2024-02-18T00:09:09.855 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-18T00:09:09.855 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957 --name yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi --description rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957/orig.config.yaml 2024-02-18T00:09:09.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T00:09:10.002 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957 branch: pacific-release description: rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} email: yweinste@redhat.com first_in_suite: false job_id: '7564957' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: pacific-release ceph: conf: client: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mgrc: 20 debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon warn on pool no app: false paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 crush_tunables: jewel flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - \(MON_DOWN\) - \(OSDMAP_FLAGS\) - flag\(s\) set - \(CACHE_POOL_NO_HIT_SET\) - \(PG_ - \(OSD_ - mons down: null - reached quota - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: pacific-release sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 owner: scheduled_yuriw@teuthology priority: 51 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 264 sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: pacific-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgEiIgn9ZRpFMBfeYM7W7ByetwlioK7CSZpkeBUEhmvUvm0L/RU4GpmH1aDZHPQdzqWONLS4ciTMHBjYvM3y8A= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK3u1MOUZPUsflX1PKVxUVIu5WPzW4XRo8+yeR5RZVHxAE5Gyaz5Jesqc7uJXNjywTxonkQbwCeRga6+De6GT5g= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - workunit: clients: client.0: - rados/test.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-17_15:55:51 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-18T00:09:10.002 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa; will attempt to use it 2024-02-18T00:09:10.003 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks 2024-02-18T00:09:10.003 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-18T00:09:10.005 INFO:teuthology.task.internal:Checking packages... 2024-02-18T00:09:10.029 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '81bd20d634209c7cb82c18be12b4b5a05643ebf1' 2024-02-18T00:09:10.029 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-18T00:09:10.029 INFO:teuthology.packaging:ref: None 2024-02-18T00:09:10.030 INFO:teuthology.packaging:tag: None 2024-02-18T00:09:10.030 INFO:teuthology.packaging:branch: pacific-release 2024-02-18T00:09:10.030 INFO:teuthology.packaging:sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:09:10.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific-release 2024-02-18T00:09:10.269 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-548.g81bd20d6 2024-02-18T00:09:10.270 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-18T00:09:10.372 INFO:teuthology.task.internal:no buildpackages task found 2024-02-18T00:09:10.372 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-18T00:09:10.379 INFO:teuthology.task.internal:Saving configuration 2024-02-18T00:09:10.393 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-18T00:09:10.402 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T00:09:10.426 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:04:05.334488', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgEiIgn9ZRpFMBfeYM7W7ByetwlioK7CSZpkeBUEhmvUvm0L/RU4GpmH1aDZHPQdzqWONLS4ciTMHBjYvM3y8A='} 2024-02-18T00:09:10.450 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:04:05.333105', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK3u1MOUZPUsflX1PKVxUVIu5WPzW4XRo8+yeR5RZVHxAE5Gyaz5Jesqc7uJXNjywTxonkQbwCeRga6+De6GT5g='} 2024-02-18T00:09:10.450 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-18T00:09:10.466 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-18T00:09:10.466 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-18T00:09:10.466 INFO:teuthology.run_tasks:Running task console_log... 2024-02-18T00:09:10.549 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f038cf9a0d0>, signals=[15]) 2024-02-18T00:09:10.549 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-18T00:09:10.571 INFO:teuthology.task.internal:Opening connections... 2024-02-18T00:09:10.571 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-02-18T00:09:10.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:09:10.656 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-02-18T00:09:10.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:09:10.731 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-18T00:09:10.772 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-02-18T00:09:10.796 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-02-18T00:09:10.796 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-02-18T00:09:10.858 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-02-18T00:09:10.858 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8" 2024-02-18T00:09:10.858 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-02-18T00:09:10.858 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-18T00:09:10.859 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T00:09:10.860 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-02-18T00:09:10.888 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-02-18T00:09:10.912 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-02-18T00:09:10.912 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-02-18T00:09:10.973 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-02-18T00:09:10.973 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8" 2024-02-18T00:09:10.973 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-02-18T00:09:10.973 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-02-18T00:09:10.973 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-18T00:09:10.974 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T00:09:10.974 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-02-18T00:09:11.004 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-18T00:09:11.038 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-18T00:09:11.045 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-18T00:09:11.045 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-02-18T00:09:11.048 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-02-18T00:09:11.069 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-18T00:09:11.075 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-18T00:09:11.075 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-02-18T00:09:11.110 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-02-18T00:09:11.142 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-18T00:09:11.203 INFO:teuthology.run_tasks:Running task kernel... 2024-02-18T00:09:11.233 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-18T00:09:11.233 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-18T00:09:11.234 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T00:09:11.234 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T00:09:11.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:09:11.255 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-02-18T00:09:11.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:09:11.257 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-02-18T00:09:11.316 INFO:teuthology.orchestra.run.smithi137.stdout:4.18.0-489.el8.x86_64 2024-02-18T00:09:11.316 INFO:teuthology.task.kernel:Running kernel on smithi137: 4.18.0-489.el8.x86_64 2024-02-18T00:09:11.316 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-02-18T00:09:11.318 INFO:teuthology.orchestra.run.smithi059.stdout:4.18.0-489.el8.x86_64 2024-02-18T00:09:11.318 INFO:teuthology.task.kernel:Running kernel on smithi059: 4.18.0-489.el8.x86_64 2024-02-18T00:09:11.318 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2024-02-18T00:09:15.106 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - AppStream 22 MB/s | 27 MB 00:01 2024-02-18T00:09:15.169 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - AppStream 20 MB/s | 27 MB 00:01 2024-02-18T00:09:21.352 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - BaseOS 7.7 MB/s | 10 MB 00:01 2024-02-18T00:09:21.426 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - BaseOS 8.5 MB/s | 10 MB 00:01 2024-02-18T00:09:23.194 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - Extras 80 kB/s | 18 kB 00:00 2024-02-18T00:09:23.368 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - Extras 72 kB/s | 18 kB 00:00 2024-02-18T00:09:23.529 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - Extras common packages 31 kB/s | 6.9 kB 00:00 2024-02-18T00:09:23.747 INFO:teuthology.orchestra.run.smithi137.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2024-02-18T00:09:23.885 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2024-02-18T00:09:24.128 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-02-18T00:09:28.116 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 3.8 MB/s | 16 MB 00:04 2024-02-18T00:09:31.897 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 279 kB/s | 24 kB 00:00 2024-02-18T00:09:34.804 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-18T00:09:34.804 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-18T00:09:34.905 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:09:34.905 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout:Install 3 Packages 2024-02-18T00:09:34.906 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:09:34.907 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 90 M 2024-02-18T00:09:34.907 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 97 M 2024-02-18T00:09:34.907 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:09:36.192 INFO:teuthology.orchestra.run.smithi137.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 9.4 MB/s | 10 MB 00:01 2024-02-18T00:09:37.359 INFO:teuthology.orchestra.run.smithi137.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 16 MB/s | 36 MB 00:02 2024-02-18T00:09:37.734 INFO:teuthology.orchestra.run.smithi137.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 16 MB/s | 43 MB 00:02 2024-02-18T00:09:37.734 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:09:37.734 INFO:teuthology.orchestra.run.smithi137.stdout:Total 32 MB/s | 90 MB 00:02 2024-02-18T00:09:38.424 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:09:38.848 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:09:38.848 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:09:40.082 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:09:40.100 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:09:42.876 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:09:43.371 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-18T00:09:43.727 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 848 kB/s | 16 MB 00:19 2024-02-18T00:09:46.908 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-18T00:09:47.353 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-18T00:09:47.677 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 297 kB/s | 24 kB 00:00 2024-02-18T00:09:50.784 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-18T00:09:50.791 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-18T00:09:50.791 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-18T00:09:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-18T00:09:50.936 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:09:50.937 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:09:50.937 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:09:50.937 INFO:teuthology.orchestra.run.smithi059.stdout:Install 3 Packages 2024-02-18T00:09:50.937 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:09:50.937 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 90 M 2024-02-18T00:09:50.938 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 97 M 2024-02-18T00:09:50.938 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:09:51.068 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:09:52.285 INFO:teuthology.orchestra.run.smithi059.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 7.9 MB/s | 10 MB 00:01 2024-02-18T00:09:53.546 INFO:teuthology.orchestra.run.smithi059.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 14 MB/s | 36 MB 00:02 2024-02-18T00:09:53.963 INFO:teuthology.orchestra.run.smithi059.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 14 MB/s | 43 MB 00:02 2024-02-18T00:09:53.963 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:09:53.963 INFO:teuthology.orchestra.run.smithi059.stdout:Total 30 MB/s | 90 MB 00:03 2024-02-18T00:09:54.681 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:09:55.113 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:09:55.113 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:09:56.414 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:09:56.445 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:09:59.365 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:09:59.845 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-18T00:10:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-18T00:10:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-18T00:10:07.642 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-18T00:10:07.974 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:13.104 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:13.128 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-18T00:10:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-18T00:10:13.950 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:13.951 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:10:13.951 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:10:13.951 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-18T00:10:13.951 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-18T00:10:13.951 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:10:13.951 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:10:14.098 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T00:10:14.315 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-18T00:10:14.316 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-18T00:10:14.316 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-18T00:10:14.316 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-18T00:10:14.316 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-18T00:10:14.317 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-18T00:10:14.317 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-02-18T00:10:14.317 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T00:10:14.528 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-18T00:10:14.529 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-18T00:10:14.529 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-02-18T00:10:14.562 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-02-18T00:10:14.562 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-02-18T00:10:14.562 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.02 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 129.el8 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Group : System Environment/Base 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 9265586 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-18T00:10:14.563 INFO:teuthology.orchestra.run.smithi137.stdout:Relocations : (not relocatable) 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : CentOS BuildSystem 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-02-18T00:10:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:10:14.565 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-02-18T00:10:14.565 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-18T00:10:14.566 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 4.18.0-539.el8.x86_64 2024-02-18T00:10:14.566 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-18T00:10:16.182 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-02-18T00:10:18.364 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-02-18T00:10:18.366 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-02-18T00:10:18.392 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.dNcnNf6CWU 2024-02-18T00:10:18.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dNcnNf6CWU 2024-02-18T00:10:18.466 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.dNcnNf6CWU 2024-02-18T00:10:18.614 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.dNcnNf6CWU is 6KB 2024-02-18T00:10:18.662 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.dNcnNf6CWU 2024-02-18T00:10:18.679 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-02-18T00:10:18.745 INFO:teuthology.orchestra.run.smithi137.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-18T00:10:18.745 INFO:teuthology.orchestra.run.smithi137.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-18T00:10:18.745 INFO:teuthology.orchestra.run.smithi137.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-18T00:10:18.746 INFO:teuthology.orchestra.run.smithi137.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-18T00:10:18.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-18T00:10:18.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-02-18T00:10:31.507 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:31.531 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:31.892 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-18T00:10:31.895 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-18T00:10:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-18T00:10:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:10:32.388 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:10:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-18T00:10:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-18T00:10:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:10:32.388 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:10:32.565 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T00:10:32.796 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-18T00:10:32.797 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-18T00:10:32.797 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-18T00:10:32.797 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-18T00:10:32.797 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-18T00:10:32.797 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-18T00:10:32.797 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=None, version=distro) 2024-02-18T00:10:32.797 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T00:10:33.018 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-18T00:10:33.018 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-18T00:10:33.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2-tools 2024-02-18T00:10:33.052 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2-tools 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.02 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 129.el8 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Group : System Environment/Base 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 9265586 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-18T00:10:33.053 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:Relocations : (not relocatable) 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : CentOS BuildSystem 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Support tools for GRUB. 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:10:33.054 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-18T00:10:33.055 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-18T00:10:33.055 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-18T00:10:33.055 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2024-02-18T00:10:33.055 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:10:33.055 INFO:teuthology.orchestra.run.smithi059.stdout:This subpackage provides tools for support of all platforms. 2024-02-18T00:10:33.056 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-18T00:10:33.056 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 4.18.0-539.el8.x86_64 2024-02-18T00:10:33.056 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-18T00:10:34.372 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2024-02-18T00:10:36.857 INFO:teuthology.orchestra.run.smithi059.stderr:done 2024-02-18T00:10:36.858 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-02-18T00:10:36.895 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.F8MhPLUECf 2024-02-18T00:10:36.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.F8MhPLUECf 2024-02-18T00:10:36.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.F8MhPLUECf 2024-02-18T00:10:37.120 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.F8MhPLUECf is 6KB 2024-02-18T00:10:37.168 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.F8MhPLUECf 2024-02-18T00:10:37.186 DEBUG:teuthology.orchestra.run.smithi059:> sudo /bin/ls /boot/loader/entries || true 2024-02-18T00:10:37.260 INFO:teuthology.orchestra.run.smithi059.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-18T00:10:37.260 INFO:teuthology.orchestra.run.smithi059.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-18T00:10:37.260 INFO:teuthology.orchestra.run.smithi059.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-18T00:10:37.261 INFO:teuthology.orchestra.run.smithi059.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-18T00:10:37.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-18T00:10:37.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2024-02-18T00:10:48.904 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-18T00:10:48.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-02-18T00:10:48.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:11:07.444 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-18T00:11:07.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-02-18T00:11:07.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:11:07.452 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-02-18T00:11:16.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-02-18T00:11:16.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:11:19.516 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-02-18T00:11:25.884 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-02-18T00:11:31.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-02-18T00:11:31.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:11:34.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-02-18T00:11:34.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:11:49.948 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-02-18T00:12:04.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-02-18T00:12:04.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:12:05.662 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-02-18T00:12:06.851 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-02-18T00:12:06.852 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-539.el8.x86_64"... 2024-02-18T00:12:06.852 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-02-18T00:12:06.952 INFO:teuthology.orchestra.run.smithi137.stdout:4.18.0-539.el8.x86_64 2024-02-18T00:12:06.953 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-18T00:12:06.953 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-18T00:12:06.953 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-18T00:12:07.954 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-18T00:12:07.955 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-18T00:12:08.031 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-02-18T00:12:08.101 DEBUG:teuthology.parallel:result is None 2024-02-18T00:12:34.891 DEBUG:teuthology.orchestra.remote:timed out 2024-02-18T00:12:46.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-02-18T00:12:46.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:12:47.230 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-02-18T00:12:47.533 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-02-18T00:12:47.533 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-539.el8.x86_64"... 2024-02-18T00:12:47.533 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-02-18T00:12:47.553 INFO:teuthology.orchestra.run.smithi059.stdout:4.18.0-539.el8.x86_64 2024-02-18T00:12:47.554 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-18T00:12:47.554 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-18T00:12:47.554 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-18T00:12:48.555 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-18T00:12:48.555 DEBUG:teuthology.orchestra.run.smithi059:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-18T00:12:48.628 INFO:teuthology.orchestra.run.smithi059.stdout:ttyS1 2024-02-18T00:12:48.641 DEBUG:teuthology.parallel:result is None 2024-02-18T00:12:48.641 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-18T00:12:48.658 INFO:teuthology.task.internal:Creating test directory... 2024-02-18T00:12:48.658 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T00:12:48.661 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T00:12:48.684 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-18T00:12:48.691 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-18T00:12:48.696 INFO:teuthology.task.internal:Creating archive directory... 2024-02-18T00:12:48.696 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T00:12:48.723 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T00:12:48.773 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-18T00:12:48.779 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-18T00:12:48.779 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T00:12:48.789 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T00:12:48.824 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T00:12:48.835 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T00:12:48.845 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T00:12:48.856 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T00:12:48.857 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-18T00:12:48.864 INFO:teuthology.task.internal:Configuring sudo... 2024-02-18T00:12:48.864 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T00:12:48.879 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T00:12:48.928 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-18T00:12:48.937 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-18T00:12:48.937 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T00:12:48.950 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T00:12:48.987 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T00:12:49.055 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T00:12:49.104 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T00:12:49.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T00:12:49.235 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:12:49.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T00:12:49.304 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T00:12:49.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T00:12:49.408 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T00:12:49.497 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T00:12:49.542 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:12:49.542 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T00:12:49.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-02-18T00:12:49.613 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-02-18T00:12:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T00:12:49.720 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T00:12:50.017 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-18T00:12:50.026 INFO:teuthology.task.internal:Starting timer... 2024-02-18T00:12:50.026 INFO:teuthology.run_tasks:Running task pcp... 2024-02-18T00:12:50.036 INFO:teuthology.run_tasks:Running task selinux... 2024-02-18T00:12:50.046 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-18T00:12:50.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-02-18T00:12:50.168 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-02-18T00:12:50.170 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-02-18T00:12:50.251 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-02-18T00:12:50.253 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-18T00:12:50.253 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-02-18T00:12:50.283 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-02-18T00:12:50.283 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-02-18T00:12:50.313 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-02-18T00:12:50.313 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2024-02-18T00:12:50.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T00:12:50.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:12:50.352 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T00:12:50.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:12:50.381 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-18T00:12:50.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-02-18T00:12:50.476 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-02-18T00:12:50.503 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-18T00:12:50.511 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-18T00:12:50.530 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-18T00:12:50.531 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-18T00:18:50.097 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2024-02-18T00:18:50.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-02-18T00:18:50.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:18:50.174 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-02-18T00:18:50.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-02-18T00:18:50.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-02-18T00:18:50.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:18:50.317 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-02-18T00:18:50.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-02-18T00:18:50.387 INFO:teuthology.run_tasks:Running task clock... 2024-02-18T00:18:50.398 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-18T00:18:50.398 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T00:18:50.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T00:18:50.401 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T00:18:50.401 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T00:18:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-18T00:18:50.448 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-18T00:18:50.457 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-18T00:18:50.471 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-18T00:18:50.515 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-02-18T00:18:50.527 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-02-18T00:18:50.541 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-18T00:18:50.541 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-02-18T00:18:50.552 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-02-18T00:18:50.555 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-18T00:18:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-18T00:18:50.578 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-18T00:18:50.626 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2024-02-18T00:18:50.630 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T00:18:50.630 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-02-18T00:18:50.630 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.630 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.630 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.630 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.636 INFO:teuthology.orchestra.run.smithi137.stderr:bash: ntpq: command not found 2024-02-18T00:18:50.639 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T00:18:50.640 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-02-18T00:18:50.640 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.640 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.640 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.640 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:18:50.641 INFO:teuthology.run_tasks:Running task pexec... 2024-02-18T00:18:50.650 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-18T00:18:50.651 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi059.front.sepia.ceph.com 2024-02-18T00:18:50.651 DEBUG:teuthology.orchestra.run.smithi059:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-18T00:18:50.651 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi137.front.sepia.ceph.com 2024-02-18T00:18:50.651 DEBUG:teuthology.orchestra.run.smithi137:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-18T00:18:51.471 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:32 ago on Sun 18 Feb 2024 12:17:19 AM UTC. 2024-02-18T00:18:51.489 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:23 ago on Sun 18 Feb 2024 12:17:28 AM UTC. 2024-02-18T00:18:53.475 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout:Resetting modules: 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:18:53.476 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:18:53.674 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-02-18T00:18:53.675 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi059.stdout:Resetting modules: 2024-02-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi059.stdout: container-tools 2024-02-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:18:53.676 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:18:53.758 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:18:53.928 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:18:54.648 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:35 ago on Sun 18 Feb 2024 12:17:19 AM UTC. 2024-02-18T00:18:54.824 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:26 ago on Sun 18 Feb 2024 12:17:28 AM UTC. 2024-02-18T00:18:56.679 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout:Installing group/module packages: 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-18T00:18:56.715 INFO:teuthology.orchestra.run.smithi137.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout:Installing module profiles: 2024-02-18T00:18:56.716 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools/common 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling module streams: 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools rhel8 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout:Install 13 Packages 2024-02-18T00:18:56.717 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:18:56.718 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 28 M 2024-02-18T00:18:56.718 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 92 M 2024-02-18T00:18:56.718 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:18:57.028 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:18:57.067 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:18:57.067 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:18:57.067 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:18:57.067 INFO:teuthology.orchestra.run.smithi059.stdout:Installing group/module packages: 2024-02-18T00:18:57.067 INFO:teuthology.orchestra.run.smithi059.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-18T00:18:57.067 INFO:teuthology.orchestra.run.smithi059.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-02-18T00:18:57.068 INFO:teuthology.orchestra.run.smithi059.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout:Installing module profiles: 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout: container-tools/common 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling module streams: 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout: container-tools rhel8 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout:Install 13 Packages 2024-02-18T00:18:57.069 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:18:57.070 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 28 M 2024-02-18T00:18:57.070 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 92 M 2024-02-18T00:18:57.070 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:18:57.233 INFO:teuthology.orchestra.run.smithi137.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 2.1 MB/s | 688 kB 00:00 2024-02-18T00:18:57.266 INFO:teuthology.orchestra.run.smithi137.stdout:(2/13): aardvark-dns-1.10.0-1.module_el8+871+55 2.8 MB/s | 1.0 MB 00:00 2024-02-18T00:18:57.320 INFO:teuthology.orchestra.run.smithi137.stdout:(3/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 437 kB/s | 38 kB 00:00 2024-02-18T00:18:57.350 INFO:teuthology.orchestra.run.smithi137.stdout:(4/13): crun-1.14-1.module_el8+871+552be958.x86 3.0 MB/s | 257 kB 00:00 2024-02-18T00:18:57.417 INFO:teuthology.orchestra.run.smithi137.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 2.6 MB/s | 180 kB 00:00 2024-02-18T00:18:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 371 kB/s | 25 kB 00:00 2024-02-18T00:18:57.543 INFO:teuthology.orchestra.run.smithi137.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.6 MB/s | 94 kB 00:00 2024-02-18T00:18:57.568 INFO:teuthology.orchestra.run.smithi059.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 2.2 MB/s | 688 kB 00:00 2024-02-18T00:18:57.658 INFO:teuthology.orchestra.run.smithi059.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 422 kB/s | 38 kB 00:00 2024-02-18T00:18:57.677 INFO:teuthology.orchestra.run.smithi137.stdout:(8/13): netavark-1.10.2-1.module_el8+885+7da147 12 MB/s | 4.3 MB 00:00 2024-02-18T00:18:57.693 INFO:teuthology.orchestra.run.smithi059.stdout:(3/13): aardvark-dns-1.10.0-1.module_el8+871+55 2.3 MB/s | 1.0 MB 00:00 2024-02-18T00:18:57.727 INFO:teuthology.orchestra.run.smithi059.stdout:(4/13): crun-1.14-1.module_el8+871+552be958.x86 3.6 MB/s | 257 kB 00:00 2024-02-18T00:18:57.794 INFO:teuthology.orchestra.run.smithi059.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 2.6 MB/s | 180 kB 00:00 2024-02-18T00:18:57.810 INFO:teuthology.orchestra.run.smithi137.stdout:(9/13): buildah-1.33.5-1.module_el8+885+7da147f 11 MB/s | 9.7 MB 00:00 2024-02-18T00:18:57.861 INFO:teuthology.orchestra.run.smithi059.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 373 kB/s | 25 kB 00:00 2024-02-18T00:18:57.869 INFO:teuthology.orchestra.run.smithi137.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 904 kB/s | 53 kB 00:00 2024-02-18T00:18:57.919 INFO:teuthology.orchestra.run.smithi137.stdout:(11/13): toolbox-0.0.99.5-1.module_el8+847+7863 10 MB/s | 2.5 MB 00:00 2024-02-18T00:18:57.928 INFO:teuthology.orchestra.run.smithi059.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 1.4 MB/s | 94 kB 00:00 2024-02-18T00:18:57.944 INFO:teuthology.orchestra.run.smithi137.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 546 kB/s | 41 kB 00:00 2024-02-18T00:18:58.053 INFO:teuthology.orchestra.run.smithi059.stdout:(8/13): netavark-1.10.2-1.module_el8+885+7da147 12 MB/s | 4.3 MB 00:00 2024-02-18T00:18:58.195 INFO:teuthology.orchestra.run.smithi059.stdout:(9/13): buildah-1.33.5-1.module_el8+885+7da147f 10 MB/s | 9.7 MB 00:00 2024-02-18T00:18:58.223 INFO:teuthology.orchestra.run.smithi137.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 13 MB/s | 8.8 MB 00:00 2024-02-18T00:18:58.223 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:18:58.223 INFO:teuthology.orchestra.run.smithi137.stdout:Total 18 MB/s | 28 MB 00:01 2024-02-18T00:18:58.262 INFO:teuthology.orchestra.run.smithi059.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 790 kB/s | 53 kB 00:00 2024-02-18T00:18:58.338 INFO:teuthology.orchestra.run.smithi059.stdout:(11/13): toolbox-0.0.99.5-1.module_el8+847+7863 8.9 MB/s | 2.5 MB 00:00 2024-02-18T00:18:58.362 INFO:teuthology.orchestra.run.smithi059.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 409 kB/s | 41 kB 00:00 2024-02-18T00:18:58.494 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:18:58.515 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:18:58.515 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi059.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 13 MB/s | 8.8 MB 00:00 2024-02-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:18:58.605 INFO:teuthology.orchestra.run.smithi059.stdout:Total 18 MB/s | 28 MB 00:01 2024-02-18T00:18:58.769 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:18:58.773 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:18:58.882 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:18:58.904 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:18:58.904 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:18:59.181 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:18:59.185 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:18:59.474 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:19:00.046 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:19:00.425 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-18T00:19:00.621 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-18T00:19:00.775 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-18T00:19:00.987 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-18T00:19:01.044 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-18T00:19:01.202 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-18T00:19:01.234 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-18T00:19:01.434 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-18T00:19:01.617 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-18T00:19:01.755 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-18T00:19:01.866 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-18T00:19:02.473 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-18T00:19:02.730 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-18T00:19:02.944 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-18T00:19:03.148 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-18T00:19:03.607 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-18T00:19:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-18T00:19:03.794 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-18T00:19:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-18T00:19:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-18T00:19:04.149 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-18T00:19:04.542 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-18T00:19:04.550 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-18T00:19:04.832 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-18T00:19:04.979 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-18T00:19:04.979 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-18T00:19:04.980 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-18T00:19:05.053 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-18T00:19:05.256 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-18T00:19:05.627 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-18T00:19:05.627 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-18T00:19:05.628 INFO:teuthology.orchestra.run.smithi137.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-18T00:19:05.629 INFO:teuthology.orchestra.run.smithi137.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-18T00:19:05.629 INFO:teuthology.orchestra.run.smithi137.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-18T00:19:05.629 INFO:teuthology.orchestra.run.smithi137.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-18T00:19:05.629 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:19:05.629 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:19:05.696 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-18T00:19:05.696 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-18T00:19:05.696 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-18T00:19:05.696 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-18T00:19:05.696 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-18T00:19:05.696 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-18T00:19:05.697 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-18T00:19:05.697 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-18T00:19:05.697 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-18T00:19:05.697 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-18T00:19:05.697 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-18T00:19:05.697 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-18T00:19:05.697 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-18T00:19:05.917 DEBUG:teuthology.parallel:result is None 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-18T00:19:06.258 INFO:teuthology.orchestra.run.smithi059.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:19:06.259 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:19:06.534 DEBUG:teuthology.parallel:result is None 2024-02-18T00:19:06.534 INFO:teuthology.run_tasks:Running task install... 2024-02-18T00:19:06.545 DEBUG:teuthology.task.install:project ceph 2024-02-18T00:19:06.546 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1'}} 2024-02-18T00:19:06.546 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1'} 2024-02-18T00:19:06.546 INFO:teuthology.task.install:Using flavor: default 2024-02-18T00:19:06.551 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-18T00:19:06.552 INFO:teuthology.task.install:extra packages: [] 2024-02-18T00:19:06.552 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1', 'tag': None, 'wait_for_package': False} 2024-02-18T00:19:06.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:19:06.554 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1', 'tag': None, 'wait_for_package': False} 2024-02-18T00:19:06.555 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:19:06.699 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/ 2024-02-18T00:19:06.699 INFO:teuthology.task.install.rpm:Package version is 16.2.14-548.g81bd20d6 2024-02-18T00:19:06.710 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/ 2024-02-18T00:19:06.710 INFO:teuthology.task.install.rpm:Package version is 16.2.14-548.g81bd20d6 2024-02-18T00:19:06.821 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-18T00:19:06.821 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:19:06.822 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-18T00:19:06.825 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-18T00:19:06.825 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:19:06.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-18T00:19:06.853 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-18T00:19:06.853 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81bd20d634209c7cb82c18be12b4b5a05643ebf1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-18T00:19:06.854 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-18T00:19:06.854 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81bd20d634209c7cb82c18be12b4b5a05643ebf1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-18T00:19:06.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-18T00:19:06.925 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-18T00:19:07.002 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-18T00:19:07.003 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-18T00:19:07.070 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-02-18T00:19:07.070 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-02-18T00:19:07.071 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-02-18T00:19:07.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-02-18T00:19:07.329 INFO:teuthology.orchestra.run.smithi137.stdout:76 files removed 2024-02-18T00:19:07.346 INFO:teuthology.orchestra.run.smithi059.stdout:76 files removed 2024-02-18T00:19:07.357 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw 2024-02-18T00:19:07.378 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw 2024-02-18T00:19:08.087 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 235 kB/s | 76 kB 00:00 2024-02-18T00:19:08.129 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 237 kB/s | 76 kB 00:00 2024-02-18T00:19:08.396 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 56 kB/s | 15 kB 00:00 2024-02-18T00:19:08.463 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 52 kB/s | 15 kB 00:00 2024-02-18T00:19:08.755 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 4.4 kB/s | 1.4 kB 00:00 2024-02-18T00:19:08.848 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 4.1 kB/s | 1.4 kB 00:00 2024-02-18T00:19:10.556 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - AppStream 15 MB/s | 27 MB 00:01 2024-02-18T00:19:10.932 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - AppStream 13 MB/s | 27 MB 00:02 2024-02-18T00:19:16.481 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - BaseOS 9.3 MB/s | 10 MB 00:01 2024-02-18T00:19:17.307 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - BaseOS 8.1 MB/s | 10 MB 00:01 2024-02-18T00:19:18.273 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - Extras 82 kB/s | 18 kB 00:00 2024-02-18T00:19:18.632 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - Extras common packages 28 kB/s | 6.9 kB 00:00 2024-02-18T00:19:19.199 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - Extras 88 kB/s | 18 kB 00:00 2024-02-18T00:19:19.525 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - Extras common packages 33 kB/s | 6.9 kB 00:00 2024-02-18T00:19:20.725 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 8 - PowerTools 2.4 MB/s | 4.8 MB 00:02 2024-02-18T00:19:21.785 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 8 - PowerTools 2.2 MB/s | 4.8 MB 00:02 2024-02-18T00:19:21.810 INFO:teuthology.orchestra.run.smithi137.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-02-18T00:19:22.895 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-02-18T00:19:24.798 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 16 MB 00:01 2024-02-18T00:19:28.812 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 413 kB/s | 24 kB 00:00 2024-02-18T00:19:32.669 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:19:32.672 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:19:32.672 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:19:32.672 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:19:32.672 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:19:32.672 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 12 M 2024-02-18T00:19:32.672 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 4.2 M 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.1 M 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 24 M 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 26 k 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 825 k 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 494 k 2024-02-18T00:19:32.673 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.8 M 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 45 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 99 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 215 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 389 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 369 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 114 k 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:19:32.674 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:19:32.675 INFO:teuthology.orchestra.run.smithi059.stdout:Install 21 Packages 2024-02-18T00:19:32.675 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 2 Packages 2024-02-18T00:19:32.675 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:19:32.675 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 58 M 2024-02-18T00:19:32.675 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:19:33.930 INFO:teuthology.orchestra.run.smithi059.stdout:(1/23): ceph-base-16.2.14-548.g81bd20d6.el8.x86 8.7 MB/s | 6.1 MB 00:00 2024-02-18T00:19:34.005 INFO:teuthology.orchestra.run.smithi059.stdout:(2/23): ceph-selinux-16.2.14-548.g81bd20d6.el8. 343 kB/s | 26 kB 00:00 2024-02-18T00:19:34.156 INFO:teuthology.orchestra.run.smithi059.stdout:(3/23): ceph-radosgw-16.2.14-548.g81bd20d6.el8. 13 MB/s | 12 MB 00:00 2024-02-18T00:19:34.189 INFO:teuthology.orchestra.run.smithi059.stdout:(4/23): libcephfs2-16.2.14-548.g81bd20d6.el8.x8 4.4 MB/s | 825 kB 00:00 2024-02-18T00:19:34.214 INFO:teuthology.orchestra.run.smithi059.stdout:(5/23): libradosstriper1-16.2.14-548.g81bd20d6. 8.3 MB/s | 494 kB 00:00 2024-02-18T00:19:34.273 INFO:teuthology.orchestra.run.smithi059.stdout:(6/23): python3-ceph-argparse-16.2.14-548.g81bd 774 kB/s | 45 kB 00:00 2024-02-18T00:19:34.357 INFO:teuthology.orchestra.run.smithi059.stdout:(7/23): python3-ceph-common-16.2.14-548.g81bd20 1.2 MB/s | 99 kB 00:00 2024-02-18T00:19:34.415 INFO:teuthology.orchestra.run.smithi059.stdout:(8/23): librgw2-16.2.14-548.g81bd20d6.el8.x86_6 17 MB/s | 3.8 MB 00:00 2024-02-18T00:19:34.440 INFO:teuthology.orchestra.run.smithi059.stdout:(9/23): python3-cephfs-16.2.14-548.g81bd20d6.el 2.5 MB/s | 215 kB 00:00 2024-02-18T00:19:34.466 INFO:teuthology.orchestra.run.smithi059.stdout:(10/23): python3-rados-16.2.14-548.g81bd20d6.el 7.6 MB/s | 389 kB 00:00 2024-02-18T00:19:34.502 INFO:teuthology.orchestra.run.smithi059.stdout:(11/23): python3-rbd-16.2.14-548.g81bd20d6.el8. 5.9 MB/s | 369 kB 00:00 2024-02-18T00:19:34.519 INFO:teuthology.orchestra.run.smithi059.stdout:(12/23): python3-rgw-16.2.14-548.g81bd20d6.el8. 2.1 MB/s | 114 kB 00:00 2024-02-18T00:19:34.786 INFO:teuthology.orchestra.run.smithi059.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 973 kB/s | 259 kB 00:00 2024-02-18T00:19:34.811 INFO:teuthology.orchestra.run.smithi059.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.1 MB/s | 677 kB 00:00 2024-02-18T00:19:34.853 INFO:teuthology.orchestra.run.smithi059.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 657 kB/s | 44 kB 00:00 2024-02-18T00:19:35.153 INFO:teuthology.orchestra.run.smithi059.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 138 kB/s | 47 kB 00:00 2024-02-18T00:19:35.292 INFO:teuthology.orchestra.run.smithi059.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 698 kB/s | 306 kB 00:00 2024-02-18T00:19:35.379 INFO:teuthology.orchestra.run.smithi059.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 685 kB/s | 59 kB 00:00 2024-02-18T00:19:35.479 INFO:teuthology.orchestra.run.smithi059.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 750 kB/s | 75 kB 00:00 2024-02-18T00:19:35.563 INFO:teuthology.orchestra.run.smithi059.stdout:(20/23): leveldb-1.22-1.el8.x86_64.rpm 442 kB/s | 181 kB 00:00 2024-02-18T00:19:36.305 INFO:teuthology.orchestra.run.smithi059.stdout:(21/23): librbd1-16.2.14-548.g81bd20d6.el8.x86_ 5.6 MB/s | 4.2 MB 00:00 2024-02-18T00:19:37.680 INFO:teuthology.orchestra.run.smithi059.stdout:(22/23): ceph-common-16.2.14-548.g81bd20d6.el8. 5.5 MB/s | 24 MB 00:04 2024-02-18T00:19:38.205 INFO:teuthology.orchestra.run.smithi059.stdout:(23/23): librados2-16.2.14-548.g81bd20d6.el8.x8 1.4 MB/s | 3.7 MB 00:02 2024-02-18T00:19:38.209 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:19:38.210 INFO:teuthology.orchestra.run.smithi059.stdout:Total 10 MB/s | 58 MB 00:05 2024-02-18T00:19:38.254 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:19:38.306 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:19:38.306 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:19:38.709 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:19:38.713 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:19:39.777 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:19:39.979 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-18T00:19:40.152 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-18T00:19:40.552 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-18T00:19:40.785 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-18T00:19:40.882 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-18T00:19:41.137 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-18T00:19:41.343 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-18T00:19:41.783 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-18T00:19:42.001 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-18T00:19:42.053 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-18T00:19:42.381 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-18T00:19:42.584 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 800 kB/s | 16 MB 00:20 2024-02-18T00:19:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-18T00:19:43.043 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-18T00:19:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-18T00:19:43.316 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-18T00:19:43.484 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 11/25 2024-02-18T00:19:43.705 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-18T00:19:43.913 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-18T00:19:44.153 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 14/25 2024-02-18T00:19:44.343 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-18T00:19:44.405 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-18T00:19:44.646 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-18T00:19:44.803 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-18T00:19:44.973 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-18T00:19:45.160 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 19/25 2024-02-18T00:19:46.359 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 399 kB/s | 24 kB 00:00 2024-02-18T00:19:47.801 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-18T00:19:48.052 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-18T00:19:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-18T00:19:49.125 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-18T00:19:49.247 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-18T00:19:49.426 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-18T00:19:49.863 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 12 M 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 4.2 M 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:19:49.866 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.1 M 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 24 M 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 26 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 825 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 494 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.8 M 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 45 k 2024-02-18T00:19:49.867 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 99 k 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 215 k 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 389 k 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 369 k 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 114 k 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout:Install 21 Packages 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 2 Packages 2024-02-18T00:19:49.868 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:19:49.869 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 58 M 2024-02-18T00:19:49.869 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:19:51.068 INFO:teuthology.orchestra.run.smithi137.stdout:(1/23): ceph-radosgw-16.2.14-548.g81bd20d6.el8. 12 MB/s | 12 MB 00:00 2024-02-18T00:19:51.177 INFO:teuthology.orchestra.run.smithi137.stdout:(2/23): ceph-base-16.2.14-548.g81bd20d6.el8.x86 5.7 MB/s | 6.1 MB 00:01 2024-02-18T00:19:51.202 INFO:teuthology.orchestra.run.smithi137.stdout:(3/23): ceph-selinux-16.2.14-548.g81bd20d6.el8. 192 kB/s | 26 kB 00:00 2024-02-18T00:19:51.269 INFO:teuthology.orchestra.run.smithi137.stdout:(4/23): libradosstriper1-16.2.14-548.g81bd20d6. 7.2 MB/s | 494 kB 00:00 2024-02-18T00:19:51.303 INFO:teuthology.orchestra.run.smithi137.stdout:(5/23): libcephfs2-16.2.14-548.g81bd20d6.el8.x8 6.4 MB/s | 825 kB 00:00 2024-02-18T00:19:51.395 INFO:teuthology.orchestra.run.smithi137.stdout:(6/23): python3-ceph-argparse-16.2.14-548.g81bd 492 kB/s | 45 kB 00:00 2024-02-18T00:19:51.520 INFO:teuthology.orchestra.run.smithi137.stdout:(7/23): librgw2-16.2.14-548.g81bd20d6.el8.x86_6 15 MB/s | 3.8 MB 00:00 2024-02-18T00:19:51.545 INFO:teuthology.orchestra.run.smithi137.stdout:(8/23): python3-ceph-common-16.2.14-548.g81bd20 656 kB/s | 99 kB 00:00 2024-02-18T00:19:51.571 INFO:teuthology.orchestra.run.smithi137.stdout:(9/23): python3-cephfs-16.2.14-548.g81bd20d6.el 4.2 MB/s | 215 kB 00:00 2024-02-18T00:19:51.630 INFO:teuthology.orchestra.run.smithi137.stdout:(10/23): python3-rbd-16.2.14-548.g81bd20d6.el8. 6.2 MB/s | 369 kB 00:00 2024-02-18T00:19:51.655 INFO:teuthology.orchestra.run.smithi137.stdout:(11/23): python3-rados-16.2.14-548.g81bd20d6.el 3.5 MB/s | 389 kB 00:00 2024-02-18T00:19:51.680 INFO:teuthology.orchestra.run.smithi137.stdout:(12/23): python3-rgw-16.2.14-548.g81bd20d6.el8. 2.2 MB/s | 114 kB 00:00 2024-02-18T00:19:51.930 INFO:teuthology.orchestra.run.smithi137.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2024-02-18T00:19:51.964 INFO:teuthology.orchestra.run.smithi137.stdout:(14/23): librdkafka-1.6.1-1.el8.x86_64.rpm 2.1 MB/s | 677 kB 00:00 2024-02-18T00:19:51.989 INFO:teuthology.orchestra.run.smithi137.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 750 kB/s | 44 kB 00:00 2024-02-18T00:19:52.231 INFO:teuthology.orchestra.run.smithi137.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2024-02-18T00:19:52.282 INFO:teuthology.orchestra.run.smithi137.stdout:(17/23): leveldb-1.22-1.el8.x86_64.rpm 3.5 MB/s | 181 kB 00:00 2024-02-18T00:19:52.307 INFO:teuthology.orchestra.run.smithi137.stdout:(18/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 137 kB/s | 47 kB 00:00 2024-02-18T00:19:52.332 INFO:teuthology.orchestra.run.smithi137.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2024-02-18T00:19:52.591 INFO:teuthology.orchestra.run.smithi137.stdout:(20/23): ceph-common-16.2.14-548.g81bd20d6.el8. 9.7 MB/s | 24 MB 00:02 2024-02-18T00:19:52.616 INFO:teuthology.orchestra.run.smithi137.stdout:(21/23): libunwind-1.3.1-3.el8.x86_64.rpm 243 kB/s | 75 kB 00:00 2024-02-18T00:19:52.858 INFO:teuthology.orchestra.run.smithi137.stdout:(22/23): librados2-16.2.14-548.g81bd20d6.el8.x8 7.2 MB/s | 3.7 MB 00:00 2024-02-18T00:19:52.945 INFO:teuthology.orchestra.run.smithi137.stdout:(23/23): librbd1-16.2.14-548.g81bd20d6.el8.x86_ 12 MB/s | 4.2 MB 00:00 2024-02-18T00:19:52.949 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:19:52.949 INFO:teuthology.orchestra.run.smithi137.stdout:Total 19 MB/s | 58 MB 00:03 2024-02-18T00:19:52.990 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:19:53.037 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:19:53.037 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:19:53.384 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:19:53.387 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:19:54.384 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:19:54.576 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2024-02-18T00:19:54.767 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-18T00:19:55.139 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2024-02-18T00:19:55.334 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-18T00:19:55.424 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-18T00:19:55.660 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-18T00:19:55.817 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-18T00:19:56.231 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-18T00:19:56.416 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-18T00:19:56.480 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 6/25 2024-02-18T00:19:56.701 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/25 2024-02-18T00:19:57.179 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/25 2024-02-18T00:19:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-18T00:19:57.444 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 9/25 2024-02-18T00:19:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/25 2024-02-18T00:19:57.763 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 11/25 2024-02-18T00:19:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-18T00:19:58.084 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-18T00:19:58.282 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 14/25 2024-02-18T00:19:58.474 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-18T00:19:58.541 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 15/25 2024-02-18T00:19:58.797 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2024-02-18T00:19:58.957 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2024-02-18T00:19:59.136 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-02-18T00:19:59.341 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 19/25 2024-02-18T00:20:01.823 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-18T00:20:02.058 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-18T00:20:02.758 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 20/25 2024-02-18T00:20:03.031 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-18T00:20:03.121 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 21/25 2024-02-18T00:20:03.257 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-18T00:20:09.450 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-18T00:20:09.450 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-02-18T00:20:09.450 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-02-18T00:20:09.450 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2024-02-18T00:20:09.451 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /run 2024-02-18T00:20:09.451 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-02-18T00:20:09.451 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-02-18T00:20:09.451 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-02-18T00:20:09.451 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-02-18T00:20:09.451 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2024-02-18T00:20:09.451 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:10.714 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-18T00:20:10.777 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-18T00:20:10.777 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-18T00:20:11.009 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 1/25 2024-02-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2/25 2024-02-18T00:20:11.650 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 6/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 7/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 8/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 9/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 10/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 11/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-18T00:20:11.651 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-18T00:20:11.652 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 24/25 2024-02-18T00:20:12.500 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: leveldb-1.22-1.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.501 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:12.502 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:12.503 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:20:12.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-test 2024-02-18T00:20:13.629 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:45 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:20:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:20:15.307 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:15.307 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:20:15.307 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:15.307 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:20:15.307 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 52 M 2024-02-18T00:20:15.307 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:20:15.307 INFO:teuthology.orchestra.run.smithi059.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 174 k 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout:Install 6 Packages 2024-02-18T00:20:15.308 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:15.309 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 53 M 2024-02-18T00:20:15.309 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 197 M 2024-02-18T00:20:15.309 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:20:15.751 INFO:teuthology.orchestra.run.smithi059.stdout:(1/6): libcephsqlite-16.2.14-548.g81bd20d6.el8. 944 kB/s | 174 kB 00:00 2024-02-18T00:20:15.818 INFO:teuthology.orchestra.run.smithi059.stdout:(2/6): jq-1.6-8.el8.x86_64.rpm 810 kB/s | 203 kB 00:00 2024-02-18T00:20:15.918 INFO:teuthology.orchestra.run.smithi059.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2024-02-18T00:20:15.985 INFO:teuthology.orchestra.run.smithi059.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-02-18T00:20:16.010 INFO:teuthology.orchestra.run.smithi059.stdout:(5/6): oniguruma-6.8.2-3.el8.x86_64.rpm 726 kB/s | 188 kB 00:00 2024-02-18T00:20:19.435 INFO:teuthology.orchestra.run.smithi059.stdout:(6/6): ceph-test-16.2.14-548.g81bd20d6.el8.x86_ 13 MB/s | 52 MB 00:03 2024-02-18T00:20:19.438 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:20:19.438 INFO:teuthology.orchestra.run.smithi059.stdout:Total 13 MB/s | 53 MB 00:04 2024-02-18T00:20:19.482 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:20:19.505 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:20:19.808 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:20:19.811 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:20:20.343 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:20:20.544 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-18T00:20:20.743 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-18T00:20:20.872 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-18T00:20:20.966 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-18T00:20:21.171 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-18T00:20:21.380 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-18T00:20:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-18T00:20:22.206 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-02-18T00:20:22.206 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-02-18T00:20:22.206 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2024-02-18T00:20:22.206 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /run 2024-02-18T00:20:22.207 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-02-18T00:20:22.207 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-02-18T00:20:22.207 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-02-18T00:20:22.207 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-02-18T00:20:22.207 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2024-02-18T00:20:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:23.361 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-18T00:20:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 23/25 2024-02-18T00:20:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-18T00:20:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-02-18T00:20:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-18T00:20:24.358 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-02-18T00:20:24.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 1/25 2024-02-18T00:20:24.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2/25 2024-02-18T00:20:24.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 3/25 2024-02-18T00:20:24.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 4/25 2024-02-18T00:20:24.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 5/25 2024-02-18T00:20:24.358 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_6 6/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 7/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8. 8/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x8 9/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 10/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 11/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 12/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 13/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-02-18T00:20:24.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : leveldb-1.22-1.el8.x86_64 19/25 2024-02-18T00:20:24.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 20/25 2024-02-18T00:20:24.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 21/25 2024-02-18T00:20:24.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 22/25 2024-02-18T00:20:24.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-02-18T00:20:24.360 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 24/25 2024-02-18T00:20:25.341 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-02-18T00:20:25.341 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:25.341 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-02-18T00:20:25.341 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.341 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.341 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: leveldb-1.22-1.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.342 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:25.343 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:20:25.676 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-test 2024-02-18T00:20:26.296 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-18T00:20:26.441 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:40 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:20:26.553 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-18T00:20:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-18T00:20:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 1/6 2024-02-18T00:20:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2/6 2024-02-18T00:20:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-18T00:20:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-18T00:20:26.849 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-18T00:20:27.316 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-18T00:20:27.316 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:27.316 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:20:27.316 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:27.316 INFO:teuthology.orchestra.run.smithi059.stdout: jq-1.6-8.el8.x86_64 2024-02-18T00:20:27.316 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:27.317 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-18T00:20:27.317 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-18T00:20:27.317 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-18T00:20:27.317 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:27.317 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:20:27.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph 2024-02-18T00:20:28.007 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:20:28.009 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:20:28.009 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:20:28.009 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:20:28.009 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 52 M 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 174 k 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout:Install 6 Packages 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:28.010 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 53 M 2024-02-18T00:20:28.011 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 197 M 2024-02-18T00:20:28.011 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:20:28.322 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:00 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:20:28.429 INFO:teuthology.orchestra.run.smithi137.stdout:(1/6): jq-1.6-8.el8.x86_64.rpm 874 kB/s | 203 kB 00:00 2024-02-18T00:20:28.496 INFO:teuthology.orchestra.run.smithi137.stdout:(2/6): libcephsqlite-16.2.14-548.g81bd20d6.el8. 580 kB/s | 174 kB 00:00 2024-02-18T00:20:28.522 INFO:teuthology.orchestra.run.smithi137.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-02-18T00:20:28.589 INFO:teuthology.orchestra.run.smithi137.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-02-18T00:20:28.755 INFO:teuthology.orchestra.run.smithi137.stdout:(5/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.2 MB/s | 323 kB 00:00 2024-02-18T00:20:29.986 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.4 k 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 2.4 M 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 1.6 M 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 245 k 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 5.2 M 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 19 M 2024-02-18T00:20:29.990 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-18T00:20:29.991 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-18T00:20:29.992 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:29.993 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:20:29.993 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:29.993 INFO:teuthology.orchestra.run.smithi059.stdout:Install 32 Packages 2024-02-18T00:20:29.993 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:29.993 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 32 M 2024-02-18T00:20:29.993 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 104 M 2024-02-18T00:20:29.993 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:20:30.647 INFO:teuthology.orchestra.run.smithi137.stdout:(6/6): ceph-test-16.2.14-548.g81bd20d6.el8.x86_ 21 MB/s | 52 MB 00:02 2024-02-18T00:20:30.649 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:20:30.649 INFO:teuthology.orchestra.run.smithi137.stdout:Total 20 MB/s | 53 MB 00:02 2024-02-18T00:20:30.692 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:20:30.712 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:20:30.712 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:20:31.000 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:20:31.003 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:20:31.475 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:20:31.725 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-18T00:20:31.905 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-18T00:20:32.019 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-18T00:20:32.094 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-18T00:20:32.283 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-18T00:20:32.461 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-18T00:20:32.595 INFO:teuthology.orchestra.run.smithi059.stdout:(1/32): ceph-16.2.14-548.g81bd20d6.el8.x86_64.r 51 kB/s | 6.4 kB 00:00 2024-02-18T00:20:32.754 INFO:teuthology.orchestra.run.smithi059.stdout:(2/32): ceph-mgr-16.2.14-548.g81bd20d6.el8.x86_ 5.7 MB/s | 1.6 MB 00:00 2024-02-18T00:20:33.221 INFO:teuthology.orchestra.run.smithi059.stdout:(3/32): ceph-mds-16.2.14-548.g81bd20d6.el8.x86_ 3.3 MB/s | 2.4 MB 00:00 2024-02-18T00:20:33.329 INFO:teuthology.orchestra.run.smithi059.stdout:(4/32): ceph-mgr-modules-core-16.2.14-548.g81bd 2.2 MB/s | 245 kB 00:00 2024-02-18T00:20:33.396 INFO:teuthology.orchestra.run.smithi059.stdout:(5/32): ceph-mon-16.2.14-548.g81bd20d6.el8.x86_ 6.5 MB/s | 5.2 MB 00:00 2024-02-18T00:20:33.538 INFO:teuthology.orchestra.run.smithi059.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 274 kB/s | 39 kB 00:00 2024-02-18T00:20:33.589 INFO:teuthology.orchestra.run.smithi059.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 606 kB/s | 157 kB 00:00 2024-02-18T00:20:33.656 INFO:teuthology.orchestra.run.smithi059.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 885 kB/s | 103 kB 00:00 2024-02-18T00:20:33.684 INFO:teuthology.orchestra.run.smithi059.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 622 kB/s | 57 kB 00:00 2024-02-18T00:20:33.890 INFO:teuthology.orchestra.run.smithi059.stdout:(10/32): ceph-osd-16.2.14-548.g81bd20d6.el8.x86 16 MB/s | 19 MB 00:01 2024-02-18T00:20:33.990 INFO:teuthology.orchestra.run.smithi059.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.3 MB/s | 457 kB 00:00 2024-02-18T00:20:34.290 INFO:teuthology.orchestra.run.smithi059.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 408 kB/s | 249 kB 00:00 2024-02-18T00:20:34.374 INFO:teuthology.orchestra.run.smithi059.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 284 kB/s | 109 kB 00:00 2024-02-18T00:20:34.416 INFO:teuthology.orchestra.run.smithi059.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 348 kB/s | 44 kB 00:00 2024-02-18T00:20:34.450 INFO:teuthology.orchestra.run.smithi059.stdout:(15/32): python3-cryptography-3.2.1-7.el8.x86_6 1.1 MB/s | 641 kB 00:00 2024-02-18T00:20:34.533 INFO:teuthology.orchestra.run.smithi059.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.4 MB/s | 173 kB 00:00 2024-02-18T00:20:34.567 INFO:teuthology.orchestra.run.smithi059.stdout:(17/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 963 kB/s | 185 kB 00:00 2024-02-18T00:20:34.592 INFO:teuthology.orchestra.run.smithi059.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 192 kB/s | 11 kB 00:00 2024-02-18T00:20:34.625 INFO:teuthology.orchestra.run.smithi059.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.1 MB/s | 384 kB 00:00 2024-02-18T00:20:34.650 INFO:teuthology.orchestra.run.smithi059.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 210 kB/s | 18 kB 00:00 2024-02-18T00:20:34.676 INFO:teuthology.orchestra.run.smithi059.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 581 kB/s | 49 kB 00:00 2024-02-18T00:20:34.703 INFO:teuthology.orchestra.run.smithi059.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 779 kB/s | 59 kB 00:00 2024-02-18T00:20:34.726 INFO:teuthology.orchestra.run.smithi059.stdout:(23/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2024-02-18T00:20:34.752 INFO:teuthology.orchestra.run.smithi059.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2024-02-18T00:20:34.777 INFO:teuthology.orchestra.run.smithi059.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 248 kB/s | 19 kB 00:00 2024-02-18T00:20:34.803 INFO:teuthology.orchestra.run.smithi059.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 317 kB/s | 24 kB 00:00 2024-02-18T00:20:34.827 INFO:teuthology.orchestra.run.smithi059.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 388 kB/s | 29 kB 00:00 2024-02-18T00:20:34.878 INFO:teuthology.orchestra.run.smithi059.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 273 kB/s | 27 kB 00:00 2024-02-18T00:20:34.911 INFO:teuthology.orchestra.run.smithi059.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.0 MB/s | 255 kB 00:00 2024-02-18T00:20:34.936 INFO:teuthology.orchestra.run.smithi059.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.3 MB/s | 251 kB 00:00 2024-02-18T00:20:34.953 INFO:teuthology.orchestra.run.smithi059.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2024-02-18T00:20:34.987 INFO:teuthology.orchestra.run.smithi059.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 305 kB/s | 23 kB 00:00 2024-02-18T00:20:34.989 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:20:34.989 INFO:teuthology.orchestra.run.smithi059.stdout:Total 6.4 MB/s | 32 MB 00:04 2024-02-18T00:20:35.072 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:20:35.114 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:20:35.115 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:20:35.440 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:20:35.443 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:20:36.626 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:20:36.855 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-18T00:20:37.056 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-18T00:20:37.089 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 5/6 2024-02-18T00:20:37.316 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-18T00:20:37.345 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-18T00:20:37.602 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 6/6 2024-02-18T00:20:37.602 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 1/6 2024-02-18T00:20:37.602 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2/6 2024-02-18T00:20:37.602 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-18T00:20:37.602 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-18T00:20:37.602 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-18T00:20:37.602 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-18T00:20:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-18T00:20:37.994 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-18T00:20:38.036 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-18T00:20:38.036 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:38.036 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:20:38.036 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:38.036 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-8.el8.x86_64 2024-02-18T00:20:38.037 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:38.037 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.8.2-3.el8.x86_64 2024-02-18T00:20:38.037 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-18T00:20:38.037 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-18T00:20:38.037 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:38.037 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:20:38.156 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-18T00:20:38.249 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph 2024-02-18T00:20:38.377 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-18T00:20:38.643 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-18T00:20:38.886 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-18T00:20:39.008 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:53 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:20:39.141 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-18T00:20:39.418 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-18T00:20:39.850 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-18T00:20:40.135 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-18T00:20:40.360 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-18T00:20:40.578 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 6.4 k 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 2.4 M 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 1.6 M 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 245 k 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 5.2 M 2024-02-18T00:20:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 19 M 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-18T00:20:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:40.584 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:20:40.585 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:20:40.585 INFO:teuthology.orchestra.run.smithi137.stdout:Install 32 Packages 2024-02-18T00:20:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:40.585 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 32 M 2024-02-18T00:20:40.585 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 104 M 2024-02-18T00:20:40.585 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:20:40.599 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-18T00:20:40.814 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-18T00:20:41.149 INFO:teuthology.orchestra.run.smithi137.stdout:(1/32): ceph-16.2.14-548.g81bd20d6.el8.x86_64.r 46 kB/s | 6.4 kB 00:00 2024-02-18T00:20:41.156 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-18T00:20:41.341 INFO:teuthology.orchestra.run.smithi137.stdout:(2/32): ceph-mgr-16.2.14-548.g81bd20d6.el8.x86_ 4.9 MB/s | 1.6 MB 00:00 2024-02-18T00:20:41.412 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-18T00:20:41.574 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-18T00:20:41.776 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-18T00:20:42.024 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-18T00:20:42.234 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-18T00:20:42.480 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-18T00:20:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-18T00:20:42.858 INFO:teuthology.orchestra.run.smithi137.stdout:(3/32): ceph-mds-16.2.14-548.g81bd20d6.el8.x86_ 1.3 MB/s | 2.4 MB 00:01 2024-02-18T00:20:43.042 INFO:teuthology.orchestra.run.smithi137.stdout:(4/32): ceph-osd-16.2.14-548.g81bd20d6.el8.x86_ 11 MB/s | 19 MB 00:01 2024-02-18T00:20:43.067 INFO:teuthology.orchestra.run.smithi137.stdout:(5/32): ceph-mgr-modules-core-16.2.14-548.g81bd 1.1 MB/s | 245 kB 00:00 2024-02-18T00:20:43.186 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-18T00:20:43.209 INFO:teuthology.orchestra.run.smithi137.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 274 kB/s | 39 kB 00:00 2024-02-18T00:20:43.292 INFO:teuthology.orchestra.run.smithi137.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 627 kB/s | 157 kB 00:00 2024-02-18T00:20:43.318 INFO:teuthology.orchestra.run.smithi137.stdout:(8/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 951 kB/s | 103 kB 00:00 2024-02-18T00:20:43.402 INFO:teuthology.orchestra.run.smithi137.stdout:(9/32): ceph-mon-16.2.14-548.g81bd20d6.el8.x86_ 2.3 MB/s | 5.2 MB 00:02 2024-02-18T00:20:43.427 INFO:teuthology.orchestra.run.smithi137.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 426 kB/s | 57 kB 00:00 2024-02-18T00:20:43.535 INFO:teuthology.orchestra.run.smithi137.stdout:(11/32): python3-werkzeug-0.12.2-4.el8.noarch.r 2.1 MB/s | 457 kB 00:00 2024-02-18T00:20:43.588 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 27/32 2024-02-18T00:20:43.809 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-18T00:20:43.902 INFO:teuthology.orchestra.run.smithi137.stdout:(12/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 497 kB/s | 249 kB 00:00 2024-02-18T00:20:43.927 INFO:teuthology.orchestra.run.smithi137.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 278 kB/s | 109 kB 00:00 2024-02-18T00:20:44.003 INFO:teuthology.orchestra.run.smithi137.stdout:(14/32): python3-cryptography-3.2.1-7.el8.x86_6 1.1 MB/s | 641 kB 00:00 2024-02-18T00:20:44.036 INFO:teuthology.orchestra.run.smithi137.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 326 kB/s | 44 kB 00:00 2024-02-18T00:20:44.087 INFO:teuthology.orchestra.run.smithi137.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.2 MB/s | 185 kB 00:00 2024-02-18T00:20:44.145 INFO:teuthology.orchestra.run.smithi137.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 192 kB/s | 11 kB 00:00 2024-02-18T00:20:44.179 INFO:teuthology.orchestra.run.smithi137.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 532 kB/s | 18 kB 00:00 2024-02-18T00:20:44.204 INFO:teuthology.orchestra.run.smithi137.stdout:(19/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 859 kB/s | 173 kB 00:00 2024-02-18T00:20:44.229 INFO:teuthology.orchestra.run.smithi137.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.0 MB/s | 384 kB 00:00 2024-02-18T00:20:44.254 INFO:teuthology.orchestra.run.smithi137.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 646 kB/s | 49 kB 00:00 2024-02-18T00:20:44.280 INFO:teuthology.orchestra.run.smithi137.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 779 kB/s | 59 kB 00:00 2024-02-18T00:20:44.310 INFO:teuthology.orchestra.run.smithi137.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2024-02-18T00:20:44.330 INFO:teuthology.orchestra.run.smithi137.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 373 kB/s | 19 kB 00:00 2024-02-18T00:20:44.356 INFO:teuthology.orchestra.run.smithi137.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 476 kB/s | 24 kB 00:00 2024-02-18T00:20:44.383 INFO:teuthology.orchestra.run.smithi137.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 556 kB/s | 29 kB 00:00 2024-02-18T00:20:44.408 INFO:teuthology.orchestra.run.smithi137.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 527 kB/s | 27 kB 00:00 2024-02-18T00:20:44.434 INFO:teuthology.orchestra.run.smithi137.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2024-02-18T00:20:44.459 INFO:teuthology.orchestra.run.smithi137.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 3.3 MB/s | 255 kB 00:00 2024-02-18T00:20:44.484 INFO:teuthology.orchestra.run.smithi137.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2024-02-18T00:20:44.513 INFO:teuthology.orchestra.run.smithi137.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.4 MB/s | 251 kB 00:00 2024-02-18T00:20:44.535 INFO:teuthology.orchestra.run.smithi137.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 305 kB/s | 23 kB 00:00 2024-02-18T00:20:44.539 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:20:44.539 INFO:teuthology.orchestra.run.smithi137.stdout:Total 8.0 MB/s | 32 MB 00:03 2024-02-18T00:20:44.620 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:20:44.658 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:20:44.659 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:20:44.976 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:20:44.979 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:20:45.426 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-18T00:20:45.585 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-18T00:20:46.072 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-18T00:20:46.081 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:20:46.244 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-18T00:20:46.296 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-18T00:20:46.472 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-18T00:20:46.483 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-18T00:20:46.669 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-18T00:20:46.687 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-18T00:20:46.750 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-18T00:20:46.802 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-18T00:20:47.023 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-18T00:20:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-18T00:20:47.216 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 1/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 3/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 4/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 5/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 6/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-18T00:20:47.217 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-18T00:20:47.218 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-18T00:20:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-18T00:20:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-18T00:20:47.219 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-18T00:20:47.220 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-18T00:20:47.418 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-18T00:20:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-18T00:20:47.830 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-18T00:20:48.040 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-18T00:20:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-18T00:20:48.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-18T00:20:48.105 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-2.6-1.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-18T00:20:48.106 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-18T00:20:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-18T00:20:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-18T00:20:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-18T00:20:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-18T00:20:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-18T00:20:48.107 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:48.107 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:20:48.277 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-18T00:20:48.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-base 2024-02-18T00:20:48.506 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-18T00:20:48.755 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-18T00:20:49.136 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-18T00:20:49.185 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:21 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:20:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-18T00:20:49.622 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-18T00:20:49.909 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-18T00:20:50.126 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-18T00:20:50.428 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-18T00:20:50.655 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-18T00:20:50.813 INFO:teuthology.orchestra.run.smithi059.stdout:Package ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:20:50.843 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-18T00:20:50.863 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:20:50.864 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:20:50.864 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:20:50.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install cephadm 2024-02-18T00:20:51.038 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-18T00:20:51.286 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-18T00:20:51.512 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-18T00:20:51.738 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-18T00:20:51.758 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:23 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:20:52.061 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-18T00:20:52.393 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-18T00:20:52.742 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 27/32 2024-02-18T00:20:52.956 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-18T00:20:53.422 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:20:53.423 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:53.423 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:20:53.423 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:53.423 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:20:53.423 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 85 k 2024-02-18T00:20:53.423 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:53.423 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:20:53.424 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:53.424 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-02-18T00:20:53.424 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:53.424 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 85 k 2024-02-18T00:20:53.424 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 352 k 2024-02-18T00:20:53.424 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:20:53.611 INFO:teuthology.orchestra.run.smithi059.stdout:cephadm-16.2.14-548.g81bd20d6.el8.noarch.rpm 458 kB/s | 85 kB 00:00 2024-02-18T00:20:53.612 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:20:53.612 INFO:teuthology.orchestra.run.smithi059.stdout:Total 455 kB/s | 85 kB 00:00 2024-02-18T00:20:53.612 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:20:53.619 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:20:53.619 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:20:53.621 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:20:53.625 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:20:53.671 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:20:54.189 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:20:54.371 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:20:54.469 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 28/32 2024-02-18T00:20:54.608 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:20:54.641 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-18T00:20:55.085 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 29/32 2024-02-18T00:20:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-18T00:20:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:20:55.454 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:55.454 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:20:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:20:55.455 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:55.455 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:20:55.498 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 30/32 2024-02-18T00:20:55.690 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-18T00:20:55.708 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 31/32 2024-02-18T00:20:55.873 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-18T00:20:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 32/32 2024-02-18T00:20:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 1/32 2024-02-18T00:20:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2/32 2024-02-18T00:20:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 3/32 2024-02-18T00:20:56.276 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 4/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 5/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8. 6/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-18T00:20:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-18T00:20:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-18T00:20:57.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-18T00:20:57.163 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:57.164 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:20:57.164 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:57.164 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:57.164 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:57.164 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:20:57.165 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:57.165 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:20:57.165 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-18T00:20:57.165 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-18T00:20:57.165 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-18T00:20:57.165 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-18T00:20:57.166 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-18T00:20:57.166 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-18T00:20:57.166 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-18T00:20:57.166 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-18T00:20:57.166 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-18T00:20:57.166 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-18T00:20:57.167 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-18T00:20:57.167 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-18T00:20:57.167 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-18T00:20:57.167 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-2.6-1.el8.noarch 2024-02-18T00:20:57.167 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-18T00:20:57.167 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-18T00:20:57.168 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-18T00:20:57.168 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-18T00:20:57.168 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-18T00:20:57.168 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-18T00:20:57.168 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-18T00:20:57.168 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-18T00:20:57.169 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-18T00:20:57.169 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-18T00:20:57.169 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-18T00:20:57.169 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-18T00:20:57.170 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:20:57.170 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:20:57.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-immutable-object-cache 2024-02-18T00:20:57.829 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-base 2024-02-18T00:20:57.961 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:29 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:20:58.588 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:12 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:20:59.619 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:20:59.619 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 161 k 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-02-18T00:20:59.620 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:20:59.621 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 161 k 2024-02-18T00:20:59.621 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 470 k 2024-02-18T00:20:59.621 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:20:59.833 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-immutable-object-cache-16.2.14-548.g81bd20 762 kB/s | 161 kB 00:00 2024-02-18T00:20:59.833 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:20:59.833 INFO:teuthology.orchestra.run.smithi059.stdout:Total 757 kB/s | 161 kB 00:00 2024-02-18T00:20:59.834 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:20:59.846 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:20:59.846 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:20:59.896 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:20:59.899 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:21:00.021 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:21:00.103 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-base-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:21:00.149 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:00.150 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:21:00.150 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:00.192 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-18T00:21:00.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install cephadm 2024-02-18T00:21:00.631 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-18T00:21:01.004 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:15 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:01.029 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-18T00:21:01.030 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:01.030 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:21:01.030 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:21:01.030 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:01.030 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:01.257 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr 2024-02-18T00:21:02.044 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:34 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:02.534 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:02.535 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:02.535 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:21:02.535 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:02.535 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:02.535 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 85 k 2024-02-18T00:21:02.535 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:02.536 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:02.536 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:02.536 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-02-18T00:21:02.536 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:02.536 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 85 k 2024-02-18T00:21:02.536 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 352 k 2024-02-18T00:21:02.536 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:02.701 INFO:teuthology.orchestra.run.smithi137.stdout:cephadm-16.2.14-548.g81bd20d6.el8.noarch.rpm 517 kB/s | 85 kB 00:00 2024-02-18T00:21:02.701 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:02.702 INFO:teuthology.orchestra.run.smithi137.stdout:Total 513 kB/s | 85 kB 00:00 2024-02-18T00:21:02.702 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:02.709 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:02.709 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:02.711 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:02.714 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:02.926 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:03.446 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:21:03.642 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:21:03.659 INFO:teuthology.orchestra.run.smithi059.stdout:Package ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:21:03.709 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:21:03.710 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:21:03.711 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:03.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-dashboard 2024-02-18T00:21:03.868 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:21:04.250 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/1 2024-02-18T00:21:04.251 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:04.251 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:21:04.251 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:04.251 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:04.251 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:04.501 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-immutable-object-cache 2024-02-18T00:21:04.592 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:36 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:05.260 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:19 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:06.252 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:21:06.254 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:06.254 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:21:06.254 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 3.1 M 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 24 k 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 15 k 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout:Install 6 Packages 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:06.255 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 3.4 M 2024-02-18T00:21:06.256 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 22 M 2024-02-18T00:21:06.256 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:21:06.719 INFO:teuthology.orchestra.run.smithi059.stdout:(1/6): ceph-grafana-dashboards-16.2.14-548.g81b 177 kB/s | 24 kB 00:00 2024-02-18T00:21:06.823 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:06.823 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:06.823 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-02-18T00:21:06.823 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:06.823 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:06.823 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 161 k 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 161 k 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 470 k 2024-02-18T00:21:06.824 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:06.850 INFO:teuthology.orchestra.run.smithi059.stdout:(2/6): ceph-prometheus-alerts-16.2.14-548.g81bd 55 kB/s | 15 kB 00:00 2024-02-18T00:21:07.111 INFO:teuthology.orchestra.run.smithi059.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 109 kB/s | 43 kB 00:00 2024-02-18T00:21:07.148 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-immutable-object-cache-16.2.14-548.g81bd20 498 kB/s | 161 kB 00:00 2024-02-18T00:21:07.148 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:07.149 INFO:teuthology.orchestra.run.smithi137.stdout:Total 496 kB/s | 161 kB 00:00 2024-02-18T00:21:07.149 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:07.159 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:07.159 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:07.206 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:07.209 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:07.211 INFO:teuthology.orchestra.run.smithi059.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 93 kB/s | 34 kB 00:00 2024-02-18T00:21:07.321 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:07.538 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-18T00:21:07.670 INFO:teuthology.orchestra.run.smithi059.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 351 kB/s | 196 kB 00:00 2024-02-18T00:21:07.887 INFO:teuthology.orchestra.run.smithi059.stdout:(6/6): ceph-mgr-dashboard-16.2.14-548.g81bd20d6 2.4 MB/s | 3.1 MB 00:01 2024-02-18T00:21:07.890 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:07.891 INFO:teuthology.orchestra.run.smithi059.stdout:Total 2.1 MB/s | 3.4 MB 00:01 2024-02-18T00:21:07.903 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:21:07.915 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:21:07.915 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:21:07.936 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-18T00:21:08.088 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:21:08.091 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:21:08.314 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6. 1/1 2024-02-18T00:21:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:08.315 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:21:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:21:08.315 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:08.315 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:08.389 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:21:08.616 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-18T00:21:08.835 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-18T00:21:09.026 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-18T00:21:09.156 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 4/6 2024-02-18T00:21:09.701 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr 2024-02-18T00:21:10.288 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 5/6 2024-02-18T00:21:10.520 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:24 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:10.597 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-18T00:21:10.897 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-18T00:21:10.897 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 1/6 2024-02-18T00:21:10.897 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 2/6 2024-02-18T00:21:10.897 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 3/6 2024-02-18T00:21:10.897 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-18T00:21:10.897 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-18T00:21:11.304 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-18T00:21:11.304 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:11.304 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:21:11.304 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:11.305 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:11.305 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:11.305 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-18T00:21:11.305 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-18T00:21:11.305 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-18T00:21:11.305 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:11.305 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:11.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-18T00:21:12.033 INFO:teuthology.orchestra.run.smithi137.stdout:Package ceph-mgr-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:21:12.080 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:12.082 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:21:12.082 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:12.193 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-dashboard 2024-02-18T00:21:12.213 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:44 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:12.951 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:26 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:13.877 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:21:13.878 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================ 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================ 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 8.6 M 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-18T00:21:13.879 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-18T00:21:13.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:13.880 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:21:13.880 INFO:teuthology.orchestra.run.smithi059.stdout:============================================================================================ 2024-02-18T00:21:13.880 INFO:teuthology.orchestra.run.smithi059.stdout:Install 8 Packages 2024-02-18T00:21:13.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:13.880 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 37 M 2024-02-18T00:21:13.880 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 210 M 2024-02-18T00:21:13.881 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:21:14.470 INFO:teuthology.orchestra.run.smithi059.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 9.3 MB/s | 4.7 MB 00:00 2024-02-18T00:21:14.485 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:14.486 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 3.1 M 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 24 k 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 15 k 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:14.487 INFO:teuthology.orchestra.run.smithi137.stdout:Install 6 Packages 2024-02-18T00:21:14.488 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:14.488 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 3.4 M 2024-02-18T00:21:14.488 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 22 M 2024-02-18T00:21:14.488 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:14.551 INFO:teuthology.orchestra.run.smithi059.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 8.2 MB/s | 4.9 MB 00:00 2024-02-18T00:21:14.618 INFO:teuthology.orchestra.run.smithi059.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 3.3 MB/s | 225 kB 00:00 2024-02-18T00:21:14.685 INFO:teuthology.orchestra.run.smithi059.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 17 MB/s | 3.7 MB 00:00 2024-02-18T00:21:15.143 INFO:teuthology.orchestra.run.smithi059.stdout:(5/8): python3-scipy-1.0.0-21.module_el8+762+77 27 MB/s | 14 MB 00:00 2024-02-18T00:21:15.343 INFO:teuthology.orchestra.run.smithi137.stdout:(1/6): ceph-prometheus-alerts-16.2.14-548.g81bd 45 kB/s | 15 kB 00:00 2024-02-18T00:21:15.368 INFO:teuthology.orchestra.run.smithi137.stdout:(2/6): ceph-grafana-dashboards-16.2.14-548.g81b 69 kB/s | 24 kB 00:00 2024-02-18T00:21:15.369 INFO:teuthology.orchestra.run.smithi059.stdout:(6/8): libgfortran-8.5.0-21.el8.x86_64.rpm 943 kB/s | 645 kB 00:00 2024-02-18T00:21:15.510 INFO:teuthology.orchestra.run.smithi137.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 237 kB/s | 34 kB 00:00 2024-02-18T00:21:15.578 INFO:teuthology.orchestra.run.smithi059.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 397 kB/s | 172 kB 00:00 2024-02-18T00:21:15.652 INFO:teuthology.orchestra.run.smithi137.stdout:(4/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.4 MB/s | 196 kB 00:00 2024-02-18T00:21:15.686 INFO:teuthology.orchestra.run.smithi137.stdout:(5/6): python3-jwt-1.6.1-2.el8.noarch.rpm 124 kB/s | 43 kB 00:00 2024-02-18T00:21:16.395 INFO:teuthology.orchestra.run.smithi059.stdout:(8/8): ceph-mgr-diskprediction-local-16.2.14-54 3.5 MB/s | 8.6 MB 00:02 2024-02-18T00:21:16.395 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:16.395 INFO:teuthology.orchestra.run.smithi059.stdout:Total 15 MB/s | 37 MB 00:02 2024-02-18T00:21:16.637 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:21:16.656 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:21:16.656 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:21:17.013 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:21:17.016 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:21:17.202 INFO:teuthology.orchestra.run.smithi137.stdout:(6/6): ceph-mgr-dashboard-16.2.14-548.g81bd20d6 1.4 MB/s | 3.1 MB 00:02 2024-02-18T00:21:17.204 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:17.205 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.3 MB/s | 3.4 MB 00:02 2024-02-18T00:21:17.217 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:17.228 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:17.228 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:17.387 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:17.390 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:17.675 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:17.727 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:21:17.900 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-18T00:21:18.145 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-18T00:21:18.356 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-18T00:21:18.363 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-18T00:21:18.476 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 4/6 2024-02-18T00:21:18.507 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-18T00:21:18.682 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-18T00:21:19.323 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-18T00:21:19.472 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-18T00:21:19.507 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 5/6 2024-02-18T00:21:19.759 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-18T00:21:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 6/6 2024-02-18T00:21:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8. 1/6 2024-02-18T00:21:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarc 2/6 2024-02-18T00:21:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.n 3/6 2024-02-18T00:21:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-18T00:21:20.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-18T00:21:20.125 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-18T00:21:20.290 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-18T00:21:20.452 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-18T00:21:20.452 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:20.452 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:21:20.452 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:20.453 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:20.453 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:20.453 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-18T00:21:20.453 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-18T00:21:20.453 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-18T00:21:20.453 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:20.453 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:20.585 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-18T00:21:21.029 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-18T00:21:21.341 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:35 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:21.364 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-18T00:21:21.640 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-18T00:21:22.881 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:22.882 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-02-18T00:21:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:21:22.882 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-02-18T00:21:22.882 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 8.6 M 2024-02-18T00:21:22.882 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:21:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout:Install 8 Packages 2024-02-18T00:21:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:22.884 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 37 M 2024-02-18T00:21:22.884 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 210 M 2024-02-18T00:21:22.884 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:23.522 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-18T00:21:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 9.5 MB/s | 4.7 MB 00:00 2024-02-18T00:21:23.833 INFO:teuthology.orchestra.run.smithi137.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 8.5 MB/s | 4.9 MB 00:00 2024-02-18T00:21:23.900 INFO:teuthology.orchestra.run.smithi137.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 3.3 MB/s | 225 kB 00:00 2024-02-18T00:21:24.017 INFO:teuthology.orchestra.run.smithi137.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 14 MB/s | 3.7 MB 00:00 2024-02-18T00:21:24.126 INFO:teuthology.orchestra.run.smithi137.stdout:(5/8): ceph-mgr-diskprediction-local-16.2.14-54 9.9 MB/s | 8.6 MB 00:00 2024-02-18T00:21:24.476 INFO:teuthology.orchestra.run.smithi137.stdout:(6/8): python3-scipy-1.0.0-21.module_el8+762+77 25 MB/s | 14 MB 00:00 2024-02-18T00:21:24.601 INFO:teuthology.orchestra.run.smithi137.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 363 kB/s | 172 kB 00:00 2024-02-18T00:21:24.668 INFO:teuthology.orchestra.run.smithi137.stdout:(8/8): libgfortran-8.5.0-21.el8.x86_64.rpm 992 kB/s | 645 kB 00:00 2024-02-18T00:21:24.669 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:24.669 INFO:teuthology.orchestra.run.smithi137.stdout:Total 21 MB/s | 37 MB 00:01 2024-02-18T00:21:24.890 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-18T00:21:24.899 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:24.918 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:24.918 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:25.098 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-18T00:21:25.257 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:25.260 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 1/8 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-18T00:21:25.422 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-18T00:21:25.846 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-18T00:21:25.852 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-18T00:21:25.853 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-18T00:21:25.853 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-18T00:21:25.853 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-18T00:21:25.853 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:25.853 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:25.968 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-18T00:21:26.013 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-rook 2024-02-18T00:21:26.099 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-18T00:21:26.235 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-18T00:21:26.811 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:58 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:26.871 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-18T00:21:27.302 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-18T00:21:27.910 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-18T00:21:28.051 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-18T00:21:28.477 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 27 k 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-18T00:21:28.479 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout:Install 13 Packages 2024-02-18T00:21:28.480 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:28.481 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 1.9 M 2024-02-18T00:21:28.481 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 24 M 2024-02-18T00:21:28.481 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:21:28.731 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-18T00:21:29.054 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-18T00:21:29.084 INFO:teuthology.orchestra.run.smithi059.stdout:(1/13): ceph-mgr-rook-16.2.14-548.g81bd20d6.el8 140 kB/s | 27 kB 00:00 2024-02-18T00:21:29.100 INFO:teuthology.orchestra.run.smithi059.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 125 kB/s | 27 kB 00:00 2024-02-18T00:21:29.117 INFO:teuthology.orchestra.run.smithi059.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 87 kB/s | 20 kB 00:00 2024-02-18T00:21:29.251 INFO:teuthology.orchestra.run.smithi059.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 756 kB/s | 126 kB 00:00 2024-02-18T00:21:29.276 INFO:teuthology.orchestra.run.smithi059.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 695 kB/s | 110 kB 00:00 2024-02-18T00:21:29.293 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-18T00:21:29.551 INFO:teuthology.orchestra.run.smithi059.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 358 kB/s | 155 kB 00:00 2024-02-18T00:21:29.585 INFO:teuthology.orchestra.run.smithi059.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 128 kB/s | 43 kB 00:00 2024-02-18T00:21:30.169 INFO:teuthology.orchestra.run.smithi059.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 37 kB/s | 33 kB 00:00 2024-02-18T00:21:30.419 INFO:teuthology.orchestra.run.smithi059.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 19 kB/s | 16 kB 00:00 2024-02-18T00:21:30.695 INFO:teuthology.orchestra.run.smithi059.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 73 kB/s | 81 kB 00:01 2024-02-18T00:21:30.962 INFO:teuthology.orchestra.run.smithi059.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 118 kB/s | 64 kB 00:00 2024-02-18T00:21:31.003 INFO:teuthology.orchestra.run.smithi059.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 198 kB/s | 61 kB 00:00 2024-02-18T00:21:31.055 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-18T00:21:31.429 INFO:teuthology.orchestra.run.smithi059.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 948 kB/s | 1.2 MB 00:01 2024-02-18T00:21:31.432 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:31.432 INFO:teuthology.orchestra.run.smithi059.stdout:Total 663 kB/s | 1.9 MB 00:02 2024-02-18T00:21:31.491 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:21:31.508 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:21:31.508 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:21:31.603 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:21:31.606 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:21:32.237 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:21:32.358 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-18T00:21:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-18T00:21:32.617 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-18T00:21:32.692 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-18T00:21:32.941 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-18T00:21:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 8/8 2024-02-18T00:21:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d 1/8 2024-02-18T00:21:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-18T00:21:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-18T00:21:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-18T00:21:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-18T00:21:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-18T00:21:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-18T00:21:33.195 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-18T00:21:33.361 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:33.362 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:33.387 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-18T00:21:33.491 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-rook 2024-02-18T00:21:33.638 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-18T00:21:33.967 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-18T00:21:34.187 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-18T00:21:34.245 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:48 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:35.264 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-18T00:21:35.552 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-18T00:21:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-18T00:21:35.781 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 27 k 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-18T00:21:35.783 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:35.784 INFO:teuthology.orchestra.run.smithi137.stdout:Install 13 Packages 2024-02-18T00:21:35.785 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:35.785 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 1.9 M 2024-02-18T00:21:35.785 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 24 M 2024-02-18T00:21:35.785 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:35.929 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-18T00:21:36.078 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 1/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-18T00:21:36.434 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-18T00:21:36.435 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-18T00:21:36.435 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-18T00:21:36.435 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-18T00:21:36.435 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-18T00:21:36.435 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-18T00:21:36.492 INFO:teuthology.orchestra.run.smithi137.stdout:(1/13): ceph-mgr-rook-16.2.14-548.g81bd20d6.el8 200 kB/s | 27 kB 00:00 2024-02-18T00:21:36.517 INFO:teuthology.orchestra.run.smithi137.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 164 kB/s | 27 kB 00:00 2024-02-18T00:21:36.542 INFO:teuthology.orchestra.run.smithi137.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 106 kB/s | 20 kB 00:00 2024-02-18T00:21:36.634 INFO:teuthology.orchestra.run.smithi137.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 890 kB/s | 126 kB 00:00 2024-02-18T00:21:36.685 INFO:teuthology.orchestra.run.smithi137.stdout:(5/13): python3-pyasn1-modules-0.3.7-6.el8.noar 776 kB/s | 110 kB 00:00 2024-02-18T00:21:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-18T00:21:36.951 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-18T00:21:36.952 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-18T00:21:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-18T00:21:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-18T00:21:36.953 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:36.953 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:36.960 INFO:teuthology.orchestra.run.smithi137.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 131 kB/s | 43 kB 00:00 2024-02-18T00:21:36.985 INFO:teuthology.orchestra.run.smithi137.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 350 kB/s | 155 kB 00:00 2024-02-18T00:21:37.068 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-mgr-cephadm 2024-02-18T00:21:37.094 INFO:teuthology.orchestra.run.smithi137.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 81 kB/s | 33 kB 00:00 2024-02-18T00:21:37.328 INFO:teuthology.orchestra.run.smithi137.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 44 kB/s | 16 kB 00:00 2024-02-18T00:21:37.470 INFO:teuthology.orchestra.run.smithi137.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 168 kB/s | 81 kB 00:00 2024-02-18T00:21:37.612 INFO:teuthology.orchestra.run.smithi137.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 226 kB/s | 64 kB 00:00 2024-02-18T00:21:37.637 INFO:teuthology.orchestra.run.smithi137.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 367 kB/s | 61 kB 00:00 2024-02-18T00:21:37.671 INFO:teuthology.orchestra.run.smithi137.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.0 MB/s | 1.2 MB 00:00 2024-02-18T00:21:37.674 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:37.674 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.0 MB/s | 1.9 MB 00:01 2024-02-18T00:21:37.731 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:37.746 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:37.747 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:37.835 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:37.837 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:37.866 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:09 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:38.457 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:38.689 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-18T00:21:38.909 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-18T00:21:39.170 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-18T00:21:39.407 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-18T00:21:39.529 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:21:39.530 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:39.530 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:21:39.530 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:39.530 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:21:39.530 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 116 k 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout:Install 6 Packages 2024-02-18T00:21:39.531 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:39.532 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 5.9 M 2024-02-18T00:21:39.532 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 24 M 2024-02-18T00:21:39.532 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:21:39.632 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-18T00:21:39.899 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-18T00:21:39.994 INFO:teuthology.orchestra.run.smithi059.stdout:(1/6): ceph-mgr-cephadm-16.2.14-548.g81bd20d6.e 403 kB/s | 116 kB 00:00 2024-02-18T00:21:40.019 INFO:teuthology.orchestra.run.smithi059.stdout:(2/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.8 MB/s | 578 kB 00:00 2024-02-18T00:21:40.185 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-18T00:21:40.220 INFO:teuthology.orchestra.run.smithi059.stdout:(3/6): python3-babel-2.5.1-7.el8.noarch.rpm 9.3 MB/s | 4.8 MB 00:00 2024-02-18T00:21:40.303 INFO:teuthology.orchestra.run.smithi059.stdout:(4/6): python3-apipkg-1.5-6.el8.noarch.rpm 69 kB/s | 21 kB 00:00 2024-02-18T00:21:40.431 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-18T00:21:41.546 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-18T00:21:41.848 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-18T00:21:42.073 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-18T00:21:42.137 INFO:teuthology.orchestra.run.smithi059.stdout:(5/6): python3-execnet-1.7.1-1.el8.noarch.rpm 168 kB/s | 355 kB 00:02 2024-02-18T00:21:42.265 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-18T00:21:42.416 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-18T00:21:42.733 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 13/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 1/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-18T00:21:42.734 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-18T00:21:43.183 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-18T00:21:43.183 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:43.183 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:21:43.183 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-18T00:21:43.184 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-18T00:21:43.185 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:43.185 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:43.305 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-mgr-cephadm 2024-02-18T00:21:43.604 INFO:teuthology.orchestra.run.smithi059.stdout:(6/6): python3-remoto-1.2.1-1.el8.noarch.rpm 14 kB/s | 49 kB 00:03 2024-02-18T00:21:43.604 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:43.605 INFO:teuthology.orchestra.run.smithi059.stdout:Total 1.4 MB/s | 5.9 MB 00:04 2024-02-18T00:21:43.662 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:21:43.673 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:21:43.674 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:21:43.783 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:21:43.786 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:21:44.066 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:58 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:44.077 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:21:44.458 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-18T00:21:44.716 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-18T00:21:45.613 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:45.614 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:45.614 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:21:45.614 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:45.614 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:45.614 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:16.2.14-548.g81bd20d6.el8 ceph-noarch 116 k 2024-02-18T00:21:45.614 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout:Install 6 Packages 2024-02-18T00:21:45.615 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:45.616 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 5.9 M 2024-02-18T00:21:45.616 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 24 M 2024-02-18T00:21:45.616 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:45.699 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-18T00:21:46.091 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-18T00:21:46.128 INFO:teuthology.orchestra.run.smithi137.stdout:(1/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 1.6 MB/s | 578 kB 00:00 2024-02-18T00:21:46.321 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-18T00:21:46.345 INFO:teuthology.orchestra.run.smithi137.stdout:(2/6): python3-babel-2.5.1-7.el8.noarch.rpm 8.5 MB/s | 4.8 MB 00:00 2024-02-18T00:21:46.379 INFO:teuthology.orchestra.run.smithi137.stdout:(3/6): python3-apipkg-1.5-6.el8.noarch.rpm 85 kB/s | 21 kB 00:00 2024-02-18T00:21:46.412 INFO:teuthology.orchestra.run.smithi137.stdout:(4/6): ceph-mgr-cephadm-16.2.14-548.g81bd20d6.e 184 kB/s | 116 kB 00:00 2024-02-18T00:21:46.535 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-18T00:21:46.546 INFO:teuthology.orchestra.run.smithi137.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 293 kB/s | 49 kB 00:00 2024-02-18T00:21:46.821 INFO:teuthology.orchestra.run.smithi137.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 748 kB/s | 355 kB 00:00 2024-02-18T00:21:46.821 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:46.822 INFO:teuthology.orchestra.run.smithi137.stdout:Total 4.9 MB/s | 5.9 MB 00:01 2024-02-18T00:21:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-18T00:21:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/6 2024-02-18T00:21:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-18T00:21:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-18T00:21:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-18T00:21:46.836 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-18T00:21:46.875 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:46.885 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:46.885 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:46.989 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:46.992 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:47.238 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:47.239 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:47.270 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:47.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-fuse 2024-02-18T00:21:47.522 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-18T00:21:47.809 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-18T00:21:48.155 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:20 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:48.748 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-18T00:21:49.100 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-18T00:21:49.360 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-18T00:21:49.548 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-18T00:21:49.813 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:21:49.814 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-02-18T00:21:49.814 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:49.814 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:21:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 911 k 2024-02-18T00:21:49.814 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-02-18T00:21:49.814 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout:Install 2 Packages 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 994 k 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 2.8 M 2024-02-18T00:21:49.815 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:21:49.840 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 6/6 2024-02-18T00:21:49.840 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 1/6 2024-02-18T00:21:49.840 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-18T00:21:49.840 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-18T00:21:49.841 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-18T00:21:49.841 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-18T00:21:50.190 INFO:teuthology.orchestra.run.smithi059.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 248 kB/s | 83 kB 00:00 2024-02-18T00:21:50.225 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-18T00:21:50.225 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:50.225 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:21:50.225 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:16.2.14-548.g81bd20d6.el8.noarch 2024-02-18T00:21:50.225 INFO:teuthology.orchestra.run.smithi137.stdout: python3-apipkg-1.5-6.el8.noarch 2024-02-18T00:21:50.225 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-18T00:21:50.225 INFO:teuthology.orchestra.run.smithi137.stdout: python3-execnet-1.7.1-1.el8.noarch 2024-02-18T00:21:50.226 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.10.1-4.el8.noarch 2024-02-18T00:21:50.226 INFO:teuthology.orchestra.run.smithi137.stdout: python3-remoto-1.2.1-1.el8.noarch 2024-02-18T00:21:50.226 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:50.226 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:50.371 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-fuse 2024-02-18T00:21:50.723 INFO:teuthology.orchestra.run.smithi059.stdout:(2/2): ceph-fuse-16.2.14-548.g81bd20d6.el8.x86_ 1.0 MB/s | 911 kB 00:00 2024-02-18T00:21:50.724 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:50.725 INFO:teuthology.orchestra.run.smithi059.stdout:Total 1.1 MB/s | 994 kB 00:00 2024-02-18T00:21:50.737 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:21:50.760 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:21:50.760 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:21:50.860 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:21:50.863 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:21:51.038 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:21:51.134 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:05 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:51.305 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-18T00:21:51.529 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-18T00:21:51.944 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-18T00:21:51.945 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/2 2024-02-18T00:21:52.398 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-18T00:21:52.399 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:52.399 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:21:52.399 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-18T00:21:52.399 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:52.399 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:52.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install librados-devel 2024-02-18T00:21:52.687 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:52.688 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:52.688 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-02-18T00:21:52.688 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 911 k 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout:Install 2 Packages 2024-02-18T00:21:52.689 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:52.690 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 994 k 2024-02-18T00:21:52.690 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 2.8 M 2024-02-18T00:21:52.691 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:53.141 INFO:teuthology.orchestra.run.smithi137.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 202 kB/s | 83 kB 00:00 2024-02-18T00:21:53.438 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:25 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:53.657 INFO:teuthology.orchestra.run.smithi137.stdout:(2/2): ceph-fuse-16.2.14-548.g81bd20d6.el8.x86_ 982 kB/s | 911 kB 00:00 2024-02-18T00:21:53.658 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:53.659 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.0 MB/s | 994 kB 00:00 2024-02-18T00:21:53.670 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:53.692 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:53.692 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:53.787 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:53.790 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:53.969 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:54.262 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-18T00:21:54.480 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-18T00:21:54.867 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2/2 2024-02-18T00:21:54.867 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/2 2024-02-18T00:21:55.119 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 131 k 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:21:55.120 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:21:55.121 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-02-18T00:21:55.121 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:55.121 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 131 k 2024-02-18T00:21:55.121 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 432 k 2024-02-18T00:21:55.121 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:21:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-18T00:21:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:55.262 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:21:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-18T00:21:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:55.262 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:21:55.493 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados-devel 2024-02-18T00:21:56.163 INFO:teuthology.orchestra.run.smithi059.stdout:librados-devel-16.2.14-548.g81bd20d6.el8.x86_64 126 kB/s | 131 kB 00:01 2024-02-18T00:21:56.164 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:56.165 INFO:teuthology.orchestra.run.smithi059.stdout:Total 126 kB/s | 131 kB 00:01 2024-02-18T00:21:56.165 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:21:56.172 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:21:56.172 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:21:56.231 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:21:56.235 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:21:56.264 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:10 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:21:56.374 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:21:56.625 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:21:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:21:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:21:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:57.339 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:21:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:21:57.339 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:21:57.339 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:21:57.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install libcephfs2 2024-02-18T00:21:57.816 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:21:57.816 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:57.816 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-02-18T00:21:57.816 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:57.817 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:21:57.817 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 131 k 2024-02-18T00:21:57.817 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:57.817 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:21:57.817 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:21:57.817 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-02-18T00:21:57.817 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:21:57.818 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 131 k 2024-02-18T00:21:57.818 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 432 k 2024-02-18T00:21:57.818 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:21:58.358 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:30 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:21:58.511 INFO:teuthology.orchestra.run.smithi137.stdout:librados-devel-16.2.14-548.g81bd20d6.el8.x86_64 189 kB/s | 131 kB 00:00 2024-02-18T00:21:58.512 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:21:58.512 INFO:teuthology.orchestra.run.smithi137.stdout:Total 189 kB/s | 131 kB 00:00 2024-02-18T00:21:58.512 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:21:58.520 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:21:58.520 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:21:58.571 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:21:58.575 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:21:58.951 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:21:59.687 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:21:59.971 INFO:teuthology.orchestra.run.smithi059.stdout:Package libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:21:59.984 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:00.021 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:00.023 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:22:00.023 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:00.113 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install libcephfs-devel 2024-02-18T00:22:00.359 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:00.360 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:00.360 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:22:00.360 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:00.360 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:00.360 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:00.605 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs2 2024-02-18T00:22:00.905 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:32 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:01.365 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:15 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:02.583 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:02.584 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:02.584 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-02-18T00:22:02.584 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:02.584 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:22:02.584 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 25 k 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 25 k 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 107 k 2024-02-18T00:22:02.585 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:22:02.719 INFO:teuthology.orchestra.run.smithi059.stdout:libcephfs-devel-16.2.14-548.g81bd20d6.el8.x86_6 191 kB/s | 25 kB 00:00 2024-02-18T00:22:02.719 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:02.719 INFO:teuthology.orchestra.run.smithi059.stdout:Total 190 kB/s | 25 kB 00:00 2024-02-18T00:22:02.720 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:22:02.723 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:22:02.724 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:22:02.726 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:22:02.729 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:22:02.783 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:22:02.865 INFO:teuthology.orchestra.run.smithi137.stdout:Package libcephfs2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:02.911 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:02.913 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:22:02.913 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:03.000 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:03.026 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install libcephfs-devel 2024-02-18T00:22:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:03.664 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:22:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:03.664 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:03.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install librados2 2024-02-18T00:22:03.790 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:17 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:04.589 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:36 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:05.365 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:05.365 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:05.365 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-02-18T00:22:05.365 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:05.365 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:22:05.366 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 25 k 2024-02-18T00:22:05.366 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:05.366 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:22:05.366 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:05.366 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-02-18T00:22:05.366 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:05.366 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 25 k 2024-02-18T00:22:05.367 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 107 k 2024-02-18T00:22:05.367 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:22:05.785 INFO:teuthology.orchestra.run.smithi137.stdout:libcephfs-devel-16.2.14-548.g81bd20d6.el8.x86_6 61 kB/s | 25 kB 00:00 2024-02-18T00:22:05.785 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:05.786 INFO:teuthology.orchestra.run.smithi137.stdout:Total 61 kB/s | 25 kB 00:00 2024-02-18T00:22:05.786 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:22:05.790 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:22:05.790 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:22:05.792 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:22:05.795 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:22:05.842 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:22:06.061 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:06.208 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:06.259 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:06.260 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:22:06.260 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:06.316 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:06.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install librbd1 2024-02-18T00:22:06.664 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:06.664 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:06.664 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:22:06.664 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:06.665 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:06.665 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:06.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librados2 2024-02-18T00:22:07.161 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:39 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:07.542 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:21 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:08.787 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:08.836 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:08.838 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:22:08.838 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:08.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-rados 2024-02-18T00:22:09.050 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:09.097 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:09.099 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:22:09.099 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:09.207 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install librbd1 2024-02-18T00:22:09.721 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:41 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:09.962 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:23 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:11.342 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:11.392 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:11.393 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:22:11.393 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:11.471 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:11.494 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-rgw 2024-02-18T00:22:11.518 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:11.519 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:22:11.519 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:11.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rados 2024-02-18T00:22:12.282 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:44 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:12.385 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:26 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:13.881 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rados-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:13.887 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:22:13.929 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:13.934 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:13.935 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:22:13.936 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:14.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-cephfs 2024-02-18T00:22:14.047 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rgw 2024-02-18T00:22:14.798 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:28 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:14.833 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:46 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:16.286 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rgw-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:16.333 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:16.334 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:22:16.334 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:16.438 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:16.454 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-cephfs 2024-02-18T00:22:16.488 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:16.489 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:22:16.489 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:16.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install python3-rbd 2024-02-18T00:22:17.209 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:31 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:17.373 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:49 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:18.733 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-cephfs-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:18.780 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:18.781 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:22:18.781 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:18.897 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install python3-rbd 2024-02-18T00:22:18.990 INFO:teuthology.orchestra.run.smithi059.stdout:Package python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:19.041 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:19.042 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2024-02-18T00:22:19.042 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:19.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-fuse 2024-02-18T00:22:19.647 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:33 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:19.929 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:51 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:21.133 INFO:teuthology.orchestra.run.smithi137.stdout:Package python3-rbd-2:16.2.14-548.g81bd20d6.el8.x86_64 is already installed. 2024-02-18T00:22:21.180 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:21.182 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2024-02-18T00:22:21.182 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:21.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-fuse 2024-02-18T00:22:21.588 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:21.589 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:21.589 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-02-18T00:22:21.589 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:21.589 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:22:21.589 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 88 k 2024-02-18T00:22:21.590 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:21.590 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:22:21.590 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:21.590 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-02-18T00:22:21.590 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:21.590 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 88 k 2024-02-18T00:22:21.591 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 240 k 2024-02-18T00:22:21.591 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:22:22.025 INFO:teuthology.orchestra.run.smithi059.stdout:rbd-fuse-16.2.14-548.g81bd20d6.el8.x86_64.rpm 204 kB/s | 88 kB 00:00 2024-02-18T00:22:22.025 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:22.025 INFO:teuthology.orchestra.run.smithi059.stdout:Total 203 kB/s | 88 kB 00:00 2024-02-18T00:22:22.026 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:22:22.037 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:22:22.037 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:22:22.092 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:22:22.095 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:22:22.211 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:22:22.214 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:36 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:22.397 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:22.701 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:23.155 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:23.155 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:23.155 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:22:23.155 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:23.156 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:23.156 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:23.414 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-mirror 2024-02-18T00:22:23.775 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 88 k 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 88 k 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 240 k 2024-02-18T00:22:23.776 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:22:24.166 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-fuse-16.2.14-548.g81bd20d6.el8.x86_64.rpm 227 kB/s | 88 kB 00:00 2024-02-18T00:22:24.167 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:24.167 INFO:teuthology.orchestra.run.smithi137.stdout:Total 226 kB/s | 88 kB 00:00 2024-02-18T00:22:24.168 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:22:24.177 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:22:24.177 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:22:24.199 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:56 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:24.226 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:22:24.229 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:22:24.333 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:22:24.508 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:24.801 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:25.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:25.168 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:25.168 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:22:25.168 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:25.168 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:25.168 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:25.399 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-mirror 2024-02-18T00:22:25.877 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repo Size 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:25.878 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-02-18T00:22:25.879 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:25.879 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 3.7 M 2024-02-18T00:22:25.879 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 15 M 2024-02-18T00:22:25.879 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:22:26.158 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:40 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:27.723 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repo Size 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 3.7 M 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:22:27.724 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:27.725 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-02-18T00:22:27.725 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:27.725 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 3.7 M 2024-02-18T00:22:27.725 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 15 M 2024-02-18T00:22:27.725 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:22:27.869 INFO:teuthology.orchestra.run.smithi059.stdout:rbd-mirror-16.2.14-548.g81bd20d6.el8.x86_64.rpm 1.8 MB/s | 3.7 MB 00:01 2024-02-18T00:22:27.870 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:27.870 INFO:teuthology.orchestra.run.smithi059.stdout:Total 1.8 MB/s | 3.7 MB 00:01 2024-02-18T00:22:27.871 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:22:27.885 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:22:27.885 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:22:27.958 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:22:27.961 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:22:28.423 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:22:28.665 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:29.102 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:29.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:29.517 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:29.517 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:22:29.517 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:29.517 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:29.518 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:29.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install rbd-nbd 2024-02-18T00:22:29.761 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-mirror-16.2.14-548.g81bd20d6.el8.x86_64.rpm 1.8 MB/s | 3.7 MB 00:02 2024-02-18T00:22:29.761 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:29.762 INFO:teuthology.orchestra.run.smithi137.stdout:Total 1.8 MB/s | 3.7 MB 00:02 2024-02-18T00:22:29.762 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:22:29.775 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:22:29.775 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:22:29.844 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:22:29.847 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:22:30.283 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:22:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:30.542 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:03:02 ago on Sun 18 Feb 2024 12:19:28 AM UTC. 2024-02-18T00:22:30.919 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:31.282 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:31.282 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:31.282 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:22:31.282 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:31.283 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:31.283 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:31.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install rbd-nbd 2024-02-18T00:22:32.197 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 175 k 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-02-18T00:22:32.198 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-02-18T00:22:32.199 INFO:teuthology.orchestra.run.smithi059.stdout:Install 1 Package 2024-02-18T00:22:32.199 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:32.199 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 175 k 2024-02-18T00:22:32.199 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 523 k 2024-02-18T00:22:32.199 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-02-18T00:22:32.291 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:46 ago on Sun 18 Feb 2024 12:19:46 AM UTC. 2024-02-18T00:22:32.705 INFO:teuthology.orchestra.run.smithi059.stdout:rbd-nbd-16.2.14-548.g81bd20d6.el8.x86_64.rpm 346 kB/s | 175 kB 00:00 2024-02-18T00:22:32.705 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:32.706 INFO:teuthology.orchestra.run.smithi059.stdout:Total 345 kB/s | 175 kB 00:00 2024-02-18T00:22:32.706 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-02-18T00:22:32.718 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-02-18T00:22:32.718 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-02-18T00:22:32.774 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-02-18T00:22:32.777 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-02-18T00:22:32.895 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-02-18T00:22:33.112 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:33.631 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:33.845 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-02-18T00:22:33.846 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:33.846 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-02-18T00:22:33.846 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:33.846 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:16.2.14-548.g81bd20d6.el8 ceph 175 k 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout:Install 1 Package 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 175 k 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 523 k 2024-02-18T00:22:33.847 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-02-18T00:22:34.010 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:34.010 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:34.010 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-02-18T00:22:34.011 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:34.011 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:22:34.011 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-02-18T00:22:34.047 INFO:teuthology.orchestra.run.smithi137.stdout:rbd-nbd-16.2.14-548.g81bd20d6.el8.x86_64.rpm 876 kB/s | 175 kB 00:00 2024-02-18T00:22:34.048 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-02-18T00:22:34.048 INFO:teuthology.orchestra.run.smithi137.stdout:Total 870 kB/s | 175 kB 00:00 2024-02-18T00:22:34.048 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-02-18T00:22:34.059 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-02-18T00:22:34.059 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-02-18T00:22:34.108 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-02-18T00:22:34.111 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-02-18T00:22:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-02-18T00:22:34.251 DEBUG:teuthology.parallel:result is None 2024-02-18T00:22:34.423 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:34.713 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:35.088 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 1/1 2024-02-18T00:22:35.088 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:35.088 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-02-18T00:22:35.088 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:16.2.14-548.g81bd20d6.el8.x86_64 2024-02-18T00:22:35.088 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:22:35.088 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-02-18T00:22:35.331 DEBUG:teuthology.parallel:result is None 2024-02-18T00:22:35.332 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:22:35.564 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-18T00:22:35.592 INFO:teuthology.orchestra.run.smithi059.stdout:16.2.14-548.g81bd20d6.el8 2024-02-18T00:22:35.593 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-548.g81bd20d6.el8 2024-02-18T00:22:35.593 INFO:teuthology.task.install:The correct ceph version 16.2.14-548.g81bd20d6 is installed. 2024-02-18T00:22:35.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:22:35.739 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-18T00:22:35.762 INFO:teuthology.orchestra.run.smithi137.stdout:16.2.14-548.g81bd20d6.el8 2024-02-18T00:22:35.762 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-548.g81bd20d6.el8 2024-02-18T00:22:35.762 INFO:teuthology.task.install:The correct ceph version 16.2.14-548.g81bd20d6 is installed. 2024-02-18T00:22:35.764 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-18T00:22:35.764 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:22:35.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T00:22:35.798 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:22:35.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T00:22:35.831 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-18T00:22:35.831 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:22:35.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T00:22:35.870 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T00:22:35.940 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:22:35.940 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T00:22:35.967 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T00:22:36.033 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-18T00:22:36.033 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:22:36.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T00:22:36.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T00:22:36.136 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:22:36.136 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T00:22:36.163 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T00:22:36.230 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-18T00:22:36.230 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:22:36.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T00:22:36.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T00:22:36.331 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:22:36.331 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T00:22:36.357 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T00:22:36.424 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-18T00:22:36.497 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mgrc': 20, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'crush_tunables': 'jewel', 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', 'flag\\(s\\) set', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(PG_', '\\(OSD_', {'mons down': None}, 'reached quota', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)'], 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1', 'cephadm_mode': 'root'} 2024-02-18T00:22:36.497 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:22:36.498 INFO:tasks.cephadm:Cluster fsid is d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:22:36.498 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-18T00:22:36.498 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.59', 'mon.c': '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]', 'mon.b': '172.21.15.137'} 2024-02-18T00:22:36.498 INFO:tasks.cephadm:First mon is mon.a on smithi059 2024-02-18T00:22:36.498 INFO:tasks.cephadm:First mgr is y 2024-02-18T00:22:36.499 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-18T00:22:36.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2024-02-18T00:22:36.531 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-02-18T00:22:36.559 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph-ci.git ref 81bd20d634209c7cb82c18be12b4b5a05643ebf1)... 2024-02-18T00:22:36.559 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/81bd20d634209c7cb82c18be12b4b5a05643ebf1/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-18T00:22:36.784 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 18 00:22 /home/ubuntu/cephtest/cephadm 2024-02-18T00:22:36.784 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent https://raw.githubusercontent.com/ceph/ceph-ci/81bd20d634209c7cb82c18be12b4b5a05643ebf1/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-18T00:22:36.983 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-rw-r--. 1 ubuntu ubuntu 355729 Feb 18 00:22 /home/ubuntu/cephtest/cephadm 2024-02-18T00:22:36.984 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-18T00:22:37.007 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-18T00:22:37.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2024-02-18T00:22:37.080 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-02-18T00:22:37.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2024-02-18T00:22:37.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-02-18T00:22:37.177 INFO:tasks.cephadm:Writing seed config... 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [client] debug objecter = 20 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [client] debug rados = 20 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-02-18T00:22:37.178 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] debug mgrc = 20 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-02-18T00:22:37.179 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-02-18T00:22:37.180 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-02-18T00:22:37.180 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-02-18T00:22:37.180 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-18T00:22:37.180 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-02-18T00:22:37.180 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:22:37.180 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-18T00:22:37.211 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d129b4f8-cdf3-11ee-95ba-87774f69a715 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mgrc = 20 debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 mon warn on pool no app = False paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 debug objecter = 20 debug rados = 20 2024-02-18T00:22:37.211 DEBUG:teuthology.orchestra.run.smithi059:mon.a> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.a.service 2024-02-18T00:22:37.253 DEBUG:teuthology.orchestra.run.smithi059:mgr.y> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.y.service 2024-02-18T00:22:37.283 INFO:journalctl@ceph.mon.a.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:22:37.284 INFO:tasks.cephadm:Bootstrapping... 2024-02-18T00:22:37.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 -v bootstrap --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.59 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-18T00:22:37.313 INFO:journalctl@ceph.mgr.y.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:22:37.570 INFO:teuthology.orchestra.run.smithi059.stderr:-------------------------------------------------------------------------------- 2024-02-18T00:22:37.570 INFO:teuthology.orchestra.run.smithi059.stderr:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1', '-v', 'bootstrap', '--fsid', 'd129b4f8-cdf3-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.59', '--skip-admin-label'] 2024-02-18T00:22:38.589 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: 4.9.3-dev 2024-02-18T00:22:38.599 INFO:teuthology.orchestra.run.smithi059.stderr:Verifying podman|docker is present... 2024-02-18T00:22:38.623 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: 4.9.3-dev 2024-02-18T00:22:38.633 INFO:teuthology.orchestra.run.smithi059.stderr:Verifying lvm2 is present... 2024-02-18T00:22:38.633 INFO:teuthology.orchestra.run.smithi059.stderr:Verifying time synchronization is in place... 2024-02-18T00:22:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-18T00:22:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: inactive 2024-02-18T00:22:38.654 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: enabled 2024-02-18T00:22:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: active 2024-02-18T00:22:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:Unit chronyd.service is enabled and running 2024-02-18T00:22:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:Repeating the final host check... 2024-02-18T00:22:38.683 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: 4.9.3-dev 2024-02-18T00:22:38.696 INFO:teuthology.orchestra.run.smithi059.stderr:podman (/bin/podman) version 4.9.3 is present 2024-02-18T00:22:38.696 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl is present 2024-02-18T00:22:38.696 INFO:teuthology.orchestra.run.smithi059.stderr:lvcreate is present 2024-02-18T00:22:38.702 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-18T00:22:38.708 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: inactive 2024-02-18T00:22:38.714 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: enabled 2024-02-18T00:22:38.720 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: active 2024-02-18T00:22:38.721 INFO:teuthology.orchestra.run.smithi059.stderr:Unit chronyd.service is enabled and running 2024-02-18T00:22:38.721 INFO:teuthology.orchestra.run.smithi059.stderr:Host looks OK 2024-02-18T00:22:38.721 INFO:teuthology.orchestra.run.smithi059.stderr:Cluster fsid: d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:22:38.721 INFO:teuthology.orchestra.run.smithi059.stderr:Acquiring lock 139946321947112 on /run/cephadm/d129b4f8-cdf3-11ee-95ba-87774f69a715.lock 2024-02-18T00:22:38.721 INFO:teuthology.orchestra.run.smithi059.stderr:Lock 139946321947112 acquired on /run/cephadm/d129b4f8-cdf3-11ee-95ba-87774f69a715.lock 2024-02-18T00:22:38.722 INFO:teuthology.orchestra.run.smithi059.stderr:Verifying IP 172.21.15.59 port 3300 ... 2024-02-18T00:22:38.722 INFO:teuthology.orchestra.run.smithi059.stderr:Verifying IP 172.21.15.59 port 6789 ... 2024-02-18T00:22:38.722 INFO:teuthology.orchestra.run.smithi059.stderr:Base mon IP(s) is [172.21.15.59:3300, 172.21.15.59:6789], mon addrv is [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-02-18T00:22:38.725 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.59 metric 100 2024-02-18T00:22:38.725 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.59 metric 100 2024-02-18T00:22:38.728 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-02-18T00:22:38.728 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-02-18T00:22:38.728 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-02-18T00:22:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-18T00:22:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-18T00:22:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-18T00:22:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-18T00:22:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2024-02-18T00:22:38.731 INFO:teuthology.orchestra.run.smithi059.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-18T00:22:38.732 INFO:teuthology.orchestra.run.smithi059.stderr:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-02-18T00:22:38.732 INFO:teuthology.orchestra.run.smithi059.stderr:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-02-18T00:22:38.732 INFO:teuthology.orchestra.run.smithi059.stderr:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-18T00:22:38.733 INFO:teuthology.orchestra.run.smithi059.stderr:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-18T00:22:38.734 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1... 2024-02-18T00:22:38.828 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1... 2024-02-18T00:22:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: Getting image source signatures 2024-02-18T00:22:39.019 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-18T00:22:39.020 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: Copying blob sha256:ec93555fb25f6296fcd66f232359fb768587e082c45b15dd379853eb0f28a963 2024-02-18T00:23:00.875 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: Copying config sha256:823b921721f579487e02cd293080702537bcba9f4f96b9041a06d7563de513aa 2024-02-18T00:23:01.067 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: Writing manifest to image destination 2024-02-18T00:23:01.263 INFO:teuthology.orchestra.run.smithi059.stderr:/bin/podman: 823b921721f579487e02cd293080702537bcba9f4f96b9041a06d7563de513aa 2024-02-18T00:23:01.937 INFO:teuthology.orchestra.run.smithi059.stderr:ceph: ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable) 2024-02-18T00:23:02.776 INFO:teuthology.orchestra.run.smithi059.stderr:Ceph version: ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable) 2024-02-18T00:23:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:Extracting ceph user uid/gid from container image... 2024-02-18T00:23:03.287 INFO:teuthology.orchestra.run.smithi059.stderr:stat: 167 167 2024-02-18T00:23:03.975 INFO:teuthology.orchestra.run.smithi059.stderr:Creating initial keys... 2024-02-18T00:23:04.876 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-authtool: AQDoTdFlmZqKLhAAROZ87Kznl3CiTajhXK1JQA== 2024-02-18T00:23:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-authtool: AQDpTdFlgNgfORAAxQKaNx8pEP3uCgWi8g4MuQ== 2024-02-18T00:23:07.259 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-authtool: AQDrTdFldgCkCBAAL879ZpcAXS68MFTExia9Ug== 2024-02-18T00:23:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:Creating initial monmap... 2024-02-18T00:23:08.362 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-18T00:23:08.362 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:08.362 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-18T00:23:08.960 INFO:teuthology.orchestra.run.smithi059.stderr:monmaptool for a [v2:172.21.15.59:3300,v1:172.21.15.59:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-18T00:23:08.960 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/monmaptool: set fsid to d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:08.960 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-18T00:23:08.961 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:23:08.961 INFO:teuthology.orchestra.run.smithi059.stderr:Creating mon... 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.484+0000 7f65c6cd6880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.485+0000 7f65c6cd6880 1 imported monmap: 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: last_changed 2024-02-18T00:23:08.218775+0000 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: created 2024-02-18T00:23:08.218775+0000 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.485+0000 7f65c6cd6880 0 /usr/bin/ceph-mon: set fsid to d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: DB SUMMARY 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.create_if_missing: 1 2024-02-18T00:23:09.557 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.env: 0x55d1a69a4080 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.fs: Posix File System 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.info_log: 0x55d1a72d3a00 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.statistics: (nil) 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.use_fsync: 0 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T00:23:09.558 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.db_log_dir: 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T00:23:09.559 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.write_buffer_manager: 0x55d1a72d9920 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T00:23:09.560 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.unordered_write: 0 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.row_cache: None 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.wal_filter: None 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T00:23:09.561 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T00:23:09.562 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_open_files: -1 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Compression algorithms supported: 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kZSTD supported: 0 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T00:23:09.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.merge_operator: 2024-02-18T00:23:09.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compaction_filter: None 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d1a71e9d18) 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-18T00:23:09.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_cache: 0x55d1a721ed10 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-18T00:23:09.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T00:23:09.567 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compression: NoCompression 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.num_levels: 7 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T00:23:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.486+0000 7f65c6cd6880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T00:23:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T00:23:09.570 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T00:23:09.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.ttl: 2592000 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T00:23:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.487+0000 7f65c6cd6880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.488+0000 7f65c6cd6880 4 rocksdb: DB pointer 0x55d1a72e7800 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.488+0000 7f65afb49700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.488+0000 7f65afb49700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T00:23:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T00:23:09.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:23:09.577 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: 2024-02-18T00:23:09.577 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.489+0000 7f65c6cd6880 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-18T00:23:09.577 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.489+0000 7f65c6cd6880 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-18T00:23:09.577 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph-mon: debug 2024-02-18T00:23:09.489+0000 7f65c6cd6880 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-02-18T00:23:10.246 INFO:teuthology.orchestra.run.smithi059.stderr:create mon.a on 2024-02-18T00:23:10.262 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-18T00:23:10.367 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-18T00:23:10.464 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715.target. 2024-02-18T00:23:10.464 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715.target. 2024-02-18T00:23:10.684 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Failed to reset failed state of unit ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.a.service: Unit ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.a.service not loaded. 2024-02-18T00:23:10.691 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Created symlink /etc/systemd/system/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715.target.wants/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.a.service → /etc/systemd/system/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@.service. 2024-02-18T00:23:10.823 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:10 smithi059 systemd[1]: Starting Ceph mon.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:23:11.388 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: disabled 2024-02-18T00:23:11.395 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: inactive 2024-02-18T00:23:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:firewalld.service is not enabled 2024-02-18T00:23:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:Not possible to enable service . firewalld.service is not available 2024-02-18T00:23:11.397 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for mon to start... 2024-02-18T00:23:11.397 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for mon... 2024-02-18T00:23:11.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:11 smithi059 bash[37755]: 721d8695a1c53d13f78a66daa9b24d02d429a01e88ae7f6ad9c2e1454c6f2958 2024-02-18T00:23:11.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:11 smithi059 systemd[1]: Started Ceph mon.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:23:11.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: cluster 2024-02-18T00:23:11.391022+0000 mon.a (mon.0) 0 : [INF] mkfs d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:11.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: cluster 2024-02-18T00:23:11.359898+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T00:23:11.950 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.930+0000 7f3313142700 1 Processor -- start 2024-02-18T00:23:11.950 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.930+0000 7f3313142700 1 -- start start 2024-02-18T00:23:11.950 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.930+0000 7f3313142700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c0fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:11.950 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.930+0000 7f3313142700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0fb8f0 con 0x7f330c0faf50 2024-02-18T00:23:11.950 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.930+0000 7f3310ede700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c0fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.930+0000 7f3310ede700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c0fb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57826/0 (socket says 172.21.15.59:57826) 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.930+0000 7f3310ede700 1 -- 172.21.15.59:0/3408729685 learned_addr learned my addr 172.21.15.59:0/3408729685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3310ede700 1 -- 172.21.15.59:0/3408729685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0fba30 con 0x7f330c0faf50 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3310ede700 1 --2- 172.21.15.59:0/3408729685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c0fb370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f3300009730 tx=0x7f33000090f0).ready entity=mon.0 client_cookie=80053cb1af94aeab server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f330b7fe700 1 -- 172.21.15.59:0/3408729685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330000b580 con 0x7f330c0faf50 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f330b7fe700 1 -- 172.21.15.59:0/3408729685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000b6e0 con 0x7f330c0faf50 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f330b7fe700 1 -- 172.21.15.59:0/3408729685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330000b990 con 0x7f330c0faf50 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3313142700 1 -- 172.21.15.59:0/3408729685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 msgr2=0x7f330c0fb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3313142700 1 --2- 172.21.15.59:0/3408729685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c0fb370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f3300009730 tx=0x7f33000090f0).stop 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3313142700 1 -- 172.21.15.59:0/3408729685 shutdown_connections 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3313142700 1 --2- 172.21.15.59:0/3408729685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c0fb370 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3313142700 1 -- 172.21.15.59:0/3408729685 >> 172.21.15.59:0/3408729685 conn(0x7f330c0f6670 msgr2=0x7f330c0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:11.951 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3313142700 1 -- 172.21.15.59:0/3408729685 shutdown_connections 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.931+0000 7f3313142700 1 -- 172.21.15.59:0/3408729685 wait complete. 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 librados: starting msgr at 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 librados: starting objecter 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 Processor -- start 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 -- start start 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 librados: setting wanted keys 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 librados: calling monclient init 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c1093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3313142700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c109890 con 0x7f330c0faf50 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3310ede700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c1093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3310ede700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c1093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57834/0 (socket says 172.21.15.59:57834) 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.932+0000 7f3310ede700 1 -- 172.21.15.59:0/3529138457 learned_addr learned my addr 172.21.15.59:0/3529138457 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3310ede700 1 -- 172.21.15.59:0/3529138457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300009370 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3310ede700 1 --2- 172.21.15.59:0/3529138457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c1093a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f3300009130 tx=0x7f3300017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 10 client.?.objecter ms_handle_connect 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 1 -- 172.21.15.59:0/3529138457 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3300011cd0 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 1 -- 172.21.15.59:0/3529138457 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3300011e30 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c109a90 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 1 -- 172.21.15.59:0/3529138457 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f330001d570 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3313142700 10 client.4098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c109f20 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3313142700 1 librados: init done 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 1 -- 172.21.15.59:0/3529138457 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f330001da00 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 1 -- 172.21.15.59:0/3529138457 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f3300026710 con 0x7f330c0faf50 2024-02-18T00:23:11.953 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 10 client.4098.objecter ms_dispatch 0x7f330c001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:11.954 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 3 client.4098.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:11.954 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 3 client.4098.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:11.954 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.933+0000 7f3309ffb700 20 client.4098.objecter dump_active .. 0 homeless 2024-02-18T00:23:11.954 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.934+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c046ce0 con 0x7f330c0faf50 2024-02-18T00:23:11.954 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:11.936+0000 7f3309ffb700 1 -- 172.21.15.59:0/3529138457 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f3300021030 con 0x7f330c0faf50 2024-02-18T00:23:12.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.006+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f330c1029e0 con 0x7f330c0faf50 2024-02-18T00:23:12.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.006+0000 7f3309ffb700 1 -- 172.21.15.59:0/3529138457 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f33000269f0 con 0x7f330c0faf50 2024-02-18T00:23:12.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: cluster: 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: id: d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: services: 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.62879s) 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: data: 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-18T00:23:12.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: pgs: 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.006+0000 7f3313142700 10 librados: watch_flush enter 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.006+0000 7f3313142700 10 librados: watch_flush exit 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.006+0000 7f3313142700 20 client.4098.objecter shutdown clearing up homeless session... 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.006+0000 7f3313142700 10 client.4098.objecter successfully canceled tick 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 msgr2=0x7f330c1093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 --2- 172.21.15.59:0/3529138457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c1093a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f3300009130 tx=0x7f3300017000).stop 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 shutdown_connections 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 --2- 172.21.15.59:0/3529138457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330c0faf50 0x7f330c1093a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 >> 172.21.15.59:0/3529138457 conn(0x7f330c0f6670 msgr2=0x7f330c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 shutdown_connections 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 -- 172.21.15.59:0/3529138457 wait complete. 2024-02-18T00:23:12.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:12.007+0000 7f3313142700 1 librados: shutdown 2024-02-18T00:23:12.600 INFO:teuthology.orchestra.run.smithi059.stderr:mon is available 2024-02-18T00:23:12.600 INFO:teuthology.orchestra.run.smithi059.stderr:Assimilating anything we can from ceph.conf... 2024-02-18T00:23:12.602 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: cluster 2024-02-18T00:23:11.378589+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T00:23:12.602 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: cluster 2024-02-18T00:23:11.378803+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0]} removed_ranks: {} 2024-02-18T00:23:12.602 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: cluster 2024-02-18T00:23:11.394858+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-02-18T00:23:12.602 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: cluster 2024-02-18T00:23:11.397022+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-18T00:23:12.603 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: cluster 2024-02-18T00:23:11.397549+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-18T00:23:12.603 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: audit 2024-02-18T00:23:12.007336+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.59:0/3529138457' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-18T00:23:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.089+0000 7f781617b700 1 Processor -- start 2024-02-18T00:23:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.089+0000 7f781617b700 1 -- start start 2024-02-18T00:23:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f781617b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f78100fec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f781617b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78100ff190 con 0x7f7810070cc0 2024-02-18T00:23:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f780f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f78100fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f780f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f78100fec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57848/0 (socket says 172.21.15.59:57848) 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f780f7fe700 1 -- 172.21.15.59:0/251295842 learned_addr learned my addr 172.21.15.59:0/251295842 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f780f7fe700 1 -- 172.21.15.59:0/251295842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100ff2d0 con 0x7f7810070cc0 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f780f7fe700 1 --2- 172.21.15.59:0/251295842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f78100fec10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f7800009730 tx=0x7f78000090f0).ready entity=mon.0 client_cookie=dafa6883efe13343 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.090+0000 7f780e7fc700 1 -- 172.21.15.59:0/251295842 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780000b580 con 0x7f7810070cc0 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f780e7fc700 1 -- 172.21.15.59:0/251295842 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780000b6e0 con 0x7f7810070cc0 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f780e7fc700 1 -- 172.21.15.59:0/251295842 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780000b990 con 0x7f7810070cc0 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f781617b700 1 -- 172.21.15.59:0/251295842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 msgr2=0x7f78100fec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f781617b700 1 --2- 172.21.15.59:0/251295842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f78100fec10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f7800009730 tx=0x7f78000090f0).stop 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f781617b700 1 -- 172.21.15.59:0/251295842 shutdown_connections 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f781617b700 1 --2- 172.21.15.59:0/251295842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f78100fec10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f781617b700 1 -- 172.21.15.59:0/251295842 >> 172.21.15.59:0/251295842 conn(0x7f78100f6670 msgr2=0x7f78100f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f781617b700 1 -- 172.21.15.59:0/251295842 shutdown_connections 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.091+0000 7f781617b700 1 -- 172.21.15.59:0/251295842 wait complete. 2024-02-18T00:23:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 librados: starting msgr at 2024-02-18T00:23:13.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 librados: starting objecter 2024-02-18T00:23:13.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 Processor -- start 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 -- start start 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 librados: setting wanted keys 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 librados: calling monclient init 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f781010a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f781617b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781010a920 con 0x7f7810070cc0 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f780f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f781010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f780f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f781010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57860/0 (socket says 172.21.15.59:57860) 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f780f7fe700 1 -- 172.21.15.59:0/2324023243 learned_addr learned my addr 172.21.15.59:0/2324023243 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.092+0000 7f780f7fe700 1 -- 172.21.15.59:0/2324023243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7800009370 con 0x7f7810070cc0 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780f7fe700 1 --2- 172.21.15.59:0/2324023243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f781010a430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f7800009130 tx=0x7f7800017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 10 client.?.objecter ms_handle_connect 0x7f7810070cc0 2024-02-18T00:23:13.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7800011cd0 con 0x7f7810070cc0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7800011e30 con 0x7f7810070cc0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781010ab20 con 0x7f7810070cc0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780001d570 con 0x7f7810070cc0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f781617b700 10 client.4100.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781010afb0 con 0x7f7810070cc0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f781617b700 1 librados: init done 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f780001da00 con 0x7f7810070cc0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f7800026710 con 0x7f7810070cc0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 10 client.4100.objecter ms_dispatch 0x7f7810001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 3 client.4100.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 3 client.4100.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.093+0000 7f780cff9700 20 client.4100.objecter dump_active .. 0 homeless 2024-02-18T00:23:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.094+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7810046ce0 con 0x7f7810070cc0 2024-02-18T00:23:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.097+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f7800021030 con 0x7f7810070cc0 2024-02-18T00:23:13.165 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.163+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f78100ff640 con 0x7f7810070cc0 2024-02-18T00:23:13.171 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.166+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f780001de90 con 0x7f7810070cc0 2024-02-18T00:23:13.171 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.166+0000 7f780cff9700 1 -- 172.21.15.59:0/2324023243 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+363 (secure 0 0 0) 0x7f78000269f0 con 0x7f7810070cc0 2024-02-18T00:23:13.171 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.171 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: [global] 2024-02-18T00:23:13.171 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: fsid = d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:13.171 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-18T00:23:13.171 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: [mgr] 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: [osd] 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 10 librados: watch_flush enter 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 10 librados: watch_flush exit 2024-02-18T00:23:13.172 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 20 client.4100.objecter shutdown clearing up homeless session... 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 10 client.4100.objecter successfully canceled tick 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 msgr2=0x7f781010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 1 --2- 172.21.15.59:0/2324023243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f781010a430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f7800009130 tx=0x7f7800017000).stop 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 shutdown_connections 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 1 --2- 172.21.15.59:0/2324023243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810070cc0 0x7f781010a430 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.167+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 >> 172.21.15.59:0/2324023243 conn(0x7f78100f6670 msgr2=0x7f78100f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.168+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 shutdown_connections 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.168+0000 7f781617b700 1 -- 172.21.15.59:0/2324023243 wait complete. 2024-02-18T00:23:13.173 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:13.168+0000 7f781617b700 1 librados: shutdown 2024-02-18T00:23:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: audit 2024-02-18T00:23:13.164917+0000 mon.a (mon.0) 8 : audit [INF] from='client.? 172.21.15.59:0/2324023243' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-18T00:23:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: audit 2024-02-18T00:23:13.167841+0000 mon.a (mon.0) 9 : audit [INF] from='client.? 172.21.15.59:0/2324023243' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-18T00:23:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:Generating new minimal ceph.conf... 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.493+0000 7f723cee1700 1 Processor -- start 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.494+0000 7f723cee1700 1 -- start start 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.494+0000 7f723cee1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f72380fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.494+0000 7f723cee1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380fcf60 con 0x7f72380fc5c0 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.494+0000 7f723659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f72380fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.494+0000 7f723659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f72380fc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57874/0 (socket says 172.21.15.59:57874) 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.494+0000 7f723659c700 1 -- 172.21.15.59:0/1359176000 learned_addr learned my addr 172.21.15.59:0/1359176000 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.494+0000 7f723659c700 1 -- 172.21.15.59:0/1359176000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380fd0a0 con 0x7f72380fc5c0 2024-02-18T00:23:14.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723659c700 1 --2- 172.21.15.59:0/1359176000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f72380fc9e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f7228009730 tx=0x7f72280090f0).ready entity=mon.0 client_cookie=61aeb2f32213bfaf server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723559a700 1 -- 172.21.15.59:0/1359176000 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f722800b580 con 0x7f72380fc5c0 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723559a700 1 -- 172.21.15.59:0/1359176000 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f722800bb90 con 0x7f72380fc5c0 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723559a700 1 -- 172.21.15.59:0/1359176000 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7228011c30 con 0x7f72380fc5c0 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723cee1700 1 -- 172.21.15.59:0/1359176000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 msgr2=0x7f72380fc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723cee1700 1 --2- 172.21.15.59:0/1359176000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f72380fc9e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f7228009730 tx=0x7f72280090f0).stop 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723cee1700 1 -- 172.21.15.59:0/1359176000 shutdown_connections 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723cee1700 1 --2- 172.21.15.59:0/1359176000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f72380fc9e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.495+0000 7f723cee1700 1 -- 172.21.15.59:0/1359176000 >> 172.21.15.59:0/1359176000 conn(0x7f72380f5e30 msgr2=0x7f72380f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 -- 172.21.15.59:0/1359176000 shutdown_connections 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 -- 172.21.15.59:0/1359176000 wait complete. 2024-02-18T00:23:14.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 librados: starting msgr at 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 librados: starting objecter 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 Processor -- start 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 -- start start 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 librados: setting wanted keys 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.496+0000 7f723cee1700 1 librados: calling monclient init 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723cee1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f723810cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723cee1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723810d390 con 0x7f72380fc5c0 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f723810cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f723810cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57886/0 (socket says 172.21.15.59:57886) 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723659c700 1 -- 172.21.15.59:0/4007471963 learned_addr learned my addr 172.21.15.59:0/4007471963 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723659c700 1 -- 172.21.15.59:0/4007471963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7228009370 con 0x7f72380fc5c0 2024-02-18T00:23:14.502 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723659c700 1 --2- 172.21.15.59:0/4007471963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f723810cea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f72280105d0 tx=0x7f72280090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f72277fe700 10 client.?.objecter ms_handle_connect 0x7f72380fc5c0 2024-02-18T00:23:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f72277fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f72277fe700 1 -- 172.21.15.59:0/4007471963 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7228011690 con 0x7f72380fc5c0 2024-02-18T00:23:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f72277fe700 1 -- 172.21.15.59:0/4007471963 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7228011a70 con 0x7f72380fc5c0 2024-02-18T00:23:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723810d590 con 0x7f72380fc5c0 2024-02-18T00:23:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.497+0000 7f72277fe700 1 -- 172.21.15.59:0/4007471963 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f722801ab80 con 0x7f72380fc5c0 2024-02-18T00:23:14.503 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f723cee1700 10 client.4102.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723810da20 con 0x7f72380fc5c0 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f723cee1700 1 librados: init done 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f72277fe700 1 -- 172.21.15.59:0/4007471963 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f72280236b0 con 0x7f72380fc5c0 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f72277fe700 1 -- 172.21.15.59:0/4007471963 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f7228022720 con 0x7f72380fc5c0 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f72277fe700 10 client.4102.objecter ms_dispatch 0x7f7238001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f72277fe700 3 client.4102.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f72277fe700 3 client.4102.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.498+0000 7f72277fe700 20 client.4102.objecter dump_active .. 0 homeless 2024-02-18T00:23:14.504 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.499+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7238046ce0 con 0x7f72380fc5c0 2024-02-18T00:23:14.505 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.502+0000 7f72277fe700 1 -- 172.21.15.59:0/4007471963 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f7228028060 con 0x7f72380fc5c0 2024-02-18T00:23:14.571 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.569+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7238106b80 con 0x7f72380fc5c0 2024-02-18T00:23:14.572 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.569+0000 7f72277fe700 1 -- 172.21.15.59:0/4007471963 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f722801e070 con 0x7f72380fc5c0 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 10 librados: watch_flush enter 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 10 librados: watch_flush exit 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 20 client.4102.objecter shutdown clearing up homeless session... 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 10 client.4102.objecter successfully canceled tick 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 msgr2=0x7f723810cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 1 --2- 172.21.15.59:0/4007471963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f723810cea0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f72280105d0 tx=0x7f72280090f0).stop 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 shutdown_connections 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 1 --2- 172.21.15.59:0/4007471963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72380fc5c0 0x7f723810cea0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.570+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 >> 172.21.15.59:0/4007471963 conn(0x7f72380f5e30 msgr2=0x7f72380f7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.571+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 shutdown_connections 2024-02-18T00:23:14.574 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.571+0000 7f723cee1700 1 -- 172.21.15.59:0/4007471963 wait complete. 2024-02-18T00:23:14.575 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:14.571+0000 7f723cee1700 1 librados: shutdown 2024-02-18T00:23:14.918 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: audit 2024-02-18T00:23:14.570965+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.59:0/4007471963' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:23:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:Restarting the monitor... 2024-02-18T00:23:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:15 smithi059 systemd[1]: Stopping Ceph mon.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:23:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:15 smithi059 bash[38093]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon.a" found: no such container 2024-02-18T00:23:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: debug 2024-02-18T00:23:16.516+0000 7f5f0ff97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T00:23:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[37770]: debug 2024-02-18T00:23:16.516+0000 7f5f0ff97700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-18T00:23:18.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:17 smithi059 bash[38133]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a 2024-02-18T00:23:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:18 smithi059 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.a.service: Succeeded. 2024-02-18T00:23:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:18 smithi059 systemd[1]: Stopped Ceph mon.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:23:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:18 smithi059 systemd[1]: Starting Ceph mon.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:23:19.201 INFO:teuthology.orchestra.run.smithi059.stderr:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-18T00:23:19.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.141+0000 7f6168467880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T00:23:19.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.141+0000 7f6168467880 0 ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable), process ceph-mon, pid 7 2024-02-18T00:23:19.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.141+0000 7f6168467880 0 pidfile_write: ignore empty --pid-file 2024-02-18T00:23:19.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 0 load: jerasure load: lrc load: isa 2024-02-18T00:23:19.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T00:23:19.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: DB SUMMARY 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: CURRENT file: CURRENT 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-18T00:23:19.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2024-02-18T00:23:19.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 94587 ; 2024-02-18T00:23:19.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T00:23:19.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.create_if_missing: 0 2024-02-18T00:23:19.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T00:23:19.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.env: 0x558eb1d39080 2024-02-18T00:23:19.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.fs: Posix File System 2024-02-18T00:23:19.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.info_log: 0x558eb2aca260 2024-02-18T00:23:19.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T00:23:19.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.statistics: (nil) 2024-02-18T00:23:19.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.use_fsync: 0 2024-02-18T00:23:19.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.db_log_dir: 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T00:23:19.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.write_buffer_manager: 0x558eb37bc3c0 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T00:23:19.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.unordered_write: 0 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.row_cache: None 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.wal_filter: None 2024-02-18T00:23:19.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T00:23:19.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T00:23:19.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T00:23:19.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T00:23:19.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T00:23:19.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T00:23:19.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T00:23:19.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T00:23:19.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T00:23:19.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T00:23:19.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T00:23:19.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T00:23:19.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T00:23:19.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T00:23:19.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T00:23:19.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T00:23:19.479 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T00:23:19.479 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T00:23:19.479 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T00:23:19.479 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T00:23:19.479 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T00:23:19.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.max_open_files: -1 2024-02-18T00:23:19.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T00:23:19.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T00:23:19.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T00:23:19.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T00:23:19.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: Compression algorithms supported: 2024-02-18T00:23:19.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.146+0000 7f6168467880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T00:23:19.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: kZSTD supported: 0 2024-02-18T00:23:19.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T00:23:19.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T00:23:19.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T00:23:19.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T00:23:19.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T00:23:19.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T00:23:19.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T00:23:19.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2024-02-18T00:23:19.483 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.483 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T00:23:19.483 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.483 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T00:23:19.483 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.merge_operator: 2024-02-18T00:23:19.483 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_filter: None 2024-02-18T00:23:19.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T00:23:19.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T00:23:19.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T00:23:19.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558eb2a6fd10) 2024-02-18T00:23:19.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cache_index_and_filter_blocks: 1 2024-02-18T00:23:19.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T00:23:19.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T00:23:19.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: pin_top_level_index_and_filter: 1 2024-02-18T00:23:19.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: index_type: 0 2024-02-18T00:23:19.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: data_block_index_type: 0 2024-02-18T00:23:19.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: index_shortening: 1 2024-02-18T00:23:19.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: data_block_hash_table_util_ratio: 0.750000 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: hash_index_allow_collision: 1 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: checksum: 1 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: no_block_cache: 0 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_cache: 0x558eb2aa4f10 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_cache_name: BinnedLRUCache 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_cache_options: 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: capacity : 536870912 2024-02-18T00:23:19.486 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: num_shard_bits : 4 2024-02-18T00:23:19.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: strict_capacity_limit : 0 2024-02-18T00:23:19.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: high_pri_pool_ratio: 0.000 2024-02-18T00:23:19.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_cache_compressed: (nil) 2024-02-18T00:23:19.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: persistent_cache: (nil) 2024-02-18T00:23:19.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_size: 4096 2024-02-18T00:23:19.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_size_deviation: 10 2024-02-18T00:23:19.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_restart_interval: 16 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: index_block_restart_interval: 1 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: metadata_block_size: 4096 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: partition_filters: 0 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: use_delta_encoding: 1 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: whole_key_filtering: 1 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: verify_compression: 0 2024-02-18T00:23:19.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: read_amp_bytes_per_bit: 0 2024-02-18T00:23:19.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: format_version: 2 2024-02-18T00:23:19.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: enable_index_compression: 1 2024-02-18T00:23:19.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: block_align: 0 2024-02-18T00:23:19.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T00:23:19.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T00:23:19.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compression: NoCompression 2024-02-18T00:23:19.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T00:23:19.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T00:23:19.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T00:23:19.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.num_levels: 7 2024-02-18T00:23:19.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T00:23:19.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T00:23:19.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T00:23:19.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T00:23:19.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T00:23:19.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T00:23:19.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T00:23:19.492 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:23:19.492 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T00:23:19.492 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T00:23:19.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T00:23:19.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T00:23:19.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T00:23:19.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:23:19.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T00:23:19.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T00:23:19.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T00:23:19.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T00:23:19.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T00:23:19.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T00:23:19.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T00:23:19.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T00:23:19.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T00:23:19.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T00:23:19.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T00:23:19.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T00:23:19.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T00:23:19.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T00:23:19.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T00:23:19.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T00:23:19.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T00:23:19.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T00:23:19.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T00:23:19.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T00:23:19.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T00:23:19.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T00:23:19.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T00:23:19.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T00:23:19.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T00:23:19.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T00:23:19.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T00:23:19.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T00:23:19.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T00:23:19.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.ttl: 2592000 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.147+0000 7f6168467880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.148+0000 7f6168467880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215799149220, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.148+0000 7f6168467880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.148+0000 7f6168467880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-18T00:23:19.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.149+0000 7f6168467880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215799150178, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 90427, "table_properties": {"data_size": 88529, "index_size": 375, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 11592, "raw_average_key_size": 49, "raw_value_size": 82168, "raw_average_value_size": 348, "num_data_blocks": 12, "num_entries": 236, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708215799, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.149+0000 7f6168467880 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.149+0000 7f6168467880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215799150609, "job": 1, "event": "recovery_finished"} 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.150+0000 7f6168467880 4 rocksdb: DB pointer 0x558eb2b6d800 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.150+0000 7f614f259700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.150+0000 7f614f259700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ** DB Stats ** 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T00:23:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ** Compaction Stats [default] ** 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: L0 2/0 89.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.8 0.00 0.00 1 0.001 0 0 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Sum 2/0 89.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.8 0.00 0.00 1 0.001 0 0 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.8 0.00 0.00 1 0.001 0 0 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ** Compaction Stats [default] ** 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.8 0.00 0.00 1 0.001 0 0 2024-02-18T00:23:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Cumulative compaction: 0.00 GB write, 27.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Interval compaction: 0.00 GB write, 27.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ** Compaction Stats [default] ** 2024-02-18T00:23:19.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: L0 2/0 89.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.8 0.00 0.00 1 0.001 0 0 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Sum 2/0 89.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.8 0.00 0.00 1 0.001 0 0 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ** Compaction Stats [default] ** 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.8 0.00 0.00 1 0.001 0 0 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:23:19.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Cumulative compaction: 0.00 GB write, 27.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.150+0000 7f6168467880 0 starting mon.a rank 0 at public addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] at bind addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 1 mon.a@-1(???) e1 preinit fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 0 mon.a@-1(???).mds e1 new map 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 0 mon.a@-1(???).mds e1 print_map 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: e1 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T00:23:19.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: legacy client fscid: -1 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: No filesystems configured 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:14.570965+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.59:0/4007471963' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:23:19.152+0000 7f6168467880 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 bash[38256]: 91c0c473f56b984b0197772f646dbd892670f0ee732865f78b5312926295d8e8 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 systemd[1]: Started Ceph mon.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:19.168788+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T00:23:19.507 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:19.168831+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0]} removed_ranks: {} 2024-02-18T00:23:19.508 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:19.169223+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-02-18T00:23:19.508 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:19.169254+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-18T00:23:19.508 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:19.169632+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-18T00:23:19.779 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.762+0000 7f9e3eed8700 1 Processor -- start 2024-02-18T00:23:19.779 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.762+0000 7f9e3eed8700 1 -- start start 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.762+0000 7f9e3eed8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e380fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.762+0000 7f9e3eed8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e380fb8e0 con 0x7f9e380faf40 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.763+0000 7f9e3cc74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e380fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.763+0000 7f9e3cc74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e380fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58048/0 (socket says 172.21.15.59:58048) 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.763+0000 7f9e3cc74700 1 -- 172.21.15.59:0/1418904917 learned_addr learned my addr 172.21.15.59:0/1418904917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.763+0000 7f9e3cc74700 1 -- 172.21.15.59:0/1418904917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e380fba20 con 0x7f9e380faf40 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.763+0000 7f9e3cc74700 1 --2- 172.21.15.59:0/1418904917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e380fb360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f9e2c009730 tx=0x7f9e2c0090f0).ready entity=mon.0 client_cookie=d1acde7d11d5aeb2 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e377fe700 1 -- 172.21.15.59:0/1418904917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e2c00b580 con 0x7f9e380faf40 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e377fe700 1 -- 172.21.15.59:0/1418904917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e2c00bb90 con 0x7f9e380faf40 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e377fe700 1 -- 172.21.15.59:0/1418904917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e2c011c30 con 0x7f9e380faf40 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e3eed8700 1 -- 172.21.15.59:0/1418904917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 msgr2=0x7f9e380fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e3eed8700 1 --2- 172.21.15.59:0/1418904917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e380fb360 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f9e2c009730 tx=0x7f9e2c0090f0).stop 2024-02-18T00:23:19.780 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e3eed8700 1 -- 172.21.15.59:0/1418904917 shutdown_connections 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e3eed8700 1 --2- 172.21.15.59:0/1418904917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e380fb360 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e3eed8700 1 -- 172.21.15.59:0/1418904917 >> 172.21.15.59:0/1418904917 conn(0x7f9e380f6660 msgr2=0x7f9e380f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e3eed8700 1 -- 172.21.15.59:0/1418904917 shutdown_connections 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.764+0000 7f9e3eed8700 1 -- 172.21.15.59:0/1418904917 wait complete. 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 librados: starting msgr at 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 librados: starting objecter 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 Processor -- start 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 -- start start 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 librados: setting wanted keys 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 librados: calling monclient init 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e38109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3eed8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3810a0a0 con 0x7f9e380faf40 2024-02-18T00:23:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3cc74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e38109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3cc74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e38109bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58060/0 (socket says 172.21.15.59:58060) 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.765+0000 7f9e3cc74700 1 -- 172.21.15.59:0/3044806817 learned_addr learned my addr 172.21.15.59:0/3044806817 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e3cc74700 1 -- 172.21.15.59:0/3044806817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c009370 con 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e3cc74700 1 --2- 172.21.15.59:0/3044806817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e38109bb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f9e2c000c00 tx=0x7f9e2c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e35ffb700 10 client.?.objecter ms_handle_connect 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e35ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e35ffb700 1 -- 172.21.15.59:0/3044806817 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e2c00b9a0 con 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e35ffb700 1 -- 172.21.15.59:0/3044806817 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e2c01aaf0 con 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e35ffb700 1 -- 172.21.15.59:0/3044806817 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e2c023ac0 con 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3810a2a0 con 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e3eed8700 10 client.14098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3810a730 con 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e3eed8700 1 librados: init done 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e35ffb700 1 -- 172.21.15.59:0/3044806817 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f9e2c02c3f0 con 0x7f9e380faf40 2024-02-18T00:23:19.782 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.766+0000 7f9e35ffb700 1 -- 172.21.15.59:0/3044806817 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f9e2c022730 con 0x7f9e380faf40 2024-02-18T00:23:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.767+0000 7f9e35ffb700 10 client.14098.objecter ms_dispatch 0x7f9e38001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.767+0000 7f9e35ffb700 3 client.14098.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.767+0000 7f9e35ffb700 3 client.14098.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.767+0000 7f9e35ffb700 20 client.14098.objecter dump_active .. 0 homeless 2024-02-18T00:23:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.767+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e38046ce0 con 0x7f9e380faf40 2024-02-18T00:23:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.769+0000 7f9e35ffb700 1 -- 172.21.15.59:0/3044806817 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f9e2c01e030 con 0x7f9e380faf40 2024-02-18T00:23:19.838 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.836+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f9e38103890 con 0x7f9e380faf40 2024-02-18T00:23:19.841 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.838+0000 7f9e35ffb700 1 -- 172.21.15.59:0/3044806817 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9e2c023520 con 0x7f9e380faf40 2024-02-18T00:23:19.842 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 10 librados: watch_flush enter 2024-02-18T00:23:19.842 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 10 librados: watch_flush exit 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 20 client.14098.objecter shutdown clearing up homeless session... 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 10 client.14098.objecter successfully canceled tick 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 msgr2=0x7f9e38109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 1 --2- 172.21.15.59:0/3044806817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e38109bb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f9e2c000c00 tx=0x7f9e2c0090f0).stop 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 shutdown_connections 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 1 --2- 172.21.15.59:0/3044806817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e380faf40 0x7f9e38109bb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 >> 172.21.15.59:0/3044806817 conn(0x7f9e380f6660 msgr2=0x7f9e380f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.839+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 shutdown_connections 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.840+0000 7f9e3eed8700 1 -- 172.21.15.59:0/3044806817 wait complete. 2024-02-18T00:23:19.843 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:19.840+0000 7f9e3eed8700 1 librados: shutdown 2024-02-18T00:23:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-18T00:23:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-18T00:23:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:Creating mgr... 2024-02-18T00:23:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:Verifying port 9283 ... 2024-02-18T00:23:20.597 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Failed to reset failed state of unit ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.y.service: Unit ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.y.service not loaded. 2024-02-18T00:23:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: Created symlink /etc/systemd/system/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715.target.wants/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@.service. 2024-02-18T00:23:20.703 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:20 smithi059 systemd[1]: Starting Ceph mgr.y for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:23:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:19.840010+0000 mon.a (mon.0) 6 : audit [INF] from='client.? 172.21.15.59:0/3044806817' entity='client.admin' 2024-02-18T00:23:21.327 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: disabled 2024-02-18T00:23:21.335 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: inactive 2024-02-18T00:23:21.335 INFO:teuthology.orchestra.run.smithi059.stderr:firewalld.service is not enabled 2024-02-18T00:23:21.335 INFO:teuthology.orchestra.run.smithi059.stderr:Not possible to enable service . firewalld.service is not available 2024-02-18T00:23:21.343 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: disabled 2024-02-18T00:23:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: inactive 2024-02-18T00:23:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:firewalld.service is not enabled 2024-02-18T00:23:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-18T00:23:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for mgr to start... 2024-02-18T00:23:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for mgr... 2024-02-18T00:23:21.568 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:21 smithi059 bash[38534]: 6cd56f3c80d4eb9663271ce987d3caa098872da2a11ba876919fdc12a2046bae 2024-02-18T00:23:21.568 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:21 smithi059 systemd[1]: Started Ceph mgr.y for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:23:21.568 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:21.360+0000 7f9eeec96680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T00:23:21.568 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:21.446+0000 7f9eeec96680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T00:23:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.918+0000 7f2306828700 1 Processor -- start 2024-02-18T00:23:21.921 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.918+0000 7f2306828700 1 -- start start 2024-02-18T00:23:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.918+0000 7f2306828700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f23000fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.918+0000 7f2306828700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000ff1d0 con 0x7f2300070c80 2024-02-18T00:23:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.918+0000 7f22fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f23000fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.918+0000 7f22fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f23000fec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58076/0 (socket says 172.21.15.59:58076) 2024-02-18T00:23:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.918+0000 7f22fffff700 1 -- 172.21.15.59:0/448610904 learned_addr learned my addr 172.21.15.59:0/448610904 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:21.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.919+0000 7f22fffff700 1 -- 172.21.15.59:0/448610904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23000ff310 con 0x7f2300070c80 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.919+0000 7f22fffff700 1 --2- 172.21.15.59:0/448610904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f23000fec50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f22f0009730 tx=0x7f22f00090f0).ready entity=mon.0 client_cookie=fc6f1236ce713df8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.919+0000 7f22feffd700 1 -- 172.21.15.59:0/448610904 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22f000b580 con 0x7f2300070c80 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.919+0000 7f22feffd700 1 -- 172.21.15.59:0/448610904 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22f000bb90 con 0x7f2300070c80 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.919+0000 7f22feffd700 1 -- 172.21.15.59:0/448610904 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22f0011c30 con 0x7f2300070c80 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 -- 172.21.15.59:0/448610904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 msgr2=0x7f23000fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 --2- 172.21.15.59:0/448610904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f23000fec50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f22f0009730 tx=0x7f22f00090f0).stop 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 -- 172.21.15.59:0/448610904 shutdown_connections 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 --2- 172.21.15.59:0/448610904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f23000fec50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 -- 172.21.15.59:0/448610904 >> 172.21.15.59:0/448610904 conn(0x7f23000f6670 msgr2=0x7f23000f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 -- 172.21.15.59:0/448610904 shutdown_connections 2024-02-18T00:23:21.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 -- 172.21.15.59:0/448610904 wait complete. 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 librados: starting msgr at 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 librados: starting objecter 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 Processor -- start 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 -- start start 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 librados: setting wanted keys 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.920+0000 7f2306828700 1 librados: calling monclient init 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f2306828700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f230010ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f2306828700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230010b270 con 0x7f2300070c80 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f22fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f230010ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f22fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f230010ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58086/0 (socket says 172.21.15.59:58086) 2024-02-18T00:23:21.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f22fffff700 1 -- 172.21.15.59:0/3939034634 learned_addr learned my addr 172.21.15.59:0/3939034634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f22fffff700 1 -- 172.21.15.59:0/3939034634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0009370 con 0x7f2300070c80 2024-02-18T00:23:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f22fffff700 1 --2- 172.21.15.59:0/3939034634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f230010ad80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f22f0006940 tx=0x7f22f00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f22fd7fa700 10 client.?.objecter ms_handle_connect 0x7f2300070c80 2024-02-18T00:23:21.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.921+0000 7f22fd7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 1 -- 172.21.15.59:0/3939034634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22f001a430 con 0x7f2300070c80 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 1 -- 172.21.15.59:0/3939034634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22f001aa40 con 0x7f2300070c80 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230010b470 con 0x7f2300070c80 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 1 -- 172.21.15.59:0/3939034634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22f0023b00 con 0x7f2300070c80 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f2306828700 10 client.14102.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230010c080 con 0x7f2300070c80 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f2306828700 1 librados: init done 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 1 -- 172.21.15.59:0/3939034634 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f22f00223c0 con 0x7f2300070c80 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 1 -- 172.21.15.59:0/3939034634 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f22f0019910 con 0x7f2300070c80 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 10 client.14102.objecter ms_dispatch 0x7f2300001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:21.929 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 3 client.14102.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 3 client.14102.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.922+0000 7f22fd7fa700 20 client.14102.objecter dump_active .. 0 homeless 2024-02-18T00:23:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.923+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230005a310 con 0x7f2300070c80 2024-02-18T00:23:21.930 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.925+0000 7f22fd7fa700 1 -- 172.21.15.59:0/3939034634 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f22f0043e50 con 0x7f2300070c80 2024-02-18T00:23:22.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.995+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2300100220 con 0x7f2300070c80 2024-02-18T00:23:22.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.996+0000 7f22fd7fa700 1 -- 172.21.15.59:0/3939034634 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7f22f0032070 con 0x7f2300070c80 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: { 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "health": { 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "checks": {}, 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mutes": [] 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum": [ 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 0 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "a" 2024-02-18T00:23:22.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "monmap": { 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osdmap": { 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-18T00:23:22.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "pgmap": { 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "fsmap": { 2024-02-18T00:23:22.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "available": false, 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "modules": [ 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "iostat", 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "nfs", 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "restful" 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:22.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "services": {} 2024-02-18T00:23:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "servicemap": { 2024-02-18T00:23:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "modified": "2024-02-18T00:23:11.391474+0000", 2024-02-18T00:23:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "services": {} 2024-02-18T00:23:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:22.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: } 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 10 librados: watch_flush enter 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 10 librados: watch_flush exit 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 20 client.14102.objecter shutdown clearing up homeless session... 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 10 client.14102.objecter successfully canceled tick 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 msgr2=0x7f230010ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 --2- 172.21.15.59:0/3939034634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f230010ad80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f22f0006940 tx=0x7f22f00090f0).stop 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 shutdown_connections 2024-02-18T00:23:22.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 --2- 172.21.15.59:0/3939034634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2300070c80 0x7f230010ad80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 >> 172.21.15.59:0/3939034634 conn(0x7f23000f6670 msgr2=0x7f23000f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 shutdown_connections 2024-02-18T00:23:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 -- 172.21.15.59:0/3939034634 wait complete. 2024-02-18T00:23:22.012 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:21.997+0000 7f2306828700 1 librados: shutdown 2024-02-18T00:23:22.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:21.997218+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.59:0/3939034634' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T00:23:22.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:21.856+0000 7f9eeec96680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T00:23:22.691 INFO:teuthology.orchestra.run.smithi059.stderr:mgr not available, waiting (1/15)... 2024-02-18T00:23:22.965 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:22.663+0000 7f9eeec96680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T00:23:22.965 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:22.745+0000 7f9eeec96680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T00:23:24.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:22.963+0000 7f9eeec96680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T00:23:24.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:23.945+0000 7f9eeec96680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T00:23:24.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:24.077+0000 7f9eeec96680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:23:24.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:24.158+0000 7f9eeec96680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T00:23:24.747 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:24.250+0000 7f9eeec96680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T00:23:24.747 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:24.450+0000 7f9eeec96680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T00:23:25.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:24.745+0000 7f9eeec96680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T00:23:25.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:24.848+0000 7f9eeec96680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.249+0000 7f1c6082a700 1 Processor -- start 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.249+0000 7f1c6082a700 1 -- start start 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c6082a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c1010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c6082a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0fcb30 con 0x7f1c5c070c20 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c5a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c1010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c5a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c1010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58088/0 (socket says 172.21.15.59:58088) 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c5a59c700 1 -- 172.21.15.59:0/293325515 learned_addr learned my addr 172.21.15.59:0/293325515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c5a59c700 1 -- 172.21.15.59:0/293325515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c1015a0 con 0x7f1c5c070c20 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c5a59c700 1 --2- 172.21.15.59:0/293325515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c1010b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1c4c009730 tx=0x7f1c4c0090f0).ready entity=mon.0 client_cookie=b1fad1454b99d258 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.250+0000 7f1c5959a700 1 -- 172.21.15.59:0/293325515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c4c00b580 con 0x7f1c5c070c20 2024-02-18T00:23:25.254 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c5959a700 1 -- 172.21.15.59:0/293325515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c4c00bb90 con 0x7f1c5c070c20 2024-02-18T00:23:25.254 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c5959a700 1 -- 172.21.15.59:0/293325515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c4c011c30 con 0x7f1c5c070c20 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c6082a700 1 -- 172.21.15.59:0/293325515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 msgr2=0x7f1c5c1010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c6082a700 1 --2- 172.21.15.59:0/293325515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c1010b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1c4c009730 tx=0x7f1c4c0090f0).stop 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c6082a700 1 -- 172.21.15.59:0/293325515 shutdown_connections 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c6082a700 1 --2- 172.21.15.59:0/293325515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c1010b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c6082a700 1 -- 172.21.15.59:0/293325515 >> 172.21.15.59:0/293325515 conn(0x7f1c5c0f6670 msgr2=0x7f1c5c0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c6082a700 1 -- 172.21.15.59:0/293325515 shutdown_connections 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.251+0000 7f1c6082a700 1 -- 172.21.15.59:0/293325515 wait complete. 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.252+0000 7f1c6082a700 1 librados: starting msgr at 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.252+0000 7f1c6082a700 1 librados: starting objecter 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.252+0000 7f1c6082a700 1 Processor -- start 2024-02-18T00:23:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.252+0000 7f1c6082a700 1 -- start start 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.252+0000 7f1c6082a700 1 librados: setting wanted keys 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.252+0000 7f1c6082a700 1 librados: calling monclient init 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c6082a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c10cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c6082a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c10d420 con 0x7f1c5c070c20 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c5a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c10cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c5a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c10cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58098/0 (socket says 172.21.15.59:58098) 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c5a59c700 1 -- 172.21.15.59:0/2627483466 learned_addr learned my addr 172.21.15.59:0/2627483466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c5a59c700 1 -- 172.21.15.59:0/2627483466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c009370 con 0x7f1c5c070c20 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c5a59c700 1 --2- 172.21.15.59:0/2627483466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c10cf30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f1c4c006940 tx=0x7f1c4c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c4b7fe700 10 client.?.objecter ms_handle_connect 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c4b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c4b7fe700 1 -- 172.21.15.59:0/2627483466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c4c01a430 con 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c4b7fe700 1 -- 172.21.15.59:0/2627483466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c4c01aa40 con 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c10d620 con 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c4b7fe700 1 -- 172.21.15.59:0/2627483466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c4c023ab0 con 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c6082a700 10 client.14104.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c10dab0 con 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.253+0000 7f1c6082a700 1 librados: init done 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.254+0000 7f1c4b7fe700 1 -- 172.21.15.59:0/2627483466 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f1c4c02c3f0 con 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.254+0000 7f1c4b7fe700 1 -- 172.21.15.59:0/2627483466 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f1c4c022730 con 0x7f1c5c070c20 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.254+0000 7f1c4b7fe700 10 client.14104.objecter ms_dispatch 0x7f1c5c001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.254+0000 7f1c4b7fe700 3 client.14104.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.254+0000 7f1c4b7fe700 3 client.14104.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.254+0000 7f1c4b7fe700 20 client.14104.objecter dump_active .. 0 homeless 2024-02-18T00:23:25.259 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.255+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c046ce0 con 0x7f1c5c070c20 2024-02-18T00:23:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.259+0000 7f1c4b7fe700 1 -- 172.21.15.59:0/2627483466 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f1c4c01e030 con 0x7f1c5c070c20 2024-02-18T00:23:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.329+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1c5c1023b0 con 0x7f1c5c070c20 2024-02-18T00:23:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c4b7fe700 1 -- 172.21.15.59:0/2627483466 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7f1c4c023410 con 0x7f1c5c070c20 2024-02-18T00:23:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: { 2024-02-18T00:23:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:23:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "health": { 2024-02-18T00:23:25.338 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "checks": {}, 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mutes": [] 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum": [ 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 0 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "a" 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "monmap": { 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:25.339 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osdmap": { 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "pgmap": { 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-18T00:23:25.340 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "fsmap": { 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "available": false, 2024-02-18T00:23:25.341 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "modules": [ 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "iostat", 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "nfs", 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "restful" 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "services": {} 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "servicemap": { 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "modified": "2024-02-18T00:23:11.391474+0000", 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "services": {} 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-18T00:23:25.342 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: } 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 10 librados: watch_flush enter 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 10 librados: watch_flush exit 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 20 client.14104.objecter shutdown clearing up homeless session... 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 10 client.14104.objecter successfully canceled tick 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 msgr2=0x7f1c5c10cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 1 --2- 172.21.15.59:0/2627483466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c10cf30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f1c4c006940 tx=0x7f1c4c0090f0).stop 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 shutdown_connections 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 1 --2- 172.21.15.59:0/2627483466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c070c20 0x7f1c5c10cf30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.330+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 >> 172.21.15.59:0/2627483466 conn(0x7f1c5c0f6670 msgr2=0x7f1c5c0f7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.331+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 shutdown_connections 2024-02-18T00:23:25.343 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.331+0000 7f1c6082a700 1 -- 172.21.15.59:0/2627483466 wait complete. 2024-02-18T00:23:25.344 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:25.331+0000 7f1c6082a700 1 librados: shutdown 2024-02-18T00:23:25.621 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:25.330927+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? 172.21.15.59:0/2627483466' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T00:23:25.977 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:25.619+0000 7f9eeec96680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T00:23:25.978 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:25.703+0000 7f9eeec96680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T00:23:25.978 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:25.797+0000 7f9eeec96680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T00:23:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:mgr not available, waiting (2/15)... 2024-02-18T00:23:26.245 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:25.975+0000 7f9eeec96680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T00:23:26.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:26.242+0000 7f9eeec96680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T00:23:26.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:26.377+0000 7f9eeec96680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T00:23:26.781 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:26.515+0000 7f9eeec96680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:23:26.781 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:26.693+0000 7f9eeec96680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T00:23:27.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:26.778+0000 7f9eeec96680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T00:23:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:26.793244+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:26.795998+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00286005s) 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.796920+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.797043+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.797165+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.797336+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.797613+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:26.811806+0000 mon.a (mon.0) 16 : cluster [INF] Manager daemon y is now available 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.820765+0000 mon.a (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.826465+0000 mon.a (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.829876+0000 mon.a (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.833109+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' 2024-02-18T00:23:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:26.835140+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.59:0/2624075140' entity='mgr.y' 2024-02-18T00:23:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:27.799326+0000 mon.a (mon.0) 22 : cluster [DBG] mgrmap e3: y(active, since 1.00619s) 2024-02-18T00:23:31.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:29.799756+0000 mon.a (mon.0) 23 : cluster [DBG] mgrmap e4: y(active, since 3s) 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.556+0000 7f002c5d2700 1 Processor -- start 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.556+0000 7f002c5d2700 1 -- start start 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.556+0000 7f002c5d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f0024101090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.556+0000 7f002c5d2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00240fcb10 con 0x7f0024070c40 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.556+0000 7f002a36e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f0024101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.556+0000 7f002a36e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f0024101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35232/0 (socket says 172.21.15.59:35232) 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.556+0000 7f002a36e700 1 -- 172.21.15.59:0/2728942091 learned_addr learned my addr 172.21.15.59:0/2728942091 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:31.623 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.557+0000 7f002a36e700 1 -- 172.21.15.59:0/2728942091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0024101580 con 0x7f0024070c40 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.557+0000 7f002a36e700 1 --2- 172.21.15.59:0/2728942091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f0024101090 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f0014009730 tx=0x7f00140090f0).ready entity=mon.0 client_cookie=315ffa61e2df3c46 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.557+0000 7f002936c700 1 -- 172.21.15.59:0/2728942091 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f001400b580 con 0x7f0024070c40 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.557+0000 7f002936c700 1 -- 172.21.15.59:0/2728942091 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f001400bb90 con 0x7f0024070c40 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.557+0000 7f002936c700 1 -- 172.21.15.59:0/2728942091 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0014011c30 con 0x7f0024070c40 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 -- 172.21.15.59:0/2728942091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 msgr2=0x7f0024101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 --2- 172.21.15.59:0/2728942091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f0024101090 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f0014009730 tx=0x7f00140090f0).stop 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 -- 172.21.15.59:0/2728942091 shutdown_connections 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 --2- 172.21.15.59:0/2728942091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f0024101090 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 -- 172.21.15.59:0/2728942091 >> 172.21.15.59:0/2728942091 conn(0x7f00240f6670 msgr2=0x7f00240f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 -- 172.21.15.59:0/2728942091 shutdown_connections 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 -- 172.21.15.59:0/2728942091 wait complete. 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 librados: starting msgr at 2024-02-18T00:23:31.624 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 librados: starting objecter 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 Processor -- start 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 -- start start 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 librados: setting wanted keys 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 librados: calling monclient init 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.558+0000 7f002c5d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f002410cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002c5d2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002410d410 con 0x7f0024070c40 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002a36e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f002410cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002a36e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f002410cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35244/0 (socket says 172.21.15.59:35244) 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002a36e700 1 -- 172.21.15.59:0/3409265267 learned_addr learned my addr 172.21.15.59:0/3409265267 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002a36e700 1 -- 172.21.15.59:0/3409265267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014009370 con 0x7f0024070c40 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002a36e700 1 --2- 172.21.15.59:0/3409265267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f002410cf20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f0014006940 tx=0x7f00140090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f001b7fe700 10 client.?.objecter ms_handle_connect 0x7f0024070c40 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f001b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:31.625 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f001b7fe700 1 -- 172.21.15.59:0/3409265267 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f001401a430 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f001b7fe700 1 -- 172.21.15.59:0/3409265267 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f001401aa40 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002410d610 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f001b7fe700 1 -- 172.21.15.59:0/3409265267 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0014023ab0 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002c5d2700 10 client.14110.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002410daa0 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.559+0000 7f002c5d2700 1 librados: init done 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.560+0000 7f001b7fe700 1 -- 172.21.15.59:0/3409265267 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 38571+0+0 (secure 0 0 0) 0x7f001401abb0 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.560+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0024046ce0 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.560+0000 7f001b7fe700 1 --2- 172.21.15.59:0/3409265267 >> v2:172.21.15.59:6800/1656814063 conn(0x7f0010033fd0 0x7f0010036490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.560+0000 7f001b7fe700 1 -- 172.21.15.59:0/3409265267 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f0014044b60 con 0x7f0024070c40 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.560+0000 7f001b7fe700 10 client.14110.objecter ms_dispatch 0x7f0024001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.560+0000 7f001b7fe700 3 client.14110.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:31.626 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.560+0000 7f001b7fe700 3 client.14110.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.561+0000 7f001b7fe700 20 client.14110.objecter dump_active .. 0 homeless 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.561+0000 7f0029b6d700 1 --2- 172.21.15.59:0/3409265267 >> v2:172.21.15.59:6800/1656814063 conn(0x7f0010033fd0 0x7f0010036490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.561+0000 7f0029b6d700 1 --2- 172.21.15.59:0/3409265267 >> v2:172.21.15.59:6800/1656814063 conn(0x7f0010033fd0 0x7f0010036490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f0020009780 tx=0x7f0020006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.561+0000 7f001b7fe700 10 client.14110.objecter ms_handle_connect 0x7f0010033fd0 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.565+0000 7f001b7fe700 1 -- 172.21.15.59:0/3409265267 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0014019570 con 0x7f0024070c40 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.804+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0024102390 con 0x7f0024070c40 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.835+0000 7f001b7fe700 1 -- 172.21.15.59:0/3409265267 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7f001401e070 con 0x7f0024070c40 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 10 librados: watch_flush enter 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 10 librados: watch_flush exit 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 20 client.14110.objecter shutdown clearing up homeless session... 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 10 client.14110.objecter successfully canceled tick 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 >> v2:172.21.15.59:6800/1656814063 conn(0x7f0010033fd0 msgr2=0x7f0010036490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 --2- 172.21.15.59:0/3409265267 >> v2:172.21.15.59:6800/1656814063 conn(0x7f0010033fd0 0x7f0010036490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f0020009780 tx=0x7f0020006cc0).stop 2024-02-18T00:23:31.627 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 msgr2=0x7f002410cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 --2- 172.21.15.59:0/3409265267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f002410cf20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f0014006940 tx=0x7f00140090f0).stop 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 shutdown_connections 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 --2- 172.21.15.59:0/3409265267 >> v2:172.21.15.59:6800/1656814063 conn(0x7f0010033fd0 0x7f0010036490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 --2- 172.21.15.59:0/3409265267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0024070c40 0x7f002410cf20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 >> 172.21.15.59:0/3409265267 conn(0x7f00240f6670 msgr2=0x7f00240f7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 shutdown_connections 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 -- 172.21.15.59:0/3409265267 wait complete. 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:30.837+0000 7f002c5d2700 1 librados: shutdown 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: { 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:23:31.628 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "health": { 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "checks": {}, 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mutes": [] 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum": [ 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 0 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "a" 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "quorum_age": 11, 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "monmap": { 2024-02-18T00:23:31.629 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osdmap": { 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "pgmap": { 2024-02-18T00:23:31.630 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "fsmap": { 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-18T00:23:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "available": true, 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "modules": [ 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "iostat", 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "nfs", 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "restful" 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ], 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "services": {} 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "servicemap": { 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "modified": "2024-02-18T00:23:11.391474+0000", 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "services": {} 2024-02-18T00:23:31.632 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: }, 2024-02-18T00:23:31.633 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-18T00:23:31.633 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: } 2024-02-18T00:23:32.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:30.836699+0000 mon.a (mon.0) 24 : audit [DBG] from='client.? 172.21.15.59:0/3409265267' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T00:23:36.746 INFO:teuthology.orchestra.run.smithi059.stderr:mgr is available 2024-02-18T00:23:38.054 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.048+0000 7ff9a36cf700 1 Processor -- start 2024-02-18T00:23:38.054 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.048+0000 7ff9a36cf700 1 -- start start 2024-02-18T00:23:38.054 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.048+0000 7ff9a36cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:38.054 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.048+0000 7ff9a36cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0730a0 con 0x7ff99c072c80 2024-02-18T00:23:38.054 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.048+0000 7ff9a146b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.048+0000 7ff9a146b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35254/0 (socket says 172.21.15.59:35254) 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.048+0000 7ff9a146b700 1 -- 172.21.15.59:0/3346982632 learned_addr learned my addr 172.21.15.59:0/3346982632 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.049+0000 7ff9a146b700 1 -- 172.21.15.59:0/3346982632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c071710 con 0x7ff99c072c80 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.049+0000 7ff9a146b700 1 --2- 172.21.15.59:0/3346982632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c071220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7ff998009730 tx=0x7ff9980090f0).ready entity=mon.0 client_cookie=d06d570c3329b3c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.049+0000 7ff993fff700 1 -- 172.21.15.59:0/3346982632 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff99800b580 con 0x7ff99c072c80 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.049+0000 7ff993fff700 1 -- 172.21.15.59:0/3346982632 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff99800bb90 con 0x7ff99c072c80 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.049+0000 7ff993fff700 1 -- 172.21.15.59:0/3346982632 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff998011c30 con 0x7ff99c072c80 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.049+0000 7ff9a36cf700 1 -- 172.21.15.59:0/3346982632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 msgr2=0x7ff99c071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.049+0000 7ff9a36cf700 1 --2- 172.21.15.59:0/3346982632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c071220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7ff998009730 tx=0x7ff9980090f0).stop 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 -- 172.21.15.59:0/3346982632 shutdown_connections 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 --2- 172.21.15.59:0/3346982632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c071220 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:38.055 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 -- 172.21.15.59:0/3346982632 >> 172.21.15.59:0/3346982632 conn(0x7ff99c0f7860 msgr2=0x7ff99c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 -- 172.21.15.59:0/3346982632 shutdown_connections 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 -- 172.21.15.59:0/3346982632 wait complete. 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 librados: starting msgr at 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 librados: starting objecter 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 Processor -- start 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 -- start start 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 librados: setting wanted keys 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.050+0000 7ff9a36cf700 1 librados: calling monclient init 2024-02-18T00:23:38.056 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a36cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c10cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a36cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c10d410 con 0x7ff99c072c80 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a146b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c10cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a146b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c10cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35256/0 (socket says 172.21.15.59:35256) 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a146b700 1 -- 172.21.15.59:0/1691745413 learned_addr learned my addr 172.21.15.59:0/1691745413 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a146b700 1 -- 172.21.15.59:0/1691745413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff998009370 con 0x7ff99c072c80 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a146b700 1 --2- 172.21.15.59:0/1691745413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c10cf20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff998009f00 tx=0x7ff9980090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9927fc700 10 client.?.objecter ms_handle_connect 0x7ff99c072c80 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9927fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9927fc700 1 -- 172.21.15.59:0/1691745413 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff99801a430 con 0x7ff99c072c80 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9927fc700 1 -- 172.21.15.59:0/1691745413 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff99801aa40 con 0x7ff99c072c80 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c10d610 con 0x7ff99c072c80 2024-02-18T00:23:38.057 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9927fc700 1 -- 172.21.15.59:0/1691745413 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff998023ad0 con 0x7ff99c072c80 2024-02-18T00:23:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a36cf700 10 client.14112.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c10daa0 con 0x7ff99c072c80 2024-02-18T00:23:38.058 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.051+0000 7ff9a36cf700 1 librados: init done 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.052+0000 7ff9927fc700 1 -- 172.21.15.59:0/1691745413 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 38571+0+0 (secure 0 0 0) 0x7ff99801abb0 con 0x7ff99c072c80 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.052+0000 7ff9927fc700 1 --2- 172.21.15.59:0/1691745413 >> v2:172.21.15.59:6800/1656814063 conn(0x7ff988033f80 0x7ff988036440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.052+0000 7ff9927fc700 1 -- 172.21.15.59:0/1691745413 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7ff998044e70 con 0x7ff99c072c80 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.052+0000 7ff9927fc700 10 client.14112.objecter ms_dispatch 0x7ff99c001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.052+0000 7ff9927fc700 3 client.14112.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.052+0000 7ff9927fc700 3 client.14112.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.053+0000 7ff9927fc700 20 client.14112.objecter dump_active .. 0 homeless 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.053+0000 7ff9a0c6a700 1 --2- 172.21.15.59:0/1691745413 >> v2:172.21.15.59:6800/1656814063 conn(0x7ff988033f80 0x7ff988036440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.053+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c05a310 con 0x7ff99c072c80 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.053+0000 7ff9927fc700 10 client.14112.objecter ms_handle_connect 0x7ff988033f80 2024-02-18T00:23:38.059 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.053+0000 7ff9a0c6a700 1 --2- 172.21.15.59:0/1691745413 >> v2:172.21.15.59:6800/1656814063 conn(0x7ff988033f80 0x7ff988036440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff98c009780 tx=0x7ff98c006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:38.060 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.058+0000 7ff9927fc700 1 -- 172.21.15.59:0/1691745413 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff99801e080 con 0x7ff99c072c80 2024-02-18T00:23:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.229+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff99c046ce0 con 0x7ff99c072c80 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.230+0000 7ff9927fc700 1 -- 172.21.15.59:0/1691745413 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+307 (secure 0 0 0) 0x7ff998019390 con 0x7ff99c072c80 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: [global] 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: fsid = d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-18T00:23:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.235 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: [mgr] 2024-02-18T00:23:38.235 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-18T00:23:38.235 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.235 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: [osd] 2024-02-18T00:23:38.235 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-18T00:23:38.235 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 10 librados: watch_flush enter 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 10 librados: watch_flush exit 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 20 client.14112.objecter shutdown clearing up homeless session... 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 10 client.14112.objecter successfully canceled tick 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 >> v2:172.21.15.59:6800/1656814063 conn(0x7ff988033f80 msgr2=0x7ff988036440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 --2- 172.21.15.59:0/1691745413 >> v2:172.21.15.59:6800/1656814063 conn(0x7ff988033f80 0x7ff988036440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff98c009780 tx=0x7ff98c006cc0).stop 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 msgr2=0x7ff99c10cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 --2- 172.21.15.59:0/1691745413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c10cf20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7ff998009f00 tx=0x7ff9980090f0).stop 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 shutdown_connections 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 --2- 172.21.15.59:0/1691745413 >> v2:172.21.15.59:6800/1656814063 conn(0x7ff988033f80 0x7ff988036440 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 --2- 172.21.15.59:0/1691745413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c072c80 0x7ff99c10cf20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:38.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.232+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 >> 172.21.15.59:0/1691745413 conn(0x7ff99c0f7860 msgr2=0x7ff99c0f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:38.238 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.233+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 shutdown_connections 2024-02-18T00:23:38.238 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.233+0000 7ff9a36cf700 1 -- 172.21.15.59:0/1691745413 wait complete. 2024-02-18T00:23:38.238 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:38.233+0000 7ff9a36cf700 1 librados: shutdown 2024-02-18T00:23:38.542 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:38.231365+0000 mon.a (mon.0) 25 : audit [INF] from='client.? 172.21.15.59:0/1691745413' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-18T00:23:38.896 INFO:teuthology.orchestra.run.smithi059.stderr:Enabling cephadm module... 2024-02-18T00:23:39.443 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.437+0000 7fe40230c700 1 Processor -- start 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.437+0000 7fe40230c700 1 -- start start 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.437+0000 7fe40230c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc0fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.437+0000 7fe40230c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0fc950 con 0x7fe3fc0fc040 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.437+0000 7fe3fbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc0fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.437+0000 7fe3fbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc0fc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35260/0 (socket says 172.21.15.59:35260) 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.437+0000 7fe3fbfff700 1 -- 172.21.15.59:0/2295282778 learned_addr learned my addr 172.21.15.59:0/2295282778 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.438+0000 7fe3fbfff700 1 -- 172.21.15.59:0/2295282778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc0fca90 con 0x7fe3fc0fc040 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.438+0000 7fe3fbfff700 1 --2- 172.21.15.59:0/2295282778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc0fc460 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fe3ec009730 tx=0x7fe3ec0090f0).ready entity=mon.0 client_cookie=4d090509e230e882 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.438+0000 7fe3faffd700 1 -- 172.21.15.59:0/2295282778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec00b580 con 0x7fe3fc0fc040 2024-02-18T00:23:39.444 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.438+0000 7fe3faffd700 1 -- 172.21.15.59:0/2295282778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3ec00bb90 con 0x7fe3fc0fc040 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.438+0000 7fe3faffd700 1 -- 172.21.15.59:0/2295282778 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec011c30 con 0x7fe3fc0fc040 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.438+0000 7fe40230c700 1 -- 172.21.15.59:0/2295282778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 msgr2=0x7fe3fc0fc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.438+0000 7fe40230c700 1 --2- 172.21.15.59:0/2295282778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc0fc460 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fe3ec009730 tx=0x7fe3ec0090f0).stop 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 -- 172.21.15.59:0/2295282778 shutdown_connections 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 --2- 172.21.15.59:0/2295282778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc0fc460 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 -- 172.21.15.59:0/2295282778 >> 172.21.15.59:0/2295282778 conn(0x7fe3fc0f7800 msgr2=0x7fe3fc0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 -- 172.21.15.59:0/2295282778 shutdown_connections 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 -- 172.21.15.59:0/2295282778 wait complete. 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 librados: starting msgr at 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 librados: starting objecter 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 Processor -- start 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 -- start start 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 librados: setting wanted keys 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.439+0000 7fe40230c700 1 librados: calling monclient init 2024-02-18T00:23:39.445 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe40230c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc072d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe40230c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0711e0 con 0x7fe3fc0fc040 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe3fbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe3fbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc072d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35276/0 (socket says 172.21.15.59:35276) 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe3fbfff700 1 -- 172.21.15.59:0/3147714947 learned_addr learned my addr 172.21.15.59:0/3147714947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe3fbfff700 1 -- 172.21.15.59:0/3147714947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec009370 con 0x7fe3fc0fc040 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe3fbfff700 1 --2- 172.21.15.59:0/3147714947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc072d60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fe3ec00bf30 tx=0x7fe3ec0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe3f97fa700 10 client.?.objecter ms_handle_connect 0x7fe3fc0fc040 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.440+0000 7fe3f97fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec01a640 con 0x7fe3fc0fc040 2024-02-18T00:23:39.446 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3ec01ac50 con 0x7fe3fc0fc040 2024-02-18T00:23:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc071380 con 0x7fe3fc0fc040 2024-02-18T00:23:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ec023ae0 con 0x7fe3fc0fc040 2024-02-18T00:23:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe40230c700 10 client.14114.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc071810 con 0x7fe3fc0fc040 2024-02-18T00:23:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe40230c700 1 librados: init done 2024-02-18T00:23:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.441+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 38571+0+0 (secure 0 0 0) 0x7fe3ec023d50 con 0x7fe3fc0fc040 2024-02-18T00:23:39.447 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3f97fa700 1 --2- 172.21.15.59:0/3147714947 >> v2:172.21.15.59:6800/1656814063 conn(0x7fe3e8033f90 0x7fe3e8036450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fe3ec044f10 con 0x7fe3fc0fc040 2024-02-18T00:23:39.448 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3f97fa700 10 client.14114.objecter ms_dispatch 0x7fe3fc001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3f97fa700 3 client.14114.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3f97fa700 3 client.14114.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3f97fa700 20 client.14114.objecter dump_active .. 0 homeless 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3fb7fe700 1 --2- 172.21.15.59:0/3147714947 >> v2:172.21.15.59:6800/1656814063 conn(0x7fe3e8033f90 0x7fe3e8036450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc05a310 con 0x7fe3fc0fc040 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3f97fa700 10 client.14114.objecter ms_handle_connect 0x7fe3e8033f90 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.442+0000 7fe3fb7fe700 1 --2- 172.21.15.59:0/3147714947 >> v2:172.21.15.59:6800/1656814063 conn(0x7fe3e8033f90 0x7fe3e8036450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fe3e4009780 tx=0x7fe3e4006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:39.449 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.447+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe3ec01e070 con 0x7fe3fc0fc040 2024-02-18T00:23:39.664 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.662+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fe3fc046ce0 con 0x7fe3fc0fc040 2024-02-18T00:23:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.714+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 5) v1 ==== 38582+0+0 (secure 0 0 0) 0x7fe3ec02c3f0 con 0x7fe3fc0fc040 2024-02-18T00:23:39.717 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.715+0000 7fe3f97fa700 1 -- 172.21.15.59:0/3147714947 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fe3ec0103a0 con 0x7fe3fc0fc040 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 10 librados: watch_flush enter 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 10 librados: watch_flush exit 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 20 client.14114.objecter shutdown clearing up homeless session... 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 10 client.14114.objecter successfully canceled tick 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 >> v2:172.21.15.59:6800/1656814063 conn(0x7fe3e8033f90 msgr2=0x7fe3e8036450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 --2- 172.21.15.59:0/3147714947 >> v2:172.21.15.59:6800/1656814063 conn(0x7fe3e8033f90 0x7fe3e8036450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fe3e4009780 tx=0x7fe3e4006cc0).stop 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 msgr2=0x7fe3fc072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 --2- 172.21.15.59:0/3147714947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc072d60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fe3ec00bf30 tx=0x7fe3ec0090f0).stop 2024-02-18T00:23:39.721 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 shutdown_connections 2024-02-18T00:23:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 --2- 172.21.15.59:0/3147714947 >> v2:172.21.15.59:6800/1656814063 conn(0x7fe3e8033f90 0x7fe3e8036450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 --2- 172.21.15.59:0/3147714947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3fc0fc040 0x7fe3fc072d60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.717+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 >> 172.21.15.59:0/3147714947 conn(0x7fe3fc0f7800 msgr2=0x7fe3fc0f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.718+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 shutdown_connections 2024-02-18T00:23:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.718+0000 7fe40230c700 1 -- 172.21.15.59:0/3147714947 wait complete. 2024-02-18T00:23:39.722 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:39.718+0000 7fe40230c700 1 librados: shutdown 2024-02-18T00:23:40.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:39.663835+0000 mon.a (mon.0) 26 : audit [INF] from='client.? 172.21.15.59:0/3147714947' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-18T00:23:40.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ignoring --setuser ceph since I am not root 2024-02-18T00:23:40.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ignoring --setgroup ceph since I am not root 2024-02-18T00:23:40.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:39.879+0000 7f152e167680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T00:23:40.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:39.963+0000 7f152e167680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T00:23:40.447 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:40.344+0000 7f152e167680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T00:23:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:39.716853+0000 mon.a (mon.0) 27 : audit [INF] from='client.? 172.21.15.59:0/3147714947' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-18T00:23:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:39.716909+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e5: y(active, since 12s) 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.986+0000 7f7798e1d700 1 Processor -- start 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.986+0000 7f7798e1d700 1 -- start start 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.986+0000 7f7798e1d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77940fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.986+0000 7f7798e1d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77940fb8f0 con 0x7f77940faf50 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.986+0000 7f779259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77940fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.986+0000 7f779259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77940fb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44958/0 (socket says 172.21.15.59:44958) 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.986+0000 7f779259c700 1 -- 172.21.15.59:0/1860388706 learned_addr learned my addr 172.21.15.59:0/1860388706 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.987+0000 7f779259c700 1 -- 172.21.15.59:0/1860388706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940fba30 con 0x7f77940faf50 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.987+0000 7f779259c700 1 --2- 172.21.15.59:0/1860388706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77940fb370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7784009730 tx=0x7f77840090f0).ready entity=mon.0 client_cookie=fe8414f4a8eb4507 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:41.008 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.987+0000 7f779159a700 1 -- 172.21.15.59:0/1860388706 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f778400b580 con 0x7f77940faf50 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.987+0000 7f779159a700 1 -- 172.21.15.59:0/1860388706 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f778400bb90 con 0x7f77940faf50 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.987+0000 7f779159a700 1 -- 172.21.15.59:0/1860388706 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7784011c30 con 0x7f77940faf50 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.987+0000 7f7798e1d700 1 -- 172.21.15.59:0/1860388706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 msgr2=0x7f77940fb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.987+0000 7f7798e1d700 1 --2- 172.21.15.59:0/1860388706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77940fb370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7784009730 tx=0x7f77840090f0).stop 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 -- 172.21.15.59:0/1860388706 shutdown_connections 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 --2- 172.21.15.59:0/1860388706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77940fb370 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 -- 172.21.15.59:0/1860388706 >> 172.21.15.59:0/1860388706 conn(0x7f77940f6670 msgr2=0x7f77940f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 -- 172.21.15.59:0/1860388706 shutdown_connections 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 -- 172.21.15.59:0/1860388706 wait complete. 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 librados: starting msgr at 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 librados: starting objecter 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 Processor -- start 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 -- start start 2024-02-18T00:23:41.009 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 librados: setting wanted keys 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.988+0000 7f7798e1d700 1 librados: calling monclient init 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f7798e1d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77941013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f7798e1d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77940ff820 con 0x7f77940faf50 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f779259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77941013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f779259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77941013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44960/0 (socket says 172.21.15.59:44960) 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f779259c700 1 -- 172.21.15.59:0/2320888479 learned_addr learned my addr 172.21.15.59:0/2320888479 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f779259c700 1 -- 172.21.15.59:0/2320888479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7784009370 con 0x7f77940faf50 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f779259c700 1 --2- 172.21.15.59:0/2320888479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77941013a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f778400bf60 tx=0x7f77840090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f77837fe700 10 client.?.objecter ms_handle_connect 0x7f77940faf50 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f77837fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f77837fe700 1 -- 172.21.15.59:0/2320888479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7784011bf0 con 0x7f77940faf50 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f77837fe700 1 -- 172.21.15.59:0/2320888479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f778401aaf0 con 0x7f77940faf50 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77940ff9c0 con 0x7f77940faf50 2024-02-18T00:23:41.010 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f77837fe700 1 -- 172.21.15.59:0/2320888479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77840229c0 con 0x7f77940faf50 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f7798e1d700 10 client.14118.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77940ffe50 con 0x7f77940faf50 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.989+0000 7f7798e1d700 1 librados: init done 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.990+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779405a310 con 0x7f77940faf50 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.990+0000 7f77837fe700 1 -- 172.21.15.59:0/2320888479 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 38582+0+0 (secure 0 0 0) 0x7f7784022b20 con 0x7f77940faf50 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.990+0000 7f77837fe700 1 --2- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 0x7f777c0364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.991+0000 7f77837fe700 1 -- 172.21.15.59:0/2320888479 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f77840447a0 con 0x7f77940faf50 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.991+0000 7f77837fe700 10 client.14118.objecter ms_dispatch 0x7f7794001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.991+0000 7f77837fe700 3 client.14118.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.991+0000 7f77837fe700 3 client.14118.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.991+0000 7f7791d9b700 1 -- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 msgr2=0x7f777c0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1656814063 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.991+0000 7f7791d9b700 1 --2- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 0x7f777c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.991+0000 7f77837fe700 20 client.14118.objecter dump_active .. 0 homeless 2024-02-18T00:23:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:40.995+0000 7f77837fe700 1 -- 172.21.15.59:0/2320888479 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7784018970 con 0x7f77940faf50 2024-02-18T00:23:41.194 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.192+0000 7f7791d9b700 1 -- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 msgr2=0x7f777c0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1656814063 2024-02-18T00:23:41.194 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.192+0000 7f7791d9b700 1 --2- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 0x7f777c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.256+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7794104aa0 con 0x7f77940faf50 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.257+0000 7f77837fe700 1 -- 172.21.15.59:0/2320888479 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f778401e070 con 0x7f77940faf50 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: { 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "available": true, 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-18T00:23:41.260 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: } 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.258+0000 7f7798e1d700 10 librados: watch_flush enter 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 10 librados: watch_flush exit 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 20 client.14118.objecter shutdown clearing up homeless session... 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 10 client.14118.objecter successfully canceled tick 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 msgr2=0x7f777c0364f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 --2- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 0x7f777c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 msgr2=0x7f77941013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 --2- 172.21.15.59:0/2320888479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77941013a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f778400bf60 tx=0x7f77840090f0).stop 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 shutdown_connections 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 --2- 172.21.15.59:0/2320888479 >> v2:172.21.15.59:6800/1656814063 conn(0x7f777c034030 0x7f777c0364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:23:41.263 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 --2- 172.21.15.59:0/2320888479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77940faf50 0x7f77941013a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 >> 172.21.15.59:0/2320888479 conn(0x7f77940f6670 msgr2=0x7f7794070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 shutdown_connections 2024-02-18T00:23:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 -- 172.21.15.59:0/2320888479 wait complete. 2024-02-18T00:23:41.264 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:41.259+0000 7f7798e1d700 1 librados: shutdown 2024-02-18T00:23:41.458 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:41.149+0000 7f152e167680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T00:23:41.458 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:41.233+0000 7f152e167680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T00:23:41.721 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:41.455+0000 7f152e167680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T00:23:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for the mgr to restart... 2024-02-18T00:23:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for mgr epoch 5... 2024-02-18T00:23:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:41.258311+0000 mon.a (mon.0) 29 : audit [DBG] from='client.? 172.21.15.59:0/2320888479' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-18T00:23:42.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd47a9fb700 1 Processor -- start 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd47a9fb700 1 -- start start 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd47a9fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd4740fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd47a9fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4740fc950 con 0x7fd4740fc040 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd473fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd4740fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd473fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd4740fc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44964/0 (socket says 172.21.15.59:44964) 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd473fff700 1 -- 172.21.15.59:0/3686213817 learned_addr learned my addr 172.21.15.59:0/3686213817 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.427+0000 7fd473fff700 1 -- 172.21.15.59:0/3686213817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740fca90 con 0x7fd4740fc040 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd473fff700 1 --2- 172.21.15.59:0/3686213817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd4740fc460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd464009730 tx=0x7fd4640090f0).ready entity=mon.0 client_cookie=185b83c3dee1bd4b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd472ffd700 1 -- 172.21.15.59:0/3686213817 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd46400b580 con 0x7fd4740fc040 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd472ffd700 1 -- 172.21.15.59:0/3686213817 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd46400bb90 con 0x7fd4740fc040 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd472ffd700 1 -- 172.21.15.59:0/3686213817 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd464011c30 con 0x7fd4740fc040 2024-02-18T00:23:42.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd47a9fb700 1 -- 172.21.15.59:0/3686213817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 msgr2=0x7fd4740fc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd47a9fb700 1 --2- 172.21.15.59:0/3686213817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd4740fc460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd464009730 tx=0x7fd4640090f0).stop 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd47a9fb700 1 -- 172.21.15.59:0/3686213817 shutdown_connections 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd47a9fb700 1 --2- 172.21.15.59:0/3686213817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd4740fc460 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd47a9fb700 1 -- 172.21.15.59:0/3686213817 >> 172.21.15.59:0/3686213817 conn(0x7fd4740f7860 msgr2=0x7fd4740f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd47a9fb700 1 -- 172.21.15.59:0/3686213817 shutdown_connections 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.428+0000 7fd47a9fb700 1 -- 172.21.15.59:0/3686213817 wait complete. 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.429+0000 7fd47a9fb700 1 librados: starting msgr at 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.429+0000 7fd47a9fb700 1 librados: starting objecter 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.429+0000 7fd47a9fb700 1 Processor -- start 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.429+0000 7fd47a9fb700 1 -- start start 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.429+0000 7fd47a9fb700 1 librados: setting wanted keys 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.429+0000 7fd47a9fb700 1 librados: calling monclient init 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd47a9fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd47410ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:42.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd47a9fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47410b220 con 0x7fd4740fc040 2024-02-18T00:23:42.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd473fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd47410ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:42.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd473fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd47410ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44974/0 (socket says 172.21.15.59:44974) 2024-02-18T00:23:42.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd473fff700 1 -- 172.21.15.59:0/2926714437 learned_addr learned my addr 172.21.15.59:0/2926714437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:42.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd473fff700 1 -- 172.21.15.59:0/2926714437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd464009370 con 0x7fd4740fc040 2024-02-18T00:23:42.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd473fff700 1 --2- 172.21.15.59:0/2926714437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd47410ad30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd464000c00 tx=0x7fd4640090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:42.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd4717fa700 10 client.?.objecter ms_handle_connect 0x7fd4740fc040 2024-02-18T00:23:42.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd4717fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.430+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd46401a430 con 0x7fd4740fc040 2024-02-18T00:23:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd46401aa40 con 0x7fd4740fc040 2024-02-18T00:23:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47410b420 con 0x7fd4740fc040 2024-02-18T00:23:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd464023b00 con 0x7fd4740fc040 2024-02-18T00:23:42.437 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd47a9fb700 10 client.14120.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47410c030 con 0x7fd4740fc040 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd47a9fb700 1 librados: init done 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 38582+0+0 (secure 0 0 0) 0x7fd46402c3f0 con 0x7fd4740fc040 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd4717fa700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 0x7fd45c0364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.431+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 --> v2:172.21.15.59:6800/1656814063 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd47405a310 con 0x7fd45c034030 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.432+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fd464045d10 con 0x7fd4740fc040 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.432+0000 7fd4717fa700 10 client.14120.objecter ms_dispatch 0x7fd474001550 osd_map(1..1 src has 1..1) v4 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.432+0000 7fd4717fa700 3 client.14120.objecter handle_osd_map got epochs [1,1] > 0 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.432+0000 7fd4717fa700 3 client.14120.objecter handle_osd_map decoding full epoch 1 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.432+0000 7fd4737fe700 1 -- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 msgr2=0x7fd45c0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1656814063 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.432+0000 7fd4717fa700 20 client.14120.objecter dump_active .. 0 homeless 2024-02-18T00:23:42.438 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.432+0000 7fd4737fe700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 0x7fd45c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:23:42.635 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.633+0000 7fd4737fe700 1 -- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 msgr2=0x7fd45c0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1656814063 2024-02-18T00:23:42.635 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:42.633+0000 7fd4737fe700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 0x7fd45c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-18T00:23:42.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:42.450+0000 7f152e167680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T00:23:42.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:42.583+0000 7f152e167680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:23:42.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:42.664+0000 7f152e167680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T00:23:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:43.034+0000 7fd4737fe700 1 -- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 msgr2=0x7fd45c0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1656814063 2024-02-18T00:23:43.036 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:43.034+0000 7fd4737fe700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 0x7fd45c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-18T00:23:43.249 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:42.756+0000 7f152e167680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T00:23:43.249 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:42.957+0000 7f152e167680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T00:23:43.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:43.247+0000 7f152e167680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T00:23:43.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:43.348+0000 7f152e167680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T00:23:43.837 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:43.835+0000 7fd4737fe700 1 -- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 msgr2=0x7fd45c0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/1656814063 2024-02-18T00:23:43.837 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:43.835+0000 7fd4737fe700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 0x7fd45c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-18T00:23:44.462 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:44.098+0000 7f152e167680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T00:23:44.463 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:44.182+0000 7f152e167680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T00:23:44.463 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:44.278+0000 7f152e167680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T00:23:44.463 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:44.460+0000 7f152e167680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T00:23:45.028 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:44.752+0000 7f152e167680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T00:23:45.029 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:44.887+0000 7f152e167680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T00:23:45.295 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:45.026+0000 7f152e167680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:23:45.295 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:45.207+0000 7f152e167680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T00:23:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:45.308+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 6) v1 ==== 38441+0+0 (secure 0 0 0) 0x7fd464023410 con 0x7fd4740fc040 2024-02-18T00:23:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:45.308+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 msgr2=0x7fd45c0364f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:23:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:45.308+0000 7fd4717fa700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 0x7fd45c0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:23:45.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:23:45.293+0000 7f152e167680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T00:23:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:45.307130+0000 mon.a (mon.0) 30 : cluster [INF] Active manager daemon y restarted 2024-02-18T00:23:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:45.307352+0000 mon.a (mon.0) 31 : cluster [INF] Activating manager daemon y 2024-02-18T00:23:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:45.310262+0000 mon.a (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:45.310411+0000 mon.a (mon.0) 33 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00313514s) 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.311139+0000 mon.a (mon.0) 34 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.311840+0000 mon.a (mon.0) 35 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.312438+0000 mon.a (mon.0) 36 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.312597+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.312726+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:45.327486+0000 mon.a (mon.0) 39 : cluster [INF] Manager daemon y is now available 2024-02-18T00:23:45.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.339783+0000 mon.a (mon.0) 40 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.309+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 7) v1 ==== 38476+0+0 (secure 0 0 0) 0x7fd46401fcf0 con 0x7fd4740fc040 2024-02-18T00:23:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.309+0000 7fd4717fa700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/3506000666 conn(0x7fd45c037910 0x7fd45c039d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.309+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 --> v2:172.21.15.59:6800/3506000666 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd47405a310 con 0x7fd45c037910 2024-02-18T00:23:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.310+0000 7fd4737fe700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/3506000666 conn(0x7fd45c037910 0x7fd45c039d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.310+0000 7fd4737fe700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/3506000666 conn(0x7fd45c037910 0x7fd45c039d00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fd468003850 tx=0x7fd468006ea0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.310+0000 7fd4717fa700 10 client.14120.objecter ms_handle_connect 0x7fd45c037910 2024-02-18T00:23:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.311+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7fd47405a310 con 0x7fd45c037910 2024-02-18T00:23:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.317+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 --> v2:172.21.15.59:6800/3506000666 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd47410b8b0 con 0x7fd45c037910 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd4717fa700 1 -- 172.21.15.59:0/2926714437 <== mgr.14116 v2:172.21.15.59:6800/3506000666 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd47410b8b0 con 0x7fd45c037910 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: { 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "initialized": true 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: } 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 10 librados: watch_flush enter 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 10 librados: watch_flush exit 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 20 client.14120.objecter shutdown clearing up homeless session... 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 10 client.14120.objecter successfully canceled tick 2024-02-18T00:23:46.322 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/3506000666 conn(0x7fd45c037910 msgr2=0x7fd45c039d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/3506000666 conn(0x7fd45c037910 0x7fd45c039d00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fd468003850 tx=0x7fd468006ea0).stop 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 msgr2=0x7fd47410ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 --2- 172.21.15.59:0/2926714437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd47410ad30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fd464000c00 tx=0x7fd4640090f0).stop 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 shutdown_connections 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 --2- 172.21.15.59:0/2926714437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4740fc040 0x7fd47410ad30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/3506000666 conn(0x7fd45c037910 0x7fd45c039d00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 --2- 172.21.15.59:0/2926714437 >> v2:172.21.15.59:6800/1656814063 conn(0x7fd45c034030 0x7fd45c0364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 >> 172.21.15.59:0/2926714437 conn(0x7fd4740f7860 msgr2=0x7fd4740f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 shutdown_connections 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 -- 172.21.15.59:0/2926714437 wait complete. 2024-02-18T00:23:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:46.318+0000 7fd47a9fb700 1 librados: shutdown 2024-02-18T00:23:46.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:23:45.337183+0000 mgr.y (mgr.14116) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-02-18T00:23:46.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.430308+0000 mon.a (mon.0) 41 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:46.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.431383+0000 mon.a (mon.0) 42 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:23:46.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.433004+0000 mon.a (mon.0) 43 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:23:46.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.434744+0000 mon.a (mon.0) 44 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:23:46.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.449036+0000 mon.a (mon.0) 45 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T00:23:46.689 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:45.451745+0000 mon.a (mon.0) 46 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T00:23:46.689 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:46.313035+0000 mon.a (mon.0) 47 : cluster [DBG] mgrmap e7: y(active, since 1.00576s) 2024-02-18T00:23:46.991 INFO:teuthology.orchestra.run.smithi059.stderr:mgr epoch 5 is available 2024-02-18T00:23:46.992 INFO:teuthology.orchestra.run.smithi059.stderr:Setting orchestrator backend to cephadm... 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.538+0000 7f1d7b75b700 1 Processor -- start 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.538+0000 7f1d7b75b700 1 -- start start 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.538+0000 7f1d7b75b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.538+0000 7f1d7b75b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740ff1c0 con 0x7f1d74070c80 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d794f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d794f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45024/0 (socket says 172.21.15.59:45024) 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d794f7700 1 -- 172.21.15.59:0/1952296561 learned_addr learned my addr 172.21.15.59:0/1952296561 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d794f7700 1 -- 172.21.15.59:0/1952296561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d740ff300 con 0x7f1d74070c80 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d794f7700 1 --2- 172.21.15.59:0/1952296561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fec40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1d64009730 tx=0x7f1d640090f0).ready entity=mon.0 client_cookie=8fa0c5ca49b22ae8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d6bfff700 1 -- 172.21.15.59:0/1952296561 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d6400b580 con 0x7f1d74070c80 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d6bfff700 1 -- 172.21.15.59:0/1952296561 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d6400bb90 con 0x7f1d74070c80 2024-02-18T00:23:47.563 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.539+0000 7f1d6bfff700 1 -- 172.21.15.59:0/1952296561 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d64011c30 con 0x7f1d74070c80 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 -- 172.21.15.59:0/1952296561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 msgr2=0x7f1d740fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 --2- 172.21.15.59:0/1952296561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fec40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1d64009730 tx=0x7f1d640090f0).stop 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 -- 172.21.15.59:0/1952296561 shutdown_connections 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 --2- 172.21.15.59:0/1952296561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fec40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 -- 172.21.15.59:0/1952296561 >> 172.21.15.59:0/1952296561 conn(0x7f1d740f6660 msgr2=0x7f1d740f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 -- 172.21.15.59:0/1952296561 shutdown_connections 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 -- 172.21.15.59:0/1952296561 wait complete. 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 librados: starting msgr at 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 librados: starting objecter 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 Processor -- start 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 -- start start 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 librados: setting wanted keys 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 librados: calling monclient init 2024-02-18T00:23:47.564 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.540+0000 7f1d7b75b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740fab40 con 0x7f1d74070c80 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d794f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d794f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45034/0 (socket says 172.21.15.59:45034) 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d794f7700 1 -- 172.21.15.59:0/941901629 learned_addr learned my addr 172.21.15.59:0/941901629 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d794f7700 1 -- 172.21.15.59:0/941901629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64009370 con 0x7f1d74070c80 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d794f7700 1 --2- 172.21.15.59:0/941901629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fc6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1d64006940 tx=0x7f1d640090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d6a7fc700 10 client.?.objecter ms_handle_connect 0x7f1d74070c80 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d6a7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d6a7fc700 1 -- 172.21.15.59:0/941901629 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d6401a430 con 0x7f1d74070c80 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d6a7fc700 1 -- 172.21.15.59:0/941901629 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1d6401aa40 con 0x7f1d74070c80 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d740face0 con 0x7f1d74070c80 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d6a7fc700 1 -- 172.21.15.59:0/941901629 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d64023b90 con 0x7f1d74070c80 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d7b75b700 10 client.14126.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:47.565 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d740fb170 con 0x7f1d74070c80 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.541+0000 7f1d7b75b700 1 librados: init done 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 1 -- 172.21.15.59:0/941901629 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7f1d64023cf0 con 0x7f1d74070c80 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 1 --2- 172.21.15.59:0/941901629 >> v2:172.21.15.59:6800/3506000666 conn(0x7f1d60033fe0 0x7f1d600364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7405a310 con 0x7f1d74070c80 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 1 -- 172.21.15.59:0/941901629 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7f1d64044770 con 0x7f1d74070c80 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 10 client.14126.objecter ms_dispatch 0x7f1d74001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 3 client.14126.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 3 client.14126.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 20 client.14126.objecter dump_active .. 0 homeless 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d78cf6700 1 --2- 172.21.15.59:0/941901629 >> v2:172.21.15.59:6800/3506000666 conn(0x7f1d60033fe0 0x7f1d600364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d6a7fc700 10 client.14126.objecter ms_handle_connect 0x7f1d60033fe0 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.542+0000 7f1d78cf6700 1 --2- 172.21.15.59:0/941901629 >> v2:172.21.15.59:6800/3506000666 conn(0x7f1d60033fe0 0x7f1d600364a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f1d70009780 tx=0x7f1d70006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:47.566 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.547+0000 7f1d6a7fc700 1 -- 172.21.15.59:0/941901629 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1d6401e070 con 0x7f1d74070c80 2024-02-18T00:23:47.737 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.734+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 --> v2:172.21.15.59:6800/3506000666 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d740fb3b0 con 0x7f1d60033fe0 2024-02-18T00:23:47.742 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.740+0000 7f1d6a7fc700 1 -- 172.21.15.59:0/941901629 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1d740fb3b0 con 0x7f1d60033fe0 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 10 librados: watch_flush enter 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 10 librados: watch_flush exit 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 20 client.14126.objecter shutdown clearing up homeless session... 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 10 client.14126.objecter successfully canceled tick 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 >> v2:172.21.15.59:6800/3506000666 conn(0x7f1d60033fe0 msgr2=0x7f1d600364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 1 --2- 172.21.15.59:0/941901629 >> v2:172.21.15.59:6800/3506000666 conn(0x7f1d60033fe0 0x7f1d600364a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f1d70009780 tx=0x7f1d70006cc0).stop 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 msgr2=0x7f1d740fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.741+0000 7f1d7b75b700 1 --2- 172.21.15.59:0/941901629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fc6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f1d64006940 tx=0x7f1d640090f0).stop 2024-02-18T00:23:47.745 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.742+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 shutdown_connections 2024-02-18T00:23:47.746 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.742+0000 7f1d7b75b700 1 --2- 172.21.15.59:0/941901629 >> v2:172.21.15.59:6800/3506000666 conn(0x7f1d60033fe0 0x7f1d600364a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:47.746 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.742+0000 7f1d7b75b700 1 --2- 172.21.15.59:0/941901629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d74070c80 0x7f1d740fc6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:47.746 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.742+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 >> 172.21.15.59:0/941901629 conn(0x7f1d740f6660 msgr2=0x7f1d740f7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:47.746 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.742+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 shutdown_connections 2024-02-18T00:23:47.746 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.742+0000 7f1d7b75b700 1 -- 172.21.15.59:0/941901629 wait complete. 2024-02-18T00:23:47.746 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:47.742+0000 7f1d7b75b700 1 librados: shutdown 2024-02-18T00:23:47.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:46.312180+0000 mgr.y (mgr.14116) 2 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-18T00:23:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:46.319032+0000 mgr.y (mgr.14116) 3 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-18T00:23:48.680 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:23:47.434993+0000 mon.a (mon.0) 48 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-02-18T00:23:48.680 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:47.736628+0000 mgr.y (mgr.14116) 4 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:23:48.680 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:47.740658+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:48.680 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:47.760303+0000 mon.a (mon.0) 50 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:23:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.977+0000 7fc6b8222700 1 Processor -- start 2024-02-18T00:23:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.977+0000 7fc6b8222700 1 -- start start 2024-02-18T00:23:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.977+0000 7fc6b8222700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b00fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.977+0000 7fc6b8222700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00fb8f0 con 0x7fc6b00faf50 2024-02-18T00:23:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.977+0000 7fc6b5fbe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b00fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.977+0000 7fc6b5fbe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b00fb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45042/0 (socket says 172.21.15.59:45042) 2024-02-18T00:23:49.003 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.977+0000 7fc6b5fbe700 1 -- 172.21.15.59:0/878153909 learned_addr learned my addr 172.21.15.59:0/878153909 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.978+0000 7fc6b5fbe700 1 -- 172.21.15.59:0/878153909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00fba30 con 0x7fc6b00faf50 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.978+0000 7fc6b5fbe700 1 --2- 172.21.15.59:0/878153909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b00fb370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc6ac009730 tx=0x7fc6ac0090f0).ready entity=mon.0 client_cookie=2e23c81c25bedeb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.978+0000 7fc6b4fbc700 1 -- 172.21.15.59:0/878153909 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac00b580 con 0x7fc6b00faf50 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.978+0000 7fc6b4fbc700 1 -- 172.21.15.59:0/878153909 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6ac00bb90 con 0x7fc6b00faf50 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.978+0000 7fc6b4fbc700 1 -- 172.21.15.59:0/878153909 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac011c30 con 0x7fc6b00faf50 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.978+0000 7fc6b8222700 1 -- 172.21.15.59:0/878153909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 msgr2=0x7fc6b00fb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.978+0000 7fc6b8222700 1 --2- 172.21.15.59:0/878153909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b00fb370 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fc6ac009730 tx=0x7fc6ac0090f0).stop 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 -- 172.21.15.59:0/878153909 shutdown_connections 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 --2- 172.21.15.59:0/878153909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b00fb370 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 -- 172.21.15.59:0/878153909 >> 172.21.15.59:0/878153909 conn(0x7fc6b00f6670 msgr2=0x7fc6b00f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 -- 172.21.15.59:0/878153909 shutdown_connections 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 -- 172.21.15.59:0/878153909 wait complete. 2024-02-18T00:23:49.004 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 librados: starting msgr at 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 librados: starting objecter 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 Processor -- start 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 -- start start 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 librados: setting wanted keys 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 librados: calling monclient init 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b01013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.979+0000 7fc6b8222700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b00ff820 con 0x7fc6b00faf50 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b5fbe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b01013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b5fbe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b01013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45050/0 (socket says 172.21.15.59:45050) 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b5fbe700 1 -- 172.21.15.59:0/3797391865 learned_addr learned my addr 172.21.15.59:0/3797391865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b5fbe700 1 -- 172.21.15.59:0/3797391865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac009370 con 0x7fc6b00faf50 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b5fbe700 1 --2- 172.21.15.59:0/3797391865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b01013a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc6ac00bf60 tx=0x7fc6ac0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6a6ffd700 10 client.?.objecter ms_handle_connect 0x7fc6b00faf50 2024-02-18T00:23:49.005 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6a6ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6a6ffd700 1 -- 172.21.15.59:0/3797391865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac011bf0 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6a6ffd700 1 -- 172.21.15.59:0/3797391865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6ac01aaf0 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b00ff9c0 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6a6ffd700 1 -- 172.21.15.59:0/3797391865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6ac0229c0 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b8222700 10 client.14128.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b00ffe50 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.980+0000 7fc6b8222700 1 librados: init done 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 1 -- 172.21.15.59:0/3797391865 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7fc6ac01a640 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b005a310 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 1 --2- 172.21.15.59:0/3797391865 >> v2:172.21.15.59:6800/3506000666 conn(0x7fc69c034030 0x7fc69c0364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 1 -- 172.21.15.59:0/3797391865 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7fc6ac044840 con 0x7fc6b00faf50 2024-02-18T00:23:49.006 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 10 client.14128.objecter ms_dispatch 0x7fc6b0001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:23:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 3 client.14128.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:23:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 3 client.14128.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:23:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 20 client.14128.objecter dump_active .. 0 homeless 2024-02-18T00:23:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6b57bd700 1 --2- 172.21.15.59:0/3797391865 >> v2:172.21.15.59:6800/3506000666 conn(0x7fc69c034030 0x7fc69c0364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6a6ffd700 10 client.14128.objecter ms_handle_connect 0x7fc69c034030 2024-02-18T00:23:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.981+0000 7fc6b57bd700 1 --2- 172.21.15.59:0/3797391865 >> v2:172.21.15.59:6800/3506000666 conn(0x7fc69c034030 0x7fc69c0364f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc6a0009780 tx=0x7fc6a0006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:49.007 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:48.985+0000 7fc6a6ffd700 1 -- 172.21.15.59:0/3797391865 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc6ac01a8f0 con 0x7fc6b00faf50 2024-02-18T00:23:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.161+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 --> v2:172.21.15.59:6800/3506000666 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6b0104b10 con 0x7fc69c034030 2024-02-18T00:23:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.162+0000 7fc6a6ffd700 1 -- 172.21.15.59:0/3797391865 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc6b0104b10 con 0x7fc69c034030 2024-02-18T00:23:49.164 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: value unchanged 2024-02-18T00:23:49.167 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 10 librados: watch_flush enter 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 10 librados: watch_flush exit 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 20 client.14128.objecter shutdown clearing up homeless session... 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 10 client.14128.objecter successfully canceled tick 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 >> v2:172.21.15.59:6800/3506000666 conn(0x7fc69c034030 msgr2=0x7fc69c0364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 --2- 172.21.15.59:0/3797391865 >> v2:172.21.15.59:6800/3506000666 conn(0x7fc69c034030 0x7fc69c0364f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fc6a0009780 tx=0x7fc6a0006cc0).stop 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 msgr2=0x7fc6b01013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 --2- 172.21.15.59:0/3797391865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b01013a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc6ac00bf60 tx=0x7fc6ac0090f0).stop 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 shutdown_connections 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 --2- 172.21.15.59:0/3797391865 >> v2:172.21.15.59:6800/3506000666 conn(0x7fc69c034030 0x7fc69c0364f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 --2- 172.21.15.59:0/3797391865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b00faf50 0x7fc6b01013a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:49.168 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 >> 172.21.15.59:0/3797391865 conn(0x7fc6b00f6670 msgr2=0x7fc6b0070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 shutdown_connections 2024-02-18T00:23:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 -- 172.21.15.59:0/3797391865 wait complete. 2024-02-18T00:23:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:49.164+0000 7fc6b8222700 1 librados: shutdown 2024-02-18T00:23:49.854 INFO:teuthology.orchestra.run.smithi059.stderr:Generating ssh key... 2024-02-18T00:23:50.080 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:49.163460+0000 mgr.y (mgr.14116) 5 : audit [DBG] from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:23:50.430 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb751d7700 1 Processor -- start 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb751d7700 1 -- start start 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb751d7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb700fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb751d7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700ff1d0 con 0x7ffb70070c80 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb6ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb700fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb6ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb700fec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52704/0 (socket says 172.21.15.59:52704) 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb6ed9d700 1 -- 172.21.15.59:0/2176593318 learned_addr learned my addr 172.21.15.59:0/2176593318 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.424+0000 7ffb6ed9d700 1 -- 172.21.15.59:0/2176593318 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700ff310 con 0x7ffb70070c80 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb6ed9d700 1 --2- 172.21.15.59:0/2176593318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb700fec50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffb60009730 tx=0x7ffb600090f0).ready entity=mon.0 client_cookie=cf54939935fd5a62 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb6dd9b700 1 -- 172.21.15.59:0/2176593318 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb6000b580 con 0x7ffb70070c80 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb6dd9b700 1 -- 172.21.15.59:0/2176593318 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb6000bb90 con 0x7ffb70070c80 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb6dd9b700 1 -- 172.21.15.59:0/2176593318 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb60011c30 con 0x7ffb70070c80 2024-02-18T00:23:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb751d7700 1 -- 172.21.15.59:0/2176593318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 msgr2=0x7ffb700fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb751d7700 1 --2- 172.21.15.59:0/2176593318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb700fec50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffb60009730 tx=0x7ffb600090f0).stop 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb751d7700 1 -- 172.21.15.59:0/2176593318 shutdown_connections 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb751d7700 1 --2- 172.21.15.59:0/2176593318 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb700fec50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb751d7700 1 -- 172.21.15.59:0/2176593318 >> 172.21.15.59:0/2176593318 conn(0x7ffb700f6670 msgr2=0x7ffb700f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb751d7700 1 -- 172.21.15.59:0/2176593318 shutdown_connections 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.425+0000 7ffb751d7700 1 -- 172.21.15.59:0/2176593318 wait complete. 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.426+0000 7ffb751d7700 1 librados: starting msgr at 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.426+0000 7ffb751d7700 1 librados: starting objecter 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.426+0000 7ffb751d7700 1 Processor -- start 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.426+0000 7ffb751d7700 1 -- start start 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.426+0000 7ffb751d7700 1 librados: setting wanted keys 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.426+0000 7ffb751d7700 1 librados: calling monclient init 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb751d7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb7010ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb751d7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7010b270 con 0x7ffb70070c80 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb6ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb7010ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb6ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb7010ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52712/0 (socket says 172.21.15.59:52712) 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb6ed9d700 1 -- 172.21.15.59:0/3916442748 learned_addr learned my addr 172.21.15.59:0/3916442748 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb6ed9d700 1 -- 172.21.15.59:0/3916442748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60009370 con 0x7ffb70070c80 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb6ed9d700 1 --2- 172.21.15.59:0/3916442748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb7010ad80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ffb60006940 tx=0x7ffb600090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb57fff700 10 client.?.objecter ms_handle_connect 0x7ffb70070c80 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.427+0000 7ffb57fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb57fff700 1 -- 172.21.15.59:0/3916442748 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb6001a430 con 0x7ffb70070c80 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7010b470 con 0x7ffb70070c80 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb57fff700 1 -- 172.21.15.59:0/3916442748 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffb6001aa40 con 0x7ffb70070c80 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb57fff700 1 -- 172.21.15.59:0/3916442748 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb60023b90 con 0x7ffb70070c80 2024-02-18T00:23:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb751d7700 10 client.14130.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:50.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7010c080 con 0x7ffb70070c80 2024-02-18T00:23:50.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb751d7700 1 librados: init done 2024-02-18T00:23:50.434 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.428+0000 7ffb57fff700 1 -- 172.21.15.59:0/3916442748 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7ffb60023cf0 con 0x7ffb70070c80 2024-02-18T00:23:50.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb57fff700 1 --2- 172.21.15.59:0/3916442748 >> v2:172.21.15.59:6800/3506000666 conn(0x7ffb58033fe0 0x7ffb580364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:50.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb57fff700 1 -- 172.21.15.59:0/3916442748 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7ffb6001a630 con 0x7ffb70070c80 2024-02-18T00:23:50.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb57fff700 10 client.14130.objecter ms_dispatch 0x7ffb70001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:23:50.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb57fff700 3 client.14130.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:23:50.435 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb57fff700 3 client.14130.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:23:50.436 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb57fff700 20 client.14130.objecter dump_active .. 0 homeless 2024-02-18T00:23:50.436 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb6e59c700 1 --2- 172.21.15.59:0/3916442748 >> v2:172.21.15.59:6800/3506000666 conn(0x7ffb58033fe0 0x7ffb580364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:50.436 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb57fff700 10 client.14130.objecter ms_handle_connect 0x7ffb58033fe0 2024-02-18T00:23:50.436 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb6e59c700 1 --2- 172.21.15.59:0/3916442748 >> v2:172.21.15.59:6800/3506000666 conn(0x7ffb58033fe0 0x7ffb580364a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7ffb64009780 tx=0x7ffb64006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:50.436 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.429+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7010b900 con 0x7ffb70070c80 2024-02-18T00:23:50.436 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.434+0000 7ffb57fff700 1 -- 172.21.15.59:0/3916442748 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ffb7010b900 con 0x7ffb70070c80 2024-02-18T00:23:50.610 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:50.608+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 --> v2:172.21.15.59:6800/3506000666 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb7010b900 con 0x7ffb58033fe0 2024-02-18T00:23:51.034 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.032+0000 7ffb57fff700 1 -- 172.21.15.59:0/3916442748 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffb7010b900 con 0x7ffb58033fe0 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 10 librados: watch_flush enter 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 10 librados: watch_flush exit 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 20 client.14130.objecter shutdown clearing up homeless session... 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 10 client.14130.objecter successfully canceled tick 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 >> v2:172.21.15.59:6800/3506000666 conn(0x7ffb58033fe0 msgr2=0x7ffb580364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 --2- 172.21.15.59:0/3916442748 >> v2:172.21.15.59:6800/3506000666 conn(0x7ffb58033fe0 0x7ffb580364a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7ffb64009780 tx=0x7ffb64006cc0).stop 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 msgr2=0x7ffb7010ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 --2- 172.21.15.59:0/3916442748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb7010ad80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ffb60006940 tx=0x7ffb600090f0).stop 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 shutdown_connections 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 --2- 172.21.15.59:0/3916442748 >> v2:172.21.15.59:6800/3506000666 conn(0x7ffb58033fe0 0x7ffb580364a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:51.037 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 --2- 172.21.15.59:0/3916442748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb70070c80 0x7ffb7010ad80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:51.038 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.033+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 >> 172.21.15.59:0/3916442748 conn(0x7ffb700f6670 msgr2=0x7ffb700f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:51.038 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.034+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 shutdown_connections 2024-02-18T00:23:51.038 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.034+0000 7ffb751d7700 1 -- 172.21.15.59:0/3916442748 wait complete. 2024-02-18T00:23:51.038 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:51.034+0000 7ffb751d7700 1 librados: shutdown 2024-02-18T00:23:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:50.609733+0000 mgr.y (mgr.14116) 6 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:23:50.609935+0000 mgr.y (mgr.14116) 7 : cephadm [INF] Generating ssh key... 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: Generating public/private rsa key pair. 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: Your identification has been saved in /tmp/tmpzew2vb44/key. 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: Your public key has been saved in /tmp/tmpzew2vb44/key.pub. 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: The key fingerprint is: 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: SHA256:CSY9FBc+KtKhOTxPjaxEzgelsobLpu0CFnlIU4suPvY ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: The key's randomart image is: 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: +---[RSA 3072]----+ 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: | ... o.o. | 2024-02-18T00:23:51.062 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: | o.o.o o | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: |o.B.o = o | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: |o@ B * + o | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: |+o& B o S | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: |*+ O . | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: |+*. . | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: |=.o | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: |.ooE | 2024-02-18T00:23:51.063 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: +----[SHA256]-----+ 2024-02-18T00:23:52.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:51.026804+0000 mon.a (mon.0) 51 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:52.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:51.032267+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:52.332 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f433a947700 1 Processor -- start 2024-02-18T00:23:52.332 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f433a947700 1 -- start start 2024-02-18T00:23:52.332 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f433a947700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f43340fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f433a947700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340fc950 con 0x7f43340fc040 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f4333fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f43340fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f4333fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f43340fc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52720/0 (socket says 172.21.15.59:52720) 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f4333fff700 1 -- 172.21.15.59:0/351635346 learned_addr learned my addr 172.21.15.59:0/351635346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.306+0000 7f4333fff700 1 -- 172.21.15.59:0/351635346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43340fca90 con 0x7f43340fc040 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f4333fff700 1 --2- 172.21.15.59:0/351635346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f43340fc460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4324009730 tx=0x7f43240090f0).ready entity=mon.0 client_cookie=bad6a9ab5ae29aea server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f4332ffd700 1 -- 172.21.15.59:0/351635346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f432400b580 con 0x7f43340fc040 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f4332ffd700 1 -- 172.21.15.59:0/351635346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f432400bb90 con 0x7f43340fc040 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f4332ffd700 1 -- 172.21.15.59:0/351635346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4324011c30 con 0x7f43340fc040 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f433a947700 1 -- 172.21.15.59:0/351635346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 msgr2=0x7f43340fc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f433a947700 1 --2- 172.21.15.59:0/351635346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f43340fc460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f4324009730 tx=0x7f43240090f0).stop 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f433a947700 1 -- 172.21.15.59:0/351635346 shutdown_connections 2024-02-18T00:23:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f433a947700 1 --2- 172.21.15.59:0/351635346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f43340fc460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f433a947700 1 -- 172.21.15.59:0/351635346 >> 172.21.15.59:0/351635346 conn(0x7f43340f7860 msgr2=0x7f43340f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f433a947700 1 -- 172.21.15.59:0/351635346 shutdown_connections 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.307+0000 7f433a947700 1 -- 172.21.15.59:0/351635346 wait complete. 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 librados: starting msgr at 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 librados: starting objecter 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 Processor -- start 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 -- start start 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 librados: setting wanted keys 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 librados: calling monclient init 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f4334072d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f433a947700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340711e0 con 0x7f43340fc040 2024-02-18T00:23:52.334 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f4333fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f4334072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f4333fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f4334072d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52724/0 (socket says 172.21.15.59:52724) 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.308+0000 7f4333fff700 1 -- 172.21.15.59:0/330469104 learned_addr learned my addr 172.21.15.59:0/330469104 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f4333fff700 1 -- 172.21.15.59:0/330469104 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324009370 con 0x7f43340fc040 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f4333fff700 1 --2- 172.21.15.59:0/330469104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f4334072d60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f432400bf30 tx=0x7f43240090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f43317fa700 10 client.?.objecter ms_handle_connect 0x7f43340fc040 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f43317fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f43317fa700 1 -- 172.21.15.59:0/330469104 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f432401a640 con 0x7f43340fc040 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f43317fa700 1 -- 172.21.15.59:0/330469104 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f432401ac50 con 0x7f43340fc040 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f43317fa700 1 -- 172.21.15.59:0/330469104 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4324023ae0 con 0x7f43340fc040 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4334071380 con 0x7f43340fc040 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f433a947700 10 client.14132.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4334071810 con 0x7f43340fc040 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.309+0000 7f433a947700 1 librados: init done 2024-02-18T00:23:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 1 -- 172.21.15.59:0/330469104 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7f4324023c40 con 0x7f43340fc040 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433405a310 con 0x7f43340fc040 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 1 --2- 172.21.15.59:0/330469104 >> v2:172.21.15.59:6800/3506000666 conn(0x7f431c034030 0x7f431c0364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 1 -- 172.21.15.59:0/330469104 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7f4324044ad0 con 0x7f43340fc040 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 10 client.14132.objecter ms_dispatch 0x7f4334001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 3 client.14132.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 3 client.14132.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 20 client.14132.objecter dump_active .. 0 homeless 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43337fe700 1 --2- 172.21.15.59:0/330469104 >> v2:172.21.15.59:6800/3506000666 conn(0x7f431c034030 0x7f431c0364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43317fa700 10 client.14132.objecter ms_handle_connect 0x7f431c034030 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.310+0000 7f43337fe700 1 --2- 172.21.15.59:0/330469104 >> v2:172.21.15.59:6800/3506000666 conn(0x7f431c034030 0x7f431c0364f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4328009780 tx=0x7f4328006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.315+0000 7f43317fa700 1 -- 172.21.15.59:0/330469104 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4324021320 con 0x7f43340fc040 2024-02-18T00:23:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.494+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 --> v2:172.21.15.59:6800/3506000666 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4334071a50 con 0x7f431c034030 2024-02-18T00:23:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.495+0000 7f43317fa700 1 -- 172.21.15.59:0/330469104 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4334071a50 con 0x7f431c034030 2024-02-18T00:23:52.497 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYK8X+slKTMtlz0QR++yTpV0cbsRmfdctrA9qSxLDP4IuIqg5efnNhND61qakklIEJBgnO2LT/JXCCNfllDI37gjIEZzO0tpmhipXXo4rsdGopIq19ANn7sAAYd2xehW4iAYx22BeDhX0lZ+ObMTzH4u4NUgQnXM0aHi9WVyNi9Mc/nfp+N0v1ES/gHjBVEWtTIVmJlrB7DDrheW6ChqmTe1N2YR0HSPa38Hl4YRtauZuhRe3V8aRit4XTHzyuGSv9Cc5LEmm9ixHiK9Pm6af64b6jDI25GjPL0dAgME5tNL6lFGRTkTFccWmu1X0GqueS+GL0SehaG4yZK+X2KhU3nImkJ7OHEggmJeh4CCeLTfJSYt7/7nauyDxhyKCU7Rj8w3gGa6Vixb37owyUs06SeJZWcwny2HlJRqaUuOSPLKD3TQbKNXMmF3uns1FdP3MShzy6GaAO0dlTstuQUXTl8LY7dHVKadPr513bt7reFcz+ZISJZpZ4pmRHjV0v93c= ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:23:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 10 librados: watch_flush enter 2024-02-18T00:23:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 10 librados: watch_flush exit 2024-02-18T00:23:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 20 client.14132.objecter shutdown clearing up homeless session... 2024-02-18T00:23:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 10 client.14132.objecter successfully canceled tick 2024-02-18T00:23:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 >> v2:172.21.15.59:6800/3506000666 conn(0x7f431c034030 msgr2=0x7f431c0364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 --2- 172.21.15.59:0/330469104 >> v2:172.21.15.59:6800/3506000666 conn(0x7f431c034030 0x7f431c0364f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4328009780 tx=0x7f4328006cc0).stop 2024-02-18T00:23:52.500 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 msgr2=0x7f4334072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 --2- 172.21.15.59:0/330469104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f4334072d60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f432400bf30 tx=0x7f43240090f0).stop 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 shutdown_connections 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 --2- 172.21.15.59:0/330469104 >> v2:172.21.15.59:6800/3506000666 conn(0x7f431c034030 0x7f431c0364f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 --2- 172.21.15.59:0/330469104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43340fc040 0x7f4334072d60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 >> 172.21.15.59:0/330469104 conn(0x7f43340f7860 msgr2=0x7f43340f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 shutdown_connections 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 -- 172.21.15.59:0/330469104 wait complete. 2024-02-18T00:23:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:52.497+0000 7f433a947700 1 librados: shutdown 2024-02-18T00:23:53.226 INFO:teuthology.orchestra.run.smithi059.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-18T00:23:53.226 INFO:teuthology.orchestra.run.smithi059.stderr:Adding key to root@localhost authorized_keys... 2024-02-18T00:23:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:Adding host smithi059... 2024-02-18T00:23:53.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:52.496304+0000 mgr.y (mgr.14116) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.798+0000 7fdf5baee700 1 Processor -- start 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.798+0000 7fdf5baee700 1 -- start start 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf5baee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf540fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf5baee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540fc950 con 0x7fdf540fc040 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf5988a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf540fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf5988a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf540fc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52732/0 (socket says 172.21.15.59:52732) 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf5988a700 1 -- 172.21.15.59:0/3688659005 learned_addr learned my addr 172.21.15.59:0/3688659005 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf5988a700 1 -- 172.21.15.59:0/3688659005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540fca90 con 0x7fdf540fc040 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf5988a700 1 --2- 172.21.15.59:0/3688659005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf540fc460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdf44009730 tx=0x7fdf440090f0).ready entity=mon.0 client_cookie=225bb22bdaae699d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.799+0000 7fdf58888700 1 -- 172.21.15.59:0/3688659005 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf4400b580 con 0x7fdf540fc040 2024-02-18T00:23:53.802 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf58888700 1 -- 172.21.15.59:0/3688659005 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf4400bb90 con 0x7fdf540fc040 2024-02-18T00:23:53.803 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf58888700 1 -- 172.21.15.59:0/3688659005 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf44011c30 con 0x7fdf540fc040 2024-02-18T00:23:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf5baee700 1 -- 172.21.15.59:0/3688659005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 msgr2=0x7fdf540fc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf5baee700 1 --2- 172.21.15.59:0/3688659005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf540fc460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdf44009730 tx=0x7fdf440090f0).stop 2024-02-18T00:23:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf5baee700 1 -- 172.21.15.59:0/3688659005 shutdown_connections 2024-02-18T00:23:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf5baee700 1 --2- 172.21.15.59:0/3688659005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf540fc460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf5baee700 1 -- 172.21.15.59:0/3688659005 >> 172.21.15.59:0/3688659005 conn(0x7fdf540f7800 msgr2=0x7fdf540f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:53.806 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf5baee700 1 -- 172.21.15.59:0/3688659005 shutdown_connections 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.800+0000 7fdf5baee700 1 -- 172.21.15.59:0/3688659005 wait complete. 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 librados: starting msgr at 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 librados: starting objecter 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 Processor -- start 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 -- start start 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 librados: setting wanted keys 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 librados: calling monclient init 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf54072d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5baee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540711e0 con 0x7fdf540fc040 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5988a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf54072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5988a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf54072d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52746/0 (socket says 172.21.15.59:52746) 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5988a700 1 -- 172.21.15.59:0/1875551689 learned_addr learned my addr 172.21.15.59:0/1875551689 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.801+0000 7fdf5988a700 1 -- 172.21.15.59:0/1875551689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44009370 con 0x7fdf540fc040 2024-02-18T00:23:53.807 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf5988a700 1 --2- 172.21.15.59:0/1875551689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf54072d60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fdf4400bf30 tx=0x7fdf440090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf4affd700 10 client.?.objecter ms_handle_connect 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf4affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf4affd700 1 -- 172.21.15.59:0/1875551689 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf4401a640 con 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf4affd700 1 -- 172.21.15.59:0/1875551689 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf4401ac50 con 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf54071380 con 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf4affd700 1 -- 172.21.15.59:0/1875551689 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf44023ae0 con 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf5baee700 10 client.14134.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf54071810 con 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.802+0000 7fdf5baee700 1 librados: init done 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 1 -- 172.21.15.59:0/1875551689 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7fdf44023c40 con 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5405a310 con 0x7fdf540fc040 2024-02-18T00:23:53.808 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 1 --2- 172.21.15.59:0/1875551689 >> v2:172.21.15.59:6800/3506000666 conn(0x7fdf40034030 0x7fdf400364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 1 -- 172.21.15.59:0/1875551689 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7fdf44044800 con 0x7fdf540fc040 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 10 client.14134.objecter ms_dispatch 0x7fdf54001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 3 client.14134.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 3 client.14134.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 20 client.14134.objecter dump_active .. 0 homeless 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf59089700 1 --2- 172.21.15.59:0/1875551689 >> v2:172.21.15.59:6800/3506000666 conn(0x7fdf40034030 0x7fdf400364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf59089700 1 --2- 172.21.15.59:0/1875551689 >> v2:172.21.15.59:6800/3506000666 conn(0x7fdf40034030 0x7fdf400364f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdf50009780 tx=0x7fdf50006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.803+0000 7fdf4affd700 10 client.14134.objecter ms_handle_connect 0x7fdf40034030 2024-02-18T00:23:53.809 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.807+0000 7fdf4affd700 1 -- 172.21.15.59:0/1875551689 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdf44023520 con 0x7fdf540fc040 2024-02-18T00:23:53.987 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:53.985+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 --> v2:172.21.15.59:6800/3506000666 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf54071a50 con 0x7fdf40034030 2024-02-18T00:23:54.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:23:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [79B blob data] 2024-02-18T00:23:54.641 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: Added host 'smithi059' with addr '172.21.15.59' 2024-02-18T00:23:54.641 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.637+0000 7fdf4affd700 1 -- 172.21.15.59:0/1875551689 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fdf54071a50 con 0x7fdf40034030 2024-02-18T00:23:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 10 librados: watch_flush enter 2024-02-18T00:23:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 10 librados: watch_flush exit 2024-02-18T00:23:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 20 client.14134.objecter shutdown clearing up homeless session... 2024-02-18T00:23:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 10 client.14134.objecter successfully canceled tick 2024-02-18T00:23:54.643 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 >> v2:172.21.15.59:6800/3506000666 conn(0x7fdf40034030 msgr2=0x7fdf400364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 --2- 172.21.15.59:0/1875551689 >> v2:172.21.15.59:6800/3506000666 conn(0x7fdf40034030 0x7fdf400364f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fdf50009780 tx=0x7fdf50006cc0).stop 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 msgr2=0x7fdf54072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 --2- 172.21.15.59:0/1875551689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf54072d60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fdf4400bf30 tx=0x7fdf440090f0).stop 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 shutdown_connections 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 --2- 172.21.15.59:0/1875551689 >> v2:172.21.15.59:6800/3506000666 conn(0x7fdf40034030 0x7fdf400364f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 --2- 172.21.15.59:0/1875551689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf540fc040 0x7fdf54072d60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 >> 172.21.15.59:0/1875551689 conn(0x7fdf540f7800 msgr2=0x7fdf540f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 shutdown_connections 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 -- 172.21.15.59:0/1875551689 wait complete. 2024-02-18T00:23:54.644 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:54.640+0000 7fdf5baee700 1 librados: shutdown 2024-02-18T00:23:55.426 INFO:teuthology.orchestra.run.smithi059.stderr:Deploying unmanaged mon service... 2024-02-18T00:23:55.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:53.987065+0000 mgr.y (mgr.14116) 9 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:23:55.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:54.638211+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:55.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:23:54.638515+0000 mgr.y (mgr.14116) 10 : cephadm [INF] Added host smithi059 2024-02-18T00:23:55.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:54.652907+0000 mon.a (mon.0) 54 : audit [DBG] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:23:56.148 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.144+0000 7f39c1761700 1 Processor -- start 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.144+0000 7f39c1761700 1 -- start start 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39c1761700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc0fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39c1761700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0ff180 con 0x7f39bc070cc0 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc0fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc0fec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52750/0 (socket says 172.21.15.59:52750) 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39baffd700 1 -- 172.21.15.59:0/3816934584 learned_addr learned my addr 172.21.15.59:0/3816934584 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39baffd700 1 -- 172.21.15.59:0/3816934584 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0ff2c0 con 0x7f39bc070cc0 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39baffd700 1 --2- 172.21.15.59:0/3816934584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc0fec00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f39ac009e50 tx=0x7f39ac009840).ready entity=mon.0 client_cookie=98ae40e4cd96595a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39b9ffb700 1 -- 172.21.15.59:0/3816934584 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39ac00e070 con 0x7f39bc070cc0 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39b9ffb700 1 -- 172.21.15.59:0/3816934584 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f39ac013420 con 0x7f39bc070cc0 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.145+0000 7f39b9ffb700 1 -- 172.21.15.59:0/3816934584 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39ac007400 con 0x7f39bc070cc0 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.146+0000 7f39c1761700 1 -- 172.21.15.59:0/3816934584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 msgr2=0x7f39bc0fec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:56.149 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.146+0000 7f39c1761700 1 --2- 172.21.15.59:0/3816934584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc0fec00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f39ac009e50 tx=0x7f39ac009840).stop 2024-02-18T00:23:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.146+0000 7f39c1761700 1 -- 172.21.15.59:0/3816934584 shutdown_connections 2024-02-18T00:23:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.146+0000 7f39c1761700 1 --2- 172.21.15.59:0/3816934584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc0fec00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:56.150 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.146+0000 7f39c1761700 1 -- 172.21.15.59:0/3816934584 >> 172.21.15.59:0/3816934584 conn(0x7f39bc0f6660 msgr2=0x7f39bc0f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.146+0000 7f39c1761700 1 -- 172.21.15.59:0/3816934584 shutdown_connections 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.146+0000 7f39c1761700 1 -- 172.21.15.59:0/3816934584 wait complete. 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 librados: starting msgr at 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 librados: starting objecter 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 Processor -- start 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 -- start start 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 librados: setting wanted keys 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 librados: calling monclient init 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc10ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.147+0000 7f39c1761700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc10b240 con 0x7f39bc070cc0 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc10ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc10ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52762/0 (socket says 172.21.15.59:52762) 2024-02-18T00:23:56.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39baffd700 1 -- 172.21.15.59:0/3927804665 learned_addr learned my addr 172.21.15.59:0/3927804665 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39baffd700 1 -- 172.21.15.59:0/3927804665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac009ae0 con 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39baffd700 1 --2- 172.21.15.59:0/3927804665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc10ad50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f39ac013aa0 tx=0x7f39ac009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39b3fff700 10 client.?.objecter ms_handle_connect 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39b3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39b3fff700 1 -- 172.21.15.59:0/3927804665 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39ac00e040 con 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39b3fff700 1 -- 172.21.15.59:0/3927804665 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f39ac00bbf0 con 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc10b440 con 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39b3fff700 1 -- 172.21.15.59:0/3927804665 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39ac01c830 con 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39c1761700 10 client.14136.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc10c050 con 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.148+0000 7f39c1761700 1 librados: init done 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39b3fff700 1 -- 172.21.15.59:0/3927804665 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7f39ac003040 con 0x7f39bc070cc0 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39b3fff700 1 --2- 172.21.15.59:0/3927804665 >> v2:172.21.15.59:6800/3506000666 conn(0x7f39a8034030 0x7f39a80364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:56.154 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39b3fff700 1 -- 172.21.15.59:0/3927804665 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7f39ac0441d0 con 0x7f39bc070cc0 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39b3fff700 10 client.14136.objecter ms_dispatch 0x7f39bc001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39b3fff700 3 client.14136.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39b3fff700 3 client.14136.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc10b8d0 con 0x7f39bc070cc0 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39b3fff700 20 client.14136.objecter dump_active .. 0 homeless 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.149+0000 7f39ba7fc700 1 --2- 172.21.15.59:0/3927804665 >> v2:172.21.15.59:6800/3506000666 conn(0x7f39a8034030 0x7f39a80364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.150+0000 7f39ba7fc700 1 --2- 172.21.15.59:0/3927804665 >> v2:172.21.15.59:6800/3506000666 conn(0x7f39a8034030 0x7f39a80364f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f39a4009780 tx=0x7f39a4006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:56.155 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.150+0000 7f39b3fff700 10 client.14136.objecter ms_handle_connect 0x7f39a8034030 2024-02-18T00:23:56.156 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.154+0000 7f39b3fff700 1 -- 172.21.15.59:0/3927804665 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f39ac025400 con 0x7f39bc070cc0 2024-02-18T00:23:56.345 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.342+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 --> v2:172.21.15.59:6800/3506000666 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f39bc1001a0 con 0x7f39a8034030 2024-02-18T00:23:56.349 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.347+0000 7f39b3fff700 1 -- 172.21.15.59:0/3927804665 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f39bc1001a0 con 0x7f39a8034030 2024-02-18T00:23:56.349 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 10 librados: watch_flush enter 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 10 librados: watch_flush exit 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 20 client.14136.objecter shutdown clearing up homeless session... 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 10 client.14136.objecter successfully canceled tick 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 >> v2:172.21.15.59:6800/3506000666 conn(0x7f39a8034030 msgr2=0x7f39a80364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 1 --2- 172.21.15.59:0/3927804665 >> v2:172.21.15.59:6800/3506000666 conn(0x7f39a8034030 0x7f39a80364f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f39a4009780 tx=0x7f39a4006cc0).stop 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 msgr2=0x7f39bc10ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.348+0000 7f39c1761700 1 --2- 172.21.15.59:0/3927804665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc10ad50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f39ac013aa0 tx=0x7f39ac009840).stop 2024-02-18T00:23:56.352 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.349+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 shutdown_connections 2024-02-18T00:23:56.353 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.349+0000 7f39c1761700 1 --2- 172.21.15.59:0/3927804665 >> v2:172.21.15.59:6800/3506000666 conn(0x7f39a8034030 0x7f39a80364f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:56.353 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.349+0000 7f39c1761700 1 --2- 172.21.15.59:0/3927804665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc070cc0 0x7f39bc10ad50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:56.353 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.349+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 >> 172.21.15.59:0/3927804665 conn(0x7f39bc0f6660 msgr2=0x7f39bc0f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:56.353 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.349+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 shutdown_connections 2024-02-18T00:23:56.353 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.349+0000 7f39c1761700 1 -- 172.21.15.59:0/3927804665 wait complete. 2024-02-18T00:23:56.353 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:56.349+0000 7f39c1761700 1 librados: shutdown 2024-02-18T00:23:57.664 INFO:teuthology.orchestra.run.smithi059.stderr:Deploying unmanaged mgr service... 2024-02-18T00:23:57.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:56.344562+0000 mgr.y (mgr.14116) 11 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:23:57.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:23:56.345784+0000 mgr.y (mgr.14116) 12 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-18T00:23:57.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:56.347893+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:57.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:57.045777+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:57.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:57.290320+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:58.188 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.185+0000 7f249a6cc700 1 Processor -- start 2024-02-18T00:23:58.188 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.185+0000 7f249a6cc700 1 -- start start 2024-02-18T00:23:58.194 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.186+0000 7f249a6cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f24940fec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:58.194 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.186+0000 7f249a6cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24940ff1e0 con 0x7f2494070c60 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.186+0000 7f2493fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f24940fec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.186+0000 7f2493fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f24940fec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52764/0 (socket says 172.21.15.59:52764) 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.186+0000 7f2493fff700 1 -- 172.21.15.59:0/2685819109 learned_addr learned my addr 172.21.15.59:0/2685819109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.186+0000 7f2493fff700 1 -- 172.21.15.59:0/2685819109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24940ff320 con 0x7f2494070c60 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.186+0000 7f2493fff700 1 --2- 172.21.15.59:0/2685819109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f24940fec60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2484009730 tx=0x7f24840090f0).ready entity=mon.0 client_cookie=f942fa6cd385fc53 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.187+0000 7f2492ffd700 1 -- 172.21.15.59:0/2685819109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f248400b580 con 0x7f2494070c60 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.187+0000 7f2492ffd700 1 -- 172.21.15.59:0/2685819109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f248400bbb0 con 0x7f2494070c60 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.187+0000 7f2492ffd700 1 -- 172.21.15.59:0/2685819109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2484011bc0 con 0x7f2494070c60 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.187+0000 7f249a6cc700 1 -- 172.21.15.59:0/2685819109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 msgr2=0x7f24940fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.187+0000 7f249a6cc700 1 --2- 172.21.15.59:0/2685819109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f24940fec60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2484009730 tx=0x7f24840090f0).stop 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 -- 172.21.15.59:0/2685819109 shutdown_connections 2024-02-18T00:23:58.195 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 --2- 172.21.15.59:0/2685819109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f24940fec60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 -- 172.21.15.59:0/2685819109 >> 172.21.15.59:0/2685819109 conn(0x7f24940f6660 msgr2=0x7f24940f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 -- 172.21.15.59:0/2685819109 shutdown_connections 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 -- 172.21.15.59:0/2685819109 wait complete. 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 librados: starting msgr at 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 librados: starting objecter 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 Processor -- start 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 -- start start 2024-02-18T00:23:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 librados: setting wanted keys 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.188+0000 7f249a6cc700 1 librados: calling monclient init 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f249a6cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f249410ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f249a6cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249410b280 con 0x7f2494070c60 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f2493fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f249410ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f2493fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f249410ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52778/0 (socket says 172.21.15.59:52778) 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f2493fff700 1 -- 172.21.15.59:0/2775498262 learned_addr learned my addr 172.21.15.59:0/2775498262 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f2493fff700 1 -- 172.21.15.59:0/2775498262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2484009370 con 0x7f2494070c60 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f2493fff700 1 --2- 172.21.15.59:0/2775498262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f249410ad90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2484006e90 tx=0x7f24840090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f24917fa700 10 client.?.objecter ms_handle_connect 0x7f2494070c60 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f24917fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.189+0000 7f24917fa700 1 -- 172.21.15.59:0/2775498262 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f248401a640 con 0x7f2494070c60 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.190+0000 7f24917fa700 1 -- 172.21.15.59:0/2775498262 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f248401ac70 con 0x7f2494070c60 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.190+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249410b480 con 0x7f2494070c60 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.190+0000 7f24917fa700 1 -- 172.21.15.59:0/2775498262 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2484023b90 con 0x7f2494070c60 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.190+0000 7f249a6cc700 10 client.14138.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.190+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249410c090 con 0x7f2494070c60 2024-02-18T00:23:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.190+0000 7f249a6cc700 1 librados: init done 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249410b910 con 0x7f2494070c60 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 1 -- 172.21.15.59:0/2775498262 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7f248402c3f0 con 0x7f2494070c60 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 1 --2- 172.21.15.59:0/2775498262 >> v2:172.21.15.59:6800/3506000666 conn(0x7f247c034030 0x7f247c0364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 1 -- 172.21.15.59:0/2775498262 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7f24840448e0 con 0x7f2494070c60 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 10 client.14138.objecter ms_dispatch 0x7f2494001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 3 client.14138.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 3 client.14138.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:23:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 20 client.14138.objecter dump_active .. 0 homeless 2024-02-18T00:23:58.200 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24937fe700 1 --2- 172.21.15.59:0/2775498262 >> v2:172.21.15.59:6800/3506000666 conn(0x7f247c034030 0x7f247c0364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:23:58.200 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24917fa700 10 client.14138.objecter ms_handle_connect 0x7f247c034030 2024-02-18T00:23:58.200 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.191+0000 7f24937fe700 1 --2- 172.21.15.59:0/2775498262 >> v2:172.21.15.59:6800/3506000666 conn(0x7f247c034030 0x7f247c0364f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f2488009780 tx=0x7f2488006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:23:58.200 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.196+0000 7f24917fa700 1 -- 172.21.15.59:0/2775498262 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f249410b910 con 0x7f2494070c60 2024-02-18T00:23:58.390 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.388+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 --> v2:172.21.15.59:6800/3506000666 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f249410b910 con 0x7f247c034030 2024-02-18T00:23:58.394 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.392+0000 7f24917fa700 1 -- 172.21.15.59:0/2775498262 <== mgr.14116 v2:172.21.15.59:6800/3506000666 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f249410b910 con 0x7f247c034030 2024-02-18T00:23:58.395 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 10 librados: watch_flush enter 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 10 librados: watch_flush exit 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 20 client.14138.objecter shutdown clearing up homeless session... 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 10 client.14138.objecter successfully canceled tick 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 >> v2:172.21.15.59:6800/3506000666 conn(0x7f247c034030 msgr2=0x7f247c0364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 --2- 172.21.15.59:0/2775498262 >> v2:172.21.15.59:6800/3506000666 conn(0x7f247c034030 0x7f247c0364f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f2488009780 tx=0x7f2488006cc0).stop 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 msgr2=0x7f249410ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 --2- 172.21.15.59:0/2775498262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f249410ad90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2484006e90 tx=0x7f24840090f0).stop 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 shutdown_connections 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 --2- 172.21.15.59:0/2775498262 >> v2:172.21.15.59:6800/3506000666 conn(0x7f247c034030 0x7f247c0364f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:58.398 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 --2- 172.21.15.59:0/2775498262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2494070c60 0x7f249410ad90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:23:58.399 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 >> 172.21.15.59:0/2775498262 conn(0x7f24940f6660 msgr2=0x7f24940f7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:23:58.399 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 shutdown_connections 2024-02-18T00:23:58.399 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 -- 172.21.15.59:0/2775498262 wait complete. 2024-02-18T00:23:58.399 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:23:58.394+0000 7f249a6cc700 1 librados: shutdown 2024-02-18T00:23:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:58.390267+0000 mgr.y (mgr.14116) 13 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:23:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:23:58.391174+0000 mgr.y (mgr.14116) 14 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-18T00:23:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:58.393284+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:23:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:23:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:23:59.135292+0000 mon.a (mon.0) 59 : audit [INF] from='mgr.14116 172.21.15.59:0/1884027000' entity='mgr.y' 2024-02-18T00:24:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.101+0000 7fe9e09fe700 1 Processor -- start 2024-02-18T00:24:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.101+0000 7fe9e09fe700 1 -- start start 2024-02-18T00:24:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.102+0000 7fe9e09fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc0fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.102+0000 7fe9e09fe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc0ff1d0 con 0x7fe9dc070c80 2024-02-18T00:24:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.102+0000 7fe9da59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc0fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.102+0000 7fe9da59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc0fec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52998/0 (socket says 172.21.15.59:52998) 2024-02-18T00:24:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.102+0000 7fe9da59c700 1 -- 172.21.15.59:0/408120632 learned_addr learned my addr 172.21.15.59:0/408120632 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.103+0000 7fe9da59c700 1 -- 172.21.15.59:0/408120632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc0ff310 con 0x7fe9dc070c80 2024-02-18T00:24:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.103+0000 7fe9da59c700 1 --2- 172.21.15.59:0/408120632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc0fec50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe9cc009730 tx=0x7fe9cc0090f0).ready entity=mon.0 client_cookie=589792adabe0d1c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.103+0000 7fe9d959a700 1 -- 172.21.15.59:0/408120632 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc00b580 con 0x7fe9dc070c80 2024-02-18T00:24:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.103+0000 7fe9d959a700 1 -- 172.21.15.59:0/408120632 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9cc00bbb0 con 0x7fe9dc070c80 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.103+0000 7fe9d959a700 1 -- 172.21.15.59:0/408120632 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc011bc0 con 0x7fe9dc070c80 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.104+0000 7fe9e09fe700 1 -- 172.21.15.59:0/408120632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 msgr2=0x7fe9dc0fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.104+0000 7fe9e09fe700 1 --2- 172.21.15.59:0/408120632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc0fec50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe9cc009730 tx=0x7fe9cc0090f0).stop 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.104+0000 7fe9e09fe700 1 -- 172.21.15.59:0/408120632 shutdown_connections 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.104+0000 7fe9e09fe700 1 --2- 172.21.15.59:0/408120632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc0fec50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.104+0000 7fe9e09fe700 1 -- 172.21.15.59:0/408120632 >> 172.21.15.59:0/408120632 conn(0x7fe9dc0f6670 msgr2=0x7fe9dc0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.104+0000 7fe9e09fe700 1 -- 172.21.15.59:0/408120632 shutdown_connections 2024-02-18T00:24:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.104+0000 7fe9e09fe700 1 -- 172.21.15.59:0/408120632 wait complete. 2024-02-18T00:24:00.108 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 librados: starting msgr at 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 librados: starting objecter 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 Processor -- start 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 -- start start 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 librados: setting wanted keys 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 librados: calling monclient init 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc108a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.105+0000 7fe9e09fe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9dc10d100 con 0x7fe9dc070c80 2024-02-18T00:24:00.109 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9da59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9da59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53008/0 (socket says 172.21.15.59:53008) 2024-02-18T00:24:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9da59c700 1 -- 172.21.15.59:0/3384464461 learned_addr learned my addr 172.21.15.59:0/3384464461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9da59c700 1 -- 172.21.15.59:0/3384464461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009370 con 0x7fe9dc070c80 2024-02-18T00:24:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9da59c700 1 --2- 172.21.15.59:0/3384464461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc108a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe9cc000c00 tx=0x7fe9cc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9cb7fe700 10 client.?.objecter ms_handle_connect 0x7fe9dc070c80 2024-02-18T00:24:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9cb7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:00.110 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9cb7fe700 1 -- 172.21.15.59:0/3384464461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc01a640 con 0x7fe9dc070c80 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc108f70 con 0x7fe9dc070c80 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9cb7fe700 1 -- 172.21.15.59:0/3384464461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9cc01ac70 con 0x7fe9dc070c80 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9cb7fe700 1 -- 172.21.15.59:0/3384464461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc023be0 con 0x7fe9dc070c80 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9e09fe700 10 client.14140.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9dc109400 con 0x7fe9dc070c80 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.106+0000 7fe9e09fe700 1 librados: init done 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.107+0000 7fe9cb7fe700 1 -- 172.21.15.59:0/3384464461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7fe9cc02c3f0 con 0x7fe9dc070c80 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9cb7fe700 1 --2- 172.21.15.59:0/3384464461 >> v2:172.21.15.59:6800/3506000666 conn(0x7fe9c4033fd0 0x7fe9c4036490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9cb7fe700 1 -- 172.21.15.59:0/3384464461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7fe9cc044a10 con 0x7fe9dc070c80 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9cb7fe700 10 client.14140.objecter ms_dispatch 0x7fe9dc001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9cb7fe700 3 client.14140.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:24:00.112 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9cb7fe700 3 client.14140.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:24:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9dc05a310 con 0x7fe9dc070c80 2024-02-18T00:24:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9cb7fe700 20 client.14140.objecter dump_active .. 0 homeless 2024-02-18T00:24:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9d9d9b700 1 --2- 172.21.15.59:0/3384464461 >> v2:172.21.15.59:6800/3506000666 conn(0x7fe9c4033fd0 0x7fe9c4036490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9cb7fe700 10 client.14140.objecter ms_handle_connect 0x7fe9c4033fd0 2024-02-18T00:24:00.113 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.108+0000 7fe9d9d9b700 1 --2- 172.21.15.59:0/3384464461 >> v2:172.21.15.59:6800/3506000666 conn(0x7fe9c4033fd0 0x7fe9c4036490 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe9d0009780 tx=0x7fe9d0006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.116+0000 7fe9cb7fe700 1 -- 172.21.15.59:0/3384464461 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe9cc01e070 con 0x7fe9dc070c80 2024-02-18T00:24:00.300 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.298+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fe9dc100230 con 0x7fe9dc070c80 2024-02-18T00:24:00.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.301+0000 7fe9cb7fe700 1 -- 172.21.15.59:0/3384464461 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fe9cc02bb40 con 0x7fe9dc070c80 2024-02-18T00:24:00.306 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 10 librados: watch_flush enter 2024-02-18T00:24:00.306 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 10 librados: watch_flush exit 2024-02-18T00:24:00.306 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 20 client.14140.objecter shutdown clearing up homeless session... 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 10 client.14140.objecter successfully canceled tick 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 >> v2:172.21.15.59:6800/3506000666 conn(0x7fe9c4033fd0 msgr2=0x7fe9c4036490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 --2- 172.21.15.59:0/3384464461 >> v2:172.21.15.59:6800/3506000666 conn(0x7fe9c4033fd0 0x7fe9c4036490 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe9d0009780 tx=0x7fe9d0006cc0).stop 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 msgr2=0x7fe9dc108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 --2- 172.21.15.59:0/3384464461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc108a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fe9cc000c00 tx=0x7fe9cc0090f0).stop 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 shutdown_connections 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 --2- 172.21.15.59:0/3384464461 >> v2:172.21.15.59:6800/3506000666 conn(0x7fe9c4033fd0 0x7fe9c4036490 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 --2- 172.21.15.59:0/3384464461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9dc070c80 0x7fe9dc108a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 >> 172.21.15.59:0/3384464461 conn(0x7fe9dc0f6670 msgr2=0x7fe9dc0f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 shutdown_connections 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 -- 172.21.15.59:0/3384464461 wait complete. 2024-02-18T00:24:00.307 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:00.303+0000 7fe9e09fe700 1 librados: shutdown 2024-02-18T00:24:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:00.302228+0000 mon.a (mon.0) 60 : audit [INF] from='client.? 172.21.15.59:0/3384464461' entity='client.admin' 2024-02-18T00:24:02.384 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.360+0000 7f728d97b700 1 Processor -- start 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.360+0000 7f728d97b700 1 -- start start 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.360+0000 7f728d97b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f72880fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.360+0000 7f728d97b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72880fc930 con 0x7f72880fc020 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.360+0000 7f7286ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f72880fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.360+0000 7f7286ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f72880fc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53018/0 (socket says 172.21.15.59:53018) 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.360+0000 7f7286ffd700 1 -- 172.21.15.59:0/2067827458 learned_addr learned my addr 172.21.15.59:0/2067827458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.361+0000 7f7286ffd700 1 -- 172.21.15.59:0/2067827458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72880fca70 con 0x7f72880fc020 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.361+0000 7f7286ffd700 1 --2- 172.21.15.59:0/2067827458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f72880fc440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7270009730 tx=0x7f72700090f0).ready entity=mon.0 client_cookie=ca49944ca02b6399 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.361+0000 7f7285ffb700 1 -- 172.21.15.59:0/2067827458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f727000b580 con 0x7f72880fc020 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.361+0000 7f7285ffb700 1 -- 172.21.15.59:0/2067827458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f727000bbb0 con 0x7f72880fc020 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.361+0000 7f7285ffb700 1 -- 172.21.15.59:0/2067827458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7270011bc0 con 0x7f72880fc020 2024-02-18T00:24:02.385 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.361+0000 7f728d97b700 1 -- 172.21.15.59:0/2067827458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 msgr2=0x7f72880fc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.361+0000 7f728d97b700 1 --2- 172.21.15.59:0/2067827458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f72880fc440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f7270009730 tx=0x7f72700090f0).stop 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 -- 172.21.15.59:0/2067827458 shutdown_connections 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 --2- 172.21.15.59:0/2067827458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f72880fc440 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 -- 172.21.15.59:0/2067827458 >> 172.21.15.59:0/2067827458 conn(0x7f72880f77c0 msgr2=0x7f72880f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 -- 172.21.15.59:0/2067827458 shutdown_connections 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 -- 172.21.15.59:0/2067827458 wait complete. 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 librados: starting msgr at 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 librados: starting objecter 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.362+0000 7f728d97b700 1 Processor -- start 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f728d97b700 1 -- start start 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f728d97b700 1 librados: setting wanted keys 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f728d97b700 1 librados: calling monclient init 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f728d97b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f728810ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f728d97b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728810b220 con 0x7f72880fc020 2024-02-18T00:24:02.386 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f7286ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f728810ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f7286ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f728810ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53026/0 (socket says 172.21.15.59:53026) 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f7286ffd700 1 -- 172.21.15.59:0/1479487963 learned_addr learned my addr 172.21.15.59:0/1479487963 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f7286ffd700 1 -- 172.21.15.59:0/1479487963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270009370 con 0x7f72880fc020 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f7286ffd700 1 --2- 172.21.15.59:0/1479487963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f728810ad30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7270000c00 tx=0x7f72700090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f727ffff700 10 client.?.objecter ms_handle_connect 0x7f72880fc020 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f727ffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.363+0000 7f727ffff700 1 -- 172.21.15.59:0/1479487963 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f727001a640 con 0x7f72880fc020 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f727ffff700 1 -- 172.21.15.59:0/1479487963 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f727001ac70 con 0x7f72880fc020 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f727ffff700 1 -- 172.21.15.59:0/1479487963 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7270023d30 con 0x7f72880fc020 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728810b420 con 0x7f72880fc020 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f728d97b700 10 client.14142.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728810c030 con 0x7f72880fc020 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f728d97b700 1 librados: init done 2024-02-18T00:24:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f727ffff700 1 -- 172.21.15.59:0/1479487963 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7f727002c3f0 con 0x7f72880fc020 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.364+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728810b8b0 con 0x7f72880fc020 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f727ffff700 1 --2- 172.21.15.59:0/1479487963 >> v2:172.21.15.59:6800/3506000666 conn(0x7f7274034030 0x7f72740364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f727ffff700 1 -- 172.21.15.59:0/1479487963 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7f7270044af0 con 0x7f72880fc020 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f727ffff700 10 client.14142.objecter ms_dispatch 0x7f7288001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f727ffff700 3 client.14142.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f727ffff700 3 client.14142.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f727ffff700 20 client.14142.objecter dump_active .. 0 homeless 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f72867fc700 1 --2- 172.21.15.59:0/1479487963 >> v2:172.21.15.59:6800/3506000666 conn(0x7f7274034030 0x7f72740364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f727ffff700 10 client.14142.objecter ms_handle_connect 0x7f7274034030 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.365+0000 7f72867fc700 1 --2- 172.21.15.59:0/1479487963 >> v2:172.21.15.59:6800/3506000666 conn(0x7f7274034030 0x7f72740364f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f7278009780 tx=0x7f7278006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:02.388 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.369+0000 7f727ffff700 1 -- 172.21.15.59:0/1479487963 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f728810b8b0 con 0x7f72880fc020 2024-02-18T00:24:02.542 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.540+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f728810b8b0 con 0x7f72880fc020 2024-02-18T00:24:02.545 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.543+0000 7f727ffff700 1 -- 172.21.15.59:0/1479487963 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f727001e020 con 0x7f72880fc020 2024-02-18T00:24:02.546 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.544+0000 7f728d97b700 10 librados: watch_flush enter 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.544+0000 7f728d97b700 10 librados: watch_flush exit 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.544+0000 7f728d97b700 20 client.14142.objecter shutdown clearing up homeless session... 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.544+0000 7f728d97b700 10 client.14142.objecter successfully canceled tick 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.544+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 >> v2:172.21.15.59:6800/3506000666 conn(0x7f7274034030 msgr2=0x7f72740364f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 --2- 172.21.15.59:0/1479487963 >> v2:172.21.15.59:6800/3506000666 conn(0x7f7274034030 0x7f72740364f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f7278009780 tx=0x7f7278006cc0).stop 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 msgr2=0x7f728810ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 --2- 172.21.15.59:0/1479487963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f728810ad30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7270000c00 tx=0x7f72700090f0).stop 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 shutdown_connections 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 --2- 172.21.15.59:0/1479487963 >> v2:172.21.15.59:6800/3506000666 conn(0x7f7274034030 0x7f72740364f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:02.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 --2- 172.21.15.59:0/1479487963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72880fc020 0x7f728810ad30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:02.549 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 >> 172.21.15.59:0/1479487963 conn(0x7f72880f77c0 msgr2=0x7f72880f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:02.549 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 shutdown_connections 2024-02-18T00:24:02.549 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 -- 172.21.15.59:0/1479487963 wait complete. 2024-02-18T00:24:02.549 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:02.545+0000 7f728d97b700 1 librados: shutdown 2024-02-18T00:24:03.454 INFO:teuthology.orchestra.run.smithi059.stderr:Enabling the dashboard module... 2024-02-18T00:24:03.720 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:02.544062+0000 mon.a (mon.0) 61 : audit [INF] from='client.? 172.21.15.59:0/1479487963' entity='client.admin' 2024-02-18T00:24:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.055+0000 7fad6795c700 1 Processor -- start 2024-02-18T00:24:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.055+0000 7fad6795c700 1 -- start start 2024-02-18T00:24:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.055+0000 7fad6795c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad600fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.055+0000 7fad6795c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad600fc950 con 0x7fad600fc040 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad656f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad600fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad656f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad600fc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53032/0 (socket says 172.21.15.59:53032) 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad656f8700 1 -- 172.21.15.59:0/1850541715 learned_addr learned my addr 172.21.15.59:0/1850541715 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad656f8700 1 -- 172.21.15.59:0/1850541715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad600fca90 con 0x7fad600fc040 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad656f8700 1 --2- 172.21.15.59:0/1850541715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad600fc460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fad50009730 tx=0x7fad500090f0).ready entity=mon.0 client_cookie=7dfa692e0e36fc69 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad57fff700 1 -- 172.21.15.59:0/1850541715 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad5000b580 con 0x7fad600fc040 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad57fff700 1 -- 172.21.15.59:0/1850541715 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad5000bbb0 con 0x7fad600fc040 2024-02-18T00:24:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.056+0000 7fad57fff700 1 -- 172.21.15.59:0/1850541715 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad50011bc0 con 0x7fad600fc040 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 -- 172.21.15.59:0/1850541715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 msgr2=0x7fad600fc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 --2- 172.21.15.59:0/1850541715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad600fc460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fad50009730 tx=0x7fad500090f0).stop 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 -- 172.21.15.59:0/1850541715 shutdown_connections 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 --2- 172.21.15.59:0/1850541715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad600fc460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 -- 172.21.15.59:0/1850541715 >> 172.21.15.59:0/1850541715 conn(0x7fad600f7800 msgr2=0x7fad600f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 -- 172.21.15.59:0/1850541715 shutdown_connections 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 -- 172.21.15.59:0/1850541715 wait complete. 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 librados: starting msgr at 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 librados: starting objecter 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.057+0000 7fad6795c700 1 Processor -- start 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 -- start start 2024-02-18T00:24:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 librados: setting wanted keys 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 librados: calling monclient init 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad6010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6010b220 con 0x7fad600fc040 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad656f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad6010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad656f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad6010ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53038/0 (socket says 172.21.15.59:53038) 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad656f8700 1 -- 172.21.15.59:0/3143582183 learned_addr learned my addr 172.21.15.59:0/3143582183 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad656f8700 1 -- 172.21.15.59:0/3143582183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad50009370 con 0x7fad600fc040 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad656f8700 1 --2- 172.21.15.59:0/3143582183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad6010ad30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fad50006e90 tx=0x7fad500090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad567fc700 10 client.?.objecter ms_handle_connect 0x7fad600fc040 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad567fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad5001a640 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6010b420 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad5001ac70 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad50023b90 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 10 client.14144.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6010bfd0 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.058+0000 7fad6795c700 1 librados: init done 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.059+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 38582+0+0 (secure 0 0 0) 0x7fad5002c3f0 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.059+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6005a310 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.059+0000 7fad567fc700 1 --2- 172.21.15.59:0/3143582183 >> v2:172.21.15.59:6800/3506000666 conn(0x7fad4c033fe0 0x7fad4c0364a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.059+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7fad50044c90 con 0x7fad600fc040 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.059+0000 7fad567fc700 10 client.14144.objecter ms_dispatch 0x7fad60001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.059+0000 7fad567fc700 3 client.14144.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.059+0000 7fad567fc700 3 client.14144.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:24:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.060+0000 7fad567fc700 20 client.14144.objecter dump_active .. 0 homeless 2024-02-18T00:24:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.060+0000 7fad64ef7700 1 --2- 172.21.15.59:0/3143582183 >> v2:172.21.15.59:6800/3506000666 conn(0x7fad4c033fe0 0x7fad4c0364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.060+0000 7fad64ef7700 1 --2- 172.21.15.59:0/3143582183 >> v2:172.21.15.59:6800/3506000666 conn(0x7fad4c033fe0 0x7fad4c0364a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fad5c009780 tx=0x7fad5c006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.060+0000 7fad567fc700 10 client.14144.objecter ms_handle_connect 0x7fad4c033fe0 2024-02-18T00:24:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.064+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fad5002b530 con 0x7fad600fc040 2024-02-18T00:24:04.283 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.281+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fad6010b8b0 con 0x7fad600fc040 2024-02-18T00:24:04.548 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.545+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 9) v1 ==== 38595+0+0 (secure 0 0 0) 0x7fad5002c6f0 con 0x7fad600fc040 2024-02-18T00:24:04.549 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.547+0000 7fad567fc700 1 -- 172.21.15.59:0/3143582183 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fad50021320 con 0x7fad600fc040 2024-02-18T00:24:04.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 10 librados: watch_flush enter 2024-02-18T00:24:04.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 10 librados: watch_flush exit 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 20 client.14144.objecter shutdown clearing up homeless session... 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 10 client.14144.objecter successfully canceled tick 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 >> v2:172.21.15.59:6800/3506000666 conn(0x7fad4c033fe0 msgr2=0x7fad4c0364a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 --2- 172.21.15.59:0/3143582183 >> v2:172.21.15.59:6800/3506000666 conn(0x7fad4c033fe0 0x7fad4c0364a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fad5c009780 tx=0x7fad5c006cc0).stop 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 msgr2=0x7fad6010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 --2- 172.21.15.59:0/3143582183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad6010ad30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fad50006e90 tx=0x7fad500090f0).stop 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 shutdown_connections 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 --2- 172.21.15.59:0/3143582183 >> v2:172.21.15.59:6800/3506000666 conn(0x7fad4c033fe0 0x7fad4c0364a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 --2- 172.21.15.59:0/3143582183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad600fc040 0x7fad6010ad30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 >> 172.21.15.59:0/3143582183 conn(0x7fad600f7800 msgr2=0x7fad600f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 shutdown_connections 2024-02-18T00:24:04.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 -- 172.21.15.59:0/3143582183 wait complete. 2024-02-18T00:24:04.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:04.549+0000 7fad6795c700 1 librados: shutdown 2024-02-18T00:24:04.797 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:04.282677+0000 mon.a (mon.0) 62 : audit [INF] from='client.? 172.21.15.59:0/3143582183' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-18T00:24:04.797 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ignoring --setuser ceph since I am not root 2024-02-18T00:24:04.797 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ignoring --setgroup ceph since I am not root 2024-02-18T00:24:04.797 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:04.710+0000 7f7590c9f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T00:24:05.179 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:04.794+0000 7f7590c9f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T00:24:05.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:05.177+0000 7f7590c9f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T00:24:05.921 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.917+0000 7f5aa5398700 1 Processor -- start 2024-02-18T00:24:05.921 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.917+0000 7f5aa5398700 1 -- start start 2024-02-18T00:24:05.921 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.917+0000 7f5aa5398700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa006a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:05.921 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.917+0000 7f5aa5398700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa006ab80 con 0x7f5aa006a1e0 2024-02-18T00:24:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.918+0000 7f5a9effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa006a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.918+0000 7f5a9effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa006a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53068/0 (socket says 172.21.15.59:53068) 2024-02-18T00:24:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.918+0000 7f5a9effd700 1 -- 172.21.15.59:0/3765811870 learned_addr learned my addr 172.21.15.59:0/3765811870 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.918+0000 7f5a9effd700 1 -- 172.21.15.59:0/3765811870 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa006acc0 con 0x7f5aa006a1e0 2024-02-18T00:24:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.918+0000 7f5a9effd700 1 --2- 172.21.15.59:0/3765811870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa006a600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5a980079a0 tx=0x7f5a9800ad00).ready entity=mon.0 client_cookie=7686508bfe0cae4a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:05.936 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.919+0000 7f5a9dffb700 1 -- 172.21.15.59:0/3765811870 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a98010e50 con 0x7f5aa006a1e0 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.919+0000 7f5a9dffb700 1 -- 172.21.15.59:0/3765811870 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9800a7f0 con 0x7f5aa006a1e0 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.919+0000 7f5a9dffb700 1 -- 172.21.15.59:0/3765811870 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a98019bd0 con 0x7f5aa006a1e0 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.920+0000 7f5aa5398700 1 -- 172.21.15.59:0/3765811870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 msgr2=0x7f5aa006a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.920+0000 7f5aa5398700 1 --2- 172.21.15.59:0/3765811870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa006a600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f5a980079a0 tx=0x7f5a9800ad00).stop 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.920+0000 7f5aa5398700 1 -- 172.21.15.59:0/3765811870 shutdown_connections 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.920+0000 7f5aa5398700 1 --2- 172.21.15.59:0/3765811870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa006a600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.920+0000 7f5aa5398700 1 -- 172.21.15.59:0/3765811870 >> 172.21.15.59:0/3765811870 conn(0x7f5aa0063ef0 msgr2=0x7f5aa0066330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.920+0000 7f5aa5398700 1 -- 172.21.15.59:0/3765811870 shutdown_connections 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.920+0000 7f5aa5398700 1 -- 172.21.15.59:0/3765811870 wait complete. 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.921+0000 7f5aa5398700 1 librados: starting msgr at 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.921+0000 7f5aa5398700 1 librados: starting objecter 2024-02-18T00:24:05.937 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.921+0000 7f5aa5398700 1 Processor -- start 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5aa5398700 1 -- start start 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5aa5398700 1 librados: setting wanted keys 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5aa5398700 1 librados: calling monclient init 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5aa5398700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa0122380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5aa5398700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0122870 con 0x7f5aa006a1e0 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5a9effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa0122380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5a9effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa0122380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53072/0 (socket says 172.21.15.59:53072) 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.922+0000 7f5a9effd700 1 -- 172.21.15.59:0/2818331792 learned_addr learned my addr 172.21.15.59:0/2818331792 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.923+0000 7f5a9effd700 1 -- 172.21.15.59:0/2818331792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9800be70 con 0x7f5aa006a1e0 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.923+0000 7f5a9effd700 1 --2- 172.21.15.59:0/2818331792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa0122380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f5a98000f80 tx=0x7f5a9800d040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.923+0000 7f5a97fff700 10 client.?.objecter ms_handle_connect 0x7f5aa006a1e0 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.923+0000 7f5a97fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:05.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.924+0000 7f5a97fff700 1 -- 172.21.15.59:0/2818331792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a98019a10 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.924+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa0122a70 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.924+0000 7f5a97fff700 1 -- 172.21.15.59:0/2818331792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a98019610 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.924+0000 7f5a97fff700 1 -- 172.21.15.59:0/2818331792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a980206c0 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.924+0000 7f5aa5398700 10 client.14148.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.924+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa0122f00 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.924+0000 7f5aa5398700 1 librados: init done 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.925+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa0046ce0 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a97fff700 1 -- 172.21.15.59:0/2818331792 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 38595+0+0 (secure 0 0 0) 0x7f5a98003040 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a97fff700 1 --2- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 0x7f5a8c036540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a97fff700 1 -- 172.21.15.59:0/2818331792 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7f5a98010070 con 0x7f5aa006a1e0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a97fff700 10 client.14148.objecter ms_dispatch 0x7f5aa0001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a97fff700 3 client.14148.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a97fff700 3 client.14148.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:24:05.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a97fff700 20 client.14148.objecter dump_active .. 0 homeless 2024-02-18T00:24:05.940 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a9e7fc700 1 -- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 msgr2=0x7f5a8c036540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3506000666 2024-02-18T00:24:05.940 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.926+0000 7f5a9e7fc700 1 --2- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 0x7f5a8c036540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:24:05.947 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:05.935+0000 7f5a97fff700 1 -- 172.21.15.59:0/2818331792 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5a98021ba0 con 0x7f5aa006a1e0 2024-02-18T00:24:05.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:04.548251+0000 mon.a (mon.0) 63 : audit [INF] from='client.? 172.21.15.59:0/3143582183' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-18T00:24:05.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:04.548317+0000 mon.a (mon.0) 64 : cluster [DBG] mgrmap e9: y(active, since 19s) 2024-02-18T00:24:06.129 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.127+0000 7f5a9e7fc700 1 -- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 msgr2=0x7f5a8c036540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3506000666 2024-02-18T00:24:06.129 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.127+0000 7f5a9e7fc700 1 --2- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 0x7f5a8c036540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-18T00:24:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.195+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5aa0069170 con 0x7f5aa006a1e0 2024-02-18T00:24:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.195+0000 7f5a97fff700 1 -- 172.21.15.59:0/2818331792 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5a98021ba0 con 0x7f5aa006a1e0 2024-02-18T00:24:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: { 2024-02-18T00:24:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "epoch": 9, 2024-02-18T00:24:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "available": true, 2024-02-18T00:24:06.198 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-18T00:24:06.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-18T00:24:06.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: } 2024-02-18T00:24:06.200 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 10 librados: watch_flush enter 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 10 librados: watch_flush exit 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 20 client.14148.objecter shutdown clearing up homeless session... 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 10 client.14148.objecter successfully canceled tick 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 msgr2=0x7f5a8c036540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 --2- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 0x7f5a8c036540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 msgr2=0x7f5aa0122380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 --2- 172.21.15.59:0/2818331792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa0122380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f5a98000f80 tx=0x7f5a9800d040).stop 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 shutdown_connections 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 --2- 172.21.15.59:0/2818331792 >> v2:172.21.15.59:6800/3506000666 conn(0x7f5a8c034080 0x7f5a8c036540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 --2- 172.21.15.59:0/2818331792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5aa006a1e0 0x7f5aa0122380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 >> 172.21.15.59:0/2818331792 conn(0x7f5aa0063ef0 msgr2=0x7f5aa0066300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 shutdown_connections 2024-02-18T00:24:06.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 -- 172.21.15.59:0/2818331792 wait complete. 2024-02-18T00:24:06.202 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:06.197+0000 7f5aa5398700 1 librados: shutdown 2024-02-18T00:24:06.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:05.985+0000 7f7590c9f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T00:24:06.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:06.068+0000 7f7590c9f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T00:24:06.573 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:06.196441+0000 mon.a (mon.0) 65 : audit [DBG] from='client.? 172.21.15.59:0/2818331792' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-18T00:24:06.573 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:06.296+0000 7f7590c9f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T00:24:06.909 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for the mgr to restart... 2024-02-18T00:24:06.909 INFO:teuthology.orchestra.run.smithi059.stderr:Waiting for mgr epoch 9... 2024-02-18T00:24:07.434 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:07.298+0000 7f7590c9f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T00:24:07.551 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.544+0000 7fa9f86fb700 1 Processor -- start 2024-02-18T00:24:07.551 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f86fb700 1 -- start start 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f86fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f01010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f86fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00fcb30 con 0x7fa9f0070c20 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f6497700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f01010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f6497700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f01010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53076/0 (socket says 172.21.15.59:53076) 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f6497700 1 -- 172.21.15.59:0/3517995036 learned_addr learned my addr 172.21.15.59:0/3517995036 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f6497700 1 -- 172.21.15.59:0/3517995036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f01015a0 con 0x7fa9f0070c20 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.545+0000 7fa9f6497700 1 --2- 172.21.15.59:0/3517995036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f01010b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa9ec009730 tx=0x7fa9ec0090f0).ready entity=mon.0 client_cookie=f98fdb9bc0fa2138 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f5495700 1 -- 172.21.15.59:0/3517995036 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec00b580 con 0x7fa9f0070c20 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f5495700 1 -- 172.21.15.59:0/3517995036 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9ec00bbb0 con 0x7fa9f0070c20 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f5495700 1 -- 172.21.15.59:0/3517995036 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec011bc0 con 0x7fa9f0070c20 2024-02-18T00:24:07.552 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f86fb700 1 -- 172.21.15.59:0/3517995036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 msgr2=0x7fa9f01010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f86fb700 1 --2- 172.21.15.59:0/3517995036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f01010b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa9ec009730 tx=0x7fa9ec0090f0).stop 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f86fb700 1 -- 172.21.15.59:0/3517995036 shutdown_connections 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f86fb700 1 --2- 172.21.15.59:0/3517995036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f01010b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f86fb700 1 -- 172.21.15.59:0/3517995036 >> 172.21.15.59:0/3517995036 conn(0x7fa9f00f6670 msgr2=0x7fa9f00f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f86fb700 1 -- 172.21.15.59:0/3517995036 shutdown_connections 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.546+0000 7fa9f86fb700 1 -- 172.21.15.59:0/3517995036 wait complete. 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.547+0000 7fa9f86fb700 1 librados: starting msgr at 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.547+0000 7fa9f86fb700 1 librados: starting objecter 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.547+0000 7fa9f86fb700 1 Processor -- start 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.547+0000 7fa9f86fb700 1 -- start start 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.547+0000 7fa9f86fb700 1 librados: setting wanted keys 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.547+0000 7fa9f86fb700 1 librados: calling monclient init 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f86fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f010cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:07.553 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f86fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f010d440 con 0x7fa9f0070c20 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f6497700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f010cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f6497700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f010cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53084/0 (socket says 172.21.15.59:53084) 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f6497700 1 -- 172.21.15.59:0/790424878 learned_addr learned my addr 172.21.15.59:0/790424878 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f6497700 1 -- 172.21.15.59:0/790424878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec009370 con 0x7fa9f0070c20 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f6497700 1 --2- 172.21.15.59:0/790424878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f010cf50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa9ec012fd0 tx=0x7fa9ec0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9e77fe700 10 client.?.objecter ms_handle_connect 0x7fa9f0070c20 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9e77fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec01a430 con 0x7fa9f0070c20 2024-02-18T00:24:07.554 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9ec01aa60 con 0x7fa9f0070c20 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f010d640 con 0x7fa9f0070c20 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9ec0239b0 con 0x7fa9f0070c20 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f86fb700 10 client.14150.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f010dad0 con 0x7fa9f0070c20 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.548+0000 7fa9f86fb700 1 librados: init done 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 38595+0+0 (secure 0 0 0) 0x7fa9ec023b10 con 0x7fa9f0070c20 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 0x7fa9dc0364f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 --> v2:172.21.15.59:6800/3506000666 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9dc036b70 con 0x7fa9dc034030 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (secure 0 0 0) 0x7fa9ec044760 con 0x7fa9f0070c20 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 10 client.14150.objecter ms_dispatch 0x7fa9f0001550 osd_map(2..2 src has 1..2) v4 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 3 client.14150.objecter handle_osd_map got epochs [2,2] > 0 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 3 client.14150.objecter handle_osd_map decoding full epoch 2 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9f5c96700 1 -- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 msgr2=0x7fa9dc0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3506000666 2024-02-18T00:24:07.555 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9f5c96700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 0x7fa9dc0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:24:07.556 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.549+0000 7fa9e77fe700 20 client.14150.objecter dump_active .. 0 homeless 2024-02-18T00:24:07.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:07.432+0000 7f7590c9f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:24:07.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:07.518+0000 7f7590c9f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T00:24:07.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:07.615+0000 7f7590c9f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T00:24:07.753 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.750+0000 7fa9f5c96700 1 -- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 msgr2=0x7fa9dc0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3506000666 2024-02-18T00:24:07.753 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:07.750+0000 7fa9f5c96700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 0x7fa9dc0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-18T00:24:08.113 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:07.817+0000 7f7590c9f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T00:24:08.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:08.151+0000 7fa9f5c96700 1 -- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 msgr2=0x7fa9dc0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3506000666 2024-02-18T00:24:08.153 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:08.151+0000 7fa9f5c96700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 0x7fa9dc0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-18T00:24:08.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:08.110+0000 7f7590c9f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T00:24:08.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:08.214+0000 7f7590c9f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T00:24:08.954 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:08.952+0000 7fa9f5c96700 1 -- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 msgr2=0x7fa9dc0364f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/3506000666 2024-02-18T00:24:08.954 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:08.952+0000 7fa9f5c96700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 0x7fa9dc0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-18T00:24:09.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:08.983+0000 7f7590c9f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T00:24:09.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:09.069+0000 7f7590c9f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T00:24:09.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:09.164+0000 7f7590c9f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T00:24:09.641 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:09.347+0000 7f7590c9f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T00:24:09.917 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:09.638+0000 7f7590c9f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T00:24:09.917 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:09.776+0000 7f7590c9f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T00:24:09.917 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:09.915+0000 7f7590c9f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:24:10.183 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:10.094+0000 7f7590c9f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T00:24:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:10.196+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 10) v1 ==== 38454+0+0 (secure 0 0 0) 0x7fa9ec029d00 con 0x7fa9f0070c20 2024-02-18T00:24:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:10.197+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 msgr2=0x7fa9dc0364f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:24:10.199 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:10.197+0000 7fa9e77fe700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 0x7fa9dc0364f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:24:10.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:24:10.181+0000 7f7590c9f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T00:24:10.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [79B blob data] 2024-02-18T00:24:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:10.195194+0000 mon.a (mon.0) 66 : cluster [INF] Active manager daemon y restarted 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:10.195488+0000 mon.a (mon.0) 67 : cluster [INF] Activating manager daemon y 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:10.199029+0000 mon.a (mon.0) 68 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:10.199144+0000 mon.a (mon.0) 69 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00377848s) 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.200086+0000 mon.a (mon.0) 70 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.200743+0000 mon.a (mon.0) 71 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.201457+0000 mon.a (mon.0) 72 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.201666+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.201786+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T00:24:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:10.215532+0000 mon.a (mon.0) 75 : cluster [INF] Manager daemon y is now available 2024-02-18T00:24:11.200 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.197+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 11) v1 ==== 38489+0+0 (secure 0 0 0) 0x7fa9ec01abd0 con 0x7fa9f0070c20 2024-02-18T00:24:11.200 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.197+0000 7fa9e77fe700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/817089002 conn(0x7fa9dc037bd0 0x7fa9dc039fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:11.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.197+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 --> v2:172.21.15.59:6800/817089002 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9dc036b70 con 0x7fa9dc037bd0 2024-02-18T00:24:11.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.198+0000 7fa9f5c96700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/817089002 conn(0x7fa9dc037bd0 0x7fa9dc039fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:11.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.198+0000 7fa9f5c96700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/817089002 conn(0x7fa9dc037bd0 0x7fa9dc039fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fa9e0003850 tx=0x7fa9e0006e60).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:11.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.198+0000 7fa9e77fe700 10 client.14150.objecter ms_handle_connect 0x7fa9dc037bd0 2024-02-18T00:24:11.201 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.199+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7fa9dc036b70 con 0x7fa9dc037bd0 2024-02-18T00:24:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 --> v2:172.21.15.59:6800/817089002 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa9f005a310 con 0x7fa9dc037bd0 2024-02-18T00:24:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9e77fe700 1 -- 172.21.15.59:0/790424878 <== mgr.14146 v2:172.21.15.59:6800/817089002 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa9f005a310 con 0x7fa9dc037bd0 2024-02-18T00:24:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: { 2024-02-18T00:24:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-02-18T00:24:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: "initialized": true 2024-02-18T00:24:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: } 2024-02-18T00:24:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 10 librados: watch_flush enter 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 10 librados: watch_flush exit 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 20 client.14150.objecter shutdown clearing up homeless session... 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 10 client.14150.objecter successfully canceled tick 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/817089002 conn(0x7fa9dc037bd0 msgr2=0x7fa9dc039fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/817089002 conn(0x7fa9dc037bd0 0x7fa9dc039fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fa9e0003850 tx=0x7fa9e0006e60).stop 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 msgr2=0x7fa9f010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 --2- 172.21.15.59:0/790424878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f010cf50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa9ec012fd0 tx=0x7fa9ec0090f0).stop 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 shutdown_connections 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/817089002 conn(0x7fa9dc037bd0 0x7fa9dc039fc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 --2- 172.21.15.59:0/790424878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f0070c20 0x7fa9f010cf50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 --2- 172.21.15.59:0/790424878 >> v2:172.21.15.59:6800/3506000666 conn(0x7fa9dc034030 0x7fa9dc0364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.205+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 >> 172.21.15.59:0/790424878 conn(0x7fa9f00f6670 msgr2=0x7fa9f00f7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.206+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 shutdown_connections 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.206+0000 7fa9f86fb700 1 -- 172.21.15.59:0/790424878 wait complete. 2024-02-18T00:24:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:11.206+0000 7fa9f86fb700 1 librados: shutdown 2024-02-18T00:24:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.314877+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:24:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.316270+0000 mon.a (mon.0) 77 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:24:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.342145+0000 mon.a (mon.0) 78 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T00:24:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:10.345243+0000 mon.a (mon.0) 79 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T00:24:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:11.200553+0000 mon.a (mon.0) 80 : cluster [DBG] mgrmap e11: y(active, since 1.00518s) 2024-02-18T00:24:12.347 INFO:teuthology.orchestra.run.smithi059.stderr:mgr epoch 9 is available 2024-02-18T00:24:12.347 INFO:teuthology.orchestra.run.smithi059.stderr:Generating a dashboard self-signed certificate... 2024-02-18T00:24:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:11.199958+0000 mgr.y (mgr.14146) 1 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-18T00:24:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:11.206542+0000 mgr.y (mgr.14146) 2 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-18T00:24:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:12.028878+0000 mon.a (mon.0) 81 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:12.933 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb058ce7700 1 Processor -- start 2024-02-18T00:24:12.933 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb058ce7700 1 -- start start 2024-02-18T00:24:12.933 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb058ce7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb0540fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:12.933 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb058ce7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540ff1d0 con 0x7fb054070c80 2024-02-18T00:24:12.933 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb0540fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:12.933 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb0540fec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48886/0 (socket says 172.21.15.59:48886) 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05259c700 1 -- 172.21.15.59:0/1658775733 learned_addr learned my addr 172.21.15.59:0/1658775733 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05259c700 1 -- 172.21.15.59:0/1658775733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540ff310 con 0x7fb054070c80 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05259c700 1 --2- 172.21.15.59:0/1658775733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb0540fec50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb03c009730 tx=0x7fb03c0090f0).ready entity=mon.0 client_cookie=aedddbee13af1d5b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05159a700 1 -- 172.21.15.59:0/1658775733 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb03c00b580 con 0x7fb054070c80 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05159a700 1 -- 172.21.15.59:0/1658775733 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb03c00bbb0 con 0x7fb054070c80 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.929+0000 7fb05159a700 1 -- 172.21.15.59:0/1658775733 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb03c011bc0 con 0x7fb054070c80 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.930+0000 7fb058ce7700 1 -- 172.21.15.59:0/1658775733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 msgr2=0x7fb0540fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.930+0000 7fb058ce7700 1 --2- 172.21.15.59:0/1658775733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb0540fec50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb03c009730 tx=0x7fb03c0090f0).stop 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.930+0000 7fb058ce7700 1 -- 172.21.15.59:0/1658775733 shutdown_connections 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.930+0000 7fb058ce7700 1 --2- 172.21.15.59:0/1658775733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb0540fec50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.930+0000 7fb058ce7700 1 -- 172.21.15.59:0/1658775733 >> 172.21.15.59:0/1658775733 conn(0x7fb0540f6670 msgr2=0x7fb0540f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.930+0000 7fb058ce7700 1 -- 172.21.15.59:0/1658775733 shutdown_connections 2024-02-18T00:24:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.930+0000 7fb058ce7700 1 -- 172.21.15.59:0/1658775733 wait complete. 2024-02-18T00:24:12.935 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 librados: starting msgr at 2024-02-18T00:24:12.935 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 librados: starting objecter 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 Processor -- start 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 -- start start 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 librados: setting wanted keys 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 librados: calling monclient init 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb05410ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb058ce7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05410b270 con 0x7fb054070c80 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb05259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb05410ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb05259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb05410ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48898/0 (socket says 172.21.15.59:48898) 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.931+0000 7fb05259c700 1 -- 172.21.15.59:0/3931574272 learned_addr learned my addr 172.21.15.59:0/3931574272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb05259c700 1 -- 172.21.15.59:0/3931574272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009370 con 0x7fb054070c80 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb05259c700 1 --2- 172.21.15.59:0/3931574272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb05410ad80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb03c000c00 tx=0x7fb03c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb04b7fe700 10 client.?.objecter ms_handle_connect 0x7fb054070c80 2024-02-18T00:24:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb04b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb03c01a640 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05410b470 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb03c01ac70 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb03c023be0 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb058ce7700 10 client.14156.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05410c0b0 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.932+0000 7fb058ce7700 1 librados: init done 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 38489+0+0 (secure 0 0 0) 0x7fb03c02c3f0 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05410b900 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb04b7fe700 1 --2- 172.21.15.59:0/3931574272 >> v2:172.21.15.59:6800/817089002 conn(0x7fb040033f00 0x7fb0400363c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7fb03c044c00 con 0x7fb054070c80 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb04b7fe700 10 client.14156.objecter ms_dispatch 0x7fb054001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:12.939 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb04b7fe700 3 client.14156.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:12.940 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb04b7fe700 3 client.14156.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:12.940 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.933+0000 7fb04b7fe700 20 client.14156.objecter dump_active .. 0 homeless 2024-02-18T00:24:12.940 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.934+0000 7fb051d9b700 1 --2- 172.21.15.59:0/3931574272 >> v2:172.21.15.59:6800/817089002 conn(0x7fb040033f00 0x7fb0400363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:12.940 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.934+0000 7fb04b7fe700 10 client.14156.objecter ms_handle_connect 0x7fb040033f00 2024-02-18T00:24:12.940 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.934+0000 7fb051d9b700 1 --2- 172.21.15.59:0/3931574272 >> v2:172.21.15.59:6800/817089002 conn(0x7fb040033f00 0x7fb0400363c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb044009780 tx=0x7fb044006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:12.941 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:12.939+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb03c04a020 con 0x7fb054070c80 2024-02-18T00:24:13.030 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.028+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 12) v1 ==== 38595+0+0 (secure 0 0 0) 0x7fb03c029d00 con 0x7fb054070c80 2024-02-18T00:24:13.122 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.120+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb054100340 con 0x7fb040033f00 2024-02-18T00:24:13.287 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.284+0000 7fb04b7fe700 1 -- 172.21.15.59:0/3931574272 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb054100340 con 0x7fb040033f00 2024-02-18T00:24:13.287 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-18T00:24:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 10 librados: watch_flush enter 2024-02-18T00:24:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 10 librados: watch_flush exit 2024-02-18T00:24:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 20 client.14156.objecter shutdown clearing up homeless session... 2024-02-18T00:24:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 10 client.14156.objecter successfully canceled tick 2024-02-18T00:24:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 >> v2:172.21.15.59:6800/817089002 conn(0x7fb040033f00 msgr2=0x7fb0400363c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 --2- 172.21.15.59:0/3931574272 >> v2:172.21.15.59:6800/817089002 conn(0x7fb040033f00 0x7fb0400363c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb044009780 tx=0x7fb044006cc0).stop 2024-02-18T00:24:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 msgr2=0x7fb05410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 --2- 172.21.15.59:0/3931574272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb05410ad80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb03c000c00 tx=0x7fb03c0090f0).stop 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 shutdown_connections 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 --2- 172.21.15.59:0/3931574272 >> v2:172.21.15.59:6800/817089002 conn(0x7fb040033f00 0x7fb0400363c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 --2- 172.21.15.59:0/3931574272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb054070c80 0x7fb05410ad80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 >> 172.21.15.59:0/3931574272 conn(0x7fb0540f6670 msgr2=0x7fb0540f7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 shutdown_connections 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.286+0000 7fb058ce7700 1 -- 172.21.15.59:0/3931574272 wait complete. 2024-02-18T00:24:13.291 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:13.287+0000 7fb058ce7700 1 librados: shutdown 2024-02-18T00:24:14.376 INFO:teuthology.orchestra.run.smithi059.stderr:Creating initial admin user... 2024-02-18T00:24:14.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:13.030312+0000 mon.a (mon.0) 82 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-02-18T00:24:14.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:13.122190+0000 mgr.y (mgr.14146) 3 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:24:14.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:13.283201+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:14.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:13.285666+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.288+0000 7f1b1a283700 1 Processor -- start 2024-02-18T00:24:15.301 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.288+0000 7f1b1a283700 1 -- start start 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b1a283700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b140fec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b1a283700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b140ff1f0 con 0x7f1b14070c60 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b140fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b140fec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48910/0 (socket says 172.21.15.59:48910) 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b13fff700 1 -- 172.21.15.59:0/3891104563 learned_addr learned my addr 172.21.15.59:0/3891104563 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b13fff700 1 -- 172.21.15.59:0/3891104563 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140ff330 con 0x7f1b14070c60 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b13fff700 1 --2- 172.21.15.59:0/3891104563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b140fec70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1b04009730 tx=0x7f1b040090f0).ready entity=mon.0 client_cookie=fd8c42de10c70608 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b12ffd700 1 -- 172.21.15.59:0/3891104563 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b0400b580 con 0x7f1b14070c60 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b12ffd700 1 -- 172.21.15.59:0/3891104563 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0400bbb0 con 0x7f1b14070c60 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.289+0000 7f1b12ffd700 1 -- 172.21.15.59:0/3891104563 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b04011bc0 con 0x7f1b14070c60 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.290+0000 7f1b1a283700 1 -- 172.21.15.59:0/3891104563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 msgr2=0x7f1b140fec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.290+0000 7f1b1a283700 1 --2- 172.21.15.59:0/3891104563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b140fec70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f1b04009730 tx=0x7f1b040090f0).stop 2024-02-18T00:24:15.302 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.290+0000 7f1b1a283700 1 -- 172.21.15.59:0/3891104563 shutdown_connections 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.290+0000 7f1b1a283700 1 --2- 172.21.15.59:0/3891104563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b140fec70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.290+0000 7f1b1a283700 1 -- 172.21.15.59:0/3891104563 >> 172.21.15.59:0/3891104563 conn(0x7f1b140f6670 msgr2=0x7f1b140f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.290+0000 7f1b1a283700 1 -- 172.21.15.59:0/3891104563 shutdown_connections 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.290+0000 7f1b1a283700 1 -- 172.21.15.59:0/3891104563 wait complete. 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 librados: starting msgr at 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 librados: starting objecter 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 Processor -- start 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 -- start start 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 librados: setting wanted keys 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 librados: calling monclient init 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b1410ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b1a283700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1410b250 con 0x7f1b14070c60 2024-02-18T00:24:15.303 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b1410ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b13fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b1410ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48920/0 (socket says 172.21.15.59:48920) 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.291+0000 7f1b13fff700 1 -- 172.21.15.59:0/4242002608 learned_addr learned my addr 172.21.15.59:0/4242002608 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b13fff700 1 -- 172.21.15.59:0/4242002608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04009370 con 0x7f1b14070c60 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b13fff700 1 --2- 172.21.15.59:0/4242002608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b1410ad60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1b04006e90 tx=0x7f1b040090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b11775700 10 client.?.objecter ms_handle_connect 0x7f1b14070c60 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b11775700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b11775700 1 -- 172.21.15.59:0/4242002608 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b0401a640 con 0x7f1b14070c60 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b11775700 1 -- 172.21.15.59:0/4242002608 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0401ac70 con 0x7f1b14070c60 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b11775700 1 -- 172.21.15.59:0/4242002608 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b04023b90 con 0x7f1b14070c60 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1410b450 con 0x7f1b14070c60 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b1a283700 10 client.14158.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1410c060 con 0x7f1b14070c60 2024-02-18T00:24:15.304 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.292+0000 7f1b1a283700 1 librados: init done 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 1 -- 172.21.15.59:0/4242002608 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 38595+0+0 (secure 0 0 0) 0x7f1b0402c3f0 con 0x7f1b14070c60 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 1 --2- 172.21.15.59:0/4242002608 >> v2:172.21.15.59:6800/817089002 conn(0x7f1b00034080 0x7f1b00036540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 1 -- 172.21.15.59:0/4242002608 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f1b04044ce0 con 0x7f1b14070c60 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 10 client.14158.objecter ms_dispatch 0x7f1b14001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 3 client.14158.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 3 client.14158.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 20 client.14158.objecter dump_active .. 0 homeless 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1410b8e0 con 0x7f1b14070c60 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b137fe700 1 --2- 172.21.15.59:0/4242002608 >> v2:172.21.15.59:6800/817089002 conn(0x7f1b00034080 0x7f1b00036540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b11775700 10 client.14158.objecter ms_handle_connect 0x7f1b00034080 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.293+0000 7f1b137fe700 1 --2- 172.21.15.59:0/4242002608 >> v2:172.21.15.59:6800/817089002 conn(0x7f1b00034080 0x7f1b00036540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1afc009780 tx=0x7f1afc006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:15.305 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.298+0000 7f1b11775700 1 -- 172.21.15.59:0/4242002608 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1b1410b8e0 con 0x7f1b14070c60 2024-02-18T00:24:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.488+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1b1405a130 con 0x7f1b00034080 2024-02-18T00:24:15.737 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.736+0000 7f1b11775700 1 -- 172.21.15.59:0/4242002608 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1b1405a130 con 0x7f1b00034080 2024-02-18T00:24:15.738 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$9CfbS8FRpJSUhoeqgAHwwewdt4TWs1aK7tC.fd949LzfAvAfFzFiO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708215855, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-18T00:24:15.740 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 10 librados: watch_flush enter 2024-02-18T00:24:15.740 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 10 librados: watch_flush exit 2024-02-18T00:24:15.740 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 20 client.14158.objecter shutdown clearing up homeless session... 2024-02-18T00:24:15.740 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 10 client.14158.objecter successfully canceled tick 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 >> v2:172.21.15.59:6800/817089002 conn(0x7f1b00034080 msgr2=0x7f1b00036540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 --2- 172.21.15.59:0/4242002608 >> v2:172.21.15.59:6800/817089002 conn(0x7f1b00034080 0x7f1b00036540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f1afc009780 tx=0x7f1afc006cc0).stop 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 msgr2=0x7f1b1410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 --2- 172.21.15.59:0/4242002608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b1410ad60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1b04006e90 tx=0x7f1b040090f0).stop 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 shutdown_connections 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 --2- 172.21.15.59:0/4242002608 >> v2:172.21.15.59:6800/817089002 conn(0x7f1b00034080 0x7f1b00036540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 --2- 172.21.15.59:0/4242002608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b14070c60 0x7f1b1410ad60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 >> 172.21.15.59:0/4242002608 conn(0x7f1b140f6670 msgr2=0x7f1b140f7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 shutdown_connections 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 -- 172.21.15.59:0/4242002608 wait complete. 2024-02-18T00:24:15.741 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:15.738+0000 7f1b1a283700 1 librados: shutdown 2024-02-18T00:24:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:15.489069+0000 mgr.y (mgr.14146) 4 : audit [DBG] from='client.14158 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:24:16.403 INFO:teuthology.orchestra.run.smithi059.stderr:Fetching dashboard port number... 2024-02-18T00:24:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:15.736271+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:17.075 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.052+0000 7ff30f960700 1 Processor -- start 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30f960700 1 -- start start 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30f960700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff3080fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30f960700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3080fb8e0 con 0x7ff3080faf40 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30d6fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff3080fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30d6fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff3080fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48924/0 (socket says 172.21.15.59:48924) 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30d6fc700 1 -- 172.21.15.59:0/343376731 learned_addr learned my addr 172.21.15.59:0/343376731 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30d6fc700 1 -- 172.21.15.59:0/343376731 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3080fba20 con 0x7ff3080faf40 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff30d6fc700 1 --2- 172.21.15.59:0/343376731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff3080fb360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff304009730 tx=0x7ff3040090f0).ready entity=mon.0 client_cookie=9762139ff266f313 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.053+0000 7ff2fffff700 1 -- 172.21.15.59:0/343376731 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff30400b580 con 0x7ff3080faf40 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff2fffff700 1 -- 172.21.15.59:0/343376731 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30400bbb0 con 0x7ff3080faf40 2024-02-18T00:24:17.076 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff2fffff700 1 -- 172.21.15.59:0/343376731 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff304011bc0 con 0x7ff3080faf40 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff30f960700 1 -- 172.21.15.59:0/343376731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 msgr2=0x7ff3080fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff30f960700 1 --2- 172.21.15.59:0/343376731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff3080fb360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff304009730 tx=0x7ff3040090f0).stop 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff30f960700 1 -- 172.21.15.59:0/343376731 shutdown_connections 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff30f960700 1 --2- 172.21.15.59:0/343376731 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff3080fb360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff30f960700 1 -- 172.21.15.59:0/343376731 >> 172.21.15.59:0/343376731 conn(0x7ff3080f6660 msgr2=0x7ff3080f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff30f960700 1 -- 172.21.15.59:0/343376731 shutdown_connections 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.054+0000 7ff30f960700 1 -- 172.21.15.59:0/343376731 wait complete. 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 librados: starting msgr at 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 librados: starting objecter 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 Processor -- start 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 -- start start 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 librados: setting wanted keys 2024-02-18T00:24:17.077 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 librados: calling monclient init 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff30810d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30f960700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308070ca0 con 0x7ff3080faf40 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30d6fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff30810d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30d6fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff30810d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48930/0 (socket says 172.21.15.59:48930) 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30d6fc700 1 -- 172.21.15.59:0/321851350 learned_addr learned my addr 172.21.15.59:0/321851350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30d6fc700 1 -- 172.21.15.59:0/321851350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009370 con 0x7ff3080faf40 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff30d6fc700 1 --2- 172.21.15.59:0/321851350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff30810d9e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff304000c00 tx=0x7ff3040090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff2fe7fc700 10 client.?.objecter ms_handle_connect 0x7ff3080faf40 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.055+0000 7ff2fe7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:17.078 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff2fe7fc700 1 -- 172.21.15.59:0/321851350 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff30401a640 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff2fe7fc700 1 -- 172.21.15.59:0/321851350 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30401ac70 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff2fe7fc700 1 -- 172.21.15.59:0/321851350 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff304023b80 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff308070ea0 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff30f960700 10 client.14160.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30810e110 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff30f960700 1 librados: init done 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff2fe7fc700 1 -- 172.21.15.59:0/321851350 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7ff30401a7a0 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.056+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30805a310 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff2fe7fc700 1 --2- 172.21.15.59:0/321851350 >> v2:172.21.15.59:6800/817089002 conn(0x7ff2f4034110 0x7ff2f40365d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff2fe7fc700 1 -- 172.21.15.59:0/321851350 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7ff304044e60 con 0x7ff3080faf40 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff2fe7fc700 10 client.14160.objecter ms_dispatch 0x7ff308001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff2fe7fc700 3 client.14160.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff2fe7fc700 3 client.14160.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:17.079 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff2fe7fc700 20 client.14160.objecter dump_active .. 0 homeless 2024-02-18T00:24:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff30cefb700 1 --2- 172.21.15.59:0/321851350 >> v2:172.21.15.59:6800/817089002 conn(0x7ff2f4034110 0x7ff2f40365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff30cefb700 1 --2- 172.21.15.59:0/321851350 >> v2:172.21.15.59:6800/817089002 conn(0x7ff2f4034110 0x7ff2f40365d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7ff2f8009780 tx=0x7ff2f8006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.057+0000 7ff2fe7fc700 10 client.14160.objecter ms_handle_connect 0x7ff2f4034110 2024-02-18T00:24:17.080 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.061+0000 7ff2fe7fc700 1 -- 172.21.15.59:0/321851350 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff304019570 con 0x7ff3080faf40 2024-02-18T00:24:17.236 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.235+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7ff308046ce0 con 0x7ff3080faf40 2024-02-18T00:24:17.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.235+0000 7ff2fe7fc700 1 -- 172.21.15.59:0/321851350 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7ff30401e070 con 0x7ff3080faf40 2024-02-18T00:24:17.237 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 8443 2024-02-18T00:24:17.239 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 10 librados: watch_flush enter 2024-02-18T00:24:17.239 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 10 librados: watch_flush exit 2024-02-18T00:24:17.239 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 20 client.14160.objecter shutdown clearing up homeless session... 2024-02-18T00:24:17.239 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 10 client.14160.objecter successfully canceled tick 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 >> v2:172.21.15.59:6800/817089002 conn(0x7ff2f4034110 msgr2=0x7ff2f40365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 --2- 172.21.15.59:0/321851350 >> v2:172.21.15.59:6800/817089002 conn(0x7ff2f4034110 0x7ff2f40365d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7ff2f8009780 tx=0x7ff2f8006cc0).stop 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 msgr2=0x7ff30810d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 --2- 172.21.15.59:0/321851350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff30810d9e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7ff304000c00 tx=0x7ff3040090f0).stop 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 shutdown_connections 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 --2- 172.21.15.59:0/321851350 >> v2:172.21.15.59:6800/817089002 conn(0x7ff2f4034110 0x7ff2f40365d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 --2- 172.21.15.59:0/321851350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3080faf40 0x7ff30810d9e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 >> 172.21.15.59:0/321851350 conn(0x7ff3080f6660 msgr2=0x7ff3080f7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 shutdown_connections 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 -- 172.21.15.59:0/321851350 wait complete. 2024-02-18T00:24:17.240 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:17.237+0000 7ff30f960700 1 librados: shutdown 2024-02-18T00:24:17.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:16.738406+0000 mon.a (mon.0) 86 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-02-18T00:24:17.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:17.235785+0000 mon.a (mon.0) 87 : audit [DBG] from='client.? 172.21.15.59:0/321851350' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-18T00:24:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: disabled 2024-02-18T00:24:17.972 INFO:teuthology.orchestra.run.smithi059.stderr:systemctl: inactive 2024-02-18T00:24:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:firewalld.service is not enabled 2024-02-18T00:24:17.973 INFO:teuthology.orchestra.run.smithi059.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-18T00:24:17.976 INFO:teuthology.orchestra.run.smithi059.stderr:Ceph Dashboard is now available at: 2024-02-18T00:24:17.976 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:17.976 INFO:teuthology.orchestra.run.smithi059.stderr: URL: https://smithi059.front.sepia.ceph.com:8443/ 2024-02-18T00:24:17.976 INFO:teuthology.orchestra.run.smithi059.stderr: User: admin 2024-02-18T00:24:17.976 INFO:teuthology.orchestra.run.smithi059.stderr: Password: s58m4vutr9 2024-02-18T00:24:17.976 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:17.977 INFO:teuthology.orchestra.run.smithi059.stderr:Enabling autotune for osd_memory_target 2024-02-18T00:24:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f6650d3e700 1 Processor -- start 2024-02-18T00:24:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f6650d3e700 1 -- start start 2024-02-18T00:24:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f6650d3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f6650d3e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0730a0 con 0x7f664c072c80 2024-02-18T00:24:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f664a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f664a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48932/0 (socket says 172.21.15.59:48932) 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f664a59c700 1 -- 172.21.15.59:0/1150908492 learned_addr learned my addr 172.21.15.59:0/1150908492 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.518+0000 7f664a59c700 1 -- 172.21.15.59:0/1150908492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c071710 con 0x7f664c072c80 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.519+0000 7f664a59c700 1 --2- 172.21.15.59:0/1150908492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c071220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6634009730 tx=0x7f66340090f0).ready entity=mon.0 client_cookie=9061ec74585056d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.519+0000 7f664959a700 1 -- 172.21.15.59:0/1150908492 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f663400b580 con 0x7f664c072c80 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.519+0000 7f664959a700 1 -- 172.21.15.59:0/1150908492 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f663400bbb0 con 0x7f664c072c80 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.519+0000 7f664959a700 1 -- 172.21.15.59:0/1150908492 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6634011bc0 con 0x7f664c072c80 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.519+0000 7f6650d3e700 1 -- 172.21.15.59:0/1150908492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 msgr2=0x7f664c071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.519+0000 7f6650d3e700 1 --2- 172.21.15.59:0/1150908492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c071220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f6634009730 tx=0x7f66340090f0).stop 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.520+0000 7f6650d3e700 1 -- 172.21.15.59:0/1150908492 shutdown_connections 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.520+0000 7f6650d3e700 1 --2- 172.21.15.59:0/1150908492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c071220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.520+0000 7f6650d3e700 1 -- 172.21.15.59:0/1150908492 >> 172.21.15.59:0/1150908492 conn(0x7f664c0f7800 msgr2=0x7f664c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.520+0000 7f6650d3e700 1 -- 172.21.15.59:0/1150908492 shutdown_connections 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.520+0000 7f6650d3e700 1 -- 172.21.15.59:0/1150908492 wait complete. 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.520+0000 7f6650d3e700 1 librados: starting msgr at 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.520+0000 7f6650d3e700 1 librados: starting objecter 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f6650d3e700 1 Processor -- start 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f6650d3e700 1 -- start start 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f6650d3e700 1 librados: setting wanted keys 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f6650d3e700 1 librados: calling monclient init 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f6650d3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c10cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f6650d3e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c10d460 con 0x7f664c072c80 2024-02-18T00:24:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f664a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c10cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f664a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c10cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48942/0 (socket says 172.21.15.59:48942) 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f664a59c700 1 -- 172.21.15.59:0/2965734820 learned_addr learned my addr 172.21.15.59:0/2965734820 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f664a59c700 1 -- 172.21.15.59:0/2965734820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634009370 con 0x7f664c072c80 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f664a59c700 1 --2- 172.21.15.59:0/2965734820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c10cf70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6634011a00 tx=0x7f66340090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f66437fe700 10 client.?.objecter ms_handle_connect 0x7f664c072c80 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.521+0000 7f66437fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.522+0000 7f66437fe700 1 -- 172.21.15.59:0/2965734820 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f663401a430 con 0x7f664c072c80 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.522+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c10d660 con 0x7f664c072c80 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.522+0000 7f66437fe700 1 -- 172.21.15.59:0/2965734820 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f663401aa60 con 0x7f664c072c80 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.522+0000 7f66437fe700 1 -- 172.21.15.59:0/2965734820 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6634023970 con 0x7f664c072c80 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.522+0000 7f6650d3e700 10 client.14162.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.522+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c10daf0 con 0x7f664c072c80 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.522+0000 7f6650d3e700 1 librados: init done 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 1 -- 172.21.15.59:0/2965734820 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f6634023bb0 con 0x7f664c072c80 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 1 --2- 172.21.15.59:0/2965734820 >> v2:172.21.15.59:6800/817089002 conn(0x7f66380340d0 0x7f6638036590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 1 -- 172.21.15.59:0/2965734820 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f6634044bb0 con 0x7f664c072c80 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 10 client.14162.objecter ms_dispatch 0x7f664c001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c046ce0 con 0x7f664c072c80 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 3 client.14162.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 3 client.14162.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 20 client.14162.objecter dump_active .. 0 homeless 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f6649d9b700 1 --2- 172.21.15.59:0/2965734820 >> v2:172.21.15.59:6800/817089002 conn(0x7f66380340d0 0x7f6638036590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f6649d9b700 1 --2- 172.21.15.59:0/2965734820 >> v2:172.21.15.59:6800/817089002 conn(0x7f66380340d0 0x7f6638036590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f663c009780 tx=0x7f663c006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.523+0000 7f66437fe700 10 client.14162.objecter ms_handle_connect 0x7f66380340d0 2024-02-18T00:24:18.530 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.527+0000 7f66437fe700 1 -- 172.21.15.59:0/2965734820 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f663401e020 con 0x7f664c072c80 2024-02-18T00:24:18.701 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.699+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f664c1023d0 con 0x7f664c072c80 2024-02-18T00:24:18.701 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.700+0000 7f66437fe700 1 -- 172.21.15.59:0/2965734820 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6634021320 con 0x7f664c072c80 2024-02-18T00:24:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 10 librados: watch_flush enter 2024-02-18T00:24:18.703 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 10 librados: watch_flush exit 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 20 client.14162.objecter shutdown clearing up homeless session... 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 10 client.14162.objecter successfully canceled tick 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 >> v2:172.21.15.59:6800/817089002 conn(0x7f66380340d0 msgr2=0x7f6638036590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 --2- 172.21.15.59:0/2965734820 >> v2:172.21.15.59:6800/817089002 conn(0x7f66380340d0 0x7f6638036590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f663c009780 tx=0x7f663c006cc0).stop 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 msgr2=0x7f664c10cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 --2- 172.21.15.59:0/2965734820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c10cf70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f6634011a00 tx=0x7f66340090f0).stop 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 shutdown_connections 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 --2- 172.21.15.59:0/2965734820 >> v2:172.21.15.59:6800/817089002 conn(0x7f66380340d0 0x7f6638036590 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 --2- 172.21.15.59:0/2965734820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f664c072c80 0x7f664c10cf70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 >> 172.21.15.59:0/2965734820 conn(0x7f664c0f7800 msgr2=0x7f664c0f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 shutdown_connections 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.701+0000 7f6650d3e700 1 -- 172.21.15.59:0/2965734820 wait complete. 2024-02-18T00:24:18.704 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:18.702+0000 7f6650d3e700 1 librados: shutdown 2024-02-18T00:24:19.924 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.911+0000 7f09c800c700 1 Processor -- start 2024-02-18T00:24:19.924 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.911+0000 7f09c800c700 1 -- start start 2024-02-18T00:24:19.924 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c800c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c00fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:19.924 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c800c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00fb8e0 con 0x7f09c00faf40 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c5da8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c00fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c5da8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c00fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33542/0 (socket says 172.21.15.59:33542) 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c5da8700 1 -- 172.21.15.59:0/365291674 learned_addr learned my addr 172.21.15.59:0/365291674 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c5da8700 1 -- 172.21.15.59:0/365291674 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00fba20 con 0x7f09c00faf40 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c5da8700 1 --2- 172.21.15.59:0/365291674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c00fb360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f09b4009730 tx=0x7f09b40090f0).ready entity=mon.0 client_cookie=dfd23307c2e34e63 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c4da6700 1 -- 172.21.15.59:0/365291674 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b400b580 con 0x7f09c00faf40 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c4da6700 1 -- 172.21.15.59:0/365291674 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b400bbb0 con 0x7f09c00faf40 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.912+0000 7f09c4da6700 1 -- 172.21.15.59:0/365291674 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b4011bc0 con 0x7f09c00faf40 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 -- 172.21.15.59:0/365291674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 msgr2=0x7f09c00fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:19.925 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 --2- 172.21.15.59:0/365291674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c00fb360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f09b4009730 tx=0x7f09b40090f0).stop 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 -- 172.21.15.59:0/365291674 shutdown_connections 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 --2- 172.21.15.59:0/365291674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c00fb360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 -- 172.21.15.59:0/365291674 >> 172.21.15.59:0/365291674 conn(0x7f09c00f6660 msgr2=0x7f09c00f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 -- 172.21.15.59:0/365291674 shutdown_connections 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 -- 172.21.15.59:0/365291674 wait complete. 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 librados: starting msgr at 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.913+0000 7f09c800c700 1 librados: starting objecter 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 Processor -- start 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 -- start start 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 librados: setting wanted keys 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 librados: calling monclient init 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c010ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c010b280 con 0x7f09c00faf40 2024-02-18T00:24:19.926 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c5da8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c010ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c5da8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c010ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33552/0 (socket says 172.21.15.59:33552) 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c5da8700 1 -- 172.21.15.59:0/1013354207 learned_addr learned my addr 172.21.15.59:0/1013354207 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c5da8700 1 -- 172.21.15.59:0/1013354207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4009370 con 0x7f09c00faf40 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c5da8700 1 --2- 172.21.15.59:0/1013354207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c010ad90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f09b4011a00 tx=0x7f09b40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09b2ffd700 10 client.?.objecter ms_handle_connect 0x7f09c00faf40 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09b2ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09b2ffd700 1 -- 172.21.15.59:0/1013354207 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b400be70 con 0x7f09c00faf40 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09b2ffd700 1 -- 172.21.15.59:0/1013354207 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b401ab10 con 0x7f09c00faf40 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c010b480 con 0x7f09c00faf40 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09b2ffd700 1 -- 172.21.15.59:0/1013354207 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09b4023b20 con 0x7f09c00faf40 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 10 client.14164.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c010c090 con 0x7f09c00faf40 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.914+0000 7f09c800c700 1 librados: init done 2024-02-18T00:24:19.927 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09b2ffd700 1 -- 172.21.15.59:0/1013354207 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f09b4023410 con 0x7f09c00faf40 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09b2ffd700 1 --2- 172.21.15.59:0/1013354207 >> v2:172.21.15.59:6800/817089002 conn(0x7f09ac034120 0x7f09ac0365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c010b910 con 0x7f09c00faf40 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09b2ffd700 1 -- 172.21.15.59:0/1013354207 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f09b4044d50 con 0x7f09c00faf40 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09b2ffd700 10 client.14164.objecter ms_dispatch 0x7f09c0001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09b2ffd700 3 client.14164.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09b2ffd700 3 client.14164.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09b2ffd700 20 client.14164.objecter dump_active .. 0 homeless 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.915+0000 7f09c55a7700 1 --2- 172.21.15.59:0/1013354207 >> v2:172.21.15.59:6800/817089002 conn(0x7f09ac034120 0x7f09ac0365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.916+0000 7f09b2ffd700 10 client.14164.objecter ms_handle_connect 0x7f09ac034120 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.916+0000 7f09c55a7700 1 --2- 172.21.15.59:0/1013354207 >> v2:172.21.15.59:6800/817089002 conn(0x7f09ac034120 0x7f09ac0365e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f09bc009780 tx=0x7f09bc006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:19.928 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:19.920+0000 7f09b2ffd700 1 -- 172.21.15.59:0/1013354207 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f09c010b910 con 0x7f09c00faf40 2024-02-18T00:24:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.185+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f09c005a310 con 0x7f09c00faf40 2024-02-18T00:24:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.188+0000 7f09b2ffd700 1 -- 172.21.15.59:0/1013354207 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v24)=0 set mgr/dashboard/cluster/status v24) v1 ==== 153+0+0 (secure 0 0 0) 0x7f09b40400c0 con 0x7f09c00faf40 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 10 librados: watch_flush enter 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 10 librados: watch_flush exit 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 20 client.14164.objecter shutdown clearing up homeless session... 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 10 client.14164.objecter successfully canceled tick 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 >> v2:172.21.15.59:6800/817089002 conn(0x7f09ac034120 msgr2=0x7f09ac0365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 1 --2- 172.21.15.59:0/1013354207 >> v2:172.21.15.59:6800/817089002 conn(0x7f09ac034120 0x7f09ac0365e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f09bc009780 tx=0x7f09bc006cc0).stop 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 msgr2=0x7f09c010ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:20.192 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.189+0000 7f09c800c700 1 --2- 172.21.15.59:0/1013354207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c010ad90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f09b4011a00 tx=0x7f09b40090f0).stop 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.190+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 shutdown_connections 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.190+0000 7f09c800c700 1 --2- 172.21.15.59:0/1013354207 >> v2:172.21.15.59:6800/817089002 conn(0x7f09ac034120 0x7f09ac0365e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.190+0000 7f09c800c700 1 --2- 172.21.15.59:0/1013354207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09c00faf40 0x7f09c010ad90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.190+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 >> 172.21.15.59:0/1013354207 conn(0x7f09c00f6660 msgr2=0x7f09c0070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.190+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 shutdown_connections 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.190+0000 7f09c800c700 1 -- 172.21.15.59:0/1013354207 wait complete. 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: 2024-02-18T00:24:20.190+0000 7f09c800c700 1 librados: shutdown 2024-02-18T00:24:20.193 INFO:teuthology.orchestra.run.smithi059.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-02-18T00:24:20.926 INFO:teuthology.orchestra.run.smithi059.stderr:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-18T00:24:20.926 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.926 INFO:teuthology.orchestra.run.smithi059.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-18T00:24:20.926 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.926 INFO:teuthology.orchestra.run.smithi059.stderr:Or, if you are only running a single cluster on this host: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: ceph telemetry on 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr:For more information see: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: https://docs.ceph.com/en/pacific/mgr/telemetry/ 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-02-18T00:24:20.927 INFO:teuthology.orchestra.run.smithi059.stderr:Bootstrap complete. 2024-02-18T00:24:20.969 INFO:tasks.cephadm:Fetching config... 2024-02-18T00:24:20.970 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:24:20.970 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-18T00:24:21.029 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-18T00:24:21.030 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:24:21.030 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-18T00:24:21.088 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-18T00:24:21.089 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:24:21.089 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-18T00:24:21.157 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-18T00:24:21.158 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:24:21.158 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-18T00:24:21.218 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-18T00:24:21.218 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYK8X+slKTMtlz0QR++yTpV0cbsRmfdctrA9qSxLDP4IuIqg5efnNhND61qakklIEJBgnO2LT/JXCCNfllDI37gjIEZzO0tpmhipXXo4rsdGopIq19ANn7sAAYd2xehW4iAYx22BeDhX0lZ+ObMTzH4u4NUgQnXM0aHi9WVyNi9Mc/nfp+N0v1ES/gHjBVEWtTIVmJlrB7DDrheW6ChqmTe1N2YR0HSPa38Hl4YRtauZuhRe3V8aRit4XTHzyuGSv9Cc5LEmm9ixHiK9Pm6af64b6jDI25GjPL0dAgME5tNL6lFGRTkTFccWmu1X0GqueS+GL0SehaG4yZK+X2KhU3nImkJ7OHEggmJeh4CCeLTfJSYt7/7nauyDxhyKCU7Rj8w3gGa6Vixb37owyUs06SeJZWcwny2HlJRqaUuOSPLKD3TQbKNXMmF3uns1FdP3MShzy6GaAO0dlTstuQUXTl8LY7dHVKadPr513bt7reFcz+ZISJZpZ4pmRHjV0v93c= ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-18T00:24:21.296 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYK8X+slKTMtlz0QR++yTpV0cbsRmfdctrA9qSxLDP4IuIqg5efnNhND61qakklIEJBgnO2LT/JXCCNfllDI37gjIEZzO0tpmhipXXo4rsdGopIq19ANn7sAAYd2xehW4iAYx22BeDhX0lZ+ObMTzH4u4NUgQnXM0aHi9WVyNi9Mc/nfp+N0v1ES/gHjBVEWtTIVmJlrB7DDrheW6ChqmTe1N2YR0HSPa38Hl4YRtauZuhRe3V8aRit4XTHzyuGSv9Cc5LEmm9ixHiK9Pm6af64b6jDI25GjPL0dAgME5tNL6lFGRTkTFccWmu1X0GqueS+GL0SehaG4yZK+X2KhU3nImkJ7OHEggmJeh4CCeLTfJSYt7/7nauyDxhyKCU7Rj8w3gGa6Vixb37owyUs06SeJZWcwny2HlJRqaUuOSPLKD3TQbKNXMmF3uns1FdP3MShzy6GaAO0dlTstuQUXTl8LY7dHVKadPr513bt7reFcz+ZISJZpZ4pmRHjV0v93c= ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:24:21.307 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYK8X+slKTMtlz0QR++yTpV0cbsRmfdctrA9qSxLDP4IuIqg5efnNhND61qakklIEJBgnO2LT/JXCCNfllDI37gjIEZzO0tpmhipXXo4rsdGopIq19ANn7sAAYd2xehW4iAYx22BeDhX0lZ+ObMTzH4u4NUgQnXM0aHi9WVyNi9Mc/nfp+N0v1ES/gHjBVEWtTIVmJlrB7DDrheW6ChqmTe1N2YR0HSPa38Hl4YRtauZuhRe3V8aRit4XTHzyuGSv9Cc5LEmm9ixHiK9Pm6af64b6jDI25GjPL0dAgME5tNL6lFGRTkTFccWmu1X0GqueS+GL0SehaG4yZK+X2KhU3nImkJ7OHEggmJeh4CCeLTfJSYt7/7nauyDxhyKCU7Rj8w3gGa6Vixb37owyUs06SeJZWcwny2HlJRqaUuOSPLKD3TQbKNXMmF3uns1FdP3MShzy6GaAO0dlTstuQUXTl8LY7dHVKadPr513bt7reFcz+ZISJZpZ4pmRHjV0v93c= ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-18T00:24:21.351 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYK8X+slKTMtlz0QR++yTpV0cbsRmfdctrA9qSxLDP4IuIqg5efnNhND61qakklIEJBgnO2LT/JXCCNfllDI37gjIEZzO0tpmhipXXo4rsdGopIq19ANn7sAAYd2xehW4iAYx22BeDhX0lZ+ObMTzH4u4NUgQnXM0aHi9WVyNi9Mc/nfp+N0v1ES/gHjBVEWtTIVmJlrB7DDrheW6ChqmTe1N2YR0HSPa38Hl4YRtauZuhRe3V8aRit4XTHzyuGSv9Cc5LEmm9ixHiK9Pm6af64b6jDI25GjPL0dAgME5tNL6lFGRTkTFccWmu1X0GqueS+GL0SehaG4yZK+X2KhU3nImkJ7OHEggmJeh4CCeLTfJSYt7/7nauyDxhyKCU7Rj8w3gGa6Vixb37owyUs06SeJZWcwny2HlJRqaUuOSPLKD3TQbKNXMmF3uns1FdP3MShzy6GaAO0dlTstuQUXTl8LY7dHVKadPr513bt7reFcz+ZISJZpZ4pmRHjV0v93c= ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:24:21.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-18T00:24:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:20.188094+0000 mon.a (mon.0) 88 : audit [INF] from='client.? 172.21.15.59:0/1013354207' entity='client.admin' 2024-02-18T00:24:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.181+0000 7f673fe74700 1 -- 172.21.15.59:0/2889894401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 msgr2=0x7f67380f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.181+0000 7f673fe74700 1 --2- 172.21.15.59:0/2889894401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f67380f68d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f6728004770 tx=0x7f6728009cb0).stop 2024-02-18T00:24:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.181+0000 7f673fe74700 1 -- 172.21.15.59:0/2889894401 shutdown_connections 2024-02-18T00:24:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.181+0000 7f673fe74700 1 --2- 172.21.15.59:0/2889894401 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f67380f68d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.181+0000 7f673fe74700 1 -- 172.21.15.59:0/2889894401 >> 172.21.15.59:0/2889894401 conn(0x7f67380f1c10 msgr2=0x7f67380f4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.181+0000 7f673fe74700 1 -- 172.21.15.59:0/2889894401 shutdown_connections 2024-02-18T00:24:24.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.181+0000 7f673fe74700 1 -- 172.21.15.59:0/2889894401 wait complete. 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 librados: starting msgr at 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 librados: starting objecter 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 Processor -- start 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 -- start start 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 librados: setting wanted keys 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 librados: calling monclient init 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f673810c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673fe74700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6738107a90 con 0x7f67380f64b0 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673dc10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f673810c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673dc10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f673810c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33566/0 (socket says 172.21.15.59:33566) 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673dc10700 1 -- 172.21.15.59:0/1477447929 learned_addr learned my addr 172.21.15.59:0/1477447929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673dc10700 1 -- 172.21.15.59:0/1477447929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6728005040 con 0x7f67380f64b0 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f673dc10700 1 --2- 172.21.15.59:0/1477447929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f673810c0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f6728013560 tx=0x7f6728009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:24.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f672effd700 10 client.?.objecter ms_handle_connect 0x7f67380f64b0 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.182+0000 7f672effd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f672effd700 1 -- 172.21.15.59:0/1477447929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f672800e030 con 0x7f67380f64b0 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f672effd700 1 -- 172.21.15.59:0/1477447929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6728009710 con 0x7f67380f64b0 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6738061190 con 0x7f67380f64b0 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f672effd700 1 -- 172.21.15.59:0/1477447929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6728018070 con 0x7f67380f64b0 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f673fe74700 10 client.14166.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673810c590 con 0x7f67380f64b0 2024-02-18T00:24:24.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f673fe74700 1 librados: init done 2024-02-18T00:24:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.183+0000 7f672effd700 1 -- 172.21.15.59:0/1477447929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f672801c450 con 0x7f67380f64b0 2024-02-18T00:24:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673805a310 con 0x7f67380f64b0 2024-02-18T00:24:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f672effd700 1 --2- 172.21.15.59:0/1477447929 >> v2:172.21.15.59:6800/817089002 conn(0x7f6724033f60 0x7f6724036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f672effd700 1 -- 172.21.15.59:0/1477447929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f67280442c0 con 0x7f67380f64b0 2024-02-18T00:24:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f672effd700 10 client.14166.objecter ms_dispatch 0x7f6738001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f672effd700 3 client.14166.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:24.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f672effd700 3 client.14166.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:24.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f672effd700 20 client.14166.objecter dump_active .. 0 homeless 2024-02-18T00:24:24.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f673d40f700 1 --2- 172.21.15.59:0/1477447929 >> v2:172.21.15.59:6800/817089002 conn(0x7f6724033f60 0x7f6724036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:24.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f672effd700 10 client.14166.objecter ms_handle_connect 0x7f6724033f60 2024-02-18T00:24:24.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.184+0000 7f673d40f700 1 --2- 172.21.15.59:0/1477447929 >> v2:172.21.15.59:6800/817089002 conn(0x7f6724033f60 0x7f6724036420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f6734009780 tx=0x7f6734006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:24.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.188+0000 7f672effd700 1 -- 172.21.15.59:0/1477447929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f672801d940 con 0x7f67380f64b0 2024-02-18T00:24:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.361+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f67380fb870 con 0x7f67380f64b0 2024-02-18T00:24:24.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.367+0000 7f672effd700 1 -- 172.21.15.59:0/1477447929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6728025b80 con 0x7f67380f64b0 2024-02-18T00:24:24.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.368+0000 7f673fe74700 10 librados: watch_flush enter 2024-02-18T00:24:24.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 10 librados: watch_flush exit 2024-02-18T00:24:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 20 client.14166.objecter shutdown clearing up homeless session... 2024-02-18T00:24:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 10 client.14166.objecter successfully canceled tick 2024-02-18T00:24:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 >> v2:172.21.15.59:6800/817089002 conn(0x7f6724033f60 msgr2=0x7f6724036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 --2- 172.21.15.59:0/1477447929 >> v2:172.21.15.59:6800/817089002 conn(0x7f6724033f60 0x7f6724036420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f6734009780 tx=0x7f6734006cc0).stop 2024-02-18T00:24:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 msgr2=0x7f673810c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:24.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 --2- 172.21.15.59:0/1477447929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f673810c0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f6728013560 tx=0x7f6728009cb0).stop 2024-02-18T00:24:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 shutdown_connections 2024-02-18T00:24:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 --2- 172.21.15.59:0/1477447929 >> v2:172.21.15.59:6800/817089002 conn(0x7f6724033f60 0x7f6724036420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 --2- 172.21.15.59:0/1477447929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67380f64b0 0x7f673810c0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 >> 172.21.15.59:0/1477447929 conn(0x7f67380f1c10 msgr2=0x7f67380f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 shutdown_connections 2024-02-18T00:24:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 -- 172.21.15.59:0/1477447929 wait complete. 2024-02-18T00:24:24.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:24.369+0000 7f673fe74700 1 librados: shutdown 2024-02-18T00:24:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:23.721798+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:24.067617+0000 mon.a (mon.0) 90 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:24:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:24.070743+0000 mon.a (mon.0) 91 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:24.072044+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:24:25.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:24.078207+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:25.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:24.367193+0000 mon.a (mon.0) 94 : audit [INF] from='client.? 172.21.15.59:0/1477447929' entity='client.admin' 2024-02-18T00:24:25.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:24.393207+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:24:25.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:24.399745+0000 mon.a (mon.0) 96 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:25.156 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-18T00:24:25.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-18T00:24:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 -- 172.21.15.59:0/4028508236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 msgr2=0x7fed8c0f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 --2- 172.21.15.59:0/4028508236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c0f79e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fed80004770 tx=0x7fed80009cb0).stop 2024-02-18T00:24:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 -- 172.21.15.59:0/4028508236 shutdown_connections 2024-02-18T00:24:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 --2- 172.21.15.59:0/4028508236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c0f79e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 -- 172.21.15.59:0/4028508236 >> 172.21.15.59:0/4028508236 conn(0x7fed8c0f2d40 msgr2=0x7fed8c0f5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 -- 172.21.15.59:0/4028508236 shutdown_connections 2024-02-18T00:24:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 -- 172.21.15.59:0/4028508236 wait complete. 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 librados: starting msgr at 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 librados: starting objecter 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.350+0000 7fed91981700 1 Processor -- start 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed91981700 1 -- start start 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed91981700 1 librados: setting wanted keys 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed91981700 1 librados: calling monclient init 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed91981700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed91981700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c107440 con 0x7fed8c0f75c0 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed8affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed8affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33592/0 (socket says 172.21.15.59:33592) 2024-02-18T00:24:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed8affd700 1 -- 172.21.15.59:0/1987391805 learned_addr learned my addr 172.21.15.59:0/1987391805 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed8affd700 1 -- 172.21.15.59:0/1987391805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80005040 con 0x7fed8c0f75c0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed8affd700 1 --2- 172.21.15.59:0/1987391805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c106f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fed80013560 tx=0x7fed80009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed73fff700 10 client.?.objecter ms_handle_connect 0x7fed8c0f75c0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed73fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed73fff700 1 -- 172.21.15.59:0/1987391805 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed8000e030 con 0x7fed8c0f75c0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed73fff700 1 -- 172.21.15.59:0/1987391805 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed80009710 con 0x7fed8c0f75c0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.351+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c107640 con 0x7fed8c0f75c0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.352+0000 7fed73fff700 1 -- 172.21.15.59:0/1987391805 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed80018070 con 0x7fed8c0f75c0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.352+0000 7fed91981700 10 client.14168.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.352+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c107ad0 con 0x7fed8c0f75c0 2024-02-18T00:24:27.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.352+0000 7fed91981700 1 librados: init done 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.352+0000 7fed73fff700 1 -- 172.21.15.59:0/1987391805 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7fed8001c450 con 0x7fed8c0f75c0 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed73fff700 1 --2- 172.21.15.59:0/1987391805 >> v2:172.21.15.59:6800/817089002 conn(0x7fed74033f60 0x7fed74036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c05a310 con 0x7fed8c0f75c0 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed73fff700 1 -- 172.21.15.59:0/1987391805 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7fed80044510 con 0x7fed8c0f75c0 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed73fff700 10 client.14168.objecter ms_dispatch 0x7fed8c001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed73fff700 3 client.14168.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed73fff700 3 client.14168.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed8a7fc700 1 --2- 172.21.15.59:0/1987391805 >> v2:172.21.15.59:6800/817089002 conn(0x7fed74033f60 0x7fed74036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed73fff700 20 client.14168.objecter dump_active .. 0 homeless 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed73fff700 10 client.14168.objecter ms_handle_connect 0x7fed74033f60 2024-02-18T00:24:27.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.353+0000 7fed8a7fc700 1 --2- 172.21.15.59:0/1987391805 >> v2:172.21.15.59:6800/817089002 conn(0x7fed74033f60 0x7fed74036420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fed7c009780 tx=0x7fed7c006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.357+0000 7fed73fff700 1 -- 172.21.15.59:0/1987391805 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fed8006ff60 con 0x7fed8c0f75c0 2024-02-18T00:24:27.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.533+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fed8c0f3cd0 con 0x7fed74033f60 2024-02-18T00:24:27.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.536+0000 7fed73fff700 1 -- 172.21.15.59:0/1987391805 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fed8c0f3cd0 con 0x7fed74033f60 2024-02-18T00:24:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 10 librados: watch_flush enter 2024-02-18T00:24:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 10 librados: watch_flush exit 2024-02-18T00:24:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 20 client.14168.objecter shutdown clearing up homeless session... 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 10 client.14168.objecter successfully canceled tick 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 >> v2:172.21.15.59:6800/817089002 conn(0x7fed74033f60 msgr2=0x7fed74036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 1 --2- 172.21.15.59:0/1987391805 >> v2:172.21.15.59:6800/817089002 conn(0x7fed74033f60 0x7fed74036420 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fed7c009780 tx=0x7fed7c006cc0).stop 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 msgr2=0x7fed8c106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.537+0000 7fed91981700 1 --2- 172.21.15.59:0/1987391805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c106f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fed80013560 tx=0x7fed80009cb0).stop 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.538+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 shutdown_connections 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.538+0000 7fed91981700 1 --2- 172.21.15.59:0/1987391805 >> v2:172.21.15.59:6800/817089002 conn(0x7fed74033f60 0x7fed74036420 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.538+0000 7fed91981700 1 --2- 172.21.15.59:0/1987391805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed8c0f75c0 0x7fed8c106f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:27.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.538+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 >> 172.21.15.59:0/1987391805 conn(0x7fed8c0f2d40 msgr2=0x7fed8c0f35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:27.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.538+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 shutdown_connections 2024-02-18T00:24:27.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.538+0000 7fed91981700 1 -- 172.21.15.59:0/1987391805 wait complete. 2024-02-18T00:24:27.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:24:27.538+0000 7fed91981700 1 librados: shutdown 2024-02-18T00:24:28.230 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-02-18T00:24:28.230 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:24:28.230 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-02-18T00:24:28.252 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:24:28.252 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:24:28.314 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-02-18T00:24:28.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch host add smithi137 2024-02-18T00:24:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:27.537068+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:27.570652+0000 mon.a (mon.0) 98 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:24:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:27.571993+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:24:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:27.572601+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:24:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:24:27.573606+0000 mgr.y (mgr.14146) 6 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.conf 2024-02-18T00:24:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:27.682643+0000 mon.a (mon.0) 101 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:27.687522+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:24:27.612643+0000 mgr.y (mgr.14146) 7 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:24:31.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:30.202850+0000 mgr.y (mgr.14146) 8 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:33.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:32.203302+0000 mgr.y (mgr.14146) 9 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:34.203571+0000 mgr.y (mgr.14146) 10 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:36.203869+0000 mgr.y (mgr.14146) 11 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:38.204135+0000 mgr.y (mgr.14146) 12 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:40.204404+0000 mgr.y (mgr.14146) 13 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:42.204715+0000 mgr.y (mgr.14146) 14 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:44.204922+0000 mgr.y (mgr.14146) 15 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:46.205190+0000 mgr.y (mgr.14146) 16 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:49.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:48.205451+0000 mgr.y (mgr.14146) 17 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:51.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:50.205809+0000 mgr.y (mgr.14146) 18 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:52.206061+0000 mgr.y (mgr.14146) 19 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:54.206378+0000 mgr.y (mgr.14146) 20 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:57.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 -- 172.21.15.137:0/4077603466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 msgr2=0x7f5d0c0f79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:57.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 --2- 172.21.15.137:0/4077603466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c0f79e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5d00004770 tx=0x7f5d00009cb0).stop 2024-02-18T00:24:57.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 -- 172.21.15.137:0/4077603466 shutdown_connections 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 --2- 172.21.15.137:0/4077603466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c0f79e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 -- 172.21.15.137:0/4077603466 >> 172.21.15.137:0/4077603466 conn(0x7f5d0c0f2d40 msgr2=0x7f5d0c0f5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 -- 172.21.15.137:0/4077603466 shutdown_connections 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 -- 172.21.15.137:0/4077603466 wait complete. 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 librados: starting msgr at 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 librados: starting objecter 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 Processor -- start 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 -- start start 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 librados: setting wanted keys 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.445+0000 7f5d12186700 1 librados: calling monclient init 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d12186700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c06cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d12186700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c06b030 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c06cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c06cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.137:42984/0 (socket says 172.21.15.137:42984) 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d0b7fe700 1 -- 172.21.15.137:0/808386686 learned_addr learned my addr 172.21.15.137:0/808386686 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d0b7fe700 1 -- 172.21.15.137:0/808386686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00005040 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d0b7fe700 1 --2- 172.21.15.137:0/808386686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c06cbb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5d00013560 tx=0x7f5d00009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d08ff9700 10 client.?.objecter ms_handle_connect 0x7f5d0c0f75c0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d08ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d08ff9700 1 -- 172.21.15.137:0/808386686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d0000e030 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.446+0000 7f5d08ff9700 1 -- 172.21.15.137:0/808386686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d00009710 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c06b1d0 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d08ff9700 1 -- 172.21.15.137:0/808386686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d00018070 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d12186700 10 client.14170.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:24:57.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c06b660 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d12186700 1 librados: init done 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d08ff9700 1 -- 172.21.15.137:0/808386686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f5d0001d420 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c05a310 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d08ff9700 1 --2- 172.21.15.137:0/808386686 >> v2:172.21.15.59:6800/817089002 conn(0x7f5cf4033f60 0x7f5cf4036420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d08ff9700 1 -- 172.21.15.137:0/808386686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f5d00044300 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d08ff9700 10 client.14170.objecter ms_dispatch 0x7f5d0c001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d08ff9700 3 client.14170.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.447+0000 7f5d08ff9700 3 client.14170.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.448+0000 7f5d08ff9700 20 client.14170.objecter dump_active .. 0 homeless 2024-02-18T00:24:57.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.448+0000 7f5d0affd700 1 --2- 172.21.15.137:0/808386686 >> v2:172.21.15.59:6800/817089002 conn(0x7f5cf4033f60 0x7f5cf4036420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:24:57.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.448+0000 7f5d08ff9700 10 client.14170.objecter ms_handle_connect 0x7f5cf4033f60 2024-02-18T00:24:57.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.448+0000 7f5d0affd700 1 --2- 172.21.15.137:0/808386686 >> v2:172.21.15.59:6800/817089002 conn(0x7f5cf4033f60 0x7f5cf4036420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5cfc009780 tx=0x7f5cfc006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:24:57.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.453+0000 7f5d08ff9700 1 -- 172.21.15.137:0/808386686 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5d00025b80 con 0x7f5d0c0f75c0 2024-02-18T00:24:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:56.206736+0000 mgr.y (mgr.14146) 21 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:57.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:57.618+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d0c0f3cd0 con 0x7f5cf4033f60 2024-02-18T00:24:58.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [90B blob data] 2024-02-18T00:24:58.247 INFO:teuthology.orchestra.run.smithi137.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-02-18T00:24:58.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.244+0000 7f5d08ff9700 1 -- 172.21.15.137:0/808386686 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5d0c0f3cd0 con 0x7f5cf4033f60 2024-02-18T00:24:58.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.246+0000 7f5d12186700 10 librados: watch_flush enter 2024-02-18T00:24:58.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 10 librados: watch_flush exit 2024-02-18T00:24:58.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 20 client.14170.objecter shutdown clearing up homeless session... 2024-02-18T00:24:58.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 10 client.14170.objecter successfully canceled tick 2024-02-18T00:24:58.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 >> v2:172.21.15.59:6800/817089002 conn(0x7f5cf4033f60 msgr2=0x7f5cf4036420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:58.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 --2- 172.21.15.137:0/808386686 >> v2:172.21.15.59:6800/817089002 conn(0x7f5cf4033f60 0x7f5cf4036420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5cfc009780 tx=0x7f5cfc006cc0).stop 2024-02-18T00:24:58.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 msgr2=0x7f5d0c06cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:24:58.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 --2- 172.21.15.137:0/808386686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c06cbb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5d00013560 tx=0x7f5d00009cb0).stop 2024-02-18T00:24:58.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 shutdown_connections 2024-02-18T00:24:58.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 --2- 172.21.15.137:0/808386686 >> v2:172.21.15.59:6800/817089002 conn(0x7f5cf4033f60 0x7f5cf4036420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:58.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 --2- 172.21.15.137:0/808386686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d0c0f75c0 0x7f5d0c06cbb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:24:58.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 >> 172.21.15.137:0/808386686 conn(0x7f5d0c0f2d40 msgr2=0x7f5d0c0f35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:24:58.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 shutdown_connections 2024-02-18T00:24:58.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 -- 172.21.15.137:0/808386686 wait complete. 2024-02-18T00:24:58.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:24:58.247+0000 7f5d12186700 1 librados: shutdown 2024-02-18T00:24:58.752 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:24:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [80B blob data] 2024-02-18T00:24:58.845 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-18T00:24:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:57.620045+0000 mgr.y (mgr.14146) 22 : audit [DBG] from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:24:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:24:58.207200+0000 mgr.y (mgr.14146) 23 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:24:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:58.244731+0000 mon.a (mon.0) 103 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:24:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:24:58.245136+0000 mgr.y (mgr.14146) 24 : cephadm [INF] Added host smithi137 2024-02-18T00:24:59.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:58.283871+0000 mon.a (mon.0) 104 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:24:59.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:24:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:24:58.885025+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.585+0000 7fccd26b1700 1 -- 172.21.15.137:0/2475423198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 msgr2=0x7fcccc0fe310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.585+0000 7fccd26b1700 1 --2- 172.21.15.137:0/2475423198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc0fe310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fccbc004770 tx=0x7fccbc009cb0).stop 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.585+0000 7fccd26b1700 1 -- 172.21.15.137:0/2475423198 shutdown_connections 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.585+0000 7fccd26b1700 1 --2- 172.21.15.137:0/2475423198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc0fe310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.585+0000 7fccd26b1700 1 -- 172.21.15.137:0/2475423198 >> 172.21.15.137:0/2475423198 conn(0x7fcccc06ce30 msgr2=0x7fcccc06f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.585+0000 7fccd26b1700 1 -- 172.21.15.137:0/2475423198 shutdown_connections 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.585+0000 7fccd26b1700 1 -- 172.21.15.137:0/2475423198 wait complete. 2024-02-18T00:25:00.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 librados: starting msgr at 2024-02-18T00:25:00.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 librados: starting objecter 2024-02-18T00:25:00.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 Processor -- start 2024-02-18T00:25:00.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 -- start start 2024-02-18T00:25:00.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 librados: setting wanted keys 2024-02-18T00:25:00.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 librados: calling monclient init 2024-02-18T00:25:00.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc10d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:00.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.586+0000 7fccd26b1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc10da90 con 0x7fcccc0fdef0 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fcccbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc10d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fcccbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc10d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.137:43000/0 (socket says 172.21.15.137:43000) 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fcccbfff700 1 -- 172.21.15.137:0/1359246810 learned_addr learned my addr 172.21.15.137:0/1359246810 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fcccbfff700 1 -- 172.21.15.137:0/1359246810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc005040 con 0x7fcccc0fdef0 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fcccbfff700 1 --2- 172.21.15.137:0/1359246810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc10d5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fccbc013560 tx=0x7fccbc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccc97fa700 10 client.?.objecter ms_handle_connect 0x7fcccc0fdef0 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccc97fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccc97fa700 1 -- 172.21.15.137:0/1359246810 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccbc00e030 con 0x7fcccc0fdef0 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccc97fa700 1 -- 172.21.15.137:0/1359246810 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccbc009710 con 0x7fcccc0fdef0 2024-02-18T00:25:00.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccc97fa700 1 -- 172.21.15.137:0/1359246810 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccbc018070 con 0x7fcccc0fdef0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc10dc90 con 0x7fcccc0fdef0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccd26b1700 10 client.14172.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc10e120 con 0x7fcccc0fdef0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.587+0000 7fccd26b1700 1 librados: init done 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.588+0000 7fccc97fa700 1 -- 172.21.15.137:0/1359246810 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7fccbc027440 con 0x7fcccc0fdef0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.588+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc05a310 con 0x7fcccc0fdef0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fccc97fa700 1 --2- 172.21.15.137:0/1359246810 >> v2:172.21.15.59:6800/817089002 conn(0x7fccb4033fb0 0x7fccb4036470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fccc97fa700 1 -- 172.21.15.137:0/1359246810 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7fccbc044340 con 0x7fcccc0fdef0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fccc97fa700 10 client.14172.objecter ms_dispatch 0x7fcccc001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fccc97fa700 3 client.14172.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fccc97fa700 3 client.14172.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fccc97fa700 20 client.14172.objecter dump_active .. 0 homeless 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fcccb7fe700 1 --2- 172.21.15.137:0/1359246810 >> v2:172.21.15.59:6800/817089002 conn(0x7fccb4033fb0 0x7fccb4036470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fcccb7fe700 1 --2- 172.21.15.137:0/1359246810 >> v2:172.21.15.59:6800/817089002 conn(0x7fccb4033fb0 0x7fccb4036470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fccc0009780 tx=0x7fccc0006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:00.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.589+0000 7fccc97fa700 10 client.14172.objecter ms_handle_connect 0x7fccb4033fb0 2024-02-18T00:25:00.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.593+0000 7fccc97fa700 1 -- 172.21.15.137:0/1359246810 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fccbc024300 con 0x7fcccc0fdef0 2024-02-18T00:25:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.769+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcccc06eea0 con 0x7fccb4033fb0 2024-02-18T00:25:00.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.772+0000 7fccc97fa700 1 -- 172.21.15.137:0/1359246810 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fcccc06eea0 con 0x7fccb4033fb0 2024-02-18T00:25:00.774 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:25:00.774 INFO:teuthology.orchestra.run.smithi137.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 10 librados: watch_flush enter 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 10 librados: watch_flush exit 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 20 client.14172.objecter shutdown clearing up homeless session... 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 10 client.14172.objecter successfully canceled tick 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 >> v2:172.21.15.59:6800/817089002 conn(0x7fccb4033fb0 msgr2=0x7fccb4036470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 --2- 172.21.15.137:0/1359246810 >> v2:172.21.15.59:6800/817089002 conn(0x7fccb4033fb0 0x7fccb4036470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fccc0009780 tx=0x7fccc0006cc0).stop 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 msgr2=0x7fcccc10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:00.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 --2- 172.21.15.137:0/1359246810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc10d5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fccbc013560 tx=0x7fccbc009cb0).stop 2024-02-18T00:25:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 shutdown_connections 2024-02-18T00:25:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 --2- 172.21.15.137:0/1359246810 >> v2:172.21.15.59:6800/817089002 conn(0x7fccb4033fb0 0x7fccb4036470 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 --2- 172.21.15.137:0/1359246810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcccc0fdef0 0x7fcccc10d5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 >> 172.21.15.137:0/1359246810 conn(0x7fcccc06ce30 msgr2=0x7fcccc06e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 shutdown_connections 2024-02-18T00:25:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 -- 172.21.15.137:0/1359246810 wait complete. 2024-02-18T00:25:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:00.773+0000 7fccd26b1700 1 librados: shutdown 2024-02-18T00:25:01.379 INFO:tasks.cephadm:Setting crush tunables to jewel 2024-02-18T00:25:01.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd crush tunables jewel 2024-02-18T00:25:01.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:00.115886+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:01.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:00.207501+0000 mgr.y (mgr.14146) 25 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:02.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:00.772371+0000 mgr.y (mgr.14146) 26 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:25:03.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.312+0000 7f6fb4d91700 1 -- 172.21.15.59:0/2694140555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 msgr2=0x7f6fb00fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.312+0000 7f6fb4d91700 1 --2- 172.21.15.59:0/2694140555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb00fc930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f6fa0004770 tx=0x7f6fa0009cb0).stop 2024-02-18T00:25:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.312+0000 7f6fb4d91700 1 -- 172.21.15.59:0/2694140555 shutdown_connections 2024-02-18T00:25:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.312+0000 7f6fb4d91700 1 --2- 172.21.15.59:0/2694140555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb00fc930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.312+0000 7f6fb4d91700 1 -- 172.21.15.59:0/2694140555 >> 172.21.15.59:0/2694140555 conn(0x7f6fb00f2100 msgr2=0x7f6fb00f4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.312+0000 7f6fb4d91700 1 -- 172.21.15.59:0/2694140555 shutdown_connections 2024-02-18T00:25:03.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.312+0000 7f6fb4d91700 1 -- 172.21.15.59:0/2694140555 wait complete. 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 librados: starting msgr at 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 librados: starting objecter 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 Processor -- start 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 -- start start 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 librados: setting wanted keys 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 librados: calling monclient init 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb010f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fb4d91700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb010f690 con 0x7f6fb00fc550 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb010f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fae59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb010f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55708/0 (socket says 172.21.15.59:55708) 2024-02-18T00:25:03.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.313+0000 7f6fae59c700 1 -- 172.21.15.59:0/1788133773 learned_addr learned my addr 172.21.15.59:0/1788133773 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6fae59c700 1 -- 172.21.15.59:0/1788133773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0005040 con 0x7f6fb00fc550 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6fae59c700 1 --2- 172.21.15.59:0/1788133773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb010f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6fa0013a30 tx=0x7f6fa0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6f9f7fe700 10 client.?.objecter ms_handle_connect 0x7f6fb00fc550 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6f9f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6f9f7fe700 1 -- 172.21.15.59:0/1788133773 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fa000e070 con 0x7f6fb00fc550 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6f9f7fe700 1 -- 172.21.15.59:0/1788133773 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa0013630 con 0x7f6fb00fc550 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb010f890 con 0x7f6fb00fc550 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6f9f7fe700 1 -- 172.21.15.59:0/1788133773 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fa0018070 con 0x7f6fb00fc550 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6fb4d91700 10 client.14174.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb010fd20 con 0x7f6fb00fc550 2024-02-18T00:25:03.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.314+0000 7f6fb4d91700 1 librados: init done 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 1 -- 172.21.15.59:0/1788133773 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f6fa00263f0 con 0x7f6fb00fc550 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb005f060 con 0x7f6fb00fc550 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 1 --2- 172.21.15.59:0/1788133773 >> v2:172.21.15.59:6800/817089002 conn(0x7f6f98034060 0x7f6f98036520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 1 -- 172.21.15.59:0/1788133773 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f6fa0043a30 con 0x7f6fb00fc550 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 10 client.14174.objecter ms_dispatch 0x7f6fb0001550 osd_map(3..3 src has 1..3) v4 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 3 client.14174.objecter handle_osd_map got epochs [3,3] > 0 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 3 client.14174.objecter handle_osd_map decoding full epoch 3 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 20 client.14174.objecter dump_active .. 0 homeless 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6fadd9b700 1 --2- 172.21.15.59:0/1788133773 >> v2:172.21.15.59:6800/817089002 conn(0x7f6f98034060 0x7f6f98036520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6f9f7fe700 10 client.14174.objecter ms_handle_connect 0x7f6f98034060 2024-02-18T00:25:03.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.315+0000 7f6fadd9b700 1 --2- 172.21.15.59:0/1788133773 >> v2:172.21.15.59:6800/817089002 conn(0x7f6f98034060 0x7f6f98036520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f6fa4009780 tx=0x7f6fa4006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:03.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.320+0000 7f6f9f7fe700 1 -- 172.21.15.59:0/1788133773 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6fa001a690 con 0x7f6fb00fc550 2024-02-18T00:25:03.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:03.495+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "jewel"} v 0) v1 -- 0x7f6fb005a310 con 0x7f6fb00fc550 2024-02-18T00:25:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:02.207752+0000 mgr.y (mgr.14146) 27 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.124+0000 7f6f9f7fe700 1 -- 172.21.15.59:0/1788133773 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "jewel"}]=0 adjusted tunables profile to jewel v4) v1 ==== 120+0+0 (secure 0 0 0) 0x7f6fa0011320 con 0x7f6fb00fc550 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.126+0000 7f6fb4d91700 10 librados: watch_flush enter 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 10 librados: watch_flush exit 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 20 client.14174.objecter shutdown clearing up homeless session... 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 10 client.14174.objecter successfully canceled tick 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 >> v2:172.21.15.59:6800/817089002 conn(0x7f6f98034060 msgr2=0x7f6f98036520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 --2- 172.21.15.59:0/1788133773 >> v2:172.21.15.59:6800/817089002 conn(0x7f6f98034060 0x7f6f98036520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f6fa4009780 tx=0x7f6fa4006cc0).stop 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 msgr2=0x7f6fb010f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 --2- 172.21.15.59:0/1788133773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb010f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6fa0013a30 tx=0x7f6fa0009cb0).stop 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 shutdown_connections 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 --2- 172.21.15.59:0/1788133773 >> v2:172.21.15.59:6800/817089002 conn(0x7f6f98034060 0x7f6f98036520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 --2- 172.21.15.59:0/1788133773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fb00fc550 0x7f6fb010f1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 >> 172.21.15.59:0/1788133773 conn(0x7f6fb00f2100 msgr2=0x7f6fb0060e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 shutdown_connections 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 -- 172.21.15.59:0/1788133773 wait complete. 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:04.127+0000 7f6fb4d91700 1 librados: shutdown 2024-02-18T00:25:04.130 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to jewel 2024-02-18T00:25:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:03.496531+0000 mon.a (mon.0) 107 : audit [INF] from='client.? 172.21.15.59:0/1788133773' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "jewel"}]: dispatch 2024-02-18T00:25:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:03.901036+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:04.810 INFO:tasks.cephadm:Adding mon.a on smithi059 2024-02-18T00:25:04.810 INFO:tasks.cephadm:Adding mon.c on smithi059 2024-02-18T00:25:04.810 INFO:tasks.cephadm:Adding mon.b on smithi137 2024-02-18T00:25:04.810 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply mon '3;smithi059:172.21.15.59=a;smithi059:[v2:172.21.15.59:3301,v1:172.21.15.59:6790]=c;smithi137:172.21.15.137=b' 2024-02-18T00:25:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:04.125691+0000 mon.a (mon.0) 109 : audit [INF] from='client.? 172.21.15.59:0/1788133773' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "jewel"}]': finished 2024-02-18T00:25:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:04.125743+0000 mon.a (mon.0) 110 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T00:25:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:04.206735+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:04.207625+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:25:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:04.207965+0000 mgr.y (mgr.14146) 28 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:05.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:04.208663+0000 mon.a (mon.0) 113 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:05.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:04.209468+0000 mon.a (mon.0) 114 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:05.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:04.210737+0000 mgr.y (mgr.14146) 29 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.conf 2024-02-18T00:25:05.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:04.232075+0000 mgr.y (mgr.14146) 30 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:25:05.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:04.260833+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:05.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:04.266093+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:06.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.639+0000 7f0aa893c700 1 -- 172.21.15.137:0/3795152587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 msgr2=0x7f0aa40fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:06.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.639+0000 7f0aa893c700 1 --2- 172.21.15.137:0/3795152587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa40fc930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0a98004770 tx=0x7f0a98009cb0).stop 2024-02-18T00:25:06.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.639+0000 7f0aa893c700 1 -- 172.21.15.137:0/3795152587 shutdown_connections 2024-02-18T00:25:06.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.639+0000 7f0aa893c700 1 --2- 172.21.15.137:0/3795152587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa40fc930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:06.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.639+0000 7f0aa893c700 1 -- 172.21.15.137:0/3795152587 >> 172.21.15.137:0/3795152587 conn(0x7f0aa406c440 msgr2=0x7f0aa406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:06.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.639+0000 7f0aa893c700 1 -- 172.21.15.137:0/3795152587 shutdown_connections 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.639+0000 7f0aa893c700 1 -- 172.21.15.137:0/3795152587 wait complete. 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 librados: starting msgr at 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 librados: starting objecter 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 Processor -- start 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 -- start start 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 librados: setting wanted keys 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 librados: calling monclient init 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa410bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa893c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa410c3f0 con 0x7f0aa40fc550 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.640+0000 7f0aa259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa410bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa410bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.137:55154/0 (socket says 172.21.15.137:55154) 2024-02-18T00:25:06.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa259c700 1 -- 172.21.15.137:0/27128960 learned_addr learned my addr 172.21.15.137:0/27128960 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:25:06.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa259c700 1 -- 172.21.15.137:0/27128960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98005040 con 0x7f0aa40fc550 2024-02-18T00:25:06.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa259c700 1 --2- 172.21.15.137:0/27128960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa410bf00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f0a98009b10 tx=0x7f0a98009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0a937fe700 10 client.?.objecter ms_handle_connect 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0a937fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0a937fe700 1 -- 172.21.15.137:0/27128960 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a9800e030 con 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa410c5f0 con 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0a937fe700 1 -- 172.21.15.137:0/27128960 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a98009430 con 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0a937fe700 1 -- 172.21.15.137:0/27128960 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a98018070 con 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa893c700 10 client.14176.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa410ca80 con 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.641+0000 7f0aa893c700 1 librados: init done 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0a937fe700 1 -- 172.21.15.137:0/27128960 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f0a9801d420 con 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa405f1f0 con 0x7f0aa40fc550 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0a937fe700 1 --2- 172.21.15.137:0/27128960 >> v2:172.21.15.59:6800/817089002 conn(0x7f0a8c034000 0x7f0a8c0364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0a937fe700 1 -- 172.21.15.137:0/27128960 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f0a98044240 con 0x7f0aa40fc550 2024-02-18T00:25:06.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0a937fe700 10 client.14176.objecter ms_dispatch 0x7f0aa4001550 osd_map(4..4 src has 1..4) v4 2024-02-18T00:25:06.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0a937fe700 3 client.14176.objecter handle_osd_map got epochs [4,4] > 0 2024-02-18T00:25:06.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0a937fe700 3 client.14176.objecter handle_osd_map decoding full epoch 4 2024-02-18T00:25:06.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0a937fe700 20 client.14176.objecter dump_active .. 0 homeless 2024-02-18T00:25:06.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.642+0000 7f0aa1d9b700 1 --2- 172.21.15.137:0/27128960 >> v2:172.21.15.59:6800/817089002 conn(0x7f0a8c034000 0x7f0a8c0364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:06.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.643+0000 7f0aa1d9b700 1 --2- 172.21.15.137:0/27128960 >> v2:172.21.15.59:6800/817089002 conn(0x7f0a8c034000 0x7f0a8c0364c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f0a94009780 tx=0x7f0a94006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:06.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.643+0000 7f0a937fe700 10 client.14176.objecter ms_handle_connect 0x7f0a8c034000 2024-02-18T00:25:06.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.647+0000 7f0a937fe700 1 -- 172.21.15.137:0/27128960 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0a98025b80 con 0x7f0aa40fc550 2024-02-18T00:25:06.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.818+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi059:[v2:172.21.15.59:3301,v1:172.21.15.59:6790]=c;smithi137:172.21.15.137=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f0aa4107580 con 0x7f0a8c034000 2024-02-18T00:25:06.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.825+0000 7f0a937fe700 1 -- 172.21.15.137:0/27128960 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0aa4107580 con 0x7f0a8c034000 2024-02-18T00:25:06.828 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mon update... 2024-02-18T00:25:06.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 10 librados: watch_flush enter 2024-02-18T00:25:06.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 10 librados: watch_flush exit 2024-02-18T00:25:06.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 20 client.14176.objecter shutdown clearing up homeless session... 2024-02-18T00:25:06.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 10 client.14176.objecter successfully canceled tick 2024-02-18T00:25:06.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 >> v2:172.21.15.59:6800/817089002 conn(0x7f0a8c034000 msgr2=0x7f0a8c0364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 --2- 172.21.15.137:0/27128960 >> v2:172.21.15.59:6800/817089002 conn(0x7f0a8c034000 0x7f0a8c0364c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f0a94009780 tx=0x7f0a94006cc0).stop 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 msgr2=0x7f0aa410bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 --2- 172.21.15.137:0/27128960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa410bf00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f0a98009b10 tx=0x7f0a98009cb0).stop 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 shutdown_connections 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 --2- 172.21.15.137:0/27128960 >> v2:172.21.15.59:6800/817089002 conn(0x7f0a8c034000 0x7f0a8c0364c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 --2- 172.21.15.137:0/27128960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0aa40fc550 0x7f0aa410bf00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 >> 172.21.15.137:0/27128960 conn(0x7f0aa406c440 msgr2=0x7f0aa40f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 shutdown_connections 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.826+0000 7f0aa893c700 1 -- 172.21.15.137:0/27128960 wait complete. 2024-02-18T00:25:06.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:06.827+0000 7f0aa893c700 1 librados: shutdown 2024-02-18T00:25:07.428 DEBUG:teuthology.orchestra.run.smithi059:mon.c> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.c.service 2024-02-18T00:25:07.431 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.b.service 2024-02-18T00:25:07.433 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T00:25:07.433 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T00:25:07.462 INFO:journalctl@ceph.mon.b.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:25:07.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:06.208258+0000 mgr.y (mgr.14146) 31 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.826171+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.836783+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.838423+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.839183+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.845270+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.847005+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.847716+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:06.821004+0000 mgr.y (mgr.14146) 32 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi059:[v2:172.21.15.59:3301,v1:172.21.15.59:6790]=c;smithi137:172.21.15.137=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:25:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:06.823381+0000 mgr.y (mgr.14146) 33 : cephadm [INF] Saving service mon spec with placement smithi059:172.21.15.59=a;smithi059:[v2:172.21.15.59:3301,v1:172.21.15.59:6790]=c;smithi137:172.21.15.137=b;count:3 2024-02-18T00:25:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:06.848381+0000 mgr.y (mgr.14146) 34 : cephadm [INF] Deploying daemon mon.b on smithi137 2024-02-18T00:25:09.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:08.208646+0000 mgr.y (mgr.14146) 35 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:10.209040+0000 mgr.y (mgr.14146) 36 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 -- 172.21.15.137:0/4157087889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 msgr2=0x7f63140f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 --2- 172.21.15.137:0/4157087889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f8a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f6304004770 tx=0x7f6304009cb0).stop 2024-02-18T00:25:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 -- 172.21.15.137:0/4157087889 shutdown_connections 2024-02-18T00:25:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 --2- 172.21.15.137:0/4157087889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f8a70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 -- 172.21.15.137:0/4157087889 >> 172.21.15.137:0/4157087889 conn(0x7f631406c440 msgr2=0x7f631406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 -- 172.21.15.137:0/4157087889 shutdown_connections 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 -- 172.21.15.137:0/4157087889 wait complete. 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 librados: starting msgr at 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 librados: starting objecter 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.406+0000 7f631a02d700 1 Processor -- start 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f631a02d700 1 -- start start 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f631a02d700 1 librados: setting wanted keys 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f631a02d700 1 librados: calling monclient init 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f631a02d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f631a02d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140f6b30 con 0x7f63140f8690 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f63137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f63137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.137:55174/0 (socket says 172.21.15.137:55174) 2024-02-18T00:25:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f63137fe700 1 -- 172.21.15.137:0/3945887698 learned_addr learned my addr 172.21.15.137:0/3945887698 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f63137fe700 1 -- 172.21.15.137:0/3945887698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304005040 con 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f63137fe700 1 --2- 172.21.15.137:0/3945887698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f6640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f6304009b10 tx=0x7f6304009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f6310ff9700 10 client.?.objecter ms_handle_connect 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.407+0000 7f6310ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f6310ff9700 1 -- 172.21.15.137:0/3945887698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f630400e030 con 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63140f6d30 con 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f6310ff9700 1 -- 172.21.15.137:0/3945887698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6304009430 con 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f6310ff9700 1 -- 172.21.15.137:0/3945887698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6304018070 con 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f631a02d700 10 client.14178.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63140f7140 con 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f631a02d700 1 librados: init done 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.408+0000 7f6310ff9700 1 -- 172.21.15.137:0/3945887698 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f630401d420 con 0x7f63140f8690 2024-02-18T00:25:12.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6310ff9700 1 --2- 172.21.15.137:0/3945887698 >> v2:172.21.15.59:6800/817089002 conn(0x7f62fc034000 0x7f62fc0364c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:12.756 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:25:12.756 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":1,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","modified":"2024-02-18T00:23:08.218775Z","created":"2024-02-18T00:23:08.218775Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-18T00:25:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6310ff9700 1 -- 172.21.15.137:0/3945887698 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f6304044250 con 0x7f63140f8690 2024-02-18T00:25:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6310ff9700 10 client.14178.objecter ms_dispatch 0x7f6314001550 osd_map(4..4 src has 1..4) v4 2024-02-18T00:25:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6310ff9700 3 client.14178.objecter handle_osd_map got epochs [4,4] > 0 2024-02-18T00:25:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6310ff9700 3 client.14178.objecter handle_osd_map decoding full epoch 4 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631405f1f0 con 0x7f63140f8690 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6310ff9700 20 client.14178.objecter dump_active .. 0 homeless 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6312ffd700 1 --2- 172.21.15.137:0/3945887698 >> v2:172.21.15.59:6800/817089002 conn(0x7f62fc034000 0x7f62fc0364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6310ff9700 10 client.14178.objecter ms_handle_connect 0x7f62fc034000 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.409+0000 7f6312ffd700 1 --2- 172.21.15.137:0/3945887698 >> v2:172.21.15.59:6800/817089002 conn(0x7f62fc034000 0x7f62fc0364c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6308009780 tx=0x7f6308006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.414+0000 7f6310ff9700 1 -- 172.21.15.137:0/3945887698 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6304009710 con 0x7f63140f8690 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.654+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f631405a310 con 0x7f63140f8690 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.655+0000 7f6310ff9700 1 -- 172.21.15.137:0/3945887698 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+723 (secure 0 0 0) 0x7f631405a310 con 0x7f63140f8690 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 10 librados: watch_flush enter 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 10 librados: watch_flush exit 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 20 client.14178.objecter shutdown clearing up homeless session... 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 10 client.14178.objecter successfully canceled tick 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 >> v2:172.21.15.59:6800/817089002 conn(0x7f62fc034000 msgr2=0x7f62fc0364c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 --2- 172.21.15.137:0/3945887698 >> v2:172.21.15.59:6800/817089002 conn(0x7f62fc034000 0x7f62fc0364c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6308009780 tx=0x7f6308006cc0).stop 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 msgr2=0x7f63140f6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 --2- 172.21.15.137:0/3945887698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f6640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f6304009b10 tx=0x7f6304009cb0).stop 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 shutdown_connections 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 --2- 172.21.15.137:0/3945887698 >> v2:172.21.15.59:6800/817089002 conn(0x7f62fc034000 0x7f62fc0364c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 --2- 172.21.15.137:0/3945887698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63140f8690 0x7f63140f6640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 >> 172.21.15.137:0/3945887698 conn(0x7f631406c440 msgr2=0x7f63140fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 shutdown_connections 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 -- 172.21.15.137:0/3945887698 wait complete. 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:12.657+0000 7f631a02d700 1 librados: shutdown 2024-02-18T00:25:12.758 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 1 2024-02-18T00:25:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 systemd[1]: Starting Ceph mon.b for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:25:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:12.209405+0000 mgr.y (mgr.14146) 37 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:12.656893+0000 mon.a (mon.0) 124 : audit [DBG] from='client.? 172.21.15.137:0/3945887698' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.799+0000 7f703ccbd880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.799+0000 7f703ccbd880 0 ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable), process ceph-mon, pid 7 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.799+0000 7f703ccbd880 0 pidfile_write: ignore empty --pid-file 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.803+0000 7f703ccbd880 0 load: jerasure load: lrc load: isa 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: DB SUMMARY 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: CURRENT file: CURRENT 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 511 ; 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.create_if_missing: 0 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T00:25:14.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.env: 0x5568efb58080 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.fs: Posix File System 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.info_log: 0x5568f1022260 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.statistics: (nil) 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.use_fsync: 0 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T00:25:14.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.db_log_dir: 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T00:25:14.258 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.write_buffer_manager: 0x5568f1d143c0 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T00:25:14.259 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.unordered_write: 0 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T00:25:14.260 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T00:25:14.262 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.row_cache: None 2024-02-18T00:25:14.262 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.wal_filter: None 2024-02-18T00:25:14.262 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T00:25:14.262 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T00:25:14.262 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T00:25:14.263 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_open_files: -1 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Compression algorithms supported: 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kZSTD supported: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T00:25:14.264 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.merge_operator: 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_filter: None 2024-02-18T00:25:14.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5568f0fc7d10) 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cache_index_and_filter_blocks: 1 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: pin_top_level_index_and_filter: 1 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: index_type: 0 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: data_block_index_type: 0 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: index_shortening: 1 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: data_block_hash_table_util_ratio: 0.750000 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: hash_index_allow_collision: 1 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: checksum: 1 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: no_block_cache: 0 2024-02-18T00:25:14.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_cache: 0x5568f0ffcf10 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_cache_name: BinnedLRUCache 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_cache_options: 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: capacity : 536870912 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: num_shard_bits : 4 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: strict_capacity_limit : 0 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: high_pri_pool_ratio: 0.000 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_cache_compressed: (nil) 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: persistent_cache: (nil) 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_size: 4096 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_size_deviation: 10 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_restart_interval: 16 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: index_block_restart_interval: 1 2024-02-18T00:25:14.267 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: metadata_block_size: 4096 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: partition_filters: 0 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: use_delta_encoding: 1 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: whole_key_filtering: 1 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: verify_compression: 0 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: read_amp_bytes_per_bit: 0 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: format_version: 2 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: enable_index_compression: 1 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: block_align: 0 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compression: NoCompression 2024-02-18T00:25:14.268 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.num_levels: 7 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T00:25:14.269 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T00:25:14.270 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T00:25:14.271 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T00:25:14.272 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.ttl: 2592000 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.804+0000 7f703ccbd880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.805+0000 7f703ccbd880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.805+0000 7f703ccbd880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.273 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.805+0000 7f703ccbd880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215913807187, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.805+0000 7f703ccbd880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.805+0000 7f703ccbd880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.805+0000 7f703ccbd880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215913807633, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1437, "table_properties": {"data_size": 523, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708215913, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.805+0000 7f703ccbd880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.806+0000 7f703ccbd880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215913808003, "job": 1, "event": "recovery_finished"} 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.806+0000 7f703ccbd880 4 rocksdb: DB pointer 0x5568f10c5800 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.807+0000 7f7023aaf700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.807+0000 7f7023aaf700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ** DB Stats ** 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T00:25:14.274 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ** Compaction Stats [default] ** 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ** Compaction Stats [default] ** 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:14.275 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.276 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ** Compaction Stats [default] ** 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ** Compaction Stats [default] ** 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:25:14.277 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.807+0000 7f703ccbd880 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.807+0000 7f703ccbd880 0 using public_addr v2:172.21.15.137:0/0 -> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.807+0000 7f703ccbd880 0 starting mon.b rank -1 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.808+0000 7f703ccbd880 1 mon.b@-1(???) e0 preinit fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f7021aab700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #9. Immutable memtables: 0. 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f702babf700 4 rocksdb: (Original Log Time 2024/02/18-00:25:13.830263) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f702babf700 4 rocksdb: [flush_job.cc:321] [default] [JOB 3] Flushing memtable with next log file: 9 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f702babf700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215913830322, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 927, "num_deletes": 10, "total_data_size": 3825316, "memory_usage": 3842816, "flush_reason": "Manual Compaction"} 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f702babf700 4 rocksdb: [flush_job.cc:350] [default] [JOB 3] Level-0 flush table #10: started 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f70262b4700 0 mon.b@-1(synchronizing).mds e1 new map 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f70262b4700 0 mon.b@-1(synchronizing).mds e1 print_map 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: e1 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: legacy client fscid: -1 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: No filesystems configured 2024-02-18T00:25:14.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f70262b4700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.828+0000 7f70262b4700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:12.209405+0000 mgr.y (mgr.14146) 37 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:12.656893+0000 mon.a (mon.0) 124 : audit [DBG] from='client.? 172.21.15.137:0/3945887698' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:25:13.829+0000 7f70262b4700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-18T00:25:14.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 bash[39497]: 50578fab0b8c2d1fda872d9eebbdebff0fa183dd12e8bc339cf02bd20ff844d5 2024-02-18T00:25:14.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:13 smithi137 systemd[1]: Started Ceph mon.b for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:25:14.395 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T00:25:14.396 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T00:25:19.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 -- 172.21.15.137:0/504469579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 msgr2=0x7f2274005890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:19.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 --2- 172.21.15.137:0/504469579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 0x7f2274005890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f228c0f6590 tx=0x7f227c00b040).stop 2024-02-18T00:25:19.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 -- 172.21.15.137:0/504469579 shutdown_connections 2024-02-18T00:25:19.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 --2- 172.21.15.137:0/504469579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 0x7f2274005890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:19.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 -- 172.21.15.137:0/504469579 >> 172.21.15.137:0/504469579 conn(0x7f228c06c440 msgr2=0x7f228c06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:19.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 -- 172.21.15.137:0/504469579 shutdown_connections 2024-02-18T00:25:19.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 -- 172.21.15.137:0/504469579 wait complete. 2024-02-18T00:25:19.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 librados: starting msgr at 2024-02-18T00:25:19.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.152+0000 7f2291198700 1 librados: starting objecter 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.153+0000 7f2291198700 1 Processor -- start 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.153+0000 7f2291198700 1 -- start start 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.153+0000 7f2291198700 1 librados: setting wanted keys 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.153+0000 7f2291198700 1 librados: calling monclient init 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f2291198700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 0x7f228c10d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f2291198700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 0x7f228c107130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f2291198700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c10dc20 con 0x7f2274003400 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f2291198700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c10dd60 con 0x7f228c10d690 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 0x7f228c107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:19.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 0x7f228c10d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228a59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 0x7f228c107130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55548/0 (socket says 172.21.15.137:55548) 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228a59c700 1 -- 172.21.15.137:0/476386069 learned_addr learned my addr 172.21.15.137:0/476386069 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228a59c700 1 -- 172.21.15.137:0/476386069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 msgr2=0x7f228c107130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228a59c700 1 -- 172.21.15.137:0/476386069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 msgr2=0x7f228c107130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228a59c700 1 --2- 172.21.15.137:0/476386069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 0x7f228c107130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228a59c700 1 --2- 172.21.15.137:0/476386069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 0x7f228c107130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228ad9d700 1 -- 172.21.15.137:0/476386069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 msgr2=0x7f228c107130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228ad9d700 1 --2- 172.21.15.137:0/476386069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 0x7f228c107130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228ad9d700 1 -- 172.21.15.137:0/476386069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c005040 con 0x7f2274003400 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f228ad9d700 1 --2- 172.21.15.137:0/476386069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 0x7f228c10d1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f227c016880 tx=0x7f227c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:19.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f2273fff700 10 client.?.objecter ms_handle_connect 0x7f2274003400 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.154+0000 7f2273fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2273fff700 1 -- 172.21.15.137:0/476386069 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f227c004ac0 con 0x7f2274003400 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2273fff700 1 -- 172.21.15.137:0/476386069 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f227c004c20 con 0x7f2274003400 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c107680 con 0x7f2274003400 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2273fff700 1 -- 172.21.15.137:0/476386069 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f227c020530 con 0x7f2274003400 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2291198700 10 client.14182.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c107c40 con 0x7f2274003400 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2291198700 1 librados: init done 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.155+0000 7f2273fff700 1 -- 172.21.15.137:0/476386069 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f227c020690 con 0x7f2274003400 2024-02-18T00:25:19.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2273fff700 1 --2- 172.21.15.137:0/476386069 >> v2:172.21.15.59:6800/817089002 conn(0x7f22780341a0 0x7f2278036660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2273fff700 1 -- 172.21.15.137:0/476386069 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f227c0453d0 con 0x7f2274003400 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2273fff700 10 client.14182.objecter ms_dispatch 0x7f228c001550 osd_map(4..4 src has 1..4) v4 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2273fff700 3 client.14182.objecter handle_osd_map got epochs [4,4] > 0 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2273fff700 3 client.14182.objecter handle_osd_map decoding full epoch 4 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2273fff700 20 client.14182.objecter dump_active .. 0 homeless 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f228a59c700 1 --2- 172.21.15.137:0/476386069 >> v2:172.21.15.59:6800/817089002 conn(0x7f22780341a0 0x7f2278036660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c05f1f0 con 0x7f2274003400 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f2273fff700 10 client.14182.objecter ms_handle_connect 0x7f22780341a0 2024-02-18T00:25:19.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.156+0000 7f228a59c700 1 --2- 172.21.15.137:0/476386069 >> v2:172.21.15.59:6800/817089002 conn(0x7f22780341a0 0x7f2278036660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2280006fd0 tx=0x7f2280008040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:19.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.161+0000 7f2273fff700 1 -- 172.21.15.137:0/476386069 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f227c01b080 con 0x7f2274003400 2024-02-18T00:25:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:13.837623+0000 mon.a (mon.0) 126 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:25:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:13.837803+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:13.837906+0000 mon.a (mon.0) 128 : cluster [INF] mon.a calling monitor election 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:14.209756+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:14.834726+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:15.834567+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:15.835857+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:16.210042+0000 mgr.y (mgr.14146) 39 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:16.834793+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:17.834953+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:18.210376+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:18.834933+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:18.867359+0000 mon.a (mon.0) 134 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:18.872444+0000 mon.a (mon.0) 135 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:18.884586+0000 mon.a (mon.0) 136 : cluster [DBG] fsmap 2024-02-18T00:25:19.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:18.884630+0000 mon.a (mon.0) 137 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:18.884856+0000 mon.a (mon.0) 138 : cluster [DBG] mgrmap e13: y(active, since 68s) 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:18.884994+0000 mon.a (mon.0) 139 : cluster [INF] overall HEALTH_OK 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:18.889615+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:18.890682+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:18.891476+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:13.837623+0000 mon.a (mon.0) 126 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:13.837803+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:13.837906+0000 mon.a (mon.0) 128 : cluster [INF] mon.a calling monitor election 2024-02-18T00:25:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:14.209756+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:14.834726+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:15.834567+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:15.835857+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:16.210042+0000 mgr.y (mgr.14146) 39 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:16.834793+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:17.834953+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:18.210376+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:18.834933+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:18.867359+0000 mon.a (mon.0) 134 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:18.872444+0000 mon.a (mon.0) 135 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:18.884586+0000 mon.a (mon.0) 136 : cluster [DBG] fsmap 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:18.884630+0000 mon.a (mon.0) 137 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:18.884856+0000 mon.a (mon.0) 138 : cluster [DBG] mgrmap e13: y(active, since 68s) 2024-02-18T00:25:19.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:18.884994+0000 mon.a (mon.0) 139 : cluster [INF] overall HEALTH_OK 2024-02-18T00:25:19.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:18.889615+0000 mon.a (mon.0) 140 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:19.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:18.890682+0000 mon.a (mon.0) 141 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:19.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:18.891476+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.392+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f228c046ce0 con 0x7f2274003400 2024-02-18T00:25:19.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.393+0000 7f2273fff700 1 -- 172.21.15.137:0/476386069 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+993 (secure 0 0 0) 0x7f227c026020 con 0x7f2274003400 2024-02-18T00:25:19.395 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:25:19.395 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":2,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","modified":"2024-02-18T00:25:13.835004Z","created":"2024-02-18T00:23:08.218775Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-18T00:25:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 10 librados: watch_flush enter 2024-02-18T00:25:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 10 librados: watch_flush exit 2024-02-18T00:25:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 20 client.14182.objecter shutdown clearing up homeless session... 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 10 client.14182.objecter successfully canceled tick 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 >> v2:172.21.15.59:6800/817089002 conn(0x7f22780341a0 msgr2=0x7f2278036660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 --2- 172.21.15.137:0/476386069 >> v2:172.21.15.59:6800/817089002 conn(0x7f22780341a0 0x7f2278036660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2280006fd0 tx=0x7f2280008040).stop 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 msgr2=0x7f228c10d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 --2- 172.21.15.137:0/476386069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 0x7f228c10d1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f227c016880 tx=0x7f227c00b040).stop 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 shutdown_connections 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 --2- 172.21.15.137:0/476386069 >> v2:172.21.15.59:6800/817089002 conn(0x7f22780341a0 0x7f2278036660 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 --2- 172.21.15.137:0/476386069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2274003400 0x7f228c10d1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 --2- 172.21.15.137:0/476386069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c10d690 0x7f228c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 >> 172.21.15.137:0/476386069 conn(0x7f228c06c440 msgr2=0x7f228c060500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 shutdown_connections 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 -- 172.21.15.137:0/476386069 wait complete. 2024-02-18T00:25:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:19.395+0000 7f2291198700 1 librados: shutdown 2024-02-18T00:25:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 2 2024-02-18T00:25:20.068 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:18.892189+0000 mgr.y (mgr.14146) 41 : cephadm [INF] Deploying daemon mon.c on smithi059 2024-02-18T00:25:20.068 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:19.393978+0000 mon.a (mon.0) 143 : audit [DBG] from='client.? 172.21.15.137:0/476386069' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:25:20.068 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:19.835723+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:18.892189+0000 mgr.y (mgr.14146) 41 : cephadm [INF] Deploying daemon mon.c on smithi059 2024-02-18T00:25:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:19.393978+0000 mon.a (mon.0) 143 : audit [DBG] from='client.? 172.21.15.137:0/476386069' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:25:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:19.835723+0000 mon.a (mon.0) 144 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:21.090 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T00:25:21.091 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T00:25:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:20.210677+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:20.210677+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:21.843 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:21 smithi059 systemd[1]: Starting Ceph mon.c for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.384+0000 7f94784a3880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.384+0000 7f94784a3880 0 ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable), process ceph-mon, pid 7 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.384+0000 7f94784a3880 0 pidfile_write: ignore empty --pid-file 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 0 load: jerasure load: lrc load: isa 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: DB SUMMARY 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: CURRENT file: CURRENT 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000003.log size: 636 ; 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.create_if_missing: 0 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T00:25:22.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.env: 0x5645109d1080 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.fs: Posix File System 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.info_log: 0x5645129cc260 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.statistics: (nil) 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.use_fsync: 0 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T00:25:22.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.db_log_dir: 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T00:25:22.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.write_buffer_manager: 0x5645136be390 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.unordered_write: 0 2024-02-18T00:25:22.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.row_cache: None 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.wal_filter: None 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.389+0000 7f94784a3880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T00:25:22.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_open_files: -1 2024-02-18T00:25:22.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Compression algorithms supported: 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kZSTD supported: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T00:25:22.760 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T00:25:22.761 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 2024-02-18T00:25:22.761 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.761 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T00:25:22.761 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.761 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T00:25:22.761 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.merge_operator: 2024-02-18T00:25:22.762 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_filter: None 2024-02-18T00:25:22.762 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564512971d10) 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cache_index_and_filter_blocks: 1 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: pin_top_level_index_and_filter: 1 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: index_type: 0 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: data_block_index_type: 0 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: index_shortening: 1 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: data_block_hash_table_util_ratio: 0.750000 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: hash_index_allow_collision: 1 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: checksum: 1 2024-02-18T00:25:22.763 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: no_block_cache: 0 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_cache: 0x5645129a6f10 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_cache_name: BinnedLRUCache 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_cache_options: 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: capacity : 536870912 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: num_shard_bits : 4 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: strict_capacity_limit : 0 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: high_pri_pool_ratio: 0.000 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_cache_compressed: (nil) 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: persistent_cache: (nil) 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_size: 4096 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_size_deviation: 10 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_restart_interval: 16 2024-02-18T00:25:22.764 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: index_block_restart_interval: 1 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: metadata_block_size: 4096 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: partition_filters: 0 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: use_delta_encoding: 1 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: whole_key_filtering: 1 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: verify_compression: 0 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: read_amp_bytes_per_bit: 0 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: format_version: 2 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: enable_index_compression: 1 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: block_align: 0 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compression: NoCompression 2024-02-18T00:25:22.765 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.num_levels: 7 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T00:25:22.766 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T00:25:22.767 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T00:25:22.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T00:25:22.769 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.ttl: 2592000 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.390+0000 7f94784a3880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.391+0000 7f94784a3880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.391+0000 7f94784a3880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215922392269, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.391+0000 7f94784a3880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.391+0000 7f94784a3880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.391+0000 7f94784a3880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215922392736, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1562, "table_properties": {"data_size": 648, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708215922, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.391+0000 7f94784a3880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.392+0000 7f94784a3880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215922393167, "job": 1, "event": "recovery_finished"} 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.392+0000 7f94784a3880 4 rocksdb: DB pointer 0x564512a6f800 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.392+0000 7f945f295700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.392+0000 7f945f295700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ** DB Stats ** 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:25:22.771 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ** Compaction Stats [default] ** 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ** Compaction Stats [default] ** 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ** Compaction Stats [default] ** 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ** Compaction Stats [default] ** 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T00:25:22.774 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.392+0000 7f94784a3880 0 mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.393+0000 7f94784a3880 0 using public_addrv [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.393+0000 7f94784a3880 0 starting mon.c rank -1 at public addrs [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] at bind addrs [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.394+0000 7f94784a3880 1 mon.c@-1(???) e0 preinit fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:25:22.775 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f945d291700 4 rocksdb: [db_impl/db_impl_write.cc:1668] [default] New memtable created with log file: #9. Immutable memtables: 0. 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f94672a5700 4 rocksdb: (Original Log Time 2024/02/18-00:25:22.413737) [db_impl/db_impl_compaction_flush.cc:2198] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f94672a5700 4 rocksdb: [flush_job.cc:321] [default] [JOB 3] Flushing memtable with next log file: 9 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f94672a5700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708215922413794, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 963, "num_deletes": 10, "total_data_size": 3991773, "memory_usage": 4009912, "flush_reason": "Manual Compaction"} 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f94672a5700 4 rocksdb: [flush_job.cc:350] [default] [JOB 3] Level-0 flush table #10: started 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f9461a9a700 0 mon.c@-1(synchronizing).mds e1 new map 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f9461a9a700 0 mon.c@-1(synchronizing).mds e1 print_map 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: e1 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: legacy client fscid: -1 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.776 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: No filesystems configured 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.412+0000 7f9461a9a700 1 mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 1 mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 1 mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 1 mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 1 mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 1 mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 0 mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:20.210677+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:25:22.413+0000 7f9461a9a700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 bash[45102]: c691c95cf31430e216e03ce52652de23bda089676d1f204f3415404f1f3fd989 2024-02-18T00:25:22.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:22 smithi059 systemd[1]: Started Ceph mon.c for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:25:27.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.861+0000 7f7bfba3f700 1 -- 172.21.15.137:0/2604210346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 msgr2=0x7f7be0005450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:27.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.861+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/2604210346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7be0005450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7bf0007ba0 tx=0x7f7bf000b040).stop 2024-02-18T00:25:27.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.861+0000 7f7bfba3f700 1 -- 172.21.15.137:0/2604210346 shutdown_connections 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.861+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/2604210346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7be0005450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.861+0000 7f7bfba3f700 1 -- 172.21.15.137:0/2604210346 >> 172.21.15.137:0/2604210346 conn(0x7f7bf406c440 msgr2=0x7f7bf406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.861+0000 7f7bfba3f700 1 -- 172.21.15.137:0/2604210346 shutdown_connections 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.861+0000 7f7bfba3f700 1 -- 172.21.15.137:0/2604210346 wait complete. 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.862+0000 7f7bfba3f700 1 librados: starting msgr at 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.862+0000 7f7bfba3f700 1 librados: starting objecter 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.862+0000 7f7bfba3f700 1 Processor -- start 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.862+0000 7f7bfba3f700 1 -- start start 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.862+0000 7f7bfba3f700 1 librados: setting wanted keys 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.862+0000 7f7bfba3f700 1 librados: calling monclient init 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.862+0000 7f7bfba3f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7bf40f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 0x7f7bf40f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 0x7f7bf4115890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:27.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4115f80 con 0x7f7bf40fc550 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf41160f0 con 0x7f7bf41134a0 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf41162b0 con 0x7f7bf40f8750 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf9fdc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 0x7f7bf4115890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7bf40f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf9fdc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 0x7f7bf4115890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44590/0 (socket says 172.21.15.137:44590) 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7bf40f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.137:40216/0 (socket says 172.21.15.137:40216) 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf9fdc700 1 -- 172.21.15.137:0/1091193110 learned_addr learned my addr 172.21.15.137:0/1091193110 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf8fda700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 0x7f7bf40f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf9fdc700 1 -- 172.21.15.137:0/1091193110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 msgr2=0x7f7bf4115890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf9fdc700 1 -- 172.21.15.137:0/1091193110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 msgr2=0x7f7bf4115890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf9fdc700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 0x7f7bf4115890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf9fdc700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 0x7f7bf4115890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:25:27.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf8fda700 1 -- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 msgr2=0x7f7bf40f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf8fda700 1 -- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 msgr2=0x7f7bf40f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf8fda700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 0x7f7bf40f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf8fda700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 0x7f7bf40f8bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 -- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 msgr2=0x7f7bf40f8bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 0x7f7bf40f8bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 -- 172.21.15.137:0/1091193110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 msgr2=0x7f7bf4115890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 0x7f7bf4115890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 -- 172.21.15.137:0/1091193110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0005040 con 0x7f7bf40fc550 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bf97db700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7bf40f8260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7bf0007b70 tx=0x7f7bf0002040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bea7fc700 10 client.?.objecter ms_handle_connect 0x7f7bf40fc550 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bea7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bea7fc700 1 -- 172.21.15.137:0/1091193110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0016460 con 0x7f7bf40fc550 2024-02-18T00:25:27.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bea7fc700 1 -- 172.21.15.137:0/1091193110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf0000bc0 con 0x7f7bf40fc550 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf4116530 con 0x7f7bf40fc550 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bea7fc700 1 -- 172.21.15.137:0/1091193110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf001f3a0 con 0x7f7bf40fc550 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 10 client.14187.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf4116a20 con 0x7f7bf40fc550 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.863+0000 7f7bfba3f700 1 librados: init done 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bea7fc700 1 -- 172.21.15.137:0/1091193110 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f7bf001f790 con 0x7f7bf40fc550 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bea7fc700 1 --2- 172.21.15.137:0/1091193110 >> v2:172.21.15.59:6800/817089002 conn(0x7f7bd80344f0 0x7f7bd80369b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bea7fc700 1 -- 172.21.15.137:0/1091193110 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f7bf0045040 con 0x7f7bf40fc550 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bea7fc700 10 client.14187.objecter ms_dispatch 0x7f7bf4001550 osd_map(4..4 src has 1..4) v4 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bea7fc700 3 client.14187.objecter handle_osd_map got epochs [4,4] > 0 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bea7fc700 3 client.14187.objecter handle_osd_map decoding full epoch 4 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bea7fc700 20 client.14187.objecter dump_active .. 0 homeless 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf405a310 con 0x7f7bf40fc550 2024-02-18T00:25:27.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.864+0000 7f7bf8fda700 1 --2- 172.21.15.137:0/1091193110 >> v2:172.21.15.59:6800/817089002 conn(0x7f7bd80344f0 0x7f7bd80369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:27.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.865+0000 7f7bea7fc700 10 client.14187.objecter ms_handle_connect 0x7f7bd80344f0 2024-02-18T00:25:27.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.865+0000 7f7bf8fda700 1 --2- 172.21.15.137:0/1091193110 >> v2:172.21.15.59:6800/817089002 conn(0x7f7bd80344f0 0x7f7bd80369b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7be4006fd0 tx=0x7f7be4008040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:27.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:27.869+0000 7f7bea7fc700 1 -- 172.21.15.137:0/1091193110 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7bf001b030 con 0x7f7bf40fc550 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:22.211075+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:22.425117+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:22.425284+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:22.425985+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:22.437993+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:22.438113+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:23.417787+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:24.211379+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:24.417766+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:24.419313+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:25.418027+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:26.211687+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:26.418199+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:27.418229+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:27.451898+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:27.457429+0000 mon.a (mon.0) 156 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0]} removed_ranks: {} 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:27.457957+0000 mon.a (mon.0) 157 : cluster [DBG] fsmap 2024-02-18T00:25:28.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:22.211075+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:22.425117+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:22.425284+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:22.425985+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:22.437993+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:22.438113+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:23.417787+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:24.211379+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:24.417766+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:24.419313+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:25.418027+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:26.211687+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:26.418199+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:27.418229+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:27.451898+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:27.457429+0000 mon.a (mon.0) 156 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0]} removed_ranks: {} 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:27.457957+0000 mon.a (mon.0) 157 : cluster [DBG] fsmap 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:27.458023+0000 mon.a (mon.0) 158 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:27.458259+0000 mon.a (mon.0) 159 : cluster [DBG] mgrmap e13: y(active, since 77s) 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:27.458464+0000 mon.a (mon.0) 160 : cluster [INF] overall HEALTH_OK 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:27.472615+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:27.474056+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:27.458023+0000 mon.a (mon.0) 158 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:27.458259+0000 mon.a (mon.0) 159 : cluster [DBG] mgrmap e13: y(active, since 77s) 2024-02-18T00:25:28.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:27.458464+0000 mon.a (mon.0) 160 : cluster [INF] overall HEALTH_OK 2024-02-18T00:25:28.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:27.472615+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:28.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:27.474056+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:28.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:22.211075+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:22.425117+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:25:28.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:22.425284+0000 mon.a (mon.0) 147 : cluster [INF] mon.a calling monitor election 2024-02-18T00:25:28.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:22.425985+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-18T00:25:28.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:22.437993+0000 mon.a (mon.0) 148 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:22.438113+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:23.417787+0000 mon.a (mon.0) 150 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:24.211379+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:24.417766+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:24.419313+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:25.418027+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:26.211687+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:26.418199+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:27.418229+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:27.451898+0000 mon.a (mon.0) 155 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:27.457429+0000 mon.a (mon.0) 156 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0]} removed_ranks: {} 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:27.457957+0000 mon.a (mon.0) 157 : cluster [DBG] fsmap 2024-02-18T00:25:28.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:27.458023+0000 mon.a (mon.0) 158 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T00:25:28.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:27.458259+0000 mon.a (mon.0) 159 : cluster [DBG] mgrmap e13: y(active, since 77s) 2024-02-18T00:25:28.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:27.458464+0000 mon.a (mon.0) 160 : cluster [INF] overall HEALTH_OK 2024-02-18T00:25:28.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:27.472615+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:28.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:27.474056+0000 mon.a (mon.0) 162 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.097+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7bf40fd8b0 con 0x7f7bf40fc550 2024-02-18T00:25:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.097+0000 7f7bea7fc700 1 -- 172.21.15.137:0/1091193110 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1259 (secure 0 0 0) 0x7f7bf0024020 con 0x7f7bf40fc550 2024-02-18T00:25:28.099 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-02-18T00:25:28.099 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":3,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","modified":"2024-02-18T00:25:22.418255Z","created":"2024-02-18T00:23:08.218775Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3301","nonce":0},{"type":"v1","addr":"172.21.15.59:6790","nonce":0}]},"addr":"172.21.15.59:6790/0","public_addr":"172.21.15.59:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-18T00:25:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 10 librados: watch_flush enter 2024-02-18T00:25:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 10 librados: watch_flush exit 2024-02-18T00:25:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 20 client.14187.objecter shutdown clearing up homeless session... 2024-02-18T00:25:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 10 client.14187.objecter successfully canceled tick 2024-02-18T00:25:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 >> v2:172.21.15.59:6800/817089002 conn(0x7f7bd80344f0 msgr2=0x7f7bd80369b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/1091193110 >> v2:172.21.15.59:6800/817089002 conn(0x7f7bd80344f0 0x7f7bd80369b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7be4006fd0 tx=0x7f7be4008040).stop 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 msgr2=0x7f7bf40f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7bf40f8260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f7bf0007b70 tx=0x7f7bf0002040).stop 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 shutdown_connections 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/1091193110 >> v2:172.21.15.59:6800/817089002 conn(0x7f7bd80344f0 0x7f7bd80369b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bf41134a0 0x7f7bf4115890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bf40fc550 0x7f7bf40f8260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 --2- 172.21.15.137:0/1091193110 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7bf40f8750 0x7f7bf40f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.099+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 >> 172.21.15.137:0/1091193110 conn(0x7f7bf406c440 msgr2=0x7f7bf40faee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.100+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 shutdown_connections 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.100+0000 7f7bfba3f700 1 -- 172.21.15.137:0/1091193110 wait complete. 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:28.100+0000 7f7bfba3f700 1 librados: shutdown 2024-02-18T00:25:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:dumped monmap epoch 3 2024-02-18T00:25:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:28.098121+0000 mon.a (mon.0) 163 : audit [DBG] from='client.? 172.21.15.137:0/1091193110' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:25:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:28.418199+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:28.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:28.098121+0000 mon.a (mon.0) 163 : audit [DBG] from='client.? 172.21.15.137:0/1091193110' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:25:28.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:28.418199+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:28.098121+0000 mon.a (mon.0) 163 : audit [DBG] from='client.? 172.21.15.137:0/1091193110' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:25:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:28.418199+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:25:29.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:28.212006+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:29.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:28.859034+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:28.212006+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:28.859034+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:29.753 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:25:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:25:29.417+0000 7f754448c700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-18T00:25:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:28.212006+0000 mgr.y (mgr.14146) 46 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:28.859034+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:30.212357+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:30.349032+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:30.212357+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:30.349032+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:31.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:30.212357+0000 mgr.y (mgr.14146) 47 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:31.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:30.349032+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:32.562 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-18T00:25:32.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-18T00:25:33.192 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.079073+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.192 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.079897+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:33.192 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.080809+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:32.081892+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.conf 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:32.082203+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.conf 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.102999+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.137072+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.143283+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:32.144588+0000 mgr.y (mgr.14146) 50 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.144895+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.145741+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:32.146535+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:32.147323+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Reconfiguring daemon mon.a on smithi059 2024-02-18T00:25:33.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:32.212665+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.079073+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.079897+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.080809+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:32.081892+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.conf 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:32.082203+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.conf 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.102999+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.137072+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.143283+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:32.144588+0000 mgr.y (mgr.14146) 50 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.144895+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.145741+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:32.146535+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:33.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:32.147323+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Reconfiguring daemon mon.a on smithi059 2024-02-18T00:25:33.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:32.212665+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.079073+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.079897+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.080809+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:32.081892+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.conf 2024-02-18T00:25:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:32.082203+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.conf 2024-02-18T00:25:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.102999+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.137072+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.143283+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:32.144588+0000 mgr.y (mgr.14146) 50 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-18T00:25:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.144895+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.145741+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:32.146535+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:32.147323+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Reconfiguring daemon mon.a on smithi059 2024-02-18T00:25:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:32.212665+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:35.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:34.071181+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:35.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:34.071689+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-18T00:25:35.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:34.071949+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:35.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:34.072574+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:34.073185+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:34.073727+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring daemon mon.c on smithi059 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:34.212904+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:34.071181+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:34.071689+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:34.071949+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:34.072574+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:34.073185+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:34.073727+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring daemon mon.c on smithi059 2024-02-18T00:25:35.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:34.212904+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:34.071181+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:34.071689+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-18T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:34.071949+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:34.072574+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:34.073185+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:34.073727+0000 mgr.y (mgr.14146) 54 : cephadm [INF] Reconfiguring daemon mon.c on smithi059 2024-02-18T00:25:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:34.212904+0000 mgr.y (mgr.14146) 55 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:35.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 -- 172.21.15.59:0/1362362276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40fdb20 msgr2=0x7f13c40fdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1362362276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40fdb20 0x7f13c40fdf00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f13b4004870 tx=0x7f13b4009cb0).stop 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 -- 172.21.15.59:0/1362362276 shutdown_connections 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1362362276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13c40f8390 0x7f13c4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1362362276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40fdb20 0x7f13c40fdf00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1362362276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13c40fe480 0x7f13c40f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 -- 172.21.15.59:0/1362362276 >> 172.21.15.59:0/1362362276 conn(0x7f13c40f3770 msgr2=0x7f13c40f5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 -- 172.21.15.59:0/1362362276 shutdown_connections 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.888+0000 7f13cc62f700 1 -- 172.21.15.59:0/1362362276 wait complete. 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 librados: starting msgr at 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 librados: starting objecter 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 Processor -- start 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 -- start start 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 librados: setting wanted keys 2024-02-18T00:25:35.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 librados: calling monclient init 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40f8390 0x7f13c410e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13c40fdb20 0x7f13c410e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13c40fe480 0x7f13c41081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41086c0 con 0x7f13c40fe480 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4108830 con 0x7f13c40fdb20 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cc62f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41089f0 con 0x7f13c40f8390 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13ca3cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40f8390 0x7f13c410e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cabcc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13c40fe480 0x7f13c41081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13ca3cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40f8390 0x7f13c410e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51062/0 (socket says 172.21.15.59:51062) 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13ca3cb700 1 -- 172.21.15.59:0/1742474334 learned_addr learned my addr 172.21.15.59:0/1742474334 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13c9bca700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13c40fdb20 0x7f13c410e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:35.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cabcc700 1 -- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40f8390 msgr2=0x7f13c410e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cabcc700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40f8390 0x7f13c410e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cabcc700 1 -- 172.21.15.59:0/1742474334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13c40fdb20 msgr2=0x7f13c410e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cabcc700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13c40fdb20 0x7f13c410e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13cabcc700 1 -- 172.21.15.59:0/1742474334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c0005040 con 0x7f13c40fe480 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.889+0000 7f13ca3cb700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40f8390 0x7f13c410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13cabcc700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13c40fe480 0x7f13c41081d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f13bc00c7a0 tx=0x7f13bc007d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13bb7fe700 10 client.?.objecter ms_handle_connect 0x7f13c40fe480 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13bb7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13bb7fe700 1 -- 172.21.15.59:0/1742474334 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc010510 con 0x7f13c40fe480 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13bb7fe700 1 -- 172.21.15.59:0/1742474334 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13bc010b40 con 0x7f13c40fe480 2024-02-18T00:25:35.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b4005040 con 0x7f13c40fe480 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13bb7fe700 1 -- 172.21.15.59:0/1742474334 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc012b10 con 0x7f13c40fe480 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13cc62f700 10 client.14193.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c4114e60 con 0x7f13c40fe480 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.890+0000 7f13cc62f700 1 librados: init done 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 1 -- 172.21.15.59:0/1742474334 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f13bc010670 con 0x7f13c40fe480 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c4060b00 con 0x7f13c40fe480 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 1 --2- 172.21.15.59:0/1742474334 >> v2:172.21.15.59:6800/817089002 conn(0x7f13b0034230 0x7f13b00366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 1 -- 172.21.15.59:0/1742474334 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f13bc042d60 con 0x7f13c40fe480 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 10 client.14193.objecter ms_dispatch 0x7f13c4001550 osd_map(4..4 src has 1..4) v4 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 3 client.14193.objecter handle_osd_map got epochs [4,4] > 0 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 3 client.14193.objecter handle_osd_map decoding full epoch 4 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 20 client.14193.objecter dump_active .. 0 homeless 2024-02-18T00:25:35.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13ca3cb700 1 --2- 172.21.15.59:0/1742474334 >> v2:172.21.15.59:6800/817089002 conn(0x7f13b0034230 0x7f13b00366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:35.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13bb7fe700 10 client.14193.objecter ms_handle_connect 0x7f13b0034230 2024-02-18T00:25:35.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.891+0000 7f13ca3cb700 1 --2- 172.21.15.59:0/1742474334 >> v2:172.21.15.59:6800/817089002 conn(0x7f13b0034230 0x7f13b00366f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f13b4009a00 tx=0x7f13b400b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:35.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:35.895+0000 7f13bb7fe700 1 -- 172.21.15.59:0/1742474334 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f13bc010cb0 con 0x7f13c40fe480 2024-02-18T00:25:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.068+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f13c405f360 con 0x7f13c40fe480 2024-02-18T00:25:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.068+0000 7f13bb7fe700 1 -- 172.21.15.59:0/1742474334 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f13bc0182b0 con 0x7f13c40fe480 2024-02-18T00:25:36.069 INFO:teuthology.orchestra.run.smithi059.stdout:# minimal ceph.conf for d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:25:36.069 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-02-18T00:25:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:25:36.070 INFO:teuthology.orchestra.run.smithi059.stdout: mon_host = [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] 2024-02-18T00:25:36.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 10 librados: watch_flush enter 2024-02-18T00:25:36.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 10 librados: watch_flush exit 2024-02-18T00:25:36.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 20 client.14193.objecter shutdown clearing up homeless session... 2024-02-18T00:25:36.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 10 client.14193.objecter successfully canceled tick 2024-02-18T00:25:36.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 >> v2:172.21.15.59:6800/817089002 conn(0x7f13b0034230 msgr2=0x7f13b00366f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1742474334 >> v2:172.21.15.59:6800/817089002 conn(0x7f13b0034230 0x7f13b00366f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f13b4009a00 tx=0x7f13b400b040).stop 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13c40fe480 msgr2=0x7f13c41081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13c40fe480 0x7f13c41081d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f13bc00c7a0 tx=0x7f13bc007d10).stop 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 shutdown_connections 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1742474334 >> v2:172.21.15.59:6800/817089002 conn(0x7f13b0034230 0x7f13b00366f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13c40fe480 0x7f13c41081d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13c40f8390 0x7f13c410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 --2- 172.21.15.59:0/1742474334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13c40fdb20 0x7f13c410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 >> 172.21.15.59:0/1742474334 conn(0x7f13c40f3770 msgr2=0x7f13c40fb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 shutdown_connections 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 -- 172.21.15.59:0/1742474334 wait complete. 2024-02-18T00:25:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:36.070+0000 7f13cc62f700 1 librados: shutdown 2024-02-18T00:25:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:36.068627+0000 mon.a (mon.0) 180 : audit [DBG] from='client.? 172.21.15.59:0/1742474334' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:36.068627+0000 mon.a (mon.0) 180 : audit [DBG] from='client.? 172.21.15.59:0/1742474334' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:36.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:36.068627+0000 mon.a (mon.0) 180 : audit [DBG] from='client.? 172.21.15.59:0/1742474334' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:36.213247+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:37.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:36.213247+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:37.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:36.213247+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:37.545 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-18T00:25:37.546 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:25:37.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2024-02-18T00:25:37.580 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:25:37.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:25:37.651 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:25:37.652 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-02-18T00:25:37.683 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:25:37.683 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:25:37.750 INFO:tasks.cephadm:Adding mgr.y on smithi059 2024-02-18T00:25:37.751 INFO:tasks.cephadm:Adding mgr.x on smithi137 2024-02-18T00:25:37.751 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply mgr '2;smithi059=y;smithi137=x' 2024-02-18T00:25:38.407 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:37.097240+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:38.407 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:37.097808+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-18T00:25:38.407 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:37.098044+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:38.407 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:37.098703+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:38.407 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:37.099298+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:38.407 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:37.099832+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Reconfiguring daemon mon.b on smithi137 2024-02-18T00:25:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:37.097240+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:37.097808+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-18T00:25:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:37.098044+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:37.098703+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:37.099298+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:37.099832+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Reconfiguring daemon mon.b on smithi137 2024-02-18T00:25:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:37.097240+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:37.097808+0000 mgr.y (mgr.14146) 57 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-18T00:25:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:37.098044+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T00:25:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:37.098703+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T00:25:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:37.099298+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:37.099832+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Reconfiguring daemon mon.b on smithi137 2024-02-18T00:25:39.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:38.213612+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:39.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:38.727764+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:39.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:38.728804+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:39.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:38.730434+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:39.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:38.731242+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:39.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:38.735547+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:38.213612+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:38.727764+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:38.728804+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:38.730434+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:38.731242+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:38.735547+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:38.213612+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:38.727764+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:38.728804+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:38.730434+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:38.731242+0000 mon.a (mon.0) 188 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:38.735547+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 -- 172.21.15.137:0/114601772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f9670 msgr2=0x7f5fd80fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 --2- 172.21.15.137:0/114601772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f9670 0x7f5fd80fff70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f5fd0009c10 tx=0x7f5fd0004d10).stop 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 -- 172.21.15.137:0/114601772 shutdown_connections 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 --2- 172.21.15.137:0/114601772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f9670 0x7f5fd80fff70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 --2- 172.21.15.137:0/114601772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fd80f7a80 0x7f5fd80f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 --2- 172.21.15.137:0/114601772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd80f8d20 0x7f5fd80f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 -- 172.21.15.137:0/114601772 >> 172.21.15.137:0/114601772 conn(0x7f5fd80f32a0 msgr2=0x7f5fd80f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 -- 172.21.15.137:0/114601772 shutdown_connections 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.899+0000 7f5fe012d700 1 -- 172.21.15.137:0/114601772 wait complete. 2024-02-18T00:25:39.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 librados: starting msgr at 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 librados: starting objecter 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 Processor -- start 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 -- start start 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 librados: setting wanted keys 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 librados: calling monclient init 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fd80f7a80 0x7f5fd8107ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f8d20 0x7f5fd8108190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd80f9670 0x7f5fd810f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810f750 con 0x7f5fd80f9670 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810f890 con 0x7f5fd80f8d20 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fe012d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810fa20 con 0x7f5fd80f7a80 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f8d20 0x7f5fd8108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f8d20 0x7f5fd8108190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37354/0 (socket says 172.21.15.137:37354) 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 -- 172.21.15.137:0/2056942521 learned_addr learned my addr 172.21.15.137:0/2056942521 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:25:39.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fddec9700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fd80f7a80 0x7f5fd8107ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fde6ca700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd80f9670 0x7f5fd810f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 -- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fd80f7a80 msgr2=0x7f5fd8107ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fd80f7a80 0x7f5fd8107ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 -- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd80f9670 msgr2=0x7f5fd810f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd80f9670 0x7f5fd810f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.900+0000 7f5fdd6c8700 1 -- 172.21.15.137:0/2056942521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00097c0 con 0x7f5fd80f8d20 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fdd6c8700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f8d20 0x7f5fd8108190 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5fd40041d0 tx=0x7f5fd4009f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fcaffd700 10 client.?.objecter ms_handle_connect 0x7f5fd80f8d20 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fcaffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fcaffd700 1 -- 172.21.15.137:0/2056942521 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4007b40 con 0x7f5fd80f8d20 2024-02-18T00:25:39.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fcaffd700 1 -- 172.21.15.137:0/2056942521 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd400f040 con 0x7f5fd80f8d20 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd810fd00 con 0x7f5fd80f8d20 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fcaffd700 1 -- 172.21.15.137:0/2056942521 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4014810 con 0x7f5fd80f8d20 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fe012d700 10 client.24106.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd8110260 con 0x7f5fd80f8d20 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.901+0000 7f5fe012d700 1 librados: init done 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 1 -- 172.21.15.137:0/2056942521 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f5fd4014c00 con 0x7f5fd80f8d20 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 1 --2- 172.21.15.137:0/2056942521 >> v2:172.21.15.59:6800/817089002 conn(0x7f5fc4034230 0x7f5fc40366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 1 -- 172.21.15.137:0/2056942521 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f5fd4042db0 con 0x7f5fd80f8d20 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 10 client.24106.objecter ms_dispatch 0x7f5fd8001550 osd_map(4..4 src has 1..4) v4 2024-02-18T00:25:39.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 3 client.24106.objecter handle_osd_map got epochs [4,4] > 0 2024-02-18T00:25:39.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 3 client.24106.objecter handle_osd_map decoding full epoch 4 2024-02-18T00:25:39.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 20 client.24106.objecter dump_active .. 0 homeless 2024-02-18T00:25:39.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd805f360 con 0x7f5fd80f8d20 2024-02-18T00:25:39.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fddec9700 1 --2- 172.21.15.137:0/2056942521 >> v2:172.21.15.59:6800/817089002 conn(0x7f5fc4034230 0x7f5fc40366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:39.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fcaffd700 10 client.24106.objecter ms_handle_connect 0x7f5fc4034230 2024-02-18T00:25:39.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.902+0000 7f5fddec9700 1 --2- 172.21.15.137:0/2056942521 >> v2:172.21.15.59:6800/817089002 conn(0x7f5fc4034230 0x7f5fc40366f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5fd8108b90 tx=0x7f5fcc009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:39.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:39.907+0000 7f5fcaffd700 1 -- 172.21.15.137:0/2056942521 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5fd401d3e0 con 0x7f5fd80f8d20 2024-02-18T00:25:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.077+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi059=y;smithi137=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fd80fed90 con 0x7f5fc4034230 2024-02-18T00:25:40.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.084+0000 7f5fcaffd700 1 -- 172.21.15.137:0/2056942521 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5fd80fed90 con 0x7f5fc4034230 2024-02-18T00:25:40.085 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled mgr update... 2024-02-18T00:25:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 10 librados: watch_flush enter 2024-02-18T00:25:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 10 librados: watch_flush exit 2024-02-18T00:25:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 20 client.24106.objecter shutdown clearing up homeless session... 2024-02-18T00:25:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 10 client.24106.objecter successfully canceled tick 2024-02-18T00:25:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 >> v2:172.21.15.59:6800/817089002 conn(0x7f5fc4034230 msgr2=0x7f5fc40366f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 --2- 172.21.15.137:0/2056942521 >> v2:172.21.15.59:6800/817089002 conn(0x7f5fc4034230 0x7f5fc40366f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5fd8108b90 tx=0x7f5fcc009040).stop 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f8d20 msgr2=0x7f5fd8108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f8d20 0x7f5fd8108190 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5fd40041d0 tx=0x7f5fd4009f60).stop 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 shutdown_connections 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 --2- 172.21.15.137:0/2056942521 >> v2:172.21.15.59:6800/817089002 conn(0x7f5fc4034230 0x7f5fc40366f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fd80f9670 0x7f5fd810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fd80f7a80 0x7f5fd8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 --2- 172.21.15.137:0/2056942521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd80f8d20 0x7f5fd8108190 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.085+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 >> 172.21.15.137:0/2056942521 conn(0x7f5fd80f32a0 msgr2=0x7f5fd80fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.086+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 shutdown_connections 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.086+0000 7f5fe012d700 1 -- 172.21.15.137:0/2056942521 wait complete. 2024-02-18T00:25:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:25:40.086+0000 7f5fe012d700 1 librados: shutdown 2024-02-18T00:25:40.768 DEBUG:teuthology.orchestra.run.smithi137:mgr.x> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.x.service 2024-02-18T00:25:40.770 INFO:tasks.cephadm:Deploying OSDs... 2024-02-18T00:25:40.771 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:25:40.771 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T00:25:40.795 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T00:25:40.795 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-02-18T00:25:40.800 INFO:journalctl@ceph.mgr.x.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:25:40.856 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T00:25:40.856 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:40.856 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 170161 Links: 1 2024-02-18T00:25:40.857 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:40.857 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:40.857 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-02-18 00:25:21.794297666 +0000 2024-02-18T00:25:40.857 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-02-18 00:18:19.174734732 +0000 2024-02-18T00:25:40.857 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-02-18 00:18:19.174734732 +0000 2024-02-18T00:25:40.857 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-02-18T00:25:40.857 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T00:25:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-02-18T00:25:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-02-18T00:25:40.927 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.00021534 s, 2.4 MB/s 2024-02-18T00:25:40.928 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T00:25:40.991 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 166294 Links: 1 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-02-18 00:25:21.794297666 +0000 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-02-18 00:18:19.525730381 +0000 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-02-18 00:18:19.525730381 +0000 2024-02-18T00:25:41.052 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-02-18T00:25:41.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T00:25:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-02-18T00:25:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-02-18T00:25:41.122 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000234751 s, 2.2 MB/s 2024-02-18T00:25:41.123 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T00:25:41.186 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-02-18T00:25:41.245 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 169166 Links: 1 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-02-18 00:25:21.795297654 +0000 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-02-18 00:18:19.859726241 +0000 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-02-18 00:18:19.859726241 +0000 2024-02-18T00:25:41.246 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-02-18T00:25:41.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.078705+0000 mgr.y (mgr.14146) 60 : audit [DBG] from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi059=y;smithi137=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:40.080140+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Saving service mgr spec with placement smithi059=y;smithi137=x;count:2 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.083955+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.104447+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.105998+0000 mon.a (mon.0) 192 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.106777+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.110910+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:41.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.112251+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.115132+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.115920+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:40.116800+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:40.117439+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Deploying daemon mgr.x on smithi137 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:40.213991+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.078705+0000 mgr.y (mgr.14146) 60 : audit [DBG] from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi059=y;smithi137=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:40.080140+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Saving service mgr spec with placement smithi059=y;smithi137=x;count:2 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.083955+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.104447+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.105998+0000 mon.a (mon.0) 192 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:41.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.106777+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:41.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.110910+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:41.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.112251+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T00:25:41.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.115132+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T00:25:41.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.115920+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T00:25:41.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:40.116800+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:41.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:40.117439+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Deploying daemon mgr.x on smithi137 2024-02-18T00:25:41.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:40.213991+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:41.281 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-02-18T00:25:41.281 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-02-18T00:25:41.282 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000193898 s, 2.6 MB/s 2024-02-18T00:25:41.282 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T00:25:41.343 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 166381 Links: 1 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-02-18 00:25:21.795297654 +0000 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-02-18 00:18:20.353720118 +0000 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-02-18 00:18:20.353720118 +0000 2024-02-18T00:25:41.402 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2024-02-18T00:25:41.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T00:25:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-02-18T00:25:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-02-18T00:25:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000215196 s, 2.4 MB/s 2024-02-18T00:25:41.470 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.078705+0000 mgr.y (mgr.14146) 60 : audit [DBG] from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi059=y;smithi137=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:40.080140+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Saving service mgr spec with placement smithi059=y;smithi137=x;count:2 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.083955+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.104447+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.105998+0000 mon.a (mon.0) 192 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.106777+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.110910+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.112251+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.115132+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T00:25:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.115920+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T00:25:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:40.116800+0000 mon.a (mon.0) 198 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:40.117439+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Deploying daemon mgr.x on smithi137 2024-02-18T00:25:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:40.213991+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:41.529 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:25:41.529 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T00:25:41.547 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T00:25:41.547 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-02-18T00:25:41.605 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T00:25:41.605 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:41.605 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 169076 Links: 1 2024-02-18T00:25:41.605 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:41.605 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:41.606 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-02-18 00:25:13.074785389 +0000 2024-02-18T00:25:41.606 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-02-18 00:18:08.662348589 +0000 2024-02-18T00:25:41.606 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-02-18 00:18:08.662348589 +0000 2024-02-18T00:25:41.606 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-02-18T00:25:41.606 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T00:25:41.672 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-02-18T00:25:41.672 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-02-18T00:25:41.672 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000236878 s, 2.2 MB/s 2024-02-18T00:25:41.673 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T00:25:41.731 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-02-18T00:25:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T00:25:41.788 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:41.789 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 165499 Links: 1 2024-02-18T00:25:41.789 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:41.789 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:41.789 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-02-18 00:25:13.074785389 +0000 2024-02-18T00:25:41.789 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-02-18 00:18:09.002342675 +0000 2024-02-18T00:25:41.789 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-02-18 00:18:09.002342675 +0000 2024-02-18T00:25:41.789 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-02-18T00:25:41.789 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T00:25:41.855 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-02-18T00:25:41.855 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-02-18T00:25:41.855 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000236026 s, 2.2 MB/s 2024-02-18T00:25:41.856 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T00:25:41.914 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 162814 Links: 1 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-02-18 00:25:13.074785389 +0000 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-02-18 00:18:09.345336708 +0000 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-02-18 00:18:09.345336708 +0000 2024-02-18T00:25:41.971 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-02-18T00:25:41.972 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T00:25:42.037 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-02-18T00:25:42.037 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-02-18T00:25:42.037 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000163023 s, 3.1 MB/s 2024-02-18T00:25:42.038 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T00:25:42.096 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-02-18T00:25:42.153 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T00:25:42.153 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T00:25:42.153 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 169212 Links: 1 2024-02-18T00:25:42.153 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T00:25:42.154 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T00:25:42.154 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-02-18 00:25:13.074785389 +0000 2024-02-18T00:25:42.154 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-02-18 00:18:09.687330760 +0000 2024-02-18T00:25:42.154 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-02-18 00:18:09.687330760 +0000 2024-02-18T00:25:42.154 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-02-18T00:25:42.154 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T00:25:42.218 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-02-18T00:25:42.219 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-02-18T00:25:42.219 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000229624 s, 2.2 MB/s 2024-02-18T00:25:42.220 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T00:25:42.279 INFO:tasks.cephadm:Deploying osd.0 on smithi059 with /dev/vg_nvme/lv_4... 2024-02-18T00:25:42.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-18T00:25:43.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:42 smithi137 systemd[1]: Starting Ceph mgr.x for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:25:43.539 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:42.214381+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:43.540 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:43.323+0000 7fa5c70d2700 1 -- 172.21.15.137:0/2050593535 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5564c9da24e0 con 0x5564c9d76c00 2024-02-18T00:25:43.540 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:43 smithi137 bash[40967]: 9ed7c255e71ea40fa6b4c42f83ec9770ec2028be146c7d9cab7fb150501e6150 2024-02-18T00:25:43.540 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:43 smithi137 systemd[1]: Started Ceph mgr.x for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:25:43.540 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:43.453+0000 7fa5d76cd680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T00:25:43.540 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:43.537+0000 7fa5d76cd680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T00:25:43.640 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:42.214381+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:43.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:42.214381+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:44.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:43.941+0000 7fa5d76cd680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T00:25:44.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:43.470565+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:44.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:43.472646+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:44.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:43.470565+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:44.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:43.472646+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:44.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:43.470565+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:43.472646+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:44.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:44.711+0000 7fa5d76cd680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T00:25:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-18T00:25:44.794 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-18T00:25:44.795 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-02-18T00:25:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-02-18T00:25:44.795 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110564 s, 948 MB/s 2024-02-18T00:25:44.795 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-02-18T00:25:45.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:44.795+0000 7fa5d76cd680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T00:25:45.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:45.003+0000 7fa5d76cd680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T00:25:45.726 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:44.214700+0000 mgr.y (mgr.14146) 65 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:44.506894+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:44.507516+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:44.508209+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:44.512266+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:44.513868+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:44.514101+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:44.514798+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:44.515340+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:44.515856+0000 mgr.y (mgr.14146) 67 : cephadm [INF] Reconfiguring daemon mgr.y on smithi059 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:45.353663+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.727 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:44.214700+0000 mgr.y (mgr.14146) 65 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:44.506894+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:44.507516+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:44.508209+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:44.512266+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:44.513868+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:44.514101+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:44.514798+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:44.515340+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:44.515856+0000 mgr.y (mgr.14146) 67 : cephadm [INF] Reconfiguring daemon mgr.y on smithi059 2024-02-18T00:25:45.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:45.353663+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:44.214700+0000 mgr.y (mgr.14146) 65 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:44.506894+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:44.507516+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:44.508209+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:44.512266+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:44.513868+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:44.514101+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:44.514798+0000 mon.a (mon.0) 206 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:44.515340+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:44.515856+0000 mgr.y (mgr.14146) 67 : cephadm [INF] Reconfiguring daemon mgr.y on smithi059 2024-02-18T00:25:45.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:45.353663+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:46.041 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2024-02-18T00:25:46.227 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:45.935+0000 7fa5d76cd680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T00:25:46.227 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:46.061+0000 7fa5d76cd680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:25:46.227 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:46.138+0000 7fa5d76cd680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T00:25:46.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:46.225+0000 7fa5d76cd680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T00:25:46.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:46.421+0000 7fa5d76cd680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T00:25:47.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:46.697+0000 7fa5d76cd680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T00:25:47.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:46.793+0000 7fa5d76cd680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T00:25:47.847 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:47.506+0000 7fa5d76cd680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T00:25:47.847 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:47.586+0000 7fa5d76cd680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T00:25:47.847 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:47.676+0000 7fa5d76cd680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T00:25:47.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:46.215129+0000 mgr.y (mgr.14146) 68 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:47.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:46.692475+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:47.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:46.693340+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:47.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:46.694674+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:47.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:46.695263+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:47.848 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:46.699289+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:47.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:46.215129+0000 mgr.y (mgr.14146) 68 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:47.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:46.692475+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:46.693340+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:46.694674+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:46.695263+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:46.699289+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:46.215129+0000 mgr.y (mgr.14146) 68 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:46.692475+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:46.693340+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:46.694674+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:46.695263+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:25:47.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:46.699289+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:25:48.364 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:47.845+0000 7fa5d76cd680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T00:25:48.365 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:48.103+0000 7fa5d76cd680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T00:25:48.365 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:48.232+0000 7fa5d76cd680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T00:25:48.618 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:48.363+0000 7fa5d76cd680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:25:48.619 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:48.534+0000 7fa5d76cd680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T00:25:48.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 -- 172.21.15.59:0/1584810080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 msgr2=0x7f36200fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:48.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 --2- 172.21.15.59:0/1584810080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f36200fff90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f3618009c60 tx=0x7f3618004c50).stop 2024-02-18T00:25:48.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 -- 172.21.15.59:0/1584810080 shutdown_connections 2024-02-18T00:25:48.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 --2- 172.21.15.59:0/1584810080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7aa0 0x7f36200f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:48.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 --2- 172.21.15.59:0/1584810080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f36200fff90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:48.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 --2- 172.21.15.59:0/1584810080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 0x7f36200f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 -- 172.21.15.59:0/1584810080 >> 172.21.15.59:0/1584810080 conn(0x7f36200f32a0 msgr2=0x7f36200f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 -- 172.21.15.59:0/1584810080 shutdown_connections 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 -- 172.21.15.59:0/1584810080 wait complete. 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 librados: starting msgr at 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 librados: starting objecter 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 Processor -- start 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 -- start start 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 librados: setting wanted keys 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.730+0000 7f36264c8700 1 librados: calling monclient init 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f36264c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7aa0 0x7f362006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f36264c8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 0x7f362006b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f36264c8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f362006b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:48.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f36264c8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362006bb40 con 0x7f36200f8d40 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f36264c8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362006bcb0 con 0x7f36200f7aa0 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f36264c8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362006be70 con 0x7f36200f9690 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 0x7f362006b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f3624a65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f362006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 0x7f362006b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49158/0 (socket says 172.21.15.59:49158) 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f3624a65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f362006b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:45684/0 (socket says 172.21.15.59:45684) 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 -- 172.21.15.59:0/1251348293 learned_addr learned my addr 172.21.15.59:0/1251348293 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361ffff700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7aa0 0x7f362006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 -- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 msgr2=0x7f362006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f362006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 -- 172.21.15.59:0/1251348293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7aa0 msgr2=0x7f362006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7aa0 0x7f362006c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 -- 172.21.15.59:0/1251348293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614005040 con 0x7f36200f8d40 2024-02-18T00:25:48.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f3624a65700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f362006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:25:48.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361f7fe700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 0x7f362006b030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3614009430 tx=0x7f3614009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361d7fa700 10 client.?.objecter ms_handle_connect 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.731+0000 7f361d7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.732+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614007a40 con 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.732+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f361400f040 con 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.732+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36180097c0 con 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.732+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36140149d0 con 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.732+0000 7f36264c8700 10 client.14211.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.732+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36201124c0 con 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.732+0000 7f36264c8700 1 librados: init done 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) v1 ==== 38638+0+0 (secure 0 0 0) 0x7f36140133c0 con 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 1 --2- 172.21.15.59:0/1251348293 >> v2:172.21.15.59:6800/817089002 conn(0x7f3608034230 0x7f36080366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (secure 0 0 0) 0x7f3614042a80 con 0x7f36200f8d40 2024-02-18T00:25:48.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 10 client.14211.objecter ms_dispatch 0x7f3620001550 osd_map(4..4 src has 1..4) v4 2024-02-18T00:25:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 3 client.14211.objecter handle_osd_map got epochs [4,4] > 0 2024-02-18T00:25:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 3 client.14211.objecter handle_osd_map decoding full epoch 4 2024-02-18T00:25:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 20 client.14211.objecter dump_active .. 0 homeless 2024-02-18T00:25:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361ffff700 1 --2- 172.21.15.59:0/1251348293 >> v2:172.21.15.59:6800/817089002 conn(0x7f3608034230 0x7f36080366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:25:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361ffff700 1 --2- 172.21.15.59:0/1251348293 >> v2:172.21.15.59:6800/817089002 conn(0x7f3608034230 0x7f36080366f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f36200f8ba0 tx=0x7f361000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:25:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f361d7fa700 10 client.14211.objecter ms_handle_connect 0x7f3608034230 2024-02-18T00:25:48.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.733+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362005f360 con 0x7f36200f8d40 2024-02-18T00:25:48.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.738+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3614014e00 con 0x7f36200f8d40 2024-02-18T00:25:48.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:48.916+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f36200fed80 con 0x7f3608034230 2024-02-18T00:25:49.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:25:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:25:48.617+0000 7fa5d76cd680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T00:25:49.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:49.516+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7f36140136c0 con 0x7f36200f8d40 2024-02-18T00:25:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:48.215409+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:48.631260+0000 mon.a (mon.0) 214 : cluster [DBG] Standby manager daemon x started 2024-02-18T00:25:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.632533+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T00:25:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.632824+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T00:25:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.633740+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T00:25:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.633980+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T00:25:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.920861+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.922724+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.923261+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:48.215409+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:48.631260+0000 mon.a (mon.0) 214 : cluster [DBG] Standby manager daemon x started 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.632533+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.632824+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.633740+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.633980+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.920861+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:25:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.922724+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:25:49.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.923261+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:48.215409+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:48.631260+0000 mon.a (mon.0) 214 : cluster [DBG] Standby manager daemon x started 2024-02-18T00:25:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.632533+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T00:25:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.632824+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T00:25:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.633740+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T00:25:50.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.633980+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.137:0/2563057178' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T00:25:50.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.920861+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:25:50.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.922724+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:25:50.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.923261+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:50.743 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:48.918720+0000 mgr.y (mgr.14146) 70 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:25:50.743 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:49.518944+0000 mon.a (mon.0) 218 : cluster [DBG] mgrmap e14: y(active, since 99s), standbys: x 2024-02-18T00:25:50.743 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:49.519062+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T00:25:50.744 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:48.918720+0000 mgr.y (mgr.14146) 70 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:25:50.744 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:49.518944+0000 mon.a (mon.0) 218 : cluster [DBG] mgrmap e14: y(active, since 99s), standbys: x 2024-02-18T00:25:50.744 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:49.519062+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T00:25:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:48.918720+0000 mgr.y (mgr.14146) 70 : audit [DBG] from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:25:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:49.518944+0000 mon.a (mon.0) 218 : cluster [DBG] mgrmap e14: y(active, since 99s), standbys: x 2024-02-18T00:25:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:49.519062+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T00:25:52.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:50.215766+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:50.215766+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:50.215766+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:51.590319+0000 mon.a (mon.0) 220 : audit [INF] from='client.? 172.21.15.59:0/4118352113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85"}]: dispatch 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:51.595419+0000 mon.a (mon.0) 221 : audit [INF] from='client.? 172.21.15.59:0/4118352113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85"}]': finished 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:51.595472+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:51.595580+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:52.165114+0000 mon.a (mon.0) 224 : audit [DBG] from='client.? 172.21.15.59:0/393342879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:51.590319+0000 mon.a (mon.0) 220 : audit [INF] from='client.? 172.21.15.59:0/4118352113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85"}]: dispatch 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:51.595419+0000 mon.a (mon.0) 221 : audit [INF] from='client.? 172.21.15.59:0/4118352113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85"}]': finished 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:51.595472+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:51.595580+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:25:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:52.165114+0000 mon.a (mon.0) 224 : audit [DBG] from='client.? 172.21.15.59:0/393342879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:25:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:51.590319+0000 mon.a (mon.0) 220 : audit [INF] from='client.? 172.21.15.59:0/4118352113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85"}]: dispatch 2024-02-18T00:25:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:51.595419+0000 mon.a (mon.0) 221 : audit [INF] from='client.? 172.21.15.59:0/4118352113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85"}]': finished 2024-02-18T00:25:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:51.595472+0000 mon.a (mon.0) 222 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-18T00:25:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:51.595580+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:25:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:52.165114+0000 mon.a (mon.0) 224 : audit [DBG] from='client.? 172.21.15.59:0/393342879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:25:53.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:53.733+0000 7f3625cc7700 10 client.14211.objecter tick 2024-02-18T00:25:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:52.215989+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:54.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:52.215989+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:54.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:52.215989+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:55.969 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:54.216374+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:55.969 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:54.216374+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:56.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:54.216374+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:57.926 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:56.216659+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:57.927 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:56.216659+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:56.216659+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:58.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:25:58.734+0000 7f3625cc7700 10 client.14211.objecter tick 2024-02-18T00:25:58.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:58.229649+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T00:25:58.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:25:58.230430+0000 mon.a (mon.0) 226 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:58.912 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:58.229649+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T00:25:58.912 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:25:58.230430+0000 mon.a (mon.0) 226 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:58.229649+0000 mon.a (mon.0) 225 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T00:25:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:25:58.230430+0000 mon.a (mon.0) 226 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:25:59.899 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:25:58.216922+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:59.899 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:25:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:25:58.231070+0000 mgr.y (mgr.14146) 76 : cephadm [INF] Deploying daemon osd.0 on smithi059 2024-02-18T00:25:59.899 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:25:58.216922+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:25:59.899 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:25:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:25:58.231070+0000 mgr.y (mgr.14146) 76 : cephadm [INF] Deploying daemon osd.0 on smithi059 2024-02-18T00:26:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:25:58.216922+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:25:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:25:58.231070+0000 mgr.y (mgr.14146) 76 : cephadm [INF] Deploying daemon osd.0 on smithi059 2024-02-18T00:26:00.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:00.217182+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:00.923 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:00.217182+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:00.217182+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:03.580 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:02.217419+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:03.581 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:02.217419+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:03.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:03.735+0000 7f3625cc7700 10 client.14211.objecter tick 2024-02-18T00:26:03.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:02.217419+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:05.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:03.749175+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:05.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:03.752640+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:05.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:04.217759+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:03.749175+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:05.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:03.752640+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:05.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:04.217759+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:05.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:03.749175+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:05.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:03.752640+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:05.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:04.217759+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:07.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:06.067556+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:07.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:06.068157+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:07.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:06.068762+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:07.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:06.072701+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:07.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:06.218045+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:07.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:06.822440+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v2:172.21.15.59:6801/3700238977' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T00:26:07.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:06.822791+0000 mon.a (mon.0) 233 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T00:26:07.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:06.067556+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:07.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:06.068157+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:07.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:06.068762+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:07.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:06.072701+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:07.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:06.218045+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:07.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:06.822440+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v2:172.21.15.59:6801/3700238977' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T00:26:07.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:06.822791+0000 mon.a (mon.0) 233 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T00:26:07.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.347+0000 7f361d7fa700 1 -- 172.21.15.59:0/1251348293 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f36200fed80 con 0x7f3608034230 2024-02-18T00:26:07.350 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 0 on host 'smithi059' 2024-02-18T00:26:07.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 10 librados: watch_flush enter 2024-02-18T00:26:07.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 10 librados: watch_flush exit 2024-02-18T00:26:07.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 20 client.14211.objecter shutdown clearing up homeless session... 2024-02-18T00:26:07.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 10 client.14211.objecter successfully canceled tick 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 >> v2:172.21.15.59:6800/817089002 conn(0x7f3608034230 msgr2=0x7f36080366f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 1 --2- 172.21.15.59:0/1251348293 >> v2:172.21.15.59:6800/817089002 conn(0x7f3608034230 0x7f36080366f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f36200f8ba0 tx=0x7f361000b040).stop 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 msgr2=0x7f362006b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.349+0000 7f36264c8700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 0x7f362006b030 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3614009430 tx=0x7f3614009cb0).stop 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 shutdown_connections 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 --2- 172.21.15.59:0/1251348293 >> v2:172.21.15.59:6800/817089002 conn(0x7f3608034230 0x7f36080366f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7aa0 0x7f362006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f9690 0x7f362006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 --2- 172.21.15.59:0/1251348293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f8d40 0x7f362006b030 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 >> 172.21.15.59:0/1251348293 conn(0x7f36200f32a0 msgr2=0x7f36200fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 shutdown_connections 2024-02-18T00:26:07.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 -- 172.21.15.59:0/1251348293 wait complete. 2024-02-18T00:26:07.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:07.350+0000 7f36264c8700 1 librados: shutdown 2024-02-18T00:26:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:06.067556+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:06.068157+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:06.068762+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:06.072701+0000 mon.a (mon.0) 232 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:06.218045+0000 mgr.y (mgr.14146) 80 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:06.822440+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 v2:172.21.15.59:6801/3700238977' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T00:26:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:06.822791+0000 mon.a (mon.0) 233 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T00:26:08.121 DEBUG:teuthology.orchestra.run.smithi059:osd.0> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.0.service 2024-02-18T00:26:08.123 INFO:tasks.cephadm:Deploying osd.1 on smithi059 with /dev/vg_nvme/lv_3... 2024-02-18T00:26:08.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-18T00:26:08.155 INFO:journalctl@ceph.osd.0.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:26:08.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:07.077039+0000 mon.a (mon.0) 234 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T00:26:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:07.077098+0000 mon.a (mon.0) 235 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-18T00:26:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:07.077211+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:07.077997+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v2:172.21.15.59:6801/3700238977' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:07.078281+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:07.348220+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:07.390931+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:08.354 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:07.077039+0000 mon.a (mon.0) 234 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T00:26:08.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:07.077098+0000 mon.a (mon.0) 235 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-18T00:26:08.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:07.077211+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:08.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:07.077997+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v2:172.21.15.59:6801/3700238977' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:08.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:07.078281+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:08.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:07.348220+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:08.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:07.390931+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:07.077039+0000 mon.a (mon.0) 234 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T00:26:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:07.077098+0000 mon.a (mon.0) 235 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-18T00:26:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:07.077211+0000 mon.a (mon.0) 236 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:07.077997+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 v2:172.21.15.59:6801/3700238977' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:07.078281+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:07.348220+0000 mon.a (mon.0) 238 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:07.390931+0000 mon.a (mon.0) 239 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:08.079668+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:26:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:08.079756+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-18T00:26:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:08.080116+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:08.084800+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:08.218294+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:08.079668+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:26:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:08.079756+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-18T00:26:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:08.080116+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:08.084800+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:08.218294+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:09.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:08.079668+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:26:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:08.079756+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-18T00:26:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:08.080116+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:08.084800+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:08.218294+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T00:26:10.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:07.835252+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:26:10.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:07.835296+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:26:10.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:09.085562+0000 mon.a (mon.0) 244 : cluster [INF] osd.0 v2:172.21.15.59:6801/3700238977 boot 2024-02-18T00:26:10.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:09.085601+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-18T00:26:10.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:09.085977+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:10.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:07.835252+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:26:10.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:07.835296+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:26:10.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:09.085562+0000 mon.a (mon.0) 244 : cluster [INF] osd.0 v2:172.21.15.59:6801/3700238977 boot 2024-02-18T00:26:10.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:09.085601+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-18T00:26:10.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:09.085977+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:07.835252+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:26:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:07.835296+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:26:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:09.085562+0000 mon.a (mon.0) 244 : cluster [INF] osd.0 v2:172.21.15.59:6801/3700238977 boot 2024-02-18T00:26:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:09.085601+0000 mon.a (mon.0) 245 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-18T00:26:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:09.085977+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:26:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:10.183501+0000 mon.a (mon.0) 247 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-18T00:26:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:10.218681+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:10.183501+0000 mon.a (mon.0) 247 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-18T00:26:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:10.218681+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:11.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:10.183501+0000 mon.a (mon.0) 247 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-18T00:26:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:10.218681+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:11.551 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-18T00:26:11.551 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-18T00:26:11.551 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-02-18T00:26:11.551 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-02-18T00:26:11.551 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110878 s, 946 MB/s 2024-02-18T00:26:11.551 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-02-18T00:26:12.870 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2024-02-18T00:26:13.610 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:12.219126+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:13.610 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:26:12.494758+0000 mgr.y (mgr.14146) 84 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:26:13.610 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:12.498802+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.610 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:12.499561+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:26:12.500015+0000 mgr.y (mgr.14146) 85 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 16102M 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:12.504352+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:12.505083+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:12.505666+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:12.509280+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:12.219126+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:26:12.494758+0000 mgr.y (mgr.14146) 84 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:12.498802+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:12.499561+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:13.611 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:26:12.500015+0000 mgr.y (mgr.14146) 85 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 16102M 2024-02-18T00:26:13.612 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:12.504352+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.612 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:12.505083+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:13.612 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:12.505666+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:13.612 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:12.509280+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:12.219126+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:26:12.494758+0000 mgr.y (mgr.14146) 84 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:26:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:12.498802+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:12.499561+0000 mon.a (mon.0) 249 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:26:12.500015+0000 mgr.y (mgr.14146) 85 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 16102M 2024-02-18T00:26:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:12.504352+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:12.505083+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:13.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:12.505666+0000 mon.a (mon.0) 252 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:13.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:12.509280+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:14.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 -- 172.21.15.59:0/807227139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 msgr2=0x7f86680f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:14.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 --2- 172.21.15.59:0/807227139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 0x7f86680f6d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f865c004870 tx=0x7f865c009cb0).stop 2024-02-18T00:26:14.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 -- 172.21.15.59:0/807227139 shutdown_connections 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 --2- 172.21.15.59:0/807227139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86680f8320 0x7f86680fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 --2- 172.21.15.59:0/807227139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 0x7f86680f6d20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 --2- 172.21.15.59:0/807227139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f7210 0x7f86680f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 -- 172.21.15.59:0/807227139 >> 172.21.15.59:0/807227139 conn(0x7f86680f2120 msgr2=0x7f86680f4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 -- 172.21.15.59:0/807227139 shutdown_connections 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 -- 172.21.15.59:0/807227139 wait complete. 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 librados: starting msgr at 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 librados: starting objecter 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 Processor -- start 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 -- start start 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 librados: setting wanted keys 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 librados: calling monclient init 2024-02-18T00:26:14.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 0x7f8668107ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.995+0000 7f866f245700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86680f7210 0x7f86681084e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866f245700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f8320 0x7f866810f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866f245700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866810fab0 con 0x7f86680f6900 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866f245700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866810fbf0 con 0x7f86680f7210 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866f245700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866810fd30 con 0x7f86680f8320 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866d7e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f8320 0x7f866810f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866cfe1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 0x7f8668107ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866d7e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f8320 0x7f866810f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39814/0 (socket says 172.21.15.59:39814) 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866d7e2700 1 -- 172.21.15.59:0/2489862587 learned_addr learned my addr 172.21.15.59:0/2489862587 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f8667fff700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86680f7210 0x7f86681084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866cfe1700 1 -- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f8320 msgr2=0x7f866810f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866cfe1700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f8320 0x7f866810f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:15.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866cfe1700 1 -- 172.21.15.59:0/2489862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86680f7210 msgr2=0x7f86681084e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866cfe1700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86680f7210 0x7f86681084e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866cfe1700 1 -- 172.21.15.59:0/2489862587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c005040 con 0x7f86680f6900 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866d7e2700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f8320 0x7f866810f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866cfe1700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 0x7f8668107ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f865c004840 tx=0x7f865c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f8665ffb700 10 client.?.objecter ms_handle_connect 0x7f86680f6900 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f8665ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f8665ffb700 1 -- 172.21.15.59:0/2489862587 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00e030 con 0x7f86680f6900 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f8665ffb700 1 -- 172.21.15.59:0/2489862587 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f865c007d50 con 0x7f86680f6900 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866810ffb0 con 0x7f86680f6900 2024-02-18T00:26:15.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.996+0000 7f8665ffb700 1 -- 172.21.15.59:0/2489862587 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c019070 con 0x7f86680f6900 2024-02-18T00:26:15.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.997+0000 7f866f245700 10 client.14238.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:26:15.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.997+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8668110300 con 0x7f86680f6900 2024-02-18T00:26:15.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.997+0000 7f866f245700 1 librados: init done 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.997+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866805ffd0 con 0x7f86680f6900 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8665ffb700 1 -- 172.21.15.59:0/2489862587 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7f865c01e420 con 0x7f86680f6900 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8665ffb700 1 --2- 172.21.15.59:0/2489862587 >> v2:172.21.15.59:6800/817089002 conn(0x7f8658062ba0 0x7f8658065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8665ffb700 1 -- 172.21.15.59:0/2489862587 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1484+0+0 (secure 0 0 0) 0x7f865c07c960 con 0x7f86680f6900 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8665ffb700 10 client.14238.objecter ms_dispatch 0x7f8668001550 osd_map(9..9 src has 1..9) v4 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8665ffb700 3 client.14238.objecter handle_osd_map got epochs [9,9] > 0 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8665ffb700 3 client.14238.objecter handle_osd_map decoding full epoch 9 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8667fff700 1 --2- 172.21.15.59:0/2489862587 >> v2:172.21.15.59:6800/817089002 conn(0x7f8658062ba0 0x7f8658065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.998+0000 7f8665ffb700 20 client.14238.objecter dump_active .. 0 homeless 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.999+0000 7f8667fff700 1 --2- 172.21.15.59:0/2489862587 >> v2:172.21.15.59:6800/817089002 conn(0x7f8658062ba0 0x7f8658065060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f86540048b0 tx=0x7f865400b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:26:15.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:14.999+0000 7f8665ffb700 10 client.14238.objecter ms_handle_connect 0x7f8658062ba0 2024-02-18T00:26:15.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:15.003+0000 7f8665ffb700 1 -- 172.21.15.59:0/2489862587 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f865c04de60 con 0x7f86680f6900 2024-02-18T00:26:15.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:15.182+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f86680616b0 con 0x7f8658062ba0 2024-02-18T00:26:15.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:14.219570+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:15.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:15.186511+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:26:15.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:15.188312+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:26:15.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:15.188807+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:15.887 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:14.219570+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:15.887 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:15.186511+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:26:15.888 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:15.188312+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:26:15.888 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:15.188807+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:15.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:14.219570+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:15.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:15.186511+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:26:15.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:15.188312+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:26:15.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:15.188807+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:15.184566+0000 mgr.y (mgr.14146) 87 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:26:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:15.184566+0000 mgr.y (mgr.14146) 87 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:26:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:15.184566+0000 mgr.y (mgr.14146) 87 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:26:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:16.219876+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:18.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:16.219876+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:18.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:16.219876+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:19.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:17.670220+0000 mon.a (mon.0) 257 : audit [INF] from='client.? 172.21.15.59:0/3426709646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0"}]: dispatch 2024-02-18T00:26:19.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:17.675934+0000 mon.a (mon.0) 258 : audit [INF] from='client.? 172.21.15.59:0/3426709646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0"}]': finished 2024-02-18T00:26:19.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:17.675970+0000 mon.a (mon.0) 259 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-18T00:26:19.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:17.676058+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:19.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:18.322027+0000 mon.a (mon.0) 261 : audit [DBG] from='client.? 172.21.15.59:0/3740559408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:26:19.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:17.670220+0000 mon.a (mon.0) 257 : audit [INF] from='client.? 172.21.15.59:0/3426709646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0"}]: dispatch 2024-02-18T00:26:19.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:17.675934+0000 mon.a (mon.0) 258 : audit [INF] from='client.? 172.21.15.59:0/3426709646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0"}]': finished 2024-02-18T00:26:19.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:17.675970+0000 mon.a (mon.0) 259 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-18T00:26:19.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:17.676058+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:19.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:18.322027+0000 mon.a (mon.0) 261 : audit [DBG] from='client.? 172.21.15.59:0/3740559408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:26:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:17.670220+0000 mon.a (mon.0) 257 : audit [INF] from='client.? 172.21.15.59:0/3426709646' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0"}]: dispatch 2024-02-18T00:26:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:17.675934+0000 mon.a (mon.0) 258 : audit [INF] from='client.? 172.21.15.59:0/3426709646' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0"}]': finished 2024-02-18T00:26:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:17.675970+0000 mon.a (mon.0) 259 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-18T00:26:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:17.676058+0000 mon.a (mon.0) 260 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:18.322027+0000 mon.a (mon.0) 261 : audit [DBG] from='client.? 172.21.15.59:0/3740559408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:26:20.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:19.998+0000 7f866ea44700 10 client.14238.objecter tick 2024-02-18T00:26:20.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:18.220142+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:18.220142+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:20.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:18.220142+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:21.770 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:20.220445+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:21.770 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:20.220445+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:20.220445+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:23.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:22.220869+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:23.966 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:22.220869+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:22.220869+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:24.318307+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T00:26:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:24.319005+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:24.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:24.318307+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T00:26:24.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:24.319005+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:25.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:24.999+0000 7f866ea44700 10 client.14238.objecter tick 2024-02-18T00:26:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:24.318307+0000 mon.a (mon.0) 262 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T00:26:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:24.319005+0000 mon.a (mon.0) 263 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:25.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:24.221099+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:25.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:26:24.319573+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Deploying daemon osd.1 on smithi059 2024-02-18T00:26:25.838 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:24.221099+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:25.839 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:26:24.319573+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Deploying daemon osd.1 on smithi059 2024-02-18T00:26:26.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:24.221099+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:26.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:26:24.319573+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Deploying daemon osd.1 on smithi059 2024-02-18T00:26:27.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:26.221428+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:27.975 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:26.221428+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:28.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:26.221428+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:29.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:28.221688+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:29.837 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:28.221688+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:30.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:30.000+0000 7f866ea44700 10 client.14238.objecter tick 2024-02-18T00:26:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:28.221688+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:29.953773+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:29.960625+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:30.222129+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:29.953773+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:29.960625+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:30.222129+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:29.953773+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:29.960625+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:30.222129+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:33.580 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:32.222421+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:33.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:32.482530+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:33.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:32.845227+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:33.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:32.845996+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:33.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:32.850990+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:33.581 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.069797+0000 mon.c (mon.2) 4 : audit [INF] from='osd.1 v2:172.21.15.59:6805/26279110' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T00:26:33.582 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.070105+0000 mon.a (mon.0) 270 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T00:26:33.582 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:32.222421+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:33.582 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:32.482530+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:33.582 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:32.845227+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:33.582 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:32.845996+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:33.583 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:32.850990+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:33.584 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.069797+0000 mon.c (mon.2) 4 : audit [INF] from='osd.1 v2:172.21.15.59:6805/26279110' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T00:26:33.584 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.070105+0000 mon.a (mon.0) 270 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T00:26:33.605 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 1 on host 'smithi059' 2024-02-18T00:26:33.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.603+0000 7f8665ffb700 1 -- 172.21.15.59:0/2489862587 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f86680616b0 con 0x7f8658062ba0 2024-02-18T00:26:33.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 10 librados: watch_flush enter 2024-02-18T00:26:33.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 10 librados: watch_flush exit 2024-02-18T00:26:33.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 20 client.14238.objecter shutdown clearing up homeless session... 2024-02-18T00:26:33.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 10 client.14238.objecter successfully canceled tick 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 >> v2:172.21.15.59:6800/817089002 conn(0x7f8658062ba0 msgr2=0x7f8658065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 --2- 172.21.15.59:0/2489862587 >> v2:172.21.15.59:6800/817089002 conn(0x7f8658062ba0 0x7f8658065060 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f86540048b0 tx=0x7f865400b040).stop 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 msgr2=0x7f8668107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 0x7f8668107ff0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f865c004840 tx=0x7f865c00b040).stop 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 shutdown_connections 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 --2- 172.21.15.59:0/2489862587 >> v2:172.21.15.59:6800/817089002 conn(0x7f8658062ba0 0x7f8658065060 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86680f8320 0x7f866810f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86680f6900 0x7f8668107ff0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 --2- 172.21.15.59:0/2489862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86680f7210 0x7f86681084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:33.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 >> 172.21.15.59:0/2489862587 conn(0x7f86680f2120 msgr2=0x7f8668060fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:26:33.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 shutdown_connections 2024-02-18T00:26:33.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 -- 172.21.15.59:0/2489862587 wait complete. 2024-02-18T00:26:33.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:33.605+0000 7f866f245700 1 librados: shutdown 2024-02-18T00:26:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:32.222421+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:32.482530+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:32.845227+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:32.845996+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:32.850990+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.069797+0000 mon.c (mon.2) 4 : audit [INF] from='osd.1 v2:172.21.15.59:6805/26279110' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T00:26:33.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.070105+0000 mon.a (mon.0) 270 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T00:26:34.399 DEBUG:teuthology.orchestra.run.smithi059:osd.1> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.1.service 2024-02-18T00:26:34.401 INFO:tasks.cephadm:Deploying osd.2 on smithi059 with /dev/vg_nvme/lv_2... 2024-02-18T00:26:34.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-18T00:26:34.439 INFO:journalctl@ceph.osd.1.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:26:34.723 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.603562+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:34.723 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.618331+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.852380+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:33.852490+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.852611+0000 mon.a (mon.0) 275 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.853059+0000 mon.c (mon.2) 5 : audit [INF] from='osd.1 v2:172.21.15.59:6805/26279110' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:33.853350+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.603562+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.618331+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.852380+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T00:26:34.724 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:33.852490+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-18T00:26:34.725 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.852611+0000 mon.a (mon.0) 275 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:34.725 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.853059+0000 mon.c (mon.2) 5 : audit [INF] from='osd.1 v2:172.21.15.59:6805/26279110' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:34.725 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:33.853350+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:35.000 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-1[53219]: debug 2024-02-18T00:26:34.855+0000 7f146ea2a700 -1 osd.1 0 waiting for initial osdmap 2024-02-18T00:26:35.000 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:26:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-1[53219]: debug 2024-02-18T00:26:34.858+0000 7f14691a2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T00:26:35.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.603562+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.618331+0000 mon.a (mon.0) 272 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.852380+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T00:26:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:33.852490+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-18T00:26:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.852611+0000 mon.a (mon.0) 275 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.853059+0000 mon.c (mon.2) 5 : audit [INF] from='osd.1 v2:172.21.15.59:6805/26279110' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:33.853350+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:26:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:34.222815+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:34.853716+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:26:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:34.853767+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-18T00:26:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:34.853877+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:34.856776+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:34.222815+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:34.853716+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:26:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:34.853767+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-18T00:26:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:34.853877+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:34.856776+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:36.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:34.222815+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:36.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:34.853716+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:26:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:34.853767+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-18T00:26:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:34.853877+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:34.856776+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:37.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:34.056902+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:34.057016+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:35.862103+0000 mon.a (mon.0) 281 : cluster [INF] osd.1 v2:172.21.15.59:6805/26279110 boot 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:35.862170+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:35.862356+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:36.223228+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:34.056902+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:34.057016+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:35.862103+0000 mon.a (mon.0) 281 : cluster [INF] osd.1 v2:172.21.15.59:6805/26279110 boot 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:35.862170+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-18T00:26:37.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:35.862356+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:37.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:36.223228+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:34.056902+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:26:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:34.057016+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:26:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:35.862103+0000 mon.a (mon.0) 281 : cluster [INF] osd.1 v2:172.21.15.59:6805/26279110 boot 2024-02-18T00:26:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:35.862170+0000 mon.a (mon.0) 282 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-18T00:26:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:35.862356+0000 mon.a (mon.0) 283 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:26:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:36.223228+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T00:26:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-18T00:26:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-18T00:26:38.158 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-02-18T00:26:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-02-18T00:26:38.158 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109171 s, 960 MB/s 2024-02-18T00:26:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-02-18T00:26:38.193 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:36.863727+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-18T00:26:38.193 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:36.863727+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-18T00:26:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:36.863727+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-18T00:26:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:38.223519+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:38.223519+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:38.223519+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:39.427 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2024-02-18T00:26:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:26:38.955474+0000 mgr.y (mgr.14146) 101 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:26:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:38.959666+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:38.973192+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:38.973787+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:26:38.974196+0000 mgr.y (mgr.14146) 102 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 8051M 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:38.978385+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:38.979433+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:38.980005+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:38.984310+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:26:38.955474+0000 mgr.y (mgr.14146) 101 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:38.959666+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:38.973192+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:38.973787+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:26:38.974196+0000 mgr.y (mgr.14146) 102 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 8051M 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:38.978385+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:38.979433+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:38.980005+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:38.984310+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:26:38.955474+0000 mgr.y (mgr.14146) 101 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:38.959666+0000 mon.a (mon.0) 285 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:38.973192+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:38.973787+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:26:38.974196+0000 mgr.y (mgr.14146) 102 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 8051M 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:38.978385+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:38.979433+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:38.980005+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:38.984310+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:41.179 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:40.223946+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:41.180 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:40.223946+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:40.223946+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 -- 172.21.15.59:0/672647951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f7a80 msgr2=0x7fce300f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 --2- 172.21.15.59:0/672647951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f7a80 0x7fce300f7ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fce20004870 tx=0x7fce20009cb0).stop 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 -- 172.21.15.59:0/672647951 shutdown_connections 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 --2- 172.21.15.59:0/672647951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce300f9670 0x7fce300fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 --2- 172.21.15.59:0/672647951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f7a80 0x7fce300f7ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 --2- 172.21.15.59:0/672647951 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f8d20 0x7fce300f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 -- 172.21.15.59:0/672647951 >> 172.21.15.59:0/672647951 conn(0x7fce300f32a0 msgr2=0x7fce300f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 -- 172.21.15.59:0/672647951 shutdown_connections 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.459+0000 7fce38485700 1 -- 172.21.15.59:0/672647951 wait complete. 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 librados: starting msgr at 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 librados: starting objecter 2024-02-18T00:26:41.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 Processor -- start 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 -- start start 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 librados: setting wanted keys 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 librados: calling monclient init 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f7a80 0x7fce3006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f8d20 0x7fce3006b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce300f9670 0x7fce3006b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3006bb40 con 0x7fce300f8d20 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3006bcb0 con 0x7fce300f9670 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce38485700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3006be70 con 0x7fce300f7a80 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f7a80 0x7fce3006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce35a20700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f8d20 0x7fce3006b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f7a80 0x7fce3006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38138/0 (socket says 172.21.15.59:38138) 2024-02-18T00:26:41.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 -- 172.21.15.59:0/2540753835 learned_addr learned my addr 172.21.15.59:0/2540753835 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36a22700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce300f9670 0x7fce3006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 -- 172.21.15.59:0/2540753835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce300f9670 msgr2=0x7fce3006b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce300f9670 0x7fce3006b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 -- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f8d20 msgr2=0x7fce3006b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f8d20 0x7fce3006b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36221700 1 -- 172.21.15.59:0/2540753835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20005040 con 0x7fce300f7a80 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce36a22700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce300f9670 0x7fce3006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.460+0000 7fce35a20700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f8d20 0x7fce3006b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce36221700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f7a80 0x7fce3006c9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fce20009ee0 tx=0x7fce2000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce277fe700 10 client.?.objecter ms_handle_connect 0x7fce300f7a80 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce277fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce277fe700 1 -- 172.21.15.59:0/2540753835 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000e030 con 0x7fce300f7a80 2024-02-18T00:26:41.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce277fe700 1 -- 172.21.15.59:0/2540753835 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce2001e420 con 0x7fce300f7a80 2024-02-18T00:26:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce301122c0 con 0x7fce300f7a80 2024-02-18T00:26:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce277fe700 1 -- 172.21.15.59:0/2540753835 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20019070 con 0x7fce300f7a80 2024-02-18T00:26:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce38485700 10 client.24173.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:26:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce301126f0 con 0x7fce300f7a80 2024-02-18T00:26:41.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.461+0000 7fce38485700 1 librados: init done 2024-02-18T00:26:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.462+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3005f360 con 0x7fce300f7a80 2024-02-18T00:26:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.462+0000 7fce277fe700 1 -- 172.21.15.59:0/2540753835 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7fce200090d0 con 0x7fce300f7a80 2024-02-18T00:26:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce277fe700 1 --2- 172.21.15.59:0/2540753835 >> v2:172.21.15.59:6800/817089002 conn(0x7fce1c062ba0 0x7fce1c065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:26:41.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce277fe700 1 -- 172.21.15.59:0/2540753835 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 1776+0+0 (secure 0 0 0) 0x7fce2007c980 con 0x7fce300f7a80 2024-02-18T00:26:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce277fe700 10 client.24173.objecter ms_dispatch 0x7fce30001550 osd_map(14..14 src has 1..14) v4 2024-02-18T00:26:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce277fe700 3 client.24173.objecter handle_osd_map got epochs [14,14] > 0 2024-02-18T00:26:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce277fe700 3 client.24173.objecter handle_osd_map decoding full epoch 14 2024-02-18T00:26:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce277fe700 20 client.24173.objecter dump_active .. 0 homeless 2024-02-18T00:26:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce35a20700 1 --2- 172.21.15.59:0/2540753835 >> v2:172.21.15.59:6800/817089002 conn(0x7fce1c062ba0 0x7fce1c065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:26:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce277fe700 10 client.24173.objecter ms_handle_connect 0x7fce1c062ba0 2024-02-18T00:26:41.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.463+0000 7fce35a20700 1 --2- 172.21.15.59:0/2540753835 >> v2:172.21.15.59:6800/817089002 conn(0x7fce1c062ba0 0x7fce1c065060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fce2c007bf0 tx=0x7fce2c00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:26:41.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.467+0000 7fce277fe700 1 -- 172.21.15.59:0/2540753835 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fce2004dda0 con 0x7fce300f7a80 2024-02-18T00:26:41.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:41.648+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fce300fedd0 con 0x7fce1c062ba0 2024-02-18T00:26:42.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:41.651913+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:26:42.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:41.653807+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:26:42.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:41.654372+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:42.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:41.651913+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:26:42.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:41.653807+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:26:42.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:41.654372+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:42.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:41.651913+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:26:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:41.653807+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:26:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:41.654372+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:43.139 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:41.649925+0000 mgr.y (mgr.14146) 104 : audit [DBG] from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:26:43.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:42.224400+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:43.140 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:41.649925+0000 mgr.y (mgr.14146) 104 : audit [DBG] from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:26:43.140 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:42.224400+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:41.649925+0000 mgr.y (mgr.14146) 104 : audit [DBG] from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:26:43.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:42.224400+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:44.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:44.275617+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.15.59:0/421480885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7"}]: dispatch 2024-02-18T00:26:44.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:44.289032+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.15.59:0/421480885' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7"}]': finished 2024-02-18T00:26:44.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:44.289085+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-18T00:26:44.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:44.289184+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:26:44.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:44.275617+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.15.59:0/421480885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7"}]: dispatch 2024-02-18T00:26:44.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:44.289032+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.15.59:0/421480885' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7"}]': finished 2024-02-18T00:26:44.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:44.289085+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-18T00:26:44.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:44.289184+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:26:44.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:44.275617+0000 mon.a (mon.0) 295 : audit [INF] from='client.? 172.21.15.59:0/421480885' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7"}]: dispatch 2024-02-18T00:26:44.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:44.289032+0000 mon.a (mon.0) 296 : audit [INF] from='client.? 172.21.15.59:0/421480885' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7"}]': finished 2024-02-18T00:26:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:44.289085+0000 mon.a (mon.0) 297 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-18T00:26:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:44.289184+0000 mon.a (mon.0) 298 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:26:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:44.224676+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:44.922817+0000 mon.a (mon.0) 299 : audit [DBG] from='client.? 172.21.15.59:0/4103188094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:26:45.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:44.224676+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:45.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:44.922817+0000 mon.a (mon.0) 299 : audit [DBG] from='client.? 172.21.15.59:0/4103188094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:26:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:44.224676+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:44.922817+0000 mon.a (mon.0) 299 : audit [DBG] from='client.? 172.21.15.59:0/4103188094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:26:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:46.462+0000 7fce37c84700 10 client.24173.objecter tick 2024-02-18T00:26:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:46.224983+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:46.224983+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:46.224983+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:49.521 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:48.225250+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:49.521 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:48.225250+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:49.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:48.225250+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:51.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:51.463+0000 7fce37c84700 10 client.24173.objecter tick 2024-02-18T00:26:51.730 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:50.225696+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:51.730 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:50.988715+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T00:26:51.730 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:50.989396+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:51.730 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:50.225696+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:51.730 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:50.988715+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T00:26:51.730 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:50.989396+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:50.225696+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:50.988715+0000 mon.a (mon.0) 300 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T00:26:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:50.989396+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:52.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:26:50.989932+0000 mgr.y (mgr.14146) 110 : cephadm [INF] Deploying daemon osd.2 on smithi059 2024-02-18T00:26:52.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:26:50.989932+0000 mgr.y (mgr.14146) 110 : cephadm [INF] Deploying daemon osd.2 on smithi059 2024-02-18T00:26:52.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:26:50.989932+0000 mgr.y (mgr.14146) 110 : cephadm [INF] Deploying daemon osd.2 on smithi059 2024-02-18T00:26:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:52.225965+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:52.225965+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:52.225965+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:54.226239+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:54.226239+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:55.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:54.226239+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:56.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:26:56.464+0000 7fce37c84700 10 client.24173.objecter tick 2024-02-18T00:26:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:56.226771+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:56.442950+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:56.485795+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:57.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:56.226771+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:57.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:56.442950+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:57.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:56.485795+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:56.226771+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:56.442950+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:56.485795+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:26:58.227220+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:59.291648+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:59.292396+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:59.293063+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:59.297070+0000 mon.a (mon.0) 307 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:26:58.227220+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:59.291648+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:59.722 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:59.292396+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:59.723 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:59.293063+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:59.723 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:26:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:59.297070+0000 mon.a (mon.0) 307 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:59.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:26:58.227220+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:26:59.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:59.291648+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:26:59.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:59.292396+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:26:59.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:59.293063+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:26:59.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:26:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:59.297070+0000 mon.a (mon.0) 307 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:00.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.371+0000 7fce277fe700 1 -- 172.21.15.59:0/2540753835 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fce300fedd0 con 0x7fce1c062ba0 2024-02-18T00:27:00.374 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 2 on host 'smithi059' 2024-02-18T00:27:00.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.373+0000 7fce38485700 10 librados: watch_flush enter 2024-02-18T00:27:00.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 10 librados: watch_flush exit 2024-02-18T00:27:00.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 20 client.24173.objecter shutdown clearing up homeless session... 2024-02-18T00:27:00.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 10 client.24173.objecter successfully canceled tick 2024-02-18T00:27:00.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 >> v2:172.21.15.59:6800/817089002 conn(0x7fce1c062ba0 msgr2=0x7fce1c065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:00.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 --2- 172.21.15.59:0/2540753835 >> v2:172.21.15.59:6800/817089002 conn(0x7fce1c062ba0 0x7fce1c065060 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fce2c007bf0 tx=0x7fce2c00b040).stop 2024-02-18T00:27:00.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f7a80 msgr2=0x7fce3006c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f7a80 0x7fce3006c9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fce20009ee0 tx=0x7fce2000b040).stop 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 shutdown_connections 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 --2- 172.21.15.59:0/2540753835 >> v2:172.21.15.59:6800/817089002 conn(0x7fce1c062ba0 0x7fce1c065060 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce300f9670 0x7fce3006b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fce300f7a80 0x7fce3006c9d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 --2- 172.21.15.59:0/2540753835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce300f8d20 0x7fce3006b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 >> 172.21.15.59:0/2540753835 conn(0x7fce300f32a0 msgr2=0x7fce300f66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 shutdown_connections 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 -- 172.21.15.59:0/2540753835 wait complete. 2024-02-18T00:27:00.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:00.374+0000 7fce38485700 1 librados: shutdown 2024-02-18T00:27:00.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:26:59.523243+0000 mon.a (mon.0) 308 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T00:27:00.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:00.371120+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:00.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:00.415218+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:00.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:26:59.523243+0000 mon.a (mon.0) 308 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T00:27:00.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:00.371120+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:00.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:00.415218+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:00.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:26:59.523243+0000 mon.a (mon.0) 308 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T00:27:00.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:00.371120+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:00.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:00.415218+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:01.137 DEBUG:teuthology.orchestra.run.smithi059:osd.2> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.2.service 2024-02-18T00:27:01.140 INFO:tasks.cephadm:Deploying osd.3 on smithi059 with /dev/vg_nvme/lv_1... 2024-02-18T00:27:01.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-18T00:27:01.172 INFO:journalctl@ceph.osd.2.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:27:01.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:00.227588+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:27:01.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:00.456123+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T00:27:01.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:00.456190+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-18T00:27:01.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:00.456314+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:01.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:00.456813+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:27:01.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:00.227588+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:27:01.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:00.456123+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T00:27:01.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:00.456190+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-18T00:27:01.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:00.456314+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:01.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:00.456813+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:27:01.665 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2[57675]: debug 2024-02-18T00:27:01.461+0000 7f0e6f7f6700 -1 osd.2 0 waiting for initial osdmap 2024-02-18T00:27:01.665 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:27:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2[57675]: debug 2024-02-18T00:27:01.464+0000 7f0e6b771700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T00:27:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:00.227588+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:27:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:00.456123+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T00:27:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:00.456190+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-18T00:27:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:00.456314+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:00.456813+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:01.456643+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:01.456715+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:01.458623+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:01.461469+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:02.461413+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:01.456643+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:01.456715+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-18T00:27:03.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:01.458623+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:01.461469+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:02.461413+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:01.456643+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 v2:172.21.15.59:6809/1153193637' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:27:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:01.456715+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-18T00:27:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:01.458623+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:01.461469+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:02.461413+0000 mon.a (mon.0) 319 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:04.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:00.510952+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:04.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:00.511001+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:04.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:02.227917+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:02.468824+0000 mon.a (mon.0) 320 : cluster [INF] osd.2 v2:172.21.15.59:6809/1153193637 boot 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:02.468863+0000 mon.a (mon.0) 321 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:02.469206+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:02.469887+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:03.474196+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:03.474256+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:03.475024+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:00.510952+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:00.511001+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:02.227917+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:27:04.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:02.468824+0000 mon.a (mon.0) 320 : cluster [INF] osd.2 v2:172.21.15.59:6809/1153193637 boot 2024-02-18T00:27:04.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:02.468863+0000 mon.a (mon.0) 321 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-18T00:27:04.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:02.469206+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:04.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:02.469887+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T00:27:04.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:03.474196+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-18T00:27:04.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:03.474256+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-18T00:27:04.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:03.475024+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:00.510952+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:00.511001+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:02.227917+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:02.468824+0000 mon.a (mon.0) 320 : cluster [INF] osd.2 v2:172.21.15.59:6809/1153193637 boot 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:02.468863+0000 mon.a (mon.0) 321 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:02.469206+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:02.469887+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:03.474196+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-18T00:27:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:03.474256+0000 mon.a (mon.0) 325 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-18T00:27:04.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:03.475024+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T00:27:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-18T00:27:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-18T00:27:04.387 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-02-18T00:27:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-02-18T00:27:04.387 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127128 s, 825 MB/s 2024-02-18T00:27:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-02-18T00:27:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:04.228209+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v96: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:04.472080+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T00:27:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:04.472174+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-18T00:27:05.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:04.228209+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v96: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:04.472080+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T00:27:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:04.472174+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-18T00:27:05.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:04.228209+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v96: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:05.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:04.472080+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T00:27:05.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:04.472174+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-18T00:27:05.815 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:05.476165+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.906466+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.907445+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.908071+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.908644+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.912535+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.913481+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.914179+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:05.918507+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:05.476165+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.906466+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.907445+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.908071+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.908644+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.912535+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.913481+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.914179+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:06.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:05.918507+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:05.476165+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.906466+0000 mon.a (mon.0) 330 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.907445+0000 mon.a (mon.0) 331 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.908071+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.908644+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.912535+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.913481+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.914179+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:05.918507+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:07.612 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:05.902031+0000 mgr.y (mgr.14146) 118 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:27:07.612 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:05.909084+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 5367M 2024-02-18T00:27:07.612 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:06.228785+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v99: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:07.612 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:05.902031+0000 mgr.y (mgr.14146) 118 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:27:07.612 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:05.909084+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 5367M 2024-02-18T00:27:07.612 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:06.228785+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v99: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:07.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:05.902031+0000 mgr.y (mgr.14146) 118 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:27:07.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:05.909084+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 5367M 2024-02-18T00:27:07.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:06.228785+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v99: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:07.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 -- 172.21.15.59:0/2700753607 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40616d0 msgr2=0x7effb40fa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:07.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 --2- 172.21.15.59:0/2700753607 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40616d0 0x7effb40fa670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7effa40047b0 tx=0x7effa4009cb0).stop 2024-02-18T00:27:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 -- 172.21.15.59:0/2700753607 shutdown_connections 2024-02-18T00:27:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 --2- 172.21.15.59:0/2700753607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40fd680 0x7effb40ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 --2- 172.21.15.59:0/2700753607 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40616d0 0x7effb40fa670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 --2- 172.21.15.59:0/2700753607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effb40fabf0 0x7effb40fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 -- 172.21.15.59:0/2700753607 >> 172.21.15.59:0/2700753607 conn(0x7effb40f2190 msgr2=0x7effb40f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:27:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 -- 172.21.15.59:0/2700753607 shutdown_connections 2024-02-18T00:27:07.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.878+0000 7effbbe30700 1 -- 172.21.15.59:0/2700753607 wait complete. 2024-02-18T00:27:07.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 librados: starting msgr at 2024-02-18T00:27:07.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 librados: starting objecter 2024-02-18T00:27:07.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 Processor -- start 2024-02-18T00:27:07.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 -- start start 2024-02-18T00:27:07.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 librados: setting wanted keys 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 librados: calling monclient init 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40616d0 0x7effb40f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40fabf0 0x7effb40f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effb40fd680 0x7effb40f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb40f7080 con 0x7effb40616d0 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb40f71f0 con 0x7effb40fd680 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effbbe30700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb40f73b0 con 0x7effb40fabf0 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effb93cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40fabf0 0x7effb40f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effb9bcc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40616d0 0x7effb40f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effb9bcc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40616d0 0x7effb40f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45718/0 (socket says 172.21.15.59:45718) 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effb9bcc700 1 -- 172.21.15.59:0/2929622662 learned_addr learned my addr 172.21.15.59:0/2929622662 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.879+0000 7effb93cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40fabf0 0x7effb40f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44516/0 (socket says 172.21.15.59:44516) 2024-02-18T00:27:07.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effba3cd700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effb40fd680 0x7effb40f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effb9bcc700 1 -- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40fabf0 msgr2=0x7effb40f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effb9bcc700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40fabf0 0x7effb40f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effb9bcc700 1 -- 172.21.15.59:0/2929622662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effb40fd680 msgr2=0x7effb40f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effb9bcc700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effb40fd680 0x7effb40f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effb9bcc700 1 -- 172.21.15.59:0/2929622662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac0097c0 con 0x7effb40616d0 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effba3cd700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effb40fd680 0x7effb40f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effb93cb700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40fabf0 0x7effb40f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effb9bcc700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40616d0 0x7effb40f7f10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7effa4000e50 tx=0x7effa4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effaaffd700 10 client.?.objecter ms_handle_connect 0x7effb40616d0 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effaaffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effaaffd700 1 -- 172.21.15.59:0/2929622662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4018070 con 0x7effb40616d0 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effaaffd700 1 -- 172.21.15.59:0/2929622662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa401d420 con 0x7effb40616d0 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa4005040 con 0x7effb40616d0 2024-02-18T00:27:07.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effaaffd700 1 -- 172.21.15.59:0/2929622662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4012650 con 0x7effb40616d0 2024-02-18T00:27:07.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effbbe30700 10 client.14292.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:27:07.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb41124d0 con 0x7effb40616d0 2024-02-18T00:27:07.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.880+0000 7effbbe30700 1 librados: init done 2024-02-18T00:27:07.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.881+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb405f360 con 0x7effb40616d0 2024-02-18T00:27:07.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 1 -- 172.21.15.59:0/2929622662 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7effa4012a80 con 0x7effb40616d0 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 1 --2- 172.21.15.59:0/2929622662 >> v2:172.21.15.59:6800/817089002 conn(0x7effa0062ba0 0x7effa0065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 1 -- 172.21.15.59:0/2929622662 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2484+0+0 (secure 0 0 0) 0x7effa407cfc0 con 0x7effb40616d0 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 10 client.14292.objecter ms_dispatch 0x7effb4001550 osd_map(21..21 src has 1..21) v4 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 3 client.14292.objecter handle_osd_map got epochs [21,21] > 0 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 3 client.14292.objecter handle_osd_map decoding full epoch 21 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effb93cb700 1 --2- 172.21.15.59:0/2929622662 >> v2:172.21.15.59:6800/817089002 conn(0x7effa0062ba0 0x7effa0065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 20 client.14292.objecter dump_active .. 0 homeless 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effaaffd700 10 client.14292.objecter ms_handle_connect 0x7effa0062ba0 2024-02-18T00:27:07.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.882+0000 7effb93cb700 1 --2- 172.21.15.59:0/2929622662 >> v2:172.21.15.59:6800/817089002 conn(0x7effa0062ba0 0x7effa0065060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7effb40614a0 tx=0x7effb000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:27:07.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:07.887+0000 7effaaffd700 1 -- 172.21.15.59:0/2929622662 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7effa404e100 con 0x7effb40616d0 2024-02-18T00:27:08.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:08.067+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7effb40f3bc0 con 0x7effa0062ba0 2024-02-18T00:27:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:08.071175+0000 mon.a (mon.0) 338 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:08.073000+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:08.073512+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:08.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:08.071175+0000 mon.a (mon.0) 338 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:08.073000+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:08.073512+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:08.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:08.071175+0000 mon.a (mon.0) 338 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:08.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:08.073000+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:08.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:08.073512+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:09.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:08.069210+0000 mgr.y (mgr.14146) 121 : audit [DBG] from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:09.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:08.229061+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v100: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:09.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:08.069210+0000 mgr.y (mgr.14146) 121 : audit [DBG] from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:09.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:08.229061+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v100: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:09.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:08.069210+0000 mgr.y (mgr.14146) 121 : audit [DBG] from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:09.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:08.229061+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v100: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:10.229321+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:10.637370+0000 mon.c (mon.2) 6 : audit [INF] from='client.? 172.21.15.59:0/636195586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]: dispatch 2024-02-18T00:27:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:10.637658+0000 mon.a (mon.0) 341 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]: dispatch 2024-02-18T00:27:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:10.644703+0000 mon.a (mon.0) 342 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]': finished 2024-02-18T00:27:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:10.644789+0000 mon.a (mon.0) 343 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-18T00:27:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:10.644912+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:11.277416+0000 mon.a (mon.0) 345 : audit [DBG] from='client.? 172.21.15.59:0/3758457202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:27:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:10.229321+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:10.637370+0000 mon.c (mon.2) 6 : audit [INF] from='client.? 172.21.15.59:0/636195586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]: dispatch 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:10.637658+0000 mon.a (mon.0) 341 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]: dispatch 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:10.644703+0000 mon.a (mon.0) 342 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]': finished 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:10.644789+0000 mon.a (mon.0) 343 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:10.644912+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:11.277416+0000 mon.a (mon.0) 345 : audit [DBG] from='client.? 172.21.15.59:0/3758457202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:10.229321+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v101: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:11.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:10.637370+0000 mon.c (mon.2) 6 : audit [INF] from='client.? 172.21.15.59:0/636195586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]: dispatch 2024-02-18T00:27:11.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:10.637658+0000 mon.a (mon.0) 341 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]: dispatch 2024-02-18T00:27:11.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:10.644703+0000 mon.a (mon.0) 342 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b"}]': finished 2024-02-18T00:27:11.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:10.644789+0000 mon.a (mon.0) 343 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-18T00:27:11.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:10.644912+0000 mon.a (mon.0) 344 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:11.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:11.277416+0000 mon.a (mon.0) 345 : audit [DBG] from='client.? 172.21.15.59:0/3758457202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:27:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:12.881+0000 7effbb62f700 10 client.14292.objecter tick 2024-02-18T00:27:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:12.229584+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:12.229584+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:13.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:12.229584+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v103: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:14.230066+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:15.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:14.230066+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:15.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:14.230066+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v104: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:16.230499+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:16.684985+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:16.685097+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:17.329926+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:17.330670+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:16.230499+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:16.684985+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:16.685097+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:17.329926+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T00:27:17.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:17.330670+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:16.230499+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:16.684985+0000 osd.0 (osd.0) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:16.685097+0000 osd.0 (osd.0) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:17.329926+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T00:27:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:17.330670+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:17.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:17.882+0000 7effbb62f700 10 client.14292.objecter tick 2024-02-18T00:27:18.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:17.331260+0000 mgr.y (mgr.14146) 127 : cephadm [INF] Deploying daemon osd.3 on smithi059 2024-02-18T00:27:18.967 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:17.505667+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-18T00:27:18.967 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:17.505808+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:18.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:17.331260+0000 mgr.y (mgr.14146) 127 : cephadm [INF] Deploying daemon osd.3 on smithi059 2024-02-18T00:27:18.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:17.505667+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-18T00:27:18.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:17.505808+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:17.331260+0000 mgr.y (mgr.14146) 127 : cephadm [INF] Deploying daemon osd.3 on smithi059 2024-02-18T00:27:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:17.505667+0000 mon.a (mon.0) 348 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-18T00:27:19.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:17.505808+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:20.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:18.230900+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:18.230900+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:18.230900+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v107: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:22.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:20.231416+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:22.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:20.231416+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:22.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:20.231416+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:22.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:22.883+0000 7effbb62f700 10 client.14292.objecter tick 2024-02-18T00:27:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:22.231920+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:23.031551+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:23.069830+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:23.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:22.231920+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:23.031551+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:23.069830+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:22.231920+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:23.031551+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:23.069830+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:24.232473+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:26.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:24.232473+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:26.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:24.232473+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.758+0000 7effaaffd700 1 -- 172.21.15.59:0/2929622662 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7effb40f3bc0 con 0x7effa0062ba0 2024-02-18T00:27:26.761 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 3 on host 'smithi059' 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.760+0000 7effbbe30700 10 librados: watch_flush enter 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.760+0000 7effbbe30700 10 librados: watch_flush exit 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.760+0000 7effbbe30700 20 client.14292.objecter shutdown clearing up homeless session... 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.760+0000 7effbbe30700 10 client.14292.objecter successfully canceled tick 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.760+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 >> v2:172.21.15.59:6800/817089002 conn(0x7effa0062ba0 msgr2=0x7effa0065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.760+0000 7effbbe30700 1 --2- 172.21.15.59:0/2929622662 >> v2:172.21.15.59:6800/817089002 conn(0x7effa0062ba0 0x7effa0065060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7effb40614a0 tx=0x7effb000b040).stop 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40616d0 msgr2=0x7effb40f7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40616d0 0x7effb40f7f10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7effa4000e50 tx=0x7effa4009cb0).stop 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 shutdown_connections 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 --2- 172.21.15.59:0/2929622662 >> v2:172.21.15.59:6800/817089002 conn(0x7effa0062ba0 0x7effa0065060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7effb40fd680 0x7effb40f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7effb40616d0 0x7effb40f7f10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 --2- 172.21.15.59:0/2929622662 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7effb40fabf0 0x7effb40f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 >> 172.21.15.59:0/2929622662 conn(0x7effb40f2190 msgr2=0x7effb40f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:27:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 shutdown_connections 2024-02-18T00:27:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 -- 172.21.15.59:0/2929622662 wait complete. 2024-02-18T00:27:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:27:26.761+0000 7effbbe30700 1 librados: shutdown 2024-02-18T00:27:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:25.906623+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:25.907409+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:25.908116+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:25.912525+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:26.143505+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T00:27:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:26.232919+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:26.758854+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:26.779914+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:25.906623+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:25.907409+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:25.908116+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:25.912525+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:26.143505+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:26.232919+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:26.758854+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:26.779914+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:25.906623+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:25.907409+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:25.908116+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:25.912525+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:26.143505+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:26.232919+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:26.758854+0000 mon.a (mon.0) 357 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:26.779914+0000 mon.a (mon.0) 358 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:27.556 DEBUG:teuthology.orchestra.run.smithi059:osd.3> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.3.service 2024-02-18T00:27:27.559 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_4... 2024-02-18T00:27:27.559 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-18T00:27:27.590 INFO:journalctl@ceph.osd.3.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:27:28.121 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:26.917445+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T00:27:28.121 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:26.917535+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-18T00:27:28.121 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:26.917671+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:28.121 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:26.918038+0000 mon.a (mon.0) 362 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:27:28.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:26.917445+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T00:27:28.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:26.917535+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-18T00:27:28.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:26.917671+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:28.122 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:26.918038+0000 mon.a (mon.0) 362 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:27:28.122 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:27:27.920+0000 7fe34badc700 -1 osd.3 0 waiting for initial osdmap 2024-02-18T00:27:28.122 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:27:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:27:27.925+0000 7fe348258700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T00:27:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:26.917445+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T00:27:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:26.917535+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-18T00:27:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:26.917671+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:26.918038+0000 mon.a (mon.0) 362 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-02-18T00:27:29.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:27.918182+0000 mon.a (mon.0) 363 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:27:29.155 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:27.918260+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-18T00:27:29.155 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:27.918618+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:29.155 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:27.922841+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:29.155 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:28.233258+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:29.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:27.918182+0000 mon.a (mon.0) 363 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:27.918260+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:27.918618+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:27.922841+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:28.233258+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:27.918182+0000 mon.a (mon.0) 363 : audit [INF] from='osd.3 v2:172.21.15.59:6813/1805700964' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:27.918260+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:27.918618+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:27.922841+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:29.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:28.233258+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T00:27:29.710 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-18T00:27:29.711 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-18T00:27:29.711 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-02-18T00:27:29.711 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-02-18T00:27:29.711 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110387 s, 950 MB/s 2024-02-18T00:27:29.711 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-02-18T00:27:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:27.180313+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:27.180357+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:28.923525+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:28.931016+0000 mon.a (mon.0) 368 : cluster [INF] osd.3 v2:172.21.15.59:6813/1805700964 boot 2024-02-18T00:27:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:28.931064+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-18T00:27:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:28.931838+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:30.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:27.180313+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:30.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:27.180357+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:30.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:28.923525+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:30.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:28.931016+0000 mon.a (mon.0) 368 : cluster [INF] osd.3 v2:172.21.15.59:6813/1805700964 boot 2024-02-18T00:27:30.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:28.931064+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-18T00:27:30.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:28.931838+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:30.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:27.180313+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:30.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:27.180357+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:30.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:28.923525+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:30.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:28.931016+0000 mon.a (mon.0) 368 : cluster [INF] osd.3 v2:172.21.15.59:6813/1805700964 boot 2024-02-18T00:27:30.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:28.931064+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-18T00:27:30.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:28.931838+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:27:30.413 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-02-18T00:27:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:29.930624+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-18T00:27:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:30.233525+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.825180+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.826097+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.826651+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.827135+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.827628+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.831627+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.833164+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.833691+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:30.837463+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:29.930624+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:30.233525+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.825180+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.826097+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.826651+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.827135+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.827628+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.831627+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.833164+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.833691+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:30.837463+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:29.930624+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:30.233525+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.825180+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.826097+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.826651+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.827135+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.827628+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:31.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.831627+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:31.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.833164+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:31.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.833691+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:31.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:30.837463+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:32.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:30.820685+0000 mgr.y (mgr.14146) 135 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:27:32.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:30.828023+0000 mgr.y (mgr.14146) 136 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 4025M 2024-02-18T00:27:32.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:30.820685+0000 mgr.y (mgr.14146) 135 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:27:32.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:30.828023+0000 mgr.y (mgr.14146) 136 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 4025M 2024-02-18T00:27:32.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:30.820685+0000 mgr.y (mgr.14146) 135 : cephadm [INF] Detected new or changed devices on smithi059 2024-02-18T00:27:32.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:30.828023+0000 mgr.y (mgr.14146) 136 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 4025M 2024-02-18T00:27:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 -- 172.21.15.137:0/107225358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f8580 msgr2=0x7fbb880fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/107225358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f8580 0x7fbb880fee80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fbb80009c10 tx=0x7fbb80004d10).stop 2024-02-18T00:27:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 -- 172.21.15.137:0/107225358 shutdown_connections 2024-02-18T00:27:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/107225358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f8580 0x7fbb880fee80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/107225358 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb880f6990 0x7fbb880f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/107225358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb880f7c30 0x7fbb880f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 -- 172.21.15.137:0/107225358 >> 172.21.15.137:0/107225358 conn(0x7fbb880f2190 msgr2=0x7fbb880f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 -- 172.21.15.137:0/107225358 shutdown_connections 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 -- 172.21.15.137:0/107225358 wait complete. 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 librados: starting msgr at 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 librados: starting objecter 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 Processor -- start 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 -- start start 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 librados: setting wanted keys 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.360+0000 7fbb8f35e700 1 librados: calling monclient init 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f6990 0x7fbb880614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb880f7c30 0x7fbb8810cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb880f8580 0x7fbb8810f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:32.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8810f820 con 0x7fbb880f8580 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8810f990 con 0x7fbb880f6990 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8810fb50 con 0x7fbb880f7c30 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f6990 0x7fbb880614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f6990 0x7fbb880614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42334/0 (socket says 172.21.15.137:42334) 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 -- 172.21.15.137:0/3056862368 learned_addr learned my addr 172.21.15.137:0/3056862368 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8c8f9700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb880f7c30 0x7fbb8810cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d8fb700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb880f8580 0x7fbb8810f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 -- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb880f7c30 msgr2=0x7fbb8810cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb880f7c30 0x7fbb8810cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 -- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb880f8580 msgr2=0x7fbb8810f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb880f8580 0x7fbb8810f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 -- 172.21.15.137:0/3056862368 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800097c0 con 0x7fbb880f6990 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8d0fa700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f6990 0x7fbb880614b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fbb84002800 tx=0x7fbb84009ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:27:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb7e7fc700 10 client.?.objecter ms_handle_connect 0x7fbb880f6990 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb7e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb7e7fc700 1 -- 172.21.15.137:0/3056862368 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84007b40 con 0x7fbb880f6990 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb7e7fc700 1 -- 172.21.15.137:0/3056862368 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb8400f040 con 0x7fbb880f6990 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8810fe30 con 0x7fbb880f6990 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb7e7fc700 1 -- 172.21.15.137:0/3056862368 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb840146f0 con 0x7fbb880f6990 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 10 client.24208.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb881102b0 con 0x7fbb880f6990 2024-02-18T00:27:32.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.361+0000 7fbb8f35e700 1 librados: init done 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.362+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8805f360 con 0x7fbb880f6990 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb7e7fc700 1 -- 172.21.15.137:0/3056862368 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7fbb84007ca0 con 0x7fbb880f6990 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb7e7fc700 1 --2- 172.21.15.137:0/3056862368 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb74062ba0 0x7fbb74065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb7e7fc700 1 -- 172.21.15.137:0/3056862368 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 2776+0+0 (secure 0 0 0) 0x7fbb8407bbc0 con 0x7fbb880f6990 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb7e7fc700 10 client.24208.objecter ms_dispatch 0x7fbb88001550 osd_map(27..27 src has 1..27) v4 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb7e7fc700 3 client.24208.objecter handle_osd_map got epochs [27,27] > 0 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb7e7fc700 3 client.24208.objecter handle_osd_map decoding full epoch 27 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb7e7fc700 20 client.24208.objecter dump_active .. 0 homeless 2024-02-18T00:27:32.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.363+0000 7fbb8c8f9700 1 --2- 172.21.15.137:0/3056862368 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb74062ba0 0x7fbb74065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:32.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.364+0000 7fbb7e7fc700 10 client.24208.objecter ms_handle_connect 0x7fbb74062ba0 2024-02-18T00:27:32.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.364+0000 7fbb8c8f9700 1 --2- 172.21.15.137:0/3056862368 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb74062ba0 0x7fbb74065060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fbb780048b0 tx=0x7fbb7800b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:27:32.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.367+0000 7fbb7e7fc700 1 -- 172.21.15.137:0/3056862368 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb8404cb40 con 0x7fbb880f6990 2024-02-18T00:27:32.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:32.535+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb880fdca0 con 0x7fbb74062ba0 2024-02-18T00:27:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:32.233846+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:32.536670+0000 mgr.y (mgr.14146) 138 : audit [DBG] from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:32.539137+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:32.541710+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:32.542483+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:32.233846+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:32.536670+0000 mgr.y (mgr.14146) 138 : audit [DBG] from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:32.539137+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:32.541710+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:32.542483+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:32.233846+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:32.536670+0000 mgr.y (mgr.14146) 138 : audit [DBG] from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:32.539137+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:33.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:32.541710+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:33.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:32.542483+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:34.234285+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:34.935353+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.137:0/523330383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]: dispatch 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:34.935774+0000 mon.a (mon.0) 384 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]: dispatch 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:34.940733+0000 mon.a (mon.0) 385 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]': finished 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:34.940820+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:34.941018+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:34.234285+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:34.935353+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.137:0/523330383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]: dispatch 2024-02-18T00:27:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:34.935774+0000 mon.a (mon.0) 384 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]: dispatch 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:34.940733+0000 mon.a (mon.0) 385 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]': finished 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:34.940820+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:34.941018+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:34.234285+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:34.935353+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.137:0/523330383' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]: dispatch 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:34.935774+0000 mon.a (mon.0) 384 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]: dispatch 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:34.940733+0000 mon.a (mon.0) 385 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231"}]': finished 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:34.940820+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-18T00:27:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:34.941018+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:36.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:35.525899+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.137:0/612143505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:27:36.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:35.525899+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.137:0/612143505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:27:36.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:35.525899+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.137:0/612143505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:27:37.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:37.362+0000 7fbb8eb5d700 10 client.24208.objecter tick 2024-02-18T00:27:37.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:36.234659+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:37.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:36.234659+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:37.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:36.234659+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:39.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:38.235183+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:39.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:38.235183+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:39.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:38.235183+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:40.235702+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:41.230698+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T00:27:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:41.231584+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:40.235702+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:41.230698+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T00:27:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:41.231584+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:41.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:40.235702+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:41.230698+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T00:27:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:41.231584+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:42.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:42.363+0000 7fbb8eb5d700 10 client.24208.objecter tick 2024-02-18T00:27:42.594 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:41.232346+0000 mgr.y (mgr.14146) 143 : cephadm [INF] Deploying daemon osd.4 on smithi137 2024-02-18T00:27:42.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:41.232346+0000 mgr.y (mgr.14146) 143 : cephadm [INF] Deploying daemon osd.4 on smithi137 2024-02-18T00:27:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:41.232346+0000 mgr.y (mgr.14146) 143 : cephadm [INF] Deploying daemon osd.4 on smithi137 2024-02-18T00:27:43.653 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:42.236163+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:42.236163+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:42.236163+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:45.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:44.236467+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:44.236467+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:45.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:44.236467+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:47.364+0000 7fbb8eb5d700 10 client.24208.objecter tick 2024-02-18T00:27:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:46.236814+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:46.498967+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:46.545216+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:46.236814+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:47.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:46.498967+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:47.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:46.545216+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:47.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:46.236814+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:47.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:46.498967+0000 mon.a (mon.0) 390 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:47.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:46.545216+0000 mon.a (mon.0) 391 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:49.399 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2024-02-18T00:27:49.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.397+0000 7fbb7e7fc700 1 -- 172.21.15.137:0/3056862368 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbb880fdca0 con 0x7fbb74062ba0 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 10 librados: watch_flush enter 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 10 librados: watch_flush exit 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 20 client.24208.objecter shutdown clearing up homeless session... 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 10 client.24208.objecter successfully canceled tick 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb74062ba0 msgr2=0x7fbb74065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/3056862368 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb74062ba0 0x7fbb74065060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fbb780048b0 tx=0x7fbb7800b040).stop 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f6990 msgr2=0x7fbb880614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f6990 0x7fbb880614b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fbb84002800 tx=0x7fbb84009ee0).stop 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 shutdown_connections 2024-02-18T00:27:49.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/3056862368 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb74062ba0 0x7fbb74065060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:49.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb880f6990 0x7fbb880614b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:49.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb880f8580 0x7fbb8810f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:49.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 --2- 172.21.15.137:0/3056862368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb880f7c30 0x7fbb8810cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:49.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 >> 172.21.15.137:0/3056862368 conn(0x7fbb880f2190 msgr2=0x7fbb880fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:27:49.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 shutdown_connections 2024-02-18T00:27:49.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 -- 172.21.15.137:0/3056862368 wait complete. 2024-02-18T00:27:49.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:49.399+0000 7fbb8f35e700 1 librados: shutdown 2024-02-18T00:27:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:48.237235+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:49.397179+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:50.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:48.237235+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:50.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:49.397179+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:48.237235+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:49.397179+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:50.502 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.4.service 2024-02-18T00:27:50.504 INFO:tasks.cephadm:Deploying osd.5 on smithi137 with /dev/vg_nvme/lv_3... 2024-02-18T00:27:50.505 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-18T00:27:50.538 INFO:journalctl@ceph.osd.4.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:27:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:49.622405+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v2:172.21.15.137:6800/2044387056' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T00:27:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:49.622623+0000 mon.a (mon.0) 393 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T00:27:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:49.622405+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v2:172.21.15.137:6800/2044387056' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T00:27:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:49.622623+0000 mon.a (mon.0) 393 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T00:27:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:49.622405+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 v2:172.21.15.137:6800/2044387056' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T00:27:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:49.622623+0000 mon.a (mon.0) 393 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T00:27:51.840 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:27:51.519+0000 7fc4f6f94700 -1 osd.4 0 waiting for initial osdmap 2024-02-18T00:27:51.840 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:27:51.523+0000 7fc4f0f0b700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T00:27:51.840 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:50.237591+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:51.840 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:50.508772+0000 mon.a (mon.0) 394 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T00:27:51.840 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:50.508835+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-18T00:27:51.840 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:50.508982+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:51.841 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:50.510468+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v2:172.21.15.137:6800/2044387056' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:27:51.841 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:50.510663+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:27:51.841 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:50.542989+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:52.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:50.237591+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:52.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:50.508772+0000 mon.a (mon.0) 394 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T00:27:52.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:50.508835+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-18T00:27:52.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:50.508982+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:52.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:50.510468+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v2:172.21.15.137:6800/2044387056' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:27:52.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:50.510663+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:27:52.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:50.542989+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:50.237591+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:50.508772+0000 mon.a (mon.0) 394 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T00:27:52.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:50.508835+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-18T00:27:52.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:50.508982+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:52.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:50.510468+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 v2:172.21.15.137:6800/2044387056' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:27:52.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:50.510663+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:27:52.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:50.542989+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:52.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:51.514187+0000 mon.a (mon.0) 399 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:27:52.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:51.514315+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-18T00:27:52.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:51.514941+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:52.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:51.520091+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:52.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:52.514494+0000 mon.a (mon.0) 403 : cluster [INF] osd.4 v2:172.21.15.137:6800/2044387056 boot 2024-02-18T00:27:52.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:52.514552+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-18T00:27:52.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:52.514755+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:51.514187+0000 mon.a (mon.0) 399 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:27:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:51.514315+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-18T00:27:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:51.514941+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:51.520091+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:52.514494+0000 mon.a (mon.0) 403 : cluster [INF] osd.4 v2:172.21.15.137:6800/2044387056 boot 2024-02-18T00:27:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:52.514552+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-18T00:27:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:52.514755+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:51.514187+0000 mon.a (mon.0) 399 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:27:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:51.514315+0000 mon.a (mon.0) 400 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-18T00:27:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:51.514941+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:51.520091+0000 mon.a (mon.0) 402 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:52.514494+0000 mon.a (mon.0) 403 : cluster [INF] osd.4 v2:172.21.15.137:6800/2044387056 boot 2024-02-18T00:27:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:52.514552+0000 mon.a (mon.0) 404 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-18T00:27:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:52.514755+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:27:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:50.604316+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:50.604367+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:52.237926+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v131: 1 pgs: 1 peering; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:53.514288+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-18T00:27:54.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:50.604316+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:54.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:50.604367+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:54.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:52.237926+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v131: 1 pgs: 1 peering; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:54.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:53.514288+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-18T00:27:54.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:50.604316+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:54.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:50.604367+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:54.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:52.237926+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v131: 1 pgs: 1 peering; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T00:27:54.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:53.514288+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-18T00:27:54.025 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-18T00:27:54.025 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-18T00:27:54.025 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-02-18T00:27:54.025 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-02-18T00:27:54.025 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.014517 s, 722 MB/s 2024-02-18T00:27:54.025 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 2024-02-18T00:27:54.025 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-02-18T00:27:54.965 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-02-18T00:27:55.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:54.238228+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v134: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:55.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:54.516400+0000 mon.a (mon.0) 407 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-18T00:27:56.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:54.238228+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v134: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:56.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:54.516400+0000 mon.a (mon.0) 407 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-18T00:27:56.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:54.238228+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v134: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:56.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:54.516400+0000 mon.a (mon.0) 407 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-18T00:27:56.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:55.684915+0000 mgr.y (mgr.14146) 151 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:27:56.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.689289+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:56.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.690325+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:56.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:27:55.690997+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 17122M 2024-02-18T00:27:56.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.695336+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:56.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.696360+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:56.795 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.697203+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.701691+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.704015+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.705524+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.706266+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:55.710525+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:55.960890+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:55.960957+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:56.796 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:56.238701+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v136: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:55.684915+0000 mgr.y (mgr.14146) 151 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:27:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.689289+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.690325+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:27:55.690997+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 17122M 2024-02-18T00:27:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.695336+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.696360+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.697203+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.701691+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.704015+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.705524+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.706266+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:55.710525+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:55.960890+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:55.960957+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:56.238701+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v136: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:55.684915+0000 mgr.y (mgr.14146) 151 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.689289+0000 mon.a (mon.0) 408 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.690325+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:27:55.690997+0000 mgr.y (mgr.14146) 152 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 17122M 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.695336+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.696360+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.697203+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.701691+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.704015+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.705524+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.706266+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:55.710525+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:55.960890+0000 osd.1 (osd.1) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:55.960957+0000 osd.1 (osd.1) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:27:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:56.238701+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v136: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:57.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 -- 172.21.15.137:0/1320661745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb720061670 msgr2=0x7fb7200fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:57.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 --2- 172.21.15.137:0/1320661745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb720061670 0x7fb7200fa6d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fb714004870 tx=0x7fb714009cb0).stop 2024-02-18T00:27:57.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 -- 172.21.15.137:0/1320661745 shutdown_connections 2024-02-18T00:27:57.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 --2- 172.21.15.137:0/1320661745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb720061670 0x7fb7200fa6d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:57.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 --2- 172.21.15.137:0/1320661745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7200fd680 0x7fb7200ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:57.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 --2- 172.21.15.137:0/1320661745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7200fac50 0x7fb7200fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 -- 172.21.15.137:0/1320661745 >> 172.21.15.137:0/1320661745 conn(0x7fb7200f2190 msgr2=0x7fb7200f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 -- 172.21.15.137:0/1320661745 shutdown_connections 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 -- 172.21.15.137:0/1320661745 wait complete. 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 librados: starting msgr at 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.037+0000 7fb7259ac700 1 librados: starting objecter 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 Processor -- start 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 -- start start 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 librados: setting wanted keys 2024-02-18T00:27:57.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 librados: calling monclient init 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb720061670 0x7fb720107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7200fac50 0x7fb720108480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7200fd680 0x7fb72010f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72010fa50 con 0x7fb7200fd680 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72010fb90 con 0x7fb7200fac50 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb7259ac700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72010fcd0 con 0x7fb720061670 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb71e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7200fac50 0x7fb720108480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb71e7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7200fac50 0x7fb720108480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45104/0 (socket says 172.21.15.137:45104) 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb71e7fc700 1 -- 172.21.15.137:0/803266729 learned_addr learned my addr 172.21.15.137:0/803266729 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb71effd700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb720061670 0x7fb720107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:57.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.038+0000 7fb71f7fe700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7200fd680 0x7fb72010f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb71e7fc700 1 -- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb720061670 msgr2=0x7fb720107f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb71e7fc700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb720061670 0x7fb720107f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb71e7fc700 1 -- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7200fd680 msgr2=0x7fb72010f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb71e7fc700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7200fd680 0x7fb72010f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb71e7fc700 1 -- 172.21.15.137:0/803266729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714005040 con 0x7fb7200fac50 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb71e7fc700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7200fac50 0x7fb720108480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fb71000c790 tx=0x7fb710007ec0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb707fff700 10 client.?.objecter ms_handle_connect 0x7fb7200fac50 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb707fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb707fff700 1 -- 172.21.15.137:0/803266729 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000e4a0 con 0x7fb7200fac50 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72010ffb0 con 0x7fb7200fac50 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb707fff700 1 -- 172.21.15.137:0/803266729 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb71000ead0 con 0x7fb7200fac50 2024-02-18T00:27:57.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb707fff700 1 -- 172.21.15.137:0/803266729 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710013aa0 con 0x7fb7200fac50 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb7259ac700 10 client.24235.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7201104e0 con 0x7fb7200fac50 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.039+0000 7fb7259ac700 1 librados: init done 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.040+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72005f360 con 0x7fb7200fac50 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.040+0000 7fb707fff700 1 -- 172.21.15.137:0/803266729 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7fb71000ec40 con 0x7fb7200fac50 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb707fff700 1 --2- 172.21.15.137:0/803266729 >> v2:172.21.15.59:6800/817089002 conn(0x7fb708062ba0 0x7fb708065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb707fff700 1 -- 172.21.15.137:0/803266729 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 3178+0+0 (secure 0 0 0) 0x7fb71007b9c0 con 0x7fb7200fac50 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb707fff700 10 client.24235.objecter ms_dispatch 0x7fb720001550 osd_map(34..34 src has 1..34) v4 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb707fff700 3 client.24235.objecter handle_osd_map got epochs [34,34] > 0 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb707fff700 3 client.24235.objecter handle_osd_map decoding full epoch 34 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb707fff700 20 client.24235.objecter dump_active .. 0 homeless 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb71effd700 1 --2- 172.21.15.137:0/803266729 >> v2:172.21.15.59:6800/817089002 conn(0x7fb708062ba0 0x7fb708065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb707fff700 10 client.24235.objecter ms_handle_connect 0x7fb708062ba0 2024-02-18T00:27:57.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.041+0000 7fb71effd700 1 --2- 172.21.15.137:0/803266729 >> v2:172.21.15.59:6800/817089002 conn(0x7fb708062ba0 0x7fb708065060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fb714004560 tx=0x7fb714009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:27:57.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.045+0000 7fb707fff700 1 -- 172.21.15.137:0/803266729 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb71004c7b0 con 0x7fb7200fac50 2024-02-18T00:27:57.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:27:57.213+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7200f3cb0 con 0x7fb708062ba0 2024-02-18T00:27:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:56.715113+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-18T00:27:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:57.215018+0000 mgr.y (mgr.14146) 154 : audit [DBG] from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:57.217363+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:57.219777+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:57.220579+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:58.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:56.715113+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-18T00:27:58.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:57.215018+0000 mgr.y (mgr.14146) 154 : audit [DBG] from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:58.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:57.217363+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:58.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:57.219777+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:58.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:57.220579+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:58.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:56.715113+0000 mon.a (mon.0) 418 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-18T00:27:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:57.215018+0000 mgr.y (mgr.14146) 154 : audit [DBG] from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:27:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:57.217363+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:27:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:57.219777+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:27:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:57.220579+0000 mon.a (mon.0) 421 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:27:58.929 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:58.239056+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:58.239056+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:27:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:58.239056+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:59.701423+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.137:0/3875568139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]: dispatch 2024-02-18T00:28:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:59.701774+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]: dispatch 2024-02-18T00:28:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:59.708803+0000 mon.a (mon.0) 423 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]': finished 2024-02-18T00:28:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:27:59.708935+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-18T00:28:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:27:59.709131+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:00.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:59.701423+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.137:0/3875568139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]: dispatch 2024-02-18T00:28:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:59.701774+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]: dispatch 2024-02-18T00:28:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:59.708803+0000 mon.a (mon.0) 423 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]': finished 2024-02-18T00:28:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:27:59.708935+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-18T00:28:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:27:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:27:59.709131+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:59.701423+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.137:0/3875568139' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]: dispatch 2024-02-18T00:28:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:59.701774+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]: dispatch 2024-02-18T00:28:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:59.708803+0000 mon.a (mon.0) 423 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01"}]': finished 2024-02-18T00:28:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:27:59.708935+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-18T00:28:00.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:27:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:27:59.709131+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:00.239621+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:00.285219+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.137:0/609657252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:00.239621+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:00.285219+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.137:0/609657252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:01.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:00.239621+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:00.285219+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.137:0/609657252' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:02.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:02.040+0000 7fb7251ab700 10 client.24235.objecter tick 2024-02-18T00:28:03.628 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:02.240020+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:02.240020+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:03.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:02.240020+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:03.405584+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:03.405651+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:04.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:03.405584+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:04.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:03.405651+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:03.405584+0000 osd.2 (osd.2) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:03.405651+0000 osd.2 (osd.2) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:04.240437+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:04.300645+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-18T00:28:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:04.300808+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:05.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:04.240437+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:05.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:04.300645+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-18T00:28:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:04.300808+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:05.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:04.240437+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:04.300645+0000 mon.a (mon.0) 426 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-18T00:28:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:04.300808+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:06.160725+0000 mon.a (mon.0) 428 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T00:28:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:06.161649+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:06.160725+0000 mon.a (mon.0) 428 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T00:28:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:06.161649+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:06.160725+0000 mon.a (mon.0) 428 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T00:28:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:06.161649+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:07.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:07.040+0000 7fb7251ab700 10 client.24235.objecter tick 2024-02-18T00:28:07.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:28:06.162423+0000 mgr.y (mgr.14146) 159 : cephadm [INF] Deploying daemon osd.5 on smithi137 2024-02-18T00:28:07.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:06.240895+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:28:06.162423+0000 mgr.y (mgr.14146) 159 : cephadm [INF] Deploying daemon osd.5 on smithi137 2024-02-18T00:28:07.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:06.240895+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:07.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:28:06.162423+0000 mgr.y (mgr.14146) 159 : cephadm [INF] Deploying daemon osd.5 on smithi137 2024-02-18T00:28:07.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:06.240895+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:09.457 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:08.241255+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:09.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:08.241255+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:09.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:08.241255+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:11.658 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:10.241681+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:11.658 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:11.271470+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:10.241681+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:11.271470+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:10.241681+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:11.271470+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:12.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:12.041+0000 7fb7251ab700 10 client.24235.objecter tick 2024-02-18T00:28:12.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:11.319179+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:12.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:11.319179+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:12.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:11.319179+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:13.626 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:12.242025+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:12.242025+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:12.242025+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:14.593 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 5 on host 'smithi137' 2024-02-18T00:28:14.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.590+0000 7fb707fff700 1 -- 172.21.15.137:0/803266729 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb7200f3cb0 con 0x7fb708062ba0 2024-02-18T00:28:14.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.592+0000 7fb7259ac700 10 librados: watch_flush enter 2024-02-18T00:28:14.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.592+0000 7fb7259ac700 10 librados: watch_flush exit 2024-02-18T00:28:14.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.592+0000 7fb7259ac700 20 client.24235.objecter shutdown clearing up homeless session... 2024-02-18T00:28:14.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.592+0000 7fb7259ac700 10 client.24235.objecter successfully canceled tick 2024-02-18T00:28:14.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.592+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 >> v2:172.21.15.59:6800/817089002 conn(0x7fb708062ba0 msgr2=0x7fb708065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:14.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.592+0000 7fb7259ac700 1 --2- 172.21.15.137:0/803266729 >> v2:172.21.15.59:6800/817089002 conn(0x7fb708062ba0 0x7fb708065060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fb714004560 tx=0x7fb714009cb0).stop 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7200fac50 msgr2=0x7fb720108480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7200fac50 0x7fb720108480 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fb71000c790 tx=0x7fb710007ec0).stop 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 shutdown_connections 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 --2- 172.21.15.137:0/803266729 >> v2:172.21.15.59:6800/817089002 conn(0x7fb708062ba0 0x7fb708065060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7200fd680 0x7fb72010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb720061670 0x7fb720107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 --2- 172.21.15.137:0/803266729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7200fac50 0x7fb720108480 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 >> 172.21.15.137:0/803266729 conn(0x7fb7200f2190 msgr2=0x7fb7200f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 shutdown_connections 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 -- 172.21.15.137:0/803266729 wait complete. 2024-02-18T00:28:14.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:14.593+0000 7fb7259ac700 1 librados: shutdown 2024-02-18T00:28:15.182 DEBUG:teuthology.orchestra.run.smithi137:osd.5> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.5.service 2024-02-18T00:28:15.184 INFO:tasks.cephadm:Deploying osd.6 on smithi137 with /dev/vg_nvme/lv_2... 2024-02-18T00:28:15.184 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-18T00:28:15.209 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.181949+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.182864+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:15.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.183773+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:15.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.191250+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.211 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:14.242438+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:15.211 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.376585+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 v2:172.21.15.137:6804/2170432263' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T00:28:15.211 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.376760+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T00:28:15.211 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.591090+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.211 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:14.608019+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:15.215 INFO:journalctl@ceph.osd.5.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:28:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.181949+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.182864+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.183773+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.191250+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:14.242438+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.376585+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 v2:172.21.15.137:6804/2170432263' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.376760+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.591090+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:14.608019+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.181949+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.182864+0000 mon.a (mon.0) 433 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.183773+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.191250+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:14.242438+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.376585+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 v2:172.21.15.137:6804/2170432263' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.376760+0000 mon.a (mon.0) 436 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T00:28:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.591090+0000 mon.a (mon.0) 437 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:15.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:14.608019+0000 mon.a (mon.0) 438 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:16.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:15.200291+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T00:28:16.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:15.200373+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-18T00:28:16.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:15.200512+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:16.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:15.201384+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 v2:172.21.15.137:6804/2170432263' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:16.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:15.201588+0000 mon.a (mon.0) 442 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:16.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:15.200291+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T00:28:16.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:15.200373+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-18T00:28:16.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:15.200512+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:16.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:15.201384+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 v2:172.21.15.137:6804/2170432263' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:16.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:15.201588+0000 mon.a (mon.0) 442 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:16.504 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:28:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:28:16.206+0000 7fb79528a700 -1 osd.5 0 waiting for initial osdmap 2024-02-18T00:28:16.504 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:28:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:28:16.212+0000 7fb78fa02700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T00:28:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:15.200291+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T00:28:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:15.200373+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-18T00:28:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:15.200512+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:15.201384+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 v2:172.21.15.137:6804/2170432263' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:15.201588+0000 mon.a (mon.0) 442 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:17.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:16.202624+0000 mon.a (mon.0) 443 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:28:17.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:16.202728+0000 mon.a (mon.0) 444 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-18T00:28:17.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:16.203135+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:17.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:16.207895+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:17.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:16.242737+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:17.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:16.202624+0000 mon.a (mon.0) 443 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:28:17.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:16.202728+0000 mon.a (mon.0) 444 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-18T00:28:17.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:16.203135+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:17.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:16.207895+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:17.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:16.242737+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:16.202624+0000 mon.a (mon.0) 443 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:28:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:16.202728+0000 mon.a (mon.0) 444 : cluster [DBG] osdmap e38: 6 total, 5 up, 6 in 2024-02-18T00:28:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:16.203135+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:16.207895+0000 mon.a (mon.0) 446 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:16.242737+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T00:28:18.371 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:15.416100+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:18.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:15.416148+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:18.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:17.209301+0000 mon.a (mon.0) 447 : cluster [INF] osd.5 v2:172.21.15.137:6804/2170432263 boot 2024-02-18T00:28:18.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:17.209340+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-18T00:28:18.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:17.209990+0000 mon.a (mon.0) 449 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:18.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:18.205798+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-18T00:28:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:15.416100+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:15.416148+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:17.209301+0000 mon.a (mon.0) 447 : cluster [INF] osd.5 v2:172.21.15.137:6804/2170432263 boot 2024-02-18T00:28:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:17.209340+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-18T00:28:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:17.209990+0000 mon.a (mon.0) 449 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:18.205798+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-18T00:28:18.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:15.416100+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:18.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:15.416148+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:18.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:17.209301+0000 mon.a (mon.0) 447 : cluster [INF] osd.5 v2:172.21.15.137:6804/2170432263 boot 2024-02-18T00:28:18.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:17.209340+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-18T00:28:18.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:17.209990+0000 mon.a (mon.0) 449 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:28:18.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:18.205798+0000 mon.a (mon.0) 450 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-18T00:28:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-18T00:28:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-18T00:28:18.680 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-02-18T00:28:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-02-18T00:28:18.680 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121857 s, 860 MB/s 2024-02-18T00:28:18.681 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-02-18T00:28:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:18.243147+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:19.207379+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-18T00:28:19.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:18.243147+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:19.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:19.207379+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-18T00:28:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:18.243147+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:19.207379+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-18T00:28:19.557 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-02-18T00:28:21.369 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:28:20.164501+0000 mgr.y (mgr.14146) 167 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:28:21.369 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:20.169468+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.369 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:20.170532+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:21.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:20.171459+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:21.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:28:20.172095+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 8561M 2024-02-18T00:28:21.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:20.176708+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:20.177735+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:21.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:20.178492+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:21.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:20.183442+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:20.243648+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:28:20.164501+0000 mgr.y (mgr.14146) 167 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:28:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:20.169468+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:20.170532+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:20.171459+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:28:20.172095+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 8561M 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:20.176708+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:20.177735+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:20.178492+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:20.183442+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:20.243648+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:28:20.164501+0000 mgr.y (mgr.14146) 167 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:20.169468+0000 mon.a (mon.0) 452 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:20.170532+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:20.171459+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:28:20.172095+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 8561M 2024-02-18T00:28:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:20.176708+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:20.177735+0000 mon.a (mon.0) 456 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:21.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:20.178492+0000 mon.a (mon.0) 457 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:21.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:20.183442+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:21.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:20.243648+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:21.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3519300665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f8580 msgr2=0x7fbb640fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3519300665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f8580 0x7fbb640fee80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbb5c0048a0 tx=0x7fbb5c009cb0).stop 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3519300665 shutdown_connections 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3519300665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f8580 0x7fbb640fee80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3519300665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640f6990 0x7fbb640f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3519300665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f7c30 0x7fbb640f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3519300665 >> 172.21.15.137:0/3519300665 conn(0x7fbb640f2190 msgr2=0x7fbb640f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3519300665 shutdown_connections 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3519300665 wait complete. 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 librados: starting msgr at 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.620+0000 7fbb6a1ae700 1 librados: starting objecter 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 Processor -- start 2024-02-18T00:28:21.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 -- start start 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 librados: setting wanted keys 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 librados: calling monclient init 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640f6990 0x7fbb64061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f7c30 0x7fbb6410cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8580 0x7fbb6410f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6410f950 con 0x7fbb640f6990 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6410fac0 con 0x7fbb640f7c30 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb6a1ae700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6410fc80 con 0x7fbb640f8580 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f7c30 0x7fbb6410cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f7c30 0x7fbb6410cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60674/0 (socket says 172.21.15.137:60674) 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 -- 172.21.15.137:0/3316659353 learned_addr learned my addr 172.21.15.137:0/3316659353 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb63fff700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8580 0x7fbb6410f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb637fe700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640f6990 0x7fbb64061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 -- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8580 msgr2=0x7fbb6410f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8580 0x7fbb6410f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 -- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640f6990 msgr2=0x7fbb64061490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640f6990 0x7fbb64061490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 -- 172.21.15.137:0/3316659353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c005040 con 0x7fbb640f7c30 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb62ffd700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f7c30 0x7fbb6410cd50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbb4c00b530 tx=0x7fbb4c0090f0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb60ff9700 10 client.?.objecter ms_handle_connect 0x7fbb640f7c30 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.621+0000 7fbb60ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.622+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3316659353 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00d4a0 con 0x7fbb640f7c30 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.622+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6410ff60 con 0x7fbb640f7c30 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.622+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3316659353 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4c00dad0 con 0x7fbb640f7c30 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.622+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3316659353 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c012a50 con 0x7fbb640f7c30 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.622+0000 7fbb6a1ae700 10 client.24262.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:28:21.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.622+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb64110370 con 0x7fbb640f7c30 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.622+0000 7fbb6a1ae700 1 librados: init done 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.623+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6405f360 con 0x7fbb640f7c30 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.623+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3316659353 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7fbb4c00d600 con 0x7fbb640f7c30 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.624+0000 7fbb60ff9700 1 --2- 172.21.15.137:0/3316659353 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb50062ba0 0x7fbb50065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.624+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3316659353 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 3470+0+0 (secure 0 0 0) 0x7fbb4c07c5a0 con 0x7fbb640f7c30 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.624+0000 7fbb60ff9700 10 client.24262.objecter ms_dispatch 0x7fbb64001550 osd_map(41..41 src has 1..41) v4 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.624+0000 7fbb60ff9700 3 client.24262.objecter handle_osd_map got epochs [41,41] > 0 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.624+0000 7fbb60ff9700 3 client.24262.objecter handle_osd_map decoding full epoch 41 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.624+0000 7fbb60ff9700 20 client.24262.objecter dump_active .. 0 homeless 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.624+0000 7fbb637fe700 1 --2- 172.21.15.137:0/3316659353 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb50062ba0 0x7fbb50065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.625+0000 7fbb60ff9700 10 client.24262.objecter ms_handle_connect 0x7fbb50062ba0 2024-02-18T00:28:21.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.625+0000 7fbb637fe700 1 --2- 172.21.15.137:0/3316659353 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb50062ba0 0x7fbb50065060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbb54007fd0 tx=0x7fbb54009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:28:21.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.628+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3316659353 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb4c04d230 con 0x7fbb640f7c30 2024-02-18T00:28:21.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:21.797+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb640fdd70 con 0x7fbb50062ba0 2024-02-18T00:28:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:21.801440+0000 mon.a (mon.0) 459 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:28:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:21.803895+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:28:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:21.804773+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:21.801440+0000 mon.a (mon.0) 459 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:28:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:21.803895+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:28:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:21.804773+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:22.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:21.801440+0000 mon.a (mon.0) 459 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:28:22.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:21.803895+0000 mon.a (mon.0) 460 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:28:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:21.804773+0000 mon.a (mon.0) 461 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:23.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:21.799092+0000 mgr.y (mgr.14146) 170 : audit [DBG] from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:28:23.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:22.244203+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:21.799092+0000 mgr.y (mgr.14146) 170 : audit [DBG] from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:28:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:22.244203+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:23.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:21.799092+0000 mgr.y (mgr.14146) 170 : audit [DBG] from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:28:23.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:22.244203+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:24.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:23.304368+0000 osd.5 (osd.5) 3 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:28:24.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:23.305635+0000 osd.5 (osd.5) 4 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:28:24.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:23.304368+0000 osd.5 (osd.5) 3 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:28:24.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:23.305635+0000 osd.5 (osd.5) 4 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:28:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:23.304368+0000 osd.5 (osd.5) 3 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:28:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:23.305635+0000 osd.5 (osd.5) 4 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:28:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:24.244751+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:24.413867+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.137:0/3324030357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]: dispatch 2024-02-18T00:28:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:24.414266+0000 mon.a (mon.0) 462 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]: dispatch 2024-02-18T00:28:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:24.419702+0000 mon.a (mon.0) 463 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]': finished 2024-02-18T00:28:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:24.419780+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:24.419908+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:25.016254+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.137:0/2142433165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:24.244751+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:24.413867+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.137:0/3324030357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]: dispatch 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:24.414266+0000 mon.a (mon.0) 462 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]: dispatch 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:24.419702+0000 mon.a (mon.0) 463 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]': finished 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:24.419780+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:24.419908+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:25.016254+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.137:0/2142433165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:24.244751+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:24.413867+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.137:0/3324030357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]: dispatch 2024-02-18T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:24.414266+0000 mon.a (mon.0) 462 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]: dispatch 2024-02-18T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:24.419702+0000 mon.a (mon.0) 463 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f"}]': finished 2024-02-18T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:24.419780+0000 mon.a (mon.0) 464 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-18T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:24.419908+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:25.016254+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.137:0/2142433165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:26.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:26.623+0000 7fbb699ad700 10 client.24262.objecter tick 2024-02-18T00:28:27.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:26.245182+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:27.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:26.245182+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:27.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:26.245182+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:28.245596+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:28.245596+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:29.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:28.245596+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:29.602420+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:29.602513+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:30.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:29.602420+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:30.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:29.602513+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:29.602420+0000 osd.3 (osd.3) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:29.602513+0000 osd.3 (osd.3) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:31.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:31.624+0000 7fbb699ad700 10 client.24262.objecter tick 2024-02-18T00:28:31.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:30.246194+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:31.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:30.311045+0000 mon.a (mon.0) 466 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-18T00:28:31.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:30.311227+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:31.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:30.534503+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:31.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:30.534596+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:31.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:30.908357+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T00:28:31.693 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:30.909232+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:30.246194+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:30.311045+0000 mon.a (mon.0) 466 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-18T00:28:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:30.311227+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:30.534503+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:30.534596+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:30.908357+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T00:28:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:30.909232+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:30.246194+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:30.311045+0000 mon.a (mon.0) 466 : cluster [DBG] osdmap e43: 7 total, 6 up, 7 in 2024-02-18T00:28:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:30.311227+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:30.534503+0000 osd.0 (osd.0) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:30.534596+0000 osd.0 (osd.0) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:30.908357+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T00:28:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:30.909232+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:32.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:28:30.909979+0000 mgr.y (mgr.14146) 176 : cephadm [INF] Deploying daemon osd.6 on smithi137 2024-02-18T00:28:32.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:31.312265+0000 mon.a (mon.0) 470 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-18T00:28:32.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:31.312897+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:32.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:28:30.909979+0000 mgr.y (mgr.14146) 176 : cephadm [INF] Deploying daemon osd.6 on smithi137 2024-02-18T00:28:32.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:31.312265+0000 mon.a (mon.0) 470 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-18T00:28:32.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:31.312897+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:32.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:28:30.909979+0000 mgr.y (mgr.14146) 176 : cephadm [INF] Deploying daemon osd.6 on smithi137 2024-02-18T00:28:32.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:31.312265+0000 mon.a (mon.0) 470 : cluster [DBG] osdmap e44: 7 total, 6 up, 7 in 2024-02-18T00:28:32.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:31.312897+0000 mon.a (mon.0) 471 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:33.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:32.246603+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:33.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:32.246603+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:32.246603+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:35.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:34.246961+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:34.246961+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:34.246961+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:36.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:36.625+0000 7fbb699ad700 10 client.24262.objecter tick 2024-02-18T00:28:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:35.963608+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:35.992204+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:36.247386+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:35.963608+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:35.992204+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:36.247386+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:37.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:35.963608+0000 mon.a (mon.0) 472 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:37.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:35.992204+0000 mon.a (mon.0) 473 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:36.247386+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:39.465 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 6 on host 'smithi137' 2024-02-18T00:28:39.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.464+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3316659353 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbb640fdd70 con 0x7fbb50062ba0 2024-02-18T00:28:39.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 10 librados: watch_flush enter 2024-02-18T00:28:39.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 10 librados: watch_flush exit 2024-02-18T00:28:39.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 20 client.24262.objecter shutdown clearing up homeless session... 2024-02-18T00:28:39.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 10 client.24262.objecter successfully canceled tick 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb50062ba0 msgr2=0x7fbb50065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3316659353 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb50062ba0 0x7fbb50065060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fbb54007fd0 tx=0x7fbb54009040).stop 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f7c30 msgr2=0x7fbb6410cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f7c30 0x7fbb6410cd50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbb4c00b530 tx=0x7fbb4c0090f0).stop 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 shutdown_connections 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3316659353 >> v2:172.21.15.59:6800/817089002 conn(0x7fbb50062ba0 0x7fbb50065060 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8580 0x7fbb6410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640f6990 0x7fbb64061490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 --2- 172.21.15.137:0/3316659353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f7c30 0x7fbb6410cd50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 >> 172.21.15.137:0/3316659353 conn(0x7fbb640f2190 msgr2=0x7fbb640f4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 shutdown_connections 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 -- 172.21.15.137:0/3316659353 wait complete. 2024-02-18T00:28:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:39.466+0000 7fbb6a1ae700 1 librados: shutdown 2024-02-18T00:28:39.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:38.213106+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:39.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:38.247865+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:39.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:38.503959+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:39.495 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:38.504894+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:39.496 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:38.510284+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:39.496 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.098098+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 v2:172.21.15.137:6808/3356009390' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T00:28:39.496 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.098395+0000 mon.a (mon.0) 478 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:38.213106+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:38.247865+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:38.503959+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:38.504894+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:38.510284+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.098098+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 v2:172.21.15.137:6808/3356009390' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.098395+0000 mon.a (mon.0) 478 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:38.213106+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:39.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:38.247865+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:38.503959+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:38.504894+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:38.510284+0000 mon.a (mon.0) 477 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.098098+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 v2:172.21.15.137:6808/3356009390' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T00:28:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.098395+0000 mon.a (mon.0) 478 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T00:28:40.204 DEBUG:teuthology.orchestra.run.smithi137:osd.6> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.6.service 2024-02-18T00:28:40.206 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_1... 2024-02-18T00:28:40.206 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-18T00:28:40.238 INFO:journalctl@ceph.osd.6.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:28:40.744 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:28:40.518+0000 7fd79f591700 -1 osd.6 0 waiting for initial osdmap 2024-02-18T00:28:40.744 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:28:40.526+0000 7fd79bd0d700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T00:28:40.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.463742+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:40.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.479321+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:40.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.512500+0000 mon.a (mon.0) 481 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T00:28:40.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:39.512572+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-02-18T00:28:40.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.512728+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:40.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.517159+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 v2:172.21.15.137:6808/3356009390' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:40.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:39.517574+0000 mon.a (mon.0) 484 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.463742+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.479321+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.512500+0000 mon.a (mon.0) 481 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:39.512572+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.512728+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.517159+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 v2:172.21.15.137:6808/3356009390' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:39.517574+0000 mon.a (mon.0) 484 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:40.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.463742+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:40.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.479321+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:28:40.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.512500+0000 mon.a (mon.0) 481 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T00:28:40.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:39.512572+0000 mon.a (mon.0) 482 : cluster [DBG] osdmap e45: 7 total, 6 up, 7 in 2024-02-18T00:28:40.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.512728+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:40.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.517159+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 v2:172.21.15.137:6808/3356009390' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:40.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:39.517574+0000 mon.a (mon.0) 484 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:28:41.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:40.248423+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:41.639 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:40.514672+0000 mon.a (mon.0) 485 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:28:41.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:40.514785+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-02-18T00:28:41.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:40.515249+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:41.640 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:40.519817+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:40.248423+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:40.514672+0000 mon.a (mon.0) 485 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:28:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:40.514785+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-02-18T00:28:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:40.515249+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:40.519817+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:40.248423+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:40.514672+0000 mon.a (mon.0) 485 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:28:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:40.514785+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e46: 7 total, 6 up, 7 in 2024-02-18T00:28:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:40.515249+0000 mon.a (mon.0) 487 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:40.519817+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:42.951 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:40.131011+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:42.951 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:40.131124+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:42.951 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:41.521682+0000 mon.a (mon.0) 489 : cluster [INF] osd.6 v2:172.21.15.137:6808/3356009390 boot 2024-02-18T00:28:42.951 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:41.521763+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-18T00:28:42.951 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:41.522644+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:40.131011+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:40.131124+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:41.521682+0000 mon.a (mon.0) 489 : cluster [INF] osd.6 v2:172.21.15.137:6808/3356009390 boot 2024-02-18T00:28:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:41.521763+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-18T00:28:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:41.522644+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:40.131011+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:28:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:40.131124+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:28:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:41.521682+0000 mon.a (mon.0) 489 : cluster [INF] osd.6 v2:172.21.15.137:6808/3356009390 boot 2024-02-18T00:28:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:41.521763+0000 mon.a (mon.0) 490 : cluster [DBG] osdmap e47: 7 total, 7 up, 7 in 2024-02-18T00:28:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:41.522644+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:28:43.368 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-18T00:28:43.369 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-18T00:28:43.369 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-02-18T00:28:43.369 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-02-18T00:28:43.369 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124111 s, 845 MB/s 2024-02-18T00:28:43.369 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-02-18T00:28:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:42.248737+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:42.530939+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-02-18T00:28:44.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:42.248737+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:44.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:42.530939+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-02-18T00:28:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:42.248737+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T00:28:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:42.530939+0000 mon.a (mon.0) 492 : cluster [DBG] osdmap e48: 7 total, 7 up, 7 in 2024-02-18T00:28:44.062 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-02-18T00:28:44.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:43.531771+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-02-18T00:28:45.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:43.531771+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-02-18T00:28:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:43.531771+0000 mon.a (mon.0) 493 : cluster [DBG] osdmap e49: 7 total, 7 up, 7 in 2024-02-18T00:28:45.729 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:44.249104+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:44.249104+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:44.249104+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/4234541083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f8580 msgr2=0x7f1c380fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/4234541083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f8580 0x7f1c380fee80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f1c30009c10 tx=0x7f1c30004d10).stop 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/4234541083 shutdown_connections 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/4234541083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f8580 0x7f1c380fee80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/4234541083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c380f6990 0x7f1c380f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/4234541083 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c380f7c30 0x7f1c380f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/4234541083 >> 172.21.15.137:0/4234541083 conn(0x7f1c380f2190 msgr2=0x7f1c380f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/4234541083 shutdown_connections 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/4234541083 wait complete. 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.242+0000 7f1c3f8fa700 1 librados: starting msgr at 2024-02-18T00:28:46.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 librados: starting objecter 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 Processor -- start 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 -- start start 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 librados: setting wanted keys 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 librados: calling monclient init 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c380f6990 0x7f1c38107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f7c30 0x7f1c38108210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c380f8580 0x7f1c3810f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3810f7e0 con 0x7f1c380f6990 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3810f920 con 0x7f1c380f7c30 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3f8fa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3810fa60 con 0x7f1c380f8580 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f7c30 0x7f1c38108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f7c30 0x7f1c38108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57520/0 (socket says 172.21.15.137:57520) 2024-02-18T00:28:46.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 -- 172.21.15.137:0/2842145997 learned_addr learned my addr 172.21.15.137:0/2842145997 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3de97700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c380f8580 0x7f1c3810f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3d696700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c380f6990 0x7f1c38107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 -- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c380f8580 msgr2=0x7f1c3810f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c380f8580 0x7f1c3810f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 -- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c380f6990 msgr2=0x7f1c38107d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c380f6990 0x7f1c38107d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 -- 172.21.15.137:0/2842145997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300097c0 con 0x7f1c380f7c30 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c3ce95700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f7c30 0x7f1c38108210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1c340041d0 tx=0x7f1c34009f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c2e7fc700 10 client.?.objecter ms_handle_connect 0x7f1c380f7c30 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.243+0000 7f1c2e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.244+0000 7f1c2e7fc700 1 -- 172.21.15.137:0/2842145997 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34007b40 con 0x7f1c380f7c30 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.244+0000 7f1c2e7fc700 1 -- 172.21.15.137:0/2842145997 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c3400f040 con 0x7f1c380f7c30 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.244+0000 7f1c2e7fc700 1 -- 172.21.15.137:0/2842145997 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34014860 con 0x7f1c380f7c30 2024-02-18T00:28:46.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.244+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3810fd40 con 0x7f1c380f7c30 2024-02-18T00:28:46.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.244+0000 7f1c3f8fa700 10 client.24289.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:28:46.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.244+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c38110250 con 0x7f1c380f7c30 2024-02-18T00:28:46.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.244+0000 7f1c3f8fa700 1 librados: init done 2024-02-18T00:28:46.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3805f360 con 0x7f1c380f7c30 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c2e7fc700 1 -- 172.21.15.137:0/2842145997 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7f1c34014c50 con 0x7f1c380f7c30 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c2e7fc700 1 --2- 172.21.15.137:0/2842145997 >> v2:172.21.15.59:6800/817089002 conn(0x7f1c24062ba0 0x7f1c24065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c2e7fc700 1 -- 172.21.15.137:0/2842145997 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3762+0+0 (secure 0 0 0) 0x7f1c3407ba20 con 0x7f1c380f7c30 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c2e7fc700 10 client.24289.objecter ms_dispatch 0x7f1c38001550 osd_map(49..49 src has 1..49) v4 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c2e7fc700 3 client.24289.objecter handle_osd_map got epochs [49,49] > 0 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c2e7fc700 3 client.24289.objecter handle_osd_map decoding full epoch 49 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c2e7fc700 20 client.24289.objecter dump_active .. 0 homeless 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.245+0000 7f1c3d696700 1 --2- 172.21.15.137:0/2842145997 >> v2:172.21.15.59:6800/817089002 conn(0x7f1c24062ba0 0x7f1c24065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.246+0000 7f1c2e7fc700 10 client.24289.objecter ms_handle_connect 0x7f1c24062ba0 2024-02-18T00:28:46.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.246+0000 7f1c3d696700 1 --2- 172.21.15.137:0/2842145997 >> v2:172.21.15.59:6800/817089002 conn(0x7f1c24062ba0 0x7f1c24065060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f1c380f7a90 tx=0x7f1c2800b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:28:46.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.250+0000 7f1c2e7fc700 1 -- 172.21.15.137:0/2842145997 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1c3404c660 con 0x7f1c380f7c30 2024-02-18T00:28:46.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:46.426+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c380610f0 con 0x7f1c24062ba0 2024-02-18T00:28:47.649 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:46.249382+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:28:46.322400+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.327324+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.328319+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.329242+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.330077+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:28:46.330726+0000 mgr.y (mgr.14146) 186 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 5707M 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.337238+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.338245+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.339112+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:47.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.346320+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.651 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.428026+0000 mgr.y (mgr.14146) 187 : audit [DBG] from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:28:47.651 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.430286+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:28:47.651 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.432534+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:28:47.651 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:46.433355+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:46.249382+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:28:46.322400+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:28:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.327324+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.328319+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.329242+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.330077+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:28:46.330726+0000 mgr.y (mgr.14146) 186 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 5707M 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.337238+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.338245+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.339112+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.346320+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.428026+0000 mgr.y (mgr.14146) 187 : audit [DBG] from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.430286+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.432534+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:46.433355+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:46.249382+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:28:46.322400+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:28:47.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.327324+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.328319+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.329242+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.330077+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:28:46.330726+0000 mgr.y (mgr.14146) 186 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 5707M 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.337238+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.338245+0000 mon.a (mon.0) 499 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.339112+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.346320+0000 mon.a (mon.0) 501 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.428026+0000 mgr.y (mgr.14146) 187 : audit [DBG] from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.430286+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.432534+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T00:28:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:46.433355+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:48.249840+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:48.969098+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.137:0/721918331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]: dispatch 2024-02-18T00:28:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:48.969475+0000 mon.a (mon.0) 505 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]: dispatch 2024-02-18T00:28:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:48.976575+0000 mon.a (mon.0) 506 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]': finished 2024-02-18T00:28:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:48.976660+0000 mon.a (mon.0) 507 : cluster [DBG] osdmap e50: 8 total, 7 up, 8 in 2024-02-18T00:28:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:48.976837+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:48.249840+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:48.969098+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.137:0/721918331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]: dispatch 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:48.969475+0000 mon.a (mon.0) 505 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]: dispatch 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:48.976575+0000 mon.a (mon.0) 506 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]': finished 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:48.976660+0000 mon.a (mon.0) 507 : cluster [DBG] osdmap e50: 8 total, 7 up, 8 in 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:48.976837+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:48.249840+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:48.969098+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.137:0/721918331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]: dispatch 2024-02-18T00:28:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:48.969475+0000 mon.a (mon.0) 505 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]: dispatch 2024-02-18T00:28:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:48.976575+0000 mon.a (mon.0) 506 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb"}]': finished 2024-02-18T00:28:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:48.976660+0000 mon.a (mon.0) 507 : cluster [DBG] osdmap e50: 8 total, 7 up, 8 in 2024-02-18T00:28:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:48.976837+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:28:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:49.573480+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.137:0/206550293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:49.573480+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.137:0/206550293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:50.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:49.573480+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.137:0/206550293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T00:28:51.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:51.245+0000 7f1c3f0f9700 10 client.24289.objecter tick 2024-02-18T00:28:51.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:50.250318+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:51.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:50.250318+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:51.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:50.250318+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:52.250798+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:52.250798+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:53.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:52.250798+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:55.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:54.251282+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:55.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:55.286992+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T00:28:55.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:28:55.287864+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:54.251282+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:55.286992+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T00:28:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:28:55.287864+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:55.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:54.251282+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:55.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:55.286992+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T00:28:55.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:28:55.287864+0000 mon.a (mon.0) 510 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:28:56.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:28:56.246+0000 7f1c3f0f9700 10 client.24289.objecter tick 2024-02-18T00:28:56.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:28:55.288579+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Deploying daemon osd.7 on smithi137 2024-02-18T00:28:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:28:55.288579+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Deploying daemon osd.7 on smithi137 2024-02-18T00:28:56.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:28:55.288579+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Deploying daemon osd.7 on smithi137 2024-02-18T00:28:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:56.251581+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:57.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:56.251581+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:56.251581+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:28:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:28:58.252078+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:59.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:28:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:28:58.252078+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:28:59.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:28:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:28:58.252078+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:01.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:01.246+0000 7f1c3f0f9700 10 client.24289.objecter tick 2024-02-18T00:29:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:00.252534+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:00.491792+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:00.497904+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:01.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:00.252534+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:01.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:00.491792+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:01.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:00.497904+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:00.252534+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:00.491792+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:00.497904+0000 mon.a (mon.0) 512 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:02.252882+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:03.362720+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:03.363643+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:03.364621+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:03.369510+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:03.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:02.252882+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:03.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:03.362720+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:03.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:03.363643+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:03.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:03.364621+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:03.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:03.369510+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:03.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:02.252882+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:03.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:03.362720+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:03.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:03.363643+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:03.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:03.364621+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:03.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:03.369510+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.235+0000 7f1c2e7fc700 1 -- 172.21.15.137:0/2842145997 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1c380610f0 con 0x7f1c24062ba0 2024-02-18T00:29:04.238 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2024-02-18T00:29:04.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 10 librados: watch_flush enter 2024-02-18T00:29:04.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 10 librados: watch_flush exit 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 20 client.24289.objecter shutdown clearing up homeless session... 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 10 client.24289.objecter successfully canceled tick 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 >> v2:172.21.15.59:6800/817089002 conn(0x7f1c24062ba0 msgr2=0x7f1c24065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/2842145997 >> v2:172.21.15.59:6800/817089002 conn(0x7f1c24062ba0 0x7f1c24065060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f1c380f7a90 tx=0x7f1c2800b040).stop 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f7c30 msgr2=0x7f1c38108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.237+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f7c30 0x7f1c38108210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1c340041d0 tx=0x7f1c34009f60).stop 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 shutdown_connections 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/2842145997 >> v2:172.21.15.59:6800/817089002 conn(0x7f1c24062ba0 0x7f1c24065060 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1c380f8580 0x7f1c3810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c380f6990 0x7f1c38107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 --2- 172.21.15.137:0/2842145997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c380f7c30 0x7f1c38108210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 >> 172.21.15.137:0/2842145997 conn(0x7f1c380f2190 msgr2=0x7f1c380609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 shutdown_connections 2024-02-18T00:29:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 -- 172.21.15.137:0/2842145997 wait complete. 2024-02-18T00:29:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:04.238+0000 7f1c3f8fa700 1 librados: shutdown 2024-02-18T00:29:04.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:03.606793+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 v2:172.21.15.137:6812/53025171' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T00:29:04.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:03.607093+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T00:29:04.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:04.236008+0000 mon.a (mon.0) 518 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:04.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:04.237752+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:03.606793+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 v2:172.21.15.137:6812/53025171' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T00:29:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:03.607093+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T00:29:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:04.236008+0000 mon.a (mon.0) 518 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:04.237752+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:03.606793+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 v2:172.21.15.137:6812/53025171' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T00:29:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:03.607093+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T00:29:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:04.236008+0000 mon.a (mon.0) 518 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:04.237752+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:04.935 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.7.service 2024-02-18T00:29:04.938 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-18T00:29:04.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-18T00:29:04.971 INFO:journalctl@ceph.osd.7.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:29:05.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:04.253385+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:04.505575+0000 mon.a (mon.0) 520 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:04.505673+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:04.505836+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:04.506398+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 v2:172.21.15.137:6812/53025171' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:04.506592+0000 mon.a (mon.0) 523 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:04.253385+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:04.505575+0000 mon.a (mon.0) 520 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:04.505673+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-18T00:29:05.808 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:04.505836+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:05.809 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:04.506398+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 v2:172.21.15.137:6812/53025171' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:29:05.809 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:04.506592+0000 mon.a (mon.0) 523 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:29:05.903 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:29:05.511+0000 7ff5aec4c700 -1 osd.7 0 waiting for initial osdmap 2024-02-18T00:29:05.903 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:29:05.519+0000 7ff5ab3c8700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T00:29:05.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:04.253385+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:05.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:04.505575+0000 mon.a (mon.0) 520 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T00:29:05.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:04.505673+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e51: 8 total, 7 up, 8 in 2024-02-18T00:29:05.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:04.505836+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:05.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:04.506398+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 v2:172.21.15.137:6812/53025171' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:29:05.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:04.506592+0000 mon.a (mon.0) 523 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-02-18T00:29:06.720 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:05.508350+0000 mon.a (mon.0) 524 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:29:06.720 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:05.508445+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-18T00:29:06.720 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:05.508782+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:06.721 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:05.512147+0000 mon.a (mon.0) 527 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:06.721 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:05.508350+0000 mon.a (mon.0) 524 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:29:06.721 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:05.508445+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-18T00:29:06.721 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:05.508782+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:06.721 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:05.512147+0000 mon.a (mon.0) 527 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:05.508350+0000 mon.a (mon.0) 524 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-02-18T00:29:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:05.508445+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e52: 8 total, 7 up, 8 in 2024-02-18T00:29:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:05.508782+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:06.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:05.512147+0000 mon.a (mon.0) 527 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:07.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 -- 172.21.15.59:0/2825955286 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 msgr2=0x7f03400f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 --2- 172.21.15.59:0/2825955286 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8090 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f03340047e0 tx=0x7f0334009cb0).stop 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 -- 172.21.15.59:0/2825955286 shutdown_connections 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 --2- 172.21.15.59:0/2825955286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03400f6990 0x7f03400f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 --2- 172.21.15.59:0/2825955286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03400f8580 0x7f03400fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 --2- 172.21.15.59:0/2825955286 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8090 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 -- 172.21.15.59:0/2825955286 >> 172.21.15.59:0/2825955286 conn(0x7f03400f2190 msgr2=0x7f03400f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 -- 172.21.15.59:0/2825955286 shutdown_connections 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.466+0000 7f0344baf700 1 -- 172.21.15.59:0/2825955286 wait complete. 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 librados: starting msgr at 2024-02-18T00:29:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 librados: starting objecter 2024-02-18T00:29:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 Processor -- start 2024-02-18T00:29:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 -- start start 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 librados: setting wanted keys 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 librados: calling monclient init 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03400f6990 0x7f03400fa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03400f8580 0x7f03400f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400f9a80 con 0x7f03400f8580 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400f9bf0 con 0x7f03400f6990 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.467+0000 7f0344baf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03400f9db0 con 0x7f03400f7c30 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:46234/0 (socket says 172.21.15.59:46234) 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 -- 172.21.15.59:0/161337727 learned_addr learned my addr 172.21.15.59:0/161337727 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:29:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033ed9d700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03400f8580 0x7f03400f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033e59c700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03400f6990 0x7f03400fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 -- 172.21.15.59:0/161337727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03400f6990 msgr2=0x7f03400fa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03400f6990 0x7f03400fa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 -- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03400f8580 msgr2=0x7f03400f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03400f8580 0x7f03400f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 -- 172.21.15.59:0/161337727 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380097c0 con 0x7f03400f7c30 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033ed9d700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03400f8580 0x7f03400f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f033dd9b700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8f70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0334007e50 tx=0x7f033400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f032f7fe700 10 client.?.objecter ms_handle_connect 0x7f03400f7c30 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f032f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f032f7fe700 1 -- 172.21.15.59:0/161337727 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334019070 con 0x7f03400f7c30 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f032f7fe700 1 -- 172.21.15.59:0/161337727 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0334013ed0 con 0x7f03400f7c30 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0334005040 con 0x7f03400f7c30 2024-02-18T00:29:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f032f7fe700 1 -- 172.21.15.59:0/161337727 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334014c30 con 0x7f03400f7c30 2024-02-18T00:29:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f0344baf700 10 client.24284.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03401125b0 con 0x7f03400f7c30 2024-02-18T00:29:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.468+0000 7f0344baf700 1 librados: init done 2024-02-18T00:29:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.470+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034005f360 con 0x7f03400f7c30 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.470+0000 7f032f7fe700 1 -- 172.21.15.59:0/161337727 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7f0334013a00 con 0x7f03400f7c30 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f032f7fe700 1 --2- 172.21.15.59:0/161337727 >> v2:172.21.15.59:6800/817089002 conn(0x7f0328062ba0 0x7f0328065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f032f7fe700 1 -- 172.21.15.59:0/161337727 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4087+0+0 (secure 0 0 0) 0x7f033407d210 con 0x7f03400f7c30 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f032f7fe700 10 client.24284.objecter ms_dispatch 0x7f0340001550 osd_map(53..53 src has 1..53) v4 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f032f7fe700 3 client.24284.objecter handle_osd_map got epochs [53,53] > 0 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f032f7fe700 3 client.24284.objecter handle_osd_map decoding full epoch 53 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f033e59c700 1 --2- 172.21.15.59:0/161337727 >> v2:172.21.15.59:6800/817089002 conn(0x7f0328062ba0 0x7f0328065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f032f7fe700 20 client.24284.objecter dump_active .. 0 homeless 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f032f7fe700 10 client.24284.objecter ms_handle_connect 0x7f0328062ba0 2024-02-18T00:29:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.471+0000 7f033e59c700 1 --2- 172.21.15.59:0/161337727 >> v2:172.21.15.59:6800/817089002 conn(0x7f0328062ba0 0x7f0328065060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f03400f7a90 tx=0x7f033000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.475+0000 7f032f7fe700 1 -- 172.21.15.59:0/161337727 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f033404dd10 con 0x7f03400f7c30 2024-02-18T00:29:07.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.647+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0340101860 con 0x7f03400f7c30 2024-02-18T00:29:07.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.647+0000 7f032f7fe700 1 -- 172.21.15.59:0/161337727 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v54) v1 ==== 74+0+130 (secure 0 0 0) 0x7f0334023030 con 0x7f03400f7c30 2024-02-18T00:29:07.650 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:29:07.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 10 librados: watch_flush enter 2024-02-18T00:29:07.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 10 librados: watch_flush exit 2024-02-18T00:29:07.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 20 client.24284.objecter shutdown clearing up homeless session... 2024-02-18T00:29:07.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 10 client.24284.objecter successfully canceled tick 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 >> v2:172.21.15.59:6800/817089002 conn(0x7f0328062ba0 msgr2=0x7f0328065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 --2- 172.21.15.59:0/161337727 >> v2:172.21.15.59:6800/817089002 conn(0x7f0328062ba0 0x7f0328065060 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f03400f7a90 tx=0x7f033000b040).stop 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 msgr2=0x7f03400f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8f70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0334007e50 tx=0x7f033400b040).stop 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 shutdown_connections 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 --2- 172.21.15.59:0/161337727 >> v2:172.21.15.59:6800/817089002 conn(0x7f0328062ba0 0x7f0328065060 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03400f6990 0x7f03400fa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03400f8580 0x7f03400f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 --2- 172.21.15.59:0/161337727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03400f7c30 0x7f03400f8f70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 >> 172.21.15.59:0/161337727 conn(0x7f03400f2190 msgr2=0x7f0340061200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 shutdown_connections 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 -- 172.21.15.59:0/161337727 wait complete. 2024-02-18T00:29:07.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:07.649+0000 7f0344baf700 1 librados: shutdown 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:04.632654+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:04.632755+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:06.167967+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:06.168040+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:06.253659+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:06.509008+0000 mon.a (mon.0) 528 : cluster [INF] osd.7 v2:172.21.15.137:6812/53025171 boot 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:06.509134+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:06.509314+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:07.800 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:07.508708+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-18T00:29:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:04.632654+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:04.632755+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:06.167967+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:06.168040+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:06.253659+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:06.509008+0000 mon.a (mon.0) 528 : cluster [INF] osd.7 v2:172.21.15.137:6812/53025171 boot 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:06.509134+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:06.509314+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:07.508708+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:04.632654+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:04.632755+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:06.167967+0000 osd.2 (osd.2) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:06.168040+0000 osd.2 (osd.2) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:06.253659+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:06.509008+0000 mon.a (mon.0) 528 : cluster [INF] osd.7 v2:172.21.15.137:6812/53025171 boot 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:06.509134+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-18T00:29:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:06.509314+0000 mon.a (mon.0) 530 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:08.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:07.508708+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-18T00:29:08.398 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1708216146,"num_in_osds":8,"osd_in_since":1708216128,"num_remapped_pgs":0} 2024-02-18T00:29:08.398 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi059 2024-02-18T00:29:08.399 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi059=foo.a' 2024-02-18T00:29:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:07.648908+0000 mon.c (mon.2) 7 : audit [DBG] from='client.? 172.21.15.59:0/161337727' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T00:29:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.035457+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.036277+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.036861+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.037405+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.037960+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.041787+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.042551+0000 mon.a (mon.0) 538 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.043119+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:08.046793+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:08.509936+0000 mon.a (mon.0) 541 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:07.648908+0000 mon.c (mon.2) 7 : audit [DBG] from='client.? 172.21.15.59:0/161337727' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.035457+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.036277+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.036861+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.037405+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.037960+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.041787+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.042551+0000 mon.a (mon.0) 538 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.043119+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:08.046793+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:08.509936+0000 mon.a (mon.0) 541 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:07.648908+0000 mon.c (mon.2) 7 : audit [DBG] from='client.? 172.21.15.59:0/161337727' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T00:29:08.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.035457+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.036277+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.036861+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.037405+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.037960+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.041787+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.042551+0000 mon.a (mon.0) 538 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.043119+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:08.046793+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:08.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:08.509936+0000 mon.a (mon.0) 541 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-18T00:29:09.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:08.030796+0000 mgr.y (mgr.14146) 199 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:29:09.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:08.038397+0000 mgr.y (mgr.14146) 200 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 4280M 2024-02-18T00:29:09.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:08.253936+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:08.030796+0000 mgr.y (mgr.14146) 199 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:29:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:08.038397+0000 mgr.y (mgr.14146) 200 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 4280M 2024-02-18T00:29:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:08.253936+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:08.030796+0000 mgr.y (mgr.14146) 199 : cephadm [INF] Detected new or changed devices on smithi137 2024-02-18T00:29:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:08.038397+0000 mgr.y (mgr.14146) 200 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 4280M 2024-02-18T00:29:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:08.253936+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 -- 172.21.15.137:0/1057024111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f8d20 msgr2=0x7f3b100f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 --2- 172.21.15.137:0/1057024111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f8d20 0x7f3b100f9180 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3b0c0048a0 tx=0x7f3b0c009cb0).stop 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 -- 172.21.15.137:0/1057024111 shutdown_connections 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 --2- 172.21.15.137:0/1057024111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b100f9670 0x7f3b100fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 --2- 172.21.15.137:0/1057024111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b100f7a80 0x7f3b100f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 --2- 172.21.15.137:0/1057024111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f8d20 0x7f3b100f9180 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 -- 172.21.15.137:0/1057024111 >> 172.21.15.137:0/1057024111 conn(0x7f3b100f32a0 msgr2=0x7f3b100f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 -- 172.21.15.137:0/1057024111 shutdown_connections 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.369+0000 7f3b17d21700 1 -- 172.21.15.137:0/1057024111 wait complete. 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 librados: starting msgr at 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 librados: starting objecter 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 Processor -- start 2024-02-18T00:29:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 -- start start 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 librados: setting wanted keys 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 librados: calling monclient init 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b100f7a80 0x7f3b10107f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b100f8d20 0x7f3b10108440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f9670 0x7f3b1010f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1010fa10 con 0x7f3b100f8d20 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1010fb50 con 0x7f3b100f9670 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b17d21700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1010fc90 con 0x7f3b100f7a80 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f9670 0x7f3b1010f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f9670 0x7f3b1010f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55654/0 (socket says 172.21.15.137:55654) 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 -- 172.21.15.137:0/3015110744 learned_addr learned my addr 172.21.15.137:0/3015110744 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b15abd700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b100f7a80 0x7f3b10107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b152bc700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b100f8d20 0x7f3b10108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 -- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b100f7a80 msgr2=0x7f3b10107f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b100f7a80 0x7f3b10107f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 -- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b100f8d20 msgr2=0x7f3b10108440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b100f8d20 0x7f3b10108440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.370+0000 7f3b162be700 1 -- 172.21.15.137:0/3015110744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c005040 con 0x7f3b100f9670 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b162be700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f9670 0x7f3b1010f520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3b0800d970 tx=0x7f3b08004e60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b06ffd700 10 client.?.objecter ms_handle_connect 0x7f3b100f9670 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b06ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b06ffd700 1 -- 172.21.15.137:0/3015110744 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08004020 con 0x7f3b100f9670 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b06ffd700 1 -- 172.21.15.137:0/3015110744 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b0800f040 con 0x7f3b100f9670 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1010ff70 con 0x7f3b100f9670 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b06ffd700 1 -- 172.21.15.137:0/3015110744 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080036c0 con 0x7f3b100f9670 2024-02-18T00:29:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b17d21700 10 client.24322.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b10110450 con 0x7f3b100f9670 2024-02-18T00:29:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.371+0000 7f3b17d21700 1 librados: init done 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.372+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b10102880 con 0x7f3b100f9670 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.372+0000 7f3b06ffd700 1 -- 172.21.15.137:0/3015110744 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7f3b08003af0 con 0x7f3b100f9670 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.372+0000 7f3b06ffd700 1 --2- 172.21.15.137:0/3015110744 >> v2:172.21.15.59:6800/817089002 conn(0x7f3afc062ba0 0x7f3afc065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b06ffd700 1 -- 172.21.15.137:0/3015110744 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4054+0+0 (secure 0 0 0) 0x7f3b0807baf0 con 0x7f3b100f9670 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b06ffd700 10 client.24322.objecter ms_dispatch 0x7f3b10001550 osd_map(55..55 src has 1..55) v4 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b06ffd700 3 client.24322.objecter handle_osd_map got epochs [55,55] > 0 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b06ffd700 3 client.24322.objecter handle_osd_map decoding full epoch 55 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b06ffd700 20 client.24322.objecter dump_active .. 0 homeless 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b15abd700 1 --2- 172.21.15.137:0/3015110744 >> v2:172.21.15.59:6800/817089002 conn(0x7f3afc062ba0 0x7f3afc065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b06ffd700 10 client.24322.objecter ms_handle_connect 0x7f3afc062ba0 2024-02-18T00:29:10.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.373+0000 7f3b15abd700 1 --2- 172.21.15.137:0/3015110744 >> v2:172.21.15.59:6800/817089002 conn(0x7f3afc062ba0 0x7f3afc065060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f3b100f8b80 tx=0x7f3b0000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:10.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.377+0000 7f3b06ffd700 1 -- 172.21.15.137:0/3015110744 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3b08048f60 con 0x7f3b100f9670 2024-02-18T00:29:10.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.548+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi059=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b100feea0 con 0x7f3afc062ba0 2024-02-18T00:29:10.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.555+0000 7f3b06ffd700 1 -- 172.21.15.137:0/3015110744 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f3b100feea0 con 0x7f3afc062ba0 2024-02-18T00:29:10.557 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled rgw.foo.a update... 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 10 librados: watch_flush enter 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 10 librados: watch_flush exit 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 20 client.24322.objecter shutdown clearing up homeless session... 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 10 client.24322.objecter successfully canceled tick 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 >> v2:172.21.15.59:6800/817089002 conn(0x7f3afc062ba0 msgr2=0x7f3afc065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 --2- 172.21.15.137:0/3015110744 >> v2:172.21.15.59:6800/817089002 conn(0x7f3afc062ba0 0x7f3afc065060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f3b100f8b80 tx=0x7f3b0000b040).stop 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f9670 msgr2=0x7f3b1010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f9670 0x7f3b1010f520 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f3b0800d970 tx=0x7f3b08004e60).stop 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 shutdown_connections 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 --2- 172.21.15.137:0/3015110744 >> v2:172.21.15.59:6800/817089002 conn(0x7f3afc062ba0 0x7f3afc065060 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b100f9670 0x7f3b1010f520 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b100f7a80 0x7f3b10107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 --2- 172.21.15.137:0/3015110744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b100f8d20 0x7f3b10108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:10.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 >> 172.21.15.137:0/3015110744 conn(0x7f3b100f32a0 msgr2=0x7f3b100f66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:10.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 shutdown_connections 2024-02-18T00:29:10.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 -- 172.21.15.137:0/3015110744 wait complete. 2024-02-18T00:29:10.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:10.557+0000 7f3b17d21700 1 librados: shutdown 2024-02-18T00:29:11.218 DEBUG:teuthology.orchestra.run.smithi059:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@rgw.foo.a.service 2024-02-18T00:29:11.221 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi137 2024-02-18T00:29:11.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool create iscsi 3 3 replicated 2024-02-18T00:29:11.255 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:29:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:10.254289+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.551318+0000 mgr.y (mgr.14146) 203 : audit [DBG] from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi059=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:10.552849+0000 mgr.y (mgr.14146) 204 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi059=foo.a;count:1 2024-02-18T00:29:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.556492+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.568851+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.570403+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.571163+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.575813+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:10.577891+0000 mgr.y (mgr.14146) 205 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi059=foo.a;count:1 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.581070+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.581758+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.586316+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.592950+0000 mon.a (mon.0) 550 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:10.593813+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:10.594522+0000 mgr.y (mgr.14146) 206 : cephadm [INF] Deploying daemon rgw.foo.a on smithi059 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:10.254289+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.551318+0000 mgr.y (mgr.14146) 203 : audit [DBG] from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi059=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:10.552849+0000 mgr.y (mgr.14146) 204 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi059=foo.a;count:1 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.556492+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.568851+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.570403+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.571163+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.575813+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:10.577891+0000 mgr.y (mgr.14146) 205 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi059=foo.a;count:1 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.581070+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.581758+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.586316+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.592950+0000 mon.a (mon.0) 550 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:10.593813+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:10.594522+0000 mgr.y (mgr.14146) 206 : cephadm [INF] Deploying daemon rgw.foo.a on smithi059 2024-02-18T00:29:11.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:10.254289+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:11.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.551318+0000 mgr.y (mgr.14146) 203 : audit [DBG] from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi059=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:11.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:10.552849+0000 mgr.y (mgr.14146) 204 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi059=foo.a;count:1 2024-02-18T00:29:11.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.556492+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.568851+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:11.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.570403+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.571163+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.575813+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:10.577891+0000 mgr.y (mgr.14146) 205 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi059=foo.a;count:1 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.581070+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.581758+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.586316+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.592950+0000 mon.a (mon.0) 550 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:10.593813+0000 mon.a (mon.0) 551 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:11.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:10.594522+0000 mgr.y (mgr.14146) 206 : cephadm [INF] Deploying daemon rgw.foo.a on smithi059 2024-02-18T00:29:13.002 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:29:12 smithi059 systemd[1]: Starting Ceph rgw.foo.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:29:13.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 -- 172.21.15.137:0/2574739540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f8d20 msgr2=0x7fa4600f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:13.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2574739540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f8d20 0x7fa4600f9180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fa44c0048a0 tx=0x7fa44c009cb0).stop 2024-02-18T00:29:13.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 -- 172.21.15.137:0/2574739540 shutdown_connections 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2574739540 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4600f9670 0x7fa4600fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2574739540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4600f7a80 0x7fa4600f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2574739540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f8d20 0x7fa4600f9180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 -- 172.21.15.137:0/2574739540 >> 172.21.15.137:0/2574739540 conn(0x7fa4600f32a0 msgr2=0x7fa4600f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 -- 172.21.15.137:0/2574739540 shutdown_connections 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.290+0000 7fa466f5d700 1 -- 172.21.15.137:0/2574739540 wait complete. 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.291+0000 7fa466f5d700 1 librados: starting msgr at 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.291+0000 7fa466f5d700 1 librados: starting objecter 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.291+0000 7fa466f5d700 1 Processor -- start 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.291+0000 7fa466f5d700 1 -- start start 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.291+0000 7fa466f5d700 1 librados: setting wanted keys 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.291+0000 7fa466f5d700 1 librados: calling monclient init 2024-02-18T00:29:13.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa466f5d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4600f7a80 0x7fa460107f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa466f5d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4600f8d20 0x7fa460108440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa466f5d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f9670 0x7fa46010f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa466f5d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46010fa10 con 0x7fa4600f7a80 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa466f5d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46010fb50 con 0x7fa4600f9670 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa466f5d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46010fc90 con 0x7fa4600f8d20 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f9670 0x7fa46010f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f9670 0x7fa46010f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55678/0 (socket says 172.21.15.137:55678) 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 -- 172.21.15.137:0/2840987822 learned_addr learned my addr 172.21.15.137:0/2840987822 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:13.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa45ffff700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4600f8d20 0x7fa460108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa464cf9700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4600f7a80 0x7fa460107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 -- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4600f8d20 msgr2=0x7fa460108440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4600f8d20 0x7fa460108440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 -- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4600f7a80 msgr2=0x7fa460107f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4600f7a80 0x7fa460107f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 -- 172.21.15.137:0/2840987822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c005040 con 0x7fa4600f9670 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa464cf9700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4600f7a80 0x7fa460107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa4654fa700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f9670 0x7fa46010f520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa45800ec20 tx=0x7fa45800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa45dffb700 10 client.?.objecter ms_handle_connect 0x7fa4600f9670 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa45dffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa45dffb700 1 -- 172.21.15.137:0/2840987822 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458009950 con 0x7fa4600f9670 2024-02-18T00:29:13.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa45dffb700 1 -- 172.21.15.137:0/2840987822 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45800f040 con 0x7fa4600f9670 2024-02-18T00:29:13.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46010ff10 con 0x7fa4600f9670 2024-02-18T00:29:13.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.292+0000 7fa45dffb700 1 -- 172.21.15.137:0/2840987822 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800c6e0 con 0x7fa4600f9670 2024-02-18T00:29:13.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.293+0000 7fa466f5d700 10 client.24337.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:13.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.293+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa460110440 con 0x7fa4600f9670 2024-02-18T00:29:13.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.293+0000 7fa466f5d700 1 librados: init done 2024-02-18T00:29:13.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.293+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46005f360 con 0x7fa4600f9670 2024-02-18T00:29:13.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 1 -- 172.21.15.137:0/2840987822 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7fa4580092e0 con 0x7fa4600f9670 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 1 --2- 172.21.15.137:0/2840987822 >> v2:172.21.15.59:6800/817089002 conn(0x7fa450062ba0 0x7fa450065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 1 -- 172.21.15.137:0/2840987822 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 4054+0+0 (secure 0 0 0) 0x7fa45807bd90 con 0x7fa4600f9670 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 10 client.24337.objecter ms_dispatch 0x7fa460001550 osd_map(55..55 src has 1..55) v4 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 3 client.24337.objecter handle_osd_map got epochs [55,55] > 0 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 3 client.24337.objecter handle_osd_map decoding full epoch 55 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa464cf9700 1 --2- 172.21.15.137:0/2840987822 >> v2:172.21.15.59:6800/817089002 conn(0x7fa450062ba0 0x7fa450065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 20 client.24337.objecter dump_active .. 0 homeless 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa45dffb700 10 client.24337.objecter ms_handle_connect 0x7fa450062ba0 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.294+0000 7fa464cf9700 1 --2- 172.21.15.137:0/2840987822 >> v2:172.21.15.59:6800/817089002 conn(0x7fa450062ba0 0x7fa450065060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa454007fd0 tx=0x7fa454009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:13.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.298+0000 7fa45dffb700 1 -- 172.21.15.137:0/2840987822 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa458049f60 con 0x7fa4600f9670 2024-02-18T00:29:13.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:13.460+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fa4600fee60 con 0x7fa4600f9670 2024-02-18T00:29:13.752 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:29:13 smithi059 bash[64789]: ff9a5e69722d4e4112e798c18d8dba60d7ae4294e141754fbf10366023d66295 2024-02-18T00:29:13.752 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:29:13 smithi059 systemd[1]: Started Ceph rgw.foo.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:29:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:12.254610+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:14.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:12.254610+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:12.254610+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:14.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.703+0000 7fa45dffb700 1 -- 172.21.15.137:0/2840987822 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'iscsi' created v57) v1 ==== 154+0+0 (secure 0 0 0) 0x7fa45804c990 con 0x7fa4600f9670 2024-02-18T00:29:14.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.708+0000 7fa466f5d700 10 librados: watch_flush enter 2024-02-18T00:29:14.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.712+0000 7fa466f5d700 10 librados: watch_flush exit 2024-02-18T00:29:14.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.712+0000 7fa466f5d700 20 client.24337.objecter shutdown clearing up homeless session... 2024-02-18T00:29:14.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.712+0000 7fa466f5d700 10 client.24337.objecter successfully canceled tick 2024-02-18T00:29:14.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.713+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 >> v2:172.21.15.59:6800/817089002 conn(0x7fa450062ba0 msgr2=0x7fa450065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:14.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.713+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2840987822 >> v2:172.21.15.59:6800/817089002 conn(0x7fa450062ba0 0x7fa450065060 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa454007fd0 tx=0x7fa454009040).stop 2024-02-18T00:29:14.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.713+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f9670 msgr2=0x7fa46010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:14.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.713+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f9670 0x7fa46010f520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa45800ec20 tx=0x7fa45800b040).stop 2024-02-18T00:29:14.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.717+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 shutdown_connections 2024-02-18T00:29:14.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.717+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2840987822 >> v2:172.21.15.59:6800/817089002 conn(0x7fa450062ba0 0x7fa450065060 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:14.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.717+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4600f9670 0x7fa46010f520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:14.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.717+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4600f7a80 0x7fa460107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:14.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.717+0000 7fa466f5d700 1 --2- 172.21.15.137:0/2840987822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4600f8d20 0x7fa460108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:14.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.717+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 >> 172.21.15.137:0/2840987822 conn(0x7fa4600f32a0 msgr2=0x7fa4600f6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:14.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.719+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 shutdown_connections 2024-02-18T00:29:14.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.719+0000 7fa466f5d700 1 -- 172.21.15.137:0/2840987822 wait complete. 2024-02-18T00:29:14.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:14.719+0000 7fa466f5d700 1 librados: shutdown 2024-02-18T00:29:14.722 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'iscsi' created 2024-02-18T00:29:15.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:12.724338+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:15.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:12.724405+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:15.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:12.724338+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:15.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:12.724405+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:15.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:13.462645+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.137:0/2840987822' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T00:29:15.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:13.699124+0000 mon.a (mon.0) 552 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T00:29:15.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:14.133035+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-18T00:29:15.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:12.724338+0000 osd.4 (osd.4) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:12.724405+0000 osd.4 (osd.4) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:13.462645+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.137:0/2840987822' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:13.699124+0000 mon.a (mon.0) 552 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:14.133035+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:14.134649+0000 mon.a (mon.0) 554 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:14.138696+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:14.139838+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:14.254925+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v198: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:15.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:14.704212+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-18T00:29:15.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:14.704438+0000 mon.a (mon.0) 558 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-18T00:29:15.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:14.704544+0000 mon.a (mon.0) 559 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-18T00:29:15.417 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool application enable iscsi rbd 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:14.134649+0000 mon.a (mon.0) 554 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:14.138696+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:14.139838+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:14.254925+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v198: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:14.704212+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:14.704438+0000 mon.a (mon.0) 558 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:14.704544+0000 mon.a (mon.0) 559 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:13.462645+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.137:0/2840987822' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T00:29:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:13.699124+0000 mon.a (mon.0) 552 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:14.133035+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:14.134649+0000 mon.a (mon.0) 554 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:14.138696+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:14.139838+0000 mon.a (mon.0) 556 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:14.254925+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v198: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:14.704212+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:14.704438+0000 mon.a (mon.0) 558 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-18T00:29:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:14.704544+0000 mon.a (mon.0) 559 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-18T00:29:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:15.388250+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:15.505435+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:15.506187+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:15.506854+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:15.511012+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-18T00:29:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:16.051362+0000 osd.1 (osd.1) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:16.051403+0000 osd.1 (osd.1) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:16.071372+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:16.074911+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.59:0/1565964696' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:16.075646+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:16.077593+0000 mon.a (mon.0) 566 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:15.388250+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:15.505435+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:15.506187+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:15.506854+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:15.511012+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:16.051362+0000 osd.1 (osd.1) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:16.051403+0000 osd.1 (osd.1) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:16.071372+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:16.074911+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.59:0/1565964696' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:16.075646+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:16.077593+0000 mon.a (mon.0) 566 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:15.388250+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:15.505435+0000 mon.a (mon.0) 561 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:15.506187+0000 mon.a (mon.0) 562 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:15.506854+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:16.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:15.511012+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-18T00:29:16.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:16.051362+0000 osd.1 (osd.1) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:16.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:16.051403+0000 osd.1 (osd.1) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:16.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:16.071372+0000 mon.a (mon.0) 564 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-18T00:29:16.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:16.074911+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.59:0/1565964696' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:16.075646+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:16.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:16.077593+0000 mon.a (mon.0) 566 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T00:29:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 -- 172.21.15.137:0/1655657371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f7c30 msgr2=0x7fca300f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 --2- 172.21.15.137:0/1655657371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f7c30 0x7fca300f8090 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fca200048a0 tx=0x7fca20009cb0).stop 2024-02-18T00:29:17.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 -- 172.21.15.137:0/1655657371 shutdown_connections 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 --2- 172.21.15.137:0/1655657371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f8580 0x7fca300fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 --2- 172.21.15.137:0/1655657371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca300f6990 0x7fca300f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 --2- 172.21.15.137:0/1655657371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f7c30 0x7fca300f8090 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 -- 172.21.15.137:0/1655657371 >> 172.21.15.137:0/1655657371 conn(0x7fca300f2190 msgr2=0x7fca300f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 -- 172.21.15.137:0/1655657371 shutdown_connections 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 -- 172.21.15.137:0/1655657371 wait complete. 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 librados: starting msgr at 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 librados: starting objecter 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.325+0000 7fca35d0b700 1 Processor -- start 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 -- start start 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 librados: setting wanted keys 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 librados: calling monclient init 2024-02-18T00:29:17.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca300f6990 0x7fca300fa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f7c30 0x7fca300f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f8580 0x7fca300f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300f9a80 con 0x7fca300f6990 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300f9bf0 con 0x7fca300f8580 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca35d0b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca300f9db0 con 0x7fca300f7c30 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f8580 0x7fca300f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f7c30 0x7fca300f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f8580 0x7fca300f9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49984/0 (socket says 172.21.15.137:49984) 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca300f6990 0x7fca300fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f7c30 0x7fca300f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.137:37152/0 (socket says 172.21.15.137:37152) 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2ffff700 1 -- 172.21.15.137:0/971009461 learned_addr learned my addr 172.21.15.137:0/971009461 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 -- 172.21.15.137:0/971009461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f8580 msgr2=0x7fca300f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:17.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f8580 0x7fca300f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 -- 172.21.15.137:0/971009461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca300f6990 msgr2=0x7fca300fa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca300f6990 0x7fca300fa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 -- 172.21.15.137:0/971009461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20005040 con 0x7fca300f7c30 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2ffff700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f8580 0x7fca300f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2effd700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f7c30 0x7fca300f8f70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fca200134f0 tx=0x7fca20009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2cff9700 10 client.?.objecter ms_handle_connect 0x7fca300f7c30 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2cff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2cff9700 1 -- 172.21.15.137:0/971009461 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000e030 con 0x7fca300f7c30 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.326+0000 7fca2cff9700 1 -- 172.21.15.137:0/971009461 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca20004100 con 0x7fca300f7c30 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.327+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca301122c0 con 0x7fca300f7c30 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.327+0000 7fca2cff9700 1 -- 172.21.15.137:0/971009461 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20018070 con 0x7fca300f7c30 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.327+0000 7fca35d0b700 10 client.24335.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:17.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.327+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca301126d0 con 0x7fca300f7c30 2024-02-18T00:29:17.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.327+0000 7fca35d0b700 1 librados: init done 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.327+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3005f360 con 0x7fca300f7c30 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2cff9700 1 -- 172.21.15.137:0/971009461 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7fca2001d420 con 0x7fca300f7c30 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2cff9700 1 --2- 172.21.15.137:0/971009461 >> v2:172.21.15.59:6800/817089002 conn(0x7fca18062ba0 0x7fca18065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2cff9700 1 -- 172.21.15.137:0/971009461 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5122+0+0 (secure 0 0 0) 0x7fca2007dd20 con 0x7fca300f7c30 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2cff9700 10 client.24335.objecter ms_dispatch 0x7fca30001550 osd_map(59..59 src has 1..59) v4 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2cff9700 3 client.24335.objecter handle_osd_map got epochs [59,59] > 0 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2cff9700 3 client.24335.objecter handle_osd_map decoding full epoch 59 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2cff9700 20 client.24335.objecter dump_active .. 0 homeless 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.328+0000 7fca2f7fe700 1 --2- 172.21.15.137:0/971009461 >> v2:172.21.15.59:6800/817089002 conn(0x7fca18062ba0 0x7fca18065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.329+0000 7fca2cff9700 10 client.24335.objecter ms_handle_connect 0x7fca18062ba0 2024-02-18T00:29:17.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.329+0000 7fca2f7fe700 1 --2- 172.21.15.137:0/971009461 >> v2:172.21.15.59:6800/817089002 conn(0x7fca18062ba0 0x7fca18065060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fca300f7a90 tx=0x7fca2400b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:17.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.333+0000 7fca2cff9700 1 -- 172.21.15.137:0/971009461 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fca2004e370 con 0x7fca300f7c30 2024-02-18T00:29:17.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:17.494+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"} v 0) v1 -- 0x7fca301128f0 con 0x7fca300f7c30 2024-02-18T00:29:18.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.080+0000 7fca2cff9700 1 -- 172.21.15.137:0/971009461 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]=0 enabled application 'rbd' on pool 'iscsi' v60) v1 ==== 147+0+0 (secure 0 0 0) 0x7fca20051990 con 0x7fca300f7c30 2024-02-18T00:29:18.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.083+0000 7fca35d0b700 10 librados: watch_flush enter 2024-02-18T00:29:18.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 10 librados: watch_flush exit 2024-02-18T00:29:18.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 20 client.24335.objecter shutdown clearing up homeless session... 2024-02-18T00:29:18.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 10 client.24335.objecter successfully canceled tick 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 >> v2:172.21.15.59:6800/817089002 conn(0x7fca18062ba0 msgr2=0x7fca18065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 --2- 172.21.15.137:0/971009461 >> v2:172.21.15.59:6800/817089002 conn(0x7fca18062ba0 0x7fca18065060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fca300f7a90 tx=0x7fca2400b040).stop 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f7c30 msgr2=0x7fca300f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f7c30 0x7fca300f8f70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fca200134f0 tx=0x7fca20009cb0).stop 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 shutdown_connections 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 --2- 172.21.15.137:0/971009461 >> v2:172.21.15.59:6800/817089002 conn(0x7fca18062ba0 0x7fca18065060 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca300f8580 0x7fca300f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca300f6990 0x7fca300fa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 --2- 172.21.15.137:0/971009461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca300f7c30 0x7fca300f8f70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 >> 172.21.15.137:0/971009461 conn(0x7fca300f2190 msgr2=0x7fca300f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 shutdown_connections 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 -- 172.21.15.137:0/971009461 wait complete. 2024-02-18T00:29:18.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:18.085+0000 7fca35d0b700 1 librados: shutdown 2024-02-18T00:29:18.088 INFO:teuthology.orchestra.run.smithi137.stderr:enabled application 'rbd' on pool 'iscsi' 2024-02-18T00:29:18.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:16.255517+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v201: 68 pgs: 2 creating+peering, 6 active+clean, 60 unknown; 358 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:18.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:17.077069+0000 mon.a (mon.0) 567 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T00:29:18.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:17.077251+0000 mon.a (mon.0) 568 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T00:29:18.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:17.077405+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-18T00:29:18.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:16.255517+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v201: 68 pgs: 2 creating+peering, 6 active+clean, 60 unknown; 358 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:18.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:17.077069+0000 mon.a (mon.0) 567 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T00:29:18.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:17.077251+0000 mon.a (mon.0) 568 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T00:29:18.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:17.077405+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-18T00:29:18.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:16.255517+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v201: 68 pgs: 2 creating+peering, 6 active+clean, 60 unknown; 358 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:17.077069+0000 mon.a (mon.0) 567 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T00:29:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:17.077251+0000 mon.a (mon.0) 568 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T00:29:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:17.077405+0000 mon.a (mon.0) 569 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-18T00:29:18.830 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply iscsi iscsi user password --placement '1;smithi137=iscsi.a' 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:17.495673+0000 mon.c (mon.2) 9 : audit [INF] from='client.? 172.21.15.137:0/971009461' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:17.639318+0000 osd.5 (osd.5) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:17.639365+0000 osd.5 (osd.5) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:17.963630+0000 mon.a (mon.0) 570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:18.078698+0000 mon.a (mon.0) 571 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:18.078754+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:18.082725+0000 mon.c (mon.2) 10 : audit [INF] from='client.? 172.21.15.59:0/1565964696' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:18.083239+0000 mon.a (mon.0) 573 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:18.083499+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:18.255893+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v204: 100 pgs: 4 creating+activating, 7 creating+peering, 40 active+clean, 49 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:17.495673+0000 mon.c (mon.2) 9 : audit [INF] from='client.? 172.21.15.137:0/971009461' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:17.639318+0000 osd.5 (osd.5) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:17.639365+0000 osd.5 (osd.5) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:17.963630+0000 mon.a (mon.0) 570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:18.078698+0000 mon.a (mon.0) 571 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:18.078754+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:18.082725+0000 mon.c (mon.2) 10 : audit [INF] from='client.? 172.21.15.59:0/1565964696' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:18.083239+0000 mon.a (mon.0) 573 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:18.083499+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:18.255893+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v204: 100 pgs: 4 creating+activating, 7 creating+peering, 40 active+clean, 49 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:17.495673+0000 mon.c (mon.2) 9 : audit [INF] from='client.? 172.21.15.137:0/971009461' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:17.639318+0000 osd.5 (osd.5) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:17.639365+0000 osd.5 (osd.5) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:17.963630+0000 mon.a (mon.0) 570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:18.078698+0000 mon.a (mon.0) 571 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:18.078754+0000 mon.a (mon.0) 572 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:18.082725+0000 mon.c (mon.2) 10 : audit [INF] from='client.? 172.21.15.59:0/1565964696' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:18.083239+0000 mon.a (mon.0) 573 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:18.083499+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T00:29:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:18.255893+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v204: 100 pgs: 4 creating+activating, 7 creating+peering, 40 active+clean, 49 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-02-18T00:29:20.238 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:19.085382+0000 mon.a (mon.0) 575 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T00:29:20.238 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:19.085483+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T00:29:20.238 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:19.085584+0000 mon.a (mon.0) 577 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-18T00:29:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:19.085382+0000 mon.a (mon.0) 575 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T00:29:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:19.085483+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T00:29:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:19.085584+0000 mon.a (mon.0) 577 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-18T00:29:20.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:19.085382+0000 mon.a (mon.0) 575 : audit [INF] from='client.? 172.21.15.59:0/2016795673' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T00:29:20.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:19.085483+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T00:29:20.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:19.085584+0000 mon.a (mon.0) 577 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-18T00:29:20.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 -- 172.21.15.137:0/1110948470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f8590 msgr2=0x7f36200fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:20.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 --2- 172.21.15.137:0/1110948470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f8590 0x7f36200fee90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3618009c10 tx=0x7f3618004d10).stop 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 -- 172.21.15.137:0/1110948470 shutdown_connections 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 --2- 172.21.15.137:0/1110948470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f8590 0x7f36200fee90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 --2- 172.21.15.137:0/1110948470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f69a0 0x7f36200f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 --2- 172.21.15.137:0/1110948470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f7c40 0x7f36200f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 -- 172.21.15.137:0/1110948470 >> 172.21.15.137:0/1110948470 conn(0x7f36200f21a0 msgr2=0x7f36200f45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 -- 172.21.15.137:0/1110948470 shutdown_connections 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 -- 172.21.15.137:0/1110948470 wait complete. 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 librados: starting msgr at 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 librados: starting objecter 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 Processor -- start 2024-02-18T00:29:20.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 -- start start 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 librados: setting wanted keys 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.754+0000 7f3628775700 1 librados: calling monclient init 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f69a0 0x7f36200614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7c40 0x7f362010ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f8590 0x7f3620111410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3620111940 con 0x7f36200f69a0 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3620111a80 con 0x7f36200f7c40 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3620111c40 con 0x7f36200f8590 2024-02-18T00:29:20.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7c40 0x7f362010ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:20.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7c40 0x7f362010ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50012/0 (socket says 172.21.15.137:50012) 2024-02-18T00:29:20.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 -- 172.21.15.137:0/3649758658 learned_addr learned my addr 172.21.15.137:0/3649758658 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:20.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3626511700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f69a0 0x7f36200614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:20.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3626d12700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f8590 0x7f3620111410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:20.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 -- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f8590 msgr2=0x7f3620111410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:20.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f8590 0x7f3620111410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 -- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f69a0 msgr2=0x7f36200614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f69a0 0x7f36200614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 -- 172.21.15.137:0/3649758658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180097c0 con 0x7f36200f7c40 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3626511700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f69a0 0x7f36200614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3626d12700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f8590 0x7f3620111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3625d10700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7c40 0x7f362010ef10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f361c00c810 tx=0x7f361c004330).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f36177fe700 10 client.?.objecter ms_handle_connect 0x7f36200f7c40 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f36177fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:20.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f36177fe700 1 -- 172.21.15.137:0/3649758658 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c0043a0 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f36177fe700 1 -- 172.21.15.137:0/3649758658 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f361c0109e0 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3620111f20 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f36177fe700 1 -- 172.21.15.137:0/3649758658 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c012ad0 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 10 client.24385.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3620112400 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.755+0000 7f3628775700 1 librados: init done 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.756+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362005f360 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f36177fe700 1 -- 172.21.15.137:0/3649758658 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7f361c010b50 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f36177fe700 1 --2- 172.21.15.137:0/3649758658 >> v2:172.21.15.59:6800/817089002 conn(0x7f360c062ba0 0x7f360c065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f36177fe700 1 -- 172.21.15.137:0/3649758658 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5857+0+0 (secure 0 0 0) 0x7f361c07c7e0 con 0x7f36200f7c40 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f36177fe700 10 client.24385.objecter ms_dispatch 0x7f3620001550 osd_map(62..62 src has 1..62) v4 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f36177fe700 3 client.24385.objecter handle_osd_map got epochs [62,62] > 0 2024-02-18T00:29:20.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f36177fe700 3 client.24385.objecter handle_osd_map decoding full epoch 62 2024-02-18T00:29:20.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f36177fe700 20 client.24385.objecter dump_active .. 0 homeless 2024-02-18T00:29:20.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.757+0000 7f3626511700 1 --2- 172.21.15.137:0/3649758658 >> v2:172.21.15.59:6800/817089002 conn(0x7f360c062ba0 0x7f360c065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:20.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.758+0000 7f36177fe700 10 client.24385.objecter ms_handle_connect 0x7f360c062ba0 2024-02-18T00:29:20.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.758+0000 7f3626511700 1 --2- 172.21.15.137:0/3649758658 >> v2:172.21.15.59:6800/817089002 conn(0x7f360c062ba0 0x7f360c065060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3610007fd0 tx=0x7f3610009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:20.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.761+0000 7f36177fe700 1 -- 172.21.15.137:0/3649758658 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f361c04cb50 con 0x7f36200f7c40 2024-02-18T00:29:20.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.932+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi137=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f3620059120 con 0x7f360c062ba0 2024-02-18T00:29:20.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.940+0000 7f36177fe700 1 -- 172.21.15.137:0/3649758658 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3610017030 con 0x7f360c062ba0 2024-02-18T00:29:20.941 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled iscsi.iscsi update... 2024-02-18T00:29:20.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 10 librados: watch_flush enter 2024-02-18T00:29:20.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 10 librados: watch_flush exit 2024-02-18T00:29:20.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 20 client.24385.objecter shutdown clearing up homeless session... 2024-02-18T00:29:20.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 10 client.24385.objecter successfully canceled tick 2024-02-18T00:29:20.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 >> v2:172.21.15.59:6800/817089002 conn(0x7f360c062ba0 msgr2=0x7f360c065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 --2- 172.21.15.137:0/3649758658 >> v2:172.21.15.59:6800/817089002 conn(0x7f360c062ba0 0x7f360c065060 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f3610007fd0 tx=0x7f3610009040).stop 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7c40 msgr2=0x7f362010ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7c40 0x7f362010ef10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f361c00c810 tx=0x7f361c004330).stop 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 shutdown_connections 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 --2- 172.21.15.137:0/3649758658 >> v2:172.21.15.59:6800/817089002 conn(0x7f360c062ba0 0x7f360c065060 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36200f8590 0x7f3620111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36200f69a0 0x7f36200614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 --2- 172.21.15.137:0/3649758658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36200f7c40 0x7f362010ef10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.941+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 >> 172.21.15.137:0/3649758658 conn(0x7f36200f21a0 msgr2=0x7f36200fae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.942+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 shutdown_connections 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.942+0000 7f3628775700 1 -- 172.21.15.137:0/3649758658 wait complete. 2024-02-18T00:29:20.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:20.942+0000 7f3628775700 1 librados: shutdown 2024-02-18T00:29:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:20.086565+0000 mon.a (mon.0) 578 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-18T00:29:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:20.091771+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.59:0/2522576766' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:20.093088+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.59:0/998759214' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:20.096066+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:20.096537+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:20.256385+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v207: 132 pgs: 4 creating+activating, 15 creating+peering, 66 active+clean, 47 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-18T00:29:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:20.939887+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:21.088177+0000 mon.a (mon.0) 582 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T00:29:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:21.088285+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T00:29:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:21.088370+0000 mon.a (mon.0) 584 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-18T00:29:21.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:20.086565+0000 mon.a (mon.0) 578 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-18T00:29:21.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:20.091771+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.59:0/2522576766' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:20.093088+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.59:0/998759214' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:20.096066+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:20.096537+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:20.256385+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v207: 132 pgs: 4 creating+activating, 15 creating+peering, 66 active+clean, 47 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-18T00:29:21.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:20.939887+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:21.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:21.088177+0000 mon.a (mon.0) 582 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T00:29:21.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:21.088285+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T00:29:21.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:21.088370+0000 mon.a (mon.0) 584 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-18T00:29:21.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:20.086565+0000 mon.a (mon.0) 578 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-18T00:29:21.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:20.091771+0000 mon.c (mon.2) 11 : audit [INF] from='client.? 172.21.15.59:0/2522576766' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:20.093088+0000 mon.c (mon.2) 12 : audit [INF] from='client.? 172.21.15.59:0/998759214' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:20.096066+0000 mon.a (mon.0) 579 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:20.096537+0000 mon.a (mon.0) 580 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T00:29:21.507 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:20.256385+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v207: 132 pgs: 4 creating+activating, 15 creating+peering, 66 active+clean, 47 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-18T00:29:21.507 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:20.939887+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:21.507 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:21.088177+0000 mon.a (mon.0) 582 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T00:29:21.507 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:21.088285+0000 mon.a (mon.0) 583 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T00:29:21.507 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:21.088370+0000 mon.a (mon.0) 584 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-18T00:29:21.639 DEBUG:teuthology.orchestra.run.smithi137:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@iscsi.iscsi.a.service 2024-02-18T00:29:21.641 INFO:tasks.cephadm:Adding prometheus.a on smithi137 2024-02-18T00:29:21.642 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply prometheus '1;smithi137=a' 2024-02-18T00:29:21.671 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:20.933544+0000 mgr.y (mgr.14146) 213 : audit [DBG] from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi137=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:20.935834+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi137=iscsi.a;count:1 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:21.093733+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.59:0/2522576766' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:21.094691+0000 mon.a (mon.0) 585 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:21.096832+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.59:0/998759214' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:21.110018+0000 mon.a (mon.0) 586 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.094933+0000 mon.a (mon.0) 587 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.095031+0000 mon.a (mon.0) 588 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T00:29:22.383 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:22.095096+0000 mon.a (mon.0) 589 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:20.933544+0000 mgr.y (mgr.14146) 213 : audit [DBG] from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi137=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:20.935834+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi137=iscsi.a;count:1 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:21.093733+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.59:0/2522576766' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:21.094691+0000 mon.a (mon.0) 585 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:21.096832+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.59:0/998759214' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:21.110018+0000 mon.a (mon.0) 586 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.094933+0000 mon.a (mon.0) 587 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.095031+0000 mon.a (mon.0) 588 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T00:29:22.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:22.095096+0000 mon.a (mon.0) 589 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:20.933544+0000 mgr.y (mgr.14146) 213 : audit [DBG] from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi137=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:20.935834+0000 mgr.y (mgr.14146) 214 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi137=iscsi.a;count:1 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:21.093733+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.59:0/2522576766' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:21.094691+0000 mon.a (mon.0) 585 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:21.096832+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.59:0/998759214' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:21.110018+0000 mon.a (mon.0) 586 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.094933+0000 mon.a (mon.0) 587 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.095031+0000 mon.a (mon.0) 588 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T00:29:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:22.095096+0000 mon.a (mon.0) 589 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-18T00:29:23.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:22.256911+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v210: 132 pgs: 2 creating+activating, 10 creating+peering, 99 active+clean, 21 unknown; 1.5 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-02-18T00:29:23.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.281709+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.289343+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.300096+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:22.303088+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-18T00:29:23.409 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.311202+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-18T00:29:23.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.317433+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-18T00:29:23.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:22.326585+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:23.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:22.327700+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi137 2024-02-18T00:29:23.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 -- 172.21.15.137:0/2277891159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 msgr2=0x7f15000f7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 --2- 172.21.15.137:0/2277891159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f15000f7ea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f14f40048a0 tx=0x7f14f4009cb0).stop 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 -- 172.21.15.137:0/2277891159 shutdown_connections 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 --2- 172.21.15.137:0/2277891159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f15000f7ea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 --2- 172.21.15.137:0/2277891159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15000f9670 0x7f15000fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 --2- 172.21.15.137:0/2277891159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15000f8d20 0x7f15000f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 -- 172.21.15.137:0/2277891159 >> 172.21.15.137:0/2277891159 conn(0x7f15000f32a0 msgr2=0x7f15000f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 -- 172.21.15.137:0/2277891159 shutdown_connections 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 -- 172.21.15.137:0/2277891159 wait complete. 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 librados: starting msgr at 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 librados: starting objecter 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 Processor -- start 2024-02-18T00:29:23.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 -- start start 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 librados: setting wanted keys 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.621+0000 7f15070ee700 1 librados: calling monclient init 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f1500107f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15000f8d20 0x7f1500108420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15000f9670 0x7f150010f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150010f9f0 con 0x7f15000f9670 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150010fb30 con 0x7f15000f7a80 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150010fc70 con 0x7f15000f8d20 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f1500107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f1500107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50030/0 (socket says 172.21.15.137:50030) 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 -- 172.21.15.137:0/2300509680 learned_addr learned my addr 172.21.15.137:0/2300509680 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f14fffff700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15000f8d20 0x7f1500108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f150568b700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15000f9670 0x7f150010f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:23.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 -- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15000f8d20 msgr2=0x7f1500108420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15000f8d20 0x7f1500108420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 -- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15000f9670 msgr2=0x7f150010f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15000f9670 0x7f150010f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 -- 172.21.15.137:0/2300509680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f4005040 con 0x7f15000f7a80 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f14fffff700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15000f8d20 0x7f1500108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f150568b700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15000f9670 0x7f150010f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f1504e8a700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f1500107f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f14f400b1a0 tx=0x7f14f4009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f14fdffb700 10 client.?.objecter ms_handle_connect 0x7f15000f7a80 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f14fdffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f14fdffb700 1 -- 172.21.15.137:0/2300509680 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f400e030 con 0x7f15000f7a80 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f14fdffb700 1 -- 172.21.15.137:0/2300509680 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f4004100 con 0x7f15000f7a80 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150010fef0 con 0x7f15000f7a80 2024-02-18T00:29:23.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f14fdffb700 1 -- 172.21.15.137:0/2300509680 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4018070 con 0x7f15000f7a80 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 10 client.24409.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15001102c0 con 0x7f15000f7a80 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.622+0000 7f15070ee700 1 librados: init done 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.623+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150005f360 con 0x7f15000f7a80 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fdffb700 1 -- 172.21.15.137:0/2300509680 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 14) v1 ==== 77029+0+0 (secure 0 0 0) 0x7f14f401c920 con 0x7f15000f7a80 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fdffb700 1 --2- 172.21.15.137:0/2300509680 >> v2:172.21.15.59:6800/817089002 conn(0x7f14f0062ba0 0x7f14f0065060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fdffb700 1 -- 172.21.15.137:0/2300509680 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5884+0+0 (secure 0 0 0) 0x7f14f407e160 con 0x7f15000f7a80 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fdffb700 10 client.24409.objecter ms_dispatch 0x7f1500001550 osd_map(64..64 src has 1..64) v4 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fdffb700 3 client.24409.objecter handle_osd_map got epochs [64,64] > 0 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fdffb700 3 client.24409.objecter handle_osd_map decoding full epoch 64 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fdffb700 20 client.24409.objecter dump_active .. 0 homeless 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.624+0000 7f14fffff700 1 --2- 172.21.15.137:0/2300509680 >> v2:172.21.15.59:6800/817089002 conn(0x7f14f0062ba0 0x7f14f0065060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.625+0000 7f14fdffb700 10 client.24409.objecter ms_handle_connect 0x7f14f0062ba0 2024-02-18T00:29:23.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.625+0000 7f14fffff700 1 --2- 172.21.15.137:0/2300509680 >> v2:172.21.15.59:6800/817089002 conn(0x7f14f0062ba0 0x7f14f0065060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f14ec004c00 tx=0x7f14ec00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:23.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.629+0000 7f14fdffb700 1 -- 172.21.15.137:0/2300509680 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f14f404e4c0 con 0x7f15000f7a80 2024-02-18T00:29:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:22.256911+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v210: 132 pgs: 2 creating+activating, 10 creating+peering, 99 active+clean, 21 unknown; 1.5 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-02-18T00:29:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.281709+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.289343+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.300096+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:22.303088+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.311202+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.317433+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:22.326585+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:22.327700+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi137 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:22.256911+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v210: 132 pgs: 2 creating+activating, 10 creating+peering, 99 active+clean, 21 unknown; 1.5 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.281709+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.289343+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.300096+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:22.303088+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-18T00:29:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.311202+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-18T00:29:23.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.317433+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-18T00:29:23.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:22.326585+0000 mon.a (mon.0) 595 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:23.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:22.327700+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi137 2024-02-18T00:29:23.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.809+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi137=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f15000fef20 con 0x7f14f0062ba0 2024-02-18T00:29:23.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.816+0000 7f14fdffb700 1 -- 172.21.15.137:0/2300509680 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f15000fef20 con 0x7f14f0062ba0 2024-02-18T00:29:23.818 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled prometheus update... 2024-02-18T00:29:23.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.818+0000 7f15070ee700 10 librados: watch_flush enter 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.818+0000 7f15070ee700 10 librados: watch_flush exit 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.818+0000 7f15070ee700 20 client.24409.objecter shutdown clearing up homeless session... 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.818+0000 7f15070ee700 10 client.24409.objecter successfully canceled tick 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 >> v2:172.21.15.59:6800/817089002 conn(0x7f14f0062ba0 msgr2=0x7f14f0065060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 --2- 172.21.15.137:0/2300509680 >> v2:172.21.15.59:6800/817089002 conn(0x7f14f0062ba0 0x7f14f0065060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f14ec004c00 tx=0x7f14ec00b040).stop 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 msgr2=0x7f1500107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f1500107f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f14f400b1a0 tx=0x7f14f4009cb0).stop 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 shutdown_connections 2024-02-18T00:29:23.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 --2- 172.21.15.137:0/2300509680 >> v2:172.21.15.59:6800/817089002 conn(0x7f14f0062ba0 0x7f14f0065060 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15000f7a80 0x7f1500107f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15000f9670 0x7f150010f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 --2- 172.21.15.137:0/2300509680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15000f8d20 0x7f1500108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:23.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 >> 172.21.15.137:0/2300509680 conn(0x7f15000f32a0 msgr2=0x7f15000f6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:23.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 shutdown_connections 2024-02-18T00:29:23.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 -- 172.21.15.137:0/2300509680 wait complete. 2024-02-18T00:29:23.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:23.819+0000 7f15070ee700 1 librados: shutdown 2024-02-18T00:29:25.048 DEBUG:teuthology.orchestra.run.smithi137:prometheus.a> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@prometheus.a.service 2024-02-18T00:29:25.050 INFO:tasks.cephadm:Adding node-exporter.a on smithi059 2024-02-18T00:29:25.050 INFO:tasks.cephadm:Adding node-exporter.b on smithi137 2024-02-18T00:29:25.050 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply node-exporter '2;smithi059=a;smithi137=b' 2024-02-18T00:29:25.074 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:23.811370+0000 mgr.y (mgr.14146) 218 : audit [DBG] from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi137=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:25.074 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:23.813106+0000 mgr.y (mgr.14146) 219 : cephadm [INF] Saving service prometheus spec with placement smithi137=a;count:1 2024-02-18T00:29:25.075 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:23.816851+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:25.075 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:24.257631+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v211: 132 pgs: 5 creating+peering, 120 active+clean, 7 unknown; 2.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 2.9 KiB/s wr, 115 op/s 2024-02-18T00:29:25.079 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:29:25.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:23.811370+0000 mgr.y (mgr.14146) 218 : audit [DBG] from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi137=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:25.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:23.813106+0000 mgr.y (mgr.14146) 219 : cephadm [INF] Saving service prometheus spec with placement smithi137=a;count:1 2024-02-18T00:29:25.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:23.816851+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:25.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:24.257631+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v211: 132 pgs: 5 creating+peering, 120 active+clean, 7 unknown; 2.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 2.9 KiB/s wr, 115 op/s 2024-02-18T00:29:25.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:23.811370+0000 mgr.y (mgr.14146) 218 : audit [DBG] from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi137=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:25.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:23.813106+0000 mgr.y (mgr.14146) 219 : cephadm [INF] Saving service prometheus spec with placement smithi137=a;count:1 2024-02-18T00:29:25.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:23.816851+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:25.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:24.257631+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v211: 132 pgs: 5 creating+peering, 120 active+clean, 7 unknown; 2.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 2.9 KiB/s wr, 115 op/s 2024-02-18T00:29:26.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:24.827001+0000 mon.a (mon.0) 597 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-18T00:29:26.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:24.827001+0000 mon.a (mon.0) 597 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-18T00:29:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:24.827001+0000 mon.a (mon.0) 597 : cluster [DBG] mgrmap e15: y(active, since 5m), standbys: x 2024-02-18T00:29:27.222 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:26 smithi137 bash[60759]: log file path now is '/var/log/tcmu-runner.log' 2024-02-18T00:29:27.222 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:26.258231+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v212: 132 pgs: 132 active+clean; 4.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 86 KiB/s rd, 4.2 KiB/s wr, 190 op/s 2024-02-18T00:29:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:26.258231+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v212: 132 pgs: 132 active+clean; 4.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 86 KiB/s rd, 4.2 KiB/s wr, 190 op/s 2024-02-18T00:29:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:26.258231+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v212: 132 pgs: 132 active+clean; 4.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 86 KiB/s rd, 4.2 KiB/s wr, 190 op/s 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.836+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/741334430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f8d20 msgr2=0x7fcc280f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.836+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/741334430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f8d20 0x7fcc280f9180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fcc240048a0 tx=0x7fcc24009cb0).stop 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.836+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/741334430 shutdown_connections 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.836+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/741334430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280f9670 0x7fcc280fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.836+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/741334430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f7a80 0x7fcc280f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/741334430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f8d20 0x7fcc280f9180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/741334430 >> 172.21.15.137:0/741334430 conn(0x7fcc280f32a0 msgr2=0x7fcc280f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/741334430 shutdown_connections 2024-02-18T00:29:27.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/741334430 wait complete. 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 librados: starting msgr at 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 librados: starting objecter 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 Processor -- start 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 -- start start 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 librados: setting wanted keys 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 librados: calling monclient init 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280f7a80 0x7fcc28107f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f8d20 0x7fcc28108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f9670 0x7fcc2810f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2810fa20 con 0x7fcc280f8d20 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2810fb60 con 0x7fcc280f9670 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2fbc0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2810fca0 con 0x7fcc280f7a80 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.837+0000 7fcc2e15d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f9670 0x7fcc2810f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:27.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f9670 0x7fcc2810f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48498/0 (socket says 172.21.15.137:48498) 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 -- 172.21.15.137:0/597790919 learned_addr learned my addr 172.21.15.137:0/597790919 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2d15b700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f8d20 0x7fcc28108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2d95c700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280f7a80 0x7fcc28107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 -- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280f7a80 msgr2=0x7fcc28107f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280f7a80 0x7fcc28107f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 -- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f8d20 msgr2=0x7fcc28108450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f8d20 0x7fcc28108450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 -- 172.21.15.137:0/597790919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc24005040 con 0x7fcc280f9670 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2e15d700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f9670 0x7fcc2810f530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fcc2000ec20 tx=0x7fcc2000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc1effd700 10 client.?.objecter ms_handle_connect 0x7fcc280f9670 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc1effd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc1effd700 1 -- 172.21.15.137:0/597790919 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20009950 con 0x7fcc280f9670 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc1effd700 1 -- 172.21.15.137:0/597790919 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc2000f040 con 0x7fcc280f9670 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2810ff20 con 0x7fcc280f9670 2024-02-18T00:29:27.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc1effd700 1 -- 172.21.15.137:0/597790919 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20007df0 con 0x7fcc280f9670 2024-02-18T00:29:27.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2fbc0700 10 client.24415.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:27.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28110330 con 0x7fcc280f9670 2024-02-18T00:29:27.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.838+0000 7fcc2fbc0700 1 librados: init done 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.839+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc28102880 con 0x7fcc280f9670 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.839+0000 7fcc1effd700 1 -- 172.21.15.137:0/597790919 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) v1 ==== 77080+0+0 (secure 0 0 0) 0x7fcc200092e0 con 0x7fcc280f9670 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.840+0000 7fcc1effd700 1 --2- 172.21.15.137:0/597790919 >> v2:172.21.15.59:6800/817089002 conn(0x7fcc14062c20 0x7fcc140650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.840+0000 7fcc1effd700 1 -- 172.21.15.137:0/597790919 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5884+0+0 (secure 0 0 0) 0x7fcc2007c700 con 0x7fcc280f9670 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.840+0000 7fcc1effd700 10 client.24415.objecter ms_dispatch 0x7fcc28001550 osd_map(64..64 src has 1..64) v4 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.840+0000 7fcc1effd700 3 client.24415.objecter handle_osd_map got epochs [64,64] > 0 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.840+0000 7fcc1effd700 3 client.24415.objecter handle_osd_map decoding full epoch 64 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.840+0000 7fcc1effd700 20 client.24415.objecter dump_active .. 0 homeless 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.840+0000 7fcc2d95c700 1 --2- 172.21.15.137:0/597790919 >> v2:172.21.15.59:6800/817089002 conn(0x7fcc14062c20 0x7fcc140650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.841+0000 7fcc1effd700 10 client.24415.objecter ms_handle_connect 0x7fcc14062c20 2024-02-18T00:29:27.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.841+0000 7fcc2d95c700 1 --2- 172.21.15.137:0/597790919 >> v2:172.21.15.59:6800/817089002 conn(0x7fcc14062c20 0x7fcc140650e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fcc280f8b80 tx=0x7fcc1800b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:27.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:27.844+0000 7fcc1effd700 1 -- 172.21.15.137:0/597790919 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcc20049f60 con 0x7fcc280f9670 2024-02-18T00:29:28.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.021+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi059=a;smithi137=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc280feef0 con 0x7fcc14062c20 2024-02-18T00:29:28.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.028+0000 7fcc1effd700 1 -- 172.21.15.137:0/597790919 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fcc280feef0 con 0x7fcc14062c20 2024-02-18T00:29:28.029 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled node-exporter update... 2024-02-18T00:29:28.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 10 librados: watch_flush enter 2024-02-18T00:29:28.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 10 librados: watch_flush exit 2024-02-18T00:29:28.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 20 client.24415.objecter shutdown clearing up homeless session... 2024-02-18T00:29:28.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 10 client.24415.objecter successfully canceled tick 2024-02-18T00:29:28.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 >> v2:172.21.15.59:6800/817089002 conn(0x7fcc14062c20 msgr2=0x7fcc140650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/597790919 >> v2:172.21.15.59:6800/817089002 conn(0x7fcc14062c20 0x7fcc140650e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fcc280f8b80 tx=0x7fcc1800b040).stop 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f9670 msgr2=0x7fcc2810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f9670 0x7fcc2810f530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fcc2000ec20 tx=0x7fcc2000b040).stop 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 shutdown_connections 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/597790919 >> v2:172.21.15.59:6800/817089002 conn(0x7fcc14062c20 0x7fcc140650e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280f9670 0x7fcc2810f530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280f7a80 0x7fcc28107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 --2- 172.21.15.137:0/597790919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f8d20 0x7fcc28108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 >> 172.21.15.137:0/597790919 conn(0x7fcc280f32a0 msgr2=0x7fcc280f66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 shutdown_connections 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 -- 172.21.15.137:0/597790919 wait complete. 2024-02-18T00:29:28.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:28.030+0000 7fcc2fbc0700 1 librados: shutdown 2024-02-18T00:29:28.688 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:28 smithi137 bash[60919]: 18ee164d05e1fbb78a2a082fff41844f707a0c3b921495f89c4e52262faa7a8c 2024-02-18T00:29:28.688 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:28 smithi137 systemd[1]: Started Ceph iscsi.iscsi.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:29:29.001 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug Started the configuration object watcher 2024-02-18T00:29:29.001 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug Checking for config object changes every 1s 2024-02-18T00:29:29.001 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug Processing osd blocklist entries for this node 2024-02-18T00:29:29.067 DEBUG:teuthology.orchestra.run.smithi059:node-exporter.a> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@node-exporter.a.service 2024-02-18T00:29:29.071 DEBUG:teuthology.orchestra.run.smithi137:node-exporter.b> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@node-exporter.b.service 2024-02-18T00:29:29.109 INFO:tasks.cephadm:Adding alertmanager.a on smithi059 2024-02-18T00:29:29.109 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply alertmanager '1;smithi059=a' 2024-02-18T00:29:29.111 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:29:29.142 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:29:29.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug Reading the configuration object to update local LIO configuration 2024-02-18T00:29:29.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug Configuration does not have an entry for this host(smithi137.front.sepia.ceph.com) - nothing to define to LIO 2024-02-18T00:29:29.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-18T00:29:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:27.698311+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:29:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:27.699566+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:29:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:28.023138+0000 mgr.y (mgr.14146) 222 : audit [DBG] from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi059=a;smithi137=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:28.024627+0000 mgr.y (mgr.14146) 223 : cephadm [INF] Saving service node-exporter spec with placement smithi059=a;smithi137=b;count:2 2024-02-18T00:29:29.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:28.028546+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:29.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:28.258831+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v213: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 90 KiB/s rd, 4.5 KiB/s wr, 194 op/s 2024-02-18T00:29:29.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:28.577783+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:29.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:28.579224+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:29.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:28.987630+0000 mon.b (mon.1) 24 : audit [DBG] from='client.? 172.21.15.137:0/2233012031' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T00:29:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:27.698311+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:29:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:27.699566+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:29:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:28.023138+0000 mgr.y (mgr.14146) 222 : audit [DBG] from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi059=a;smithi137=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:28.024627+0000 mgr.y (mgr.14146) 223 : cephadm [INF] Saving service node-exporter spec with placement smithi059=a;smithi137=b;count:2 2024-02-18T00:29:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:28.028546+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:29.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:28.258831+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v213: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 90 KiB/s rd, 4.5 KiB/s wr, 194 op/s 2024-02-18T00:29:29.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:28.577783+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:29.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:28.579224+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:29.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:28.987630+0000 mon.b (mon.1) 24 : audit [DBG] from='client.? 172.21.15.137:0/2233012031' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:27.698311+0000 osd.7 (osd.7) 3 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:27.699566+0000 osd.7 (osd.7) 4 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:28.023138+0000 mgr.y (mgr.14146) 222 : audit [DBG] from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi059=a;smithi137=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:28.024627+0000 mgr.y (mgr.14146) 223 : cephadm [INF] Saving service node-exporter spec with placement smithi059=a;smithi137=b;count:2 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:28.028546+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:28.258831+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v213: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 90 KiB/s rd, 4.5 KiB/s wr, 194 op/s 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:28.577783+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:28.579224+0000 mon.a (mon.0) 600 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:29.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:28.987630+0000 mon.b (mon.1) 24 : audit [DBG] from='client.? 172.21.15.137:0/2233012031' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 -- 172.21.15.137:0/224283915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f7c30 msgr2=0x7f2bac0f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 --2- 172.21.15.137:0/224283915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f7c30 0x7f2bac0f8090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f2ba80048a0 tx=0x7f2ba8009cb0).stop 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 -- 172.21.15.137:0/224283915 shutdown_connections 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 --2- 172.21.15.137:0/224283915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac0f8580 0x7f2bac0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 --2- 172.21.15.137:0/224283915 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2bac0f6990 0x7f2bac0f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 --2- 172.21.15.137:0/224283915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f7c30 0x7f2bac0f8090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 -- 172.21.15.137:0/224283915 >> 172.21.15.137:0/224283915 conn(0x7f2bac0f2190 msgr2=0x7f2bac0f45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.451+0000 7f2bb4448700 1 -- 172.21.15.137:0/224283915 shutdown_connections 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 -- 172.21.15.137:0/224283915 wait complete. 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 librados: starting msgr at 2024-02-18T00:29:31.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 librados: starting objecter 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 Processor -- start 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 -- start start 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 librados: setting wanted keys 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 librados: calling monclient init 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2bac0f6990 0x7f2bac107f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac0f7c30 0x7f2bac108470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f8580 0x7f2bac10f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac10fa40 con 0x7f2bac0f7c30 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac10fb80 con 0x7f2bac0f8580 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.452+0000 7f2bb4448700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac10fcc0 con 0x7f2bac0f6990 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f8580 0x7f2bac10f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f8580 0x7f2bac10f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48564/0 (socket says 172.21.15.137:48564) 2024-02-18T00:29:31.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb21e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2bac0f6990 0x7f2bac107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 -- 172.21.15.137:0/217181742 learned_addr learned my addr 172.21.15.137:0/217181742 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb19e3700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac0f7c30 0x7f2bac108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 -- 172.21.15.137:0/217181742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2bac0f6990 msgr2=0x7f2bac107f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2bac0f6990 0x7f2bac107f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 -- 172.21.15.137:0/217181742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac0f7c30 msgr2=0x7f2bac108470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac0f7c30 0x7f2bac108470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 -- 172.21.15.137:0/217181742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba8005040 con 0x7f2bac0f8580 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb29e5700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f8580 0x7f2bac10f550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f2ba400c870 tx=0x7f2ba4007dd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2b9f7fe700 10 client.?.objecter ms_handle_connect 0x7f2bac0f8580 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2b9f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2b9f7fe700 1 -- 172.21.15.137:0/217181742 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4010510 con 0x7f2bac0f8580 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2b9f7fe700 1 -- 172.21.15.137:0/217181742 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ba4010b40 con 0x7f2bac0f8580 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac10ffa0 con 0x7f2bac0f8580 2024-02-18T00:29:31.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2b9f7fe700 1 -- 172.21.15.137:0/217181742 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4012ae0 con 0x7f2bac0f8580 2024-02-18T00:29:31.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb4448700 10 client.24442.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:31.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac110480 con 0x7f2bac0f8580 2024-02-18T00:29:31.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.453+0000 7f2bb4448700 1 librados: init done 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.454+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac102870 con 0x7f2bac0f8580 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.454+0000 7f2b9f7fe700 1 -- 172.21.15.137:0/217181742 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) v1 ==== 77080+0+0 (secure 0 0 0) 0x7f2ba4010670 con 0x7f2bac0f8580 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.455+0000 7f2b9f7fe700 1 --2- 172.21.15.137:0/217181742 >> v2:172.21.15.59:6800/817089002 conn(0x7f2b98062c20 0x7f2b980650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.455+0000 7f2b9f7fe700 1 -- 172.21.15.137:0/217181742 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5884+0+0 (secure 0 0 0) 0x7f2ba407c950 con 0x7f2bac0f8580 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.455+0000 7f2b9f7fe700 10 client.24442.objecter ms_dispatch 0x7f2bac001550 osd_map(64..64 src has 1..64) v4 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.455+0000 7f2b9f7fe700 3 client.24442.objecter handle_osd_map got epochs [64,64] > 0 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.455+0000 7f2b9f7fe700 3 client.24442.objecter handle_osd_map decoding full epoch 64 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.455+0000 7f2b9f7fe700 20 client.24442.objecter dump_active .. 0 homeless 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.455+0000 7f2bb21e4700 1 --2- 172.21.15.137:0/217181742 >> v2:172.21.15.59:6800/817089002 conn(0x7f2b98062c20 0x7f2b980650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.456+0000 7f2b9f7fe700 10 client.24442.objecter ms_handle_connect 0x7f2b98062c20 2024-02-18T00:29:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.456+0000 7f2bb21e4700 1 --2- 172.21.15.137:0/217181742 >> v2:172.21.15.59:6800/817089002 conn(0x7f2b98062c20 0x7f2b980650e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2bac0f7a90 tx=0x7f2ba0009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:31.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.459+0000 7f2b9f7fe700 1 -- 172.21.15.137:0/217181742 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2ba404ccd0 con 0x7f2bac0f8580 2024-02-18T00:29:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:30.259496+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v214: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 85 KiB/s rd, 3.9 KiB/s wr, 180 op/s 2024-02-18T00:29:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:30.402213+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:31.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.685+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi059=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bac0fddb0 con 0x7f2b98062c20 2024-02-18T00:29:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.692+0000 7f2b9f7fe700 1 -- 172.21.15.137:0/217181742 <== mgr.14146 v2:172.21.15.59:6800/817089002 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f2bac0fddb0 con 0x7f2b98062c20 2024-02-18T00:29:31.694 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled alertmanager update... 2024-02-18T00:29:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 10 librados: watch_flush enter 2024-02-18T00:29:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 10 librados: watch_flush exit 2024-02-18T00:29:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 20 client.24442.objecter shutdown clearing up homeless session... 2024-02-18T00:29:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 10 client.24442.objecter successfully canceled tick 2024-02-18T00:29:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 >> v2:172.21.15.59:6800/817089002 conn(0x7f2b98062c20 msgr2=0x7f2b980650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 --2- 172.21.15.137:0/217181742 >> v2:172.21.15.59:6800/817089002 conn(0x7f2b98062c20 0x7f2b980650e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f2bac0f7a90 tx=0x7f2ba0009040).stop 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f8580 msgr2=0x7f2bac10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f8580 0x7f2bac10f550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f2ba400c870 tx=0x7f2ba4007dd0).stop 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 shutdown_connections 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 --2- 172.21.15.137:0/217181742 >> v2:172.21.15.59:6800/817089002 conn(0x7f2b98062c20 0x7f2b980650e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2bac0f8580 0x7f2bac10f550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2bac0f6990 0x7f2bac107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 --2- 172.21.15.137:0/217181742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2bac0f7c30 0x7f2bac108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 >> 172.21.15.137:0/217181742 conn(0x7f2bac0f2190 msgr2=0x7f2bac0f4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.694+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 shutdown_connections 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.695+0000 7f2bb4448700 1 -- 172.21.15.137:0/217181742 wait complete. 2024-02-18T00:29:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:31.695+0000 7f2bb4448700 1 librados: shutdown 2024-02-18T00:29:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:30.259496+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v214: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 85 KiB/s rd, 3.9 KiB/s wr, 180 op/s 2024-02-18T00:29:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:30.402213+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:30.259496+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v214: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 85 KiB/s rd, 3.9 KiB/s wr, 180 op/s 2024-02-18T00:29:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:30.402213+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:32.576 DEBUG:teuthology.orchestra.run.smithi059:alertmanager.a> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@alertmanager.a.service 2024-02-18T00:29:32.579 INFO:tasks.cephadm:Adding grafana.a on smithi137 2024-02-18T00:29:32.579 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph orch apply grafana '1;smithi137=a' 2024-02-18T00:29:32.614 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:-- Logs begin at Sun 2024-02-18 00:12:00 UTC. -- 2024-02-18T00:29:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:31.687332+0000 mgr.y (mgr.14146) 226 : audit [DBG] from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi059=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:31.688749+0000 mgr.y (mgr.14146) 227 : cephadm [INF] Saving service alertmanager spec with placement smithi059=a;count:1 2024-02-18T00:29:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:31.692741+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:32.260307+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v215: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s rd, 3.5 KiB/s wr, 168 op/s 2024-02-18T00:29:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:31.687332+0000 mgr.y (mgr.14146) 226 : audit [DBG] from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi059=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:31.688749+0000 mgr.y (mgr.14146) 227 : cephadm [INF] Saving service alertmanager spec with placement smithi059=a;count:1 2024-02-18T00:29:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:31.692741+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:32.260307+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v215: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s rd, 3.5 KiB/s wr, 168 op/s 2024-02-18T00:29:33.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:31.687332+0000 mgr.y (mgr.14146) 226 : audit [DBG] from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi059=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:29:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:31.688749+0000 mgr.y (mgr.14146) 227 : cephadm [INF] Saving service alertmanager spec with placement smithi059=a;count:1 2024-02-18T00:29:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:31.692741+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:32.260307+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v215: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s rd, 3.5 KiB/s wr, 168 op/s 2024-02-18T00:29:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:33.128733+0000 mon.a (mon.0) 603 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:33.129693+0000 mon.a (mon.0) 604 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:33.130641+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:33.135402+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:33.140668+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-18T00:29:34.255 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ignoring --setuser ceph since I am not root 2024-02-18T00:29:34.255 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ignoring --setgroup ceph since I am not root 2024-02-18T00:29:34.255 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:34.174+0000 7f9a57920700 1 -- 172.21.15.137:0/1975421850 <== mon.1 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557d157d04e0 con 0x557d157a7400 2024-02-18T00:29:34.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:33.128733+0000 mon.a (mon.0) 603 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:34.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:33.129693+0000 mon.a (mon.0) 604 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:34.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:33.130641+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:34.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:33.135402+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:34.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:33.140668+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-18T00:29:34.398 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:33.128733+0000 mon.a (mon.0) 603 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:33.129693+0000 mon.a (mon.0) 604 : audit [DBG] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:33.130641+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:33.135402+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:33.140668+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ignoring --setuser ceph since I am not root 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ignoring --setgroup ceph since I am not root 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:34.183+0000 7f11a9941700 1 -- 172.21.15.59:0/1521712922 <== mon.2 v2:172.21.15.59:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b4f1404e0 con 0x561b4f116c00 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:34.311+0000 7f11b9b1e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T00:29:34.399 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:34.396+0000 7f11b9b1e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T00:29:34.578 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:34.304+0000 7f9a67afd680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T00:29:34.578 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:34.389+0000 7f9a67afd680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T00:29:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 -- 172.21.15.137:0/1480951076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f7c30 msgr2=0x7fc3f40f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 --2- 172.21.15.137:0/1480951076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f7c30 0x7fc3f40f8090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc3e40048a0 tx=0x7fc3e4009cb0).stop 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 -- 172.21.15.137:0/1480951076 shutdown_connections 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 --2- 172.21.15.137:0/1480951076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3f40f8580 0x7fc3f40fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 --2- 172.21.15.137:0/1480951076 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3f40f6990 0x7fc3f40f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 --2- 172.21.15.137:0/1480951076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f7c30 0x7fc3f40f8090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 -- 172.21.15.137:0/1480951076 >> 172.21.15.137:0/1480951076 conn(0x7fc3f40f21b0 msgr2=0x7fc3f40f45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 -- 172.21.15.137:0/1480951076 shutdown_connections 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 -- 172.21.15.137:0/1480951076 wait complete. 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 librados: starting msgr at 2024-02-18T00:29:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 librados: starting objecter 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 Processor -- start 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 -- start start 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 librados: setting wanted keys 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.837+0000 7fc3f950c700 1 librados: calling monclient init 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f950c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f6990 0x7fc3f40fa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f950c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3f40f7c30 0x7fc3f40f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f950c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3f40f8580 0x7fc3f40f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f950c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40f9a80 con 0x7fc3f40f7c30 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f950c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40f9bf0 con 0x7fc3f40f6990 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f950c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40f9db0 con 0x7fc3f40f8580 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f6990 0x7fc3f40fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f6990 0x7fc3f40fa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38664/0 (socket says 172.21.15.137:38664) 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 -- 172.21.15.137:0/1389043728 learned_addr learned my addr 172.21.15.137:0/1389043728 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3f40f7c30 0x7fc3f40f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f37fe700 1 --2- 172.21.15.137:0/1389043728 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3f40f8580 0x7fc3f40f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 -- 172.21.15.137:0/1389043728 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3f40f8580 msgr2=0x7fc3f40f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 --2- 172.21.15.137:0/1389043728 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc3f40f8580 0x7fc3f40f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 -- 172.21.15.137:0/1389043728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3f40f7c30 msgr2=0x7fc3f40f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 --2- 172.21.15.137:0/1389043728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3f40f7c30 0x7fc3f40f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 -- 172.21.15.137:0/1389043728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4005040 con 0x7fc3f40f6990 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f2ffd700 1 --2- 172.21.15.137:0/1389043728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f6990 0x7fc3f40fa910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fc3e8002830 tx=0x7fc3e8009ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3dbfff700 10 client.?.objecter ms_handle_connect 0x7fc3f40f6990 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3dbfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007b40 con 0x7fc3f40f6990 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3e800f040 con 0x7fc3f40f6990 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f4112110 con 0x7fc3f40f6990 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.838+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e80147d0 con 0x7fc3f40f6990 2024-02-18T00:29:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.839+0000 7fc3f950c700 10 client.24460.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:34.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.839+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f41125b0 con 0x7fc3f40f6990 2024-02-18T00:29:34.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.839+0000 7fc3f950c700 1 librados: init done 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 77094+0+0 (secure 0 0 0) 0x7fc3e8014b80 con 0x7fc3f40f6990 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f405f360 con 0x7fc3f40f6990 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3dbfff700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 0x7fc3dc065130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3f27fc700 1 -- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 msgr2=0x7fc3dc065130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817089002 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5884+0+0 (secure 0 0 0) 0x7fc3e807c9e0 con 0x7fc3f40f6990 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3dbfff700 10 client.24460.objecter ms_dispatch 0x7fc3f4001550 osd_map(64..64 src has 1..64) v4 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3dbfff700 3 client.24460.objecter handle_osd_map got epochs [64,64] > 0 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3dbfff700 3 client.24460.objecter handle_osd_map decoding full epoch 64 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 0x7fc3dc065130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-18T00:29:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.840+0000 7fc3dbfff700 20 client.24460.objecter dump_active .. 0 homeless 2024-02-18T00:29:34.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:34.844+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc3e804cc30 con 0x7fc3f40f6990 2024-02-18T00:29:35.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:34.764+0000 7f9a67afd680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T00:29:35.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:35.023+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 --> v2:172.21.15.59:6800/817089002 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi137=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3f4061170 con 0x7fc3dc062c70 2024-02-18T00:29:35.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:35.041+0000 7fc3f27fc700 1 -- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 msgr2=0x7fc3dc065130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817089002 2024-02-18T00:29:35.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:35.041+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 0x7fc3dc065130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-18T00:29:35.142 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:34.780+0000 7f11b9b1e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T00:29:35.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:35.442+0000 7fc3f27fc700 1 -- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 msgr2=0x7fc3dc065130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817089002 2024-02-18T00:29:35.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:35.442+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 0x7fc3dc065130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-18T00:29:35.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:34.144252+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-18T00:29:35.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:34.144305+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-18T00:29:35.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:34.144252+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-18T00:29:35.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:34.144305+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-18T00:29:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:34.144252+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14146 172.21.15.59:0/972793065' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-18T00:29:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:34.144305+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e16: y(active, since 5m), standbys: x 2024-02-18T00:29:35.808 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:35.511+0000 7f9a67afd680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T00:29:35.808 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:35.595+0000 7f9a67afd680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T00:29:35.808 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:35.806+0000 7f9a67afd680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T00:29:35.874 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:35.572+0000 7f11b9b1e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T00:29:35.874 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:35.654+0000 7f11b9b1e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T00:29:35.874 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:35.873+0000 7f11b9b1e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T00:29:36.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:36.243+0000 7fc3f27fc700 1 -- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 msgr2=0x7fc3dc065130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817089002 2024-02-18T00:29:36.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:36.243+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 0x7fc3dc065130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-18T00:29:37.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:36.729+0000 7f9a67afd680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T00:29:37.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:36.856+0000 7f9a67afd680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:29:37.004 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:36.934+0000 7f9a67afd680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T00:29:37.162 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:36.850+0000 7f11b9b1e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T00:29:37.162 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:36.984+0000 7f11b9b1e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:29:37.162 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:37.065+0000 7f11b9b1e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T00:29:37.498 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:37.023+0000 7f9a67afd680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T00:29:37.498 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:37.221+0000 7f9a67afd680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T00:29:37.499 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:37.496+0000 7f9a67afd680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T00:29:37.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:37.161+0000 7f11b9b1e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T00:29:37.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:37.364+0000 7f11b9b1e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T00:29:37.753 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:37.595+0000 7f9a67afd680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T00:29:37.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:37.844+0000 7fc3f27fc700 1 -- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 msgr2=0x7fc3dc065130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817089002 2024-02-18T00:29:37.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:37.844+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 0x7fc3dc065130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-02-18T00:29:38.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:37.656+0000 7f11b9b1e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T00:29:38.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:37.758+0000 7f11b9b1e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T00:29:38.656 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:38.310+0000 7f9a67afd680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T00:29:38.656 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:38.391+0000 7f9a67afd680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T00:29:38.656 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:38.483+0000 7f9a67afd680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T00:29:38.877 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:38.513+0000 7f11b9b1e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T00:29:38.877 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:38.599+0000 7f11b9b1e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T00:29:38.877 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:38.694+0000 7f11b9b1e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T00:29:39.178 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:38.654+0000 7f9a67afd680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T00:29:39.178 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:38.915+0000 7f9a67afd680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T00:29:39.178 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:39.044+0000 7f9a67afd680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T00:29:39.178 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:39.176+0000 7f9a67afd680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:29:39.420 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:38.876+0000 7f11b9b1e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T00:29:39.420 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:39.142+0000 7f11b9b1e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T00:29:39.420 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:39.279+0000 7f11b9b1e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T00:29:39.420 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:39.419+0000 7f11b9b1e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T00:29:39.430 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:39.346+0000 7f9a67afd680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T00:29:39.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:39.499+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 17) v1 ==== 77094+0+0 (secure 0 0 0) 0x7fc3e807a790 con 0x7fc3f40f6990 2024-02-18T00:29:39.691 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:39.603+0000 7f11b9b1e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T00:29:39.691 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.443119+0000 mon.a (mon.0) 610 : cluster [DBG] Standby manager daemon x restarted 2024-02-18T00:29:39.691 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.443278+0000 mon.a (mon.0) 611 : cluster [DBG] Standby manager daemon x started 2024-02-18T00:29:39.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.444639+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T00:29:39.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.444987+0000 mon.b (mon.1) 26 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T00:29:39.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.447011+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T00:29:39.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.447915+0000 mon.b (mon.1) 28 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T00:29:39.692 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.443119+0000 mon.a (mon.0) 610 : cluster [DBG] Standby manager daemon x restarted 2024-02-18T00:29:39.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.443278+0000 mon.a (mon.0) 611 : cluster [DBG] Standby manager daemon x started 2024-02-18T00:29:39.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.444639+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T00:29:39.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.444987+0000 mon.b (mon.1) 26 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T00:29:39.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.447011+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T00:29:39.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.447915+0000 mon.b (mon.1) 28 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T00:29:39.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:39.712+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 18) v1 ==== 76859+0+0 (secure 0 0 0) 0x7fc3e807ab10 con 0x7fc3f40f6990 2024-02-18T00:29:39.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:39.713+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 msgr2=0x7fc3dc065130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:29:39.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:39.713+0000 7fc3dbfff700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/817089002 conn(0x7fc3dc062c70 0x7fc3dc065130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: debug 2024-02-18T00:29:39.428+0000 7f9a67afd680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: [18/Feb/2024:00:29:39] ENGINE Bus STARTING 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: CherryPy Checker: 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: The Application mounted at '' has an empty config. 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: [18/Feb/2024:00:29:39] ENGINE Serving on http://:::9283 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: [18/Feb/2024:00:29:39] ENGINE Bus STARTED 2024-02-18T00:29:39.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.443119+0000 mon.a (mon.0) 610 : cluster [DBG] Standby manager daemon x restarted 2024-02-18T00:29:39.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.443278+0000 mon.a (mon.0) 611 : cluster [DBG] Standby manager daemon x started 2024-02-18T00:29:39.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.444639+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T00:29:39.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.444987+0000 mon.b (mon.1) 26 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T00:29:39.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.447011+0000 mon.b (mon.1) 27 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T00:29:39.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.447915+0000 mon.b (mon.1) 28 : audit [DBG] from='mgr.? 172.21.15.137:0/3772329849' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T00:29:39.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:39.840+0000 7fc3f8d0b700 10 client.24460.objecter tick 2024-02-18T00:29:39.950 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:29:39.690+0000 7f11b9b1e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T00:29:39.950 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:29:39] ENGINE Bus STARTING 2024-02-18T00:29:39.950 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [80B blob data] 2024-02-18T00:29:39.950 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [79B blob data] 2024-02-18T00:29:40.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: CherryPy Checker: 2024-02-18T00:29:40.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: The Application mounted at '' has an empty config. 2024-02-18T00:29:40.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: 2024-02-18T00:29:40.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:29:40] ENGINE Serving on http://:::9283 2024-02-18T00:29:40.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:29:40] ENGINE Bus STARTED 2024-02-18T00:29:40.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.717+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mon.1 v2:172.21.15.137:3300/0 9 ==== mgrmap(e 19) v1 ==== 76894+0+0 (secure 0 0 0) 0x7fc3e804d8d0 con 0x7fc3f40f6990 2024-02-18T00:29:40.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.717+0000 7fc3dbfff700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc3dc06c150 0x7fc3dc06e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:40.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.717+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi137=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3f4061170 con 0x7fc3dc06c150 2024-02-18T00:29:40.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.718+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc3dc06c150 0x7fc3dc06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:40.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.718+0000 7fc3dbfff700 10 client.24460.objecter ms_handle_connect 0x7fc3dc06c150 2024-02-18T00:29:40.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.718+0000 7fc3f27fc700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc3dc06c150 0x7fc3dc06e540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc3e4009da0 tx=0x7fc3e4009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:40.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.726+0000 7fc3dbfff700 1 -- 172.21.15.137:0/1389043728 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc3f4061170 con 0x7fc3dc06c150 2024-02-18T00:29:40.728 INFO:teuthology.orchestra.run.smithi137.stdout:Scheduled grafana update... 2024-02-18T00:29:40.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 10 librados: watch_flush enter 2024-02-18T00:29:40.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 10 librados: watch_flush exit 2024-02-18T00:29:40.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 20 client.24460.objecter shutdown clearing up homeless session... 2024-02-18T00:29:40.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 10 client.24460.objecter successfully canceled tick 2024-02-18T00:29:40.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc3dc06c150 msgr2=0x7fc3dc06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:40.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 --2- 172.21.15.137:0/1389043728 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc3dc06c150 0x7fc3dc06e540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc3e4009da0 tx=0x7fc3e4009cb0).stop 2024-02-18T00:29:40.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f6990 msgr2=0x7fc3f40fa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:40.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 --2- 172.21.15.137:0/1389043728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3f40f6990 0x7fc3f40fa910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fc3e8002830 tx=0x7fc3e8009ee0).stop 2024-02-18T00:29:40.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f37fe700 1 -- 172.21.15.137:0/1389043728 reap_dead start 2024-02-18T00:29:40.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 shutdown_connections 2024-02-18T00:29:40.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 >> 172.21.15.137:0/1389043728 conn(0x7fc3f40f21b0 msgr2=0x7fc3f4060a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:40.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 shutdown_connections 2024-02-18T00:29:40.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 -- 172.21.15.137:0/1389043728 wait complete. 2024-02-18T00:29:40.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:40.728+0000 7fc3f950c700 1 librados: shutdown 2024-02-18T00:29:40.759 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.500315+0000 mon.a (mon.0) 612 : cluster [DBG] mgrmap e17: y(active, since 5m), standbys: x 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.703717+0000 mon.a (mon.0) 613 : cluster [INF] Active manager daemon y restarted 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.703998+0000 mon.a (mon.0) 614 : cluster [INF] Activating manager daemon y 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.712456+0000 mon.a (mon.0) 615 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.713586+0000 mon.a (mon.0) 616 : cluster [DBG] mgrmap e18: y(active, starting, since 0.00967678s), standbys: x 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.716214+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.716357+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:29:40.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.716473+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.717252+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.717415+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.717557+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.717780+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.717983+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.718173+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.718428+0000 mon.a (mon.0) 626 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.718726+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.718941+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.719160+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.719361+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T00:29:40.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.719507+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T00:29:40.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.719905+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T00:29:40.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:39.722357+0000 mon.a (mon.0) 633 : cluster [INF] Manager daemon y is now available 2024-02-18T00:29:40.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.824085+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:40.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.825720+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:40.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.852203+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:40.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.876931+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T00:29:40.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:39.899667+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T00:29:40.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.500315+0000 mon.a (mon.0) 612 : cluster [DBG] mgrmap e17: y(active, since 5m), standbys: x 2024-02-18T00:29:40.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.703717+0000 mon.a (mon.0) 613 : cluster [INF] Active manager daemon y restarted 2024-02-18T00:29:40.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.703998+0000 mon.a (mon.0) 614 : cluster [INF] Activating manager daemon y 2024-02-18T00:29:40.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.712456+0000 mon.a (mon.0) 615 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-18T00:29:40.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.713586+0000 mon.a (mon.0) 616 : cluster [DBG] mgrmap e18: y(active, starting, since 0.00967678s), standbys: x 2024-02-18T00:29:40.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.716214+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:29:40.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.716357+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.716473+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.717252+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.717415+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.717557+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.717780+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.717983+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.718173+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.718428+0000 mon.a (mon.0) 626 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.718726+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:29:40.804 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.500315+0000 mon.a (mon.0) 612 : cluster [DBG] mgrmap e17: y(active, since 5m), standbys: x 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.703717+0000 mon.a (mon.0) 613 : cluster [INF] Active manager daemon y restarted 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.703998+0000 mon.a (mon.0) 614 : cluster [INF] Activating manager daemon y 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.712456+0000 mon.a (mon.0) 615 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.713586+0000 mon.a (mon.0) 616 : cluster [DBG] mgrmap e18: y(active, starting, since 0.00967678s), standbys: x 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.716214+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.716357+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.716473+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.717252+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.717415+0000 mon.a (mon.0) 621 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.717557+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.717780+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.717983+0000 mon.a (mon.0) 624 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.718173+0000 mon.a (mon.0) 625 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T00:29:40.805 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.718428+0000 mon.a (mon.0) 626 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.718726+0000 mon.a (mon.0) 627 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.718941+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.719160+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.719361+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.719507+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.719905+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:39.722357+0000 mon.a (mon.0) 633 : cluster [INF] Manager daemon y is now available 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.824085+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.825720+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:40.806 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.852203+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.876931+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:39.899667+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.718941+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.719160+0000 mon.a (mon.0) 629 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.719361+0000 mon.a (mon.0) 630 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.719507+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.719905+0000 mon.a (mon.0) 632 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:39.722357+0000 mon.a (mon.0) 633 : cluster [INF] Manager daemon y is now available 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.824085+0000 mon.a (mon.0) 634 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.825720+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.852203+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:40.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.876931+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T00:29:40.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:39.899667+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T00:29:41.657 DEBUG:teuthology.orchestra.run.smithi137:grafana.a> sudo journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@grafana.a.service 2024-02-18T00:29:41.660 INFO:tasks.cephadm:Setting up client nodes... 2024-02-18T00:29:41.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-18T00:29:41.689 INFO:journalctl@ceph.grafana.a.smithi137.stdout:-- Logs begin at Sun 2024-02-18 00:11:45 UTC. -- 2024-02-18T00:29:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:40.716997+0000 mon.a (mon.0) 639 : cluster [DBG] mgrmap e19: y(active, since 1.01309s), standbys: x 2024-02-18T00:29:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:40.723265+0000 mgr.y (mgr.24454) 3 : cephadm [INF] Saving service grafana spec with placement smithi137=a;count:1 2024-02-18T00:29:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:40.723721+0000 mgr.y (mgr.24454) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:40.726680+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:42.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:41.511533+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:40.716997+0000 mon.a (mon.0) 639 : cluster [DBG] mgrmap e19: y(active, since 1.01309s), standbys: x 2024-02-18T00:29:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:40.723265+0000 mgr.y (mgr.24454) 3 : cephadm [INF] Saving service grafana spec with placement smithi137=a;count:1 2024-02-18T00:29:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:40.723721+0000 mgr.y (mgr.24454) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:40.726680+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:41.511533+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:40.716997+0000 mon.a (mon.0) 639 : cluster [DBG] mgrmap e19: y(active, since 1.01309s), standbys: x 2024-02-18T00:29:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:40.723265+0000 mgr.y (mgr.24454) 3 : cephadm [INF] Saving service grafana spec with placement smithi137=a;count:1 2024-02-18T00:29:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:40.723721+0000 mgr.y (mgr.24454) 4 : cluster [DBG] pgmap v3: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:40.726680+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:41.511533+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:41.718642+0000 mgr.y (mgr.24454) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:41.815944+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:41.816661+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:41.817225+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:41.817751+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:41.818180+0000 mgr.y (mgr.24454) 6 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 3769M 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:41.822099+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.648218+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:41.718642+0000 mgr.y (mgr.24454) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:41.815944+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:41.816661+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:41.817225+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:41.817751+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:41.818180+0000 mgr.y (mgr.24454) 6 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 3769M 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:41.822099+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.648218+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:41.718642+0000 mgr.y (mgr.24454) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:41.815944+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:41.816661+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:41.817225+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:41.817751+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:41.818180+0000 mgr.y (mgr.24454) 6 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 3769M 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:41.822099+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.648218+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:43.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 -- 172.21.15.59:0/597175370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c0f9480 msgr2=0x7f538c101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:43.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 --2- 172.21.15.59:0/597175370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c0f9480 0x7f538c101c20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f5380004870 tx=0x7f5380009cb0).stop 2024-02-18T00:29:43.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 -- 172.21.15.59:0/597175370 shutdown_connections 2024-02-18T00:29:43.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 --2- 172.21.15.59:0/597175370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f538c102110 0x7f538c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:43.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 --2- 172.21.15.59:0/597175370 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f8bb0 0x7f538c0f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 --2- 172.21.15.59:0/597175370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c0f9480 0x7f538c101c20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 -- 172.21.15.59:0/597175370 >> 172.21.15.59:0/597175370 conn(0x7f538c0f2640 msgr2=0x7f538c0f4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 -- 172.21.15.59:0/597175370 shutdown_connections 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.965+0000 7f5393a94700 1 -- 172.21.15.59:0/597175370 wait complete. 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 librados: starting msgr at 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 librados: starting objecter 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 Processor -- start 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 -- start start 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 librados: setting wanted keys 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 librados: calling monclient init 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f538c0f8bb0 0x7f538c10e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:43.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f9480 0x7f538c10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c102110 0x7f538c1083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c108890 con 0x7f538c102110 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c108a00 con 0x7f538c0f8bb0 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5393a94700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c108bc0 con 0x7f538c0f9480 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f539102f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f9480 0x7f538c10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f539102f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f9480 0x7f538c10e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:36264/0 (socket says 172.21.15.59:36264) 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f539102f700 1 -- 172.21.15.59:0/3412291858 learned_addr learned my addr 172.21.15.59:0/3412291858 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5392031700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c102110 0x7f538c1083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.966+0000 7f5391830700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f538c0f8bb0 0x7f538c10e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5392031700 1 -- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f9480 msgr2=0x7f538c10e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5392031700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f9480 0x7f538c10e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5392031700 1 -- 172.21.15.59:0/3412291858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f538c0f8bb0 msgr2=0x7f538c10e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5392031700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f538c0f8bb0 0x7f538c10e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:43.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5392031700 1 -- 172.21.15.59:0/3412291858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5380005040 con 0x7f538c102110 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f539102f700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f9480 0x7f538c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5392031700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c102110 0x7f538c1083a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f538c061fa0 tx=0x7f5384004db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f537ef78700 10 client.?.objecter ms_handle_connect 0x7f538c102110 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f537ef78700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f537ef78700 1 -- 172.21.15.59:0/3412291858 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384004020 con 0x7f538c102110 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c108ea0 con 0x7f538c102110 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f537ef78700 1 -- 172.21.15.59:0/3412291858 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f538400f040 con 0x7f538c102110 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f537ef78700 1 -- 172.21.15.59:0/3412291858 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840036c0 con 0x7f538c102110 2024-02-18T00:29:43.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5393a94700 10 client.14523.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c115030 con 0x7f538c102110 2024-02-18T00:29:43.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.967+0000 7f5393a94700 1 librados: init done 2024-02-18T00:29:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.968+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c05f860 con 0x7f538c102110 2024-02-18T00:29:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.968+0000 7f537ef78700 1 -- 172.21.15.59:0/3412291858 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5384003af0 con 0x7f538c102110 2024-02-18T00:29:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f537ef78700 1 --2- 172.21.15.59:0/3412291858 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5378062c90 0x7f5378065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f537ef78700 1 -- 172.21.15.59:0/3412291858 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f538407c380 con 0x7f538c102110 2024-02-18T00:29:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f5391830700 1 --2- 172.21.15.59:0/3412291858 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5378062c90 0x7f5378065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:43.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f537ef78700 10 client.14523.objecter ms_dispatch 0x7f538c001550 osd_map(67..67 src has 1..67) v4 2024-02-18T00:29:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f537ef78700 3 client.14523.objecter handle_osd_map got epochs [67,67] > 0 2024-02-18T00:29:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f537ef78700 3 client.14523.objecter handle_osd_map decoding full epoch 67 2024-02-18T00:29:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f537ef78700 20 client.14523.objecter dump_active .. 0 homeless 2024-02-18T00:29:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f537ef78700 10 client.14523.objecter ms_handle_connect 0x7f5378062c90 2024-02-18T00:29:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.969+0000 7f5391830700 1 --2- 172.21.15.59:0/3412291858 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5378062c90 0x7f5378065150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f53880048b0 tx=0x7f538800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:43.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:43.973+0000 7f537ef78700 1 -- 172.21.15.59:0/3412291858 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5384048f60 con 0x7f538c102110 2024-02-18T00:29:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:42.402329+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:42.402378+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:42.720064+0000 mon.a (mon.0) 648 : cluster [DBG] mgrmap e20: y(active, since 3s), standbys: x 2024-02-18T00:29:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:42.823412+0000 mon.a (mon.0) 649 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-18T00:29:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.823726+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.825667+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.935082+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.935770+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.936382+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.936962+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:42.937454+0000 mgr.y (mgr.24454) 7 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 4024M 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.943108+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.944015+0000 mon.a (mon.0) 657 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.944684+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:42.945802+0000 mgr.y (mgr.24454) 8 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.conf 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:42.946072+0000 mgr.y (mgr.24454) 9 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.conf 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:42.962030+0000 mgr.y (mgr.24454) 10 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:29:44.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:42.983740+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:43.002507+0000 mgr.y (mgr.24454) 11 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:29:44.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:43.066078+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:43.070642+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:43.081519+0000 mgr.y (mgr.24454) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi059 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:42.402329+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:42.402378+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:42.720064+0000 mon.a (mon.0) 648 : cluster [DBG] mgrmap e20: y(active, since 3s), standbys: x 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:42.823412+0000 mon.a (mon.0) 649 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.823726+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.825667+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.935082+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.935770+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.936382+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.936962+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:42.937454+0000 mgr.y (mgr.24454) 7 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 4024M 2024-02-18T00:29:44.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.943108+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.944015+0000 mon.a (mon.0) 657 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.944684+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:42.945802+0000 mgr.y (mgr.24454) 8 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.conf 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:42.946072+0000 mgr.y (mgr.24454) 9 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.conf 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:42.962030+0000 mgr.y (mgr.24454) 10 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:42.983740+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:43.002507+0000 mgr.y (mgr.24454) 11 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:29:44.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:43.066078+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:43.070642+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:43.081519+0000 mgr.y (mgr.24454) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi059 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:42.402329+0000 osd.0 (osd.0) 7 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:42.402378+0000 osd.0 (osd.0) 8 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:42.720064+0000 mon.a (mon.0) 648 : cluster [DBG] mgrmap e20: y(active, since 3s), standbys: x 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:42.823412+0000 mon.a (mon.0) 649 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.823726+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.825667+0000 mon.a (mon.0) 651 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.935082+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.935770+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.936382+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.936962+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:42.937454+0000 mgr.y (mgr.24454) 7 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 4024M 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.943108+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.944015+0000 mon.a (mon.0) 657 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.944684+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:42.945802+0000 mgr.y (mgr.24454) 8 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.conf 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:42.946072+0000 mgr.y (mgr.24454) 9 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.conf 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:42.962030+0000 mgr.y (mgr.24454) 10 : cephadm [INF] Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:42.983740+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:43.002507+0000 mgr.y (mgr.24454) 11 : cephadm [INF] Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:43.066078+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:43.070642+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:43.081519+0000 mgr.y (mgr.24454) 12 : cephadm [INF] Deploying daemon alertmanager.a on smithi059 2024-02-18T00:29:44.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.242+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f538c1070a0 con 0x7f538c102110 2024-02-18T00:29:44.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.248+0000 7f537ef78700 1 -- 172.21.15.59:0/3412291858 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5384050060 con 0x7f538c102110 2024-02-18T00:29:44.250 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-02-18T00:29:44.250 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQB4T9FlZ/93DhAAt1iULY6+R73WW4MsFRCPsA== 2024-02-18T00:29:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 10 librados: watch_flush enter 2024-02-18T00:29:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 10 librados: watch_flush exit 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 20 client.14523.objecter shutdown clearing up homeless session... 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 10 client.14523.objecter successfully canceled tick 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5378062c90 msgr2=0x7f5378065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 --2- 172.21.15.59:0/3412291858 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5378062c90 0x7f5378065150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f53880048b0 tx=0x7f538800b040).stop 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c102110 msgr2=0x7f538c1083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c102110 0x7f538c1083a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f538c061fa0 tx=0x7f5384004db0).stop 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 shutdown_connections 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 --2- 172.21.15.59:0/3412291858 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5378062c90 0x7f5378065150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f538c0f8bb0 0x7f538c10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c102110 0x7f538c1083a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:44.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 --2- 172.21.15.59:0/3412291858 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f538c0f9480 0x7f538c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.250+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 >> 172.21.15.59:0/3412291858 conn(0x7f538c0f2640 msgr2=0x7f538c0f3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.251+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 shutdown_connections 2024-02-18T00:29:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.251+0000 7f5393a94700 1 -- 172.21.15.59:0/3412291858 wait complete. 2024-02-18T00:29:44.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:44.251+0000 7f5393a94700 1 librados: shutdown 2024-02-18T00:29:45.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:43.718995+0000 mgr.y (mgr.24454) 13 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:45.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:43.825012+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T00:29:45.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:43.825086+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-18T00:29:45.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:44.242594+0000 mon.a (mon.0) 664 : audit [INF] from='client.? 172.21.15.59:0/3412291858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:45.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:44.248818+0000 mon.a (mon.0) 665 : audit [INF] from='client.? 172.21.15.59:0/3412291858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T00:29:45.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:44.611344+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-18T00:29:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:43.718995+0000 mgr.y (mgr.24454) 13 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:45.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:43.825012+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T00:29:45.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:43.825086+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-18T00:29:45.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:44.242594+0000 mon.a (mon.0) 664 : audit [INF] from='client.? 172.21.15.59:0/3412291858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:45.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:44.248818+0000 mon.a (mon.0) 665 : audit [INF] from='client.? 172.21.15.59:0/3412291858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T00:29:45.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:44.611344+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-18T00:29:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:43.718995+0000 mgr.y (mgr.24454) 13 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:43.825012+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T00:29:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:43.825086+0000 mon.a (mon.0) 663 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-18T00:29:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:44.242594+0000 mon.a (mon.0) 664 : audit [INF] from='client.? 172.21.15.59:0/3412291858' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:44.248818+0000 mon.a (mon.0) 665 : audit [INF] from='client.? 172.21.15.59:0/3412291858' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T00:29:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:44.611344+0000 mon.a (mon.0) 666 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-18T00:29:45.381 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:29:45.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-18T00:29:45.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-18T00:29:45.424 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-18T00:29:46.829 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:45.719309+0000 mgr.y (mgr.24454) 14 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:45.719309+0000 mgr.y (mgr.24454) 14 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:45.719309+0000 mgr.y (mgr.24454) 14 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:29:47.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.359+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3252595482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f8390 msgr2=0x7fbb64104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:47.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.359+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3252595482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f8390 0x7fbb64104990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fbb5c009be0 tx=0x7fbb5c004d10).stop 2024-02-18T00:29:47.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.359+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3252595482 shutdown_connections 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.359+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3252595482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640f8390 0x7fbb64104990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.359+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3252595482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640fdb20 0x7fbb640fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.359+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3252595482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640fe480 0x7fbb640f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.359+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3252595482 >> 172.21.15.137:0/3252595482 conn(0x7fbb640f3770 msgr2=0x7fbb640f5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3252595482 shutdown_connections 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3252595482 wait complete. 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 librados: starting msgr at 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 librados: starting objecter 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 Processor -- start 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 -- start start 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 librados: setting wanted keys 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.360+0000 7fbb6a03e700 1 librados: calling monclient init 2024-02-18T00:29:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8390 0x7fbb6410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640fdb20 0x7fbb6410e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640fe480 0x7fbb64107fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64108490 con 0x7fbb640fdb20 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64108600 con 0x7fbb640fe480 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb641087c0 con 0x7fbb640f8390 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640fe480 0x7fbb64107fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb637fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8390 0x7fbb6410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640fe480 0x7fbb64107fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56810/0 (socket says 172.21.15.137:56810) 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 -- 172.21.15.137:0/3367780252 learned_addr learned my addr 172.21.15.137:0/3367780252 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb62ffd700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640fdb20 0x7fbb6410e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 -- 172.21.15.137:0/3367780252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8390 msgr2=0x7fbb6410ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8390 0x7fbb6410ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 -- 172.21.15.137:0/3367780252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640fdb20 msgr2=0x7fbb6410e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:47.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640fdb20 0x7fbb6410e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 -- 172.21.15.137:0/3367780252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c0097c0 con 0x7fbb640fe480 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb63fff700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640fe480 0x7fbb64107fa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbb5c004a30 tx=0x7fbb5c004d10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb60ff9700 10 client.?.objecter ms_handle_connect 0x7fbb640fe480 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb60ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3367780252 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00e040 con 0x7fbb640fe480 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb64108a40 con 0x7fbb640fe480 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3367780252 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb5c018040 con 0x7fbb640fe480 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3367780252 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c01c3f0 con 0x7fbb640fe480 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 10 client.24484.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb64114be0 con 0x7fbb640fe480 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.361+0000 7fbb6a03e700 1 librados: init done 2024-02-18T00:29:47.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.362+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb64060b00 con 0x7fbb640fe480 2024-02-18T00:29:47.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3367780252 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fbb5c013420 con 0x7fbb640fe480 2024-02-18T00:29:47.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb60ff9700 1 --2- 172.21.15.137:0/3367780252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbb4c062c90 0x7fbb4c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:47.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3367780252 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6056+0+0 (secure 0 0 0) 0x7fbb5c07db80 con 0x7fbb640fe480 2024-02-18T00:29:47.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb60ff9700 10 client.24484.objecter ms_dispatch 0x7fbb64001550 osd_map(68..68 src has 1..68) v4 2024-02-18T00:29:47.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb60ff9700 3 client.24484.objecter handle_osd_map got epochs [68,68] > 0 2024-02-18T00:29:47.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb60ff9700 3 client.24484.objecter handle_osd_map decoding full epoch 68 2024-02-18T00:29:47.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb637fe700 1 --2- 172.21.15.137:0/3367780252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbb4c062c90 0x7fbb4c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:47.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.363+0000 7fbb60ff9700 20 client.24484.objecter dump_active .. 0 homeless 2024-02-18T00:29:47.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.364+0000 7fbb60ff9700 10 client.24484.objecter ms_handle_connect 0x7fbb4c062c90 2024-02-18T00:29:47.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.364+0000 7fbb637fe700 1 --2- 172.21.15.137:0/3367780252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbb4c062c90 0x7fbb4c065150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fbb54004de0 tx=0x7fbb5400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:47.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.368+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3367780252 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb5c051910 con 0x7fbb640fe480 2024-02-18T00:29:47.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.590+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fbb64107080 con 0x7fbb640fe480 2024-02-18T00:29:47.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.594+0000 7fbb60ff9700 1 -- 172.21.15.137:0/3367780252 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fbb5c021070 con 0x7fbb640fe480 2024-02-18T00:29:47.596 INFO:teuthology.orchestra.run.smithi137.stdout:[client.1] 2024-02-18T00:29:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB7T9Fllg9HIxAADtDQ7laB1kcCC2xXAZPfHA== 2024-02-18T00:29:47.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 10 librados: watch_flush enter 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 10 librados: watch_flush exit 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 20 client.24484.objecter shutdown clearing up homeless session... 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 10 client.24484.objecter successfully canceled tick 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbb4c062c90 msgr2=0x7fbb4c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3367780252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbb4c062c90 0x7fbb4c065150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fbb54004de0 tx=0x7fbb5400b040).stop 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640fe480 msgr2=0x7fbb64107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640fe480 0x7fbb64107fa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fbb5c004a30 tx=0x7fbb5c004d10).stop 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 shutdown_connections 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3367780252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbb4c062c90 0x7fbb4c065150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb640fe480 0x7fbb64107fa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb640f8390 0x7fbb6410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 --2- 172.21.15.137:0/3367780252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb640fdb20 0x7fbb6410e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:47.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 >> 172.21.15.137:0/3367780252 conn(0x7fbb640f3770 msgr2=0x7fbb640f4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:47.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 shutdown_connections 2024-02-18T00:29:47.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 -- 172.21.15.137:0/3367780252 wait complete. 2024-02-18T00:29:47.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-02-18T00:29:47.596+0000 7fbb6a03e700 1 librados: shutdown 2024-02-18T00:29:47.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:47.591420+0000 mon.b (mon.1) 29 : audit [INF] from='client.? 172.21.15.137:0/3367780252' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:47.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:47.591715+0000 mon.a (mon.0) 667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:47.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:47.594884+0000 mon.a (mon.0) 668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T00:29:48.183 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-02-18T00:29:48.183 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-18T00:29:48.183 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-18T00:29:48.224 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T00:29:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:47.591420+0000 mon.b (mon.1) 29 : audit [INF] from='client.? 172.21.15.137:0/3367780252' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:47.591715+0000 mon.a (mon.0) 667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:47.594884+0000 mon.a (mon.0) 668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T00:29:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:47.591420+0000 mon.b (mon.1) 29 : audit [INF] from='client.? 172.21.15.137:0/3367780252' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:47.591715+0000 mon.a (mon.0) 667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T00:29:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:47.594884+0000 mon.a (mon.0) 668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T00:29:49.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:47.387340+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:49.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:47.387389+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:49.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:47.719981+0000 mgr.y (mgr.24454) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:29:49.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:47.831594+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-18T00:29:49.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:47.387340+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:49.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:47.387389+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:49.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:47.719981+0000 mgr.y (mgr.24454) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:29:49.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:47.831594+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-18T00:29:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:47.387340+0000 osd.3 (osd.3) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:47.387389+0000 osd.3 (osd.3) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:47.719981+0000 mgr.y (mgr.24454) 15 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:29:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:47.831594+0000 mon.a (mon.0) 669 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-18T00:29:49.850 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:29:50.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:49.613508+0000 mon.a (mon.0) 670 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-18T00:29:50.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:49.720370+0000 mgr.y (mgr.24454) 16 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 0 op/s 2024-02-18T00:29:50.879 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:49.613508+0000 mon.a (mon.0) 670 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-18T00:29:50.879 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:49.720370+0000 mgr.y (mgr.24454) 16 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 0 op/s 2024-02-18T00:29:51.004 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:29:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:29:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:49.613508+0000 mon.a (mon.0) 670 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-18T00:29:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:49.720370+0000 mgr.y (mgr.24454) 16 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 0 op/s 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:50.470954+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:50.471042+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:50.617457+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:50.730483+0000 mgr.y (mgr.24454) 17 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:50.470954+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:50.471042+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:50.617457+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-18T00:29:51.893 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:50.730483+0000 mgr.y (mgr.24454) 17 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:29:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:50.470954+0000 osd.6 (osd.6) 3 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:29:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:50.471042+0000 osd.6 (osd.6) 4 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:29:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:50.617457+0000 mon.a (mon.0) 671 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-18T00:29:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:50.730483+0000 mgr.y (mgr.24454) 17 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:29:52.162 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:52 smithi059 systemd[1]: Starting Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:29:52.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:51.720998+0000 mgr.y (mgr.24454) 18 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:29:52.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:51.720998+0000 mgr.y (mgr.24454) 18 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:29:53.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:51.720998+0000 mgr.y (mgr.24454) 18 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 -- 172.21.15.59:0/1973492210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 msgr2=0x7f39bc0f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 --2- 172.21.15.59:0/1973492210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 0x7f39bc0f7d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f39a4004870 tx=0x7f39a4009cb0).stop 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 -- 172.21.15.59:0/1973492210 shutdown_connections 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 --2- 172.21.15.59:0/1973492210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39bc0f9ab0 0x7f39bc0f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 --2- 172.21.15.59:0/1973492210 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f8440 0x7f39bc104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 --2- 172.21.15.59:0/1973492210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 0x7f39bc0f7d60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 -- 172.21.15.59:0/1973492210 >> 172.21.15.59:0/1973492210 conn(0x7f39bc0f3800 msgr2=0x7f39bc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 -- 172.21.15.59:0/1973492210 shutdown_connections 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.123+0000 7f39c1b40700 1 -- 172.21.15.59:0/1973492210 wait complete. 2024-02-18T00:29:53.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.124+0000 7f39c1b40700 1 librados: starting msgr at 2024-02-18T00:29:53.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.124+0000 7f39c1b40700 1 librados: starting objecter 2024-02-18T00:29:53.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.124+0000 7f39c1b40700 1 Processor -- start 2024-02-18T00:29:53.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 -- start start 2024-02-18T00:29:53.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 librados: setting wanted keys 2024-02-18T00:29:53.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 librados: calling monclient init 2024-02-18T00:29:53.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39bc0f8440 0x7f39bc06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:53.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f9ab0 0x7f39bc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:53.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 0x7f39bc06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:53.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc116570 con 0x7f39bc0fa380 2024-02-18T00:29:53.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1166e0 con 0x7f39bc0f8440 2024-02-18T00:29:53.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39c1b40700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1168a0 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f9ab0 0x7f39bc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f9ab0 0x7f39bc06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:48632/0 (socket says 172.21.15.59:48632) 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 -- 172.21.15.59:0/4074207956 learned_addr learned my addr 172.21.15.59:0/4074207956 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39bb7fe700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39bc0f8440 0x7f39bc06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39bbfff700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 0x7f39bc06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 -- 172.21.15.59:0/4074207956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39bc0f8440 msgr2=0x7f39bc06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39bc0f8440 0x7f39bc06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 -- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 msgr2=0x7f39bc06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 0x7f39bc06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39baffd700 1 -- 172.21.15.59:0/4074207956 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a4005040 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.125+0000 7f39bbfff700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 0x7f39bc06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.126+0000 7f39baffd700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f9ab0 0x7f39bc06b5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f39a4009eb0 tx=0x7f39a400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.126+0000 7f39b8ff9700 10 client.?.objecter ms_handle_connect 0x7f39bc0f9ab0 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.126+0000 7f39b8ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.126+0000 7f39b8ff9700 1 -- 172.21.15.59:0/4074207956 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a400e070 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.126+0000 7f39b8ff9700 1 -- 172.21.15.59:0/4074207956 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a4013da0 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.126+0000 7f39b8ff9700 1 -- 172.21.15.59:0/4074207956 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4019070 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.127+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc116b20 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.127+0000 7f39c1b40700 10 client.24440.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:53.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.127+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc116fb0 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.127+0000 7f39c1b40700 1 librados: init done 2024-02-18T00:29:53.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.128+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc05ee80 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39b8ff9700 1 -- 172.21.15.59:0/4074207956 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f39a401e420 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39b8ff9700 1 --2- 172.21.15.59:0/4074207956 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39a8062c90 0x7f39a8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39b8ff9700 1 -- 172.21.15.59:0/4074207956 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f39a407d830 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39b8ff9700 10 client.24440.objecter ms_dispatch 0x7f39bc001550 osd_map(71..71 src has 1..71) v4 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39b8ff9700 3 client.24440.objecter handle_osd_map got epochs [71,71] > 0 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39b8ff9700 3 client.24440.objecter handle_osd_map decoding full epoch 71 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39bb7fe700 1 --2- 172.21.15.59:0/4074207956 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39a8062c90 0x7f39a8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.129+0000 7f39b8ff9700 20 client.24440.objecter dump_active .. 0 homeless 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.130+0000 7f39b8ff9700 10 client.24440.objecter ms_handle_connect 0x7f39a8062c90 2024-02-18T00:29:53.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.130+0000 7f39bb7fe700 1 --2- 172.21.15.59:0/4074207956 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39a8062c90 0x7f39a8065150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f39ac004dc0 tx=0x7f39ac00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:53.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.133+0000 7f39b8ff9700 1 -- 172.21.15.59:0/4074207956 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f39a40515b0 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.308+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39bc107190 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.309+0000 7f39b8ff9700 1 -- 172.21.15.59:0/4074207956 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+19003 (secure 0 0 0) 0x7f39a4023020 con 0x7f39bc0f9ab0 2024-02-18T00:29:53.312 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:29:53.312 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":71,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","created":"2024-02-18T00:23:11.390869+0000","modified":"2024-02-18T00:29:50.609439+0000","last_up_change":"2024-02-18T00:29:06.503960+0000","last_in_change":"2024-02-18T00:28:48.970063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T00:27:02.909667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-18T00:29:13.697458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-18T00:29:14.132992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T00:29:15.191206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T00:29:17.526023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T00:29:19.600478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"6e729102-80e2-4a4e-92ca-600ee8b1ed85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":3700238977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3700238977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3700238977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6803","nonce":3700238977}]},"public_addr":"172.21.15.59:6801/3700238977","cluster_addr":"172.21.15.59:6802/3700238977","heartbeat_back_addr":"172.21.15.59:6804/3700238977","heartbeat_front_addr":"172.21.15.59:6803/3700238977","state":["exists","up"]},{"osd":1,"uuid":"f67a7206-6d31-497f-97e0-672d134b13d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":26279110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":26279110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":26279110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6807","nonce":26279110}]},"public_addr":"172.21.15.59:6805/26279110","cluster_addr":"172.21.15.59:6806/26279110","heartbeat_back_addr":"172.21.15.59:6808/26279110","heartbeat_front_addr":"172.21.15.59:6807/26279110","state":["exists","up"]},{"osd":2,"uuid":"d6223dbd-e539-4770-9e5a-7b756304f8f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1153193637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1153193637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1153193637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6811","nonce":1153193637}]},"public_addr":"172.21.15.59:6809/1153193637","cluster_addr":"172.21.15.59:6810/1153193637","heartbeat_back_addr":"172.21.15.59:6812/1153193637","heartbeat_front_addr":"172.21.15.59:6811/1153193637","state":["exists","up"]},{"osd":3,"uuid":"fff275df-95e0-41f6-96c3-e531f05de32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1805700964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1805700964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1805700964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6815","nonce":1805700964}]},"public_addr":"172.21.15.59:6813/1805700964","cluster_addr":"172.21.15.59:6814/1805700964","heartbeat_back_addr":"172.21.15.59:6816/1805700964","heartbeat_front_addr":"172.21.15.59:6815/1805700964","state":["exists","up"]},{"osd":4,"uuid":"a0bae50b-4f29-4310-ac5f-bc7285464231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2044387056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":2044387056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":2044387056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2044387056}]},"public_addr":"172.21.15.137:6800/2044387056","cluster_addr":"172.21.15.137:6801/2044387056","heartbeat_back_addr":"172.21.15.137:6803/2044387056","heartbeat_front_addr":"172.21.15.137:6802/2044387056","state":["exists","up"]},{"osd":5,"uuid":"513a4192-45fb-4444-a1f9-4bd2f2a5fa01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2170432263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":2170432263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":2170432263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2170432263}]},"public_addr":"172.21.15.137:6804/2170432263","cluster_addr":"172.21.15.137:6805/2170432263","heartbeat_back_addr":"172.21.15.137:6807/2170432263","heartbeat_front_addr":"172.21.15.137:6806/2170432263","state":["exists","up"]},{"osd":6,"uuid":"5bbb7683-ea5a-490c-8c8b-67b53c45406f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3356009390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3356009390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6811","nonce":3356009390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3356009390}]},"public_addr":"172.21.15.137:6808/3356009390","cluster_addr":"172.21.15.137:6809/3356009390","heartbeat_back_addr":"172.21.15.137:6811/3356009390","heartbeat_front_addr":"172.21.15.137:6810/3356009390","state":["exists","up"]},{"osd":7,"uuid":"951d317b-7a34-41a0-b1a1-a2851b2070bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":53025171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":53025171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6815","nonce":53025171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":53025171}]},"public_addr":"172.21.15.137:6812/53025171","cluster_addr":"172.21.15.137:6813/53025171","heartbeat_back_addr":"172.21.15.137:6815/53025171","heartbeat_front_addr":"172.21.15.137:6814/53025171","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:42.402380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:16.051406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:06.168044+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:47.387404+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:12.724408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:17.639367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:50.471046+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:04.632761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1243102711":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/817089002":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/3174805070":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/3506000666":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/4167740619":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/999852549":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/3575161033":"2024-02-19T00:23:45.307159+0000","172.21.15.59:0/2403979355":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/642821245":"2024-02-19T00:23:45.307159+0000","172.21.15.59:6800/1656814063":"2024-02-19T00:23:45.307159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T00:29:53.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 10 librados: watch_flush enter 2024-02-18T00:29:53.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 10 librados: watch_flush exit 2024-02-18T00:29:53.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 20 client.24440.objecter shutdown clearing up homeless session... 2024-02-18T00:29:53.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 10 client.24440.objecter successfully canceled tick 2024-02-18T00:29:53.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39a8062c90 msgr2=0x7f39a8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:53.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 --2- 172.21.15.59:0/4074207956 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39a8062c90 0x7f39a8065150 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f39ac004dc0 tx=0x7f39ac00b040).stop 2024-02-18T00:29:53.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f9ab0 msgr2=0x7f39bc06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:53.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f9ab0 0x7f39bc06b5c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f39a4009eb0 tx=0x7f39a400b040).stop 2024-02-18T00:29:53.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 shutdown_connections 2024-02-18T00:29:53.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 --2- 172.21.15.59:0/4074207956 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39a8062c90 0x7f39a8065150 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39bc0f8440 0x7f39bc06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39bc0fa380 0x7f39bc06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 --2- 172.21.15.59:0/4074207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39bc0f9ab0 0x7f39bc06b5c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:53.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 >> 172.21.15.59:0/4074207956 conn(0x7f39bc0f3800 msgr2=0x7f39bc0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:53.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 shutdown_connections 2024-02-18T00:29:53.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 -- 172.21.15.59:0/4074207956 wait complete. 2024-02-18T00:29:53.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:53.311+0000 7f39c1b40700 1 librados: shutdown 2024-02-18T00:29:53.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:53.310007+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.59:0/4074207956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:29:53.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:53.310007+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.59:0/4074207956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:29:54.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:53.310007+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.59:0/4074207956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:53.991Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:53.991Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:53.993Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.59 port=9094 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:53.994Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:54.020Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:54.020Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:54.022Z caller=main.go:518 msg=Listening address=:9093 2024-02-18T00:29:54.252 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:54.023Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-18T00:29:54.445 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-18T00:27:02.909667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-02-18T00:29:13.697458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'iscsi', 'create_time': '2024-02-18T00:29:14.132992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-18T00:29:15.191206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-18T00:29:17.526023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-18T00:29:19.600478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}] 2024-02-18T00:29:54.446 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-18T00:29:54.752 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:54 smithi059 bash[67252]: e12473b0feb1805468a509c1a8ce79ebff0634d3f79955862dedc8b15b088340 2024-02-18T00:29:54.752 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:54 smithi059 systemd[1]: Started Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:29:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:53.721427+0000 mgr.y (mgr.24454) 19 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:29:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:54.615465+0000 mon.a (mon.0) 672 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-18T00:29:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:53.721427+0000 mgr.y (mgr.24454) 19 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:29:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:54.615465+0000 mon.a (mon.0) 672 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-18T00:29:54.793 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:29:55.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:53.721427+0000 mgr.y (mgr.24454) 19 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:29:55.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:54.615465+0000 mon.a (mon.0) 672 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-18T00:29:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:54.674751+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:29:54.677371+0000 mgr.y (mgr.24454) 20 : cephadm [INF] Deploying daemon node-exporter.a on smithi059 2024-02-18T00:29:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:54.871373+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:54.674751+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:29:54.677371+0000 mgr.y (mgr.24454) 20 : cephadm [INF] Deploying daemon node-exporter.a on smithi059 2024-02-18T00:29:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:54.871373+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:56.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:54.674751+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:56.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:29:54.677371+0000 mgr.y (mgr.24454) 20 : cephadm [INF] Deploying daemon node-exporter.a on smithi059 2024-02-18T00:29:56.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:54.871373+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:29:56.173 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:29:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:29:55.994Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000090647s 2024-02-18T00:29:56.174 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:55 smithi059 systemd[1]: Starting Ceph node-exporter.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:29:56.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:55.721917+0000 mgr.y (mgr.24454) 21 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-02-18T00:29:56.821 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:55.721917+0000 mgr.y (mgr.24454) 21 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-02-18T00:29:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:55.721917+0000 mgr.y (mgr.24454) 21 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-02-18T00:29:57.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 -- 172.21.15.59:0/2482306259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f9610 msgr2=0x7fb308101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 --2- 172.21.15.59:0/2482306259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f9610 0x7fb308101d20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fb2f0004870 tx=0x7fb2f0009cb0).stop 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 -- 172.21.15.59:0/2482306259 shutdown_connections 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 --2- 172.21.15.59:0/2482306259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb308102210 0x7fb308104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 --2- 172.21.15.59:0/2482306259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3080f8cb0 0x7fb3080f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 --2- 172.21.15.59:0/2482306259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f9610 0x7fb308101d20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 -- 172.21.15.59:0/2482306259 >> 172.21.15.59:0/2482306259 conn(0x7fb3080f2820 msgr2=0x7fb3080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 -- 172.21.15.59:0/2482306259 shutdown_connections 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 -- 172.21.15.59:0/2482306259 wait complete. 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 librados: starting msgr at 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 librados: starting objecter 2024-02-18T00:29:57.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 Processor -- start 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 -- start start 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 librados: setting wanted keys 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.080+0000 7fb30e993700 1 librados: calling monclient init 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30e993700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f8cb0 0x7fb3080f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30e993700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3080f9610 0x7fb3080f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30e993700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb308102210 0x7fb3080f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30e993700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081166a0 con 0x7fb3080f8cb0 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30e993700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb308116810 con 0x7fb308102210 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30e993700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081169d0 con 0x7fb3080f9610 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f8cb0 0x7fb3080f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb3077fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3080f9610 0x7fb3080f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f8cb0 0x7fb3080f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59064/0 (socket says 172.21.15.59:59064) 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 -- 172.21.15.59:0/755112447 learned_addr learned my addr 172.21.15.59:0/755112447 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:29:57.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30cf30700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb308102210 0x7fb3080f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 -- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3080f9610 msgr2=0x7fb3080f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3080f9610 0x7fb3080f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 -- 172.21.15.59:0/755112447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb308102210 msgr2=0x7fb3080f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb308102210 0x7fb3080f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 -- 172.21.15.59:0/755112447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0005040 con 0x7fb3080f8cb0 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb3077fe700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3080f9610 0x7fb3080f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb307fff700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f8cb0 0x7fb3080f6c60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb2f800e970 tx=0x7fb2f800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb3057fa700 10 client.?.objecter ms_handle_connect 0x7fb3080f8cb0 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb3057fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb3057fa700 1 -- 172.21.15.59:0/755112447 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f800ee60 con 0x7fb3080f8cb0 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb3057fa700 1 -- 172.21.15.59:0/755112447 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f800f040 con 0x7fb3080f8cb0 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb308116cb0 con 0x7fb3080f8cb0 2024-02-18T00:29:57.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.081+0000 7fb3057fa700 1 -- 172.21.15.59:0/755112447 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8007ea0 con 0x7fb3080f8cb0 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.082+0000 7fb30e993700 10 client.14541.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.082+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3081171a0 con 0x7fb3080f8cb0 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.082+0000 7fb30e993700 1 librados: init done 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.082+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30805fa60 con 0x7fb3080f8cb0 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3057fa700 1 -- 172.21.15.59:0/755112447 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb2f800cb10 con 0x7fb3080f8cb0 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3057fa700 1 --2- 172.21.15.59:0/755112447 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2f4062c90 0x7fb2f4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3057fa700 1 -- 172.21.15.59:0/755112447 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6056+0+0 (secure 0 0 0) 0x7fb2f807c5f0 con 0x7fb3080f8cb0 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3057fa700 10 client.14541.objecter ms_dispatch 0x7fb308001550 osd_map(72..72 src has 1..72) v4 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3057fa700 3 client.14541.objecter handle_osd_map got epochs [72,72] > 0 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3057fa700 3 client.14541.objecter handle_osd_map decoding full epoch 72 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3077fe700 1 --2- 172.21.15.59:0/755112447 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2f4062c90 0x7fb2f4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.083+0000 7fb3057fa700 20 client.14541.objecter dump_active .. 0 homeless 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.084+0000 7fb3057fa700 10 client.14541.objecter ms_handle_connect 0x7fb2f4062c90 2024-02-18T00:29:57.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.084+0000 7fb3077fe700 1 --2- 172.21.15.59:0/755112447 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2f4062c90 0x7fb2f4065150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb2f0004ee0 tx=0x7fb2f000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:29:57.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.088+0000 7fb3057fa700 1 -- 172.21.15.59:0/755112447 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb2f8050380 con 0x7fb3080f8cb0 2024-02-18T00:29:57.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.267+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb3080f8140 con 0x7fb3080f8cb0 2024-02-18T00:29:57.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.268+0000 7fb3057fa700 1 -- 172.21.15.59:0/755112447 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v72) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb2f8019020 con 0x7fb3080f8cb0 2024-02-18T00:29:57.270 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-02-18T00:29:57.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 10 librados: watch_flush enter 2024-02-18T00:29:57.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 10 librados: watch_flush exit 2024-02-18T00:29:57.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 20 client.14541.objecter shutdown clearing up homeless session... 2024-02-18T00:29:57.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 10 client.14541.objecter successfully canceled tick 2024-02-18T00:29:57.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2f4062c90 msgr2=0x7fb2f4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 --2- 172.21.15.59:0/755112447 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2f4062c90 0x7fb2f4065150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fb2f0004ee0 tx=0x7fb2f000b040).stop 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f8cb0 msgr2=0x7fb3080f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f8cb0 0x7fb3080f6c60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fb2f800e970 tx=0x7fb2f800b040).stop 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 shutdown_connections 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 --2- 172.21.15.59:0/755112447 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2f4062c90 0x7fb2f4065150 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb308102210 0x7fb3080f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3080f8cb0 0x7fb3080f6c60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 --2- 172.21.15.59:0/755112447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3080f9610 0x7fb3080f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 >> 172.21.15.59:0/755112447 conn(0x7fb3080f2820 msgr2=0x7fb3080f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 shutdown_connections 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 -- 172.21.15.59:0/755112447 wait complete. 2024-02-18T00:29:57.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:29:57.270+0000 7fb30e993700 1 librados: shutdown 2024-02-18T00:29:57.656 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:57 smithi059 bash[67705]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-02-18T00:29:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:29:57.269402+0000 mon.a (mon.0) 675 : audit [DBG] from='client.? 172.21.15.59:0/755112447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-18T00:29:58.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:29:57.269402+0000 mon.a (mon.0) 675 : audit [DBG] from='client.? 172.21.15.59:0/755112447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-18T00:29:58.002 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:57 smithi059 bash[67705]: Getting image source signatures 2024-02-18T00:29:58.002 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:57 smithi059 bash[67705]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-02-18T00:29:58.002 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:57 smithi059 bash[67705]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-02-18T00:29:58.002 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:57 smithi059 bash[67705]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-02-18T00:29:58.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:29:57.269402+0000 mon.a (mon.0) 675 : audit [DBG] from='client.? 172.21.15.59:0/755112447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-18T00:29:58.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool get .rgw.root pg_num 2024-02-18T00:29:58.604 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:29:58.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:29:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:57.722382+0000 mgr.y (mgr.24454) 22 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:29:58.988 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:58 smithi059 bash[67705]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-02-18T00:29:58.988 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:29:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:57.722382+0000 mgr.y (mgr.24454) 22 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:29:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:29:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:57.722382+0000 mgr.y (mgr.24454) 22 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:29:59.252 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:29:59 smithi059 bash[67705]: Writing manifest to image destination 2024-02-18T00:30:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:29:59.722692+0000 mgr.y (mgr.24454) 23 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:30:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:00.000134+0000 mon.a (mon.0) 676 : cluster [INF] overall HEALTH_OK 2024-02-18T00:30:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:29:59.722692+0000 mgr.y (mgr.24454) 23 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:30:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:00.000134+0000 mon.a (mon.0) 676 : cluster [INF] overall HEALTH_OK 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.776Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.777Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.777Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.777Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=arp 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-18T00:30:01.003 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=edac 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-18T00:30:01.004 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=os 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-18T00:30:01.005 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=stat 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=time 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=uname 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-18T00:30:01.006 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-18T00:30:01.007 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.778Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-18T00:30:01.007 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a[67906]: ts=2024-02-18T00:30:00.779Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-18T00:30:01.007 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 bash[67705]: c0d4f8f718a54925d09ca2d747fd3acfd0e8faa77fc061244e402780ff9fa919 2024-02-18T00:30:01.007 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:30:00 smithi059 systemd[1]: Started Ceph node-exporter.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:30:01.007 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:30:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:30:01.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:29:59.722692+0000 mgr.y (mgr.24454) 23 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:30:01.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:00.000134+0000 mon.a (mon.0) 676 : cluster [INF] overall HEALTH_OK 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 -- 172.21.15.59:0/1274218866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 msgr2=0x7f109c104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 --2- 172.21.15.59:0/1274218866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c104190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f1098004870 tx=0x7f1098009cb0).stop 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 -- 172.21.15.59:0/1274218866 shutdown_connections 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 --2- 172.21.15.59:0/1274218866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f109c0f6b20 0x7f109c0f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 --2- 172.21.15.59:0/1274218866 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 0x7f109c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 --2- 172.21.15.59:0/1274218866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c104190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 -- 172.21.15.59:0/1274218866 >> 172.21.15.59:0/1274218866 conn(0x7f109c0f2820 msgr2=0x7f109c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 -- 172.21.15.59:0/1274218866 shutdown_connections 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.084+0000 7f10a3373700 1 -- 172.21.15.59:0/1274218866 wait complete. 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 librados: starting msgr at 2024-02-18T00:30:01.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 librados: starting objecter 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 Processor -- start 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 -- start start 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 librados: setting wanted keys 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 librados: calling monclient init 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f109c0f6b20 0x7f109c110330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c110820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 0x7f109c10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c10a9c0 con 0x7f109c0f73f0 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c10ab30 con 0x7f109c0f6b20 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a3373700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c10acf0 con 0x7f109c104680 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 0x7f109c10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a090e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c110820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 0x7f109c10a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44128/0 (socket says 172.21.15.59:44128) 2024-02-18T00:30:01.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 -- 172.21.15.59:0/2603706584 learned_addr learned my addr 172.21.15.59:0/2603706584 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a090e700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c110820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58684/0 (socket says 172.21.15.59:58684) 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a110f700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f109c0f6b20 0x7f109c110330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 -- 172.21.15.59:0/2603706584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f109c0f6b20 msgr2=0x7f109c110330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f109c0f6b20 0x7f109c110330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 -- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 msgr2=0x7f109c110820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c110820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a1910700 1 -- 172.21.15.59:0/2603706584 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098005040 con 0x7f109c104680 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.085+0000 7f10a090e700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10a1910700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 0x7f109c10a4d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f109400ec20 tx=0x7f109400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10927fc700 10 client.?.objecter ms_handle_connect 0x7f109c104680 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10927fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10927fc700 1 -- 172.21.15.59:0/2603706584 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094009950 con 0x7f109c104680 2024-02-18T00:30:01.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10927fc700 1 -- 172.21.15.59:0/2603706584 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f109400f040 con 0x7f109c104680 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c10af70 con 0x7f109c104680 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10927fc700 1 -- 172.21.15.59:0/2603706584 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094007df0 con 0x7f109c104680 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10a3373700 10 client.24452.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c117160 con 0x7f109c104680 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.086+0000 7f10a3373700 1 librados: init done 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.087+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c05fa60 con 0x7f109c104680 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.087+0000 7f10927fc700 1 -- 172.21.15.59:0/2603706584 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f10940092e0 con 0x7f109c104680 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.087+0000 7f10927fc700 1 --2- 172.21.15.59:0/2603706584 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1088062c90 0x7f1088065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10a110f700 1 --2- 172.21.15.59:0/2603706584 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1088062c90 0x7f1088065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:01.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10927fc700 1 -- 172.21.15.59:0/2603706584 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f109407c680 con 0x7f109c104680 2024-02-18T00:30:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10927fc700 10 client.24452.objecter ms_dispatch 0x7f109c001550 osd_map(72..72 src has 1..72) v4 2024-02-18T00:30:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10927fc700 3 client.24452.objecter handle_osd_map got epochs [72,72] > 0 2024-02-18T00:30:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10927fc700 3 client.24452.objecter handle_osd_map decoding full epoch 72 2024-02-18T00:30:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10927fc700 20 client.24452.objecter dump_active .. 0 homeless 2024-02-18T00:30:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10927fc700 10 client.24452.objecter ms_handle_connect 0x7f1088062c90 2024-02-18T00:30:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.088+0000 7f10a110f700 1 --2- 172.21.15.59:0/2603706584 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1088062c90 0x7f1088065150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f108c004dc0 tx=0x7f108c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:01.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.092+0000 7f10927fc700 1 -- 172.21.15.59:0/2603706584 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1094049f60 con 0x7f109c104680 2024-02-18T00:30:01.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.265+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"} v 0) v1 -- 0x7f109c1093f0 con 0x7f109c104680 2024-02-18T00:30:01.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.265+0000 7f10927fc700 1 -- 172.21.15.59:0/2603706584 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]=0 v72) v1 ==== 98+0+11 (secure 0 0 0) 0x7f109407ae60 con 0x7f109c104680 2024-02-18T00:30:01.268 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-02-18T00:30:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 10 librados: watch_flush enter 2024-02-18T00:30:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 10 librados: watch_flush exit 2024-02-18T00:30:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 20 client.24452.objecter shutdown clearing up homeless session... 2024-02-18T00:30:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 10 client.24452.objecter successfully canceled tick 2024-02-18T00:30:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1088062c90 msgr2=0x7f1088065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 --2- 172.21.15.59:0/2603706584 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1088062c90 0x7f1088065150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f108c004dc0 tx=0x7f108c00b040).stop 2024-02-18T00:30:01.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 msgr2=0x7f109c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:01.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 0x7f109c10a4d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f109400ec20 tx=0x7f109400b040).stop 2024-02-18T00:30:01.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 shutdown_connections 2024-02-18T00:30:01.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 --2- 172.21.15.59:0/2603706584 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1088062c90 0x7f1088065150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f109c0f6b20 0x7f109c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f109c104680 0x7f109c10a4d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 --2- 172.21.15.59:0/2603706584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f109c0f73f0 0x7f109c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:01.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 >> 172.21.15.59:0/2603706584 conn(0x7f109c0f2820 msgr2=0x7f109c0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:01.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 shutdown_connections 2024-02-18T00:30:01.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 -- 172.21.15.59:0/2603706584 wait complete. 2024-02-18T00:30:01.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:01.267+0000 7f10a3373700 1 librados: shutdown 2024-02-18T00:30:01.754 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:01 smithi137 systemd[1]: Starting Ceph node-exporter.b for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:30:01.754 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:01 smithi137 bash[62618]: Trying to pull quay.io/prometheus/node-exporter:v1.3.1... 2024-02-18T00:30:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:00.739224+0000 mgr.y (mgr.24454) 24 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:00.914214+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:02.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:30:00.915118+0000 mgr.y (mgr.24454) 25 : cephadm [INF] Deploying daemon node-exporter.b on smithi137 2024-02-18T00:30:02.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:01.266764+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.59:0/2603706584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-18T00:30:02.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:00.739224+0000 mgr.y (mgr.24454) 24 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:02.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:00.914214+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:02.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:30:00.915118+0000 mgr.y (mgr.24454) 25 : cephadm [INF] Deploying daemon node-exporter.b on smithi137 2024-02-18T00:30:02.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:01.266764+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.59:0/2603706584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-18T00:30:02.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool get iscsi pg_num 2024-02-18T00:30:02.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:00.739224+0000 mgr.y (mgr.24454) 24 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:02.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:00.914214+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:02.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:30:00.915118+0000 mgr.y (mgr.24454) 25 : cephadm [INF] Deploying daemon node-exporter.b on smithi137 2024-02-18T00:30:02.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:01.266764+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.59:0/2603706584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-18T00:30:02.341 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:02.503 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:02 smithi137 bash[62618]: Getting image source signatures 2024-02-18T00:30:02.504 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:02 smithi137 bash[62618]: Copying blob sha256:b5db1e299295edf3005515ab7879c1df64a33c185d3a7a23aa4dcaa17d26f7b3 2024-02-18T00:30:02.504 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:02 smithi137 bash[62618]: Copying blob sha256:aa2a8d90b84cb2a9c422e7005cd166a008ccf22ef5d7d4f07128478585ce35ea 2024-02-18T00:30:02.504 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:02 smithi137 bash[62618]: Copying blob sha256:b45d31ee2d7f9f452678a85b0c837c29e12089f31ee8dbac6c8c24dfa4054a30 2024-02-18T00:30:03.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:01.723319+0000 mgr.y (mgr.24454) 26 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:30:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:01.723319+0000 mgr.y (mgr.24454) 26 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:30:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:01.723319+0000 mgr.y (mgr.24454) 26 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:30:04.149 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:30:03.998Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003767962s 2024-02-18T00:30:04.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 -- 172.21.15.59:0/3373075570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 msgr2=0x7f5a500fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:04.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 --2- 172.21.15.59:0/3373075570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 0x7f5a500fcf90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f5a440047b0 tx=0x7f5a44009cb0).stop 2024-02-18T00:30:04.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 -- 172.21.15.59:0/3373075570 shutdown_connections 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 --2- 172.21.15.59:0/3373075570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 0x7f5a50103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 --2- 172.21.15.59:0/3373075570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 0x7f5a500fcf90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 --2- 172.21.15.59:0/3373075570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a500fd510 0x7f5a500f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 -- 172.21.15.59:0/3373075570 >> 172.21.15.59:0/3373075570 conn(0x7f5a500f2820 msgr2=0x7f5a500f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 -- 172.21.15.59:0/3373075570 shutdown_connections 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 -- 172.21.15.59:0/3373075570 wait complete. 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 librados: starting msgr at 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.430+0000 7f5a56e79700 1 librados: starting objecter 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 Processor -- start 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 -- start start 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 librados: setting wanted keys 2024-02-18T00:30:04.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 librados: calling monclient init 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 0x7f5a501103b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 0x7f5a501108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a500fd510 0x7f5a5010a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5010aa40 con 0x7f5a500f7400 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5010abb0 con 0x7f5a500fd510 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a56e79700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5010ad70 con 0x7f5a500fcbb0 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 0x7f5a501103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 0x7f5a501103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58708/0 (socket says 172.21.15.59:58708) 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 -- 172.21.15.59:0/1125207399 learned_addr learned my addr 172.21.15.59:0/1125207399 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a4ffff700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 0x7f5a501108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a55416700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a500fd510 0x7f5a5010a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 -- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 msgr2=0x7f5a501108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 0x7f5a501108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 -- 172.21.15.59:0/1125207399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a500fd510 msgr2=0x7f5a5010a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a500fd510 0x7f5a5010a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a54c15700 1 -- 172.21.15.59:0/1125207399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480097c0 con 0x7f5a500f7400 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a4ffff700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 0x7f5a501108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.431+0000 7f5a55416700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a500fd510 0x7f5a5010a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a54c15700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 0x7f5a501103b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f5a44009b10 tx=0x7f5a44009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a4dffb700 10 client.?.objecter ms_handle_connect 0x7f5a500f7400 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a4dffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a4dffb700 1 -- 172.21.15.59:0/1125207399 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44018070 con 0x7f5a500f7400 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a4dffb700 1 -- 172.21.15.59:0/1125207399 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a44013e70 con 0x7f5a500f7400 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a44005040 con 0x7f5a500f7400 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a4dffb700 1 -- 172.21.15.59:0/1125207399 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4401ddb0 con 0x7f5a500f7400 2024-02-18T00:30:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a56e79700 10 client.14553.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5010b210 con 0x7f5a500f7400 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.432+0000 7f5a56e79700 1 librados: init done 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.433+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5010b400 con 0x7f5a500f7400 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.433+0000 7f5a4dffb700 1 -- 172.21.15.59:0/1125207399 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5a4401d420 con 0x7f5a500f7400 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4dffb700 1 --2- 172.21.15.59:0/1125207399 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a40062c90 0x7f5a40065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4dffb700 1 -- 172.21.15.59:0/1125207399 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f5a4407df40 con 0x7f5a500f7400 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4dffb700 10 client.14553.objecter ms_dispatch 0x7f5a50001550 osd_map(72..72 src has 1..72) v4 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4dffb700 3 client.14553.objecter handle_osd_map got epochs [72,72] > 0 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4dffb700 3 client.14553.objecter handle_osd_map decoding full epoch 72 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4ffff700 1 --2- 172.21.15.59:0/1125207399 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a40062c90 0x7f5a40065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4dffb700 20 client.14553.objecter dump_active .. 0 homeless 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4dffb700 10 client.14553.objecter ms_handle_connect 0x7f5a40062c90 2024-02-18T00:30:04.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.434+0000 7f5a4ffff700 1 --2- 172.21.15.59:0/1125207399 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a40062c90 0x7f5a40065150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5a5010bcf0 tx=0x7f5a3c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:04.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.438+0000 7f5a4dffb700 1 -- 172.21.15.59:0/1125207399 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5a44051bc0 con 0x7f5a500f7400 2024-02-18T00:30:04.503 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 bash[62618]: Copying config sha256:1dbe0e931976487e20e5cfb272087e08a9779c88fd5e9617ed7042dd9751ec26 2024-02-18T00:30:04.504 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 bash[62618]: Writing manifest to image destination 2024-02-18T00:30:04.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.609+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"} v 0) v1 -- 0x7f5a5005fa60 con 0x7f5a500f7400 2024-02-18T00:30:04.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.610+0000 7f5a4dffb700 1 -- 172.21.15.59:0/1125207399 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]=0 v72) v1 ==== 94+0+10 (secure 0 0 0) 0x7f5a5005fa60 con 0x7f5a500f7400 2024-02-18T00:30:04.612 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 3 2024-02-18T00:30:04.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.611+0000 7f5a56e79700 10 librados: watch_flush enter 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.611+0000 7f5a56e79700 10 librados: watch_flush exit 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.611+0000 7f5a56e79700 20 client.14553.objecter shutdown clearing up homeless session... 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.611+0000 7f5a56e79700 10 client.14553.objecter successfully canceled tick 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a40062c90 msgr2=0x7f5a40065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 --2- 172.21.15.59:0/1125207399 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a40062c90 0x7f5a40065150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5a5010bcf0 tx=0x7f5a3c00b040).stop 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 msgr2=0x7f5a501103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 0x7f5a501103b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f5a44009b10 tx=0x7f5a44009cb0).stop 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 shutdown_connections 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 --2- 172.21.15.59:0/1125207399 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a40062c90 0x7f5a40065150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a500fd510 0x7f5a5010a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a500f7400 0x7f5a501103b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 --2- 172.21.15.59:0/1125207399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a500fcbb0 0x7f5a501108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:04.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 >> 172.21.15.59:0/1125207399 conn(0x7f5a500f2820 msgr2=0x7f5a50060d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:04.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 shutdown_connections 2024-02-18T00:30:04.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 -- 172.21.15.59:0/1125207399 wait complete. 2024-02-18T00:30:04.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:04.612+0000 7f5a56e79700 1 librados: shutdown 2024-02-18T00:30:05.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:03.723696+0000 mgr.y (mgr.24454) 27 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:30:05.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:04.611205+0000 mon.a (mon.0) 678 : audit [DBG] from='client.? 172.21.15.59:0/1125207399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-18T00:30:05.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:03.723696+0000 mgr.y (mgr.24454) 27 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:30:05.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:04.611205+0000 mon.a (mon.0) 678 : audit [DBG] from='client.? 172.21.15.59:0/1125207399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-18T00:30:05.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:03.723696+0000 mgr.y (mgr.24454) 27 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:30:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:04.611205+0000 mon.a (mon.0) 678 : audit [DBG] from='client.? 172.21.15.59:0/1125207399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-18T00:30:05.254 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.955Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-18T00:30:05.254 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.955Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.956Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.956Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.956Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.956Z caller=node_exporter.go:115 level=info collector=arp 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.956Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.956Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-18T00:30:05.255 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-18T00:30:05.256 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-18T00:30:05.256 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=edac 2024-02-18T00:30:05.256 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-18T00:30:05.256 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-18T00:30:05.256 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-18T00:30:05.257 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=os 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-18T00:30:05.258 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-18T00:30:05.259 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=stat 2024-02-18T00:30:05.259 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-18T00:30:05.259 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-18T00:30:05.259 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-18T00:30:05.259 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=time 2024-02-18T00:30:05.259 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-18T00:30:05.260 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=uname 2024-02-18T00:30:05.260 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-18T00:30:05.260 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-18T00:30:05.260 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-18T00:30:05.260 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-18T00:30:05.260 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b[62668]: ts=2024-02-18T00:30:04.957Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-18T00:30:05.261 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:05 smithi137 bash[62618]: a6cd18af7ca02763ff669e48c04397f651d5b56f2b440c22c2e09fa944826329 2024-02-18T00:30:05.261 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:30:05 smithi137 systemd[1]: Started Ceph node-exporter.b for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:30:05.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool get default.rgw.log pg_num 2024-02-18T00:30:05.886 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:06.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:05.113529+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:06.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:30:05.123657+0000 mgr.y (mgr.24454) 28 : cephadm [INF] Deploying daemon prometheus.a on smithi137 2024-02-18T00:30:06.403 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:05.113529+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:06.403 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:30:05.123657+0000 mgr.y (mgr.24454) 28 : cephadm [INF] Deploying daemon prometheus.a on smithi137 2024-02-18T00:30:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:05.113529+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:30:05.123657+0000 mgr.y (mgr.24454) 28 : cephadm [INF] Deploying daemon prometheus.a on smithi137 2024-02-18T00:30:07.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:05.724176+0000 mgr.y (mgr.24454) 29 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:30:07.225 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:05.724176+0000 mgr.y (mgr.24454) 29 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:30:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:05.724176+0000 mgr.y (mgr.24454) 29 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:30:07.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 -- 172.21.15.59:0/1098474453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340fd510 msgr2=0x7fa3340f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:07.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 --2- 172.21.15.59:0/1098474453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340fd510 0x7fa3340f6d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fa31c004870 tx=0x7fa31c009cb0).stop 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 -- 172.21.15.59:0/1098474453 shutdown_connections 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 --2- 172.21.15.59:0/1098474453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3340f7400 0x7fa334103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 --2- 172.21.15.59:0/1098474453 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3340fcbb0 0x7fa3340fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 --2- 172.21.15.59:0/1098474453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340fd510 0x7fa3340f6d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 -- 172.21.15.59:0/1098474453 >> 172.21.15.59:0/1098474453 conn(0x7fa3340f2820 msgr2=0x7fa3340f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 -- 172.21.15.59:0/1098474453 shutdown_connections 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.790+0000 7fa33a792700 1 -- 172.21.15.59:0/1098474453 wait complete. 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 librados: starting msgr at 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 librados: starting objecter 2024-02-18T00:30:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 Processor -- start 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 -- start start 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 librados: setting wanted keys 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 librados: calling monclient init 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340f7400 0x7fa3340f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3340fcbb0 0x7fa3340f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.791+0000 7fa33a792700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3340fd510 0x7fa3340f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa33a792700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341166a0 con 0x7fa3340f7400 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa33a792700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa334116810 con 0x7fa3340fd510 2024-02-18T00:30:07.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa33a792700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3341169d0 con 0x7fa3340fcbb0 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa333fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340f7400 0x7fa3340f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3337fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3340fcbb0 0x7fa3340f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa333fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340f7400 0x7fa3340f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58720/0 (socket says 172.21.15.59:58720) 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa333fff700 1 -- 172.21.15.59:0/1011179707 learned_addr learned my addr 172.21.15.59:0/1011179707 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa338d2f700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3340fd510 0x7fa3340f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3337fe700 1 -- 172.21.15.59:0/1011179707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3340fd510 msgr2=0x7fa3340f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3337fe700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3340fd510 0x7fa3340f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3337fe700 1 -- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340f7400 msgr2=0x7fa3340f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3337fe700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340f7400 0x7fa3340f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3337fe700 1 -- 172.21.15.59:0/1011179707 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324005040 con 0x7fa3340fcbb0 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa333fff700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340f7400 0x7fa3340f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3337fe700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3340fcbb0 0x7fa3340f8ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa31c004870 tx=0x7fa31c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:07.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3317fa700 10 client.?.objecter ms_handle_connect 0x7fa3340fcbb0 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3317fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3317fa700 1 -- 172.21.15.59:0/1011179707 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00e070 con 0x7fa3340fcbb0 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3317fa700 1 -- 172.21.15.59:0/1011179707 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa31c009710 con 0x7fa3340fcbb0 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c005040 con 0x7fa3340fcbb0 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa3317fa700 1 -- 172.21.15.59:0/1011179707 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c019070 con 0x7fa3340fcbb0 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa33a792700 10 client.24464.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa334116ec0 con 0x7fa3340fcbb0 2024-02-18T00:30:07.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.792+0000 7fa33a792700 1 librados: init done 2024-02-18T00:30:07.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.793+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33405fa60 con 0x7fa3340fcbb0 2024-02-18T00:30:07.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa3317fa700 1 -- 172.21.15.59:0/1011179707 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa31c0090d0 con 0x7fa3340fcbb0 2024-02-18T00:30:07.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa3317fa700 1 --2- 172.21.15.59:0/1011179707 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa320062c90 0x7fa320065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:07.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa3317fa700 1 -- 172.21.15.59:0/1011179707 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6056+0+0 (secure 0 0 0) 0x7fa31c07d140 con 0x7fa3340fcbb0 2024-02-18T00:30:07.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa3317fa700 10 client.24464.objecter ms_dispatch 0x7fa334001550 osd_map(72..72 src has 1..72) v4 2024-02-18T00:30:07.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa3317fa700 3 client.24464.objecter handle_osd_map got epochs [72,72] > 0 2024-02-18T00:30:07.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa3317fa700 3 client.24464.objecter handle_osd_map decoding full epoch 72 2024-02-18T00:30:07.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa333fff700 1 --2- 172.21.15.59:0/1011179707 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa320062c90 0x7fa320065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:07.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.794+0000 7fa3317fa700 20 client.24464.objecter dump_active .. 0 homeless 2024-02-18T00:30:07.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.795+0000 7fa3317fa700 10 client.24464.objecter ms_handle_connect 0x7fa320062c90 2024-02-18T00:30:07.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.795+0000 7fa333fff700 1 --2- 172.21.15.59:0/1011179707 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa320062c90 0x7fa320065150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa324009810 tx=0x7fa32400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:07.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.798+0000 7fa3317fa700 1 -- 172.21.15.59:0/1011179707 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa31c050e20 con 0x7fa3340fcbb0 2024-02-18T00:30:07.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.969+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"} v 0) v1 -- 0x7fa3341060e0 con 0x7fa3340fcbb0 2024-02-18T00:30:07.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.969+0000 7fa3317fa700 1 -- 172.21.15.59:0/1011179707 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa31c0230a0 con 0x7fa3340fcbb0 2024-02-18T00:30:07.971 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 10 librados: watch_flush enter 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 10 librados: watch_flush exit 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 20 client.24464.objecter shutdown clearing up homeless session... 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 10 client.24464.objecter successfully canceled tick 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa320062c90 msgr2=0x7fa320065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 --2- 172.21.15.59:0/1011179707 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa320062c90 0x7fa320065150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fa324009810 tx=0x7fa32400b040).stop 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3340fcbb0 msgr2=0x7fa3340f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3340fcbb0 0x7fa3340f8ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa31c004870 tx=0x7fa31c00b040).stop 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 shutdown_connections 2024-02-18T00:30:07.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 --2- 172.21.15.59:0/1011179707 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa320062c90 0x7fa320065150 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3340fd510 0x7fa3340f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3340f7400 0x7fa3340f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 --2- 172.21.15.59:0/1011179707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3340fcbb0 0x7fa3340f8ea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 >> 172.21.15.59:0/1011179707 conn(0x7fa3340f2820 msgr2=0x7fa334060e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 shutdown_connections 2024-02-18T00:30:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 -- 172.21.15.59:0/1011179707 wait complete. 2024-02-18T00:30:07.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:07.971+0000 7fa33a792700 1 librados: shutdown 2024-02-18T00:30:08.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:07.970507+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? 172.21.15.59:0/1011179707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-18T00:30:08.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:07.970507+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? 172.21.15.59:0/1011179707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-18T00:30:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:07.970507+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? 172.21.15.59:0/1011179707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-18T00:30:08.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool get default.rgw.control pg_num 2024-02-18T00:30:08.974 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:07.724550+0000 mgr.y (mgr.24454) 30 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:09.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:07.724550+0000 mgr.y (mgr.24454) 30 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:09.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:07.724550+0000 mgr.y (mgr.24454) 30 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:10.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:09.121286+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:10.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:09.121353+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:10.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:09.126560+0000 mon.a (mon.0) 680 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-18T00:30:10.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:09.616651+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-18T00:30:10.340 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:09.887696+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:10.340 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:09.121286+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:10.340 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:09.121353+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:10.340 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:09.126560+0000 mon.a (mon.0) 680 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-18T00:30:10.340 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:09.616651+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-18T00:30:10.340 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:09.887696+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:09.121286+0000 osd.7 (osd.7) 5 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:09.121353+0000 osd.7 (osd.7) 6 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:09.126560+0000 mon.a (mon.0) 680 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-18T00:30:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:09.616651+0000 mon.a (mon.0) 681 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-18T00:30:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:09.887696+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 -- 172.21.15.59:0/3737496028 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc0f8d70 msgr2=0x7f80cc0f9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 --2- 172.21.15.59:0/3737496028 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc0f8d70 0x7f80cc0f9150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f80bc004870 tx=0x7f80bc009cb0).stop 2024-02-18T00:30:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 -- 172.21.15.59:0/3737496028 shutdown_connections 2024-02-18T00:30:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 --2- 172.21.15.59:0/3737496028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc1022d0 0x7f80cc1046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 --2- 172.21.15.59:0/3737496028 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc0f8d70 0x7f80cc0f9150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 --2- 172.21.15.59:0/3737496028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80cc0f9640 0x7f80cc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:10.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 -- 172.21.15.59:0/3737496028 >> 172.21.15.59:0/3737496028 conn(0x7f80cc0f2820 msgr2=0x7f80cc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 -- 172.21.15.59:0/3737496028 shutdown_connections 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 -- 172.21.15.59:0/3737496028 wait complete. 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 librados: starting msgr at 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.930+0000 7f80d381a700 1 librados: starting objecter 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 Processor -- start 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 -- start start 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 librados: setting wanted keys 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 librados: calling monclient init 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc0f8d70 0x7f80cc0f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:10.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80cc0f9640 0x7f80cc0f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc1022d0 0x7f80cc0f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc116630 con 0x7f80cc0f8d70 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1167a0 con 0x7f80cc0f9640 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d381a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc116960 con 0x7f80cc1022d0 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc0f8d70 0x7f80cc0f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc0f8d70 0x7f80cc0f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59010/0 (socket says 172.21.15.59:59010) 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 -- 172.21.15.59:0/847728515 learned_addr learned my addr 172.21.15.59:0/847728515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d1db7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc1022d0 0x7f80cc0f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d0db5700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80cc0f9640 0x7f80cc0f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 -- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc1022d0 msgr2=0x7f80cc0f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:10.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc1022d0 0x7f80cc0f7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 -- 172.21.15.59:0/847728515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80cc0f9640 msgr2=0x7f80cc0f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80cc0f9640 0x7f80cc0f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 -- 172.21.15.59:0/847728515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40097c0 con 0x7f80cc0f8d70 2024-02-18T00:30:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d1db7700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc1022d0 0x7f80cc0f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:10.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d0db5700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80cc0f9640 0x7f80cc0f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:30:10.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80d15b6700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc0f8d70 0x7f80cc0f6d20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f80bc009eb0 tx=0x7f80bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:10.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80c27fc700 10 client.?.objecter ms_handle_connect 0x7f80cc0f8d70 2024-02-18T00:30:10.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.931+0000 7f80c27fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:10.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80c27fc700 1 -- 172.21.15.59:0/847728515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00e070 con 0x7f80cc0f8d70 2024-02-18T00:30:10.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80c27fc700 1 -- 172.21.15.59:0/847728515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80bc013da0 con 0x7f80cc0f8d70 2024-02-18T00:30:10.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80bc005040 con 0x7f80cc0f8d70 2024-02-18T00:30:10.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80c27fc700 1 -- 172.21.15.59:0/847728515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc019070 con 0x7f80cc0f8d70 2024-02-18T00:30:10.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80d381a700 10 client.14565.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:10.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc116e50 con 0x7f80cc0f8d70 2024-02-18T00:30:10.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80d381a700 1 librados: init done 2024-02-18T00:30:10.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.932+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc05fa60 con 0x7f80cc0f8d70 2024-02-18T00:30:10.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.933+0000 7f80c27fc700 1 -- 172.21.15.59:0/847728515 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f80bc01e420 con 0x7f80cc0f8d70 2024-02-18T00:30:10.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80c27fc700 1 --2- 172.21.15.59:0/847728515 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80b8062c90 0x7f80b8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:10.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80c27fc700 1 -- 172.21.15.59:0/847728515 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f80bc07d9a0 con 0x7f80cc0f8d70 2024-02-18T00:30:10.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80c27fc700 10 client.14565.objecter ms_dispatch 0x7f80cc001550 osd_map(74..74 src has 1..74) v4 2024-02-18T00:30:10.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80c27fc700 3 client.14565.objecter handle_osd_map got epochs [74,74] > 0 2024-02-18T00:30:10.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80c27fc700 3 client.14565.objecter handle_osd_map decoding full epoch 74 2024-02-18T00:30:10.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80d0db5700 1 --2- 172.21.15.59:0/847728515 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80b8062c90 0x7f80b8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:10.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80c27fc700 20 client.14565.objecter dump_active .. 0 homeless 2024-02-18T00:30:10.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80c27fc700 10 client.14565.objecter ms_handle_connect 0x7f80b8062c90 2024-02-18T00:30:10.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.934+0000 7f80d0db5700 1 --2- 172.21.15.59:0/847728515 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80b8062c90 0x7f80b8065150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f80cc0f84c0 tx=0x7f80c8009d60).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:10.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:10.937+0000 7f80c27fc700 1 -- 172.21.15.59:0/847728515 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f80bc051720 con 0x7f80cc0f8d70 2024-02-18T00:30:11.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:30:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:30:11.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.109+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"} v 0) v1 -- 0x7f80cc107290 con 0x7f80cc0f8d70 2024-02-18T00:30:11.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.109+0000 7f80c27fc700 1 -- 172.21.15.59:0/847728515 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]=0 v74) v1 ==== 108+0+11 (secure 0 0 0) 0x7f80bc051310 con 0x7f80cc0f8d70 2024-02-18T00:30:11.112 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-02-18T00:30:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 10 librados: watch_flush enter 2024-02-18T00:30:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 10 librados: watch_flush exit 2024-02-18T00:30:11.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 20 client.14565.objecter shutdown clearing up homeless session... 2024-02-18T00:30:11.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 10 client.14565.objecter successfully canceled tick 2024-02-18T00:30:11.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80b8062c90 msgr2=0x7f80b8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:11.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 --2- 172.21.15.59:0/847728515 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80b8062c90 0x7f80b8065150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f80cc0f84c0 tx=0x7f80c8009d60).stop 2024-02-18T00:30:11.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc0f8d70 msgr2=0x7f80cc0f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:11.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc0f8d70 0x7f80cc0f6d20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f80bc009eb0 tx=0x7f80bc00b040).stop 2024-02-18T00:30:11.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 shutdown_connections 2024-02-18T00:30:11.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80cc1022d0 0x7f80cc0f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:11.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80cc0f8d70 0x7f80cc0f6d20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:11.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 --2- 172.21.15.59:0/847728515 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80b8062c90 0x7f80b8065150 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:11.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 --2- 172.21.15.59:0/847728515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80cc0f9640 0x7f80cc0f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:11.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 >> 172.21.15.59:0/847728515 conn(0x7f80cc0f2820 msgr2=0x7f80cc0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:11.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 shutdown_connections 2024-02-18T00:30:11.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 -- 172.21.15.59:0/847728515 wait complete. 2024-02-18T00:30:11.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:11.111+0000 7f80d381a700 1 librados: shutdown 2024-02-18T00:30:11.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:09.724969+0000 mgr.y (mgr.24454) 31 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:11.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:11.110793+0000 mon.a (mon.0) 683 : audit [DBG] from='client.? 172.21.15.59:0/847728515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-18T00:30:11.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:09.724969+0000 mgr.y (mgr.24454) 31 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:11.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:11.110793+0000 mon.a (mon.0) 683 : audit [DBG] from='client.? 172.21.15.59:0/847728515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-18T00:30:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:09.724969+0000 mgr.y (mgr.24454) 31 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:11.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:11.110793+0000 mon.a (mon.0) 683 : audit [DBG] from='client.? 172.21.15.59:0/847728515' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-18T00:30:11.897 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool get default.rgw.meta pg_num 2024-02-18T00:30:12.180 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:12.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:10.750073+0000 mgr.y (mgr.24454) 32 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:10.750073+0000 mgr.y (mgr.24454) 32 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:12.425 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:10.750073+0000 mgr.y (mgr.24454) 32 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:13.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:11.725591+0000 mgr.y (mgr.24454) 33 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:13.436 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:11.725591+0000 mgr.y (mgr.24454) 33 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:11.725591+0000 mgr.y (mgr.24454) 33 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 -- 172.21.15.59:0/921775895 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 msgr2=0x7fc5a00f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 --2- 172.21.15.59:0/921775895 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a00f9090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc5940047b0 tx=0x7fc594009cb0).stop 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 -- 172.21.15.59:0/921775895 shutdown_connections 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 --2- 172.21.15.59:0/921775895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5a0102210 0x7fc5a0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 --2- 172.21.15.59:0/921775895 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a00f9090 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 --2- 172.21.15.59:0/921775895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5a00f9610 0x7fc5a0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 -- 172.21.15.59:0/921775895 >> 172.21.15.59:0/921775895 conn(0x7fc5a00f2820 msgr2=0x7fc5a00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 -- 172.21.15.59:0/921775895 shutdown_connections 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.035+0000 7fc5a5609700 1 -- 172.21.15.59:0/921775895 wait complete. 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 librados: starting msgr at 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 librados: starting objecter 2024-02-18T00:30:14.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 Processor -- start 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 -- start start 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 librados: setting wanted keys 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 librados: calling monclient init 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a010e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5a00f9610 0x7fc5a010e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5a0102210 0x7fc5a01083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a0108890 con 0x7fc5a0102210 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a0108a00 con 0x7fc5a00f9610 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc5a5609700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a0108bc0 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.036+0000 7fc59effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a010e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a010e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37650/0 (socket says 172.21.15.59:37650) 2024-02-18T00:30:14.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 -- 172.21.15.59:0/3789465523 learned_addr learned my addr 172.21.15.59:0/3789465523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59f7fe700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5a0102210 0x7fc5a01083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59e7fc700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5a00f9610 0x7fc5a010e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 -- 172.21.15.59:0/3789465523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5a00f9610 msgr2=0x7fc5a010e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5a00f9610 0x7fc5a010e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 -- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5a0102210 msgr2=0x7fc5a01083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5a0102210 0x7fc5a01083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 -- 172.21.15.59:0/3789465523 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590005040 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59f7fe700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5a0102210 0x7fc5a01083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc59effd700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a010e200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fc594009b10 tx=0x7fc594009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc587fff700 10 client.?.objecter ms_handle_connect 0x7fc5a00f8cb0 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc587fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc587fff700 1 -- 172.21.15.59:0/3789465523 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594018070 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc587fff700 1 -- 172.21.15.59:0/3789465523 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc594013da0 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc594005040 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc587fff700 1 -- 172.21.15.59:0/3789465523 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59401dcc0 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc5a5609700 10 client.24476.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:14.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a0109000 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.037+0000 7fc5a5609700 1 librados: init done 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.038+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a005fa60 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc587fff700 1 -- 172.21.15.59:0/3789465523 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc59401d420 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc587fff700 1 --2- 172.21.15.59:0/3789465523 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc588062c90 0x7fc588065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc587fff700 1 -- 172.21.15.59:0/3789465523 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6056+0+0 (secure 0 0 0) 0x7fc59407df30 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc587fff700 10 client.24476.objecter ms_dispatch 0x7fc5a0001550 osd_map(74..74 src has 1..74) v4 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc587fff700 3 client.24476.objecter handle_osd_map got epochs [74,74] > 0 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc587fff700 3 client.24476.objecter handle_osd_map decoding full epoch 74 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc59e7fc700 1 --2- 172.21.15.59:0/3789465523 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc588062c90 0x7fc588065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.039+0000 7fc587fff700 20 client.24476.objecter dump_active .. 0 homeless 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.040+0000 7fc587fff700 10 client.24476.objecter ms_handle_connect 0x7fc588062c90 2024-02-18T00:30:14.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.040+0000 7fc59e7fc700 1 --2- 172.21.15.59:0/3789465523 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc588062c90 0x7fc588065150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc5a0109b40 tx=0x7fc59000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.044+0000 7fc587fff700 1 -- 172.21.15.59:0/3789465523 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc594051c10 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.215+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"} v 0) v1 -- 0x7fc5a0107290 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.215+0000 7fc587fff700 1 -- 172.21.15.59:0/3789465523 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]=0 v74) v1 ==== 105+0+11 (secure 0 0 0) 0x7fc594051800 con 0x7fc5a00f8cb0 2024-02-18T00:30:14.217 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-02-18T00:30:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 10 librados: watch_flush enter 2024-02-18T00:30:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 10 librados: watch_flush exit 2024-02-18T00:30:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 20 client.24476.objecter shutdown clearing up homeless session... 2024-02-18T00:30:14.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 10 client.24476.objecter successfully canceled tick 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc588062c90 msgr2=0x7fc588065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 --2- 172.21.15.59:0/3789465523 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc588062c90 0x7fc588065150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc5a0109b40 tx=0x7fc59000b040).stop 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 msgr2=0x7fc5a010e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a010e200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fc594009b10 tx=0x7fc594009cb0).stop 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 shutdown_connections 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5a0102210 0x7fc5a01083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5a00f8cb0 0x7fc5a010e200 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 --2- 172.21.15.59:0/3789465523 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc588062c90 0x7fc588065150 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 --2- 172.21.15.59:0/3789465523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5a00f9610 0x7fc5a010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 >> 172.21.15.59:0/3789465523 conn(0x7fc5a00f2820 msgr2=0x7fc5a00f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 shutdown_connections 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 -- 172.21.15.59:0/3789465523 wait complete. 2024-02-18T00:30:14.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:14.217+0000 7fc5a5609700 1 librados: shutdown 2024-02-18T00:30:14.905 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-18T00:30:14.905 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-18T00:30:14.905 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-18T00:30:15.143 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:14 smithi137 systemd[1]: Starting Ceph prometheus.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:30:15.187 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:15.446 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:13.725954+0000 mgr.y (mgr.24454) 34 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:15.446 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:14.216806+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.59:0/3789465523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-18T00:30:15.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:13.725954+0000 mgr.y (mgr.24454) 34 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:15.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:14.216806+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.59:0/3789465523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-18T00:30:15.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:13.725954+0000 mgr.y (mgr.24454) 34 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:15.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:14.216806+0000 mon.c (mon.2) 18 : audit [DBG] from='client.? 172.21.15.59:0/3789465523' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.445Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.445Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.445Z caller=main.go:518 level=info host_details="(Linux 4.18.0-539.el8.x86_64 #1 SMP Mon Feb 5 18:34:56 UTC 2024 x86_64 smithi137 (none))" 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.445Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.445Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.446Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.447Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.448Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-18T00:30:15.754 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.450Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.450Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.005µs 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.450Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.450Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.450Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=47.361µs wal_replay_duration=288.27µs total_replay_duration=356.865µs 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.451Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.451Z caller=main.go:947 level=info msg="TSDB started" 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.451Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.466Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.330461ms db_storage=618ns remote_storage=1.11µs web_handler=279ns query_engine=563ns scrape=452.271µs scrape_sd=25.059µs notify=34.871µs notify_sd=5.585µs rules=14.592427ms 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:30:15.466Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 bash[63129]: c52135923cf3583a6aed32145507a9228dbfc75c459d3db151371f08417bbd27 2024-02-18T00:30:15.755 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:30:15 smithi137 systemd[1]: Started Ceph prometheus.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:30:16.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:15] ENGINE Bus STOPPING 2024-02-18T00:30:16.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:15] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:30:16.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:15] ENGINE Bus STOPPED 2024-02-18T00:30:16.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:15] ENGINE Bus STARTING 2024-02-18T00:30:16.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:15] ENGINE Serving on http://:::9283 2024-02-18T00:30:16.003 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:15] ENGINE Bus STARTED 2024-02-18T00:30:16.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:15.592323+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:15.726488+0000 mgr.y (mgr.24454) 35 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:15.848048+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:15.853282+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:15.855573+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:15.856006+0000 mgr.y (mgr.24454) 36 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:15.861752+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:30:15.871139+0000 mgr.y (mgr.24454) 37 : cephadm [INF] Deploying daemon grafana.a on smithi137 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:15.592323+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:15.726488+0000 mgr.y (mgr.24454) 35 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:15.848048+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:15.853282+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.899 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:15.855573+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T00:30:16.899 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:15.856006+0000 mgr.y (mgr.24454) 36 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T00:30:16.899 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:15.861752+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:16.899 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:30:15.871139+0000 mgr.y (mgr.24454) 37 : cephadm [INF] Deploying daemon grafana.a on smithi137 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:15.592323+0000 mon.a (mon.0) 684 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:15.726488+0000 mgr.y (mgr.24454) 35 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:15.848048+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:15.853282+0000 mon.a (mon.0) 686 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:15.855573+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:15.856006+0000 mgr.y (mgr.24454) 36 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:15.861752+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:30:15.871139+0000 mgr.y (mgr.24454) 37 : cephadm [INF] Deploying daemon grafana.a on smithi137 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 -- 172.21.15.59:0/1590553415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200fdbd0 msgr2=0x7f18200fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 --2- 172.21.15.59:0/1590553415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200fdbd0 0x7f18200fdfb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f18100047b0 tx=0x7f1810009cb0).stop 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 -- 172.21.15.59:0/1590553415 shutdown_connections 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 --2- 172.21.15.59:0/1590553415 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200f6a90 0x7f1820104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 --2- 172.21.15.59:0/1590553415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200fdbd0 0x7f18200fdfb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 --2- 172.21.15.59:0/1590553415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18200fe530 0x7f18200f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 -- 172.21.15.59:0/1590553415 >> 172.21.15.59:0/1590553415 conn(0x7f182006c370 msgr2=0x7f182006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 -- 172.21.15.59:0/1590553415 shutdown_connections 2024-02-18T00:30:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 -- 172.21.15.59:0/1590553415 wait complete. 2024-02-18T00:30:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 librados: starting msgr at 2024-02-18T00:30:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 librados: starting objecter 2024-02-18T00:30:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.191+0000 7f182589d700 1 Processor -- start 2024-02-18T00:30:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 -- start start 2024-02-18T00:30:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 librados: setting wanted keys 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 librados: calling monclient init 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200f6a90 0x7f182010e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200fdbd0 0x7f182010e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18200fe530 0x7f1820108340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1820108830 con 0x7f18200f6a90 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18201089a0 con 0x7f18200fe530 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f182589d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1820108b60 con 0x7f18200fdbd0 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181e7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200fdbd0 0x7f182010e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200f6a90 0x7f182010e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200f6a90 0x7f182010e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59048/0 (socket says 172.21.15.59:59048) 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 -- 172.21.15.59:0/2214079022 learned_addr learned my addr 172.21.15.59:0/2214079022 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181e7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200fdbd0 0x7f182010e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37674/0 (socket says 172.21.15.59:37674) 2024-02-18T00:30:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181f7fe700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18200fe530 0x7f1820108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 -- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200fdbd0 msgr2=0x7f182010e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200fdbd0 0x7f182010e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 -- 172.21.15.59:0/2214079022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18200fe530 msgr2=0x7f1820108340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18200fe530 0x7f1820108340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 -- 172.21.15.59:0/2214079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18180097c0 con 0x7f18200f6a90 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181e7fc700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200fdbd0 0x7f182010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181f7fe700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18200fe530 0x7f1820108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f181effd700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200f6a90 0x7f182010e1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f1810009b10 tx=0x7f1810009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f1807fff700 10 client.?.objecter ms_handle_connect 0x7f18200f6a90 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.192+0000 7f1807fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.193+0000 7f1807fff700 1 -- 172.21.15.59:0/2214079022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810018070 con 0x7f18200f6a90 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.193+0000 7f1807fff700 1 -- 172.21.15.59:0/2214079022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1810013da0 con 0x7f18200f6a90 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.193+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1810005040 con 0x7f18200f6a90 2024-02-18T00:30:17.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.193+0000 7f1807fff700 1 -- 172.21.15.59:0/2214079022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181001dc10 con 0x7f18200f6a90 2024-02-18T00:30:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.193+0000 7f182589d700 10 client.14577.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.193+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1820109000 con 0x7f18200f6a90 2024-02-18T00:30:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.193+0000 7f182589d700 1 librados: init done 2024-02-18T00:30:17.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.194+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18201091f0 con 0x7f18200f6a90 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.194+0000 7f1807fff700 1 -- 172.21.15.59:0/2214079022 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f181001d420 con 0x7f18200f6a90 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f1807fff700 1 --2- 172.21.15.59:0/2214079022 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1808062c90 0x7f1808065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f1807fff700 1 -- 172.21.15.59:0/2214079022 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f181007de50 con 0x7f18200f6a90 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f1807fff700 10 client.14577.objecter ms_dispatch 0x7f1820001550 osd_map(74..74 src has 1..74) v4 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f1807fff700 3 client.14577.objecter handle_osd_map got epochs [74,74] > 0 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f1807fff700 3 client.14577.objecter handle_osd_map decoding full epoch 74 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f181e7fc700 1 --2- 172.21.15.59:0/2214079022 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1808062c90 0x7f1808065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f1807fff700 20 client.14577.objecter dump_active .. 0 homeless 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f1807fff700 10 client.14577.objecter ms_handle_connect 0x7f1808062c90 2024-02-18T00:30:17.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.195+0000 7f181e7fc700 1 --2- 172.21.15.59:0/2214079022 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1808062c90 0x7f1808065150 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1820109ae0 tx=0x7f181400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:17.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.199+0000 7f1807fff700 1 -- 172.21.15.59:0/2214079022 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1810051b30 con 0x7f18200f6a90 2024-02-18T00:30:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.414+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1820107100 con 0x7f18200f6a90 2024-02-18T00:30:17.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.417+0000 7f1807fff700 1 -- 172.21.15.59:0/2214079022 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v20) v1 ==== 74+0+151936 (secure 0 0 0) 0x7f1810051720 con 0x7f18200f6a90 2024-02-18T00:30:17.422 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:30:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 10 librados: watch_flush enter 2024-02-18T00:30:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 10 librados: watch_flush exit 2024-02-18T00:30:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 20 client.14577.objecter shutdown clearing up homeless session... 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 10 client.14577.objecter successfully canceled tick 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1808062c90 msgr2=0x7f1808065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 --2- 172.21.15.59:0/2214079022 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1808062c90 0x7f1808065150 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1820109ae0 tx=0x7f181400b040).stop 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200f6a90 msgr2=0x7f182010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200f6a90 0x7f182010e1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f1810009b10 tx=0x7f1810009cb0).stop 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 shutdown_connections 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 --2- 172.21.15.59:0/2214079022 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1808062c90 0x7f1808065150 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18200fe530 0x7f1820108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18200f6a90 0x7f182010e1a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 --2- 172.21.15.59:0/2214079022 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18200fdbd0 0x7f182010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.419+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 >> 172.21.15.59:0/2214079022 conn(0x7f182006c370 msgr2=0x7f18200fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.420+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 shutdown_connections 2024-02-18T00:30:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.420+0000 7f182589d700 1 -- 172.21.15.59:0/2214079022 wait complete. 2024-02-18T00:30:17.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:17.420+0000 7f182589d700 1 librados: shutdown 2024-02-18T00:30:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:17.416069+0000 mon.a (mon.0) 689 : audit [DBG] from='client.? 172.21.15.59:0/2214079022' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T00:30:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:17.416069+0000 mon.a (mon.0) 689 : audit [DBG] from='client.? 172.21.15.59:0/2214079022' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T00:30:18.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:17.416069+0000 mon.a (mon.0) 689 : audit [DBG] from='client.? 172.21.15.59:0/2214079022' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T00:30:18.214 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"active_gid":24454,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3721891797}]},"active_addr":"172.21.15.59:6800/3721891797","active_change":"2024-02-18T00:29:39.703899+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":24448,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.59:8443/","prometheus":"http://172.21.15.59:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":65,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3961685966}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":417091922}]}]} 2024-02-18T00:30:18.217 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-18T00:30:18.217 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-18T00:30:18.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T00:30:18.501 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:18.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:17.726798+0000 mgr.y (mgr.24454) 38 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:18.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:18.098742+0000 osd.5 (osd.5) 7 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:30:18.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:18.101768+0000 osd.5 (osd.5) 8 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:30:18.862 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:17.726798+0000 mgr.y (mgr.24454) 38 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:18.862 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:18.098742+0000 osd.5 (osd.5) 7 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:30:18.863 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:18.101768+0000 osd.5 (osd.5) 8 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:30:19.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:17.726798+0000 mgr.y (mgr.24454) 38 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:18.098742+0000 osd.5 (osd.5) 7 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:30:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:18.101768+0000 osd.5 (osd.5) 8 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:30:20.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:17.981826+0000 osd.4 (osd.4) 5 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:30:20.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:17.985416+0000 osd.4 (osd.4) 6 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:30:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:17.981826+0000 osd.4 (osd.4) 5 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:30:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:17.985416+0000 osd.4 (osd.4) 6 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:30:20.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:17.981826+0000 osd.4 (osd.4) 5 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:30:20.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:17.985416+0000 osd.4 (osd.4) 6 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:30:20.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:19.028644+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.d scrub starts 2024-02-18T00:30:20.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:19.032091+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.d scrub ok 2024-02-18T00:30:20.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:19.477508+0000 osd.6 (osd.6) 5 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:30:20.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:19.478750+0000 osd.6 (osd.6) 6 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:30:20.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:19.725743+0000 osd.3 (osd.3) 7 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:30:20.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:19.726595+0000 osd.3 (osd.3) 8 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:30:20.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:19.727140+0000 mgr.y (mgr.24454) 39 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1012 B/s rd, 0 op/s 2024-02-18T00:30:20.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:19.897210+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:20.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:19.028644+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.d scrub starts 2024-02-18T00:30:20.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:19.032091+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.d scrub ok 2024-02-18T00:30:20.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:19.477508+0000 osd.6 (osd.6) 5 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:30:20.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:19.478750+0000 osd.6 (osd.6) 6 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:30:20.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:19.725743+0000 osd.3 (osd.3) 7 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:30:20.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:19.726595+0000 osd.3 (osd.3) 8 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:30:20.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:19.727140+0000 mgr.y (mgr.24454) 39 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1012 B/s rd, 0 op/s 2024-02-18T00:30:20.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:19.897210+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:21.004 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:30:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:19.028644+0000 osd.4 (osd.4) 7 : cluster [DBG] 4.d scrub starts 2024-02-18T00:30:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:19.032091+0000 osd.4 (osd.4) 8 : cluster [DBG] 4.d scrub ok 2024-02-18T00:30:21.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:19.477508+0000 osd.6 (osd.6) 5 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:30:21.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:19.478750+0000 osd.6 (osd.6) 6 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:30:21.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:19.725743+0000 osd.3 (osd.3) 7 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:30:21.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:19.726595+0000 osd.3 (osd.3) 8 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:30:21.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:19.727140+0000 mgr.y (mgr.24454) 39 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1012 B/s rd, 0 op/s 2024-02-18T00:30:21.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:19.897210+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 -- 172.21.15.59:0/561722191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80f8420 msgr2=0x7ff6c8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 --2- 172.21.15.59:0/561722191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80f8420 0x7ff6c8104a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff6c0009c30 tx=0x7ff6c0004c50).stop 2024-02-18T00:30:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 -- 172.21.15.59:0/561722191 shutdown_connections 2024-02-18T00:30:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 --2- 172.21.15.59:0/561722191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80f8420 0x7ff6c8104a20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 --2- 172.21.15.59:0/561722191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80fdbb0 0x7ff6c80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 --2- 172.21.15.59:0/561722191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c80fe510 0x7ff6c80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 -- 172.21.15.59:0/561722191 >> 172.21.15.59:0/561722191 conn(0x7ff6c80f3800 msgr2=0x7ff6c80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 -- 172.21.15.59:0/561722191 shutdown_connections 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.674+0000 7ff6ce724700 1 -- 172.21.15.59:0/561722191 wait complete. 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 librados: starting msgr at 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 librados: starting objecter 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 Processor -- start 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 -- start start 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 librados: setting wanted keys 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 librados: calling monclient init 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80f8420 0x7ff6c810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c80fdbb0 0x7ff6c810e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:21.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80fe510 0x7ff6c81080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c81085a0 con 0x7ff6c80f8420 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8108710 con 0x7ff6c80fdbb0 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6ce724700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c81088d0 con 0x7ff6c80fe510 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80f8420 0x7ff6c810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6cccc1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80fe510 0x7ff6c81080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80f8420 0x7ff6c810df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48412/0 (socket says 172.21.15.59:48412) 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6c7fff700 1 -- 172.21.15.59:0/1151438825 learned_addr learned my addr 172.21.15.59:0/1151438825 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.675+0000 7ff6c77fe700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c80fdbb0 0x7ff6c810e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6cccc1700 1 -- 172.21.15.59:0/1151438825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c80fdbb0 msgr2=0x7ff6c810e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6cccc1700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c80fdbb0 0x7ff6c810e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6cccc1700 1 -- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80f8420 msgr2=0x7ff6c810df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6cccc1700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80f8420 0x7ff6c810df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6cccc1700 1 -- 172.21.15.59:0/1151438825 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8005040 con 0x7ff6c80fe510 2024-02-18T00:30:21.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6c7fff700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80f8420 0x7ff6c810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6cccc1700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80fe510 0x7ff6c81080b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ff6c0009ad0 tx=0x7ff6c0004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6c57fa700 10 client.?.objecter ms_handle_connect 0x7ff6c80fe510 2024-02-18T00:30:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6c57fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6c57fa700 1 -- 172.21.15.59:0/1151438825 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000e040 con 0x7ff6c80fe510 2024-02-18T00:30:21.156 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:30:21.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6c57fa700 1 -- 172.21.15.59:0/1151438825 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c0018040 con 0x7ff6c80fe510 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c00097c0 con 0x7ff6c80fe510 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6c57fa700 1 -- 172.21.15.59:0/1151438825 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0003a30 con 0x7ff6c80fe510 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6ce724700 10 client.24488.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c8108d90 con 0x7ff6c80fe510 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.676+0000 7ff6ce724700 1 librados: init done 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.677+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c805ee80 con 0x7ff6c80fe510 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.677+0000 7ff6c57fa700 1 -- 172.21.15.59:0/1151438825 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff6c0004120 con 0x7ff6c80fe510 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c57fa700 1 --2- 172.21.15.59:0/1151438825 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff6b0062c90 0x7ff6b0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c57fa700 1 -- 172.21.15.59:0/1151438825 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6056+0+0 (secure 0 0 0) 0x7ff6c007d800 con 0x7ff6c80fe510 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c7fff700 1 --2- 172.21.15.59:0/1151438825 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff6b0062c90 0x7ff6b0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c57fa700 10 client.24488.objecter ms_dispatch 0x7ff6c8001550 osd_map(74..74 src has 1..74) v4 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c57fa700 3 client.24488.objecter handle_osd_map got epochs [74,74] > 0 2024-02-18T00:30:21.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c57fa700 3 client.24488.objecter handle_osd_map decoding full epoch 74 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c57fa700 20 client.24488.objecter dump_active .. 0 homeless 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c57fa700 10 client.24488.objecter ms_handle_connect 0x7ff6b0062c90 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.678+0000 7ff6c7fff700 1 --2- 172.21.15.59:0/1151438825 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff6b0062c90 0x7ff6b0065150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff6b8004d40 tx=0x7ff6b800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.682+0000 7ff6c57fa700 1 -- 172.21.15.59:0/1151438825 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff6c00514e0 con 0x7ff6c80fe510 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.853+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6c8109630 con 0x7ff6c80fe510 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.854+0000 7ff6c57fa700 1 -- 172.21.15.59:0/1151438825 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19003 (secure 0 0 0) 0x7ff6c0003e60 con 0x7ff6c80fe510 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 10 librados: watch_flush enter 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 10 librados: watch_flush exit 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 20 client.24488.objecter shutdown clearing up homeless session... 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 10 client.24488.objecter successfully canceled tick 2024-02-18T00:30:21.158 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","created":"2024-02-18T00:23:11.390869+0000","modified":"2024-02-18T00:30:09.612772+0000","last_up_change":"2024-02-18T00:29:06.503960+0000","last_in_change":"2024-02-18T00:28:48.970063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T00:27:02.909667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-18T00:29:13.697458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-18T00:29:14.132992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T00:29:15.191206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T00:29:17.526023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T00:29:19.600478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"6e729102-80e2-4a4e-92ca-600ee8b1ed85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":3700238977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3700238977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3700238977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6803","nonce":3700238977}]},"public_addr":"172.21.15.59:6801/3700238977","cluster_addr":"172.21.15.59:6802/3700238977","heartbeat_back_addr":"172.21.15.59:6804/3700238977","heartbeat_front_addr":"172.21.15.59:6803/3700238977","state":["exists","up"]},{"osd":1,"uuid":"f67a7206-6d31-497f-97e0-672d134b13d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":26279110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":26279110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":26279110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6807","nonce":26279110}]},"public_addr":"172.21.15.59:6805/26279110","cluster_addr":"172.21.15.59:6806/26279110","heartbeat_back_addr":"172.21.15.59:6808/26279110","heartbeat_front_addr":"172.21.15.59:6807/26279110","state":["exists","up"]},{"osd":2,"uuid":"d6223dbd-e539-4770-9e5a-7b756304f8f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1153193637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1153193637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1153193637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6811","nonce":1153193637}]},"public_addr":"172.21.15.59:6809/1153193637","cluster_addr":"172.21.15.59:6810/1153193637","heartbeat_back_addr":"172.21.15.59:6812/1153193637","heartbeat_front_addr":"172.21.15.59:6811/1153193637","state":["exists","up"]},{"osd":3,"uuid":"fff275df-95e0-41f6-96c3-e531f05de32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1805700964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1805700964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1805700964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6815","nonce":1805700964}]},"public_addr":"172.21.15.59:6813/1805700964","cluster_addr":"172.21.15.59:6814/1805700964","heartbeat_back_addr":"172.21.15.59:6816/1805700964","heartbeat_front_addr":"172.21.15.59:6815/1805700964","state":["exists","up"]},{"osd":4,"uuid":"a0bae50b-4f29-4310-ac5f-bc7285464231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2044387056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":2044387056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":2044387056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2044387056}]},"public_addr":"172.21.15.137:6800/2044387056","cluster_addr":"172.21.15.137:6801/2044387056","heartbeat_back_addr":"172.21.15.137:6803/2044387056","heartbeat_front_addr":"172.21.15.137:6802/2044387056","state":["exists","up"]},{"osd":5,"uuid":"513a4192-45fb-4444-a1f9-4bd2f2a5fa01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2170432263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":2170432263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":2170432263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2170432263}]},"public_addr":"172.21.15.137:6804/2170432263","cluster_addr":"172.21.15.137:6805/2170432263","heartbeat_back_addr":"172.21.15.137:6807/2170432263","heartbeat_front_addr":"172.21.15.137:6806/2170432263","state":["exists","up"]},{"osd":6,"uuid":"5bbb7683-ea5a-490c-8c8b-67b53c45406f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3356009390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3356009390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6811","nonce":3356009390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3356009390}]},"public_addr":"172.21.15.137:6808/3356009390","cluster_addr":"172.21.15.137:6809/3356009390","heartbeat_back_addr":"172.21.15.137:6811/3356009390","heartbeat_front_addr":"172.21.15.137:6810/3356009390","state":["exists","up"]},{"osd":7,"uuid":"951d317b-7a34-41a0-b1a1-a2851b2070bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":53025171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":53025171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6815","nonce":53025171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":53025171}]},"public_addr":"172.21.15.137:6812/53025171","cluster_addr":"172.21.15.137:6813/53025171","heartbeat_back_addr":"172.21.15.137:6815/53025171","heartbeat_front_addr":"172.21.15.137:6814/53025171","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:42.402380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:16.051406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:06.168044+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:47.387404+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:12.724408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:17.639367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:50.471046+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:30:09.121357+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1243102711":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/817089002":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/3174805070":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/3506000666":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/4167740619":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/999852549":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/3575161033":"2024-02-19T00:23:45.307159+0000","172.21.15.59:0/2403979355":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/642821245":"2024-02-19T00:23:45.307159+0000","172.21.15.59:6800/1656814063":"2024-02-19T00:23:45.307159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff6b0062c90 msgr2=0x7ff6b0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 --2- 172.21.15.59:0/1151438825 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff6b0062c90 0x7ff6b0065150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff6b8004d40 tx=0x7ff6b800b040).stop 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80fe510 msgr2=0x7ff6c81080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80fe510 0x7ff6c81080b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7ff6c0009ad0 tx=0x7ff6c0004c50).stop 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 shutdown_connections 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6c80fe510 0x7ff6c81080b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6c80f8420 0x7ff6c810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 --2- 172.21.15.59:0/1151438825 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff6b0062c90 0x7ff6b0065150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 --2- 172.21.15.59:0/1151438825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c80fdbb0 0x7ff6c810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 >> 172.21.15.59:0/1151438825 conn(0x7ff6c80f3800 msgr2=0x7ff6c80f5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:21.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 shutdown_connections 2024-02-18T00:30:21.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 -- 172.21.15.59:0/1151438825 wait complete. 2024-02-18T00:30:21.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:20.856+0000 7ff6ce724700 1 librados: shutdown 2024-02-18T00:30:21.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:30:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:30:22.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:20.210238+0000 osd.1 (osd.1) 7 : cluster [DBG] 2.9 deep-scrub starts 2024-02-18T00:30:22.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:20.213166+0000 osd.1 (osd.1) 8 : cluster [DBG] 2.9 deep-scrub ok 2024-02-18T00:30:22.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:20.622815+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:30:22.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:20.623818+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:30:22.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:20.760774+0000 mgr.y (mgr.24454) 40 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:22.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:20.855229+0000 mon.c (mon.2) 19 : audit [DBG] from='client.? 172.21.15.59:0/1151438825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:30:22.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:20.210238+0000 osd.1 (osd.1) 7 : cluster [DBG] 2.9 deep-scrub starts 2024-02-18T00:30:22.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:20.213166+0000 osd.1 (osd.1) 8 : cluster [DBG] 2.9 deep-scrub ok 2024-02-18T00:30:22.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:20.622815+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:30:22.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:20.623818+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:30:22.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:20.760774+0000 mgr.y (mgr.24454) 40 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:22.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:20.855229+0000 mon.c (mon.2) 19 : audit [DBG] from='client.? 172.21.15.59:0/1151438825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:30:22.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:20.210238+0000 osd.1 (osd.1) 7 : cluster [DBG] 2.9 deep-scrub starts 2024-02-18T00:30:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:20.213166+0000 osd.1 (osd.1) 8 : cluster [DBG] 2.9 deep-scrub ok 2024-02-18T00:30:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:20.622815+0000 osd.0 (osd.0) 9 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:30:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:20.623818+0000 osd.0 (osd.0) 10 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:30:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:20.760774+0000 mgr.y (mgr.24454) 40 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:20.855229+0000 mon.c (mon.2) 19 : audit [DBG] from='client.? 172.21.15.59:0/1151438825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:30:22.096 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-18T00:30:22.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T00:30:22.383 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:22.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:22] "GET /metrics HTTP/1.1" 200 172140 "" "Prometheus/2.33.4" 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.091435+0000 osd.4 (osd.4) 9 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.092903+0000 osd.4 (osd.4) 10 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.463753+0000 osd.6 (osd.6) 7 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.465244+0000 osd.6 (osd.6) 8 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.671915+0000 osd.0 (osd.0) 11 : cluster [DBG] 2.13 deep-scrub starts 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.674894+0000 osd.0 (osd.0) 12 : cluster [DBG] 2.13 deep-scrub ok 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.727890+0000 mgr.y (mgr.24454) 41 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.732100+0000 osd.3 (osd.3) 9 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:21.733428+0000 osd.3 (osd.3) 10 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:30:22.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.538165+0000 osd.7 (osd.7) 7 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:30:22.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.538942+0000 osd.7 (osd.7) 8 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:30:22.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.091435+0000 osd.4 (osd.4) 9 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:30:22.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.092903+0000 osd.4 (osd.4) 10 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:30:22.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.463753+0000 osd.6 (osd.6) 7 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:30:22.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.465244+0000 osd.6 (osd.6) 8 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:30:22.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.671915+0000 osd.0 (osd.0) 11 : cluster [DBG] 2.13 deep-scrub starts 2024-02-18T00:30:22.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.674894+0000 osd.0 (osd.0) 12 : cluster [DBG] 2.13 deep-scrub ok 2024-02-18T00:30:22.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.727890+0000 mgr.y (mgr.24454) 41 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:22.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.732100+0000 osd.3 (osd.3) 9 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:30:22.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:21.733428+0000 osd.3 (osd.3) 10 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:30:22.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.538165+0000 osd.7 (osd.7) 7 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:30:22.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.538942+0000 osd.7 (osd.7) 8 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.091435+0000 osd.4 (osd.4) 9 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.092903+0000 osd.4 (osd.4) 10 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.463753+0000 osd.6 (osd.6) 7 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.465244+0000 osd.6 (osd.6) 8 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.671915+0000 osd.0 (osd.0) 11 : cluster [DBG] 2.13 deep-scrub starts 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.674894+0000 osd.0 (osd.0) 12 : cluster [DBG] 2.13 deep-scrub ok 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.727890+0000 mgr.y (mgr.24454) 41 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.732100+0000 osd.3 (osd.3) 9 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:30:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:21.733428+0000 osd.3 (osd.3) 10 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:30:23.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.538165+0000 osd.7 (osd.7) 7 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:30:23.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.538942+0000 osd.7 (osd.7) 8 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:30:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.272783+0000 osd.1 (osd.1) 9 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:30:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.276251+0000 osd.1 (osd.1) 10 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:30:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.509449+0000 osd.6 (osd.6) 9 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:30:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.511248+0000 osd.6 (osd.6) 10 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:30:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.714132+0000 osd.3 (osd.3) 11 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:22.716222+0000 osd.3 (osd.3) 12 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:23.424984+0000 osd.2 (osd.2) 7 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:23.426945+0000 osd.2 (osd.2) 8 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.272783+0000 osd.1 (osd.1) 9 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.276251+0000 osd.1 (osd.1) 10 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.509449+0000 osd.6 (osd.6) 9 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.511248+0000 osd.6 (osd.6) 10 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.714132+0000 osd.3 (osd.3) 11 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:22.716222+0000 osd.3 (osd.3) 12 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:23.424984+0000 osd.2 (osd.2) 7 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:30:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:23.426945+0000 osd.2 (osd.2) 8 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:30:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.272783+0000 osd.1 (osd.1) 9 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:30:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.276251+0000 osd.1 (osd.1) 10 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:30:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.509449+0000 osd.6 (osd.6) 9 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:30:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.511248+0000 osd.6 (osd.6) 10 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:30:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.714132+0000 osd.3 (osd.3) 11 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:30:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:22.716222+0000 osd.3 (osd.3) 12 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:30:24.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:23.424984+0000 osd.2 (osd.2) 7 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:30:24.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:23.426945+0000 osd.2 (osd.2) 8 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:30:24.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 -- 172.21.15.59:0/2955209323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c102120 msgr2=0x7faf8c104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:24.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 --2- 172.21.15.59:0/2955209323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c102120 0x7faf8c104510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7faf84009be0 tx=0x7faf84004d10).stop 2024-02-18T00:30:24.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 -- 172.21.15.59:0/2955209323 shutdown_connections 2024-02-18T00:30:24.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 --2- 172.21.15.59:0/2955209323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf8c0f8bc0 0x7faf8c0f8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 --2- 172.21.15.59:0/2955209323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c102120 0x7faf8c104510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 --2- 172.21.15.59:0/2955209323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf8c0f9520 0x7faf8c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 -- 172.21.15.59:0/2955209323 >> 172.21.15.59:0/2955209323 conn(0x7faf8c0f26f0 msgr2=0x7faf8c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 -- 172.21.15.59:0/2955209323 shutdown_connections 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 -- 172.21.15.59:0/2955209323 wait complete. 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 librados: starting msgr at 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 librados: starting objecter 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.342+0000 7faf91508700 1 Processor -- start 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 -- start start 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 librados: setting wanted keys 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 librados: calling monclient init 2024-02-18T00:30:24.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf8c0f8bc0 0x7faf8c10df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c0f9520 0x7faf8c10e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf8c102120 0x7faf8c108130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c108620 con 0x7faf8c102120 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c108790 con 0x7faf8c0f8bc0 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf91508700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c108950 con 0x7faf8c0f9520 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c0f9520 0x7faf8c10e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf8c102120 0x7faf8c108130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c0f9520 0x7faf8c10e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:54264/0 (socket says 172.21.15.59:54264) 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8a7fc700 1 -- 172.21.15.59:0/842597533 learned_addr learned my addr 172.21.15.59:0/842597533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8affd700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf8c0f8bc0 0x7faf8c10df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8b7fe700 1 -- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c0f9520 msgr2=0x7faf8c10e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8b7fe700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c0f9520 0x7faf8c10e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8b7fe700 1 -- 172.21.15.59:0/842597533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf8c0f8bc0 msgr2=0x7faf8c10df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8b7fe700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf8c0f8bc0 0x7faf8c10df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8b7fe700 1 -- 172.21.15.59:0/842597533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80005040 con 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.343+0000 7faf8a7fc700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c0f9520 0x7faf8c10e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf8b7fe700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf8c102120 0x7faf8c108130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7faf84009bb0 tx=0x7faf84004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf73fff700 10 client.?.objecter ms_handle_connect 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf73fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf73fff700 1 -- 172.21.15.59:0/842597533 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8400e040 con 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf840097c0 con 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf73fff700 1 -- 172.21.15.59:0/842597533 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf84018040 con 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf73fff700 1 -- 172.21.15.59:0/842597533 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84013e90 con 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf91508700 10 client.14589.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c108de0 con 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.344+0000 7faf91508700 1 librados: init done 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.345+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c05f990 con 0x7faf8c102120 2024-02-18T00:30:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.345+0000 7faf73fff700 1 -- 172.21.15.59:0/842597533 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7faf84013520 con 0x7faf8c102120 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.345+0000 7faf73fff700 1 --2- 172.21.15.59:0/842597533 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf74062c90 0x7faf74065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf73fff700 1 -- 172.21.15.59:0/842597533 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6056+0+0 (secure 0 0 0) 0x7faf8407d490 con 0x7faf8c102120 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf73fff700 10 client.14589.objecter ms_dispatch 0x7faf8c001550 osd_map(74..74 src has 1..74) v4 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf73fff700 3 client.14589.objecter handle_osd_map got epochs [74,74] > 0 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf73fff700 3 client.14589.objecter handle_osd_map decoding full epoch 74 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf8affd700 1 --2- 172.21.15.59:0/842597533 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf74062c90 0x7faf74065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf73fff700 20 client.14589.objecter dump_active .. 0 homeless 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf73fff700 10 client.14589.objecter ms_handle_connect 0x7faf74062c90 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.346+0000 7faf8affd700 1 --2- 172.21.15.59:0/842597533 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf74062c90 0x7faf74065150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7faf7c004dc0 tx=0x7faf7c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.350+0000 7faf73fff700 1 -- 172.21.15.59:0/842597533 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faf84051300 con 0x7faf8c102120 2024-02-18T00:30:24.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.522+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf8c108fd0 con 0x7faf8c102120 2024-02-18T00:30:24.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.523+0000 7faf73fff700 1 -- 172.21.15.59:0/842597533 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+19003 (secure 0 0 0) 0x7faf8402ea50 con 0x7faf8c102120 2024-02-18T00:30:24.525 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:30:24.526 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","created":"2024-02-18T00:23:11.390869+0000","modified":"2024-02-18T00:30:09.612772+0000","last_up_change":"2024-02-18T00:29:06.503960+0000","last_in_change":"2024-02-18T00:28:48.970063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T00:27:02.909667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-18T00:29:13.697458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-18T00:29:14.132992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T00:29:15.191206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T00:29:17.526023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T00:29:19.600478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"6e729102-80e2-4a4e-92ca-600ee8b1ed85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":3700238977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3700238977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3700238977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6803","nonce":3700238977}]},"public_addr":"172.21.15.59:6801/3700238977","cluster_addr":"172.21.15.59:6802/3700238977","heartbeat_back_addr":"172.21.15.59:6804/3700238977","heartbeat_front_addr":"172.21.15.59:6803/3700238977","state":["exists","up"]},{"osd":1,"uuid":"f67a7206-6d31-497f-97e0-672d134b13d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":26279110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":26279110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":26279110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6807","nonce":26279110}]},"public_addr":"172.21.15.59:6805/26279110","cluster_addr":"172.21.15.59:6806/26279110","heartbeat_back_addr":"172.21.15.59:6808/26279110","heartbeat_front_addr":"172.21.15.59:6807/26279110","state":["exists","up"]},{"osd":2,"uuid":"d6223dbd-e539-4770-9e5a-7b756304f8f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1153193637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1153193637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1153193637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6811","nonce":1153193637}]},"public_addr":"172.21.15.59:6809/1153193637","cluster_addr":"172.21.15.59:6810/1153193637","heartbeat_back_addr":"172.21.15.59:6812/1153193637","heartbeat_front_addr":"172.21.15.59:6811/1153193637","state":["exists","up"]},{"osd":3,"uuid":"fff275df-95e0-41f6-96c3-e531f05de32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1805700964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1805700964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1805700964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6815","nonce":1805700964}]},"public_addr":"172.21.15.59:6813/1805700964","cluster_addr":"172.21.15.59:6814/1805700964","heartbeat_back_addr":"172.21.15.59:6816/1805700964","heartbeat_front_addr":"172.21.15.59:6815/1805700964","state":["exists","up"]},{"osd":4,"uuid":"a0bae50b-4f29-4310-ac5f-bc7285464231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2044387056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":2044387056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":2044387056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2044387056}]},"public_addr":"172.21.15.137:6800/2044387056","cluster_addr":"172.21.15.137:6801/2044387056","heartbeat_back_addr":"172.21.15.137:6803/2044387056","heartbeat_front_addr":"172.21.15.137:6802/2044387056","state":["exists","up"]},{"osd":5,"uuid":"513a4192-45fb-4444-a1f9-4bd2f2a5fa01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2170432263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":2170432263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":2170432263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2170432263}]},"public_addr":"172.21.15.137:6804/2170432263","cluster_addr":"172.21.15.137:6805/2170432263","heartbeat_back_addr":"172.21.15.137:6807/2170432263","heartbeat_front_addr":"172.21.15.137:6806/2170432263","state":["exists","up"]},{"osd":6,"uuid":"5bbb7683-ea5a-490c-8c8b-67b53c45406f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3356009390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3356009390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6811","nonce":3356009390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3356009390}]},"public_addr":"172.21.15.137:6808/3356009390","cluster_addr":"172.21.15.137:6809/3356009390","heartbeat_back_addr":"172.21.15.137:6811/3356009390","heartbeat_front_addr":"172.21.15.137:6810/3356009390","state":["exists","up"]},{"osd":7,"uuid":"951d317b-7a34-41a0-b1a1-a2851b2070bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":53025171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":53025171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6815","nonce":53025171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":53025171}]},"public_addr":"172.21.15.137:6812/53025171","cluster_addr":"172.21.15.137:6813/53025171","heartbeat_back_addr":"172.21.15.137:6815/53025171","heartbeat_front_addr":"172.21.15.137:6814/53025171","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:42.402380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:16.051406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:06.168044+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:47.387404+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:12.724408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:17.639367+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:29:50.471046+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:30:09.121357+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1243102711":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/817089002":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/3174805070":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/3506000666":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/4167740619":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/999852549":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/3575161033":"2024-02-19T00:23:45.307159+0000","172.21.15.59:0/2403979355":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/642821245":"2024-02-19T00:23:45.307159+0000","172.21.15.59:6800/1656814063":"2024-02-19T00:23:45.307159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T00:30:24.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 10 librados: watch_flush enter 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 10 librados: watch_flush exit 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 20 client.14589.objecter shutdown clearing up homeless session... 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 10 client.14589.objecter successfully canceled tick 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf74062c90 msgr2=0x7faf74065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 1 --2- 172.21.15.59:0/842597533 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf74062c90 0x7faf74065150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7faf7c004dc0 tx=0x7faf7c00b040).stop 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf8c102120 msgr2=0x7faf8c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.524+0000 7faf91508700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf8c102120 0x7faf8c108130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7faf84009bb0 tx=0x7faf84004d10).stop 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 shutdown_connections 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 --2- 172.21.15.59:0/842597533 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf74062c90 0x7faf74065150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf8c0f8bc0 0x7faf8c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf8c102120 0x7faf8c108130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 --2- 172.21.15.59:0/842597533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf8c0f9520 0x7faf8c10e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:24.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 >> 172.21.15.59:0/842597533 conn(0x7faf8c0f26f0 msgr2=0x7faf8c0f32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:24.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 shutdown_connections 2024-02-18T00:30:24.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 -- 172.21.15.59:0/842597533 wait complete. 2024-02-18T00:30:24.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:24.525+0000 7faf91508700 1 librados: shutdown 2024-02-18T00:30:24.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:23.728346+0000 mgr.y (mgr.24454) 42 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:24.404886+0000 osd.2 (osd.2) 9 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:24.405724+0000 osd.2 (osd.2) 10 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:24.523518+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? 172.21.15.59:0/842597533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:24.600452+0000 osd.7 (osd.7) 9 : cluster [DBG] 2.b scrub starts 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:24.601436+0000 osd.7 (osd.7) 10 : cluster [DBG] 2.b scrub ok 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:23.728346+0000 mgr.y (mgr.24454) 42 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:24.404886+0000 osd.2 (osd.2) 9 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:24.405724+0000 osd.2 (osd.2) 10 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:30:24.920 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:24.523518+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? 172.21.15.59:0/842597533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:30:24.921 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:24.600452+0000 osd.7 (osd.7) 9 : cluster [DBG] 2.b scrub starts 2024-02-18T00:30:24.921 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:24.601436+0000 osd.7 (osd.7) 10 : cluster [DBG] 2.b scrub ok 2024-02-18T00:30:24.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:23.728346+0000 mgr.y (mgr.24454) 42 : cluster [DBG] pgmap v34: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:30:24.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:24.404886+0000 osd.2 (osd.2) 9 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:30:24.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:24.405724+0000 osd.2 (osd.2) 10 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:30:24.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:24.523518+0000 mon.a (mon.0) 691 : audit [DBG] from='client.? 172.21.15.59:0/842597533' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:30:24.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:24.600452+0000 osd.7 (osd.7) 9 : cluster [DBG] 2.b scrub starts 2024-02-18T00:30:24.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:24.601436+0000 osd.7 (osd.7) 10 : cluster [DBG] 2.b scrub ok 2024-02-18T00:30:25.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-18T00:30:25.545 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:27.122 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:25.634330+0000 osd.7 (osd.7) 11 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:30:27.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:25.635255+0000 osd.7 (osd.7) 12 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:30:27.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:25.728962+0000 mgr.y (mgr.24454) 43 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:27.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:25.801328+0000 osd.3 (osd.3) 13 : cluster [DBG] 2.1b deep-scrub starts 2024-02-18T00:30:27.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:25.803717+0000 osd.3 (osd.3) 14 : cluster [DBG] 2.1b deep-scrub ok 2024-02-18T00:30:27.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:26.098115+0000 osd.4 (osd.4) 11 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:30:27.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:26.098977+0000 osd.4 (osd.4) 12 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:30:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:25.634330+0000 osd.7 (osd.7) 11 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:30:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:25.635255+0000 osd.7 (osd.7) 12 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:30:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:25.728962+0000 mgr.y (mgr.24454) 43 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:25.801328+0000 osd.3 (osd.3) 13 : cluster [DBG] 2.1b deep-scrub starts 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:25.803717+0000 osd.3 (osd.3) 14 : cluster [DBG] 2.1b deep-scrub ok 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:26.098115+0000 osd.4 (osd.4) 11 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:26.098977+0000 osd.4 (osd.4) 12 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:25.634330+0000 osd.7 (osd.7) 11 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:25.635255+0000 osd.7 (osd.7) 12 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:25.728962+0000 mgr.y (mgr.24454) 43 : cluster [DBG] pgmap v35: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:25.801328+0000 osd.3 (osd.3) 13 : cluster [DBG] 2.1b deep-scrub starts 2024-02-18T00:30:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:25.803717+0000 osd.3 (osd.3) 14 : cluster [DBG] 2.1b deep-scrub ok 2024-02-18T00:30:27.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:26.098115+0000 osd.4 (osd.4) 11 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:30:27.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:26.098977+0000 osd.4 (osd.4) 12 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:30:27.722 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 systemd[1]: Starting Ceph grafana.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:30:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 -- 172.21.15.59:0/174393719 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad780f8d50 msgr2=0x7fad780f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 --2- 172.21.15.59:0/174393719 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad780f8d50 0x7fad780f9130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fad600047b0 tx=0x7fad60009cb0).stop 2024-02-18T00:30:27.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 -- 172.21.15.59:0/174393719 shutdown_connections 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 --2- 172.21.15.59:0/174393719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad781022b0 0x7fad781046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 --2- 172.21.15.59:0/174393719 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad780f8d50 0x7fad780f9130 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 --2- 172.21.15.59:0/174393719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad780f9620 0x7fad78101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 -- 172.21.15.59:0/174393719 >> 172.21.15.59:0/174393719 conn(0x7fad780f2820 msgr2=0x7fad780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 -- 172.21.15.59:0/174393719 shutdown_connections 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 -- 172.21.15.59:0/174393719 wait complete. 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 librados: starting msgr at 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 librados: starting objecter 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.877+0000 7fad7d3bf700 1 Processor -- start 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 -- start start 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 librados: setting wanted keys 2024-02-18T00:30:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 librados: calling monclient init 2024-02-18T00:30:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad780f8d50 0x7fad7810e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad780f9620 0x7fad7810e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad781022b0 0x7fad78108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78108940 con 0x7fad780f9620 2024-02-18T00:30:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78108ab0 con 0x7fad780f8d50 2024-02-18T00:30:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad7d3bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad78108c70 con 0x7fad781022b0 2024-02-18T00:30:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad767fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad780f9620 0x7fad7810e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad781022b0 0x7fad78108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad76ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad780f8d50 0x7fad7810e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad767fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad780f9620 0x7fad7810e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48448/0 (socket says 172.21.15.59:48448) 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad767fc700 1 -- 172.21.15.59:0/1551716812 learned_addr learned my addr 172.21.15.59:0/1551716812 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad777fe700 1 -- 172.21.15.59:0/1551716812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad780f8d50 msgr2=0x7fad7810e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad777fe700 1 --2- 172.21.15.59:0/1551716812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad780f8d50 0x7fad7810e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad777fe700 1 -- 172.21.15.59:0/1551716812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad780f9620 msgr2=0x7fad7810e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.878+0000 7fad777fe700 1 --2- 172.21.15.59:0/1551716812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad780f9620 0x7fad7810e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad777fe700 1 -- 172.21.15.59:0/1551716812 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68005040 con 0x7fad781022b0 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad767fc700 1 --2- 172.21.15.59:0/1551716812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad780f9620 0x7fad7810e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad777fe700 1 --2- 172.21.15.59:0/1551716812 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad781022b0 0x7fad78108450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fad7000d920 tx=0x7fad70004da0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:27.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad6ffff700 10 client.?.objecter ms_handle_connect 0x7fad781022b0 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad6ffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70004020 con 0x7fad781022b0 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad7000f040 con 0x7fad781022b0 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad60005040 con 0x7fad781022b0 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad7d3bf700 10 client.24500.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700036c0 con 0x7fad781022b0 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad781150e0 con 0x7fad781022b0 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.879+0000 7fad7d3bf700 1 librados: init done 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.880+0000 7fad7d3bf700 10 client.24500.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.880+0000 7fad7d3bf700 15 client.24500.objecter _session_command_op_assign -1 1 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.880+0000 7fad7d3bf700 20 client.24500.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.880+0000 7fad7d3bf700 10 client.24500.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.880+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad7805fa60 con 0x7fad781022b0 2024-02-18T00:30:27.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.880+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fad70004180 con 0x7fad781022b0 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad64062c90 0x7fad64065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6056+0+0 (secure 0 0 0) 0x7fad7007c250 con 0x7fad781022b0 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 10 client.24500.objecter ms_dispatch 0x7fad78001550 osd_map(74..74 src has 1..74) v4 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 3 client.24500.objecter handle_osd_map got epochs [74,74] > 0 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 3 client.24500.objecter handle_osd_map decoding full epoch 74 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad76ffd700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad64062c90 0x7fad64065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 10 client.24500.objecter checking command 1 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad6406c530 0x7fad6406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 20 client.24500.objecter _get_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 20 client.24500.objecter put_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 15 client.24500.objecter _session_command_op_remove -1 1 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 20 client.24500.objecter _get_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 20 client.24500.objecter get_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 15 client.24500.objecter _session_command_op_assign 0 1 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 20 client.24500.objecter put_session s=0x7fad6406be80 osd=0 4 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 10 client.24500.objecter _send_command 1 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad6406f010 con 0x7fad6406c530 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 20 client.24500.objecter dump_active .. 0 homeless 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad7007e8c0 con 0x7fad781022b0 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad767fc700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad6406c530 0x7fad6406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 10 client.24500.objecter ms_handle_connect 0x7fad64062c90 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad76ffd700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad64062c90 0x7fad64065150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fad60005010 tx=0x7fad60009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad6ffff700 10 client.24500.objecter ms_handle_connect 0x7fad6406c530 2024-02-18T00:30:27.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.881+0000 7fad767fc700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad6406c530 0x7fad6406e9d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.882+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad6406f010 con 0x7fad6406c530 2024-02-18T00:30:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.882+0000 7fad6ffff700 10 client.24500.objecter ms_dispatch 0x7fad78001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.882+0000 7fad6ffff700 10 client.24500.objecter _finish_command 1 = system:0 2024-02-18T00:30:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.882+0000 7fad6ffff700 20 client.24500.objecter put_session s=0x7fad6406be80 osd=0 4 2024-02-18T00:30:27.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.882+0000 7fad6ffff700 15 client.24500.objecter _session_command_op_remove 0 1 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 10 client.24500.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 15 client.24500.objecter _session_command_op_assign -1 2 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 20 client.24500.objecter _get_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 20 client.24500.objecter put_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 20 client.24500.objecter _get_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 15 client.24500.objecter _session_command_op_remove -1 2 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 20 client.24500.objecter get_session s=0x7fad6406be80 osd=0 3 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 15 client.24500.objecter _session_command_op_assign 0 2 2024-02-18T00:30:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 20 client.24500.objecter put_session s=0x7fad6406be80 osd=0 4 2024-02-18T00:30:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 10 client.24500.objecter _send_command 2 2024-02-18T00:30:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.896+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad7805ee80 con 0x7fad6406c530 2024-02-18T00:30:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad6ffff700 1 -- 172.21.15.59:0/1551716812 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad7805ee80 con 0x7fad6406c530 2024-02-18T00:30:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad6ffff700 10 client.24500.objecter ms_dispatch 0x7fad78001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad6ffff700 10 client.24500.objecter _finish_command 2 = system:0 2024-02-18T00:30:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad6ffff700 20 client.24500.objecter put_session s=0x7fad6406be80 osd=0 4 2024-02-18T00:30:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad6ffff700 15 client.24500.objecter _session_command_op_remove 0 2 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 10 librados: watch_flush enter 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 10 librados: watch_flush exit 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 10 client.24500.objecter close_session for osd.0 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad6406c530 msgr2=0x7fad6406e9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad6406c530 0x7fad6406e9d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 20 client.24500.objecter put_session s=0x7fad6406be80 osd=0 1 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 20 client.24500.objecter shutdown clearing up homeless session... 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 10 client.24500.objecter successfully canceled tick 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad64062c90 msgr2=0x7fad64065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 --2- 172.21.15.59:0/1551716812 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad64062c90 0x7fad64065150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fad60005010 tx=0x7fad60009cb0).stop 2024-02-18T00:30:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad781022b0 msgr2=0x7fad78108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 --2- 172.21.15.59:0/1551716812 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad781022b0 0x7fad78108450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fad7000d920 tx=0x7fad70004da0).stop 2024-02-18T00:30:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad777fe700 1 -- 172.21.15.59:0/1551716812 reap_dead start 2024-02-18T00:30:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 shutdown_connections 2024-02-18T00:30:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 >> 172.21.15.59:0/1551716812 conn(0x7fad780f2820 msgr2=0x7fad780f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 shutdown_connections 2024-02-18T00:30:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 -- 172.21.15.59:0/1551716812 wait complete. 2024-02-18T00:30:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:27.897+0000 7fad7d3bf700 1 librados: shutdown 2024-02-18T00:30:27.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:26.653630+0000 osd.7 (osd.7) 13 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:30:27.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:26.654537+0000 osd.7 (osd.7) 14 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:30:27.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:27.056415+0000 osd.4 (osd.4) 13 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:30:27.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:27.058411+0000 osd.4 (osd.4) 14 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:30:27.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:27.607166+0000 osd.7 (osd.7) 15 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:30:27.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:27.607976+0000 osd.7 (osd.7) 16 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:30:27.921 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:26.653630+0000 osd.7 (osd.7) 13 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:30:27.922 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:26.654537+0000 osd.7 (osd.7) 14 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:30:27.922 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:27.056415+0000 osd.4 (osd.4) 13 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:30:27.922 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:27.058411+0000 osd.4 (osd.4) 14 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:30:27.922 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:27.607166+0000 osd.7 (osd.7) 15 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:30:27.922 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:27.607976+0000 osd.7 (osd.7) 16 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:30:27.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:26.653630+0000 osd.7 (osd.7) 13 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:30:27.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:26.654537+0000 osd.7 (osd.7) 14 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:30:27.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:27.056415+0000 osd.4 (osd.4) 13 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:30:27.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:27.058411+0000 osd.4 (osd.4) 14 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:30:27.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:27.607166+0000 osd.7 (osd.7) 15 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:30:27.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:27.607976+0000 osd.7 (osd.7) 16 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:30:27.975 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-02-18T00:30:27.975 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-02-18T00:30:27.975 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-02-18T00:30:27.975 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-02-18T00:30:27.975 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-18T00:30:27.975 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-18T00:30:27.975 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="App mode production" logger=settings 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2024-02-18T00:30:27.976 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2024-02-18T00:30:27.977 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2024-02-18T00:30:27.978 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Add index user.login/user.email" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_service_account column to user" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_email - v1" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_org_id - v1" 2024-02-18T00:30:28.226 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_code - v1" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_status - v1" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create temp_user v2" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v2" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v2" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v2" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v2" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="copy temp_user v1 to v2" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="drop temp_user_tmp_qwerty" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:27+0000 lvl=info msg="Executing migration" logger=migrator id="Set created for temp users that will otherwise prematurely expire" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2024-02-18T00:30:28.227 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_user_id - v1" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 bash[63490]: 91b9e85045e7beadd14f57fb5494ae1be2b2f2202cdd1d29d9dea3e0eeefc9b6 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 systemd[1]: Started Ceph grafana.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2024-02-18T00:30:28.228 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2024-02-18T00:30:28.229 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-18T00:30:28.230 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_title" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="delete tags for deleted dashboards" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="delete stars for deleted dashboards" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_is_folder" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2024-02-18T00:30:28.231 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2024-02-18T00:30:28.232 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add uid column" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid value" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index datasource_org_id_uid" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index datasource_org_id_is_default" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2024-02-18T00:30:28.233 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add service account foreign key" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-18T00:30:28.234 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-02-18T00:30:28.235 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-02-18T00:30:28.236 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-02-18T00:30:28.237 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-02-18T00:30:28.238 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-18T00:30:28.239 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-02-18T00:30:28.239 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-02-18T00:30:28.239 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-02-18T00:30:28.239 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-02-18T00:30:28.239 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-02-18T00:30:28.239 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-02-18T00:30:28.239 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-02-18T00:30:28.240 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-18T00:30:28.241 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-02-18T00:30:28.482 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-18T00:30:28.482 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-02-18T00:30:28.482 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-18T00:30:28.483 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-02-18T00:30:28.484 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-02-18T00:30:28.485 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-02-18T00:30:28.486 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-02-18T00:30:28.487 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-02-18T00:30:28.488 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-02-18T00:30:28.489 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-02-18T00:30:28.490 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-02-18T00:30:28.491 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-02-18T00:30:28.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-02-18T00:30:28.493 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-02-18T00:30:28.494 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=526.989001ms 2024-02-18T00:30:28.495 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-02-18T00:30:28.495 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-02-18T00:30:28.495 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-02-18T00:30:28.654 INFO:teuthology.orchestra.run.smithi059.stdout:34359738421 2024-02-18T00:30:28.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-02-18T00:30:28.754 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:30:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:30:28+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-02-18T00:30:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:27.729453+0000 mgr.y (mgr.24454) 44 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:30:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:28.071685+0000 mon.a (mon.0) 692 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:28.072870+0000 mon.a (mon.0) 693 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:28.334944+0000 osd.2 (osd.2) 11 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:28.335018+0000 osd.2 (osd.2) 12 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:29.487 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:27.729453+0000 mgr.y (mgr.24454) 44 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:30:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:28.071685+0000 mon.a (mon.0) 692 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:28.072870+0000 mon.a (mon.0) 693 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:28.334944+0000 osd.2 (osd.2) 11 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:28.335018+0000 osd.2 (osd.2) 12 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:27.729453+0000 mgr.y (mgr.24454) 44 : cluster [DBG] pgmap v36: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:30:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:28.071685+0000 mon.a (mon.0) 692 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:28.072870+0000 mon.a (mon.0) 693 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:28.334944+0000 osd.2 (osd.2) 11 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:28.335018+0000 osd.2 (osd.2) 12 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:30.079 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.060593+0000 osd.4 (osd.4) 15 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:30:30.080 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.061386+0000 osd.4 (osd.4) 16 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:30:30.080 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.078957+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-18T00:30:30.080 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.622158+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-18T00:30:30.080 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.771586+0000 osd.0 (osd.0) 13 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:30:30.080 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.772389+0000 osd.0 (osd.0) 14 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:30:30.080 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:29.903551+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:30.488 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.060593+0000 osd.4 (osd.4) 15 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:30:30.489 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.061386+0000 osd.4 (osd.4) 16 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:30:30.489 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.078957+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-18T00:30:30.489 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.622158+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-18T00:30:30.489 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.771586+0000 osd.0 (osd.0) 13 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:30:30.489 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.772389+0000 osd.0 (osd.0) 14 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:30:30.489 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:29.903551+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:30.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.060593+0000 osd.4 (osd.4) 15 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:30:30.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.061386+0000 osd.4 (osd.4) 16 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:30:30.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.078957+0000 mon.a (mon.0) 694 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-18T00:30:30.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.622158+0000 mon.a (mon.0) 695 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-18T00:30:30.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.771586+0000 osd.0 (osd.0) 13 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:30:30.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.772389+0000 osd.0 (osd.0) 14 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:30:30.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:29.903551+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:31.061 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:30:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.327230+0000 osd.1 (osd.1) 11 : cluster [DBG] 2.a scrub starts 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.330121+0000 osd.1 (osd.1) 12 : cluster [DBG] 2.a scrub ok 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.537992+0000 osd.6 (osd.6) 11 : cluster [DBG] 4.a scrub starts 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.539868+0000 osd.6 (osd.6) 12 : cluster [DBG] 4.a scrub ok 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.729980+0000 mgr.y (mgr.24454) 45 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.788248+0000 osd.3 (osd.3) 15 : cluster [DBG] 6.b scrub starts 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:29.789090+0000 osd.3 (osd.3) 16 : cluster [DBG] 6.b scrub ok 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:30.044339+0000 osd.5 (osd.5) 9 : cluster [DBG] 6.a scrub starts 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:30.045213+0000 osd.5 (osd.5) 10 : cluster [DBG] 6.a scrub ok 2024-02-18T00:30:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:30.775551+0000 osd.0 (osd.0) 15 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:30.776607+0000 osd.0 (osd.0) 16 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.327230+0000 osd.1 (osd.1) 11 : cluster [DBG] 2.a scrub starts 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.330121+0000 osd.1 (osd.1) 12 : cluster [DBG] 2.a scrub ok 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.537992+0000 osd.6 (osd.6) 11 : cluster [DBG] 4.a scrub starts 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.539868+0000 osd.6 (osd.6) 12 : cluster [DBG] 4.a scrub ok 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.729980+0000 mgr.y (mgr.24454) 45 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.788248+0000 osd.3 (osd.3) 15 : cluster [DBG] 6.b scrub starts 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:29.789090+0000 osd.3 (osd.3) 16 : cluster [DBG] 6.b scrub ok 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:30.044339+0000 osd.5 (osd.5) 9 : cluster [DBG] 6.a scrub starts 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:30.045213+0000 osd.5 (osd.5) 10 : cluster [DBG] 6.a scrub ok 2024-02-18T00:30:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:30.775551+0000 osd.0 (osd.0) 15 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:30:31.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:30.776607+0000 osd.0 (osd.0) 16 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.327230+0000 osd.1 (osd.1) 11 : cluster [DBG] 2.a scrub starts 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.330121+0000 osd.1 (osd.1) 12 : cluster [DBG] 2.a scrub ok 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.537992+0000 osd.6 (osd.6) 11 : cluster [DBG] 4.a scrub starts 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.539868+0000 osd.6 (osd.6) 12 : cluster [DBG] 4.a scrub ok 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.729980+0000 mgr.y (mgr.24454) 45 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.788248+0000 osd.3 (osd.3) 15 : cluster [DBG] 6.b scrub starts 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:29.789090+0000 osd.3 (osd.3) 16 : cluster [DBG] 6.b scrub ok 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:30.044339+0000 osd.5 (osd.5) 9 : cluster [DBG] 6.a scrub starts 2024-02-18T00:30:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:30.045213+0000 osd.5 (osd.5) 10 : cluster [DBG] 6.a scrub ok 2024-02-18T00:30:31.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:30.775551+0000 osd.0 (osd.0) 15 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:30:31.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:30.776607+0000 osd.0 (osd.0) 16 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:30:31.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 -- 172.21.15.59:0/3240162344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e8102210 msgr2=0x7f08e8104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 --2- 172.21.15.59:0/3240162344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e8102210 0x7f08e8104600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f08e0009c30 tx=0x7f08e0004c50).stop 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 -- 172.21.15.59:0/3240162344 shutdown_connections 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 --2- 172.21.15.59:0/3240162344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e80f8cb0 0x7f08e80f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 --2- 172.21.15.59:0/3240162344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e8102210 0x7f08e8104600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 --2- 172.21.15.59:0/3240162344 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08e80f9610 0x7f08e8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 -- 172.21.15.59:0/3240162344 >> 172.21.15.59:0/3240162344 conn(0x7f08e80f2820 msgr2=0x7f08e80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 -- 172.21.15.59:0/3240162344 shutdown_connections 2024-02-18T00:30:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.590+0000 7f08f04d2700 1 -- 172.21.15.59:0/3240162344 wait complete. 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 librados: starting msgr at 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 librados: starting objecter 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 Processor -- start 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 -- start start 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 librados: setting wanted keys 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 librados: calling monclient init 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e80f8cb0 0x7f08e810e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.591+0000 7f08f04d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e80f9610 0x7f08e810e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08f04d2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08e8102210 0x7f08e81081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08f04d2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e81086e0 con 0x7f08e80f9610 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08f04d2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8108850 con 0x7f08e80f8cb0 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08f04d2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e81089c0 con 0x7f08e8102210 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e80f9610 0x7f08e810e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e80f9610 0x7f08e810e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35694/0 (socket says 172.21.15.59:35694) 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 -- 172.21.15.59:0/1316615421 learned_addr learned my addr 172.21.15.59:0/1316615421 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eea6f700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08e8102210 0x7f08e81081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08ee26e700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e80f8cb0 0x7f08e810e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 -- 172.21.15.59:0/1316615421 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08e8102210 msgr2=0x7f08e81081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08e8102210 0x7f08e81081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 -- 172.21.15.59:0/1316615421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e80f8cb0 msgr2=0x7f08e810e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e80f8cb0 0x7f08e810e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 -- 172.21.15.59:0/1316615421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4005040 con 0x7f08e80f9610 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eea6f700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08e8102210 0x7f08e81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08ee26e700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08e80f8cb0 0x7f08e810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08eda6d700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e80f9610 0x7f08e810e500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f08d80048f0 tx=0x7f08d8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08df7fe700 10 client.?.objecter ms_handle_connect 0x7f08e80f9610 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08df7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8007b00 con 0x7f08e80f9610 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d8005040 con 0x7f08e80f9610 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08d800f040 con 0x7f08e80f9610 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.592+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8014990 con 0x7f08e80f9610 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 10 client.14601.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e00097c0 con 0x7f08e80f9610 2024-02-18T00:30:31.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 1 librados: init done 2024-02-18T00:30:31.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 10 client.14601.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:31.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 15 client.14601.objecter _session_command_op_assign -1 1 2024-02-18T00:30:31.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 20 client.14601.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:31.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 10 client.14601.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:31.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.593+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08e805fa60 con 0x7f08e80f9610 2024-02-18T00:30:31.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.594+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f08d8010500 con 0x7f08e80f9610 2024-02-18T00:30:31.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.594+0000 7f08df7fe700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08d4062c90 0x7f08d4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:31.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f08d807c540 con 0x7f08e80f9610 2024-02-18T00:30:31.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 10 client.14601.objecter ms_dispatch 0x7f08e8001550 osd_map(76..76 src has 1..76) v4 2024-02-18T00:30:31.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 3 client.14601.objecter handle_osd_map got epochs [76,76] > 0 2024-02-18T00:30:31.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 3 client.14601.objecter handle_osd_map decoding full epoch 76 2024-02-18T00:30:31.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08ee26e700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08d4062c90 0x7f08d4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 10 client.14601.objecter checking command 1 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6805/26279110 conn(0x7f08d406c4b0 0x7f08d406e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 20 client.14601.objecter _get_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 20 client.14601.objecter put_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 15 client.14601.objecter _session_command_op_remove -1 1 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 20 client.14601.objecter _get_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 20 client.14601.objecter get_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 15 client.14601.objecter _session_command_op_assign 1 1 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 20 client.14601.objecter put_session s=0x7f08d406be00 osd=1 4 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 10 client.14601.objecter _send_command 1 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 --> v2:172.21.15.59:6805/26279110 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08d406ef90 con 0x7f08d406c4b0 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 20 client.14601.objecter dump_active .. 0 homeless 2024-02-18T00:30:31.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d807eaa0 con 0x7f08e80f9610 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08eea6f700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6805/26279110 conn(0x7f08d406c4b0 0x7f08d406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 10 client.14601.objecter ms_handle_connect 0x7f08d4062c90 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08ee26e700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08d4062c90 0x7f08d4065150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f08e4004c80 tx=0x7f08e400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08eea6f700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6805/26279110 conn(0x7f08d406c4b0 0x7f08d406e950 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.595+0000 7f08df7fe700 10 client.14601.objecter ms_handle_connect 0x7f08d406c4b0 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.596+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f08d406ef90 con 0x7f08d406c4b0 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.596+0000 7f08df7fe700 10 client.14601.objecter ms_dispatch 0x7f08e8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.596+0000 7f08df7fe700 10 client.14601.objecter _finish_command 1 = system:0 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.596+0000 7f08df7fe700 20 client.14601.objecter put_session s=0x7f08d406be00 osd=1 4 2024-02-18T00:30:31.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.596+0000 7f08df7fe700 15 client.14601.objecter _session_command_op_remove 1 1 2024-02-18T00:30:31.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 10 client.14601.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:31.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 15 client.14601.objecter _session_command_op_assign -1 2 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 20 client.14601.objecter _get_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 20 client.14601.objecter put_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 20 client.14601.objecter _get_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 15 client.14601.objecter _session_command_op_remove -1 2 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 20 client.14601.objecter get_session s=0x7f08d406be00 osd=1 3 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 15 client.14601.objecter _session_command_op_assign 1 2 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 20 client.14601.objecter put_session s=0x7f08d406be00 osd=1 4 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 10 client.14601.objecter _send_command 2 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.611+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 --> v2:172.21.15.59:6805/26279110 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08e805ee80 con 0x7f08d406c4b0 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08df7fe700 1 -- 172.21.15.59:0/1316615421 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08e805ee80 con 0x7f08d406c4b0 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08df7fe700 10 client.14601.objecter ms_dispatch 0x7f08e8001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08df7fe700 10 client.14601.objecter _finish_command 2 = system:0 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08df7fe700 20 client.14601.objecter put_session s=0x7f08d406be00 osd=1 4 2024-02-18T00:30:31.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08df7fe700 15 client.14601.objecter _session_command_op_remove 1 2 2024-02-18T00:30:31.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 10 librados: watch_flush enter 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 10 librados: watch_flush exit 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 10 client.14601.objecter close_session for osd.1 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6805/26279110 conn(0x7f08d406c4b0 msgr2=0x7f08d406e950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6805/26279110 conn(0x7f08d406c4b0 0x7f08d406e950 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 20 client.14601.objecter put_session s=0x7f08d406be00 osd=1 1 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 20 client.14601.objecter shutdown clearing up homeless session... 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 10 client.14601.objecter successfully canceled tick 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08d4062c90 msgr2=0x7f08d4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 --2- 172.21.15.59:0/1316615421 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08d4062c90 0x7f08d4065150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f08e4004c80 tx=0x7f08e400b040).stop 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e80f9610 msgr2=0x7f08e810e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 --2- 172.21.15.59:0/1316615421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08e80f9610 0x7f08e810e500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f08d80048f0 tx=0x7f08d8009cb0).stop 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08eea6f700 1 -- 172.21.15.59:0/1316615421 reap_dead start 2024-02-18T00:30:31.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 shutdown_connections 2024-02-18T00:30:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 >> 172.21.15.59:0/1316615421 conn(0x7f08e80f2820 msgr2=0x7f08e80f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 shutdown_connections 2024-02-18T00:30:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 -- 172.21.15.59:0/1316615421 wait complete. 2024-02-18T00:30:31.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:31.612+0000 7f08f04d2700 1 librados: shutdown 2024-02-18T00:30:32.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:30.320452+0000 osd.1 (osd.1) 13 : cluster [DBG] 6.1d deep-scrub starts 2024-02-18T00:30:32.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:30.321326+0000 osd.1 (osd.1) 14 : cluster [DBG] 6.1d deep-scrub ok 2024-02-18T00:30:32.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:30.771281+0000 mgr.y (mgr.24454) 46 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:32.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:32] "GET /metrics HTTP/1.1" 200 172146 "" "Prometheus/2.33.4" 2024-02-18T00:30:32.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:30.320452+0000 osd.1 (osd.1) 13 : cluster [DBG] 6.1d deep-scrub starts 2024-02-18T00:30:32.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:30.321326+0000 osd.1 (osd.1) 14 : cluster [DBG] 6.1d deep-scrub ok 2024-02-18T00:30:32.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:30.771281+0000 mgr.y (mgr.24454) 46 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:32.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:30.320452+0000 osd.1 (osd.1) 13 : cluster [DBG] 6.1d deep-scrub starts 2024-02-18T00:30:32.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:30.321326+0000 osd.1 (osd.1) 14 : cluster [DBG] 6.1d deep-scrub ok 2024-02-18T00:30:32.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:30.771281+0000 mgr.y (mgr.24454) 46 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:32.524 INFO:teuthology.orchestra.run.smithi059.stdout:55834574896 2024-02-18T00:30:32.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-18T00:30:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:31.633476+0000 osd.7 (osd.7) 17 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:30:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:31.634305+0000 osd.7 (osd.7) 18 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:30:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:31.730560+0000 mgr.y (mgr.24454) 47 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:30:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:31.809810+0000 osd.3 (osd.3) 17 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:30:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:31.810600+0000 osd.3 (osd.3) 18 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:32.026239+0000 osd.4 (osd.4) 17 : cluster [DBG] 4.e scrub starts 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:32.029748+0000 osd.4 (osd.4) 18 : cluster [DBG] 4.e scrub ok 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:32.363128+0000 osd.2 (osd.2) 13 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:32.363985+0000 osd.2 (osd.2) 14 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:32.420337+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:31.633476+0000 osd.7 (osd.7) 17 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:31.634305+0000 osd.7 (osd.7) 18 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:31.730560+0000 mgr.y (mgr.24454) 47 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:31.809810+0000 osd.3 (osd.3) 17 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:31.810600+0000 osd.3 (osd.3) 18 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:32.026239+0000 osd.4 (osd.4) 17 : cluster [DBG] 4.e scrub starts 2024-02-18T00:30:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:32.029748+0000 osd.4 (osd.4) 18 : cluster [DBG] 4.e scrub ok 2024-02-18T00:30:33.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:32.363128+0000 osd.2 (osd.2) 13 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:30:33.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:32.363985+0000 osd.2 (osd.2) 14 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:30:33.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:32.420337+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:31.633476+0000 osd.7 (osd.7) 17 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:31.634305+0000 osd.7 (osd.7) 18 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:31.730560+0000 mgr.y (mgr.24454) 47 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:31.809810+0000 osd.3 (osd.3) 17 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:31.810600+0000 osd.3 (osd.3) 18 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:32.026239+0000 osd.4 (osd.4) 17 : cluster [DBG] 4.e scrub starts 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:32.029748+0000 osd.4 (osd.4) 18 : cluster [DBG] 4.e scrub ok 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:32.363128+0000 osd.2 (osd.2) 13 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:30:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:32.363985+0000 osd.2 (osd.2) 14 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:30:33.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:32.420337+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:32.269272+0000 osd.1 (osd.1) 15 : cluster [DBG] 3.0 deep-scrub starts 2024-02-18T00:30:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:32.271177+0000 osd.1 (osd.1) 16 : cluster [DBG] 3.0 deep-scrub ok 2024-02-18T00:30:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:33.044271+0000 osd.4 (osd.4) 19 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:30:34.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:33.047322+0000 osd.4 (osd.4) 20 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:30:34.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:33.563529+0000 osd.7 (osd.7) 19 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:30:34.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:33.564513+0000 osd.7 (osd.7) 20 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:30:34.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:32.269272+0000 osd.1 (osd.1) 15 : cluster [DBG] 3.0 deep-scrub starts 2024-02-18T00:30:34.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:32.271177+0000 osd.1 (osd.1) 16 : cluster [DBG] 3.0 deep-scrub ok 2024-02-18T00:30:34.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:33.044271+0000 osd.4 (osd.4) 19 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:30:34.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:33.047322+0000 osd.4 (osd.4) 20 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:30:34.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:33.563529+0000 osd.7 (osd.7) 19 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:30:34.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:33.564513+0000 osd.7 (osd.7) 20 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:30:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:32.269272+0000 osd.1 (osd.1) 15 : cluster [DBG] 3.0 deep-scrub starts 2024-02-18T00:30:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:32.271177+0000 osd.1 (osd.1) 16 : cluster [DBG] 3.0 deep-scrub ok 2024-02-18T00:30:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:33.044271+0000 osd.4 (osd.4) 19 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:30:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:33.047322+0000 osd.4 (osd.4) 20 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:30:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:33.563529+0000 osd.7 (osd.7) 19 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:30:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:33.564513+0000 osd.7 (osd.7) 20 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:30:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:33.730873+0000 mgr.y (mgr.24454) 48 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T00:30:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:34.097235+0000 osd.5 (osd.5) 11 : cluster [DBG] 6.7 deep-scrub starts 2024-02-18T00:30:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:34.100170+0000 osd.5 (osd.5) 12 : cluster [DBG] 6.7 deep-scrub ok 2024-02-18T00:30:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:34.351683+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:34.352492+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:30:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:34.353116+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:30:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:34.357794+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:33.730873+0000 mgr.y (mgr.24454) 48 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T00:30:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:34.097235+0000 osd.5 (osd.5) 11 : cluster [DBG] 6.7 deep-scrub starts 2024-02-18T00:30:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:34.100170+0000 osd.5 (osd.5) 12 : cluster [DBG] 6.7 deep-scrub ok 2024-02-18T00:30:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:34.351683+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:34.352492+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:30:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:34.353116+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:30:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:34.357794+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:33.730873+0000 mgr.y (mgr.24454) 48 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:34.097235+0000 osd.5 (osd.5) 11 : cluster [DBG] 6.7 deep-scrub starts 2024-02-18T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:34.100170+0000 osd.5 (osd.5) 12 : cluster [DBG] 6.7 deep-scrub ok 2024-02-18T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:34.351683+0000 mon.a (mon.0) 698 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:34.352492+0000 mon.a (mon.0) 699 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:34.353116+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:30:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:34.357794+0000 mon.a (mon.0) 701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:30:34.361727+0000 mgr.y (mgr.24454) 49 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-18T00:30:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:30:34.364782+0000 mgr.y (mgr.24454) 50 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi059 2024-02-18T00:30:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:34.599080+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:34.599226+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.368698+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-18T00:30:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.531913+0000 osd.7 (osd.7) 21 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.533623+0000 osd.7 (osd.7) 22 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:30:34.361727+0000 mgr.y (mgr.24454) 49 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:30:34.364782+0000 mgr.y (mgr.24454) 50 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi059 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:34.599080+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:34.599226+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.368698+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.531913+0000 osd.7 (osd.7) 21 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:30:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.533623+0000 osd.7 (osd.7) 22 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:30:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:30:34.361727+0000 mgr.y (mgr.24454) 49 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-18T00:30:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:30:34.364782+0000 mgr.y (mgr.24454) 50 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi059 2024-02-18T00:30:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:34.599080+0000 osd.5 (osd.5) 13 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:34.599226+0000 osd.5 (osd.5) 14 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.368698+0000 mon.a (mon.0) 702 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-18T00:30:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.531913+0000 osd.7 (osd.7) 21 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:30:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.533623+0000 osd.7 (osd.7) 22 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:30:37.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.175923+0000 osd.1 (osd.1) 17 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:30:37.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.176985+0000 osd.1 (osd.1) 18 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:30:37.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.731578+0000 mgr.y (mgr.24454) 51 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:30:37.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.822105+0000 osd.3 (osd.3) 19 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:30:37.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:35.823583+0000 osd.3 (osd.3) 20 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:30:37.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:36.040238+0000 osd.4 (osd.4) 21 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:30:37.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:36.043206+0000 osd.4 (osd.4) 22 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:30:37.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.175923+0000 osd.1 (osd.1) 17 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:30:37.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.176985+0000 osd.1 (osd.1) 18 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:30:37.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.731578+0000 mgr.y (mgr.24454) 51 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:30:37.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.822105+0000 osd.3 (osd.3) 19 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:30:37.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:35.823583+0000 osd.3 (osd.3) 20 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:30:37.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:36.040238+0000 osd.4 (osd.4) 21 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:30:37.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:36.043206+0000 osd.4 (osd.4) 22 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:30:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.175923+0000 osd.1 (osd.1) 17 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:30:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.176985+0000 osd.1 (osd.1) 18 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:30:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.731578+0000 mgr.y (mgr.24454) 51 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:30:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.822105+0000 osd.3 (osd.3) 19 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:30:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:35.823583+0000 osd.3 (osd.3) 20 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:30:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:36.040238+0000 osd.4 (osd.4) 21 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:30:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:36.043206+0000 osd.4 (osd.4) 22 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:30:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 -- 172.21.15.59:0/3422688012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0fd510 msgr2=0x7f9d3c0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:38.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 --2- 172.21.15.59:0/3422688012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0fd510 0x7f9d3c0f6d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f9d300047b0 tx=0x7f9d30009cb0).stop 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 -- 172.21.15.59:0/3422688012 shutdown_connections 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 --2- 172.21.15.59:0/3422688012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d3c0fcbb0 0x7f9d3c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 --2- 172.21.15.59:0/3422688012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d3c0f7400 0x7f9d3c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 --2- 172.21.15.59:0/3422688012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0fd510 0x7f9d3c0f6d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 -- 172.21.15.59:0/3422688012 >> 172.21.15.59:0/3422688012 conn(0x7f9d3c0f2820 msgr2=0x7f9d3c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 -- 172.21.15.59:0/3422688012 shutdown_connections 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.191+0000 7f9d41298700 1 -- 172.21.15.59:0/3422688012 wait complete. 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.192+0000 7f9d41298700 1 librados: starting msgr at 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.192+0000 7f9d41298700 1 librados: starting objecter 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.192+0000 7f9d41298700 1 Processor -- start 2024-02-18T00:30:38.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.192+0000 7f9d41298700 1 -- start start 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.192+0000 7f9d41298700 1 librados: setting wanted keys 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.192+0000 7f9d41298700 1 librados: calling monclient init 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0f7400 0x7f9d3c10e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d3c0fcbb0 0x7f9d3c10e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d3c0fd510 0x7f9d3c108400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c1088f0 con 0x7f9d3c0f7400 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c108a60 con 0x7f9d3c0fcbb0 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3c108c20 con 0x7f9d3c0fd510 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0f7400 0x7f9d3c10e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d3c0fd510 0x7f9d3c108400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0f7400 0x7f9d3c10e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35714/0 (socket says 172.21.15.59:35714) 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 -- 172.21.15.59:0/1602576685 learned_addr learned my addr 172.21.15.59:0/1602576685 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:38.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3a7fc700 1 --2- 172.21.15.59:0/1602576685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d3c0fcbb0 0x7f9d3c10e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 -- 172.21.15.59:0/1602576685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d3c0fd510 msgr2=0x7f9d3c108400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 --2- 172.21.15.59:0/1602576685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d3c0fd510 0x7f9d3c108400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 -- 172.21.15.59:0/1602576685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d3c0fcbb0 msgr2=0x7f9d3c10e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 --2- 172.21.15.59:0/1602576685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d3c0fcbb0 0x7f9d3c10e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 -- 172.21.15.59:0/1602576685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d34005040 con 0x7f9d3c0f7400 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3b7fe700 1 --2- 172.21.15.59:0/1602576685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d3c0fd510 0x7f9d3c108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d3affd700 1 --2- 172.21.15.59:0/1602576685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0f7400 0x7f9d3c10e260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9d2c00cb90 tx=0x7f9d2c004d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d23fff700 10 client.?.objecter ms_handle_connect 0x7f9d3c0f7400 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d23fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00a7a0 con 0x7f9d3c0f7400 2024-02-18T00:30:38.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d30005040 con 0x7f9d3c0f7400 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d2c00add0 con 0x7f9d3c0f7400 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c012d90 con 0x7f9d3c0f7400 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 10 client.14607.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3c115090 con 0x7f9d3c0f7400 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.193+0000 7f9d41298700 1 librados: init done 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.194+0000 7f9d41298700 10 client.14607.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.194+0000 7f9d41298700 15 client.14607.objecter _session_command_op_assign -1 1 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.194+0000 7f9d41298700 20 client.14607.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.194+0000 7f9d41298700 10 client.14607.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.194+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d3c05fa60 con 0x7f9d3c0f7400 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.195+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9d2c012400 con 0x7f9d3c0f7400 2024-02-18T00:30:38.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.195+0000 7f9d23fff700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d24062c90 0x7f9d24065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.195+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6056+0+0 (secure 0 0 0) 0x7f9d2c07c670 con 0x7f9d3c0f7400 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.195+0000 7f9d23fff700 10 client.14607.objecter ms_dispatch 0x7f9d3c001550 osd_map(77..77 src has 1..77) v4 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.195+0000 7f9d23fff700 3 client.14607.objecter handle_osd_map got epochs [77,77] > 0 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.195+0000 7f9d23fff700 3 client.14607.objecter handle_osd_map decoding full epoch 77 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.195+0000 7f9d3a7fc700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d24062c90 0x7f9d24065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 10 client.14607.objecter checking command 1 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6809/1153193637 conn(0x7f9d2406c4b0 0x7f9d2406e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 20 client.14607.objecter _get_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 20 client.14607.objecter put_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 15 client.14607.objecter _session_command_op_remove -1 1 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 20 client.14607.objecter _get_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 20 client.14607.objecter get_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 15 client.14607.objecter _session_command_op_assign 2 1 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 20 client.14607.objecter put_session s=0x7f9d2406be00 osd=2 4 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 10 client.14607.objecter _send_command 1 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 --> v2:172.21.15.59:6809/1153193637 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d2406ef90 con 0x7f9d2406c4b0 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 20 client.14607.objecter dump_active .. 0 homeless 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d2c07ec90 con 0x7f9d3c0f7400 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 10 client.14607.objecter ms_handle_connect 0x7f9d24062c90 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d3a7fc700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d24062c90 0x7f9d24065150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f9d30009da0 tx=0x7f9d30009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d3b7fe700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6809/1153193637 conn(0x7f9d2406c4b0 0x7f9d2406e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d23fff700 10 client.14607.objecter ms_handle_connect 0x7f9d2406c4b0 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.196+0000 7f9d3b7fe700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6809/1153193637 conn(0x7f9d2406c4b0 0x7f9d2406e950 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:38.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.197+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9d2406ef90 con 0x7f9d2406c4b0 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.197+0000 7f9d23fff700 10 client.14607.objecter ms_dispatch 0x7f9d3c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.197+0000 7f9d23fff700 10 client.14607.objecter _finish_command 1 = system:0 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.197+0000 7f9d23fff700 20 client.14607.objecter put_session s=0x7f9d2406be00 osd=2 4 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.197+0000 7f9d23fff700 15 client.14607.objecter _session_command_op_remove 2 1 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 10 client.14607.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 15 client.14607.objecter _session_command_op_assign -1 2 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 20 client.14607.objecter _get_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 20 client.14607.objecter put_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 20 client.14607.objecter _get_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 15 client.14607.objecter _session_command_op_remove -1 2 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 20 client.14607.objecter get_session s=0x7f9d2406be00 osd=2 3 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 15 client.14607.objecter _session_command_op_assign 2 2 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 20 client.14607.objecter put_session s=0x7f9d2406be00 osd=2 4 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 10 client.14607.objecter _send_command 2 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 --> v2:172.21.15.59:6809/1153193637 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d3c05ee80 con 0x7f9d2406c4b0 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d23fff700 1 -- 172.21.15.59:0/1602576685 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d3c05ee80 con 0x7f9d2406c4b0 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d23fff700 10 client.14607.objecter ms_dispatch 0x7f9d3c001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d23fff700 10 client.14607.objecter _finish_command 2 = system:0 2024-02-18T00:30:38.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d23fff700 20 client.14607.objecter put_session s=0x7f9d2406be00 osd=2 4 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.211+0000 7f9d23fff700 15 client.14607.objecter _session_command_op_remove 2 2 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 10 librados: watch_flush enter 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 10 librados: watch_flush exit 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 10 client.14607.objecter close_session for osd.2 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6809/1153193637 conn(0x7f9d2406c4b0 msgr2=0x7f9d2406e950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6809/1153193637 conn(0x7f9d2406c4b0 0x7f9d2406e950 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 20 client.14607.objecter put_session s=0x7f9d2406be00 osd=2 1 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 20 client.14607.objecter shutdown clearing up homeless session... 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 10 client.14607.objecter successfully canceled tick 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d24062c90 msgr2=0x7f9d24065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 --2- 172.21.15.59:0/1602576685 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d24062c90 0x7f9d24065150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f9d30009da0 tx=0x7f9d30009cb0).stop 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0f7400 msgr2=0x7f9d3c10e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 --2- 172.21.15.59:0/1602576685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d3c0f7400 0x7f9d3c10e260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f9d2c00cb90 tx=0x7f9d2c004d70).stop 2024-02-18T00:30:38.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d3b7fe700 1 -- 172.21.15.59:0/1602576685 reap_dead start 2024-02-18T00:30:38.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 shutdown_connections 2024-02-18T00:30:38.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 >> 172.21.15.59:0/1602576685 conn(0x7f9d3c0f2820 msgr2=0x7f9d3c060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:38.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 shutdown_connections 2024-02-18T00:30:38.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 -- 172.21.15.59:0/1602576685 wait complete. 2024-02-18T00:30:38.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:38.212+0000 7f9d41298700 1 librados: shutdown 2024-02-18T00:30:38.971 INFO:teuthology.orchestra.run.smithi059.stdout:77309411374 2024-02-18T00:30:38.971 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-18T00:30:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:37.537688+0000 osd.6 (osd.6) 13 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:30:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:37.538696+0000 osd.6 (osd.6) 14 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:30:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:37.731937+0000 mgr.y (mgr.24454) 52 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.055128+0000 osd.4 (osd.4) 23 : cluster [DBG] 2.f scrub starts 2024-02-18T00:30:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.058591+0000 osd.4 (osd.4) 24 : cluster [DBG] 2.f scrub ok 2024-02-18T00:30:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.315961+0000 osd.2 (osd.2) 15 : cluster [DBG] 2.c scrub starts 2024-02-18T00:30:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.316955+0000 osd.2 (osd.2) 16 : cluster [DBG] 2.c scrub ok 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.558524+0000 osd.7 (osd.7) 23 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.560080+0000 osd.7 (osd.7) 24 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:37.537688+0000 osd.6 (osd.6) 13 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:37.538696+0000 osd.6 (osd.6) 14 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:37.731937+0000 mgr.y (mgr.24454) 52 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.055128+0000 osd.4 (osd.4) 23 : cluster [DBG] 2.f scrub starts 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.058591+0000 osd.4 (osd.4) 24 : cluster [DBG] 2.f scrub ok 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.315961+0000 osd.2 (osd.2) 15 : cluster [DBG] 2.c scrub starts 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.316955+0000 osd.2 (osd.2) 16 : cluster [DBG] 2.c scrub ok 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.558524+0000 osd.7 (osd.7) 23 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:30:39.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.560080+0000 osd.7 (osd.7) 24 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:37.537688+0000 osd.6 (osd.6) 13 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:37.538696+0000 osd.6 (osd.6) 14 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:37.731937+0000 mgr.y (mgr.24454) 52 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.055128+0000 osd.4 (osd.4) 23 : cluster [DBG] 2.f scrub starts 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.058591+0000 osd.4 (osd.4) 24 : cluster [DBG] 2.f scrub ok 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.315961+0000 osd.2 (osd.2) 15 : cluster [DBG] 2.c scrub starts 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.316955+0000 osd.2 (osd.2) 16 : cluster [DBG] 2.c scrub ok 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.558524+0000 osd.7 (osd.7) 23 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:30:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.560080+0000 osd.7 (osd.7) 24 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:30:39.927 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:40.160 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:39 smithi059 systemd[1]: Stopping Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:30:40.160 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:39 smithi059 bash[70860]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager.a" found: no such container 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.506668+0000 osd.6 (osd.6) 15 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.507634+0000 osd.6 (osd.6) 16 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.893902+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:38.895649+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:39.298442+0000 osd.2 (osd.2) 17 : cluster [DBG] 5.a scrub starts 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:39.299274+0000 osd.2 (osd.2) 18 : cluster [DBG] 5.a scrub ok 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:39.624858+0000 mon.a (mon.0) 703 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-18T00:30:40.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:39.726561+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-02-18T00:30:40.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:39.726773+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-18T00:30:40.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:39.726926+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-02-18T00:30:40.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:39.727072+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-02-18T00:30:40.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:40.078039+0000 osd.1 (osd.1) 19 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:30:40.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:40.080923+0000 osd.1 (osd.1) 20 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:30:40.162 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.506668+0000 osd.6 (osd.6) 15 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.507634+0000 osd.6 (osd.6) 16 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.893902+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:38.895649+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:39.298442+0000 osd.2 (osd.2) 17 : cluster [DBG] 5.a scrub starts 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:39.299274+0000 osd.2 (osd.2) 18 : cluster [DBG] 5.a scrub ok 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:39.624858+0000 mon.a (mon.0) 703 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:39.726561+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-02-18T00:30:40.163 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:39.726773+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-18T00:30:40.164 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:39.726926+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-02-18T00:30:40.164 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:39.727072+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-02-18T00:30:40.164 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:40.078039+0000 osd.1 (osd.1) 19 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:30:40.164 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:40.080923+0000 osd.1 (osd.1) 20 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:30:40.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[67300]: level=info ts=2024-02-18T00:30:40.366Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-18T00:30:40.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.506668+0000 osd.6 (osd.6) 15 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.507634+0000 osd.6 (osd.6) 16 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.893902+0000 osd.0 (osd.0) 17 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:38.895649+0000 osd.0 (osd.0) 18 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:39.298442+0000 osd.2 (osd.2) 17 : cluster [DBG] 5.a scrub starts 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:39.299274+0000 osd.2 (osd.2) 18 : cluster [DBG] 5.a scrub ok 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:39.624858+0000 mon.a (mon.0) 703 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:39.726561+0000 mon.a (mon.0) 704 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]: dispatch 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:39.726773+0000 mon.a (mon.0) 705 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:39.726926+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]: dispatch 2024-02-18T00:30:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:39.727072+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-02-18T00:30:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:40.078039+0000 osd.1 (osd.1) 19 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:30:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:40.080923+0000 osd.1 (osd.1) 20 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:30:41.062 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:30:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:30:41.359 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:41 smithi059 bash[70931]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a 2024-02-18T00:30:41.359 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:39.732309+0000 mgr.y (mgr.24454) 53 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T00:30:41.359 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:39.826046+0000 osd.3 (osd.3) 21 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:30:41.359 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:39.827715+0000 osd.3 (osd.3) 22 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:30:41.359 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:40.054252+0000 osd.5 (osd.5) 15 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:30:41.359 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:40.055244+0000 osd.5 (osd.5) 16 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:30:41.359 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:40.634796+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:40.634878+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:40.634942+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:40.635005+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:40.635035+0000 mon.a (mon.0) 712 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.038587+0000 osd.1 (osd.1) 21 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.040591+0000 osd.1 (osd.1) 22 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:39.732309+0000 mgr.y (mgr.24454) 53 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:39.826046+0000 osd.3 (osd.3) 21 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:30:41.360 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:39.827715+0000 osd.3 (osd.3) 22 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:40.054252+0000 osd.5 (osd.5) 15 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:40.055244+0000 osd.5 (osd.5) 16 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:40.634796+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:40.634878+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:40.634942+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:40.635005+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:40.635035+0000 mon.a (mon.0) 712 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.038587+0000 osd.1 (osd.1) 21 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:30:41.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.040591+0000 osd.1 (osd.1) 22 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:30:41.505 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:30:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:39.732309+0000 mgr.y (mgr.24454) 53 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T00:30:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:39.826046+0000 osd.3 (osd.3) 21 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:30:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:39.827715+0000 osd.3 (osd.3) 22 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:40.054252+0000 osd.5 (osd.5) 15 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:40.055244+0000 osd.5 (osd.5) 16 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:40.634796+0000 mon.a (mon.0) 708 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [7, 2]}]': finished 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:40.634878+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:40.634942+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 5]}]': finished 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:40.635005+0000 mon.a (mon.0) 711 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:40.635035+0000 mon.a (mon.0) 712 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-18T00:30:41.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.038587+0000 osd.1 (osd.1) 21 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:30:41.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.040591+0000 osd.1 (osd.1) 22 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:30:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:40.782214+0000 mgr.y (mgr.24454) 54 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:40.853952+0000 osd.3 (osd.3) 23 : cluster [DBG] 6.c scrub starts 2024-02-18T00:30:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:40.854923+0000 osd.3 (osd.3) 24 : cluster [DBG] 6.c scrub ok 2024-02-18T00:30:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.062230+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.19 deep-scrub starts 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.063233+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.19 deep-scrub ok 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.298323+0000 osd.2 (osd.2) 19 : cluster [DBG] 6.f scrub starts 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.301401+0000 osd.2 (osd.2) 20 : cluster [DBG] 6.f scrub ok 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.635549+0000 mon.a (mon.0) 713 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:40.782214+0000 mgr.y (mgr.24454) 54 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:40.853952+0000 osd.3 (osd.3) 23 : cluster [DBG] 6.c scrub starts 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:40.854923+0000 osd.3 (osd.3) 24 : cluster [DBG] 6.c scrub ok 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.062230+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.19 deep-scrub starts 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.063233+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.19 deep-scrub ok 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.298323+0000 osd.2 (osd.2) 19 : cluster [DBG] 6.f scrub starts 2024-02-18T00:30:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.301401+0000 osd.2 (osd.2) 20 : cluster [DBG] 6.f scrub ok 2024-02-18T00:30:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.635549+0000 mon.a (mon.0) 713 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-18T00:30:42.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:42] "GET /metrics HTTP/1.1" 200 172145 "" "Prometheus/2.33.4" 2024-02-18T00:30:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:40.782214+0000 mgr.y (mgr.24454) 54 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:40.853952+0000 osd.3 (osd.3) 23 : cluster [DBG] 6.c scrub starts 2024-02-18T00:30:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:40.854923+0000 osd.3 (osd.3) 24 : cluster [DBG] 6.c scrub ok 2024-02-18T00:30:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.062230+0000 osd.5 (osd.5) 17 : cluster [DBG] 6.19 deep-scrub starts 2024-02-18T00:30:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.063233+0000 osd.5 (osd.5) 18 : cluster [DBG] 6.19 deep-scrub ok 2024-02-18T00:30:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.298323+0000 osd.2 (osd.2) 19 : cluster [DBG] 6.f scrub starts 2024-02-18T00:30:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.301401+0000 osd.2 (osd.2) 20 : cluster [DBG] 6.f scrub ok 2024-02-18T00:30:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.635549+0000 mon.a (mon.0) 713 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-18T00:30:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 -- 172.21.15.59:0/2481325540 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc40f8cb0 msgr2=0x7fbdc40f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 --2- 172.21.15.59:0/2481325540 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc40f8cb0 0x7fbdc40f9090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fbdb40047b0 tx=0x7fbdb4009cb0).stop 2024-02-18T00:30:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 -- 172.21.15.59:0/2481325540 shutdown_connections 2024-02-18T00:30:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 --2- 172.21.15.59:0/2481325540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdc4102210 0x7fbdc4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 --2- 172.21.15.59:0/2481325540 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc40f8cb0 0x7fbdc40f9090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 --2- 172.21.15.59:0/2481325540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc40f9610 0x7fbdc4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:42.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 -- 172.21.15.59:0/2481325540 >> 172.21.15.59:0/2481325540 conn(0x7fbdc40f2820 msgr2=0x7fbdc40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 -- 172.21.15.59:0/2481325540 shutdown_connections 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.846+0000 7fbdca57e700 1 -- 172.21.15.59:0/2481325540 wait complete. 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 librados: starting msgr at 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 librados: starting objecter 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 Processor -- start 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 -- start start 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 librados: setting wanted keys 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 librados: calling monclient init 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdc40f8cb0 0x7fbdc40f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc40f9610 0x7fbdc40f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc4102210 0x7fbdc40f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc41166b0 con 0x7fbdc40f9610 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc4116820 con 0x7fbdc40f8cb0 2024-02-18T00:30:42.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.847+0000 7fbdca57e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc41169e0 con 0x7fbdc4102210 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc40f9610 0x7fbdc40f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc8b1b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc4102210 0x7fbdc40f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc40f9610 0x7fbdc40f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55566/0 (socket says 172.21.15.59:55566) 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc37fe700 1 -- 172.21.15.59:0/3984975553 learned_addr learned my addr 172.21.15.59:0/3984975553 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc3fff700 1 --2- 172.21.15.59:0/3984975553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdc40f8cb0 0x7fbdc40f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc8b1b700 1 -- 172.21.15.59:0/3984975553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdc40f8cb0 msgr2=0x7fbdc40f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc8b1b700 1 --2- 172.21.15.59:0/3984975553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdc40f8cb0 0x7fbdc40f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc8b1b700 1 -- 172.21.15.59:0/3984975553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc40f9610 msgr2=0x7fbdc40f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc8b1b700 1 --2- 172.21.15.59:0/3984975553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc40f9610 0x7fbdc40f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc8b1b700 1 -- 172.21.15.59:0/3984975553 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8005040 con 0x7fbdc4102210 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc37fe700 1 --2- 172.21.15.59:0/3984975553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc40f9610 0x7fbdc40f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc8b1b700 1 --2- 172.21.15.59:0/3984975553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc4102210 0x7fbdc40f7640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fbdbc00eb40 tx=0x7fbdbc00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:42.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc17fa700 10 client.?.objecter ms_handle_connect 0x7fbdc4102210 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc17fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc009870 con 0x7fbdc4102210 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdbc00f040 con 0x7fbdc4102210 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb4005040 con 0x7fbdc4102210 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc007d60 con 0x7fbdc4102210 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdca57e700 10 client.24518.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc4116f70 con 0x7fbdc4102210 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.848+0000 7fbdca57e700 1 librados: init done 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.849+0000 7fbdca57e700 10 client.24518.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.849+0000 7fbdca57e700 15 client.24518.objecter _session_command_op_assign -1 1 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.849+0000 7fbdca57e700 20 client.24518.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.849+0000 7fbdca57e700 10 client.24518.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.849+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbdc405fa60 con 0x7fbdc4102210 2024-02-18T00:30:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.849+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fbdbc0040b0 con 0x7fbdc4102210 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbdac062c90 0x7fbdac065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fbdbc07c760 con 0x7fbdc4102210 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 10 client.24518.objecter ms_dispatch 0x7fbdc4001550 osd_map(80..80 src has 1..80) v4 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 3 client.24518.objecter handle_osd_map got epochs [80,80] > 0 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 3 client.24518.objecter handle_osd_map decoding full epoch 80 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc3fff700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbdac062c90 0x7fbdac065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 10 client.24518.objecter checking command 1 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6813/1805700964 conn(0x7fbdac06c650 0x7fbdac06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 20 client.24518.objecter _get_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 20 client.24518.objecter put_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 15 client.24518.objecter _session_command_op_remove -1 1 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 20 client.24518.objecter _get_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 20 client.24518.objecter get_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 15 client.24518.objecter _session_command_op_assign 3 1 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 20 client.24518.objecter put_session s=0x7fbdac06bfa0 osd=3 4 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 10 client.24518.objecter _send_command 1 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 --> v2:172.21.15.59:6813/1805700964 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdac06f130 con 0x7fbdac06c650 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 20 client.24518.objecter dump_active .. 0 homeless 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdbc07ed80 con 0x7fbdc4102210 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc37fe700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6813/1805700964 conn(0x7fbdac06c650 0x7fbdac06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 10 client.24518.objecter ms_handle_connect 0x7fbdac062c90 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc3fff700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbdac062c90 0x7fbdac065150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fbdb4005010 tx=0x7fbdb4009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc37fe700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6813/1805700964 conn(0x7fbdac06c650 0x7fbdac06eaf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.850+0000 7fbdc17fa700 10 client.24518.objecter ms_handle_connect 0x7fbdac06c650 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.851+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fbdac06f130 con 0x7fbdac06c650 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.851+0000 7fbdc17fa700 10 client.24518.objecter ms_dispatch 0x7fbdc4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.851+0000 7fbdc17fa700 10 client.24518.objecter _finish_command 1 = system:0 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.851+0000 7fbdc17fa700 20 client.24518.objecter put_session s=0x7fbdac06bfa0 osd=3 4 2024-02-18T00:30:42.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.851+0000 7fbdc17fa700 15 client.24518.objecter _session_command_op_remove 3 1 2024-02-18T00:30:42.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 10 client.24518.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:42.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 15 client.24518.objecter _session_command_op_assign -1 2 2024-02-18T00:30:42.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 20 client.24518.objecter _get_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 20 client.24518.objecter put_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 20 client.24518.objecter _get_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 15 client.24518.objecter _session_command_op_remove -1 2 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 20 client.24518.objecter get_session s=0x7fbdac06bfa0 osd=3 3 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 15 client.24518.objecter _session_command_op_assign 3 2 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 20 client.24518.objecter put_session s=0x7fbdac06bfa0 osd=3 4 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 10 client.24518.objecter _send_command 2 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.865+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 --> v2:172.21.15.59:6813/1805700964 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbdc405ee80 con 0x7fbdac06c650 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdc17fa700 1 -- 172.21.15.59:0/3984975553 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbdc405ee80 con 0x7fbdac06c650 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdc17fa700 10 client.24518.objecter ms_dispatch 0x7fbdc4001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdc17fa700 10 client.24518.objecter _finish_command 2 = system:0 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdc17fa700 20 client.24518.objecter put_session s=0x7fbdac06bfa0 osd=3 4 2024-02-18T00:30:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdc17fa700 15 client.24518.objecter _session_command_op_remove 3 2 2024-02-18T00:30:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 10 librados: watch_flush enter 2024-02-18T00:30:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 10 librados: watch_flush exit 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 10 client.24518.objecter close_session for osd.3 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6813/1805700964 conn(0x7fbdac06c650 msgr2=0x7fbdac06eaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6813/1805700964 conn(0x7fbdac06c650 0x7fbdac06eaf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 20 client.24518.objecter put_session s=0x7fbdac06bfa0 osd=3 1 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 20 client.24518.objecter shutdown clearing up homeless session... 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 10 client.24518.objecter successfully canceled tick 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbdac062c90 msgr2=0x7fbdac065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 --2- 172.21.15.59:0/3984975553 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbdac062c90 0x7fbdac065150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fbdb4005010 tx=0x7fbdb4009cb0).stop 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc4102210 msgr2=0x7fbdc40f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 --2- 172.21.15.59:0/3984975553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdc4102210 0x7fbdc40f7640 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fbdbc00eb40 tx=0x7fbdbc00b040).stop 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdc8b1b700 1 -- 172.21.15.59:0/3984975553 reap_dead start 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 shutdown_connections 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 >> 172.21.15.59:0/3984975553 conn(0x7fbdc40f2820 msgr2=0x7fbdc40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 shutdown_connections 2024-02-18T00:30:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 -- 172.21.15.59:0/3984975553 wait complete. 2024-02-18T00:30:42.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:42.866+0000 7fbdca57e700 1 librados: shutdown 2024-02-18T00:30:42.888 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:42 smithi059 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-18T00:30:42.888 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:42 smithi059 systemd[1]: Stopped Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:30:42.888 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:42 smithi059 systemd[1]: Starting Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:30:43.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.538099+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:30:43.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.540000+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:30:43.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:41.732655+0000 mgr.y (mgr.24454) 55 : cluster [DBG] pgmap v49: 132 pgs: 4 peering, 128 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:30:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.538099+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:30:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.540000+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:30:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:41.732655+0000 mgr.y (mgr.24454) 55 : cluster [DBG] pgmap v49: 132 pgs: 4 peering, 128 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:30:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.538099+0000 osd.6 (osd.6) 17 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:30:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.540000+0000 osd.6 (osd.6) 18 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:30:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:41.732655+0000 mgr.y (mgr.24454) 55 : cluster [DBG] pgmap v49: 132 pgs: 4 peering, 128 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.865Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.865Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.867Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.59 port=9094 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.868Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.895Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.895Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.898Z caller=main.go:518 msg=Listening address=:9093 2024-02-18T00:30:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:43.898Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:42.768627+0000 osd.4 (osd.4) 25 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:42.768666+0000 osd.4 (osd.4) 26 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.115560+0000 osd.1 (osd.1) 23 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.115614+0000 osd.1 (osd.1) 24 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.220626+0000 osd.2 (osd.2) 21 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.221536+0000 osd.2 (osd.2) 22 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.540309+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.541451+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:30:44.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.643005+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:42.768627+0000 osd.4 (osd.4) 25 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:42.768666+0000 osd.4 (osd.4) 26 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.115560+0000 osd.1 (osd.1) 23 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.115614+0000 osd.1 (osd.1) 24 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.220626+0000 osd.2 (osd.2) 21 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.221536+0000 osd.2 (osd.2) 22 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.540309+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.541451+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:30:44.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.643005+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-18T00:30:44.386 INFO:teuthology.orchestra.run.smithi059.stdout:111669149737 2024-02-18T00:30:44.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:42.768627+0000 osd.4 (osd.4) 25 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:42.768666+0000 osd.4 (osd.4) 26 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.115560+0000 osd.1 (osd.1) 23 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.115614+0000 osd.1 (osd.1) 24 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.220626+0000 osd.2 (osd.2) 21 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.221536+0000 osd.2 (osd.2) 22 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.540309+0000 osd.7 (osd.7) 25 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.541451+0000 osd.7 (osd.7) 26 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:30:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.643005+0000 mon.a (mon.0) 714 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-18T00:30:44.647 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:44 smithi059 bash[71147]: 3a24148431bcf197ecf33039d4045a2736cdd139867bda72295fab2d55c211ea 2024-02-18T00:30:44.647 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:44 smithi059 systemd[1]: Started Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:30:44.647 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STOPPING 2024-02-18T00:30:44.647 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:30:44.647 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STOPPED 2024-02-18T00:30:44.647 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STARTING 2024-02-18T00:30:44.647 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Serving on http://:::9283 2024-02-18T00:30:44.647 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STARTED 2024-02-18T00:30:44.648 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STOPPING 2024-02-18T00:30:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:44.993 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:30:44.993 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STOPPED 2024-02-18T00:30:44.993 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STARTING 2024-02-18T00:30:44.993 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Serving on http://:::9283 2024-02-18T00:30:44.993 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STARTED 2024-02-18T00:30:44.993 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STOPPING 2024-02-18T00:30:44.994 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:30:44.994 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STOPPED 2024-02-18T00:30:44.994 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STARTING 2024-02-18T00:30:44.994 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Serving on http://:::9283 2024-02-18T00:30:44.994 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:30:44] ENGINE Bus STARTED 2024-02-18T00:30:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.733036+0000 mgr.y (mgr.24454) 56 : cluster [DBG] pgmap v51: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-02-18T00:30:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.817549+0000 osd.3 (osd.3) 25 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:30:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:43.819430+0000 osd.3 (osd.3) 26 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:30:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:44.100977+0000 osd.4 (osd.4) 27 : cluster [DBG] 6.e scrub starts 2024-02-18T00:30:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:44.103895+0000 osd.4 (osd.4) 28 : cluster [DBG] 6.e scrub ok 2024-02-18T00:30:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:44.492437+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:30:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:44.493864+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.510371+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.513646+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.514824+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi059.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.519136+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.523981+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.526627+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.531289+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.535717+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.539042+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.544591+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.548059+0000 mon.a (mon.0) 725 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:44.564662+0000 osd.7 (osd.7) 27 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:44.565621+0000 osd.7 (osd.7) 28 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:30:45.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:44.627303+0000 mon.a (mon.0) 726 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.733036+0000 mgr.y (mgr.24454) 56 : cluster [DBG] pgmap v51: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.817549+0000 osd.3 (osd.3) 25 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:43.819430+0000 osd.3 (osd.3) 26 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:44.100977+0000 osd.4 (osd.4) 27 : cluster [DBG] 6.e scrub starts 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:44.103895+0000 osd.4 (osd.4) 28 : cluster [DBG] 6.e scrub ok 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:44.492437+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:44.493864+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.510371+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.513646+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.514824+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi059.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.519136+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.523981+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.526627+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.531289+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.535717+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.539042+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.544591+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.548059+0000 mon.a (mon.0) 725 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:44.564662+0000 osd.7 (osd.7) 27 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:44.565621+0000 osd.7 (osd.7) 28 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:30:45.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:44.627303+0000 mon.a (mon.0) 726 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-18T00:30:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.733036+0000 mgr.y (mgr.24454) 56 : cluster [DBG] pgmap v51: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-02-18T00:30:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.817549+0000 osd.3 (osd.3) 25 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:30:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:43.819430+0000 osd.3 (osd.3) 26 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:30:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:44.100977+0000 osd.4 (osd.4) 27 : cluster [DBG] 6.e scrub starts 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:44.103895+0000 osd.4 (osd.4) 28 : cluster [DBG] 6.e scrub ok 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:44.492437+0000 osd.6 (osd.6) 19 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:44.493864+0000 osd.6 (osd.6) 20 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.510371+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.513646+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.514824+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi059.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.519136+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.523981+0000 mon.a (mon.0) 719 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.526627+0000 mon.a (mon.0) 720 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.531289+0000 mon.a (mon.0) 721 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.535717+0000 mon.a (mon.0) 722 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.539042+0000 mon.a (mon.0) 723 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.544591+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.548059+0000 mon.a (mon.0) 725 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:44.564662+0000 osd.7 (osd.7) 27 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:30:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:44.565621+0000 osd.7 (osd.7) 28 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:30:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:44.627303+0000 mon.a (mon.0) 726 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-18T00:30:46.122 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:45.869Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000662672s 2024-02-18T00:30:46.493 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.513994+0000 mgr.y (mgr.24454) 57 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T00:30:46.493 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.515074+0000 mgr.y (mgr.24454) 58 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi059.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.524355+0000 mgr.y (mgr.24454) 59 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.527015+0000 mgr.y (mgr.24454) 60 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.536104+0000 mgr.y (mgr.24454) 61 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:44.539356+0000 mgr.y (mgr.24454) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.049224+0000 osd.5 (osd.5) 19 : cluster [DBG] 4.f deep-scrub starts 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.055393+0000 osd.5 (osd.5) 20 : cluster [DBG] 4.f deep-scrub ok 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.061447+0000 osd.4 (osd.4) 29 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.062324+0000 osd.4 (osd.4) 30 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.272951+0000 osd.2 (osd.2) 23 : cluster [DBG] 5.b scrub starts 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.273916+0000 osd.2 (osd.2) 24 : cluster [DBG] 5.b scrub ok 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.462760+0000 osd.6 (osd.6) 21 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.463622+0000 osd.6 (osd.6) 22 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.529018+0000 osd.7 (osd.7) 29 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:30:46.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.530287+0000 osd.7 (osd.7) 30 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.984704+0000 osd.1 (osd.1) 25 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.987656+0000 osd.1 (osd.1) 26 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.513994+0000 mgr.y (mgr.24454) 57 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.515074+0000 mgr.y (mgr.24454) 58 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi059.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.524355+0000 mgr.y (mgr.24454) 59 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.527015+0000 mgr.y (mgr.24454) 60 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.536104+0000 mgr.y (mgr.24454) 61 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:44.539356+0000 mgr.y (mgr.24454) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.049224+0000 osd.5 (osd.5) 19 : cluster [DBG] 4.f deep-scrub starts 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.055393+0000 osd.5 (osd.5) 20 : cluster [DBG] 4.f deep-scrub ok 2024-02-18T00:30:46.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.061447+0000 osd.4 (osd.4) 29 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.062324+0000 osd.4 (osd.4) 30 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.272951+0000 osd.2 (osd.2) 23 : cluster [DBG] 5.b scrub starts 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.273916+0000 osd.2 (osd.2) 24 : cluster [DBG] 5.b scrub ok 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.462760+0000 osd.6 (osd.6) 21 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.463622+0000 osd.6 (osd.6) 22 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.529018+0000 osd.7 (osd.7) 29 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.530287+0000 osd.7 (osd.7) 30 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.984704+0000 osd.1 (osd.1) 25 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:30:46.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.987656+0000 osd.1 (osd.1) 26 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.513994+0000 mgr.y (mgr.24454) 57 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.515074+0000 mgr.y (mgr.24454) 58 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi059.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.524355+0000 mgr.y (mgr.24454) 59 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.527015+0000 mgr.y (mgr.24454) 60 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi137.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.536104+0000 mgr.y (mgr.24454) 61 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:44.539356+0000 mgr.y (mgr.24454) 62 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi137.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.049224+0000 osd.5 (osd.5) 19 : cluster [DBG] 4.f deep-scrub starts 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.055393+0000 osd.5 (osd.5) 20 : cluster [DBG] 4.f deep-scrub ok 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.061447+0000 osd.4 (osd.4) 29 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.062324+0000 osd.4 (osd.4) 30 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.272951+0000 osd.2 (osd.2) 23 : cluster [DBG] 5.b scrub starts 2024-02-18T00:30:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.273916+0000 osd.2 (osd.2) 24 : cluster [DBG] 5.b scrub ok 2024-02-18T00:30:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.462760+0000 osd.6 (osd.6) 21 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:30:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.463622+0000 osd.6 (osd.6) 22 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:30:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.529018+0000 osd.7 (osd.7) 29 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:30:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.530287+0000 osd.7 (osd.7) 30 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:30:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.984704+0000 osd.1 (osd.1) 25 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:30:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.987656+0000 osd.1 (osd.1) 26 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:30:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.759+0000 7f398fb67700 1 -- 172.21.15.59:0/952508662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fcbb0 msgr2=0x7f39880fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.759+0000 7f398fb67700 1 --2- 172.21.15.59:0/952508662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fcbb0 0x7f39880fcf90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3978004870 tx=0x7f3978009cb0).stop 2024-02-18T00:30:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.759+0000 7f398fb67700 1 -- 172.21.15.59:0/952508662 shutdown_connections 2024-02-18T00:30:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 --2- 172.21.15.59:0/952508662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39880f7400 0x7f3988103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 --2- 172.21.15.59:0/952508662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fcbb0 0x7f39880fcf90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 --2- 172.21.15.59:0/952508662 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39880fd510 0x7f39880f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 -- 172.21.15.59:0/952508662 >> 172.21.15.59:0/952508662 conn(0x7f39880f2820 msgr2=0x7f39880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 -- 172.21.15.59:0/952508662 shutdown_connections 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 -- 172.21.15.59:0/952508662 wait complete. 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 librados: starting msgr at 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 librados: starting objecter 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 Processor -- start 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 -- start start 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 librados: setting wanted keys 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 librados: calling monclient init 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39880f7400 0x7f39880f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39880fcbb0 0x7f39880f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fd510 0x7f39880f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39881166d0 con 0x7f39880fd510 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988116840 con 0x7f39880fcbb0 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.760+0000 7f398fb67700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3988116a00 con 0x7f39880f7400 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39880f7400 0x7f39880f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398e104700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fd510 0x7f39880f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39880f7400 0x7f39880f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55744/0 (socket says 172.21.15.59:55744) 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398e104700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fd510 0x7f39880f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55598/0 (socket says 172.21.15.59:55598) 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 -- 172.21.15.59:0/1430317292 learned_addr learned my addr 172.21.15.59:0/1430317292 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d102700 1 --2- 172.21.15.59:0/1430317292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39880fcbb0 0x7f39880f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 -- 172.21.15.59:0/1430317292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39880fcbb0 msgr2=0x7f39880f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 --2- 172.21.15.59:0/1430317292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39880fcbb0 0x7f39880f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 -- 172.21.15.59:0/1430317292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fd510 msgr2=0x7f39880f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 --2- 172.21.15.59:0/1430317292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fd510 0x7f39880f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 -- 172.21.15.59:0/1430317292 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c005040 con 0x7f39880f7400 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398e104700 1 --2- 172.21.15.59:0/1430317292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39880fd510 0x7f39880f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398d903700 1 --2- 172.21.15.59:0/1430317292 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39880f7400 0x7f39880f89b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3978009eb0 tx=0x7f397800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f3986ffd700 10 client.?.objecter ms_handle_connect 0x7f39880f7400 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f3986ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397800e070 con 0x7f39880f7400 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3978005040 con 0x7f39880f7400 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3978013da0 con 0x7f39880f7400 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978019070 con 0x7f39880f7400 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398fb67700 10 client.24524.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3988116ef0 con 0x7f39880f7400 2024-02-18T00:30:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.761+0000 7f398fb67700 1 librados: init done 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.762+0000 7f398fb67700 10 client.24524.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.762+0000 7f398fb67700 15 client.24524.objecter _session_command_op_assign -1 1 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.762+0000 7f398fb67700 20 client.24524.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.762+0000 7f398fb67700 10 client.24524.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.762+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f398805fa60 con 0x7f39880f7400 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f397801e420 con 0x7f39880f7400 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3970062c90 0x7f3970065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:46.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f397807d510 con 0x7f39880f7400 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f398d102700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3970062c90 0x7f3970065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 10 client.24524.objecter ms_dispatch 0x7f3988001550 osd_map(82..82 src has 1..82) v4 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 3 client.24524.objecter handle_osd_map got epochs [82,82] > 0 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 3 client.24524.objecter handle_osd_map decoding full epoch 82 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 10 client.24524.objecter checking command 1 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.137:6800/2044387056 conn(0x7f397006c650 0x7f397006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 20 client.24524.objecter _get_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 20 client.24524.objecter put_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 15 client.24524.objecter _session_command_op_remove -1 1 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 20 client.24524.objecter _get_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 20 client.24524.objecter get_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 15 client.24524.objecter _session_command_op_assign 4 1 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 20 client.24524.objecter put_session s=0x7f397006bfa0 osd=4 4 2024-02-18T00:30:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 10 client.24524.objecter _send_command 1 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 --> v2:172.21.15.137:6800/2044387056 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f397006f130 con 0x7f397006c650 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 20 client.24524.objecter dump_active .. 0 homeless 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.763+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f397807fb30 con 0x7f39880f7400 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.764+0000 7f398e104700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.137:6800/2044387056 conn(0x7f397006c650 0x7f397006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.764+0000 7f3986ffd700 10 client.24524.objecter ms_handle_connect 0x7f397006c650 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.764+0000 7f398e104700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.137:6800/2044387056 conn(0x7f397006c650 0x7f397006eaf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.764+0000 7f3986ffd700 10 client.24524.objecter ms_handle_connect 0x7f3970062c90 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.764+0000 7f398d102700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3970062c90 0x7f3970065150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f39880fa100 tx=0x7f397c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.765+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f397006f130 con 0x7f397006c650 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.765+0000 7f3986ffd700 10 client.24524.objecter ms_dispatch 0x7f3988001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.765+0000 7f3986ffd700 10 client.24524.objecter _finish_command 1 = system:0 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.765+0000 7f3986ffd700 20 client.24524.objecter put_session s=0x7f397006bfa0 osd=4 4 2024-02-18T00:30:46.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.765+0000 7f3986ffd700 15 client.24524.objecter _session_command_op_remove 4 1 2024-02-18T00:30:46.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 10 client.24524.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:46.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 15 client.24524.objecter _session_command_op_assign -1 2 2024-02-18T00:30:46.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 20 client.24524.objecter _get_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 20 client.24524.objecter put_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 20 client.24524.objecter _get_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 15 client.24524.objecter _session_command_op_remove -1 2 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 20 client.24524.objecter get_session s=0x7f397006bfa0 osd=4 3 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 15 client.24524.objecter _session_command_op_assign 4 2 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 20 client.24524.objecter put_session s=0x7f397006bfa0 osd=4 4 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 10 client.24524.objecter _send_command 2 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 --> v2:172.21.15.137:6800/2044387056 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f398805ee80 con 0x7f397006c650 2024-02-18T00:30:46.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f3986ffd700 1 -- 172.21.15.59:0/1430317292 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f398805ee80 con 0x7f397006c650 2024-02-18T00:30:46.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f3986ffd700 10 client.24524.objecter ms_dispatch 0x7f3988001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:46.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f3986ffd700 10 client.24524.objecter _finish_command 2 = system:0 2024-02-18T00:30:46.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f3986ffd700 20 client.24524.objecter put_session s=0x7f397006bfa0 osd=4 4 2024-02-18T00:30:46.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f3986ffd700 15 client.24524.objecter _session_command_op_remove 4 2 2024-02-18T00:30:46.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 10 librados: watch_flush enter 2024-02-18T00:30:46.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 10 librados: watch_flush exit 2024-02-18T00:30:46.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 10 client.24524.objecter close_session for osd.4 2024-02-18T00:30:46.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 >> v2:172.21.15.137:6800/2044387056 conn(0x7f397006c650 msgr2=0x7f397006eaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:46.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.137:6800/2044387056 conn(0x7f397006c650 0x7f397006eaf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:46.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 20 client.24524.objecter put_session s=0x7f397006bfa0 osd=4 1 2024-02-18T00:30:46.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 20 client.24524.objecter shutdown clearing up homeless session... 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 10 client.24524.objecter successfully canceled tick 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3970062c90 msgr2=0x7f3970065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 1 --2- 172.21.15.59:0/1430317292 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3970062c90 0x7f3970065150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f39880fa100 tx=0x7f397c00b040).stop 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39880f7400 msgr2=0x7f39880f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398fb67700 1 --2- 172.21.15.59:0/1430317292 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39880f7400 0x7f39880f89b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3978009eb0 tx=0x7f397800b040).stop 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.780+0000 7f398e104700 1 -- 172.21.15.59:0/1430317292 reap_dead start 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.781+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 shutdown_connections 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.781+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 >> 172.21.15.59:0/1430317292 conn(0x7f39880f2820 msgr2=0x7f39880f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.781+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 shutdown_connections 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.781+0000 7f398fb67700 1 -- 172.21.15.59:0/1430317292 wait complete. 2024-02-18T00:30:46.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:46.781+0000 7f398fb67700 1 librados: shutdown 2024-02-18T00:30:47.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:45.733798+0000 mgr.y (mgr.24454) 63 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 21 B/s, 3 objects/s recovering 2024-02-18T00:30:47.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.061609+0000 osd.4 (osd.4) 31 : cluster [DBG] 4.1b deep-scrub starts 2024-02-18T00:30:47.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.066549+0000 osd.4 (osd.4) 32 : cluster [DBG] 4.1b deep-scrub ok 2024-02-18T00:30:47.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.307263+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.308449+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.503632+0000 osd.7 (osd.7) 31 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.504418+0000 osd.7 (osd.7) 32 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.764451+0000 osd.0 (osd.0) 19 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.767894+0000 osd.0 (osd.0) 20 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:45.733798+0000 mgr.y (mgr.24454) 63 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 21 B/s, 3 objects/s recovering 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.061609+0000 osd.4 (osd.4) 31 : cluster [DBG] 4.1b deep-scrub starts 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.066549+0000 osd.4 (osd.4) 32 : cluster [DBG] 4.1b deep-scrub ok 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.307263+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.308449+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:30:47.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.503632+0000 osd.7 (osd.7) 31 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:30:47.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.504418+0000 osd.7 (osd.7) 32 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:30:47.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.764451+0000 osd.0 (osd.0) 19 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:30:47.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.767894+0000 osd.0 (osd.0) 20 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:45.733798+0000 mgr.y (mgr.24454) 63 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 21 B/s, 3 objects/s recovering 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.061609+0000 osd.4 (osd.4) 31 : cluster [DBG] 4.1b deep-scrub starts 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.066549+0000 osd.4 (osd.4) 32 : cluster [DBG] 4.1b deep-scrub ok 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.307263+0000 osd.2 (osd.2) 25 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.308449+0000 osd.2 (osd.2) 26 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.503632+0000 osd.7 (osd.7) 31 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.504418+0000 osd.7 (osd.7) 32 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.764451+0000 osd.0 (osd.0) 19 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:30:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.767894+0000 osd.0 (osd.0) 20 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:30:47.997 INFO:teuthology.orchestra.run.smithi059.stdout:133143986213 2024-02-18T00:30:47.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.912515+0000 osd.3 (osd.3) 27 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:46.913332+0000 osd.3 (osd.3) 28 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.274526+0000 osd.2 (osd.2) 27 : cluster [DBG] 2.e scrub starts 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.275323+0000 osd.2 (osd.2) 28 : cluster [DBG] 2.e scrub ok 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.439173+0000 osd.6 (osd.6) 23 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.440001+0000 osd.6 (osd.6) 24 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.649296+0000 osd.0 (osd.0) 21 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.649353+0000 osd.0 (osd.0) 22 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:48.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.730616+0000 osd.0 (osd.0) 23 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:30:48.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.731748+0000 osd.0 (osd.0) 24 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:30:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.912515+0000 osd.3 (osd.3) 27 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:30:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:46.913332+0000 osd.3 (osd.3) 28 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.274526+0000 osd.2 (osd.2) 27 : cluster [DBG] 2.e scrub starts 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.275323+0000 osd.2 (osd.2) 28 : cluster [DBG] 2.e scrub ok 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.439173+0000 osd.6 (osd.6) 23 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.440001+0000 osd.6 (osd.6) 24 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.649296+0000 osd.0 (osd.0) 21 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.649353+0000 osd.0 (osd.0) 22 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.730616+0000 osd.0 (osd.0) 23 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:30:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.731748+0000 osd.0 (osd.0) 24 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:30:48.425 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.912515+0000 osd.3 (osd.3) 27 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:30:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:46.913332+0000 osd.3 (osd.3) 28 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:30:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.274526+0000 osd.2 (osd.2) 27 : cluster [DBG] 2.e scrub starts 2024-02-18T00:30:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.275323+0000 osd.2 (osd.2) 28 : cluster [DBG] 2.e scrub ok 2024-02-18T00:30:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.439173+0000 osd.6 (osd.6) 23 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:30:48.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.440001+0000 osd.6 (osd.6) 24 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:30:48.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.649296+0000 osd.0 (osd.0) 21 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:48.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.649353+0000 osd.0 (osd.0) 22 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:48.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.730616+0000 osd.0 (osd.0) 23 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:30:48.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.731748+0000 osd.0 (osd.0) 24 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:30:49.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:47.734403+0000 mgr.y (mgr.24454) 64 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.0 KiB/s rd, 167 B/s wr, 14 op/s; 18 B/s, 2 objects/s recovering 2024-02-18T00:30:49.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.141478+0000 mon.a (mon.0) 727 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.261208+0000 osd.2 (osd.2) 29 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.261997+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.450044+0000 osd.6 (osd.6) 25 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.450872+0000 osd.6 (osd.6) 26 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.685207+0000 osd.0 (osd.0) 25 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.688897+0000 osd.0 (osd.0) 26 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:48.778245+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:49.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.979546+0000 osd.1 (osd.1) 27 : cluster [DBG] 2.d scrub starts 2024-02-18T00:30:49.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.980288+0000 osd.1 (osd.1) 28 : cluster [DBG] 2.d scrub ok 2024-02-18T00:30:49.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:47.734403+0000 mgr.y (mgr.24454) 64 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.0 KiB/s rd, 167 B/s wr, 14 op/s; 18 B/s, 2 objects/s recovering 2024-02-18T00:30:49.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.141478+0000 mon.a (mon.0) 727 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-18T00:30:49.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.261208+0000 osd.2 (osd.2) 29 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:30:49.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.261997+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:30:49.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.450044+0000 osd.6 (osd.6) 25 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:30:49.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.450872+0000 osd.6 (osd.6) 26 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:30:49.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.685207+0000 osd.0 (osd.0) 25 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:30:49.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.688897+0000 osd.0 (osd.0) 26 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:30:49.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:48.778245+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:49.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.979546+0000 osd.1 (osd.1) 27 : cluster [DBG] 2.d scrub starts 2024-02-18T00:30:49.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.980288+0000 osd.1 (osd.1) 28 : cluster [DBG] 2.d scrub ok 2024-02-18T00:30:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:47.734403+0000 mgr.y (mgr.24454) 64 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.0 KiB/s rd, 167 B/s wr, 14 op/s; 18 B/s, 2 objects/s recovering 2024-02-18T00:30:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.141478+0000 mon.a (mon.0) 727 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-18T00:30:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.261208+0000 osd.2 (osd.2) 29 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:30:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.261997+0000 osd.2 (osd.2) 30 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:30:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.450044+0000 osd.6 (osd.6) 25 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:30:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.450872+0000 osd.6 (osd.6) 26 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:30:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.685207+0000 osd.0 (osd.0) 25 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:30:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.688897+0000 osd.0 (osd.0) 26 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:30:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:48.778245+0000 mon.a (mon.0) 728 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.979546+0000 osd.1 (osd.1) 27 : cluster [DBG] 2.d scrub starts 2024-02-18T00:30:49.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.980288+0000 osd.1 (osd.1) 28 : cluster [DBG] 2.d scrub ok 2024-02-18T00:30:50.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.889865+0000 osd.3 (osd.3) 29 : cluster [DBG] 2.1e deep-scrub starts 2024-02-18T00:30:50.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.890673+0000 osd.3 (osd.3) 30 : cluster [DBG] 2.1e deep-scrub ok 2024-02-18T00:30:50.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.990023+0000 osd.5 (osd.5) 21 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:30:50.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:48.990746+0000 osd.5 (osd.5) 22 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:30:50.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:49.587228+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:30:50.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:49.588264+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:30:50.223 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:49.628264+0000 mon.a (mon.0) 729 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-18T00:30:50.224 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.889865+0000 osd.3 (osd.3) 29 : cluster [DBG] 2.1e deep-scrub starts 2024-02-18T00:30:50.224 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.890673+0000 osd.3 (osd.3) 30 : cluster [DBG] 2.1e deep-scrub ok 2024-02-18T00:30:50.224 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.990023+0000 osd.5 (osd.5) 21 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:30:50.224 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:48.990746+0000 osd.5 (osd.5) 22 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:30:50.224 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:49.587228+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:30:50.224 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:49.588264+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:30:50.224 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:49.628264+0000 mon.a (mon.0) 729 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-18T00:30:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.889865+0000 osd.3 (osd.3) 29 : cluster [DBG] 2.1e deep-scrub starts 2024-02-18T00:30:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.890673+0000 osd.3 (osd.3) 30 : cluster [DBG] 2.1e deep-scrub ok 2024-02-18T00:30:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.990023+0000 osd.5 (osd.5) 21 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:30:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:48.990746+0000 osd.5 (osd.5) 22 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:30:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:49.587228+0000 osd.7 (osd.7) 33 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:30:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:49.588264+0000 osd.7 (osd.7) 34 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:30:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:49.628264+0000 mon.a (mon.0) 729 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 -- 172.21.15.59:0/3572344347 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 msgr2=0x7f5ea40f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 --2- 172.21.15.59:0/3572344347 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 0x7f5ea40f6d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5e98004870 tx=0x7f5e98009cb0).stop 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 -- 172.21.15.59:0/3572344347 shutdown_connections 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 --2- 172.21.15.59:0/3572344347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ea40fcbb0 0x7f5ea40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 --2- 172.21.15.59:0/3572344347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea40f7400 0x7f5ea4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 --2- 172.21.15.59:0/3572344347 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 0x7f5ea40f6d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 -- 172.21.15.59:0/3572344347 >> 172.21.15.59:0/3572344347 conn(0x7f5ea40f2820 msgr2=0x7f5ea40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 -- 172.21.15.59:0/3572344347 shutdown_connections 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.680+0000 7f5eac68d700 1 -- 172.21.15.59:0/3572344347 wait complete. 2024-02-18T00:30:50.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 librados: starting msgr at 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 librados: starting objecter 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 Processor -- start 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 -- start start 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 librados: setting wanted keys 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 librados: calling monclient init 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea40f7400 0x7f5ea410e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ea40fcbb0 0x7f5ea410e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 0x7f5ea4108300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea41087f0 con 0x7f5ea40f7400 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4108960 con 0x7f5ea40fcbb0 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eac68d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea4108b20 con 0x7f5ea40fd510 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea40f7400 0x7f5ea410e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:50.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaac2a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 0x7f5ea4108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea40f7400 0x7f5ea410e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47768/0 (socket says 172.21.15.59:47768) 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 -- 172.21.15.59:0/1563509990 learned_addr learned my addr 172.21.15.59:0/1563509990 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5ea9c28700 1 --2- 172.21.15.59:0/1563509990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ea40fcbb0 0x7f5ea410e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 -- 172.21.15.59:0/1563509990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 msgr2=0x7f5ea4108300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 --2- 172.21.15.59:0/1563509990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 0x7f5ea4108300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 -- 172.21.15.59:0/1563509990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ea40fcbb0 msgr2=0x7f5ea410e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 --2- 172.21.15.59:0/1563509990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ea40fcbb0 0x7f5ea410e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaa429700 1 -- 172.21.15.59:0/1563509990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0097c0 con 0x7f5ea40f7400 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.681+0000 7f5eaac2a700 1 --2- 172.21.15.59:0/1563509990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ea40fd510 0x7f5ea4108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5eaa429700 1 --2- 172.21.15.59:0/1563509990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea40f7400 0x7f5ea410e1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f5ea000c810 tx=0x7f5ea000cb20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5e977fe700 10 client.?.objecter ms_handle_connect 0x7f5ea40f7400 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5e977fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:50.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0010510 con 0x7f5ea40f7400 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea0010b40 con 0x7f5ea40f7400 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e98005040 con 0x7f5ea40f7400 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0012a90 con 0x7f5ea40f7400 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5eac68d700 10 client.14625.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea4109110 con 0x7f5ea40f7400 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.682+0000 7f5eac68d700 1 librados: init done 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.683+0000 7f5eac68d700 10 client.14625.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.683+0000 7f5eac68d700 15 client.14625.objecter _session_command_op_assign -1 1 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.683+0000 7f5eac68d700 20 client.14625.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.683+0000 7f5eac68d700 10 client.14625.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.683+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ea405fa60 con 0x7f5ea40f7400 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.683+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5ea0010cb0 con 0x7f5ea40f7400 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.683+0000 7f5e977fe700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5e90062c90 0x7f5e90065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:50.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f5ea007ca10 con 0x7f5ea40f7400 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 10 client.14625.objecter ms_dispatch 0x7f5ea4001550 osd_map(84..84 src has 1..84) v4 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 3 client.14625.objecter handle_osd_map got epochs [84,84] > 0 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 3 client.14625.objecter handle_osd_map decoding full epoch 84 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5ea9c28700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5e90062c90 0x7f5e90065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 10 client.14625.objecter checking command 1 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5e9006c650 0x7f5e9006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 20 client.14625.objecter _get_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 20 client.14625.objecter put_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 15 client.14625.objecter _session_command_op_remove -1 1 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 20 client.14625.objecter _get_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 20 client.14625.objecter get_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 15 client.14625.objecter _session_command_op_assign 5 1 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 20 client.14625.objecter put_session s=0x7f5e9006bfa0 osd=5 4 2024-02-18T00:30:50.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 10 client.14625.objecter _send_command 1 2024-02-18T00:30:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 --> v2:172.21.15.137:6804/2170432263 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e9006f130 con 0x7f5e9006c650 2024-02-18T00:30:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 20 client.14625.objecter dump_active .. 0 homeless 2024-02-18T00:30:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea0081220 con 0x7f5ea40f7400 2024-02-18T00:30:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 10 client.14625.objecter ms_handle_connect 0x7f5e90062c90 2024-02-18T00:30:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5ea9c28700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5e90062c90 0x7f5e90065150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5e980043d0 tx=0x7f5e9800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:50.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5eaac2a700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5e9006c650 0x7f5e9006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5e977fe700 10 client.14625.objecter ms_handle_connect 0x7f5e9006c650 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.684+0000 7f5eaac2a700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5e9006c650 0x7f5e9006eaf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.685+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5e9006f130 con 0x7f5e9006c650 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.685+0000 7f5e977fe700 10 client.14625.objecter ms_dispatch 0x7f5ea4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.685+0000 7f5e977fe700 10 client.14625.objecter _finish_command 1 = system:0 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.685+0000 7f5e977fe700 20 client.14625.objecter put_session s=0x7f5e9006bfa0 osd=5 4 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.685+0000 7f5e977fe700 15 client.14625.objecter _session_command_op_remove 5 1 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 10 client.14625.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 15 client.14625.objecter _session_command_op_assign -1 2 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 20 client.14625.objecter _get_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 20 client.14625.objecter put_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 20 client.14625.objecter _get_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 15 client.14625.objecter _session_command_op_remove -1 2 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 20 client.14625.objecter get_session s=0x7f5e9006bfa0 osd=5 3 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 15 client.14625.objecter _session_command_op_assign 5 2 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 20 client.14625.objecter put_session s=0x7f5e9006bfa0 osd=5 4 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 10 client.14625.objecter _send_command 2 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.699+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 --> v2:172.21.15.137:6804/2170432263 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ea405ee80 con 0x7f5e9006c650 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5e977fe700 1 -- 172.21.15.59:0/1563509990 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5ea405ee80 con 0x7f5e9006c650 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5e977fe700 10 client.14625.objecter ms_dispatch 0x7f5ea4001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5e977fe700 10 client.14625.objecter _finish_command 2 = system:0 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5e977fe700 20 client.14625.objecter put_session s=0x7f5e9006bfa0 osd=5 4 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5e977fe700 15 client.14625.objecter _session_command_op_remove 5 2 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 10 librados: watch_flush enter 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 10 librados: watch_flush exit 2024-02-18T00:30:50.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 10 client.14625.objecter close_session for osd.5 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5e9006c650 msgr2=0x7f5e9006eaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5e9006c650 0x7f5e9006eaf0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 20 client.14625.objecter put_session s=0x7f5e9006bfa0 osd=5 1 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 20 client.14625.objecter shutdown clearing up homeless session... 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 10 client.14625.objecter successfully canceled tick 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5e90062c90 msgr2=0x7f5e90065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 --2- 172.21.15.59:0/1563509990 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5e90062c90 0x7f5e90065150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f5e980043d0 tx=0x7f5e9800b040).stop 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea40f7400 msgr2=0x7f5ea410e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 --2- 172.21.15.59:0/1563509990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea40f7400 0x7f5ea410e1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f5ea000c810 tx=0x7f5ea000cb20).stop 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eaac2a700 1 -- 172.21.15.59:0/1563509990 reap_dead start 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 shutdown_connections 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.700+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 >> 172.21.15.59:0/1563509990 conn(0x7f5ea40f2820 msgr2=0x7f5ea40f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.701+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 shutdown_connections 2024-02-18T00:30:50.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.701+0000 7f5eac68d700 1 -- 172.21.15.59:0/1563509990 wait complete. 2024-02-18T00:30:50.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:50.701+0000 7f5eac68d700 1 librados: shutdown 2024-02-18T00:30:51.061 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:30:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:30:51.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:49.734752+0000 mgr.y (mgr.24454) 65 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 170 B/s wr, 14 op/s; 18 B/s, 2 objects/s recovering 2024-02-18T00:30:51.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:50.763599+0000 osd.0 (osd.0) 27 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:30:51.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:50.764658+0000 osd.0 (osd.0) 28 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:30:51.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:50.988681+0000 osd.1 (osd.1) 29 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:30:51.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:50.991588+0000 osd.1 (osd.1) 30 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:30:51.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:49.734752+0000 mgr.y (mgr.24454) 65 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 170 B/s wr, 14 op/s; 18 B/s, 2 objects/s recovering 2024-02-18T00:30:51.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:50.763599+0000 osd.0 (osd.0) 27 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:30:51.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:50.764658+0000 osd.0 (osd.0) 28 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:30:51.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:50.988681+0000 osd.1 (osd.1) 29 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:30:51.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:50.991588+0000 osd.1 (osd.1) 30 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:30:51.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:30:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:30:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:49.734752+0000 mgr.y (mgr.24454) 65 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 170 B/s wr, 14 op/s; 18 B/s, 2 objects/s recovering 2024-02-18T00:30:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:50.763599+0000 osd.0 (osd.0) 27 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:30:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:50.764658+0000 osd.0 (osd.0) 28 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:30:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:50.988681+0000 osd.1 (osd.1) 29 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:30:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:50.991588+0000 osd.1 (osd.1) 30 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:30:51.580 INFO:teuthology.orchestra.run.smithi059.stdout:167503724576 2024-02-18T00:30:51.581 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-18T00:30:52.435 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:52.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:30:52] "GET /metrics HTTP/1.1" 200 172145 "" "Prometheus/2.33.4" 2024-02-18T00:30:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:50.793345+0000 mgr.y (mgr.24454) 66 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:50.883684+0000 osd.3 (osd.3) 31 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:30:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:50.885640+0000 osd.3 (osd.3) 32 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:30:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:51.227037+0000 osd.2 (osd.2) 31 : cluster [DBG] 5.d scrub starts 2024-02-18T00:30:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:51.228897+0000 osd.2 (osd.2) 32 : cluster [DBG] 5.d scrub ok 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:51.990860+0000 osd.1 (osd.1) 31 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:51.994327+0000 osd.1 (osd.1) 32 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:50.793345+0000 mgr.y (mgr.24454) 66 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:50.883684+0000 osd.3 (osd.3) 31 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:50.885640+0000 osd.3 (osd.3) 32 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:51.227037+0000 osd.2 (osd.2) 31 : cluster [DBG] 5.d scrub starts 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:51.228897+0000 osd.2 (osd.2) 32 : cluster [DBG] 5.d scrub ok 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:51.990860+0000 osd.1 (osd.1) 31 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:30:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:51.994327+0000 osd.1 (osd.1) 32 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:50.793345+0000 mgr.y (mgr.24454) 66 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:50.883684+0000 osd.3 (osd.3) 31 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:50.885640+0000 osd.3 (osd.3) 32 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:51.227037+0000 osd.2 (osd.2) 31 : cluster [DBG] 5.d scrub starts 2024-02-18T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:51.228897+0000 osd.2 (osd.2) 32 : cluster [DBG] 5.d scrub ok 2024-02-18T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:51.990860+0000 osd.1 (osd.1) 31 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:30:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:51.994327+0000 osd.1 (osd.1) 32 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:30:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:51.735319+0000 mgr.y (mgr.24454) 67 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.8 KiB/s rd, 144 B/s wr, 13 op/s; 15 B/s, 1 objects/s recovering 2024-02-18T00:30:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:52.069491+0000 osd.5 (osd.5) 23 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:30:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:52.072614+0000 osd.5 (osd.5) 24 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:30:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:53.055065+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:51.735319+0000 mgr.y (mgr.24454) 67 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.8 KiB/s rd, 144 B/s wr, 13 op/s; 15 B/s, 1 objects/s recovering 2024-02-18T00:30:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:52.069491+0000 osd.5 (osd.5) 23 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:30:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:52.072614+0000 osd.5 (osd.5) 24 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:30:53.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:53.055065+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:51.735319+0000 mgr.y (mgr.24454) 67 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 6.8 KiB/s rd, 144 B/s wr, 13 op/s; 15 B/s, 1 objects/s recovering 2024-02-18T00:30:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:52.069491+0000 osd.5 (osd.5) 23 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:30:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:52.072614+0000 osd.5 (osd.5) 24 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:30:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:53.055065+0000 mon.a (mon.0) 730 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:54.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:30:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:30:53.872Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003697751s 2024-02-18T00:30:54.420 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:52.922057+0000 osd.3 (osd.3) 33 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:30:54.420 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:52.923011+0000 osd.3 (osd.3) 34 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:30:54.420 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:53.586110+0000 osd.7 (osd.7) 35 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:53.587791+0000 osd.7 (osd.7) 36 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:53.679922+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:53.680622+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:53.685874+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:53.689836+0000 mon.a (mon.0) 734 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.007397+0000 osd.1 (osd.1) 33 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.008192+0000 osd.1 (osd.1) 34 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:52.922057+0000 osd.3 (osd.3) 33 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:52.923011+0000 osd.3 (osd.3) 34 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:30:54.421 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:53.586110+0000 osd.7 (osd.7) 35 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:30:54.422 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:53.587791+0000 osd.7 (osd.7) 36 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:30:54.422 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:53.679922+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:30:54.422 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:53.680622+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:30:54.422 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:53.685874+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:54.422 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:53.689836+0000 mon.a (mon.0) 734 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:54.422 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.007397+0000 osd.1 (osd.1) 33 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:30:54.422 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.008192+0000 osd.1 (osd.1) 34 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:30:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:52.922057+0000 osd.3 (osd.3) 33 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:30:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:52.923011+0000 osd.3 (osd.3) 34 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:30:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:53.586110+0000 osd.7 (osd.7) 35 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:30:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:53.587791+0000 osd.7 (osd.7) 36 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:30:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:53.679922+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:30:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:53.680622+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:30:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:53.685874+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:30:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:53.689836+0000 mon.a (mon.0) 734 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:30:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.007397+0000 osd.1 (osd.1) 33 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:30:54.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.008192+0000 osd.1 (osd.1) 34 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:30:54.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 -- 172.21.15.59:0/592825501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 msgr2=0x7f9fb00f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:54.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 --2- 172.21.15.59:0/592825501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 0x7f9fb00f7d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f9fa0004870 tx=0x7f9fa0009cb0).stop 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 -- 172.21.15.59:0/592825501 shutdown_connections 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 --2- 172.21.15.59:0/592825501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fb00fdbb0 0x7f9fb00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 --2- 172.21.15.59:0/592825501 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fb00f8420 0x7f9fb0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 --2- 172.21.15.59:0/592825501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 0x7f9fb00f7d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 -- 172.21.15.59:0/592825501 >> 172.21.15.59:0/592825501 conn(0x7f9fb00f3800 msgr2=0x7f9fb00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 -- 172.21.15.59:0/592825501 shutdown_connections 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 -- 172.21.15.59:0/592825501 wait complete. 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 librados: starting msgr at 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.663+0000 7f9fb779e700 1 librados: starting objecter 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 Processor -- start 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 -- start start 2024-02-18T00:30:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 librados: setting wanted keys 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 librados: calling monclient init 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fb00f8420 0x7f9fb010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fb00fdbb0 0x7f9fb010e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 0x7f9fb0108150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0108640 con 0x7f9fb00fe510 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb01087b0 con 0x7f9fb00fdbb0 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb779e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0108970 con 0x7f9fb00f8420 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 0x7f9fb0108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb553a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fb00f8420 0x7f9fb010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 0x7f9fb0108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47780/0 (socket says 172.21.15.59:47780) 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 -- 172.21.15.59:0/380805439 learned_addr learned my addr 172.21.15.59:0/380805439 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb553a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fb00f8420 0x7f9fb010dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:48892/0 (socket says 172.21.15.59:48892) 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb4d39700 1 --2- 172.21.15.59:0/380805439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fb00fdbb0 0x7f9fb010e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 -- 172.21.15.59:0/380805439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fb00f8420 msgr2=0x7f9fb010dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 --2- 172.21.15.59:0/380805439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9fb00f8420 0x7f9fb010dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 -- 172.21.15.59:0/380805439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fb00fdbb0 msgr2=0x7f9fb010e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 --2- 172.21.15.59:0/380805439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fb00fdbb0 0x7f9fb010e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 -- 172.21.15.59:0/380805439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa0005040 con 0x7f9fb00fe510 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fb5d3b700 1 --2- 172.21.15.59:0/380805439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 0x7f9fb0108150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f9fa800ec20 tx=0x7f9fa800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fa67fc700 10 client.?.objecter ms_handle_connect 0x7f9fb00fe510 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.664+0000 7f9fa67fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8009950 con 0x7f9fb00fe510 2024-02-18T00:30:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa800f040 con 0x7f9fb00fe510 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb0108bf0 con 0x7f9fb00fe510 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8007df0 con 0x7f9fb00fe510 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fb779e700 10 client.14631.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb0114de0 con 0x7f9fb00fe510 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fb779e700 1 librados: init done 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.665+0000 7f9fb779e700 10 client.14631.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fb779e700 15 client.14631.objecter _session_command_op_assign -1 1 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fb779e700 20 client.14631.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fb779e700 10 client.14631.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9fb005ee80 con 0x7f9fb00fe510 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9fa80092e0 con 0x7f9fb00fe510 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9f9c062c90 0x7f9f9c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f9fa807c670 con 0x7f9fb00fe510 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 10 client.14631.objecter ms_dispatch 0x7f9fb0001550 osd_map(84..84 src has 1..84) v4 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 3 client.14631.objecter handle_osd_map got epochs [84,84] > 0 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 3 client.14631.objecter handle_osd_map decoding full epoch 84 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fb553a700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9f9c062c90 0x7f9f9c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 10 client.14631.objecter checking command 1 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.137:6808/3356009390 conn(0x7f9f9c06c6d0 0x7f9f9c06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 20 client.14631.objecter _get_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 20 client.14631.objecter put_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 15 client.14631.objecter _session_command_op_remove -1 1 2024-02-18T00:30:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 20 client.14631.objecter _get_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 20 client.14631.objecter get_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 15 client.14631.objecter _session_command_op_assign 6 1 2024-02-18T00:30:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 20 client.14631.objecter put_session s=0x7f9f9c06c020 osd=6 4 2024-02-18T00:30:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 10 client.14631.objecter _send_command 1 2024-02-18T00:30:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 --> v2:172.21.15.137:6808/3356009390 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f9c06f1b0 con 0x7f9f9c06c6d0 2024-02-18T00:30:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 20 client.14631.objecter dump_active .. 0 homeless 2024-02-18T00:30:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.666+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fa807ece0 con 0x7f9fb00fe510 2024-02-18T00:30:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.667+0000 7f9fb4d39700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.137:6808/3356009390 conn(0x7f9f9c06c6d0 0x7f9f9c06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.667+0000 7f9fa67fc700 10 client.14631.objecter ms_handle_connect 0x7f9f9c062c90 2024-02-18T00:30:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.667+0000 7f9fb553a700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9f9c062c90 0x7f9f9c065150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9fac004dc0 tx=0x7f9fac00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.667+0000 7f9fa67fc700 10 client.14631.objecter ms_handle_connect 0x7f9f9c06c6d0 2024-02-18T00:30:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.667+0000 7f9fb4d39700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.137:6808/3356009390 conn(0x7f9f9c06c6d0 0x7f9f9c06eb70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.668+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9f9c06f1b0 con 0x7f9f9c06c6d0 2024-02-18T00:30:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.668+0000 7f9fa67fc700 10 client.14631.objecter ms_dispatch 0x7f9fb0001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.668+0000 7f9fa67fc700 10 client.14631.objecter _finish_command 1 = system:0 2024-02-18T00:30:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.668+0000 7f9fa67fc700 20 client.14631.objecter put_session s=0x7f9f9c06c020 osd=6 4 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.668+0000 7f9fa67fc700 15 client.14631.objecter _session_command_op_remove 6 1 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 10 client.14631.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 15 client.14631.objecter _session_command_op_assign -1 2 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 20 client.14631.objecter _get_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 20 client.14631.objecter put_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 20 client.14631.objecter _get_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 15 client.14631.objecter _session_command_op_remove -1 2 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 20 client.14631.objecter get_session s=0x7f9f9c06c020 osd=6 3 2024-02-18T00:30:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 15 client.14631.objecter _session_command_op_assign 6 2 2024-02-18T00:30:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 20 client.14631.objecter put_session s=0x7f9f9c06c020 osd=6 4 2024-02-18T00:30:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 10 client.14631.objecter _send_command 2 2024-02-18T00:30:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 --> v2:172.21.15.137:6808/3356009390 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9fb01070a0 con 0x7f9f9c06c6d0 2024-02-18T00:30:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fa67fc700 1 -- 172.21.15.59:0/380805439 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9fb01070a0 con 0x7f9f9c06c6d0 2024-02-18T00:30:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fa67fc700 10 client.14631.objecter ms_dispatch 0x7f9fb0001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:54.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fa67fc700 10 client.14631.objecter _finish_command 2 = system:0 2024-02-18T00:30:54.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fa67fc700 20 client.14631.objecter put_session s=0x7f9f9c06c020 osd=6 4 2024-02-18T00:30:54.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.682+0000 7f9fa67fc700 15 client.14631.objecter _session_command_op_remove 6 2 2024-02-18T00:30:54.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 10 librados: watch_flush enter 2024-02-18T00:30:54.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 10 librados: watch_flush exit 2024-02-18T00:30:54.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 10 client.14631.objecter close_session for osd.6 2024-02-18T00:30:54.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 >> v2:172.21.15.137:6808/3356009390 conn(0x7f9f9c06c6d0 msgr2=0x7f9f9c06eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:54.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.137:6808/3356009390 conn(0x7f9f9c06c6d0 0x7f9f9c06eb70 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:54.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 20 client.14631.objecter put_session s=0x7f9f9c06c020 osd=6 1 2024-02-18T00:30:54.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 20 client.14631.objecter shutdown clearing up homeless session... 2024-02-18T00:30:54.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 10 client.14631.objecter successfully canceled tick 2024-02-18T00:30:54.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9f9c062c90 msgr2=0x7f9f9c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:54.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 --2- 172.21.15.59:0/380805439 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9f9c062c90 0x7f9f9c065150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9fac004dc0 tx=0x7f9fac00b040).stop 2024-02-18T00:30:54.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 msgr2=0x7f9fb0108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:54.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 --2- 172.21.15.59:0/380805439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9fb00fe510 0x7f9fb0108150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f9fa800ec20 tx=0x7f9fa800b040).stop 2024-02-18T00:30:54.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb5d3b700 1 -- 172.21.15.59:0/380805439 reap_dead start 2024-02-18T00:30:54.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 shutdown_connections 2024-02-18T00:30:54.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 >> 172.21.15.59:0/380805439 conn(0x7f9fb00f3800 msgr2=0x7f9fb00fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:54.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 shutdown_connections 2024-02-18T00:30:54.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 -- 172.21.15.59:0/380805439 wait complete. 2024-02-18T00:30:54.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:54.683+0000 7f9fb779e700 1 librados: shutdown 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:53.735673+0000 mgr.y (mgr.24454) 68 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.590001+0000 osd.7 (osd.7) 37 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.590766+0000 osd.7 (osd.7) 38 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.685767+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.685812+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:53.735673+0000 mgr.y (mgr.24454) 68 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.590001+0000 osd.7 (osd.7) 37 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.590766+0000 osd.7 (osd.7) 38 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.685767+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.685812+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:53.735673+0000 mgr.y (mgr.24454) 68 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:30:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.590001+0000 osd.7 (osd.7) 37 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:30:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.590766+0000 osd.7 (osd.7) 38 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:30:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.685767+0000 osd.6 (osd.6) 27 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:30:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.685812+0000 osd.6 (osd.6) 28 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:30:55.603 INFO:teuthology.orchestra.run.smithi059.stdout:201863462941 2024-02-18T00:30:55.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-18T00:30:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.985198+0000 osd.3 (osd.3) 35 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.987043+0000 osd.3 (osd.3) 36 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.994434+0000 osd.4 (osd.4) 33 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:54.996538+0000 osd.4 (osd.4) 34 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:55.689275+0000 mon.a (mon.0) 735 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.985198+0000 osd.3 (osd.3) 35 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.987043+0000 osd.3 (osd.3) 36 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.994434+0000 osd.4 (osd.4) 33 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:54.996538+0000 osd.4 (osd.4) 34 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:30:56.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:55.689275+0000 mon.a (mon.0) 735 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-18T00:30:56.393 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:30:56.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.985198+0000 osd.3 (osd.3) 35 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:30:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.987043+0000 osd.3 (osd.3) 36 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:30:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.994434+0000 osd.4 (osd.4) 33 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:30:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:54.996538+0000 osd.4 (osd.4) 34 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:30:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:55.689275+0000 mon.a (mon.0) 735 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-18T00:30:57.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:55.736224+0000 mgr.y (mgr.24454) 69 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:30:57.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:56.032979+0000 osd.3 (osd.3) 37 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:30:57.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:56.034484+0000 osd.3 (osd.3) 38 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:30:57.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:55.736224+0000 mgr.y (mgr.24454) 69 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:30:57.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:56.032979+0000 osd.3 (osd.3) 37 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:30:57.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:56.034484+0000 osd.3 (osd.3) 38 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:30:57.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:55.736224+0000 mgr.y (mgr.24454) 69 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:30:57.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:56.032979+0000 osd.3 (osd.3) 37 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:30:57.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:56.034484+0000 osd.3 (osd.3) 38 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:30:58.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:58.026699+0000 osd.1 (osd.1) 35 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:30:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:58.030170+0000 osd.1 (osd.1) 36 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:30:58.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:58.026699+0000 osd.1 (osd.1) 35 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:30:58.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:58.030170+0000 osd.1 (osd.1) 36 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:30:58.467 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:58.026699+0000 osd.1 (osd.1) 35 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:30:58.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:58.030170+0000 osd.1 (osd.1) 36 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:30:58.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 -- 172.21.15.59:0/658139767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 msgr2=0x7f0d940f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 --2- 172.21.15.59:0/658139767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 0x7f0d940f8d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0d7c004870 tx=0x7f0d7c009cb0).stop 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 -- 172.21.15.59:0/658139767 shutdown_connections 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 --2- 172.21.15.59:0/658139767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d940f7330 0x7f0d94103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 --2- 172.21.15.59:0/658139767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 0x7f0d940f8d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 --2- 172.21.15.59:0/658139767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d940f9270 0x7f0d940f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 -- 172.21.15.59:0/658139767 >> 172.21.15.59:0/658139767 conn(0x7f0d940f26f0 msgr2=0x7f0d940f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 -- 172.21.15.59:0/658139767 shutdown_connections 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 -- 172.21.15.59:0/658139767 wait complete. 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 librados: starting msgr at 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 librados: starting objecter 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.732+0000 7f0d99a2f700 1 Processor -- start 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 -- start start 2024-02-18T00:30:58.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 librados: setting wanted keys 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 librados: calling monclient init 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d940f7330 0x7f0d9410e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 0x7f0d9410e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d940f9270 0x7f0d94108330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94108820 con 0x7f0d940f89a0 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94108990 con 0x7f0d940f7330 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d99a2f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94108b50 con 0x7f0d940f9270 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 0x7f0d9410e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 0x7f0d9410e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47810/0 (socket says 172.21.15.59:47810) 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 -- 172.21.15.59:0/923727771 learned_addr learned my addr 172.21.15.59:0/923727771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d937fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d940f9270 0x7f0d94108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:58.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d92ffd700 1 --2- 172.21.15.59:0/923727771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d940f7330 0x7f0d9410e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 -- 172.21.15.59:0/923727771 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d940f9270 msgr2=0x7f0d94108330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 --2- 172.21.15.59:0/923727771 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d940f9270 0x7f0d94108330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 -- 172.21.15.59:0/923727771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d940f7330 msgr2=0x7f0d9410e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 --2- 172.21.15.59:0/923727771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d940f7330 0x7f0d9410e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d927fc700 1 -- 172.21.15.59:0/923727771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d84005040 con 0x7f0d940f89a0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.733+0000 7f0d937fe700 1 --2- 172.21.15.59:0/923727771 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d940f9270 0x7f0d94108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d927fc700 1 --2- 172.21.15.59:0/923727771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 0x7f0d9410e680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0d840048f0 tx=0x7f0d84009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d8bfff700 10 client.?.objecter ms_handle_connect 0x7f0d940f89a0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d8bfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84007b00 con 0x7f0d940f89a0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8400f040 con 0x7f0d940f89a0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c005040 con 0x7f0d940f89a0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84014990 con 0x7f0d940f89a0 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d99a2f700 10 client.14637.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:58.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d94109190 con 0x7f0d940f89a0 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d99a2f700 1 librados: init done 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.734+0000 7f0d99a2f700 10 client.14637.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.735+0000 7f0d99a2f700 15 client.14637.objecter _session_command_op_assign -1 1 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.735+0000 7f0d99a2f700 20 client.14637.objecter _get_session osd=-1 returning homeless 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.735+0000 7f0d99a2f700 10 client.14637.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.735+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d9405ee80 con 0x7f0d940f89a0 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.735+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0d84010500 con 0x7f0d940f89a0 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0d80062c90 0x7f0d80065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:58.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f0d8407c890 con 0x7f0d940f89a0 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 10 client.14637.objecter ms_dispatch 0x7f0d94001550 osd_map(85..85 src has 1..85) v4 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 3 client.14637.objecter handle_osd_map got epochs [85,85] > 0 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 3 client.14637.objecter handle_osd_map decoding full epoch 85 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d92ffd700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0d80062c90 0x7f0d80065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 10 client.14637.objecter checking command 1 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.137:6812/53025171 conn(0x7f0d8006c650 0x7f0d8006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 20 client.14637.objecter _get_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 20 client.14637.objecter put_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 15 client.14637.objecter _session_command_op_remove -1 1 2024-02-18T00:30:58.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 20 client.14637.objecter _get_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 20 client.14637.objecter get_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 15 client.14637.objecter _session_command_op_assign 7 1 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 20 client.14637.objecter put_session s=0x7f0d8006bfa0 osd=7 4 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 10 client.14637.objecter _send_command 1 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 --> v2:172.21.15.137:6812/53025171 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d8006f130 con 0x7f0d8006c650 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 20 client.14637.objecter dump_active .. 0 homeless 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d8407cc40 con 0x7f0d940f89a0 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d92ffd700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0d80062c90 0x7f0d80065150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0d7c0043d0 tx=0x7f0d7c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 10 client.14637.objecter ms_handle_connect 0x7f0d80062c90 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d937fe700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.137:6812/53025171 conn(0x7f0d8006c650 0x7f0d8006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d8bfff700 10 client.14637.objecter ms_handle_connect 0x7f0d8006c650 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.736+0000 7f0d937fe700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.137:6812/53025171 conn(0x7f0d8006c650 0x7f0d8006eaf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.737+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0d8006f130 con 0x7f0d8006c650 2024-02-18T00:30:58.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.737+0000 7f0d8bfff700 10 client.14637.objecter ms_dispatch 0x7f0d94001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.737+0000 7f0d8bfff700 10 client.14637.objecter _finish_command 1 = system:0 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.737+0000 7f0d8bfff700 20 client.14637.objecter put_session s=0x7f0d8006bfa0 osd=7 4 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.737+0000 7f0d8bfff700 15 client.14637.objecter _session_command_op_remove 7 1 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 10 client.14637.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 15 client.14637.objecter _session_command_op_assign -1 2 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 20 client.14637.objecter _get_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 20 client.14637.objecter put_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 20 client.14637.objecter _get_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 15 client.14637.objecter _session_command_op_remove -1 2 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 20 client.14637.objecter get_session s=0x7f0d8006bfa0 osd=7 3 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 15 client.14637.objecter _session_command_op_assign 7 2 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 20 client.14637.objecter put_session s=0x7f0d8006bfa0 osd=7 4 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 10 client.14637.objecter _send_command 2 2024-02-18T00:30:58.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 --> v2:172.21.15.137:6812/53025171 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0d94106040 con 0x7f0d8006c650 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d8bfff700 1 -- 172.21.15.59:0/923727771 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0d94106040 con 0x7f0d8006c650 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d8bfff700 10 client.14637.objecter ms_dispatch 0x7f0d94001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d8bfff700 10 client.14637.objecter _finish_command 2 = system:0 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d8bfff700 20 client.14637.objecter put_session s=0x7f0d8006bfa0 osd=7 4 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d8bfff700 15 client.14637.objecter _session_command_op_remove 7 2 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.752+0000 7f0d99a2f700 10 librados: watch_flush enter 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 10 librados: watch_flush exit 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 10 client.14637.objecter close_session for osd.7 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 >> v2:172.21.15.137:6812/53025171 conn(0x7f0d8006c650 msgr2=0x7f0d8006eaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.137:6812/53025171 conn(0x7f0d8006c650 0x7f0d8006eaf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 20 client.14637.objecter put_session s=0x7f0d8006bfa0 osd=7 1 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 20 client.14637.objecter shutdown clearing up homeless session... 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 10 client.14637.objecter successfully canceled tick 2024-02-18T00:30:58.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0d80062c90 msgr2=0x7f0d80065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 --2- 172.21.15.59:0/923727771 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0d80062c90 0x7f0d80065150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f0d7c0043d0 tx=0x7f0d7c00b040).stop 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 msgr2=0x7f0d9410e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 --2- 172.21.15.59:0/923727771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d940f89a0 0x7f0d9410e680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f0d840048f0 tx=0x7f0d84009cb0).stop 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d937fe700 1 -- 172.21.15.59:0/923727771 reap_dead start 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 shutdown_connections 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 >> 172.21.15.59:0/923727771 conn(0x7f0d940f26f0 msgr2=0x7f0d940f49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 shutdown_connections 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 -- 172.21.15.59:0/923727771 wait complete. 2024-02-18T00:30:58.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:30:58.753+0000 7f0d99a2f700 1 librados: shutdown 2024-02-18T00:30:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:57.736693+0000 mgr.y (mgr.24454) 70 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:58.599240+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:30:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:58.601690+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:30:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:59.056060+0000 osd.1 (osd.1) 37 : cluster [DBG] 5.c scrub starts 2024-02-18T00:30:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:59.058944+0000 osd.1 (osd.1) 38 : cluster [DBG] 5.c scrub ok 2024-02-18T00:30:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:57.736693+0000 mgr.y (mgr.24454) 70 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:58.599240+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:30:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:58.601690+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:30:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:59.056060+0000 osd.1 (osd.1) 37 : cluster [DBG] 5.c scrub starts 2024-02-18T00:30:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:30:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:59.058944+0000 osd.1 (osd.1) 38 : cluster [DBG] 5.c scrub ok 2024-02-18T00:30:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:57.736693+0000 mgr.y (mgr.24454) 70 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:30:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:58.599240+0000 osd.7 (osd.7) 39 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:30:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:58.601690+0000 osd.7 (osd.7) 40 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:30:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:59.056060+0000 osd.1 (osd.1) 37 : cluster [DBG] 5.c scrub starts 2024-02-18T00:30:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:30:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:59.058944+0000 osd.1 (osd.1) 38 : cluster [DBG] 5.c scrub ok 2024-02-18T00:30:59.563 INFO:teuthology.orchestra.run.smithi059.stdout:227633266712 2024-02-18T00:30:59.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-18T00:31:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:59.121345+0000 osd.3 (osd.3) 39 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:31:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:59.122213+0000 osd.3 (osd.3) 40 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:31:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:59.633045+0000 mon.a (mon.0) 736 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-18T00:31:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:30:59.767994+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:00.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:00.088038+0000 osd.1 (osd.1) 39 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:31:00.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:00.091165+0000 osd.1 (osd.1) 40 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:31:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:59.121345+0000 osd.3 (osd.3) 39 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:31:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:59.122213+0000 osd.3 (osd.3) 40 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:31:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:59.633045+0000 mon.a (mon.0) 736 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-18T00:31:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:30:59.767994+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:00.088038+0000 osd.1 (osd.1) 39 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:31:00.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:00.091165+0000 osd.1 (osd.1) 40 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:31:00.461 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:59.121345+0000 osd.3 (osd.3) 39 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:31:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:59.122213+0000 osd.3 (osd.3) 40 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:31:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:59.633045+0000 mon.a (mon.0) 736 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-18T00:31:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:30:59.767994+0000 mon.a (mon.0) 737 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:00.088038+0000 osd.1 (osd.1) 39 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:31:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:00.091165+0000 osd.1 (osd.1) 40 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:31:01.061 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:31:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:31:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:30:59.737016+0000 mgr.y (mgr.24454) 71 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:00.082288+0000 osd.3 (osd.3) 41 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:31:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:00.083328+0000 osd.3 (osd.3) 42 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:31:01.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:30:59.737016+0000 mgr.y (mgr.24454) 71 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:01.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:00.082288+0000 osd.3 (osd.3) 41 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:31:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:00.083328+0000 osd.3 (osd.3) 42 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:31:01.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:31:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:31:01.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:30:59.737016+0000 mgr.y (mgr.24454) 71 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:01.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:00.082288+0000 osd.3 (osd.3) 41 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:31:01.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:00.083328+0000 osd.3 (osd.3) 42 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:31:02.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:02] "GET /metrics HTTP/1.1" 200 172153 "" "Prometheus/2.33.4" 2024-02-18T00:31:02.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:00.803135+0000 mgr.y (mgr.24454) 72 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:02.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:01.009225+0000 osd.5 (osd.5) 25 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:31:02.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:01.012680+0000 osd.5 (osd.5) 26 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:31:02.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:02.088614+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:02.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:02.089463+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:31:02.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:02.090208+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:31:02.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:02.094453+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:02.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:00.803135+0000 mgr.y (mgr.24454) 72 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:02.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:01.009225+0000 osd.5 (osd.5) 25 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:31:02.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:01.012680+0000 osd.5 (osd.5) 26 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:31:02.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:02.088614+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:02.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:02.089463+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:31:02.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:02.090208+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:31:02.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:02.094453+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:02.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:00.803135+0000 mgr.y (mgr.24454) 72 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:02.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:01.009225+0000 osd.5 (osd.5) 25 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:31:02.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:01.012680+0000 osd.5 (osd.5) 26 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:31:02.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:02.088614+0000 mon.a (mon.0) 738 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:02.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:02.089463+0000 mon.a (mon.0) 739 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:31:02.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:02.090208+0000 mon.a (mon.0) 740 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:31:02.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:02.094453+0000 mon.a (mon.0) 741 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:03.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:01.737577+0000 mgr.y (mgr.24454) 73 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:01.881808+0000 osd.0 (osd.0) 29 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:01.883417+0000 osd.0 (osd.0) 30 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:02.144352+0000 osd.1 (osd.1) 41 : cluster [DBG] 5.1d deep-scrub starts 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:02.145245+0000 osd.1 (osd.1) 42 : cluster [DBG] 5.1d deep-scrub ok 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:01.737577+0000 mgr.y (mgr.24454) 73 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:01.881808+0000 osd.0 (osd.0) 29 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:01.883417+0000 osd.0 (osd.0) 30 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:02.144352+0000 osd.1 (osd.1) 41 : cluster [DBG] 5.1d deep-scrub starts 2024-02-18T00:31:03.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:02.145245+0000 osd.1 (osd.1) 42 : cluster [DBG] 5.1d deep-scrub ok 2024-02-18T00:31:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4005610735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80fe510 msgr2=0x7f0fb80f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4005610735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80fe510 0x7f0fb80f7d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f0fa8004870 tx=0x7f0fa8009cb0).stop 2024-02-18T00:31:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4005610735 shutdown_connections 2024-02-18T00:31:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4005610735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb80fdbb0 0x7f0fb80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4005610735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80f8420 0x7f0fb8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4005610735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80fe510 0x7f0fb80f7d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4005610735 >> 172.21.15.59:0/4005610735 conn(0x7f0fb80f3800 msgr2=0x7f0fb80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4005610735 shutdown_connections 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.209+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4005610735 wait complete. 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 librados: starting msgr at 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 librados: starting objecter 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 Processor -- start 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 -- start start 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 librados: setting wanted keys 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 librados: calling monclient init 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80f8420 0x7f0fb806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80fdbb0 0x7f0fb806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb806bab0 0x7f0fb8116610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb8116b00 con 0x7f0fb80f8420 2024-02-18T00:31:03.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb8116c40 con 0x7f0fb806bab0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbfc34700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb8116db0 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbd1cf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80fdbb0 0x7f0fb806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbd1cf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80fdbb0 0x7f0fb806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56950/0 (socket says 172.21.15.59:56950) 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbd1cf700 1 -- 172.21.15.59:0/4166751778 learned_addr learned my addr 172.21.15.59:0/4166751778 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbd9d0700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80f8420 0x7f0fb806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.210+0000 7f0fbe1d1700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb806bab0 0x7f0fb8116610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbd1cf700 1 -- 172.21.15.59:0/4166751778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb806bab0 msgr2=0x7f0fb8116610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbd1cf700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb806bab0 0x7f0fb8116610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbd1cf700 1 -- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80f8420 msgr2=0x7f0fb806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbd1cf700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80f8420 0x7f0fb806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbd1cf700 1 -- 172.21.15.59:0/4166751778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00097c0 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbd1cf700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80fdbb0 0x7f0fb806b5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0fa8004710 tx=0x7f0fa800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0faeffd700 10 client.?.objecter ms_handle_connect 0x7f0fb80fdbb0 2024-02-18T00:31:03.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0faeffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0faeffd700 1 -- 172.21.15.59:0/4166751778 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800e070 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0faeffd700 1 -- 172.21.15.59:0/4166751778 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa8013c70 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa8005040 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0faeffd700 1 -- 172.21.15.59:0/4166751778 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8019070 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbfc34700 10 client.24548.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb8117380 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.211+0000 7f0fbfc34700 1 librados: init done 2024-02-18T00:31:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.212+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb805ee80 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0faeffd700 1 -- 172.21.15.59:0/4166751778 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0fa801e420 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0faeffd700 1 --2- 172.21.15.59:0/4166751778 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0fa4062c90 0x7f0fa4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0faeffd700 1 -- 172.21.15.59:0/4166751778 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f0fa807d300 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0faeffd700 10 client.24548.objecter ms_dispatch 0x7f0fb8001550 osd_map(86..86 src has 1..86) v4 2024-02-18T00:31:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0fbd9d0700 1 --2- 172.21.15.59:0/4166751778 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0fa4062c90 0x7f0fa4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:03.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0faeffd700 3 client.24548.objecter handle_osd_map got epochs [86,86] > 0 2024-02-18T00:31:03.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0faeffd700 3 client.24548.objecter handle_osd_map decoding full epoch 86 2024-02-18T00:31:03.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.213+0000 7f0faeffd700 20 client.24548.objecter dump_active .. 0 homeless 2024-02-18T00:31:03.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.214+0000 7f0fbd9d0700 1 --2- 172.21.15.59:0/4166751778 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0fa4062c90 0x7f0fa4065150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0fb4004dc0 tx=0x7f0fb400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:03.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.214+0000 7f0faeffd700 10 client.24548.objecter ms_handle_connect 0x7f0fa4062c90 2024-02-18T00:31:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.217+0000 7f0faeffd700 1 -- 172.21.15.59:0/4166751778 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0fa8050f70 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.389+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0fb806c5e0 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.390+0000 7f0faeffd700 1 -- 172.21.15.59:0/4166751778 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0fa8023020 con 0x7f0fb80fdbb0 2024-02-18T00:31:03.391 INFO:teuthology.orchestra.run.smithi059.stdout:34359738428 2024-02-18T00:31:03.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 10 librados: watch_flush enter 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 10 librados: watch_flush exit 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 20 client.24548.objecter shutdown clearing up homeless session... 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 10 client.24548.objecter successfully canceled tick 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0fa4062c90 msgr2=0x7f0fa4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4166751778 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0fa4062c90 0x7f0fa4065150 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0fb4004dc0 tx=0x7f0fb400b040).stop 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80fdbb0 msgr2=0x7f0fb806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80fdbb0 0x7f0fb806b5c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0fa8004710 tx=0x7f0fa800b040).stop 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 shutdown_connections 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4166751778 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0fa4062c90 0x7f0fa4065150 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0fb806bab0 0x7f0fb8116610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fb80f8420 0x7f0fb806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 --2- 172.21.15.59:0/4166751778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0fb80fdbb0 0x7f0fb806b5c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.391+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 >> 172.21.15.59:0/4166751778 conn(0x7f0fb80f3800 msgr2=0x7f0fb80fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.392+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 shutdown_connections 2024-02-18T00:31:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.392+0000 7f0fbfc34700 1 -- 172.21.15.59:0/4166751778 wait complete. 2024-02-18T00:31:03.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:03.392+0000 7f0fbfc34700 1 librados: shutdown 2024-02-18T00:31:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:01.737577+0000 mgr.y (mgr.24454) 73 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:01.881808+0000 osd.0 (osd.0) 29 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:31:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:01.883417+0000 osd.0 (osd.0) 30 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:31:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:02.144352+0000 osd.1 (osd.1) 41 : cluster [DBG] 5.1d deep-scrub starts 2024-02-18T00:31:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:02.145245+0000 osd.1 (osd.1) 42 : cluster [DBG] 5.1d deep-scrub ok 2024-02-18T00:31:04.135 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738421 got 34359738428 for osd.0 2024-02-18T00:31:04.136 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-18T00:31:04.420 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:02.921761+0000 osd.0 (osd.0) 31 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:31:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:02.922767+0000 osd.0 (osd.0) 32 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:31:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:03.390109+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.59:0/4166751778' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T00:31:04.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:02.921761+0000 osd.0 (osd.0) 31 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:31:04.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:02.922767+0000 osd.0 (osd.0) 32 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:31:04.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:03.390109+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.59:0/4166751778' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T00:31:04.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:02.921761+0000 osd.0 (osd.0) 31 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:31:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:02.922767+0000 osd.0 (osd.0) 32 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:31:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:03.390109+0000 mon.c (mon.2) 20 : audit [DBG] from='client.? 172.21.15.59:0/4166751778' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T00:31:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:03.738095+0000 mgr.y (mgr.24454) 74 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:03.939644+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:31:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:03.941187+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:31:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:04.043656+0000 osd.3 (osd.3) 43 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:31:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:04.044590+0000 osd.3 (osd.3) 44 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:31:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:05.055204+0000 osd.3 (osd.3) 45 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:31:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:05.056859+0000 osd.3 (osd.3) 46 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:31:05.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:03.738095+0000 mgr.y (mgr.24454) 74 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:05.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:03.939644+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:31:05.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:03.941187+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:31:05.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:04.043656+0000 osd.3 (osd.3) 43 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:31:05.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:04.044590+0000 osd.3 (osd.3) 44 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:31:05.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:05.055204+0000 osd.3 (osd.3) 45 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:31:05.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:05.056859+0000 osd.3 (osd.3) 46 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:03.738095+0000 mgr.y (mgr.24454) 74 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:03.939644+0000 osd.4 (osd.4) 35 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:03.941187+0000 osd.4 (osd.4) 36 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:04.043656+0000 osd.3 (osd.3) 43 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:04.044590+0000 osd.3 (osd.3) 44 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:05.055204+0000 osd.3 (osd.3) 45 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:31:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:05.056859+0000 osd.3 (osd.3) 46 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 -- 172.21.15.59:0/146948625 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200fcbb0 msgr2=0x7f01200fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 --2- 172.21.15.59:0/146948625 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200fcbb0 0x7f01200fcf90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f01100047b0 tx=0x7f0110009cb0).stop 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 -- 172.21.15.59:0/146948625 shutdown_connections 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 --2- 172.21.15.59:0/146948625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200f7400 0x7f0120103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 --2- 172.21.15.59:0/146948625 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200fcbb0 0x7f01200fcf90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 --2- 172.21.15.59:0/146948625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01200fd510 0x7f01200f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 -- 172.21.15.59:0/146948625 >> 172.21.15.59:0/146948625 conn(0x7f01200f2820 msgr2=0x7f01200f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 -- 172.21.15.59:0/146948625 shutdown_connections 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 -- 172.21.15.59:0/146948625 wait complete. 2024-02-18T00:31:06.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 librados: starting msgr at 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 librados: starting objecter 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.398+0000 7f0125e8a700 1 Processor -- start 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 -- start start 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 librados: setting wanted keys 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 librados: calling monclient init 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200f7400 0x7f01201092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01200fcbb0 0x7f01201097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200fd510 0x7f0120109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0120114430 con 0x7f01200fd510 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201145a0 con 0x7f01200fcbb0 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f0125e8a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0120114760 con 0x7f01200f7400 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200f7400 0x7f01201092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200fd510 0x7f0120109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:06.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200f7400 0x7f01201092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56974/0 (socket says 172.21.15.59:56974) 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200fd510 0x7f0120109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40564/0 (socket says 172.21.15.59:40564) 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011f7fe700 1 -- 172.21.15.59:0/679405692 learned_addr learned my addr 172.21.15.59:0/679405692 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011effd700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01200fcbb0 0x7f01201097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 -- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200f7400 msgr2=0x7f01201092d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200f7400 0x7f01201092d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 -- 172.21.15.59:0/679405692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01200fcbb0 msgr2=0x7f01201097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01200fcbb0 0x7f01201097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 -- 172.21.15.59:0/679405692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180097c0 con 0x7f01200fd510 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011f7fe700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200f7400 0x7f01201092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011ffff700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200fd510 0x7f0120109cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f0118001df0 tx=0x7f0118004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011cff9700 10 client.?.objecter ms_handle_connect 0x7f01200fd510 2024-02-18T00:31:06.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011cff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.399+0000 7f011cff9700 1 -- 172.21.15.59:0/679405692 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180099f0 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.400+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0110005040 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.400+0000 7f011cff9700 1 -- 172.21.15.59:0/679405692 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01180095a0 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.400+0000 7f011cff9700 1 -- 172.21.15.59:0/679405692 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118007d20 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.400+0000 7f0125e8a700 10 client.14649.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.400+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0120114d50 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.400+0000 7f0125e8a700 1 librados: init done 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.401+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012005fa60 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.401+0000 7f011cff9700 1 -- 172.21.15.59:0/679405692 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f011800fb80 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.401+0000 7f011cff9700 1 --2- 172.21.15.59:0/679405692 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0108062c90 0x7f0108065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011f7fe700 1 --2- 172.21.15.59:0/679405692 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0108062c90 0x7f0108065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011cff9700 1 -- 172.21.15.59:0/679405692 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f011807d350 con 0x7f01200fd510 2024-02-18T00:31:06.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011cff9700 10 client.14649.objecter ms_dispatch 0x7f0120001550 osd_map(86..86 src has 1..86) v4 2024-02-18T00:31:06.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011cff9700 3 client.14649.objecter handle_osd_map got epochs [86,86] > 0 2024-02-18T00:31:06.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011cff9700 3 client.14649.objecter handle_osd_map decoding full epoch 86 2024-02-18T00:31:06.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011cff9700 20 client.14649.objecter dump_active .. 0 homeless 2024-02-18T00:31:06.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011cff9700 10 client.14649.objecter ms_handle_connect 0x7f0108062c90 2024-02-18T00:31:06.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.402+0000 7f011f7fe700 1 --2- 172.21.15.59:0/679405692 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0108062c90 0x7f0108065150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0110004e20 tx=0x7f0110009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:06.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.406+0000 7f011cff9700 1 -- 172.21.15.59:0/679405692 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0118050fc0 con 0x7f01200fd510 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:05.069959+0000 osd.5 (osd.5) 27 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:05.070926+0000 osd.5 (osd.5) 28 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:06.047696+0000 osd.3 (osd.3) 47 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:06.048884+0000 osd.3 (osd.3) 48 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:05.069959+0000 osd.5 (osd.5) 27 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:05.070926+0000 osd.5 (osd.5) 28 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:06.047696+0000 osd.3 (osd.3) 47 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:31:06.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:06.048884+0000 osd.3 (osd.3) 48 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:31:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:05.069959+0000 osd.5 (osd.5) 27 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:31:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:05.070926+0000 osd.5 (osd.5) 28 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:31:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:06.047696+0000 osd.3 (osd.3) 47 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:31:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:06.048884+0000 osd.3 (osd.3) 48 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:31:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.581+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f01201061d0 con 0x7f01200fd510 2024-02-18T00:31:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.581+0000 7f011cff9700 1 -- 172.21.15.59:0/679405692 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0118050bb0 con 0x7f01200fd510 2024-02-18T00:31:06.583 INFO:teuthology.orchestra.run.smithi059.stdout:55834574903 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 10 librados: watch_flush enter 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 10 librados: watch_flush exit 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 20 client.14649.objecter shutdown clearing up homeless session... 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 10 client.14649.objecter successfully canceled tick 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0108062c90 msgr2=0x7f0108065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 --2- 172.21.15.59:0/679405692 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0108062c90 0x7f0108065150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0110004e20 tx=0x7f0110009cb0).stop 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200fd510 msgr2=0x7f0120109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200fd510 0x7f0120109cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f0118001df0 tx=0x7f0118004d10).stop 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 shutdown_connections 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01200fd510 0x7f0120109cd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01200f7400 0x7f01201092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 --2- 172.21.15.59:0/679405692 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0108062c90 0x7f0108065150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 --2- 172.21.15.59:0/679405692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01200fcbb0 0x7f01201097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 >> 172.21.15.59:0/679405692 conn(0x7f01200f2820 msgr2=0x7f01200fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 shutdown_connections 2024-02-18T00:31:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 -- 172.21.15.59:0/679405692 wait complete. 2024-02-18T00:31:06.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:06.583+0000 7f0125e8a700 1 librados: shutdown 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:05.738769+0000 mgr.y (mgr.24454) 75 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:05.993973+0000 osd.4 (osd.4) 37 : cluster [DBG] 5.e scrub starts 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:05.997456+0000 osd.4 (osd.4) 38 : cluster [DBG] 5.e scrub ok 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:06.039452+0000 osd.5 (osd.5) 29 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:06.040421+0000 osd.5 (osd.5) 30 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:06.582642+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.59:0/679405692' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.065634+0000 osd.3 (osd.3) 49 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.066525+0000 osd.3 (osd.3) 50 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:31:07.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:05.738769+0000 mgr.y (mgr.24454) 75 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:05.993973+0000 osd.4 (osd.4) 37 : cluster [DBG] 5.e scrub starts 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:05.997456+0000 osd.4 (osd.4) 38 : cluster [DBG] 5.e scrub ok 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:06.039452+0000 osd.5 (osd.5) 29 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:06.040421+0000 osd.5 (osd.5) 30 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:06.582642+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.59:0/679405692' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.065634+0000 osd.3 (osd.3) 49 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.066525+0000 osd.3 (osd.3) 50 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:31:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:05.738769+0000 mgr.y (mgr.24454) 75 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:05.993973+0000 osd.4 (osd.4) 37 : cluster [DBG] 5.e scrub starts 2024-02-18T00:31:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:05.997456+0000 osd.4 (osd.4) 38 : cluster [DBG] 5.e scrub ok 2024-02-18T00:31:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:06.039452+0000 osd.5 (osd.5) 29 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:31:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:06.040421+0000 osd.5 (osd.5) 30 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:31:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:06.582642+0000 mon.a (mon.0) 742 : audit [DBG] from='client.? 172.21.15.59:0/679405692' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T00:31:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.065634+0000 osd.3 (osd.3) 49 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:31:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.066525+0000 osd.3 (osd.3) 50 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:31:07.527 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574896 got 55834574903 for osd.1 2024-02-18T00:31:07.528 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-18T00:31:07.811 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.027867+0000 osd.5 (osd.5) 31 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:31:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.029672+0000 osd.5 (osd.5) 32 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:31:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.027867+0000 osd.5 (osd.5) 31 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:31:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.029672+0000 osd.5 (osd.5) 32 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:31:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.027867+0000 osd.5 (osd.5) 31 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:31:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.029672+0000 osd.5 (osd.5) 32 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:31:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.739088+0000 mgr.y (mgr.24454) 76 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.941309+0000 osd.4 (osd.4) 39 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:31:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.942429+0000 osd.4 (osd.4) 40 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:31:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.949648+0000 osd.0 (osd.0) 33 : cluster [DBG] 4.b scrub starts 2024-02-18T00:31:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:07.953260+0000 osd.0 (osd.0) 34 : cluster [DBG] 4.b scrub ok 2024-02-18T00:31:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:08.044855+0000 osd.5 (osd.5) 33 : cluster [DBG] 2.2 deep-scrub starts 2024-02-18T00:31:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:08.045940+0000 osd.5 (osd.5) 34 : cluster [DBG] 2.2 deep-scrub ok 2024-02-18T00:31:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.739088+0000 mgr.y (mgr.24454) 76 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.941309+0000 osd.4 (osd.4) 39 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:31:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.942429+0000 osd.4 (osd.4) 40 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:31:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.949648+0000 osd.0 (osd.0) 33 : cluster [DBG] 4.b scrub starts 2024-02-18T00:31:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:07.953260+0000 osd.0 (osd.0) 34 : cluster [DBG] 4.b scrub ok 2024-02-18T00:31:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:08.044855+0000 osd.5 (osd.5) 33 : cluster [DBG] 2.2 deep-scrub starts 2024-02-18T00:31:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:08.045940+0000 osd.5 (osd.5) 34 : cluster [DBG] 2.2 deep-scrub ok 2024-02-18T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.739088+0000 mgr.y (mgr.24454) 76 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.941309+0000 osd.4 (osd.4) 39 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.942429+0000 osd.4 (osd.4) 40 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.949648+0000 osd.0 (osd.0) 33 : cluster [DBG] 4.b scrub starts 2024-02-18T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:07.953260+0000 osd.0 (osd.0) 34 : cluster [DBG] 4.b scrub ok 2024-02-18T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:08.044855+0000 osd.5 (osd.5) 33 : cluster [DBG] 2.2 deep-scrub starts 2024-02-18T00:31:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:08.045940+0000 osd.5 (osd.5) 34 : cluster [DBG] 2.2 deep-scrub ok 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 -- 172.21.15.59:0/795033732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fd510 msgr2=0x7fb2240f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 --2- 172.21.15.59:0/795033732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fd510 0x7fb2240f6d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fb20c0047b0 tx=0x7fb20c009cb0).stop 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 -- 172.21.15.59:0/795033732 shutdown_connections 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 --2- 172.21.15.59:0/795033732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2240fcbb0 0x7fb2240fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 --2- 172.21.15.59:0/795033732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240f7400 0x7fb224103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 --2- 172.21.15.59:0/795033732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fd510 0x7fb2240f6d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 -- 172.21.15.59:0/795033732 >> 172.21.15.59:0/795033732 conn(0x7fb2240f2820 msgr2=0x7fb2240f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 -- 172.21.15.59:0/795033732 shutdown_connections 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.901+0000 7fb22a574700 1 -- 172.21.15.59:0/795033732 wait complete. 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 librados: starting msgr at 2024-02-18T00:31:09.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 librados: starting objecter 2024-02-18T00:31:09.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 Processor -- start 2024-02-18T00:31:09.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 -- start start 2024-02-18T00:31:09.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 librados: setting wanted keys 2024-02-18T00:31:09.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 librados: calling monclient init 2024-02-18T00:31:09.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2240f7400 0x7fb22410d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fcbb0 0x7fb22410d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240fd510 0x7fb224107220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224107710 con 0x7fb2240fcbb0 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224107880 con 0x7fb2240f7400 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.902+0000 7fb22a574700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb224107a40 con 0x7fb2240fd510 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb228b11700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240fd510 0x7fb224107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fcbb0 0x7fb22410d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fcbb0 0x7fb22410d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41894/0 (socket says 172.21.15.59:41894) 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb228b11700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240fd510 0x7fb224107220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:48824/0 (socket says 172.21.15.59:48824) 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb228b11700 1 -- 172.21.15.59:0/3677714049 learned_addr learned my addr 172.21.15.59:0/3677714049 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb223fff700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2240f7400 0x7fb22410d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:09.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 -- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240fd510 msgr2=0x7fb224107220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240fd510 0x7fb224107220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 -- 172.21.15.59:0/3677714049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2240f7400 msgr2=0x7fb22410d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2240f7400 0x7fb22410d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 -- 172.21.15.59:0/3677714049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c0097c0 con 0x7fb2240fcbb0 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb228b11700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240fd510 0x7fb224107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2217fa700 10 client.?.objecter ms_handle_connect 0x7fb2240fcbb0 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2217fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2237fe700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fcbb0 0x7fb22410d570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb20c0047b0 tx=0x7fb20c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2217fa700 1 -- 172.21.15.59:0/3677714049 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c018070 con 0x7fb2240fcbb0 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c005040 con 0x7fb2240fcbb0 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2217fa700 1 -- 172.21.15.59:0/3677714049 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb20c013da0 con 0x7fb2240fcbb0 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb2217fa700 1 -- 172.21.15.59:0/3677714049 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c01dbd0 con 0x7fb2240fcbb0 2024-02-18T00:31:09.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb22a574700 10 client.14655.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:09.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb224107ee0 con 0x7fb2240fcbb0 2024-02-18T00:31:09.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.903+0000 7fb22a574700 1 librados: init done 2024-02-18T00:31:09.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.904+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2241080d0 con 0x7fb2240fcbb0 2024-02-18T00:31:09.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.905+0000 7fb2217fa700 1 -- 172.21.15.59:0/3677714049 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb20c01d420 con 0x7fb2240fcbb0 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.905+0000 7fb2217fa700 1 --2- 172.21.15.59:0/3677714049 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb210062c90 0x7fb210065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb2217fa700 1 -- 172.21.15.59:0/3677714049 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fb20c07e230 con 0x7fb2240fcbb0 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb2217fa700 10 client.14655.objecter ms_dispatch 0x7fb224001550 osd_map(86..86 src has 1..86) v4 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb2217fa700 3 client.14655.objecter handle_osd_map got epochs [86,86] > 0 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb2217fa700 3 client.14655.objecter handle_osd_map decoding full epoch 86 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb223fff700 1 --2- 172.21.15.59:0/3677714049 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb210062c90 0x7fb210065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb2217fa700 20 client.14655.objecter dump_active .. 0 homeless 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb2217fa700 10 client.14655.objecter ms_handle_connect 0x7fb210062c90 2024-02-18T00:31:09.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.906+0000 7fb223fff700 1 --2- 172.21.15.59:0/3677714049 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb210062c90 0x7fb210065150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb2140041d0 tx=0x7fb214009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:09.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:09.909+0000 7fb2217fa700 1 -- 172.21.15.59:0/3677714049 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb20c051f40 con 0x7fb2240fcbb0 2024-02-18T00:31:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.083+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb22405fa60 con 0x7fb2240fcbb0 2024-02-18T00:31:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.084+0000 7fb2217fa700 1 -- 172.21.15.59:0/3677714049 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb20c051b30 con 0x7fb2240fcbb0 2024-02-18T00:31:10.086 INFO:teuthology.orchestra.run.smithi059.stdout:77309411380 2024-02-18T00:31:10.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.085+0000 7fb22a574700 10 librados: watch_flush enter 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.085+0000 7fb22a574700 10 librados: watch_flush exit 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.085+0000 7fb22a574700 20 client.14655.objecter shutdown clearing up homeless session... 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.085+0000 7fb22a574700 10 client.14655.objecter successfully canceled tick 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb210062c90 msgr2=0x7fb210065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 --2- 172.21.15.59:0/3677714049 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb210062c90 0x7fb210065150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb2140041d0 tx=0x7fb214009040).stop 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fcbb0 msgr2=0x7fb22410d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fcbb0 0x7fb22410d570 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fb20c0047b0 tx=0x7fb20c009cb0).stop 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 shutdown_connections 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 --2- 172.21.15.59:0/3677714049 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb210062c90 0x7fb210065150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2240f7400 0x7fb22410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:10.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2240fd510 0x7fb224107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 --2- 172.21.15.59:0/3677714049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2240fcbb0 0x7fb22410d570 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 >> 172.21.15.59:0/3677714049 conn(0x7fb2240f2820 msgr2=0x7fb2240f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 shutdown_connections 2024-02-18T00:31:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 -- 172.21.15.59:0/3677714049 wait complete. 2024-02-18T00:31:10.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:10.086+0000 7fb22a574700 1 librados: shutdown 2024-02-18T00:31:10.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:10.085290+0000 mon.a (mon.0) 743 : audit [DBG] from='client.? 172.21.15.59:0/3677714049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T00:31:10.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:10.085290+0000 mon.a (mon.0) 743 : audit [DBG] from='client.? 172.21.15.59:0/3677714049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T00:31:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:10.085290+0000 mon.a (mon.0) 743 : audit [DBG] from='client.? 172.21.15.59:0/3677714049' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T00:31:10.841 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411374 got 77309411380 for osd.2 2024-02-18T00:31:10.842 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-18T00:31:11.128 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:11.203 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:31:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:31:11.203 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:31:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:31:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:09.739459+0000 mgr.y (mgr.24454) 77 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-02-18T00:31:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:11.017182+0000 osd.1 (osd.1) 43 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:31:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:11.020115+0000 osd.1 (osd.1) 44 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:31:11.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:09.739459+0000 mgr.y (mgr.24454) 77 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-02-18T00:31:11.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:11.017182+0000 osd.1 (osd.1) 43 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:31:11.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:11.020115+0000 osd.1 (osd.1) 44 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:31:11.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:09.739459+0000 mgr.y (mgr.24454) 77 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-02-18T00:31:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:11.017182+0000 osd.1 (osd.1) 43 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:31:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:11.020115+0000 osd.1 (osd.1) 44 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:31:12.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:12] "GET /metrics HTTP/1.1" 200 172168 "" "Prometheus/2.33.4" 2024-02-18T00:31:12.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:10.814092+0000 mgr.y (mgr.24454) 78 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:12.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:10.998555+0000 osd.4 (osd.4) 41 : cluster [DBG] 4.c scrub starts 2024-02-18T00:31:12.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:11.002034+0000 osd.4 (osd.4) 42 : cluster [DBG] 4.c scrub ok 2024-02-18T00:31:12.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:11.740304+0000 mgr.y (mgr.24454) 79 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:12.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:10.814092+0000 mgr.y (mgr.24454) 78 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:12.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:10.998555+0000 osd.4 (osd.4) 41 : cluster [DBG] 4.c scrub starts 2024-02-18T00:31:12.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:11.002034+0000 osd.4 (osd.4) 42 : cluster [DBG] 4.c scrub ok 2024-02-18T00:31:12.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:11.740304+0000 mgr.y (mgr.24454) 79 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:10.814092+0000 mgr.y (mgr.24454) 78 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:10.998555+0000 osd.4 (osd.4) 41 : cluster [DBG] 4.c scrub starts 2024-02-18T00:31:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:11.002034+0000 osd.4 (osd.4) 42 : cluster [DBG] 4.c scrub ok 2024-02-18T00:31:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:11.740304+0000 mgr.y (mgr.24454) 79 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 -- 172.21.15.59:0/2740790658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80fdbb0 msgr2=0x7efdf80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 --2- 172.21.15.59:0/2740790658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80fdbb0 0x7efdf80fdf90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7efdf4004870 tx=0x7efdf4009cb0).stop 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 -- 172.21.15.59:0/2740790658 shutdown_connections 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 --2- 172.21.15.59:0/2740790658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf80f8420 0x7efdf8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 --2- 172.21.15.59:0/2740790658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80fdbb0 0x7efdf80fdf90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 --2- 172.21.15.59:0/2740790658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fe510 0x7efdf80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 -- 172.21.15.59:0/2740790658 >> 172.21.15.59:0/2740790658 conn(0x7efdf80f3800 msgr2=0x7efdf80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 -- 172.21.15.59:0/2740790658 shutdown_connections 2024-02-18T00:31:13.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 -- 172.21.15.59:0/2740790658 wait complete. 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 librados: starting msgr at 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.039+0000 7efe0043e700 1 librados: starting objecter 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 Processor -- start 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 -- start start 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 librados: setting wanted keys 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 librados: calling monclient init 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80f8420 0x7efdf810e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fdbb0 0x7efdf810e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf80fe510 0x7efdf8108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf8108770 con 0x7efdf80f8420 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf81088e0 con 0x7efdf80fe510 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efe0043e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf8108aa0 con 0x7efdf80fdbb0 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fdbb0 0x7efdf810e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:13.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfe1da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80f8420 0x7efdf810e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fdbb0 0x7efdf810e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:48848/0 (socket says 172.21.15.59:48848) 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 -- 172.21.15.59:0/348718169 learned_addr learned my addr 172.21.15.59:0/348718169 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfe9db700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf80fe510 0x7efdf8108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 -- 172.21.15.59:0/348718169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf80fe510 msgr2=0x7efdf8108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf80fe510 0x7efdf8108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 -- 172.21.15.59:0/348718169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80f8420 msgr2=0x7efdf810e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80f8420 0x7efdf810e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 -- 172.21.15.59:0/348718169 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8005040 con 0x7efdf80fdbb0 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdfd9d9700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fdbb0 0x7efdf810e580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7efde80048f0 tx=0x7efde8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdef7fe700 10 client.?.objecter ms_handle_connect 0x7efdf80fdbb0 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.040+0000 7efdef7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efdef7fe700 1 -- 172.21.15.59:0/348718169 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8007b00 con 0x7efdf80fdbb0 2024-02-18T00:31:13.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efdef7fe700 1 -- 172.21.15.59:0/348718169 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde800f040 con 0x7efdf80fdbb0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4005040 con 0x7efdf80fdbb0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efdef7fe700 1 -- 172.21.15.59:0/348718169 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8014a00 con 0x7efdf80fdbb0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efe0043e700 10 client.24566.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf81090e0 con 0x7efdf80fdbb0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efe0043e700 1 librados: init done 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.041+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf805ee80 con 0x7efdf80fdbb0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.042+0000 7efdef7fe700 1 -- 172.21.15.59:0/348718169 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efde800e580 con 0x7efdf80fdbb0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdef7fe700 1 --2- 172.21.15.59:0/348718169 >> v2:172.21.15.59:6800/3721891797 conn(0x7efde4062c90 0x7efde4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdef7fe700 1 -- 172.21.15.59:0/348718169 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6172+0+0 (secure 0 0 0) 0x7efde807c940 con 0x7efdf80fdbb0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdef7fe700 10 client.24566.objecter ms_dispatch 0x7efdf8001550 osd_map(86..86 src has 1..86) v4 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdef7fe700 3 client.24566.objecter handle_osd_map got epochs [86,86] > 0 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdef7fe700 3 client.24566.objecter handle_osd_map decoding full epoch 86 2024-02-18T00:31:13.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdfe1da700 1 --2- 172.21.15.59:0/348718169 >> v2:172.21.15.59:6800/3721891797 conn(0x7efde4062c90 0x7efde4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdef7fe700 20 client.24566.objecter dump_active .. 0 homeless 2024-02-18T00:31:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdef7fe700 10 client.24566.objecter ms_handle_connect 0x7efde4062c90 2024-02-18T00:31:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.043+0000 7efdfe1da700 1 --2- 172.21.15.59:0/348718169 >> v2:172.21.15.59:6800/3721891797 conn(0x7efde4062c90 0x7efde4065150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7efdf40043d0 tx=0x7efdf400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:13.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.046+0000 7efdef7fe700 1 -- 172.21.15.59:0/348718169 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efde8050650 con 0x7efdf80fdbb0 2024-02-18T00:31:13.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:12.040985+0000 osd.4 (osd.4) 43 : cluster [DBG] 6.17 deep-scrub starts 2024-02-18T00:31:13.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:12.043989+0000 osd.4 (osd.4) 44 : cluster [DBG] 6.17 deep-scrub ok 2024-02-18T00:31:13.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.219+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efdf8107060 con 0x7efdf80fdbb0 2024-02-18T00:31:13.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.220+0000 7efdef7fe700 1 -- 172.21.15.59:0/348718169 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efde8050240 con 0x7efdf80fdbb0 2024-02-18T00:31:13.222 INFO:teuthology.orchestra.run.smithi059.stdout:111669149743 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.221+0000 7efe0043e700 10 librados: watch_flush enter 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 10 librados: watch_flush exit 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 20 client.24566.objecter shutdown clearing up homeless session... 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 10 client.24566.objecter successfully canceled tick 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 >> v2:172.21.15.59:6800/3721891797 conn(0x7efde4062c90 msgr2=0x7efde4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 --2- 172.21.15.59:0/348718169 >> v2:172.21.15.59:6800/3721891797 conn(0x7efde4062c90 0x7efde4065150 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7efdf40043d0 tx=0x7efdf400b040).stop 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fdbb0 msgr2=0x7efdf810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fdbb0 0x7efdf810e580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7efde80048f0 tx=0x7efde8009cb0).stop 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 shutdown_connections 2024-02-18T00:31:13.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 --2- 172.21.15.59:0/348718169 >> v2:172.21.15.59:6800/3721891797 conn(0x7efde4062c90 0x7efde4065150 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdf80fe510 0x7efdf8108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdf80f8420 0x7efdf810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 --2- 172.21.15.59:0/348718169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdf80fdbb0 0x7efdf810e580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 >> 172.21.15.59:0/348718169 conn(0x7efdf80f3800 msgr2=0x7efdf80fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 shutdown_connections 2024-02-18T00:31:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 -- 172.21.15.59:0/348718169 wait complete. 2024-02-18T00:31:13.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:13.222+0000 7efe0043e700 1 librados: shutdown 2024-02-18T00:31:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:12.040985+0000 osd.4 (osd.4) 43 : cluster [DBG] 6.17 deep-scrub starts 2024-02-18T00:31:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:12.043989+0000 osd.4 (osd.4) 44 : cluster [DBG] 6.17 deep-scrub ok 2024-02-18T00:31:13.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:12.040985+0000 osd.4 (osd.4) 43 : cluster [DBG] 6.17 deep-scrub starts 2024-02-18T00:31:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:12.043989+0000 osd.4 (osd.4) 44 : cluster [DBG] 6.17 deep-scrub ok 2024-02-18T00:31:14.135 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149737 got 111669149743 for osd.3 2024-02-18T00:31:14.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-18T00:31:14.416 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:14.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:13.221253+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.59:0/348718169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T00:31:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:13.740647+0000 mgr.y (mgr.24454) 80 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:14.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:13.221253+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.59:0/348718169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T00:31:14.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:13.740647+0000 mgr.y (mgr.24454) 80 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:13.221253+0000 mon.c (mon.2) 21 : audit [DBG] from='client.? 172.21.15.59:0/348718169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T00:31:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:13.740647+0000 mgr.y (mgr.24454) 80 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:13.959063+0000 osd.0 (osd.0) 35 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:31:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:13.960084+0000 osd.0 (osd.0) 36 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:31:15.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:13.959063+0000 osd.0 (osd.0) 35 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:31:15.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:13.960084+0000 osd.0 (osd.0) 36 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:31:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:13.959063+0000 osd.0 (osd.0) 35 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:31:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:13.960084+0000 osd.0 (osd.0) 36 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 -- 172.21.15.59:0/4074243054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380f7400 msgr2=0x7ff338103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 --2- 172.21.15.59:0/4074243054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380f7400 0x7ff338103a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ff330009be0 tx=0x7ff330004d10).stop 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 -- 172.21.15.59:0/4074243054 shutdown_connections 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 --2- 172.21.15.59:0/4074243054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3380fcbb0 0x7ff3380fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 --2- 172.21.15.59:0/4074243054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380f7400 0x7ff338103a20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 --2- 172.21.15.59:0/4074243054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 0x7ff3380f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 -- 172.21.15.59:0/4074243054 >> 172.21.15.59:0/4074243054 conn(0x7ff3380f2820 msgr2=0x7ff3380f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 -- 172.21.15.59:0/4074243054 shutdown_connections 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 -- 172.21.15.59:0/4074243054 wait complete. 2024-02-18T00:31:16.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 librados: starting msgr at 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 librados: starting objecter 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 Processor -- start 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 -- start start 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 librados: setting wanted keys 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.423+0000 7ff33e57b700 1 librados: calling monclient init 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33e57b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3380f7400 0x7ff33810e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33e57b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380fcbb0 0x7ff33810e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33e57b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 0x7ff3381081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33e57b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3381086e0 con 0x7ff3380fd510 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33e57b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff338108850 con 0x7ff3380f7400 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33e57b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff338108a10 con 0x7ff3380fcbb0 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33cb18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 0x7ff3381081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3377fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380fcbb0 0x7ff33810e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:16.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33cb18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 0x7ff3381081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41932/0 (socket says 172.21.15.59:41932) 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33cb18700 1 -- 172.21.15.59:0/969121346 learned_addr learned my addr 172.21.15.59:0/969121346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff337fff700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3380f7400 0x7ff33810e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3377fe700 1 -- 172.21.15.59:0/969121346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3380f7400 msgr2=0x7ff33810e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3377fe700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3380f7400 0x7ff33810e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3377fe700 1 -- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 msgr2=0x7ff3381081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3377fe700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 0x7ff3381081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3377fe700 1 -- 172.21.15.59:0/969121346 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c005040 con 0x7ff3380fcbb0 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff33cb18700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 0x7ff3381081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff337fff700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3380f7400 0x7ff33810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3357fa700 10 client.?.objecter ms_handle_connect 0x7ff3380fcbb0 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3357fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.424+0000 7ff3377fe700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380fcbb0 0x7ff33810e4f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff32c004920 tx=0x7ff32c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.425+0000 7ff3357fa700 1 -- 172.21.15.59:0/969121346 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c007a40 con 0x7ff3380fcbb0 2024-02-18T00:31:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.425+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3300097c0 con 0x7ff3380fcbb0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.425+0000 7ff3357fa700 1 -- 172.21.15.59:0/969121346 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff32c00f040 con 0x7ff3380fcbb0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.425+0000 7ff3357fa700 1 -- 172.21.15.59:0/969121346 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c014a00 con 0x7ff3380fcbb0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.425+0000 7ff33e57b700 10 client.24572.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.425+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff338109050 con 0x7ff3380fcbb0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.425+0000 7ff33e57b700 1 librados: init done 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.426+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33805fa60 con 0x7ff3380fcbb0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.426+0000 7ff3357fa700 1 -- 172.21.15.59:0/969121346 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff32c010530 con 0x7ff3380fcbb0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.426+0000 7ff3357fa700 1 --2- 172.21.15.59:0/969121346 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff320062c90 0x7ff320065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff3357fa700 1 -- 172.21.15.59:0/969121346 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6172+0+0 (secure 0 0 0) 0x7ff32c07c670 con 0x7ff3380fcbb0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff3357fa700 10 client.24572.objecter ms_dispatch 0x7ff338001550 osd_map(86..86 src has 1..86) v4 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff3357fa700 3 client.24572.objecter handle_osd_map got epochs [86,86] > 0 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff3357fa700 3 client.24572.objecter handle_osd_map decoding full epoch 86 2024-02-18T00:31:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff337fff700 1 --2- 172.21.15.59:0/969121346 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff320062c90 0x7ff320065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:16.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff3357fa700 20 client.24572.objecter dump_active .. 0 homeless 2024-02-18T00:31:16.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff3357fa700 10 client.24572.objecter ms_handle_connect 0x7ff320062c90 2024-02-18T00:31:16.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.427+0000 7ff337fff700 1 --2- 172.21.15.59:0/969121346 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff320062c90 0x7ff320065150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff328004b80 tx=0x7ff328009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:16.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.431+0000 7ff3357fa700 1 -- 172.21.15.59:0/969121346 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff32c049f60 con 0x7ff3380fcbb0 2024-02-18T00:31:16.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.604+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff338106180 con 0x7ff3380fcbb0 2024-02-18T00:31:16.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.604+0000 7ff3357fa700 1 -- 172.21.15.59:0/969121346 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff32c0501c0 con 0x7ff3380fcbb0 2024-02-18T00:31:16.606 INFO:teuthology.orchestra.run.smithi059.stdout:133143986218 2024-02-18T00:31:16.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 10 librados: watch_flush enter 2024-02-18T00:31:16.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 10 librados: watch_flush exit 2024-02-18T00:31:16.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 20 client.24572.objecter shutdown clearing up homeless session... 2024-02-18T00:31:16.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 10 client.24572.objecter successfully canceled tick 2024-02-18T00:31:16.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff320062c90 msgr2=0x7ff320065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 --2- 172.21.15.59:0/969121346 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff320062c90 0x7ff320065150 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff328004b80 tx=0x7ff328009040).stop 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380fcbb0 msgr2=0x7ff33810e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380fcbb0 0x7ff33810e4f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7ff32c004920 tx=0x7ff32c009cb0).stop 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 shutdown_connections 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 --2- 172.21.15.59:0/969121346 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff320062c90 0x7ff320065150 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3380f7400 0x7ff33810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3380fd510 0x7ff3381081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 --2- 172.21.15.59:0/969121346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3380fcbb0 0x7ff33810e4f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 >> 172.21.15.59:0/969121346 conn(0x7ff3380f2820 msgr2=0x7ff3380f44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 shutdown_connections 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.606+0000 7ff33e57b700 1 -- 172.21.15.59:0/969121346 wait complete. 2024-02-18T00:31:16.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:16.607+0000 7ff33e57b700 1 librados: shutdown 2024-02-18T00:31:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:15.741379+0000 mgr.y (mgr.24454) 81 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:16.017566+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.f scrub starts 2024-02-18T00:31:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:16.020553+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.f scrub ok 2024-02-18T00:31:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:16.605959+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.59:0/969121346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T00:31:17.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:15.741379+0000 mgr.y (mgr.24454) 81 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:17.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:16.017566+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.f scrub starts 2024-02-18T00:31:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:16.020553+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.f scrub ok 2024-02-18T00:31:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:16.605959+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.59:0/969121346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T00:31:17.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:15.741379+0000 mgr.y (mgr.24454) 81 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:16.017566+0000 osd.5 (osd.5) 35 : cluster [DBG] 5.f scrub starts 2024-02-18T00:31:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:16.020553+0000 osd.5 (osd.5) 36 : cluster [DBG] 5.f scrub ok 2024-02-18T00:31:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:16.605959+0000 mon.c (mon.2) 22 : audit [DBG] from='client.? 172.21.15.59:0/969121346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T00:31:17.497 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986213 got 133143986218 for osd.4 2024-02-18T00:31:17.498 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-18T00:31:17.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:16.438475+0000 osd.3 (osd.3) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:16.438524+0000 osd.3 (osd.3) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:16.642758+0000 mon.a (mon.0) 744 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-18T00:31:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:16.438475+0000 osd.3 (osd.3) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:16.438524+0000 osd.3 (osd.3) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:16.642758+0000 mon.a (mon.0) 744 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-18T00:31:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:16.438475+0000 osd.3 (osd.3) 51 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:16.438524+0000 osd.3 (osd.3) 52 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:16.642758+0000 mon.a (mon.0) 744 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-18T00:31:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:17.741687+0000 mgr.y (mgr.24454) 82 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:18.027840+0000 osd.5 (osd.5) 37 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:31:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:18.031298+0000 osd.5 (osd.5) 38 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:31:18.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:17.741687+0000 mgr.y (mgr.24454) 82 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:18.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:18.027840+0000 osd.5 (osd.5) 37 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:31:18.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:18.031298+0000 osd.5 (osd.5) 38 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:31:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:17.741687+0000 mgr.y (mgr.24454) 82 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:18.027840+0000 osd.5 (osd.5) 37 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:31:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:18.031298+0000 osd.5 (osd.5) 38 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 -- 172.21.15.59:0/2038549195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fcbb0 msgr2=0x7fa9580fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 --2- 172.21.15.59:0/2038549195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fcbb0 0x7fa9580fcf90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fa9540047b0 tx=0x7fa954009cb0).stop 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 -- 172.21.15.59:0/2038549195 shutdown_connections 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 --2- 172.21.15.59:0/2038549195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580f7400 0x7fa958103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 --2- 172.21.15.59:0/2038549195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fcbb0 0x7fa9580fcf90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 --2- 172.21.15.59:0/2038549195 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580fd510 0x7fa9580f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 -- 172.21.15.59:0/2038549195 >> 172.21.15.59:0/2038549195 conn(0x7fa9580f2820 msgr2=0x7fa9580f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:19.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 -- 172.21.15.59:0/2038549195 shutdown_connections 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.960+0000 7fa95fd09700 1 -- 172.21.15.59:0/2038549195 wait complete. 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 librados: starting msgr at 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 librados: starting objecter 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 Processor -- start 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 -- start start 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 librados: setting wanted keys 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 librados: calling monclient init 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580f7400 0x7fa958107130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580fcbb0 0x7fa958107620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fd510 0x7fa958107b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581167b0 con 0x7fa9580fd510 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9581168f0 con 0x7fa9580fcbb0 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95fd09700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa958116a60 con 0x7fa9580f7400 2024-02-18T00:31:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95daa5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580f7400 0x7fa958107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95e2a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fd510 0x7fa958107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95daa5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580f7400 0x7fa958107130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56926/0 (socket says 172.21.15.59:56926) 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95daa5700 1 -- 172.21.15.59:0/4218346775 learned_addr learned my addr 172.21.15.59:0/4218346775 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.961+0000 7fa95d2a4700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580fcbb0 0x7fa958107620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95daa5700 1 -- 172.21.15.59:0/4218346775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580fcbb0 msgr2=0x7fa958107620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95daa5700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580fcbb0 0x7fa958107620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95daa5700 1 -- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fd510 msgr2=0x7fa958107b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95daa5700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fd510 0x7fa958107b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95daa5700 1 -- 172.21.15.59:0/4218346775 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c005040 con 0x7fa9580f7400 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95e2a6700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fd510 0x7fa958107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95daa5700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580f7400 0x7fa958107130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa954009b10 tx=0x7fa954009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa94affd700 10 client.?.objecter ms_handle_connect 0x7fa9580f7400 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa94affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:19.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa94affd700 1 -- 172.21.15.59:0/4218346775 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954018070 con 0x7fa9580f7400 2024-02-18T00:31:19.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa94affd700 1 -- 172.21.15.59:0/4218346775 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa954013e70 con 0x7fa9580f7400 2024-02-18T00:31:19.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa954005040 con 0x7fa9580f7400 2024-02-18T00:31:19.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa94affd700 1 -- 172.21.15.59:0/4218346775 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95401ddb0 con 0x7fa9580f7400 2024-02-18T00:31:19.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95fd09700 10 client.24578.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:19.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa958116f50 con 0x7fa9580f7400 2024-02-18T00:31:19.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.962+0000 7fa95fd09700 1 librados: init done 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.963+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95805fa60 con 0x7fa9580f7400 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.963+0000 7fa94affd700 1 -- 172.21.15.59:0/4218346775 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa95401d420 con 0x7fa9580f7400 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa94affd700 1 --2- 172.21.15.59:0/4218346775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa944062c90 0x7fa944065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa94affd700 1 -- 172.21.15.59:0/4218346775 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fa95407de50 con 0x7fa9580f7400 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa94affd700 10 client.24578.objecter ms_dispatch 0x7fa958001550 osd_map(88..88 src has 1..88) v4 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa94affd700 3 client.24578.objecter handle_osd_map got epochs [88,88] > 0 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa94affd700 3 client.24578.objecter handle_osd_map decoding full epoch 88 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa95d2a4700 1 --2- 172.21.15.59:0/4218346775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa944062c90 0x7fa944065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:19.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa94affd700 20 client.24578.objecter dump_active .. 0 homeless 2024-02-18T00:31:19.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa94affd700 10 client.24578.objecter ms_handle_connect 0x7fa944062c90 2024-02-18T00:31:19.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.964+0000 7fa95d2a4700 1 --2- 172.21.15.59:0/4218346775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa944062c90 0x7fa944065150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa958061700 tx=0x7fa94c009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:19.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:19.968+0000 7fa94affd700 1 -- 172.21.15.59:0/4218346775 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa954051b60 con 0x7fa9580f7400 2024-02-18T00:31:20.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:18.064368+0000 osd.0 (osd.0) 37 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:31:20.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:18.065783+0000 osd.0 (osd.0) 38 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:31:20.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:19.638668+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-18T00:31:20.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:18.064368+0000 osd.0 (osd.0) 37 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:31:20.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:18.065783+0000 osd.0 (osd.0) 38 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:31:20.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:19.638668+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-18T00:31:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:18.064368+0000 osd.0 (osd.0) 37 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:31:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:18.065783+0000 osd.0 (osd.0) 38 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:31:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:19.638668+0000 mon.a (mon.0) 745 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-18T00:31:20.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.144+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa9581060e0 con 0x7fa9580f7400 2024-02-18T00:31:20.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.144+0000 7fa94affd700 1 -- 172.21.15.59:0/4218346775 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa954051750 con 0x7fa9580f7400 2024-02-18T00:31:20.147 INFO:teuthology.orchestra.run.smithi059.stdout:167503724582 2024-02-18T00:31:20.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 10 librados: watch_flush enter 2024-02-18T00:31:20.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 10 librados: watch_flush exit 2024-02-18T00:31:20.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 20 client.24578.objecter shutdown clearing up homeless session... 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 10 client.24578.objecter successfully canceled tick 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa944062c90 msgr2=0x7fa944065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 --2- 172.21.15.59:0/4218346775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa944062c90 0x7fa944065150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa958061700 tx=0x7fa94c009cb0).stop 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580f7400 msgr2=0x7fa958107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580f7400 0x7fa958107130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fa954009b10 tx=0x7fa954009cb0).stop 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 shutdown_connections 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9580fd510 0x7fa958107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa9580f7400 0x7fa958107130 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 --2- 172.21.15.59:0/4218346775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa944062c90 0x7fa944065150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.146+0000 7fa95fd09700 1 --2- 172.21.15.59:0/4218346775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9580fcbb0 0x7fa958107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.147+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 >> 172.21.15.59:0/4218346775 conn(0x7fa9580f2820 msgr2=0x7fa9580fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.147+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 shutdown_connections 2024-02-18T00:31:20.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.147+0000 7fa95fd09700 1 -- 172.21.15.59:0/4218346775 wait complete. 2024-02-18T00:31:20.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:20.147+0000 7fa95fd09700 1 librados: shutdown 2024-02-18T00:31:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:19.742163+0000 mgr.y (mgr.24454) 83 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:20.145928+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.59:0/4218346775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T00:31:21.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:19.742163+0000 mgr.y (mgr.24454) 83 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:21.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:20.145928+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.59:0/4218346775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T00:31:21.004 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:31:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:31:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:19.742163+0000 mgr.y (mgr.24454) 83 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:20.145928+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.59:0/4218346775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T00:31:21.166 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724576 got 167503724582 for osd.5 2024-02-18T00:31:21.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-18T00:31:21.449 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:21.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:31:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:31:22.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:20.131507+0000 osd.0 (osd.0) 39 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:31:22.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:20.132874+0000 osd.0 (osd.0) 40 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:31:22.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:20.825435+0000 mgr.y (mgr.24454) 84 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:22.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:20.131507+0000 osd.0 (osd.0) 39 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:31:22.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:20.132874+0000 osd.0 (osd.0) 40 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:31:22.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:20.825435+0000 mgr.y (mgr.24454) 84 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:22.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:20.131507+0000 osd.0 (osd.0) 39 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:31:22.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:20.132874+0000 osd.0 (osd.0) 40 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:31:22.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:20.825435+0000 mgr.y (mgr.24454) 84 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:22.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:22] "GET /metrics HTTP/1.1" 200 172168 "" "Prometheus/2.33.4" 2024-02-18T00:31:23.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:21.742788+0000 mgr.y (mgr.24454) 85 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:23.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:21.742788+0000 mgr.y (mgr.24454) 85 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:23.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:21.742788+0000 mgr.y (mgr.24454) 85 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:23.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.636+0000 7f40f74b1700 1 -- 172.21.15.59:0/3333276184 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fe510 msgr2=0x7f40f00f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.636+0000 7f40f74b1700 1 --2- 172.21.15.59:0/3333276184 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fe510 0x7f40f00f7d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f40e00047b0 tx=0x7f40e0009cb0).stop 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.636+0000 7f40f74b1700 1 -- 172.21.15.59:0/3333276184 shutdown_connections 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.636+0000 7f40f74b1700 1 --2- 172.21.15.59:0/3333276184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40f00f8420 0x7f40f0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.636+0000 7f40f74b1700 1 --2- 172.21.15.59:0/3333276184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fdbb0 0x7f40f00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.636+0000 7f40f74b1700 1 --2- 172.21.15.59:0/3333276184 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fe510 0x7f40f00f7d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.636+0000 7f40f74b1700 1 -- 172.21.15.59:0/3333276184 >> 172.21.15.59:0/3333276184 conn(0x7f40f00f3800 msgr2=0x7f40f00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 -- 172.21.15.59:0/3333276184 shutdown_connections 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 -- 172.21.15.59:0/3333276184 wait complete. 2024-02-18T00:31:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 librados: starting msgr at 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 librados: starting objecter 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 Processor -- start 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 -- start start 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 librados: setting wanted keys 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 librados: calling monclient init 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40f00f8420 0x7f40f010e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.637+0000 7f40f74b1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fdbb0 0x7f40f010e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fe510 0x7f40f0108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0108730 con 0x7f40f00fe510 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01088a0 con 0x7f40f00f8420 2024-02-18T00:31:23.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0108a60 con 0x7f40f00fdbb0 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fe510 0x7f40f0108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fe510 0x7f40f0108240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56088/0 (socket says 172.21.15.59:56088) 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 -- 172.21.15.59:0/1759475353 learned_addr learned my addr 172.21.15.59:0/1759475353 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f524d700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40f00f8420 0x7f40f010e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f4a4c700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fdbb0 0x7f40f010e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 -- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fdbb0 msgr2=0x7f40f010e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fdbb0 0x7f40f010e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 -- 172.21.15.59:0/1759475353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40f00f8420 msgr2=0x7f40f010e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40f00f8420 0x7f40f010e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 -- 172.21.15.59:0/1759475353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec005040 con 0x7f40f00fe510 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f5a4e700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fe510 0x7f40f0108240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f40e800d970 tx=0x7f40e8004e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40e67fc700 10 client.?.objecter ms_handle_connect 0x7f40f00fe510 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40e67fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40e67fc700 1 -- 172.21.15.59:0/1759475353 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e800de60 con 0x7f40f00fe510 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40e67fc700 1 -- 172.21.15.59:0/1759475353 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e800f040 con 0x7f40f00fe510 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e0005040 con 0x7f40f00fe510 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40e67fc700 1 -- 172.21.15.59:0/1759475353 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e80036c0 con 0x7f40f00fe510 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 10 client.14679.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f01090a0 con 0x7f40f00fe510 2024-02-18T00:31:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.638+0000 7f40f74b1700 1 librados: init done 2024-02-18T00:31:23.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.639+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f005ee80 con 0x7f40f00fe510 2024-02-18T00:31:23.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40e67fc700 1 -- 172.21.15.59:0/1759475353 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f40e8003ab0 con 0x7f40f00fe510 2024-02-18T00:31:23.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40e67fc700 1 --2- 172.21.15.59:0/1759475353 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40dc062c90 0x7f40dc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:23.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40e67fc700 1 -- 172.21.15.59:0/1759475353 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f40e807c5c0 con 0x7f40f00fe510 2024-02-18T00:31:23.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40e67fc700 10 client.14679.objecter ms_dispatch 0x7f40f0001550 osd_map(88..88 src has 1..88) v4 2024-02-18T00:31:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40e67fc700 3 client.14679.objecter handle_osd_map got epochs [88,88] > 0 2024-02-18T00:31:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40e67fc700 3 client.14679.objecter handle_osd_map decoding full epoch 88 2024-02-18T00:31:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40f524d700 1 --2- 172.21.15.59:0/1759475353 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40dc062c90 0x7f40dc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.640+0000 7f40e67fc700 20 client.14679.objecter dump_active .. 0 homeless 2024-02-18T00:31:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.641+0000 7f40e67fc700 10 client.14679.objecter ms_handle_connect 0x7f40dc062c90 2024-02-18T00:31:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.641+0000 7f40f524d700 1 --2- 172.21.15.59:0/1759475353 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40dc062c90 0x7f40dc065150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f40ec004870 tx=0x7f40ec00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.644+0000 7f40e67fc700 1 -- 172.21.15.59:0/1759475353 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f40e8050230 con 0x7f40f00fe510 2024-02-18T00:31:23.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:23.015363+0000 osd.5 (osd.5) 39 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:31:23.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:23.018454+0000 osd.5 (osd.5) 40 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:31:23.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.818+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f40f0109800 con 0x7f40f00fe510 2024-02-18T00:31:23.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.819+0000 7f40e67fc700 1 -- 172.21.15.59:0/1759475353 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f40e804fe20 con 0x7f40f00fe510 2024-02-18T00:31:23.821 INFO:teuthology.orchestra.run.smithi059.stdout:201863462947 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 10 librados: watch_flush enter 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 10 librados: watch_flush exit 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 20 client.14679.objecter shutdown clearing up homeless session... 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 10 client.14679.objecter successfully canceled tick 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40dc062c90 msgr2=0x7f40dc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 --2- 172.21.15.59:0/1759475353 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40dc062c90 0x7f40dc065150 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f40ec004870 tx=0x7f40ec00b040).stop 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fe510 msgr2=0x7f40f0108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fe510 0x7f40f0108240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f40e800d970 tx=0x7f40e8004e60).stop 2024-02-18T00:31:23.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 shutdown_connections 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 --2- 172.21.15.59:0/1759475353 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40dc062c90 0x7f40dc065150 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40f00f8420 0x7f40f010e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f00fe510 0x7f40f0108240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 --2- 172.21.15.59:0/1759475353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40f00fdbb0 0x7f40f010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 >> 172.21.15.59:0/1759475353 conn(0x7f40f00f3800 msgr2=0x7f40f00f52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 shutdown_connections 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 -- 172.21.15.59:0/1759475353 wait complete. 2024-02-18T00:31:23.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:23.821+0000 7f40f74b1700 1 librados: shutdown 2024-02-18T00:31:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:23.015363+0000 osd.5 (osd.5) 39 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:31:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:23.018454+0000 osd.5 (osd.5) 40 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:31:24.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:23.015363+0000 osd.5 (osd.5) 39 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:31:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:23.018454+0000 osd.5 (osd.5) 40 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:31:24.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462941 got 201863462947 for osd.6 2024-02-18T00:31:24.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-18T00:31:24.970 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:23.239099+0000 osd.0 (osd.0) 41 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:31:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:23.240093+0000 osd.0 (osd.0) 42 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:31:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:23.743103+0000 mgr.y (mgr.24454) 86 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:23.820289+0000 mon.a (mon.0) 746 : audit [DBG] from='client.? 172.21.15.59:0/1759475353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T00:31:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:23.984854+0000 osd.5 (osd.5) 41 : cluster [DBG] 6.d deep-scrub starts 2024-02-18T00:31:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:23.987785+0000 osd.5 (osd.5) 42 : cluster [DBG] 6.d deep-scrub ok 2024-02-18T00:31:25.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:23.239099+0000 osd.0 (osd.0) 41 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:31:25.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:23.240093+0000 osd.0 (osd.0) 42 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:31:25.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:23.743103+0000 mgr.y (mgr.24454) 86 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:25.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:23.820289+0000 mon.a (mon.0) 746 : audit [DBG] from='client.? 172.21.15.59:0/1759475353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T00:31:25.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:23.984854+0000 osd.5 (osd.5) 41 : cluster [DBG] 6.d deep-scrub starts 2024-02-18T00:31:25.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:23.987785+0000 osd.5 (osd.5) 42 : cluster [DBG] 6.d deep-scrub ok 2024-02-18T00:31:25.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:23.239099+0000 osd.0 (osd.0) 41 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:31:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:23.240093+0000 osd.0 (osd.0) 42 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:31:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:23.743103+0000 mgr.y (mgr.24454) 86 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:23.820289+0000 mon.a (mon.0) 746 : audit [DBG] from='client.? 172.21.15.59:0/1759475353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T00:31:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:23.984854+0000 osd.5 (osd.5) 41 : cluster [DBG] 6.d deep-scrub starts 2024-02-18T00:31:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:23.987785+0000 osd.5 (osd.5) 42 : cluster [DBG] 6.d deep-scrub ok 2024-02-18T00:31:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.878+0000 7f8e5cafe700 1 -- 172.21.15.59:0/1199261042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fe510 msgr2=0x7f8e580f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.878+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/1199261042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fe510 0x7f8e580f7d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8e400047b0 tx=0x7f8e40009cb0).stop 2024-02-18T00:31:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 -- 172.21.15.59:0/1199261042 shutdown_connections 2024-02-18T00:31:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/1199261042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e580fdbb0 0x7f8e580fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/1199261042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580f8420 0x7f8e58104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/1199261042 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fe510 0x7f8e580f7d40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:26.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 -- 172.21.15.59:0/1199261042 >> 172.21.15.59:0/1199261042 conn(0x7f8e580f3800 msgr2=0x7f8e580f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 -- 172.21.15.59:0/1199261042 shutdown_connections 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 -- 172.21.15.59:0/1199261042 wait complete. 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 librados: starting msgr at 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 librados: starting objecter 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 Processor -- start 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 -- start start 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 librados: setting wanted keys 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.879+0000 7f8e5cafe700 1 librados: calling monclient init 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e580f8420 0x7f8e58110230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fdbb0 0x7f8e58110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580fe510 0x7f8e5810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5810a8c0 con 0x7f8e580fe510 2024-02-18T00:31:26.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5810aa30 con 0x7f8e580f8420 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5810abf0 con 0x7f8e580fdbb0 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fdbb0 0x7f8e58110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e56d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580fe510 0x7f8e5810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fdbb0 0x7f8e58110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56956/0 (socket says 172.21.15.59:56956) 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 -- 172.21.15.59:0/602918831 learned_addr learned my addr 172.21.15.59:0/602918831 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e56d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580fe510 0x7f8e5810a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56110/0 (socket says 172.21.15.59:56110) 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5659c700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e580f8420 0x7f8e58110230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 -- 172.21.15.59:0/602918831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e580f8420 msgr2=0x7f8e58110230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e580f8420 0x7f8e58110230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 -- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580fe510 msgr2=0x7f8e5810a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580fe510 0x7f8e5810a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 -- 172.21.15.59:0/602918831 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e500097c0 con 0x7f8e580fdbb0 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e56d9d700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580fe510 0x7f8e5810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:26.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e55d9b700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fdbb0 0x7f8e58110720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e400044e0 tx=0x7f8e40007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e4f7fe700 10 client.?.objecter ms_handle_connect 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e4f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e4f7fe700 1 -- 172.21.15.59:0/602918831 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40018070 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e4f7fe700 1 -- 172.21.15.59:0/602918831 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e400093b0 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40005040 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e4f7fe700 1 -- 172.21.15.59:0/602918831 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4001dd90 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 10 client.24590.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5810b110 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.880+0000 7f8e5cafe700 1 librados: init done 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.881+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5805ee80 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e4f7fe700 1 -- 172.21.15.59:0/602918831 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8e4001d420 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e4f7fe700 1 --2- 172.21.15.59:0/602918831 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8e44062c90 0x7f8e44065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e4f7fe700 1 -- 172.21.15.59:0/602918831 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f8e4007df00 con 0x7f8e580fdbb0 2024-02-18T00:31:26.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e4f7fe700 10 client.24590.objecter ms_dispatch 0x7f8e58001550 osd_map(88..88 src has 1..88) v4 2024-02-18T00:31:26.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e4f7fe700 3 client.24590.objecter handle_osd_map got epochs [88,88] > 0 2024-02-18T00:31:26.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e4f7fe700 3 client.24590.objecter handle_osd_map decoding full epoch 88 2024-02-18T00:31:26.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e5659c700 1 --2- 172.21.15.59:0/602918831 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8e44062c90 0x7f8e44065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:26.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.882+0000 7f8e4f7fe700 20 client.24590.objecter dump_active .. 0 homeless 2024-02-18T00:31:26.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.883+0000 7f8e4f7fe700 10 client.24590.objecter ms_handle_connect 0x7f8e44062c90 2024-02-18T00:31:26.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.883+0000 7f8e5659c700 1 --2- 172.21.15.59:0/602918831 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8e44062c90 0x7f8e44065150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8e48004dc0 tx=0x7f8e4800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:26.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:26.886+0000 7f8e4f7fe700 1 -- 172.21.15.59:0/602918831 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8e40051b10 con 0x7f8e580fdbb0 2024-02-18T00:31:27.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:25.296010+0000 osd.6 (osd.6) 29 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:31:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:25.298057+0000 osd.6 (osd.6) 30 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:31:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:25.743620+0000 mgr.y (mgr.24454) 87 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:31:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:25.296010+0000 osd.6 (osd.6) 29 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:31:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:25.298057+0000 osd.6 (osd.6) 30 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:31:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:25.743620+0000 mgr.y (mgr.24454) 87 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:31:27.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.058+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8e5810b980 con 0x7f8e580fdbb0 2024-02-18T00:31:27.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.058+0000 7f8e4f7fe700 1 -- 172.21.15.59:0/602918831 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e40051700 con 0x7f8e580fdbb0 2024-02-18T00:31:27.060 INFO:teuthology.orchestra.run.smithi059.stdout:227633266718 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 10 librados: watch_flush enter 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 10 librados: watch_flush exit 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 20 client.24590.objecter shutdown clearing up homeless session... 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 10 client.24590.objecter successfully canceled tick 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8e44062c90 msgr2=0x7f8e44065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/602918831 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8e44062c90 0x7f8e44065150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f8e48004dc0 tx=0x7f8e4800b040).stop 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fdbb0 msgr2=0x7f8e58110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fdbb0 0x7f8e58110720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f8e400044e0 tx=0x7f8e40007f90).stop 2024-02-18T00:31:27.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 shutdown_connections 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/602918831 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8e44062c90 0x7f8e44065150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e580f8420 0x7f8e58110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e580fe510 0x7f8e5810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 --2- 172.21.15.59:0/602918831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8e580fdbb0 0x7f8e58110720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 >> 172.21.15.59:0/602918831 conn(0x7f8e580f3800 msgr2=0x7f8e58103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 shutdown_connections 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 -- 172.21.15.59:0/602918831 wait complete. 2024-02-18T00:31:27.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:27.060+0000 7f8e5cafe700 1 librados: shutdown 2024-02-18T00:31:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:25.296010+0000 osd.6 (osd.6) 29 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:31:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:25.298057+0000 osd.6 (osd.6) 30 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:31:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:25.743620+0000 mgr.y (mgr.24454) 87 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:31:27.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266712 got 227633266718 for osd.7 2024-02-18T00:31:27.915 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-18T00:31:27.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:31:28.197 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:27.059764+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.59:0/602918831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T00:31:28.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:27.059764+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.59:0/602918831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T00:31:28.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:27.059764+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.59:0/602918831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T00:31:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:27.744146+0000 mgr.y (mgr.24454) 88 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:29.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:27.744146+0000 mgr.y (mgr.24454) 88 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:27.744146+0000 mgr.y (mgr.24454) 88 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:30.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 -- 172.21.15.59:0/2571919832 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00fdbb0 msgr2=0x7f52a00fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:30.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 --2- 172.21.15.59:0/2571919832 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00fdbb0 0x7f52a00fdf90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f529c004870 tx=0x7f529c009cb0).stop 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 -- 172.21.15.59:0/2571919832 shutdown_connections 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 --2- 172.21.15.59:0/2571919832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00f8420 0x7f52a0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 --2- 172.21.15.59:0/2571919832 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00fdbb0 0x7f52a00fdf90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 --2- 172.21.15.59:0/2571919832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52a00fe510 0x7f52a00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 -- 172.21.15.59:0/2571919832 >> 172.21.15.59:0/2571919832 conn(0x7f52a00f3800 msgr2=0x7f52a00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 -- 172.21.15.59:0/2571919832 shutdown_connections 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.537+0000 7f52a74a7700 1 -- 172.21.15.59:0/2571919832 wait complete. 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 librados: starting msgr at 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 librados: starting objecter 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 Processor -- start 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 -- start start 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 librados: setting wanted keys 2024-02-18T00:31:30.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 librados: calling monclient init 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00f8420 0x7f52a010e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52a00fdbb0 0x7f52a010e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00fe510 0x7f52a0108270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a0108760 con 0x7f52a00fe510 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a01088d0 con 0x7f52a00fdbb0 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a74a7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a0108a90 con 0x7f52a00f8420 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5243700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00f8420 0x7f52a010e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5a44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00fe510 0x7f52a0108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:30.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5243700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00f8420 0x7f52a010e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59340/0 (socket says 172.21.15.59:59340) 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5243700 1 -- 172.21.15.59:0/1889293338 learned_addr learned my addr 172.21.15.59:0/1889293338 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5a44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00fe510 0x7f52a0108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36224/0 (socket says 172.21.15.59:36224) 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a4a42700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52a00fdbb0 0x7f52a010e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5a44700 1 -- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00f8420 msgr2=0x7f52a010e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5a44700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00f8420 0x7f52a010e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5a44700 1 -- 172.21.15.59:0/1889293338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52a00fdbb0 msgr2=0x7f52a010e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.538+0000 7f52a5a44700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52a00fdbb0 0x7f52a010e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52a5a44700 1 -- 172.21.15.59:0/1889293338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980097c0 con 0x7f52a00fe510 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52a5243700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00f8420 0x7f52a010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52a5a44700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00fe510 0x7f52a0108270 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5298001cb0 tx=0x7f5298004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52967fc700 10 client.?.objecter ms_handle_connect 0x7f52a00fe510 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52967fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:30.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52967fc700 1 -- 172.21.15.59:0/1889293338 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52980079b0 con 0x7f52a00fe510 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52967fc700 1 -- 172.21.15.59:0/1889293338 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52980090d0 con 0x7f52a00fe510 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c005040 con 0x7f52a00fe510 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52967fc700 1 -- 172.21.15.59:0/1889293338 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800fc20 con 0x7f52a00fe510 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52a74a7700 10 client.14691.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a0109080 con 0x7f52a00fe510 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.539+0000 7f52a74a7700 1 librados: init done 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.540+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a005ee80 con 0x7f52a00fe510 2024-02-18T00:31:30.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.540+0000 7f52967fc700 1 -- 172.21.15.59:0/1889293338 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f529800fd80 con 0x7f52a00fe510 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52967fc700 1 --2- 172.21.15.59:0/1889293338 >> v2:172.21.15.59:6800/3721891797 conn(0x7f528c062c90 0x7f528c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52967fc700 1 -- 172.21.15.59:0/1889293338 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f529807d2e0 con 0x7f52a00fe510 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52967fc700 10 client.14691.objecter ms_dispatch 0x7f52a0001550 osd_map(88..88 src has 1..88) v4 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52967fc700 3 client.14691.objecter handle_osd_map got epochs [88,88] > 0 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52967fc700 3 client.14691.objecter handle_osd_map decoding full epoch 88 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52a5243700 1 --2- 172.21.15.59:0/1889293338 >> v2:172.21.15.59:6800/3721891797 conn(0x7f528c062c90 0x7f528c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52967fc700 20 client.14691.objecter dump_active .. 0 homeless 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52a5243700 1 --2- 172.21.15.59:0/1889293338 >> v2:172.21.15.59:6800/3721891797 conn(0x7f528c062c90 0x7f528c065150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f529c0043d0 tx=0x7f529c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:30.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.541+0000 7f52967fc700 10 client.14691.objecter ms_handle_connect 0x7f528c062c90 2024-02-18T00:31:30.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.545+0000 7f52967fc700 1 -- 172.21.15.59:0/1889293338 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5298050f50 con 0x7f52a00fe510 2024-02-18T00:31:30.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.714+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52a0109860 con 0x7f528c062c90 2024-02-18T00:31:30.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.727+0000 7f52967fc700 1 -- 172.21.15.59:0/1889293338 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357206 (secure 0 0 0) 0x7f52a0109860 con 0x7f528c062c90 2024-02-18T00:31:30.735 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 10 librados: watch_flush enter 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 10 librados: watch_flush exit 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 20 client.14691.objecter shutdown clearing up homeless session... 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 10 client.14691.objecter successfully canceled tick 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 >> v2:172.21.15.59:6800/3721891797 conn(0x7f528c062c90 msgr2=0x7f528c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 --2- 172.21.15.59:0/1889293338 >> v2:172.21.15.59:6800/3721891797 conn(0x7f528c062c90 0x7f528c065150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f529c0043d0 tx=0x7f529c00b040).stop 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00fe510 msgr2=0x7f52a0108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00fe510 0x7f52a0108270 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5298001cb0 tx=0x7f5298004c50).stop 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 shutdown_connections 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52a00fe510 0x7f52a0108270 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52a00f8420 0x7f52a010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 --2- 172.21.15.59:0/1889293338 >> v2:172.21.15.59:6800/3721891797 conn(0x7f528c062c90 0x7f528c065150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 --2- 172.21.15.59:0/1889293338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52a00fdbb0 0x7f52a010e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:30.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 >> 172.21.15.59:0/1889293338 conn(0x7f52a00f3800 msgr2=0x7f52a00f53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:30.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 shutdown_connections 2024-02-18T00:31:30.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 -- 172.21.15.59:0/1889293338 wait complete. 2024-02-18T00:31:30.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:30.730+0000 7f52a74a7700 1 librados: shutdown 2024-02-18T00:31:30.740 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:31:31.133 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:29.227213+0000 osd.3 (osd.3) 53 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:31:31.133 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:29.228790+0000 osd.3 (osd.3) 54 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:31:31.133 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:29.744472+0000 mgr.y (mgr.24454) 89 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-02-18T00:31:31.134 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:29.227213+0000 osd.3 (osd.3) 53 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:31:31.134 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:29.228790+0000 osd.3 (osd.3) 54 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:31:31.134 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:29.744472+0000 mgr.y (mgr.24454) 89 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-02-18T00:31:31.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:31:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:31:31.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:31:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:31:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:29.227213+0000 osd.3 (osd.3) 53 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:31:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:29.228790+0000 osd.3 (osd.3) 54 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:31:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:29.744472+0000 mgr.y (mgr.24454) 89 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1013 B/s rd, 0 op/s 2024-02-18T00:31:31.628 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-02-18T00:31:29.744294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1226,"num_read_kb":946,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":674,"ondisk_log_size":674,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2383988,"kb_used_data":7220,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747577228,"statfs":{"total":767960285184,"available":765519081472,"internally_reserved":0,"allocated":7393280,"data_stored":3093754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.108555"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644500+0000","last_change":"2024-02-18T00:30:46.913439+0000","last_active":"2024-02-18T00:31:16.644500+0000","last_peered":"2024-02-18T00:31:16.644500+0000","last_clean":"2024-02-18T00:31:16.644500+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:31:16.644500+0000","last_undegraded":"2024-02-18T00:31:16.644500+0000","last_fullsized":"2024-02-18T00:31:16.644500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.913340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.913340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645115+0000","last_change":"2024-02-18T00:31:07.942520+0000","last_active":"2024-02-18T00:31:16.645115+0000","last_peered":"2024-02-18T00:31:16.645115+0000","last_clean":"2024-02-18T00:31:16.645115+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:31:16.645115+0000","last_undegraded":"2024-02-18T00:31:16.645115+0000","last_fullsized":"2024-02-18T00:31:16.645115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.942435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.942435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977255+0000","last_change":"2024-02-18T00:31:02.922852+0000","last_active":"2024-02-18T00:31:16.977255+0000","last_peered":"2024-02-18T00:31:16.977255+0000","last_clean":"2024-02-18T00:31:16.977255+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:31:16.977255+0000","last_undegraded":"2024-02-18T00:31:16.977255+0000","last_fullsized":"2024-02-18T00:31:16.977255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.922771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.922771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":128,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644821+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:31:16.644821+0000","last_peered":"2024-02-18T00:31:16.644821+0000","last_clean":"2024-02-18T00:31:16.644821+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:31:16.644821+0000","last_undegraded":"2024-02-18T00:31:16.644821+0000","last_fullsized":"2024-02-18T00:31:16.644821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645333+0000","last_change":"2024-02-18T00:30:29.061462+0000","last_active":"2024-02-18T00:31:16.645333+0000","last_peered":"2024-02-18T00:31:16.645333+0000","last_clean":"2024-02-18T00:31:16.645333+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:31:16.645333+0000","last_undegraded":"2024-02-18T00:31:16.645333+0000","last_fullsized":"2024-02-18T00:31:16.645333+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.061390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.061390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645060+0000","last_change":"2024-02-18T00:30:50.991652+0000","last_active":"2024-02-18T00:31:16.645060+0000","last_peered":"2024-02-18T00:31:16.645060+0000","last_clean":"2024-02-18T00:31:16.645060+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:31:16.645060+0000","last_undegraded":"2024-02-18T00:31:16.645060+0000","last_fullsized":"2024-02-18T00:31:16.645060+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.991592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.991592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645331+0000","last_change":"2024-02-18T00:30:48.890743+0000","last_active":"2024-02-18T00:31:16.645331+0000","last_peered":"2024-02-18T00:31:16.645331+0000","last_clean":"2024-02-18T00:31:16.645331+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:31:16.645331+0000","last_undegraded":"2024-02-18T00:31:16.645331+0000","last_fullsized":"2024-02-18T00:31:16.645331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.890677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645284+0000","last_change":"2024-02-18T00:31:03.941281+0000","last_active":"2024-02-18T00:31:16.645284+0000","last_peered":"2024-02-18T00:31:16.645284+0000","last_clean":"2024-02-18T00:31:16.645284+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:31:16.645284+0000","last_undegraded":"2024-02-18T00:31:16.645284+0000","last_fullsized":"2024-02-18T00:31:16.645284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:03.941195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:03.941195+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976631+0000","last_change":"2024-02-18T00:30:41.063319+0000","last_active":"2024-02-18T00:31:16.976631+0000","last_peered":"2024-02-18T00:31:16.976631+0000","last_clean":"2024-02-18T00:31:16.976631+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:31:16.976631+0000","last_undegraded":"2024-02-18T00:31:16.976631+0000","last_fullsized":"2024-02-18T00:31:16.976631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.063240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:41.063240+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.063240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976614+0000","last_change":"2024-02-18T00:30:25.635340+0000","last_active":"2024-02-18T00:31:16.976614+0000","last_peered":"2024-02-18T00:31:16.976614+0000","last_clean":"2024-02-18T00:31:16.976614+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:31:16.976614+0000","last_undegraded":"2024-02-18T00:31:16.976614+0000","last_fullsized":"2024-02-18T00:31:16.976614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:25.635259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:25.635259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977679+0000","last_change":"2024-02-18T00:30:22.539011+0000","last_active":"2024-02-18T00:31:16.977679+0000","last_peered":"2024-02-18T00:31:16.977679+0000","last_clean":"2024-02-18T00:31:16.977679+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:31:16.977679+0000","last_undegraded":"2024-02-18T00:31:16.977679+0000","last_fullsized":"2024-02-18T00:31:16.977679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:22.538946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:22.538946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645647+0000","last_change":"2024-02-18T00:30:46.066618+0000","last_active":"2024-02-18T00:31:16.645647+0000","last_peered":"2024-02-18T00:31:16.645647+0000","last_clean":"2024-02-18T00:31:16.645647+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:31:16.645647+0000","last_undegraded":"2024-02-18T00:31:16.645647+0000","last_fullsized":"2024-02-18T00:31:16.645647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.066555+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976147+0000","last_change":"2024-02-18T00:30:30.776677+0000","last_active":"2024-02-18T00:31:16.976147+0000","last_peered":"2024-02-18T00:31:16.976147+0000","last_clean":"2024-02-18T00:31:16.976147+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:31:16.976147+0000","last_undegraded":"2024-02-18T00:31:16.976147+0000","last_fullsized":"2024-02-18T00:31:16.976147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.776611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.776611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977353+0000","last_change":"2024-02-18T00:30:48.990813+0000","last_active":"2024-02-18T00:31:16.977353+0000","last_peered":"2024-02-18T00:31:16.977353+0000","last_clean":"2024-02-18T00:31:16.977353+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:31:16.977353+0000","last_undegraded":"2024-02-18T00:31:16.977353+0000","last_fullsized":"2024-02-18T00:31:16.977353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.990751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.990751+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977340+0000","last_change":"2024-02-18T00:30:26.654619+0000","last_active":"2024-02-18T00:31:16.977340+0000","last_peered":"2024-02-18T00:31:16.977340+0000","last_clean":"2024-02-18T00:31:16.977340+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:31:16.977340+0000","last_undegraded":"2024-02-18T00:31:16.977340+0000","last_fullsized":"2024-02-18T00:31:16.977340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.654541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.654541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'22","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645497+0000","last_change":"2024-02-18T00:30:27.058513+0000","last_active":"2024-02-18T00:31:16.645497+0000","last_peered":"2024-02-18T00:31:16.645497+0000","last_clean":"2024-02-18T00:31:16.645497+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:31:16.645497+0000","last_undegraded":"2024-02-18T00:31:16.645497+0000","last_fullsized":"2024-02-18T00:31:16.645497+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:30:27.058416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.058416+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645307+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:31:16.645307+0000","last_peered":"2024-02-18T00:31:16.645307+0000","last_clean":"2024-02-18T00:31:16.645307+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:31:16.645307+0000","last_undegraded":"2024-02-18T00:31:16.645307+0000","last_fullsized":"2024-02-18T00:31:16.645307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645666+0000","last_change":"2024-02-18T00:30:33.047434+0000","last_active":"2024-02-18T00:31:16.645666+0000","last_peered":"2024-02-18T00:31:16.645666+0000","last_clean":"2024-02-18T00:31:16.645666+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:31:16.645666+0000","last_undegraded":"2024-02-18T00:31:16.645666+0000","last_fullsized":"2024-02-18T00:31:16.645666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.047327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.047327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644603+0000","last_change":"2024-02-18T00:30:25.803798+0000","last_active":"2024-02-18T00:31:16.644603+0000","last_peered":"2024-02-18T00:31:16.644603+0000","last_clean":"2024-02-18T00:31:16.644603+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:31:16.644603+0000","last_undegraded":"2024-02-18T00:31:16.644603+0000","last_fullsized":"2024-02-18T00:31:16.644603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:30:25.803723+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645057+0000","last_change":"2024-02-18T00:30:50.885765+0000","last_active":"2024-02-18T00:31:16.645057+0000","last_peered":"2024-02-18T00:31:16.645057+0000","last_clean":"2024-02-18T00:31:16.645057+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:31:16.645057+0000","last_undegraded":"2024-02-18T00:31:16.645057+0000","last_fullsized":"2024-02-18T00:31:16.645057+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:30:50.885651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.885651+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646046+0000","last_change":"2024-02-18T00:30:36.043298+0000","last_active":"2024-02-18T00:31:16.646046+0000","last_peered":"2024-02-18T00:31:16.646046+0000","last_clean":"2024-02-18T00:31:16.646046+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:31:16.646046+0000","last_undegraded":"2024-02-18T00:31:16.646046+0000","last_fullsized":"2024-02-18T00:31:16.646046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:36.043211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:36.043211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646172+0000","last_change":"2024-02-18T00:31:02.145322+0000","last_active":"2024-02-18T00:31:16.646172+0000","last_peered":"2024-02-18T00:31:16.646172+0000","last_clean":"2024-02-18T00:31:16.646172+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:31:16.646172+0000","last_undegraded":"2024-02-18T00:31:16.646172+0000","last_fullsized":"2024-02-18T00:31:16.646172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.145253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978470+0000","last_change":"2024-02-18T00:30:48.450965+0000","last_active":"2024-02-18T00:31:16.978470+0000","last_peered":"2024-02-18T00:31:16.978470+0000","last_clean":"2024-02-18T00:31:16.978470+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:31:16.978470+0000","last_undegraded":"2024-02-18T00:31:16.978470+0000","last_fullsized":"2024-02-18T00:31:16.978470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.450877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.450877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976836+0000","last_change":"2024-02-18T00:30:23.427028+0000","last_active":"2024-02-18T00:31:16.976836+0000","last_peered":"2024-02-18T00:31:16.976836+0000","last_clean":"2024-02-18T00:31:16.976836+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:31:16.976836+0000","last_undegraded":"2024-02-18T00:31:16.976836+0000","last_fullsized":"2024-02-18T00:31:16.976836+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:30:23.426950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:23.426950+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645705+0000","last_change":"2024-02-18T00:30:30.321398+0000","last_active":"2024-02-18T00:31:16.645705+0000","last_peered":"2024-02-18T00:31:16.645705+0000","last_clean":"2024-02-18T00:31:16.645705+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:31:16.645705+0000","last_undegraded":"2024-02-18T00:31:16.645705+0000","last_fullsized":"2024-02-18T00:31:16.645705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.321330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.240187+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:31:23.240187+0000","last_peered":"2024-02-18T00:31:23.240187+0000","last_clean":"2024-02-18T00:31:23.240187+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:31:23.240187+0000","last_undegraded":"2024-02-18T00:31:23.240187+0000","last_fullsized":"2024-02-18T00:31:23.240187+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644497+0000","last_change":"2024-02-18T00:30:19.726673+0000","last_active":"2024-02-18T00:31:16.644497+0000","last_peered":"2024-02-18T00:31:16.644497+0000","last_clean":"2024-02-18T00:31:16.644497+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:31:16.644497+0000","last_undegraded":"2024-02-18T00:31:16.644497+0000","last_fullsized":"2024-02-18T00:31:16.644497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:19.726599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.726599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977675+0000","last_change":"2024-02-18T00:30:41.540091+0000","last_active":"2024-02-18T00:31:16.977675+0000","last_peered":"2024-02-18T00:31:16.977675+0000","last_clean":"2024-02-18T00:31:16.977675+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:31:16.977675+0000","last_undegraded":"2024-02-18T00:31:16.977675+0000","last_fullsized":"2024-02-18T00:31:16.977675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:41.540007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.540007+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977559+0000","last_change":"2024-02-18T00:30:38.560205+0000","last_active":"2024-02-18T00:31:16.977559+0000","last_peered":"2024-02-18T00:31:16.977559+0000","last_clean":"2024-02-18T00:31:16.977559+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:31:16.977559+0000","last_undegraded":"2024-02-18T00:31:16.977559+0000","last_fullsized":"2024-02-18T00:31:16.977559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:30:38.560085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.560085+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.518463+0000","last_change":"2024-02-18T00:30:21.465322+0000","last_active":"2024-02-18T00:31:20.518463+0000","last_peered":"2024-02-18T00:31:20.518463+0000","last_clean":"2024-02-18T00:31:20.518463+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:31:20.518463+0000","last_undegraded":"2024-02-18T00:31:20.518463+0000","last_fullsized":"2024-02-18T00:31:20.518463+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:21.465249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.465249+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976449+0000","last_change":"2024-02-18T00:31:06.040501+0000","last_active":"2024-02-18T00:31:16.976449+0000","last_peered":"2024-02-18T00:31:16.976449+0000","last_clean":"2024-02-18T00:31:16.976449+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:31:16.976449+0000","last_undegraded":"2024-02-18T00:31:16.976449+0000","last_fullsized":"2024-02-18T00:31:16.976449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.040425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.040425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976957+0000","last_change":"2024-02-18T00:30:38.895738+0000","last_active":"2024-02-18T00:31:16.976957+0000","last_peered":"2024-02-18T00:31:16.976957+0000","last_clean":"2024-02-18T00:31:16.976957+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:31:16.976957+0000","last_undegraded":"2024-02-18T00:31:16.976957+0000","last_fullsized":"2024-02-18T00:31:16.976957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:30:38.895655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.895655+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644795+0000","last_change":"2024-02-18T00:30:59.122281+0000","last_active":"2024-02-18T00:31:16.644795+0000","last_peered":"2024-02-18T00:31:16.644795+0000","last_clean":"2024-02-18T00:31:16.644795+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:31:16.644795+0000","last_undegraded":"2024-02-18T00:31:16.644795+0000","last_fullsized":"2024-02-18T00:31:16.644795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.122216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.122216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977001+0000","last_change":"2024-02-18T00:30:31.634376+0000","last_active":"2024-02-18T00:31:16.977001+0000","last_peered":"2024-02-18T00:31:16.977001+0000","last_clean":"2024-02-18T00:31:16.977001+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:31:16.977001+0000","last_undegraded":"2024-02-18T00:31:16.977001+0000","last_fullsized":"2024-02-18T00:31:16.977001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.634311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.634311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977942+0000","last_change":"2024-02-18T00:30:44.493983+0000","last_active":"2024-02-18T00:31:16.977942+0000","last_peered":"2024-02-18T00:31:16.977942+0000","last_clean":"2024-02-18T00:31:16.977942+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:31:16.977942+0000","last_undegraded":"2024-02-18T00:31:16.977942+0000","last_fullsized":"2024-02-18T00:31:16.977942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.493871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.493871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644852+0000","last_change":"2024-02-18T00:30:43.819520+0000","last_active":"2024-02-18T00:31:16.644852+0000","last_peered":"2024-02-18T00:31:16.644852+0000","last_clean":"2024-02-18T00:31:16.644852+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:31:16.644852+0000","last_undegraded":"2024-02-18T00:31:16.644852+0000","last_fullsized":"2024-02-18T00:31:16.644852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:43.819434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.819434+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976441+0000","last_change":"2024-02-18T00:30:35.533752+0000","last_active":"2024-02-18T00:31:16.976441+0000","last_peered":"2024-02-18T00:31:16.976441+0000","last_clean":"2024-02-18T00:31:16.976441+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:31:16.976441+0000","last_undegraded":"2024-02-18T00:31:16.976441+0000","last_fullsized":"2024-02-18T00:31:16.976441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.533630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.533630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978149+0000","last_change":"2024-02-18T00:30:47.440072+0000","last_active":"2024-02-18T00:31:16.978149+0000","last_peered":"2024-02-18T00:31:16.978149+0000","last_clean":"2024-02-18T00:31:16.978149+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:31:16.978149+0000","last_undegraded":"2024-02-18T00:31:16.978149+0000","last_fullsized":"2024-02-18T00:31:16.978149+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.440005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.440005+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978064+0000","last_change":"2024-02-18T00:30:18.101882+0000","last_active":"2024-02-18T00:31:16.978064+0000","last_peered":"2024-02-18T00:31:16.978064+0000","last_clean":"2024-02-18T00:31:16.978064+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:31:16.978064+0000","last_undegraded":"2024-02-18T00:31:16.978064+0000","last_fullsized":"2024-02-18T00:31:16.978064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:18.101776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:18.101776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644451+0000","last_change":"2024-02-18T00:31:00.083411+0000","last_active":"2024-02-18T00:31:16.644451+0000","last_peered":"2024-02-18T00:31:16.644451+0000","last_clean":"2024-02-18T00:31:16.644451+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:31:16.644451+0000","last_undegraded":"2024-02-18T00:31:16.644451+0000","last_fullsized":"2024-02-18T00:31:16.644451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:31:00.083332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.083332+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644359+0000","last_change":"2024-02-18T00:31:06.048967+0000","last_active":"2024-02-18T00:31:16.644359+0000","last_peered":"2024-02-18T00:31:16.644359+0000","last_clean":"2024-02-18T00:31:16.644359+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:31:16.644359+0000","last_undegraded":"2024-02-18T00:31:16.644359+0000","last_fullsized":"2024-02-18T00:31:16.644359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.048889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.048889+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645358+0000","last_change":"2024-02-18T00:30:45.987723+0000","last_active":"2024-02-18T00:31:16.645358+0000","last_peered":"2024-02-18T00:31:16.645358+0000","last_clean":"2024-02-18T00:31:16.645358+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:31:16.645358+0000","last_undegraded":"2024-02-18T00:31:16.645358+0000","last_fullsized":"2024-02-18T00:31:16.645358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.987660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.987660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645853+0000","last_change":"2024-02-18T00:30:54.008260+0000","last_active":"2024-02-18T00:31:16.645853+0000","last_peered":"2024-02-18T00:31:16.645853+0000","last_clean":"2024-02-18T00:31:16.645853+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:31:16.645853+0000","last_undegraded":"2024-02-18T00:31:16.645853+0000","last_fullsized":"2024-02-18T00:31:16.645853+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.008196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.008196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645424+0000","last_change":"2024-02-18T00:30:17.985487+0000","last_active":"2024-02-18T00:31:16.645424+0000","last_peered":"2024-02-18T00:31:16.645424+0000","last_clean":"2024-02-18T00:31:16.645424+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:31:16.645424+0000","last_undegraded":"2024-02-18T00:31:16.645424+0000","last_fullsized":"2024-02-18T00:31:16.645424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:30:17.985420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:17.985420+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975810+0000","last_change":"2024-02-18T00:31:13.960170+0000","last_active":"2024-02-18T00:31:16.975810+0000","last_peered":"2024-02-18T00:31:16.975810+0000","last_clean":"2024-02-18T00:31:16.975810+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:31:16.975810+0000","last_undegraded":"2024-02-18T00:31:16.975810+0000","last_fullsized":"2024-02-18T00:31:16.975810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:13.960089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:13.960089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644853+0000","last_change":"2024-02-18T00:30:31.810682+0000","last_active":"2024-02-18T00:31:16.644853+0000","last_peered":"2024-02-18T00:31:16.644853+0000","last_clean":"2024-02-18T00:31:16.644853+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:31:16.644853+0000","last_undegraded":"2024-02-18T00:31:16.644853+0000","last_fullsized":"2024-02-18T00:31:16.644853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.810604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.810604+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976874+0000","last_change":"2024-02-18T00:30:19.478866+0000","last_active":"2024-02-18T00:31:16.976874+0000","last_peered":"2024-02-18T00:31:16.976874+0000","last_clean":"2024-02-18T00:31:16.976874+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:31:16.976874+0000","last_undegraded":"2024-02-18T00:31:16.976874+0000","last_fullsized":"2024-02-18T00:31:16.976874+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:19.478758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.478758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645332+0000","last_change":"2024-02-18T00:30:51.994431+0000","last_active":"2024-02-18T00:31:16.645332+0000","last_peered":"2024-02-18T00:31:16.645332+0000","last_clean":"2024-02-18T00:31:16.645332+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:31:16.645332+0000","last_undegraded":"2024-02-18T00:31:16.645332+0000","last_fullsized":"2024-02-18T00:31:16.645332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:30:51.994334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.994334+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646060+0000","last_change":"2024-02-18T00:31:12.044114+0000","last_active":"2024-02-18T00:31:16.646060+0000","last_peered":"2024-02-18T00:31:16.646060+0000","last_clean":"2024-02-18T00:31:16.646060+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:31:16.646060+0000","last_undegraded":"2024-02-18T00:31:16.646060+0000","last_fullsized":"2024-02-18T00:31:16.646060+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:12.043994+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.132282+0000","last_change":"2024-02-18T00:30:35.823674+0000","last_active":"2024-02-18T00:31:20.132282+0000","last_peered":"2024-02-18T00:31:20.132282+0000","last_clean":"2024-02-18T00:31:20.132282+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:31:20.132282+0000","last_undegraded":"2024-02-18T00:31:20.132282+0000","last_fullsized":"2024-02-18T00:31:20.132282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:35.823589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.823589+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976606+0000","last_change":"2024-02-18T00:30:21.675011+0000","last_active":"2024-02-18T00:31:16.976606+0000","last_peered":"2024-02-18T00:31:16.976606+0000","last_clean":"2024-02-18T00:31:16.976606+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:31:16.976606+0000","last_undegraded":"2024-02-18T00:31:16.976606+0000","last_fullsized":"2024-02-18T00:31:16.976606+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.674903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:18.031433+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:31:18.031433+0000","last_peered":"2024-02-18T00:31:18.031433+0000","last_clean":"2024-02-18T00:31:18.031433+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:31:18.031433+0000","last_undegraded":"2024-02-18T00:31:18.031433+0000","last_fullsized":"2024-02-18T00:31:18.031433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976579+0000","last_change":"2024-02-18T00:30:29.772460+0000","last_active":"2024-02-18T00:31:16.976579+0000","last_peered":"2024-02-18T00:31:16.976579+0000","last_clean":"2024-02-18T00:31:16.976579+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:31:16.976579+0000","last_undegraded":"2024-02-18T00:31:16.976579+0000","last_fullsized":"2024-02-18T00:31:16.976579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.772394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.772394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.878394+0000","last_change":"2024-02-18T00:31:01.012802+0000","last_active":"2024-02-18T00:31:20.878394+0000","last_peered":"2024-02-18T00:31:20.878394+0000","last_clean":"2024-02-18T00:31:20.878394+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:31:20.878394+0000","last_undegraded":"2024-02-18T00:31:20.878394+0000","last_fullsized":"2024-02-18T00:31:20.878394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:01.012688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.012688+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977678+0000","last_change":"2024-02-18T00:31:05.071051+0000","last_active":"2024-02-18T00:31:16.977678+0000","last_peered":"2024-02-18T00:31:16.977678+0000","last_clean":"2024-02-18T00:31:16.977678+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:31:16.977678+0000","last_undegraded":"2024-02-18T00:31:16.977678+0000","last_fullsized":"2024-02-18T00:31:16.977678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:05.070930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.070930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644791+0000","last_change":"2024-02-18T00:30:54.987124+0000","last_active":"2024-02-18T00:31:16.644791+0000","last_peered":"2024-02-18T00:31:16.644791+0000","last_clean":"2024-02-18T00:31:16.644791+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:31:16.644791+0000","last_undegraded":"2024-02-18T00:31:16.644791+0000","last_fullsized":"2024-02-18T00:31:16.644791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:30:54.987049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.987049+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977242+0000","last_change":"2024-02-18T00:30:33.564597+0000","last_active":"2024-02-18T00:31:16.977242+0000","last_peered":"2024-02-18T00:31:16.977242+0000","last_clean":"2024-02-18T00:31:16.977242+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:31:16.977242+0000","last_undegraded":"2024-02-18T00:31:16.977242+0000","last_fullsized":"2024-02-18T00:31:16.977242+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.564517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.564517+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.018547+0000","last_change":"2024-02-18T00:31:23.018547+0000","last_active":"2024-02-18T00:31:23.018547+0000","last_peered":"2024-02-18T00:31:23.018547+0000","last_clean":"2024-02-18T00:31:23.018547+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:31:23.018547+0000","last_undegraded":"2024-02-18T00:31:23.018547+0000","last_fullsized":"2024-02-18T00:31:23.018547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.018460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.018460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978161+0000","last_change":"2024-02-18T00:30:38.507742+0000","last_active":"2024-02-18T00:31:16.978161+0000","last_peered":"2024-02-18T00:31:16.978161+0000","last_clean":"2024-02-18T00:31:16.978161+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:31:16.978161+0000","last_undegraded":"2024-02-18T00:31:16.978161+0000","last_fullsized":"2024-02-18T00:31:16.978161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.507637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.507637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":86,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645263+0000","last_change":"2024-02-18T00:30:41.648179+0000","last_active":"2024-02-18T00:31:16.645263+0000","last_peered":"2024-02-18T00:31:16.645263+0000","last_clean":"2024-02-18T00:31:16.645263+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:31:16.645263+0000","last_undegraded":"2024-02-18T00:31:16.645263+0000","last_fullsized":"2024-02-18T00:31:16.645263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:30:21.733434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.733434+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976542+0000","last_change":"2024-02-18T00:30:46.308529+0000","last_active":"2024-02-18T00:31:16.976542+0000","last_peered":"2024-02-18T00:31:16.976542+0000","last_clean":"2024-02-18T00:31:16.976542+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:31:16.976542+0000","last_undegraded":"2024-02-18T00:31:16.976542+0000","last_fullsized":"2024-02-18T00:31:16.976542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.308454+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645348+0000","last_change":"2024-02-18T00:30:52.923082+0000","last_active":"2024-02-18T00:31:16.645348+0000","last_peered":"2024-02-18T00:31:16.645348+0000","last_clean":"2024-02-18T00:31:16.645348+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:31:16.645348+0000","last_undegraded":"2024-02-18T00:31:16.645348+0000","last_fullsized":"2024-02-18T00:31:16.645348+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.923014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.923014+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975828+0000","last_change":"2024-02-18T00:30:32.364058+0000","last_active":"2024-02-18T00:31:16.975828+0000","last_peered":"2024-02-18T00:31:16.975828+0000","last_clean":"2024-02-18T00:31:16.975828+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:31:16.975828+0000","last_undegraded":"2024-02-18T00:31:16.975828+0000","last_fullsized":"2024-02-18T00:31:16.975828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:32.363988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.363988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:18.065909+0000","last_change":"2024-02-18T00:31:18.065868+0000","last_active":"2024-02-18T00:31:18.065909+0000","last_peered":"2024-02-18T00:31:18.065909+0000","last_clean":"2024-02-18T00:31:18.065909+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:31:18.065909+0000","last_undegraded":"2024-02-18T00:31:18.065909+0000","last_fullsized":"2024-02-18T00:31:18.065909+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:31:18.065789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.065789+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644695+0000","last_change":"2024-02-18T00:30:29.789176+0000","last_active":"2024-02-18T00:31:16.644695+0000","last_peered":"2024-02-18T00:31:16.644695+0000","last_clean":"2024-02-18T00:31:16.644695+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:31:16.644695+0000","last_undegraded":"2024-02-18T00:31:16.644695+0000","last_fullsized":"2024-02-18T00:31:16.644695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.789093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.789093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645951+0000","last_change":"2024-02-18T00:30:38.058675+0000","last_active":"2024-02-18T00:31:16.645951+0000","last_peered":"2024-02-18T00:31:16.645951+0000","last_clean":"2024-02-18T00:31:16.645951+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:31:16.645951+0000","last_undegraded":"2024-02-18T00:31:16.645951+0000","last_fullsized":"2024-02-18T00:31:16.645951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:30:38.058595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.058595+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645303+0000","last_change":"2024-02-18T00:30:54.996645+0000","last_active":"2024-02-18T00:31:16.645303+0000","last_peered":"2024-02-18T00:31:16.645303+0000","last_clean":"2024-02-18T00:31:16.645303+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:31:16.645303+0000","last_undegraded":"2024-02-18T00:31:16.645303+0000","last_fullsized":"2024-02-18T00:31:16.645303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:30:54.996545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.996545+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975812+0000","last_change":"2024-02-18T00:30:43.221614+0000","last_active":"2024-02-18T00:31:16.975812+0000","last_peered":"2024-02-18T00:31:16.975812+0000","last_clean":"2024-02-18T00:31:16.975812+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:31:16.975812+0000","last_undegraded":"2024-02-18T00:31:16.975812+0000","last_fullsized":"2024-02-18T00:31:16.975812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.221541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.221541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.132994+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:31:20.132994+0000","last_peered":"2024-02-18T00:31:20.132994+0000","last_clean":"2024-02-18T00:31:20.132994+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:31:20.132994+0000","last_undegraded":"2024-02-18T00:31:20.132994+0000","last_fullsized":"2024-02-18T00:31:20.132994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645731+0000","last_change":"2024-02-18T00:30:40.081002+0000","last_active":"2024-02-18T00:31:16.645731+0000","last_peered":"2024-02-18T00:31:16.645731+0000","last_clean":"2024-02-18T00:31:16.645731+0000","last_became_active":"2024-02-18T00:29:14.715019+0000","last_became_peered":"2024-02-18T00:29:14.715019+0000","last_unstale":"2024-02-18T00:31:16.645731+0000","last_undegraded":"2024-02-18T00:31:16.645731+0000","last_fullsized":"2024-02-18T00:31:16.645731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.080928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.080928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'16","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976775+0000","last_change":"2024-02-18T00:30:41.649899+0000","last_active":"2024-02-18T00:31:16.976775+0000","last_peered":"2024-02-18T00:31:16.976775+0000","last_clean":"2024-02-18T00:31:16.976775+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:31:16.976775+0000","last_undegraded":"2024-02-18T00:31:16.976775+0000","last_fullsized":"2024-02-18T00:31:16.976775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:30:22.276257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:22.276257+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:19.639802+0000","last_change":"2024-02-18T00:30:46.767980+0000","last_active":"2024-02-18T00:31:19.639802+0000","last_peered":"2024-02-18T00:31:19.639802+0000","last_clean":"2024-02-18T00:31:19.639802+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:31:19.639802+0000","last_undegraded":"2024-02-18T00:31:19.639802+0000","last_fullsized":"2024-02-18T00:31:19.639802+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:46.767898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.767898+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645110+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:31:16.645110+0000","last_peered":"2024-02-18T00:31:16.645110+0000","last_clean":"2024-02-18T00:31:16.645110+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:31:16.645110+0000","last_undegraded":"2024-02-18T00:31:16.645110+0000","last_fullsized":"2024-02-18T00:31:16.645110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977772+0000","last_change":"2024-02-18T00:31:08.046030+0000","last_active":"2024-02-18T00:31:16.977772+0000","last_peered":"2024-02-18T00:31:16.977772+0000","last_clean":"2024-02-18T00:31:16.977772+0000","last_became_active":"2024-02-18T00:29:14.715157+0000","last_became_peered":"2024-02-18T00:29:14.715157+0000","last_unstale":"2024-02-18T00:31:16.977772+0000","last_undegraded":"2024-02-18T00:31:16.977772+0000","last_fullsized":"2024-02-18T00:31:16.977772+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:31:08.045946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.004834+0000","last_change":"2024-02-18T00:30:58.030242+0000","last_active":"2024-02-18T00:31:20.004834+0000","last_peered":"2024-02-18T00:31:20.004834+0000","last_clean":"2024-02-18T00:31:20.004834+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:31:20.004834+0000","last_undegraded":"2024-02-18T00:31:20.004834+0000","last_fullsized":"2024-02-18T00:31:20.004834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:30:58.030175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.030175+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976440+0000","last_change":"2024-02-18T00:30:47.731838+0000","last_active":"2024-02-18T00:31:16.976440+0000","last_peered":"2024-02-18T00:31:16.976440+0000","last_clean":"2024-02-18T00:31:16.976440+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:31:16.976440+0000","last_undegraded":"2024-02-18T00:31:16.976440+0000","last_fullsized":"2024-02-18T00:31:16.976440+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.731752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.731752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978469+0000","last_change":"2024-02-18T00:30:58.601762+0000","last_active":"2024-02-18T00:31:16.978469+0000","last_peered":"2024-02-18T00:31:16.978469+0000","last_clean":"2024-02-18T00:31:16.978469+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:31:16.978469+0000","last_undegraded":"2024-02-18T00:31:16.978469+0000","last_fullsized":"2024-02-18T00:31:16.978469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.601694+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":170,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.909499+0000","last_change":"2024-02-18T00:30:32.271251+0000","last_active":"2024-02-18T00:31:23.909499+0000","last_peered":"2024-02-18T00:31:23.909499+0000","last_clean":"2024-02-18T00:31:23.909499+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:31:23.909499+0000","last_undegraded":"2024-02-18T00:31:23.909499+0000","last_fullsized":"2024-02-18T00:31:23.909499+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.271183+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975870+0000","last_change":"2024-02-18T00:30:48.262064+0000","last_active":"2024-02-18T00:31:16.975870+0000","last_peered":"2024-02-18T00:31:16.975870+0000","last_clean":"2024-02-18T00:31:16.975870+0000","last_became_active":"2024-02-18T00:29:14.713812+0000","last_became_peered":"2024-02-18T00:29:14.713812+0000","last_unstale":"2024-02-18T00:31:16.975870+0000","last_undegraded":"2024-02-18T00:31:16.975870+0000","last_fullsized":"2024-02-18T00:31:16.975870+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.262001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.262001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":85,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646327+0000","last_change":"2024-02-18T00:30:41.040669+0000","last_active":"2024-02-18T00:31:16.646327+0000","last_peered":"2024-02-18T00:31:16.646327+0000","last_clean":"2024-02-18T00:31:16.646327+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:31:16.646327+0000","last_undegraded":"2024-02-18T00:31:16.646327+0000","last_fullsized":"2024-02-18T00:31:16.646327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:30:41.040595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.040595+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976455+0000","last_change":"2024-02-18T00:30:24.405804+0000","last_active":"2024-02-18T00:31:16.976455+0000","last_peered":"2024-02-18T00:31:16.976455+0000","last_clean":"2024-02-18T00:31:16.976455+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:31:16.976455+0000","last_undegraded":"2024-02-18T00:31:16.976455+0000","last_fullsized":"2024-02-18T00:31:16.976455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.405729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.405729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978001+0000","last_change":"2024-02-18T00:30:34.100267+0000","last_active":"2024-02-18T00:31:16.978001+0000","last_peered":"2024-02-18T00:31:16.978001+0000","last_clean":"2024-02-18T00:31:16.978001+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:31:16.978001+0000","last_undegraded":"2024-02-18T00:31:16.978001+0000","last_fullsized":"2024-02-18T00:31:16.978001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_clean_scrub_stamp":"2024-02-18T00:30:34.100174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644617+0000","last_change":"2024-02-18T00:30:56.034602+0000","last_active":"2024-02-18T00:31:16.644617+0000","last_peered":"2024-02-18T00:31:16.644617+0000","last_clean":"2024-02-18T00:31:16.644617+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:31:16.644617+0000","last_undegraded":"2024-02-18T00:31:16.644617+0000","last_fullsized":"2024-02-18T00:31:16.644617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:56.034493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:56.034493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977267+0000","last_change":"2024-02-18T00:30:40.055404+0000","last_active":"2024-02-18T00:31:16.977267+0000","last_peered":"2024-02-18T00:31:16.977267+0000","last_clean":"2024-02-18T00:31:16.977267+0000","last_became_active":"2024-02-18T00:29:14.716595+0000","last_became_peered":"2024-02-18T00:29:14.716595+0000","last_unstale":"2024-02-18T00:31:16.977267+0000","last_undegraded":"2024-02-18T00:31:16.977267+0000","last_fullsized":"2024-02-18T00:31:16.977267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.055253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.055253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976838+0000","last_change":"2024-02-18T00:30:46.504493+0000","last_active":"2024-02-18T00:31:16.976838+0000","last_peered":"2024-02-18T00:31:16.976838+0000","last_clean":"2024-02-18T00:31:16.976838+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:31:16.976838+0000","last_undegraded":"2024-02-18T00:31:16.976838+0000","last_fullsized":"2024-02-18T00:31:16.976838+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.504422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.504422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:25.298149+0000","last_change":"2024-02-18T00:31:25.298149+0000","last_active":"2024-02-18T00:31:25.298149+0000","last_peered":"2024-02-18T00:31:25.298149+0000","last_clean":"2024-02-18T00:31:25.298149+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:31:25.298149+0000","last_undegraded":"2024-02-18T00:31:25.298149+0000","last_fullsized":"2024-02-18T00:31:25.298149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:31:25.298062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:25.298062+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977464+0000","last_change":"2024-02-18T00:30:27.608046+0000","last_active":"2024-02-18T00:31:16.977464+0000","last_peered":"2024-02-18T00:31:16.977464+0000","last_clean":"2024-02-18T00:31:16.977464+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:31:16.977464+0000","last_undegraded":"2024-02-18T00:31:16.977464+0000","last_fullsized":"2024-02-18T00:31:16.977464+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:27.607981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.607981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645600+0000","last_change":"2024-02-18T00:31:00.091252+0000","last_active":"2024-02-18T00:31:16.645600+0000","last_peered":"2024-02-18T00:31:16.645600+0000","last_clean":"2024-02-18T00:31:16.645600+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:31:16.645600+0000","last_undegraded":"2024-02-18T00:31:16.645600+0000","last_fullsized":"2024-02-18T00:31:16.645600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:00.091169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.091169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976219+0000","last_change":"2024-02-18T00:30:20.623950+0000","last_active":"2024-02-18T00:31:16.976219+0000","last_peered":"2024-02-18T00:31:16.976219+0000","last_clean":"2024-02-18T00:31:16.976219+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:31:16.976219+0000","last_undegraded":"2024-02-18T00:31:16.976219+0000","last_fullsized":"2024-02-18T00:31:16.976219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.623829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.623829+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976764+0000","last_change":"2024-02-18T00:30:54.590835+0000","last_active":"2024-02-18T00:31:16.976764+0000","last_peered":"2024-02-18T00:31:16.976764+0000","last_clean":"2024-02-18T00:31:16.976764+0000","last_became_active":"2024-02-18T00:29:14.714355+0000","last_became_peered":"2024-02-18T00:29:14.714355+0000","last_unstale":"2024-02-18T00:31:16.976764+0000","last_undegraded":"2024-02-18T00:31:16.976764+0000","last_fullsized":"2024-02-18T00:31:16.976764+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.590771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.590771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976670+0000","last_change":"2024-02-18T00:31:01.883504+0000","last_active":"2024-02-18T00:31:16.976670+0000","last_peered":"2024-02-18T00:31:16.976670+0000","last_clean":"2024-02-18T00:31:16.976670+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:31:16.976670+0000","last_undegraded":"2024-02-18T00:31:16.976670+0000","last_fullsized":"2024-02-18T00:31:16.976670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:01.883424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.883424+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977907+0000","last_change":"2024-02-18T00:30:52.072719+0000","last_active":"2024-02-18T00:31:16.977907+0000","last_peered":"2024-02-18T00:31:16.977907+0000","last_clean":"2024-02-18T00:31:16.977907+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:31:16.977907+0000","last_undegraded":"2024-02-18T00:31:16.977907+0000","last_fullsized":"2024-02-18T00:31:16.977907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.072619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.072619+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645041+0000","last_change":"2024-02-18T00:31:11.020203+0000","last_active":"2024-02-18T00:31:16.645041+0000","last_peered":"2024-02-18T00:31:16.645041+0000","last_clean":"2024-02-18T00:31:16.645041+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:31:16.645041+0000","last_undegraded":"2024-02-18T00:31:16.645041+0000","last_fullsized":"2024-02-18T00:31:16.645041+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:11.020119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.020119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978589+0000","last_change":"2024-02-18T00:30:53.587885+0000","last_active":"2024-02-18T00:31:16.978589+0000","last_peered":"2024-02-18T00:31:16.978589+0000","last_clean":"2024-02-18T00:31:16.978589+0000","last_became_active":"2024-02-18T00:29:14.715756+0000","last_became_peered":"2024-02-18T00:29:14.715756+0000","last_unstale":"2024-02-18T00:31:16.978589+0000","last_undegraded":"2024-02-18T00:31:16.978589+0000","last_fullsized":"2024-02-18T00:31:16.978589+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:30:53.587796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:53.587796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976564+0000","last_change":"2024-02-18T00:30:48.688989+0000","last_active":"2024-02-18T00:31:16.976564+0000","last_peered":"2024-02-18T00:31:16.976564+0000","last_clean":"2024-02-18T00:31:16.976564+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:31:16.976564+0000","last_undegraded":"2024-02-18T00:31:16.976564+0000","last_fullsized":"2024-02-18T00:31:16.976564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:30:48.688901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.688901+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978389+0000","last_change":"2024-02-18T00:30:45.463697+0000","last_active":"2024-02-18T00:31:16.978389+0000","last_peered":"2024-02-18T00:31:16.978389+0000","last_clean":"2024-02-18T00:31:16.978389+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:31:16.978389+0000","last_undegraded":"2024-02-18T00:31:16.978389+0000","last_fullsized":"2024-02-18T00:31:16.978389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645664+0000","last_change":"2024-02-18T00:30:45.062422+0000","last_active":"2024-02-18T00:31:16.645664+0000","last_peered":"2024-02-18T00:31:16.645664+0000","last_clean":"2024-02-18T00:31:16.645664+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:31:16.645664+0000","last_undegraded":"2024-02-18T00:31:16.645664+0000","last_fullsized":"2024-02-18T00:31:16.645664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.062339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.062339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646272+0000","last_change":"2024-02-18T00:30:35.177065+0000","last_active":"2024-02-18T00:31:16.646272+0000","last_peered":"2024-02-18T00:31:16.646272+0000","last_clean":"2024-02-18T00:31:16.646272+0000","last_became_active":"2024-02-18T00:29:14.714616+0000","last_became_peered":"2024-02-18T00:29:14.714616+0000","last_unstale":"2024-02-18T00:31:16.646272+0000","last_undegraded":"2024-02-18T00:31:16.646272+0000","last_fullsized":"2024-02-18T00:31:16.646272+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.176991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.176991+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"64'30","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644349+0000","last_change":"2024-02-18T00:30:39.827787+0000","last_active":"2024-02-18T00:31:16.644349+0000","last_peered":"2024-02-18T00:31:16.644349+0000","last_clean":"2024-02-18T00:31:16.644349+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:31:16.644349+0000","last_undegraded":"2024-02-18T00:31:16.644349+0000","last_fullsized":"2024-02-18T00:31:16.644349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'30","last_scrub_stamp":"2024-02-18T00:30:39.827720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.827720+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645451+0000","last_change":"2024-02-18T00:30:26.099064+0000","last_active":"2024-02-18T00:31:16.645451+0000","last_peered":"2024-02-18T00:31:16.645451+0000","last_clean":"2024-02-18T00:31:16.645451+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:31:16.645451+0000","last_undegraded":"2024-02-18T00:31:16.645451+0000","last_fullsized":"2024-02-18T00:31:16.645451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.098982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.098982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977642+0000","last_change":"2024-02-18T00:30:44.565713+0000","last_active":"2024-02-18T00:31:16.977642+0000","last_peered":"2024-02-18T00:31:16.977642+0000","last_clean":"2024-02-18T00:31:16.977642+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:31:16.977642+0000","last_undegraded":"2024-02-18T00:31:16.977642+0000","last_fullsized":"2024-02-18T00:31:16.977642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.565626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.565626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977858+0000","last_change":"2024-02-18T00:30:37.538788+0000","last_active":"2024-02-18T00:31:16.977858+0000","last_peered":"2024-02-18T00:31:16.977858+0000","last_clean":"2024-02-18T00:31:16.977858+0000","last_became_active":"2024-02-18T00:29:14.714186+0000","last_became_peered":"2024-02-18T00:29:14.714186+0000","last_unstale":"2024-02-18T00:31:16.977858+0000","last_undegraded":"2024-02-18T00:31:16.977858+0000","last_fullsized":"2024-02-18T00:31:16.977858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:37.538700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:37.538700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645236+0000","last_change":"2024-02-18T00:30:21.092976+0000","last_active":"2024-02-18T00:31:16.645236+0000","last_peered":"2024-02-18T00:31:16.645236+0000","last_clean":"2024-02-18T00:31:16.645236+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:31:16.645236+0000","last_undegraded":"2024-02-18T00:31:16.645236+0000","last_fullsized":"2024-02-18T00:31:16.645236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:30:21.092908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.092908+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.131160+0000","last_change":"2024-02-18T00:30:22.716297+0000","last_active":"2024-02-18T00:31:20.131160+0000","last_peered":"2024-02-18T00:31:20.131160+0000","last_clean":"2024-02-18T00:31:20.131160+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:31:20.131160+0000","last_undegraded":"2024-02-18T00:31:20.131160+0000","last_fullsized":"2024-02-18T00:31:20.131160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:30:22.716226+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644566+0000","last_change":"2024-02-18T00:30:40.855003+0000","last_active":"2024-02-18T00:31:16.644566+0000","last_peered":"2024-02-18T00:31:16.644566+0000","last_clean":"2024-02-18T00:31:16.644566+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:31:16.644566+0000","last_undegraded":"2024-02-18T00:31:16.644566+0000","last_fullsized":"2024-02-18T00:31:16.644566+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.854928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.854928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977743+0000","last_change":"2024-02-18T00:31:16.020640+0000","last_active":"2024-02-18T00:31:16.977743+0000","last_peered":"2024-02-18T00:31:16.977743+0000","last_clean":"2024-02-18T00:31:16.977743+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:31:16.977743+0000","last_undegraded":"2024-02-18T00:31:16.977743+0000","last_fullsized":"2024-02-18T00:31:16.977743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:16.020558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:16.020558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976891+0000","last_change":"2024-02-18T00:30:45.530383+0000","last_active":"2024-02-18T00:31:16.976891+0000","last_peered":"2024-02-18T00:31:16.976891+0000","last_clean":"2024-02-18T00:31:16.976891+0000","last_became_active":"2024-02-18T00:29:14.713770+0000","last_became_peered":"2024-02-18T00:29:14.713770+0000","last_unstale":"2024-02-18T00:31:16.976891+0000","last_undegraded":"2024-02-18T00:31:16.976891+0000","last_fullsized":"2024-02-18T00:31:16.976891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.530294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.530294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646083+0000","last_change":"2024-02-18T00:30:32.029864+0000","last_active":"2024-02-18T00:31:16.646083+0000","last_peered":"2024-02-18T00:31:16.646083+0000","last_clean":"2024-02-18T00:31:16.646083+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:31:16.646083+0000","last_undegraded":"2024-02-18T00:31:16.646083+0000","last_fullsized":"2024-02-18T00:31:16.646083+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:32.029757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.029757+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.987881+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:31:23.987881+0000","last_peered":"2024-02-18T00:31:23.987881+0000","last_clean":"2024-02-18T00:31:23.987881+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:31:23.987881+0000","last_undegraded":"2024-02-18T00:31:23.987881+0000","last_fullsized":"2024-02-18T00:31:23.987881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:19.640720+0000","last_change":"2024-02-18T00:31:05.997541+0000","last_active":"2024-02-18T00:31:19.640720+0000","last_peered":"2024-02-18T00:31:19.640720+0000","last_clean":"2024-02-18T00:31:19.640720+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:31:19.640720+0000","last_undegraded":"2024-02-18T00:31:19.640720+0000","last_fullsized":"2024-02-18T00:31:19.640720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:05.997460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.997460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645143+0000","last_change":"2024-02-18T00:30:20.213263+0000","last_active":"2024-02-18T00:31:16.645143+0000","last_peered":"2024-02-18T00:31:16.645143+0000","last_clean":"2024-02-18T00:31:16.645143+0000","last_became_active":"2024-02-18T00:29:14.713732+0000","last_became_peered":"2024-02-18T00:29:14.713732+0000","last_unstale":"2024-02-18T00:31:16.645143+0000","last_undegraded":"2024-02-18T00:31:16.645143+0000","last_fullsized":"2024-02-18T00:31:16.645143+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.213170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977393+0000","last_change":"2024-02-18T00:30:45.055464+0000","last_active":"2024-02-18T00:31:16.977393+0000","last_peered":"2024-02-18T00:31:16.977393+0000","last_clean":"2024-02-18T00:31:16.977393+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:31:16.977393+0000","last_undegraded":"2024-02-18T00:31:16.977393+0000","last_fullsized":"2024-02-18T00:31:16.977393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.055398+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645825+0000","last_change":"2024-02-18T00:30:44.103971+0000","last_active":"2024-02-18T00:31:16.645825+0000","last_peered":"2024-02-18T00:31:16.645825+0000","last_clean":"2024-02-18T00:31:16.645825+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:31:16.645825+0000","last_undegraded":"2024-02-18T00:31:16.645825+0000","last_fullsized":"2024-02-18T00:31:16.645825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.103899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.103899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:19.637612+0000","last_change":"2024-02-18T00:30:51.228973+0000","last_active":"2024-02-18T00:31:19.637612+0000","last_peered":"2024-02-18T00:31:19.637612+0000","last_clean":"2024-02-18T00:31:19.637612+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:31:19.637612+0000","last_undegraded":"2024-02-18T00:31:19.637612+0000","last_fullsized":"2024-02-18T00:31:19.637612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:51.228902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.228902+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645285+0000","last_change":"2024-02-18T00:30:29.330196+0000","last_active":"2024-02-18T00:31:16.645285+0000","last_peered":"2024-02-18T00:31:16.645285+0000","last_clean":"2024-02-18T00:31:16.645285+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:31:16.645285+0000","last_undegraded":"2024-02-18T00:31:16.645285+0000","last_fullsized":"2024-02-18T00:31:16.645285+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.330124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.330124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645550+0000","last_change":"2024-02-18T00:31:11.002136+0000","last_active":"2024-02-18T00:31:16.645550+0000","last_peered":"2024-02-18T00:31:16.645550+0000","last_clean":"2024-02-18T00:31:16.645550+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:31:16.645550+0000","last_undegraded":"2024-02-18T00:31:16.645550+0000","last_fullsized":"2024-02-18T00:31:16.645550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:31:11.002039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.002039+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977085+0000","last_change":"2024-02-18T00:30:41.301486+0000","last_active":"2024-02-18T00:31:16.977085+0000","last_peered":"2024-02-18T00:31:16.977085+0000","last_clean":"2024-02-18T00:31:16.977085+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:31:16.977085+0000","last_undegraded":"2024-02-18T00:31:16.977085+0000","last_fullsized":"2024-02-18T00:31:16.977085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.301405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.301405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645652+0000","last_change":"2024-02-18T00:30:59.059018+0000","last_active":"2024-02-18T00:31:16.645652+0000","last_peered":"2024-02-18T00:31:16.645652+0000","last_clean":"2024-02-18T00:31:16.645652+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:31:16.645652+0000","last_undegraded":"2024-02-18T00:31:16.645652+0000","last_fullsized":"2024-02-18T00:31:16.645652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.058947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.058947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977236+0000","last_change":"2024-02-18T00:30:24.601515+0000","last_active":"2024-02-18T00:31:16.977236+0000","last_peered":"2024-02-18T00:31:16.977236+0000","last_clean":"2024-02-18T00:31:16.977236+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:31:16.977236+0000","last_undegraded":"2024-02-18T00:31:16.977236+0000","last_fullsized":"2024-02-18T00:31:16.977236+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.601442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.601442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646032+0000","last_change":"2024-02-18T00:30:19.032163+0000","last_active":"2024-02-18T00:31:16.646032+0000","last_peered":"2024-02-18T00:31:16.646032+0000","last_clean":"2024-02-18T00:31:16.646032+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:31:16.646032+0000","last_undegraded":"2024-02-18T00:31:16.646032+0000","last_fullsized":"2024-02-18T00:31:16.646032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:19.032095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.032095+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976515+0000","last_change":"2024-02-18T00:30:43.541530+0000","last_active":"2024-02-18T00:31:16.976515+0000","last_peered":"2024-02-18T00:31:16.976515+0000","last_clean":"2024-02-18T00:31:16.976515+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:31:16.976515+0000","last_undegraded":"2024-02-18T00:31:16.976515+0000","last_fullsized":"2024-02-18T00:31:16.976515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976371+0000","last_change":"2024-02-18T00:30:45.273987+0000","last_active":"2024-02-18T00:31:16.976371+0000","last_peered":"2024-02-18T00:31:16.976371+0000","last_clean":"2024-02-18T00:31:16.976371+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:31:16.976371+0000","last_undegraded":"2024-02-18T00:31:16.976371+0000","last_fullsized":"2024-02-18T00:31:16.976371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.273920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.273920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976771+0000","last_change":"2024-02-18T00:30:38.317025+0000","last_active":"2024-02-18T00:31:16.976771+0000","last_peered":"2024-02-18T00:31:16.976771+0000","last_clean":"2024-02-18T00:31:16.976771+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:31:16.976771+0000","last_undegraded":"2024-02-18T00:31:16.976771+0000","last_fullsized":"2024-02-18T00:31:16.976771+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.316960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.316960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978338+0000","last_change":"2024-02-18T00:30:29.539942+0000","last_active":"2024-02-18T00:31:16.978338+0000","last_peered":"2024-02-18T00:31:16.978338+0000","last_clean":"2024-02-18T00:31:16.978338+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:31:16.978338+0000","last_undegraded":"2024-02-18T00:31:16.978338+0000","last_fullsized":"2024-02-18T00:31:16.978338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:30:29.539874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.539874+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976013+0000","last_change":"2024-02-18T00:30:50.764741+0000","last_active":"2024-02-18T00:31:16.976013+0000","last_peered":"2024-02-18T00:31:16.976013+0000","last_clean":"2024-02-18T00:31:16.976013+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:31:16.976013+0000","last_undegraded":"2024-02-18T00:31:16.976013+0000","last_fullsized":"2024-02-18T00:31:16.976013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.764665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.764665+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976905+0000","last_change":"2024-02-18T00:30:39.299351+0000","last_active":"2024-02-18T00:31:16.976905+0000","last_peered":"2024-02-18T00:31:16.976905+0000","last_clean":"2024-02-18T00:31:16.976905+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:31:16.976905+0000","last_undegraded":"2024-02-18T00:31:16.976905+0000","last_fullsized":"2024-02-18T00:31:16.976905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:39.299285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.299285+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645980+0000","last_change":"2024-02-18T00:30:48.980366+0000","last_active":"2024-02-18T00:31:16.645980+0000","last_peered":"2024-02-18T00:31:16.645980+0000","last_clean":"2024-02-18T00:31:16.645980+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:31:16.645980+0000","last_undegraded":"2024-02-18T00:31:16.645980+0000","last_fullsized":"2024-02-18T00:31:16.645980+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.980292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.980292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976654+0000","last_change":"2024-02-18T00:31:07.953329+0000","last_active":"2024-02-18T00:31:16.976654+0000","last_peered":"2024-02-18T00:31:16.976654+0000","last_clean":"2024-02-18T00:31:16.976654+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:31:16.976654+0000","last_undegraded":"2024-02-18T00:31:16.976654+0000","last_fullsized":"2024-02-18T00:31:16.976654+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:07.953265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.953265+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977591+0000","last_change":"2024-02-18T00:30:30.045287+0000","last_active":"2024-02-18T00:31:16.977591+0000","last_peered":"2024-02-18T00:31:16.977591+0000","last_clean":"2024-02-18T00:31:16.977591+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:31:16.977591+0000","last_undegraded":"2024-02-18T00:31:16.977591+0000","last_fullsized":"2024-02-18T00:31:16.977591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.045217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.045217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976603+0000","last_change":"2024-02-18T00:30:47.275395+0000","last_active":"2024-02-18T00:31:16.976603+0000","last_peered":"2024-02-18T00:31:16.976603+0000","last_clean":"2024-02-18T00:31:16.976603+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:31:16.976603+0000","last_undegraded":"2024-02-18T00:31:16.976603+0000","last_fullsized":"2024-02-18T00:31:16.976603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.275328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.275328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976356+0000","last_change":"2024-02-18T00:31:07.029773+0000","last_active":"2024-02-18T00:31:16.976356+0000","last_peered":"2024-02-18T00:31:16.976356+0000","last_clean":"2024-02-18T00:31:16.976356+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:31:16.976356+0000","last_undegraded":"2024-02-18T00:31:16.976356+0000","last_fullsized":"2024-02-18T00:31:16.976356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:31:07.029677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.029677+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.126669+0000","last_change":"2024-02-18T00:30:49.588342+0000","last_active":"2024-02-18T00:31:20.126669+0000","last_peered":"2024-02-18T00:31:20.126669+0000","last_clean":"2024-02-18T00:31:20.126669+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:31:20.126669+0000","last_undegraded":"2024-02-18T00:31:20.126669+0000","last_fullsized":"2024-02-18T00:31:20.126669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:49.588269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:49.588269+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":595,"ondisk_log_size":595,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266718,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":929792,"data_stored":387426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.505,"15min":0.502},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84899999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.88,"15min":0.88},"last":0.60299999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.54400000000000004}]},{"osd":3,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.34000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.39400000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.61499999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.40799999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.51200000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.432},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.5}]}]},{"osd":6,"up_from":47,"seq":201863462948,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":909312,"data_stored":386261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.505,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.437}]},{"osd":2,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51200000000000001,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.65300000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.629},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.80400000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.505},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.075,"15min":1.075},"last":0.748},{"interface":"front","average":{"1min":0.503,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447128,"statfs":{"total":95995035648,"available":95689859072,"internally_reserved":0,"allocated":950272,"data_stored":387084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.35999999999999999,"15min":0.311},"min":{"1min":0.29299999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.371},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.32200000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.373},"min":{"1min":0.27700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.32900000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.47899999999999998,"15min":0.48699999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.67800000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.439,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82499999999999996,"5min":1.278,"15min":1.278},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.63},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79400000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66200000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":925696,"data_stored":387067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.48599999999999999,"15min":0.59799999999999998},"min":{"1min":0.23300000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.78000000000000003,"5min":6.7720000000000002,"15min":6.7720000000000002},"last":0.308},{"interface":"front","average":{"1min":0.378,"5min":0.38100000000000001,"15min":0.34699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.623,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.26300000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.23799999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.28499999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.42999999999999999,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.47999999999999998,"15min":0.501},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.32400000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84899999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.73799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.64500000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66900000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56899999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411384,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":905216,"data_stored":386338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39500000000000002,"15min":0.374},"min":{"1min":0.26100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.40400000000000003,"15min":0.378},"min":{"1min":0.29699999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.51600000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.32100000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.38600000000000001},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55200000000000005,"5min":2.306,"15min":2.306},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.42299999999999999,"15min":0.40100000000000002},"min":{"1min":0.27400000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58899999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.371}]},{"osd":3,"last update":"Sun Feb 18 00:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46999999999999997,"15min":0.46000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.34999999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.65700000000000003,"5min":1.675,"15min":1.675},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91200000000000003,"5min":1.173,"15min":1.173},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.41499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.62,"15min":0.627},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.66500000000000004,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.753,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72399999999999998,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89300000000000002,"5min":1.242,"15min":1.242},"last":0.69599999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149746,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298016,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447136,"statfs":{"total":95995035648,"available":95689867264,"internally_reserved":0,"allocated":942080,"data_stored":386736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.38400000000000001,"15min":0.39400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.58999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53400000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.436}]},{"osd":1,"last update":"Sun Feb 18 00:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57599999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.58299999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.372}]},{"osd":2,"last update":"Sun Feb 18 00:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.436,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56000000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.42899999999999999,"15min":0.432},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59499999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:31:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77900000000000003,"5min":1.083,"15min":1.083},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.59299999999999997,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.94199999999999995,"5min":1.139,"15min":1.139},"last":0.71499999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82299999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86199999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.78400000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60599999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986221,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":929792,"data_stored":387221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":2.339,"15min":2.339},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":1.288,"15min":1.288},"last":0.59299999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55900000000000005,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65200000000000002,"5min":1.204,"15min":1.204},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70499999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61699999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":1.48,"15min":1.48},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.497}]},{"osd":3,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58099999999999996,"15min":0.60499999999999998},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69399999999999995,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.71699999999999997,"5min":1.133,"15min":1.133},"last":0.70199999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35299999999999998},"min":{"1min":0.29499999999999998,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.36799999999999999,"15min":0.35999999999999999},"min":{"1min":0.26800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.68300000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:30:46 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.752},{"interface":"front","average":{"1min":0.504,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.71699999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724584,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":901120,"data_stored":385621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53900000000000003}]},{"osd":1,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":1.306,"15min":1.306},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":1.272,"15min":1.272},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78800000000000003,"5min":1.173,"15min":1.173},"last":0.623},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63100000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":1.238,"15min":1.238},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72099999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.438},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.378},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.495},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45600000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:31:31.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:31:31.920 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:30.355134+0000 osd.6 (osd.6) 31 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:30.356807+0000 osd.6 (osd.6) 32 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:30.716142+0000 mgr.y (mgr.24454) 90 : audit [DBG] from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:30.786754+0000 osd.5 (osd.5) 43 : cluster [DBG] 2.16 deep-scrub starts 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:30.789629+0000 osd.5 (osd.5) 44 : cluster [DBG] 2.16 deep-scrub ok 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:30.830250+0000 mgr.y (mgr.24454) 91 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:30.355134+0000 osd.6 (osd.6) 31 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:30.356807+0000 osd.6 (osd.6) 32 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:31:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:30.716142+0000 mgr.y (mgr.24454) 90 : audit [DBG] from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:30.786754+0000 osd.5 (osd.5) 43 : cluster [DBG] 2.16 deep-scrub starts 2024-02-18T00:31:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:30.789629+0000 osd.5 (osd.5) 44 : cluster [DBG] 2.16 deep-scrub ok 2024-02-18T00:31:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:30.830250+0000 mgr.y (mgr.24454) 91 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:30.355134+0000 osd.6 (osd.6) 31 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:31:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:30.356807+0000 osd.6 (osd.6) 32 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:31:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:30.716142+0000 mgr.y (mgr.24454) 90 : audit [DBG] from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:30.786754+0000 osd.5 (osd.5) 43 : cluster [DBG] 2.16 deep-scrub starts 2024-02-18T00:31:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:30.789629+0000 osd.5 (osd.5) 44 : cluster [DBG] 2.16 deep-scrub ok 2024-02-18T00:31:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:30.830250+0000 mgr.y (mgr.24454) 91 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:32.481 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:32] "GET /metrics HTTP/1.1" 200 172164 "" "Prometheus/2.33.4" 2024-02-18T00:31:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:31.745044+0000 mgr.y (mgr.24454) 92 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:32.008420+0000 osd.4 (osd.4) 45 : cluster [DBG] 4.d scrub starts 2024-02-18T00:31:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:32.011890+0000 osd.4 (osd.4) 46 : cluster [DBG] 4.d scrub ok 2024-02-18T00:31:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:32.119982+0000 osd.7 (osd.7) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:32.120027+0000 osd.7 (osd.7) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:31.745044+0000 mgr.y (mgr.24454) 92 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:32.008420+0000 osd.4 (osd.4) 45 : cluster [DBG] 4.d scrub starts 2024-02-18T00:31:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:32.011890+0000 osd.4 (osd.4) 46 : cluster [DBG] 4.d scrub ok 2024-02-18T00:31:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:32.119982+0000 osd.7 (osd.7) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:33.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:32.120027+0000 osd.7 (osd.7) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:33.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:31.745044+0000 mgr.y (mgr.24454) 92 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:33.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:32.008420+0000 osd.4 (osd.4) 45 : cluster [DBG] 4.d scrub starts 2024-02-18T00:31:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:32.011890+0000 osd.4 (osd.4) 46 : cluster [DBG] 4.d scrub ok 2024-02-18T00:31:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:32.119982+0000 osd.7 (osd.7) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:32.120027+0000 osd.7 (osd.7) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:33.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 -- 172.21.15.59:0/1621160498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480fe510 msgr2=0x7f6e480f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:33.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 --2- 172.21.15.59:0/1621160498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480fe510 0x7f6e480f7d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6e30004870 tx=0x7f6e30009cb0).stop 2024-02-18T00:31:33.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 -- 172.21.15.59:0/1621160498 shutdown_connections 2024-02-18T00:31:33.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 --2- 172.21.15.59:0/1621160498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e480f8420 0x7f6e48104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:33.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 --2- 172.21.15.59:0/1621160498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e480fdbb0 0x7f6e480fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:33.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 --2- 172.21.15.59:0/1621160498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480fe510 0x7f6e480f7d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:33.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 -- 172.21.15.59:0/1621160498 >> 172.21.15.59:0/1621160498 conn(0x7f6e480f3800 msgr2=0x7f6e480f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:33.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 -- 172.21.15.59:0/1621160498 shutdown_connections 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.890+0000 7f6e4e381700 1 -- 172.21.15.59:0/1621160498 wait complete. 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 librados: starting msgr at 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 librados: starting objecter 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 Processor -- start 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 -- start start 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 librados: setting wanted keys 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 librados: calling monclient init 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480f8420 0x7f6e4810e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e480fdbb0 0x7f6e4810e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e480fe510 0x7f6e481082b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e481087a0 con 0x7f6e480fe510 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48108910 con 0x7f6e480fdbb0 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4e381700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48108ad0 con 0x7f6e480f8420 2024-02-18T00:31:33.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e47fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480f8420 0x7f6e4810e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e47fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480f8420 0x7f6e4810e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59348/0 (socket says 172.21.15.59:59348) 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e47fff700 1 -- 172.21.15.59:0/4101564661 learned_addr learned my addr 172.21.15.59:0/4101564661 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.891+0000 7f6e4c91e700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e480fe510 0x7f6e481082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e3ffff700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e480fdbb0 0x7f6e4810e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e47fff700 1 -- 172.21.15.59:0/4101564661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e480fdbb0 msgr2=0x7f6e4810e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e47fff700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e480fdbb0 0x7f6e4810e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e47fff700 1 -- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e480fe510 msgr2=0x7f6e481082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e47fff700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e480fe510 0x7f6e481082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e47fff700 1 -- 172.21.15.59:0/4101564661 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e38005040 con 0x7f6e480f8420 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e4c91e700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e480fe510 0x7f6e481082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e47fff700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480f8420 0x7f6e4810e110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6e380048f0 tx=0x7f6e38009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e45ffb700 10 client.?.objecter ms_handle_connect 0x7f6e480f8420 2024-02-18T00:31:33.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e45ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e45ffb700 1 -- 172.21.15.59:0/4101564661 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38007b00 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e30005040 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e45ffb700 1 -- 172.21.15.59:0/4101564661 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e3800f040 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e45ffb700 1 -- 172.21.15.59:0/4101564661 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38014a00 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e4e381700 10 client.24602.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e48114f40 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.892+0000 7f6e4e381700 1 librados: init done 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.893+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4805ee80 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.893+0000 7f6e45ffb700 1 -- 172.21.15.59:0/4101564661 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6e3800e580 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e45ffb700 1 --2- 172.21.15.59:0/4101564661 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6e34062c90 0x7f6e34065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e45ffb700 1 -- 172.21.15.59:0/4101564661 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f6e3807c600 con 0x7f6e480f8420 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e45ffb700 10 client.24602.objecter ms_dispatch 0x7f6e48001550 osd_map(89..89 src has 1..89) v4 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e45ffb700 3 client.24602.objecter handle_osd_map got epochs [89,89] > 0 2024-02-18T00:31:33.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e45ffb700 3 client.24602.objecter handle_osd_map decoding full epoch 89 2024-02-18T00:31:33.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e3ffff700 1 --2- 172.21.15.59:0/4101564661 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6e34062c90 0x7f6e34065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:33.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e45ffb700 20 client.24602.objecter dump_active .. 0 homeless 2024-02-18T00:31:33.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e45ffb700 10 client.24602.objecter ms_handle_connect 0x7f6e34062c90 2024-02-18T00:31:33.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.894+0000 7f6e3ffff700 1 --2- 172.21.15.59:0/4101564661 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6e34062c90 0x7f6e34065150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f6e30004ee0 tx=0x7f6e3000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:33.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:33.898+0000 7f6e45ffb700 1 -- 172.21.15.59:0/4101564661 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6e38049f60 con 0x7f6e480f8420 2024-02-18T00:31:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:32.208987+0000 osd.3 (osd.3) 55 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:31:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:32.210194+0000 osd.3 (osd.3) 56 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:31:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:32.823761+0000 mon.a (mon.0) 747 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-18T00:31:34.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:32.208987+0000 osd.3 (osd.3) 55 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:31:34.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:32.210194+0000 osd.3 (osd.3) 56 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:31:34.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:32.823761+0000 mon.a (mon.0) 747 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-18T00:31:34.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.067+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e4805fa60 con 0x7f6e34062c90 2024-02-18T00:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.077+0000 7f6e45ffb700 1 -- 172.21.15.59:0/4101564661 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357251 (secure 0 0 0) 0x7f6e4805fa60 con 0x7f6e34062c90 2024-02-18T00:31:34.084 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.080+0000 7f6e4e381700 10 librados: watch_flush enter 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 10 librados: watch_flush exit 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 20 client.24602.objecter shutdown clearing up homeless session... 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 10 client.24602.objecter successfully canceled tick 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6e34062c90 msgr2=0x7f6e34065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 --2- 172.21.15.59:0/4101564661 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6e34062c90 0x7f6e34065150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f6e30004ee0 tx=0x7f6e3000b040).stop 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480f8420 msgr2=0x7f6e4810e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480f8420 0x7f6e4810e110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f6e380048f0 tx=0x7f6e38009cb0).stop 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 shutdown_connections 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e480fe510 0x7f6e481082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e480f8420 0x7f6e4810e110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 --2- 172.21.15.59:0/4101564661 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6e34062c90 0x7f6e34065150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 --2- 172.21.15.59:0/4101564661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e480fdbb0 0x7f6e4810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 >> 172.21.15.59:0/4101564661 conn(0x7f6e480f3800 msgr2=0x7f6e480f5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 shutdown_connections 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 -- 172.21.15.59:0/4101564661 wait complete. 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:34.081+0000 7f6e4e381700 1 librados: shutdown 2024-02-18T00:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:31:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:32.208987+0000 osd.3 (osd.3) 55 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:31:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:32.210194+0000 osd.3 (osd.3) 56 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:31:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:32.823761+0000 mon.a (mon.0) 747 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-18T00:31:34.924 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-02-18T00:31:33.745197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1231,"num_read_kb":951,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":674,"ondisk_log_size":674,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2383988,"kb_used_data":7220,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747577228,"statfs":{"total":767960285184,"available":765519081472,"internally_reserved":0,"allocated":7393280,"data_stored":3093754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002253"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644500+0000","last_change":"2024-02-18T00:30:46.913439+0000","last_active":"2024-02-18T00:31:16.644500+0000","last_peered":"2024-02-18T00:31:16.644500+0000","last_clean":"2024-02-18T00:31:16.644500+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:31:16.644500+0000","last_undegraded":"2024-02-18T00:31:16.644500+0000","last_fullsized":"2024-02-18T00:31:16.644500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.913340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.913340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645115+0000","last_change":"2024-02-18T00:31:07.942520+0000","last_active":"2024-02-18T00:31:16.645115+0000","last_peered":"2024-02-18T00:31:16.645115+0000","last_clean":"2024-02-18T00:31:16.645115+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:31:16.645115+0000","last_undegraded":"2024-02-18T00:31:16.645115+0000","last_fullsized":"2024-02-18T00:31:16.645115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.942435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.942435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977255+0000","last_change":"2024-02-18T00:31:02.922852+0000","last_active":"2024-02-18T00:31:16.977255+0000","last_peered":"2024-02-18T00:31:16.977255+0000","last_clean":"2024-02-18T00:31:16.977255+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:31:16.977255+0000","last_undegraded":"2024-02-18T00:31:16.977255+0000","last_fullsized":"2024-02-18T00:31:16.977255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.922771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.922771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":128,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644821+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:31:16.644821+0000","last_peered":"2024-02-18T00:31:16.644821+0000","last_clean":"2024-02-18T00:31:16.644821+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:31:16.644821+0000","last_undegraded":"2024-02-18T00:31:16.644821+0000","last_fullsized":"2024-02-18T00:31:16.644821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645333+0000","last_change":"2024-02-18T00:30:29.061462+0000","last_active":"2024-02-18T00:31:16.645333+0000","last_peered":"2024-02-18T00:31:16.645333+0000","last_clean":"2024-02-18T00:31:16.645333+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:31:16.645333+0000","last_undegraded":"2024-02-18T00:31:16.645333+0000","last_fullsized":"2024-02-18T00:31:16.645333+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.061390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.061390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645060+0000","last_change":"2024-02-18T00:30:50.991652+0000","last_active":"2024-02-18T00:31:16.645060+0000","last_peered":"2024-02-18T00:31:16.645060+0000","last_clean":"2024-02-18T00:31:16.645060+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:31:16.645060+0000","last_undegraded":"2024-02-18T00:31:16.645060+0000","last_fullsized":"2024-02-18T00:31:16.645060+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.991592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.991592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645331+0000","last_change":"2024-02-18T00:30:48.890743+0000","last_active":"2024-02-18T00:31:16.645331+0000","last_peered":"2024-02-18T00:31:16.645331+0000","last_clean":"2024-02-18T00:31:16.645331+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:31:16.645331+0000","last_undegraded":"2024-02-18T00:31:16.645331+0000","last_fullsized":"2024-02-18T00:31:16.645331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.890677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645284+0000","last_change":"2024-02-18T00:31:03.941281+0000","last_active":"2024-02-18T00:31:16.645284+0000","last_peered":"2024-02-18T00:31:16.645284+0000","last_clean":"2024-02-18T00:31:16.645284+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:31:16.645284+0000","last_undegraded":"2024-02-18T00:31:16.645284+0000","last_fullsized":"2024-02-18T00:31:16.645284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:03.941195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:03.941195+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976631+0000","last_change":"2024-02-18T00:30:41.063319+0000","last_active":"2024-02-18T00:31:16.976631+0000","last_peered":"2024-02-18T00:31:16.976631+0000","last_clean":"2024-02-18T00:31:16.976631+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:31:16.976631+0000","last_undegraded":"2024-02-18T00:31:16.976631+0000","last_fullsized":"2024-02-18T00:31:16.976631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.063240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:41.063240+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.063240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976614+0000","last_change":"2024-02-18T00:30:25.635340+0000","last_active":"2024-02-18T00:31:16.976614+0000","last_peered":"2024-02-18T00:31:16.976614+0000","last_clean":"2024-02-18T00:31:16.976614+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:31:16.976614+0000","last_undegraded":"2024-02-18T00:31:16.976614+0000","last_fullsized":"2024-02-18T00:31:16.976614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:25.635259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:25.635259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977679+0000","last_change":"2024-02-18T00:30:22.539011+0000","last_active":"2024-02-18T00:31:16.977679+0000","last_peered":"2024-02-18T00:31:16.977679+0000","last_clean":"2024-02-18T00:31:16.977679+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:31:16.977679+0000","last_undegraded":"2024-02-18T00:31:16.977679+0000","last_fullsized":"2024-02-18T00:31:16.977679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:22.538946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:22.538946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645647+0000","last_change":"2024-02-18T00:30:46.066618+0000","last_active":"2024-02-18T00:31:16.645647+0000","last_peered":"2024-02-18T00:31:16.645647+0000","last_clean":"2024-02-18T00:31:16.645647+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:31:16.645647+0000","last_undegraded":"2024-02-18T00:31:16.645647+0000","last_fullsized":"2024-02-18T00:31:16.645647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.066555+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976147+0000","last_change":"2024-02-18T00:30:30.776677+0000","last_active":"2024-02-18T00:31:16.976147+0000","last_peered":"2024-02-18T00:31:16.976147+0000","last_clean":"2024-02-18T00:31:16.976147+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:31:16.976147+0000","last_undegraded":"2024-02-18T00:31:16.976147+0000","last_fullsized":"2024-02-18T00:31:16.976147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.776611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.776611+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977353+0000","last_change":"2024-02-18T00:30:48.990813+0000","last_active":"2024-02-18T00:31:16.977353+0000","last_peered":"2024-02-18T00:31:16.977353+0000","last_clean":"2024-02-18T00:31:16.977353+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:31:16.977353+0000","last_undegraded":"2024-02-18T00:31:16.977353+0000","last_fullsized":"2024-02-18T00:31:16.977353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.990751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.990751+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977340+0000","last_change":"2024-02-18T00:30:26.654619+0000","last_active":"2024-02-18T00:31:16.977340+0000","last_peered":"2024-02-18T00:31:16.977340+0000","last_clean":"2024-02-18T00:31:16.977340+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:31:16.977340+0000","last_undegraded":"2024-02-18T00:31:16.977340+0000","last_fullsized":"2024-02-18T00:31:16.977340+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.654541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.654541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'22","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645497+0000","last_change":"2024-02-18T00:30:27.058513+0000","last_active":"2024-02-18T00:31:16.645497+0000","last_peered":"2024-02-18T00:31:16.645497+0000","last_clean":"2024-02-18T00:31:16.645497+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:31:16.645497+0000","last_undegraded":"2024-02-18T00:31:16.645497+0000","last_fullsized":"2024-02-18T00:31:16.645497+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:30:27.058416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.058416+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645307+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:31:16.645307+0000","last_peered":"2024-02-18T00:31:16.645307+0000","last_clean":"2024-02-18T00:31:16.645307+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:31:16.645307+0000","last_undegraded":"2024-02-18T00:31:16.645307+0000","last_fullsized":"2024-02-18T00:31:16.645307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645666+0000","last_change":"2024-02-18T00:30:33.047434+0000","last_active":"2024-02-18T00:31:16.645666+0000","last_peered":"2024-02-18T00:31:16.645666+0000","last_clean":"2024-02-18T00:31:16.645666+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:31:16.645666+0000","last_undegraded":"2024-02-18T00:31:16.645666+0000","last_fullsized":"2024-02-18T00:31:16.645666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.047327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.047327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644603+0000","last_change":"2024-02-18T00:30:25.803798+0000","last_active":"2024-02-18T00:31:16.644603+0000","last_peered":"2024-02-18T00:31:16.644603+0000","last_clean":"2024-02-18T00:31:16.644603+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:31:16.644603+0000","last_undegraded":"2024-02-18T00:31:16.644603+0000","last_fullsized":"2024-02-18T00:31:16.644603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:30:25.803723+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645057+0000","last_change":"2024-02-18T00:30:50.885765+0000","last_active":"2024-02-18T00:31:16.645057+0000","last_peered":"2024-02-18T00:31:16.645057+0000","last_clean":"2024-02-18T00:31:16.645057+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:31:16.645057+0000","last_undegraded":"2024-02-18T00:31:16.645057+0000","last_fullsized":"2024-02-18T00:31:16.645057+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:30:50.885651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.885651+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646046+0000","last_change":"2024-02-18T00:30:36.043298+0000","last_active":"2024-02-18T00:31:16.646046+0000","last_peered":"2024-02-18T00:31:16.646046+0000","last_clean":"2024-02-18T00:31:16.646046+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:31:16.646046+0000","last_undegraded":"2024-02-18T00:31:16.646046+0000","last_fullsized":"2024-02-18T00:31:16.646046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:36.043211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:36.043211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646172+0000","last_change":"2024-02-18T00:31:02.145322+0000","last_active":"2024-02-18T00:31:16.646172+0000","last_peered":"2024-02-18T00:31:16.646172+0000","last_clean":"2024-02-18T00:31:16.646172+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:31:16.646172+0000","last_undegraded":"2024-02-18T00:31:16.646172+0000","last_fullsized":"2024-02-18T00:31:16.646172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.145253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978470+0000","last_change":"2024-02-18T00:30:48.450965+0000","last_active":"2024-02-18T00:31:16.978470+0000","last_peered":"2024-02-18T00:31:16.978470+0000","last_clean":"2024-02-18T00:31:16.978470+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:31:16.978470+0000","last_undegraded":"2024-02-18T00:31:16.978470+0000","last_fullsized":"2024-02-18T00:31:16.978470+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.450877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.450877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976836+0000","last_change":"2024-02-18T00:30:23.427028+0000","last_active":"2024-02-18T00:31:16.976836+0000","last_peered":"2024-02-18T00:31:16.976836+0000","last_clean":"2024-02-18T00:31:16.976836+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:31:16.976836+0000","last_undegraded":"2024-02-18T00:31:16.976836+0000","last_fullsized":"2024-02-18T00:31:16.976836+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:30:23.426950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:23.426950+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645705+0000","last_change":"2024-02-18T00:30:30.321398+0000","last_active":"2024-02-18T00:31:16.645705+0000","last_peered":"2024-02-18T00:31:16.645705+0000","last_clean":"2024-02-18T00:31:16.645705+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:31:16.645705+0000","last_undegraded":"2024-02-18T00:31:16.645705+0000","last_fullsized":"2024-02-18T00:31:16.645705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.321330+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.240187+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:31:23.240187+0000","last_peered":"2024-02-18T00:31:23.240187+0000","last_clean":"2024-02-18T00:31:23.240187+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:31:23.240187+0000","last_undegraded":"2024-02-18T00:31:23.240187+0000","last_fullsized":"2024-02-18T00:31:23.240187+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644497+0000","last_change":"2024-02-18T00:30:19.726673+0000","last_active":"2024-02-18T00:31:16.644497+0000","last_peered":"2024-02-18T00:31:16.644497+0000","last_clean":"2024-02-18T00:31:16.644497+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:31:16.644497+0000","last_undegraded":"2024-02-18T00:31:16.644497+0000","last_fullsized":"2024-02-18T00:31:16.644497+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:19.726599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.726599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977675+0000","last_change":"2024-02-18T00:30:41.540091+0000","last_active":"2024-02-18T00:31:16.977675+0000","last_peered":"2024-02-18T00:31:16.977675+0000","last_clean":"2024-02-18T00:31:16.977675+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:31:16.977675+0000","last_undegraded":"2024-02-18T00:31:16.977675+0000","last_fullsized":"2024-02-18T00:31:16.977675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:41.540007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.540007+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977559+0000","last_change":"2024-02-18T00:30:38.560205+0000","last_active":"2024-02-18T00:31:16.977559+0000","last_peered":"2024-02-18T00:31:16.977559+0000","last_clean":"2024-02-18T00:31:16.977559+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:31:16.977559+0000","last_undegraded":"2024-02-18T00:31:16.977559+0000","last_fullsized":"2024-02-18T00:31:16.977559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:30:38.560085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.560085+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:30.356896+0000","last_change":"2024-02-18T00:31:30.356896+0000","last_active":"2024-02-18T00:31:30.356896+0000","last_peered":"2024-02-18T00:31:30.356896+0000","last_clean":"2024-02-18T00:31:30.356896+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:31:30.356896+0000","last_undegraded":"2024-02-18T00:31:30.356896+0000","last_fullsized":"2024-02-18T00:31:30.356896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:30.356811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.356811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976449+0000","last_change":"2024-02-18T00:31:06.040501+0000","last_active":"2024-02-18T00:31:16.976449+0000","last_peered":"2024-02-18T00:31:16.976449+0000","last_clean":"2024-02-18T00:31:16.976449+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:31:16.976449+0000","last_undegraded":"2024-02-18T00:31:16.976449+0000","last_fullsized":"2024-02-18T00:31:16.976449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.040425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.040425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976957+0000","last_change":"2024-02-18T00:30:38.895738+0000","last_active":"2024-02-18T00:31:16.976957+0000","last_peered":"2024-02-18T00:31:16.976957+0000","last_clean":"2024-02-18T00:31:16.976957+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:31:16.976957+0000","last_undegraded":"2024-02-18T00:31:16.976957+0000","last_fullsized":"2024-02-18T00:31:16.976957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:30:38.895655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.895655+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644795+0000","last_change":"2024-02-18T00:30:59.122281+0000","last_active":"2024-02-18T00:31:16.644795+0000","last_peered":"2024-02-18T00:31:16.644795+0000","last_clean":"2024-02-18T00:31:16.644795+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:31:16.644795+0000","last_undegraded":"2024-02-18T00:31:16.644795+0000","last_fullsized":"2024-02-18T00:31:16.644795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.122216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.122216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977001+0000","last_change":"2024-02-18T00:30:31.634376+0000","last_active":"2024-02-18T00:31:16.977001+0000","last_peered":"2024-02-18T00:31:16.977001+0000","last_clean":"2024-02-18T00:31:16.977001+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:31:16.977001+0000","last_undegraded":"2024-02-18T00:31:16.977001+0000","last_fullsized":"2024-02-18T00:31:16.977001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.634311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.634311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977942+0000","last_change":"2024-02-18T00:30:44.493983+0000","last_active":"2024-02-18T00:31:16.977942+0000","last_peered":"2024-02-18T00:31:16.977942+0000","last_clean":"2024-02-18T00:31:16.977942+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:31:16.977942+0000","last_undegraded":"2024-02-18T00:31:16.977942+0000","last_fullsized":"2024-02-18T00:31:16.977942+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.493871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.493871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644852+0000","last_change":"2024-02-18T00:30:43.819520+0000","last_active":"2024-02-18T00:31:16.644852+0000","last_peered":"2024-02-18T00:31:16.644852+0000","last_clean":"2024-02-18T00:31:16.644852+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:31:16.644852+0000","last_undegraded":"2024-02-18T00:31:16.644852+0000","last_fullsized":"2024-02-18T00:31:16.644852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:43.819434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.819434+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976441+0000","last_change":"2024-02-18T00:30:35.533752+0000","last_active":"2024-02-18T00:31:16.976441+0000","last_peered":"2024-02-18T00:31:16.976441+0000","last_clean":"2024-02-18T00:31:16.976441+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:31:16.976441+0000","last_undegraded":"2024-02-18T00:31:16.976441+0000","last_fullsized":"2024-02-18T00:31:16.976441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.533630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.533630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978149+0000","last_change":"2024-02-18T00:30:47.440072+0000","last_active":"2024-02-18T00:31:16.978149+0000","last_peered":"2024-02-18T00:31:16.978149+0000","last_clean":"2024-02-18T00:31:16.978149+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:31:16.978149+0000","last_undegraded":"2024-02-18T00:31:16.978149+0000","last_fullsized":"2024-02-18T00:31:16.978149+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.440005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.440005+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978064+0000","last_change":"2024-02-18T00:30:18.101882+0000","last_active":"2024-02-18T00:31:16.978064+0000","last_peered":"2024-02-18T00:31:16.978064+0000","last_clean":"2024-02-18T00:31:16.978064+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:31:16.978064+0000","last_undegraded":"2024-02-18T00:31:16.978064+0000","last_fullsized":"2024-02-18T00:31:16.978064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:18.101776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:18.101776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644451+0000","last_change":"2024-02-18T00:31:00.083411+0000","last_active":"2024-02-18T00:31:16.644451+0000","last_peered":"2024-02-18T00:31:16.644451+0000","last_clean":"2024-02-18T00:31:16.644451+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:31:16.644451+0000","last_undegraded":"2024-02-18T00:31:16.644451+0000","last_fullsized":"2024-02-18T00:31:16.644451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:31:00.083332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.083332+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644359+0000","last_change":"2024-02-18T00:31:06.048967+0000","last_active":"2024-02-18T00:31:16.644359+0000","last_peered":"2024-02-18T00:31:16.644359+0000","last_clean":"2024-02-18T00:31:16.644359+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:31:16.644359+0000","last_undegraded":"2024-02-18T00:31:16.644359+0000","last_fullsized":"2024-02-18T00:31:16.644359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.048889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.048889+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645358+0000","last_change":"2024-02-18T00:30:45.987723+0000","last_active":"2024-02-18T00:31:16.645358+0000","last_peered":"2024-02-18T00:31:16.645358+0000","last_clean":"2024-02-18T00:31:16.645358+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:31:16.645358+0000","last_undegraded":"2024-02-18T00:31:16.645358+0000","last_fullsized":"2024-02-18T00:31:16.645358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.987660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.987660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645853+0000","last_change":"2024-02-18T00:30:54.008260+0000","last_active":"2024-02-18T00:31:16.645853+0000","last_peered":"2024-02-18T00:31:16.645853+0000","last_clean":"2024-02-18T00:31:16.645853+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:31:16.645853+0000","last_undegraded":"2024-02-18T00:31:16.645853+0000","last_fullsized":"2024-02-18T00:31:16.645853+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.008196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.008196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645424+0000","last_change":"2024-02-18T00:30:17.985487+0000","last_active":"2024-02-18T00:31:16.645424+0000","last_peered":"2024-02-18T00:31:16.645424+0000","last_clean":"2024-02-18T00:31:16.645424+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:31:16.645424+0000","last_undegraded":"2024-02-18T00:31:16.645424+0000","last_fullsized":"2024-02-18T00:31:16.645424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:30:17.985420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:17.985420+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975810+0000","last_change":"2024-02-18T00:31:13.960170+0000","last_active":"2024-02-18T00:31:16.975810+0000","last_peered":"2024-02-18T00:31:16.975810+0000","last_clean":"2024-02-18T00:31:16.975810+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:31:16.975810+0000","last_undegraded":"2024-02-18T00:31:16.975810+0000","last_fullsized":"2024-02-18T00:31:16.975810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:13.960089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:13.960089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644853+0000","last_change":"2024-02-18T00:30:31.810682+0000","last_active":"2024-02-18T00:31:16.644853+0000","last_peered":"2024-02-18T00:31:16.644853+0000","last_clean":"2024-02-18T00:31:16.644853+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:31:16.644853+0000","last_undegraded":"2024-02-18T00:31:16.644853+0000","last_fullsized":"2024-02-18T00:31:16.644853+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.810604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.810604+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976874+0000","last_change":"2024-02-18T00:30:19.478866+0000","last_active":"2024-02-18T00:31:16.976874+0000","last_peered":"2024-02-18T00:31:16.976874+0000","last_clean":"2024-02-18T00:31:16.976874+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:31:16.976874+0000","last_undegraded":"2024-02-18T00:31:16.976874+0000","last_fullsized":"2024-02-18T00:31:16.976874+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:19.478758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.478758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645332+0000","last_change":"2024-02-18T00:30:51.994431+0000","last_active":"2024-02-18T00:31:16.645332+0000","last_peered":"2024-02-18T00:31:16.645332+0000","last_clean":"2024-02-18T00:31:16.645332+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:31:16.645332+0000","last_undegraded":"2024-02-18T00:31:16.645332+0000","last_fullsized":"2024-02-18T00:31:16.645332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:30:51.994334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.994334+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646060+0000","last_change":"2024-02-18T00:31:12.044114+0000","last_active":"2024-02-18T00:31:16.646060+0000","last_peered":"2024-02-18T00:31:16.646060+0000","last_clean":"2024-02-18T00:31:16.646060+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:31:16.646060+0000","last_undegraded":"2024-02-18T00:31:16.646060+0000","last_fullsized":"2024-02-18T00:31:16.646060+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:12.043994+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.283905+0000","last_change":"2024-02-18T00:30:35.823674+0000","last_active":"2024-02-18T00:31:23.283905+0000","last_peered":"2024-02-18T00:31:23.283905+0000","last_clean":"2024-02-18T00:31:23.283905+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:31:23.283905+0000","last_undegraded":"2024-02-18T00:31:23.283905+0000","last_fullsized":"2024-02-18T00:31:23.283905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:35.823589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.823589+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976606+0000","last_change":"2024-02-18T00:30:21.675011+0000","last_active":"2024-02-18T00:31:16.976606+0000","last_peered":"2024-02-18T00:31:16.976606+0000","last_clean":"2024-02-18T00:31:16.976606+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:31:16.976606+0000","last_undegraded":"2024-02-18T00:31:16.976606+0000","last_fullsized":"2024-02-18T00:31:16.976606+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.674903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:18.031433+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:31:18.031433+0000","last_peered":"2024-02-18T00:31:18.031433+0000","last_clean":"2024-02-18T00:31:18.031433+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:31:18.031433+0000","last_undegraded":"2024-02-18T00:31:18.031433+0000","last_fullsized":"2024-02-18T00:31:18.031433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976579+0000","last_change":"2024-02-18T00:30:29.772460+0000","last_active":"2024-02-18T00:31:16.976579+0000","last_peered":"2024-02-18T00:31:16.976579+0000","last_clean":"2024-02-18T00:31:16.976579+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:31:16.976579+0000","last_undegraded":"2024-02-18T00:31:16.976579+0000","last_fullsized":"2024-02-18T00:31:16.976579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.772394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.772394+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.284131+0000","last_change":"2024-02-18T00:31:01.012802+0000","last_active":"2024-02-18T00:31:23.284131+0000","last_peered":"2024-02-18T00:31:23.284131+0000","last_clean":"2024-02-18T00:31:23.284131+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:31:23.284131+0000","last_undegraded":"2024-02-18T00:31:23.284131+0000","last_fullsized":"2024-02-18T00:31:23.284131+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:01.012688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.012688+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977678+0000","last_change":"2024-02-18T00:31:05.071051+0000","last_active":"2024-02-18T00:31:16.977678+0000","last_peered":"2024-02-18T00:31:16.977678+0000","last_clean":"2024-02-18T00:31:16.977678+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:31:16.977678+0000","last_undegraded":"2024-02-18T00:31:16.977678+0000","last_fullsized":"2024-02-18T00:31:16.977678+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:05.070930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.070930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644791+0000","last_change":"2024-02-18T00:30:54.987124+0000","last_active":"2024-02-18T00:31:16.644791+0000","last_peered":"2024-02-18T00:31:16.644791+0000","last_clean":"2024-02-18T00:31:16.644791+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:31:16.644791+0000","last_undegraded":"2024-02-18T00:31:16.644791+0000","last_fullsized":"2024-02-18T00:31:16.644791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:30:54.987049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.987049+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977242+0000","last_change":"2024-02-18T00:30:33.564597+0000","last_active":"2024-02-18T00:31:16.977242+0000","last_peered":"2024-02-18T00:31:16.977242+0000","last_clean":"2024-02-18T00:31:16.977242+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:31:16.977242+0000","last_undegraded":"2024-02-18T00:31:16.977242+0000","last_fullsized":"2024-02-18T00:31:16.977242+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.564517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.564517+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.018547+0000","last_change":"2024-02-18T00:31:23.018547+0000","last_active":"2024-02-18T00:31:23.018547+0000","last_peered":"2024-02-18T00:31:23.018547+0000","last_clean":"2024-02-18T00:31:23.018547+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:31:23.018547+0000","last_undegraded":"2024-02-18T00:31:23.018547+0000","last_fullsized":"2024-02-18T00:31:23.018547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.018460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.018460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978161+0000","last_change":"2024-02-18T00:30:38.507742+0000","last_active":"2024-02-18T00:31:16.978161+0000","last_peered":"2024-02-18T00:31:16.978161+0000","last_clean":"2024-02-18T00:31:16.978161+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:31:16.978161+0000","last_undegraded":"2024-02-18T00:31:16.978161+0000","last_fullsized":"2024-02-18T00:31:16.978161+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.507637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.507637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:29.228915+0000","last_change":"2024-02-18T00:31:29.228915+0000","last_active":"2024-02-18T00:31:29.228915+0000","last_peered":"2024-02-18T00:31:29.228915+0000","last_clean":"2024-02-18T00:31:29.228915+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:31:29.228915+0000","last_undegraded":"2024-02-18T00:31:29.228915+0000","last_fullsized":"2024-02-18T00:31:29.228915+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:29.228799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:29.228799+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976542+0000","last_change":"2024-02-18T00:30:46.308529+0000","last_active":"2024-02-18T00:31:16.976542+0000","last_peered":"2024-02-18T00:31:16.976542+0000","last_clean":"2024-02-18T00:31:16.976542+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:31:16.976542+0000","last_undegraded":"2024-02-18T00:31:16.976542+0000","last_fullsized":"2024-02-18T00:31:16.976542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.308454+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645348+0000","last_change":"2024-02-18T00:30:52.923082+0000","last_active":"2024-02-18T00:31:16.645348+0000","last_peered":"2024-02-18T00:31:16.645348+0000","last_clean":"2024-02-18T00:31:16.645348+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:31:16.645348+0000","last_undegraded":"2024-02-18T00:31:16.645348+0000","last_fullsized":"2024-02-18T00:31:16.645348+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.923014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.923014+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975828+0000","last_change":"2024-02-18T00:30:32.364058+0000","last_active":"2024-02-18T00:31:16.975828+0000","last_peered":"2024-02-18T00:31:16.975828+0000","last_clean":"2024-02-18T00:31:16.975828+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:31:16.975828+0000","last_undegraded":"2024-02-18T00:31:16.975828+0000","last_fullsized":"2024-02-18T00:31:16.975828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:32.363988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.363988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:18.065909+0000","last_change":"2024-02-18T00:31:18.065868+0000","last_active":"2024-02-18T00:31:18.065909+0000","last_peered":"2024-02-18T00:31:18.065909+0000","last_clean":"2024-02-18T00:31:18.065909+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:31:18.065909+0000","last_undegraded":"2024-02-18T00:31:18.065909+0000","last_fullsized":"2024-02-18T00:31:18.065909+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:31:18.065789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.065789+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644695+0000","last_change":"2024-02-18T00:30:29.789176+0000","last_active":"2024-02-18T00:31:16.644695+0000","last_peered":"2024-02-18T00:31:16.644695+0000","last_clean":"2024-02-18T00:31:16.644695+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:31:16.644695+0000","last_undegraded":"2024-02-18T00:31:16.644695+0000","last_fullsized":"2024-02-18T00:31:16.644695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.789093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.789093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645951+0000","last_change":"2024-02-18T00:30:38.058675+0000","last_active":"2024-02-18T00:31:16.645951+0000","last_peered":"2024-02-18T00:31:16.645951+0000","last_clean":"2024-02-18T00:31:16.645951+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:31:16.645951+0000","last_undegraded":"2024-02-18T00:31:16.645951+0000","last_fullsized":"2024-02-18T00:31:16.645951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:30:38.058595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.058595+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645303+0000","last_change":"2024-02-18T00:30:54.996645+0000","last_active":"2024-02-18T00:31:16.645303+0000","last_peered":"2024-02-18T00:31:16.645303+0000","last_clean":"2024-02-18T00:31:16.645303+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:31:16.645303+0000","last_undegraded":"2024-02-18T00:31:16.645303+0000","last_fullsized":"2024-02-18T00:31:16.645303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:30:54.996545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.996545+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975812+0000","last_change":"2024-02-18T00:30:43.221614+0000","last_active":"2024-02-18T00:31:16.975812+0000","last_peered":"2024-02-18T00:31:16.975812+0000","last_clean":"2024-02-18T00:31:16.975812+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:31:16.975812+0000","last_undegraded":"2024-02-18T00:31:16.975812+0000","last_fullsized":"2024-02-18T00:31:16.975812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.221541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.221541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:20.132994+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:31:20.132994+0000","last_peered":"2024-02-18T00:31:20.132994+0000","last_clean":"2024-02-18T00:31:20.132994+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:31:20.132994+0000","last_undegraded":"2024-02-18T00:31:20.132994+0000","last_fullsized":"2024-02-18T00:31:20.132994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645731+0000","last_change":"2024-02-18T00:30:40.081002+0000","last_active":"2024-02-18T00:31:16.645731+0000","last_peered":"2024-02-18T00:31:16.645731+0000","last_clean":"2024-02-18T00:31:16.645731+0000","last_became_active":"2024-02-18T00:29:14.715019+0000","last_became_peered":"2024-02-18T00:29:14.715019+0000","last_unstale":"2024-02-18T00:31:16.645731+0000","last_undegraded":"2024-02-18T00:31:16.645731+0000","last_fullsized":"2024-02-18T00:31:16.645731+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.080928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.080928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'16","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976775+0000","last_change":"2024-02-18T00:30:41.649899+0000","last_active":"2024-02-18T00:31:16.976775+0000","last_peered":"2024-02-18T00:31:16.976775+0000","last_clean":"2024-02-18T00:31:16.976775+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:31:16.976775+0000","last_undegraded":"2024-02-18T00:31:16.976775+0000","last_fullsized":"2024-02-18T00:31:16.976775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:30:22.276257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:22.276257+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.283833+0000","last_change":"2024-02-18T00:30:46.767980+0000","last_active":"2024-02-18T00:31:23.283833+0000","last_peered":"2024-02-18T00:31:23.283833+0000","last_clean":"2024-02-18T00:31:23.283833+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:31:23.283833+0000","last_undegraded":"2024-02-18T00:31:23.283833+0000","last_fullsized":"2024-02-18T00:31:23.283833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:46.767898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.767898+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645110+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:31:16.645110+0000","last_peered":"2024-02-18T00:31:16.645110+0000","last_clean":"2024-02-18T00:31:16.645110+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:31:16.645110+0000","last_undegraded":"2024-02-18T00:31:16.645110+0000","last_fullsized":"2024-02-18T00:31:16.645110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977772+0000","last_change":"2024-02-18T00:31:08.046030+0000","last_active":"2024-02-18T00:31:16.977772+0000","last_peered":"2024-02-18T00:31:16.977772+0000","last_clean":"2024-02-18T00:31:16.977772+0000","last_became_active":"2024-02-18T00:29:14.715157+0000","last_became_peered":"2024-02-18T00:29:14.715157+0000","last_unstale":"2024-02-18T00:31:16.977772+0000","last_undegraded":"2024-02-18T00:31:16.977772+0000","last_fullsized":"2024-02-18T00:31:16.977772+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:31:08.045946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":133,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.283839+0000","last_change":"2024-02-18T00:30:58.030242+0000","last_active":"2024-02-18T00:31:23.283839+0000","last_peered":"2024-02-18T00:31:23.283839+0000","last_clean":"2024-02-18T00:31:23.283839+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:31:23.283839+0000","last_undegraded":"2024-02-18T00:31:23.283839+0000","last_fullsized":"2024-02-18T00:31:23.283839+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:30:58.030175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.030175+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976440+0000","last_change":"2024-02-18T00:30:47.731838+0000","last_active":"2024-02-18T00:31:16.976440+0000","last_peered":"2024-02-18T00:31:16.976440+0000","last_clean":"2024-02-18T00:31:16.976440+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:31:16.976440+0000","last_undegraded":"2024-02-18T00:31:16.976440+0000","last_fullsized":"2024-02-18T00:31:16.976440+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.731752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.731752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978469+0000","last_change":"2024-02-18T00:30:58.601762+0000","last_active":"2024-02-18T00:31:16.978469+0000","last_peered":"2024-02-18T00:31:16.978469+0000","last_clean":"2024-02-18T00:31:16.978469+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:31:16.978469+0000","last_undegraded":"2024-02-18T00:31:16.978469+0000","last_fullsized":"2024-02-18T00:31:16.978469+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.601694+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":175,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:28.919062+0000","last_change":"2024-02-18T00:30:32.271251+0000","last_active":"2024-02-18T00:31:28.919062+0000","last_peered":"2024-02-18T00:31:28.919062+0000","last_clean":"2024-02-18T00:31:28.919062+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:31:28.919062+0000","last_undegraded":"2024-02-18T00:31:28.919062+0000","last_fullsized":"2024-02-18T00:31:28.919062+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.271183+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.975870+0000","last_change":"2024-02-18T00:30:48.262064+0000","last_active":"2024-02-18T00:31:16.975870+0000","last_peered":"2024-02-18T00:31:16.975870+0000","last_clean":"2024-02-18T00:31:16.975870+0000","last_became_active":"2024-02-18T00:29:14.713812+0000","last_became_peered":"2024-02-18T00:29:14.713812+0000","last_unstale":"2024-02-18T00:31:16.975870+0000","last_undegraded":"2024-02-18T00:31:16.975870+0000","last_fullsized":"2024-02-18T00:31:16.975870+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.262001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.262001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":85,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646327+0000","last_change":"2024-02-18T00:30:41.040669+0000","last_active":"2024-02-18T00:31:16.646327+0000","last_peered":"2024-02-18T00:31:16.646327+0000","last_clean":"2024-02-18T00:31:16.646327+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:31:16.646327+0000","last_undegraded":"2024-02-18T00:31:16.646327+0000","last_fullsized":"2024-02-18T00:31:16.646327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:30:41.040595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.040595+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976455+0000","last_change":"2024-02-18T00:30:24.405804+0000","last_active":"2024-02-18T00:31:16.976455+0000","last_peered":"2024-02-18T00:31:16.976455+0000","last_clean":"2024-02-18T00:31:16.976455+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:31:16.976455+0000","last_undegraded":"2024-02-18T00:31:16.976455+0000","last_fullsized":"2024-02-18T00:31:16.976455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.405729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.405729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978001+0000","last_change":"2024-02-18T00:30:34.100267+0000","last_active":"2024-02-18T00:31:16.978001+0000","last_peered":"2024-02-18T00:31:16.978001+0000","last_clean":"2024-02-18T00:31:16.978001+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:31:16.978001+0000","last_undegraded":"2024-02-18T00:31:16.978001+0000","last_fullsized":"2024-02-18T00:31:16.978001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_clean_scrub_stamp":"2024-02-18T00:30:34.100174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644617+0000","last_change":"2024-02-18T00:30:56.034602+0000","last_active":"2024-02-18T00:31:16.644617+0000","last_peered":"2024-02-18T00:31:16.644617+0000","last_clean":"2024-02-18T00:31:16.644617+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:31:16.644617+0000","last_undegraded":"2024-02-18T00:31:16.644617+0000","last_fullsized":"2024-02-18T00:31:16.644617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:56.034493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:56.034493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977267+0000","last_change":"2024-02-18T00:30:40.055404+0000","last_active":"2024-02-18T00:31:16.977267+0000","last_peered":"2024-02-18T00:31:16.977267+0000","last_clean":"2024-02-18T00:31:16.977267+0000","last_became_active":"2024-02-18T00:29:14.716595+0000","last_became_peered":"2024-02-18T00:29:14.716595+0000","last_unstale":"2024-02-18T00:31:16.977267+0000","last_undegraded":"2024-02-18T00:31:16.977267+0000","last_fullsized":"2024-02-18T00:31:16.977267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.055253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.055253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976838+0000","last_change":"2024-02-18T00:30:46.504493+0000","last_active":"2024-02-18T00:31:16.976838+0000","last_peered":"2024-02-18T00:31:16.976838+0000","last_clean":"2024-02-18T00:31:16.976838+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:31:16.976838+0000","last_undegraded":"2024-02-18T00:31:16.976838+0000","last_fullsized":"2024-02-18T00:31:16.976838+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.504422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.504422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:25.298149+0000","last_change":"2024-02-18T00:31:25.298149+0000","last_active":"2024-02-18T00:31:25.298149+0000","last_peered":"2024-02-18T00:31:25.298149+0000","last_clean":"2024-02-18T00:31:25.298149+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:31:25.298149+0000","last_undegraded":"2024-02-18T00:31:25.298149+0000","last_fullsized":"2024-02-18T00:31:25.298149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:31:25.298062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:25.298062+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977464+0000","last_change":"2024-02-18T00:30:27.608046+0000","last_active":"2024-02-18T00:31:16.977464+0000","last_peered":"2024-02-18T00:31:16.977464+0000","last_clean":"2024-02-18T00:31:16.977464+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:31:16.977464+0000","last_undegraded":"2024-02-18T00:31:16.977464+0000","last_fullsized":"2024-02-18T00:31:16.977464+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:27.607981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.607981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645600+0000","last_change":"2024-02-18T00:31:00.091252+0000","last_active":"2024-02-18T00:31:16.645600+0000","last_peered":"2024-02-18T00:31:16.645600+0000","last_clean":"2024-02-18T00:31:16.645600+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:31:16.645600+0000","last_undegraded":"2024-02-18T00:31:16.645600+0000","last_fullsized":"2024-02-18T00:31:16.645600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:00.091169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.091169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976219+0000","last_change":"2024-02-18T00:30:20.623950+0000","last_active":"2024-02-18T00:31:16.976219+0000","last_peered":"2024-02-18T00:31:16.976219+0000","last_clean":"2024-02-18T00:31:16.976219+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:31:16.976219+0000","last_undegraded":"2024-02-18T00:31:16.976219+0000","last_fullsized":"2024-02-18T00:31:16.976219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.623829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.623829+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976764+0000","last_change":"2024-02-18T00:30:54.590835+0000","last_active":"2024-02-18T00:31:16.976764+0000","last_peered":"2024-02-18T00:31:16.976764+0000","last_clean":"2024-02-18T00:31:16.976764+0000","last_became_active":"2024-02-18T00:29:14.714355+0000","last_became_peered":"2024-02-18T00:29:14.714355+0000","last_unstale":"2024-02-18T00:31:16.976764+0000","last_undegraded":"2024-02-18T00:31:16.976764+0000","last_fullsized":"2024-02-18T00:31:16.976764+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.590771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.590771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976670+0000","last_change":"2024-02-18T00:31:01.883504+0000","last_active":"2024-02-18T00:31:16.976670+0000","last_peered":"2024-02-18T00:31:16.976670+0000","last_clean":"2024-02-18T00:31:16.976670+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:31:16.976670+0000","last_undegraded":"2024-02-18T00:31:16.976670+0000","last_fullsized":"2024-02-18T00:31:16.976670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:01.883424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.883424+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977907+0000","last_change":"2024-02-18T00:30:52.072719+0000","last_active":"2024-02-18T00:31:16.977907+0000","last_peered":"2024-02-18T00:31:16.977907+0000","last_clean":"2024-02-18T00:31:16.977907+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:31:16.977907+0000","last_undegraded":"2024-02-18T00:31:16.977907+0000","last_fullsized":"2024-02-18T00:31:16.977907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.072619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.072619+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645041+0000","last_change":"2024-02-18T00:31:11.020203+0000","last_active":"2024-02-18T00:31:16.645041+0000","last_peered":"2024-02-18T00:31:16.645041+0000","last_clean":"2024-02-18T00:31:16.645041+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:31:16.645041+0000","last_undegraded":"2024-02-18T00:31:16.645041+0000","last_fullsized":"2024-02-18T00:31:16.645041+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:11.020119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.020119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978589+0000","last_change":"2024-02-18T00:30:53.587885+0000","last_active":"2024-02-18T00:31:16.978589+0000","last_peered":"2024-02-18T00:31:16.978589+0000","last_clean":"2024-02-18T00:31:16.978589+0000","last_became_active":"2024-02-18T00:29:14.715756+0000","last_became_peered":"2024-02-18T00:29:14.715756+0000","last_unstale":"2024-02-18T00:31:16.978589+0000","last_undegraded":"2024-02-18T00:31:16.978589+0000","last_fullsized":"2024-02-18T00:31:16.978589+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:30:53.587796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:53.587796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976564+0000","last_change":"2024-02-18T00:30:48.688989+0000","last_active":"2024-02-18T00:31:16.976564+0000","last_peered":"2024-02-18T00:31:16.976564+0000","last_clean":"2024-02-18T00:31:16.976564+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:31:16.976564+0000","last_undegraded":"2024-02-18T00:31:16.976564+0000","last_fullsized":"2024-02-18T00:31:16.976564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:30:48.688901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.688901+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978389+0000","last_change":"2024-02-18T00:30:45.463697+0000","last_active":"2024-02-18T00:31:16.978389+0000","last_peered":"2024-02-18T00:31:16.978389+0000","last_clean":"2024-02-18T00:31:16.978389+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:31:16.978389+0000","last_undegraded":"2024-02-18T00:31:16.978389+0000","last_fullsized":"2024-02-18T00:31:16.978389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645664+0000","last_change":"2024-02-18T00:30:45.062422+0000","last_active":"2024-02-18T00:31:16.645664+0000","last_peered":"2024-02-18T00:31:16.645664+0000","last_clean":"2024-02-18T00:31:16.645664+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:31:16.645664+0000","last_undegraded":"2024-02-18T00:31:16.645664+0000","last_fullsized":"2024-02-18T00:31:16.645664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.062339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.062339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646272+0000","last_change":"2024-02-18T00:30:35.177065+0000","last_active":"2024-02-18T00:31:16.646272+0000","last_peered":"2024-02-18T00:31:16.646272+0000","last_clean":"2024-02-18T00:31:16.646272+0000","last_became_active":"2024-02-18T00:29:14.714616+0000","last_became_peered":"2024-02-18T00:29:14.714616+0000","last_unstale":"2024-02-18T00:31:16.646272+0000","last_undegraded":"2024-02-18T00:31:16.646272+0000","last_fullsized":"2024-02-18T00:31:16.646272+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.176991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.176991+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"64'30","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644349+0000","last_change":"2024-02-18T00:30:39.827787+0000","last_active":"2024-02-18T00:31:16.644349+0000","last_peered":"2024-02-18T00:31:16.644349+0000","last_clean":"2024-02-18T00:31:16.644349+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:31:16.644349+0000","last_undegraded":"2024-02-18T00:31:16.644349+0000","last_fullsized":"2024-02-18T00:31:16.644349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'30","last_scrub_stamp":"2024-02-18T00:30:39.827720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.827720+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645451+0000","last_change":"2024-02-18T00:30:26.099064+0000","last_active":"2024-02-18T00:31:16.645451+0000","last_peered":"2024-02-18T00:31:16.645451+0000","last_clean":"2024-02-18T00:31:16.645451+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:31:16.645451+0000","last_undegraded":"2024-02-18T00:31:16.645451+0000","last_fullsized":"2024-02-18T00:31:16.645451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.098982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.098982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977642+0000","last_change":"2024-02-18T00:30:44.565713+0000","last_active":"2024-02-18T00:31:16.977642+0000","last_peered":"2024-02-18T00:31:16.977642+0000","last_clean":"2024-02-18T00:31:16.977642+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:31:16.977642+0000","last_undegraded":"2024-02-18T00:31:16.977642+0000","last_fullsized":"2024-02-18T00:31:16.977642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.565626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.565626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977858+0000","last_change":"2024-02-18T00:30:37.538788+0000","last_active":"2024-02-18T00:31:16.977858+0000","last_peered":"2024-02-18T00:31:16.977858+0000","last_clean":"2024-02-18T00:31:16.977858+0000","last_became_active":"2024-02-18T00:29:14.714186+0000","last_became_peered":"2024-02-18T00:29:14.714186+0000","last_unstale":"2024-02-18T00:31:16.977858+0000","last_undegraded":"2024-02-18T00:31:16.977858+0000","last_fullsized":"2024-02-18T00:31:16.977858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:37.538700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:37.538700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645236+0000","last_change":"2024-02-18T00:30:21.092976+0000","last_active":"2024-02-18T00:31:16.645236+0000","last_peered":"2024-02-18T00:31:16.645236+0000","last_clean":"2024-02-18T00:31:16.645236+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:31:16.645236+0000","last_undegraded":"2024-02-18T00:31:16.645236+0000","last_fullsized":"2024-02-18T00:31:16.645236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:30:21.092908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.092908+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":77,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.283921+0000","last_change":"2024-02-18T00:30:22.716297+0000","last_active":"2024-02-18T00:31:23.283921+0000","last_peered":"2024-02-18T00:31:23.283921+0000","last_clean":"2024-02-18T00:31:23.283921+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:31:23.283921+0000","last_undegraded":"2024-02-18T00:31:23.283921+0000","last_fullsized":"2024-02-18T00:31:23.283921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:30:22.716226+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.644566+0000","last_change":"2024-02-18T00:30:40.855003+0000","last_active":"2024-02-18T00:31:16.644566+0000","last_peered":"2024-02-18T00:31:16.644566+0000","last_clean":"2024-02-18T00:31:16.644566+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:31:16.644566+0000","last_undegraded":"2024-02-18T00:31:16.644566+0000","last_fullsized":"2024-02-18T00:31:16.644566+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.854928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.854928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977743+0000","last_change":"2024-02-18T00:31:16.020640+0000","last_active":"2024-02-18T00:31:16.977743+0000","last_peered":"2024-02-18T00:31:16.977743+0000","last_clean":"2024-02-18T00:31:16.977743+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:31:16.977743+0000","last_undegraded":"2024-02-18T00:31:16.977743+0000","last_fullsized":"2024-02-18T00:31:16.977743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:16.020558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:16.020558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976891+0000","last_change":"2024-02-18T00:30:45.530383+0000","last_active":"2024-02-18T00:31:16.976891+0000","last_peered":"2024-02-18T00:31:16.976891+0000","last_clean":"2024-02-18T00:31:16.976891+0000","last_became_active":"2024-02-18T00:29:14.713770+0000","last_became_peered":"2024-02-18T00:29:14.713770+0000","last_unstale":"2024-02-18T00:31:16.976891+0000","last_undegraded":"2024-02-18T00:31:16.976891+0000","last_fullsized":"2024-02-18T00:31:16.976891+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.530294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.530294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646083+0000","last_change":"2024-02-18T00:30:32.029864+0000","last_active":"2024-02-18T00:31:16.646083+0000","last_peered":"2024-02-18T00:31:16.646083+0000","last_clean":"2024-02-18T00:31:16.646083+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:31:16.646083+0000","last_undegraded":"2024-02-18T00:31:16.646083+0000","last_fullsized":"2024-02-18T00:31:16.646083+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:32.029757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.029757+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.987881+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:31:23.987881+0000","last_peered":"2024-02-18T00:31:23.987881+0000","last_clean":"2024-02-18T00:31:23.987881+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:31:23.987881+0000","last_undegraded":"2024-02-18T00:31:23.987881+0000","last_fullsized":"2024-02-18T00:31:23.987881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.284113+0000","last_change":"2024-02-18T00:31:05.997541+0000","last_active":"2024-02-18T00:31:23.284113+0000","last_peered":"2024-02-18T00:31:23.284113+0000","last_clean":"2024-02-18T00:31:23.284113+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:31:23.284113+0000","last_undegraded":"2024-02-18T00:31:23.284113+0000","last_fullsized":"2024-02-18T00:31:23.284113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:05.997460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.997460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645143+0000","last_change":"2024-02-18T00:30:20.213263+0000","last_active":"2024-02-18T00:31:16.645143+0000","last_peered":"2024-02-18T00:31:16.645143+0000","last_clean":"2024-02-18T00:31:16.645143+0000","last_became_active":"2024-02-18T00:29:14.713732+0000","last_became_peered":"2024-02-18T00:29:14.713732+0000","last_unstale":"2024-02-18T00:31:16.645143+0000","last_undegraded":"2024-02-18T00:31:16.645143+0000","last_fullsized":"2024-02-18T00:31:16.645143+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.213170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977393+0000","last_change":"2024-02-18T00:30:45.055464+0000","last_active":"2024-02-18T00:31:16.977393+0000","last_peered":"2024-02-18T00:31:16.977393+0000","last_clean":"2024-02-18T00:31:16.977393+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:31:16.977393+0000","last_undegraded":"2024-02-18T00:31:16.977393+0000","last_fullsized":"2024-02-18T00:31:16.977393+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.055398+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645825+0000","last_change":"2024-02-18T00:30:44.103971+0000","last_active":"2024-02-18T00:31:16.645825+0000","last_peered":"2024-02-18T00:31:16.645825+0000","last_clean":"2024-02-18T00:31:16.645825+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:31:16.645825+0000","last_undegraded":"2024-02-18T00:31:16.645825+0000","last_fullsized":"2024-02-18T00:31:16.645825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.103899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.103899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.283890+0000","last_change":"2024-02-18T00:30:51.228973+0000","last_active":"2024-02-18T00:31:23.283890+0000","last_peered":"2024-02-18T00:31:23.283890+0000","last_clean":"2024-02-18T00:31:23.283890+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:31:23.283890+0000","last_undegraded":"2024-02-18T00:31:23.283890+0000","last_fullsized":"2024-02-18T00:31:23.283890+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:51.228902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.228902+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645285+0000","last_change":"2024-02-18T00:30:29.330196+0000","last_active":"2024-02-18T00:31:16.645285+0000","last_peered":"2024-02-18T00:31:16.645285+0000","last_clean":"2024-02-18T00:31:16.645285+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:31:16.645285+0000","last_undegraded":"2024-02-18T00:31:16.645285+0000","last_fullsized":"2024-02-18T00:31:16.645285+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.330124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.330124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":58,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645550+0000","last_change":"2024-02-18T00:31:11.002136+0000","last_active":"2024-02-18T00:31:16.645550+0000","last_peered":"2024-02-18T00:31:16.645550+0000","last_clean":"2024-02-18T00:31:16.645550+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:31:16.645550+0000","last_undegraded":"2024-02-18T00:31:16.645550+0000","last_fullsized":"2024-02-18T00:31:16.645550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:31:11.002039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.002039+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977085+0000","last_change":"2024-02-18T00:30:41.301486+0000","last_active":"2024-02-18T00:31:16.977085+0000","last_peered":"2024-02-18T00:31:16.977085+0000","last_clean":"2024-02-18T00:31:16.977085+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:31:16.977085+0000","last_undegraded":"2024-02-18T00:31:16.977085+0000","last_fullsized":"2024-02-18T00:31:16.977085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.301405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.301405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645652+0000","last_change":"2024-02-18T00:30:59.059018+0000","last_active":"2024-02-18T00:31:16.645652+0000","last_peered":"2024-02-18T00:31:16.645652+0000","last_clean":"2024-02-18T00:31:16.645652+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:31:16.645652+0000","last_undegraded":"2024-02-18T00:31:16.645652+0000","last_fullsized":"2024-02-18T00:31:16.645652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.058947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.058947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977236+0000","last_change":"2024-02-18T00:30:24.601515+0000","last_active":"2024-02-18T00:31:16.977236+0000","last_peered":"2024-02-18T00:31:16.977236+0000","last_clean":"2024-02-18T00:31:16.977236+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:31:16.977236+0000","last_undegraded":"2024-02-18T00:31:16.977236+0000","last_fullsized":"2024-02-18T00:31:16.977236+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.601442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.601442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.646032+0000","last_change":"2024-02-18T00:30:19.032163+0000","last_active":"2024-02-18T00:31:16.646032+0000","last_peered":"2024-02-18T00:31:16.646032+0000","last_clean":"2024-02-18T00:31:16.646032+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:31:16.646032+0000","last_undegraded":"2024-02-18T00:31:16.646032+0000","last_fullsized":"2024-02-18T00:31:16.646032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:19.032095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.032095+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976515+0000","last_change":"2024-02-18T00:30:43.541530+0000","last_active":"2024-02-18T00:31:16.976515+0000","last_peered":"2024-02-18T00:31:16.976515+0000","last_clean":"2024-02-18T00:31:16.976515+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:31:16.976515+0000","last_undegraded":"2024-02-18T00:31:16.976515+0000","last_fullsized":"2024-02-18T00:31:16.976515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976371+0000","last_change":"2024-02-18T00:30:45.273987+0000","last_active":"2024-02-18T00:31:16.976371+0000","last_peered":"2024-02-18T00:31:16.976371+0000","last_clean":"2024-02-18T00:31:16.976371+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:31:16.976371+0000","last_undegraded":"2024-02-18T00:31:16.976371+0000","last_fullsized":"2024-02-18T00:31:16.976371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.273920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.273920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976771+0000","last_change":"2024-02-18T00:30:38.317025+0000","last_active":"2024-02-18T00:31:16.976771+0000","last_peered":"2024-02-18T00:31:16.976771+0000","last_clean":"2024-02-18T00:31:16.976771+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:31:16.976771+0000","last_undegraded":"2024-02-18T00:31:16.976771+0000","last_fullsized":"2024-02-18T00:31:16.976771+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.316960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.316960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.978338+0000","last_change":"2024-02-18T00:30:29.539942+0000","last_active":"2024-02-18T00:31:16.978338+0000","last_peered":"2024-02-18T00:31:16.978338+0000","last_clean":"2024-02-18T00:31:16.978338+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:31:16.978338+0000","last_undegraded":"2024-02-18T00:31:16.978338+0000","last_fullsized":"2024-02-18T00:31:16.978338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:30:29.539874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.539874+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976013+0000","last_change":"2024-02-18T00:30:50.764741+0000","last_active":"2024-02-18T00:31:16.976013+0000","last_peered":"2024-02-18T00:31:16.976013+0000","last_clean":"2024-02-18T00:31:16.976013+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:31:16.976013+0000","last_undegraded":"2024-02-18T00:31:16.976013+0000","last_fullsized":"2024-02-18T00:31:16.976013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.764665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.764665+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976905+0000","last_change":"2024-02-18T00:30:39.299351+0000","last_active":"2024-02-18T00:31:16.976905+0000","last_peered":"2024-02-18T00:31:16.976905+0000","last_clean":"2024-02-18T00:31:16.976905+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:31:16.976905+0000","last_undegraded":"2024-02-18T00:31:16.976905+0000","last_fullsized":"2024-02-18T00:31:16.976905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:39.299285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.299285+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.645980+0000","last_change":"2024-02-18T00:30:48.980366+0000","last_active":"2024-02-18T00:31:16.645980+0000","last_peered":"2024-02-18T00:31:16.645980+0000","last_clean":"2024-02-18T00:31:16.645980+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:31:16.645980+0000","last_undegraded":"2024-02-18T00:31:16.645980+0000","last_fullsized":"2024-02-18T00:31:16.645980+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.980292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.980292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976654+0000","last_change":"2024-02-18T00:31:07.953329+0000","last_active":"2024-02-18T00:31:16.976654+0000","last_peered":"2024-02-18T00:31:16.976654+0000","last_clean":"2024-02-18T00:31:16.976654+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:31:16.976654+0000","last_undegraded":"2024-02-18T00:31:16.976654+0000","last_fullsized":"2024-02-18T00:31:16.976654+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:07.953265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.953265+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.977591+0000","last_change":"2024-02-18T00:30:30.045287+0000","last_active":"2024-02-18T00:31:16.977591+0000","last_peered":"2024-02-18T00:31:16.977591+0000","last_clean":"2024-02-18T00:31:16.977591+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:31:16.977591+0000","last_undegraded":"2024-02-18T00:31:16.977591+0000","last_fullsized":"2024-02-18T00:31:16.977591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.045217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.045217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976603+0000","last_change":"2024-02-18T00:30:47.275395+0000","last_active":"2024-02-18T00:31:16.976603+0000","last_peered":"2024-02-18T00:31:16.976603+0000","last_clean":"2024-02-18T00:31:16.976603+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:31:16.976603+0000","last_undegraded":"2024-02-18T00:31:16.976603+0000","last_fullsized":"2024-02-18T00:31:16.976603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.275328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.275328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-18T00:31:16.976356+0000","last_change":"2024-02-18T00:31:07.029773+0000","last_active":"2024-02-18T00:31:16.976356+0000","last_peered":"2024-02-18T00:31:16.976356+0000","last_clean":"2024-02-18T00:31:16.976356+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:31:16.976356+0000","last_undegraded":"2024-02-18T00:31:16.976356+0000","last_fullsized":"2024-02-18T00:31:16.976356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:31:07.029677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.029677+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-18T00:31:23.284113+0000","last_change":"2024-02-18T00:30:49.588342+0000","last_active":"2024-02-18T00:31:23.284113+0000","last_peered":"2024-02-18T00:31:23.284113+0000","last_clean":"2024-02-18T00:31:23.284113+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:31:23.284113+0000","last_undegraded":"2024-02-18T00:31:23.284113+0000","last_fullsized":"2024-02-18T00:31:23.284113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:49.588269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:49.588269+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":595,"ondisk_log_size":595,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":127,"num_read_kb":122,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266719,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":929792,"data_stored":387426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.505,"15min":0.502},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.65200000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84899999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.88,"15min":0.88},"last":0.621}]},{"osd":2,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.34000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.39400000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.436},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.39500000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.41899999999999998}]}]},{"osd":6,"up_from":47,"seq":201863462949,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":909312,"data_stored":386261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.505,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.627}]},{"osd":1,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51200000000000001,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.49099999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55500000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.075,"15min":1.075},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447128,"statfs":{"total":95995035648,"available":95689859072,"internally_reserved":0,"allocated":950272,"data_stored":387084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.35999999999999999,"15min":0.311},"min":{"1min":0.29299999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.32200000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.53900000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.373},"min":{"1min":0.27700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47699999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:30:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59499999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.47899999999999998,"15min":0.48699999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.55600000000000005}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.83699999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.439,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82499999999999996,"5min":1.278,"15min":1.278},"last":0.69499999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79400000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.86299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":925696,"data_stored":387067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.48599999999999999,"15min":0.59799999999999998},"min":{"1min":0.23300000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.78000000000000003,"5min":6.7720000000000002,"15min":6.7720000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38100000000000001,"15min":0.34699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.623,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.27700000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.23799999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.308},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.29699999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.42999999999999999,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.47999999999999998,"15min":0.501},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.373}]},{"osd":4,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84899999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.73799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60699999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.621}]},{"osd":7,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.65000000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":905216,"data_stored":386338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39500000000000002,"15min":0.374},"min":{"1min":0.26100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.40400000000000003,"15min":0.378},"min":{"1min":0.29699999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.51600000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.38600000000000001},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55200000000000005,"5min":2.306,"15min":2.306},"last":0.495},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.42299999999999999,"15min":0.40100000000000002},"min":{"1min":0.27400000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58899999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.50700000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:30:35 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46999999999999997,"15min":0.46000000000000002},"min":{"1min":0.30499999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.34999999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.65700000000000003,"5min":1.675,"15min":1.675},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91200000000000003,"5min":1.173,"15min":1.173},"last":0.71799999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81399999999999995,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61199999999999999,"15min":0.624},"min":{"1min":0.46000000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.755}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.66500000000000004,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.753,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72399999999999998,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89300000000000002,"5min":1.242,"15min":1.242},"last":0.66300000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149747,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298016,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447136,"statfs":{"total":95995035648,"available":95689867264,"internally_reserved":0,"allocated":942080,"data_stored":386736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.38400000000000001,"15min":0.39400000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.58999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.53400000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.48699999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57599999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.25600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.58299999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:30:31 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.436,"15min":0.438},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.56000000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.42899999999999999,"15min":0.432},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59499999999999997,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:31:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77900000000000003,"5min":1.083,"15min":1.083},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.46899999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.441},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96599999999999997,"5min":1.139,"15min":1.139},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82299999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86199999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.629},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59999999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986222,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":929792,"data_stored":387221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":2.339,"15min":2.339},"last":0.5},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":1.288,"15min":1.288},"last":0.54400000000000004}]},{"osd":1,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55900000000000005,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65200000000000002,"5min":1.204,"15min":1.204},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70499999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.55600000000000005}]},{"osd":2,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":1.48,"15min":1.48},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.47699999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58099999999999996,"15min":0.60499999999999998},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69399999999999995,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.71699999999999997,"5min":1.133,"15min":1.133},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.38,"15min":0.35999999999999999},"min":{"1min":0.33300000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.443}]},{"osd":6,"last update":"Sun Feb 18 00:30:46 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.45600000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58499999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724585,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":901120,"data_stored":385621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.439}]},{"osd":1,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":1.306,"15min":1.306},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":1.272,"15min":1.272},"last":0.49299999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78800000000000003,"5min":1.173,"15min":1.173},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.502}]},{"osd":3,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":1.238,"15min":1.238},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72099999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.52500000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.378},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.315}]},{"osd":6,"last update":"Sun Feb 18 00:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.40200000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:31:34.931 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-18T00:31:34.931 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-18T00:31:34.931 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-18T00:31:34.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-18T00:31:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:35.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:33.697882+0000 osd.2 (osd.2) 33 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:33.699548+0000 osd.2 (osd.2) 34 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:33.745375+0000 mgr.y (mgr.24454) 93 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:34.068742+0000 mgr.y (mgr.24454) 94 : audit [DBG] from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:34.641777+0000 mon.a (mon.0) 748 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:33.697882+0000 osd.2 (osd.2) 33 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:33.699548+0000 osd.2 (osd.2) 34 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:33.745375+0000 mgr.y (mgr.24454) 93 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:34.068742+0000 mgr.y (mgr.24454) 94 : audit [DBG] from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:35.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:34.641777+0000 mon.a (mon.0) 748 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-18T00:31:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:33.697882+0000 osd.2 (osd.2) 33 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:31:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:33.699548+0000 osd.2 (osd.2) 34 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:31:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:33.745375+0000 mgr.y (mgr.24454) 93 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:34.068742+0000 mgr.y (mgr.24454) 94 : audit [DBG] from='client.24602 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:34.641777+0000 mon.a (mon.0) 748 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-18T00:31:36.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:35.746174+0000 mgr.y (mgr.24454) 95 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:36.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:36.501607+0000 osd.7 (osd.7) 43 : cluster [DBG] 2.1d deep-scrub starts 2024-02-18T00:31:36.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:36.502895+0000 osd.7 (osd.7) 44 : cluster [DBG] 2.1d deep-scrub ok 2024-02-18T00:31:36.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:35.746174+0000 mgr.y (mgr.24454) 95 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:36.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:36.501607+0000 osd.7 (osd.7) 43 : cluster [DBG] 2.1d deep-scrub starts 2024-02-18T00:31:36.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:36.502895+0000 osd.7 (osd.7) 44 : cluster [DBG] 2.1d deep-scrub ok 2024-02-18T00:31:37.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 -- 172.21.15.59:0/3115812034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f8cb0 msgr2=0x7f59740f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:37.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 --2- 172.21.15.59:0/3115812034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f8cb0 0x7f59740f9090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5970004870 tx=0x7f5970009cb0).stop 2024-02-18T00:31:37.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 -- 172.21.15.59:0/3115812034 shutdown_connections 2024-02-18T00:31:37.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 --2- 172.21.15.59:0/3115812034 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5974102210 0x7f5974104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 --2- 172.21.15.59:0/3115812034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f8cb0 0x7f59740f9090 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 --2- 172.21.15.59:0/3115812034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59740f9610 0x7f5974101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 -- 172.21.15.59:0/3115812034 >> 172.21.15.59:0/3115812034 conn(0x7f59740f2820 msgr2=0x7f59740f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:37.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 -- 172.21.15.59:0/3115812034 shutdown_connections 2024-02-18T00:31:37.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.201+0000 7f597c406700 1 -- 172.21.15.59:0/3115812034 wait complete. 2024-02-18T00:31:37.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 librados: starting msgr at 2024-02-18T00:31:37.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 librados: starting objecter 2024-02-18T00:31:37.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 Processor -- start 2024-02-18T00:31:37.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 -- start start 2024-02-18T00:31:37.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 librados: setting wanted keys 2024-02-18T00:31:37.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 librados: calling monclient init 2024-02-18T00:31:37.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59740f8cb0 0x7f597410e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f9610 0x7f597410e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974102210 0x7f5974108340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5974108830 con 0x7f59740f9610 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59741089a0 con 0x7f5974102210 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597c406700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5974108b60 con 0x7f59740f8cb0 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597a1a2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59740f8cb0 0x7f597410e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f59799a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f9610 0x7f597410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597a1a2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59740f8cb0 0x7f597410e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59360/0 (socket says 172.21.15.59:59360) 2024-02-18T00:31:37.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597a1a2700 1 -- 172.21.15.59:0/3254837874 learned_addr learned my addr 172.21.15.59:0/3254837874 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:37.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597a9a3700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974102210 0x7f5974108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:37.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f59799a1700 1 -- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59740f8cb0 msgr2=0x7f597410e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:37.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f59799a1700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59740f8cb0 0x7f597410e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f59799a1700 1 -- 172.21.15.59:0/3254837874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974102210 msgr2=0x7f5974108340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:37.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f59799a1700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974102210 0x7f5974108340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f59799a1700 1 -- 172.21.15.59:0/3254837874 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0097c0 con 0x7f59740f9610 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.202+0000 7f597a1a2700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59740f8cb0 0x7f597410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f59799a1700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f9610 0x7f597410e690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f5964002780 tx=0x7f5964009e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f596b7fe700 10 client.?.objecter ms_handle_connect 0x7f59740f9610 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f596b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f596b7fe700 1 -- 172.21.15.59:0/3254837874 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964007b40 con 0x7f59740f9610 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f596b7fe700 1 -- 172.21.15.59:0/3254837874 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f596400f040 con 0x7f59740f9610 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5970005040 con 0x7f59740f9610 2024-02-18T00:31:37.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f596b7fe700 1 -- 172.21.15.59:0/3254837874 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964014850 con 0x7f59740f9610 2024-02-18T00:31:37.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f597c406700 10 client.14703.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:37.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59741091a0 con 0x7f59740f9610 2024-02-18T00:31:37.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.203+0000 7f597c406700 1 librados: init done 2024-02-18T00:31:37.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.204+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597405fa60 con 0x7f59740f9610 2024-02-18T00:31:37.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.204+0000 7f596b7fe700 1 -- 172.21.15.59:0/3254837874 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5964007ca0 con 0x7f59740f9610 2024-02-18T00:31:37.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.204+0000 7f596b7fe700 1 --2- 172.21.15.59:0/3254837874 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5960062c90 0x7f5960065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:37.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f596b7fe700 1 -- 172.21.15.59:0/3254837874 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f596407c9c0 con 0x7f59740f9610 2024-02-18T00:31:37.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f597a1a2700 1 --2- 172.21.15.59:0/3254837874 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5960062c90 0x7f5960065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:37.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f596b7fe700 10 client.14703.objecter ms_dispatch 0x7f5974001550 osd_map(90..90 src has 1..90) v4 2024-02-18T00:31:37.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f596b7fe700 3 client.14703.objecter handle_osd_map got epochs [90,90] > 0 2024-02-18T00:31:37.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f596b7fe700 3 client.14703.objecter handle_osd_map decoding full epoch 90 2024-02-18T00:31:37.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f596b7fe700 20 client.14703.objecter dump_active .. 0 homeless 2024-02-18T00:31:37.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f596b7fe700 10 client.14703.objecter ms_handle_connect 0x7f5960062c90 2024-02-18T00:31:37.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.205+0000 7f597a1a2700 1 --2- 172.21.15.59:0/3254837874 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5960062c90 0x7f5960065150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f59700043d0 tx=0x7f597000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:37.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.209+0000 7f596b7fe700 1 -- 172.21.15.59:0/3254837874 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5964050630 con 0x7f59740f9610 2024-02-18T00:31:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:35.746174+0000 mgr.y (mgr.24454) 95 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:36.501607+0000 osd.7 (osd.7) 43 : cluster [DBG] 2.1d deep-scrub starts 2024-02-18T00:31:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:36.502895+0000 osd.7 (osd.7) 44 : cluster [DBG] 2.1d deep-scrub ok 2024-02-18T00:31:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.421+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5974107290 con 0x7f59740f9610 2024-02-18T00:31:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.421+0000 7f596b7fe700 1 -- 172.21.15.59:0/3254837874 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f5964050220 con 0x7f59740f9610 2024-02-18T00:31:37.423 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 10 librados: watch_flush enter 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 10 librados: watch_flush exit 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 20 client.14703.objecter shutdown clearing up homeless session... 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 10 client.14703.objecter successfully canceled tick 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5960062c90 msgr2=0x7f5960065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 --2- 172.21.15.59:0/3254837874 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5960062c90 0x7f5960065150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f59700043d0 tx=0x7f597000b040).stop 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f9610 msgr2=0x7f597410e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f9610 0x7f597410e690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f5964002780 tx=0x7f5964009e40).stop 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 shutdown_connections 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 --2- 172.21.15.59:0/3254837874 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5960062c90 0x7f5960065150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5974102210 0x7f5974108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59740f8cb0 0x7f597410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 --2- 172.21.15.59:0/3254837874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59740f9610 0x7f597410e690 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 >> 172.21.15.59:0/3254837874 conn(0x7f59740f2820 msgr2=0x7f59740f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 shutdown_connections 2024-02-18T00:31:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 -- 172.21.15.59:0/3254837874 wait complete. 2024-02-18T00:31:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:37.423+0000 7f597c406700 1 librados: shutdown 2024-02-18T00:31:38.093 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:36.139629+0000 osd.0 (osd.0) 43 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:31:38.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:36.140625+0000 osd.0 (osd.0) 44 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:31:38.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:37.422637+0000 mon.a (mon.0) 749 : audit [DBG] from='client.? 172.21.15.59:0/3254837874' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T00:31:38.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:36.139629+0000 osd.0 (osd.0) 43 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:31:38.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:36.140625+0000 osd.0 (osd.0) 44 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:31:38.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:37.422637+0000 mon.a (mon.0) 749 : audit [DBG] from='client.? 172.21.15.59:0/3254837874' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T00:31:38.114 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-18T00:31:38.114 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-18T00:31:38.114 INFO:tasks.cephadm:Setup complete, yielding 2024-02-18T00:31:38.115 INFO:teuthology.run_tasks:Running task thrashosds... 2024-02-18T00:31:38.126 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-02-18T00:31:38.127 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'pacific-release'}, 'ceph': {'conf': {'client': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mgrc': 20, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon warn on pool no app': False, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'crush_tunables': 'jewel', 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', 'flag\\(s\\) set', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(PG_', '\\(OSD_', {'mons down': None}, 'reached quota', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)'], 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'pacific-release', 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1'}} 2024-02-18T00:31:38.127 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-02-18T00:31:38.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-02-18T00:31:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:36.139629+0000 osd.0 (osd.0) 43 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:31:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:36.140625+0000 osd.0 (osd.0) 44 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:31:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:37.422637+0000 mon.a (mon.0) 749 : audit [DBG] from='client.? 172.21.15.59:0/3254837874' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T00:31:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:37.149057+0000 osd.0 (osd.0) 45 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:37.150212+0000 osd.0 (osd.0) 46 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:37.746514+0000 mgr.y (mgr.24454) 96 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:37.935936+0000 osd.1 (osd.1) 45 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:37.936891+0000 osd.1 (osd.1) 46 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:37.149057+0000 osd.0 (osd.0) 45 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:37.150212+0000 osd.0 (osd.0) 46 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:37.746514+0000 mgr.y (mgr.24454) 96 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:38.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:37.935936+0000 osd.1 (osd.1) 45 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:31:38.964 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:37.936891+0000 osd.1 (osd.1) 46 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:31:39.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:37.149057+0000 osd.0 (osd.0) 45 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:31:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:37.150212+0000 osd.0 (osd.0) 46 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:31:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:37.746514+0000 mgr.y (mgr.24454) 96 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:37.935936+0000 osd.1 (osd.1) 45 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:31:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:37.936891+0000 osd.1 (osd.1) 46 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:31:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:38.265450+0000 osd.6 (osd.6) 33 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:31:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:38.266555+0000 osd.6 (osd.6) 34 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:31:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:38.976719+0000 osd.1 (osd.1) 47 : cluster [DBG] 2.6 deep-scrub starts 2024-02-18T00:31:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:38.977838+0000 osd.1 (osd.1) 48 : cluster [DBG] 2.6 deep-scrub ok 2024-02-18T00:31:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:38.265450+0000 osd.6 (osd.6) 33 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:31:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:38.266555+0000 osd.6 (osd.6) 34 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:31:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:38.976719+0000 osd.1 (osd.1) 47 : cluster [DBG] 2.6 deep-scrub starts 2024-02-18T00:31:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:38.977838+0000 osd.1 (osd.1) 48 : cluster [DBG] 2.6 deep-scrub ok 2024-02-18T00:31:40.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:38.265450+0000 osd.6 (osd.6) 33 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:31:40.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:38.266555+0000 osd.6 (osd.6) 34 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:31:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:38.976719+0000 osd.1 (osd.1) 47 : cluster [DBG] 2.6 deep-scrub starts 2024-02-18T00:31:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:38.977838+0000 osd.1 (osd.1) 48 : cluster [DBG] 2.6 deep-scrub ok 2024-02-18T00:31:40.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.632+0000 7f95e7c88700 1 -- 172.21.15.59:0/3339556623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 msgr2=0x7f95e00f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.632+0000 7f95e7c88700 1 --2- 172.21.15.59:0/3339556623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 0x7f95e00f6e80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f95dc004870 tx=0x7f95dc009cb0).stop 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.632+0000 7f95e7c88700 1 -- 172.21.15.59:0/3339556623 shutdown_connections 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.632+0000 7f95e7c88700 1 --2- 172.21.15.59:0/3339556623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95e0104600 0x7f95e01069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.632+0000 7f95e7c88700 1 --2- 172.21.15.59:0/3339556623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 0x7f95e00f6e80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.632+0000 7f95e7c88700 1 --2- 172.21.15.59:0/3339556623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e00f7370 0x7f95e0104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.632+0000 7f95e7c88700 1 -- 172.21.15.59:0/3339556623 >> 172.21.15.59:0/3339556623 conn(0x7f95e00f2820 msgr2=0x7f95e00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 -- 172.21.15.59:0/3339556623 shutdown_connections 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 -- 172.21.15.59:0/3339556623 wait complete. 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 librados: starting msgr at 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 librados: starting objecter 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 Processor -- start 2024-02-18T00:31:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 -- start start 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 librados: setting wanted keys 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 librados: calling monclient init 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 0x7f95e01103a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e00f7370 0x7f95e0110890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95e0104600 0x7f95e010a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e010aa30 con 0x7f95e00f6aa0 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e010aba0 con 0x7f95e0104600 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.633+0000 7f95e7c88700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e010ad60 con 0x7f95e00f7370 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5a24700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 0x7f95e01103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5223700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e00f7370 0x7f95e0110890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5a24700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 0x7f95e01103a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49206/0 (socket says 172.21.15.59:49206) 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5a24700 1 -- 172.21.15.59:0/2848138888 learned_addr learned my addr 172.21.15.59:0/2848138888 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:40.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e6225700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95e0104600 0x7f95e010a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5223700 1 -- 172.21.15.59:0/2848138888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95e0104600 msgr2=0x7f95e010a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5223700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95e0104600 0x7f95e010a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5223700 1 -- 172.21.15.59:0/2848138888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 msgr2=0x7f95e01103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5223700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 0x7f95e01103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5223700 1 -- 172.21.15.59:0/2848138888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0005040 con 0x7f95e00f7370 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e5223700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e00f7370 0x7f95e0110890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f95d0004920 tx=0x7f95d0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95d6ffd700 10 client.?.objecter ms_handle_connect 0x7f95e00f7370 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95d6ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95d6ffd700 1 -- 172.21.15.59:0/2848138888 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0007a40 con 0x7f95e00f7370 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95d6ffd700 1 -- 172.21.15.59:0/2848138888 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d000f040 con 0x7f95e00f7370 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95dc005040 con 0x7f95e00f7370 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95d6ffd700 1 -- 172.21.15.59:0/2848138888 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0014a70 con 0x7f95e00f7370 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e7c88700 10 client.24614.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e010b3a0 con 0x7f95e00f7370 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.634+0000 7f95e7c88700 1 librados: init done 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.635+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e005fa60 con 0x7f95e00f7370 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95d6ffd700 1 -- 172.21.15.59:0/2848138888 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f95d0010530 con 0x7f95e00f7370 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95d6ffd700 1 --2- 172.21.15.59:0/2848138888 >> v2:172.21.15.59:6800/3721891797 conn(0x7f95cc062c90 0x7f95cc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95e5a24700 1 --2- 172.21.15.59:0/2848138888 >> v2:172.21.15.59:6800/3721891797 conn(0x7f95cc062c90 0x7f95cc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95d6ffd700 1 -- 172.21.15.59:0/2848138888 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f95d007c5b0 con 0x7f95e00f7370 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95d6ffd700 10 client.24614.objecter ms_dispatch 0x7f95e0001550 osd_map(90..90 src has 1..90) v4 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95d6ffd700 3 client.24614.objecter handle_osd_map got epochs [90,90] > 0 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95d6ffd700 3 client.24614.objecter handle_osd_map decoding full epoch 90 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.636+0000 7f95d6ffd700 20 client.24614.objecter dump_active .. 0 homeless 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.637+0000 7f95e5a24700 1 --2- 172.21.15.59:0/2848138888 >> v2:172.21.15.59:6800/3721891797 conn(0x7f95cc062c90 0x7f95cc065150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f95dc009a00 tx=0x7f95dc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.637+0000 7f95d6ffd700 10 client.24614.objecter ms_handle_connect 0x7f95cc062c90 2024-02-18T00:31:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.640+0000 7f95d6ffd700 1 -- 172.21.15.59:0/2848138888 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f95d0049f60 con 0x7f95e00f7370 2024-02-18T00:31:40.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.814+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) v1 -- 0x7f95e0109420 con 0x7f95e00f7370 2024-02-18T00:31:40.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.818+0000 7f95d6ffd700 1 -- 172.21.15.59:0/2848138888 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v27)=0 v27) v1 ==== 136+0+0 (secure 0 0 0) 0x7f95d00501c0 con 0x7f95e00f7370 2024-02-18T00:31:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.820+0000 7f95e7c88700 10 librados: watch_flush enter 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.820+0000 7f95e7c88700 10 librados: watch_flush exit 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.820+0000 7f95e7c88700 20 client.24614.objecter shutdown clearing up homeless session... 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.820+0000 7f95e7c88700 10 client.24614.objecter successfully canceled tick 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 >> v2:172.21.15.59:6800/3721891797 conn(0x7f95cc062c90 msgr2=0x7f95cc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 --2- 172.21.15.59:0/2848138888 >> v2:172.21.15.59:6800/3721891797 conn(0x7f95cc062c90 0x7f95cc065150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f95dc009a00 tx=0x7f95dc00b040).stop 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e00f7370 msgr2=0x7f95e0110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e00f7370 0x7f95e0110890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f95d0004920 tx=0x7f95d0009cb0).stop 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 shutdown_connections 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 --2- 172.21.15.59:0/2848138888 >> v2:172.21.15.59:6800/3721891797 conn(0x7f95cc062c90 0x7f95cc065150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95e0104600 0x7f95e010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e00f6aa0 0x7f95e01103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 --2- 172.21.15.59:0/2848138888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95e00f7370 0x7f95e0110890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 >> 172.21.15.59:0/2848138888 conn(0x7f95e00f2820 msgr2=0x7f95e00f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:40.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 shutdown_connections 2024-02-18T00:31:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 -- 172.21.15.59:0/2848138888 wait complete. 2024-02-18T00:31:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:40.821+0000 7f95e7c88700 1 librados: shutdown 2024-02-18T00:31:41.195 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:31:40] ENGINE Bus STOPPING 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:31:40] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:31:40] ENGINE Bus STOPPED 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:31:40] ENGINE Bus STARTING 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:31:40] ENGINE Serving on http://:::9283 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:31:40] ENGINE Bus STARTED 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:39.292876+0000 osd.3 (osd.3) 57 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:39.293993+0000 osd.3 (osd.3) 58 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:39.746847+0000 mgr.y (mgr.24454) 97 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:41.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:40.092491+0000 osd.4 (osd.4) 47 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:40.094409+0000 osd.4 (osd.4) 48 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:40.819820+0000 mon.a (mon.0) 750 : audit [INF] from='client.? ' entity='client.admin' 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:40.832615+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:40.834585+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:40.835521+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:40.840753+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:39.292876+0000 osd.3 (osd.3) 57 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:39.293993+0000 osd.3 (osd.3) 58 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:39.746847+0000 mgr.y (mgr.24454) 97 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:40.092491+0000 osd.4 (osd.4) 47 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:40.094409+0000 osd.4 (osd.4) 48 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:31:41.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:40.819820+0000 mon.a (mon.0) 750 : audit [INF] from='client.? ' entity='client.admin' 2024-02-18T00:31:41.198 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:40.832615+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:31:41.198 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:40.834585+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:31:41.198 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:40.835521+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:31:41.198 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:40.840753+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:41.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:31:41.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:31:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:39.292876+0000 osd.3 (osd.3) 57 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:31:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:39.293993+0000 osd.3 (osd.3) 58 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:31:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:39.746847+0000 mgr.y (mgr.24454) 97 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:31:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:40.092491+0000 osd.4 (osd.4) 47 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:31:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:40.094409+0000 osd.4 (osd.4) 48 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:31:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:40.819820+0000 mon.a (mon.0) 750 : audit [INF] from='client.? ' entity='client.admin' 2024-02-18T00:31:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:40.832615+0000 mon.a (mon.0) 751 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:31:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:40.834585+0000 mon.a (mon.0) 752 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:31:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:40.835521+0000 mon.a (mon.0) 753 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:31:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:40.840753+0000 mon.a (mon.0) 754 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:31:41.682 INFO:tasks.thrashosds:Beginning thrashosds... 2024-02-18T00:31:41.682 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-18T00:31:41.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:31:41.966 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:42.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:40.248873+0000 osd.6 (osd.6) 35 : cluster [DBG] 4.a scrub starts 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:40.251587+0000 osd.6 (osd.6) 36 : cluster [DBG] 4.a scrub ok 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:40.589822+0000 osd.2 (osd.2) 35 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:40.592079+0000 osd.2 (osd.2) 36 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:40.841012+0000 mgr.y (mgr.24454) 98 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:41.606619+0000 osd.7 (osd.7) 45 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:41.607561+0000 osd.7 (osd.7) 46 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:40.248873+0000 osd.6 (osd.6) 35 : cluster [DBG] 4.a scrub starts 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:40.251587+0000 osd.6 (osd.6) 36 : cluster [DBG] 4.a scrub ok 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:40.589822+0000 osd.2 (osd.2) 35 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:31:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:40.592079+0000 osd.2 (osd.2) 36 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:31:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:40.841012+0000 mgr.y (mgr.24454) 98 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:41.606619+0000 osd.7 (osd.7) 45 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:31:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:41.607561+0000 osd.7 (osd.7) 46 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:31:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:40.248873+0000 osd.6 (osd.6) 35 : cluster [DBG] 4.a scrub starts 2024-02-18T00:31:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:40.251587+0000 osd.6 (osd.6) 36 : cluster [DBG] 4.a scrub ok 2024-02-18T00:31:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:40.589822+0000 osd.2 (osd.2) 35 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:31:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:40.592079+0000 osd.2 (osd.2) 36 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:31:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:40.841012+0000 mgr.y (mgr.24454) 98 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:41.606619+0000 osd.7 (osd.7) 45 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:31:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:41.607561+0000 osd.7 (osd.7) 46 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:31:42.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:42] "GET /metrics HTTP/1.1" 200 172171 "" "Prometheus/2.33.4" 2024-02-18T00:31:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:41.109409+0000 osd.0 (osd.0) 47 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:31:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:41.112536+0000 osd.0 (osd.0) 48 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:31:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:41.747460+0000 mgr.y (mgr.24454) 99 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:31:43.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:41.109409+0000 osd.0 (osd.0) 47 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:31:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:41.112536+0000 osd.0 (osd.0) 48 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:31:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:41.747460+0000 mgr.y (mgr.24454) 99 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:31:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:41.109409+0000 osd.0 (osd.0) 47 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:31:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:41.112536+0000 osd.0 (osd.0) 48 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:31:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:41.747460+0000 mgr.y (mgr.24454) 99 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:31:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 -- 172.21.15.59:0/812268565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 msgr2=0x7fdfe40f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:43.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 --2- 172.21.15.59:0/812268565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe40f9090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fdfd4004870 tx=0x7fdfd4009cb0).stop 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 -- 172.21.15.59:0/812268565 shutdown_connections 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 --2- 172.21.15.59:0/812268565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfe4102210 0x7fdfe4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 --2- 172.21.15.59:0/812268565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe40f9090 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 --2- 172.21.15.59:0/812268565 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdfe40f9610 0x7fdfe4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 -- 172.21.15.59:0/812268565 >> 172.21.15.59:0/812268565 conn(0x7fdfe40f2820 msgr2=0x7fdfe40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 -- 172.21.15.59:0/812268565 shutdown_connections 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.907+0000 7fdfea52b700 1 -- 172.21.15.59:0/812268565 wait complete. 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 librados: starting msgr at 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 librados: starting objecter 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 Processor -- start 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 -- start start 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 librados: setting wanted keys 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 librados: calling monclient init 2024-02-18T00:31:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe410e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfe40f9610 0x7fdfe410e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdfe4102210 0x7fdfe41084c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe41089b0 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe4108b20 con 0x7fdfe40f9610 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfea52b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe4108ce0 con 0x7fdfe4102210 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfe3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe410e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfe3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe410e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49226/0 (socket says 172.21.15.59:49226) 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfe3fff700 1 -- 172.21.15.59:0/2597744556 learned_addr learned my addr 172.21.15.59:0/2597744556 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.908+0000 7fdfe8ac8700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdfe4102210 0x7fdfe41084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe37fe700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfe40f9610 0x7fdfe410e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe3fff700 1 -- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdfe4102210 msgr2=0x7fdfe41084c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe3fff700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdfe4102210 0x7fdfe41084c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe3fff700 1 -- 172.21.15.59:0/2597744556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfe40f9610 msgr2=0x7fdfe410e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe3fff700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfe40f9610 0x7fdfe410e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe3fff700 1 -- 172.21.15.59:0/2597744556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc005040 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe37fe700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfe40f9610 0x7fdfe410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe3fff700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe410e320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fdfd4009eb0 tx=0x7fdfd400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe17fa700 10 client.?.objecter ms_handle_connect 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe17fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe17fa700 1 -- 172.21.15.59:0/2597744556 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400e070 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe17fa700 1 -- 172.21.15.59:0/2597744556 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfd4013da0 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd4005040 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfe17fa700 1 -- 172.21.15.59:0/2597744556 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4019070 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfea52b700 10 client.14715.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe4109180 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.909+0000 7fdfea52b700 1 librados: init done 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.910+0000 7fdfe17fa700 1 -- 172.21.15.59:0/2597744556 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fdfd401e420 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.910+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe4109370 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe17fa700 1 --2- 172.21.15.59:0/2597744556 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdfd0062c90 0x7fdfd0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe17fa700 1 -- 172.21.15.59:0/2597744556 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fdfd407d5a0 con 0x7fdfe40f8cb0 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe17fa700 10 client.14715.objecter ms_dispatch 0x7fdfe4001550 osd_map(90..90 src has 1..90) v4 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe17fa700 3 client.14715.objecter handle_osd_map got epochs [90,90] > 0 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe17fa700 3 client.14715.objecter handle_osd_map decoding full epoch 90 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe37fe700 1 --2- 172.21.15.59:0/2597744556 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdfd0062c90 0x7fdfd0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe17fa700 20 client.14715.objecter dump_active .. 0 homeless 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe17fa700 10 client.14715.objecter ms_handle_connect 0x7fdfd0062c90 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.911+0000 7fdfe37fe700 1 --2- 172.21.15.59:0/2597744556 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdfd0062c90 0x7fdfd0065150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fdfcc007fd0 tx=0x7fdfcc009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:43.915+0000 7fdfe17fa700 1 -- 172.21.15.59:0/2597744556 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdfd4051280 con 0x7fdfe40f8cb0 2024-02-18T00:31:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:42.846763+0000 osd.5 (osd.5) 45 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:31:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:42.847811+0000 osd.5 (osd.5) 46 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:31:44.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:42.846763+0000 osd.5 (osd.5) 45 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:31:44.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:42.847811+0000 osd.5 (osd.5) 46 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:31:44.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.081+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfe4107290 con 0x7fdfd0062c90 2024-02-18T00:31:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.091+0000 7fdfe17fa700 1 -- 172.21.15.59:0/2597744556 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357309 (secure 0 0 0) 0x7fdfe4107290 con 0x7fdfd0062c90 2024-02-18T00:31:44.098 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 10 librados: watch_flush enter 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 10 librados: watch_flush exit 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 20 client.14715.objecter shutdown clearing up homeless session... 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 10 client.14715.objecter successfully canceled tick 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdfd0062c90 msgr2=0x7fdfd0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 --2- 172.21.15.59:0/2597744556 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdfd0062c90 0x7fdfd0065150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fdfcc007fd0 tx=0x7fdfcc009cb0).stop 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 msgr2=0x7fdfe410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe410e320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fdfd4009eb0 tx=0x7fdfd400b040).stop 2024-02-18T00:31:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 shutdown_connections 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdfe4102210 0x7fdfe41084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdfe40f8cb0 0x7fdfe410e320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 --2- 172.21.15.59:0/2597744556 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdfd0062c90 0x7fdfd0065150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 --2- 172.21.15.59:0/2597744556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfe40f9610 0x7fdfe410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 >> 172.21.15.59:0/2597744556 conn(0x7fdfe40f2820 msgr2=0x7fdfe40f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 shutdown_connections 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.094+0000 7fdfea52b700 1 -- 172.21.15.59:0/2597744556 wait complete. 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:44.095+0000 7fdfea52b700 1 librados: shutdown 2024-02-18T00:31:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:31:44.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:42.846763+0000 osd.5 (osd.5) 45 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:31:44.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:42.847811+0000 osd.5 (osd.5) 46 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:31:44.870 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-02-18T00:31:43.747648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1241,"num_read_kb":961,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":674,"ondisk_log_size":674,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2384180,"kb_used_data":7412,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747577036,"statfs":{"total":767960285184,"available":765518884864,"internally_reserved":0,"allocated":7589888,"data_stored":3195978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002451"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197352+0000","last_change":"2024-02-18T00:30:46.913439+0000","last_active":"2024-02-18T00:31:33.197352+0000","last_peered":"2024-02-18T00:31:33.197352+0000","last_clean":"2024-02-18T00:31:33.197352+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:31:33.197352+0000","last_undegraded":"2024-02-18T00:31:33.197352+0000","last_fullsized":"2024-02-18T00:31:33.197352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.913340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.913340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287432+0000","last_change":"2024-02-18T00:31:07.942520+0000","last_active":"2024-02-18T00:31:33.287432+0000","last_peered":"2024-02-18T00:31:33.287432+0000","last_clean":"2024-02-18T00:31:33.287432+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:31:33.287432+0000","last_undegraded":"2024-02-18T00:31:33.287432+0000","last_fullsized":"2024-02-18T00:31:33.287432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.942435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.942435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825257+0000","last_change":"2024-02-18T00:31:02.922852+0000","last_active":"2024-02-18T00:31:32.825257+0000","last_peered":"2024-02-18T00:31:32.825257+0000","last_clean":"2024-02-18T00:31:32.825257+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:31:32.825257+0000","last_undegraded":"2024-02-18T00:31:32.825257+0000","last_fullsized":"2024-02-18T00:31:32.825257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.922771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.922771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":130,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.196865+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:31:33.196865+0000","last_peered":"2024-02-18T00:31:33.196865+0000","last_clean":"2024-02-18T00:31:33.196865+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:31:33.196865+0000","last_undegraded":"2024-02-18T00:31:33.196865+0000","last_fullsized":"2024-02-18T00:31:33.196865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289114+0000","last_change":"2024-02-18T00:30:29.061462+0000","last_active":"2024-02-18T00:31:33.289114+0000","last_peered":"2024-02-18T00:31:33.289114+0000","last_clean":"2024-02-18T00:31:33.289114+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:31:33.289114+0000","last_undegraded":"2024-02-18T00:31:33.289114+0000","last_fullsized":"2024-02-18T00:31:33.289114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.061390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.061390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287775+0000","last_change":"2024-02-18T00:30:50.991652+0000","last_active":"2024-02-18T00:31:33.287775+0000","last_peered":"2024-02-18T00:31:33.287775+0000","last_clean":"2024-02-18T00:31:33.287775+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:31:33.287775+0000","last_undegraded":"2024-02-18T00:31:33.287775+0000","last_fullsized":"2024-02-18T00:31:33.287775+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.991592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.991592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197820+0000","last_change":"2024-02-18T00:30:48.890743+0000","last_active":"2024-02-18T00:31:33.197820+0000","last_peered":"2024-02-18T00:31:33.197820+0000","last_clean":"2024-02-18T00:31:33.197820+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:31:33.197820+0000","last_undegraded":"2024-02-18T00:31:33.197820+0000","last_fullsized":"2024-02-18T00:31:33.197820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.890677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288095+0000","last_change":"2024-02-18T00:31:03.941281+0000","last_active":"2024-02-18T00:31:33.288095+0000","last_peered":"2024-02-18T00:31:33.288095+0000","last_clean":"2024-02-18T00:31:33.288095+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:31:33.288095+0000","last_undegraded":"2024-02-18T00:31:33.288095+0000","last_fullsized":"2024-02-18T00:31:33.288095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:03.941195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:03.941195+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826040+0000","last_change":"2024-02-18T00:30:41.063319+0000","last_active":"2024-02-18T00:31:32.826040+0000","last_peered":"2024-02-18T00:31:32.826040+0000","last_clean":"2024-02-18T00:31:32.826040+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:31:32.826040+0000","last_undegraded":"2024-02-18T00:31:32.826040+0000","last_fullsized":"2024-02-18T00:31:32.826040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.063240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:41.063240+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.063240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289134+0000","last_change":"2024-02-18T00:30:25.635340+0000","last_active":"2024-02-18T00:31:33.289134+0000","last_peered":"2024-02-18T00:31:33.289134+0000","last_clean":"2024-02-18T00:31:33.289134+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:31:33.289134+0000","last_undegraded":"2024-02-18T00:31:33.289134+0000","last_fullsized":"2024-02-18T00:31:33.289134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:25.635259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:25.635259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:36.502977+0000","last_change":"2024-02-18T00:31:36.502977+0000","last_active":"2024-02-18T00:31:36.502977+0000","last_peered":"2024-02-18T00:31:36.502977+0000","last_clean":"2024-02-18T00:31:36.502977+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:31:36.502977+0000","last_undegraded":"2024-02-18T00:31:36.502977+0000","last_fullsized":"2024-02-18T00:31:36.502977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.502900+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289000+0000","last_change":"2024-02-18T00:30:46.066618+0000","last_active":"2024-02-18T00:31:33.289000+0000","last_peered":"2024-02-18T00:31:33.289000+0000","last_clean":"2024-02-18T00:31:33.289000+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:31:33.289000+0000","last_undegraded":"2024-02-18T00:31:33.289000+0000","last_fullsized":"2024-02-18T00:31:33.289000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.066555+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:36.140716+0000","last_change":"2024-02-18T00:31:36.140716+0000","last_active":"2024-02-18T00:31:36.140716+0000","last_peered":"2024-02-18T00:31:36.140716+0000","last_clean":"2024-02-18T00:31:36.140716+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:31:36.140716+0000","last_undegraded":"2024-02-18T00:31:36.140716+0000","last_fullsized":"2024-02-18T00:31:36.140716+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.140630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.140630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826531+0000","last_change":"2024-02-18T00:30:48.990813+0000","last_active":"2024-02-18T00:31:32.826531+0000","last_peered":"2024-02-18T00:31:32.826531+0000","last_clean":"2024-02-18T00:31:32.826531+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:31:32.826531+0000","last_undegraded":"2024-02-18T00:31:32.826531+0000","last_fullsized":"2024-02-18T00:31:32.826531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.990751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.990751+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289064+0000","last_change":"2024-02-18T00:30:26.654619+0000","last_active":"2024-02-18T00:31:33.289064+0000","last_peered":"2024-02-18T00:31:33.289064+0000","last_clean":"2024-02-18T00:31:33.289064+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:31:33.289064+0000","last_undegraded":"2024-02-18T00:31:33.289064+0000","last_fullsized":"2024-02-18T00:31:33.289064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.654541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.654541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'22","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:40.094511+0000","last_change":"2024-02-18T00:31:40.094511+0000","last_active":"2024-02-18T00:31:40.094511+0000","last_peered":"2024-02-18T00:31:40.094511+0000","last_clean":"2024-02-18T00:31:40.094511+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:31:40.094511+0000","last_undegraded":"2024-02-18T00:31:40.094511+0000","last_fullsized":"2024-02-18T00:31:40.094511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:31:40.094414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.094414+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197873+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:31:33.197873+0000","last_peered":"2024-02-18T00:31:33.197873+0000","last_clean":"2024-02-18T00:31:33.197873+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:31:33.197873+0000","last_undegraded":"2024-02-18T00:31:33.197873+0000","last_fullsized":"2024-02-18T00:31:33.197873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289125+0000","last_change":"2024-02-18T00:30:33.047434+0000","last_active":"2024-02-18T00:31:33.289125+0000","last_peered":"2024-02-18T00:31:33.289125+0000","last_clean":"2024-02-18T00:31:33.289125+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:31:33.289125+0000","last_undegraded":"2024-02-18T00:31:33.289125+0000","last_fullsized":"2024-02-18T00:31:33.289125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.047327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.047327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197537+0000","last_change":"2024-02-18T00:31:32.210269+0000","last_active":"2024-02-18T00:31:33.197537+0000","last_peered":"2024-02-18T00:31:33.197537+0000","last_clean":"2024-02-18T00:31:33.197537+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:31:33.197537+0000","last_undegraded":"2024-02-18T00:31:33.197537+0000","last_fullsized":"2024-02-18T00:31:33.197537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:31:32.210198+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.210198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197615+0000","last_change":"2024-02-18T00:30:50.885765+0000","last_active":"2024-02-18T00:31:33.197615+0000","last_peered":"2024-02-18T00:31:33.197615+0000","last_clean":"2024-02-18T00:31:33.197615+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:31:33.197615+0000","last_undegraded":"2024-02-18T00:31:33.197615+0000","last_fullsized":"2024-02-18T00:31:33.197615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:30:50.885651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.885651+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289558+0000","last_change":"2024-02-18T00:30:36.043298+0000","last_active":"2024-02-18T00:31:33.289558+0000","last_peered":"2024-02-18T00:31:33.289558+0000","last_clean":"2024-02-18T00:31:33.289558+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:31:33.289558+0000","last_undegraded":"2024-02-18T00:31:33.289558+0000","last_fullsized":"2024-02-18T00:31:33.289558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:36.043211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:36.043211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288594+0000","last_change":"2024-02-18T00:31:02.145322+0000","last_active":"2024-02-18T00:31:33.288594+0000","last_peered":"2024-02-18T00:31:33.288594+0000","last_clean":"2024-02-18T00:31:33.288594+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:31:33.288594+0000","last_undegraded":"2024-02-18T00:31:33.288594+0000","last_fullsized":"2024-02-18T00:31:33.288594+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.145253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288434+0000","last_change":"2024-02-18T00:30:48.450965+0000","last_active":"2024-02-18T00:31:33.288434+0000","last_peered":"2024-02-18T00:31:33.288434+0000","last_clean":"2024-02-18T00:31:33.288434+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:31:33.288434+0000","last_undegraded":"2024-02-18T00:31:33.288434+0000","last_fullsized":"2024-02-18T00:31:33.288434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.450877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.450877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:40.592166+0000","last_change":"2024-02-18T00:31:40.592166+0000","last_active":"2024-02-18T00:31:40.592166+0000","last_peered":"2024-02-18T00:31:40.592166+0000","last_clean":"2024-02-18T00:31:40.592166+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:31:40.592166+0000","last_undegraded":"2024-02-18T00:31:40.592166+0000","last_fullsized":"2024-02-18T00:31:40.592166+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:40.592083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.592083+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:37.936961+0000","last_change":"2024-02-18T00:31:37.936961+0000","last_active":"2024-02-18T00:31:37.936961+0000","last_peered":"2024-02-18T00:31:37.936961+0000","last_clean":"2024-02-18T00:31:37.936961+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:31:37.936961+0000","last_undegraded":"2024-02-18T00:31:37.936961+0000","last_fullsized":"2024-02-18T00:31:37.936961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.936896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.936896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825366+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:31:32.825366+0000","last_peered":"2024-02-18T00:31:32.825366+0000","last_clean":"2024-02-18T00:31:32.825366+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:31:32.825366+0000","last_undegraded":"2024-02-18T00:31:32.825366+0000","last_fullsized":"2024-02-18T00:31:32.825366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197170+0000","last_change":"2024-02-18T00:30:19.726673+0000","last_active":"2024-02-18T00:31:33.197170+0000","last_peered":"2024-02-18T00:31:33.197170+0000","last_clean":"2024-02-18T00:31:33.197170+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:31:33.197170+0000","last_undegraded":"2024-02-18T00:31:33.197170+0000","last_fullsized":"2024-02-18T00:31:33.197170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:19.726599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.726599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287317+0000","last_change":"2024-02-18T00:30:41.540091+0000","last_active":"2024-02-18T00:31:33.287317+0000","last_peered":"2024-02-18T00:31:33.287317+0000","last_clean":"2024-02-18T00:31:33.287317+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:31:33.287317+0000","last_undegraded":"2024-02-18T00:31:33.287317+0000","last_fullsized":"2024-02-18T00:31:33.287317+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:41.540007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.540007+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.290090+0000","last_change":"2024-02-18T00:30:38.560205+0000","last_active":"2024-02-18T00:31:33.290090+0000","last_peered":"2024-02-18T00:31:33.290090+0000","last_clean":"2024-02-18T00:31:33.290090+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:31:33.290090+0000","last_undegraded":"2024-02-18T00:31:33.290090+0000","last_fullsized":"2024-02-18T00:31:33.290090+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:30:38.560085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.560085+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:35.160824+0000","last_change":"2024-02-18T00:31:30.356896+0000","last_active":"2024-02-18T00:31:35.160824+0000","last_peered":"2024-02-18T00:31:35.160824+0000","last_clean":"2024-02-18T00:31:35.160824+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:31:35.160824+0000","last_undegraded":"2024-02-18T00:31:35.160824+0000","last_fullsized":"2024-02-18T00:31:35.160824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:30.356811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.356811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826004+0000","last_change":"2024-02-18T00:31:06.040501+0000","last_active":"2024-02-18T00:31:32.826004+0000","last_peered":"2024-02-18T00:31:32.826004+0000","last_clean":"2024-02-18T00:31:32.826004+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:31:32.826004+0000","last_undegraded":"2024-02-18T00:31:32.826004+0000","last_fullsized":"2024-02-18T00:31:32.826004+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.040425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.040425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":74,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825360+0000","last_change":"2024-02-18T00:30:38.895738+0000","last_active":"2024-02-18T00:31:32.825360+0000","last_peered":"2024-02-18T00:31:32.825360+0000","last_clean":"2024-02-18T00:31:32.825360+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:31:32.825360+0000","last_undegraded":"2024-02-18T00:31:32.825360+0000","last_fullsized":"2024-02-18T00:31:32.825360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:30:38.895655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.895655+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197705+0000","last_change":"2024-02-18T00:30:59.122281+0000","last_active":"2024-02-18T00:31:33.197705+0000","last_peered":"2024-02-18T00:31:33.197705+0000","last_clean":"2024-02-18T00:31:33.197705+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:31:33.197705+0000","last_undegraded":"2024-02-18T00:31:33.197705+0000","last_fullsized":"2024-02-18T00:31:33.197705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.122216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.122216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288685+0000","last_change":"2024-02-18T00:30:31.634376+0000","last_active":"2024-02-18T00:31:33.288685+0000","last_peered":"2024-02-18T00:31:33.288685+0000","last_clean":"2024-02-18T00:31:33.288685+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:31:33.288685+0000","last_undegraded":"2024-02-18T00:31:33.288685+0000","last_fullsized":"2024-02-18T00:31:33.288685+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.634311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.634311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288635+0000","last_change":"2024-02-18T00:30:44.493983+0000","last_active":"2024-02-18T00:31:33.288635+0000","last_peered":"2024-02-18T00:31:33.288635+0000","last_clean":"2024-02-18T00:31:33.288635+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:31:33.288635+0000","last_undegraded":"2024-02-18T00:31:33.288635+0000","last_fullsized":"2024-02-18T00:31:33.288635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.493871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.493871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197354+0000","last_change":"2024-02-18T00:30:43.819520+0000","last_active":"2024-02-18T00:31:33.197354+0000","last_peered":"2024-02-18T00:31:33.197354+0000","last_clean":"2024-02-18T00:31:33.197354+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:31:33.197354+0000","last_undegraded":"2024-02-18T00:31:33.197354+0000","last_fullsized":"2024-02-18T00:31:33.197354+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:43.819434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.819434+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288985+0000","last_change":"2024-02-18T00:30:35.533752+0000","last_active":"2024-02-18T00:31:33.288985+0000","last_peered":"2024-02-18T00:31:33.288985+0000","last_clean":"2024-02-18T00:31:33.288985+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:31:33.288985+0000","last_undegraded":"2024-02-18T00:31:33.288985+0000","last_fullsized":"2024-02-18T00:31:33.288985+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.533630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.533630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288110+0000","last_change":"2024-02-18T00:30:47.440072+0000","last_active":"2024-02-18T00:31:33.288110+0000","last_peered":"2024-02-18T00:31:33.288110+0000","last_clean":"2024-02-18T00:31:33.288110+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:31:33.288110+0000","last_undegraded":"2024-02-18T00:31:33.288110+0000","last_fullsized":"2024-02-18T00:31:33.288110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.440005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.440005+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826793+0000","last_change":"2024-02-18T00:31:30.789715+0000","last_active":"2024-02-18T00:31:32.826793+0000","last_peered":"2024-02-18T00:31:32.826793+0000","last_clean":"2024-02-18T00:31:32.826793+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:31:32.826793+0000","last_undegraded":"2024-02-18T00:31:32.826793+0000","last_fullsized":"2024-02-18T00:31:32.826793+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.789634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197121+0000","last_change":"2024-02-18T00:31:00.083411+0000","last_active":"2024-02-18T00:31:33.197121+0000","last_peered":"2024-02-18T00:31:33.197121+0000","last_clean":"2024-02-18T00:31:33.197121+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:31:33.197121+0000","last_undegraded":"2024-02-18T00:31:33.197121+0000","last_fullsized":"2024-02-18T00:31:33.197121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:31:00.083332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.083332+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197078+0000","last_change":"2024-02-18T00:31:06.048967+0000","last_active":"2024-02-18T00:31:33.197078+0000","last_peered":"2024-02-18T00:31:33.197078+0000","last_clean":"2024-02-18T00:31:33.197078+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:31:33.197078+0000","last_undegraded":"2024-02-18T00:31:33.197078+0000","last_fullsized":"2024-02-18T00:31:33.197078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.048889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.048889+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288169+0000","last_change":"2024-02-18T00:30:45.987723+0000","last_active":"2024-02-18T00:31:33.288169+0000","last_peered":"2024-02-18T00:31:33.288169+0000","last_clean":"2024-02-18T00:31:33.288169+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:31:33.288169+0000","last_undegraded":"2024-02-18T00:31:33.288169+0000","last_fullsized":"2024-02-18T00:31:33.288169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.987660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.987660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288403+0000","last_change":"2024-02-18T00:30:54.008260+0000","last_active":"2024-02-18T00:31:33.288403+0000","last_peered":"2024-02-18T00:31:33.288403+0000","last_clean":"2024-02-18T00:31:33.288403+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:31:33.288403+0000","last_undegraded":"2024-02-18T00:31:33.288403+0000","last_fullsized":"2024-02-18T00:31:33.288403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.008196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.008196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288841+0000","last_change":"2024-02-18T00:30:17.985487+0000","last_active":"2024-02-18T00:31:33.288841+0000","last_peered":"2024-02-18T00:31:33.288841+0000","last_clean":"2024-02-18T00:31:33.288841+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:31:33.288841+0000","last_undegraded":"2024-02-18T00:31:33.288841+0000","last_fullsized":"2024-02-18T00:31:33.288841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:30:17.985420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:17.985420+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823337+0000","last_change":"2024-02-18T00:31:13.960170+0000","last_active":"2024-02-18T00:31:32.823337+0000","last_peered":"2024-02-18T00:31:32.823337+0000","last_clean":"2024-02-18T00:31:32.823337+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:31:32.823337+0000","last_undegraded":"2024-02-18T00:31:32.823337+0000","last_fullsized":"2024-02-18T00:31:32.823337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:13.960089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:13.960089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197533+0000","last_change":"2024-02-18T00:30:31.810682+0000","last_active":"2024-02-18T00:31:33.197533+0000","last_peered":"2024-02-18T00:31:33.197533+0000","last_clean":"2024-02-18T00:31:33.197533+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:31:33.197533+0000","last_undegraded":"2024-02-18T00:31:33.197533+0000","last_fullsized":"2024-02-18T00:31:33.197533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.810604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.810604+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.266692+0000","last_change":"2024-02-18T00:31:38.266692+0000","last_active":"2024-02-18T00:31:38.266692+0000","last_peered":"2024-02-18T00:31:38.266692+0000","last_clean":"2024-02-18T00:31:38.266692+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:31:38.266692+0000","last_undegraded":"2024-02-18T00:31:38.266692+0000","last_fullsized":"2024-02-18T00:31:38.266692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.266577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.266577+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287094+0000","last_change":"2024-02-18T00:30:51.994431+0000","last_active":"2024-02-18T00:31:33.287094+0000","last_peered":"2024-02-18T00:31:33.287094+0000","last_clean":"2024-02-18T00:31:33.287094+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:31:33.287094+0000","last_undegraded":"2024-02-18T00:31:33.287094+0000","last_fullsized":"2024-02-18T00:31:33.287094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:30:51.994334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.994334+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289780+0000","last_change":"2024-02-18T00:31:12.044114+0000","last_active":"2024-02-18T00:31:33.289780+0000","last_peered":"2024-02-18T00:31:33.289780+0000","last_clean":"2024-02-18T00:31:33.289780+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:31:33.289780+0000","last_undegraded":"2024-02-18T00:31:33.289780+0000","last_fullsized":"2024-02-18T00:31:33.289780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:12.043994+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:35.159503+0000","last_change":"2024-02-18T00:30:35.823674+0000","last_active":"2024-02-18T00:31:35.159503+0000","last_peered":"2024-02-18T00:31:35.159503+0000","last_clean":"2024-02-18T00:31:35.159503+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:31:35.159503+0000","last_undegraded":"2024-02-18T00:31:35.159503+0000","last_fullsized":"2024-02-18T00:31:35.159503+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:35.823589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.823589+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824327+0000","last_change":"2024-02-18T00:30:21.675011+0000","last_active":"2024-02-18T00:31:32.824327+0000","last_peered":"2024-02-18T00:31:32.824327+0000","last_clean":"2024-02-18T00:31:32.824327+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:31:32.824327+0000","last_undegraded":"2024-02-18T00:31:32.824327+0000","last_fullsized":"2024-02-18T00:31:32.824327+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.674903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826869+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:31:32.826869+0000","last_peered":"2024-02-18T00:31:32.826869+0000","last_clean":"2024-02-18T00:31:32.826869+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:31:32.826869+0000","last_undegraded":"2024-02-18T00:31:32.826869+0000","last_fullsized":"2024-02-18T00:31:32.826869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:37.150285+0000","last_change":"2024-02-18T00:31:37.150285+0000","last_active":"2024-02-18T00:31:37.150285+0000","last_peered":"2024-02-18T00:31:37.150285+0000","last_clean":"2024-02-18T00:31:37.150285+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:31:37.150285+0000","last_undegraded":"2024-02-18T00:31:37.150285+0000","last_fullsized":"2024-02-18T00:31:37.150285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.150215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:35.161013+0000","last_change":"2024-02-18T00:31:01.012802+0000","last_active":"2024-02-18T00:31:35.161013+0000","last_peered":"2024-02-18T00:31:35.161013+0000","last_clean":"2024-02-18T00:31:35.161013+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:31:35.161013+0000","last_undegraded":"2024-02-18T00:31:35.161013+0000","last_fullsized":"2024-02-18T00:31:35.161013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:01.012688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.012688+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826143+0000","last_change":"2024-02-18T00:31:05.071051+0000","last_active":"2024-02-18T00:31:32.826143+0000","last_peered":"2024-02-18T00:31:32.826143+0000","last_clean":"2024-02-18T00:31:32.826143+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:31:32.826143+0000","last_undegraded":"2024-02-18T00:31:32.826143+0000","last_fullsized":"2024-02-18T00:31:32.826143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:05.070930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.070930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197787+0000","last_change":"2024-02-18T00:30:54.987124+0000","last_active":"2024-02-18T00:31:33.197787+0000","last_peered":"2024-02-18T00:31:33.197787+0000","last_clean":"2024-02-18T00:31:33.197787+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:31:33.197787+0000","last_undegraded":"2024-02-18T00:31:33.197787+0000","last_fullsized":"2024-02-18T00:31:33.197787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:30:54.987049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.987049+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.290026+0000","last_change":"2024-02-18T00:30:33.564597+0000","last_active":"2024-02-18T00:31:33.290026+0000","last_peered":"2024-02-18T00:31:33.290026+0000","last_clean":"2024-02-18T00:31:33.290026+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:31:33.290026+0000","last_undegraded":"2024-02-18T00:31:33.290026+0000","last_fullsized":"2024-02-18T00:31:33.290026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.564517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.564517+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826538+0000","last_change":"2024-02-18T00:31:23.018547+0000","last_active":"2024-02-18T00:31:32.826538+0000","last_peered":"2024-02-18T00:31:32.826538+0000","last_clean":"2024-02-18T00:31:32.826538+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:31:32.826538+0000","last_undegraded":"2024-02-18T00:31:32.826538+0000","last_fullsized":"2024-02-18T00:31:32.826538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.018460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.018460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288205+0000","last_change":"2024-02-18T00:30:38.507742+0000","last_active":"2024-02-18T00:31:33.288205+0000","last_peered":"2024-02-18T00:31:33.288205+0000","last_clean":"2024-02-18T00:31:33.288205+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:31:33.288205+0000","last_undegraded":"2024-02-18T00:31:33.288205+0000","last_fullsized":"2024-02-18T00:31:33.288205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.507637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.507637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197821+0000","last_change":"2024-02-18T00:31:29.228915+0000","last_active":"2024-02-18T00:31:33.197821+0000","last_peered":"2024-02-18T00:31:33.197821+0000","last_clean":"2024-02-18T00:31:33.197821+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:31:33.197821+0000","last_undegraded":"2024-02-18T00:31:33.197821+0000","last_fullsized":"2024-02-18T00:31:33.197821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:29.228799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:29.228799+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823646+0000","last_change":"2024-02-18T00:30:46.308529+0000","last_active":"2024-02-18T00:31:32.823646+0000","last_peered":"2024-02-18T00:31:32.823646+0000","last_clean":"2024-02-18T00:31:32.823646+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:31:32.823646+0000","last_undegraded":"2024-02-18T00:31:32.823646+0000","last_fullsized":"2024-02-18T00:31:32.823646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.308454+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197916+0000","last_change":"2024-02-18T00:30:52.923082+0000","last_active":"2024-02-18T00:31:33.197916+0000","last_peered":"2024-02-18T00:31:33.197916+0000","last_clean":"2024-02-18T00:31:33.197916+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:31:33.197916+0000","last_undegraded":"2024-02-18T00:31:33.197916+0000","last_fullsized":"2024-02-18T00:31:33.197916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.923014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.923014+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823108+0000","last_change":"2024-02-18T00:30:32.364058+0000","last_active":"2024-02-18T00:31:32.823108+0000","last_peered":"2024-02-18T00:31:32.823108+0000","last_clean":"2024-02-18T00:31:32.823108+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:31:32.823108+0000","last_undegraded":"2024-02-18T00:31:32.823108+0000","last_fullsized":"2024-02-18T00:31:32.823108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:32.363988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.363988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825270+0000","last_change":"2024-02-18T00:31:18.065868+0000","last_active":"2024-02-18T00:31:32.825270+0000","last_peered":"2024-02-18T00:31:32.825270+0000","last_clean":"2024-02-18T00:31:32.825270+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:31:32.825270+0000","last_undegraded":"2024-02-18T00:31:32.825270+0000","last_fullsized":"2024-02-18T00:31:32.825270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:31:18.065789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.065789+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197638+0000","last_change":"2024-02-18T00:30:29.789176+0000","last_active":"2024-02-18T00:31:33.197638+0000","last_peered":"2024-02-18T00:31:33.197638+0000","last_clean":"2024-02-18T00:31:33.197638+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:31:33.197638+0000","last_undegraded":"2024-02-18T00:31:33.197638+0000","last_fullsized":"2024-02-18T00:31:33.197638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.789093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.789093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289617+0000","last_change":"2024-02-18T00:30:38.058675+0000","last_active":"2024-02-18T00:31:33.289617+0000","last_peered":"2024-02-18T00:31:33.289617+0000","last_clean":"2024-02-18T00:31:33.289617+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:31:33.289617+0000","last_undegraded":"2024-02-18T00:31:33.289617+0000","last_fullsized":"2024-02-18T00:31:33.289617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:30:38.058595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.058595+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288827+0000","last_change":"2024-02-18T00:30:54.996645+0000","last_active":"2024-02-18T00:31:33.288827+0000","last_peered":"2024-02-18T00:31:33.288827+0000","last_clean":"2024-02-18T00:31:33.288827+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:31:33.288827+0000","last_undegraded":"2024-02-18T00:31:33.288827+0000","last_fullsized":"2024-02-18T00:31:33.288827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:30:54.996545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.996545+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823106+0000","last_change":"2024-02-18T00:30:43.221614+0000","last_active":"2024-02-18T00:31:32.823106+0000","last_peered":"2024-02-18T00:31:32.823106+0000","last_clean":"2024-02-18T00:31:32.823106+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:31:32.823106+0000","last_undegraded":"2024-02-18T00:31:32.823106+0000","last_fullsized":"2024-02-18T00:31:32.823106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.221541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.221541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823423+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:31:32.823423+0000","last_peered":"2024-02-18T00:31:32.823423+0000","last_clean":"2024-02-18T00:31:32.823423+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:31:32.823423+0000","last_undegraded":"2024-02-18T00:31:32.823423+0000","last_fullsized":"2024-02-18T00:31:32.823423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288319+0000","last_change":"2024-02-18T00:30:40.081002+0000","last_active":"2024-02-18T00:31:33.288319+0000","last_peered":"2024-02-18T00:31:33.288319+0000","last_clean":"2024-02-18T00:31:33.288319+0000","last_became_active":"2024-02-18T00:29:14.715019+0000","last_became_peered":"2024-02-18T00:29:14.715019+0000","last_unstale":"2024-02-18T00:31:33.288319+0000","last_undegraded":"2024-02-18T00:31:33.288319+0000","last_fullsized":"2024-02-18T00:31:33.288319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.080928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.080928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'16","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.699656+0000","last_change":"2024-02-18T00:31:33.699627+0000","last_active":"2024-02-18T00:31:33.699656+0000","last_peered":"2024-02-18T00:31:33.699656+0000","last_clean":"2024-02-18T00:31:33.699656+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:31:33.699656+0000","last_undegraded":"2024-02-18T00:31:33.699656+0000","last_fullsized":"2024-02-18T00:31:33.699656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:33.699554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:33.699554+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:34.643901+0000","last_change":"2024-02-18T00:30:46.767980+0000","last_active":"2024-02-18T00:31:34.643901+0000","last_peered":"2024-02-18T00:31:34.643901+0000","last_clean":"2024-02-18T00:31:34.643901+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:31:34.643901+0000","last_undegraded":"2024-02-18T00:31:34.643901+0000","last_fullsized":"2024-02-18T00:31:34.643901+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:46.767898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.767898+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197650+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:31:33.197650+0000","last_peered":"2024-02-18T00:31:33.197650+0000","last_clean":"2024-02-18T00:31:33.197650+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:31:33.197650+0000","last_undegraded":"2024-02-18T00:31:33.197650+0000","last_fullsized":"2024-02-18T00:31:33.197650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826245+0000","last_change":"2024-02-18T00:31:08.046030+0000","last_active":"2024-02-18T00:31:32.826245+0000","last_peered":"2024-02-18T00:31:32.826245+0000","last_clean":"2024-02-18T00:31:32.826245+0000","last_became_active":"2024-02-18T00:29:14.715157+0000","last_became_peered":"2024-02-18T00:29:14.715157+0000","last_unstale":"2024-02-18T00:31:32.826245+0000","last_undegraded":"2024-02-18T00:31:32.826245+0000","last_fullsized":"2024-02-18T00:31:32.826245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:31:08.045946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:35.160776+0000","last_change":"2024-02-18T00:30:58.030242+0000","last_active":"2024-02-18T00:31:35.160776+0000","last_peered":"2024-02-18T00:31:35.160776+0000","last_clean":"2024-02-18T00:31:35.160776+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:31:35.160776+0000","last_undegraded":"2024-02-18T00:31:35.160776+0000","last_fullsized":"2024-02-18T00:31:35.160776+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:30:58.030175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.030175+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825196+0000","last_change":"2024-02-18T00:30:47.731838+0000","last_active":"2024-02-18T00:31:32.825196+0000","last_peered":"2024-02-18T00:31:32.825196+0000","last_clean":"2024-02-18T00:31:32.825196+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:31:32.825196+0000","last_undegraded":"2024-02-18T00:31:32.825196+0000","last_fullsized":"2024-02-18T00:31:32.825196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.731752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.731752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.290183+0000","last_change":"2024-02-18T00:30:58.601762+0000","last_active":"2024-02-18T00:31:33.290183+0000","last_peered":"2024-02-18T00:31:33.290183+0000","last_clean":"2024-02-18T00:31:33.290183+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:31:33.290183+0000","last_undegraded":"2024-02-18T00:31:33.290183+0000","last_fullsized":"2024-02-18T00:31:33.290183+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.601694+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":187,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.937792+0000","last_change":"2024-02-18T00:30:32.271251+0000","last_active":"2024-02-18T00:31:38.937792+0000","last_peered":"2024-02-18T00:31:38.937792+0000","last_clean":"2024-02-18T00:31:38.937792+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:31:38.937792+0000","last_undegraded":"2024-02-18T00:31:38.937792+0000","last_fullsized":"2024-02-18T00:31:38.937792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.271183+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":137,"num_read_kb":132,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823133+0000","last_change":"2024-02-18T00:30:48.262064+0000","last_active":"2024-02-18T00:31:32.823133+0000","last_peered":"2024-02-18T00:31:32.823133+0000","last_clean":"2024-02-18T00:31:32.823133+0000","last_became_active":"2024-02-18T00:29:14.713812+0000","last_became_peered":"2024-02-18T00:29:14.713812+0000","last_unstale":"2024-02-18T00:31:32.823133+0000","last_undegraded":"2024-02-18T00:31:32.823133+0000","last_fullsized":"2024-02-18T00:31:32.823133+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.262001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.262001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287205+0000","last_change":"2024-02-18T00:30:41.040669+0000","last_active":"2024-02-18T00:31:33.287205+0000","last_peered":"2024-02-18T00:31:33.287205+0000","last_clean":"2024-02-18T00:31:33.287205+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:31:33.287205+0000","last_undegraded":"2024-02-18T00:31:33.287205+0000","last_fullsized":"2024-02-18T00:31:33.287205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:30:41.040595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.040595+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823286+0000","last_change":"2024-02-18T00:30:24.405804+0000","last_active":"2024-02-18T00:31:32.823286+0000","last_peered":"2024-02-18T00:31:32.823286+0000","last_clean":"2024-02-18T00:31:32.823286+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:31:32.823286+0000","last_undegraded":"2024-02-18T00:31:32.823286+0000","last_fullsized":"2024-02-18T00:31:32.823286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.405729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.405729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826965+0000","last_change":"2024-02-18T00:30:34.100267+0000","last_active":"2024-02-18T00:31:32.826965+0000","last_peered":"2024-02-18T00:31:32.826965+0000","last_clean":"2024-02-18T00:31:32.826965+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:31:32.826965+0000","last_undegraded":"2024-02-18T00:31:32.826965+0000","last_fullsized":"2024-02-18T00:31:32.826965+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_clean_scrub_stamp":"2024-02-18T00:30:34.100174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197308+0000","last_change":"2024-02-18T00:30:56.034602+0000","last_active":"2024-02-18T00:31:33.197308+0000","last_peered":"2024-02-18T00:31:33.197308+0000","last_clean":"2024-02-18T00:31:33.197308+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:31:33.197308+0000","last_undegraded":"2024-02-18T00:31:33.197308+0000","last_fullsized":"2024-02-18T00:31:33.197308+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:56.034493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:56.034493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826651+0000","last_change":"2024-02-18T00:30:40.055404+0000","last_active":"2024-02-18T00:31:32.826651+0000","last_peered":"2024-02-18T00:31:32.826651+0000","last_clean":"2024-02-18T00:31:32.826651+0000","last_became_active":"2024-02-18T00:29:14.716595+0000","last_became_peered":"2024-02-18T00:29:14.716595+0000","last_unstale":"2024-02-18T00:31:32.826651+0000","last_undegraded":"2024-02-18T00:31:32.826651+0000","last_fullsized":"2024-02-18T00:31:32.826651+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.055253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.055253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288414+0000","last_change":"2024-02-18T00:30:46.504493+0000","last_active":"2024-02-18T00:31:33.288414+0000","last_peered":"2024-02-18T00:31:33.288414+0000","last_clean":"2024-02-18T00:31:33.288414+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:31:33.288414+0000","last_undegraded":"2024-02-18T00:31:33.288414+0000","last_fullsized":"2024-02-18T00:31:33.288414+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.504422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.504422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288384+0000","last_change":"2024-02-18T00:31:25.298149+0000","last_active":"2024-02-18T00:31:33.288384+0000","last_peered":"2024-02-18T00:31:33.288384+0000","last_clean":"2024-02-18T00:31:33.288384+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:31:33.288384+0000","last_undegraded":"2024-02-18T00:31:33.288384+0000","last_fullsized":"2024-02-18T00:31:33.288384+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:31:25.298062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:25.298062+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289969+0000","last_change":"2024-02-18T00:30:27.608046+0000","last_active":"2024-02-18T00:31:33.289969+0000","last_peered":"2024-02-18T00:31:33.289969+0000","last_clean":"2024-02-18T00:31:33.289969+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:31:33.289969+0000","last_undegraded":"2024-02-18T00:31:33.289969+0000","last_fullsized":"2024-02-18T00:31:33.289969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:27.607981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.607981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288195+0000","last_change":"2024-02-18T00:31:00.091252+0000","last_active":"2024-02-18T00:31:33.288195+0000","last_peered":"2024-02-18T00:31:33.288195+0000","last_clean":"2024-02-18T00:31:33.288195+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:31:33.288195+0000","last_undegraded":"2024-02-18T00:31:33.288195+0000","last_fullsized":"2024-02-18T00:31:33.288195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:00.091169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.091169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824393+0000","last_change":"2024-02-18T00:30:20.623950+0000","last_active":"2024-02-18T00:31:32.824393+0000","last_peered":"2024-02-18T00:31:32.824393+0000","last_clean":"2024-02-18T00:31:32.824393+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:31:32.824393+0000","last_undegraded":"2024-02-18T00:31:32.824393+0000","last_fullsized":"2024-02-18T00:31:32.824393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.623829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.623829+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288230+0000","last_change":"2024-02-18T00:30:54.590835+0000","last_active":"2024-02-18T00:31:33.288230+0000","last_peered":"2024-02-18T00:31:33.288230+0000","last_clean":"2024-02-18T00:31:33.288230+0000","last_became_active":"2024-02-18T00:29:14.714355+0000","last_became_peered":"2024-02-18T00:29:14.714355+0000","last_unstale":"2024-02-18T00:31:33.288230+0000","last_undegraded":"2024-02-18T00:31:33.288230+0000","last_fullsized":"2024-02-18T00:31:33.288230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.590771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.590771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825175+0000","last_change":"2024-02-18T00:31:01.883504+0000","last_active":"2024-02-18T00:31:32.825175+0000","last_peered":"2024-02-18T00:31:32.825175+0000","last_clean":"2024-02-18T00:31:32.825175+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:31:32.825175+0000","last_undegraded":"2024-02-18T00:31:32.825175+0000","last_fullsized":"2024-02-18T00:31:32.825175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:01.883424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.883424+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826954+0000","last_change":"2024-02-18T00:30:52.072719+0000","last_active":"2024-02-18T00:31:32.826954+0000","last_peered":"2024-02-18T00:31:32.826954+0000","last_clean":"2024-02-18T00:31:32.826954+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:31:32.826954+0000","last_undegraded":"2024-02-18T00:31:32.826954+0000","last_fullsized":"2024-02-18T00:31:32.826954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.072619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.072619+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287701+0000","last_change":"2024-02-18T00:31:11.020203+0000","last_active":"2024-02-18T00:31:33.287701+0000","last_peered":"2024-02-18T00:31:33.287701+0000","last_clean":"2024-02-18T00:31:33.287701+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:31:33.287701+0000","last_undegraded":"2024-02-18T00:31:33.287701+0000","last_fullsized":"2024-02-18T00:31:33.287701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:11.020119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.020119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.290216+0000","last_change":"2024-02-18T00:30:53.587885+0000","last_active":"2024-02-18T00:31:33.290216+0000","last_peered":"2024-02-18T00:31:33.290216+0000","last_clean":"2024-02-18T00:31:33.290216+0000","last_became_active":"2024-02-18T00:29:14.715756+0000","last_became_peered":"2024-02-18T00:29:14.715756+0000","last_unstale":"2024-02-18T00:31:33.290216+0000","last_undegraded":"2024-02-18T00:31:33.290216+0000","last_fullsized":"2024-02-18T00:31:33.290216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:30:53.587796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:53.587796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824338+0000","last_change":"2024-02-18T00:30:48.688989+0000","last_active":"2024-02-18T00:31:32.824338+0000","last_peered":"2024-02-18T00:31:32.824338+0000","last_clean":"2024-02-18T00:31:32.824338+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:31:32.824338+0000","last_undegraded":"2024-02-18T00:31:32.824338+0000","last_fullsized":"2024-02-18T00:31:32.824338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:30:48.688901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.688901+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288335+0000","last_change":"2024-02-18T00:30:45.463697+0000","last_active":"2024-02-18T00:31:33.288335+0000","last_peered":"2024-02-18T00:31:33.288335+0000","last_clean":"2024-02-18T00:31:33.288335+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:31:33.288335+0000","last_undegraded":"2024-02-18T00:31:33.288335+0000","last_fullsized":"2024-02-18T00:31:33.288335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289217+0000","last_change":"2024-02-18T00:30:45.062422+0000","last_active":"2024-02-18T00:31:33.289217+0000","last_peered":"2024-02-18T00:31:33.289217+0000","last_clean":"2024-02-18T00:31:33.289217+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:31:33.289217+0000","last_undegraded":"2024-02-18T00:31:33.289217+0000","last_fullsized":"2024-02-18T00:31:33.289217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.062339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.062339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.977920+0000","last_change":"2024-02-18T00:31:38.977920+0000","last_active":"2024-02-18T00:31:38.977920+0000","last_peered":"2024-02-18T00:31:38.977920+0000","last_clean":"2024-02-18T00:31:38.977920+0000","last_became_active":"2024-02-18T00:29:14.714616+0000","last_became_peered":"2024-02-18T00:29:14.714616+0000","last_unstale":"2024-02-18T00:31:38.977920+0000","last_undegraded":"2024-02-18T00:31:38.977920+0000","last_fullsized":"2024-02-18T00:31:38.977920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"64'30","reported_seq":107,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197038+0000","last_change":"2024-02-18T00:30:39.827787+0000","last_active":"2024-02-18T00:31:33.197038+0000","last_peered":"2024-02-18T00:31:33.197038+0000","last_clean":"2024-02-18T00:31:33.197038+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:31:33.197038+0000","last_undegraded":"2024-02-18T00:31:33.197038+0000","last_fullsized":"2024-02-18T00:31:33.197038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'30","last_scrub_stamp":"2024-02-18T00:30:39.827720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.827720+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289059+0000","last_change":"2024-02-18T00:30:26.099064+0000","last_active":"2024-02-18T00:31:33.289059+0000","last_peered":"2024-02-18T00:31:33.289059+0000","last_clean":"2024-02-18T00:31:33.289059+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:31:33.289059+0000","last_undegraded":"2024-02-18T00:31:33.289059+0000","last_fullsized":"2024-02-18T00:31:33.289059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.098982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.098982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289781+0000","last_change":"2024-02-18T00:30:44.565713+0000","last_active":"2024-02-18T00:31:33.289781+0000","last_peered":"2024-02-18T00:31:33.289781+0000","last_clean":"2024-02-18T00:31:33.289781+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:31:33.289781+0000","last_undegraded":"2024-02-18T00:31:33.289781+0000","last_fullsized":"2024-02-18T00:31:33.289781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.565626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.565626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288509+0000","last_change":"2024-02-18T00:30:37.538788+0000","last_active":"2024-02-18T00:31:33.288509+0000","last_peered":"2024-02-18T00:31:33.288509+0000","last_clean":"2024-02-18T00:31:33.288509+0000","last_became_active":"2024-02-18T00:29:14.714186+0000","last_became_peered":"2024-02-18T00:29:14.714186+0000","last_unstale":"2024-02-18T00:31:33.288509+0000","last_undegraded":"2024-02-18T00:31:33.288509+0000","last_fullsized":"2024-02-18T00:31:33.288509+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:37.538700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:37.538700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287356+0000","last_change":"2024-02-18T00:30:21.092976+0000","last_active":"2024-02-18T00:31:33.287356+0000","last_peered":"2024-02-18T00:31:33.287356+0000","last_clean":"2024-02-18T00:31:33.287356+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:31:33.287356+0000","last_undegraded":"2024-02-18T00:31:33.287356+0000","last_fullsized":"2024-02-18T00:31:33.287356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:30:21.092908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.092908+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":86,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:39.294080+0000","last_change":"2024-02-18T00:31:39.294080+0000","last_active":"2024-02-18T00:31:39.294080+0000","last_peered":"2024-02-18T00:31:39.294080+0000","last_clean":"2024-02-18T00:31:39.294080+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:31:39.294080+0000","last_undegraded":"2024-02-18T00:31:39.294080+0000","last_fullsized":"2024-02-18T00:31:39.294080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:31:39.294006+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:31:39.294006+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197607+0000","last_change":"2024-02-18T00:30:40.855003+0000","last_active":"2024-02-18T00:31:33.197607+0000","last_peered":"2024-02-18T00:31:33.197607+0000","last_clean":"2024-02-18T00:31:33.197607+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:31:33.197607+0000","last_undegraded":"2024-02-18T00:31:33.197607+0000","last_fullsized":"2024-02-18T00:31:33.197607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.854928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.854928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826906+0000","last_change":"2024-02-18T00:31:16.020640+0000","last_active":"2024-02-18T00:31:32.826906+0000","last_peered":"2024-02-18T00:31:32.826906+0000","last_clean":"2024-02-18T00:31:32.826906+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:31:32.826906+0000","last_undegraded":"2024-02-18T00:31:32.826906+0000","last_fullsized":"2024-02-18T00:31:32.826906+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:16.020558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:16.020558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288503+0000","last_change":"2024-02-18T00:30:45.530383+0000","last_active":"2024-02-18T00:31:33.288503+0000","last_peered":"2024-02-18T00:31:33.288503+0000","last_clean":"2024-02-18T00:31:33.288503+0000","last_became_active":"2024-02-18T00:29:14.713770+0000","last_became_peered":"2024-02-18T00:29:14.713770+0000","last_unstale":"2024-02-18T00:31:33.288503+0000","last_undegraded":"2024-02-18T00:31:33.288503+0000","last_fullsized":"2024-02-18T00:31:33.288503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.530294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.530294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289657+0000","last_change":"2024-02-18T00:30:32.029864+0000","last_active":"2024-02-18T00:31:33.289657+0000","last_peered":"2024-02-18T00:31:33.289657+0000","last_clean":"2024-02-18T00:31:33.289657+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:31:33.289657+0000","last_undegraded":"2024-02-18T00:31:33.289657+0000","last_fullsized":"2024-02-18T00:31:33.289657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:32.029757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.029757+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826372+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:31:32.826372+0000","last_peered":"2024-02-18T00:31:32.826372+0000","last_clean":"2024-02-18T00:31:32.826372+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:31:32.826372+0000","last_undegraded":"2024-02-18T00:31:32.826372+0000","last_fullsized":"2024-02-18T00:31:32.826372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:34.641640+0000","last_change":"2024-02-18T00:31:05.997541+0000","last_active":"2024-02-18T00:31:34.641640+0000","last_peered":"2024-02-18T00:31:34.641640+0000","last_clean":"2024-02-18T00:31:34.641640+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:31:34.641640+0000","last_undegraded":"2024-02-18T00:31:34.641640+0000","last_fullsized":"2024-02-18T00:31:34.641640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:05.997460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.997460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287899+0000","last_change":"2024-02-18T00:30:20.213263+0000","last_active":"2024-02-18T00:31:33.287899+0000","last_peered":"2024-02-18T00:31:33.287899+0000","last_clean":"2024-02-18T00:31:33.287899+0000","last_became_active":"2024-02-18T00:29:14.713732+0000","last_became_peered":"2024-02-18T00:29:14.713732+0000","last_unstale":"2024-02-18T00:31:33.287899+0000","last_undegraded":"2024-02-18T00:31:33.287899+0000","last_fullsized":"2024-02-18T00:31:33.287899+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.213170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826613+0000","last_change":"2024-02-18T00:30:45.055464+0000","last_active":"2024-02-18T00:31:32.826613+0000","last_peered":"2024-02-18T00:31:32.826613+0000","last_clean":"2024-02-18T00:31:32.826613+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:31:32.826613+0000","last_undegraded":"2024-02-18T00:31:32.826613+0000","last_fullsized":"2024-02-18T00:31:32.826613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.055398+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289394+0000","last_change":"2024-02-18T00:30:44.103971+0000","last_active":"2024-02-18T00:31:33.289394+0000","last_peered":"2024-02-18T00:31:33.289394+0000","last_clean":"2024-02-18T00:31:33.289394+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:31:33.289394+0000","last_undegraded":"2024-02-18T00:31:33.289394+0000","last_fullsized":"2024-02-18T00:31:33.289394+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.103899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.103899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":83,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:34.640812+0000","last_change":"2024-02-18T00:30:51.228973+0000","last_active":"2024-02-18T00:31:34.640812+0000","last_peered":"2024-02-18T00:31:34.640812+0000","last_clean":"2024-02-18T00:31:34.640812+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:31:34.640812+0000","last_undegraded":"2024-02-18T00:31:34.640812+0000","last_fullsized":"2024-02-18T00:31:34.640812+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:51.228902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.228902+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288099+0000","last_change":"2024-02-18T00:30:29.330196+0000","last_active":"2024-02-18T00:31:33.288099+0000","last_peered":"2024-02-18T00:31:33.288099+0000","last_clean":"2024-02-18T00:31:33.288099+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:31:33.288099+0000","last_undegraded":"2024-02-18T00:31:33.288099+0000","last_fullsized":"2024-02-18T00:31:33.288099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.330124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.330124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288963+0000","last_change":"2024-02-18T00:31:11.002136+0000","last_active":"2024-02-18T00:31:33.288963+0000","last_peered":"2024-02-18T00:31:33.288963+0000","last_clean":"2024-02-18T00:31:33.288963+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:31:33.288963+0000","last_undegraded":"2024-02-18T00:31:33.288963+0000","last_fullsized":"2024-02-18T00:31:33.288963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:31:11.002039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.002039+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824202+0000","last_change":"2024-02-18T00:30:41.301486+0000","last_active":"2024-02-18T00:31:32.824202+0000","last_peered":"2024-02-18T00:31:32.824202+0000","last_clean":"2024-02-18T00:31:32.824202+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:31:32.824202+0000","last_undegraded":"2024-02-18T00:31:32.824202+0000","last_fullsized":"2024-02-18T00:31:32.824202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.301405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.301405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288214+0000","last_change":"2024-02-18T00:30:59.059018+0000","last_active":"2024-02-18T00:31:33.288214+0000","last_peered":"2024-02-18T00:31:33.288214+0000","last_clean":"2024-02-18T00:31:33.288214+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:31:33.288214+0000","last_undegraded":"2024-02-18T00:31:33.288214+0000","last_fullsized":"2024-02-18T00:31:33.288214+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.058947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.058947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289329+0000","last_change":"2024-02-18T00:30:24.601515+0000","last_active":"2024-02-18T00:31:33.289329+0000","last_peered":"2024-02-18T00:31:33.289329+0000","last_clean":"2024-02-18T00:31:33.289329+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:31:33.289329+0000","last_undegraded":"2024-02-18T00:31:33.289329+0000","last_fullsized":"2024-02-18T00:31:33.289329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.601442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.601442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":78,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287478+0000","last_change":"2024-02-18T00:31:32.011975+0000","last_active":"2024-02-18T00:31:33.287478+0000","last_peered":"2024-02-18T00:31:33.287478+0000","last_clean":"2024-02-18T00:31:33.287478+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:31:33.287478+0000","last_undegraded":"2024-02-18T00:31:33.287478+0000","last_fullsized":"2024-02-18T00:31:33.287478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:32.011896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.011896+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287846+0000","last_change":"2024-02-18T00:30:43.541530+0000","last_active":"2024-02-18T00:31:33.287846+0000","last_peered":"2024-02-18T00:31:33.287846+0000","last_clean":"2024-02-18T00:31:33.287846+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:31:33.287846+0000","last_undegraded":"2024-02-18T00:31:33.287846+0000","last_fullsized":"2024-02-18T00:31:33.287846+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823556+0000","last_change":"2024-02-18T00:30:45.273987+0000","last_active":"2024-02-18T00:31:32.823556+0000","last_peered":"2024-02-18T00:31:32.823556+0000","last_clean":"2024-02-18T00:31:32.823556+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:31:32.823556+0000","last_undegraded":"2024-02-18T00:31:32.823556+0000","last_fullsized":"2024-02-18T00:31:32.823556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.273920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.273920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823774+0000","last_change":"2024-02-18T00:30:38.317025+0000","last_active":"2024-02-18T00:31:32.823774+0000","last_peered":"2024-02-18T00:31:32.823774+0000","last_clean":"2024-02-18T00:31:32.823774+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:31:32.823774+0000","last_undegraded":"2024-02-18T00:31:32.823774+0000","last_fullsized":"2024-02-18T00:31:32.823774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.316960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.316960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:40.251680+0000","last_change":"2024-02-18T00:31:40.251680+0000","last_active":"2024-02-18T00:31:40.251680+0000","last_peered":"2024-02-18T00:31:40.251680+0000","last_clean":"2024-02-18T00:31:40.251680+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:31:40.251680+0000","last_undegraded":"2024-02-18T00:31:40.251680+0000","last_fullsized":"2024-02-18T00:31:40.251680+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:31:40.251596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.251596+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823338+0000","last_change":"2024-02-18T00:30:50.764741+0000","last_active":"2024-02-18T00:31:32.823338+0000","last_peered":"2024-02-18T00:31:32.823338+0000","last_clean":"2024-02-18T00:31:32.823338+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:31:32.823338+0000","last_undegraded":"2024-02-18T00:31:32.823338+0000","last_fullsized":"2024-02-18T00:31:32.823338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.764665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.764665+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823451+0000","last_change":"2024-02-18T00:30:39.299351+0000","last_active":"2024-02-18T00:31:32.823451+0000","last_peered":"2024-02-18T00:31:32.823451+0000","last_clean":"2024-02-18T00:31:32.823451+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:31:32.823451+0000","last_undegraded":"2024-02-18T00:31:32.823451+0000","last_fullsized":"2024-02-18T00:31:32.823451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:39.299285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.299285+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288506+0000","last_change":"2024-02-18T00:30:48.980366+0000","last_active":"2024-02-18T00:31:33.288506+0000","last_peered":"2024-02-18T00:31:33.288506+0000","last_clean":"2024-02-18T00:31:33.288506+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:31:33.288506+0000","last_undegraded":"2024-02-18T00:31:33.288506+0000","last_fullsized":"2024-02-18T00:31:33.288506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.980292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.980292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824416+0000","last_change":"2024-02-18T00:31:07.953329+0000","last_active":"2024-02-18T00:31:32.824416+0000","last_peered":"2024-02-18T00:31:32.824416+0000","last_clean":"2024-02-18T00:31:32.824416+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:31:32.824416+0000","last_undegraded":"2024-02-18T00:31:32.824416+0000","last_fullsized":"2024-02-18T00:31:32.824416+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:07.953265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.953265+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826220+0000","last_change":"2024-02-18T00:30:30.045287+0000","last_active":"2024-02-18T00:31:32.826220+0000","last_peered":"2024-02-18T00:31:32.826220+0000","last_clean":"2024-02-18T00:31:32.826220+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:31:32.826220+0000","last_undegraded":"2024-02-18T00:31:32.826220+0000","last_fullsized":"2024-02-18T00:31:32.826220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.045217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.045217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824491+0000","last_change":"2024-02-18T00:30:47.275395+0000","last_active":"2024-02-18T00:31:32.824491+0000","last_peered":"2024-02-18T00:31:32.824491+0000","last_clean":"2024-02-18T00:31:32.824491+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:31:32.824491+0000","last_undegraded":"2024-02-18T00:31:32.824491+0000","last_fullsized":"2024-02-18T00:31:32.824491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.275328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.275328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826105+0000","last_change":"2024-02-18T00:31:07.029773+0000","last_active":"2024-02-18T00:31:32.826105+0000","last_peered":"2024-02-18T00:31:32.826105+0000","last_clean":"2024-02-18T00:31:32.826105+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:31:32.826105+0000","last_undegraded":"2024-02-18T00:31:32.826105+0000","last_fullsized":"2024-02-18T00:31:32.826105+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:31:07.029677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.029677+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:35.160543+0000","last_change":"2024-02-18T00:30:49.588342+0000","last_active":"2024-02-18T00:31:35.160543+0000","last_peered":"2024-02-18T00:31:35.160543+0000","last_clean":"2024-02-18T00:31:35.160543+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:31:35.160543+0000","last_undegraded":"2024-02-18T00:31:35.160543+0000","last_fullsized":"2024-02-18T00:31:35.160543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:49.588269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:49.588269+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":595,"ondisk_log_size":595,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":137,"num_read_kb":132,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266721,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298028,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447124,"statfs":{"total":95995035648,"available":95689854976,"internally_reserved":0,"allocated":954368,"data_stored":400204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.505,"15min":0.502},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57399999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84899999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.88,"15min":0.88},"last":0.60699999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.58499999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.34000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.39400000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.495}]},{"osd":4,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.505},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42499999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56200000000000006}]}]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298008,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447144,"statfs":{"total":95995035648,"available":95689875456,"internally_reserved":0,"allocated":933888,"data_stored":399039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.505,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51200000000000001,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.623}]},{"osd":4,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60999999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.075,"15min":1.075},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298048,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447104,"statfs":{"total":95995035648,"available":95689834496,"internally_reserved":0,"allocated":974848,"data_stored":399862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.35999999999999999,"15min":0.311},"min":{"1min":0.29299999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.32200000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.41899999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.373},"min":{"1min":0.27700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.32400000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.55500000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.441}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.63600000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.439,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82499999999999996,"5min":1.278,"15min":1.278},"last":0.59999999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79400000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65400000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447128,"statfs":{"total":95995035648,"available":95689859072,"internally_reserved":0,"allocated":950272,"data_stored":399845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.48599999999999999,"15min":0.59799999999999998},"min":{"1min":0.23300000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.78000000000000003,"5min":6.7720000000000002,"15min":6.7720000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.38100000000000001,"15min":0.34699999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.623,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.498}]},{"osd":2,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.23799999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Sun Feb 18 00:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.42999999999999999,"15min":0.442},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.47999999999999998,"15min":0.501},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.56200000000000006,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40699999999999997}]},{"osd":4,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84899999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.73799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.67700000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64900000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":929792,"data_stored":399116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39500000000000002,"15min":0.374},"min":{"1min":0.26100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.40400000000000003,"15min":0.378},"min":{"1min":0.29699999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.51600000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.33600000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.38600000000000001},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55200000000000005,"5min":2.306,"15min":2.306},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.42299999999999999,"15min":0.40100000000000002},"min":{"1min":0.27400000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58899999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.55600000000000005,"5min":1.296,"15min":1.296},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47799999999999998,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61899999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.65700000000000003,"5min":1.675,"15min":1.675},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91200000000000003,"5min":1.173,"15min":1.173},"last":0.76200000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81399999999999995,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61199999999999999,"15min":0.624},"min":{"1min":0.46000000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.66500000000000004,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.753,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.56299999999999994}]},{"osd":7,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72399999999999998,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89300000000000002,"5min":1.242,"15min":1.242},"last":0.72999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298040,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447112,"statfs":{"total":95995035648,"available":95689842688,"internally_reserved":0,"allocated":966656,"data_stored":399514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.374,"15min":0.39000000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56000000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.49299999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.437},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64600000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.495},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41999999999999998,"15min":0.42899999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.439}]},{"osd":4,"last update":"Sun Feb 18 00:31:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77900000000000003,"5min":1.083,"15min":1.083},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96599999999999997,"5min":1.139,"15min":1.139},"last":0.65500000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82299999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86199999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83699999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81000000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986224,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298028,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447124,"statfs":{"total":95995035648,"available":95689854976,"internally_reserved":0,"allocated":954368,"data_stored":399999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":2.339,"15min":2.339},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":1.288,"15min":1.288},"last":0.72199999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55900000000000005,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65200000000000002,"5min":1.204,"15min":1.204},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70499999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70999999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":1.48,"15min":1.48},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.46200000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58099999999999996,"15min":0.60499999999999998},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69399999999999995,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.71699999999999997,"5min":1.133,"15min":1.133},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.38,"15min":0.35999999999999999},"min":{"1min":0.33300000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38300000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:30:46 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.44500000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41399999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724587,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":925696,"data_stored":398399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.66900000000000004}]},{"osd":1,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":1.306,"15min":1.306},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":1.272,"15min":1.272},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78800000000000003,"5min":1.173,"15min":1.173},"last":0.621},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":1.238,"15min":1.238},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72099999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.378},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.42399999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41099999999999998,"15min":0.40300000000000002},"min":{"1min":0.33100000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.65800000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.442},{"interface":"front","average":{"1min":0.495,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.505}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:31:44.876 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:31:45.160 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:43.747830+0000 mgr.y (mgr.24454) 100 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:44.082784+0000 mgr.y (mgr.24454) 101 : audit [DBG] from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:45.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:43.747830+0000 mgr.y (mgr.24454) 100 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:45.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:44.082784+0000 mgr.y (mgr.24454) 101 : audit [DBG] from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:43.747830+0000 mgr.y (mgr.24454) 100 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:44.082784+0000 mgr.y (mgr.24454) 101 : audit [DBG] from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:46.192 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:44.842227+0000 osd.5 (osd.5) 47 : cluster [DBG] 6.19 deep-scrub starts 2024-02-18T00:31:46.192 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:44.843484+0000 osd.5 (osd.5) 48 : cluster [DBG] 6.19 deep-scrub ok 2024-02-18T00:31:46.193 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:44.842227+0000 osd.5 (osd.5) 47 : cluster [DBG] 6.19 deep-scrub starts 2024-02-18T00:31:46.193 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:44.843484+0000 osd.5 (osd.5) 48 : cluster [DBG] 6.19 deep-scrub ok 2024-02-18T00:31:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:44.842227+0000 osd.5 (osd.5) 47 : cluster [DBG] 6.19 deep-scrub starts 2024-02-18T00:31:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:44.843484+0000 osd.5 (osd.5) 48 : cluster [DBG] 6.19 deep-scrub ok 2024-02-18T00:31:47.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/1474754391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040f8420 msgr2=0x7f2f04104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:47.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/1474754391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040f8420 0x7f2f04104a20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f2efc009be0 tx=0x7f2efc004d10).stop 2024-02-18T00:31:47.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/1474754391 shutdown_connections 2024-02-18T00:31:47.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/1474754391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f040fdbb0 0x7f2f040fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/1474754391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040f8420 0x7f2f04104a20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/1474754391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f040fe510 0x7f2f040f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/1474754391 >> 172.21.15.59:0/1474754391 conn(0x7f2f040f3800 msgr2=0x7f2f040f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/1474754391 shutdown_connections 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/1474754391 wait complete. 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.080+0000 7f2f0a1c0700 1 librados: starting msgr at 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 librados: starting objecter 2024-02-18T00:31:47.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 Processor -- start 2024-02-18T00:31:47.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 -- start start 2024-02-18T00:31:47.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 librados: setting wanted keys 2024-02-18T00:31:47.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 librados: calling monclient init 2024-02-18T00:31:47.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f040f8420 0x7f2f0410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:47.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040fdbb0 0x7f2f0410e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:47.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f040fe510 0x7f2f04108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:47.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04108510 con 0x7f2f040fdbb0 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04108680 con 0x7f2f040f8420 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f0a1c0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04108840 con 0x7f2f040fe510 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f02ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040fdbb0 0x7f2f0410e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f03fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f040fe510 0x7f2f04108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f02ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040fdbb0 0x7f2f0410e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49250/0 (socket says 172.21.15.59:49250) 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f02ffd700 1 -- 172.21.15.59:0/411501902 learned_addr learned my addr 172.21.15.59:0/411501902 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.081+0000 7f2f037fe700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f040f8420 0x7f2f0410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f02ffd700 1 -- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f040fe510 msgr2=0x7f2f04108020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f02ffd700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f040fe510 0x7f2f04108020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f02ffd700 1 -- 172.21.15.59:0/411501902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f040f8420 msgr2=0x7f2f0410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f02ffd700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f040f8420 0x7f2f0410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f02ffd700 1 -- 172.21.15.59:0/411501902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec005040 con 0x7f2f040fdbb0 2024-02-18T00:31:47.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f03fff700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f040fe510 0x7f2f04108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f02ffd700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040fdbb0 0x7f2f0410e370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f2eec0048f0 tx=0x7f2eec009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f00ff9700 10 client.?.objecter ms_handle_connect 0x7f2f040fdbb0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f00ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f00ff9700 1 -- 172.21.15.59:0/411501902 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec007b00 con 0x7f2f040fdbb0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f00ff9700 1 -- 172.21.15.59:0/411501902 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eec00f040 con 0x7f2f040fdbb0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc0097c0 con 0x7f2f040fdbb0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f00ff9700 1 -- 172.21.15.59:0/411501902 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec014990 con 0x7f2f040fdbb0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f0a1c0700 10 client.14721.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f04114cb0 con 0x7f2f040fdbb0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.082+0000 7f2f0a1c0700 1 librados: init done 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.083+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0405ee80 con 0x7f2f040fdbb0 2024-02-18T00:31:47.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.083+0000 7f2f00ff9700 1 -- 172.21.15.59:0/411501902 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2eec010500 con 0x7f2f040fdbb0 2024-02-18T00:31:47.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f00ff9700 1 --2- 172.21.15.59:0/411501902 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ef0062c90 0x7f2ef0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:47.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f00ff9700 1 -- 172.21.15.59:0/411501902 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f2eec07c4b0 con 0x7f2f040fdbb0 2024-02-18T00:31:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f00ff9700 10 client.14721.objecter ms_dispatch 0x7f2f04001550 osd_map(90..90 src has 1..90) v4 2024-02-18T00:31:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f00ff9700 3 client.14721.objecter handle_osd_map got epochs [90,90] > 0 2024-02-18T00:31:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f00ff9700 3 client.14721.objecter handle_osd_map decoding full epoch 90 2024-02-18T00:31:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f037fe700 1 --2- 172.21.15.59:0/411501902 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ef0062c90 0x7f2ef0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f00ff9700 20 client.14721.objecter dump_active .. 0 homeless 2024-02-18T00:31:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f00ff9700 10 client.14721.objecter ms_handle_connect 0x7f2ef0062c90 2024-02-18T00:31:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.084+0000 7f2f037fe700 1 --2- 172.21.15.59:0/411501902 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ef0062c90 0x7f2ef0065150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2ef4004dc0 tx=0x7f2ef400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:47.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.088+0000 7f2f00ff9700 1 -- 172.21.15.59:0/411501902 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2eec049f60 con 0x7f2f040fdbb0 2024-02-18T00:31:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:45.653162+0000 osd.7 (osd.7) 47 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:31:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:45.654822+0000 osd.7 (osd.7) 48 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:31:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:45.748470+0000 mgr.y (mgr.24454) 102 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:31:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:45.653162+0000 osd.7 (osd.7) 47 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:31:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:45.654822+0000 osd.7 (osd.7) 48 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:31:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:45.748470+0000 mgr.y (mgr.24454) 102 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:31:47.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:45.653162+0000 osd.7 (osd.7) 47 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:31:47.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:45.654822+0000 osd.7 (osd.7) 48 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:31:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:45.748470+0000 mgr.y (mgr.24454) 102 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:31:47.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.255+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f041095a0 con 0x7f2ef0062c90 2024-02-18T00:31:47.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.266+0000 7f2f00ff9700 1 -- 172.21.15.59:0/411501902 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357218 (secure 0 0 0) 0x7f2f041095a0 con 0x7f2ef0062c90 2024-02-18T00:31:47.272 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 10 librados: watch_flush enter 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 10 librados: watch_flush exit 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 20 client.14721.objecter shutdown clearing up homeless session... 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 10 client.14721.objecter successfully canceled tick 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ef0062c90 msgr2=0x7f2ef0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/411501902 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ef0062c90 0x7f2ef0065150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2ef4004dc0 tx=0x7f2ef400b040).stop 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040fdbb0 msgr2=0x7f2f0410e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.268+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040fdbb0 0x7f2f0410e370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f2eec0048f0 tx=0x7f2eec009cb0).stop 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 shutdown_connections 2024-02-18T00:31:47.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/411501902 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ef0062c90 0x7f2ef0065150 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f040f8420 0x7f2f0410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f040fe510 0x7f2f04108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 --2- 172.21.15.59:0/411501902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f040fdbb0 0x7f2f0410e370 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 >> 172.21.15.59:0/411501902 conn(0x7f2f040f3800 msgr2=0x7f2f040f5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 shutdown_connections 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 -- 172.21.15.59:0/411501902 wait complete. 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:47.269+0000 7f2f0a1c0700 1 librados: shutdown 2024-02-18T00:31:47.276 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:31:48.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:46.198186+0000 osd.6 (osd.6) 37 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:31:48.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:46.199148+0000 osd.6 (osd.6) 38 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:31:48.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:46.848673+0000 osd.5 (osd.5) 49 : cluster [DBG] 6.7 deep-scrub starts 2024-02-18T00:31:48.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:46.851567+0000 osd.5 (osd.5) 50 : cluster [DBG] 6.7 deep-scrub ok 2024-02-18T00:31:48.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:46.957640+0000 osd.1 (osd.1) 49 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:31:48.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:46.960652+0000 osd.1 (osd.1) 50 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:31:48.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:47.501728+0000 osd.2 (osd.2) 37 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:31:48.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:47.502656+0000 osd.2 (osd.2) 38 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:31:48.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:46.198186+0000 osd.6 (osd.6) 37 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:31:48.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:46.199148+0000 osd.6 (osd.6) 38 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:31:48.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:46.848673+0000 osd.5 (osd.5) 49 : cluster [DBG] 6.7 deep-scrub starts 2024-02-18T00:31:48.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:46.851567+0000 osd.5 (osd.5) 50 : cluster [DBG] 6.7 deep-scrub ok 2024-02-18T00:31:48.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:46.957640+0000 osd.1 (osd.1) 49 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:31:48.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:46.960652+0000 osd.1 (osd.1) 50 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:31:48.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:47.501728+0000 osd.2 (osd.2) 37 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:31:48.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:47.502656+0000 osd.2 (osd.2) 38 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:31:48.099 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-02-18T00:31:45.748035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1246,"num_read_kb":966,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":674,"ondisk_log_size":674,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2384180,"kb_used_data":7412,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747577036,"statfs":{"total":767960285184,"available":765518884864,"internally_reserved":0,"allocated":7589888,"data_stored":3195978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.108600"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197352+0000","last_change":"2024-02-18T00:30:46.913439+0000","last_active":"2024-02-18T00:31:33.197352+0000","last_peered":"2024-02-18T00:31:33.197352+0000","last_clean":"2024-02-18T00:31:33.197352+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:31:33.197352+0000","last_undegraded":"2024-02-18T00:31:33.197352+0000","last_fullsized":"2024-02-18T00:31:33.197352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.913340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.913340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287432+0000","last_change":"2024-02-18T00:31:07.942520+0000","last_active":"2024-02-18T00:31:33.287432+0000","last_peered":"2024-02-18T00:31:33.287432+0000","last_clean":"2024-02-18T00:31:33.287432+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:31:33.287432+0000","last_undegraded":"2024-02-18T00:31:33.287432+0000","last_fullsized":"2024-02-18T00:31:33.287432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.942435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.942435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825257+0000","last_change":"2024-02-18T00:31:02.922852+0000","last_active":"2024-02-18T00:31:32.825257+0000","last_peered":"2024-02-18T00:31:32.825257+0000","last_clean":"2024-02-18T00:31:32.825257+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:31:32.825257+0000","last_undegraded":"2024-02-18T00:31:32.825257+0000","last_fullsized":"2024-02-18T00:31:32.825257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.922771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.922771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":130,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.196865+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:31:33.196865+0000","last_peered":"2024-02-18T00:31:33.196865+0000","last_clean":"2024-02-18T00:31:33.196865+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:31:33.196865+0000","last_undegraded":"2024-02-18T00:31:33.196865+0000","last_fullsized":"2024-02-18T00:31:33.196865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289114+0000","last_change":"2024-02-18T00:30:29.061462+0000","last_active":"2024-02-18T00:31:33.289114+0000","last_peered":"2024-02-18T00:31:33.289114+0000","last_clean":"2024-02-18T00:31:33.289114+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:31:33.289114+0000","last_undegraded":"2024-02-18T00:31:33.289114+0000","last_fullsized":"2024-02-18T00:31:33.289114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.061390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.061390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287775+0000","last_change":"2024-02-18T00:30:50.991652+0000","last_active":"2024-02-18T00:31:33.287775+0000","last_peered":"2024-02-18T00:31:33.287775+0000","last_clean":"2024-02-18T00:31:33.287775+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:31:33.287775+0000","last_undegraded":"2024-02-18T00:31:33.287775+0000","last_fullsized":"2024-02-18T00:31:33.287775+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.991592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.991592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197820+0000","last_change":"2024-02-18T00:30:48.890743+0000","last_active":"2024-02-18T00:31:33.197820+0000","last_peered":"2024-02-18T00:31:33.197820+0000","last_clean":"2024-02-18T00:31:33.197820+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:31:33.197820+0000","last_undegraded":"2024-02-18T00:31:33.197820+0000","last_fullsized":"2024-02-18T00:31:33.197820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.890677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288095+0000","last_change":"2024-02-18T00:31:03.941281+0000","last_active":"2024-02-18T00:31:33.288095+0000","last_peered":"2024-02-18T00:31:33.288095+0000","last_clean":"2024-02-18T00:31:33.288095+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:31:33.288095+0000","last_undegraded":"2024-02-18T00:31:33.288095+0000","last_fullsized":"2024-02-18T00:31:33.288095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:03.941195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:03.941195+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:44.843600+0000","last_change":"2024-02-18T00:31:44.843600+0000","last_active":"2024-02-18T00:31:44.843600+0000","last_peered":"2024-02-18T00:31:44.843600+0000","last_clean":"2024-02-18T00:31:44.843600+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:31:44.843600+0000","last_undegraded":"2024-02-18T00:31:44.843600+0000","last_fullsized":"2024-02-18T00:31:44.843600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:31:44.843494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:41.607649+0000","last_change":"2024-02-18T00:31:41.607649+0000","last_active":"2024-02-18T00:31:41.607649+0000","last_peered":"2024-02-18T00:31:41.607649+0000","last_clean":"2024-02-18T00:31:41.607649+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:31:41.607649+0000","last_undegraded":"2024-02-18T00:31:41.607649+0000","last_fullsized":"2024-02-18T00:31:41.607649+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.607567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:36.502977+0000","last_change":"2024-02-18T00:31:36.502977+0000","last_active":"2024-02-18T00:31:36.502977+0000","last_peered":"2024-02-18T00:31:36.502977+0000","last_clean":"2024-02-18T00:31:36.502977+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:31:36.502977+0000","last_undegraded":"2024-02-18T00:31:36.502977+0000","last_fullsized":"2024-02-18T00:31:36.502977+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.502900+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":73,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289000+0000","last_change":"2024-02-18T00:30:46.066618+0000","last_active":"2024-02-18T00:31:33.289000+0000","last_peered":"2024-02-18T00:31:33.289000+0000","last_clean":"2024-02-18T00:31:33.289000+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:31:33.289000+0000","last_undegraded":"2024-02-18T00:31:33.289000+0000","last_fullsized":"2024-02-18T00:31:33.289000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.066555+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:36.140716+0000","last_change":"2024-02-18T00:31:36.140716+0000","last_active":"2024-02-18T00:31:36.140716+0000","last_peered":"2024-02-18T00:31:36.140716+0000","last_clean":"2024-02-18T00:31:36.140716+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:31:36.140716+0000","last_undegraded":"2024-02-18T00:31:36.140716+0000","last_fullsized":"2024-02-18T00:31:36.140716+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.140630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.140630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826531+0000","last_change":"2024-02-18T00:30:48.990813+0000","last_active":"2024-02-18T00:31:32.826531+0000","last_peered":"2024-02-18T00:31:32.826531+0000","last_clean":"2024-02-18T00:31:32.826531+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:31:32.826531+0000","last_undegraded":"2024-02-18T00:31:32.826531+0000","last_fullsized":"2024-02-18T00:31:32.826531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.990751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.990751+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289064+0000","last_change":"2024-02-18T00:30:26.654619+0000","last_active":"2024-02-18T00:31:33.289064+0000","last_peered":"2024-02-18T00:31:33.289064+0000","last_clean":"2024-02-18T00:31:33.289064+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:31:33.289064+0000","last_undegraded":"2024-02-18T00:31:33.289064+0000","last_fullsized":"2024-02-18T00:31:33.289064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.654541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.654541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'22","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:40.094511+0000","last_change":"2024-02-18T00:31:40.094511+0000","last_active":"2024-02-18T00:31:40.094511+0000","last_peered":"2024-02-18T00:31:40.094511+0000","last_clean":"2024-02-18T00:31:40.094511+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:31:40.094511+0000","last_undegraded":"2024-02-18T00:31:40.094511+0000","last_fullsized":"2024-02-18T00:31:40.094511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:31:40.094414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.094414+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197873+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:31:33.197873+0000","last_peered":"2024-02-18T00:31:33.197873+0000","last_clean":"2024-02-18T00:31:33.197873+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:31:33.197873+0000","last_undegraded":"2024-02-18T00:31:33.197873+0000","last_fullsized":"2024-02-18T00:31:33.197873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289125+0000","last_change":"2024-02-18T00:30:33.047434+0000","last_active":"2024-02-18T00:31:33.289125+0000","last_peered":"2024-02-18T00:31:33.289125+0000","last_clean":"2024-02-18T00:31:33.289125+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:31:33.289125+0000","last_undegraded":"2024-02-18T00:31:33.289125+0000","last_fullsized":"2024-02-18T00:31:33.289125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.047327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.047327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197537+0000","last_change":"2024-02-18T00:31:32.210269+0000","last_active":"2024-02-18T00:31:33.197537+0000","last_peered":"2024-02-18T00:31:33.197537+0000","last_clean":"2024-02-18T00:31:33.197537+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:31:33.197537+0000","last_undegraded":"2024-02-18T00:31:33.197537+0000","last_fullsized":"2024-02-18T00:31:33.197537+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:31:32.210198+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.210198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":83,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197615+0000","last_change":"2024-02-18T00:30:50.885765+0000","last_active":"2024-02-18T00:31:33.197615+0000","last_peered":"2024-02-18T00:31:33.197615+0000","last_clean":"2024-02-18T00:31:33.197615+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:31:33.197615+0000","last_undegraded":"2024-02-18T00:31:33.197615+0000","last_fullsized":"2024-02-18T00:31:33.197615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:30:50.885651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.885651+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289558+0000","last_change":"2024-02-18T00:30:36.043298+0000","last_active":"2024-02-18T00:31:33.289558+0000","last_peered":"2024-02-18T00:31:33.289558+0000","last_clean":"2024-02-18T00:31:33.289558+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:31:33.289558+0000","last_undegraded":"2024-02-18T00:31:33.289558+0000","last_fullsized":"2024-02-18T00:31:33.289558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:36.043211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:36.043211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288594+0000","last_change":"2024-02-18T00:31:02.145322+0000","last_active":"2024-02-18T00:31:33.288594+0000","last_peered":"2024-02-18T00:31:33.288594+0000","last_clean":"2024-02-18T00:31:33.288594+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:31:33.288594+0000","last_undegraded":"2024-02-18T00:31:33.288594+0000","last_fullsized":"2024-02-18T00:31:33.288594+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.145253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288434+0000","last_change":"2024-02-18T00:30:48.450965+0000","last_active":"2024-02-18T00:31:33.288434+0000","last_peered":"2024-02-18T00:31:33.288434+0000","last_clean":"2024-02-18T00:31:33.288434+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:31:33.288434+0000","last_undegraded":"2024-02-18T00:31:33.288434+0000","last_fullsized":"2024-02-18T00:31:33.288434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.450877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.450877+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":75,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:40.592166+0000","last_change":"2024-02-18T00:31:40.592166+0000","last_active":"2024-02-18T00:31:40.592166+0000","last_peered":"2024-02-18T00:31:40.592166+0000","last_clean":"2024-02-18T00:31:40.592166+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:31:40.592166+0000","last_undegraded":"2024-02-18T00:31:40.592166+0000","last_fullsized":"2024-02-18T00:31:40.592166+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:40.592083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.592083+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:37.936961+0000","last_change":"2024-02-18T00:31:37.936961+0000","last_active":"2024-02-18T00:31:37.936961+0000","last_peered":"2024-02-18T00:31:37.936961+0000","last_clean":"2024-02-18T00:31:37.936961+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:31:37.936961+0000","last_undegraded":"2024-02-18T00:31:37.936961+0000","last_fullsized":"2024-02-18T00:31:37.936961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.936896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.936896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825366+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:31:32.825366+0000","last_peered":"2024-02-18T00:31:32.825366+0000","last_clean":"2024-02-18T00:31:32.825366+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:31:32.825366+0000","last_undegraded":"2024-02-18T00:31:32.825366+0000","last_fullsized":"2024-02-18T00:31:32.825366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197170+0000","last_change":"2024-02-18T00:30:19.726673+0000","last_active":"2024-02-18T00:31:33.197170+0000","last_peered":"2024-02-18T00:31:33.197170+0000","last_clean":"2024-02-18T00:31:33.197170+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:31:33.197170+0000","last_undegraded":"2024-02-18T00:31:33.197170+0000","last_fullsized":"2024-02-18T00:31:33.197170+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:19.726599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:19.726599+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287317+0000","last_change":"2024-02-18T00:30:41.540091+0000","last_active":"2024-02-18T00:31:33.287317+0000","last_peered":"2024-02-18T00:31:33.287317+0000","last_clean":"2024-02-18T00:31:33.287317+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:31:33.287317+0000","last_undegraded":"2024-02-18T00:31:33.287317+0000","last_fullsized":"2024-02-18T00:31:33.287317+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:41.540007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.540007+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:45.654910+0000","last_change":"2024-02-18T00:31:45.654910+0000","last_active":"2024-02-18T00:31:45.654910+0000","last_peered":"2024-02-18T00:31:45.654910+0000","last_clean":"2024-02-18T00:31:45.654910+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:31:45.654910+0000","last_undegraded":"2024-02-18T00:31:45.654910+0000","last_fullsized":"2024-02-18T00:31:45.654910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:31:45.654827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:45.654827+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.287204+0000","last_change":"2024-02-18T00:31:30.356896+0000","last_active":"2024-02-18T00:31:38.287204+0000","last_peered":"2024-02-18T00:31:38.287204+0000","last_clean":"2024-02-18T00:31:38.287204+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:31:38.287204+0000","last_undegraded":"2024-02-18T00:31:38.287204+0000","last_fullsized":"2024-02-18T00:31:38.287204+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:30.356811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.356811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826004+0000","last_change":"2024-02-18T00:31:06.040501+0000","last_active":"2024-02-18T00:31:32.826004+0000","last_peered":"2024-02-18T00:31:32.826004+0000","last_clean":"2024-02-18T00:31:32.826004+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:31:32.826004+0000","last_undegraded":"2024-02-18T00:31:32.826004+0000","last_fullsized":"2024-02-18T00:31:32.826004+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.040425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.040425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":74,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825360+0000","last_change":"2024-02-18T00:30:38.895738+0000","last_active":"2024-02-18T00:31:32.825360+0000","last_peered":"2024-02-18T00:31:32.825360+0000","last_clean":"2024-02-18T00:31:32.825360+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:31:32.825360+0000","last_undegraded":"2024-02-18T00:31:32.825360+0000","last_fullsized":"2024-02-18T00:31:32.825360+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:30:38.895655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.895655+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197705+0000","last_change":"2024-02-18T00:30:59.122281+0000","last_active":"2024-02-18T00:31:33.197705+0000","last_peered":"2024-02-18T00:31:33.197705+0000","last_clean":"2024-02-18T00:31:33.197705+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:31:33.197705+0000","last_undegraded":"2024-02-18T00:31:33.197705+0000","last_fullsized":"2024-02-18T00:31:33.197705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.122216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.122216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288685+0000","last_change":"2024-02-18T00:30:31.634376+0000","last_active":"2024-02-18T00:31:33.288685+0000","last_peered":"2024-02-18T00:31:33.288685+0000","last_clean":"2024-02-18T00:31:33.288685+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:31:33.288685+0000","last_undegraded":"2024-02-18T00:31:33.288685+0000","last_fullsized":"2024-02-18T00:31:33.288685+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.634311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.634311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288635+0000","last_change":"2024-02-18T00:30:44.493983+0000","last_active":"2024-02-18T00:31:33.288635+0000","last_peered":"2024-02-18T00:31:33.288635+0000","last_clean":"2024-02-18T00:31:33.288635+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:31:33.288635+0000","last_undegraded":"2024-02-18T00:31:33.288635+0000","last_fullsized":"2024-02-18T00:31:33.288635+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.493871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.493871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197354+0000","last_change":"2024-02-18T00:30:43.819520+0000","last_active":"2024-02-18T00:31:33.197354+0000","last_peered":"2024-02-18T00:31:33.197354+0000","last_clean":"2024-02-18T00:31:33.197354+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:31:33.197354+0000","last_undegraded":"2024-02-18T00:31:33.197354+0000","last_fullsized":"2024-02-18T00:31:33.197354+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:43.819434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.819434+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288985+0000","last_change":"2024-02-18T00:30:35.533752+0000","last_active":"2024-02-18T00:31:33.288985+0000","last_peered":"2024-02-18T00:31:33.288985+0000","last_clean":"2024-02-18T00:31:33.288985+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:31:33.288985+0000","last_undegraded":"2024-02-18T00:31:33.288985+0000","last_fullsized":"2024-02-18T00:31:33.288985+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.533630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.533630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288110+0000","last_change":"2024-02-18T00:30:47.440072+0000","last_active":"2024-02-18T00:31:33.288110+0000","last_peered":"2024-02-18T00:31:33.288110+0000","last_clean":"2024-02-18T00:31:33.288110+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:31:33.288110+0000","last_undegraded":"2024-02-18T00:31:33.288110+0000","last_fullsized":"2024-02-18T00:31:33.288110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.440005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.440005+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826793+0000","last_change":"2024-02-18T00:31:30.789715+0000","last_active":"2024-02-18T00:31:32.826793+0000","last_peered":"2024-02-18T00:31:32.826793+0000","last_clean":"2024-02-18T00:31:32.826793+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:31:32.826793+0000","last_undegraded":"2024-02-18T00:31:32.826793+0000","last_fullsized":"2024-02-18T00:31:32.826793+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.789634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197121+0000","last_change":"2024-02-18T00:31:00.083411+0000","last_active":"2024-02-18T00:31:33.197121+0000","last_peered":"2024-02-18T00:31:33.197121+0000","last_clean":"2024-02-18T00:31:33.197121+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:31:33.197121+0000","last_undegraded":"2024-02-18T00:31:33.197121+0000","last_fullsized":"2024-02-18T00:31:33.197121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:31:00.083332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.083332+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197078+0000","last_change":"2024-02-18T00:31:06.048967+0000","last_active":"2024-02-18T00:31:33.197078+0000","last_peered":"2024-02-18T00:31:33.197078+0000","last_clean":"2024-02-18T00:31:33.197078+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:31:33.197078+0000","last_undegraded":"2024-02-18T00:31:33.197078+0000","last_fullsized":"2024-02-18T00:31:33.197078+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.048889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.048889+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288169+0000","last_change":"2024-02-18T00:30:45.987723+0000","last_active":"2024-02-18T00:31:33.288169+0000","last_peered":"2024-02-18T00:31:33.288169+0000","last_clean":"2024-02-18T00:31:33.288169+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:31:33.288169+0000","last_undegraded":"2024-02-18T00:31:33.288169+0000","last_fullsized":"2024-02-18T00:31:33.288169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.987660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.987660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288403+0000","last_change":"2024-02-18T00:30:54.008260+0000","last_active":"2024-02-18T00:31:33.288403+0000","last_peered":"2024-02-18T00:31:33.288403+0000","last_clean":"2024-02-18T00:31:33.288403+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:31:33.288403+0000","last_undegraded":"2024-02-18T00:31:33.288403+0000","last_fullsized":"2024-02-18T00:31:33.288403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.008196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.008196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":64,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288841+0000","last_change":"2024-02-18T00:30:17.985487+0000","last_active":"2024-02-18T00:31:33.288841+0000","last_peered":"2024-02-18T00:31:33.288841+0000","last_clean":"2024-02-18T00:31:33.288841+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:31:33.288841+0000","last_undegraded":"2024-02-18T00:31:33.288841+0000","last_fullsized":"2024-02-18T00:31:33.288841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:30:17.985420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:17.985420+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823337+0000","last_change":"2024-02-18T00:31:13.960170+0000","last_active":"2024-02-18T00:31:32.823337+0000","last_peered":"2024-02-18T00:31:32.823337+0000","last_clean":"2024-02-18T00:31:32.823337+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:31:32.823337+0000","last_undegraded":"2024-02-18T00:31:32.823337+0000","last_fullsized":"2024-02-18T00:31:32.823337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:13.960089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:13.960089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197533+0000","last_change":"2024-02-18T00:30:31.810682+0000","last_active":"2024-02-18T00:31:33.197533+0000","last_peered":"2024-02-18T00:31:33.197533+0000","last_clean":"2024-02-18T00:31:33.197533+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:31:33.197533+0000","last_undegraded":"2024-02-18T00:31:33.197533+0000","last_fullsized":"2024-02-18T00:31:33.197533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.810604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.810604+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.266692+0000","last_change":"2024-02-18T00:31:38.266692+0000","last_active":"2024-02-18T00:31:38.266692+0000","last_peered":"2024-02-18T00:31:38.266692+0000","last_clean":"2024-02-18T00:31:38.266692+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:31:38.266692+0000","last_undegraded":"2024-02-18T00:31:38.266692+0000","last_fullsized":"2024-02-18T00:31:38.266692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.266577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.266577+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287094+0000","last_change":"2024-02-18T00:30:51.994431+0000","last_active":"2024-02-18T00:31:33.287094+0000","last_peered":"2024-02-18T00:31:33.287094+0000","last_clean":"2024-02-18T00:31:33.287094+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:31:33.287094+0000","last_undegraded":"2024-02-18T00:31:33.287094+0000","last_fullsized":"2024-02-18T00:31:33.287094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:30:51.994334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.994334+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289780+0000","last_change":"2024-02-18T00:31:12.044114+0000","last_active":"2024-02-18T00:31:33.289780+0000","last_peered":"2024-02-18T00:31:33.289780+0000","last_clean":"2024-02-18T00:31:33.289780+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:31:33.289780+0000","last_undegraded":"2024-02-18T00:31:33.289780+0000","last_fullsized":"2024-02-18T00:31:33.289780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:12.043994+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.286747+0000","last_change":"2024-02-18T00:30:35.823674+0000","last_active":"2024-02-18T00:31:38.286747+0000","last_peered":"2024-02-18T00:31:38.286747+0000","last_clean":"2024-02-18T00:31:38.286747+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:31:38.286747+0000","last_undegraded":"2024-02-18T00:31:38.286747+0000","last_fullsized":"2024-02-18T00:31:38.286747+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:35.823589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.823589+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:41.112608+0000","last_change":"2024-02-18T00:31:41.112608+0000","last_active":"2024-02-18T00:31:41.112608+0000","last_peered":"2024-02-18T00:31:41.112608+0000","last_clean":"2024-02-18T00:31:41.112608+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:31:41.112608+0000","last_undegraded":"2024-02-18T00:31:41.112608+0000","last_fullsized":"2024-02-18T00:31:41.112608+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.112540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.112540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826869+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:31:32.826869+0000","last_peered":"2024-02-18T00:31:32.826869+0000","last_clean":"2024-02-18T00:31:32.826869+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:31:32.826869+0000","last_undegraded":"2024-02-18T00:31:32.826869+0000","last_fullsized":"2024-02-18T00:31:32.826869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:37.150285+0000","last_change":"2024-02-18T00:31:37.150285+0000","last_active":"2024-02-18T00:31:37.150285+0000","last_peered":"2024-02-18T00:31:37.150285+0000","last_clean":"2024-02-18T00:31:37.150285+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:31:37.150285+0000","last_undegraded":"2024-02-18T00:31:37.150285+0000","last_fullsized":"2024-02-18T00:31:37.150285+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.150215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.287208+0000","last_change":"2024-02-18T00:31:01.012802+0000","last_active":"2024-02-18T00:31:38.287208+0000","last_peered":"2024-02-18T00:31:38.287208+0000","last_clean":"2024-02-18T00:31:38.287208+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:31:38.287208+0000","last_undegraded":"2024-02-18T00:31:38.287208+0000","last_fullsized":"2024-02-18T00:31:38.287208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:01.012688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.012688+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826143+0000","last_change":"2024-02-18T00:31:05.071051+0000","last_active":"2024-02-18T00:31:32.826143+0000","last_peered":"2024-02-18T00:31:32.826143+0000","last_clean":"2024-02-18T00:31:32.826143+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:31:32.826143+0000","last_undegraded":"2024-02-18T00:31:32.826143+0000","last_fullsized":"2024-02-18T00:31:32.826143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:05.070930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.070930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197787+0000","last_change":"2024-02-18T00:30:54.987124+0000","last_active":"2024-02-18T00:31:33.197787+0000","last_peered":"2024-02-18T00:31:33.197787+0000","last_clean":"2024-02-18T00:31:33.197787+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:31:33.197787+0000","last_undegraded":"2024-02-18T00:31:33.197787+0000","last_fullsized":"2024-02-18T00:31:33.197787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:30:54.987049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.987049+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.290026+0000","last_change":"2024-02-18T00:30:33.564597+0000","last_active":"2024-02-18T00:31:33.290026+0000","last_peered":"2024-02-18T00:31:33.290026+0000","last_clean":"2024-02-18T00:31:33.290026+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:31:33.290026+0000","last_undegraded":"2024-02-18T00:31:33.290026+0000","last_fullsized":"2024-02-18T00:31:33.290026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.564517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.564517+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826538+0000","last_change":"2024-02-18T00:31:23.018547+0000","last_active":"2024-02-18T00:31:32.826538+0000","last_peered":"2024-02-18T00:31:32.826538+0000","last_clean":"2024-02-18T00:31:32.826538+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:31:32.826538+0000","last_undegraded":"2024-02-18T00:31:32.826538+0000","last_fullsized":"2024-02-18T00:31:32.826538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.018460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.018460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288205+0000","last_change":"2024-02-18T00:30:38.507742+0000","last_active":"2024-02-18T00:31:33.288205+0000","last_peered":"2024-02-18T00:31:33.288205+0000","last_clean":"2024-02-18T00:31:33.288205+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:31:33.288205+0000","last_undegraded":"2024-02-18T00:31:33.288205+0000","last_fullsized":"2024-02-18T00:31:33.288205+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.507637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.507637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197821+0000","last_change":"2024-02-18T00:31:29.228915+0000","last_active":"2024-02-18T00:31:33.197821+0000","last_peered":"2024-02-18T00:31:33.197821+0000","last_clean":"2024-02-18T00:31:33.197821+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:31:33.197821+0000","last_undegraded":"2024-02-18T00:31:33.197821+0000","last_fullsized":"2024-02-18T00:31:33.197821+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:29.228799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:29.228799+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823646+0000","last_change":"2024-02-18T00:30:46.308529+0000","last_active":"2024-02-18T00:31:32.823646+0000","last_peered":"2024-02-18T00:31:32.823646+0000","last_clean":"2024-02-18T00:31:32.823646+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:31:32.823646+0000","last_undegraded":"2024-02-18T00:31:32.823646+0000","last_fullsized":"2024-02-18T00:31:32.823646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.308454+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197916+0000","last_change":"2024-02-18T00:30:52.923082+0000","last_active":"2024-02-18T00:31:33.197916+0000","last_peered":"2024-02-18T00:31:33.197916+0000","last_clean":"2024-02-18T00:31:33.197916+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:31:33.197916+0000","last_undegraded":"2024-02-18T00:31:33.197916+0000","last_fullsized":"2024-02-18T00:31:33.197916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.923014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.923014+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823108+0000","last_change":"2024-02-18T00:30:32.364058+0000","last_active":"2024-02-18T00:31:32.823108+0000","last_peered":"2024-02-18T00:31:32.823108+0000","last_clean":"2024-02-18T00:31:32.823108+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:31:32.823108+0000","last_undegraded":"2024-02-18T00:31:32.823108+0000","last_fullsized":"2024-02-18T00:31:32.823108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:32.363988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.363988+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825270+0000","last_change":"2024-02-18T00:31:18.065868+0000","last_active":"2024-02-18T00:31:32.825270+0000","last_peered":"2024-02-18T00:31:32.825270+0000","last_clean":"2024-02-18T00:31:32.825270+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:31:32.825270+0000","last_undegraded":"2024-02-18T00:31:32.825270+0000","last_fullsized":"2024-02-18T00:31:32.825270+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:31:18.065789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.065789+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197638+0000","last_change":"2024-02-18T00:30:29.789176+0000","last_active":"2024-02-18T00:31:33.197638+0000","last_peered":"2024-02-18T00:31:33.197638+0000","last_clean":"2024-02-18T00:31:33.197638+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:31:33.197638+0000","last_undegraded":"2024-02-18T00:31:33.197638+0000","last_fullsized":"2024-02-18T00:31:33.197638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.789093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.789093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289617+0000","last_change":"2024-02-18T00:30:38.058675+0000","last_active":"2024-02-18T00:31:33.289617+0000","last_peered":"2024-02-18T00:31:33.289617+0000","last_clean":"2024-02-18T00:31:33.289617+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:31:33.289617+0000","last_undegraded":"2024-02-18T00:31:33.289617+0000","last_fullsized":"2024-02-18T00:31:33.289617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:30:38.058595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.058595+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":98,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288827+0000","last_change":"2024-02-18T00:30:54.996645+0000","last_active":"2024-02-18T00:31:33.288827+0000","last_peered":"2024-02-18T00:31:33.288827+0000","last_clean":"2024-02-18T00:31:33.288827+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:31:33.288827+0000","last_undegraded":"2024-02-18T00:31:33.288827+0000","last_fullsized":"2024-02-18T00:31:33.288827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:30:54.996545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.996545+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823106+0000","last_change":"2024-02-18T00:30:43.221614+0000","last_active":"2024-02-18T00:31:32.823106+0000","last_peered":"2024-02-18T00:31:32.823106+0000","last_clean":"2024-02-18T00:31:32.823106+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:31:32.823106+0000","last_undegraded":"2024-02-18T00:31:32.823106+0000","last_fullsized":"2024-02-18T00:31:32.823106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.221541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.221541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823423+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:31:32.823423+0000","last_peered":"2024-02-18T00:31:32.823423+0000","last_clean":"2024-02-18T00:31:32.823423+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:31:32.823423+0000","last_undegraded":"2024-02-18T00:31:32.823423+0000","last_fullsized":"2024-02-18T00:31:32.823423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288319+0000","last_change":"2024-02-18T00:30:40.081002+0000","last_active":"2024-02-18T00:31:33.288319+0000","last_peered":"2024-02-18T00:31:33.288319+0000","last_clean":"2024-02-18T00:31:33.288319+0000","last_became_active":"2024-02-18T00:29:14.715019+0000","last_became_peered":"2024-02-18T00:29:14.715019+0000","last_unstale":"2024-02-18T00:31:33.288319+0000","last_undegraded":"2024-02-18T00:31:33.288319+0000","last_fullsized":"2024-02-18T00:31:33.288319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.080928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.080928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'16","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.699656+0000","last_change":"2024-02-18T00:31:33.699627+0000","last_active":"2024-02-18T00:31:33.699656+0000","last_peered":"2024-02-18T00:31:33.699656+0000","last_clean":"2024-02-18T00:31:33.699656+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:31:33.699656+0000","last_undegraded":"2024-02-18T00:31:33.699656+0000","last_fullsized":"2024-02-18T00:31:33.699656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:33.699554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:33.699554+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.286683+0000","last_change":"2024-02-18T00:30:46.767980+0000","last_active":"2024-02-18T00:31:38.286683+0000","last_peered":"2024-02-18T00:31:38.286683+0000","last_clean":"2024-02-18T00:31:38.286683+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:31:38.286683+0000","last_undegraded":"2024-02-18T00:31:38.286683+0000","last_fullsized":"2024-02-18T00:31:38.286683+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:46.767898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.767898+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197650+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:31:33.197650+0000","last_peered":"2024-02-18T00:31:33.197650+0000","last_clean":"2024-02-18T00:31:33.197650+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:31:33.197650+0000","last_undegraded":"2024-02-18T00:31:33.197650+0000","last_fullsized":"2024-02-18T00:31:33.197650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826245+0000","last_change":"2024-02-18T00:31:08.046030+0000","last_active":"2024-02-18T00:31:32.826245+0000","last_peered":"2024-02-18T00:31:32.826245+0000","last_clean":"2024-02-18T00:31:32.826245+0000","last_became_active":"2024-02-18T00:29:14.715157+0000","last_became_peered":"2024-02-18T00:29:14.715157+0000","last_unstale":"2024-02-18T00:31:32.826245+0000","last_undegraded":"2024-02-18T00:31:32.826245+0000","last_fullsized":"2024-02-18T00:31:32.826245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:31:08.045946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.286690+0000","last_change":"2024-02-18T00:30:58.030242+0000","last_active":"2024-02-18T00:31:38.286690+0000","last_peered":"2024-02-18T00:31:38.286690+0000","last_clean":"2024-02-18T00:31:38.286690+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:31:38.286690+0000","last_undegraded":"2024-02-18T00:31:38.286690+0000","last_fullsized":"2024-02-18T00:31:38.286690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:30:58.030175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.030175+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825196+0000","last_change":"2024-02-18T00:30:47.731838+0000","last_active":"2024-02-18T00:31:32.825196+0000","last_peered":"2024-02-18T00:31:32.825196+0000","last_clean":"2024-02-18T00:31:32.825196+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:31:32.825196+0000","last_undegraded":"2024-02-18T00:31:32.825196+0000","last_fullsized":"2024-02-18T00:31:32.825196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.731752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.731752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.290183+0000","last_change":"2024-02-18T00:30:58.601762+0000","last_active":"2024-02-18T00:31:33.290183+0000","last_peered":"2024-02-18T00:31:33.290183+0000","last_clean":"2024-02-18T00:31:33.290183+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:31:33.290183+0000","last_undegraded":"2024-02-18T00:31:33.290183+0000","last_fullsized":"2024-02-18T00:31:33.290183+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.601694+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":192,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:43.948049+0000","last_change":"2024-02-18T00:30:32.271251+0000","last_active":"2024-02-18T00:31:43.948049+0000","last_peered":"2024-02-18T00:31:43.948049+0000","last_clean":"2024-02-18T00:31:43.948049+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:31:43.948049+0000","last_undegraded":"2024-02-18T00:31:43.948049+0000","last_fullsized":"2024-02-18T00:31:43.948049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.271183+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":142,"num_read_kb":137,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823133+0000","last_change":"2024-02-18T00:30:48.262064+0000","last_active":"2024-02-18T00:31:32.823133+0000","last_peered":"2024-02-18T00:31:32.823133+0000","last_clean":"2024-02-18T00:31:32.823133+0000","last_became_active":"2024-02-18T00:29:14.713812+0000","last_became_peered":"2024-02-18T00:29:14.713812+0000","last_unstale":"2024-02-18T00:31:32.823133+0000","last_undegraded":"2024-02-18T00:31:32.823133+0000","last_fullsized":"2024-02-18T00:31:32.823133+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.262001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.262001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287205+0000","last_change":"2024-02-18T00:30:41.040669+0000","last_active":"2024-02-18T00:31:33.287205+0000","last_peered":"2024-02-18T00:31:33.287205+0000","last_clean":"2024-02-18T00:31:33.287205+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:31:33.287205+0000","last_undegraded":"2024-02-18T00:31:33.287205+0000","last_fullsized":"2024-02-18T00:31:33.287205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:30:41.040595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.040595+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823286+0000","last_change":"2024-02-18T00:30:24.405804+0000","last_active":"2024-02-18T00:31:32.823286+0000","last_peered":"2024-02-18T00:31:32.823286+0000","last_clean":"2024-02-18T00:31:32.823286+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:31:32.823286+0000","last_undegraded":"2024-02-18T00:31:32.823286+0000","last_fullsized":"2024-02-18T00:31:32.823286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.405729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.405729+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826965+0000","last_change":"2024-02-18T00:30:34.100267+0000","last_active":"2024-02-18T00:31:32.826965+0000","last_peered":"2024-02-18T00:31:32.826965+0000","last_clean":"2024-02-18T00:31:32.826965+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:31:32.826965+0000","last_undegraded":"2024-02-18T00:31:32.826965+0000","last_fullsized":"2024-02-18T00:31:32.826965+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:34.100174+0000","last_clean_scrub_stamp":"2024-02-18T00:30:34.100174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197308+0000","last_change":"2024-02-18T00:30:56.034602+0000","last_active":"2024-02-18T00:31:33.197308+0000","last_peered":"2024-02-18T00:31:33.197308+0000","last_clean":"2024-02-18T00:31:33.197308+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:31:33.197308+0000","last_undegraded":"2024-02-18T00:31:33.197308+0000","last_fullsized":"2024-02-18T00:31:33.197308+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:56.034493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:56.034493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:42.847902+0000","last_change":"2024-02-18T00:31:42.847902+0000","last_active":"2024-02-18T00:31:42.847902+0000","last_peered":"2024-02-18T00:31:42.847902+0000","last_clean":"2024-02-18T00:31:42.847902+0000","last_became_active":"2024-02-18T00:29:14.716595+0000","last_became_peered":"2024-02-18T00:29:14.716595+0000","last_unstale":"2024-02-18T00:31:42.847902+0000","last_undegraded":"2024-02-18T00:31:42.847902+0000","last_fullsized":"2024-02-18T00:31:42.847902+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288414+0000","last_change":"2024-02-18T00:30:46.504493+0000","last_active":"2024-02-18T00:31:33.288414+0000","last_peered":"2024-02-18T00:31:33.288414+0000","last_clean":"2024-02-18T00:31:33.288414+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:31:33.288414+0000","last_undegraded":"2024-02-18T00:31:33.288414+0000","last_fullsized":"2024-02-18T00:31:33.288414+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.504422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.504422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":72,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288384+0000","last_change":"2024-02-18T00:31:25.298149+0000","last_active":"2024-02-18T00:31:33.288384+0000","last_peered":"2024-02-18T00:31:33.288384+0000","last_clean":"2024-02-18T00:31:33.288384+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:31:33.288384+0000","last_undegraded":"2024-02-18T00:31:33.288384+0000","last_fullsized":"2024-02-18T00:31:33.288384+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:31:25.298062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:25.298062+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289969+0000","last_change":"2024-02-18T00:30:27.608046+0000","last_active":"2024-02-18T00:31:33.289969+0000","last_peered":"2024-02-18T00:31:33.289969+0000","last_clean":"2024-02-18T00:31:33.289969+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:31:33.289969+0000","last_undegraded":"2024-02-18T00:31:33.289969+0000","last_fullsized":"2024-02-18T00:31:33.289969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:27.607981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.607981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288195+0000","last_change":"2024-02-18T00:31:00.091252+0000","last_active":"2024-02-18T00:31:33.288195+0000","last_peered":"2024-02-18T00:31:33.288195+0000","last_clean":"2024-02-18T00:31:33.288195+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:31:33.288195+0000","last_undegraded":"2024-02-18T00:31:33.288195+0000","last_fullsized":"2024-02-18T00:31:33.288195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:00.091169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.091169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824393+0000","last_change":"2024-02-18T00:30:20.623950+0000","last_active":"2024-02-18T00:31:32.824393+0000","last_peered":"2024-02-18T00:31:32.824393+0000","last_clean":"2024-02-18T00:31:32.824393+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:31:32.824393+0000","last_undegraded":"2024-02-18T00:31:32.824393+0000","last_fullsized":"2024-02-18T00:31:32.824393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.623829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.623829+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288230+0000","last_change":"2024-02-18T00:30:54.590835+0000","last_active":"2024-02-18T00:31:33.288230+0000","last_peered":"2024-02-18T00:31:33.288230+0000","last_clean":"2024-02-18T00:31:33.288230+0000","last_became_active":"2024-02-18T00:29:14.714355+0000","last_became_peered":"2024-02-18T00:29:14.714355+0000","last_unstale":"2024-02-18T00:31:33.288230+0000","last_undegraded":"2024-02-18T00:31:33.288230+0000","last_fullsized":"2024-02-18T00:31:33.288230+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.590771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.590771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.825175+0000","last_change":"2024-02-18T00:31:01.883504+0000","last_active":"2024-02-18T00:31:32.825175+0000","last_peered":"2024-02-18T00:31:32.825175+0000","last_clean":"2024-02-18T00:31:32.825175+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:31:32.825175+0000","last_undegraded":"2024-02-18T00:31:32.825175+0000","last_fullsized":"2024-02-18T00:31:32.825175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:01.883424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.883424+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826954+0000","last_change":"2024-02-18T00:30:52.072719+0000","last_active":"2024-02-18T00:31:32.826954+0000","last_peered":"2024-02-18T00:31:32.826954+0000","last_clean":"2024-02-18T00:31:32.826954+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:31:32.826954+0000","last_undegraded":"2024-02-18T00:31:32.826954+0000","last_fullsized":"2024-02-18T00:31:32.826954+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.072619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.072619+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287701+0000","last_change":"2024-02-18T00:31:11.020203+0000","last_active":"2024-02-18T00:31:33.287701+0000","last_peered":"2024-02-18T00:31:33.287701+0000","last_clean":"2024-02-18T00:31:33.287701+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:31:33.287701+0000","last_undegraded":"2024-02-18T00:31:33.287701+0000","last_fullsized":"2024-02-18T00:31:33.287701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:11.020119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.020119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.290216+0000","last_change":"2024-02-18T00:30:53.587885+0000","last_active":"2024-02-18T00:31:33.290216+0000","last_peered":"2024-02-18T00:31:33.290216+0000","last_clean":"2024-02-18T00:31:33.290216+0000","last_became_active":"2024-02-18T00:29:14.715756+0000","last_became_peered":"2024-02-18T00:29:14.715756+0000","last_unstale":"2024-02-18T00:31:33.290216+0000","last_undegraded":"2024-02-18T00:31:33.290216+0000","last_fullsized":"2024-02-18T00:31:33.290216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:30:53.587796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:53.587796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824338+0000","last_change":"2024-02-18T00:30:48.688989+0000","last_active":"2024-02-18T00:31:32.824338+0000","last_peered":"2024-02-18T00:31:32.824338+0000","last_clean":"2024-02-18T00:31:32.824338+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:31:32.824338+0000","last_undegraded":"2024-02-18T00:31:32.824338+0000","last_fullsized":"2024-02-18T00:31:32.824338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:30:48.688901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.688901+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288335+0000","last_change":"2024-02-18T00:30:45.463697+0000","last_active":"2024-02-18T00:31:33.288335+0000","last_peered":"2024-02-18T00:31:33.288335+0000","last_clean":"2024-02-18T00:31:33.288335+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:31:33.288335+0000","last_undegraded":"2024-02-18T00:31:33.288335+0000","last_fullsized":"2024-02-18T00:31:33.288335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289217+0000","last_change":"2024-02-18T00:30:45.062422+0000","last_active":"2024-02-18T00:31:33.289217+0000","last_peered":"2024-02-18T00:31:33.289217+0000","last_clean":"2024-02-18T00:31:33.289217+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:31:33.289217+0000","last_undegraded":"2024-02-18T00:31:33.289217+0000","last_fullsized":"2024-02-18T00:31:33.289217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.062339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.062339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.977920+0000","last_change":"2024-02-18T00:31:38.977920+0000","last_active":"2024-02-18T00:31:38.977920+0000","last_peered":"2024-02-18T00:31:38.977920+0000","last_clean":"2024-02-18T00:31:38.977920+0000","last_became_active":"2024-02-18T00:29:14.714616+0000","last_became_peered":"2024-02-18T00:29:14.714616+0000","last_unstale":"2024-02-18T00:31:38.977920+0000","last_undegraded":"2024-02-18T00:31:38.977920+0000","last_fullsized":"2024-02-18T00:31:38.977920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"64'30","reported_seq":107,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197038+0000","last_change":"2024-02-18T00:30:39.827787+0000","last_active":"2024-02-18T00:31:33.197038+0000","last_peered":"2024-02-18T00:31:33.197038+0000","last_clean":"2024-02-18T00:31:33.197038+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:31:33.197038+0000","last_undegraded":"2024-02-18T00:31:33.197038+0000","last_fullsized":"2024-02-18T00:31:33.197038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'30","last_scrub_stamp":"2024-02-18T00:30:39.827720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.827720+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289059+0000","last_change":"2024-02-18T00:30:26.099064+0000","last_active":"2024-02-18T00:31:33.289059+0000","last_peered":"2024-02-18T00:31:33.289059+0000","last_clean":"2024-02-18T00:31:33.289059+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:31:33.289059+0000","last_undegraded":"2024-02-18T00:31:33.289059+0000","last_fullsized":"2024-02-18T00:31:33.289059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.098982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.098982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289781+0000","last_change":"2024-02-18T00:30:44.565713+0000","last_active":"2024-02-18T00:31:33.289781+0000","last_peered":"2024-02-18T00:31:33.289781+0000","last_clean":"2024-02-18T00:31:33.289781+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:31:33.289781+0000","last_undegraded":"2024-02-18T00:31:33.289781+0000","last_fullsized":"2024-02-18T00:31:33.289781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.565626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.565626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288509+0000","last_change":"2024-02-18T00:30:37.538788+0000","last_active":"2024-02-18T00:31:33.288509+0000","last_peered":"2024-02-18T00:31:33.288509+0000","last_clean":"2024-02-18T00:31:33.288509+0000","last_became_active":"2024-02-18T00:29:14.714186+0000","last_became_peered":"2024-02-18T00:29:14.714186+0000","last_unstale":"2024-02-18T00:31:33.288509+0000","last_undegraded":"2024-02-18T00:31:33.288509+0000","last_fullsized":"2024-02-18T00:31:33.288509+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:37.538700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:37.538700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287356+0000","last_change":"2024-02-18T00:30:21.092976+0000","last_active":"2024-02-18T00:31:33.287356+0000","last_peered":"2024-02-18T00:31:33.287356+0000","last_clean":"2024-02-18T00:31:33.287356+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:31:33.287356+0000","last_undegraded":"2024-02-18T00:31:33.287356+0000","last_fullsized":"2024-02-18T00:31:33.287356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:30:21.092908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:21.092908+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":87,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:39.294106+0000","last_change":"2024-02-18T00:31:39.294080+0000","last_active":"2024-02-18T00:31:39.294106+0000","last_peered":"2024-02-18T00:31:39.294106+0000","last_clean":"2024-02-18T00:31:39.294106+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:31:39.294106+0000","last_undegraded":"2024-02-18T00:31:39.294106+0000","last_fullsized":"2024-02-18T00:31:39.294106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:31:39.294006+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:31:39.294006+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.197607+0000","last_change":"2024-02-18T00:30:40.855003+0000","last_active":"2024-02-18T00:31:33.197607+0000","last_peered":"2024-02-18T00:31:33.197607+0000","last_clean":"2024-02-18T00:31:33.197607+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:31:33.197607+0000","last_undegraded":"2024-02-18T00:31:33.197607+0000","last_fullsized":"2024-02-18T00:31:33.197607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:40.854928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:40.854928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826906+0000","last_change":"2024-02-18T00:31:16.020640+0000","last_active":"2024-02-18T00:31:32.826906+0000","last_peered":"2024-02-18T00:31:32.826906+0000","last_clean":"2024-02-18T00:31:32.826906+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:31:32.826906+0000","last_undegraded":"2024-02-18T00:31:32.826906+0000","last_fullsized":"2024-02-18T00:31:32.826906+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:16.020558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:16.020558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288503+0000","last_change":"2024-02-18T00:30:45.530383+0000","last_active":"2024-02-18T00:31:33.288503+0000","last_peered":"2024-02-18T00:31:33.288503+0000","last_clean":"2024-02-18T00:31:33.288503+0000","last_became_active":"2024-02-18T00:29:14.713770+0000","last_became_peered":"2024-02-18T00:29:14.713770+0000","last_unstale":"2024-02-18T00:31:33.288503+0000","last_undegraded":"2024-02-18T00:31:33.288503+0000","last_fullsized":"2024-02-18T00:31:33.288503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.530294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.530294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289657+0000","last_change":"2024-02-18T00:30:32.029864+0000","last_active":"2024-02-18T00:31:33.289657+0000","last_peered":"2024-02-18T00:31:33.289657+0000","last_clean":"2024-02-18T00:31:33.289657+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:31:33.289657+0000","last_undegraded":"2024-02-18T00:31:33.289657+0000","last_fullsized":"2024-02-18T00:31:33.289657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:32.029757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:32.029757+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826372+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:31:32.826372+0000","last_peered":"2024-02-18T00:31:32.826372+0000","last_clean":"2024-02-18T00:31:32.826372+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:31:32.826372+0000","last_undegraded":"2024-02-18T00:31:32.826372+0000","last_fullsized":"2024-02-18T00:31:32.826372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.287332+0000","last_change":"2024-02-18T00:31:05.997541+0000","last_active":"2024-02-18T00:31:38.287332+0000","last_peered":"2024-02-18T00:31:38.287332+0000","last_clean":"2024-02-18T00:31:38.287332+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:31:38.287332+0000","last_undegraded":"2024-02-18T00:31:38.287332+0000","last_fullsized":"2024-02-18T00:31:38.287332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:05.997460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.997460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287899+0000","last_change":"2024-02-18T00:30:20.213263+0000","last_active":"2024-02-18T00:31:33.287899+0000","last_peered":"2024-02-18T00:31:33.287899+0000","last_clean":"2024-02-18T00:31:33.287899+0000","last_became_active":"2024-02-18T00:29:14.713732+0000","last_became_peered":"2024-02-18T00:29:14.713732+0000","last_unstale":"2024-02-18T00:31:33.287899+0000","last_undegraded":"2024-02-18T00:31:33.287899+0000","last_fullsized":"2024-02-18T00:31:33.287899+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:30:20.213170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826613+0000","last_change":"2024-02-18T00:30:45.055464+0000","last_active":"2024-02-18T00:31:32.826613+0000","last_peered":"2024-02-18T00:31:32.826613+0000","last_clean":"2024-02-18T00:31:32.826613+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:31:32.826613+0000","last_undegraded":"2024-02-18T00:31:32.826613+0000","last_fullsized":"2024-02-18T00:31:32.826613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:30:45.055398+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.055398+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289394+0000","last_change":"2024-02-18T00:30:44.103971+0000","last_active":"2024-02-18T00:31:33.289394+0000","last_peered":"2024-02-18T00:31:33.289394+0000","last_clean":"2024-02-18T00:31:33.289394+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:31:33.289394+0000","last_undegraded":"2024-02-18T00:31:33.289394+0000","last_fullsized":"2024-02-18T00:31:33.289394+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.103899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.103899+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.286675+0000","last_change":"2024-02-18T00:30:51.228973+0000","last_active":"2024-02-18T00:31:38.286675+0000","last_peered":"2024-02-18T00:31:38.286675+0000","last_clean":"2024-02-18T00:31:38.286675+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:31:38.286675+0000","last_undegraded":"2024-02-18T00:31:38.286675+0000","last_fullsized":"2024-02-18T00:31:38.286675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:51.228902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.228902+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288099+0000","last_change":"2024-02-18T00:30:29.330196+0000","last_active":"2024-02-18T00:31:33.288099+0000","last_peered":"2024-02-18T00:31:33.288099+0000","last_clean":"2024-02-18T00:31:33.288099+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:31:33.288099+0000","last_undegraded":"2024-02-18T00:31:33.288099+0000","last_fullsized":"2024-02-18T00:31:33.288099+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.330124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.330124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288963+0000","last_change":"2024-02-18T00:31:11.002136+0000","last_active":"2024-02-18T00:31:33.288963+0000","last_peered":"2024-02-18T00:31:33.288963+0000","last_clean":"2024-02-18T00:31:33.288963+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:31:33.288963+0000","last_undegraded":"2024-02-18T00:31:33.288963+0000","last_fullsized":"2024-02-18T00:31:33.288963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:31:11.002039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.002039+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824202+0000","last_change":"2024-02-18T00:30:41.301486+0000","last_active":"2024-02-18T00:31:32.824202+0000","last_peered":"2024-02-18T00:31:32.824202+0000","last_clean":"2024-02-18T00:31:32.824202+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:31:32.824202+0000","last_undegraded":"2024-02-18T00:31:32.824202+0000","last_fullsized":"2024-02-18T00:31:32.824202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.301405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.301405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288214+0000","last_change":"2024-02-18T00:30:59.059018+0000","last_active":"2024-02-18T00:31:33.288214+0000","last_peered":"2024-02-18T00:31:33.288214+0000","last_clean":"2024-02-18T00:31:33.288214+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:31:33.288214+0000","last_undegraded":"2024-02-18T00:31:33.288214+0000","last_fullsized":"2024-02-18T00:31:33.288214+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.058947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.058947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.289329+0000","last_change":"2024-02-18T00:30:24.601515+0000","last_active":"2024-02-18T00:31:33.289329+0000","last_peered":"2024-02-18T00:31:33.289329+0000","last_clean":"2024-02-18T00:31:33.289329+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:31:33.289329+0000","last_undegraded":"2024-02-18T00:31:33.289329+0000","last_fullsized":"2024-02-18T00:31:33.289329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:24.601442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:24.601442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":78,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287478+0000","last_change":"2024-02-18T00:31:32.011975+0000","last_active":"2024-02-18T00:31:33.287478+0000","last_peered":"2024-02-18T00:31:33.287478+0000","last_clean":"2024-02-18T00:31:33.287478+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:31:33.287478+0000","last_undegraded":"2024-02-18T00:31:33.287478+0000","last_fullsized":"2024-02-18T00:31:33.287478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:32.011896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.011896+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.287846+0000","last_change":"2024-02-18T00:30:43.541530+0000","last_active":"2024-02-18T00:31:33.287846+0000","last_peered":"2024-02-18T00:31:33.287846+0000","last_clean":"2024-02-18T00:31:33.287846+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:31:33.287846+0000","last_undegraded":"2024-02-18T00:31:33.287846+0000","last_fullsized":"2024-02-18T00:31:33.287846+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823556+0000","last_change":"2024-02-18T00:30:45.273987+0000","last_active":"2024-02-18T00:31:32.823556+0000","last_peered":"2024-02-18T00:31:32.823556+0000","last_clean":"2024-02-18T00:31:32.823556+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:31:32.823556+0000","last_undegraded":"2024-02-18T00:31:32.823556+0000","last_fullsized":"2024-02-18T00:31:32.823556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.273920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.273920+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823774+0000","last_change":"2024-02-18T00:30:38.317025+0000","last_active":"2024-02-18T00:31:32.823774+0000","last_peered":"2024-02-18T00:31:32.823774+0000","last_clean":"2024-02-18T00:31:32.823774+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:31:32.823774+0000","last_undegraded":"2024-02-18T00:31:32.823774+0000","last_fullsized":"2024-02-18T00:31:32.823774+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:38.316960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:38.316960+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:40.251680+0000","last_change":"2024-02-18T00:31:40.251680+0000","last_active":"2024-02-18T00:31:40.251680+0000","last_peered":"2024-02-18T00:31:40.251680+0000","last_clean":"2024-02-18T00:31:40.251680+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:31:40.251680+0000","last_undegraded":"2024-02-18T00:31:40.251680+0000","last_fullsized":"2024-02-18T00:31:40.251680+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:31:40.251596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.251596+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823338+0000","last_change":"2024-02-18T00:30:50.764741+0000","last_active":"2024-02-18T00:31:32.823338+0000","last_peered":"2024-02-18T00:31:32.823338+0000","last_clean":"2024-02-18T00:31:32.823338+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:31:32.823338+0000","last_undegraded":"2024-02-18T00:31:32.823338+0000","last_fullsized":"2024-02-18T00:31:32.823338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.764665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.764665+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.823451+0000","last_change":"2024-02-18T00:30:39.299351+0000","last_active":"2024-02-18T00:31:32.823451+0000","last_peered":"2024-02-18T00:31:32.823451+0000","last_clean":"2024-02-18T00:31:32.823451+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:31:32.823451+0000","last_undegraded":"2024-02-18T00:31:32.823451+0000","last_fullsized":"2024-02-18T00:31:32.823451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:39.299285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:39.299285+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:33.288506+0000","last_change":"2024-02-18T00:30:48.980366+0000","last_active":"2024-02-18T00:31:33.288506+0000","last_peered":"2024-02-18T00:31:33.288506+0000","last_clean":"2024-02-18T00:31:33.288506+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:31:33.288506+0000","last_undegraded":"2024-02-18T00:31:33.288506+0000","last_fullsized":"2024-02-18T00:31:33.288506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.980292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.980292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824416+0000","last_change":"2024-02-18T00:31:07.953329+0000","last_active":"2024-02-18T00:31:32.824416+0000","last_peered":"2024-02-18T00:31:32.824416+0000","last_clean":"2024-02-18T00:31:32.824416+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:31:32.824416+0000","last_undegraded":"2024-02-18T00:31:32.824416+0000","last_fullsized":"2024-02-18T00:31:32.824416+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:07.953265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.953265+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826220+0000","last_change":"2024-02-18T00:30:30.045287+0000","last_active":"2024-02-18T00:31:32.826220+0000","last_peered":"2024-02-18T00:31:32.826220+0000","last_clean":"2024-02-18T00:31:32.826220+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:31:32.826220+0000","last_undegraded":"2024-02-18T00:31:32.826220+0000","last_fullsized":"2024-02-18T00:31:32.826220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.045217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.045217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.824491+0000","last_change":"2024-02-18T00:30:47.275395+0000","last_active":"2024-02-18T00:31:32.824491+0000","last_peered":"2024-02-18T00:31:32.824491+0000","last_clean":"2024-02-18T00:31:32.824491+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:31:32.824491+0000","last_undegraded":"2024-02-18T00:31:32.824491+0000","last_fullsized":"2024-02-18T00:31:32.824491+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.275328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.275328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-02-18T00:31:32.826105+0000","last_change":"2024-02-18T00:31:07.029773+0000","last_active":"2024-02-18T00:31:32.826105+0000","last_peered":"2024-02-18T00:31:32.826105+0000","last_clean":"2024-02-18T00:31:32.826105+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:31:32.826105+0000","last_undegraded":"2024-02-18T00:31:32.826105+0000","last_fullsized":"2024-02-18T00:31:32.826105+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:31:07.029677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.029677+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-02-18T00:31:38.287208+0000","last_change":"2024-02-18T00:30:49.588342+0000","last_active":"2024-02-18T00:31:38.287208+0000","last_peered":"2024-02-18T00:31:38.287208+0000","last_clean":"2024-02-18T00:31:38.287208+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:31:38.287208+0000","last_undegraded":"2024-02-18T00:31:38.287208+0000","last_fullsized":"2024-02-18T00:31:38.287208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:49.588269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:49.588269+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":595,"ondisk_log_size":595,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":142,"num_read_kb":137,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266722,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298028,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447124,"statfs":{"total":95995035648,"available":95689854976,"internally_reserved":0,"allocated":954368,"data_stored":400204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.505,"15min":0.502},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.497}]},{"osd":1,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84899999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.88,"15min":0.88},"last":0.58299999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48099999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.34000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.39400000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.432},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45800000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55500000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298008,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447144,"statfs":{"total":95995035648,"available":95689875456,"internally_reserved":0,"allocated":933888,"data_stored":399039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.504},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.505,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.626}]},{"osd":1,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.49399999999999999,"15min":0.48699999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.56799999999999995}]},{"osd":2,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.52500000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51200000000000001,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.52500000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.28199999999999997,"5min":0.254,"15min":0.254},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.39700000000000002},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.38100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41599999999999998,"15min":0.40600000000000003},"min":{"1min":0.308,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.61499999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.075,"15min":1.075},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298048,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447104,"statfs":{"total":95995035648,"available":95689834496,"internally_reserved":0,"allocated":974848,"data_stored":399862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.35999999999999999,"15min":0.311},"min":{"1min":0.29299999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.36199999999999999,"15min":0.32200000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.27200000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.373},"min":{"1min":0.27700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.30399999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.55500000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.34300000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.58699999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.66200000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.439,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82499999999999996,"5min":1.278,"15min":1.278},"last":0.52000000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79400000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.52900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298024,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447128,"statfs":{"total":95995035648,"available":95689859072,"internally_reserved":0,"allocated":950272,"data_stored":399845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.42399999999999999,"15min":0.57699999999999996},"min":{"1min":0.221,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46500000000000002,"5min":6.7720000000000002,"15min":6.7720000000000002},"last":0.221},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38800000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.29199999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.23799999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.23000000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.254},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46200000000000002,"15min":0.495},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.314}]},{"osd":4,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.54400000000000004},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84899999999999998,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.45100000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.73799999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55500000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71199999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46500000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":929792,"data_stored":399116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39500000000000002,"15min":0.374},"min":{"1min":0.26100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.40400000000000003,"15min":0.378},"min":{"1min":0.29699999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.51600000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.28699999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.38600000000000001},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55200000000000005,"5min":2.306,"15min":2.306},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.42299999999999999,"15min":0.40100000000000002},"min":{"1min":0.27400000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58899999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.38500000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.55600000000000005,"5min":1.296,"15min":1.296},"last":0.371},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47799999999999998,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61899999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.35699999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:30:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.65700000000000003,"5min":1.675,"15min":1.675},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91200000000000003,"5min":1.173,"15min":1.173},"last":0.61399999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81399999999999995,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61199999999999999,"15min":0.624},"min":{"1min":0.46000000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.66500000000000004,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.621},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64800000000000002,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.753,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72399999999999998,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89300000000000002,"5min":1.242,"15min":1.242},"last":0.63800000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298040,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447112,"statfs":{"total":95995035648,"available":95689842688,"internally_reserved":0,"allocated":966656,"data_stored":399514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.374,"15min":0.39000000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56000000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.34599999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.49299999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.36699999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.437},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64600000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41999999999999998,"15min":0.42899999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.434}]},{"osd":4,"last update":"Sun Feb 18 00:31:01 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56899999999999995,"15min":0.57499999999999996},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77900000000000003,"5min":1.083,"15min":1.083},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69999999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96599999999999997,"5min":1.139,"15min":1.139},"last":0.61499999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:30:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82299999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86199999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80700000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81599999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298028,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447124,"statfs":{"total":95995035648,"available":95689854976,"internally_reserved":0,"allocated":954368,"data_stored":399999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.375,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68200000000000005,"5min":2.339,"15min":2.339},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.41099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":1.288,"15min":1.288},"last":0.66600000000000004}]},{"osd":1,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55900000000000005,"15min":0.57399999999999995},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.65200000000000002,"5min":1.204,"15min":1.204},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55400000000000005,"15min":0.56200000000000006},"min":{"1min":0.35899999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70499999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.64900000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.59199999999999997,"15min":0.61399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66000000000000003,"5min":1.48,"15min":1.48},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56599999999999995,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.72999999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56399999999999995}]},{"osd":3,"last update":"Sun Feb 18 00:30:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58099999999999996,"15min":0.60499999999999998},"min":{"1min":0.36199999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.69399999999999995,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.41999999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.71699999999999997,"5min":1.133,"15min":1.133},"last":0.48399999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.38,"15min":0.35999999999999999},"min":{"1min":0.33300000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:30:46 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.623}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.83299999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298000,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447152,"statfs":{"total":95995035648,"available":95689883648,"internally_reserved":0,"allocated":925696,"data_stored":398399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.68899999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":1.306,"15min":1.306},"last":0.751},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":1.272,"15min":1.272},"last":0.626}]},{"osd":2,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78800000000000003,"5min":1.173,"15min":1.173},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.73199999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":1.238,"15min":1.238},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72099999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.71199999999999997}]},{"osd":4,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.378},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.54000000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:31:48.107 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-18T00:31:48.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:46.198186+0000 osd.6 (osd.6) 37 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:46.199148+0000 osd.6 (osd.6) 38 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:46.848673+0000 osd.5 (osd.5) 49 : cluster [DBG] 6.7 deep-scrub starts 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:46.851567+0000 osd.5 (osd.5) 50 : cluster [DBG] 6.7 deep-scrub ok 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:46.957640+0000 osd.1 (osd.1) 49 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:46.960652+0000 osd.1 (osd.1) 50 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:47.501728+0000 osd.2 (osd.2) 37 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:31:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:47.502656+0000 osd.2 (osd.2) 38 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:31:48.394 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:48.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:47.257056+0000 mgr.y (mgr.24454) 103 : audit [DBG] from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:48.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:47.748860+0000 mgr.y (mgr.24454) 104 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:48.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:47.970964+0000 osd.1 (osd.1) 51 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:31:48.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:47.973958+0000 osd.1 (osd.1) 52 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:31:48.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:48.593608+0000 osd.7 (osd.7) 49 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:31:48.967 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:48.594632+0000 osd.7 (osd.7) 50 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:31:48.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:47.257056+0000 mgr.y (mgr.24454) 103 : audit [DBG] from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:48.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:47.748860+0000 mgr.y (mgr.24454) 104 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:48.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:47.970964+0000 osd.1 (osd.1) 51 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:31:48.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:47.973958+0000 osd.1 (osd.1) 52 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:31:48.968 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:48.593608+0000 osd.7 (osd.7) 49 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:31:48.968 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:48.594632+0000 osd.7 (osd.7) 50 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:31:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:47.257056+0000 mgr.y (mgr.24454) 103 : audit [DBG] from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:31:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:47.748860+0000 mgr.y (mgr.24454) 104 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:47.970964+0000 osd.1 (osd.1) 51 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:31:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:47.973958+0000 osd.1 (osd.1) 52 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:31:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:48.593608+0000 osd.7 (osd.7) 49 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:31:49.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:48.594632+0000 osd.7 (osd.7) 50 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:31:50.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 -- 172.21.15.59:0/1434540691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f8d50 msgr2=0x7f0e6c0f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 --2- 172.21.15.59:0/1434540691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f8d50 0x7f0e6c0f9130 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f0e5c004870 tx=0x7f0e5c009cb0).stop 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 -- 172.21.15.59:0/1434540691 shutdown_connections 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 --2- 172.21.15.59:0/1434540691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e6c1022b0 0x7f0e6c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 --2- 172.21.15.59:0/1434540691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f8d50 0x7f0e6c0f9130 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 --2- 172.21.15.59:0/1434540691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e6c0f9620 0x7f0e6c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 -- 172.21.15.59:0/1434540691 >> 172.21.15.59:0/1434540691 conn(0x7f0e6c0f2820 msgr2=0x7f0e6c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 -- 172.21.15.59:0/1434540691 shutdown_connections 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.435+0000 7f0e73624700 1 -- 172.21.15.59:0/1434540691 wait complete. 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 librados: starting msgr at 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 librados: starting objecter 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 Processor -- start 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 -- start start 2024-02-18T00:31:50.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 librados: setting wanted keys 2024-02-18T00:31:50.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 librados: calling monclient init 2024-02-18T00:31:50.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e6c0f8d50 0x7f0e6c10e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f9620 0x7f0e6c10e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e6c1022b0 0x7f0e6c1084f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1089e0 con 0x7f0e6c0f9620 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c108b50 con 0x7f0e6c0f8d50 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.436+0000 7f0e73624700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c108d10 con 0x7f0e6c1022b0 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e70bbf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f9620 0x7f0e6c10e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e70bbf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f9620 0x7f0e6c10e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38868/0 (socket says 172.21.15.59:38868) 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e70bbf700 1 -- 172.21.15.59:0/3106262282 learned_addr learned my addr 172.21.15.59:0/3106262282 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e71bc1700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e6c1022b0 0x7f0e6c1084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e71bc1700 1 -- 172.21.15.59:0/3106262282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e6c0f8d50 msgr2=0x7f0e6c10e300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e713c0700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e6c0f8d50 0x7f0e6c10e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:50.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e71bc1700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e6c0f8d50 0x7f0e6c10e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e71bc1700 1 -- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f9620 msgr2=0x7f0e6c10e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e71bc1700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f9620 0x7f0e6c10e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e71bc1700 1 -- 172.21.15.59:0/3106262282 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640097c0 con 0x7f0e6c1022b0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e70bbf700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f9620 0x7f0e6c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e71bc1700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e6c1022b0 0x7f0e6c1084f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0e6c060200 tx=0x7f0e64004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e627fc700 10 client.?.objecter ms_handle_connect 0x7f0e6c1022b0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e627fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e627fc700 1 -- 172.21.15.59:0/3106262282 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e640079b0 con 0x7f0e6c1022b0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e627fc700 1 -- 172.21.15.59:0/3106262282 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e640090d0 con 0x7f0e6c1022b0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5c005040 con 0x7f0e6c1022b0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e627fc700 1 -- 172.21.15.59:0/3106262282 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6400fc90 con 0x7f0e6c1022b0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e73624700 10 client.24632.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c109350 con 0x7f0e6c1022b0 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.437+0000 7f0e73624700 1 librados: init done 2024-02-18T00:31:50.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.438+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c05fa60 con 0x7f0e6c1022b0 2024-02-18T00:31:50.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e627fc700 1 -- 172.21.15.59:0/3106262282 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0e64007b10 con 0x7f0e6c1022b0 2024-02-18T00:31:50.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e627fc700 1 --2- 172.21.15.59:0/3106262282 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e58062c90 0x7f0e58065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:50.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e627fc700 1 -- 172.21.15.59:0/3106262282 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f0e6407d240 con 0x7f0e6c1022b0 2024-02-18T00:31:50.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e627fc700 10 client.24632.objecter ms_dispatch 0x7f0e6c001550 osd_map(90..90 src has 1..90) v4 2024-02-18T00:31:50.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e627fc700 3 client.24632.objecter handle_osd_map got epochs [90,90] > 0 2024-02-18T00:31:50.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e627fc700 3 client.24632.objecter handle_osd_map decoding full epoch 90 2024-02-18T00:31:50.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e713c0700 1 --2- 172.21.15.59:0/3106262282 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e58062c90 0x7f0e58065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:50.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.439+0000 7f0e627fc700 20 client.24632.objecter dump_active .. 0 homeless 2024-02-18T00:31:50.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.440+0000 7f0e627fc700 10 client.24632.objecter ms_handle_connect 0x7f0e58062c90 2024-02-18T00:31:50.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.440+0000 7f0e713c0700 1 --2- 172.21.15.59:0/3106262282 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e58062c90 0x7f0e58065150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0e5c004840 tx=0x7f0e5c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:50.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.443+0000 7f0e627fc700 1 -- 172.21.15.59:0/3106262282 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0e64050f50 con 0x7f0e6c1022b0 2024-02-18T00:31:50.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.616+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f0e6c107290 con 0x7f0e6c1022b0 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.617+0000 7f0e627fc700 1 -- 172.21.15.59:0/3106262282 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) v1 ==== 56+0+4010 (secure 0 0 0) 0x7f0e64050b40 con 0x7f0e6c1022b0 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 90 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-02-18T00:23:11.390869+0000 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-02-18T00:31:34.637557+0000 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:crush_version 18 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:full_ratio 0.95 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:backfillfull_ratio 0.9 2024-02-18T00:31:50.619 INFO:teuthology.orchestra.run.smithi059.stdout:nearfull_ratio 0.85 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:require_min_compat_client luminous 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:min_compat_client luminous 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:require_osd_release pacific 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:stretch_mode_enabled false 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:pool 1 'device_health_metrics' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 67 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr_devicehealth 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:pool 2 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 58 flags hashpspool stripe_width 0 application rgw 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:pool 3 'iscsi' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 60 flags hashpspool stripe_width 0 application rbd 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 60 flags hashpspool stripe_width 0 application rgw 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 62 flags hashpspool stripe_width 0 application rgw 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 64 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:max_osd 8 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 up in weight 1 up_from 8 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6801/3700238977 v2:172.21.15.59:6802/3700238977 exists,up 6e729102-80e2-4a4e-92ca-600ee8b1ed85 2024-02-18T00:31:50.620 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 up in weight 1 up_from 13 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6805/26279110 v2:172.21.15.59:6806/26279110 exists,up f67a7206-6d31-497f-97e0-672d134b13d0 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6809/1153193637 v2:172.21.15.59:6810/1153193637 exists,up d6223dbd-e539-4770-9e5a-7b756304f8f7 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 up in weight 1 up_from 26 up_thru 79 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6813/1805700964 v2:172.21.15.59:6814/1805700964 exists,up fff275df-95e0-41f6-96c3-e531f05de32b 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 up in weight 1 up_from 31 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6800/2044387056 v2:172.21.15.137:6801/2044387056 exists,up a0bae50b-4f29-4310-ac5f-bc7285464231 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 up in weight 1 up_from 39 up_thru 79 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6804/2170432263 v2:172.21.15.137:6805/2170432263 exists,up 513a4192-45fb-4444-a1f9-4bd2f2a5fa01 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 up in weight 1 up_from 47 up_thru 60 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6808/3356009390 v2:172.21.15.137:6809/3356009390 exists,up 5bbb7683-ea5a-490c-8c8b-67b53c45406f 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 up in weight 1 up_from 53 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6812/53025171 v2:172.21.15.137:6813/53025171 exists,up 951d317b-7a34-41a0-b1a1-a2851b2070bb 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.12 [7,2] 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.2 [1,2] 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.f [1,5] 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.17 [1,2] 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/1243102711 expires 2024-02-19T00:29:39.703735+0000 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/817089002 expires 2024-02-19T00:29:39.703735+0000 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3174805070 expires 2024-02-19T00:29:39.703735+0000 2024-02-18T00:31:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/3506000666 expires 2024-02-19T00:24:10.195218+0000 2024-02-18T00:31:50.622 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/4167740619 expires 2024-02-19T00:29:39.703735+0000 2024-02-18T00:31:50.622 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/999852549 expires 2024-02-19T00:24:10.195218+0000 2024-02-18T00:31:50.622 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3575161033 expires 2024-02-19T00:23:45.307159+0000 2024-02-18T00:31:50.622 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2403979355 expires 2024-02-19T00:24:10.195218+0000 2024-02-18T00:31:50.622 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/642821245 expires 2024-02-19T00:23:45.307159+0000 2024-02-18T00:31:50.622 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/1656814063 expires 2024-02-19T00:23:45.307159+0000 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.618+0000 7f0e73624700 10 librados: watch_flush enter 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.618+0000 7f0e73624700 10 librados: watch_flush exit 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.618+0000 7f0e73624700 20 client.24632.objecter shutdown clearing up homeless session... 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.618+0000 7f0e73624700 10 client.24632.objecter successfully canceled tick 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e58062c90 msgr2=0x7f0e58065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 --2- 172.21.15.59:0/3106262282 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e58062c90 0x7f0e58065150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0e5c004840 tx=0x7f0e5c00b040).stop 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e6c1022b0 msgr2=0x7f0e6c1084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e6c1022b0 0x7f0e6c1084f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0e6c060200 tx=0x7f0e64004c50).stop 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 shutdown_connections 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 --2- 172.21.15.59:0/3106262282 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e58062c90 0x7f0e58065150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e6c0f8d50 0x7f0e6c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e6c1022b0 0x7f0e6c1084f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 --2- 172.21.15.59:0/3106262282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e6c0f9620 0x7f0e6c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:50.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 >> 172.21.15.59:0/3106262282 conn(0x7f0e6c0f2820 msgr2=0x7f0e6c0f3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:50.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 shutdown_connections 2024-02-18T00:31:50.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 -- 172.21.15.59:0/3106262282 wait complete. 2024-02-18T00:31:50.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:50.619+0000 7f0e73624700 1 librados: shutdown 2024-02-18T00:31:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:49.511629+0000 osd.3 (osd.3) 59 : cluster [DBG] 2.19 deep-scrub starts 2024-02-18T00:31:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:49.512569+0000 osd.3 (osd.3) 60 : cluster [DBG] 2.19 deep-scrub ok 2024-02-18T00:31:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:49.749231+0000 mgr.y (mgr.24454) 105 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:50.618209+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.59:0/3106262282' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T00:31:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:49.511629+0000 osd.3 (osd.3) 59 : cluster [DBG] 2.19 deep-scrub starts 2024-02-18T00:31:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:49.512569+0000 osd.3 (osd.3) 60 : cluster [DBG] 2.19 deep-scrub ok 2024-02-18T00:31:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:49.749231+0000 mgr.y (mgr.24454) 105 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:50.618209+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.59:0/3106262282' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T00:31:51.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:31:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:31:51.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:31:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:31:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:49.511629+0000 osd.3 (osd.3) 59 : cluster [DBG] 2.19 deep-scrub starts 2024-02-18T00:31:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:49.512569+0000 osd.3 (osd.3) 60 : cluster [DBG] 2.19 deep-scrub ok 2024-02-18T00:31:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:49.749231+0000 mgr.y (mgr.24454) 105 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:31:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:50.618209+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.59:0/3106262282' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T00:31:51.502 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6801/3700238977 v2:172.21.15.59:6802/3700238977 exists,up 6e729102-80e2-4a4e-92ca-600ee8b1ed85', 'osd.1 up in weight 1 up_from 13 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6805/26279110 v2:172.21.15.59:6806/26279110 exists,up f67a7206-6d31-497f-97e0-672d134b13d0', 'osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6809/1153193637 v2:172.21.15.59:6810/1153193637 exists,up d6223dbd-e539-4770-9e5a-7b756304f8f7', 'osd.3 up in weight 1 up_from 26 up_thru 79 down_at 0 last_clean_interval [0,0) v2:172.21.15.59:6813/1805700964 v2:172.21.15.59:6814/1805700964 exists,up fff275df-95e0-41f6-96c3-e531f05de32b', 'osd.4 up in weight 1 up_from 31 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6800/2044387056 v2:172.21.15.137:6801/2044387056 exists,up a0bae50b-4f29-4310-ac5f-bc7285464231', 'osd.5 up in weight 1 up_from 39 up_thru 79 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6804/2170432263 v2:172.21.15.137:6805/2170432263 exists,up 513a4192-45fb-4444-a1f9-4bd2f2a5fa01', 'osd.6 up in weight 1 up_from 47 up_thru 60 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6808/3356009390 v2:172.21.15.137:6809/3356009390 exists,up 5bbb7683-ea5a-490c-8c8b-67b53c45406f', 'osd.7 up in weight 1 up_from 53 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.137:6812/53025171 v2:172.21.15.137:6813/53025171 exists,up 951d317b-7a34-41a0-b1a1-a2851b2070bb'] 2024-02-18T00:31:51.502 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon mon.a config show 2024-02-18T00:31:51.783 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:50.840015+0000 osd.5 (osd.5) 51 : cluster [DBG] 4.f deep-scrub starts 2024-02-18T00:31:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:50.843987+0000 osd.5 (osd.5) 52 : cluster [DBG] 4.f deep-scrub ok 2024-02-18T00:31:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:50.852031+0000 mgr.y (mgr.24454) 106 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:50.985500+0000 osd.1 (osd.1) 53 : cluster [DBG] 2.a deep-scrub starts 2024-02-18T00:31:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:50.988425+0000 osd.1 (osd.1) 54 : cluster [DBG] 2.a deep-scrub ok 2024-02-18T00:31:52.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:50.840015+0000 osd.5 (osd.5) 51 : cluster [DBG] 4.f deep-scrub starts 2024-02-18T00:31:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:50.843987+0000 osd.5 (osd.5) 52 : cluster [DBG] 4.f deep-scrub ok 2024-02-18T00:31:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:50.852031+0000 mgr.y (mgr.24454) 106 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:50.985500+0000 osd.1 (osd.1) 53 : cluster [DBG] 2.a deep-scrub starts 2024-02-18T00:31:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:50.988425+0000 osd.1 (osd.1) 54 : cluster [DBG] 2.a deep-scrub ok 2024-02-18T00:31:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:50.840015+0000 osd.5 (osd.5) 51 : cluster [DBG] 4.f deep-scrub starts 2024-02-18T00:31:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:50.843987+0000 osd.5 (osd.5) 52 : cluster [DBG] 4.f deep-scrub ok 2024-02-18T00:31:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:50.852031+0000 mgr.y (mgr.24454) 106 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:31:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:50.985500+0000 osd.1 (osd.1) 53 : cluster [DBG] 2.a deep-scrub starts 2024-02-18T00:31:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:50.988425+0000 osd.1 (osd.1) 54 : cluster [DBG] 2.a deep-scrub ok 2024-02-18T00:31:52.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:31:52] "GET /metrics HTTP/1.1" 200 172171 "" "Prometheus/2.33.4" 2024-02-18T00:31:53.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:51.185337+0000 osd.6 (osd.6) 39 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:31:53.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:51.186370+0000 osd.6 (osd.6) 40 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:31:53.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:51.749781+0000 mgr.y (mgr.24454) 107 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:53.173 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:51.764124+0000 osd.1 (osd.1) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:53.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:51.764203+0000 osd.1 (osd.1) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:53.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:51.887587+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-18T00:31:53.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:52.065673+0000 osd.4 (osd.4) 49 : cluster [DBG] 2.f scrub starts 2024-02-18T00:31:53.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:52.069178+0000 osd.4 (osd.4) 50 : cluster [DBG] 2.f scrub ok 2024-02-18T00:31:53.174 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:51.185337+0000 osd.6 (osd.6) 39 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:31:53.175 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:51.186370+0000 osd.6 (osd.6) 40 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:31:53.175 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:51.749781+0000 mgr.y (mgr.24454) 107 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:53.175 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:51.764124+0000 osd.1 (osd.1) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:53.175 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:51.764203+0000 osd.1 (osd.1) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:53.175 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:51.887587+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-18T00:31:53.175 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:52.065673+0000 osd.4 (osd.4) 49 : cluster [DBG] 2.f scrub starts 2024-02-18T00:31:53.176 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:52.069178+0000 osd.4 (osd.4) 50 : cluster [DBG] 2.f scrub ok 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:51.185337+0000 osd.6 (osd.6) 39 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:51.186370+0000 osd.6 (osd.6) 40 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:51.749781+0000 mgr.y (mgr.24454) 107 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:51.764124+0000 osd.1 (osd.1) 55 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:51.764203+0000 osd.1 (osd.1) 56 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:51.887587+0000 mon.a (mon.0) 755 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:52.065673+0000 osd.4 (osd.4) 49 : cluster [DBG] 2.f scrub starts 2024-02-18T00:31:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:52.069178+0000 osd.4 (osd.4) 50 : cluster [DBG] 2.f scrub ok 2024-02-18T00:31:53.778 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:31:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mon.a", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster": "ceph", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "admin_socket_mode": "", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_client_required": "cephx, none", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_cluster_required": "cephx", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_debug": "true", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_service_required": "cephx", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_service_ticket_ttl": "240.000000", 2024-02-18T00:31:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_supported": "", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio": "true", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio_max_queue_depth": "1024", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio_poll_ms": "250", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_aio_reap_max": "16", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_async_discard": "false", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_block_size": "4096", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_aio": "false", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_debug_inflight_ios": "false", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_enable_discard": "false", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_flock_retry": "3", 2024-02-18T00:31:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_flock_retry_interval": "0.100000", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_inject_crash": "0", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_inject_crash_flush_delay": "2", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_ioring": "false", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_ioring_hipri": "false", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_ioring_sqthread_poll": "false", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bdev_type": "", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_alloc_size": "1048576", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_allocator": "hybrid", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_buffered_io": "true", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_check_for_zeros": "false", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_check_volume_selector_often": "false", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-02-18T00:31:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_compact_log_sync": "false", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_log_compact_min_size": "16777216", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_log_replay_check_allocations": "true", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_max_log_runway": "4194304", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_max_prefetch": "1048576", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_min_flush_size": "524288", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_min_log_runway": "1048576", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_replay_recovery": "false", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_shared_alloc_size": "65536", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_sync_write": "false", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-02-18T00:31:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_allocator": "hybrid", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blobid_prealloc": "10240", 2024-02-18T00:31:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_create": "true", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_db_create": "false", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_db_path": "", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_db_size": "0", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_path": "", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_preallocate_file": "false", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_size": "107374182400", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_wal_create": "false", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_wal_path": "", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_block_wal_size": "100663296", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bluefs": "true", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-02-18T00:31:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_bluefs_env_mirror": "false", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_autotune": "true", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_size": "0", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-02-18T00:31:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_type": "2q", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_clone_cow": "true", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_algorithm": "snappy", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_max_blob_size": "0", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_min_blob_size": "0", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_mode": "none", 2024-02-18T00:31:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_csum_type": "crc32c", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_enforce_settings": "default", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_freelist": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_fsck_abort": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_inject_bug21040": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_inject_read_err": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_legacy_omap": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_prefill": "0.000000", 2024-02-18T00:31:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_small_allocations": "0", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_default_buffered_read": "true", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_default_buffered_write": "false", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_deferred_batch_ops": "0", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-02-18T00:31:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mkfs": "true", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mount": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_umount": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-02-18T00:31:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_ignore_data_csum": "false", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_kvbackend": "rocksdb", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_log_op_age": "5.000000", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_alloc_size": "0", 2024-02-18T00:31:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_blob_size": "0", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_defer_interval": "3.000000", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_max_deferred_txc": "32", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_min_alloc_size": "0", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_nid_prealloc": "1024", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_prefer_deferred_size": "0", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_retry_disk_reads": "3", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_cf": "true", 2024-02-18T00:31:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_rocksdb_options_annex": "", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_coremask": "0x1", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_io_sleep": "5", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_max_io_completion": "0", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_spdk_mem": "512", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_sync_submit_transaction": "false", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_bytes": "67108864", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_cost_per_io": "0", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-02-18T00:31:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_tracing": "false", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_volume_selection_reserved": "0", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_action_update_interval": "2", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-02-18T00:31:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_mount_timeout": "10", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_cluster_require_signatures": "false", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_cluster_require_version": "2", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_require_signatures": "false", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_require_version": "2", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_service_require_signatures": "false", 2024-02-18T00:31:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_service_require_version": "2", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "cephx_sign_messages": "true", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "chdir": "", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_acl_type": "", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_asio_thread_count": "2", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_cache_mid": "0.750000", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_cache_size": "16384", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_caps_release_delay": "5", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_check_pool_perm": "true", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_collect_and_send_global_metrics": "false", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_debug_force_sync_read": "false", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_debug_getattr_caps": "false", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_debug_inject_tick_delay": "0", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-02-18T00:31:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_die_on_failed_remount": "false", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_dirsize_rbytes": "true", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_force_lazyio": "false", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_fs": "", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_inject_fixed_oldest_tid": "false", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_inject_release_failure": "false", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_max_inline_size": "4096", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_max_retries_on_remount_failure": "5", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mds_namespace": "", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_metadata": "", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mount_gid": "-1", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mount_timeout": "300", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mount_uid": "-1", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_mountpoint": "/", 2024-02-18T00:31:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "client_notify_timeout": "10", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc": "true", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_max_dirty": "104857600", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_max_dirty_age": "5.000000", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_max_objects": "1000", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_size": "209715200", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_oc_target_dirty": "8388608", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_permissions": "true", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_quota": "true", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_quota_df": "true", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_readahead_max_bytes": "0", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_readahead_max_periods": "4", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_readahead_min": "131072", 2024-02-18T00:31:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_reconnect_stale": "false", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_shutdown_timeout": "30", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_snapdir": ".snap", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_tick_interval": "1", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_trace": "", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_try_dentry_invalidate": "false", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_use_faked_inos": "false", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_use_random_mds": "false", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_graylog": "false", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_graylog_port": "12201", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_monitors": "default=true", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_syslog": "false", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "clog_to_syslog_level": "info", 2024-02-18T00:31:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "-", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_network": "", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_network_interface": "", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zlib_isal": "false", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zlib_level": "5", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zlib_winsize": "-15", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "compressor_zstd_level": "1", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "crimson_osd_obc_lru_size": "10", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_location": "", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_location_hook": "", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_location_hook_timeout": "10", 2024-02-18T00:31:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "daemonize": "false", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_allow_any_pool_priority": "false", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_asok": "1/5", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_asok_assert_abort": "false", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_asserts_on_shutdown": "true", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_auth": "1/5", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_bdev": "1/3", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_bluefs": "1/5", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_bluestore": "1/5", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_buffer": "0/1", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_cephfs_mirror": "0/5", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_cephsqlite": "0/5", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_civetweb": "1/10", 2024-02-18T00:31:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_client": "0/5", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_compressor": "1/5", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_context": "0/1", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_crush": "1/1", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_crypto": "1/5", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_deliberately_leak_memory": "false", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_disable_randomized_ping": "false", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_dpdk": "1/5", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_eventtrace": "1/5", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_filer": "0/1", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_filestore": "1/3", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_finisher": "1/1", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_fuse": "1/5", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_heartbeat_testing_span": "0", 2024-02-18T00:31:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_heartbeatmap": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_immutable_obj_cache": "0/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_javaclient": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_journal": "1/3", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_journaler": "0/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_kstore": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_leveldb": "4/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_lockdep": "0/1", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_balancer": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_locker": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_log": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_log_expire": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mds_migrator": "1/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_memdb": "4/5", 2024-02-18T00:31:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mgr": "2/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mgrc": "20/20", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_mon": "20/20", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_monc": "0/10", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_ms": "1/1", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_none": "0/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_objclass": "0/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_objectcacher": "0/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_objecter": "0/1", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_optracker": "0/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_osd": "1/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_paxos": "20/20", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_perfcounter": "1/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_prioritycache": "1/5", 2024-02-18T00:31:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rados": "0/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd": "0/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd_mirror": "0/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd_pwl": "0/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rbd_replay": "0/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_refs": "0/0", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_reserver": "1/1", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw": "1/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rgw_sync": "1/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_rocksdb": "4/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_striper": "0/1", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_test": "0/5", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_throttle": "1/1", 2024-02-18T00:31:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_timer": "0/1", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "debug_tp": "0/5", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "defer_client_eviction_on_laggy_osds": "true", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "device_failure_prediction_mode": "none", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "err_to_graylog": "false", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "err_to_stderr": "true", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "err_to_syslog": "false", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event_tracing": "false", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "fake_statfs_for_testing": "0", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "fatal_signal_handlers": "true", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "filer_max_purge_ops": "10", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "filer_max_truncate_ops": "128", 2024-02-18T00:31:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_apply_finisher_threads": "1", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_blackhole": "false", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_btrfs_clone_range": "true", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_btrfs_snap": "true", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_caller_concurrency": "10", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_collect_device_partition_information": "true", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_commit_timeout": "600.000000", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_inject_read_err": "false", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_omap_check": "false", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_random_read_err": "0.000000", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_debug_verify_split": "false", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_dump_file": "", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fadvise": "true", 2024-02-18T00:31:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fail_eio": "true", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fd_cache_shards": "16", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fd_cache_size": "128", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fiemap": "false", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fiemap_threshold": "4096", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_index_retry_probability": "0.000000", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_inject_stall": "0", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_journal_parallel": "false", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_journal_trailing": "false", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_journal_writeahead": "false", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_kill_at": "0", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size": "0", 2024-02-18T00:31:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs": "0", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs_other": "2", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_sync_interval": "5.000000", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size": "0", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_merge_threshold": "-10", 2024-02-18T00:31:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_min_sync_interval": "0.010000", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_odsync_write": "false", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_omap_backend": "rocksdb", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_omap_backend_path": "", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_omap_header_cache_size": "1024", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_ondisk_finisher_threads": "1", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_op_thread_timeout": "60", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_op_threads": "2", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_punch_hole": "false", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-02-18T00:31:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_bytes": "104857600", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_queue_max_ops": "50", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_seek_data_hole": "false", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_sloppy_crc": "false", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-02-18T00:31:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_splice": "false", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_split_multiple": "2", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_split_rand_factor": "20", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_update_to": "1000", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_enable": "true", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-02-18T00:31:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_xfs_extsize": "false", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "filestore_zfs_snap": "false", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fio_dir": "/tmp/fio", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_allow_other": "true", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_atomic_o_trunc": "true", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_big_writes": "true", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_debug": "false", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_default_permissions": "false", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_disable_pagecache": "false", 2024-02-18T00:31:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_max_write": "0", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_multithreaded": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_require_active_mds": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_set_user_groups": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_splice_move": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_splice_read": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_splice_write": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_syncfs_on_mksnap": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "fuse_use_invalidate_cb": "true", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "gss_target_name": "ceph", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_file": "", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_inject_failure": "0", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_interval": "5", 2024-02-18T00:31:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "host": "", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_max_size": "1073741824", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-02-18T00:31:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "immutable_object_cache_watermark": "0.900000", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "inject_early_sigterm": "false", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_aio": "true", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_align_min_size": "65536", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_block_align": "true", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_block_size": "4096", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_dio": "true", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_discard": "false", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_force_aio": "false", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_ignore_corruption": "false", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_max_write_bytes": "10485760", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_max_write_entries": "100", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_replay_from": "0", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_high_multiple": "0.000000", 2024-02-18T00:31:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_throttle_max_multiple": "0.000000", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_write_header_frequency": "0", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "journal_zero_on_create": "false", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "journaler_prefetch_periods": "10", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "journaler_prezero_periods": "5", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "journaler_write_head_interval": "15", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "keyfile": "", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_backend": "rocksdb", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_default_stripe_size": "65536", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_fsck_on_mount": "false", 2024-02-18T00:31:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_fsck_on_mount_deep": "true", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_max_bytes": "67108864", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_max_ops": "512", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_nid_prealloc": "1024", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_onode_map_size": "1024", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_sync_submit_transaction": "false", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "kstore_sync_transaction": "false", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_block_size": "65536", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_bloom_size": "0", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_cache_size": "536870912", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_compact_on_mount": "false", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_compression": "false", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_log": "", 2024-02-18T00:31:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_log_to_ceph_log": "true", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_max_open_files": "0", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_paranoid": "false", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "leveldb_write_buffer_size": "33554432", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "librados_thread_count": "2", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "lockdep": "false", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "lockdep_force_backtrace": "false", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "log_coarse_timestamps": "true", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "log_flush_on_exit": "false", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "log_graylog_host": "127.0.0.1", 2024-02-18T00:31:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "log_graylog_port": "12201", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_max_new": "1000", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_max_recent": "10000", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_stderr_prefix": "debug ", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_stop_at_utilization": "0.970000", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": "true", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_graylog": "false", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_stderr": "false", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_syslog": "false", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "max_rotating_auth_attempts": "10", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_action_on_write_error": "1", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_alternate_name_max": "8192", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_asio_thread_count": "2", 2024-02-18T00:31:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_export_pin": "true", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_dirs": "true", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_interval": "5", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_fragment_size_max": "100000", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_idle_threshold": "0.000000", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_interval": "10", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_max": "-1", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_max_until": "-1", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_merge_size": "50", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_midchunk": "0.300000", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_min_rebalance": "0.100000", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_min_start": "0.200000", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_minchunk": "0.001000", 2024-02-18T00:31:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_mode": "0", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_need_max": "1.200000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_need_min": "0.800000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_sample_interval": "3.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_bits": "3", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_rd": "25000.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_size": "10000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_split_wr": "10000.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_target_decay": "10.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_beacon_grace": "15.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_beacon_interval": "4.000000", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_beacon_mon_down_grace": "60", 2024-02-18T00:31:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_memory_limit": "4294967296", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_mid": "0.700000", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_release_free_interval": "10", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_reservation": "0.050000", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_trim_interval": "1", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache_trim_threshold": "262144", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_client_delegate_inos_pct": "50", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_client_prealloc_inos": "1000", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_connect_bootstrapping": "false", 2024-02-18T00:31:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_damage_table_max_entries": "10000", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_auth_pins": "false", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_frag": "false", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_scatterstat": "false", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_debug_subtrees": "false", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_decay_halflife": "5.000000", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_default_dir_hash": "2", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_defer_session_stale": "true", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_deny_all_reconnect": "false", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dir_keys_per_op": "16384", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dir_max_commit_size": "10", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dir_max_entries": "0", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dirstat_min_interval": "1.000000", 2024-02-18T00:31:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_after_rejoin": "false", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_on_map": "false", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_threshold_file": "0", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_early_reply": "true", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_enable_op_tracker": "true", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_enforce_unique_name": "true", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_distributed": "true", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_random": "true", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_forward_all_requests_to_auth": "false", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-02-18T00:31:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_health_cache_threshold": "1.500000", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_health_summarize_threshold": "10", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_heartbeat_grace": "15.000000", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_heartbeat_reset_grace": "1000", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_health_dummy": "false", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_migrator_session_race": "false", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_join_fs": "", 2024-02-18T00:31:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_journal_format": "1", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_create_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_export_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_import_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_journal_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_journal_expire_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_journal_replay_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_link_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_mdstable_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_openc_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_rename_at": "0", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_events_per_segment": "1024", 2024-02-18T00:31:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_max_events": "-1", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_max_segments": "128", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_pause": "false", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_segment_size": "0", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_skip_corrupt_events": "false", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log_warn_factor": "2.000000", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_caps_per_client": "1048576", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_completed_flushes": "100000", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_completed_requests": "100000", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_export_size": "20971520", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_file_recover": "32", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_purge_files": "64", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_purge_ops": "8192", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-02-18T00:31:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_snaps_per_dir": "100", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_max_xattr_pairs_size": "65536", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_metrics_update_interval": "2", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_min_caps_per_client": "100", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_min_caps_working_set": "10000", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_numa_node": "-1", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_complaint_time": "30.000000", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_history_duration": "600", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_history_size": "20", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_op_log_threshold": "5", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_ping_grace": "15", 2024-02-18T00:31:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_ping_interval": "5", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_max_caps": "30000", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_max_decay_threshold": "131072", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_recall_warning_threshold": "262144", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_reconnect_timeout": "45.000000", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_replay_interval": "1.000000", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_root_ino_gid": "0", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_root_ino_uid": "0", 2024-02-18T00:31:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_blocklist_on_evict": "true", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_blocklist_on_timeout": "true", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_session_metadata_threshold": "16777216", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_shutdown_check": "0", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_skip_ino": "0", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sleep_rank_change": "0.000000", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_snap_max_uid": "4294967294", 2024-02-18T00:31:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_snap_min_uid": "0", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_snap_rstat": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_standby_replay_damaged": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_task_status_update_interval": "2.000000", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_thrash_exports": "0", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_thrash_fragments": "0", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_tick_interval": "5.000000", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_valgrind_exit": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_verify_backtrace": "1", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_verify_scatter": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_wipe_ino_prealloc": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_wipe_sessions": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool_debug": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_debug_omit_block_device_write": "false", 2024-02-18T00:31:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_device_bytes": "1073741824", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_page_set": "false", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "memstore_page_size": "65536", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_client_bytes": "134217728", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_client_messages": "512", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_connect_retry_interval": "1.000000", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_disabled_modules": "", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_max_pg_num_change": "128", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mds_bytes": "134217728", 2024-02-18T00:31:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mds_messages": "128", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mon_bytes": "134217728", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_mon_messages": "128", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_osd_bytes": "536870912", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_osd_messages": "8192", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_service_beacon_grace": "60.000000", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_standby_modules": "true", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_stats_period": "5", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_stats_threshold": "5", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_tick_period": "2", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_accept_timeout_factor": "2.000000", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_allow_pool_delete": "true", 2024-02-18T00:31:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_allow_pool_size_one": "false", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_auth_validate_all_caps": "true", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_bytes": "104857600", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_directed_command_retry": "2", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval": "3.000000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_hunt_parallel": "3", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_log_interval": "1.000000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_ping_interval": "10.000000", 2024-02-18T00:31:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_client_ping_timeout": "30.000000", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_clock_drift_allowed": "1.000000", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_file_level": "debug", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_file": "true", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_graylog": "false", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_stderr": "true", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_compact_on_bootstrap": "false", 2024-02-18T00:31:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_compact_on_start": "false", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_compact_on_trim": "true", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_con_tracker_persist_interval": "10", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_con_tracker_score_halflife": "43200", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_config_key_max_entry_size": "65536", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_cpu_threads": "4", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_crush_min_required_version": "hammer", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_daemon_bytes": "419430400", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data_avail_crit": "5", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data_avail_warn": "5", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_data_size_warn": "16106127360", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_block_osdmap_trim": "false", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-02-18T00:31:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_dump_json": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_dump_transactions": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_extra_checks": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_require_octopus": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_no_require_pacific": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_delta_reset_interval": "10.000000", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_dns_srv_name": "ceph-mon", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_down_mkfs_grace": "60", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_election_default_strategy": "1", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_election_timeout": "5.000000", 2024-02-18T00:31:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_elector_ping_divisor": "2", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_elector_ping_timeout": "2.000000", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_enable_op_tracker": "true", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_fake_pool_delete": "false", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_force_quorum_join": "false", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_globalid_prealloc": "10000", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_detail_to_clog": "true", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_log_update_period": "5", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_max_detail": "50", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_to_clog": "true", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_to_clog_interval": "600", 2024-02-18T00:31:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_host": "[v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0]", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_host_override": "", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_initial_members": "", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_keyvaluedb": "rocksdb", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_lease": "5.000000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_log_max_summary": "50", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_log_entries_per_event": "4096", 2024-02-18T00:31:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_log_epochs": "500", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_mdsmap_epochs": "500", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_mgrmap_epochs": "500", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_osd": "10000", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_pg_per_osd": "10000", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_pool_pg_num": "65536", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mds_force_trim_to": "0", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mds_skip_sanity": "false", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_memory_autotune": "true", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_memory_target": "2147483648", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_beacon_grace": "30", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-02-18T00:31:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_digest_period": "5", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_inactive_grace": "60", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_mkfs_grace": "240", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_min_osdmap_epochs": "50", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_complaint_time": "30", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_duration": "600", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_size": "20", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_slow_op_size": "20", 2024-02-18T00:31:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_history_slow_op_threshold": "10", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_op_log_threshold": "5", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_auto_mark_in": "false", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_auto_mark_new_in": "true", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_cache_size": "500", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_cache_size_min": "134217728", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_crush_smoke_test": "true", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_destroyed_out_interval": "600", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_down_out_interval": "600", 2024-02-18T00:31:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_force_trim_to": "0", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_full_ratio": "0.950000", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_laggy_halflife": "3600", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_laggy_max_interval": "300", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_laggy_weight": "0.300000", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_max_creating_pgs": "1024", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_max_initial_pgs": "1024", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_min_down_reporters": "2", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-02-18T00:31:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_prime_pg_temp": "true", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_report_timeout": "900", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_warn_num_repaired": "10", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osd_warn_op_age": "32.000000", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_interval": "2", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_min": "15", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_stuck_threshold": "60", 2024-02-18T00:31:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_min_objects": "10000", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_min_per_osd": "0", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pool_quota_crit_threshold": "0", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_pool_quota_warn_threshold": "0", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_probe_timeout": "2.000000", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_max_change": "0.050000", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_max_osds": "4", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-02-18T00:31:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_interval": "86400", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_max_keys": "100", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_scrub_timeout": "300", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_session_timeout": "300", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_smart_report_timeout": "5", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stat_smooth_intervals": "6", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_pool_min_size": "2", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_pool_size": "4", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_subscribe_interval": "86400.000000", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_debug": "false", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_max_payload_keys": "2000", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_max_payload_size": "1048576", 2024-02-18T00:31:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_provider_kill_at": "0", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_requester_kill_at": "0", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_sync_timeout": "60.000000", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_target_pg_per_osd": "100", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_tick_interval": "5", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_timecheck_interval": "300.000000", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_older_version_delay": "604800", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-02-18T00:31:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_misplaced": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_older_version": "true", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_pool_no_app": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_on_too_few_osds": "false", 2024-02-18T00:31:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "monmap": "", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_op_threads": "3", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_buffer_size": "131072", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_cm": "false", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_device_name": "", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_dscp": "96", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_gid_idx": "0", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_local_gid": "", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_polling_us": "1000", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_port_num": "1", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-02-18T00:31:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_roce_ver": "1", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_send_buffers": "1024", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_sl": "3", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_support_srq": "true", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_rdma_type": "ib", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_async_reap_threshold": "5", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_before_connect": "false", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_ipv4": "true", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_ipv6": "false", 2024-02-18T00:31:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_msgr1": "false", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_msgr2": "true", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_port_max": "7300", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_port_min": "6800", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_prefer_ipv4": "false", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_retry_count": "3", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_bind_retry_delay": "5", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_client": "false", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_mds": "false", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_mgr": "false", 2024-02-18T00:31:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_mon": "false", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_blackhole_osd": "false", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_client_mode": "crc secure", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_cluster_mode": "crc secure", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_cluster_type": "", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_connection_idle_timeout": "900", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_connection_ready_timeout": "10", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_crc_data": "true", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_crc_header": "true", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_bad_msg": "false", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_bug": "true", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_old_message": "true", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_skipped_message": "false", 2024-02-18T00:31:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_die_on_unhandled_msg": "false", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_coremask": "0xF", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_hugepages": "", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_hw_flow_control": "true", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_lro": "true", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_memory_channel": "4", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_pmd": "", 2024-02-18T00:31:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_port_id": "0", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dump_corrupt_message_level": "1", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_dump_on_send": "false", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_initial_backoff": "0.200000", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_delay_max": "1.000000", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_delay_probability": "0.000000", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_delay_type": "", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_internal_delays": "0.000000", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_inject_socket_failures": "0", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_learn_addr_from_peer": "true", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_max_accept_failures": "4", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_max_backoff": "15.000000", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_mon_client_mode": "secure crc", 2024-02-18T00:31:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_mon_cluster_mode": "secure crc", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_mon_service_mode": "secure crc", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_pq_min_cost": "65536", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_public_type": "", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_service_mode": "crc secure", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_listen_backlog": "512", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_nodelay": "true", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_tcp_rcvbuf": "0", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "ms_type": "async+posix", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "no_config_file": "false", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_completion_locks_per_session": "32", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_debug_inject_relock_delay": "false", 2024-02-18T00:31:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_inflight_op_bytes": "104857600", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_inflight_ops": "1024", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_inject_no_watch_ping": "false", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_tick_interval": "5.000000", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter_timeout": "10.000000", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "objectstore_blackhole": "false", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "openssl_engine_opts": "", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_delay_time": "5.000000", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_hist_halflife": "1000", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_max_low_ops": "2", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_max_ops": "4", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_quantize_effort": "0.100000", 2024-02-18T00:31:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_agent_slop": "0.020000", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_allow_recovery_below_min_size": "true", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_asio_thread_count": "2", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_async_recovery_min_cost": "100", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_auto_mark_unfound_lost": "false", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backfill_retry_interval": "30.000000", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backfill_scan_max": "512", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backfill_scan_min": "64", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backoff_on_degraded": "false", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backoff_on_peering": "false", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_backoff_on_unfound": "true", 2024-02-18T00:31:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_beacon_report_interval": "300", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_duration": "30", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_max_block_size": "67108864", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bench_small_size_max_iops": "100", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_blkin_trace_all": "false", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_check_for_log_corruption": "false", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-02-18T00:31:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_class_update_on_start": "true", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_message_cap": "256", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_message_size_cap": "524288000", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_op_priority": "63", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_client_watch_timeout": "30", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_command_max_records": "256", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_command_thread_suicide_timeout": "900", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_command_thread_timeout": "600", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_compact_on_start": "false", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_chooseleaf_type": "1", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_initial_weight": "-1.000000", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_update_on_start": "true", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_crush_update_weight_set": "true", 2024-02-18T00:31:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_drop_ping_duration": "0", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_feed_pullee": "-1", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_misdirected_ops": "false", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_no_acting_change": "false", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_no_purge_strays": "false", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_op_order": "false", 2024-02-18T00:31:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_pg_log_writeout": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_pretend_recovery_active": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_shutdown": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_cached_snaps": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_missing_on_start": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_snaps": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_keys": "1024", 2024-02-18T00:31:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_stride": "524288", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_default_data_pool_replay_window": "45", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_default_notify_timeout": "30", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep": "0.000000", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_discard_disconnected_ops": "true", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_enable_op_tracker": "true", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-02-18T00:31:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_info": "true", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_shutdown": "true", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_function_tracing": "false", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_grace": "20", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_interval": "6", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_min_peers": "10", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_min_size": "2000", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_stale": "600", 2024-02-18T00:31:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_hit_set_max_size": "100000", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_hit_set_min_size": "1000", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_journal_flush_on_shutdown": "true", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_journal_size": "5120", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_kill_backfill_at": "0", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_loop_before_reset_tphandle": "64", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_cache_size": "50", 2024-02-18T00:31:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_dedup": "true", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_message_max": "40", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_message_max_bytes": "10485760", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_map_share_max_epochs": "40", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_attr_name_len": "100", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_attr_size": "0", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_backfills": "1", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_markdown_count": "5", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_markdown_period": "600", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_object_name_len": "2048", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_object_namespace_len": "256", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_object_size": "134217728", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_omap_entries_per_request": "1024", 2024-02-18T00:31:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pg_blocked_by": "16", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pg_log_entries": "10000", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_pgls": "1024", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_push_cost": "8388608", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_push_objects": "10", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_scrubs": "1", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_trimming_pgs": "2", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_write_op_reply_len": "64", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_max_write_size": "90", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_cost_per_byte_usec": "0.000000", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_cost_per_byte_usec_hdd": "5.200000", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_cost_per_byte_usec_ssd": "0.011000", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_cost_per_io_usec": "0.000000", 2024-02-18T00:31:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_cost_per_io_usec_hdd": "25000.000000", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_cost_per_io_usec_ssd": "50.000000", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_profile": "high_client_ops", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-02-18T00:31:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_client_lim": "999999", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_client_res": "1", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mclock_skip_benchmark": "false", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_base": "805306368", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_cache_min": "134217728", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_target": "4294967296", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_target_autotune": "false", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_min_pg_log_entries": "250", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_min_recovery_priority": "0", 2024-02-18T00:31:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_heartbeat_interval": "30", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_report_interval": "5", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_report_max_in_flight": "2", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_num_cache_shards": "32", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_num_op_tracker_shard": "32", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_numa_auto_affinity": "true", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_numa_node": "-1", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_numa_prefer_iface": "true", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objecter_finishers": "1", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objectstore": "bluestore", 2024-02-18T00:31:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objectstore_fuse": "false", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_objectstore_tracing": "false", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_complaint_time": "30.000000", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_duration": "600", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_size": "20", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_slow_op_size": "20", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_log_threshold": "5", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_shards": "0", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_shards_hdd": "5", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_shards_ssd": "8", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_threads_per_shard": "0", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-02-18T00:31:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_pq_min_cost": "65536", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_queue": "wpq", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_queue_cut_off": "high", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_thread_suicide_timeout": "150", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_op_thread_timeout": "15", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_open_classes_on_start": "true", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_os_flags": "0", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_peering_op_priority": "255", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_delete_cost": "1048576", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_delete_priority": "5", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-02-18T00:31:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_log_dups_tracked": "3000", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_log_trim_max": "10000", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_log_trim_min": "100", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_object_context_cache_count": "64", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pg_stat_report_interval_max": "500", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_crush_rule": "-1", 2024-02-18T00:31:53.872 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_ec_fast_read": "false", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_bulk": "false", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_nodelete": "false", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_flags": "0", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_min_size": "0", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_pg_num": "32", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_pgp_num": "0", 2024-02-18T00:31:53.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_size": "3", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_default_type": "replicated", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pool_use_gmt_hitset": "true", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_push_per_object_cost": "1000", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_read_ec_check_for_errors": "false", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recover_clone_overlap": "true", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recover_clone_overlap_limit": "10", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_cost": "20971520", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_delay_start": "0.000000", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_active": "0", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_active_hdd": "3", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_active_ssd": "10", 2024-02-18T00:31:53.874 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_chunk": "8388608", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_max_single_start": "1", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_op_priority": "3", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_op_warn_multiple": "16", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_priority": "5", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_retry_interval": "30.000000", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep": "0.000000", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_repair_during_recovery": "false", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_requested_scrub_priority": "120", 2024-02-18T00:31:53.875 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_rollback_to_cluster_snap": "", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_auto_repair": "false", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_begin_hour": "0", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_begin_week_day": "0", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_chunk_max": "25", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_chunk_min": "5", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_cost": "52428800", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_during_recovery": "false", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_end_hour": "0", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_end_week_day": "0", 2024-02-18T00:31:53.876 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_invalid_stats": "true", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_load_threshold": "0.500000", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_max_interval": "604800.000000", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_max_preemptions": "5", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_min_interval": "86400.000000", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_priority": "5", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_scrub_sleep": "0.000000", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_shutdown_pgref_assert": "false", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_skip_data_digest": "false", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_smart_report_timeout": "5", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_cost": "1048576", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_priority": "5", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep": "0.000000", 2024-02-18T00:31:53.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_target_transaction_size": "30", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_default_cache_mode": "writeback", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-02-18T00:31:53.878 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_tracing": "false", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_use_stale_snap": "false", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "osdc_blkin_trace_all": "false", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_kill_at": "0", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_max_join_drift": "10", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_min": "500", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_min_wait": "0.050000", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_propose_interval": "1.000000", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_service_trim_max": "500", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_service_trim_max_multiplier": "20", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_service_trim_min": "10", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_stash_full_interval": "25", 2024-02-18T00:31:53.879 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_trim_max": "500", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "paxos_trim_min": "250", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "perf": "true", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "pid_file": "", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "-", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrv": "", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "public_bind_addr": "-", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "public_network": "", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "public_network_interface": "", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "qat_compressor_enabled": "false", 2024-02-18T00:31:53.880 INFO:teuthology.orchestra.run.smithi059.stdout: "rados_mon_op_timeout": "0", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rados_osd_op_timeout": "0", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rados_tracing": "false", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_atime_update_interval": "60", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_balance_parent_reads": "false", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_balance_snap_reads": "false", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_blkin_trace_all": "false", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_blocklist_expire_seconds": "0", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_blocklist_on_break_lock": "true", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache": "true", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_block_writes_upfront": "false", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_max_dirty": "25165824", 2024-02-18T00:31:53.881 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_max_dirty_object": "0", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_policy": "writearound", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_size": "33554432", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_target_dirty": "16777216", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_clone_copy_on_read": "false", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_compression_hint": "none", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_concurrent_management_ops": "10", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_clone_format": "auto", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_data_pool": "", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_features": "61", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_format": "2", 2024-02-18T00:31:53.882 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_map_options": "", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_order": "22", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_pool": "rbd", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_stripe_count": "0", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_default_stripe_unit": "0", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_disable_zero_copy_writes": "true", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_discard_granularity_bytes": "65536", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_enable_alloc_hint": "true", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_io_scheduler": "simple", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_commit_age": "5.000000", 2024-02-18T00:31:53.883 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_flush_interval": "0", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_order": "24", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_pool": "", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_journal_splay_width": "4", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_localize_parent_reads": "false", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_localize_snap_reads": "false", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-02-18T00:31:53.884 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_type": "simple", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_autotune": "true", 2024-02-18T00:31:53.885 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_base": "805306368", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_memory_target": "4294967296", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_delete_delay": "0", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_replay_delay": "0", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-02-18T00:31:53.886 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_move_to_trash_on_remove": "false", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_mtime_update_interval": "60", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_non_blocking_aio": "true", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_op_thread_timeout": "60", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_op_threads": "1", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_parent_cache_enabled": "false", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_persistent_cache_mode": "disabled", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_persistent_cache_path": "/tmp", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_persistent_cache_size": "1073741824", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_plugins": "", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_bps_burst": "0", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-02-18T00:31:53.887 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_bps_limit": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_iops_burst": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_iops_limit": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_bps_burst": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_bps_limit": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_iops_burst": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_read_iops_limit": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_schedule_tick_min": "50", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_bps_burst": "0", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-02-18T00:31:53.888 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_bps_limit": "0", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_iops_burst": "0", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_qos_write_iops_limit": "0", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_quiesce_notification_attempts": "10", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_read_from_replica_policy": "default", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_readahead_max_bytes": "524288", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_readahead_trigger_requests": "10", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_request_timed_out_seconds": "30", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_skip_partial_discard": "true", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_tracing": "false", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_validate_names": "true", 2024-02-18T00:31:53.889 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_validate_pool": "true", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "restapi_base_url": "", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "restapi_log_level": "", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_acl_grants_max_num": "100", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_admin_entry": "admin", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_barbican_url": "", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_index_max_aio": "128", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_bucket_quota_ttl": "600", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cache_enabled": "true", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cache_expiry_interval": "900", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cache_lru_size": "10000", 2024-02-18T00:31:53.890 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_content_length_compat": "false", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_copy_obj_progress": "true", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cors_rules_max_num": "100", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_cross_domain_policy": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_default_encryption_key": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_addr": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_ca_path": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_client_cert": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_client_key": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_password": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_kmip_username": "", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_require_ssl": "true", 2024-02-18T00:31:53.891 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_suppress_logs": "true", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_addr": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_auth": "token", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_namespace": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_prefix": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_token_file": "", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-02-18T00:31:53.892 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_buffersize": "524288", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_low_speed_limit": "1024", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_low_speed_time": "300", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_changes_size": "1000", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_num_shards": "128", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_log_window": "30", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_data_notify_interval_msec": "200", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_data_log_backing": "fifo", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-02-18T00:31:53.893 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_region_info_oid": "default.region", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_defer_to_bucket_acls": "", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_data_res": "500.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-02-18T00:31:53.894 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dns_name": "", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dns_s3website_name": "", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_dynamic_resharding": "true", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_gc_threads": "true", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_lc_threads": "true", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_ops_log": "false", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_quota_threads": "true", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_static_website": "false", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enable_usage_log": "false", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_enforce_swift_acls": "true", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_exit_timeout_secs": "120", 2024-02-18T00:31:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_expose_bucket": "false", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_extended_http_attrs": "", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_fcgi_socket_backlog": "1024", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_frontends": "beast port=7480", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_concurrent_io": "10", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_deferred": "50", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_objs": "32", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_queue_size": "134213632", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_max_trim_chunk": "16", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_obj_min_wait": "7200", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_processor_max_time": "3600", 2024-02-18T00:31:53.896 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_gc_processor_period": "3600", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_get_obj_window_size": "16777216", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_healthcheck_disabling_path": "", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_host": "", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ignore_get_invalid_range": "false", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_init_timeout": "300", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_domain": "", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_password": "", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_password_path": "", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_project": "", 2024-02-18T00:31:53.897 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_tenant": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_token": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_token_path": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_admin_user": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_api_version": "2", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_domain": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_password": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_project": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_tenant": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_barbican_user": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_implicit_tenants": "false", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_token_cache_size": "10000", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_url": "", 2024-02-18T00:31:53.898 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_keystone_verify_ssl": "true", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_debug_interval": "-1", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_lock_max_time": "90", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_objs": "32", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_rules": "1000", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_worker": "3", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_max_wp_worker": "3", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lc_thread_delay": "0", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_dnattr": "uid", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_searchfilter": "", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ldap_uri": "ldaps://", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-02-18T00:31:53.899 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_http_headers": "", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_nonexistent_bucket": "false", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_log_object_name_utc": "false", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_attr_name_len": "0", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_attr_size": "0", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_attrs_num_in_req": "0", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_chunk_size": "4194304", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_concurrent_requests": "1024", 2024-02-18T00:31:53.900 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_dynamic_shards": "1999", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_listing_results": "1000", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_notify_retries": "10", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_objs_per_shard": "100000", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_put_param_size": "1048576", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_put_size": "5368709120", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_max_slo_entries": "1000", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_md_log_max_shards": "64", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_md_notify_interval_msec": "200", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_mp_lock_max_time": "600", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_multipart_min_part_size": "5242880", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-02-18T00:31:53.901 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_fhcache_size": "2017", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_lru_lanes": "5", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_max_gc": "300", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_gc_threads": "false", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_lc_threads": "false", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_quota_threads": "false", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_run_sync_thread": "false", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_num_async_rados_threads": "32", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_num_control_oids": "8", 2024-02-18T00:31:53.902 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_num_rados_handles": "1", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_numa_node": "-1", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_obj_stripe_size": "4194304", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_objexp_chunk_size": "100", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_objexp_gc_interval": "600", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_objexp_hints_num_shards": "127", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_op_thread_timeout": "600", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_op_tracing": "false", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_opa_token": "", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_opa_url": "", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_opa_verify_ssl": "true", 2024-02-18T00:31:53.903 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_file_path": "", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_rados": "true", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_ops_log_socket_path": "", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_push_interval": "2.000000", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_push_interval_max": "30.000000", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_period_root_pool": ".rgw.root", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_port": "", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_print_continue": "true", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_print_prohibited_content_length": "false", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-02-18T00:31:53.904 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rados_tracing": "false", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_realm": "", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_region": "", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_region_root_pool": ".rgw.root", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_relaxed_region_enforcement": "false", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_request_uri": "", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_batch_size": "64", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_max_aio": "128", 2024-02-18T00:31:53.905 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_num_logs": "16", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_reshard_thread_interval": "600", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_resolve_cname": "false", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_rest_getusage_op_compat": "false", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_run_sync_thread": "true", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_keystone": "false", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_ldap": "false", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_rados": "true", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_auth_use_sts": "false", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_s3_success_create_obj_status": "0", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_scheduler_type": "throttler", 2024-02-18T00:31:53.906 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_script_uri": "", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_service_provider_name": "", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_socket_path": "", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_client_id": "", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_client_secret": "", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_entry": "sts", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_key": "sts", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_max_session_duration": "43200", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_min_session_duration": "900", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sts_token_introspection_url": "", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_account_in_url": "false", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_auth_entry": "auth", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_auth_url": "", 2024-02-18T00:31:53.907 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_custom_header": "", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_enforce_content_length": "false", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_need_stats": "true", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_tenant_name": "", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_token_expiration": "86400", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_url": "", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_url_prefix": "swift", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_swift_versioning_enabled": "false", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_lease_period": "120", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_interval": "1200", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-02-18T00:31:53.908 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_obj_etag_verify": "false", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_trace_history_size": "4096", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_thread_pool_size": "512", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_comment": "", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_createby": "", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_encoding": "", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_flag": "false", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_origin": "", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_sha_unit": "524288", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_torrent_tracker": "", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_trust_forwarded_https": "false", 2024-02-18T00:31:53.909 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_log_tick_interval": "30", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_max_shards": "32", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_usage_max_user_shards": "1", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_use_opa_authz": "false", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_default_quota_max_size": "-1", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_max_buckets": "1000", 2024-02-18T00:31:53.910 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_sync_interval": "86400", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_user_unique_email": "true", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_verify_ssl": "true", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_website_routing_rules_max_num": "50", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zone": "", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zonegroup": "", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_block_size": "4096", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-02-18T00:31:53.911 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_shard_bits": "4", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_size": "536870912", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cache_type": "binned_lru", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cf_compact_on_deletion": "false", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_collect_compaction_stats": "false", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_collect_extended_stats": "false", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_collect_memory_stats": "false", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_index_type": "binary_search", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_log_to_ceph_log": "true", 2024-02-18T00:31:53.912 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_metadata_block_size": "4096", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_partition_filters": "false", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_perf": "false", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "rotating_keys_renewal_timeout": "10", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "run_dir": "/var/run/ceph", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "setgroup": "ceph", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "setuser": "ceph", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "setuser_match_path": "", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_misplaced_ratio": "0.050000", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "thp": "false", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "threadpool_default_timeout": "60", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "threadpool_empty_queue_max_wait": "2", 2024-02-18T00:31:53.913 INFO:teuthology.orchestra.run.smithi059.stdout: "throttler_perf_counter": "true" 2024-02-18T00:31:53.914 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:31:54.190 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:52.443946+0000 osd.3 (osd.3) 61 : cluster [DBG] 6.b scrub starts 2024-02-18T00:31:54.190 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:52.444806+0000 osd.3 (osd.3) 62 : cluster [DBG] 6.b scrub ok 2024-02-18T00:31:54.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:52.443946+0000 osd.3 (osd.3) 61 : cluster [DBG] 6.b scrub starts 2024-02-18T00:31:54.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:52.444806+0000 osd.3 (osd.3) 62 : cluster [DBG] 6.b scrub ok 2024-02-18T00:31:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:52.443946+0000 osd.3 (osd.3) 61 : cluster [DBG] 6.b scrub starts 2024-02-18T00:31:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:52.444806+0000 osd.3 (osd.3) 62 : cluster [DBG] 6.b scrub ok 2024-02-18T00:31:54.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:53.750108+0000 mgr.y (mgr.24454) 108 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:53.975346+0000 osd.0 (osd.0) 49 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:53.976374+0000 osd.0 (osd.0) 50 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:54.071022+0000 osd.4 (osd.4) 51 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:54.074583+0000 osd.4 (osd.4) 52 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:54.615228+0000 osd.7 (osd.7) 51 : cluster [DBG] 2.b scrub starts 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:54.616208+0000 osd.7 (osd.7) 52 : cluster [DBG] 2.b scrub ok 2024-02-18T00:31:54.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:54.647440+0000 mon.a (mon.0) 756 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-18T00:31:54.955 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:55.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:53.750108+0000 mgr.y (mgr.24454) 108 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:55.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:53.975346+0000 osd.0 (osd.0) 49 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:31:55.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:53.976374+0000 osd.0 (osd.0) 50 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:31:55.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:54.071022+0000 osd.4 (osd.4) 51 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:31:55.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:54.074583+0000 osd.4 (osd.4) 52 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:31:55.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:54.615228+0000 osd.7 (osd.7) 51 : cluster [DBG] 2.b scrub starts 2024-02-18T00:31:55.231 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:54.616208+0000 osd.7 (osd.7) 52 : cluster [DBG] 2.b scrub ok 2024-02-18T00:31:55.231 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:54.647440+0000 mon.a (mon.0) 756 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:53.750108+0000 mgr.y (mgr.24454) 108 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:53.975346+0000 osd.0 (osd.0) 49 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:53.976374+0000 osd.0 (osd.0) 50 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:54.071022+0000 osd.4 (osd.4) 51 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:54.074583+0000 osd.4 (osd.4) 52 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:54.615228+0000 osd.7 (osd.7) 51 : cluster [DBG] 2.b scrub starts 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:54.616208+0000 osd.7 (osd.7) 52 : cluster [DBG] 2.b scrub ok 2024-02-18T00:31:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:54.647440+0000 mon.a (mon.0) 756 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:54.137350+0000 osd.6 (osd.6) 41 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:54.138307+0000 osd.6 (osd.6) 42 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:55.003443+0000 osd.1 (osd.1) 57 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:55.005557+0000 osd.1 (osd.1) 58 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:55.014325+0000 osd.0 (osd.0) 51 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:55.015954+0000 osd.0 (osd.0) 52 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:55.100719+0000 osd.4 (osd.4) 53 : cluster [DBG] 4.e scrub starts 2024-02-18T00:31:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:55.104190+0000 osd.4 (osd.4) 54 : cluster [DBG] 4.e scrub ok 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:54.137350+0000 osd.6 (osd.6) 41 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:54.138307+0000 osd.6 (osd.6) 42 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:55.003443+0000 osd.1 (osd.1) 57 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:55.005557+0000 osd.1 (osd.1) 58 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:55.014325+0000 osd.0 (osd.0) 51 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:55.015954+0000 osd.0 (osd.0) 52 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:55.100719+0000 osd.4 (osd.4) 53 : cluster [DBG] 4.e scrub starts 2024-02-18T00:31:56.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:55.104190+0000 osd.4 (osd.4) 54 : cluster [DBG] 4.e scrub ok 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:54.137350+0000 osd.6 (osd.6) 41 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:54.138307+0000 osd.6 (osd.6) 42 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:55.003443+0000 osd.1 (osd.1) 57 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:55.005557+0000 osd.1 (osd.1) 58 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:55.014325+0000 osd.0 (osd.0) 51 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:55.015954+0000 osd.0 (osd.0) 52 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:55.100719+0000 osd.4 (osd.4) 53 : cluster [DBG] 4.e scrub starts 2024-02-18T00:31:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:55.104190+0000 osd.4 (osd.4) 54 : cluster [DBG] 4.e scrub ok 2024-02-18T00:31:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 -- 172.21.15.59:0/713033376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800f8420 msgr2=0x7fd080104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 --2- 172.21.15.59:0/713033376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800f8420 0x7fd080104a20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fd078009be0 tx=0x7fd078004d10).stop 2024-02-18T00:31:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 -- 172.21.15.59:0/713033376 shutdown_connections 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 --2- 172.21.15.59:0/713033376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800fdbb0 0x7fd0800fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 --2- 172.21.15.59:0/713033376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800f8420 0x7fd080104a20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 --2- 172.21.15.59:0/713033376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0800fe510 0x7fd0800f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 -- 172.21.15.59:0/713033376 >> 172.21.15.59:0/713033376 conn(0x7fd0800f3800 msgr2=0x7fd0800f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 -- 172.21.15.59:0/713033376 shutdown_connections 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.993+0000 7fd086269700 1 -- 172.21.15.59:0/713033376 wait complete. 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 librados: starting msgr at 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 librados: starting objecter 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 Processor -- start 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 -- start start 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 librados: setting wanted keys 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 librados: calling monclient init 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800f8420 0x7fd08006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:56.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0800fdbb0 0x7fd08006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800fe510 0x7fd08006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080116330 con 0x7fd0800f8420 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0801164a0 con 0x7fd0800fe510 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.994+0000 7fd086269700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd080116660 con 0x7fd0800fdbb0 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0800fdbb0 0x7fd08006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0800fdbb0 0x7fd08006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60516/0 (socket says 172.21.15.59:60516) 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 -- 172.21.15.59:0/113144374 learned_addr learned my addr 172.21.15.59:0/113144374 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07ffff700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800f8420 0x7fd08006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800fe510 0x7fd08006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800fe510 msgr2=0x7fd08006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800fe510 0x7fd08006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800f8420 msgr2=0x7fd08006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800f8420 0x7fd08006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780097c0 con 0x7fd0800fdbb0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07f7fe700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0800fdbb0 0x7fd08006b5c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd07400e970 tx=0x7fd07400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07d775700 10 client.?.objecter ms_handle_connect 0x7fd0800fdbb0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07d775700 10 client.?.objecter resend_mon_ops 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074004660 con 0x7fd0800fdbb0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07400f040 con 0x7fd0800fdbb0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd080116940 con 0x7fd0800fdbb0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074014790 con 0x7fd0800fdbb0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd086269700 10 client.24638.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd080116f00 con 0x7fd0800fdbb0 2024-02-18T00:31:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.995+0000 7fd086269700 1 librados: init done 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.996+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd08005ee80 con 0x7fd0800fdbb0 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd0740090d0 con 0x7fd0800fdbb0 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd068062c90 0x7fd068065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fd07407c570 con 0x7fd0800fdbb0 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 10 client.24638.objecter ms_dispatch 0x7fd080001550 osd_map(92..92 src has 1..92) v4 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07ffff700 1 --2- 172.21.15.59:0/113144374 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd068062c90 0x7fd068065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 3 client.24638.objecter handle_osd_map got epochs [92,92] > 0 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 3 client.24638.objecter handle_osd_map decoding full epoch 92 2024-02-18T00:31:57.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 20 client.24638.objecter dump_active .. 0 homeless 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1259 (secure 0 0 0) 0x7fd074083090 con 0x7fd0800fdbb0 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07d775700 10 client.24638.objecter ms_handle_connect 0x7fd068062c90 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd07ffff700 1 --2- 172.21.15.59:0/113144374 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd068062c90 0x7fd068065150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd070004dc0 tx=0x7fd07000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.997+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800fe950 0x7fd0800fed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080107110 con 0x7fd0800fe950 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800fe950 0x7fd0800fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd08006c5b0 con 0x7fd0800fe950 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800fe950 0x7fd0800fed50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd078004a30 tx=0x7fd078004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 10 client.24638.objecter ms_handle_connect 0x7fd0800fe950 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 10 client.24638.objecter resend_mon_ops 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07800e040 con 0x7fd0800fe950 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.0 v2:172.21.15.59:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7fd078018040 con 0x7fd0800fe950 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800fe950 msgr2=0x7fd0800fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800fe950 0x7fd0800fed50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fd078004a30 tx=0x7fd078004d10).stop 2024-02-18T00:31:57.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800fe950 msgr2=0x7fd0800fed50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:31:57.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:56.998+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800fe950 0x7fd0800fed50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:57.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:55.750718+0000 mgr.y (mgr.24454) 109 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:57.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:56.061866+0000 osd.4 (osd.4) 55 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:31:57.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:56.063553+0000 osd.4 (osd.4) 56 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:31:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:55.750718+0000 mgr.y (mgr.24454) 109 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:56.061866+0000 osd.4 (osd.4) 55 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:31:57.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:56.063553+0000 osd.4 (osd.4) 56 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.005+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800ff240 0x7fd08011b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.005+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800ff640 con 0x7fd0800ff240 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.005+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800ff240 0x7fd08011b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.005+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) v1 -- 0x7fd0800ff7b0 con 0x7fd0800ff240 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.006+0000 7fd07d775700 10 client.24638.objecter ms_handle_connect 0x7fd0800ff240 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.006+0000 7fd07d775700 10 client.24638.objecter resend_mon_ops 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.006+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800ff240 0x7fd08011b110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd078005010 tx=0x7fd078004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.006+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07802d070 con 0x7fd0800ff240 2024-02-18T00:31:57.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.007+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.0 v2:172.21.15.59:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '0' (not observed, change may require restart) ) v1 ==== 83+0+3 (secure 0 0 0) 0x7fd0780079e0 con 0x7fd0800ff240 2024-02-18T00:31:57.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.007+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800ff240 msgr2=0x7fd08011b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.007+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800ff240 0x7fd08011b110 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd078005010 tx=0x7fd078004d10).stop 2024-02-18T00:31:57.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.007+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800ff240 msgr2=0x7fd08011b110 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:31:57.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.007+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0800ff240 0x7fd08011b110 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0x7fd078005010 tx=0x7fd078004d10).stop 2024-02-18T00:31:57.009 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a: {} 2024-02-18T00:31:57.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.007+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800273c0 0x7fd0800277d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.007+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080027cc0 con 0x7fd0800273c0 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.008+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800273c0 0x7fd0800277d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.008+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd080027e80 con 0x7fd0800273c0 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.008+0000 7fd07d775700 10 client.24638.objecter ms_handle_connect 0x7fd0800273c0 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.008+0000 7fd07d775700 10 client.24638.objecter resend_mon_ops 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.008+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800273c0 0x7fd0800277d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fd07800b530 tx=0x7fd078028000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.008+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078029900 con 0x7fd0800273c0 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.009+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.1 v2:172.21.15.137:3300/0 2 ==== command_reply(tid 4: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7fd07801fd10 con 0x7fd0800273c0 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.009+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800273c0 msgr2=0x7fd0800277d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.009+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800273c0 0x7fd0800277d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fd07800b530 tx=0x7fd078028000).stop 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.009+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800273c0 msgr2=0x7fd0800277d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.009+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0800273c0 0x7fd0800277d0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0x7fd07800b530 tx=0x7fd078028000).stop 2024-02-18T00:31:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.009+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 reap_dead start 2024-02-18T00:31:57.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.015+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd08006b0d0 0x7fd08006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:57.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.015+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080028050 con 0x7fd08006b0d0 2024-02-18T00:31:57.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.015+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd08006b0d0 0x7fd08006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:57.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.016+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) v1 -- 0x7fd080028190 con 0x7fd08006b0d0 2024-02-18T00:31:57.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.016+0000 7fd07d775700 10 client.24638.objecter ms_handle_connect 0x7fd08006b0d0 2024-02-18T00:31:57.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.016+0000 7fd07d775700 10 client.24638.objecter resend_mon_ops 2024-02-18T00:31:57.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.016+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd08006b0d0 0x7fd08006bab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd078009790 tx=0x7fd078034000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.016+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0780338f0 con 0x7fd08006b0d0 2024-02-18T00:31:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.017+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.1 v2:172.21.15.137:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '0' (not observed, change may require restart) ) v1 ==== 83+0+3 (secure 0 0 0) 0x7fd0780296f0 con 0x7fd08006b0d0 2024-02-18T00:31:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.017+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd08006b0d0 msgr2=0x7fd08006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.017+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd08006b0d0 0x7fd08006bab0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd078009790 tx=0x7fd078034000).stop 2024-02-18T00:31:57.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.017+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd08006b0d0 msgr2=0x7fd08006bab0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:31:57.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.017+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd08006b0d0 0x7fd08006bab0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:57.020 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b: {} 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd080028360 0x7fd08011d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd080028050 con 0x7fd080028360 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd080028360 0x7fd08011d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd080028740 con 0x7fd080028360 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd07d775700 10 client.24638.objecter ms_handle_connect 0x7fd080028360 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd07d775700 10 client.24638.objecter resend_mon_ops 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd080028360 0x7fd08011d610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd07800b530 tx=0x7fd078040070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.018+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078047500 con 0x7fd080028360 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.019+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 2 ==== command_reply(tid 6: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7fd07804d070 con 0x7fd080028360 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.019+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd080028360 msgr2=0x7fd08011d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.019+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd080028360 0x7fd08011d610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd07800b530 tx=0x7fd078040070).stop 2024-02-18T00:31:57.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.019+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd080028360 msgr2=0x7fd08011d610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:31:57.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.019+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd080028360 0x7fd08011d610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.025+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd08011db00 0x7fd08011fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:31:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.025+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0801219c0 con 0x7fd08011db00 2024-02-18T00:31:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.025+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd08011db00 0x7fd08011fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:31:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.025+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) v1 -- 0x7fd080121b30 con 0x7fd08011db00 2024-02-18T00:31:57.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.026+0000 7fd07d775700 10 client.24638.objecter ms_handle_connect 0x7fd08011db00 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.026+0000 7fd07d775700 10 client.24638.objecter resend_mon_ops 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.026+0000 7fd084806700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd08011db00 0x7fd08011fef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd078009790 tx=0x7fd07804b070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.026+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0780479f0 con 0x7fd08011db00 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 <== mon.2 v2:172.21.15.59:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '0' (not observed, change may require restart) ) v1 ==== 83+0+3 (secure 0 0 0) 0x7fd078055070 con 0x7fd08011db00 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd08011db00 msgr2=0x7fd08011fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd08011db00 0x7fd08011fef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd078009790 tx=0x7fd07804b070).stop 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd07d775700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd08011db00 msgr2=0x7fd08011fef0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:31:57.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd07d775700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd08011db00 0x7fd08011fef0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:31:57.030 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c: {} 2024-02-18T00:31:57.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 10 librados: watch_flush enter 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 10 librados: watch_flush exit 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 20 client.24638.objecter shutdown clearing up homeless session... 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 10 client.24638.objecter successfully canceled tick 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd068062c90 msgr2=0x7fd068065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd068062c90 0x7fd068065150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fd070004dc0 tx=0x7fd07000b040).stop 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0800fdbb0 msgr2=0x7fd08006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 --2- 172.21.15.59:0/113144374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0800fdbb0 0x7fd08006b5c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd07400e970 tx=0x7fd07400b040).stop 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd084806700 1 -- 172.21.15.59:0/113144374 reap_dead start 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 shutdown_connections 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 >> 172.21.15.59:0/113144374 conn(0x7fd0800f3800 msgr2=0x7fd0800f5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 shutdown_connections 2024-02-18T00:31:57.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 -- 172.21.15.59:0/113144374 wait complete. 2024-02-18T00:31:57.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:31:57.027+0000 7fd086269700 1 librados: shutdown 2024-02-18T00:31:57.032 INFO:teuthology.orchestra.run.smithi059.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-18T00:31:57.032 INFO:teuthology.orchestra.run.smithi059.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-18T00:31:57.032 INFO:teuthology.orchestra.run.smithi059.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2024-02-18T00:31:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:55.750718+0000 mgr.y (mgr.24454) 109 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:56.061866+0000 osd.4 (osd.4) 55 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:31:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:56.063553+0000 osd.4 (osd.4) 56 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:31:57.821 INFO:teuthology.run_tasks:Running task workunit... 2024-02-18T00:31:57.834 INFO:tasks.workunit:Pulling workunits from ref 81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:31:57.834 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-18T00:31:57.834 DEBUG:teuthology.orchestra.run.smithi059:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-18T00:31:57.835 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-02-18T00:31:57.835 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:31:57.835 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:31:57.835 INFO:tasks.thrashosds.thrasher:fixing pg num pool .rgw.root 2024-02-18T00:31:57.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:31:57.836 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-02-18T00:31:57.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:31:57.837 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-02-18T00:31:57.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:31:57.837 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-02-18T00:31:57.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-18T00:31:57.837 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-02-18T00:31:57.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:31:57.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:31:57.869 INFO:teuthology.orchestra.run.smithi059.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-18T00:31:57.869 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-18T00:31:57.882 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:57.911 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:31:57.914 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-18T00:31:57.914 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-18T00:31:57.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:31:57.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:31:57.978 INFO:tasks.workunit:timeout=3h 2024-02-18T00:31:57.979 INFO:tasks.workunit:cleanup=True 2024-02-18T00:31:57.979 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:31:58.017 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:31:58.041 INFO:tasks.workunit.client.0.smithi059.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-18T00:31:58.056 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:58.079 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:31:58.081 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:31:58.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:31:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:58.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:31:58.196 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:58.226 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:56.164000+0000 osd.6 (osd.6) 43 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:31:58.226 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:56.165915+0000 osd.6 (osd.6) 44 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:31:58.226 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:31:56.998411+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.59:0/113144374' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:31:58.226 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:58.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:56.164000+0000 osd.6 (osd.6) 43 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:31:58.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:56.165915+0000 osd.6 (osd.6) 44 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:31:58.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:31:56.998411+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.59:0/113144374' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:31:58.250 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:31:58.252 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:31:58.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:31:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:56.164000+0000 osd.6 (osd.6) 43 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:31:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:56.165915+0000 osd.6 (osd.6) 44 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:31:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:31:56.998411+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? 172.21.15.59:0/113144374' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:31:58.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:31:58.382 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:58.531 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:31:58.534 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:31:58.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:31:58.635 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:31:58.670 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:58.691 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:31:58.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:31:58.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:31:58.795 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:31:58.826 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:58.845 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:31:58.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:31:58.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:31:58.949 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:31:58.979 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:58.998 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:31:59.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:31:59.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:31:59.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:31:59.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.500377+0000 osd.3 (osd.3) 63 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:31:59.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.501293+0000 osd.3 (osd.3) 64 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:31:59.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.560540+0000 osd.2 (osd.2) 39 : cluster [DBG] 2.e scrub starts 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.561432+0000 osd.2 (osd.2) 40 : cluster [DBG] 2.e scrub ok 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.682703+0000 osd.2 (osd.2) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.682748+0000 osd.2 (osd.2) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.751130+0000 mgr.y (mgr.24454) 110 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.885513+0000 osd.5 (osd.5) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.885586+0000 osd.5 (osd.5) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:57.921412+0000 mon.a (mon.0) 757 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.061619+0000 osd.4 (osd.4) 57 : cluster [DBG] 6.e scrub starts 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.064553+0000 osd.4 (osd.4) 58 : cluster [DBG] 6.e scrub ok 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.500377+0000 osd.3 (osd.3) 63 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:31:59.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.501293+0000 osd.3 (osd.3) 64 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.560540+0000 osd.2 (osd.2) 39 : cluster [DBG] 2.e scrub starts 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.561432+0000 osd.2 (osd.2) 40 : cluster [DBG] 2.e scrub ok 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.682703+0000 osd.2 (osd.2) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.682748+0000 osd.2 (osd.2) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.751130+0000 mgr.y (mgr.24454) 110 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.885513+0000 osd.5 (osd.5) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.885586+0000 osd.5 (osd.5) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:57.921412+0000 mon.a (mon.0) 757 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.061619+0000 osd.4 (osd.4) 57 : cluster [DBG] 6.e scrub starts 2024-02-18T00:31:59.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.064553+0000 osd.4 (osd.4) 58 : cluster [DBG] 6.e scrub ok 2024-02-18T00:31:59.132 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:31:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.500377+0000 osd.3 (osd.3) 63 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:31:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.501293+0000 osd.3 (osd.3) 64 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:31:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.560540+0000 osd.2 (osd.2) 39 : cluster [DBG] 2.e scrub starts 2024-02-18T00:31:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.561432+0000 osd.2 (osd.2) 40 : cluster [DBG] 2.e scrub ok 2024-02-18T00:31:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.682703+0000 osd.2 (osd.2) 41 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.682748+0000 osd.2 (osd.2) 42 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.751130+0000 mgr.y (mgr.24454) 110 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:31:59.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.885513+0000 osd.5 (osd.5) 53 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:31:59.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.885586+0000 osd.5 (osd.5) 54 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:31:59.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:57.921412+0000 mon.a (mon.0) 757 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-18T00:31:59.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.061619+0000 osd.4 (osd.4) 57 : cluster [DBG] 6.e scrub starts 2024-02-18T00:31:59.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.064553+0000 osd.4 (osd.4) 58 : cluster [DBG] 6.e scrub ok 2024-02-18T00:31:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:31:59.739 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:00.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.197466+0000 osd.6 (osd.6) 45 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:32:00.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.198534+0000 osd.6 (osd.6) 46 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:32:00.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.529342+0000 osd.3 (osd.3) 65 : cluster [DBG] 6.c scrub starts 2024-02-18T00:32:00.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.530765+0000 osd.3 (osd.3) 66 : cluster [DBG] 6.c scrub ok 2024-02-18T00:32:00.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.854862+0000 osd.5 (osd.5) 55 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:32:00.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.855976+0000 osd.5 (osd.5) 56 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:32:00.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.944401+0000 osd.1 (osd.1) 59 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:32:00.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:58.945447+0000 osd.1 (osd.1) 60 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:32:00.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:59.647749+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-18T00:32:00.183 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:00.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.197466+0000 osd.6 (osd.6) 45 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.198534+0000 osd.6 (osd.6) 46 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.529342+0000 osd.3 (osd.3) 65 : cluster [DBG] 6.c scrub starts 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.530765+0000 osd.3 (osd.3) 66 : cluster [DBG] 6.c scrub ok 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.854862+0000 osd.5 (osd.5) 55 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.855976+0000 osd.5 (osd.5) 56 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.944401+0000 osd.1 (osd.1) 59 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:58.945447+0000 osd.1 (osd.1) 60 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:32:00.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:31:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:59.647749+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-18T00:32:00.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:00.185 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.197466+0000 osd.6 (osd.6) 45 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.198534+0000 osd.6 (osd.6) 46 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.529342+0000 osd.3 (osd.3) 65 : cluster [DBG] 6.c scrub starts 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.530765+0000 osd.3 (osd.3) 66 : cluster [DBG] 6.c scrub ok 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.854862+0000 osd.5 (osd.5) 55 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.855976+0000 osd.5 (osd.5) 56 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.944401+0000 osd.1 (osd.1) 59 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:58.945447+0000 osd.1 (osd.1) 60 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:32:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:31:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:59.647749+0000 mon.a (mon.0) 758 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-18T00:32:00.286 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:00.316 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:00.440 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:00.443 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:00.443 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:00.544 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:00.576 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:00.598 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:00.601 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:00.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:00.702 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:00.732 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:59.579282+0000 osd.3 (osd.3) 67 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:59.580516+0000 osd.3 (osd.3) 68 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:31:59.751785+0000 mgr.y (mgr.24454) 111 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:00.526690+0000 osd.2 (osd.2) 43 : cluster [DBG] 5.a scrub starts 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:00.527577+0000 osd.2 (osd.2) 44 : cluster [DBG] 5.a scrub ok 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:59.579282+0000 osd.3 (osd.3) 67 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:59.580516+0000 osd.3 (osd.3) 68 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:32:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:31:59.751785+0000 mgr.y (mgr.24454) 111 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:00.526690+0000 osd.2 (osd.2) 43 : cluster [DBG] 5.a scrub starts 2024-02-18T00:32:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:00.527577+0000 osd.2 (osd.2) 44 : cluster [DBG] 5.a scrub ok 2024-02-18T00:32:01.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:32:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:32:01.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:32:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:32:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:59.579282+0000 osd.3 (osd.3) 67 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:32:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:59.580516+0000 osd.3 (osd.3) 68 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:32:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:31:59.751785+0000 mgr.y (mgr.24454) 111 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:01.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:00.526690+0000 osd.2 (osd.2) 43 : cluster [DBG] 5.a scrub starts 2024-02-18T00:32:01.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:00.527577+0000 osd.2 (osd.2) 44 : cluster [DBG] 5.a scrub ok 2024-02-18T00:32:02.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:00.863367+0000 mgr.y (mgr.24454) 112 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:02.209 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:02] "GET /metrics HTTP/1.1" 200 172175 "" "Prometheus/2.33.4" 2024-02-18T00:32:02.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:00.863367+0000 mgr.y (mgr.24454) 112 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:02.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:00.863367+0000 mgr.y (mgr.24454) 112 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:02.982 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:02.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:02.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:03.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:03.116 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:01.546304+0000 osd.3 (osd.3) 69 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:32:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:01.547755+0000 osd.3 (osd.3) 70 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:32:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:01.752641+0000 mgr.y (mgr.24454) 113 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:32:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:02.059789+0000 osd.4 (osd.4) 59 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:32:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:02.063341+0000 osd.4 (osd.4) 60 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:32:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:02.506071+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.b scrub starts 2024-02-18T00:32:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:02.507004+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.b scrub ok 2024-02-18T00:32:03.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:01.546304+0000 osd.3 (osd.3) 69 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:32:03.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:01.547755+0000 osd.3 (osd.3) 70 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:32:03.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:01.752641+0000 mgr.y (mgr.24454) 113 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:32:03.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:02.059789+0000 osd.4 (osd.4) 59 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:32:03.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:02.063341+0000 osd.4 (osd.4) 60 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:32:03.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:02.506071+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.b scrub starts 2024-02-18T00:32:03.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:02.507004+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.b scrub ok 2024-02-18T00:32:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:01.546304+0000 osd.3 (osd.3) 69 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:32:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:01.547755+0000 osd.3 (osd.3) 70 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:32:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:01.752641+0000 mgr.y (mgr.24454) 113 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:32:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:02.059789+0000 osd.4 (osd.4) 59 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:32:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:02.063341+0000 osd.4 (osd.4) 60 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:32:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:02.506071+0000 osd.2 (osd.2) 45 : cluster [DBG] 5.b scrub starts 2024-02-18T00:32:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:02.507004+0000 osd.2 (osd.2) 46 : cluster [DBG] 5.b scrub ok 2024-02-18T00:32:04.019 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:02.545716+0000 osd.3 (osd.3) 71 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:32:04.019 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:02.547438+0000 osd.3 (osd.3) 72 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:32:04.020 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:02.545716+0000 osd.3 (osd.3) 71 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:32:04.020 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:02.547438+0000 osd.3 (osd.3) 72 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:32:04.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:02.545716+0000 osd.3 (osd.3) 71 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:32:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:02.547438+0000 osd.3 (osd.3) 72 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:32:04.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.252+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2935349751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec0f8cb0 msgr2=0x7f39ec0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.252+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2935349751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec0f8cb0 0x7f39ec0f9090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f39e0004870 tx=0x7f39e0009cb0).stop 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.252+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2935349751 shutdown_connections 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.252+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2935349751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39ec102210 0x7f39ec104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2935349751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec0f8cb0 0x7f39ec0f9090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2935349751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39ec0f9610 0x7f39ec101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2935349751 >> 172.21.15.59:0/2935349751 conn(0x7f39ec0f2820 msgr2=0x7f39ec0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2935349751 shutdown_connections 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2935349751 wait complete. 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 librados: starting msgr at 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 librados: starting objecter 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 Processor -- start 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 -- start start 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 librados: setting wanted keys 2024-02-18T00:32:04.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 librados: calling monclient init 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39ec0f8cb0 0x7f39ec0f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39ec0f9610 0x7f39ec0f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec102210 0x7f39ec0f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec116620 con 0x7f39ec0f8cb0 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec116790 con 0x7f39ec0f9610 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.253+0000 7f39f2ccb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec116950 con 0x7f39ec102210 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f0a67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39ec0f8cb0 0x7f39ec0f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec102210 0x7f39ec0f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f0a67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39ec0f8cb0 0x7f39ec0f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46098/0 (socket says 172.21.15.59:46098) 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec102210 0x7f39ec0f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:48594/0 (socket says 172.21.15.59:48594) 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f0a67700 1 -- 172.21.15.59:0/2201999077 learned_addr learned my addr 172.21.15.59:0/2201999077 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39ebfff700 1 --2- 172.21.15.59:0/2201999077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39ec0f9610 0x7f39ec0f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 -- 172.21.15.59:0/2201999077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39ec0f9610 msgr2=0x7f39ec0f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 --2- 172.21.15.59:0/2201999077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39ec0f9610 0x7f39ec0f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 -- 172.21.15.59:0/2201999077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39ec0f8cb0 msgr2=0x7f39ec0f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 --2- 172.21.15.59:0/2201999077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39ec0f8cb0 0x7f39ec0f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 -- 172.21.15.59:0/2201999077 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0005040 con 0x7f39ec102210 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39ec0f8cb0 0x7f39ec0f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f1268700 1 --2- 172.21.15.59:0/2201999077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec102210 0x7f39ec0f7640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f39e400eb40 tx=0x7f39e400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39e9ffb700 10 client.?.objecter ms_handle_connect 0x7f39ec102210 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39e9ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4009870 con 0x7f39ec102210 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e400f040 con 0x7f39ec102210 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec116bd0 con 0x7f39ec102210 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.254+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4007e10 con 0x7f39ec102210 2024-02-18T00:32:04.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.255+0000 7f39f2ccb700 10 client.24650.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.255+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec117070 con 0x7f39ec102210 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.255+0000 7f39f2ccb700 1 librados: init done 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.255+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f39ec05fa60 con 0x7f39ec102210 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f39e40092e0 con 0x7f39ec102210 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39dc062c90 0x7f39dc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f39e407c7d0 con 0x7f39ec102210 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 osd_map(94..94 src has 1..94) v4 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 3 client.24650.objecter handle_osd_map got epochs [94,94] > 0 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 3 client.24650.objecter handle_osd_map decoding full epoch 94 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39dc062c90 0x7f39dc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 20 client.24650.objecter dump_active .. 0 homeless 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.256+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v94) v1 ==== 54+0+16 (secure 0 0 0) 0x7f39e407cb70 con 0x7f39ec102210 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 1 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6801/3700238977 conn(0x7f39ec0fc1a0 0x7f39ec0fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39dc062c90 0x7f39dc065150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f39e00043d0 tx=0x7f39e000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 1 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 0 1 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec05ee80 osd=0 4 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 10 client.24650.objecter _send_command 1 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec107290 con 0x7f39ec0fc1a0 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39dc062c90 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39ebfff700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6801/3700238977 conn(0x7f39ec0fc1a0 0x7f39ec0fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec0fc1a0 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.257+0000 7f39ebfff700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6801/3700238977 conn(0x7f39ec0fc1a0 0x7f39ec0fc5a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.258+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39ec107290 con 0x7f39ec0fc1a0 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.258+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.258+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 1 = system:0 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.258+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec05ee80 osd=0 4 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.258+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 0 1 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 2 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 2 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec05ee80 osd=0 3 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 0 2 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec05ee80 osd=0 4 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 10 client.24650.objecter _send_command 2 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.273+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec0273c0 con 0x7f39ec0fc1a0 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec0273c0 con 0x7f39ec0fc1a0 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.302 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:32:04.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec05ee80 osd=0 4 2024-02-18T00:32:04.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 0 2 2024-02-18T00:32:04.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.274+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6805/26279110 conn(0x7f39ec0598f0 0x7f39ec059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 1 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec0286b0 osd=1 4 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 10 client.24650.objecter _send_command 3 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec028820 con 0x7f39ec0598f0 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6805/26279110 conn(0x7f39ec0598f0 0x7f39ec059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6805/26279110 conn(0x7f39ec0598f0 0x7f39ec059d10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.275+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec0598f0 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.276+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39e001a040 con 0x7f39ec0598f0 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.276+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.276+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 3 = system:0 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.276+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec0286b0 osd=1 4 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.276+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 1 3 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 4 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 4 2024-02-18T00:32:04.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec0286b0 osd=1 3 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 1 4 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec0286b0 osd=1 4 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 10 client.24650.objecter _send_command 4 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.291+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec059360 con 0x7f39ec0598f0 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec059360 con 0x7f39ec0598f0 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec0286b0 osd=1 4 2024-02-18T00:32:04.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 1 4 2024-02-18T00:32:04.307 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:32:04.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 5 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6809/1153193637 conn(0x7f39ec028210 0x7f39ec11e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 5 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 2 5 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec059770 osd=2 4 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 10 client.24650.objecter _send_command 5 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec121060 con 0x7f39ec028210 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39ebfff700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6809/1153193637 conn(0x7f39ec028210 0x7f39ec11e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec028210 2024-02-18T00:32:04.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.293+0000 7f39ebfff700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6809/1153193637 conn(0x7f39ec028210 0x7f39ec11e410 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.294+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39ec121060 con 0x7f39ec028210 2024-02-18T00:32:04.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.294+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:32:04.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.294+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 5 = system:0 2024-02-18T00:32:04.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.294+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec059770 osd=2 4 2024-02-18T00:32:04.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.294+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 2 5 2024-02-18T00:32:04.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 6 2024-02-18T00:32:04.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 6 2024-02-18T00:32:04.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec059770 osd=2 3 2024-02-18T00:32:04.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 2 6 2024-02-18T00:32:04.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec059770 osd=2 4 2024-02-18T00:32:04.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 10 client.24650.objecter _send_command 6 2024-02-18T00:32:04.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.309+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec027830 con 0x7f39ec028210 2024-02-18T00:32:04.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec027830 con 0x7f39ec028210 2024-02-18T00:32:04.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec059770 osd=2 4 2024-02-18T00:32:04.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 2 6 2024-02-18T00:32:04.312 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 7 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6813/1805700964 conn(0x7f39ec11fa80 0x7f39ec11fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 7 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 3 7 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec11ba50 osd=3 4 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 10 client.24650.objecter _send_command 7 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec11c190 con 0x7f39ec11fa80 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.310+0000 7f39f1268700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6813/1805700964 conn(0x7f39ec11fa80 0x7f39ec11fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.311+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec11fa80 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.311+0000 7f39f1268700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6813/1805700964 conn(0x7f39ec11fa80 0x7f39ec11fec0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.312+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39ec11c190 con 0x7f39ec11fa80 2024-02-18T00:32:04.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.312+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:32:04.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.312+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 7 = system:0 2024-02-18T00:32:04.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.312+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec11ba50 osd=3 4 2024-02-18T00:32:04.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.312+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 3 7 2024-02-18T00:32:04.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 8 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 8 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec11ba50 osd=3 3 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 3 8 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec11ba50 osd=3 4 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 10 client.24650.objecter _send_command 8 2024-02-18T00:32:04.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.327+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec120b80 con 0x7f39ec11fa80 2024-02-18T00:32:04.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec120b80 con 0x7f39ec11fa80 2024-02-18T00:32:04.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec11ba50 osd=3 4 2024-02-18T00:32:04.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 3 8 2024-02-18T00:32:04.331 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 9 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39ec123290 0x7f39ec125680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 9 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 4 9 2024-02-18T00:32:04.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec1203b0 osd=4 4 2024-02-18T00:32:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 10 client.24650.objecter _send_command 9 2024-02-18T00:32:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.329+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec120b60 con 0x7f39ec123290 2024-02-18T00:32:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.330+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39ec123290 0x7f39ec125680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.330+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec123290 2024-02-18T00:32:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.330+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39ec123290 0x7f39ec125680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.331+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39ec120b60 con 0x7f39ec123290 2024-02-18T00:32:04.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.331+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:32:04.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.331+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 9 = system:0 2024-02-18T00:32:04.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.331+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec1203b0 osd=4 4 2024-02-18T00:32:04.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.331+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 4 9 2024-02-18T00:32:04.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 10 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 10 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec1203b0 osd=4 3 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 4 10 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec1203b0 osd=4 4 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 10 client.24650.objecter _send_command 10 2024-02-18T00:32:04.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.346+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec120b60 con 0x7f39ec123290 2024-02-18T00:32:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.347+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec120b60 con 0x7f39ec123290 2024-02-18T00:32:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.347+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.347+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.347+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec1203b0 osd=4 4 2024-02-18T00:32:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.347+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 4 10 2024-02-18T00:32:04.350 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:32:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 11 2024-02-18T00:32:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6804/2170432263 conn(0x7f39ec126080 0x7f39ec126500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 11 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 5 11 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec120b60 osd=5 4 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 10 client.24650.objecter _send_command 11 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec128150 con 0x7f39ec126080 2024-02-18T00:32:04.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39ebfff700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6804/2170432263 conn(0x7f39ec126080 0x7f39ec126500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec126080 2024-02-18T00:32:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.348+0000 7f39ebfff700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6804/2170432263 conn(0x7f39ec126080 0x7f39ec126500 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.349+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39ec128150 con 0x7f39ec126080 2024-02-18T00:32:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.349+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:32:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.349+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 11 = system:0 2024-02-18T00:32:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.349+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec120b60 osd=5 4 2024-02-18T00:32:04.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.349+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 5 11 2024-02-18T00:32:04.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 12 2024-02-18T00:32:04.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 12 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec120b60 osd=5 3 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 5 12 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec120b60 osd=5 4 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 10 client.24650.objecter _send_command 12 2024-02-18T00:32:04.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.364+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec128150 con 0x7f39ec126080 2024-02-18T00:32:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec128150 con 0x7f39ec126080 2024-02-18T00:32:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec120b60 osd=5 4 2024-02-18T00:32:04.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 5 12 2024-02-18T00:32:04.368 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 13 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39ec12a8a0 0x7f39ec12ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 13 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 6 13 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec128150 osd=6 4 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 10 client.24650.objecter _send_command 13 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec12b390 con 0x7f39ec12a8a0 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.366+0000 7f39f1268700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39ec12a8a0 0x7f39ec12ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.367+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec12a8a0 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.367+0000 7f39f1268700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39ec12a8a0 0x7f39ec12ad50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.368+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39e4009590 con 0x7f39ec12a8a0 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.368+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.368+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 13 = system:0 2024-02-18T00:32:04.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.368+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec128150 osd=6 4 2024-02-18T00:32:04.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.368+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 6 13 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 14 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 14 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec128150 osd=6 3 2024-02-18T00:32:04.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 6 14 2024-02-18T00:32:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec128150 osd=6 4 2024-02-18T00:32:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 10 client.24650.objecter _send_command 14 2024-02-18T00:32:04.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.382+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec12d5a0 con 0x7f39ec12a8a0 2024-02-18T00:32:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec12d5a0 con 0x7f39ec12a8a0 2024-02-18T00:32:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec128150 osd=6 4 2024-02-18T00:32:04.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 6 14 2024-02-18T00:32:04.386 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:32:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 15 2024-02-18T00:32:04.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6812/53025171 conn(0x7f39ec12dc80 0x7f39ec12e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 15 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 7 15 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec12d5a0 osd=7 4 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 10 client.24650.objecter _send_command 15 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.384+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec131960 con 0x7f39ec12dc80 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.385+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6812/53025171 conn(0x7f39ec12dc80 0x7f39ec12e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.385+0000 7f39e9ffb700 10 client.24650.objecter ms_handle_connect 0x7f39ec12dc80 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.385+0000 7f39f0a67700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6812/53025171 conn(0x7f39ec12dc80 0x7f39ec12e0e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:04.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.386+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f39ec131960 con 0x7f39ec12dc80 2024-02-18T00:32:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.386+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:32:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.386+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 15 = system:0 2024-02-18T00:32:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.386+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec12d5a0 osd=7 4 2024-02-18T00:32:04.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.386+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 7 15 2024-02-18T00:32:04.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 10 client.24650.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:04.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign -1 16 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 20 client.24650.objecter _get_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_remove -1 16 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 20 client.24650.objecter get_session s=0x7f39ec12d5a0 osd=7 3 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 15 client.24650.objecter _session_command_op_assign 7 16 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec12d5a0 osd=7 4 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 10 client.24650.objecter _send_command 16 2024-02-18T00:32:04.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.401+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f39ec1307e0 con 0x7f39ec12dc80 2024-02-18T00:32:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.402+0000 7f39e9ffb700 1 -- 172.21.15.59:0/2201999077 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f39ec1307e0 con 0x7f39ec12dc80 2024-02-18T00:32:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.402+0000 7f39e9ffb700 10 client.24650.objecter ms_dispatch 0x7f39ec001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.402+0000 7f39e9ffb700 10 client.24650.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.402+0000 7f39e9ffb700 20 client.24650.objecter put_session s=0x7f39ec12d5a0 osd=7 4 2024-02-18T00:32:04.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.402+0000 7f39e9ffb700 15 client.24650.objecter _session_command_op_remove 7 16 2024-02-18T00:32:04.405 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:32:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 librados: watch_flush enter 2024-02-18T00:32:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 librados: watch_flush exit 2024-02-18T00:32:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.0 2024-02-18T00:32:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6801/3700238977 conn(0x7f39ec0fc1a0 msgr2=0x7f39ec0fc5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6801/3700238977 conn(0x7f39ec0fc1a0 0x7f39ec0fc5a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec05ee80 osd=0 1 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.1 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6805/26279110 conn(0x7f39ec0598f0 msgr2=0x7f39ec059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6805/26279110 conn(0x7f39ec0598f0 0x7f39ec059d10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec0286b0 osd=1 1 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.2 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6809/1153193637 conn(0x7f39ec028210 msgr2=0x7f39ec11e410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6809/1153193637 conn(0x7f39ec028210 0x7f39ec11e410 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec059770 osd=2 1 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.3 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6813/1805700964 conn(0x7f39ec11fa80 msgr2=0x7f39ec11fec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6813/1805700964 conn(0x7f39ec11fa80 0x7f39ec11fec0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec11ba50 osd=3 1 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.4 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f1268700 1 -- 172.21.15.59:0/2201999077 reap_dead start 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39ec123290 msgr2=0x7f39ec125680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39ec123290 0x7f39ec125680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f1268700 1 -- 172.21.15.59:0/2201999077 reap_dead start 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec1203b0 osd=4 1 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.5 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6804/2170432263 conn(0x7f39ec126080 msgr2=0x7f39ec126500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6804/2170432263 conn(0x7f39ec126080 0x7f39ec126500 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec120b60 osd=5 1 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.6 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39ec12a8a0 msgr2=0x7f39ec12ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39ec12a8a0 0x7f39ec12ad50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec128150 osd=6 1 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter close_session for osd.7 2024-02-18T00:32:04.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6812/53025171 conn(0x7f39ec12dc80 msgr2=0x7f39ec12e0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.137:6812/53025171 conn(0x7f39ec12dc80 0x7f39ec12e0e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter put_session s=0x7f39ec12d5a0 osd=7 1 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 20 client.24650.objecter shutdown clearing up homeless session... 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 10 client.24650.objecter successfully canceled tick 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39dc062c90 msgr2=0x7f39dc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> v2:172.21.15.59:6800/3721891797 conn(0x7f39dc062c90 0x7f39dc065150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f39e00043d0 tx=0x7f39e000b040).stop 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec102210 msgr2=0x7f39ec0f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 --2- 172.21.15.59:0/2201999077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39ec102210 0x7f39ec0f7640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f39e400eb40 tx=0x7f39e400b040).stop 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f1268700 1 -- 172.21.15.59:0/2201999077 reap_dead start 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 shutdown_connections 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 >> 172.21.15.59:0/2201999077 conn(0x7f39ec0f2820 msgr2=0x7f39ec0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 shutdown_connections 2024-02-18T00:32:04.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 -- 172.21.15.59:0/2201999077 wait complete. 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:04.403+0000 7f39f2ccb700 1 librados: shutdown 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.411 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.412 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:32:04.529 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:32:04.529 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:32:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:04.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:04.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:04.870 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:04.903 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:05.002 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:05.005 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:05.006 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:05.021 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:03.753038+0000 mgr.y (mgr.24454) 114 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:03.930019+0000 osd.1 (osd.1) 61 : cluster [DBG] 2.d deep-scrub starts 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:03.930959+0000 osd.1 (osd.1) 62 : cluster [DBG] 2.d deep-scrub ok 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:04.257544+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? 172.21.15.59:0/2201999077' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:04.459989+0000 osd.2 (osd.2) 47 : cluster [DBG] 2.c scrub starts 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:04.460964+0000 osd.2 (osd.2) 48 : cluster [DBG] 2.c scrub ok 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:03.753038+0000 mgr.y (mgr.24454) 114 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:03.930019+0000 osd.1 (osd.1) 61 : cluster [DBG] 2.d deep-scrub starts 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:03.930959+0000 osd.1 (osd.1) 62 : cluster [DBG] 2.d deep-scrub ok 2024-02-18T00:32:05.022 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:04.257544+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? 172.21.15.59:0/2201999077' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:05.023 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:04.459989+0000 osd.2 (osd.2) 47 : cluster [DBG] 2.c scrub starts 2024-02-18T00:32:05.023 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:04.460964+0000 osd.2 (osd.2) 48 : cluster [DBG] 2.c scrub ok 2024-02-18T00:32:05.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:05.140 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:05.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:05.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:03.753038+0000 mgr.y (mgr.24454) 114 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:03.930019+0000 osd.1 (osd.1) 61 : cluster [DBG] 2.d deep-scrub starts 2024-02-18T00:32:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:03.930959+0000 osd.1 (osd.1) 62 : cluster [DBG] 2.d deep-scrub ok 2024-02-18T00:32:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:04.257544+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? 172.21.15.59:0/2201999077' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:04.459989+0000 osd.2 (osd.2) 47 : cluster [DBG] 2.c scrub starts 2024-02-18T00:32:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:04.460964+0000 osd.2 (osd.2) 48 : cluster [DBG] 2.c scrub ok 2024-02-18T00:32:05.264 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:05.372 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:05.378 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:05.378 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:05.479 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:05.532 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:05.534 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:05.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:05.636 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:05.666 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:05.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:05.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:05.791 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:05.820 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:05.874 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:05.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:05.878 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:05.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:06.110 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:06.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:04.950899+0000 osd.1 (osd.1) 63 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:32:06.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:04.954355+0000 osd.1 (osd.1) 64 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:32:06.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.023194+0000 osd.0 (osd.0) 53 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:32:06.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.026646+0000 osd.0 (osd.0) 54 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.448306+0000 osd.2 (osd.2) 49 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.449277+0000 osd.2 (osd.2) 50 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:04.950899+0000 osd.1 (osd.1) 63 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:04.954355+0000 osd.1 (osd.1) 64 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.023194+0000 osd.0 (osd.0) 53 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.026646+0000 osd.0 (osd.0) 54 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.448306+0000 osd.2 (osd.2) 49 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:32:06.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.449277+0000 osd.2 (osd.2) 50 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:32:06.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:06.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:06.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:06.244 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:04.950899+0000 osd.1 (osd.1) 63 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:32:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:04.954355+0000 osd.1 (osd.1) 64 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:32:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.023194+0000 osd.0 (osd.0) 53 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:32:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.026646+0000 osd.0 (osd.0) 54 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:32:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.448306+0000 osd.2 (osd.2) 49 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:32:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.449277+0000 osd.2 (osd.2) 50 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:32:06.270 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:06.285 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:06.286 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:06.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:06.416 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:06.854 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-18T00:32:06.868 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:06.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:06.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:06.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:07.006 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:07.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.105+0000 7f0288ddb700 1 -- 172.21.15.59:0/3956563237 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fdbd0 msgr2=0x7f02840fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.105+0000 7f0288ddb700 1 --2- 172.21.15.59:0/3956563237 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fdbd0 0x7f02840fdfb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0274004870 tx=0x7f0274009cb0).stop 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 -- 172.21.15.59:0/3956563237 shutdown_connections 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 --2- 172.21.15.59:0/3956563237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02840f6a90 0x7f0284104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 --2- 172.21.15.59:0/3956563237 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fdbd0 0x7f02840fdfb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 --2- 172.21.15.59:0/3956563237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02840fe530 0x7f02840f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 -- 172.21.15.59:0/3956563237 >> 172.21.15.59:0/3956563237 conn(0x7f028406c370 msgr2=0x7f028406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 -- 172.21.15.59:0/3956563237 shutdown_connections 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 -- 172.21.15.59:0/3956563237 wait complete. 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 librados: starting msgr at 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.106+0000 7f0288ddb700 1 librados: starting objecter 2024-02-18T00:32:07.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 Processor -- start 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 -- start start 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 librados: setting wanted keys 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 librados: calling monclient init 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02840f6a90 0x7f02840633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02840fdbd0 0x7f0284063900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fe530 0x7f028411c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028411cd70 con 0x7f02840fdbd0 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028411ceb0 con 0x7f02840f6a90 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0288ddb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028411d070 con 0x7f02840fe530 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0282d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fe530 0x7f028411c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0282d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fe530 0x7f028411c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:48634/0 (socket says 172.21.15.59:48634) 2024-02-18T00:32:07.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0282d9d700 1 -- 172.21.15.59:0/454391748 learned_addr learned my addr 172.21.15.59:0/454391748 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.107+0000 7f0281d9b700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02840fdbd0 0x7f0284063900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f0282d9d700 1 -- 172.21.15.59:0/454391748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02840f6a90 msgr2=0x7f02840633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f0282d9d700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02840f6a90 0x7f02840633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f0282d9d700 1 -- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02840fdbd0 msgr2=0x7f0284063900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f0282d9d700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02840fdbd0 0x7f0284063900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f0282d9d700 1 -- 172.21.15.59:0/454391748 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0278005040 con 0x7f02840fe530 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f0281d9b700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02840fdbd0 0x7f0284063900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f0282d9d700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fe530 0x7f028411c880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f027c00ec20 tx=0x7f027c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f02737fe700 10 client.?.objecter ms_handle_connect 0x7f02840fe530 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f02737fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f02737fe700 1 -- 172.21.15.59:0/454391748 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c004020 con 0x7f02840fe530 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f02737fe700 1 -- 172.21.15.59:0/454391748 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027c00f040 con 0x7f02840fe530 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.108+0000 7f02737fe700 1 -- 172.21.15.59:0/454391748 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00c790 con 0x7f02840fe530 2024-02-18T00:32:07.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.109+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0274005040 con 0x7f02840fe530 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.109+0000 7f0288ddb700 10 client.24662.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.109+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028411d5d0 con 0x7f02840fe530 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.109+0000 7f0288ddb700 1 librados: init done 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.110+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028411d230 con 0x7f02840fe530 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.111+0000 7f02737fe700 1 -- 172.21.15.59:0/454391748 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f027c0094f0 con 0x7f02840fe530 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.111+0000 7f02737fe700 1 --2- 172.21.15.59:0/454391748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f026c062ce0 0x7f026c0651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.111+0000 7f02737fe700 1 -- 172.21.15.59:0/454391748 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(95..95 src has 44..95) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f027c07cd80 con 0x7f02840fe530 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.112+0000 7f02737fe700 10 client.24662.objecter ms_dispatch 0x7f0284001550 osd_map(95..95 src has 44..95) v4 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.112+0000 7f02737fe700 3 client.24662.objecter handle_osd_map got epochs [95,95] > 0 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.112+0000 7f02737fe700 3 client.24662.objecter handle_osd_map decoding full epoch 95 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.112+0000 7f02737fe700 20 client.24662.objecter dump_active .. 0 homeless 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.112+0000 7f028259c700 1 --2- 172.21.15.59:0/454391748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f026c062ce0 0x7f026c0651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:07.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.112+0000 7f02737fe700 10 client.24662.objecter ms_handle_connect 0x7f026c062ce0 2024-02-18T00:32:07.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.112+0000 7f028259c700 1 --2- 172.21.15.59:0/454391748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f026c062ce0 0x7f026c0651a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0274009bc0 tx=0x7f0274009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:07.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.116+0000 7f02737fe700 1 -- 172.21.15.59:0/454391748 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f027c0509b0 con 0x7f02840fe530 2024-02-18T00:32:07.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 -- 172.21.15.59:0/1537384641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 msgr2=0x7f686c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 --2- 172.21.15.59:0/1537384641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 0x7f686c104a20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f6864009be0 tx=0x7f6864004d10).stop 2024-02-18T00:32:07.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 -- 172.21.15.59:0/1537384641 shutdown_connections 2024-02-18T00:32:07.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 --2- 172.21.15.59:0/1537384641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c0fdbb0 0x7f686c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 --2- 172.21.15.59:0/1537384641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 0x7f686c104a20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 --2- 172.21.15.59:0/1537384641 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fe510 0x7f686c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 -- 172.21.15.59:0/1537384641 >> 172.21.15.59:0/1537384641 conn(0x7f686c0f3800 msgr2=0x7f686c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 -- 172.21.15.59:0/1537384641 shutdown_connections 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 -- 172.21.15.59:0/1537384641 wait complete. 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 librados: starting msgr at 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.477+0000 7f6873707700 1 librados: starting objecter 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 Processor -- start 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 -- start start 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 librados: setting wanted keys 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 librados: calling monclient init 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 0x7f686c10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fdbb0 0x7f686c10e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c0fe510 0x7f686c107f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c108460 con 0x7f686c0f8420 2024-02-18T00:32:07.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1085d0 con 0x7f686c0fe510 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6873707700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c108790 con 0x7f686c0fdbb0 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fdbb0 0x7f686c10e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f68714a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 0x7f686c10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fdbb0 0x7f686c10e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:48618/0 (socket says 172.21.15.59:48618) 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 -- 172.21.15.59:0/3626760473 learned_addr learned my addr 172.21.15.59:0/3626760473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6871ca4700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c0fe510 0x7f686c107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 -- 172.21.15.59:0/3626760473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c0fe510 msgr2=0x7f686c107f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c0fe510 0x7f686c107f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 -- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 msgr2=0x7f686c10ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 0x7f686c10ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f6870ca2700 1 -- 172.21.15.59:0/3626760473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640097c0 con 0x7f686c0fdbb0 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.478+0000 7f68714a3700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 0x7f686c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:32:07.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f6870ca2700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fdbb0 0x7f686c10e2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f685c0041d0 tx=0x7f685c009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f68627fc700 10 client.?.objecter ms_handle_connect 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f68627fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f68627fc700 1 -- 172.21.15.59:0/3626760473 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c007b40 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c108a70 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f68627fc700 1 -- 172.21.15.59:0/3626760473 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f685c00f040 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f68627fc700 1 -- 172.21.15.59:0/3626760473 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c014730 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f6873707700 10 client.24656.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c108df0 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.479+0000 7f6873707700 1 librados: init done 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.480+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c107110 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.480+0000 7f68627fc700 1 -- 172.21.15.59:0/3626760473 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f685c009110 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.480+0000 7f68627fc700 1 --2- 172.21.15.59:0/3626760473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6858062c90 0x7f6858065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68627fc700 1 -- 172.21.15.59:0/3626760473 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(95..95 src has 44..95) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f685c07ca70 con 0x7f686c0fdbb0 2024-02-18T00:32:07.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68627fc700 10 client.24656.objecter ms_dispatch 0x7f686c001550 osd_map(95..95 src has 44..95) v4 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68627fc700 3 client.24656.objecter handle_osd_map got epochs [95,95] > 0 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68627fc700 3 client.24656.objecter handle_osd_map decoding full epoch 95 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68714a3700 1 --2- 172.21.15.59:0/3626760473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6858062c90 0x7f6858065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68627fc700 20 client.24656.objecter dump_active .. 0 homeless 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68627fc700 10 client.24656.objecter ms_handle_connect 0x7f6858062c90 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.481+0000 7f68714a3700 1 --2- 172.21.15.59:0/3626760473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6858062c90 0x7f6858065150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f68680103c0 tx=0x7f686800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.485+0000 7f68627fc700 1 -- 172.21.15.59:0/3626760473 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f685c050750 con 0x7f686c0fdbb0 2024-02-18T00:32:07.234 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:32:07.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.680+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f686c05fb60 con 0x7f6858062c90 2024-02-18T00:32:07.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:06.692+0000 7f68627fc700 1 -- 172.21.15.59:0/3626760473 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357249 (secure 0 0 0) 0x7f686c05fb60 con 0x7f6858062c90 2024-02-18T00:32:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.230+0000 7f6873707700 10 librados: watch_flush enter 2024-02-18T00:32:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 10 librados: watch_flush exit 2024-02-18T00:32:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 20 client.24656.objecter shutdown clearing up homeless session... 2024-02-18T00:32:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 10 client.24656.objecter successfully canceled tick 2024-02-18T00:32:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6858062c90 msgr2=0x7f6858065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 --2- 172.21.15.59:0/3626760473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6858062c90 0x7f6858065150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f68680103c0 tx=0x7f686800b040).stop 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fdbb0 msgr2=0x7f686c10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fdbb0 0x7f686c10e2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f685c0041d0 tx=0x7f685c009f60).stop 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 shutdown_connections 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 --2- 172.21.15.59:0/3626760473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6858062c90 0x7f6858065150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c0fe510 0x7f686c107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f686c0f8420 0x7f686c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 --2- 172.21.15.59:0/3626760473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f686c0fdbb0 0x7f686c10e2c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 >> 172.21.15.59:0/3626760473 conn(0x7f686c0f3800 msgr2=0x7f686c0f50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 shutdown_connections 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 -- 172.21.15.59:0/3626760473 wait complete. 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.231+0000 7f6873707700 1 librados: shutdown 2024-02-18T00:32:07.243 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:32:07.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.650494+0000 osd.3 (osd.3) 73 : cluster [DBG] 4.0 deep-scrub starts 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.656335+0000 osd.3 (osd.3) 74 : cluster [DBG] 4.0 deep-scrub ok 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.753603+0000 mgr.y (mgr.24454) 115 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.789279+0000 osd.5 (osd.5) 57 : cluster [DBG] 6.a scrub starts 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.790087+0000 osd.5 (osd.5) 58 : cluster [DBG] 6.a scrub ok 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.831248+0000 osd.4 (osd.4) 61 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.831283+0000 osd.4 (osd.4) 62 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.946197+0000 mon.a (mon.0) 759 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.971325+0000 osd.4 (osd.4) 63 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:32:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:05.972298+0000 osd.4 (osd.4) 64 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.650494+0000 osd.3 (osd.3) 73 : cluster [DBG] 4.0 deep-scrub starts 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.656335+0000 osd.3 (osd.3) 74 : cluster [DBG] 4.0 deep-scrub ok 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.753603+0000 mgr.y (mgr.24454) 115 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.789279+0000 osd.5 (osd.5) 57 : cluster [DBG] 6.a scrub starts 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.790087+0000 osd.5 (osd.5) 58 : cluster [DBG] 6.a scrub ok 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.831248+0000 osd.4 (osd.4) 61 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.831283+0000 osd.4 (osd.4) 62 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.946197+0000 mon.a (mon.0) 759 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.971325+0000 osd.4 (osd.4) 63 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:32:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:05.972298+0000 osd.4 (osd.4) 64 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.650494+0000 osd.3 (osd.3) 73 : cluster [DBG] 4.0 deep-scrub starts 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.656335+0000 osd.3 (osd.3) 74 : cluster [DBG] 4.0 deep-scrub ok 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.753603+0000 mgr.y (mgr.24454) 115 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.789279+0000 osd.5 (osd.5) 57 : cluster [DBG] 6.a scrub starts 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.790087+0000 osd.5 (osd.5) 58 : cluster [DBG] 6.a scrub ok 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.831248+0000 osd.4 (osd.4) 61 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.831283+0000 osd.4 (osd.4) 62 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.946197+0000 mon.a (mon.0) 759 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.971325+0000 osd.4 (osd.4) 63 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:32:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:05.972298+0000 osd.4 (osd.4) 64 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:32:07.259 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:07.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:07.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:07.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.311+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f028405fa40 con 0x7f02840fe530 2024-02-18T00:32:07.362 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:07.391 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:07.590 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:07.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:07.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:07.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:07.722 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:07.795 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:07.811 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:07.811 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:07.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:32:07.912 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:07.942 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.958+0000 7f02737fe700 1 -- 172.21.15.59:0/454391748 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v96) v1 ==== 87+0+0 (secure 0 0 0) 0x7f027c0505a0 con 0x7f02840fe530 2024-02-18T00:32:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 10 librados: watch_flush enter 2024-02-18T00:32:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 10 librados: watch_flush exit 2024-02-18T00:32:07.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 20 client.24662.objecter shutdown clearing up homeless session... 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 10 client.24662.objecter successfully canceled tick 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f026c062ce0 msgr2=0x7f026c0651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 --2- 172.21.15.59:0/454391748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f026c062ce0 0x7f026c0651a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0274009bc0 tx=0x7f0274009cb0).stop 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fe530 msgr2=0x7f028411c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fe530 0x7f028411c880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f027c00ec20 tx=0x7f027c00b040).stop 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 shutdown_connections 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 --2- 172.21.15.59:0/454391748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f026c062ce0 0x7f026c0651a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02840f6a90 0x7f02840633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f02840fe530 0x7f028411c880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 --2- 172.21.15.59:0/454391748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02840fdbd0 0x7f0284063900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:07.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.962+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 >> 172.21.15.59:0/454391748 conn(0x7f028406c370 msgr2=0x7f02840f58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:07.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.964+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 shutdown_connections 2024-02-18T00:32:07.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.965+0000 7f0288ddb700 1 -- 172.21.15.59:0/454391748 wait complete. 2024-02-18T00:32:07.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:07.965+0000 7f0288ddb700 1 librados: shutdown 2024-02-18T00:32:07.967 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:06.682192+0000 mgr.y (mgr.24454) 116 : audit [DBG] from='client.24656 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:06.738976+0000 osd.7 (osd.7) 53 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:06.739939+0000 osd.7 (osd.7) 54 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:06.882026+0000 osd.1 (osd.1) 65 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:06.883015+0000 osd.1 (osd.1) 66 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.007357+0000 osd.0 (osd.0) 55 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.008535+0000 osd.0 (osd.0) 56 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:07.313110+0000 mon.c (mon.2) 28 : audit [INF] from='client.? 172.21.15.59:0/454391748' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:07.313569+0000 mon.a (mon.0) 760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:32:08.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.446324+0000 osd.2 (osd.2) 51 : cluster [DBG] 6.f scrub starts 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.449245+0000 osd.2 (osd.2) 52 : cluster [DBG] 6.f scrub ok 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:06.682192+0000 mgr.y (mgr.24454) 116 : audit [DBG] from='client.24656 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:06.738976+0000 osd.7 (osd.7) 53 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:06.739939+0000 osd.7 (osd.7) 54 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:06.882026+0000 osd.1 (osd.1) 65 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:06.883015+0000 osd.1 (osd.1) 66 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.007357+0000 osd.0 (osd.0) 55 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.008535+0000 osd.0 (osd.0) 56 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:07.313110+0000 mon.c (mon.2) 28 : audit [INF] from='client.? 172.21.15.59:0/454391748' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:07.313569+0000 mon.a (mon.0) 760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:32:08.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.446324+0000 osd.2 (osd.2) 51 : cluster [DBG] 6.f scrub starts 2024-02-18T00:32:08.213 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.449245+0000 osd.2 (osd.2) 52 : cluster [DBG] 6.f scrub ok 2024-02-18T00:32:08.229 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-02-18T00:32:05.942817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1266,"num_read_kb":986,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":684,"ondisk_log_size":684,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2384564,"kb_used_data":7796,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747576652,"statfs":{"total":767960285184,"available":765518491648,"internally_reserved":0,"allocated":7983104,"data_stored":3400826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.021527"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311193+0000","last_change":"2024-02-18T00:30:46.913439+0000","last_active":"2024-02-18T00:31:58.311193+0000","last_peered":"2024-02-18T00:31:58.311193+0000","last_clean":"2024-02-18T00:31:58.311193+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:31:58.311193+0000","last_undegraded":"2024-02-18T00:31:58.311193+0000","last_fullsized":"2024-02-18T00:31:58.311193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.913340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.913340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.917877+0000","last_change":"2024-02-18T00:31:07.942520+0000","last_active":"2024-02-18T00:31:57.917877+0000","last_peered":"2024-02-18T00:31:57.917877+0000","last_clean":"2024-02-18T00:31:57.917877+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:31:57.917877+0000","last_undegraded":"2024-02-18T00:31:57.917877+0000","last_fullsized":"2024-02-18T00:31:57.917877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.942435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.942435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310758+0000","last_change":"2024-02-18T00:31:02.922852+0000","last_active":"2024-02-18T00:31:58.310758+0000","last_peered":"2024-02-18T00:31:58.310758+0000","last_clean":"2024-02-18T00:31:58.310758+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:31:58.310758+0000","last_undegraded":"2024-02-18T00:31:58.310758+0000","last_fullsized":"2024-02-18T00:31:58.310758+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.922771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.922771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":134,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311448+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:31:58.311448+0000","last_peered":"2024-02-18T00:31:58.311448+0000","last_clean":"2024-02-18T00:31:58.311448+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:31:58.311448+0000","last_undegraded":"2024-02-18T00:31:58.311448+0000","last_fullsized":"2024-02-18T00:31:58.311448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918094+0000","last_change":"2024-02-18T00:30:29.061462+0000","last_active":"2024-02-18T00:31:57.918094+0000","last_peered":"2024-02-18T00:31:57.918094+0000","last_clean":"2024-02-18T00:31:57.918094+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:31:57.918094+0000","last_undegraded":"2024-02-18T00:31:57.918094+0000","last_fullsized":"2024-02-18T00:31:57.918094+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.061390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.061390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.924921+0000","last_change":"2024-02-18T00:30:50.991652+0000","last_active":"2024-02-18T00:31:57.924921+0000","last_peered":"2024-02-18T00:31:57.924921+0000","last_clean":"2024-02-18T00:31:57.924921+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:31:57.924921+0000","last_undegraded":"2024-02-18T00:31:57.924921+0000","last_fullsized":"2024-02-18T00:31:57.924921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.991592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.991592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.312216+0000","last_change":"2024-02-18T00:30:48.890743+0000","last_active":"2024-02-18T00:31:58.312216+0000","last_peered":"2024-02-18T00:31:58.312216+0000","last_clean":"2024-02-18T00:31:58.312216+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:31:58.312216+0000","last_undegraded":"2024-02-18T00:31:58.312216+0000","last_fullsized":"2024-02-18T00:31:58.312216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.890677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":80,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918040+0000","last_change":"2024-02-18T00:31:03.941281+0000","last_active":"2024-02-18T00:31:57.918040+0000","last_peered":"2024-02-18T00:31:57.918040+0000","last_clean":"2024-02-18T00:31:57.918040+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:31:57.918040+0000","last_undegraded":"2024-02-18T00:31:57.918040+0000","last_fullsized":"2024-02-18T00:31:57.918040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:03.941195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:03.941195+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310670+0000","last_change":"2024-02-18T00:31:44.843600+0000","last_active":"2024-02-18T00:31:58.310670+0000","last_peered":"2024-02-18T00:31:58.310670+0000","last_clean":"2024-02-18T00:31:58.310670+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:31:58.310670+0000","last_undegraded":"2024-02-18T00:31:58.310670+0000","last_fullsized":"2024-02-18T00:31:58.310670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:31:44.843494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200468+0000","last_change":"2024-02-18T00:31:41.607649+0000","last_active":"2024-02-18T00:31:58.200468+0000","last_peered":"2024-02-18T00:31:58.200468+0000","last_clean":"2024-02-18T00:31:58.200468+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:31:58.200468+0000","last_undegraded":"2024-02-18T00:31:58.200468+0000","last_fullsized":"2024-02-18T00:31:58.200468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.607567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200403+0000","last_change":"2024-02-18T00:31:36.502977+0000","last_active":"2024-02-18T00:31:58.200403+0000","last_peered":"2024-02-18T00:31:58.200403+0000","last_clean":"2024-02-18T00:31:58.200403+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:31:58.200403+0000","last_undegraded":"2024-02-18T00:31:58.200403+0000","last_fullsized":"2024-02-18T00:31:58.200403+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.502900+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:02.063451+0000","last_change":"2024-02-18T00:32:02.063451+0000","last_active":"2024-02-18T00:32:02.063451+0000","last_peered":"2024-02-18T00:32:02.063451+0000","last_clean":"2024-02-18T00:32:02.063451+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:32:02.063451+0000","last_undegraded":"2024-02-18T00:32:02.063451+0000","last_fullsized":"2024-02-18T00:32:02.063451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:32:02.063346+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.063346+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310674+0000","last_change":"2024-02-18T00:31:36.140716+0000","last_active":"2024-02-18T00:31:58.310674+0000","last_peered":"2024-02-18T00:31:58.310674+0000","last_clean":"2024-02-18T00:31:58.310674+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:31:58.310674+0000","last_undegraded":"2024-02-18T00:31:58.310674+0000","last_fullsized":"2024-02-18T00:31:58.310674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.140630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.140630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.856076+0000","last_change":"2024-02-18T00:31:58.856050+0000","last_active":"2024-02-18T00:31:58.856076+0000","last_peered":"2024-02-18T00:31:58.856076+0000","last_clean":"2024-02-18T00:31:58.856076+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:31:58.856076+0000","last_undegraded":"2024-02-18T00:31:58.856076+0000","last_fullsized":"2024-02-18T00:31:58.856076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.855979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.855979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200283+0000","last_change":"2024-02-18T00:30:26.654619+0000","last_active":"2024-02-18T00:31:58.200283+0000","last_peered":"2024-02-18T00:31:58.200283+0000","last_clean":"2024-02-18T00:31:58.200283+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:31:58.200283+0000","last_undegraded":"2024-02-18T00:31:58.200283+0000","last_fullsized":"2024-02-18T00:31:58.200283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.654541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.654541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'22","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918180+0000","last_change":"2024-02-18T00:31:40.094511+0000","last_active":"2024-02-18T00:31:57.918180+0000","last_peered":"2024-02-18T00:31:57.918180+0000","last_clean":"2024-02-18T00:31:57.918180+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:31:57.918180+0000","last_undegraded":"2024-02-18T00:31:57.918180+0000","last_fullsized":"2024-02-18T00:31:57.918180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:31:40.094414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.094414+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.312048+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:31:58.312048+0000","last_peered":"2024-02-18T00:31:58.312048+0000","last_clean":"2024-02-18T00:31:58.312048+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:31:58.312048+0000","last_undegraded":"2024-02-18T00:31:58.312048+0000","last_fullsized":"2024-02-18T00:31:58.312048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918545+0000","last_change":"2024-02-18T00:30:33.047434+0000","last_active":"2024-02-18T00:31:57.918545+0000","last_peered":"2024-02-18T00:31:57.918545+0000","last_clean":"2024-02-18T00:31:57.918545+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:31:57.918545+0000","last_undegraded":"2024-02-18T00:31:57.918545+0000","last_fullsized":"2024-02-18T00:31:57.918545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.047327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.047327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":63,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311402+0000","last_change":"2024-02-18T00:31:32.210269+0000","last_active":"2024-02-18T00:31:58.311402+0000","last_peered":"2024-02-18T00:31:58.311402+0000","last_clean":"2024-02-18T00:31:58.311402+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:31:58.311402+0000","last_undegraded":"2024-02-18T00:31:58.311402+0000","last_fullsized":"2024-02-18T00:31:58.311402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:31:32.210198+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.210198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":87,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311721+0000","last_change":"2024-02-18T00:30:50.885765+0000","last_active":"2024-02-18T00:31:58.311721+0000","last_peered":"2024-02-18T00:31:58.311721+0000","last_clean":"2024-02-18T00:31:58.311721+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:31:58.311721+0000","last_undegraded":"2024-02-18T00:31:58.311721+0000","last_fullsized":"2024-02-18T00:31:58.311721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:30:50.885651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.885651+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918836+0000","last_change":"2024-02-18T00:30:36.043298+0000","last_active":"2024-02-18T00:31:57.918836+0000","last_peered":"2024-02-18T00:31:57.918836+0000","last_clean":"2024-02-18T00:31:57.918836+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:31:57.918836+0000","last_undegraded":"2024-02-18T00:31:57.918836+0000","last_fullsized":"2024-02-18T00:31:57.918836+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:36.043211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:36.043211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.926489+0000","last_change":"2024-02-18T00:31:02.145322+0000","last_active":"2024-02-18T00:31:57.926489+0000","last_peered":"2024-02-18T00:31:57.926489+0000","last_clean":"2024-02-18T00:31:57.926489+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:31:57.926489+0000","last_undegraded":"2024-02-18T00:31:57.926489+0000","last_fullsized":"2024-02-18T00:31:57.926489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.145253+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.198954+0000","last_change":"2024-02-18T00:31:54.138392+0000","last_active":"2024-02-18T00:31:58.198954+0000","last_peered":"2024-02-18T00:31:58.198954+0000","last_clean":"2024-02-18T00:31:58.198954+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:31:58.198954+0000","last_undegraded":"2024-02-18T00:31:58.198954+0000","last_fullsized":"2024-02-18T00:31:58.198954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:54.138311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.138311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.922866+0000","last_change":"2024-02-18T00:31:40.592166+0000","last_active":"2024-02-18T00:31:57.922866+0000","last_peered":"2024-02-18T00:31:57.922866+0000","last_clean":"2024-02-18T00:31:57.922866+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:31:57.922866+0000","last_undegraded":"2024-02-18T00:31:57.922866+0000","last_fullsized":"2024-02-18T00:31:57.922866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:40.592083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.592083+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.926009+0000","last_change":"2024-02-18T00:31:37.936961+0000","last_active":"2024-02-18T00:31:57.926009+0000","last_peered":"2024-02-18T00:31:57.926009+0000","last_clean":"2024-02-18T00:31:57.926009+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:31:57.926009+0000","last_undegraded":"2024-02-18T00:31:57.926009+0000","last_fullsized":"2024-02-18T00:31:57.926009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.936896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.936896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311139+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:31:58.311139+0000","last_peered":"2024-02-18T00:31:58.311139+0000","last_clean":"2024-02-18T00:31:58.311139+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:31:58.311139+0000","last_undegraded":"2024-02-18T00:31:58.311139+0000","last_fullsized":"2024-02-18T00:31:58.311139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311528+0000","last_change":"2024-02-18T00:31:49.512647+0000","last_active":"2024-02-18T00:31:58.311528+0000","last_peered":"2024-02-18T00:31:58.311528+0000","last_clean":"2024-02-18T00:31:58.311528+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:31:58.311528+0000","last_undegraded":"2024-02-18T00:31:58.311528+0000","last_fullsized":"2024-02-18T00:31:58.311528+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:31:49.512572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.199548+0000","last_change":"2024-02-18T00:31:56.165995+0000","last_active":"2024-02-18T00:31:58.199548+0000","last_peered":"2024-02-18T00:31:58.199548+0000","last_clean":"2024-02-18T00:31:58.199548+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:31:58.199548+0000","last_undegraded":"2024-02-18T00:31:58.199548+0000","last_fullsized":"2024-02-18T00:31:58.199548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:56.165920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.165920+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200709+0000","last_change":"2024-02-18T00:31:45.654910+0000","last_active":"2024-02-18T00:31:58.200709+0000","last_peered":"2024-02-18T00:31:58.200709+0000","last_clean":"2024-02-18T00:31:58.200709+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:31:58.200709+0000","last_undegraded":"2024-02-18T00:31:58.200709+0000","last_fullsized":"2024-02-18T00:31:58.200709+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:31:45.654827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:45.654827+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":97,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:31:59.648353+0000","last_change":"2024-02-18T00:31:30.356896+0000","last_active":"2024-02-18T00:31:59.648353+0000","last_peered":"2024-02-18T00:31:59.648353+0000","last_clean":"2024-02-18T00:31:59.648353+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:31:59.648353+0000","last_undegraded":"2024-02-18T00:31:59.648353+0000","last_fullsized":"2024-02-18T00:31:59.648353+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:30.356811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.356811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310574+0000","last_change":"2024-02-18T00:31:06.040501+0000","last_active":"2024-02-18T00:31:58.310574+0000","last_peered":"2024-02-18T00:31:58.310574+0000","last_clean":"2024-02-18T00:31:58.310574+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:31:58.310574+0000","last_undegraded":"2024-02-18T00:31:58.310574+0000","last_fullsized":"2024-02-18T00:31:58.310574+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.040425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.040425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":83,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311122+0000","last_change":"2024-02-18T00:31:55.016021+0000","last_active":"2024-02-18T00:31:58.311122+0000","last_peered":"2024-02-18T00:31:58.311122+0000","last_clean":"2024-02-18T00:31:58.311122+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:31:58.311122+0000","last_undegraded":"2024-02-18T00:31:58.311122+0000","last_fullsized":"2024-02-18T00:31:58.311122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:55.015959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.015959+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311558+0000","last_change":"2024-02-18T00:30:59.122281+0000","last_active":"2024-02-18T00:31:58.311558+0000","last_peered":"2024-02-18T00:31:58.311558+0000","last_clean":"2024-02-18T00:31:58.311558+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:31:58.311558+0000","last_undegraded":"2024-02-18T00:31:58.311558+0000","last_fullsized":"2024-02-18T00:31:58.311558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.122216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.122216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200202+0000","last_change":"2024-02-18T00:30:31.634376+0000","last_active":"2024-02-18T00:31:58.200202+0000","last_peered":"2024-02-18T00:31:58.200202+0000","last_clean":"2024-02-18T00:31:58.200202+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:31:58.200202+0000","last_undegraded":"2024-02-18T00:31:58.200202+0000","last_fullsized":"2024-02-18T00:31:58.200202+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:31.634311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:31.634311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.199252+0000","last_change":"2024-02-18T00:31:51.186460+0000","last_active":"2024-02-18T00:31:58.199252+0000","last_peered":"2024-02-18T00:31:58.199252+0000","last_clean":"2024-02-18T00:31:58.199252+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:31:58.199252+0000","last_undegraded":"2024-02-18T00:31:58.199252+0000","last_fullsized":"2024-02-18T00:31:58.199252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:51.186373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:51.186373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":83,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311812+0000","last_change":"2024-02-18T00:30:43.819520+0000","last_active":"2024-02-18T00:31:58.311812+0000","last_peered":"2024-02-18T00:31:58.311812+0000","last_clean":"2024-02-18T00:31:58.311812+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:31:58.311812+0000","last_undegraded":"2024-02-18T00:31:58.311812+0000","last_fullsized":"2024-02-18T00:31:58.311812+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:43.819434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.819434+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200080+0000","last_change":"2024-02-18T00:30:35.533752+0000","last_active":"2024-02-18T00:31:58.200080+0000","last_peered":"2024-02-18T00:31:58.200080+0000","last_clean":"2024-02-18T00:31:58.200080+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:31:58.200080+0000","last_undegraded":"2024-02-18T00:31:58.200080+0000","last_fullsized":"2024-02-18T00:31:58.200080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.533630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.533630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.198581+0000","last_change":"2024-02-18T00:30:47.440072+0000","last_active":"2024-02-18T00:31:58.198581+0000","last_peered":"2024-02-18T00:31:58.198581+0000","last_clean":"2024-02-18T00:31:58.198581+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:31:58.198581+0000","last_undegraded":"2024-02-18T00:31:58.198581+0000","last_fullsized":"2024-02-18T00:31:58.198581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.440005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.440005+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311386+0000","last_change":"2024-02-18T00:31:30.789715+0000","last_active":"2024-02-18T00:31:58.311386+0000","last_peered":"2024-02-18T00:31:58.311386+0000","last_clean":"2024-02-18T00:31:58.311386+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:31:58.311386+0000","last_undegraded":"2024-02-18T00:31:58.311386+0000","last_fullsized":"2024-02-18T00:31:58.311386+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.789634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310516+0000","last_change":"2024-02-18T00:31:00.083411+0000","last_active":"2024-02-18T00:31:58.310516+0000","last_peered":"2024-02-18T00:31:58.310516+0000","last_clean":"2024-02-18T00:31:58.310516+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:31:58.310516+0000","last_undegraded":"2024-02-18T00:31:58.310516+0000","last_fullsized":"2024-02-18T00:31:58.310516+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:31:00.083332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.083332+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311294+0000","last_change":"2024-02-18T00:31:06.048967+0000","last_active":"2024-02-18T00:31:58.311294+0000","last_peered":"2024-02-18T00:31:58.311294+0000","last_clean":"2024-02-18T00:31:58.311294+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:31:58.311294+0000","last_undegraded":"2024-02-18T00:31:58.311294+0000","last_fullsized":"2024-02-18T00:31:58.311294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.048889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.048889+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.925659+0000","last_change":"2024-02-18T00:30:45.987723+0000","last_active":"2024-02-18T00:31:57.925659+0000","last_peered":"2024-02-18T00:31:57.925659+0000","last_clean":"2024-02-18T00:31:57.925659+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:31:57.925659+0000","last_undegraded":"2024-02-18T00:31:57.925659+0000","last_fullsized":"2024-02-18T00:31:57.925659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.987660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.987660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.926172+0000","last_change":"2024-02-18T00:30:54.008260+0000","last_active":"2024-02-18T00:31:57.926172+0000","last_peered":"2024-02-18T00:31:57.926172+0000","last_clean":"2024-02-18T00:31:57.926172+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:31:57.926172+0000","last_undegraded":"2024-02-18T00:31:57.926172+0000","last_fullsized":"2024-02-18T00:31:57.926172+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.008196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.008196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":73,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918310+0000","last_change":"2024-02-18T00:31:54.074682+0000","last_active":"2024-02-18T00:31:57.918310+0000","last_peered":"2024-02-18T00:31:57.918310+0000","last_clean":"2024-02-18T00:31:57.918310+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:31:57.918310+0000","last_undegraded":"2024-02-18T00:31:57.918310+0000","last_fullsized":"2024-02-18T00:31:57.918310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:31:54.074589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.074589+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310286+0000","last_change":"2024-02-18T00:31:13.960170+0000","last_active":"2024-02-18T00:31:58.310286+0000","last_peered":"2024-02-18T00:31:58.310286+0000","last_clean":"2024-02-18T00:31:58.310286+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:31:58.310286+0000","last_undegraded":"2024-02-18T00:31:58.310286+0000","last_fullsized":"2024-02-18T00:31:58.310286+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:13.960089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:13.960089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311451+0000","last_change":"2024-02-18T00:31:57.501367+0000","last_active":"2024-02-18T00:31:58.311451+0000","last_peered":"2024-02-18T00:31:58.311451+0000","last_clean":"2024-02-18T00:31:58.311451+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:31:58.311451+0000","last_undegraded":"2024-02-18T00:31:58.311451+0000","last_fullsized":"2024-02-18T00:31:58.311451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.501298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.501298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.199195+0000","last_change":"2024-02-18T00:31:38.266692+0000","last_active":"2024-02-18T00:31:58.199195+0000","last_peered":"2024-02-18T00:31:58.199195+0000","last_clean":"2024-02-18T00:31:58.199195+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:31:58.199195+0000","last_undegraded":"2024-02-18T00:31:58.199195+0000","last_fullsized":"2024-02-18T00:31:58.199195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.266577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.266577+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":65,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.925571+0000","last_change":"2024-02-18T00:30:51.994431+0000","last_active":"2024-02-18T00:31:57.925571+0000","last_peered":"2024-02-18T00:31:57.925571+0000","last_clean":"2024-02-18T00:31:57.925571+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:31:57.925571+0000","last_undegraded":"2024-02-18T00:31:57.925571+0000","last_fullsized":"2024-02-18T00:31:57.925571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:30:51.994334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.994334+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918867+0000","last_change":"2024-02-18T00:31:12.044114+0000","last_active":"2024-02-18T00:31:57.918867+0000","last_peered":"2024-02-18T00:31:57.918867+0000","last_clean":"2024-02-18T00:31:57.918867+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:31:57.918867+0000","last_undegraded":"2024-02-18T00:31:57.918867+0000","last_fullsized":"2024-02-18T00:31:57.918867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:12.043994+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":94,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:01.547919+0000","last_change":"2024-02-18T00:32:01.547864+0000","last_active":"2024-02-18T00:32:01.547919+0000","last_peered":"2024-02-18T00:32:01.547919+0000","last_clean":"2024-02-18T00:32:01.547919+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:32:01.547919+0000","last_undegraded":"2024-02-18T00:32:01.547919+0000","last_fullsized":"2024-02-18T00:32:01.547919+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:01.547763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:01.547763+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311107+0000","last_change":"2024-02-18T00:31:41.112608+0000","last_active":"2024-02-18T00:31:58.311107+0000","last_peered":"2024-02-18T00:31:58.311107+0000","last_clean":"2024-02-18T00:31:58.311107+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:31:58.311107+0000","last_undegraded":"2024-02-18T00:31:58.311107+0000","last_fullsized":"2024-02-18T00:31:58.311107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.112540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.112540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":80,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311296+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:31:58.311296+0000","last_peered":"2024-02-18T00:31:58.311296+0000","last_clean":"2024-02-18T00:31:58.311296+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:31:58.311296+0000","last_undegraded":"2024-02-18T00:31:58.311296+0000","last_fullsized":"2024-02-18T00:31:58.311296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310766+0000","last_change":"2024-02-18T00:31:37.150285+0000","last_active":"2024-02-18T00:31:58.310766+0000","last_peered":"2024-02-18T00:31:58.310766+0000","last_clean":"2024-02-18T00:31:58.310766+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:31:58.310766+0000","last_undegraded":"2024-02-18T00:31:58.310766+0000","last_fullsized":"2024-02-18T00:31:58.310766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.150215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":89,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:31:59.987264+0000","last_change":"2024-02-18T00:31:01.012802+0000","last_active":"2024-02-18T00:31:59.987264+0000","last_peered":"2024-02-18T00:31:59.987264+0000","last_clean":"2024-02-18T00:31:59.987264+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:31:59.987264+0000","last_undegraded":"2024-02-18T00:31:59.987264+0000","last_fullsized":"2024-02-18T00:31:59.987264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:01.012688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.012688+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310804+0000","last_change":"2024-02-18T00:31:05.071051+0000","last_active":"2024-02-18T00:31:58.310804+0000","last_peered":"2024-02-18T00:31:58.310804+0000","last_clean":"2024-02-18T00:31:58.310804+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:31:58.310804+0000","last_undegraded":"2024-02-18T00:31:58.310804+0000","last_fullsized":"2024-02-18T00:31:58.310804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:05.070930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.070930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":99,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:02.547561+0000","last_change":"2024-02-18T00:32:02.547561+0000","last_active":"2024-02-18T00:32:02.547561+0000","last_peered":"2024-02-18T00:32:02.547561+0000","last_clean":"2024-02-18T00:32:02.547561+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:32:02.547561+0000","last_undegraded":"2024-02-18T00:32:02.547561+0000","last_fullsized":"2024-02-18T00:32:02.547561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:32:02.547447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.547447+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200394+0000","last_change":"2024-02-18T00:30:33.564597+0000","last_active":"2024-02-18T00:31:58.200394+0000","last_peered":"2024-02-18T00:31:58.200394+0000","last_clean":"2024-02-18T00:31:58.200394+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:31:58.200394+0000","last_undegraded":"2024-02-18T00:31:58.200394+0000","last_fullsized":"2024-02-18T00:31:58.200394+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.564517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.564517+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311354+0000","last_change":"2024-02-18T00:31:23.018547+0000","last_active":"2024-02-18T00:31:58.311354+0000","last_peered":"2024-02-18T00:31:58.311354+0000","last_clean":"2024-02-18T00:31:58.311354+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:31:58.311354+0000","last_undegraded":"2024-02-18T00:31:58.311354+0000","last_fullsized":"2024-02-18T00:31:58.311354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.018460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.018460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.198602+0000","last_change":"2024-02-18T00:31:46.199231+0000","last_active":"2024-02-18T00:31:58.198602+0000","last_peered":"2024-02-18T00:31:58.198602+0000","last_clean":"2024-02-18T00:31:58.198602+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:31:58.198602+0000","last_undegraded":"2024-02-18T00:31:58.198602+0000","last_fullsized":"2024-02-18T00:31:58.198602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.199153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.199153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":97,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311987+0000","last_change":"2024-02-18T00:31:29.228915+0000","last_active":"2024-02-18T00:31:58.311987+0000","last_peered":"2024-02-18T00:31:58.311987+0000","last_clean":"2024-02-18T00:31:58.311987+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:31:58.311987+0000","last_undegraded":"2024-02-18T00:31:58.311987+0000","last_fullsized":"2024-02-18T00:31:58.311987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:29.228799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:29.228799+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.922545+0000","last_change":"2024-02-18T00:30:46.308529+0000","last_active":"2024-02-18T00:31:57.922545+0000","last_peered":"2024-02-18T00:31:57.922545+0000","last_clean":"2024-02-18T00:31:57.922545+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:31:57.922545+0000","last_undegraded":"2024-02-18T00:31:57.922545+0000","last_fullsized":"2024-02-18T00:31:57.922545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.308454+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:59.580627+0000","last_change":"2024-02-18T00:31:59.580602+0000","last_active":"2024-02-18T00:31:59.580627+0000","last_peered":"2024-02-18T00:31:59.580627+0000","last_clean":"2024-02-18T00:31:59.580627+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:31:59.580627+0000","last_undegraded":"2024-02-18T00:31:59.580627+0000","last_fullsized":"2024-02-18T00:31:59.580627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:59.580519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:59.580519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:05.449395+0000","last_change":"2024-02-18T00:32:05.449395+0000","last_active":"2024-02-18T00:32:05.449395+0000","last_peered":"2024-02-18T00:32:05.449395+0000","last_clean":"2024-02-18T00:32:05.449395+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:32:05.449395+0000","last_undegraded":"2024-02-18T00:32:05.449395+0000","last_fullsized":"2024-02-18T00:32:05.449395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.449286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.449286+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":94,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310941+0000","last_change":"2024-02-18T00:31:18.065868+0000","last_active":"2024-02-18T00:31:58.310941+0000","last_peered":"2024-02-18T00:31:58.310941+0000","last_clean":"2024-02-18T00:31:58.310941+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:31:58.310941+0000","last_undegraded":"2024-02-18T00:31:58.310941+0000","last_fullsized":"2024-02-18T00:31:58.310941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:31:18.065789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.065789+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311504+0000","last_change":"2024-02-18T00:31:52.444880+0000","last_active":"2024-02-18T00:31:58.311504+0000","last_peered":"2024-02-18T00:31:58.311504+0000","last_clean":"2024-02-18T00:31:58.311504+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:31:58.311504+0000","last_undegraded":"2024-02-18T00:31:58.311504+0000","last_fullsized":"2024-02-18T00:31:58.311504+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:52.444811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:52.444811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":69,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918739+0000","last_change":"2024-02-18T00:31:52.069257+0000","last_active":"2024-02-18T00:31:57.918739+0000","last_peered":"2024-02-18T00:31:57.918739+0000","last_clean":"2024-02-18T00:31:57.918739+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:31:57.918739+0000","last_undegraded":"2024-02-18T00:31:57.918739+0000","last_fullsized":"2024-02-18T00:31:57.918739+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:31:52.069184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:52.069184+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":102,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918099+0000","last_change":"2024-02-18T00:30:54.996645+0000","last_active":"2024-02-18T00:31:57.918099+0000","last_peered":"2024-02-18T00:31:57.918099+0000","last_clean":"2024-02-18T00:31:57.918099+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:31:57.918099+0000","last_undegraded":"2024-02-18T00:31:57.918099+0000","last_fullsized":"2024-02-18T00:31:57.918099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:30:54.996545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.996545+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.921768+0000","last_change":"2024-02-18T00:30:43.221614+0000","last_active":"2024-02-18T00:31:57.921768+0000","last_peered":"2024-02-18T00:31:57.921768+0000","last_clean":"2024-02-18T00:31:57.921768+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:31:57.921768+0000","last_undegraded":"2024-02-18T00:31:57.921768+0000","last_fullsized":"2024-02-18T00:31:57.921768+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.221541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.221541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310291+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:31:58.310291+0000","last_peered":"2024-02-18T00:31:58.310291+0000","last_clean":"2024-02-18T00:31:58.310291+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:31:58.310291+0000","last_undegraded":"2024-02-18T00:31:58.310291+0000","last_fullsized":"2024-02-18T00:31:58.310291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.926976+0000","last_change":"2024-02-18T00:31:47.974066+0000","last_active":"2024-02-18T00:31:57.926976+0000","last_peered":"2024-02-18T00:31:57.926976+0000","last_clean":"2024-02-18T00:31:57.926976+0000","last_became_active":"2024-02-18T00:29:14.715019+0000","last_became_peered":"2024-02-18T00:29:14.715019+0000","last_unstale":"2024-02-18T00:31:57.926976+0000","last_undegraded":"2024-02-18T00:31:57.926976+0000","last_fullsized":"2024-02-18T00:31:57.926976+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:47.973962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:47.973962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'16","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.923723+0000","last_change":"2024-02-18T00:31:33.699627+0000","last_active":"2024-02-18T00:31:57.923723+0000","last_peered":"2024-02-18T00:31:57.923723+0000","last_clean":"2024-02-18T00:31:57.923723+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:31:57.923723+0000","last_undegraded":"2024-02-18T00:31:57.923723+0000","last_fullsized":"2024-02-18T00:31:57.923723+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:33.699554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:33.699554+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":93,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:05.026757+0000","last_change":"2024-02-18T00:32:05.026757+0000","last_active":"2024-02-18T00:32:05.026757+0000","last_peered":"2024-02-18T00:32:05.026757+0000","last_clean":"2024-02-18T00:32:05.026757+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:32:05.026757+0000","last_undegraded":"2024-02-18T00:32:05.026757+0000","last_fullsized":"2024-02-18T00:32:05.026757+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:05.026652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.026652+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311967+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:31:58.311967+0000","last_peered":"2024-02-18T00:31:58.311967+0000","last_clean":"2024-02-18T00:31:58.311967+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:31:58.311967+0000","last_undegraded":"2024-02-18T00:31:58.311967+0000","last_fullsized":"2024-02-18T00:31:58.311967+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310838+0000","last_change":"2024-02-18T00:31:08.046030+0000","last_active":"2024-02-18T00:31:58.310838+0000","last_peered":"2024-02-18T00:31:58.310838+0000","last_clean":"2024-02-18T00:31:58.310838+0000","last_became_active":"2024-02-18T00:29:14.715157+0000","last_became_peered":"2024-02-18T00:29:14.715157+0000","last_unstale":"2024-02-18T00:31:58.310838+0000","last_undegraded":"2024-02-18T00:31:58.310838+0000","last_fullsized":"2024-02-18T00:31:58.310838+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:31:08.045946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":151,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:04.954445+0000","last_change":"2024-02-18T00:32:04.954445+0000","last_active":"2024-02-18T00:32:04.954445+0000","last_peered":"2024-02-18T00:32:04.954445+0000","last_clean":"2024-02-18T00:32:04.954445+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:32:04.954445+0000","last_undegraded":"2024-02-18T00:32:04.954445+0000","last_fullsized":"2024-02-18T00:32:04.954445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:32:04.954359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.954359+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310967+0000","last_change":"2024-02-18T00:30:47.731838+0000","last_active":"2024-02-18T00:31:58.310967+0000","last_peered":"2024-02-18T00:31:58.310967+0000","last_clean":"2024-02-18T00:31:58.310967+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:31:58.310967+0000","last_undegraded":"2024-02-18T00:31:58.310967+0000","last_fullsized":"2024-02-18T00:31:58.310967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:47.731752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:47.731752+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200565+0000","last_change":"2024-02-18T00:30:58.601762+0000","last_active":"2024-02-18T00:31:58.200565+0000","last_peered":"2024-02-18T00:31:58.200565+0000","last_clean":"2024-02-18T00:31:58.200565+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:31:58.200565+0000","last_undegraded":"2024-02-18T00:31:58.200565+0000","last_fullsized":"2024-02-18T00:31:58.200565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.601694+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":221,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:03.986902+0000","last_change":"2024-02-18T00:31:58.945520+0000","last_active":"2024-02-18T00:32:03.986902+0000","last_peered":"2024-02-18T00:32:03.986902+0000","last_clean":"2024-02-18T00:32:03.986902+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:32:03.986902+0000","last_undegraded":"2024-02-18T00:32:03.986902+0000","last_fullsized":"2024-02-18T00:32:03.986902+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:31:58.945452+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.945452+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":162,"num_read_kb":157,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.921839+0000","last_change":"2024-02-18T00:30:48.262064+0000","last_active":"2024-02-18T00:31:57.921839+0000","last_peered":"2024-02-18T00:31:57.921839+0000","last_clean":"2024-02-18T00:31:57.921839+0000","last_became_active":"2024-02-18T00:29:14.713812+0000","last_became_peered":"2024-02-18T00:29:14.713812+0000","last_unstale":"2024-02-18T00:31:57.921839+0000","last_undegraded":"2024-02-18T00:31:57.921839+0000","last_fullsized":"2024-02-18T00:31:57.921839+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.262001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.262001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":96,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.927363+0000","last_change":"2024-02-18T00:31:55.005638+0000","last_active":"2024-02-18T00:31:57.927363+0000","last_peered":"2024-02-18T00:31:57.927363+0000","last_clean":"2024-02-18T00:31:57.927363+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:31:57.927363+0000","last_undegraded":"2024-02-18T00:31:57.927363+0000","last_fullsized":"2024-02-18T00:31:57.927363+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:31:55.005563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.005563+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":52,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.922734+0000","last_change":"2024-02-18T00:31:47.502734+0000","last_active":"2024-02-18T00:31:57.922734+0000","last_peered":"2024-02-18T00:31:57.922734+0000","last_clean":"2024-02-18T00:31:57.922734+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:31:57.922734+0000","last_undegraded":"2024-02-18T00:31:57.922734+0000","last_fullsized":"2024-02-18T00:31:57.922734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:47.502660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:47.502660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311528+0000","last_change":"2024-02-18T00:31:46.851657+0000","last_active":"2024-02-18T00:31:58.311528+0000","last_peered":"2024-02-18T00:31:58.311528+0000","last_clean":"2024-02-18T00:31:58.311528+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:31:58.311528+0000","last_undegraded":"2024-02-18T00:31:58.311528+0000","last_fullsized":"2024-02-18T00:31:58.311528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.851573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310876+0000","last_change":"2024-02-18T00:30:56.034602+0000","last_active":"2024-02-18T00:31:58.310876+0000","last_peered":"2024-02-18T00:31:58.310876+0000","last_clean":"2024-02-18T00:31:58.310876+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:31:58.310876+0000","last_undegraded":"2024-02-18T00:31:58.310876+0000","last_fullsized":"2024-02-18T00:31:58.310876+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:56.034493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:56.034493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310922+0000","last_change":"2024-02-18T00:31:42.847902+0000","last_active":"2024-02-18T00:31:58.310922+0000","last_peered":"2024-02-18T00:31:58.310922+0000","last_clean":"2024-02-18T00:31:58.310922+0000","last_became_active":"2024-02-18T00:29:14.716595+0000","last_became_peered":"2024-02-18T00:29:14.716595+0000","last_unstale":"2024-02-18T00:31:58.310922+0000","last_undegraded":"2024-02-18T00:31:58.310922+0000","last_fullsized":"2024-02-18T00:31:58.310922+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200000+0000","last_change":"2024-02-18T00:30:46.504493+0000","last_active":"2024-02-18T00:31:58.200000+0000","last_peered":"2024-02-18T00:31:58.200000+0000","last_clean":"2024-02-18T00:31:58.200000+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:31:58.200000+0000","last_undegraded":"2024-02-18T00:31:58.200000+0000","last_fullsized":"2024-02-18T00:31:58.200000+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.504422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.504422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.199489+0000","last_change":"2024-02-18T00:31:25.298149+0000","last_active":"2024-02-18T00:31:58.199489+0000","last_peered":"2024-02-18T00:31:58.199489+0000","last_clean":"2024-02-18T00:31:58.199489+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:31:58.199489+0000","last_undegraded":"2024-02-18T00:31:58.199489+0000","last_fullsized":"2024-02-18T00:31:58.199489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:31:25.298062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:25.298062+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200567+0000","last_change":"2024-02-18T00:30:27.608046+0000","last_active":"2024-02-18T00:31:58.200567+0000","last_peered":"2024-02-18T00:31:58.200567+0000","last_clean":"2024-02-18T00:31:58.200567+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:31:58.200567+0000","last_undegraded":"2024-02-18T00:31:58.200567+0000","last_fullsized":"2024-02-18T00:31:58.200567+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:27.607981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.607981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.926733+0000","last_change":"2024-02-18T00:31:00.091252+0000","last_active":"2024-02-18T00:31:57.926733+0000","last_peered":"2024-02-18T00:31:57.926733+0000","last_clean":"2024-02-18T00:31:57.926733+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:31:57.926733+0000","last_undegraded":"2024-02-18T00:31:57.926733+0000","last_fullsized":"2024-02-18T00:31:57.926733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:00.091169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.091169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":54,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310640+0000","last_change":"2024-02-18T00:31:53.976449+0000","last_active":"2024-02-18T00:31:58.310640+0000","last_peered":"2024-02-18T00:31:58.310640+0000","last_clean":"2024-02-18T00:31:58.310640+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:31:58.310640+0000","last_undegraded":"2024-02-18T00:31:58.310640+0000","last_fullsized":"2024-02-18T00:31:58.310640+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:53.976377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:31:53.976377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.199985+0000","last_change":"2024-02-18T00:30:54.590835+0000","last_active":"2024-02-18T00:31:58.199985+0000","last_peered":"2024-02-18T00:31:58.199985+0000","last_clean":"2024-02-18T00:31:58.199985+0000","last_became_active":"2024-02-18T00:29:14.714355+0000","last_became_peered":"2024-02-18T00:29:14.714355+0000","last_unstale":"2024-02-18T00:31:58.199985+0000","last_undegraded":"2024-02-18T00:31:58.199985+0000","last_fullsized":"2024-02-18T00:31:58.199985+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.590771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.590771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":66,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310852+0000","last_change":"2024-02-18T00:31:01.883504+0000","last_active":"2024-02-18T00:31:58.310852+0000","last_peered":"2024-02-18T00:31:58.310852+0000","last_clean":"2024-02-18T00:31:58.310852+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:31:58.310852+0000","last_undegraded":"2024-02-18T00:31:58.310852+0000","last_fullsized":"2024-02-18T00:31:58.310852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:01.883424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.883424+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311516+0000","last_change":"2024-02-18T00:30:52.072719+0000","last_active":"2024-02-18T00:31:58.311516+0000","last_peered":"2024-02-18T00:31:58.311516+0000","last_clean":"2024-02-18T00:31:58.311516+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:31:58.311516+0000","last_undegraded":"2024-02-18T00:31:58.311516+0000","last_fullsized":"2024-02-18T00:31:58.311516+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.072619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.072619+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.924840+0000","last_change":"2024-02-18T00:31:11.020203+0000","last_active":"2024-02-18T00:31:57.924840+0000","last_peered":"2024-02-18T00:31:57.924840+0000","last_clean":"2024-02-18T00:31:57.924840+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:31:57.924840+0000","last_undegraded":"2024-02-18T00:31:57.924840+0000","last_fullsized":"2024-02-18T00:31:57.924840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:11.020119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.020119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200713+0000","last_change":"2024-02-18T00:30:53.587885+0000","last_active":"2024-02-18T00:31:58.200713+0000","last_peered":"2024-02-18T00:31:58.200713+0000","last_clean":"2024-02-18T00:31:58.200713+0000","last_became_active":"2024-02-18T00:29:14.715756+0000","last_became_peered":"2024-02-18T00:29:14.715756+0000","last_unstale":"2024-02-18T00:31:58.200713+0000","last_undegraded":"2024-02-18T00:31:58.200713+0000","last_fullsized":"2024-02-18T00:31:58.200713+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:30:53.587796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:53.587796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":99,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311010+0000","last_change":"2024-02-18T00:30:48.688989+0000","last_active":"2024-02-18T00:31:58.311010+0000","last_peered":"2024-02-18T00:31:58.311010+0000","last_clean":"2024-02-18T00:31:58.311010+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:31:58.311010+0000","last_undegraded":"2024-02-18T00:31:58.311010+0000","last_fullsized":"2024-02-18T00:31:58.311010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:30:48.688901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.688901+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.198838+0000","last_change":"2024-02-18T00:30:45.463697+0000","last_active":"2024-02-18T00:31:58.198838+0000","last_peered":"2024-02-18T00:31:58.198838+0000","last_clean":"2024-02-18T00:31:58.198838+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:31:58.198838+0000","last_undegraded":"2024-02-18T00:31:58.198838+0000","last_fullsized":"2024-02-18T00:31:58.198838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918618+0000","last_change":"2024-02-18T00:30:45.062422+0000","last_active":"2024-02-18T00:31:57.918618+0000","last_peered":"2024-02-18T00:31:57.918618+0000","last_clean":"2024-02-18T00:31:57.918618+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:31:57.918618+0000","last_undegraded":"2024-02-18T00:31:57.918618+0000","last_fullsized":"2024-02-18T00:31:57.918618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.062339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.062339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.927273+0000","last_change":"2024-02-18T00:31:38.977920+0000","last_active":"2024-02-18T00:31:57.927273+0000","last_peered":"2024-02-18T00:31:57.927273+0000","last_clean":"2024-02-18T00:31:57.927273+0000","last_became_active":"2024-02-18T00:29:14.714616+0000","last_became_peered":"2024-02-18T00:29:14.714616+0000","last_unstale":"2024-02-18T00:31:57.927273+0000","last_undegraded":"2024-02-18T00:31:57.927273+0000","last_fullsized":"2024-02-18T00:31:57.927273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":126,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:05.656439+0000","last_change":"2024-02-18T00:32:05.656439+0000","last_active":"2024-02-18T00:32:05.656439+0000","last_peered":"2024-02-18T00:32:05.656439+0000","last_clean":"2024-02-18T00:32:05.656439+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:32:05.656439+0000","last_undegraded":"2024-02-18T00:32:05.656439+0000","last_fullsized":"2024-02-18T00:32:05.656439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.656344+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918332+0000","last_change":"2024-02-18T00:30:26.099064+0000","last_active":"2024-02-18T00:31:57.918332+0000","last_peered":"2024-02-18T00:31:57.918332+0000","last_clean":"2024-02-18T00:31:57.918332+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:31:57.918332+0000","last_undegraded":"2024-02-18T00:31:57.918332+0000","last_fullsized":"2024-02-18T00:31:57.918332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.098982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.098982+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200488+0000","last_change":"2024-02-18T00:30:44.565713+0000","last_active":"2024-02-18T00:31:58.200488+0000","last_peered":"2024-02-18T00:31:58.200488+0000","last_clean":"2024-02-18T00:31:58.200488+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:31:58.200488+0000","last_undegraded":"2024-02-18T00:31:58.200488+0000","last_fullsized":"2024-02-18T00:31:58.200488+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.565626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.565626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.199202+0000","last_change":"2024-02-18T00:31:58.198693+0000","last_active":"2024-02-18T00:31:58.199202+0000","last_peered":"2024-02-18T00:31:58.199202+0000","last_clean":"2024-02-18T00:31:58.199202+0000","last_became_active":"2024-02-18T00:29:14.714186+0000","last_became_peered":"2024-02-18T00:29:14.714186+0000","last_unstale":"2024-02-18T00:31:58.199202+0000","last_undegraded":"2024-02-18T00:31:58.199202+0000","last_fullsized":"2024-02-18T00:31:58.199202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.198542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.198542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":90,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918095+0000","last_change":"2024-02-18T00:31:56.063646+0000","last_active":"2024-02-18T00:31:57.918095+0000","last_peered":"2024-02-18T00:31:57.918095+0000","last_clean":"2024-02-18T00:31:57.918095+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:31:57.918095+0000","last_undegraded":"2024-02-18T00:31:57.918095+0000","last_fullsized":"2024-02-18T00:31:57.918095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:31:56.063560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.063560+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":95,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:31:59.648848+0000","last_change":"2024-02-18T00:31:39.294080+0000","last_active":"2024-02-18T00:31:59.648848+0000","last_peered":"2024-02-18T00:31:59.648848+0000","last_clean":"2024-02-18T00:31:59.648848+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:31:59.648848+0000","last_undegraded":"2024-02-18T00:31:59.648848+0000","last_fullsized":"2024-02-18T00:31:59.648848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:31:39.294006+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:31:39.294006+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.530891+0000","last_change":"2024-02-18T00:31:58.530850+0000","last_active":"2024-02-18T00:31:58.530891+0000","last_peered":"2024-02-18T00:31:58.530891+0000","last_clean":"2024-02-18T00:31:58.530891+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:31:58.530891+0000","last_undegraded":"2024-02-18T00:31:58.530891+0000","last_fullsized":"2024-02-18T00:31:58.530891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.530768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.530768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311462+0000","last_change":"2024-02-18T00:31:16.020640+0000","last_active":"2024-02-18T00:31:58.311462+0000","last_peered":"2024-02-18T00:31:58.311462+0000","last_clean":"2024-02-18T00:31:58.311462+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:31:58.311462+0000","last_undegraded":"2024-02-18T00:31:58.311462+0000","last_fullsized":"2024-02-18T00:31:58.311462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:16.020558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:16.020558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200092+0000","last_change":"2024-02-18T00:31:48.594719+0000","last_active":"2024-02-18T00:31:58.200092+0000","last_peered":"2024-02-18T00:31:58.200092+0000","last_clean":"2024-02-18T00:31:58.200092+0000","last_became_active":"2024-02-18T00:29:14.713770+0000","last_became_peered":"2024-02-18T00:29:14.713770+0000","last_unstale":"2024-02-18T00:31:58.200092+0000","last_undegraded":"2024-02-18T00:31:58.200092+0000","last_fullsized":"2024-02-18T00:31:58.200092+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:48.594640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:48.594640+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":88,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918838+0000","last_change":"2024-02-18T00:31:55.104289+0000","last_active":"2024-02-18T00:31:57.918838+0000","last_peered":"2024-02-18T00:31:57.918838+0000","last_clean":"2024-02-18T00:31:57.918838+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:31:57.918838+0000","last_undegraded":"2024-02-18T00:31:57.918838+0000","last_fullsized":"2024-02-18T00:31:57.918838+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:55.104195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.104195+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311089+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:31:58.311089+0000","last_peered":"2024-02-18T00:31:58.311089+0000","last_clean":"2024-02-18T00:31:58.311089+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:31:58.311089+0000","last_undegraded":"2024-02-18T00:31:58.311089+0000","last_fullsized":"2024-02-18T00:31:58.311089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":89,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:31:59.987601+0000","last_change":"2024-02-18T00:31:05.997541+0000","last_active":"2024-02-18T00:31:59.987601+0000","last_peered":"2024-02-18T00:31:59.987601+0000","last_clean":"2024-02-18T00:31:59.987601+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:31:59.987601+0000","last_undegraded":"2024-02-18T00:31:59.987601+0000","last_fullsized":"2024-02-18T00:31:59.987601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:05.997460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.997460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.925064+0000","last_change":"2024-02-18T00:31:46.960767+0000","last_active":"2024-02-18T00:31:57.925064+0000","last_peered":"2024-02-18T00:31:57.925064+0000","last_clean":"2024-02-18T00:31:57.925064+0000","last_became_active":"2024-02-18T00:29:14.713732+0000","last_became_peered":"2024-02-18T00:29:14.713732+0000","last_unstale":"2024-02-18T00:31:57.925064+0000","last_undegraded":"2024-02-18T00:31:57.925064+0000","last_fullsized":"2024-02-18T00:31:57.925064+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.960661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.960661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":57,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311324+0000","last_change":"2024-02-18T00:31:50.844080+0000","last_active":"2024-02-18T00:31:58.311324+0000","last_peered":"2024-02-18T00:31:58.311324+0000","last_clean":"2024-02-18T00:31:58.311324+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:31:58.311324+0000","last_undegraded":"2024-02-18T00:31:58.311324+0000","last_fullsized":"2024-02-18T00:31:58.311324+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.843994+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":50,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.064685+0000","last_change":"2024-02-18T00:31:58.064645+0000","last_active":"2024-02-18T00:31:58.064685+0000","last_peered":"2024-02-18T00:31:58.064685+0000","last_clean":"2024-02-18T00:31:58.064685+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:31:58.064685+0000","last_undegraded":"2024-02-18T00:31:58.064685+0000","last_fullsized":"2024-02-18T00:31:58.064685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.064559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.064559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":92,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:31:59.986481+0000","last_change":"2024-02-18T00:30:51.228973+0000","last_active":"2024-02-18T00:31:59.986481+0000","last_peered":"2024-02-18T00:31:59.986481+0000","last_clean":"2024-02-18T00:31:59.986481+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:31:59.986481+0000","last_undegraded":"2024-02-18T00:31:59.986481+0000","last_fullsized":"2024-02-18T00:31:59.986481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:51.228902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.228902+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.925477+0000","last_change":"2024-02-18T00:31:50.988509+0000","last_active":"2024-02-18T00:31:57.925477+0000","last_peered":"2024-02-18T00:31:57.925477+0000","last_clean":"2024-02-18T00:31:57.925477+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:31:57.925477+0000","last_undegraded":"2024-02-18T00:31:57.925477+0000","last_fullsized":"2024-02-18T00:31:57.925477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.988429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":64,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918407+0000","last_change":"2024-02-18T00:31:11.002136+0000","last_active":"2024-02-18T00:31:57.918407+0000","last_peered":"2024-02-18T00:31:57.918407+0000","last_clean":"2024-02-18T00:31:57.918407+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:31:57.918407+0000","last_undegraded":"2024-02-18T00:31:57.918407+0000","last_fullsized":"2024-02-18T00:31:57.918407+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:31:11.002039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.002039+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.922406+0000","last_change":"2024-02-18T00:30:41.301486+0000","last_active":"2024-02-18T00:31:57.922406+0000","last_peered":"2024-02-18T00:31:57.922406+0000","last_clean":"2024-02-18T00:31:57.922406+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:31:57.922406+0000","last_undegraded":"2024-02-18T00:31:57.922406+0000","last_fullsized":"2024-02-18T00:31:57.922406+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:41.301405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:41.301405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.926820+0000","last_change":"2024-02-18T00:30:59.059018+0000","last_active":"2024-02-18T00:31:57.926820+0000","last_peered":"2024-02-18T00:31:57.926820+0000","last_clean":"2024-02-18T00:31:57.926820+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:31:57.926820+0000","last_undegraded":"2024-02-18T00:31:57.926820+0000","last_fullsized":"2024-02-18T00:31:57.926820+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.058947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.058947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.200208+0000","last_change":"2024-02-18T00:31:54.616292+0000","last_active":"2024-02-18T00:31:58.200208+0000","last_peered":"2024-02-18T00:31:58.200208+0000","last_clean":"2024-02-18T00:31:58.200208+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:31:58.200208+0000","last_undegraded":"2024-02-18T00:31:58.200208+0000","last_fullsized":"2024-02-18T00:31:58.200208+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:54.616213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.616213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":82,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.918708+0000","last_change":"2024-02-18T00:31:32.011975+0000","last_active":"2024-02-18T00:31:57.918708+0000","last_peered":"2024-02-18T00:31:57.918708+0000","last_clean":"2024-02-18T00:31:57.918708+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:31:57.918708+0000","last_undegraded":"2024-02-18T00:31:57.918708+0000","last_fullsized":"2024-02-18T00:31:57.918708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:32.011896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.011896+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.199911+0000","last_change":"2024-02-18T00:30:43.541530+0000","last_active":"2024-02-18T00:31:58.199911+0000","last_peered":"2024-02-18T00:31:58.199911+0000","last_clean":"2024-02-18T00:31:58.199911+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:31:58.199911+0000","last_undegraded":"2024-02-18T00:31:58.199911+0000","last_fullsized":"2024-02-18T00:31:58.199911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:02.507109+0000","last_change":"2024-02-18T00:32:02.507109+0000","last_active":"2024-02-18T00:32:02.507109+0000","last_peered":"2024-02-18T00:32:02.507109+0000","last_clean":"2024-02-18T00:32:02.507109+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:32:02.507109+0000","last_undegraded":"2024-02-18T00:32:02.507109+0000","last_fullsized":"2024-02-18T00:32:02.507109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:02.507010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.507010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:04.461036+0000","last_change":"2024-02-18T00:32:04.461036+0000","last_active":"2024-02-18T00:32:04.461036+0000","last_peered":"2024-02-18T00:32:04.461036+0000","last_clean":"2024-02-18T00:32:04.461036+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:32:04.461036+0000","last_undegraded":"2024-02-18T00:32:04.461036+0000","last_fullsized":"2024-02-18T00:32:04.461036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:04.460968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.460968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":101,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.198772+0000","last_change":"2024-02-18T00:31:40.251680+0000","last_active":"2024-02-18T00:31:58.198772+0000","last_peered":"2024-02-18T00:31:58.198772+0000","last_clean":"2024-02-18T00:31:58.198772+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:31:58.198772+0000","last_undegraded":"2024-02-18T00:31:58.198772+0000","last_fullsized":"2024-02-18T00:31:58.198772+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:31:40.251596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.251596+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310382+0000","last_change":"2024-02-18T00:30:50.764741+0000","last_active":"2024-02-18T00:31:58.310382+0000","last_peered":"2024-02-18T00:31:58.310382+0000","last_clean":"2024-02-18T00:31:58.310382+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:31:58.310382+0000","last_undegraded":"2024-02-18T00:31:58.310382+0000","last_fullsized":"2024-02-18T00:31:58.310382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.764665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.764665+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:00.527642+0000","last_change":"2024-02-18T00:32:00.527642+0000","last_active":"2024-02-18T00:32:00.527642+0000","last_peered":"2024-02-18T00:32:00.527642+0000","last_clean":"2024-02-18T00:32:00.527642+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:32:00.527642+0000","last_undegraded":"2024-02-18T00:32:00.527642+0000","last_fullsized":"2024-02-18T00:32:00.527642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:00.527580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:00.527580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:32:03.931078+0000","last_change":"2024-02-18T00:32:03.931078+0000","last_active":"2024-02-18T00:32:03.931078+0000","last_peered":"2024-02-18T00:32:03.931078+0000","last_clean":"2024-02-18T00:32:03.931078+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:32:03.931078+0000","last_undegraded":"2024-02-18T00:32:03.931078+0000","last_fullsized":"2024-02-18T00:32:03.931078+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:32:03.930967+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":79,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.311197+0000","last_change":"2024-02-18T00:31:07.953329+0000","last_active":"2024-02-18T00:31:58.311197+0000","last_peered":"2024-02-18T00:31:58.311197+0000","last_clean":"2024-02-18T00:31:58.311197+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:31:58.311197+0000","last_undegraded":"2024-02-18T00:31:58.311197+0000","last_fullsized":"2024-02-18T00:31:58.311197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:07.953265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.953265+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310750+0000","last_change":"2024-02-18T00:30:30.045287+0000","last_active":"2024-02-18T00:31:58.310750+0000","last_peered":"2024-02-18T00:31:58.310750+0000","last_clean":"2024-02-18T00:31:58.310750+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:31:58.310750+0000","last_undegraded":"2024-02-18T00:31:58.310750+0000","last_fullsized":"2024-02-18T00:31:58.310750+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:30.045217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:30.045217+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:57.922879+0000","last_change":"2024-02-18T00:31:57.561503+0000","last_active":"2024-02-18T00:31:57.922879+0000","last_peered":"2024-02-18T00:31:57.922879+0000","last_clean":"2024-02-18T00:31:57.922879+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:31:57.922879+0000","last_undegraded":"2024-02-18T00:31:57.922879+0000","last_fullsized":"2024-02-18T00:31:57.922879+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.561435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.561435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":104,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-02-18T00:31:58.310620+0000","last_change":"2024-02-18T00:31:07.029773+0000","last_active":"2024-02-18T00:31:58.310620+0000","last_peered":"2024-02-18T00:31:58.310620+0000","last_clean":"2024-02-18T00:31:58.310620+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:31:58.310620+0000","last_undegraded":"2024-02-18T00:31:58.310620+0000","last_fullsized":"2024-02-18T00:31:58.310620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:31:07.029677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.029677+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":89,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-02-18T00:31:59.987069+0000","last_change":"2024-02-18T00:30:49.588342+0000","last_active":"2024-02-18T00:31:59.987069+0000","last_peered":"2024-02-18T00:31:59.987069+0000","last_clean":"2024-02-18T00:31:59.987069+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:31:59.987069+0000","last_undegraded":"2024-02-18T00:31:59.987069+0000","last_fullsized":"2024-02-18T00:31:59.987069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:49.588269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:49.588269+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":605,"ondisk_log_size":605,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":162,"num_read_kb":157,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266726,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":1003520,"data_stored":425810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78200000000000003,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.505,"15min":0.502},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.38700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84899999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86799999999999999,"5min":0.88,"15min":0.88},"last":0.36399999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.51000000000000001},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.378}]},{"osd":3,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.34000000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.39400000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.497}]},{"osd":4,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.41799999999999998,"15min":0.41199999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41299999999999998,"15min":0.40799999999999997},"min":{"1min":0.30599999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.52700000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.435},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.54100000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56599999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298056,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447096,"statfs":{"total":95995035648,"available":95689826304,"internally_reserved":0,"allocated":983040,"data_stored":424645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.39500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67500000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.497},"min":{"1min":0.42599999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.627,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.42799999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67000000000000004,"5min":0.874,"15min":0.874},"last":0.434},{"interface":"front","average":{"1min":0.503,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.41099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61299999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69999999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.40200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.59499999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.42399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.67500000000000004,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80400000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.438,"15min":0.41399999999999998},"min":{"1min":0.38900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.121,"15min":1.121},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.44800000000000001,"15min":0.42199999999999999},"min":{"1min":0.35999999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72999999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51700000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.64000000000000001,"5min":1.075,"15min":1.075},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66700000000000004,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298096,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447056,"statfs":{"total":95995035648,"available":95689785344,"internally_reserved":0,"allocated":1024000,"data_stored":425468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.374,"15min":0.316},"min":{"1min":0.25800000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.49199999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.372,"15min":0.32500000000000001},"min":{"1min":0.26300000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53900000000000003,"5min":1.129,"15min":1.129},"last":0.46200000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.373},"min":{"1min":0.27700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41399999999999998,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.35299999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.55500000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.377}]},{"osd":4,"last update":"Sun Feb 18 00:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":1.347,"15min":1.347},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81999999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.51000000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.627}]},{"osd":6,"last update":"Sun Feb 18 00:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80500000000000005,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83699999999999997,"5min":1.278,"15min":1.278},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:31:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.79400000000000004,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298072,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447080,"statfs":{"total":95995035648,"available":95689809920,"internally_reserved":0,"allocated":999424,"data_stored":425451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.42399999999999999,"15min":0.57699999999999996},"min":{"1min":0.221,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46500000000000002,"5min":6.7720000000000002,"15min":6.7720000000000002},"last":0.248},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38800000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.33500000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.39200000000000002},"min":{"1min":0.23799999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.28299999999999997}]},{"osd":3,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46200000000000002,"15min":0.495},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.307}]},{"osd":4,"last update":"Sun Feb 18 00:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80800000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.45500000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60899999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71199999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64500000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298052,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447100,"statfs":{"total":95995035648,"available":95689830400,"internally_reserved":0,"allocated":978944,"data_stored":424722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39500000000000002,"15min":0.374},"min":{"1min":0.26100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.31},{"interface":"front","average":{"1min":0.379,"5min":0.40400000000000003,"15min":0.378},"min":{"1min":0.29699999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.51600000000000001,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40799999999999997,"15min":0.38600000000000001},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.55200000000000005,"5min":2.306,"15min":2.306},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.42299999999999999,"15min":0.40100000000000002},"min":{"1min":0.27400000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58899999999999997,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.55600000000000005,"5min":1.296,"15min":1.296},"last":0.378},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47799999999999998,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61899999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.32300000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85099999999999998,"5min":1.675,"15min":1.675},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":1.173,"15min":1.173},"last":0.502}]},{"osd":5,"last update":"Sun Feb 18 00:31:28 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.41499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81399999999999995,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61199999999999999,"15min":0.624},"min":{"1min":0.46000000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85399999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61499999999999999,"15min":0.64000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69699999999999995,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90100000000000002,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.46100000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67500000000000004,"15min":0.69599999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72399999999999998,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89300000000000002,"5min":1.242,"15min":1.242},"last":0.52000000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298088,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447064,"statfs":{"total":95995035648,"available":95689793536,"internally_reserved":0,"allocated":1015808,"data_stored":425120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.374,"15min":0.39000000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47699999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40899999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56000000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.27300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.27800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.505,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.315},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.28399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.49299999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Sun Feb 18 00:31:31 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.434,"15min":0.437},"min":{"1min":0.29799999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64600000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41999999999999998,"15min":0.42899999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.26000000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:32:02 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.46899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.878,"5min":1.083,"15min":1.083},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98199999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:31:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82499999999999996,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60199999999999998,"15min":0.60999999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96599999999999997,"5min":1.139,"15min":1.139},"last":0.69399999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94699999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.623,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94799999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81000000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80100000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70499999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":1003520,"data_stored":425605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":2.339,"15min":2.339},"last":0.442},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72199999999999998,"5min":1.288,"15min":1.288},"last":0.56499999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.745,"5min":1.204,"15min":1.204},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78200000000000003,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.61199999999999999},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81399999999999995,"5min":1.48,"15min":1.48},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79300000000000004,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":1.133,"15min":1.133},"last":0.497}]},{"osd":5,"last update":"Sun Feb 18 00:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.38,"15min":0.35999999999999999},"min":{"1min":0.33300000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.36799999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.51900000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.25,"15min":0.25},"max":{"1min":0.69099999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.436},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:31:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.504,"5min":0.47499999999999998,"15min":0.46999999999999997},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53500000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298048,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447104,"statfs":{"total":95995035648,"available":95689834496,"internally_reserved":0,"allocated":974848,"data_stored":424005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.46400000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":1.306,"15min":1.306},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56299999999999994},"min":{"1min":0.40699999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.80000000000000004,"5min":1.272,"15min":1.272},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.39000000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78800000000000003,"5min":1.173,"15min":1.173},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.442,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42799999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":1.238,"15min":1.238},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.55600000000000005},"min":{"1min":0.36699999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72099999999999997,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:31:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"min":{"1min":0.29399999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.495},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.40000000000000002,"15min":0.378},"min":{"1min":0.28699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44}]},{"osd":6,"last update":"Sun Feb 18 00:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.55900000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:31:10 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.47199999999999998,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.53400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:32:08.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool set .rgw.root pgp_num 32 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:06.682192+0000 mgr.y (mgr.24454) 116 : audit [DBG] from='client.24656 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:06.738976+0000 osd.7 (osd.7) 53 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:06.739939+0000 osd.7 (osd.7) 54 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:06.882026+0000 osd.1 (osd.1) 65 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:06.883015+0000 osd.1 (osd.1) 66 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.007357+0000 osd.0 (osd.0) 55 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.008535+0000 osd.0 (osd.0) 56 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:07.313110+0000 mon.c (mon.2) 28 : audit [INF] from='client.? 172.21.15.59:0/454391748' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:07.313569+0000 mon.a (mon.0) 760 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.446324+0000 osd.2 (osd.2) 51 : cluster [DBG] 6.f scrub starts 2024-02-18T00:32:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.449245+0000 osd.2 (osd.2) 52 : cluster [DBG] 6.f scrub ok 2024-02-18T00:32:09.104 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:09.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:09.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:09.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:09.235 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.257321+0000 osd.6 (osd.6) 47 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.258234+0000 osd.6 (osd.6) 48 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.753949+0000 mgr.y (mgr.24454) 117 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.758974+0000 osd.7 (osd.7) 55 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.759959+0000 osd.7 (osd.7) 56 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.942026+0000 mon.a (mon.0) 761 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:07.955842+0000 mon.a (mon.0) 762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:07.955886+0000 mon.a (mon.0) 763 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:08.081159+0000 osd.0 (osd.0) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:09.236 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:08.081231+0000 osd.0 (osd.0) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.257321+0000 osd.6 (osd.6) 47 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.258234+0000 osd.6 (osd.6) 48 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.753949+0000 mgr.y (mgr.24454) 117 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.758974+0000 osd.7 (osd.7) 55 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.759959+0000 osd.7 (osd.7) 56 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.942026+0000 mon.a (mon.0) 761 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:07.955842+0000 mon.a (mon.0) 762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:07.955886+0000 mon.a (mon.0) 763 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:08.081159+0000 osd.0 (osd.0) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:09.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:08.081231+0000 osd.0 (osd.0) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:09.238 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.257321+0000 osd.6 (osd.6) 47 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.258234+0000 osd.6 (osd.6) 48 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.753949+0000 mgr.y (mgr.24454) 117 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.758974+0000 osd.7 (osd.7) 55 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.759959+0000 osd.7 (osd.7) 56 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.942026+0000 mon.a (mon.0) 761 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:07.955842+0000 mon.a (mon.0) 762 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:07.955886+0000 mon.a (mon.0) 763 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:08.081159+0000 osd.0 (osd.0) 57 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:08.081231+0000 osd.0 (osd.0) 58 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:09.438 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:09.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:09.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:09.446 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:09.543 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:09.576 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:09.597 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:09.599 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:09.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:09.701 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:09.731 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:09.751 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:09.753 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:09.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:09.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:09.887 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:10.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:08.961447+0000 mon.a (mon.0) 764 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-18T00:32:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:09.874026+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:32:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:08.961447+0000 mon.a (mon.0) 764 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-18T00:32:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:09.874026+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:32:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:08.961447+0000 mon.a (mon.0) 764 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-18T00:32:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:09.874026+0000 mon.a (mon.0) 765 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:32:10.818 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:11.121 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:32:11.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:09.754313+0000 mgr.y (mgr.24454) 118 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:11.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:09.754313+0000 mgr.y (mgr.24454) 118 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:11.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:32:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:32:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:09.754313+0000 mgr.y (mgr.24454) 118 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:11.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:32:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:32:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:11.846 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:11.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:11.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:11.983 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:12.105 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:10.874492+0000 mgr.y (mgr.24454) 119 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:12.105 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:10.874492+0000 mgr.y (mgr.24454) 119 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:12.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:10.874492+0000 mgr.y (mgr.24454) 119 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:12.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:12] "GET /metrics HTTP/1.1" 200 172239 "" "Prometheus/2.33.4" 2024-02-18T00:32:12.512 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:32:12.512 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:32:12.534 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:12.538 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:12.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:12.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:12.669 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:12.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:12.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:12.846 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:12.879 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:12.900 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:12.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:12.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:13.003 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:13.033 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:13.056 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:13.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:13.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:13.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:11.588913+0000 osd.6 (osd.6) 49 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:13.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:11.588971+0000 osd.6 (osd.6) 50 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:13.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:11.754892+0000 mgr.y (mgr.24454) 120 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:13.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:11.975347+0000 mon.a (mon.0) 766 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-18T00:32:13.186 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:13.205 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:13.207 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:13.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:13.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:11.588913+0000 osd.6 (osd.6) 49 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:11.588971+0000 osd.6 (osd.6) 50 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:11.754892+0000 mgr.y (mgr.24454) 120 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:11.975347+0000 mon.a (mon.0) 766 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-18T00:32:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:11.588913+0000 osd.6 (osd.6) 49 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:11.588971+0000 osd.6 (osd.6) 50 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:11.754892+0000 mgr.y (mgr.24454) 120 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:11.975347+0000 mon.a (mon.0) 766 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-18T00:32:13.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:13.336 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:13.375 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:13.378 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:13.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:13.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-18T00:32:13.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:13.547 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:13.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:13.550 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:13.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.854+0000 7faf87412700 1 -- 172.21.15.59:0/890065920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fd510 msgr2=0x7faf800f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.854+0000 7faf87412700 1 --2- 172.21.15.59:0/890065920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fd510 0x7faf800f6d40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7faf70004870 tx=0x7faf70009cb0).stop 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 -- 172.21.15.59:0/890065920 shutdown_connections 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 --2- 172.21.15.59:0/890065920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf800fcbb0 0x7faf800fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 --2- 172.21.15.59:0/890065920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf800f7400 0x7faf80103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 --2- 172.21.15.59:0/890065920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fd510 0x7faf800f6d40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 -- 172.21.15.59:0/890065920 >> 172.21.15.59:0/890065920 conn(0x7faf800f2820 msgr2=0x7faf800f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:13.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 -- 172.21.15.59:0/890065920 shutdown_connections 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 -- 172.21.15.59:0/890065920 wait complete. 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 librados: starting msgr at 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 librados: starting objecter 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.855+0000 7faf87412700 1 Processor -- start 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 -- start start 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 librados: setting wanted keys 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 librados: calling monclient init 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf800f7400 0x7faf80104350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fcbb0 0x7faf80104840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf800fd510 0x7faf80104d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf801166c0 con 0x7faf800fcbb0 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf80116830 con 0x7faf800f7400 2024-02-18T00:32:13.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf87412700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf801169f0 con 0x7faf800fd510 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fcbb0 0x7faf80104840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fcbb0 0x7faf80104840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41180/0 (socket says 172.21.15.59:41180) 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 -- 172.21.15.59:0/1552172698 learned_addr learned my addr 172.21.15.59:0/1552172698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf859af700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf800fd510 0x7faf80104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf800f7400 0x7faf80104350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 -- 172.21.15.59:0/1552172698 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf800fd510 msgr2=0x7faf80104d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf800fd510 0x7faf80104d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 -- 172.21.15.59:0/1552172698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf800f7400 msgr2=0x7faf80104350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf800f7400 0x7faf80104350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 -- 172.21.15.59:0/1552172698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf70005040 con 0x7faf800fcbb0 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf800f7400 0x7faf80104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf849ad700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fcbb0 0x7faf80104840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7faf700043a0 tx=0x7faf7000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf767fc700 10 client.?.objecter ms_handle_connect 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.856+0000 7faf767fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.857+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7000e070 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.857+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf70009710 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.857+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70019070 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.857+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf80116c70 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.857+0000 7faf87412700 10 client.14763.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.857+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf80117070 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.857+0000 7faf87412700 1 librados: init done 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.858+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7faf8005fa60 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.859+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7faf700090d0 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.859+0000 7faf767fc700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf6c062c90 0x7faf6c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.859+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 44..98) v4 ==== 6172+0+0 (secure 0 0 0) 0x7faf7007d380 con 0x7faf800fcbb0 2024-02-18T00:32:13.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.859+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 osd_map(98..98 src has 44..98) v4 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.859+0000 7faf767fc700 3 client.14763.objecter handle_osd_map got epochs [98,98] > 0 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.859+0000 7faf767fc700 3 client.14763.objecter handle_osd_map decoding full epoch 98 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.859+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf6c062c90 0x7faf6c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf767fc700 20 client.14763.objecter dump_active .. 0 homeless 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v98) v1 ==== 54+0+16 (secure 0 0 0) 0x7faf70084070 con 0x7faf800fcbb0 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 10 client.14763.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 1 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf6c062c90 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf6c062c90 0x7faf6c065150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7faf7c007fd0 tx=0x7faf7c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6801/3700238977 conn(0x7faf800fdea0 0x7faf800fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 1 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 0 1 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8005ee80 osd=0 4 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 10 client.14763.objecter _send_command 1 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf80107220 con 0x7faf800fdea0 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf859af700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6801/3700238977 conn(0x7faf800fdea0 0x7faf800fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf800fdea0 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.860+0000 7faf859af700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6801/3700238977 conn(0x7faf800fdea0 0x7faf800fe2e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.862+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf80107220 con 0x7faf800fdea0 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.862+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.862+0000 7faf767fc700 10 client.14763.objecter _finish_command 1 = system:0 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.862+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf8005ee80 osd=0 4 2024-02-18T00:32:13.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.862+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 0 1 2024-02-18T00:32:13.875 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:13.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 10 client.14763.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 2 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 2 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf8005ee80 osd=0 3 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 0 2 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8005ee80 osd=0 4 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 10 client.14763.objecter _send_command 2 2024-02-18T00:32:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.883+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf800273c0 con 0x7faf800fdea0 2024-02-18T00:32:13.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.885+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf800273c0 con 0x7faf800fdea0 2024-02-18T00:32:13.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.885+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:13.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.885+0000 7faf767fc700 10 client.14763.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:13.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.885+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf8005ee80 osd=0 4 2024-02-18T00:32:13.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.885+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 0 2 2024-02-18T00:32:13.887 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:32:13.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 10 client.14763.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:13.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 3 2024-02-18T00:32:13.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6805/26279110 conn(0x7faf800598f0 0x7faf80059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 3 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 1 3 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf800286b0 osd=1 4 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 10 client.14763.objecter _send_command 3 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf80028820 con 0x7faf800598f0 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6805/26279110 conn(0x7faf800598f0 0x7faf80059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6805/26279110 conn(0x7faf800598f0 0x7faf80059d10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.886+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf800598f0 2024-02-18T00:32:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.887+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf7c00f040 con 0x7faf800598f0 2024-02-18T00:32:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.887+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:32:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.887+0000 7faf767fc700 10 client.14763.objecter _finish_command 3 = system:0 2024-02-18T00:32:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.887+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf800286b0 osd=1 4 2024-02-18T00:32:13.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.887+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 1 3 2024-02-18T00:32:13.908 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:13.911 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:13.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 10 client.14763.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 4 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 4 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf800286b0 osd=1 3 2024-02-18T00:32:13.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 1 4 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf800286b0 osd=1 4 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 10 client.14763.objecter _send_command 4 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.913+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf80059360 con 0x7faf800598f0 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.914+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf80059360 con 0x7faf800598f0 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.914+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.914+0000 7faf767fc700 10 client.14763.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.914+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf800286b0 osd=1 4 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.914+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 1 4 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 10 client.14763.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 5 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6809/1153193637 conn(0x7faf80028210 0x7faf8011e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 5 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 2 5 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80059770 osd=2 4 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 10 client.14763.objecter _send_command 5 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.915+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf80121090 con 0x7faf80028210 2024-02-18T00:32:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.916+0000 7faf849ad700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6809/1153193637 conn(0x7faf80028210 0x7faf8011e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.921+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf80028210 2024-02-18T00:32:13.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.921+0000 7faf849ad700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6809/1153193637 conn(0x7faf80028210 0x7faf8011e3b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.922+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf80121090 con 0x7faf80028210 2024-02-18T00:32:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.922+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:32:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.922+0000 7faf767fc700 10 client.14763.objecter _finish_command 5 = system:0 2024-02-18T00:32:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.922+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80059770 osd=2 4 2024-02-18T00:32:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.922+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 2 5 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 10 client.14763.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 6 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 6 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80059770 osd=2 3 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 2 6 2024-02-18T00:32:13.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80059770 osd=2 4 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 10 client.14763.objecter _send_command 6 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.946+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf800277d0 con 0x7faf80028210 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf800277d0 con 0x7faf80028210 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf767fc700 10 client.14763.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80059770 osd=2 4 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 2 6 2024-02-18T00:32:13.951 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 10 client.14763.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 7 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6813/1805700964 conn(0x7faf8011fa20 0x7faf8011fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 7 2024-02-18T00:32:13.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 3 7 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8011b9f0 osd=3 4 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 10 client.14763.objecter _send_command 7 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf8011c100 con 0x7faf8011fa20 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.949+0000 7faf859af700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6813/1805700964 conn(0x7faf8011fa20 0x7faf8011fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.950+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf8011fa20 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.950+0000 7faf859af700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6813/1805700964 conn(0x7faf8011fa20 0x7faf8011fe60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.951+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf8011c100 con 0x7faf8011fa20 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.951+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.951+0000 7faf767fc700 10 client.14763.objecter _finish_command 7 = system:0 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.951+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf8011b9f0 osd=3 4 2024-02-18T00:32:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.951+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 3 7 2024-02-18T00:32:13.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 10 client.14763.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:13.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 8 2024-02-18T00:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 8 2024-02-18T00:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf8011b9f0 osd=3 3 2024-02-18T00:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 3 8 2024-02-18T00:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8011b9f0 osd=3 4 2024-02-18T00:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 10 client.14763.objecter _send_command 8 2024-02-18T00:32:13.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.969+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf80120b20 con 0x7faf8011fa20 2024-02-18T00:32:13.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.970+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf80120b20 con 0x7faf8011fa20 2024-02-18T00:32:13.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.970+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:13.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.970+0000 7faf767fc700 10 client.14763.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:13.973 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.970+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf8011b9f0 osd=3 4 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.970+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 3 8 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 10 client.14763.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 9 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6800/2044387056 conn(0x7faf801232c0 0x7faf801256b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 9 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 4 9 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120350 osd=4 4 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 10 client.14763.objecter _send_command 9 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf80120a90 con 0x7faf801232c0 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6800/2044387056 conn(0x7faf801232c0 0x7faf801256b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf801232c0 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.971+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6800/2044387056 conn(0x7faf801232c0 0x7faf801256b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.972+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf80120a90 con 0x7faf801232c0 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.972+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.972+0000 7faf767fc700 10 client.14763.objecter _finish_command 9 = system:0 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.972+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80120350 osd=4 4 2024-02-18T00:32:13.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.972+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 4 9 2024-02-18T00:32:13.980 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:13.859741+0000 mon.a (mon.0) 767 : audit [DBG] from='client.? 172.21.15.59:0/1552172698' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 10 client.14763.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 10 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 10 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80120350 osd=4 3 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 4 10 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120350 osd=4 4 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 10 client.14763.objecter _send_command 10 2024-02-18T00:32:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.993+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf80120a90 con 0x7faf801232c0 2024-02-18T00:32:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf80120a90 con 0x7faf801232c0 2024-02-18T00:32:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf767fc700 10 client.14763.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80120350 osd=4 4 2024-02-18T00:32:13.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 4 10 2024-02-18T00:32:13.996 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 10 client.14763.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 11 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6804/2170432263 conn(0x7faf80127740 0x7faf80127ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 11 2024-02-18T00:32:13.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 5 11 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120a90 osd=5 4 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 10 client.14763.objecter _send_command 11 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.995+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf8012b4c0 con 0x7faf80127740 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.996+0000 7faf849ad700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6804/2170432263 conn(0x7faf80127740 0x7faf80127ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.996+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf80127740 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.996+0000 7faf849ad700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6804/2170432263 conn(0x7faf80127740 0x7faf80127ba0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.997+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf8012b4c0 con 0x7faf80127740 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.997+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:32:13.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.997+0000 7faf767fc700 10 client.14763.objecter _finish_command 11 = system:0 2024-02-18T00:32:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.997+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80120a90 osd=5 4 2024-02-18T00:32:13.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:13.997+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 5 11 2024-02-18T00:32:14.013 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 10 client.14763.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 12 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 12 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80120a90 osd=5 3 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 5 12 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120a90 osd=5 4 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 10 client.14763.objecter _send_command 12 2024-02-18T00:32:14.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.022+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf80125ba0 con 0x7faf80127740 2024-02-18T00:32:14.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.024+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf80125ba0 con 0x7faf80127740 2024-02-18T00:32:14.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.024+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:14.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.024+0000 7faf767fc700 10 client.14763.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.024+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80120a90 osd=5 4 2024-02-18T00:32:14.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.024+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 5 12 2024-02-18T00:32:14.025 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 10 client.14763.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 13 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6808/3356009390 conn(0x7faf80126220 0x7faf80126660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 13 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 6 13 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80125ba0 osd=6 4 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 10 client.14763.objecter _send_command 13 2024-02-18T00:32:14.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf80126d00 con 0x7faf80126220 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.025+0000 7faf859af700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6808/3356009390 conn(0x7faf80126220 0x7faf80126660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.026+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf80126220 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.026+0000 7faf859af700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6808/3356009390 conn(0x7faf80126220 0x7faf80126660 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.027+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf80126d00 con 0x7faf80126220 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.027+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.027+0000 7faf767fc700 10 client.14763.objecter _finish_command 13 = system:0 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.027+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80125ba0 osd=6 4 2024-02-18T00:32:14.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.027+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 6 13 2024-02-18T00:32:14.050 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:14.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 10 client.14763.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 14 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 14 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80125ba0 osd=6 3 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 6 14 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80125ba0 osd=6 4 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 10 client.14763.objecter _send_command 14 2024-02-18T00:32:14.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.055+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf80126d00 con 0x7faf80126220 2024-02-18T00:32:14.058 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:32:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf80126d00 con 0x7faf80126220 2024-02-18T00:32:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:14.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf767fc700 10 client.14763.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80125ba0 osd=6 4 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 6 14 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 10 client.14763.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 15 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6812/53025171 conn(0x7faf8012c860 0x7faf8012ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 15 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 7 15 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80126d00 osd=7 4 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 10 client.14763.objecter _send_command 15 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.057+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf801318a0 con 0x7faf8012c860 2024-02-18T00:32:14.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.058+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6812/53025171 conn(0x7faf8012c860 0x7faf8012ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.058+0000 7faf767fc700 10 client.14763.objecter ms_handle_connect 0x7faf8012c860 2024-02-18T00:32:14.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.058+0000 7faf851ae700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6812/53025171 conn(0x7faf8012c860 0x7faf8012ec50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.059+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faf801318a0 con 0x7faf8012c860 2024-02-18T00:32:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.059+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:32:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.059+0000 7faf767fc700 10 client.14763.objecter _finish_command 15 = system:0 2024-02-18T00:32:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.059+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80126d00 osd=7 4 2024-02-18T00:32:14.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.059+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 7 15 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 10 client.14763.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign -1 16 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 20 client.14763.objecter _get_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 15 client.14763.objecter _session_command_op_remove -1 16 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 20 client.14763.objecter get_session s=0x7faf80126d00 osd=7 3 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 15 client.14763.objecter _session_command_op_assign 7 16 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80126d00 osd=7 4 2024-02-18T00:32:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 10 client.14763.objecter _send_command 16 2024-02-18T00:32:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.079+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7faf8012f140 con 0x7faf8012c860 2024-02-18T00:32:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf767fc700 1 -- 172.21.15.59:0/1552172698 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7faf8012f140 con 0x7faf8012c860 2024-02-18T00:32:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf767fc700 10 client.14763.objecter ms_dispatch 0x7faf80001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf767fc700 10 client.14763.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf767fc700 20 client.14763.objecter put_session s=0x7faf80126d00 osd=7 4 2024-02-18T00:32:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf767fc700 15 client.14763.objecter _session_command_op_remove 7 16 2024-02-18T00:32:14.082 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:32:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 librados: watch_flush enter 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 librados: watch_flush exit 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 client.14763.objecter close_session for osd.0 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6801/3700238977 conn(0x7faf800fdea0 msgr2=0x7faf800fe2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6801/3700238977 conn(0x7faf800fdea0 0x7faf800fe2e0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8005ee80 osd=0 1 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 client.14763.objecter close_session for osd.1 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6805/26279110 conn(0x7faf800598f0 msgr2=0x7faf80059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6805/26279110 conn(0x7faf800598f0 0x7faf80059d10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf800286b0 osd=1 1 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 client.14763.objecter close_session for osd.2 2024-02-18T00:32:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6809/1153193637 conn(0x7faf80028210 msgr2=0x7faf8011e3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6809/1153193637 conn(0x7faf80028210 0x7faf8011e3b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80059770 osd=2 1 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 client.14763.objecter close_session for osd.3 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6813/1805700964 conn(0x7faf8011fa20 msgr2=0x7faf8011fe60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6813/1805700964 conn(0x7faf8011fa20 0x7faf8011fe60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf8011b9f0 osd=3 1 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 client.14763.objecter close_session for osd.4 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6800/2044387056 conn(0x7faf801232c0 msgr2=0x7faf801256b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6800/2044387056 conn(0x7faf801232c0 0x7faf801256b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120350 osd=4 1 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 client.14763.objecter close_session for osd.5 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6804/2170432263 conn(0x7faf80127740 msgr2=0x7faf80127ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6804/2170432263 conn(0x7faf80127740 0x7faf80127ba0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80120a90 osd=5 1 2024-02-18T00:32:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 10 client.14763.objecter close_session for osd.6 2024-02-18T00:32:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6808/3356009390 conn(0x7faf80126220 msgr2=0x7faf80126660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.081+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6808/3356009390 conn(0x7faf80126220 0x7faf80126660 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80125ba0 osd=6 1 2024-02-18T00:32:14.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 10 client.14763.objecter close_session for osd.7 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6812/53025171 conn(0x7faf8012c860 msgr2=0x7faf8012ec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.137:6812/53025171 conn(0x7faf8012c860 0x7faf8012ec50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf859af700 1 -- 172.21.15.59:0/1552172698 reap_dead start 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 20 client.14763.objecter put_session s=0x7faf80126d00 osd=7 1 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 20 client.14763.objecter shutdown clearing up homeless session... 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 10 client.14763.objecter successfully canceled tick 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf859af700 1 -- 172.21.15.59:0/1552172698 reap_dead start 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf859af700 1 -- 172.21.15.59:0/1552172698 reap_dead start 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf859af700 1 -- 172.21.15.59:0/1552172698 reap_dead start 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf859af700 1 -- 172.21.15.59:0/1552172698 reap_dead start 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf6c062c90 msgr2=0x7faf6c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf6c062c90 0x7faf6c065150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7faf7c007fd0 tx=0x7faf7c00b040).stop 2024-02-18T00:32:14.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fcbb0 msgr2=0x7faf80104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fcbb0 0x7faf80104840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7faf700043a0 tx=0x7faf7000b040).stop 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 shutdown_connections 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf800fcbb0 0x7faf80104840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 --2- 172.21.15.59:0/1552172698 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf6c062c90 0x7faf6c065150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 >> 172.21.15.59:0/1552172698 conn(0x7faf800f2820 msgr2=0x7faf800f4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 shutdown_connections 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 -- 172.21.15.59:0/1552172698 wait complete. 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:14.082+0000 7faf87412700 1 librados: shutdown 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.088 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.089 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:32:14.244 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:13.859741+0000 mon.a (mon.0) 767 : audit [DBG] from='client.? 172.21.15.59:0/1552172698' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:13.859741+0000 mon.a (mon.0) 767 : audit [DBG] from='client.? 172.21.15.59:0/1552172698' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:13.755261+0000 mgr.y (mgr.24454) 121 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:15.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:13.755261+0000 mgr.y (mgr.24454) 121 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:15.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:13.755261+0000 mgr.y (mgr.24454) 121 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:15.705 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:15.710 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:15.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:15.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:15.852 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:16.010 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:16.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:16.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:16.117 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:16.149 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:16.170 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:16.172 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:16.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:16.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:16.313 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:16.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:32:16.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:15.755996+0000 mgr.y (mgr.24454) 122 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:32:17.073 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:17.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:17.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:17.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 -- 172.21.15.59:0/3095030682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0f7400 msgr2=0x7fbdfc103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 --2- 172.21.15.59:0/3095030682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc103a20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fbdf4009be0 tx=0x7fbdf4004d10).stop 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 -- 172.21.15.59:0/3095030682 shutdown_connections 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 --2- 172.21.15.59:0/3095030682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0fcbb0 0x7fbdfc0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 --2- 172.21.15.59:0/3095030682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc103a20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 --2- 172.21.15.59:0/3095030682 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdfc0fd510 0x7fbdfc0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 -- 172.21.15.59:0/3095030682 >> 172.21.15.59:0/3095030682 conn(0x7fbdfc0f2820 msgr2=0x7fbdfc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 -- 172.21.15.59:0/3095030682 shutdown_connections 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.198+0000 7fbe016cc700 1 -- 172.21.15.59:0/3095030682 wait complete. 2024-02-18T00:32:17.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 librados: starting msgr at 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 librados: starting objecter 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 Processor -- start 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 -- start start 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 librados: setting wanted keys 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 librados: calling monclient init 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0fcbb0 0x7fbdfc0628c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdfc0fd510 0x7fbdfc062db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc063420 con 0x7fbdfc0fcbb0 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc063590 con 0x7fbdfc0f7400 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.199+0000 7fbe016cc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc063750 con 0x7fbdfc0fd510 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc06a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:47136/0 (socket says 172.21.15.59:47136) 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 -- 172.21.15.59:0/2405332458 learned_addr learned my addr 172.21.15.59:0/2405332458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfa7fc700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0fcbb0 0x7fbdfc0628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfb7fe700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdfc0fd510 0x7fbdfc062db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 -- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdfc0fd510 msgr2=0x7fbdfc062db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdfc0fd510 0x7fbdfc062db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 -- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0fcbb0 msgr2=0x7fbdfc0628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0fcbb0 0x7fbdfc0628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 -- 172.21.15.59:0/2405332458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf40097c0 con 0x7fbdfc0f7400 2024-02-18T00:32:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbdfaffd700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc06a7d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbdec002820 tx=0x7fbdec002b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbde3fff700 10 client.?.objecter ms_handle_connect 0x7fbdfc0f7400 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.200+0000 7fbde3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.201+0000 7fbde3fff700 1 -- 172.21.15.59:0/2405332458 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec007b40 con 0x7fbdfc0f7400 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.201+0000 7fbde3fff700 1 -- 172.21.15.59:0/2405332458 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdec00f040 con 0x7fbdfc0f7400 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.201+0000 7fbde3fff700 1 -- 172.21.15.59:0/2405332458 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec014710 con 0x7fbdfc0f7400 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.201+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc109f90 con 0x7fbdfc0f7400 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.201+0000 7fbe016cc700 10 client.24676.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.201+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc10a430 con 0x7fbdfc0f7400 2024-02-18T00:32:17.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.201+0000 7fbe016cc700 1 librados: init done 2024-02-18T00:32:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.203+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdfc05fa60 con 0x7fbdfc0f7400 2024-02-18T00:32:17.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.204+0000 7fbde3fff700 1 -- 172.21.15.59:0/2405332458 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fbdec014b40 con 0x7fbdfc0f7400 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.204+0000 7fbde3fff700 1 --2- 172.21.15.59:0/2405332458 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbde4062c90 0x7fbde4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.204+0000 7fbde3fff700 1 -- 172.21.15.59:0/2405332458 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(98..98 src has 44..98) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fbdec07c7a0 con 0x7fbdfc0f7400 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.204+0000 7fbde3fff700 10 client.24676.objecter ms_dispatch 0x7fbdfc001550 osd_map(98..98 src has 44..98) v4 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.204+0000 7fbde3fff700 3 client.24676.objecter handle_osd_map got epochs [98,98] > 0 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.204+0000 7fbde3fff700 3 client.24676.objecter handle_osd_map decoding full epoch 98 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.204+0000 7fbde3fff700 20 client.24676.objecter dump_active .. 0 homeless 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.205+0000 7fbdfa7fc700 1 --2- 172.21.15.59:0/2405332458 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbde4062c90 0x7fbde4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:17.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.205+0000 7fbde3fff700 10 client.24676.objecter ms_handle_connect 0x7fbde4062c90 2024-02-18T00:32:17.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.205+0000 7fbdfa7fc700 1 --2- 172.21.15.59:0/2405332458 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbde4062c90 0x7fbde4065150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fbdfc064060 tx=0x7fbdf0009060).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.211+0000 7fbde3fff700 1 -- 172.21.15.59:0/2405332458 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbdec050370 con 0x7fbdfc0f7400 2024-02-18T00:32:17.220 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:15.755996+0000 mgr.y (mgr.24454) 122 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:32:17.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:15.755996+0000 mgr.y (mgr.24454) 122 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:32:17.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.415+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"} v 0) v1 -- 0x7fbdfc05ee80 con 0x7fbdfc0f7400 2024-02-18T00:32:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:17.677 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:17.677 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:17.778 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:17.810 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:17.833 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:17.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:17.836 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:17.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:17.977 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:17.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.989+0000 7fbde3fff700 1 -- 172.21.15.59:0/2405332458 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]=0 set pool 2 pgp_num to 32 v99) v1 ==== 136+0+0 (secure 0 0 0) 0x7fbdec04ff60 con 0x7fbdfc0f7400 2024-02-18T00:32:17.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.991+0000 7fbe016cc700 10 librados: watch_flush enter 2024-02-18T00:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.991+0000 7fbe016cc700 10 librados: watch_flush exit 2024-02-18T00:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.991+0000 7fbe016cc700 20 client.24676.objecter shutdown clearing up homeless session... 2024-02-18T00:32:17.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.991+0000 7fbe016cc700 10 client.24676.objecter successfully canceled tick 2024-02-18T00:32:18.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.997+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbde4062c90 msgr2=0x7fbde4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:18.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.997+0000 7fbe016cc700 1 --2- 172.21.15.59:0/2405332458 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbde4062c90 0x7fbde4065150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fbdfc064060 tx=0x7fbdf0009060).stop 2024-02-18T00:32:18.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.997+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0f7400 msgr2=0x7fbdfc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:18.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:17.997+0000 7fbe016cc700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc06a7d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbdec002820 tx=0x7fbdec002b30).stop 2024-02-18T00:32:18.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.004+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 shutdown_connections 2024-02-18T00:32:18.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.004+0000 7fbe016cc700 1 --2- 172.21.15.59:0/2405332458 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbde4062c90 0x7fbde4065150 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:18.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.004+0000 7fbe016cc700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbdfc0f7400 0x7fbdfc06a7d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:18.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.004+0000 7fbe016cc700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbdfc0fd510 0x7fbdfc062db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:18.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.004+0000 7fbe016cc700 1 --2- 172.21.15.59:0/2405332458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdfc0fcbb0 0x7fbdfc0628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:18.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.004+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 >> 172.21.15.59:0/2405332458 conn(0x7fbdfc0f2820 msgr2=0x7fbdfc0fac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:18.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.005+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 shutdown_connections 2024-02-18T00:32:18.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.005+0000 7fbe016cc700 1 -- 172.21.15.59:0/2405332458 wait complete. 2024-02-18T00:32:18.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:18.005+0000 7fbe016cc700 1 librados: shutdown 2024-02-18T00:32:18.009 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 2 pgp_num to 32 2024-02-18T00:32:18.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:17.416555+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.59:0/2405332458' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-18T00:32:18.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:17.417074+0000 mon.a (mon.0) 768 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-18T00:32:18.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:17.416555+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.59:0/2405332458' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-18T00:32:18.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:17.417074+0000 mon.a (mon.0) 768 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-18T00:32:18.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:17.416555+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.59:0/2405332458' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-18T00:32:18.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:17.417074+0000 mon.a (mon.0) 768 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]: dispatch 2024-02-18T00:32:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:17.756350+0000 mgr.y (mgr.24454) 123 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:32:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:17.986668+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]': finished 2024-02-18T00:32:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:17.986747+0000 mon.a (mon.0) 770 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-18T00:32:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:17.756350+0000 mgr.y (mgr.24454) 123 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:32:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:17.986668+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]': finished 2024-02-18T00:32:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:17.986747+0000 mon.a (mon.0) 770 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-18T00:32:19.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:17.756350+0000 mgr.y (mgr.24454) 123 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:32:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:17.986668+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "32"}]': finished 2024-02-18T00:32:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:17.986747+0000 mon.a (mon.0) 770 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-18T00:32:19.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 -- 172.21.15.59:0/61750124 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006b660 msgr2=0x7f204006bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:19.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 --2- 172.21.15.59:0/61750124 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006b660 0x7f204006bae0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f2030004870 tx=0x7f2030009cb0).stop 2024-02-18T00:32:19.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 -- 172.21.15.59:0/61750124 shutdown_connections 2024-02-18T00:32:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 --2- 172.21.15.59:0/61750124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2040108720 0x7f204010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 --2- 172.21.15.59:0/61750124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006cc20 0x7f204006b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 --2- 172.21.15.59:0/61750124 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006b660 0x7f204006bae0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 -- 172.21.15.59:0/61750124 >> 172.21.15.59:0/61750124 conn(0x7f20400f3800 msgr2=0x7f20400f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.264+0000 7f2045223700 1 -- 172.21.15.59:0/61750124 shutdown_connections 2024-02-18T00:32:19.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.265+0000 7f2045223700 1 -- 172.21.15.59:0/61750124 wait complete. 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.265+0000 7f2045223700 1 librados: starting msgr at 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.265+0000 7f2045223700 1 librados: starting objecter 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.265+0000 7f2045223700 1 Processor -- start 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 -- start start 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 librados: setting wanted keys 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 librados: calling monclient init 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006b660 0x7f20401147f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:19.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006cc20 0x7f2040114ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:19.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f203ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006b660 0x7f20401147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:19.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f203ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006b660 0x7f20401147f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41220/0 (socket says 172.21.15.59:41220) 2024-02-18T00:32:19.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2040108720 0x7f204010e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:19.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204010ee80 con 0x7f204006b660 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204010efc0 con 0x7f2040108720 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f2045223700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204010f180 con 0x7f204006cc20 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f203ed9d700 1 -- 172.21.15.59:0/1159974792 learned_addr learned my addr 172.21.15.59:0/1159974792 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.266+0000 7f203e59c700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006cc20 0x7f2040114ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f203ed9d700 1 -- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006cc20 msgr2=0x7f2040114ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f203ed9d700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006cc20 0x7f2040114ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f203ed9d700 1 -- 172.21.15.59:0/1159974792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2040108720 msgr2=0x7f204010e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f203ed9d700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2040108720 0x7f204010e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f203ed9d700 1 -- 172.21.15.59:0/1159974792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2030005040 con 0x7f204006b660 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f203ed9d700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006b660 0x7f20401147f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f203400e970 tx=0x7f203400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f2027fff700 10 client.?.objecter ms_handle_connect 0x7f204006b660 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f2027fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:19.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f2027fff700 1 -- 172.21.15.59:0/1159974792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203400ee60 con 0x7f204006b660 2024-02-18T00:32:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f2027fff700 1 -- 172.21.15.59:0/1159974792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f203400f040 con 0x7f204006b660 2024-02-18T00:32:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.267+0000 7f2027fff700 1 -- 172.21.15.59:0/1159974792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034007ea0 con 0x7f204006b660 2024-02-18T00:32:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.268+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204010f460 con 0x7f204006b660 2024-02-18T00:32:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.268+0000 7f2045223700 10 client.14775.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.268+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204011b5f0 con 0x7f204006b660 2024-02-18T00:32:19.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.268+0000 7f2045223700 1 librados: init done 2024-02-18T00:32:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.269+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204005ee80 con 0x7f204006b660 2024-02-18T00:32:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.270+0000 7f2027fff700 1 -- 172.21.15.59:0/1159974792 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f20340092e0 con 0x7f204006b660 2024-02-18T00:32:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.270+0000 7f2027fff700 1 --2- 172.21.15.59:0/1159974792 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2028062c90 0x7f2028065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:19.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f203e59c700 1 --2- 172.21.15.59:0/1159974792 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2028062c90 0x7f2028065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f2027fff700 1 -- 172.21.15.59:0/1159974792 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(99..99 src has 44..99) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f203407c480 con 0x7f204006b660 2024-02-18T00:32:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f2027fff700 10 client.14775.objecter ms_dispatch 0x7f2040001550 osd_map(99..99 src has 44..99) v4 2024-02-18T00:32:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f2027fff700 3 client.14775.objecter handle_osd_map got epochs [99,99] > 0 2024-02-18T00:32:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f2027fff700 3 client.14775.objecter handle_osd_map decoding full epoch 99 2024-02-18T00:32:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f2027fff700 20 client.14775.objecter dump_active .. 0 homeless 2024-02-18T00:32:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f203e59c700 1 --2- 172.21.15.59:0/1159974792 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2028062c90 0x7f2028065150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2030005010 tx=0x7f203000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:19.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.271+0000 7f2027fff700 10 client.14775.objecter ms_handle_connect 0x7f2028062c90 2024-02-18T00:32:19.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.275+0000 7f2027fff700 1 -- 172.21.15.59:0/1159974792 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2034049f60 con 0x7f204006b660 2024-02-18T00:32:19.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:19.489+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f20400008d0 con 0x7f204006b660 2024-02-18T00:32:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:19.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:19.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:19.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:19.699 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:19.720 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:19.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:19.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:19.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:19.965 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.008+0000 7f2027fff700 1 -- 172.21.15.59:0/1159974792 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2034019090 con 0x7f204006b660 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 10 librados: watch_flush enter 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 10 librados: watch_flush exit 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 20 client.14775.objecter shutdown clearing up homeless session... 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 10 client.14775.objecter successfully canceled tick 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2028062c90 msgr2=0x7f2028065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 1 --2- 172.21.15.59:0/1159974792 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2028062c90 0x7f2028065150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2030005010 tx=0x7f203000b040).stop 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006b660 msgr2=0x7f20401147f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:20.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.011+0000 7f2045223700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006b660 0x7f20401147f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f203400e970 tx=0x7f203400b040).stop 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.013+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 shutdown_connections 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.013+0000 7f2045223700 1 --2- 172.21.15.59:0/1159974792 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2028062c90 0x7f2028065150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.013+0000 7f2045223700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2040108720 0x7f204010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.013+0000 7f2045223700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204006b660 0x7f20401147f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.013+0000 7f2045223700 1 --2- 172.21.15.59:0/1159974792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204006cc20 0x7f2040114ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.013+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 >> 172.21.15.59:0/1159974792 conn(0x7f20400f3800 msgr2=0x7f20400f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.017+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 shutdown_connections 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.017+0000 7f2045223700 1 -- 172.21.15.59:0/1159974792 wait complete. 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:20.017+0000 7f2045223700 1 librados: shutdown 2024-02-18T00:32:20.020 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:32:20.235 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:19.490044+0000 mon.a (mon.0) 771 : audit [INF] from='client.? 172.21.15.59:0/1159974792' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:20.236 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:19.490044+0000 mon.a (mon.0) 771 : audit [INF] from='client.? 172.21.15.59:0/1159974792' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:19.490044+0000 mon.a (mon.0) 771 : audit [INF] from='client.? 172.21.15.59:0/1159974792' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:20.707 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:20.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:20.717 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:20.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:20.866 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:19.360018+0000 osd.3 (osd.3) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:19.360083+0000 osd.3 (osd.3) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:19.756713+0000 mgr.y (mgr.24454) 124 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:19.995316+0000 mon.a (mon.0) 772 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:20.007673+0000 mon.a (mon.0) 773 : audit [INF] from='client.? 172.21.15.59:0/1159974792' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:32:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:20.007759+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-18T00:32:21.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:19.360018+0000 osd.3 (osd.3) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:21.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:19.360083+0000 osd.3 (osd.3) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:21.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:19.756713+0000 mgr.y (mgr.24454) 124 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:21.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:19.995316+0000 mon.a (mon.0) 772 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:21.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:20.007673+0000 mon.a (mon.0) 773 : audit [INF] from='client.? 172.21.15.59:0/1159974792' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:32:21.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:20.007759+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-18T00:32:21.201 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:32:21.201 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:32:21.201 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:32:21.201 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24329.0:64 5.3 5:c609908c:::notify.5:head [watch watch cookie 94665234049200] snapc 0=[] ondisk+write+known_if_redirected e61)", 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:19.111863+0000", 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 182.08460973999999, 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021016160000000002, 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24329", 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1565964696", 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 64 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.111863+0000", 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.111863+0000", 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.111862+0000", 2024-02-18T00:32:21.203 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3430000000000001e-06 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.111868+0000", 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3760999999999999e-05 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.111882+0000", 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7049999999999994e-06 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.204 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.111892+0000", 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00042555000000000002 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.112317+0000", 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3464999999999998e-05 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.112361+0000", 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00069654500000000002 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113057+0000", 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2997000000000006e-05 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113130+0000", 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017445500000000001 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113305+0000", 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041541499999999998 2024-02-18T00:32:21.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113720+0000", 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000136379 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113857+0000", 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7319e-05 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.207 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113894+0000", 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0590999999999999e-05 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113925+0000", 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0161999999999999e-05 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:19.113965+0000", 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021016160000000002 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.208 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:76 4.b 4:d242335b:gc::gc.2:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.163009+0000", 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.03346404199999, 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046791070000000001, 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 76 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163009+0000", 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163009+0000", 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163008+0000", 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6499999999999996e-06 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163015+0000", 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2965e-05 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163028+0000", 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.098e-06 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163037+0000", 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1894000000000002e-05 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.211 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163079+0000", 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0387999999999999e-05 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163130+0000", 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013633100000000001 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,4", 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163266+0000", 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00073022699999999996 2024-02-18T00:32:21.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163996+0000", 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00088861300000000001 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.164885+0000", 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027092050000000001 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167594+0000", 2024-02-18T00:32:21.213 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9511000000000001e-05 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167644+0000", 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4474000000000001e-05 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167688+0000", 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0046791070000000001 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:77 4.0 e64/58 4:05588ce8:lc::lc.6:head v 64'3, mlcod=64'1)", 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.163734+0000", 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.03273876599999, 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0038805979999999999, 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163734+0000", 2024-02-18T00:32:21.215 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163734+0000", 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163733+0000", 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9090000000000006e-06 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163742+0000", 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0709e-05 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163753+0000", 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7050000000000005e-06 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.217 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163762+0000", 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3535999999999995e-05 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163835+0000", 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9714000000000001e-05 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.163855+0000", 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037226030000000001 2024-02-18T00:32:21.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167577+0000", 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7311000000000002e-05 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167615+0000", 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0038805979999999999 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:87 4.b 4:d5e0c857:lc::lc.5:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.165881+0000", 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.030591566, 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001889368, 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 87 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.220 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.165881+0000", 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.165881+0000", 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.165880+0000", 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7959999999999998e-06 2024-02-18T00:32:21.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.165887+0000", 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2153999999999999e-05 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.165899+0000", 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0180000000000004e-06 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.165906+0000", 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4703000000000001e-05 2024-02-18T00:32:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.165961+0000", 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0853999999999996e-05 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.166032+0000", 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000181893 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,4", 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.166214+0000", 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00057692500000000005 2024-02-18T00:32:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.166791+0000", 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000228254 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167019+0000", 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00067865299999999998 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.224 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167698+0000", 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7796999999999997e-05 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167735+0000", 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5332e-05 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.167771+0000", 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001889368 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:128 4.3 4:c3a682d4:reshard::reshard.0000000004:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.179017+0000", 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.01745585699999, 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0036152580000000001, 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 128 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179017+0000", 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179017+0000", 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179016+0000", 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9519999999999994e-06 2024-02-18T00:32:21.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179023+0000", 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1088e-05 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179035+0000", 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7119999999999997e-06 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.228 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179040+0000", 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9910000000000002e-05 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179080+0000", 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000110035 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179190+0000", 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015111000000000001 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,7", 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179341+0000", 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031501329999999998 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182491+0000", 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.345e-06 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182499+0000", 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1028999999999997e-05 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.230 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182530+0000", 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3486999999999997e-05 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182593+0000", 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9008999999999999e-05 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182632+0000", 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0036152580000000001 2024-02-18T00:32:21.231 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop_reply(client.24347.0:128 4.3 e64/58 ondisk, result = 0)", 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.179816+0000", 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.01665650800001, 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026831469999999999, 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.232 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179816+0000", 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179816+0000", 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179815+0000", 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7950000000000002e-06 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179822+0000", 2024-02-18T00:32:21.233 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4443999999999999e-05 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179837+0000", 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8020000000000003e-06 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179843+0000", 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026325889999999998 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182476+0000", 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0839999999999992e-06 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182484+0000", 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5549999999999999e-05 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.235 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182500+0000", 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026831469999999999 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop_reply(client.24347.0:128 4.3 e64/58 ondisk, result = 0)", 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.179873+0000", 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.01659969900001, 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027651300000000002, 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:32:21.236 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179873+0000", 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179873+0000", 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.237 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179873+0000", 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.255e-06 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179880+0000", 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0739999999999993e-06 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179888+0000", 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9529999999999998e-06 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.238 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.179893+0000", 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026251270000000001 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182518+0000", 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.981e-06 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182524+0000", 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011428 2024-02-18T00:32:21.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182638+0000", 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027651300000000002 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:131 4.1e 4:7e82ab18:lc::lc.13:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.182397+0000", 2024-02-18T00:32:21.240 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.01407576700001, 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023504540000000001, 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 131 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182397+0000", 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182397+0000", 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182395+0000", 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.045e-05 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182406+0000", 2024-02-18T00:32:21.242 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2153e-05 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182418+0000", 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8000000000000001e-06 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182425+0000", 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6550000000000004e-05 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.243 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182501+0000", 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012316900000000001 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182624+0000", 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000208968 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182833+0000", 2024-02-18T00:32:21.244 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00065253100000000003 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.183486+0000", 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00088422999999999996 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184370+0000", 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029465999999999997 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.245 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184665+0000", 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5189999999999999e-05 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184710+0000", 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7648999999999998e-05 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184748+0000", 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023504540000000001 2024-02-18T00:32:21.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:157 4.6 4:66924383:reshard::reshard.0000000005:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.187290+0000", 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.009182944, 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022095449999999998, 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.247 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 157 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187290+0000", 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187290+0000", 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:21.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187289+0000", 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9120000000000001e-06 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187296+0000", 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4370000000000004e-06 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187303+0000", 2024-02-18T00:32:21.249 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8520000000000003e-06 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187308+0000", 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020589 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187513+0000", 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4306000000000002e-05 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.250 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187598+0000", 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013956299999999999 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187737+0000", 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001274692 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189012+0000", 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7326999999999999e-05 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.251 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189029+0000", 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000416401 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189446+0000", 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2864000000000002e-05 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189479+0000", 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0962e-05 2024-02-18T00:32:21.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189500+0000", 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022095449999999998 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:167 4.6 4:66924383:reshard::reshard.0000000005:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.190448+0000", 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.00602465, 2024-02-18T00:32:21.253 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019772589999999999, 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 167 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190448+0000", 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190448+0000", 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190448+0000", 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6979999999999997e-06 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190452+0000", 2024-02-18T00:32:21.255 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2490000000000001e-06 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190458+0000", 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5619999999999999e-06 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190461+0000", 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4436000000000004e-05 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.256 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190536+0000", 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2192000000000004e-05 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190598+0000", 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000114808 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190713+0000", 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00086500499999999996 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.257 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.191578+0000", 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00076663600000000001 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192344+0000", 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9572000000000001e-05 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192374+0000", 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1776999999999997e-05 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192406+0000", 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0032000000000001e-05 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192426+0000", 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019772589999999999 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.259 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:168 4.1e 4:7e82ab18:lc::lc.13:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.190471+0000", 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.00600230699999, 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027676979999999999, 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 168 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.260 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190471+0000", 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190471+0000", 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190470+0000", 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2899999999999998e-06 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.261 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190473+0000", 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8939999999999998e-06 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190479+0000", 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3749999999999996e-06 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190484+0000", 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00038715499999999998 2024-02-18T00:32:21.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190871+0000", 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0444000000000001e-05 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.190911+0000", 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5013999999999995e-05 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.191006+0000", 2024-02-18T00:32:21.263 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000576626 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.191583+0000", 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015392050000000001 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193122+0000", 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6903000000000002e-05 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.264 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193159+0000", 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4586999999999998e-05 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193204+0000", 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4674e-05 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193238+0000", 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027676979999999999 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.265 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:175 4.3 4:c60da066:gc::gc.4:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.192183+0000", 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.00428987800001, 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020883949999999998, 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 175 2024-02-18T00:32:21.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192183+0000", 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192183+0000", 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192182+0000", 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3579999999999999e-06 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192189+0000", 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1486e-05 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.268 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192200+0000", 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7999999999999998e-06 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192205+0000", 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000274231 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192479+0000", 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9332e-05 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192538+0000", 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016199000000000001 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,7", 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.192700+0000", 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00066777099999999995 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.270 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193368+0000", 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00079531399999999998 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.194164+0000", 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1065000000000002e-05 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.194195+0000", 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3739999999999998e-05 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.194238+0000", 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.311e-05 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.194271+0000", 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020883949999999998 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:183 4.3 4:c134263f:lc::lc.25:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.193683+0000", 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.00279033300001, 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024560570000000002, 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 183 2024-02-18T00:32:21.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193683+0000", 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193683+0000", 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193682+0000", 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8970000000000001e-06 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193686+0000", 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1190000000000002e-06 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193692+0000", 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4050000000000002e-06 2024-02-18T00:32:21.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.193696+0000", 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00060086799999999995 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.194297+0000", 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5231999999999999e-05 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.276 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.194392+0000", 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016724400000000001 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,7", 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.194560+0000", 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00072719799999999997 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.195287+0000", 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00077488899999999996 2024-02-18T00:32:21.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196062+0000", 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2578999999999998e-05 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196084+0000", 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5656e-05 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.278 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196110+0000", 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8591000000000001e-05 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196139+0000", 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024560570000000002 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:191 4.16 4:6dc3b799:reshard::reshard.0000000006:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.279 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.196518+0000", 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 178.99995514700001, 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020049960000000002, 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 191 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.280 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196518+0000", 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196518+0000", 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196517+0000", 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0139999999999999e-06 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196520+0000", 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0980000000000004e-06 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196524+0000", 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6429999999999999e-06 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196527+0000", 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8941e-05 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.282 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196556+0000", 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5176000000000002e-05 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196611+0000", 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016717699999999999 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.196778+0000", 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00077762399999999998 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.283 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.197556+0000", 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000198946 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.197755+0000", 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068251399999999995 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.198437+0000", 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0040999999999997e-05 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.284 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.198488+0000", 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5237000000000002e-05 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.198523+0000", 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020049960000000002 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:215 4.3 4:c134263f:lc::lc.25:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.200825+0000", 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 178.99564758899999, 2024-02-18T00:32:21.285 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019938220000000001, 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 215 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.200825+0000", 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.286 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.200825+0000", 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.200824+0000", 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.468e-06 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.200830+0000", 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1795e-05 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.287 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.200842+0000", 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6960000000000005e-06 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.200849+0000", 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026514399999999998 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.201114+0000", 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2103999999999999e-05 2024-02-18T00:32:21.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.201187+0000", 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000168923 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,7", 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.201355+0000", 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064393700000000002 2024-02-18T00:32:21.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.201999+0000", 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00071081200000000001 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.202710+0000", 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0531000000000003e-05 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.290 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.202741+0000", 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4107000000000003e-05 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.202785+0000", 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4313000000000002e-05 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.291 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.202819+0000", 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019938220000000001 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:233 4.6 4:64760185:::obj_delete_at_hint.0000000010:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.203924+0000", 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 178.99254915200001, 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00195035, 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.292 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 233 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.203924+0000", 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.293 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.203924+0000", 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.203923+0000", 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8689999999999998e-06 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.203929+0000", 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5056e-05 2024-02-18T00:32:21.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.203945+0000", 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8560000000000001e-06 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.203951+0000", 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4420999999999999e-05 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.295 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.204016+0000", 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010378399999999999 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.204120+0000", 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00021543199999999999 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.204335+0000", 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00081021700000000001 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.205145+0000", 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00039811499999999999 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.205543+0000", 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025842900000000002 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.205802+0000", 2024-02-18T00:32:21.297 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9920000000000003e-05 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.205852+0000", 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2444000000000001e-05 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.205874+0000", 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00195035 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:332 4.1e 4:7e82ab18:lc::lc.13:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.225899+0000", 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 178.97057342700001, 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020490719999999999, 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 332 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225899+0000", 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225899+0000", 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.300 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225899+0000", 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4389999999999999e-06 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225904+0000", 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.13e-05 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.301 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225915+0000", 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9899999999999997e-06 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225920+0000", 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8246999999999999e-05 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.302 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225949+0000", 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6612e-05 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.225995+0000", 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000179555 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:32:21.303 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.226175+0000", 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00039774599999999998 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.226573+0000", 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00065766299999999995 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.227230+0000", 2024-02-18T00:32:21.304 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00067174200000000002 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.227902+0000", 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.673e-05 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.227929+0000", 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9891999999999999e-05 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.305 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.227949+0000", 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020490719999999999 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:543 4.16 4:6b0d4d95:lc::lc.20:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.276942+0000", 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 178.91953094900001, 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018722350000000001, 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.306 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 543 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276942+0000", 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.307 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276942+0000", 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276941+0000", 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5480000000000004e-06 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276948+0000", 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2797e-05 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276961+0000", 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0110000000000003e-06 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276968+0000", 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1654999999999999e-05 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.277020+0000", 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6988000000000001e-05 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.309 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.277107+0000", 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000200776 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.277308+0000", 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00048292499999999999 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.277791+0000", 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000104471 2024-02-18T00:32:21.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.277895+0000", 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00083298400000000005 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278728+0000", 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7092000000000002e-05 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278775+0000", 2024-02-18T00:32:21.311 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9026999999999998e-05 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278814+0000", 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018722350000000001 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:622 4.19 e64/58 4:994d36d1:lc::lc.9:head v 64'20, mlcod=64'18)", 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.301239+0000", 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 178.89523432999999, 2024-02-18T00:32:21.312 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020028720000000002, 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301239+0000", 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301239+0000", 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.313 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301237+0000", 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0359e-05 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301248+0000", 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7651000000000001e-05 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301265+0000", 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7610000000000002e-06 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301272+0000", 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3146000000000002e-05 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301325+0000", 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1353e-05 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.301347+0000", 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001859744 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.315 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.303206+0000", 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5026e-05 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.303241+0000", 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020028720000000002 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.316 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:696 4.16 4:6d4431ed:lc::lc.7:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.326573+0000", 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 178.86989979500001, 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019451379999999999, 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 696 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.317 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326573+0000", 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326573+0000", 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326572+0000", 2024-02-18T00:32:21.318 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0209e-05 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326582+0000", 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3465999999999999e-05 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326595+0000", 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6939999999999998e-06 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.319 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326602+0000", 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1950000000000002e-05 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326654+0000", 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1669999999999996e-05 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326736+0000", 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00018011700000000001 2024-02-18T00:32:21.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.326916+0000", 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00053847300000000003 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.327454+0000", 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2886000000000002e-05 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.327507+0000", 2024-02-18T00:32:21.321 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00092814700000000004 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.328435+0000", 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.689e-05 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.328482+0000", 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5970999999999998e-05 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:21.322 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.328518+0000", 2024-02-18T00:32:21.323 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019451379999999999 2024-02-18T00:32:21.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.323 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:21.323 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:21.323 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:32:21.324 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:32:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:32:21.324 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:32:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:32:21.324 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:19.360018+0000 osd.3 (osd.3) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:21.324 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:19.360083+0000 osd.3 (osd.3) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:21.324 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:19.756713+0000 mgr.y (mgr.24454) 124 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:21.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:19.995316+0000 mon.a (mon.0) 772 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:21.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:20.007673+0000 mon.a (mon.0) 773 : audit [INF] from='client.? 172.21.15.59:0/1159974792' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:32:21.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:20.007759+0000 mon.a (mon.0) 774 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-18T00:32:22.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:20.885365+0000 mgr.y (mgr.24454) 125 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:22.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:20.885365+0000 mgr.y (mgr.24454) 125 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:22.212 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:32:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:22] "GET /metrics HTTP/1.1" 200 172239 "" "Prometheus/2.33.4" 2024-02-18T00:32:22.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-18T00:32:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:20.885365+0000 mgr.y (mgr.24454) 125 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:22.618 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:22.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:22.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:22.723 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:22.755 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:22.776 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:22.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:22.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:22.880 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:22.912 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:22.934 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:22.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:22.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:23.038 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:23.067 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:23.087 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:23.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:23.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:23.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:23.222 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:21.757442+0000 mgr.y (mgr.24454) 126 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:23.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:21.757442+0000 mgr.y (mgr.24454) 126 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:23.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:21.757442+0000 mgr.y (mgr.24454) 126 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:23.579 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:23.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:23.680 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:23.712 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:23.734 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:23.736 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:23.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:23.838 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:23.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:23.892 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:23.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:24.027 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:24.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:24.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:24.119 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:24.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.118+0000 7f7d10f9a700 1 -- 172.21.15.59:0/2393146563 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 msgr2=0x7f7d0c0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.118+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/2393146563 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c0f6d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f7cfc004870 tx=0x7f7cfc009cb0).stop 2024-02-18T00:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 -- 172.21.15.59:0/2393146563 shutdown_connections 2024-02-18T00:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/2393146563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d0c0f7400 0x7f7d0c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/2393146563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d0c0fcbb0 0x7f7d0c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/2393146563 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c0f6d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 -- 172.21.15.59:0/2393146563 >> 172.21.15.59:0/2393146563 conn(0x7f7d0c0f2820 msgr2=0x7f7d0c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 -- 172.21.15.59:0/2393146563 shutdown_connections 2024-02-18T00:32:24.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 -- 172.21.15.59:0/2393146563 wait complete. 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 librados: starting msgr at 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.119+0000 7f7d10f9a700 1 librados: starting objecter 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 Processor -- start 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 -- start start 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 librados: setting wanted keys 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 librados: calling monclient init 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d0c0f7400 0x7f7d0c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d0c0fcbb0 0x7f7d0c0628c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c062db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c063420 con 0x7f7d0c0fcbb0 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c063590 con 0x7f7d0c0f7400 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.120+0000 7f7d10f9a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c063750 con 0x7f7d0c0fd510 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c062db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c062db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56192/0 (socket says 172.21.15.59:56192) 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 -- 172.21.15.59:0/562467742 learned_addr learned my addr 172.21.15.59:0/562467742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d09d9b700 1 --2- 172.21.15.59:0/562467742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d0c0fcbb0 0x7f7d0c0628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d0c0f7400 0x7f7d0c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 -- 172.21.15.59:0/562467742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d0c0f7400 msgr2=0x7f7d0c06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 --2- 172.21.15.59:0/562467742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d0c0f7400 0x7f7d0c06a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 -- 172.21.15.59:0/562467742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d0c0fcbb0 msgr2=0x7f7d0c0628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 --2- 172.21.15.59:0/562467742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d0c0fcbb0 0x7f7d0c0628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 -- 172.21.15.59:0/562467742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc005040 con 0x7f7d0c0fd510 2024-02-18T00:32:24.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7d0ad9d700 1 --2- 172.21.15.59:0/562467742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c062db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7d0402f980 tx=0x7f7d04004dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7cfb7fe700 10 client.?.objecter ms_handle_connect 0x7f7d0c0fd510 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.121+0000 7f7cfb7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.122+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0402fe70 con 0x7f7d0c0fd510 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.122+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d0402dc30 con 0x7f7d0c0fd510 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.122+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04003870 con 0x7f7d0c0fd510 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.122+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c10a3c0 con 0x7f7d0c0fd510 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.122+0000 7f7d10f9a700 10 client.24683.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.122+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c10a800 con 0x7f7d0c0fd510 2024-02-18T00:32:24.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.122+0000 7f7d10f9a700 1 librados: init done 2024-02-18T00:32:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.123+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f7d0c05fa60 con 0x7f7d0c0fd510 2024-02-18T00:32:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.124+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f7d04003ca0 con 0x7f7d0c0fd510 2024-02-18T00:32:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.125+0000 7f7cfb7fe700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cf4062ce0 0x7f7cf40651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.125+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(100..100 src has 44..100) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f7d0409e7c0 con 0x7f7d0c0fd510 2024-02-18T00:32:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.125+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 osd_map(100..100 src has 44..100) v4 2024-02-18T00:32:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.125+0000 7f7cfb7fe700 3 client.24683.objecter handle_osd_map got epochs [100,100] > 0 2024-02-18T00:32:24.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.125+0000 7f7cfb7fe700 3 client.24683.objecter handle_osd_map decoding full epoch 100 2024-02-18T00:32:24.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.125+0000 7f7cfb7fe700 20 client.24683.objecter dump_active .. 0 homeless 2024-02-18T00:32:24.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.125+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cf4062ce0 0x7f7cf40651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.126+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7cf4062ce0 2024-02-18T00:32:24.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.126+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cf4062ce0 0x7f7cf40651a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7d000092f0 tx=0x7f7d00009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.126+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v100) v1 ==== 54+0+16 (secure 0 0 0) 0x7f7d040a5070 con 0x7f7d0c0fd510 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 1 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7d0c0fdec0 0x7f7d0c0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 1 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 0 1 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c05ee80 osd=0 4 2024-02-18T00:32:24.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 10 client.24683.objecter _send_command 1 2024-02-18T00:32:24.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.127+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c105010 con 0x7f7d0c0fdec0 2024-02-18T00:32:24.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.129+0000 7f7d09d9b700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7d0c0fdec0 0x7f7d0c0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.129+0000 7f7d09d9b700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7d0c0fdec0 0x7f7d0c0fe2e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.129+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c0fdec0 2024-02-18T00:32:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.130+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d0c105010 con 0x7f7d0c0fdec0 2024-02-18T00:32:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.130+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:32:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.130+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 1 = system:0 2024-02-18T00:32:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.130+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c05ee80 osd=0 4 2024-02-18T00:32:24.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.130+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 0 1 2024-02-18T00:32:24.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 2 2024-02-18T00:32:24.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 2 2024-02-18T00:32:24.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c05ee80 osd=0 3 2024-02-18T00:32:24.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 0 2 2024-02-18T00:32:24.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c05ee80 osd=0 4 2024-02-18T00:32:24.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 10 client.24683.objecter _send_command 2 2024-02-18T00:32:24.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.160+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c0273c0 con 0x7f7d0c0fdec0 2024-02-18T00:32:24.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c0273c0 con 0x7f7d0c0fdec0 2024-02-18T00:32:24.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c05ee80 osd=0 4 2024-02-18T00:32:24.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 0 2 2024-02-18T00:32:24.165 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:32:24.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6805/26279110 conn(0x7f7d0c0598f0 0x7f7d0c059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 1 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c028710 osd=1 4 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 10 client.24683.objecter _send_command 3 2024-02-18T00:32:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.163+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c111260 con 0x7f7d0c0598f0 2024-02-18T00:32:24.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.166+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6805/26279110 conn(0x7f7d0c0598f0 0x7f7d0c059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.166+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6805/26279110 conn(0x7f7d0c0598f0 0x7f7d0c059d10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.166+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c0598f0 2024-02-18T00:32:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.167+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d0c111260 con 0x7f7d0c0598f0 2024-02-18T00:32:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.167+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:32:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.167+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 3 = system:0 2024-02-18T00:32:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.167+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c028710 osd=1 4 2024-02-18T00:32:24.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.167+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 1 3 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 4 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 4 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c028710 osd=1 3 2024-02-18T00:32:24.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 1 4 2024-02-18T00:32:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c028710 osd=1 4 2024-02-18T00:32:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 10 client.24683.objecter _send_command 4 2024-02-18T00:32:24.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.196+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c059360 con 0x7f7d0c0598f0 2024-02-18T00:32:24.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c059360 con 0x7f7d0c0598f0 2024-02-18T00:32:24.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c028710 osd=1 4 2024-02-18T00:32:24.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 1 4 2024-02-18T00:32:24.201 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 5 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7d0c028270 0x7f7d0c10fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 5 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 2 5 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c059770 osd=2 4 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 10 client.24683.objecter _send_command 5 2024-02-18T00:32:24.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c110170 con 0x7f7d0c028270 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.199+0000 7f7d09d9b700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7d0c028270 0x7f7d0c10fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.200+0000 7f7d09d9b700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7d0c028270 0x7f7d0c10fc80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.200+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c028270 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.201+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d0c110170 con 0x7f7d0c028270 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.201+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.201+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 5 = system:0 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.201+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c059770 osd=2 4 2024-02-18T00:32:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.201+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 2 5 2024-02-18T00:32:24.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:24.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 6 2024-02-18T00:32:24.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 6 2024-02-18T00:32:24.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c059770 osd=2 3 2024-02-18T00:32:24.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 2 6 2024-02-18T00:32:24.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c059770 osd=2 4 2024-02-18T00:32:24.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 10 client.24683.objecter _send_command 6 2024-02-18T00:32:24.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.230+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c0277d0 con 0x7f7d0c028270 2024-02-18T00:32:24.235 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c0277d0 con 0x7f7d0c028270 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c059770 osd=2 4 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 2 6 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 7 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7d0c113490 0x7f7d0c1158c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 7 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 3 7 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c027be0 osd=3 4 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 10 client.24683.objecter _send_command 7 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c111070 con 0x7f7d0c113490 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.233+0000 7f7d0ad9d700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7d0c113490 0x7f7d0c1158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.234+0000 7f7d0ad9d700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7d0c113490 0x7f7d0c1158c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.234+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c113490 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.235+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d0c111070 con 0x7f7d0c113490 2024-02-18T00:32:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.235+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:32:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.235+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 7 = system:0 2024-02-18T00:32:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.235+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c027be0 osd=3 4 2024-02-18T00:32:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.235+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 3 7 2024-02-18T00:32:24.255 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 8 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 8 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c027be0 osd=3 3 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 3 8 2024-02-18T00:32:24.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c027be0 osd=3 4 2024-02-18T00:32:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 10 client.24683.objecter _send_command 8 2024-02-18T00:32:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.265+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c1105d0 con 0x7f7d0c113490 2024-02-18T00:32:24.276 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:32:24.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c1105d0 con 0x7f7d0c113490 2024-02-18T00:32:24.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c027be0 osd=3 4 2024-02-18T00:32:24.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 3 8 2024-02-18T00:32:24.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 9 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7d0c10f7e0 0x7f7d0c11afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 9 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 4 9 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110390 osd=4 4 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 10 client.24683.objecter _send_command 9 2024-02-18T00:32:24.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.272+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c110530 con 0x7f7d0c10f7e0 2024-02-18T00:32:24.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.275+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7d0c10f7e0 0x7f7d0c11afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.275+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c10f7e0 2024-02-18T00:32:24.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.275+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7d0c10f7e0 0x7f7d0c11afe0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.278+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d0c110530 con 0x7f7d0c10f7e0 2024-02-18T00:32:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.278+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:32:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.278+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 9 = system:0 2024-02-18T00:32:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.278+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c110390 osd=4 4 2024-02-18T00:32:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.278+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 4 9 2024-02-18T00:32:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 10 2024-02-18T00:32:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 10 2024-02-18T00:32:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c110390 osd=4 3 2024-02-18T00:32:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 4 10 2024-02-18T00:32:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110390 osd=4 4 2024-02-18T00:32:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 10 client.24683.objecter _send_command 10 2024-02-18T00:32:24.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.308+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c110530 con 0x7f7d0c10f7e0 2024-02-18T00:32:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.310+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c110530 con 0x7f7d0c10f7e0 2024-02-18T00:32:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.310+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.310+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.310+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c110390 osd=4 4 2024-02-18T00:32:24.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.310+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 4 10 2024-02-18T00:32:24.313 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:32:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 11 2024-02-18T00:32:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7d0c11ba40 0x7f7d0c11be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 11 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 5 11 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110530 osd=5 4 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 10 client.24683.objecter _send_command 11 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c122d70 con 0x7f7d0c11ba40 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.311+0000 7f7d09d9b700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7d0c11ba40 0x7f7d0c11be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.312+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c11ba40 2024-02-18T00:32:24.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.312+0000 7f7d09d9b700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7d0c11ba40 0x7f7d0c11be60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.312+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d0c122d70 con 0x7f7d0c11ba40 2024-02-18T00:32:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.312+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:32:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.312+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 11 = system:0 2024-02-18T00:32:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.312+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c110530 osd=5 4 2024-02-18T00:32:24.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.312+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 5 11 2024-02-18T00:32:24.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 12 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 12 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c110530 osd=5 3 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 5 12 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110530 osd=5 4 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 10 client.24683.objecter _send_command 12 2024-02-18T00:32:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.342+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c120610 con 0x7f7d0c11ba40 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c120610 con 0x7f7d0c11ba40 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c110530 osd=5 4 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 5 12 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 13 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7d0c120cf0 0x7f7d0c121150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 13 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 6 13 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c120610 osd=6 4 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 10 client.24683.objecter _send_command 13 2024-02-18T00:32:24.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.344+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c122d50 con 0x7f7d0c120cf0 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.345+0000 7f7d0ad9d700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7d0c120cf0 0x7f7d0c121150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.345+0000 7f7d0ad9d700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7d0c120cf0 0x7f7d0c121150 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.345+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c120cf0 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.346+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d0c122d50 con 0x7f7d0c120cf0 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.346+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.346+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 13 = system:0 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.346+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c120610 osd=6 4 2024-02-18T00:32:24.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.346+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 6 13 2024-02-18T00:32:24.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 14 2024-02-18T00:32:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 14 2024-02-18T00:32:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c120610 osd=6 3 2024-02-18T00:32:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 6 14 2024-02-18T00:32:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c120610 osd=6 4 2024-02-18T00:32:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 10 client.24683.objecter _send_command 14 2024-02-18T00:32:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.375+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c122d50 con 0x7f7d0c120cf0 2024-02-18T00:32:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.377+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c122d50 con 0x7f7d0c120cf0 2024-02-18T00:32:24.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.377+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.377+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.377+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c120610 osd=6 4 2024-02-18T00:32:24.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.377+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 6 14 2024-02-18T00:32:24.380 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 15 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6812/53025171 conn(0x7f7d0c1254d0 0x7f7d0c125910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 15 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 7 15 2024-02-18T00:32:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c122d50 osd=7 4 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 10 client.24683.objecter _send_command 15 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d0c125f80 con 0x7f7d0c1254d0 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6812/53025171 conn(0x7f7d0c1254d0 0x7f7d0c125910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7cfb7fe700 10 client.24683.objecter ms_handle_connect 0x7f7d0c1254d0 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.378+0000 7f7d0a59c700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6812/53025171 conn(0x7f7d0c1254d0 0x7f7d0c125910 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.379+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7d00009880 con 0x7f7d0c1254d0 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.379+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.379+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 15 = system:0 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.379+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c122d50 osd=7 4 2024-02-18T00:32:24.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.379+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 7 15 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 10 client.24683.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign -1 16 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 20 client.24683.objecter _get_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_remove -1 16 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 20 client.24683.objecter get_session s=0x7f7d0c122d50 osd=7 3 2024-02-18T00:32:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 15 client.24683.objecter _session_command_op_assign 7 16 2024-02-18T00:32:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c122d50 osd=7 4 2024-02-18T00:32:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 10 client.24683.objecter _send_command 16 2024-02-18T00:32:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.408+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7d0c128140 con 0x7f7d0c1254d0 2024-02-18T00:32:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.410+0000 7f7cfb7fe700 1 -- 172.21.15.59:0/562467742 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7d0c128140 con 0x7f7d0c1254d0 2024-02-18T00:32:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.410+0000 7f7cfb7fe700 10 client.24683.objecter ms_dispatch 0x7f7d0c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.410+0000 7f7cfb7fe700 10 client.24683.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.410+0000 7f7cfb7fe700 20 client.24683.objecter put_session s=0x7f7d0c122d50 osd=7 4 2024-02-18T00:32:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.410+0000 7f7cfb7fe700 15 client.24683.objecter _session_command_op_remove 7 16 2024-02-18T00:32:24.413 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:32:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 librados: watch_flush enter 2024-02-18T00:32:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 librados: watch_flush exit 2024-02-18T00:32:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.0 2024-02-18T00:32:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7d0c0fdec0 msgr2=0x7f7d0c0fe2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7d0c0fdec0 0x7f7d0c0fe2e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c05ee80 osd=0 1 2024-02-18T00:32:24.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.1 2024-02-18T00:32:24.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6805/26279110 conn(0x7f7d0c0598f0 msgr2=0x7f7d0c059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6805/26279110 conn(0x7f7d0c0598f0 0x7f7d0c059d10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c028710 osd=1 1 2024-02-18T00:32:24.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.2 2024-02-18T00:32:24.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7d0c028270 msgr2=0x7f7d0c10fc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7d0c028270 0x7f7d0c10fc80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c059770 osd=2 1 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.3 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7d0c113490 msgr2=0x7f7d0c1158c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7d0c113490 0x7f7d0c1158c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d0ad9d700 1 -- 172.21.15.59:0/562467742 reap_dead start 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c027be0 osd=3 1 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.4 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7d0c10f7e0 msgr2=0x7f7d0c11afe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7d0c10f7e0 0x7f7d0c11afe0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110390 osd=4 1 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.5 2024-02-18T00:32:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7d0c11ba40 msgr2=0x7f7d0c11be60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7d0c11ba40 0x7f7d0c11be60 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c110530 osd=5 1 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.6 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7d0c120cf0 msgr2=0x7f7d0c121150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7d0c120cf0 0x7f7d0c121150 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c120610 osd=6 1 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter close_session for osd.7 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6812/53025171 conn(0x7f7d0c1254d0 msgr2=0x7f7d0c125910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.137:6812/53025171 conn(0x7f7d0c1254d0 0x7f7d0c125910 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter put_session s=0x7f7d0c122d50 osd=7 1 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 20 client.24683.objecter shutdown clearing up homeless session... 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d10f9a700 10 client.24683.objecter successfully canceled tick 2024-02-18T00:32:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.411+0000 7f7d0ad9d700 1 -- 172.21.15.59:0/562467742 reap_dead start 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cf4062ce0 msgr2=0x7f7cf40651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cf4062ce0 0x7f7cf40651a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f7d000092f0 tx=0x7f7d00009cb0).stop 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 msgr2=0x7f7d0c062db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c062db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f7d0402f980 tx=0x7f7d04004dd0).stop 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 shutdown_connections 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d0c0fd510 0x7f7d0c062db0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 --2- 172.21.15.59:0/562467742 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cf4062ce0 0x7f7cf40651a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 >> 172.21.15.59:0/562467742 conn(0x7f7d0c0f2820 msgr2=0x7f7d0c0f43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 shutdown_connections 2024-02-18T00:32:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 -- 172.21.15.59:0/562467742 wait complete. 2024-02-18T00:32:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:24.412+0000 7f7d10f9a700 1 librados: shutdown 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:32:24.465 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:24.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:24.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:24.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:24.609 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:24.630 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:32:24.996 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:32:24.997 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:32:24.997 INFO:tasks.thrashosds.thrasher:Growing pool .rgw.root 2024-02-18T00:32:24.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:32:25.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:23.758058+0000 mgr.y (mgr.24454) 127 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:25.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:24.127211+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? 172.21.15.59:0/562467742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:25.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:23.758058+0000 mgr.y (mgr.24454) 127 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:25.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:24.127211+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? 172.21.15.59:0/562467742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:23.758058+0000 mgr.y (mgr.24454) 127 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:32:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:24.127211+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? 172.21.15.59:0/562467742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:25.809 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:25.812 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:25.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:25.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:25.946 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:25.966 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:25.969 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:25.970 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:26.071 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:26.101 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:26.121 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:26.124 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:26.124 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:26.225 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:26.274 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:26.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:26.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:26.377 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:26.405 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:26.424 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:26.426 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:26.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:26.528 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:26.554 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:26.573 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:26.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:26.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:26.678 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:26.704 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:26.724 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:26.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:26.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:26.828 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:26.854 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:26.873 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:26.876 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:26.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:26.977 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:27.020 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:27.056 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:27.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:27.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:27.162 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:27.186 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:25.758601+0000 mgr.y (mgr.24454) 128 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:27.188 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:27.209 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:27.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:27.311 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:27.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:25.758601+0000 mgr.y (mgr.24454) 128 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:27.339 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:25.758601+0000 mgr.y (mgr.24454) 128 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:27.340 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:29.021 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:32:29.021 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:32:29.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:32:29.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:27.758971+0000 mgr.y (mgr.24454) 129 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:32:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:27.758971+0000 mgr.y (mgr.24454) 129 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:32:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:27.758971+0000 mgr.y (mgr.24454) 129 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:32:31.087 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:31.090 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:31.096 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:31.106 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:31.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:31.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-18T00:32:31.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:31.241 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:31.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:32:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:32:31.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:32:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:32:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:29.759323+0000 mgr.y (mgr.24454) 130 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:31.424 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:29.759323+0000 mgr.y (mgr.24454) 130 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:31.424 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:29.759323+0000 mgr.y (mgr.24454) 130 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:31.450 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:31.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:31.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:31.557 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:31.589 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:31.610 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:31.613 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:31.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:31.714 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:31.752 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:32.291 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:30.896246+0000 mgr.y (mgr.24454) 131 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:32.291 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:32:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:32] "GET /metrics HTTP/1.1" 200 172245 "" "Prometheus/2.33.4" 2024-02-18T00:32:32.292 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:30.896246+0000 mgr.y (mgr.24454) 131 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:30.896246+0000 mgr.y (mgr.24454) 131 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:32.595 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:32.606 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:32.615 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:32.616 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:32.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:32.770 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:32.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:32.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:32.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:32.912 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:33.020 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:33.040 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:33.049 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:33.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:33.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:33.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:31.760057+0000 mgr.y (mgr.24454) 132 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:32:33.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:31.760057+0000 mgr.y (mgr.24454) 132 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:32:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:33.384 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:33.388 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:33.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:33.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:31.760057+0000 mgr.y (mgr.24454) 132 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:32:33.521 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:33.673 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:33.676 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:33.676 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:33.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:33.808 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:34.265 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:34.268 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:34.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:34.372 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:34.689 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:34.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:34.693 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:34.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:34.822 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:35.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:35.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:35.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:35.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:33.760439+0000 mgr.y (mgr.24454) 133 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:35.316 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:35.338 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:35.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:35.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:35.426 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:33.760439+0000 mgr.y (mgr.24454) 133 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:35.427 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:33.760439+0000 mgr.y (mgr.24454) 133 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:32:35.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:35.471 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:35.491 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:35.494 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:35.494 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:35.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:35.661 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:35.668 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:35.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:35.769 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:35.822 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:35.825 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:35.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.772+0000 7f2d33e81700 1 -- 172.21.15.59:0/494570609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0f7400 msgr2=0x7f2d2c0fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.772+0000 7f2d33e81700 1 --2- 172.21.15.59:0/494570609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c0fbaf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2d280047b0 tx=0x7f2d28009cb0).stop 2024-02-18T00:32:35.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.772+0000 7f2d33e81700 1 -- 172.21.15.59:0/494570609 shutdown_connections 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.772+0000 7f2d33e81700 1 --2- 172.21.15.59:0/494570609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d2c0fbfe0 0x7f2d2c0fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.772+0000 7f2d33e81700 1 --2- 172.21.15.59:0/494570609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f6aa0 0x7f2d2c0f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.772+0000 7f2d33e81700 1 --2- 172.21.15.59:0/494570609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c0fbaf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.772+0000 7f2d33e81700 1 -- 172.21.15.59:0/494570609 >> 172.21.15.59:0/494570609 conn(0x7f2d2c0f2820 msgr2=0x7f2d2c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.773+0000 7f2d33e81700 1 -- 172.21.15.59:0/494570609 shutdown_connections 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.773+0000 7f2d33e81700 1 -- 172.21.15.59:0/494570609 wait complete. 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.773+0000 7f2d33e81700 1 librados: starting msgr at 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.773+0000 7f2d33e81700 1 librados: starting objecter 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.773+0000 7f2d33e81700 1 Processor -- start 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.774+0000 7f2d33e81700 1 -- start start 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.774+0000 7f2d33e81700 1 librados: setting wanted keys 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.774+0000 7f2d33e81700 1 librados: calling monclient init 2024-02-18T00:32:35.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.774+0000 7f2d33e81700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d2c0f6aa0 0x7f2d2c114890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.775+0000 7f2d33e81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.775+0000 7f2d3141c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.775+0000 7f2d3141c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c114d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58522/0 (socket says 172.21.15.59:58522) 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d33e81700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0fbfe0 0x7f2d2c10ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d33e81700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c10ef20 con 0x7f2d2c0f7400 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d33e81700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c10f060 con 0x7f2d2c0fbfe0 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d33e81700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c10f220 con 0x7f2d2c0f6aa0 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d3141c700 1 -- 172.21.15.59:0/3443367181 learned_addr learned my addr 172.21.15.59:0/3443367181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d3241e700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0fbfe0 0x7f2d2c10ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d3141c700 1 -- 172.21.15.59:0/3443367181 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d2c0f6aa0 msgr2=0x7f2d2c114890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d3141c700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d2c0f6aa0 0x7f2d2c114890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d3141c700 1 -- 172.21.15.59:0/3443367181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0fbfe0 msgr2=0x7f2d2c10ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d3141c700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0fbfe0 0x7f2d2c10ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.777+0000 7f2d3141c700 1 -- 172.21.15.59:0/3443367181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28005040 con 0x7f2d2c0f7400 2024-02-18T00:32:35.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d3141c700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c114d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f2d280044e0 tx=0x7f2d28007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d1effd700 10 client.?.objecter ms_handle_connect 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d1effd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d1effd700 1 -- 172.21.15.59:0/3443367181 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d28018070 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d1effd700 1 -- 172.21.15.59:0/3443367181 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d2801d420 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d1effd700 1 -- 172.21.15.59:0/3443367181 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2801c650 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c10f4a0 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d33e81700 10 client.14787.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c11b690 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.778+0000 7f2d33e81700 1 librados: init done 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.779+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c10f8e0 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.780+0000 7f2d1effd700 1 -- 172.21.15.59:0/3443367181 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2d28013ad0 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.781+0000 7f2d1effd700 1 --2- 172.21.15.59:0/3443367181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2d18062c90 0x7f2d18065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.781+0000 7f2d1effd700 1 -- 172.21.15.59:0/3443367181 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 44..100) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f2d2807def0 con 0x7f2d2c0f7400 2024-02-18T00:32:35.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.781+0000 7f2d1effd700 10 client.14787.objecter ms_dispatch 0x7f2d2c001550 osd_map(100..100 src has 44..100) v4 2024-02-18T00:32:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.781+0000 7f2d1effd700 3 client.14787.objecter handle_osd_map got epochs [100,100] > 0 2024-02-18T00:32:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.781+0000 7f2d1effd700 3 client.14787.objecter handle_osd_map decoding full epoch 100 2024-02-18T00:32:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.781+0000 7f2d1effd700 20 client.14787.objecter dump_active .. 0 homeless 2024-02-18T00:32:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.785+0000 7f2d1effd700 1 -- 172.21.15.59:0/3443367181 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2d28051aa0 con 0x7f2d2c0f7400 2024-02-18T00:32:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.786+0000 7f2d31c1d700 1 --2- 172.21.15.59:0/3443367181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2d18062c90 0x7f2d18065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.786+0000 7f2d1effd700 10 client.14787.objecter ms_handle_connect 0x7f2d18062c90 2024-02-18T00:32:35.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:35.786+0000 7f2d31c1d700 1 --2- 172.21.15.59:0/3443367181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2d18062c90 0x7f2d18065150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f2d20004c60 tx=0x7f2d2000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:35.926 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:35.966 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:36.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.020+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f2d2c10ff10 con 0x7f2d2c0f7400 2024-02-18T00:32:36.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:36.021547+0000 mon.a (mon.0) 775 : audit [INF] from='client.? 172.21.15.59:0/3443367181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:32:36.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:36.021547+0000 mon.a (mon.0) 775 : audit [INF] from='client.? 172.21.15.59:0/3443367181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:32:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.062+0000 7f2d1effd700 1 -- 172.21.15.59:0/3443367181 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v101) v1 ==== 91+0+0 (secure 0 0 0) 0x7f2d28051690 con 0x7f2d2c0f7400 2024-02-18T00:32:36.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.065+0000 7f2d33e81700 10 librados: watch_flush enter 2024-02-18T00:32:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 10 librados: watch_flush exit 2024-02-18T00:32:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 20 client.14787.objecter shutdown clearing up homeless session... 2024-02-18T00:32:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 10 client.14787.objecter successfully canceled tick 2024-02-18T00:32:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2d18062c90 msgr2=0x7f2d18065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 --2- 172.21.15.59:0/3443367181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2d18062c90 0x7f2d18065150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f2d20004c60 tx=0x7f2d2000b040).stop 2024-02-18T00:32:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f7400 msgr2=0x7f2d2c114d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c114d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f2d280044e0 tx=0x7f2d28007f90).stop 2024-02-18T00:32:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 shutdown_connections 2024-02-18T00:32:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 --2- 172.21.15.59:0/3443367181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2d18062c90 0x7f2d18065150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d2c0fbfe0 0x7f2d2c10ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d2c0f6aa0 0x7f2d2c114890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 --2- 172.21.15.59:0/3443367181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d2c0f7400 0x7f2d2c114d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:36.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.067+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 >> 172.21.15.59:0/3443367181 conn(0x7f2d2c0f2820 msgr2=0x7f2d2c0f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:36.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.069+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 shutdown_connections 2024-02-18T00:32:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.069+0000 7f2d33e81700 1 -- 172.21.15.59:0/3443367181 wait complete. 2024-02-18T00:32:36.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:36.070+0000 7f2d33e81700 1 librados: shutdown 2024-02-18T00:32:36.074 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:32:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:36.021547+0000 mon.a (mon.0) 775 : audit [INF] from='client.? 172.21.15.59:0/3443367181' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:32:36.822 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:36.829 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:36.830 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:36.930 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:36.971 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:37.083 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:35.761192+0000 mgr.y (mgr.24454) 134 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:37.083 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.047101+0000 mon.a (mon.0) 776 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:37.083 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:36.060203+0000 mon.a (mon.0) 777 : audit [INF] from='client.? 172.21.15.59:0/3443367181' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:32:37.083 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.060275+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-18T00:32:37.083 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.520140+0000 osd.2 (osd.2) 53 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:32:37.083 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.521623+0000 osd.2 (osd.2) 54 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:32:37.084 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:35.761192+0000 mgr.y (mgr.24454) 134 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:37.084 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.047101+0000 mon.a (mon.0) 776 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:37.084 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:36.060203+0000 mon.a (mon.0) 777 : audit [INF] from='client.? 172.21.15.59:0/3443367181' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:32:37.084 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.060275+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-18T00:32:37.084 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.520140+0000 osd.2 (osd.2) 53 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:32:37.084 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.521623+0000 osd.2 (osd.2) 54 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:32:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:35.761192+0000 mgr.y (mgr.24454) 134 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:32:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.047101+0000 mon.a (mon.0) 776 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:32:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:36.060203+0000 mon.a (mon.0) 777 : audit [INF] from='client.? 172.21.15.59:0/3443367181' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:32:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.060275+0000 mon.a (mon.0) 778 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-18T00:32:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.520140+0000 osd.2 (osd.2) 53 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:32:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.521623+0000 osd.2 (osd.2) 54 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:32:37.927 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:32:37.928 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.718463+0000 osd.7 (osd.7) 57 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.719479+0000 osd.7 (osd.7) 58 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.731180+0000 osd.3 (osd.3) 77 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.732276+0000 osd.3 (osd.3) 78 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.945016+0000 osd.5 (osd.5) 59 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.948472+0000 osd.5 (osd.5) 60 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.953225+0000 osd.4 (osd.4) 65 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:32:38.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:36.954036+0000 osd.4 (osd.4) 66 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.718463+0000 osd.7 (osd.7) 57 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.719479+0000 osd.7 (osd.7) 58 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.731180+0000 osd.3 (osd.3) 77 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.732276+0000 osd.3 (osd.3) 78 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.945016+0000 osd.5 (osd.5) 59 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.948472+0000 osd.5 (osd.5) 60 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.953225+0000 osd.4 (osd.4) 65 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:32:38.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:36.954036+0000 osd.4 (osd.4) 66 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:32:38.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.422+0000 7fa7fb77d700 1 -- 172.21.15.59:0/1443108897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 msgr2=0x7fa7f4104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:38.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.422+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/1443108897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f4104170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa7e4004870 tx=0x7fa7e4009cb0).stop 2024-02-18T00:32:38.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.422+0000 7fa7fb77d700 1 -- 172.21.15.59:0/1443108897 shutdown_connections 2024-02-18T00:32:38.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.422+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/1443108897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f4104660 0x7fa7f4106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.422+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/1443108897 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f40f6b00 0x7fa7f40f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.422+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/1443108897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f4104170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.422+0000 7fa7fb77d700 1 -- 172.21.15.59:0/1443108897 >> 172.21.15.59:0/1443108897 conn(0x7fa7f40f2820 msgr2=0x7fa7f40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:38.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 -- 172.21.15.59:0/1443108897 shutdown_connections 2024-02-18T00:32:38.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 -- 172.21.15.59:0/1443108897 wait complete. 2024-02-18T00:32:38.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 librados: starting msgr at 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 librados: starting objecter 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 Processor -- start 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 -- start start 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 librados: setting wanted keys 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.423+0000 7fa7fb77d700 1 librados: calling monclient init 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7fb77d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f40f6b00 0x7fa7f41103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7fb77d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f41108c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7fb77d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f4104660 0x7fa7f410a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7fb77d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f410aa60 con 0x7fa7f40f73d0 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7fb77d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f410abd0 con 0x7fa7f4104660 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7fb77d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f410ad90 con 0x7fa7f40f6b00 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f41108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:38.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f41108c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58542/0 (socket says 172.21.15.59:58542) 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 -- 172.21.15.59:0/3321238275 learned_addr learned my addr 172.21.15.59:0/3321238275 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f9519700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f40f6b00 0x7fa7f41103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f9d1a700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f4104660 0x7fa7f410a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 -- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f40f6b00 msgr2=0x7fa7f41103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f40f6b00 0x7fa7f41103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 -- 172.21.15.59:0/3321238275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f4104660 msgr2=0x7fa7f410a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f4104660 0x7fa7f410a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 -- 172.21.15.59:0/3321238275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e4005040 con 0x7fa7f40f73d0 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7f8d18700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f41108c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fa7e4009eb0 tx=0x7fa7e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7ea7fc700 10 client.?.objecter ms_handle_connect 0x7fa7f40f73d0 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.424+0000 7fa7ea7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.425+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/3321238275 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400e070 con 0x7fa7f40f73d0 2024-02-18T00:32:38.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.425+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f410b010 con 0x7fa7f40f73d0 2024-02-18T00:32:38.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.425+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/3321238275 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7e4013da0 con 0x7fa7f40f73d0 2024-02-18T00:32:38.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.425+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/3321238275 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4019070 con 0x7fa7f40f73d0 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.425+0000 7fa7fb77d700 10 client.14793.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.425+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f4117200 con 0x7fa7f40f73d0 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.425+0000 7fa7fb77d700 1 librados: init done 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.426+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f405fa60 con 0x7fa7f40f73d0 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.427+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/3321238275 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa7e401e420 con 0x7fa7f40f73d0 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.427+0000 7fa7ea7fc700 1 --2- 172.21.15.59:0/3321238275 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.427+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/3321238275 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(101..101 src has 44..101) v4 ==== 6172+0+0 (secure 0 0 0) 0x7fa7e407d5a0 con 0x7fa7f40f73d0 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.427+0000 7fa7ea7fc700 10 client.14793.objecter ms_dispatch 0x7fa7f4001550 osd_map(101..101 src has 44..101) v4 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.427+0000 7fa7ea7fc700 3 client.14793.objecter handle_osd_map got epochs [101,101] > 0 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.427+0000 7fa7ea7fc700 3 client.14793.objecter handle_osd_map decoding full epoch 101 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.427+0000 7fa7ea7fc700 20 client.14793.objecter dump_active .. 0 homeless 2024-02-18T00:32:38.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.428+0000 7fa7f9519700 1 --2- 172.21.15.59:0/3321238275 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.428+0000 7fa7ea7fc700 10 client.14793.objecter ms_handle_connect 0x7fa7e0062c90 2024-02-18T00:32:38.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.428+0000 7fa7f9519700 1 --2- 172.21.15.59:0/3321238275 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa7f00048b0 tx=0x7fa7f000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:38.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.433+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/3321238275 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa7e4051280 con 0x7fa7f40f73d0 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.718463+0000 osd.7 (osd.7) 57 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.719479+0000 osd.7 (osd.7) 58 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.731180+0000 osd.3 (osd.3) 77 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.732276+0000 osd.3 (osd.3) 78 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.945016+0000 osd.5 (osd.5) 59 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.948472+0000 osd.5 (osd.5) 60 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.953225+0000 osd.4 (osd.4) 65 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:32:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:36.954036+0000 osd.4 (osd.4) 66 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:32:38.524 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:38.529 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:38.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:38.630 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:38.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.647+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7f405ef20 con 0x7fa7e0062c90 2024-02-18T00:32:38.662 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:32:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.659+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/3321238275 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357042 (secure 0 0 0) 0x7fa7f405ef20 con 0x7fa7e0062c90 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.664+0000 7fa7fb77d700 10 librados: watch_flush enter 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.664+0000 7fa7fb77d700 10 librados: watch_flush exit 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.664+0000 7fa7fb77d700 20 client.14793.objecter shutdown clearing up homeless session... 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.664+0000 7fa7fb77d700 10 client.14793.objecter successfully canceled tick 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 msgr2=0x7fa7e0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/3321238275 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fa7f00048b0 tx=0x7fa7f000b040).stop 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 msgr2=0x7fa7f41108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f41108c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fa7e4009eb0 tx=0x7fa7e400b040).stop 2024-02-18T00:32:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 shutdown_connections 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/3321238275 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f4104660 0x7fa7f410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f40f6b00 0x7fa7f41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 --2- 172.21.15.59:0/3321238275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f73d0 0x7fa7f41108c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 >> 172.21.15.59:0/3321238275 conn(0x7fa7f40f2820 msgr2=0x7fa7f40f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 shutdown_connections 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 -- 172.21.15.59:0/3321238275 wait complete. 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:38.665+0000 7fa7fb77d700 1 librados: shutdown 2024-02-18T00:32:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:32:38.877 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:38.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:38.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:38.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-18T00:32:38.982 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:39.036 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:39.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:39.039 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:39.140 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:39.169 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:39.188 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:39.191 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:39.192 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:39.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:37.226600+0000 osd.6 (osd.6) 51 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:32:39.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:37.227530+0000 osd.6 (osd.6) 52 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:32:39.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:37.761567+0000 mgr.y (mgr.24454) 135 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:39.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:37.811944+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:32:39.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:37.815403+0000 osd.1 (osd.1) 68 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:32:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:37.226600+0000 osd.6 (osd.6) 51 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:32:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:37.227530+0000 osd.6 (osd.6) 52 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:32:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:37.761567+0000 mgr.y (mgr.24454) 135 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:37.811944+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:32:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:37.815403+0000 osd.1 (osd.1) 68 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:32:39.292 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:39.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:37.226600+0000 osd.6 (osd.6) 51 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:32:39.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:37.227530+0000 osd.6 (osd.6) 52 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:32:39.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:37.761567+0000 mgr.y (mgr.24454) 135 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:39.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:37.811944+0000 osd.1 (osd.1) 67 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:32:39.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:37.815403+0000 osd.1 (osd.1) 68 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:32:39.319 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:39.338 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:39.340 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:39.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:39.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:39.475 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:39.499 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:39.512 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:39.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:39.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:39.642 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:39.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:32:39.850 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:39.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:39.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:39.896 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-02-18T00:32:37.761352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1296,"num_read_kb":1016,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":684,"ondisk_log_size":684,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386492,"kb_used_data":5628,"kb_used_omap":0,"kb_used_meta":2380800,"kb_avail":747574724,"statfs":{"total":767960285184,"available":765516517376,"internally_reserved":0,"allocated":5763072,"data_stored":2722872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2437939200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002599"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070015+0000","last_change":"2024-02-18T00:30:46.913439+0000","last_active":"2024-02-18T00:32:18.070015+0000","last_peered":"2024-02-18T00:32:18.070015+0000","last_clean":"2024-02-18T00:32:18.070015+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:32:18.070015+0000","last_undegraded":"2024-02-18T00:32:18.070015+0000","last_fullsized":"2024-02-18T00:32:18.070015+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.913340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.913340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065052+0000","last_change":"2024-02-18T00:31:07.942520+0000","last_active":"2024-02-18T00:32:18.065052+0000","last_peered":"2024-02-18T00:32:18.065052+0000","last_clean":"2024-02-18T00:32:18.065052+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:32:18.065052+0000","last_undegraded":"2024-02-18T00:32:18.065052+0000","last_fullsized":"2024-02-18T00:32:18.065052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.942435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.942435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.988325+0000","last_change":"2024-02-18T00:31:02.922852+0000","last_active":"2024-02-18T00:32:17.988325+0000","last_peered":"2024-02-18T00:32:17.988325+0000","last_clean":"2024-02-18T00:32:17.988325+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:32:17.988325+0000","last_undegraded":"2024-02-18T00:32:17.988325+0000","last_fullsized":"2024-02-18T00:32:17.988325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:02.922771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:02.922771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":140,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070658+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:32:18.070658+0000","last_peered":"2024-02-18T00:32:18.070658+0000","last_clean":"2024-02-18T00:32:18.070658+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:32:18.070658+0000","last_undegraded":"2024-02-18T00:32:18.070658+0000","last_fullsized":"2024-02-18T00:32:18.070658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065420+0000","last_change":"2024-02-18T00:30:29.061462+0000","last_active":"2024-02-18T00:32:18.065420+0000","last_peered":"2024-02-18T00:32:18.065420+0000","last_clean":"2024-02-18T00:32:18.065420+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:32:18.065420+0000","last_undegraded":"2024-02-18T00:32:18.065420+0000","last_fullsized":"2024-02-18T00:32:18.065420+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:29.061390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:29.061390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.068767+0000","last_change":"2024-02-18T00:30:50.991652+0000","last_active":"2024-02-18T00:32:18.068767+0000","last_peered":"2024-02-18T00:32:18.068767+0000","last_clean":"2024-02-18T00:32:18.068767+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:32:18.068767+0000","last_undegraded":"2024-02-18T00:32:18.068767+0000","last_fullsized":"2024-02-18T00:32:18.068767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.991592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.991592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070402+0000","last_change":"2024-02-18T00:30:48.890743+0000","last_active":"2024-02-18T00:32:18.070402+0000","last_peered":"2024-02-18T00:32:18.070402+0000","last_clean":"2024-02-18T00:32:18.070402+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:32:18.070402+0000","last_undegraded":"2024-02-18T00:32:18.070402+0000","last_fullsized":"2024-02-18T00:32:18.070402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.890677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":86,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065287+0000","last_change":"2024-02-18T00:31:03.941281+0000","last_active":"2024-02-18T00:32:18.065287+0000","last_peered":"2024-02-18T00:32:18.065287+0000","last_clean":"2024-02-18T00:32:18.065287+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:32:18.065287+0000","last_undegraded":"2024-02-18T00:32:18.065287+0000","last_fullsized":"2024-02-18T00:32:18.065287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:03.941195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:03.941195+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065115+0000","last_change":"2024-02-18T00:31:44.843600+0000","last_active":"2024-02-18T00:32:18.065115+0000","last_peered":"2024-02-18T00:32:18.065115+0000","last_clean":"2024-02-18T00:32:18.065115+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:32:18.065115+0000","last_undegraded":"2024-02-18T00:32:18.065115+0000","last_fullsized":"2024-02-18T00:32:18.065115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:31:44.843494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.985924+0000","last_change":"2024-02-18T00:31:41.607649+0000","last_active":"2024-02-18T00:32:17.985924+0000","last_peered":"2024-02-18T00:32:17.985924+0000","last_clean":"2024-02-18T00:32:17.985924+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:32:17.985924+0000","last_undegraded":"2024-02-18T00:32:17.985924+0000","last_fullsized":"2024-02-18T00:32:17.985924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.607567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986444+0000","last_change":"2024-02-18T00:31:36.502977+0000","last_active":"2024-02-18T00:32:17.986444+0000","last_peered":"2024-02-18T00:32:17.986444+0000","last_clean":"2024-02-18T00:32:17.986444+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:32:17.986444+0000","last_undegraded":"2024-02-18T00:32:17.986444+0000","last_fullsized":"2024-02-18T00:32:17.986444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.502900+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":88,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.075921+0000","last_change":"2024-02-18T00:32:02.063451+0000","last_active":"2024-02-18T00:32:18.075921+0000","last_peered":"2024-02-18T00:32:18.075921+0000","last_clean":"2024-02-18T00:32:18.075921+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:32:18.075921+0000","last_undegraded":"2024-02-18T00:32:18.075921+0000","last_fullsized":"2024-02-18T00:32:18.075921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:32:02.063346+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.063346+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986937+0000","last_change":"2024-02-18T00:31:36.140716+0000","last_active":"2024-02-18T00:32:17.986937+0000","last_peered":"2024-02-18T00:32:17.986937+0000","last_clean":"2024-02-18T00:32:17.986937+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:32:17.986937+0000","last_undegraded":"2024-02-18T00:32:17.986937+0000","last_fullsized":"2024-02-18T00:32:17.986937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.140630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.140630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065607+0000","last_change":"2024-02-18T00:31:58.856050+0000","last_active":"2024-02-18T00:32:18.065607+0000","last_peered":"2024-02-18T00:32:18.065607+0000","last_clean":"2024-02-18T00:32:18.065607+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:32:18.065607+0000","last_undegraded":"2024-02-18T00:32:18.065607+0000","last_fullsized":"2024-02-18T00:32:18.065607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.855979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.855979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986252+0000","last_change":"2024-02-18T00:30:26.654619+0000","last_active":"2024-02-18T00:32:17.986252+0000","last_peered":"2024-02-18T00:32:17.986252+0000","last_clean":"2024-02-18T00:32:17.986252+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:32:17.986252+0000","last_undegraded":"2024-02-18T00:32:17.986252+0000","last_fullsized":"2024-02-18T00:32:17.986252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:26.654541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:26.654541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"64'22","reported_seq":107,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065433+0000","last_change":"2024-02-18T00:31:40.094511+0000","last_active":"2024-02-18T00:32:18.065433+0000","last_peered":"2024-02-18T00:32:18.065433+0000","last_clean":"2024-02-18T00:32:18.065433+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:32:18.065433+0000","last_undegraded":"2024-02-18T00:32:18.065433+0000","last_fullsized":"2024-02-18T00:32:18.065433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:31:40.094414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.094414+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.076516+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:32:18.076516+0000","last_peered":"2024-02-18T00:32:18.076516+0000","last_clean":"2024-02-18T00:32:18.076516+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:32:18.076516+0000","last_undegraded":"2024-02-18T00:32:18.076516+0000","last_fullsized":"2024-02-18T00:32:18.076516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066135+0000","last_change":"2024-02-18T00:30:33.047434+0000","last_active":"2024-02-18T00:32:18.066135+0000","last_peered":"2024-02-18T00:32:18.066135+0000","last_clean":"2024-02-18T00:32:18.066135+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:32:18.066135+0000","last_undegraded":"2024-02-18T00:32:18.066135+0000","last_fullsized":"2024-02-18T00:32:18.066135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:33.047327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:33.047327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":69,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070240+0000","last_change":"2024-02-18T00:31:32.210269+0000","last_active":"2024-02-18T00:32:18.070240+0000","last_peered":"2024-02-18T00:32:18.070240+0000","last_clean":"2024-02-18T00:32:18.070240+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:32:18.070240+0000","last_undegraded":"2024-02-18T00:32:18.070240+0000","last_fullsized":"2024-02-18T00:32:18.070240+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:31:32.210198+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.210198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":93,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.072111+0000","last_change":"2024-02-18T00:30:50.885765+0000","last_active":"2024-02-18T00:32:18.072111+0000","last_peered":"2024-02-18T00:32:18.072111+0000","last_clean":"2024-02-18T00:32:18.072111+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:32:18.072111+0000","last_undegraded":"2024-02-18T00:32:18.072111+0000","last_fullsized":"2024-02-18T00:32:18.072111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:30:50.885651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.885651+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.075923+0000","last_change":"2024-02-18T00:30:36.043298+0000","last_active":"2024-02-18T00:32:18.075923+0000","last_peered":"2024-02-18T00:32:18.075923+0000","last_clean":"2024-02-18T00:32:18.075923+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:32:18.075923+0000","last_undegraded":"2024-02-18T00:32:18.075923+0000","last_fullsized":"2024-02-18T00:32:18.075923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:36.043211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:36.043211+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066740+0000","last_change":"2024-02-18T00:32:06.883082+0000","last_active":"2024-02-18T00:32:18.066740+0000","last_peered":"2024-02-18T00:32:18.066740+0000","last_clean":"2024-02-18T00:32:18.066740+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:32:18.066740+0000","last_undegraded":"2024-02-18T00:32:18.066740+0000","last_fullsized":"2024-02-18T00:32:18.066740+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:06.883019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:32:06.883019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067370+0000","last_change":"2024-02-18T00:31:54.138392+0000","last_active":"2024-02-18T00:32:18.067370+0000","last_peered":"2024-02-18T00:32:18.067370+0000","last_clean":"2024-02-18T00:32:18.067370+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:32:18.067370+0000","last_undegraded":"2024-02-18T00:32:18.067370+0000","last_fullsized":"2024-02-18T00:32:18.067370+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:54.138311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.138311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987649+0000","last_change":"2024-02-18T00:31:40.592166+0000","last_active":"2024-02-18T00:32:17.987649+0000","last_peered":"2024-02-18T00:32:17.987649+0000","last_clean":"2024-02-18T00:32:17.987649+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:32:17.987649+0000","last_undegraded":"2024-02-18T00:32:17.987649+0000","last_fullsized":"2024-02-18T00:32:17.987649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:40.592083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.592083+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066513+0000","last_change":"2024-02-18T00:31:37.936961+0000","last_active":"2024-02-18T00:32:18.066513+0000","last_peered":"2024-02-18T00:32:18.066513+0000","last_clean":"2024-02-18T00:32:18.066513+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:32:18.066513+0000","last_undegraded":"2024-02-18T00:32:18.066513+0000","last_fullsized":"2024-02-18T00:32:18.066513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.936896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:30.321330+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.936896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":52,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.988134+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:32:17.988134+0000","last_peered":"2024-02-18T00:32:17.988134+0000","last_clean":"2024-02-18T00:32:17.988134+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:32:17.988134+0000","last_undegraded":"2024-02-18T00:32:17.988134+0000","last_fullsized":"2024-02-18T00:32:17.988134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070706+0000","last_change":"2024-02-18T00:31:49.512647+0000","last_active":"2024-02-18T00:32:18.070706+0000","last_peered":"2024-02-18T00:32:18.070706+0000","last_clean":"2024-02-18T00:32:18.070706+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:32:18.070706+0000","last_undegraded":"2024-02-18T00:32:18.070706+0000","last_fullsized":"2024-02-18T00:32:18.070706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:31:49.512572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":94,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067678+0000","last_change":"2024-02-18T00:31:56.165995+0000","last_active":"2024-02-18T00:32:18.067678+0000","last_peered":"2024-02-18T00:32:18.067678+0000","last_clean":"2024-02-18T00:32:18.067678+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:32:18.067678+0000","last_undegraded":"2024-02-18T00:32:18.067678+0000","last_fullsized":"2024-02-18T00:32:18.067678+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:56.165920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.165920+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986449+0000","last_change":"2024-02-18T00:31:45.654910+0000","last_active":"2024-02-18T00:32:17.986449+0000","last_peered":"2024-02-18T00:32:17.986449+0000","last_clean":"2024-02-18T00:32:17.986449+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:32:17.986449+0000","last_undegraded":"2024-02-18T00:32:17.986449+0000","last_fullsized":"2024-02-18T00:32:17.986449+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:31:45.654827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:45.654827+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":109,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.296330+0000","last_change":"2024-02-18T00:31:30.356896+0000","last_active":"2024-02-18T00:32:28.296330+0000","last_peered":"2024-02-18T00:32:28.296330+0000","last_clean":"2024-02-18T00:32:28.296330+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:32:28.296330+0000","last_undegraded":"2024-02-18T00:32:28.296330+0000","last_fullsized":"2024-02-18T00:32:28.296330+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:30.356811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.356811+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065067+0000","last_change":"2024-02-18T00:31:06.040501+0000","last_active":"2024-02-18T00:32:18.065067+0000","last_peered":"2024-02-18T00:32:18.065067+0000","last_clean":"2024-02-18T00:32:18.065067+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:32:18.065067+0000","last_undegraded":"2024-02-18T00:32:18.065067+0000","last_fullsized":"2024-02-18T00:32:18.065067+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.040425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.040425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":89,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.988095+0000","last_change":"2024-02-18T00:31:55.016021+0000","last_active":"2024-02-18T00:32:17.988095+0000","last_peered":"2024-02-18T00:32:17.988095+0000","last_clean":"2024-02-18T00:32:17.988095+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:32:17.988095+0000","last_undegraded":"2024-02-18T00:32:17.988095+0000","last_fullsized":"2024-02-18T00:32:17.988095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:55.015959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.015959+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070466+0000","last_change":"2024-02-18T00:30:59.122281+0000","last_active":"2024-02-18T00:32:18.070466+0000","last_peered":"2024-02-18T00:32:18.070466+0000","last_clean":"2024-02-18T00:32:18.070466+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:32:18.070466+0000","last_undegraded":"2024-02-18T00:32:18.070466+0000","last_fullsized":"2024-02-18T00:32:18.070466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.122216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.122216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986044+0000","last_change":"2024-02-18T00:32:07.760043+0000","last_active":"2024-02-18T00:32:17.986044+0000","last_peered":"2024-02-18T00:32:17.986044+0000","last_clean":"2024-02-18T00:32:17.986044+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:32:17.986044+0000","last_undegraded":"2024-02-18T00:32:17.986044+0000","last_fullsized":"2024-02-18T00:32:17.986044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.759964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.759964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067773+0000","last_change":"2024-02-18T00:31:51.186460+0000","last_active":"2024-02-18T00:32:18.067773+0000","last_peered":"2024-02-18T00:32:18.067773+0000","last_clean":"2024-02-18T00:32:18.067773+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:32:18.067773+0000","last_undegraded":"2024-02-18T00:32:18.067773+0000","last_fullsized":"2024-02-18T00:32:18.067773+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:51.186373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:51.186373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":89,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070862+0000","last_change":"2024-02-18T00:30:43.819520+0000","last_active":"2024-02-18T00:32:18.070862+0000","last_peered":"2024-02-18T00:32:18.070862+0000","last_clean":"2024-02-18T00:32:18.070862+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:32:18.070862+0000","last_undegraded":"2024-02-18T00:32:18.070862+0000","last_fullsized":"2024-02-18T00:32:18.070862+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:30:43.819434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.819434+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.985846+0000","last_change":"2024-02-18T00:30:35.533752+0000","last_active":"2024-02-18T00:32:17.985846+0000","last_peered":"2024-02-18T00:32:17.985846+0000","last_clean":"2024-02-18T00:32:17.985846+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:32:17.985846+0000","last_undegraded":"2024-02-18T00:32:17.985846+0000","last_fullsized":"2024-02-18T00:32:17.985846+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:35.533630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:35.533630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067060+0000","last_change":"2024-02-18T00:32:07.258321+0000","last_active":"2024-02-18T00:32:18.067060+0000","last_peered":"2024-02-18T00:32:18.067060+0000","last_clean":"2024-02-18T00:32:18.067060+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:32:18.067060+0000","last_undegraded":"2024-02-18T00:32:18.067060+0000","last_fullsized":"2024-02-18T00:32:18.067060+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.258239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.258239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066408+0000","last_change":"2024-02-18T00:31:30.789715+0000","last_active":"2024-02-18T00:32:18.066408+0000","last_peered":"2024-02-18T00:32:18.066408+0000","last_clean":"2024-02-18T00:32:18.066408+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:32:18.066408+0000","last_undegraded":"2024-02-18T00:32:18.066408+0000","last_fullsized":"2024-02-18T00:32:18.066408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.789634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070046+0000","last_change":"2024-02-18T00:31:00.083411+0000","last_active":"2024-02-18T00:32:18.070046+0000","last_peered":"2024-02-18T00:32:18.070046+0000","last_clean":"2024-02-18T00:32:18.070046+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:32:18.070046+0000","last_undegraded":"2024-02-18T00:32:18.070046+0000","last_fullsized":"2024-02-18T00:32:18.070046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:31:00.083332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.083332+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070585+0000","last_change":"2024-02-18T00:31:06.048967+0000","last_active":"2024-02-18T00:32:18.070585+0000","last_peered":"2024-02-18T00:32:18.070585+0000","last_clean":"2024-02-18T00:32:18.070585+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:32:18.070585+0000","last_undegraded":"2024-02-18T00:32:18.070585+0000","last_fullsized":"2024-02-18T00:32:18.070585+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:06.048889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:06.048889+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066598+0000","last_change":"2024-02-18T00:30:45.987723+0000","last_active":"2024-02-18T00:32:18.066598+0000","last_peered":"2024-02-18T00:32:18.066598+0000","last_clean":"2024-02-18T00:32:18.066598+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:32:18.066598+0000","last_undegraded":"2024-02-18T00:32:18.066598+0000","last_fullsized":"2024-02-18T00:32:18.066598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.987660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.987660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066593+0000","last_change":"2024-02-18T00:30:54.008260+0000","last_active":"2024-02-18T00:32:18.066593+0000","last_peered":"2024-02-18T00:32:18.066593+0000","last_clean":"2024-02-18T00:32:18.066593+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:32:18.066593+0000","last_undegraded":"2024-02-18T00:32:18.066593+0000","last_fullsized":"2024-02-18T00:32:18.066593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.008196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.008196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066133+0000","last_change":"2024-02-18T00:31:54.074682+0000","last_active":"2024-02-18T00:32:18.066133+0000","last_peered":"2024-02-18T00:32:18.066133+0000","last_clean":"2024-02-18T00:32:18.066133+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:32:18.066133+0000","last_undegraded":"2024-02-18T00:32:18.066133+0000","last_fullsized":"2024-02-18T00:32:18.066133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:31:54.074589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.074589+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986491+0000","last_change":"2024-02-18T00:31:13.960170+0000","last_active":"2024-02-18T00:32:17.986491+0000","last_peered":"2024-02-18T00:32:17.986491+0000","last_clean":"2024-02-18T00:32:17.986491+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:32:17.986491+0000","last_undegraded":"2024-02-18T00:32:17.986491+0000","last_fullsized":"2024-02-18T00:32:17.986491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:13.960089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:13.960089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070314+0000","last_change":"2024-02-18T00:31:57.501367+0000","last_active":"2024-02-18T00:32:18.070314+0000","last_peered":"2024-02-18T00:32:18.070314+0000","last_clean":"2024-02-18T00:32:18.070314+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:32:18.070314+0000","last_undegraded":"2024-02-18T00:32:18.070314+0000","last_fullsized":"2024-02-18T00:32:18.070314+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.501298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.501298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067259+0000","last_change":"2024-02-18T00:31:38.266692+0000","last_active":"2024-02-18T00:32:18.067259+0000","last_peered":"2024-02-18T00:32:18.067259+0000","last_clean":"2024-02-18T00:32:18.067259+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:32:18.067259+0000","last_undegraded":"2024-02-18T00:32:18.067259+0000","last_fullsized":"2024-02-18T00:32:18.067259+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.266577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.266577+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":71,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066521+0000","last_change":"2024-02-18T00:30:51.994431+0000","last_active":"2024-02-18T00:32:18.066521+0000","last_peered":"2024-02-18T00:32:18.066521+0000","last_clean":"2024-02-18T00:32:18.066521+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:32:18.066521+0000","last_undegraded":"2024-02-18T00:32:18.066521+0000","last_fullsized":"2024-02-18T00:32:18.066521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:30:51.994334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.994334+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.076033+0000","last_change":"2024-02-18T00:31:12.044114+0000","last_active":"2024-02-18T00:32:18.076033+0000","last_peered":"2024-02-18T00:32:18.076033+0000","last_clean":"2024-02-18T00:32:18.076033+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:32:18.076033+0000","last_undegraded":"2024-02-18T00:32:18.076033+0000","last_fullsized":"2024-02-18T00:32:18.076033+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:12.043994+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":106,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.296217+0000","last_change":"2024-02-18T00:32:01.547864+0000","last_active":"2024-02-18T00:32:28.296217+0000","last_peered":"2024-02-18T00:32:28.296217+0000","last_clean":"2024-02-18T00:32:28.296217+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:32:28.296217+0000","last_undegraded":"2024-02-18T00:32:28.296217+0000","last_fullsized":"2024-02-18T00:32:28.296217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:01.547763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:01.547763+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":61,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.990126+0000","last_change":"2024-02-18T00:31:41.112608+0000","last_active":"2024-02-18T00:32:17.990126+0000","last_peered":"2024-02-18T00:32:17.990126+0000","last_clean":"2024-02-18T00:32:17.990126+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:32:17.990126+0000","last_undegraded":"2024-02-18T00:32:17.990126+0000","last_fullsized":"2024-02-18T00:32:17.990126+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.112540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.112540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":86,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066482+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:32:18.066482+0000","last_peered":"2024-02-18T00:32:18.066482+0000","last_clean":"2024-02-18T00:32:18.066482+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:32:18.066482+0000","last_undegraded":"2024-02-18T00:32:18.066482+0000","last_fullsized":"2024-02-18T00:32:18.066482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987901+0000","last_change":"2024-02-18T00:31:37.150285+0000","last_active":"2024-02-18T00:32:17.987901+0000","last_peered":"2024-02-18T00:32:17.987901+0000","last_clean":"2024-02-18T00:32:17.987901+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:32:17.987901+0000","last_undegraded":"2024-02-18T00:32:17.987901+0000","last_fullsized":"2024-02-18T00:32:17.987901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.150215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":101,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.296330+0000","last_change":"2024-02-18T00:31:01.012802+0000","last_active":"2024-02-18T00:32:28.296330+0000","last_peered":"2024-02-18T00:32:28.296330+0000","last_clean":"2024-02-18T00:32:28.296330+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:32:28.296330+0000","last_undegraded":"2024-02-18T00:32:28.296330+0000","last_fullsized":"2024-02-18T00:32:28.296330+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:01.012688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.012688+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":66,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065396+0000","last_change":"2024-02-18T00:31:05.071051+0000","last_active":"2024-02-18T00:32:18.065396+0000","last_peered":"2024-02-18T00:32:18.065396+0000","last_clean":"2024-02-18T00:32:18.065396+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:32:18.065396+0000","last_undegraded":"2024-02-18T00:32:18.065396+0000","last_fullsized":"2024-02-18T00:32:18.065396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:05.070930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.070930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":105,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.071073+0000","last_change":"2024-02-18T00:32:02.547561+0000","last_active":"2024-02-18T00:32:18.071073+0000","last_peered":"2024-02-18T00:32:18.071073+0000","last_clean":"2024-02-18T00:32:18.071073+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:32:18.071073+0000","last_undegraded":"2024-02-18T00:32:18.071073+0000","last_fullsized":"2024-02-18T00:32:18.071073+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:32:02.547447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.547447+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986291+0000","last_change":"2024-02-18T00:32:06.740021+0000","last_active":"2024-02-18T00:32:17.986291+0000","last_peered":"2024-02-18T00:32:17.986291+0000","last_clean":"2024-02-18T00:32:17.986291+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:32:17.986291+0000","last_undegraded":"2024-02-18T00:32:17.986291+0000","last_fullsized":"2024-02-18T00:32:17.986291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:06.739943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:06.739943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066374+0000","last_change":"2024-02-18T00:31:23.018547+0000","last_active":"2024-02-18T00:32:18.066374+0000","last_peered":"2024-02-18T00:32:18.066374+0000","last_clean":"2024-02-18T00:32:18.066374+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:32:18.066374+0000","last_undegraded":"2024-02-18T00:32:18.066374+0000","last_fullsized":"2024-02-18T00:32:18.066374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.018460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.018460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067207+0000","last_change":"2024-02-18T00:31:46.199231+0000","last_active":"2024-02-18T00:32:18.067207+0000","last_peered":"2024-02-18T00:32:18.067207+0000","last_clean":"2024-02-18T00:32:18.067207+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:32:18.067207+0000","last_undegraded":"2024-02-18T00:32:18.067207+0000","last_fullsized":"2024-02-18T00:32:18.067207+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.199153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.199153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":103,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.076465+0000","last_change":"2024-02-18T00:31:29.228915+0000","last_active":"2024-02-18T00:32:18.076465+0000","last_peered":"2024-02-18T00:32:18.076465+0000","last_clean":"2024-02-18T00:32:18.076465+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:32:18.076465+0000","last_undegraded":"2024-02-18T00:32:18.076465+0000","last_fullsized":"2024-02-18T00:32:18.076465+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:29.228799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:29.228799+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987322+0000","last_change":"2024-02-18T00:30:46.308529+0000","last_active":"2024-02-18T00:32:17.987322+0000","last_peered":"2024-02-18T00:32:17.987322+0000","last_clean":"2024-02-18T00:32:17.987322+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:32:17.987322+0000","last_undegraded":"2024-02-18T00:32:17.987322+0000","last_fullsized":"2024-02-18T00:32:17.987322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.308454+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.076558+0000","last_change":"2024-02-18T00:31:59.580602+0000","last_active":"2024-02-18T00:32:18.076558+0000","last_peered":"2024-02-18T00:32:18.076558+0000","last_clean":"2024-02-18T00:32:18.076558+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:32:18.076558+0000","last_undegraded":"2024-02-18T00:32:18.076558+0000","last_fullsized":"2024-02-18T00:32:18.076558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:59.580519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:59.580519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986708+0000","last_change":"2024-02-18T00:32:05.449395+0000","last_active":"2024-02-18T00:32:17.986708+0000","last_peered":"2024-02-18T00:32:17.986708+0000","last_clean":"2024-02-18T00:32:17.986708+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:32:17.986708+0000","last_undegraded":"2024-02-18T00:32:17.986708+0000","last_fullsized":"2024-02-18T00:32:17.986708+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.449286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.449286+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":100,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987987+0000","last_change":"2024-02-18T00:31:18.065868+0000","last_active":"2024-02-18T00:32:17.987987+0000","last_peered":"2024-02-18T00:32:17.987987+0000","last_clean":"2024-02-18T00:32:17.987987+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:32:17.987987+0000","last_undegraded":"2024-02-18T00:32:17.987987+0000","last_fullsized":"2024-02-18T00:32:17.987987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:31:18.065789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.065789+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070389+0000","last_change":"2024-02-18T00:31:52.444880+0000","last_active":"2024-02-18T00:32:18.070389+0000","last_peered":"2024-02-18T00:32:18.070389+0000","last_clean":"2024-02-18T00:32:18.070389+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:32:18.070389+0000","last_undegraded":"2024-02-18T00:32:18.070389+0000","last_fullsized":"2024-02-18T00:32:18.070389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:52.444811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:52.444811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":75,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.075882+0000","last_change":"2024-02-18T00:31:52.069257+0000","last_active":"2024-02-18T00:32:18.075882+0000","last_peered":"2024-02-18T00:32:18.075882+0000","last_clean":"2024-02-18T00:32:18.075882+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:32:18.075882+0000","last_undegraded":"2024-02-18T00:32:18.075882+0000","last_fullsized":"2024-02-18T00:32:18.075882+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:31:52.069184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:52.069184+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":108,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065606+0000","last_change":"2024-02-18T00:30:54.996645+0000","last_active":"2024-02-18T00:32:18.065606+0000","last_peered":"2024-02-18T00:32:18.065606+0000","last_clean":"2024-02-18T00:32:18.065606+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:32:18.065606+0000","last_undegraded":"2024-02-18T00:32:18.065606+0000","last_fullsized":"2024-02-18T00:32:18.065606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:30:54.996545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.996545+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986612+0000","last_change":"2024-02-18T00:30:43.221614+0000","last_active":"2024-02-18T00:32:17.986612+0000","last_peered":"2024-02-18T00:32:17.986612+0000","last_clean":"2024-02-18T00:32:17.986612+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:32:17.986612+0000","last_undegraded":"2024-02-18T00:32:17.986612+0000","last_fullsized":"2024-02-18T00:32:17.986612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.221541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.221541+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986573+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:32:17.986573+0000","last_peered":"2024-02-18T00:32:17.986573+0000","last_clean":"2024-02-18T00:32:17.986573+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:32:17.986573+0000","last_undegraded":"2024-02-18T00:32:17.986573+0000","last_fullsized":"2024-02-18T00:32:17.986573+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067323+0000","last_change":"2024-02-18T00:31:47.974066+0000","last_active":"2024-02-18T00:32:18.067323+0000","last_peered":"2024-02-18T00:32:18.067323+0000","last_clean":"2024-02-18T00:32:18.067323+0000","last_became_active":"2024-02-18T00:29:14.715019+0000","last_became_peered":"2024-02-18T00:29:14.715019+0000","last_unstale":"2024-02-18T00:32:18.067323+0000","last_undegraded":"2024-02-18T00:32:18.067323+0000","last_fullsized":"2024-02-18T00:32:18.067323+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:47.973962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:47.973962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"64'16","reported_seq":49,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986646+0000","last_change":"2024-02-18T00:31:33.699627+0000","last_active":"2024-02-18T00:32:17.986646+0000","last_peered":"2024-02-18T00:32:17.986646+0000","last_clean":"2024-02-18T00:32:17.986646+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:32:17.986646+0000","last_undegraded":"2024-02-18T00:32:17.986646+0000","last_fullsized":"2024-02-18T00:32:17.986646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:33.699554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:33.699554+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":105,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.295687+0000","last_change":"2024-02-18T00:32:05.026757+0000","last_active":"2024-02-18T00:32:28.295687+0000","last_peered":"2024-02-18T00:32:28.295687+0000","last_clean":"2024-02-18T00:32:28.295687+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:32:28.295687+0000","last_undegraded":"2024-02-18T00:32:28.295687+0000","last_fullsized":"2024-02-18T00:32:28.295687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:05.026652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.026652+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070139+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:32:18.070139+0000","last_peered":"2024-02-18T00:32:18.070139+0000","last_clean":"2024-02-18T00:32:18.070139+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:32:18.070139+0000","last_undegraded":"2024-02-18T00:32:18.070139+0000","last_fullsized":"2024-02-18T00:32:18.070139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065504+0000","last_change":"2024-02-18T00:31:08.046030+0000","last_active":"2024-02-18T00:32:18.065504+0000","last_peered":"2024-02-18T00:32:18.065504+0000","last_clean":"2024-02-18T00:32:18.065504+0000","last_became_active":"2024-02-18T00:29:14.715157+0000","last_became_peered":"2024-02-18T00:29:14.715157+0000","last_unstale":"2024-02-18T00:32:18.065504+0000","last_undegraded":"2024-02-18T00:32:18.065504+0000","last_fullsized":"2024-02-18T00:32:18.065504+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:31:08.045946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":163,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.295684+0000","last_change":"2024-02-18T00:32:04.954445+0000","last_active":"2024-02-18T00:32:28.295684+0000","last_peered":"2024-02-18T00:32:28.295684+0000","last_clean":"2024-02-18T00:32:28.295684+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:32:28.295684+0000","last_undegraded":"2024-02-18T00:32:28.295684+0000","last_fullsized":"2024-02-18T00:32:28.295684+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:32:04.954359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.954359+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987781+0000","last_change":"2024-02-18T00:32:07.008657+0000","last_active":"2024-02-18T00:32:17.987781+0000","last_peered":"2024-02-18T00:32:17.987781+0000","last_clean":"2024-02-18T00:32:17.987781+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:32:17.987781+0000","last_undegraded":"2024-02-18T00:32:17.987781+0000","last_fullsized":"2024-02-18T00:32:17.987781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.008543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.008543+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986508+0000","last_change":"2024-02-18T00:30:58.601762+0000","last_active":"2024-02-18T00:32:17.986508+0000","last_peered":"2024-02-18T00:32:17.986508+0000","last_clean":"2024-02-18T00:32:17.986508+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:32:17.986508+0000","last_undegraded":"2024-02-18T00:32:17.986508+0000","last_fullsized":"2024-02-18T00:32:17.986508+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:30:58.601694+0000","last_clean_scrub_stamp":"2024-02-18T00:30:58.601694+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":257,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:34.041374+0000","last_change":"2024-02-18T00:31:58.945520+0000","last_active":"2024-02-18T00:32:34.041374+0000","last_peered":"2024-02-18T00:32:34.041374+0000","last_clean":"2024-02-18T00:32:34.041374+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:32:34.041374+0000","last_undegraded":"2024-02-18T00:32:34.041374+0000","last_fullsized":"2024-02-18T00:32:34.041374+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:31:58.945452+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.945452+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":192,"num_read_kb":187,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986938+0000","last_change":"2024-02-18T00:30:48.262064+0000","last_active":"2024-02-18T00:32:17.986938+0000","last_peered":"2024-02-18T00:32:17.986938+0000","last_clean":"2024-02-18T00:32:17.986938+0000","last_became_active":"2024-02-18T00:29:14.713812+0000","last_became_peered":"2024-02-18T00:29:14.713812+0000","last_unstale":"2024-02-18T00:32:17.986938+0000","last_undegraded":"2024-02-18T00:32:17.986938+0000","last_fullsized":"2024-02-18T00:32:17.986938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:48.262001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.262001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":102,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067051+0000","last_change":"2024-02-18T00:31:55.005638+0000","last_active":"2024-02-18T00:32:18.067051+0000","last_peered":"2024-02-18T00:32:18.067051+0000","last_clean":"2024-02-18T00:32:18.067051+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:32:18.067051+0000","last_undegraded":"2024-02-18T00:32:18.067051+0000","last_fullsized":"2024-02-18T00:32:18.067051+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:31:55.005563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.005563+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986972+0000","last_change":"2024-02-18T00:31:47.502734+0000","last_active":"2024-02-18T00:32:17.986972+0000","last_peered":"2024-02-18T00:32:17.986972+0000","last_clean":"2024-02-18T00:32:17.986972+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:32:17.986972+0000","last_undegraded":"2024-02-18T00:32:17.986972+0000","last_fullsized":"2024-02-18T00:32:17.986972+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:47.502660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:47.502660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066619+0000","last_change":"2024-02-18T00:31:46.851657+0000","last_active":"2024-02-18T00:32:18.066619+0000","last_peered":"2024-02-18T00:32:18.066619+0000","last_clean":"2024-02-18T00:32:18.066619+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:32:18.066619+0000","last_undegraded":"2024-02-18T00:32:18.066619+0000","last_fullsized":"2024-02-18T00:32:18.066619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.851573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.071700+0000","last_change":"2024-02-18T00:30:56.034602+0000","last_active":"2024-02-18T00:32:18.071700+0000","last_peered":"2024-02-18T00:32:18.071700+0000","last_clean":"2024-02-18T00:32:18.071700+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:32:18.071700+0000","last_undegraded":"2024-02-18T00:32:18.071700+0000","last_fullsized":"2024-02-18T00:32:18.071700+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:56.034493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:30:56.034493+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065522+0000","last_change":"2024-02-18T00:31:42.847902+0000","last_active":"2024-02-18T00:32:18.065522+0000","last_peered":"2024-02-18T00:32:18.065522+0000","last_clean":"2024-02-18T00:32:18.065522+0000","last_became_active":"2024-02-18T00:29:14.716595+0000","last_became_peered":"2024-02-18T00:29:14.716595+0000","last_unstale":"2024-02-18T00:32:18.065522+0000","last_undegraded":"2024-02-18T00:32:18.065522+0000","last_fullsized":"2024-02-18T00:32:18.065522+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.985756+0000","last_change":"2024-02-18T00:30:46.504493+0000","last_active":"2024-02-18T00:32:17.985756+0000","last_peered":"2024-02-18T00:32:17.985756+0000","last_clean":"2024-02-18T00:32:17.985756+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:32:17.985756+0000","last_undegraded":"2024-02-18T00:32:17.985756+0000","last_fullsized":"2024-02-18T00:32:17.985756+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:46.504422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:30:46.504422+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":82,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067518+0000","last_change":"2024-02-18T00:31:25.298149+0000","last_active":"2024-02-18T00:32:18.067518+0000","last_peered":"2024-02-18T00:32:18.067518+0000","last_clean":"2024-02-18T00:32:18.067518+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:32:18.067518+0000","last_undegraded":"2024-02-18T00:32:18.067518+0000","last_fullsized":"2024-02-18T00:32:18.067518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:31:25.298062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:25.298062+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986357+0000","last_change":"2024-02-18T00:30:27.608046+0000","last_active":"2024-02-18T00:32:17.986357+0000","last_peered":"2024-02-18T00:32:17.986357+0000","last_clean":"2024-02-18T00:32:17.986357+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:32:17.986357+0000","last_undegraded":"2024-02-18T00:32:17.986357+0000","last_fullsized":"2024-02-18T00:32:17.986357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:27.607981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:27.607981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067177+0000","last_change":"2024-02-18T00:31:00.091252+0000","last_active":"2024-02-18T00:32:18.067177+0000","last_peered":"2024-02-18T00:32:18.067177+0000","last_clean":"2024-02-18T00:32:18.067177+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:32:18.067177+0000","last_undegraded":"2024-02-18T00:32:18.067177+0000","last_fullsized":"2024-02-18T00:32:18.067177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:00.091169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:00.091169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":60,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987340+0000","last_change":"2024-02-18T00:31:53.976449+0000","last_active":"2024-02-18T00:32:17.987340+0000","last_peered":"2024-02-18T00:32:17.987340+0000","last_clean":"2024-02-18T00:32:17.987340+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:32:17.987340+0000","last_undegraded":"2024-02-18T00:32:17.987340+0000","last_fullsized":"2024-02-18T00:32:17.987340+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:53.976377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:31:53.976377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.985777+0000","last_change":"2024-02-18T00:30:54.590835+0000","last_active":"2024-02-18T00:32:17.985777+0000","last_peered":"2024-02-18T00:32:17.985777+0000","last_clean":"2024-02-18T00:32:17.985777+0000","last_became_active":"2024-02-18T00:29:14.714355+0000","last_became_peered":"2024-02-18T00:29:14.714355+0000","last_unstale":"2024-02-18T00:32:17.985777+0000","last_undegraded":"2024-02-18T00:32:17.985777+0000","last_fullsized":"2024-02-18T00:32:17.985777+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:54.590771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:54.590771+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":72,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987944+0000","last_change":"2024-02-18T00:31:01.883504+0000","last_active":"2024-02-18T00:32:17.987944+0000","last_peered":"2024-02-18T00:32:17.987944+0000","last_clean":"2024-02-18T00:32:17.987944+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:32:17.987944+0000","last_undegraded":"2024-02-18T00:32:17.987944+0000","last_fullsized":"2024-02-18T00:32:17.987944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:01.883424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:01.883424+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066607+0000","last_change":"2024-02-18T00:30:52.072719+0000","last_active":"2024-02-18T00:32:18.066607+0000","last_peered":"2024-02-18T00:32:18.066607+0000","last_clean":"2024-02-18T00:32:18.066607+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:32:18.066607+0000","last_undegraded":"2024-02-18T00:32:18.066607+0000","last_fullsized":"2024-02-18T00:32:18.066607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:52.072619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:52.072619+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.068693+0000","last_change":"2024-02-18T00:31:11.020203+0000","last_active":"2024-02-18T00:32:18.068693+0000","last_peered":"2024-02-18T00:32:18.068693+0000","last_clean":"2024-02-18T00:32:18.068693+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:32:18.068693+0000","last_undegraded":"2024-02-18T00:32:18.068693+0000","last_fullsized":"2024-02-18T00:32:18.068693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:11.020119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.020119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":64,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986569+0000","last_change":"2024-02-18T00:30:53.587885+0000","last_active":"2024-02-18T00:32:17.986569+0000","last_peered":"2024-02-18T00:32:17.986569+0000","last_clean":"2024-02-18T00:32:17.986569+0000","last_became_active":"2024-02-18T00:29:14.715756+0000","last_became_peered":"2024-02-18T00:29:14.715756+0000","last_unstale":"2024-02-18T00:32:17.986569+0000","last_undegraded":"2024-02-18T00:32:17.986569+0000","last_fullsized":"2024-02-18T00:32:17.986569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:30:53.587796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:30:53.587796+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":105,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.990105+0000","last_change":"2024-02-18T00:30:48.688989+0000","last_active":"2024-02-18T00:32:17.990105+0000","last_peered":"2024-02-18T00:32:17.990105+0000","last_clean":"2024-02-18T00:32:17.990105+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:32:17.990105+0000","last_undegraded":"2024-02-18T00:32:17.990105+0000","last_fullsized":"2024-02-18T00:32:17.990105+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:30:48.688901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:30:48.688901+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067190+0000","last_change":"2024-02-18T00:30:45.463697+0000","last_active":"2024-02-18T00:32:18.067190+0000","last_peered":"2024-02-18T00:32:18.067190+0000","last_clean":"2024-02-18T00:32:18.067190+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:32:18.067190+0000","last_undegraded":"2024-02-18T00:32:18.067190+0000","last_fullsized":"2024-02-18T00:32:18.067190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.463628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.463628+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065782+0000","last_change":"2024-02-18T00:30:45.062422+0000","last_active":"2024-02-18T00:32:18.065782+0000","last_peered":"2024-02-18T00:32:18.065782+0000","last_clean":"2024-02-18T00:32:18.065782+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:32:18.065782+0000","last_undegraded":"2024-02-18T00:32:18.065782+0000","last_fullsized":"2024-02-18T00:32:18.065782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:45.062339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:45.062339+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066943+0000","last_change":"2024-02-18T00:31:38.977920+0000","last_active":"2024-02-18T00:32:18.066943+0000","last_peered":"2024-02-18T00:32:18.066943+0000","last_clean":"2024-02-18T00:32:18.066943+0000","last_became_active":"2024-02-18T00:29:14.714616+0000","last_became_peered":"2024-02-18T00:29:14.714616+0000","last_unstale":"2024-02-18T00:32:18.066943+0000","last_undegraded":"2024-02-18T00:32:18.066943+0000","last_fullsized":"2024-02-18T00:32:18.066943+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":132,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.069907+0000","last_change":"2024-02-18T00:32:05.656439+0000","last_active":"2024-02-18T00:32:18.069907+0000","last_peered":"2024-02-18T00:32:18.069907+0000","last_clean":"2024-02-18T00:32:18.069907+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:32:18.069907+0000","last_undegraded":"2024-02-18T00:32:18.069907+0000","last_fullsized":"2024-02-18T00:32:18.069907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.656344+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066119+0000","last_change":"2024-02-18T00:32:05.972387+0000","last_active":"2024-02-18T00:32:18.066119+0000","last_peered":"2024-02-18T00:32:18.066119+0000","last_clean":"2024-02-18T00:32:18.066119+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:32:18.066119+0000","last_undegraded":"2024-02-18T00:32:18.066119+0000","last_fullsized":"2024-02-18T00:32:18.066119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.972303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.972303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986364+0000","last_change":"2024-02-18T00:30:44.565713+0000","last_active":"2024-02-18T00:32:17.986364+0000","last_peered":"2024-02-18T00:32:17.986364+0000","last_clean":"2024-02-18T00:32:17.986364+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:32:17.986364+0000","last_undegraded":"2024-02-18T00:32:17.986364+0000","last_fullsized":"2024-02-18T00:32:17.986364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:44.565626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:44.565626+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067764+0000","last_change":"2024-02-18T00:31:58.198693+0000","last_active":"2024-02-18T00:32:18.067764+0000","last_peered":"2024-02-18T00:32:18.067764+0000","last_clean":"2024-02-18T00:32:18.067764+0000","last_became_active":"2024-02-18T00:29:14.714186+0000","last_became_peered":"2024-02-18T00:29:14.714186+0000","last_unstale":"2024-02-18T00:32:18.067764+0000","last_undegraded":"2024-02-18T00:32:18.067764+0000","last_fullsized":"2024-02-18T00:32:18.067764+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.198542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.198542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":96,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065496+0000","last_change":"2024-02-18T00:31:56.063646+0000","last_active":"2024-02-18T00:32:18.065496+0000","last_peered":"2024-02-18T00:32:18.065496+0000","last_clean":"2024-02-18T00:32:18.065496+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:32:18.065496+0000","last_undegraded":"2024-02-18T00:32:18.065496+0000","last_fullsized":"2024-02-18T00:32:18.065496+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:31:56.063560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.063560+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":107,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.296016+0000","last_change":"2024-02-18T00:31:39.294080+0000","last_active":"2024-02-18T00:32:28.296016+0000","last_peered":"2024-02-18T00:32:28.296016+0000","last_clean":"2024-02-18T00:32:28.296016+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:32:28.296016+0000","last_undegraded":"2024-02-18T00:32:28.296016+0000","last_fullsized":"2024-02-18T00:32:28.296016+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:31:39.294006+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:31:39.294006+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.070944+0000","last_change":"2024-02-18T00:31:58.530850+0000","last_active":"2024-02-18T00:32:18.070944+0000","last_peered":"2024-02-18T00:32:18.070944+0000","last_clean":"2024-02-18T00:32:18.070944+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:32:18.070944+0000","last_undegraded":"2024-02-18T00:32:18.070944+0000","last_fullsized":"2024-02-18T00:32:18.070944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.530768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.530768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066504+0000","last_change":"2024-02-18T00:31:16.020640+0000","last_active":"2024-02-18T00:32:18.066504+0000","last_peered":"2024-02-18T00:32:18.066504+0000","last_clean":"2024-02-18T00:32:18.066504+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:32:18.066504+0000","last_undegraded":"2024-02-18T00:32:18.066504+0000","last_fullsized":"2024-02-18T00:32:18.066504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:16.020558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:16.020558+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.985862+0000","last_change":"2024-02-18T00:31:48.594719+0000","last_active":"2024-02-18T00:32:17.985862+0000","last_peered":"2024-02-18T00:32:17.985862+0000","last_clean":"2024-02-18T00:32:17.985862+0000","last_became_active":"2024-02-18T00:29:14.713770+0000","last_became_peered":"2024-02-18T00:29:14.713770+0000","last_unstale":"2024-02-18T00:32:17.985862+0000","last_undegraded":"2024-02-18T00:32:17.985862+0000","last_fullsized":"2024-02-18T00:32:17.985862+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:48.594640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:48.594640+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":94,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.076079+0000","last_change":"2024-02-18T00:31:55.104289+0000","last_active":"2024-02-18T00:32:18.076079+0000","last_peered":"2024-02-18T00:32:18.076079+0000","last_clean":"2024-02-18T00:32:18.076079+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:32:18.076079+0000","last_undegraded":"2024-02-18T00:32:18.076079+0000","last_fullsized":"2024-02-18T00:32:18.076079+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:55.104195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.104195+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066198+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:32:18.066198+0000","last_peered":"2024-02-18T00:32:18.066198+0000","last_clean":"2024-02-18T00:32:18.066198+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:32:18.066198+0000","last_undegraded":"2024-02-18T00:32:18.066198+0000","last_fullsized":"2024-02-18T00:32:18.066198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":101,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.296403+0000","last_change":"2024-02-18T00:31:05.997541+0000","last_active":"2024-02-18T00:32:28.296403+0000","last_peered":"2024-02-18T00:32:28.296403+0000","last_clean":"2024-02-18T00:32:28.296403+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:32:28.296403+0000","last_undegraded":"2024-02-18T00:32:28.296403+0000","last_fullsized":"2024-02-18T00:32:28.296403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:31:05.997460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.997460+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.068907+0000","last_change":"2024-02-18T00:31:46.960767+0000","last_active":"2024-02-18T00:32:18.068907+0000","last_peered":"2024-02-18T00:32:18.068907+0000","last_clean":"2024-02-18T00:32:18.068907+0000","last_became_active":"2024-02-18T00:29:14.713732+0000","last_became_peered":"2024-02-18T00:29:14.713732+0000","last_unstale":"2024-02-18T00:32:18.068907+0000","last_undegraded":"2024-02-18T00:32:18.068907+0000","last_fullsized":"2024-02-18T00:32:18.068907+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.960661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.960661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":63,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066282+0000","last_change":"2024-02-18T00:31:50.844080+0000","last_active":"2024-02-18T00:32:18.066282+0000","last_peered":"2024-02-18T00:32:18.066282+0000","last_clean":"2024-02-18T00:32:18.066282+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:32:18.066282+0000","last_undegraded":"2024-02-18T00:32:18.066282+0000","last_fullsized":"2024-02-18T00:32:18.066282+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.843994+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065700+0000","last_change":"2024-02-18T00:31:58.064645+0000","last_active":"2024-02-18T00:32:18.065700+0000","last_peered":"2024-02-18T00:32:18.065700+0000","last_clean":"2024-02-18T00:32:18.065700+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:32:18.065700+0000","last_undegraded":"2024-02-18T00:32:18.065700+0000","last_fullsized":"2024-02-18T00:32:18.065700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.064559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.064559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":104,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.295957+0000","last_change":"2024-02-18T00:30:51.228973+0000","last_active":"2024-02-18T00:32:28.295957+0000","last_peered":"2024-02-18T00:32:28.295957+0000","last_clean":"2024-02-18T00:32:28.295957+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:32:28.295957+0000","last_undegraded":"2024-02-18T00:32:28.295957+0000","last_fullsized":"2024-02-18T00:32:28.295957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:51.228902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:51.228902+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066436+0000","last_change":"2024-02-18T00:31:50.988509+0000","last_active":"2024-02-18T00:32:18.066436+0000","last_peered":"2024-02-18T00:32:18.066436+0000","last_clean":"2024-02-18T00:32:18.066436+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:32:18.066436+0000","last_undegraded":"2024-02-18T00:32:18.066436+0000","last_fullsized":"2024-02-18T00:32:18.066436+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.988429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":70,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065897+0000","last_change":"2024-02-18T00:31:11.002136+0000","last_active":"2024-02-18T00:32:18.065897+0000","last_peered":"2024-02-18T00:32:18.065897+0000","last_clean":"2024-02-18T00:32:18.065897+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:32:18.065897+0000","last_undegraded":"2024-02-18T00:32:18.065897+0000","last_fullsized":"2024-02-18T00:32:18.065897+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:31:11.002039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:11.002039+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.988055+0000","last_change":"2024-02-18T00:32:07.449328+0000","last_active":"2024-02-18T00:32:17.988055+0000","last_peered":"2024-02-18T00:32:17.988055+0000","last_clean":"2024-02-18T00:32:17.988055+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:32:17.988055+0000","last_undegraded":"2024-02-18T00:32:17.988055+0000","last_fullsized":"2024-02-18T00:32:17.988055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.449249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.449249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":53,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067225+0000","last_change":"2024-02-18T00:30:59.059018+0000","last_active":"2024-02-18T00:32:18.067225+0000","last_peered":"2024-02-18T00:32:18.067225+0000","last_clean":"2024-02-18T00:32:18.067225+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:32:18.067225+0000","last_undegraded":"2024-02-18T00:32:18.067225+0000","last_fullsized":"2024-02-18T00:32:18.067225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:59.058947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:59.058947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986136+0000","last_change":"2024-02-18T00:31:54.616292+0000","last_active":"2024-02-18T00:32:17.986136+0000","last_peered":"2024-02-18T00:32:17.986136+0000","last_clean":"2024-02-18T00:32:17.986136+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:32:17.986136+0000","last_undegraded":"2024-02-18T00:32:17.986136+0000","last_fullsized":"2024-02-18T00:32:17.986136+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:54.616213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.616213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":88,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.076174+0000","last_change":"2024-02-18T00:31:32.011975+0000","last_active":"2024-02-18T00:32:18.076174+0000","last_peered":"2024-02-18T00:32:18.076174+0000","last_clean":"2024-02-18T00:32:18.076174+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:32:18.076174+0000","last_undegraded":"2024-02-18T00:32:18.076174+0000","last_fullsized":"2024-02-18T00:32:18.076174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:32.011896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.011896+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":51,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.985668+0000","last_change":"2024-02-18T00:30:43.541530+0000","last_active":"2024-02-18T00:32:17.985668+0000","last_peered":"2024-02-18T00:32:17.985668+0000","last_clean":"2024-02-18T00:32:17.985668+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:32:17.985668+0000","last_undegraded":"2024-02-18T00:32:17.985668+0000","last_fullsized":"2024-02-18T00:32:17.985668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:43.541458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:43.541458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.988331+0000","last_change":"2024-02-18T00:32:02.507109+0000","last_active":"2024-02-18T00:32:17.988331+0000","last_peered":"2024-02-18T00:32:17.988331+0000","last_clean":"2024-02-18T00:32:17.988331+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:32:17.988331+0000","last_undegraded":"2024-02-18T00:32:17.988331+0000","last_fullsized":"2024-02-18T00:32:17.988331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:02.507010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.507010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987567+0000","last_change":"2024-02-18T00:32:04.461036+0000","last_active":"2024-02-18T00:32:17.987567+0000","last_peered":"2024-02-18T00:32:17.987567+0000","last_clean":"2024-02-18T00:32:17.987567+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:32:17.987567+0000","last_undegraded":"2024-02-18T00:32:17.987567+0000","last_fullsized":"2024-02-18T00:32:17.987567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:04.460968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.460968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":107,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.067145+0000","last_change":"2024-02-18T00:31:40.251680+0000","last_active":"2024-02-18T00:32:18.067145+0000","last_peered":"2024-02-18T00:32:18.067145+0000","last_clean":"2024-02-18T00:32:18.067145+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:32:18.067145+0000","last_undegraded":"2024-02-18T00:32:18.067145+0000","last_fullsized":"2024-02-18T00:32:18.067145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:31:40.251596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:40.251596+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":50,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987080+0000","last_change":"2024-02-18T00:30:50.764741+0000","last_active":"2024-02-18T00:32:17.987080+0000","last_peered":"2024-02-18T00:32:17.987080+0000","last_clean":"2024-02-18T00:32:17.987080+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:32:17.987080+0000","last_undegraded":"2024-02-18T00:32:17.987080+0000","last_fullsized":"2024-02-18T00:32:17.987080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:30:50.764665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:30:50.764665+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":58,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.986834+0000","last_change":"2024-02-18T00:32:00.527642+0000","last_active":"2024-02-18T00:32:17.986834+0000","last_peered":"2024-02-18T00:32:17.986834+0000","last_clean":"2024-02-18T00:32:17.986834+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:32:17.986834+0000","last_undegraded":"2024-02-18T00:32:17.986834+0000","last_fullsized":"2024-02-18T00:32:17.986834+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:00.527580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:00.527580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.066671+0000","last_change":"2024-02-18T00:32:03.931078+0000","last_active":"2024-02-18T00:32:18.066671+0000","last_peered":"2024-02-18T00:32:18.066671+0000","last_clean":"2024-02-18T00:32:18.066671+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:32:18.066671+0000","last_undegraded":"2024-02-18T00:32:18.066671+0000","last_fullsized":"2024-02-18T00:32:18.066671+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:32:03.930967+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.990206+0000","last_change":"2024-02-18T00:31:07.953329+0000","last_active":"2024-02-18T00:32:17.990206+0000","last_peered":"2024-02-18T00:32:17.990206+0000","last_clean":"2024-02-18T00:32:17.990206+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:32:17.990206+0000","last_undegraded":"2024-02-18T00:32:17.990206+0000","last_fullsized":"2024-02-18T00:32:17.990206+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:07.953265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.953265+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":56,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065289+0000","last_change":"2024-02-18T00:32:05.790164+0000","last_active":"2024-02-18T00:32:18.065289+0000","last_peered":"2024-02-18T00:32:18.065289+0000","last_clean":"2024-02-18T00:32:18.065289+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:32:18.065289+0000","last_undegraded":"2024-02-18T00:32:18.065289+0000","last_fullsized":"2024-02-18T00:32:18.065289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.790091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.790091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":62,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:17.987116+0000","last_change":"2024-02-18T00:31:57.561503+0000","last_active":"2024-02-18T00:32:17.987116+0000","last_peered":"2024-02-18T00:32:17.987116+0000","last_clean":"2024-02-18T00:32:17.987116+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:32:17.987116+0000","last_undegraded":"2024-02-18T00:32:17.987116+0000","last_fullsized":"2024-02-18T00:32:17.987116+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.561435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.561435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":110,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-02-18T00:32:18.065193+0000","last_change":"2024-02-18T00:31:07.029773+0000","last_active":"2024-02-18T00:32:18.065193+0000","last_peered":"2024-02-18T00:32:18.065193+0000","last_clean":"2024-02-18T00:32:18.065193+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:32:18.065193+0000","last_undegraded":"2024-02-18T00:32:18.065193+0000","last_fullsized":"2024-02-18T00:32:18.065193+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:31:07.029677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.029677+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":101,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-02-18T00:32:28.296332+0000","last_change":"2024-02-18T00:30:49.588342+0000","last_active":"2024-02-18T00:32:28.296332+0000","last_peered":"2024-02-18T00:32:28.296332+0000","last_clean":"2024-02-18T00:32:28.296332+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:32:28.296332+0000","last_undegraded":"2024-02-18T00:32:28.296332+0000","last_fullsized":"2024-02-18T00:32:28.296332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:30:49.588269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:30:49.588269+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":605,"ondisk_log_size":605,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":192,"num_read_kb":187,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266732,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447348,"statfs":{"total":95995035648,"available":95690084352,"internally_reserved":0,"allocated":724992,"data_stored":340987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51400000000000001,"15min":0.502},"min":{"1min":0.39500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.746,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.38,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71099999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.50800000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72299999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79100000000000004,"5min":0.88,"15min":0.88},"last":0.64100000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64100000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.499},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.378,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76600000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60199999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.44600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.45100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.74299999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42899999999999999,"15min":0.41599999999999998},"min":{"1min":0.33900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66000000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.42899999999999999,"15min":0.41299999999999998},"min":{"1min":0.35699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.57299999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44700000000000001,"15min":0.441},"min":{"1min":0.36499999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.621,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.46899999999999997,"15min":0.45700000000000002},"min":{"1min":0.36099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:32:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.503,"15min":0.48899999999999999},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71299999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.755},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.41899999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73699999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447368,"statfs":{"total":95995035648,"available":95690104832,"internally_reserved":0,"allocated":704512,"data_stored":339822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.39500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67500000000000004,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51300000000000001,"15min":0.497},"min":{"1min":0.42599999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.627,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.35299999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.42799999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.67000000000000004,"5min":0.874,"15min":0.874},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.41099999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61299999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54200000000000004}]},{"osd":2,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.40400000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.66300000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.45700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69999999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.40200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.59499999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.42399999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65300000000000002,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57999999999999996}]},{"osd":4,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.67500000000000004,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.42299999999999999,"15min":0.40300000000000002},"min":{"1min":0.29199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80400000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63900000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:31:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.438,"15min":0.41399999999999998},"min":{"1min":0.38900000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.121,"15min":1.121},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.44800000000000001,"15min":0.42199999999999999},"min":{"1min":0.35999999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72999999999999998,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51200000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.379,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":1.075,"15min":1.075},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.40799999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84199999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301924,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":93443228,"statfs":{"total":95995035648,"available":95685865472,"internally_reserved":0,"allocated":749568,"data_stored":340960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.374,"15min":0.316},"min":{"1min":0.25800000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.49199999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.372,"15min":0.32500000000000001},"min":{"1min":0.26300000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53900000000000003,"5min":1.129,"15min":1.129},"last":0.41099999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38,"15min":0.373},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.627,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.39500000000000002},"min":{"1min":0.27300000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.012,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:31:33 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.434,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.55500000000000005,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33300000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.53400000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.44500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76400000000000001,"5min":1.347,"15min":1.347},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81999999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.45600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.038,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.45000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.298,"5min":1.427,"15min":1.427},"last":0.752}]},{"osd":6,"last update":"Sun Feb 18 00:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80500000000000005,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.625,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83699999999999997,"5min":1.278,"15min":1.278},"last":0.51500000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.441,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.393,"15min":1.393},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.67600000000000005,"15min":0.66100000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.65000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297800,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447352,"statfs":{"total":95995035648,"available":95690088448,"internally_reserved":0,"allocated":720896,"data_stored":340628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.42399999999999999,"15min":0.57699999999999996},"min":{"1min":0.221,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46500000000000002,"5min":6.7720000000000002,"15min":6.7720000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38800000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.32800000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:32:08 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40400000000000003,"15min":0.39200000000000002},"min":{"1min":0.26000000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.55600000000000005,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.23000000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.56799999999999995,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.48799999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.442},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.46200000000000002,"15min":0.495},"min":{"1min":0.314,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.57499999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42999999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:31:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.38600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80800000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.46600000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71499999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:31:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.625},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.71199999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:32:08 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82799999999999996,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.872,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.752}]}]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297780,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447372,"statfs":{"total":95995035648,"available":95690108928,"internally_reserved":0,"allocated":700416,"data_stored":339899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39700000000000002,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.79200000000000004,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.504},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.40000000000000002,"15min":0.376},"min":{"1min":0.25900000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.54000000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41299999999999998,"15min":0.38800000000000001},"min":{"1min":0.28799999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":2.306,"15min":2.306},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.73899999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.436}]},{"osd":3,"last update":"Sun Feb 18 00:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.33600000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.55600000000000005,"5min":1.296,"15min":1.296},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.47799999999999998,"15min":0.47699999999999998},"min":{"1min":0.35199999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61899999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.59899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85099999999999998,"5min":1.675,"15min":1.675},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":1.173,"15min":1.173},"last":0.80900000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.46999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93999999999999995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:31:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61499999999999999,"15min":0.64000000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69699999999999995,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.439,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90100000000000002,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65600000000000003,"15min":0.68999999999999995},"min":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86199999999999999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.52000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83899999999999997,"5min":1.242,"15min":1.242},"last":0.83999999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297816,"kb_used_data":720,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447336,"statfs":{"total":95995035648,"available":95690072064,"internally_reserved":0,"allocated":737280,"data_stored":340297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.373,"15min":0.39000000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.27300000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44},{"interface":"front","average":{"1min":0.496,"5min":0.40799999999999997,"15min":0.40699999999999997},"min":{"1min":0.26600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.32400000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.436,"15min":0.438},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:32:02 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.46899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.878,"5min":1.083,"15min":1.083},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98199999999999998,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.58699999999999997,"15min":0.59999999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97199999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82999999999999996,"5min":1.139,"15min":1.139},"last":0.46400000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94699999999999995,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.623,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.94799999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.59499999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90700000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.46000000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89200000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52000000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447348,"statfs":{"total":95995035648,"available":95690084352,"internally_reserved":0,"allocated":724992,"data_stored":340782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83399999999999996,"5min":2.339,"15min":2.339},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54600000000000004,"15min":0.56499999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72199999999999998,"5min":1.288,"15min":1.288},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.745,"5min":1.204,"15min":1.204},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.78200000000000003,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.64600000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58499999999999996,"15min":0.61199999999999999},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81399999999999995,"5min":1.48,"15min":1.48},"last":0.752},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.66100000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.60299999999999998},"min":{"1min":0.44700000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79300000000000004,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.39600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":1.133,"15min":1.133},"last":0.73499999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.40500000000000003,"15min":0.36799999999999999},"min":{"1min":0.36399999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.42099999999999999,"15min":0.377},"min":{"1min":0.309,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.48199999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:31:51 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"min":{"1min":0.309,"5min":0.25,"15min":0.25},"max":{"1min":0.69099999999999995,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.436},"min":{"1min":0.34699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.58499999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:32:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.495,"15min":0.48299999999999998},"min":{"1min":0.39300000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.49299999999999999,"15min":0.47599999999999998},"min":{"1min":0.41299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54900000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724598,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297780,"kb_used_data":684,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447372,"statfs":{"total":95995035648,"available":95690108928,"internally_reserved":0,"allocated":700416,"data_stored":339497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68000000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52500000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.50900000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":1.306,"15min":1.306},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.432,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":1.272,"15min":1.272},"last":0.53400000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72399999999999998,"5min":1.173,"15min":1.173},"last":0.435},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58399999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.45100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64900000000000002,"5min":1.238,"15min":1.238},"last":0.499},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.746,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.374},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.40000000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:31:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.33300000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.34999999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46899999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:32:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.39500000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.48799999999999999,"15min":0.47399999999999998},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73099999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:32:39.903 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-02-18T00:32:39.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool set .rgw.root pg_num 42 2024-02-18T00:32:39.965 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:39.994 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:40.014 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:40.016 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:40.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:40.117 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:40.144 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:40.163 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:40.165 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:40.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.168+0000 7f3927530700 1 -- 172.21.15.59:0/3839511027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f6aa0 msgr2=0x7f39200f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.168+0000 7f3927530700 1 --2- 172.21.15.59:0/3839511027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f6aa0 0x7f39200f6e80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f3910004870 tx=0x7f3910009cb0).stop 2024-02-18T00:32:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.172+0000 7f3927530700 1 -- 172.21.15.59:0/3839511027 shutdown_connections 2024-02-18T00:32:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.172+0000 7f3927530700 1 --2- 172.21.15.59:0/3839511027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f6aa0 0x7f39200f6e80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.172+0000 7f3927530700 1 --2- 172.21.15.59:0/3839511027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39200fbfe0 0x7f39200fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:40.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.172+0000 7f3927530700 1 --2- 172.21.15.59:0/3839511027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39200f7400 0x7f39200fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:40.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.172+0000 7f3927530700 1 -- 172.21.15.59:0/3839511027 >> 172.21.15.59:0/3839511027 conn(0x7f39200f2820 msgr2=0x7f39200f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:40.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.173+0000 7f3927530700 1 -- 172.21.15.59:0/3839511027 shutdown_connections 2024-02-18T00:32:40.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.173+0000 7f3927530700 1 -- 172.21.15.59:0/3839511027 wait complete. 2024-02-18T00:32:40.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 librados: starting msgr at 2024-02-18T00:32:40.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 librados: starting objecter 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 Processor -- start 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 -- start start 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 librados: setting wanted keys 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 librados: calling monclient init 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39200f6aa0 0x7f3920103d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f7400 0x7f3920104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39200fbfe0 0x7f3920123010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3920104a80 con 0x7f39200fbfe0 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3920062700 con 0x7f39200f7400 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.174+0000 7f3927530700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200628c0 con 0x7f39200f6aa0 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.175+0000 7f39252cc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39200f6aa0 0x7f3920103d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.175+0000 7f39252cc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39200f6aa0 0x7f3920103d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:50994/0 (socket says 172.21.15.59:50994) 2024-02-18T00:32:40.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.175+0000 7f39252cc700 1 -- 172.21.15.59:0/3447348279 learned_addr learned my addr 172.21.15.59:0/3447348279 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.176+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f7400 0x7f3920104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.176+0000 7f3924acb700 1 -- 172.21.15.59:0/3447348279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39200f6aa0 msgr2=0x7f3920103d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.176+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f39200f6aa0 0x7f3920103d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.176+0000 7f3924acb700 1 -- 172.21.15.59:0/3447348279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39200fbfe0 msgr2=0x7f3920123010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.176+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39200fbfe0 0x7f3920123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.177+0000 7f3924acb700 1 -- 172.21.15.59:0/3447348279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910005040 con 0x7f39200f7400 2024-02-18T00:32:40.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.177+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f7400 0x7f3920104260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f391c00b4e0 tx=0x7f391c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f39167fc700 10 client.?.objecter ms_handle_connect 0x7f39200f7400 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f39167fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c002320 con 0x7f39200f7400 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f391c002950 con 0x7f39200f7400 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3920062ba0 con 0x7f39200f7400 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c012a20 con 0x7f39200f7400 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f3927530700 10 client.24703.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3920063090 con 0x7f39200f7400 2024-02-18T00:32:40.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.179+0000 7f3927530700 1 librados: init done 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.180+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f391c01c440 con 0x7f39200f7400 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f3904004f30 con 0x7f39200f7400 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39167fc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6800/3721891797 conn(0x7f390c062ce0 0x7f390c0651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 44..101) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f391c07da90 con 0x7f39200f7400 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 osd_map(101..101 src has 44..101) v4 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39167fc700 3 client.24703.objecter handle_osd_map got epochs [101,101] > 0 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39167fc700 3 client.24703.objecter handle_osd_map decoding full epoch 101 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39167fc700 20 client.24703.objecter dump_active .. 0 homeless 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6800/3721891797 conn(0x7f390c062ce0 0x7f390c0651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.181+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v101) v1 ==== 54+0+16 (secure 0 0 0) 0x7f391c084070 con 0x7f39200f7400 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 10 client.24703.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 1 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6801/3700238977 conn(0x7f3904004560 0x7f39040049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 1 2024-02-18T00:32:40.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 0 1 2024-02-18T00:32:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f39040051d0 osd=0 4 2024-02-18T00:32:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 10 client.24703.objecter _send_command 1 2024-02-18T00:32:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3904008960 con 0x7f3904004560 2024-02-18T00:32:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f390c062ce0 2024-02-18T00:32:40.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.182+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6800/3721891797 conn(0x7f390c062ce0 0x7f390c0651a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f3910009da0 tx=0x7f3910009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.188+0000 7f3925acd700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6801/3700238977 conn(0x7f3904004560 0x7f39040049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.188+0000 7f3925acd700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6801/3700238977 conn(0x7f3904004560 0x7f39040049a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.188+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f3904004560 2024-02-18T00:32:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.189+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3904008960 con 0x7f3904004560 2024-02-18T00:32:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.189+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:32:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.189+0000 7f39167fc700 10 client.24703.objecter _finish_command 1 = system:0 2024-02-18T00:32:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.189+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f39040051d0 osd=0 4 2024-02-18T00:32:40.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.189+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 0 1 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 10 client.24703.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 2 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 2 2024-02-18T00:32:40.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f39040051d0 osd=0 3 2024-02-18T00:32:40.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 0 2 2024-02-18T00:32:40.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f39040051d0 osd=0 4 2024-02-18T00:32:40.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 10 client.24703.objecter _send_command 2 2024-02-18T00:32:40.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.221+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3904003fa0 con 0x7f3904004560 2024-02-18T00:32:40.231 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:32:40.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.228+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3904003fa0 con 0x7f3904004560 2024-02-18T00:32:40.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.228+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.228+0000 7f39167fc700 10 client.24703.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.228+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f39040051d0 osd=0 4 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.228+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 0 2 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 10 client.24703.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 3 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6805/26279110 conn(0x7f3904002740 0x7f3904002bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 3 2024-02-18T00:32:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 1 3 2024-02-18T00:32:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904007fb0 osd=1 4 2024-02-18T00:32:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 10 client.24703.objecter _send_command 3 2024-02-18T00:32:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.229+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3904008700 con 0x7f3904002740 2024-02-18T00:32:40.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.231+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6805/26279110 conn(0x7f3904002740 0x7f3904002bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.233+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6805/26279110 conn(0x7f3904002740 0x7f3904002bd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.233+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f3904002740 2024-02-18T00:32:40.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.235+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3904008700 con 0x7f3904002740 2024-02-18T00:32:40.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.235+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:32:40.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.235+0000 7f39167fc700 10 client.24703.objecter _finish_command 3 = system:0 2024-02-18T00:32:40.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.235+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904007fb0 osd=1 4 2024-02-18T00:32:40.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.235+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 1 3 2024-02-18T00:32:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:38.648620+0000 mgr.y (mgr.24454) 136 : audit [DBG] from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:32:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:38.664795+0000 osd.7 (osd.7) 59 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:32:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:38.665823+0000 osd.7 (osd.7) 60 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:32:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:38.801767+0000 osd.1 (osd.1) 69 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:32:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:38.804770+0000 osd.1 (osd.1) 70 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:32:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:38.982249+0000 osd.4 (osd.4) 67 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:38.985174+0000 osd.4 (osd.4) 68 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:38.648620+0000 mgr.y (mgr.24454) 136 : audit [DBG] from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:38.664795+0000 osd.7 (osd.7) 59 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:38.665823+0000 osd.7 (osd.7) 60 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:38.801767+0000 osd.1 (osd.1) 69 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:38.804770+0000 osd.1 (osd.1) 70 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:38.982249+0000 osd.4 (osd.4) 67 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:32:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:38.985174+0000 osd.4 (osd.4) 68 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:32:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:38.648620+0000 mgr.y (mgr.24454) 136 : audit [DBG] from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:32:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:38.664795+0000 osd.7 (osd.7) 59 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:32:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:38.665823+0000 osd.7 (osd.7) 60 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:32:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:38.801767+0000 osd.1 (osd.1) 69 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:32:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:38.804770+0000 osd.1 (osd.1) 70 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:32:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:38.982249+0000 osd.4 (osd.4) 67 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:32:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:38.985174+0000 osd.4 (osd.4) 68 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:32:40.260 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 10 client.24703.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 4 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 4 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904007fb0 osd=1 3 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 1 4 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904007fb0 osd=1 4 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 10 client.24703.objecter _send_command 4 2024-02-18T00:32:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.264+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3904003890 con 0x7f3904002740 2024-02-18T00:32:40.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:40.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.267+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3904003890 con 0x7f3904002740 2024-02-18T00:32:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.267+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.267+0000 7f39167fc700 10 client.24703.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.267+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904007fb0 osd=1 4 2024-02-18T00:32:40.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.267+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 1 4 2024-02-18T00:32:40.270 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 10 client.24703.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 5 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6809/1153193637 conn(0x7f390400aff0 0x7f390400d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 5 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 2 5 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904003b40 osd=2 4 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 10 client.24703.objecter _send_command 5 2024-02-18T00:32:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3904010030 con 0x7f390400aff0 2024-02-18T00:32:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.269+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6809/1153193637 conn(0x7f390400aff0 0x7f390400d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.270+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f390400aff0 2024-02-18T00:32:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.270+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6809/1153193637 conn(0x7f390400aff0 0x7f390400d3e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.271+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3904010030 con 0x7f390400aff0 2024-02-18T00:32:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.271+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:32:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.271+0000 7f39167fc700 10 client.24703.objecter _finish_command 5 = system:0 2024-02-18T00:32:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.271+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904003b40 osd=2 4 2024-02-18T00:32:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.271+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 2 5 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 10 client.24703.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 6 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 6 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904003b40 osd=2 3 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 2 6 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904003b40 osd=2 4 2024-02-18T00:32:40.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 10 client.24703.objecter _send_command 6 2024-02-18T00:32:40.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.300+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3904003300 con 0x7f390400aff0 2024-02-18T00:32:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.303+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3904003300 con 0x7f390400aff0 2024-02-18T00:32:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.303+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.303+0000 7f39167fc700 10 client.24703.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.303+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904003b40 osd=2 4 2024-02-18T00:32:40.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.303+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 2 6 2024-02-18T00:32:40.308 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 10 client.24703.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 7 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6813/1805700964 conn(0x7f390400ea50 0x7f390400eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 7 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 3 7 2024-02-18T00:32:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904001bb0 osd=3 4 2024-02-18T00:32:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 10 client.24703.objecter _send_command 7 2024-02-18T00:32:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3904002230 con 0x7f390400ea50 2024-02-18T00:32:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.305+0000 7f3925acd700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6813/1805700964 conn(0x7f390400ea50 0x7f390400eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.307+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f390400ea50 2024-02-18T00:32:40.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.307+0000 7f3925acd700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6813/1805700964 conn(0x7f390400ea50 0x7f390400eec0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.310+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3904002230 con 0x7f390400ea50 2024-02-18T00:32:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.310+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:32:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.310+0000 7f39167fc700 10 client.24703.objecter _finish_command 7 = system:0 2024-02-18T00:32:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.310+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904001bb0 osd=3 4 2024-02-18T00:32:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.310+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 3 7 2024-02-18T00:32:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:40.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 10 client.24703.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 8 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 8 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904001bb0 osd=3 3 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 3 8 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904001bb0 osd=3 4 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 10 client.24703.objecter _send_command 8 2024-02-18T00:32:40.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.345+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f390400fb80 con 0x7f390400ea50 2024-02-18T00:32:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f390400fb80 con 0x7f390400ea50 2024-02-18T00:32:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f39167fc700 10 client.24703.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904001bb0 osd=3 4 2024-02-18T00:32:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 3 8 2024-02-18T00:32:40.350 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 10 client.24703.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 9 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39040121e0 0x7f39040145d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 9 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 4 9 2024-02-18T00:32:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400f3b0 osd=4 4 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 10 client.24703.objecter _send_command 9 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f390400fb50 con 0x7f39040121e0 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.348+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39040121e0 0x7f39040145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.349+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f39040121e0 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.349+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39040121e0 0x7f39040145d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.350+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f390400fb50 con 0x7f39040121e0 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.350+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.350+0000 7f39167fc700 10 client.24703.objecter _finish_command 9 = system:0 2024-02-18T00:32:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.350+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f390400f3b0 osd=4 4 2024-02-18T00:32:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.350+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 4 9 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 10 client.24703.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 10 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 10 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f390400f3b0 osd=4 3 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 4 10 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400f3b0 osd=4 4 2024-02-18T00:32:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 10 client.24703.objecter _send_command 10 2024-02-18T00:32:40.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.378+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f390400fb10 con 0x7f39040121e0 2024-02-18T00:32:40.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f390400fb10 con 0x7f39040121e0 2024-02-18T00:32:40.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f39167fc700 10 client.24703.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f390400f3b0 osd=4 4 2024-02-18T00:32:40.382 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:32:40.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 4 10 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 10 client.24703.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 11 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6804/2170432263 conn(0x7f3904016640 0x7f3904016aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 11 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 5 11 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400fb10 osd=5 4 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 10 client.24703.objecter _send_command 11 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f390401a3e0 con 0x7f3904016640 2024-02-18T00:32:40.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.380+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6804/2170432263 conn(0x7f3904016640 0x7f3904016aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.381+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f3904016640 2024-02-18T00:32:40.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.381+0000 7f3924acb700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6804/2170432263 conn(0x7f3904016640 0x7f3904016aa0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.381+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f390401a3e0 con 0x7f3904016640 2024-02-18T00:32:40.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.381+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:32:40.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.381+0000 7f39167fc700 10 client.24703.objecter _finish_command 11 = system:0 2024-02-18T00:32:40.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.381+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f390400fb10 osd=5 4 2024-02-18T00:32:40.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.381+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 5 11 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 10 client.24703.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 12 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 12 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f390400fb10 osd=5 3 2024-02-18T00:32:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 5 12 2024-02-18T00:32:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400fb10 osd=5 4 2024-02-18T00:32:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 10 client.24703.objecter _send_command 12 2024-02-18T00:32:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.410+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3904014ac0 con 0x7f3904016640 2024-02-18T00:32:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3904014ac0 con 0x7f3904016640 2024-02-18T00:32:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f39167fc700 10 client.24703.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f390400fb10 osd=5 4 2024-02-18T00:32:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 5 12 2024-02-18T00:32:40.414 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 10 client.24703.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 13 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39040151e0 0x7f3904015620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 13 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 6 13 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904014ac0 osd=6 4 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 10 client.24703.objecter _send_command 13 2024-02-18T00:32:40.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3904015cc0 con 0x7f39040151e0 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.412+0000 7f3925acd700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39040151e0 0x7f3904015620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.413+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f39040151e0 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.413+0000 7f3925acd700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39040151e0 0x7f3904015620 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.413+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3904015cc0 con 0x7f39040151e0 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.413+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.414+0000 7f39167fc700 10 client.24703.objecter _finish_command 13 = system:0 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.414+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904014ac0 osd=6 4 2024-02-18T00:32:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.414+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 6 13 2024-02-18T00:32:40.441 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 10 client.24703.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 14 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 14 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904014ac0 osd=6 3 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 6 14 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904014ac0 osd=6 4 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 10 client.24703.objecter _send_command 14 2024-02-18T00:32:40.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.437+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f3904015cc0 con 0x7f39040151e0 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.438+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f3904015cc0 con 0x7f39040151e0 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.438+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.438+0000 7f39167fc700 10 client.24703.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.438+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904014ac0 osd=6 4 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.438+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 6 14 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 10 client.24703.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 15 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6812/53025171 conn(0x7f390401b780 0x7f390401db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 15 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 7 15 2024-02-18T00:32:40.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904015cc0 osd=7 4 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 10 client.24703.objecter _send_command 15 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3904020800 con 0x7f390401b780 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6812/53025171 conn(0x7f390401b780 0x7f390401db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f39167fc700 10 client.24703.objecter ms_handle_connect 0x7f390401b780 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.439+0000 7f39252cc700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6812/53025171 conn(0x7f390401b780 0x7f390401db70 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.440+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3904020800 con 0x7f390401b780 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.440+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.440+0000 7f39167fc700 10 client.24703.objecter _finish_command 15 = system:0 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.440+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904015cc0 osd=7 4 2024-02-18T00:32:40.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.440+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 7 15 2024-02-18T00:32:40.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 10 client.24703.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:40.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign -1 16 2024-02-18T00:32:40.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 20 client.24703.objecter _get_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 15 client.24703.objecter _session_command_op_remove -1 16 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 20 client.24703.objecter get_session s=0x7f3904015cc0 osd=7 3 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 15 client.24703.objecter _session_command_op_assign 7 16 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904015cc0 osd=7 4 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 10 client.24703.objecter _send_command 16 2024-02-18T00:32:40.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.456+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f390401e060 con 0x7f390401b780 2024-02-18T00:32:40.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.457+0000 7f39167fc700 1 -- 172.21.15.59:0/3447348279 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f390401e060 con 0x7f390401b780 2024-02-18T00:32:40.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.457+0000 7f39167fc700 10 client.24703.objecter ms_dispatch 0x7f3920001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:40.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.457+0000 7f39167fc700 10 client.24703.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.457+0000 7f39167fc700 20 client.24703.objecter put_session s=0x7f3904015cc0 osd=7 4 2024-02-18T00:32:40.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.457+0000 7f39167fc700 15 client.24703.objecter _session_command_op_remove 7 16 2024-02-18T00:32:40.460 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 librados: watch_flush enter 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 librados: watch_flush exit 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.0 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6801/3700238977 conn(0x7f3904004560 msgr2=0x7f39040049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6801/3700238977 conn(0x7f3904004560 0x7f39040049a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f39040051d0 osd=0 1 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.1 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6805/26279110 conn(0x7f3904002740 msgr2=0x7f3904002bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6805/26279110 conn(0x7f3904002740 0x7f3904002bd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904007fb0 osd=1 1 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.2 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6809/1153193637 conn(0x7f390400aff0 msgr2=0x7f390400d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6809/1153193637 conn(0x7f390400aff0 0x7f390400d3e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904003b40 osd=2 1 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.3 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6813/1805700964 conn(0x7f390400ea50 msgr2=0x7f390400eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6813/1805700964 conn(0x7f390400ea50 0x7f390400eec0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904001bb0 osd=3 1 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.4 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39040121e0 msgr2=0x7f39040145d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6800/2044387056 conn(0x7f39040121e0 0x7f39040145d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3925acd700 1 -- 172.21.15.59:0/3447348279 reap_dead start 2024-02-18T00:32:40.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400f3b0 osd=4 1 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.5 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6804/2170432263 conn(0x7f3904016640 msgr2=0x7f3904016aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6804/2170432263 conn(0x7f3904016640 0x7f3904016aa0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f390400fb10 osd=5 1 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.6 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39040151e0 msgr2=0x7f3904015620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6808/3356009390 conn(0x7f39040151e0 0x7f3904015620 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3925acd700 1 -- 172.21.15.59:0/3447348279 reap_dead start 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904014ac0 osd=6 1 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter close_session for osd.7 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6812/53025171 conn(0x7f390401b780 msgr2=0x7f390401db70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.137:6812/53025171 conn(0x7f390401b780 0x7f390401db70 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter put_session s=0x7f3904015cc0 osd=7 1 2024-02-18T00:32:40.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 20 client.24703.objecter shutdown clearing up homeless session... 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 10 client.24703.objecter successfully canceled tick 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3925acd700 1 -- 172.21.15.59:0/3447348279 reap_dead start 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6800/3721891797 conn(0x7f390c062ce0 msgr2=0x7f390c0651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6800/3721891797 conn(0x7f390c062ce0 0x7f390c0651a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f3910009da0 tx=0x7f3910009cb0).stop 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f7400 msgr2=0x7f3920104260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f7400 0x7f3920104260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f391c00b4e0 tx=0x7f391c009060).stop 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.458+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 shutdown_connections 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.459+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39200f7400 0x7f3920104260 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0x7f391c00b4e0 tx=0x7f391c009060).stop 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.459+0000 7f3927530700 1 --2- 172.21.15.59:0/3447348279 >> v2:172.21.15.59:6800/3721891797 conn(0x7f390c062ce0 0x7f390c0651a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.459+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 >> 172.21.15.59:0/3447348279 conn(0x7f39200f2820 msgr2=0x7f39200f42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.459+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 shutdown_connections 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.459+0000 7f3927530700 1 -- 172.21.15.59:0/3447348279 wait complete. 2024-02-18T00:32:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:40.459+0000 7f3927530700 1 librados: shutdown 2024-02-18T00:32:40.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.465 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:32:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:39.761947+0000 mgr.y (mgr.24454) 137 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:39.971665+0000 osd.0 (osd.0) 59 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:39.973033+0000 osd.0 (osd.0) 60 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:40.182695+0000 mon.b (mon.1) 31 : audit [DBG] from='client.? 172.21.15.59:0/3447348279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:40.475697+0000 osd.2 (osd.2) 55 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:40.476840+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:40.847030+0000 mon.a (mon.0) 779 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:32:41.188 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:39.761947+0000 mgr.y (mgr.24454) 137 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:41.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:39.971665+0000 osd.0 (osd.0) 59 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:32:41.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:39.973033+0000 osd.0 (osd.0) 60 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:32:41.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:40.182695+0000 mon.b (mon.1) 31 : audit [DBG] from='client.? 172.21.15.59:0/3447348279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:41.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:40.475697+0000 osd.2 (osd.2) 55 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:32:41.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:40.476840+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:32:41.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:40.847030+0000 mon.a (mon.0) 779 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:32:41.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:32:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:32:41.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:32:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:39.761947+0000 mgr.y (mgr.24454) 137 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:39.971665+0000 osd.0 (osd.0) 59 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:32:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:39.973033+0000 osd.0 (osd.0) 60 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:32:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:40.182695+0000 mon.b (mon.1) 31 : audit [DBG] from='client.? 172.21.15.59:0/3447348279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:40.475697+0000 osd.2 (osd.2) 55 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:32:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:40.476840+0000 osd.2 (osd.2) 56 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:32:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:40.847030+0000 mon.a (mon.0) 779 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:32:41.680 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:41.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:41.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:41.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:41.833 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:41.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:41.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:41.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:40.773512+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:32:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:40.775664+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:32:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:40.791345+0000 osd.1 (osd.1) 71 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:40.792574+0000 osd.1 (osd.1) 72 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:40.899128+0000 mgr.y (mgr.24454) 138 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:41.024224+0000 osd.4 (osd.4) 69 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:41.027141+0000 osd.4 (osd.4) 70 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:41.316565+0000 mon.a (mon.0) 780 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:41.317329+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:41.322919+0000 mon.a (mon.0) 782 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:42] "GET /metrics HTTP/1.1" 200 172238 "" "Prometheus/2.33.4" 2024-02-18T00:32:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:40.773512+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:40.775664+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:40.791345+0000 osd.1 (osd.1) 71 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:40.792574+0000 osd.1 (osd.1) 72 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:40.899128+0000 mgr.y (mgr.24454) 138 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:41.024224+0000 osd.4 (osd.4) 69 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:41.027141+0000 osd.4 (osd.4) 70 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:41.316565+0000 mon.a (mon.0) 780 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:41.317329+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:32:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:41.322919+0000 mon.a (mon.0) 782 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:32:42.428 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:42.431 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:42.431 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:40.773512+0000 osd.3 (osd.3) 79 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:40.775664+0000 osd.3 (osd.3) 80 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:40.791345+0000 osd.1 (osd.1) 71 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:40.792574+0000 osd.1 (osd.1) 72 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:40.899128+0000 mgr.y (mgr.24454) 138 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:41.024224+0000 osd.4 (osd.4) 69 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:41.027141+0000 osd.4 (osd.4) 70 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:41.316565+0000 mon.a (mon.0) 780 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:32:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:41.317329+0000 mon.a (mon.0) 781 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:32:42.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:41.322919+0000 mon.a (mon.0) 782 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:32:42.532 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:42.564 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:42.586 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:42.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:42.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:42.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:42.721 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:42.741 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:42.744 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:42.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:42.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:42.883 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:43.014 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:43.018 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:43.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:43.059 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:32:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:32:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:32:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:32:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.14403.0:88 4.2 4:4347d321:::bucket.sync-source-hints.:head [call,read 0~0] snapc 0=[] ondisk+read+known_if_redirected e59)", 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:17.080439+0000", 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 205.974865005, 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021931049999999999, 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.14403", 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2016795673", 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 88 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.060 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.080439+0000", 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.080439+0000", 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.061 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.080437+0000", 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1733e-05 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.080458+0000", 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4415e-05 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.080473+0000", 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7189999999999996e-06 2024-02-18T00:32:43.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.080482+0000", 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00081045499999999996 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.081293+0000", 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.024e-05 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.081303+0000", 2024-02-18T00:32:43.063 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011944429999999999 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.082498+0000", 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013419699999999999 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:17.082632+0000", 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021931049999999999 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:9 4.1d e64/58 4:beaa4be4:gc::gc.7:head v 64'1, mlcod=0'0)", 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.111766+0000", 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.94353722599999, 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025387119999999998, 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111766+0000", 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111766+0000", 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111765+0000", 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7216000000000002e-05 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111793+0000", 2024-02-18T00:32:43.066 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5423000000000001e-05 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111808+0000", 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0560000000000005e-06 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111816+0000", 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9107999999999997e-05 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.067 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111875+0000", 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7830999999999999e-05 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.111893+0000", 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023783670000000002 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114271+0000", 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3657e-05 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.068 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114305+0000", 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025387119999999998 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24329.0:220 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94665234060176] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.113622+0000", 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.94168207300001, 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037061540000000001, 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24329", 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1565964696", 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 220 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.070 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113622+0000", 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113622+0000", 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.071 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113620+0000", 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1270000000000003e-06 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113628+0000", 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2339e-05 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.072 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113641+0000", 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3410000000000002e-06 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113646+0000", 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1100999999999997e-05 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113697+0000", 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6935999999999998e-05 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.073 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113754+0000", 2024-02-18T00:32:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010702800000000001 2024-02-18T00:32:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.074 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:32:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.113861+0000", 2024-02-18T00:32:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00074225199999999999 2024-02-18T00:32:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114603+0000", 2024-02-18T00:32:43.075 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002381055 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.116984+0000", 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000237288 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.117222+0000", 2024-02-18T00:32:43.079 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8600000000000003e-05 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.117260+0000", 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7416999999999995e-05 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.117328+0000", 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037061540000000001 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:10 4.12 4:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=18b,call rgw_gc.rgw_gc_queue_init in=23b,call version.set in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.114798+0000", 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.94050516300001, 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024770119999999998, 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 10 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.081 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114798+0000", 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114798+0000", 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114797+0000", 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4130000000000006e-06 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.082 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114806+0000", 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4613000000000001e-05 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114820+0000", 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1229999999999999e-06 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114825+0000", 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9799000000000002e-05 2024-02-18T00:32:43.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114875+0000", 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8208000000000007e-05 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.114963+0000", 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016747800000000001 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:32:43.084 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.115131+0000", 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000829972 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.115961+0000", 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011585639999999999 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.117119+0000", 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6923999999999998e-05 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.085 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.117186+0000", 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7988999999999997e-05 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.117234+0000", 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1205000000000003e-05 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.117275+0000", 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024770119999999998 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.086 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:36 4.2 4:4455f95b:lc::lc.27:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.150864+0000", 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.904439962, 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021822320000000001, 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 36 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.150864+0000", 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.150864+0000", 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.088 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.150862+0000", 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.285e-06 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.150871+0000", 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5225000000000001e-05 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.150886+0000", 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0290000000000005e-06 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.089 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.150894+0000", 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6567000000000001e-05 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.150961+0000", 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116829 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.151078+0000", 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020716800000000001 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.090 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4,5", 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.151285+0000", 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00080357200000000001 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.152088+0000", 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045153400000000002 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.152540+0000", 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040011400000000003 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.091 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.152940+0000", 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4882000000000007e-05 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.153005+0000", 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1013999999999997e-05 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.153046+0000", 2024-02-18T00:32:43.092 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021822320000000001 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:46 4.f 4:f1983ba0:::obj_delete_at_hint.0000000001:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.154630+0000", 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.90067343199999, 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025702590000000001, 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 46 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154630+0000", 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154630+0000", 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.094 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154629+0000", 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6565000000000001e-05 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154645+0000", 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0554e-05 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154666+0000", 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6990000000000003e-06 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.095 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154673+0000", 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6220999999999997e-05 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154720+0000", 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000135962 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.154856+0000", 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000205396 2024-02-18T00:32:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,4", 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.155061+0000", 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000551623 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.155613+0000", 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010543 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.155718+0000", 2024-02-18T00:32:43.097 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001372802 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157091+0000", 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1904999999999999e-05 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157143+0000", 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7637999999999999e-05 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.098 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157200+0000", 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025702590000000001 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.099 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:55 4.f 4:f1983ba0:::obj_delete_at_hint.0000000001:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.157846+0000", 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.89745717, 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021532399999999998, 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 55 2024-02-18T00:32:43.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157846+0000", 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157846+0000", 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.101 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157845+0000", 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1720000000000007e-06 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157853+0000", 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3699999999999999e-05 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157867+0000", 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.102 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157875+0000", 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7222000000000003e-05 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.157922+0000", 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000258531 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.158181+0000", 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000206138 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.103 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,4", 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.158387+0000", 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061961700000000004 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.159006+0000", 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00050367399999999996 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.104 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.159510+0000", 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040660700000000002 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.159917+0000", 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4666000000000002e-05 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.159961+0000", 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8386999999999997e-05 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.105 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.160000+0000", 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021532399999999998 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:113 4.4 4:2166d8e1:gc::gc.3:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.174389+0000", 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.880914499, 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041726230000000003, 2024-02-18T00:32:43.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 113 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174389+0000", 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.107 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174389+0000", 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174387+0000", 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1711e-05 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174399+0000", 2024-02-18T00:32:43.108 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9123999999999999e-05 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174418+0000", 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2930000000000006e-06 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174427+0000", 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8952e-05 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.109 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174496+0000", 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9109000000000003e-05 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174595+0000", 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025804499999999999 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.174853+0000", 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00054988300000000003 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.110 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.175403+0000", 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029322400000000002 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.175697+0000", 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027802399999999998 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.178477+0000", 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7800999999999999e-05 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.178525+0000", 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7172999999999998e-05 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.178562+0000", 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0041726230000000003 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.112 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:140 4.4 4:2166d8e1:gc::gc.3:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.182190+0000", 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.87311348899999, 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024533200000000002, 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 140 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.113 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182190+0000", 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182190+0000", 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182189+0000", 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2939999999999996e-06 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.114 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182195+0000", 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4720000000000001e-06 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182204+0000", 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4380000000000003e-06 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182210+0000", 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9246999999999998e-05 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.115 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182259+0000", 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8848e-05 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182348+0000", 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000185716 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.182533+0000", 2024-02-18T00:32:43.116 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00059533700000000004 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.183129+0000", 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001011818 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184141+0000", 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041031500000000002 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.117 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184551+0000", 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3633e-05 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184605+0000", 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8927000000000002e-05 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.184643+0000", 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024533200000000002 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.118 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:155 4.4 4:25ca39bb:::obj_delete_at_hint.0000000005:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.186998+0000", 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.868305357, 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022168190000000001, 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 155 2024-02-18T00:32:43.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.186998+0000", 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.186998+0000", 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.120 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.186997+0000", 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4919999999999996e-06 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187005+0000", 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4931e-05 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187019+0000", 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9539999999999998e-06 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.121 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187026+0000", 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4760999999999999e-05 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187091+0000", 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5983000000000006e-05 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187187+0000", 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000214825 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.122 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.187402+0000", 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000741743 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.188144+0000", 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00064243899999999997 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.188786+0000", 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00033582000000000002 2024-02-18T00:32:43.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189122+0000", 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5478e-05 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189177+0000", 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7622e-05 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.189215+0000", 2024-02-18T00:32:43.124 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022168190000000001 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:245 4.7 4:e1cdb60d:reshard::reshard.0000000008:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.206354+0000", 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.848949645, 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002258712, 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.125 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 245 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206354+0000", 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206354+0000", 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206353+0000", 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5310000000000004e-06 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206360+0000", 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.094e-05 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206371+0000", 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.268e-06 2024-02-18T00:32:43.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206377+0000", 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9916999999999997e-05 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206417+0000", 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4453000000000001e-05 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.128 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206482+0000", 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014217900000000001 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,5", 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.206624+0000", 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041295299999999997 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.207037+0000", 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00054223699999999999 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.207579+0000", 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00098482600000000006 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.208564+0000", 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8342000000000001e-05 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.208592+0000", 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0407999999999998e-05 2024-02-18T00:32:43.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.208613+0000", 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002258712 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:424 4.4 4:227f63b5:gc::gc.14:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.246861+0000", 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.80844224399999, 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021000089999999999, 2024-02-18T00:32:43.131 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 424 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.246861+0000", 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.132 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.246861+0000", 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.246860+0000", 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6190000000000002e-06 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.246868+0000", 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3036e-05 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.133 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.246881+0000", 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9550000000000003e-06 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.246888+0000", 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8755999999999999e-05 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.246937+0000", 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8621999999999994e-05 2024-02-18T00:32:43.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.247015+0000", 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022224899999999999 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.247238+0000", 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041552599999999999 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.135 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.247653+0000", 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00072857499999999997 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.248382+0000", 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049236099999999997 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.248874+0000", 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7076000000000001e-05 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.136 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.248921+0000", 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0321000000000002e-05 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.248961+0000", 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0021000089999999999 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:531 4.7 4:e622f76a:gc::gc.20:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.273806+0000", 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.781497261, 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026768669999999999, 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 531 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.138 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.273806+0000", 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.273806+0000", 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.273805+0000", 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9650000000000008e-06 2024-02-18T00:32:43.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.273813+0000", 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4484999999999999e-05 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.273827+0000", 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7390000000000002e-06 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.273834+0000", 2024-02-18T00:32:43.140 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00144406 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.275278+0000", 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9111999999999997e-05 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.275357+0000", 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00018043200000000001 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.141 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,5", 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.275538+0000", 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00072274299999999997 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276260+0000", 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0790000000000002e-06 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276265+0000", 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013280099999999999 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276397+0000", 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8693999999999998e-05 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276446+0000", 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7178999999999998e-05 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.276483+0000", 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026768669999999999 2024-02-18T00:32:43.143 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24347.0:549 4.7 4:e622f76a:gc::gc.20:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected e64)", 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.278471+0000", 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.77683279199999, 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028549040000000001, 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24347", 2024-02-18T00:32:43.144 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2522576766", 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 549 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278471+0000", 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278471+0000", 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.145 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278470+0000", 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6730000000000003e-06 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278476+0000", 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4011e-05 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278490+0000", 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7449999999999998e-06 2024-02-18T00:32:43.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278497+0000", 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9990999999999997e-05 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278537+0000", 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3110999999999998e-05 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278610+0000", 2024-02-18T00:32:43.147 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000170061 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,5", 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.278780+0000", 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00053726000000000002 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.279318+0000", 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000118139 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:32:43.148 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.279436+0000", 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0018216319999999999 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.281257+0000", 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4993999999999997e-05 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.281292+0000", 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3371000000000003e-05 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.149 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.281326+0000", 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0028549040000000001 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:609 4.d e64/58 4:b72b1478:::obj_delete_at_hint.0000000033:head v 64'6, mlcod=64'4)", 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.295380+0000", 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.75992359700001, 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024581939999999999, 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.150 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295380+0000", 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295380+0000", 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295379+0000", 2024-02-18T00:32:43.151 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0949999999999996e-06 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295387+0000", 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3885e-05 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295401+0000", 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0079999999999999e-06 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.152 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295407+0000", 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9225999999999999e-05 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295456+0000", 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0319999999999999e-05 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.295477+0000", 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023238159999999998 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.153 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297800+0000", 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7746000000000001e-05 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297838+0000", 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024581939999999999 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:619 4.a e64/58 4:5395e6f3:lc::lc.4:head v 64'10, mlcod=64'8)", 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.297194+0000", 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.758109158, 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003362819, 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297194+0000", 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.155 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297194+0000", 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297194+0000", 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2389999999999999e-06 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297200+0000", 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0047000000000001e-05 2024-02-18T00:32:43.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297210+0000", 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4619999999999998e-06 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297214+0000", 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5822999999999999e-05 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297250+0000", 2024-02-18T00:32:43.157 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1477999999999999e-05 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297262+0000", 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032584210000000001 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.300520+0000", 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7107999999999998e-05 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.158 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.300557+0000", 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003362819 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:620 4.1b e64/58 4:db9d130d:gc::gc.24:head v 64'6, mlcod=64'4)", 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.297976+0000", 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.75732767599999, 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053279290000000003, 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.159 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297976+0000", 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297976+0000", 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297975+0000", 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6179999999999997e-06 2024-02-18T00:32:43.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297983+0000", 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1453e-05 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.297994+0000", 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8749999999999997e-06 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298000+0000", 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9978000000000002e-05 2024-02-18T00:32:43.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298050+0000", 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1749999999999999e-05 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298062+0000", 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052104819999999998 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.303272+0000", 2024-02-18T00:32:43.162 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1463e-05 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.303304+0000", 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053279290000000003 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:621 4.d e64/58 4:b72b1478:::obj_delete_at_hint.0000000033:head v 64'7, mlcod=64'5)", 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.298879+0000", 2024-02-18T00:32:43.163 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.75642424899999, 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0044331980000000002, 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298879+0000", 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298879+0000", 2024-02-18T00:32:43.164 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298879+0000", 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4340000000000003e-06 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298886+0000", 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4700000000000008e-06 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.165 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298896+0000", 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7209999999999997e-06 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298902+0000", 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4515999999999998e-05 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298957+0000", 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4408000000000001e-05 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.166 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.298981+0000", 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043038099999999999 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.303285+0000", 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7529999999999999e-05 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.303313+0000", 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0044331980000000002 2024-02-18T00:32:43.167 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:650 4.1f e64/58 4:ffe20264:lc::lc.0:head v 64'6, mlcod=64'4)", 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.312789+0000", 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.742514731, 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002633135, 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.168 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312789+0000", 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312789+0000", 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312788+0000", 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4239999999999998e-06 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.169 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312795+0000", 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5939999999999996e-06 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312802+0000", 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1309999999999999e-06 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312806+0000", 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0417999999999998e-05 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312847+0000", 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3773000000000001e-05 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.312870+0000", 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025242609999999999 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.315395+0000", 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7321000000000002e-05 2024-02-18T00:32:43.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.315422+0000", 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002633135 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24347.0:652 4.1f e64/58 4:fde4dd55:gc::gc.26:head v 64'7, mlcod=64'4)", 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:29:22.313545+0000", 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 200.741758463, 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020144220000000001, 2024-02-18T00:32:43.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313545+0000", 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313545+0000", 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.173 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313545+0000", 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3600000000000004e-06 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313550+0000", 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1099999999999992e-06 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313559+0000", 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8890000000000004e-06 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313563+0000", 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9014000000000003e-05 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313602+0000", 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4229000000000001e-05 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.313626+0000", 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0019126410000000001 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.315539+0000", 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0778e-05 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:29:22.315560+0000", 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020144220000000001 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:32:43.176 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:32:43.177 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:32:43.177 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:43.209 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:43.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:32:43.231 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:43.233 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:43.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:43.334 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:43.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:41.658335+0000 osd.7 (osd.7) 61 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:32:43.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:41.659217+0000 osd.7 (osd.7) 62 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:32:43.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:41.762662+0000 mgr.y (mgr.24454) 139 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:43.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:42.032841+0000 osd.4 (osd.4) 71 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:32:43.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:42.033756+0000 osd.4 (osd.4) 72 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:32:43.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:42.461689+0000 osd.2 (osd.2) 57 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:32:43.362 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:42.462768+0000 osd.2 (osd.2) 58 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:32:43.363 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:43.384 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:43.386 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:43.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:43.487 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:43.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:41.658335+0000 osd.7 (osd.7) 61 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:41.659217+0000 osd.7 (osd.7) 62 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:41.762662+0000 mgr.y (mgr.24454) 139 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:42.032841+0000 osd.4 (osd.4) 71 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:42.033756+0000 osd.4 (osd.4) 72 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:42.461689+0000 osd.2 (osd.2) 57 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:42.462768+0000 osd.2 (osd.2) 58 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:41.658335+0000 osd.7 (osd.7) 61 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:41.659217+0000 osd.7 (osd.7) 62 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:32:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:41.762662+0000 mgr.y (mgr.24454) 139 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:42.032841+0000 osd.4 (osd.4) 71 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:32:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:42.033756+0000 osd.4 (osd.4) 72 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:32:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:42.461689+0000 osd.2 (osd.2) 57 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:32:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:42.462768+0000 osd.2 (osd.2) 58 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:32:43.518 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:43.538 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:43.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:43.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:43.642 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:43.671 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:43.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:43.693 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:43.694 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:43.727 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:43.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:43.828 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:44.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.016905+0000 osd.4 (osd.4) 73 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:32:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:44.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-18T00:32:44.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:44.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:44.291 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:44.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.018836+0000 osd.4 (osd.4) 74 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:32:44.461 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.016905+0000 osd.4 (osd.4) 73 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:32:44.461 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.018836+0000 osd.4 (osd.4) 74 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:32:44.474 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:44.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:44.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:44.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.016905+0000 osd.4 (osd.4) 73 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:32:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.018836+0000 osd.4 (osd.4) 74 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:32:44.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:44.610 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:45.048 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:45.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:45.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:45.152 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:45.184 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:45.206 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:45.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:45.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:45.310 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.644010+0000 osd.7 (osd.7) 63 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.644917+0000 osd.7 (osd.7) 64 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.762984+0000 mgr.y (mgr.24454) 140 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.770762+0000 osd.1 (osd.1) 73 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.773797+0000 osd.1 (osd.1) 74 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.805788+0000 osd.3 (osd.3) 81 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.807048+0000 osd.3 (osd.3) 82 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.910512+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.913519+0000 osd.5 (osd.5) 62 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:32:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.966096+0000 osd.4 (osd.4) 75 : cluster [DBG] 4.c scrub starts 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:43.969563+0000 osd.4 (osd.4) 76 : cluster [DBG] 4.c scrub ok 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.644010+0000 osd.7 (osd.7) 63 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.644917+0000 osd.7 (osd.7) 64 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.762984+0000 mgr.y (mgr.24454) 140 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.770762+0000 osd.1 (osd.1) 73 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.773797+0000 osd.1 (osd.1) 74 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.805788+0000 osd.3 (osd.3) 81 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.807048+0000 osd.3 (osd.3) 82 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.910512+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.913519+0000 osd.5 (osd.5) 62 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:32:45.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.966096+0000 osd.4 (osd.4) 75 : cluster [DBG] 4.c scrub starts 2024-02-18T00:32:45.339 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:43.969563+0000 osd.4 (osd.4) 76 : cluster [DBG] 4.c scrub ok 2024-02-18T00:32:45.339 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:45.367 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:45.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:45.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:45.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:45.502 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:45.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.644010+0000 osd.7 (osd.7) 63 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.644917+0000 osd.7 (osd.7) 64 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.762984+0000 mgr.y (mgr.24454) 140 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.770762+0000 osd.1 (osd.1) 73 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.773797+0000 osd.1 (osd.1) 74 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.805788+0000 osd.3 (osd.3) 81 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.807048+0000 osd.3 (osd.3) 82 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.910512+0000 osd.5 (osd.5) 61 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.913519+0000 osd.5 (osd.5) 62 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.966096+0000 osd.4 (osd.4) 75 : cluster [DBG] 4.c scrub starts 2024-02-18T00:32:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:43.969563+0000 osd.4 (osd.4) 76 : cluster [DBG] 4.c scrub ok 2024-02-18T00:32:45.523 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:45.526 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:45.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:45.627 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:45.680 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:45.683 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:45.684 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:45.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:32:45.814 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:45.880 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:45.887 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:32:45.890 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:45.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:32:45.991 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:46.023 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:46.045 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:46.047 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:46.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:46.081 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:44.351609+0000 osd.6 (osd.6) 53 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:32:46.081 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:44.353563+0000 osd.6 (osd.6) 54 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:32:46.149 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:46.179 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:46.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:44.351609+0000 osd.6 (osd.6) 53 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:32:46.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:44.353563+0000 osd.6 (osd.6) 54 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:32:46.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:44.748491+0000 osd.1 (osd.1) 75 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:32:46.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:44.751531+0000 osd.1 (osd.1) 76 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:32:46.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:44.766589+0000 osd.3 (osd.3) 83 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:44.768878+0000 osd.3 (osd.3) 84 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:45.000538+0000 osd.0 (osd.0) 61 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:45.002399+0000 osd.0 (osd.0) 62 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:44.748491+0000 osd.1 (osd.1) 75 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:44.751531+0000 osd.1 (osd.1) 76 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:44.766589+0000 osd.3 (osd.3) 83 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:44.768878+0000 osd.3 (osd.3) 84 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:45.000538+0000 osd.0 (osd.0) 61 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:32:46.462 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:45.002399+0000 osd.0 (osd.0) 62 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:44.351609+0000 osd.6 (osd.6) 53 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:44.353563+0000 osd.6 (osd.6) 54 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:44.748491+0000 osd.1 (osd.1) 75 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:44.751531+0000 osd.1 (osd.1) 76 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:44.766589+0000 osd.3 (osd.3) 83 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:44.768878+0000 osd.3 (osd.3) 84 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:45.000538+0000 osd.0 (osd.0) 61 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:32:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:45.002399+0000 osd.0 (osd.0) 62 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:32:46.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.694+0000 7f3e687df700 1 -- 172.21.15.59:0/3768530052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fd410 msgr2=0x7f3e600f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.694+0000 7f3e687df700 1 --2- 172.21.15.59:0/3768530052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fd410 0x7f3e600f6c40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f3e54004870 tx=0x7f3e54009cb0).stop 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.694+0000 7f3e687df700 1 -- 172.21.15.59:0/3768530052 shutdown_connections 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.694+0000 7f3e687df700 1 --2- 172.21.15.59:0/3768530052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e600f7320 0x7f3e60103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.694+0000 7f3e687df700 1 --2- 172.21.15.59:0/3768530052 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600fcab0 0x7f3e600fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.694+0000 7f3e687df700 1 --2- 172.21.15.59:0/3768530052 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fd410 0x7f3e600f6c40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.694+0000 7f3e687df700 1 -- 172.21.15.59:0/3768530052 >> 172.21.15.59:0/3768530052 conn(0x7f3e600f26e0 msgr2=0x7f3e600f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 -- 172.21.15.59:0/3768530052 shutdown_connections 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 -- 172.21.15.59:0/3768530052 wait complete. 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 librados: starting msgr at 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 librados: starting objecter 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 Processor -- start 2024-02-18T00:32:46.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 -- start start 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 librados: setting wanted keys 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.695+0000 7f3e687df700 1 librados: calling monclient init 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e687df700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600f7320 0x7f3e60106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e687df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fcab0 0x7f3e60107470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e687df700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e600fd410 0x7f3e60107960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e687df700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e60116600 con 0x7f3e600fcab0 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e687df700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e60116740 con 0x7f3e600fd410 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e687df700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e601168b0 con 0x7f3e600f7320 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600f7320 0x7f3e60106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e65d7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fcab0 0x7f3e60107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e65d7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fcab0 0x7f3e60107470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36312/0 (socket says 172.21.15.59:36312) 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600f7320 0x7f3e60106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51020/0 (socket says 172.21.15.59:51020) 2024-02-18T00:32:46.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e65d7a700 1 -- 172.21.15.59:0/3884895608 learned_addr learned my addr 172.21.15.59:0/3884895608 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 -- 172.21.15.59:0/3884895608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e600fd410 msgr2=0x7f3e60107960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e600fd410 0x7f3e60107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 -- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fcab0 msgr2=0x7f3e60107470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fcab0 0x7f3e60107470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 -- 172.21.15.59:0/3884895608 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54005040 con 0x7f3e600f7320 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e65d7a700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fcab0 0x7f3e60107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e6657b700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600f7320 0x7f3e60106f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f3e5000e8f0 tx=0x7f3e5000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e5f7fe700 10 client.?.objecter ms_handle_connect 0x7f3e600f7320 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e5f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e5f7fe700 1 -- 172.21.15.59:0/3884895608 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50004620 con 0x7f3e600f7320 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e5f7fe700 1 -- 172.21.15.59:0/3884895608 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e5000f040 con 0x7f3e600f7320 2024-02-18T00:32:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.696+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e60116b90 con 0x7f3e600f7320 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.697+0000 7f3e5f7fe700 1 -- 172.21.15.59:0/3884895608 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500146f0 con 0x7f3e600f7320 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.697+0000 7f3e687df700 10 client.24698.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.697+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e60117150 con 0x7f3e600f7320 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.697+0000 7f3e687df700 1 librados: init done 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.697+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6005f950 con 0x7f3e600f7320 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e5f7fe700 1 -- 172.21.15.59:0/3884895608 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f3e5000c3d0 con 0x7f3e600f7320 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e5f7fe700 1 --2- 172.21.15.59:0/3884895608 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3e48062c90 0x7f3e48065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e5f7fe700 1 -- 172.21.15.59:0/3884895608 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(101..101 src has 44..101) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f3e5007c520 con 0x7f3e600f7320 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e5f7fe700 10 client.24698.objecter ms_dispatch 0x7f3e60001550 osd_map(101..101 src has 44..101) v4 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e5f7fe700 3 client.24698.objecter handle_osd_map got epochs [101,101] > 0 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e5f7fe700 3 client.24698.objecter handle_osd_map decoding full epoch 101 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e65d7a700 1 --2- 172.21.15.59:0/3884895608 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3e48062c90 0x7f3e48065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.698+0000 7f3e5f7fe700 20 client.24698.objecter dump_active .. 0 homeless 2024-02-18T00:32:46.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.699+0000 7f3e5f7fe700 10 client.24698.objecter ms_handle_connect 0x7f3e48062c90 2024-02-18T00:32:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.699+0000 7f3e65d7a700 1 --2- 172.21.15.59:0/3884895608 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3e48062c90 0x7f3e48065150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f3e540043d0 tx=0x7f3e5400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.702+0000 7f3e5f7fe700 1 -- 172.21.15.59:0/3884895608 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3e50050280 con 0x7f3e600f7320 2024-02-18T00:32:46.769 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:46.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:46.906+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f3e60061550 con 0x7f3e600f7320 2024-02-18T00:32:47.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.087+0000 7f3e5f7fe700 1 -- 172.21.15.59:0/3884895608 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v102) v1 ==== 101+0+0 (secure 0 0 0) 0x7f3e5004fe70 con 0x7f3e600f7320 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 10 librados: watch_flush enter 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 10 librados: watch_flush exit 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 20 client.24698.objecter shutdown clearing up homeless session... 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 10 client.24698.objecter successfully canceled tick 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3e48062c90 msgr2=0x7f3e48065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 --2- 172.21.15.59:0/3884895608 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3e48062c90 0x7f3e48065150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f3e540043d0 tx=0x7f3e5400b040).stop 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600f7320 msgr2=0x7f3e60106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600f7320 0x7f3e60106f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f3e5000e8f0 tx=0x7f3e5000b040).stop 2024-02-18T00:32:47.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 shutdown_connections 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 --2- 172.21.15.59:0/3884895608 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3e48062c90 0x7f3e48065150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e600fd410 0x7f3e60107960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e600f7320 0x7f3e60106f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 --2- 172.21.15.59:0/3884895608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e600fcab0 0x7f3e60107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 >> 172.21.15.59:0/3884895608 conn(0x7f3e600f26e0 msgr2=0x7f3e600f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 shutdown_connections 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 -- 172.21.15.59:0/3884895608 wait complete. 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:47.090+0000 7f3e687df700 1 librados: shutdown 2024-02-18T00:32:47.093 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:45.393343+0000 osd.6 (osd.6) 55 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:45.395079+0000 osd.6 (osd.6) 56 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:45.748818+0000 osd.1 (osd.1) 77 : cluster [DBG] 5.c scrub starts 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:45.751867+0000 osd.1 (osd.1) 78 : cluster [DBG] 5.c scrub ok 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:45.763569+0000 mgr.y (mgr.24454) 141 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:46.412248+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.d scrub starts 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:46.413890+0000 osd.2 (osd.2) 60 : cluster [DBG] 5.d scrub ok 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:46.907617+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.59:0/3884895608' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:46.907952+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:45.393343+0000 osd.6 (osd.6) 55 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:45.395079+0000 osd.6 (osd.6) 56 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:45.748818+0000 osd.1 (osd.1) 77 : cluster [DBG] 5.c scrub starts 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:45.751867+0000 osd.1 (osd.1) 78 : cluster [DBG] 5.c scrub ok 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:45.763569+0000 mgr.y (mgr.24454) 141 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:46.412248+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.d scrub starts 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:46.413890+0000 osd.2 (osd.2) 60 : cluster [DBG] 5.d scrub ok 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:46.907617+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.59:0/3884895608' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:46.907952+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:45.393343+0000 osd.6 (osd.6) 55 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:45.395079+0000 osd.6 (osd.6) 56 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:45.748818+0000 osd.1 (osd.1) 77 : cluster [DBG] 5.c scrub starts 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:45.751867+0000 osd.1 (osd.1) 78 : cluster [DBG] 5.c scrub ok 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:45.763569+0000 mgr.y (mgr.24454) 141 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:46.412248+0000 osd.2 (osd.2) 59 : cluster [DBG] 5.d scrub starts 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:46.413890+0000 osd.2 (osd.2) 60 : cluster [DBG] 5.d scrub ok 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:46.907617+0000 mon.c (mon.2) 30 : audit [INF] from='client.? 172.21.15.59:0/3884895608' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:46.907952+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:32:47.597 INFO:tasks.workunit.client.0.smithi059.stderr:Updating files: 90% (10969/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-02-18T00:32:48.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:46.708203+0000 osd.3 (osd.3) 85 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:32:48.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:46.709305+0000 osd.3 (osd.3) 86 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:32:48.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:46.875441+0000 osd.5 (osd.5) 63 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:46.877247+0000 osd.5 (osd.5) 64 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.079808+0000 mon.a (mon.0) 784 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.079840+0000 mon.a (mon.0) 785 : cluster [INF] Cluster is now healthy 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:47.087555+0000 mon.a (mon.0) 786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.087617+0000 mon.a (mon.0) 787 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.564519+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.565445+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:46.708203+0000 osd.3 (osd.3) 85 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:46.709305+0000 osd.3 (osd.3) 86 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:32:48.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:46.875441+0000 osd.5 (osd.5) 63 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:32:48.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:46.877247+0000 osd.5 (osd.5) 64 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:32:48.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.079808+0000 mon.a (mon.0) 784 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:32:48.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.079840+0000 mon.a (mon.0) 785 : cluster [INF] Cluster is now healthy 2024-02-18T00:32:48.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:47.087555+0000 mon.a (mon.0) 786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:32:48.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.087617+0000 mon.a (mon.0) 787 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-18T00:32:48.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.564519+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:32:48.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.565445+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:46.708203+0000 osd.3 (osd.3) 85 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:46.709305+0000 osd.3 (osd.3) 86 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:46.875441+0000 osd.5 (osd.5) 63 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:46.877247+0000 osd.5 (osd.5) 64 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.079808+0000 mon.a (mon.0) 784 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.079840+0000 mon.a (mon.0) 785 : cluster [INF] Cluster is now healthy 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:47.087555+0000 mon.a (mon.0) 786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:32:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.087617+0000 mon.a (mon.0) 787 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-18T00:32:48.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.564519+0000 osd.7 (osd.7) 65 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:32:48.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.565445+0000 osd.7 (osd.7) 66 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:32:48.688 INFO:tasks.workunit.client.0.smithi059.stderr:Note: switching to '81bd20d634209c7cb82c18be12b4b5a05643ebf1'. 2024-02-18T00:32:48.688 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-02-18T00:32:48.688 INFO:tasks.workunit.client.0.smithi059.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-18T00:32:48.688 INFO:tasks.workunit.client.0.smithi059.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-18T00:32:48.688 INFO:tasks.workunit.client.0.smithi059.stderr:state without impacting any branches by switching back to a branch. 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: git switch -c 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr:Or undo this operation with: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: git switch - 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-02-18T00:32:48.689 INFO:tasks.workunit.client.0.smithi059.stderr:HEAD is now at 81bd20d6342 Merge pull request #55523 from ljflores/wip-tracker-64343-pacific 2024-02-18T00:32:48.695 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-18T00:32:48.761 INFO:tasks.workunit.client.0.smithi059.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-18T00:32:48.764 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-18T00:32:48.764 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-18T00:32:49.330 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:49.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:49.336 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:49.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.763880+0000 mgr.y (mgr.24454) 142 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:49.364 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.912115+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.f deep-scrub starts 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:47.915088+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.f deep-scrub ok 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:48.393384+0000 osd.2 (osd.2) 61 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:48.395157+0000 osd.2 (osd.2) 62 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.763880+0000 mgr.y (mgr.24454) 142 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.912115+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.f deep-scrub starts 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:47.915088+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.f deep-scrub ok 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:48.393384+0000 osd.2 (osd.2) 61 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:32:49.365 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:48.395157+0000 osd.2 (osd.2) 62 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:32:49.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:49.470 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.763880+0000 mgr.y (mgr.24454) 142 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:49.628 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.912115+0000 osd.5 (osd.5) 65 : cluster [DBG] 5.f deep-scrub starts 2024-02-18T00:32:49.632 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:47.915088+0000 osd.5 (osd.5) 66 : cluster [DBG] 5.f deep-scrub ok 2024-02-18T00:32:49.632 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:48.393384+0000 osd.2 (osd.2) 61 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:32:49.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:48.395157+0000 osd.2 (osd.2) 62 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:32:49.669 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:49.672 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:49.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:49.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:49.802 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:50.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 -- 172.21.15.59:0/3300417107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c0f8c80 msgr2=0x7f0f9c0f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/3300417107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c0f8c80 0x7f0f9c0f9060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f0f900047b0 tx=0x7f0f90009cb0).stop 2024-02-18T00:32:50.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 -- 172.21.15.59:0/3300417107 shutdown_connections 2024-02-18T00:32:50.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/3300417107 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f9c1021e0 0x7f0f9c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/3300417107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c0f8c80 0x7f0f9c0f9060 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/3300417107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c0f9550 0x7f0f9c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 -- 172.21.15.59:0/3300417107 >> 172.21.15.59:0/3300417107 conn(0x7f0f9c0f26f0 msgr2=0x7f0f9c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:50.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 -- 172.21.15.59:0/3300417107 shutdown_connections 2024-02-18T00:32:50.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.946+0000 7f0fa2e52700 1 -- 172.21.15.59:0/3300417107 wait complete. 2024-02-18T00:32:50.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 librados: starting msgr at 2024-02-18T00:32:50.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 librados: starting objecter 2024-02-18T00:32:50.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 Processor -- start 2024-02-18T00:32:50.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 -- start start 2024-02-18T00:32:50.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 librados: setting wanted keys 2024-02-18T00:32:50.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 librados: calling monclient init 2024-02-18T00:32:50.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f9c0f8c80 0x7f0f9c110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c0f9550 0x7f0f9c1107f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c1021e0 0x7f0f9c10a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c10a9d0 con 0x7f0f9c1021e0 2024-02-18T00:32:50.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c10ab40 con 0x7f0f9c0f9550 2024-02-18T00:32:50.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa2e52700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c10acb0 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa0bee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f9c0f8c80 0x7f0f9c110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa0bee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f9c0f8c80 0x7f0f9c110300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55072/0 (socket says 172.21.15.59:55072) 2024-02-18T00:32:50.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa13ef700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c1021e0 0x7f0f9c10a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0fa0bee700 1 -- 172.21.15.59:0/2531141525 learned_addr learned my addr 172.21.15.59:0/2531141525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:50.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.947+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c0f9550 0x7f0f9c1107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa0bee700 1 -- 172.21.15.59:0/2531141525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c0f9550 msgr2=0x7f0f9c1107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c0f9550 0x7f0f9c1107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa0bee700 1 -- 172.21.15.59:0/2531141525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c1021e0 msgr2=0x7f0f9c10a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c1021e0 0x7f0f9c10a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa0bee700 1 -- 172.21.15.59:0/2531141525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940097c0 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f9c0f9550 0x7f0f9c1107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa13ef700 1 --2- 172.21.15.59:0/2531141525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f9c1021e0 0x7f0f9c10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f9c0f8c80 0x7f0f9c110300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0f90009b10 tx=0x7f0f90009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0f99ffb700 10 client.?.objecter ms_handle_connect 0x7f0f9c0f8c80 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0f99ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f90018070 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f90013e70 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f90005040 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9001ddb0 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa2e52700 10 client.24704.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:50.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c10b1a0 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.948+0000 7f0fa2e52700 1 librados: init done 2024-02-18T00:32:50.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.949+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f0f9c10b3c0 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.950+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0f9001d420 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.950+0000 7f0f99ffb700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f8c062c90 0x7f0f8c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(102..102 src has 44..102) v4 ==== 6172+0+0 (secure 0 0 0) 0x7f0f9007e5c0 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f8c062c90 0x7f0f8c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 osd_map(102..102 src has 44..102) v4 2024-02-18T00:32:50.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 3 client.24704.objecter handle_osd_map got epochs [102,102] > 0 2024-02-18T00:32:50.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 3 client.24704.objecter handle_osd_map decoding full epoch 102 2024-02-18T00:32:50.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 20 client.24704.objecter dump_active .. 0 homeless 2024-02-18T00:32:50.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v102) v1 ==== 54+0+16 (secure 0 0 0) 0x7f0f9007eaf0 con 0x7f0f9c0f8c80 2024-02-18T00:32:50.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f8c062c90 2024-02-18T00:32:50.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f8c062c90 0x7f0f8c065150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0f9c10bc80 tx=0x7f0f8800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.234 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:32:50.234 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:32:50.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 1 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0f9c0fe5b0 0x7f0f9c0fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 1 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 0 1 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c05f990 osd=0 4 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 10 client.24704.objecter _send_command 1 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c05ee80 con 0x7f0f9c0fe5b0 2024-02-18T00:32:50.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa13ef700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0f9c0fe5b0 0x7f0f9c0fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c0fe5b0 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.951+0000 7f0fa13ef700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0f9c0fe5b0 0x7f0f9c0fe990 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.952+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9c05ee80 con 0x7f0f9c0fe5b0 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.952+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.952+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 1 = system:0 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.952+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c05f990 osd=0 4 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.952+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 0 1 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 2 2024-02-18T00:32:50.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 2 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c05f990 osd=0 3 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 0 2 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c05f990 osd=0 4 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 10 client.24704.objecter _send_command 2 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.967+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c107130 con 0x7f0f9c0fe5b0 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c107130 con 0x7f0f9c0fe5b0 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c05f990 osd=0 4 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 0 2 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 3 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6805/26279110 conn(0x7f0f9c0598f0 0x7f0f9c059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 3 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 1 3 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c028540 osd=1 4 2024-02-18T00:32:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 10 client.24704.objecter _send_command 3 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c0286b0 con 0x7f0f9c0598f0 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.969+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6805/26279110 conn(0x7f0f9c0598f0 0x7f0f9c059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.970+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c0598f0 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.970+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6805/26279110 conn(0x7f0f9c0598f0 0x7f0f9c059d10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.970+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9c0286b0 con 0x7f0f9c0598f0 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.970+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.970+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 3 = system:0 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.970+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c028540 osd=1 4 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.970+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 1 3 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 4 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 4 2024-02-18T00:32:50.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c028540 osd=1 3 2024-02-18T00:32:50.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 1 4 2024-02-18T00:32:50.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c028540 osd=1 4 2024-02-18T00:32:50.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 10 client.24704.objecter _send_command 4 2024-02-18T00:32:50.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.986+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c059360 con 0x7f0f9c0598f0 2024-02-18T00:32:50.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c059360 con 0x7f0f9c0598f0 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c028540 osd=1 4 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 1 4 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 5 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0f9c0280a0 0x7f0f9c11e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 5 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 2 5 2024-02-18T00:32:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c059770 osd=2 4 2024-02-18T00:32:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 10 client.24704.objecter _send_command 5 2024-02-18T00:32:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c120f70 con 0x7f0f9c0280a0 2024-02-18T00:32:50.246 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:32:50.246 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:32:50.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0f9c0280a0 0x7f0f9c11e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c0280a0 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.988+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0f9c0280a0 0x7f0f9c11e320 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.989+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9c120f70 con 0x7f0f9c0280a0 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.989+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.989+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 5 = system:0 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.989+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c059770 osd=2 4 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:49.989+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 2 5 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.004+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 6 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 6 2024-02-18T00:32:50.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c059770 osd=2 3 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 2 6 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c059770 osd=2 4 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 10 client.24704.objecter _send_command 6 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.005+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c027600 con 0x7f0f9c0280a0 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c027600 con 0x7f0f9c0280a0 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c059770 osd=2 4 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 2 6 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 7 2024-02-18T00:32:50.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0f9c11f990 0x7f0f9c11fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 7 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 3 7 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c11b960 osd=3 4 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 10 client.24704.objecter _send_command 7 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.006+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c11c030 con 0x7f0f9c11f990 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0fa13ef700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0f9c11f990 0x7f0f9c11fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c11f990 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0fa13ef700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0f9c11f990 0x7f0f9c11fe00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9c11c030 con 0x7f0f9c11f990 2024-02-18T00:32:50.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 7 = system:0 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c11b960 osd=3 4 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.007+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 3 7 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 8 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 8 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c11b960 osd=3 3 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 3 8 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c11b960 osd=3 4 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 10 client.24704.objecter _send_command 8 2024-02-18T00:32:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.022+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c120ac0 con 0x7f0f9c11f990 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c120ac0 con 0x7f0f9c11f990 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c11b960 osd=3 4 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 3 8 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 9 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0f9c1230f0 0x7f0f9c1254e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 9 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 4 9 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c1202f0 osd=4 4 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 10 client.24704.objecter _send_command 9 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c120a10 con 0x7f0f9c1230f0 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.024+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0f9c1230f0 0x7f0f9c1254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.025+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0f9c1230f0 0x7f0f9c1254e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.025+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c1230f0 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.025+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9c120a10 con 0x7f0f9c1230f0 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.025+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.025+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 9 = system:0 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.025+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c1202f0 osd=4 4 2024-02-18T00:32:50.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.025+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 4 9 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 10 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 10 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c1202f0 osd=4 3 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 4 10 2024-02-18T00:32:50.253 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c1202f0 osd=4 4 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 10 client.24704.objecter _send_command 10 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.040+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c120a10 con 0x7f0f9c1230f0 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c120a10 con 0x7f0f9c1230f0 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c1202f0 osd=4 4 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 4 10 2024-02-18T00:32:50.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 11 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0f9c125f20 0x7f0f9c126340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 11 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 5 11 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c120a10 osd=5 4 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 10 client.24704.objecter _send_command 11 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c127f90 con 0x7f0f9c125f20 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.042+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0f9c125f20 0x7f0f9c126340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.043+0000 7f0f9bfff700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0f9c125f20 0x7f0f9c126340 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.043+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c125f20 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.044+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9c127f90 con 0x7f0f9c125f20 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.044+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.044+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 11 = system:0 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.044+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c120a10 osd=5 4 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.044+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 5 11 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 12 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 12 2024-02-18T00:32:50.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c120a10 osd=5 3 2024-02-18T00:32:50.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 5 12 2024-02-18T00:32:50.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c120a10 osd=5 4 2024-02-18T00:32:50.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 10 client.24704.objecter _send_command 12 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.058+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c127f90 con 0x7f0f9c125f20 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c127f90 con 0x7f0f9c125f20 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c120a10 osd=5 4 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 5 12 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 13 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0f9c12a710 0x7f0f9c12ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 13 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 6 13 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c127f90 osd=6 4 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 10 client.24704.objecter _send_command 13 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c12b1c0 con 0x7f0f9c12a710 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.060+0000 7f0fa13ef700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0f9c12a710 0x7f0f9c12ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.061+0000 7f0fa13ef700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0f9c12a710 0x7f0f9c12ab50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.061+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c12a710 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.062+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9400f040 con 0x7f0f9c12a710 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.062+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.062+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 13 = system:0 2024-02-18T00:32:50.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.062+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c127f90 osd=6 4 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.062+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 6 13 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 14 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 14 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c127f90 osd=6 3 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 6 14 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c127f90 osd=6 4 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 10 client.24704.objecter _send_command 14 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.076+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c12d450 con 0x7f0f9c12a710 2024-02-18T00:32:50.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.078+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c12d450 con 0x7f0f9c12a710 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.078+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.078+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.078+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c127f90 osd=6 4 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.078+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 6 14 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.078+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.078+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 15 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6812/53025171 conn(0x7f0f9c12db00 0x7f0f9c12df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 15 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 7 15 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c12d450 osd=7 4 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 10 client.24704.objecter _send_command 15 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f9c131810 con 0x7f0f9c12db00 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6812/53025171 conn(0x7f0f9c12db00 0x7f0f9c12df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0fa0bee700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6812/53025171 conn(0x7f0f9c12db00 0x7f0f9c12df60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:50.262 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.079+0000 7f0f99ffb700 10 client.24704.objecter ms_handle_connect 0x7f0f9c12db00 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.080+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0f9c131810 con 0x7f0f9c12db00 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.080+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.080+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 15 = system:0 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.080+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c12d450 osd=7 4 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.080+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 7 15 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 10 client.24704.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:32:50.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign -1 16 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 20 client.24704.objecter _get_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_remove -1 16 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 20 client.24704.objecter get_session s=0x7f0f9c12d450 osd=7 3 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 15 client.24704.objecter _session_command_op_assign 7 16 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c12d450 osd=7 4 2024-02-18T00:32:50.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 10 client.24704.objecter _send_command 16 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.096+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0f9c130690 con 0x7f0f9c12db00 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0f99ffb700 1 -- 172.21.15.59:0/2531141525 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0f9c130690 con 0x7f0f9c12db00 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0f99ffb700 10 client.24704.objecter ms_dispatch 0x7f0f9c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0f99ffb700 10 client.24704.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0f99ffb700 20 client.24704.objecter put_session s=0x7f0f9c12d450 osd=7 4 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0f99ffb700 15 client.24704.objecter _session_command_op_remove 7 16 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 librados: watch_flush enter 2024-02-18T00:32:50.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 librados: watch_flush exit 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.0 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0f9c0fe5b0 msgr2=0x7f0f9c0fe990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0f9c0fe5b0 0x7f0f9c0fe990 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c05f990 osd=0 1 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.1 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6805/26279110 conn(0x7f0f9c0598f0 msgr2=0x7f0f9c059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6805/26279110 conn(0x7f0f9c0598f0 0x7f0f9c059d10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c028540 osd=1 1 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.2 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0f9c0280a0 msgr2=0x7f0f9c11e320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0f9c0280a0 0x7f0f9c11e320 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c059770 osd=2 1 2024-02-18T00:32:50.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.3 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0f9c11f990 msgr2=0x7f0f9c11fe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0f9c11f990 0x7f0f9c11fe00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa13ef700 1 -- 172.21.15.59:0/2531141525 reap_dead start 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c11b960 osd=3 1 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.4 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0f9c1230f0 msgr2=0x7f0f9c1254e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0f9c1230f0 0x7f0f9c1254e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa13ef700 1 -- 172.21.15.59:0/2531141525 reap_dead start 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c1202f0 osd=4 1 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.5 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0f9c125f20 msgr2=0x7f0f9c126340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0f9c125f20 0x7f0f9c126340 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c120a10 osd=5 1 2024-02-18T00:32:50.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.6 2024-02-18T00:32:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0f9c12a710 msgr2=0x7f0f9c12ab50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0f9c12a710 0x7f0f9c12ab50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c127f90 osd=6 1 2024-02-18T00:32:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter close_session for osd.7 2024-02-18T00:32:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6812/53025171 conn(0x7f0f9c12db00 msgr2=0x7f0f9c12df60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.137:6812/53025171 conn(0x7f0f9c12db00 0x7f0f9c12df60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:50.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter put_session s=0x7f0f9c12d450 osd=7 1 2024-02-18T00:32:50.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 20 client.24704.objecter shutdown clearing up homeless session... 2024-02-18T00:32:50.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 10 client.24704.objecter successfully canceled tick 2024-02-18T00:32:50.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f8c062c90 msgr2=0x7f0f8c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f8c062c90 0x7f0f8c065150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f0f9c10bc80 tx=0x7f0f8800b040).stop 2024-02-18T00:32:50.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f9c0f8c80 msgr2=0x7f0f9c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:50.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.098+0000 7f0fa2e52700 1 --2- 172.21.15.59:0/2531141525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f9c0f8c80 0x7f0f9c110300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0f90009b10 tx=0x7f0f90009cb0).stop 2024-02-18T00:32:50.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.099+0000 7f0fa13ef700 1 -- 172.21.15.59:0/2531141525 reap_dead start 2024-02-18T00:32:50.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.099+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 shutdown_connections 2024-02-18T00:32:50.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.099+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 >> 172.21.15.59:0/2531141525 conn(0x7f0f9c0f26f0 msgr2=0x7f0f9c0f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:50.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.099+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 shutdown_connections 2024-02-18T00:32:50.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.099+0000 7f0fa2e52700 1 -- 172.21.15.59:0/2531141525 wait complete. 2024-02-18T00:32:50.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:50.099+0000 7f0fa2e52700 1 librados: shutdown 2024-02-18T00:32:50.271 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.271 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.271 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.271 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:32:50.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:48.721366+0000 osd.3 (osd.3) 87 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:32:50.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:48.722495+0000 osd.3 (osd.3) 88 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:32:50.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:49.088765+0000 osd.4 (osd.4) 77 : cluster [DBG] 5.e scrub starts 2024-02-18T00:32:50.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:49.092287+0000 osd.4 (osd.4) 78 : cluster [DBG] 5.e scrub ok 2024-02-18T00:32:50.274 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:49.395112+0000 osd.2 (osd.2) 63 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:32:50.274 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:49.398860+0000 osd.2 (osd.2) 64 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:32:50.274 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:49.950879+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.59:0/2531141525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:50.274 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:48.721366+0000 osd.3 (osd.3) 87 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:32:50.275 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:48.722495+0000 osd.3 (osd.3) 88 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:32:50.275 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:49.088765+0000 osd.4 (osd.4) 77 : cluster [DBG] 5.e scrub starts 2024-02-18T00:32:50.275 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:49.092287+0000 osd.4 (osd.4) 78 : cluster [DBG] 5.e scrub ok 2024-02-18T00:32:50.275 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:49.395112+0000 osd.2 (osd.2) 63 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:32:50.275 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:49.398860+0000 osd.2 (osd.2) 64 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:32:50.275 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:49.950879+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.59:0/2531141525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:50.276 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-18T00:32:50.434 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-18T00:32:50.476 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-18T00:32:50.477 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-18T00:32:50.478 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-18T00:32:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:48.721366+0000 osd.3 (osd.3) 87 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:32:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:48.722495+0000 osd.3 (osd.3) 88 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:32:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:49.088765+0000 osd.4 (osd.4) 77 : cluster [DBG] 5.e scrub starts 2024-02-18T00:32:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:49.092287+0000 osd.4 (osd.4) 78 : cluster [DBG] 5.e scrub ok 2024-02-18T00:32:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:49.395112+0000 osd.2 (osd.2) 63 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:32:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:49.398860+0000 osd.2 (osd.2) 64 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:32:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:49.950879+0000 mon.c (mon.2) 31 : audit [DBG] from='client.? 172.21.15.59:0/2531141525' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:32:50.516 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-18T00:32:50.519 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-02-18T00:32:50.519 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-18T00:32:50.578 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-18T00:32:50.579 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-18T00:32:50.579 DEBUG:teuthology.orchestra.run.smithi059:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81bd20d634209c7cb82c18be12b4b5a05643ebf1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-18T00:32:50.714 INFO:tasks.workunit.client.0.smithi059.stderr:+ parallel=1 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' '' = --serial ']' 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ color= 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' -t 1 ']' 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ declare -A pids 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_aio 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_aio' 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_aio 2024-02-18T00:32:50.715 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_aio 2024-02-18T00:32:50.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82872 2024-02-18T00:32:50.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_aio on pid 82872' 2024-02-18T00:32:50.717 INFO:tasks.workunit.client.0.smithi059.stdout:test api_aio on pid 82872 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82872 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_aio_pp 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_aio_pp' 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_aio_pp 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_aio 2024-02-18T00:32:50.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-18T00:32:50.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_aio: /' 2024-02-18T00:32:50.719 INFO:tasks.workunit.client.0.smithi059.stdout:test api_aio_pp on pid 82880 2024-02-18T00:32:50.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_aio_pp 2024-02-18T00:32:50.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82880 2024-02-18T00:32:50.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_aio_pp on pid 82880' 2024-02-18T00:32:50.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82880 2024-02-18T00:32:50.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-18T00:32:50.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_aio_pp 2024-02-18T00:32:50.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-18T00:32:50.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-18T00:32:50.722 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_io 2024-02-18T00:32:50.722 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_io' 2024-02-18T00:32:50.722 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_io 2024-02-18T00:32:50.722 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.724 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_io 2024-02-18T00:32:50.724 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82888 2024-02-18T00:32:50.724 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_io on pid 82888' 2024-02-18T00:32:50.724 INFO:tasks.workunit.client.0.smithi059.stdout:test api_io on pid 82888 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82888 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_io_pp 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_io_pp' 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_io_pp 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_io 2024-02-18T00:32:50.726 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_io.log 2024-02-18T00:32:50.727 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_io: /' 2024-02-18T00:32:50.727 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_io_pp 2024-02-18T00:32:50.727 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82896 2024-02-18T00:32:50.727 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_io_pp on pid 82896' 2024-02-18T00:32:50.727 INFO:tasks.workunit.client.0.smithi059.stdout:test api_io_pp on pid 82896 2024-02-18T00:32:50.728 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82896 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_asio 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_asio' 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_asio 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_io_pp 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_asio 2024-02-18T00:32:50.729 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82904 2024-02-18T00:32:50.730 INFO:tasks.workunit.client.0.smithi059.stdout:test api_asio on pid 82904 2024-02-18T00:32:50.731 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_asio on pid 82904' 2024-02-18T00:32:50.731 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82904 2024-02-18T00:32:50.731 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.731 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-18T00:32:50.731 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.731 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_list 2024-02-18T00:32:50.731 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_list' 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_list 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_asio 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_asio: /' 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_list 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82912 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_list on pid 82912' 2024-02-18T00:32:50.732 INFO:tasks.workunit.client.0.smithi059.stdout:test api_list on pid 82912 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82912 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_lock 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_lock' 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_lock 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_list 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_list.log 2024-02-18T00:32:50.734 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_list: /' 2024-02-18T00:32:50.735 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_lock 2024-02-18T00:32:50.735 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82920 2024-02-18T00:32:50.735 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_lock on pid 82920' 2024-02-18T00:32:50.735 INFO:tasks.workunit.client.0.smithi059.stdout:test api_lock on pid 82920 2024-02-18T00:32:50.736 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82920 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_lock_pp 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_lock_pp' 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_lock_pp 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_lock 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_lock: /' 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_lock_pp 2024-02-18T00:32:50.737 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82928 2024-02-18T00:32:50.738 INFO:tasks.workunit.client.0.smithi059.stdout:test api_lock_pp on pid 82928 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_lock_pp on pid 82928' 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82928 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_misc 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_misc' 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_misc 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.739 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_lock_pp 2024-02-18T00:32:50.740 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-18T00:32:50.740 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-18T00:32:50.740 INFO:tasks.workunit.client.0.smithi059.stdout:test api_misc on pid 82936 2024-02-18T00:32:50.741 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_misc 2024-02-18T00:32:50.741 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82936 2024-02-18T00:32:50.741 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_misc on pid 82936' 2024-02-18T00:32:50.741 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82936 2024-02-18T00:32:50.741 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.741 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.741 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_misc_pp 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_misc_pp' 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_misc_pp 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_misc 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_misc: /' 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_misc_pp 2024-02-18T00:32:50.742 INFO:tasks.workunit.client.0.smithi059.stdout:test api_misc_pp on pid 82944 2024-02-18T00:32:50.743 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82944 2024-02-18T00:32:50.743 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_misc_pp on pid 82944' 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82944 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_tier_pp 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_tier_pp' 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_tier_pp 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_misc_pp 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_tier_pp 2024-02-18T00:32:50.744 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82952 2024-02-18T00:32:50.745 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_tier_pp on pid 82952' 2024-02-18T00:32:50.745 INFO:tasks.workunit.client.0.smithi059.stdout:test api_tier_pp on pid 82952 2024-02-18T00:32:50.746 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82952 2024-02-18T00:32:50.746 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.746 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_pool 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_pool' 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_pool 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_tier_pp 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_pool 2024-02-18T00:32:50.747 INFO:tasks.workunit.client.0.smithi059.stdout:test api_pool on pid 82960 2024-02-18T00:32:50.748 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82960 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_pool on pid 82960' 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82960 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_snapshots 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_snapshots' 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_snapshots 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_pool 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_pool: /' 2024-02-18T00:32:50.749 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_snapshots 2024-02-18T00:32:50.750 INFO:tasks.workunit.client.0.smithi059.stdout:test api_snapshots on pid 82968 2024-02-18T00:32:50.751 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82968 2024-02-18T00:32:50.751 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_snapshots on pid 82968' 2024-02-18T00:32:50.751 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82968 2024-02-18T00:32:50.751 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.751 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.751 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-18T00:32:50.751 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_snapshots_pp 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_snapshots_pp' 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_snapshots_pp 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_snapshots 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_snapshots_pp 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82976 2024-02-18T00:32:50.752 INFO:tasks.workunit.client.0.smithi059.stdout:test api_snapshots_pp on pid 82976 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_snapshots_pp on pid 82976' 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82976 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_stat 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_stat' 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_stat 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_snapshots_pp 2024-02-18T00:32:50.754 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-18T00:32:50.755 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-18T00:32:50.755 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_stat 2024-02-18T00:32:50.755 INFO:tasks.workunit.client.0.smithi059.stdout:test api_stat on pid 82984 2024-02-18T00:32:50.756 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82984 2024-02-18T00:32:50.756 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_stat on pid 82984' 2024-02-18T00:32:50.756 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82984 2024-02-18T00:32:50.756 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.756 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-18T00:32:50.756 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.756 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_stat_pp 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_stat_pp' 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_stat_pp 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_stat 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_stat: /' 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_stat_pp 2024-02-18T00:32:50.757 INFO:tasks.workunit.client.0.smithi059.stdout:test api_stat_pp on pid 82992 2024-02-18T00:32:50.758 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82992 2024-02-18T00:32:50.758 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_stat_pp on pid 82992' 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=82992 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_watch_notify 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_watch_notify' 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_watch_notify 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_stat_pp 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-18T00:32:50.759 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_watch_notify 2024-02-18T00:32:50.760 INFO:tasks.workunit.client.0.smithi059.stdout:test api_watch_notify on pid 83000 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83000 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_watch_notify on pid 83000' 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83000 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_watch_notify_pp 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_watch_notify_pp' 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_watch_notify_pp 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_watch_notify 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-18T00:32:50.761 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-18T00:32:50.762 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_watch_notify_pp 2024-02-18T00:32:50.762 INFO:tasks.workunit.client.0.smithi059.stdout:test api_watch_notify_pp on pid 83008 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83008 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_watch_notify_pp on pid 83008' 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83008 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_cmd 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_cmd' 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_cmd 2024-02-18T00:32:50.763 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.764 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-02-18T00:32:50.764 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-18T00:32:50.764 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-18T00:32:50.764 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_cmd 2024-02-18T00:32:50.764 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83016 2024-02-18T00:32:50.764 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_cmd on pid 83016' 2024-02-18T00:32:50.765 INFO:tasks.workunit.client.0.smithi059.stdout:test api_cmd on pid 83016 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83016 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_cmd_pp 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_cmd_pp' 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_cmd_pp 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_cmd 2024-02-18T00:32:50.766 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-18T00:32:50.767 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_cmd: /' 2024-02-18T00:32:50.767 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_cmd_pp 2024-02-18T00:32:50.767 INFO:tasks.workunit.client.0.smithi059.stdout:test api_cmd_pp on pid 83024 2024-02-18T00:32:50.768 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83024 2024-02-18T00:32:50.768 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_cmd_pp on pid 83024' 2024-02-18T00:32:50.768 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83024 2024-02-18T00:32:50.768 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_service 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_service' 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_service 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_cmd_pp 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-18T00:32:50.769 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-18T00:32:50.770 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_service 2024-02-18T00:32:50.770 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83032 2024-02-18T00:32:50.770 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_service on pid 83032' 2024-02-18T00:32:50.770 INFO:tasks.workunit.client.0.smithi059.stdout:test api_service on pid 83032 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83032 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_service_pp 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_service_pp' 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_service_pp 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_service 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_service.log 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_service: /' 2024-02-18T00:32:50.772 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_service_pp 2024-02-18T00:32:50.773 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83040 2024-02-18T00:32:50.773 INFO:tasks.workunit.client.0.smithi059.stdout:test api_service_pp on pid 83040 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_service_pp on pid 83040' 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83040 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_c_write_operations 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_c_write_operations' 2024-02-18T00:32:50.774 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_c_write_operations 2024-02-18T00:32:50.775 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.775 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_service_pp 2024-02-18T00:32:50.775 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-18T00:32:50.775 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-18T00:32:50.775 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_c_write_operations 2024-02-18T00:32:50.775 INFO:tasks.workunit.client.0.smithi059.stdout:test api_c_write_operations on pid 83048 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83048 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_c_write_operations on pid 83048' 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83048 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s api_c_read_operations 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' api_c_read_operations' 2024-02-18T00:32:50.776 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo api_c_read_operations 2024-02-18T00:32:50.777 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.777 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_c_write_operations 2024-02-18T00:32:50.777 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-18T00:32:50.777 INFO:tasks.workunit.client.0.smithi059.stdout:test api_c_read_operations on pid 83056 2024-02-18T00:32:50.778 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-18T00:32:50.778 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=api_c_read_operations 2024-02-18T00:32:50.778 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83056 2024-02-18T00:32:50.778 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test api_c_read_operations on pid 83056' 2024-02-18T00:32:50.778 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83056 2024-02-18T00:32:50.778 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s list_parallel 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' list_parallel' 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo list_parallel 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_api_c_read_operations 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=list_parallel 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83064 2024-02-18T00:32:50.779 INFO:tasks.workunit.client.0.smithi059.stdout:test list_parallel on pid 83064 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test list_parallel on pid 83064' 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83064 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s open_pools_parallel 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' open_pools_parallel' 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo open_pools_parallel 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_list_parallel 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-18T00:32:50.781 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ list_parallel: /' 2024-02-18T00:32:50.782 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:32:50.782 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=open_pools_parallel 2024-02-18T00:32:50.783 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83072 2024-02-18T00:32:50.783 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test open_pools_parallel on pid 83072' 2024-02-18T00:32:50.783 INFO:tasks.workunit.client.0.smithi059.stdout:test open_pools_parallel on pid 83072 2024-02-18T00:32:50.784 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83072 2024-02-18T00:32:50.784 INFO:tasks.workunit.client.0.smithi059.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T00:32:50.784 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:++ printf %25s delete_pools_parallel 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:+ r=' delete_pools_parallel' 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:++ echo delete_pools_parallel 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:++ awk '{print $1}' 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_open_pools_parallel 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:+ ff=delete_pools_parallel 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83080 2024-02-18T00:32:50.785 INFO:tasks.workunit.client.0.smithi059.stdout:test delete_pools_parallel on pid 83080 2024-02-18T00:32:50.786 INFO:tasks.workunit.client.0.smithi059.stderr:+ echo 'test delete_pools_parallel on pid 83080' 2024-02-18T00:32:50.786 INFO:tasks.workunit.client.0.smithi059.stderr:+ pids[$f]=83080 2024-02-18T00:32:50.786 INFO:tasks.workunit.client.0.smithi059.stderr:+ ret=0 2024-02-18T00:32:50.786 INFO:tasks.workunit.client.0.smithi059.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T00:32:50.787 INFO:tasks.workunit.client.0.smithi059.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-18T00:32:50.787 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:32:50.787 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82968 2024-02-18T00:32:50.787 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82968 2024-02-18T00:32:50.787 INFO:tasks.workunit.client.0.smithi059.stderr:+ ceph_test_rados_delete_pools_parallel 2024-02-18T00:32:50.788 INFO:tasks.workunit.client.0.smithi059.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-18T00:32:50.788 INFO:tasks.workunit.client.0.smithi059.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-18T00:32:50.856 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:50.860 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:50.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:50.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:51.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:32:51.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:32:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:32:52.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:49.764319+0000 mgr.y (mgr.24454) 143 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:49.926012+0000 osd.5 (osd.5) 67 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:49.926991+0000 osd.5 (osd.5) 68 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.004009+0000 osd.0 (osd.0) 63 : cluster [DBG] 4.b scrub starts 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.007589+0000 osd.0 (osd.0) 64 : cluster [DBG] 4.b scrub ok 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.120104+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.121514+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.593564+0000 osd.7 (osd.7) 67 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.594794+0000 osd.7 (osd.7) 68 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:49.764319+0000 mgr.y (mgr.24454) 143 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:52.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:49.926012+0000 osd.5 (osd.5) 67 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:32:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:49.926991+0000 osd.5 (osd.5) 68 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:32:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.004009+0000 osd.0 (osd.0) 63 : cluster [DBG] 4.b scrub starts 2024-02-18T00:32:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.007589+0000 osd.0 (osd.0) 64 : cluster [DBG] 4.b scrub ok 2024-02-18T00:32:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.120104+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:32:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.121514+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:32:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.593564+0000 osd.7 (osd.7) 67 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:32:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.594794+0000 osd.7 (osd.7) 68 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:49.764319+0000 mgr.y (mgr.24454) 143 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:49.926012+0000 osd.5 (osd.5) 67 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:49.926991+0000 osd.5 (osd.5) 68 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.004009+0000 osd.0 (osd.0) 63 : cluster [DBG] 4.b scrub starts 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.007589+0000 osd.0 (osd.0) 64 : cluster [DBG] 4.b scrub ok 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.120104+0000 osd.4 (osd.4) 79 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.121514+0000 osd.4 (osd.4) 80 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.593564+0000 osd.7 (osd.7) 67 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:32:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.594794+0000 osd.7 (osd.7) 68 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:32:52.423 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:52.428 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:52.428 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:52.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:32:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:32:52] "GET /metrics HTTP/1.1" 200 172238 "" "Prometheus/2.33.4" 2024-02-18T00:32:52.529 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:52.561 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:52.583 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:52.585 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:52.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:52.687 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:52.716 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:52.735 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:52.737 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:52.738 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:52.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:52.874 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:52.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:52.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:53.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:53.117 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:53.218 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:53.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 -- 172.21.15.59:0/2469856531 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 msgr2=0x7f8440104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 --2- 172.21.15.59:0/2469856531 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f8440104a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f8438009be0 tx=0x7f8438004d10).stop 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 -- 172.21.15.59:0/2469856531 shutdown_connections 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 --2- 172.21.15.59:0/2469856531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84400fdbb0 0x7f84400fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 --2- 172.21.15.59:0/2469856531 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f8440104a20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 --2- 172.21.15.59:0/2469856531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84400fe510 0x7f84400f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 -- 172.21.15.59:0/2469856531 >> 172.21.15.59:0/2469856531 conn(0x7f84400f3800 msgr2=0x7f84400f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 -- 172.21.15.59:0/2469856531 shutdown_connections 2024-02-18T00:32:53.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.225+0000 7f8447fb4700 1 -- 172.21.15.59:0/2469856531 wait complete. 2024-02-18T00:32:53.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.226+0000 7f8447fb4700 1 librados: starting msgr at 2024-02-18T00:32:53.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.226+0000 7f8447fb4700 1 librados: starting objecter 2024-02-18T00:32:53.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 Processor -- start 2024-02-18T00:32:53.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 -- start start 2024-02-18T00:32:53.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 librados: setting wanted keys 2024-02-18T00:32:53.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 librados: calling monclient init 2024-02-18T00:32:53.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f844006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84400fdbb0 0x7f844006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84400fe510 0x7f844006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84401163c0 con 0x7f84400fe510 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8440116530 con 0x7f84400fdbb0 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.227+0000 7f8447fb4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84401166f0 con 0x7f84400f8420 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f844006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8446551700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84400fe510 0x7f844006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:53.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f844006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55694/0 (socket says 172.21.15.59:55694) 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 -- 172.21.15.59:0/651305083 learned_addr learned my addr 172.21.15.59:0/651305083 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f844554f700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84400fdbb0 0x7f844006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 -- 172.21.15.59:0/651305083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84400fdbb0 msgr2=0x7f844006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84400fdbb0 0x7f844006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 -- 172.21.15.59:0/651305083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84400fe510 msgr2=0x7f844006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84400fe510 0x7f844006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 -- 172.21.15.59:0/651305083 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84380097c0 con 0x7f84400f8420 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8445d50700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f844006b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f843c00e970 tx=0x7f843c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8436ffd700 10 client.?.objecter ms_handle_connect 0x7f84400f8420 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8436ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8436ffd700 1 -- 172.21.15.59:0/651305083 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c004660 con 0x7f84400f8420 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8436ffd700 1 -- 172.21.15.59:0/651305083 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843c00f040 con 0x7f84400f8420 2024-02-18T00:32:53.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84401169d0 con 0x7f84400f8420 2024-02-18T00:32:53.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8436ffd700 1 -- 172.21.15.59:0/651305083 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c014790 con 0x7f84400f8420 2024-02-18T00:32:53.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8447fb4700 10 client.24836.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:53.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8440116f90 con 0x7f84400f8420 2024-02-18T00:32:53.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.228+0000 7f8447fb4700 1 librados: init done 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.229+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8440107110 con 0x7f84400f8420 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f8436ffd700 1 -- 172.21.15.59:0/651305083 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f843c0090d0 con 0x7f84400f8420 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f8436ffd700 1 --2- 172.21.15.59:0/651305083 >> v2:172.21.15.59:6800/3721891797 conn(0x7f842c062c90 0x7f842c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f8436ffd700 1 -- 172.21.15.59:0/651305083 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(103..103 src has 44..103) v4 ==== 6929+0+0 (secure 0 0 0) 0x7f843c07c870 con 0x7f84400f8420 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f8436ffd700 10 client.24836.objecter ms_dispatch 0x7f8440001550 osd_map(103..103 src has 44..103) v4 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f8436ffd700 3 client.24836.objecter handle_osd_map got epochs [103,103] > 0 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f8436ffd700 3 client.24836.objecter handle_osd_map decoding full epoch 103 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f844554f700 1 --2- 172.21.15.59:0/651305083 >> v2:172.21.15.59:6800/3721891797 conn(0x7f842c062c90 0x7f842c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.230+0000 7f8436ffd700 20 client.24836.objecter dump_active .. 0 homeless 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.231+0000 7f8436ffd700 10 client.24836.objecter ms_handle_connect 0x7f842c062c90 2024-02-18T00:32:53.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.231+0000 7f844554f700 1 --2- 172.21.15.59:0/651305083 >> v2:172.21.15.59:6800/3721891797 conn(0x7f842c062c90 0x7f842c065150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f844006c870 tx=0x7f843000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:53.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.234+0000 7f8436ffd700 1 -- 172.21.15.59:0/651305083 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f843c04af60 con 0x7f84400f8420 2024-02-18T00:32:53.320 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:53.347 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:53.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:53.418+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"} v 0) v1 -- 0x7f844005fb70 con 0x7f84400f8420 2024-02-18T00:32:53.496 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:32:53.496 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:32:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:53.506 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:53.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:53.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:32:53.608 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:53.640 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:53.662 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:53.664 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:53.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:53.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:53.794 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:54.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-18T00:32:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:54.250 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:54.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:54.288 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:54.303 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:54.357 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:54.389 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:54.410 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:54.413 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:54.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:54.514 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:54.543 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:54.563 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:54.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:54.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:54.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:54.907 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:54.911 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:54.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:55.012 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:55.044 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:55.065 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:55.067 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:55.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:55.169 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:55.199 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:55.349 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:55.350 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:55.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:32:55.479 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:55.498 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:32:55.501 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:55.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:32:55.602 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:55.634 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:55.656 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:55.658 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:55.658 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:55.756 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:32:55.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:55.792 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:56.541 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:56.553 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:56.553 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:56.654 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:56.707 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:56.710 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:56.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:56.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:56.842 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:56.862 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:56.864 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:56.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:56.965 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:32:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:57.358 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:32:57.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:57.361 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:32:57.462 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:32:57.490 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:57.509 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:32:57.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:57.512 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:32:57.615 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:50.905196+0000 mgr.y (mgr.24454) 144 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.943090+0000 osd.5 (osd.5) 69 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:50.948232+0000 osd.5 (osd.5) 70 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:51.334848+0000 osd.7 (osd.7) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:51.334940+0000 osd.7 (osd.7) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:51.631289+0000 osd.7 (osd.7) 71 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:51.632884+0000 osd.7 (osd.7) 72 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:32:57.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:51.912193+0000 mon.a (mon.0) 788 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:32:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:57.662 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:57.664 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:57.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:50.905196+0000 mgr.y (mgr.24454) 144 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.943090+0000 osd.5 (osd.5) 69 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:50.948232+0000 osd.5 (osd.5) 70 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:51.334848+0000 osd.7 (osd.7) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:51.334940+0000 osd.7 (osd.7) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:51.631289+0000 osd.7 (osd.7) 71 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:51.632884+0000 osd.7 (osd.7) 72 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:32:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:51.912193+0000 mon.a (mon.0) 788 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:50.905196+0000 mgr.y (mgr.24454) 144 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.943090+0000 osd.5 (osd.5) 69 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:50.948232+0000 osd.5 (osd.5) 70 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:51.334848+0000 osd.7 (osd.7) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:51.334940+0000 osd.7 (osd.7) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:51.631289+0000 osd.7 (osd.7) 71 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:51.632884+0000 osd.7 (osd.7) 72 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:32:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:51.912193+0000 mon.a (mon.0) 788 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:32:57.766 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:32:57.792 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:57.811 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:32:57.813 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:57.814 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:32:57.915 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:32:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:57.959 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:32:57.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:57.962 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:32:58.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:58.092 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:58.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:58.229+0000 7f84477b3700 10 client.24836.objecter tick 2024-02-18T00:32:58.995 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:32:58.998 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:58.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:32:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.046+0000 7f5a3763e700 1 -- 172.21.15.59:0/1084320689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fdbb0 msgr2=0x7f5a300fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.046+0000 7f5a3763e700 1 --2- 172.21.15.59:0/1084320689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fdbb0 0x7f5a300fdf90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f5a2c0047b0 tx=0x7f5a2c009cb0).stop 2024-02-18T00:32:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.050+0000 7f5a3763e700 1 -- 172.21.15.59:0/1084320689 shutdown_connections 2024-02-18T00:32:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.050+0000 7f5a3763e700 1 --2- 172.21.15.59:0/1084320689 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a300f8420 0x7f5a30104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.050+0000 7f5a3763e700 1 --2- 172.21.15.59:0/1084320689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fdbb0 0x7f5a300fdf90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.050+0000 7f5a3763e700 1 --2- 172.21.15.59:0/1084320689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a300fe510 0x7f5a300f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.050+0000 7f5a3763e700 1 -- 172.21.15.59:0/1084320689 >> 172.21.15.59:0/1084320689 conn(0x7f5a300f3800 msgr2=0x7f5a300f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:59.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.053+0000 7f5a3763e700 1 -- 172.21.15.59:0/1084320689 shutdown_connections 2024-02-18T00:32:59.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.053+0000 7f5a3763e700 1 -- 172.21.15.59:0/1084320689 wait complete. 2024-02-18T00:32:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.054+0000 7f5a3763e700 1 librados: starting msgr at 2024-02-18T00:32:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.054+0000 7f5a3763e700 1 librados: starting objecter 2024-02-18T00:32:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.054+0000 7f5a3763e700 1 Processor -- start 2024-02-18T00:32:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.054+0000 7f5a3763e700 1 -- start start 2024-02-18T00:32:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.054+0000 7f5a3763e700 1 librados: setting wanted keys 2024-02-18T00:32:59.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.054+0000 7f5a3763e700 1 librados: calling monclient init 2024-02-18T00:32:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a3763e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a300f8420 0x7f5a30114780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a3763e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a300fdbb0 0x7f5a30114c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a3763e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fe510 0x7f5a301013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a3763e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301018e0 con 0x7f5a300fe510 2024-02-18T00:32:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a3763e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30101a50 con 0x7f5a300fdbb0 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a3763e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30101c10 con 0x7f5a300f8420 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fe510 0x7f5a301013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fe510 0x7f5a301013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53568/0 (socket says 172.21.15.59:53568) 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 learned_addr learned my addr 172.21.15.59:0/890395403 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a34bd9700 1 --2- 172.21.15.59:0/890395403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a300fdbb0 0x7f5a30114c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a300f8420 0x7f5a30114780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a300f8420 msgr2=0x7f5a30114780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 --2- 172.21.15.59:0/890395403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a300f8420 0x7f5a30114780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a300fdbb0 msgr2=0x7f5a30114c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 --2- 172.21.15.59:0/890395403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a300fdbb0 0x7f5a30114c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280097c0 con 0x7f5a300fe510 2024-02-18T00:32:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.055+0000 7f5a35bdb700 1 --2- 172.21.15.59:0/890395403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fe510 0x7f5a301013f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f5a28001d70 tx=0x7f5a28004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:51.697684+0000 osd.3 (osd.3) 89 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:32:59.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:51.698510+0000 osd.3 (osd.3) 90 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:32:59.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:51.765333+0000 mgr.y (mgr.24454) 145 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:59.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:51.924609+0000 mon.b (mon.1) 32 : audit [INF] from='client.? 172.21.15.59:0/1230350963' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:32:59.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:53.419955+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.59:0/651305083' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:32:59.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:57.246192+0000 mon.a (mon.0) 789 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-18T00:32:59.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.257396+0000 mon.a (mon.0) 790 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:32:59.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:51.697684+0000 osd.3 (osd.3) 89 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:32:59.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:51.698510+0000 osd.3 (osd.3) 90 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:32:59.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:51.765333+0000 mgr.y (mgr.24454) 145 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:59.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:51.924609+0000 mon.b (mon.1) 32 : audit [INF] from='client.? 172.21.15.59:0/1230350963' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:32:59.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:53.419955+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.59:0/651305083' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:32:59.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:57.246192+0000 mon.a (mon.0) 789 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-18T00:32:59.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:32:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.257396+0000 mon.a (mon.0) 790 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:32:59.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.060+0000 7f5a267fc700 10 client.?.objecter ms_handle_connect 0x7f5a300fe510 2024-02-18T00:32:59.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.060+0000 7f5a267fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:32:59.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.061+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28009a10 con 0x7f5a300fe510 2024-02-18T00:32:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.061+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a20005040 con 0x7f5a300fe510 2024-02-18T00:32:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.061+0000 7f5a3763e700 10 client.15042.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:32:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.061+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c005040 con 0x7f5a300fe510 2024-02-18T00:32:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.061+0000 7f5a3763e700 1 librados: init done 2024-02-18T00:32:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.062+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a280090d0 con 0x7f5a300fe510 2024-02-18T00:32:59.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.062+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28007cc0 con 0x7f5a300fe510 2024-02-18T00:32:59.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.062+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f5a14004f30 con 0x7f5a300fe510 2024-02-18T00:32:59.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.064+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5a2800fb30 con 0x7f5a300fe510 2024-02-18T00:32:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.064+0000 7f5a267fc700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a1c062ce0 0x7f5a1c0651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.064+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(104..104 src has 44..104) v4 ==== 11367+0+0 (secure 0 0 0) 0x7f5a2807e8a0 con 0x7f5a300fe510 2024-02-18T00:32:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.064+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 osd_map(104..104 src has 44..104) v4 2024-02-18T00:32:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.064+0000 7f5a267fc700 3 client.15042.objecter handle_osd_map got epochs [104,104] > 0 2024-02-18T00:32:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.064+0000 7f5a267fc700 3 client.15042.objecter handle_osd_map decoding full epoch 104 2024-02-18T00:32:59.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.065+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a1c062ce0 0x7f5a1c0651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.065+0000 7f5a267fc700 20 client.15042.objecter dump_active .. 0 homeless 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.065+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v104) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5a2807f0c0 con 0x7f5a300fe510 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.065+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a1c062ce0 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.065+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a1c062ce0 0x7f5a1c0651a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5a2c009da0 tx=0x7f5a2c007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 10 client.15042.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 1 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5a14004560 0x7f5a14004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 1 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 0 1 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140051d0 osd=0 4 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 10 client.15042.objecter _send_command 1 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a14008960 con 0x7f5a14004560 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a34bd9700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5a14004560 0x7f5a14004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a34bd9700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5a14004560 0x7f5a14004980 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.067+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a14004560 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.068+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a14008960 con 0x7f5a14004560 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.068+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.068+0000 7f5a267fc700 10 client.15042.objecter _finish_command 1 = system:0 2024-02-18T00:32:59.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.068+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a140051d0 osd=0 4 2024-02-18T00:32:59.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.068+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 0 1 2024-02-18T00:32:59.099 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 10 client.15042.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 2 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 2 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a140051d0 osd=0 3 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 0 2 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140051d0 osd=0 4 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 10 client.15042.objecter _send_command 2 2024-02-18T00:32:59.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.115+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a14003fd0 con 0x7f5a14004560 2024-02-18T00:32:59.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a14003fd0 con 0x7f5a14004560 2024-02-18T00:32:59.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a267fc700 10 client.15042.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a140051d0 osd=0 4 2024-02-18T00:32:59.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 0 2 2024-02-18T00:32:59.128 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:32:59.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 10 client.15042.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6805/26279110 conn(0x7f5a140084c0 0x7f5a14002740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 1 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14004380 osd=1 4 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 10 client.15042.objecter _send_command 3 2024-02-18T00:32:59.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.126+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a14002dc0 con 0x7f5a140084c0 2024-02-18T00:32:59.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.129+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6805/26279110 conn(0x7f5a140084c0 0x7f5a14002740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.133 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:59.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.132+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6805/26279110 conn(0x7f5a140084c0 0x7f5a14002740 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.132+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a140084c0 2024-02-18T00:32:59.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.133+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a14002dc0 con 0x7f5a140084c0 2024-02-18T00:32:59.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.133+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:32:59.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.133+0000 7f5a267fc700 10 client.15042.objecter _finish_command 3 = system:0 2024-02-18T00:32:59.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.133+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a14004380 osd=1 4 2024-02-18T00:32:59.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.133+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 1 3 2024-02-18T00:32:59.154 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:59.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:59.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:59.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 10 client.15042.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 4 2024-02-18T00:32:59.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 4 2024-02-18T00:32:59.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a14004380 osd=1 3 2024-02-18T00:32:59.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 1 4 2024-02-18T00:32:59.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14004380 osd=1 4 2024-02-18T00:32:59.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 10 client.15042.objecter _send_command 4 2024-02-18T00:32:59.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.154+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a14007ae0 con 0x7f5a140084c0 2024-02-18T00:32:59.158 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:32:59.158 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:32:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a14007ae0 con 0x7f5a140084c0 2024-02-18T00:32:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a267fc700 10 client.15042.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a14004380 osd=1 4 2024-02-18T00:32:59.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 1 4 2024-02-18T00:32:59.159 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 10 client.15042.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 5 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5a14003920 0x7f5a1400d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 5 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 2 5 2024-02-18T00:32:59.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140037b0 osd=2 4 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 10 client.15042.objecter _send_command 5 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a1400fdb0 con 0x7f5a14003920 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.157+0000 7f5a34bd9700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5a14003920 0x7f5a1400d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.158+0000 7f5a34bd9700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5a14003920 0x7f5a1400d000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.158+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a14003920 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.159+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a1400fdb0 con 0x7f5a14003920 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.159+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.159+0000 7f5a267fc700 10 client.15042.objecter _finish_command 5 = system:0 2024-02-18T00:32:59.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.159+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a140037b0 osd=2 4 2024-02-18T00:32:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.159+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 2 5 2024-02-18T00:32:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 10 client.15042.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 6 2024-02-18T00:32:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 6 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a140037b0 osd=2 3 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 2 6 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140037b0 osd=2 4 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 10 client.15042.objecter _send_command 6 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.174+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a140032e0 con 0x7f5a14003920 2024-02-18T00:32:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a140032e0 con 0x7f5a14003920 2024-02-18T00:32:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a267fc700 10 client.15042.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a140037b0 osd=2 4 2024-02-18T00:32:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 2 6 2024-02-18T00:32:59.178 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 10 client.15042.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 7 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5a1400e6a0 0x7f5a1400eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 7 2024-02-18T00:32:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 3 7 2024-02-18T00:32:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14001b90 osd=3 4 2024-02-18T00:32:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 10 client.15042.objecter _send_command 7 2024-02-18T00:32:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a14002280 con 0x7f5a1400e6a0 2024-02-18T00:32:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a35bdb700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5a1400e6a0 0x7f5a1400eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a1400e6a0 2024-02-18T00:32:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.176+0000 7f5a35bdb700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5a1400e6a0 0x7f5a1400eb30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.177+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a14002280 con 0x7f5a1400e6a0 2024-02-18T00:32:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.177+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:32:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.177+0000 7f5a267fc700 10 client.15042.objecter _finish_command 7 = system:0 2024-02-18T00:32:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.177+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a14001b90 osd=3 4 2024-02-18T00:32:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.177+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 3 7 2024-02-18T00:32:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 10 client.15042.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 8 2024-02-18T00:32:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 8 2024-02-18T00:32:59.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a14001b90 osd=3 3 2024-02-18T00:32:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 3 8 2024-02-18T00:32:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14001b90 osd=3 4 2024-02-18T00:32:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 10 client.15042.objecter _send_command 8 2024-02-18T00:32:59.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.193+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a1400f7f0 con 0x7f5a1400e6a0 2024-02-18T00:32:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a1400f7f0 con 0x7f5a1400e6a0 2024-02-18T00:32:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a267fc700 10 client.15042.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a14001b90 osd=3 4 2024-02-18T00:32:59.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 3 8 2024-02-18T00:32:59.197 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:32:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 10 client.15042.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 9 2024-02-18T00:32:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5a14011fe0 0x7f5a140143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 9 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 4 9 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1400f020 osd=4 4 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 10 client.15042.objecter _send_command 9 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a14017020 con 0x7f5a14011fe0 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.195+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5a14011fe0 0x7f5a140143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.196+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a14011fe0 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.196+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5a14011fe0 0x7f5a140143d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.196+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a14017020 con 0x7f5a14011fe0 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.196+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.196+0000 7f5a267fc700 10 client.15042.objecter _finish_command 9 = system:0 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.196+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a1400f020 osd=4 4 2024-02-18T00:32:59.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.196+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 4 9 2024-02-18T00:32:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 10 client.15042.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 10 2024-02-18T00:32:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 10 2024-02-18T00:32:59.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a1400f020 osd=4 3 2024-02-18T00:32:59.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 4 10 2024-02-18T00:32:59.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1400f020 osd=4 4 2024-02-18T00:32:59.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 10 client.15042.objecter _send_command 10 2024-02-18T00:32:59.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.211+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a140148c0 con 0x7f5a14011fe0 2024-02-18T00:32:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a140148c0 con 0x7f5a14011fe0 2024-02-18T00:32:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a267fc700 10 client.15042.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a1400f020 osd=4 4 2024-02-18T00:32:59.215 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 4 10 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 10 client.15042.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 11 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5a14014fd0 0x7f5a140173c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 11 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 5 11 2024-02-18T00:32:59.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140148c0 osd=5 4 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 10 client.15042.objecter _send_command 11 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.213+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a1401a120 con 0x7f5a14014fd0 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.214+0000 7f5a34bd9700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5a14014fd0 0x7f5a140173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.214+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a14014fd0 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.214+0000 7f5a34bd9700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5a14014fd0 0x7f5a140173c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.215+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a1401a120 con 0x7f5a14014fd0 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.215+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.215+0000 7f5a267fc700 10 client.15042.objecter _finish_command 11 = system:0 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.215+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a140148c0 osd=5 4 2024-02-18T00:32:59.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.215+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 5 11 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 10 client.15042.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 12 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 12 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a140148c0 osd=5 3 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 5 12 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140148c0 osd=5 4 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 10 client.15042.objecter _send_command 12 2024-02-18T00:32:59.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.229+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a1401a120 con 0x7f5a14014fd0 2024-02-18T00:32:59.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a1401a120 con 0x7f5a14014fd0 2024-02-18T00:32:59.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a267fc700 10 client.15042.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a140148c0 osd=5 4 2024-02-18T00:32:59.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 5 12 2024-02-18T00:32:59.236 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:32:59.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 10 client.15042.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 13 2024-02-18T00:32:59.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5a14017e60 0x7f5a140182c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 13 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 6 13 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1401a120 osd=6 4 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 10 client.15042.objecter _send_command 13 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a14019ef0 con 0x7f5a14017e60 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.231+0000 7f5a35bdb700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5a14017e60 0x7f5a140182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.232+0000 7f5a35bdb700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5a14017e60 0x7f5a140182c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.232+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a14017e60 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.232+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a14019ef0 con 0x7f5a14017e60 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.232+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:32:59.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.232+0000 7f5a267fc700 10 client.15042.objecter _finish_command 13 = system:0 2024-02-18T00:32:59.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.232+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a1401a120 osd=6 4 2024-02-18T00:32:59.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.232+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 6 13 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 10 client.15042.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 14 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 14 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a1401a120 osd=6 3 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 6 14 2024-02-18T00:32:59.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1401a120 osd=6 4 2024-02-18T00:32:59.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 10 client.15042.objecter _send_command 14 2024-02-18T00:32:59.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.249+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a14019ef0 con 0x7f5a14017e60 2024-02-18T00:32:59.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.251+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a14019ef0 con 0x7f5a14017e60 2024-02-18T00:32:59.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.251+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.251+0000 7f5a267fc700 10 client.15042.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.251+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a1401a120 osd=6 4 2024-02-18T00:32:59.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.251+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 6 14 2024-02-18T00:32:59.254 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:32:59.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 10 client.15042.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 15 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6812/53025171 conn(0x7f5a1401c980 0x7f5a1401cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 15 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 7 15 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14019ef0 osd=7 4 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 10 client.15042.objecter _send_command 15 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a14020710 con 0x7f5a1401c980 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6812/53025171 conn(0x7f5a1401c980 0x7f5a1401cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a267fc700 10 client.15042.objecter ms_handle_connect 0x7f5a1401c980 2024-02-18T00:32:59.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.252+0000 7f5a353da700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6812/53025171 conn(0x7f5a1401c980 0x7f5a1401cda0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:32:59.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.253+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5a14020710 con 0x7f5a1401c980 2024-02-18T00:32:59.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.253+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:32:59.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.253+0000 7f5a267fc700 10 client.15042.objecter _finish_command 15 = system:0 2024-02-18T00:32:59.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.253+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a14019ef0 osd=7 4 2024-02-18T00:32:59.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.253+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 7 15 2024-02-18T00:32:59.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:32:59.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 10 client.15042.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign -1 16 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 20 client.15042.objecter _get_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_remove -1 16 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 20 client.15042.objecter get_session s=0x7f5a14019ef0 osd=7 3 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 15 client.15042.objecter _session_command_op_assign 7 16 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14019ef0 osd=7 4 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 10 client.15042.objecter _send_command 16 2024-02-18T00:32:59.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.272+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5a140188a0 con 0x7f5a1401c980 2024-02-18T00:32:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.273+0000 7f5a267fc700 1 -- 172.21.15.59:0/890395403 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5a140188a0 con 0x7f5a1401c980 2024-02-18T00:32:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.273+0000 7f5a267fc700 10 client.15042.objecter ms_dispatch 0x7f5a30001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:32:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.273+0000 7f5a267fc700 10 client.15042.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.273+0000 7f5a267fc700 20 client.15042.objecter put_session s=0x7f5a14019ef0 osd=7 4 2024-02-18T00:32:59.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.273+0000 7f5a267fc700 15 client.15042.objecter _session_command_op_remove 7 16 2024-02-18T00:32:59.276 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 librados: watch_flush enter 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 librados: watch_flush exit 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.0 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5a14004560 msgr2=0x7f5a14004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5a14004560 0x7f5a14004980 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140051d0 osd=0 1 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.1 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6805/26279110 conn(0x7f5a140084c0 msgr2=0x7f5a14002740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6805/26279110 conn(0x7f5a140084c0 0x7f5a14002740 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14004380 osd=1 1 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.2 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5a14003920 msgr2=0x7f5a1400d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5a14003920 0x7f5a1400d000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140037b0 osd=2 1 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.3 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5a1400e6a0 msgr2=0x7f5a1400eb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5a1400e6a0 0x7f5a1400eb30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14001b90 osd=3 1 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.4 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5a14011fe0 msgr2=0x7f5a140143d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5a14011fe0 0x7f5a140143d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1400f020 osd=4 1 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.5 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5a14014fd0 msgr2=0x7f5a140173c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5a14014fd0 0x7f5a140173c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a140148c0 osd=5 1 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.6 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5a14017e60 msgr2=0x7f5a140182c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5a14017e60 0x7f5a140182c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a1401a120 osd=6 1 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter close_session for osd.7 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6812/53025171 conn(0x7f5a1401c980 msgr2=0x7f5a1401cda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.137:6812/53025171 conn(0x7f5a1401c980 0x7f5a1401cda0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter put_session s=0x7f5a14019ef0 osd=7 1 2024-02-18T00:32:59.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 20 client.15042.objecter shutdown clearing up homeless session... 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 10 client.15042.objecter successfully canceled tick 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 reap_dead start 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 reap_dead start 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 reap_dead start 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 reap_dead start 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a35bdb700 1 -- 172.21.15.59:0/890395403 reap_dead start 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a1c062ce0 msgr2=0x7f5a1c0651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a1c062ce0 0x7f5a1c0651a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5a2c009da0 tx=0x7f5a2c007f90).stop 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fe510 msgr2=0x7f5a301013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fe510 0x7f5a301013f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f5a28001d70 tx=0x7f5a28004d10).stop 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 shutdown_connections 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a1c062ce0 0x7f5a1c0651a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 --2- 172.21.15.59:0/890395403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a300fe510 0x7f5a301013f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 >> 172.21.15.59:0/890395403 conn(0x7f5a300f3800 msgr2=0x7f5a300fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 shutdown_connections 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 -- 172.21.15.59:0/890395403 wait complete. 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:32:59.274+0000 7f5a3763e700 1 librados: shutdown 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:32:59.285 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:59.304 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:32:59.306 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:32:59.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:32:59.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:32:59.437 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:32:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:51.697684+0000 osd.3 (osd.3) 89 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:32:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:51.698510+0000 osd.3 (osd.3) 90 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:32:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:51.765333+0000 mgr.y (mgr.24454) 145 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:32:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:51.924609+0000 mon.b (mon.1) 32 : audit [INF] from='client.? 172.21.15.59:0/1230350963' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:32:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:53.419955+0000 mon.c (mon.2) 32 : audit [INF] from='client.? 172.21.15.59:0/651305083' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:32:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:57.246192+0000 mon.a (mon.0) 789 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-18T00:32:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:32:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.257396+0000 mon.a (mon.0) 790 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:00.551 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-18T00:33:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:00.590 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:00.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:00.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:00.723 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:00.745 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:00.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:00.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:00.849 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:00.879 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:00.899 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:00.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:00.902 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:01.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:01.004 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:33:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:33:01.033 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:01.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 -- 172.21.15.59:0/1968140994 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df40f8d10 msgr2=0x7f8df40f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:01.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/1968140994 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df40f8d10 0x7f8df40f90f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f8df0004870 tx=0x7f8df0009cb0).stop 2024-02-18T00:33:01.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 -- 172.21.15.59:0/1968140994 shutdown_connections 2024-02-18T00:33:01.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/1968140994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df4102270 0x7f8df4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/1968140994 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df40f8d10 0x7f8df40f90f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/1968140994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 0x7f8df4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 -- 172.21.15.59:0/1968140994 >> 172.21.15.59:0/1968140994 conn(0x7f8df40f2820 msgr2=0x7f8df40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 -- 172.21.15.59:0/1968140994 shutdown_connections 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 -- 172.21.15.59:0/1968140994 wait complete. 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 librados: starting msgr at 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 librados: starting objecter 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.250+0000 7f8dfc79c700 1 Processor -- start 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 -- start start 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 librados: setting wanted keys 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 librados: calling monclient init 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df40f8d10 0x7f8df4110430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 0x7f8df4110920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df4102270 0x7f8df410a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:01.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df410aac0 con 0x7f8df40f9670 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df410ac00 con 0x7f8df40f8d10 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfc79c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df410adc0 con 0x7f8df4102270 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8df9d37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 0x7f8df4110920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfad39700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df4102270 0x7f8df410a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8df9d37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 0x7f8df4110920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51974/0 (socket says 172.21.15.59:51974) 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8df9d37700 1 -- 172.21.15.59:0/683218601 learned_addr learned my addr 172.21.15.59:0/683218601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfa538700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df40f8d10 0x7f8df4110430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfad39700 1 -- 172.21.15.59:0/683218601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df40f8d10 msgr2=0x7f8df4110430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfad39700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df40f8d10 0x7f8df4110430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfad39700 1 -- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 msgr2=0x7f8df4110920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfad39700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 0x7f8df4110920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfad39700 1 -- 172.21.15.59:0/683218601 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4005040 con 0x7f8df4102270 2024-02-18T00:33:01.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8df9d37700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 0x7f8df4110920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8dfad39700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df4102270 0x7f8df410a5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f8dec00eb40 tx=0x7f8dec00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8deb7fe700 10 client.?.objecter ms_handle_connect 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8deb7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.251+0000 7f8deb7fe700 1 -- 172.21.15.59:0/683218601 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec009870 con 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.252+0000 7f8deb7fe700 1 -- 172.21.15.59:0/683218601 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dec00f040 con 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.252+0000 7f8deb7fe700 1 -- 172.21.15.59:0/683218601 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec007e10 con 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.252+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df0005040 con 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.252+0000 7f8dfc79c700 10 client.24962.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.252+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df410b350 con 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.252+0000 7f8dfc79c700 1 librados: init done 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df405fa60 con 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8deb7fe700 1 -- 172.21.15.59:0/683218601 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8dec004120 con 0x7f8df4102270 2024-02-18T00:33:01.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8deb7fe700 1 --2- 172.21.15.59:0/683218601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8de0062c90 0x7f8de0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8deb7fe700 1 -- 172.21.15.59:0/683218601 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(105..105 src has 44..105) v4 ==== 12869+0+0 (secure 0 0 0) 0x7f8dec07e220 con 0x7f8df4102270 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8deb7fe700 10 client.24962.objecter ms_dispatch 0x7f8df4001550 osd_map(105..105 src has 44..105) v4 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8deb7fe700 3 client.24962.objecter handle_osd_map got epochs [105,105] > 0 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8deb7fe700 3 client.24962.objecter handle_osd_map decoding full epoch 105 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.253+0000 7f8dfa538700 1 --2- 172.21.15.59:0/683218601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8de0062c90 0x7f8de0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.254+0000 7f8deb7fe700 20 client.24962.objecter dump_active .. 0 homeless 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.254+0000 7f8deb7fe700 10 client.24962.objecter ms_handle_connect 0x7f8de0062c90 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.254+0000 7f8dfa538700 1 --2- 172.21.15.59:0/683218601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8de0062c90 0x7f8de0065150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8df0004ee0 tx=0x7f8df000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:01.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.258+0000 7f8deb7fe700 1 -- 172.21.15.59:0/683218601 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8dec049f60 con 0x7f8df4102270 2024-02-18T00:33:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:01.339 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:33:01.340 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:01.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:01.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.377+0000 7f8436ffd700 1 -- 172.21.15.59:0/651305083 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]=0 set pool 2 pg_num to 42 v106) v1 ==== 134+0+0 (secure 0 0 0) 0x7f843c0190b0 con 0x7f84400f8420 2024-02-18T00:33:01.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.384+0000 7f8447fb4700 10 librados: watch_flush enter 2024-02-18T00:33:01.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.390+0000 7f8447fb4700 10 librados: watch_flush exit 2024-02-18T00:33:01.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.390+0000 7f8447fb4700 20 client.24836.objecter shutdown clearing up homeless session... 2024-02-18T00:33:01.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.390+0000 7f8447fb4700 10 client.24836.objecter successfully canceled tick 2024-02-18T00:33:01.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.393+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 >> v2:172.21.15.59:6800/3721891797 conn(0x7f842c062c90 msgr2=0x7f842c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.393+0000 7f8447fb4700 1 --2- 172.21.15.59:0/651305083 >> v2:172.21.15.59:6800/3721891797 conn(0x7f842c062c90 0x7f842c065150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f844006c870 tx=0x7f843000b040).stop 2024-02-18T00:33:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.393+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 msgr2=0x7f844006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:01.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.393+0000 7f8447fb4700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f844006b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f843c00e970 tx=0x7f843c00b040).stop 2024-02-18T00:33:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.395+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 shutdown_connections 2024-02-18T00:33:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.395+0000 7f8447fb4700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84400fe510 0x7f844006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.395+0000 7f8447fb4700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84400f8420 0x7f844006b0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.395+0000 7f8447fb4700 1 --2- 172.21.15.59:0/651305083 >> v2:172.21.15.59:6800/3721891797 conn(0x7f842c062c90 0x7f842c065150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.395+0000 7f8447fb4700 1 --2- 172.21.15.59:0/651305083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84400fdbb0 0x7f844006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.395+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 >> 172.21.15.59:0/651305083 conn(0x7f84400f3800 msgr2=0x7f84400fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.400+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 shutdown_connections 2024-02-18T00:33:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.400+0000 7f8447fb4700 1 -- 172.21.15.59:0/651305083 wait complete. 2024-02-18T00:33:01.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.408+0000 7f8447fb4700 1 librados: shutdown 2024-02-18T00:33:01.421 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 2 pg_num to 42 2024-02-18T00:33:01.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:01.484 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:01.513 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:01.515 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:01.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:01.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:01.556+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f8df410af80 con 0x7f8df4102270 2024-02-18T00:33:01.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:01.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:52.321977+0000 osd.6 (osd.6) 57 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:33:01.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:52.322925+0000 osd.6 (osd.6) 58 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:33:01.633 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:52.883966+0000 osd.1 (osd.1) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:52.884017+0000 osd.1 (osd.1) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:53.111462+0000 osd.4 (osd.4) 81 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:53.114416+0000 osd.4 (osd.4) 82 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:53.368491+0000 osd.6 (osd.6) 59 : cluster [DBG] 4.a scrub starts 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:53.370864+0000 osd.6 (osd.6) 60 : cluster [DBG] 4.a scrub ok 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:53.765726+0000 mgr.y (mgr.24454) 146 : cluster [DBG] pgmap v138: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:53.809866+0000 osd.1 (osd.1) 81 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:53.812888+0000 osd.1 (osd.1) 82 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:54.632118+0000 osd.7 (osd.7) 73 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:54.632906+0000 osd.7 (osd.7) 74 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:54.835962+0000 osd.1 (osd.1) 83 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:54.839009+0000 osd.1 (osd.1) 84 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:33:01.634 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:55.587344+0000 osd.7 (osd.7) 75 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:55.588743+0000 osd.7 (osd.7) 76 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:55.766471+0000 mgr.y (mgr.24454) 147 : cluster [DBG] pgmap v139: 196 pgs: 51 creating+peering, 132 active+clean, 13 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:55.955576+0000 osd.0 (osd.0) 65 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:55.959037+0000 osd.0 (osd.0) 66 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:55.975637+0000 osd.5 (osd.5) 71 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:55.976573+0000 osd.5 (osd.5) 72 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.272358+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.59:0/3164350885' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.280885+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.59:0/1346273899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.286732+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.59:0/3512082368' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.287995+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.59:0/521262486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.289525+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.59:0/3991581620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.289748+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.59:0/1794179512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.295025+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.59:0/428925405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.295292+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:57.295489+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.59:0/1293437097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:57.385227+0000 osd.6 (osd.6) 61 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:57.386087+0000 osd.6 (osd.6) 62 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:57.766957+0000 mgr.y (mgr.24454) 148 : cluster [DBG] pgmap v141: 580 pgs: 51 creating+peering, 132 active+clean, 397 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:58.038126+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:58.038879+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:59.027272+0000 mon.a (mon.0) 791 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.041741+0000 mon.a (mon.0) 792 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.044637+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.045045+0000 mon.a (mon.0) 794 : audit [INF] from='client.? 172.21.15.59:0/464889388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi059-82925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.051177+0000 mon.a (mon.0) 795 : audit [INF] from='client.? 172.21.15.59:0/839866406' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.051340+0000 mon.a (mon.0) 796 : audit [INF] from='client.? 172.21.15.59:0/512189418' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi059-82882-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.051827+0000 mon.a (mon.0) 797 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.055414+0000 mon.a (mon.0) 798 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.056237+0000 mon.a (mon.0) 799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.056584+0000 mon.a (mon.0) 800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.056915+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.057202+0000 mon.a (mon.0) 802 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.062389+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.062573+0000 mon.a (mon.0) 804 : audit [DBG] from='client.? 172.21.15.59:0/1162212462' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.062841+0000 mon.a (mon.0) 805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.063209+0000 mon.a (mon.0) 806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.063525+0000 mon.a (mon.0) 807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.637 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.064609+0000 mon.a (mon.0) 808 : audit [DBG] from='client.? 172.21.15.59:0/890395403' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:52.321977+0000 osd.6 (osd.6) 57 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:52.322925+0000 osd.6 (osd.6) 58 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:52.883966+0000 osd.1 (osd.1) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:52.884017+0000 osd.1 (osd.1) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:53.111462+0000 osd.4 (osd.4) 81 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:53.114416+0000 osd.4 (osd.4) 82 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:53.368491+0000 osd.6 (osd.6) 59 : cluster [DBG] 4.a scrub starts 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:53.370864+0000 osd.6 (osd.6) 60 : cluster [DBG] 4.a scrub ok 2024-02-18T00:33:01.638 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:53.765726+0000 mgr.y (mgr.24454) 146 : cluster [DBG] pgmap v138: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:53.809866+0000 osd.1 (osd.1) 81 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:53.812888+0000 osd.1 (osd.1) 82 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:54.632118+0000 osd.7 (osd.7) 73 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:54.632906+0000 osd.7 (osd.7) 74 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:54.835962+0000 osd.1 (osd.1) 83 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:54.839009+0000 osd.1 (osd.1) 84 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:55.587344+0000 osd.7 (osd.7) 75 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:33:01.639 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:55.588743+0000 osd.7 (osd.7) 76 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:55.766471+0000 mgr.y (mgr.24454) 147 : cluster [DBG] pgmap v139: 196 pgs: 51 creating+peering, 132 active+clean, 13 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:55.955576+0000 osd.0 (osd.0) 65 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:55.959037+0000 osd.0 (osd.0) 66 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:55.975637+0000 osd.5 (osd.5) 71 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:55.976573+0000 osd.5 (osd.5) 72 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.272358+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.59:0/3164350885' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.280885+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.59:0/1346273899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.286732+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.59:0/3512082368' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.640 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.287995+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.59:0/521262486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.289525+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.59:0/3991581620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.289748+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.59:0/1794179512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.295025+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.59:0/428925405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.295292+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:57.295489+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.59:0/1293437097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:57.385227+0000 osd.6 (osd.6) 61 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:57.386087+0000 osd.6 (osd.6) 62 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:57.766957+0000 mgr.y (mgr.24454) 148 : cluster [DBG] pgmap v141: 580 pgs: 51 creating+peering, 132 active+clean, 397 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:58.038126+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:58.038879+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:59.027272+0000 mon.a (mon.0) 791 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.041741+0000 mon.a (mon.0) 792 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.044637+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.641 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.045045+0000 mon.a (mon.0) 794 : audit [INF] from='client.? 172.21.15.59:0/464889388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi059-82925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.051177+0000 mon.a (mon.0) 795 : audit [INF] from='client.? 172.21.15.59:0/839866406' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.051340+0000 mon.a (mon.0) 796 : audit [INF] from='client.? 172.21.15.59:0/512189418' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi059-82882-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.051827+0000 mon.a (mon.0) 797 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.055414+0000 mon.a (mon.0) 798 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.056237+0000 mon.a (mon.0) 799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.056584+0000 mon.a (mon.0) 800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.056915+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.057202+0000 mon.a (mon.0) 802 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.062389+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.062573+0000 mon.a (mon.0) 804 : audit [DBG] from='client.? 172.21.15.59:0/1162212462' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.062841+0000 mon.a (mon.0) 805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.063209+0000 mon.a (mon.0) 806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.063525+0000 mon.a (mon.0) 807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.642 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.064609+0000 mon.a (mon.0) 808 : audit [DBG] from='client.? 172.21.15.59:0/890395403' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:01.673 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:52.321977+0000 osd.6 (osd.6) 57 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:33:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:52.322925+0000 osd.6 (osd.6) 58 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:33:01.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:52.883966+0000 osd.1 (osd.1) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:52.884017+0000 osd.1 (osd.1) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:53.111462+0000 osd.4 (osd.4) 81 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:53.114416+0000 osd.4 (osd.4) 82 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:53.368491+0000 osd.6 (osd.6) 59 : cluster [DBG] 4.a scrub starts 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:53.370864+0000 osd.6 (osd.6) 60 : cluster [DBG] 4.a scrub ok 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:53.765726+0000 mgr.y (mgr.24454) 146 : cluster [DBG] pgmap v138: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:53.809866+0000 osd.1 (osd.1) 81 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:53.812888+0000 osd.1 (osd.1) 82 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:54.632118+0000 osd.7 (osd.7) 73 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:54.632906+0000 osd.7 (osd.7) 74 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:54.835962+0000 osd.1 (osd.1) 83 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:54.839009+0000 osd.1 (osd.1) 84 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:33:01.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:55.587344+0000 osd.7 (osd.7) 75 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:55.588743+0000 osd.7 (osd.7) 76 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:55.766471+0000 mgr.y (mgr.24454) 147 : cluster [DBG] pgmap v139: 196 pgs: 51 creating+peering, 132 active+clean, 13 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:55.955576+0000 osd.0 (osd.0) 65 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:55.959037+0000 osd.0 (osd.0) 66 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:55.975637+0000 osd.5 (osd.5) 71 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:55.976573+0000 osd.5 (osd.5) 72 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.272358+0000 mon.c (mon.2) 33 : audit [INF] from='client.? 172.21.15.59:0/3164350885' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.280885+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.59:0/1346273899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.286732+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.59:0/3512082368' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.287995+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.59:0/521262486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.289525+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.59:0/3991581620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.289748+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.59:0/1794179512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.295025+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.59:0/428925405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.295292+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:57.295489+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.59:0/1293437097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:57.385227+0000 osd.6 (osd.6) 61 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:57.386087+0000 osd.6 (osd.6) 62 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:57.766957+0000 mgr.y (mgr.24454) 148 : cluster [DBG] pgmap v141: 580 pgs: 51 creating+peering, 132 active+clean, 397 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:58.038126+0000 osd.4 (osd.4) 83 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:58.038879+0000 osd.4 (osd.4) 84 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:59.027272+0000 mon.a (mon.0) 791 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.041741+0000 mon.a (mon.0) 792 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.044637+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.045045+0000 mon.a (mon.0) 794 : audit [INF] from='client.? 172.21.15.59:0/464889388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi059-82925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.051177+0000 mon.a (mon.0) 795 : audit [INF] from='client.? 172.21.15.59:0/839866406' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.051340+0000 mon.a (mon.0) 796 : audit [INF] from='client.? 172.21.15.59:0/512189418' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi059-82882-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.051827+0000 mon.a (mon.0) 797 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.055414+0000 mon.a (mon.0) 798 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.056237+0000 mon.a (mon.0) 799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.056584+0000 mon.a (mon.0) 800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.056915+0000 mon.a (mon.0) 801 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.057202+0000 mon.a (mon.0) 802 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.062389+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.062573+0000 mon.a (mon.0) 804 : audit [DBG] from='client.? 172.21.15.59:0/1162212462' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.062841+0000 mon.a (mon.0) 805 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.063209+0000 mon.a (mon.0) 806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.063525+0000 mon.a (mon.0) 807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:01.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.064609+0000 mon.a (mon.0) 808 : audit [DBG] from='client.? 172.21.15.59:0/890395403' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:01.881 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:01.885 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:01.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:01.986 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:02.023 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:02.161 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:02.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:02.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:02.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:02.407 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:02] "GET /metrics HTTP/1.1" 200 216828 "" "Prometheus/2.33.4" 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:58.391682+0000 osd.6 (osd.6) 63 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:58.392439+0000 osd.6 (osd.6) 64 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:58.997662+0000 osd.0 (osd.0) 67 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:58.998746+0000 osd.0 (osd.0) 68 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.075292+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.59:0/1733075084' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.092439+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.59:0/1917601440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.101958+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.59:0/1001788920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:32:59.123013+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? 172.21.15.59:0/3681537941' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:59.767975+0000 mgr.y (mgr.24454) 149 : cluster [DBG] pgmap v143: 708 pgs: 51 creating+peering, 132 active+clean, 525 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 652 B/s rd, 0 op/s 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:59.799261+0000 osd.1 (osd.1) 85 : cluster [DBG] 6.1d deep-scrub starts 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:59.800343+0000 osd.1 (osd.1) 86 : cluster [DBG] 6.1d deep-scrub ok 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:59.990898+0000 osd.5 (osd.5) 73 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:33:02.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:32:59.991754+0000 osd.5 (osd.5) 74 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:00.004758+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.1a deep-scrub starts 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:00.013045+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.1a deep-scrub ok 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:00.916706+0000 mgr.y (mgr.24454) 150 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.148915+0000 mon.c (mon.2) 36 : audit [DBG] from='client.? 172.21.15.59:0/3145287853' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:01.315448+0000 mon.a (mon.0) 809 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.315615+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.322793+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.323028+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.324241+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.348762+0000 mon.a (mon.0) 814 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.348903+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.349193+0000 mon.a (mon.0) 816 : audit [INF] from='client.? 172.21.15.59:0/464889388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi059-82925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.410 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.349532+0000 mon.a (mon.0) 817 : audit [INF] from='client.? 172.21.15.59:0/839866406' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.349665+0000 mon.a (mon.0) 818 : audit [INF] from='client.? 172.21.15.59:0/512189418' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi059-82882-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.349750+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]': finished 2024-02-18T00:33:02.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.349871+0000 mon.a (mon.0) 820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350106+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350214+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350310+0000 mon.a (mon.0) 823 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350404+0000 mon.a (mon.0) 824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350482+0000 mon.a (mon.0) 825 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350559+0000 mon.a (mon.0) 826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350633+0000 mon.a (mon.0) 827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.350707+0000 mon.a (mon.0) 828 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.362588+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.402508+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.59:0/1844909298' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.557964+0000 mon.c (mon.2) 38 : audit [INF] from='client.? 172.21.15.59:0/683218601' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:01.879257+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.882879+0000 mon.a (mon.0) 830 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi059-83013-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.885065+0000 mon.a (mon.0) 831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.888812+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-18T00:33:02.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:01.889398+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:58.391682+0000 osd.6 (osd.6) 63 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:33:02.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:58.392439+0000 osd.6 (osd.6) 64 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:33:02.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:58.997662+0000 osd.0 (osd.0) 67 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:33:02.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:58.998746+0000 osd.0 (osd.0) 68 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:33:02.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.075292+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.59:0/1733075084' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.092439+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.59:0/1917601440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.413 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.101958+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.59:0/1001788920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:32:59.123013+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? 172.21.15.59:0/3681537941' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:59.767975+0000 mgr.y (mgr.24454) 149 : cluster [DBG] pgmap v143: 708 pgs: 51 creating+peering, 132 active+clean, 525 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 652 B/s rd, 0 op/s 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:59.799261+0000 osd.1 (osd.1) 85 : cluster [DBG] 6.1d deep-scrub starts 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:59.800343+0000 osd.1 (osd.1) 86 : cluster [DBG] 6.1d deep-scrub ok 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:59.990898+0000 osd.5 (osd.5) 73 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:32:59.991754+0000 osd.5 (osd.5) 74 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:00.004758+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.1a deep-scrub starts 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:00.013045+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.1a deep-scrub ok 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:00.916706+0000 mgr.y (mgr.24454) 150 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.148915+0000 mon.c (mon.2) 36 : audit [DBG] from='client.? 172.21.15.59:0/3145287853' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:01.315448+0000 mon.a (mon.0) 809 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.315615+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.322793+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.414 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.323028+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.324241+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.348762+0000 mon.a (mon.0) 814 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.348903+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.349193+0000 mon.a (mon.0) 816 : audit [INF] from='client.? 172.21.15.59:0/464889388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi059-82925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.349532+0000 mon.a (mon.0) 817 : audit [INF] from='client.? 172.21.15.59:0/839866406' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.349665+0000 mon.a (mon.0) 818 : audit [INF] from='client.? 172.21.15.59:0/512189418' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi059-82882-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.349750+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.349871+0000 mon.a (mon.0) 820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350106+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350214+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350310+0000 mon.a (mon.0) 823 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350404+0000 mon.a (mon.0) 824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350482+0000 mon.a (mon.0) 825 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350559+0000 mon.a (mon.0) 826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350633+0000 mon.a (mon.0) 827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.350707+0000 mon.a (mon.0) 828 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.362588+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.402508+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.59:0/1844909298' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.557964+0000 mon.c (mon.2) 38 : audit [INF] from='client.? 172.21.15.59:0/683218601' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:02.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:01.879257+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-18T00:33:02.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.882879+0000 mon.a (mon.0) 830 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi059-83013-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:02.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.885065+0000 mon.a (mon.0) 831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.888812+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-18T00:33:02.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:01.889398+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.572 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:02.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:02.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:02.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:02.677 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:02.725 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:02.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:58.391682+0000 osd.6 (osd.6) 63 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:33:02.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:58.392439+0000 osd.6 (osd.6) 64 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:33:02.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:58.997662+0000 osd.0 (osd.0) 67 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:33:02.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:58.998746+0000 osd.0 (osd.0) 68 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:33:02.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.075292+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.59:0/1733075084' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.092439+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.59:0/1917601440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.101958+0000 mon.c (mon.2) 34 : audit [INF] from='client.? 172.21.15.59:0/1001788920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:32:59.123013+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? 172.21.15.59:0/3681537941' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:59.767975+0000 mgr.y (mgr.24454) 149 : cluster [DBG] pgmap v143: 708 pgs: 51 creating+peering, 132 active+clean, 525 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 652 B/s rd, 0 op/s 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:59.799261+0000 osd.1 (osd.1) 85 : cluster [DBG] 6.1d deep-scrub starts 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:59.800343+0000 osd.1 (osd.1) 86 : cluster [DBG] 6.1d deep-scrub ok 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:59.990898+0000 osd.5 (osd.5) 73 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:32:59.991754+0000 osd.5 (osd.5) 74 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:00.004758+0000 osd.4 (osd.4) 85 : cluster [DBG] 4.1a deep-scrub starts 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:00.013045+0000 osd.4 (osd.4) 86 : cluster [DBG] 4.1a deep-scrub ok 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:00.916706+0000 mgr.y (mgr.24454) 150 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.148915+0000 mon.c (mon.2) 36 : audit [DBG] from='client.? 172.21.15.59:0/3145287853' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:02.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:01.315448+0000 mon.a (mon.0) 809 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.315615+0000 mon.a (mon.0) 810 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.322793+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.323028+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.324241+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.348762+0000 mon.a (mon.0) 814 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.348903+0000 mon.a (mon.0) 815 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi059-82893-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.349193+0000 mon.a (mon.0) 816 : audit [INF] from='client.? 172.21.15.59:0/464889388' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi059-82925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.349532+0000 mon.a (mon.0) 817 : audit [INF] from='client.? 172.21.15.59:0/839866406' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.349665+0000 mon.a (mon.0) 818 : audit [INF] from='client.? 172.21.15.59:0/512189418' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi059-82882-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.349750+0000 mon.a (mon.0) 819 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num", "val": "42"}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.349871+0000 mon.a (mon.0) 820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi059-82997-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350106+0000 mon.a (mon.0) 821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi059-82901-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350214+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi059-82989-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350310+0000 mon.a (mon.0) 823 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi059-82981-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350404+0000 mon.a (mon.0) 824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi059-82973-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350482+0000 mon.a (mon.0) 825 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi059-82957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350559+0000 mon.a (mon.0) 826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi059-82877-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350633+0000 mon.a (mon.0) 827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.350707+0000 mon.a (mon.0) 828 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi059-83061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.362588+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.402508+0000 mon.c (mon.2) 37 : audit [INF] from='client.? 172.21.15.59:0/1844909298' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.557964+0000 mon.c (mon.2) 38 : audit [INF] from='client.? 172.21.15.59:0/683218601' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:01.879257+0000 mon.a (mon.0) 829 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.882879+0000 mon.a (mon.0) 830 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi059-83013-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.885065+0000 mon.a (mon.0) 831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.888812+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-18T00:33:02.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:01.889398+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:02.842 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:02.846 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:02.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:02.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:02.983 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:03.007 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:03.011 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:03.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:03.113 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:03.149 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:03.433 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [----------] Global test environment set-up. 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5095 ms) 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ OK ] LibRadosService.RegisterLate (33 ms) 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: cluster: 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: id: d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: health: HEALTH_OK 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.434 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: services: 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 7m) 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 4m) 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: data: 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: pgs: 132 active+clean 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: io: 2024-02-18T00:33:03.435 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: cluster: 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: id: d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: health: HEALTH_OK 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: services: 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 7m) 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 4m) 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-18T00:33:03.436 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: data: 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: pgs: 132 active+clean 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: io: 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: cluster: 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: id: d129b4f8-cdf3-11ee-95ba-87774f69a715 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: health: HEALTH_OK 2024-02-18T00:33:03.437 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: services: 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 7m) 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 4m) 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: data: 2024-02-18T00:33:03.438 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: pools: 29 pools, 868 pgs 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: pgs: 22.465% pgs unknown 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 59.217% pgs not active 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 514 creating+peering 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 195 unknown 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 159 active+clean 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:03.439 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: io: 2024-02-18T00:33:03.603 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:03.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:03.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:03.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:33:03.708 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:03.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:01.768668+0000 mgr.y (mgr.24454) 151 : cluster [DBG] pgmap v145: 868 pgs: 514 creating+peering, 159 active+clean, 195 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:33:03.710 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:01.768668+0000 mgr.y (mgr.24454) 151 : cluster [DBG] pgmap v145: 868 pgs: 514 creating+peering, 159 active+clean, 195 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:33:03.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:01.768668+0000 mgr.y (mgr.24454) 151 : cluster [DBG] pgmap v145: 868 pgs: 514 creating+peering, 159 active+clean, 195 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:33:03.735 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:03.754 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:03.757 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:03.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:03.858 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:03.887 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:03.910 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:03.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:03.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:04.014 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:04.044 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:04.076 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:04.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:04.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:04.180 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:04.209 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:04.232 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:04.234 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:04.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:04.335 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:04.362 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:04.381 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:04.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:04.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:04.484 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:04.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.485+0000 7f8deb7fe700 1 -- 172.21.15.59:0/683218601 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v109) v1 ==== 87+0+0 (secure 0 0 0) 0x7f8dec050460 con 0x7f8df4102270 2024-02-18T00:33:04.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.489+0000 7f8dfc79c700 10 librados: watch_flush enter 2024-02-18T00:33:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.497+0000 7f8dfc79c700 10 librados: watch_flush exit 2024-02-18T00:33:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.497+0000 7f8dfc79c700 20 client.24962.objecter shutdown clearing up homeless session... 2024-02-18T00:33:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.497+0000 7f8dfc79c700 10 client.24962.objecter successfully canceled tick 2024-02-18T00:33:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.501+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8de0062c90 msgr2=0x7f8de0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.501+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/683218601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8de0062c90 0x7f8de0065150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f8df0004ee0 tx=0x7f8df000b040).stop 2024-02-18T00:33:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.501+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df4102270 msgr2=0x7f8df410a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.501+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df4102270 0x7f8df410a5d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f8dec00eb40 tx=0x7f8dec00b040).stop 2024-02-18T00:33:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.502+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 shutdown_connections 2024-02-18T00:33:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.502+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/683218601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8de0062c90 0x7f8de0065150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.502+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df40f8d10 0x7f8df4110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.502+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8df4102270 0x7f8df410a5d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.502+0000 7f8dfc79c700 1 --2- 172.21.15.59:0/683218601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df40f9670 0x7f8df4110920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.502+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 >> 172.21.15.59:0/683218601 conn(0x7f8df40f2820 msgr2=0x7f8df40f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:04.507 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-02-18T00:33:04.507 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-02-18T00:33:04.507 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-18T00:33:04.507 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-02-18T00:33:04.507 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (210 ms) 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (22 ms) 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (23 ms) 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (47 ms) 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (6 ms) 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-18T00:33:04.508 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (15 ms) 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (8 ms) 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (3 ms) 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (4 ms) 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (9 ms) 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-18T00:33:04.509 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (10 ms) 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (7 ms) 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (6 ms) 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (382 ms total) 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (13607 ms total) 2024-02-18T00:33:04.510 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-02-18T00:33:04.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.522+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 shutdown_connections 2024-02-18T00:33:04.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.524+0000 7f8dfc79c700 1 -- 172.21.15.59:0/683218601 wait complete. 2024-02-18T00:33:04.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:04.532+0000 7f8dfc79c700 1 librados: shutdown 2024-02-18T00:33:04.544 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:33:04.555 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:04.600 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:04.603 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:04.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:04.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:01.074266+0000 osd.0 (osd.0) 69 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:33:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:01.075165+0000 osd.0 (osd.0) 70 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:01.753584+0000 osd.7 (osd.7) 77 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:01.754413+0000 osd.7 (osd.7) 78 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:02.048203+0000 osd.4 (osd.4) 87 : cluster [DBG] 4.d scrub starts 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:02.051674+0000 osd.4 (osd.4) 88 : cluster [DBG] 4.d scrub ok 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:02.401645+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:02.401739+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:02.401876+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:02.408657+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.59:0/2985092008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:02.546635+0000 osd.3 (osd.3) 91 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:02.549072+0000 osd.3 (osd.3) 92 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:02.915705+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:02.915872+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:02.915972+0000 mon.a (mon.0) 839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.041032+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.042866+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.410338+0000 mon.a (mon.0) 840 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:03.431929+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? 172.21.15.59:0/1201162779' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:03.434476+0000 mon.a (mon.0) 841 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi059-83013-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.437309+0000 mon.a (mon.0) 842 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:03.444927+0000 mon.a (mon.0) 843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:03.450955+0000 mon.a (mon.0) 844 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:03.457784+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:03.969881+0000 mon.a (mon.0) 846 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:01.074266+0000 osd.0 (osd.0) 69 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:01.075165+0000 osd.0 (osd.0) 70 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:01.753584+0000 osd.7 (osd.7) 77 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:01.754413+0000 osd.7 (osd.7) 78 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:02.048203+0000 osd.4 (osd.4) 87 : cluster [DBG] 4.d scrub starts 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:02.051674+0000 osd.4 (osd.4) 88 : cluster [DBG] 4.d scrub ok 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:02.401645+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:02.401739+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:02.401876+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:02.408657+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.59:0/2985092008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:02.546635+0000 osd.3 (osd.3) 91 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:02.549072+0000 osd.3 (osd.3) 92 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:02.915705+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:02.915872+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:02.915972+0000 mon.a (mon.0) 839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.041032+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.042866+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.410338+0000 mon.a (mon.0) 840 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:03.431929+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? 172.21.15.59:0/1201162779' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:03.434476+0000 mon.a (mon.0) 841 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi059-83013-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.437309+0000 mon.a (mon.0) 842 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-18T00:33:04.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:03.444927+0000 mon.a (mon.0) 843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:04.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:03.450955+0000 mon.a (mon.0) 844 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:03.457784+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:03.969881+0000 mon.a (mon.0) 846 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-18T00:33:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:01.074266+0000 osd.0 (osd.0) 69 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:33:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:01.075165+0000 osd.0 (osd.0) 70 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:33:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:01.753584+0000 osd.7 (osd.7) 77 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:33:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:01.754413+0000 osd.7 (osd.7) 78 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:02.048203+0000 osd.4 (osd.4) 87 : cluster [DBG] 4.d scrub starts 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:02.051674+0000 osd.4 (osd.4) 88 : cluster [DBG] 4.d scrub ok 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:02.401645+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi059-83005-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:02.401739+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi059-82933-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:02.401876+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:02.408657+0000 mon.c (mon.2) 39 : audit [INF] from='client.? 172.21.15.59:0/2985092008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:02.546635+0000 osd.3 (osd.3) 91 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:02.549072+0000 osd.3 (osd.3) 92 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:02.915705+0000 mon.a (mon.0) 837 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83029-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:02.915872+0000 mon.a (mon.0) 838 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:02.915972+0000 mon.a (mon.0) 839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi059-82917-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.041032+0000 osd.0 (osd.0) 71 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.042866+0000 osd.0 (osd.0) 72 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:33:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.410338+0000 mon.a (mon.0) 840 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-18T00:33:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:03.431929+0000 mon.c (mon.2) 40 : audit [DBG] from='client.? 172.21.15.59:0/1201162779' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T00:33:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:03.434476+0000 mon.a (mon.0) 841 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi059-83013-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.437309+0000 mon.a (mon.0) 842 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-18T00:33:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:03.444927+0000 mon.a (mon.0) 843 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:03.450955+0000 mon.a (mon.0) 844 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:03.457784+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:03.969881+0000 mon.a (mon.0) 846 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-18T00:33:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [----------] Global test environment set-up. 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-18T00:33:04.806 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (12 ms) 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (5 ms) 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (5 ms) 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-18T00:33:04.807 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (10 ms) 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (8 ms) 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (5 ms) 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [----------] 12 tests from AsioRados (52 ms total) 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [----------] Global test environment tear-down 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (12900 ms total) 2024-02-18T00:33:04.808 INFO:tasks.workunit.client.0.smithi059.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_1_[83127]: starting. 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_1_[83127]: creating pool ceph_test_rados_list_parallel.smithi059-83069 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_1_[83127]: created object 0... 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_1_[83127]: created object 25... 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_1_[83127]: created object 49... 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_1_[83127]: finishing. 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_1_[83127]: shutting down. 2024-02-18T00:33:04.863 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_2_[83128]: starting. 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_2_[83128]: listing objects. 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_2_[83128]: listed object 0... 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_2_[83128]: listed object 25... 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_2_[83128]: saw 50 objects 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_2_[83128]: shutting down. 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_3_[85450]: starting. 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_3_[85450]: creating pool ceph_test_rados_list_parallel.smithi059-83069 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_3_[85450]: created object 0... 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_3_[85450]: created object 25... 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_3_[85450]: created object 49... 2024-02-18T00:33:04.864 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_3_[85450]: finishing. 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_3_[85450]: shutting down. 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_4_[85451]: starting. 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_4_[85451]: listing objects. 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_4_[85451]: listed object 0... 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_4_[85451]: listed object 25... 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_4_[85451]: saw 43 objects 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_4_[85451]: shutting down. 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_5_[85452]: starting. 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_5_[85452]: removed 25 objects... 2024-02-18T00:33:04.865 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_5_[85452]: removed half of the objects 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_5_[85452]: removed 50 objects... 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_5_[85452]: removed 50 objects 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_5_[85452]: shutting down. 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_6_[85732]: starting. 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_6_[85732]: creating pool ceph_test_rados_list_parallel.smithi059-83069 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_6_[85732]: created object 0... 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_6_[85732]: created object 25... 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_6_[85732]: created object 49... 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_6_[85732]: finishing. 2024-02-18T00:33:04.866 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_6_[85732]: shutting down. 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_7_[85733]: starting. 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_7_[85733]: listing objects. 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_7_[85733]: listed object 0... 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_7_[85733]: listed object 25... 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_7_[85733]: listed object 50... 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_7_[85733]: saw 53 objects 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_7_[85733]: shutting down. 2024-02-18T00:33:04.867 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_8_[85734]: starting. 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_8_[85734]: added 25 objects... 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_8_[85734]: added half of the objects 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_8_[85734]: added 50 objects... 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_8_[85734]: added 50 objects 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_8_[85734]: shutting down. 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.868 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:04.892 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:04.896 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:04.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:04.998 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:05.048 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:05.050 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:05.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:05.106 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ********************** api_list: [==========] Running 11 tests from 3 test suites. 2024-02-18T00:33:05.106 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] Global test environment set-up. 2024-02-18T00:33:05.106 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-18T00:33:05.106 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-18T00:33:05.106 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosList.ListObjects (1888 ms) 2024-02-18T00:33:05.106 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-18T00:33:05.106 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (31 ms) 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset foo1,foo2,foo3 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo1 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo2 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo3 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset foo1,foo4,foo5 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo4 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo5 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo1 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset foo6,foo7 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo7 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo6 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-18T00:33:05.107 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns1:foo4 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns1:foo5 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns2:foo7 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns2:foo6 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns1:foo1 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: :foo1 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: :foo2 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: :foo3 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (97 ms) 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 1 0 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 10 0 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 13 0 2024-02-18T00:33:05.108 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 7 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 14 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 0 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 15 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 11 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 5 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 8 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 6 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 3 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 4 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 12 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 9 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 2 0 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (79 ms) 2024-02-18T00:33:05.109 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: x cursor=MIN 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=1 cursor=26:02547ec2:::1:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=10 cursor=26:52ea6a34:::10:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=13 cursor=26:566253c9:::13:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=7 cursor=26:5c6b0b28:::7:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=14 cursor=26:62a1935d:::14:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=0 cursor=26:6cac518f:::0:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=15 cursor=26:863748b0:::15:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=11 cursor=26:89d3ae78:::11:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=5 cursor=26:b29083e3:::5:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=8 cursor=26:bd63b0f1:::8:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=6 cursor=26:c4fdafeb:::6:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=3 cursor=26:cfc208b3:::3:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=4 cursor=26:d83876eb:::4:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=12 cursor=26:de5d7c5f:::12:head 2024-02-18T00:33:05.110 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=9 cursor=26:e960b815:::9:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > oid=2 cursor=26:f905c69b:::2:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=1 cursor=26:02547ec2:::1:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:02547ec2:::1:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:02547ec2:::1:head -> 1 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=10 cursor=26:52ea6a34:::10:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:52ea6a34:::10:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:52ea6a34:::10:head -> 10 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=13 cursor=26:566253c9:::13:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:566253c9:::13:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:566253c9:::13:head -> 13 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=7 cursor=26:5c6b0b28:::7:head 2024-02-18T00:33:05.111 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:5c6b0b28:::7:head 2024-02-18T00:33:05.112 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:5c6b0b28:::7:head -> 7 2024-02-18T00:33:05.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:05.174 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=14 cursor=26:62a1935d:::14:head 2024-02-18T00:33:05.174 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:62a1935d:::14:head 2024-02-18T00:33:05.174 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:62a1935d:::14:head -> 14 2024-02-18T00:33:05.174 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=0 cursor=26:6cac518f:::0:head 2024-02-18T00:33:05.174 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:6cac518f:::0:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:6cac518f:::0:head -> 0 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=15 cursor=26:863748b0:::15:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:863748b0:::15:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:863748b0:::15:head -> 15 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=11 cursor=26:89d3ae78:::11:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:89d3ae78:::11:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:89d3ae78:::11:head -> 11 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=5 cursor=26:b29083e3:::5:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:b29083e3:::5:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:b29083e3:::5:head -> 5 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=8 cursor=26:bd63b0f1:::8:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:bd63b0f1:::8:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:bd63b0f1:::8:head -> 8 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=6 cursor=26:c4fdafeb:::6:head 2024-02-18T00:33:05.175 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:c4fdafeb:::6:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:c4fdafeb:::6:head -> 6 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=3 cursor=26:cfc208b3:::3:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:cfc208b3:::3:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:cfc208b3:::3:head -> 3 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=4 cursor=26:d83876eb:::4:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:d83876eb:::4:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:d83876eb:::4:head -> 4 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=12 cursor=26:de5d7c5f:::12:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:de5d7c5f:::12:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:de5d7c5f:::12:head -> 12 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=9 cursor=26:e960b815:::9:head 2024-02-18T00:33:05.176 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:e960b815:::9:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:e960b815:::9:head -> 9 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : oid=2 cursor=26:f905c69b:::2:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:f905c69b:::2:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:f905c69b:::2:head -> 2 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:5c6b0b28:::7:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:5c6b0b28:::7:head expected=26:5c6b0b28:::7:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:5c6b0b28:::7:head -> 7 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=7 expected=7 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:89d3ae78:::11:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:89d3ae78:::11:head expected=26:89d3ae78:::11:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:89d3ae78:::11:head -> 11 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=11 expected=11 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:863748b0:::15:head 2024-02-18T00:33:05.177 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:863748b0:::15:head expected=26:863748b0:::15:head 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:863748b0:::15:head -> 15 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=15 expected=15 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:6cac518f:::0:head 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:6cac518f:::0:head expected=26:6cac518f:::0:head 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:6cac518f:::0:head -> 0 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=0 expected=0 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:62a1935d:::14:head 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:62a1935d:::14:head expected=26:62a1935d:::14:head 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:62a1935d:::14:head -> 14 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=14 expected=14 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:f905c69b:::2:head 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:f905c69b:::2:head expected=26:f905c69b:::2:head 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:f905c69b:::2:head -> 2 2024-02-18T00:33:05.178 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=2 expected=2 2024-02-18T00:33:05.179 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:e960b815:::9:head 2024-02-18T00:33:05.179 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:e960b815:::9:head expected=26:e960b815:::9:head 2024-02-18T00:33:05.179 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:e960b815:::9:head -> 9 2024-02-18T00:33:05.179 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=9 expected=9 2024-02-18T00:33:05.179 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:de5d7c5f:::12:head 2024-02-18T00:33:05.179 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:de5d7c5f:::12:head expected=26:de5d7c5f:::12:head 2024-02-18T00:33:05.179 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:de5d7c5f:::12:head -> 12 2024-02-18T00:33:05.184 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:05.386 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:05.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:05.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:05.492 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24760.0:2 11.1f 11:f9b7e329:test-rados-api-smithi059-82973-2::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.486348+0000", 2024-02-18T00:33:05.526 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.293260332, 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.085234769000000002, 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24760", 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3991581620", 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486348+0000", 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486348+0000", 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486346+0000", 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7677999999999999e-05 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486364+0000", 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0370000000000001e-05 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486374+0000", 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3960000000000006e-06 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486382+0000", 2024-02-18T00:33:05.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000219044 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486601+0000", 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8040000000000002e-06 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.486609+0000", 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010230504 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.496840+0000", 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000119448 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.496959+0000", 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000154864 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497114+0000", 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020249207000000002 2024-02-18T00:33:05.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.517363+0000", 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022774626999999999 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.540138+0000", 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.031317190000000002 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.571455+0000", 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0222999999999994e-05 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.571535+0000", 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7284000000000003e-05 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.571583+0000", 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.085234769000000002 2024-02-18T00:33:05.533 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24841.0:2 16.d e106/104 16:b4e7e639:test-rados-api-smithi059-83061-3::testobj:head v 106'1, mlcod=0'0)", 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.497464+0000", 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.2821441280000001, 2024-02-18T00:33:05.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044963041000000002, 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497464+0000", 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497464+0000", 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:05.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497464+0000", 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0304e-05 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497474+0000", 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5339999999999999e-06 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497479+0000", 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8510000000000004e-06 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497485+0000", 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00063065699999999996 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.498116+0000", 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6269999999999998e-05 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.498132+0000", 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044268123999999999 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.542400+0000", 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6650000000000001e-05 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.542427+0000", 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044963041000000002 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24817.0:2 14.19 e106/104 14:9a945d6d:test-rados-api-smithi059-82981-2::foo:head v 106'1, mlcod=0'0)", 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.497644+0000", 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.2819636870000002, 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042076765000000002, 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497644+0000", 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497644+0000", 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497644+0000", 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3510000000000004e-06 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497648+0000", 2024-02-18T00:33:05.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9570000000000001e-06 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497653+0000", 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6290000000000001e-06 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.497658+0000", 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00097780999999999996 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.498636+0000", 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5201999999999999e-05 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.498651+0000", 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.04104153 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.539693+0000", 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8595999999999999e-05 2024-02-18T00:33:05.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.539721+0000", 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.042076765000000002 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop_reply(client.24760.0:2 11.1f e106/104 ondisk, result = 0)", 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.511156+0000", 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.2684515869999999, 2024-02-18T00:33:05.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0062098589999999999, 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.511156+0000", 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.511156+0000", 2024-02-18T00:33:05.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.511156+0000", 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6060000000000002e-06 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.511161+0000", 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3220000000000004e-06 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.511169+0000", 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0090000000000001e-06 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.511176+0000", 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061709379999999999 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.517346+0000", 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9819999999999997e-06 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.517355+0000", 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0900000000000001e-05 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.517366+0000", 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0062098589999999999 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24841.0:7 16.17 16:e8385c9e:test-rados-api-smithi059-83061-4::testobj:head [write 0~8] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.586392+0000", 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.193215919, 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012152002, 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24841", 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1293437097", 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 7 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586392+0000", 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586392+0000", 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586391+0000", 2024-02-18T00:33:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0849999999999997e-06 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586399+0000", 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0967e-05 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586410+0000", 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1829999999999999e-06 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586416+0000", 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1254000000000002e-05 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586458+0000", 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000112184 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586570+0000", 2024-02-18T00:33:05.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014782099999999999 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,1", 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.586718+0000", 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032502970000000001 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.589968+0000", 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7961e-05 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.589986+0000", 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084653060000000006 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.598451+0000", 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2782999999999999e-05 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.598504+0000", 2024-02-18T00:33:05.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0210000000000003e-05 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.598544+0000", 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012152002 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24766.0:25 12.a 12:53b30445:test-rados-api-smithi059-82901-9::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.628591+0000", 2024-02-18T00:33:05.554 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 3.151017408, 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0078502610000000007, 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24766", 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1346273899", 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 25 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628591+0000", 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628591+0000", 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628588+0000", 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2928999999999999e-05 2024-02-18T00:33:05.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628601+0000", 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3458000000000001e-05 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628624+0000", 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0509999999999999e-05 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628635+0000", 2024-02-18T00:33:05.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3893000000000001e-05 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628689+0000", 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014219499999999999 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628831+0000", 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000150147 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,3", 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.628981+0000", 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049170799999999999 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.629473+0000", 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001814373 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.631287+0000", 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0050979329999999998 2024-02-18T00:33:05.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.636385+0000", 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3386000000000002e-05 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.636418+0000", 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2433999999999999e-05 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.636441+0000", 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0078502610000000007 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:303 17.f e106/104 17:f7837858:test-rados-api-smithi059-82949-15::foo:head v 106'129, mlcod=106'127)", 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.895391+0000", 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 2.884217145, 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053366949999999998, 2024-02-18T00:33:05.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895391+0000", 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895391+0000", 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895390+0000", 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7950000000000008e-06 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895398+0000", 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5340000000000005e-06 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895407+0000", 2024-02-18T00:33:05.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8689999999999998e-06 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895414+0000", 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0987000000000003e-05 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895485+0000", 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1107999999999998e-05 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.895506+0000", 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00517952 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.900686+0000", 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1681000000000003e-05 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.900728+0000", 2024-02-18T00:33:05.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053366949999999998 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24853.0:2 21.15 21:ad551702:::test:head [stat] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.454911+0000", 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 2.324696629, 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.080637693999999996, 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:05.566 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24853", 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1917601440", 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454911+0000", 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454911+0000", 2024-02-18T00:33:05.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454910+0000", 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8229999999999997e-06 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454917+0000", 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7750000000000004e-06 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454926+0000", 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5510000000000002e-06 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454934+0000", 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5192999999999999e-05 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454979+0000", 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2559999999999996e-06 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.454986+0000", 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.080444880999999996 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.535431+0000", 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000117827 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.535549+0000", 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.080637693999999996 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.570 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:977 17.f e107/104 17:f7837858:test-rados-api-smithi059-82949-15::foo:head v 107'466, mlcod=106'464)", 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.499420+0000", 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 2.2801883950000001, 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084731719999999993, 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.499420+0000", 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.499420+0000", 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.499419+0000", 2024-02-18T00:33:05.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5289999999999999e-06 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.499425+0000", 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9710000000000001e-06 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.499431+0000", 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3219999999999998e-06 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.499437+0000", 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0055022810000000004 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.504939+0000", 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1421e-05 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.504961+0000", 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0029044840000000001 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.507865+0000", 2024-02-18T00:33:05.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7645e-05 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.507893+0000", 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0084731719999999993 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:1081 17.f e107/104 17:f7837858:test-rados-api-smithi059-82949-15::foo:head v 107'518, mlcod=107'516)", 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.666257+0000", 2024-02-18T00:33:05.575 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 2.1133513640000001, 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040797910000000001, 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666257+0000", 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666257+0000", 2024-02-18T00:33:05.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:05.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666255+0000", 2024-02-18T00:33:05.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2199e-05 2024-02-18T00:33:05.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666267+0000", 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6458999999999999e-05 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666284+0000", 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7389999999999999e-06 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.578 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666292+0000", 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.58e-05 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666357+0000", 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5289000000000001e-05 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.666383+0000", 2024-02-18T00:33:05.579 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039230150000000002 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.670306+0000", 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0719999999999997e-05 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.670336+0000", 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0040797910000000001 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:1235 17.f e107/104 17:f7837858:test-rados-api-smithi059-82949-15::foo:head v 107'595, mlcod=107'593)", 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.812301+0000", 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.967307073, 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047060449999999998, 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.581 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812301+0000", 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812301+0000", 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812300+0000", 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2880000000000004e-06 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.582 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812307+0000", 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1482e-05 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812319+0000", 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4520000000000002e-06 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.583 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812325+0000", 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7510000000000003e-05 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812383+0000", 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8236999999999998e-05 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.812401+0000", 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0045784450000000004 2024-02-18T00:33:05.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.816979+0000", 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7670000000000001e-05 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.817007+0000", 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047060449999999998 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.585 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:1921 17.f e107/104 17:f7837858:test-rados-api-smithi059-82949-15::foo:head v 107'938, mlcod=107'936)", 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.443388+0000", 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.33622029, 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0054991220000000004, 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443388+0000", 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.586 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443388+0000", 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443387+0000", 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3799999999999999e-06 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.587 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443394+0000", 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2570000000000001e-06 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443401+0000", 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9040000000000003e-06 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443408+0000", 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000158317 2024-02-18T00:33:05.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443566+0000", 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9927000000000001e-05 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.443586+0000", 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0052439699999999997 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.589 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.448830+0000", 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6886999999999999e-05 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.448887+0000", 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0054991220000000004 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25003.0:3 22.16 22:681f27cd:::2.obj:head [write 0~105] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:33:05.590 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.491357+0000", 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.288250634, 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047604539999999999, 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25003", 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/445777972", 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 3 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491357+0000", 2024-02-18T00:33:05.591 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491357+0000", 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491356+0000", 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1909999999999997e-06 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491362+0000", 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0530000000000001e-05 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491372+0000", 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3369999999999996e-06 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491379+0000", 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9910000000000002e-05 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491419+0000", 2024-02-18T00:33:05.593 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1566999999999999e-05 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491510+0000", 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015843500000000001 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,7", 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.491669+0000", 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00088947699999999995 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.492558+0000", 2024-02-18T00:33:05.594 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0011178049999999999 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.493676+0000", 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023853429999999998 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.496061+0000", 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2679999999999999e-05 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.496094+0000", 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3881999999999999e-05 2024-02-18T00:33:05.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.496118+0000", 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0047604539999999999 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25003.0:24 22.16 22:69f6b4d7:::23.obj:head [write 0~141] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:33:05.596 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.545633+0000", 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 1.2339745879999999, 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037255439999999999, 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25003", 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/445777972", 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 24 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545633+0000", 2024-02-18T00:33:05.597 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545633+0000", 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545632+0000", 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2269999999999993e-06 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.598 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545640+0000", 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0017000000000001e-05 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545650+0000", 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0260000000000004e-06 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545656+0000", 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3513000000000003e-05 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.599 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545690+0000", 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6669999999999999e-05 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545736+0000", 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5740000000000002e-05 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.600 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,7", 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.545822+0000", 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003413892 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.549236+0000", 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.272e-06 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.549243+0000", 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9264999999999998e-05 2024-02-18T00:33:05.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.549272+0000", 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8609999999999997e-05 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.549321+0000", 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7988999999999998e-05 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.602 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.549359+0000", 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0037255439999999999 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.5.0:88 17.19 e108/104 -19:9c638acd:::temp_17.19_0_24256_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.871627+0000", 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.90798151199999999, 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021493643999999999, 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.603 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.871627+0000", 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.871627+0000", 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.871624+0000", 2024-02-18T00:33:05.604 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0089847120000000006 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.880609+0000", 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5344e-05 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.880634+0000", 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0404999999999999e-05 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.605 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.880645+0000", 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7953e-05 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.880693+0000", 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7008000000000002e-05 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.880730+0000", 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012313456 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.606 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.893043+0000", 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7053999999999998e-05 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.893120+0000", 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021493643999999999 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.5.0:89 17.19 e108/104 -19:9c638acd:::temp_17.19_0_24256_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.894328+0000", 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.885279598, 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019789787, 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.894328+0000", 2024-02-18T00:33:05.608 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.894328+0000", 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.894327+0000", 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0094669369999999999 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.609 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.903794+0000", 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.281e-05 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.903816+0000", 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6090000000000001e-06 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.903825+0000", 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4752e-05 2024-02-18T00:33:05.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.903880+0000", 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7396e-05 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.903907+0000", 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010133918 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.914041+0000", 2024-02-18T00:33:05.611 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7212999999999994e-05 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.914118+0000", 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.019789787 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.5.0:90 17.19 e108/104 -19:9c638acd:::temp_17.19_0_24256_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.913429+0000", 2024-02-18T00:33:05.612 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.866179425, 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017674481999999998, 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.913429+0000", 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.913429+0000", 2024-02-18T00:33:05.613 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.913427+0000", 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0080309060000000008 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921458+0000", 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2487e-05 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.614 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921480+0000", 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6806000000000001e-05 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921497+0000", 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5045e-05 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921542+0000", 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7518000000000001e-05 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.615 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921560+0000", 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0095103779999999999 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.931070+0000", 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3009999999999997e-05 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.931103+0000", 2024-02-18T00:33:05.616 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017674481999999998 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2064 17.19 e108/104 17:9c638acd:test-rados-api-smithi059-82949-16::big.copy2:head v 108'1, mlcod=0'0)", 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.921518+0000", 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.85809049999999998, 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.009638008, 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.617 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921518+0000", 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921518+0000", 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.618 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921516+0000", 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4989999999999999e-06 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921523+0000", 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8420000000000004e-06 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921528+0000", 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0149999999999999e-05 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.619 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.921538+0000", 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00334637 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.924884+0000", 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4387e-05 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.620 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.924898+0000", 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0062249760000000001 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.931123+0000", 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.2193000000000002e-05 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.931156+0000", 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.009638008 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2069 17.3 e108/104 17:c1ec3694:test-rados-api-smithi059-82949-17::big:head v 108'1, mlcod=0'0)", 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.047081+0000", 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.73252686499999997, 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059647784000000002, 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.047081+0000", 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.622 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.047081+0000", 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.047080+0000", 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.029251637 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.076332+0000", 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2527999999999997e-05 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.623 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.076374+0000", 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7289999999999999e-05 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.076392+0000", 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0770999999999998e-05 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.624 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.076452+0000", 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9717000000000001e-05 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.076472+0000", 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030214181 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.106686+0000", 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2789e-05 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.625 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.106729+0000", 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059647784000000002 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.25025.0:13 22.14 e108/105 22:2bc39789:::34.obj2:head v 108'1, mlcod=0'0)", 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.128457+0000", 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.65115115899999998, 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0051709520000000004, 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:05.626 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128457+0000", 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128457+0000", 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:05.627 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128455+0000", 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2454e-05 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128468+0000", 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0775e-05 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128488+0000", 2024-02-18T00:33:05.628 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.328e-06 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128496+0000", 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3652999999999997e-05 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128539+0000", 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7450999999999999e-05 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.629 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.128557+0000", 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0050264519999999998 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.133583+0000", 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4487000000000003e-05 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.133628+0000", 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0051709520000000004 2024-02-18T00:33:05.630 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:05.631 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:05.631 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:33:05.632 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:05.632 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:05.633 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:05.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:05.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.043639+0000 osd.4 (osd.4) 89 : cluster [DBG] 2.f scrub starts 2024-02-18T00:33:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.047124+0000 osd.4 (osd.4) 90 : cluster [DBG] 2.f scrub ok 2024-02-18T00:33:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.769868+0000 mgr.y (mgr.24454) 152 : cluster [DBG] pgmap v148: 886 pgs: 418 creating+peering, 159 active+clean, 309 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:33:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.337751+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.338482+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.339002+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.339481+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.339966+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.340464+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.351800+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.354557+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.357118+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.359850+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:04.455832+0000 mon.a (mon.0) 847 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.474758+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.474846+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.474952+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.475335+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.511245+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.525111+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.59:0/2984699099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.598510+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.628508+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.658036+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.040475+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.044443+0000 mon.a (mon.0) 853 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.044959+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.045809+0000 mon.a (mon.0) 855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.043639+0000 osd.4 (osd.4) 89 : cluster [DBG] 2.f scrub starts 2024-02-18T00:33:05.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.047124+0000 osd.4 (osd.4) 90 : cluster [DBG] 2.f scrub ok 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.769868+0000 mgr.y (mgr.24454) 152 : cluster [DBG] pgmap v148: 886 pgs: 418 creating+peering, 159 active+clean, 309 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.337751+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.338482+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.339002+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.339481+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.339966+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.340464+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.351800+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.354557+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:05.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.357118+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.359850+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:04.455832+0000 mon.a (mon.0) 847 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.474758+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.474846+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.474952+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.475335+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.511245+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.525111+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.59:0/2984699099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.598510+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.628508+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.658036+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.040475+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-18T00:33:05.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.044443+0000 mon.a (mon.0) 853 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:05.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.044959+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.045809+0000 mon.a (mon.0) 855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:05.763 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:05.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:05.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:05.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:05.920 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:06.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.043639+0000 osd.4 (osd.4) 89 : cluster [DBG] 2.f scrub starts 2024-02-18T00:33:06.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.047124+0000 osd.4 (osd.4) 90 : cluster [DBG] 2.f scrub ok 2024-02-18T00:33:06.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.769868+0000 mgr.y (mgr.24454) 152 : cluster [DBG] pgmap v148: 886 pgs: 418 creating+peering, 159 active+clean, 309 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:33:06.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.337751+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.338482+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.339002+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.339481+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.339966+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.340464+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.351800+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.354557+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.357118+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.359850+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:04.455832+0000 mon.a (mon.0) 847 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.474758+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.474846+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:06.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.474952+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83021-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.475335+0000 mon.a (mon.0) 851 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.511245+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.525111+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.59:0/2984699099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.598510+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.628508+0000 mon.c (mon.2) 41 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.658036+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.040475+0000 mon.a (mon.0) 852 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.044443+0000 mon.a (mon.0) 853 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.044959+0000 mon.a (mon.0) 854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:06.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.045809+0000 mon.a (mon.0) 855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:06.023 INFO:tasks.workunit.client.0.smithi059.stdout: api_lis********* 2024-02-18T00:33:06.023 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_9_[85862]: starting. 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_9_[85862]: creating pool ceph_test_rados_list_parallel.smithi059-83069 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_9_[85862]: created object 0... 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_9_[85862]: created object 25... 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_9_[85862]: created object 49... 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_9_[85862]: finishing. 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_9_[85862]: shutting down. 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: starting. 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: listing objects. 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: listed object 0... 2024-02-18T00:33:06.024 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: listed object 25... 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: listed object 50... 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: listed object 75... 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: saw 97 objects 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_10_[85863]: shutting down. 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_11_[85864]: starting. 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_11_[85864]: added 25 objects... 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_11_[85864]: added half of the objects 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_11_[85864]: added 50 objects... 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_11_[85864]: added 50 objects 2024-02-18T00:33:06.025 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_11_[85864]: shutting down. 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_13_[85866]: starting. 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_13_[85866]: removed 25 objects... 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_13_[85866]: removed half of the objects 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_13_[85866]: removed 50 objects... 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_13_[85866]: removed 50 objects 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_13_[85866]: shutting down. 2024-02-18T00:33:06.026 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_12_[85865]: starting. 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_12_[85865]: added 25 objects... 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_12_[85865]: added half of the objects 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_12_[85865]: added 50 objects... 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_12_[85865]: added 50 objects 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_12_[85865]: shutting down. 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.027 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_14_[86147]: starting. 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_14_[86147]: creating pool ceph_test_rados_list_parallel.smithi059-83069 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_14_[86147]: created object 0... 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_14_[86147]: created object 25... 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_14_[86147]: created object 49... 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_14_[86147]: finishing. 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_14_[86147]: shutting down. 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: starting. 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: listing objects. 2024-02-18T00:33:06.028 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: listed object 0... 2024-02-18T00:33:06.270 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:06.271 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-18T00:33:06.272 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:06.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:06.374 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:06.401 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:06.421 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:06.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:06.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:06.525 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:06.551 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:06.569 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:06.572 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:06.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:06.673 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:06.700 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:06.718 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:06.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:06.721 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:06.822 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:06.848 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:06.866 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:06.869 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:06.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:06.900 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: listed object 2 api_cmd_pp: Running main() from gmock_main.cc 2024-02-18T00:33:06.900 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-18T00:33:06.900 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-18T00:33:06.900 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (120 ms) 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (888 ms) 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (14975 ms) 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (15983 ms total) 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (15983 ms total) 2024-02-18T00:33:06.901 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (8159 ms) 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-18T00:33:06.948 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (70 ms) 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (7771 ms) 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.502497+0000 mon.a [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.508603+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.514667+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.514806+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.544960+0000 mon.b [INF] from='client.? 172.21.15.59:0/2631054062' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.976351+0000 mon.c [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.976849+0000 mon.a [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:06.949 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.980427+0000 mon.a [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:06.950 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.980962+0000 mon.a [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:06.950 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.981440+0000 mon.a [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:06.950 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.982173+0000 mon.a [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:06.950 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.985284+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:06.950 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:05.986646+0000 mon.b [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:06.970 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.807828+0000 osd.7 (osd.7) 79 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:33:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:03.808883+0000 osd.7 (osd.7) 80 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.337871+0000 mgr.y (mgr.24454) 153 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.338516+0000 mgr.y (mgr.24454) 154 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.339035+0000 mgr.y (mgr.24454) 155 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.339501+0000 mgr.y (mgr.24454) 156 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.339989+0000 mgr.y (mgr.24454) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.340510+0000 mgr.y (mgr.24454) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.351954+0000 mgr.y (mgr.24454) 159 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.355623+0000 mgr.y (mgr.24454) 160 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.358504+0000 mgr.y (mgr.24454) 161 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:04.361378+0000 mgr.y (mgr.24454) 162 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:07.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.251706+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.502497+0000 mon.a (mon.0) 856 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.508603+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.514667+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.514806+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.544960+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.59:0/2631054062' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.772127+0000 mgr.y (mgr.24454) 163 : cluster [DBG] pgmap v151: 750 pgs: 49 creating+peering, 686 active+clean, 15 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 693 op/s; 218 B/s, 0 objects/s recovering 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.975956+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.976351+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.976849+0000 mon.a (mon.0) 861 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:07.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.980427+0000 mon.a (mon.0) 862 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.980962+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.981440+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.982173+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.985284+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:05.986646+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.252741+0000 mon.b (mon.1) 61 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.426756+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.427602+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:07.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.428430+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:07.019 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.807828+0000 osd.7 (osd.7) 79 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:03.808883+0000 osd.7 (osd.7) 80 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.337871+0000 mgr.y (mgr.24454) 153 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.338516+0000 mgr.y (mgr.24454) 154 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.339035+0000 mgr.y (mgr.24454) 155 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.339501+0000 mgr.y (mgr.24454) 156 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.339989+0000 mgr.y (mgr.24454) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.340510+0000 mgr.y (mgr.24454) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.351954+0000 mgr.y (mgr.24454) 159 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.355623+0000 mgr.y (mgr.24454) 160 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.358504+0000 mgr.y (mgr.24454) 161 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:04.361378+0000 mgr.y (mgr.24454) 162 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.251706+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.502497+0000 mon.a (mon.0) 856 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.508603+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.514667+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.807828+0000 osd.7 (osd.7) 79 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:03.808883+0000 osd.7 (osd.7) 80 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:33:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.337871+0000 mgr.y (mgr.24454) 153 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.338516+0000 mgr.y (mgr.24454) 154 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.339035+0000 mgr.y (mgr.24454) 155 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.339501+0000 mgr.y (mgr.24454) 156 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.339989+0000 mgr.y (mgr.24454) 157 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.340510+0000 mgr.y (mgr.24454) 158 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.351954+0000 mgr.y (mgr.24454) 159 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.355623+0000 mgr.y (mgr.24454) 160 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.358504+0000 mgr.y (mgr.24454) 161 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:04.361378+0000 mgr.y (mgr.24454) 162 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.251706+0000 mon.b (mon.1) 58 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.502497+0000 mon.a (mon.0) 856 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.508603+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:07.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.514667+0000 mon.a (mon.0) 858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.514806+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.544960+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.59:0/2631054062' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.772127+0000 mgr.y (mgr.24454) 163 : cluster [DBG] pgmap v151: 750 pgs: 49 creating+peering, 686 active+clean, 15 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 693 op/s; 218 B/s, 0 objects/s recovering 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.975956+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.976351+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.976849+0000 mon.a (mon.0) 861 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.980427+0000 mon.a (mon.0) 862 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.980962+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:07.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.981440+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:07.256 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.982173+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:07.256 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.985284+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:07.256 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:05.986646+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:07.256 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.252741+0000 mon.b (mon.1) 61 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:07.256 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.426756+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:07.256 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.427602+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:07.256 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.428430+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:07.259 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.514806+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi059-82877-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:07.259 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.544960+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.59:0/2631054062' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.772127+0000 mgr.y (mgr.24454) 163 : cluster [DBG] pgmap v151: 750 pgs: 49 creating+peering, 686 active+clean, 15 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 693 op/s; 218 B/s, 0 objects/s recovering 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.975956+0000 mon.a (mon.0) 860 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.976351+0000 mon.c (mon.2) 42 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.976849+0000 mon.a (mon.0) 861 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.980427+0000 mon.a (mon.0) 862 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.980962+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.981440+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:07.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.982173+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:07.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.985284+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:07.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:05.986646+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:07.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.252741+0000 mon.b (mon.1) 61 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:07.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.426756+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:07.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.427602+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:07.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.428430+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:07.267 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:07.272 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:07.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:07.284 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:07.374 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:07.401 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:07.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-15... 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: listed object 50... 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: listed object 75... 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: listed object 100... 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: listed object 125... 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: saw 150 objects 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_15_[86148]: shutting down. 2024-02-18T00:33:07.433 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_16_[86149]: starting. 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_16_[86149]: added 25 objects... 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_16_[86149]: added half of the objects 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_16_[86149]: added 50 objects... 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_16_[86149]: added 50 objects 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: process_16_[86149]: shutting down. 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.434 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******************************* 2024-02-18T00:33:07.435 INFO:tasks.workunit.client.0.smithi059.stdout: list_parallel: ******* SUCCESS ********** 2024-02-18T00:33:07.586 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:33:07.586 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:33:07.587 INFO:tasks.thrashosds.thrasher:Shrinking pool iscsi 2024-02-18T00:33:07.587 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool iscsi by 10 min 8 2024-02-18T00:33:07.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:33:08.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:04.985589+0000 osd.4 (osd.4) 91 : cluster [DBG] 17.2 deep-scrub starts 2024-02-18T00:33:08.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:04.987862+0000 osd.4 (osd.4) 92 : cluster [DBG] 17.2 deep-scrub ok 2024-02-18T00:33:08.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.820320+0000 osd.7 (osd.7) 81 : cluster [DBG] 17.0 deep-scrub starts 2024-02-18T00:33:08.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.821670+0000 osd.7 (osd.7) 82 : cluster [DBG] 17.0 deep-scrub ok 2024-02-18T00:33:08.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.968784+0000 osd.0 (osd.0) 73 : cluster [DBG] 17.1 deep-scrub starts 2024-02-18T00:33:08.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.970216+0000 osd.4 (osd.4) 93 : cluster [DBG] 17.3 deep-scrub starts 2024-02-18T00:33:08.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:05.970526+0000 osd.0 (osd.0) 74 : cluster [DBG] 17.1 deep-scrub ok 2024-02-18T00:33:08.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.020443+0000 osd.4 (osd.4) 94 : cluster [DBG] 17.3 deep-scrub ok 2024-02-18T00:33:08.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.869836+0000 mon.a (mon.0) 869 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:08.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.880468+0000 mon.a (mon.0) 870 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.880553+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.350 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.880610+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.880810+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:08.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.881574+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-18T00:33:08.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.894279+0000 mon.a (mon.0) 875 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:08.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.895234+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.351 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.895911+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.59:0/1278661838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.895947+0000 mon.a (mon.0) 876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.904236+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.904831+0000 mon.a (mon.0) 878 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.905317+0000 mon.a (mon.0) 879 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.906814+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.907018+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:08.352 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.907743+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.908039+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.921276+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.921665+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.922047+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.353 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.922624+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.928323+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.928890+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.938692+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-18T00:33:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.939590+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T00:33:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.945601+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.354 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.947001+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:06.949140+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:07.253777+0000 mon.b (mon.1) 67 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:08.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:07.404270+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:07.410537+0000 mon.a (mon.0) 890 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.328402+0000 mon.a (mon.0) 891 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:08.356 INFO:tasks.workunit.client.0.smithi059.stdout:8T00:33:06.426756+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:08.356 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.427602+0000 mon.b [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:08.356 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.428430+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.357 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.869836+0000 mon.a [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:08.357 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.880468+0000 mon.a [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.357 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.880553+0000 mon.a [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.357 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.880610+0000 mon.a [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.357 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.880810+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:08.358 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.894279+0000 mon.a [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:08.358 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.895234+0000 mon.b [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.358 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.895911+0000 mon.b [INF] from='client.? 172.21.15.59:0/1278661838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.358 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.895947+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:08.359 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.904236+0000 mon.a [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:08.359 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.904831+0000 mon.a [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:08.359 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.905317+0000 mon.a [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.359 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.906814+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:08.359 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.907018+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:08.360 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.907743+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.360 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.908039+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.360 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.921276+0000 mon.b [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.361 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.921665+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.361 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.922047+0000 mon.c [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.361 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.922624+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.362 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.928323+0000 mon.b [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.362 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.928890+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.362 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.938692+0000 client.admin [INF] onexx 2024-02-18T00:33:08.362 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.939590+0000 mon.a [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T00:33:08.362 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.945601+0000 mon.c [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.362 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.947001+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.363 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:06.949140+0000 mon.c [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.363 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:07.404270+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.363 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:07.410537+0000 mon.a [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:04.985589+0000 osd.4 (osd.4) 91 : cluster [DBG] 17.2 deep-scrub starts 2024-02-18T00:33:08.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:04.987862+0000 osd.4 (osd.4) 92 : cluster [DBG] 17.2 deep-scrub ok 2024-02-18T00:33:08.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.820320+0000 osd.7 (osd.7) 81 : cluster [DBG] 17.0 deep-scrub starts 2024-02-18T00:33:08.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.821670+0000 osd.7 (osd.7) 82 : cluster [DBG] 17.0 deep-scrub ok 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.968784+0000 osd.0 (osd.0) 73 : cluster [DBG] 17.1 deep-scrub starts 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.970216+0000 osd.4 (osd.4) 93 : cluster [DBG] 17.3 deep-scrub starts 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:05.970526+0000 osd.0 (osd.0) 74 : cluster [DBG] 17.1 deep-scrub ok 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.020443+0000 osd.4 (osd.4) 94 : cluster [DBG] 17.3 deep-scrub ok 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.869836+0000 mon.a (mon.0) 869 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.880468+0000 mon.a (mon.0) 870 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.880553+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.880610+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.880810+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.881574+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.894279+0000 mon.a (mon.0) 875 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.895234+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.895911+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.59:0/1278661838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.895947+0000 mon.a (mon.0) 876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.904236+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.904831+0000 mon.a (mon.0) 878 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.905317+0000 mon.a (mon.0) 879 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.906814+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.907018+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.907743+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.908039+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.921276+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.921665+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.922047+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.922624+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.928323+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.928890+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.938692+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.939590+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.945601+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.947001+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:06.949140+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:07.253777+0000 mon.b (mon.1) 67 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:07.404270+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:07.410537+0000 mon.a (mon.0) 890 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.328402+0000 mon.a (mon.0) 891 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:04.985589+0000 osd.4 (osd.4) 91 : cluster [DBG] 17.2 deep-scrub starts 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:04.987862+0000 osd.4 (osd.4) 92 : cluster [DBG] 17.2 deep-scrub ok 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.820320+0000 osd.7 (osd.7) 81 : cluster [DBG] 17.0 deep-scrub starts 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.821670+0000 osd.7 (osd.7) 82 : cluster [DBG] 17.0 deep-scrub ok 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.968784+0000 osd.0 (osd.0) 73 : cluster [DBG] 17.1 deep-scrub starts 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.970216+0000 osd.4 (osd.4) 93 : cluster [DBG] 17.3 deep-scrub starts 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:05.970526+0000 osd.0 (osd.0) 74 : cluster [DBG] 17.1 deep-scrub ok 2024-02-18T00:33:08.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.020443+0000 osd.4 (osd.4) 94 : cluster [DBG] 17.3 deep-scrub ok 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.869836+0000 mon.a (mon.0) 869 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.880468+0000 mon.a (mon.0) 870 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.880553+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.880610+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi059-82925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.880810+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.881574+0000 mon.a (mon.0) 874 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.894279+0000 mon.a (mon.0) 875 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]: dispatch 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.895234+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.895911+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.59:0/1278661838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.895947+0000 mon.a (mon.0) 876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.904236+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.904831+0000 mon.a (mon.0) 878 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.905317+0000 mon.a (mon.0) 879 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.906814+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.907018+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.907743+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.908039+0000 mon.a (mon.0) 883 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.921276+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.921665+0000 mon.a (mon.0) 884 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.922047+0000 mon.c (mon.2) 43 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.922624+0000 mon.a (mon.0) 885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.928323+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.928890+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.938692+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.939590+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.945601+0000 mon.c (mon.2) 44 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.947001+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:06.949140+0000 mon.c (mon.2) 45 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:07.253777+0000 mon.b (mon.1) 67 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:08.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:07.404270+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:08.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:07.410537+0000 mon.a (mon.0) 890 : audit [INF] from='client.? 172.21.15.59:0/4176758285' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi059-83013-1"}]': finished 2024-02-18T00:33:08.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.328402+0000 mon.a (mon.0) 891 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:08.845 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:08.866 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:08.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:08.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:09.004 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:09.096 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:09.098 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:09.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:09.100 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:09.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:09.228 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_1_[83129]: starting. 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_1_[83129]: creating pool ceph_test_rados_open_pools_parallel.smithi059-83077 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_1_[83129]: created object 0... 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_1_[83129]: created object 25... 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_1_[83129]: created object 49... 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_1_[83129]: finishing. 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_1_[83129]: shutting down. 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_2_[83130]: starting. 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_2_[83130]: rados_pool_create. 2024-02-18T00:33:09.304 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_2_[83130]: rados_ioctx_create. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_2_[83130]: shutting down. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: ******************************* 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_3_[85966]: starting. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_3_[85966]: creating pool ceph_test_rados_open_pools_parallel.smithi059-83077 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_3_[85966]: created object 0... 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_3_[85966]: created object 25... 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_3_[85966]: created object 49... 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_3_[85966]: finishing. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_3_[85966]: shutting down. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: ******************************* 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_4_[85967]: starting. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_4_[85967]: rados_pool_create. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_4_[85967]: rados_ioctx_create. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: process_4_[85967]: shutting down. 2024-02-18T00:33:09.305 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: ******************************* 2024-02-18T00:33:09.306 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: ******************************* 2024-02-18T00:33:09.306 INFO:tasks.workunit.client.0.smithi059.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-18T00:33:09.778 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:09.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:09.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:09.883 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:09.910 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:09.929 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:09.931 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:09.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:10.032 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:10.059 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:10.077 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:10.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:10.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:10.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:10.213 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:10.313 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:10.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:10.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:10.324 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:33:10.327 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:33:10.327 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:33:10.327 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:33:10.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:10.448 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:10.468 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:10.472 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:10.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:10.573 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:10.600 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:10.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:10.622 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:10.722 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:10.751 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:10.780 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:10.782 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:10.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:10.884 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:10.911 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:10.930 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:10.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:10.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:11.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:11.063 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.074+0000 7faa3aa4a700 1 -- 172.21.15.59:0/418522249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa3406b9c0 msgr2=0x7faa34103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.074+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/418522249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa3406b9c0 0x7faa34103fd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7faa28004870 tx=0x7faa28009cb0).stop 2024-02-18T00:33:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.075+0000 7faa3aa4a700 1 -- 172.21.15.59:0/418522249 shutdown_connections 2024-02-18T00:33:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.075+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/418522249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa3406b0f0 0x7faa3406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.075+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/418522249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa341044c0 0x7faa341068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.075+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/418522249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa3406b9c0 0x7faa34103fd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.075+0000 7faa3aa4a700 1 -- 172.21.15.59:0/418522249 >> 172.21.15.59:0/418522249 conn(0x7faa340f3800 msgr2=0x7faa340f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:11.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.075+0000 7faa3aa4a700 1 -- 172.21.15.59:0/418522249 shutdown_connections 2024-02-18T00:33:11.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.075+0000 7faa3aa4a700 1 -- 172.21.15.59:0/418522249 wait complete. 2024-02-18T00:33:11.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.076+0000 7faa3aa4a700 1 librados: starting msgr at 2024-02-18T00:33:11.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.076+0000 7faa3aa4a700 1 librados: starting objecter 2024-02-18T00:33:11.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.076+0000 7faa3aa4a700 1 Processor -- start 2024-02-18T00:33:11.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.076+0000 7faa3aa4a700 1 -- start start 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.076+0000 7faa3aa4a700 1 librados: setting wanted keys 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.076+0000 7faa3aa4a700 1 librados: calling monclient init 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa3aa4a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa3406b0f0 0x7faa340fd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa3aa4a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa3406b9c0 0x7faa340fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa3aa4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa341044c0 0x7faa340f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa3aa4a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa340f8b20 con 0x7faa3406b9c0 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa3aa4a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa340f8c90 con 0x7faa341044c0 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa3aa4a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa340f8e50 con 0x7faa3406b0f0 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa3406b0f0 0x7faa340fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa3406b0f0 0x7faa340fd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44882/0 (socket says 172.21.15.59:44882) 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 -- 172.21.15.59:0/3014528981 learned_addr learned my addr 172.21.15.59:0/3014528981 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa38fe7700 1 --2- 172.21.15.59:0/3014528981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa341044c0 0x7faa340f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 -- 172.21.15.59:0/3014528981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa341044c0 msgr2=0x7faa340f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa341044c0 0x7faa340f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 -- 172.21.15.59:0/3014528981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa3406b9c0 msgr2=0x7faa340fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa3406b9c0 0x7faa340fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 -- 172.21.15.59:0/3014528981 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa28005040 con 0x7faa3406b0f0 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa3406b0f0 0x7faa340fd310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7faa240027f0 tx=0x7faa24002bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa317fa700 10 client.?.objecter ms_handle_connect 0x7faa3406b0f0 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa317fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.077+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24007b40 con 0x7faa3406b0f0 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.078+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa2400f040 con 0x7faa3406b0f0 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.078+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2400d7f0 con 0x7faa3406b0f0 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.078+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa340f9130 con 0x7faa3406b0f0 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.078+0000 7faa3aa4a700 10 client.25154.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:11.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.078+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa34103300 con 0x7faa3406b0f0 2024-02-18T00:33:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.078+0000 7faa3aa4a700 1 librados: init done 2024-02-18T00:33:11.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.079+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7faa3405ee80 con 0x7faa3406b0f0 2024-02-18T00:33:11.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.085+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7faa2400dc20 con 0x7faa3406b0f0 2024-02-18T00:33:11.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.086+0000 7faa317fa700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6800/3721891797 conn(0x7faa1c062c90 0x7faa1c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.086+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(114..114 src has 44..114) v4 ==== 16370+0+0 (secure 0 0 0) 0x7faa2407f600 con 0x7faa3406b0f0 2024-02-18T00:33:11.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.086+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 osd_map(114..114 src has 44..114) v4 2024-02-18T00:33:11.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.086+0000 7faa317fa700 3 client.25154.objecter handle_osd_map got epochs [114,114] > 0 2024-02-18T00:33:11.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.086+0000 7faa317fa700 3 client.25154.objecter handle_osd_map decoding full epoch 114 2024-02-18T00:33:11.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.086+0000 7faa317fa700 20 client.25154.objecter dump_active .. 0 homeless 2024-02-18T00:33:11.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.086+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v114) v1 ==== 54+0+16 (secure 0 0 0) 0x7faa2407fa20 con 0x7faa3406b0f0 2024-02-18T00:33:11.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.087+0000 7faa337fe700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6800/3721891797 conn(0x7faa1c062c90 0x7faa1c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.088+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa1c062c90 2024-02-18T00:33:11.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.088+0000 7faa337fe700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6800/3721891797 conn(0x7faa1c062c90 0x7faa1c065150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7faa28005010 tx=0x7faa2800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 1 2024-02-18T00:33:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6801/3700238977 conn(0x7faa340273c0 0x7faa340277a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 1 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 0 1 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34109310 osd=0 4 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 10 client.25154.objecter _send_command 1 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa34104260 con 0x7faa340273c0 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa38fe7700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6801/3700238977 conn(0x7faa340273c0 0x7faa340277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa340273c0 2024-02-18T00:33:11.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.091+0000 7faa38fe7700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6801/3700238977 conn(0x7faa340273c0 0x7faa340277a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.117+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa34104260 con 0x7faa340273c0 2024-02-18T00:33:11.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.117+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:33:11.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.117+0000 7faa317fa700 10 client.25154.objecter _finish_command 1 = system:0 2024-02-18T00:33:11.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.117+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34109310 osd=0 4 2024-02-18T00:33:11.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.117+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 0 1 2024-02-18T00:33:11.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 2 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 2 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34109310 osd=0 3 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 0 2 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34109310 osd=0 4 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 10 client.25154.objecter _send_command 2 2024-02-18T00:33:11.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.133+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa3405fa60 con 0x7faa340273c0 2024-02-18T00:33:11.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.134+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa3405fa60 con 0x7faa340273c0 2024-02-18T00:33:11.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.134+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.134+0000 7faa317fa700 10 client.25154.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.134+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34109310 osd=0 4 2024-02-18T00:33:11.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.134+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 0 2 2024-02-18T00:33:11.136 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6805/26279110 conn(0x7faa341037c0 0x7faa340646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 1 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3405fbd0 osd=1 4 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 10 client.25154.objecter _send_command 3 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa34065d60 con 0x7faa341037c0 2024-02-18T00:33:11.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6805/26279110 conn(0x7faa341037c0 0x7faa340646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6805/26279110 conn(0x7faa341037c0 0x7faa340646f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.135+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa341037c0 2024-02-18T00:33:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.136+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa34065d60 con 0x7faa341037c0 2024-02-18T00:33:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.136+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:33:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.136+0000 7faa317fa700 10 client.25154.objecter _finish_command 3 = system:0 2024-02-18T00:33:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.136+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa3405fbd0 osd=1 4 2024-02-18T00:33:11.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.136+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 1 3 2024-02-18T00:33:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 4 2024-02-18T00:33:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 4 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa3405fbd0 osd=1 3 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 1 4 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3405fbd0 osd=1 4 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 10 client.25154.objecter _send_command 4 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.150+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa34027ed0 con 0x7faa341037c0 2024-02-18T00:33:11.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa34027ed0 con 0x7faa341037c0 2024-02-18T00:33:11.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa317fa700 10 client.25154.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa3405fbd0 osd=1 4 2024-02-18T00:33:11.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 1 4 2024-02-18T00:33:11.154 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 5 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6809/1153193637 conn(0x7faa34059e00 0x7faa341339d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 5 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 2 5 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa340282e0 osd=2 4 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 10 client.25154.objecter _send_command 5 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa34136620 con 0x7faa34059e00 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.152+0000 7faa337fe700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6809/1153193637 conn(0x7faa34059e00 0x7faa341339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.153+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa34059e00 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.153+0000 7faa337fe700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6809/1153193637 conn(0x7faa34059e00 0x7faa341339d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.153+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa34136620 con 0x7faa34059e00 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.153+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.153+0000 7faa317fa700 10 client.25154.objecter _finish_command 5 = system:0 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.153+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa340282e0 osd=2 4 2024-02-18T00:33:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.153+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 2 5 2024-02-18T00:33:11.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 6 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 6 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa340282e0 osd=2 3 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 2 6 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa340282e0 osd=2 4 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 10 client.25154.objecter _send_command 6 2024-02-18T00:33:11.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.168+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa34059360 con 0x7faa34059e00 2024-02-18T00:33:11.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa34059360 con 0x7faa34059e00 2024-02-18T00:33:11.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa317fa700 10 client.25154.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa340282e0 osd=2 4 2024-02-18T00:33:11.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 2 6 2024-02-18T00:33:11.171 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 7 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6813/1805700964 conn(0x7faa340658c0 0x7faa34135040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 7 2024-02-18T00:33:11.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 3 7 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34059770 osd=3 4 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 10 client.25154.objecter _send_command 7 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa34135610 con 0x7faa340658c0 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.169+0000 7faa38fe7700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6813/1805700964 conn(0x7faa340658c0 0x7faa34135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.170+0000 7faa38fe7700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6813/1805700964 conn(0x7faa340658c0 0x7faa34135040 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.170+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa340658c0 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.170+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa34135610 con 0x7faa340658c0 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.170+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.170+0000 7faa317fa700 10 client.25154.objecter _finish_command 7 = system:0 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.170+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34059770 osd=3 4 2024-02-18T00:33:11.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.170+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 3 7 2024-02-18T00:33:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 8 2024-02-18T00:33:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 8 2024-02-18T00:33:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34059770 osd=3 3 2024-02-18T00:33:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 3 8 2024-02-18T00:33:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34059770 osd=3 4 2024-02-18T00:33:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 10 client.25154.objecter _send_command 8 2024-02-18T00:33:11.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.186+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa34064e20 con 0x7faa340658c0 2024-02-18T00:33:11.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa34064e20 con 0x7faa340658c0 2024-02-18T00:33:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa317fa700 10 client.25154.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34059770 osd=3 4 2024-02-18T00:33:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 3 8 2024-02-18T00:33:11.192 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:33:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 9 2024-02-18T00:33:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6800/2044387056 conn(0x7faa341387d0 0x7faa3413abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 9 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 4 9 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34064e20 osd=4 4 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 10 client.25154.objecter _send_command 9 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa3413d810 con 0x7faa341387d0 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.190+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6800/2044387056 conn(0x7faa341387d0 0x7faa3413abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.191+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa341387d0 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.191+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6800/2044387056 conn(0x7faa341387d0 0x7faa3413abc0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.191+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa3413d810 con 0x7faa341387d0 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.191+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:33:11.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.191+0000 7faa317fa700 10 client.25154.objecter _finish_command 9 = system:0 2024-02-18T00:33:11.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.191+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34064e20 osd=4 4 2024-02-18T00:33:11.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.191+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 4 9 2024-02-18T00:33:11.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:11.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 10 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 10 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34064e20 osd=4 3 2024-02-18T00:33:11.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 4 10 2024-02-18T00:33:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34064e20 osd=4 4 2024-02-18T00:33:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 10 client.25154.objecter _send_command 10 2024-02-18T00:33:11.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.207+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa3413b0b0 con 0x7faa341387d0 2024-02-18T00:33:11.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa3413b0b0 con 0x7faa341387d0 2024-02-18T00:33:11.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa317fa700 10 client.25154.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34064e20 osd=4 4 2024-02-18T00:33:11.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 4 10 2024-02-18T00:33:11.211 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:33:11.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 11 2024-02-18T00:33:11.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6804/2170432263 conn(0x7faa3413b790 0x7faa3413bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 11 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 5 11 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413b0b0 osd=5 4 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 10 client.25154.objecter _send_command 11 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa3413d840 con 0x7faa3413b790 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.209+0000 7faa337fe700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6804/2170432263 conn(0x7faa3413b790 0x7faa3413bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.210+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa3413b790 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.210+0000 7faa337fe700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6804/2170432263 conn(0x7faa3413b790 0x7faa3413bbf0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.210+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa3413d840 con 0x7faa3413b790 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.210+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:33:11.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.210+0000 7faa317fa700 10 client.25154.objecter _finish_command 11 = system:0 2024-02-18T00:33:11.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.210+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa3413b0b0 osd=5 4 2024-02-18T00:33:11.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.210+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 5 11 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 12 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 12 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa3413b0b0 osd=5 3 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 5 12 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413b0b0 osd=5 4 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 10 client.25154.objecter _send_command 12 2024-02-18T00:33:11.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.225+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa3413d840 con 0x7faa3413b790 2024-02-18T00:33:11.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa3413d840 con 0x7faa3413b790 2024-02-18T00:33:11.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa317fa700 10 client.25154.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa3413b0b0 osd=5 4 2024-02-18T00:33:11.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 5 12 2024-02-18T00:33:11.229 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 13 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6808/3356009390 conn(0x7faa3413ff70 0x7faa341403b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 13 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 6 13 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413d840 osd=6 4 2024-02-18T00:33:11.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 10 client.25154.objecter _send_command 13 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa34140a20 con 0x7faa3413ff70 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa38fe7700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6808/3356009390 conn(0x7faa3413ff70 0x7faa341403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa3413ff70 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.227+0000 7faa38fe7700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6808/3356009390 conn(0x7faa3413ff70 0x7faa341403b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.228+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa2c00f040 con 0x7faa3413ff70 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.228+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.228+0000 7faa317fa700 10 client.25154.objecter _finish_command 13 = system:0 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.228+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa3413d840 osd=6 4 2024-02-18T00:33:11.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.228+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 6 13 2024-02-18T00:33:11.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 14 2024-02-18T00:33:11.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 14 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa3413d840 osd=6 3 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 6 14 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413d840 osd=6 4 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 10 client.25154.objecter _send_command 14 2024-02-18T00:33:11.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.242+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa34142cb0 con 0x7faa3413ff70 2024-02-18T00:33:11.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa34142cb0 con 0x7faa3413ff70 2024-02-18T00:33:11.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa317fa700 10 client.25154.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa3413d840 osd=6 4 2024-02-18T00:33:11.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 6 14 2024-02-18T00:33:11.247 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:33:11.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:11.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 15 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6812/53025171 conn(0x7faa341433a0 0x7faa34143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 15 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 7 15 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34142cb0 osd=7 4 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 10 client.25154.objecter _send_command 15 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.244+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa341470b0 con 0x7faa341433a0 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.245+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6812/53025171 conn(0x7faa341433a0 0x7faa34143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.245+0000 7faa317fa700 10 client.25154.objecter ms_handle_connect 0x7faa341433a0 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.245+0000 7faa33fff700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6812/53025171 conn(0x7faa341433a0 0x7faa34143800 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.246+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faa341470b0 con 0x7faa341433a0 2024-02-18T00:33:11.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.246+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:33:11.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.246+0000 7faa317fa700 10 client.25154.objecter _finish_command 15 = system:0 2024-02-18T00:33:11.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.246+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34142cb0 osd=7 4 2024-02-18T00:33:11.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.246+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 7 15 2024-02-18T00:33:11.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:33:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:33:11.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:33:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.363426+0000 osd.6 (osd.6) 65 : cluster [DBG] 17.5 deep-scrub starts 2024-02-18T00:33:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.364351+0000 osd.6 (osd.6) 66 : cluster [DBG] 17.5 deep-scrub ok 2024-02-18T00:33:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.774990+0000 osd.7 (osd.7) 83 : cluster [DBG] 17.8 deep-scrub starts 2024-02-18T00:33:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:06.776391+0000 osd.7 (osd.7) 84 : cluster [DBG] 17.8 deep-scrub ok 2024-02-18T00:33:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:07.772819+0000 mgr.y (mgr.24454) 164 : cluster [DBG] pgmap v154: 622 pgs: 26 creating+peering, 462 active+clean, 134 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 546 op/s; 218 B/s, 0 objects/s recovering 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.254885+0000 mon.b (mon.1) 68 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.328683+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.328799+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.328947+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.329047+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.329209+0000 mon.a (mon.0) 896 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:08.329554+0000 mon.a (mon.0) 897 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.329789+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.330013+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.330069+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.330769+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.330944+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T00:33:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.342464+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.348859+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:08.352825+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.353211+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.360032+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.360657+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.365236+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.365804+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.366434+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 10 client.25154.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign -1 16 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 20 client.25154.objecter _get_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_remove -1 16 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 20 client.25154.objecter get_session s=0x7faa34142cb0 osd=7 3 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 15 client.25154.objecter _session_command_op_assign 7 16 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34142cb0 osd=7 4 2024-02-18T00:33:11.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 10 client.25154.objecter _send_command 16 2024-02-18T00:33:11.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.260+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7faa341470b0 con 0x7faa341433a0 2024-02-18T00:33:11.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa317fa700 1 -- 172.21.15.59:0/3014528981 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7faa341470b0 con 0x7faa341433a0 2024-02-18T00:33:11.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa317fa700 10 client.25154.objecter ms_dispatch 0x7faa34001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:11.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa317fa700 10 client.25154.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa317fa700 20 client.25154.objecter put_session s=0x7faa34142cb0 osd=7 4 2024-02-18T00:33:11.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa317fa700 15 client.25154.objecter _session_command_op_remove 7 16 2024-02-18T00:33:11.264 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 librados: watch_flush enter 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 librados: watch_flush exit 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.0 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6801/3700238977 conn(0x7faa340273c0 msgr2=0x7faa340277a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6801/3700238977 conn(0x7faa340273c0 0x7faa340277a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34109310 osd=0 1 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.1 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6805/26279110 conn(0x7faa341037c0 msgr2=0x7faa340646f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6805/26279110 conn(0x7faa341037c0 0x7faa340646f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3405fbd0 osd=1 1 2024-02-18T00:33:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.2 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6809/1153193637 conn(0x7faa34059e00 msgr2=0x7faa341339d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6809/1153193637 conn(0x7faa34059e00 0x7faa341339d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa340282e0 osd=2 1 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.3 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6813/1805700964 conn(0x7faa340658c0 msgr2=0x7faa34135040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6813/1805700964 conn(0x7faa340658c0 0x7faa34135040 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34059770 osd=3 1 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.4 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6800/2044387056 conn(0x7faa341387d0 msgr2=0x7faa3413abc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6800/2044387056 conn(0x7faa341387d0 0x7faa3413abc0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34064e20 osd=4 1 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.5 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6804/2170432263 conn(0x7faa3413b790 msgr2=0x7faa3413bbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6804/2170432263 conn(0x7faa3413b790 0x7faa3413bbf0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413b0b0 osd=5 1 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.6 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6808/3356009390 conn(0x7faa3413ff70 msgr2=0x7faa341403b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6808/3356009390 conn(0x7faa3413ff70 0x7faa341403b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa3413d840 osd=6 1 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter close_session for osd.7 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6812/53025171 conn(0x7faa341433a0 msgr2=0x7faa34143800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.137:6812/53025171 conn(0x7faa341433a0 0x7faa34143800 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter put_session s=0x7faa34142cb0 osd=7 1 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 20 client.25154.objecter shutdown clearing up homeless session... 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 10 client.25154.objecter successfully canceled tick 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa38fe7700 1 -- 172.21.15.59:0/3014528981 reap_dead start 2024-02-18T00:33:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa38fe7700 1 -- 172.21.15.59:0/3014528981 reap_dead start 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa38fe7700 1 -- 172.21.15.59:0/3014528981 reap_dead start 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa38fe7700 1 -- 172.21.15.59:0/3014528981 reap_dead start 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa38fe7700 1 -- 172.21.15.59:0/3014528981 reap_dead start 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6800/3721891797 conn(0x7faa1c062c90 msgr2=0x7faa1c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6800/3721891797 conn(0x7faa1c062c90 0x7faa1c065150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7faa28005010 tx=0x7faa2800b040).stop 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa3406b0f0 msgr2=0x7faa340fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.262+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa3406b0f0 0x7faa340fd310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7faa240027f0 tx=0x7faa24002bb0).stop 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.263+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 shutdown_connections 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.263+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa3406b0f0 0x7faa340fd310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.263+0000 7faa3aa4a700 1 --2- 172.21.15.59:0/3014528981 >> v2:172.21.15.59:6800/3721891797 conn(0x7faa1c062c90 0x7faa1c065150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.263+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 >> 172.21.15.59:0/3014528981 conn(0x7faa340f3800 msgr2=0x7faa340f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.263+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 shutdown_connections 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.263+0000 7faa3aa4a700 1 -- 172.21.15.59:0/3014528981 wait complete. 2024-02-18T00:33:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:11.263+0000 7faa3aa4a700 1 librados: shutdown 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:33:11.298 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.363426+0000 osd.6 (osd.6) 65 : cluster [DBG] 17.5 deep-scrub starts 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.364351+0000 osd.6 (osd.6) 66 : cluster [DBG] 17.5 deep-scrub ok 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.774990+0000 osd.7 (osd.7) 83 : cluster [DBG] 17.8 deep-scrub starts 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:06.776391+0000 osd.7 (osd.7) 84 : cluster [DBG] 17.8 deep-scrub ok 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:07.772819+0000 mgr.y (mgr.24454) 164 : cluster [DBG] pgmap v154: 622 pgs: 26 creating+peering, 462 active+clean, 134 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 546 op/s; 218 B/s, 0 objects/s recovering 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.254885+0000 mon.b (mon.1) 68 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.328683+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.328799+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:11.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.328947+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.329047+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.329209+0000 mon.a (mon.0) 896 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:08.329554+0000 mon.a (mon.0) 897 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.329789+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.363426+0000 osd.6 (osd.6) 65 : cluster [DBG] 17.5 deep-scrub starts 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.364351+0000 osd.6 (osd.6) 66 : cluster [DBG] 17.5 deep-scrub ok 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.774990+0000 osd.7 (osd.7) 83 : cluster [DBG] 17.8 deep-scrub starts 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:06.776391+0000 osd.7 (osd.7) 84 : cluster [DBG] 17.8 deep-scrub ok 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:07.772819+0000 mgr.y (mgr.24454) 164 : cluster [DBG] pgmap v154: 622 pgs: 26 creating+peering, 462 active+clean, 134 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 546 op/s; 218 B/s, 0 objects/s recovering 2024-02-18T00:33:11.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.254885+0000 mon.b (mon.1) 68 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:11.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.328683+0000 mon.a (mon.0) 892 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi059-82882-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:11.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.328799+0000 mon.a (mon.0) 893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:11.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.328947+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi059-82893-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.329047+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi059-82989-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.329209+0000 mon.a (mon.0) 896 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:11.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:08.329554+0000 mon.a (mon.0) 897 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-18T00:33:11.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.329789+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:11.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.330013+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:11.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.330069+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:11.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.330769+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:11.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.330944+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T00:33:11.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.342464+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:11.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.348859+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:08.352825+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.353211+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.360032+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.360657+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.365236+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.365804+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.366434+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.330013+0000 mon.c (mon.2) 46 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:11.334 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.330069+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.330769+0000 mon.c (mon.2) 47 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.330944+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.59:0/2397326679' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.342464+0000 mon.a (mon.0) 898 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.348859+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:08.352825+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.353211+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.360032+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:11.335 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.360657+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:11.336 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.365236+0000 mon.a (mon.0) 903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:11.336 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.365804+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:11.336 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.366434+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T00:33:11.336 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:11.534 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:11.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:11.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:11.549 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-18T00:33:11.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:11.668 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: seed 82901 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-18T00:33:12.205 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (218 ms) 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (20 ms) 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (6 ms) 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (9 ms) 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (6 ms) 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (10 ms) 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-18T00:33:12.206 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (10 ms) 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (8 ms) 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (6 ms) 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (7 ms) 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-18T00:33:12.207 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4 ms) 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (12 ms) 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (4 ms) 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (5 ms) 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (7 ms) 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (347 ms total) 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-18T00:33:12.208 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1079 ms) 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (13 ms) 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (6 ms) 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-18T00:33:12.209 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (4 ms) 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:08.026104+0000 osd.0 (osd.0) 75 : cluster [DBG] 17.4 deep-scrub starts 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:08.067953+0000 osd.0 (osd.0) 76 : cluster [DBG] 17.4 deep-scrub ok 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:08.649416+0000 osd.3 (osd.3) 93 : cluster [DBG] 17.9 deep-scrub starts 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:08.650647+0000 osd.3 (osd.3) 94 : cluster [DBG] 17.9 deep-scrub ok 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.823405+0000 mon.a (mon.0) 906 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.823493+0000 mon.a (mon.0) 907 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:08.823612+0000 mon.a (mon.0) 908 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:09.255998+0000 mon.b (mon.1) 72 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:09.298100+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:09.298734+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:09.429156+0000 osd.6 (osd.6) 67 : cluster [DBG] 17.6 deep-scrub starts 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:09.430592+0000 osd.6 (osd.6) 68 : cluster [DBG] 17.6 deep-scrub ok 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:09.773783+0000 mgr.y (mgr.24454) 165 : cluster [DBG] pgmap v156: 574 pgs: 430 active+clean, 144 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 35 MiB/s wr, 513 op/s; 204 B/s, 0 objects/s recovering 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:10.174711+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:08.026104+0000 osd.0 (osd.0) 75 : cluster [DBG] 17.4 deep-scrub starts 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:08.067953+0000 osd.0 (osd.0) 76 : cluster [DBG] 17.4 deep-scrub ok 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:08.649416+0000 osd.3 (osd.3) 93 : cluster [DBG] 17.9 deep-scrub starts 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:08.650647+0000 osd.3 (osd.3) 94 : cluster [DBG] 17.9 deep-scrub ok 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.823405+0000 mon.a (mon.0) 906 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.823493+0000 mon.a (mon.0) 907 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:12.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:08.823612+0000 mon.a (mon.0) 908 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:09.255998+0000 mon.b (mon.1) 72 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:09.298100+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:09.298734+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:09.429156+0000 osd.6 (osd.6) 67 : cluster [DBG] 17.6 deep-scrub starts 2024-02-18T00:33:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:09.430592+0000 osd.6 (osd.6) 68 : cluster [DBG] 17.6 deep-scrub ok 2024-02-18T00:33:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:09.773783+0000 mgr.y (mgr.24454) 165 : cluster [DBG] pgmap v156: 574 pgs: 430 active+clean, 144 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 35 MiB/s wr, 513 op/s; 204 B/s, 0 objects/s recovering 2024-02-18T00:33:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:10.174711+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:10.175858+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:10.256947+0000 mon.b (mon.1) 74 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:10.927064+0000 mgr.y (mgr.24454) 166 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:10.997815+0000 osd.0 (osd.0) 77 : cluster [DBG] 17.7 deep-scrub starts 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:10.999154+0000 osd.0 (osd.0) 78 : cluster [DBG] 17.7 deep-scrub ok 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:11.060853+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.067218+0000 mon.a (mon.0) 912 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.074319+0000 mon.a (mon.0) 913 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.077096+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.078333+0000 mon.a (mon.0) 915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.084454+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.084543+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.084621+0000 mon.a (mon.0) 918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:12.213 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.085232+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? 172.21.15.59:0/3014528981' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.086079+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.59:0/2128402209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.093575+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.59:0/146077577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.257963+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:11.975331+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.975492+0000 mon.a (mon.0) 920 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.977010+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:11.977451+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:10.175858+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:10.256947+0000 mon.b (mon.1) 74 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:10.927064+0000 mgr.y (mgr.24454) 166 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:10.997815+0000 osd.0 (osd.0) 77 : cluster [DBG] 17.7 deep-scrub starts 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:10.999154+0000 osd.0 (osd.0) 78 : cluster [DBG] 17.7 deep-scrub ok 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:11.060853+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.067218+0000 mon.a (mon.0) 912 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.074319+0000 mon.a (mon.0) 913 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.077096+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.078333+0000 mon.a (mon.0) 915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.084454+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.084543+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.084621+0000 mon.a (mon.0) 918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.085232+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? 172.21.15.59:0/3014528981' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.086079+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.59:0/2128402209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.215 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.093575+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.59:0/146077577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.216 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.257963+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.216 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:11.975331+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-18T00:33:12.216 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.975492+0000 mon.a (mon.0) 920 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:12.216 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.977010+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.216 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:11.977451+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:33:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:12] "GET /metrics HTTP/1.1" 200 224171 "" "Prometheus/2.33.4" 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:08.026104+0000 osd.0 (osd.0) 75 : cluster [DBG] 17.4 deep-scrub starts 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:08.067953+0000 osd.0 (osd.0) 76 : cluster [DBG] 17.4 deep-scrub ok 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:08.649416+0000 osd.3 (osd.3) 93 : cluster [DBG] 17.9 deep-scrub starts 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:08.650647+0000 osd.3 (osd.3) 94 : cluster [DBG] 17.9 deep-scrub ok 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.823405+0000 mon.a (mon.0) 906 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.823493+0000 mon.a (mon.0) 907 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:08.823612+0000 mon.a (mon.0) 908 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:09.255998+0000 mon.b (mon.1) 72 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:09.298100+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:09.298734+0000 mon.c (mon.2) 48 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:09.429156+0000 osd.6 (osd.6) 67 : cluster [DBG] 17.6 deep-scrub starts 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:09.430592+0000 osd.6 (osd.6) 68 : cluster [DBG] 17.6 deep-scrub ok 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:09.773783+0000 mgr.y (mgr.24454) 165 : cluster [DBG] pgmap v156: 574 pgs: 430 active+clean, 144 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 35 MiB/s wr, 513 op/s; 204 B/s, 0 objects/s recovering 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:10.174711+0000 mon.a (mon.0) 910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:10.175858+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:10.256947+0000 mon.b (mon.1) 74 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:10.927064+0000 mgr.y (mgr.24454) 166 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:10.997815+0000 osd.0 (osd.0) 77 : cluster [DBG] 17.7 deep-scrub starts 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:10.999154+0000 osd.0 (osd.0) 78 : cluster [DBG] 17.7 deep-scrub ok 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:11.060853+0000 mon.a (mon.0) 911 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-18T00:33:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.067218+0000 mon.a (mon.0) 912 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.074319+0000 mon.a (mon.0) 913 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.077096+0000 mon.a (mon.0) 914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.078333+0000 mon.a (mon.0) 915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.084454+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.084543+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.084621+0000 mon.a (mon.0) 918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.085232+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? 172.21.15.59:0/3014528981' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.086079+0000 mon.c (mon.2) 50 : audit [INF] from='client.? 172.21.15.59:0/2128402209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.093575+0000 mon.c (mon.2) 51 : audit [INF] from='client.? 172.21.15.59:0/146077577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.257963+0000 mon.b (mon.1) 75 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:11.975331+0000 mon.a (mon.0) 919 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.975492+0000 mon.a (mon.0) 920 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.977010+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:11.977451+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:12.554 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN delete_pools_parallel: process_1_[83131]: starting. 2024-02-18T00:33:12.554 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_1_[83131]: creating pool ceph_test_rados_delete_pools_parallel.smithi059-83081 2024-02-18T00:33:12.554 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_1_[83131]: created object 0... 2024-02-18T00:33:12.554 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_1_[83131]: created object 25... 2024-02-18T00:33:12.554 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_1_[83131]: created object 49... 2024-02-18T00:33:12.554 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_1_[83131]: finishing. 2024-02-18T00:33:12.554 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_1_[83131]: shutting down. 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_2_[83132]: starting. 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_2_[83132]: deleting pool ceph_test_rados_delete_pools_parallel.smithi059-83081 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_2_[83132]: shutting down. 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: ******************************* 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_3_[86455]: starting. 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_3_[86455]: creating pool ceph_test_rados_delete_pools_parallel.smithi059-83081 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_3_[86455]: created object 0... 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_3_[86455]: created object 25... 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_3_[86455]: created object 49... 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_3_[86455]: finishing. 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_3_[86455]: shutting down. 2024-02-18T00:33:12.555 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: ******************************* 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_5_[86457]: starting. 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_5_[86457]: listing objects. 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_5_[86457]: listed object 0... 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_5_[86457]: listed object 25... 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_5_[86457]: saw 50 objects 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_5_[86457]: shutting down. 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: ******************************* 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_4_[86456]: starting. 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_4_[86456]: deleting pool ceph_test_rados_delete_pools_parallel.smithi059-83081 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: process_4_[86456]: shutting down. 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: ******************************* 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: ******************************* 2024-02-18T00:33:12.556 INFO:tasks.workunit.client.0.smithi059.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-18T00:33:12.657 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:12.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:12.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:12.716 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:12.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:13.075 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:08.328402+0000 mon.a [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:08.823405+0000 mon.a [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi059-82997-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:08.823493+0000 mon.a [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi059-82925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:08.823612+0000 mon.a [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:09.298100+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:09.298734+0000 mon.c [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:10.174711+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi059-82882-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:10.175858+0000 mon.b [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.067218+0000 mon.a [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.074319+0000 mon.a [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T00:33:13.076 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.077096+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:13.077 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.078333+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.077 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.084454+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi059-82989-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:13.085 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.084543+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi059-82893-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:13.085 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.084621+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi059-82901-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:13.085 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.086079+0000 mon.c [INF] from='client.? 172.21.15.59:0/2128402209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.085 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.093575+0000 mon.c [INF] from='client.? 172.21.15.59:0/146077577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.085 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.975492+0000 mon.a [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.977010+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:11.977451+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.085475+0000 mon.a [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.542459+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.542574+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.542662+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.543208+0000 mon.a [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.544077+0000 mon.b [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.545191+0000 mon.c [INF] from='client.? 172.21.15.59:0/694722472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.545949+0000 mon.a [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:33:13.086 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.555388+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:11.774832+0000 mgr.y (mgr.24454) 167 : cluster [DBG] pgmap v158: 662 pgs: 34 creating+peering, 526 active+clean, 102 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:33:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.085475+0000 mon.a (mon.0) 923 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.259091+0000 mon.b (mon.1) 76 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.542459+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.542574+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.542662+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:12.542829+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:12.543208+0000 mon.a (mon.0) 928 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.544077+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.545191+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.59:0/694722472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.545949+0000 mon.a (mon.0) 929 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.555388+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.555546+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.558576+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:12.558910+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:13.074859+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-18T00:33:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.075069+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:11.774832+0000 mgr.y (mgr.24454) 167 : cluster [DBG] pgmap v158: 662 pgs: 34 creating+peering, 526 active+clean, 102 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.085475+0000 mon.a (mon.0) 923 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.259091+0000 mon.b (mon.1) 76 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.542459+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.542574+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.542662+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:12.542829+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:12.543208+0000 mon.a (mon.0) 928 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.544077+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.545191+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.59:0/694722472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.545949+0000 mon.a (mon.0) 929 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.555388+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.555546+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.558576+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:12.558910+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:13.074859+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.075069+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T00:33:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:11.774832+0000 mgr.y (mgr.24454) 167 : cluster [DBG] pgmap v158: 662 pgs: 34 creating+peering, 526 active+clean, 102 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.085475+0000 mon.a (mon.0) 923 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-4", "overlaypool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.259091+0000 mon.b (mon.1) 76 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.542459+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.542574+0000 mon.a (mon.0) 925 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi059-82877-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.542662+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:12.542829+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:12.543208+0000 mon.a (mon.0) 928 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.544077+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.545191+0000 mon.c (mon.2) 52 : audit [INF] from='client.? 172.21.15.59:0/694722472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.545949+0000 mon.a (mon.0) 929 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.555388+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.555546+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.558576+0000 mon.a (mon.0) 931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:13.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:12.558910+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:13.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:13.074859+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-18T00:33:13.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.075069+0000 mon.a (mon.0) 933 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T00:33:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:13.083505+0000 mon.a (mon.0) 934 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.087212+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-18T00:33:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:13.087544+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-18T00:33:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.087793+0000 mon.a (mon.0) 936 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T00:33:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.091931+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.092054+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-4-cache", "mode": "writeback"}]': finished 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.092251+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.092391+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.092505+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]': finished 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:13.092578+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.096928+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.097895+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.098253+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:13.083505+0000 mon.a (mon.0) 934 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.087212+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-18T00:33:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:13.087544+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.087793+0000 mon.a (mon.0) 936 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.091931+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.092054+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-4-cache", "mode": "writeback"}]': finished 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.092251+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.092391+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.092505+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]': finished 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:13.092578+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.096928+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.097895+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:14.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.098253+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.101020+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.104378+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.104587+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.109270+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.109650+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.109807+0000 mon.a (mon.0) 947 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.209046+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.209561+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.260309+0000 mon.b (mon.1) 84 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:13.586817+0000 mon.a (mon.0) 948 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-4"}]: dispatch 2024-02-18T00:33:14.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.027020+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.101020+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.104378+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.104587+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.109270+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.109650+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.109807+0000 mon.a (mon.0) 947 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.209046+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.209561+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.260309+0000 mon.b (mon.1) 84 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:13.586817+0000 mon.a (mon.0) 948 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-4"}]: dispatch 2024-02-18T00:33:14.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.027020+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:13.083505+0000 mon.a (mon.0) 934 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.087212+0000 mon.a (mon.0) 935 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:13.087544+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.087793+0000 mon.a (mon.0) 936 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.091931+0000 mon.a (mon.0) 937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi059-82933-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.092054+0000 mon.a (mon.0) 938 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-4-cache", "mode": "writeback"}]': finished 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.092251+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.092391+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:14.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.092505+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]': finished 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:13.092578+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.096928+0000 mon.c (mon.2) 53 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.097895+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.098253+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.101020+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.104378+0000 mon.b (mon.1) 80 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.104587+0000 mon.b (mon.1) 81 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.109270+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.109650+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.109807+0000 mon.a (mon.0) 947 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.209046+0000 mon.b (mon.1) 82 : audit [DBG] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.209561+0000 mon.b (mon.1) 83 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:14.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.260309+0000 mon.b (mon.1) 84 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:14.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:13.586817+0000 mon.a (mon.0) 948 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-4"}]: dispatch 2024-02-18T00:33:14.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.027020+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:14.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:33:15.040 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:15.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:15.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:15.125 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02- api_stat: Running main() from gmock_main.cc 2024-02-18T00:33:15.125 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-02-18T00:33:15.125 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [----------] Global test environment set-up. 2024-02-18T00:33:15.125 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-02-18T00:33:15.125 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStat.Stat (98 ms) 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStat.StatNS (161 ms) 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [----------] 4 tests from LibRadosStat (261 ms total) 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1080 ms) 2024-02-18T00:33:15.126 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (9 ms) 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (1 ms) 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1091 ms total) 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [----------] Global test environment tear-down 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (24171 ms total) 2024-02-18T00:33:15.127 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat: [ PASSED ] 8 tests. 2024-02-18T00:33:15.132 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-18T00:33:15.132 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-18T00:33:15.132 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-18T00:33:15.132 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-18T00:33:15.132 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: seed 82997 2024-02-18T00:33:15.132 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (195 ms) 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (24 ms) 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (4 ms) 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (17 ms) 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (240 ms total) 2024-02-18T00:33:15.133 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: 2024-02-18T00:33:15.134 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-18T00:33:15.134 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-18T00:33:15.134 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1097 ms) 2024-02-18T00:33:15.134 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-18T00:33:15.134 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (374 ms) 2024-02-18T00:33:15.134 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-18T00:33:15.134 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (5 ms) 2024-02-18T00:33:15.135 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-18T00:33:15.135 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (7 ms) 2024-02-18T00:33:15.135 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1483 ms total) 2024-02-18T00:33:15.135 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: 2024-02-18T00:33:15.135 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-18T00:33:15.135 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (24220 ms total) 2024-02-18T00:33:15.135 INFO:tasks.workunit.client.0.smithi059.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: ] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (7 ms) 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (7 ms) 2024-02-18T00:33:15.136 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-18T00:33:15.137 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (8 ms) 2024-02-18T00:33:15.138 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-18T00:33:15.138 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-02-18T00:33:15.138 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-18T00:33:15.138 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-02-18T00:33:15.138 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1173 ms total) 2024-02-18T00:33:15.139 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: 2024-02-18T00:33:15.139 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-18T00:33:15.139 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24240 ms total) 2024-02-18T00:33:15.139 INFO:tasks.workunit.client.0.smithi059.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-18T00:33:15.143 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: Running main() from gmock_main.cc 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [==========] Running 23 tests from 2 test suites. 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [----------] Global test environment set-up. 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [----------] 13 tests from LibRadosIo 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (242 ms) 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-18T00:33:15.147 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: no timeout :/ 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: no timeout :/ 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: no timeout :/ 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: no timeout :/ 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: no timeout :/ 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (307 ms) 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (10 ms) 2024-02-18T00:33:15.148 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.Checksum (5 ms) 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (3 ms) 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (6 ms) 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-02-18T00:33:15.149 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (3 ms) 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (2 ms) 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.RmXattr (9 ms) 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [----------] 13 tests from LibRadosIo (601 ms total) 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-18T00:33:15.150 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1072 ms) 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (8 ms) 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (4 ms) 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-02-18T00:33:15.151 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (11 ms) 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (6 ms) 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1126 ms total) 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [----------] Global test environment tear-down 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [==========] 23 tests from 2 test suites ran. (24241 ms total) 2024-02-18T00:33:15.152 INFO:tasks.workunit.client.0.smithi059.stdout: api_io: [ PASSED ] 23 tests. 2024-02-18T00:33:15.176 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:15.195 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:15.199 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:15.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:15.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:15.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:13.775370+0000 mgr.y (mgr.24454) 168 : cluster [DBG] pgmap v161: 610 pgs: 27 creating+peering, 494 active+clean, 89 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-18T00:33:15.326 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.090816+0000 mon.a (mon.0) 950 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.099237+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.099315+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.099396+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.099447+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.099497+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.099654+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-4"}]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.099793+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:14.099874+0000 mon.a (mon.0) 958 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.102721+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.103937+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:15.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.104606+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:13.775370+0000 mgr.y (mgr.24454) 168 : cluster [DBG] pgmap v161: 610 pgs: 27 creating+peering, 494 active+clean, 89 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.090816+0000 mon.a (mon.0) 950 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.099237+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.099315+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.099396+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.099447+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.099497+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.099654+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-4"}]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.099793+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:15.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:14.099874+0000 mon.a (mon.0) 958 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.102721+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.103937+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.104606+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.104893+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.105760+0000 mon.a (mon.0) 961 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.109001+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.110083+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.110193+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:15.329 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.110746+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.111188+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.59:0/2294550938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.112531+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.118180+0000 mon.a (mon.0) 965 : audit [INF] from='client.? 172.21.15.59:0/3857141535' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi059-82877-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.123123+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2"}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.125944+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.131600+0000 mon.a (mon.0) 966 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.104893+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:15.330 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.105760+0000 mon.a (mon.0) 961 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.109001+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.110083+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.110193+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.110746+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.111188+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.59:0/2294550938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.112531+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.118180+0000 mon.a (mon.0) 965 : audit [INF] from='client.? 172.21.15.59:0/3857141535' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi059-82877-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.123123+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2"}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.125944+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.131600+0000 mon.a (mon.0) 966 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:15.331 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.131984+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.162939+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.261392+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.604684+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.605538+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.605878+0000 mon.a (mon.0) 969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.606509+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:14.606717+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.131984+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.332 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.162939+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.261392+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.604684+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.605538+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.605878+0000 mon.a (mon.0) 969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.606509+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:14.606717+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:15.333 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:13.775370+0000 mgr.y (mgr.24454) 168 : cluster [DBG] pgmap v161: 610 pgs: 27 creating+peering, 494 active+clean, 89 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-02-18T00:33:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.090816+0000 mon.a (mon.0) 950 : audit [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-18T00:33:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.099237+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.099315+0000 mon.a (mon.0) 952 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.099396+0000 mon.a (mon.0) 953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.099447+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.099497+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.099654+0000 mon.a (mon.0) 956 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-4"}]': finished 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.099793+0000 mon.a (mon.0) 957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:14.099874+0000 mon.a (mon.0) 958 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.102721+0000 mon.c (mon.2) 54 : audit [INF] from='client.? 172.21.15.59:0/4203989618' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.103937+0000 mon.b (mon.1) 85 : audit [INF] from='client.? 172.21.15.59:0/2343408540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.104606+0000 mon.a (mon.0) 959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]: dispatch 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.104893+0000 mon.a (mon.0) 960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]: dispatch 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.105760+0000 mon.a (mon.0) 961 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]: dispatch 2024-02-18T00:33:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.109001+0000 mon.b (mon.1) 86 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.110083+0000 mon.b (mon.1) 87 : audit [INF] from='client.? 172.21.15.59:0/1926350508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.110193+0000 mon.a (mon.0) 962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.110746+0000 mon.a (mon.0) 963 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.111188+0000 mon.b (mon.1) 88 : audit [INF] from='client.? 172.21.15.59:0/2294550938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.112531+0000 mon.a (mon.0) 964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.118180+0000 mon.a (mon.0) 965 : audit [INF] from='client.? 172.21.15.59:0/3857141535' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi059-82877-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.123123+0000 mon.b (mon.1) 89 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2"}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.125944+0000 mon.b (mon.1) 90 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.131600+0000 mon.a (mon.0) 966 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.131984+0000 mon.a (mon.0) 967 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:15.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.162939+0000 mon.b (mon.1) 91 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.261392+0000 mon.b (mon.1) 92 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:15.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.604684+0000 mon.a (mon.0) 968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.605538+0000 mon.b (mon.1) 93 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.605878+0000 mon.a (mon.0) 969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:15.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.606509+0000 mon.b (mon.1) 94 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:15.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:14.606717+0000 mon.a (mon.0) 970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:16.096 INFO:tasks.workunit.client.0.smithi059.stdout:18T00:33:12.555546+0000 mon.b [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:16.096 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.558576+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:16.096 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:12.558910+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pg_num": 4}]: dispatch 2024-02-18T00:33:16.096 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:13.074859+0000 client.admin [INF] threexx 2024-02-18T00:33:16.096 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: got: 2024-02-18T00:33:13.075069+0000 mon.a [INF] from='client.? 172.21.15.59:0/4020970402' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T00:33:16.096 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (9187 ms) 2024-02-18T00:33:16.096 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (25187 ms total) 2024-02-18T00:33:16.097 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: 2024-02-18T00:33:16.097 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-18T00:33:16.097 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (25187 ms total) 2024-02-18T00:33:16.097 INFO:tasks.workunit.client.0.smithi059.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-18T00:33:16.279 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:16.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:16.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:16.309 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:16.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:16.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:15.096834+0000 mon.a (mon.0) 971 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:16.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102069+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:16.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102192+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102292+0000 mon.a (mon.0) 974 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102459+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102560+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102651+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102751+0000 mon.a (mon.0) 978 : audit [INF] from='client.? 172.21.15.59:0/3857141535' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi059-82877-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102884+0000 mon.a (mon.0) 979 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.102966+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.103068+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:15.103128+0000 mon.a (mon.0) 982 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.115900+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]: dispatch 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.118070+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.59:0/1888077111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.120373+0000 mon.a (mon.0) 983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]: dispatch 2024-02-18T00:33:16.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.120641+0000 mon.a (mon.0) 984 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:16.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.121526+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:16.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.123774+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-18T00:33:16.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.125929+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:33:16.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.128653+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:16.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.129392+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:33:16.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.131162+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:16.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:15.262447+0000 mon.b (mon.1) 99 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:15.096834+0000 mon.a (mon.0) 971 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102069+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102192+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102292+0000 mon.a (mon.0) 974 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102459+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102560+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102651+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102751+0000 mon.a (mon.0) 978 : audit [INF] from='client.? 172.21.15.59:0/3857141535' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi059-82877-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102884+0000 mon.a (mon.0) 979 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.102966+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.103068+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:16.419 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:15.103128+0000 mon.a (mon.0) 982 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.115900+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.118070+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.59:0/1888077111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.120373+0000 mon.a (mon.0) 983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.120641+0000 mon.a (mon.0) 984 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.121526+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.123774+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.125929+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.128653+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.129392+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.131162+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:16.420 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:15.262447+0000 mon.b (mon.1) 99 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:16.421 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:15.096834+0000 mon.a (mon.0) 971 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102069+0000 mon.a (mon.0) 972 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi059-82901-22"}]': finished 2024-02-18T00:33:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102192+0000 mon.a (mon.0) 973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi059-82989-6"}]': finished 2024-02-18T00:33:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102292+0000 mon.a (mon.0) 974 : audit [INF] from='client.? 172.21.15.59:0/933229290' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi059-82997-7"}]': finished 2024-02-18T00:33:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102459+0000 mon.a (mon.0) 975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]': finished 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102560+0000 mon.a (mon.0) 976 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi059-82893-15"}]': finished 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102651+0000 mon.a (mon.0) 977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102751+0000 mon.a (mon.0) 978 : audit [INF] from='client.? 172.21.15.59:0/3857141535' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi059-82877-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102884+0000 mon.a (mon.0) 979 : audit [INF] from='client.? 172.21.15.59:0/3309993667' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-4", "tierpool": "test-rados-api-smithi059-82957-4-cache"}]': finished 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.102966+0000 mon.a (mon.0) 980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi059-82941-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.103068+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:15.103128+0000 mon.a (mon.0) 982 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.115900+0000 mon.b (mon.1) 95 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]: dispatch 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.118070+0000 mon.c (mon.2) 55 : audit [INF] from='client.? 172.21.15.59:0/1888077111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.120373+0000 mon.a (mon.0) 983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]: dispatch 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.120641+0000 mon.a (mon.0) 984 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.121526+0000 mon.b (mon.1) 96 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:16.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.123774+0000 mon.b (mon.1) 97 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-18T00:33:16.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.125929+0000 mon.b (mon.1) 98 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:33:16.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.128653+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:16.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.129392+0000 mon.a (mon.0) 986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:33:16.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.131162+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:16.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:15.262447+0000 mon.b (mon.1) 99 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:15.776979+0000 mgr.y (mgr.24454) 169 : cluster [DBG] pgmap v164: 634 pgs: 32 creating+peering, 602 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.7 KiB/s rd, 3 op/s 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:16.099236+0000 mon.a (mon.0) 988 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.109247+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]': finished 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.109363+0000 mon.a (mon.0) 990 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.264650+0000 mon.b (mon.1) 100 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.580931+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.580999+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:16.581104+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.582022+0000 mon.a (mon.0) 994 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.583556+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:16.585561+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:15.776979+0000 mgr.y (mgr.24454) 169 : cluster [DBG] pgmap v164: 634 pgs: 32 creating+peering, 602 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.7 KiB/s rd, 3 op/s 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:16.099236+0000 mon.a (mon.0) 988 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.109247+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]': finished 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.109363+0000 mon.a (mon.0) 990 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.264650+0000 mon.b (mon.1) 100 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.580931+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.580999+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:16.581104+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.582022+0000 mon.a (mon.0) 994 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.583556+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:16.585561+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:15.776979+0000 mgr.y (mgr.24454) 169 : cluster [DBG] pgmap v164: 634 pgs: 32 creating+peering, 602 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.7 KiB/s rd, 3 op/s 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:16.099236+0000 mon.a (mon.0) 988 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.109247+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "mode": "writeback"}]': finished 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.109363+0000 mon.a (mon.0) 990 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.264650+0000 mon.b (mon.1) 100 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.580931+0000 mon.a (mon.0) 991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.580999+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:16.581104+0000 mon.a (mon.0) 993 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-18T00:33:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.582022+0000 mon.a (mon.0) 994 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]: dispatch 2024-02-18T00:33:17.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.583556+0000 mon.b (mon.1) 101 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:33:17.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:16.585561+0000 mon.a (mon.0) 995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:33:17.974 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: Running main() from gmock_main.cc 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [----------] Global test environment set-up. 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (16 ms) 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.LockShared (4 ms) 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1004 ms) 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1022 ms) 2024-02-18T00:33:17.975 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (8 ms) 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.Unlock (14 ms) 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (4 ms) 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [----------] 8 tests from LibRadosLock (2075 ms total) 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1106 ms) 2024-02-18T00:33:17.976 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (4 ms) 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1012 ms) 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1068 ms) 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-02-18T00:33:17.977 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3205 ms total) 2024-02-18T00:33:17.978 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: 2024-02-18T00:33:17.978 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [----------] Global test environment tear-down 2024-02-18T00:33:17.978 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (27029 ms total) 2024-02-18T00:33:17.978 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-18T00:33:18.183 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:18.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:18.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:18.287 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:18.319 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:18.340 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:18.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:18.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:18.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:18.476 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:18.683 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:18.686 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:18.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.266321+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.498998+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.499101+0000 mon.a (mon.0) 997 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.499240+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.504009+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.59:0/556484139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.504898+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.505081+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.59:0/2002381519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.536015+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:17.777395+0000 mgr.y (mgr.24454) 170 : cluster [DBG] pgmap v167: 506 pgs: 72 unknown, 434 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-18T00:33:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:17.942649+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:17.942803+0000 mon.a (mon.0) 1000 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:17.943229+0000 mon.a (mon.0) 1001 : cluster [WRN] pool 'PoolQuotaPP_smithi059-82882-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:17.944541+0000 mon.a (mon.0) 1002 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.266321+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.498998+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.499101+0000 mon.a (mon.0) 997 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.499240+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.504009+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.59:0/556484139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.504898+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.505081+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.59:0/2002381519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.536015+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:17.777395+0000 mgr.y (mgr.24454) 170 : cluster [DBG] pgmap v167: 506 pgs: 72 unknown, 434 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:17.942649+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:17.942803+0000 mon.a (mon.0) 1000 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:17.943229+0000 mon.a (mon.0) 1001 : cluster [WRN] pool 'PoolQuotaPP_smithi059-82882-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:17.944541+0000 mon.a (mon.0) 1002 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.948519+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:17.955541+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.959679+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:18.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.960123+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:17.960578+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.267383+0000 mon.b (mon.1) 104 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.405055+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? 172.21.15.59:0/310625823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi059-82973-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.948519+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:17.955541+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.959679+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.960123+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:17.960578+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.267383+0000 mon.b (mon.1) 104 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:18.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.405055+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? 172.21.15.59:0/310625823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi059-82973-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.266321+0000 mon.b (mon.1) 102 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:18.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.498998+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi059-83005-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:18.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.499101+0000 mon.a (mon.0) 997 : audit [INF] from='client.? 172.21.15.59:0/1750819307' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi059-82925-10"}]': finished 2024-02-18T00:33:18.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.499240+0000 mon.a (mon.0) 998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T00:33:18.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.504009+0000 mon.c (mon.2) 56 : audit [INF] from='client.? 172.21.15.59:0/556484139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.504898+0000 mon.c (mon.2) 57 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.505081+0000 mon.c (mon.2) 58 : audit [INF] from='client.? 172.21.15.59:0/2002381519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.536015+0000 mon.b (mon.1) 103 : audit [INF] from='client.? 172.21.15.59:0/3659768887' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:17.777395+0000 mgr.y (mgr.24454) 170 : cluster [DBG] pgmap v167: 506 pgs: 72 unknown, 434 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:17.942649+0000 mon.a (mon.0) 999 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:17.942803+0000 mon.a (mon.0) 1000 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:17.943229+0000 mon.a (mon.0) 1001 : cluster [WRN] pool 'PoolQuotaPP_smithi059-82882-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:17.944541+0000 mon.a (mon.0) 1002 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.948519+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:17.955541+0000 mon.a (mon.0) 1004 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.959679+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.960123+0000 mon.a (mon.0) 1006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:17.960578+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.267383+0000 mon.b (mon.1) 104 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:18.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.405055+0000 mon.a (mon.0) 1008 : audit [INF] from='client.? 172.21.15.59:0/310625823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi059-82973-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:18.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:18.816 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:19.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-18T00:33:19.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-18T00:33:19.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-18T00:33:19.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-18T00:33:19.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: notify 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1336 ms) 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (16 ms) 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1352 ms total) 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: notify 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (11 ms) 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: notify 2024-02-18T00:33:19.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3624 ms) 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (5 ms) 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (6 ms) 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 511101108225 notifier_gid 25141 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 511101108224 notifier_gid 25141 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-18T00:33:19.228 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 511101108225 notifier_gid 25141 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 511101108226 notifier_gid 25141 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 511101108226 notifier_gid 25141 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (4 ms) 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 511101108227 notifier_gid 25141 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: trying... 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 511101108226 notifier_gid 25141 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: timed out 2024-02-18T00:33:19.229 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: flushing 2024-02-18T00:33:19.611 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:33:19.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 -- 172.21.15.59:0/2717476892 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80f6a90 msgr2=0x7f1ac8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:19.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2717476892 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80f6a90 0x7f1ac8104a20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f1ac0009be0 tx=0x7f1ac0004d10).stop 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 -- 172.21.15.59:0/2717476892 shutdown_connections 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2717476892 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80f6a90 0x7f1ac8104a20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2717476892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac80fdbd0 0x7f1ac80fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2717476892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ac80fe530 0x7f1ac80f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 -- 172.21.15.59:0/2717476892 >> 172.21.15.59:0/2717476892 conn(0x7f1ac806c370 msgr2=0x7f1ac806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 -- 172.21.15.59:0/2717476892 shutdown_connections 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 -- 172.21.15.59:0/2717476892 wait complete. 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 librados: starting msgr at 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 librados: starting objecter 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 Processor -- start 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 -- start start 2024-02-18T00:33:19.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 librados: setting wanted keys 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.912+0000 7f1ad012d700 1 librados: calling monclient init 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ad012d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac80f6a90 0x7f1ac80f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ad012d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80fdbd0 0x7f1ac80f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ad012d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ac80fe530 0x7f1ac80f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ad012d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8116490 con 0x7f1ac80f6a90 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ad012d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8116600 con 0x7f1ac80fe530 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ad012d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac81167c0 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80fdbd0 0x7f1ac80f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80fdbd0 0x7f1ac80f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:45094/0 (socket says 172.21.15.59:45094) 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 -- 172.21.15.59:0/2058884919 learned_addr learned my addr 172.21.15.59:0/2058884919 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acdec9700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac80f6a90 0x7f1ac80f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ace6ca700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ac80fe530 0x7f1ac80f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:19.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 -- 172.21.15.59:0/2058884919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ac80fe530 msgr2=0x7f1ac80f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ac80fe530 0x7f1ac80f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 -- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac80f6a90 msgr2=0x7f1ac80f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac80f6a90 0x7f1ac80f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 -- 172.21.15.59:0/2058884919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8005040 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ace6ca700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ac80fe530 0x7f1ac80f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1acd6c8700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80fdbd0 0x7f1ac80f7b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f1ac400b530 tx=0x7f1ac40090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1abeffd700 10 client.?.objecter ms_handle_connect 0x7f1ac80fdbd0 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1abeffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4002320 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac4002950 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40129a0 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.913+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac00097c0 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.914+0000 7f1ad012d700 10 client.25214.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.914+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac8116db0 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.914+0000 7f1ad012d700 1 librados: init done 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac805f950 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1ac4002480 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 1 --2- 172.21.15.59:0/2058884919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1ab4062c90 0x7f1ab4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(121..121 src has 44..121) v4 ==== 12718+0+0 (secure 0 0 0) 0x7f1ac407e410 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 10 client.25214.objecter ms_dispatch 0x7f1ac8001550 osd_map(121..121 src has 44..121) v4 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 3 client.25214.objecter handle_osd_map got epochs [121,121] > 0 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 3 client.25214.objecter handle_osd_map decoding full epoch 121 2024-02-18T00:33:19.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1acdec9700 1 --2- 172.21.15.59:0/2058884919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1ab4062c90 0x7f1ab4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 10 client.25214.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=122}) v3 -- 0x7f1ab4078350 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 20 client.25214.objecter dump_active .. 0 homeless 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1abeffd700 10 client.25214.objecter ms_handle_connect 0x7f1ab4062c90 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.915+0000 7f1acdec9700 1 --2- 172.21.15.59:0/2058884919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1ab4062c90 0x7f1ab4065150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f1ab8009350 tx=0x7f1ab800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.919+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1ac40506d0 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.953+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mon.2 v2:172.21.15.59:3301/0 7 ==== osd_map(122..122 src has 44..122) v4 ==== 2585+0+0 (secure 0 0 0) 0x7f1ac4002ac0 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.953+0000 7f1abeffd700 10 client.25214.objecter ms_dispatch 0x7f1ac8001550 osd_map(122..122 src has 44..122) v4 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.953+0000 7f1abeffd700 3 client.25214.objecter handle_osd_map got epochs [122,122] > 121 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.953+0000 7f1abeffd700 3 client.25214.objecter handle_osd_map decoding incremental epoch 122 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.953+0000 7f1abeffd700 10 client.25214.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:19.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.953+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=123}) v3 -- 0x7f1ab406bc80 con 0x7f1ac80fdbd0 2024-02-18T00:33:19.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:18.953+0000 7f1abeffd700 20 client.25214.objecter dump_active .. 0 homeless 2024-02-18T00:33:19.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.192+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ac80f8b60 con 0x7f1ab4062c90 2024-02-18T00:33:19.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.265+0000 7f1abeffd700 1 -- 172.21.15.59:0/2058884919 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+1287882 (secure 0 0 0) 0x7f1ac80f8b60 con 0x7f1ab4062c90 2024-02-18T00:33:19.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 10 librados: watch_flush enter 2024-02-18T00:33:19.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 10 librados: watch_flush exit 2024-02-18T00:33:19.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 20 client.25214.objecter shutdown clearing up homeless session... 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 10 client.25214.objecter successfully canceled tick 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1ab4062c90 msgr2=0x7f1ab4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2058884919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1ab4062c90 0x7f1ab4065150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f1ab8009350 tx=0x7f1ab800b040).stop 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80fdbd0 msgr2=0x7f1ac80f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.644+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80fdbd0 0x7f1ac80f7b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f1ac400b530 tx=0x7f1ac40090f0).stop 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 shutdown_connections 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2058884919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1ab4062c90 0x7f1ab4065150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ac80fe530 0x7f1ac80f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ac80f6a90 0x7f1ac80f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 --2- 172.21.15.59:0/2058884919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ac80fdbd0 0x7f1ac80f7b70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:19.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 >> 172.21.15.59:0/2058884919 conn(0x7f1ac806c370 msgr2=0x7f1ac80f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:19.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 shutdown_connections 2024-02-18T00:33:19.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.645+0000 7f1ad012d700 1 -- 172.21.15.59:0/2058884919 wait complete. 2024-02-18T00:33:19.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:19.646+0000 7f1ad012d700 1 librados: shutdown 2024-02-18T00:33:19.669 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_watc api_lock_pp: Running main() from gmock_main.cc 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: seed 82933 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (100 ms) 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (9 ms) 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1030 ms) 2024-02-18T00:33:20.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1160 ms) 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (24 ms) 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (20 ms) 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (10 ms) 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (6 ms) 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2359 ms total) 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-18T00:33:20.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1060 ms) 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (2 ms) 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1006 ms) 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1009 ms) 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (5 ms) 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-18T00:33:20.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (115 ms) 2024-02-18T00:33:20.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-18T00:33:20.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (5 ms) 2024-02-18T00:33:20.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3206 ms total) 2024-02-18T00:33:20.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: 2024-02-18T00:33:20.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-18T00:33:20.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (29135 ms total) 2024-02-18T00:33:20.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-18T00:33:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.956487+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.310 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.956607+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:20.310 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.956716+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.956816+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]': finished 2024-02-18T00:33:20.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.956995+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? 172.21.15.59:0/310625823' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi059-82973-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:18.957086+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-18T00:33:20.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.962113+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:20.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.967359+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:20.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.970831+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.59:0/1897995270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:20.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:18.987542+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:20.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:19.194511+0000 mgr.y (mgr.24454) 171 : audit [DBG] from='client.25214 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:33:20.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:19.268432+0000 mon.b (mon.1) 105 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:20.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.956487+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.956607+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:20.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.956716+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.956816+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]': finished 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.956995+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? 172.21.15.59:0/310625823' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi059-82973-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:18.957086+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.962113+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.967359+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.970831+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.59:0/1897995270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:18.987542+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:19.194511+0000 mgr.y (mgr.24454) 171 : audit [DBG] from='client.25214 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:33:20.314 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:19.268432+0000 mon.b (mon.1) 105 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:20.315 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.956487+0000 mon.a (mon.0) 1009 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.315 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.956607+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:20.315 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.956716+0000 mon.a (mon.0) 1011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi059-82877-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.315 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.956816+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi059-82941-1","app":"app1","key":"key1"}]': finished 2024-02-18T00:33:20.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.956995+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? 172.21.15.59:0/310625823' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi059-82973-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:20.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:18.957086+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-18T00:33:20.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.962113+0000 mon.c (mon.2) 59 : audit [INF] from='client.? 172.21.15.59:0/1439959833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:20.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.967359+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]: dispatch 2024-02-18T00:33:20.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.970831+0000 mon.c (mon.2) 60 : audit [INF] from='client.? 172.21.15.59:0/1897995270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:20.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:18.987542+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:20.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:19.194511+0000 mgr.y (mgr.24454) 171 : audit [DBG] from='client.25214 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:33:20.317 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:19.268432+0000 mon.b (mon.1) 105 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:20.568 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:21.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:19.778398+0000 mgr.y (mgr.24454) 172 : cluster [DBG] pgmap v170: 538 pgs: 136 unknown, 402 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:21.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:19.829338+0000 osd.5 (osd.5) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:21.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:19.829421+0000 osd.5 (osd.5) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:21.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:19.966198+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:21.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:19.966335+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:21.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:19.966499+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-18T00:33:21.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:19.972770+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:21.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:19.984982+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:21.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:20.011675+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:21.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:20.015595+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:21.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:20.269368+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:21.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:33:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:19.778398+0000 mgr.y (mgr.24454) 172 : cluster [DBG] pgmap v170: 538 pgs: 136 unknown, 402 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:19.829338+0000 osd.5 (osd.5) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:19.829421+0000 osd.5 (osd.5) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:19.966198+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:19.966335+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:19.966499+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:19.972770+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:19.984982+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:21.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:20.011675+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:21.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:20.015595+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:21.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:20.269368+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:21.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:19.778398+0000 mgr.y (mgr.24454) 172 : cluster [DBG] pgmap v170: 538 pgs: 136 unknown, 402 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:21.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:19.829338+0000 osd.5 (osd.5) 75 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:21.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:19.829421+0000 osd.5 (osd.5) 76 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:19.966198+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi059-82933-10"}]': finished 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:19.966335+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:19.966499+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:19.972770+0000 mon.b (mon.1) 106 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:19.984982+0000 mon.c (mon.2) 61 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:20.011675+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:20.015595+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:21.257 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:20.269368+0000 mon.b (mon.1) 107 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:21.376 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-02-18T00:33:18.953898+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151032697,"num_objects":285,"num_object_clones":0,"num_object_copies":855,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":0,"num_read":2354,"num_read_kb":50223,"num_write":1788,"num_write_kb":153120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":983,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1884,"ondisk_log_size":1884,"up":1206,"acting":1206,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1812,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2898972,"kb_used_data":452572,"kb_used_omap":0,"kb_used_meta":2446336,"kb_avail":747062244,"statfs":{"total":767960285184,"available":764991737856,"internally_reserved":0,"allocated":463433728,"data_stored":458193495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2505048064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21006,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":8,"num_write":9,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.853945"},"pg_stats":[{"pgid":"61.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.133003+0000","last_change":"2024-02-18T00:33:13.176264+0000","last_active":"2024-02-18T00:33:14.133003+0000","last_peered":"2024-02-18T00:33:14.133003+0000","last_clean":"2024-02-18T00:33:14.133003+0000","last_became_active":"2024-02-18T00:33:13.175988+0000","last_became_peered":"2024-02-18T00:33:13.175988+0000","last_unstale":"2024-02-18T00:33:14.133003+0000","last_undegraded":"2024-02-18T00:33:14.133003+0000","last_fullsized":"2024-02-18T00:33:14.133003+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.12","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106832+0000","last_change":"2024-02-18T00:33:12.169788+0000","last_active":"2024-02-18T00:33:14.106832+0000","last_peered":"2024-02-18T00:33:14.106832+0000","last_clean":"2024-02-18T00:33:14.106832+0000","last_became_active":"2024-02-18T00:33:12.169061+0000","last_became_peered":"2024-02-18T00:33:12.169061+0000","last_unstale":"2024-02-18T00:33:14.106832+0000","last_undegraded":"2024-02-18T00:33:14.106832+0000","last_fullsized":"2024-02-18T00:33:14.106832+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.16","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.105785+0000","last_change":"2024-02-18T00:33:07.528124+0000","last_active":"2024-02-18T00:33:14.105785+0000","last_peered":"2024-02-18T00:33:14.105785+0000","last_clean":"2024-02-18T00:33:14.105785+0000","last_became_active":"2024-02-18T00:33:07.520618+0000","last_became_peered":"2024-02-18T00:33:07.520618+0000","last_unstale":"2024-02-18T00:33:14.105785+0000","last_undegraded":"2024-02-18T00:33:14.105785+0000","last_fullsized":"2024-02-18T00:33:14.105785+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138878+0000","last_change":"2024-02-18T00:33:05.581923+0000","last_active":"2024-02-18T00:33:14.138878+0000","last_peered":"2024-02-18T00:33:14.138878+0000","last_clean":"2024-02-18T00:33:14.138878+0000","last_became_active":"2024-02-18T00:33:05.581506+0000","last_became_peered":"2024-02-18T00:33:05.581506+0000","last_unstale":"2024-02-18T00:33:14.138878+0000","last_undegraded":"2024-02-18T00:33:14.138878+0000","last_fullsized":"2024-02-18T00:33:14.138878+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"61.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106841+0000","last_change":"2024-02-18T00:33:13.180556+0000","last_active":"2024-02-18T00:33:14.106841+0000","last_peered":"2024-02-18T00:33:14.106841+0000","last_clean":"2024-02-18T00:33:14.106841+0000","last_became_active":"2024-02-18T00:33:13.180256+0000","last_became_peered":"2024-02-18T00:33:13.180256+0000","last_unstale":"2024-02-18T00:33:14.106841+0000","last_undegraded":"2024-02-18T00:33:14.106841+0000","last_fullsized":"2024-02-18T00:33:14.106841+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.13","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.120753+0000","last_change":"2024-02-18T00:33:12.169684+0000","last_active":"2024-02-18T00:33:14.120753+0000","last_peered":"2024-02-18T00:33:14.120753+0000","last_clean":"2024-02-18T00:33:14.120753+0000","last_became_active":"2024-02-18T00:33:12.167115+0000","last_became_peered":"2024-02-18T00:33:12.167115+0000","last_unstale":"2024-02-18T00:33:14.120753+0000","last_undegraded":"2024-02-18T00:33:14.120753+0000","last_fullsized":"2024-02-18T00:33:14.120753+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.17","version":"117'1","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.188589+0000","last_change":"2024-02-18T00:33:07.486481+0000","last_active":"2024-02-18T00:33:14.188589+0000","last_peered":"2024-02-18T00:33:14.188589+0000","last_clean":"2024-02-18T00:33:14.188589+0000","last_became_active":"2024-02-18T00:33:07.486168+0000","last_became_peered":"2024-02-18T00:33:07.486168+0000","last_unstale":"2024-02-18T00:33:14.188589+0000","last_undegraded":"2024-02-18T00:33:14.188589+0000","last_fullsized":"2024-02-18T00:33:14.188589+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.142448+0000","last_change":"2024-02-18T00:33:05.560917+0000","last_active":"2024-02-18T00:33:14.142448+0000","last_peered":"2024-02-18T00:33:14.142448+0000","last_clean":"2024-02-18T00:33:14.142448+0000","last_became_active":"2024-02-18T00:33:05.560630+0000","last_became_peered":"2024-02-18T00:33:05.560630+0000","last_unstale":"2024-02-18T00:33:14.142448+0000","last_undegraded":"2024-02-18T00:33:14.142448+0000","last_fullsized":"2024-02-18T00:33:14.142448+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"61.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.f","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137452+0000","last_change":"2024-02-18T00:33:13.182928+0000","last_active":"2024-02-18T00:33:14.137452+0000","last_peered":"2024-02-18T00:33:14.137452+0000","last_clean":"2024-02-18T00:33:14.137452+0000","last_became_active":"2024-02-18T00:33:13.182750+0000","last_became_peered":"2024-02-18T00:33:13.182750+0000","last_unstale":"2024-02-18T00:33:14.137452+0000","last_undegraded":"2024-02-18T00:33:14.137452+0000","last_fullsized":"2024-02-18T00:33:14.137452+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.10","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.100985+0000","last_change":"2024-02-18T00:33:12.158979+0000","last_active":"2024-02-18T00:33:14.100985+0000","last_peered":"2024-02-18T00:33:14.100985+0000","last_clean":"2024-02-18T00:33:14.100985+0000","last_became_active":"2024-02-18T00:33:12.158548+0000","last_became_peered":"2024-02-18T00:33:12.158548+0000","last_unstale":"2024-02-18T00:33:14.100985+0000","last_undegraded":"2024-02-18T00:33:14.100985+0000","last_fullsized":"2024-02-18T00:33:14.100985+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.14","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124858+0000","last_change":"2024-02-18T00:33:07.488438+0000","last_active":"2024-02-18T00:33:14.124858+0000","last_peered":"2024-02-18T00:33:14.124858+0000","last_clean":"2024-02-18T00:33:14.124858+0000","last_became_active":"2024-02-18T00:33:07.488020+0000","last_became_peered":"2024-02-18T00:33:07.488020+0000","last_unstale":"2024-02-18T00:33:14.124858+0000","last_undegraded":"2024-02-18T00:33:14.124858+0000","last_fullsized":"2024-02-18T00:33:14.124858+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.112268+0000","last_change":"2024-02-18T00:33:05.591088+0000","last_active":"2024-02-18T00:33:14.112268+0000","last_peered":"2024-02-18T00:33:14.112268+0000","last_clean":"2024-02-18T00:33:14.112268+0000","last_became_active":"2024-02-18T00:33:05.590799+0000","last_became_peered":"2024-02-18T00:33:05.590799+0000","last_unstale":"2024-02-18T00:33:14.112268+0000","last_undegraded":"2024-02-18T00:33:14.112268+0000","last_fullsized":"2024-02-18T00:33:14.112268+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"61.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138207+0000","last_change":"2024-02-18T00:33:13.140774+0000","last_active":"2024-02-18T00:33:14.138207+0000","last_peered":"2024-02-18T00:33:14.138207+0000","last_clean":"2024-02-18T00:33:14.138207+0000","last_became_active":"2024-02-18T00:33:13.140452+0000","last_became_peered":"2024-02-18T00:33:13.140452+0000","last_unstale":"2024-02-18T00:33:14.138207+0000","last_undegraded":"2024-02-18T00:33:14.138207+0000","last_fullsized":"2024-02-18T00:33:14.138207+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.11","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128310+0000","last_change":"2024-02-18T00:33:12.136333+0000","last_active":"2024-02-18T00:33:14.128310+0000","last_peered":"2024-02-18T00:33:14.128310+0000","last_clean":"2024-02-18T00:33:14.128310+0000","last_became_active":"2024-02-18T00:33:12.135512+0000","last_became_peered":"2024-02-18T00:33:12.135512+0000","last_unstale":"2024-02-18T00:33:14.128310+0000","last_undegraded":"2024-02-18T00:33:14.128310+0000","last_fullsized":"2024-02-18T00:33:14.128310+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.15","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.116597+0000","last_change":"2024-02-18T00:33:07.524335+0000","last_active":"2024-02-18T00:33:14.116597+0000","last_peered":"2024-02-18T00:33:14.116597+0000","last_clean":"2024-02-18T00:33:14.116597+0000","last_became_active":"2024-02-18T00:33:07.522427+0000","last_became_peered":"2024-02-18T00:33:07.522427+0000","last_unstale":"2024-02-18T00:33:14.116597+0000","last_undegraded":"2024-02-18T00:33:14.116597+0000","last_fullsized":"2024-02-18T00:33:14.116597+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106259+0000","last_change":"2024-02-18T00:33:05.576865+0000","last_active":"2024-02-18T00:33:14.106259+0000","last_peered":"2024-02-18T00:33:14.106259+0000","last_clean":"2024-02-18T00:33:14.106259+0000","last_became_active":"2024-02-18T00:33:05.576577+0000","last_became_peered":"2024-02-18T00:33:05.576577+0000","last_unstale":"2024-02-18T00:33:14.106259+0000","last_undegraded":"2024-02-18T00:33:14.106259+0000","last_fullsized":"2024-02-18T00:33:14.106259+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"61.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.111887+0000","last_change":"2024-02-18T00:33:13.166112+0000","last_active":"2024-02-18T00:33:14.111887+0000","last_peered":"2024-02-18T00:33:14.111887+0000","last_clean":"2024-02-18T00:33:14.111887+0000","last_became_active":"2024-02-18T00:33:13.165794+0000","last_became_peered":"2024-02-18T00:33:13.165794+0000","last_unstale":"2024-02-18T00:33:14.111887+0000","last_undegraded":"2024-02-18T00:33:14.111887+0000","last_fullsized":"2024-02-18T00:33:14.111887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.16","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106911+0000","last_change":"2024-02-18T00:33:12.166816+0000","last_active":"2024-02-18T00:33:14.106911+0000","last_peered":"2024-02-18T00:33:14.106911+0000","last_clean":"2024-02-18T00:33:14.106911+0000","last_became_active":"2024-02-18T00:33:12.166501+0000","last_became_peered":"2024-02-18T00:33:12.166501+0000","last_unstale":"2024-02-18T00:33:14.106911+0000","last_undegraded":"2024-02-18T00:33:14.106911+0000","last_fullsized":"2024-02-18T00:33:14.106911+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.12","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122078+0000","last_change":"2024-02-18T00:33:07.479453+0000","last_active":"2024-02-18T00:33:14.122078+0000","last_peered":"2024-02-18T00:33:14.122078+0000","last_clean":"2024-02-18T00:33:14.122078+0000","last_became_active":"2024-02-18T00:33:07.479016+0000","last_became_peered":"2024-02-18T00:33:07.479016+0000","last_unstale":"2024-02-18T00:33:14.122078+0000","last_undegraded":"2024-02-18T00:33:14.122078+0000","last_fullsized":"2024-02-18T00:33:14.122078+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.110419+0000","last_change":"2024-02-18T00:33:05.588506+0000","last_active":"2024-02-18T00:33:14.110419+0000","last_peered":"2024-02-18T00:33:14.110419+0000","last_clean":"2024-02-18T00:33:14.110419+0000","last_became_active":"2024-02-18T00:33:05.588248+0000","last_became_peered":"2024-02-18T00:33:05.588248+0000","last_unstale":"2024-02-18T00:33:14.110419+0000","last_undegraded":"2024-02-18T00:33:14.110419+0000","last_fullsized":"2024-02-18T00:33:14.110419+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"61.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102171+0000","last_change":"2024-02-18T00:33:13.173187+0000","last_active":"2024-02-18T00:33:14.102171+0000","last_peered":"2024-02-18T00:33:14.102171+0000","last_clean":"2024-02-18T00:33:14.102171+0000","last_became_active":"2024-02-18T00:33:13.172912+0000","last_became_peered":"2024-02-18T00:33:13.172912+0000","last_unstale":"2024-02-18T00:33:14.102171+0000","last_undegraded":"2024-02-18T00:33:14.102171+0000","last_fullsized":"2024-02-18T00:33:14.102171+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.17","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135466+0000","last_change":"2024-02-18T00:33:12.121235+0000","last_active":"2024-02-18T00:33:14.135466+0000","last_peered":"2024-02-18T00:33:14.135466+0000","last_clean":"2024-02-18T00:33:14.135466+0000","last_became_active":"2024-02-18T00:33:12.120324+0000","last_became_peered":"2024-02-18T00:33:12.120324+0000","last_unstale":"2024-02-18T00:33:14.135466+0000","last_undegraded":"2024-02-18T00:33:14.135466+0000","last_fullsized":"2024-02-18T00:33:14.135466+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.13","version":"115'2","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.139753+0000","last_change":"2024-02-18T00:33:07.503730+0000","last_active":"2024-02-18T00:33:14.139753+0000","last_peered":"2024-02-18T00:33:14.139753+0000","last_clean":"2024-02-18T00:33:14.139753+0000","last_became_active":"2024-02-18T00:33:07.503024+0000","last_became_peered":"2024-02-18T00:33:07.503024+0000","last_unstale":"2024-02-18T00:33:14.139753+0000","last_undegraded":"2024-02-18T00:33:14.139753+0000","last_fullsized":"2024-02-18T00:33:14.139753+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8192,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102263+0000","last_change":"2024-02-18T00:33:05.593560+0000","last_active":"2024-02-18T00:33:14.102263+0000","last_peered":"2024-02-18T00:33:14.102263+0000","last_clean":"2024-02-18T00:33:14.102263+0000","last_became_active":"2024-02-18T00:33:05.593261+0000","last_became_peered":"2024-02-18T00:33:05.593261+0000","last_unstale":"2024-02-18T00:33:14.102263+0000","last_undegraded":"2024-02-18T00:33:14.102263+0000","last_fullsized":"2024-02-18T00:33:14.102263+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"61.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.120669+0000","last_change":"2024-02-18T00:33:13.150944+0000","last_active":"2024-02-18T00:33:14.120669+0000","last_peered":"2024-02-18T00:33:14.120669+0000","last_clean":"2024-02-18T00:33:14.120669+0000","last_became_active":"2024-02-18T00:33:13.150511+0000","last_became_peered":"2024-02-18T00:33:13.150511+0000","last_unstale":"2024-02-18T00:33:14.120669+0000","last_undegraded":"2024-02-18T00:33:14.120669+0000","last_fullsized":"2024-02-18T00:33:14.120669+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.14","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102162+0000","last_change":"2024-02-18T00:33:12.152338+0000","last_active":"2024-02-18T00:33:14.102162+0000","last_peered":"2024-02-18T00:33:14.102162+0000","last_clean":"2024-02-18T00:33:14.102162+0000","last_became_active":"2024-02-18T00:33:12.151163+0000","last_became_peered":"2024-02-18T00:33:12.151163+0000","last_unstale":"2024-02-18T00:33:14.102162+0000","last_undegraded":"2024-02-18T00:33:14.102162+0000","last_fullsized":"2024-02-18T00:33:14.102162+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.10","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113771+0000","last_change":"2024-02-18T00:33:07.485443+0000","last_active":"2024-02-18T00:33:14.113771+0000","last_peered":"2024-02-18T00:33:14.113771+0000","last_clean":"2024-02-18T00:33:14.113771+0000","last_became_active":"2024-02-18T00:33:07.485146+0000","last_became_peered":"2024-02-18T00:33:07.485146+0000","last_unstale":"2024-02-18T00:33:14.113771+0000","last_undegraded":"2024-02-18T00:33:14.113771+0000","last_fullsized":"2024-02-18T00:33:14.113771+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124038+0000","last_change":"2024-02-18T00:33:05.592933+0000","last_active":"2024-02-18T00:33:14.124038+0000","last_peered":"2024-02-18T00:33:14.124038+0000","last_clean":"2024-02-18T00:33:14.124038+0000","last_became_active":"2024-02-18T00:33:05.592651+0000","last_became_peered":"2024-02-18T00:33:05.592651+0000","last_unstale":"2024-02-18T00:33:14.124038+0000","last_undegraded":"2024-02-18T00:33:14.124038+0000","last_fullsized":"2024-02-18T00:33:14.124038+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"61.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"57.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.112843+0000","last_change":"2024-02-18T00:33:13.179390+0000","last_active":"2024-02-18T00:33:14.112843+0000","last_peered":"2024-02-18T00:33:14.112843+0000","last_clean":"2024-02-18T00:33:14.112843+0000","last_became_active":"2024-02-18T00:33:13.179070+0000","last_became_peered":"2024-02-18T00:33:13.179070+0000","last_unstale":"2024-02-18T00:33:14.112843+0000","last_undegraded":"2024-02-18T00:33:14.112843+0000","last_fullsized":"2024-02-18T00:33:14.112843+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.15","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109438+0000","last_change":"2024-02-18T00:33:12.150467+0000","last_active":"2024-02-18T00:33:14.109438+0000","last_peered":"2024-02-18T00:33:14.109438+0000","last_clean":"2024-02-18T00:33:14.109438+0000","last_became_active":"2024-02-18T00:33:12.150225+0000","last_became_peered":"2024-02-18T00:33:12.150225+0000","last_unstale":"2024-02-18T00:33:14.109438+0000","last_undegraded":"2024-02-18T00:33:14.109438+0000","last_fullsized":"2024-02-18T00:33:14.109438+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.11","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.099201+0000","last_change":"2024-02-18T00:33:07.471272+0000","last_active":"2024-02-18T00:33:14.099201+0000","last_peered":"2024-02-18T00:33:14.099201+0000","last_clean":"2024-02-18T00:33:14.099201+0000","last_became_active":"2024-02-18T00:33:07.458468+0000","last_became_peered":"2024-02-18T00:33:07.458468+0000","last_unstale":"2024-02-18T00:33:14.099201+0000","last_undegraded":"2024-02-18T00:33:14.099201+0000","last_fullsized":"2024-02-18T00:33:14.099201+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122213+0000","last_change":"2024-02-18T00:33:05.591825+0000","last_active":"2024-02-18T00:33:14.122213+0000","last_peered":"2024-02-18T00:33:14.122213+0000","last_clean":"2024-02-18T00:33:14.122213+0000","last_became_active":"2024-02-18T00:33:05.591547+0000","last_became_peered":"2024-02-18T00:33:05.591547+0000","last_unstale":"2024-02-18T00:33:14.122213+0000","last_undegraded":"2024-02-18T00:33:14.122213+0000","last_fullsized":"2024-02-18T00:33:14.122213+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"61.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"53.3","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141295+0000","last_change":"2024-02-18T00:33:14.137458+0000","last_active":"2024-02-18T00:33:14.141295+0000","last_peered":"2024-02-18T00:33:14.141295+0000","last_clean":"2024-02-18T00:33:14.141295+0000","last_became_active":"2024-02-18T00:33:14.137156+0000","last_became_peered":"2024-02-18T00:33:14.137156+0000","last_unstale":"2024-02-18T00:33:14.141295+0000","last_undegraded":"2024-02-18T00:33:14.141295+0000","last_fullsized":"2024-02-18T00:33:14.141295+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_clean_scrub_stamp":"2024-02-18T00:33:13.082745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132219+0000","last_change":"2024-02-18T00:33:13.161210+0000","last_active":"2024-02-18T00:33:14.132219+0000","last_peered":"2024-02-18T00:33:14.132219+0000","last_clean":"2024-02-18T00:33:14.132219+0000","last_became_active":"2024-02-18T00:33:13.160885+0000","last_became_peered":"2024-02-18T00:33:13.160885+0000","last_unstale":"2024-02-18T00:33:14.132219+0000","last_undegraded":"2024-02-18T00:33:14.132219+0000","last_fullsized":"2024-02-18T00:33:14.132219+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.1a","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.129118+0000","last_change":"2024-02-18T00:33:12.140630+0000","last_active":"2024-02-18T00:33:14.129118+0000","last_peered":"2024-02-18T00:33:14.129118+0000","last_clean":"2024-02-18T00:33:14.129118+0000","last_became_active":"2024-02-18T00:33:12.140288+0000","last_became_peered":"2024-02-18T00:33:12.140288+0000","last_unstale":"2024-02-18T00:33:14.129118+0000","last_undegraded":"2024-02-18T00:33:14.129118+0000","last_fullsized":"2024-02-18T00:33:14.129118+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.1e","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127095+0000","last_change":"2024-02-18T00:33:07.485737+0000","last_active":"2024-02-18T00:33:14.127095+0000","last_peered":"2024-02-18T00:33:14.127095+0000","last_clean":"2024-02-18T00:33:14.127095+0000","last_became_active":"2024-02-18T00:33:07.485435+0000","last_became_peered":"2024-02-18T00:33:07.485435+0000","last_unstale":"2024-02-18T00:33:14.127095+0000","last_undegraded":"2024-02-18T00:33:14.127095+0000","last_fullsized":"2024-02-18T00:33:14.127095+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130051+0000","last_change":"2024-02-18T00:33:05.557565+0000","last_active":"2024-02-18T00:33:14.130051+0000","last_peered":"2024-02-18T00:33:14.130051+0000","last_clean":"2024-02-18T00:33:14.130051+0000","last_became_active":"2024-02-18T00:33:05.557302+0000","last_became_peered":"2024-02-18T00:33:05.557302+0000","last_unstale":"2024-02-18T00:33:14.130051+0000","last_undegraded":"2024-02-18T00:33:14.130051+0000","last_fullsized":"2024-02-18T00:33:14.130051+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"61.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"53.2","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.148777+0000","last_change":"2024-02-18T00:33:14.147506+0000","last_active":"2024-02-18T00:33:14.148777+0000","last_peered":"2024-02-18T00:33:14.148777+0000","last_clean":"2024-02-18T00:33:14.148777+0000","last_became_active":"2024-02-18T00:33:14.147031+0000","last_became_peered":"2024-02-18T00:33:14.147031+0000","last_unstale":"2024-02-18T00:33:14.148777+0000","last_undegraded":"2024-02-18T00:33:14.148777+0000","last_fullsized":"2024-02-18T00:33:14.148777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_clean_scrub_stamp":"2024-02-18T00:33:13.082745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104034+0000","last_change":"2024-02-18T00:33:13.131288+0000","last_active":"2024-02-18T00:33:14.104034+0000","last_peered":"2024-02-18T00:33:14.104034+0000","last_clean":"2024-02-18T00:33:14.104034+0000","last_became_active":"2024-02-18T00:33:13.130912+0000","last_became_peered":"2024-02-18T00:33:13.130912+0000","last_unstale":"2024-02-18T00:33:14.104034+0000","last_undegraded":"2024-02-18T00:33:14.104034+0000","last_fullsized":"2024-02-18T00:33:14.104034+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.1b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.139666+0000","last_change":"2024-02-18T00:33:12.133293+0000","last_active":"2024-02-18T00:33:14.139666+0000","last_peered":"2024-02-18T00:33:14.139666+0000","last_clean":"2024-02-18T00:33:14.139666+0000","last_became_active":"2024-02-18T00:33:12.132996+0000","last_became_peered":"2024-02-18T00:33:12.132996+0000","last_unstale":"2024-02-18T00:33:14.139666+0000","last_undegraded":"2024-02-18T00:33:14.139666+0000","last_fullsized":"2024-02-18T00:33:14.139666+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.1f","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108962+0000","last_change":"2024-02-18T00:33:07.486457+0000","last_active":"2024-02-18T00:33:14.108962+0000","last_peered":"2024-02-18T00:33:14.108962+0000","last_clean":"2024-02-18T00:33:14.108962+0000","last_became_active":"2024-02-18T00:33:07.485940+0000","last_became_peered":"2024-02-18T00:33:07.485940+0000","last_unstale":"2024-02-18T00:33:14.108962+0000","last_undegraded":"2024-02-18T00:33:14.108962+0000","last_fullsized":"2024-02-18T00:33:14.108962+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107327+0000","last_change":"2024-02-18T00:33:05.594525+0000","last_active":"2024-02-18T00:33:14.107327+0000","last_peered":"2024-02-18T00:33:14.107327+0000","last_clean":"2024-02-18T00:33:14.107327+0000","last_became_active":"2024-02-18T00:33:05.594248+0000","last_became_peered":"2024-02-18T00:33:05.594248+0000","last_unstale":"2024-02-18T00:33:14.107327+0000","last_undegraded":"2024-02-18T00:33:14.107327+0000","last_fullsized":"2024-02-18T00:33:14.107327+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"61.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"53.1","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.144078+0000","last_change":"2024-02-18T00:33:14.137560+0000","last_active":"2024-02-18T00:33:14.144078+0000","last_peered":"2024-02-18T00:33:14.144078+0000","last_clean":"2024-02-18T00:33:14.144078+0000","last_became_active":"2024-02-18T00:33:14.133644+0000","last_became_peered":"2024-02-18T00:33:14.133644+0000","last_unstale":"2024-02-18T00:33:14.144078+0000","last_undegraded":"2024-02-18T00:33:14.144078+0000","last_fullsized":"2024-02-18T00:33:14.144078+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_clean_scrub_stamp":"2024-02-18T00:33:13.082745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.7","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113858+0000","last_change":"2024-02-18T00:33:13.152434+0000","last_active":"2024-02-18T00:33:14.113858+0000","last_peered":"2024-02-18T00:33:14.113858+0000","last_clean":"2024-02-18T00:33:14.113858+0000","last_became_active":"2024-02-18T00:33:13.152141+0000","last_became_peered":"2024-02-18T00:33:13.152141+0000","last_unstale":"2024-02-18T00:33:14.113858+0000","last_undegraded":"2024-02-18T00:33:14.113858+0000","last_fullsized":"2024-02-18T00:33:14.113858+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.18","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.119087+0000","last_change":"2024-02-18T00:33:12.146722+0000","last_active":"2024-02-18T00:33:14.119087+0000","last_peered":"2024-02-18T00:33:14.119087+0000","last_clean":"2024-02-18T00:33:14.119087+0000","last_became_active":"2024-02-18T00:33:12.145977+0000","last_became_peered":"2024-02-18T00:33:12.145977+0000","last_unstale":"2024-02-18T00:33:14.119087+0000","last_undegraded":"2024-02-18T00:33:14.119087+0000","last_fullsized":"2024-02-18T00:33:14.119087+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.1c","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104259+0000","last_change":"2024-02-18T00:33:07.486120+0000","last_active":"2024-02-18T00:33:14.104259+0000","last_peered":"2024-02-18T00:33:14.104259+0000","last_clean":"2024-02-18T00:33:14.104259+0000","last_became_active":"2024-02-18T00:33:07.485584+0000","last_became_peered":"2024-02-18T00:33:07.485584+0000","last_unstale":"2024-02-18T00:33:14.104259+0000","last_undegraded":"2024-02-18T00:33:14.104259+0000","last_fullsized":"2024-02-18T00:33:14.104259+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137227+0000","last_change":"2024-02-18T00:33:05.577654+0000","last_active":"2024-02-18T00:33:14.137227+0000","last_peered":"2024-02-18T00:33:14.137227+0000","last_clean":"2024-02-18T00:33:14.137227+0000","last_became_active":"2024-02-18T00:33:05.577374+0000","last_became_peered":"2024-02-18T00:33:05.577374+0000","last_unstale":"2024-02-18T00:33:14.137227+0000","last_undegraded":"2024-02-18T00:33:14.137227+0000","last_fullsized":"2024-02-18T00:33:14.137227+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"61.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"53.0","version":"0'0","reported_seq":10,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.149590+0000","last_change":"2024-02-18T00:33:14.148891+0000","last_active":"2024-02-18T00:33:14.149590+0000","last_peered":"2024-02-18T00:33:14.149590+0000","last_clean":"2024-02-18T00:33:14.149590+0000","last_became_active":"2024-02-18T00:33:14.148598+0000","last_became_peered":"2024-02-18T00:33:14.148598+0000","last_unstale":"2024-02-18T00:33:14.149590+0000","last_undegraded":"2024-02-18T00:33:14.149590+0000","last_fullsized":"2024-02-18T00:33:14.149590+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:13.082745+0000","last_clean_scrub_stamp":"2024-02-18T00:33:13.082745+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.6","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.134376+0000","last_change":"2024-02-18T00:33:13.178541+0000","last_active":"2024-02-18T00:33:14.134376+0000","last_peered":"2024-02-18T00:33:14.134376+0000","last_clean":"2024-02-18T00:33:14.134376+0000","last_became_active":"2024-02-18T00:33:13.178304+0000","last_became_peered":"2024-02-18T00:33:13.178304+0000","last_unstale":"2024-02-18T00:33:14.134376+0000","last_undegraded":"2024-02-18T00:33:14.134376+0000","last_fullsized":"2024-02-18T00:33:14.134376+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.19","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103879+0000","last_change":"2024-02-18T00:33:12.156275+0000","last_active":"2024-02-18T00:33:14.103879+0000","last_peered":"2024-02-18T00:33:14.103879+0000","last_clean":"2024-02-18T00:33:14.103879+0000","last_became_active":"2024-02-18T00:33:12.155963+0000","last_became_peered":"2024-02-18T00:33:12.155963+0000","last_unstale":"2024-02-18T00:33:14.103879+0000","last_undegraded":"2024-02-18T00:33:14.103879+0000","last_fullsized":"2024-02-18T00:33:14.103879+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.1d","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.129439+0000","last_change":"2024-02-18T00:33:07.447387+0000","last_active":"2024-02-18T00:33:14.129439+0000","last_peered":"2024-02-18T00:33:14.129439+0000","last_clean":"2024-02-18T00:33:14.129439+0000","last_became_active":"2024-02-18T00:33:07.447089+0000","last_became_peered":"2024-02-18T00:33:07.447089+0000","last_unstale":"2024-02-18T00:33:14.129439+0000","last_undegraded":"2024-02-18T00:33:14.129439+0000","last_fullsized":"2024-02-18T00:33:14.129439+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.134455+0000","last_change":"2024-02-18T00:33:05.588589+0000","last_active":"2024-02-18T00:33:14.134455+0000","last_peered":"2024-02-18T00:33:14.134455+0000","last_clean":"2024-02-18T00:33:14.134455+0000","last_became_active":"2024-02-18T00:33:05.588290+0000","last_became_peered":"2024-02-18T00:33:05.588290+0000","last_unstale":"2024-02-18T00:33:14.134455+0000","last_undegraded":"2024-02-18T00:33:14.134455+0000","last_fullsized":"2024-02-18T00:33:14.134455+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"61.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.126400+0000","last_change":"2024-02-18T00:33:13.167068+0000","last_active":"2024-02-18T00:33:14.126400+0000","last_peered":"2024-02-18T00:33:14.126400+0000","last_clean":"2024-02-18T00:33:14.126400+0000","last_became_active":"2024-02-18T00:33:13.164988+0000","last_became_peered":"2024-02-18T00:33:13.164988+0000","last_unstale":"2024-02-18T00:33:14.126400+0000","last_undegraded":"2024-02-18T00:33:14.126400+0000","last_fullsized":"2024-02-18T00:33:14.126400+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.1e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.133409+0000","last_change":"2024-02-18T00:33:12.154291+0000","last_active":"2024-02-18T00:33:14.133409+0000","last_peered":"2024-02-18T00:33:14.133409+0000","last_clean":"2024-02-18T00:33:14.133409+0000","last_became_active":"2024-02-18T00:33:12.150516+0000","last_became_peered":"2024-02-18T00:33:12.150516+0000","last_unstale":"2024-02-18T00:33:14.133409+0000","last_undegraded":"2024-02-18T00:33:14.133409+0000","last_fullsized":"2024-02-18T00:33:14.133409+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.1a","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104067+0000","last_change":"2024-02-18T00:33:07.479890+0000","last_active":"2024-02-18T00:33:14.104067+0000","last_peered":"2024-02-18T00:33:14.104067+0000","last_clean":"2024-02-18T00:33:14.104067+0000","last_became_active":"2024-02-18T00:33:07.476802+0000","last_became_peered":"2024-02-18T00:33:07.476802+0000","last_unstale":"2024-02-18T00:33:14.104067+0000","last_undegraded":"2024-02-18T00:33:14.104067+0000","last_fullsized":"2024-02-18T00:33:14.104067+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113827+0000","last_change":"2024-02-18T00:33:05.591422+0000","last_active":"2024-02-18T00:33:14.113827+0000","last_peered":"2024-02-18T00:33:14.113827+0000","last_clean":"2024-02-18T00:33:14.113827+0000","last_became_active":"2024-02-18T00:33:05.591105+0000","last_became_peered":"2024-02-18T00:33:05.591105+0000","last_unstale":"2024-02-18T00:33:14.113827+0000","last_undegraded":"2024-02-18T00:33:14.113827+0000","last_fullsized":"2024-02-18T00:33:14.113827+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"61.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113870+0000","last_change":"2024-02-18T00:33:13.154609+0000","last_active":"2024-02-18T00:33:14.113870+0000","last_peered":"2024-02-18T00:33:14.113870+0000","last_clean":"2024-02-18T00:33:14.113870+0000","last_became_active":"2024-02-18T00:33:13.154324+0000","last_became_peered":"2024-02-18T00:33:13.154324+0000","last_unstale":"2024-02-18T00:33:14.113870+0000","last_undegraded":"2024-02-18T00:33:14.113870+0000","last_fullsized":"2024-02-18T00:33:14.113870+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.1f","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137588+0000","last_change":"2024-02-18T00:33:12.161894+0000","last_active":"2024-02-18T00:33:14.137588+0000","last_peered":"2024-02-18T00:33:14.137588+0000","last_clean":"2024-02-18T00:33:14.137588+0000","last_became_active":"2024-02-18T00:33:12.160406+0000","last_became_peered":"2024-02-18T00:33:12.160406+0000","last_unstale":"2024-02-18T00:33:14.137588+0000","last_undegraded":"2024-02-18T00:33:14.137588+0000","last_fullsized":"2024-02-18T00:33:14.137588+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.1b","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124193+0000","last_change":"2024-02-18T00:33:07.475304+0000","last_active":"2024-02-18T00:33:14.124193+0000","last_peered":"2024-02-18T00:33:14.124193+0000","last_clean":"2024-02-18T00:33:14.124193+0000","last_became_active":"2024-02-18T00:33:07.474989+0000","last_became_peered":"2024-02-18T00:33:07.474989+0000","last_unstale":"2024-02-18T00:33:14.124193+0000","last_undegraded":"2024-02-18T00:33:14.124193+0000","last_fullsized":"2024-02-18T00:33:14.124193+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102516+0000","last_change":"2024-02-18T00:33:05.574365+0000","last_active":"2024-02-18T00:33:14.102516+0000","last_peered":"2024-02-18T00:33:14.102516+0000","last_clean":"2024-02-18T00:33:14.102516+0000","last_became_active":"2024-02-18T00:33:05.574066+0000","last_became_peered":"2024-02-18T00:33:05.574066+0000","last_unstale":"2024-02-18T00:33:14.102516+0000","last_undegraded":"2024-02-18T00:33:14.102516+0000","last_fullsized":"2024-02-18T00:33:14.102516+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"61.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.139399+0000","last_change":"2024-02-18T00:33:13.169706+0000","last_active":"2024-02-18T00:33:14.139399+0000","last_peered":"2024-02-18T00:33:14.139399+0000","last_clean":"2024-02-18T00:33:14.139399+0000","last_became_active":"2024-02-18T00:33:13.169446+0000","last_became_peered":"2024-02-18T00:33:13.169446+0000","last_unstale":"2024-02-18T00:33:14.139399+0000","last_undegraded":"2024-02-18T00:33:14.139399+0000","last_fullsized":"2024-02-18T00:33:14.139399+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.1c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.142253+0000","last_change":"2024-02-18T00:33:12.143886+0000","last_active":"2024-02-18T00:33:14.142253+0000","last_peered":"2024-02-18T00:33:14.142253+0000","last_clean":"2024-02-18T00:33:14.142253+0000","last_became_active":"2024-02-18T00:33:12.143472+0000","last_became_peered":"2024-02-18T00:33:12.143472+0000","last_unstale":"2024-02-18T00:33:14.142253+0000","last_undegraded":"2024-02-18T00:33:14.142253+0000","last_fullsized":"2024-02-18T00:33:14.142253+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.18","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.101062+0000","last_change":"2024-02-18T00:33:07.482819+0000","last_active":"2024-02-18T00:33:14.101062+0000","last_peered":"2024-02-18T00:33:14.101062+0000","last_clean":"2024-02-18T00:33:14.101062+0000","last_became_active":"2024-02-18T00:33:07.482537+0000","last_became_peered":"2024-02-18T00:33:07.482537+0000","last_unstale":"2024-02-18T00:33:14.101062+0000","last_undegraded":"2024-02-18T00:33:14.101062+0000","last_fullsized":"2024-02-18T00:33:14.101062+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117031+0000","last_change":"2024-02-18T00:33:05.591088+0000","last_active":"2024-02-18T00:33:14.117031+0000","last_peered":"2024-02-18T00:33:14.117031+0000","last_clean":"2024-02-18T00:33:14.117031+0000","last_became_active":"2024-02-18T00:33:05.590791+0000","last_became_peered":"2024-02-18T00:33:05.590791+0000","last_unstale":"2024-02-18T00:33:14.117031+0000","last_undegraded":"2024-02-18T00:33:14.117031+0000","last_fullsized":"2024-02-18T00:33:14.117031+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"61.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.116434+0000","last_change":"2024-02-18T00:33:13.167303+0000","last_active":"2024-02-18T00:33:14.116434+0000","last_peered":"2024-02-18T00:33:14.116434+0000","last_clean":"2024-02-18T00:33:14.116434+0000","last_became_active":"2024-02-18T00:33:13.167010+0000","last_became_peered":"2024-02-18T00:33:13.167010+0000","last_unstale":"2024-02-18T00:33:14.116434+0000","last_undegraded":"2024-02-18T00:33:14.116434+0000","last_fullsized":"2024-02-18T00:33:14.116434+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.1d","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.105075+0000","last_change":"2024-02-18T00:33:12.167340+0000","last_active":"2024-02-18T00:33:14.105075+0000","last_peered":"2024-02-18T00:33:14.105075+0000","last_clean":"2024-02-18T00:33:14.105075+0000","last_became_active":"2024-02-18T00:33:12.166575+0000","last_became_peered":"2024-02-18T00:33:12.166575+0000","last_unstale":"2024-02-18T00:33:14.105075+0000","last_undegraded":"2024-02-18T00:33:14.105075+0000","last_fullsized":"2024-02-18T00:33:14.105075+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.19","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.110324+0000","last_change":"2024-02-18T00:33:07.477953+0000","last_active":"2024-02-18T00:33:14.110324+0000","last_peered":"2024-02-18T00:33:14.110324+0000","last_clean":"2024-02-18T00:33:14.110324+0000","last_became_active":"2024-02-18T00:33:07.477671+0000","last_became_peered":"2024-02-18T00:33:07.477671+0000","last_unstale":"2024-02-18T00:33:14.110324+0000","last_undegraded":"2024-02-18T00:33:14.110324+0000","last_fullsized":"2024-02-18T00:33:14.110324+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.119900+0000","last_change":"2024-02-18T00:33:05.595242+0000","last_active":"2024-02-18T00:33:14.119900+0000","last_peered":"2024-02-18T00:33:14.119900+0000","last_clean":"2024-02-18T00:33:14.119900+0000","last_became_active":"2024-02-18T00:33:05.594948+0000","last_became_peered":"2024-02-18T00:33:05.594948+0000","last_unstale":"2024-02-18T00:33:14.119900+0000","last_undegraded":"2024-02-18T00:33:14.119900+0000","last_fullsized":"2024-02-18T00:33:14.119900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"61.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.1d","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.126846+0000","last_change":"2024-02-18T00:33:13.171793+0000","last_active":"2024-02-18T00:33:14.126846+0000","last_peered":"2024-02-18T00:33:14.126846+0000","last_clean":"2024-02-18T00:33:14.126846+0000","last_became_active":"2024-02-18T00:33:13.171322+0000","last_became_peered":"2024-02-18T00:33:13.171322+0000","last_unstale":"2024-02-18T00:33:14.126846+0000","last_undegraded":"2024-02-18T00:33:14.126846+0000","last_fullsized":"2024-02-18T00:33:14.126846+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.2","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121871+0000","last_change":"2024-02-18T00:33:12.128241+0000","last_active":"2024-02-18T00:33:14.121871+0000","last_peered":"2024-02-18T00:33:14.121871+0000","last_clean":"2024-02-18T00:33:14.121871+0000","last_became_active":"2024-02-18T00:33:12.127912+0000","last_became_peered":"2024-02-18T00:33:12.127912+0000","last_unstale":"2024-02-18T00:33:14.121871+0000","last_undegraded":"2024-02-18T00:33:14.121871+0000","last_fullsized":"2024-02-18T00:33:14.121871+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.6","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138417+0000","last_change":"2024-02-18T00:33:07.472969+0000","last_active":"2024-02-18T00:33:14.138417+0000","last_peered":"2024-02-18T00:33:14.138417+0000","last_clean":"2024-02-18T00:33:14.138417+0000","last_became_active":"2024-02-18T00:33:07.469196+0000","last_became_peered":"2024-02-18T00:33:07.469196+0000","last_unstale":"2024-02-18T00:33:14.138417+0000","last_undegraded":"2024-02-18T00:33:14.138417+0000","last_fullsized":"2024-02-18T00:33:14.138417+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.136632+0000","last_change":"2024-02-18T00:33:05.581799+0000","last_active":"2024-02-18T00:33:14.136632+0000","last_peered":"2024-02-18T00:33:14.136632+0000","last_clean":"2024-02-18T00:33:14.136632+0000","last_became_active":"2024-02-18T00:33:05.581404+0000","last_became_peered":"2024-02-18T00:33:05.581404+0000","last_unstale":"2024-02-18T00:33:14.136632+0000","last_undegraded":"2024-02-18T00:33:14.136632+0000","last_fullsized":"2024-02-18T00:33:14.136632+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"61.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.1c","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107175+0000","last_change":"2024-02-18T00:33:13.182690+0000","last_active":"2024-02-18T00:33:14.107175+0000","last_peered":"2024-02-18T00:33:14.107175+0000","last_clean":"2024-02-18T00:33:14.107175+0000","last_became_active":"2024-02-18T00:33:13.182395+0000","last_became_peered":"2024-02-18T00:33:13.182395+0000","last_unstale":"2024-02-18T00:33:14.107175+0000","last_undegraded":"2024-02-18T00:33:14.107175+0000","last_fullsized":"2024-02-18T00:33:14.107175+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.3","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.100667+0000","last_change":"2024-02-18T00:33:12.158708+0000","last_active":"2024-02-18T00:33:14.100667+0000","last_peered":"2024-02-18T00:33:14.100667+0000","last_clean":"2024-02-18T00:33:14.100667+0000","last_became_active":"2024-02-18T00:33:12.158419+0000","last_became_peered":"2024-02-18T00:33:12.158419+0000","last_unstale":"2024-02-18T00:33:14.100667+0000","last_undegraded":"2024-02-18T00:33:14.100667+0000","last_fullsized":"2024-02-18T00:33:14.100667+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"40.7","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.147245+0000","last_change":"2024-02-18T00:33:07.506906+0000","last_active":"2024-02-18T00:33:14.147245+0000","last_peered":"2024-02-18T00:33:14.147245+0000","last_clean":"2024-02-18T00:33:14.147245+0000","last_became_active":"2024-02-18T00:33:07.506634+0000","last_became_peered":"2024-02-18T00:33:07.506634+0000","last_unstale":"2024-02-18T00:33:14.147245+0000","last_undegraded":"2024-02-18T00:33:14.147245+0000","last_fullsized":"2024-02-18T00:33:14.147245+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.133384+0000","last_change":"2024-02-18T00:33:05.578309+0000","last_active":"2024-02-18T00:33:14.133384+0000","last_peered":"2024-02-18T00:33:14.133384+0000","last_clean":"2024-02-18T00:33:14.133384+0000","last_became_active":"2024-02-18T00:33:05.577884+0000","last_became_peered":"2024-02-18T00:33:05.577884+0000","last_unstale":"2024-02-18T00:33:14.133384+0000","last_undegraded":"2024-02-18T00:33:14.133384+0000","last_fullsized":"2024-02-18T00:33:14.133384+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"61.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.1f","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.105692+0000","last_change":"2024-02-18T00:33:13.175268+0000","last_active":"2024-02-18T00:33:14.105692+0000","last_peered":"2024-02-18T00:33:14.105692+0000","last_clean":"2024-02-18T00:33:14.105692+0000","last_became_active":"2024-02-18T00:33:13.174941+0000","last_became_peered":"2024-02-18T00:33:13.174941+0000","last_unstale":"2024-02-18T00:33:14.105692+0000","last_undegraded":"2024-02-18T00:33:14.105692+0000","last_fullsized":"2024-02-18T00:33:14.105692+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.0","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127314+0000","last_change":"2024-02-18T00:33:12.130947+0000","last_active":"2024-02-18T00:33:14.127314+0000","last_peered":"2024-02-18T00:33:14.127314+0000","last_clean":"2024-02-18T00:33:14.127314+0000","last_became_active":"2024-02-18T00:33:12.130628+0000","last_became_peered":"2024-02-18T00:33:12.130628+0000","last_unstale":"2024-02-18T00:33:14.127314+0000","last_undegraded":"2024-02-18T00:33:14.127314+0000","last_fullsized":"2024-02-18T00:33:14.127314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.4","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127543+0000","last_change":"2024-02-18T00:33:07.488287+0000","last_active":"2024-02-18T00:33:14.127543+0000","last_peered":"2024-02-18T00:33:14.127543+0000","last_clean":"2024-02-18T00:33:14.127543+0000","last_became_active":"2024-02-18T00:33:07.487634+0000","last_became_peered":"2024-02-18T00:33:07.487634+0000","last_unstale":"2024-02-18T00:33:14.127543+0000","last_undegraded":"2024-02-18T00:33:14.127543+0000","last_fullsized":"2024-02-18T00:33:14.127543+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.131478+0000","last_change":"2024-02-18T00:33:05.573140+0000","last_active":"2024-02-18T00:33:14.131478+0000","last_peered":"2024-02-18T00:33:14.131478+0000","last_clean":"2024-02-18T00:33:14.131478+0000","last_became_active":"2024-02-18T00:33:05.572784+0000","last_became_peered":"2024-02-18T00:33:05.572784+0000","last_unstale":"2024-02-18T00:33:14.131478+0000","last_undegraded":"2024-02-18T00:33:14.131478+0000","last_fullsized":"2024-02-18T00:33:14.131478+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"61.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.1e","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.105580+0000","last_change":"2024-02-18T00:33:13.164463+0000","last_active":"2024-02-18T00:33:14.105580+0000","last_peered":"2024-02-18T00:33:14.105580+0000","last_clean":"2024-02-18T00:33:14.105580+0000","last_became_active":"2024-02-18T00:33:13.164131+0000","last_became_peered":"2024-02-18T00:33:13.164131+0000","last_unstale":"2024-02-18T00:33:14.105580+0000","last_undegraded":"2024-02-18T00:33:14.105580+0000","last_fullsized":"2024-02-18T00:33:14.105580+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.1","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121587+0000","last_change":"2024-02-18T00:33:12.134904+0000","last_active":"2024-02-18T00:33:14.121587+0000","last_peered":"2024-02-18T00:33:14.121587+0000","last_clean":"2024-02-18T00:33:14.121587+0000","last_became_active":"2024-02-18T00:33:12.134092+0000","last_became_peered":"2024-02-18T00:33:12.134092+0000","last_unstale":"2024-02-18T00:33:14.121587+0000","last_undegraded":"2024-02-18T00:33:14.121587+0000","last_fullsized":"2024-02-18T00:33:14.121587+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.5","version":"118'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.130435+0000","last_change":"2024-02-18T00:33:07.478129+0000","last_active":"2024-02-18T00:33:15.130435+0000","last_peered":"2024-02-18T00:33:15.130435+0000","last_clean":"2024-02-18T00:33:15.130435+0000","last_became_active":"2024-02-18T00:33:07.477861+0000","last_became_peered":"2024-02-18T00:33:07.477861+0000","last_unstale":"2024-02-18T00:33:15.130435+0000","last_undegraded":"2024-02-18T00:33:15.130435+0000","last_fullsized":"2024-02-18T00:33:15.130435+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8192,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132120+0000","last_change":"2024-02-18T00:33:05.601719+0000","last_active":"2024-02-18T00:33:14.132120+0000","last_peered":"2024-02-18T00:33:14.132120+0000","last_clean":"2024-02-18T00:33:14.132120+0000","last_became_active":"2024-02-18T00:33:05.601456+0000","last_became_peered":"2024-02-18T00:33:05.601456+0000","last_unstale":"2024-02-18T00:33:14.132120+0000","last_undegraded":"2024-02-18T00:33:14.132120+0000","last_fullsized":"2024-02-18T00:33:14.132120+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"61.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.1b","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114047+0000","last_change":"2024-02-18T00:33:13.179350+0000","last_active":"2024-02-18T00:33:14.114047+0000","last_peered":"2024-02-18T00:33:14.114047+0000","last_clean":"2024-02-18T00:33:14.114047+0000","last_became_active":"2024-02-18T00:33:13.179052+0000","last_became_peered":"2024-02-18T00:33:13.179052+0000","last_unstale":"2024-02-18T00:33:14.114047+0000","last_undegraded":"2024-02-18T00:33:14.114047+0000","last_fullsized":"2024-02-18T00:33:14.114047+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"44.4","version":"115'3","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104266+0000","last_change":"2024-02-18T00:33:12.156419+0000","last_active":"2024-02-18T00:33:14.104266+0000","last_peered":"2024-02-18T00:33:14.104266+0000","last_clean":"2024-02-18T00:33:14.104266+0000","last_became_active":"2024-02-18T00:33:12.156112+0000","last_became_peered":"2024-02-18T00:33:12.156112+0000","last_unstale":"2024-02-18T00:33:14.104266+0000","last_undegraded":"2024-02-18T00:33:14.104266+0000","last_fullsized":"2024-02-18T00:33:14.104266+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.0","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113678+0000","last_change":"2024-02-18T00:33:07.477593+0000","last_active":"2024-02-18T00:33:14.113678+0000","last_peered":"2024-02-18T00:33:14.113678+0000","last_clean":"2024-02-18T00:33:14.113678+0000","last_became_active":"2024-02-18T00:33:07.477234+0000","last_became_peered":"2024-02-18T00:33:07.477234+0000","last_unstale":"2024-02-18T00:33:14.113678+0000","last_undegraded":"2024-02-18T00:33:14.113678+0000","last_fullsized":"2024-02-18T00:33:14.113678+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,6],"acting":[1,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130731+0000","last_change":"2024-02-18T00:33:05.591844+0000","last_active":"2024-02-18T00:33:14.130731+0000","last_peered":"2024-02-18T00:33:14.130731+0000","last_clean":"2024-02-18T00:33:14.130731+0000","last_became_active":"2024-02-18T00:33:05.591564+0000","last_became_peered":"2024-02-18T00:33:05.591564+0000","last_unstale":"2024-02-18T00:33:14.130731+0000","last_undegraded":"2024-02-18T00:33:14.130731+0000","last_fullsized":"2024-02-18T00:33:14.130731+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"61.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.1a","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104159+0000","last_change":"2024-02-18T00:33:13.164050+0000","last_active":"2024-02-18T00:33:14.104159+0000","last_peered":"2024-02-18T00:33:14.104159+0000","last_clean":"2024-02-18T00:33:14.104159+0000","last_became_active":"2024-02-18T00:33:13.163452+0000","last_became_peered":"2024-02-18T00:33:13.163452+0000","last_unstale":"2024-02-18T00:33:14.104159+0000","last_undegraded":"2024-02-18T00:33:14.104159+0000","last_fullsized":"2024-02-18T00:33:14.104159+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.5","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140655+0000","last_change":"2024-02-18T00:33:12.145377+0000","last_active":"2024-02-18T00:33:14.140655+0000","last_peered":"2024-02-18T00:33:14.140655+0000","last_clean":"2024-02-18T00:33:14.140655+0000","last_became_active":"2024-02-18T00:33:12.145097+0000","last_became_peered":"2024-02-18T00:33:12.145097+0000","last_unstale":"2024-02-18T00:33:14.140655+0000","last_undegraded":"2024-02-18T00:33:14.140655+0000","last_fullsized":"2024-02-18T00:33:14.140655+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.1","version":"116'1","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117368+0000","last_change":"2024-02-18T00:33:07.473777+0000","last_active":"2024-02-18T00:33:14.117368+0000","last_peered":"2024-02-18T00:33:14.117368+0000","last_clean":"2024-02-18T00:33:14.117368+0000","last_became_active":"2024-02-18T00:33:07.473623+0000","last_became_peered":"2024-02-18T00:33:07.473623+0000","last_unstale":"2024-02-18T00:33:14.117368+0000","last_undegraded":"2024-02-18T00:33:14.117368+0000","last_fullsized":"2024-02-18T00:33:14.117368+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4096,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109738+0000","last_change":"2024-02-18T00:33:05.600449+0000","last_active":"2024-02-18T00:33:14.109738+0000","last_peered":"2024-02-18T00:33:14.109738+0000","last_clean":"2024-02-18T00:33:14.109738+0000","last_became_active":"2024-02-18T00:33:05.600082+0000","last_became_peered":"2024-02-18T00:33:05.600082+0000","last_unstale":"2024-02-18T00:33:14.109738+0000","last_undegraded":"2024-02-18T00:33:14.109738+0000","last_fullsized":"2024-02-18T00:33:14.109738+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"61.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127495+0000","last_change":"2024-02-18T00:33:13.133830+0000","last_active":"2024-02-18T00:33:14.127495+0000","last_peered":"2024-02-18T00:33:14.127495+0000","last_clean":"2024-02-18T00:33:14.127495+0000","last_became_active":"2024-02-18T00:33:13.133132+0000","last_became_peered":"2024-02-18T00:33:13.133132+0000","last_unstale":"2024-02-18T00:33:14.127495+0000","last_undegraded":"2024-02-18T00:33:14.127495+0000","last_fullsized":"2024-02-18T00:33:14.127495+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.7","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130845+0000","last_change":"2024-02-18T00:33:12.136423+0000","last_active":"2024-02-18T00:33:14.130845+0000","last_peered":"2024-02-18T00:33:14.130845+0000","last_clean":"2024-02-18T00:33:14.130845+0000","last_became_active":"2024-02-18T00:33:12.136049+0000","last_became_peered":"2024-02-18T00:33:12.136049+0000","last_unstale":"2024-02-18T00:33:14.130845+0000","last_undegraded":"2024-02-18T00:33:14.130845+0000","last_fullsized":"2024-02-18T00:33:14.130845+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.3","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.120957+0000","last_change":"2024-02-18T00:33:07.498957+0000","last_active":"2024-02-18T00:33:14.120957+0000","last_peered":"2024-02-18T00:33:14.120957+0000","last_clean":"2024-02-18T00:33:14.120957+0000","last_became_active":"2024-02-18T00:33:07.498560+0000","last_became_peered":"2024-02-18T00:33:07.498560+0000","last_unstale":"2024-02-18T00:33:14.120957+0000","last_undegraded":"2024-02-18T00:33:14.120957+0000","last_fullsized":"2024-02-18T00:33:14.120957+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114902+0000","last_change":"2024-02-18T00:33:05.595547+0000","last_active":"2024-02-18T00:33:14.114902+0000","last_peered":"2024-02-18T00:33:14.114902+0000","last_clean":"2024-02-18T00:33:14.114902+0000","last_became_active":"2024-02-18T00:33:05.595374+0000","last_became_peered":"2024-02-18T00:33:05.595374+0000","last_unstale":"2024-02-18T00:33:14.114902+0000","last_undegraded":"2024-02-18T00:33:14.114902+0000","last_fullsized":"2024-02-18T00:33:14.114902+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113681+0000","last_change":"2024-02-18T00:33:05.591442+0000","last_active":"2024-02-18T00:33:14.113681+0000","last_peered":"2024-02-18T00:33:14.113681+0000","last_clean":"2024-02-18T00:33:14.113681+0000","last_became_active":"2024-02-18T00:33:05.591101+0000","last_became_peered":"2024-02-18T00:33:05.591101+0000","last_unstale":"2024-02-18T00:33:14.113681+0000","last_undegraded":"2024-02-18T00:33:14.113681+0000","last_fullsized":"2024-02-18T00:33:14.113681+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.960661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.960661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"61.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117466+0000","last_change":"2024-02-18T00:33:13.164719+0000","last_active":"2024-02-18T00:33:14.117466+0000","last_peered":"2024-02-18T00:33:14.117466+0000","last_clean":"2024-02-18T00:33:14.117466+0000","last_became_active":"2024-02-18T00:33:13.164376+0000","last_became_peered":"2024-02-18T00:33:13.164376+0000","last_unstale":"2024-02-18T00:33:14.117466+0000","last_undegraded":"2024-02-18T00:33:14.117466+0000","last_fullsized":"2024-02-18T00:33:14.117466+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"44.6","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.133995+0000","last_change":"2024-02-18T00:33:12.154467+0000","last_active":"2024-02-18T00:33:14.133995+0000","last_peered":"2024-02-18T00:33:14.133995+0000","last_clean":"2024-02-18T00:33:14.133995+0000","last_became_active":"2024-02-18T00:33:12.150926+0000","last_became_peered":"2024-02-18T00:33:12.150926+0000","last_unstale":"2024-02-18T00:33:14.133995+0000","last_undegraded":"2024-02-18T00:33:14.133995+0000","last_fullsized":"2024-02-18T00:33:14.133995+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"40.2","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130014+0000","last_change":"2024-02-18T00:33:07.519130+0000","last_active":"2024-02-18T00:33:14.130014+0000","last_peered":"2024-02-18T00:33:14.130014+0000","last_clean":"2024-02-18T00:33:14.130014+0000","last_became_active":"2024-02-18T00:33:07.515941+0000","last_became_peered":"2024-02-18T00:33:07.515941+0000","last_unstale":"2024-02-18T00:33:14.130014+0000","last_undegraded":"2024-02-18T00:33:14.130014+0000","last_fullsized":"2024-02-18T00:33:14.130014+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108363+0000","last_change":"2024-02-18T00:33:05.585449+0000","last_active":"2024-02-18T00:33:14.108363+0000","last_peered":"2024-02-18T00:33:14.108363+0000","last_clean":"2024-02-18T00:33:14.108363+0000","last_became_active":"2024-02-18T00:33:05.585192+0000","last_became_peered":"2024-02-18T00:33:05.585192+0000","last_unstale":"2024-02-18T00:33:14.108363+0000","last_undegraded":"2024-02-18T00:33:14.108363+0000","last_fullsized":"2024-02-18T00:33:14.108363+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117410+0000","last_change":"2024-02-18T00:33:05.558071+0000","last_active":"2024-02-18T00:33:14.117410+0000","last_peered":"2024-02-18T00:33:14.117410+0000","last_clean":"2024-02-18T00:33:14.117410+0000","last_became_active":"2024-02-18T00:33:05.557534+0000","last_became_peered":"2024-02-18T00:33:05.557534+0000","last_unstale":"2024-02-18T00:33:14.117410+0000","last_undegraded":"2024-02-18T00:33:14.117410+0000","last_fullsized":"2024-02-18T00:33:14.117410+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:48.594640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:48.594640+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"61.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.16","version":"116'3","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137875+0000","last_change":"2024-02-18T00:33:13.133297+0000","last_active":"2024-02-18T00:33:14.137875+0000","last_peered":"2024-02-18T00:33:14.137875+0000","last_clean":"2024-02-18T00:33:14.137875+0000","last_became_active":"2024-02-18T00:33:13.132997+0000","last_became_peered":"2024-02-18T00:33:13.132997+0000","last_unstale":"2024-02-18T00:33:14.137875+0000","last_undegraded":"2024-02-18T00:33:14.137875+0000","last_fullsized":"2024-02-18T00:33:14.137875+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":512,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.9","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.126090+0000","last_change":"2024-02-18T00:33:12.113161+0000","last_active":"2024-02-18T00:33:14.126090+0000","last_peered":"2024-02-18T00:33:14.126090+0000","last_clean":"2024-02-18T00:33:14.126090+0000","last_became_active":"2024-02-18T00:33:12.111335+0000","last_became_peered":"2024-02-18T00:33:12.111335+0000","last_unstale":"2024-02-18T00:33:14.126090+0000","last_undegraded":"2024-02-18T00:33:14.126090+0000","last_fullsized":"2024-02-18T00:33:14.126090+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"40.d","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.116671+0000","last_change":"2024-02-18T00:33:07.524137+0000","last_active":"2024-02-18T00:33:14.116671+0000","last_peered":"2024-02-18T00:33:14.116671+0000","last_clean":"2024-02-18T00:33:14.116671+0000","last_became_active":"2024-02-18T00:33:07.522168+0000","last_became_peered":"2024-02-18T00:33:07.522168+0000","last_unstale":"2024-02-18T00:33:14.116671+0000","last_undegraded":"2024-02-18T00:33:14.116671+0000","last_fullsized":"2024-02-18T00:33:14.116671+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127488+0000","last_change":"2024-02-18T00:33:05.584077+0000","last_active":"2024-02-18T00:33:14.127488+0000","last_peered":"2024-02-18T00:33:14.127488+0000","last_clean":"2024-02-18T00:33:14.127488+0000","last_became_active":"2024-02-18T00:33:05.583775+0000","last_became_peered":"2024-02-18T00:33:05.583775+0000","last_unstale":"2024-02-18T00:33:14.127488+0000","last_undegraded":"2024-02-18T00:33:14.127488+0000","last_fullsized":"2024-02-18T00:33:14.127488+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.112499+0000","last_change":"2024-02-18T00:33:05.577170+0000","last_active":"2024-02-18T00:33:14.112499+0000","last_peered":"2024-02-18T00:33:14.112499+0000","last_clean":"2024-02-18T00:33:14.112499+0000","last_became_active":"2024-02-18T00:33:05.576817+0000","last_became_peered":"2024-02-18T00:33:05.576817+0000","last_unstale":"2024-02-18T00:33:14.112499+0000","last_undegraded":"2024-02-18T00:33:14.112499+0000","last_fullsized":"2024-02-18T00:33:14.112499+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.198542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.198542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"61.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130697+0000","last_change":"2024-02-18T00:33:13.179504+0000","last_active":"2024-02-18T00:33:14.130697+0000","last_peered":"2024-02-18T00:33:14.130697+0000","last_clean":"2024-02-18T00:33:14.130697+0000","last_became_active":"2024-02-18T00:33:13.176379+0000","last_became_peered":"2024-02-18T00:33:13.176379+0000","last_unstale":"2024-02-18T00:33:14.130697+0000","last_undegraded":"2024-02-18T00:33:14.130697+0000","last_fullsized":"2024-02-18T00:33:14.130697+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.8","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.119179+0000","last_change":"2024-02-18T00:33:12.129687+0000","last_active":"2024-02-18T00:33:14.119179+0000","last_peered":"2024-02-18T00:33:14.119179+0000","last_clean":"2024-02-18T00:33:14.119179+0000","last_became_active":"2024-02-18T00:33:12.129288+0000","last_became_peered":"2024-02-18T00:33:12.129288+0000","last_unstale":"2024-02-18T00:33:14.119179+0000","last_undegraded":"2024-02-18T00:33:14.119179+0000","last_fullsized":"2024-02-18T00:33:14.119179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.c","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.133177+0000","last_change":"2024-02-18T00:33:07.497200+0000","last_active":"2024-02-18T00:33:14.133177+0000","last_peered":"2024-02-18T00:33:14.133177+0000","last_clean":"2024-02-18T00:33:14.133177+0000","last_became_active":"2024-02-18T00:33:07.496934+0000","last_became_peered":"2024-02-18T00:33:07.496934+0000","last_unstale":"2024-02-18T00:33:14.133177+0000","last_undegraded":"2024-02-18T00:33:14.133177+0000","last_fullsized":"2024-02-18T00:33:14.133177+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114825+0000","last_change":"2024-02-18T00:33:05.588070+0000","last_active":"2024-02-18T00:33:14.114825+0000","last_peered":"2024-02-18T00:33:14.114825+0000","last_clean":"2024-02-18T00:33:14.114825+0000","last_became_active":"2024-02-18T00:33:05.587793+0000","last_became_peered":"2024-02-18T00:33:05.587793+0000","last_unstale":"2024-02-18T00:33:14.114825+0000","last_undegraded":"2024-02-18T00:33:14.114825+0000","last_fullsized":"2024-02-18T00:33:14.114825+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107750+0000","last_change":"2024-02-18T00:33:05.575605+0000","last_active":"2024-02-18T00:33:14.107750+0000","last_peered":"2024-02-18T00:33:14.107750+0000","last_clean":"2024-02-18T00:33:14.107750+0000","last_became_active":"2024-02-18T00:33:05.575295+0000","last_became_peered":"2024-02-18T00:33:05.575295+0000","last_unstale":"2024-02-18T00:33:14.107750+0000","last_undegraded":"2024-02-18T00:33:14.107750+0000","last_fullsized":"2024-02-18T00:33:14.107750+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"61.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.14","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132955+0000","last_change":"2024-02-18T00:33:13.175920+0000","last_active":"2024-02-18T00:33:14.132955+0000","last_peered":"2024-02-18T00:33:14.132955+0000","last_clean":"2024-02-18T00:33:14.132955+0000","last_became_active":"2024-02-18T00:33:13.166627+0000","last_became_peered":"2024-02-18T00:33:13.166627+0000","last_unstale":"2024-02-18T00:33:14.132955+0000","last_undegraded":"2024-02-18T00:33:14.132955+0000","last_fullsized":"2024-02-18T00:33:14.132955+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"44.b","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.123419+0000","last_change":"2024-02-18T00:33:12.149634+0000","last_active":"2024-02-18T00:33:14.123419+0000","last_peered":"2024-02-18T00:33:14.123419+0000","last_clean":"2024-02-18T00:33:14.123419+0000","last_became_active":"2024-02-18T00:33:12.149305+0000","last_became_peered":"2024-02-18T00:33:12.149305+0000","last_unstale":"2024-02-18T00:33:14.123419+0000","last_undegraded":"2024-02-18T00:33:14.123419+0000","last_fullsized":"2024-02-18T00:33:14.123419+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.f","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.133635+0000","last_change":"2024-02-18T00:33:07.489032+0000","last_active":"2024-02-18T00:33:14.133635+0000","last_peered":"2024-02-18T00:33:14.133635+0000","last_clean":"2024-02-18T00:33:14.133635+0000","last_became_active":"2024-02-18T00:33:07.488810+0000","last_became_peered":"2024-02-18T00:33:07.488810+0000","last_unstale":"2024-02-18T00:33:14.133635+0000","last_undegraded":"2024-02-18T00:33:14.133635+0000","last_fullsized":"2024-02-18T00:33:14.133635+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127280+0000","last_change":"2024-02-18T00:33:05.577505+0000","last_active":"2024-02-18T00:33:14.127280+0000","last_peered":"2024-02-18T00:33:14.127280+0000","last_clean":"2024-02-18T00:33:14.127280+0000","last_became_active":"2024-02-18T00:33:05.577209+0000","last_became_peered":"2024-02-18T00:33:05.577209+0000","last_unstale":"2024-02-18T00:33:14.127280+0000","last_undegraded":"2024-02-18T00:33:14.127280+0000","last_fullsized":"2024-02-18T00:33:14.127280+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140302+0000","last_change":"2024-02-18T00:33:05.668879+0000","last_active":"2024-02-18T00:33:14.140302+0000","last_peered":"2024-02-18T00:33:14.140302+0000","last_clean":"2024-02-18T00:33:14.140302+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:33:14.140302+0000","last_undegraded":"2024-02-18T00:33:14.140302+0000","last_fullsized":"2024-02-18T00:33:14.140302+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:32:51.632894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:51.632894+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":437,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"61.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.15","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109048+0000","last_change":"2024-02-18T00:33:13.172696+0000","last_active":"2024-02-18T00:33:14.109048+0000","last_peered":"2024-02-18T00:33:14.109048+0000","last_clean":"2024-02-18T00:33:14.109048+0000","last_became_active":"2024-02-18T00:33:13.172420+0000","last_became_peered":"2024-02-18T00:33:13.172420+0000","last_unstale":"2024-02-18T00:33:14.109048+0000","last_undegraded":"2024-02-18T00:33:14.109048+0000","last_fullsized":"2024-02-18T00:33:14.109048+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"44.a","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103896+0000","last_change":"2024-02-18T00:33:12.172426+0000","last_active":"2024-02-18T00:33:14.103896+0000","last_peered":"2024-02-18T00:33:14.103896+0000","last_clean":"2024-02-18T00:33:14.103896+0000","last_became_active":"2024-02-18T00:33:12.172145+0000","last_became_peered":"2024-02-18T00:33:12.172145+0000","last_unstale":"2024-02-18T00:33:14.103896+0000","last_undegraded":"2024-02-18T00:33:14.103896+0000","last_fullsized":"2024-02-18T00:33:14.103896+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"40.e","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137964+0000","last_change":"2024-02-18T00:33:07.486592+0000","last_active":"2024-02-18T00:33:14.137964+0000","last_peered":"2024-02-18T00:33:14.137964+0000","last_clean":"2024-02-18T00:33:14.137964+0000","last_became_active":"2024-02-18T00:33:07.486294+0000","last_became_peered":"2024-02-18T00:33:07.486294+0000","last_unstale":"2024-02-18T00:33:14.137964+0000","last_undegraded":"2024-02-18T00:33:14.137964+0000","last_fullsized":"2024-02-18T00:33:14.137964+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.136708+0000","last_change":"2024-02-18T00:33:05.582047+0000","last_active":"2024-02-18T00:33:14.136708+0000","last_peered":"2024-02-18T00:33:14.136708+0000","last_clean":"2024-02-18T00:33:14.136708+0000","last_became_active":"2024-02-18T00:33:05.581707+0000","last_became_peered":"2024-02-18T00:33:05.581707+0000","last_unstale":"2024-02-18T00:33:14.136708+0000","last_undegraded":"2024-02-18T00:33:14.136708+0000","last_fullsized":"2024-02-18T00:33:14.136708+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127203+0000","last_change":"2024-02-18T00:33:05.593083+0000","last_active":"2024-02-18T00:33:14.127203+0000","last_peered":"2024-02-18T00:33:14.127203+0000","last_clean":"2024-02-18T00:33:14.127203+0000","last_became_active":"2024-02-18T00:33:05.592776+0000","last_became_peered":"2024-02-18T00:33:05.592776+0000","last_unstale":"2024-02-18T00:33:14.127203+0000","last_undegraded":"2024-02-18T00:33:14.127203+0000","last_fullsized":"2024-02-18T00:33:14.127203+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.839018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.839018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"61.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.12","version":"116'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118012+0000","last_change":"2024-02-18T00:33:13.166252+0000","last_active":"2024-02-18T00:33:14.118012+0000","last_peered":"2024-02-18T00:33:14.118012+0000","last_clean":"2024-02-18T00:33:14.118012+0000","last_became_active":"2024-02-18T00:33:13.166017+0000","last_became_peered":"2024-02-18T00:33:13.166017+0000","last_unstale":"2024-02-18T00:33:14.118012+0000","last_undegraded":"2024-02-18T00:33:14.118012+0000","last_fullsized":"2024-02-18T00:33:14.118012+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"44.d","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124434+0000","last_change":"2024-02-18T00:33:12.122922+0000","last_active":"2024-02-18T00:33:14.124434+0000","last_peered":"2024-02-18T00:33:14.124434+0000","last_clean":"2024-02-18T00:33:14.124434+0000","last_became_active":"2024-02-18T00:33:12.122579+0000","last_became_peered":"2024-02-18T00:33:12.122579+0000","last_unstale":"2024-02-18T00:33:14.124434+0000","last_undegraded":"2024-02-18T00:33:14.124434+0000","last_fullsized":"2024-02-18T00:33:14.124434+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"40.9","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121824+0000","last_change":"2024-02-18T00:33:07.441054+0000","last_active":"2024-02-18T00:33:14.121824+0000","last_peered":"2024-02-18T00:33:14.121824+0000","last_clean":"2024-02-18T00:33:14.121824+0000","last_became_active":"2024-02-18T00:33:07.440763+0000","last_became_peered":"2024-02-18T00:33:07.440763+0000","last_unstale":"2024-02-18T00:33:14.121824+0000","last_undegraded":"2024-02-18T00:33:14.121824+0000","last_fullsized":"2024-02-18T00:33:14.121824+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109887+0000","last_change":"2024-02-18T00:33:05.594559+0000","last_active":"2024-02-18T00:33:14.109887+0000","last_peered":"2024-02-18T00:33:14.109887+0000","last_clean":"2024-02-18T00:33:14.109887+0000","last_became_active":"2024-02-18T00:33:05.594278+0000","last_became_peered":"2024-02-18T00:33:05.594278+0000","last_unstale":"2024-02-18T00:33:14.109887+0000","last_undegraded":"2024-02-18T00:33:14.109887+0000","last_fullsized":"2024-02-18T00:33:14.109887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140075+0000","last_change":"2024-02-18T00:33:05.575803+0000","last_active":"2024-02-18T00:33:14.140075+0000","last_peered":"2024-02-18T00:33:14.140075+0000","last_clean":"2024-02-18T00:33:14.140075+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:33:14.140075+0000","last_undegraded":"2024-02-18T00:33:14.140075+0000","last_fullsized":"2024-02-18T00:33:14.140075+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"61.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.13","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.120581+0000","last_change":"2024-02-18T00:33:13.173929+0000","last_active":"2024-02-18T00:33:14.120581+0000","last_peered":"2024-02-18T00:33:14.120581+0000","last_clean":"2024-02-18T00:33:14.120581+0000","last_became_active":"2024-02-18T00:33:13.173641+0000","last_became_peered":"2024-02-18T00:33:13.173641+0000","last_unstale":"2024-02-18T00:33:14.120581+0000","last_undegraded":"2024-02-18T00:33:14.120581+0000","last_fullsized":"2024-02-18T00:33:14.120581+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"44.c","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128389+0000","last_change":"2024-02-18T00:33:12.120034+0000","last_active":"2024-02-18T00:33:14.128389+0000","last_peered":"2024-02-18T00:33:14.128389+0000","last_clean":"2024-02-18T00:33:14.128389+0000","last_became_active":"2024-02-18T00:33:12.119653+0000","last_became_peered":"2024-02-18T00:33:12.119653+0000","last_unstale":"2024-02-18T00:33:14.128389+0000","last_undegraded":"2024-02-18T00:33:14.128389+0000","last_fullsized":"2024-02-18T00:33:14.128389+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"40.8","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.101101+0000","last_change":"2024-02-18T00:33:07.526546+0000","last_active":"2024-02-18T00:33:14.101101+0000","last_peered":"2024-02-18T00:33:14.101101+0000","last_clean":"2024-02-18T00:33:14.101101+0000","last_became_active":"2024-02-18T00:33:07.521957+0000","last_became_peered":"2024-02-18T00:33:07.521957+0000","last_unstale":"2024-02-18T00:33:14.101101+0000","last_undegraded":"2024-02-18T00:33:14.101101+0000","last_fullsized":"2024-02-18T00:33:14.101101+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.131855+0000","last_change":"2024-02-18T00:33:05.587963+0000","last_active":"2024-02-18T00:33:14.131855+0000","last_peered":"2024-02-18T00:33:14.131855+0000","last_clean":"2024-02-18T00:33:14.131855+0000","last_became_active":"2024-02-18T00:33:05.587687+0000","last_became_peered":"2024-02-18T00:33:05.587687+0000","last_unstale":"2024-02-18T00:33:14.131855+0000","last_undegraded":"2024-02-18T00:33:14.131855+0000","last_fullsized":"2024-02-18T00:33:14.131855+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.120666+0000","last_change":"2024-02-18T00:33:05.592130+0000","last_active":"2024-02-18T00:33:14.120666+0000","last_peered":"2024-02-18T00:33:14.120666+0000","last_clean":"2024-02-18T00:33:14.120666+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:33:14.120666+0000","last_undegraded":"2024-02-18T00:33:14.120666+0000","last_fullsized":"2024-02-18T00:33:14.120666+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.991759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.991759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"61.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.10","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127225+0000","last_change":"2024-02-18T00:33:13.134485+0000","last_active":"2024-02-18T00:33:14.127225+0000","last_peered":"2024-02-18T00:33:14.127225+0000","last_clean":"2024-02-18T00:33:14.127225+0000","last_became_active":"2024-02-18T00:33:13.134204+0000","last_became_peered":"2024-02-18T00:33:13.134204+0000","last_unstale":"2024-02-18T00:33:14.127225+0000","last_undegraded":"2024-02-18T00:33:14.127225+0000","last_fullsized":"2024-02-18T00:33:14.127225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"44.f","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108715+0000","last_change":"2024-02-18T00:33:12.155890+0000","last_active":"2024-02-18T00:33:14.108715+0000","last_peered":"2024-02-18T00:33:14.108715+0000","last_clean":"2024-02-18T00:33:14.108715+0000","last_became_active":"2024-02-18T00:33:12.155050+0000","last_became_peered":"2024-02-18T00:33:12.155050+0000","last_unstale":"2024-02-18T00:33:14.108715+0000","last_undegraded":"2024-02-18T00:33:14.108715+0000","last_fullsized":"2024-02-18T00:33:14.108715+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"40.b","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.100878+0000","last_change":"2024-02-18T00:33:07.492716+0000","last_active":"2024-02-18T00:33:14.100878+0000","last_peered":"2024-02-18T00:33:14.100878+0000","last_clean":"2024-02-18T00:33:14.100878+0000","last_became_active":"2024-02-18T00:33:07.492327+0000","last_became_peered":"2024-02-18T00:33:07.492327+0000","last_unstale":"2024-02-18T00:33:14.100878+0000","last_undegraded":"2024-02-18T00:33:14.100878+0000","last_fullsized":"2024-02-18T00:33:14.100878+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114720+0000","last_change":"2024-02-18T00:33:05.595758+0000","last_active":"2024-02-18T00:33:14.114720+0000","last_peered":"2024-02-18T00:33:14.114720+0000","last_clean":"2024-02-18T00:33:14.114720+0000","last_became_active":"2024-02-18T00:33:05.595429+0000","last_became_peered":"2024-02-18T00:33:05.595429+0000","last_unstale":"2024-02-18T00:33:14.114720+0000","last_undegraded":"2024-02-18T00:33:14.114720+0000","last_fullsized":"2024-02-18T00:33:14.114720+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109324+0000","last_change":"2024-02-18T00:33:05.591899+0000","last_active":"2024-02-18T00:33:14.109324+0000","last_peered":"2024-02-18T00:33:14.109324+0000","last_clean":"2024-02-18T00:33:14.109324+0000","last_became_active":"2024-02-18T00:33:05.591566+0000","last_became_peered":"2024-02-18T00:33:05.591566+0000","last_unstale":"2024-02-18T00:33:14.109324+0000","last_undegraded":"2024-02-18T00:33:14.109324+0000","last_fullsized":"2024-02-18T00:33:14.109324+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.521630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.521630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"61.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:18.948741+0000","last_change":"2024-02-18T00:33:18.948741+0000","last_active":"2024-02-18T00:33:18.948741+0000","last_peered":"2024-02-18T00:33:18.948741+0000","last_clean":"2024-02-18T00:33:18.948741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:18.948741+0000","last_undegraded":"2024-02-18T00:33:18.948741+0000","last_fullsized":"2024-02-18T00:33:18.948741+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:18.948741+0000","last_clean_scrub_stamp":"2024-02-18T00:33:18.948741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"60.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.943349+0000","last_change":"2024-02-18T00:33:17.943349+0000","last_active":"2024-02-18T00:33:17.943349+0000","last_peered":"2024-02-18T00:33:17.943349+0000","last_clean":"2024-02-18T00:33:17.943349+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.943349+0000","last_undegraded":"2024-02-18T00:33:17.943349+0000","last_fullsized":"2024-02-18T00:33:17.943349+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.943349+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.943349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"59.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"58.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:17.490160+0000","last_change":"2024-02-18T00:33:17.490160+0000","last_active":"2024-02-18T00:33:17.490160+0000","last_peered":"2024-02-18T00:33:17.490160+0000","last_clean":"2024-02-18T00:33:17.490160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:17.490160+0000","last_undegraded":"2024-02-18T00:33:17.490160+0000","last_fullsized":"2024-02-18T00:33:17.490160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:17.490160+0000","last_clean_scrub_stamp":"2024-02-18T00:33:17.490160+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":11,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109863+0000","last_change":"2024-02-18T00:33:13.155349+0000","last_active":"2024-02-18T00:33:14.109863+0000","last_peered":"2024-02-18T00:33:14.109863+0000","last_clean":"2024-02-18T00:33:14.109863+0000","last_became_active":"2024-02-18T00:33:13.154940+0000","last_became_peered":"2024-02-18T00:33:13.154940+0000","last_unstale":"2024-02-18T00:33:14.109863+0000","last_undegraded":"2024-02-18T00:33:14.109863+0000","last_fullsized":"2024-02-18T00:33:14.109863+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"44.e","version":"0'0","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.131933+0000","last_change":"2024-02-18T00:33:12.158769+0000","last_active":"2024-02-18T00:33:14.131933+0000","last_peered":"2024-02-18T00:33:14.131933+0000","last_clean":"2024-02-18T00:33:14.131933+0000","last_became_active":"2024-02-18T00:33:12.158332+0000","last_became_peered":"2024-02-18T00:33:12.158332+0000","last_unstale":"2024-02-18T00:33:14.131933+0000","last_undegraded":"2024-02-18T00:33:14.131933+0000","last_fullsized":"2024-02-18T00:33:14.131933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:08.815731+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.815731+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"40.a","version":"0'0","reported_seq":15,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.129504+0000","last_change":"2024-02-18T00:33:07.518959+0000","last_active":"2024-02-18T00:33:14.129504+0000","last_peered":"2024-02-18T00:33:14.129504+0000","last_clean":"2024-02-18T00:33:14.129504+0000","last_became_active":"2024-02-18T00:33:07.515485+0000","last_became_peered":"2024-02-18T00:33:07.515485+0000","last_unstale":"2024-02-18T00:33:14.129504+0000","last_undegraded":"2024-02-18T00:33:14.129504+0000","last_fullsized":"2024-02-18T00:33:14.129504+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.868502+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.868502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122140+0000","last_change":"2024-02-18T00:33:05.577075+0000","last_active":"2024-02-18T00:33:14.122140+0000","last_peered":"2024-02-18T00:33:14.122140+0000","last_clean":"2024-02-18T00:33:14.122140+0000","last_became_active":"2024-02-18T00:33:05.576778+0000","last_became_peered":"2024-02-18T00:33:05.576778+0000","last_unstale":"2024-02-18T00:33:14.122140+0000","last_undegraded":"2024-02-18T00:33:14.122140+0000","last_fullsized":"2024-02-18T00:33:14.122140+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:04.454172+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.454172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,7],"acting":[4,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102752+0000","last_change":"2024-02-18T00:33:05.574422+0000","last_active":"2024-02-18T00:33:14.102752+0000","last_peered":"2024-02-18T00:33:14.102752+0000","last_clean":"2024-02-18T00:33:14.102752+0000","last_became_active":"2024-02-18T00:33:05.574089+0000","last_became_peered":"2024-02-18T00:33:05.574089+0000","last_unstale":"2024-02-18T00:33:14.102752+0000","last_undegraded":"2024-02-18T00:33:14.102752+0000","last_fullsized":"2024-02-18T00:33:14.102752+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.754417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.754417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.16","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.139783+0000","last_change":"2024-02-18T00:33:02.535342+0000","last_active":"2024-02-18T00:33:14.139783+0000","last_peered":"2024-02-18T00:33:14.139783+0000","last_clean":"2024-02-18T00:33:14.139783+0000","last_became_active":"2024-02-18T00:33:02.534729+0000","last_became_peered":"2024-02-18T00:33:02.534729+0000","last_unstale":"2024-02-18T00:33:14.139783+0000","last_undegraded":"2024-02-18T00:33:14.139783+0000","last_fullsized":"2024-02-18T00:33:14.139783+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117186+0000","last_change":"2024-02-18T00:33:04.663714+0000","last_active":"2024-02-18T00:33:14.117186+0000","last_peered":"2024-02-18T00:33:14.117186+0000","last_clean":"2024-02-18T00:33:14.117186+0000","last_became_active":"2024-02-18T00:33:04.663445+0000","last_became_peered":"2024-02-18T00:33:04.663445+0000","last_unstale":"2024-02-18T00:33:14.117186+0000","last_undegraded":"2024-02-18T00:33:14.117186+0000","last_fullsized":"2024-02-18T00:33:14.117186+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":132,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106958+0000","last_change":"2024-02-18T00:32:50.594884+0000","last_active":"2024-02-18T00:33:14.106958+0000","last_peered":"2024-02-18T00:33:14.106958+0000","last_clean":"2024-02-18T00:33:14.106958+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:33:14.106958+0000","last_undegraded":"2024-02-18T00:33:14.106958+0000","last_fullsized":"2024-02-18T00:33:14.106958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:50.594799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.594799+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":133,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113932+0000","last_change":"2024-02-18T00:32:46.877329+0000","last_active":"2024-02-18T00:33:14.113932+0000","last_peered":"2024-02-18T00:33:14.113932+0000","last_clean":"2024-02-18T00:33:14.113932+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:33:14.113932+0000","last_undegraded":"2024-02-18T00:33:14.113932+0000","last_fullsized":"2024-02-18T00:33:14.113932+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:32:46.877252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.877252+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118239+0000","last_change":"2024-02-18T00:31:57.561503+0000","last_active":"2024-02-18T00:33:14.118239+0000","last_peered":"2024-02-18T00:33:14.118239+0000","last_clean":"2024-02-18T00:33:14.118239+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:33:14.118239+0000","last_undegraded":"2024-02-18T00:33:14.118239+0000","last_fullsized":"2024-02-18T00:33:14.118239+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.561435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.561435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.110908+0000","last_change":"2024-02-18T00:32:05.790164+0000","last_active":"2024-02-18T00:33:14.110908+0000","last_peered":"2024-02-18T00:33:14.110908+0000","last_clean":"2024-02-18T00:33:14.110908+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:33:14.110908+0000","last_undegraded":"2024-02-18T00:33:14.110908+0000","last_fullsized":"2024-02-18T00:33:14.110908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.790091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.790091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.134405+0000","last_change":"2024-02-18T00:33:15.134660+0000","last_active":"2024-02-18T00:33:15.134405+0000","last_peered":"2024-02-18T00:33:15.134405+0000","last_clean":"2024-02-18T00:33:15.134405+0000","last_became_active":"2024-02-18T00:33:01.556421+0000","last_became_peered":"2024-02-18T00:33:01.556421+0000","last_unstale":"2024-02-18T00:33:15.134405+0000","last_undegraded":"2024-02-18T00:33:15.134405+0000","last_fullsized":"2024-02-18T00:33:15.134405+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.1d","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.111043+0000","last_change":"2024-02-18T00:33:01.472792+0000","last_active":"2024-02-18T00:33:14.111043+0000","last_peered":"2024-02-18T00:33:14.111043+0000","last_clean":"2024-02-18T00:33:14.111043+0000","last_became_active":"2024-02-18T00:33:01.472120+0000","last_became_peered":"2024-02-18T00:33:01.472120+0000","last_unstale":"2024-02-18T00:33:14.111043+0000","last_undegraded":"2024-02-18T00:33:14.111043+0000","last_fullsized":"2024-02-18T00:33:14.111043+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137346+0000","last_change":"2024-02-18T00:33:04.600509+0000","last_active":"2024-02-18T00:33:14.137346+0000","last_peered":"2024-02-18T00:33:14.137346+0000","last_clean":"2024-02-18T00:33:14.137346+0000","last_became_active":"2024-02-18T00:33:04.600249+0000","last_became_peered":"2024-02-18T00:33:04.600249+0000","last_unstale":"2024-02-18T00:33:14.137346+0000","last_undegraded":"2024-02-18T00:33:14.137346+0000","last_fullsized":"2024-02-18T00:33:14.137346+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132297+0000","last_change":"2024-02-18T00:33:02.535442+0000","last_active":"2024-02-18T00:33:14.132297+0000","last_peered":"2024-02-18T00:33:14.132297+0000","last_clean":"2024-02-18T00:33:14.132297+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:33:14.132297+0000","last_undegraded":"2024-02-18T00:33:14.132297+0000","last_fullsized":"2024-02-18T00:33:14.132297+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127208+0000","last_change":"2024-02-18T00:32:50.007673+0000","last_active":"2024-02-18T00:33:14.127208+0000","last_peered":"2024-02-18T00:33:14.127208+0000","last_clean":"2024-02-18T00:33:14.127208+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:33:14.127208+0000","last_undegraded":"2024-02-18T00:33:14.127208+0000","last_fullsized":"2024-02-18T00:33:14.127208+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.007594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.007594+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127939+0000","last_change":"2024-02-18T00:32:03.931078+0000","last_active":"2024-02-18T00:33:14.127939+0000","last_peered":"2024-02-18T00:33:14.127939+0000","last_clean":"2024-02-18T00:33:14.127939+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:33:14.127939+0000","last_undegraded":"2024-02-18T00:33:14.127939+0000","last_fullsized":"2024-02-18T00:33:14.127939+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:32:03.930967+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113868+0000","last_change":"2024-02-18T00:32:00.527642+0000","last_active":"2024-02-18T00:33:14.113868+0000","last_peered":"2024-02-18T00:33:14.113868+0000","last_clean":"2024-02-18T00:33:14.113868+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:33:14.113868+0000","last_undegraded":"2024-02-18T00:33:14.113868+0000","last_fullsized":"2024-02-18T00:33:14.113868+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:00.527580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:00.527580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121541+0000","last_change":"2024-02-18T00:32:39.973147+0000","last_active":"2024-02-18T00:33:14.121541+0000","last_peered":"2024-02-18T00:33:14.121541+0000","last_clean":"2024-02-18T00:33:14.121541+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:33:14.121541+0000","last_undegraded":"2024-02-18T00:33:14.121541+0000","last_fullsized":"2024-02-18T00:33:14.121541+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:39.973040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:39.973040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.110654+0000","last_change":"2024-02-18T00:33:15.111008+0000","last_active":"2024-02-18T00:33:15.110654+0000","last_peered":"2024-02-18T00:33:15.110654+0000","last_clean":"2024-02-18T00:33:15.110654+0000","last_became_active":"2024-02-18T00:33:01.546263+0000","last_became_peered":"2024-02-18T00:33:01.546263+0000","last_unstale":"2024-02-18T00:33:15.110654+0000","last_undegraded":"2024-02-18T00:33:15.110654+0000","last_fullsized":"2024-02-18T00:33:15.110654+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.1e","version":"106'2","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132216+0000","last_change":"2024-02-18T00:33:01.463387+0000","last_active":"2024-02-18T00:33:14.132216+0000","last_peered":"2024-02-18T00:33:14.132216+0000","last_clean":"2024-02-18T00:33:14.132216+0000","last_became_active":"2024-02-18T00:33:01.462544+0000","last_became_peered":"2024-02-18T00:33:01.462544+0000","last_unstale":"2024-02-18T00:33:14.132216+0000","last_undegraded":"2024-02-18T00:33:14.132216+0000","last_fullsized":"2024-02-18T00:33:14.132216+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.14","version":"109'2","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.112921+0000","last_change":"2024-02-18T00:33:02.511910+0000","last_active":"2024-02-18T00:33:14.112921+0000","last_peered":"2024-02-18T00:33:14.112921+0000","last_clean":"2024-02-18T00:33:14.112921+0000","last_became_active":"2024-02-18T00:33:02.511292+0000","last_became_peered":"2024-02-18T00:33:02.511292+0000","last_unstale":"2024-02-18T00:33:14.112921+0000","last_undegraded":"2024-02-18T00:33:14.112921+0000","last_fullsized":"2024-02-18T00:33:14.112921+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.147951+0000","last_change":"2024-02-18T00:33:04.645450+0000","last_active":"2024-02-18T00:33:14.147951+0000","last_peered":"2024-02-18T00:33:14.147951+0000","last_clean":"2024-02-18T00:33:14.147951+0000","last_became_active":"2024-02-18T00:33:04.644990+0000","last_became_peered":"2024-02-18T00:33:04.644990+0000","last_unstale":"2024-02-18T00:33:14.147951+0000","last_undegraded":"2024-02-18T00:33:14.147951+0000","last_fullsized":"2024-02-18T00:33:14.147951+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":130,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121473+0000","last_change":"2024-02-18T00:32:53.370947+0000","last_active":"2024-02-18T00:33:14.121473+0000","last_peered":"2024-02-18T00:33:14.121473+0000","last_clean":"2024-02-18T00:33:14.121473+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:33:14.121473+0000","last_undegraded":"2024-02-18T00:33:14.121473+0000","last_fullsized":"2024-02-18T00:33:14.121473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:32:53.370869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.370869+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108481+0000","last_change":"2024-02-18T00:32:04.461036+0000","last_active":"2024-02-18T00:33:14.108481+0000","last_peered":"2024-02-18T00:33:14.108481+0000","last_clean":"2024-02-18T00:33:14.108481+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:33:14.108481+0000","last_undegraded":"2024-02-18T00:33:14.108481+0000","last_fullsized":"2024-02-18T00:33:14.108481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:04.460968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.460968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104773+0000","last_change":"2024-02-18T00:32:02.507109+0000","last_active":"2024-02-18T00:33:14.104773+0000","last_peered":"2024-02-18T00:33:14.104773+0000","last_clean":"2024-02-18T00:33:14.104773+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:33:14.104773+0000","last_undegraded":"2024-02-18T00:33:14.104773+0000","last_fullsized":"2024-02-18T00:33:14.104773+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:02.507010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.507010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103267+0000","last_change":"2024-02-18T00:32:38.665917+0000","last_active":"2024-02-18T00:33:14.103267+0000","last_peered":"2024-02-18T00:33:14.103267+0000","last_clean":"2024-02-18T00:33:14.103267+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:33:14.103267+0000","last_undegraded":"2024-02-18T00:33:14.103267+0000","last_fullsized":"2024-02-18T00:33:14.103267+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.665830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.665830+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.109163+0000","last_change":"2024-02-18T00:33:15.110013+0000","last_active":"2024-02-18T00:33:15.109163+0000","last_peered":"2024-02-18T00:33:15.109163+0000","last_clean":"2024-02-18T00:33:15.109163+0000","last_became_active":"2024-02-18T00:33:01.510830+0000","last_became_peered":"2024-02-18T00:33:01.510830+0000","last_unstale":"2024-02-18T00:33:15.109163+0000","last_undegraded":"2024-02-18T00:33:15.109163+0000","last_fullsized":"2024-02-18T00:33:15.109163+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.1f","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135767+0000","last_change":"2024-02-18T00:33:01.571369+0000","last_active":"2024-02-18T00:33:14.135767+0000","last_peered":"2024-02-18T00:33:14.135767+0000","last_clean":"2024-02-18T00:33:14.135767+0000","last_became_active":"2024-02-18T00:33:01.570706+0000","last_became_peered":"2024-02-18T00:33:01.570706+0000","last_unstale":"2024-02-18T00:33:14.135767+0000","last_undegraded":"2024-02-18T00:33:14.135767+0000","last_fullsized":"2024-02-18T00:33:14.135767+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"109'9","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125329+0000","last_change":"2024-02-18T00:33:02.492423+0000","last_active":"2024-02-18T00:33:14.125329+0000","last_peered":"2024-02-18T00:33:14.125329+0000","last_clean":"2024-02-18T00:33:14.125329+0000","last_became_active":"2024-02-18T00:33:02.492133+0000","last_became_peered":"2024-02-18T00:33:02.492133+0000","last_unstale":"2024-02-18T00:33:14.125329+0000","last_undegraded":"2024-02-18T00:33:14.125329+0000","last_fullsized":"2024-02-18T00:33:14.125329+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114260+0000","last_change":"2024-02-18T00:33:02.523582+0000","last_active":"2024-02-18T00:33:14.114260+0000","last_peered":"2024-02-18T00:33:14.114260+0000","last_clean":"2024-02-18T00:33:14.114260+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:33:14.114260+0000","last_undegraded":"2024-02-18T00:33:14.114260+0000","last_fullsized":"2024-02-18T00:33:14.114260+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":111,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138192+0000","last_change":"2024-02-18T00:33:02.051762+0000","last_active":"2024-02-18T00:33:14.138192+0000","last_peered":"2024-02-18T00:33:14.138192+0000","last_clean":"2024-02-18T00:33:14.138192+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:33:14.138192+0000","last_undegraded":"2024-02-18T00:33:14.138192+0000","last_fullsized":"2024-02-18T00:33:14.138192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:02.051679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.051679+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114930+0000","last_change":"2024-02-18T00:31:54.616292+0000","last_active":"2024-02-18T00:33:14.114930+0000","last_peered":"2024-02-18T00:33:14.114930+0000","last_clean":"2024-02-18T00:33:14.114930+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:33:14.114930+0000","last_undegraded":"2024-02-18T00:33:14.114930+0000","last_fullsized":"2024-02-18T00:33:14.114930+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:54.616213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.616213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128189+0000","last_change":"2024-02-18T00:32:45.751973+0000","last_active":"2024-02-18T00:33:14.128189+0000","last_peered":"2024-02-18T00:33:14.128189+0000","last_clean":"2024-02-18T00:33:14.128189+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:33:14.128189+0000","last_undegraded":"2024-02-18T00:33:14.128189+0000","last_fullsized":"2024-02-18T00:33:14.128189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:45.751874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.751874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121761+0000","last_change":"2024-02-18T00:32:07.449328+0000","last_active":"2024-02-18T00:33:14.121761+0000","last_peered":"2024-02-18T00:33:14.121761+0000","last_clean":"2024-02-18T00:33:14.121761+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:33:14.121761+0000","last_undegraded":"2024-02-18T00:33:14.121761+0000","last_fullsized":"2024-02-18T00:33:14.121761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.449249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.449249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.119527+0000","last_change":"2024-02-18T00:33:15.120665+0000","last_active":"2024-02-18T00:33:15.119527+0000","last_peered":"2024-02-18T00:33:15.119527+0000","last_clean":"2024-02-18T00:33:15.119527+0000","last_became_active":"2024-02-18T00:33:01.547029+0000","last_became_peered":"2024-02-18T00:33:01.547029+0000","last_unstale":"2024-02-18T00:33:15.119527+0000","last_undegraded":"2024-02-18T00:33:15.119527+0000","last_fullsized":"2024-02-18T00:33:15.119527+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.18","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102501+0000","last_change":"2024-02-18T00:33:01.559343+0000","last_active":"2024-02-18T00:33:14.102501+0000","last_peered":"2024-02-18T00:33:14.102501+0000","last_clean":"2024-02-18T00:33:14.102501+0000","last_became_active":"2024-02-18T00:33:01.559068+0000","last_became_peered":"2024-02-18T00:33:01.559068+0000","last_unstale":"2024-02-18T00:33:14.102501+0000","last_undegraded":"2024-02-18T00:33:14.102501+0000","last_fullsized":"2024-02-18T00:33:14.102501+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130957+0000","last_change":"2024-02-18T00:33:02.533251+0000","last_active":"2024-02-18T00:33:14.130957+0000","last_peered":"2024-02-18T00:33:14.130957+0000","last_clean":"2024-02-18T00:33:14.130957+0000","last_became_active":"2024-02-18T00:33:02.532428+0000","last_became_peered":"2024-02-18T00:33:02.532428+0000","last_unstale":"2024-02-18T00:33:14.130957+0000","last_undegraded":"2024-02-18T00:33:14.130957+0000","last_fullsized":"2024-02-18T00:33:14.130957+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121831+0000","last_change":"2024-02-18T00:33:04.608115+0000","last_active":"2024-02-18T00:33:14.121831+0000","last_peered":"2024-02-18T00:33:14.121831+0000","last_clean":"2024-02-18T00:33:14.121831+0000","last_became_active":"2024-02-18T00:33:04.607229+0000","last_became_peered":"2024-02-18T00:33:04.607229+0000","last_unstale":"2024-02-18T00:33:14.121831+0000","last_undegraded":"2024-02-18T00:33:14.121831+0000","last_fullsized":"2024-02-18T00:33:14.121831+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130189+0000","last_change":"2024-02-18T00:32:43.969645+0000","last_active":"2024-02-18T00:33:14.130189+0000","last_peered":"2024-02-18T00:33:14.130189+0000","last_clean":"2024-02-18T00:33:14.130189+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:33:14.130189+0000","last_undegraded":"2024-02-18T00:33:14.130189+0000","last_fullsized":"2024-02-18T00:33:14.130189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:32:43.969566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.969566+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122812+0000","last_change":"2024-02-18T00:31:50.988509+0000","last_active":"2024-02-18T00:33:14.122812+0000","last_peered":"2024-02-18T00:33:14.122812+0000","last_clean":"2024-02-18T00:33:14.122812+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:33:14.122812+0000","last_undegraded":"2024-02-18T00:33:14.122812+0000","last_fullsized":"2024-02-18T00:33:14.122812+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.988429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":135,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109971+0000","last_change":"2024-02-18T00:32:46.413965+0000","last_active":"2024-02-18T00:33:14.109971+0000","last_peered":"2024-02-18T00:33:14.109971+0000","last_clean":"2024-02-18T00:33:14.109971+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:33:14.109971+0000","last_undegraded":"2024-02-18T00:33:14.109971+0000","last_fullsized":"2024-02-18T00:33:14.109971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:46.413894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.413894+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.139510+0000","last_change":"2024-02-18T00:31:58.064645+0000","last_active":"2024-02-18T00:33:14.139510+0000","last_peered":"2024-02-18T00:33:14.139510+0000","last_clean":"2024-02-18T00:33:14.139510+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:33:14.139510+0000","last_undegraded":"2024-02-18T00:33:14.139510+0000","last_fullsized":"2024-02-18T00:33:14.139510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.064559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.064559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"110'1","reported_seq":40,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.125765+0000","last_change":"2024-02-18T00:33:15.125948+0000","last_active":"2024-02-18T00:33:15.125765+0000","last_peered":"2024-02-18T00:33:15.125765+0000","last_clean":"2024-02-18T00:33:15.125765+0000","last_became_active":"2024-02-18T00:33:01.589795+0000","last_became_peered":"2024-02-18T00:33:01.589795+0000","last_unstale":"2024-02-18T00:33:15.125765+0000","last_undegraded":"2024-02-18T00:33:15.125765+0000","last_fullsized":"2024-02-18T00:33:15.125765+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.19","version":"108'1","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.111190+0000","last_change":"2024-02-18T00:33:01.434399+0000","last_active":"2024-02-18T00:33:14.111190+0000","last_peered":"2024-02-18T00:33:14.111190+0000","last_clean":"2024-02-18T00:33:14.111190+0000","last_became_active":"2024-02-18T00:33:01.433012+0000","last_became_peered":"2024-02-18T00:33:01.433012+0000","last_unstale":"2024-02-18T00:33:14.111190+0000","last_undegraded":"2024-02-18T00:33:14.111190+0000","last_fullsized":"2024-02-18T00:33:14.111190+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165836,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":24577,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.11","version":"109'7","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127216+0000","last_change":"2024-02-18T00:33:04.669390+0000","last_active":"2024-02-18T00:33:14.127216+0000","last_peered":"2024-02-18T00:33:14.127216+0000","last_clean":"2024-02-18T00:33:14.127216+0000","last_became_active":"2024-02-18T00:33:04.669099+0000","last_became_peered":"2024-02-18T00:33:04.669099+0000","last_unstale":"2024-02-18T00:33:14.127216+0000","last_undegraded":"2024-02-18T00:33:14.127216+0000","last_fullsized":"2024-02-18T00:33:14.127216+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.123473+0000","last_change":"2024-02-18T00:33:02.536636+0000","last_active":"2024-02-18T00:33:14.123473+0000","last_peered":"2024-02-18T00:33:14.123473+0000","last_clean":"2024-02-18T00:33:14.123473+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:33:14.123473+0000","last_undegraded":"2024-02-18T00:33:14.123473+0000","last_fullsized":"2024-02-18T00:33:14.123473+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":81,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115204+0000","last_change":"2024-02-18T00:31:50.844080+0000","last_active":"2024-02-18T00:33:14.115204+0000","last_peered":"2024-02-18T00:33:14.115204+0000","last_clean":"2024-02-18T00:33:14.115204+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:33:14.115204+0000","last_undegraded":"2024-02-18T00:33:14.115204+0000","last_fullsized":"2024-02-18T00:33:14.115204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.843994+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118560+0000","last_change":"2024-02-18T00:33:04.715878+0000","last_active":"2024-02-18T00:33:14.118560+0000","last_peered":"2024-02-18T00:33:14.118560+0000","last_clean":"2024-02-18T00:33:14.118560+0000","last_became_active":"2024-02-18T00:33:04.715539+0000","last_became_peered":"2024-02-18T00:33:04.715539+0000","last_unstale":"2024-02-18T00:33:14.118560+0000","last_undegraded":"2024-02-18T00:33:14.118560+0000","last_fullsized":"2024-02-18T00:33:14.118560+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.960661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.960661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":132,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.139429+0000","last_change":"2024-02-18T00:32:49.092381+0000","last_active":"2024-02-18T00:33:14.139429+0000","last_peered":"2024-02-18T00:33:14.139429+0000","last_clean":"2024-02-18T00:33:14.139429+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:33:14.139429+0000","last_undegraded":"2024-02-18T00:33:14.139429+0000","last_fullsized":"2024-02-18T00:33:14.139429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:49.092293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.092293+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141686+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:33:14.141686+0000","last_peered":"2024-02-18T00:33:14.141686+0000","last_clean":"2024-02-18T00:33:14.141686+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:33:14.141686+0000","last_undegraded":"2024-02-18T00:33:14.141686+0000","last_fullsized":"2024-02-18T00:33:14.141686+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.114979+0000","last_change":"2024-02-18T00:33:15.115141+0000","last_active":"2024-02-18T00:33:15.114979+0000","last_peered":"2024-02-18T00:33:15.114979+0000","last_clean":"2024-02-18T00:33:15.114979+0000","last_became_active":"2024-02-18T00:33:01.551695+0000","last_became_peered":"2024-02-18T00:33:01.551695+0000","last_unstale":"2024-02-18T00:33:15.114979+0000","last_undegraded":"2024-02-18T00:33:15.114979+0000","last_fullsized":"2024-02-18T00:33:15.114979+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.1a","version":"106'3","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104965+0000","last_change":"2024-02-18T00:33:01.569847+0000","last_active":"2024-02-18T00:33:14.104965+0000","last_peered":"2024-02-18T00:33:14.104965+0000","last_clean":"2024-02-18T00:33:14.104965+0000","last_became_active":"2024-02-18T00:33:01.569569+0000","last_became_peered":"2024-02-18T00:33:01.569569+0000","last_unstale":"2024-02-18T00:33:14.104965+0000","last_undegraded":"2024-02-18T00:33:14.104965+0000","last_fullsized":"2024-02-18T00:33:14.104965+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104531+0000","last_change":"2024-02-18T00:33:04.703417+0000","last_active":"2024-02-18T00:33:14.104531+0000","last_peered":"2024-02-18T00:33:14.104531+0000","last_clean":"2024-02-18T00:33:14.104531+0000","last_became_active":"2024-02-18T00:33:04.701206+0000","last_became_peered":"2024-02-18T00:33:04.701206+0000","last_unstale":"2024-02-18T00:33:14.104531+0000","last_undegraded":"2024-02-18T00:33:14.104531+0000","last_fullsized":"2024-02-18T00:33:14.104531+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124606+0000","last_change":"2024-02-18T00:33:02.533944+0000","last_active":"2024-02-18T00:33:14.124606+0000","last_peered":"2024-02-18T00:33:14.124606+0000","last_clean":"2024-02-18T00:33:14.124606+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:33:14.124606+0000","last_undegraded":"2024-02-18T00:33:14.124606+0000","last_fullsized":"2024-02-18T00:33:14.124606+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":112,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138798+0000","last_change":"2024-02-18T00:31:55.104289+0000","last_active":"2024-02-18T00:33:14.138798+0000","last_peered":"2024-02-18T00:33:14.138798+0000","last_clean":"2024-02-18T00:33:14.138798+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:33:14.138798+0000","last_undegraded":"2024-02-18T00:33:14.138798+0000","last_fullsized":"2024-02-18T00:33:14.138798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:55.104195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.104195+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103709+0000","last_change":"2024-02-18T00:33:04.610117+0000","last_active":"2024-02-18T00:33:14.103709+0000","last_peered":"2024-02-18T00:33:14.103709+0000","last_clean":"2024-02-18T00:33:14.103709+0000","last_became_active":"2024-02-18T00:33:04.609915+0000","last_became_peered":"2024-02-18T00:33:04.609915+0000","last_unstale":"2024-02-18T00:33:14.103709+0000","last_undegraded":"2024-02-18T00:33:14.103709+0000","last_fullsized":"2024-02-18T00:33:14.103709+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:48.594640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:48.594640+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115758+0000","last_change":"2024-02-18T00:32:47.915174+0000","last_active":"2024-02-18T00:33:14.115758+0000","last_peered":"2024-02-18T00:33:14.115758+0000","last_clean":"2024-02-18T00:33:14.115758+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:33:14.115758+0000","last_undegraded":"2024-02-18T00:33:14.115758+0000","last_fullsized":"2024-02-18T00:33:14.115758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:32:47.915092+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.142821+0000","last_change":"2024-02-18T00:31:58.530850+0000","last_active":"2024-02-18T00:33:14.142821+0000","last_peered":"2024-02-18T00:33:14.142821+0000","last_clean":"2024-02-18T00:33:14.142821+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:33:14.142821+0000","last_undegraded":"2024-02-18T00:33:14.142821+0000","last_fullsized":"2024-02-18T00:33:14.142821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.530768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.530768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.136083+0000","last_change":"2024-02-18T00:33:15.136339+0000","last_active":"2024-02-18T00:33:15.136083+0000","last_peered":"2024-02-18T00:33:15.136083+0000","last_clean":"2024-02-18T00:33:15.136083+0000","last_became_active":"2024-02-18T00:33:01.556223+0000","last_became_peered":"2024-02-18T00:33:01.556223+0000","last_unstale":"2024-02-18T00:33:15.136083+0000","last_undegraded":"2024-02-18T00:33:15.136083+0000","last_fullsized":"2024-02-18T00:33:15.136083+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.1b","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124518+0000","last_change":"2024-02-18T00:33:01.450143+0000","last_active":"2024-02-18T00:33:14.124518+0000","last_peered":"2024-02-18T00:33:14.124518+0000","last_clean":"2024-02-18T00:33:14.124518+0000","last_became_active":"2024-02-18T00:33:01.449972+0000","last_became_peered":"2024-02-18T00:33:01.449972+0000","last_unstale":"2024-02-18T00:33:14.124518+0000","last_undegraded":"2024-02-18T00:33:14.124518+0000","last_fullsized":"2024-02-18T00:33:14.124518+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135222+0000","last_change":"2024-02-18T00:33:02.511006+0000","last_active":"2024-02-18T00:33:14.135222+0000","last_peered":"2024-02-18T00:33:14.135222+0000","last_clean":"2024-02-18T00:33:14.135222+0000","last_became_active":"2024-02-18T00:33:02.510597+0000","last_became_peered":"2024-02-18T00:33:02.510597+0000","last_unstale":"2024-02-18T00:33:14.135222+0000","last_undegraded":"2024-02-18T00:33:14.135222+0000","last_fullsized":"2024-02-18T00:33:14.135222+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130906+0000","last_change":"2024-02-18T00:33:02.483392+0000","last_active":"2024-02-18T00:33:14.130906+0000","last_peered":"2024-02-18T00:33:14.130906+0000","last_clean":"2024-02-18T00:33:14.130906+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:33:14.130906+0000","last_undegraded":"2024-02-18T00:33:14.130906+0000","last_fullsized":"2024-02-18T00:33:14.130906+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":133,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118244+0000","last_change":"2024-02-18T00:31:39.294080+0000","last_active":"2024-02-18T00:33:14.118244+0000","last_peered":"2024-02-18T00:33:14.118244+0000","last_clean":"2024-02-18T00:33:14.118244+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:33:14.118244+0000","last_undegraded":"2024-02-18T00:33:14.118244+0000","last_fullsized":"2024-02-18T00:33:14.118244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:31:39.294006+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:31:39.294006+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":114,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122672+0000","last_change":"2024-02-18T00:31:56.063646+0000","last_active":"2024-02-18T00:33:14.122672+0000","last_peered":"2024-02-18T00:33:14.122672+0000","last_clean":"2024-02-18T00:33:14.122672+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:33:14.122672+0000","last_undegraded":"2024-02-18T00:33:14.122672+0000","last_fullsized":"2024-02-18T00:33:14.122672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:31:56.063560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.063560+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135137+0000","last_change":"2024-02-18T00:33:04.695273+0000","last_active":"2024-02-18T00:33:14.135137+0000","last_peered":"2024-02-18T00:33:14.135137+0000","last_clean":"2024-02-18T00:33:14.135137+0000","last_became_active":"2024-02-18T00:33:04.694880+0000","last_became_peered":"2024-02-18T00:33:04.694880+0000","last_unstale":"2024-02-18T00:33:14.135137+0000","last_undegraded":"2024-02-18T00:33:14.135137+0000","last_fullsized":"2024-02-18T00:33:14.135137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.198542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.198542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115459+0000","last_change":"2024-02-18T00:32:47.565531+0000","last_active":"2024-02-18T00:33:14.115459+0000","last_peered":"2024-02-18T00:33:14.115459+0000","last_clean":"2024-02-18T00:33:14.115459+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:33:14.115459+0000","last_undegraded":"2024-02-18T00:33:14.115459+0000","last_fullsized":"2024-02-18T00:33:14.115459+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:47.565450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:47.565450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"116'1","reported_seq":40,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.125404+0000","last_change":"2024-02-18T00:33:15.125694+0000","last_active":"2024-02-18T00:33:15.125404+0000","last_peered":"2024-02-18T00:33:15.125404+0000","last_clean":"2024-02-18T00:33:15.125404+0000","last_became_active":"2024-02-18T00:33:01.578876+0000","last_became_peered":"2024-02-18T00:33:01.578876+0000","last_unstale":"2024-02-18T00:33:15.125404+0000","last_undegraded":"2024-02-18T00:33:15.125404+0000","last_fullsized":"2024-02-18T00:33:15.125404+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.14","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.143369+0000","last_change":"2024-02-18T00:33:01.532614+0000","last_active":"2024-02-18T00:33:14.143369+0000","last_peered":"2024-02-18T00:33:14.143369+0000","last_clean":"2024-02-18T00:33:14.143369+0000","last_became_active":"2024-02-18T00:33:01.516952+0000","last_became_peered":"2024-02-18T00:33:01.516952+0000","last_unstale":"2024-02-18T00:33:14.143369+0000","last_undegraded":"2024-02-18T00:33:14.143369+0000","last_fullsized":"2024-02-18T00:33:14.143369+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137362+0000","last_change":"2024-02-18T00:33:02.493878+0000","last_active":"2024-02-18T00:33:14.137362+0000","last_peered":"2024-02-18T00:33:14.137362+0000","last_clean":"2024-02-18T00:33:14.137362+0000","last_became_active":"2024-02-18T00:33:02.493597+0000","last_became_peered":"2024-02-18T00:33:02.493597+0000","last_unstale":"2024-02-18T00:33:14.137362+0000","last_undegraded":"2024-02-18T00:33:14.137362+0000","last_fullsized":"2024-02-18T00:33:14.137362+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141146+0000","last_change":"2024-02-18T00:33:02.477932+0000","last_active":"2024-02-18T00:33:14.141146+0000","last_peered":"2024-02-18T00:33:14.141146+0000","last_clean":"2024-02-18T00:33:14.141146+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:33:14.141146+0000","last_undegraded":"2024-02-18T00:33:14.141146+0000","last_fullsized":"2024-02-18T00:33:14.141146+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124970+0000","last_change":"2024-02-18T00:32:05.972387+0000","last_active":"2024-02-18T00:33:14.124970+0000","last_peered":"2024-02-18T00:33:14.124970+0000","last_clean":"2024-02-18T00:33:14.124970+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:33:14.124970+0000","last_undegraded":"2024-02-18T00:33:14.124970+0000","last_fullsized":"2024-02-18T00:33:14.124970+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.972303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.972303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":150,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118157+0000","last_change":"2024-02-18T00:32:05.656439+0000","last_active":"2024-02-18T00:33:14.118157+0000","last_peered":"2024-02-18T00:33:14.118157+0000","last_clean":"2024-02-18T00:33:14.118157+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:33:14.118157+0000","last_undegraded":"2024-02-18T00:33:14.118157+0000","last_fullsized":"2024-02-18T00:33:14.118157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.656344+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132444+0000","last_change":"2024-02-18T00:33:04.715734+0000","last_active":"2024-02-18T00:33:14.132444+0000","last_peered":"2024-02-18T00:33:14.132444+0000","last_clean":"2024-02-18T00:33:14.132444+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:33:14.132444+0000","last_undegraded":"2024-02-18T00:33:14.132444+0000","last_fullsized":"2024-02-18T00:33:14.132444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130869+0000","last_change":"2024-02-18T00:32:42.033835+0000","last_active":"2024-02-18T00:33:14.130869+0000","last_peered":"2024-02-18T00:33:14.130869+0000","last_clean":"2024-02-18T00:33:14.130869+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:33:14.130869+0000","last_undegraded":"2024-02-18T00:33:14.130869+0000","last_fullsized":"2024-02-18T00:33:14.130869+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:42.033759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:42.033759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.129718+0000","last_change":"2024-02-18T00:33:15.130105+0000","last_active":"2024-02-18T00:33:15.129718+0000","last_peered":"2024-02-18T00:33:15.129718+0000","last_clean":"2024-02-18T00:33:15.129718+0000","last_became_active":"2024-02-18T00:33:01.448830+0000","last_became_peered":"2024-02-18T00:33:01.448830+0000","last_unstale":"2024-02-18T00:33:15.129718+0000","last_undegraded":"2024-02-18T00:33:15.129718+0000","last_fullsized":"2024-02-18T00:33:15.129718+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.15","version":"108'4","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141067+0000","last_change":"2024-02-18T00:33:01.549700+0000","last_active":"2024-02-18T00:33:14.141067+0000","last_peered":"2024-02-18T00:33:14.141067+0000","last_clean":"2024-02-18T00:33:14.141067+0000","last_became_active":"2024-02-18T00:33:01.548851+0000","last_became_peered":"2024-02-18T00:33:01.548851+0000","last_unstale":"2024-02-18T00:33:14.141067+0000","last_undegraded":"2024-02-18T00:33:14.141067+0000","last_fullsized":"2024-02-18T00:33:14.141067+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"109'7","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.110253+0000","last_change":"2024-02-18T00:33:04.700942+0000","last_active":"2024-02-18T00:33:14.110253+0000","last_peered":"2024-02-18T00:33:14.110253+0000","last_clean":"2024-02-18T00:33:14.110253+0000","last_became_active":"2024-02-18T00:33:04.700673+0000","last_became_peered":"2024-02-18T00:33:04.700673+0000","last_unstale":"2024-02-18T00:33:14.110253+0000","last_undegraded":"2024-02-18T00:33:14.110253+0000","last_fullsized":"2024-02-18T00:33:14.110253+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141476+0000","last_change":"2024-02-18T00:33:04.650781+0000","last_active":"2024-02-18T00:33:14.141476+0000","last_peered":"2024-02-18T00:33:14.141476+0000","last_clean":"2024-02-18T00:33:14.141476+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:33:14.141476+0000","last_undegraded":"2024-02-18T00:33:14.141476+0000","last_fullsized":"2024-02-18T00:33:14.141476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.120407+0000","last_change":"2024-02-18T00:32:37.227614+0000","last_active":"2024-02-18T00:33:14.120407+0000","last_peered":"2024-02-18T00:33:14.120407+0000","last_clean":"2024-02-18T00:33:14.120407+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:33:14.120407+0000","last_undegraded":"2024-02-18T00:33:14.120407+0000","last_fullsized":"2024-02-18T00:33:14.120407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:37.227536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:37.227536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":128,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127045+0000","last_change":"2024-02-18T00:32:55.959110+0000","last_active":"2024-02-18T00:33:14.127045+0000","last_peered":"2024-02-18T00:33:14.127045+0000","last_clean":"2024-02-18T00:33:14.127045+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:33:14.127045+0000","last_undegraded":"2024-02-18T00:33:14.127045+0000","last_fullsized":"2024-02-18T00:33:14.127045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:32:55.959041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.959041+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":96,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107408+0000","last_change":"2024-02-18T00:33:04.581054+0000","last_active":"2024-02-18T00:33:14.107408+0000","last_peered":"2024-02-18T00:33:14.107408+0000","last_clean":"2024-02-18T00:33:14.107408+0000","last_became_active":"2024-02-18T00:33:04.576123+0000","last_became_peered":"2024-02-18T00:33:04.576123+0000","last_unstale":"2024-02-18T00:33:14.107408+0000","last_undegraded":"2024-02-18T00:33:14.107408+0000","last_fullsized":"2024-02-18T00:33:14.107408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:32:51.632894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:51.632894+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":436,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118644+0000","last_change":"2024-02-18T00:32:44.751653+0000","last_active":"2024-02-18T00:33:14.118644+0000","last_peered":"2024-02-18T00:33:14.118644+0000","last_clean":"2024-02-18T00:33:14.118644+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:33:14.118644+0000","last_undegraded":"2024-02-18T00:33:14.118644+0000","last_fullsized":"2024-02-18T00:33:14.118644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:44.751537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.751537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.129792+0000","last_change":"2024-02-18T00:33:15.131841+0000","last_active":"2024-02-18T00:33:15.129792+0000","last_peered":"2024-02-18T00:33:15.129792+0000","last_clean":"2024-02-18T00:33:15.129792+0000","last_became_active":"2024-02-18T00:33:01.441352+0000","last_became_peered":"2024-02-18T00:33:01.441352+0000","last_unstale":"2024-02-18T00:33:15.129792+0000","last_undegraded":"2024-02-18T00:33:15.129792+0000","last_fullsized":"2024-02-18T00:33:15.129792+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.16","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135763+0000","last_change":"2024-02-18T00:33:01.486028+0000","last_active":"2024-02-18T00:33:14.135763+0000","last_peered":"2024-02-18T00:33:14.135763+0000","last_clean":"2024-02-18T00:33:14.135763+0000","last_became_active":"2024-02-18T00:33:01.485748+0000","last_became_peered":"2024-02-18T00:33:01.485748+0000","last_unstale":"2024-02-18T00:33:14.135763+0000","last_undegraded":"2024-02-18T00:33:14.135763+0000","last_fullsized":"2024-02-18T00:33:14.135763+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"109'2","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128644+0000","last_change":"2024-02-18T00:33:02.534383+0000","last_active":"2024-02-18T00:33:14.128644+0000","last_peered":"2024-02-18T00:33:14.128644+0000","last_clean":"2024-02-18T00:33:14.128644+0000","last_became_active":"2024-02-18T00:33:02.533911+0000","last_became_peered":"2024-02-18T00:33:02.533911+0000","last_unstale":"2024-02-18T00:33:14.128644+0000","last_undegraded":"2024-02-18T00:33:14.128644+0000","last_fullsized":"2024-02-18T00:33:14.128644+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141512+0000","last_change":"2024-02-18T00:33:04.587776+0000","last_active":"2024-02-18T00:33:14.141512+0000","last_peered":"2024-02-18T00:33:14.141512+0000","last_clean":"2024-02-18T00:33:14.141512+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:33:14.141512+0000","last_undegraded":"2024-02-18T00:33:14.141512+0000","last_fullsized":"2024-02-18T00:33:14.141512+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115901+0000","last_change":"2024-02-18T00:32:43.913604+0000","last_active":"2024-02-18T00:33:14.115901+0000","last_peered":"2024-02-18T00:33:14.115901+0000","last_clean":"2024-02-18T00:33:14.115901+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:33:14.115901+0000","last_undegraded":"2024-02-18T00:33:14.115901+0000","last_fullsized":"2024-02-18T00:33:14.115901+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.913522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.913522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107037+0000","last_change":"2024-02-18T00:32:45.002522+0000","last_active":"2024-02-18T00:33:14.107037+0000","last_peered":"2024-02-18T00:33:14.107037+0000","last_clean":"2024-02-18T00:33:14.107037+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:33:14.107037+0000","last_undegraded":"2024-02-18T00:33:14.107037+0000","last_fullsized":"2024-02-18T00:33:14.107037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:32:45.002406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.002406+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103355+0000","last_change":"2024-02-18T00:33:04.610118+0000","last_active":"2024-02-18T00:33:14.103355+0000","last_peered":"2024-02-18T00:33:14.103355+0000","last_clean":"2024-02-18T00:33:14.103355+0000","last_became_active":"2024-02-18T00:33:04.609908+0000","last_became_peered":"2024-02-18T00:33:04.609908+0000","last_unstale":"2024-02-18T00:33:14.103355+0000","last_undegraded":"2024-02-18T00:33:14.103355+0000","last_fullsized":"2024-02-18T00:33:14.103355+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.754417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.754417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121092+0000","last_change":"2024-02-18T00:31:53.976449+0000","last_active":"2024-02-18T00:33:14.121092+0000","last_peered":"2024-02-18T00:33:14.121092+0000","last_clean":"2024-02-18T00:33:14.121092+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:33:14.121092+0000","last_undegraded":"2024-02-18T00:33:14.121092+0000","last_fullsized":"2024-02-18T00:33:14.121092+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:53.976377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:31:53.976377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128319+0000","last_change":"2024-02-18T00:32:53.813004+0000","last_active":"2024-02-18T00:33:14.128319+0000","last_peered":"2024-02-18T00:33:14.128319+0000","last_clean":"2024-02-18T00:33:14.128319+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:33:14.128319+0000","last_undegraded":"2024-02-18T00:33:14.128319+0000","last_fullsized":"2024-02-18T00:33:14.128319+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.812897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.812897+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"116'6","reported_seq":45,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.159310+0000","last_change":"2024-02-18T00:33:15.159482+0000","last_active":"2024-02-18T00:33:15.159310+0000","last_peered":"2024-02-18T00:33:15.159310+0000","last_clean":"2024-02-18T00:33:15.159310+0000","last_became_active":"2024-02-18T00:33:01.584099+0000","last_became_peered":"2024-02-18T00:33:01.584099+0000","last_unstale":"2024-02-18T00:33:15.159310+0000","last_undegraded":"2024-02-18T00:33:15.159310+0000","last_fullsized":"2024-02-18T00:33:15.159310+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.13","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141440+0000","last_change":"2024-02-18T00:33:01.585846+0000","last_active":"2024-02-18T00:33:14.141440+0000","last_peered":"2024-02-18T00:33:14.141440+0000","last_clean":"2024-02-18T00:33:14.141440+0000","last_became_active":"2024-02-18T00:33:01.584536+0000","last_became_peered":"2024-02-18T00:33:01.584536+0000","last_unstale":"2024-02-18T00:33:14.141440+0000","last_undegraded":"2024-02-18T00:33:14.141440+0000","last_fullsized":"2024-02-18T00:33:14.141440+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1b","version":"109'10","reported_seq":30,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124137+0000","last_change":"2024-02-18T00:33:02.509211+0000","last_active":"2024-02-18T00:33:14.124137+0000","last_peered":"2024-02-18T00:33:14.124137+0000","last_clean":"2024-02-18T00:33:14.124137+0000","last_became_active":"2024-02-18T00:33:02.508134+0000","last_became_peered":"2024-02-18T00:33:02.508134+0000","last_unstale":"2024-02-18T00:33:14.124137+0000","last_undegraded":"2024-02-18T00:33:14.124137+0000","last_fullsized":"2024-02-18T00:33:14.124137+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102108+0000","last_change":"2024-02-18T00:33:04.592626+0000","last_active":"2024-02-18T00:33:14.102108+0000","last_peered":"2024-02-18T00:33:14.102108+0000","last_clean":"2024-02-18T00:33:14.102108+0000","last_became_active":"2024-02-18T00:33:04.591598+0000","last_became_peered":"2024-02-18T00:33:04.591598+0000","last_unstale":"2024-02-18T00:33:14.102108+0000","last_undegraded":"2024-02-18T00:33:14.102108+0000","last_fullsized":"2024-02-18T00:33:14.102108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.112411+0000","last_change":"2024-02-18T00:32:43.644999+0000","last_active":"2024-02-18T00:33:14.112411+0000","last_peered":"2024-02-18T00:33:14.112411+0000","last_clean":"2024-02-18T00:33:14.112411+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:33:14.112411+0000","last_undegraded":"2024-02-18T00:33:14.112411+0000","last_fullsized":"2024-02-18T00:33:14.112411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.644921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":105,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125539+0000","last_change":"2024-02-18T00:32:44.353654+0000","last_active":"2024-02-18T00:33:14.125539+0000","last_peered":"2024-02-18T00:33:14.125539+0000","last_clean":"2024-02-18T00:33:14.125539+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:33:14.125539+0000","last_undegraded":"2024-02-18T00:33:14.125539+0000","last_fullsized":"2024-02-18T00:33:14.125539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:32:44.353571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.353571+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103540+0000","last_change":"2024-02-18T00:32:54.632979+0000","last_active":"2024-02-18T00:33:14.103540+0000","last_peered":"2024-02-18T00:33:14.103540+0000","last_clean":"2024-02-18T00:33:14.103540+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:33:14.103540+0000","last_undegraded":"2024-02-18T00:33:14.103540+0000","last_fullsized":"2024-02-18T00:33:14.103540+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.632911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.632911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117212+0000","last_change":"2024-02-18T00:33:04.697173+0000","last_active":"2024-02-18T00:33:14.117212+0000","last_peered":"2024-02-18T00:33:14.117212+0000","last_clean":"2024-02-18T00:33:14.117212+0000","last_became_active":"2024-02-18T00:33:04.696891+0000","last_became_peered":"2024-02-18T00:33:04.696891+0000","last_unstale":"2024-02-18T00:33:14.117212+0000","last_undegraded":"2024-02-18T00:33:14.117212+0000","last_fullsized":"2024-02-18T00:33:14.117212+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.131806+0000","last_change":"2024-02-18T00:33:02.549194+0000","last_active":"2024-02-18T00:33:14.131806+0000","last_peered":"2024-02-18T00:33:14.131806+0000","last_clean":"2024-02-18T00:33:14.131806+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:33:14.131806+0000","last_undegraded":"2024-02-18T00:33:14.131806+0000","last_fullsized":"2024-02-18T00:33:14.131806+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:02.549081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.549081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115988+0000","last_change":"2024-02-18T00:31:46.851657+0000","last_active":"2024-02-18T00:33:14.115988+0000","last_peered":"2024-02-18T00:33:14.115988+0000","last_clean":"2024-02-18T00:33:14.115988+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:33:14.115988+0000","last_undegraded":"2024-02-18T00:33:14.115988+0000","last_fullsized":"2024-02-18T00:33:14.115988+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.851573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.127301+0000","last_change":"2024-02-18T00:33:15.127484+0000","last_active":"2024-02-18T00:33:15.127301+0000","last_peered":"2024-02-18T00:33:15.127301+0000","last_clean":"2024-02-18T00:33:15.127301+0000","last_became_active":"2024-02-18T00:33:01.451681+0000","last_became_peered":"2024-02-18T00:33:01.451681+0000","last_unstale":"2024-02-18T00:33:15.127301+0000","last_undegraded":"2024-02-18T00:33:15.127301+0000","last_fullsized":"2024-02-18T00:33:15.127301+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.10","version":"108'3","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117809+0000","last_change":"2024-02-18T00:33:01.534477+0000","last_active":"2024-02-18T00:33:14.117809+0000","last_peered":"2024-02-18T00:33:14.117809+0000","last_clean":"2024-02-18T00:33:14.117809+0000","last_became_active":"2024-02-18T00:33:01.514058+0000","last_became_peered":"2024-02-18T00:33:01.514058+0000","last_unstale":"2024-02-18T00:33:14.117809+0000","last_undegraded":"2024-02-18T00:33:14.117809+0000","last_fullsized":"2024-02-18T00:33:14.117809+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118772+0000","last_change":"2024-02-18T00:33:02.511940+0000","last_active":"2024-02-18T00:33:14.118772+0000","last_peered":"2024-02-18T00:33:14.118772+0000","last_clean":"2024-02-18T00:33:14.118772+0000","last_became_active":"2024-02-18T00:33:02.511501+0000","last_became_peered":"2024-02-18T00:33:02.511501+0000","last_unstale":"2024-02-18T00:33:14.118772+0000","last_undegraded":"2024-02-18T00:33:14.118772+0000","last_fullsized":"2024-02-18T00:33:14.118772+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108387+0000","last_change":"2024-02-18T00:33:04.649428+0000","last_active":"2024-02-18T00:33:14.108387+0000","last_peered":"2024-02-18T00:33:14.108387+0000","last_clean":"2024-02-18T00:33:14.108387+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:33:14.108387+0000","last_undegraded":"2024-02-18T00:33:14.108387+0000","last_fullsized":"2024-02-18T00:33:14.108387+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118423+0000","last_change":"2024-02-18T00:31:47.502734+0000","last_active":"2024-02-18T00:33:14.118423+0000","last_peered":"2024-02-18T00:33:14.118423+0000","last_clean":"2024-02-18T00:33:14.118423+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:33:14.118423+0000","last_undegraded":"2024-02-18T00:33:14.118423+0000","last_fullsized":"2024-02-18T00:33:14.118423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:47.502660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:47.502660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":120,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138933+0000","last_change":"2024-02-18T00:31:55.005638+0000","last_active":"2024-02-18T00:33:14.138933+0000","last_peered":"2024-02-18T00:33:14.138933+0000","last_clean":"2024-02-18T00:33:14.138933+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:33:14.138933+0000","last_undegraded":"2024-02-18T00:33:14.138933+0000","last_fullsized":"2024-02-18T00:33:14.138933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:31:55.005563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.005563+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.099698+0000","last_change":"2024-02-18T00:33:04.665195+0000","last_active":"2024-02-18T00:33:14.099698+0000","last_peered":"2024-02-18T00:33:14.099698+0000","last_clean":"2024-02-18T00:33:14.099698+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:33:14.099698+0000","last_undegraded":"2024-02-18T00:33:14.099698+0000","last_fullsized":"2024-02-18T00:33:14.099698+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.521630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.521630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":315,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.121344+0000","last_change":"2024-02-18T00:31:58.945520+0000","last_active":"2024-02-18T00:33:15.121344+0000","last_peered":"2024-02-18T00:33:15.121344+0000","last_clean":"2024-02-18T00:33:15.121344+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:33:15.121344+0000","last_undegraded":"2024-02-18T00:33:15.121344+0000","last_fullsized":"2024-02-18T00:33:15.121344+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:31:58.945452+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.945452+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":232,"num_read_kb":227,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107241+0000","last_change":"2024-02-18T00:32:55.588834+0000","last_active":"2024-02-18T00:33:14.107241+0000","last_peered":"2024-02-18T00:33:14.107241+0000","last_clean":"2024-02-18T00:33:14.107241+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:33:14.107241+0000","last_undegraded":"2024-02-18T00:33:14.107241+0000","last_fullsized":"2024-02-18T00:33:14.107241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.588750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.136674+0000","last_change":"2024-02-18T00:33:15.136856+0000","last_active":"2024-02-18T00:33:15.136674+0000","last_peered":"2024-02-18T00:33:15.136674+0000","last_clean":"2024-02-18T00:33:15.136674+0000","last_became_active":"2024-02-18T00:33:01.436093+0000","last_became_peered":"2024-02-18T00:33:01.436093+0000","last_unstale":"2024-02-18T00:33:15.136674+0000","last_undegraded":"2024-02-18T00:33:15.136674+0000","last_fullsized":"2024-02-18T00:33:15.136674+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.12","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.145855+0000","last_change":"2024-02-18T00:33:01.553808+0000","last_active":"2024-02-18T00:33:14.145855+0000","last_peered":"2024-02-18T00:33:14.145855+0000","last_clean":"2024-02-18T00:33:14.145855+0000","last_became_active":"2024-02-18T00:33:01.551635+0000","last_became_peered":"2024-02-18T00:33:01.551635+0000","last_unstale":"2024-02-18T00:33:14.145855+0000","last_undegraded":"2024-02-18T00:33:14.145855+0000","last_fullsized":"2024-02-18T00:33:14.145855+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122571+0000","last_change":"2024-02-18T00:33:02.487192+0000","last_active":"2024-02-18T00:33:14.122571+0000","last_peered":"2024-02-18T00:33:14.122571+0000","last_clean":"2024-02-18T00:33:14.122571+0000","last_became_active":"2024-02-18T00:33:02.486911+0000","last_became_peered":"2024-02-18T00:33:02.486911+0000","last_unstale":"2024-02-18T00:33:14.122571+0000","last_undegraded":"2024-02-18T00:33:14.122571+0000","last_fullsized":"2024-02-18T00:33:14.122571+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125054+0000","last_change":"2024-02-18T00:33:02.499565+0000","last_active":"2024-02-18T00:33:14.125054+0000","last_peered":"2024-02-18T00:33:14.125054+0000","last_clean":"2024-02-18T00:33:14.125054+0000","last_became_active":"2024-02-18T00:33:02.499219+0000","last_became_peered":"2024-02-18T00:33:02.499219+0000","last_unstale":"2024-02-18T00:33:14.125054+0000","last_undegraded":"2024-02-18T00:33:14.125054+0000","last_fullsized":"2024-02-18T00:33:14.125054+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117630+0000","last_change":"2024-02-18T00:32:07.008657+0000","last_active":"2024-02-18T00:33:14.117630+0000","last_peered":"2024-02-18T00:33:14.117630+0000","last_clean":"2024-02-18T00:33:14.117630+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:33:14.117630+0000","last_undegraded":"2024-02-18T00:33:14.117630+0000","last_fullsized":"2024-02-18T00:33:14.117630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.008543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.008543+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":189,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127972+0000","last_change":"2024-02-18T00:32:04.954445+0000","last_active":"2024-02-18T00:33:14.127972+0000","last_peered":"2024-02-18T00:33:14.127972+0000","last_clean":"2024-02-18T00:33:14.127972+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:33:14.127972+0000","last_undegraded":"2024-02-18T00:33:14.127972+0000","last_fullsized":"2024-02-18T00:33:14.127972+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:32:04.954359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.954359+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117182+0000","last_change":"2024-02-18T00:33:04.569398+0000","last_active":"2024-02-18T00:33:14.117182+0000","last_peered":"2024-02-18T00:33:14.117182+0000","last_clean":"2024-02-18T00:33:14.117182+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:33:14.117182+0000","last_undegraded":"2024-02-18T00:33:14.117182+0000","last_fullsized":"2024-02-18T00:33:14.117182+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.991759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.991759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140127+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:33:14.140127+0000","last_peered":"2024-02-18T00:33:14.140127+0000","last_clean":"2024-02-18T00:33:14.140127+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:33:14.140127+0000","last_undegraded":"2024-02-18T00:33:14.140127+0000","last_fullsized":"2024-02-18T00:33:14.140127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.122147+0000","last_change":"2024-02-18T00:33:15.122853+0000","last_active":"2024-02-18T00:33:15.122147+0000","last_peered":"2024-02-18T00:33:15.122147+0000","last_clean":"2024-02-18T00:33:15.122147+0000","last_became_active":"2024-02-18T00:33:01.426568+0000","last_became_peered":"2024-02-18T00:33:01.426568+0000","last_unstale":"2024-02-18T00:33:15.122147+0000","last_undegraded":"2024-02-18T00:33:15.122147+0000","last_fullsized":"2024-02-18T00:33:15.122147+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.11","version":"108'3","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127966+0000","last_change":"2024-02-18T00:33:01.502827+0000","last_active":"2024-02-18T00:33:14.127966+0000","last_peered":"2024-02-18T00:33:14.127966+0000","last_clean":"2024-02-18T00:33:14.127966+0000","last_became_active":"2024-02-18T00:33:01.502464+0000","last_became_peered":"2024-02-18T00:33:01.502464+0000","last_unstale":"2024-02-18T00:33:14.127966+0000","last_undegraded":"2024-02-18T00:33:14.127966+0000","last_fullsized":"2024-02-18T00:33:14.127966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165843,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107783+0000","last_change":"2024-02-18T00:33:02.522197+0000","last_active":"2024-02-18T00:33:14.107783+0000","last_peered":"2024-02-18T00:33:14.107783+0000","last_clean":"2024-02-18T00:33:14.107783+0000","last_became_active":"2024-02-18T00:33:02.521934+0000","last_became_peered":"2024-02-18T00:33:02.521934+0000","last_unstale":"2024-02-18T00:33:14.107783+0000","last_undegraded":"2024-02-18T00:33:14.107783+0000","last_fullsized":"2024-02-18T00:33:14.107783+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115432+0000","last_change":"2024-02-18T00:33:04.719631+0000","last_active":"2024-02-18T00:33:14.115432+0000","last_peered":"2024-02-18T00:33:14.115432+0000","last_clean":"2024-02-18T00:33:14.115432+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:33:14.115432+0000","last_undegraded":"2024-02-18T00:33:14.115432+0000","last_fullsized":"2024-02-18T00:33:14.115432+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":131,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127293+0000","last_change":"2024-02-18T00:32:05.026757+0000","last_active":"2024-02-18T00:33:14.127293+0000","last_peered":"2024-02-18T00:33:14.127293+0000","last_clean":"2024-02-18T00:33:14.127293+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:33:14.127293+0000","last_undegraded":"2024-02-18T00:33:14.127293+0000","last_fullsized":"2024-02-18T00:33:14.127293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:05.026652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.026652+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114106+0000","last_change":"2024-02-18T00:32:49.398933+0000","last_active":"2024-02-18T00:33:14.114106+0000","last_peered":"2024-02-18T00:33:14.114106+0000","last_clean":"2024-02-18T00:33:14.114106+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:33:14.114106+0000","last_undegraded":"2024-02-18T00:33:14.114106+0000","last_fullsized":"2024-02-18T00:33:14.114106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.398867+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128049+0000","last_change":"2024-02-18T00:33:04.676887+0000","last_active":"2024-02-18T00:33:14.128049+0000","last_peered":"2024-02-18T00:33:14.128049+0000","last_clean":"2024-02-18T00:33:14.128049+0000","last_became_active":"2024-02-18T00:33:04.676371+0000","last_became_peered":"2024-02-18T00:33:04.676371+0000","last_unstale":"2024-02-18T00:33:14.128049+0000","last_undegraded":"2024-02-18T00:33:14.128049+0000","last_fullsized":"2024-02-18T00:33:14.128049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.839018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.839018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.101568+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:33:14.101568+0000","last_peered":"2024-02-18T00:33:14.101568+0000","last_clean":"2024-02-18T00:33:14.101568+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:33:14.101568+0000","last_undegraded":"2024-02-18T00:33:14.101568+0000","last_fullsized":"2024-02-18T00:33:14.101568+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.117302+0000","last_change":"2024-02-18T00:33:15.117472+0000","last_active":"2024-02-18T00:33:15.117302+0000","last_peered":"2024-02-18T00:33:15.117302+0000","last_clean":"2024-02-18T00:33:15.117302+0000","last_became_active":"2024-02-18T00:33:01.580618+0000","last_became_peered":"2024-02-18T00:33:01.580618+0000","last_unstale":"2024-02-18T00:33:15.117302+0000","last_undegraded":"2024-02-18T00:33:15.117302+0000","last_fullsized":"2024-02-18T00:33:15.117302+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.17","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135693+0000","last_change":"2024-02-18T00:33:01.507106+0000","last_active":"2024-02-18T00:33:14.135693+0000","last_peered":"2024-02-18T00:33:14.135693+0000","last_clean":"2024-02-18T00:33:14.135693+0000","last_became_active":"2024-02-18T00:33:01.506457+0000","last_became_peered":"2024-02-18T00:33:01.506457+0000","last_unstale":"2024-02-18T00:33:14.135693+0000","last_undegraded":"2024-02-18T00:33:14.135693+0000","last_fullsized":"2024-02-18T00:33:14.135693+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138408+0000","last_change":"2024-02-18T00:33:02.484120+0000","last_active":"2024-02-18T00:33:14.138408+0000","last_peered":"2024-02-18T00:33:14.138408+0000","last_clean":"2024-02-18T00:33:14.138408+0000","last_became_active":"2024-02-18T00:33:02.483807+0000","last_became_peered":"2024-02-18T00:33:02.483807+0000","last_unstale":"2024-02-18T00:33:14.138408+0000","last_undegraded":"2024-02-18T00:33:14.138408+0000","last_fullsized":"2024-02-18T00:33:14.138408+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130824+0000","last_change":"2024-02-18T00:33:02.483549+0000","last_active":"2024-02-18T00:33:14.130824+0000","last_peered":"2024-02-18T00:33:14.130824+0000","last_clean":"2024-02-18T00:33:14.130824+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:33:14.130824+0000","last_undegraded":"2024-02-18T00:33:14.130824+0000","last_fullsized":"2024-02-18T00:33:14.130824+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102657+0000","last_change":"2024-02-18T00:32:40.476946+0000","last_active":"2024-02-18T00:33:14.102657+0000","last_peered":"2024-02-18T00:33:14.102657+0000","last_clean":"2024-02-18T00:33:14.102657+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:33:14.102657+0000","last_undegraded":"2024-02-18T00:33:14.102657+0000","last_fullsized":"2024-02-18T00:33:14.102657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:40.476846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.476846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":131,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122777+0000","last_change":"2024-02-18T00:32:43.018921+0000","last_active":"2024-02-18T00:33:14.122777+0000","last_peered":"2024-02-18T00:33:14.122777+0000","last_clean":"2024-02-18T00:33:14.122777+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:33:14.122777+0000","last_undegraded":"2024-02-18T00:33:14.122777+0000","last_fullsized":"2024-02-18T00:33:14.122777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:32:43.018841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.018841+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":98,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.136148+0000","last_change":"2024-02-18T00:33:03.047192+0000","last_active":"2024-02-18T00:33:14.136148+0000","last_peered":"2024-02-18T00:33:14.136148+0000","last_clean":"2024-02-18T00:33:14.136148+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:33:14.136148+0000","last_undegraded":"2024-02-18T00:33:14.136148+0000","last_fullsized":"2024-02-18T00:33:14.136148+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:33:03.047128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.047128+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140656+0000","last_change":"2024-02-18T00:31:52.444880+0000","last_active":"2024-02-18T00:33:14.140656+0000","last_peered":"2024-02-18T00:33:14.140656+0000","last_clean":"2024-02-18T00:33:14.140656+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:33:14.140656+0000","last_undegraded":"2024-02-18T00:33:14.140656+0000","last_fullsized":"2024-02-18T00:33:14.140656+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:52.444811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:52.444811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.124843+0000","last_change":"2024-02-18T00:33:15.125068+0000","last_active":"2024-02-18T00:33:15.124843+0000","last_peered":"2024-02-18T00:33:15.124843+0000","last_clean":"2024-02-18T00:33:15.124843+0000","last_became_active":"2024-02-18T00:33:01.569919+0000","last_became_peered":"2024-02-18T00:33:01.569919+0000","last_unstale":"2024-02-18T00:33:15.124843+0000","last_undegraded":"2024-02-18T00:33:15.124843+0000","last_fullsized":"2024-02-18T00:33:15.124843+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.1c","version":"108'4","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127884+0000","last_change":"2024-02-18T00:33:01.489945+0000","last_active":"2024-02-18T00:33:14.127884+0000","last_peered":"2024-02-18T00:33:14.127884+0000","last_clean":"2024-02-18T00:33:14.127884+0000","last_became_active":"2024-02-18T00:33:01.488576+0000","last_became_peered":"2024-02-18T00:33:01.488576+0000","last_unstale":"2024-02-18T00:33:14.127884+0000","last_undegraded":"2024-02-18T00:33:14.127884+0000","last_fullsized":"2024-02-18T00:33:14.127884+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165835,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"109'2","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114923+0000","last_change":"2024-02-18T00:33:02.493422+0000","last_active":"2024-02-18T00:33:14.114923+0000","last_peered":"2024-02-18T00:33:14.114923+0000","last_clean":"2024-02-18T00:33:14.114923+0000","last_became_active":"2024-02-18T00:33:02.492817+0000","last_became_peered":"2024-02-18T00:33:02.492817+0000","last_unstale":"2024-02-18T00:33:14.114923+0000","last_undegraded":"2024-02-18T00:33:14.114923+0000","last_fullsized":"2024-02-18T00:33:14.114923+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124785+0000","last_change":"2024-02-18T00:33:02.533610+0000","last_active":"2024-02-18T00:33:14.124785+0000","last_peered":"2024-02-18T00:33:14.124785+0000","last_clean":"2024-02-18T00:33:14.124785+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:33:14.124785+0000","last_undegraded":"2024-02-18T00:33:14.124785+0000","last_fullsized":"2024-02-18T00:33:14.124785+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":123,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107119+0000","last_change":"2024-02-18T00:33:03.042961+0000","last_active":"2024-02-18T00:33:14.107119+0000","last_peered":"2024-02-18T00:33:14.107119+0000","last_clean":"2024-02-18T00:33:14.107119+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:33:14.107119+0000","last_undegraded":"2024-02-18T00:33:14.107119+0000","last_fullsized":"2024-02-18T00:33:14.107119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:33:03.042870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.042870+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103216+0000","last_change":"2024-02-18T00:32:05.449395+0000","last_active":"2024-02-18T00:33:14.103216+0000","last_peered":"2024-02-18T00:33:14.103216+0000","last_clean":"2024-02-18T00:33:14.103216+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:33:14.103216+0000","last_undegraded":"2024-02-18T00:33:14.103216+0000","last_fullsized":"2024-02-18T00:33:14.103216+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.449286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.449286+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132077+0000","last_change":"2024-02-18T00:31:59.580602+0000","last_active":"2024-02-18T00:33:14.132077+0000","last_peered":"2024-02-18T00:33:14.132077+0000","last_clean":"2024-02-18T00:33:14.132077+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:33:14.132077+0000","last_undegraded":"2024-02-18T00:33:14.132077+0000","last_fullsized":"2024-02-18T00:33:14.132077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:59.580519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:59.580519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108141+0000","last_change":"2024-02-18T00:32:42.462891+0000","last_active":"2024-02-18T00:33:14.108141+0000","last_peered":"2024-02-18T00:33:14.108141+0000","last_clean":"2024-02-18T00:33:14.108141+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:33:14.108141+0000","last_undegraded":"2024-02-18T00:33:14.108141+0000","last_fullsized":"2024-02-18T00:33:14.108141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:42.462777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:32:42.462777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.113587+0000","last_change":"2024-02-18T00:33:15.113999+0000","last_active":"2024-02-18T00:33:15.113587+0000","last_peered":"2024-02-18T00:33:15.113587+0000","last_clean":"2024-02-18T00:33:15.113587+0000","last_became_active":"2024-02-18T00:33:01.554736+0000","last_became_peered":"2024-02-18T00:33:01.554736+0000","last_unstale":"2024-02-18T00:33:15.113587+0000","last_undegraded":"2024-02-18T00:33:15.113587+0000","last_fullsized":"2024-02-18T00:33:15.113587+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.3","version":"108'1","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.124695+0000","last_change":"2024-02-18T00:33:06.020549+0000","last_active":"2024-02-18T00:33:14.124695+0000","last_peered":"2024-02-18T00:33:14.124695+0000","last_clean":"2024-02-18T00:33:14.124695+0000","last_became_active":"2024-02-18T00:33:01.438240+0000","last_became_peered":"2024-02-18T00:33:01.438240+0000","last_unstale":"2024-02-18T00:33:14.124695+0000","last_undegraded":"2024-02-18T00:33:14.124695+0000","last_fullsized":"2024-02-18T00:33:14.124695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'1","last_scrub_stamp":"2024-02-18T00:33:06.020448+0000","last_deep_scrub":"108'1","last_deep_scrub_stamp":"2024-02-18T00:33:06.020448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.020448+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121912+0000","last_change":"2024-02-18T00:33:04.608305+0000","last_active":"2024-02-18T00:33:14.121912+0000","last_peered":"2024-02-18T00:33:14.121912+0000","last_clean":"2024-02-18T00:33:14.121912+0000","last_became_active":"2024-02-18T00:33:04.607683+0000","last_became_peered":"2024-02-18T00:33:04.607683+0000","last_unstale":"2024-02-18T00:33:14.121912+0000","last_undegraded":"2024-02-18T00:33:14.121912+0000","last_fullsized":"2024-02-18T00:33:14.121912+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108029+0000","last_change":"2024-02-18T00:33:04.706011+0000","last_active":"2024-02-18T00:33:14.108029+0000","last_peered":"2024-02-18T00:33:14.108029+0000","last_clean":"2024-02-18T00:33:14.108029+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:33:14.108029+0000","last_undegraded":"2024-02-18T00:33:14.108029+0000","last_fullsized":"2024-02-18T00:33:14.108029+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":121,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.131874+0000","last_change":"2024-02-18T00:31:29.228915+0000","last_active":"2024-02-18T00:33:14.131874+0000","last_peered":"2024-02-18T00:33:14.131874+0000","last_clean":"2024-02-18T00:33:14.131874+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:33:14.131874+0000","last_undegraded":"2024-02-18T00:33:14.131874+0000","last_fullsized":"2024-02-18T00:33:14.131874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:29.228799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:29.228799+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":85,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114957+0000","last_change":"2024-02-18T00:32:58.392505+0000","last_active":"2024-02-18T00:33:14.114957+0000","last_peered":"2024-02-18T00:33:14.114957+0000","last_clean":"2024-02-18T00:33:14.114957+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:33:14.114957+0000","last_undegraded":"2024-02-18T00:33:14.114957+0000","last_fullsized":"2024-02-18T00:33:14.114957+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.392443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135346+0000","last_change":"2024-02-18T00:32:50.948349+0000","last_active":"2024-02-18T00:33:14.135346+0000","last_peered":"2024-02-18T00:33:14.135346+0000","last_clean":"2024-02-18T00:33:14.135346+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:33:14.135346+0000","last_undegraded":"2024-02-18T00:33:14.135346+0000","last_fullsized":"2024-02-18T00:33:14.135346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:50.948240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.948240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106877+0000","last_change":"2024-02-18T00:32:06.740021+0000","last_active":"2024-02-18T00:33:14.106877+0000","last_peered":"2024-02-18T00:33:14.106877+0000","last_clean":"2024-02-18T00:33:14.106877+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:33:14.106877+0000","last_undegraded":"2024-02-18T00:33:14.106877+0000","last_fullsized":"2024-02-18T00:33:14.106877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:06.739943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:06.739943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.114593+0000","last_change":"2024-02-18T00:33:15.114753+0000","last_active":"2024-02-18T00:33:15.114593+0000","last_peered":"2024-02-18T00:33:15.114593+0000","last_clean":"2024-02-18T00:33:15.114593+0000","last_became_active":"2024-02-18T00:33:01.580766+0000","last_became_peered":"2024-02-18T00:33:01.580766+0000","last_unstale":"2024-02-18T00:33:15.114593+0000","last_undegraded":"2024-02-18T00:33:15.114593+0000","last_fullsized":"2024-02-18T00:33:15.114593+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.2","version":"108'8","reported_seq":36,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.129502+0000","last_change":"2024-02-18T00:33:04.987932+0000","last_active":"2024-02-18T00:33:14.129502+0000","last_peered":"2024-02-18T00:33:14.129502+0000","last_clean":"2024-02-18T00:33:14.129502+0000","last_became_active":"2024-02-18T00:33:01.499199+0000","last_became_peered":"2024-02-18T00:33:01.499199+0000","last_unstale":"2024-02-18T00:33:14.129502+0000","last_undegraded":"2024-02-18T00:33:14.129502+0000","last_fullsized":"2024-02-18T00:33:14.129502+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'8","last_scrub_stamp":"2024-02-18T00:33:04.987867+0000","last_deep_scrub":"108'8","last_deep_scrub_stamp":"2024-02-18T00:33:04.987867+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.987867+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3888,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.a","version":"109'10","reported_seq":30,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108242+0000","last_change":"2024-02-18T00:33:04.707926+0000","last_active":"2024-02-18T00:33:14.108242+0000","last_peered":"2024-02-18T00:33:14.108242+0000","last_clean":"2024-02-18T00:33:14.108242+0000","last_became_active":"2024-02-18T00:33:04.701946+0000","last_became_peered":"2024-02-18T00:33:04.701946+0000","last_unstale":"2024-02-18T00:33:14.108242+0000","last_undegraded":"2024-02-18T00:33:14.108242+0000","last_fullsized":"2024-02-18T00:33:14.108242+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.101543+0000","last_change":"2024-02-18T00:33:04.730258+0000","last_active":"2024-02-18T00:33:14.101543+0000","last_peered":"2024-02-18T00:33:14.101543+0000","last_clean":"2024-02-18T00:33:14.101543+0000","last_became_active":"2024-02-18T00:33:04.729050+0000","last_became_peered":"2024-02-18T00:33:04.729050+0000","last_unstale":"2024-02-18T00:33:14.101543+0000","last_undegraded":"2024-02-18T00:33:14.101543+0000","last_fullsized":"2024-02-18T00:33:14.101543+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":123,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.143051+0000","last_change":"2024-02-18T00:32:02.547561+0000","last_active":"2024-02-18T00:33:14.143051+0000","last_peered":"2024-02-18T00:33:14.143051+0000","last_clean":"2024-02-18T00:33:14.143051+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:33:14.143051+0000","last_undegraded":"2024-02-18T00:33:14.143051+0000","last_fullsized":"2024-02-18T00:33:14.143051+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:32:02.547447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.547447+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.111000+0000","last_change":"2024-02-18T00:32:55.976686+0000","last_active":"2024-02-18T00:33:14.111000+0000","last_peered":"2024-02-18T00:33:14.111000+0000","last_clean":"2024-02-18T00:33:14.111000+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:33:14.111000+0000","last_undegraded":"2024-02-18T00:33:14.111000+0000","last_fullsized":"2024-02-18T00:33:14.111000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:55.976578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.976578+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":132,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141459+0000","last_change":"2024-02-18T00:32:36.948567+0000","last_active":"2024-02-18T00:33:14.141459+0000","last_peered":"2024-02-18T00:33:14.141459+0000","last_clean":"2024-02-18T00:33:14.141459+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:33:14.141459+0000","last_undegraded":"2024-02-18T00:33:14.141459+0000","last_fullsized":"2024-02-18T00:33:14.141459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:36.948476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.948476+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106897+0000","last_change":"2024-02-18T00:31:37.150285+0000","last_active":"2024-02-18T00:33:14.106897+0000","last_peered":"2024-02-18T00:33:14.106897+0000","last_clean":"2024-02-18T00:33:14.106897+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:33:14.106897+0000","last_undegraded":"2024-02-18T00:33:14.106897+0000","last_fullsized":"2024-02-18T00:33:14.106897+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.150215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.122742+0000","last_change":"2024-02-18T00:33:15.122888+0000","last_active":"2024-02-18T00:33:15.122742+0000","last_peered":"2024-02-18T00:33:15.122742+0000","last_clean":"2024-02-18T00:33:15.122742+0000","last_became_active":"2024-02-18T00:33:01.469083+0000","last_became_peered":"2024-02-18T00:33:01.469083+0000","last_unstale":"2024-02-18T00:33:15.122742+0000","last_undegraded":"2024-02-18T00:33:15.122742+0000","last_fullsized":"2024-02-18T00:33:15.122742+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.1","version":"106'2","reported_seq":30,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108654+0000","last_change":"2024-02-18T00:33:05.970604+0000","last_active":"2024-02-18T00:33:14.108654+0000","last_peered":"2024-02-18T00:33:14.108654+0000","last_clean":"2024-02-18T00:33:14.108654+0000","last_became_active":"2024-02-18T00:33:01.577813+0000","last_became_peered":"2024-02-18T00:33:01.577813+0000","last_unstale":"2024-02-18T00:33:14.108654+0000","last_undegraded":"2024-02-18T00:33:14.108654+0000","last_fullsized":"2024-02-18T00:33:14.108654+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'2","last_scrub_stamp":"2024-02-18T00:33:05.970532+0000","last_deep_scrub":"106'2","last_deep_scrub_stamp":"2024-02-18T00:33:05.970532+0000","last_clean_scrub_stamp":"2024-02-18T00:33:05.970532+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"109'2","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140356+0000","last_change":"2024-02-18T00:33:04.682773+0000","last_active":"2024-02-18T00:33:14.140356+0000","last_peered":"2024-02-18T00:33:14.140356+0000","last_clean":"2024-02-18T00:33:14.140356+0000","last_became_active":"2024-02-18T00:33:04.679723+0000","last_became_peered":"2024-02-18T00:33:04.679723+0000","last_unstale":"2024-02-18T00:33:14.140356+0000","last_undegraded":"2024-02-18T00:33:14.140356+0000","last_fullsized":"2024-02-18T00:33:14.140356+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125765+0000","last_change":"2024-02-18T00:33:02.431426+0000","last_active":"2024-02-18T00:33:14.125765+0000","last_peered":"2024-02-18T00:33:14.125765+0000","last_clean":"2024-02-18T00:33:14.125765+0000","last_became_active":"2024-02-18T00:33:02.430642+0000","last_became_peered":"2024-02-18T00:33:02.430642+0000","last_unstale":"2024-02-18T00:33:14.125765+0000","last_undegraded":"2024-02-18T00:33:14.125765+0000","last_fullsized":"2024-02-18T00:33:14.125765+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141234+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:33:14.141234+0000","last_peered":"2024-02-18T00:33:14.141234+0000","last_clean":"2024-02-18T00:33:14.141234+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:33:14.141234+0000","last_undegraded":"2024-02-18T00:33:14.141234+0000","last_fullsized":"2024-02-18T00:33:14.141234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127120+0000","last_change":"2024-02-18T00:31:41.112608+0000","last_active":"2024-02-18T00:33:14.127120+0000","last_peered":"2024-02-18T00:33:14.127120+0000","last_clean":"2024-02-18T00:33:14.127120+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:33:14.127120+0000","last_undegraded":"2024-02-18T00:33:14.127120+0000","last_fullsized":"2024-02-18T00:33:14.127120+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.112540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.112540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":132,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.143250+0000","last_change":"2024-02-18T00:32:01.547864+0000","last_active":"2024-02-18T00:33:14.143250+0000","last_peered":"2024-02-18T00:33:14.143250+0000","last_clean":"2024-02-18T00:33:14.143250+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:33:14.143250+0000","last_undegraded":"2024-02-18T00:33:14.143250+0000","last_fullsized":"2024-02-18T00:33:14.143250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:01.547763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:01.547763+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.136462+0000","last_change":"2024-02-18T00:32:53.114523+0000","last_active":"2024-02-18T00:33:14.136462+0000","last_peered":"2024-02-18T00:33:14.136462+0000","last_clean":"2024-02-18T00:33:14.136462+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:33:14.136462+0000","last_undegraded":"2024-02-18T00:33:14.136462+0000","last_fullsized":"2024-02-18T00:33:14.136462+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.114424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.114424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.131527+0000","last_change":"2024-02-18T00:33:15.132184+0000","last_active":"2024-02-18T00:33:15.131527+0000","last_peered":"2024-02-18T00:33:15.131527+0000","last_clean":"2024-02-18T00:33:15.131527+0000","last_became_active":"2024-02-18T00:33:01.555569+0000","last_became_peered":"2024-02-18T00:33:01.555569+0000","last_unstale":"2024-02-18T00:33:15.131527+0000","last_undegraded":"2024-02-18T00:33:15.131527+0000","last_fullsized":"2024-02-18T00:33:15.131527+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.0","version":"106'1","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102854+0000","last_change":"2024-02-18T00:33:05.821752+0000","last_active":"2024-02-18T00:33:14.102854+0000","last_peered":"2024-02-18T00:33:14.102854+0000","last_clean":"2024-02-18T00:33:14.102854+0000","last_became_active":"2024-02-18T00:33:01.406671+0000","last_became_peered":"2024-02-18T00:33:01.406671+0000","last_unstale":"2024-02-18T00:33:14.102854+0000","last_undegraded":"2024-02-18T00:33:14.102854+0000","last_fullsized":"2024-02-18T00:33:14.102854+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'1","last_scrub_stamp":"2024-02-18T00:33:05.821674+0000","last_deep_scrub":"106'1","last_deep_scrub_stamp":"2024-02-18T00:33:05.821674+0000","last_clean_scrub_stamp":"2024-02-18T00:33:05.821674+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108035+0000","last_change":"2024-02-18T00:33:04.704619+0000","last_active":"2024-02-18T00:33:14.108035+0000","last_peered":"2024-02-18T00:33:14.108035+0000","last_clean":"2024-02-18T00:33:14.108035+0000","last_became_active":"2024-02-18T00:33:04.703531+0000","last_became_peered":"2024-02-18T00:33:04.703531+0000","last_unstale":"2024-02-18T00:33:14.108035+0000","last_undegraded":"2024-02-18T00:33:14.108035+0000","last_fullsized":"2024-02-18T00:33:14.108035+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.116996+0000","last_change":"2024-02-18T00:33:02.532649+0000","last_active":"2024-02-18T00:33:14.116996+0000","last_peered":"2024-02-18T00:33:14.116996+0000","last_clean":"2024-02-18T00:33:14.116996+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:33:14.116996+0000","last_undegraded":"2024-02-18T00:33:14.116996+0000","last_fullsized":"2024-02-18T00:33:14.116996+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":94,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122895+0000","last_change":"2024-02-18T00:32:37.815524+0000","last_active":"2024-02-18T00:33:14.122895+0000","last_peered":"2024-02-18T00:33:14.122895+0000","last_clean":"2024-02-18T00:33:14.122895+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:33:14.122895+0000","last_undegraded":"2024-02-18T00:33:14.122895+0000","last_fullsized":"2024-02-18T00:33:14.122895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:32:37.815411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:37.815411+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":85,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127789+0000","last_change":"2024-02-18T00:32:52.323010+0000","last_active":"2024-02-18T00:33:14.127789+0000","last_peered":"2024-02-18T00:33:14.127789+0000","last_clean":"2024-02-18T00:33:14.127789+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:33:14.127789+0000","last_undegraded":"2024-02-18T00:33:14.127789+0000","last_fullsized":"2024-02-18T00:33:14.127789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:52.322930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:52.322930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140582+0000","last_change":"2024-02-18T00:31:57.501367+0000","last_active":"2024-02-18T00:33:14.140582+0000","last_peered":"2024-02-18T00:33:14.140582+0000","last_clean":"2024-02-18T00:33:14.140582+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:33:14.140582+0000","last_undegraded":"2024-02-18T00:33:14.140582+0000","last_fullsized":"2024-02-18T00:33:14.140582+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.501298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.501298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.101374+0000","last_change":"2024-02-18T00:33:01.075249+0000","last_active":"2024-02-18T00:33:14.101374+0000","last_peered":"2024-02-18T00:33:14.101374+0000","last_clean":"2024-02-18T00:33:14.101374+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:33:14.101374+0000","last_undegraded":"2024-02-18T00:33:14.101374+0000","last_fullsized":"2024-02-18T00:33:14.101374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.075169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.075169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.120660+0000","last_change":"2024-02-18T00:33:15.121695+0000","last_active":"2024-02-18T00:33:15.120660+0000","last_peered":"2024-02-18T00:33:15.120660+0000","last_clean":"2024-02-18T00:33:15.120660+0000","last_became_active":"2024-02-18T00:33:01.537343+0000","last_became_peered":"2024-02-18T00:33:01.537343+0000","last_unstale":"2024-02-18T00:33:15.120660+0000","last_undegraded":"2024-02-18T00:33:15.120660+0000","last_fullsized":"2024-02-18T00:33:15.120660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.7","version":"0'0","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114668+0000","last_change":"2024-02-18T00:33:10.999268+0000","last_active":"2024-02-18T00:33:14.114668+0000","last_peered":"2024-02-18T00:33:14.114668+0000","last_clean":"2024-02-18T00:33:14.114668+0000","last_became_active":"2024-02-18T00:33:01.567237+0000","last_became_peered":"2024-02-18T00:33:01.567237+0000","last_unstale":"2024-02-18T00:33:14.114668+0000","last_undegraded":"2024-02-18T00:33:14.114668+0000","last_fullsized":"2024-02-18T00:33:14.114668+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:10.999164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:10.999164+0000","last_clean_scrub_stamp":"2024-02-18T00:33:10.999164+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138772+0000","last_change":"2024-02-18T00:33:04.600039+0000","last_active":"2024-02-18T00:33:14.138772+0000","last_peered":"2024-02-18T00:33:14.138772+0000","last_clean":"2024-02-18T00:33:14.138772+0000","last_became_active":"2024-02-18T00:33:04.599784+0000","last_became_peered":"2024-02-18T00:33:04.599784+0000","last_unstale":"2024-02-18T00:33:14.138772+0000","last_undegraded":"2024-02-18T00:33:14.138772+0000","last_fullsized":"2024-02-18T00:33:14.138772+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106815+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:33:14.106815+0000","last_peered":"2024-02-18T00:33:14.106815+0000","last_clean":"2024-02-18T00:33:14.106815+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:33:14.106815+0000","last_undegraded":"2024-02-18T00:33:14.106815+0000","last_fullsized":"2024-02-18T00:33:14.106815+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":97,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125420+0000","last_change":"2024-02-18T00:31:54.074682+0000","last_active":"2024-02-18T00:33:14.125420+0000","last_peered":"2024-02-18T00:33:14.125420+0000","last_clean":"2024-02-18T00:33:14.125420+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:33:14.125420+0000","last_undegraded":"2024-02-18T00:33:14.125420+0000","last_fullsized":"2024-02-18T00:33:14.125420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:31:54.074589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.074589+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127858+0000","last_change":"2024-02-18T00:32:40.792705+0000","last_active":"2024-02-18T00:33:14.127858+0000","last_peered":"2024-02-18T00:33:14.127858+0000","last_clean":"2024-02-18T00:33:14.127858+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:33:14.127858+0000","last_undegraded":"2024-02-18T00:33:14.127858+0000","last_fullsized":"2024-02-18T00:33:14.127858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:40.792593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.792593+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.122973+0000","last_change":"2024-02-18T00:32:43.773879+0000","last_active":"2024-02-18T00:33:14.122973+0000","last_peered":"2024-02-18T00:33:14.122973+0000","last_clean":"2024-02-18T00:33:14.122973+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:33:14.122973+0000","last_undegraded":"2024-02-18T00:33:14.122973+0000","last_fullsized":"2024-02-18T00:33:14.122973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.773801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.773801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.126918+0000","last_change":"2024-02-18T00:32:46.709442+0000","last_active":"2024-02-18T00:33:14.126918+0000","last_peered":"2024-02-18T00:33:14.126918+0000","last_clean":"2024-02-18T00:33:14.126918+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:33:14.126918+0000","last_undegraded":"2024-02-18T00:33:14.126918+0000","last_fullsized":"2024-02-18T00:33:14.126918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:46.709314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.709314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"106'1","reported_seq":40,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.123810+0000","last_change":"2024-02-18T00:33:15.124006+0000","last_active":"2024-02-18T00:33:15.123810+0000","last_peered":"2024-02-18T00:33:15.123810+0000","last_clean":"2024-02-18T00:33:15.123810+0000","last_became_active":"2024-02-18T00:33:01.494413+0000","last_became_peered":"2024-02-18T00:33:01.494413+0000","last_unstale":"2024-02-18T00:33:15.123810+0000","last_undegraded":"2024-02-18T00:33:15.123810+0000","last_fullsized":"2024-02-18T00:33:15.123810+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.6","version":"106'1","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.129405+0000","last_change":"2024-02-18T00:33:09.430682+0000","last_active":"2024-02-18T00:33:14.129405+0000","last_peered":"2024-02-18T00:33:14.129405+0000","last_clean":"2024-02-18T00:33:14.129405+0000","last_became_active":"2024-02-18T00:33:01.493762+0000","last_became_peered":"2024-02-18T00:33:01.493762+0000","last_unstale":"2024-02-18T00:33:14.129405+0000","last_undegraded":"2024-02-18T00:33:14.129405+0000","last_fullsized":"2024-02-18T00:33:14.129405+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'1","last_scrub_stamp":"2024-02-18T00:33:09.430597+0000","last_deep_scrub":"106'1","last_deep_scrub_stamp":"2024-02-18T00:33:09.430597+0000","last_clean_scrub_stamp":"2024-02-18T00:33:09.430597+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135508+0000","last_change":"2024-02-18T00:33:02.492453+0000","last_active":"2024-02-18T00:33:14.135508+0000","last_peered":"2024-02-18T00:33:14.135508+0000","last_clean":"2024-02-18T00:33:14.135508+0000","last_became_active":"2024-02-18T00:33:02.491650+0000","last_became_peered":"2024-02-18T00:33:02.491650+0000","last_unstale":"2024-02-18T00:33:14.135508+0000","last_undegraded":"2024-02-18T00:33:14.135508+0000","last_fullsized":"2024-02-18T00:33:14.135508+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.126042+0000","last_change":"2024-02-18T00:33:02.499776+0000","last_active":"2024-02-18T00:33:14.126042+0000","last_peered":"2024-02-18T00:33:14.126042+0000","last_clean":"2024-02-18T00:33:14.126042+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:33:14.126042+0000","last_undegraded":"2024-02-18T00:33:14.126042+0000","last_fullsized":"2024-02-18T00:33:14.126042+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117996+0000","last_change":"2024-02-18T00:32:48.722576+0000","last_active":"2024-02-18T00:33:14.117996+0000","last_peered":"2024-02-18T00:33:14.117996+0000","last_clean":"2024-02-18T00:33:14.117996+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:33:14.117996+0000","last_undegraded":"2024-02-18T00:33:14.117996+0000","last_fullsized":"2024-02-18T00:33:14.117996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:32:48.722501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:48.722501+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135429+0000","last_change":"2024-02-18T00:31:30.789715+0000","last_active":"2024-02-18T00:33:14.135429+0000","last_peered":"2024-02-18T00:33:14.135429+0000","last_clean":"2024-02-18T00:33:14.135429+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:33:14.135429+0000","last_undegraded":"2024-02-18T00:33:14.135429+0000","last_fullsized":"2024-02-18T00:33:14.135429+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.789634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114881+0000","last_change":"2024-02-18T00:32:07.258321+0000","last_active":"2024-02-18T00:33:14.114881+0000","last_peered":"2024-02-18T00:33:14.114881+0000","last_clean":"2024-02-18T00:33:14.114881+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:33:14.114881+0000","last_undegraded":"2024-02-18T00:33:14.114881+0000","last_fullsized":"2024-02-18T00:33:14.114881+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.258239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.258239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.112935+0000","last_change":"2024-02-18T00:32:36.719564+0000","last_active":"2024-02-18T00:33:14.112935+0000","last_peered":"2024-02-18T00:33:14.112935+0000","last_clean":"2024-02-18T00:33:14.112935+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:33:14.112935+0000","last_undegraded":"2024-02-18T00:33:14.112935+0000","last_fullsized":"2024-02-18T00:33:14.112935+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.719484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.719484+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.112821+0000","last_change":"2024-02-18T00:33:15.113499+0000","last_active":"2024-02-18T00:33:15.112821+0000","last_peered":"2024-02-18T00:33:15.112821+0000","last_clean":"2024-02-18T00:33:15.112821+0000","last_became_active":"2024-02-18T00:33:01.574821+0000","last_became_peered":"2024-02-18T00:33:01.574821+0000","last_unstale":"2024-02-18T00:33:15.112821+0000","last_undegraded":"2024-02-18T00:33:15.112821+0000","last_fullsized":"2024-02-18T00:33:15.112821+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.5","version":"0'0","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125935+0000","last_change":"2024-02-18T00:33:06.364448+0000","last_active":"2024-02-18T00:33:14.125935+0000","last_peered":"2024-02-18T00:33:14.125935+0000","last_clean":"2024-02-18T00:33:14.125935+0000","last_became_active":"2024-02-18T00:33:01.494037+0000","last_became_peered":"2024-02-18T00:33:01.494037+0000","last_unstale":"2024-02-18T00:33:14.125935+0000","last_undegraded":"2024-02-18T00:33:14.125935+0000","last_fullsized":"2024-02-18T00:33:14.125935+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.364359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.364359+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.364359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138402+0000","last_change":"2024-02-18T00:33:02.493509+0000","last_active":"2024-02-18T00:33:14.138402+0000","last_peered":"2024-02-18T00:33:14.138402+0000","last_clean":"2024-02-18T00:33:14.138402+0000","last_became_active":"2024-02-18T00:33:02.492366+0000","last_became_peered":"2024-02-18T00:33:02.492366+0000","last_unstale":"2024-02-18T00:33:14.138402+0000","last_undegraded":"2024-02-18T00:33:14.138402+0000","last_fullsized":"2024-02-18T00:33:14.138402+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.133891+0000","last_change":"2024-02-18T00:33:02.495634+0000","last_active":"2024-02-18T00:33:14.133891+0000","last_peered":"2024-02-18T00:33:14.133891+0000","last_clean":"2024-02-18T00:33:14.133891+0000","last_became_active":"2024-02-18T00:33:02.495368+0000","last_became_peered":"2024-02-18T00:33:02.495368+0000","last_unstale":"2024-02-18T00:33:14.133891+0000","last_undegraded":"2024-02-18T00:33:14.133891+0000","last_fullsized":"2024-02-18T00:33:14.133891+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":112,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127598+0000","last_change":"2024-02-18T00:32:40.775783+0000","last_active":"2024-02-18T00:33:14.127598+0000","last_peered":"2024-02-18T00:33:14.127598+0000","last_clean":"2024-02-18T00:33:14.127598+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:33:14.127598+0000","last_undegraded":"2024-02-18T00:33:14.127598+0000","last_fullsized":"2024-02-18T00:33:14.127598+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:32:40.775672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.775672+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135059+0000","last_change":"2024-02-18T00:31:51.186460+0000","last_active":"2024-02-18T00:33:14.135059+0000","last_peered":"2024-02-18T00:33:14.135059+0000","last_clean":"2024-02-18T00:33:14.135059+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:33:14.135059+0000","last_undegraded":"2024-02-18T00:33:14.135059+0000","last_fullsized":"2024-02-18T00:33:14.135059+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:51.186373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:51.186373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.103958+0000","last_change":"2024-02-18T00:32:07.760043+0000","last_active":"2024-02-18T00:33:14.103958+0000","last_peered":"2024-02-18T00:33:14.103958+0000","last_clean":"2024-02-18T00:33:14.103958+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:33:14.103958+0000","last_undegraded":"2024-02-18T00:33:14.103958+0000","last_fullsized":"2024-02-18T00:33:14.103958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.759964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.759964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.141222+0000","last_change":"2024-02-18T00:32:51.698582+0000","last_active":"2024-02-18T00:33:14.141222+0000","last_peered":"2024-02-18T00:33:14.141222+0000","last_clean":"2024-02-18T00:33:14.141222+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:33:14.141222+0000","last_undegraded":"2024-02-18T00:33:14.141222+0000","last_fullsized":"2024-02-18T00:33:14.141222+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:51.698513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:51.698513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.127051+0000","last_change":"2024-02-18T00:33:15.128083+0000","last_active":"2024-02-18T00:33:15.127051+0000","last_peered":"2024-02-18T00:33:15.127051+0000","last_clean":"2024-02-18T00:33:15.127051+0000","last_became_active":"2024-02-18T00:33:01.417085+0000","last_became_peered":"2024-02-18T00:33:01.417085+0000","last_unstale":"2024-02-18T00:33:15.127051+0000","last_undegraded":"2024-02-18T00:33:15.127051+0000","last_fullsized":"2024-02-18T00:33:15.127051+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.4","version":"112'6","reported_seq":43,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.106860+0000","last_change":"2024-02-18T00:33:08.068061+0000","last_active":"2024-02-18T00:33:14.106860+0000","last_peered":"2024-02-18T00:33:14.106860+0000","last_clean":"2024-02-18T00:33:14.106860+0000","last_became_active":"2024-02-18T00:33:01.526381+0000","last_became_peered":"2024-02-18T00:33:01.526381+0000","last_unstale":"2024-02-18T00:33:14.106860+0000","last_undegraded":"2024-02-18T00:33:14.106860+0000","last_fullsized":"2024-02-18T00:33:14.106860+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6","last_scrub_stamp":"2024-02-18T00:33:08.067963+0000","last_deep_scrub":"112'6","last_deep_scrub_stamp":"2024-02-18T00:33:08.067963+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.067963+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165852,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24583,"num_write":3,"num_write_kb":24579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.0","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132068+0000","last_change":"2024-02-18T00:33:02.516955+0000","last_active":"2024-02-18T00:33:14.132068+0000","last_peered":"2024-02-18T00:33:14.132068+0000","last_clean":"2024-02-18T00:33:14.132068+0000","last_became_active":"2024-02-18T00:33:02.516681+0000","last_became_peered":"2024-02-18T00:33:02.516681+0000","last_unstale":"2024-02-18T00:33:14.132068+0000","last_undegraded":"2024-02-18T00:33:14.132068+0000","last_fullsized":"2024-02-18T00:33:14.132068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114857+0000","last_change":"2024-02-18T00:33:02.504339+0000","last_active":"2024-02-18T00:33:14.114857+0000","last_peered":"2024-02-18T00:33:14.114857+0000","last_clean":"2024-02-18T00:33:14.114857+0000","last_became_active":"2024-02-18T00:33:02.503627+0000","last_became_peered":"2024-02-18T00:33:02.503627+0000","last_unstale":"2024-02-18T00:33:14.114857+0000","last_undegraded":"2024-02-18T00:33:14.114857+0000","last_fullsized":"2024-02-18T00:33:14.114857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":107,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109200+0000","last_change":"2024-02-18T00:31:55.016021+0000","last_active":"2024-02-18T00:33:14.109200+0000","last_peered":"2024-02-18T00:33:14.109200+0000","last_clean":"2024-02-18T00:33:14.109200+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:33:14.109200+0000","last_undegraded":"2024-02-18T00:33:14.109200+0000","last_fullsized":"2024-02-18T00:33:14.109200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:55.015959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.015959+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113962+0000","last_change":"2024-02-18T00:32:49.927116+0000","last_active":"2024-02-18T00:33:14.113962+0000","last_peered":"2024-02-18T00:33:14.113962+0000","last_clean":"2024-02-18T00:33:14.113962+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:33:14.113962+0000","last_undegraded":"2024-02-18T00:33:14.113962+0000","last_fullsized":"2024-02-18T00:33:14.113962+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:49.926996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.926996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":140,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135307+0000","last_change":"2024-02-18T00:32:45.395174+0000","last_active":"2024-02-18T00:33:14.135307+0000","last_peered":"2024-02-18T00:33:14.135307+0000","last_clean":"2024-02-18T00:33:14.135307+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:33:14.135307+0000","last_undegraded":"2024-02-18T00:33:14.135307+0000","last_fullsized":"2024-02-18T00:33:14.135307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:45.395085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.395085+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.112809+0000","last_change":"2024-02-18T00:31:45.654910+0000","last_active":"2024-02-18T00:33:14.112809+0000","last_peered":"2024-02-18T00:33:14.112809+0000","last_clean":"2024-02-18T00:33:14.112809+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:33:14.112809+0000","last_undegraded":"2024-02-18T00:33:14.112809+0000","last_fullsized":"2024-02-18T00:33:14.112809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:31:45.654827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:45.654827+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.111794+0000","last_change":"2024-02-18T00:33:15.115363+0000","last_active":"2024-02-18T00:33:15.111794+0000","last_peered":"2024-02-18T00:33:15.111794+0000","last_clean":"2024-02-18T00:33:15.111794+0000","last_became_active":"2024-02-18T00:33:01.485533+0000","last_became_peered":"2024-02-18T00:33:01.485533+0000","last_unstale":"2024-02-18T00:33:15.111794+0000","last_undegraded":"2024-02-18T00:33:15.111794+0000","last_fullsized":"2024-02-18T00:33:15.111794+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.b","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121920+0000","last_change":"2024-02-18T00:33:01.581012+0000","last_active":"2024-02-18T00:33:14.121920+0000","last_peered":"2024-02-18T00:33:14.121920+0000","last_clean":"2024-02-18T00:33:14.121920+0000","last_became_active":"2024-02-18T00:33:01.579869+0000","last_became_peered":"2024-02-18T00:33:01.579869+0000","last_unstale":"2024-02-18T00:33:14.121920+0000","last_undegraded":"2024-02-18T00:33:14.121920+0000","last_fullsized":"2024-02-18T00:33:14.121920+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125225+0000","last_change":"2024-02-18T00:33:02.504123+0000","last_active":"2024-02-18T00:33:14.125225+0000","last_peered":"2024-02-18T00:33:14.125225+0000","last_clean":"2024-02-18T00:33:14.125225+0000","last_became_active":"2024-02-18T00:33:02.503440+0000","last_became_peered":"2024-02-18T00:33:02.503440+0000","last_unstale":"2024-02-18T00:33:14.125225+0000","last_undegraded":"2024-02-18T00:33:14.125225+0000","last_fullsized":"2024-02-18T00:33:14.125225+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118081+0000","last_change":"2024-02-18T00:33:02.513760+0000","last_active":"2024-02-18T00:33:14.118081+0000","last_peered":"2024-02-18T00:33:14.118081+0000","last_clean":"2024-02-18T00:33:14.118081+0000","last_became_active":"2024-02-18T00:33:02.513275+0000","last_became_peered":"2024-02-18T00:33:02.513275+0000","last_unstale":"2024-02-18T00:33:14.118081+0000","last_undegraded":"2024-02-18T00:33:14.118081+0000","last_fullsized":"2024-02-18T00:33:14.118081+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":112,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.135394+0000","last_change":"2024-02-18T00:31:56.165995+0000","last_active":"2024-02-18T00:33:14.135394+0000","last_peered":"2024-02-18T00:33:14.135394+0000","last_clean":"2024-02-18T00:33:14.135394+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:33:14.135394+0000","last_undegraded":"2024-02-18T00:33:14.135394+0000","last_fullsized":"2024-02-18T00:33:14.135394+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:56.165920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.165920+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127129+0000","last_change":"2024-02-18T00:31:49.512647+0000","last_active":"2024-02-18T00:33:14.127129+0000","last_peered":"2024-02-18T00:33:14.127129+0000","last_clean":"2024-02-18T00:33:14.127129+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:33:14.127129+0000","last_undegraded":"2024-02-18T00:33:14.127129+0000","last_fullsized":"2024-02-18T00:33:14.127129+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:31:49.512572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.109276+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:33:14.109276+0000","last_peered":"2024-02-18T00:33:14.109276+0000","last_clean":"2024-02-18T00:33:14.109276+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:33:14.109276+0000","last_undegraded":"2024-02-18T00:33:14.109276+0000","last_fullsized":"2024-02-18T00:33:14.109276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127695+0000","last_change":"2024-02-18T00:32:59.800515+0000","last_active":"2024-02-18T00:33:14.127695+0000","last_peered":"2024-02-18T00:33:14.127695+0000","last_clean":"2024-02-18T00:33:14.127695+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:33:14.127695+0000","last_undegraded":"2024-02-18T00:33:14.127695+0000","last_fullsized":"2024-02-18T00:33:14.127695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.800355+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"118'1","reported_seq":40,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.122406+0000","last_change":"2024-02-18T00:33:15.122406+0000","last_active":"2024-02-18T00:33:15.122406+0000","last_peered":"2024-02-18T00:33:15.122406+0000","last_clean":"2024-02-18T00:33:15.122406+0000","last_became_active":"2024-02-18T00:33:01.472354+0000","last_became_peered":"2024-02-18T00:33:01.472354+0000","last_unstale":"2024-02-18T00:33:15.122406+0000","last_undegraded":"2024-02-18T00:33:15.122406+0000","last_fullsized":"2024-02-18T00:33:15.122406+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,3],"acting":[7,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.a","version":"106'2","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118883+0000","last_change":"2024-02-18T00:33:01.428299+0000","last_active":"2024-02-18T00:33:14.118883+0000","last_peered":"2024-02-18T00:33:14.118883+0000","last_clean":"2024-02-18T00:33:14.118883+0000","last_became_active":"2024-02-18T00:33:01.428013+0000","last_became_peered":"2024-02-18T00:33:01.428013+0000","last_unstale":"2024-02-18T00:33:14.118883+0000","last_undegraded":"2024-02-18T00:33:14.118883+0000","last_fullsized":"2024-02-18T00:33:14.118883+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.123548+0000","last_change":"2024-02-18T00:33:02.537930+0000","last_active":"2024-02-18T00:33:14.123548+0000","last_peered":"2024-02-18T00:33:14.123548+0000","last_clean":"2024-02-18T00:33:14.123548+0000","last_became_active":"2024-02-18T00:33:02.536038+0000","last_became_peered":"2024-02-18T00:33:02.536038+0000","last_unstale":"2024-02-18T00:33:14.123548+0000","last_undegraded":"2024-02-18T00:33:14.123548+0000","last_fullsized":"2024-02-18T00:33:14.123548+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.105814+0000","last_change":"2024-02-18T00:33:04.698462+0000","last_active":"2024-02-18T00:33:14.105814+0000","last_peered":"2024-02-18T00:33:14.105814+0000","last_clean":"2024-02-18T00:33:14.105814+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:33:14.105814+0000","last_undegraded":"2024-02-18T00:33:14.105814+0000","last_fullsized":"2024-02-18T00:33:14.105814+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108531+0000","last_change":"2024-02-18T00:32:48.395232+0000","last_active":"2024-02-18T00:33:14.108531+0000","last_peered":"2024-02-18T00:33:14.108531+0000","last_clean":"2024-02-18T00:33:14.108531+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:33:14.108531+0000","last_undegraded":"2024-02-18T00:33:14.108531+0000","last_fullsized":"2024-02-18T00:33:14.108531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:48.395162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:48.395162+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":85,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.120330+0000","last_change":"2024-02-18T00:32:57.386162+0000","last_active":"2024-02-18T00:33:14.120330+0000","last_peered":"2024-02-18T00:33:14.120330+0000","last_clean":"2024-02-18T00:33:14.120330+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:33:14.120330+0000","last_undegraded":"2024-02-18T00:33:14.120330+0000","last_fullsized":"2024-02-18T00:33:14.120330+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.386090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.386090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128022+0000","last_change":"2024-02-18T00:32:06.883082+0000","last_active":"2024-02-18T00:33:14.128022+0000","last_peered":"2024-02-18T00:33:14.128022+0000","last_clean":"2024-02-18T00:33:14.128022+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:33:14.128022+0000","last_undegraded":"2024-02-18T00:33:14.128022+0000","last_fullsized":"2024-02-18T00:33:14.128022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:06.883019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:32:06.883019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138908+0000","last_change":"2024-02-18T00:32:38.985290+0000","last_active":"2024-02-18T00:33:14.138908+0000","last_peered":"2024-02-18T00:33:14.138908+0000","last_clean":"2024-02-18T00:33:14.138908+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:33:14.138908+0000","last_undegraded":"2024-02-18T00:33:14.138908+0000","last_fullsized":"2024-02-18T00:33:14.138908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.985181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.985181+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.143023+0000","last_change":"2024-02-18T00:33:15.144471+0000","last_active":"2024-02-18T00:33:15.143023+0000","last_peered":"2024-02-18T00:33:15.143023+0000","last_clean":"2024-02-18T00:33:15.143023+0000","last_became_active":"2024-02-18T00:33:01.595209+0000","last_became_peered":"2024-02-18T00:33:01.595209+0000","last_unstale":"2024-02-18T00:33:15.143023+0000","last_undegraded":"2024-02-18T00:33:15.143023+0000","last_fullsized":"2024-02-18T00:33:15.143023+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.9","version":"106'6","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.126786+0000","last_change":"2024-02-18T00:33:08.650721+0000","last_active":"2024-02-18T00:33:14.126786+0000","last_peered":"2024-02-18T00:33:14.126786+0000","last_clean":"2024-02-18T00:33:14.126786+0000","last_became_active":"2024-02-18T00:33:01.593606+0000","last_became_peered":"2024-02-18T00:33:01.593606+0000","last_unstale":"2024-02-18T00:33:14.126786+0000","last_undegraded":"2024-02-18T00:33:14.126786+0000","last_fullsized":"2024-02-18T00:33:14.126786+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6","last_scrub_stamp":"2024-02-18T00:33:08.650652+0000","last_deep_scrub":"106'6","last_deep_scrub_stamp":"2024-02-18T00:33:08.650652+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.650652+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3","version":"109'5","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115313+0000","last_change":"2024-02-18T00:33:02.529166+0000","last_active":"2024-02-18T00:33:14.115313+0000","last_peered":"2024-02-18T00:33:14.115313+0000","last_clean":"2024-02-18T00:33:14.115313+0000","last_became_active":"2024-02-18T00:33:02.528960+0000","last_became_peered":"2024-02-18T00:33:02.528960+0000","last_unstale":"2024-02-18T00:33:14.115313+0000","last_undegraded":"2024-02-18T00:33:14.115313+0000","last_fullsized":"2024-02-18T00:33:14.115313+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.132151+0000","last_change":"2024-02-18T00:33:02.473416+0000","last_active":"2024-02-18T00:33:14.132151+0000","last_peered":"2024-02-18T00:33:14.132151+0000","last_clean":"2024-02-18T00:33:14.132151+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:33:14.132151+0000","last_undegraded":"2024-02-18T00:33:14.132151+0000","last_fullsized":"2024-02-18T00:33:14.132151+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":116,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.137427+0000","last_change":"2024-02-18T00:32:44.768998+0000","last_active":"2024-02-18T00:33:14.137427+0000","last_peered":"2024-02-18T00:33:14.137427+0000","last_clean":"2024-02-18T00:33:14.137427+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:33:14.137427+0000","last_undegraded":"2024-02-18T00:33:14.137427+0000","last_fullsized":"2024-02-18T00:33:14.137427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:32:44.768886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.768886+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":87,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140503+0000","last_change":"2024-02-18T00:31:32.210269+0000","last_active":"2024-02-18T00:33:14.140503+0000","last_peered":"2024-02-18T00:33:14.140503+0000","last_clean":"2024-02-18T00:33:14.140503+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:33:14.140503+0000","last_undegraded":"2024-02-18T00:33:14.140503+0000","last_fullsized":"2024-02-18T00:33:14.140503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:31:32.210198+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.210198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.130270+0000","last_change":"2024-02-18T00:32:41.027232+0000","last_active":"2024-02-18T00:33:14.130270+0000","last_peered":"2024-02-18T00:33:14.130270+0000","last_clean":"2024-02-18T00:33:14.130270+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:33:14.130270+0000","last_undegraded":"2024-02-18T00:33:14.130270+0000","last_fullsized":"2024-02-18T00:33:14.130270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:41.027153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:41.027153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.131794+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:33:14.131794+0000","last_peered":"2024-02-18T00:33:14.131794+0000","last_clean":"2024-02-18T00:33:14.131794+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:33:14.131794+0000","last_undegraded":"2024-02-18T00:33:14.131794+0000","last_fullsized":"2024-02-18T00:33:14.131794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.139513+0000","last_change":"2024-02-18T00:33:15.141052+0000","last_active":"2024-02-18T00:33:15.139513+0000","last_peered":"2024-02-18T00:33:15.139513+0000","last_clean":"2024-02-18T00:33:15.139513+0000","last_became_active":"2024-02-18T00:33:01.453414+0000","last_became_peered":"2024-02-18T00:33:01.453414+0000","last_unstale":"2024-02-18T00:33:15.139513+0000","last_undegraded":"2024-02-18T00:33:15.139513+0000","last_fullsized":"2024-02-18T00:33:15.139513+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.8","version":"106'6","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.104294+0000","last_change":"2024-02-18T00:33:06.776464+0000","last_active":"2024-02-18T00:33:14.104294+0000","last_peered":"2024-02-18T00:33:14.104294+0000","last_clean":"2024-02-18T00:33:14.104294+0000","last_became_active":"2024-02-18T00:33:01.427985+0000","last_became_peered":"2024-02-18T00:33:01.427985+0000","last_unstale":"2024-02-18T00:33:14.104294+0000","last_undegraded":"2024-02-18T00:33:14.104294+0000","last_fullsized":"2024-02-18T00:33:14.104294+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6","last_scrub_stamp":"2024-02-18T00:33:06.776396+0000","last_deep_scrub":"106'6","last_deep_scrub_stamp":"2024-02-18T00:33:06.776396+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.776396+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.108434+0000","last_change":"2024-02-18T00:33:04.688881+0000","last_active":"2024-02-18T00:33:14.108434+0000","last_peered":"2024-02-18T00:33:14.108434+0000","last_clean":"2024-02-18T00:33:14.108434+0000","last_became_active":"2024-02-18T00:33:04.688607+0000","last_became_peered":"2024-02-18T00:33:04.688607+0000","last_unstale":"2024-02-18T00:33:14.108434+0000","last_undegraded":"2024-02-18T00:33:14.108434+0000","last_fullsized":"2024-02-18T00:33:14.108434+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115563+0000","last_change":"2024-02-18T00:33:04.595768+0000","last_active":"2024-02-18T00:33:14.115563+0000","last_peered":"2024-02-18T00:33:14.115563+0000","last_clean":"2024-02-18T00:33:14.115563+0000","last_became_active":"2024-02-18T00:33:04.595086+0000","last_became_peered":"2024-02-18T00:33:04.595086+0000","last_unstale":"2024-02-18T00:33:14.115563+0000","last_undegraded":"2024-02-18T00:33:14.115563+0000","last_fullsized":"2024-02-18T00:33:14.115563+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":136,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.129869+0000","last_change":"2024-02-18T00:33:00.013117+0000","last_active":"2024-02-18T00:33:14.129869+0000","last_peered":"2024-02-18T00:33:14.129869+0000","last_clean":"2024-02-18T00:33:14.129869+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:33:14.129869+0000","last_undegraded":"2024-02-18T00:33:14.129869+0000","last_fullsized":"2024-02-18T00:33:14.129869+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:33:00.013051+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:33:00.013051+0000","last_clean_scrub_stamp":"2024-02-18T00:33:00.013051+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107871+0000","last_change":"2024-02-18T00:32:41.659287+0000","last_active":"2024-02-18T00:33:14.107871+0000","last_peered":"2024-02-18T00:33:14.107871+0000","last_clean":"2024-02-18T00:33:14.107871+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:33:14.107871+0000","last_undegraded":"2024-02-18T00:33:14.107871+0000","last_fullsized":"2024-02-18T00:33:14.107871+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:41.659220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:41.659220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117184+0000","last_change":"2024-02-18T00:31:58.856050+0000","last_active":"2024-02-18T00:33:14.117184+0000","last_peered":"2024-02-18T00:33:14.117184+0000","last_clean":"2024-02-18T00:33:14.117184+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:33:14.117184+0000","last_undegraded":"2024-02-18T00:33:14.117184+0000","last_fullsized":"2024-02-18T00:33:14.117184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.855979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.855979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.102318+0000","last_change":"2024-02-18T00:31:36.140716+0000","last_active":"2024-02-18T00:33:14.102318+0000","last_peered":"2024-02-18T00:33:14.102318+0000","last_clean":"2024-02-18T00:33:14.102318+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:33:14.102318+0000","last_undegraded":"2024-02-18T00:33:14.102318+0000","last_fullsized":"2024-02-18T00:33:14.102318+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.140630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.140630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.121775+0000","last_change":"2024-02-18T00:33:15.121957+0000","last_active":"2024-02-18T00:33:15.121775+0000","last_peered":"2024-02-18T00:33:15.121775+0000","last_clean":"2024-02-18T00:33:15.121775+0000","last_became_active":"2024-02-18T00:33:01.463106+0000","last_became_peered":"2024-02-18T00:33:01.463106+0000","last_unstale":"2024-02-18T00:33:15.121775+0000","last_undegraded":"2024-02-18T00:33:15.121775+0000","last_fullsized":"2024-02-18T00:33:15.121775+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.f","version":"108'1001","reported_seq":2023,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.139136+0000","last_change":"2024-02-18T00:33:01.535111+0000","last_active":"2024-02-18T00:33:14.139136+0000","last_peered":"2024-02-18T00:33:14.139136+0000","last_clean":"2024-02-18T00:33:14.139136+0000","last_became_active":"2024-02-18T00:33:01.532648+0000","last_became_peered":"2024-02-18T00:33:01.532648+0000","last_unstale":"2024-02-18T00:33:14.139136+0000","last_undegraded":"2024-02-18T00:33:14.139136+0000","last_fullsized":"2024-02-18T00:33:14.139136+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1001,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1000,"num_read_kb":0,"num_write":1000,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127616+0000","last_change":"2024-02-18T00:33:02.511002+0000","last_active":"2024-02-18T00:33:14.127616+0000","last_peered":"2024-02-18T00:33:14.127616+0000","last_clean":"2024-02-18T00:33:14.127616+0000","last_became_active":"2024-02-18T00:33:02.510701+0000","last_became_peered":"2024-02-18T00:33:02.510701+0000","last_unstale":"2024-02-18T00:33:14.127616+0000","last_undegraded":"2024-02-18T00:33:14.127616+0000","last_fullsized":"2024-02-18T00:33:14.127616+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117671+0000","last_change":"2024-02-18T00:33:04.678891+0000","last_active":"2024-02-18T00:33:14.117671+0000","last_peered":"2024-02-18T00:33:14.117671+0000","last_clean":"2024-02-18T00:33:14.117671+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:33:14.117671+0000","last_undegraded":"2024-02-18T00:33:14.117671+0000","last_fullsized":"2024-02-18T00:33:14.117671+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.125185+0000","last_change":"2024-02-18T00:32:02.063451+0000","last_active":"2024-02-18T00:33:14.125185+0000","last_peered":"2024-02-18T00:33:14.125185+0000","last_clean":"2024-02-18T00:33:14.125185+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:33:14.125185+0000","last_undegraded":"2024-02-18T00:33:14.125185+0000","last_fullsized":"2024-02-18T00:33:14.125185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:32:02.063346+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.063346+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":85,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107071+0000","last_change":"2024-02-18T00:33:03.808951+0000","last_active":"2024-02-18T00:33:14.107071+0000","last_peered":"2024-02-18T00:33:14.107071+0000","last_clean":"2024-02-18T00:33:14.107071+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:33:14.107071+0000","last_undegraded":"2024-02-18T00:33:14.107071+0000","last_fullsized":"2024-02-18T00:33:14.107071+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:03.808887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.808887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.113023+0000","last_change":"2024-02-18T00:31:41.607649+0000","last_active":"2024-02-18T00:33:14.113023+0000","last_peered":"2024-02-18T00:33:14.113023+0000","last_clean":"2024-02-18T00:33:14.113023+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:33:14.113023+0000","last_undegraded":"2024-02-18T00:33:14.113023+0000","last_fullsized":"2024-02-18T00:33:14.113023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.607567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.110999+0000","last_change":"2024-02-18T00:31:44.843600+0000","last_active":"2024-02-18T00:33:14.110999+0000","last_peered":"2024-02-18T00:33:14.110999+0000","last_clean":"2024-02-18T00:33:14.110999+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:33:14.110999+0000","last_undegraded":"2024-02-18T00:33:14.110999+0000","last_fullsized":"2024-02-18T00:33:14.110999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:31:44.843494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.106288+0000","last_change":"2024-02-18T00:33:15.106567+0000","last_active":"2024-02-18T00:33:15.106288+0000","last_peered":"2024-02-18T00:33:15.106288+0000","last_clean":"2024-02-18T00:33:15.106288+0000","last_became_active":"2024-02-18T00:33:01.439073+0000","last_became_peered":"2024-02-18T00:33:01.439073+0000","last_unstale":"2024-02-18T00:33:15.106288+0000","last_undegraded":"2024-02-18T00:33:15.106288+0000","last_fullsized":"2024-02-18T00:33:15.106288+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.e","version":"106'2","reported_seq":24,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.117592+0000","last_change":"2024-02-18T00:33:01.542256+0000","last_active":"2024-02-18T00:33:14.117592+0000","last_peered":"2024-02-18T00:33:14.117592+0000","last_clean":"2024-02-18T00:33:14.117592+0000","last_became_active":"2024-02-18T00:33:01.541655+0000","last_became_peered":"2024-02-18T00:33:01.541655+0000","last_unstale":"2024-02-18T00:33:14.117592+0000","last_undegraded":"2024-02-18T00:33:14.117592+0000","last_fullsized":"2024-02-18T00:33:14.117592+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"109'9","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.115536+0000","last_change":"2024-02-18T00:33:02.512609+0000","last_active":"2024-02-18T00:33:14.115536+0000","last_peered":"2024-02-18T00:33:14.115536+0000","last_clean":"2024-02-18T00:33:14.115536+0000","last_became_active":"2024-02-18T00:33:02.512297+0000","last_became_peered":"2024-02-18T00:33:02.512297+0000","last_unstale":"2024-02-18T00:33:14.115536+0000","last_undegraded":"2024-02-18T00:33:14.115536+0000","last_fullsized":"2024-02-18T00:33:14.115536+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.138284+0000","last_change":"2024-02-18T00:33:04.657140+0000","last_active":"2024-02-18T00:33:14.138284+0000","last_peered":"2024-02-18T00:33:14.138284+0000","last_clean":"2024-02-18T00:33:14.138284+0000","last_became_active":"2024-02-18T00:33:04.656649+0000","last_became_peered":"2024-02-18T00:33:04.656649+0000","last_unstale":"2024-02-18T00:33:14.138284+0000","last_undegraded":"2024-02-18T00:33:14.138284+0000","last_fullsized":"2024-02-18T00:33:14.138284+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":109,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128140+0000","last_change":"2024-02-18T00:32:50.121604+0000","last_active":"2024-02-18T00:33:14.128140+0000","last_peered":"2024-02-18T00:33:14.128140+0000","last_clean":"2024-02-18T00:33:14.128140+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:33:14.128140+0000","last_undegraded":"2024-02-18T00:33:14.128140+0000","last_fullsized":"2024-02-18T00:33:14.128140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.121518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.121518+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140354+0000","last_change":"2024-02-18T00:32:43.807178+0000","last_active":"2024-02-18T00:33:14.140354+0000","last_peered":"2024-02-18T00:33:14.140354+0000","last_clean":"2024-02-18T00:33:14.140354+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:33:14.140354+0000","last_undegraded":"2024-02-18T00:33:14.140354+0000","last_fullsized":"2024-02-18T00:33:14.140354+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.807055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.807055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.118475+0000","last_change":"2024-02-18T00:32:38.804852+0000","last_active":"2024-02-18T00:33:14.118475+0000","last_peered":"2024-02-18T00:33:14.118475+0000","last_clean":"2024-02-18T00:33:14.118475+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:33:14.118475+0000","last_undegraded":"2024-02-18T00:33:14.118475+0000","last_fullsized":"2024-02-18T00:33:14.118475+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.804774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.804774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.129659+0000","last_change":"2024-02-18T00:32:36.954112+0000","last_active":"2024-02-18T00:33:14.129659+0000","last_peered":"2024-02-18T00:33:14.129659+0000","last_clean":"2024-02-18T00:33:14.129659+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:33:14.129659+0000","last_undegraded":"2024-02-18T00:33:14.129659+0000","last_fullsized":"2024-02-18T00:33:14.129659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.954039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.954039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.113570+0000","last_change":"2024-02-18T00:33:15.113775+0000","last_active":"2024-02-18T00:33:15.113570+0000","last_peered":"2024-02-18T00:33:15.113570+0000","last_clean":"2024-02-18T00:33:15.113570+0000","last_became_active":"2024-02-18T00:33:01.487845+0000","last_became_peered":"2024-02-18T00:33:01.487845+0000","last_unstale":"2024-02-18T00:33:15.113570+0000","last_undegraded":"2024-02-18T00:33:15.113570+0000","last_fullsized":"2024-02-18T00:33:15.113570+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.d","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.105993+0000","last_change":"2024-02-18T00:33:01.472972+0000","last_active":"2024-02-18T00:33:14.105993+0000","last_peered":"2024-02-18T00:33:14.105993+0000","last_clean":"2024-02-18T00:33:14.105993+0000","last_became_active":"2024-02-18T00:33:01.472652+0000","last_became_peered":"2024-02-18T00:33:01.472652+0000","last_unstale":"2024-02-18T00:33:14.105993+0000","last_undegraded":"2024-02-18T00:33:14.105993+0000","last_fullsized":"2024-02-18T00:33:14.105993+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.121419+0000","last_change":"2024-02-18T00:33:04.691417+0000","last_active":"2024-02-18T00:33:14.121419+0000","last_peered":"2024-02-18T00:33:14.121419+0000","last_clean":"2024-02-18T00:33:14.121419+0000","last_became_active":"2024-02-18T00:33:04.691013+0000","last_became_peered":"2024-02-18T00:33:04.691013+0000","last_unstale":"2024-02-18T00:33:14.121419+0000","last_undegraded":"2024-02-18T00:33:14.121419+0000","last_fullsized":"2024-02-18T00:33:14.121419+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.144300+0000","last_change":"2024-02-18T00:33:04.720986+0000","last_active":"2024-02-18T00:33:14.144300+0000","last_peered":"2024-02-18T00:33:14.144300+0000","last_clean":"2024-02-18T00:33:14.144300+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:33:14.144300+0000","last_undegraded":"2024-02-18T00:33:14.144300+0000","last_fullsized":"2024-02-18T00:33:14.144300+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":158,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.127055+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:33:14.127055+0000","last_peered":"2024-02-18T00:33:14.127055+0000","last_clean":"2024-02-18T00:33:14.127055+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:33:14.127055+0000","last_undegraded":"2024-02-18T00:33:14.127055+0000","last_fullsized":"2024-02-18T00:33:14.127055+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.114961+0000","last_change":"2024-02-18T00:32:58.998813+0000","last_active":"2024-02-18T00:33:14.114961+0000","last_peered":"2024-02-18T00:33:14.114961+0000","last_clean":"2024-02-18T00:33:14.114961+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:33:14.114961+0000","last_undegraded":"2024-02-18T00:33:14.114961+0000","last_fullsized":"2024-02-18T00:33:14.114961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.998750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.128551+0000","last_change":"2024-02-18T00:32:58.038986+0000","last_active":"2024-02-18T00:33:14.128551+0000","last_peered":"2024-02-18T00:33:14.128551+0000","last_clean":"2024-02-18T00:33:14.128551+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:33:14.128551+0000","last_undegraded":"2024-02-18T00:33:14.128551+0000","last_fullsized":"2024-02-18T00:33:14.128551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.038886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.038886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.140224+0000","last_change":"2024-02-18T00:32:36.732383+0000","last_active":"2024-02-18T00:33:14.140224+0000","last_peered":"2024-02-18T00:33:14.140224+0000","last_clean":"2024-02-18T00:33:14.140224+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:33:14.140224+0000","last_undegraded":"2024-02-18T00:33:14.140224+0000","last_fullsized":"2024-02-18T00:33:14.140224+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.732282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.732282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":39,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-02-18T00:33:15.108896+0000","last_change":"2024-02-18T00:33:15.109513+0000","last_active":"2024-02-18T00:33:15.108896+0000","last_peered":"2024-02-18T00:33:15.108896+0000","last_clean":"2024-02-18T00:33:15.108896+0000","last_became_active":"2024-02-18T00:33:01.441007+0000","last_became_peered":"2024-02-18T00:33:01.441007+0000","last_unstale":"2024-02-18T00:33:15.108896+0000","last_undegraded":"2024-02-18T00:33:15.108896+0000","last_fullsized":"2024-02-18T00:33:15.108896+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"}]},{"pgid":"17.c","version":"0'0","reported_seq":22,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-02-18T00:33:14.107518+0000","last_change":"2024-02-18T00:33:01.463611+0000","last_active":"2024-02-18T00:33:14.107518+0000","last_peered":"2024-02-18T00:33:14.107518+0000","last_clean":"2024-02-18T00:33:14.107518+0000","last_became_active":"2024-02-18T00:33:01.463209+0000","last_became_peered":"2024-02-18T00:33:01.463209+0000","last_unstale":"2024-02-18T00:33:14.107518+0000","last_undegraded":"2024-02-18T00:33:14.107518+0000","last_fullsized":"2024-02-18T00:33:14.107518+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":61,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":60,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":59,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":58,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":57,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":53,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":6},{"poolid":44,"num_pg":32,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":150998978,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1011,"num_read_kb":49161,"num_write":1043,"num_write_kb":152978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":453267456,"data_stored":452996934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1056,"ondisk_log_size":1056,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":617,"ondisk_log_size":617,"up":96,"acting":96,"num_store_stats":8},{"poolid":40,"num_pg":32,"stat_sum":{"num_bytes":24576,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":73728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":232,"num_read_kb":227,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":651,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":515,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266740,"num_pgs":266,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":331088,"kb_used_data":25800,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93414064,"statfs":{"total":95995035648,"available":95656001536,"internally_reserved":0,"allocated":26419200,"data_stored":25798523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.504,"15min":0.499},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.627,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.24299999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.315},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.38}]},{"osd":2,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60999999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.42799999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.307},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56899999999999995,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":3.351,"5min":3.351,"15min":3.351},"last":0.48699999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42899999999999999,"15min":0.41599999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.41499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34300000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46100000000000002,"15min":0.44500000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47299999999999998,"15min":0.45800000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72199999999999998,"5min":0.747,"15min":0.747},"last":0.442}]},{"osd":6,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.755,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73699999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.45500000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":196,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":380296,"kb_used_data":75008,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93364856,"statfs":{"total":95995035648,"available":95605612544,"internally_reserved":0,"allocated":76808192,"data_stored":76164108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.505},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.76800000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.498}]},{"osd":1,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68100000000000005,"5min":0.874,"15min":0.874},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.39700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72199999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.372}]},{"osd":2,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.50700000000000001},"min":{"1min":0.40999999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.64300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73999999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.438}]},{"osd":3,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78800000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.437,"15min":0.40699999999999997},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.69899999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41299999999999998},"min":{"1min":0.41199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63900000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.35299999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.42099999999999999},"min":{"1min":0.40600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65100000000000002,"5min":1.121,"15min":1.121},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.432},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71199999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.53000000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.51000000000000001,"15min":0.50700000000000001},"min":{"1min":0.379,"5min":0.318,"15min":0.318},"max":{"1min":0.748,"5min":1.075,"15min":1.075},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.40799999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84199999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.45500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":208,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":384400,"kb_used_data":75016,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93360752,"statfs":{"total":95995035648,"available":95601410048,"internally_reserved":0,"allocated":76816384,"data_stored":76139633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39200000000000002,"15min":0.32200000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.20100000000000001},"max":{"1min":1.0780000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.433,"15min":0.34499999999999997},"min":{"1min":0.27500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.9770000000000003,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.38,"15min":0.373},"min":{"1min":0.27900000000000003,"5min":0.245,"15min":0.245},"max":{"1min":0.627,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41899999999999998,"15min":0.39500000000000002},"min":{"1min":0.27300000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":1.012,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.35199999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:32:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.48399999999999999},"min":{"1min":0.29299999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.32000000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1559999999999999,"5min":1.347,"15min":1.347},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.2370000000000001,"5min":4.2370000000000001,"15min":4.2370000000000001},"last":0.44600000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.45600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.038,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.45000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.298,"5min":1.427,"15min":1.427},"last":0.50700000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.49399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.071,"5min":1.278,"15min":1.278},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.502,"5min":0.441,"15min":0.441},"max":{"1min":0.90200000000000002,"5min":1.393,"15min":1.393},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.67600000000000005,"15min":0.66100000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":4.2850000000000001,"5min":4.2850000000000001,"15min":4.2850000000000001},"last":0.54300000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":228,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":355720,"kb_used_data":50432,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93389432,"statfs":{"total":95995035648,"available":95630778368,"internally_reserved":0,"allocated":51642368,"data_stored":50976988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.41899999999999998,"15min":0.57499999999999996},"min":{"1min":0.22,"5min":0.22,"15min":0.20699999999999999},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":6.7720000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.41199999999999998,"15min":0.35699999999999998},"min":{"1min":0.24299999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.28799999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":4.9790000000000001,"5min":1.323,"15min":0.69799999999999995},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.215},"max":{"1min":90.850999999999999,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.441},{"interface":"front","average":{"1min":5.0149999999999997,"5min":1.3180000000000001,"15min":0.69499999999999995},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.224},"max":{"1min":80.537000000000006,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.318}]},{"osd":3,"last update":"Sun Feb 18 00:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.29099999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.1339999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45700000000000002,"15min":0.49299999999999999},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41599999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:33:01 2024","interfaces":[{"interface":"back","average":{"1min":4.3789999999999996,"5min":1.323,"15min":0.80200000000000005},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":57.914000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":4.867,"5min":1.444,"15min":0.88800000000000001},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":90.070999999999998,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.53500000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:32:27 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.46600000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56899999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.59299999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":4.8940000000000001,"5min":1.48,"15min":0.89800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":90.025000000000006,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.627},{"interface":"front","average":{"1min":4.4119999999999999,"5min":1.359,"15min":0.84499999999999997},"min":{"1min":0.434,"5min":0.37,"15min":0.37},"max":{"1min":57.962000000000003,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.51300000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":231,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":355740,"kb_used_data":50452,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93389412,"statfs":{"total":95995035648,"available":95630757888,"internally_reserved":0,"allocated":51662848,"data_stored":50986743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39700000000000002,"15min":0.375},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.79200000000000004,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.40000000000000002,"15min":0.376},"min":{"1min":0.25900000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.54000000000000004,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":0.34999999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.41299999999999998,"15min":0.38800000000000001},"min":{"1min":0.28799999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":2.306,"15min":2.306},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.40300000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.73899999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.251}]},{"osd":3,"last update":"Sun Feb 18 00:32:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.46700000000000003,"15min":0.45900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.91800000000000004,"5min":1.296,"15min":1.296},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.308}]},{"osd":4,"last update":"Sun Feb 18 00:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.5580000000000001,"5min":1.675,"15min":1.675},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96199999999999997,"5min":1.173,"15min":1.173},"last":0.5}]},{"osd":5,"last update":"Sun Feb 18 00:32:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.46999999999999997,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.39700000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93999999999999995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.625,"15min":0.64300000000000002},"min":{"1min":0.44900000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.63},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65600000000000003,"15min":0.68999999999999995},"min":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86199999999999999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65800000000000003,"15min":0.67200000000000004},"min":{"1min":0.52000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83899999999999997,"5min":1.242,"15min":1.242},"last":0.59099999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":240,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":355712,"kb_used_data":50424,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93389440,"statfs":{"total":95995035648,"available":95630786560,"internally_reserved":0,"allocated":51634176,"data_stored":50973637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.373,"15min":0.39000000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.27300000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.35199999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.40799999999999997,"15min":0.40699999999999997},"min":{"1min":0.26600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.27200000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.436,"15min":0.438},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.37}]},{"osd":4,"last update":"Sun Feb 18 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.8010000000000002,"5min":3.8010000000000002,"15min":3.8010000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.627,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.41599999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:32:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.58699999999999997,"15min":0.59999999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97199999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82999999999999996,"5min":1.139,"15min":1.139},"last":0.40500000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83799999999999997,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.625},"min":{"1min":0.41599999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.66800000000000004,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.2759999999999998,"5min":6.2759999999999998,"15min":6.2759999999999998},"last":0.47599999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986243,"num_pgs":217,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":380308,"kb_used_data":75020,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93364844,"statfs":{"total":95995035648,"available":95605600256,"internally_reserved":0,"allocated":76820480,"data_stored":76148082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.3239999999999998,"5min":6.3239999999999998,"15min":6.3239999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.307}]},{"osd":1,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.752,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.44700000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2110000000000001,"5min":1.48,"15min":1.48},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.438,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.022,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.435},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:32:29 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.40500000000000003,"15min":0.36799999999999999},"min":{"1min":0.36399999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.42099999999999999,"15min":0.377},"min":{"1min":0.309,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.44600000000000001,"15min":0.40500000000000003},"min":{"1min":0.34100000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.71699999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.378},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.44600000000000001},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.52900000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.747,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.505},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.505,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51800000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":226,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":355708,"kb_used_data":50420,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93389444,"statfs":{"total":95995035648,"available":95630790656,"internally_reserved":0,"allocated":51630080,"data_stored":51005781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.51600000000000001,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68000000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52500000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68899999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.316}]},{"osd":1,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56399999999999995,"15min":0.58399999999999996},"min":{"1min":0.36099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77700000000000002,"5min":1.306,"15min":1.306},"last":0.375},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.432,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":1.272,"15min":1.272},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.40699999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72399999999999998,"5min":1.173,"15min":1.173},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58399999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73199999999999998,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.442}]},{"osd":3,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.45100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64900000000000002,"5min":1.238,"15min":1.238},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.746,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:32:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.40799999999999997,"15min":0.374},"min":{"1min":0.32200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68600000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.42599999999999999,"15min":0.38700000000000001},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46700000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.44900000000000001,"15min":0.41399999999999998},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.621,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46899999999999997,"15min":0.42199999999999999},"min":{"1min":0.33900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.751,"15min":0.751},"last":0.501}]},{"osd":7,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":3,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50380800,"data_stored":50335600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75534336,"data_stored":75497544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50360320,"data_stored":50331696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50364416,"data_stored":50331707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":75558912,"data_stored":75501451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50356224,"data_stored":50335559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":75522048,"data_stored":75497513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25190400,"data_stored":25165864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":40,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":44,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":11,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":53,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":53,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":53,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":53,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":53,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":53,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:33:21.402 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -7, currently 3 2024-02-18T00:33:21.411 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:21.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:21.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:21.518 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:21.545 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:21.565 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:21.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:21.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:21.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:21.784 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:22.017 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:22.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:22.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:22.127 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:22.154 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:22.173 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:22.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:22.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:22.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:22.302 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:22.321 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:22.324 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:22.324 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:22.360 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:33:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:22] "GET /metrics HTTP/1.1" 200 224171 "" "Prometheus/2.33.4" 2024-02-18T00:33:22.411 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-18T00:33:22.411 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-18T00:33:22.411 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-18T00:33:22.411 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-18T00:33:22.411 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-18T00:33:22.411 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify_test_cb 2024-02-18T00:33:22.411 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (138 ms) 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94878363365584 err -107 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1011 ms) 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94878363377232 err -107 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1086 ms) 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 468151435264 cookie 94878363377232 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (64 ms) 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-18T00:33:22.412 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 468151435264 cookie 94878363398352 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (41 ms) 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 468151435265 cookie 94878363398352 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (11 ms) 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 468151435266 cookie 94878363398352 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 468151435266 cookie 94878363414496 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (32 ms) 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 468151435264 cookie 94878363398352 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 472446402561 cookie 94878363398352 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3012 ms) 2024-02-18T00:33:22.413 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94878363414496 err -107 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_cb from 24847 notify_id 493921239043 cookie 94878363414496 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5008 ms) 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94878363414496 err -107 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11407 ms total) 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-18T00:33:22.414 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: watch_notify_test_cb 2024-02-18T00:33:22.415 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1566 ms) 2024-02-18T00:33:22.415 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1566 ms total) 2024-02-18T00:33:22.415 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: 2024-02-18T00:33:22.415 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-18T00:33:22.415 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (31510 ms total) 2024-02-18T00:33:22.415 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-18T00:33:22.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:23.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-18T00:33:23.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-18T00:33:23.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-18T00:33:23.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-18T00:33:23.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5080 ms) 2024-02-18T00:33:23.025 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (50 ms) 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (21167 ms) 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: attempt 0 of 20 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: failed to deregister: 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: { 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "epoch": 18, 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "modified": "2024-02-18T00:33:17.777117+0000", 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "services": { 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "laundry": { 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "daemons": { 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "summary": "", 2024-02-18T00:33:23.026 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "close-test-pid83045": { 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_epoch": 18, 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_stamp": "2024-02-18T00:33:17.290427+0000", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "gid": 25202, 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "addr": "172.21.15.59:0/4193659469", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "metadata": { 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro": "centos", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-18T00:33:23.027 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_version": "8", 2024-02-18T00:33:23.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "foo": "bar", 2024-02-18T00:33:23.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "hostname": "smithi059", 2024-02-18T00:33:23.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_description": "#1 SMP Mon Feb 5 18:34:56 UTC 2024", 2024-02-18T00:33:23.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_version": "4.18.0-539.el8.x86_64", 2024-02-18T00:33:23.028 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_total_kb": "32543852", 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "os": "Linux", 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "this": "that" 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: }, 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "task_status": {} 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: }, 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "pid83037": { 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_epoch": 17, 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_stamp": "2024-02-18T00:33:03.548735+0000", 2024-02-18T00:33:23.031 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "gid": 25018, 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "addr": "172.21.15.59:0/1113402902", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "metadata": { 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro": "centos", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_version": "8", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "foo": "bar", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "hostname": "smithi059", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_description": "#1 SMP Mon Feb 5 18:34:56 UTC 2024", 2024-02-18T00:33:23.032 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_version": "4.18.0-539.el8.x86_64", 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_total_kb": "32543852", 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "os": "Linux", 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "this": "that" 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: }, 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "task_status": {} 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: }, 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "rgw": { 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "daemons": { 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "summary": "", 2024-02-18T00:33:23.033 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "24347": { 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_epoch": 12, 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_stamp": "2024-02-18T00:29:22.198518+0000", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "gid": 24347, 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "addr": "172.21.15.59:0/2522576766", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "metadata": { 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "container_hostname": "smithi059", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro": "centos", 2024-02-18T00:33:23.034 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_version": "8", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "hostname": "smithi059", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "id": "foo.a", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_description": "#1 SMP Mon Feb 5 18:34:56 UTC 2024", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_version": "4.18.0-539.el8.x86_64", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_total_kb": "32543852", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "num_handles": "1", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "os": "Linux", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "pid": "7", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "realm_id": "", 2024-02-18T00:33:23.035 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "realm_name": "", 2024-02-18T00:33:23.036 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zone_id": "e67bc818-d879-487a-b572-f16f51d463f9", 2024-02-18T00:33:23.036 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zone_name": "default", 2024-02-18T00:33:23.036 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zonegroup_id": "1a312463-4a64-495f-95ab-5ad353fdc6fb", 2024-02-18T00:33:23.036 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-18T00:33:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.205+0000 7fbe75a97700 1 -- 172.21.15.59:0/3883223134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe700f6b10 msgr2=0x7fbe701030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.205+0000 7fbe75a97700 1 --2- 172.21.15.59:0/3883223134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe700f6b10 0x7fbe701030b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fbe6800acf0 tx=0x7fbe68004d10).stop 2024-02-18T00:33:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.205+0000 7fbe75a97700 1 -- 172.21.15.59:0/3883223134 shutdown_connections 2024-02-18T00:33:23.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.205+0000 7fbe75a97700 1 --2- 172.21.15.59:0/3883223134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 0x7fbe700fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.205+0000 7fbe75a97700 1 --2- 172.21.15.59:0/3883223134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe700f6b10 0x7fbe701030b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.205+0000 7fbe75a97700 1 --2- 172.21.15.59:0/3883223134 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe700fcba0 0x7fbe700f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.205+0000 7fbe75a97700 1 -- 172.21.15.59:0/3883223134 >> 172.21.15.59:0/3883223134 conn(0x7fbe7006c370 msgr2=0x7fbe7006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.206+0000 7fbe75a97700 1 -- 172.21.15.59:0/3883223134 shutdown_connections 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.206+0000 7fbe75a97700 1 -- 172.21.15.59:0/3883223134 wait complete. 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.207+0000 7fbe75a97700 1 librados: starting msgr at 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.207+0000 7fbe75a97700 1 librados: starting objecter 2024-02-18T00:33:23.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.207+0000 7fbe75a97700 1 Processor -- start 2024-02-18T00:33:23.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.207+0000 7fbe75a97700 1 -- start start 2024-02-18T00:33:23.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.207+0000 7fbe75a97700 1 librados: setting wanted keys 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.207+0000 7fbe75a97700 1 librados: calling monclient init 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe75a97700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe700f6b10 0x7fbe700653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe75a97700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 0x7fbe700658b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe75a97700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe700fcba0 0x7fbe7011c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe75a97700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7011cc80 con 0x7fbe700f6b10 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe75a97700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7011cdc0 con 0x7fbe700fc240 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe75a97700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70062e60 con 0x7fbe700fcba0 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe6effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 0x7fbe700658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:23.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe6effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 0x7fbe700658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:50170/0 (socket says 172.21.15.59:50170) 2024-02-18T00:33:23.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.208+0000 7fbe6effd700 1 -- 172.21.15.59:0/68804166 learned_addr learned my addr 172.21.15.59:0/68804166 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:23.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.210+0000 7fbe6ffff700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe700fcba0 0x7fbe7011c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:23.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.210+0000 7fbe6ffff700 1 -- 172.21.15.59:0/68804166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 msgr2=0x7fbe700658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:23.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.210+0000 7fbe6ffff700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 0x7fbe700658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:23.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.210+0000 7fbe6ffff700 1 -- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe700f6b10 msgr2=0x7fbe700653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:23.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.210+0000 7fbe6ffff700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe700f6b10 0x7fbe700653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:23.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.210+0000 7fbe6ffff700 1 -- 172.21.15.59:0/68804166 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6800a8d0 con 0x7fbe700fcba0 2024-02-18T00:33:23.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.211+0000 7fbe6effd700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 0x7fbe700658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:33:23.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.211+0000 7fbe6ffff700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe700fcba0 0x7fbe7011c790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fbe6800acc0 tx=0x7fbe68004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:23.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe6cf74700 10 client.?.objecter ms_handle_connect 0x7fbe700fcba0 2024-02-18T00:33:23.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe6cf74700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:23.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800f040 con 0x7fbe700fcba0 2024-02-18T00:33:23.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe6800a460 con 0x7fbe700fcba0 2024-02-18T00:33:23.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe700630e0 con 0x7fbe700fcba0 2024-02-18T00:33:23.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe75a97700 10 client.27173.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:23.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe70063570 con 0x7fbe700fcba0 2024-02-18T00:33:23.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.212+0000 7fbe75a97700 1 librados: init done 2024-02-18T00:33:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.213+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68027a90 con 0x7fbe700fcba0 2024-02-18T00:33:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.214+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c004f50 con 0x7fbe700fcba0 2024-02-18T00:33:23.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.215+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fbe68024070 con 0x7fbe700fcba0 2024-02-18T00:33:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.215+0000 7fbe6cf74700 1 --2- 172.21.15.59:0/68804166 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbe58062d10 0x7fbe580651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6f7fe700 1 --2- 172.21.15.59:0/68804166 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbe58062d10 0x7fbe580651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(125..125 src has 44..125) v4 ==== 11803+0+0 (secure 0 0 0) 0x7fbe68087fb0 con 0x7fbe700fcba0 2024-02-18T00:33:23.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 10 client.27173.objecter ms_dispatch 0x7fbe70001550 osd_map(125..125 src has 44..125) v4 2024-02-18T00:33:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 3 client.27173.objecter handle_osd_map got epochs [125,125] > 0 2024-02-18T00:33:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 3 client.27173.objecter handle_osd_map decoding full epoch 125 2024-02-18T00:33:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 10 client.27173.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=126}) v3 -- 0x7fbe58077de0 con 0x7fbe700fcba0 2024-02-18T00:33:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 20 client.27173.objecter dump_active .. 0 homeless 2024-02-18T00:33:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6f7fe700 1 --2- 172.21.15.59:0/68804166 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbe58062d10 0x7fbe580651d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fbe64009640 tx=0x7fbe64006b60).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:23.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.216+0000 7fbe6cf74700 10 client.27173.objecter ms_handle_connect 0x7fbe58062d10 2024-02-18T00:33:23.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.226+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbe6805a660 con 0x7fbe700fcba0 2024-02-18T00:33:23.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:20.936256+0000 mgr.y (mgr.24454) 173 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:23.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.108710+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:23.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.108869+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:21.109033+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.123570+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.128214+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? 172.21.15.59:0/130929383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.135312+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.59:0/3622381454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.152949+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.154080+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:21.272433+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:20.936256+0000 mgr.y (mgr.24454) 173 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.108710+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.108869+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:21.109033+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-18T00:33:23.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.123570+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:23.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.128214+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? 172.21.15.59:0/130929383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.135312+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.59:0/3622381454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.152949+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:23.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.154080+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:21.272433+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:23.323 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:23.331 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:23.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.398+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 7 ==== osd_map(126..126 src has 44..126) v4 ==== 1787+0+0 (secure 0 0 0) 0x7fbe6805a890 con 0x7fbe700fcba0 2024-02-18T00:33:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.398+0000 7fbe6cf74700 10 client.27173.objecter ms_dispatch 0x7fbe70001550 osd_map(126..126 src has 44..126) v4 2024-02-18T00:33:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.398+0000 7fbe6cf74700 3 client.27173.objecter handle_osd_map got epochs [126,126] > 125 2024-02-18T00:33:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.398+0000 7fbe6cf74700 3 client.27173.objecter handle_osd_map decoding incremental epoch 126 2024-02-18T00:33:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.398+0000 7fbe6cf74700 10 client.27173.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.398+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=127}) v3 -- 0x7fbe58075850 con 0x7fbe700fcba0 2024-02-18T00:33:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.398+0000 7fbe6cf74700 20 client.27173.objecter dump_active .. 0 homeless 2024-02-18T00:33:23.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:20.936256+0000 mgr.y (mgr.24454) 173 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.108710+0000 mon.a (mon.0) 1022 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.108869+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:21.109033+0000 mon.a (mon.0) 1024 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.123570+0000 mon.b (mon.1) 108 : audit [INF] from='client.? 172.21.15.59:0/1718434587' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.128214+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? 172.21.15.59:0/130929383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.135312+0000 mon.b (mon.1) 109 : audit [INF] from='client.? 172.21.15.59:0/3622381454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.152949+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]: dispatch 2024-02-18T00:33:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.154080+0000 mon.a (mon.0) 1027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:21.272433+0000 mon.b (mon.1) 110 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:23.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:23.613+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fbe5c0054d0 con 0x7fbe700fcba0 2024-02-18T00:33:23.668 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:23.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-18T00:33:23.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:23.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ OK ] LibRadosService.StatusFormat (7493 ms) 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ OK ] LibRadosService.Status (20139 ms) 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [----------] 4 tests from LibRadosService (32760 ms total) 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [----------] Global test environment tear-down 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [==========] 4 tests from 1 test suite ran. (32760 ms total) 2024-02-18T00:33:23.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_service: [ PASSED ] 4 tests. 2024-02-18T00:33:23.953 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:23.956 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:23.957 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:24.058 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:24.096 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:33:24.097 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:33:24.136 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:24.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:24.649+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 8 ==== osd_map(127..127 src has 44..127) v4 ==== 2795+0+0 (secure 0 0 0) 0x7fbe6802fdf0 con 0x7fbe700fcba0 2024-02-18T00:33:24.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:24.649+0000 7fbe6cf74700 10 client.27173.objecter ms_dispatch 0x7fbe70001550 osd_map(127..127 src has 44..127) v4 2024-02-18T00:33:24.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:24.649+0000 7fbe6cf74700 3 client.27173.objecter handle_osd_map got epochs [127,127] > 126 2024-02-18T00:33:24.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:24.649+0000 7fbe6cf74700 3 client.27173.objecter handle_osd_map decoding incremental epoch 127 2024-02-18T00:33:24.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:24.649+0000 7fbe6cf74700 20 client.27173.objecter dump_active .. 0 homeless 2024-02-18T00:33:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:21.779256+0000 mgr.y (mgr.24454) 174 : cluster [DBG] pgmap v173: 498 pgs: 32 creating+peering, 64 unknown, 402 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 3.5 KiB/s wr, 3 op/s 2024-02-18T00:33:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.273649+0000 mon.b (mon.1) 111 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.378004+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.59:0/130929383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.378087+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.378185+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.396729+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.416733+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.59:0/2043219380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:22.976262+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:22.986322+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.990424+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.993627+0000 mon.a (mon.0) 1034 : audit [DBG] from='client.? 172.21.15.59:0/3430214106' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.997284+0000 mgr.y (mgr.24454) 175 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:22.998408+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:23.274622+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:23.374865+0000 mon.a (mon.0) 1036 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:23.381834+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:23.381978+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:23.382077+0000 mon.a (mon.0) 1039 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:23.387194+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.59:0/1950635727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:23.390543+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:23.535495+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:23.618137+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.59:0/68804166' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.064065+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:21.779256+0000 mgr.y (mgr.24454) 174 : cluster [DBG] pgmap v173: 498 pgs: 32 creating+peering, 64 unknown, 402 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 3.5 KiB/s wr, 3 op/s 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.273649+0000 mon.b (mon.1) 111 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.378004+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.59:0/130929383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.378087+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:24.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.378185+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.396729+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.416733+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.59:0/2043219380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:22.976262+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:22.986322+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.990424+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.993627+0000 mon.a (mon.0) 1034 : audit [DBG] from='client.? 172.21.15.59:0/3430214106' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.997284+0000 mgr.y (mgr.24454) 175 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:22.998408+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:23.274622+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:23.374865+0000 mon.a (mon.0) 1036 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:23.381834+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:23.381978+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:23.382077+0000 mon.a (mon.0) 1039 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-18T00:33:24.758 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:23.387194+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.59:0/1950635727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:23.390543+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:24.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:23.535495+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:24.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:23.618137+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.59:0/68804166' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:24.759 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.064065+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:21.779256+0000 mgr.y (mgr.24454) 174 : cluster [DBG] pgmap v173: 498 pgs: 32 creating+peering, 64 unknown, 402 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 3.5 KiB/s wr, 3 op/s 2024-02-18T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.273649+0000 mon.b (mon.1) 111 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.378004+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.59:0/130929383' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.378087+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi059-83005-12"}]': finished 2024-02-18T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.378185+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi059-82877-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.396729+0000 mon.c (mon.2) 62 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.416733+0000 mon.b (mon.1) 112 : audit [INF] from='client.? 172.21.15.59:0/2043219380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:22.976262+0000 mon.a (mon.0) 1031 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:22.986322+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.990424+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.993627+0000 mon.a (mon.0) 1034 : audit [DBG] from='client.? 172.21.15.59:0/3430214106' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.997284+0000 mgr.y (mgr.24454) 175 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:22.998408+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:23.274622+0000 mon.b (mon.1) 113 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:23.374865+0000 mon.a (mon.0) 1036 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:23.381834+0000 mon.a (mon.0) 1037 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:23.381978+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi059-82981-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:23.382077+0000 mon.a (mon.0) 1039 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:23.387194+0000 mon.b (mon.1) 114 : audit [INF] from='client.? 172.21.15.59:0/1950635727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:23.390543+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:23.535495+0000 mon.c (mon.2) 63 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:25.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:23.618137+0000 mon.c (mon.2) 64 : audit [INF] from='client.? 172.21.15.59:0/68804166' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:25.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.064065+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:25.436 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:25.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:25.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:25.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-18T00:33:25.541 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:25.619 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:23.785637+0000 mgr.y (mgr.24454) 176 : cluster [DBG] pgmap v176: 498 pgs: 32 creating+peering, 96 unknown, 370 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 3 op/s 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:24.241089+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:24.241196+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.299659+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.632360+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.650919+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.651162+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:24.652600+0000 mon.a (mon.0) 1045 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.659507+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:25.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.667076+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.59:0/3631926661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.667876+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:25.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:24.674356+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:25.301061+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.653 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:25.335793+0000 mon.a (mon.0) 1048 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:25.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.712+0000 7fbe6cf74700 1 -- 172.21.15.59:0/68804166 <== mon.2 v2:172.21.15.59:3301/0 9 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbe6805a250 con 0x7fbe700fcba0 2024-02-18T00:33:25.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.717+0000 7fbe75a97700 10 librados: watch_flush enter 2024-02-18T00:33:25.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.717+0000 7fbe75a97700 10 librados: watch_flush exit 2024-02-18T00:33:25.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.717+0000 7fbe75a97700 20 client.27173.objecter shutdown clearing up homeless session... 2024-02-18T00:33:25.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.717+0000 7fbe75a97700 10 client.27173.objecter successfully canceled tick 2024-02-18T00:33:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.729+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbe58062d10 msgr2=0x7fbe580651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.729+0000 7fbe75a97700 1 --2- 172.21.15.59:0/68804166 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbe58062d10 0x7fbe580651d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fbe64009640 tx=0x7fbe64006b60).stop 2024-02-18T00:33:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.729+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe700fcba0 msgr2=0x7fbe7011c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:25.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.729+0000 7fbe75a97700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe700fcba0 0x7fbe7011c790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7fbe6800acc0 tx=0x7fbe68004d10).stop 2024-02-18T00:33:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.743+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 shutdown_connections 2024-02-18T00:33:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.743+0000 7fbe75a97700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe700fcba0 0x7fbe7011c790 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.743+0000 7fbe75a97700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe700f6b10 0x7fbe700653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.743+0000 7fbe75a97700 1 --2- 172.21.15.59:0/68804166 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbe58062d10 0x7fbe580651d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.743+0000 7fbe75a97700 1 --2- 172.21.15.59:0/68804166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe700fc240 0x7fbe700658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:25.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.743+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 >> 172.21.15.59:0/68804166 conn(0x7fbe7006c370 msgr2=0x7fbe700f5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.745+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 shutdown_connections 2024-02-18T00:33:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.747+0000 7fbe75a97700 1 -- 172.21.15.59:0/68804166 wait complete. 2024-02-18T00:33:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:25.751+0000 7fbe75a97700 1 librados: shutdown 2024-02-18T00:33:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:25.785 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:33:25.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:25.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:25.903 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:25.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:23.785637+0000 mgr.y (mgr.24454) 176 : cluster [DBG] pgmap v176: 498 pgs: 32 creating+peering, 96 unknown, 370 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 3 op/s 2024-02-18T00:33:25.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:24.241089+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:25.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:24.241196+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:25.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.299659+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.632360+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:25.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.650919+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:25.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.651162+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:25.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:24.652600+0000 mon.a (mon.0) 1045 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-18T00:33:25.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.659507+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:25.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.667076+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.59:0/3631926661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.667876+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:25.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:24.674356+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:25.301061+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:25.335793+0000 mon.a (mon.0) 1048 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:25.929 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:25.947 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:25.950 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:25.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:25.975 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:23.785637+0000 mgr.y (mgr.24454) 176 : cluster [DBG] pgmap v176: 498 pgs: 32 creating+peering, 96 unknown, 370 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 3 op/s 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:24.241089+0000 osd.6 (osd.6) 69 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:24.241196+0000 osd.6 (osd.6) 70 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.299659+0000 mon.b (mon.1) 115 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.632360+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.650919+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.651162+0000 mon.a (mon.0) 1044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:24.652600+0000 mon.a (mon.0) 1045 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.659507+0000 mon.c (mon.2) 65 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.667076+0000 mon.b (mon.1) 116 : audit [INF] from='client.? 172.21.15.59:0/3631926661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.667876+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:24.674356+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:25.301061+0000 mon.b (mon.1) 117 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:25.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:25.335793+0000 mon.a (mon.0) 1048 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:26.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:26.077 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:26.096 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:26.098 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:26.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:26.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:26.308 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:26.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:33:26.403 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:33:26.404 INFO:tasks.thrashosds.thrasher:Shrinking pool .rgw.root 2024-02-18T00:33:26.404 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .rgw.root by 10 min 8 2024-02-18T00:33:26.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:33:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.501+0000 7fed5f951700 1 -- 172.21.15.59:0/275402758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed580fe510 msgr2=0x7fed580f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.501+0000 7fed5f951700 1 --2- 172.21.15.59:0/275402758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed580fe510 0x7fed580f7d40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7fed48004870 tx=0x7fed48009cb0).stop 2024-02-18T00:33:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.501+0000 7fed5f951700 1 -- 172.21.15.59:0/275402758 shutdown_connections 2024-02-18T00:33:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.501+0000 7fed5f951700 1 --2- 172.21.15.59:0/275402758 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed580f8420 0x7fed58104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.501+0000 7fed5f951700 1 --2- 172.21.15.59:0/275402758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed580fdbb0 0x7fed580fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.501+0000 7fed5f951700 1 --2- 172.21.15.59:0/275402758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed580fe510 0x7fed580f7d40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.501+0000 7fed5f951700 1 -- 172.21.15.59:0/275402758 >> 172.21.15.59:0/275402758 conn(0x7fed580f3800 msgr2=0x7fed580f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:26.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.540+0000 7fed5f951700 1 -- 172.21.15.59:0/275402758 shutdown_connections 2024-02-18T00:33:26.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.540+0000 7fed5f951700 1 -- 172.21.15.59:0/275402758 wait complete. 2024-02-18T00:33:26.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.540+0000 7fed5f951700 1 librados: starting msgr at 2024-02-18T00:33:26.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.540+0000 7fed5f951700 1 librados: starting objecter 2024-02-18T00:33:26.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 Processor -- start 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 -- start start 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 librados: setting wanted keys 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 librados: calling monclient init 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed580f8420 0x7fed58063310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed580fdbb0 0x7fed58063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed580fe510 0x7fed58109740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed58109c30 con 0x7fed580f8420 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed58109da0 con 0x7fed580fdbb0 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.541+0000 7fed5f951700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed58109f60 con 0x7fed580fe510 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.542+0000 7fed5deee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed580fe510 0x7fed58109740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.542+0000 7fed5deee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed580fe510 0x7fed58109740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:36530/0 (socket says 172.21.15.59:36530) 2024-02-18T00:33:26.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.542+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 learned_addr learned my addr 172.21.15.59:0/207470252 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:26.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.547+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed580fdbb0 msgr2=0x7fed58063800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T00:33:26.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.547+0000 7fed5deee700 1 --2- 172.21.15.59:0/207470252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed580fdbb0 0x7fed58063800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:26.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.547+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed580f8420 msgr2=0x7fed58063310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.547+0000 7fed5deee700 1 --2- 172.21.15.59:0/207470252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed580f8420 0x7fed58063310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:26.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.547+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48005040 con 0x7fed580fe510 2024-02-18T00:33:26.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.547+0000 7fed5deee700 1 --2- 172.21.15.59:0/207470252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed580fe510 0x7fed58109740 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fed5000bc30 tx=0x7fed50009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.549+0000 7fed4e7fc700 10 client.?.objecter ms_handle_connect 0x7fed580fe510 2024-02-18T00:33:26.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.549+0000 7fed4e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.560+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5000c7a0 con 0x7fed580fe510 2024-02-18T00:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.560+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed50010070 con 0x7fed580fe510 2024-02-18T00:33:26.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.560+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50007920 con 0x7fed580fe510 2024-02-18T00:33:26.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.564+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5810a240 con 0x7fed580fe510 2024-02-18T00:33:26.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.564+0000 7fed5f951700 10 client.28754.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:26.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.564+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5810a6e0 con 0x7fed580fe510 2024-02-18T00:33:26.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.564+0000 7fed5f951700 1 librados: init done 2024-02-18T00:33:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.568+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fed3c004f30 con 0x7fed580fe510 2024-02-18T00:33:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fed50007cd0 con 0x7fed580fe510 2024-02-18T00:33:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed44062d20 0x7fed440651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(128..128 src has 44..128) v4 ==== 11038+0+0 (secure 0 0 0) 0x7fed5007d8a0 con 0x7fed580fe510 2024-02-18T00:33:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 osd_map(128..128 src has 44..128) v4 2024-02-18T00:33:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 3 client.28754.objecter handle_osd_map got epochs [128,128] > 0 2024-02-18T00:33:26.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 3 client.28754.objecter handle_osd_map decoding full epoch 128 2024-02-18T00:33:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 20 client.28754.objecter dump_active .. 0 homeless 2024-02-18T00:33:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.585+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v128) v1 ==== 54+0+16 (secure 0 0 0) 0x7fed5007dbc0 con 0x7fed580fe510 2024-02-18T00:33:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.587+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed44062d20 0x7fed440651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.587+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed44062d20 0x7fed440651e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fed54009640 tx=0x7fed54006b60).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.587+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed44062d20 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 10 client.28754.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 1 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6801/3700238977 conn(0x7fed3c004560 0x7fed3c0049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 1 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 0 1 2024-02-18T00:33:26.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0051d0 osd=0 4 2024-02-18T00:33:26.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 10 client.28754.objecter _send_command 1 2024-02-18T00:33:26.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.596+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c008960 con 0x7fed3c004560 2024-02-18T00:33:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.600+0000 7fed5ceec700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6801/3700238977 conn(0x7fed3c004560 0x7fed3c0049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.601+0000 7fed5ceec700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6801/3700238977 conn(0x7fed3c004560 0x7fed3c0049a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.601+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c004560 2024-02-18T00:33:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.602+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed3c008960 con 0x7fed3c004560 2024-02-18T00:33:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.602+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:33:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.602+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 1 = system:0 2024-02-18T00:33:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.602+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c0051d0 osd=0 4 2024-02-18T00:33:26.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.602+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 0 1 2024-02-18T00:33:26.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 10 client.28754.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 2 2024-02-18T00:33:26.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 2 2024-02-18T00:33:26.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c0051d0 osd=0 3 2024-02-18T00:33:26.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 0 2 2024-02-18T00:33:26.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0051d0 osd=0 4 2024-02-18T00:33:26.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 10 client.28754.objecter _send_command 2 2024-02-18T00:33:26.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.626+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c0081f0 con 0x7fed3c004560 2024-02-18T00:33:26.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.630+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c0081f0 con 0x7fed3c004560 2024-02-18T00:33:26.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.630+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.630+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.630+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c0051d0 osd=0 4 2024-02-18T00:33:26.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.630+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 0 2 2024-02-18T00:33:26.633 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:33:26.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 10 client.28754.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 3 2024-02-18T00:33:26.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6805/26279110 conn(0x7fed3c002c90 0x7fed3c003100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 3 2024-02-18T00:33:26.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 1 3 2024-02-18T00:33:26.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0085e0 osd=1 4 2024-02-18T00:33:26.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 10 client.28754.objecter _send_command 3 2024-02-18T00:33:26.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.631+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c003620 con 0x7fed3c002c90 2024-02-18T00:33:26.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.633+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6805/26279110 conn(0x7fed3c002c90 0x7fed3c003100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.634+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6805/26279110 conn(0x7fed3c002c90 0x7fed3c003100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.634+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c002c90 2024-02-18T00:33:26.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.635+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed3c003620 con 0x7fed3c002c90 2024-02-18T00:33:26.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.635+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:33:26.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.635+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 3 = system:0 2024-02-18T00:33:26.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.635+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c0085e0 osd=1 4 2024-02-18T00:33:26.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.635+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 1 3 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 10 client.28754.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 4 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 4 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c0085e0 osd=1 3 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 1 4 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0085e0 osd=1 4 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 10 client.28754.objecter _send_command 4 2024-02-18T00:33:26.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.667+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c007d20 con 0x7fed3c002c90 2024-02-18T00:33:26.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c007d20 con 0x7fed3c002c90 2024-02-18T00:33:26.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c0085e0 osd=1 4 2024-02-18T00:33:26.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 1 4 2024-02-18T00:33:26.672 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:33:26.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed5f951700 10 client.28754.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 5 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.669+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6809/1153193637 conn(0x7fed3c004180 0x7fed3c00d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 5 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 2 5 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c004010 osd=2 4 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 10 client.28754.objecter _send_command 5 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.670+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c00ff50 con 0x7fed3c004180 2024-02-18T00:33:26.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.671+0000 7fed5ceec700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6809/1153193637 conn(0x7fed3c004180 0x7fed3c00d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.671+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c004180 2024-02-18T00:33:26.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.671+0000 7fed5ceec700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6809/1153193637 conn(0x7fed3c004180 0x7fed3c00d090 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.674+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed3c00ff50 con 0x7fed3c004180 2024-02-18T00:33:26.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.674+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:33:26.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.674+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 5 = system:0 2024-02-18T00:33:26.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.674+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c004010 osd=2 4 2024-02-18T00:33:26.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.674+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 2 5 2024-02-18T00:33:26.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 10 client.28754.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 6 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 6 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c004010 osd=2 3 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 2 6 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c004010 osd=2 4 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 10 client.28754.objecter _send_command 6 2024-02-18T00:33:26.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.695+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c003a80 con 0x7fed3c004180 2024-02-18T00:33:26.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.698+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c003a80 con 0x7fed3c004180 2024-02-18T00:33:26.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.699+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.699+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.699+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c004010 osd=2 4 2024-02-18T00:33:26.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.699+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 2 6 2024-02-18T00:33:26.717 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 10 client.28754.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 7 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6813/1805700964 conn(0x7fed3c0020c0 0x7fed3c00e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 7 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 3 7 2024-02-18T00:33:26.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c003e30 osd=3 4 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 10 client.28754.objecter _send_command 7 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.701+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c00eea0 con 0x7fed3c0020c0 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.705+0000 7fed5deee700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6813/1805700964 conn(0x7fed3c0020c0 0x7fed3c00e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.705+0000 7fed5deee700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6813/1805700964 conn(0x7fed3c0020c0 0x7fed3c00e820 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.705+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c0020c0 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.706+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed3c00eea0 con 0x7fed3c0020c0 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.706+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.706+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 7 = system:0 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.706+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c003e30 osd=3 4 2024-02-18T00:33:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.706+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 3 7 2024-02-18T00:33:26.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 10 client.28754.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 8 2024-02-18T00:33:26.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 8 2024-02-18T00:33:26.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c003e30 osd=3 3 2024-02-18T00:33:26.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 3 8 2024-02-18T00:33:26.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c003e30 osd=3 4 2024-02-18T00:33:26.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 10 client.28754.objecter _send_command 8 2024-02-18T00:33:26.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.751+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c001680 con 0x7fed3c0020c0 2024-02-18T00:33:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.754+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c001680 con 0x7fed3c0020c0 2024-02-18T00:33:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.754+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.754+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.754+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c003e30 osd=3 4 2024-02-18T00:33:26.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.754+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 3 8 2024-02-18T00:33:26.762 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:33:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 10 client.28754.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 9 2024-02-18T00:33:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6800/2044387056 conn(0x7fed3c012180 0x7fed3c014570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 9 2024-02-18T00:33:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 4 9 2024-02-18T00:33:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c001440 osd=4 4 2024-02-18T00:33:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 10 client.28754.objecter _send_command 9 2024-02-18T00:33:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.760+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c001670 con 0x7fed3c012180 2024-02-18T00:33:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.761+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6800/2044387056 conn(0x7fed3c012180 0x7fed3c014570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.761+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6800/2044387056 conn(0x7fed3c012180 0x7fed3c014570 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.761+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c012180 2024-02-18T00:33:26.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.762+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed5401dba0 con 0x7fed3c012180 2024-02-18T00:33:26.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.762+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:33:26.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.762+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 9 = system:0 2024-02-18T00:33:26.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.762+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c001440 osd=4 4 2024-02-18T00:33:26.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.762+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 4 9 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 10 client.28754.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 10 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 10 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c001440 osd=4 3 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 4 10 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c001440 osd=4 4 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 10 client.28754.objecter _send_command 10 2024-02-18T00:33:26.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.795+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c014a60 con 0x7fed3c012180 2024-02-18T00:33:26.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.804+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c014a60 con 0x7fed3c012180 2024-02-18T00:33:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.804+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.804+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.804+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c001440 osd=4 4 2024-02-18T00:33:26.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.804+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 4 10 2024-02-18T00:33:26.806 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 10 client.28754.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 11 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6804/2170432263 conn(0x7fed3c016860 0x7fed3c016ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 11 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 5 11 2024-02-18T00:33:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0161e0 osd=5 4 2024-02-18T00:33:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 10 client.28754.objecter _send_command 11 2024-02-18T00:33:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.805+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c01a430 con 0x7fed3c016860 2024-02-18T00:33:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.806+0000 7fed5ceec700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6804/2170432263 conn(0x7fed3c016860 0x7fed3c016ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.806+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c016860 2024-02-18T00:33:26.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.806+0000 7fed5ceec700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6804/2170432263 conn(0x7fed3c016860 0x7fed3c016ca0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.807+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed3c01a430 con 0x7fed3c016860 2024-02-18T00:33:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.807+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:33:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.807+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 11 = system:0 2024-02-18T00:33:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.807+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c0161e0 osd=5 4 2024-02-18T00:33:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.807+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 5 11 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 10 client.28754.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 12 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 12 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c0161e0 osd=5 3 2024-02-18T00:33:26.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 5 12 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0161e0 osd=5 4 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 10 client.28754.objecter _send_command 12 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.840+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c01a430 con 0x7fed3c016860 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.841+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c01a430 con 0x7fed3c016860 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.841+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.841+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.841+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c0161e0 osd=5 4 2024-02-18T00:33:26.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.841+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 5 12 2024-02-18T00:33:26.844 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:33:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 10 client.28754.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 13 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6808/3356009390 conn(0x7fed3c019820 0x7fed3c019c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 13 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 6 13 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c01a430 osd=6 4 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 10 client.28754.objecter _send_command 13 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c01d7e0 con 0x7fed3c019820 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.843+0000 7fed5deee700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6808/3356009390 conn(0x7fed3c019820 0x7fed3c019c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.844+0000 7fed5deee700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6808/3356009390 conn(0x7fed3c019820 0x7fed3c019c60 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.844+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c019820 2024-02-18T00:33:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.844+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed3c01d7e0 con 0x7fed3c019820 2024-02-18T00:33:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.844+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:33:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.844+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 13 = system:0 2024-02-18T00:33:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.844+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c01a430 osd=6 4 2024-02-18T00:33:26.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.844+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 6 13 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 10 client.28754.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 14 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 14 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c01a430 osd=6 3 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 6 14 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c01a430 osd=6 4 2024-02-18T00:33:26.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 10 client.28754.objecter _send_command 14 2024-02-18T00:33:26.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.880+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c014b20 con 0x7fed3c019820 2024-02-18T00:33:26.889 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:33:26.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.886+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c014b20 con 0x7fed3c019820 2024-02-18T00:33:26.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.886+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.886+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.886+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c01a430 osd=6 4 2024-02-18T00:33:26.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.886+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 6 14 2024-02-18T00:33:26.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 10 client.28754.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 15 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6812/53025171 conn(0x7fed3c0151d0 0x7fed3c015630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 15 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 7 15 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c014b20 osd=7 4 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 10 client.28754.objecter _send_command 15 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.890+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed3c015c50 con 0x7fed3c0151d0 2024-02-18T00:33:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.898+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6812/53025171 conn(0x7fed3c0151d0 0x7fed3c015630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.902+0000 7fed5d6ed700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6812/53025171 conn(0x7fed3c0151d0 0x7fed3c015630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.902+0000 7fed4e7fc700 10 client.28754.objecter ms_handle_connect 0x7fed3c0151d0 2024-02-18T00:33:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.904+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fed3c015c50 con 0x7fed3c0151d0 2024-02-18T00:33:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.904+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:33:26.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.904+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 15 = system:0 2024-02-18T00:33:26.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.904+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c014b20 osd=7 4 2024-02-18T00:33:26.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.904+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 7 15 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 10 client.28754.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign -1 16 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 20 client.28754.objecter _get_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 15 client.28754.objecter _session_command_op_remove -1 16 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 20 client.28754.objecter get_session s=0x7fed3c014b20 osd=7 3 2024-02-18T00:33:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 15 client.28754.objecter _session_command_op_assign 7 16 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c014b20 osd=7 4 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 10 client.28754.objecter _send_command 16 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.922+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fed3c015c50 con 0x7fed3c0151d0 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.927+0000 7fed4e7fc700 1 -- 172.21.15.59:0/207470252 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fed3c015c50 con 0x7fed3c0151d0 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.927+0000 7fed4e7fc700 10 client.28754.objecter ms_dispatch 0x7fed58001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.927+0000 7fed4e7fc700 10 client.28754.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.927+0000 7fed4e7fc700 20 client.28754.objecter put_session s=0x7fed3c014b20 osd=7 4 2024-02-18T00:33:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.927+0000 7fed4e7fc700 15 client.28754.objecter _session_command_op_remove 7 16 2024-02-18T00:33:26.934 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 librados: watch_flush enter 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 librados: watch_flush exit 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.0 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6801/3700238977 conn(0x7fed3c004560 msgr2=0x7fed3c0049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6801/3700238977 conn(0x7fed3c004560 0x7fed3c0049a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0051d0 osd=0 1 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.1 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6805/26279110 conn(0x7fed3c002c90 msgr2=0x7fed3c003100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6805/26279110 conn(0x7fed3c002c90 0x7fed3c003100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0085e0 osd=1 1 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.2 2024-02-18T00:33:26.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6809/1153193637 conn(0x7fed3c004180 msgr2=0x7fed3c00d090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6809/1153193637 conn(0x7fed3c004180 0x7fed3c00d090 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c004010 osd=2 1 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.3 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6813/1805700964 conn(0x7fed3c0020c0 msgr2=0x7fed3c00e820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6813/1805700964 conn(0x7fed3c0020c0 0x7fed3c00e820 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c003e30 osd=3 1 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.4 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6800/2044387056 conn(0x7fed3c012180 msgr2=0x7fed3c014570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6800/2044387056 conn(0x7fed3c012180 0x7fed3c014570 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c001440 osd=4 1 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.5 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6804/2170432263 conn(0x7fed3c016860 msgr2=0x7fed3c016ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6804/2170432263 conn(0x7fed3c016860 0x7fed3c016ca0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c0161e0 osd=5 1 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.6 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6808/3356009390 conn(0x7fed3c019820 msgr2=0x7fed3c019c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6808/3356009390 conn(0x7fed3c019820 0x7fed3c019c60 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c01a430 osd=6 1 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter close_session for osd.7 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6812/53025171 conn(0x7fed3c0151d0 msgr2=0x7fed3c015630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.137:6812/53025171 conn(0x7fed3c0151d0 0x7fed3c015630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter put_session s=0x7fed3c014b20 osd=7 1 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 20 client.28754.objecter shutdown clearing up homeless session... 2024-02-18T00:33:26.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.932+0000 7fed5f951700 10 client.28754.objecter successfully canceled tick 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed44062d20 msgr2=0x7fed440651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed44062d20 0x7fed440651e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fed54009640 tx=0x7fed54006b60).stop 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed580fe510 msgr2=0x7fed58109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5f951700 1 --2- 172.21.15.59:0/207470252 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed580fe510 0x7fed58109740 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fed5000bc30 tx=0x7fed50009840).stop 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 reap_dead start 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 reap_dead start 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 reap_dead start 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 reap_dead start 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 reap_dead start 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5deee700 1 -- 172.21.15.59:0/207470252 reap_dead start 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 shutdown_connections 2024-02-18T00:33:26.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.933+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 >> 172.21.15.59:0/207470252 conn(0x7fed580f3800 msgr2=0x7fed580f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:26.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.936+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 shutdown_connections 2024-02-18T00:33:26.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.936+0000 7fed5f951700 1 -- 172.21.15.59:0/207470252 wait complete. 2024-02-18T00:33:26.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:26.942+0000 7fed5f951700 1 librados: shutdown 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:33:26.948 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:25.649246+0000 mon.a (mon.0) 1049 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:25.660659+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:25.660720+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]': finished 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:25.660818+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:25.661027+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:25.688478+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? 172.21.15.59:0/3676653592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:25.791082+0000 mgr.y (mgr.24454) 177 : cluster [DBG] pgmap v179: 498 pgs: 30 creating+peering, 27 creating+activating, 2 unknown, 439 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:25.909865+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:27.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:26.097374+0000 osd.2 (osd.2) 65 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:26.097457+0000 osd.2 (osd.2) 66 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:26.313718+0000 mon.b (mon.1) 118 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:27.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:26.583310+0000 mon.c (mon.2) 67 : audit [DBG] from='client.? 172.21.15.59:0/207470252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:25.649246+0000 mon.a (mon.0) 1049 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:25.660659+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:25.660720+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]': finished 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:25.660818+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:25.661027+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:25.688478+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? 172.21.15.59:0/3676653592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:25.791082+0000 mgr.y (mgr.24454) 177 : cluster [DBG] pgmap v179: 498 pgs: 30 creating+peering, 27 creating+activating, 2 unknown, 439 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T00:33:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:25.909865+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:27.260 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:26.097374+0000 osd.2 (osd.2) 65 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:27.260 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:26.097457+0000 osd.2 (osd.2) 66 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:27.260 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:26.313718+0000 mon.b (mon.1) 118 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:27.260 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:26.583310+0000 mon.c (mon.2) 67 : audit [DBG] from='client.? 172.21.15.59:0/207470252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:27.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:25.649246+0000 mon.a (mon.0) 1049 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:27.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:25.660659+0000 mon.a (mon.0) 1050 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:33:27.260 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:25.660720+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-7"}]': finished 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:25.660818+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi059-82877-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:25.661027+0000 mon.a (mon.0) 1053 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:25.688478+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? 172.21.15.59:0/3676653592' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:25.791082+0000 mgr.y (mgr.24454) 177 : cluster [DBG] pgmap v179: 498 pgs: 30 creating+peering, 27 creating+activating, 2 unknown, 439 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:25.909865+0000 mon.c (mon.2) 66 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:26.097374+0000 osd.2 (osd.2) 65 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:26.097457+0000 osd.2 (osd.2) 66 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:26.313718+0000 mon.b (mon.1) 118 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:27.261 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:26.583310+0000 mon.c (mon.2) 67 : audit [DBG] from='client.? 172.21.15.59:0/207470252' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:27.464 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:27.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:27.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:27.569 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:27.595 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:27.614 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:27.616 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:27.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:27.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:28.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:26.961292+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:28.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.000078+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.59:0/3676653592' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:28.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.000370+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:27.000449+0000 mon.a (mon.0) 1058 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.015018+0000 mon.b (mon.1) 119 : audit [INF] from='client.? 172.21.15.59:0/158288180' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.015864+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.017101+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.017991+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.035722+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.59:0/157459887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.036650+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:27.319951+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:27.364696+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:28.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:27.364821+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:26.961292+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.000078+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.59:0/3676653592' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.000370+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:27.000449+0000 mon.a (mon.0) 1058 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.015018+0000 mon.b (mon.1) 119 : audit [INF] from='client.? 172.21.15.59:0/158288180' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.015864+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.017101+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.017991+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.035722+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.59:0/157459887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.036650+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:27.319951+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:27.364696+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:28.073 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:27.364821+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:28.074 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:28.074 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:28.095 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:28.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:28.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:28.159 INFO:tasks.workunit.client.0.smithi059.stdout: }, 2024-02-18T00:33:28.159 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "task_status": {} 2024-02-18T00:33:28.159 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:28.159 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: attempt 1 of 20 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: failed to deregister: 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: { 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "epoch": 20, 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "modified": "2024-02-18T00:33:23.785267+0000", 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "services": { 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "laundry": { 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "daemons": { 2024-02-18T00:33:28.160 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "summary": "", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "close-test-pid83045": { 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_epoch": 20, 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_stamp": "2024-02-18T00:33:23.044043+0000", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "gid": 27098, 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "addr": "172.21.15.59:0/2784234171", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "metadata": { 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro": "centos", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-18T00:33:28.161 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_version": "8", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "foo": "bar", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "hostname": "smithi059", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_description": "#1 SMP Mon Feb 5 18:34:56 UTC 2024", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_version": "4.18.0-539.el8.x86_64", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_total_kb": "32543852", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "os": "Linux", 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "this": "that" 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: }, 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "task_status": {} 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: }, 2024-02-18T00:33:28.162 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "rgw": { 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "daemons": { 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "summary": "", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "24347": { 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_epoch": 12, 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "start_stamp": "2024-02-18T00:29:22.198518+0000", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "gid": 24347, 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "addr": "172.21.15.59:0/2522576766", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "metadata": { 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "container_hostname": "smithi059", 2024-02-18T00:33:28.163 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9", 2024-02-18T00:33:28.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:26.961292+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.000078+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.59:0/3676653592' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.000370+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:27.000449+0000 mon.a (mon.0) 1058 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.015018+0000 mon.b (mon.1) 119 : audit [INF] from='client.? 172.21.15.59:0/158288180' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.015864+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.017101+0000 mon.c (mon.2) 68 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.017991+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]: dispatch 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.035722+0000 mon.c (mon.2) 69 : audit [INF] from='client.? 172.21.15.59:0/157459887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.036650+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:28.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:27.319951+0000 mon.b (mon.1) 120 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:28.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:27.364696+0000 osd.4 (osd.4) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:28.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:27.364821+0000 osd.4 (osd.4) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:28.336 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:28.476 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:28.482 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:28.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:28.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:28.609 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:28.628 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:28.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:28.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:28.732 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:28.764 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:28.783 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:28.785 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:28.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:28.886 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:28.913 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:28.932 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:28.935 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:28.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:29.036 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:29.062 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:29.081 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:29.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:29.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:29.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:33:29.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:29.300 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_p api_c_write_operations: Running main() from gmock_main.cc 2024-02-18T00:33:29.300 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (13606 ms) 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (5655 ms) 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3935 ms) 2024-02-18T00:33:29.301 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3427 ms) 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3599 ms) 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4532 ms) 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3627 ms) 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (38381 ms total) 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (38381 ms total) 2024-02-18T00:33:29.302 INFO:tasks.workunit.client.0.smithi059.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-18T00:33:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:29.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:29.449 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:29.450 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:27.792188+0000 mgr.y (mgr.24454) 178 : cluster [DBG] pgmap v181: 530 pgs: 30 creating+peering, 66 unknown, 434 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.018277+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.018372+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]': finished 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.018444+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:28.018586+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.064575+0000 mon.b (mon.1) 121 : audit [INF] from='client.? 172.21.15.59:0/4028720008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.087705+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.093778+0000 mon.b (mon.1) 122 : audit [DBG] from='client.? 172.21.15.59:0/3357798051' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.095039+0000 mgr.y (mgr.24454) 179 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.316981+0000 mon.b (mon.1) 123 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]: dispatch 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:28.321804+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:29.451 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.260815+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]: dispatch 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:27.792188+0000 mgr.y (mgr.24454) 178 : cluster [DBG] pgmap v181: 530 pgs: 30 creating+peering, 66 unknown, 434 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.018277+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.018372+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]': finished 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.018444+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:28.018586+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.064575+0000 mon.b (mon.1) 121 : audit [INF] from='client.? 172.21.15.59:0/4028720008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.087705+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:29.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.093778+0000 mon.b (mon.1) 122 : audit [DBG] from='client.? 172.21.15.59:0/3357798051' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:29.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.095039+0000 mgr.y (mgr.24454) 179 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:29.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.316981+0000 mon.b (mon.1) 123 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]: dispatch 2024-02-18T00:33:29.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:28.321804+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:29.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.260815+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]: dispatch 2024-02-18T00:33:29.550 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:29.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:27.792188+0000 mgr.y (mgr.24454) 178 : cluster [DBG] pgmap v181: 530 pgs: 30 creating+peering, 66 unknown, 434 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-18T00:33:29.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.018277+0000 mon.a (mon.0) 1062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-83053-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:29.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.018372+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-7"}]': finished 2024-02-18T00:33:29.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.018444+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:29.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:28.018586+0000 mon.a (mon.0) 1065 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-18T00:33:29.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.064575+0000 mon.b (mon.1) 121 : audit [INF] from='client.? 172.21.15.59:0/4028720008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:29.576 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.087705+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:29.577 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.093778+0000 mon.b (mon.1) 122 : audit [DBG] from='client.? 172.21.15.59:0/3357798051' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:29.577 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.095039+0000 mgr.y (mgr.24454) 179 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:29.577 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.316981+0000 mon.b (mon.1) 123 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]: dispatch 2024-02-18T00:33:29.577 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:28.321804+0000 mon.b (mon.1) 124 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:29.577 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.260815+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]: dispatch 2024-02-18T00:33:29.578 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:29.597 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:29.599 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:29.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:29.701 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:29.851 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:29.892 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:29.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:29.993 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:30.020 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:30.039 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:30.041 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:30.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:30.142 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:30.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:33:30.237 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:30.352 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:30.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:30.364 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:30.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:30.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.286625+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.286798+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]': finished 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:29.286865+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.292915+0000 mon.b (mon.1) 125 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.303843+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.320956+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.59:0/3002694168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.331365+0000 mon.b (mon.1) 127 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.373384+0000 mon.b (mon.1) 128 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:29.792872+0000 mgr.y (mgr.24454) 180 : cluster [DBG] pgmap v184: 498 pgs: 128 unknown, 370 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 744 B/s wr, 2 op/s 2024-02-18T00:33:30.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.937440+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:30.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.941520+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.951591+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.954319+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.968840+0000 mon.b (mon.1) 130 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:30.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:29.969380+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:30.510 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.286625+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:30.510 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.286798+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]': finished 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:29.286865+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.292915+0000 mon.b (mon.1) 125 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.303843+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.320956+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.59:0/3002694168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.331365+0000 mon.b (mon.1) 127 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.373384+0000 mon.b (mon.1) 128 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:29.792872+0000 mgr.y (mgr.24454) 180 : cluster [DBG] pgmap v184: 498 pgs: 128 unknown, 370 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 744 B/s wr, 2 op/s 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.937440+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.941520+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.951591+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.954319+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.968840+0000 mon.b (mon.1) 130 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:30.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:29.969380+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:30.561 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.286625+0000 mon.a (mon.0) 1068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi059-82877-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.286798+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4"}]': finished 2024-02-18T00:33:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:29.286865+0000 mon.a (mon.0) 1070 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-18T00:33:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.292915+0000 mon.b (mon.1) 125 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.303843+0000 mon.a (mon.0) 1071 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]: dispatch 2024-02-18T00:33:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.320956+0000 mon.b (mon.1) 126 : audit [INF] from='client.? 172.21.15.59:0/3002694168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.331365+0000 mon.b (mon.1) 127 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.373384+0000 mon.b (mon.1) 128 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:29.792872+0000 mgr.y (mgr.24454) 180 : cluster [DBG] pgmap v184: 498 pgs: 128 unknown, 370 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 744 B/s wr, 2 op/s 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.937440+0000 mon.a (mon.0) 1072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.941520+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.951591+0000 mon.b (mon.1) 129 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.954319+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.968840+0000 mon.b (mon.1) 130 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:30.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:29.969380+0000 mon.a (mon.0) 1075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:31.073 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:31.122 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:31.126 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:31.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:31.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:31.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:33:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:33:31.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:33:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:33:31.417 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:31.507 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:31.549 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:31.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:31.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:31.823 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:31.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:31.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:31.971 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:31.998 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:32.016 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:32.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:32.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:32.120 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:32.145 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:32.165 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:32.167 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:32.168 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:32.218 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:32] "GET /metrics HTTP/1.1" 200 208825 "" "Prometheus/2.33.4" 2024-02-18T00:33:32.268 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:32.379 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24760.0:2 11.1f e106/104 11:f9b7e329:test-rados-api-smithi059-82973-2::foo:head v 106'1, mlcod=0'0)", 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.544957+0000", 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.774721261, 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022698870999999999, 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.544957+0000", 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.544957+0000", 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.381 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.544957+0000", 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5720000000000003e-06 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.544965+0000", 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5290000000000008e-06 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.544974+0000", 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.737e-06 2024-02-18T00:33:32.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.544980+0000", 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020573409999999999 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.547038+0000", 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8539999999999999e-05 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.383 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.547056+0000", 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020573556 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.567630+0000", 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5976999999999999e-05 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.567656+0000", 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022698870999999999 2024-02-18T00:33:32.384 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:4 17.1c e106/104 17:387e3012:test-rados-api-smithi059-82949-3::aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:head v 106'1, mlcod=0'0)", 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.549284+0000", 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.770394178, 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018781722000000001, 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.385 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549284+0000", 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549284+0000", 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.386 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549283+0000", 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3411000000000002e-05 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549317+0000", 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1949999999999992e-06 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549326+0000", 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6259999999999997e-06 2024-02-18T00:33:32.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549333+0000", 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00029272 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549625+0000", 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2076000000000002e-05 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.388 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.549668+0000", 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018352606 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.568020+0000", 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5711000000000003e-05 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.389 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.568066+0000", 2024-02-18T00:33:32.390 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018781722000000001 2024-02-18T00:33:32.390 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.390 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.390 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.390 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24841.0:2 16.d e106/104 16:b4e7e639:test-rados-api-smithi059-83061-3::testobj:head v 106'1, mlcod=0'0)", 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.558532+0000", 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.761146305, 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0098767290000000008, 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558532+0000", 2024-02-18T00:33:32.391 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558532+0000", 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558532+0000", 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5130000000000001e-06 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.392 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558535+0000", 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3000000000000002e-06 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558538+0000", 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8800000000000001e-06 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558542+0000", 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00071394899999999996 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.393 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.559256+0000", 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7017999999999999e-05 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.559273+0000", 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0091110419999999998 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.394 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.568384+0000", 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4332999999999999e-05 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.568409+0000", 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0098767290000000008 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24766.0:2 12.4 e106/104 12:216d7194:test-rados-api-smithi059-82901-3::foo:head v 106'1, mlcod=0'0)", 2024-02-18T00:33:32.395 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.558880+0000", 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.760798493999999, 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0098142439999999997, 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558880+0000", 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.396 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558880+0000", 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967296 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558880+0000", 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1569999999999998e-06 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558882+0000", 2024-02-18T00:33:32.397 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.773e-06 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558885+0000", 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0249999999999998e-06 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.558888+0000", 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032651000000000002 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.559214+0000", 2024-02-18T00:33:32.398 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6889999999999999e-05 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.559231+0000", 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0094487600000000005 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.568680+0000", 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4279e-05 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.568694+0000", 2024-02-18T00:33:32.399 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0098142439999999997 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24841.0:15 16.16 16:6971d125:test-rados-api-smithi059-83061-5::testobj:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e106)", 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:01.619276+0000", 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 30.700401956, 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0073807969999999997, 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:32.400 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24841", 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1293437097", 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 15 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619276+0000", 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619276+0000", 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619275+0000", 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7009999999999999e-06 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619282+0000", 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5832999999999999e-05 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619298+0000", 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5399999999999998e-06 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619305+0000", 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9855999999999999e-05 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619345+0000", 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9962999999999994e-05 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619415+0000", 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0494000000000006e-05 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.403 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,6", 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619486+0000", 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049231700000000004 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.619978+0000", 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000153368 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.620131+0000", 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0064384139999999999 2024-02-18T00:33:32.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.626570+0000", 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7620999999999997e-05 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.626617+0000", 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9876000000000002e-05 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.405 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:01.626657+0000", 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0073807969999999997 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24865.0:3 29.f 29:f4b50b61:::1.obj:head [write 0~198] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.519469+0000", 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 29.800209113000001, 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2357815539999999, 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24865", 2024-02-18T00:33:32.406 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2678422317", 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 3 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519469+0000", 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519469+0000", 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.407 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519468+0000", 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8630000000000002e-06 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519474+0000", 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1486e-05 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.408 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519486+0000", 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.977e-06 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519494+0000", 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7311999999999997e-05 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519541+0000", 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1619999999999997e-06 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:33:32.409 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.519550+0000", 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2124678910000002 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.732018+0000", 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011129300000000001 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.732130+0000", 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015139299999999999 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.410 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.732281+0000", 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013114880000000001 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.745396+0000", 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035140900000000001 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.745747+0000", 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0094409690000000004 2024-02-18T00:33:32.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.755188+0000", 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1968999999999998e-05 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.755220+0000", 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.0505999999999999e-05 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.412 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.755251+0000", 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2357815539999999 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:2059 17.11 17:8d70d4b8:test-rados-api-smithi059-82949-16::big:head [writefull 0~25165836] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.659372+0000", 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.660306418000001, 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.081239354999999999, 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.413 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2059 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.659372+0000", 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.414 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.659372+0000", 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.659369+0000", 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.014247381 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.673617+0000", 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5166999999999997e-05 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.415 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.673652+0000", 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2853999999999999e-05 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.673665+0000", 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.915e-05 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.673734+0000", 2024-02-18T00:33:32.416 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000145544 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.673880+0000", 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000183769 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,4", 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.674063+0000", 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062714080000000005 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.417 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.736777+0000", 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00044547400000000001 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.737223+0000", 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0033048909999999999 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.740528+0000", 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7123999999999999e-05 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.418 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.740565+0000", 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6329999999999999e-05 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.740611+0000", 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.081239354999999999 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.419 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(osd.0.8:6 17.11 17:8d70d4b8:test-rados-api-smithi059-82949-16::big:head [assert-version v3,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e108)", 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.763676+0000", 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.556001888000001, 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010828492, 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763676+0000", 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763676+0000", 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763674+0000", 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2782000000000001e-05 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763687+0000", 2024-02-18T00:33:32.421 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7816999999999999e-05 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763704+0000", 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1471000000000001e-05 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763716+0000", 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9397999999999997e-05 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.422 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763765+0000", 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0051999999999999e-05 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.763835+0000", 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010669514 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.774505+0000", 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010828492 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.423 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(osd.0.8:7 17.11 17:8d70d4b8:test-rados-api-smithi059-82949-16::big:head [assert-version v3,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e108)", 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.786483+0000", 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.533194900000002, 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.009113744, 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786483+0000", 2024-02-18T00:33:32.424 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786483+0000", 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999962 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786480+0000", 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7526000000000001e-05 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786497+0000", 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6469e-05 2024-02-18T00:33:32.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786524+0000", 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3373999999999999e-05 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786537+0000", 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1390999999999994e-05 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786618+0000", 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9512999999999995e-05 2024-02-18T00:33:32.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.786708+0000", 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0088891690000000006 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.795597+0000", 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.009113744 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.427 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(18.15 epoch 108/104 rep_tid 87 entries 108'2 (0'0) error 18:ae6b78ae:test-rados-api-smithi059-82925-5::foo:head by client.14892.0:10 0.000000 -17 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.812600+0000", 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.507077801000001, 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010386688, 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812600+0000", 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.428 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812600+0000", 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812600+0000", 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5040000000000001e-05 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812625+0000", 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1627000000000001e-05 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.429 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812646+0000", 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4169999999999998e-06 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812656+0000", 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4792999999999999e-05 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812750+0000", 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010236764000000001 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.430 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.822987+0000", 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010386688 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.14892.0:15 18.9 18:92155e6d:test-rados-api-smithi059-82925-7::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.939697+0000", 2024-02-18T00:33:32.431 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.379981785999998, 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010893435999999999, 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.14892", 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/464889388", 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 15 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.432 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939697+0000", 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939697+0000", 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939695+0000", 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8680000000000001e-06 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.433 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939702+0000", 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0584000000000001e-05 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939713+0000", 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.083e-06 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939719+0000", 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0561e-05 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939759+0000", 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3642999999999999e-05 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939843+0000", 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012881599999999999 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4,5", 2024-02-18T00:33:32.435 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.939972+0000", 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00036959900000000002 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.940341+0000", 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001999827 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.942341+0000", 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0081609109999999999 2024-02-18T00:33:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.950502+0000", 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9289000000000002e-05 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.950551+0000", 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8677000000000003e-05 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.437 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.950590+0000", 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010893435999999999 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2074 17.1c e108/104 17:38a558b5:test-rados-api-smithi059-82949-17::big3:head v 108'2, mlcod=0'0)", 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.249215+0000", 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.070463465, 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.060924395999999999, 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.249215+0000", 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.249215+0000", 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.249213+0000", 2024-02-18T00:33:32.439 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013099369 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262312+0000", 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1510000000000001e-05 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262354+0000", 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3818999999999999e-05 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.440 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262368+0000", 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011035900000000001 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262478+0000", 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1514999999999999e-05 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262510+0000", 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047601631999999998 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.310111+0000", 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8065999999999999e-05 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.310139+0000", 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.060924395999999999 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25025.0:36 22.12 22:4803cc70:::35.obj2:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.250034+0000", 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.069644355000001, 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012668097, 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25025", 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 36 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.443 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250034+0000", 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250034+0000", 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250031+0000", 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5656000000000001e-05 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.444 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250047+0000", 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4873000000000001e-05 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250072+0000", 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2266000000000001e-05 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250084+0000", 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.622e-05 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250140+0000", 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115559 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250256+0000", 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116279 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,7", 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250372+0000", 2024-02-18T00:33:32.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00053804600000000003 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.250910+0000", 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0027937349999999999 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.253704+0000", 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0088686219999999996 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262572+0000", 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4518999999999996e-05 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262647+0000", 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5279999999999999e-05 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.262702+0000", 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012668097 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.25025.0:40 22.c e108/105 22:33eb54d8:::39.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.269059+0000", 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.050619254000001, 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030859444999999999, 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269059+0000", 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269059+0000", 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269058+0000", 2024-02-18T00:33:32.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2991e-05 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269071+0000", 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7163000000000002e-05 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269088+0000", 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0175e-05 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269098+0000", 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6989000000000003e-05 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269195+0000", 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8621e-05 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269214+0000", 2024-02-18T00:33:32.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030653297999999999 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.299867+0000", 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1452999999999997e-05 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.299918+0000", 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030859444999999999 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.25025.0:41 22.e e108/105 22:7639cdab:::46.obj2:head v 108'9, mlcod=108'7)", 2024-02-18T00:33:32.454 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.302637+0000", 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 28.017040905000002, 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00717835, 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302637+0000", 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302637+0000", 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302636+0000", 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1408e-05 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302647+0000", 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0797e-05 2024-02-18T00:33:32.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302658+0000", 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9950000000000002e-06 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302664+0000", 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5448999999999998e-05 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302710+0000", 2024-02-18T00:33:32.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9885999999999999e-05 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.302729+0000", 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0070621549999999996 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.309792+0000", 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4130000000000001e-05 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.309816+0000", 2024-02-18T00:33:32.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00717835 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2076 17.1c e108/104 17:38a558b5:test-rados-api-smithi059-82949-17::big3:head v 108'4, mlcod=108'2)", 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.327823+0000", 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.991854896, 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0089897469999999993, 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.327823+0000", 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.327823+0000", 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.327822+0000", 2024-02-18T00:33:32.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001142749 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.328965+0000", 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0985999999999999e-05 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.328986+0000", 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3560999999999999e-05 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.329000+0000", 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2438999999999997e-05 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.329052+0000", 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7566000000000001e-05 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.329070+0000", 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0077101460000000002 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.336780+0000", 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3439000000000003e-05 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.336813+0000", 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0089897469999999993 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(23.a epoch 109/105 rep_tid 112 entries 109'4 (0'0) error 23:5074b981:test-rados-api-smithi059-83005-4::foo:head by client.24847.0:15 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 108'2)", 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.584436+0000", 2024-02-18T00:33:32.463 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.735242048, 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024581105999999998, 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.584436+0000", 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.464 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.584436+0000", 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.584436+0000", 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.1430000000000001e-06 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.584443+0000", 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1406000000000001e-05 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.584455+0000", 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8889999999999997e-06 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.584464+0000", 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00040077400000000001 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.584865+0000", 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024152173999999998 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.609017+0000", 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024581105999999998 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24847.0:16 23.a e109/105 23:5267809f:test-rados-api-smithi059-83005-5::foo:head v 109'5, mlcod=108'3)", 2024-02-18T00:33:32.467 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.656175+0000", 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.663503108, 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020745076000000001, 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.656175+0000", 2024-02-18T00:33:32.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.656175+0000", 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.656174+0000", 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2269999999999999e-05 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.656186+0000", 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0856000000000001e-05 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.656197+0000", 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.297e-06 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.656204+0000", 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023862480000000001 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.658590+0000", 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8087000000000001e-05 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.658608+0000", 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.018282987000000001 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.676891+0000", 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9402000000000001e-05 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.676920+0000", 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020745076000000001 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24847.0:22 23.12 23:48aa9aec:test-rados-api-smithi059-83005-6::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.743884+0000", 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.575793992000001, 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010121371000000001, 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24847", 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1733075084", 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 22 2024-02-18T00:33:32.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.743884+0000", 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.743884+0000", 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.474 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.743882+0000", 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4739999999999994e-06 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.743891+0000", 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2557e-05 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.743904+0000", 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0859999999999999e-05 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.743915+0000", 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012929700000000001 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.744044+0000", 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016783999999999999 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.744212+0000", 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022445700000000001 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.744436+0000", 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0036874540000000002 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.748124+0000", 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000421197 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.748545+0000", 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053682010000000004 2024-02-18T00:33:32.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.753913+0000", 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0918999999999998e-05 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.753964+0000", 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1566999999999997e-05 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.754006+0000", 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010121371000000001 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.15132.0:4 22.12 22:4a6d8757:::3.obj:head [write 0~115] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.749499+0000", 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 27.570179705000001, 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012887941999999999, 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:33:32.479 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.15132", 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3906319504", 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 4 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749499+0000", 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749499+0000", 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999967 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749495+0000", 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3458000000000001e-05 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749509+0000", 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2336000000000001e-05 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749531+0000", 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1518999999999999e-05 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749542+0000", 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2795999999999999e-05 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:33:32.482 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749615+0000", 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011618200000000001 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749731+0000", 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00023549700000000001 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,7", 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.749967+0000", 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00087268999999999997 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.750840+0000", 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0075474610000000001 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.758387+0000", 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039174819999999999 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.762305+0000", 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6468000000000003e-05 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.762351+0000", 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.5487000000000001e-05 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.762387+0000", 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012887941999999999 2024-02-18T00:33:32.485 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.486 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.486 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:33:32.486 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:33:32.486 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:33:32.486 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:30.283140+0000 mon.a (mon.0) 1076 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.309716+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]': finished 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.309925+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.310042+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:30.310100+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.313862+0000 mon.b (mon.1) 131 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.320148+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.321873+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.330984+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.334417+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.335180+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.340592+0000 mon.b (mon.1) 133 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:30.947790+0000 mgr.y (mgr.24454) 181 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:33.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:30.283140+0000 mon.a (mon.0) 1076 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:33.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.309716+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]': finished 2024-02-18T00:33:33.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.309925+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.310042+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:30.310100+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.313862+0000 mon.b (mon.1) 131 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.320148+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.321873+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.330984+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.334417+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.335180+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.340592+0000 mon.b (mon.1) 133 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:30.947790+0000 mgr.y (mgr.24454) 181 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:30.283140+0000 mon.a (mon.0) 1076 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:33.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.309716+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6"}]': finished 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.309925+0000 mon.a (mon.0) 1078 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi059-82882-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.310042+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:30.310100+0000 mon.a (mon.0) 1080 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.313862+0000 mon.b (mon.1) 131 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.320148+0000 mon.c (mon.2) 70 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.321873+0000 mon.a (mon.0) 1081 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.330984+0000 mon.b (mon.1) 132 : audit [INF] from='client.? 172.21.15.59:0/822740143' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.334417+0000 mon.a (mon.0) 1082 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.335180+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.340592+0000 mon.b (mon.1) 133 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.009 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:30.947790+0000 mgr.y (mgr.24454) 181 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:33.107 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:33.122 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:33.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:33.224 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:33.224 INFO:tasks.workunit.client.0.smithi059.stdout:h_notify_pp: flushed 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3005 ms) 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: trying... 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 523986010115 notifier_gid 25141 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: timed out 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: flushing 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: flushed 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3017 ms) 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: List watches 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: notify2 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 532575944708 notifier_gid 25141 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: notify2 done 2024-02-18T00:33:33.225 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: watch_check 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: unwatch2 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: flushing 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: done 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3016 ms) 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: List watches 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: notify2 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: handle_notify cookie 94686821711440 notify_id 545460846597 notifier_gid 25141 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: notify2 done 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: watch_check 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: unwatch2 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: flushing 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: done 2024-02-18T00:33:33.226 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3058 ms) 2024-02-18T00:33:33.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (15772 ms total) 2024-02-18T00:33:33.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: 2024-02-18T00:33:33.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-18T00:33:33.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (42302 ms total) 2024-02-18T00:33:33.227 INFO:tasks.workunit.client.0.smithi059.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-18T00:33:33.249 INFO:tasks.workunit.client.0.smithi059.stdout:p: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-18T00:33:33.249 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro": "centos", 2024-02-18T00:33:33.249 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-18T00:33:33.249 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "distro_version": "8", 2024-02-18T00:33:33.249 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-18T00:33:33.249 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "hostname": "smithi059", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "id": "foo.a", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_description": "#1 SMP Mon Feb 5 18:34:56 UTC 2024", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "kernel_version": "4.18.0-539.el8.x86_64", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "mem_total_kb": "32543852", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "num_handles": "1", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "os": "Linux", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "pid": "7", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "realm_id": "", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "realm_name": "", 2024-02-18T00:33:33.250 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zone_id": "e67bc818-d879-487a-b572-f16f51d463f9", 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zone_name": "default", 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zonegroup_id": "1a312463-4a64-495f-95ab-5ad353fdc6fb", 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: }, 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: "task_status": {} 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: } 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: attempt 2 of 20 2024-02-18T00:33:33.251 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (15973 ms) 2024-02-18T00:33:33.252 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (42270 ms total) 2024-02-18T00:33:33.252 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: 2024-02-18T00:33:33.252 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-18T00:33:33.252 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (42270 ms total) 2024-02-18T00:33:33.252 INFO:tasks.workunit.client.0.smithi059.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-18T00:33:33.253 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:33.272 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:33.274 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:33.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:33.375 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:33.401 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:33.420 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:33.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:33.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:33.524 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:33.638 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:33.699 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-18T00:33:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:33.858 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:33.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:33.861 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:31.314718+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:31.341855+0000 mon.b (mon.1) 134 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:31.399388+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:31.796816+0000 mgr.y (mgr.24454) 182 : cluster [DBG] pgmap v187: 430 pgs: 32 creating+peering, 32 unknown, 366 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:31.924641+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:31.924855+0000 mon.a (mon.0) 1086 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:31.951855+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.59:0/841879442' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:31.957299+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:32.346077+0000 mon.b (mon.1) 135 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:32.577216+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:32.588085+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:32.588255+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:33.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:32.588757+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:32.625769+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:32.629516+0000 mon.c (mon.2) 74 : audit [INF] from='client.? 172.21.15.59:0/1112125661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.166123+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.166811+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.176129+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:33.177125+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.191056+0000 mon.b (mon.1) 136 : audit [INF] from='client.? 172.21.15.59:0/3112604729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.211671+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.239718+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? 172.21.15.59:0/2138943324' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.240242+0000 mgr.y (mgr.24454) 183 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:33.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:33.350521+0000 mon.b (mon.1) 137 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:31.314718+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:31.341855+0000 mon.b (mon.1) 134 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:31.399388+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:31.796816+0000 mgr.y (mgr.24454) 182 : cluster [DBG] pgmap v187: 430 pgs: 32 creating+peering, 32 unknown, 366 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:31.924641+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:31.924855+0000 mon.a (mon.0) 1086 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:31.951855+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.59:0/841879442' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:31.957299+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:32.346077+0000 mon.b (mon.1) 135 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:32.577216+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:32.588085+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:32.588255+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:33.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:32.588757+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:32.625769+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:32.629516+0000 mon.c (mon.2) 74 : audit [INF] from='client.? 172.21.15.59:0/1112125661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.166123+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.166811+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.176129+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:33.177125+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.191056+0000 mon.b (mon.1) 136 : audit [INF] from='client.? 172.21.15.59:0/3112604729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.211671+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.239718+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? 172.21.15.59:0/2138943324' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.240242+0000 mgr.y (mgr.24454) 183 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:33.865 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:33.350521+0000 mon.b (mon.1) 137 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.960 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:33.973 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:31.314718+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:31.341855+0000 mon.b (mon.1) 134 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:31.399388+0000 mon.c (mon.2) 71 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:31.796816+0000 mgr.y (mgr.24454) 182 : cluster [DBG] pgmap v187: 430 pgs: 32 creating+peering, 32 unknown, 366 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:31.924641+0000 mon.a (mon.0) 1085 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi059-83013-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:31.924855+0000 mon.a (mon.0) 1086 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:31.951855+0000 mon.c (mon.2) 72 : audit [INF] from='client.? 172.21.15.59:0/841879442' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:31.957299+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:32.346077+0000 mon.b (mon.1) 135 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:32.577216+0000 mon.a (mon.0) 1088 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:32.588085+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi059-82973-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:32.588255+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:32.588757+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:32.625769+0000 mon.c (mon.2) 73 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:32.629516+0000 mon.c (mon.2) 74 : audit [INF] from='client.? 172.21.15.59:0/1112125661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.166123+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.166811+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.176129+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi059-82877-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:33.177125+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.191056+0000 mon.b (mon.1) 136 : audit [INF] from='client.? 172.21.15.59:0/3112604729' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.211671+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:33.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.239718+0000 mon.c (mon.2) 75 : audit [DBG] from='client.? 172.21.15.59:0/2138943324' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:33.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.240242+0000 mgr.y (mgr.24454) 183 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T00:33:33.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:33.350521+0000 mon.b (mon.1) 137 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:33.988 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:34.011 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:34.013 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:34.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:34.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:34.515 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:34.546 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:34.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:34.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:34.764 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:34.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.896+0000 7fa748774700 1 -- 172.21.15.59:0/2105083600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa74006a530 msgr2=0x7fa74006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.896+0000 7fa748774700 1 --2- 172.21.15.59:0/2105083600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa74006a530 0x7fa74006a9b0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 rx=0x7fa73800bfa0 tx=0x7fa738009850).stop 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.896+0000 7fa748774700 1 -- 172.21.15.59:0/2105083600 shutdown_connections 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.896+0000 7fa748774700 1 --2- 172.21.15.59:0/2105083600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa74006a530 0x7fa74006a9b0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.896+0000 7fa748774700 1 --2- 172.21.15.59:0/2105083600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7401034d0 0x7fa7401038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.896+0000 7fa748774700 1 --2- 172.21.15.59:0/2105083600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa740103e30 0x7fa740069ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.896+0000 7fa748774700 1 -- 172.21.15.59:0/2105083600 >> 172.21.15.59:0/2105083600 conn(0x7fa740064640 msgr2=0x7fa740064a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.898+0000 7fa748774700 1 -- 172.21.15.59:0/2105083600 shutdown_connections 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.898+0000 7fa748774700 1 -- 172.21.15.59:0/2105083600 wait complete. 2024-02-18T00:33:34.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.898+0000 7fa748774700 1 librados: starting msgr at 2024-02-18T00:33:34.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.898+0000 7fa748774700 1 librados: starting objecter 2024-02-18T00:33:34.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.898+0000 7fa748774700 1 Processor -- start 2024-02-18T00:33:34.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.901+0000 7fa748774700 1 -- start start 2024-02-18T00:33:34.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 librados: setting wanted keys 2024-02-18T00:33:34.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 librados: calling monclient init 2024-02-18T00:33:34.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74006a530 0x7fa740125770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:34.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7401034d0 0x7fa740125c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:34.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa74011f910 0x7fa74011fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:34.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa740120280 con 0x7fa74006a530 2024-02-18T00:33:34.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401203f0 con 0x7fa7401034d0 2024-02-18T00:33:34.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa748774700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401205b0 con 0x7fa74011f910 2024-02-18T00:33:34.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa745d0f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7401034d0 0x7fa740125c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:34.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa745d0f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7401034d0 0x7fa740125c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:33996/0 (socket says 172.21.15.59:33996) 2024-02-18T00:33:34.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.902+0000 7fa745d0f700 1 -- 172.21.15.59:0/530441213 learned_addr learned my addr 172.21.15.59:0/530441213 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.904+0000 7fa746510700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74006a530 0x7fa740125770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa746d11700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa74011f910 0x7fa74011fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa745d0f700 1 -- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa74011f910 msgr2=0x7fa74011fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa745d0f700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa74011f910 0x7fa74011fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa745d0f700 1 -- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74006a530 msgr2=0x7fa740125770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa745d0f700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74006a530 0x7fa740125770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa745d0f700 1 -- 172.21.15.59:0/530441213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73800bde0 con 0x7fa7401034d0 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa746d11700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa74011f910 0x7fa74011fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:33:34.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.906+0000 7fa745d0f700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7401034d0 0x7fa740125c60 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7fa73c00b440 tx=0x7fa73c006f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:34.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.907+0000 7fa7377fe700 10 client.?.objecter ms_handle_connect 0x7fa7401034d0 2024-02-18T00:33:34.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.908+0000 7fa7377fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.908+0000 7fa7377fe700 1 -- 172.21.15.59:0/530441213 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c012410 con 0x7fa7401034d0 2024-02-18T00:33:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.908+0000 7fa7377fe700 1 -- 172.21.15.59:0/530441213 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa73c012a40 con 0x7fa7401034d0 2024-02-18T00:33:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.908+0000 7fa7377fe700 1 -- 172.21.15.59:0/530441213 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c002870 con 0x7fa7401034d0 2024-02-18T00:33:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.908+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74012c570 con 0x7fa7401034d0 2024-02-18T00:33:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.909+0000 7fa748774700 10 client.33976.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.909+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74012c9a0 con 0x7fa7401034d0 2024-02-18T00:33:34.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.909+0000 7fa748774700 1 librados: init done 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.912+0000 7fa7377fe700 1 -- 172.21.15.59:0/530441213 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa73c012570 con 0x7fa7401034d0 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.912+0000 7fa7377fe700 1 --2- 172.21.15.59:0/530441213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa72c062ce0 0x7fa72c0651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa7377fe700 1 -- 172.21.15.59:0/530441213 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(136..136 src has 44..136) v4 ==== 10668+0+0 (secure 0 0 0) 0x7fa73c07d790 con 0x7fa7401034d0 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa7377fe700 10 client.33976.objecter ms_dispatch 0x7fa740001550 osd_map(136..136 src has 44..136) v4 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa7377fe700 3 client.33976.objecter handle_osd_map got epochs [136,136] > 0 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa7377fe700 3 client.33976.objecter handle_osd_map decoding full epoch 136 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa7377fe700 20 client.33976.objecter dump_active .. 0 homeless 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa746510700 1 --2- 172.21.15.59:0/530441213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa72c062ce0 0x7fa72c0651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:34.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa746510700 1 --2- 172.21.15.59:0/530441213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa72c062ce0 0x7fa72c0651a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7fa7300048b0 tx=0x7fa73000c040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:34.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.913+0000 7fa7377fe700 10 client.33976.objecter ms_handle_connect 0x7fa72c062ce0 2024-02-18T00:33:34.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.919+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa740060280 con 0x7fa7401034d0 2024-02-18T00:33:34.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:34.927+0000 7fa7377fe700 1 -- 172.21.15.59:0/530441213 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa73c0502b0 con 0x7fa7401034d0 2024-02-18T00:33:34.999 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:35.007 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:35.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:35.108 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.267+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa740120df0 con 0x7fa72c062ce0 2024-02-18T00:33:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:35.342 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:33:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.338+0000 7fa7377fe700 1 -- 172.21.15.59:0/530441213 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+1058047 (secure 0 0 0) 0x7fa740120df0 con 0x7fa72c062ce0 2024-02-18T00:33:35.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.354+0000 7fa748774700 10 librados: watch_flush enter 2024-02-18T00:33:35.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 10 librados: watch_flush exit 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 20 client.33976.objecter shutdown clearing up homeless session... 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 10 client.33976.objecter successfully canceled tick 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa72c062ce0 msgr2=0x7fa72c0651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 --2- 172.21.15.59:0/530441213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa72c062ce0 0x7fa72c0651a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7fa7300048b0 tx=0x7fa73000c040).stop 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7401034d0 msgr2=0x7fa740125c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7401034d0 0x7fa740125c60 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7fa73c00b440 tx=0x7fa73c006f90).stop 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 shutdown_connections 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa74011f910 0x7fa74011fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa74006a530 0x7fa740125770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 --2- 172.21.15.59:0/530441213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa72c062ce0 0x7fa72c0651a0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 --2- 172.21.15.59:0/530441213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7401034d0 0x7fa740125c60 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.357+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 >> 172.21.15.59:0/530441213 conn(0x7fa740064640 msgr2=0x7fa7401043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.362+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 shutdown_connections 2024-02-18T00:33:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.362+0000 7fa748774700 1 -- 172.21.15.59:0/530441213 wait complete. 2024-02-18T00:33:35.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.370+0000 7fa748774700 1 librados: shutdown 2024-02-18T00:33:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:33:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:35.459 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:35.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:35.560 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 -- 172.21.15.59:0/3245205399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17580692d0 msgr2=0x7f17580696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 --2- 172.21.15.59:0/3245205399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17580692d0 0x7f17580696b0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 rx=0x7f1748004870 tx=0x7f1748009cb0).stop 2024-02-18T00:33:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 -- 172.21.15.59:0/3245205399 shutdown_connections 2024-02-18T00:33:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 --2- 172.21.15.59:0/3245205399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17580692d0 0x7f17580696b0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 --2- 172.21.15.59:0/3245205399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17581044a0 0x7f1758104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 --2- 172.21.15.59:0/3245205399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1758069ba0 0x7f1758103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:35.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 -- 172.21.15.59:0/3245205399 >> 172.21.15.59:0/3245205399 conn(0x7f1758063db0 msgr2=0x7f1758067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 -- 172.21.15.59:0/3245205399 shutdown_connections 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.631+0000 7f175f79a700 1 -- 172.21.15.59:0/3245205399 wait complete. 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 librados: starting msgr at 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 librados: starting objecter 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 Processor -- start 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 -- start start 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 librados: setting wanted keys 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 librados: calling monclient init 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17580692d0 0x7f175810cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1758069ba0 0x7f175810d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17581044a0 0x7f17581117f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175810db70 con 0x7f1758069ba0 2024-02-18T00:33:35.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1758111ce0 con 0x7f17581044a0 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175f79a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1758111ea0 con 0x7f17580692d0 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175d536700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17580692d0 0x7f175810cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175d536700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17580692d0 0x7f175810cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43400/0 (socket says 172.21.15.59:43400) 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.632+0000 7f175d536700 1 -- 172.21.15.59:0/1140132757 learned_addr learned my addr 172.21.15.59:0/1140132757 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175d536700 1 -- 172.21.15.59:0/1140132757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17581044a0 msgr2=0x7f17581117f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175d536700 1 --2- 172.21.15.59:0/1140132757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17581044a0 0x7f17581117f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175d536700 1 -- 172.21.15.59:0/1140132757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1758069ba0 msgr2=0x7f175810d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175d536700 1 --2- 172.21.15.59:0/1140132757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1758069ba0 0x7f175810d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175d536700 1 -- 172.21.15.59:0/1140132757 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748005040 con 0x7f17580692d0 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175d536700 1 --2- 172.21.15.59:0/1140132757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17580692d0 0x7f175810cd60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f1748009b10 tx=0x7f1748007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f174e7fc700 10 client.?.objecter ms_handle_connect 0x7f17580692d0 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f174e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f174e7fc700 1 -- 172.21.15.59:0/1140132757 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174800e030 con 0x7f17580692d0 2024-02-18T00:33:35.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f174e7fc700 1 -- 172.21.15.59:0/1140132757 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f174801d420 con 0x7f17580692d0 2024-02-18T00:33:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f174e7fc700 1 -- 172.21.15.59:0/1140132757 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748018070 con 0x7f17580692d0 2024-02-18T00:33:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1758112120 con 0x7f17580692d0 2024-02-18T00:33:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175f79a700 10 client.33098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1758112560 con 0x7f17580692d0 2024-02-18T00:33:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.633+0000 7f175f79a700 1 librados: init done 2024-02-18T00:33:35.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.634+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1758060280 con 0x7f17580692d0 2024-02-18T00:33:35.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.644+0000 7f174e7fc700 1 -- 172.21.15.59:0/1140132757 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f17480092e0 con 0x7f17580692d0 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.645+0000 7f174e7fc700 1 --2- 172.21.15.59:0/1140132757 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1744062ce0 0x7f17440651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.645+0000 7f174e7fc700 1 -- 172.21.15.59:0/1140132757 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(137..137 src has 44..137) v4 ==== 11060+0+0 (secure 0 0 0) 0x7f174807f420 con 0x7f17580692d0 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.645+0000 7f174e7fc700 10 client.33098.objecter ms_dispatch 0x7f1758001550 osd_map(137..137 src has 44..137) v4 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.645+0000 7f174e7fc700 3 client.33098.objecter handle_osd_map got epochs [137,137] > 0 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.645+0000 7f174e7fc700 3 client.33098.objecter handle_osd_map decoding full epoch 137 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.645+0000 7f174e7fc700 20 client.33098.objecter dump_active .. 0 homeless 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.645+0000 7f174e7fc700 1 -- 172.21.15.59:0/1140132757 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f174807f8b0 con 0x7f17580692d0 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.646+0000 7f175cd35700 1 --2- 172.21.15.59:0/1140132757 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1744062ce0 0x7f17440651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.647+0000 7f175cd35700 1 --2- 172.21.15.59:0/1140132757 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1744062ce0 0x7f17440651a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7f1758112860 tx=0x7f1754006c30).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.647+0000 7f174e7fc700 10 client.33098.objecter ms_handle_connect 0x7f1744062ce0 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:33.799009+0000 mgr.y (mgr.24454) 184 : cluster [DBG] pgmap v190: 470 pgs: 32 creating+peering, 104 unknown, 334 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.190629+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]': finished 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.190772+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.190957+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:34.191043+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.253448+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]: dispatch 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.262632+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]: dispatch 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.356592+0000 mon.b (mon.1) 138 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.800846+0000 mon.b (mon.1) 139 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.801543+0000 mon.b (mon.1) 140 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.803780+0000 mon.b (mon.1) 141 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.804478+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:35.666 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.807329+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:35.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.811499+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:35.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.815462+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:35.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.816034+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:35.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.816778+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:35.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.817312+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:35.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:35.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:35.916 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:35.930 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:33:35.930 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:33:35.930 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:33:35.930 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:33:35.948 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:35.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:33.799009+0000 mgr.y (mgr.24454) 184 : cluster [DBG] pgmap v190: 470 pgs: 32 creating+peering, 104 unknown, 334 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:35.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.190629+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]': finished 2024-02-18T00:33:35.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.190772+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:35.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.190957+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:35.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:34.191043+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-18T00:33:35.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.253448+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]: dispatch 2024-02-18T00:33:35.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.262632+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]: dispatch 2024-02-18T00:33:35.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.356592+0000 mon.b (mon.1) 138 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:35.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.800846+0000 mon.b (mon.1) 139 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:35.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.801543+0000 mon.b (mon.1) 140 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:35.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.803780+0000 mon.b (mon.1) 141 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:35.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.804478+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:35.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.807329+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:35.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.811499+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:35.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.815462+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:35.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.816034+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:35.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.816778+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:35.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.817312+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:35.964 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:33.799009+0000 mgr.y (mgr.24454) 184 : cluster [DBG] pgmap v190: 470 pgs: 32 creating+peering, 104 unknown, 334 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:35.964 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.190629+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-9"}]': finished 2024-02-18T00:33:35.964 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.190772+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:35.964 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.190957+0000 mon.a (mon.0) 1099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi059-82882-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:35.964 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:34.191043+0000 mon.a (mon.0) 1100 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-18T00:33:35.965 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.253448+0000 mon.c (mon.2) 76 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]: dispatch 2024-02-18T00:33:35.965 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.262632+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]: dispatch 2024-02-18T00:33:35.965 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.356592+0000 mon.b (mon.1) 138 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:35.965 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.800846+0000 mon.b (mon.1) 139 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:35.965 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.801543+0000 mon.b (mon.1) 140 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:35.965 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.803780+0000 mon.b (mon.1) 141 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:35.966 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.804478+0000 mon.b (mon.1) 142 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:35.966 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.807329+0000 mon.b (mon.1) 143 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:35.966 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.811499+0000 mon.b (mon.1) 144 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:35.966 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.815462+0000 mon.b (mon.1) 145 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:35.966 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.816034+0000 mon.b (mon.1) 146 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:35.966 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.816778+0000 mon.b (mon.1) 147 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:35.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.817312+0000 mon.b (mon.1) 148 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:35.968 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:35.971 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:35.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:35.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:35.987+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f175805ee80 con 0x7f17580692d0 2024-02-18T00:33:36.072 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:36.100 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:36.223 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:36.226 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:36.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:36.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:36.352 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:36.383 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-02-18T00:33:34.190960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":151007994,"num_objects":276,"num_object_clones":1,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":0,"num_read":2375,"num_read_kb":50242,"num_write":1784,"num_write_kb":153097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":983,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1879,"ondisk_log_size":1879,"up":1098,"acting":1098,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1399,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2918060,"kb_used_data":455264,"kb_used_omap":0,"kb_used_meta":2462720,"kb_avail":747043156,"statfs":{"total":767960285184,"available":764972191744,"internally_reserved":0,"allocated":466190336,"data_stored":459792258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2521825280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1408,"num_objects":-13,"num_object_clones":1,"num_object_copies":-39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":9,"num_write":-7,"num_write_kb":-8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":-13,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.882642"},"pg_stats":[{"pgid":"80.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.311114+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.329537+0000","last_change":"2024-02-18T00:33:28.146845+0000","last_active":"2024-02-18T00:33:29.329537+0000","last_peered":"2024-02-18T00:33:29.329537+0000","last_clean":"2024-02-18T00:33:29.329537+0000","last_became_active":"2024-02-18T00:33:23.620302+0000","last_became_peered":"2024-02-18T00:33:23.620302+0000","last_unstale":"2024-02-18T00:33:29.329537+0000","last_undegraded":"2024-02-18T00:33:29.329537+0000","last_fullsized":"2024-02-18T00:33:29.329537+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.321005+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.317490+0000","last_change":"2024-02-18T00:33:28.059109+0000","last_active":"2024-02-18T00:33:29.317490+0000","last_peered":"2024-02-18T00:33:29.317490+0000","last_clean":"2024-02-18T00:33:29.317490+0000","last_became_active":"2024-02-18T00:33:23.636764+0000","last_became_peered":"2024-02-18T00:33:23.636764+0000","last_unstale":"2024-02-18T00:33:29.317490+0000","last_undegraded":"2024-02-18T00:33:29.317490+0000","last_fullsized":"2024-02-18T00:33:29.317490+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.330137+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296058+0000","last_change":"2024-02-18T00:33:28.034837+0000","last_active":"2024-02-18T00:33:29.296058+0000","last_peered":"2024-02-18T00:33:29.296058+0000","last_clean":"2024-02-18T00:33:29.296058+0000","last_became_active":"2024-02-18T00:33:23.503952+0000","last_became_peered":"2024-02-18T00:33:23.503952+0000","last_unstale":"2024-02-18T00:33:29.296058+0000","last_undegraded":"2024-02-18T00:33:29.296058+0000","last_fullsized":"2024-02-18T00:33:29.296058+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.319629+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.358865+0000","last_change":"2024-02-18T00:33:28.131694+0000","last_active":"2024-02-18T00:33:29.358865+0000","last_peered":"2024-02-18T00:33:29.358865+0000","last_clean":"2024-02-18T00:33:29.358865+0000","last_became_active":"2024-02-18T00:33:23.615724+0000","last_became_peered":"2024-02-18T00:33:23.615724+0000","last_unstale":"2024-02-18T00:33:29.358865+0000","last_undegraded":"2024-02-18T00:33:29.358865+0000","last_fullsized":"2024-02-18T00:33:29.358865+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.355043+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299238+0000","last_change":"2024-02-18T00:33:28.024991+0000","last_active":"2024-02-18T00:33:29.299238+0000","last_peered":"2024-02-18T00:33:29.299238+0000","last_clean":"2024-02-18T00:33:29.299238+0000","last_became_active":"2024-02-18T00:33:23.599663+0000","last_became_peered":"2024-02-18T00:33:23.599663+0000","last_unstale":"2024-02-18T00:33:29.299238+0000","last_undegraded":"2024-02-18T00:33:29.299238+0000","last_fullsized":"2024-02-18T00:33:29.299238+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.330260+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312839+0000","last_change":"2024-02-18T00:33:28.038900+0000","last_active":"2024-02-18T00:33:29.312839+0000","last_peered":"2024-02-18T00:33:29.312839+0000","last_clean":"2024-02-18T00:33:29.312839+0000","last_became_active":"2024-02-18T00:33:23.475463+0000","last_became_peered":"2024-02-18T00:33:23.475463+0000","last_unstale":"2024-02-18T00:33:29.312839+0000","last_undegraded":"2024-02-18T00:33:29.312839+0000","last_fullsized":"2024-02-18T00:33:29.312839+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.317609+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.1b","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.327531+0000","last_change":"2024-02-18T00:33:28.025469+0000","last_active":"2024-02-18T00:33:29.327531+0000","last_peered":"2024-02-18T00:33:29.327531+0000","last_clean":"2024-02-18T00:33:29.327531+0000","last_became_active":"2024-02-18T00:33:23.628491+0000","last_became_peered":"2024-02-18T00:33:23.628491+0000","last_unstale":"2024-02-18T00:33:29.327531+0000","last_undegraded":"2024-02-18T00:33:29.327531+0000","last_fullsized":"2024-02-18T00:33:29.327531+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.332247+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.293084+0000","last_change":"2024-02-18T00:33:28.023753+0000","last_active":"2024-02-18T00:33:29.293084+0000","last_peered":"2024-02-18T00:33:29.293084+0000","last_clean":"2024-02-18T00:33:29.293084+0000","last_became_active":"2024-02-18T00:33:23.499256+0000","last_became_peered":"2024-02-18T00:33:23.499256+0000","last_unstale":"2024-02-18T00:33:29.293084+0000","last_undegraded":"2024-02-18T00:33:29.293084+0000","last_fullsized":"2024-02-18T00:33:29.293084+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.314586+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311074+0000","last_change":"2024-02-18T00:33:28.046925+0000","last_active":"2024-02-18T00:33:29.311074+0000","last_peered":"2024-02-18T00:33:29.311074+0000","last_clean":"2024-02-18T00:33:29.311074+0000","last_became_active":"2024-02-18T00:33:23.501291+0000","last_became_peered":"2024-02-18T00:33:23.501291+0000","last_unstale":"2024-02-18T00:33:29.311074+0000","last_undegraded":"2024-02-18T00:33:29.311074+0000","last_fullsized":"2024-02-18T00:33:29.311074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.319419+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.317848+0000","last_change":"2024-02-18T00:33:28.039934+0000","last_active":"2024-02-18T00:33:29.317848+0000","last_peered":"2024-02-18T00:33:29.317848+0000","last_clean":"2024-02-18T00:33:29.317848+0000","last_became_active":"2024-02-18T00:33:23.566091+0000","last_became_peered":"2024-02-18T00:33:23.566091+0000","last_unstale":"2024-02-18T00:33:29.317848+0000","last_undegraded":"2024-02-18T00:33:29.317848+0000","last_fullsized":"2024-02-18T00:33:29.317848+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.322892+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.17","version":"132'7","reported_seq":35,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-02-18T00:33:30.412376+0000","last_change":"2024-02-18T00:33:28.042245+0000","last_active":"2024-02-18T00:33:30.412376+0000","last_peered":"2024-02-18T00:33:30.412376+0000","last_clean":"2024-02-18T00:33:30.412376+0000","last_became_active":"2024-02-18T00:33:23.488902+0000","last_became_peered":"2024-02-18T00:33:23.488902+0000","last_unstale":"2024-02-18T00:33:30.412376+0000","last_undegraded":"2024-02-18T00:33:30.412376+0000","last_fullsized":"2024-02-18T00:33:30.412376+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":512,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.333308+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.16","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.358974+0000","last_change":"2024-02-18T00:33:28.131831+0000","last_active":"2024-02-18T00:33:29.358974+0000","last_peered":"2024-02-18T00:33:29.358974+0000","last_clean":"2024-02-18T00:33:29.358974+0000","last_became_active":"2024-02-18T00:33:23.636643+0000","last_became_peered":"2024-02-18T00:33:23.636643+0000","last_unstale":"2024-02-18T00:33:29.358974+0000","last_undegraded":"2024-02-18T00:33:29.358974+0000","last_fullsized":"2024-02-18T00:33:29.358974+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.383249+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306419+0000","last_change":"2024-02-18T00:33:28.067993+0000","last_active":"2024-02-18T00:33:29.306419+0000","last_peered":"2024-02-18T00:33:29.306419+0000","last_clean":"2024-02-18T00:33:29.306419+0000","last_became_active":"2024-02-18T00:33:23.626811+0000","last_became_peered":"2024-02-18T00:33:23.626811+0000","last_unstale":"2024-02-18T00:33:29.306419+0000","last_undegraded":"2024-02-18T00:33:29.306419+0000","last_fullsized":"2024-02-18T00:33:29.306419+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.383690+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.290434+0000","last_change":"2024-02-18T00:33:28.019583+0000","last_active":"2024-02-18T00:33:29.290434+0000","last_peered":"2024-02-18T00:33:29.290434+0000","last_clean":"2024-02-18T00:33:29.290434+0000","last_became_active":"2024-02-18T00:33:23.625484+0000","last_became_peered":"2024-02-18T00:33:23.625484+0000","last_unstale":"2024-02-18T00:33:29.290434+0000","last_undegraded":"2024-02-18T00:33:29.290434+0000","last_fullsized":"2024-02-18T00:33:29.290434+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.382740+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.13","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.317468+0000","last_change":"2024-02-18T00:33:28.029292+0000","last_active":"2024-02-18T00:33:29.317468+0000","last_peered":"2024-02-18T00:33:29.317468+0000","last_clean":"2024-02-18T00:33:29.317468+0000","last_became_active":"2024-02-18T00:33:23.489005+0000","last_became_peered":"2024-02-18T00:33:23.489005+0000","last_unstale":"2024-02-18T00:33:29.317468+0000","last_undegraded":"2024-02-18T00:33:29.317468+0000","last_fullsized":"2024-02-18T00:33:29.317468+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.314360+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.12","version":"130'6","reported_seq":29,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320764+0000","last_change":"2024-02-18T00:33:29.060064+0000","last_active":"2024-02-18T00:33:29.320764+0000","last_peered":"2024-02-18T00:33:29.320764+0000","last_clean":"2024-02-18T00:33:29.320764+0000","last_became_active":"2024-02-18T00:33:23.490075+0000","last_became_peered":"2024-02-18T00:33:23.490075+0000","last_unstale":"2024-02-18T00:33:29.320764+0000","last_undegraded":"2024-02-18T00:33:29.320764+0000","last_fullsized":"2024-02-18T00:33:29.320764+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.310789+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.d","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.337331+0000","last_change":"2024-02-18T00:33:28.130377+0000","last_active":"2024-02-18T00:33:29.337331+0000","last_peered":"2024-02-18T00:33:29.337331+0000","last_clean":"2024-02-18T00:33:29.337331+0000","last_became_active":"2024-02-18T00:33:23.489723+0000","last_became_peered":"2024-02-18T00:33:23.489723+0000","last_unstale":"2024-02-18T00:33:29.337331+0000","last_undegraded":"2024-02-18T00:33:29.337331+0000","last_fullsized":"2024-02-18T00:33:29.337331+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.351779+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306279+0000","last_change":"2024-02-18T00:33:28.047154+0000","last_active":"2024-02-18T00:33:29.306279+0000","last_peered":"2024-02-18T00:33:29.306279+0000","last_clean":"2024-02-18T00:33:29.306279+0000","last_became_active":"2024-02-18T00:33:23.566498+0000","last_became_peered":"2024-02-18T00:33:23.566498+0000","last_unstale":"2024-02-18T00:33:29.306279+0000","last_undegraded":"2024-02-18T00:33:29.306279+0000","last_fullsized":"2024-02-18T00:33:29.306279+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.376473+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305617+0000","last_change":"2024-02-18T00:33:28.147005+0000","last_active":"2024-02-18T00:33:29.305617+0000","last_peered":"2024-02-18T00:33:29.305617+0000","last_clean":"2024-02-18T00:33:29.305617+0000","last_became_active":"2024-02-18T00:33:23.621779+0000","last_became_peered":"2024-02-18T00:33:23.621779+0000","last_unstale":"2024-02-18T00:33:29.305617+0000","last_undegraded":"2024-02-18T00:33:29.305617+0000","last_fullsized":"2024-02-18T00:33:29.305617+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.351528+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.319996+0000","last_change":"2024-02-18T00:33:28.041493+0000","last_active":"2024-02-18T00:33:29.319996+0000","last_peered":"2024-02-18T00:33:29.319996+0000","last_clean":"2024-02-18T00:33:29.319996+0000","last_became_active":"2024-02-18T00:33:23.599543+0000","last_became_peered":"2024-02-18T00:33:23.599543+0000","last_unstale":"2024-02-18T00:33:29.319996+0000","last_undegraded":"2024-02-18T00:33:29.319996+0000","last_fullsized":"2024-02-18T00:33:29.319996+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.311238+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.9","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.334327+0000","last_change":"2024-02-18T00:33:28.052866+0000","last_active":"2024-02-18T00:33:29.334327+0000","last_peered":"2024-02-18T00:33:29.334327+0000","last_clean":"2024-02-18T00:33:29.334327+0000","last_became_active":"2024-02-18T00:33:23.474433+0000","last_became_peered":"2024-02-18T00:33:23.474433+0000","last_unstale":"2024-02-18T00:33:29.334327+0000","last_undegraded":"2024-02-18T00:33:29.334327+0000","last_fullsized":"2024-02-18T00:33:29.334327+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.338790+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312888+0000","last_change":"2024-02-18T00:33:28.039029+0000","last_active":"2024-02-18T00:33:29.312888+0000","last_peered":"2024-02-18T00:33:29.312888+0000","last_clean":"2024-02-18T00:33:29.312888+0000","last_became_active":"2024-02-18T00:33:23.477241+0000","last_became_peered":"2024-02-18T00:33:23.477241+0000","last_unstale":"2024-02-18T00:33:29.312888+0000","last_undegraded":"2024-02-18T00:33:29.312888+0000","last_fullsized":"2024-02-18T00:33:29.312888+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.338154+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.291354+0000","last_change":"2024-02-18T00:33:28.025790+0000","last_active":"2024-02-18T00:33:29.291354+0000","last_peered":"2024-02-18T00:33:29.291354+0000","last_clean":"2024-02-18T00:33:29.291354+0000","last_became_active":"2024-02-18T00:33:23.415100+0000","last_became_peered":"2024-02-18T00:33:23.415100+0000","last_unstale":"2024-02-18T00:33:29.291354+0000","last_undegraded":"2024-02-18T00:33:29.291354+0000","last_fullsized":"2024-02-18T00:33:29.291354+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"78.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.318871+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309109+0000","last_change":"2024-02-18T00:33:28.038184+0000","last_active":"2024-02-18T00:33:29.309109+0000","last_peered":"2024-02-18T00:33:29.309109+0000","last_clean":"2024-02-18T00:33:29.309109+0000","last_became_active":"2024-02-18T00:33:23.592942+0000","last_became_peered":"2024-02-18T00:33:23.592942+0000","last_unstale":"2024-02-18T00:33:29.309109+0000","last_undegraded":"2024-02-18T00:33:29.309109+0000","last_fullsized":"2024-02-18T00:33:29.309109+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.337516+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.337424+0000","last_change":"2024-02-18T00:33:28.130511+0000","last_active":"2024-02-18T00:33:29.337424+0000","last_peered":"2024-02-18T00:33:29.337424+0000","last_clean":"2024-02-18T00:33:29.337424+0000","last_became_active":"2024-02-18T00:33:23.635584+0000","last_became_peered":"2024-02-18T00:33:23.635584+0000","last_unstale":"2024-02-18T00:33:29.337424+0000","last_undegraded":"2024-02-18T00:33:29.337424+0000","last_fullsized":"2024-02-18T00:33:29.337424+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.352122+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.4","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306387+0000","last_change":"2024-02-18T00:33:28.050283+0000","last_active":"2024-02-18T00:33:29.306387+0000","last_peered":"2024-02-18T00:33:29.306387+0000","last_clean":"2024-02-18T00:33:29.306387+0000","last_became_active":"2024-02-18T00:33:23.454291+0000","last_became_peered":"2024-02-18T00:33:23.454291+0000","last_unstale":"2024-02-18T00:33:29.306387+0000","last_undegraded":"2024-02-18T00:33:29.306387+0000","last_fullsized":"2024-02-18T00:33:29.306387+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.376688+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311745+0000","last_change":"2024-02-18T00:33:28.048891+0000","last_active":"2024-02-18T00:33:29.311745+0000","last_peered":"2024-02-18T00:33:29.311745+0000","last_clean":"2024-02-18T00:33:29.311745+0000","last_became_active":"2024-02-18T00:33:23.458121+0000","last_became_peered":"2024-02-18T00:33:23.458121+0000","last_unstale":"2024-02-18T00:33:29.311745+0000","last_undegraded":"2024-02-18T00:33:29.311745+0000","last_fullsized":"2024-02-18T00:33:29.311745+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.375570+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314423+0000","last_change":"2024-02-18T00:33:28.040802+0000","last_active":"2024-02-18T00:33:29.314423+0000","last_peered":"2024-02-18T00:33:29.314423+0000","last_clean":"2024-02-18T00:33:29.314423+0000","last_became_active":"2024-02-18T00:33:23.456515+0000","last_became_peered":"2024-02-18T00:33:23.456515+0000","last_unstale":"2024-02-18T00:33:29.314423+0000","last_undegraded":"2024-02-18T00:33:29.314423+0000","last_fullsized":"2024-02-18T00:33:29.314423+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.318675+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307244+0000","last_change":"2024-02-18T00:33:28.046398+0000","last_active":"2024-02-18T00:33:29.307244+0000","last_peered":"2024-02-18T00:33:29.307244+0000","last_clean":"2024-02-18T00:33:29.307244+0000","last_became_active":"2024-02-18T00:33:23.458253+0000","last_became_peered":"2024-02-18T00:33:23.458253+0000","last_unstale":"2024-02-18T00:33:29.307244+0000","last_undegraded":"2024-02-18T00:33:29.307244+0000","last_fullsized":"2024-02-18T00:33:29.307244+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.314925+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309507+0000","last_change":"2024-02-18T00:33:28.050789+0000","last_active":"2024-02-18T00:33:29.309507+0000","last_peered":"2024-02-18T00:33:29.309507+0000","last_clean":"2024-02-18T00:33:29.309507+0000","last_became_active":"2024-02-18T00:33:23.471031+0000","last_became_peered":"2024-02-18T00:33:23.471031+0000","last_unstale":"2024-02-18T00:33:29.309507+0000","last_undegraded":"2024-02-18T00:33:29.309507+0000","last_fullsized":"2024-02-18T00:33:29.309507+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.382404+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295566+0000","last_change":"2024-02-18T00:33:28.032497+0000","last_active":"2024-02-18T00:33:29.295566+0000","last_peered":"2024-02-18T00:33:29.295566+0000","last_clean":"2024-02-18T00:33:29.295566+0000","last_became_active":"2024-02-18T00:33:23.457348+0000","last_became_peered":"2024-02-18T00:33:23.457348+0000","last_unstale":"2024-02-18T00:33:29.295566+0000","last_undegraded":"2024-02-18T00:33:29.295566+0000","last_fullsized":"2024-02-18T00:33:29.295566+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"80.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:33.158633+0000","last_change":"2024-02-18T00:33:33.158633+0000","last_active":"2024-02-18T00:33:33.158633+0000","last_peered":"2024-02-18T00:33:33.158633+0000","last_clean":"2024-02-18T00:33:33.158633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:33.158633+0000","last_undegraded":"2024-02-18T00:33:33.158633+0000","last_fullsized":"2024-02-18T00:33:33.158633+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:33.158633+0000","last_clean_scrub_stamp":"2024-02-18T00:33:33.158633+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"79.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:33:32.565590+0000","last_change":"2024-02-18T00:33:32.565590+0000","last_active":"2024-02-18T00:33:32.565590+0000","last_peered":"2024-02-18T00:33:32.565590+0000","last_clean":"2024-02-18T00:33:32.565590+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:32.565590+0000","last_undegraded":"2024-02-18T00:33:32.565590+0000","last_fullsized":"2024-02-18T00:33:32.565590+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:32.565590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:32.565590+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"76.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:33:30.282448+0000","last_change":"2024-02-18T00:33:30.331916+0000","last_active":"2024-02-18T00:33:30.282448+0000","last_peered":"2024-02-18T00:33:30.282448+0000","last_clean":"2024-02-18T00:33:30.282448+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:33:30.282448+0000","last_undegraded":"2024-02-18T00:33:30.282448+0000","last_fullsized":"2024-02-18T00:33:30.282448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:30.282448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:30.282448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"0'0","reported_seq":23,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296067+0000","last_change":"2024-02-18T00:33:28.062033+0000","last_active":"2024-02-18T00:33:29.296067+0000","last_peered":"2024-02-18T00:33:29.296067+0000","last_clean":"2024-02-18T00:33:29.296067+0000","last_became_active":"2024-02-18T00:33:23.490771+0000","last_became_peered":"2024-02-18T00:33:23.490771+0000","last_unstale":"2024-02-18T00:33:29.296067+0000","last_undegraded":"2024-02-18T00:33:29.296067+0000","last_fullsized":"2024-02-18T00:33:29.296067+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"2"}]},{"pgid":"62.0","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310271+0000","last_change":"2024-02-18T00:33:21.197393+0000","last_active":"2024-02-18T00:33:29.310271+0000","last_peered":"2024-02-18T00:33:29.310271+0000","last_clean":"2024-02-18T00:33:29.310271+0000","last_became_active":"2024-02-18T00:33:21.197032+0000","last_became_peered":"2024-02-18T00:33:21.197032+0000","last_unstale":"2024-02-18T00:33:29.310271+0000","last_undegraded":"2024-02-18T00:33:29.310271+0000","last_fullsized":"2024-02-18T00:33:29.310271+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.d","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302193+0000","last_change":"2024-02-18T00:33:13.176264+0000","last_active":"2024-02-18T00:33:29.302193+0000","last_peered":"2024-02-18T00:33:29.302193+0000","last_clean":"2024-02-18T00:33:29.302193+0000","last_became_active":"2024-02-18T00:33:13.175988+0000","last_became_peered":"2024-02-18T00:33:13.175988+0000","last_unstale":"2024-02-18T00:33:29.302193+0000","last_undegraded":"2024-02-18T00:33:29.302193+0000","last_fullsized":"2024-02-18T00:33:29.302193+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.334482+0000","last_change":"2024-02-18T00:33:21.215047+0000","last_active":"2024-02-18T00:33:29.334482+0000","last_peered":"2024-02-18T00:33:29.334482+0000","last_clean":"2024-02-18T00:33:29.334482+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:33:29.334482+0000","last_undegraded":"2024-02-18T00:33:29.334482+0000","last_fullsized":"2024-02-18T00:33:29.334482+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.c","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306727+0000","last_change":"2024-02-18T00:33:13.180556+0000","last_active":"2024-02-18T00:33:29.306727+0000","last_peered":"2024-02-18T00:33:29.306727+0000","last_clean":"2024-02-18T00:33:29.306727+0000","last_became_active":"2024-02-18T00:33:13.180256+0000","last_became_peered":"2024-02-18T00:33:13.180256+0000","last_unstale":"2024-02-18T00:33:29.306727+0000","last_undegraded":"2024-02-18T00:33:29.306727+0000","last_fullsized":"2024-02-18T00:33:29.306727+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.2","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295505+0000","last_change":"2024-02-18T00:33:21.143767+0000","last_active":"2024-02-18T00:33:29.295505+0000","last_peered":"2024-02-18T00:33:29.295505+0000","last_clean":"2024-02-18T00:33:29.295505+0000","last_became_active":"2024-02-18T00:33:21.143454+0000","last_became_peered":"2024-02-18T00:33:21.143454+0000","last_unstale":"2024-02-18T00:33:29.295505+0000","last_undegraded":"2024-02-18T00:33:29.295505+0000","last_fullsized":"2024-02-18T00:33:29.295505+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.f","version":"116'1","reported_seq":27,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296318+0000","last_change":"2024-02-18T00:33:13.182928+0000","last_active":"2024-02-18T00:33:29.296318+0000","last_peered":"2024-02-18T00:33:29.296318+0000","last_clean":"2024-02-18T00:33:29.296318+0000","last_became_active":"2024-02-18T00:33:13.182750+0000","last_became_peered":"2024-02-18T00:33:13.182750+0000","last_unstale":"2024-02-18T00:33:29.296318+0000","last_undegraded":"2024-02-18T00:33:29.296318+0000","last_fullsized":"2024-02-18T00:33:29.296318+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.3","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.346926+0000","last_change":"2024-02-18T00:33:21.169478+0000","last_active":"2024-02-18T00:33:29.346926+0000","last_peered":"2024-02-18T00:33:29.346926+0000","last_clean":"2024-02-18T00:33:29.346926+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:33:29.346926+0000","last_undegraded":"2024-02-18T00:33:29.346926+0000","last_fullsized":"2024-02-18T00:33:29.346926+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.e","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295739+0000","last_change":"2024-02-18T00:33:13.140774+0000","last_active":"2024-02-18T00:33:29.295739+0000","last_peered":"2024-02-18T00:33:29.295739+0000","last_clean":"2024-02-18T00:33:29.295739+0000","last_became_active":"2024-02-18T00:33:13.140452+0000","last_became_peered":"2024-02-18T00:33:13.140452+0000","last_unstale":"2024-02-18T00:33:29.295739+0000","last_undegraded":"2024-02-18T00:33:29.295739+0000","last_fullsized":"2024-02-18T00:33:29.295739+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.4","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320680+0000","last_change":"2024-02-18T00:33:21.182315+0000","last_active":"2024-02-18T00:33:29.320680+0000","last_peered":"2024-02-18T00:33:29.320680+0000","last_clean":"2024-02-18T00:33:29.320680+0000","last_became_active":"2024-02-18T00:33:21.180879+0000","last_became_peered":"2024-02-18T00:33:21.180879+0000","last_unstale":"2024-02-18T00:33:29.320680+0000","last_undegraded":"2024-02-18T00:33:29.320680+0000","last_fullsized":"2024-02-18T00:33:29.320680+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.9","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.294280+0000","last_change":"2024-02-18T00:33:13.166112+0000","last_active":"2024-02-18T00:33:29.294280+0000","last_peered":"2024-02-18T00:33:29.294280+0000","last_clean":"2024-02-18T00:33:29.294280+0000","last_became_active":"2024-02-18T00:33:13.165794+0000","last_became_peered":"2024-02-18T00:33:13.165794+0000","last_unstale":"2024-02-18T00:33:29.294280+0000","last_undegraded":"2024-02-18T00:33:29.294280+0000","last_fullsized":"2024-02-18T00:33:29.294280+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.5","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.303560+0000","last_change":"2024-02-18T00:33:21.205790+0000","last_active":"2024-02-18T00:33:29.303560+0000","last_peered":"2024-02-18T00:33:29.303560+0000","last_clean":"2024-02-18T00:33:29.303560+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:33:29.303560+0000","last_undegraded":"2024-02-18T00:33:29.303560+0000","last_fullsized":"2024-02-18T00:33:29.303560+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.8","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311386+0000","last_change":"2024-02-18T00:33:13.173187+0000","last_active":"2024-02-18T00:33:29.311386+0000","last_peered":"2024-02-18T00:33:29.311386+0000","last_clean":"2024-02-18T00:33:29.311386+0000","last_became_active":"2024-02-18T00:33:13.172912+0000","last_became_peered":"2024-02-18T00:33:13.172912+0000","last_unstale":"2024-02-18T00:33:29.311386+0000","last_undegraded":"2024-02-18T00:33:29.311386+0000","last_fullsized":"2024-02-18T00:33:29.311386+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.6","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.308911+0000","last_change":"2024-02-18T00:33:21.197586+0000","last_active":"2024-02-18T00:33:29.308911+0000","last_peered":"2024-02-18T00:33:29.308911+0000","last_clean":"2024-02-18T00:33:29.308911+0000","last_became_active":"2024-02-18T00:33:21.197086+0000","last_became_peered":"2024-02-18T00:33:21.197086+0000","last_unstale":"2024-02-18T00:33:29.308911+0000","last_undegraded":"2024-02-18T00:33:29.308911+0000","last_fullsized":"2024-02-18T00:33:29.308911+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.b","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.327290+0000","last_change":"2024-02-18T00:33:13.150944+0000","last_active":"2024-02-18T00:33:29.327290+0000","last_peered":"2024-02-18T00:33:29.327290+0000","last_clean":"2024-02-18T00:33:29.327290+0000","last_became_active":"2024-02-18T00:33:13.150511+0000","last_became_peered":"2024-02-18T00:33:13.150511+0000","last_unstale":"2024-02-18T00:33:29.327290+0000","last_undegraded":"2024-02-18T00:33:29.327290+0000","last_fullsized":"2024-02-18T00:33:29.327290+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.7","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.362415+0000","last_change":"2024-02-18T00:33:21.200335+0000","last_active":"2024-02-18T00:33:29.362415+0000","last_peered":"2024-02-18T00:33:29.362415+0000","last_clean":"2024-02-18T00:33:29.362415+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:33:29.362415+0000","last_undegraded":"2024-02-18T00:33:29.362415+0000","last_fullsized":"2024-02-18T00:33:29.362415+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.a","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.304494+0000","last_change":"2024-02-18T00:33:13.179390+0000","last_active":"2024-02-18T00:33:29.304494+0000","last_peered":"2024-02-18T00:33:29.304494+0000","last_clean":"2024-02-18T00:33:29.304494+0000","last_became_active":"2024-02-18T00:33:13.179070+0000","last_became_peered":"2024-02-18T00:33:13.179070+0000","last_unstale":"2024-02-18T00:33:29.304494+0000","last_undegraded":"2024-02-18T00:33:29.304494+0000","last_fullsized":"2024-02-18T00:33:29.304494+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.8","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310193+0000","last_change":"2024-02-18T00:33:21.204653+0000","last_active":"2024-02-18T00:33:29.310193+0000","last_peered":"2024-02-18T00:33:29.310193+0000","last_clean":"2024-02-18T00:33:29.310193+0000","last_became_active":"2024-02-18T00:33:21.204152+0000","last_became_peered":"2024-02-18T00:33:21.204152+0000","last_unstale":"2024-02-18T00:33:29.310193+0000","last_undegraded":"2024-02-18T00:33:29.310193+0000","last_fullsized":"2024-02-18T00:33:29.310193+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.5","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302106+0000","last_change":"2024-02-18T00:33:13.161210+0000","last_active":"2024-02-18T00:33:29.302106+0000","last_peered":"2024-02-18T00:33:29.302106+0000","last_clean":"2024-02-18T00:33:29.302106+0000","last_became_active":"2024-02-18T00:33:13.160885+0000","last_became_peered":"2024-02-18T00:33:13.160885+0000","last_unstale":"2024-02-18T00:33:29.302106+0000","last_undegraded":"2024-02-18T00:33:29.302106+0000","last_fullsized":"2024-02-18T00:33:29.302106+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.9","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.294822+0000","last_change":"2024-02-18T00:33:21.193298+0000","last_active":"2024-02-18T00:33:29.294822+0000","last_peered":"2024-02-18T00:33:29.294822+0000","last_clean":"2024-02-18T00:33:29.294822+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:33:29.294822+0000","last_undegraded":"2024-02-18T00:33:29.294822+0000","last_fullsized":"2024-02-18T00:33:29.294822+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.4","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320963+0000","last_change":"2024-02-18T00:33:13.131288+0000","last_active":"2024-02-18T00:33:29.320963+0000","last_peered":"2024-02-18T00:33:29.320963+0000","last_clean":"2024-02-18T00:33:29.320963+0000","last_became_active":"2024-02-18T00:33:13.130912+0000","last_became_peered":"2024-02-18T00:33:13.130912+0000","last_unstale":"2024-02-18T00:33:29.320963+0000","last_undegraded":"2024-02-18T00:33:29.320963+0000","last_fullsized":"2024-02-18T00:33:29.320963+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.a","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.303097+0000","last_change":"2024-02-18T00:33:21.197499+0000","last_active":"2024-02-18T00:33:29.303097+0000","last_peered":"2024-02-18T00:33:29.303097+0000","last_clean":"2024-02-18T00:33:29.303097+0000","last_became_active":"2024-02-18T00:33:21.197236+0000","last_became_peered":"2024-02-18T00:33:21.197236+0000","last_unstale":"2024-02-18T00:33:29.303097+0000","last_undegraded":"2024-02-18T00:33:29.303097+0000","last_fullsized":"2024-02-18T00:33:29.303097+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.7","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311367+0000","last_change":"2024-02-18T00:33:13.152434+0000","last_active":"2024-02-18T00:33:29.311367+0000","last_peered":"2024-02-18T00:33:29.311367+0000","last_clean":"2024-02-18T00:33:29.311367+0000","last_became_active":"2024-02-18T00:33:13.152141+0000","last_became_peered":"2024-02-18T00:33:13.152141+0000","last_unstale":"2024-02-18T00:33:29.311367+0000","last_undegraded":"2024-02-18T00:33:29.311367+0000","last_fullsized":"2024-02-18T00:33:29.311367+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.b","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305512+0000","last_change":"2024-02-18T00:33:21.190838+0000","last_active":"2024-02-18T00:33:29.305512+0000","last_peered":"2024-02-18T00:33:29.305512+0000","last_clean":"2024-02-18T00:33:29.305512+0000","last_became_active":"2024-02-18T00:33:21.190579+0000","last_became_peered":"2024-02-18T00:33:21.190579+0000","last_unstale":"2024-02-18T00:33:29.305512+0000","last_undegraded":"2024-02-18T00:33:29.305512+0000","last_fullsized":"2024-02-18T00:33:29.305512+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.6","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315833+0000","last_change":"2024-02-18T00:33:13.178541+0000","last_active":"2024-02-18T00:33:29.315833+0000","last_peered":"2024-02-18T00:33:29.315833+0000","last_clean":"2024-02-18T00:33:29.315833+0000","last_became_active":"2024-02-18T00:33:13.178304+0000","last_became_peered":"2024-02-18T00:33:13.178304+0000","last_unstale":"2024-02-18T00:33:29.315833+0000","last_undegraded":"2024-02-18T00:33:29.315833+0000","last_fullsized":"2024-02-18T00:33:29.315833+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.c","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.318583+0000","last_change":"2024-02-18T00:33:21.191019+0000","last_active":"2024-02-18T00:33:29.318583+0000","last_peered":"2024-02-18T00:33:29.318583+0000","last_clean":"2024-02-18T00:33:29.318583+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:33:29.318583+0000","last_undegraded":"2024-02-18T00:33:29.318583+0000","last_fullsized":"2024-02-18T00:33:29.318583+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.1","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305374+0000","last_change":"2024-02-18T00:33:13.167068+0000","last_active":"2024-02-18T00:33:29.305374+0000","last_peered":"2024-02-18T00:33:29.305374+0000","last_clean":"2024-02-18T00:33:29.305374+0000","last_became_active":"2024-02-18T00:33:13.164988+0000","last_became_peered":"2024-02-18T00:33:13.164988+0000","last_unstale":"2024-02-18T00:33:29.305374+0000","last_undegraded":"2024-02-18T00:33:29.305374+0000","last_fullsized":"2024-02-18T00:33:29.305374+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.303483+0000","last_change":"2024-02-18T00:33:21.178176+0000","last_active":"2024-02-18T00:33:29.303483+0000","last_peered":"2024-02-18T00:33:29.303483+0000","last_clean":"2024-02-18T00:33:29.303483+0000","last_became_active":"2024-02-18T00:33:21.177885+0000","last_became_peered":"2024-02-18T00:33:21.177885+0000","last_unstale":"2024-02-18T00:33:29.303483+0000","last_undegraded":"2024-02-18T00:33:29.303483+0000","last_fullsized":"2024-02-18T00:33:29.303483+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.0","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299560+0000","last_change":"2024-02-18T00:33:13.154609+0000","last_active":"2024-02-18T00:33:29.299560+0000","last_peered":"2024-02-18T00:33:29.299560+0000","last_clean":"2024-02-18T00:33:29.299560+0000","last_became_active":"2024-02-18T00:33:13.154324+0000","last_became_peered":"2024-02-18T00:33:13.154324+0000","last_unstale":"2024-02-18T00:33:29.299560+0000","last_undegraded":"2024-02-18T00:33:29.299560+0000","last_fullsized":"2024-02-18T00:33:29.299560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.e","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320077+0000","last_change":"2024-02-18T00:33:21.155283+0000","last_active":"2024-02-18T00:33:29.320077+0000","last_peered":"2024-02-18T00:33:29.320077+0000","last_clean":"2024-02-18T00:33:29.320077+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:33:29.320077+0000","last_undegraded":"2024-02-18T00:33:29.320077+0000","last_fullsized":"2024-02-18T00:33:29.320077+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.3","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.321227+0000","last_change":"2024-02-18T00:33:13.169706+0000","last_active":"2024-02-18T00:33:29.321227+0000","last_peered":"2024-02-18T00:33:29.321227+0000","last_clean":"2024-02-18T00:33:29.321227+0000","last_became_active":"2024-02-18T00:33:13.169446+0000","last_became_peered":"2024-02-18T00:33:13.169446+0000","last_unstale":"2024-02-18T00:33:29.321227+0000","last_undegraded":"2024-02-18T00:33:29.321227+0000","last_fullsized":"2024-02-18T00:33:29.321227+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.f","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296148+0000","last_change":"2024-02-18T00:33:21.176269+0000","last_active":"2024-02-18T00:33:29.296148+0000","last_peered":"2024-02-18T00:33:29.296148+0000","last_clean":"2024-02-18T00:33:29.296148+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:33:29.296148+0000","last_undegraded":"2024-02-18T00:33:29.296148+0000","last_fullsized":"2024-02-18T00:33:29.296148+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.2","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315354+0000","last_change":"2024-02-18T00:33:13.167303+0000","last_active":"2024-02-18T00:33:29.315354+0000","last_peered":"2024-02-18T00:33:29.315354+0000","last_clean":"2024-02-18T00:33:29.315354+0000","last_became_active":"2024-02-18T00:33:13.167010+0000","last_became_peered":"2024-02-18T00:33:13.167010+0000","last_unstale":"2024-02-18T00:33:29.315354+0000","last_undegraded":"2024-02-18T00:33:29.315354+0000","last_fullsized":"2024-02-18T00:33:29.315354+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.10","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328344+0000","last_change":"2024-02-18T00:33:21.214944+0000","last_active":"2024-02-18T00:33:29.328344+0000","last_peered":"2024-02-18T00:33:29.328344+0000","last_clean":"2024-02-18T00:33:29.328344+0000","last_became_active":"2024-02-18T00:33:21.214672+0000","last_became_peered":"2024-02-18T00:33:21.214672+0000","last_unstale":"2024-02-18T00:33:29.328344+0000","last_undegraded":"2024-02-18T00:33:29.328344+0000","last_fullsized":"2024-02-18T00:33:29.328344+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1d","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306661+0000","last_change":"2024-02-18T00:33:13.171793+0000","last_active":"2024-02-18T00:33:29.306661+0000","last_peered":"2024-02-18T00:33:29.306661+0000","last_clean":"2024-02-18T00:33:29.306661+0000","last_became_active":"2024-02-18T00:33:13.171322+0000","last_became_peered":"2024-02-18T00:33:13.171322+0000","last_unstale":"2024-02-18T00:33:29.306661+0000","last_undegraded":"2024-02-18T00:33:29.306661+0000","last_fullsized":"2024-02-18T00:33:29.306661+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.334407+0000","last_change":"2024-02-18T00:33:21.220131+0000","last_active":"2024-02-18T00:33:29.334407+0000","last_peered":"2024-02-18T00:33:29.334407+0000","last_clean":"2024-02-18T00:33:29.334407+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:33:29.334407+0000","last_undegraded":"2024-02-18T00:33:29.334407+0000","last_fullsized":"2024-02-18T00:33:29.334407+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"51.1c","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306805+0000","last_change":"2024-02-18T00:33:13.182690+0000","last_active":"2024-02-18T00:33:29.306805+0000","last_peered":"2024-02-18T00:33:29.306805+0000","last_clean":"2024-02-18T00:33:29.306805+0000","last_became_active":"2024-02-18T00:33:13.182395+0000","last_became_peered":"2024-02-18T00:33:13.182395+0000","last_unstale":"2024-02-18T00:33:29.306805+0000","last_undegraded":"2024-02-18T00:33:29.306805+0000","last_fullsized":"2024-02-18T00:33:29.306805+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.12","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.348805+0000","last_change":"2024-02-18T00:33:21.200686+0000","last_active":"2024-02-18T00:33:29.348805+0000","last_peered":"2024-02-18T00:33:29.348805+0000","last_clean":"2024-02-18T00:33:29.348805+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:33:29.348805+0000","last_undegraded":"2024-02-18T00:33:29.348805+0000","last_fullsized":"2024-02-18T00:33:29.348805+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.1f","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312582+0000","last_change":"2024-02-18T00:33:13.175268+0000","last_active":"2024-02-18T00:33:29.312582+0000","last_peered":"2024-02-18T00:33:29.312582+0000","last_clean":"2024-02-18T00:33:29.312582+0000","last_became_active":"2024-02-18T00:33:13.174941+0000","last_became_peered":"2024-02-18T00:33:13.174941+0000","last_unstale":"2024-02-18T00:33:29.312582+0000","last_undegraded":"2024-02-18T00:33:29.312582+0000","last_fullsized":"2024-02-18T00:33:29.312582+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.13","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.326623+0000","last_change":"2024-02-18T00:33:21.224632+0000","last_active":"2024-02-18T00:33:29.326623+0000","last_peered":"2024-02-18T00:33:29.326623+0000","last_clean":"2024-02-18T00:33:29.326623+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:33:29.326623+0000","last_undegraded":"2024-02-18T00:33:29.326623+0000","last_fullsized":"2024-02-18T00:33:29.326623+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"51.1e","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.335384+0000","last_change":"2024-02-18T00:33:13.164463+0000","last_active":"2024-02-18T00:33:29.335384+0000","last_peered":"2024-02-18T00:33:29.335384+0000","last_clean":"2024-02-18T00:33:29.335384+0000","last_became_active":"2024-02-18T00:33:13.164131+0000","last_became_peered":"2024-02-18T00:33:13.164131+0000","last_unstale":"2024-02-18T00:33:29.335384+0000","last_undegraded":"2024-02-18T00:33:29.335384+0000","last_fullsized":"2024-02-18T00:33:29.335384+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295552+0000","last_change":"2024-02-18T00:33:21.225573+0000","last_active":"2024-02-18T00:33:29.295552+0000","last_peered":"2024-02-18T00:33:29.295552+0000","last_clean":"2024-02-18T00:33:29.295552+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:33:29.295552+0000","last_undegraded":"2024-02-18T00:33:29.295552+0000","last_fullsized":"2024-02-18T00:33:29.295552+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.1b","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307302+0000","last_change":"2024-02-18T00:33:13.179350+0000","last_active":"2024-02-18T00:33:29.307302+0000","last_peered":"2024-02-18T00:33:29.307302+0000","last_clean":"2024-02-18T00:33:29.307302+0000","last_became_active":"2024-02-18T00:33:13.179052+0000","last_became_peered":"2024-02-18T00:33:13.179052+0000","last_unstale":"2024-02-18T00:33:29.307302+0000","last_undegraded":"2024-02-18T00:33:29.307302+0000","last_fullsized":"2024-02-18T00:33:29.307302+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.17","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.308529+0000","last_change":"2024-02-18T00:33:21.138761+0000","last_active":"2024-02-18T00:33:29.308529+0000","last_peered":"2024-02-18T00:33:29.308529+0000","last_clean":"2024-02-18T00:33:29.308529+0000","last_became_active":"2024-02-18T00:33:21.138414+0000","last_became_peered":"2024-02-18T00:33:21.138414+0000","last_unstale":"2024-02-18T00:33:29.308529+0000","last_undegraded":"2024-02-18T00:33:29.308529+0000","last_fullsized":"2024-02-18T00:33:29.308529+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.1a","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310458+0000","last_change":"2024-02-18T00:33:13.164050+0000","last_active":"2024-02-18T00:33:29.310458+0000","last_peered":"2024-02-18T00:33:29.310458+0000","last_clean":"2024-02-18T00:33:29.310458+0000","last_became_active":"2024-02-18T00:33:13.163452+0000","last_became_peered":"2024-02-18T00:33:13.163452+0000","last_unstale":"2024-02-18T00:33:29.310458+0000","last_undegraded":"2024-02-18T00:33:29.310458+0000","last_fullsized":"2024-02-18T00:33:29.310458+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.15","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.301543+0000","last_change":"2024-02-18T00:33:21.198854+0000","last_active":"2024-02-18T00:33:29.301543+0000","last_peered":"2024-02-18T00:33:29.301543+0000","last_clean":"2024-02-18T00:33:29.301543+0000","last_became_active":"2024-02-18T00:33:21.198486+0000","last_became_peered":"2024-02-18T00:33:21.198486+0000","last_unstale":"2024-02-18T00:33:29.301543+0000","last_undegraded":"2024-02-18T00:33:29.301543+0000","last_fullsized":"2024-02-18T00:33:29.301543+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.18","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.291530+0000","last_change":"2024-02-18T00:33:13.133830+0000","last_active":"2024-02-18T00:33:29.291530+0000","last_peered":"2024-02-18T00:33:29.291530+0000","last_clean":"2024-02-18T00:33:29.291530+0000","last_became_active":"2024-02-18T00:33:13.133132+0000","last_became_peered":"2024-02-18T00:33:13.133132+0000","last_unstale":"2024-02-18T00:33:29.291530+0000","last_undegraded":"2024-02-18T00:33:29.291530+0000","last_fullsized":"2024-02-18T00:33:29.291530+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,2],"acting":[4,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":32,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.294434+0000","last_change":"2024-02-18T00:33:05.591442+0000","last_active":"2024-02-18T00:33:29.294434+0000","last_peered":"2024-02-18T00:33:29.294434+0000","last_clean":"2024-02-18T00:33:29.294434+0000","last_became_active":"2024-02-18T00:33:05.591101+0000","last_became_peered":"2024-02-18T00:33:05.591101+0000","last_unstale":"2024-02-18T00:33:29.294434+0000","last_undegraded":"2024-02-18T00:33:29.294434+0000","last_fullsized":"2024-02-18T00:33:29.294434+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.960661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.960661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306517+0000","last_change":"2024-02-18T00:33:21.200063+0000","last_active":"2024-02-18T00:33:29.306517+0000","last_peered":"2024-02-18T00:33:29.306517+0000","last_clean":"2024-02-18T00:33:29.306517+0000","last_became_active":"2024-02-18T00:33:21.199758+0000","last_became_peered":"2024-02-18T00:33:21.199758+0000","last_unstale":"2024-02-18T00:33:29.306517+0000","last_undegraded":"2024-02-18T00:33:29.306517+0000","last_fullsized":"2024-02-18T00:33:29.306517+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.19","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.335588+0000","last_change":"2024-02-18T00:33:13.164719+0000","last_active":"2024-02-18T00:33:29.335588+0000","last_peered":"2024-02-18T00:33:29.335588+0000","last_clean":"2024-02-18T00:33:29.335588+0000","last_became_active":"2024-02-18T00:33:13.164376+0000","last_became_peered":"2024-02-18T00:33:13.164376+0000","last_unstale":"2024-02-18T00:33:29.335588+0000","last_undegraded":"2024-02-18T00:33:29.335588+0000","last_fullsized":"2024-02-18T00:33:29.335588+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":32,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311701+0000","last_change":"2024-02-18T00:33:05.558071+0000","last_active":"2024-02-18T00:33:29.311701+0000","last_peered":"2024-02-18T00:33:29.311701+0000","last_clean":"2024-02-18T00:33:29.311701+0000","last_became_active":"2024-02-18T00:33:05.557534+0000","last_became_peered":"2024-02-18T00:33:05.557534+0000","last_unstale":"2024-02-18T00:33:29.311701+0000","last_undegraded":"2024-02-18T00:33:29.311701+0000","last_fullsized":"2024-02-18T00:33:29.311701+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:48.594640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:48.594640+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.291480+0000","last_change":"2024-02-18T00:33:21.225639+0000","last_active":"2024-02-18T00:33:29.291480+0000","last_peered":"2024-02-18T00:33:29.291480+0000","last_clean":"2024-02-18T00:33:29.291480+0000","last_became_active":"2024-02-18T00:33:21.225365+0000","last_became_peered":"2024-02-18T00:33:21.225365+0000","last_unstale":"2024-02-18T00:33:29.291480+0000","last_undegraded":"2024-02-18T00:33:29.291480+0000","last_fullsized":"2024-02-18T00:33:29.291480+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"51.16","version":"116'3","reported_seq":29,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295643+0000","last_change":"2024-02-18T00:33:13.133297+0000","last_active":"2024-02-18T00:33:29.295643+0000","last_peered":"2024-02-18T00:33:29.295643+0000","last_clean":"2024-02-18T00:33:29.295643+0000","last_became_active":"2024-02-18T00:33:13.132997+0000","last_became_peered":"2024-02-18T00:33:13.132997+0000","last_unstale":"2024-02-18T00:33:29.295643+0000","last_undegraded":"2024-02-18T00:33:29.295643+0000","last_fullsized":"2024-02-18T00:33:29.295643+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":512,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312658+0000","last_change":"2024-02-18T00:33:05.577170+0000","last_active":"2024-02-18T00:33:29.312658+0000","last_peered":"2024-02-18T00:33:29.312658+0000","last_clean":"2024-02-18T00:33:29.312658+0000","last_became_active":"2024-02-18T00:33:05.576817+0000","last_became_peered":"2024-02-18T00:33:05.576817+0000","last_unstale":"2024-02-18T00:33:29.312658+0000","last_undegraded":"2024-02-18T00:33:29.312658+0000","last_fullsized":"2024-02-18T00:33:29.312658+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.198542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.198542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1a","version":"126'1","reported_seq":19,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295439+0000","last_change":"2024-02-18T00:33:21.200854+0000","last_active":"2024-02-18T00:33:29.295439+0000","last_peered":"2024-02-18T00:33:29.295439+0000","last_clean":"2024-02-18T00:33:29.295439+0000","last_became_active":"2024-02-18T00:33:21.200536+0000","last_became_peered":"2024-02-18T00:33:21.200536+0000","last_unstale":"2024-02-18T00:33:29.295439+0000","last_undegraded":"2024-02-18T00:33:29.295439+0000","last_fullsized":"2024-02-18T00:33:29.295439+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.17","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311846+0000","last_change":"2024-02-18T00:33:13.179504+0000","last_active":"2024-02-18T00:33:29.311846+0000","last_peered":"2024-02-18T00:33:29.311846+0000","last_clean":"2024-02-18T00:33:29.311846+0000","last_became_active":"2024-02-18T00:33:13.176379+0000","last_became_peered":"2024-02-18T00:33:13.176379+0000","last_unstale":"2024-02-18T00:33:29.311846+0000","last_undegraded":"2024-02-18T00:33:29.311846+0000","last_fullsized":"2024-02-18T00:33:29.311846+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":32,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309585+0000","last_change":"2024-02-18T00:33:05.575605+0000","last_active":"2024-02-18T00:33:29.309585+0000","last_peered":"2024-02-18T00:33:29.309585+0000","last_clean":"2024-02-18T00:33:29.309585+0000","last_became_active":"2024-02-18T00:33:05.575295+0000","last_became_peered":"2024-02-18T00:33:05.575295+0000","last_unstale":"2024-02-18T00:33:29.309585+0000","last_undegraded":"2024-02-18T00:33:29.309585+0000","last_fullsized":"2024-02-18T00:33:29.309585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.19","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.294331+0000","last_change":"2024-02-18T00:33:21.200080+0000","last_active":"2024-02-18T00:33:29.294331+0000","last_peered":"2024-02-18T00:33:29.294331+0000","last_clean":"2024-02-18T00:33:29.294331+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:33:29.294331+0000","last_undegraded":"2024-02-18T00:33:29.294331+0000","last_fullsized":"2024-02-18T00:33:29.294331+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.14","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.319640+0000","last_change":"2024-02-18T00:33:13.175920+0000","last_active":"2024-02-18T00:33:29.319640+0000","last_peered":"2024-02-18T00:33:29.319640+0000","last_clean":"2024-02-18T00:33:29.319640+0000","last_became_active":"2024-02-18T00:33:13.166627+0000","last_became_peered":"2024-02-18T00:33:13.166627+0000","last_unstale":"2024-02-18T00:33:29.319640+0000","last_undegraded":"2024-02-18T00:33:29.319640+0000","last_fullsized":"2024-02-18T00:33:29.319640+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":47,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307978+0000","last_change":"2024-02-18T00:33:05.668879+0000","last_active":"2024-02-18T00:33:29.307978+0000","last_peered":"2024-02-18T00:33:29.307978+0000","last_clean":"2024-02-18T00:33:29.307978+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:33:29.307978+0000","last_undegraded":"2024-02-18T00:33:29.307978+0000","last_fullsized":"2024-02-18T00:33:29.307978+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:32:51.632894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:51.632894+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":437,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.301208+0000","last_change":"2024-02-18T00:33:21.203075+0000","last_active":"2024-02-18T00:33:29.301208+0000","last_peered":"2024-02-18T00:33:29.301208+0000","last_clean":"2024-02-18T00:33:29.301208+0000","last_became_active":"2024-02-18T00:33:21.202614+0000","last_became_peered":"2024-02-18T00:33:21.202614+0000","last_unstale":"2024-02-18T00:33:29.301208+0000","last_undegraded":"2024-02-18T00:33:29.301208+0000","last_fullsized":"2024-02-18T00:33:29.301208+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"51.15","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.338624+0000","last_change":"2024-02-18T00:33:13.172696+0000","last_active":"2024-02-18T00:33:29.338624+0000","last_peered":"2024-02-18T00:33:29.338624+0000","last_clean":"2024-02-18T00:33:29.338624+0000","last_became_active":"2024-02-18T00:33:13.172420+0000","last_became_peered":"2024-02-18T00:33:13.172420+0000","last_unstale":"2024-02-18T00:33:29.338624+0000","last_undegraded":"2024-02-18T00:33:29.338624+0000","last_fullsized":"2024-02-18T00:33:29.338624+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":32,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.303006+0000","last_change":"2024-02-18T00:33:05.593083+0000","last_active":"2024-02-18T00:33:29.303006+0000","last_peered":"2024-02-18T00:33:29.303006+0000","last_clean":"2024-02-18T00:33:29.303006+0000","last_became_active":"2024-02-18T00:33:05.592776+0000","last_became_peered":"2024-02-18T00:33:05.592776+0000","last_unstale":"2024-02-18T00:33:29.303006+0000","last_undegraded":"2024-02-18T00:33:29.303006+0000","last_fullsized":"2024-02-18T00:33:29.303006+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.839018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.839018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.362376+0000","last_change":"2024-02-18T00:33:21.200077+0000","last_active":"2024-02-18T00:33:29.362376+0000","last_peered":"2024-02-18T00:33:29.362376+0000","last_clean":"2024-02-18T00:33:29.362376+0000","last_became_active":"2024-02-18T00:33:21.199831+0000","last_became_peered":"2024-02-18T00:33:21.199831+0000","last_unstale":"2024-02-18T00:33:29.362376+0000","last_undegraded":"2024-02-18T00:33:29.362376+0000","last_fullsized":"2024-02-18T00:33:29.362376+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"51.12","version":"116'1","reported_seq":27,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295702+0000","last_change":"2024-02-18T00:33:13.166252+0000","last_active":"2024-02-18T00:33:29.295702+0000","last_peered":"2024-02-18T00:33:29.295702+0000","last_clean":"2024-02-18T00:33:29.295702+0000","last_became_active":"2024-02-18T00:33:13.166017+0000","last_became_peered":"2024-02-18T00:33:13.166017+0000","last_unstale":"2024-02-18T00:33:29.295702+0000","last_undegraded":"2024-02-18T00:33:29.295702+0000","last_fullsized":"2024-02-18T00:33:29.295702+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":32,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.322370+0000","last_change":"2024-02-18T00:33:05.575803+0000","last_active":"2024-02-18T00:33:29.322370+0000","last_peered":"2024-02-18T00:33:29.322370+0000","last_clean":"2024-02-18T00:33:29.322370+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:33:29.322370+0000","last_undegraded":"2024-02-18T00:33:29.322370+0000","last_fullsized":"2024-02-18T00:33:29.322370+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.308832+0000","last_change":"2024-02-18T00:33:21.197924+0000","last_active":"2024-02-18T00:33:29.308832+0000","last_peered":"2024-02-18T00:33:29.308832+0000","last_clean":"2024-02-18T00:33:29.308832+0000","last_became_active":"2024-02-18T00:33:21.197666+0000","last_became_peered":"2024-02-18T00:33:21.197666+0000","last_unstale":"2024-02-18T00:33:29.308832+0000","last_undegraded":"2024-02-18T00:33:29.308832+0000","last_fullsized":"2024-02-18T00:33:29.308832+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"51.13","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.327214+0000","last_change":"2024-02-18T00:33:13.173929+0000","last_active":"2024-02-18T00:33:29.327214+0000","last_peered":"2024-02-18T00:33:29.327214+0000","last_clean":"2024-02-18T00:33:29.327214+0000","last_became_active":"2024-02-18T00:33:13.173641+0000","last_became_peered":"2024-02-18T00:33:13.173641+0000","last_unstale":"2024-02-18T00:33:29.327214+0000","last_undegraded":"2024-02-18T00:33:29.327214+0000","last_fullsized":"2024-02-18T00:33:29.327214+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320978+0000","last_change":"2024-02-18T00:33:05.592130+0000","last_active":"2024-02-18T00:33:29.320978+0000","last_peered":"2024-02-18T00:33:29.320978+0000","last_clean":"2024-02-18T00:33:29.320978+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:33:29.320978+0000","last_undegraded":"2024-02-18T00:33:29.320978+0000","last_fullsized":"2024-02-18T00:33:29.320978+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.991759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.991759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305194+0000","last_change":"2024-02-18T00:33:21.133096+0000","last_active":"2024-02-18T00:33:29.305194+0000","last_peered":"2024-02-18T00:33:29.305194+0000","last_clean":"2024-02-18T00:33:29.305194+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:33:29.305194+0000","last_undegraded":"2024-02-18T00:33:29.305194+0000","last_fullsized":"2024-02-18T00:33:29.305194+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"51.10","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.292269+0000","last_change":"2024-02-18T00:33:13.134485+0000","last_active":"2024-02-18T00:33:29.292269+0000","last_peered":"2024-02-18T00:33:29.292269+0000","last_clean":"2024-02-18T00:33:29.292269+0000","last_became_active":"2024-02-18T00:33:13.134204+0000","last_became_peered":"2024-02-18T00:33:13.134204+0000","last_unstale":"2024-02-18T00:33:29.292269+0000","last_undegraded":"2024-02-18T00:33:29.292269+0000","last_fullsized":"2024-02-18T00:33:29.292269+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.356656+0000","last_change":"2024-02-18T00:33:05.591899+0000","last_active":"2024-02-18T00:33:29.356656+0000","last_peered":"2024-02-18T00:33:29.356656+0000","last_clean":"2024-02-18T00:33:29.356656+0000","last_became_active":"2024-02-18T00:33:05.591566+0000","last_became_peered":"2024-02-18T00:33:05.591566+0000","last_unstale":"2024-02-18T00:33:29.356656+0000","last_undegraded":"2024-02-18T00:33:29.356656+0000","last_fullsized":"2024-02-18T00:33:29.356656+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.521630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.521630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"0'0","reported_seq":17,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302827+0000","last_change":"2024-02-18T00:33:21.155005+0000","last_active":"2024-02-18T00:33:29.302827+0000","last_peered":"2024-02-18T00:33:29.302827+0000","last_clean":"2024-02-18T00:33:29.302827+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:33:29.302827+0000","last_undegraded":"2024-02-18T00:33:29.302827+0000","last_fullsized":"2024-02-18T00:33:29.302827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"51.11","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302696+0000","last_change":"2024-02-18T00:33:13.155349+0000","last_active":"2024-02-18T00:33:29.302696+0000","last_peered":"2024-02-18T00:33:29.302696+0000","last_clean":"2024-02-18T00:33:29.302696+0000","last_became_active":"2024-02-18T00:33:13.154940+0000","last_became_peered":"2024-02-18T00:33:13.154940+0000","last_unstale":"2024-02-18T00:33:29.302696+0000","last_undegraded":"2024-02-18T00:33:29.302696+0000","last_fullsized":"2024-02-18T00:33:29.302696+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:12.079296+0000","last_clean_scrub_stamp":"2024-02-18T00:33:12.079296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311582+0000","last_change":"2024-02-18T00:33:05.574422+0000","last_active":"2024-02-18T00:33:29.311582+0000","last_peered":"2024-02-18T00:33:29.311582+0000","last_clean":"2024-02-18T00:33:29.311582+0000","last_became_active":"2024-02-18T00:33:05.574089+0000","last_became_peered":"2024-02-18T00:33:05.574089+0000","last_unstale":"2024-02-18T00:33:29.311582+0000","last_undegraded":"2024-02-18T00:33:29.311582+0000","last_fullsized":"2024-02-18T00:33:29.311582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.754417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.754417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.16","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296541+0000","last_change":"2024-02-18T00:33:02.535342+0000","last_active":"2024-02-18T00:33:29.296541+0000","last_peered":"2024-02-18T00:33:29.296541+0000","last_clean":"2024-02-18T00:33:29.296541+0000","last_became_active":"2024-02-18T00:33:02.534729+0000","last_became_peered":"2024-02-18T00:33:02.534729+0000","last_unstale":"2024-02-18T00:33:29.296541+0000","last_undegraded":"2024-02-18T00:33:29.296541+0000","last_fullsized":"2024-02-18T00:33:29.296541+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320234+0000","last_change":"2024-02-18T00:33:04.663714+0000","last_active":"2024-02-18T00:33:29.320234+0000","last_peered":"2024-02-18T00:33:29.320234+0000","last_clean":"2024-02-18T00:33:29.320234+0000","last_became_active":"2024-02-18T00:33:04.663445+0000","last_became_peered":"2024-02-18T00:33:04.663445+0000","last_unstale":"2024-02-18T00:33:29.320234+0000","last_undegraded":"2024-02-18T00:33:29.320234+0000","last_fullsized":"2024-02-18T00:33:29.320234+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"64'8","reported_seq":149,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311663+0000","last_change":"2024-02-18T00:32:50.594884+0000","last_active":"2024-02-18T00:33:29.311663+0000","last_peered":"2024-02-18T00:33:29.311663+0000","last_clean":"2024-02-18T00:33:29.311663+0000","last_became_active":"2024-02-18T00:29:19.101012+0000","last_became_peered":"2024-02-18T00:29:19.101012+0000","last_unstale":"2024-02-18T00:33:29.311663+0000","last_undegraded":"2024-02-18T00:33:29.311663+0000","last_fullsized":"2024-02-18T00:33:29.311663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:50.594799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.594799+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":147,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299661+0000","last_change":"2024-02-18T00:32:46.877329+0000","last_active":"2024-02-18T00:33:29.299661+0000","last_peered":"2024-02-18T00:33:29.299661+0000","last_clean":"2024-02-18T00:33:29.299661+0000","last_became_active":"2024-02-18T00:29:17.800167+0000","last_became_peered":"2024-02-18T00:29:17.800167+0000","last_unstale":"2024-02-18T00:33:29.299661+0000","last_undegraded":"2024-02-18T00:33:29.299661+0000","last_fullsized":"2024-02-18T00:33:29.299661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:32:46.877252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.877252+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.360004+0000","last_change":"2024-02-18T00:31:57.561503+0000","last_active":"2024-02-18T00:33:29.360004+0000","last_peered":"2024-02-18T00:33:29.360004+0000","last_clean":"2024-02-18T00:33:29.360004+0000","last_became_active":"2024-02-18T00:29:14.710326+0000","last_became_peered":"2024-02-18T00:29:14.710326+0000","last_unstale":"2024-02-18T00:33:29.360004+0000","last_undegraded":"2024-02-18T00:33:29.360004+0000","last_fullsized":"2024-02-18T00:33:29.360004+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.561435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.561435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305534+0000","last_change":"2024-02-18T00:32:05.790164+0000","last_active":"2024-02-18T00:33:29.305534+0000","last_peered":"2024-02-18T00:33:29.305534+0000","last_clean":"2024-02-18T00:33:29.305534+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:33:29.305534+0000","last_undegraded":"2024-02-18T00:33:29.305534+0000","last_fullsized":"2024-02-18T00:33:29.305534+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.790091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.790091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311265+0000","last_change":"2024-02-18T00:33:01.472792+0000","last_active":"2024-02-18T00:33:29.311265+0000","last_peered":"2024-02-18T00:33:29.311265+0000","last_clean":"2024-02-18T00:33:29.311265+0000","last_became_active":"2024-02-18T00:33:01.472120+0000","last_became_peered":"2024-02-18T00:33:01.472120+0000","last_unstale":"2024-02-18T00:33:29.311265+0000","last_undegraded":"2024-02-18T00:33:29.311265+0000","last_fullsized":"2024-02-18T00:33:29.311265+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302326+0000","last_change":"2024-02-18T00:33:04.600509+0000","last_active":"2024-02-18T00:33:29.302326+0000","last_peered":"2024-02-18T00:33:29.302326+0000","last_clean":"2024-02-18T00:33:29.302326+0000","last_became_active":"2024-02-18T00:33:04.600249+0000","last_became_peered":"2024-02-18T00:33:04.600249+0000","last_unstale":"2024-02-18T00:33:29.302326+0000","last_undegraded":"2024-02-18T00:33:29.302326+0000","last_fullsized":"2024-02-18T00:33:29.302326+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.301735+0000","last_change":"2024-02-18T00:33:02.535442+0000","last_active":"2024-02-18T00:33:29.301735+0000","last_peered":"2024-02-18T00:33:29.301735+0000","last_clean":"2024-02-18T00:33:29.301735+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:33:29.301735+0000","last_undegraded":"2024-02-18T00:33:29.301735+0000","last_fullsized":"2024-02-18T00:33:29.301735+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":122,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328342+0000","last_change":"2024-02-18T00:32:50.007673+0000","last_active":"2024-02-18T00:33:29.328342+0000","last_peered":"2024-02-18T00:33:29.328342+0000","last_clean":"2024-02-18T00:33:29.328342+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:33:29.328342+0000","last_undegraded":"2024-02-18T00:33:29.328342+0000","last_fullsized":"2024-02-18T00:33:29.328342+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.007594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.007594+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.333320+0000","last_change":"2024-02-18T00:32:03.931078+0000","last_active":"2024-02-18T00:33:29.333320+0000","last_peered":"2024-02-18T00:33:29.333320+0000","last_clean":"2024-02-18T00:33:29.333320+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:33:29.333320+0000","last_undegraded":"2024-02-18T00:33:29.333320+0000","last_fullsized":"2024-02-18T00:33:29.333320+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:32:03.930967+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.350127+0000","last_change":"2024-02-18T00:32:00.527642+0000","last_active":"2024-02-18T00:33:29.350127+0000","last_peered":"2024-02-18T00:33:29.350127+0000","last_clean":"2024-02-18T00:33:29.350127+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:33:29.350127+0000","last_undegraded":"2024-02-18T00:33:29.350127+0000","last_fullsized":"2024-02-18T00:33:29.350127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:00.527580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:00.527580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314297+0000","last_change":"2024-02-18T00:32:39.973147+0000","last_active":"2024-02-18T00:33:29.314297+0000","last_peered":"2024-02-18T00:33:29.314297+0000","last_clean":"2024-02-18T00:33:29.314297+0000","last_became_active":"2024-02-18T00:29:21.091776+0000","last_became_peered":"2024-02-18T00:29:21.091776+0000","last_unstale":"2024-02-18T00:33:29.314297+0000","last_undegraded":"2024-02-18T00:33:29.314297+0000","last_fullsized":"2024-02-18T00:33:29.314297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:39.973040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:39.973040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"106'2","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.301657+0000","last_change":"2024-02-18T00:33:01.463387+0000","last_active":"2024-02-18T00:33:29.301657+0000","last_peered":"2024-02-18T00:33:29.301657+0000","last_clean":"2024-02-18T00:33:29.301657+0000","last_became_active":"2024-02-18T00:33:01.462544+0000","last_became_peered":"2024-02-18T00:33:01.462544+0000","last_unstale":"2024-02-18T00:33:29.301657+0000","last_undegraded":"2024-02-18T00:33:29.301657+0000","last_fullsized":"2024-02-18T00:33:29.301657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.14","version":"109'2","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.308132+0000","last_change":"2024-02-18T00:33:02.511910+0000","last_active":"2024-02-18T00:33:29.308132+0000","last_peered":"2024-02-18T00:33:29.308132+0000","last_clean":"2024-02-18T00:33:29.308132+0000","last_became_active":"2024-02-18T00:33:02.511292+0000","last_became_peered":"2024-02-18T00:33:02.511292+0000","last_unstale":"2024-02-18T00:33:29.308132+0000","last_undegraded":"2024-02-18T00:33:29.308132+0000","last_fullsized":"2024-02-18T00:33:29.308132+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.316831+0000","last_change":"2024-02-18T00:33:04.645450+0000","last_active":"2024-02-18T00:33:29.316831+0000","last_peered":"2024-02-18T00:33:29.316831+0000","last_clean":"2024-02-18T00:33:29.316831+0000","last_became_active":"2024-02-18T00:33:04.644990+0000","last_became_peered":"2024-02-18T00:33:04.644990+0000","last_unstale":"2024-02-18T00:33:29.316831+0000","last_undegraded":"2024-02-18T00:33:29.316831+0000","last_fullsized":"2024-02-18T00:33:29.316831+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":144,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.321126+0000","last_change":"2024-02-18T00:32:53.370947+0000","last_active":"2024-02-18T00:33:29.321126+0000","last_peered":"2024-02-18T00:33:29.321126+0000","last_clean":"2024-02-18T00:33:29.321126+0000","last_became_active":"2024-02-18T00:29:17.800098+0000","last_became_peered":"2024-02-18T00:29:17.800098+0000","last_unstale":"2024-02-18T00:33:29.321126+0000","last_undegraded":"2024-02-18T00:33:29.321126+0000","last_fullsized":"2024-02-18T00:33:29.321126+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:32:53.370869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.370869+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.370273+0000","last_change":"2024-02-18T00:32:04.461036+0000","last_active":"2024-02-18T00:33:29.370273+0000","last_peered":"2024-02-18T00:33:29.370273+0000","last_clean":"2024-02-18T00:33:29.370273+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:33:29.370273+0000","last_undegraded":"2024-02-18T00:33:29.370273+0000","last_fullsized":"2024-02-18T00:33:29.370273+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:04.460968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.460968+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.351893+0000","last_change":"2024-02-18T00:32:02.507109+0000","last_active":"2024-02-18T00:33:29.351893+0000","last_peered":"2024-02-18T00:33:29.351893+0000","last_clean":"2024-02-18T00:33:29.351893+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:33:29.351893+0000","last_undegraded":"2024-02-18T00:33:29.351893+0000","last_fullsized":"2024-02-18T00:33:29.351893+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:02.507010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.507010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314321+0000","last_change":"2024-02-18T00:32:38.665917+0000","last_active":"2024-02-18T00:33:29.314321+0000","last_peered":"2024-02-18T00:33:29.314321+0000","last_clean":"2024-02-18T00:33:29.314321+0000","last_became_active":"2024-02-18T00:29:21.101623+0000","last_became_peered":"2024-02-18T00:29:21.101623+0000","last_unstale":"2024-02-18T00:33:29.314321+0000","last_undegraded":"2024-02-18T00:33:29.314321+0000","last_fullsized":"2024-02-18T00:33:29.314321+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.665830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.665830+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.317549+0000","last_change":"2024-02-18T00:33:01.571369+0000","last_active":"2024-02-18T00:33:29.317549+0000","last_peered":"2024-02-18T00:33:29.317549+0000","last_clean":"2024-02-18T00:33:29.317549+0000","last_became_active":"2024-02-18T00:33:01.570706+0000","last_became_peered":"2024-02-18T00:33:01.570706+0000","last_unstale":"2024-02-18T00:33:29.317549+0000","last_undegraded":"2024-02-18T00:33:29.317549+0000","last_fullsized":"2024-02-18T00:33:29.317549+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"109'9","reported_seq":43,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.292952+0000","last_change":"2024-02-18T00:33:02.492423+0000","last_active":"2024-02-18T00:33:29.292952+0000","last_peered":"2024-02-18T00:33:29.292952+0000","last_clean":"2024-02-18T00:33:29.292952+0000","last_became_active":"2024-02-18T00:33:02.492133+0000","last_became_peered":"2024-02-18T00:33:02.492133+0000","last_unstale":"2024-02-18T00:33:29.292952+0000","last_undegraded":"2024-02-18T00:33:29.292952+0000","last_fullsized":"2024-02-18T00:33:29.292952+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299942+0000","last_change":"2024-02-18T00:33:02.523582+0000","last_active":"2024-02-18T00:33:29.299942+0000","last_peered":"2024-02-18T00:33:29.299942+0000","last_clean":"2024-02-18T00:33:29.299942+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:33:29.299942+0000","last_undegraded":"2024-02-18T00:33:29.299942+0000","last_fullsized":"2024-02-18T00:33:29.299942+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":125,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307838+0000","last_change":"2024-02-18T00:33:02.051762+0000","last_active":"2024-02-18T00:33:29.307838+0000","last_peered":"2024-02-18T00:33:29.307838+0000","last_clean":"2024-02-18T00:33:29.307838+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:33:29.307838+0000","last_undegraded":"2024-02-18T00:33:29.307838+0000","last_fullsized":"2024-02-18T00:33:29.307838+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:02.051679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.051679+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307559+0000","last_change":"2024-02-18T00:31:54.616292+0000","last_active":"2024-02-18T00:33:29.307559+0000","last_peered":"2024-02-18T00:33:29.307559+0000","last_clean":"2024-02-18T00:33:29.307559+0000","last_became_active":"2024-02-18T00:29:14.716661+0000","last_became_peered":"2024-02-18T00:29:14.716661+0000","last_unstale":"2024-02-18T00:33:29.307559+0000","last_undegraded":"2024-02-18T00:33:29.307559+0000","last_fullsized":"2024-02-18T00:33:29.307559+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:54.616213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.616213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328895+0000","last_change":"2024-02-18T00:32:45.751973+0000","last_active":"2024-02-18T00:33:29.328895+0000","last_peered":"2024-02-18T00:33:29.328895+0000","last_clean":"2024-02-18T00:33:29.328895+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:33:29.328895+0000","last_undegraded":"2024-02-18T00:33:29.328895+0000","last_fullsized":"2024-02-18T00:33:29.328895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:45.751874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.751874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.362855+0000","last_change":"2024-02-18T00:32:07.449328+0000","last_active":"2024-02-18T00:33:29.362855+0000","last_peered":"2024-02-18T00:33:29.362855+0000","last_clean":"2024-02-18T00:33:29.362855+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:33:29.362855+0000","last_undegraded":"2024-02-18T00:33:29.362855+0000","last_fullsized":"2024-02-18T00:33:29.362855+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.449249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.449249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315102+0000","last_change":"2024-02-18T00:33:01.559343+0000","last_active":"2024-02-18T00:33:29.315102+0000","last_peered":"2024-02-18T00:33:29.315102+0000","last_clean":"2024-02-18T00:33:29.315102+0000","last_became_active":"2024-02-18T00:33:01.559068+0000","last_became_peered":"2024-02-18T00:33:01.559068+0000","last_unstale":"2024-02-18T00:33:29.315102+0000","last_undegraded":"2024-02-18T00:33:29.315102+0000","last_fullsized":"2024-02-18T00:33:29.315102+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311189+0000","last_change":"2024-02-18T00:33:02.533251+0000","last_active":"2024-02-18T00:33:29.311189+0000","last_peered":"2024-02-18T00:33:29.311189+0000","last_clean":"2024-02-18T00:33:29.311189+0000","last_became_active":"2024-02-18T00:33:02.532428+0000","last_became_peered":"2024-02-18T00:33:02.532428+0000","last_unstale":"2024-02-18T00:33:29.311189+0000","last_undegraded":"2024-02-18T00:33:29.311189+0000","last_fullsized":"2024-02-18T00:33:29.311189+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314378+0000","last_change":"2024-02-18T00:33:04.608115+0000","last_active":"2024-02-18T00:33:29.314378+0000","last_peered":"2024-02-18T00:33:29.314378+0000","last_clean":"2024-02-18T00:33:29.314378+0000","last_became_active":"2024-02-18T00:33:04.607229+0000","last_became_peered":"2024-02-18T00:33:04.607229+0000","last_unstale":"2024-02-18T00:33:29.314378+0000","last_undegraded":"2024-02-18T00:33:29.314378+0000","last_fullsized":"2024-02-18T00:33:29.314378+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":107,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310587+0000","last_change":"2024-02-18T00:32:43.969645+0000","last_active":"2024-02-18T00:33:29.310587+0000","last_peered":"2024-02-18T00:33:29.310587+0000","last_clean":"2024-02-18T00:33:29.310587+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:33:29.310587+0000","last_undegraded":"2024-02-18T00:33:29.310587+0000","last_fullsized":"2024-02-18T00:33:29.310587+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:32:43.969566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.969566+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299319+0000","last_change":"2024-02-18T00:31:50.988509+0000","last_active":"2024-02-18T00:33:29.299319+0000","last_peered":"2024-02-18T00:33:29.299319+0000","last_clean":"2024-02-18T00:33:29.299319+0000","last_became_active":"2024-02-18T00:29:14.709682+0000","last_became_peered":"2024-02-18T00:29:14.709682+0000","last_unstale":"2024-02-18T00:33:29.299319+0000","last_undegraded":"2024-02-18T00:33:29.299319+0000","last_fullsized":"2024-02-18T00:33:29.299319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.988429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"64'8","reported_seq":152,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.338984+0000","last_change":"2024-02-18T00:32:46.413965+0000","last_active":"2024-02-18T00:33:29.338984+0000","last_peered":"2024-02-18T00:33:29.338984+0000","last_clean":"2024-02-18T00:33:29.338984+0000","last_became_active":"2024-02-18T00:29:19.096623+0000","last_became_peered":"2024-02-18T00:29:19.096623+0000","last_unstale":"2024-02-18T00:33:29.338984+0000","last_undegraded":"2024-02-18T00:33:29.338984+0000","last_fullsized":"2024-02-18T00:33:29.338984+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:46.413894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.413894+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296402+0000","last_change":"2024-02-18T00:31:58.064645+0000","last_active":"2024-02-18T00:33:29.296402+0000","last_peered":"2024-02-18T00:33:29.296402+0000","last_clean":"2024-02-18T00:33:29.296402+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:33:29.296402+0000","last_undegraded":"2024-02-18T00:33:29.296402+0000","last_fullsized":"2024-02-18T00:33:29.296402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.064559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.064559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"108'1","reported_seq":41,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.303559+0000","last_change":"2024-02-18T00:33:01.434399+0000","last_active":"2024-02-18T00:33:29.303559+0000","last_peered":"2024-02-18T00:33:29.303559+0000","last_clean":"2024-02-18T00:33:29.303559+0000","last_became_active":"2024-02-18T00:33:01.433012+0000","last_became_peered":"2024-02-18T00:33:01.433012+0000","last_unstale":"2024-02-18T00:33:29.303559+0000","last_undegraded":"2024-02-18T00:33:29.303559+0000","last_fullsized":"2024-02-18T00:33:29.303559+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165836,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":24577,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.11","version":"109'7","reported_seq":41,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306093+0000","last_change":"2024-02-18T00:33:04.669390+0000","last_active":"2024-02-18T00:33:29.306093+0000","last_peered":"2024-02-18T00:33:29.306093+0000","last_clean":"2024-02-18T00:33:29.306093+0000","last_became_active":"2024-02-18T00:33:04.669099+0000","last_became_peered":"2024-02-18T00:33:04.669099+0000","last_unstale":"2024-02-18T00:33:29.306093+0000","last_undegraded":"2024-02-18T00:33:29.306093+0000","last_fullsized":"2024-02-18T00:33:29.306093+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299622+0000","last_change":"2024-02-18T00:33:02.536636+0000","last_active":"2024-02-18T00:33:29.299622+0000","last_peered":"2024-02-18T00:33:29.299622+0000","last_clean":"2024-02-18T00:33:29.299622+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:33:29.299622+0000","last_undegraded":"2024-02-18T00:33:29.299622+0000","last_fullsized":"2024-02-18T00:33:29.299622+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":95,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311558+0000","last_change":"2024-02-18T00:31:50.844080+0000","last_active":"2024-02-18T00:33:29.311558+0000","last_peered":"2024-02-18T00:33:29.311558+0000","last_clean":"2024-02-18T00:33:29.311558+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:33:29.311558+0000","last_undegraded":"2024-02-18T00:33:29.311558+0000","last_fullsized":"2024-02-18T00:33:29.311558+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.843994+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.336626+0000","last_change":"2024-02-18T00:33:04.715878+0000","last_active":"2024-02-18T00:33:29.336626+0000","last_peered":"2024-02-18T00:33:29.336626+0000","last_clean":"2024-02-18T00:33:29.336626+0000","last_became_active":"2024-02-18T00:33:04.715539+0000","last_became_peered":"2024-02-18T00:33:04.715539+0000","last_unstale":"2024-02-18T00:33:29.336626+0000","last_undegraded":"2024-02-18T00:33:29.336626+0000","last_fullsized":"2024-02-18T00:33:29.336626+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.960661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.960661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":149,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296302+0000","last_change":"2024-02-18T00:32:49.092381+0000","last_active":"2024-02-18T00:33:29.296302+0000","last_peered":"2024-02-18T00:33:29.296302+0000","last_clean":"2024-02-18T00:33:29.296302+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:33:29.296302+0000","last_undegraded":"2024-02-18T00:33:29.296302+0000","last_fullsized":"2024-02-18T00:33:29.296302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:49.092293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.092293+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":83,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.313911+0000","last_change":"2024-02-18T00:31:23.987881+0000","last_active":"2024-02-18T00:33:29.313911+0000","last_peered":"2024-02-18T00:33:29.313911+0000","last_clean":"2024-02-18T00:33:29.313911+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:33:29.313911+0000","last_undegraded":"2024-02-18T00:33:29.313911+0000","last_fullsized":"2024-02-18T00:33:29.313911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.987791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1a","version":"106'3","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310669+0000","last_change":"2024-02-18T00:33:01.569847+0000","last_active":"2024-02-18T00:33:29.310669+0000","last_peered":"2024-02-18T00:33:29.310669+0000","last_clean":"2024-02-18T00:33:29.310669+0000","last_became_active":"2024-02-18T00:33:01.569569+0000","last_became_peered":"2024-02-18T00:33:01.569569+0000","last_unstale":"2024-02-18T00:33:29.310669+0000","last_undegraded":"2024-02-18T00:33:29.310669+0000","last_fullsized":"2024-02-18T00:33:29.310669+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.335400+0000","last_change":"2024-02-18T00:33:04.703417+0000","last_active":"2024-02-18T00:33:29.335400+0000","last_peered":"2024-02-18T00:33:29.335400+0000","last_clean":"2024-02-18T00:33:29.335400+0000","last_became_active":"2024-02-18T00:33:04.701206+0000","last_became_peered":"2024-02-18T00:33:04.701206+0000","last_unstale":"2024-02-18T00:33:29.335400+0000","last_undegraded":"2024-02-18T00:33:29.335400+0000","last_fullsized":"2024-02-18T00:33:29.335400+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.292393+0000","last_change":"2024-02-18T00:33:02.533944+0000","last_active":"2024-02-18T00:33:29.292393+0000","last_peered":"2024-02-18T00:33:29.292393+0000","last_clean":"2024-02-18T00:33:29.292393+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:33:29.292393+0000","last_undegraded":"2024-02-18T00:33:29.292393+0000","last_fullsized":"2024-02-18T00:33:29.292393+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296025+0000","last_change":"2024-02-18T00:31:55.104289+0000","last_active":"2024-02-18T00:33:29.296025+0000","last_peered":"2024-02-18T00:33:29.296025+0000","last_clean":"2024-02-18T00:33:29.296025+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:33:29.296025+0000","last_undegraded":"2024-02-18T00:33:29.296025+0000","last_fullsized":"2024-02-18T00:33:29.296025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:55.104195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.104195+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314720+0000","last_change":"2024-02-18T00:33:04.610117+0000","last_active":"2024-02-18T00:33:29.314720+0000","last_peered":"2024-02-18T00:33:29.314720+0000","last_clean":"2024-02-18T00:33:29.314720+0000","last_became_active":"2024-02-18T00:33:04.609915+0000","last_became_peered":"2024-02-18T00:33:04.609915+0000","last_unstale":"2024-02-18T00:33:29.314720+0000","last_undegraded":"2024-02-18T00:33:29.314720+0000","last_fullsized":"2024-02-18T00:33:29.314720+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:48.594640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:48.594640+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311926+0000","last_change":"2024-02-18T00:32:47.915174+0000","last_active":"2024-02-18T00:33:29.311926+0000","last_peered":"2024-02-18T00:33:29.311926+0000","last_clean":"2024-02-18T00:33:29.311926+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:33:29.311926+0000","last_undegraded":"2024-02-18T00:33:29.311926+0000","last_fullsized":"2024-02-18T00:33:29.311926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:32:47.915092+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.325453+0000","last_change":"2024-02-18T00:31:58.530850+0000","last_active":"2024-02-18T00:33:29.325453+0000","last_peered":"2024-02-18T00:33:29.325453+0000","last_clean":"2024-02-18T00:33:29.325453+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:33:29.325453+0000","last_undegraded":"2024-02-18T00:33:29.325453+0000","last_fullsized":"2024-02-18T00:33:29.325453+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.530768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.530768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.292316+0000","last_change":"2024-02-18T00:33:01.450143+0000","last_active":"2024-02-18T00:33:29.292316+0000","last_peered":"2024-02-18T00:33:29.292316+0000","last_clean":"2024-02-18T00:33:29.292316+0000","last_became_active":"2024-02-18T00:33:01.449972+0000","last_became_peered":"2024-02-18T00:33:01.449972+0000","last_unstale":"2024-02-18T00:33:29.292316+0000","last_undegraded":"2024-02-18T00:33:29.292316+0000","last_fullsized":"2024-02-18T00:33:29.292316+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309901+0000","last_change":"2024-02-18T00:33:02.511006+0000","last_active":"2024-02-18T00:33:29.309901+0000","last_peered":"2024-02-18T00:33:29.309901+0000","last_clean":"2024-02-18T00:33:29.309901+0000","last_became_active":"2024-02-18T00:33:02.510597+0000","last_became_peered":"2024-02-18T00:33:02.510597+0000","last_unstale":"2024-02-18T00:33:29.309901+0000","last_undegraded":"2024-02-18T00:33:29.309901+0000","last_fullsized":"2024-02-18T00:33:29.309901+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310835+0000","last_change":"2024-02-18T00:33:02.483392+0000","last_active":"2024-02-18T00:33:29.310835+0000","last_peered":"2024-02-18T00:33:29.310835+0000","last_clean":"2024-02-18T00:33:29.310835+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:33:29.310835+0000","last_undegraded":"2024-02-18T00:33:29.310835+0000","last_fullsized":"2024-02-18T00:33:29.310835+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":150,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312276+0000","last_change":"2024-02-18T00:31:39.294080+0000","last_active":"2024-02-18T00:33:29.312276+0000","last_peered":"2024-02-18T00:33:29.312276+0000","last_clean":"2024-02-18T00:33:29.312276+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:33:29.312276+0000","last_undegraded":"2024-02-18T00:33:29.312276+0000","last_fullsized":"2024-02-18T00:33:29.312276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:31:39.294006+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:31:39.294006+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":128,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299382+0000","last_change":"2024-02-18T00:31:56.063646+0000","last_active":"2024-02-18T00:33:29.299382+0000","last_peered":"2024-02-18T00:33:29.299382+0000","last_clean":"2024-02-18T00:33:29.299382+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:33:29.299382+0000","last_undegraded":"2024-02-18T00:33:29.299382+0000","last_fullsized":"2024-02-18T00:33:29.299382+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:31:56.063560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.063560+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309817+0000","last_change":"2024-02-18T00:33:04.695273+0000","last_active":"2024-02-18T00:33:29.309817+0000","last_peered":"2024-02-18T00:33:29.309817+0000","last_clean":"2024-02-18T00:33:29.309817+0000","last_became_active":"2024-02-18T00:33:04.694880+0000","last_became_peered":"2024-02-18T00:33:04.694880+0000","last_unstale":"2024-02-18T00:33:29.309817+0000","last_undegraded":"2024-02-18T00:33:29.309817+0000","last_fullsized":"2024-02-18T00:33:29.309817+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.198542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.198542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307935+0000","last_change":"2024-02-18T00:32:47.565531+0000","last_active":"2024-02-18T00:33:29.307935+0000","last_peered":"2024-02-18T00:33:29.307935+0000","last_clean":"2024-02-18T00:33:29.307935+0000","last_became_active":"2024-02-18T00:29:21.243619+0000","last_became_peered":"2024-02-18T00:29:21.243619+0000","last_unstale":"2024-02-18T00:33:29.307935+0000","last_undegraded":"2024-02-18T00:33:29.307935+0000","last_fullsized":"2024-02-18T00:33:29.307935+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:47.565450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:47.565450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.326104+0000","last_change":"2024-02-18T00:33:01.532614+0000","last_active":"2024-02-18T00:33:29.326104+0000","last_peered":"2024-02-18T00:33:29.326104+0000","last_clean":"2024-02-18T00:33:29.326104+0000","last_became_active":"2024-02-18T00:33:01.516952+0000","last_became_peered":"2024-02-18T00:33:01.516952+0000","last_unstale":"2024-02-18T00:33:29.326104+0000","last_undegraded":"2024-02-18T00:33:29.326104+0000","last_fullsized":"2024-02-18T00:33:29.326104+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.316519+0000","last_change":"2024-02-18T00:33:02.493878+0000","last_active":"2024-02-18T00:33:29.316519+0000","last_peered":"2024-02-18T00:33:29.316519+0000","last_clean":"2024-02-18T00:33:29.316519+0000","last_became_active":"2024-02-18T00:33:02.493597+0000","last_became_peered":"2024-02-18T00:33:02.493597+0000","last_unstale":"2024-02-18T00:33:29.316519+0000","last_undegraded":"2024-02-18T00:33:29.316519+0000","last_fullsized":"2024-02-18T00:33:29.316519+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312974+0000","last_change":"2024-02-18T00:33:02.477932+0000","last_active":"2024-02-18T00:33:29.312974+0000","last_peered":"2024-02-18T00:33:29.312974+0000","last_clean":"2024-02-18T00:33:29.312974+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:33:29.312974+0000","last_undegraded":"2024-02-18T00:33:29.312974+0000","last_fullsized":"2024-02-18T00:33:29.312974+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299759+0000","last_change":"2024-02-18T00:32:05.972387+0000","last_active":"2024-02-18T00:33:29.299759+0000","last_peered":"2024-02-18T00:33:29.299759+0000","last_clean":"2024-02-18T00:33:29.299759+0000","last_became_active":"2024-02-18T00:29:19.097813+0000","last_became_peered":"2024-02-18T00:29:19.097813+0000","last_unstale":"2024-02-18T00:33:29.299759+0000","last_undegraded":"2024-02-18T00:33:29.299759+0000","last_fullsized":"2024-02-18T00:33:29.299759+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.972303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.972303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":164,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312180+0000","last_change":"2024-02-18T00:32:05.656439+0000","last_active":"2024-02-18T00:33:29.312180+0000","last_peered":"2024-02-18T00:33:29.312180+0000","last_clean":"2024-02-18T00:33:29.312180+0000","last_became_active":"2024-02-18T00:29:17.801350+0000","last_became_peered":"2024-02-18T00:29:17.801350+0000","last_unstale":"2024-02-18T00:33:29.312180+0000","last_undegraded":"2024-02-18T00:33:29.312180+0000","last_fullsized":"2024-02-18T00:33:29.312180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.656344+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.301810+0000","last_change":"2024-02-18T00:33:04.715734+0000","last_active":"2024-02-18T00:33:29.301810+0000","last_peered":"2024-02-18T00:33:29.301810+0000","last_clean":"2024-02-18T00:33:29.301810+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:33:29.301810+0000","last_undegraded":"2024-02-18T00:33:29.301810+0000","last_fullsized":"2024-02-18T00:33:29.301810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310581+0000","last_change":"2024-02-18T00:32:42.033835+0000","last_active":"2024-02-18T00:33:29.310581+0000","last_peered":"2024-02-18T00:33:29.310581+0000","last_clean":"2024-02-18T00:33:29.310581+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:33:29.310581+0000","last_undegraded":"2024-02-18T00:33:29.310581+0000","last_fullsized":"2024-02-18T00:33:29.310581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:42.033759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:42.033759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.15","version":"108'4","reported_seq":40,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312891+0000","last_change":"2024-02-18T00:33:01.549700+0000","last_active":"2024-02-18T00:33:29.312891+0000","last_peered":"2024-02-18T00:33:29.312891+0000","last_clean":"2024-02-18T00:33:29.312891+0000","last_became_active":"2024-02-18T00:33:01.548851+0000","last_became_peered":"2024-02-18T00:33:01.548851+0000","last_unstale":"2024-02-18T00:33:29.312891+0000","last_undegraded":"2024-02-18T00:33:29.312891+0000","last_fullsized":"2024-02-18T00:33:29.312891+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"109'7","reported_seq":41,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.339136+0000","last_change":"2024-02-18T00:33:04.700942+0000","last_active":"2024-02-18T00:33:29.339136+0000","last_peered":"2024-02-18T00:33:29.339136+0000","last_clean":"2024-02-18T00:33:29.339136+0000","last_became_active":"2024-02-18T00:33:04.700673+0000","last_became_peered":"2024-02-18T00:33:04.700673+0000","last_unstale":"2024-02-18T00:33:29.339136+0000","last_undegraded":"2024-02-18T00:33:29.339136+0000","last_fullsized":"2024-02-18T00:33:29.339136+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.333305+0000","last_change":"2024-02-18T00:33:04.650781+0000","last_active":"2024-02-18T00:33:29.333305+0000","last_peered":"2024-02-18T00:33:29.333305+0000","last_clean":"2024-02-18T00:33:29.333305+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:33:29.333305+0000","last_undegraded":"2024-02-18T00:33:29.333305+0000","last_fullsized":"2024-02-18T00:33:29.333305+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.297613+0000","last_change":"2024-02-18T00:32:37.227614+0000","last_active":"2024-02-18T00:33:29.297613+0000","last_peered":"2024-02-18T00:33:29.297613+0000","last_clean":"2024-02-18T00:33:29.297613+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:33:29.297613+0000","last_undegraded":"2024-02-18T00:33:29.297613+0000","last_fullsized":"2024-02-18T00:33:29.297613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:37.227536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:37.227536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":142,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328126+0000","last_change":"2024-02-18T00:32:55.959110+0000","last_active":"2024-02-18T00:33:29.328126+0000","last_peered":"2024-02-18T00:33:29.328126+0000","last_clean":"2024-02-18T00:33:29.328126+0000","last_became_active":"2024-02-18T00:29:17.799838+0000","last_became_peered":"2024-02-18T00:29:17.799838+0000","last_unstale":"2024-02-18T00:33:29.328126+0000","last_undegraded":"2024-02-18T00:33:29.328126+0000","last_fullsized":"2024-02-18T00:33:29.328126+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:32:55.959041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.959041+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":110,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310923+0000","last_change":"2024-02-18T00:33:04.581054+0000","last_active":"2024-02-18T00:33:29.310923+0000","last_peered":"2024-02-18T00:33:29.310923+0000","last_clean":"2024-02-18T00:33:29.310923+0000","last_became_active":"2024-02-18T00:33:04.576123+0000","last_became_peered":"2024-02-18T00:33:04.576123+0000","last_unstale":"2024-02-18T00:33:29.310923+0000","last_undegraded":"2024-02-18T00:33:29.310923+0000","last_fullsized":"2024-02-18T00:33:29.310923+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:32:51.632894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:51.632894+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":436,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.336710+0000","last_change":"2024-02-18T00:32:44.751653+0000","last_active":"2024-02-18T00:33:29.336710+0000","last_peered":"2024-02-18T00:33:29.336710+0000","last_clean":"2024-02-18T00:33:29.336710+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:33:29.336710+0000","last_undegraded":"2024-02-18T00:33:29.336710+0000","last_fullsized":"2024-02-18T00:33:29.336710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:44.751537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.751537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.316208+0000","last_change":"2024-02-18T00:33:01.486028+0000","last_active":"2024-02-18T00:33:29.316208+0000","last_peered":"2024-02-18T00:33:29.316208+0000","last_clean":"2024-02-18T00:33:29.316208+0000","last_became_active":"2024-02-18T00:33:01.485748+0000","last_became_peered":"2024-02-18T00:33:01.485748+0000","last_unstale":"2024-02-18T00:33:29.316208+0000","last_undegraded":"2024-02-18T00:33:29.316208+0000","last_fullsized":"2024-02-18T00:33:29.316208+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"109'2","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.291986+0000","last_change":"2024-02-18T00:33:02.534383+0000","last_active":"2024-02-18T00:33:29.291986+0000","last_peered":"2024-02-18T00:33:29.291986+0000","last_clean":"2024-02-18T00:33:29.291986+0000","last_became_active":"2024-02-18T00:33:02.533911+0000","last_became_peered":"2024-02-18T00:33:02.533911+0000","last_unstale":"2024-02-18T00:33:29.291986+0000","last_undegraded":"2024-02-18T00:33:29.291986+0000","last_fullsized":"2024-02-18T00:33:29.291986+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.323762+0000","last_change":"2024-02-18T00:33:04.587776+0000","last_active":"2024-02-18T00:33:29.323762+0000","last_peered":"2024-02-18T00:33:29.323762+0000","last_clean":"2024-02-18T00:33:29.323762+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:33:29.323762+0000","last_undegraded":"2024-02-18T00:33:29.323762+0000","last_fullsized":"2024-02-18T00:33:29.323762+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312072+0000","last_change":"2024-02-18T00:32:43.913604+0000","last_active":"2024-02-18T00:33:29.312072+0000","last_peered":"2024-02-18T00:33:29.312072+0000","last_clean":"2024-02-18T00:33:29.312072+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:33:29.312072+0000","last_undegraded":"2024-02-18T00:33:29.312072+0000","last_fullsized":"2024-02-18T00:33:29.312072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.913522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.913522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":109,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.337040+0000","last_change":"2024-02-18T00:32:45.002522+0000","last_active":"2024-02-18T00:33:29.337040+0000","last_peered":"2024-02-18T00:33:29.337040+0000","last_clean":"2024-02-18T00:33:29.337040+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:33:29.337040+0000","last_undegraded":"2024-02-18T00:33:29.337040+0000","last_fullsized":"2024-02-18T00:33:29.337040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:32:45.002406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.002406+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314396+0000","last_change":"2024-02-18T00:33:04.610118+0000","last_active":"2024-02-18T00:33:29.314396+0000","last_peered":"2024-02-18T00:33:29.314396+0000","last_clean":"2024-02-18T00:33:29.314396+0000","last_became_active":"2024-02-18T00:33:04.609908+0000","last_became_peered":"2024-02-18T00:33:04.609908+0000","last_unstale":"2024-02-18T00:33:29.314396+0000","last_undegraded":"2024-02-18T00:33:29.314396+0000","last_fullsized":"2024-02-18T00:33:29.314396+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.754417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.754417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":92,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311540+0000","last_change":"2024-02-18T00:31:53.976449+0000","last_active":"2024-02-18T00:33:29.311540+0000","last_peered":"2024-02-18T00:33:29.311540+0000","last_clean":"2024-02-18T00:33:29.311540+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:33:29.311540+0000","last_undegraded":"2024-02-18T00:33:29.311540+0000","last_fullsized":"2024-02-18T00:33:29.311540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:53.976377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:31:53.976377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328983+0000","last_change":"2024-02-18T00:32:53.813004+0000","last_active":"2024-02-18T00:33:29.328983+0000","last_peered":"2024-02-18T00:33:29.328983+0000","last_clean":"2024-02-18T00:33:29.328983+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:33:29.328983+0000","last_undegraded":"2024-02-18T00:33:29.328983+0000","last_fullsized":"2024-02-18T00:33:29.328983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.812897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.812897+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.323688+0000","last_change":"2024-02-18T00:33:01.585846+0000","last_active":"2024-02-18T00:33:29.323688+0000","last_peered":"2024-02-18T00:33:29.323688+0000","last_clean":"2024-02-18T00:33:29.323688+0000","last_became_active":"2024-02-18T00:33:01.584536+0000","last_became_peered":"2024-02-18T00:33:01.584536+0000","last_unstale":"2024-02-18T00:33:29.323688+0000","last_undegraded":"2024-02-18T00:33:29.323688+0000","last_fullsized":"2024-02-18T00:33:29.323688+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1b","version":"109'10","reported_seq":44,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.301449+0000","last_change":"2024-02-18T00:33:02.509211+0000","last_active":"2024-02-18T00:33:29.301449+0000","last_peered":"2024-02-18T00:33:29.301449+0000","last_clean":"2024-02-18T00:33:29.301449+0000","last_became_active":"2024-02-18T00:33:02.508134+0000","last_became_peered":"2024-02-18T00:33:02.508134+0000","last_unstale":"2024-02-18T00:33:29.301449+0000","last_undegraded":"2024-02-18T00:33:29.301449+0000","last_fullsized":"2024-02-18T00:33:29.301449+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314702+0000","last_change":"2024-02-18T00:33:04.592626+0000","last_active":"2024-02-18T00:33:29.314702+0000","last_peered":"2024-02-18T00:33:29.314702+0000","last_clean":"2024-02-18T00:33:29.314702+0000","last_became_active":"2024-02-18T00:33:04.591598+0000","last_became_peered":"2024-02-18T00:33:04.591598+0000","last_unstale":"2024-02-18T00:33:29.314702+0000","last_undegraded":"2024-02-18T00:33:29.314702+0000","last_fullsized":"2024-02-18T00:33:29.314702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307846+0000","last_change":"2024-02-18T00:32:43.644999+0000","last_active":"2024-02-18T00:33:29.307846+0000","last_peered":"2024-02-18T00:33:29.307846+0000","last_clean":"2024-02-18T00:33:29.307846+0000","last_became_active":"2024-02-18T00:29:19.097300+0000","last_became_peered":"2024-02-18T00:29:19.097300+0000","last_unstale":"2024-02-18T00:33:29.307846+0000","last_undegraded":"2024-02-18T00:33:29.307846+0000","last_fullsized":"2024-02-18T00:33:29.307846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.644921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":119,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320048+0000","last_change":"2024-02-18T00:32:44.353654+0000","last_active":"2024-02-18T00:33:29.320048+0000","last_peered":"2024-02-18T00:33:29.320048+0000","last_clean":"2024-02-18T00:33:29.320048+0000","last_became_active":"2024-02-18T00:29:17.800136+0000","last_became_peered":"2024-02-18T00:29:17.800136+0000","last_unstale":"2024-02-18T00:33:29.320048+0000","last_undegraded":"2024-02-18T00:33:29.320048+0000","last_fullsized":"2024-02-18T00:33:29.320048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:32:44.353571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.353571+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314526+0000","last_change":"2024-02-18T00:32:54.632979+0000","last_active":"2024-02-18T00:33:29.314526+0000","last_peered":"2024-02-18T00:33:29.314526+0000","last_clean":"2024-02-18T00:33:29.314526+0000","last_became_active":"2024-02-18T00:29:08.534553+0000","last_became_peered":"2024-02-18T00:29:08.534553+0000","last_unstale":"2024-02-18T00:33:29.314526+0000","last_undegraded":"2024-02-18T00:33:29.314526+0000","last_fullsized":"2024-02-18T00:33:29.314526+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.632911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.632911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.317299+0000","last_change":"2024-02-18T00:33:04.697173+0000","last_active":"2024-02-18T00:33:29.317299+0000","last_peered":"2024-02-18T00:33:29.317299+0000","last_clean":"2024-02-18T00:33:29.317299+0000","last_became_active":"2024-02-18T00:33:04.696891+0000","last_became_peered":"2024-02-18T00:33:04.696891+0000","last_unstale":"2024-02-18T00:33:29.317299+0000","last_undegraded":"2024-02-18T00:33:29.317299+0000","last_fullsized":"2024-02-18T00:33:29.317299+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":93,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315070+0000","last_change":"2024-02-18T00:33:02.549194+0000","last_active":"2024-02-18T00:33:29.315070+0000","last_peered":"2024-02-18T00:33:29.315070+0000","last_clean":"2024-02-18T00:33:29.315070+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:33:29.315070+0000","last_undegraded":"2024-02-18T00:33:29.315070+0000","last_fullsized":"2024-02-18T00:33:29.315070+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:02.549081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.549081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.319370+0000","last_change":"2024-02-18T00:31:46.851657+0000","last_active":"2024-02-18T00:33:29.319370+0000","last_peered":"2024-02-18T00:33:29.319370+0000","last_clean":"2024-02-18T00:33:29.319370+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:33:29.319370+0000","last_undegraded":"2024-02-18T00:33:29.319370+0000","last_fullsized":"2024-02-18T00:33:29.319370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.851573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"108'3","reported_seq":41,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311939+0000","last_change":"2024-02-18T00:33:01.534477+0000","last_active":"2024-02-18T00:33:29.311939+0000","last_peered":"2024-02-18T00:33:29.311939+0000","last_clean":"2024-02-18T00:33:29.311939+0000","last_became_active":"2024-02-18T00:33:01.514058+0000","last_became_peered":"2024-02-18T00:33:01.514058+0000","last_unstale":"2024-02-18T00:33:29.311939+0000","last_undegraded":"2024-02-18T00:33:29.311939+0000","last_fullsized":"2024-02-18T00:33:29.311939+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.336873+0000","last_change":"2024-02-18T00:33:02.511940+0000","last_active":"2024-02-18T00:33:29.336873+0000","last_peered":"2024-02-18T00:33:29.336873+0000","last_clean":"2024-02-18T00:33:29.336873+0000","last_became_active":"2024-02-18T00:33:02.511501+0000","last_became_peered":"2024-02-18T00:33:02.511501+0000","last_unstale":"2024-02-18T00:33:29.336873+0000","last_undegraded":"2024-02-18T00:33:29.336873+0000","last_fullsized":"2024-02-18T00:33:29.336873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320526+0000","last_change":"2024-02-18T00:33:04.649428+0000","last_active":"2024-02-18T00:33:29.320526+0000","last_peered":"2024-02-18T00:33:29.320526+0000","last_clean":"2024-02-18T00:33:29.320526+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:33:29.320526+0000","last_undegraded":"2024-02-18T00:33:29.320526+0000","last_fullsized":"2024-02-18T00:33:29.320526+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.360209+0000","last_change":"2024-02-18T00:31:47.502734+0000","last_active":"2024-02-18T00:33:29.360209+0000","last_peered":"2024-02-18T00:33:29.360209+0000","last_clean":"2024-02-18T00:33:29.360209+0000","last_became_active":"2024-02-18T00:29:19.096898+0000","last_became_peered":"2024-02-18T00:29:19.096898+0000","last_unstale":"2024-02-18T00:33:29.360209+0000","last_undegraded":"2024-02-18T00:33:29.360209+0000","last_fullsized":"2024-02-18T00:33:29.360209+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:47.502660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:47.502660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":134,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305680+0000","last_change":"2024-02-18T00:31:55.005638+0000","last_active":"2024-02-18T00:33:29.305680+0000","last_peered":"2024-02-18T00:33:29.305680+0000","last_clean":"2024-02-18T00:33:29.305680+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:33:29.305680+0000","last_undegraded":"2024-02-18T00:33:29.305680+0000","last_fullsized":"2024-02-18T00:33:29.305680+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:31:55.005563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.005563+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.355732+0000","last_change":"2024-02-18T00:33:04.665195+0000","last_active":"2024-02-18T00:33:29.355732+0000","last_peered":"2024-02-18T00:33:29.355732+0000","last_clean":"2024-02-18T00:33:29.355732+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:33:29.355732+0000","last_undegraded":"2024-02-18T00:33:29.355732+0000","last_fullsized":"2024-02-18T00:33:29.355732+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.521630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.521630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":344,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:30.232570+0000","last_change":"2024-02-18T00:31:58.945520+0000","last_active":"2024-02-18T00:33:30.232570+0000","last_peered":"2024-02-18T00:33:30.232570+0000","last_clean":"2024-02-18T00:33:30.232570+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:33:30.232570+0000","last_undegraded":"2024-02-18T00:33:30.232570+0000","last_fullsized":"2024-02-18T00:33:30.232570+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:31:58.945452+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.945452+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":247,"num_read_kb":242,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310792+0000","last_change":"2024-02-18T00:32:55.588834+0000","last_active":"2024-02-18T00:33:29.310792+0000","last_peered":"2024-02-18T00:33:29.310792+0000","last_clean":"2024-02-18T00:33:29.310792+0000","last_became_active":"2024-02-18T00:29:21.243902+0000","last_became_peered":"2024-02-18T00:29:21.243902+0000","last_unstale":"2024-02-18T00:33:29.310792+0000","last_undegraded":"2024-02-18T00:33:29.310792+0000","last_fullsized":"2024-02-18T00:33:29.310792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.588750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315412+0000","last_change":"2024-02-18T00:33:01.553808+0000","last_active":"2024-02-18T00:33:29.315412+0000","last_peered":"2024-02-18T00:33:29.315412+0000","last_clean":"2024-02-18T00:33:29.315412+0000","last_became_active":"2024-02-18T00:33:01.551635+0000","last_became_peered":"2024-02-18T00:33:01.551635+0000","last_unstale":"2024-02-18T00:33:29.315412+0000","last_undegraded":"2024-02-18T00:33:29.315412+0000","last_fullsized":"2024-02-18T00:33:29.315412+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299084+0000","last_change":"2024-02-18T00:33:02.487192+0000","last_active":"2024-02-18T00:33:29.299084+0000","last_peered":"2024-02-18T00:33:29.299084+0000","last_clean":"2024-02-18T00:33:29.299084+0000","last_became_active":"2024-02-18T00:33:02.486911+0000","last_became_peered":"2024-02-18T00:33:02.486911+0000","last_unstale":"2024-02-18T00:33:29.299084+0000","last_undegraded":"2024-02-18T00:33:29.299084+0000","last_fullsized":"2024-02-18T00:33:29.299084+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299832+0000","last_change":"2024-02-18T00:33:02.499565+0000","last_active":"2024-02-18T00:33:29.299832+0000","last_peered":"2024-02-18T00:33:29.299832+0000","last_clean":"2024-02-18T00:33:29.299832+0000","last_became_active":"2024-02-18T00:33:02.499219+0000","last_became_peered":"2024-02-18T00:33:02.499219+0000","last_unstale":"2024-02-18T00:33:29.299832+0000","last_undegraded":"2024-02-18T00:33:29.299832+0000","last_fullsized":"2024-02-18T00:33:29.299832+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":89,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320769+0000","last_change":"2024-02-18T00:32:07.008657+0000","last_active":"2024-02-18T00:33:29.320769+0000","last_peered":"2024-02-18T00:33:29.320769+0000","last_clean":"2024-02-18T00:33:29.320769+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:33:29.320769+0000","last_undegraded":"2024-02-18T00:33:29.320769+0000","last_fullsized":"2024-02-18T00:33:29.320769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.008543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.008543+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":206,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328237+0000","last_change":"2024-02-18T00:32:04.954445+0000","last_active":"2024-02-18T00:33:29.328237+0000","last_peered":"2024-02-18T00:33:29.328237+0000","last_clean":"2024-02-18T00:33:29.328237+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:33:29.328237+0000","last_undegraded":"2024-02-18T00:33:29.328237+0000","last_fullsized":"2024-02-18T00:33:29.328237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:32:04.954359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:04.954359+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.318037+0000","last_change":"2024-02-18T00:33:04.569398+0000","last_active":"2024-02-18T00:33:29.318037+0000","last_peered":"2024-02-18T00:33:29.318037+0000","last_clean":"2024-02-18T00:33:29.318037+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:33:29.318037+0000","last_undegraded":"2024-02-18T00:33:29.318037+0000","last_fullsized":"2024-02-18T00:33:29.318037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.991759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.991759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":83,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.332834+0000","last_change":"2024-02-18T00:31:07.066613+0000","last_active":"2024-02-18T00:33:29.332834+0000","last_peered":"2024-02-18T00:33:29.332834+0000","last_clean":"2024-02-18T00:33:29.332834+0000","last_became_active":"2024-02-18T00:29:21.095514+0000","last_became_peered":"2024-02-18T00:29:21.095514+0000","last_unstale":"2024-02-18T00:33:29.332834+0000","last_undegraded":"2024-02-18T00:33:29.332834+0000","last_fullsized":"2024-02-18T00:33:29.332834+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:07.066540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:07.066540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.11","version":"108'3","reported_seq":47,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306747+0000","last_change":"2024-02-18T00:33:01.502827+0000","last_active":"2024-02-18T00:33:29.306747+0000","last_peered":"2024-02-18T00:33:29.306747+0000","last_clean":"2024-02-18T00:33:29.306747+0000","last_became_active":"2024-02-18T00:33:01.502464+0000","last_became_peered":"2024-02-18T00:33:01.502464+0000","last_unstale":"2024-02-18T00:33:29.306747+0000","last_undegraded":"2024-02-18T00:33:29.306747+0000","last_fullsized":"2024-02-18T00:33:29.306747+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165843,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307124+0000","last_change":"2024-02-18T00:33:02.522197+0000","last_active":"2024-02-18T00:33:29.307124+0000","last_peered":"2024-02-18T00:33:29.307124+0000","last_clean":"2024-02-18T00:33:29.307124+0000","last_became_active":"2024-02-18T00:33:02.521934+0000","last_became_peered":"2024-02-18T00:33:02.521934+0000","last_unstale":"2024-02-18T00:33:29.307124+0000","last_undegraded":"2024-02-18T00:33:29.307124+0000","last_fullsized":"2024-02-18T00:33:29.307124+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.319049+0000","last_change":"2024-02-18T00:33:04.719631+0000","last_active":"2024-02-18T00:33:29.319049+0000","last_peered":"2024-02-18T00:33:29.319049+0000","last_clean":"2024-02-18T00:33:29.319049+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:33:29.319049+0000","last_undegraded":"2024-02-18T00:33:29.319049+0000","last_fullsized":"2024-02-18T00:33:29.319049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":148,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328429+0000","last_change":"2024-02-18T00:32:05.026757+0000","last_active":"2024-02-18T00:33:29.328429+0000","last_peered":"2024-02-18T00:33:29.328429+0000","last_clean":"2024-02-18T00:33:29.328429+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:33:29.328429+0000","last_undegraded":"2024-02-18T00:33:29.328429+0000","last_fullsized":"2024-02-18T00:33:29.328429+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:05.026652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.026652+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":92,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.350786+0000","last_change":"2024-02-18T00:32:49.398933+0000","last_active":"2024-02-18T00:33:29.350786+0000","last_peered":"2024-02-18T00:33:29.350786+0000","last_clean":"2024-02-18T00:33:29.350786+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:33:29.350786+0000","last_undegraded":"2024-02-18T00:33:29.350786+0000","last_fullsized":"2024-02-18T00:33:29.350786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.398867+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":108,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328753+0000","last_change":"2024-02-18T00:33:04.676887+0000","last_active":"2024-02-18T00:33:29.328753+0000","last_peered":"2024-02-18T00:33:29.328753+0000","last_clean":"2024-02-18T00:33:29.328753+0000","last_became_active":"2024-02-18T00:33:04.676371+0000","last_became_peered":"2024-02-18T00:33:04.676371+0000","last_unstale":"2024-02-18T00:33:29.328753+0000","last_undegraded":"2024-02-18T00:33:29.328753+0000","last_fullsized":"2024-02-18T00:33:29.328753+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.839018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.839018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":82,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314054+0000","last_change":"2024-02-18T00:31:20.132994+0000","last_active":"2024-02-18T00:33:29.314054+0000","last_peered":"2024-02-18T00:33:29.314054+0000","last_clean":"2024-02-18T00:33:29.314054+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:33:29.314054+0000","last_undegraded":"2024-02-18T00:33:29.314054+0000","last_fullsized":"2024-02-18T00:33:29.314054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:20.132882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:20.132882+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.317406+0000","last_change":"2024-02-18T00:33:01.507106+0000","last_active":"2024-02-18T00:33:29.317406+0000","last_peered":"2024-02-18T00:33:29.317406+0000","last_clean":"2024-02-18T00:33:29.317406+0000","last_became_active":"2024-02-18T00:33:01.506457+0000","last_became_peered":"2024-02-18T00:33:01.506457+0000","last_unstale":"2024-02-18T00:33:29.317406+0000","last_undegraded":"2024-02-18T00:33:29.317406+0000","last_fullsized":"2024-02-18T00:33:29.317406+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305445+0000","last_change":"2024-02-18T00:33:02.484120+0000","last_active":"2024-02-18T00:33:29.305445+0000","last_peered":"2024-02-18T00:33:29.305445+0000","last_clean":"2024-02-18T00:33:29.305445+0000","last_became_active":"2024-02-18T00:33:02.483807+0000","last_became_peered":"2024-02-18T00:33:02.483807+0000","last_unstale":"2024-02-18T00:33:29.305445+0000","last_undegraded":"2024-02-18T00:33:29.305445+0000","last_fullsized":"2024-02-18T00:33:29.305445+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310794+0000","last_change":"2024-02-18T00:33:02.483549+0000","last_active":"2024-02-18T00:33:29.310794+0000","last_peered":"2024-02-18T00:33:29.310794+0000","last_clean":"2024-02-18T00:33:29.310794+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:33:29.310794+0000","last_undegraded":"2024-02-18T00:33:29.310794+0000","last_fullsized":"2024-02-18T00:33:29.310794+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.334623+0000","last_change":"2024-02-18T00:32:40.476946+0000","last_active":"2024-02-18T00:33:29.334623+0000","last_peered":"2024-02-18T00:33:29.334623+0000","last_clean":"2024-02-18T00:33:29.334623+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:33:29.334623+0000","last_undegraded":"2024-02-18T00:33:29.334623+0000","last_fullsized":"2024-02-18T00:33:29.334623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:40.476846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.476846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":145,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299466+0000","last_change":"2024-02-18T00:32:43.018921+0000","last_active":"2024-02-18T00:33:29.299466+0000","last_peered":"2024-02-18T00:33:29.299466+0000","last_clean":"2024-02-18T00:33:29.299466+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:33:29.299466+0000","last_undegraded":"2024-02-18T00:33:29.299466+0000","last_fullsized":"2024-02-18T00:33:29.299466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:32:43.018841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.018841+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":112,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.300849+0000","last_change":"2024-02-18T00:33:03.047192+0000","last_active":"2024-02-18T00:33:29.300849+0000","last_peered":"2024-02-18T00:33:29.300849+0000","last_clean":"2024-02-18T00:33:29.300849+0000","last_became_active":"2024-02-18T00:29:14.715981+0000","last_became_peered":"2024-02-18T00:29:14.715981+0000","last_unstale":"2024-02-18T00:33:29.300849+0000","last_undegraded":"2024-02-18T00:33:29.300849+0000","last_fullsized":"2024-02-18T00:33:29.300849+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:33:03.047128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.047128+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.323457+0000","last_change":"2024-02-18T00:31:52.444880+0000","last_active":"2024-02-18T00:33:29.323457+0000","last_peered":"2024-02-18T00:33:29.323457+0000","last_clean":"2024-02-18T00:33:29.323457+0000","last_became_active":"2024-02-18T00:29:21.104458+0000","last_became_peered":"2024-02-18T00:29:21.104458+0000","last_unstale":"2024-02-18T00:33:29.323457+0000","last_undegraded":"2024-02-18T00:33:29.323457+0000","last_fullsized":"2024-02-18T00:33:29.323457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:52.444811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:52.444811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1c","version":"108'4","reported_seq":40,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328145+0000","last_change":"2024-02-18T00:33:01.489945+0000","last_active":"2024-02-18T00:33:29.328145+0000","last_peered":"2024-02-18T00:33:29.328145+0000","last_clean":"2024-02-18T00:33:29.328145+0000","last_became_active":"2024-02-18T00:33:01.488576+0000","last_became_peered":"2024-02-18T00:33:01.488576+0000","last_unstale":"2024-02-18T00:33:29.328145+0000","last_undegraded":"2024-02-18T00:33:29.328145+0000","last_fullsized":"2024-02-18T00:33:29.328145+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165835,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"109'2","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.332312+0000","last_change":"2024-02-18T00:33:02.493422+0000","last_active":"2024-02-18T00:33:29.332312+0000","last_peered":"2024-02-18T00:33:29.332312+0000","last_clean":"2024-02-18T00:33:29.332312+0000","last_became_active":"2024-02-18T00:33:02.492817+0000","last_became_peered":"2024-02-18T00:33:02.492817+0000","last_unstale":"2024-02-18T00:33:29.332312+0000","last_undegraded":"2024-02-18T00:33:29.332312+0000","last_fullsized":"2024-02-18T00:33:29.332312+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.292562+0000","last_change":"2024-02-18T00:33:02.533610+0000","last_active":"2024-02-18T00:33:29.292562+0000","last_peered":"2024-02-18T00:33:29.292562+0000","last_clean":"2024-02-18T00:33:29.292562+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:33:29.292562+0000","last_undegraded":"2024-02-18T00:33:29.292562+0000","last_fullsized":"2024-02-18T00:33:29.292562+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":137,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.337143+0000","last_change":"2024-02-18T00:33:03.042961+0000","last_active":"2024-02-18T00:33:29.337143+0000","last_peered":"2024-02-18T00:33:29.337143+0000","last_clean":"2024-02-18T00:33:29.337143+0000","last_became_active":"2024-02-18T00:29:17.799833+0000","last_became_peered":"2024-02-18T00:29:17.799833+0000","last_unstale":"2024-02-18T00:33:29.337143+0000","last_undegraded":"2024-02-18T00:33:29.337143+0000","last_fullsized":"2024-02-18T00:33:29.337143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:33:03.042870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.042870+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.334546+0000","last_change":"2024-02-18T00:32:05.449395+0000","last_active":"2024-02-18T00:33:29.334546+0000","last_peered":"2024-02-18T00:33:29.334546+0000","last_clean":"2024-02-18T00:33:29.334546+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:33:29.334546+0000","last_undegraded":"2024-02-18T00:33:29.334546+0000","last_fullsized":"2024-02-18T00:33:29.334546+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.449286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.449286+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.316618+0000","last_change":"2024-02-18T00:31:59.580602+0000","last_active":"2024-02-18T00:33:29.316618+0000","last_peered":"2024-02-18T00:33:29.316618+0000","last_clean":"2024-02-18T00:33:29.316618+0000","last_became_active":"2024-02-18T00:29:19.100136+0000","last_became_peered":"2024-02-18T00:29:19.100136+0000","last_unstale":"2024-02-18T00:33:29.316618+0000","last_undegraded":"2024-02-18T00:33:29.316618+0000","last_fullsized":"2024-02-18T00:33:29.316618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:59.580519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:59.580519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.369957+0000","last_change":"2024-02-18T00:32:42.462891+0000","last_active":"2024-02-18T00:33:29.369957+0000","last_peered":"2024-02-18T00:33:29.369957+0000","last_clean":"2024-02-18T00:33:29.369957+0000","last_became_active":"2024-02-18T00:29:21.094616+0000","last_became_peered":"2024-02-18T00:29:21.094616+0000","last_unstale":"2024-02-18T00:33:29.369957+0000","last_undegraded":"2024-02-18T00:33:29.369957+0000","last_fullsized":"2024-02-18T00:33:29.369957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:42.462777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:32:42.462777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3","version":"108'1","reported_seq":42,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.292478+0000","last_change":"2024-02-18T00:33:06.020549+0000","last_active":"2024-02-18T00:33:29.292478+0000","last_peered":"2024-02-18T00:33:29.292478+0000","last_clean":"2024-02-18T00:33:29.292478+0000","last_became_active":"2024-02-18T00:33:01.438240+0000","last_became_peered":"2024-02-18T00:33:01.438240+0000","last_unstale":"2024-02-18T00:33:29.292478+0000","last_undegraded":"2024-02-18T00:33:29.292478+0000","last_fullsized":"2024-02-18T00:33:29.292478+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'1","last_scrub_stamp":"2024-02-18T00:33:06.020448+0000","last_deep_scrub":"108'1","last_deep_scrub_stamp":"2024-02-18T00:33:06.020448+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.020448+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314457+0000","last_change":"2024-02-18T00:33:04.608305+0000","last_active":"2024-02-18T00:33:29.314457+0000","last_peered":"2024-02-18T00:33:29.314457+0000","last_clean":"2024-02-18T00:33:29.314457+0000","last_became_active":"2024-02-18T00:33:04.607683+0000","last_became_peered":"2024-02-18T00:33:04.607683+0000","last_unstale":"2024-02-18T00:33:29.314457+0000","last_undegraded":"2024-02-18T00:33:29.314457+0000","last_fullsized":"2024-02-18T00:33:29.314457+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310997+0000","last_change":"2024-02-18T00:33:04.706011+0000","last_active":"2024-02-18T00:33:29.310997+0000","last_peered":"2024-02-18T00:33:29.310997+0000","last_clean":"2024-02-18T00:33:29.310997+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:33:29.310997+0000","last_undegraded":"2024-02-18T00:33:29.310997+0000","last_fullsized":"2024-02-18T00:33:29.310997+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":135,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312735+0000","last_change":"2024-02-18T00:31:29.228915+0000","last_active":"2024-02-18T00:33:29.312735+0000","last_peered":"2024-02-18T00:33:29.312735+0000","last_clean":"2024-02-18T00:33:29.312735+0000","last_became_active":"2024-02-18T00:30:41.642511+0000","last_became_peered":"2024-02-18T00:30:41.642511+0000","last_unstale":"2024-02-18T00:33:29.312735+0000","last_undegraded":"2024-02-18T00:33:29.312735+0000","last_fullsized":"2024-02-18T00:33:29.312735+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:31:29.228799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:29.228799+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305804+0000","last_change":"2024-02-18T00:32:58.392505+0000","last_active":"2024-02-18T00:33:29.305804+0000","last_peered":"2024-02-18T00:33:29.305804+0000","last_clean":"2024-02-18T00:33:29.305804+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:33:29.305804+0000","last_undegraded":"2024-02-18T00:33:29.305804+0000","last_fullsized":"2024-02-18T00:33:29.305804+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.392443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.316597+0000","last_change":"2024-02-18T00:32:50.948349+0000","last_active":"2024-02-18T00:33:29.316597+0000","last_peered":"2024-02-18T00:33:29.316597+0000","last_clean":"2024-02-18T00:33:29.316597+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:33:29.316597+0000","last_undegraded":"2024-02-18T00:33:29.316597+0000","last_fullsized":"2024-02-18T00:33:29.316597+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:50.948240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.948240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310643+0000","last_change":"2024-02-18T00:32:06.740021+0000","last_active":"2024-02-18T00:33:29.310643+0000","last_peered":"2024-02-18T00:33:29.310643+0000","last_clean":"2024-02-18T00:33:29.310643+0000","last_became_active":"2024-02-18T00:29:21.244262+0000","last_became_peered":"2024-02-18T00:29:21.244262+0000","last_unstale":"2024-02-18T00:33:29.310643+0000","last_undegraded":"2024-02-18T00:33:29.310643+0000","last_fullsized":"2024-02-18T00:33:29.310643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:06.739943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:06.739943+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"108'8","reported_seq":50,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309657+0000","last_change":"2024-02-18T00:33:04.987932+0000","last_active":"2024-02-18T00:33:29.309657+0000","last_peered":"2024-02-18T00:33:29.309657+0000","last_clean":"2024-02-18T00:33:29.309657+0000","last_became_active":"2024-02-18T00:33:01.499199+0000","last_became_peered":"2024-02-18T00:33:01.499199+0000","last_unstale":"2024-02-18T00:33:29.309657+0000","last_undegraded":"2024-02-18T00:33:29.309657+0000","last_fullsized":"2024-02-18T00:33:29.309657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'8","last_scrub_stamp":"2024-02-18T00:33:04.987867+0000","last_deep_scrub":"108'8","last_deep_scrub_stamp":"2024-02-18T00:33:04.987867+0000","last_clean_scrub_stamp":"2024-02-18T00:33:04.987867+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3888,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.a","version":"109'10","reported_seq":44,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311260+0000","last_change":"2024-02-18T00:33:04.707926+0000","last_active":"2024-02-18T00:33:29.311260+0000","last_peered":"2024-02-18T00:33:29.311260+0000","last_clean":"2024-02-18T00:33:29.311260+0000","last_became_active":"2024-02-18T00:33:04.701946+0000","last_became_peered":"2024-02-18T00:33:04.701946+0000","last_unstale":"2024-02-18T00:33:29.311260+0000","last_undegraded":"2024-02-18T00:33:29.311260+0000","last_fullsized":"2024-02-18T00:33:29.311260+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.356580+0000","last_change":"2024-02-18T00:33:04.730258+0000","last_active":"2024-02-18T00:33:29.356580+0000","last_peered":"2024-02-18T00:33:29.356580+0000","last_clean":"2024-02-18T00:33:29.356580+0000","last_became_active":"2024-02-18T00:33:04.729050+0000","last_became_peered":"2024-02-18T00:33:04.729050+0000","last_unstale":"2024-02-18T00:33:29.356580+0000","last_undegraded":"2024-02-18T00:33:29.356580+0000","last_fullsized":"2024-02-18T00:33:29.356580+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":137,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.325717+0000","last_change":"2024-02-18T00:32:02.547561+0000","last_active":"2024-02-18T00:33:29.325717+0000","last_peered":"2024-02-18T00:33:29.325717+0000","last_clean":"2024-02-18T00:33:29.325717+0000","last_became_active":"2024-02-18T00:29:17.801278+0000","last_became_peered":"2024-02-18T00:29:17.801278+0000","last_unstale":"2024-02-18T00:33:29.325717+0000","last_undegraded":"2024-02-18T00:33:29.325717+0000","last_fullsized":"2024-02-18T00:33:29.325717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:32:02.547447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.547447+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":103,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305683+0000","last_change":"2024-02-18T00:32:55.976686+0000","last_active":"2024-02-18T00:33:29.305683+0000","last_peered":"2024-02-18T00:33:29.305683+0000","last_clean":"2024-02-18T00:33:29.305683+0000","last_became_active":"2024-02-18T00:30:41.642130+0000","last_became_peered":"2024-02-18T00:30:41.642130+0000","last_unstale":"2024-02-18T00:33:29.305683+0000","last_undegraded":"2024-02-18T00:33:29.305683+0000","last_fullsized":"2024-02-18T00:33:29.305683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:55.976578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.976578+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":149,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.313248+0000","last_change":"2024-02-18T00:32:36.948567+0000","last_active":"2024-02-18T00:33:29.313248+0000","last_peered":"2024-02-18T00:33:29.313248+0000","last_clean":"2024-02-18T00:33:29.313248+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:33:29.313248+0000","last_undegraded":"2024-02-18T00:33:29.313248+0000","last_fullsized":"2024-02-18T00:33:29.313248+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:36.948476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.948476+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.336143+0000","last_change":"2024-02-18T00:31:37.150285+0000","last_active":"2024-02-18T00:33:29.336143+0000","last_peered":"2024-02-18T00:33:29.336143+0000","last_clean":"2024-02-18T00:33:29.336143+0000","last_became_active":"2024-02-18T00:29:21.100270+0000","last_became_peered":"2024-02-18T00:29:21.100270+0000","last_unstale":"2024-02-18T00:33:29.336143+0000","last_undegraded":"2024-02-18T00:33:29.336143+0000","last_fullsized":"2024-02-18T00:33:29.336143+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:37.150215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:37.150215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"106'2","reported_seq":44,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.311317+0000","last_change":"2024-02-18T00:33:05.970604+0000","last_active":"2024-02-18T00:33:29.311317+0000","last_peered":"2024-02-18T00:33:29.311317+0000","last_clean":"2024-02-18T00:33:29.311317+0000","last_became_active":"2024-02-18T00:33:01.577813+0000","last_became_peered":"2024-02-18T00:33:01.577813+0000","last_unstale":"2024-02-18T00:33:29.311317+0000","last_undegraded":"2024-02-18T00:33:29.311317+0000","last_fullsized":"2024-02-18T00:33:29.311317+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'2","last_scrub_stamp":"2024-02-18T00:33:05.970532+0000","last_deep_scrub":"106'2","last_deep_scrub_stamp":"2024-02-18T00:33:05.970532+0000","last_clean_scrub_stamp":"2024-02-18T00:33:05.970532+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"109'2","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.322681+0000","last_change":"2024-02-18T00:33:04.682773+0000","last_active":"2024-02-18T00:33:29.322681+0000","last_peered":"2024-02-18T00:33:29.322681+0000","last_clean":"2024-02-18T00:33:29.322681+0000","last_became_active":"2024-02-18T00:33:04.679723+0000","last_became_peered":"2024-02-18T00:33:04.679723+0000","last_unstale":"2024-02-18T00:33:29.322681+0000","last_undegraded":"2024-02-18T00:33:29.322681+0000","last_fullsized":"2024-02-18T00:33:29.322681+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.295788+0000","last_change":"2024-02-18T00:33:02.431426+0000","last_active":"2024-02-18T00:33:29.295788+0000","last_peered":"2024-02-18T00:33:29.295788+0000","last_clean":"2024-02-18T00:33:29.295788+0000","last_became_active":"2024-02-18T00:33:02.430642+0000","last_became_peered":"2024-02-18T00:33:02.430642+0000","last_unstale":"2024-02-18T00:33:29.295788+0000","last_undegraded":"2024-02-18T00:33:29.295788+0000","last_fullsized":"2024-02-18T00:33:29.295788+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":118,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.313054+0000","last_change":"2024-02-18T00:31:18.031405+0000","last_active":"2024-02-18T00:33:29.313054+0000","last_peered":"2024-02-18T00:33:29.313054+0000","last_clean":"2024-02-18T00:33:29.313054+0000","last_became_active":"2024-02-18T00:29:17.800211+0000","last_became_peered":"2024-02-18T00:29:17.800211+0000","last_unstale":"2024-02-18T00:33:29.313054+0000","last_undegraded":"2024-02-18T00:33:29.313054+0000","last_fullsized":"2024-02-18T00:33:29.313054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:31:18.031302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:18.031302+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":93,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.328218+0000","last_change":"2024-02-18T00:31:41.112608+0000","last_active":"2024-02-18T00:33:29.328218+0000","last_peered":"2024-02-18T00:33:29.328218+0000","last_clean":"2024-02-18T00:33:29.328218+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:33:29.328218+0000","last_undegraded":"2024-02-18T00:33:29.328218+0000","last_fullsized":"2024-02-18T00:33:29.328218+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.112540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.112540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"64'8","reported_seq":149,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.325938+0000","last_change":"2024-02-18T00:32:01.547864+0000","last_active":"2024-02-18T00:33:29.325938+0000","last_peered":"2024-02-18T00:33:29.325938+0000","last_clean":"2024-02-18T00:33:29.325938+0000","last_became_active":"2024-02-18T00:29:19.103748+0000","last_became_peered":"2024-02-18T00:29:19.103748+0000","last_unstale":"2024-02-18T00:33:29.325938+0000","last_undegraded":"2024-02-18T00:33:29.325938+0000","last_fullsized":"2024-02-18T00:33:29.325938+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:01.547763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:01.547763+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.301131+0000","last_change":"2024-02-18T00:32:53.114523+0000","last_active":"2024-02-18T00:33:29.301131+0000","last_peered":"2024-02-18T00:33:29.301131+0000","last_clean":"2024-02-18T00:33:29.301131+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:33:29.301131+0000","last_undegraded":"2024-02-18T00:33:29.301131+0000","last_fullsized":"2024-02-18T00:33:29.301131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.114424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.114424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"106'1","reported_seq":42,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314232+0000","last_change":"2024-02-18T00:33:05.821752+0000","last_active":"2024-02-18T00:33:29.314232+0000","last_peered":"2024-02-18T00:33:29.314232+0000","last_clean":"2024-02-18T00:33:29.314232+0000","last_became_active":"2024-02-18T00:33:01.406671+0000","last_became_peered":"2024-02-18T00:33:01.406671+0000","last_unstale":"2024-02-18T00:33:29.314232+0000","last_undegraded":"2024-02-18T00:33:29.314232+0000","last_fullsized":"2024-02-18T00:33:29.314232+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'1","last_scrub_stamp":"2024-02-18T00:33:05.821674+0000","last_deep_scrub":"106'1","last_deep_scrub_stamp":"2024-02-18T00:33:05.821674+0000","last_clean_scrub_stamp":"2024-02-18T00:33:05.821674+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.369779+0000","last_change":"2024-02-18T00:33:04.704619+0000","last_active":"2024-02-18T00:33:29.369779+0000","last_peered":"2024-02-18T00:33:29.369779+0000","last_clean":"2024-02-18T00:33:29.369779+0000","last_became_active":"2024-02-18T00:33:04.703531+0000","last_became_peered":"2024-02-18T00:33:04.703531+0000","last_unstale":"2024-02-18T00:33:29.369779+0000","last_undegraded":"2024-02-18T00:33:29.369779+0000","last_fullsized":"2024-02-18T00:33:29.369779+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.319556+0000","last_change":"2024-02-18T00:33:02.532649+0000","last_active":"2024-02-18T00:33:29.319556+0000","last_peered":"2024-02-18T00:33:29.319556+0000","last_clean":"2024-02-18T00:33:29.319556+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:33:29.319556+0000","last_undegraded":"2024-02-18T00:33:29.319556+0000","last_fullsized":"2024-02-18T00:33:29.319556+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":108,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299422+0000","last_change":"2024-02-18T00:32:37.815524+0000","last_active":"2024-02-18T00:33:29.299422+0000","last_peered":"2024-02-18T00:33:29.299422+0000","last_clean":"2024-02-18T00:33:29.299422+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:33:29.299422+0000","last_undegraded":"2024-02-18T00:33:29.299422+0000","last_fullsized":"2024-02-18T00:33:29.299422+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:32:37.815411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:37.815411+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.303141+0000","last_change":"2024-02-18T00:32:52.323010+0000","last_active":"2024-02-18T00:33:29.303141+0000","last_peered":"2024-02-18T00:33:29.303141+0000","last_clean":"2024-02-18T00:33:29.303141+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:33:29.303141+0000","last_undegraded":"2024-02-18T00:33:29.303141+0000","last_fullsized":"2024-02-18T00:33:29.303141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:52.322930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:52.322930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.322901+0000","last_change":"2024-02-18T00:31:57.501367+0000","last_active":"2024-02-18T00:33:29.322901+0000","last_peered":"2024-02-18T00:33:29.322901+0000","last_clean":"2024-02-18T00:33:29.322901+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:33:29.322901+0000","last_undegraded":"2024-02-18T00:33:29.322901+0000","last_fullsized":"2024-02-18T00:33:29.322901+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.501298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.501298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314011+0000","last_change":"2024-02-18T00:33:01.075249+0000","last_active":"2024-02-18T00:33:29.314011+0000","last_peered":"2024-02-18T00:33:29.314011+0000","last_clean":"2024-02-18T00:33:29.314011+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:33:29.314011+0000","last_undegraded":"2024-02-18T00:33:29.314011+0000","last_fullsized":"2024-02-18T00:33:29.314011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.075169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.075169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":41,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320367+0000","last_change":"2024-02-18T00:33:10.999268+0000","last_active":"2024-02-18T00:33:29.320367+0000","last_peered":"2024-02-18T00:33:29.320367+0000","last_clean":"2024-02-18T00:33:29.320367+0000","last_became_active":"2024-02-18T00:33:01.567237+0000","last_became_peered":"2024-02-18T00:33:01.567237+0000","last_unstale":"2024-02-18T00:33:29.320367+0000","last_undegraded":"2024-02-18T00:33:29.320367+0000","last_fullsized":"2024-02-18T00:33:29.320367+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:10.999164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:10.999164+0000","last_clean_scrub_stamp":"2024-02-18T00:33:10.999164+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302869+0000","last_change":"2024-02-18T00:33:04.600039+0000","last_active":"2024-02-18T00:33:29.302869+0000","last_peered":"2024-02-18T00:33:29.302869+0000","last_clean":"2024-02-18T00:33:29.302869+0000","last_became_active":"2024-02-18T00:33:04.599784+0000","last_became_peered":"2024-02-18T00:33:04.599784+0000","last_unstale":"2024-02-18T00:33:29.302869+0000","last_undegraded":"2024-02-18T00:33:29.302869+0000","last_fullsized":"2024-02-18T00:33:29.302869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.335620+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:33:29.335620+0000","last_peered":"2024-02-18T00:33:29.335620+0000","last_clean":"2024-02-18T00:33:29.335620+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:33:29.335620+0000","last_undegraded":"2024-02-18T00:33:29.335620+0000","last_fullsized":"2024-02-18T00:33:29.335620+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":111,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.293037+0000","last_change":"2024-02-18T00:31:54.074682+0000","last_active":"2024-02-18T00:33:29.293037+0000","last_peered":"2024-02-18T00:33:29.293037+0000","last_clean":"2024-02-18T00:33:29.293037+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:33:29.293037+0000","last_undegraded":"2024-02-18T00:33:29.293037+0000","last_fullsized":"2024-02-18T00:33:29.293037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:31:54.074589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.074589+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.329688+0000","last_change":"2024-02-18T00:32:40.792705+0000","last_active":"2024-02-18T00:33:29.329688+0000","last_peered":"2024-02-18T00:33:29.329688+0000","last_clean":"2024-02-18T00:33:29.329688+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:33:29.329688+0000","last_undegraded":"2024-02-18T00:33:29.329688+0000","last_fullsized":"2024-02-18T00:33:29.329688+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:40.792593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.792593+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299501+0000","last_change":"2024-02-18T00:32:43.773879+0000","last_active":"2024-02-18T00:33:29.299501+0000","last_peered":"2024-02-18T00:33:29.299501+0000","last_clean":"2024-02-18T00:33:29.299501+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:33:29.299501+0000","last_undegraded":"2024-02-18T00:33:29.299501+0000","last_fullsized":"2024-02-18T00:33:29.299501+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.773801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.773801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305753+0000","last_change":"2024-02-18T00:32:46.709442+0000","last_active":"2024-02-18T00:33:29.305753+0000","last_peered":"2024-02-18T00:33:29.305753+0000","last_clean":"2024-02-18T00:33:29.305753+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:33:29.305753+0000","last_undegraded":"2024-02-18T00:33:29.305753+0000","last_fullsized":"2024-02-18T00:33:29.305753+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:46.709314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.709314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.6","version":"106'1","reported_seq":42,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320831+0000","last_change":"2024-02-18T00:33:09.430682+0000","last_active":"2024-02-18T00:33:29.320831+0000","last_peered":"2024-02-18T00:33:29.320831+0000","last_clean":"2024-02-18T00:33:29.320831+0000","last_became_active":"2024-02-18T00:33:01.493762+0000","last_became_peered":"2024-02-18T00:33:01.493762+0000","last_unstale":"2024-02-18T00:33:29.320831+0000","last_undegraded":"2024-02-18T00:33:29.320831+0000","last_fullsized":"2024-02-18T00:33:29.320831+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'1","last_scrub_stamp":"2024-02-18T00:33:09.430597+0000","last_deep_scrub":"106'1","last_deep_scrub_stamp":"2024-02-18T00:33:09.430597+0000","last_clean_scrub_stamp":"2024-02-18T00:33:09.430597+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315989+0000","last_change":"2024-02-18T00:33:02.492453+0000","last_active":"2024-02-18T00:33:29.315989+0000","last_peered":"2024-02-18T00:33:29.315989+0000","last_clean":"2024-02-18T00:33:29.315989+0000","last_became_active":"2024-02-18T00:33:02.491650+0000","last_became_peered":"2024-02-18T00:33:02.491650+0000","last_unstale":"2024-02-18T00:33:29.315989+0000","last_undegraded":"2024-02-18T00:33:29.315989+0000","last_fullsized":"2024-02-18T00:33:29.315989+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320252+0000","last_change":"2024-02-18T00:33:02.499776+0000","last_active":"2024-02-18T00:33:29.320252+0000","last_peered":"2024-02-18T00:33:29.320252+0000","last_clean":"2024-02-18T00:33:29.320252+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:33:29.320252+0000","last_undegraded":"2024-02-18T00:33:29.320252+0000","last_fullsized":"2024-02-18T00:33:29.320252+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":98,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312025+0000","last_change":"2024-02-18T00:32:48.722576+0000","last_active":"2024-02-18T00:33:29.312025+0000","last_peered":"2024-02-18T00:33:29.312025+0000","last_clean":"2024-02-18T00:33:29.312025+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:33:29.312025+0000","last_undegraded":"2024-02-18T00:33:29.312025+0000","last_fullsized":"2024-02-18T00:33:29.312025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:32:48.722501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:48.722501+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315914+0000","last_change":"2024-02-18T00:31:30.789715+0000","last_active":"2024-02-18T00:33:29.315914+0000","last_peered":"2024-02-18T00:33:29.315914+0000","last_clean":"2024-02-18T00:33:29.315914+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:33:29.315914+0000","last_undegraded":"2024-02-18T00:33:29.315914+0000","last_fullsized":"2024-02-18T00:33:29.315914+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:31:30.789634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296539+0000","last_change":"2024-02-18T00:32:07.258321+0000","last_active":"2024-02-18T00:33:29.296539+0000","last_peered":"2024-02-18T00:33:29.296539+0000","last_clean":"2024-02-18T00:33:29.296539+0000","last_became_active":"2024-02-18T00:29:19.099475+0000","last_became_peered":"2024-02-18T00:29:19.099475+0000","last_unstale":"2024-02-18T00:33:29.296539+0000","last_undegraded":"2024-02-18T00:33:29.296539+0000","last_fullsized":"2024-02-18T00:33:29.296539+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.258239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.258239+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.304577+0000","last_change":"2024-02-18T00:32:36.719564+0000","last_active":"2024-02-18T00:33:29.304577+0000","last_peered":"2024-02-18T00:33:29.304577+0000","last_clean":"2024-02-18T00:33:29.304577+0000","last_became_active":"2024-02-18T00:29:21.094738+0000","last_became_peered":"2024-02-18T00:29:21.094738+0000","last_unstale":"2024-02-18T00:33:29.304577+0000","last_undegraded":"2024-02-18T00:33:29.304577+0000","last_fullsized":"2024-02-18T00:33:29.304577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.719484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.719484+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":41,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.320178+0000","last_change":"2024-02-18T00:33:06.364448+0000","last_active":"2024-02-18T00:33:29.320178+0000","last_peered":"2024-02-18T00:33:29.320178+0000","last_clean":"2024-02-18T00:33:29.320178+0000","last_became_active":"2024-02-18T00:33:01.494037+0000","last_became_peered":"2024-02-18T00:33:01.494037+0000","last_unstale":"2024-02-18T00:33:29.320178+0000","last_undegraded":"2024-02-18T00:33:29.320178+0000","last_fullsized":"2024-02-18T00:33:29.320178+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:06.364359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:06.364359+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.364359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305394+0000","last_change":"2024-02-18T00:33:02.493509+0000","last_active":"2024-02-18T00:33:29.305394+0000","last_peered":"2024-02-18T00:33:29.305394+0000","last_clean":"2024-02-18T00:33:29.305394+0000","last_became_active":"2024-02-18T00:33:02.492366+0000","last_became_peered":"2024-02-18T00:33:02.492366+0000","last_unstale":"2024-02-18T00:33:29.305394+0000","last_undegraded":"2024-02-18T00:33:29.305394+0000","last_fullsized":"2024-02-18T00:33:29.305394+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.318961+0000","last_change":"2024-02-18T00:33:02.495634+0000","last_active":"2024-02-18T00:33:29.318961+0000","last_peered":"2024-02-18T00:33:29.318961+0000","last_clean":"2024-02-18T00:33:29.318961+0000","last_became_active":"2024-02-18T00:33:02.495368+0000","last_became_peered":"2024-02-18T00:33:02.495368+0000","last_unstale":"2024-02-18T00:33:29.318961+0000","last_undegraded":"2024-02-18T00:33:29.318961+0000","last_fullsized":"2024-02-18T00:33:29.318961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306555+0000","last_change":"2024-02-18T00:32:40.775783+0000","last_active":"2024-02-18T00:33:29.306555+0000","last_peered":"2024-02-18T00:33:29.306555+0000","last_clean":"2024-02-18T00:33:29.306555+0000","last_became_active":"2024-02-18T00:29:17.801400+0000","last_became_peered":"2024-02-18T00:29:17.801400+0000","last_unstale":"2024-02-18T00:33:29.306555+0000","last_undegraded":"2024-02-18T00:33:29.306555+0000","last_fullsized":"2024-02-18T00:33:29.306555+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:32:40.775672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.775672+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309700+0000","last_change":"2024-02-18T00:31:51.186460+0000","last_active":"2024-02-18T00:33:29.309700+0000","last_peered":"2024-02-18T00:33:29.309700+0000","last_clean":"2024-02-18T00:33:29.309700+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:33:29.309700+0000","last_undegraded":"2024-02-18T00:33:29.309700+0000","last_fullsized":"2024-02-18T00:33:29.309700+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:51.186373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:51.186373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314920+0000","last_change":"2024-02-18T00:32:07.760043+0000","last_active":"2024-02-18T00:33:29.314920+0000","last_peered":"2024-02-18T00:33:29.314920+0000","last_clean":"2024-02-18T00:33:29.314920+0000","last_became_active":"2024-02-18T00:29:19.101667+0000","last_became_peered":"2024-02-18T00:29:19.101667+0000","last_unstale":"2024-02-18T00:33:29.314920+0000","last_undegraded":"2024-02-18T00:33:29.314920+0000","last_fullsized":"2024-02-18T00:33:29.314920+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.759964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.759964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.323558+0000","last_change":"2024-02-18T00:32:51.698582+0000","last_active":"2024-02-18T00:33:29.323558+0000","last_peered":"2024-02-18T00:33:29.323558+0000","last_clean":"2024-02-18T00:33:29.323558+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:33:29.323558+0000","last_undegraded":"2024-02-18T00:33:29.323558+0000","last_fullsized":"2024-02-18T00:33:29.323558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:51.698513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:51.698513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"112'6","reported_seq":57,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.321433+0000","last_change":"2024-02-18T00:33:08.068061+0000","last_active":"2024-02-18T00:33:29.321433+0000","last_peered":"2024-02-18T00:33:29.321433+0000","last_clean":"2024-02-18T00:33:29.321433+0000","last_became_active":"2024-02-18T00:33:01.526381+0000","last_became_peered":"2024-02-18T00:33:01.526381+0000","last_unstale":"2024-02-18T00:33:29.321433+0000","last_undegraded":"2024-02-18T00:33:29.321433+0000","last_fullsized":"2024-02-18T00:33:29.321433+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6","last_scrub_stamp":"2024-02-18T00:33:08.067963+0000","last_deep_scrub":"112'6","last_deep_scrub_stamp":"2024-02-18T00:33:08.067963+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.067963+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165852,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24583,"num_write":3,"num_write_kb":24579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.0","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299555+0000","last_change":"2024-02-18T00:33:02.516955+0000","last_active":"2024-02-18T00:33:29.299555+0000","last_peered":"2024-02-18T00:33:29.299555+0000","last_clean":"2024-02-18T00:33:29.299555+0000","last_became_active":"2024-02-18T00:33:02.516681+0000","last_became_peered":"2024-02-18T00:33:02.516681+0000","last_unstale":"2024-02-18T00:33:29.299555+0000","last_undegraded":"2024-02-18T00:33:29.299555+0000","last_fullsized":"2024-02-18T00:33:29.299555+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307494+0000","last_change":"2024-02-18T00:33:02.504339+0000","last_active":"2024-02-18T00:33:29.307494+0000","last_peered":"2024-02-18T00:33:29.307494+0000","last_clean":"2024-02-18T00:33:29.307494+0000","last_became_active":"2024-02-18T00:33:02.503627+0000","last_became_peered":"2024-02-18T00:33:02.503627+0000","last_unstale":"2024-02-18T00:33:29.307494+0000","last_undegraded":"2024-02-18T00:33:29.307494+0000","last_fullsized":"2024-02-18T00:33:29.307494+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"64'16","reported_seq":121,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.337359+0000","last_change":"2024-02-18T00:31:55.016021+0000","last_active":"2024-02-18T00:33:29.337359+0000","last_peered":"2024-02-18T00:33:29.337359+0000","last_clean":"2024-02-18T00:33:29.337359+0000","last_became_active":"2024-02-18T00:29:17.799960+0000","last_became_peered":"2024-02-18T00:29:17.799960+0000","last_unstale":"2024-02-18T00:33:29.337359+0000","last_undegraded":"2024-02-18T00:33:29.337359+0000","last_fullsized":"2024-02-18T00:33:29.337359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'16","last_scrub_stamp":"2024-02-18T00:31:55.015959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.015959+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299742+0000","last_change":"2024-02-18T00:32:49.927116+0000","last_active":"2024-02-18T00:33:29.299742+0000","last_peered":"2024-02-18T00:33:29.299742+0000","last_clean":"2024-02-18T00:33:29.299742+0000","last_became_active":"2024-02-18T00:29:14.713882+0000","last_became_peered":"2024-02-18T00:29:14.713882+0000","last_unstale":"2024-02-18T00:33:29.299742+0000","last_undegraded":"2024-02-18T00:33:29.299742+0000","last_fullsized":"2024-02-18T00:33:29.299742+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:49.926996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.926996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"64'8","reported_seq":157,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309983+0000","last_change":"2024-02-18T00:32:45.395174+0000","last_active":"2024-02-18T00:33:29.309983+0000","last_peered":"2024-02-18T00:33:29.309983+0000","last_clean":"2024-02-18T00:33:29.309983+0000","last_became_active":"2024-02-18T00:29:19.099047+0000","last_became_peered":"2024-02-18T00:29:19.099047+0000","last_unstale":"2024-02-18T00:33:29.309983+0000","last_undegraded":"2024-02-18T00:33:29.309983+0000","last_fullsized":"2024-02-18T00:33:29.309983+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:45.395085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.395085+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":89,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.308007+0000","last_change":"2024-02-18T00:31:45.654910+0000","last_active":"2024-02-18T00:33:29.308007+0000","last_peered":"2024-02-18T00:33:29.308007+0000","last_clean":"2024-02-18T00:33:29.308007+0000","last_became_active":"2024-02-18T00:29:21.096065+0000","last_became_peered":"2024-02-18T00:29:21.096065+0000","last_unstale":"2024-02-18T00:33:29.308007+0000","last_undegraded":"2024-02-18T00:33:29.308007+0000","last_fullsized":"2024-02-18T00:33:29.308007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:31:45.654827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:45.654827+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.327667+0000","last_change":"2024-02-18T00:33:01.581012+0000","last_active":"2024-02-18T00:33:29.327667+0000","last_peered":"2024-02-18T00:33:29.327667+0000","last_clean":"2024-02-18T00:33:29.327667+0000","last_became_active":"2024-02-18T00:33:01.579869+0000","last_became_peered":"2024-02-18T00:33:01.579869+0000","last_unstale":"2024-02-18T00:33:29.327667+0000","last_undegraded":"2024-02-18T00:33:29.327667+0000","last_fullsized":"2024-02-18T00:33:29.327667+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299973+0000","last_change":"2024-02-18T00:33:02.504123+0000","last_active":"2024-02-18T00:33:29.299973+0000","last_peered":"2024-02-18T00:33:29.299973+0000","last_clean":"2024-02-18T00:33:29.299973+0000","last_became_active":"2024-02-18T00:33:02.503440+0000","last_became_peered":"2024-02-18T00:33:02.503440+0000","last_unstale":"2024-02-18T00:33:29.299973+0000","last_undegraded":"2024-02-18T00:33:29.299973+0000","last_fullsized":"2024-02-18T00:33:29.299973+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305100+0000","last_change":"2024-02-18T00:33:02.513760+0000","last_active":"2024-02-18T00:33:29.305100+0000","last_peered":"2024-02-18T00:33:29.305100+0000","last_clean":"2024-02-18T00:33:29.305100+0000","last_became_active":"2024-02-18T00:33:02.513275+0000","last_became_peered":"2024-02-18T00:33:02.513275+0000","last_unstale":"2024-02-18T00:33:29.305100+0000","last_undegraded":"2024-02-18T00:33:29.305100+0000","last_fullsized":"2024-02-18T00:33:29.305100+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":126,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310065+0000","last_change":"2024-02-18T00:31:56.165995+0000","last_active":"2024-02-18T00:33:29.310065+0000","last_peered":"2024-02-18T00:33:29.310065+0000","last_clean":"2024-02-18T00:33:29.310065+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:33:29.310065+0000","last_undegraded":"2024-02-18T00:33:29.310065+0000","last_fullsized":"2024-02-18T00:33:29.310065+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:31:56.165920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:56.165920+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305978+0000","last_change":"2024-02-18T00:31:49.512647+0000","last_active":"2024-02-18T00:33:29.305978+0000","last_peered":"2024-02-18T00:33:29.305978+0000","last_clean":"2024-02-18T00:33:29.305978+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:33:29.305978+0000","last_undegraded":"2024-02-18T00:33:29.305978+0000","last_fullsized":"2024-02-18T00:33:29.305978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:31:49.512572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":84,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.337434+0000","last_change":"2024-02-18T00:31:23.240187+0000","last_active":"2024-02-18T00:33:29.337434+0000","last_peered":"2024-02-18T00:33:29.337434+0000","last_clean":"2024-02-18T00:33:29.337434+0000","last_became_active":"2024-02-18T00:29:19.091746+0000","last_became_peered":"2024-02-18T00:29:19.091746+0000","last_unstale":"2024-02-18T00:33:29.337434+0000","last_undegraded":"2024-02-18T00:33:29.337434+0000","last_fullsized":"2024-02-18T00:33:29.337434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:23.240097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:23.240097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":93,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.329425+0000","last_change":"2024-02-18T00:32:59.800515+0000","last_active":"2024-02-18T00:33:29.329425+0000","last_peered":"2024-02-18T00:33:29.329425+0000","last_clean":"2024-02-18T00:33:29.329425+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:33:29.329425+0000","last_undegraded":"2024-02-18T00:33:29.329425+0000","last_fullsized":"2024-02-18T00:33:29.329425+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.800355+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"106'2","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305186+0000","last_change":"2024-02-18T00:33:01.428299+0000","last_active":"2024-02-18T00:33:29.305186+0000","last_peered":"2024-02-18T00:33:29.305186+0000","last_clean":"2024-02-18T00:33:29.305186+0000","last_became_active":"2024-02-18T00:33:01.428013+0000","last_became_peered":"2024-02-18T00:33:01.428013+0000","last_unstale":"2024-02-18T00:33:29.305186+0000","last_undegraded":"2024-02-18T00:33:29.305186+0000","last_fullsized":"2024-02-18T00:33:29.305186+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299693+0000","last_change":"2024-02-18T00:33:02.537930+0000","last_active":"2024-02-18T00:33:29.299693+0000","last_peered":"2024-02-18T00:33:29.299693+0000","last_clean":"2024-02-18T00:33:29.299693+0000","last_became_active":"2024-02-18T00:33:02.536038+0000","last_became_peered":"2024-02-18T00:33:02.536038+0000","last_unstale":"2024-02-18T00:33:29.299693+0000","last_undegraded":"2024-02-18T00:33:29.299693+0000","last_fullsized":"2024-02-18T00:33:29.299693+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":47,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.356503+0000","last_change":"2024-02-18T00:33:04.698462+0000","last_active":"2024-02-18T00:33:29.356503+0000","last_peered":"2024-02-18T00:33:29.356503+0000","last_clean":"2024-02-18T00:33:29.356503+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:33:29.356503+0000","last_undegraded":"2024-02-18T00:33:29.356503+0000","last_fullsized":"2024-02-18T00:33:29.356503+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":122,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.370353+0000","last_change":"2024-02-18T00:32:48.395232+0000","last_active":"2024-02-18T00:33:29.370353+0000","last_peered":"2024-02-18T00:33:29.370353+0000","last_clean":"2024-02-18T00:33:29.370353+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:33:29.370353+0000","last_undegraded":"2024-02-18T00:33:29.370353+0000","last_fullsized":"2024-02-18T00:33:29.370353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:48.395162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:48.395162+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296487+0000","last_change":"2024-02-18T00:32:57.386162+0000","last_active":"2024-02-18T00:33:29.296487+0000","last_peered":"2024-02-18T00:33:29.296487+0000","last_clean":"2024-02-18T00:33:29.296487+0000","last_became_active":"2024-02-18T00:29:14.717295+0000","last_became_peered":"2024-02-18T00:29:14.717295+0000","last_unstale":"2024-02-18T00:33:29.296487+0000","last_undegraded":"2024-02-18T00:33:29.296487+0000","last_fullsized":"2024-02-18T00:33:29.296487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.386090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.386090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.333463+0000","last_change":"2024-02-18T00:32:06.883082+0000","last_active":"2024-02-18T00:33:29.333463+0000","last_peered":"2024-02-18T00:33:29.333463+0000","last_clean":"2024-02-18T00:33:29.333463+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:33:29.333463+0000","last_undegraded":"2024-02-18T00:33:29.333463+0000","last_fullsized":"2024-02-18T00:33:29.333463+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:06.883019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:32:06.883019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.296114+0000","last_change":"2024-02-18T00:32:38.985290+0000","last_active":"2024-02-18T00:33:29.296114+0000","last_peered":"2024-02-18T00:33:29.296114+0000","last_clean":"2024-02-18T00:33:29.296114+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:33:29.296114+0000","last_undegraded":"2024-02-18T00:33:29.296114+0000","last_fullsized":"2024-02-18T00:33:29.296114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.985181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.985181+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"106'6","reported_seq":47,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305572+0000","last_change":"2024-02-18T00:33:08.650721+0000","last_active":"2024-02-18T00:33:29.305572+0000","last_peered":"2024-02-18T00:33:29.305572+0000","last_clean":"2024-02-18T00:33:29.305572+0000","last_became_active":"2024-02-18T00:33:01.593606+0000","last_became_peered":"2024-02-18T00:33:01.593606+0000","last_unstale":"2024-02-18T00:33:29.305572+0000","last_undegraded":"2024-02-18T00:33:29.305572+0000","last_fullsized":"2024-02-18T00:33:29.305572+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6","last_scrub_stamp":"2024-02-18T00:33:08.650652+0000","last_deep_scrub":"106'6","last_deep_scrub_stamp":"2024-02-18T00:33:08.650652+0000","last_clean_scrub_stamp":"2024-02-18T00:33:08.650652+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3","version":"109'5","reported_seq":39,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307779+0000","last_change":"2024-02-18T00:33:02.529166+0000","last_active":"2024-02-18T00:33:29.307779+0000","last_peered":"2024-02-18T00:33:29.307779+0000","last_clean":"2024-02-18T00:33:29.307779+0000","last_became_active":"2024-02-18T00:33:02.528960+0000","last_became_peered":"2024-02-18T00:33:02.528960+0000","last_unstale":"2024-02-18T00:33:29.307779+0000","last_undegraded":"2024-02-18T00:33:29.307779+0000","last_fullsized":"2024-02-18T00:33:29.307779+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.299633+0000","last_change":"2024-02-18T00:33:02.473416+0000","last_active":"2024-02-18T00:33:29.299633+0000","last_peered":"2024-02-18T00:33:29.299633+0000","last_clean":"2024-02-18T00:33:29.299633+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:33:29.299633+0000","last_undegraded":"2024-02-18T00:33:29.299633+0000","last_fullsized":"2024-02-18T00:33:29.299633+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":130,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302411+0000","last_change":"2024-02-18T00:32:44.768998+0000","last_active":"2024-02-18T00:33:29.302411+0000","last_peered":"2024-02-18T00:33:29.302411+0000","last_clean":"2024-02-18T00:33:29.302411+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:33:29.302411+0000","last_undegraded":"2024-02-18T00:33:29.302411+0000","last_fullsized":"2024-02-18T00:33:29.302411+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:32:44.768886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.768886+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":101,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.322826+0000","last_change":"2024-02-18T00:31:32.210269+0000","last_active":"2024-02-18T00:33:29.322826+0000","last_peered":"2024-02-18T00:33:29.322826+0000","last_clean":"2024-02-18T00:33:29.322826+0000","last_became_active":"2024-02-18T00:29:14.713943+0000","last_became_peered":"2024-02-18T00:29:14.713943+0000","last_unstale":"2024-02-18T00:33:29.322826+0000","last_undegraded":"2024-02-18T00:33:29.322826+0000","last_fullsized":"2024-02-18T00:33:29.322826+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:31:32.210198+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:31:32.210198+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310665+0000","last_change":"2024-02-18T00:32:41.027232+0000","last_active":"2024-02-18T00:33:29.310665+0000","last_peered":"2024-02-18T00:33:29.310665+0000","last_clean":"2024-02-18T00:33:29.310665+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:33:29.310665+0000","last_undegraded":"2024-02-18T00:33:29.310665+0000","last_fullsized":"2024-02-18T00:33:29.310665+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:41.027153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:41.027153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":83,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.312643+0000","last_change":"2024-02-18T00:31:04.044662+0000","last_active":"2024-02-18T00:33:29.312643+0000","last_peered":"2024-02-18T00:33:29.312643+0000","last_clean":"2024-02-18T00:33:29.312643+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:33:29.312643+0000","last_undegraded":"2024-02-18T00:33:29.312643+0000","last_fullsized":"2024-02-18T00:33:29.312643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:04.044596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:04.044596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"106'6","reported_seq":47,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.315062+0000","last_change":"2024-02-18T00:33:06.776464+0000","last_active":"2024-02-18T00:33:29.315062+0000","last_peered":"2024-02-18T00:33:29.315062+0000","last_clean":"2024-02-18T00:33:29.315062+0000","last_became_active":"2024-02-18T00:33:01.427985+0000","last_became_peered":"2024-02-18T00:33:01.427985+0000","last_unstale":"2024-02-18T00:33:29.315062+0000","last_undegraded":"2024-02-18T00:33:29.315062+0000","last_fullsized":"2024-02-18T00:33:29.315062+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6","last_scrub_stamp":"2024-02-18T00:33:06.776396+0000","last_deep_scrub":"106'6","last_deep_scrub_stamp":"2024-02-18T00:33:06.776396+0000","last_clean_scrub_stamp":"2024-02-18T00:33:06.776396+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.370201+0000","last_change":"2024-02-18T00:33:04.688881+0000","last_active":"2024-02-18T00:33:29.370201+0000","last_peered":"2024-02-18T00:33:29.370201+0000","last_clean":"2024-02-18T00:33:29.370201+0000","last_became_active":"2024-02-18T00:33:04.688607+0000","last_became_peered":"2024-02-18T00:33:04.688607+0000","last_unstale":"2024-02-18T00:33:29.370201+0000","last_undegraded":"2024-02-18T00:33:29.370201+0000","last_fullsized":"2024-02-18T00:33:29.370201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.319200+0000","last_change":"2024-02-18T00:33:04.595768+0000","last_active":"2024-02-18T00:33:29.319200+0000","last_peered":"2024-02-18T00:33:29.319200+0000","last_clean":"2024-02-18T00:33:29.319200+0000","last_became_active":"2024-02-18T00:33:04.595086+0000","last_became_peered":"2024-02-18T00:33:04.595086+0000","last_unstale":"2024-02-18T00:33:29.319200+0000","last_undegraded":"2024-02-18T00:33:29.319200+0000","last_fullsized":"2024-02-18T00:33:29.319200+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":150,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309884+0000","last_change":"2024-02-18T00:33:00.013117+0000","last_active":"2024-02-18T00:33:29.309884+0000","last_peered":"2024-02-18T00:33:29.309884+0000","last_clean":"2024-02-18T00:33:29.309884+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:33:29.309884+0000","last_undegraded":"2024-02-18T00:33:29.309884+0000","last_fullsized":"2024-02-18T00:33:29.309884+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:33:00.013051+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:33:00.013051+0000","last_clean_scrub_stamp":"2024-02-18T00:33:00.013051+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307645+0000","last_change":"2024-02-18T00:32:41.659287+0000","last_active":"2024-02-18T00:33:29.307645+0000","last_peered":"2024-02-18T00:33:29.307645+0000","last_clean":"2024-02-18T00:33:29.307645+0000","last_became_active":"2024-02-18T00:29:14.712320+0000","last_became_peered":"2024-02-18T00:29:14.712320+0000","last_unstale":"2024-02-18T00:33:29.307645+0000","last_undegraded":"2024-02-18T00:33:29.307645+0000","last_fullsized":"2024-02-18T00:33:29.307645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:41.659220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:41.659220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.317360+0000","last_change":"2024-02-18T00:31:58.856050+0000","last_active":"2024-02-18T00:33:29.317360+0000","last_peered":"2024-02-18T00:33:29.317360+0000","last_clean":"2024-02-18T00:33:29.317360+0000","last_became_active":"2024-02-18T00:29:19.096471+0000","last_became_peered":"2024-02-18T00:29:19.096471+0000","last_unstale":"2024-02-18T00:33:29.317360+0000","last_undegraded":"2024-02-18T00:33:29.317360+0000","last_fullsized":"2024-02-18T00:33:29.317360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.855979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.855979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.314895+0000","last_change":"2024-02-18T00:31:36.140716+0000","last_active":"2024-02-18T00:33:29.314895+0000","last_peered":"2024-02-18T00:33:29.314895+0000","last_clean":"2024-02-18T00:33:29.314895+0000","last_became_active":"2024-02-18T00:29:21.101578+0000","last_became_peered":"2024-02-18T00:29:21.101578+0000","last_unstale":"2024-02-18T00:33:29.314895+0000","last_undegraded":"2024-02-18T00:33:29.314895+0000","last_fullsized":"2024-02-18T00:33:29.314895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:36.140630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:36.140630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"108'1001","reported_seq":2037,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305756+0000","last_change":"2024-02-18T00:33:01.535111+0000","last_active":"2024-02-18T00:33:29.305756+0000","last_peered":"2024-02-18T00:33:29.305756+0000","last_clean":"2024-02-18T00:33:29.305756+0000","last_became_active":"2024-02-18T00:33:01.532648+0000","last_became_peered":"2024-02-18T00:33:01.532648+0000","last_unstale":"2024-02-18T00:33:29.305756+0000","last_undegraded":"2024-02-18T00:33:29.305756+0000","last_fullsized":"2024-02-18T00:33:29.305756+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1001,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1000,"num_read_kb":0,"num_write":1000,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.307305+0000","last_change":"2024-02-18T00:33:02.511002+0000","last_active":"2024-02-18T00:33:29.307305+0000","last_peered":"2024-02-18T00:33:29.307305+0000","last_clean":"2024-02-18T00:33:29.307305+0000","last_became_active":"2024-02-18T00:33:02.510701+0000","last_became_peered":"2024-02-18T00:33:02.510701+0000","last_unstale":"2024-02-18T00:33:29.307305+0000","last_undegraded":"2024-02-18T00:33:29.307305+0000","last_fullsized":"2024-02-18T00:33:29.307305+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.359880+0000","last_change":"2024-02-18T00:33:04.678891+0000","last_active":"2024-02-18T00:33:29.359880+0000","last_peered":"2024-02-18T00:33:29.359880+0000","last_clean":"2024-02-18T00:33:29.359880+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:33:29.359880+0000","last_undegraded":"2024-02-18T00:33:29.359880+0000","last_fullsized":"2024-02-18T00:33:29.359880+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":120,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.292808+0000","last_change":"2024-02-18T00:32:02.063451+0000","last_active":"2024-02-18T00:33:29.292808+0000","last_peered":"2024-02-18T00:33:29.292808+0000","last_clean":"2024-02-18T00:33:29.292808+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:33:29.292808+0000","last_undegraded":"2024-02-18T00:33:29.292808+0000","last_fullsized":"2024-02-18T00:33:29.292808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:32:02.063346+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.063346+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":99,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310643+0000","last_change":"2024-02-18T00:33:03.808951+0000","last_active":"2024-02-18T00:33:29.310643+0000","last_peered":"2024-02-18T00:33:29.310643+0000","last_clean":"2024-02-18T00:33:29.310643+0000","last_became_active":"2024-02-18T00:29:14.714498+0000","last_became_peered":"2024-02-18T00:29:14.714498+0000","last_unstale":"2024-02-18T00:33:29.310643+0000","last_undegraded":"2024-02-18T00:33:29.310643+0000","last_fullsized":"2024-02-18T00:33:29.310643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:03.808887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.808887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.304657+0000","last_change":"2024-02-18T00:31:41.607649+0000","last_active":"2024-02-18T00:33:29.304657+0000","last_peered":"2024-02-18T00:33:29.304657+0000","last_clean":"2024-02-18T00:33:29.304657+0000","last_became_active":"2024-02-18T00:29:19.100545+0000","last_became_peered":"2024-02-18T00:29:19.100545+0000","last_unstale":"2024-02-18T00:33:29.304657+0000","last_undegraded":"2024-02-18T00:33:29.304657+0000","last_fullsized":"2024-02-18T00:33:29.304657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.607567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.303388+0000","last_change":"2024-02-18T00:31:44.843600+0000","last_active":"2024-02-18T00:33:29.303388+0000","last_peered":"2024-02-18T00:33:29.303388+0000","last_clean":"2024-02-18T00:33:29.303388+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:33:29.303388+0000","last_undegraded":"2024-02-18T00:33:29.303388+0000","last_fullsized":"2024-02-18T00:33:29.303388+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:31:44.843494+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.e","version":"106'2","reported_seq":38,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.359796+0000","last_change":"2024-02-18T00:33:01.542256+0000","last_active":"2024-02-18T00:33:29.359796+0000","last_peered":"2024-02-18T00:33:29.359796+0000","last_clean":"2024-02-18T00:33:29.359796+0000","last_became_active":"2024-02-18T00:33:01.541655+0000","last_became_peered":"2024-02-18T00:33:01.541655+0000","last_unstale":"2024-02-18T00:33:29.359796+0000","last_undegraded":"2024-02-18T00:33:29.359796+0000","last_fullsized":"2024-02-18T00:33:29.359796+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"109'9","reported_seq":43,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.313754+0000","last_change":"2024-02-18T00:33:02.512609+0000","last_active":"2024-02-18T00:33:29.313754+0000","last_peered":"2024-02-18T00:33:29.313754+0000","last_clean":"2024-02-18T00:33:29.313754+0000","last_became_active":"2024-02-18T00:33:02.512297+0000","last_became_peered":"2024-02-18T00:33:02.512297+0000","last_unstale":"2024-02-18T00:33:29.313754+0000","last_undegraded":"2024-02-18T00:33:29.313754+0000","last_fullsized":"2024-02-18T00:33:29.313754+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.302781+0000","last_change":"2024-02-18T00:33:04.657140+0000","last_active":"2024-02-18T00:33:29.302781+0000","last_peered":"2024-02-18T00:33:29.302781+0000","last_clean":"2024-02-18T00:33:29.302781+0000","last_became_active":"2024-02-18T00:33:04.656649+0000","last_became_peered":"2024-02-18T00:33:04.656649+0000","last_unstale":"2024-02-18T00:33:29.302781+0000","last_undegraded":"2024-02-18T00:33:29.302781+0000","last_fullsized":"2024-02-18T00:33:29.302781+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":123,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.291615+0000","last_change":"2024-02-18T00:32:50.121604+0000","last_active":"2024-02-18T00:33:29.291615+0000","last_peered":"2024-02-18T00:33:29.291615+0000","last_clean":"2024-02-18T00:33:29.291615+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:33:29.291615+0000","last_undegraded":"2024-02-18T00:33:29.291615+0000","last_fullsized":"2024-02-18T00:33:29.291615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.121518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.121518+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":94,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.333036+0000","last_change":"2024-02-18T00:32:43.807178+0000","last_active":"2024-02-18T00:33:29.333036+0000","last_peered":"2024-02-18T00:33:29.333036+0000","last_clean":"2024-02-18T00:33:29.333036+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:33:29.333036+0000","last_undegraded":"2024-02-18T00:33:29.333036+0000","last_fullsized":"2024-02-18T00:33:29.333036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.807055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.807055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.336548+0000","last_change":"2024-02-18T00:32:38.804852+0000","last_active":"2024-02-18T00:33:29.336548+0000","last_peered":"2024-02-18T00:33:29.336548+0000","last_clean":"2024-02-18T00:33:29.336548+0000","last_became_active":"2024-02-18T00:29:19.096963+0000","last_became_peered":"2024-02-18T00:29:19.096963+0000","last_unstale":"2024-02-18T00:33:29.336548+0000","last_undegraded":"2024-02-18T00:33:29.336548+0000","last_fullsized":"2024-02-18T00:33:29.336548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.804774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.804774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.309807+0000","last_change":"2024-02-18T00:32:36.954112+0000","last_active":"2024-02-18T00:33:29.309807+0000","last_peered":"2024-02-18T00:33:29.309807+0000","last_clean":"2024-02-18T00:33:29.309807+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:33:29.309807+0000","last_undegraded":"2024-02-18T00:33:29.309807+0000","last_fullsized":"2024-02-18T00:33:29.309807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.954039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.954039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.310525+0000","last_change":"2024-02-18T00:33:01.472972+0000","last_active":"2024-02-18T00:33:29.310525+0000","last_peered":"2024-02-18T00:33:29.310525+0000","last_clean":"2024-02-18T00:33:29.310525+0000","last_became_active":"2024-02-18T00:33:01.472652+0000","last_became_peered":"2024-02-18T00:33:01.472652+0000","last_unstale":"2024-02-18T00:33:29.310525+0000","last_undegraded":"2024-02-18T00:33:29.310525+0000","last_fullsized":"2024-02-18T00:33:29.310525+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.362573+0000","last_change":"2024-02-18T00:33:04.691417+0000","last_active":"2024-02-18T00:33:29.362573+0000","last_peered":"2024-02-18T00:33:29.362573+0000","last_clean":"2024-02-18T00:33:29.362573+0000","last_became_active":"2024-02-18T00:33:04.691013+0000","last_became_peered":"2024-02-18T00:33:04.691013+0000","last_unstale":"2024-02-18T00:33:29.362573+0000","last_undegraded":"2024-02-18T00:33:29.362573+0000","last_fullsized":"2024-02-18T00:33:29.362573+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":34,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.326359+0000","last_change":"2024-02-18T00:33:04.720986+0000","last_active":"2024-02-18T00:33:29.326359+0000","last_peered":"2024-02-18T00:33:29.326359+0000","last_clean":"2024-02-18T00:33:29.326359+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:33:29.326359+0000","last_undegraded":"2024-02-18T00:33:29.326359+0000","last_fullsized":"2024-02-18T00:33:29.326359+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":172,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.305902+0000","last_change":"2024-02-18T00:31:05.056932+0000","last_active":"2024-02-18T00:33:29.305902+0000","last_peered":"2024-02-18T00:33:29.305902+0000","last_clean":"2024-02-18T00:33:29.305902+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:33:29.305902+0000","last_undegraded":"2024-02-18T00:33:29.305902+0000","last_fullsized":"2024-02-18T00:33:29.305902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:31:05.056863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:05.056863+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":93,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.318688+0000","last_change":"2024-02-18T00:32:58.998813+0000","last_active":"2024-02-18T00:33:29.318688+0000","last_peered":"2024-02-18T00:33:29.318688+0000","last_clean":"2024-02-18T00:33:29.318688+0000","last_became_active":"2024-02-18T00:29:14.715835+0000","last_became_peered":"2024-02-18T00:29:14.715835+0000","last_unstale":"2024-02-18T00:33:29.318688+0000","last_undegraded":"2024-02-18T00:33:29.318688+0000","last_fullsized":"2024-02-18T00:33:29.318688+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.998750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":90,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.291907+0000","last_change":"2024-02-18T00:32:58.038986+0000","last_active":"2024-02-18T00:33:29.291907+0000","last_peered":"2024-02-18T00:33:29.291907+0000","last_clean":"2024-02-18T00:33:29.291907+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:33:29.291907+0000","last_undegraded":"2024-02-18T00:33:29.291907+0000","last_fullsized":"2024-02-18T00:33:29.291907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.038886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.038886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":88,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.322514+0000","last_change":"2024-02-18T00:32:36.732383+0000","last_active":"2024-02-18T00:33:29.322514+0000","last_peered":"2024-02-18T00:33:29.322514+0000","last_clean":"2024-02-18T00:33:29.322514+0000","last_became_active":"2024-02-18T00:29:21.243843+0000","last_became_peered":"2024-02-18T00:29:21.243843+0000","last_unstale":"2024-02-18T00:33:29.322514+0000","last_undegraded":"2024-02-18T00:33:29.322514+0000","last_fullsized":"2024-02-18T00:33:29.322514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.732282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.732282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":36,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-02-18T00:33:29.306989+0000","last_change":"2024-02-18T00:33:01.463611+0000","last_active":"2024-02-18T00:33:29.306989+0000","last_peered":"2024-02-18T00:33:29.306989+0000","last_clean":"2024-02-18T00:33:29.306989+0000","last_became_active":"2024-02-18T00:33:01.463209+0000","last_became_peered":"2024-02-18T00:33:01.463209+0000","last_unstale":"2024-02-18T00:33:29.306989+0000","last_undegraded":"2024-02-18T00:33:29.306989+0000","last_fullsized":"2024-02-18T00:33:29.306989+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":512,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":96,"acting":96,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":150998978,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1011,"num_read_kb":49161,"num_write":1043,"num_write_kb":152978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":453267456,"data_stored":452996934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1056,"ondisk_log_size":1056,"up":96,"acting":96,"num_store_stats":8},{"poolid":80,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":79,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":8},{"poolid":78,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":617,"ondisk_log_size":617,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":247,"num_read_kb":242,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":76,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8},{"poolid":51,"num_pg":32,"stat_sum":{"num_bytes":651,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266743,"num_pgs":189,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":335524,"kb_used_data":26140,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93409628,"statfs":{"total":95995035648,"available":95651459072,"internally_reserved":0,"allocated":26767360,"data_stored":26013591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.504,"15min":0.499},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.627,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.42599999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.54900000000000004,"15min":0.54300000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":3.23},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":3.254}]},{"osd":2,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":3.3969999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.60999999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":2.5590000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56899999999999995,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":3.351,"5min":3.351,"15min":3.351},"last":3.5070000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.42899999999999999,"15min":0.41599999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.435,"15min":0.41499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66800000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46100000000000002,"15min":0.44500000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47299999999999998,"15min":0.45800000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72199999999999998,"5min":0.747,"15min":0.747},"last":0.36899999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50800000000000001,"15min":0.49099999999999999},"min":{"1min":0.36799999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.755,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73699999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.40600000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":380616,"kb_used_data":75328,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93364536,"statfs":{"total":95995035648,"available":95605284864,"internally_reserved":0,"allocated":77135872,"data_stored":76347373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51000000000000001,"15min":0.49099999999999999},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66000000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.76800000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.38400000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34300000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.68100000000000005,"5min":0.874,"15min":0.874},"last":5.0590000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.496,"15min":0.48799999999999999},"min":{"1min":0.39700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72199999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":1.8939999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.50700000000000001},"min":{"1min":0.40999999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.64300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73999999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":2.9470000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78800000000000003,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.39300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":1.988}]},{"osd":4,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.437,"15min":0.40699999999999997},"min":{"1min":0.34999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.69899999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.41299999999999998},"min":{"1min":0.41199999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63900000000000001,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.373}]},{"osd":5,"last update":"Sun Feb 18 00:32:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.42099999999999999},"min":{"1min":0.40600000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65100000000000002,"5min":1.121,"15min":1.121},"last":0.315},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.432},"min":{"1min":0.38600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71199999999999997,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.39900000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.57899999999999996,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":5.1230000000000002,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":2.8809999999999998,"5min":1.002,"15min":0.68600000000000005},"min":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":32.985999999999997,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.41499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":185,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":384760,"kb_used_data":75372,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93360392,"statfs":{"total":95995035648,"available":95601041408,"internally_reserved":0,"allocated":77180928,"data_stored":76347157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.39200000000000002,"15min":0.32200000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.20100000000000001},"max":{"1min":1.0780000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":10.606999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.433,"15min":0.34499999999999997},"min":{"1min":0.27500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.9770000000000003,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.34899999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:33:16 2024","interfaces":[{"interface":"back","average":{"1min":3.1800000000000002,"5min":0.94199999999999995,"15min":0.56000000000000005},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":49.201000000000001,"5min":49.201000000000001,"15min":49.201000000000001},"last":9.2530000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":0.60999999999999999,"15min":0.45900000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":17.596,"5min":17.596,"15min":17.596},"last":10.654}]},{"osd":3,"last update":"Sun Feb 18 00:32:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.435,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":9.2859999999999996},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.48399999999999999},"min":{"1min":0.29299999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.121,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":10.688000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1559999999999999,"5min":1.347,"15min":1.347},"last":9.0489999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.2370000000000001,"5min":4.2370000000000001,"15min":4.2370000000000001},"last":9.4019999999999992}]},{"osd":5,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":2.351,"5min":0.95399999999999996,"15min":0.71599999999999997},"min":{"1min":0.378,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":12.611000000000001,"5min":12.611000000000001,"15min":12.611000000000001},"last":9.1679999999999993},{"interface":"front","average":{"1min":2.5870000000000002,"5min":1.0129999999999999,"15min":0.748},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":17.417000000000002,"5min":17.417000000000002,"15min":17.417000000000002},"last":9.0039999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:32:51 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.41599999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":4.3140000000000001},"last":9.4359999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63200000000000001,"15min":0.63300000000000001},"min":{"1min":0.49399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.071,"5min":1.278,"15min":1.278},"last":9.2200000000000006}]},{"osd":7,"last update":"Sun Feb 18 00:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":0.81999999999999995,"15min":0.69499999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.452000000000002,"5min":17.452000000000002,"15min":17.452000000000002},"last":9.0830000000000002},{"interface":"front","average":{"1min":1.3220000000000001,"5min":0.80900000000000005,"15min":0.70499999999999996},"min":{"1min":0.51500000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":12.805,"5min":12.805,"15min":12.805},"last":9.3610000000000007}]}]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":201,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":360148,"kb_used_data":50764,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93385004,"statfs":{"total":95995035648,"available":95626244096,"internally_reserved":0,"allocated":51982336,"data_stored":51184244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.41899999999999998,"15min":0.57499999999999996},"min":{"1min":0.22,"5min":0.22,"15min":0.20699999999999999},"max":{"1min":4.3140000000000001,"5min":4.3140000000000001,"15min":6.7720000000000002},"last":16.844000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.41199999999999998,"15min":0.35699999999999998},"min":{"1min":0.24299999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":3.8220000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":4.9790000000000001,"5min":1.323,"15min":0.69799999999999995},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.215},"max":{"1min":90.850999999999999,"5min":90.850999999999999,"15min":90.850999999999999},"last":3.847},{"interface":"front","average":{"1min":5.0149999999999997,"5min":1.3180000000000001,"15min":0.69499999999999995},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.224},"max":{"1min":80.537000000000006,"5min":80.537000000000006,"15min":80.537000000000006},"last":2.3100000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.29099999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.1339999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":4.8840000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45700000000000002,"15min":0.49299999999999999},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":4.915}]},{"osd":4,"last update":"Sun Feb 18 00:33:01 2024","interfaces":[{"interface":"back","average":{"1min":4.3789999999999996,"5min":1.323,"15min":0.80200000000000005},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":57.914000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":1.405},{"interface":"front","average":{"1min":4.867,"5min":1.444,"15min":0.88800000000000001},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":90.070999999999998,"5min":90.070999999999998,"15min":90.070999999999998},"last":3.673}]},{"osd":5,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":4.5549999999999997,"5min":1.385,"15min":0.85599999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":90.049000000000007,"5min":90.049000000000007,"15min":90.049000000000007},"last":3.7360000000000002},{"interface":"front","average":{"1min":4.9039999999999999,"5min":1.4350000000000001,"15min":0.83799999999999997},"min":{"1min":0.45800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":57.991,"5min":57.991,"15min":57.991},"last":1.3320000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:32:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.439,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":4.8490000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":3.798}]},{"osd":7,"last update":"Sun Feb 18 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":4.8940000000000001,"5min":1.48,"15min":0.89800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":90.025000000000006,"5min":90.025000000000006,"15min":90.025000000000006},"last":3.7679999999999998},{"interface":"front","average":{"1min":4.4119999999999999,"5min":1.359,"15min":0.84499999999999997},"min":{"1min":0.434,"5min":0.37,"15min":0.37},"max":{"1min":57.962000000000003,"5min":57.962000000000003,"15min":57.962000000000003},"last":1.373}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":161,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":360136,"kb_used_data":50748,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93385016,"statfs":{"total":95995035648,"available":95626256384,"internally_reserved":0,"allocated":51965952,"data_stored":51181012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.52500000000000002,"15min":0.41699999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":12.144,"5min":12.144,"15min":12.144},"last":2.1499999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.53200000000000003,"15min":0.41999999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":12.335000000000001,"5min":12.335000000000001,"15min":12.335000000000001},"last":11.324}]},{"osd":1,"last update":"Sun Feb 18 00:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.51800000000000002,"15min":0.42299999999999999},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":9.6489999999999991,"5min":9.6489999999999991,"15min":9.6489999999999991},"last":2.044},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.46500000000000002,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.219,"15min":0.219},"max":{"1min":3.5009999999999999,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":4.5179999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:32:37 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.46700000000000003,"15min":0.45900000000000002},"min":{"1min":0.29799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.91800000000000004,"5min":1.296,"15min":1.296},"last":6.556},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.47999999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":4.3230000000000004}]},{"osd":4,"last update":"Sun Feb 18 00:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.5580000000000001,"5min":1.675,"15min":1.675},"last":4.3520000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96199999999999997,"5min":1.173,"15min":1.173},"last":2.1040000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":0.754,"15min":0.63900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.766,"5min":6.766,"15min":6.766},"last":5.2370000000000001},{"interface":"front","average":{"1min":1.4019999999999999,"5min":0.76300000000000001,"15min":0.67500000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":9.6159999999999997,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":2.129}]},{"osd":6,"last update":"Sun Feb 18 00:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.625,"15min":0.64300000000000002},"min":{"1min":0.44900000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":5.2729999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64300000000000002,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":4.383}]},{"osd":7,"last update":"Sun Feb 18 00:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.68700000000000006,"15min":0.69999999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":5.2969999999999997},{"interface":"front","average":{"1min":1.153,"5min":0.753,"15min":0.70399999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5830000000000002,"5min":9.5830000000000002,"15min":9.5830000000000002},"last":2.0750000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":187,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":356044,"kb_used_data":50756,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93389108,"statfs":{"total":95995035648,"available":95630446592,"internally_reserved":0,"allocated":51974144,"data_stored":51180637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.373,"15min":0.39000000000000001},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.27300000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":6.8700000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40500000000000003,"15min":0.41999999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":6.8479999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.40799999999999997,"15min":0.40699999999999997},"min":{"1min":0.26600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":19.738}]},{"osd":2,"last update":"Sun Feb 18 00:32:35 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.436,"15min":0.438},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":19.681000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42799999999999999,"15min":0.432},"min":{"1min":0.26000000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":1.2190000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.8010000000000002,"5min":3.8010000000000002,"15min":3.8010000000000002},"last":10.131},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.627,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.65300000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.66400000000000003,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":7.4249999999999998,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.2390000000000001,"5min":0.73499999999999999,"15min":0.65500000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.7750000000000004,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":6.7750000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83799999999999997,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":10.189},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.625},"min":{"1min":0.41599999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88600000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.63}]},{"osd":7,"last update":"Sun Feb 18 00:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":6.8220000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.66800000000000004,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.2759999999999998,"5min":6.2759999999999998,"15min":6.2759999999999998},"last":10.164999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":181,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":384788,"kb_used_data":75400,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93360364,"statfs":{"total":95995035648,"available":95601012736,"internally_reserved":0,"allocated":77209600,"data_stored":76352530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.3239999999999998,"5min":6.3239999999999998,"15min":6.3239999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":3.1640000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.752,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2110000000000001,"5min":1.48,"15min":1.48},"last":3.0070000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":3.0510000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.438,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.022,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":3.2959999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.495}]},{"osd":5,"last update":"Sun Feb 18 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41799999999999998,"15min":0.372},"min":{"1min":0.246,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.438,"15min":0.38300000000000001},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.248}]},{"osd":6,"last update":"Sun Feb 18 00:32:52 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.44600000000000001,"15min":0.40500000000000003},"min":{"1min":0.34100000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.71699999999999997,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.48699999999999999,"15min":0.44600000000000001},"min":{"1min":0.371,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.33500000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.747,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.315},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.505,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.28000000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":143,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":356044,"kb_used_data":50756,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":93389108,"statfs":{"total":95995035648,"available":95630446592,"internally_reserved":0,"allocated":51974144,"data_stored":51185714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.49299999999999999,"15min":0.52500000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71899999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.34999999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":0.64000000000000001,"15min":0.58199999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":12.744,"5min":12.744,"15min":12.744},"last":0.75600000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.3369999999999997,"5min":5.3369999999999997,"15min":5.3369999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.879}]},{"osd":2,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.72499999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.53700000000000003,"15min":0.55100000000000005},"min":{"1min":0.33600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67600000000000005,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.94599999999999995}]},{"osd":4,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.54100000000000004,"15min":0.41899999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":11.818,"5min":11.818,"15min":11.818},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.48199999999999998,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.274,"5min":4.274,"15min":4.274},"last":0.40899999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:32:47 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.44900000000000001,"15min":0.41399999999999998},"min":{"1min":0.376,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.621,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.46899999999999997,"15min":0.42199999999999999},"min":{"1min":0.33900000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.751,"15min":0.751},"last":0.437}]},{"osd":7,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.50600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50380800,"data_stored":50335600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":75534336,"data_stored":75497544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50360320,"data_stored":50331696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50364416,"data_stored":50331707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":75558912,"data_stored":75501451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50356224,"data_stored":50335559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":75522048,"data_stored":75497513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25190400,"data_stored":25165864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":11,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":51,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":76,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:33:36.399 INFO:tasks.cephadm.ceph_manager.ceph:too many creating 2024-02-18T00:33:36.425 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.588+0000 7f174e7fc700 1 -- 172.21.15.59:0/1140132757 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v138) v1 ==== 91+0+0 (secure 0 0 0) 0x7f1748051d30 con 0x7f17580692d0 2024-02-18T00:33:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.592+0000 7f175f79a700 10 librados: watch_flush enter 2024-02-18T00:33:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.598+0000 7f175f79a700 10 librados: watch_flush exit 2024-02-18T00:33:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.598+0000 7f175f79a700 20 client.33098.objecter shutdown clearing up homeless session... 2024-02-18T00:33:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.598+0000 7f175f79a700 10 client.33098.objecter successfully canceled tick 2024-02-18T00:33:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.599+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1744062ce0 msgr2=0x7f17440651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.599+0000 7f175f79a700 1 --2- 172.21.15.59:0/1140132757 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1744062ce0 0x7f17440651a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7f1758112860 tx=0x7f1754006c30).stop 2024-02-18T00:33:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.599+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17580692d0 msgr2=0x7f175810cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.599+0000 7f175f79a700 1 --2- 172.21.15.59:0/1140132757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17580692d0 0x7f175810cd60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f1748009b10 tx=0x7f1748007f90).stop 2024-02-18T00:33:36.645 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-18T00:33:36.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.609+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 shutdown_connections 2024-02-18T00:33:36.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.609+0000 7f175f79a700 1 --2- 172.21.15.59:0/1140132757 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1744062ce0 0x7f17440651a0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:36.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.609+0000 7f175f79a700 1 --2- 172.21.15.59:0/1140132757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17581044a0 0x7f17581117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:36.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.609+0000 7f175f79a700 1 --2- 172.21.15.59:0/1140132757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17580692d0 0x7f175810cd60 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:36.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.609+0000 7f175f79a700 1 --2- 172.21.15.59:0/1140132757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1758069ba0 0x7f175810d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:36.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.609+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 >> 172.21.15.59:0/1140132757 conn(0x7f1758063db0 msgr2=0x7f1758105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:36.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.632+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 shutdown_connections 2024-02-18T00:33:36.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.643+0000 7f175f79a700 1 -- 172.21.15.59:0/1140132757 wait complete. 2024-02-18T00:33:36.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:36.645+0000 7f175f79a700 1 librados: shutdown 2024-02-18T00:33:36.672 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:33:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:34.649605+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:34.649672+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.800985+0000 mgr.y (mgr.24454) 185 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.801637+0000 mgr.y (mgr.24454) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.803862+0000 mgr.y (mgr.24454) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.805674+0000 mgr.y (mgr.24454) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.809428+0000 mgr.y (mgr.24454) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.814518+0000 mgr.y (mgr.24454) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.815519+0000 mgr.y (mgr.24454) 191 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.816057+0000 mgr.y (mgr.24454) 192 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.816808+0000 mgr.y (mgr.24454) 193 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:34.817359+0000 mgr.y (mgr.24454) 194 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:34.855047+0000 osd.7 (osd.7) 85 : cluster [DBG] 17.0 deep-scrub starts 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:34.857255+0000 osd.7 (osd.7) 86 : cluster [DBG] 17.0 deep-scrub ok 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:35.270254+0000 mgr.y (mgr.24454) 195 : audit [DBG] from='client.33976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:35.368485+0000 mon.b (mon.1) 149 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:37.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:35.536410+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:34.649605+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:34.649672+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.800985+0000 mgr.y (mgr.24454) 185 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.801637+0000 mgr.y (mgr.24454) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.803862+0000 mgr.y (mgr.24454) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.805674+0000 mgr.y (mgr.24454) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.809428+0000 mgr.y (mgr.24454) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.814518+0000 mgr.y (mgr.24454) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.815519+0000 mgr.y (mgr.24454) 191 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.816057+0000 mgr.y (mgr.24454) 192 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:37.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.816808+0000 mgr.y (mgr.24454) 193 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:34.817359+0000 mgr.y (mgr.24454) 194 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:34.855047+0000 osd.7 (osd.7) 85 : cluster [DBG] 17.0 deep-scrub starts 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:34.857255+0000 osd.7 (osd.7) 86 : cluster [DBG] 17.0 deep-scrub ok 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:35.270254+0000 mgr.y (mgr.24454) 195 : audit [DBG] from='client.33976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:35.368485+0000 mon.b (mon.1) 149 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:35.536410+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:35.586988+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]': finished 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:35.587201+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:35.605776+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.59:0/3967728574' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:35.624514+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:35.800607+0000 mgr.y (mgr.24454) 196 : cluster [DBG] pgmap v193: 470 pgs: 64 creating+peering, 406 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:35.876689+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:35.989203+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.59:0/1140132757' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.247897+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.253005+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.370161+0000 mon.b (mon.1) 151 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:36.572005+0000 mon.a (mon.0) 1108 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.581086+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.581249+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.581311+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:36.581360+0000 mon.a (mon.0) 1112 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.586777+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.59:0/3037089745' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi059-82882-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.586801+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:37.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:36.587653+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:35.586988+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]': finished 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:35.587201+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:35.605776+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.59:0/3967728574' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:35.624514+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:35.800607+0000 mgr.y (mgr.24454) 196 : cluster [DBG] pgmap v193: 470 pgs: 64 creating+peering, 406 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:35.876689+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:35.989203+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.59:0/1140132757' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.247897+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:37.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.253005+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.370161+0000 mon.b (mon.1) 151 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:36.572005+0000 mon.a (mon.0) 1108 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.581086+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.581249+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.581311+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:36.581360+0000 mon.a (mon.0) 1112 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.586777+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.59:0/3037089745' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi059-82882-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.586801+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:37.008 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:36.587653+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:34.649605+0000 osd.0 (osd.0) 79 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:34.649672+0000 osd.0 (osd.0) 80 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.800985+0000 mgr.y (mgr.24454) 185 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.801637+0000 mgr.y (mgr.24454) 186 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.803862+0000 mgr.y (mgr.24454) 187 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.805674+0000 mgr.y (mgr.24454) 188 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.809428+0000 mgr.y (mgr.24454) 189 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.814518+0000 mgr.y (mgr.24454) 190 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.815519+0000 mgr.y (mgr.24454) 191 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-02-18T00:33:37.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.816057+0000 mgr.y (mgr.24454) 192 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.816808+0000 mgr.y (mgr.24454) 193 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:34.817359+0000 mgr.y (mgr.24454) 194 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:34.855047+0000 osd.7 (osd.7) 85 : cluster [DBG] 17.0 deep-scrub starts 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:34.857255+0000 osd.7 (osd.7) 86 : cluster [DBG] 17.0 deep-scrub ok 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:35.270254+0000 mgr.y (mgr.24454) 195 : audit [DBG] from='client.33976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:35.368485+0000 mon.b (mon.1) 149 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:35.536410+0000 mon.a (mon.0) 1102 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:35.586988+0000 mon.a (mon.0) 1103 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-9", "mode": "writeback"}]': finished 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:35.587201+0000 mon.a (mon.0) 1104 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:35.605776+0000 mon.b (mon.1) 150 : audit [INF] from='client.? 172.21.15.59:0/3967728574' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:35.624514+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:35.800607+0000 mgr.y (mgr.24454) 196 : cluster [DBG] pgmap v193: 470 pgs: 64 creating+peering, 406 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:35.876689+0000 mon.c (mon.2) 77 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:37.010 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:35.989203+0000 mon.c (mon.2) 78 : audit [INF] from='client.? 172.21.15.59:0/1140132757' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.247897+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.253005+0000 mon.a (mon.0) 1107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.370161+0000 mon.b (mon.1) 151 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:36.572005+0000 mon.a (mon.0) 1108 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.581086+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi059-82877-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.581249+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.581311+0000 mon.a (mon.0) 1111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:36.581360+0000 mon.a (mon.0) 1112 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.586777+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.59:0/3037089745' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi059-82882-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.586801+0000 mon.c (mon.2) 79 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:37.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:36.587653+0000 mon.a (mon.0) 1114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]: dispatch 2024-02-18T00:33:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:37.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:37.156 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:37.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:37.286 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:37.309 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:37.311 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:37.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:37.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:37.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:37.694 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:37.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:37.795 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:37.824 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:37.845 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:37.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:37.847 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:37.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:38.052 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:38.096 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:38.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:38.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:38.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:38.263 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:36.622281+0000 mon.a (mon.0) 1115 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:36.841762+0000 osd.3 (osd.3) 95 : cluster [DBG] 17.9 deep-scrub starts 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:36.855944+0000 osd.3 (osd.3) 96 : cluster [DBG] 17.9 deep-scrub ok 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:37.371361+0000 mon.b (mon.1) 152 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.580974+0000 mon.a (mon.0) 1116 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:37.592164+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? 172.21.15.59:0/3037089745' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi059-82882-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:37.592233+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]': finished 2024-02-18T00:33:38.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.592445+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-18T00:33:38.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:36.622281+0000 mon.a (mon.0) 1115 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:38.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:36.841762+0000 osd.3 (osd.3) 95 : cluster [DBG] 17.9 deep-scrub starts 2024-02-18T00:33:38.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:36.855944+0000 osd.3 (osd.3) 96 : cluster [DBG] 17.9 deep-scrub ok 2024-02-18T00:33:38.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:37.371361+0000 mon.b (mon.1) 152 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:38.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.580974+0000 mon.a (mon.0) 1116 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:38.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:37.592164+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? 172.21.15.59:0/3037089745' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi059-82882-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:38.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:37.592233+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]': finished 2024-02-18T00:33:38.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.592445+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-18T00:33:38.335 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:38.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:38.344 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:38.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:38.622 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:36.622281+0000 mon.a (mon.0) 1115 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:38.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:36.841762+0000 osd.3 (osd.3) 95 : cluster [DBG] 17.9 deep-scrub starts 2024-02-18T00:33:38.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:36.855944+0000 osd.3 (osd.3) 96 : cluster [DBG] 17.9 deep-scrub ok 2024-02-18T00:33:38.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:37.371361+0000 mon.b (mon.1) 152 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:38.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.580974+0000 mon.a (mon.0) 1116 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:38.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:37.592164+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? 172.21.15.59:0/3037089745' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi059-82882-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:38.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:37.592233+0000 mon.a (mon.0) 1118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-9"}]': finished 2024-02-18T00:33:38.623 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.592445+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-18T00:33:38.709 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:38.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:38.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:38.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:38.935 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:33:38.935 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:33:38.935 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:33:38.935 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:33:38.935 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:33:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:39.241 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:39.251 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:39.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:39.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:39.481 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:39.672 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.207618+0000 osd.4 (osd.4) 97 : cluster [DBG] 17.2 deep-scrub starts 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.215944+0000 osd.4 (osd.4) 98 : cluster [DBG] 17.2 deep-scrub ok 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.801118+0000 mgr.y (mgr.24454) 197 : cluster [DBG] pgmap v196: 406 pgs: 32 unknown, 374 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.806524+0000 osd.2 (osd.2) 67 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.808916+0000 osd.2 (osd.2) 68 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.955000+0000 osd.0 (osd.0) 81 : cluster [DBG] 17.1 deep-scrub starts 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:37.962273+0000 osd.0 (osd.0) 82 : cluster [DBG] 17.1 deep-scrub ok 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:38.379059+0000 mon.b (mon.1) 153 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:38.897769+0000 osd.3 (osd.3) 97 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:38.914680+0000 osd.3 (osd.3) 98 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:38.976809+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-18T00:33:39.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:38.997130+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.59:0/3438207028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi059-82877-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.207618+0000 osd.4 (osd.4) 97 : cluster [DBG] 17.2 deep-scrub starts 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.215944+0000 osd.4 (osd.4) 98 : cluster [DBG] 17.2 deep-scrub ok 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.801118+0000 mgr.y (mgr.24454) 197 : cluster [DBG] pgmap v196: 406 pgs: 32 unknown, 374 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.806524+0000 osd.2 (osd.2) 67 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.808916+0000 osd.2 (osd.2) 68 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.955000+0000 osd.0 (osd.0) 81 : cluster [DBG] 17.1 deep-scrub starts 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:37.962273+0000 osd.0 (osd.0) 82 : cluster [DBG] 17.1 deep-scrub ok 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:38.379059+0000 mon.b (mon.1) 153 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:38.897769+0000 osd.3 (osd.3) 97 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:38.914680+0000 osd.3 (osd.3) 98 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:38.976809+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-18T00:33:39.678 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:38.997130+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.59:0/3438207028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi059-82877-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:39.679 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:39.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:39.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.207618+0000 osd.4 (osd.4) 97 : cluster [DBG] 17.2 deep-scrub starts 2024-02-18T00:33:39.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.215944+0000 osd.4 (osd.4) 98 : cluster [DBG] 17.2 deep-scrub ok 2024-02-18T00:33:39.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.801118+0000 mgr.y (mgr.24454) 197 : cluster [DBG] pgmap v196: 406 pgs: 32 unknown, 374 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-02-18T00:33:39.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.806524+0000 osd.2 (osd.2) 67 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:33:39.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.808916+0000 osd.2 (osd.2) 68 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:33:39.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.955000+0000 osd.0 (osd.0) 81 : cluster [DBG] 17.1 deep-scrub starts 2024-02-18T00:33:39.698 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:37.962273+0000 osd.0 (osd.0) 82 : cluster [DBG] 17.1 deep-scrub ok 2024-02-18T00:33:39.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:38.379059+0000 mon.b (mon.1) 153 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:39.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:38.897769+0000 osd.3 (osd.3) 97 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:33:39.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:38.914680+0000 osd.3 (osd.3) 98 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:33:39.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:38.976809+0000 mon.a (mon.0) 1120 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-18T00:33:39.699 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:38.997130+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.59:0/3438207028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi059-82877-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:39.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-18T00:33:39.781 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:39.810 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:39.834 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:39.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:39.837 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.884+0000 7f740e793700 1 -- 172.21.15.59:0/1908929674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7408069d00 msgr2=0x7f740810d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.884+0000 7f740e793700 1 --2- 172.21.15.59:0/1908929674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7408069d00 0x7f740810d070 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7f73fc0047b0 tx=0x7f73fc009cb0).stop 2024-02-18T00:33:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.884+0000 7f740e793700 1 -- 172.21.15.59:0/1908929674 shutdown_connections 2024-02-18T00:33:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.884+0000 7f740e793700 1 --2- 172.21.15.59:0/1908929674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74081001f0 0x7f74081005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.884+0000 7f740e793700 1 --2- 172.21.15.59:0/1908929674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7408069d00 0x7f740810d070 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.884+0000 7f740e793700 1 --2- 172.21.15.59:0/1908929674 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7408069390 0x7f7408069810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:39.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.884+0000 7f740e793700 1 -- 172.21.15.59:0/1908929674 >> 172.21.15.59:0/1908929674 conn(0x7f7408063db0 msgr2=0x7f7408067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:39.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.891+0000 7f740e793700 1 -- 172.21.15.59:0/1908929674 shutdown_connections 2024-02-18T00:33:39.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.893+0000 7f740e793700 1 -- 172.21.15.59:0/1908929674 wait complete. 2024-02-18T00:33:39.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.893+0000 7f740e793700 1 librados: starting msgr at 2024-02-18T00:33:39.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.894+0000 7f740e793700 1 librados: starting objecter 2024-02-18T00:33:39.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.894+0000 7f740e793700 1 Processor -- start 2024-02-18T00:33:39.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 -- start start 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 librados: setting wanted keys 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 librados: calling monclient init 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7408069390 0x7f7408102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7408069d00 0x7f7408100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74081001f0 0x7f74081015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7408101ca0 con 0x7f74081001f0 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7408101e10 con 0x7f7408069390 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f740e793700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7408114b60 con 0x7f7408069d00 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.896+0000 7f7407fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7408069390 0x7f7408102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74077fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7408069d00 0x7f7408100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f7407fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7408069390 0x7f7408102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:43862/0 (socket says 172.21.15.59:43862) 2024-02-18T00:33:39.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f7407fff700 1 -- 172.21.15.59:0/222508900 learned_addr learned my addr 172.21.15.59:0/222508900 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:39.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f740cd30700 1 --2- 172.21.15.59:0/222508900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74081001f0 0x7f74081015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74077fe700 1 -- 172.21.15.59:0/222508900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7408069390 msgr2=0x7f7408102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74077fe700 1 --2- 172.21.15.59:0/222508900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7408069390 0x7f7408102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74077fe700 1 -- 172.21.15.59:0/222508900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74081001f0 msgr2=0x7f74081015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74077fe700 1 --2- 172.21.15.59:0/222508900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74081001f0 0x7f74081015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74077fe700 1 -- 172.21.15.59:0/222508900 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc005040 con 0x7f7408069d00 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74077fe700 1 --2- 172.21.15.59:0/222508900 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7408069d00 0x7f7408100fb0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f73f8009a00 tx=0x7f73f800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74057fa700 10 client.?.objecter ms_handle_connect 0x7f7408069d00 2024-02-18T00:33:39.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.897+0000 7f74057fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.898+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8004630 con 0x7f7408069d00 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.898+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f800f040 con 0x7f7408069d00 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.898+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8014830 con 0x7f7408069d00 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.902+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7408114e40 con 0x7f7408069d00 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.902+0000 7f740e793700 10 client.35294.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.902+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74081152b0 con 0x7f7408069d00 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.902+0000 7f740e793700 1 librados: init done 2024-02-18T00:33:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.906+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f7408060280 con 0x7f7408069d00 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.910+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f73f80092e0 con 0x7f7408069d00 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.911+0000 7f74057fa700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6800/3721891797 conn(0x7f73f0062d20 0x7f73f00651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.911+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(141..141 src has 44..141) v4 ==== 10372+0+0 (secure 0 0 0) 0x7f73f807d930 con 0x7f7408069d00 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.911+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 osd_map(141..141 src has 44..141) v4 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.911+0000 7f74057fa700 3 client.35294.objecter handle_osd_map got epochs [141,141] > 0 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.911+0000 7f74057fa700 3 client.35294.objecter handle_osd_map decoding full epoch 141 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.911+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6800/3721891797 conn(0x7f73f0062d20 0x7f73f00651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:39.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.911+0000 7f74057fa700 20 client.35294.objecter dump_active .. 0 homeless 2024-02-18T00:33:39.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.913+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f73f0062d20 2024-02-18T00:33:39.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:39.913+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6800/3721891797 conn(0x7f73f0062d20 0x7f73f00651e0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 rx=0x7f7400004550 tx=0x7f7400004e00).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:39.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:39.988 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:40.012 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:40.054 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:40.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:40.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:40.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:33:40.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:40.265 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:40.295 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:40.297 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:40.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:40.398 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:40.426 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:40.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.486+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v141) v1 ==== 54+0+16 (secure 0 0 0) 0x7f73f80505a0 con 0x7f7408069d00 2024-02-18T00:33:40.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 10 client.35294.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 1 2024-02-18T00:33:40.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74080598f0 0x7f7408059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 1 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 0 1 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740810f8b0 osd=0 4 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 10 client.35294.objecter _send_command 1 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.488+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f740805ee80 con 0x7f74080598f0 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.491+0000 7f740cd30700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74080598f0 0x7f7408059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.491+0000 7f740cd30700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74080598f0 0x7f7408059cd0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.491+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f74080598f0 2024-02-18T00:33:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.500+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f740805ee80 con 0x7f74080598f0 2024-02-18T00:33:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.501+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:33:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.501+0000 7f74057fa700 10 client.35294.objecter _finish_command 1 = system:0 2024-02-18T00:33:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.501+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740810f8b0 osd=0 4 2024-02-18T00:33:40.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.501+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 0 1 2024-02-18T00:33:40.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:39.284884+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? 172.21.15.59:0/3438207028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi059-82877-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:40.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:39.284953+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-18T00:33:40.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:39.297075+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:40.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:39.388032+0000 mon.b (mon.1) 155 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:40.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:39.802827+0000 mgr.y (mgr.24454) 198 : cluster [DBG] pgmap v199: 406 pgs: 64 unknown, 342 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:40.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:39.876932+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:40.522 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:39.284884+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? 172.21.15.59:0/3438207028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi059-82877-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:40.522 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:39.284953+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-18T00:33:40.522 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:39.297075+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:40.522 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:39.388032+0000 mon.b (mon.1) 155 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:40.522 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:39.802827+0000 mgr.y (mgr.24454) 198 : cluster [DBG] pgmap v199: 406 pgs: 64 unknown, 342 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:40.522 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:39.876932+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:40.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 10 client.35294.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 2 2024-02-18T00:33:40.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 2 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740810f8b0 osd=0 3 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 0 2 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740810f8b0 osd=0 4 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 10 client.35294.objecter _send_command 2 2024-02-18T00:33:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.532+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74080273c0 con 0x7f74080598f0 2024-02-18T00:33:40.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.565+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74080273c0 con 0x7f74080598f0 2024-02-18T00:33:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.565+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.565+0000 7f74057fa700 10 client.35294.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.565+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740810f8b0 osd=0 4 2024-02-18T00:33:40.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.565+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 0 2 2024-02-18T00:33:40.573 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:33:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 10 client.35294.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 3 2024-02-18T00:33:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6805/26279110 conn(0x7f7408107c50 0x7f7408108030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 3 2024-02-18T00:33:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 1 3 2024-02-18T00:33:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408028710 osd=1 4 2024-02-18T00:33:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 10 client.35294.objecter _send_command 3 2024-02-18T00:33:40.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.574+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f740811bc50 con 0x7f7408107c50 2024-02-18T00:33:40.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.590+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6805/26279110 conn(0x7f7408107c50 0x7f7408108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.593+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6805/26279110 conn(0x7f7408107c50 0x7f7408108030 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.593+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f7408107c50 2024-02-18T00:33:40.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.598+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f740811bc50 con 0x7f7408107c50 2024-02-18T00:33:40.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.598+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:33:40.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.598+0000 7f74057fa700 10 client.35294.objecter _finish_command 3 = system:0 2024-02-18T00:33:40.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.598+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f7408028710 osd=1 4 2024-02-18T00:33:40.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.598+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 1 3 2024-02-18T00:33:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 10 client.35294.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 4 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 4 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f7408028710 osd=1 3 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 1 4 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408028710 osd=1 4 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 10 client.35294.objecter _send_command 4 2024-02-18T00:33:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.623+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f740811cff0 con 0x7f7408107c50 2024-02-18T00:33:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f740811cff0 con 0x7f7408107c50 2024-02-18T00:33:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f74057fa700 10 client.35294.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f7408028710 osd=1 4 2024-02-18T00:33:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 1 4 2024-02-18T00:33:40.632 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:33:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 10 client.35294.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 5 2024-02-18T00:33:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7408028270 0x7f740811b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 5 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 2 5 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740811d1c0 osd=2 4 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 10 client.35294.objecter _send_command 5 2024-02-18T00:33:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.630+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f740811b9f0 con 0x7f7408028270 2024-02-18T00:33:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.633+0000 7f74077fe700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7408028270 0x7f740811b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.635+0000 7f74077fe700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7408028270 0x7f740811b500 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.636+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f7408028270 2024-02-18T00:33:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.638+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f740811b9f0 con 0x7f7408028270 2024-02-18T00:33:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.638+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:33:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.638+0000 7f74057fa700 10 client.35294.objecter _finish_command 5 = system:0 2024-02-18T00:33:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.638+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740811d1c0 osd=2 4 2024-02-18T00:33:40.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.638+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 2 5 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 10 client.35294.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 6 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 6 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740811d1c0 osd=2 3 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 2 6 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740811d1c0 osd=2 4 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 10 client.35294.objecter _send_command 6 2024-02-18T00:33:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.668+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7408134380 con 0x7f7408028270 2024-02-18T00:33:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.672+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7408134380 con 0x7f7408028270 2024-02-18T00:33:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.672+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.672+0000 7f74057fa700 10 client.35294.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.672+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740811d1c0 osd=2 4 2024-02-18T00:33:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.672+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 2 6 2024-02-18T00:33:40.678 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:33:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 10 client.35294.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 7 2024-02-18T00:33:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7408027940 0x7f74081376e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 7 2024-02-18T00:33:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 3 7 2024-02-18T00:33:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f74080277d0 osd=3 4 2024-02-18T00:33:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 10 client.35294.objecter _send_command 7 2024-02-18T00:33:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.677+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f740811cb50 con 0x7f7408027940 2024-02-18T00:33:40.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.678+0000 7f740cd30700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7408027940 0x7f74081376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.678+0000 7f740cd30700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7408027940 0x7f74081376e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.678+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f7408027940 2024-02-18T00:33:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.680+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f740811cb50 con 0x7f7408027940 2024-02-18T00:33:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.680+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:33:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.680+0000 7f74057fa700 10 client.35294.objecter _finish_command 7 = system:0 2024-02-18T00:33:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.680+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f74080277d0 osd=3 4 2024-02-18T00:33:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.680+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 3 7 2024-02-18T00:33:40.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 10 client.35294.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 8 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 8 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f74080277d0 osd=3 3 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 3 8 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f74080277d0 osd=3 4 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 10 client.35294.objecter _send_command 8 2024-02-18T00:33:40.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.705+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7408137bd0 con 0x7f7408027940 2024-02-18T00:33:40.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.708+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7408137bd0 con 0x7f7408027940 2024-02-18T00:33:40.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.708+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.708+0000 7f74057fa700 10 client.35294.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.708+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f74080277d0 osd=3 4 2024-02-18T00:33:40.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.708+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 3 8 2024-02-18T00:33:40.711 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:33:40.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 10 client.35294.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 9 2024-02-18T00:33:40.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7408138250 0x7f740813a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 9 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 4 9 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408137bd0 osd=4 4 2024-02-18T00:33:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 10 client.35294.objecter _send_command 9 2024-02-18T00:33:40.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.714+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f740813d300 con 0x7f7408138250 2024-02-18T00:33:40.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.717+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7408138250 0x7f740813a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.723+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7408138250 0x7f740813a640 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.723+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f7408138250 2024-02-18T00:33:40.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.724+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f740813d300 con 0x7f7408138250 2024-02-18T00:33:40.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.724+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:33:40.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.724+0000 7f74057fa700 10 client.35294.objecter _finish_command 9 = system:0 2024-02-18T00:33:40.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.724+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f7408137bd0 osd=4 4 2024-02-18T00:33:40.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.724+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 4 9 2024-02-18T00:33:40.736 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:40.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 10 client.35294.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 10 2024-02-18T00:33:40.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 10 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f7408137bd0 osd=4 3 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 4 10 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408137bd0 osd=4 4 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 10 client.35294.objecter _send_command 10 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.761+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f740813d300 con 0x7f7408138250 2024-02-18T00:33:40.765 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:33:40.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.763+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f740813d300 con 0x7f7408138250 2024-02-18T00:33:40.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.763+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.763+0000 7f74057fa700 10 client.35294.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.763+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f7408137bd0 osd=4 4 2024-02-18T00:33:40.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.763+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 4 10 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 10 client.35294.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 11 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6804/2170432263 conn(0x7f740813b080 0x7f740813b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 11 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 5 11 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813d300 osd=5 4 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 10 client.35294.objecter _send_command 11 2024-02-18T00:33:40.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.767+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74081406b0 con 0x7f740813b080 2024-02-18T00:33:40.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.768+0000 7f74077fe700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6804/2170432263 conn(0x7f740813b080 0x7f740813b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.775 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:40.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:39.284884+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? 172.21.15.59:0/3438207028' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi059-82877-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:40.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:39.284953+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-18T00:33:40.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:39.297075+0000 mon.b (mon.1) 154 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:40.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:39.388032+0000 mon.b (mon.1) 155 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:40.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:39.802827+0000 mgr.y (mgr.24454) 198 : cluster [DBG] pgmap v199: 406 pgs: 64 unknown, 342 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T00:33:40.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:39.876932+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:40.778 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:40.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:40.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.775+0000 7f74077fe700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6804/2170432263 conn(0x7f740813b080 0x7f740813b4e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.775+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f740813b080 2024-02-18T00:33:40.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.783+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74081406b0 con 0x7f740813b080 2024-02-18T00:33:40.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.783+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:33:40.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.783+0000 7f74057fa700 10 client.35294.objecter _finish_command 11 = system:0 2024-02-18T00:33:40.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.783+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740813d300 osd=5 4 2024-02-18T00:33:40.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.783+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 5 11 2024-02-18T00:33:40.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 10 client.35294.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 12 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 12 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740813d300 osd=5 3 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 5 12 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813d300 osd=5 4 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 10 client.35294.objecter _send_command 12 2024-02-18T00:33:40.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.808+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f740813bac0 con 0x7f740813b080 2024-02-18T00:33:40.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.810+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f740813bac0 con 0x7f740813b080 2024-02-18T00:33:40.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.810+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.810+0000 7f74057fa700 10 client.35294.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.810+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740813d300 osd=5 4 2024-02-18T00:33:40.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.810+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 5 12 2024-02-18T00:33:40.819 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:33:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 10 client.35294.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 13 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6808/3356009390 conn(0x7f740813c140 0x7f740813c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 13 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 6 13 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813bac0 osd=6 4 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 10 client.35294.objecter _send_command 13 2024-02-18T00:33:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.821+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f740813cc00 con 0x7f740813c140 2024-02-18T00:33:40.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.833+0000 7f740cd30700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6808/3356009390 conn(0x7f740813c140 0x7f740813c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.833+0000 7f740cd30700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6808/3356009390 conn(0x7f740813c140 0x7f740813c560 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.833+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f740813c140 2024-02-18T00:33:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.835+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f740813cc00 con 0x7f740813c140 2024-02-18T00:33:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.835+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:33:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.835+0000 7f74057fa700 10 client.35294.objecter _finish_command 13 = system:0 2024-02-18T00:33:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.835+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740813bac0 osd=6 4 2024-02-18T00:33:40.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.835+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 6 13 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 10 client.35294.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 14 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 14 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740813bac0 osd=6 3 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 6 14 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813bac0 osd=6 4 2024-02-18T00:33:40.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 10 client.35294.objecter _send_command 14 2024-02-18T00:33:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.870+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f740813cc00 con 0x7f740813c140 2024-02-18T00:33:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.872+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f740813cc00 con 0x7f740813c140 2024-02-18T00:33:40.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.872+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.872+0000 7f74057fa700 10 client.35294.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.872+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740813bac0 osd=6 4 2024-02-18T00:33:40.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.872+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 6 14 2024-02-18T00:33:40.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:40.888 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:33:40.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 10 client.35294.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 15 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6812/53025171 conn(0x7f7408141a50 0x7f7408143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 15 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 7 15 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813cc00 osd=7 4 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 10 client.35294.objecter _send_command 15 2024-02-18T00:33:40.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.887+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7408146a90 con 0x7f7408141a50 2024-02-18T00:33:40.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.891+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6812/53025171 conn(0x7f7408141a50 0x7f7408143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.896+0000 7f7407fff700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6812/53025171 conn(0x7f7408141a50 0x7f7408143e40 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:40.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.896+0000 7f74057fa700 10 client.35294.objecter ms_handle_connect 0x7f7408141a50 2024-02-18T00:33:40.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.897+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7408146a90 con 0x7f7408141a50 2024-02-18T00:33:40.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.897+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:33:40.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.897+0000 7f74057fa700 10 client.35294.objecter _finish_command 15 = system:0 2024-02-18T00:33:40.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.897+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740813cc00 osd=7 4 2024-02-18T00:33:40.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.897+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 7 15 2024-02-18T00:33:40.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 10 client.35294.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:33:40.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign -1 16 2024-02-18T00:33:40.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 20 client.35294.objecter _get_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 15 client.35294.objecter _session_command_op_remove -1 16 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 20 client.35294.objecter get_session s=0x7f740813cc00 osd=7 3 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 15 client.35294.objecter _session_command_op_assign 7 16 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813cc00 osd=7 4 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 10 client.35294.objecter _send_command 16 2024-02-18T00:33:40.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.927+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7408144330 con 0x7f7408141a50 2024-02-18T00:33:40.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.933+0000 7f74057fa700 1 -- 172.21.15.59:0/222508900 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7408144330 con 0x7f7408141a50 2024-02-18T00:33:40.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.933+0000 7f74057fa700 10 client.35294.objecter ms_dispatch 0x7f7408001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:33:40.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.933+0000 7f74057fa700 10 client.35294.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.933+0000 7f74057fa700 20 client.35294.objecter put_session s=0x7f740813cc00 osd=7 4 2024-02-18T00:33:40.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.933+0000 7f74057fa700 15 client.35294.objecter _session_command_op_remove 7 16 2024-02-18T00:33:40.935 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 librados: watch_flush enter 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 librados: watch_flush exit 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.0 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74080598f0 msgr2=0x7f7408059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74080598f0 0x7f7408059cd0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740810f8b0 osd=0 1 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.1 2024-02-18T00:33:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6805/26279110 conn(0x7f7408107c50 msgr2=0x7f7408108030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6805/26279110 conn(0x7f7408107c50 0x7f7408108030 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408028710 osd=1 1 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.2 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7408028270 msgr2=0x7f740811b500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7408028270 0x7f740811b500 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740811d1c0 osd=2 1 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.3 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7408027940 msgr2=0x7f74081376e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7408027940 0x7f74081376e0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f74080277d0 osd=3 1 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.4 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7408138250 msgr2=0x7f740813a640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7408138250 0x7f740813a640 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f7408137bd0 osd=4 1 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.5 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6804/2170432263 conn(0x7f740813b080 msgr2=0x7f740813b4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6804/2170432263 conn(0x7f740813b080 0x7f740813b4e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813d300 osd=5 1 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.6 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6808/3356009390 conn(0x7f740813c140 msgr2=0x7f740813c560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6808/3356009390 conn(0x7f740813c140 0x7f740813c560 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813bac0 osd=6 1 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter close_session for osd.7 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6812/53025171 conn(0x7f7408141a50 msgr2=0x7f7408143e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.137:6812/53025171 conn(0x7f7408141a50 0x7f7408143e40 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter put_session s=0x7f740813cc00 osd=7 1 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 20 client.35294.objecter shutdown clearing up homeless session... 2024-02-18T00:33:40.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740e793700 10 client.35294.objecter successfully canceled tick 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740cd30700 1 -- 172.21.15.59:0/222508900 reap_dead start 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740cd30700 1 -- 172.21.15.59:0/222508900 reap_dead start 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740cd30700 1 -- 172.21.15.59:0/222508900 reap_dead start 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740cd30700 1 -- 172.21.15.59:0/222508900 reap_dead start 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.938+0000 7f740cd30700 1 -- 172.21.15.59:0/222508900 reap_dead start 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.939+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6800/3721891797 conn(0x7f73f0062d20 msgr2=0x7f73f00651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.939+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6800/3721891797 conn(0x7f73f0062d20 0x7f73f00651e0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 rx=0x7f7400004550 tx=0x7f7400004e00).stop 2024-02-18T00:33:40.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.939+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7408069d00 msgr2=0x7f7408100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.939+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7408069d00 0x7f7408100fb0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f73f8009a00 tx=0x7f73f800b040).stop 2024-02-18T00:33:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.940+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 shutdown_connections 2024-02-18T00:33:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.940+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7408069d00 0x7f7408100fb0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.940+0000 7f740e793700 1 --2- 172.21.15.59:0/222508900 >> v2:172.21.15.59:6800/3721891797 conn(0x7f73f0062d20 0x7f73f00651e0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.940+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 >> 172.21.15.59:0/222508900 conn(0x7f7408063db0 msgr2=0x7f7408105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:40.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.947+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 shutdown_connections 2024-02-18T00:33:40.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.948+0000 7f740e793700 1 -- 172.21.15.59:0/222508900 wait complete. 2024-02-18T00:33:40.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:40.949+0000 7f740e793700 1 librados: shutdown 2024-02-18T00:33:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.969 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:33:40.970 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:33:41.011 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:41.061 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:33:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:33:41.062 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:33:41.132 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:41.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:41.139 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:41.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:41.373 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:41.401 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:33:41.401 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:33:41.401 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:33:41.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd out 7 2024-02-18T00:33:41.415 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:41.415 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:41.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:41.516 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:41.546 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:41.599 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:41.602 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:41.602 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:41.703 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:41.732 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:41.751 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:41.754 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:41.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:41.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:38.821748+0000 osd.7 (osd.7) 87 : cluster [DBG] 17.8 deep-scrub starts 2024-02-18T00:33:41.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:38.826586+0000 osd.7 (osd.7) 88 : cluster [DBG] 17.8 deep-scrub ok 2024-02-18T00:33:41.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:39.047799+0000 osd.5 (osd.5) 77 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:33:41.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:39.051647+0000 osd.5 (osd.5) 78 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:33:41.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:39.910997+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? 172.21.15.59:0/222508900' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:41.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:39.968857+0000 osd.0 (osd.0) 83 : cluster [DBG] 17.4 deep-scrub starts 2024-02-18T00:33:41.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.052424+0000 osd.0 (osd.0) 84 : cluster [DBG] 17.4 deep-scrub ok 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.090382+0000 osd.5 (osd.5) 79 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.095419+0000 osd.5 (osd.5) 80 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:40.395647+0000 mon.b (mon.1) 156 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:40.481592+0000 mon.a (mon.0) 1125 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:40.506277+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.506794+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:40.533264+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.59:0/2802807173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:40.533763+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:40.542794+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:41.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.089565+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:41.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:41.917 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:33:41.917 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:33:41.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:38.821748+0000 osd.7 (osd.7) 87 : cluster [DBG] 17.8 deep-scrub starts 2024-02-18T00:33:41.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:38.826586+0000 osd.7 (osd.7) 88 : cluster [DBG] 17.8 deep-scrub ok 2024-02-18T00:33:41.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:39.047799+0000 osd.5 (osd.5) 77 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:33:41.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:39.051647+0000 osd.5 (osd.5) 78 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:33:41.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:39.910997+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? 172.21.15.59:0/222508900' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:41.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:39.968857+0000 osd.0 (osd.0) 83 : cluster [DBG] 17.4 deep-scrub starts 2024-02-18T00:33:41.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.052424+0000 osd.0 (osd.0) 84 : cluster [DBG] 17.4 deep-scrub ok 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.090382+0000 osd.5 (osd.5) 79 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.095419+0000 osd.5 (osd.5) 80 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:40.395647+0000 mon.b (mon.1) 156 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:40.481592+0000 mon.a (mon.0) 1125 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:40.506277+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.506794+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:40.533264+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.59:0/2802807173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:40.533763+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:40.542794+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:41.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.089565+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:38.821748+0000 osd.7 (osd.7) 87 : cluster [DBG] 17.8 deep-scrub starts 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:38.826586+0000 osd.7 (osd.7) 88 : cluster [DBG] 17.8 deep-scrub ok 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:39.047799+0000 osd.5 (osd.5) 77 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:39.051647+0000 osd.5 (osd.5) 78 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:39.910997+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? 172.21.15.59:0/222508900' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:39.968857+0000 osd.0 (osd.0) 83 : cluster [DBG] 17.4 deep-scrub starts 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.052424+0000 osd.0 (osd.0) 84 : cluster [DBG] 17.4 deep-scrub ok 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.090382+0000 osd.5 (osd.5) 79 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.095419+0000 osd.5 (osd.5) 80 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:40.395647+0000 mon.b (mon.1) 156 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:40.481592+0000 mon.a (mon.0) 1125 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:42.086 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:40.506277+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82965-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:42.087 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.506794+0000 mon.a (mon.0) 1127 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-18T00:33:42.087 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:40.533264+0000 mon.b (mon.1) 157 : audit [INF] from='client.? 172.21.15.59:0/2802807173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:42.087 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:40.533763+0000 mon.a (mon.0) 1128 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:42.087 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:40.542794+0000 mon.c (mon.2) 81 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:42.087 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.089565+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:42.097 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:42.125 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:42.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:42.226 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:42.254 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:42.276 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:42.284 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:42.284 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:42.385 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:42.412 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:42.503 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:42] "GET /metrics HTTP/1.1" 200 199674 "" "Prometheus/2.33.4" 2024-02-18T00:33:42.609 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:42.612 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:42.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:42.626 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-18T00:33:42.714 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:42.858 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:42.970 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:39.800458+0000 osd.7 (osd.7) 89 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:39.801496+0000 osd.7 (osd.7) 90 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.823322+0000 osd.2 (osd.2) 69 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.834871+0000 osd.2 (osd.2) 70 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.949267+0000 osd.0 (osd.0) 85 : cluster [DBG] 17.7 deep-scrub starts 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:40.951362+0000 osd.0 (osd.0) 86 : cluster [DBG] 17.7 deep-scrub ok 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:40.960247+0000 mgr.y (mgr.24454) 199 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.054482+0000 osd.5 (osd.5) 81 : cluster [DBG] 6.d scrub starts 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.057656+0000 osd.5 (osd.5) 82 : cluster [DBG] 6.d scrub ok 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.140798+0000 osd.1 (osd.1) 87 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:33:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.153860+0000 osd.1 (osd.1) 88 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.321832+0000 osd.4 (osd.4) 99 : cluster [DBG] 17.3 deep-scrub starts 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.409272+0000 mon.b (mon.1) 158 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.446433+0000 osd.4 (osd.4) 100 : cluster [DBG] 17.3 deep-scrub ok 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.735763+0000 mon.a (mon.0) 1130 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.762682+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.762819+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.762972+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.772818+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.59:0/3352845223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.780147+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.794917+0000 mon.b (mon.1) 160 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.803489+0000 mgr.y (mgr.24454) 200 : cluster [DBG] pgmap v202: 502 pgs: 64 unknown, 64 creating+peering, 1 active+clean+snaptrim, 373 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.824689+0000 osd.7 (osd.7) 91 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:41.826178+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T00:33:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.843887+0000 osd.7 (osd.7) 92 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:33:43.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.861471+0000 osd.3 (osd.3) 99 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:33:43.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.866524+0000 osd.3 (osd.3) 100 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:33:43.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.421894+0000 mon.b (mon.1) 161 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:43.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.774518+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.774654+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]': finished 2024-02-18T00:33:43.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:42.774797+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-18T00:33:43.173 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:43.186 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:39.800458+0000 osd.7 (osd.7) 89 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:39.801496+0000 osd.7 (osd.7) 90 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.823322+0000 osd.2 (osd.2) 69 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.834871+0000 osd.2 (osd.2) 70 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.949267+0000 osd.0 (osd.0) 85 : cluster [DBG] 17.7 deep-scrub starts 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:40.951362+0000 osd.0 (osd.0) 86 : cluster [DBG] 17.7 deep-scrub ok 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:40.960247+0000 mgr.y (mgr.24454) 199 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:43.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.054482+0000 osd.5 (osd.5) 81 : cluster [DBG] 6.d scrub starts 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.057656+0000 osd.5 (osd.5) 82 : cluster [DBG] 6.d scrub ok 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.140798+0000 osd.1 (osd.1) 87 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.153860+0000 osd.1 (osd.1) 88 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.321832+0000 osd.4 (osd.4) 99 : cluster [DBG] 17.3 deep-scrub starts 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.409272+0000 mon.b (mon.1) 158 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.446433+0000 osd.4 (osd.4) 100 : cluster [DBG] 17.3 deep-scrub ok 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.735763+0000 mon.a (mon.0) 1130 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.762682+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.762819+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.762972+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.772818+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.59:0/3352845223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.780147+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:43.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.794917+0000 mon.b (mon.1) 160 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.803489+0000 mgr.y (mgr.24454) 200 : cluster [DBG] pgmap v202: 502 pgs: 64 unknown, 64 creating+peering, 1 active+clean+snaptrim, 373 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.824689+0000 osd.7 (osd.7) 91 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:41.826178+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.843887+0000 osd.7 (osd.7) 92 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.861471+0000 osd.3 (osd.3) 99 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.866524+0000 osd.3 (osd.3) 100 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.421894+0000 mon.b (mon.1) 161 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.774518+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.774654+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]': finished 2024-02-18T00:33:43.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:42.774797+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:39.800458+0000 osd.7 (osd.7) 89 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:39.801496+0000 osd.7 (osd.7) 90 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.823322+0000 osd.2 (osd.2) 69 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.834871+0000 osd.2 (osd.2) 70 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.949267+0000 osd.0 (osd.0) 85 : cluster [DBG] 17.7 deep-scrub starts 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:40.951362+0000 osd.0 (osd.0) 86 : cluster [DBG] 17.7 deep-scrub ok 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:40.960247+0000 mgr.y (mgr.24454) 199 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.054482+0000 osd.5 (osd.5) 81 : cluster [DBG] 6.d scrub starts 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.057656+0000 osd.5 (osd.5) 82 : cluster [DBG] 6.d scrub ok 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.140798+0000 osd.1 (osd.1) 87 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.153860+0000 osd.1 (osd.1) 88 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.321832+0000 osd.4 (osd.4) 99 : cluster [DBG] 17.3 deep-scrub starts 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.409272+0000 mon.b (mon.1) 158 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.446433+0000 osd.4 (osd.4) 100 : cluster [DBG] 17.3 deep-scrub ok 2024-02-18T00:33:43.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.735763+0000 mon.a (mon.0) 1130 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.762682+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi059-82882-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.762819+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.762972+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.772818+0000 mon.b (mon.1) 159 : audit [INF] from='client.? 172.21.15.59:0/3352845223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.780147+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.794917+0000 mon.b (mon.1) 160 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.803489+0000 mgr.y (mgr.24454) 200 : cluster [DBG] pgmap v202: 502 pgs: 64 unknown, 64 creating+peering, 1 active+clean+snaptrim, 373 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.824689+0000 osd.7 (osd.7) 91 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:41.826178+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.843887+0000 osd.7 (osd.7) 92 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.861471+0000 osd.3 (osd.3) 99 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.866524+0000 osd.3 (osd.3) 100 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.421894+0000 mon.b (mon.1) 161 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:43.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.774518+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi059-82877-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:43.192 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.774654+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache", "force_nonempty":""}]': finished 2024-02-18T00:33:43.192 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:42.774797+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-18T00:33:43.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:43.193 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:43.293 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:43.321 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:43.340 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:43.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:43.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:43.444 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:43.470 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:43.498 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:43.501 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:43.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:43.602 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:43.634 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:43.833 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:43.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:43.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:43.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:44.088 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:44.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:33:44.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.995455+0000 osd.0 (osd.0) 87 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:33:44.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:41.996909+0000 osd.0 (osd.0) 88 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:33:44.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:42.412521+0000 osd.6 (osd.6) 71 : cluster [DBG] 17.5 deep-scrub starts 2024-02-18T00:33:44.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:42.414895+0000 osd.6 (osd.6) 72 : cluster [DBG] 17.5 deep-scrub ok 2024-02-18T00:33:44.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:42.766187+0000 osd.2 (osd.2) 71 : cluster [DBG] 5.a scrub starts 2024-02-18T00:33:44.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:42.768623+0000 osd.2 (osd.2) 72 : cluster [DBG] 5.a scrub ok 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.813975+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.844272+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.844509+0000 mon.a (mon.0) 1140 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.876437+0000 mon.a (mon.0) 1141 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.901422+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:42.908479+0000 mon.a (mon.0) 1142 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:43.342571+0000 osd.4 (osd.4) 101 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:43.346083+0000 osd.4 (osd.4) 102 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:43.433619+0000 mon.b (mon.1) 163 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:43.565329+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:43.580552+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:44.618 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:43.580772+0000 mon.a (mon.0) 1145 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-18T00:33:44.619 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:43.586845+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.59:0/1851023858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.619 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:43.589880+0000 mon.b (mon.1) 165 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]: dispatch 2024-02-18T00:33:44.619 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:43.625464+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.619 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:43.625702+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]: dispatch 2024-02-18T00:33:44.619 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:43.807268+0000 mgr.y (mgr.24454) 201 : cluster [DBG] pgmap v205: 470 pgs: 64 unknown, 32 creating+peering, 1 active+clean+snaptrim, 373 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-02-18T00:33:44.619 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:43.925094+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:33:44.619 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:43.926294+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.995455+0000 osd.0 (osd.0) 87 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:41.996909+0000 osd.0 (osd.0) 88 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:42.412521+0000 osd.6 (osd.6) 71 : cluster [DBG] 17.5 deep-scrub starts 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:42.414895+0000 osd.6 (osd.6) 72 : cluster [DBG] 17.5 deep-scrub ok 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:42.766187+0000 osd.2 (osd.2) 71 : cluster [DBG] 5.a scrub starts 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:42.768623+0000 osd.2 (osd.2) 72 : cluster [DBG] 5.a scrub ok 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.813975+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.844272+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.844509+0000 mon.a (mon.0) 1140 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:33:44.621 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.876437+0000 mon.a (mon.0) 1141 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.901422+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:42.908479+0000 mon.a (mon.0) 1142 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:43.342571+0000 osd.4 (osd.4) 101 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:43.346083+0000 osd.4 (osd.4) 102 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:43.433619+0000 mon.b (mon.1) 163 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:43.565329+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:43.580552+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:43.580772+0000 mon.a (mon.0) 1145 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:43.586845+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.59:0/1851023858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:43.589880+0000 mon.b (mon.1) 165 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]: dispatch 2024-02-18T00:33:44.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:43.625464+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:43.625702+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]: dispatch 2024-02-18T00:33:44.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:43.807268+0000 mgr.y (mgr.24454) 201 : cluster [DBG] pgmap v205: 470 pgs: 64 unknown, 32 creating+peering, 1 active+clean+snaptrim, 373 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-02-18T00:33:44.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:43.925094+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:33:44.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:43.926294+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:33:44.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.995455+0000 osd.0 (osd.0) 87 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:33:44.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:41.996909+0000 osd.0 (osd.0) 88 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:33:44.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:42.412521+0000 osd.6 (osd.6) 71 : cluster [DBG] 17.5 deep-scrub starts 2024-02-18T00:33:44.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:42.414895+0000 osd.6 (osd.6) 72 : cluster [DBG] 17.5 deep-scrub ok 2024-02-18T00:33:44.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:42.766187+0000 osd.2 (osd.2) 71 : cluster [DBG] 5.a scrub starts 2024-02-18T00:33:44.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:42.768623+0000 osd.2 (osd.2) 72 : cluster [DBG] 5.a scrub ok 2024-02-18T00:33:44.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.813975+0000 mon.b (mon.1) 162 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.844272+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.844509+0000 mon.a (mon.0) 1140 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.876437+0000 mon.a (mon.0) 1141 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.901422+0000 mon.c (mon.2) 82 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:42.908479+0000 mon.a (mon.0) 1142 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:43.342571+0000 osd.4 (osd.4) 101 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:43.346083+0000 osd.4 (osd.4) 102 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:43.433619+0000 mon.b (mon.1) 163 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:43.565329+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:43.580552+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82965-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:44.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:43.580772+0000 mon.a (mon.0) 1145 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-18T00:33:44.726 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:43.586845+0000 mon.b (mon.1) 164 : audit [INF] from='client.? 172.21.15.59:0/1851023858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.726 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:43.589880+0000 mon.b (mon.1) 165 : audit [INF] from='client.? 172.21.15.59:0/3720201279' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]: dispatch 2024-02-18T00:33:44.726 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:43.625464+0000 mon.a (mon.0) 1146 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:44.726 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:43.625702+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]: dispatch 2024-02-18T00:33:44.726 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:43.807268+0000 mgr.y (mgr.24454) 201 : cluster [DBG] pgmap v205: 470 pgs: 64 unknown, 32 creating+peering, 1 active+clean+snaptrim, 373 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 4 op/s 2024-02-18T00:33:44.726 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:43.925094+0000 osd.3 (osd.3) 101 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:33:44.726 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:43.926294+0000 osd.3 (osd.3) 102 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:33:44.990 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:33:44.990 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:33:45.003 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:45.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:45.006 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:45.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:45.138 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:45.236 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:45.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:45.238 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.195+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2478132238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ce0069ba0 msgr2=0x7f3ce0103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.195+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2478132238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ce0069ba0 0x7f3ce0103fb0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 rx=0x7f3cd00048a0 tx=0x7f3cd0009cb0).stop 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.199+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2478132238 shutdown_connections 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.199+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2478132238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 0x7f3ce0104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.199+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2478132238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce00692d0 0x7f3ce00696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.199+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2478132238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ce0069ba0 0x7f3ce0103fb0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.199+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2478132238 >> 172.21.15.59:0/2478132238 conn(0x7f3ce0063db0 msgr2=0x7f3ce0067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.203+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2478132238 shutdown_connections 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.203+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2478132238 wait complete. 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 librados: starting msgr at 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 librados: starting objecter 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 Processor -- start 2024-02-18T00:33:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 -- start start 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 librados: setting wanted keys 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 librados: calling monclient init 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce00692d0 0x7f3ce010cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ce0069ba0 0x7f3ce010d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 0x7f3ce01117f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce010db70 con 0x7f3ce00692d0 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0111ce0 con 0x7f3ce01044a0 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.204+0000 7f3ce5b8f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0111ea0 con 0x7f3ce0069ba0 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 0x7f3ce01117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 0x7f3ce01117f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:56658/0 (socket says 172.21.15.59:56658) 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 -- 172.21.15.59:0/2262640023 learned_addr learned my addr 172.21.15.59:0/2262640023 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 -- 172.21.15.59:0/2262640023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ce0069ba0 msgr2=0x7f3ce010d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 --2- 172.21.15.59:0/2262640023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ce0069ba0 0x7f3ce010d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 -- 172.21.15.59:0/2262640023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce00692d0 msgr2=0x7f3ce010cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 --2- 172.21.15.59:0/2262640023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce00692d0 0x7f3ce010cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.205+0000 7f3cdffff700 1 -- 172.21.15.59:0/2262640023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd0005040 con 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3cdffff700 1 --2- 172.21.15.59:0/2262640023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 0x7f3ce01117f0 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 rx=0x7f3cd400b460 tx=0x7f3cd4006f20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3cdcff9700 10 client.?.objecter ms_handle_connect 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3cdcff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3cdcff9700 1 -- 172.21.15.59:0/2262640023 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4012410 con 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce0112180 con 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3ce5b8f700 10 client.40003.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce0112670 con 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.206+0000 7f3ce5b8f700 1 librados: init done 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.207+0000 7f3cdcff9700 1 -- 172.21.15.59:0/2262640023 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd4012a40 con 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.207+0000 7f3cdcff9700 1 -- 172.21.15.59:0/2262640023 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400d940 con 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.209+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc004f50 con 0x7f3ce01044a0 2024-02-18T00:33:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.213+0000 7f3cdcff9700 1 -- 172.21.15.59:0/2262640023 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f3cd400dd70 con 0x7f3ce01044a0 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.213+0000 7f3cdcff9700 1 --2- 172.21.15.59:0/2262640023 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3cc8062c90 0x7f3cc8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.213+0000 7f3cdcff9700 1 -- 172.21.15.59:0/2262640023 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(146..146 src has 44..146) v4 ==== 11087+0+0 (secure 0 0 0) 0x7f3cd407dc20 con 0x7f3ce01044a0 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.213+0000 7f3cdcff9700 10 client.40003.objecter ms_dispatch 0x7f3ce0001550 osd_map(146..146 src has 44..146) v4 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.213+0000 7f3cdcff9700 3 client.40003.objecter handle_osd_map got epochs [146,146] > 0 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.213+0000 7f3cdcff9700 3 client.40003.objecter handle_osd_map decoding full epoch 146 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.214+0000 7f3cdcff9700 20 client.40003.objecter dump_active .. 0 homeless 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.214+0000 7f3cdf7fe700 1 --2- 172.21.15.59:0/2262640023 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3cc8062c90 0x7f3cc8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.214+0000 7f3cdf7fe700 1 --2- 172.21.15.59:0/2262640023 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3cc8062c90 0x7f3cc8065150 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 rx=0x7f3cd800acf0 tx=0x7f3cd8009370).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.215+0000 7f3cdcff9700 10 client.40003.objecter ms_handle_connect 0x7f3cc8062c90 2024-02-18T00:33:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.218+0000 7f3cdcff9700 1 -- 172.21.15.59:0/2262640023 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3cd40504c0 con 0x7f3ce01044a0 2024-02-18T00:33:45.339 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:45.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:44.004293+0000 osd.5 (osd.5) 83 : cluster [DBG] 4.f scrub starts 2024-02-18T00:33:45.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:44.007799+0000 osd.5 (osd.5) 84 : cluster [DBG] 4.f scrub ok 2024-02-18T00:33:45.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:44.346549+0000 osd.3 (osd.3) 103 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:45.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:44.346655+0000 osd.3 (osd.3) 104 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:45.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:44.412560+0000 osd.6 (osd.6) 73 : cluster [DBG] 17.6 deep-scrub starts 2024-02-18T00:33:45.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:44.433468+0000 osd.6 (osd.6) 74 : cluster [DBG] 17.6 deep-scrub ok 2024-02-18T00:33:45.417 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.434704+0000 mon.b (mon.1) 166 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.593320+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.593492+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.593565+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]': finished 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:44.593722+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.599573+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.620031+0000 mon.a (mon.0) 1152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.624719+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.625538+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.644524+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.59:0/611161834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:45.418 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:44.645890+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:45.465 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:45.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.548+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f3ccc005200 con 0x7f3ce01044a0 2024-02-18T00:33:45.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.608+0000 7f3cdcff9700 1 -- 172.21.15.59:0/2262640023 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v147) v1 ==== 101+0+0 (secure 0 0 0) 0x7f3cd40500b0 con 0x7f3ce01044a0 2024-02-18T00:33:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 10 librados: watch_flush enter 2024-02-18T00:33:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 10 librados: watch_flush exit 2024-02-18T00:33:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 20 client.40003.objecter shutdown clearing up homeless session... 2024-02-18T00:33:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 10 client.40003.objecter successfully canceled tick 2024-02-18T00:33:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3cc8062c90 msgr2=0x7f3cc8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2262640023 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3cc8062c90 0x7f3cc8065150 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 rx=0x7f3cd800acf0 tx=0x7f3cd8009370).stop 2024-02-18T00:33:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 msgr2=0x7f3ce01117f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.613+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2262640023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 0x7f3ce01117f0 secure :-1 s=READY pgs=3422 cs=0 l=1 rev1=1 rx=0x7f3cd400b460 tx=0x7f3cd4006f20).stop 2024-02-18T00:33:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 shutdown_connections 2024-02-18T00:33:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2262640023 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3cc8062c90 0x7f3cc8065150 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2262640023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce01044a0 0x7f3ce01117f0 unknown :-1 s=CLOSED pgs=3422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2262640023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ce00692d0 0x7f3ce010cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 --2- 172.21.15.59:0/2262640023 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ce0069ba0 0x7f3ce010d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 >> 172.21.15.59:0/2262640023 conn(0x7f3ce0063db0 msgr2=0x7f3ce00fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 shutdown_connections 2024-02-18T00:33:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 -- 172.21.15.59:0/2262640023 wait complete. 2024-02-18T00:33:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:45.614+0000 7f3ce5b8f700 1 librados: shutdown 2024-02-18T00:33:45.628 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:33:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:44.004293+0000 osd.5 (osd.5) 83 : cluster [DBG] 4.f scrub starts 2024-02-18T00:33:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:44.007799+0000 osd.5 (osd.5) 84 : cluster [DBG] 4.f scrub ok 2024-02-18T00:33:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:44.346549+0000 osd.3 (osd.3) 103 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:44.346655+0000 osd.3 (osd.3) 104 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:44.412560+0000 osd.6 (osd.6) 73 : cluster [DBG] 17.6 deep-scrub starts 2024-02-18T00:33:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:44.433468+0000 osd.6 (osd.6) 74 : cluster [DBG] 17.6 deep-scrub ok 2024-02-18T00:33:45.655 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.434704+0000 mon.b (mon.1) 166 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.593320+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.593492+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.593565+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]': finished 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:44.593722+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.599573+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.620031+0000 mon.a (mon.0) 1152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.624719+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.625538+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.644524+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.59:0/611161834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:45.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:44.645890+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:44.004293+0000 osd.5 (osd.5) 83 : cluster [DBG] 4.f scrub starts 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:44.007799+0000 osd.5 (osd.5) 84 : cluster [DBG] 4.f scrub ok 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:44.346549+0000 osd.3 (osd.3) 103 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:44.346655+0000 osd.3 (osd.3) 104 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:44.412560+0000 osd.6 (osd.6) 73 : cluster [DBG] 17.6 deep-scrub starts 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:44.433468+0000 osd.6 (osd.6) 74 : cluster [DBG] 17.6 deep-scrub ok 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.434704+0000 mon.b (mon.1) 166 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.593320+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.593492+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.593565+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82965-10", "tierpool":"test-rados-api-smithi059-82965-10-cache"}]': finished 2024-02-18T00:33:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:44.593722+0000 mon.a (mon.0) 1151 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-18T00:33:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.599573+0000 mon.b (mon.1) 167 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.620031+0000 mon.a (mon.0) 1152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.624719+0000 mon.c (mon.2) 83 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.625538+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.644524+0000 mon.c (mon.2) 84 : audit [INF] from='client.? 172.21.15.59:0/611161834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:44.645890+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:45.810 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-18T00:33:46.086 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:46.282 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:46.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:46.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 -- 172.21.15.59:0/1271203842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100f6b10 msgr2=0x7f26101030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 --2- 172.21.15.59:0/1271203842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100f6b10 0x7f26101030b0 secure :-1 s=READY pgs=3581 cs=0 l=1 rev1=1 rx=0x7f2608009be0 tx=0x7f2608004d10).stop 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 -- 172.21.15.59:0/1271203842 shutdown_connections 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 --2- 172.21.15.59:0/1271203842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100f6b10 0x7f26101030b0 unknown :-1 s=CLOSED pgs=3581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 --2- 172.21.15.59:0/1271203842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26100fc240 0x7f26100fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 --2- 172.21.15.59:0/1271203842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26100fcba0 0x7f26100f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 -- 172.21.15.59:0/1271203842 >> 172.21.15.59:0/1271203842 conn(0x7f261006c370 msgr2=0x7f261006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 -- 172.21.15.59:0/1271203842 shutdown_connections 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 -- 172.21.15.59:0/1271203842 wait complete. 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 librados: starting msgr at 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.296+0000 7f2617975700 1 librados: starting objecter 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 Processor -- start 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 -- start start 2024-02-18T00:33:46.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 librados: setting wanted keys 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 librados: calling monclient init 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26100f6b10 0x7f26100feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26100fc240 0x7f26100ff020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100fcba0 0x7f261011a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261011aa80 con 0x7f26100fc240 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261010acd0 con 0x7f26100fcba0 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2617975700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261010ae90 con 0x7f26100f6b10 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2615f12700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100fcba0 0x7f261011a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2615f12700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100fcba0 0x7f261011a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:59568/0 (socket says 172.21.15.59:59568) 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.297+0000 7f2615f12700 1 -- 172.21.15.59:0/67644891 learned_addr learned my addr 172.21.15.59:0/67644891 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f2615f12700 1 -- 172.21.15.59:0/67644891 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26100f6b10 msgr2=0x7f26100feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f2615f12700 1 --2- 172.21.15.59:0/67644891 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26100f6b10 0x7f26100feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f2615f12700 1 -- 172.21.15.59:0/67644891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26100fc240 msgr2=0x7f26100ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:46.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f2615f12700 1 --2- 172.21.15.59:0/67644891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26100fc240 0x7f26100ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f2615f12700 1 -- 172.21.15.59:0/67644891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26080097c0 con 0x7f26100fcba0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f2615f12700 1 --2- 172.21.15.59:0/67644891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100fcba0 0x7f261011a590 secure :-1 s=READY pgs=3582 cs=0 l=1 rev1=1 rx=0x7f2608004ed0 tx=0x7f2608004d10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f26067fc700 10 client.?.objecter ms_handle_connect 0x7f26100fcba0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f26067fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f26067fc700 1 -- 172.21.15.59:0/67644891 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800e030 con 0x7f26100fcba0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f26067fc700 1 -- 172.21.15.59:0/67644891 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f260800be60 con 0x7f26100fcba0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.298+0000 7f26067fc700 1 -- 172.21.15.59:0/67644891 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608018070 con 0x7f26100fcba0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.299+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261010b110 con 0x7f26100fcba0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.299+0000 7f2617975700 10 client.40696.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.299+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261010b520 con 0x7f26100fcba0 2024-02-18T00:33:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.299+0000 7f2617975700 1 librados: init done 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.300+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261005fa60 con 0x7f26100fcba0 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.300+0000 7f26067fc700 1 -- 172.21.15.59:0/67644891 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2608013840 con 0x7f26100fcba0 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.300+0000 7f26067fc700 1 --2- 172.21.15.59:0/67644891 >> v2:172.21.15.59:6800/3721891797 conn(0x7f25fc062c90 0x7f25fc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.300+0000 7f26067fc700 1 -- 172.21.15.59:0/67644891 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(147..147 src has 44..147) v4 ==== 10033+0+0 (secure 0 0 0) 0x7f260807f240 con 0x7f26100fcba0 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.300+0000 7f26067fc700 10 client.40696.objecter ms_dispatch 0x7f2610001550 osd_map(147..147 src has 44..147) v4 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.300+0000 7f26067fc700 3 client.40696.objecter handle_osd_map got epochs [147,147] > 0 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.300+0000 7f26067fc700 3 client.40696.objecter handle_osd_map decoding full epoch 147 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.301+0000 7f2615711700 1 --2- 172.21.15.59:0/67644891 >> v2:172.21.15.59:6800/3721891797 conn(0x7f25fc062c90 0x7f25fc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:46.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.301+0000 7f26067fc700 20 client.40696.objecter dump_active .. 0 homeless 2024-02-18T00:33:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.301+0000 7f26067fc700 10 client.40696.objecter ms_handle_connect 0x7f25fc062c90 2024-02-18T00:33:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.301+0000 7f2615711700 1 --2- 172.21.15.59:0/67644891 >> v2:172.21.15.59:6800/3721891797 conn(0x7f25fc062c90 0x7f25fc065150 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7f260c009700 tx=0x7f260c006c30).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.305+0000 7f26067fc700 1 -- 172.21.15.59:0/67644891 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2608051fd0 con 0x7f26100fcba0 2024-02-18T00:33:46.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:46.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:46.631+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd out", "ids": ["7"]} v 0) v1 -- 0x7f26100ffe00 con 0x7f26100fcba0 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.301657+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.e scrub starts 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.311255+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.e scrub ok 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.441721+0000 mon.b (mon.1) 168 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.549787+0000 mon.b (mon.1) 169 : audit [INF] from='client.? 172.21.15.59:0/2262640023' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.550972+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.594600+0000 mon.a (mon.0) 1156 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.606236+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:46.636 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.606352+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]': finished 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.606442+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.606556+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.606602+0000 mon.a (mon.0) 1161 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.613021+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]: dispatch 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.613767+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]: dispatch 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.619463+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:45.636841+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.808937+0000 mgr.y (mgr.24454) 202 : cluster [DBG] pgmap v208: 430 pgs: 1 active+clean+snaptrim, 429 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 256 KiB/s wr, 2 op/s 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.900317+0000 osd.3 (osd.3) 105 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:33:46.637 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.906509+0000 osd.3 (osd.3) 106 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:33:46.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.301657+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.e scrub starts 2024-02-18T00:33:46.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.311255+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.e scrub ok 2024-02-18T00:33:46.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.441721+0000 mon.b (mon.1) 168 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:46.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.549787+0000 mon.b (mon.1) 169 : audit [INF] from='client.? 172.21.15.59:0/2262640023' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:46.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.550972+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:46.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.594600+0000 mon.a (mon.0) 1156 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:33:46.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.606236+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.606352+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]': finished 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.606442+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.606556+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.606602+0000 mon.a (mon.0) 1161 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.613021+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]: dispatch 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.613767+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]: dispatch 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.619463+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:45.636841+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.808937+0000 mgr.y (mgr.24454) 202 : cluster [DBG] pgmap v208: 430 pgs: 1 active+clean+snaptrim, 429 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 256 KiB/s wr, 2 op/s 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.900317+0000 osd.3 (osd.3) 105 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.906509+0000 osd.3 (osd.3) 106 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:33:46.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:33:46.638+0000 7f6151a5e700 -1 mon.a@0(leader).osd e148 definitely_dead 0 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.301657+0000 osd.4 (osd.4) 103 : cluster [DBG] 4.e scrub starts 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.311255+0000 osd.4 (osd.4) 104 : cluster [DBG] 4.e scrub ok 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.441721+0000 mon.b (mon.1) 168 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.549787+0000 mon.b (mon.1) 169 : audit [INF] from='client.? 172.21.15.59:0/2262640023' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.550972+0000 mon.a (mon.0) 1155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.594600+0000 mon.a (mon.0) 1156 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.606236+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.606352+0000 mon.a (mon.0) 1158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-11"}]': finished 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.606442+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi059-82877-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.606556+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.606602+0000 mon.a (mon.0) 1161 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-18T00:33:46.651 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.613021+0000 mon.c (mon.2) 85 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]: dispatch 2024-02-18T00:33:46.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.613767+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]: dispatch 2024-02-18T00:33:46.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.619463+0000 mon.b (mon.1) 170 : audit [INF] from='client.? 172.21.15.59:0/510272679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:46.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:45.636841+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]: dispatch 2024-02-18T00:33:46.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.808937+0000 mgr.y (mgr.24454) 202 : cluster [DBG] pgmap v208: 430 pgs: 1 active+clean+snaptrim, 429 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 256 KiB/s wr, 2 op/s 2024-02-18T00:33:46.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.900317+0000 osd.3 (osd.3) 105 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:33:46.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.906509+0000 osd.3 (osd.3) 106 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: Running main() from gmock_main.cc 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [----------] Global test environment set-up. 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolList (12510 ms) 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (4903 ms) 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4779 ms) 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-18T00:33:46.722 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3476 ms) 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3412 ms) 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5718 ms) 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5641 ms) 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (6278 ms) 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (9081 ms) 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [----------] 9 tests from LibRadosPools (55798 ms total) 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [----------] Global test environment tear-down 2024-02-18T00:33:46.723 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (55798 ms total) 2024-02-18T00:33:46.724 INFO:tasks.workunit.client.0.smithi059.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-18T00:33:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:46.989 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:46.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:46.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:47.094 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:47.121 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:47.570 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:47.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:47.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.648+0000 7f26067fc700 1 -- 172.21.15.59:0/67644891 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["7"]}]=0 marked out osd.7. v149) v1 ==== 87+0+0 (secure 0 0 0) 0x7f2608051bc0 con 0x7f26100fcba0 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 10 librados: watch_flush enter 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 10 librados: watch_flush exit 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 20 client.40696.objecter shutdown clearing up homeless session... 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 10 client.40696.objecter successfully canceled tick 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 >> v2:172.21.15.59:6800/3721891797 conn(0x7f25fc062c90 msgr2=0x7f25fc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 1 --2- 172.21.15.59:0/67644891 >> v2:172.21.15.59:6800/3721891797 conn(0x7f25fc062c90 0x7f25fc065150 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7f260c009700 tx=0x7f260c006c30).stop 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100fcba0 msgr2=0x7f261011a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:47.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.651+0000 7f2617975700 1 --2- 172.21.15.59:0/67644891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100fcba0 0x7f261011a590 secure :-1 s=READY pgs=3582 cs=0 l=1 rev1=1 rx=0x7f2608004ed0 tx=0x7f2608004d10).stop 2024-02-18T00:33:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.656+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 shutdown_connections 2024-02-18T00:33:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.656+0000 7f2617975700 1 --2- 172.21.15.59:0/67644891 >> v2:172.21.15.59:6800/3721891797 conn(0x7f25fc062c90 0x7f25fc065150 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.656+0000 7f2617975700 1 --2- 172.21.15.59:0/67644891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26100fcba0 0x7f261011a590 unknown :-1 s=CLOSED pgs=3582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.656+0000 7f2617975700 1 --2- 172.21.15.59:0/67644891 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26100f6b10 0x7f26100feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.656+0000 7f2617975700 1 --2- 172.21.15.59:0/67644891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26100fc240 0x7f26100ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:33:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.656+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 >> 172.21.15.59:0/67644891 conn(0x7f261006c370 msgr2=0x7f26100f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:47.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.670+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 shutdown_connections 2024-02-18T00:33:47.674 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:33:47.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.673+0000 7f2617975700 1 -- 172.21.15.59:0/67644891 wait complete. 2024-02-18T00:33:47.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:47.676+0000 7f2617975700 1 librados: shutdown 2024-02-18T00:33:47.702 INFO:teuthology.orchestra.run.smithi059.stderr:marked out osd.7. 2024-02-18T00:33:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.785961+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:33:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.793227+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:33:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.809671+0000 osd.2 (osd.2) 73 : cluster [DBG] 6.f scrub starts 2024-02-18T00:33:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:45.813506+0000 osd.2 (osd.2) 74 : cluster [DBG] 6.f scrub ok 2024-02-18T00:33:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.443795+0000 mon.b (mon.1) 171 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.602954+0000 mon.a (mon.0) 1164 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.604851+0000 mon.a (mon.0) 1165 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.623629+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]': finished 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.623747+0000 mon.a (mon.0) 1167 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.623835+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.629118+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.59:0/3190096847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.630696+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.637410+0000 mon.b (mon.1) 172 : audit [INF] from='client.? 172.21.15.59:0/67644891' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.639768+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.639882+0000 mon.a (mon.0) 1171 : cluster [INF] Client client.admin marked osd.7 out, while it was still marked up 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.788851+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.791653+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:47.770 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.785961+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.793227+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.809671+0000 osd.2 (osd.2) 73 : cluster [DBG] 6.f scrub starts 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:45.813506+0000 osd.2 (osd.2) 74 : cluster [DBG] 6.f scrub ok 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.443795+0000 mon.b (mon.1) 171 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.602954+0000 mon.a (mon.0) 1164 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.604851+0000 mon.a (mon.0) 1165 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.623629+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]': finished 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.623747+0000 mon.a (mon.0) 1167 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.623835+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.629118+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.59:0/3190096847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.630696+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:47.771 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.637410+0000 mon.b (mon.1) 172 : audit [INF] from='client.? 172.21.15.59:0/67644891' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-18T00:33:47.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.639768+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-18T00:33:47.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.639882+0000 mon.a (mon.0) 1171 : cluster [INF] Client client.admin marked osd.7 out, while it was still marked up 2024-02-18T00:33:47.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.788851+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:47.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.791653+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.785961+0000 osd.7 (osd.7) 93 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.793227+0000 osd.7 (osd.7) 94 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.809671+0000 osd.2 (osd.2) 73 : cluster [DBG] 6.f scrub starts 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:45.813506+0000 osd.2 (osd.2) 74 : cluster [DBG] 6.f scrub ok 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.443795+0000 mon.b (mon.1) 171 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.602954+0000 mon.a (mon.0) 1164 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.604851+0000 mon.a (mon.0) 1165 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.623629+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-11", "mode": "writeback"}]': finished 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.623747+0000 mon.a (mon.0) 1167 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi059-82973-9"}]': finished 2024-02-18T00:33:47.777 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.623835+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-18T00:33:47.778 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.629118+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.59:0/3190096847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:47.778 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.630696+0000 mon.a (mon.0) 1169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:47.778 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.637410+0000 mon.b (mon.1) 172 : audit [INF] from='client.? 172.21.15.59:0/67644891' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-18T00:33:47.778 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.639768+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["7"]}]: dispatch 2024-02-18T00:33:47.778 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.639882+0000 mon.a (mon.0) 1171 : cluster [INF] Client client.admin marked osd.7 out, while it was still marked up 2024-02-18T00:33:47.778 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.788851+0000 mon.b (mon.1) 173 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:47.778 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.791653+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.051 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.023579+0000 osd.0 (osd.0) 89 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.025290+0000 osd.0 (osd.0) 90 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.367073+0000 osd.6 (osd.6) 75 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.368002+0000 osd.6 (osd.6) 76 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.791663+0000 osd.7 (osd.7) 95 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:46.792574+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.807513+0000 osd.7 (osd.7) 96 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.855419+0000 osd.2 (osd.2) 75 : cluster [DBG] 2.c scrub starts 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.861227+0000 osd.2 (osd.2) 76 : cluster [DBG] 2.c scrub ok 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.891410+0000 osd.3 (osd.3) 107 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:33:48.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:46.897436+0000 osd.3 (osd.3) 108 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:47.195745+0000 osd.1 (osd.1) 89 : cluster [DBG] 2.9 deep-scrub starts 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:47.199003+0000 osd.1 (osd.1) 90 : cluster [DBG] 2.9 deep-scrub ok 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.377446+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.378195+0000 mon.b (mon.1) 175 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.403942+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.447118+0000 mon.b (mon.1) 176 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.641563+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.641712+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.641884+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:47.641957+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e149: 8 total, 8 up, 7 in 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.659079+0000 mon.b (mon.1) 177 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.659378+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.59:0/3442492812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:48.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.668522+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:47.668973+0000 mon.a (mon.0) 1180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:48.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:47.819721+0000 mgr.y (mgr.24454) 203 : cluster [DBG] pgmap v211: 430 pgs: 64 unknown, 1 active+clean+snaptrim, 365 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 1020 B/s rd, 256 KiB/s wr, 2 op/s 2024-02-18T00:33:48.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.023579+0000 osd.0 (osd.0) 89 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.025290+0000 osd.0 (osd.0) 90 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.367073+0000 osd.6 (osd.6) 75 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.368002+0000 osd.6 (osd.6) 76 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.791663+0000 osd.7 (osd.7) 95 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:46.792574+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.807513+0000 osd.7 (osd.7) 96 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.855419+0000 osd.2 (osd.2) 75 : cluster [DBG] 2.c scrub starts 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.861227+0000 osd.2 (osd.2) 76 : cluster [DBG] 2.c scrub ok 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.891410+0000 osd.3 (osd.3) 107 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:46.897436+0000 osd.3 (osd.3) 108 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:47.195745+0000 osd.1 (osd.1) 89 : cluster [DBG] 2.9 deep-scrub starts 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:47.199003+0000 osd.1 (osd.1) 90 : cluster [DBG] 2.9 deep-scrub ok 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.377446+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.378195+0000 mon.b (mon.1) 175 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.403942+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.447118+0000 mon.b (mon.1) 176 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.641563+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.641712+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.641884+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:47.641957+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e149: 8 total, 8 up, 7 in 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.659079+0000 mon.b (mon.1) 177 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.659378+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.59:0/3442492812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.668522+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:47.668973+0000 mon.a (mon.0) 1180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:47.819721+0000 mgr.y (mgr.24454) 203 : cluster [DBG] pgmap v211: 430 pgs: 64 unknown, 1 active+clean+snaptrim, 365 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 1020 B/s rd, 256 KiB/s wr, 2 op/s 2024-02-18T00:33:48.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.023579+0000 osd.0 (osd.0) 89 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.025290+0000 osd.0 (osd.0) 90 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.367073+0000 osd.6 (osd.6) 75 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.368002+0000 osd.6 (osd.6) 76 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.791663+0000 osd.7 (osd.7) 95 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:46.792574+0000 mon.b (mon.1) 174 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.807513+0000 osd.7 (osd.7) 96 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.855419+0000 osd.2 (osd.2) 75 : cluster [DBG] 2.c scrub starts 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.861227+0000 osd.2 (osd.2) 76 : cluster [DBG] 2.c scrub ok 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.891410+0000 osd.3 (osd.3) 107 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:46.897436+0000 osd.3 (osd.3) 108 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:47.195745+0000 osd.1 (osd.1) 89 : cluster [DBG] 2.9 deep-scrub starts 2024-02-18T00:33:48.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:47.199003+0000 osd.1 (osd.1) 90 : cluster [DBG] 2.9 deep-scrub ok 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.377446+0000 mon.a (mon.0) 1173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.378195+0000 mon.b (mon.1) 175 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.403942+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.447118+0000 mon.b (mon.1) 176 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.641563+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi059-82882-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.641712+0000 mon.a (mon.0) 1176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["7"]}]': finished 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.641884+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:47.641957+0000 mon.a (mon.0) 1178 : cluster [DBG] osdmap e149: 8 total, 8 up, 7 in 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.659079+0000 mon.b (mon.1) 177 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.659378+0000 mon.b (mon.1) 178 : audit [INF] from='client.? 172.21.15.59:0/3442492812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.668522+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:33:48.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:47.668973+0000 mon.a (mon.0) 1180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:48.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:47.819721+0000 mgr.y (mgr.24454) 203 : cluster [DBG] pgmap v211: 430 pgs: 64 unknown, 1 active+clean+snaptrim, 365 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 1020 B/s rd, 256 KiB/s wr, 2 op/s 2024-02-18T00:33:48.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:33:49.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:48.007974+0000 osd.0 (osd.0) 91 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:33:49.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:48.010815+0000 osd.0 (osd.0) 92 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:33:49.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:48.450935+0000 mon.b (mon.1) 179 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:49.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:48.949363+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:49.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:49.221526+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:49.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:49.221735+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e150: 8 total, 8 up, 7 in 2024-02-18T00:33:49.412 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:49.229986+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:48.007974+0000 osd.0 (osd.0) 91 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:33:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:48.010815+0000 osd.0 (osd.0) 92 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:33:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:48.450935+0000 mon.b (mon.1) 179 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:48.949363+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:49.221526+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:49.221735+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e150: 8 total, 8 up, 7 in 2024-02-18T00:33:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:49.229986+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:48.007974+0000 osd.0 (osd.0) 91 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:33:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:48.010815+0000 osd.0 (osd.0) 92 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:33:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:48.450935+0000 mon.b (mon.1) 179 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:48.949363+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:49.221526+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi059-82877-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:49.221735+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e150: 8 total, 8 up, 7 in 2024-02-18T00:33:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:49.229986+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:33:50.367 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:33:50.367 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:33:50.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] Global test environment set-up. 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: 2024-02-18T00:32:51.033+0000 7fa8cf5f6700 -1 failed for service _ceph-mon._tcp 2024-02-18T00:33:50.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: 2024-02-18T00:32:51.033+0000 7fa8cf5f6700 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (135 ms) 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2007 ms) 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2142 ms total) 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: started 0x55be22bfbf70 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: started 0x55be22c05450 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: started 2 aios 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: waiting 0x55be22bfbf70 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: waiting 0x55be22c05450 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: done. 2024-02-18T00:33:50.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (15854 ms) 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (15854 ms total) 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.Exec (97 ms) 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (5 ms) 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (4 ms) 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5788 ms) 2024-02-18T00:33:50.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (31551 ms) 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] 8 tests from LibRadosMisc (37445 ms total) 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [----------] Global test environment tear-down 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (59716 ms total) 2024-02-18T00:33:50.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-18T00:33:50.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:49.459909+0000 mon.b (mon.1) 180 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:50.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:49.574234+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:33:50.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:49.574362+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:50.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:49.574449+0000 mon.a (mon.0) 1186 : cluster [DBG] osdmap e151: 8 total, 8 up, 7 in 2024-02-18T00:33:50.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:49.578777+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:50.719 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:49.580978+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:50.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:49.459909+0000 mon.b (mon.1) 180 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:49.574234+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:49.574362+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:49.574449+0000 mon.a (mon.0) 1186 : cluster [DBG] osdmap e151: 8 total, 8 up, 7 in 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:49.578777+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:49.580978+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:49.459909+0000 mon.b (mon.1) 180 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:49.574234+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi059-82973-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:33:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:49.574362+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:33:50.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:49.574449+0000 mon.a (mon.0) 1186 : cluster [DBG] osdmap e151: 8 total, 8 up, 7 in 2024-02-18T00:33:50.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:49.578777+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:50.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:49.580978+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]: dispatch 2024-02-18T00:33:51.043 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-18T00:33:51.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:33:51.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:33:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:33:51.312 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:51.786 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:49.341306+0000 osd.6 (osd.6) 77 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:33:51.786 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:49.344633+0000 osd.6 (osd.6) 78 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:33:51.786 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:49.820455+0000 mgr.y (mgr.24454) 204 : cluster [DBG] pgmap v214: 374 pgs: 8 unknown, 1 active+clean+snaptrim, 365 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:33:51.786 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:49.896792+0000 osd.2 (osd.2) 77 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:33:51.786 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:49.897806+0000 osd.2 (osd.2) 78 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:33:51.786 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:50.286147+0000 osd.4 (osd.4) 105 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:50.286999+0000 osd.4 (osd.4) 106 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:50.461037+0000 mon.b (mon.1) 181 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:50.572061+0000 mon.a (mon.0) 1188 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:50.584117+0000 mon.a (mon.0) 1189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]': finished 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:50.584385+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e152: 8 total, 8 up, 7 in 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:50.586811+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.59:0/3804121060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:50.586985+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.59:0/2509190237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:50.587217+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:50.587533+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:50.764516+0000 osd.3 (osd.3) 109 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:50.765363+0000 osd.3 (osd.3) 110 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:50.971260+0000 mgr.y (mgr.24454) 205 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:51.201637+0000 osd.1 (osd.1) 91 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:33:51.787 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:51.202644+0000 osd.1 (osd.1) 92 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:33:51.788 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:51.462180+0000 mon.b (mon.1) 182 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:49.341306+0000 osd.6 (osd.6) 77 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:49.344633+0000 osd.6 (osd.6) 78 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:49.820455+0000 mgr.y (mgr.24454) 204 : cluster [DBG] pgmap v214: 374 pgs: 8 unknown, 1 active+clean+snaptrim, 365 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:49.896792+0000 osd.2 (osd.2) 77 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:49.897806+0000 osd.2 (osd.2) 78 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:50.286147+0000 osd.4 (osd.4) 105 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:50.286999+0000 osd.4 (osd.4) 106 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:33:52.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:50.461037+0000 mon.b (mon.1) 181 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:50.572061+0000 mon.a (mon.0) 1188 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:50.584117+0000 mon.a (mon.0) 1189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]': finished 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:50.584385+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e152: 8 total, 8 up, 7 in 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:50.586811+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.59:0/3804121060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:50.586985+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.59:0/2509190237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:50.587217+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:50.587533+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:50.764516+0000 osd.3 (osd.3) 109 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:33:52.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:50.765363+0000 osd.3 (osd.3) 110 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:49.341306+0000 osd.6 (osd.6) 77 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:49.344633+0000 osd.6 (osd.6) 78 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:49.820455+0000 mgr.y (mgr.24454) 204 : cluster [DBG] pgmap v214: 374 pgs: 8 unknown, 1 active+clean+snaptrim, 365 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:49.896792+0000 osd.2 (osd.2) 77 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:49.897806+0000 osd.2 (osd.2) 78 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:50.286147+0000 osd.4 (osd.4) 105 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:50.286999+0000 osd.4 (osd.4) 106 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:50.461037+0000 mon.b (mon.1) 181 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:50.572061+0000 mon.a (mon.0) 1188 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:50.584117+0000 mon.a (mon.0) 1189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-11"}]': finished 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:50.584385+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e152: 8 total, 8 up, 7 in 2024-02-18T00:33:52.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:50.586811+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.59:0/3804121060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:50.586985+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.59:0/2509190237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:50.587217+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:50.587533+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:50.764516+0000 osd.3 (osd.3) 109 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:50.765363+0000 osd.3 (osd.3) 110 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:50.971260+0000 mgr.y (mgr.24454) 205 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:51.201637+0000 osd.1 (osd.1) 91 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:51.202644+0000 osd.1 (osd.1) 92 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:51.462180+0000 mon.b (mon.1) 182 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:50.971260+0000 mgr.y (mgr.24454) 205 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:51.201637+0000 osd.1 (osd.1) 91 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:51.202644+0000 osd.1 (osd.1) 92 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:33:52.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:51.462180+0000 mon.b (mon.1) 182 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:52.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:33:52] "GET /metrics HTTP/1.1" 200 199674 "" "Prometheus/2.33.4" 2024-02-18T00:33:52.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:51.294261+0000 osd.6 (osd.6) 79 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:33:52.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:51.295090+0000 osd.6 (osd.6) 80 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:33:52.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:51.587483+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:52.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:51.587600+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:52.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:51.587850+0000 mon.a (mon.0) 1195 : cluster [DBG] osdmap e153: 8 total, 8 up, 7 in 2024-02-18T00:33:52.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:51.820896+0000 mgr.y (mgr.24454) 206 : cluster [DBG] pgmap v217: 374 pgs: 64 creating+peering, 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 304 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 12 MiB/s, 499 keys/s, 28 objects/s recovering 2024-02-18T00:33:52.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:52.463186+0000 mon.b (mon.1) 183 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:53.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:51.294261+0000 osd.6 (osd.6) 79 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:51.295090+0000 osd.6 (osd.6) 80 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:51.587483+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:51.587600+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:51.587850+0000 mon.a (mon.0) 1195 : cluster [DBG] osdmap e153: 8 total, 8 up, 7 in 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:51.820896+0000 mgr.y (mgr.24454) 206 : cluster [DBG] pgmap v217: 374 pgs: 64 creating+peering, 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 304 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 12 MiB/s, 499 keys/s, 28 objects/s recovering 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:52.463186+0000 mon.b (mon.1) 183 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:51.294261+0000 osd.6 (osd.6) 79 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:51.295090+0000 osd.6 (osd.6) 80 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:33:53.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:51.587483+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi059-82877-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:51.587600+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi059-82882-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:51.587850+0000 mon.a (mon.0) 1195 : cluster [DBG] osdmap e153: 8 total, 8 up, 7 in 2024-02-18T00:33:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:51.820896+0000 mgr.y (mgr.24454) 206 : cluster [DBG] pgmap v217: 374 pgs: 64 creating+peering, 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 304 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 12 MiB/s, 499 keys/s, 28 objects/s recovering 2024-02-18T00:33:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:52.463186+0000 mon.b (mon.1) 183 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:53.098 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:33:53.098 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:33:53.342 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:33:53.345 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:53.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:33:53.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 -- 172.21.15.59:0/423793809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d40f9610 msgr2=0x7fc5d4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 --2- 172.21.15.59:0/423793809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d40f9610 0x7fc5d4101d20 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 rx=0x7fc5c8004870 tx=0x7fc5c8009cb0).stop 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 -- 172.21.15.59:0/423793809 shutdown_connections 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 --2- 172.21.15.59:0/423793809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5d40f8cb0 0x7fc5d40f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 --2- 172.21.15.59:0/423793809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5d4102210 0x7fc5d4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 --2- 172.21.15.59:0/423793809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d40f9610 0x7fc5d4101d20 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 -- 172.21.15.59:0/423793809 >> 172.21.15.59:0/423793809 conn(0x7fc5d40f2820 msgr2=0x7fc5d40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 -- 172.21.15.59:0/423793809 shutdown_connections 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.366+0000 7fc5da783700 1 -- 172.21.15.59:0/423793809 wait complete. 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 librados: starting msgr at 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 librados: starting objecter 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 Processor -- start 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 -- start start 2024-02-18T00:33:53.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 librados: setting wanted keys 2024-02-18T00:33:53.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 librados: calling monclient init 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5d40f8cb0 0x7fc5d40f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5d40f9610 0x7fc5d40f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d4102210 0x7fc5d40f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4116610 con 0x7fc5d40f8cb0 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4116780 con 0x7fc5d40f9610 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.367+0000 7fc5da783700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4116940 con 0x7fc5d4102210 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d4102210 0x7fc5d40f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d4102210 0x7fc5d40f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:52638/0 (socket says 172.21.15.59:52638) 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 learned_addr learned my addr 172.21.15.59:0/177156093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5d40f8cb0 0x7fc5d40f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d37fe700 1 --2- 172.21.15.59:0/177156093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5d40f9610 0x7fc5d40f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5d40f9610 msgr2=0x7fc5d40f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 --2- 172.21.15.59:0/177156093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5d40f9610 0x7fc5d40f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5d40f8cb0 msgr2=0x7fc5d40f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 --2- 172.21.15.59:0/177156093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5d40f8cb0 0x7fc5d40f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8005040 con 0x7fc5d4102210 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d8d20700 1 --2- 172.21.15.59:0/177156093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d4102210 0x7fc5d40f7640 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 rx=0x7fc5cc00d940 tx=0x7fc5cc004e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d17fa700 10 client.?.objecter ms_handle_connect 0x7fc5d4102210 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d17fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:53.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc004020 con 0x7fc5d4102210 2024-02-18T00:33:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d4116c20 con 0x7fc5d4102210 2024-02-18T00:33:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5cc00f040 con 0x7fc5d4102210 2024-02-18T00:33:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0036c0 con 0x7fc5d4102210 2024-02-18T00:33:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5da783700 10 client.40226.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d4117110 con 0x7fc5d4102210 2024-02-18T00:33:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.368+0000 7fc5da783700 1 librados: init done 2024-02-18T00:33:53.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.369+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fc5c0004f30 con 0x7fc5d4102210 2024-02-18T00:33:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.371+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc5cc004180 con 0x7fc5d4102210 2024-02-18T00:33:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.371+0000 7fc5d17fa700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5bc062c90 0x7fc5bc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.371+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(154..154 src has 44..154) v4 ==== 9648+0+0 (secure 0 0 0) 0x7fc5cc07dc30 con 0x7fc5d4102210 2024-02-18T00:33:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.371+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 osd_map(154..154 src has 44..154) v4 2024-02-18T00:33:53.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.371+0000 7fc5d17fa700 3 client.40226.objecter handle_osd_map got epochs [154,154] > 0 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.371+0000 7fc5d17fa700 3 client.40226.objecter handle_osd_map decoding full epoch 154 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.371+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5bc062c90 0x7fc5bc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5d17fa700 20 client.40226.objecter dump_active .. 0 homeless 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v154) v1 ==== 54+0+16 (secure 0 0 0) 0x7fc5cc07e0b0 con 0x7fc5d4102210 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 10 client.40226.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 1 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5c0004560 0x7fc5c0004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 1 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 0 1 2024-02-18T00:33:53.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00051d0 osd=0 4 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 10 client.40226.objecter _send_command 1 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c0008960 con 0x7fc5c0004560 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5bc062c90 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5bc062c90 0x7fc5bc065150 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 rx=0x7fc5c4004d20 tx=0x7fc5c4009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5d37fe700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5c0004560 0x7fc5c0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c0004560 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.372+0000 7fc5d37fe700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5c0004560 0x7fc5c0004980 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.373+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c0008960 con 0x7fc5c0004560 2024-02-18T00:33:53.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.373+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:33:53.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.373+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 1 = system:0 2024-02-18T00:33:53.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.373+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c00051d0 osd=0 4 2024-02-18T00:33:53.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.373+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 0 1 2024-02-18T00:33:53.379 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 10 client.40226.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 2 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 2 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c00051d0 osd=0 3 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 0 2 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00051d0 osd=0 4 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 10 client.40226.objecter _send_command 2 2024-02-18T00:33:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.390+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c00081f0 con 0x7fc5c0004560 2024-02-18T00:33:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.391+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c00081f0 con 0x7fc5c0004560 2024-02-18T00:33:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.391+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.391+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.391+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c00051d0 osd=0 4 2024-02-18T00:33:53.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.391+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 0 2 2024-02-18T00:33:53.394 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:33:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 10 client.40226.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5c0002c90 0x7fc5c0003140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 1 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00085a0 osd=1 4 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 10 client.40226.objecter _send_command 3 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c0003630 con 0x7fc5c0002c90 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.392+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5c0002c90 0x7fc5c0003140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.393+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c0002c90 2024-02-18T00:33:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.393+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5c0002c90 0x7fc5c0003140 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.394+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c0003630 con 0x7fc5c0002c90 2024-02-18T00:33:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.394+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:33:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.394+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 3 = system:0 2024-02-18T00:33:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.394+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c00085a0 osd=1 4 2024-02-18T00:33:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.394+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 1 3 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 10 client.40226.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 4 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 4 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c00085a0 osd=1 3 2024-02-18T00:33:53.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 1 4 2024-02-18T00:33:53.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00085a0 osd=1 4 2024-02-18T00:33:53.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 10 client.40226.objecter _send_command 4 2024-02-18T00:33:53.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.412+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c0003fd0 con 0x7fc5c0002c90 2024-02-18T00:33:53.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c0003fd0 con 0x7fc5c0002c90 2024-02-18T00:33:53.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c00085a0 osd=1 4 2024-02-18T00:33:53.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 1 4 2024-02-18T00:33:53.417 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 10 client.40226.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 5 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5c0007dd0 0x7fc5c000d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 5 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 2 5 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0007c60 osd=2 4 2024-02-18T00:33:53.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 10 client.40226.objecter _send_command 5 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c000fe70 con 0x7fc5c0007dd0 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.415+0000 7fc5d37fe700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5c0007dd0 0x7fc5c000d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.416+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c0007dd0 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.416+0000 7fc5d37fe700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5c0007dd0 0x7fc5c000d000 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.416+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c000fe70 con 0x7fc5c0007dd0 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.416+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.416+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 5 = system:0 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.416+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0007c60 osd=2 4 2024-02-18T00:33:53.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.416+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 2 5 2024-02-18T00:33:53.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 10 client.40226.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 6 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 6 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0007c60 osd=2 3 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 2 6 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0007c60 osd=2 4 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 10 client.40226.objecter _send_command 6 2024-02-18T00:33:53.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.431+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c0003a40 con 0x7fc5c0007dd0 2024-02-18T00:33:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.432+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c0003a40 con 0x7fc5c0007dd0 2024-02-18T00:33:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.432+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.432+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.432+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0007c60 osd=2 4 2024-02-18T00:33:53.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.432+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 2 6 2024-02-18T00:33:53.435 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:33:53.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 10 client.40226.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 7 2024-02-18T00:33:53.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5c00020e0 0x7fc5c000e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 7 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 3 7 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0003e50 osd=3 4 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 10 client.40226.objecter _send_command 7 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c000ee20 con 0x7fc5c00020e0 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5d8d20700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5c00020e0 0x7fc5c000e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c00020e0 2024-02-18T00:33:53.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.433+0000 7fc5d8d20700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5c00020e0 0x7fc5c000e740 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.435+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c000ee20 con 0x7fc5c00020e0 2024-02-18T00:33:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.435+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:33:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.435+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 7 = system:0 2024-02-18T00:33:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.435+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0003e50 osd=3 4 2024-02-18T00:33:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.435+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 3 7 2024-02-18T00:33:53.446 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 10 client.40226.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 8 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 8 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0003e50 osd=3 3 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 3 8 2024-02-18T00:33:53.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0003e50 osd=3 4 2024-02-18T00:33:53.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 10 client.40226.objecter _send_command 8 2024-02-18T00:33:53.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.473+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c0001600 con 0x7fc5c00020e0 2024-02-18T00:33:53.476 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:53.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c0001600 con 0x7fc5c00020e0 2024-02-18T00:33:53.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0003e50 osd=3 4 2024-02-18T00:33:53.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 3 8 2024-02-18T00:33:53.479 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:33:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 10 client.40226.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 9 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5c0012050 0x7fc5c0014440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 9 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 4 9 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00013c0 osd=4 4 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 10 client.40226.objecter _send_command 9 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c0017090 con 0x7fc5c0012050 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.477+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5c0012050 0x7fc5c0014440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.478+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c0012050 2024-02-18T00:33:53.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.478+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5c0012050 0x7fc5c0014440 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.478+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c0017090 con 0x7fc5c0012050 2024-02-18T00:33:53.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.478+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:33:53.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.478+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 9 = system:0 2024-02-18T00:33:53.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.478+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c00013c0 osd=4 4 2024-02-18T00:33:53.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.478+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 4 9 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 10 client.40226.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 10 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 10 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c00013c0 osd=4 3 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 4 10 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00013c0 osd=4 4 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 10 client.40226.objecter _send_command 10 2024-02-18T00:33:53.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.493+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c0014930 con 0x7fc5c0012050 2024-02-18T00:33:53.496 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c0014930 con 0x7fc5c0012050 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c00013c0 osd=4 4 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 4 10 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 10 client.40226.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 11 2024-02-18T00:33:53.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5c0015040 0x7fc5c0017430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 11 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 5 11 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0014930 osd=5 4 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 10 client.40226.objecter _send_command 11 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c001a1b0 con 0x7fc5c0015040 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.495+0000 7fc5d37fe700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5c0015040 0x7fc5c0017430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.496+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c0015040 2024-02-18T00:33:53.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.496+0000 7fc5d37fe700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5c0015040 0x7fc5c0017430 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.496+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c001a1b0 con 0x7fc5c0015040 2024-02-18T00:33:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.496+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:33:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.496+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 11 = system:0 2024-02-18T00:33:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.496+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0014930 osd=5 4 2024-02-18T00:33:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.496+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 5 11 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 10 client.40226.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 12 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 12 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0014930 osd=5 3 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 5 12 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0014930 osd=5 4 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 10 client.40226.objecter _send_command 12 2024-02-18T00:33:53.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.511+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c001a1b0 con 0x7fc5c0015040 2024-02-18T00:33:53.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c001a1b0 con 0x7fc5c0015040 2024-02-18T00:33:53.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0014930 osd=5 4 2024-02-18T00:33:53.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 5 12 2024-02-18T00:33:53.515 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 10 client.40226.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 13 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5c0017ed0 0x7fc5c0018330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 13 2024-02-18T00:33:53.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 6 13 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c001a1b0 osd=6 4 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 10 client.40226.objecter _send_command 13 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c0019f60 con 0x7fc5c0017ed0 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.513+0000 7fc5d8d20700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5c0017ed0 0x7fc5c0018330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.514+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c0017ed0 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.514+0000 7fc5d8d20700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5c0017ed0 0x7fc5c0018330 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.514+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c0019f60 con 0x7fc5c0017ed0 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.514+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.514+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 13 = system:0 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.514+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c001a1b0 osd=6 4 2024-02-18T00:33:53.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.514+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 6 13 2024-02-18T00:33:53.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 10 client.40226.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 14 2024-02-18T00:33:53.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 14 2024-02-18T00:33:53.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c001a1b0 osd=6 3 2024-02-18T00:33:53.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 6 14 2024-02-18T00:33:53.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c001a1b0 osd=6 4 2024-02-18T00:33:53.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 10 client.40226.objecter _send_command 14 2024-02-18T00:33:53.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.540+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c0019f60 con 0x7fc5c0017ed0 2024-02-18T00:33:53.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c0019f60 con 0x7fc5c0017ed0 2024-02-18T00:33:53.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c001a1b0 osd=6 4 2024-02-18T00:33:53.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 6 14 2024-02-18T00:33:53.544 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:33:53.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 10 client.40226.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:33:53.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 15 2024-02-18T00:33:53.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5c001c920 0x7fc5c001cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:53.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 15 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 7 15 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0019f60 osd=7 4 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 10 client.40226.objecter _send_command 15 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.542+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5c001d3d0 con 0x7fc5c001c920 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.543+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5c001c920 0x7fc5c001cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.543+0000 7fc5d17fa700 10 client.40226.objecter ms_handle_connect 0x7fc5c001c920 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.543+0000 7fc5d3fff700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5c001c920 0x7fc5c001cd60 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.544+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5c001d3d0 con 0x7fc5c001c920 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.544+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.544+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 15 = system:0 2024-02-18T00:33:53.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.544+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0019f60 osd=7 4 2024-02-18T00:33:53.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.544+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 7 15 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 10 client.40226.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign -1 16 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 20 client.40226.objecter _get_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 15 client.40226.objecter _session_command_op_remove -1 16 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 20 client.40226.objecter get_session s=0x7fc5c0019f60 osd=7 3 2024-02-18T00:33:53.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 15 client.40226.objecter _session_command_op_assign 7 16 2024-02-18T00:33:53.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0019f60 osd=7 4 2024-02-18T00:33:53.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 10 client.40226.objecter _send_command 16 2024-02-18T00:33:53.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.558+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc5c001d3d0 con 0x7fc5c001c920 2024-02-18T00:33:53.561 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:33:53.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d17fa700 1 -- 172.21.15.59:0/177156093 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc5c001d3d0 con 0x7fc5c001c920 2024-02-18T00:33:53.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d17fa700 10 client.40226.objecter ms_dispatch 0x7fc5d4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:33:53.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d17fa700 10 client.40226.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d17fa700 20 client.40226.objecter put_session s=0x7fc5c0019f60 osd=7 4 2024-02-18T00:33:53.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d17fa700 15 client.40226.objecter _session_command_op_remove 7 16 2024-02-18T00:33:53.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 librados: watch_flush enter 2024-02-18T00:33:53.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 librados: watch_flush exit 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.0 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5c0004560 msgr2=0x7fc5c0004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5c0004560 0x7fc5c0004980 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00051d0 osd=0 1 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.1 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5c0002c90 msgr2=0x7fc5c0003140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5c0002c90 0x7fc5c0003140 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00085a0 osd=1 1 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.2 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5c0007dd0 msgr2=0x7fc5c000d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5c0007dd0 0x7fc5c000d000 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0007c60 osd=2 1 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.3 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5c00020e0 msgr2=0x7fc5c000e740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5c00020e0 0x7fc5c000e740 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0003e50 osd=3 1 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.4 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5c0012050 msgr2=0x7fc5c0014440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5c0012050 0x7fc5c0014440 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c00013c0 osd=4 1 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.5 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 reap_dead start 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5c0015040 msgr2=0x7fc5c0017430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5c0015040 0x7fc5c0017430 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 reap_dead start 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 reap_dead start 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0014930 osd=5 1 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.6 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5c0017ed0 msgr2=0x7fc5c0018330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5c0017ed0 0x7fc5c0018330 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c001a1b0 osd=6 1 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter close_session for osd.7 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5c001c920 msgr2=0x7fc5c001cd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5c001c920 0x7fc5c001cd60 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter put_session s=0x7fc5c0019f60 osd=7 1 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 20 client.40226.objecter shutdown clearing up homeless session... 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 10 client.40226.objecter successfully canceled tick 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5bc062c90 msgr2=0x7fc5bc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5bc062c90 0x7fc5bc065150 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 rx=0x7fc5c4004d20 tx=0x7fc5c4009040).stop 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d4102210 msgr2=0x7fc5d40f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5da783700 1 --2- 172.21.15.59:0/177156093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5d4102210 0x7fc5d40f7640 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 rx=0x7fc5cc00d940 tx=0x7fc5cc004e40).stop 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.560+0000 7fc5d8d20700 1 -- 172.21.15.59:0/177156093 reap_dead start 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.561+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 shutdown_connections 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.561+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 >> 172.21.15.59:0/177156093 conn(0x7fc5d40f2820 msgr2=0x7fc5d40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.561+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 shutdown_connections 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.561+0000 7fc5da783700 1 -- 172.21.15.59:0/177156093 wait complete. 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:53.561+0000 7fc5da783700 1 librados: shutdown 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:33:53.607 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:52.249113+0000 osd.6 (osd.6) 81 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:33:53.607 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:52.250844+0000 osd.6 (osd.6) 82 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:33:53.607 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:52.582772+0000 mon.a (mon.0) 1196 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:53.607 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:52.594099+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e154: 8 total, 8 up, 7 in 2024-02-18T00:33:53.607 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:52.624543+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:53.608 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:52.625081+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:53.608 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:53.371207+0000 mon.c (mon.2) 92 : audit [DBG] from='client.? 172.21.15.59:0/177156093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:53.608 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:53.464045+0000 mon.b (mon.1) 184 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:53.608 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:53.579356+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:53.608 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.579511+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e155: 8 total, 8 up, 7 in 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:52.249113+0000 osd.6 (osd.6) 81 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:52.250844+0000 osd.6 (osd.6) 82 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:52.582772+0000 mon.a (mon.0) 1196 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:52.594099+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e154: 8 total, 8 up, 7 in 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:52.624543+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:52.625081+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:53.371207+0000 mon.c (mon.2) 92 : audit [DBG] from='client.? 172.21.15.59:0/177156093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:53.464045+0000 mon.b (mon.1) 184 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:53.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:53.579356+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:53.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.579511+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e155: 8 total, 8 up, 7 in 2024-02-18T00:33:53.890 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-18T00:33:53.928 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:53.928 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:33:53.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:52.249113+0000 osd.6 (osd.6) 81 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:52.250844+0000 osd.6 (osd.6) 82 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:52.582772+0000 mon.a (mon.0) 1196 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:52.594099+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e154: 8 total, 8 up, 7 in 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:52.624543+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:52.625081+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:53.371207+0000 mon.c (mon.2) 92 : audit [DBG] from='client.? 172.21.15.59:0/177156093' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:33:53.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:53.464045+0000 mon.b (mon.1) 184 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:53.987 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:53.579356+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:53.987 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.579511+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e155: 8 total, 8 up, 7 in 2024-02-18T00:33:54.030 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:54.056 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:54.076 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:54.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:54.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:54.179 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:54.211 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:54.239 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:54.560 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:54.563 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:54.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:54.625 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.003772+0000 osd.2 (osd.2) 79 : cluster [DBG] 5.b scrub starts 2024-02-18T00:33:54.625 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.004894+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.b scrub ok 2024-02-18T00:33:54.625 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.100140+0000 osd.0 (osd.0) 93 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:33:54.625 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.101239+0000 osd.0 (osd.0) 94 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:33:54.625 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.318995+0000 osd.4 (osd.4) 107 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.320352+0000 osd.4 (osd.4) 108 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:53.589904+0000 mon.b (mon.1) 185 : audit [INF] from='client.? 172.21.15.59:0/3195038165' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:53.590982+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:53.611619+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.59:0/970717787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:53.612560+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:53.821667+0000 mgr.y (mgr.24454) 207 : cluster [DBG] pgmap v220: 406 pgs: 96 unknown, 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 304 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 12 MiB/s, 499 keys/s, 28 objects/s recovering 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:54.012555+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:54.013459+0000 osd.5 (osd.5) 86 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:54.194365+0000 osd.1 (osd.1) 93 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:54.197971+0000 osd.1 (osd.1) 94 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:54.465066+0000 mon.b (mon.1) 186 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:54.578824+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:54.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:54.579010+0000 mon.a (mon.0) 1204 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:54.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:54.579097+0000 mon.a (mon.0) 1205 : cluster [DBG] osdmap e156: 8 total, 8 up, 7 in 2024-02-18T00:33:54.664 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:54.695 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.003772+0000 osd.2 (osd.2) 79 : cluster [DBG] 5.b scrub starts 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.004894+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.b scrub ok 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.100140+0000 osd.0 (osd.0) 93 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.101239+0000 osd.0 (osd.0) 94 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.318995+0000 osd.4 (osd.4) 107 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.320352+0000 osd.4 (osd.4) 108 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:53.589904+0000 mon.b (mon.1) 185 : audit [INF] from='client.? 172.21.15.59:0/3195038165' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:53.590982+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:53.611619+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.59:0/970717787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:53.612560+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:53.821667+0000 mgr.y (mgr.24454) 207 : cluster [DBG] pgmap v220: 406 pgs: 96 unknown, 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 304 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 12 MiB/s, 499 keys/s, 28 objects/s recovering 2024-02-18T00:33:54.705 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:54.012555+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:33:54.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:54.013459+0000 osd.5 (osd.5) 86 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:33:54.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:54.194365+0000 osd.1 (osd.1) 93 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:33:54.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:54.197971+0000 osd.1 (osd.1) 94 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:33:54.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:54.465066+0000 mon.b (mon.1) 186 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:54.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:54.578824+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:54.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:54.579010+0000 mon.a (mon.0) 1204 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:54.706 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:54.579097+0000 mon.a (mon.0) 1205 : cluster [DBG] osdmap e156: 8 total, 8 up, 7 in 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.003772+0000 osd.2 (osd.2) 79 : cluster [DBG] 5.b scrub starts 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.004894+0000 osd.2 (osd.2) 80 : cluster [DBG] 5.b scrub ok 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.100140+0000 osd.0 (osd.0) 93 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.101239+0000 osd.0 (osd.0) 94 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.318995+0000 osd.4 (osd.4) 107 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.320352+0000 osd.4 (osd.4) 108 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:53.589904+0000 mon.b (mon.1) 185 : audit [INF] from='client.? 172.21.15.59:0/3195038165' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:53.590982+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:53.611619+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.59:0/970717787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:53.612560+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:53.821667+0000 mgr.y (mgr.24454) 207 : cluster [DBG] pgmap v220: 406 pgs: 96 unknown, 4 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 304 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 0 B/s rd, 0 op/s; 12 MiB/s, 499 keys/s, 28 objects/s recovering 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:54.012555+0000 osd.5 (osd.5) 85 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:54.013459+0000 osd.5 (osd.5) 86 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:54.194365+0000 osd.1 (osd.1) 93 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:54.197971+0000 osd.1 (osd.1) 94 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:54.465066+0000 mon.b (mon.1) 186 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:54.578824+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi059-82877-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:54.579010+0000 mon.a (mon.0) 1204 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi059-82882-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:55.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:54.579097+0000 mon.a (mon.0) 1205 : cluster [DBG] osdmap e156: 8 total, 8 up, 7 in 2024-02-18T00:33:55.046 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:55.048 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:55.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:55.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:55.179 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:55.829 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:55.832 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:55.833 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:55.934 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:55.962 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:55.999 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:56.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:56.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:56.102 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:56.126 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:33:56.126 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:33:56.126 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:33:56.126 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:33:56.133 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:56.155 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:56.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:56.256 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 -- 172.21.15.59:0/3389566542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380f8420 msgr2=0x7f8838104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 --2- 172.21.15.59:0/3389566542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380f8420 0x7f8838104a20 secure :-1 s=READY pgs=4022 cs=0 l=1 rev1=1 rx=0x7f8830009c30 tx=0x7f8830004c50).stop 2024-02-18T00:33:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 -- 172.21.15.59:0/3389566542 shutdown_connections 2024-02-18T00:33:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 --2- 172.21.15.59:0/3389566542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380f8420 0x7f8838104a20 unknown :-1 s=CLOSED pgs=4022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 --2- 172.21.15.59:0/3389566542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380fdbb0 0x7f88380fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 --2- 172.21.15.59:0/3389566542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380fe510 0x7f88380f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 -- 172.21.15.59:0/3389566542 >> 172.21.15.59:0/3389566542 conn(0x7f88380f3800 msgr2=0x7f88380f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 -- 172.21.15.59:0/3389566542 shutdown_connections 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.260+0000 7f883e313700 1 -- 172.21.15.59:0/3389566542 wait complete. 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 librados: starting msgr at 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 librados: starting objecter 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 Processor -- start 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 -- start start 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 librados: setting wanted keys 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 librados: calling monclient init 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380f8420 0x7f883806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380fdbb0 0x7f883806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380fe510 0x7f883806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838116480 con 0x7f88380fdbb0 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381165f0 con 0x7f88380fe510 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883e313700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88381167b0 con 0x7f88380f8420 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f8837fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380f8420 0x7f883806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f882ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380fdbb0 0x7f883806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f8837fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380f8420 0x7f883806b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:52696/0 (socket says 172.21.15.59:52696) 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f8837fff700 1 -- 172.21.15.59:0/3726603093 learned_addr learned my addr 172.21.15.59:0/3726603093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.261+0000 7f883c8b0700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380fe510 0x7f883806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8837fff700 1 -- 172.21.15.59:0/3726603093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380fe510 msgr2=0x7f883806bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8837fff700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380fe510 0x7f883806bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8837fff700 1 -- 172.21.15.59:0/3726603093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380fdbb0 msgr2=0x7f883806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8837fff700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380fdbb0 0x7f883806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8837fff700 1 -- 172.21.15.59:0/3726603093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300097c0 con 0x7f88380f8420 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8837fff700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380f8420 0x7f883806b0d0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 rx=0x7f882800e990 tx=0x7f882800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8835ffb700 10 client.?.objecter ms_handle_connect 0x7f88380f8420 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8835ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8835ffb700 1 -- 172.21.15.59:0/3726603093 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828004680 con 0x7f88380f8420 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8835ffb700 1 -- 172.21.15.59:0/3726603093 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882800f040 con 0x7f88380f8420 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8838116a30 con 0x7f88380f8420 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f8835ffb700 1 -- 172.21.15.59:0/3726603093 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280146b0 con 0x7f88380f8420 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.262+0000 7f883e313700 10 client.40244.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:33:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.263+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8838116ea0 con 0x7f88380f8420 2024-02-18T00:33:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.263+0000 7f883e313700 1 librados: init done 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.263+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883805ee80 con 0x7f88380f8420 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.264+0000 7f8835ffb700 1 -- 172.21.15.59:0/3726603093 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8828014ae0 con 0x7f88380f8420 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.264+0000 7f8835ffb700 1 --2- 172.21.15.59:0/3726603093 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8824062c90 0x7f8824065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.264+0000 7f8835ffb700 1 -- 172.21.15.59:0/3726603093 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(157..157 src has 44..157) v4 ==== 9617+0+0 (secure 0 0 0) 0x7f882807d390 con 0x7f88380f8420 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.264+0000 7f8835ffb700 10 client.40244.objecter ms_dispatch 0x7f8838001550 osd_map(157..157 src has 44..157) v4 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.264+0000 7f8835ffb700 3 client.40244.objecter handle_osd_map got epochs [157,157] > 0 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.264+0000 7f8835ffb700 3 client.40244.objecter handle_osd_map decoding full epoch 157 2024-02-18T00:33:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.264+0000 7f882ffff700 1 --2- 172.21.15.59:0/3726603093 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8824062c90 0x7f8824065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:33:56.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.265+0000 7f8835ffb700 20 client.40244.objecter dump_active .. 0 homeless 2024-02-18T00:33:56.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.265+0000 7f8835ffb700 10 client.40244.objecter ms_handle_connect 0x7f8824062c90 2024-02-18T00:33:56.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.265+0000 7f882ffff700 1 --2- 172.21.15.59:0/3726603093 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8824062c90 0x7f8824065150 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 rx=0x7f883806c870 tx=0x7f882000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:33:56.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.269+0000 7f8835ffb700 1 -- 172.21.15.59:0/3726603093 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8828049f60 con 0x7f88380f8420 2024-02-18T00:33:56.283 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:56.381 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:56.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:56.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:56.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:33:56.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.453+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f883805fa60 con 0x7f88380f8420 2024-02-18T00:33:56.484 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:33:56.511 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:56.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.585+0000 7f8835ffb700 1 -- 172.21.15.59:0/3726603093 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v158) v1 ==== 87+0+0 (secure 0 0 0) 0x7f8828050320 con 0x7f88380f8420 2024-02-18T00:33:56.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.592+0000 7f883e313700 10 librados: watch_flush enter 2024-02-18T00:33:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.594+0000 7f883e313700 10 librados: watch_flush exit 2024-02-18T00:33:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.594+0000 7f883e313700 20 client.40244.objecter shutdown clearing up homeless session... 2024-02-18T00:33:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.594+0000 7f883e313700 10 client.40244.objecter successfully canceled tick 2024-02-18T00:33:56.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.597+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8824062c90 msgr2=0x7f8824065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.597+0000 7f883e313700 1 --2- 172.21.15.59:0/3726603093 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8824062c90 0x7f8824065150 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 rx=0x7f883806c870 tx=0x7f882000b040).stop 2024-02-18T00:33:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.597+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380f8420 msgr2=0x7f883806b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:33:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.597+0000 7f883e313700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380f8420 0x7f883806b0d0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 rx=0x7f882800e990 tx=0x7f882800b040).stop 2024-02-18T00:33:56.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.604+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 shutdown_connections 2024-02-18T00:33:56.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.604+0000 7f883e313700 1 --2- 172.21.15.59:0/3726603093 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8824062c90 0x7f8824065150 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.604+0000 7f883e313700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88380fe510 0x7f883806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.604+0000 7f883e313700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88380f8420 0x7f883806b0d0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.604+0000 7f883e313700 1 --2- 172.21.15.59:0/3726603093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88380fdbb0 0x7f883806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:33:56.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.604+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 >> 172.21.15.59:0/3726603093 conn(0x7f88380f3800 msgr2=0x7f88380f50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:33:56.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.610+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 shutdown_connections 2024-02-18T00:33:56.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.610+0000 7f883e313700 1 -- 172.21.15.59:0/3726603093 wait complete. 2024-02-18T00:33:56.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:33:56.610+0000 7f883e313700 1 librados: shutdown 2024-02-18T00:33:56.613 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:33:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:54.752016+0000 osd.3 (osd.3) 111 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:33:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:54.752981+0000 osd.3 (osd.3) 112 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:33:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:54.887655+0000 mon.a (mon.0) 1206 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:55.466204+0000 mon.b (mon.1) 187 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.578853+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e157: 8 total, 8 up, 7 in 2024-02-18T00:33:56.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:54.752016+0000 osd.3 (osd.3) 111 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:33:56.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:54.752981+0000 osd.3 (osd.3) 112 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:33:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:54.887655+0000 mon.a (mon.0) 1206 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:55.466204+0000 mon.b (mon.1) 187 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.578853+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e157: 8 total, 8 up, 7 in 2024-02-18T00:33:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:54.752016+0000 osd.3 (osd.3) 111 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:33:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:54.752981+0000 osd.3 (osd.3) 112 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:33:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:54.887655+0000 mon.a (mon.0) 1206 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:33:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:55.466204+0000 mon.b (mon.1) 187 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.578853+0000 mon.a (mon.0) 1207 : cluster [DBG] osdmap e157: 8 total, 8 up, 7 in 2024-02-18T00:33:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:56.939 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-18T00:33:56.960 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:33:56.962 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:56.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:33:57.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:33:57.092 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:57.224 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:33:57.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.264517+0000 osd.6 (osd.6) 83 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:33:57.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.265369+0000 osd.6 (osd.6) 84 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:33:57.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.269020+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:57.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.269281+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:57.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.782127+0000 osd.3 (osd.3) 113 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.782952+0000 osd.3 (osd.3) 114 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:55.822688+0000 mgr.y (mgr.24454) 208 : cluster [DBG] pgmap v223: 342 pgs: 342 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 5.2 KiB/s rd, 79 KiB/s wr, 76 op/s 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:56.117620+0000 osd.0 (osd.0) 95 : cluster [DBG] 4.1e deep-scrub starts 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:56.121867+0000 osd.0 (osd.0) 96 : cluster [DBG] 4.1e deep-scrub ok 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:56.295447+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:56.296271+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.454478+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.59:0/3726603093' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.467381+0000 mon.b (mon.1) 188 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.514426+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:56.576875+0000 mon.a (mon.0) 1209 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.585269+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:56.585368+0000 mon.a (mon.0) 1211 : cluster [DBG] osdmap e158: 8 total, 8 up, 7 in 2024-02-18T00:33:57.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.589283+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.59:0/2445655238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.264517+0000 osd.6 (osd.6) 83 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.265369+0000 osd.6 (osd.6) 84 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.269020+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.269281+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.782127+0000 osd.3 (osd.3) 113 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.782952+0000 osd.3 (osd.3) 114 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:55.822688+0000 mgr.y (mgr.24454) 208 : cluster [DBG] pgmap v223: 342 pgs: 342 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 5.2 KiB/s rd, 79 KiB/s wr, 76 op/s 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:56.117620+0000 osd.0 (osd.0) 95 : cluster [DBG] 4.1e deep-scrub starts 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:56.121867+0000 osd.0 (osd.0) 96 : cluster [DBG] 4.1e deep-scrub ok 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:56.295447+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:33:57.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:56.296271+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.454478+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.59:0/3726603093' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.467381+0000 mon.b (mon.1) 188 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.514426+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:56.576875+0000 mon.a (mon.0) 1209 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.585269+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:56.585368+0000 mon.a (mon.0) 1211 : cluster [DBG] osdmap e158: 8 total, 8 up, 7 in 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.589283+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.59:0/2445655238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.594191+0000 mon.b (mon.1) 189 : audit [INF] from='client.? 172.21.15.59:0/3396975013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.602663+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:56.602935+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.468409+0000 mon.b (mon.1) 190 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.469264+0000 mon.b (mon.1) 191 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.469476+0000 mon.a (mon.0) 1214 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.594191+0000 mon.b (mon.1) 189 : audit [INF] from='client.? 172.21.15.59:0/3396975013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.602663+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:56.602935+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.468409+0000 mon.b (mon.1) 190 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.469264+0000 mon.b (mon.1) 191 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:33:57.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.469476+0000 mon.a (mon.0) 1214 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:33:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:33:57.745 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:57.745 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.264517+0000 osd.6 (osd.6) 83 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.265369+0000 osd.6 (osd.6) 84 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.269020+0000 osd.1 (osd.1) 95 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.269281+0000 osd.1 (osd.1) 96 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.782127+0000 osd.3 (osd.3) 113 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.782952+0000 osd.3 (osd.3) 114 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:55.822688+0000 mgr.y (mgr.24454) 208 : cluster [DBG] pgmap v223: 342 pgs: 342 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 5.2 KiB/s rd, 79 KiB/s wr, 76 op/s 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:56.117620+0000 osd.0 (osd.0) 95 : cluster [DBG] 4.1e deep-scrub starts 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:56.121867+0000 osd.0 (osd.0) 96 : cluster [DBG] 4.1e deep-scrub ok 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:56.295447+0000 osd.6 (osd.6) 85 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:56.296271+0000 osd.6 (osd.6) 86 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:33:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.454478+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.59:0/3726603093' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.467381+0000 mon.b (mon.1) 188 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.514426+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:56.576875+0000 mon.a (mon.0) 1209 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.585269+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:56.585368+0000 mon.a (mon.0) 1211 : cluster [DBG] osdmap e158: 8 total, 8 up, 7 in 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.589283+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.59:0/2445655238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.594191+0000 mon.b (mon.1) 189 : audit [INF] from='client.? 172.21.15.59:0/3396975013' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.602663+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:56.602935+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.468409+0000 mon.b (mon.1) 190 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.469264+0000 mon.b (mon.1) 191 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:33:57.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.469476+0000 mon.a (mon.0) 1214 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:33:57.801 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:33:57.801 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:33:57.801 INFO:tasks.thrashosds.thrasher:fixing pg num pool .rgw.root 2024-02-18T00:33:57.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:33:57.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:57.874 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:57.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:57.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:58.076 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:33:58.103 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:58.176 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:33:58.184 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:58.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:33:58.286 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:58.313 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:58.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:58.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:58.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:58.904 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:33:58.923 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:33:58.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:58.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:33:59.008 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:33:59.008 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:33:59.008 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:33:59.008 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:33:59.008 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:33:59.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:59.162 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:59.165 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:59.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:59.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:33:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:59.394 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:33:59.449 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:33:59.452 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:59.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:33:59.553 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:33:59.584 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:59.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.587653+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:59.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.587754+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:59.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.587967+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]': finished 2024-02-18T00:33:59.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:57.588041+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e159: 8 total, 8 up, 7 in 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.593912+0000 mon.b (mon.1) 192 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.600876+0000 mon.b (mon.1) 193 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.605046+0000 mon.b (mon.1) 194 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.607422+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.755173+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.755551+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:57.823235+0000 mgr.y (mgr.24454) 209 : cluster [DBG] pgmap v226: 406 pgs: 64 unknown, 342 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 5.2 KiB/s rd, 79 KiB/s wr, 76 op/s 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:57.825324+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.576409+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.576556+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.576607+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-18T00:33:59.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:58.576641+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e160: 8 total, 8 up, 7 in 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.587653+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.587754+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.587967+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]': finished 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:57.588041+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e159: 8 total, 8 up, 7 in 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.593912+0000 mon.b (mon.1) 192 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.600876+0000 mon.b (mon.1) 193 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.605046+0000 mon.b (mon.1) 194 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.607422+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.755173+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.755551+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:57.823235+0000 mgr.y (mgr.24454) 209 : cluster [DBG] pgmap v226: 406 pgs: 64 unknown, 342 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 5.2 KiB/s rd, 79 KiB/s wr, 76 op/s 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:57.825324+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-18T00:33:59.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.576409+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T00:33:59.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.576556+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:59.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.576607+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-18T00:33:59.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:33:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:58.576641+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e160: 8 total, 8 up, 7 in 2024-02-18T00:33:59.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.587653+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi059-82882-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:59.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.587754+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi059-82877-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:33:59.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.587967+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]': finished 2024-02-18T00:33:59.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:57.588041+0000 mon.a (mon.0) 1218 : cluster [DBG] osdmap e159: 8 total, 8 up, 7 in 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.593912+0000 mon.b (mon.1) 192 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.600876+0000 mon.b (mon.1) 193 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.605046+0000 mon.b (mon.1) 194 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.607422+0000 mon.a (mon.0) 1219 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.755173+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.755551+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:57.823235+0000 mgr.y (mgr.24454) 209 : cluster [DBG] pgmap v226: 406 pgs: 64 unknown, 342 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 5.2 KiB/s rd, 79 KiB/s wr, 76 op/s 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:57.825324+0000 mon.a (mon.0) 1221 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.576409+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.576556+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.576607+0000 mon.a (mon.0) 1224 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pgp_num_actual", "val": "11"}]': finished 2024-02-18T00:33:59.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:33:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:58.576641+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e160: 8 total, 8 up, 7 in 2024-02-18T00:33:59.777 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-18T00:33:59.792 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:33:59.794 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:33:59.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:33:59.895 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:33:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:33:59.998 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:00.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:00.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:00.060 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 -- 172.21.15.59:0/1011425370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 msgr2=0x7fd3100fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 --2- 172.21.15.59:0/1011425370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd3100fdf90 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 rx=0x7fd30c004870 tx=0x7fd30c009cb0).stop 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 -- 172.21.15.59:0/1011425370 shutdown_connections 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 --2- 172.21.15.59:0/1011425370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3100f8420 0x7fd310104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 --2- 172.21.15.59:0/1011425370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd3100fdf90 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 --2- 172.21.15.59:0/1011425370 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3100fe510 0x7fd3100f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 -- 172.21.15.59:0/1011425370 >> 172.21.15.59:0/1011425370 conn(0x7fd3100f3800 msgr2=0x7fd3100f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 -- 172.21.15.59:0/1011425370 shutdown_connections 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.092+0000 7fd317ee3700 1 -- 172.21.15.59:0/1011425370 wait complete. 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 librados: starting msgr at 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 librados: starting objecter 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 Processor -- start 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 -- start start 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 librados: setting wanted keys 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 librados: calling monclient init 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3100f8420 0x7fd31010e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd31010e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3100fe510 0x7fd310108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd310108770 con 0x7fd3100fdbb0 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3101088e0 con 0x7fd3100fe510 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.093+0000 7fd317ee3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd310108aa0 con 0x7fd3100f8420 2024-02-18T00:34:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd315c7f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3100f8420 0x7fd31010e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd31010e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd315c7f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3100f8420 0x7fd31010e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38980/0 (socket says 172.21.15.59:38980) 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd31010e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37308/0 (socket says 172.21.15.59:37308) 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd315c7f700 1 -- 172.21.15.59:0/3810994920 learned_addr learned my addr 172.21.15.59:0/3810994920 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd316480700 1 --2- 172.21.15.59:0/3810994920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3100fe510 0x7fd310108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 -- 172.21.15.59:0/3810994920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3100f8420 msgr2=0x7fd31010e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 --2- 172.21.15.59:0/3810994920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3100f8420 0x7fd31010e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 -- 172.21.15.59:0/3810994920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3100fe510 msgr2=0x7fd310108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 --2- 172.21.15.59:0/3810994920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3100fe510 0x7fd310108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 -- 172.21.15.59:0/3810994920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd300005040 con 0x7fd3100fdbb0 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3100f8420 0x7fd31010e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd31547e700 1 --2- 172.21.15.59:0/3810994920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd31010e580 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 rx=0x7fd300004920 tx=0x7fd300009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd306ffd700 10 client.?.objecter ms_handle_connect 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd306ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000e030 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd30000f040 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c005040 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300014850 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd317ee3700 10 client.30438.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3101090e0 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.094+0000 7fd317ee3700 1 librados: init done 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.095+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd310107060 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd300010500 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd2fc062c90 0x7fd2fc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(161..161 src has 44..161) v4 ==== 9555+0+0 (secure 0 0 0) 0x7fd30007e240 con 0x7fd3100fdbb0 2024-02-18T00:34:00.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 osd_map(161..161 src has 44..161) v4 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 3 client.30438.objecter handle_osd_map got epochs [161,161] > 0 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 3 client.30438.objecter handle_osd_map decoding full epoch 161 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd2fc062c90 0x7fd2fc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 20 client.30438.objecter dump_active .. 0 homeless 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.096+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v161) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd30007e6e0 con 0x7fd3100fdbb0 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 10 client.30438.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 1 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd2fc062c90 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd2fc062c90 0x7fd2fc065150 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 rx=0x7fd30c009a00 tx=0x7fd30c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd3100fee60 0x7fd3100ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 1 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 0 1 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310109710 osd=0 4 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 10 client.30438.objecter _send_command 1 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3100ff730 con 0x7fd3100fee60 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd316480700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd3100fee60 0x7fd3100ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd3100fee60 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.097+0000 7fd316480700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd3100fee60 0x7fd3100ff240 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.098+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd3100ff730 con 0x7fd3100fee60 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.098+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.098+0000 7fd306ffd700 10 client.30438.objecter _finish_command 1 = system:0 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.098+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310109710 osd=0 4 2024-02-18T00:34:00.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.098+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 0 1 2024-02-18T00:34:00.108 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 10 client.30438.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 2 2024-02-18T00:34:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 2 2024-02-18T00:34:00.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310109710 osd=0 3 2024-02-18T00:34:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 0 2 2024-02-18T00:34:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310109710 osd=0 4 2024-02-18T00:34:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 10 client.30438.objecter _send_command 2 2024-02-18T00:34:00.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.113+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd3100ff8f0 con 0x7fd3100fee60 2024-02-18T00:34:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.115+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd3100ff8f0 con 0x7fd3100fee60 2024-02-18T00:34:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.115+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.115+0000 7fd306ffd700 10 client.30438.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.115+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310109710 osd=0 4 2024-02-18T00:34:00.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.115+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 0 2 2024-02-18T00:34:00.117 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:34:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 10 client.30438.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6805/26279110 conn(0x7fd310028540 0x7fd310118f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 1 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3100fd4a0 osd=1 4 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 10 client.30438.objecter _send_command 3 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3100fd610 con 0x7fd310028540 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6805/26279110 conn(0x7fd310028540 0x7fd310118f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd310028540 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.116+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6805/26279110 conn(0x7fd310028540 0x7fd310118f40 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.117+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd3100fd610 con 0x7fd310028540 2024-02-18T00:34:00.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.117+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:34:00.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.117+0000 7fd306ffd700 10 client.30438.objecter _finish_command 3 = system:0 2024-02-18T00:34:00.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.117+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd3100fd4a0 osd=1 4 2024-02-18T00:34:00.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.117+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 1 3 2024-02-18T00:34:00.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 10 client.30438.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 4 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 4 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd3100fd4a0 osd=1 3 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 1 4 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3100fd4a0 osd=1 4 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 10 client.30438.objecter _send_command 4 2024-02-18T00:34:00.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.132+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd31005a040 con 0x7fd310028540 2024-02-18T00:34:00.135 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd31005a040 con 0x7fd310028540 2024-02-18T00:34:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd306ffd700 10 client.30438.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd3100fd4a0 osd=1 4 2024-02-18T00:34:00.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 1 4 2024-02-18T00:34:00.137 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:34:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 10 client.30438.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 5 2024-02-18T00:34:00.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd3100594d0 0x7fd31011c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 5 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 2 5 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310059360 osd=2 4 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 10 client.30438.objecter _send_command 5 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.134+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3100280a0 con 0x7fd3100594d0 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd31547e700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd3100594d0 0x7fd31011c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd3100594d0 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd31547e700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd3100594d0 0x7fd31011c5c0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd3100280a0 con 0x7fd3100594d0 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:34:00.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd306ffd700 10 client.30438.objecter _finish_command 5 = system:0 2024-02-18T00:34:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310059360 osd=2 4 2024-02-18T00:34:00.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.135+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 2 5 2024-02-18T00:34:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 10 client.30438.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 6 2024-02-18T00:34:00.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 6 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310059360 osd=2 3 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 2 6 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310059360 osd=2 4 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 10 client.30438.objecter _send_command 6 2024-02-18T00:34:00.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.150+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd31011cab0 con 0x7fd3100594d0 2024-02-18T00:34:00.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd31011cab0 con 0x7fd3100594d0 2024-02-18T00:34:00.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd306ffd700 10 client.30438.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310059360 osd=2 4 2024-02-18T00:34:00.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 2 6 2024-02-18T00:34:00.153 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 10 client.30438.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 7 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd310027770 0x7fd31011fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 7 2024-02-18T00:34:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 3 7 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310027600 osd=3 4 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 10 client.30438.objecter _send_command 7 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd31011a2b0 con 0x7fd310027770 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.151+0000 7fd316480700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd310027770 0x7fd31011fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.152+0000 7fd316480700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd310027770 0x7fd31011fe10 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.152+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd310027770 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.152+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd31011a2b0 con 0x7fd310027770 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.152+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.152+0000 7fd306ffd700 10 client.30438.objecter _finish_command 7 = system:0 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.152+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310027600 osd=3 4 2024-02-18T00:34:00.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.152+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 3 7 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 10 client.30438.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 8 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 8 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310027600 osd=3 3 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 3 8 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310027600 osd=3 4 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 10 client.30438.objecter _send_command 8 2024-02-18T00:34:00.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.167+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd310119670 con 0x7fd310027770 2024-02-18T00:34:00.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.168+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd310119670 con 0x7fd310027770 2024-02-18T00:34:00.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.168+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.168+0000 7fd306ffd700 10 client.30438.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.168+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310027600 osd=3 4 2024-02-18T00:34:00.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.168+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 3 8 2024-02-18T00:34:00.170 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:34:00.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 10 client.30438.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 9 2024-02-18T00:34:00.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd31011d900 0x7fd310123490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 9 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 4 9 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310119430 osd=4 4 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 10 client.30438.objecter _send_command 9 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3101195a0 con 0x7fd31011d900 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd31011d900 0x7fd310123490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd31011d900 0x7fd310123490 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.169+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd31011d900 2024-02-18T00:34:00.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.170+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd3101195a0 con 0x7fd31011d900 2024-02-18T00:34:00.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.170+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:34:00.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.170+0000 7fd306ffd700 10 client.30438.objecter _finish_command 9 = system:0 2024-02-18T00:34:00.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.170+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310119430 osd=4 4 2024-02-18T00:34:00.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.170+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 4 9 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 10 client.30438.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 10 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 10 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310119430 osd=4 3 2024-02-18T00:34:00.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 4 10 2024-02-18T00:34:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310119430 osd=4 4 2024-02-18T00:34:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 10 client.30438.objecter _send_command 10 2024-02-18T00:34:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.185+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd3101195a0 con 0x7fd31011d900 2024-02-18T00:34:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.186+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd3101195a0 con 0x7fd31011d900 2024-02-18T00:34:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.186+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.186+0000 7fd306ffd700 10 client.30438.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.186+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310119430 osd=4 4 2024-02-18T00:34:00.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.186+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 4 10 2024-02-18T00:34:00.189 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:34:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 10 client.30438.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 11 2024-02-18T00:34:00.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd310123e90 0x7fd3101262a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 11 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 5 11 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3101195a0 osd=5 4 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 10 client.30438.objecter _send_command 11 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd310129070 con 0x7fd310123e90 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd31547e700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd310123e90 0x7fd3101262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd310123e90 2024-02-18T00:34:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.187+0000 7fd31547e700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd310123e90 0x7fd3101262a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.188+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd310129070 con 0x7fd310123e90 2024-02-18T00:34:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.188+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:34:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.188+0000 7fd306ffd700 10 client.30438.objecter _finish_command 11 = system:0 2024-02-18T00:34:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.188+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd3101195a0 osd=5 4 2024-02-18T00:34:00.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.188+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 5 11 2024-02-18T00:34:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 10 client.30438.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 12 2024-02-18T00:34:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 12 2024-02-18T00:34:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd3101195a0 osd=5 3 2024-02-18T00:34:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 5 12 2024-02-18T00:34:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3101195a0 osd=5 4 2024-02-18T00:34:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 10 client.30438.objecter _send_command 12 2024-02-18T00:34:00.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.202+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd310129070 con 0x7fd310123e90 2024-02-18T00:34:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd310129070 con 0x7fd310123e90 2024-02-18T00:34:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd306ffd700 10 client.30438.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd3101195a0 osd=5 4 2024-02-18T00:34:00.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 5 12 2024-02-18T00:34:00.207 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:34:00.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 10 client.30438.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 13 2024-02-18T00:34:00.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd310128370 0x7fd3101287f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 13 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 6 13 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310129070 osd=6 4 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 10 client.30438.objecter _send_command 13 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd310128e60 con 0x7fd310128370 2024-02-18T00:34:00.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.204+0000 7fd316480700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd310128370 0x7fd3101287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.205+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd310128370 2024-02-18T00:34:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.205+0000 7fd316480700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd310128370 0x7fd3101287f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.205+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd310128e60 con 0x7fd310128370 2024-02-18T00:34:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.205+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:34:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.205+0000 7fd306ffd700 10 client.30438.objecter _finish_command 13 = system:0 2024-02-18T00:34:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.205+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310129070 osd=6 4 2024-02-18T00:34:00.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.205+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 6 13 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 10 client.30438.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 14 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 14 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310129070 osd=6 3 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 6 14 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310129070 osd=6 4 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 10 client.30438.objecter _send_command 14 2024-02-18T00:34:00.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.220+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd310128e60 con 0x7fd310128370 2024-02-18T00:34:00.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd310128e60 con 0x7fd310128370 2024-02-18T00:34:00.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd306ffd700 10 client.30438.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310129070 osd=6 4 2024-02-18T00:34:00.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 6 14 2024-02-18T00:34:00.224 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 10 client.30438.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 15 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6812/53025171 conn(0x7fd31012b7e0 0x7fd31012bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 15 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 7 15 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310128e60 osd=7 4 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 10 client.30438.objecter _send_command 15 2024-02-18T00:34:00.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd31012c2b0 con 0x7fd31012b7e0 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.221+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6812/53025171 conn(0x7fd31012b7e0 0x7fd31012bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.222+0000 7fd315c7f700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6812/53025171 conn(0x7fd31012b7e0 0x7fd31012bc40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.222+0000 7fd306ffd700 10 client.30438.objecter ms_handle_connect 0x7fd31012b7e0 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.222+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd31012c2b0 con 0x7fd31012b7e0 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.223+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.223+0000 7fd306ffd700 10 client.30438.objecter _finish_command 15 = system:0 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.223+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310128e60 osd=7 4 2024-02-18T00:34:00.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.223+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 7 15 2024-02-18T00:34:00.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 10 client.30438.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign -1 16 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 20 client.30438.objecter _get_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_remove -1 16 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 20 client.30438.objecter get_session s=0x7fd310128e60 osd=7 3 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 15 client.30438.objecter _session_command_op_assign 7 16 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310128e60 osd=7 4 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 10 client.30438.objecter _send_command 16 2024-02-18T00:34:00.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.237+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fd31012c2b0 con 0x7fd31012b7e0 2024-02-18T00:34:00.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd306ffd700 1 -- 172.21.15.59:0/3810994920 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fd31012c2b0 con 0x7fd31012b7e0 2024-02-18T00:34:00.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd306ffd700 10 client.30438.objecter ms_dispatch 0x7fd310001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:00.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd306ffd700 10 client.30438.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd306ffd700 20 client.30438.objecter put_session s=0x7fd310128e60 osd=7 4 2024-02-18T00:34:00.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd306ffd700 15 client.30438.objecter _session_command_op_remove 7 16 2024-02-18T00:34:00.241 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 librados: watch_flush enter 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 librados: watch_flush exit 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.0 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd3100fee60 msgr2=0x7fd3100ff240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd3100fee60 0x7fd3100ff240 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310109710 osd=0 1 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.1 2024-02-18T00:34:00.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6805/26279110 conn(0x7fd310028540 msgr2=0x7fd310118f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6805/26279110 conn(0x7fd310028540 0x7fd310118f40 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3100fd4a0 osd=1 1 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.2 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd3100594d0 msgr2=0x7fd31011c5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd3100594d0 0x7fd31011c5c0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310059360 osd=2 1 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.3 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd310027770 msgr2=0x7fd31011fe10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd310027770 0x7fd31011fe10 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd316480700 1 -- 172.21.15.59:0/3810994920 reap_dead start 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310027600 osd=3 1 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.4 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd31011d900 msgr2=0x7fd310123490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd31011d900 0x7fd310123490 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310119430 osd=4 1 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.5 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd310123e90 msgr2=0x7fd3101262a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd310123e90 0x7fd3101262a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd3101195a0 osd=5 1 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.6 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd310128370 msgr2=0x7fd3101287f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd310128370 0x7fd3101287f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310129070 osd=6 1 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter close_session for osd.7 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6812/53025171 conn(0x7fd31012b7e0 msgr2=0x7fd31012bc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.137:6812/53025171 conn(0x7fd31012b7e0 0x7fd31012bc40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter put_session s=0x7fd310128e60 osd=7 1 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 20 client.30438.objecter shutdown clearing up homeless session... 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 10 client.30438.objecter successfully canceled tick 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd316480700 1 -- 172.21.15.59:0/3810994920 reap_dead start 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd316480700 1 -- 172.21.15.59:0/3810994920 reap_dead start 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd316480700 1 -- 172.21.15.59:0/3810994920 reap_dead start 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd2fc062c90 msgr2=0x7fd2fc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd2fc062c90 0x7fd2fc065150 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 rx=0x7fd30c009a00 tx=0x7fd30c00b040).stop 2024-02-18T00:34:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd316480700 1 -- 172.21.15.59:0/3810994920 reap_dead start 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 msgr2=0x7fd31010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd31010e580 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 rx=0x7fd300004920 tx=0x7fd300009cb0).stop 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 shutdown_connections 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3100fdbb0 0x7fd31010e580 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 --2- 172.21.15.59:0/3810994920 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd2fc062c90 0x7fd2fc065150 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 >> 172.21.15.59:0/3810994920 conn(0x7fd3100f3800 msgr2=0x7fd3100fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 shutdown_connections 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 -- 172.21.15.59:0/3810994920 wait complete. 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:00.239+0000 7fd317ee3700 1 librados: shutdown 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:34:00.333 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:00.336 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:00.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:00.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:00.465 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:00.618 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: Running main() from gmock_main.cc 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (4184 ms) 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (4626 ms) 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2931 ms) 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2012 ms) 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (13753 ms total) 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-02-18T00:34:00.619 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4645 ms) 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4452 ms) 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (9097 ms total) 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (4054 ms) 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1634 ms) 2024-02-18T00:34:00.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (3471 ms) 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1822 ms) 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (10982 ms total) 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4015 ms) 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4011 ms) 2024-02-18T00:34:00.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8027 ms total) 2024-02-18T00:34:00.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: 2024-02-18T00:34:00.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-18T00:34:00.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (69727 ms total) 2024-02-18T00:34:00.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-02-18T00:34:00.625 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83048 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83048 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82904 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82904 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83008 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83008 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82976 2024-02-18T00:34:00.626 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82976 2024-02-18T00:34:00.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.579814+0000 mon.b (mon.1) 195 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:00.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.581029+0000 mon.b (mon.1) 196 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.590515+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.596573+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:58.619541+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.589282+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.589403+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]': finished 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:59.589586+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e161: 8 total, 8 up, 7 in 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.598187+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.59:0/2504946421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi059-82877-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.599092+0000 mon.b (mon.1) 197 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.600238+0000 mon.b (mon.1) 198 : audit [INF] from='client.? 172.21.15.59:0/1844002495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.607915+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]: dispatch 2024-02-18T00:34:00.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.608670+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]: dispatch 2024-02-18T00:34:00.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.610855+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:00.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:33:59.611265+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:00.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:33:59.823687+0000 mgr.y (mgr.24454) 210 : cluster [DBG] pgmap v229: 398 pgs: 64 unknown, 334 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:00.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:00.016410+0000 mon.a (mon.0) 1235 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:00.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:00.097050+0000 mon.a (mon.0) 1236 : audit [DBG] from='client.? 172.21.15.59:0/3810994920' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.579814+0000 mon.b (mon.1) 195 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.581029+0000 mon.b (mon.1) 196 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.590515+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.596573+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:58.619541+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.589282+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.589403+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]': finished 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:59.589586+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e161: 8 total, 8 up, 7 in 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.598187+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.59:0/2504946421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi059-82877-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.599092+0000 mon.b (mon.1) 197 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.600238+0000 mon.b (mon.1) 198 : audit [INF] from='client.? 172.21.15.59:0/1844002495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.607915+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.579814+0000 mon.b (mon.1) 195 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.581029+0000 mon.b (mon.1) 196 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.590515+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.596573+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:58.619541+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.589282+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.589403+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-13"}]': finished 2024-02-18T00:34:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:59.589586+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e161: 8 total, 8 up, 7 in 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.598187+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? 172.21.15.59:0/2504946421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi059-82877-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.599092+0000 mon.b (mon.1) 197 : audit [INF] from='client.? 172.21.15.59:0/4226878741' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.600238+0000 mon.b (mon.1) 198 : audit [INF] from='client.? 172.21.15.59:0/1844002495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.607915+0000 mon.c (mon.2) 98 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]: dispatch 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.608670+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]: dispatch 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.610855+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:33:59.611265+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:33:59.823687+0000 mgr.y (mgr.24454) 210 : cluster [DBG] pgmap v229: 398 pgs: 64 unknown, 334 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:00.016410+0000 mon.a (mon.0) 1235 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:00.097050+0000 mon.a (mon.0) 1236 : audit [DBG] from='client.? 172.21.15.59:0/3810994920' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:01.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.608670+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]: dispatch 2024-02-18T00:34:01.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.610855+0000 mon.a (mon.0) 1233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]: dispatch 2024-02-18T00:34:01.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:33:59.611265+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:01.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:33:59.823687+0000 mgr.y (mgr.24454) 210 : cluster [DBG] pgmap v229: 398 pgs: 64 unknown, 334 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:01.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:00.016410+0000 mon.a (mon.0) 1235 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:01.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:00.097050+0000 mon.a (mon.0) 1236 : audit [DBG] from='client.? 172.21.15.59:0/3810994920' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:01.114 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:01.117 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:01.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:01.218 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:01.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:34:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:34:01.243 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:34:01.245 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:01.444 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:01.448 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:01.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:01.549 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:01.576 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:01.674 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:01.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:01.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3683855730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 msgr2=0x7f5a2c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3683855730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c103a20 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 rx=0x7f5a24009be0 tx=0x7f5a24004d10).stop 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3683855730 shutdown_connections 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3683855730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0fcbb0 0x7f5a2c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3683855730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c103a20 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3683855730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fd510 0x7f5a2c0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3683855730 >> 172.21.15.59:0/3683855730 conn(0x7f5a2c0f2820 msgr2=0x7f5a2c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3683855730 shutdown_connections 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.442+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3683855730 wait complete. 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 librados: starting msgr at 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 librados: starting objecter 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 Processor -- start 2024-02-18T00:34:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 -- start start 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 librados: setting wanted keys 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 librados: calling monclient init 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c10ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fcbb0 0x7f5a2c10d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0fd510 0x7f5a2c107050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c107540 con 0x7f5a2c0fcbb0 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c1076b0 con 0x7f5a2c0fd510 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a33dfa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c107870 con 0x7f5a2c0f7400 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a31395700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fcbb0 0x7f5a2c10d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a31b96700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c10ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a31b96700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c10ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39002/0 (socket says 172.21.15.59:39002) 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a31395700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fcbb0 0x7f5a2c10d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37326/0 (socket says 172.21.15.59:37326) 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.443+0000 7f5a31b96700 1 -- 172.21.15.59:0/3740544473 learned_addr learned my addr 172.21.15.59:0/3740544473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a32397700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0fd510 0x7f5a2c107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a31b96700 1 -- 172.21.15.59:0/3740544473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0fd510 msgr2=0x7f5a2c107050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a31b96700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0fd510 0x7f5a2c107050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a31b96700 1 -- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fcbb0 msgr2=0x7f5a2c10d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a31b96700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fcbb0 0x7f5a2c10d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a31b96700 1 -- 172.21.15.59:0/3740544473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c005040 con 0x7f5a2c0f7400 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a31395700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fcbb0 0x7f5a2c10d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a31b96700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c10ce60 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 rx=0x7f5a280041d0 tx=0x7f5a28009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a22ffd700 10 client.?.objecter ms_handle_connect 0x7f5a2c0f7400 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a22ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a22ffd700 1 -- 172.21.15.59:0/3740544473 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28007b40 con 0x7f5a2c0f7400 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a240097c0 con 0x7f5a2c0f7400 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a22ffd700 1 -- 172.21.15.59:0/3740544473 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a2800f040 con 0x7f5a2c0f7400 2024-02-18T00:34:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a22ffd700 1 -- 172.21.15.59:0/3740544473 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28014730 con 0x7f5a2c0f7400 2024-02-18T00:34:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a33dfa700 10 client.40262.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c113c90 con 0x7f5a2c0f7400 2024-02-18T00:34:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.444+0000 7f5a33dfa700 1 librados: init done 2024-02-18T00:34:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.445+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c05fa60 con 0x7f5a2c0f7400 2024-02-18T00:34:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a22ffd700 1 -- 172.21.15.59:0/3740544473 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5a28014b60 con 0x7f5a2c0f7400 2024-02-18T00:34:01.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a22ffd700 1 --2- 172.21.15.59:0/3740544473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a18062c90 0x7f5a18065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:01.775 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a22ffd700 1 -- 172.21.15.59:0/3740544473 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(162..162 src has 44..162) v4 ==== 9456+0+0 (secure 0 0 0) 0x7f5a2807db80 con 0x7f5a2c0f7400 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a22ffd700 10 client.40262.objecter ms_dispatch 0x7f5a2c001550 osd_map(162..162 src has 44..162) v4 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a22ffd700 3 client.40262.objecter handle_osd_map got epochs [162,162] > 0 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a22ffd700 3 client.40262.objecter handle_osd_map decoding full epoch 162 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a31395700 1 --2- 172.21.15.59:0/3740544473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a18062c90 0x7f5a18065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.446+0000 7f5a22ffd700 20 client.40262.objecter dump_active .. 0 homeless 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.447+0000 7f5a31395700 1 --2- 172.21.15.59:0/3740544473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a18062c90 0x7f5a18065150 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 rx=0x7f5a2c1087a0 tx=0x7f5a1c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.447+0000 7f5a22ffd700 10 client.40262.objecter ms_handle_connect 0x7f5a18062c90 2024-02-18T00:34:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.450+0000 7f5a22ffd700 1 -- 172.21.15.59:0/3740544473 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5a28050b60 con 0x7f5a2c0f7400 2024-02-18T00:34:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.651+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a2c05ee80 con 0x7f5a18062c90 2024-02-18T00:34:01.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.693+0000 7f5a22ffd700 1 -- 172.21.15.59:0/3740544473 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+821351 (secure 0 0 0) 0x7f5a2c05ee80 con 0x7f5a18062c90 2024-02-18T00:34:01.781 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:01.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 10 librados: watch_flush enter 2024-02-18T00:34:01.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 10 librados: watch_flush exit 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 20 client.40262.objecter shutdown clearing up homeless session... 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 10 client.40262.objecter successfully canceled tick 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a18062c90 msgr2=0x7f5a18065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3740544473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a18062c90 0x7f5a18065150 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 rx=0x7f5a2c1087a0 tx=0x7f5a1c00b040).stop 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 msgr2=0x7f5a2c10ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c10ce60 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 rx=0x7f5a280041d0 tx=0x7f5a28009f60).stop 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 shutdown_connections 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3740544473 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5a18062c90 0x7f5a18065150 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a2c0fd510 0x7f5a2c107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5a2c0f7400 0x7f5a2c10ce60 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 --2- 172.21.15.59:0/3740544473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a2c0fcbb0 0x7f5a2c10d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.773+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 >> 172.21.15.59:0/3740544473 conn(0x7f5a2c0f2820 msgr2=0x7f5a2c0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.774+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 shutdown_connections 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.774+0000 7f5a33dfa700 1 -- 172.21.15.59:0/3740544473 wait complete. 2024-02-18T00:34:01.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:01.774+0000 7f5a33dfa700 1 librados: shutdown 2024-02-18T00:34:01.790 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:00.585679+0000 mon.a (mon.0) 1237 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:00.586767+0000 mon.a (mon.0) 1238 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:00.595396+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? 172.21.15.59:0/2504946421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi059-82877-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:00.595501+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]': finished 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:00.595565+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:00.595656+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:00.595769+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e162: 8 total, 8 up, 7 in 2024-02-18T00:34:01.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:00.981922+0000 mgr.y (mgr.24454) 211 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.283238+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.283416+0000 mgr.y (mgr.24454) 212 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.283791+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.283895+0000 mgr.y (mgr.24454) 213 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.284286+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.284398+0000 mgr.y (mgr.24454) 214 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.284752+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:00.585679+0000 mon.a (mon.0) 1237 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:01.800 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:00.586767+0000 mon.a (mon.0) 1238 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:00.595396+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? 172.21.15.59:0/2504946421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi059-82877-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:00.595501+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]': finished 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:00.595565+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:00.595656+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:00.595769+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e162: 8 total, 8 up, 7 in 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:00.981922+0000 mgr.y (mgr.24454) 211 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.283238+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.283416+0000 mgr.y (mgr.24454) 212 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.283791+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.283895+0000 mgr.y (mgr.24454) 213 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.284286+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.284398+0000 mgr.y (mgr.24454) 214 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-02-18T00:34:01.801 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.284752+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.284855+0000 mgr.y (mgr.24454) 215 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.285190+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.285288+0000 mgr.y (mgr.24454) 216 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.285606+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.285707+0000 mgr.y (mgr.24454) 217 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.286034+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.286144+0000 mgr.y (mgr.24454) 218 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.286478+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-02-18T00:34:01.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.286586+0000 mgr.y (mgr.24454) 219 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.286929+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.287033+0000 mgr.y (mgr.24454) 220 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.287369+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.287473+0000 mgr.y (mgr.24454) 221 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.284855+0000 mgr.y (mgr.24454) 215 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.285190+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.285288+0000 mgr.y (mgr.24454) 216 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.285606+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.285707+0000 mgr.y (mgr.24454) 217 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.286034+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.286144+0000 mgr.y (mgr.24454) 218 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-02-18T00:34:01.803 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.286478+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-02-18T00:34:01.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.286586+0000 mgr.y (mgr.24454) 219 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-02-18T00:34:01.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.286929+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-02-18T00:34:01.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.287033+0000 mgr.y (mgr.24454) 220 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-02-18T00:34:01.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.287369+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-02-18T00:34:01.804 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.287473+0000 mgr.y (mgr.24454) 221 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:00.585679+0000 mon.a (mon.0) 1237 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:00.586767+0000 mon.a (mon.0) 1238 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:00.595396+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? 172.21.15.59:0/2504946421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi059-82877-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:00.595501+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-13", "mode": "writeback"}]': finished 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:00.595565+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi059-82973-14"}]': finished 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:00.595656+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi059-82882-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:00.595769+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e162: 8 total, 8 up, 7 in 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:00.981922+0000 mgr.y (mgr.24454) 211 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.283238+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.283416+0000 mgr.y (mgr.24454) 212 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.283791+0000 mon.c (mon.2) 100 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-02-18T00:34:01.819 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.283895+0000 mgr.y (mgr.24454) 213 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.284286+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.284398+0000 mgr.y (mgr.24454) 214 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.284752+0000 mon.c (mon.2) 102 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.284855+0000 mgr.y (mgr.24454) 215 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.285190+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.285288+0000 mgr.y (mgr.24454) 216 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.285606+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.285707+0000 mgr.y (mgr.24454) 217 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.286034+0000 mon.c (mon.2) 105 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.286144+0000 mgr.y (mgr.24454) 218 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.286478+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.286586+0000 mgr.y (mgr.24454) 219 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-02-18T00:34:01.820 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.286929+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-02-18T00:34:01.821 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.287033+0000 mgr.y (mgr.24454) 220 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-02-18T00:34:01.821 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.287369+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-02-18T00:34:01.821 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.287473+0000 mgr.y (mgr.24454) 221 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-02-18T00:34:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:01.898 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:01.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:01.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:34:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:34:02.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:02.032 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:02.066 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:02.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:02.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:02.170 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:02.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:02] "GET /metrics HTTP/1.1" 200 200229 "" "Prometheus/2.33.4" 2024-02-18T00:34:02.521 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-18T00:34:02.550 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-02-18T00:34:01.595432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":226515240,"num_objects":822,"num_object_clones":410,"num_object_copies":2466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":784,"num_whiteouts":0,"num_read":2519,"num_read_kb":50347,"num_write":2311,"num_write_kb":228375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":50338257,"num_keys_recovered":2000,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":39,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2891,"ondisk_log_size":2891,"up":1002,"acting":1002,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1142,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2778036,"kb_used_data":685304,"kb_used_omap":0,"kb_used_meta":2092672,"kb_avail":653438028,"statfs":{"total":671965249536,"available":669120540672,"internally_reserved":0,"allocated":701751296,"data_stored":688172957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2142896128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5508,"num_objects":242,"num_object_clones":214,"num_object_copies":726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":204,"num_whiteouts":0,"num_read":115,"num_read_kb":69,"num_write":201,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":39,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010663"},"pg_stats":[{"pgid":"97.1e","version":"162'1","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.677738+0000","last_change":"2024-02-18T00:33:53.608605+0000","last_active":"2024-02-18T00:34:00.677738+0000","last_peered":"2024-02-18T00:34:00.677738+0000","last_clean":"2024-02-18T00:34:00.677738+0000","last_became_active":"2024-02-18T00:33:53.608322+0000","last_became_peered":"2024-02-18T00:33:53.608322+0000","last_unstale":"2024-02-18T00:34:00.677738+0000","last_undegraded":"2024-02-18T00:34:00.677738+0000","last_fullsized":"2024-02-18T00:34:00.677738+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"97.1f","version":"162'1","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.683231+0000","last_change":"2024-02-18T00:33:53.610444+0000","last_active":"2024-02-18T00:34:00.683231+0000","last_peered":"2024-02-18T00:34:00.683231+0000","last_clean":"2024-02-18T00:34:00.683231+0000","last_became_active":"2024-02-18T00:33:53.610070+0000","last_became_peered":"2024-02-18T00:33:53.610070+0000","last_unstale":"2024-02-18T00:34:00.683231+0000","last_undegraded":"2024-02-18T00:34:00.683231+0000","last_fullsized":"2024-02-18T00:34:00.683231+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.1c","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595427+0000","last_change":"2024-02-18T00:33:53.600727+0000","last_active":"2024-02-18T00:33:59.595427+0000","last_peered":"2024-02-18T00:33:59.595427+0000","last_clean":"2024-02-18T00:33:59.595427+0000","last_became_active":"2024-02-18T00:33:53.600450+0000","last_became_peered":"2024-02-18T00:33:53.600450+0000","last_unstale":"2024-02-18T00:33:59.595427+0000","last_undegraded":"2024-02-18T00:33:59.595427+0000","last_fullsized":"2024-02-18T00:33:59.595427+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"97.1d","version":"162'1","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.675101+0000","last_change":"2024-02-18T00:33:53.615410+0000","last_active":"2024-02-18T00:34:00.675101+0000","last_peered":"2024-02-18T00:34:00.675101+0000","last_clean":"2024-02-18T00:34:00.675101+0000","last_became_active":"2024-02-18T00:33:53.615102+0000","last_became_peered":"2024-02-18T00:33:53.615102+0000","last_unstale":"2024-02-18T00:34:00.675101+0000","last_undegraded":"2024-02-18T00:34:00.675101+0000","last_fullsized":"2024-02-18T00:34:00.675101+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"97.1a","version":"162'3","reported_seq":21,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.725135+0000","last_change":"2024-02-18T00:33:53.617125+0000","last_active":"2024-02-18T00:34:00.725135+0000","last_peered":"2024-02-18T00:34:00.725135+0000","last_clean":"2024-02-18T00:34:00.725135+0000","last_became_active":"2024-02-18T00:33:53.616692+0000","last_became_peered":"2024-02-18T00:33:53.616692+0000","last_unstale":"2024-02-18T00:34:00.725135+0000","last_undegraded":"2024-02-18T00:34:00.725135+0000","last_fullsized":"2024-02-18T00:34:00.725135+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.1b","version":"162'2","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.641376+0000","last_change":"2024-02-18T00:33:53.609703+0000","last_active":"2024-02-18T00:34:00.641376+0000","last_peered":"2024-02-18T00:34:00.641376+0000","last_clean":"2024-02-18T00:34:00.641376+0000","last_became_active":"2024-02-18T00:33:53.609275+0000","last_became_peered":"2024-02-18T00:33:53.609275+0000","last_unstale":"2024-02-18T00:34:00.641376+0000","last_undegraded":"2024-02-18T00:34:00.641376+0000","last_fullsized":"2024-02-18T00:34:00.641376+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"97.18","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594123+0000","last_change":"2024-02-18T00:33:53.592273+0000","last_active":"2024-02-18T00:33:59.594123+0000","last_peered":"2024-02-18T00:33:59.594123+0000","last_clean":"2024-02-18T00:33:59.594123+0000","last_became_active":"2024-02-18T00:33:53.591909+0000","last_became_peered":"2024-02-18T00:33:53.591909+0000","last_unstale":"2024-02-18T00:33:59.594123+0000","last_undegraded":"2024-02-18T00:33:59.594123+0000","last_fullsized":"2024-02-18T00:33:59.594123+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"97.19","version":"162'5","reported_seq":25,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.732340+0000","last_change":"2024-02-18T00:33:53.586949+0000","last_active":"2024-02-18T00:34:00.732340+0000","last_peered":"2024-02-18T00:34:00.732340+0000","last_clean":"2024-02-18T00:34:00.732340+0000","last_became_active":"2024-02-18T00:33:53.586621+0000","last_became_peered":"2024-02-18T00:33:53.586621+0000","last_unstale":"2024-02-18T00:34:00.732340+0000","last_undegraded":"2024-02-18T00:34:00.732340+0000","last_fullsized":"2024-02-18T00:34:00.732340+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":5,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"97.16","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597827+0000","last_change":"2024-02-18T00:33:53.594602+0000","last_active":"2024-02-18T00:33:59.597827+0000","last_peered":"2024-02-18T00:33:59.597827+0000","last_clean":"2024-02-18T00:33:59.597827+0000","last_became_active":"2024-02-18T00:33:53.594310+0000","last_became_peered":"2024-02-18T00:33:53.594310+0000","last_unstale":"2024-02-18T00:33:59.597827+0000","last_undegraded":"2024-02-18T00:33:59.597827+0000","last_fullsized":"2024-02-18T00:33:59.597827+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"97.17","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590767+0000","last_change":"2024-02-18T00:33:53.588091+0000","last_active":"2024-02-18T00:33:59.590767+0000","last_peered":"2024-02-18T00:33:59.590767+0000","last_clean":"2024-02-18T00:33:59.590767+0000","last_became_active":"2024-02-18T00:33:53.587815+0000","last_became_peered":"2024-02-18T00:33:53.587815+0000","last_unstale":"2024-02-18T00:33:59.590767+0000","last_undegraded":"2024-02-18T00:33:59.590767+0000","last_fullsized":"2024-02-18T00:33:59.590767+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"97.14","version":"162'3","reported_seq":21,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.742791+0000","last_change":"2024-02-18T00:33:53.589478+0000","last_active":"2024-02-18T00:34:00.742791+0000","last_peered":"2024-02-18T00:34:00.742791+0000","last_clean":"2024-02-18T00:34:00.742791+0000","last_became_active":"2024-02-18T00:33:53.589200+0000","last_became_peered":"2024-02-18T00:33:53.589200+0000","last_unstale":"2024-02-18T00:34:00.742791+0000","last_undegraded":"2024-02-18T00:34:00.742791+0000","last_fullsized":"2024-02-18T00:34:00.742791+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"97.15","version":"162'2","reported_seq":20,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.671358+0000","last_change":"2024-02-18T00:33:53.613209+0000","last_active":"2024-02-18T00:34:00.671358+0000","last_peered":"2024-02-18T00:34:00.671358+0000","last_clean":"2024-02-18T00:34:00.671358+0000","last_became_active":"2024-02-18T00:33:53.612935+0000","last_became_peered":"2024-02-18T00:33:53.612935+0000","last_unstale":"2024-02-18T00:34:00.671358+0000","last_undegraded":"2024-02-18T00:34:00.671358+0000","last_fullsized":"2024-02-18T00:34:00.671358+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"97.12","version":"162'2","reported_seq":20,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.695058+0000","last_change":"2024-02-18T00:33:53.613358+0000","last_active":"2024-02-18T00:34:00.695058+0000","last_peered":"2024-02-18T00:34:00.695058+0000","last_clean":"2024-02-18T00:34:00.695058+0000","last_became_active":"2024-02-18T00:33:53.613066+0000","last_became_peered":"2024-02-18T00:33:53.613066+0000","last_unstale":"2024-02-18T00:34:00.695058+0000","last_undegraded":"2024-02-18T00:34:00.695058+0000","last_fullsized":"2024-02-18T00:34:00.695058+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.13","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591281+0000","last_change":"2024-02-18T00:33:53.593120+0000","last_active":"2024-02-18T00:33:59.591281+0000","last_peered":"2024-02-18T00:33:59.591281+0000","last_clean":"2024-02-18T00:33:59.591281+0000","last_became_active":"2024-02-18T00:33:53.592880+0000","last_became_peered":"2024-02-18T00:33:53.592880+0000","last_unstale":"2024-02-18T00:33:59.591281+0000","last_undegraded":"2024-02-18T00:33:59.591281+0000","last_fullsized":"2024-02-18T00:33:59.591281+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"97.10","version":"162'2","reported_seq":20,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.649564+0000","last_change":"2024-02-18T00:33:53.608691+0000","last_active":"2024-02-18T00:34:00.649564+0000","last_peered":"2024-02-18T00:34:00.649564+0000","last_clean":"2024-02-18T00:34:00.649564+0000","last_became_active":"2024-02-18T00:33:53.608425+0000","last_became_peered":"2024-02-18T00:33:53.608425+0000","last_unstale":"2024-02-18T00:34:00.649564+0000","last_undegraded":"2024-02-18T00:34:00.649564+0000","last_fullsized":"2024-02-18T00:34:00.649564+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"97.11","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.603535+0000","last_change":"2024-02-18T00:33:53.607811+0000","last_active":"2024-02-18T00:33:59.603535+0000","last_peered":"2024-02-18T00:33:59.603535+0000","last_clean":"2024-02-18T00:33:59.603535+0000","last_became_active":"2024-02-18T00:33:53.607531+0000","last_became_peered":"2024-02-18T00:33:53.607531+0000","last_unstale":"2024-02-18T00:33:59.603535+0000","last_undegraded":"2024-02-18T00:33:59.603535+0000","last_fullsized":"2024-02-18T00:33:59.603535+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.e","version":"162'2","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.728658+0000","last_change":"2024-02-18T00:33:53.608549+0000","last_active":"2024-02-18T00:34:00.728658+0000","last_peered":"2024-02-18T00:34:00.728658+0000","last_clean":"2024-02-18T00:34:00.728658+0000","last_became_active":"2024-02-18T00:33:53.607595+0000","last_became_peered":"2024-02-18T00:33:53.607595+0000","last_unstale":"2024-02-18T00:34:00.728658+0000","last_undegraded":"2024-02-18T00:34:00.728658+0000","last_fullsized":"2024-02-18T00:34:00.728658+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.f","version":"162'1","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.698269+0000","last_change":"2024-02-18T00:33:53.600100+0000","last_active":"2024-02-18T00:34:00.698269+0000","last_peered":"2024-02-18T00:34:00.698269+0000","last_clean":"2024-02-18T00:34:00.698269+0000","last_became_active":"2024-02-18T00:33:53.599833+0000","last_became_peered":"2024-02-18T00:33:53.599833+0000","last_unstale":"2024-02-18T00:34:00.698269+0000","last_undegraded":"2024-02-18T00:34:00.698269+0000","last_fullsized":"2024-02-18T00:34:00.698269+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"97.c","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.614642+0000","last_change":"2024-02-18T00:33:53.616191+0000","last_active":"2024-02-18T00:33:59.614642+0000","last_peered":"2024-02-18T00:33:59.614642+0000","last_clean":"2024-02-18T00:33:59.614642+0000","last_became_active":"2024-02-18T00:33:53.615899+0000","last_became_peered":"2024-02-18T00:33:53.615899+0000","last_unstale":"2024-02-18T00:33:59.614642+0000","last_undegraded":"2024-02-18T00:33:59.614642+0000","last_fullsized":"2024-02-18T00:33:59.614642+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.d","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.611053+0000","last_change":"2024-02-18T00:33:53.612120+0000","last_active":"2024-02-18T00:33:59.611053+0000","last_peered":"2024-02-18T00:33:59.611053+0000","last_clean":"2024-02-18T00:33:59.611053+0000","last_became_active":"2024-02-18T00:33:53.607798+0000","last_became_peered":"2024-02-18T00:33:53.607798+0000","last_unstale":"2024-02-18T00:33:59.611053+0000","last_undegraded":"2024-02-18T00:33:59.611053+0000","last_fullsized":"2024-02-18T00:33:59.611053+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.a","version":"162'1","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.692350+0000","last_change":"2024-02-18T00:33:53.611520+0000","last_active":"2024-02-18T00:34:00.692350+0000","last_peered":"2024-02-18T00:34:00.692350+0000","last_clean":"2024-02-18T00:34:00.692350+0000","last_became_active":"2024-02-18T00:33:53.611212+0000","last_became_peered":"2024-02-18T00:33:53.611212+0000","last_unstale":"2024-02-18T00:34:00.692350+0000","last_undegraded":"2024-02-18T00:34:00.692350+0000","last_fullsized":"2024-02-18T00:34:00.692350+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"97.b","version":"162'2","reported_seq":20,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.737176+0000","last_change":"2024-02-18T00:33:53.610734+0000","last_active":"2024-02-18T00:34:00.737176+0000","last_peered":"2024-02-18T00:34:00.737176+0000","last_clean":"2024-02-18T00:34:00.737176+0000","last_became_active":"2024-02-18T00:33:53.610535+0000","last_became_peered":"2024-02-18T00:33:53.610535+0000","last_unstale":"2024-02-18T00:34:00.737176+0000","last_undegraded":"2024-02-18T00:34:00.737176+0000","last_fullsized":"2024-02-18T00:34:00.737176+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.8","version":"162'3","reported_seq":71,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.717760+0000","last_change":"2024-02-18T00:33:53.595993+0000","last_active":"2024-02-18T00:34:00.717760+0000","last_peered":"2024-02-18T00:34:00.717760+0000","last_clean":"2024-02-18T00:34:00.717760+0000","last_became_active":"2024-02-18T00:33:53.595689+0000","last_became_peered":"2024-02-18T00:33:53.595689+0000","last_unstale":"2024-02-18T00:34:00.717760+0000","last_undegraded":"2024-02-18T00:34:00.717760+0000","last_fullsized":"2024-02-18T00:34:00.717760+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":2,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5],"acting":[1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"97.9","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589464+0000","last_change":"2024-02-18T00:33:53.587912+0000","last_active":"2024-02-18T00:33:59.589464+0000","last_peered":"2024-02-18T00:33:59.589464+0000","last_clean":"2024-02-18T00:33:59.589464+0000","last_became_active":"2024-02-18T00:33:53.587559+0000","last_became_peered":"2024-02-18T00:33:53.587559+0000","last_unstale":"2024-02-18T00:33:59.589464+0000","last_undegraded":"2024-02-18T00:33:59.589464+0000","last_fullsized":"2024-02-18T00:33:59.589464+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"97.6","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597923+0000","last_change":"2024-02-18T00:33:53.598227+0000","last_active":"2024-02-18T00:33:59.597923+0000","last_peered":"2024-02-18T00:33:59.597923+0000","last_clean":"2024-02-18T00:33:59.597923+0000","last_became_active":"2024-02-18T00:33:53.597919+0000","last_became_peered":"2024-02-18T00:33:53.597919+0000","last_unstale":"2024-02-18T00:33:59.597923+0000","last_undegraded":"2024-02-18T00:33:59.597923+0000","last_fullsized":"2024-02-18T00:33:59.597923+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"97.7","version":"162'3","reported_seq":21,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.749281+0000","last_change":"2024-02-18T00:33:53.595977+0000","last_active":"2024-02-18T00:34:00.749281+0000","last_peered":"2024-02-18T00:34:00.749281+0000","last_clean":"2024-02-18T00:34:00.749281+0000","last_became_active":"2024-02-18T00:33:53.595685+0000","last_became_peered":"2024-02-18T00:33:53.595685+0000","last_unstale":"2024-02-18T00:34:00.749281+0000","last_undegraded":"2024-02-18T00:34:00.749281+0000","last_fullsized":"2024-02-18T00:34:00.749281+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"97.4","version":"162'1","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.654755+0000","last_change":"2024-02-18T00:33:53.597288+0000","last_active":"2024-02-18T00:34:00.654755+0000","last_peered":"2024-02-18T00:34:00.654755+0000","last_clean":"2024-02-18T00:34:00.654755+0000","last_became_active":"2024-02-18T00:33:53.597027+0000","last_became_peered":"2024-02-18T00:33:53.597027+0000","last_unstale":"2024-02-18T00:34:00.654755+0000","last_undegraded":"2024-02-18T00:34:00.654755+0000","last_fullsized":"2024-02-18T00:34:00.654755+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"97.5","version":"162'1","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.686644+0000","last_change":"2024-02-18T00:33:53.615522+0000","last_active":"2024-02-18T00:34:00.686644+0000","last_peered":"2024-02-18T00:34:00.686644+0000","last_clean":"2024-02-18T00:34:00.686644+0000","last_became_active":"2024-02-18T00:33:53.615198+0000","last_became_peered":"2024-02-18T00:33:53.615198+0000","last_unstale":"2024-02-18T00:34:00.686644+0000","last_undegraded":"2024-02-18T00:34:00.686644+0000","last_fullsized":"2024-02-18T00:34:00.686644+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"97.2","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.617603+0000","last_change":"2024-02-18T00:33:53.616940+0000","last_active":"2024-02-18T00:33:59.617603+0000","last_peered":"2024-02-18T00:33:59.617603+0000","last_clean":"2024-02-18T00:33:59.617603+0000","last_became_active":"2024-02-18T00:33:53.616292+0000","last_became_peered":"2024-02-18T00:33:53.616292+0000","last_unstale":"2024-02-18T00:33:59.617603+0000","last_undegraded":"2024-02-18T00:33:59.617603+0000","last_fullsized":"2024-02-18T00:33:59.617603+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"97.3","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589848+0000","last_change":"2024-02-18T00:33:53.603901+0000","last_active":"2024-02-18T00:33:59.589848+0000","last_peered":"2024-02-18T00:33:59.589848+0000","last_clean":"2024-02-18T00:33:59.589848+0000","last_became_active":"2024-02-18T00:33:53.603569+0000","last_became_peered":"2024-02-18T00:33:53.603569+0000","last_unstale":"2024-02-18T00:33:59.589848+0000","last_undegraded":"2024-02-18T00:33:59.589848+0000","last_fullsized":"2024-02-18T00:33:59.589848+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"97.0","version":"162'2","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.722342+0000","last_change":"2024-02-18T00:33:53.609431+0000","last_active":"2024-02-18T00:34:00.722342+0000","last_peered":"2024-02-18T00:34:00.722342+0000","last_clean":"2024-02-18T00:34:00.722342+0000","last_became_active":"2024-02-18T00:33:53.609179+0000","last_became_peered":"2024-02-18T00:33:53.609179+0000","last_unstale":"2024-02-18T00:34:00.722342+0000","last_undegraded":"2024-02-18T00:34:00.722342+0000","last_fullsized":"2024-02-18T00:34:00.722342+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"97.1","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590002+0000","last_change":"2024-02-18T00:33:53.614562+0000","last_active":"2024-02-18T00:33:59.590002+0000","last_peered":"2024-02-18T00:33:59.590002+0000","last_clean":"2024-02-18T00:33:59.590002+0000","last_became_active":"2024-02-18T00:33:53.614308+0000","last_became_peered":"2024-02-18T00:33:53.614308+0000","last_unstale":"2024-02-18T00:33:59.590002+0000","last_undegraded":"2024-02-18T00:33:59.590002+0000","last_fullsized":"2024-02-18T00:33:59.590002+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:52.583076+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.583076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.1d","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596411+0000","last_change":"2024-02-18T00:33:50.618546+0000","last_active":"2024-02-18T00:33:59.596411+0000","last_peered":"2024-02-18T00:33:59.596411+0000","last_clean":"2024-02-18T00:33:59.596411+0000","last_became_active":"2024-02-18T00:33:23.620302+0000","last_became_peered":"2024-02-18T00:33:23.620302+0000","last_unstale":"2024-02-18T00:33:59.596411+0000","last_undegraded":"2024-02-18T00:33:59.596411+0000","last_fullsized":"2024-02-18T00:33:59.596411+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597370+0000","last_change":"2024-02-18T00:33:50.604349+0000","last_active":"2024-02-18T00:33:59.597370+0000","last_peered":"2024-02-18T00:33:59.597370+0000","last_clean":"2024-02-18T00:33:59.597370+0000","last_became_active":"2024-02-18T00:33:23.636764+0000","last_became_peered":"2024-02-18T00:33:23.636764+0000","last_unstale":"2024-02-18T00:33:59.597370+0000","last_undegraded":"2024-02-18T00:33:59.597370+0000","last_fullsized":"2024-02-18T00:33:59.597370+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590951+0000","last_change":"2024-02-18T00:33:50.598265+0000","last_active":"2024-02-18T00:33:59.590951+0000","last_peered":"2024-02-18T00:33:59.590951+0000","last_clean":"2024-02-18T00:33:59.590951+0000","last_became_active":"2024-02-18T00:33:23.503952+0000","last_became_peered":"2024-02-18T00:33:23.503952+0000","last_unstale":"2024-02-18T00:33:59.590951+0000","last_undegraded":"2024-02-18T00:33:59.590951+0000","last_fullsized":"2024-02-18T00:33:59.590951+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1e","version":"150'1","reported_seq":91,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594207+0000","last_change":"2024-02-18T00:33:51.608864+0000","last_active":"2024-02-18T00:33:59.594207+0000","last_peered":"2024-02-18T00:33:59.594207+0000","last_clean":"2024-02-18T00:33:59.594207+0000","last_became_active":"2024-02-18T00:33:23.615724+0000","last_became_peered":"2024-02-18T00:33:23.615724+0000","last_unstale":"2024-02-18T00:33:59.594207+0000","last_undegraded":"2024-02-18T00:33:59.594207+0000","last_fullsized":"2024-02-18T00:33:59.594207+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589887+0000","last_change":"2024-02-18T00:33:50.597355+0000","last_active":"2024-02-18T00:33:59.589887+0000","last_peered":"2024-02-18T00:33:59.589887+0000","last_clean":"2024-02-18T00:33:59.589887+0000","last_became_active":"2024-02-18T00:33:23.599663+0000","last_became_peered":"2024-02-18T00:33:23.599663+0000","last_unstale":"2024-02-18T00:33:59.589887+0000","last_undegraded":"2024-02-18T00:33:59.589887+0000","last_fullsized":"2024-02-18T00:33:59.589887+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590731+0000","last_change":"2024-02-18T00:33:50.598406+0000","last_active":"2024-02-18T00:33:59.590731+0000","last_peered":"2024-02-18T00:33:59.590731+0000","last_clean":"2024-02-18T00:33:59.590731+0000","last_became_active":"2024-02-18T00:33:23.475463+0000","last_became_peered":"2024-02-18T00:33:23.475463+0000","last_unstale":"2024-02-18T00:33:59.590731+0000","last_undegraded":"2024-02-18T00:33:59.590731+0000","last_fullsized":"2024-02-18T00:33:59.590731+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1b","version":"0'0","reported_seq":98,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.609924+0000","last_change":"2024-02-18T00:33:50.610876+0000","last_active":"2024-02-18T00:33:59.609924+0000","last_peered":"2024-02-18T00:33:59.609924+0000","last_clean":"2024-02-18T00:33:59.609924+0000","last_became_active":"2024-02-18T00:33:48.748495+0000","last_became_peered":"2024-02-18T00:33:48.748495+0000","last_unstale":"2024-02-18T00:33:59.609924+0000","last_undegraded":"2024-02-18T00:33:59.609924+0000","last_fullsized":"2024-02-18T00:33:59.609924+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1a","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589666+0000","last_change":"2024-02-18T00:33:50.598856+0000","last_active":"2024-02-18T00:33:59.589666+0000","last_peered":"2024-02-18T00:33:59.589666+0000","last_clean":"2024-02-18T00:33:59.589666+0000","last_became_active":"2024-02-18T00:33:23.499256+0000","last_became_peered":"2024-02-18T00:33:23.499256+0000","last_unstale":"2024-02-18T00:33:59.589666+0000","last_undegraded":"2024-02-18T00:33:59.589666+0000","last_fullsized":"2024-02-18T00:33:59.589666+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":30,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601331+0000","last_change":"2024-02-18T00:33:50.609717+0000","last_active":"2024-02-18T00:33:59.601331+0000","last_peered":"2024-02-18T00:33:59.601331+0000","last_clean":"2024-02-18T00:33:59.601331+0000","last_became_active":"2024-02-18T00:33:48.714483+0000","last_became_peered":"2024-02-18T00:33:48.714483+0000","last_unstale":"2024-02-18T00:33:59.601331+0000","last_undegraded":"2024-02-18T00:33:59.601331+0000","last_fullsized":"2024-02-18T00:33:59.601331+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.14","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600883+0000","last_change":"2024-02-18T00:33:50.606337+0000","last_active":"2024-02-18T00:33:59.600883+0000","last_peered":"2024-02-18T00:33:59.600883+0000","last_clean":"2024-02-18T00:33:59.600883+0000","last_became_active":"2024-02-18T00:33:23.566091+0000","last_became_peered":"2024-02-18T00:33:23.566091+0000","last_unstale":"2024-02-18T00:33:59.600883+0000","last_undegraded":"2024-02-18T00:33:59.600883+0000","last_fullsized":"2024-02-18T00:33:59.600883+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.17","version":"149'13","reported_seq":107,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.607617+0000","last_change":"2024-02-18T00:33:50.611143+0000","last_active":"2024-02-18T00:33:59.607617+0000","last_peered":"2024-02-18T00:33:59.607617+0000","last_clean":"2024-02-18T00:33:59.607617+0000","last_became_active":"2024-02-18T00:33:23.488902+0000","last_became_peered":"2024-02-18T00:33:23.488902+0000","last_unstale":"2024-02-18T00:33:59.607617+0000","last_undegraded":"2024-02-18T00:33:59.607617+0000","last_fullsized":"2024-02-18T00:33:59.607617+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":640,"num_objects":3,"num_object_clones":1,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.16","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594295+0000","last_change":"2024-02-18T00:33:51.609110+0000","last_active":"2024-02-18T00:33:59.594295+0000","last_peered":"2024-02-18T00:33:59.594295+0000","last_clean":"2024-02-18T00:33:59.594295+0000","last_became_active":"2024-02-18T00:33:23.636643+0000","last_became_peered":"2024-02-18T00:33:23.636643+0000","last_unstale":"2024-02-18T00:33:59.594295+0000","last_undegraded":"2024-02-18T00:33:59.594295+0000","last_fullsized":"2024-02-18T00:33:59.594295+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592946+0000","last_change":"2024-02-18T00:33:50.600471+0000","last_active":"2024-02-18T00:33:59.592946+0000","last_peered":"2024-02-18T00:33:59.592946+0000","last_clean":"2024-02-18T00:33:59.592946+0000","last_became_active":"2024-02-18T00:33:23.626811+0000","last_became_peered":"2024-02-18T00:33:23.626811+0000","last_unstale":"2024-02-18T00:33:59.592946+0000","last_undegraded":"2024-02-18T00:33:59.592946+0000","last_fullsized":"2024-02-18T00:33:59.592946+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589112+0000","last_change":"2024-02-18T00:33:50.597487+0000","last_active":"2024-02-18T00:33:59.589112+0000","last_peered":"2024-02-18T00:33:59.589112+0000","last_clean":"2024-02-18T00:33:59.589112+0000","last_became_active":"2024-02-18T00:33:23.625484+0000","last_became_peered":"2024-02-18T00:33:23.625484+0000","last_unstale":"2024-02-18T00:33:59.589112+0000","last_undegraded":"2024-02-18T00:33:59.589112+0000","last_fullsized":"2024-02-18T00:33:59.589112+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.13","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601076+0000","last_change":"2024-02-18T00:33:50.602133+0000","last_active":"2024-02-18T00:33:59.601076+0000","last_peered":"2024-02-18T00:33:59.601076+0000","last_clean":"2024-02-18T00:33:59.601076+0000","last_became_active":"2024-02-18T00:33:23.489005+0000","last_became_peered":"2024-02-18T00:33:23.489005+0000","last_unstale":"2024-02-18T00:33:59.601076+0000","last_undegraded":"2024-02-18T00:33:59.601076+0000","last_fullsized":"2024-02-18T00:33:59.601076+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.12","version":"142'26","reported_seq":117,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592056+0000","last_change":"2024-02-18T00:33:50.608217+0000","last_active":"2024-02-18T00:33:59.592056+0000","last_peered":"2024-02-18T00:33:59.592056+0000","last_clean":"2024-02-18T00:33:59.592056+0000","last_became_active":"2024-02-18T00:33:23.490075+0000","last_became_peered":"2024-02-18T00:33:23.490075+0000","last_unstale":"2024-02-18T00:33:59.592056+0000","last_undegraded":"2024-02-18T00:33:59.592056+0000","last_fullsized":"2024-02-18T00:33:59.592056+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1280,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.d","version":"0'0","reported_seq":98,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593358+0000","last_change":"2024-02-18T00:33:51.608325+0000","last_active":"2024-02-18T00:33:59.593358+0000","last_peered":"2024-02-18T00:33:59.593358+0000","last_clean":"2024-02-18T00:33:59.593358+0000","last_became_active":"2024-02-18T00:33:48.744435+0000","last_became_peered":"2024-02-18T00:33:48.744435+0000","last_unstale":"2024-02-18T00:33:59.593358+0000","last_undegraded":"2024-02-18T00:33:59.593358+0000","last_fullsized":"2024-02-18T00:33:59.593358+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.c","version":"0'0","reported_seq":30,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597503+0000","last_change":"2024-02-18T00:33:50.604507+0000","last_active":"2024-02-18T00:33:59.597503+0000","last_peered":"2024-02-18T00:33:59.597503+0000","last_clean":"2024-02-18T00:33:59.597503+0000","last_became_active":"2024-02-18T00:33:48.666029+0000","last_became_peered":"2024-02-18T00:33:48.666029+0000","last_unstale":"2024-02-18T00:33:59.597503+0000","last_undegraded":"2024-02-18T00:33:59.597503+0000","last_fullsized":"2024-02-18T00:33:59.597503+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600266+0000","last_change":"2024-02-18T00:33:50.617076+0000","last_active":"2024-02-18T00:33:59.600266+0000","last_peered":"2024-02-18T00:33:59.600266+0000","last_clean":"2024-02-18T00:33:59.600266+0000","last_became_active":"2024-02-18T00:33:23.621779+0000","last_became_peered":"2024-02-18T00:33:23.621779+0000","last_unstale":"2024-02-18T00:33:59.600266+0000","last_undegraded":"2024-02-18T00:33:59.600266+0000","last_fullsized":"2024-02-18T00:33:59.600266+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.611347+0000","last_change":"2024-02-18T00:33:50.607945+0000","last_active":"2024-02-18T00:33:59.611347+0000","last_peered":"2024-02-18T00:33:59.611347+0000","last_clean":"2024-02-18T00:33:59.611347+0000","last_became_active":"2024-02-18T00:33:23.599543+0000","last_became_peered":"2024-02-18T00:33:23.599543+0000","last_unstale":"2024-02-18T00:33:59.611347+0000","last_undegraded":"2024-02-18T00:33:59.611347+0000","last_fullsized":"2024-02-18T00:33:59.611347+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.9","version":"0'0","reported_seq":98,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591448+0000","last_change":"2024-02-18T00:33:50.601718+0000","last_active":"2024-02-18T00:33:59.591448+0000","last_peered":"2024-02-18T00:33:59.591448+0000","last_clean":"2024-02-18T00:33:59.591448+0000","last_became_active":"2024-02-18T00:33:48.687829+0000","last_became_peered":"2024-02-18T00:33:48.687829+0000","last_unstale":"2024-02-18T00:33:59.591448+0000","last_undegraded":"2024-02-18T00:33:59.591448+0000","last_fullsized":"2024-02-18T00:33:59.591448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590806+0000","last_change":"2024-02-18T00:33:50.608800+0000","last_active":"2024-02-18T00:33:59.590806+0000","last_peered":"2024-02-18T00:33:59.590806+0000","last_clean":"2024-02-18T00:33:59.590806+0000","last_became_active":"2024-02-18T00:33:23.477241+0000","last_became_peered":"2024-02-18T00:33:23.477241+0000","last_unstale":"2024-02-18T00:33:59.590806+0000","last_undegraded":"2024-02-18T00:33:59.590806+0000","last_fullsized":"2024-02-18T00:33:59.590806+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4],"acting":[0,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.b","version":"0'0","reported_seq":98,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590056+0000","last_change":"2024-02-18T00:33:50.597784+0000","last_active":"2024-02-18T00:33:59.590056+0000","last_peered":"2024-02-18T00:33:59.590056+0000","last_clean":"2024-02-18T00:33:59.590056+0000","last_became_active":"2024-02-18T00:33:48.635844+0000","last_became_peered":"2024-02-18T00:33:48.635844+0000","last_unstale":"2024-02-18T00:33:59.590056+0000","last_undegraded":"2024-02-18T00:33:59.590056+0000","last_fullsized":"2024-02-18T00:33:59.590056+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.617743+0000","last_change":"2024-02-18T00:33:50.606860+0000","last_active":"2024-02-18T00:33:59.617743+0000","last_peered":"2024-02-18T00:33:59.617743+0000","last_clean":"2024-02-18T00:33:59.617743+0000","last_became_active":"2024-02-18T00:33:23.592942+0000","last_became_peered":"2024-02-18T00:33:23.592942+0000","last_unstale":"2024-02-18T00:33:59.617743+0000","last_undegraded":"2024-02-18T00:33:59.617743+0000","last_fullsized":"2024-02-18T00:33:59.617743+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593469+0000","last_change":"2024-02-18T00:33:51.608593+0000","last_active":"2024-02-18T00:33:59.593469+0000","last_peered":"2024-02-18T00:33:59.593469+0000","last_clean":"2024-02-18T00:33:59.593469+0000","last_became_active":"2024-02-18T00:33:23.635584+0000","last_became_peered":"2024-02-18T00:33:23.635584+0000","last_unstale":"2024-02-18T00:33:59.593469+0000","last_undegraded":"2024-02-18T00:33:59.593469+0000","last_fullsized":"2024-02-18T00:33:59.593469+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"66.4","version":"0'0","reported_seq":30,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600778+0000","last_change":"2024-02-18T00:33:50.606183+0000","last_active":"2024-02-18T00:33:59.600778+0000","last_peered":"2024-02-18T00:33:59.600778+0000","last_clean":"2024-02-18T00:33:59.600778+0000","last_became_active":"2024-02-18T00:33:48.673040+0000","last_became_peered":"2024-02-18T00:33:48.673040+0000","last_unstale":"2024-02-18T00:33:59.600778+0000","last_undegraded":"2024-02-18T00:33:59.600778+0000","last_fullsized":"2024-02-18T00:33:59.600778+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":30,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.606516+0000","last_change":"2024-02-18T00:33:50.616747+0000","last_active":"2024-02-18T00:33:59.606516+0000","last_peered":"2024-02-18T00:33:59.606516+0000","last_clean":"2024-02-18T00:33:59.606516+0000","last_became_active":"2024-02-18T00:33:48.692427+0000","last_became_peered":"2024-02-18T00:33:48.692427+0000","last_unstale":"2024-02-18T00:33:59.606516+0000","last_undegraded":"2024-02-18T00:33:59.606516+0000","last_fullsized":"2024-02-18T00:33:59.606516+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.6","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601773+0000","last_change":"2024-02-18T00:33:50.599300+0000","last_active":"2024-02-18T00:33:59.601773+0000","last_peered":"2024-02-18T00:33:59.601773+0000","last_clean":"2024-02-18T00:33:59.601773+0000","last_became_active":"2024-02-18T00:33:23.456515+0000","last_became_peered":"2024-02-18T00:33:23.456515+0000","last_unstale":"2024-02-18T00:33:59.601773+0000","last_undegraded":"2024-02-18T00:33:59.601773+0000","last_fullsized":"2024-02-18T00:33:59.601773+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"146'5","reported_seq":99,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592131+0000","last_change":"2024-02-18T00:33:50.600353+0000","last_active":"2024-02-18T00:33:59.592131+0000","last_peered":"2024-02-18T00:33:59.592131+0000","last_clean":"2024-02-18T00:33:59.592131+0000","last_became_active":"2024-02-18T00:33:48.694103+0000","last_became_peered":"2024-02-18T00:33:48.694103+0000","last_unstale":"2024-02-18T00:33:59.592131+0000","last_undegraded":"2024-02-18T00:33:59.592131+0000","last_fullsized":"2024-02-18T00:33:59.592131+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.0","version":"0'0","reported_seq":30,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590592+0000","last_change":"2024-02-18T00:33:50.598253+0000","last_active":"2024-02-18T00:33:59.590592+0000","last_peered":"2024-02-18T00:33:59.590592+0000","last_clean":"2024-02-18T00:33:59.590592+0000","last_became_active":"2024-02-18T00:33:48.741269+0000","last_became_peered":"2024-02-18T00:33:48.741269+0000","last_unstale":"2024-02-18T00:33:59.590592+0000","last_undegraded":"2024-02-18T00:33:59.590592+0000","last_fullsized":"2024-02-18T00:33:59.590592+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":90,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594804+0000","last_change":"2024-02-18T00:33:50.608063+0000","last_active":"2024-02-18T00:33:59.594804+0000","last_peered":"2024-02-18T00:33:59.594804+0000","last_clean":"2024-02-18T00:33:59.594804+0000","last_became_active":"2024-02-18T00:33:23.457348+0000","last_became_peered":"2024-02-18T00:33:23.457348+0000","last_unstale":"2024-02-18T00:33:59.594804+0000","last_undegraded":"2024-02-18T00:33:59.594804+0000","last_fullsized":"2024-02-18T00:33:59.594804+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"0'0","reported_seq":98,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594191+0000","last_change":"2024-02-18T00:33:50.601973+0000","last_active":"2024-02-18T00:33:59.594191+0000","last_peered":"2024-02-18T00:33:59.594191+0000","last_clean":"2024-02-18T00:33:59.594191+0000","last_became_active":"2024-02-18T00:33:48.701731+0000","last_became_peered":"2024-02-18T00:33:48.701731+0000","last_unstale":"2024-02-18T00:33:59.594191+0000","last_undegraded":"2024-02-18T00:33:59.594191+0000","last_fullsized":"2024-02-18T00:33:59.594191+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:22.359279+0000","last_clean_scrub_stamp":"2024-02-18T00:33:22.359279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.0","version":"159'36","reported_seq":52,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.720241+0000","last_change":"2024-02-18T00:33:49.929435+0000","last_active":"2024-02-18T00:34:00.720241+0000","last_peered":"2024-02-18T00:34:00.720241+0000","last_clean":"2024-02-18T00:34:00.720241+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:34:00.720241+0000","last_undegraded":"2024-02-18T00:34:00.720241+0000","last_fullsized":"2024-02-18T00:34:00.720241+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"0'0","reported_seq":52,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591627+0000","last_change":"2024-02-18T00:33:47.891249+0000","last_active":"2024-02-18T00:33:59.591627+0000","last_peered":"2024-02-18T00:33:59.591627+0000","last_clean":"2024-02-18T00:33:59.591627+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:33:59.591627+0000","last_undegraded":"2024-02-18T00:33:59.591627+0000","last_fullsized":"2024-02-18T00:33:59.591627+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"159'36","reported_seq":81,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591704+0000","last_change":"2024-02-18T00:33:48.731235+0000","last_active":"2024-02-18T00:33:59.591704+0000","last_peered":"2024-02-18T00:33:59.591704+0000","last_clean":"2024-02-18T00:33:59.591704+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:33:59.591704+0000","last_undegraded":"2024-02-18T00:33:59.591704+0000","last_fullsized":"2024-02-18T00:33:59.591704+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"159'41","reported_seq":82,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.786569+0000","last_change":"2024-02-18T00:33:49.929195+0000","last_active":"2024-02-18T00:34:00.786569+0000","last_peered":"2024-02-18T00:34:00.786569+0000","last_clean":"2024-02-18T00:34:00.786569+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:34:00.786569+0000","last_undegraded":"2024-02-18T00:34:00.786569+0000","last_fullsized":"2024-02-18T00:34:00.786569+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":213,"num_objects":21,"num_object_clones":16,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"159'63","reported_seq":98,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.653487+0000","last_change":"2024-02-18T00:33:48.725155+0000","last_active":"2024-02-18T00:34:00.653487+0000","last_peered":"2024-02-18T00:34:00.653487+0000","last_clean":"2024-02-18T00:34:00.653487+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:34:00.653487+0000","last_undegraded":"2024-02-18T00:34:00.653487+0000","last_fullsized":"2024-02-18T00:34:00.653487+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":364,"num_objects":35,"num_object_clones":28,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"159'45","reported_seq":79,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.685632+0000","last_change":"2024-02-18T00:33:47.658160+0000","last_active":"2024-02-18T00:34:00.685632+0000","last_peered":"2024-02-18T00:34:00.685632+0000","last_clean":"2024-02-18T00:34:00.685632+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:34:00.685632+0000","last_undegraded":"2024-02-18T00:34:00.685632+0000","last_fullsized":"2024-02-18T00:34:00.685632+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":260,"num_objects":25,"num_object_clones":20,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"159'63","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595672+0000","last_change":"2024-02-18T00:33:49.929686+0000","last_active":"2024-02-18T00:33:59.595672+0000","last_peered":"2024-02-18T00:33:59.595672+0000","last_clean":"2024-02-18T00:33:59.595672+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:33:59.595672+0000","last_undegraded":"2024-02-18T00:33:59.595672+0000","last_fullsized":"2024-02-18T00:33:59.595672+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":63,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":364,"num_objects":35,"num_object_clones":28,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"159'36","reported_seq":77,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.747478+0000","last_change":"2024-02-18T00:33:49.850224+0000","last_active":"2024-02-18T00:34:00.747478+0000","last_peered":"2024-02-18T00:34:00.747478+0000","last_clean":"2024-02-18T00:34:00.747478+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:34:00.747478+0000","last_undegraded":"2024-02-18T00:34:00.747478+0000","last_fullsized":"2024-02-18T00:34:00.747478+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"159'36","reported_seq":53,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.699997+0000","last_change":"2024-02-18T00:33:48.712112+0000","last_active":"2024-02-18T00:34:00.699997+0000","last_peered":"2024-02-18T00:34:00.699997+0000","last_clean":"2024-02-18T00:34:00.699997+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:34:00.699997+0000","last_undegraded":"2024-02-18T00:34:00.699997+0000","last_fullsized":"2024-02-18T00:34:00.699997+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"159'27","reported_seq":67,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589834+0000","last_change":"2024-02-18T00:33:47.646434+0000","last_active":"2024-02-18T00:33:59.589834+0000","last_peered":"2024-02-18T00:33:59.589834+0000","last_clean":"2024-02-18T00:33:59.589834+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:33:59.589834+0000","last_undegraded":"2024-02-18T00:33:59.589834+0000","last_fullsized":"2024-02-18T00:33:59.589834+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":156,"num_objects":15,"num_object_clones":12,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"159'18","reported_seq":41,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.691308+0000","last_change":"2024-02-18T00:33:49.850472+0000","last_active":"2024-02-18T00:34:00.691308+0000","last_peered":"2024-02-18T00:34:00.691308+0000","last_clean":"2024-02-18T00:34:00.691308+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:34:00.691308+0000","last_undegraded":"2024-02-18T00:34:00.691308+0000","last_fullsized":"2024-02-18T00:34:00.691308+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":104,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"159'27","reported_seq":50,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.762707+0000","last_change":"2024-02-18T00:33:48.748034+0000","last_active":"2024-02-18T00:34:00.762707+0000","last_peered":"2024-02-18T00:34:00.762707+0000","last_clean":"2024-02-18T00:34:00.762707+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:34:00.762707+0000","last_undegraded":"2024-02-18T00:34:00.762707+0000","last_fullsized":"2024-02-18T00:34:00.762707+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":156,"num_objects":15,"num_object_clones":12,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"159'9","reported_seq":57,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597606+0000","last_change":"2024-02-18T00:33:47.731085+0000","last_active":"2024-02-18T00:33:59.597606+0000","last_peered":"2024-02-18T00:33:59.597606+0000","last_clean":"2024-02-18T00:33:59.597606+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:33:59.597606+0000","last_undegraded":"2024-02-18T00:33:59.597606+0000","last_fullsized":"2024-02-18T00:33:59.597606+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"159'10","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591818+0000","last_change":"2024-02-18T00:33:48.942577+0000","last_active":"2024-02-18T00:33:59.591818+0000","last_peered":"2024-02-18T00:33:59.591818+0000","last_clean":"2024-02-18T00:33:59.591818+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:33:59.591818+0000","last_undegraded":"2024-02-18T00:33:59.591818+0000","last_fullsized":"2024-02-18T00:33:59.591818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"159'36","reported_seq":75,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.727091+0000","last_change":"2024-02-18T00:33:47.696539+0000","last_active":"2024-02-18T00:34:00.727091+0000","last_peered":"2024-02-18T00:34:00.727091+0000","last_clean":"2024-02-18T00:34:00.727091+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:34:00.727091+0000","last_undegraded":"2024-02-18T00:34:00.727091+0000","last_fullsized":"2024-02-18T00:34:00.727091+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"159'36","reported_seq":74,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.697222+0000","last_change":"2024-02-18T00:33:47.658428+0000","last_active":"2024-02-18T00:34:00.697222+0000","last_peered":"2024-02-18T00:34:00.697222+0000","last_clean":"2024-02-18T00:34:00.697222+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:34:00.697222+0000","last_undegraded":"2024-02-18T00:34:00.697222+0000","last_fullsized":"2024-02-18T00:34:00.697222+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"159'32","reported_seq":92,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.648131+0000","last_change":"2024-02-18T00:33:50.845059+0000","last_active":"2024-02-18T00:34:00.648131+0000","last_peered":"2024-02-18T00:34:00.648131+0000","last_clean":"2024-02-18T00:34:00.648131+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:34:00.648131+0000","last_undegraded":"2024-02-18T00:34:00.648131+0000","last_fullsized":"2024-02-18T00:34:00.648131+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":156,"num_objects":15,"num_object_clones":12,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"159'9","reported_seq":57,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591547+0000","last_change":"2024-02-18T00:33:47.891050+0000","last_active":"2024-02-18T00:33:59.591547+0000","last_peered":"2024-02-18T00:33:59.591547+0000","last_clean":"2024-02-18T00:33:59.591547+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:33:59.591547+0000","last_undegraded":"2024-02-18T00:33:59.591547+0000","last_fullsized":"2024-02-18T00:33:59.591547+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"159'36","reported_seq":76,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.693899+0000","last_change":"2024-02-18T00:33:47.768270+0000","last_active":"2024-02-18T00:34:00.693899+0000","last_peered":"2024-02-18T00:34:00.693899+0000","last_clean":"2024-02-18T00:34:00.693899+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:34:00.693899+0000","last_undegraded":"2024-02-18T00:34:00.693899+0000","last_fullsized":"2024-02-18T00:34:00.693899+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"159'9","reported_seq":57,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.609814+0000","last_change":"2024-02-18T00:33:47.714621+0000","last_active":"2024-02-18T00:33:59.609814+0000","last_peered":"2024-02-18T00:33:59.609814+0000","last_clean":"2024-02-18T00:33:59.609814+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:33:59.609814+0000","last_undegraded":"2024-02-18T00:33:59.609814+0000","last_fullsized":"2024-02-18T00:33:59.609814+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"159'14","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591017+0000","last_change":"2024-02-18T00:33:49.875569+0000","last_active":"2024-02-18T00:33:59.591017+0000","last_peered":"2024-02-18T00:33:59.591017+0000","last_clean":"2024-02-18T00:33:59.591017+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:33:59.591017+0000","last_undegraded":"2024-02-18T00:33:59.591017+0000","last_fullsized":"2024-02-18T00:33:59.591017+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"159'9","reported_seq":66,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594369+0000","last_change":"2024-02-18T00:33:48.672626+0000","last_active":"2024-02-18T00:33:59.594369+0000","last_peered":"2024-02-18T00:33:59.594369+0000","last_clean":"2024-02-18T00:33:59.594369+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:33:59.594369+0000","last_undegraded":"2024-02-18T00:33:59.594369+0000","last_fullsized":"2024-02-18T00:33:59.594369+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"159'54","reported_seq":95,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.670118+0000","last_change":"2024-02-18T00:33:48.712914+0000","last_active":"2024-02-18T00:34:00.670118+0000","last_peered":"2024-02-18T00:34:00.670118+0000","last_clean":"2024-02-18T00:34:00.670118+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:34:00.670118+0000","last_undegraded":"2024-02-18T00:34:00.670118+0000","last_fullsized":"2024-02-18T00:34:00.670118+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":312,"num_objects":30,"num_object_clones":24,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":71,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596256+0000","last_change":"2024-02-18T00:33:48.630133+0000","last_active":"2024-02-18T00:33:59.596256+0000","last_peered":"2024-02-18T00:33:59.596256+0000","last_clean":"2024-02-18T00:33:59.596256+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:33:59.596256+0000","last_undegraded":"2024-02-18T00:33:59.596256+0000","last_fullsized":"2024-02-18T00:33:59.596256+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.960661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.960661+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"159'59","reported_seq":107,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.773026+0000","last_change":"2024-02-18T00:33:51.931833+0000","last_active":"2024-02-18T00:34:00.773026+0000","last_peered":"2024-02-18T00:34:00.773026+0000","last_clean":"2024-02-18T00:34:00.773026+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:34:00.773026+0000","last_undegraded":"2024-02-18T00:34:00.773026+0000","last_fullsized":"2024-02-18T00:34:00.773026+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":59,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":317,"num_objects":31,"num_object_clones":24,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":71,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592487+0000","last_change":"2024-02-18T00:33:48.701421+0000","last_active":"2024-02-18T00:33:59.592487+0000","last_peered":"2024-02-18T00:33:59.592487+0000","last_clean":"2024-02-18T00:33:59.592487+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:33:59.592487+0000","last_undegraded":"2024-02-18T00:33:59.592487+0000","last_fullsized":"2024-02-18T00:33:59.592487+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:48.594640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:48.594640+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"159'18","reported_seq":73,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.639577+0000","last_change":"2024-02-18T00:33:48.680482+0000","last_active":"2024-02-18T00:34:00.639577+0000","last_peered":"2024-02-18T00:34:00.639577+0000","last_clean":"2024-02-18T00:34:00.639577+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:34:00.639577+0000","last_undegraded":"2024-02-18T00:34:00.639577+0000","last_fullsized":"2024-02-18T00:34:00.639577+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":104,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":27,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591352+0000","last_change":"2024-02-18T00:33:48.733882+0000","last_active":"2024-02-18T00:33:59.591352+0000","last_peered":"2024-02-18T00:33:59.591352+0000","last_clean":"2024-02-18T00:33:59.591352+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:33:59.591352+0000","last_undegraded":"2024-02-18T00:33:59.591352+0000","last_fullsized":"2024-02-18T00:33:59.591352+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.198542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.198542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"159'55","reported_seq":102,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.723627+0000","last_change":"2024-02-18T00:33:48.708452+0000","last_active":"2024-02-18T00:34:00.723627+0000","last_peered":"2024-02-18T00:34:00.723627+0000","last_clean":"2024-02-18T00:34:00.723627+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:34:00.723627+0000","last_undegraded":"2024-02-18T00:34:00.723627+0000","last_fullsized":"2024-02-18T00:34:00.723627+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":316,"num_objects":31,"num_object_clones":24,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":25,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594084+0000","last_change":"2024-02-18T00:33:48.637145+0000","last_active":"2024-02-18T00:33:59.594084+0000","last_peered":"2024-02-18T00:33:59.594084+0000","last_clean":"2024-02-18T00:33:59.594084+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:33:59.594084+0000","last_undegraded":"2024-02-18T00:33:59.594084+0000","last_fullsized":"2024-02-18T00:33:59.594084+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:31:38.977852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"159'36","reported_seq":81,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.730801+0000","last_change":"2024-02-18T00:33:47.653849+0000","last_active":"2024-02-18T00:34:00.730801+0000","last_peered":"2024-02-18T00:34:00.730801+0000","last_clean":"2024-02-18T00:34:00.730801+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:34:00.730801+0000","last_undegraded":"2024-02-18T00:34:00.730801+0000","last_fullsized":"2024-02-18T00:34:00.730801+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":36,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":208,"num_objects":20,"num_object_clones":16,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":82,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592853+0000","last_change":"2024-02-18T00:33:43.346148+0000","last_active":"2024-02-18T00:33:59.592853+0000","last_peered":"2024-02-18T00:33:59.592853+0000","last_clean":"2024-02-18T00:33:59.592853+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:33:59.592853+0000","last_undegraded":"2024-02-18T00:33:59.592853+0000","last_fullsized":"2024-02-18T00:33:59.592853+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:43.346088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.346088+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"159'9","reported_seq":66,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592302+0000","last_change":"2024-02-18T00:33:48.693268+0000","last_active":"2024-02-18T00:33:59.592302+0000","last_peered":"2024-02-18T00:33:59.592302+0000","last_clean":"2024-02-18T00:33:59.592302+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:33:59.592302+0000","last_undegraded":"2024-02-18T00:33:59.592302+0000","last_fullsized":"2024-02-18T00:33:59.592302+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":71,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593290+0000","last_change":"2024-02-18T00:33:48.729510+0000","last_active":"2024-02-18T00:33:59.593290+0000","last_peered":"2024-02-18T00:33:59.593290+0000","last_clean":"2024-02-18T00:33:59.593290+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:33:59.593290+0000","last_undegraded":"2024-02-18T00:33:59.593290+0000","last_fullsized":"2024-02-18T00:33:59.593290+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.839018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.839018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"159'27","reported_seq":78,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.682016+0000","last_change":"2024-02-18T00:33:49.929918+0000","last_active":"2024-02-18T00:34:00.682016+0000","last_peered":"2024-02-18T00:34:00.682016+0000","last_clean":"2024-02-18T00:34:00.682016+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:34:00.682016+0000","last_undegraded":"2024-02-18T00:34:00.682016+0000","last_fullsized":"2024-02-18T00:34:00.682016+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":156,"num_objects":15,"num_object_clones":12,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":67,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594757+0000","last_change":"2024-02-18T00:33:55.783013+0000","last_active":"2024-02-18T00:33:59.594757+0000","last_peered":"2024-02-18T00:33:59.594757+0000","last_clean":"2024-02-18T00:33:59.594757+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:33:59.594757+0000","last_undegraded":"2024-02-18T00:33:59.594757+0000","last_fullsized":"2024-02-18T00:33:59.594757+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.782955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.782955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"159'18","reported_seq":41,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.676660+0000","last_change":"2024-02-18T00:33:48.733067+0000","last_active":"2024-02-18T00:34:00.676660+0000","last_peered":"2024-02-18T00:34:00.676660+0000","last_clean":"2024-02-18T00:34:00.676660+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:34:00.676660+0000","last_undegraded":"2024-02-18T00:34:00.676660+0000","last_fullsized":"2024-02-18T00:34:00.676660+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":104,"num_objects":10,"num_object_clones":8,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":69,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592240+0000","last_change":"2024-02-18T00:33:49.344703+0000","last_active":"2024-02-18T00:33:59.592240+0000","last_peered":"2024-02-18T00:33:59.592240+0000","last_clean":"2024-02-18T00:33:59.592240+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:33:59.592240+0000","last_undegraded":"2024-02-18T00:33:59.592240+0000","last_fullsized":"2024-02-18T00:33:59.592240+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.344637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.344637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"159'9","reported_seq":59,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.673766+0000","last_change":"2024-02-18T00:33:47.694682+0000","last_active":"2024-02-18T00:34:00.673766+0000","last_peered":"2024-02-18T00:34:00.673766+0000","last_clean":"2024-02-18T00:34:00.673766+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:34:00.673766+0000","last_undegraded":"2024-02-18T00:34:00.673766+0000","last_fullsized":"2024-02-18T00:34:00.673766+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593711+0000","last_change":"2024-02-18T00:33:48.736504+0000","last_active":"2024-02-18T00:33:59.593711+0000","last_peered":"2024-02-18T00:33:59.593711+0000","last_clean":"2024-02-18T00:33:59.593711+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:33:59.593711+0000","last_undegraded":"2024-02-18T00:33:59.593711+0000","last_fullsized":"2024-02-18T00:33:59.593711+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.834878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.834878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"159'9","reported_seq":57,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593201+0000","last_change":"2024-02-18T00:33:47.684299+0000","last_active":"2024-02-18T00:33:59.593201+0000","last_peered":"2024-02-18T00:33:59.593201+0000","last_clean":"2024-02-18T00:33:59.593201+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:33:59.593201+0000","last_undegraded":"2024-02-18T00:33:59.593201+0000","last_fullsized":"2024-02-18T00:33:59.593201+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":4,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":96,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599846+0000","last_change":"2024-02-18T00:33:48.629409+0000","last_active":"2024-02-18T00:33:59.599846+0000","last_peered":"2024-02-18T00:33:59.599846+0000","last_clean":"2024-02-18T00:33:59.599846+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:33:59.599846+0000","last_undegraded":"2024-02-18T00:33:59.599846+0000","last_fullsized":"2024-02-18T00:33:59.599846+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.754417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.754417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.16","version":"109'5","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.635621+0000","last_change":"2024-02-18T00:33:59.635375+0000","last_active":"2024-02-18T00:33:59.635621+0000","last_peered":"2024-02-18T00:33:59.635621+0000","last_clean":"2024-02-18T00:33:59.635621+0000","last_became_active":"2024-02-18T00:33:59.622391+0000","last_became_peered":"2024-02-18T00:33:59.622391+0000","last_unstale":"2024-02-18T00:33:59.635621+0000","last_undegraded":"2024-02-18T00:33:59.635621+0000","last_fullsized":"2024-02-18T00:33:59.635621+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.611795+0000","last_change":"2024-02-18T00:33:48.727714+0000","last_active":"2024-02-18T00:33:59.611795+0000","last_peered":"2024-02-18T00:33:59.611795+0000","last_clean":"2024-02-18T00:33:59.611795+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:33:59.611795+0000","last_undegraded":"2024-02-18T00:33:59.611795+0000","last_fullsized":"2024-02-18T00:33:59.611795+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":50,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596878+0000","last_change":"2024-02-18T00:33:48.709199+0000","last_active":"2024-02-18T00:33:59.596878+0000","last_peered":"2024-02-18T00:33:59.596878+0000","last_clean":"2024-02-18T00:33:59.596878+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:33:59.596878+0000","last_undegraded":"2024-02-18T00:33:59.596878+0000","last_fullsized":"2024-02-18T00:33:59.596878+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:50.594799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.594799+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":206,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599934+0000","last_change":"2024-02-18T00:33:48.645777+0000","last_active":"2024-02-18T00:33:59.599934+0000","last_peered":"2024-02-18T00:33:59.599934+0000","last_clean":"2024-02-18T00:33:59.599934+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:33:59.599934+0000","last_undegraded":"2024-02-18T00:33:59.599934+0000","last_fullsized":"2024-02-18T00:33:59.599934+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:32:46.877252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.877252+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":133,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596129+0000","last_change":"2024-02-18T00:33:48.691330+0000","last_active":"2024-02-18T00:33:59.596129+0000","last_peered":"2024-02-18T00:33:59.596129+0000","last_clean":"2024-02-18T00:33:59.596129+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:33:59.596129+0000","last_undegraded":"2024-02-18T00:33:59.596129+0000","last_fullsized":"2024-02-18T00:33:59.596129+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:57.561435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:57.561435+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601151+0000","last_change":"2024-02-18T00:32:05.790164+0000","last_active":"2024-02-18T00:33:59.601151+0000","last_peered":"2024-02-18T00:33:59.601151+0000","last_clean":"2024-02-18T00:33:59.601151+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:33:59.601151+0000","last_undegraded":"2024-02-18T00:33:59.601151+0000","last_fullsized":"2024-02-18T00:33:59.601151+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.790091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.790091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":25,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593710+0000","last_change":"2024-02-18T00:33:48.726230+0000","last_active":"2024-02-18T00:33:59.593710+0000","last_peered":"2024-02-18T00:33:59.593710+0000","last_clean":"2024-02-18T00:33:59.593710+0000","last_became_active":"2024-02-18T00:33:48.725862+0000","last_became_peered":"2024-02-18T00:33:48.725862+0000","last_unstale":"2024-02-18T00:33:59.593710+0000","last_undegraded":"2024-02-18T00:33:59.593710+0000","last_fullsized":"2024-02-18T00:33:59.593710+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.626365+0000","last_change":"2024-02-18T00:33:59.624410+0000","last_active":"2024-02-18T00:33:59.626365+0000","last_peered":"2024-02-18T00:33:59.626365+0000","last_clean":"2024-02-18T00:33:59.626365+0000","last_became_active":"2024-02-18T00:33:59.623284+0000","last_became_peered":"2024-02-18T00:33:59.623284+0000","last_unstale":"2024-02-18T00:33:59.626365+0000","last_undegraded":"2024-02-18T00:33:59.626365+0000","last_fullsized":"2024-02-18T00:33:59.626365+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599751+0000","last_change":"2024-02-18T00:33:02.535442+0000","last_active":"2024-02-18T00:33:59.599751+0000","last_peered":"2024-02-18T00:33:59.599751+0000","last_clean":"2024-02-18T00:33:59.599751+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:33:59.599751+0000","last_undegraded":"2024-02-18T00:33:59.599751+0000","last_fullsized":"2024-02-18T00:33:59.599751+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":152,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.610686+0000","last_change":"2024-02-18T00:32:50.007673+0000","last_active":"2024-02-18T00:33:59.610686+0000","last_peered":"2024-02-18T00:33:59.610686+0000","last_clean":"2024-02-18T00:33:59.610686+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:33:59.610686+0000","last_undegraded":"2024-02-18T00:33:59.610686+0000","last_fullsized":"2024-02-18T00:33:59.610686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.007594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.007594+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597650+0000","last_change":"2024-02-18T00:32:03.931078+0000","last_active":"2024-02-18T00:33:59.597650+0000","last_peered":"2024-02-18T00:33:59.597650+0000","last_clean":"2024-02-18T00:33:59.597650+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:33:59.597650+0000","last_undegraded":"2024-02-18T00:33:59.597650+0000","last_fullsized":"2024-02-18T00:33:59.597650+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:32:03.930967+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":125,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594313+0000","last_change":"2024-02-18T00:33:42.768749+0000","last_active":"2024-02-18T00:33:59.594313+0000","last_peered":"2024-02-18T00:33:59.594313+0000","last_clean":"2024-02-18T00:33:59.594313+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:33:59.594313+0000","last_undegraded":"2024-02-18T00:33:59.594313+0000","last_fullsized":"2024-02-18T00:33:59.594313+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.768630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.768630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":126,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.604834+0000","last_change":"2024-02-18T00:33:48.734849+0000","last_active":"2024-02-18T00:33:59.604834+0000","last_peered":"2024-02-18T00:33:59.604834+0000","last_clean":"2024-02-18T00:33:59.604834+0000","last_became_active":"2024-02-18T00:33:48.734414+0000","last_became_peered":"2024-02-18T00:33:48.734414+0000","last_unstale":"2024-02-18T00:33:59.604834+0000","last_undegraded":"2024-02-18T00:33:59.604834+0000","last_fullsized":"2024-02-18T00:33:59.604834+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:39.973040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:39.973040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"106'2","reported_seq":82,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599673+0000","last_change":"2024-02-18T00:33:48.796853+0000","last_active":"2024-02-18T00:33:59.599673+0000","last_peered":"2024-02-18T00:33:59.599673+0000","last_clean":"2024-02-18T00:33:59.599673+0000","last_became_active":"2024-02-18T00:33:48.709644+0000","last_became_peered":"2024-02-18T00:33:48.709644+0000","last_unstale":"2024-02-18T00:33:59.599673+0000","last_undegraded":"2024-02-18T00:33:59.599673+0000","last_fullsized":"2024-02-18T00:33:59.599673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.14","version":"109'2","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.647941+0000","last_change":"2024-02-18T00:33:59.647620+0000","last_active":"2024-02-18T00:33:59.647941+0000","last_peered":"2024-02-18T00:33:59.647941+0000","last_clean":"2024-02-18T00:33:59.647941+0000","last_became_active":"2024-02-18T00:33:59.628045+0000","last_became_peered":"2024-02-18T00:33:59.628045+0000","last_unstale":"2024-02-18T00:33:59.647941+0000","last_undegraded":"2024-02-18T00:33:59.647941+0000","last_fullsized":"2024-02-18T00:33:59.647941+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.607118+0000","last_change":"2024-02-18T00:33:48.745303+0000","last_active":"2024-02-18T00:33:59.607118+0000","last_peered":"2024-02-18T00:33:59.607118+0000","last_clean":"2024-02-18T00:33:59.607118+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:33:59.607118+0000","last_undegraded":"2024-02-18T00:33:59.607118+0000","last_fullsized":"2024-02-18T00:33:59.607118+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":205,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592382+0000","last_change":"2024-02-18T00:33:48.952048+0000","last_active":"2024-02-18T00:33:59.592382+0000","last_peered":"2024-02-18T00:33:59.592382+0000","last_clean":"2024-02-18T00:33:59.592382+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:33:59.592382+0000","last_undegraded":"2024-02-18T00:33:59.592382+0000","last_fullsized":"2024-02-18T00:33:59.592382+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:32:53.370869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.370869+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595593+0000","last_change":"2024-02-18T00:33:46.861306+0000","last_active":"2024-02-18T00:33:59.595593+0000","last_peered":"2024-02-18T00:33:59.595593+0000","last_clean":"2024-02-18T00:33:59.595593+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:33:59.595593+0000","last_undegraded":"2024-02-18T00:33:59.595593+0000","last_fullsized":"2024-02-18T00:33:59.595593+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.861233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.861233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":125,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590447+0000","last_change":"2024-02-18T00:33:53.005012+0000","last_active":"2024-02-18T00:33:59.590447+0000","last_peered":"2024-02-18T00:33:59.590447+0000","last_clean":"2024-02-18T00:33:59.590447+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:33:59.590447+0000","last_undegraded":"2024-02-18T00:33:59.590447+0000","last_fullsized":"2024-02-18T00:33:59.590447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.004903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.004903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597797+0000","last_change":"2024-02-18T00:33:48.695636+0000","last_active":"2024-02-18T00:33:59.597797+0000","last_peered":"2024-02-18T00:33:59.597797+0000","last_clean":"2024-02-18T00:33:59.597797+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:33:59.597797+0000","last_undegraded":"2024-02-18T00:33:59.597797+0000","last_fullsized":"2024-02-18T00:33:59.597797+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.665830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.665830+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":75,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595556+0000","last_change":"2024-02-18T00:33:48.672466+0000","last_active":"2024-02-18T00:33:59.595556+0000","last_peered":"2024-02-18T00:33:59.595556+0000","last_clean":"2024-02-18T00:33:59.595556+0000","last_became_active":"2024-02-18T00:33:48.672037+0000","last_became_peered":"2024-02-18T00:33:48.672037+0000","last_unstale":"2024-02-18T00:33:59.595556+0000","last_undegraded":"2024-02-18T00:33:59.595556+0000","last_fullsized":"2024-02-18T00:33:59.595556+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"109'9","reported_seq":42,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.416711+0000","last_change":"2024-02-18T00:34:00.416383+0000","last_active":"2024-02-18T00:34:00.416711+0000","last_peered":"2024-02-18T00:34:00.416711+0000","last_clean":"2024-02-18T00:34:00.416711+0000","last_became_active":"2024-02-18T00:33:59.628637+0000","last_became_peered":"2024-02-18T00:33:59.628637+0000","last_unstale":"2024-02-18T00:34:00.416711+0000","last_undegraded":"2024-02-18T00:34:00.416711+0000","last_fullsized":"2024-02-18T00:34:00.416711+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600212+0000","last_change":"2024-02-18T00:33:02.523582+0000","last_active":"2024-02-18T00:33:59.600212+0000","last_peered":"2024-02-18T00:33:59.600212+0000","last_clean":"2024-02-18T00:33:59.600212+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:33:59.600212+0000","last_undegraded":"2024-02-18T00:33:59.600212+0000","last_fullsized":"2024-02-18T00:33:59.600212+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":155,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592701+0000","last_change":"2024-02-18T00:33:02.051762+0000","last_active":"2024-02-18T00:33:59.592701+0000","last_peered":"2024-02-18T00:33:59.592701+0000","last_clean":"2024-02-18T00:33:59.592701+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:33:59.592701+0000","last_undegraded":"2024-02-18T00:33:59.592701+0000","last_fullsized":"2024-02-18T00:33:59.592701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:02.051679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.051679+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591340+0000","last_change":"2024-02-18T00:33:48.669006+0000","last_active":"2024-02-18T00:33:59.591340+0000","last_peered":"2024-02-18T00:33:59.591340+0000","last_clean":"2024-02-18T00:33:59.591340+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:33:59.591340+0000","last_undegraded":"2024-02-18T00:33:59.591340+0000","last_fullsized":"2024-02-18T00:33:59.591340+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:54.616213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.616213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599184+0000","last_change":"2024-02-18T00:32:45.751973+0000","last_active":"2024-02-18T00:33:59.599184+0000","last_peered":"2024-02-18T00:33:59.599184+0000","last_clean":"2024-02-18T00:33:59.599184+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:33:59.599184+0000","last_undegraded":"2024-02-18T00:33:59.599184+0000","last_fullsized":"2024-02-18T00:33:59.599184+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:45.751874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.751874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":123,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601052+0000","last_change":"2024-02-18T00:33:45.813625+0000","last_active":"2024-02-18T00:33:59.601052+0000","last_peered":"2024-02-18T00:33:59.601052+0000","last_clean":"2024-02-18T00:33:59.601052+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:33:59.601052+0000","last_undegraded":"2024-02-18T00:33:59.601052+0000","last_fullsized":"2024-02-18T00:33:59.601052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.813514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.813514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":75,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.605336+0000","last_change":"2024-02-18T00:33:48.757100+0000","last_active":"2024-02-18T00:33:59.605336+0000","last_peered":"2024-02-18T00:33:59.605336+0000","last_clean":"2024-02-18T00:33:59.605336+0000","last_became_active":"2024-02-18T00:33:48.741851+0000","last_became_peered":"2024-02-18T00:33:48.741851+0000","last_unstale":"2024-02-18T00:33:59.605336+0000","last_undegraded":"2024-02-18T00:33:59.605336+0000","last_fullsized":"2024-02-18T00:33:59.605336+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.629487+0000","last_change":"2024-02-18T00:33:59.627271+0000","last_active":"2024-02-18T00:33:59.629487+0000","last_peered":"2024-02-18T00:33:59.629487+0000","last_clean":"2024-02-18T00:33:59.629487+0000","last_became_active":"2024-02-18T00:33:59.626904+0000","last_became_peered":"2024-02-18T00:33:59.626904+0000","last_unstale":"2024-02-18T00:33:59.629487+0000","last_undegraded":"2024-02-18T00:33:59.629487+0000","last_fullsized":"2024-02-18T00:33:59.629487+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.604918+0000","last_change":"2024-02-18T00:33:48.733994+0000","last_active":"2024-02-18T00:33:59.604918+0000","last_peered":"2024-02-18T00:33:59.604918+0000","last_clean":"2024-02-18T00:33:59.604918+0000","last_became_active":"2024-02-18T00:33:48.732220+0000","last_became_peered":"2024-02-18T00:33:48.732220+0000","last_unstale":"2024-02-18T00:33:59.604918+0000","last_undegraded":"2024-02-18T00:33:59.604918+0000","last_fullsized":"2024-02-18T00:33:59.604918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":137,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590555+0000","last_change":"2024-02-18T00:32:43.969645+0000","last_active":"2024-02-18T00:33:59.590555+0000","last_peered":"2024-02-18T00:33:59.590555+0000","last_clean":"2024-02-18T00:33:59.590555+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:33:59.590555+0000","last_undegraded":"2024-02-18T00:33:59.590555+0000","last_fullsized":"2024-02-18T00:33:59.590555+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:32:43.969566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.969566+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":133,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594985+0000","last_change":"2024-02-18T00:33:48.704797+0000","last_active":"2024-02-18T00:33:59.594985+0000","last_peered":"2024-02-18T00:33:59.594985+0000","last_clean":"2024-02-18T00:33:59.594985+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:33:59.594985+0000","last_undegraded":"2024-02-18T00:33:59.594985+0000","last_fullsized":"2024-02-18T00:33:59.594985+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:31:50.988429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":201,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593711+0000","last_change":"2024-02-18T00:33:48.802648+0000","last_active":"2024-02-18T00:33:59.593711+0000","last_peered":"2024-02-18T00:33:59.593711+0000","last_clean":"2024-02-18T00:33:59.593711+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:33:59.593711+0000","last_undegraded":"2024-02-18T00:33:59.593711+0000","last_fullsized":"2024-02-18T00:33:59.593711+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:46.413894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.413894+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594694+0000","last_change":"2024-02-18T00:31:58.064645+0000","last_active":"2024-02-18T00:33:59.594694+0000","last_peered":"2024-02-18T00:33:59.594694+0000","last_clean":"2024-02-18T00:33:59.594694+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:33:59.594694+0000","last_undegraded":"2024-02-18T00:33:59.594694+0000","last_fullsized":"2024-02-18T00:33:59.594694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.064559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.064559+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"136'2","reported_seq":75,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600712+0000","last_change":"2024-02-18T00:33:01.434399+0000","last_active":"2024-02-18T00:33:59.600712+0000","last_peered":"2024-02-18T00:33:59.600712+0000","last_clean":"2024-02-18T00:33:59.600712+0000","last_became_active":"2024-02-18T00:33:01.433012+0000","last_became_peered":"2024-02-18T00:33:01.433012+0000","last_unstale":"2024-02-18T00:33:59.600712+0000","last_undegraded":"2024-02-18T00:33:59.600712+0000","last_fullsized":"2024-02-18T00:33:59.600712+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":50331664,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":24577,"num_write":2,"num_write_kb":49154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.11","version":"109'7","reported_seq":19,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.627357+0000","last_change":"2024-02-18T00:33:59.627084+0000","last_active":"2024-02-18T00:33:59.627357+0000","last_peered":"2024-02-18T00:33:59.627357+0000","last_clean":"2024-02-18T00:33:59.627357+0000","last_became_active":"2024-02-18T00:33:59.604140+0000","last_became_peered":"2024-02-18T00:33:59.604140+0000","last_unstale":"2024-02-18T00:33:59.627357+0000","last_undegraded":"2024-02-18T00:33:59.627357+0000","last_fullsized":"2024-02-18T00:33:59.627357+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595176+0000","last_change":"2024-02-18T00:33:02.536636+0000","last_active":"2024-02-18T00:33:59.595176+0000","last_peered":"2024-02-18T00:33:59.595176+0000","last_clean":"2024-02-18T00:33:59.595176+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:33:59.595176+0000","last_undegraded":"2024-02-18T00:33:59.595176+0000","last_fullsized":"2024-02-18T00:33:59.595176+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":130,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.603217+0000","last_change":"2024-02-18T00:33:44.007884+0000","last_active":"2024-02-18T00:33:59.603217+0000","last_peered":"2024-02-18T00:33:59.603217+0000","last_clean":"2024-02-18T00:33:59.603217+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:33:59.603217+0000","last_undegraded":"2024-02-18T00:33:59.603217+0000","last_fullsized":"2024-02-18T00:33:59.603217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:33:44.007804+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:33:44.007804+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594490+0000","last_change":"2024-02-18T00:33:48.713658+0000","last_active":"2024-02-18T00:33:59.594490+0000","last_peered":"2024-02-18T00:33:59.594490+0000","last_clean":"2024-02-18T00:33:59.594490+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:33:59.594490+0000","last_undegraded":"2024-02-18T00:33:59.594490+0000","last_fullsized":"2024-02-18T00:33:59.594490+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:33:47.199009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":185,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594612+0000","last_change":"2024-02-18T00:32:49.092381+0000","last_active":"2024-02-18T00:33:59.594612+0000","last_peered":"2024-02-18T00:33:59.594612+0000","last_clean":"2024-02-18T00:33:59.594612+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:33:59.594612+0000","last_undegraded":"2024-02-18T00:33:59.594612+0000","last_fullsized":"2024-02-18T00:33:59.594612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:49.092293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.092293+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601430+0000","last_change":"2024-02-18T00:33:41.057722+0000","last_active":"2024-02-18T00:33:59.601430+0000","last_peered":"2024-02-18T00:33:59.601430+0000","last_clean":"2024-02-18T00:33:59.601430+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:33:59.601430+0000","last_undegraded":"2024-02-18T00:33:59.601430+0000","last_fullsized":"2024-02-18T00:33:59.601430+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.057659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.057659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1a","version":"106'3","reported_seq":69,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.618264+0000","last_change":"2024-02-18T00:33:01.569847+0000","last_active":"2024-02-18T00:33:59.618264+0000","last_peered":"2024-02-18T00:33:59.618264+0000","last_clean":"2024-02-18T00:33:59.618264+0000","last_became_active":"2024-02-18T00:33:01.569569+0000","last_became_peered":"2024-02-18T00:33:01.569569+0000","last_unstale":"2024-02-18T00:33:59.618264+0000","last_undegraded":"2024-02-18T00:33:59.618264+0000","last_fullsized":"2024-02-18T00:33:59.618264+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.615397+0000","last_change":"2024-02-18T00:33:59.615109+0000","last_active":"2024-02-18T00:33:59.615397+0000","last_peered":"2024-02-18T00:33:59.615397+0000","last_clean":"2024-02-18T00:33:59.615397+0000","last_became_active":"2024-02-18T00:33:59.614727+0000","last_became_peered":"2024-02-18T00:33:59.614727+0000","last_unstale":"2024-02-18T00:33:59.615397+0000","last_undegraded":"2024-02-18T00:33:59.615397+0000","last_fullsized":"2024-02-18T00:33:59.615397+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590598+0000","last_change":"2024-02-18T00:33:02.533944+0000","last_active":"2024-02-18T00:33:59.590598+0000","last_peered":"2024-02-18T00:33:59.590598+0000","last_clean":"2024-02-18T00:33:59.590598+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:33:59.590598+0000","last_undegraded":"2024-02-18T00:33:59.590598+0000","last_fullsized":"2024-02-18T00:33:59.590598+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":161,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594237+0000","last_change":"2024-02-18T00:33:45.311343+0000","last_active":"2024-02-18T00:33:59.594237+0000","last_peered":"2024-02-18T00:33:59.594237+0000","last_clean":"2024-02-18T00:33:59.594237+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:33:59.594237+0000","last_undegraded":"2024-02-18T00:33:59.594237+0000","last_fullsized":"2024-02-18T00:33:59.594237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:45.311259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.311259+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":95,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591223+0000","last_change":"2024-02-18T00:33:48.742196+0000","last_active":"2024-02-18T00:33:59.591223+0000","last_peered":"2024-02-18T00:33:59.591223+0000","last_clean":"2024-02-18T00:33:59.591223+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:33:59.591223+0000","last_undegraded":"2024-02-18T00:33:59.591223+0000","last_fullsized":"2024-02-18T00:33:59.591223+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.793236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.793236+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.603409+0000","last_change":"2024-02-18T00:32:47.915174+0000","last_active":"2024-02-18T00:33:59.603409+0000","last_peered":"2024-02-18T00:33:59.603409+0000","last_clean":"2024-02-18T00:33:59.603409+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:33:59.603409+0000","last_undegraded":"2024-02-18T00:33:59.603409+0000","last_fullsized":"2024-02-18T00:33:59.603409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:32:47.915092+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597807+0000","last_change":"2024-02-18T00:31:58.530850+0000","last_active":"2024-02-18T00:33:59.597807+0000","last_peered":"2024-02-18T00:33:59.597807+0000","last_clean":"2024-02-18T00:33:59.597807+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:33:59.597807+0000","last_undegraded":"2024-02-18T00:33:59.597807+0000","last_fullsized":"2024-02-18T00:33:59.597807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.530768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.530768+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":66,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590516+0000","last_change":"2024-02-18T00:33:01.450143+0000","last_active":"2024-02-18T00:33:59.590516+0000","last_peered":"2024-02-18T00:33:59.590516+0000","last_clean":"2024-02-18T00:33:59.590516+0000","last_became_active":"2024-02-18T00:33:01.449972+0000","last_became_peered":"2024-02-18T00:33:01.449972+0000","last_unstale":"2024-02-18T00:33:59.590516+0000","last_undegraded":"2024-02-18T00:33:59.590516+0000","last_fullsized":"2024-02-18T00:33:59.590516+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"109'5","reported_seq":31,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.880058+0000","last_change":"2024-02-18T00:33:59.879870+0000","last_active":"2024-02-18T00:33:59.880058+0000","last_peered":"2024-02-18T00:33:59.880058+0000","last_clean":"2024-02-18T00:33:59.880058+0000","last_became_active":"2024-02-18T00:33:59.606454+0000","last_became_peered":"2024-02-18T00:33:59.606454+0000","last_unstale":"2024-02-18T00:33:59.880058+0000","last_undegraded":"2024-02-18T00:33:59.880058+0000","last_fullsized":"2024-02-18T00:33:59.880058+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590851+0000","last_change":"2024-02-18T00:33:02.483392+0000","last_active":"2024-02-18T00:33:59.590851+0000","last_peered":"2024-02-18T00:33:59.590851+0000","last_clean":"2024-02-18T00:33:59.590851+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:33:59.590851+0000","last_undegraded":"2024-02-18T00:33:59.590851+0000","last_fullsized":"2024-02-18T00:33:59.590851+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":191,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592900+0000","last_change":"2024-02-18T00:33:54.753052+0000","last_active":"2024-02-18T00:33:59.592900+0000","last_peered":"2024-02-18T00:33:59.592900+0000","last_clean":"2024-02-18T00:33:59.592900+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:33:59.592900+0000","last_undegraded":"2024-02-18T00:33:59.592900+0000","last_fullsized":"2024-02-18T00:33:59.592900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:33:54.752985+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.752985+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":163,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591560+0000","last_change":"2024-02-18T00:33:53.320429+0000","last_active":"2024-02-18T00:33:59.591560+0000","last_peered":"2024-02-18T00:33:59.591560+0000","last_clean":"2024-02-18T00:33:59.591560+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:33:59.591560+0000","last_undegraded":"2024-02-18T00:33:59.591560+0000","last_fullsized":"2024-02-18T00:33:59.591560+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:33:53.320356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.320356+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594479+0000","last_change":"2024-02-18T00:33:48.672342+0000","last_active":"2024-02-18T00:33:59.594479+0000","last_peered":"2024-02-18T00:33:59.594479+0000","last_clean":"2024-02-18T00:33:59.594479+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:33:59.594479+0000","last_undegraded":"2024-02-18T00:33:59.594479+0000","last_fullsized":"2024-02-18T00:33:59.594479+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.368010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.368010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595080+0000","last_change":"2024-02-18T00:33:48.672321+0000","last_active":"2024-02-18T00:33:59.595080+0000","last_peered":"2024-02-18T00:33:59.595080+0000","last_clean":"2024-02-18T00:33:59.595080+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:33:59.595080+0000","last_undegraded":"2024-02-18T00:33:59.595080+0000","last_fullsized":"2024-02-18T00:33:59.595080+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:47.565450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:47.565450+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.14","version":"136'1","reported_seq":70,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.598543+0000","last_change":"2024-02-18T00:33:01.532614+0000","last_active":"2024-02-18T00:33:59.598543+0000","last_peered":"2024-02-18T00:33:59.598543+0000","last_clean":"2024-02-18T00:33:59.598543+0000","last_became_active":"2024-02-18T00:33:01.516952+0000","last_became_peered":"2024-02-18T00:33:01.516952+0000","last_unstale":"2024-02-18T00:33:59.598543+0000","last_undegraded":"2024-02-18T00:33:59.598543+0000","last_fullsized":"2024-02-18T00:33:59.598543+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.623224+0000","last_change":"2024-02-18T00:33:59.622769+0000","last_active":"2024-02-18T00:33:59.623224+0000","last_peered":"2024-02-18T00:33:59.623224+0000","last_clean":"2024-02-18T00:33:59.623224+0000","last_became_active":"2024-02-18T00:33:59.622390+0000","last_became_peered":"2024-02-18T00:33:59.622390+0000","last_unstale":"2024-02-18T00:33:59.623224+0000","last_undegraded":"2024-02-18T00:33:59.623224+0000","last_fullsized":"2024-02-18T00:33:59.623224+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601118+0000","last_change":"2024-02-18T00:33:02.477932+0000","last_active":"2024-02-18T00:33:59.601118+0000","last_peered":"2024-02-18T00:33:59.601118+0000","last_clean":"2024-02-18T00:33:59.601118+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:33:59.601118+0000","last_undegraded":"2024-02-18T00:33:59.601118+0000","last_fullsized":"2024-02-18T00:33:59.601118+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":134,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591913+0000","last_change":"2024-02-18T00:33:50.287103+0000","last_active":"2024-02-18T00:33:59.591913+0000","last_peered":"2024-02-18T00:33:59.591913+0000","last_clean":"2024-02-18T00:33:59.591913+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:33:59.591913+0000","last_undegraded":"2024-02-18T00:33:59.591913+0000","last_fullsized":"2024-02-18T00:33:59.591913+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.287007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.287007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":225,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592808+0000","last_change":"2024-02-18T00:33:49.076501+0000","last_active":"2024-02-18T00:33:59.592808+0000","last_peered":"2024-02-18T00:33:59.592808+0000","last_clean":"2024-02-18T00:33:59.592808+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:33:59.592808+0000","last_undegraded":"2024-02-18T00:33:59.592808+0000","last_fullsized":"2024-02-18T00:33:59.592808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.656344+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":138,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599826+0000","last_change":"2024-02-18T00:33:51.202714+0000","last_active":"2024-02-18T00:33:59.599826+0000","last_peered":"2024-02-18T00:33:59.599826+0000","last_clean":"2024-02-18T00:33:59.599826+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:33:59.599826+0000","last_undegraded":"2024-02-18T00:33:59.599826+0000","last_fullsized":"2024-02-18T00:33:59.599826+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.202648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.202648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590390+0000","last_change":"2024-02-18T00:32:42.033835+0000","last_active":"2024-02-18T00:33:59.590390+0000","last_peered":"2024-02-18T00:33:59.590390+0000","last_clean":"2024-02-18T00:33:59.590390+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:33:59.590390+0000","last_undegraded":"2024-02-18T00:33:59.590390+0000","last_fullsized":"2024-02-18T00:33:59.590390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:42.033759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:42.033759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.15","version":"108'4","reported_seq":87,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601094+0000","last_change":"2024-02-18T00:33:49.190447+0000","last_active":"2024-02-18T00:33:59.601094+0000","last_peered":"2024-02-18T00:33:59.601094+0000","last_clean":"2024-02-18T00:33:59.601094+0000","last_became_active":"2024-02-18T00:33:48.734971+0000","last_became_peered":"2024-02-18T00:33:48.734971+0000","last_unstale":"2024-02-18T00:33:59.601094+0000","last_undegraded":"2024-02-18T00:33:59.601094+0000","last_fullsized":"2024-02-18T00:33:59.601094+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":25165828,"num_keys_recovered":1000,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"109'7","reported_seq":19,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.641180+0000","last_change":"2024-02-18T00:33:59.640750+0000","last_active":"2024-02-18T00:33:59.641180+0000","last_peered":"2024-02-18T00:33:59.641180+0000","last_clean":"2024-02-18T00:33:59.641180+0000","last_became_active":"2024-02-18T00:33:59.623654+0000","last_became_peered":"2024-02-18T00:33:59.623654+0000","last_unstale":"2024-02-18T00:33:59.641180+0000","last_undegraded":"2024-02-18T00:33:59.641180+0000","last_fullsized":"2024-02-18T00:33:59.641180+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600392+0000","last_change":"2024-02-18T00:33:04.650781+0000","last_active":"2024-02-18T00:33:59.600392+0000","last_peered":"2024-02-18T00:33:59.600392+0000","last_clean":"2024-02-18T00:33:59.600392+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:33:59.600392+0000","last_undegraded":"2024-02-18T00:33:59.600392+0000","last_fullsized":"2024-02-18T00:33:59.600392+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591973+0000","last_change":"2024-02-18T00:32:37.227614+0000","last_active":"2024-02-18T00:33:59.591973+0000","last_peered":"2024-02-18T00:33:59.591973+0000","last_clean":"2024-02-18T00:33:59.591973+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:33:59.591973+0000","last_undegraded":"2024-02-18T00:33:59.591973+0000","last_fullsized":"2024-02-18T00:33:59.591973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:37.227536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:37.227536+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":201,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.610517+0000","last_change":"2024-02-18T00:33:49.663562+0000","last_active":"2024-02-18T00:33:59.610517+0000","last_peered":"2024-02-18T00:33:59.610517+0000","last_clean":"2024-02-18T00:33:59.610517+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:33:59.610517+0000","last_undegraded":"2024-02-18T00:33:59.610517+0000","last_fullsized":"2024-02-18T00:33:59.610517+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:32:55.959041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.959041+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":32,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.612010+0000","last_change":"2024-02-18T00:33:48.734242+0000","last_active":"2024-02-18T00:33:59.612010+0000","last_peered":"2024-02-18T00:33:59.612010+0000","last_clean":"2024-02-18T00:33:59.612010+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:33:59.612010+0000","last_undegraded":"2024-02-18T00:33:59.612010+0000","last_fullsized":"2024-02-18T00:33:59.612010+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:46.807518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.807518+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":436,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594574+0000","last_change":"2024-02-18T00:32:44.751653+0000","last_active":"2024-02-18T00:33:59.594574+0000","last_peered":"2024-02-18T00:33:59.594574+0000","last_clean":"2024-02-18T00:33:59.594574+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:33:59.594574+0000","last_undegraded":"2024-02-18T00:33:59.594574+0000","last_fullsized":"2024-02-18T00:33:59.594574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:44.751537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.751537+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":66,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.602183+0000","last_change":"2024-02-18T00:33:01.486028+0000","last_active":"2024-02-18T00:33:59.602183+0000","last_peered":"2024-02-18T00:33:59.602183+0000","last_clean":"2024-02-18T00:33:59.602183+0000","last_became_active":"2024-02-18T00:33:01.485748+0000","last_became_peered":"2024-02-18T00:33:01.485748+0000","last_unstale":"2024-02-18T00:33:59.602183+0000","last_undegraded":"2024-02-18T00:33:59.602183+0000","last_fullsized":"2024-02-18T00:33:59.602183+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.18","version":"109'2","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.621545+0000","last_change":"2024-02-18T00:33:59.620670+0000","last_active":"2024-02-18T00:33:59.621545+0000","last_peered":"2024-02-18T00:33:59.621545+0000","last_clean":"2024-02-18T00:33:59.621545+0000","last_became_active":"2024-02-18T00:33:59.614401+0000","last_became_peered":"2024-02-18T00:33:59.614401+0000","last_unstale":"2024-02-18T00:33:59.621545+0000","last_undegraded":"2024-02-18T00:33:59.621545+0000","last_fullsized":"2024-02-18T00:33:59.621545+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595435+0000","last_change":"2024-02-18T00:33:04.587776+0000","last_active":"2024-02-18T00:33:59.595435+0000","last_peered":"2024-02-18T00:33:59.595435+0000","last_clean":"2024-02-18T00:33:59.595435+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:33:59.595435+0000","last_undegraded":"2024-02-18T00:33:59.595435+0000","last_fullsized":"2024-02-18T00:33:59.595435+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.603545+0000","last_change":"2024-02-18T00:32:43.913604+0000","last_active":"2024-02-18T00:33:59.603545+0000","last_peered":"2024-02-18T00:33:59.603545+0000","last_clean":"2024-02-18T00:33:59.603545+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:33:59.603545+0000","last_undegraded":"2024-02-18T00:33:59.603545+0000","last_fullsized":"2024-02-18T00:33:59.603545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.913522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.913522+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":139,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.611975+0000","last_change":"2024-02-18T00:32:45.002522+0000","last_active":"2024-02-18T00:33:59.611975+0000","last_peered":"2024-02-18T00:33:59.611975+0000","last_clean":"2024-02-18T00:33:59.611975+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:33:59.611975+0000","last_undegraded":"2024-02-18T00:33:59.611975+0000","last_fullsized":"2024-02-18T00:33:59.611975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:32:45.002406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.002406+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.607353+0000","last_change":"2024-02-18T00:33:48.735745+0000","last_active":"2024-02-18T00:33:59.607353+0000","last_peered":"2024-02-18T00:33:59.607353+0000","last_clean":"2024-02-18T00:33:59.607353+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:33:59.607353+0000","last_undegraded":"2024-02-18T00:33:59.607353+0000","last_fullsized":"2024-02-18T00:33:59.607353+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.843892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.843892+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":122,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.604556+0000","last_change":"2024-02-18T00:31:53.976449+0000","last_active":"2024-02-18T00:33:59.604556+0000","last_peered":"2024-02-18T00:33:59.604556+0000","last_clean":"2024-02-18T00:33:59.604556+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:33:59.604556+0000","last_undegraded":"2024-02-18T00:33:59.604556+0000","last_fullsized":"2024-02-18T00:33:59.604556+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:53.976377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:31:53.976377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599265+0000","last_change":"2024-02-18T00:32:53.813004+0000","last_active":"2024-02-18T00:33:59.599265+0000","last_peered":"2024-02-18T00:33:59.599265+0000","last_clean":"2024-02-18T00:33:59.599265+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:33:59.599265+0000","last_undegraded":"2024-02-18T00:33:59.599265+0000","last_fullsized":"2024-02-18T00:33:59.599265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.812897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.812897+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":75,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595361+0000","last_change":"2024-02-18T00:33:48.748699+0000","last_active":"2024-02-18T00:33:59.595361+0000","last_peered":"2024-02-18T00:33:59.595361+0000","last_clean":"2024-02-18T00:33:59.595361+0000","last_became_active":"2024-02-18T00:33:48.742690+0000","last_became_peered":"2024-02-18T00:33:48.742690+0000","last_unstale":"2024-02-18T00:33:59.595361+0000","last_undegraded":"2024-02-18T00:33:59.595361+0000","last_fullsized":"2024-02-18T00:33:59.595361+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1b","version":"109'10","reported_seq":19,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.507279+0000","last_change":"2024-02-18T00:34:00.506923+0000","last_active":"2024-02-18T00:34:00.507279+0000","last_peered":"2024-02-18T00:34:00.507279+0000","last_clean":"2024-02-18T00:34:00.507279+0000","last_became_active":"2024-02-18T00:33:59.628124+0000","last_became_peered":"2024-02-18T00:33:59.628124+0000","last_unstale":"2024-02-18T00:34:00.507279+0000","last_undegraded":"2024-02-18T00:34:00.507279+0000","last_fullsized":"2024-02-18T00:34:00.507279+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591438+0000","last_change":"2024-02-18T00:33:48.731380+0000","last_active":"2024-02-18T00:33:59.591438+0000","last_peered":"2024-02-18T00:33:59.591438+0000","last_clean":"2024-02-18T00:33:59.591438+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:33:59.591438+0000","last_undegraded":"2024-02-18T00:33:59.591438+0000","last_fullsized":"2024-02-18T00:33:59.591438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594774+0000","last_change":"2024-02-18T00:33:48.739564+0000","last_active":"2024-02-18T00:33:59.594774+0000","last_peered":"2024-02-18T00:33:59.594774+0000","last_clean":"2024-02-18T00:33:59.594774+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:33:59.594774+0000","last_undegraded":"2024-02-18T00:33:59.594774+0000","last_fullsized":"2024-02-18T00:33:59.594774+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.644921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.644921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":174,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593881+0000","last_change":"2024-02-18T00:33:49.043325+0000","last_active":"2024-02-18T00:33:59.593881+0000","last_peered":"2024-02-18T00:33:59.593881+0000","last_clean":"2024-02-18T00:33:59.593881+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:33:59.593881+0000","last_undegraded":"2024-02-18T00:33:59.593881+0000","last_fullsized":"2024-02-18T00:33:59.593881+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:32:44.353571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.353571+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":58,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591361+0000","last_change":"2024-02-18T00:33:48.731235+0000","last_active":"2024-02-18T00:33:59.591361+0000","last_peered":"2024-02-18T00:33:59.591361+0000","last_clean":"2024-02-18T00:33:59.591361+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:33:59.591361+0000","last_undegraded":"2024-02-18T00:33:59.591361+0000","last_fullsized":"2024-02-18T00:33:59.591361+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.632911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.632911+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":142,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600919+0000","last_change":"2024-02-18T00:33:48.670701+0000","last_active":"2024-02-18T00:33:59.600919+0000","last_peered":"2024-02-18T00:33:59.600919+0000","last_clean":"2024-02-18T00:33:59.600919+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:33:59.600919+0000","last_undegraded":"2024-02-18T00:33:59.600919+0000","last_fullsized":"2024-02-18T00:33:59.600919+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:42.847815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:31:42.847815+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":123,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593994+0000","last_change":"2024-02-18T00:33:02.549194+0000","last_active":"2024-02-18T00:33:59.593994+0000","last_peered":"2024-02-18T00:33:59.593994+0000","last_clean":"2024-02-18T00:33:59.593994+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:33:59.593994+0000","last_undegraded":"2024-02-18T00:33:59.593994+0000","last_fullsized":"2024-02-18T00:33:59.593994+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:02.549081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.549081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.603607+0000","last_change":"2024-02-18T00:31:46.851657+0000","last_active":"2024-02-18T00:33:59.603607+0000","last_peered":"2024-02-18T00:33:59.603607+0000","last_clean":"2024-02-18T00:33:59.603607+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:33:59.603607+0000","last_undegraded":"2024-02-18T00:33:59.603607+0000","last_fullsized":"2024-02-18T00:33:59.603607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:31:46.851573+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"108'3","reported_seq":86,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594625+0000","last_change":"2024-02-18T00:33:49.053068+0000","last_active":"2024-02-18T00:33:59.594625+0000","last_peered":"2024-02-18T00:33:59.594625+0000","last_clean":"2024-02-18T00:33:59.594625+0000","last_became_active":"2024-02-18T00:33:48.734090+0000","last_became_peered":"2024-02-18T00:33:48.734090+0000","last_unstale":"2024-02-18T00:33:59.594625+0000","last_undegraded":"2024-02-18T00:33:59.594625+0000","last_fullsized":"2024-02-18T00:33:59.594625+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":15,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":15,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.637412+0000","last_change":"2024-02-18T00:33:59.637128+0000","last_active":"2024-02-18T00:33:59.637412+0000","last_peered":"2024-02-18T00:33:59.637412+0000","last_clean":"2024-02-18T00:33:59.637412+0000","last_became_active":"2024-02-18T00:33:59.636810+0000","last_became_peered":"2024-02-18T00:33:59.636810+0000","last_unstale":"2024-02-18T00:33:59.637412+0000","last_undegraded":"2024-02-18T00:33:59.637412+0000","last_fullsized":"2024-02-18T00:33:59.637412+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.618901+0000","last_change":"2024-02-18T00:33:04.649428+0000","last_active":"2024-02-18T00:33:59.618901+0000","last_peered":"2024-02-18T00:33:59.618901+0000","last_clean":"2024-02-18T00:33:59.618901+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:33:59.618901+0000","last_undegraded":"2024-02-18T00:33:59.618901+0000","last_fullsized":"2024-02-18T00:33:59.618901+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":134,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596332+0000","last_change":"2024-02-18T00:33:49.897870+0000","last_active":"2024-02-18T00:33:59.596332+0000","last_peered":"2024-02-18T00:33:59.596332+0000","last_clean":"2024-02-18T00:33:59.596332+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:33:59.596332+0000","last_undegraded":"2024-02-18T00:33:59.596332+0000","last_fullsized":"2024-02-18T00:33:59.596332+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.897812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.897812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":164,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600352+0000","last_change":"2024-02-18T00:31:55.005638+0000","last_active":"2024-02-18T00:33:59.600352+0000","last_peered":"2024-02-18T00:33:59.600352+0000","last_clean":"2024-02-18T00:33:59.600352+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:33:59.600352+0000","last_undegraded":"2024-02-18T00:33:59.600352+0000","last_fullsized":"2024-02-18T00:33:59.600352+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:31:55.005563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:55.005563+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590783+0000","last_change":"2024-02-18T00:33:37.809035+0000","last_active":"2024-02-18T00:33:59.590783+0000","last_peered":"2024-02-18T00:33:59.590783+0000","last_clean":"2024-02-18T00:33:59.590783+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:33:59.590783+0000","last_undegraded":"2024-02-18T00:33:59.590783+0000","last_fullsized":"2024-02-18T00:33:59.590783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:37.808923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.808923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":409,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:34:00.421003+0000","last_change":"2024-02-18T00:33:41.154008+0000","last_active":"2024-02-18T00:34:00.421003+0000","last_peered":"2024-02-18T00:34:00.421003+0000","last_clean":"2024-02-18T00:34:00.421003+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:34:00.421003+0000","last_undegraded":"2024-02-18T00:34:00.421003+0000","last_fullsized":"2024-02-18T00:34:00.421003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:33:41.153868+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.153868+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":277,"num_read_kb":272,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":61,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594181+0000","last_change":"2024-02-18T00:33:48.840999+0000","last_active":"2024-02-18T00:33:59.594181+0000","last_peered":"2024-02-18T00:33:59.594181+0000","last_clean":"2024-02-18T00:33:59.594181+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:33:59.594181+0000","last_undegraded":"2024-02-18T00:33:59.594181+0000","last_fullsized":"2024-02-18T00:33:59.594181+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.588750+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":66,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594353+0000","last_change":"2024-02-18T00:33:01.553808+0000","last_active":"2024-02-18T00:33:59.594353+0000","last_peered":"2024-02-18T00:33:59.594353+0000","last_clean":"2024-02-18T00:33:59.594353+0000","last_became_active":"2024-02-18T00:33:01.551635+0000","last_became_peered":"2024-02-18T00:33:01.551635+0000","last_unstale":"2024-02-18T00:33:59.594353+0000","last_undegraded":"2024-02-18T00:33:59.594353+0000","last_fullsized":"2024-02-18T00:33:59.594353+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1a","version":"109'5","reported_seq":18,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.820160+0000","last_change":"2024-02-18T00:33:59.819858+0000","last_active":"2024-02-18T00:33:59.820160+0000","last_peered":"2024-02-18T00:33:59.820160+0000","last_clean":"2024-02-18T00:33:59.820160+0000","last_became_active":"2024-02-18T00:33:59.635283+0000","last_became_peered":"2024-02-18T00:33:59.635283+0000","last_unstale":"2024-02-18T00:33:59.820160+0000","last_undegraded":"2024-02-18T00:33:59.820160+0000","last_fullsized":"2024-02-18T00:33:59.820160+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591990+0000","last_change":"2024-02-18T00:33:48.670413+0000","last_active":"2024-02-18T00:33:59.591990+0000","last_peered":"2024-02-18T00:33:59.591990+0000","last_clean":"2024-02-18T00:33:59.591990+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:33:59.591990+0000","last_undegraded":"2024-02-18T00:33:59.591990+0000","last_fullsized":"2024-02-18T00:33:59.591990+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":119,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.612570+0000","last_change":"2024-02-18T00:32:07.008657+0000","last_active":"2024-02-18T00:33:59.612570+0000","last_peered":"2024-02-18T00:33:59.612570+0000","last_clean":"2024-02-18T00:33:59.612570+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:33:59.612570+0000","last_undegraded":"2024-02-18T00:33:59.612570+0000","last_fullsized":"2024-02-18T00:33:59.612570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.008543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.008543+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":247,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.598911+0000","last_change":"2024-02-18T00:33:54.198053+0000","last_active":"2024-02-18T00:33:59.598911+0000","last_peered":"2024-02-18T00:33:59.598911+0000","last_clean":"2024-02-18T00:33:59.598911+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:33:59.598911+0000","last_undegraded":"2024-02-18T00:33:59.598911+0000","last_fullsized":"2024-02-18T00:33:59.598911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:33:54.197976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.197976+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":133,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601376+0000","last_change":"2024-02-18T00:33:04.569398+0000","last_active":"2024-02-18T00:33:59.601376+0000","last_peered":"2024-02-18T00:33:59.601376+0000","last_clean":"2024-02-18T00:33:59.601376+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:33:59.601376+0000","last_undegraded":"2024-02-18T00:33:59.601376+0000","last_fullsized":"2024-02-18T00:33:59.601376+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.991759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.991759+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.598377+0000","last_change":"2024-02-18T00:33:48.688410+0000","last_active":"2024-02-18T00:33:59.598377+0000","last_peered":"2024-02-18T00:33:59.598377+0000","last_clean":"2024-02-18T00:33:59.598377+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:33:59.598377+0000","last_undegraded":"2024-02-18T00:33:59.598377+0000","last_fullsized":"2024-02-18T00:33:59.598377+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:43.926301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.926301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.11","version":"108'3","reported_seq":77,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594591+0000","last_change":"2024-02-18T00:33:01.502827+0000","last_active":"2024-02-18T00:33:59.594591+0000","last_peered":"2024-02-18T00:33:59.594591+0000","last_clean":"2024-02-18T00:33:59.594591+0000","last_became_active":"2024-02-18T00:33:01.502464+0000","last_became_peered":"2024-02-18T00:33:01.502464+0000","last_unstale":"2024-02-18T00:33:59.594591+0000","last_undegraded":"2024-02-18T00:33:59.594591+0000","last_fullsized":"2024-02-18T00:33:59.594591+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165843,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":24578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.634632+0000","last_change":"2024-02-18T00:33:59.628645+0000","last_active":"2024-02-18T00:33:59.634632+0000","last_peered":"2024-02-18T00:33:59.634632+0000","last_clean":"2024-02-18T00:33:59.634632+0000","last_became_active":"2024-02-18T00:33:59.628391+0000","last_became_peered":"2024-02-18T00:33:59.628391+0000","last_unstale":"2024-02-18T00:33:59.634632+0000","last_undegraded":"2024-02-18T00:33:59.634632+0000","last_fullsized":"2024-02-18T00:33:59.634632+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.607721+0000","last_change":"2024-02-18T00:33:04.719631+0000","last_active":"2024-02-18T00:33:59.607721+0000","last_peered":"2024-02-18T00:33:59.607721+0000","last_clean":"2024-02-18T00:33:59.607721+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:33:59.607721+0000","last_undegraded":"2024-02-18T00:33:59.607721+0000","last_fullsized":"2024-02-18T00:33:59.607721+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":184,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.610774+0000","last_change":"2024-02-18T00:32:05.026757+0000","last_active":"2024-02-18T00:33:59.610774+0000","last_peered":"2024-02-18T00:33:59.610774+0000","last_clean":"2024-02-18T00:33:59.610774+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:33:59.610774+0000","last_undegraded":"2024-02-18T00:33:59.610774+0000","last_fullsized":"2024-02-18T00:33:59.610774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:05.026652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.026652+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":122,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596976+0000","last_change":"2024-02-18T00:32:49.398933+0000","last_active":"2024-02-18T00:33:59.596976+0000","last_peered":"2024-02-18T00:33:59.596976+0000","last_clean":"2024-02-18T00:33:59.596976+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:33:59.596976+0000","last_undegraded":"2024-02-18T00:33:59.596976+0000","last_fullsized":"2024-02-18T00:33:59.596976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.398867+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":147,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599048+0000","last_change":"2024-02-18T00:33:48.759454+0000","last_active":"2024-02-18T00:33:59.599048+0000","last_peered":"2024-02-18T00:33:59.599048+0000","last_clean":"2024-02-18T00:33:59.599048+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:33:59.599048+0000","last_undegraded":"2024-02-18T00:33:59.599048+0000","last_fullsized":"2024-02-18T00:33:59.599048+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:54.839018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:54.839018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":117,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590963+0000","last_change":"2024-02-18T00:33:48.010937+0000","last_active":"2024-02-18T00:33:59.590963+0000","last_peered":"2024-02-18T00:33:59.590963+0000","last_clean":"2024-02-18T00:33:59.590963+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:33:59.590963+0000","last_undegraded":"2024-02-18T00:33:59.590963+0000","last_fullsized":"2024-02-18T00:33:59.590963+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:48.010821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:48.010821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":66,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595736+0000","last_change":"2024-02-18T00:33:01.507106+0000","last_active":"2024-02-18T00:33:59.595736+0000","last_peered":"2024-02-18T00:33:59.595736+0000","last_clean":"2024-02-18T00:33:59.595736+0000","last_became_active":"2024-02-18T00:33:01.506457+0000","last_became_peered":"2024-02-18T00:33:01.506457+0000","last_unstale":"2024-02-18T00:33:59.595736+0000","last_undegraded":"2024-02-18T00:33:59.595736+0000","last_fullsized":"2024-02-18T00:33:59.595736+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.17","version":"109'5","reported_seq":15,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.966091+0000","last_change":"2024-02-18T00:33:59.965802+0000","last_active":"2024-02-18T00:33:59.966091+0000","last_peered":"2024-02-18T00:33:59.966091+0000","last_clean":"2024-02-18T00:33:59.966091+0000","last_became_active":"2024-02-18T00:33:59.604379+0000","last_became_peered":"2024-02-18T00:33:59.604379+0000","last_unstale":"2024-02-18T00:33:59.966091+0000","last_undegraded":"2024-02-18T00:33:59.966091+0000","last_fullsized":"2024-02-18T00:33:59.966091+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590778+0000","last_change":"2024-02-18T00:33:02.483549+0000","last_active":"2024-02-18T00:33:59.590778+0000","last_peered":"2024-02-18T00:33:59.590778+0000","last_clean":"2024-02-18T00:33:59.590778+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:33:59.590778+0000","last_undegraded":"2024-02-18T00:33:59.590778+0000","last_fullsized":"2024-02-18T00:33:59.590778+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597669+0000","last_change":"2024-02-18T00:32:40.476946+0000","last_active":"2024-02-18T00:33:59.597669+0000","last_peered":"2024-02-18T00:33:59.597669+0000","last_clean":"2024-02-18T00:33:59.597669+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:33:59.597669+0000","last_undegraded":"2024-02-18T00:33:59.597669+0000","last_fullsized":"2024-02-18T00:33:59.597669+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:40.476846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.476846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":175,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591650+0000","last_change":"2024-02-18T00:32:43.018921+0000","last_active":"2024-02-18T00:33:59.591650+0000","last_peered":"2024-02-18T00:33:59.591650+0000","last_clean":"2024-02-18T00:33:59.591650+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:33:59.591650+0000","last_undegraded":"2024-02-18T00:33:59.591650+0000","last_fullsized":"2024-02-18T00:33:59.591650+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:32:43.018841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.018841+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":157,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591606+0000","last_change":"2024-02-18T00:33:48.943060+0000","last_active":"2024-02-18T00:33:59.591606+0000","last_peered":"2024-02-18T00:33:59.591606+0000","last_clean":"2024-02-18T00:33:59.591606+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:33:59.591606+0000","last_undegraded":"2024-02-18T00:33:59.591606+0000","last_fullsized":"2024-02-18T00:33:59.591606+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:33:03.047128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.047128+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595175+0000","last_change":"2024-02-18T00:33:48.711292+0000","last_active":"2024-02-18T00:33:59.595175+0000","last_peered":"2024-02-18T00:33:59.595175+0000","last_clean":"2024-02-18T00:33:59.595175+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:33:59.595175+0000","last_undegraded":"2024-02-18T00:33:59.595175+0000","last_fullsized":"2024-02-18T00:33:59.595175+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:52.444811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:52.444811+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1c","version":"108'4","reported_seq":74,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595741+0000","last_change":"2024-02-18T00:33:01.489945+0000","last_active":"2024-02-18T00:33:59.595741+0000","last_peered":"2024-02-18T00:33:59.595741+0000","last_clean":"2024-02-18T00:33:59.595741+0000","last_became_active":"2024-02-18T00:33:01.488576+0000","last_became_peered":"2024-02-18T00:33:01.488576+0000","last_unstale":"2024-02-18T00:33:59.595741+0000","last_undegraded":"2024-02-18T00:33:59.595741+0000","last_fullsized":"2024-02-18T00:33:59.595741+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165835,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"109'2","reported_seq":79,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.604676+0000","last_change":"2024-02-18T00:33:48.849336+0000","last_active":"2024-02-18T00:33:59.604676+0000","last_peered":"2024-02-18T00:33:59.604676+0000","last_clean":"2024-02-18T00:33:59.604676+0000","last_became_active":"2024-02-18T00:33:48.692512+0000","last_became_peered":"2024-02-18T00:33:48.692512+0000","last_unstale":"2024-02-18T00:33:59.604676+0000","last_undegraded":"2024-02-18T00:33:59.604676+0000","last_fullsized":"2024-02-18T00:33:59.604676+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590769+0000","last_change":"2024-02-18T00:33:02.533610+0000","last_active":"2024-02-18T00:33:59.590769+0000","last_peered":"2024-02-18T00:33:59.590769+0000","last_clean":"2024-02-18T00:33:59.590769+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:33:59.590769+0000","last_undegraded":"2024-02-18T00:33:59.590769+0000","last_fullsized":"2024-02-18T00:33:59.590769+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":190,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.612019+0000","last_change":"2024-02-18T00:33:48.958400+0000","last_active":"2024-02-18T00:33:59.612019+0000","last_peered":"2024-02-18T00:33:59.612019+0000","last_clean":"2024-02-18T00:33:59.612019+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:33:59.612019+0000","last_undegraded":"2024-02-18T00:33:59.612019+0000","last_fullsized":"2024-02-18T00:33:59.612019+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:33:03.042870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.042870+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597598+0000","last_change":"2024-02-18T00:32:05.449395+0000","last_active":"2024-02-18T00:33:59.597598+0000","last_peered":"2024-02-18T00:33:59.597598+0000","last_clean":"2024-02-18T00:33:59.597598+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:33:59.597598+0000","last_undegraded":"2024-02-18T00:33:59.597598+0000","last_fullsized":"2024-02-18T00:33:59.597598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:05.449286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:05.449286+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.606927+0000","last_change":"2024-02-18T00:33:48.744464+0000","last_active":"2024-02-18T00:33:59.606927+0000","last_peered":"2024-02-18T00:33:59.606927+0000","last_clean":"2024-02-18T00:33:59.606927+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:33:59.606927+0000","last_undegraded":"2024-02-18T00:33:59.606927+0000","last_fullsized":"2024-02-18T00:33:59.606927+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:59.580519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:59.580519+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594700+0000","last_change":"2024-02-18T00:33:48.738806+0000","last_active":"2024-02-18T00:33:59.594700+0000","last_peered":"2024-02-18T00:33:59.594700+0000","last_clean":"2024-02-18T00:33:59.594700+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:33:59.594700+0000","last_undegraded":"2024-02-18T00:33:59.594700+0000","last_fullsized":"2024-02-18T00:33:59.594700+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:42.462777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:32:42.462777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3","version":"142'2","reported_seq":82,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590682+0000","last_change":"2024-02-18T00:33:41.446503+0000","last_active":"2024-02-18T00:33:59.590682+0000","last_peered":"2024-02-18T00:33:59.590682+0000","last_clean":"2024-02-18T00:33:59.590682+0000","last_became_active":"2024-02-18T00:33:01.438240+0000","last_became_peered":"2024-02-18T00:33:01.438240+0000","last_unstale":"2024-02-18T00:33:59.590682+0000","last_undegraded":"2024-02-18T00:33:59.590682+0000","last_fullsized":"2024-02-18T00:33:59.590682+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'2","last_scrub_stamp":"2024-02-18T00:33:41.446439+0000","last_deep_scrub":"142'2","last_deep_scrub_stamp":"2024-02-18T00:33:41.446439+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.446439+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.605000+0000","last_change":"2024-02-18T00:33:48.728749+0000","last_active":"2024-02-18T00:33:59.605000+0000","last_peered":"2024-02-18T00:33:59.605000+0000","last_clean":"2024-02-18T00:33:59.605000+0000","last_became_active":"2024-02-18T00:33:48.728292+0000","last_became_peered":"2024-02-18T00:33:48.728292+0000","last_unstale":"2024-02-18T00:33:59.605000+0000","last_undegraded":"2024-02-18T00:33:59.605000+0000","last_fullsized":"2024-02-18T00:33:59.605000+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.618596+0000","last_change":"2024-02-18T00:33:04.706011+0000","last_active":"2024-02-18T00:33:59.618596+0000","last_peered":"2024-02-18T00:33:59.618596+0000","last_clean":"2024-02-18T00:33:59.618596+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:33:59.618596+0000","last_undegraded":"2024-02-18T00:33:59.618596+0000","last_fullsized":"2024-02-18T00:33:59.618596+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":189,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.606833+0000","last_change":"2024-02-18T00:33:48.861148+0000","last_active":"2024-02-18T00:33:59.606833+0000","last_peered":"2024-02-18T00:33:59.606833+0000","last_clean":"2024-02-18T00:33:59.606833+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:33:59.606833+0000","last_undegraded":"2024-02-18T00:33:59.606833+0000","last_fullsized":"2024-02-18T00:33:59.606833+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:33:46.897445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.897445+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596706+0000","last_change":"2024-02-18T00:32:58.392505+0000","last_active":"2024-02-18T00:33:59.596706+0000","last_peered":"2024-02-18T00:33:59.596706+0000","last_clean":"2024-02-18T00:33:59.596706+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:33:59.596706+0000","last_undegraded":"2024-02-18T00:33:59.596706+0000","last_fullsized":"2024-02-18T00:33:59.596706+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.392443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.392443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601914+0000","last_change":"2024-02-18T00:32:50.948349+0000","last_active":"2024-02-18T00:33:59.601914+0000","last_peered":"2024-02-18T00:33:59.601914+0000","last_clean":"2024-02-18T00:33:59.601914+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:33:59.601914+0000","last_undegraded":"2024-02-18T00:33:59.601914+0000","last_fullsized":"2024-02-18T00:33:59.601914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:50.948240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.948240+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":29,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594230+0000","last_change":"2024-02-18T00:33:55.265441+0000","last_active":"2024-02-18T00:33:59.594230+0000","last_peered":"2024-02-18T00:33:59.594230+0000","last_clean":"2024-02-18T00:33:59.594230+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:33:59.594230+0000","last_undegraded":"2024-02-18T00:33:59.594230+0000","last_fullsized":"2024-02-18T00:33:59.594230+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.265373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.265373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2","version":"138'11","reported_seq":89,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589775+0000","last_change":"2024-02-18T00:33:37.216038+0000","last_active":"2024-02-18T00:33:59.589775+0000","last_peered":"2024-02-18T00:33:59.589775+0000","last_clean":"2024-02-18T00:33:59.589775+0000","last_became_active":"2024-02-18T00:33:01.499199+0000","last_became_peered":"2024-02-18T00:33:01.499199+0000","last_unstale":"2024-02-18T00:33:59.589775+0000","last_undegraded":"2024-02-18T00:33:59.589775+0000","last_fullsized":"2024-02-18T00:33:59.589775+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'11","last_scrub_stamp":"2024-02-18T00:33:37.215954+0000","last_deep_scrub":"138'11","last_deep_scrub_stamp":"2024-02-18T00:33:37.215954+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.215954+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":3888,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.a","version":"109'10","reported_seq":74,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.618818+0000","last_change":"2024-02-18T00:33:04.707926+0000","last_active":"2024-02-18T00:33:59.618818+0000","last_peered":"2024-02-18T00:33:59.618818+0000","last_clean":"2024-02-18T00:33:59.618818+0000","last_became_active":"2024-02-18T00:33:04.701946+0000","last_became_peered":"2024-02-18T00:33:04.701946+0000","last_unstale":"2024-02-18T00:33:59.618818+0000","last_undegraded":"2024-02-18T00:33:59.618818+0000","last_fullsized":"2024-02-18T00:33:59.618818+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593623+0000","last_change":"2024-02-18T00:33:04.730258+0000","last_active":"2024-02-18T00:33:59.593623+0000","last_peered":"2024-02-18T00:33:59.593623+0000","last_clean":"2024-02-18T00:33:59.593623+0000","last_became_active":"2024-02-18T00:33:04.729050+0000","last_became_peered":"2024-02-18T00:33:04.729050+0000","last_unstale":"2024-02-18T00:33:59.593623+0000","last_undegraded":"2024-02-18T00:33:59.593623+0000","last_fullsized":"2024-02-18T00:33:59.593623+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":192,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.598066+0000","last_change":"2024-02-18T00:33:48.884823+0000","last_active":"2024-02-18T00:33:59.598066+0000","last_peered":"2024-02-18T00:33:59.598066+0000","last_clean":"2024-02-18T00:33:59.598066+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:33:59.598066+0000","last_undegraded":"2024-02-18T00:33:59.598066+0000","last_fullsized":"2024-02-18T00:33:59.598066+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:32:02.547447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.547447+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":142,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601415+0000","last_change":"2024-02-18T00:33:48.735156+0000","last_active":"2024-02-18T00:33:59.601415+0000","last_peered":"2024-02-18T00:33:59.601415+0000","last_clean":"2024-02-18T00:33:59.601415+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:33:59.601415+0000","last_undegraded":"2024-02-18T00:33:59.601415+0000","last_fullsized":"2024-02-18T00:33:59.601415+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:55.976578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:55.976578+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":185,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601325+0000","last_change":"2024-02-18T00:32:36.948567+0000","last_active":"2024-02-18T00:33:59.601325+0000","last_peered":"2024-02-18T00:33:59.601325+0000","last_clean":"2024-02-18T00:33:59.601325+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:33:59.601325+0000","last_undegraded":"2024-02-18T00:33:59.601325+0000","last_fullsized":"2024-02-18T00:33:59.601325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:36.948476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.948476+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.611887+0000","last_change":"2024-02-18T00:33:53.101420+0000","last_active":"2024-02-18T00:33:59.611887+0000","last_peered":"2024-02-18T00:33:59.611887+0000","last_clean":"2024-02-18T00:33:59.611887+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:33:59.611887+0000","last_undegraded":"2024-02-18T00:33:59.611887+0000","last_fullsized":"2024-02-18T00:33:59.611887+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.101251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.101251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"139'4","reported_seq":81,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.603993+0000","last_change":"2024-02-18T00:33:37.962396+0000","last_active":"2024-02-18T00:33:59.603993+0000","last_peered":"2024-02-18T00:33:59.603993+0000","last_clean":"2024-02-18T00:33:59.603993+0000","last_became_active":"2024-02-18T00:33:01.577813+0000","last_became_peered":"2024-02-18T00:33:01.577813+0000","last_unstale":"2024-02-18T00:33:59.603993+0000","last_undegraded":"2024-02-18T00:33:59.603993+0000","last_fullsized":"2024-02-18T00:33:59.603993+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-02-18T00:33:37.962280+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-02-18T00:33:37.962280+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.962280+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"109'2","reported_seq":17,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.720529+0000","last_change":"2024-02-18T00:33:59.720068+0000","last_active":"2024-02-18T00:33:59.720529+0000","last_peered":"2024-02-18T00:33:59.720529+0000","last_clean":"2024-02-18T00:33:59.720529+0000","last_became_active":"2024-02-18T00:33:59.627607+0000","last_became_peered":"2024-02-18T00:33:59.627607+0000","last_unstale":"2024-02-18T00:33:59.720529+0000","last_undegraded":"2024-02-18T00:33:59.720529+0000","last_fullsized":"2024-02-18T00:33:59.720529+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593756+0000","last_change":"2024-02-18T00:33:48.730891+0000","last_active":"2024-02-18T00:33:59.593756+0000","last_peered":"2024-02-18T00:33:59.593756+0000","last_clean":"2024-02-18T00:33:59.593756+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:33:59.593756+0000","last_undegraded":"2024-02-18T00:33:59.593756+0000","last_fullsized":"2024-02-18T00:33:59.593756+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":174,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601219+0000","last_change":"2024-02-18T00:33:49.276437+0000","last_active":"2024-02-18T00:33:59.601219+0000","last_peered":"2024-02-18T00:33:59.601219+0000","last_clean":"2024-02-18T00:33:59.601219+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:33:59.601219+0000","last_undegraded":"2024-02-18T00:33:59.601219+0000","last_fullsized":"2024-02-18T00:33:59.601219+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:33:40.095425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.095425+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":123,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.610597+0000","last_change":"2024-02-18T00:31:41.112608+0000","last_active":"2024-02-18T00:33:59.610597+0000","last_peered":"2024-02-18T00:33:59.610597+0000","last_clean":"2024-02-18T00:33:59.610597+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:33:59.610597+0000","last_undegraded":"2024-02-18T00:33:59.610597+0000","last_fullsized":"2024-02-18T00:33:59.610597+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:41.112540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:31:41.112540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"150'9","reported_seq":198,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.598319+0000","last_change":"2024-02-18T00:33:48.839748+0000","last_active":"2024-02-18T00:33:59.598319+0000","last_peered":"2024-02-18T00:33:59.598319+0000","last_clean":"2024-02-18T00:33:59.598319+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:33:59.598319+0000","last_undegraded":"2024-02-18T00:33:59.598319+0000","last_fullsized":"2024-02-18T00:33:59.598319+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:01.547763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:01.547763+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591889+0000","last_change":"2024-02-18T00:32:53.114523+0000","last_active":"2024-02-18T00:33:59.591889+0000","last_peered":"2024-02-18T00:33:59.591889+0000","last_clean":"2024-02-18T00:33:59.591889+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:33:59.591889+0000","last_undegraded":"2024-02-18T00:33:59.591889+0000","last_fullsized":"2024-02-18T00:33:59.591889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.114424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.114424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"136'2","reported_seq":75,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593783+0000","last_change":"2024-02-18T00:33:48.720462+0000","last_active":"2024-02-18T00:33:59.593783+0000","last_peered":"2024-02-18T00:33:59.593783+0000","last_clean":"2024-02-18T00:33:59.593783+0000","last_became_active":"2024-02-18T00:33:48.684637+0000","last_became_peered":"2024-02-18T00:33:48.684637+0000","last_unstale":"2024-02-18T00:33:59.593783+0000","last_undegraded":"2024-02-18T00:33:59.593783+0000","last_fullsized":"2024-02-18T00:33:59.593783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'2","last_scrub_stamp":"2024-02-18T00:33:34.857262+0000","last_deep_scrub":"136'2","last_deep_scrub_stamp":"2024-02-18T00:33:34.857262+0000","last_clean_scrub_stamp":"2024-02-18T00:33:34.857262+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":7,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.634584+0000","last_change":"2024-02-18T00:33:59.626045+0000","last_active":"2024-02-18T00:33:59.634584+0000","last_peered":"2024-02-18T00:33:59.634584+0000","last_clean":"2024-02-18T00:33:59.634584+0000","last_became_active":"2024-02-18T00:33:59.623162+0000","last_became_peered":"2024-02-18T00:33:59.623162+0000","last_unstale":"2024-02-18T00:33:59.634584+0000","last_undegraded":"2024-02-18T00:33:59.634584+0000","last_fullsized":"2024-02-18T00:33:59.634584+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.603824+0000","last_change":"2024-02-18T00:33:02.532649+0000","last_active":"2024-02-18T00:33:59.603824+0000","last_peered":"2024-02-18T00:33:59.603824+0000","last_clean":"2024-02-18T00:33:59.603824+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:33:59.603824+0000","last_undegraded":"2024-02-18T00:33:59.603824+0000","last_fullsized":"2024-02-18T00:33:59.603824+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":138,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595071+0000","last_change":"2024-02-18T00:32:37.815524+0000","last_active":"2024-02-18T00:33:59.595071+0000","last_peered":"2024-02-18T00:33:59.595071+0000","last_clean":"2024-02-18T00:33:59.595071+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:33:59.595071+0000","last_undegraded":"2024-02-18T00:33:59.595071+0000","last_fullsized":"2024-02-18T00:33:59.595071+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:32:37.815411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:37.815411+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593399+0000","last_change":"2024-02-18T00:32:52.323010+0000","last_active":"2024-02-18T00:33:59.593399+0000","last_peered":"2024-02-18T00:33:59.593399+0000","last_clean":"2024-02-18T00:33:59.593399+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:33:59.593399+0000","last_undegraded":"2024-02-18T00:33:59.593399+0000","last_fullsized":"2024-02-18T00:33:59.593399+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:52.322930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:52.322930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":125,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595098+0000","last_change":"2024-02-18T00:33:50.765435+0000","last_active":"2024-02-18T00:33:59.595098+0000","last_peered":"2024-02-18T00:33:59.595098+0000","last_clean":"2024-02-18T00:33:59.595098+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:33:59.595098+0000","last_undegraded":"2024-02-18T00:33:59.595098+0000","last_fullsized":"2024-02-18T00:33:59.595098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.765368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.765368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":117,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590887+0000","last_change":"2024-02-18T00:33:01.075249+0000","last_active":"2024-02-18T00:33:59.590887+0000","last_peered":"2024-02-18T00:33:59.590887+0000","last_clean":"2024-02-18T00:33:59.590887+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:33:59.590887+0000","last_undegraded":"2024-02-18T00:33:59.590887+0000","last_fullsized":"2024-02-18T00:33:59.590887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.075169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.075169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":85,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.607348+0000","last_change":"2024-02-18T00:33:48.741294+0000","last_active":"2024-02-18T00:33:59.607348+0000","last_peered":"2024-02-18T00:33:59.607348+0000","last_clean":"2024-02-18T00:33:59.607348+0000","last_became_active":"2024-02-18T00:33:48.736024+0000","last_became_peered":"2024-02-18T00:33:48.736024+0000","last_unstale":"2024-02-18T00:33:59.607348+0000","last_undegraded":"2024-02-18T00:33:59.607348+0000","last_fullsized":"2024-02-18T00:33:59.607348+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.951370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:40.951370+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.951370+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"109'5","reported_seq":16,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.645388+0000","last_change":"2024-02-18T00:33:59.644771+0000","last_active":"2024-02-18T00:33:59.645388+0000","last_peered":"2024-02-18T00:33:59.645388+0000","last_clean":"2024-02-18T00:33:59.645388+0000","last_became_active":"2024-02-18T00:33:59.624212+0000","last_became_peered":"2024-02-18T00:33:59.624212+0000","last_unstale":"2024-02-18T00:33:59.645388+0000","last_undegraded":"2024-02-18T00:33:59.645388+0000","last_fullsized":"2024-02-18T00:33:59.645388+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.611930+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:33:59.611930+0000","last_peered":"2024-02-18T00:33:59.611930+0000","last_clean":"2024-02-18T00:33:59.611930+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:33:59.611930+0000","last_undegraded":"2024-02-18T00:33:59.611930+0000","last_fullsized":"2024-02-18T00:33:59.611930+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":141,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593846+0000","last_change":"2024-02-18T00:31:54.074682+0000","last_active":"2024-02-18T00:33:59.593846+0000","last_peered":"2024-02-18T00:33:59.593846+0000","last_clean":"2024-02-18T00:33:59.593846+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:33:59.593846+0000","last_undegraded":"2024-02-18T00:33:59.593846+0000","last_fullsized":"2024-02-18T00:33:59.593846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:31:54.074589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:31:54.074589+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597140+0000","last_change":"2024-02-18T00:32:40.792705+0000","last_active":"2024-02-18T00:33:59.597140+0000","last_peered":"2024-02-18T00:33:59.597140+0000","last_clean":"2024-02-18T00:33:59.597140+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:33:59.597140+0000","last_undegraded":"2024-02-18T00:33:59.597140+0000","last_fullsized":"2024-02-18T00:33:59.597140+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:40.792593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.792593+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595061+0000","last_change":"2024-02-18T00:32:43.773879+0000","last_active":"2024-02-18T00:33:59.595061+0000","last_peered":"2024-02-18T00:33:59.595061+0000","last_clean":"2024-02-18T00:33:59.595061+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:33:59.595061+0000","last_undegraded":"2024-02-18T00:33:59.595061+0000","last_fullsized":"2024-02-18T00:33:59.595061+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.773801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.773801+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592601+0000","last_change":"2024-02-18T00:32:46.709442+0000","last_active":"2024-02-18T00:33:59.592601+0000","last_peered":"2024-02-18T00:33:59.592601+0000","last_clean":"2024-02-18T00:33:59.592601+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:33:59.592601+0000","last_undegraded":"2024-02-18T00:33:59.592601+0000","last_fullsized":"2024-02-18T00:33:59.592601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:46.709314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:46.709314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.6","version":"145'2","reported_seq":78,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596741+0000","last_change":"2024-02-18T00:33:44.433587+0000","last_active":"2024-02-18T00:33:59.596741+0000","last_peered":"2024-02-18T00:33:59.596741+0000","last_clean":"2024-02-18T00:33:59.596741+0000","last_became_active":"2024-02-18T00:33:01.493762+0000","last_became_peered":"2024-02-18T00:33:01.493762+0000","last_unstale":"2024-02-18T00:33:59.596741+0000","last_undegraded":"2024-02-18T00:33:59.596741+0000","last_fullsized":"2024-02-18T00:33:59.596741+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"145'2","last_scrub_stamp":"2024-02-18T00:33:44.433478+0000","last_deep_scrub":"145'2","last_deep_scrub_stamp":"2024-02-18T00:33:44.433478+0000","last_clean_scrub_stamp":"2024-02-18T00:33:44.433478+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":65,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.623472+0000","last_change":"2024-02-18T00:33:59.623101+0000","last_active":"2024-02-18T00:33:59.623472+0000","last_peered":"2024-02-18T00:33:59.623472+0000","last_clean":"2024-02-18T00:33:59.623472+0000","last_became_active":"2024-02-18T00:33:59.622824+0000","last_became_peered":"2024-02-18T00:33:59.622824+0000","last_unstale":"2024-02-18T00:33:59.623472+0000","last_undegraded":"2024-02-18T00:33:59.623472+0000","last_fullsized":"2024-02-18T00:33:59.623472+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594114+0000","last_change":"2024-02-18T00:33:02.499776+0000","last_active":"2024-02-18T00:33:59.594114+0000","last_peered":"2024-02-18T00:33:59.594114+0000","last_clean":"2024-02-18T00:33:59.594114+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:33:59.594114+0000","last_undegraded":"2024-02-18T00:33:59.594114+0000","last_fullsized":"2024-02-18T00:33:59.594114+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":128,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592644+0000","last_change":"2024-02-18T00:32:48.722576+0000","last_active":"2024-02-18T00:33:59.592644+0000","last_peered":"2024-02-18T00:33:59.592644+0000","last_clean":"2024-02-18T00:33:59.592644+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:33:59.592644+0000","last_undegraded":"2024-02-18T00:33:59.592644+0000","last_fullsized":"2024-02-18T00:33:59.592644+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:32:48.722501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:48.722501+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601983+0000","last_change":"2024-02-18T00:33:39.051726+0000","last_active":"2024-02-18T00:33:59.601983+0000","last_peered":"2024-02-18T00:33:59.601983+0000","last_clean":"2024-02-18T00:33:59.601983+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:33:59.601983+0000","last_undegraded":"2024-02-18T00:33:59.601983+0000","last_fullsized":"2024-02-18T00:33:59.601983+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.051653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.051653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":134,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596703+0000","last_change":"2024-02-18T00:33:56.296342+0000","last_active":"2024-02-18T00:33:59.596703+0000","last_peered":"2024-02-18T00:33:59.596703+0000","last_clean":"2024-02-18T00:33:59.596703+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:33:59.596703+0000","last_undegraded":"2024-02-18T00:33:59.596703+0000","last_fullsized":"2024-02-18T00:33:59.596703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:56.296274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.296274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596890+0000","last_change":"2024-02-18T00:33:48.676256+0000","last_active":"2024-02-18T00:33:59.596890+0000","last_peered":"2024-02-18T00:33:59.596890+0000","last_clean":"2024-02-18T00:33:59.596890+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:33:59.596890+0000","last_undegraded":"2024-02-18T00:33:59.596890+0000","last_fullsized":"2024-02-18T00:33:59.596890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.719484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.719484+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":76,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594039+0000","last_change":"2024-02-18T00:33:42.414998+0000","last_active":"2024-02-18T00:33:59.594039+0000","last_peered":"2024-02-18T00:33:59.594039+0000","last_clean":"2024-02-18T00:33:59.594039+0000","last_became_active":"2024-02-18T00:33:01.494037+0000","last_became_peered":"2024-02-18T00:33:01.494037+0000","last_unstale":"2024-02-18T00:33:59.594039+0000","last_undegraded":"2024-02-18T00:33:59.594039+0000","last_fullsized":"2024-02-18T00:33:59.594039+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.414902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:42.414902+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.414902+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.621855+0000","last_change":"2024-02-18T00:33:59.605476+0000","last_active":"2024-02-18T00:33:59.621855+0000","last_peered":"2024-02-18T00:33:59.621855+0000","last_clean":"2024-02-18T00:33:59.621855+0000","last_became_active":"2024-02-18T00:33:59.604847+0000","last_became_peered":"2024-02-18T00:33:59.604847+0000","last_unstale":"2024-02-18T00:33:59.621855+0000","last_undegraded":"2024-02-18T00:33:59.621855+0000","last_fullsized":"2024-02-18T00:33:59.621855+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601385+0000","last_change":"2024-02-18T00:33:48.714793+0000","last_active":"2024-02-18T00:33:59.601385+0000","last_peered":"2024-02-18T00:33:59.601385+0000","last_clean":"2024-02-18T00:33:59.601385+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:33:59.601385+0000","last_undegraded":"2024-02-18T00:33:59.601385+0000","last_fullsized":"2024-02-18T00:33:59.601385+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":179,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593052+0000","last_change":"2024-02-18T00:33:48.727740+0000","last_active":"2024-02-18T00:33:59.593052+0000","last_peered":"2024-02-18T00:33:59.593052+0000","last_clean":"2024-02-18T00:33:59.593052+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:33:59.593052+0000","last_undegraded":"2024-02-18T00:33:59.593052+0000","last_fullsized":"2024-02-18T00:33:59.593052+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:32:40.775672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:40.775672+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594439+0000","last_change":"2024-02-18T00:33:51.295157+0000","last_active":"2024-02-18T00:33:59.594439+0000","last_peered":"2024-02-18T00:33:59.594439+0000","last_clean":"2024-02-18T00:33:59.594439+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:33:59.594439+0000","last_undegraded":"2024-02-18T00:33:59.594439+0000","last_fullsized":"2024-02-18T00:33:59.594439+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.295093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.295093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589192+0000","last_change":"2024-02-18T00:33:48.742173+0000","last_active":"2024-02-18T00:33:59.589192+0000","last_peered":"2024-02-18T00:33:59.589192+0000","last_clean":"2024-02-18T00:33:59.589192+0000","last_became_active":"2024-02-18T00:33:48.741677+0000","last_became_peered":"2024-02-18T00:33:48.741677+0000","last_unstale":"2024-02-18T00:33:59.589192+0000","last_undegraded":"2024-02-18T00:33:59.589192+0000","last_fullsized":"2024-02-18T00:33:59.589192+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:07.759964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:07.759964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595254+0000","last_change":"2024-02-18T00:32:51.698582+0000","last_active":"2024-02-18T00:33:59.595254+0000","last_peered":"2024-02-18T00:33:59.595254+0000","last_clean":"2024-02-18T00:33:59.595254+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:33:59.595254+0000","last_undegraded":"2024-02-18T00:33:59.595254+0000","last_fullsized":"2024-02-18T00:33:59.595254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:51.698513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:51.698513+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"112'6","reported_seq":92,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.615255+0000","last_change":"2024-02-18T00:33:40.052581+0000","last_active":"2024-02-18T00:33:59.615255+0000","last_peered":"2024-02-18T00:33:59.615255+0000","last_clean":"2024-02-18T00:33:59.615255+0000","last_became_active":"2024-02-18T00:33:01.526381+0000","last_became_peered":"2024-02-18T00:33:01.526381+0000","last_unstale":"2024-02-18T00:33:59.615255+0000","last_undegraded":"2024-02-18T00:33:59.615255+0000","last_fullsized":"2024-02-18T00:33:59.615255+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'6","last_scrub_stamp":"2024-02-18T00:33:40.052435+0000","last_deep_scrub":"112'6","last_deep_scrub_stamp":"2024-02-18T00:33:40.052435+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.052435+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165852,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":24583,"num_write":3,"num_write_kb":24579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.0","version":"109'5","reported_seq":69,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594130+0000","last_change":"2024-02-18T00:33:02.516955+0000","last_active":"2024-02-18T00:33:59.594130+0000","last_peered":"2024-02-18T00:33:59.594130+0000","last_clean":"2024-02-18T00:33:59.594130+0000","last_became_active":"2024-02-18T00:33:02.516681+0000","last_became_peered":"2024-02-18T00:33:02.516681+0000","last_unstale":"2024-02-18T00:33:59.594130+0000","last_undegraded":"2024-02-18T00:33:59.594130+0000","last_fullsized":"2024-02-18T00:33:59.594130+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590369+0000","last_change":"2024-02-18T00:33:48.680037+0000","last_active":"2024-02-18T00:33:59.590369+0000","last_peered":"2024-02-18T00:33:59.590369+0000","last_clean":"2024-02-18T00:33:59.590369+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:33:59.590369+0000","last_undegraded":"2024-02-18T00:33:59.590369+0000","last_fullsized":"2024-02-18T00:33:59.590369+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":185,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.612865+0000","last_change":"2024-02-18T00:33:56.121942+0000","last_active":"2024-02-18T00:33:59.612865+0000","last_peered":"2024-02-18T00:33:59.612865+0000","last_clean":"2024-02-18T00:33:59.612865+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:33:59.612865+0000","last_undegraded":"2024-02-18T00:33:59.612865+0000","last_fullsized":"2024-02-18T00:33:59.612865+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.121872+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":133,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600056+0000","last_change":"2024-02-18T00:33:48.634995+0000","last_active":"2024-02-18T00:33:59.600056+0000","last_peered":"2024-02-18T00:33:59.600056+0000","last_clean":"2024-02-18T00:33:59.600056+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:33:59.600056+0000","last_undegraded":"2024-02-18T00:33:59.600056+0000","last_fullsized":"2024-02-18T00:33:59.600056+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:49.926996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.926996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":206,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594613+0000","last_change":"2024-02-18T00:33:48.784978+0000","last_active":"2024-02-18T00:33:59.594613+0000","last_peered":"2024-02-18T00:33:59.594613+0000","last_clean":"2024-02-18T00:33:59.594613+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:33:59.594613+0000","last_undegraded":"2024-02-18T00:33:59.594613+0000","last_fullsized":"2024-02-18T00:33:59.594613+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:32:45.395085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:45.395085+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":37,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601372+0000","last_change":"2024-02-18T00:33:48.780105+0000","last_active":"2024-02-18T00:33:59.601372+0000","last_peered":"2024-02-18T00:33:59.601372+0000","last_clean":"2024-02-18T00:33:59.601372+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:33:59.601372+0000","last_undegraded":"2024-02-18T00:33:59.601372+0000","last_fullsized":"2024-02-18T00:33:59.601372+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:31:45.654827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:31:45.654827+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":75,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.610041+0000","last_change":"2024-02-18T00:33:48.752577+0000","last_active":"2024-02-18T00:33:59.610041+0000","last_peered":"2024-02-18T00:33:59.610041+0000","last_clean":"2024-02-18T00:33:59.610041+0000","last_became_active":"2024-02-18T00:33:48.752058+0000","last_became_peered":"2024-02-18T00:33:48.752058+0000","last_unstale":"2024-02-18T00:33:59.610041+0000","last_undegraded":"2024-02-18T00:33:59.610041+0000","last_fullsized":"2024-02-18T00:33:59.610041+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1","version":"109'5","reported_seq":69,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592072+0000","last_change":"2024-02-18T00:33:02.504123+0000","last_active":"2024-02-18T00:33:59.592072+0000","last_peered":"2024-02-18T00:33:59.592072+0000","last_clean":"2024-02-18T00:33:59.592072+0000","last_became_active":"2024-02-18T00:33:02.503440+0000","last_became_peered":"2024-02-18T00:33:02.503440+0000","last_unstale":"2024-02-18T00:33:59.592072+0000","last_undegraded":"2024-02-18T00:33:59.592072+0000","last_fullsized":"2024-02-18T00:33:59.592072+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593100+0000","last_change":"2024-02-18T00:33:48.698785+0000","last_active":"2024-02-18T00:33:59.593100+0000","last_peered":"2024-02-18T00:33:59.593100+0000","last_clean":"2024-02-18T00:33:59.593100+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:33:59.593100+0000","last_undegraded":"2024-02-18T00:33:59.593100+0000","last_fullsized":"2024-02-18T00:33:59.593100+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":161,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594662+0000","last_change":"2024-02-18T00:33:52.250914+0000","last_active":"2024-02-18T00:33:59.594662+0000","last_peered":"2024-02-18T00:33:59.594662+0000","last_clean":"2024-02-18T00:33:59.594662+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:33:59.594662+0000","last_undegraded":"2024-02-18T00:33:59.594662+0000","last_fullsized":"2024-02-18T00:33:59.594662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:52.250848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.250848+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592715+0000","last_change":"2024-02-18T00:31:49.512647+0000","last_active":"2024-02-18T00:33:59.592715+0000","last_peered":"2024-02-18T00:33:59.592715+0000","last_clean":"2024-02-18T00:33:59.592715+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:33:59.592715+0000","last_undegraded":"2024-02-18T00:33:59.592715+0000","last_fullsized":"2024-02-18T00:33:59.592715+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:31:49.512572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":128,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.612940+0000","last_change":"2024-02-18T00:33:48.751672+0000","last_active":"2024-02-18T00:33:59.612940+0000","last_peered":"2024-02-18T00:33:59.612940+0000","last_clean":"2024-02-18T00:33:59.612940+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:33:59.612940+0000","last_undegraded":"2024-02-18T00:33:59.612940+0000","last_fullsized":"2024-02-18T00:33:59.612940+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.025296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.025296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":123,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596335+0000","last_change":"2024-02-18T00:32:59.800515+0000","last_active":"2024-02-18T00:33:59.596335+0000","last_peered":"2024-02-18T00:33:59.596335+0000","last_clean":"2024-02-18T00:33:59.596335+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:33:59.596335+0000","last_undegraded":"2024-02-18T00:33:59.596335+0000","last_fullsized":"2024-02-18T00:33:59.596335+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.800355+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"106'2","reported_seq":40,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590597+0000","last_change":"2024-02-18T00:33:48.827156+0000","last_active":"2024-02-18T00:33:59.590597+0000","last_peered":"2024-02-18T00:33:59.590597+0000","last_clean":"2024-02-18T00:33:59.590597+0000","last_became_active":"2024-02-18T00:33:48.668875+0000","last_became_peered":"2024-02-18T00:33:48.668875+0000","last_unstale":"2024-02-18T00:33:59.590597+0000","last_undegraded":"2024-02-18T00:33:59.590597+0000","last_fullsized":"2024-02-18T00:33:59.590597+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":7,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595248+0000","last_change":"2024-02-18T00:33:02.537930+0000","last_active":"2024-02-18T00:33:59.595248+0000","last_peered":"2024-02-18T00:33:59.595248+0000","last_clean":"2024-02-18T00:33:59.595248+0000","last_became_active":"2024-02-18T00:33:02.536038+0000","last_became_peered":"2024-02-18T00:33:02.536038+0000","last_unstale":"2024-02-18T00:33:59.595248+0000","last_undegraded":"2024-02-18T00:33:59.595248+0000","last_fullsized":"2024-02-18T00:33:59.595248+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":77,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593538+0000","last_change":"2024-02-18T00:33:04.698462+0000","last_active":"2024-02-18T00:33:59.593538+0000","last_peered":"2024-02-18T00:33:59.593538+0000","last_clean":"2024-02-18T00:33:59.593538+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:33:59.593538+0000","last_undegraded":"2024-02-18T00:33:59.593538+0000","last_fullsized":"2024-02-18T00:33:59.593538+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":152,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595677+0000","last_change":"2024-02-18T00:32:48.395232+0000","last_active":"2024-02-18T00:33:59.595677+0000","last_peered":"2024-02-18T00:33:59.595677+0000","last_clean":"2024-02-18T00:33:59.595677+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:33:59.595677+0000","last_undegraded":"2024-02-18T00:33:59.595677+0000","last_fullsized":"2024-02-18T00:33:59.595677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:48.395162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:48.395162+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":138,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591899+0000","last_change":"2024-02-18T00:33:48.669825+0000","last_active":"2024-02-18T00:33:59.591899+0000","last_peered":"2024-02-18T00:33:59.591899+0000","last_clean":"2024-02-18T00:33:59.591899+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:33:59.591899+0000","last_undegraded":"2024-02-18T00:33:59.591899+0000","last_fullsized":"2024-02-18T00:33:59.591899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.386090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.386090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597789+0000","last_change":"2024-02-18T00:32:06.883082+0000","last_active":"2024-02-18T00:33:59.597789+0000","last_peered":"2024-02-18T00:33:59.597789+0000","last_clean":"2024-02-18T00:33:59.597789+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:33:59.597789+0000","last_undegraded":"2024-02-18T00:33:59.597789+0000","last_fullsized":"2024-02-18T00:33:59.597789+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:06.883019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:32:06.883019+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594385+0000","last_change":"2024-02-18T00:32:38.985290+0000","last_active":"2024-02-18T00:33:59.594385+0000","last_peered":"2024-02-18T00:33:59.594385+0000","last_clean":"2024-02-18T00:33:59.594385+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:33:59.594385+0000","last_undegraded":"2024-02-18T00:33:59.594385+0000","last_fullsized":"2024-02-18T00:33:59.594385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.985181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.985181+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"138'7","reported_seq":83,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592432+0000","last_change":"2024-02-18T00:33:36.856080+0000","last_active":"2024-02-18T00:33:59.592432+0000","last_peered":"2024-02-18T00:33:59.592432+0000","last_clean":"2024-02-18T00:33:59.592432+0000","last_became_active":"2024-02-18T00:33:01.593606+0000","last_became_peered":"2024-02-18T00:33:01.593606+0000","last_unstale":"2024-02-18T00:33:59.592432+0000","last_undegraded":"2024-02-18T00:33:59.592432+0000","last_fullsized":"2024-02-18T00:33:59.592432+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"138'7","last_scrub_stamp":"2024-02-18T00:33:36.855953+0000","last_deep_scrub":"138'7","last_deep_scrub_stamp":"2024-02-18T00:33:36.855953+0000","last_clean_scrub_stamp":"2024-02-18T00:33:36.855953+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3","version":"109'5","reported_seq":42,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601085+0000","last_change":"2024-02-18T00:33:49.063013+0000","last_active":"2024-02-18T00:33:59.601085+0000","last_peered":"2024-02-18T00:33:59.601085+0000","last_clean":"2024-02-18T00:33:59.601085+0000","last_became_active":"2024-02-18T00:33:48.736951+0000","last_became_peered":"2024-02-18T00:33:48.736951+0000","last_unstale":"2024-02-18T00:33:59.601085+0000","last_undegraded":"2024-02-18T00:33:59.601085+0000","last_fullsized":"2024-02-18T00:33:59.601085+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594132+0000","last_change":"2024-02-18T00:33:02.473416+0000","last_active":"2024-02-18T00:33:59.594132+0000","last_peered":"2024-02-18T00:33:59.594132+0000","last_clean":"2024-02-18T00:33:59.594132+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:33:59.594132+0000","last_undegraded":"2024-02-18T00:33:59.594132+0000","last_fullsized":"2024-02-18T00:33:59.594132+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":160,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599382+0000","last_change":"2024-02-18T00:32:44.768998+0000","last_active":"2024-02-18T00:33:59.599382+0000","last_peered":"2024-02-18T00:33:59.599382+0000","last_clean":"2024-02-18T00:33:59.599382+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:33:59.599382+0000","last_undegraded":"2024-02-18T00:33:59.599382+0000","last_fullsized":"2024-02-18T00:33:59.599382+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:32:44.768886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:44.768886+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":149,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595057+0000","last_change":"2024-02-18T00:33:48.748311+0000","last_active":"2024-02-18T00:33:59.595057+0000","last_peered":"2024-02-18T00:33:59.595057+0000","last_clean":"2024-02-18T00:33:59.595057+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:33:59.595057+0000","last_undegraded":"2024-02-18T00:33:59.595057+0000","last_fullsized":"2024-02-18T00:33:59.595057+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:33:45.906516+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.906516+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590646+0000","last_change":"2024-02-18T00:32:41.027232+0000","last_active":"2024-02-18T00:33:59.590646+0000","last_peered":"2024-02-18T00:33:59.590646+0000","last_clean":"2024-02-18T00:33:59.590646+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:33:59.590646+0000","last_undegraded":"2024-02-18T00:33:59.590646+0000","last_fullsized":"2024-02-18T00:33:59.590646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:41.027153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:41.027153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.606771+0000","last_change":"2024-02-18T00:33:38.914876+0000","last_active":"2024-02-18T00:33:59.606771+0000","last_peered":"2024-02-18T00:33:59.606771+0000","last_clean":"2024-02-18T00:33:59.606771+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:33:59.606771+0000","last_undegraded":"2024-02-18T00:33:59.606771+0000","last_fullsized":"2024-02-18T00:33:59.606771+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:38.914691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:38.914691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"140'10","reported_seq":89,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.597884+0000","last_change":"2024-02-18T00:33:48.799111+0000","last_active":"2024-02-18T00:33:59.597884+0000","last_peered":"2024-02-18T00:33:59.597884+0000","last_clean":"2024-02-18T00:33:59.597884+0000","last_became_active":"2024-02-18T00:33:48.743202+0000","last_became_peered":"2024-02-18T00:33:48.743202+0000","last_unstale":"2024-02-18T00:33:59.597884+0000","last_undegraded":"2024-02-18T00:33:59.597884+0000","last_fullsized":"2024-02-18T00:33:59.597884+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'10","last_scrub_stamp":"2024-02-18T00:33:38.826590+0000","last_deep_scrub":"140'10","last_deep_scrub_stamp":"2024-02-18T00:33:38.826590+0000","last_clean_scrub_stamp":"2024-02-18T00:33:38.826590+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2807,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594944+0000","last_change":"2024-02-18T00:33:04.688881+0000","last_active":"2024-02-18T00:33:59.594944+0000","last_peered":"2024-02-18T00:33:59.594944+0000","last_clean":"2024-02-18T00:33:59.594944+0000","last_became_active":"2024-02-18T00:33:04.688607+0000","last_became_peered":"2024-02-18T00:33:04.688607+0000","last_unstale":"2024-02-18T00:33:59.594944+0000","last_undegraded":"2024-02-18T00:33:59.594944+0000","last_fullsized":"2024-02-18T00:33:59.594944+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.607826+0000","last_change":"2024-02-18T00:33:48.765117+0000","last_active":"2024-02-18T00:33:59.607826+0000","last_peered":"2024-02-18T00:33:59.607826+0000","last_clean":"2024-02-18T00:33:59.607826+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:33:59.607826+0000","last_undegraded":"2024-02-18T00:33:59.607826+0000","last_fullsized":"2024-02-18T00:33:59.607826+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":180,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590042+0000","last_change":"2024-02-18T00:33:00.013117+0000","last_active":"2024-02-18T00:33:59.590042+0000","last_peered":"2024-02-18T00:33:59.590042+0000","last_clean":"2024-02-18T00:33:59.590042+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:33:59.590042+0000","last_undegraded":"2024-02-18T00:33:59.590042+0000","last_fullsized":"2024-02-18T00:33:59.590042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:33:00.013051+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:33:00.013051+0000","last_clean_scrub_stamp":"2024-02-18T00:33:00.013051+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601274+0000","last_change":"2024-02-18T00:33:48.674444+0000","last_active":"2024-02-18T00:33:59.601274+0000","last_peered":"2024-02-18T00:33:59.601274+0000","last_clean":"2024-02-18T00:33:59.601274+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:33:59.601274+0000","last_undegraded":"2024-02-18T00:33:59.601274+0000","last_fullsized":"2024-02-18T00:33:59.601274+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:41.659220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:41.659220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600912+0000","last_change":"2024-02-18T00:33:48.736897+0000","last_active":"2024-02-18T00:33:59.600912+0000","last_peered":"2024-02-18T00:33:59.600912+0000","last_clean":"2024-02-18T00:33:59.600912+0000","last_became_active":"2024-02-18T00:33:48.736206+0000","last_became_peered":"2024-02-18T00:33:48.736206+0000","last_unstale":"2024-02-18T00:33:59.600912+0000","last_undegraded":"2024-02-18T00:33:59.600912+0000","last_fullsized":"2024-02-18T00:33:59.600912+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:31:58.855979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:31:58.855979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":131,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591633+0000","last_change":"2024-02-18T00:33:48.649527+0000","last_active":"2024-02-18T00:33:59.591633+0000","last_peered":"2024-02-18T00:33:59.591633+0000","last_clean":"2024-02-18T00:33:59.591633+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:33:59.591633+0000","last_undegraded":"2024-02-18T00:33:59.591633+0000","last_fullsized":"2024-02-18T00:33:59.591633+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.996913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.996913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"108'1001","reported_seq":2067,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600490+0000","last_change":"2024-02-18T00:33:01.535111+0000","last_active":"2024-02-18T00:33:59.600490+0000","last_peered":"2024-02-18T00:33:59.600490+0000","last_clean":"2024-02-18T00:33:59.600490+0000","last_became_active":"2024-02-18T00:33:01.532648+0000","last_became_peered":"2024-02-18T00:33:01.532648+0000","last_unstale":"2024-02-18T00:33:59.600490+0000","last_undegraded":"2024-02-18T00:33:59.600490+0000","last_fullsized":"2024-02-18T00:33:59.600490+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1001,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1000,"num_read_kb":0,"num_write":1000,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.596194+0000","last_change":"2024-02-18T00:33:02.511002+0000","last_active":"2024-02-18T00:33:59.596194+0000","last_peered":"2024-02-18T00:33:59.596194+0000","last_clean":"2024-02-18T00:33:59.596194+0000","last_became_active":"2024-02-18T00:33:02.510701+0000","last_became_peered":"2024-02-18T00:33:02.510701+0000","last_unstale":"2024-02-18T00:33:59.596194+0000","last_undegraded":"2024-02-18T00:33:59.596194+0000","last_fullsized":"2024-02-18T00:33:59.596194+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595996+0000","last_change":"2024-02-18T00:33:04.678891+0000","last_active":"2024-02-18T00:33:59.595996+0000","last_peered":"2024-02-18T00:33:59.595996+0000","last_clean":"2024-02-18T00:33:59.595996+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:33:59.595996+0000","last_undegraded":"2024-02-18T00:33:59.595996+0000","last_fullsized":"2024-02-18T00:33:59.595996+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":150,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.591009+0000","last_change":"2024-02-18T00:32:02.063451+0000","last_active":"2024-02-18T00:33:59.591009+0000","last_peered":"2024-02-18T00:33:59.591009+0000","last_clean":"2024-02-18T00:33:59.591009+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:33:59.591009+0000","last_undegraded":"2024-02-18T00:33:59.591009+0000","last_fullsized":"2024-02-18T00:33:59.591009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:32:02.063346+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:32:02.063346+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.593962+0000","last_change":"2024-02-18T00:33:48.729504+0000","last_active":"2024-02-18T00:33:59.593962+0000","last_peered":"2024-02-18T00:33:59.593962+0000","last_clean":"2024-02-18T00:33:59.593962+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:33:59.593962+0000","last_undegraded":"2024-02-18T00:33:59.593962+0000","last_fullsized":"2024-02-18T00:33:59.593962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:03.808887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:36.502900+0000","last_clean_scrub_stamp":"2024-02-18T00:33:03.808887+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":24,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.590202+0000","last_change":"2024-02-18T00:33:48.733307+0000","last_active":"2024-02-18T00:33:59.590202+0000","last_peered":"2024-02-18T00:33:59.590202+0000","last_clean":"2024-02-18T00:33:59.590202+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:33:59.590202+0000","last_undegraded":"2024-02-18T00:33:59.590202+0000","last_fullsized":"2024-02-18T00:33:59.590202+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.801504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.801504+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":123,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600613+0000","last_change":"2024-02-18T00:33:54.013575+0000","last_active":"2024-02-18T00:33:59.600613+0000","last_peered":"2024-02-18T00:33:59.600613+0000","last_clean":"2024-02-18T00:33:59.600613+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:33:59.600613+0000","last_undegraded":"2024-02-18T00:33:59.600613+0000","last_fullsized":"2024-02-18T00:33:59.600613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:54.013467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.013467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.e","version":"106'2","reported_seq":68,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595924+0000","last_change":"2024-02-18T00:33:01.542256+0000","last_active":"2024-02-18T00:33:59.595924+0000","last_peered":"2024-02-18T00:33:59.595924+0000","last_clean":"2024-02-18T00:33:59.595924+0000","last_became_active":"2024-02-18T00:33:01.541655+0000","last_became_peered":"2024-02-18T00:33:01.541655+0000","last_unstale":"2024-02-18T00:33:59.595924+0000","last_undegraded":"2024-02-18T00:33:59.595924+0000","last_fullsized":"2024-02-18T00:33:59.595924+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"109'9","reported_seq":48,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.595751+0000","last_change":"2024-02-18T00:33:48.836585+0000","last_active":"2024-02-18T00:33:59.595751+0000","last_peered":"2024-02-18T00:33:59.595751+0000","last_clean":"2024-02-18T00:33:59.595751+0000","last_became_active":"2024-02-18T00:33:48.685227+0000","last_became_peered":"2024-02-18T00:33:48.685227+0000","last_unstale":"2024-02-18T00:33:59.595751+0000","last_undegraded":"2024-02-18T00:33:59.595751+0000","last_fullsized":"2024-02-18T00:33:59.595751+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":73,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599854+0000","last_change":"2024-02-18T00:33:48.714160+0000","last_active":"2024-02-18T00:33:59.599854+0000","last_peered":"2024-02-18T00:33:59.599854+0000","last_clean":"2024-02-18T00:33:59.599854+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:33:59.599854+0000","last_undegraded":"2024-02-18T00:33:59.599854+0000","last_fullsized":"2024-02-18T00:33:59.599854+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"64'15","reported_seq":153,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589187+0000","last_change":"2024-02-18T00:32:50.121604+0000","last_active":"2024-02-18T00:33:59.589187+0000","last_peered":"2024-02-18T00:33:59.589187+0000","last_clean":"2024-02-18T00:33:59.589187+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:33:59.589187+0000","last_undegraded":"2024-02-18T00:33:59.589187+0000","last_fullsized":"2024-02-18T00:33:59.589187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.121518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.121518+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":124,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.600194+0000","last_change":"2024-02-18T00:32:43.807178+0000","last_active":"2024-02-18T00:33:59.600194+0000","last_peered":"2024-02-18T00:33:59.600194+0000","last_clean":"2024-02-18T00:33:59.600194+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:33:59.600194+0000","last_undegraded":"2024-02-18T00:33:59.600194+0000","last_fullsized":"2024-02-18T00:33:59.600194+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:43.807055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:32:43.807055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":129,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594411+0000","last_change":"2024-02-18T00:33:48.711008+0000","last_active":"2024-02-18T00:33:59.594411+0000","last_peered":"2024-02-18T00:33:59.594411+0000","last_clean":"2024-02-18T00:33:59.594411+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:33:59.594411+0000","last_undegraded":"2024-02-18T00:33:59.594411+0000","last_fullsized":"2024-02-18T00:33:59.594411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:38.804774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:38.804774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":118,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589952+0000","last_change":"2024-02-18T00:32:36.954112+0000","last_active":"2024-02-18T00:33:59.589952+0000","last_peered":"2024-02-18T00:33:59.589952+0000","last_clean":"2024-02-18T00:33:59.589952+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:33:59.589952+0000","last_undegraded":"2024-02-18T00:33:59.589952+0000","last_fullsized":"2024-02-18T00:33:59.589952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.954039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.954039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":25,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.601585+0000","last_change":"2024-02-18T00:33:48.670965+0000","last_active":"2024-02-18T00:33:59.601585+0000","last_peered":"2024-02-18T00:33:59.601585+0000","last_clean":"2024-02-18T00:33:59.601585+0000","last_became_active":"2024-02-18T00:33:48.670623+0000","last_became_peered":"2024-02-18T00:33:48.670623+0000","last_unstale":"2024-02-18T00:33:59.601585+0000","last_undegraded":"2024-02-18T00:33:59.601585+0000","last_fullsized":"2024-02-18T00:33:59.601585+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.599958+0000","last_change":"2024-02-18T00:33:04.691417+0000","last_active":"2024-02-18T00:33:59.599958+0000","last_peered":"2024-02-18T00:33:59.599958+0000","last_clean":"2024-02-18T00:33:59.599958+0000","last_became_active":"2024-02-18T00:33:04.691013+0000","last_became_peered":"2024-02-18T00:33:04.691013+0000","last_unstale":"2024-02-18T00:33:59.599958+0000","last_undegraded":"2024-02-18T00:33:59.599958+0000","last_fullsized":"2024-02-18T00:33:59.599958+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":64,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.598792+0000","last_change":"2024-02-18T00:33:04.720986+0000","last_active":"2024-02-18T00:33:59.598792+0000","last_peered":"2024-02-18T00:33:59.598792+0000","last_clean":"2024-02-18T00:33:59.598792+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:33:59.598792+0000","last_undegraded":"2024-02-18T00:33:59.598792+0000","last_fullsized":"2024-02-18T00:33:59.598792+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":209,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.592697+0000","last_change":"2024-02-18T00:33:41.866669+0000","last_active":"2024-02-18T00:33:59.592697+0000","last_peered":"2024-02-18T00:33:59.592697+0000","last_clean":"2024-02-18T00:33:59.592697+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:33:59.592697+0000","last_undegraded":"2024-02-18T00:33:59.592697+0000","last_fullsized":"2024-02-18T00:33:59.592697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:33:41.866531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.866531+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":132,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.607487+0000","last_change":"2024-02-18T00:33:48.763096+0000","last_active":"2024-02-18T00:33:59.607487+0000","last_peered":"2024-02-18T00:33:59.607487+0000","last_clean":"2024-02-18T00:33:59.607487+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:33:59.607487+0000","last_undegraded":"2024-02-18T00:33:59.607487+0000","last_fullsized":"2024-02-18T00:33:59.607487+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.998750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.998750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":120,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.589335+0000","last_change":"2024-02-18T00:32:58.038986+0000","last_active":"2024-02-18T00:33:59.589335+0000","last_peered":"2024-02-18T00:33:59.589335+0000","last_clean":"2024-02-18T00:33:59.589335+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:33:59.589335+0000","last_undegraded":"2024-02-18T00:33:59.589335+0000","last_fullsized":"2024-02-18T00:33:59.589335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:58.038886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:32:58.038886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":127,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.594901+0000","last_change":"2024-02-18T00:33:48.699939+0000","last_active":"2024-02-18T00:33:59.594901+0000","last_peered":"2024-02-18T00:33:59.594901+0000","last_clean":"2024-02-18T00:33:59.594901+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:33:59.594901+0000","last_undegraded":"2024-02-18T00:33:59.594901+0000","last_fullsized":"2024-02-18T00:33:59.594901+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:36.732282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:32:36.732282+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.c","version":"136'1","reported_seq":67,"reported_epoch":161,"state":"active+clean","last_fresh":"2024-02-18T00:33:59.615366+0000","last_change":"2024-02-18T00:33:49.173780+0000","last_active":"2024-02-18T00:33:59.615366+0000","last_peered":"2024-02-18T00:33:59.615366+0000","last_clean":"2024-02-18T00:33:59.615366+0000","last_became_active":"2024-02-18T00:33:48.733341+0000","last_became_peered":"2024-02-18T00:33:48.733341+0000","last_unstale":"2024-02-18T00:33:59.615366+0000","last_undegraded":"2024-02-18T00:33:59.615366+0000","last_fullsized":"2024-02-18T00:33:59.615366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:57.248206+0000","last_clean_scrub_stamp":"2024-02-18T00:32:57.248206+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":25165828,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":24577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":25165828,"num_keys_recovered":1000,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":97,"num_pg":32,"stat_sum":{"num_bytes":418,"num_objects":38,"num_object_clones":9,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":39,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":507904,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":96,"acting":96,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":11,"num_read_kb":4,"num_write":28,"num_write_kb":1049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":6144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":96,"acting":96,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":6528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":5227,"num_objects":505,"num_object_clones":400,"num_object_copies":1515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":36,"num_read_kb":1,"num_write":510,"num_write_kb":509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6225920,"data_stored":15714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":922,"ondisk_log_size":922,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":226499262,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1011,"num_read_kb":49161,"num_write":1047,"num_write_kb":226712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":50334492,"num_keys_recovered":2000,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":730173440,"data_stored":729832278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1072,"ondisk_log_size":1072,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":623,"ondisk_log_size":623,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":277,"num_read_kb":272,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1759,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266749,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":143,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":423188,"kb_used_data":125964,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93321964,"statfs":{"total":95995035648,"available":95561691136,"internally_reserved":0,"allocated":128987136,"data_stored":127069768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":10.933,"5min":2.601,"15min":1.1879999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":69.194999999999993,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.24099999999999999},{"interface":"front","average":{"1min":3.1080000000000001,"5min":1.044,"15min":0.67400000000000004},"min":{"1min":0.38400000000000001,"5min":0.307,"15min":0.307},"max":{"1min":9.7539999999999996,"5min":9.7539999999999996,"15min":9.7539999999999996},"last":0.311}]},{"osd":1,"last update":"Sun Feb 18 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":12.468,"5min":2.911,"15min":1.2989999999999999},"min":{"1min":0.41199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":76.108999999999995,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.27900000000000003},{"interface":"front","average":{"1min":12.256,"5min":2.8500000000000001,"15min":1.272},"min":{"1min":0.33100000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":71.872,"5min":71.872,"15min":71.872},"last":0.38}]},{"osd":2,"last update":"Sun Feb 18 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":4.2309999999999999,"5min":1.2569999999999999,"15min":0.755},"min":{"1min":0.32800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":39.939999999999998,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":6.8559999999999999,"5min":1.804,"15min":0.94399999999999995},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":39.673000000000002,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.35899999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":4.3940000000000001,"5min":1.3089999999999999,"15min":0.77100000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":21.219000000000001,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":3.8149999999999999,"5min":1.1879999999999999,"15min":0.73999999999999999},"min":{"1min":0.44800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.40300000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":0.621,"15min":0.46800000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":8.1479999999999997,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.26100000000000001},{"interface":"front","average":{"1min":3.3580000000000001,"5min":1.0429999999999999,"15min":0.60999999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":35.18,"5min":35.18,"15min":35.18},"last":0.32400000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:33:51 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":0.61599999999999999,"15min":0.47299999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":10.606,"5min":10.606,"15min":10.606},"last":0.40899999999999997},{"interface":"front","average":{"1min":3.6280000000000001,"5min":1.123,"15min":0.64700000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":35.119,"5min":35.119,"15min":35.119},"last":0.39800000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.57899999999999996,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.318,"15min":0.318},"max":{"1min":5.1230000000000002,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.38900000000000001},{"interface":"front","average":{"1min":2.8809999999999998,"5min":1.002,"15min":0.68600000000000005},"min":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":32.985999999999997,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.37}]}]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":178,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":423196,"kb_used_data":125972,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93321956,"statfs":{"total":95995035648,"available":95561682944,"internally_reserved":0,"allocated":128995328,"data_stored":127069725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":4.7220000000000004,"5min":1.2569999999999999,"15min":0.61699999999999999},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":32.106999999999999,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":4.9509999999999996,"5min":1.3420000000000001,"15min":0.65500000000000003},"min":{"1min":0.28000000000000003,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":43.137,"5min":43.137,"15min":43.137},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:33:16 2024","interfaces":[{"interface":"back","average":{"1min":3.1800000000000002,"5min":0.94199999999999995,"15min":0.56000000000000005},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":49.201000000000001,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":0.60999999999999999,"15min":0.45900000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":17.596,"5min":17.596,"15min":17.596},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:33:37 2024","interfaces":[{"interface":"back","average":{"1min":3.7719999999999998,"5min":1.097,"15min":0.67400000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":37.524000000000001,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.27200000000000002},{"interface":"front","average":{"1min":3.5550000000000002,"5min":1.081,"15min":0.68799999999999994},"min":{"1min":0.29699999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":32.036000000000001,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:32:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1559999999999999,"5min":1.347,"15min":1.347},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.63800000000000001,"15min":0.58399999999999996},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":4.2370000000000001,"5min":4.2370000000000001,"15min":4.2370000000000001},"last":0.39700000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":2.351,"5min":0.95399999999999996,"15min":0.71599999999999997},"min":{"1min":0.378,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":12.611000000000001,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":2.5870000000000002,"5min":1.0129999999999999,"15min":0.748},"min":{"1min":0.41199999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":17.417000000000002,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.40400000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:33:54 2024","interfaces":[{"interface":"back","average":{"1min":3.274,"5min":1.1990000000000001,"15min":0.81799999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":12.834,"5min":12.834,"15min":12.834},"last":0.46200000000000002},{"interface":"front","average":{"1min":3.2509999999999999,"5min":1.1559999999999999,"15min":0.80800000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":12.651999999999999,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.35499999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":0.81999999999999995,"15min":0.69499999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.452000000000002,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":0.80900000000000005,"15min":0.70499999999999996},"min":{"1min":0.51500000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":12.805,"5min":12.805,"15min":12.805},"last":0.42799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":160,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":373980,"kb_used_data":76756,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93371172,"statfs":{"total":95995035648,"available":95612080128,"internally_reserved":0,"allocated":78598144,"data_stored":76742260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:33:49 2024","interfaces":[{"interface":"back","average":{"1min":9.5649999999999995,"5min":2.2490000000000001,"15min":1.169},"min":{"1min":0.223,"5min":0.22,"15min":0.20699999999999999},"max":{"1min":133.38999999999999,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":5.8550000000000004,"5min":1.494,"15min":0.72499999999999998},"min":{"1min":0.217,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":90.623000000000005,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.57699999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":4.9790000000000001,"5min":1.323,"15min":0.69799999999999995},"min":{"1min":0.28000000000000003,"5min":0.23799999999999999,"15min":0.215},"max":{"1min":90.850999999999999,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":5.0149999999999997,"5min":1.3180000000000001,"15min":0.69499999999999995},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.224},"max":{"1min":80.537000000000006,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.46700000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:33:49 2024","interfaces":[{"interface":"back","average":{"1min":4.0270000000000001,"5min":1.1539999999999999,"15min":0.68400000000000005},"min":{"1min":0.26200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":19.006,"5min":19.006,"15min":19.006},"last":0.63200000000000001},{"interface":"front","average":{"1min":9.3320000000000007,"5min":2.2210000000000001,"15min":1.081},"min":{"1min":0.29699999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":146.71899999999999,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:33:01 2024","interfaces":[{"interface":"back","average":{"1min":4.3789999999999996,"5min":1.323,"15min":0.80200000000000005},"min":{"1min":0.34799999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":57.914000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":4.867,"5min":1.444,"15min":0.88800000000000001},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":90.070999999999998,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.89000000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":4.5549999999999997,"5min":1.385,"15min":0.85599999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":90.049000000000007,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.70299999999999996},{"interface":"front","average":{"1min":4.9039999999999999,"5min":1.4350000000000001,"15min":0.83799999999999997},"min":{"1min":0.45800000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":57.991,"5min":57.991,"15min":57.991},"last":0.83499999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:33:49 2024","interfaces":[{"interface":"back","average":{"1min":3.5550000000000002,"5min":1.1899999999999999,"15min":0.78100000000000003},"min":{"1min":0.41799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":19.039000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":6.1079999999999997,"5min":1.7050000000000001,"15min":0.95099999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":89.989999999999995,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.753}]},{"osd":7,"last update":"Sun Feb 18 00:33:09 2024","interfaces":[{"interface":"back","average":{"1min":4.8940000000000001,"5min":1.48,"15min":0.89800000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":90.025000000000006,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.84799999999999998},{"interface":"front","average":{"1min":4.4119999999999999,"5min":1.359,"15min":0.84499999999999997},"min":{"1min":0.434,"5min":0.37,"15min":0.37},"max":{"1min":57.962000000000003,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.71299999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":146,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":374412,"kb_used_data":77188,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93370740,"statfs":{"total":95995035648,"available":95611637760,"internally_reserved":0,"allocated":79040512,"data_stored":76740967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.52500000000000002,"15min":0.41699999999999998},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":12.144,"5min":12.144,"15min":12.144},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.53200000000000003,"15min":0.41999999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":12.335000000000001,"5min":12.335000000000001,"15min":12.335000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.51800000000000002,"15min":0.42299999999999999},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":9.6489999999999991,"5min":9.6489999999999991,"15min":9.6489999999999991},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.46500000000000002,"15min":0.41499999999999998},"min":{"1min":0.23899999999999999,"5min":0.219,"15min":0.219},"max":{"1min":3.5009999999999999,"5min":3.5009999999999999,"15min":3.5009999999999999},"last":0.438}]},{"osd":3,"last update":"Sun Feb 18 00:33:39 2024","interfaces":[{"interface":"back","average":{"1min":2.544,"5min":0.88400000000000001,"15min":0.59799999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":22.712,"5min":22.712,"15min":22.712},"last":0.32600000000000001},{"interface":"front","average":{"1min":2.456,"5min":0.88400000000000001,"15min":0.61199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":22.359000000000002,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.39900000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:33:00 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59699999999999998,"15min":0.58699999999999997},"min":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.5580000000000001,"5min":1.675,"15min":1.675},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.42899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.96199999999999997,"5min":1.173,"15min":1.173},"last":0.497}]},{"osd":5,"last update":"Sun Feb 18 00:33:30 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":0.754,"15min":0.63900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":6.766,"5min":6.766,"15min":6.766},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":0.76300000000000001,"15min":0.67500000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":9.6159999999999997,"5min":9.6159999999999997,"15min":9.6159999999999997},"last":0.41999999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:33:49 2024","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":0.81100000000000005,"15min":0.70499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.9379999999999999,"5min":0.89700000000000002,"15min":0.745},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":18.861000000000001,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.68700000000000006,"15min":0.69999999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.153,"5min":0.753,"15min":0.70399999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5830000000000002,"5min":9.5830000000000002,"15min":9.5830000000000002},"last":0.44700000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":158,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":386424,"kb_used_data":77040,"kb_used_omap":0,"kb_used_meta":309376,"kb_avail":93358728,"statfs":{"total":95995035648,"available":95599337472,"internally_reserved":0,"allocated":78888960,"data_stored":76735986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316801024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.42399999999999999,"15min":0.40699999999999997},"min":{"1min":0.22900000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":3.8340000000000001,"5min":3.8340000000000001,"15min":3.8340000000000001},"last":0.24299999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":0.57599999999999996,"15min":0.46800000000000003},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":6.8700000000000001,"5min":6.8700000000000001,"15min":6.8700000000000001},"last":0.20599999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:33:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.54900000000000004,"15min":0.46800000000000003},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":3.141,"5min":0.95499999999999996,"15min":0.58899999999999997},"min":{"1min":0.252,"5min":0.25,"15min":0.25},"max":{"1min":23.606999999999999,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:33:37 2024","interfaces":[{"interface":"back","average":{"1min":2.2130000000000001,"5min":0.79100000000000004,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":19.681000000000001,"5min":19.681000000000001,"15min":19.681000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.497,"15min":0.45500000000000002},"min":{"1min":0.26200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.0179999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.276}]},{"osd":4,"last update":"Sun Feb 18 00:33:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.8010000000000002,"5min":3.8010000000000002,"15min":3.8010000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.627,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:33:27 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.66400000000000003,"15min":0.626},"min":{"1min":0.38600000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":7.4249999999999998,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":0.73499999999999999,"15min":0.65500000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.7750000000000004,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.35999999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:33:52 2024","interfaces":[{"interface":"back","average":{"1min":2.6920000000000002,"5min":1.022,"15min":0.73899999999999999},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":10.189,"5min":10.189,"15min":10.189},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.238,"5min":0.745,"15min":0.66600000000000004},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":6.4279999999999999,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.626,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.66800000000000004,"15min":0.61899999999999999},"min":{"1min":0.46500000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":6.2759999999999998,"5min":6.2759999999999998,"15min":6.2759999999999998},"last":0.70799999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":193,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":398664,"kb_used_data":101436,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93346488,"statfs":{"total":95995035648,"available":95586803712,"internally_reserved":0,"allocated":103870464,"data_stored":101906013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":6.3239999999999998,"5min":6.3239999999999998,"15min":6.3239999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.57099999999999995},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.0819999999999999,"5min":3.0819999999999999,"15min":3.0819999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.2119999999999997,"5min":6.2119999999999997,"15min":6.2119999999999997},"last":0.623},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.56299999999999994},"min":{"1min":0.313,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.752,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.52500000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.60899999999999999},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2110000000000001,"5min":1.48,"15min":1.48},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.7109999999999999,"5min":3.7109999999999999,"15min":3.7109999999999999},"last":0.70699999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:33:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57399999999999995,"15min":0.60199999999999998},"min":{"1min":0.438,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.022,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.58499999999999996,"15min":0.56100000000000005},"min":{"1min":0.32000000000000001,"5min":0.31,"15min":0.31},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":0.60499999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:33:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41799999999999998,"15min":0.372},"min":{"1min":0.246,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.438,"15min":0.38300000000000001},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.317}]},{"osd":6,"last update":"Sun Feb 18 00:33:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.442,"15min":0.40400000000000003},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52100000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.308},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.48499999999999999,"15min":0.44500000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.34100000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.48799999999999999},"min":{"1min":0.32800000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.747,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.505,"15min":0.47999999999999998},"min":{"1min":0.311,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.33300000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":164,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":398172,"kb_used_data":100948,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93346980,"statfs":{"total":95995035648,"available":95587307520,"internally_reserved":0,"allocated":103370752,"data_stored":101908238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.49299999999999999,"15min":0.52500000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.71899999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":1.1259999999999999,"5min":0.64000000000000001,"15min":0.58199999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":12.744,"5min":12.744,"15min":12.744},"last":0.36099999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":5.3369999999999997,"5min":5.3369999999999997,"15min":5.3369999999999997},"last":0.432},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.628,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.52100000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.44},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.59899999999999998,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.51200000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.373,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":4.1699999999999999,"5min":4.1699999999999999,"15min":4.1699999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.53700000000000003,"15min":0.55100000000000005},"min":{"1min":0.33600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67600000000000005,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.48099999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:33:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.54100000000000004,"15min":0.41899999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":11.818,"5min":11.818,"15min":11.818},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.48199999999999998,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.274,"5min":4.274,"15min":4.274},"last":0.53000000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:33:48 2024","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":0.63500000000000001,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":18.605,"5min":18.605,"15min":18.605},"last":0.496},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.51300000000000001,"15min":0.437},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.2089999999999996,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.502}]},{"osd":7,"last update":"Sun Feb 18 00:33:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.54100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":2778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":75567104,"data_stored":75504250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":125878272,"data_stored":125829207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75534336,"data_stored":75500324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":75538432,"data_stored":75497542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":100728832,"data_stored":100667279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":100700160,"data_stored":100670015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":125861888,"data_stored":125829169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50364416,"data_stored":50334492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":2036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1245184,"data_stored":3143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1130496,"data_stored":2865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":921600,"data_stored":2315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":1092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":864256,"data_stored":2188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":97,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":97,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":97,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":97,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":97,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":97,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":97,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:34:02.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool set .rgw.root pgp_num 42 2024-02-18T00:34:02.564 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:02.565 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:02.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:02.601 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:02.666 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:02.694 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:02.838 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:01.297404+0000 osd.1 (osd.1) 97 : cluster [DBG] 97.8 scrub starts 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:01.298727+0000 osd.1 (osd.1) 98 : cluster [DBG] 97.8 scrub ok 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:01.598067+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e163: 8 total, 8 up, 7 in 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:01.654444+0000 mgr.y (mgr.24454) 222 : audit [DBG] from='client.40262 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:01.824391+0000 mgr.y (mgr.24454) 223 : cluster [DBG] pgmap v232: 334 pgs: 334 active+clean; 216 MiB data, 2.6 GiB used, 623 GiB / 626 GiB avail; 17 KiB/s rd, 43 KiB/s wr, 78 op/s; 287 B/s, 3 objects/s recovering 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:02.526945+0000 mon.a (mon.0) 1245 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:01.297404+0000 osd.1 (osd.1) 97 : cluster [DBG] 97.8 scrub starts 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:01.298727+0000 osd.1 (osd.1) 98 : cluster [DBG] 97.8 scrub ok 2024-02-18T00:34:02.878 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:01.598067+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e163: 8 total, 8 up, 7 in 2024-02-18T00:34:02.879 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:01.654444+0000 mgr.y (mgr.24454) 222 : audit [DBG] from='client.40262 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:34:02.879 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:01.824391+0000 mgr.y (mgr.24454) 223 : cluster [DBG] pgmap v232: 334 pgs: 334 active+clean; 216 MiB data, 2.6 GiB used, 623 GiB / 626 GiB avail; 17 KiB/s rd, 43 KiB/s wr, 78 op/s; 287 B/s, 3 objects/s recovering 2024-02-18T00:34:02.879 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:02.526945+0000 mon.a (mon.0) 1245 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T00:34:02.892 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:02.898 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:02.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:02.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:03.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:01.297404+0000 osd.1 (osd.1) 97 : cluster [DBG] 97.8 scrub starts 2024-02-18T00:34:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:01.298727+0000 osd.1 (osd.1) 98 : cluster [DBG] 97.8 scrub ok 2024-02-18T00:34:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:01.598067+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e163: 8 total, 8 up, 7 in 2024-02-18T00:34:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:01.654444+0000 mgr.y (mgr.24454) 222 : audit [DBG] from='client.40262 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:34:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:01.824391+0000 mgr.y (mgr.24454) 223 : cluster [DBG] pgmap v232: 334 pgs: 334 active+clean; 216 MiB data, 2.6 GiB used, 623 GiB / 626 GiB avail; 17 KiB/s rd, 43 KiB/s wr, 78 op/s; 287 B/s, 3 objects/s recovering 2024-02-18T00:34:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:02.526945+0000 mon.a (mon.0) 1245 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T00:34:03.027 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:03.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:34:03.877 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:02.403874+0000 osd.4 (osd.4) 109 : cluster [DBG] 97.3 scrub starts 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:02.407567+0000 osd.4 (osd.4) 110 : cluster [DBG] 97.3 scrub ok 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:02.604777+0000 mon.a (mon.0) 1246 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:02.604837+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e164: 8 total, 8 up, 7 in 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:02.617641+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.59:0/975494703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:02.619886+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:02.627778+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.59:0/1083930469' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:02.628235+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:03.580970+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:03.581054+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:03.880 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:03.581092+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e165: 8 total, 8 up, 7 in 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:02.403874+0000 osd.4 (osd.4) 109 : cluster [DBG] 97.3 scrub starts 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:02.407567+0000 osd.4 (osd.4) 110 : cluster [DBG] 97.3 scrub ok 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:02.604777+0000 mon.a (mon.0) 1246 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:02.604837+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e164: 8 total, 8 up, 7 in 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:02.617641+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.59:0/975494703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:02.619886+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:02.627778+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.59:0/1083930469' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:02.628235+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:03.580970+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:03.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:03.581054+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:03.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:03.581092+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e165: 8 total, 8 up, 7 in 2024-02-18T00:34:03.882 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:03.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:03.983 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:04.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:02.403874+0000 osd.4 (osd.4) 109 : cluster [DBG] 97.3 scrub starts 2024-02-18T00:34:04.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:02.407567+0000 osd.4 (osd.4) 110 : cluster [DBG] 97.3 scrub ok 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:02.604777+0000 mon.a (mon.0) 1246 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:02.604837+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e164: 8 total, 8 up, 7 in 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:02.617641+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.59:0/975494703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:02.619886+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:02.627778+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.59:0/1083930469' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:02.628235+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:03.580970+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi059-82882-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:03.581054+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi059-82877-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:03.581092+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e165: 8 total, 8 up, 7 in 2024-02-18T00:34:04.010 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:04.210 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:04.213 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:04.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:04.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1834597151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f89a0 msgr2=0x7fe9ec0f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1834597151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f89a0 0x7fe9ec0f8d80 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7fe9dc004870 tx=0x7fe9dc009cb0).stop 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1834597151 shutdown_connections 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1834597151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ec0f7330 0x7fe9ec103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1834597151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f89a0 0x7fe9ec0f8d80 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1834597151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f9270 0x7fe9ec0f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1834597151 >> 172.21.15.59:0/1834597151 conn(0x7fe9ec0f26f0 msgr2=0x7fe9ec0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1834597151 shutdown_connections 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.500+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1834597151 wait complete. 2024-02-18T00:34:04.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 librados: starting msgr at 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 librados: starting objecter 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 Processor -- start 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 -- start start 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 librados: setting wanted keys 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 librados: calling monclient init 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f7330 0x7fe9ec10d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f89a0 0x7fe9ec10d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ec0f9270 0x7fe9ec107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec107760 con 0x7fe9ec0f7330 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec1078d0 con 0x7fe9ec0f9270 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9f13e7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec107a90 con 0x7fe9ec0f89a0 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9ea7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f89a0 0x7fe9ec10d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9eaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f7330 0x7fe9ec10d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:04.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9eaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f7330 0x7fe9ec10d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37364/0 (socket says 172.21.15.59:37364) 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9ea7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f89a0 0x7fe9ec10d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39046/0 (socket says 172.21.15.59:39046) 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.501+0000 7fe9eaffd700 1 -- 172.21.15.59:0/1720071096 learned_addr learned my addr 172.21.15.59:0/1720071096 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9eb7fe700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ec0f9270 0x7fe9ec107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9eaffd700 1 -- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f89a0 msgr2=0x7fe9ec10d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9eaffd700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f89a0 0x7fe9ec10d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9eaffd700 1 -- 172.21.15.59:0/1720071096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ec0f9270 msgr2=0x7fe9ec107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9eaffd700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ec0f9270 0x7fe9ec107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9eaffd700 1 -- 172.21.15.59:0/1720071096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d4005040 con 0x7fe9ec0f7330 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9ea7fc700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f89a0 0x7fe9ec10d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9eaffd700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f7330 0x7fe9ec10d0d0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7fe9dc009eb0 tx=0x7fe9dc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9e3fff700 10 client.?.objecter ms_handle_connect 0x7fe9ec0f7330 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9e3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9e3fff700 1 -- 172.21.15.59:0/1720071096 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00e070 con 0x7fe9ec0f7330 2024-02-18T00:34:04.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9dc005040 con 0x7fe9ec0f7330 2024-02-18T00:34:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9e3fff700 1 -- 172.21.15.59:0/1720071096 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9dc013da0 con 0x7fe9ec0f7330 2024-02-18T00:34:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9f13e7700 10 client.30459.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9e3fff700 1 -- 172.21.15.59:0/1720071096 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc019070 con 0x7fe9ec0f7330 2024-02-18T00:34:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec108000 con 0x7fe9ec0f7330 2024-02-18T00:34:04.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.502+0000 7fe9f13e7700 1 librados: init done 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.503+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec05ee80 con 0x7fe9ec0f7330 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 1 -- 172.21.15.59:0/1720071096 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe9dc01e420 con 0x7fe9ec0f7330 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 1 --2- 172.21.15.59:0/1720071096 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe9d8062c90 0x7fe9d8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 1 -- 172.21.15.59:0/1720071096 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(165..165 src has 44..165) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe9dc07e1e0 con 0x7fe9ec0f7330 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 10 client.30459.objecter ms_dispatch 0x7fe9ec001550 osd_map(165..165 src has 44..165) v4 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 3 client.30459.objecter handle_osd_map got epochs [165,165] > 0 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 3 client.30459.objecter handle_osd_map decoding full epoch 165 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9ea7fc700 1 --2- 172.21.15.59:0/1720071096 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe9d8062c90 0x7fe9d8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:04.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 20 client.30459.objecter dump_active .. 0 homeless 2024-02-18T00:34:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9e3fff700 10 client.30459.objecter ms_handle_connect 0x7fe9d8062c90 2024-02-18T00:34:04.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.504+0000 7fe9ea7fc700 1 --2- 172.21.15.59:0/1720071096 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe9d8062c90 0x7fe9d8065150 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 rx=0x7fe9ec108a10 tx=0x7fe9d400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.508+0000 7fe9e3fff700 1 -- 172.21.15.59:0/1720071096 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe9dc051140 con 0x7fe9ec0f7330 2024-02-18T00:34:04.541 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:04.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:04.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:04.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:04.676 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:34:04.676 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:34:04.686 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:04.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:04.743+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fe9ec108750 con 0x7fe9ec0f7330 2024-02-18T00:34:04.876 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:03.170079+0000 osd.0 (osd.0) 97 : cluster [DBG] 97.2 scrub starts 2024-02-18T00:34:04.876 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:03.173754+0000 osd.0 (osd.0) 98 : cluster [DBG] 97.2 scrub ok 2024-02-18T00:34:04.876 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:03.385118+0000 osd.4 (osd.4) 111 : cluster [DBG] 97.9 scrub starts 2024-02-18T00:34:04.876 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:03.386955+0000 osd.4 (osd.4) 112 : cluster [DBG] 97.9 scrub ok 2024-02-18T00:34:04.876 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:03.824955+0000 mgr.y (mgr.24454) 224 : cluster [DBG] pgmap v235: 398 pgs: 64 unknown, 334 active+clean; 216 MiB data, 2.6 GiB used, 623 GiB / 626 GiB avail; 21 KiB/s rd, 51 KiB/s wr, 80 op/s; 287 B/s, 3 objects/s recovering 2024-02-18T00:34:04.876 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:04.580676+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e166: 8 total, 8 up, 7 in 2024-02-18T00:34:04.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:03.170079+0000 osd.0 (osd.0) 97 : cluster [DBG] 97.2 scrub starts 2024-02-18T00:34:04.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:03.173754+0000 osd.0 (osd.0) 98 : cluster [DBG] 97.2 scrub ok 2024-02-18T00:34:04.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:03.385118+0000 osd.4 (osd.4) 111 : cluster [DBG] 97.9 scrub starts 2024-02-18T00:34:04.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:03.386955+0000 osd.4 (osd.4) 112 : cluster [DBG] 97.9 scrub ok 2024-02-18T00:34:04.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:03.824955+0000 mgr.y (mgr.24454) 224 : cluster [DBG] pgmap v235: 398 pgs: 64 unknown, 334 active+clean; 216 MiB data, 2.6 GiB used, 623 GiB / 626 GiB avail; 21 KiB/s rd, 51 KiB/s wr, 80 op/s; 287 B/s, 3 objects/s recovering 2024-02-18T00:34:04.877 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:04.580676+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e166: 8 total, 8 up, 7 in 2024-02-18T00:34:04.891 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:04.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:04.894 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:04.995 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:03.170079+0000 osd.0 (osd.0) 97 : cluster [DBG] 97.2 scrub starts 2024-02-18T00:34:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:03.173754+0000 osd.0 (osd.0) 98 : cluster [DBG] 97.2 scrub ok 2024-02-18T00:34:05.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:03.385118+0000 osd.4 (osd.4) 111 : cluster [DBG] 97.9 scrub starts 2024-02-18T00:34:05.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:03.386955+0000 osd.4 (osd.4) 112 : cluster [DBG] 97.9 scrub ok 2024-02-18T00:34:05.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:03.824955+0000 mgr.y (mgr.24454) 224 : cluster [DBG] pgmap v235: 398 pgs: 64 unknown, 334 active+clean; 216 MiB data, 2.6 GiB used, 623 GiB / 626 GiB avail; 21 KiB/s rd, 51 KiB/s wr, 80 op/s; 287 B/s, 3 objects/s recovering 2024-02-18T00:34:05.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:04.580676+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e166: 8 total, 8 up, 7 in 2024-02-18T00:34:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:05.046 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:05.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:05.054 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:05.155 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:05.185 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:05.418 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-18T00:34:05.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.590+0000 7fe9e3fff700 1 -- 172.21.15.59:0/1720071096 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v167) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe9dc050d30 con 0x7fe9ec0f7330 2024-02-18T00:34:05.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.596+0000 7fe9f13e7700 10 librados: watch_flush enter 2024-02-18T00:34:05.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.598+0000 7fe9f13e7700 10 librados: watch_flush exit 2024-02-18T00:34:05.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.598+0000 7fe9f13e7700 20 client.30459.objecter shutdown clearing up homeless session... 2024-02-18T00:34:05.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.598+0000 7fe9f13e7700 10 client.30459.objecter successfully canceled tick 2024-02-18T00:34:05.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.603+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe9d8062c90 msgr2=0x7fe9d8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:05.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.603+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1720071096 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe9d8062c90 0x7fe9d8065150 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 rx=0x7fe9ec108a10 tx=0x7fe9d400b040).stop 2024-02-18T00:34:05.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.603+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f7330 msgr2=0x7fe9ec10d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:05.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.603+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f7330 0x7fe9ec10d0d0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7fe9dc009eb0 tx=0x7fe9dc00b040).stop 2024-02-18T00:34:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.605+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 shutdown_connections 2024-02-18T00:34:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.605+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1720071096 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe9d8062c90 0x7fe9d8065150 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.605+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9ec0f9270 0x7fe9ec107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.605+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9ec0f7330 0x7fe9ec10d0d0 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.605+0000 7fe9f13e7700 1 --2- 172.21.15.59:0/1720071096 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9ec0f89a0 0x7fe9ec10d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:05.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.605+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 >> 172.21.15.59:0/1720071096 conn(0x7fe9ec0f26f0 msgr2=0x7fe9ec0fb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:05.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.607+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 shutdown_connections 2024-02-18T00:34:05.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.607+0000 7fe9f13e7700 1 -- 172.21.15.59:0/1720071096 wait complete. 2024-02-18T00:34:05.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:05.607+0000 7fe9f13e7700 1 librados: shutdown 2024-02-18T00:34:05.610 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:34:05.696 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:04.062597+0000 osd.2 (osd.2) 81 : cluster [DBG] 97.1 scrub starts 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:04.063401+0000 osd.2 (osd.2) 82 : cluster [DBG] 97.1 scrub ok 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:04.843252+0000 mon.b (mon.1) 199 : audit [DBG] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:04.844123+0000 mon.b (mon.1) 200 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.519702+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.59:0/1720071096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.520022+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:05.578125+0000 mon.a (mon.0) 1256 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.588301+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? 172.21.15.59:0/1720071096' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.588363+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:05.588428+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e167: 8 total, 8 up, 7 in 2024-02-18T00:34:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.591466+0000 mon.b (mon.1) 201 : audit [INF] from='client.? 172.21.15.59:0/2143884008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.592672+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.596298+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? 172.21.15.59:0/3141847512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82877-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.599505+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2"}]: dispatch 2024-02-18T00:34:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.600081+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.601075+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:04.062597+0000 osd.2 (osd.2) 81 : cluster [DBG] 97.1 scrub starts 2024-02-18T00:34:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:04.063401+0000 osd.2 (osd.2) 82 : cluster [DBG] 97.1 scrub ok 2024-02-18T00:34:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:04.843252+0000 mon.b (mon.1) 199 : audit [DBG] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T00:34:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:04.844123+0000 mon.b (mon.1) 200 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.519702+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.59:0/1720071096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.520022+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:05.578125+0000 mon.a (mon.0) 1256 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.588301+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? 172.21.15.59:0/1720071096' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.588363+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:05.588428+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e167: 8 total, 8 up, 7 in 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.591466+0000 mon.b (mon.1) 201 : audit [INF] from='client.? 172.21.15.59:0/2143884008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.592672+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.596298+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? 172.21.15.59:0/3141847512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82877-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.599505+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2"}]: dispatch 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.600081+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.601075+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:04.062597+0000 osd.2 (osd.2) 81 : cluster [DBG] 97.1 scrub starts 2024-02-18T00:34:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:04.063401+0000 osd.2 (osd.2) 82 : cluster [DBG] 97.1 scrub ok 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:04.843252+0000 mon.b (mon.1) 199 : audit [DBG] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:04.844123+0000 mon.b (mon.1) 200 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.519702+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.59:0/1720071096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.520022+0000 mon.a (mon.0) 1255 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:05.578125+0000 mon.a (mon.0) 1256 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.588301+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? 172.21.15.59:0/1720071096' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.588363+0000 mon.a (mon.0) 1258 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:05.588428+0000 mon.a (mon.0) 1259 : cluster [DBG] osdmap e167: 8 total, 8 up, 7 in 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.591466+0000 mon.b (mon.1) 201 : audit [INF] from='client.? 172.21.15.59:0/2143884008' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.592672+0000 mon.a (mon.0) 1260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.596298+0000 mon.a (mon.0) 1261 : audit [INF] from='client.? 172.21.15.59:0/3141847512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82877-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.599505+0000 mon.b (mon.1) 202 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2"}]: dispatch 2024-02-18T00:34:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.600081+0000 mon.b (mon.1) 203 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.601075+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:06.641 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:05.259614+0000 osd.6 (osd.6) 87 : cluster [DBG] 97.0 scrub starts 2024-02-18T00:34:06.641 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:05.268668+0000 osd.6 (osd.6) 88 : cluster [DBG] 97.0 scrub ok 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:05.826418+0000 mgr.y (mgr.24454) 225 : cluster [DBG] pgmap v238: 397 pgs: 64 creating+peering, 333 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 37 KiB/s rd, 1.2 KiB/s wr, 50 op/s 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:05.827542+0000 mon.a (mon.0) 1263 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:06.587617+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:06.587784+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.59:0/3141847512' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82877-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:06.587875+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:06.587984+0000 mon.a (mon.0) 1267 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:06.588033+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e168: 8 total, 8 up, 7 in 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:06.608190+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:06.613915+0000 mon.b (mon.1) 205 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:06.642 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:06.616046+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:05.259614+0000 osd.6 (osd.6) 87 : cluster [DBG] 97.0 scrub starts 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:05.268668+0000 osd.6 (osd.6) 88 : cluster [DBG] 97.0 scrub ok 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:05.826418+0000 mgr.y (mgr.24454) 225 : cluster [DBG] pgmap v238: 397 pgs: 64 creating+peering, 333 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 37 KiB/s rd, 1.2 KiB/s wr, 50 op/s 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:05.827542+0000 mon.a (mon.0) 1263 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:06.587617+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:06.587784+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.59:0/3141847512' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82877-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:06.587875+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:06.891 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:06.587984+0000 mon.a (mon.0) 1267 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-18T00:34:06.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:06.588033+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e168: 8 total, 8 up, 7 in 2024-02-18T00:34:06.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:06.608190+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:06.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:06.613915+0000 mon.b (mon.1) 205 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:06.892 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:06.616046+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:07.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:05.259614+0000 osd.6 (osd.6) 87 : cluster [DBG] 97.0 scrub starts 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:05.268668+0000 osd.6 (osd.6) 88 : cluster [DBG] 97.0 scrub ok 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:05.826418+0000 mgr.y (mgr.24454) 225 : cluster [DBG] pgmap v238: 397 pgs: 64 creating+peering, 333 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 37 KiB/s rd, 1.2 KiB/s wr, 50 op/s 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:05.827542+0000 mon.a (mon.0) 1263 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:06.587617+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi059-82882-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:06.587784+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.59:0/3141847512' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82877-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:06.587875+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi059-82949-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:06.587984+0000 mon.a (mon.0) 1267 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:06.588033+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e168: 8 total, 8 up, 7 in 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:06.608190+0000 mon.b (mon.1) 204 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:07.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:06.613915+0000 mon.b (mon.1) 205 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:07.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:06.616046+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T00:34:07.404 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:34:07.404 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:34:07.437 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:07.440 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:07.441 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:07.541 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:07.573 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:07.717 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:07.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:07.721 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:07.822 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:07.855 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:07.987 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:07.989 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:07.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:08.090 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:08.099 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-18T00:34:08.121 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:08.366 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:34:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 -- 172.21.15.59:0/739231838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac4104680 msgr2=0x7fdac4106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 --2- 172.21.15.59:0/739231838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac4104680 0x7fdac4106a70 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 rx=0x7fdabc009be0 tx=0x7fdabc004d10).stop 2024-02-18T00:34:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 -- 172.21.15.59:0/739231838 shutdown_connections 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 --2- 172.21.15.59:0/739231838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac40f6b20 0x7fdac40f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 --2- 172.21.15.59:0/739231838 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac4104680 0x7fdac4106a70 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 --2- 172.21.15.59:0/739231838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac40f73f0 0x7fdac4104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 -- 172.21.15.59:0/739231838 >> 172.21.15.59:0/739231838 conn(0x7fdac40f2820 msgr2=0x7fdac40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 -- 172.21.15.59:0/739231838 shutdown_connections 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.097+0000 7fdac899b700 1 -- 172.21.15.59:0/739231838 wait complete. 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.098+0000 7fdac899b700 1 librados: starting msgr at 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.098+0000 7fdac899b700 1 librados: starting objecter 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 Processor -- start 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 -- start start 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 librados: setting wanted keys 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 librados: calling monclient init 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac40f6b20 0x7fdac4110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac40f73f0 0x7fdac4110650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac4104680 0x7fdac410a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac410a840 con 0x7fdac4104680 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac410a9b0 con 0x7fdac40f6b20 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac899b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac410ab70 con 0x7fdac40f73f0 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac1d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac40f73f0 0x7fdac4110650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac1d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac40f73f0 0x7fdac4110650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39138/0 (socket says 172.21.15.59:39138) 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac1d9b700 1 -- 172.21.15.59:0/2900799437 learned_addr learned my addr 172.21.15.59:0/2900799437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.099+0000 7fdac259c700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac40f6b20 0x7fdac4110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac1d9b700 1 -- 172.21.15.59:0/2900799437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac40f6b20 msgr2=0x7fdac4110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac1d9b700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac40f6b20 0x7fdac4110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac1d9b700 1 -- 172.21.15.59:0/2900799437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac4104680 msgr2=0x7fdac410a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac1d9b700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac4104680 0x7fdac410a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:34:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac1d9b700 1 -- 172.21.15.59:0/2900799437 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0097c0 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac1d9b700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac40f73f0 0x7fdac4110650 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 rx=0x7fdab80027f0 tx=0x7fdab8002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdab37fe700 10 client.?.objecter ms_handle_connect 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdab37fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdab37fe700 1 -- 172.21.15.59:0/2900799437 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8007b40 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac410ae50 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac899b700 10 client.40304.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac410b1d0 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdab37fe700 1 -- 172.21.15.59:0/2900799437 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdab800f040 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdac899b700 1 librados: init done 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.100+0000 7fdab37fe700 1 -- 172.21.15.59:0/2900799437 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab800d7e0 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.101+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa4004f50 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.102+0000 7fdab37fe700 1 -- 172.21.15.59:0/2900799437 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fdab8007ca0 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.102+0000 7fdab37fe700 1 --2- 172.21.15.59:0/2900799437 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaac062c90 0x7fdaac065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.102+0000 7fdab37fe700 1 -- 172.21.15.59:0/2900799437 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(169..169 src has 44..169) v4 ==== 8703+0+0 (secure 0 0 0) 0x7fdab807d1a0 con 0x7fdac40f73f0 2024-02-18T00:34:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.102+0000 7fdab37fe700 10 client.40304.objecter ms_dispatch 0x7fdac4001550 osd_map(169..169 src has 44..169) v4 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.103+0000 7fdab37fe700 3 client.40304.objecter handle_osd_map got epochs [169,169] > 0 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.103+0000 7fdab37fe700 3 client.40304.objecter handle_osd_map decoding full epoch 169 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.103+0000 7fdac259c700 1 --2- 172.21.15.59:0/2900799437 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaac062c90 0x7fdaac065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.103+0000 7fdab37fe700 20 client.40304.objecter dump_active .. 0 homeless 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.103+0000 7fdab37fe700 10 client.40304.objecter ms_handle_connect 0x7fdaac062c90 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.103+0000 7fdac259c700 1 --2- 172.21.15.59:0/2900799437 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaac062c90 0x7fdaac065150 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 rx=0x7fdab40048b0 tx=0x7fdab400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.107+0000 7fdab37fe700 1 -- 172.21.15.59:0/2900799437 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdab8050440 con 0x7fdac40f73f0 2024-02-18T00:34:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.284+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"} v 0) v1 -- 0x7fdaa4005780 con 0x7fdac40f73f0 2024-02-18T00:34:08.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.616+0000 7fdab37fe700 1 -- 172.21.15.59:0/2900799437 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]=0 set pool 2 pgp_num to 42 v170) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdab8004570 con 0x7fdac40f73f0 2024-02-18T00:34:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.622+0000 7fdac899b700 10 librados: watch_flush enter 2024-02-18T00:34:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.623+0000 7fdac899b700 10 librados: watch_flush exit 2024-02-18T00:34:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.623+0000 7fdac899b700 20 client.40304.objecter shutdown clearing up homeless session... 2024-02-18T00:34:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.623+0000 7fdac899b700 10 client.40304.objecter successfully canceled tick 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.624+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaac062c90 msgr2=0x7fdaac065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.624+0000 7fdac899b700 1 --2- 172.21.15.59:0/2900799437 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaac062c90 0x7fdaac065150 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 rx=0x7fdab40048b0 tx=0x7fdab400b040).stop 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.624+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac40f73f0 msgr2=0x7fdac4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.624+0000 7fdac899b700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac40f73f0 0x7fdac4110650 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 rx=0x7fdab80027f0 tx=0x7fdab8002b00).stop 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.627+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 shutdown_connections 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.627+0000 7fdac899b700 1 --2- 172.21.15.59:0/2900799437 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaac062c90 0x7fdaac065150 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.627+0000 7fdac899b700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac40f6b20 0x7fdac4110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.628+0000 7fdac899b700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdac4104680 0x7fdac410a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.628+0000 7fdac899b700 1 --2- 172.21.15.59:0/2900799437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdac40f73f0 0x7fdac4110650 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.628+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 >> 172.21.15.59:0/2900799437 conn(0x7fdac40f2820 msgr2=0x7fdac40f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.629+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 shutdown_connections 2024-02-18T00:34:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.629+0000 7fdac899b700 1 -- 172.21.15.59:0/2900799437 wait complete. 2024-02-18T00:34:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:08.629+0000 7fdac899b700 1 librados: shutdown 2024-02-18T00:34:08.635 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 2 pgp_num to 42 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:06.089128+0000 osd.2 (osd.2) 83 : cluster [DBG] 97.7 scrub starts 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:06.092580+0000 osd.2 (osd.2) 84 : cluster [DBG] 97.7 scrub ok 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:07.590301+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:07.590471+0000 mon.a (mon.0) 1271 : cluster [DBG] osdmap e169: 8 total, 8 up, 7 in 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:07.592150+0000 mon.b (mon.1) 206 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:07.593812+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:07.595822+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.59:0/1783214352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:08.661 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:07.597543+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:06.089128+0000 osd.2 (osd.2) 83 : cluster [DBG] 97.7 scrub starts 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:06.092580+0000 osd.2 (osd.2) 84 : cluster [DBG] 97.7 scrub ok 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:07.590301+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:07.590471+0000 mon.a (mon.0) 1271 : cluster [DBG] osdmap e169: 8 total, 8 up, 7 in 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:07.592150+0000 mon.b (mon.1) 206 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:07.593812+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:07.595822+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.59:0/1783214352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:08.662 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:07.597543+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:08.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:06.089128+0000 osd.2 (osd.2) 83 : cluster [DBG] 97.7 scrub starts 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:06.092580+0000 osd.2 (osd.2) 84 : cluster [DBG] 97.7 scrub ok 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:07.590301+0000 mon.a (mon.0) 1270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:07.590471+0000 mon.a (mon.0) 1271 : cluster [DBG] osdmap e169: 8 total, 8 up, 7 in 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:07.592150+0000 mon.b (mon.1) 206 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:07.593812+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:07.595822+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.59:0/1783214352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:08.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:07.597543+0000 mon.a (mon.0) 1273 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.477 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:09.479 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:09.479 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:09.580 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:09.619 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:09.696 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:09.698 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:09.699 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 -- 172.21.15.59:0/726788669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f9610 msgr2=0x7f40c8101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 --2- 172.21.15.59:0/726788669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f9610 0x7f40c8101d20 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 rx=0x7f40bc004870 tx=0x7f40bc009cb0).stop 2024-02-18T00:34:09.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 -- 172.21.15.59:0/726788669 shutdown_connections 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 --2- 172.21.15.59:0/726788669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c80f8cb0 0x7f40c80f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 --2- 172.21.15.59:0/726788669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40c8102210 0x7f40c8104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 --2- 172.21.15.59:0/726788669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f9610 0x7f40c8101d20 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 -- 172.21.15.59:0/726788669 >> 172.21.15.59:0/726788669 conn(0x7f40c80f2820 msgr2=0x7f40c80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 -- 172.21.15.59:0/726788669 shutdown_connections 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 -- 172.21.15.59:0/726788669 wait complete. 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 librados: starting msgr at 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 librados: starting objecter 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 Processor -- start 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 -- start start 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 librados: setting wanted keys 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.691+0000 7f40cfd60700 1 librados: calling monclient init 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f8cb0 0x7f40c810e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40c80f9610 0x7f40c810e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8102210 0x7f40c8108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8108840 con 0x7f40c80f8cb0 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c81089b0 con 0x7f40c8102210 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8108b70 con 0x7f40c80f9610 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f8cb0 0x7f40c810e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cd2fb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40c80f9610 0x7f40c810e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f8cb0 0x7f40c810e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57632/0 (socket says 172.21.15.59:57632) 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 -- 172.21.15.59:0/3975832200 learned_addr learned my addr 172.21.15.59:0/3975832200 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8102210 0x7f40c8108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 -- 172.21.15.59:0/3975832200 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40c80f9610 msgr2=0x7f40c810e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f40c80f9610 0x7f40c810e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 -- 172.21.15.59:0/3975832200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8102210 msgr2=0x7f40c8108350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8102210 0x7f40c8108350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 -- 172.21.15.59:0/3975832200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00097c0 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40c8102210 0x7f40c8108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f8cb0 0x7f40c810e1b0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 rx=0x7f40c40027f0 tx=0x7f40c4002b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40baffd700 10 client.?.objecter ms_handle_connect 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40baffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4007b40 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c400f040 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc005040 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c400d7e0 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 10 client.30501.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c81091b0 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.692+0000 7f40cfd60700 1 librados: init done 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.693+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f40c805fa60 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f40c400dc10 con 0x7f40c80f8cb0 2024-02-18T00:34:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40b4062c90 0x7f40b4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(171..171 src has 44..171) v4 ==== 9491+0+0 (secure 0 0 0) 0x7f40c407d9f0 con 0x7f40c80f8cb0 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 osd_map(171..171 src has 44..171) v4 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 3 client.30501.objecter handle_osd_map got epochs [171,171] > 0 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 3 client.30501.objecter handle_osd_map decoding full epoch 171 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40cd2fb700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40b4062c90 0x7f40b4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 20 client.30501.objecter dump_active .. 0 homeless 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.694+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v171) v1 ==== 54+0+16 (secure 0 0 0) 0x7f40c407e0a0 con 0x7f40c80f8cb0 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 10 client.30501.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 1 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40b4062c90 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cd2fb700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40b4062c90 0x7f40b4065150 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 rx=0x7f40bc009a00 tx=0x7f40bc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6801/3700238977 conn(0x7f40c80fc1e0 0x7f40c80fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 1 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 0 1 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c805ee80 osd=0 4 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 10 client.30501.objecter _send_command 1 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c8107290 con 0x7f40c80fc1e0 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6801/3700238977 conn(0x7f40c80fc1e0 0x7f40c80fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c80fc1e0 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.695+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6801/3700238977 conn(0x7f40c80fc1e0 0x7f40c80fc5c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.696+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c8107290 con 0x7f40c80fc1e0 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.696+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:34:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.696+0000 7f40baffd700 10 client.30501.objecter _finish_command 1 = system:0 2024-02-18T00:34:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.696+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c805ee80 osd=0 4 2024-02-18T00:34:09.722 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:34:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.696+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 0 1 2024-02-18T00:34:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 10 client.30501.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 2 2024-02-18T00:34:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 2 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c805ee80 osd=0 3 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 0 2 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c805ee80 osd=0 4 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 10 client.30501.objecter _send_command 2 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.711+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c80273c0 con 0x7f40c80fc1e0 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c80273c0 con 0x7f40c80fc1e0 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40baffd700 10 client.30501.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c805ee80 osd=0 4 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 0 2 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 10 client.30501.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6805/26279110 conn(0x7f40c80598f0 0x7f40c8059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 1 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8028710 osd=1 4 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 10 client.30501.objecter _send_command 3 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c811b820 con 0x7f40c80598f0 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.712+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6805/26279110 conn(0x7f40c80598f0 0x7f40c8059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.713+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c80598f0 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.713+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6805/26279110 conn(0x7f40c80598f0 0x7f40c8059cf0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.713+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c811b820 con 0x7f40c80598f0 2024-02-18T00:34:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.713+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:34:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.713+0000 7f40baffd700 10 client.30501.objecter _finish_command 3 = system:0 2024-02-18T00:34:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.713+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8028710 osd=1 4 2024-02-18T00:34:09.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.713+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 1 3 2024-02-18T00:34:09.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 10 client.30501.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 4 2024-02-18T00:34:09.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 4 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8028710 osd=1 3 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 1 4 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8028710 osd=1 4 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 10 client.30501.objecter _send_command 4 2024-02-18T00:34:09.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.729+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c8059360 con 0x7f40c80598f0 2024-02-18T00:34:09.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.730+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c8059360 con 0x7f40c80598f0 2024-02-18T00:34:09.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.730+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.730+0000 7f40baffd700 10 client.30501.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.730+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8028710 osd=1 4 2024-02-18T00:34:09.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.730+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 1 4 2024-02-18T00:34:09.732 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:34:09.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 10 client.30501.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 5 2024-02-18T00:34:09.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6809/1153193637 conn(0x7f40c8028270 0x7f40c811a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 5 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 2 5 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8059770 osd=2 4 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 10 client.30501.objecter _send_command 5 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c811a730 con 0x7f40c8028270 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cd2fb700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6809/1153193637 conn(0x7f40c8028270 0x7f40c811a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c8028270 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.731+0000 7f40cd2fb700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6809/1153193637 conn(0x7f40c8028270 0x7f40c811a240 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.732+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c811a730 con 0x7f40c8028270 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.732+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:34:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.732+0000 7f40baffd700 10 client.30501.objecter _finish_command 5 = system:0 2024-02-18T00:34:09.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.732+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8059770 osd=2 4 2024-02-18T00:34:09.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.732+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 2 5 2024-02-18T00:34:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 10 client.30501.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 6 2024-02-18T00:34:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 6 2024-02-18T00:34:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8059770 osd=2 3 2024-02-18T00:34:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 2 6 2024-02-18T00:34:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8059770 osd=2 4 2024-02-18T00:34:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 10 client.30501.objecter _send_command 6 2024-02-18T00:34:09.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.747+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c80277d0 con 0x7f40c8028270 2024-02-18T00:34:09.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c80277d0 con 0x7f40c8028270 2024-02-18T00:34:09.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40baffd700 10 client.30501.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8059770 osd=2 4 2024-02-18T00:34:09.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 2 6 2024-02-18T00:34:09.751 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:34:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 10 client.30501.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 7 2024-02-18T00:34:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6813/1805700964 conn(0x7f40c811d9a0 0x7f40c811fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 7 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 3 7 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811b120 osd=3 4 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 10 client.30501.objecter _send_command 7 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c81229e0 con 0x7f40c811d9a0 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.749+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6813/1805700964 conn(0x7f40c811d9a0 0x7f40c811fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.750+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c811d9a0 2024-02-18T00:34:09.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.750+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6813/1805700964 conn(0x7f40c811d9a0 0x7f40c811fd90 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:07.316436+0000 osd.6 (osd.6) 89 : cluster [DBG] 97.4 scrub starts 2024-02-18T00:34:09.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:07.318577+0000 osd.6 (osd.6) 90 : cluster [DBG] 97.4 scrub ok 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:07.826880+0000 mgr.y (mgr.24454) 226 : cluster [DBG] pgmap v241: 333 pgs: 32 unknown, 301 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 36 KiB/s rd, 46 op/s 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.285556+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.59:0/2900799437' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.535797+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.599217+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.599340+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.599613+0000 mon.a (mon.0) 1277 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]': finished 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:08.599691+0000 mon.a (mon.0) 1278 : cluster [DBG] osdmap e170: 8 total, 8 up, 7 in 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.612814+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? 172.21.15.59:0/1213186724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.613876+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.614751+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.629501+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.59:0/1235599235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:08.630466+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.750+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c81229e0 con 0x7f40c811d9a0 2024-02-18T00:34:09.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.750+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:34:09.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.750+0000 7f40baffd700 10 client.30501.objecter _finish_command 7 = system:0 2024-02-18T00:34:09.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.750+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c811b120 osd=3 4 2024-02-18T00:34:09.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.750+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 3 7 2024-02-18T00:34:09.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:07.316436+0000 osd.6 (osd.6) 89 : cluster [DBG] 97.4 scrub starts 2024-02-18T00:34:09.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:07.318577+0000 osd.6 (osd.6) 90 : cluster [DBG] 97.4 scrub ok 2024-02-18T00:34:09.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:07.826880+0000 mgr.y (mgr.24454) 226 : cluster [DBG] pgmap v241: 333 pgs: 32 unknown, 301 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 36 KiB/s rd, 46 op/s 2024-02-18T00:34:09.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.285556+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.59:0/2900799437' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.535797+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.599217+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.599340+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.599613+0000 mon.a (mon.0) 1277 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]': finished 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:08.599691+0000 mon.a (mon.0) 1278 : cluster [DBG] osdmap e170: 8 total, 8 up, 7 in 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.612814+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? 172.21.15.59:0/1213186724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.613876+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.614751+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.629501+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.59:0/1235599235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:08.630466+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.757 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 10 client.30501.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 8 2024-02-18T00:34:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 8 2024-02-18T00:34:09.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c811b120 osd=3 3 2024-02-18T00:34:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 3 8 2024-02-18T00:34:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811b120 osd=3 4 2024-02-18T00:34:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 10 client.30501.objecter _send_command 8 2024-02-18T00:34:09.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.765+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c811ab90 con 0x7f40c811d9a0 2024-02-18T00:34:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c811ab90 con 0x7f40c811d9a0 2024-02-18T00:34:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40baffd700 10 client.30501.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c811b120 osd=3 4 2024-02-18T00:34:09.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 3 8 2024-02-18T00:34:09.769 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 10 client.30501.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 9 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6800/2044387056 conn(0x7f40c811ad30 0x7f40c8121400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 9 2024-02-18T00:34:09.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 4 9 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811ab90 osd=4 4 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 10 client.30501.objecter _send_command 9 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.767+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c8119da0 con 0x7f40c811ad30 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.768+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6800/2044387056 conn(0x7f40c811ad30 0x7f40c8121400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.768+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c811ad30 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.768+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6800/2044387056 conn(0x7f40c811ad30 0x7f40c8121400 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.769+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c8119da0 con 0x7f40c811ad30 2024-02-18T00:34:09.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.769+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:34:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.769+0000 7f40baffd700 10 client.30501.objecter _finish_command 9 = system:0 2024-02-18T00:34:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.769+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c811ab90 osd=4 4 2024-02-18T00:34:09.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.769+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 4 9 2024-02-18T00:34:09.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 10 client.30501.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 10 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 10 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c811ab90 osd=4 3 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 4 10 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811ab90 osd=4 4 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 10 client.30501.objecter _send_command 10 2024-02-18T00:34:09.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.785+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c8119da0 con 0x7f40c811ad30 2024-02-18T00:34:09.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.786+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c8119da0 con 0x7f40c811ad30 2024-02-18T00:34:09.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40baffd700 10 client.30501.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c811ab90 osd=4 4 2024-02-18T00:34:09.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 4 10 2024-02-18T00:34:09.789 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:34:09.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 10 client.30501.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 11 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6804/2170432263 conn(0x7f40c8121e00 0x7f40c8122260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 11 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 5 11 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8119da0 osd=5 4 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 10 client.30501.objecter _send_command 11 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c8122750 con 0x7f40c8121e00 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cd2fb700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6804/2170432263 conn(0x7f40c8121e00 0x7f40c8122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40cd2fb700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6804/2170432263 conn(0x7f40c8121e00 0x7f40c8122260 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.787+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c8121e00 2024-02-18T00:34:09.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.788+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c8122750 con 0x7f40c8121e00 2024-02-18T00:34:09.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.788+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:34:09.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.788+0000 7f40baffd700 10 client.30501.objecter _finish_command 11 = system:0 2024-02-18T00:34:09.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.788+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8119da0 osd=5 4 2024-02-18T00:34:09.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.788+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 5 11 2024-02-18T00:34:09.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:09.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 10 client.30501.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 12 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 12 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8119da0 osd=5 3 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 5 12 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8119da0 osd=5 4 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 10 client.30501.objecter _send_command 12 2024-02-18T00:34:09.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.803+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c8122750 con 0x7f40c8121e00 2024-02-18T00:34:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c8122750 con 0x7f40c8121e00 2024-02-18T00:34:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40baffd700 10 client.30501.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8119da0 osd=5 4 2024-02-18T00:34:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 5 12 2024-02-18T00:34:09.807 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:34:09.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 10 client.30501.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 13 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6808/3356009390 conn(0x7f40c8127080 0x7f40c8129470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 13 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 6 13 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8122750 osd=6 4 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 10 client.30501.objecter _send_command 13 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c812c180 con 0x7f40c8127080 2024-02-18T00:34:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6808/3356009390 conn(0x7f40c8127080 0x7f40c8129470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c8127080 2024-02-18T00:34:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.805+0000 7f40ce2fd700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6808/3356009390 conn(0x7f40c8127080 0x7f40c8129470 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.806+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c812c180 con 0x7f40c8127080 2024-02-18T00:34:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.806+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:34:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.806+0000 7f40baffd700 10 client.30501.objecter _finish_command 13 = system:0 2024-02-18T00:34:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.806+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8122750 osd=6 4 2024-02-18T00:34:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.806+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 6 13 2024-02-18T00:34:09.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 10 client.30501.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 14 2024-02-18T00:34:09.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 14 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c8122750 osd=6 3 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 6 14 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8122750 osd=6 4 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 10 client.30501.objecter _send_command 14 2024-02-18T00:34:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.821+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c812c180 con 0x7f40c8127080 2024-02-18T00:34:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c812c180 con 0x7f40c8127080 2024-02-18T00:34:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40baffd700 10 client.30501.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c8122750 osd=6 4 2024-02-18T00:34:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 6 14 2024-02-18T00:34:09.825 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 10 client.30501.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 15 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6812/53025171 conn(0x7f40c812b5a0 0x7f40c812ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 15 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 7 15 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c812c180 osd=7 4 2024-02-18T00:34:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 10 client.30501.objecter _send_command 15 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40c812f530 con 0x7f40c812b5a0 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.823+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6812/53025171 conn(0x7f40c812b5a0 0x7f40c812ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.824+0000 7f40baffd700 10 client.30501.objecter ms_handle_connect 0x7f40c812b5a0 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.824+0000 7f40cdafc700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6812/53025171 conn(0x7f40c812b5a0 0x7f40c812ba20 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.824+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f40c812f530 con 0x7f40c812b5a0 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.824+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.824+0000 7f40baffd700 10 client.30501.objecter _finish_command 15 = system:0 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.824+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c812c180 osd=7 4 2024-02-18T00:34:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.824+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 7 15 2024-02-18T00:34:09.830 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 10 client.30501.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign -1 16 2024-02-18T00:34:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 20 client.30501.objecter _get_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_remove -1 16 2024-02-18T00:34:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 20 client.30501.objecter get_session s=0x7f40c812c180 osd=7 3 2024-02-18T00:34:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 15 client.30501.objecter _session_command_op_assign 7 16 2024-02-18T00:34:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c812c180 osd=7 4 2024-02-18T00:34:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 10 client.30501.objecter _send_command 16 2024-02-18T00:34:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.839+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f40c812e3b0 con 0x7f40c812b5a0 2024-02-18T00:34:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40baffd700 1 -- 172.21.15.59:0/3975832200 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f40c812e3b0 con 0x7f40c812b5a0 2024-02-18T00:34:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40baffd700 10 client.30501.objecter ms_dispatch 0x7f40c8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40baffd700 10 client.30501.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.844 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:34:09.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40baffd700 20 client.30501.objecter put_session s=0x7f40c812c180 osd=7 4 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40baffd700 15 client.30501.objecter _session_command_op_remove 7 16 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 librados: watch_flush enter 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 librados: watch_flush exit 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.0 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6801/3700238977 conn(0x7f40c80fc1e0 msgr2=0x7f40c80fc5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6801/3700238977 conn(0x7f40c80fc1e0 0x7f40c80fc5c0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c805ee80 osd=0 1 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.1 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6805/26279110 conn(0x7f40c80598f0 msgr2=0x7f40c8059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6805/26279110 conn(0x7f40c80598f0 0x7f40c8059cf0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8028710 osd=1 1 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.2 2024-02-18T00:34:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6809/1153193637 conn(0x7f40c8028270 msgr2=0x7f40c811a240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6809/1153193637 conn(0x7f40c8028270 0x7f40c811a240 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8059770 osd=2 1 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.3 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6813/1805700964 conn(0x7f40c811d9a0 msgr2=0x7f40c811fd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6813/1805700964 conn(0x7f40c811d9a0 0x7f40c811fd90 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811b120 osd=3 1 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.4 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6800/2044387056 conn(0x7f40c811ad30 msgr2=0x7f40c8121400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6800/2044387056 conn(0x7f40c811ad30 0x7f40c8121400 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40ce2fd700 1 -- 172.21.15.59:0/3975832200 reap_dead start 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c811ab90 osd=4 1 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.5 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6804/2170432263 conn(0x7f40c8121e00 msgr2=0x7f40c8122260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6804/2170432263 conn(0x7f40c8121e00 0x7f40c8122260 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40ce2fd700 1 -- 172.21.15.59:0/3975832200 reap_dead start 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40ce2fd700 1 -- 172.21.15.59:0/3975832200 reap_dead start 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8119da0 osd=5 1 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.6 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6808/3356009390 conn(0x7f40c8127080 msgr2=0x7f40c8129470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6808/3356009390 conn(0x7f40c8127080 0x7f40c8129470 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c8122750 osd=6 1 2024-02-18T00:34:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 10 client.30501.objecter close_session for osd.7 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6812/53025171 conn(0x7f40c812b5a0 msgr2=0x7f40c812ba20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.137:6812/53025171 conn(0x7f40c812b5a0 0x7f40c812ba20 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter put_session s=0x7f40c812c180 osd=7 1 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.841+0000 7f40cfd60700 20 client.30501.objecter shutdown clearing up homeless session... 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 10 client.30501.objecter successfully canceled tick 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40b4062c90 msgr2=0x7f40b4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> v2:172.21.15.59:6800/3721891797 conn(0x7f40b4062c90 0x7f40b4065150 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 rx=0x7f40bc009a00 tx=0x7f40bc00b040).stop 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f8cb0 msgr2=0x7f40c810e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 --2- 172.21.15.59:0/3975832200 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40c80f8cb0 0x7f40c810e1b0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 rx=0x7f40c40027f0 tx=0x7f40c4002b00).stop 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40ce2fd700 1 -- 172.21.15.59:0/3975832200 reap_dead start 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 shutdown_connections 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 >> 172.21.15.59:0/3975832200 conn(0x7f40c80f2820 msgr2=0x7f40c80f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 shutdown_connections 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 -- 172.21.15.59:0/3975832200 wait complete. 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:09.842+0000 7f40cfd60700 1 librados: shutdown 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:34:09.954 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:07.316436+0000 osd.6 (osd.6) 89 : cluster [DBG] 97.4 scrub starts 2024-02-18T00:34:09.954 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:07.318577+0000 osd.6 (osd.6) 90 : cluster [DBG] 97.4 scrub ok 2024-02-18T00:34:09.954 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:07.826880+0000 mgr.y (mgr.24454) 226 : cluster [DBG] pgmap v241: 333 pgs: 32 unknown, 301 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail; 36 KiB/s rd, 46 op/s 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.285556+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.59:0/2900799437' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.535797+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.599217+0000 mon.a (mon.0) 1275 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.599340+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82981-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.599613+0000 mon.a (mon.0) 1277 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".rgw.root", "var": "pgp_num", "val": "42"}]': finished 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:08.599691+0000 mon.a (mon.0) 1278 : cluster [DBG] osdmap e170: 8 total, 8 up, 7 in 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.612814+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? 172.21.15.59:0/1213186724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.613876+0000 mon.b (mon.1) 207 : audit [INF] from='client.? 172.21.15.59:0/147524826' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.614751+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.629501+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.59:0/1235599235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:09.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:08.630466+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:10.191 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24865.0:2 29.1a 29:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.365742+0000", 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 68.822186754000001, 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1529440639999999, 2024-02-18T00:34:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24865", 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2678422317", 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365742+0000", 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365742+0000", 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365740+0000", 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9699999999999994e-06 2024-02-18T00:34:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365750+0000", 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4426e-05 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365774+0000", 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.149e-06 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365783+0000", 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0027213300000000001 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.368505+0000", 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4719999999999995e-06 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:34:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.368513+0000", 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1456162480000001 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.514130+0000", 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012966900000000001 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.514259+0000", 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014994800000000001 2024-02-18T00:34:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.514409+0000", 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016852989999999999 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.516094+0000", 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002244242 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518339+0000", 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00025869000000000001 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518597+0000", 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8600999999999998e-05 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518646+0000", 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0485000000000003e-05 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518686+0000", 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1529440639999999 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24805.0:2 13.18 13:1e6cc557:test-rados-api-smithi059-82957-2::foo:head [undirty] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.401592+0000", 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 68.786337001999996, 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.17911491299999999, 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24805", 2024-02-18T00:34:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1794179512", 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401592+0000", 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401592+0000", 2024-02-18T00:34:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401590+0000", 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0469999999999999e-06 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401597+0000", 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9949999999999994e-06 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401606+0000", 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2190000000000001e-06 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401612+0000", 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001212917 2024-02-18T00:34:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.402825+0000", 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8909999999999995e-06 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.402833+0000", 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.060490740000000001 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.463323+0000", 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012003899999999999 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.463443+0000", 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11726376199999999 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.580707+0000", 2024-02-18T00:34:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.17911491299999999 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24829.0:2 15.f e106/104 15:f318d819:test-rados-api-smithi059-82989-2::foo:head v 106'1, mlcod=0'0)", 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 68.754181111999998, 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011160974000000001, 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4830000000000005e-06 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433757+0000", 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1699999999999997e-06 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433765+0000", 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.888e-06 2024-02-18T00:34:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433772+0000", 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016121700000000001 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433933+0000", 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6643e-05 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433950+0000", 2024-02-18T00:34:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010933965 2024-02-18T00:34:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.444884+0000", 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5137e-05 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.444909+0000", 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011160974000000001 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24805.0:3 13.18 13:1e6cc557:test-rados-api-smithi059-82957-2::foo:head [create] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.582528+0000", 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 68.605401618000002, 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011408494, 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24805", 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1794179512", 2024-02-18T00:34:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 3 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582528+0000", 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582528+0000", 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:34:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582526+0000", 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5890000000000002e-06 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582532+0000", 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0080000000000008e-06 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582541+0000", 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4349999999999999e-06 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582545+0000", 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6727e-05 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582572+0000", 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9898999999999998e-05 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582622+0000", 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8269000000000005e-05 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,4", 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582710+0000", 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030607300000000002 2024-02-18T00:34:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.583016+0000", 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000163521 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.583180+0000", 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010674022999999999 2024-02-18T00:34:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.593854+0000", 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7469999999999998e-05 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.593901+0000", 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4650000000000002e-05 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.593936+0000", 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011408494 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24766.0:23 12.d 12:b6d4da3a:test-rados-api-smithi059-82901-8::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.619767+0000", 2024-02-18T00:34:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 68.568162032000004, 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0068538059999999996, 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24766", 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1346273899", 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 23 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619767+0000", 2024-02-18T00:34:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619767+0000", 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619766+0000", 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5389999999999996e-06 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619776+0000", 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4699999999999999e-06 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619782+0000", 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.709e-06 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619786+0000", 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5612999999999999e-05 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619822+0000", 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8061999999999997e-05 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619920+0000", 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013012500000000001 2024-02-18T00:34:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.620050+0000", 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00036723700000000002 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.620417+0000", 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00046972000000000002 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.620887+0000", 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0056526720000000001 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.626539+0000", 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4203999999999999e-05 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.626584+0000", 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7239e-05 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.626621+0000", 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0068538059999999996 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25003.0:1 22.1a e107/105 22:5c6776db:::0.obj:head v 107'5, mlcod=107'3)", 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.436676+0000", 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 66.751253452, 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012922347000000001, 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436676+0000", 2024-02-18T00:34:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436676+0000", 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436675+0000", 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0831e-05 2024-02-18T00:34:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436685+0000", 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0688e-05 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436696+0000", 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6769999999999995e-06 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436704+0000", 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0079016120000000006 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.444605+0000", 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4494999999999998e-05 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.444630+0000", 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049239829999999998 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.449554+0000", 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4261000000000002e-05 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.449598+0000", 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012922347000000001 2024-02-18T00:34:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:3 22.d 22:b27a9ef8:::17.obj2:head [write 0~255] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.054788+0000", 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 66.133141197, 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0077752630000000001, 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 3 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054788+0000", 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054788+0000", 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054787+0000", 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8359999999999992e-06 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054794+0000", 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3886999999999999e-05 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054808+0000", 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3310000000000004e-06 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054817+0000", 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7200999999999995e-05 2024-02-18T00:34:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054894+0000", 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020316 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.055097+0000", 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026296499999999999 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 4,6", 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.055360+0000", 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0030637559999999999 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.058424+0000", 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00054202500000000002 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.058966+0000", 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003509968 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062476+0000", 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0881999999999999e-05 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062517+0000", 2024-02-18T00:34:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6555999999999999e-05 2024-02-18T00:34:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062563+0000", 2024-02-18T00:34:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0077752630000000001 2024-02-18T00:34:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:6 22.2 e108/105 22:47a9abb2:::36.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.068716+0000", 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 66.119213412999997, 2024-02-18T00:34:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034977148999999999, 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068716+0000", 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068716+0000", 2024-02-18T00:34:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068714+0000", 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.577e-05 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068730+0000", 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3647e-05 2024-02-18T00:34:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068744+0000", 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8710000000000002e-06 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068750+0000", 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5821e-05 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068805+0000", 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1854000000000001e-05 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068827+0000", 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034833981999999999 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103661+0000", 2024-02-18T00:34:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1584000000000001e-05 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103693+0000", 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034977148999999999 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2070 17.15 e108/104 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head v 108'1, mlcod=0'0)", 2024-02-18T00:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.136909+0000", 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 66.051020144000006, 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.066111226999999995, 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136909+0000", 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136909+0000", 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136907+0000", 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017153972999999999 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154061+0000", 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6050999999999999e-05 2024-02-18T00:34:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154077+0000", 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4019999999999996e-06 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154084+0000", 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2176999999999998e-05 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154147+0000", 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.448e-05 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154161+0000", 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.048825701999999999 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.202987+0000", 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3439000000000003e-05 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.203020+0000", 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.066111226999999995 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:20 22.1 e108/105 22:83282cb0:::1.obj2:head v 108'14, mlcod=108'12)", 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.158484+0000", 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 66.029445238999998, 2024-02-18T00:34:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042476860999999998, 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158484+0000", 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158484+0000", 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:34:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158483+0000", 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3191e-05 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158496+0000", 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6535999999999999e-05 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158512+0000", 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8220000000000002e-06 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158522+0000", 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0906999999999999e-05 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158573+0000", 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0628e-05 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158594+0000", 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042318445000000003 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.200912+0000", 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8708999999999997e-05 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.200961+0000", 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042476860999999998 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2073 17.15 e108/104 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head v 108'4, mlcod=108'2)", 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.971052720000003, 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0089590280000000008, 2024-02-18T00:34:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:34:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00083967099999999995 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217715+0000", 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9789e-05 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217735+0000", 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0609999999999998e-06 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217742+0000", 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2679000000000002e-05 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217795+0000", 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2875e-05 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217818+0000", 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0079678040000000002 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225786+0000", 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9818999999999999e-05 2024-02-18T00:34:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225835+0000", 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0089590280000000008 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:32 22.8 22:12e893d3:::7.obj2:head [write 0~135] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.219993+0000", 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.967935994000001, 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.006045323, 2024-02-18T00:34:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 32 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.219993+0000", 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.219993+0000", 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.219991+0000", 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0264e-05 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220001+0000", 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6555999999999999e-05 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220018+0000", 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0730000000000001e-05 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220028+0000", 2024-02-18T00:34:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9993000000000001e-05 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220068+0000", 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010894 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220177+0000", 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000111055 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,4", 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220288+0000", 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074838599999999997 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.221037+0000", 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000118282 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.221155+0000", 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047722520000000003 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225927+0000", 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8940000000000002e-05 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225986+0000", 2024-02-18T00:34:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2339000000000002e-05 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.226038+0000", 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.006045323 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:40 22.c 22:33eb54d8:::39.obj2:head [write 0~150] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:34:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.268821+0000", 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.919108182000002, 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.033196955, 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 40 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268821+0000", 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268821+0000", 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:34:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268820+0000", 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.49e-06 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268825+0000", 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0638999999999999e-05 2024-02-18T00:34:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268836+0000", 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1050000000000003e-06 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268841+0000", 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5590999999999999e-05 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268877+0000", 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1233000000000001e-05 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268948+0000", 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4674999999999997e-05 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,3", 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.269043+0000", 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029585499999999998 2024-02-18T00:34:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.269338+0000", 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.030906822 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.300245+0000", 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001703366 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.301949+0000", 2024-02-18T00:34:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7935000000000002e-05 2024-02-18T00:34:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.301987+0000", 2024-02-18T00:34:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1309999999999997e-05 2024-02-18T00:34:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302018+0000", 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.033196955 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:41 22.e e108/105 22:7639cdab:::46.obj2:head v 108'9, mlcod=108'7)", 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.302685+0000", 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.885244184000001, 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018774604, 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302685+0000", 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302685+0000", 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302684+0000", 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0800000000000006e-06 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302692+0000", 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4999999999999999e-06 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302701+0000", 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8539999999999999e-06 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302705+0000", 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1548000000000003e-05 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302746+0000", 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3828e-05 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302770+0000", 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018660823 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321431+0000", 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8781000000000001e-05 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321460+0000", 2024-02-18T00:34:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018774604 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25052.0:14 22.a 22:54367c00:::22.obj:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109)", 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.891467+0000", 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.296462278000007, 2024-02-18T00:34:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0070211850000000001, 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25052", 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1321287304", 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 14 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891467+0000", 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891467+0000", 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891466+0000", 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3839999999999999e-06 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891471+0000", 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2650000000000002e-06 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891480+0000", 2024-02-18T00:34:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4100000000000001e-06 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891484+0000", 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9938999999999999e-05 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891514+0000", 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0243e-05 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891564+0000", 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7618000000000002e-05 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891632+0000", 2024-02-18T00:34:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00022058700000000001 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891853+0000", 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00024304200000000001 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.892096+0000", 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0063404799999999999 2024-02-18T00:34:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.898436+0000", 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7296000000000001e-05 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.898463+0000", 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4664999999999998e-05 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.898488+0000", 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0070211850000000001 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25073.0:3 37.f 37:f4b50b61:::1.obj:head [write 0~198] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:07.512122+0000", 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.675807223, 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020910159000000001, 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25073", 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1549676353", 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 3 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512122+0000", 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512122+0000", 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512121+0000", 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3009999999999997e-06 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512126+0000", 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6290000000000001e-06 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512134+0000", 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0050000000000001e-06 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512140+0000", 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5095000000000002e-05 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512175+0000", 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000106903 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512282+0000", 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015298800000000001 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512435+0000", 2024-02-18T00:34:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00044063199999999999 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512875+0000", 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00018640599999999999 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.513062+0000", 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019895076000000001 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.532957+0000", 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7423999999999999e-05 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.532994+0000", 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7999e-05 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.533032+0000", 2024-02-18T00:34:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020910159000000001 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.15264.0:19 60.10 60:0b42fc87:test-rados-api-smithi059-82973-8::foo:head [delete] snapc 7=[7,6] ondisk+write+peerstat_old+known_if_redirected e130)", 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.023202+0000", 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 42.164727454999998, 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.043374965000000001, 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.15264", 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/310625823", 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 19 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023202+0000", 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023202+0000", 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023200+0000", 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6118999999999999e-05 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023216+0000", 2024-02-18T00:34:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.452e-05 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023230+0000", 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9570000000000008e-06 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023239+0000", 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5216999999999997e-05 2024-02-18T00:34:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023294+0000", 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2409999999999995e-06 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023303+0000", 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026498173 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.049801+0000", 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8725000000000002e-05 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.049890+0000", 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000145439 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.050035+0000", 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001137379 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.051172+0000", 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029129900000000002 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.051464+0000", 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015008858999999999 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.066473+0000", 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8291999999999997e-05 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.066531+0000", 2024-02-18T00:34:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5847999999999999e-05 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.066577+0000", 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.043374965000000001 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30748.0:1 78.4s0 78:3cba11e7:test-rados-api-smithi059-82973-10::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:32.611721+0000", 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.576208538000003, 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6967851380000001, 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30748", 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/510272679", 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:34:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611721+0000", 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611721+0000", 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611719+0000", 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.153e-05 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611731+0000", 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1849999999999998e-06 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611738+0000", 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6439999999999997e-06 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611744+0000", 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4613000000000002e-05 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611778+0000", 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.396e-06 2024-02-18T00:34:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611781+0000", 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6366086819999999 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.248389+0000", 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012518499999999999 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.248515+0000", 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030601100000000002 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.248821+0000", 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047360799999999998 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:34:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.249294+0000", 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.059161694000000001 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.308456+0000", 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9768999999999997e-05 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.308506+0000", 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6967851380000001 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2079 17.c 17:36e2cd47:test-rados-api-smithi059-82949-17::big2.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e136)", 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.511355+0000", 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 35.676574510999998, 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.14033522800000001, 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2079 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511355+0000", 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511355+0000", 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511353+0000", 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3270000000000003e-06 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511360+0000", 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2292e-05 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511373+0000", 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7450000000000001e-06 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511378+0000", 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0411999999999999e-05 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511419+0000", 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010539099999999999 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511524+0000", 2024-02-18T00:34:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.069820673999999999 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,5", 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.581345+0000", 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0099325850000000007 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.591278+0000", 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056372900000000001 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.591841+0000", 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.059783511999999997 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.651625+0000", 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7512999999999998e-05 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.651662+0000", 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7617999999999999e-05 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.651690+0000", 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.14033522800000001 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.324 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.33712.0:7 82.c 82:3197012c:::foo_rmxattr:head [setxattr attr2 (12) in=5b] snapc 0=[] ondisk+write+known_if_redirected e138)", 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:36.773678+0000", 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 33.414251047999997, 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058598080000000002, 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.33712", 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3967728574", 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 7 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:34:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773678+0000", 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773678+0000", 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773677+0000", 2024-02-18T00:34:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9460000000000001e-06 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773684+0000", 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0764000000000001e-05 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773695+0000", 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9930000000000003e-06 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773700+0000", 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1032000000000002e-05 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773751+0000", 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4580000000000007e-05 2024-02-18T00:34:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773835+0000", 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014667200000000001 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,4", 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773982+0000", 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00032207600000000002 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:34:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.774304+0000", 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017701699999999999 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.774481+0000", 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049669709999999997 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.779448+0000", 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9666000000000002e-05 2024-02-18T00:34:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.779498+0000", 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0355999999999998e-05 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.779538+0000", 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058598080000000002 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:34:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:34:10.332 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:34:10.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:10.333 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:10.333 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:10.434 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:10.463 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:10.634 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-18T00:34:10.634 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: seed 82949 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (8 ms) 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (231 ms) 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-18T00:34:10.635 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (8 ms) 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (12 ms) 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (25 ms) 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (5 ms) 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (6 ms) 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (6 ms) 2024-02-18T00:34:10.636 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (15 ms) 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (4 ms) 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1939 ms) 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-18T00:34:10.637 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (364 ms) 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: done waiting, doing copies 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: done waiting 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60817 ms) 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (11 ms) 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (14 ms) 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4761 ms) 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-18T00:34:10.638 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-18T00:34:10.639 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-18T00:34:10.639 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-18T00:34:10.639 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-18T00:34:10.639 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-18T00:34:10.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:09.599799+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? 172.21.15.59:0/1213186724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:09.599875+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]': finished 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:09.600012+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:09.600095+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e171: 8 total, 8 up, 7 in 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:09.695085+0000 mon.a (mon.0) 1286 : audit [DBG] from='client.? 172.21.15.59:0/3975832200' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:09.827412+0000 mgr.y (mgr.24454) 227 : cluster [DBG] pgmap v244: 396 pgs: 96 unknown, 300 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:09.828948+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:09.887792+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:10.025502+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:10.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:09.599799+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? 172.21.15.59:0/1213186724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:09.599875+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]': finished 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:09.600012+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:09.600095+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e171: 8 total, 8 up, 7 in 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:09.695085+0000 mon.a (mon.0) 1286 : audit [DBG] from='client.? 172.21.15.59:0/3975832200' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:09.827412+0000 mgr.y (mgr.24454) 227 : cluster [DBG] pgmap v244: 396 pgs: 96 unknown, 300 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:09.828948+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:09.887792+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:10.025502+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:09.599799+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? 172.21.15.59:0/1213186724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:10.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:09.599875+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi059-82949-1","app":"app1","key":"key1"}]': finished 2024-02-18T00:34:10.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:09.600012+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi059-82877-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:10.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:09.600095+0000 mon.a (mon.0) 1285 : cluster [DBG] osdmap e171: 8 total, 8 up, 7 in 2024-02-18T00:34:10.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:09.695085+0000 mon.a (mon.0) 1286 : audit [DBG] from='client.? 172.21.15.59:0/3975832200' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:10.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:09.827412+0000 mgr.y (mgr.24454) 227 : cluster [DBG] pgmap v244: 396 pgs: 96 unknown, 300 active+clean; 216 MiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:10.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:09.828948+0000 mon.a (mon.0) 1287 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-18T00:34:10.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:09.887792+0000 mon.a (mon.0) 1288 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:10.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:10.025502+0000 mon.a (mon.0) 1289 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:10.912 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:10.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:10.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:10.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-18T00:34:11.017 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:11.048 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:11.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:34:11.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:34:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:34:11.398 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:11.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:11.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:11.503 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:11.532 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:11.550 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:11.552 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:11.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:11.654 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:11.699 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:11.702 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:11.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:10.454102+0000 osd.7 (osd.7) 97 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:10.454160+0000 osd.7 (osd.7) 98 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:10.605565+0000 mon.a (mon.0) 1290 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-18T00:34:11.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:10.605633+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e172: 8 total, 8 up, 7 in 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:10.671185+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:10.992965+0000 mgr.y (mgr.24454) 228 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.538447+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.539373+0000 mon.b (mon.1) 209 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.539596+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.540133+0000 mon.b (mon.1) 210 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.540296+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:10.454102+0000 osd.7 (osd.7) 97 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:10.454160+0000 osd.7 (osd.7) 98 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:10.605565+0000 mon.a (mon.0) 1290 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-18T00:34:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:10.605633+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e172: 8 total, 8 up, 7 in 2024-02-18T00:34:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:10.671185+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:10.992965+0000 mgr.y (mgr.24454) 228 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.538447+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.539373+0000 mon.b (mon.1) 209 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.539596+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.540133+0000 mon.b (mon.1) 210 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:11.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.540296+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:11.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:11.831 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:11.952 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:11.956 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:11.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:11.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:10.454102+0000 osd.7 (osd.7) 97 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:10.454160+0000 osd.7 (osd.7) 98 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:10.605565+0000 mon.a (mon.0) 1290 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-18T00:34:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:10.605633+0000 mon.a (mon.0) 1291 : cluster [DBG] osdmap e172: 8 total, 8 up, 7 in 2024-02-18T00:34:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:10.671185+0000 mon.b (mon.1) 208 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:10.992965+0000 mgr.y (mgr.24454) 228 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.538447+0000 mon.a (mon.0) 1292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.539373+0000 mon.b (mon.1) 209 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.539596+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.540133+0000 mon.b (mon.1) 210 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.540296+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.061 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:12.107 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:12.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:12.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:12.211 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:12.237 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:12.255 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:12.258 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:12.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:12.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:12.386 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:12.405 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:12.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:12.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:12.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:12] "GET /metrics HTTP/1.1" 200 198420 "" "Prometheus/2.33.4" 2024-02-18T00:34:12.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:12.536 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:12.553 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:12.555 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:12.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:12.583 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:34:12.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:12.690 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:12.714 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:12.719 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:12.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.614513+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:11.614604+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e173: 8 total, 8 up, 7 in 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.616475+0000 mon.b (mon.1) 211 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.617296+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.618699+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.59:0/169508354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.627210+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.640198+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.59:0/1838859307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.640834+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.671521+0000 mon.b (mon.1) 213 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.671700+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.672780+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.672885+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.673394+0000 mon.b (mon.1) 215 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:11.673501+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:11.827848+0000 mgr.y (mgr.24454) 229 : cluster [DBG] pgmap v247: 332 pgs: 2 peering, 64 unknown, 266 active+clean; 15 KiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:12.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:12.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.614513+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:12.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:11.614604+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e173: 8 total, 8 up, 7 in 2024-02-18T00:34:12.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.616475+0000 mon.b (mon.1) 211 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.848 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.617296+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.618699+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.59:0/169508354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.627210+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.640198+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.59:0/1838859307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.640834+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.671521+0000 mon.b (mon.1) 213 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.671700+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.672780+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.672885+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.673394+0000 mon.b (mon.1) 215 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:11.673501+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.849 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:11.827848+0000 mgr.y (mgr.24454) 229 : cluster [DBG] pgmap v247: 332 pgs: 2 peering, 64 unknown, 266 active+clean; 15 KiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.614513+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82949-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:11.614604+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e173: 8 total, 8 up, 7 in 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.616475+0000 mon.b (mon.1) 211 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.617296+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.618699+0000 mon.b (mon.1) 212 : audit [INF] from='client.? 172.21.15.59:0/169508354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.627210+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.640198+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.59:0/1838859307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.640834+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.671521+0000 mon.b (mon.1) 213 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.671700+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.672780+0000 mon.b (mon.1) 214 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.672885+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:12.851 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.673394+0000 mon.b (mon.1) 215 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.851 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:11.673501+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:12.851 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:11.827848+0000 mgr.y (mgr.24454) 229 : cluster [DBG] pgmap v247: 332 pgs: 2 peering, 64 unknown, 266 active+clean; 15 KiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:13.051 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:13.056 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:13.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:13.157 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:13.185 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:13.203 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:13.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:13.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:13.307 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 -- 172.21.15.59:0/73924888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c0f8cb0 msgr2=0x7f8a8c0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 --2- 172.21.15.59:0/73924888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0f9090 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 rx=0x7f8a7c0047b0 tx=0x7f8a7c009cb0).stop 2024-02-18T00:34:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 -- 172.21.15.59:0/73924888 shutdown_connections 2024-02-18T00:34:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 --2- 172.21.15.59:0/73924888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c102210 0x7f8a8c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 --2- 172.21.15.59:0/73924888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0f9090 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 --2- 172.21.15.59:0/73924888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a8c0f9610 0x7f8a8c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 -- 172.21.15.59:0/73924888 >> 172.21.15.59:0/73924888 conn(0x7f8a8c0f2820 msgr2=0x7f8a8c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 -- 172.21.15.59:0/73924888 shutdown_connections 2024-02-18T00:34:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.305+0000 7f8a922f5700 1 -- 172.21.15.59:0/73924888 wait complete. 2024-02-18T00:34:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 librados: starting msgr at 2024-02-18T00:34:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 librados: starting objecter 2024-02-18T00:34:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 Processor -- start 2024-02-18T00:34:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 -- start start 2024-02-18T00:34:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 librados: setting wanted keys 2024-02-18T00:34:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 librados: calling monclient init 2024-02-18T00:34:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0633a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a8c0f9610 0x7f8a8c063890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c102210 0x7f8a8c066c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c067150 con 0x7f8a8c102210 2024-02-18T00:34:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0672c0 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.306+0000 7f8a922f5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c067480 con 0x7f8a8c0f9610 2024-02-18T00:34:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a90892700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c102210 0x7f8a8c066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0633a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:47246/0 (socket says 172.21.15.59:47246) 2024-02-18T00:34:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a90892700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c102210 0x7f8a8c066c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57738/0 (socket says 172.21.15.59:57738) 2024-02-18T00:34:13.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 -- 172.21.15.59:0/508947691 learned_addr learned my addr 172.21.15.59:0/508947691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:13.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8b7fe700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a8c0f9610 0x7f8a8c063890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:13.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 -- 172.21.15.59:0/508947691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a8c0f9610 msgr2=0x7f8a8c063890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:13.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a8c0f9610 0x7f8a8c063890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:13.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 -- 172.21.15.59:0/508947691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c102210 msgr2=0x7f8a8c066c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:13.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c102210 0x7f8a8c066c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:13.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 -- 172.21.15.59:0/508947691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c005040 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a8bfff700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0633a0 secure :-1 s=READY pgs=4037 cs=0 l=1 rev1=1 rx=0x7f8a7c009b10 tx=0x7f8a7c009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:13.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a897fa700 10 client.?.objecter ms_handle_connect 0x7f8a8c0f8cb0 2024-02-18T00:34:13.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a897fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:13.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a897fa700 1 -- 172.21.15.59:0/508947691 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c018070 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c067700 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a897fa700 1 -- 172.21.15.59:0/508947691 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a7c013da0 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a897fa700 1 -- 172.21.15.59:0/508947691 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c01dcc0 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a922f5700 10 client.42922.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c067c90 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.307+0000 7f8a922f5700 1 librados: init done 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c05fa60 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a897fa700 1 -- 172.21.15.59:0/508947691 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8a7c01c5d0 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a897fa700 1 --2- 172.21.15.59:0/508947691 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8a78062c90 0x7f8a78065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a897fa700 1 -- 172.21.15.59:0/508947691 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(174..174 src has 44..174) v4 ==== 9401+0+0 (secure 0 0 0) 0x7f8a7c07ed10 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a897fa700 10 client.42922.objecter ms_dispatch 0x7f8a8c001550 osd_map(174..174 src has 44..174) v4 2024-02-18T00:34:13.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a897fa700 3 client.42922.objecter handle_osd_map got epochs [174,174] > 0 2024-02-18T00:34:13.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a897fa700 3 client.42922.objecter handle_osd_map decoding full epoch 174 2024-02-18T00:34:13.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a8b7fe700 1 --2- 172.21.15.59:0/508947691 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8a78062c90 0x7f8a78065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:13.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.309+0000 7f8a897fa700 20 client.42922.objecter dump_active .. 0 homeless 2024-02-18T00:34:13.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.310+0000 7f8a897fa700 10 client.42922.objecter ms_handle_connect 0x7f8a78062c90 2024-02-18T00:34:13.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.310+0000 7f8a8b7fe700 1 --2- 172.21.15.59:0/508947691 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8a78062c90 0x7f8a78065150 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 rx=0x7f8a8c064980 tx=0x7f8a7400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:13.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.314+0000 7f8a897fa700 1 -- 172.21.15.59:0/508947691 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8a7c051df0 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.331 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:13.352 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:13.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:13.354 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:13.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:13.482 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:13.500 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:13.503 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:13.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:13.507+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f8a8c05ee80 con 0x7f8a8c0f8cb0 2024-02-18T00:34:13.605 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:13.631 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:13.652 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:13.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:13.655 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:13.756 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:13.782 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:13.800 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:13.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:13.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:13.904 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:13.931 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:13.949 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:13.951 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:13.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:14.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.028+0000 7f8a897fa700 1 -- 172.21.15.59:0/508947691 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v175) v1 ==== 91+0+0 (secure 0 0 0) 0x7f8a7c0519e0 con 0x7f8a8c0f8cb0 2024-02-18T00:34:14.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.031+0000 7f8a922f5700 10 librados: watch_flush enter 2024-02-18T00:34:14.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.031+0000 7f8a922f5700 10 librados: watch_flush exit 2024-02-18T00:34:14.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.031+0000 7f8a922f5700 20 client.42922.objecter shutdown clearing up homeless session... 2024-02-18T00:34:14.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.031+0000 7f8a922f5700 10 client.42922.objecter successfully canceled tick 2024-02-18T00:34:14.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8a78062c90 msgr2=0x7f8a78065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:14.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 --2- 172.21.15.59:0/508947691 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8a78062c90 0x7f8a78065150 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 rx=0x7f8a8c064980 tx=0x7f8a7400b040).stop 2024-02-18T00:34:14.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c0f8cb0 msgr2=0x7f8a8c0633a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:14.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0633a0 secure :-1 s=READY pgs=4037 cs=0 l=1 rev1=1 rx=0x7f8a7c009b10 tx=0x7f8a7c009cb0).stop 2024-02-18T00:34:14.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 shutdown_connections 2024-02-18T00:34:14.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 --2- 172.21.15.59:0/508947691 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8a78062c90 0x7f8a78065150 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a8c0f8cb0 0x7f8a8c0633a0 unknown :-1 s=CLOSED pgs=4037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a8c102210 0x7f8a8c066c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 --2- 172.21.15.59:0/508947691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a8c0f9610 0x7f8a8c063890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 >> 172.21.15.59:0/508947691 conn(0x7f8a8c0f2820 msgr2=0x7f8a8c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 shutdown_connections 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 -- 172.21.15.59:0/508947691 wait complete. 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:14.034+0000 7f8a922f5700 1 librados: shutdown 2024-02-18T00:34:14.037 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:34:14.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:14.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.079867+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:14.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.080070+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:14.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.080211+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:14.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:13.080311+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e174: 8 total, 8 up, 7 in 2024-02-18T00:34:14.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.080897+0000 mon.b (mon.1) 216 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:14.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.085642+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.509648+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.59:0/508947691' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.510078+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:13.578266+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.079867+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.080070+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.080211+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:13.080311+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e174: 8 total, 8 up, 7 in 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.080897+0000 mon.b (mon.1) 216 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.085642+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.509648+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.59:0/508947691' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:34:14.059 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.510078+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:34:14.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:13.578266+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:14.474 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:34:14.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 0 1 2024-02-18T00:34:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.079867+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi059-82877-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.080070+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi059-82882-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.080211+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:14.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:13.080311+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e174: 8 total, 8 up, 7 in 2024-02-18T00:34:14.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.080897+0000 mon.b (mon.1) 216 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:14.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.085642+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:14.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.509648+0000 mon.b (mon.1) 217 : audit [INF] from='client.? 172.21.15.59:0/508947691' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:34:14.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.510078+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:34:14.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:13.578266+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:14.541 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:14.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:14.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:14.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:14.680 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:14.805 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:14.808 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:14.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:14.909 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:14.968 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:14.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:15.069 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:15.116 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:15.119 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:15.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:15.220 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:15.246 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:15.267 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:15.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:15.368 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:15.395 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:15.413 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:15.416 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:15.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.586160+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:13.667268+0000 mon.b (mon.1) 218 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:13.828536+0000 mgr.y (mgr.24454) 230 : cluster [DBG] pgmap v250: 275 pgs: 1 peering, 8 unknown, 266 active+clean; 15 KiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:14.029387+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:34:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:14.029447+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e175: 8 total, 8 up, 7 in 2024-02-18T00:34:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:14.037628+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-18T00:34:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:14.668158+0000 mon.b (mon.1) 219 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:15.517 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:15.544 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:15.562 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:15.565 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:15.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:15.666 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:15.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.586160+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:13.667268+0000 mon.b (mon.1) 218 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:13.828536+0000 mgr.y (mgr.24454) 230 : cluster [DBG] pgmap v250: 275 pgs: 1 peering, 8 unknown, 266 active+clean; 15 KiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:14.029387+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:14.029447+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e175: 8 total, 8 up, 7 in 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:14.037628+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:14.668158+0000 mon.b (mon.1) 219 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.586160+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82949-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:13.667268+0000 mon.b (mon.1) 218 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:13.828536+0000 mgr.y (mgr.24454) 230 : cluster [DBG] pgmap v250: 275 pgs: 1 peering, 8 unknown, 266 active+clean; 15 KiB data, 2.7 GiB used, 623 GiB / 626 GiB avail 2024-02-18T00:34:15.693 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:14.029387+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:34:15.694 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:14.029447+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e175: 8 total, 8 up, 7 in 2024-02-18T00:34:15.694 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:14.037628+0000 mon.a (mon.0) 1313 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-18T00:34:15.694 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:14.668158+0000 mon.b (mon.1) 219 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:15.695 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:16.145 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:16.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:16.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:16.248 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:16.270 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:34:16.270 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:34:16.270 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:34:16.271 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:34:16.276 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:16.295 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:16.297 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:16.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:16.399 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:16.425 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:16.443 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:16.446 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:16.446 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:16.547 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:16.572 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:14.636184+0000 osd.3 (osd.3) 115 : cluster [DBG] 97.5 scrub starts 2024-02-18T00:34:16.572 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:14.638784+0000 osd.3 (osd.3) 116 : cluster [DBG] 97.5 scrub ok 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:14.941494+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:14.941696+0000 mon.a (mon.0) 1315 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:14.941843+0000 mon.a (mon.0) 1316 : cluster [DBG] osdmap e176: 8 total, 8 up, 7 in 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:14.947398+0000 mon.b (mon.1) 220 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:14.949051+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.59:0/167636954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.044902+0000 osd.5 (osd.5) 87 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.045672+0000 osd.5 (osd.5) 88 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.230017+0000 osd.6 (osd.6) 91 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.230933+0000 osd.6 (osd.6) 92 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.391542+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.391902+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.573 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.669082+0000 mon.b (mon.1) 221 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.829706+0000 mgr.y (mgr.24454) 231 : cluster [DBG] pgmap v252: 347 pgs: 72 creating+peering, 2 peering, 273 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.931949+0000 osd.0 (osd.0) 99 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.934966+0000 osd.0 (osd.0) 100 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.944311+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.944429+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:15.944511+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e177: 8 total, 8 up, 7 in 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.950700+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.59:0/481047181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi059-82882-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.956571+0000 mon.b (mon.1) 222 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:15.957005+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.575 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:16.592 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:16.595 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:16.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:16.696 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:16.723 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:16.741 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:16.744 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:16.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:14.636184+0000 osd.3 (osd.3) 115 : cluster [DBG] 97.5 scrub starts 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:14.638784+0000 osd.3 (osd.3) 116 : cluster [DBG] 97.5 scrub ok 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:14.941494+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:14.941696+0000 mon.a (mon.0) 1315 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:14.941843+0000 mon.a (mon.0) 1316 : cluster [DBG] osdmap e176: 8 total, 8 up, 7 in 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:14.947398+0000 mon.b (mon.1) 220 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:14.949051+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.59:0/167636954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.044902+0000 osd.5 (osd.5) 87 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:34:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.045672+0000 osd.5 (osd.5) 88 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.230017+0000 osd.6 (osd.6) 91 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.230933+0000 osd.6 (osd.6) 92 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.391542+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.391902+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.669082+0000 mon.b (mon.1) 221 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.829706+0000 mgr.y (mgr.24454) 231 : cluster [DBG] pgmap v252: 347 pgs: 72 creating+peering, 2 peering, 273 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.931949+0000 osd.0 (osd.0) 99 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.934966+0000 osd.0 (osd.0) 100 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.944311+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.944429+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:15.944511+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e177: 8 total, 8 up, 7 in 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.950700+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.59:0/481047181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi059-82882-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.956571+0000 mon.b (mon.1) 222 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:15.957005+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:14.636184+0000 osd.3 (osd.3) 115 : cluster [DBG] 97.5 scrub starts 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:14.638784+0000 osd.3 (osd.3) 116 : cluster [DBG] 97.5 scrub ok 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:14.941494+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi059-82981-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:14.941696+0000 mon.a (mon.0) 1315 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:14.941843+0000 mon.a (mon.0) 1316 : cluster [DBG] osdmap e176: 8 total, 8 up, 7 in 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:14.947398+0000 mon.b (mon.1) 220 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:14.949051+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.59:0/167636954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.044902+0000 osd.5 (osd.5) 87 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.045672+0000 osd.5 (osd.5) 88 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.230017+0000 osd.6 (osd.6) 91 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.230933+0000 osd.6 (osd.6) 92 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.391542+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.391902+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.669082+0000 mon.b (mon.1) 221 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.829706+0000 mgr.y (mgr.24454) 231 : cluster [DBG] pgmap v252: 347 pgs: 72 creating+peering, 2 peering, 273 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.931949+0000 osd.0 (osd.0) 99 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.934966+0000 osd.0 (osd.0) 100 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.944311+0000 mon.a (mon.0) 1319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.944429+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi059-82877-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:15.944511+0000 mon.a (mon.0) 1321 : cluster [DBG] osdmap e177: 8 total, 8 up, 7 in 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.950700+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.59:0/481047181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi059-82882-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.956571+0000 mon.b (mon.1) 222 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:15.957005+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:16.845 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:16.866 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-18T00:34:16.871 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:16.890 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:16.892 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:16.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:16.917 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:16.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:17.043 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:17.244 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:17.247 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:17.247 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:17.348 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:17.375 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:17.394 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:17.397 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:17.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:17.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:16.669932+0000 mon.b (mon.1) 223 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:17.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:16.945741+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? 172.21.15.59:0/481047181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi059-82882-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:17.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:16.945865+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:17.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:16.946193+0000 mon.a (mon.0) 1326 : cluster [DBG] osdmap e178: 8 total, 8 up, 7 in 2024-02-18T00:34:17.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:16.669932+0000 mon.b (mon.1) 223 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:17.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:16.945741+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? 172.21.15.59:0/481047181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi059-82882-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:17.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:16.945865+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:17.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:16.946193+0000 mon.a (mon.0) 1326 : cluster [DBG] osdmap e178: 8 total, 8 up, 7 in 2024-02-18T00:34:17.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 -- 172.21.15.59:0/156853118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fcbb0 msgr2=0x7f07700fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 --2- 172.21.15.59:0/156853118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fcbb0 0x7f07700fcf90 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 rx=0x7f0764004870 tx=0x7f0764009cb0).stop 2024-02-18T00:34:17.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 -- 172.21.15.59:0/156853118 shutdown_connections 2024-02-18T00:34:17.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 --2- 172.21.15.59:0/156853118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07700f7400 0x7f0770103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 --2- 172.21.15.59:0/156853118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fcbb0 0x7f07700fcf90 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 --2- 172.21.15.59:0/156853118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07700fd510 0x7f07700f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 -- 172.21.15.59:0/156853118 >> 172.21.15.59:0/156853118 conn(0x7f07700f2820 msgr2=0x7f07700f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 -- 172.21.15.59:0/156853118 shutdown_connections 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 -- 172.21.15.59:0/156853118 wait complete. 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 librados: starting msgr at 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.476+0000 7f0776238700 1 librados: starting objecter 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 Processor -- start 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 -- start start 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 librados: setting wanted keys 2024-02-18T00:34:17.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 librados: calling monclient init 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07700f7400 0x7f07700f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07700fcbb0 0x7f07700f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fd510 0x7f07700f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770116670 con 0x7f07700f7400 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07701167b0 con 0x7f07700fcbb0 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f0776238700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0770116970 con 0x7f07700fd510 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07700f7400 0x7f07700f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07700f7400 0x7f07700f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57816/0 (socket says 172.21.15.59:57816) 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076f7fe700 1 -- 172.21.15.59:0/4216020798 learned_addr learned my addr 172.21.15.59:0/4216020798 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fd510 0x7f07700f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076effd700 1 --2- 172.21.15.59:0/4216020798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07700fcbb0 0x7f07700f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076ffff700 1 -- 172.21.15.59:0/4216020798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07700fcbb0 msgr2=0x7f07700f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07700fcbb0 0x7f07700f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076ffff700 1 -- 172.21.15.59:0/4216020798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07700f7400 msgr2=0x7f07700f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07700f7400 0x7f07700f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.477+0000 7f076ffff700 1 -- 172.21.15.59:0/4216020798 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680097c0 con 0x7f07700fd510 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07700f7400 0x7f07700f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fd510 0x7f07700f9390 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 rx=0x7f0768001cb0 tx=0x7f0768004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f076cff9700 10 client.?.objecter ms_handle_connect 0x7f07700fd510 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f076cff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680079b0 con 0x7f07700fd510 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07680090d0 con 0x7f07700fd510 2024-02-18T00:34:17.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076800fc20 con 0x7f07700fd510 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0764005040 con 0x7f07700fd510 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f0776238700 10 client.40373.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0770116f60 con 0x7f07700fd510 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.478+0000 7f0776238700 1 librados: init done 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.479+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f077005fa60 con 0x7f07700fd510 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.479+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f076800fd80 con 0x7f07700fd510 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0758062c90 0x7f0758065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(178..178 src has 44..178) v4 ==== 10422+0+0 (secure 0 0 0) 0x7f076807e300 con 0x7f07700fd510 2024-02-18T00:34:17.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 osd_map(178..178 src has 44..178) v4 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 3 client.40373.objecter handle_osd_map got epochs [178,178] > 0 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 3 client.40373.objecter handle_osd_map decoding full epoch 178 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0758062c90 0x7f0758065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 20 client.40373.objecter dump_active .. 0 homeless 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v178) v1 ==== 54+0+16 (secure 0 0 0) 0x7f0768085070 con 0x7f07700fd510 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f0758062c90 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0758062c90 0x7f0758065150 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 rx=0x7f07640043d0 tx=0x7f076400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 10 client.40373.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 1 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6801/3700238977 conn(0x7f07700ffed0 0x7f07701002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 1 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 0 1 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077005ee80 osd=0 4 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 10 client.40373.objecter _send_command 1 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07701060e0 con 0x7f07700ffed0 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.480+0000 7f076effd700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6801/3700238977 conn(0x7f07700ffed0 0x7f07701002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.481+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f07700ffed0 2024-02-18T00:34:17.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.481+0000 7f076effd700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6801/3700238977 conn(0x7f07700ffed0 0x7f07701002b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.482+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f07701060e0 con 0x7f07700ffed0 2024-02-18T00:34:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.482+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:34:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.482+0000 7f076cff9700 10 client.40373.objecter _finish_command 1 = system:0 2024-02-18T00:34:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.482+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077005ee80 osd=0 4 2024-02-18T00:34:17.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.482+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 0 1 2024-02-18T00:34:17.498 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 10 client.40373.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 2 2024-02-18T00:34:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 2 2024-02-18T00:34:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077005ee80 osd=0 3 2024-02-18T00:34:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 0 2 2024-02-18T00:34:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077005ee80 osd=0 4 2024-02-18T00:34:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 10 client.40373.objecter _send_command 2 2024-02-18T00:34:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.497+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f07700273c0 con 0x7f07700ffed0 2024-02-18T00:34:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f07700273c0 con 0x7f07700ffed0 2024-02-18T00:34:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f076cff9700 10 client.40373.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077005ee80 osd=0 4 2024-02-18T00:34:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 0 2 2024-02-18T00:34:17.502 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 10 client.40373.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6805/26279110 conn(0x7f07700598f0 0x7f0770059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 1 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770028710 osd=1 4 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 10 client.40373.objecter _send_command 3 2024-02-18T00:34:17.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f077011d930 con 0x7f07700598f0 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.499+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6805/26279110 conn(0x7f07700598f0 0x7f0770059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.500+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f07700598f0 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.500+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6805/26279110 conn(0x7f07700598f0 0x7f0770059cf0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.500+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f077011d930 con 0x7f07700598f0 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.500+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.500+0000 7f076cff9700 10 client.40373.objecter _finish_command 3 = system:0 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.500+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f0770028710 osd=1 4 2024-02-18T00:34:17.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.500+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 1 3 2024-02-18T00:34:17.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:34:17.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 10 client.40373.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 4 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 4 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f0770028710 osd=1 3 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 1 4 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770028710 osd=1 4 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 10 client.40373.objecter _send_command 4 2024-02-18T00:34:17.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.515+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0770059360 con 0x7f07700598f0 2024-02-18T00:34:17.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0770059360 con 0x7f07700598f0 2024-02-18T00:34:17.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f076cff9700 10 client.40373.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f0770028710 osd=1 4 2024-02-18T00:34:17.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 1 4 2024-02-18T00:34:17.519 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:34:17.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 10 client.40373.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 5 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6809/1153193637 conn(0x7f077011c350 0x7f077011c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 5 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.517+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 2 5 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.518+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770027d60 osd=2 4 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.518+0000 7f0776238700 10 client.40373.objecter _send_command 5 2024-02-18T00:34:17.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.518+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07700283e0 con 0x7f077011c350 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.518+0000 7f076effd700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6809/1153193637 conn(0x7f077011c350 0x7f077011c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.518+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f077011c350 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.518+0000 7f076effd700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6809/1153193637 conn(0x7f077011c350 0x7f077011c790 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.519+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f07700283e0 con 0x7f077011c350 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.519+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.519+0000 7f076cff9700 10 client.40373.objecter _finish_command 5 = system:0 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.519+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f0770027d60 osd=2 4 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.519+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 2 5 2024-02-18T00:34:17.522 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:17.524 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:17.542 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 10 client.40373.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 6 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 6 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f0770027d60 osd=2 3 2024-02-18T00:34:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 2 6 2024-02-18T00:34:17.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770027d60 osd=2 4 2024-02-18T00:34:17.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 10 client.40373.objecter _send_command 6 2024-02-18T00:34:17.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.541+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f077011d450 con 0x7f077011c350 2024-02-18T00:34:17.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:17.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:17.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.544+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f077011d450 con 0x7f077011c350 2024-02-18T00:34:17.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.544+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.544+0000 7f076cff9700 10 client.40373.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.544+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f0770027d60 osd=2 4 2024-02-18T00:34:17.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.544+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 2 6 2024-02-18T00:34:17.547 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:34:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 10 client.40373.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 7 2024-02-18T00:34:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0770027940 0x7f0770121fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 7 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 3 7 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f07700277d0 osd=3 4 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 10 client.40373.objecter _send_command 7 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0770124c20 con 0x7f0770027940 2024-02-18T00:34:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0770027940 0x7f0770121fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f0770027940 2024-02-18T00:34:17.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.545+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0770027940 0x7f0770121fd0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.546+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0770124c20 con 0x7f0770027940 2024-02-18T00:34:17.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.546+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:34:17.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.546+0000 7f076cff9700 10 client.40373.objecter _finish_command 7 = system:0 2024-02-18T00:34:17.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.546+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f07700277d0 osd=3 4 2024-02-18T00:34:17.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.546+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 3 7 2024-02-18T00:34:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.568+0000 7f0776238700 10 client.40373.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 8 2024-02-18T00:34:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 8 2024-02-18T00:34:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f07700277d0 osd=3 3 2024-02-18T00:34:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 3 8 2024-02-18T00:34:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f07700277d0 osd=3 4 2024-02-18T00:34:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 10 client.40373.objecter _send_command 8 2024-02-18T00:34:17.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.569+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f077011cec0 con 0x7f0770027940 2024-02-18T00:34:17.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f077011cec0 con 0x7f0770027940 2024-02-18T00:34:17.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f076cff9700 10 client.40373.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f07700277d0 osd=3 4 2024-02-18T00:34:17.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 3 8 2024-02-18T00:34:17.574 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:34:17.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 10 client.40373.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 9 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6800/2044387056 conn(0x7f077011d030 0x7f0770123640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 9 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 4 9 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011cec0 osd=4 4 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 10 client.40373.objecter _send_command 9 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.571+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f077011beb0 con 0x7f077011d030 2024-02-18T00:34:17.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.572+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6800/2044387056 conn(0x7f077011d030 0x7f0770123640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.572+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f077011d030 2024-02-18T00:34:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.572+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6800/2044387056 conn(0x7f077011d030 0x7f0770123640 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.573+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f077011beb0 con 0x7f077011d030 2024-02-18T00:34:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.573+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:34:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.573+0000 7f076cff9700 10 client.40373.objecter _finish_command 9 = system:0 2024-02-18T00:34:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.573+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077011cec0 osd=4 4 2024-02-18T00:34:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.573+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 4 9 2024-02-18T00:34:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 10 client.40373.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 10 2024-02-18T00:34:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 10 2024-02-18T00:34:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077011cec0 osd=4 3 2024-02-18T00:34:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 4 10 2024-02-18T00:34:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011cec0 osd=4 4 2024-02-18T00:34:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 10 client.40373.objecter _send_command 10 2024-02-18T00:34:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.588+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f077011beb0 con 0x7f077011d030 2024-02-18T00:34:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f077011beb0 con 0x7f077011d030 2024-02-18T00:34:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f076cff9700 10 client.40373.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077011cec0 osd=4 4 2024-02-18T00:34:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 4 10 2024-02-18T00:34:17.592 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:34:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 10 client.40373.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 11 2024-02-18T00:34:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0770124040 0x7f07701244a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 11 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 5 11 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011beb0 osd=5 4 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 10 client.40373.objecter _send_command 11 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.590+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0770124990 con 0x7f0770124040 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.591+0000 7f076effd700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0770124040 0x7f07701244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.591+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f0770124040 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.591+0000 7f076effd700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0770124040 0x7f07701244a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.592+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0770124990 con 0x7f0770124040 2024-02-18T00:34:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.592+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:34:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.592+0000 7f076cff9700 10 client.40373.objecter _finish_command 11 = system:0 2024-02-18T00:34:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.592+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077011beb0 osd=5 4 2024-02-18T00:34:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.592+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 5 11 2024-02-18T00:34:17.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 10 client.40373.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 12 2024-02-18T00:34:17.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 12 2024-02-18T00:34:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077011beb0 osd=5 3 2024-02-18T00:34:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 5 12 2024-02-18T00:34:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011beb0 osd=5 4 2024-02-18T00:34:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 10 client.40373.objecter _send_command 12 2024-02-18T00:34:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.607+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0770124990 con 0x7f0770124040 2024-02-18T00:34:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0770124990 con 0x7f0770124040 2024-02-18T00:34:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f076cff9700 10 client.40373.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077011beb0 osd=5 4 2024-02-18T00:34:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 5 12 2024-02-18T00:34:17.612 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f0776238700 10 client.40373.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 13 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.609+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6808/3356009390 conn(0x7f07701292c0 0x7f077012b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 13 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 6 13 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770124990 osd=6 4 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 10 client.40373.objecter _send_command 13 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f077012e3a0 con 0x7f07701292c0 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6808/3356009390 conn(0x7f07701292c0 0x7f077012b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f07701292c0 2024-02-18T00:34:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.610+0000 7f076ffff700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6808/3356009390 conn(0x7f07701292c0 0x7f077012b6b0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.611+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f077012e3a0 con 0x7f07701292c0 2024-02-18T00:34:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.611+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:34:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.611+0000 7f076cff9700 10 client.40373.objecter _finish_command 13 = system:0 2024-02-18T00:34:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.611+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f0770124990 osd=6 4 2024-02-18T00:34:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.611+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 6 13 2024-02-18T00:34:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 10 client.40373.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 14 2024-02-18T00:34:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 14 2024-02-18T00:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f0770124990 osd=6 3 2024-02-18T00:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 6 14 2024-02-18T00:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770124990 osd=6 4 2024-02-18T00:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 10 client.40373.objecter _send_command 14 2024-02-18T00:34:17.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.626+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f077012e3a0 con 0x7f07701292c0 2024-02-18T00:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f077012e3a0 con 0x7f07701292c0 2024-02-18T00:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f076cff9700 10 client.40373.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f0770124990 osd=6 4 2024-02-18T00:34:17.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 6 14 2024-02-18T00:34:17.630 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 10 client.40373.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 15 2024-02-18T00:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6812/53025171 conn(0x7f077012d6c0 0x7f077012db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 15 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 7 15 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077012e3a0 osd=7 4 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 10 client.40373.objecter _send_command 15 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.628+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f077012e190 con 0x7f077012d6c0 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.629+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6812/53025171 conn(0x7f077012d6c0 0x7f077012db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.629+0000 7f076cff9700 10 client.40373.objecter ms_handle_connect 0x7f077012d6c0 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.629+0000 7f076f7fe700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6812/53025171 conn(0x7f077012d6c0 0x7f077012db20 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.630+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f077012e190 con 0x7f077012d6c0 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.630+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.630+0000 7f076cff9700 10 client.40373.objecter _finish_command 15 = system:0 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.630+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077012e3a0 osd=7 4 2024-02-18T00:34:17.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.630+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 7 15 2024-02-18T00:34:17.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:17.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 10 client.40373.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:17.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign -1 16 2024-02-18T00:34:17.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 20 client.40373.objecter _get_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 15 client.40373.objecter _session_command_op_remove -1 16 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 20 client.40373.objecter get_session s=0x7f077012e3a0 osd=7 3 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 15 client.40373.objecter _session_command_op_assign 7 16 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077012e3a0 osd=7 4 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 10 client.40373.objecter _send_command 16 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.645+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f077012e190 con 0x7f077012d6c0 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.646+0000 7f076cff9700 1 -- 172.21.15.59:0/4216020798 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f077012e190 con 0x7f077012d6c0 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.646+0000 7f076cff9700 10 client.40373.objecter ms_dispatch 0x7f0770001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.646+0000 7f076cff9700 10 client.40373.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.646+0000 7f076cff9700 20 client.40373.objecter put_session s=0x7f077012e3a0 osd=7 4 2024-02-18T00:34:17.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.646+0000 7f076cff9700 15 client.40373.objecter _session_command_op_remove 7 16 2024-02-18T00:34:17.650 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:34:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 librados: watch_flush enter 2024-02-18T00:34:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 librados: watch_flush exit 2024-02-18T00:34:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.0 2024-02-18T00:34:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6801/3700238977 conn(0x7f07700ffed0 msgr2=0x7f07701002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6801/3700238977 conn(0x7f07700ffed0 0x7f07701002b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077005ee80 osd=0 1 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.1 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6805/26279110 conn(0x7f07700598f0 msgr2=0x7f0770059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6805/26279110 conn(0x7f07700598f0 0x7f0770059cf0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770028710 osd=1 1 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.2 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6809/1153193637 conn(0x7f077011c350 msgr2=0x7f077011c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6809/1153193637 conn(0x7f077011c350 0x7f077011c790 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770027d60 osd=2 1 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.3 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0770027940 msgr2=0x7f0770121fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0770027940 0x7f0770121fd0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f07700277d0 osd=3 1 2024-02-18T00:34:17.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.4 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6800/2044387056 conn(0x7f077011d030 msgr2=0x7f0770123640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6800/2044387056 conn(0x7f077011d030 0x7f0770123640 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011cec0 osd=4 1 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f076ffff700 1 -- 172.21.15.59:0/4216020798 reap_dead start 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.5 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0770124040 msgr2=0x7f07701244a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0770124040 0x7f07701244a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f076ffff700 1 -- 172.21.15.59:0/4216020798 reap_dead start 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f076ffff700 1 -- 172.21.15.59:0/4216020798 reap_dead start 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077011beb0 osd=5 1 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.6 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6808/3356009390 conn(0x7f07701292c0 msgr2=0x7f077012b6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6808/3356009390 conn(0x7f07701292c0 0x7f077012b6b0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f0770124990 osd=6 1 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter close_session for osd.7 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6812/53025171 conn(0x7f077012d6c0 msgr2=0x7f077012db20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.137:6812/53025171 conn(0x7f077012d6c0 0x7f077012db20 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter put_session s=0x7f077012e3a0 osd=7 1 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 20 client.40373.objecter shutdown clearing up homeless session... 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 10 client.40373.objecter successfully canceled tick 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0758062c90 msgr2=0x7f0758065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0758062c90 0x7f0758065150 secure :-1 s=READY pgs=2972 cs=0 l=1 rev1=1 rx=0x7f07640043d0 tx=0x7f076400b040).stop 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fd510 msgr2=0x7f07700f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 --2- 172.21.15.59:0/4216020798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f07700fd510 0x7f07700f9390 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 rx=0x7f0768001cb0 tx=0x7f0768004c50).stop 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f076ffff700 1 -- 172.21.15.59:0/4216020798 reap_dead start 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 shutdown_connections 2024-02-18T00:34:17.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 >> 172.21.15.59:0/4216020798 conn(0x7f07700f2820 msgr2=0x7f07700f4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 shutdown_connections 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 -- 172.21.15.59:0/4216020798 wait complete. 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:17.647+0000 7f0776238700 1 librados: shutdown 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.655 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:34:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:16.669932+0000 mon.b (mon.1) 223 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:16.945741+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? 172.21.15.59:0/481047181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi059-82882-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:16.945865+0000 mon.a (mon.0) 1325 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82949-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:16.946193+0000 mon.a (mon.0) 1326 : cluster [DBG] osdmap e178: 8 total, 8 up, 7 in 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:16.686907+0000 osd.3 (osd.3) 117 : cluster [DBG] 97.6 scrub starts 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:16.689387+0000 osd.3 (osd.3) 118 : cluster [DBG] 97.6 scrub ok 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:17.013597+0000 osd.5 (osd.5) 89 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:17.016471+0000 osd.5 (osd.5) 90 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:17.271020+0000 osd.6 (osd.6) 93 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:17.271863+0000 osd.6 (osd.6) 94 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:17.481093+0000 mon.c (mon.2) 116 : audit [DBG] from='client.? 172.21.15.59:0/4216020798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:18.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:17.670891+0000 mon.b (mon.1) 224 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:17.830343+0000 mgr.y (mgr.24454) 232 : cluster [DBG] pgmap v255: 346 pgs: 32 unknown, 40 creating+peering, 1 peering, 273 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 4.7 KiB/s wr, 10 op/s; 30 B/s, 0 objects/s recovering 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:17.831530+0000 mon.a (mon.0) 1327 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:17.945811+0000 mon.a (mon.0) 1328 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:17.945888+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e179: 8 total, 8 up, 7 in 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:17.961931+0000 mon.b (mon.1) 225 : audit [INF] from='client.? 172.21.15.59:0/2310105786' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:17.973713+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:16.686907+0000 osd.3 (osd.3) 117 : cluster [DBG] 97.6 scrub starts 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:16.689387+0000 osd.3 (osd.3) 118 : cluster [DBG] 97.6 scrub ok 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:17.013597+0000 osd.5 (osd.5) 89 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:17.016471+0000 osd.5 (osd.5) 90 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:34:18.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:17.271020+0000 osd.6 (osd.6) 93 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:17.271863+0000 osd.6 (osd.6) 94 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:17.481093+0000 mon.c (mon.2) 116 : audit [DBG] from='client.? 172.21.15.59:0/4216020798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:17.670891+0000 mon.b (mon.1) 224 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:17.830343+0000 mgr.y (mgr.24454) 232 : cluster [DBG] pgmap v255: 346 pgs: 32 unknown, 40 creating+peering, 1 peering, 273 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 4.7 KiB/s wr, 10 op/s; 30 B/s, 0 objects/s recovering 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:17.831530+0000 mon.a (mon.0) 1327 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:17.945811+0000 mon.a (mon.0) 1328 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:17.945888+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e179: 8 total, 8 up, 7 in 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:17.961931+0000 mon.b (mon.1) 225 : audit [INF] from='client.? 172.21.15.59:0/2310105786' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:18.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:17.973713+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:16.686907+0000 osd.3 (osd.3) 117 : cluster [DBG] 97.6 scrub starts 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:16.689387+0000 osd.3 (osd.3) 118 : cluster [DBG] 97.6 scrub ok 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:17.013597+0000 osd.5 (osd.5) 89 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:17.016471+0000 osd.5 (osd.5) 90 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:17.271020+0000 osd.6 (osd.6) 93 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:17.271863+0000 osd.6 (osd.6) 94 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:17.481093+0000 mon.c (mon.2) 116 : audit [DBG] from='client.? 172.21.15.59:0/4216020798' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:17.670891+0000 mon.b (mon.1) 224 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:17.830343+0000 mgr.y (mgr.24454) 232 : cluster [DBG] pgmap v255: 346 pgs: 32 unknown, 40 creating+peering, 1 peering, 273 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 4.7 KiB/s wr, 10 op/s; 30 B/s, 0 objects/s recovering 2024-02-18T00:34:18.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:17.831530+0000 mon.a (mon.0) 1327 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-18T00:34:18.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:17.945811+0000 mon.a (mon.0) 1328 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-18T00:34:18.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:17.945888+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e179: 8 total, 8 up, 7 in 2024-02-18T00:34:18.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:17.961931+0000 mon.b (mon.1) 225 : audit [INF] from='client.? 172.21.15.59:0/2310105786' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:18.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:17.973713+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:19.030 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:19.033 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:19.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:19.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:19.165 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:19.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:19.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:19.619 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:19.652 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:19.673 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:19.675 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:19.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.298248+0000 osd.4 (osd.4) 113 : cluster [DBG] 2.b scrub starts 2024-02-18T00:34:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.301133+0000 osd.4 (osd.4) 114 : cluster [DBG] 2.b scrub ok 2024-02-18T00:34:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.316674+0000 osd.6 (osd.6) 95 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:34:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.318180+0000 osd.6 (osd.6) 96 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:34:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.356428+0000 osd.1 (osd.1) 99 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:34:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.357365+0000 osd.1 (osd.1) 100 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:34:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:18.672068+0000 mon.b (mon.1) 226 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:18.947746+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.947930+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e180: 8 total, 8 up, 7 in 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:18.957968+0000 mon.b (mon.1) 227 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:18.962760+0000 mon.b (mon.1) 228 : audit [INF] from='client.? 172.21.15.59:0/1773310280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:18.964181+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:18.964648+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.298248+0000 osd.4 (osd.4) 113 : cluster [DBG] 2.b scrub starts 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.301133+0000 osd.4 (osd.4) 114 : cluster [DBG] 2.b scrub ok 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.316674+0000 osd.6 (osd.6) 95 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:34:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.318180+0000 osd.6 (osd.6) 96 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.356428+0000 osd.1 (osd.1) 99 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.357365+0000 osd.1 (osd.1) 100 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:18.672068+0000 mon.b (mon.1) 226 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:18.947746+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.947930+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e180: 8 total, 8 up, 7 in 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:18.957968+0000 mon.b (mon.1) 227 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:18.962760+0000 mon.b (mon.1) 228 : audit [INF] from='client.? 172.21.15.59:0/1773310280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:18.964181+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:18.964648+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.298248+0000 osd.4 (osd.4) 113 : cluster [DBG] 2.b scrub starts 2024-02-18T00:34:19.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.301133+0000 osd.4 (osd.4) 114 : cluster [DBG] 2.b scrub ok 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.316674+0000 osd.6 (osd.6) 95 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.318180+0000 osd.6 (osd.6) 96 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.356428+0000 osd.1 (osd.1) 99 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.357365+0000 osd.1 (osd.1) 100 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:18.672068+0000 mon.b (mon.1) 226 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:18.947746+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi059-82877-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.947930+0000 mon.a (mon.0) 1332 : cluster [DBG] osdmap e180: 8 total, 8 up, 7 in 2024-02-18T00:34:19.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:18.957968+0000 mon.b (mon.1) 227 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:19.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:18.962760+0000 mon.b (mon.1) 228 : audit [INF] from='client.? 172.21.15.59:0/1773310280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:19.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:18.964181+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:19.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:18.964648+0000 mon.a (mon.0) 1334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:19.777 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:19.806 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:19.826 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:19.828 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:19.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:19.930 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:20.002 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:20.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:20.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:20.105 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:20.132 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:20.150 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:20.153 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:20.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:20.254 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:20.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.274+0000 7fc4ef554700 1 -- 172.21.15.59:0/545669691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e8102210 msgr2=0x7fc4e8104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.274+0000 7fc4ef554700 1 --2- 172.21.15.59:0/545669691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e8102210 0x7fc4e8104600 secure :-1 s=READY pgs=4043 cs=0 l=1 rev1=1 rx=0x7fc4e0009c30 tx=0x7fc4e0004c50).stop 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.274+0000 7fc4ef554700 1 -- 172.21.15.59:0/545669691 shutdown_connections 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.274+0000 7fc4ef554700 1 --2- 172.21.15.59:0/545669691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e8102210 0x7fc4e8104600 unknown :-1 s=CLOSED pgs=4043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.274+0000 7fc4ef554700 1 --2- 172.21.15.59:0/545669691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e80f8cb0 0x7fc4e80f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.274+0000 7fc4ef554700 1 --2- 172.21.15.59:0/545669691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e80f9610 0x7fc4e8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.274+0000 7fc4ef554700 1 -- 172.21.15.59:0/545669691 >> 172.21.15.59:0/545669691 conn(0x7fc4e80f2820 msgr2=0x7fc4e80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 -- 172.21.15.59:0/545669691 shutdown_connections 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 -- 172.21.15.59:0/545669691 wait complete. 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 librados: starting msgr at 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 librados: starting objecter 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 Processor -- start 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 -- start start 2024-02-18T00:34:20.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 librados: setting wanted keys 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 librados: calling monclient init 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80f8cb0 0x7fc4e80f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e80f9610 0x7fc4e80f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e8102210 0x7fc4e80f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80f89c0 con 0x7fc4e80f9610 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80f8b30 con 0x7fc4e80f8cb0 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.275+0000 7fc4ef554700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e811ad50 con 0x7fc4e8102210 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4edaf1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e8102210 0x7fc4e80f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4edaf1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e8102210 0x7fc4e80f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:50878/0 (socket says 172.21.15.59:50878) 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4edaf1700 1 -- 172.21.15.59:0/4132862142 learned_addr learned my addr 172.21.15.59:0/4132862142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ecaef700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e80f9610 0x7fc4e80f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ed2f0700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80f8cb0 0x7fc4e80f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:20.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ecaef700 1 -- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e8102210 msgr2=0x7fc4e80f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ecaef700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e8102210 0x7fc4e80f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ecaef700 1 -- 172.21.15.59:0/4132862142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80f8cb0 msgr2=0x7fc4e80f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ecaef700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80f8cb0 0x7fc4e80f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ecaef700 1 -- 172.21.15.59:0/4132862142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e4005040 con 0x7fc4e80f9610 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4edaf1700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e8102210 0x7fc4e80f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4ecaef700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e80f9610 0x7fc4e80f7150 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 rx=0x7fc4d8009220 tx=0x7fc4d8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4de7fc700 10 client.?.objecter ms_handle_connect 0x7fc4e80f9610 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.276+0000 7fc4de7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:20.281 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:20.299 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:20.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:20.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:20.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:20.433 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:20.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.441+0000 7fc4de7fc700 1 -- 172.21.15.59:0/4132862142 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800e030 con 0x7fc4e80f9610 2024-02-18T00:34:20.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.441+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d8005040 con 0x7fc4e80f9610 2024-02-18T00:34:20.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.441+0000 7fc4ef554700 10 client.30567.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:20.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.441+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e00097c0 con 0x7fc4e80f9610 2024-02-18T00:34:20.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.441+0000 7fc4ef554700 1 librados: init done 2024-02-18T00:34:20.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.442+0000 7fc4de7fc700 1 -- 172.21.15.59:0/4132862142 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d800f040 con 0x7fc4e80f9610 2024-02-18T00:34:20.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.442+0000 7fc4de7fc700 1 -- 172.21.15.59:0/4132862142 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8014740 con 0x7fc4e80f9610 2024-02-18T00:34:20.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.443+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc004f50 con 0x7fc4e80f9610 2024-02-18T00:34:20.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.443+0000 7fc4de7fc700 1 -- 172.21.15.59:0/4132862142 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc4d8010700 con 0x7fc4e80f9610 2024-02-18T00:34:20.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4de7fc700 1 --2- 172.21.15.59:0/4132862142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc4d4062c90 0x7fc4d4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:20.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4de7fc700 1 -- 172.21.15.59:0/4132862142 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(181..181 src has 44..181) v4 ==== 9343+0+0 (secure 0 0 0) 0x7fc4d807d540 con 0x7fc4e80f9610 2024-02-18T00:34:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4de7fc700 10 client.30567.objecter ms_dispatch 0x7fc4e8001550 osd_map(181..181 src has 44..181) v4 2024-02-18T00:34:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4de7fc700 3 client.30567.objecter handle_osd_map got epochs [181,181] > 0 2024-02-18T00:34:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4de7fc700 3 client.30567.objecter handle_osd_map decoding full epoch 181 2024-02-18T00:34:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4ed2f0700 1 --2- 172.21.15.59:0/4132862142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc4d4062c90 0x7fc4d4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4de7fc700 20 client.30567.objecter dump_active .. 0 homeless 2024-02-18T00:34:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4de7fc700 10 client.30567.objecter ms_handle_connect 0x7fc4d4062c90 2024-02-18T00:34:20.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.444+0000 7fc4ed2f0700 1 --2- 172.21.15.59:0/4132862142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc4d4062c90 0x7fc4d4065150 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 rx=0x7fc4e4009350 tx=0x7fc4e400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:20.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.453+0000 7fc4de7fc700 1 -- 172.21.15.59:0/4132862142 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc4d8050510 con 0x7fc4e80f9610 2024-02-18T00:34:20.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.649+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) v1 -- 0x7fc4cc005200 con 0x7fc4e80f9610 2024-02-18T00:34:20.685 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.975735+0000 osd.0 (osd.0) 101 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:18.980608+0000 osd.0 (osd.0) 102 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:19.321628+0000 osd.4 (osd.4) 115 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:19.325126+0000 osd.4 (osd.4) 116 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:19.673602+0000 mon.b (mon.1) 229 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:19.831004+0000 mgr.y (mgr.24454) 233 : cluster [DBG] pgmap v258: 338 pgs: 64 unknown, 8 creating+peering, 1 peering, 265 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:19.924088+0000 osd.0 (osd.0) 103 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:34:20.687 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:19.925107+0000 osd.0 (osd.0) 104 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:19.949061+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:19.949129+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:19.949250+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e181: 8 total, 8 up, 7 in 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:19.953192+0000 mon.b (mon.1) 230 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:19.954095+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.975735+0000 osd.0 (osd.0) 101 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:18.980608+0000 osd.0 (osd.0) 102 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:19.321628+0000 osd.4 (osd.4) 115 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:34:20.688 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:19.325126+0000 osd.4 (osd.4) 116 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:19.673602+0000 mon.b (mon.1) 229 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:19.831004+0000 mgr.y (mgr.24454) 233 : cluster [DBG] pgmap v258: 338 pgs: 64 unknown, 8 creating+peering, 1 peering, 265 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:19.924088+0000 osd.0 (osd.0) 103 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:19.925107+0000 osd.0 (osd.0) 104 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:19.949061+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:19.949129+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:19.949250+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e181: 8 total, 8 up, 7 in 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:19.953192+0000 mon.b (mon.1) 230 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:20.689 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:19.954095+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:20.690 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:20.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.975735+0000 osd.0 (osd.0) 101 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:34:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:18.980608+0000 osd.0 (osd.0) 102 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:34:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:19.321628+0000 osd.4 (osd.4) 115 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:34:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:19.325126+0000 osd.4 (osd.4) 116 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:34:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:19.673602+0000 mon.b (mon.1) 229 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:19.831004+0000 mgr.y (mgr.24454) 233 : cluster [DBG] pgmap v258: 338 pgs: 64 unknown, 8 creating+peering, 1 peering, 265 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:19.924088+0000 osd.0 (osd.0) 103 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:34:20.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:19.925107+0000 osd.0 (osd.0) 104 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:34:20.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:19.949061+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:20.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:19.949129+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi059-82882-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:20.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:19.949250+0000 mon.a (mon.0) 1337 : cluster [DBG] osdmap e181: 8 total, 8 up, 7 in 2024-02-18T00:34:20.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:19.953192+0000 mon.b (mon.1) 230 : audit [INF] from='client.? 172.21.15.59:0/955395839' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:20.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:19.954095+0000 mon.a (mon.0) 1338 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]: dispatch 2024-02-18T00:34:20.792 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:20.818 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:20.837 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:20.839 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:20.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:20.940 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:20.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.953+0000 7fc4de7fc700 1 -- 172.21.15.59:0/4132862142 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v182) v1 ==== 113+0+0 (secure 0 0 0) 0x7fc4d8019ed0 con 0x7fc4e80f9610 2024-02-18T00:34:20.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.958+0000 7fc4ef554700 10 librados: watch_flush enter 2024-02-18T00:34:20.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.958+0000 7fc4ef554700 10 librados: watch_flush exit 2024-02-18T00:34:20.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.958+0000 7fc4ef554700 20 client.30567.objecter shutdown clearing up homeless session... 2024-02-18T00:34:20.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.958+0000 7fc4ef554700 10 client.30567.objecter successfully canceled tick 2024-02-18T00:34:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.960+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc4d4062c90 msgr2=0x7fc4d4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.960+0000 7fc4ef554700 1 --2- 172.21.15.59:0/4132862142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc4d4062c90 0x7fc4d4065150 secure :-1 s=READY pgs=2975 cs=0 l=1 rev1=1 rx=0x7fc4e4009350 tx=0x7fc4e400b040).stop 2024-02-18T00:34:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.960+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e80f9610 msgr2=0x7fc4e80f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:20.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.960+0000 7fc4ef554700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e80f9610 0x7fc4e80f7150 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 rx=0x7fc4d8009220 tx=0x7fc4d8009cb0).stop 2024-02-18T00:34:20.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.966+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 shutdown_connections 2024-02-18T00:34:20.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.966+0000 7fc4ef554700 1 --2- 172.21.15.59:0/4132862142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc4d4062c90 0x7fc4d4065150 unknown :-1 s=CLOSED pgs=2975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.966+0000 7fc4ef554700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4e80f8cb0 0x7fc4e80f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.966+0000 7fc4ef554700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4e8102210 0x7fc4e80f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.966+0000 7fc4ef554700 1 --2- 172.21.15.59:0/4132862142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e80f9610 0x7fc4e80f7150 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:20.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.966+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 >> 172.21.15.59:0/4132862142 conn(0x7fc4e80f2820 msgr2=0x7fc4e80f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:20.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.971+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 shutdown_connections 2024-02-18T00:34:20.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.976+0000 7fc4ef554700 1 -- 172.21.15.59:0/4132862142 wait complete. 2024-02-18T00:34:20.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:20.980+0000 7fc4ef554700 1 librados: shutdown 2024-02-18T00:34:20.983 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.0 to 1 (10000) 2024-02-18T00:34:20.999 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:21.248 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:21.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:21.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:21.255 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:34:21.255 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:34:21.355 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:21.385 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:21.407 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:21.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:21.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:21.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:21.512 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:21.541 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:21.982 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 1 1 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.311357+0000 osd.6 (osd.6) 97 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.312091+0000 osd.6 (osd.6) 98 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.359308+0000 osd.4 (osd.4) 117 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.362779+0000 osd.4 (osd.4) 118 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:20.650989+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.59:0/4132862142' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:20.674933+0000 mon.b (mon.1) 231 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:20.952984+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:20.953165+0000 mon.a (mon.0) 1341 : audit [INF] from='client.? 172.21.15.59:0/4132862142' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.953227+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e182: 8 total, 8 up, 7 in 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:20.975843+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? 172.21.15.59:0/2128212082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi059-82877-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.991659+0000 osd.5 (osd.5) 91 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.995121+0000 osd.5 (osd.5) 92 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.003078+0000 mgr.y (mgr.24454) 234 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.676260+0000 mon.b (mon.1) 232 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.311357+0000 osd.6 (osd.6) 97 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.312091+0000 osd.6 (osd.6) 98 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.359308+0000 osd.4 (osd.4) 117 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:34:22.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.362779+0000 osd.4 (osd.4) 118 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:20.650989+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.59:0/4132862142' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:20.674933+0000 mon.b (mon.1) 231 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:20.952984+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:20.953165+0000 mon.a (mon.0) 1341 : audit [INF] from='client.? 172.21.15.59:0/4132862142' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.953227+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e182: 8 total, 8 up, 7 in 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:20.975843+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? 172.21.15.59:0/2128212082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi059-82877-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.991659+0000 osd.5 (osd.5) 91 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.995121+0000 osd.5 (osd.5) 92 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.003078+0000 mgr.y (mgr.24454) 234 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:22.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.676260+0000 mon.b (mon.1) 232 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.311357+0000 osd.6 (osd.6) 97 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.312091+0000 osd.6 (osd.6) 98 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.359308+0000 osd.4 (osd.4) 117 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.362779+0000 osd.4 (osd.4) 118 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:20.650989+0000 mon.a (mon.0) 1339 : audit [INF] from='client.? 172.21.15.59:0/4132862142' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:20.674933+0000 mon.b (mon.1) 231 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:20.952984+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82949-24"}]': finished 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:20.953165+0000 mon.a (mon.0) 1341 : audit [INF] from='client.? 172.21.15.59:0/4132862142' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.953227+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e182: 8 total, 8 up, 7 in 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:20.975843+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? 172.21.15.59:0/2128212082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi059-82877-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.991659+0000 osd.5 (osd.5) 91 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:34:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.995121+0000 osd.5 (osd.5) 92 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:34:22.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.003078+0000 mgr.y (mgr.24454) 234 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:22.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.676260+0000 mon.b (mon.1) 232 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:22.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:22] "GET /metrics HTTP/1.1" 200 198420 "" "Prometheus/2.33.4" 2024-02-18T00:34:22.635 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:34:22.635 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:34:22.668 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:34:23.024 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:23.028 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:23.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:23.129 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:23.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.997878+0000 osd.2 (osd.2) 85 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:34:23.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:20.999464+0000 osd.2 (osd.2) 86 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:34:23.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:21.831400+0000 mgr.y (mgr.24454) 235 : cluster [DBG] pgmap v261: 305 pgs: 32 unknown, 1 peering, 272 active+clean; 16 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-18T00:34:23.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.890243+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:21.949181+0000 mon.a (mon.0) 1345 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.966858+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? 172.21.15.59:0/2128212082' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi059-82877-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.967056+0000 mon.a (mon.0) 1347 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:21.967117+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e183: 8 total, 8 up, 7 in 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.980028+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? 172.21.15.59:0/1196007280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi059-82949-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.991384+0000 mon.b (mon.1) 233 : audit [INF] from='client.? 172.21.15.59:0/3581811467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:21.991799+0000 mon.a (mon.0) 1350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:22.355224+0000 osd.4 (osd.4) 119 : cluster [DBG] 6.e scrub starts 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:22.358098+0000 osd.4 (osd.4) 120 : cluster [DBG] 6.e scrub ok 2024-02-18T00:34:23.158 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:22.677463+0000 mon.b (mon.1) 234 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:23.159 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:23.180 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:23.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:23.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.997878+0000 osd.2 (osd.2) 85 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:34:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:20.999464+0000 osd.2 (osd.2) 86 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:34:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:21.831400+0000 mgr.y (mgr.24454) 235 : cluster [DBG] pgmap v261: 305 pgs: 32 unknown, 1 peering, 272 active+clean; 16 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-18T00:34:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.890243+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-18T00:34:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:21.949181+0000 mon.a (mon.0) 1345 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.966858+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? 172.21.15.59:0/2128212082' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi059-82877-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.967056+0000 mon.a (mon.0) 1347 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:21.967117+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e183: 8 total, 8 up, 7 in 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.980028+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? 172.21.15.59:0/1196007280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi059-82949-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.991384+0000 mon.b (mon.1) 233 : audit [INF] from='client.? 172.21.15.59:0/3581811467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:21.991799+0000 mon.a (mon.0) 1350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:22.355224+0000 osd.4 (osd.4) 119 : cluster [DBG] 6.e scrub starts 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:22.358098+0000 osd.4 (osd.4) 120 : cluster [DBG] 6.e scrub ok 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:22.677463+0000 mon.b (mon.1) 234 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.997878+0000 osd.2 (osd.2) 85 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:20.999464+0000 osd.2 (osd.2) 86 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:21.831400+0000 mgr.y (mgr.24454) 235 : cluster [DBG] pgmap v261: 305 pgs: 32 unknown, 1 peering, 272 active+clean; 16 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-18T00:34:23.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.890243+0000 mon.a (mon.0) 1344 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:21.949181+0000 mon.a (mon.0) 1345 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.966858+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? 172.21.15.59:0/2128212082' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi059-82877-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.967056+0000 mon.a (mon.0) 1347 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:21.967117+0000 mon.a (mon.0) 1348 : cluster [DBG] osdmap e183: 8 total, 8 up, 7 in 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.980028+0000 mon.a (mon.0) 1349 : audit [INF] from='client.? 172.21.15.59:0/1196007280' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi059-82949-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.991384+0000 mon.b (mon.1) 233 : audit [INF] from='client.? 172.21.15.59:0/3581811467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:21.991799+0000 mon.a (mon.0) 1350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:22.355224+0000 osd.4 (osd.4) 119 : cluster [DBG] 6.e scrub starts 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:22.358098+0000 osd.4 (osd.4) 120 : cluster [DBG] 6.e scrub ok 2024-02-18T00:34:23.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:22.677463+0000 mon.b (mon.1) 234 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:23.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:23.311 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:23.330 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:23.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:23.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:23.340 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:23.340 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:23.433 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:23.461 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:23.480 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:23.483 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:23.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:23.584 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:23.617 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:23.636 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:23.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:23.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:23.740 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:23.766 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:23.785 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:23.787 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:23.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:23.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:23.918 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:24.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-18T00:34:24.468 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:24.472 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:24.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:24.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:24.623 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:22.826128+0000 osd.5 (osd.5) 93 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:22.826555+0000 osd.5 (osd.5) 94 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:22.916350+0000 osd.0 (osd.0) 105 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:22.917593+0000 osd.0 (osd.0) 106 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:22.966023+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? 172.21.15.59:0/1196007280' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi059-82949-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:22.966138+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:22.966262+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e184: 8 total, 8 up, 7 in 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:23.033230+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:23.033656+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:23.034017+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi059-82877-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:23.585513+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi059-82877-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.585593+0000 mon.a (mon.0) 1358 : cluster [DBG] osdmap e185: 8 total, 8 up, 7 in 2024-02-18T00:34:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:23.598574+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi059-82877-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:22.826128+0000 osd.5 (osd.5) 93 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:22.826555+0000 osd.5 (osd.5) 94 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:22.916350+0000 osd.0 (osd.0) 105 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:22.917593+0000 osd.0 (osd.0) 106 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:22.966023+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? 172.21.15.59:0/1196007280' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi059-82949-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:22.966138+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:22.966262+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e184: 8 total, 8 up, 7 in 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:23.033230+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:23.033656+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:23.034017+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi059-82877-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:23.585513+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi059-82877-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.585593+0000 mon.a (mon.0) 1358 : cluster [DBG] osdmap e185: 8 total, 8 up, 7 in 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:23.598574+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi059-82877-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:22.826128+0000 osd.5 (osd.5) 93 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:22.826555+0000 osd.5 (osd.5) 94 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:22.916350+0000 osd.0 (osd.0) 105 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:22.917593+0000 osd.0 (osd.0) 106 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:22.966023+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? 172.21.15.59:0/1196007280' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi059-82949-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:22.966138+0000 mon.a (mon.0) 1352 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi059-82882-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:22.966262+0000 mon.a (mon.0) 1353 : cluster [DBG] osdmap e184: 8 total, 8 up, 7 in 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:23.033230+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:23.033656+0000 mon.a (mon.0) 1355 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:24.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:23.034017+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi059-82877-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:23.585513+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi059-82877-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.585593+0000 mon.a (mon.0) 1358 : cluster [DBG] osdmap e185: 8 total, 8 up, 7 in 2024-02-18T00:34:24.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:23.598574+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi059-82877-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.256983+0000 osd.6 (osd.6) 99 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.258449+0000 osd.6 (osd.6) 100 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.292166+0000 osd.1 (osd.1) 101 : cluster [DBG] 2.a scrub starts 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.295046+0000 osd.1 (osd.1) 102 : cluster [DBG] 2.a scrub ok 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.373538+0000 osd.4 (osd.4) 121 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.374325+0000 osd.4 (osd.4) 122 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:23.678650+0000 mon.b (mon.1) 235 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:25.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:23.831799+0000 mgr.y (mgr.24454) 236 : cluster [DBG] pgmap v265: 272 pgs: 1 peering, 271 active+clean; 16 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.8 KiB/s wr, 3 op/s; 32 B/s, 0 objects/s recovering 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:24.261749+0000 osd.6 (osd.6) 101 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:24.262532+0000 osd.6 (osd.6) 102 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:24.325514+0000 osd.4 (osd.4) 123 : cluster [DBG] 4.c scrub starts 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:24.328986+0000 osd.4 (osd.4) 124 : cluster [DBG] 4.c scrub ok 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:24.333843+0000 osd.1 (osd.1) 103 : cluster [DBG] 2.d scrub starts 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:24.334807+0000 osd.1 (osd.1) 104 : cluster [DBG] 2.d scrub ok 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:24.504422+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:24.597703+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:24.679769+0000 mon.b (mon.1) 237 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.034900+0000 mon.a (mon.0) 1361 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-18T00:34:25.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.256983+0000 osd.6 (osd.6) 99 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.258449+0000 osd.6 (osd.6) 100 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.292166+0000 osd.1 (osd.1) 101 : cluster [DBG] 2.a scrub starts 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.295046+0000 osd.1 (osd.1) 102 : cluster [DBG] 2.a scrub ok 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.373538+0000 osd.4 (osd.4) 121 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.374325+0000 osd.4 (osd.4) 122 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:23.678650+0000 mon.b (mon.1) 235 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:23.831799+0000 mgr.y (mgr.24454) 236 : cluster [DBG] pgmap v265: 272 pgs: 1 peering, 271 active+clean; 16 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.8 KiB/s wr, 3 op/s; 32 B/s, 0 objects/s recovering 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:24.261749+0000 osd.6 (osd.6) 101 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:24.262532+0000 osd.6 (osd.6) 102 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:24.325514+0000 osd.4 (osd.4) 123 : cluster [DBG] 4.c scrub starts 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:24.328986+0000 osd.4 (osd.4) 124 : cluster [DBG] 4.c scrub ok 2024-02-18T00:34:25.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:24.333843+0000 osd.1 (osd.1) 103 : cluster [DBG] 2.d scrub starts 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:24.334807+0000 osd.1 (osd.1) 104 : cluster [DBG] 2.d scrub ok 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:24.504422+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:24.597703+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:24.679769+0000 mon.b (mon.1) 237 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.034900+0000 mon.a (mon.0) 1361 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.034957+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e186: 8 total, 8 up, 7 in 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.035894+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.59:0/3999107041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.036792+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.037509+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.59:0/3617236382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.042609+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.043133+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.034957+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e186: 8 total, 8 up, 7 in 2024-02-18T00:34:25.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.035894+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.59:0/3999107041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.036792+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:25.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.037509+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.59:0/3617236382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.042609+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.043133+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.256983+0000 osd.6 (osd.6) 99 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.258449+0000 osd.6 (osd.6) 100 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.292166+0000 osd.1 (osd.1) 101 : cluster [DBG] 2.a scrub starts 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.295046+0000 osd.1 (osd.1) 102 : cluster [DBG] 2.a scrub ok 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.373538+0000 osd.4 (osd.4) 121 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.374325+0000 osd.4 (osd.4) 122 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:23.678650+0000 mon.b (mon.1) 235 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:23.831799+0000 mgr.y (mgr.24454) 236 : cluster [DBG] pgmap v265: 272 pgs: 1 peering, 271 active+clean; 16 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1.8 KiB/s wr, 3 op/s; 32 B/s, 0 objects/s recovering 2024-02-18T00:34:25.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:24.261749+0000 osd.6 (osd.6) 101 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:24.262532+0000 osd.6 (osd.6) 102 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:24.325514+0000 osd.4 (osd.4) 123 : cluster [DBG] 4.c scrub starts 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:24.328986+0000 osd.4 (osd.4) 124 : cluster [DBG] 4.c scrub ok 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:24.333843+0000 osd.1 (osd.1) 103 : cluster [DBG] 2.d scrub starts 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:24.334807+0000 osd.1 (osd.1) 104 : cluster [DBG] 2.d scrub ok 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:24.504422+0000 mon.a (mon.0) 1360 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:24.597703+0000 mon.b (mon.1) 236 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:24.679769+0000 mon.b (mon.1) 237 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.034900+0000 mon.a (mon.0) 1361 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.034957+0000 mon.a (mon.0) 1362 : cluster [DBG] osdmap e186: 8 total, 8 up, 7 in 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.035894+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.59:0/3999107041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.036792+0000 mon.a (mon.0) 1363 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.037509+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.59:0/3617236382' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.042609+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.043133+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:25.883 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:25.885 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:25.885 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:25.986 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:26.609 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-18T00:34:26.609 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: seed 82981 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (4191 ms) 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4613 ms) 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2927 ms) 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2016 ms) 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4853 ms) 2024-02-18T00:34:26.610 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (18600 ms total) 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4736 ms) 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (5043 ms) 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (8613 ms) 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3816 ms) 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2292 ms) 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-18T00:34:26.611 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi059-82981-7 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17702 ms) 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (42203 ms total) 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5029 ms) 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5037 ms total) 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-18T00:34:26.612 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2570 ms) 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2008 ms) 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2009 ms) 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1609 ms) 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8196 ms total) 2024-02-18T00:34:26.613 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: 2024-02-18T00:34:26.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.383578+0000 osd.1 (osd.1) 105 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.385560+0000 osd.1 (osd.1) 106 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.590390+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi059-82877-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.590566+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.590691+0000 mon.a (mon.0) 1368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.590771+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.590845+0000 mon.a (mon.0) 1370 : cluster [DBG] osdmap e187: 8 total, 8 up, 7 in 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.593215+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.596354+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:25.680891+0000 mon.b (mon.1) 239 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.682634+0000 osd.3 (osd.3) 119 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.683436+0000 osd.3 (osd.3) 120 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:34:26.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.832863+0000 mgr.y (mgr.24454) 237 : cluster [DBG] pgmap v268: 336 pgs: 8 creating+peering, 2 clean+premerge+peered, 326 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 3.4 KiB/s rd, 2 op/s 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.960398+0000 osd.5 (osd.5) 95 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:25.961297+0000 osd.5 (osd.5) 96 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.383578+0000 osd.1 (osd.1) 105 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.385560+0000 osd.1 (osd.1) 106 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.590390+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi059-82877-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.590566+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.590691+0000 mon.a (mon.0) 1368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.590771+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:26.815 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.590845+0000 mon.a (mon.0) 1370 : cluster [DBG] osdmap e187: 8 total, 8 up, 7 in 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.593215+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.596354+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:25.680891+0000 mon.b (mon.1) 239 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.682634+0000 osd.3 (osd.3) 119 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.683436+0000 osd.3 (osd.3) 120 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.832863+0000 mgr.y (mgr.24454) 237 : cluster [DBG] pgmap v268: 336 pgs: 8 creating+peering, 2 clean+premerge+peered, 326 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 3.4 KiB/s rd, 2 op/s 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.960398+0000 osd.5 (osd.5) 95 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:34:26.816 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:25.961297+0000 osd.5 (osd.5) 96 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.383578+0000 osd.1 (osd.1) 105 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.385560+0000 osd.1 (osd.1) 106 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.590390+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi059-82877-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.590566+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.590691+0000 mon.a (mon.0) 1368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi059-82882-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.590771+0000 mon.a (mon.0) 1369 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi059-82949-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.590845+0000 mon.a (mon.0) 1370 : cluster [DBG] osdmap e187: 8 total, 8 up, 7 in 2024-02-18T00:34:27.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.593215+0000 mon.b (mon.1) 238 : audit [INF] from='client.? 172.21.15.59:0/2278224535' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.596354+0000 mon.a (mon.0) 1371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]: dispatch 2024-02-18T00:34:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:25.680891+0000 mon.b (mon.1) 239 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.682634+0000 osd.3 (osd.3) 119 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:34:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.683436+0000 osd.3 (osd.3) 120 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:34:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.832863+0000 mgr.y (mgr.24454) 237 : cluster [DBG] pgmap v268: 336 pgs: 8 creating+peering, 2 clean+premerge+peered, 326 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 3.4 KiB/s rd, 2 op/s 2024-02-18T00:34:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.960398+0000 osd.5 (osd.5) 95 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:34:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:25.961297+0000 osd.5 (osd.5) 96 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:34:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 -- 172.21.15.59:0/495562434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 msgr2=0x7fecac106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/495562434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac106a70 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 rx=0x7feca4009be0 tx=0x7feca4004d10).stop 2024-02-18T00:34:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 -- 172.21.15.59:0/495562434 shutdown_connections 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/495562434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecac0f6b20 0x7fecac0f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/495562434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac106a70 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/495562434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecac0f73f0 0x7fecac104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 -- 172.21.15.59:0/495562434 >> 172.21.15.59:0/495562434 conn(0x7fecac0f2820 msgr2=0x7fecac0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 -- 172.21.15.59:0/495562434 shutdown_connections 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.355+0000 7fecb1b4b700 1 -- 172.21.15.59:0/495562434 wait complete. 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 librados: starting msgr at 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 librados: starting objecter 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 Processor -- start 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 -- start start 2024-02-18T00:34:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 librados: setting wanted keys 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 librados: calling monclient init 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecac0f6b20 0x7fecac110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecac0f73f0 0x7fecac1105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac10a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac10a790 con 0x7fecac0f73f0 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac10a900 con 0x7fecac0f6b20 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.356+0000 7fecb1b4b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac10aac0 con 0x7fecac104680 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac10a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecaaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecac0f73f0 0x7fecac1105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac10a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51048/0 (socket says 172.21.15.59:51048) 2024-02-18T00:34:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 -- 172.21.15.59:0/68398168 learned_addr learned my addr 172.21.15.59:0/68398168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecab7fe700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecac0f6b20 0x7fecac110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 -- 172.21.15.59:0/68398168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecac0f6b20 msgr2=0x7fecac110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecac0f6b20 0x7fecac110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 -- 172.21.15.59:0/68398168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecac0f73f0 msgr2=0x7fecac1105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecac0f73f0 0x7fecac1105f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 -- 172.21.15.59:0/68398168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca40097c0 con 0x7fecac104680 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7feca8ff9700 10 client.?.objecter ms_handle_connect 0x7fecac104680 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecabfff700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac10a2a0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 rx=0x7feca4004ed0 tx=0x7feca4004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7feca8ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7feca8ff9700 1 -- 172.21.15.59:0/68398168 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca400e040 con 0x7fecac104680 2024-02-18T00:34:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac10ad40 con 0x7fecac104680 2024-02-18T00:34:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7feca8ff9700 1 -- 172.21.15.59:0/68398168 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca4018040 con 0x7fecac104680 2024-02-18T00:34:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7feca8ff9700 1 -- 172.21.15.59:0/68398168 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca401c3f0 con 0x7fecac104680 2024-02-18T00:34:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecb1b4b700 10 client.40430.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac116f30 con 0x7fecac104680 2024-02-18T00:34:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.357+0000 7fecb1b4b700 1 librados: init done 2024-02-18T00:34:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.358+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac05fa60 con 0x7fecac104680 2024-02-18T00:34:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7feca8ff9700 1 -- 172.21.15.59:0/68398168 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7feca401c9f0 con 0x7fecac104680 2024-02-18T00:34:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7feca8ff9700 1 --2- 172.21.15.59:0/68398168 >> v2:172.21.15.59:6800/3721891797 conn(0x7fec94062c90 0x7fec94065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7feca8ff9700 1 -- 172.21.15.59:0/68398168 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(188..188 src has 44..188) v4 ==== 8781+0+0 (secure 0 0 0) 0x7feca407e760 con 0x7fecac104680 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7feca8ff9700 10 client.40430.objecter ms_dispatch 0x7fecac001550 osd_map(188..188 src has 44..188) v4 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7feca8ff9700 3 client.40430.objecter handle_osd_map got epochs [188,188] > 0 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7feca8ff9700 3 client.40430.objecter handle_osd_map decoding full epoch 188 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7fecab7fe700 1 --2- 172.21.15.59:0/68398168 >> v2:172.21.15.59:6800/3721891797 conn(0x7fec94062c90 0x7fec94065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.360+0000 7feca8ff9700 20 client.40430.objecter dump_active .. 0 homeless 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.361+0000 7feca8ff9700 10 client.40430.objecter ms_handle_connect 0x7fec94062c90 2024-02-18T00:34:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.361+0000 7fecab7fe700 1 --2- 172.21.15.59:0/68398168 >> v2:172.21.15.59:6800/3721891797 conn(0x7fec94062c90 0x7fec94065150 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 rx=0x7fec9c004e60 tx=0x7fec9c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:27.366 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:27.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.365+0000 7feca8ff9700 1 -- 172.21.15.59:0/68398168 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feca4051a10 con 0x7fecac104680 2024-02-18T00:34:27.369 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:27.370 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:27.471 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:27.502 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:27.524 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:27.526 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:27.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:27.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.553+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fecac1093f0 con 0x7fecac104680 2024-02-18T00:34:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.621+0000 7feca8ff9700 1 -- 172.21.15.59:0/68398168 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v189) v1 ==== 101+0+0 (secure 0 0 0) 0x7feca4021030 con 0x7fecac104680 2024-02-18T00:34:27.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.628+0000 7fecb1b4b700 10 librados: watch_flush enter 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.628+0000 7fecb1b4b700 10 librados: watch_flush exit 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.628+0000 7fecb1b4b700 20 client.40430.objecter shutdown clearing up homeless session... 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.628+0000 7fecb1b4b700 10 client.40430.objecter successfully canceled tick 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.629+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 >> v2:172.21.15.59:6800/3721891797 conn(0x7fec94062c90 msgr2=0x7fec94065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.629+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/68398168 >> v2:172.21.15.59:6800/3721891797 conn(0x7fec94062c90 0x7fec94065150 secure :-1 s=READY pgs=2984 cs=0 l=1 rev1=1 rx=0x7fec9c004e60 tx=0x7fec9c00b040).stop 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.629+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 msgr2=0x7fecac10a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.629+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac10a2a0 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 rx=0x7feca4004ed0 tx=0x7feca4004d10).stop 2024-02-18T00:34:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 shutdown_connections 2024-02-18T00:34:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/68398168 >> v2:172.21.15.59:6800/3721891797 conn(0x7fec94062c90 0x7fec94065150 unknown :-1 s=CLOSED pgs=2984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecac0f6b20 0x7fecac110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecac104680 0x7fecac10a2a0 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 --2- 172.21.15.59:0/68398168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecac0f73f0 0x7fecac1105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 >> 172.21.15.59:0/68398168 conn(0x7fecac0f2820 msgr2=0x7fecac0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 shutdown_connections 2024-02-18T00:34:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 -- 172.21.15.59:0/68398168 wait complete. 2024-02-18T00:34:27.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.630+0000 7fecb1b4b700 1 librados: shutdown 2024-02-18T00:34:27.634 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:34:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 -- 172.21.15.59:0/351909185 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1540692d0 msgr2=0x7fb154069750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 --2- 172.21.15.59:0/351909185 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1540692d0 0x7fb154069750 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 rx=0x7fb1480047b0 tx=0x7fb148009cb0).stop 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 -- 172.21.15.59:0/351909185 shutdown_connections 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 --2- 172.21.15.59:0/351909185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1541001a0 0x7fb154100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 --2- 172.21.15.59:0/351909185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 0x7fb15410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 --2- 172.21.15.59:0/351909185 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1540692d0 0x7fb154069750 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 -- 172.21.15.59:0/351909185 >> 172.21.15.59:0/351909185 conn(0x7fb154063db0 msgr2=0x7fb154067560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 -- 172.21.15.59:0/351909185 shutdown_connections 2024-02-18T00:34:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 -- 172.21.15.59:0/351909185 wait complete. 2024-02-18T00:34:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 librados: starting msgr at 2024-02-18T00:34:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 librados: starting objecter 2024-02-18T00:34:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.640+0000 7fb159cd7700 1 Processor -- start 2024-02-18T00:34:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 -- start start 2024-02-18T00:34:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 librados: setting wanted keys 2024-02-18T00:34:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 librados: calling monclient init 2024-02-18T00:34:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1540692d0 0x7fb154100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 0x7fb1541014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1541001a0 0x7fb154101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15411f020 con 0x7fb154069c40 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15411f190 con 0x7fb1540692d0 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb159cd7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15411f350 con 0x7fb1541001a0 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb152ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 0x7fb1541014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb153fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1541001a0 0x7fb154101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb152ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 0x7fb1541014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34068/0 (socket says 172.21.15.59:34068) 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb152ffd700 1 -- 172.21.15.59:0/3444791220 learned_addr learned my addr 172.21.15.59:0/3444791220 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb153fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1541001a0 0x7fb154101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51076/0 (socket says 172.21.15.59:51076) 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb1537fe700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1540692d0 0x7fb154100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb153fff700 1 -- 172.21.15.59:0/3444791220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1540692d0 msgr2=0x7fb154100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb153fff700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1540692d0 0x7fb154100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb153fff700 1 -- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 msgr2=0x7fb1541014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb153fff700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 0x7fb1541014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb153fff700 1 -- 172.21.15.59:0/3444791220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c0097c0 con 0x7fb1541001a0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.641+0000 7fb152ffd700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 0x7fb1541014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb153fff700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1541001a0 0x7fb154101990 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 rx=0x7fb14c001df0 tx=0x7fb14c004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb150ff9700 10 client.?.objecter ms_handle_connect 0x7fb1541001a0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb150ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb150ff9700 1 -- 172.21.15.59:0/3444791220 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0099f0 con 0x7fb1541001a0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb148005040 con 0x7fb1541001a0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb150ff9700 1 -- 172.21.15.59:0/3444791220 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb14c0090d0 con 0x7fb1541001a0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb150ff9700 1 -- 172.21.15.59:0/3444791220 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c007c60 con 0x7fb1541001a0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb159cd7700 10 client.40436.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15411f990 con 0x7fb1541001a0 2024-02-18T00:34:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.642+0000 7fb159cd7700 1 librados: init done 2024-02-18T00:34:27.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.643+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb154060280 con 0x7fb1541001a0 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.644+0000 7fb150ff9700 1 -- 172.21.15.59:0/3444791220 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb14c00fad0 con 0x7fb1541001a0 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb150ff9700 1 --2- 172.21.15.59:0/3444791220 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb13c062c90 0x7fb13c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb150ff9700 1 -- 172.21.15.59:0/3444791220 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(189..189 src has 44..189) v4 ==== 9425+0+0 (secure 0 0 0) 0x7fb14c07df50 con 0x7fb1541001a0 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb150ff9700 10 client.40436.objecter ms_dispatch 0x7fb154001550 osd_map(189..189 src has 44..189) v4 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb150ff9700 3 client.40436.objecter handle_osd_map got epochs [189,189] > 0 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb150ff9700 3 client.40436.objecter handle_osd_map decoding full epoch 189 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb1537fe700 1 --2- 172.21.15.59:0/3444791220 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb13c062c90 0x7fb13c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb150ff9700 20 client.40436.objecter dump_active .. 0 homeless 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb150ff9700 10 client.40436.objecter ms_handle_connect 0x7fb13c062c90 2024-02-18T00:34:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.645+0000 7fb1537fe700 1 --2- 172.21.15.59:0/3444791220 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb13c062c90 0x7fb13c065150 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 rx=0x7fb1440048b0 tx=0x7fb14400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.649+0000 7fb150ff9700 1 -- 172.21.15.59:0/3444791220 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb14c050f40 con 0x7fb1541001a0 2024-02-18T00:34:27.655 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:27.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.059021+0000 osd.2 (osd.2) 87 : cluster [DBG] 2.e scrub starts 2024-02-18T00:34:27.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.060091+0000 osd.2 (osd.2) 88 : cluster [DBG] 2.e scrub ok 2024-02-18T00:34:27.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.273112+0000 osd.4 (osd.4) 125 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:34:27.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.276052+0000 osd.4 (osd.4) 126 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:34:27.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.296965+0000 osd.6 (osd.6) 103 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:34:27.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.297948+0000 osd.6 (osd.6) 104 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:34:27.667 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.339347+0000 osd.1 (osd.1) 107 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.340115+0000 osd.1 (osd.1) 108 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.597900+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.598061+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e188: 8 total, 8 up, 7 in 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.639886+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.640601+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.641340+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.641925+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.642975+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.644361+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:27.668 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:26.681882+0000 mon.b (mon.1) 240 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:27.669 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.699953+0000 osd.3 (osd.3) 121 : cluster [DBG] 6.b scrub starts 2024-02-18T00:34:27.669 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:26.701154+0000 osd.3 (osd.3) 122 : cluster [DBG] 6.b scrub ok 2024-02-18T00:34:27.669 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.555380+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.59:0/68398168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:27.669 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.555742+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:27.669 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.059021+0000 osd.2 (osd.2) 87 : cluster [DBG] 2.e scrub starts 2024-02-18T00:34:27.669 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.060091+0000 osd.2 (osd.2) 88 : cluster [DBG] 2.e scrub ok 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.273112+0000 osd.4 (osd.4) 125 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.276052+0000 osd.4 (osd.4) 126 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.296965+0000 osd.6 (osd.6) 103 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.297948+0000 osd.6 (osd.6) 104 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.339347+0000 osd.1 (osd.1) 107 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.340115+0000 osd.1 (osd.1) 108 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.597900+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.598061+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e188: 8 total, 8 up, 7 in 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.639886+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.640601+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.670 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.641340+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.641925+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.642975+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.644361+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:26.681882+0000 mon.b (mon.1) 240 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.699953+0000 osd.3 (osd.3) 121 : cluster [DBG] 6.b scrub starts 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:26.701154+0000 osd.3 (osd.3) 122 : cluster [DBG] 6.b scrub ok 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.555380+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.59:0/68398168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:27.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.555742+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:27.675 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:27.677 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:27.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:27.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.059021+0000 osd.2 (osd.2) 87 : cluster [DBG] 2.e scrub starts 2024-02-18T00:34:27.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.060091+0000 osd.2 (osd.2) 88 : cluster [DBG] 2.e scrub ok 2024-02-18T00:34:27.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.273112+0000 osd.4 (osd.4) 125 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:34:27.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.276052+0000 osd.4 (osd.4) 126 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.296965+0000 osd.6 (osd.6) 103 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.297948+0000 osd.6 (osd.6) 104 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.339347+0000 osd.1 (osd.1) 107 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.340115+0000 osd.1 (osd.1) 108 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.597900+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi059-82981-15"}]': finished 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.598061+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e188: 8 total, 8 up, 7 in 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.639886+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.640601+0000 mon.a (mon.0) 1374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.641340+0000 mon.c (mon.2) 120 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.641925+0000 mon.a (mon.0) 1375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:27.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.642975+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:27.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.644361+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:27.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:26.681882+0000 mon.b (mon.1) 240 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:27.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.699953+0000 osd.3 (osd.3) 121 : cluster [DBG] 6.b scrub starts 2024-02-18T00:34:27.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:26.701154+0000 osd.3 (osd.3) 122 : cluster [DBG] 6.b scrub ok 2024-02-18T00:34:27.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.555380+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.59:0/68398168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:27.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.555742+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:27.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:27.809 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:27.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:27.846+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7fb15405ee80 con 0x7fb1541001a0 2024-02-18T00:34:27.910 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:27.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:27.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:28.014 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:28.043 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:28.062 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:28.065 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:28.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:28.167 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:28.283 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:28.294 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:28.297 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:28.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:28.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:28.425 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:28.584 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:28.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:28.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:28.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:28.724 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:28.744 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:28.746 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:28.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:28.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:28.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.297613+0000 osd.4 (osd.4) 127 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:34:28.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.300566+0000 osd.4 (osd.4) 128 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:34:28.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.594974+0000 mon.a (mon.0) 1378 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:34:28.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.611352+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:28.872 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.611559+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.611620+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e189: 8 total, 8 up, 7 in 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.615655+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.623486+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.625343+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.59:0/3063108600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.625363+0000 mon.a (mon.0) 1383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.626072+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.628553+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.59:0/3615606666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.630180+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.682740+0000 mon.b (mon.1) 241 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.740381+0000 osd.3 (osd.3) 123 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.741877+0000 osd.3 (osd.3) 124 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:34:28.873 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.833267+0000 mgr.y (mgr.24454) 238 : cluster [DBG] pgmap v271: 327 pgs: 64 unknown, 1 clean+premerge+peered, 262 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:34:28.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.834300+0000 mon.a (mon.0) 1386 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-18T00:34:28.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.848250+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.59:0/3444791220' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:34:28.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:27.848562+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:34:28.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.926964+0000 osd.0 (osd.0) 107 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:34:28.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:27.927867+0000 osd.0 (osd.0) 108 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:34:28.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:28.582466+0000 mon.a (mon.0) 1388 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:28.874 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:28.892 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:28.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:28.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:28.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.297613+0000 osd.4 (osd.4) 127 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:34:28.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.300566+0000 osd.4 (osd.4) 128 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:34:28.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.594974+0000 mon.a (mon.0) 1378 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:34:28.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.611352+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:28.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.611559+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:28.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.611620+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e189: 8 total, 8 up, 7 in 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.615655+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.623486+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.625343+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.59:0/3063108600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.625363+0000 mon.a (mon.0) 1383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.626072+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.628553+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.59:0/3615606666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.630180+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.682740+0000 mon.b (mon.1) 241 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.740381+0000 osd.3 (osd.3) 123 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.741877+0000 osd.3 (osd.3) 124 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.833267+0000 mgr.y (mgr.24454) 238 : cluster [DBG] pgmap v271: 327 pgs: 64 unknown, 1 clean+premerge+peered, 262 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.834300+0000 mon.a (mon.0) 1386 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-18T00:34:28.924 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.848250+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.59:0/3444791220' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:27.848562+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.926964+0000 osd.0 (osd.0) 107 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:27.927867+0000 osd.0 (osd.0) 108 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:28.582466+0000 mon.a (mon.0) 1388 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.297613+0000 osd.4 (osd.4) 127 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.300566+0000 osd.4 (osd.4) 128 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.594974+0000 mon.a (mon.0) 1378 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.611352+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:28.925 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.611559+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.611620+0000 mon.a (mon.0) 1381 : cluster [DBG] osdmap e189: 8 total, 8 up, 7 in 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.615655+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.623486+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.625343+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.59:0/3063108600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.625363+0000 mon.a (mon.0) 1383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.626072+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.628553+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.59:0/3615606666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.630180+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.682740+0000 mon.b (mon.1) 241 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.740381+0000 osd.3 (osd.3) 123 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.741877+0000 osd.3 (osd.3) 124 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.833267+0000 mgr.y (mgr.24454) 238 : cluster [DBG] pgmap v271: 327 pgs: 64 unknown, 1 clean+premerge+peered, 262 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.834300+0000 mon.a (mon.0) 1386 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-18T00:34:28.926 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.848250+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.59:0/3444791220' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:34:28.927 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:27.848562+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:34:28.927 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.926964+0000 osd.0 (osd.0) 107 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:34:28.927 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:27.927867+0000 osd.0 (osd.0) 108 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:34:28.927 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:28.582466+0000 mon.a (mon.0) 1388 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:28.995 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:29.022 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:29.040 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:29.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:29.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:29.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.070+0000 7fb150ff9700 1 -- 172.21.15.59:0/3444791220 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v190) v1 ==== 113+0+0 (secure 0 0 0) 0x7fb14c0191b0 con 0x7fb1541001a0 2024-02-18T00:34:29.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 10 librados: watch_flush enter 2024-02-18T00:34:29.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 10 librados: watch_flush exit 2024-02-18T00:34:29.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 20 client.40436.objecter shutdown clearing up homeless session... 2024-02-18T00:34:29.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 10 client.40436.objecter successfully canceled tick 2024-02-18T00:34:29.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb13c062c90 msgr2=0x7fb13c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:29.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 --2- 172.21.15.59:0/3444791220 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb13c062c90 0x7fb13c065150 secure :-1 s=READY pgs=2985 cs=0 l=1 rev1=1 rx=0x7fb1440048b0 tx=0x7fb14400b040).stop 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1541001a0 msgr2=0x7fb154101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1541001a0 0x7fb154101990 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 rx=0x7fb14c001df0 tx=0x7fb14c004d10).stop 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 shutdown_connections 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 --2- 172.21.15.59:0/3444791220 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb13c062c90 0x7fb13c065150 unknown :-1 s=CLOSED pgs=2985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1540692d0 0x7fb154100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1541001a0 0x7fb154101990 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 --2- 172.21.15.59:0/3444791220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb154069c40 0x7fb1541014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.074+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 >> 172.21.15.59:0/3444791220 conn(0x7fb154063db0 msgr2=0x7fb154105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.075+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 shutdown_connections 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.075+0000 7fb159cd7700 1 -- 172.21.15.59:0/3444791220 wait complete. 2024-02-18T00:34:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:29.075+0000 7fb159cd7700 1 librados: shutdown 2024-02-18T00:34:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.1 to 1 (10000) 2024-02-18T00:34:29.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:29.173 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [----------] Global test environment set-up. 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.TooBig (12027 ms) 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (5413 ms) 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-18T00:34:29.669 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (4783 ms) 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3470 ms) 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3414 ms) 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3441 ms) 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3595 ms) 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3307 ms) 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3947 ms) 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-18T00:34:29.670 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3319 ms) 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2948 ms) 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3082 ms) 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3098 ms) 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2851 ms) 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.Flush (3031 ms) 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-18T00:34:29.671 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2973 ms) 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3017 ms) 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2993 ms) 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (2991 ms) 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3017 ms) 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3004 ms) 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3420 ms) 2024-02-18T00:34:29.672 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (2918 ms) 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3025 ms) 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3013 ms) 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [----------] 25 tests from LibRadosAio (92097 ms total) 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-18T00:34:29.673 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (6680 ms) 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:28.114319+0000 osd.2 (osd.2) 89 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:28.115140+0000 osd.2 (osd.2) 90 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:28.289018+0000 osd.6 (osd.6) 105 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:28.290860+0000 osd.6 (osd.6) 106 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:28.622620+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:28.684038+0000 mon.b (mon.1) 242 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:28.725530+0000 osd.3 (osd.3) 125 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:28.730951+0000 osd.3 (osd.3) 126 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.070875+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.070952+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:29.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.071050+0000 mon.a (mon.0) 1392 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.071122+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:29.071201+0000 mon.a (mon.0) 1394 : cluster [DBG] osdmap e190: 8 total, 8 up, 7 in 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.072156+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:28.114319+0000 osd.2 (osd.2) 89 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:28.115140+0000 osd.2 (osd.2) 90 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:28.289018+0000 osd.6 (osd.6) 105 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:28.290860+0000 osd.6 (osd.6) 106 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:28.622620+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:28.684038+0000 mon.b (mon.1) 242 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:28.725530+0000 osd.3 (osd.3) 125 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:34:29.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:28.730951+0000 osd.3 (osd.3) 126 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:34:29.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.070875+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:29.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.070952+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:29.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.071050+0000 mon.a (mon.0) 1392 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-18T00:34:29.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.071122+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:34:29.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:29.071201+0000 mon.a (mon.0) 1394 : cluster [DBG] osdmap e190: 8 total, 8 up, 7 in 2024-02-18T00:34:29.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.072156+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:28.114319+0000 osd.2 (osd.2) 89 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:28.115140+0000 osd.2 (osd.2) 90 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:28.289018+0000 osd.6 (osd.6) 105 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:28.290860+0000 osd.6 (osd.6) 106 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:28.622620+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:28.684038+0000 mon.b (mon.1) 242 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:28.725530+0000 osd.3 (osd.3) 125 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:28.730951+0000 osd.3 (osd.3) 126 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.070875+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi059-82949-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.070952+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi059-82882-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.071050+0000 mon.a (mon.0) 1392 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-18T00:34:30.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.071122+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:34:30.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:29.071201+0000 mon.a (mon.0) 1394 : cluster [DBG] osdmap e190: 8 total, 8 up, 7 in 2024-02-18T00:34:30.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.072156+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]: dispatch 2024-02-18T00:34:30.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:29.233062+0000 osd.4 (osd.4) 129 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:29.235949+0000 osd.4 (osd.4) 130 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:29.251677+0000 osd.6 (osd.6) 107 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:29.252826+0000 osd.6 (osd.6) 108 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.626113+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.626228+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:29.626342+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e191: 8 total, 8 up, 7 in 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.666353+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.668916+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.675436+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.676022+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.678248+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.678739+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:30.646 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.684983+0000 mon.b (mon.1) 243 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:29.692920+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:29.833664+0000 mgr.y (mgr.24454) 239 : cluster [DBG] pgmap v274: 271 pgs: 8 unknown, 1 clean+premerge+peered, 262 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.127116+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.128205+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi059-82877-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:29.233062+0000 osd.4 (osd.4) 129 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:29.235949+0000 osd.4 (osd.4) 130 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:29.251677+0000 osd.6 (osd.6) 107 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:29.252826+0000 osd.6 (osd.6) 108 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.626113+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.626228+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:30.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:29.626342+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e191: 8 total, 8 up, 7 in 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.666353+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.668916+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.675436+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.676022+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.678248+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.678739+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.684983+0000 mon.b (mon.1) 243 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:29.692920+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:29.833664+0000 mgr.y (mgr.24454) 239 : cluster [DBG] pgmap v274: 271 pgs: 8 unknown, 1 clean+premerge+peered, 262 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.127116+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:30.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.128205+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi059-82877-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:30.995 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:34:30.995 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:29.233062+0000 osd.4 (osd.4) 129 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:29.235949+0000 osd.4 (osd.4) 130 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:29.251677+0000 osd.6 (osd.6) 107 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:29.252826+0000 osd.6 (osd.6) 108 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.626113+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.626228+0000 mon.a (mon.0) 1397 : audit [INF] from='client.? 172.21.15.59:0/1062093826' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi059-82877-26"}]': finished 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:29.626342+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e191: 8 total, 8 up, 7 in 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.666353+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.668916+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.675436+0000 mon.c (mon.2) 128 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.676022+0000 mon.a (mon.0) 1400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.678248+0000 mon.c (mon.2) 129 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.678739+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.684983+0000 mon.b (mon.1) 243 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:29.692920+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:29.833664+0000 mgr.y (mgr.24454) 239 : cluster [DBG] pgmap v274: 271 pgs: 8 unknown, 1 clean+premerge+peered, 262 active+clean; 15 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.127116+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.128205+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi059-82877-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:31.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:34:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:34:31.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:34:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:34:31.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 2 1 2024-02-18T00:34:32.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:32] "GET /metrics HTTP/1.1" 200 193847 "" "Prometheus/2.33.4" 2024-02-18T00:34:32.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.066340+0000 osd.2 (osd.2) 91 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.067145+0000 osd.2 (osd.2) 92 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.152476+0000 osd.2 (osd.2) 93 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.154194+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.222742+0000 osd.4 (osd.4) 131 : cluster [DBG] 6.1e deep-scrub starts 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.225621+0000 osd.4 (osd.4) 132 : cluster [DBG] 6.1e deep-scrub ok 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.316837+0000 osd.6 (osd.6) 109 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:32.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.317597+0000 osd.6 (osd.6) 110 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.633300+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.633592+0000 mon.a (mon.0) 1406 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi059-82877-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.637303+0000 mon.a (mon.0) 1407 : cluster [DBG] osdmap e192: 8 total, 8 up, 7 in 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.639566+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi059-82877-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.641051+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? 172.21.15.59:0/3324537352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi059-82882-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.645063+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.652117+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.664284+0000 osd.3 (osd.3) 127 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.666761+0000 osd.3 (osd.3) 128 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:30.685965+0000 mon.b (mon.1) 244 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.875064+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.a scrub starts 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:30.876367+0000 osd.5 (osd.5) 98 : cluster [DBG] 6.a scrub ok 2024-02-18T00:34:32.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:31.013009+0000 mgr.y (mgr.24454) 240 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:32.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.066340+0000 osd.2 (osd.2) 91 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:32.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.067145+0000 osd.2 (osd.2) 92 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:32.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.152476+0000 osd.2 (osd.2) 93 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:34:32.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.154194+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:34:32.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.222742+0000 osd.4 (osd.4) 131 : cluster [DBG] 6.1e deep-scrub starts 2024-02-18T00:34:32.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.225621+0000 osd.4 (osd.4) 132 : cluster [DBG] 6.1e deep-scrub ok 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.316837+0000 osd.6 (osd.6) 109 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.317597+0000 osd.6 (osd.6) 110 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.633300+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.633592+0000 mon.a (mon.0) 1406 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi059-82877-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.637303+0000 mon.a (mon.0) 1407 : cluster [DBG] osdmap e192: 8 total, 8 up, 7 in 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.639566+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi059-82877-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.641051+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? 172.21.15.59:0/3324537352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi059-82882-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:32.819 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.645063+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:32.820 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.652117+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:32.820 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.664284+0000 osd.3 (osd.3) 127 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:34:32.820 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.666761+0000 osd.3 (osd.3) 128 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:34:32.820 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:30.685965+0000 mon.b (mon.1) 244 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:32.820 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.875064+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.a scrub starts 2024-02-18T00:34:32.820 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:30.876367+0000 osd.5 (osd.5) 98 : cluster [DBG] 6.a scrub ok 2024-02-18T00:34:32.820 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:31.013009+0000 mgr.y (mgr.24454) 240 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:32.824 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:32.829 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:32.829 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:32.930 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:32.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.066340+0000 osd.2 (osd.2) 91 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:32.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.067145+0000 osd.2 (osd.2) 92 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:32.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.152476+0000 osd.2 (osd.2) 93 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.154194+0000 osd.2 (osd.2) 94 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.222742+0000 osd.4 (osd.4) 131 : cluster [DBG] 6.1e deep-scrub starts 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.225621+0000 osd.4 (osd.4) 132 : cluster [DBG] 6.1e deep-scrub ok 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.316837+0000 osd.6 (osd.6) 109 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.317597+0000 osd.6 (osd.6) 110 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.633300+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.633592+0000 mon.a (mon.0) 1406 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi059-82877-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.637303+0000 mon.a (mon.0) 1407 : cluster [DBG] osdmap e192: 8 total, 8 up, 7 in 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.639566+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi059-82877-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.641051+0000 mon.a (mon.0) 1409 : audit [INF] from='client.? 172.21.15.59:0/3324537352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi059-82882-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.645063+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.652117+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:32.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.664284+0000 osd.3 (osd.3) 127 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:34:32.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.666761+0000 osd.3 (osd.3) 128 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:34:32.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:30.685965+0000 mon.b (mon.1) 244 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:32.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.875064+0000 osd.5 (osd.5) 97 : cluster [DBG] 6.a scrub starts 2024-02-18T00:34:32.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:30.876367+0000 osd.5 (osd.5) 98 : cluster [DBG] 6.a scrub ok 2024-02-18T00:34:32.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:31.013009+0000 mgr.y (mgr.24454) 240 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:32.963 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:32.987 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:32.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:33.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/244940513 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 msgr2=0x7fdeb0106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/244940513 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb0106830 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 rx=0x7fdea8009be0 tx=0x7fdea8004d10).stop 2024-02-18T00:34:33.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/244940513 shutdown_connections 2024-02-18T00:34:33.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/244940513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb006b170 0x7fdeb006b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/244940513 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb0106830 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/244940513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb006ba40 0x7fdeb0103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/244940513 >> 172.21.15.59:0/244940513 conn(0x7fdeb00f3800 msgr2=0x7fdeb00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/244940513 shutdown_connections 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/244940513 wait complete. 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 librados: starting msgr at 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.049+0000 7fdeb6bc3700 1 librados: starting objecter 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 Processor -- start 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 -- start start 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 librados: setting wanted keys 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 librados: calling monclient init 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb006b170 0x7fdeb01101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb006ba40 0x7fdeb01106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb010a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb010a850 con 0x7fdeb006ba40 2024-02-18T00:34:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb010a9c0 con 0x7fdeb006b170 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb6bc3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb010ab80 con 0x7fdeb0104440 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb5160700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb010a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeaffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb006ba40 0x7fdeb01106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb495f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb006b170 0x7fdeb01101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb495f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb006b170 0x7fdeb01101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:43920/0 (socket says 172.21.15.59:43920) 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb5160700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb010a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43508/0 (socket says 172.21.15.59:43508) 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeaffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb006ba40 0x7fdeb01106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47350/0 (socket says 172.21.15.59:47350) 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.050+0000 7fdeb495f700 1 -- 172.21.15.59:0/4245931136 learned_addr learned my addr 172.21.15.59:0/4245931136 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb5160700 1 -- 172.21.15.59:0/4245931136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb006b170 msgr2=0x7fdeb01101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb5160700 1 --2- 172.21.15.59:0/4245931136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb006b170 0x7fdeb01101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb5160700 1 -- 172.21.15.59:0/4245931136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb006ba40 msgr2=0x7fdeb01106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb5160700 1 --2- 172.21.15.59:0/4245931136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdeb006ba40 0x7fdeb01106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb5160700 1 -- 172.21.15.59:0/4245931136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c005040 con 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb006b170 0x7fdeb01101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb5160700 1 --2- 172.21.15.59:0/4245931136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb010a360 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 rx=0x7fdea8009bb0 tx=0x7fdea8004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeadffb700 10 client.?.objecter ms_handle_connect 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeadffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea800e040 con 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea8018040 con 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8013e90 con 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea80097c0 con 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb6bc3700 10 client.40466.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb010b010 con 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.051+0000 7fdeb6bc3700 1 librados: init done 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.052+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fdeb005ee80 con 0x7fdeb0104440 2024-02-18T00:34:33.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fdea8013520 con 0x7fdeb0104440 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdea0062c90 0x7fdea0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(194..194 src has 44..194) v4 ==== 10677+0+0 (secure 0 0 0) 0x7fdea807f170 con 0x7fdeb0104440 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 osd_map(194..194 src has 44..194) v4 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 3 client.40466.objecter handle_osd_map got epochs [194,194] > 0 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 3 client.40466.objecter handle_osd_map decoding full epoch 194 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdea0062c90 0x7fdea0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 20 client.40466.objecter dump_active .. 0 homeless 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.053+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v194) v1 ==== 54+0+16 (secure 0 0 0) 0x7fdea807f650 con 0x7fdeb0104440 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 1 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdea0062c90 0x7fdea0065150 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 rx=0x7fdea4004fd0 tx=0x7fdea400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdea0062c90 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdeb00fe410 0x7fdeb00fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 1 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 0 1 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0109330 osd=0 4 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 1 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb010b840 con 0x7fdeb00fe410 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeaffff700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdeb00fe410 0x7fdeb00fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb00fe410 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.054+0000 7fdeaffff700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdeb00fe410 0x7fdeb00fe7f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.055+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdeb010b840 con 0x7fdeb00fe410 2024-02-18T00:34:33.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.055+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:34:33.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.055+0000 7fdeadffb700 10 client.40466.objecter _finish_command 1 = system:0 2024-02-18T00:34:33.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.055+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0109330 osd=0 4 2024-02-18T00:34:33.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.055+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 0 1 2024-02-18T00:34:33.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 2 2024-02-18T00:34:33.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 2 2024-02-18T00:34:33.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0109330 osd=0 3 2024-02-18T00:34:33.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 0 2 2024-02-18T00:34:33.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0109330 osd=0 4 2024-02-18T00:34:33.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 2 2024-02-18T00:34:33.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.070+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb00273c0 con 0x7fdeb00fe410 2024-02-18T00:34:33.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb00273c0 con 0x7fdeb00fe410 2024-02-18T00:34:33.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeadffb700 10 client.40466.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0109330 osd=0 4 2024-02-18T00:34:33.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 0 2 2024-02-18T00:34:33.074 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:34:33.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 3 2024-02-18T00:34:33.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6805/26279110 conn(0x7fdeb00598f0 0x7fdeb0059d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 3 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 1 3 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0028710 osd=1 4 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 3 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.072+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb011d770 con 0x7fdeb00598f0 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.073+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6805/26279110 conn(0x7fdeb00598f0 0x7fdeb0059d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.073+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb00598f0 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.073+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6805/26279110 conn(0x7fdeb00598f0 0x7fdeb0059d50 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.074+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdeb011d770 con 0x7fdeb00598f0 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.074+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.074+0000 7fdeadffb700 10 client.40466.objecter _finish_command 3 = system:0 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.074+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0028710 osd=1 4 2024-02-18T00:34:33.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.074+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 1 3 2024-02-18T00:34:33.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:33.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 4 2024-02-18T00:34:33.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 4 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0028710 osd=1 3 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 1 4 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0028710 osd=1 4 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 4 2024-02-18T00:34:33.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.100+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb0059360 con 0x7fdeb00598f0 2024-02-18T00:34:33.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb0059360 con 0x7fdeb00598f0 2024-02-18T00:34:33.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeadffb700 10 client.40466.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0028710 osd=1 4 2024-02-18T00:34:33.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 1 4 2024-02-18T00:34:33.103 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:34:33.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 5 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdeb0028270 0x7fdeb011c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.101+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 5 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 2 5 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0059770 osd=2 4 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 5 2024-02-18T00:34:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb011c680 con 0x7fdeb0028270 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeaffff700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdeb0028270 0x7fdeb011c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb0028270 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.102+0000 7fdeaffff700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdeb0028270 0x7fdeb011c190 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.103+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdeb011c680 con 0x7fdeb0028270 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.103+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.103+0000 7fdeadffb700 10 client.40466.objecter _finish_command 5 = system:0 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.103+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0059770 osd=2 4 2024-02-18T00:34:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.103+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 2 5 2024-02-18T00:34:33.118 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:33.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 6 2024-02-18T00:34:33.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 6 2024-02-18T00:34:33.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0059770 osd=2 3 2024-02-18T00:34:33.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 2 6 2024-02-18T00:34:33.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0059770 osd=2 4 2024-02-18T00:34:33.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 6 2024-02-18T00:34:33.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.124+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb00277d0 con 0x7fdeb0028270 2024-02-18T00:34:33.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb00277d0 con 0x7fdeb0028270 2024-02-18T00:34:33.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeadffb700 10 client.40466.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0059770 osd=2 4 2024-02-18T00:34:33.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 2 6 2024-02-18T00:34:33.128 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 7 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdeb011f8f0 0x7fdeb0121ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 7 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 3 7 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0027bb0 osd=3 4 2024-02-18T00:34:33.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 7 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb011d580 con 0x7fdeb011f8f0 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.126+0000 7fdeb5160700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdeb011f8f0 0x7fdeb0121ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.127+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb011f8f0 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.127+0000 7fdeb5160700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdeb011f8f0 0x7fdeb0121ce0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.128+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdeb011d580 con 0x7fdeb011f8f0 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.128+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.128+0000 7fdeadffb700 10 client.40466.objecter _finish_command 7 = system:0 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.128+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0027bb0 osd=3 4 2024-02-18T00:34:33.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.128+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 3 7 2024-02-18T00:34:33.131 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:33.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 8 2024-02-18T00:34:33.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 8 2024-02-18T00:34:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0027bb0 osd=3 3 2024-02-18T00:34:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 3 8 2024-02-18T00:34:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0027bb0 osd=3 4 2024-02-18T00:34:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 8 2024-02-18T00:34:33.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.146+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb011d580 con 0x7fdeb011f8f0 2024-02-18T00:34:33.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb011d580 con 0x7fdeb011f8f0 2024-02-18T00:34:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeadffb700 10 client.40466.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0027bb0 osd=3 4 2024-02-18T00:34:33.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 3 8 2024-02-18T00:34:33.151 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 9 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdeb011b7e0 0x7fdeb011bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 9 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 4 9 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb011d580 osd=4 4 2024-02-18T00:34:33.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 9 2024-02-18T00:34:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb0127ac0 con 0x7fdeb011b7e0 2024-02-18T00:34:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.149+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdeb011b7e0 0x7fdeb011bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.150+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb011b7e0 2024-02-18T00:34:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.150+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdeb011b7e0 0x7fdeb011bbe0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.153 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.152+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdeb0127ac0 con 0x7fdeb011b7e0 2024-02-18T00:34:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.152+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:34:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.152+0000 7fdeadffb700 10 client.40466.objecter _finish_command 9 = system:0 2024-02-18T00:34:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.152+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb011d580 osd=4 4 2024-02-18T00:34:33.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.152+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 4 9 2024-02-18T00:34:33.157 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:33.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 10 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 10 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb011d580 osd=4 3 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 4 10 2024-02-18T00:34:33.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb011d580 osd=4 4 2024-02-18T00:34:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 10 2024-02-18T00:34:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.178+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb0125360 con 0x7fdeb011b7e0 2024-02-18T00:34:33.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb0125360 con 0x7fdeb011b7e0 2024-02-18T00:34:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeadffb700 10 client.40466.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb011d580 osd=4 4 2024-02-18T00:34:33.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 4 10 2024-02-18T00:34:33.182 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:34:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 11 2024-02-18T00:34:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdeb0125a40 0x7fdeb0125ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 11 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 5 11 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0125360 osd=5 4 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 11 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb012ae40 con 0x7fdeb0125a40 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.180+0000 7fdeaffff700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdeb0125a40 0x7fdeb0125ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.181+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb0125a40 2024-02-18T00:34:33.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.181+0000 7fdeaffff700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdeb0125a40 0x7fdeb0125ea0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.181+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdeb012ae40 con 0x7fdeb0125a40 2024-02-18T00:34:33.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.182+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:34:33.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.182+0000 7fdeadffb700 10 client.40466.objecter _finish_command 11 = system:0 2024-02-18T00:34:33.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.182+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0125360 osd=5 4 2024-02-18T00:34:33.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.182+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 5 11 2024-02-18T00:34:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 12 2024-02-18T00:34:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 12 2024-02-18T00:34:33.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0125360 osd=5 3 2024-02-18T00:34:33.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 5 12 2024-02-18T00:34:33.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0125360 osd=5 4 2024-02-18T00:34:33.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 12 2024-02-18T00:34:33.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.196+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb01264b0 con 0x7fdeb0125a40 2024-02-18T00:34:33.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.198+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb01264b0 con 0x7fdeb0125a40 2024-02-18T00:34:33.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.198+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.198+0000 7fdeadffb700 10 client.40466.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.198+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0125360 osd=5 4 2024-02-18T00:34:33.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.198+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 5 12 2024-02-18T00:34:33.201 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 13 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdeb01269c0 0x7fdeb0126e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 13 2024-02-18T00:34:33.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 6 13 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb01277d0 osd=6 4 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 13 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb0127310 con 0x7fdeb01269c0 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb5160700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdeb01269c0 0x7fdeb0126e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb01269c0 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.199+0000 7fdeb5160700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdeb01269c0 0x7fdeb0126e20 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.200+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdea8034e90 con 0x7fdeb01269c0 2024-02-18T00:34:33.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.200+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:34:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.200+0000 7fdeadffb700 10 client.40466.objecter _finish_command 13 = system:0 2024-02-18T00:34:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.200+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb01277d0 osd=6 4 2024-02-18T00:34:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.200+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 6 13 2024-02-18T00:34:33.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 14 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 14 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb01277d0 osd=6 3 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 6 14 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb01277d0 osd=6 4 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 14 2024-02-18T00:34:33.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.215+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb0127530 con 0x7fdeb01269c0 2024-02-18T00:34:33.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb0127530 con 0x7fdeb01269c0 2024-02-18T00:34:33.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeadffb700 10 client.40466.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb01277d0 osd=6 4 2024-02-18T00:34:33.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 6 14 2024-02-18T00:34:33.219 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 15 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6812/53025171 conn(0x7fdeb012c1e0 0x7fdeb012e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 15 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 7 15 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0127530 osd=7 4 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 15 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb0131350 con 0x7fdeb012c1e0 2024-02-18T00:34:33.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6812/53025171 conn(0x7fdeb012c1e0 0x7fdeb012e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:33.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeadffb700 10 client.40466.objecter ms_handle_connect 0x7fdeb012c1e0 2024-02-18T00:34:33.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.217+0000 7fdeb495f700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6812/53025171 conn(0x7fdeb012c1e0 0x7fdeb012e5d0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:33.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.218+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdeb0131350 con 0x7fdeb012c1e0 2024-02-18T00:34:33.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.218+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:34:33.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.218+0000 7fdeadffb700 10 client.40466.objecter _finish_command 15 = system:0 2024-02-18T00:34:33.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.218+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0127530 osd=7 4 2024-02-18T00:34:33.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.218+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 7 15 2024-02-18T00:34:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 10 client.40466.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:34:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign -1 16 2024-02-18T00:34:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 20 client.40466.objecter _get_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_remove -1 16 2024-02-18T00:34:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 20 client.40466.objecter get_session s=0x7fdeb0127530 osd=7 3 2024-02-18T00:34:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 15 client.40466.objecter _session_command_op_assign 7 16 2024-02-18T00:34:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0127530 osd=7 4 2024-02-18T00:34:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 10 client.40466.objecter _send_command 16 2024-02-18T00:34:33.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.233+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fdeb0131350 con 0x7fdeb012c1e0 2024-02-18T00:34:33.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.235+0000 7fdeadffb700 1 -- 172.21.15.59:0/4245931136 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fdeb0131350 con 0x7fdeb012c1e0 2024-02-18T00:34:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.235+0000 7fdeadffb700 10 client.40466.objecter ms_dispatch 0x7fdeb0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:34:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.235+0000 7fdeadffb700 10 client.40466.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.235+0000 7fdeadffb700 20 client.40466.objecter put_session s=0x7fdeb0127530 osd=7 4 2024-02-18T00:34:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.235+0000 7fdeadffb700 15 client.40466.objecter _session_command_op_remove 7 16 2024-02-18T00:34:33.240 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:34:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 librados: watch_flush enter 2024-02-18T00:34:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 librados: watch_flush exit 2024-02-18T00:34:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.0 2024-02-18T00:34:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdeb00fe410 msgr2=0x7fdeb00fe7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdeb00fe410 0x7fdeb00fe7f0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0109330 osd=0 1 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.1 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6805/26279110 conn(0x7fdeb00598f0 msgr2=0x7fdeb0059d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6805/26279110 conn(0x7fdeb00598f0 0x7fdeb0059d50 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0028710 osd=1 1 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.2 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdeb0028270 msgr2=0x7fdeb011c190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdeb0028270 0x7fdeb011c190 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0059770 osd=2 1 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.3 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdeb011f8f0 msgr2=0x7fdeb0121ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdeb011f8f0 0x7fdeb0121ce0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0027bb0 osd=3 1 2024-02-18T00:34:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.4 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdeb011b7e0 msgr2=0x7fdeb011bbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdeb011b7e0 0x7fdeb011bbe0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb011d580 osd=4 1 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb5160700 1 -- 172.21.15.59:0/4245931136 reap_dead start 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.5 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdeb0125a40 msgr2=0x7fdeb0125ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdeb0125a40 0x7fdeb0125ea0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb5160700 1 -- 172.21.15.59:0/4245931136 reap_dead start 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0125360 osd=5 1 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.6 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdeb01269c0 msgr2=0x7fdeb0126e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdeb01269c0 0x7fdeb0126e20 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb01277d0 osd=6 1 2024-02-18T00:34:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter close_session for osd.7 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6812/53025171 conn(0x7fdeb012c1e0 msgr2=0x7fdeb012e5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb5160700 1 -- 172.21.15.59:0/4245931136 reap_dead start 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6812/53025171 conn(0x7fdeb012c1e0 0x7fdeb012e5d0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter put_session s=0x7fdeb0127530 osd=7 1 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 20 client.40466.objecter shutdown clearing up homeless session... 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 10 client.40466.objecter successfully canceled tick 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdea0062c90 msgr2=0x7fdea0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdea0062c90 0x7fdea0065150 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 rx=0x7fdea4004fd0 tx=0x7fdea400b040).stop 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 msgr2=0x7fdeb010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb010a360 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 rx=0x7fdea8009bb0 tx=0x7fdea8004d10).stop 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 shutdown_connections 2024-02-18T00:34:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.137:6812/53025171 conn(0x7fdeb012c1e0 0x7fdeb012e5d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdeb0104440 0x7fdeb010a360 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 --2- 172.21.15.59:0/4245931136 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdea0062c90 0x7fdea0065150 unknown :-1 s=CLOSED pgs=2990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 >> 172.21.15.59:0/4245931136 conn(0x7fdeb00f3800 msgr2=0x7fdeb00f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 shutdown_connections 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 -- 172.21.15.59:0/4245931136 wait complete. 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:33.236+0000 7fdeb6bc3700 1 librados: shutdown 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:34:33.259 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:33.311 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:33.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:33.412 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:33.441 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:33.460 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:33.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:33.464 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:33.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:33.604 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:33.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:31.240217+0000 osd.6 (osd.6) 111 : cluster [DBG] 2.1d deep-scrub starts 2024-02-18T00:34:33.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:31.241142+0000 osd.6 (osd.6) 112 : cluster [DBG] 2.1d deep-scrub ok 2024-02-18T00:34:33.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:31.407173+0000 osd.1 (osd.1) 109 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:34:33.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:31.410077+0000 osd.1 (osd.1) 110 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:34:33.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:31.686939+0000 mon.b (mon.1) 245 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:31.834071+0000 mgr.y (mgr.24454) 241 : cluster [DBG] pgmap v277: 302 pgs: 32 creating+peering, 1 peering, 269 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.020102+0000 osd.0 (osd.0) 109 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.021208+0000 osd.0 (osd.0) 110 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:32.079907+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? 172.21.15.59:0/3324537352' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi059-82882-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.168293+0000 osd.4 (osd.4) 133 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.170189+0000 osd.4 (osd.4) 134 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.217393+0000 osd.6 (osd.6) 113 : cluster [DBG] 26.0 scrub starts 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.218712+0000 osd.6 (osd.6) 114 : cluster [DBG] 26.0 scrub ok 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.415249+0000 osd.1 (osd.1) 111 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.418713+0000 osd.1 (osd.1) 112 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.507047+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e193: 8 total, 8 up, 7 in 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:32.515679+0000 mon.a (mon.0) 1413 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-18T00:34:33.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:32.635654+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi059-82877-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:32.635744+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:32.635824+0000 mon.a (mon.0) 1416 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:32.635927+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e194: 8 total, 8 up, 7 in 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:32.688035+0000 mon.b (mon.1) 246 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:33.054163+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? 172.21.15.59:0/4245931136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:31.240217+0000 osd.6 (osd.6) 111 : cluster [DBG] 2.1d deep-scrub starts 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:31.241142+0000 osd.6 (osd.6) 112 : cluster [DBG] 2.1d deep-scrub ok 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:31.407173+0000 osd.1 (osd.1) 109 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:34:33.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:31.410077+0000 osd.1 (osd.1) 110 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:31.686939+0000 mon.b (mon.1) 245 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:31.834071+0000 mgr.y (mgr.24454) 241 : cluster [DBG] pgmap v277: 302 pgs: 32 creating+peering, 1 peering, 269 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.020102+0000 osd.0 (osd.0) 109 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.021208+0000 osd.0 (osd.0) 110 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:32.079907+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? 172.21.15.59:0/3324537352' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi059-82882-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.168293+0000 osd.4 (osd.4) 133 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.170189+0000 osd.4 (osd.4) 134 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.217393+0000 osd.6 (osd.6) 113 : cluster [DBG] 26.0 scrub starts 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.218712+0000 osd.6 (osd.6) 114 : cluster [DBG] 26.0 scrub ok 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.415249+0000 osd.1 (osd.1) 111 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.418713+0000 osd.1 (osd.1) 112 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.507047+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e193: 8 total, 8 up, 7 in 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:32.515679+0000 mon.a (mon.0) 1413 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-18T00:34:33.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:32.635654+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi059-82877-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:33.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:32.635744+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:33.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:32.635824+0000 mon.a (mon.0) 1416 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-18T00:34:33.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:32.635927+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e194: 8 total, 8 up, 7 in 2024-02-18T00:34:33.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:32.688035+0000 mon.b (mon.1) 246 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:33.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:33.054163+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? 172.21.15.59:0/4245931136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:33.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:31.240217+0000 osd.6 (osd.6) 111 : cluster [DBG] 2.1d deep-scrub starts 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:31.241142+0000 osd.6 (osd.6) 112 : cluster [DBG] 2.1d deep-scrub ok 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:31.407173+0000 osd.1 (osd.1) 109 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:31.410077+0000 osd.1 (osd.1) 110 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:31.686939+0000 mon.b (mon.1) 245 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:31.834071+0000 mgr.y (mgr.24454) 241 : cluster [DBG] pgmap v277: 302 pgs: 32 creating+peering, 1 peering, 269 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.020102+0000 osd.0 (osd.0) 109 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.021208+0000 osd.0 (osd.0) 110 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:32.079907+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? 172.21.15.59:0/3324537352' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi059-82882-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.168293+0000 osd.4 (osd.4) 133 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.170189+0000 osd.4 (osd.4) 134 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.217393+0000 osd.6 (osd.6) 113 : cluster [DBG] 26.0 scrub starts 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.218712+0000 osd.6 (osd.6) 114 : cluster [DBG] 26.0 scrub ok 2024-02-18T00:34:33.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.415249+0000 osd.1 (osd.1) 111 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.418713+0000 osd.1 (osd.1) 112 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.507047+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e193: 8 total, 8 up, 7 in 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:32.515679+0000 mon.a (mon.0) 1413 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:32.635654+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi059-82877-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:32.635744+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi059-82949-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:32.635824+0000 mon.a (mon.0) 1416 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:32.635927+0000 mon.a (mon.0) 1417 : cluster [DBG] osdmap e194: 8 total, 8 up, 7 in 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:32.688035+0000 mon.b (mon.1) 246 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:33.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:33.054163+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? 172.21.15.59:0/4245931136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:34.055 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:34:34.119 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:34.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:33.254984+0000 osd.2 (osd.2) 95 : cluster [DBG] 5.d scrub starts 2024-02-18T00:34:34.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:33.256468+0000 osd.2 (osd.2) 96 : cluster [DBG] 5.d scrub ok 2024-02-18T00:34:34.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:33.583892+0000 mon.a (mon.0) 1418 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:34.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:33.590273+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e195: 8 total, 8 up, 7 in 2024-02-18T00:34:34.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:33.601699+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.59:0/157508739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:34.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:33.603309+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:33.689179+0000 mon.b (mon.1) 247 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:33.834813+0000 mgr.y (mgr.24454) 242 : cluster [DBG] pgmap v280: 318 pgs: 48 unknown, 1 peering, 269 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:33.254984+0000 osd.2 (osd.2) 95 : cluster [DBG] 5.d scrub starts 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:33.256468+0000 osd.2 (osd.2) 96 : cluster [DBG] 5.d scrub ok 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:33.583892+0000 mon.a (mon.0) 1418 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:33.590273+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e195: 8 total, 8 up, 7 in 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:33.601699+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.59:0/157508739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:33.603309+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:33.689179+0000 mon.b (mon.1) 247 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:34.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:33.834813+0000 mgr.y (mgr.24454) 242 : cluster [DBG] pgmap v280: 318 pgs: 48 unknown, 1 peering, 269 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:33.254984+0000 osd.2 (osd.2) 95 : cluster [DBG] 5.d scrub starts 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:33.256468+0000 osd.2 (osd.2) 96 : cluster [DBG] 5.d scrub ok 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:33.583892+0000 mon.a (mon.0) 1418 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:33.590273+0000 mon.a (mon.0) 1419 : cluster [DBG] osdmap e195: 8 total, 8 up, 7 in 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:33.601699+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.59:0/157508739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:33.603309+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:33.689179+0000 mon.b (mon.1) 247 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:33.834813+0000 mgr.y (mgr.24454) 242 : cluster [DBG] pgmap v280: 318 pgs: 48 unknown, 1 peering, 269 active+clean; 23 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T00:34:35.381 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:34:35.381 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:34:35.381 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:34:35.381 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.206439+0000 osd.4 (osd.4) 135 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.207293+0000 osd.4 (osd.4) 136 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.228103+0000 osd.2 (osd.2) 97 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.229319+0000 osd.2 (osd.2) 98 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.352281+0000 osd.1 (osd.1) 113 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.355412+0000 osd.1 (osd.1) 114 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:34.598376+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.598538+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e196: 8 total, 8 up, 7 in 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:34.602297+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:34.603081+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:34.614082+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:34.690032+0000 mon.b (mon.1) 248 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:34.706608+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:34.706911+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.776063+0000 osd.5 (osd.5) 99 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:34.779527+0000 osd.5 (osd.5) 100 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:34:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.206439+0000 osd.4 (osd.4) 135 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.207293+0000 osd.4 (osd.4) 136 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.228103+0000 osd.2 (osd.2) 97 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.229319+0000 osd.2 (osd.2) 98 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.352281+0000 osd.1 (osd.1) 113 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.355412+0000 osd.1 (osd.1) 114 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:34.598376+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.598538+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e196: 8 total, 8 up, 7 in 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:34.602297+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:34.603081+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:34.614082+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:34.690032+0000 mon.b (mon.1) 248 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:34.706608+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:34.706911+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:35.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.776063+0000 osd.5 (osd.5) 99 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:34:35.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:34.779527+0000 osd.5 (osd.5) 100 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:34:35.896 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:35.900 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:35.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:36.001 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.206439+0000 osd.4 (osd.4) 135 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.207293+0000 osd.4 (osd.4) 136 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.228103+0000 osd.2 (osd.2) 97 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.229319+0000 osd.2 (osd.2) 98 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.352281+0000 osd.1 (osd.1) 113 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.355412+0000 osd.1 (osd.1) 114 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:34.598376+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi059-82882-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.598538+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e196: 8 total, 8 up, 7 in 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:34.602297+0000 mon.c (mon.2) 133 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:34.603081+0000 mon.a (mon.0) 1423 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:34.614082+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:36.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:34.690032+0000 mon.b (mon.1) 248 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:36.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:34.706608+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:36.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:34.706911+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:36.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.776063+0000 osd.5 (osd.5) 99 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:34:36.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:34.779527+0000 osd.5 (osd.5) 100 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:34:36.033 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:36.054 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:36.057 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:36.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:36.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:36.188 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:36.209 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:36.212 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:36.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:36.313 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:36.342 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:36.362 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:36.364 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:36.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:36.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:36.498 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:36.622 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.241436+0000 osd.6 (osd.6) 115 : cluster [DBG] 26.6 scrub starts 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.243057+0000 osd.6 (osd.6) 116 : cluster [DBG] 26.6 scrub ok 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.249647+0000 osd.4 (osd.4) 137 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.250528+0000 osd.4 (osd.4) 138 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.258814+0000 osd.2 (osd.2) 99 : cluster [DBG] 5.4 deep-scrub starts 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.259760+0000 osd.2 (osd.2) 100 : cluster [DBG] 5.4 deep-scrub ok 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.602203+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.602328+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:36.623 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.602423+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:36.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplP (68243 ms total) 2024-02-18T00:34:36.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: 2024-02-18T00:34:36.628 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (119 ms) 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (119 ms total) 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (25 ms) 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (7 ms) 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (32 ms total) 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-18T00:34:36.629 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (30 ms) 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (3 ms) 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (33 ms total) 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (3 ms) 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (5 ms) 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (8 ms total) 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-18T00:34:36.630 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (978 ms) 2024-02-18T00:34:36.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (978 ms total) 2024-02-18T00:34:36.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: 2024-02-18T00:34:36.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-18T00:34:36.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (105656 ms total) 2024-02-18T00:34:36.631 INFO:tasks.workunit.client.0.smithi059.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-18T00:34:36.747 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:36.750 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:36.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:36.852 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:36.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.241436+0000 osd.6 (osd.6) 115 : cluster [DBG] 26.6 scrub starts 2024-02-18T00:34:36.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.243057+0000 osd.6 (osd.6) 116 : cluster [DBG] 26.6 scrub ok 2024-02-18T00:34:36.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.249647+0000 osd.4 (osd.4) 137 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:34:36.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.250528+0000 osd.4 (osd.4) 138 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:34:36.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.258814+0000 osd.2 (osd.2) 99 : cluster [DBG] 5.4 deep-scrub starts 2024-02-18T00:34:36.878 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.259760+0000 osd.2 (osd.2) 100 : cluster [DBG] 5.4 deep-scrub ok 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.602203+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.602328+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.602423+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.602485+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e197: 8 total, 8 up, 7 in 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.604341+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.607456+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.614361+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.615074+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.615289+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:36.879 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:35.691118+0000 mon.b (mon.1) 249 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.602485+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e197: 8 total, 8 up, 7 in 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.604341+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.607456+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.614361+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.615074+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.615289+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:35.691118+0000 mon.b (mon.1) 249 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.698975+0000 osd.3 (osd.3) 129 : cluster [DBG] 6.c scrub starts 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.700179+0000 osd.3 (osd.3) 130 : cluster [DBG] 6.c scrub ok 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.764152+0000 osd.5 (osd.5) 101 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.765055+0000 osd.5 (osd.5) 102 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:34:36.880 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:35.835659+0000 mgr.y (mgr.24454) 243 : cluster [DBG] pgmap v283: 269 pgs: 14 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 198 active+clean; 46 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.074883+0000 mon.a (mon.0) 1433 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:36.597563+0000 mon.a (mon.0) 1434 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.603292+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.698975+0000 osd.3 (osd.3) 129 : cluster [DBG] 6.c scrub starts 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.700179+0000 osd.3 (osd.3) 130 : cluster [DBG] 6.c scrub ok 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.764152+0000 osd.5 (osd.5) 101 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.765055+0000 osd.5 (osd.5) 102 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:35.835659+0000 mgr.y (mgr.24454) 243 : cluster [DBG] pgmap v283: 269 pgs: 14 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 198 active+clean; 46 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-02-18T00:34:36.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.074883+0000 mon.a (mon.0) 1433 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:36.597563+0000 mon.a (mon.0) 1434 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.603292+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.603386+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]': finished 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.603463+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.603572+0000 mon.a (mon.0) 1438 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:36.603620+0000 mon.a (mon.0) 1439 : cluster [DBG] osdmap e198: 8 total, 8 up, 7 in 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.603386+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]': finished 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.603463+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.603572+0000 mon.a (mon.0) 1438 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-18T00:34:36.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:36.603620+0000 mon.a (mon.0) 1439 : cluster [DBG] osdmap e198: 8 total, 8 up, 7 in 2024-02-18T00:34:36.883 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:36.979 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:36.982 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:36.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.241436+0000 osd.6 (osd.6) 115 : cluster [DBG] 26.6 scrub starts 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.243057+0000 osd.6 (osd.6) 116 : cluster [DBG] 26.6 scrub ok 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.249647+0000 osd.4 (osd.4) 137 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.250528+0000 osd.4 (osd.4) 138 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.258814+0000 osd.2 (osd.2) 99 : cluster [DBG] 5.4 deep-scrub starts 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.259760+0000 osd.2 (osd.2) 100 : cluster [DBG] 5.4 deep-scrub ok 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.602203+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.602328+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.602423+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:37.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.602485+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e197: 8 total, 8 up, 7 in 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.604341+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]: dispatch 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.607456+0000 mon.c (mon.2) 135 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.614361+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.59:0/1749639172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.615074+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]: dispatch 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.615289+0000 mon.a (mon.0) 1432 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]: dispatch 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:35.691118+0000 mon.b (mon.1) 249 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.698975+0000 osd.3 (osd.3) 129 : cluster [DBG] 6.c scrub starts 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.700179+0000 osd.3 (osd.3) 130 : cluster [DBG] 6.c scrub ok 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.764152+0000 osd.5 (osd.5) 101 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.765055+0000 osd.5 (osd.5) 102 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:35.835659+0000 mgr.y (mgr.24454) 243 : cluster [DBG] pgmap v283: 269 pgs: 14 active+clean+snaptrim, 57 active+clean+snaptrim_wait, 198 active+clean; 46 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.074883+0000 mon.a (mon.0) 1433 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-18T00:34:37.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:36.597563+0000 mon.a (mon.0) 1434 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:37.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.603292+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? 172.21.15.59:0/2395362722' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi059-82877-27"}]': finished 2024-02-18T00:34:37.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.603386+0000 mon.a (mon.0) 1436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-13"}]': finished 2024-02-18T00:34:37.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.603463+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi059-82949-36"}]': finished 2024-02-18T00:34:37.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.603572+0000 mon.a (mon.0) 1438 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-18T00:34:37.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:36.603620+0000 mon.a (mon.0) 1439 : cluster [DBG] osdmap e198: 8 total, 8 up, 7 in 2024-02-18T00:34:37.083 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:37.112 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:37.130 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:37.132 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:37.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:37.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:37.465 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:37.468 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:37.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:37.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-18T00:34:37.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:37.597 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:37.655 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:37.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:37.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:37.761 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:37.816 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:37.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:37.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:37.946 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:38.072 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:38.077 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:38.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:38.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:36.034353+0000 osd.0 (osd.0) 111 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:34:38.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:36.037975+0000 osd.0 (osd.0) 112 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:34:38.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.619499+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.59:0/303679292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:38.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.626344+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:38.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.634604+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:38.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:36.034353+0000 osd.0 (osd.0) 111 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:34:38.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:36.037975+0000 osd.0 (osd.0) 112 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:34:38.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.619499+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.59:0/303679292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:38.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.626344+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:38.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.634604+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.634938+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.635485+0000 mon.c (mon.2) 139 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.635853+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.636556+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.637026+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:36.692025+0000 mon.b (mon.1) 250 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:36.697177+0000 osd.3 (osd.3) 131 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:36.698872+0000 osd.3 (osd.3) 132 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:37.604649+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:36.034353+0000 osd.0 (osd.0) 111 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:36.037975+0000 osd.0 (osd.0) 112 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:34:38.205 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.619499+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.59:0/303679292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.626344+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.634604+0000 mon.c (mon.2) 138 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.634938+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.635485+0000 mon.c (mon.2) 139 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.635853+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.636556+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.637026+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:36.692025+0000 mon.b (mon.1) 250 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:36.697177+0000 osd.3 (osd.3) 131 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:36.698872+0000 osd.3 (osd.3) 132 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:37.604649+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:37.604786+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:38.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:37.604879+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e199: 8 total, 8 up, 7 in 2024-02-18T00:34:38.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:37.618727+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:37.619301+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.207 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:37.604786+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:38.207 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:37.604879+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e199: 8 total, 8 up, 7 in 2024-02-18T00:34:38.207 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:37.618727+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.207 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:37.619301+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.207 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:38.317 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.634938+0000 mon.a (mon.0) 1441 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.635485+0000 mon.c (mon.2) 139 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.635853+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.636556+0000 mon.c (mon.2) 140 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.637026+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:36.692025+0000 mon.b (mon.1) 250 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:36.697177+0000 osd.3 (osd.3) 131 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:36.698872+0000 osd.3 (osd.3) 132 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:37.604649+0000 mon.a (mon.0) 1444 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi059-82882-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:37.604786+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi059-82877-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:37.604879+0000 mon.a (mon.0) 1446 : cluster [DBG] osdmap e199: 8 total, 8 up, 7 in 2024-02-18T00:34:38.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:37.618727+0000 mon.c (mon.2) 141 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:37.619301+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:38.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 -- 172.21.15.59:0/3868017927 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fdbb0 msgr2=0x7f3a500fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 --2- 172.21.15.59:0/3868017927 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fdbb0 0x7f3a500fdf90 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 rx=0x7f3a400047b0 tx=0x7f3a40009cb0).stop 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 -- 172.21.15.59:0/3868017927 shutdown_connections 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 --2- 172.21.15.59:0/3868017927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a500f8420 0x7f3a50104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 --2- 172.21.15.59:0/3868017927 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fdbb0 0x7f3a500fdf90 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 --2- 172.21.15.59:0/3868017927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500fe510 0x7f3a500f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 -- 172.21.15.59:0/3868017927 >> 172.21.15.59:0/3868017927 conn(0x7f3a500f3800 msgr2=0x7f3a500f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 -- 172.21.15.59:0/3868017927 shutdown_connections 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 -- 172.21.15.59:0/3868017927 wait complete. 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 librados: starting msgr at 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 librados: starting objecter 2024-02-18T00:34:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.524+0000 7f3a5545f700 1 Processor -- start 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 -- start start 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 librados: setting wanted keys 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 librados: calling monclient init 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500f8420 0x7f3a5010e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a500fdbb0 0x7f3a5010e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fe510 0x7f3a501081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501086b0 con 0x7f3a500fdbb0 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50108820 con 0x7f3a500f8420 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a5545f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501089e0 con 0x7f3a500fe510 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a4effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500f8420 0x7f3a5010e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a4effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500f8420 0x7f3a5010e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:43986/0 (socket says 172.21.15.59:43986) 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.525+0000 7f3a4effd700 1 -- 172.21.15.59:0/1402134351 learned_addr learned my addr 172.21.15.59:0/1402134351 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4f7fe700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fe510 0x7f3a501081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4effd700 1 -- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fe510 msgr2=0x7f3a501081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4e7fc700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a500fdbb0 0x7f3a5010e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4effd700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fe510 0x7f3a501081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4effd700 1 -- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a500fdbb0 msgr2=0x7f3a5010e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4effd700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a500fdbb0 0x7f3a5010e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4effd700 1 -- 172.21.15.59:0/1402134351 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a480097c0 con 0x7f3a500f8420 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4f7fe700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fe510 0x7f3a501081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4e7fc700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a500fdbb0 0x7f3a5010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a4effd700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500f8420 0x7f3a5010e020 secure :-1 s=READY pgs=4049 cs=0 l=1 rev1=1 rx=0x7f3a400044e0 tx=0x7f3a40007f90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a37fff700 10 client.?.objecter ms_handle_connect 0x7f3a500f8420 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a37fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a37fff700 1 -- 172.21.15.59:0/1402134351 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40018070 con 0x7f3a500f8420 2024-02-18T00:34:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a40005040 con 0x7f3a500f8420 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a37fff700 1 -- 172.21.15.59:0/1402134351 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a4001d420 con 0x7f3a500f8420 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a37fff700 1 -- 172.21.15.59:0/1402134351 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4001c650 con 0x7f3a500f8420 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a5545f700 10 client.43057.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a50108e80 con 0x7f3a500f8420 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.526+0000 7f3a5545f700 1 librados: init done 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.527+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a50109070 con 0x7f3a500f8420 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 1 -- 172.21.15.59:0/1402134351 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f3a40013ad0 con 0x7f3a500f8420 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 1 --2- 172.21.15.59:0/1402134351 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3a38062c90 0x7f3a38065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 1 -- 172.21.15.59:0/1402134351 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(199..199 src has 44..199) v4 ==== 9262+0+0 (secure 0 0 0) 0x7f3a4007ebd0 con 0x7f3a500f8420 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 10 client.43057.objecter ms_dispatch 0x7f3a50001550 osd_map(199..199 src has 44..199) v4 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 3 client.43057.objecter handle_osd_map got epochs [199,199] > 0 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 3 client.43057.objecter handle_osd_map decoding full epoch 199 2024-02-18T00:34:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a4e7fc700 1 --2- 172.21.15.59:0/1402134351 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3a38062c90 0x7f3a38065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 20 client.43057.objecter dump_active .. 0 homeless 2024-02-18T00:34:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a37fff700 10 client.43057.objecter ms_handle_connect 0x7f3a38062c90 2024-02-18T00:34:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.528+0000 7f3a4e7fc700 1 --2- 172.21.15.59:0/1402134351 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3a38062c90 0x7f3a38065150 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 rx=0x7f3a44009a00 tx=0x7f3a4400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.533+0000 7f3a37fff700 1 -- 172.21.15.59:0/1402134351 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3a40051bf0 con 0x7f3a500f8420 2024-02-18T00:34:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:38.741+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) v1 -- 0x7f3a50060280 con 0x7f3a500f8420 2024-02-18T00:34:38.859 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:38.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:38.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:38.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:38.991 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:39.010 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:39.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:39.027 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:39.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:39.155 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:39.416 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:39.419 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:39.423 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:39.424 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:39.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:39.552 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:39.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.616+0000 7f3a37fff700 1 -- 172.21.15.59:0/1402134351 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v201) v1 ==== 113+0+0 (secure 0 0 0) 0x7f3a400517e0 con 0x7f3a500f8420 2024-02-18T00:34:39.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.621+0000 7f3a5545f700 10 librados: watch_flush enter 2024-02-18T00:34:39.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.622+0000 7f3a5545f700 10 librados: watch_flush exit 2024-02-18T00:34:39.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.622+0000 7f3a5545f700 20 client.43057.objecter shutdown clearing up homeless session... 2024-02-18T00:34:39.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.622+0000 7f3a5545f700 10 client.43057.objecter successfully canceled tick 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3a38062c90 msgr2=0x7f3a38065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 --2- 172.21.15.59:0/1402134351 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3a38062c90 0x7f3a38065150 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 rx=0x7f3a44009a00 tx=0x7f3a4400b040).stop 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500f8420 msgr2=0x7f3a5010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500f8420 0x7f3a5010e020 secure :-1 s=READY pgs=4049 cs=0 l=1 rev1=1 rx=0x7f3a400044e0 tx=0x7f3a40007f90).stop 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 shutdown_connections 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 --2- 172.21.15.59:0/1402134351 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3a38062c90 0x7f3a38065150 secure :-1 s=CLOSED pgs=2993 cs=0 l=1 rev1=1 rx=0x7f3a44009a00 tx=0x7f3a4400b040).stop 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a500f8420 0x7f3a5010e020 unknown :-1 s=CLOSED pgs=4049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a500fe510 0x7f3a501081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 --2- 172.21.15.59:0/1402134351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a500fdbb0 0x7f3a5010e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:39.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.623+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 >> 172.21.15.59:0/1402134351 conn(0x7f3a500f3800 msgr2=0x7f3a500f5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:39.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.629+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 shutdown_connections 2024-02-18T00:34:39.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.629+0000 7f3a5545f700 1 -- 172.21.15.59:0/1402134351 wait complete. 2024-02-18T00:34:39.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:39.631+0000 7f3a5545f700 1 librados: shutdown 2024-02-18T00:34:39.638 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.2 to 1 (10000) 2024-02-18T00:34:39.744 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:37.212986+0000 osd.2 (osd.2) 101 : cluster [DBG] 26.f scrub starts 2024-02-18T00:34:39.744 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:37.216470+0000 osd.2 (osd.2) 102 : cluster [DBG] 26.f scrub ok 2024-02-18T00:34:39.744 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:37.692927+0000 mon.b (mon.1) 251 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:39.744 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:37.836060+0000 mgr.y (mgr.24454) 244 : cluster [DBG] pgmap v286: 269 pgs: 11 active+clean+snaptrim, 32 unknown, 28 active+clean+snaptrim_wait, 198 active+clean; 44 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:38.693881+0000 mon.b (mon.1) 252 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:38.743314+0000 mon.b (mon.1) 253 : audit [INF] from='client.? 172.21.15.59:0/1402134351' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.047801+0000 mon.a (mon.0) 1448 : cluster [DBG] osdmap e200: 8 total, 8 up, 7 in 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.048969+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.049137+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.049849+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:37.212986+0000 osd.2 (osd.2) 101 : cluster [DBG] 26.f scrub starts 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:37.216470+0000 osd.2 (osd.2) 102 : cluster [DBG] 26.f scrub ok 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:37.692927+0000 mon.b (mon.1) 251 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:39.745 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:37.836060+0000 mgr.y (mgr.24454) 244 : cluster [DBG] pgmap v286: 269 pgs: 11 active+clean+snaptrim, 32 unknown, 28 active+clean+snaptrim_wait, 198 active+clean; 44 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-18T00:34:39.746 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:38.693881+0000 mon.b (mon.1) 252 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:39.746 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:38.743314+0000 mon.b (mon.1) 253 : audit [INF] from='client.? 172.21.15.59:0/1402134351' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:34:39.746 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.047801+0000 mon.a (mon.0) 1448 : cluster [DBG] osdmap e200: 8 total, 8 up, 7 in 2024-02-18T00:34:39.746 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.048969+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:39.746 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.049137+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:34:39.746 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.049849+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:37.212986+0000 osd.2 (osd.2) 101 : cluster [DBG] 26.f scrub starts 2024-02-18T00:34:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:37.216470+0000 osd.2 (osd.2) 102 : cluster [DBG] 26.f scrub ok 2024-02-18T00:34:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:37.692927+0000 mon.b (mon.1) 251 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:37.836060+0000 mgr.y (mgr.24454) 244 : cluster [DBG] pgmap v286: 269 pgs: 11 active+clean+snaptrim, 32 unknown, 28 active+clean+snaptrim_wait, 198 active+clean; 44 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-18T00:34:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:38.693881+0000 mon.b (mon.1) 252 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:38.743314+0000 mon.b (mon.1) 253 : audit [INF] from='client.? 172.21.15.59:0/1402134351' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:34:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.047801+0000 mon.a (mon.0) 1448 : cluster [DBG] osdmap e200: 8 total, 8 up, 7 in 2024-02-18T00:34:40.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.048969+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:40.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.049137+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:34:40.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.049849+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 -- 172.21.15.59:0/1566593639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fdbb0 msgr2=0x7f4fc80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/1566593639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fdbb0 0x7f4fc80fdf90 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 rx=0x7f4fc40047b0 tx=0x7f4fc4009cb0).stop 2024-02-18T00:34:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 -- 172.21.15.59:0/1566593639 shutdown_connections 2024-02-18T00:34:40.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/1566593639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fc80f8420 0x7f4fc8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/1566593639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fdbb0 0x7f4fc80fdf90 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/1566593639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc80fe510 0x7f4fc80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 -- 172.21.15.59:0/1566593639 >> 172.21.15.59:0/1566593639 conn(0x7f4fc80f3800 msgr2=0x7f4fc80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 -- 172.21.15.59:0/1566593639 shutdown_connections 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 -- 172.21.15.59:0/1566593639 wait complete. 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 librados: starting msgr at 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 librados: starting objecter 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 Processor -- start 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 -- start start 2024-02-18T00:34:40.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 librados: setting wanted keys 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.356+0000 7f4fcf65c700 1 librados: calling monclient init 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcf65c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fc80f8420 0x7f4fc810e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcf65c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc80fdbb0 0x7f4fc810e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcf65c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fe510 0x7f4fc81081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcf65c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81086d0 con 0x7f4fc80f8420 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcf65c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8108840 con 0x7f4fc80fdbb0 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcf65c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8108a00 con 0x7f4fc80fe510 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fe510 0x7f4fc81081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcd3f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fc80f8420 0x7f4fc810e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fe510 0x7f4fc81081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51504/0 (socket says 172.21.15.59:51504) 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 -- 172.21.15.59:0/3842345078 learned_addr learned my addr 172.21.15.59:0/3842345078 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fccbf7700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc80fdbb0 0x7f4fc810e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 -- 172.21.15.59:0/3842345078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc80fdbb0 msgr2=0x7f4fc810e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc80fdbb0 0x7f4fc810e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 -- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fc80f8420 msgr2=0x7f4fc810e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fc80f8420 0x7f4fc810e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 -- 172.21.15.59:0/3842345078 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00097c0 con 0x7f4fc80fe510 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcd3f8700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fc80f8420 0x7f4fc810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcdbf9700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fe510 0x7f4fc81081e0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 rx=0x7f4fc0001df0 tx=0x7f4fc0004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fbe7fc700 10 client.?.objecter ms_handle_connect 0x7f4fc80fe510 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fbe7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fbe7fc700 1 -- 172.21.15.59:0/3842345078 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0009a10 con 0x7f4fc80fe510 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fbe7fc700 1 -- 172.21.15.59:0/3842345078 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc00095a0 con 0x7f4fc80fe510 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fbe7fc700 1 -- 172.21.15.59:0/3842345078 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0007e30 con 0x7f4fc80fe510 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.357+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc4005040 con 0x7f4fc80fe510 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.358+0000 7f4fcf65c700 10 client.40493.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.358+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc8109040 con 0x7f4fc80fe510 2024-02-18T00:34:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.358+0000 7f4fcf65c700 1 librados: init done 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.358+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8060280 con 0x7f4fc80fe510 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.359+0000 7f4fbe7fc700 1 -- 172.21.15.59:0/3842345078 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4fc00090d0 con 0x7f4fc80fe510 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.359+0000 7f4fbe7fc700 1 --2- 172.21.15.59:0/3842345078 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fb4062c90 0x7f4fb4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.359+0000 7f4fcd3f8700 1 --2- 172.21.15.59:0/3842345078 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fb4062c90 0x7f4fb4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.359+0000 7f4fbe7fc700 1 -- 172.21.15.59:0/3842345078 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(201..201 src has 44..201) v4 ==== 10123+0+0 (secure 0 0 0) 0x7f4fc007e260 con 0x7f4fc80fe510 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.359+0000 7f4fbe7fc700 10 client.40493.objecter ms_dispatch 0x7f4fc8001550 osd_map(201..201 src has 44..201) v4 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.359+0000 7f4fbe7fc700 3 client.40493.objecter handle_osd_map got epochs [201,201] > 0 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.359+0000 7f4fbe7fc700 3 client.40493.objecter handle_osd_map decoding full epoch 201 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.360+0000 7f4fbe7fc700 20 client.40493.objecter dump_active .. 0 homeless 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.360+0000 7f4fbe7fc700 10 client.40493.objecter ms_handle_connect 0x7f4fb4062c90 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.360+0000 7f4fcd3f8700 1 --2- 172.21.15.59:0/3842345078 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fb4062c90 0x7f4fb4065150 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 rx=0x7f4fc40044a0 tx=0x7f4fc4007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:40.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.364+0000 7f4fbe7fc700 1 -- 172.21.15.59:0/3842345078 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4fc0050fa0 con 0x7f4fc80fe510 2024-02-18T00:34:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:40.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:40.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:40.484 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:40.516 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:40.539 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:40.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:40.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:40.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.554+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f4fc81096c0 con 0x7f4fc80fe510 2024-02-18T00:34:40.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.616+0000 7f4fbe7fc700 1 -- 172.21.15.59:0/3842345078 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v202) v1 ==== 87+0+0 (secure 0 0 0) 0x7f4fc0050b90 con 0x7f4fc80fe510 2024-02-18T00:34:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 10 librados: watch_flush enter 2024-02-18T00:34:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 10 librados: watch_flush exit 2024-02-18T00:34:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 20 client.40493.objecter shutdown clearing up homeless session... 2024-02-18T00:34:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 10 client.40493.objecter successfully canceled tick 2024-02-18T00:34:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fb4062c90 msgr2=0x7f4fb4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/3842345078 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fb4062c90 0x7f4fb4065150 secure :-1 s=READY pgs=2995 cs=0 l=1 rev1=1 rx=0x7f4fc40044a0 tx=0x7f4fc4007f90).stop 2024-02-18T00:34:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fe510 msgr2=0x7f4fc81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.620+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fe510 0x7f4fc81081e0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 rx=0x7f4fc0001df0 tx=0x7f4fc0004d10).stop 2024-02-18T00:34:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.623+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 shutdown_connections 2024-02-18T00:34:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.623+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fc80fe510 0x7f4fc81081e0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.623+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fc80f8420 0x7f4fc810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.623+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/3842345078 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fb4062c90 0x7f4fb4065150 unknown :-1 s=CLOSED pgs=2995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.623+0000 7f4fcf65c700 1 --2- 172.21.15.59:0/3842345078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fc80fdbb0 0x7f4fc810e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.623+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 >> 172.21.15.59:0/3842345078 conn(0x7f4fc80f3800 msgr2=0x7f4fc80f5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.626+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 shutdown_connections 2024-02-18T00:34:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.627+0000 7f4fcf65c700 1 -- 172.21.15.59:0/3842345078 wait complete. 2024-02-18T00:34:40.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:40.628+0000 7f4fcf65c700 1 librados: shutdown 2024-02-18T00:34:40.639 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:34:40.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:40.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:38.425810+0000 osd.1 (osd.1) 115 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:34:40.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:38.428851+0000 osd.1 (osd.1) 116 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:34:40.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:38.741861+0000 osd.3 (osd.3) 133 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:34:40.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:38.743387+0000 osd.3 (osd.3) 134 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:34:40.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.018681+0000 osd.0 (osd.0) 113 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.020047+0000 osd.0 (osd.0) 114 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.365261+0000 osd.4 (osd.4) 139 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.366154+0000 osd.4 (osd.4) 140 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.450313+0000 osd.1 (osd.1) 117 : cluster [DBG] 5.c scrub starts 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.453301+0000 osd.1 (osd.1) 118 : cluster [DBG] 5.c scrub ok 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.609304+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.609523+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.609631+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.609758+0000 mon.a (mon.0) 1454 : cluster [DBG] osdmap e201: 8 total, 8 up, 7 in 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.626695+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? 172.21.15.59:0/3276465537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi059-82882-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.695048+0000 mon.b (mon.1) 254 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.757627+0000 osd.3 (osd.3) 135 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:34:40.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.758791+0000 osd.3 (osd.3) 136 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:39.836639+0000 mgr.y (mgr.24454) 245 : cluster [DBG] pgmap v289: 308 pgs: 11 active+clean+snaptrim, 72 unknown, 28 active+clean+snaptrim_wait, 197 active+clean; 44 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.837736+0000 mon.a (mon.0) 1456 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:39.893401+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:38.425810+0000 osd.1 (osd.1) 115 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:38.428851+0000 osd.1 (osd.1) 116 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:38.741861+0000 osd.3 (osd.3) 133 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:38.743387+0000 osd.3 (osd.3) 134 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.018681+0000 osd.0 (osd.0) 113 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.020047+0000 osd.0 (osd.0) 114 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.365261+0000 osd.4 (osd.4) 139 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:34:40.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.366154+0000 osd.4 (osd.4) 140 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.450313+0000 osd.1 (osd.1) 117 : cluster [DBG] 5.c scrub starts 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.453301+0000 osd.1 (osd.1) 118 : cluster [DBG] 5.c scrub ok 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.609304+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.609523+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.609631+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.609758+0000 mon.a (mon.0) 1454 : cluster [DBG] osdmap e201: 8 total, 8 up, 7 in 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.626695+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? 172.21.15.59:0/3276465537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi059-82882-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.695048+0000 mon.b (mon.1) 254 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.757627+0000 osd.3 (osd.3) 135 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.758791+0000 osd.3 (osd.3) 136 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:39.836639+0000 mgr.y (mgr.24454) 245 : cluster [DBG] pgmap v289: 308 pgs: 11 active+clean+snaptrim, 72 unknown, 28 active+clean+snaptrim_wait, 197 active+clean; 44 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.837736+0000 mon.a (mon.0) 1456 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-18T00:34:40.665 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:39.893401+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:40.675 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:41.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:38.425810+0000 osd.1 (osd.1) 115 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:34:41.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:38.428851+0000 osd.1 (osd.1) 116 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:34:41.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:38.741861+0000 osd.3 (osd.3) 133 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:34:41.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:38.743387+0000 osd.3 (osd.3) 134 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:34:41.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.018681+0000 osd.0 (osd.0) 113 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:34:41.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.020047+0000 osd.0 (osd.0) 114 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.365261+0000 osd.4 (osd.4) 139 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.366154+0000 osd.4 (osd.4) 140 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.450313+0000 osd.1 (osd.1) 117 : cluster [DBG] 5.c scrub starts 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.453301+0000 osd.1 (osd.1) 118 : cluster [DBG] 5.c scrub ok 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.609304+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi059-82877-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.609523+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.609631+0000 mon.a (mon.0) 1453 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.609758+0000 mon.a (mon.0) 1454 : cluster [DBG] osdmap e201: 8 total, 8 up, 7 in 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.626695+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? 172.21.15.59:0/3276465537' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi059-82882-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.695048+0000 mon.b (mon.1) 254 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.757627+0000 osd.3 (osd.3) 135 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:34:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.758791+0000 osd.3 (osd.3) 136 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:34:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:39.836639+0000 mgr.y (mgr.24454) 245 : cluster [DBG] pgmap v289: 308 pgs: 11 active+clean+snaptrim, 72 unknown, 28 active+clean+snaptrim_wait, 197 active+clean; 44 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.837736+0000 mon.a (mon.0) 1456 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-02-18T00:34:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:39.893401+0000 mon.a (mon.0) 1457 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:41.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 3 1 2024-02-18T00:34:41.131 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:41.135 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:41.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:41.236 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:41.268 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:41.289 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:41.291 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:34:41.292 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:34:41.292 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:41.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:41.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:41.476 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:41.480 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:41.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:41.582 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:40.243094+0000 osd.2 (osd.2) 103 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:40.244762+0000 osd.2 (osd.2) 104 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.555441+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.59:0/3842345078' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.555737+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:40.606440+0000 mon.a (mon.0) 1459 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.613067+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? 172.21.15.59:0/3276465537' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi059-82882-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.613248+0000 mon.a (mon.0) 1461 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.613300+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:40.613352+0000 mon.a (mon.0) 1463 : cluster [DBG] osdmap e202: 8 total, 8 up, 7 in 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.643224+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.643723+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:40.696395+0000 mon.b (mon.1) 255 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:41.611 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:41.023842+0000 mgr.y (mgr.24454) 246 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:41.613 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:41.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:41.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:41.740 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:40.243094+0000 osd.2 (osd.2) 103 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:40.244762+0000 osd.2 (osd.2) 104 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.555441+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.59:0/3842345078' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.555737+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:40.606440+0000 mon.a (mon.0) 1459 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.613067+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? 172.21.15.59:0/3276465537' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi059-82882-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.613248+0000 mon.a (mon.0) 1461 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.613300+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:40.613352+0000 mon.a (mon.0) 1463 : cluster [DBG] osdmap e202: 8 total, 8 up, 7 in 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.643224+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.643723+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:40.696395+0000 mon.b (mon.1) 255 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:41.023842+0000 mgr.y (mgr.24454) 246 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:40.243094+0000 osd.2 (osd.2) 103 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:40.244762+0000 osd.2 (osd.2) 104 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.555441+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.59:0/3842345078' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.555737+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:40.606440+0000 mon.a (mon.0) 1459 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.613067+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? 172.21.15.59:0/3276465537' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi059-82882-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.613248+0000 mon.a (mon.0) 1461 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "21"}]': finished 2024-02-18T00:34:41.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.613300+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:34:41.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:40.613352+0000 mon.a (mon.0) 1463 : cluster [DBG] osdmap e202: 8 total, 8 up, 7 in 2024-02-18T00:34:41.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.643224+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:41.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.643723+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:41.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:40.696395+0000 mon.b (mon.1) 255 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:41.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:41.023842+0000 mgr.y (mgr.24454) 246 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:41.768 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:41.787 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:41.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:41.790 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:41.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:41.917 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:42.485 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:42] "GET /metrics HTTP/1.1" 200 196802 "" "Prometheus/2.33.4" 2024-02-18T00:34:42.567 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:42.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:42.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:42.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:42.703 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:42.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:41.499147+0000 osd.4 (osd.4) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:41.499308+0000 osd.4 (osd.4) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:41.615335+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:41.615448+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e203: 8 total, 8 up, 7 in 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:41.617314+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:41.617590+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:41.618271+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:41.618534+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:41.697479+0000 mon.b (mon.1) 256 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:42.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:41.837067+0000 mgr.y (mgr.24454) 247 : cluster [DBG] pgmap v292: 268 pgs: 2 peering, 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 228 active+clean; 52 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:41.499147+0000 osd.4 (osd.4) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:41.499308+0000 osd.4 (osd.4) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:41.615335+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:41.615448+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e203: 8 total, 8 up, 7 in 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:41.617314+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:41.617590+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:41.618271+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:41.618534+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:41.697479+0000 mon.b (mon.1) 256 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:41.837067+0000 mgr.y (mgr.24454) 247 : cluster [DBG] pgmap v292: 268 pgs: 2 peering, 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 228 active+clean; 52 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:41.499147+0000 osd.4 (osd.4) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:41.499308+0000 osd.4 (osd.4) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:41.615335+0000 mon.a (mon.0) 1465 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:41.615448+0000 mon.a (mon.0) 1466 : cluster [DBG] osdmap e203: 8 total, 8 up, 7 in 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:41.617314+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:41.617590+0000 mon.c (mon.2) 146 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:41.618271+0000 mon.a (mon.0) 1467 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:41.618534+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:41.697479+0000 mon.b (mon.1) 256 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:43.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:41.837067+0000 mgr.y (mgr.24454) 247 : cluster [DBG] pgmap v292: 268 pgs: 2 peering, 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 228 active+clean; 52 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-18T00:34:43.054 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:43.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:43.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:43.097 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:43.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:43.188 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:43.387 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:43.390 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:43.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:43.491 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:43.519 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:43.538 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:43.540 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:43.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:43.641 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:43.667 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:43.668 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:34:43.668 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:34:43.668 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:34:43.668 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:34:43.668 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:34:43.687 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:43.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:43.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:43.694 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.622654+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]': finished 2024-02-18T00:34:43.694 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.622759+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:43.694 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:42.622908+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e204: 8 total, 8 up, 7 in 2024-02-18T00:34:43.694 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.634062+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.634822+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.634901+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.635130+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.59:0/2214316156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.635566+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.635807+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.636391+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.636808+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.698479+0000 mon.b (mon.1) 257 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:42.914403+0000 mon.a (mon.0) 1476 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.622654+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]': finished 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.622759+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:43.695 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:42.622908+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e204: 8 total, 8 up, 7 in 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.634062+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.634822+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.634901+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.635130+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.59:0/2214316156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.635566+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.635807+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.636391+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.636808+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.698479+0000 mon.b (mon.1) 257 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:42.914403+0000 mon.a (mon.0) 1476 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:43.588045+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.592919+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.593050+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.593103+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.593181+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]': finished 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:43.593240+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e205: 8 total, 8 up, 7 in 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:43.588045+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.592919+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.593050+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.593103+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:43.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.593181+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]': finished 2024-02-18T00:34:43.698 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:43.593240+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e205: 8 total, 8 up, 7 in 2024-02-18T00:34:43.790 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:44.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.622654+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-15"}]': finished 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.622759+0000 mon.a (mon.0) 1470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:42.622908+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e204: 8 total, 8 up, 7 in 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.634062+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.59:0/2213758534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.634822+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]: dispatch 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.634901+0000 mon.c (mon.2) 148 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.635130+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.59:0/2214316156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.635566+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.635807+0000 mon.a (mon.0) 1474 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:44.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.636391+0000 mon.c (mon.2) 150 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]: dispatch 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.636808+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]: dispatch 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.698479+0000 mon.b (mon.1) 257 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:42.914403+0000 mon.a (mon.0) 1476 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:43.588045+0000 mon.a (mon.0) 1477 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.592919+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi059-82877-28"}]': finished 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.593050+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.593103+0000 mon.a (mon.0) 1480 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi059-82882-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.593181+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-15", "mode": "writeback"}]': finished 2024-02-18T00:34:44.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:43.593240+0000 mon.a (mon.0) 1482 : cluster [DBG] osdmap e205: 8 total, 8 up, 7 in 2024-02-18T00:34:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:44.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:44.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:44.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:44.205 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:44.259 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:44.261 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:44.261 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:44.362 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:44.392 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:44.413 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:44.415 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:44.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:44.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:34:44.516 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:44.577 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:44.626 INFO:tasks.workunit.client.0.smithi059.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (3964 ms) 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3997 ms) 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4000 ms) 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11961 ms total) 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (113699 ms total) 2024-02-18T00:34:44.627 INFO:tasks.workunit.client.0.smithi059.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-18T00:34:44.628 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:34:44.628 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82936 2024-02-18T00:34:44.628 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82936 2024-02-18T00:34:44.629 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:34:44.629 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82928 2024-02-18T00:34:44.629 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82928 2024-02-18T00:34:44.629 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:34:44.629 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82912 2024-02-18T00:34:44.629 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82912 2024-02-18T00:34:44.670 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:44.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:44.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:44.684 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.609313+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.684 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.624326+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.684 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.638318+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.684 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.638850+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.639252+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi059-82877-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.699668+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.725635+0000 mon.a (mon.0) 1487 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:43.837468+0000 mgr.y (mgr.24454) 248 : cluster [DBG] pgmap v295: 291 pgs: 1 peering, 12 active+clean+snaptrim, 32 unknown, 19 active+clean+snaptrim_wait, 227 active+clean; 12 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 0 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:43.838607+0000 mon.a (mon.0) 1488 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.169801+0000 mon.a (mon.0) 1489 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.170526+0000 mon.a (mon.0) 1490 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.174866+0000 mon.a (mon.0) 1491 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.594696+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.594823+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi059-82877-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.594895+0000 mon.a (mon.0) 1494 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-18T00:34:44.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:44.594949+0000 mon.a (mon.0) 1495 : cluster [DBG] osdmap e206: 8 total, 8 up, 7 in 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.595874+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi059-82877-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.609313+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.624326+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.638318+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.638850+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.639252+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi059-82877-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.699668+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.725635+0000 mon.a (mon.0) 1487 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:43.837468+0000 mgr.y (mgr.24454) 248 : cluster [DBG] pgmap v295: 291 pgs: 1 peering, 12 active+clean+snaptrim, 32 unknown, 19 active+clean+snaptrim_wait, 227 active+clean; 12 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 0 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:43.838607+0000 mon.a (mon.0) 1488 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-18T00:34:44.686 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.169801+0000 mon.a (mon.0) 1489 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:34:44.687 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.170526+0000 mon.a (mon.0) 1490 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:34:44.687 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.174866+0000 mon.a (mon.0) 1491 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:44.687 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.594696+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:44.687 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.594823+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi059-82877-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:44.687 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.594895+0000 mon.a (mon.0) 1494 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-18T00:34:44.687 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:44.594949+0000 mon.a (mon.0) 1495 : cluster [DBG] osdmap e206: 8 total, 8 up, 7 in 2024-02-18T00:34:44.687 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.595874+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi059-82877-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.609313+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.59:0/1886861710' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.624326+0000 mon.a (mon.0) 1483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.638318+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.638850+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.639252+0000 mon.a (mon.0) 1486 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi059-82877-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.699668+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.725635+0000 mon.a (mon.0) 1487 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:43.837468+0000 mgr.y (mgr.24454) 248 : cluster [DBG] pgmap v295: 291 pgs: 1 peering, 12 active+clean+snaptrim, 32 unknown, 19 active+clean+snaptrim_wait, 227 active+clean; 12 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 0 B/s wr, 0 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:43.838607+0000 mon.a (mon.0) 1488 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "20"}]: dispatch 2024-02-18T00:34:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.169801+0000 mon.a (mon.0) 1489 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:34:44.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.170526+0000 mon.a (mon.0) 1490 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:34:44.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.174866+0000 mon.a (mon.0) 1491 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:44.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.594696+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi059-82981-20"}]': finished 2024-02-18T00:34:44.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.594823+0000 mon.a (mon.0) 1493 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi059-82877-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:44.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.594895+0000 mon.a (mon.0) 1494 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "20"}]': finished 2024-02-18T00:34:44.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:44.594949+0000 mon.a (mon.0) 1495 : cluster [DBG] osdmap e206: 8 total, 8 up, 7 in 2024-02-18T00:34:44.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.595874+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi059-82877-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:44.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:44.804 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:45.156 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:45.160 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:45.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:45.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:45.293 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:45.314 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:45.316 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:45.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:45.418 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:45.449 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:45.470 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:45.473 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:45.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:45.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:45.624 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:46.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-18T00:34:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:44.487941+0000 osd.0 (osd.0) 115 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:44.488307+0000 osd.0 (osd.0) 116 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.646308+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:44.700708+0000 mon.b (mon.1) 259 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:45.085160+0000 mon.a (mon.0) 1497 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:45.085870+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:44.487941+0000 osd.0 (osd.0) 115 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:44.488307+0000 osd.0 (osd.0) 116 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.646308+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:44.700708+0000 mon.b (mon.1) 259 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:45.085160+0000 mon.a (mon.0) 1497 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:45.085870+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:44.487941+0000 osd.0 (osd.0) 115 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:34:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:44.488307+0000 osd.0 (osd.0) 116 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:34:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.646308+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:44.700708+0000 mon.b (mon.1) 259 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:45.085160+0000 mon.a (mon.0) 1497 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:34:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:45.085870+0000 mon.a (mon.0) 1498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:47.076 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:47.080 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:47.080 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:47.080 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:47.181 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:47.213 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:47.234 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:47.237 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:47.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:47.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:47.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:45.702021+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:47.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:45.838412+0000 mgr.y (mgr.24454) 249 : cluster [DBG] pgmap v298: 291 pgs: 23 creating+peering, 2 clean+premerge+peered, 12 active+clean+snaptrim, 9 unknown, 44 active+clean+snaptrim_wait, 201 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:47.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.040410+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:47.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:46.040505+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e207: 8 total, 8 up, 7 in 2024-02-18T00:34:47.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.041115+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:47.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.043583+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.047787+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? 172.21.15.59:0/2634309772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi059-82882-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:46.595175+0000 mon.a (mon.0) 1503 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.607706+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi059-82877-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.607895+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]': finished 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.608008+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? 172.21.15.59:0/2634309772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi059-82882-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:46.608059+0000 mon.a (mon.0) 1507 : cluster [DBG] osdmap e208: 8 total, 8 up, 7 in 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:46.703313+0000 mon.b (mon.1) 261 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:45.702021+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:45.838412+0000 mgr.y (mgr.24454) 249 : cluster [DBG] pgmap v298: 291 pgs: 23 creating+peering, 2 clean+premerge+peered, 12 active+clean+snaptrim, 9 unknown, 44 active+clean+snaptrim_wait, 201 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.040410+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:47.367 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:46.040505+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e207: 8 total, 8 up, 7 in 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.041115+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.043583+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.047787+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? 172.21.15.59:0/2634309772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi059-82882-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:46.595175+0000 mon.a (mon.0) 1503 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.607706+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi059-82877-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.607895+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]': finished 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.608008+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? 172.21.15.59:0/2634309772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi059-82882-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:46.608059+0000 mon.a (mon.0) 1507 : cluster [DBG] osdmap e208: 8 total, 8 up, 7 in 2024-02-18T00:34:47.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:46.703313+0000 mon.b (mon.1) 261 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:47.369 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:45.702021+0000 mon.b (mon.1) 260 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:45.838412+0000 mgr.y (mgr.24454) 249 : cluster [DBG] pgmap v298: 291 pgs: 23 creating+peering, 2 clean+premerge+peered, 12 active+clean+snaptrim, 9 unknown, 44 active+clean+snaptrim_wait, 201 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.040410+0000 mon.a (mon.0) 1499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:46.040505+0000 mon.a (mon.0) 1500 : cluster [DBG] osdmap e207: 8 total, 8 up, 7 in 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.041115+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.043583+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]: dispatch 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.047787+0000 mon.a (mon.0) 1502 : audit [INF] from='client.? 172.21.15.59:0/2634309772' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi059-82882-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:46.595175+0000 mon.a (mon.0) 1503 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.607706+0000 mon.a (mon.0) 1504 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi059-82877-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.607895+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-15"}]': finished 2024-02-18T00:34:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.608008+0000 mon.a (mon.0) 1506 : audit [INF] from='client.? 172.21.15.59:0/2634309772' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi059-82882-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:46.608059+0000 mon.a (mon.0) 1507 : cluster [DBG] osdmap e208: 8 total, 8 up, 7 in 2024-02-18T00:34:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:46.703313+0000 mon.b (mon.1) 261 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:47.520 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:47.523 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:47.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [==========] Running 56 tests from 4 test suites. 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (12498 ms) 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (21249 ms) 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-18T00:34:47.620 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (7313 ms) 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3639 ms) 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3677 ms) 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3511 ms) 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2821 ms) 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3943 ms) 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3029 ms) 2024-02-18T00:34:47.621 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2984 ms) 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3006 ms) 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3012 ms) 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (2972 ms) 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3018 ms) 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3014 ms) 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-18T00:34:47.622 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3415 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3928 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3018 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (2623 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3037 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3021 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3005 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (2948 ms) 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-18T00:34:47.623 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3440 ms) 2024-02-18T00:34:47.624 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-18T00:34:47.624 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2573 ms) 2024-02-18T00:34:47.624 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-18T00:34:47.624 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2985 ms) 2024-02-18T00:34:47.624 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-18T00:34:47.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:47.686 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:48.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 -- 172.21.15.59:0/3672079578 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40fe510 msgr2=0x7ff2d40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 --2- 172.21.15.59:0/3672079578 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40fe510 0x7ff2d40f7d40 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 rx=0x7ff2c4004870 tx=0x7ff2c4009cb0).stop 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 -- 172.21.15.59:0/3672079578 shutdown_connections 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 --2- 172.21.15.59:0/3672079578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d40f8420 0x7ff2d4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 --2- 172.21.15.59:0/3672079578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 0x7ff2d40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 --2- 172.21.15.59:0/3672079578 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40fe510 0x7ff2d40f7d40 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 -- 172.21.15.59:0/3672079578 >> 172.21.15.59:0/3672079578 conn(0x7ff2d40f3800 msgr2=0x7ff2d40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 -- 172.21.15.59:0/3672079578 shutdown_connections 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 -- 172.21.15.59:0/3672079578 wait complete. 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 librados: starting msgr at 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 librados: starting objecter 2024-02-18T00:34:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 Processor -- start 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 -- start start 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 librados: setting wanted keys 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.534+0000 7ff2db71b700 1 librados: calling monclient init 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40f8420 0x7ff2d410e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 0x7ff2d410e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d40fe510 0x7ff2d4108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4108750 con 0x7ff2d40fdbb0 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d41088c0 con 0x7ff2d40fe510 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4108a80 con 0x7ff2d40f8420 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d8cb6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 0x7ff2d410e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d8cb6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 0x7ff2d410e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60760/0 (socket says 172.21.15.59:60760) 2024-02-18T00:34:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d8cb6700 1 -- 172.21.15.59:0/2871605691 learned_addr learned my addr 172.21.15.59:0/2871605691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d94b7700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40f8420 0x7ff2d410e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d9cb8700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d40fe510 0x7ff2d4108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d94b7700 1 -- 172.21.15.59:0/2871605691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d40fe510 msgr2=0x7ff2d4108260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d94b7700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d40fe510 0x7ff2d4108260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d94b7700 1 -- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 msgr2=0x7ff2d410e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d94b7700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 0x7ff2d410e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d94b7700 1 -- 172.21.15.59:0/2871605691 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d0005040 con 0x7ff2d40f8420 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d8cb6700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 0x7ff2d410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d9cb8700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d40fe510 0x7ff2d4108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2d94b7700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40f8420 0x7ff2d410e0c0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 rx=0x7ff2d0004920 tx=0x7ff2d0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2ca7fc700 10 client.?.objecter ms_handle_connect 0x7ff2d40f8420 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2ca7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2ca7fc700 1 -- 172.21.15.59:0/2871605691 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0007a40 con 0x7ff2d40f8420 2024-02-18T00:34:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2ca7fc700 1 -- 172.21.15.59:0/2871605691 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2d000f040 con 0x7ff2d40f8420 2024-02-18T00:34:48.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2ca7fc700 1 -- 172.21.15.59:0/2871605691 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0014a70 con 0x7ff2d40f8420 2024-02-18T00:34:48.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c4005040 con 0x7ff2d40f8420 2024-02-18T00:34:48.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.535+0000 7ff2db71b700 10 client.40520.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:48.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.536+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d41090c0 con 0x7ff2d40f8420 2024-02-18T00:34:48.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.536+0000 7ff2db71b700 1 librados: init done 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.536+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d41070a0 con 0x7ff2d40f8420 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2ca7fc700 1 -- 172.21.15.59:0/2871605691 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff2d0010530 con 0x7ff2d40f8420 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2ca7fc700 1 --2- 172.21.15.59:0/2871605691 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff2c0062c90 0x7ff2c0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2ca7fc700 1 -- 172.21.15.59:0/2871605691 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(209..209 src has 44..209) v4 ==== 8544+0+0 (secure 0 0 0) 0x7ff2d007ce80 con 0x7ff2d40f8420 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2ca7fc700 10 client.40520.objecter ms_dispatch 0x7ff2d4001550 osd_map(209..209 src has 44..209) v4 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2ca7fc700 3 client.40520.objecter handle_osd_map got epochs [209,209] > 0 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2ca7fc700 3 client.40520.objecter handle_osd_map decoding full epoch 209 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2d8cb6700 1 --2- 172.21.15.59:0/2871605691 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff2c0062c90 0x7ff2c0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.537+0000 7ff2ca7fc700 20 client.40520.objecter dump_active .. 0 homeless 2024-02-18T00:34:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.538+0000 7ff2d8cb6700 1 --2- 172.21.15.59:0/2871605691 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff2c0062c90 0x7ff2c0065150 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 rx=0x7ff2c4009a00 tx=0x7ff2c400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.538+0000 7ff2ca7fc700 10 client.40520.objecter ms_handle_connect 0x7ff2c0062c90 2024-02-18T00:34:48.542 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:48.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.541+0000 7ff2ca7fc700 1 -- 172.21.15.59:0/2871605691 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff2d0049f60 con 0x7ff2d40f8420 2024-02-18T00:34:48.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:48.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:48.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:48.680 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:48.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.756+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7ff2d405faf0 con 0x7ff2d40f8420 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:47.607833+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e209: 8 total, 8 up, 7 in 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:47.704787+0000 mon.b (mon.1) 262 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:47.838888+0000 mgr.y (mgr.24454) 250 : cluster [DBG] pgmap v301: 234 pgs: 1 clean+premerge+peered, 12 active+clean+snaptrim, 8 unknown, 19 active+clean+snaptrim_wait, 194 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.069995+0000 mon.a (mon.0) 1509 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:47.607833+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e209: 8 total, 8 up, 7 in 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:47.704787+0000 mon.b (mon.1) 262 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:47.838888+0000 mgr.y (mgr.24454) 250 : cluster [DBG] pgmap v301: 234 pgs: 1 clean+premerge+peered, 12 active+clean+snaptrim, 8 unknown, 19 active+clean+snaptrim_wait, 194 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:34:48.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.069995+0000 mon.a (mon.0) 1509 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-18T00:34:48.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 -- 172.21.15.59:0/3599407877 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 msgr2=0x7f2900103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 --2- 172.21.15.59:0/3599407877 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900103a20 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 rx=0x7f28f8009be0 tx=0x7f28f8004d10).stop 2024-02-18T00:34:48.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 -- 172.21.15.59:0/3599407877 shutdown_connections 2024-02-18T00:34:48.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 --2- 172.21.15.59:0/3599407877 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900103a20 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 --2- 172.21.15.59:0/3599407877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29000fcbb0 0x7f29000fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 --2- 172.21.15.59:0/3599407877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29000fd510 0x7f29000f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 -- 172.21.15.59:0/3599407877 >> 172.21.15.59:0/3599407877 conn(0x7f29000f2820 msgr2=0x7f29000f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 -- 172.21.15.59:0/3599407877 shutdown_connections 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 -- 172.21.15.59:0/3599407877 wait complete. 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 librados: starting msgr at 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.786+0000 7f2905d1c700 1 librados: starting objecter 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 Processor -- start 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 -- start start 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 librados: setting wanted keys 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 librados: calling monclient init 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29000fcbb0 0x7f2900107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29000fd510 0x7f2900107800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29001164a0 con 0x7f29000fcbb0 2024-02-18T00:34:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29001165e0 con 0x7f29000fd510 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f2905d1c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2900116750 con 0x7f29000f7400 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f28ff7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f28ff7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51578/0 (socket says 172.21.15.59:51578) 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f28ff7fe700 1 -- 172.21.15.59:0/596669088 learned_addr learned my addr 172.21.15.59:0/596669088 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.787+0000 7f28feffd700 1 --2- 172.21.15.59:0/596669088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29000fcbb0 0x7f2900107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28fffff700 1 --2- 172.21.15.59:0/596669088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29000fd510 0x7f2900107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28ff7fe700 1 -- 172.21.15.59:0/596669088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29000fd510 msgr2=0x7f2900107800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29000fd510 0x7f2900107800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28ff7fe700 1 -- 172.21.15.59:0/596669088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29000fcbb0 msgr2=0x7f2900107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29000fcbb0 0x7f2900107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28ff7fe700 1 -- 172.21.15.59:0/596669088 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4005040 con 0x7f29000f7400 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900106e20 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 rx=0x7f28f40048f0 tx=0x7f28f4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28fcff9700 10 client.?.objecter ms_handle_connect 0x7f29000f7400 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28fcff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4007b00 con 0x7f29000f7400 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28f400f040 con 0x7f29000f7400 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f80097c0 con 0x7f29000f7400 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4014a00 con 0x7f29000f7400 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f2905d1c700 10 client.40526.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2900116d90 con 0x7f29000f7400 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.788+0000 7f2905d1c700 1 librados: init done 2024-02-18T00:34:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.789+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f290005fa60 con 0x7f29000f7400 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.789+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f28f400e580 con 0x7f29000f7400 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6800/3721891797 conn(0x7f28e8062c90 0x7f28e8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(210..210 src has 44..210) v4 ==== 8879+0+0 (secure 0 0 0) 0x7f28f407d120 con 0x7f29000f7400 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 osd_map(210..210 src has 44..210) v4 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 3 client.40526.objecter handle_osd_map got epochs [210,210] > 0 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 3 client.40526.objecter handle_osd_map decoding full epoch 210 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28feffd700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6800/3721891797 conn(0x7f28e8062c90 0x7f28e8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 20 client.40526.objecter dump_active .. 0 homeless 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v210) v1 ==== 54+0+16 (secure 0 0 0) 0x7f28f4084070 con 0x7f29000f7400 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 10 client.40526.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28feffd700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6800/3721891797 conn(0x7f28e8062c90 0x7f28e8065150 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 rx=0x7f2900061700 tx=0x7f28f0009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f28e8062c90 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 1 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6801/3700238977 conn(0x7f29000fdea0 0x7f29000fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 1 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 0 1 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290005ee80 osd=0 4 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 10 client.40526.objecter _send_command 1 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29001060e0 con 0x7f29000fdea0 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.790+0000 7f28fffff700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6801/3700238977 conn(0x7f29000fdea0 0x7f29000fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.791+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f29000fdea0 2024-02-18T00:34:48.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.791+0000 7f28fffff700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6801/3700238977 conn(0x7f29000fdea0 0x7f29000fe280 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.791+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f29001060e0 con 0x7f29000fdea0 2024-02-18T00:34:48.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.791+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:34:48.939 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.791+0000 7f28fcff9700 10 client.40526.objecter _finish_command 1 = system:0 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.791+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f290005ee80 osd=0 4 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.791+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 0 1 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 10 client.40526.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 2 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 2 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f290005ee80 osd=0 3 2024-02-18T00:34:48.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 0 2 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290005ee80 osd=0 4 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 10 client.40526.objecter _send_command 2 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.806+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f29000fe9b0 con 0x7f29000fdea0 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.807+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f28f800b770 con 0x7f29000fdea0 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.807+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.807+0000 7f28fcff9700 10 client.40526.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.807+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f290005ee80 osd=0 4 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.807+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 0 2 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 10 client.40526.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 3 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6805/26279110 conn(0x7f29000598f0 0x7f2900059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 3 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 1 3 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900028540 osd=1 4 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 10 client.40526.objecter _send_command 3 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29000286b0 con 0x7f29000598f0 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6805/26279110 conn(0x7f29000598f0 0x7f2900059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f29000598f0 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.808+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6805/26279110 conn(0x7f29000598f0 0x7f2900059cf0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.809+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f29000286b0 con 0x7f29000598f0 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.809+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.809+0000 7f28fcff9700 10 client.40526.objecter _finish_command 3 = system:0 2024-02-18T00:34:48.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.809+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900028540 osd=1 4 2024-02-18T00:34:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.809+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 1 3 2024-02-18T00:34:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 10 client.40526.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 4 2024-02-18T00:34:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.943 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 4 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900028540 osd=1 3 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 1 4 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900028540 osd=1 4 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 10 client.40526.objecter _send_command 4 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.825+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2900059360 con 0x7f29000598f0 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2900059360 con 0x7f29000598f0 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f28fcff9700 10 client.40526.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900028540 osd=1 4 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 1 4 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 10 client.40526.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 5 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6809/1153193637 conn(0x7f29000280a0 0x7f290011e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 5 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 2 5 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900059770 osd=2 4 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 10 client.40526.objecter _send_command 5 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.826+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2900120e10 con 0x7f29000280a0 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28feffd700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6809/1153193637 conn(0x7f29000280a0 0x7f290011e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f29000280a0 2024-02-18T00:34:48.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28feffd700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6809/1153193637 conn(0x7f29000280a0 0x7f290011e1c0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2900120e10 con 0x7f29000280a0 2024-02-18T00:34:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:34:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28fcff9700 10 client.40526.objecter _finish_command 5 = system:0 2024-02-18T00:34:48.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900059770 osd=2 4 2024-02-18T00:34:48.946 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.827+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 2 5 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 10 client.40526.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 6 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 6 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900059770 osd=2 3 2024-02-18T00:34:48.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 2 6 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900059770 osd=2 4 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 10 client.40526.objecter _send_command 6 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.842+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f29000276c0 con 0x7f29000280a0 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f29000276c0 con 0x7f29000280a0 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f28fcff9700 10 client.40526.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900059770 osd=2 4 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 2 6 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 10 client.40526.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 7 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6813/1805700964 conn(0x7f290011f830 0x7f290011fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 7 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 3 7 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900027940 osd=3 4 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 10 client.40526.objecter _send_command 7 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.843+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2900120180 con 0x7f290011f830 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fffff700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6813/1805700964 conn(0x7f290011f830 0x7f290011fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f290011f830 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fffff700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6813/1805700964 conn(0x7f290011f830 0x7f290011fc90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2900120180 con 0x7f290011f830 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:34:48.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fcff9700 10 client.40526.objecter _finish_command 7 = system:0 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900027940 osd=3 4 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.844+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 3 7 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 10 client.40526.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 8 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 8 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900027940 osd=3 3 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 3 8 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900027940 osd=3 4 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 10 client.40526.objecter _send_command 8 2024-02-18T00:34:48.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.859+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2900120b70 con 0x7f290011f830 2024-02-18T00:34:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2900120b70 con 0x7f290011f830 2024-02-18T00:34:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28fcff9700 10 client.40526.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900027940 osd=3 4 2024-02-18T00:34:48.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 3 8 2024-02-18T00:34:48.951 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 10 client.40526.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 9 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2900123010 0x7f2900125400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 9 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 4 9 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900120b70 osd=4 4 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 10 client.40526.objecter _send_command 9 2024-02-18T00:34:48.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2900128050 con 0x7f2900123010 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2900123010 0x7f2900125400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2900123010 0x7f2900125400 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.861+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f2900123010 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.862+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2900128050 con 0x7f2900123010 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.862+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.862+0000 7f28fcff9700 10 client.40526.objecter _finish_command 9 = system:0 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.862+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900120b70 osd=4 4 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.862+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 4 9 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 10 client.40526.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 10 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 10 2024-02-18T00:34:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f2900120b70 osd=4 3 2024-02-18T00:34:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 4 10 2024-02-18T00:34:48.954 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:34:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900120b70 osd=4 4 2024-02-18T00:34:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 10 client.40526.objecter _send_command 10 2024-02-18T00:34:48.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.877+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f29001258f0 con 0x7f2900123010 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f29001258f0 con 0x7f2900123010 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f28fcff9700 10 client.40526.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f2900120b70 osd=4 4 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 4 10 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 10 client.40526.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 11 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2900125f70 0x7f2900128360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 11 2024-02-18T00:34:48.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 5 11 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f29001258f0 osd=5 4 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 10 client.40526.objecter _send_command 11 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290012b120 con 0x7f2900125f70 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.879+0000 7f28feffd700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2900125f70 0x7f2900128360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.880+0000 7f28feffd700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2900125f70 0x7f2900128360 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.880+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f2900125f70 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.881+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290012b120 con 0x7f2900125f70 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.881+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.881+0000 7f28fcff9700 10 client.40526.objecter _finish_command 11 = system:0 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.881+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f29001258f0 osd=5 4 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.881+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 5 11 2024-02-18T00:34:48.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 10 client.40526.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 12 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 12 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f29001258f0 osd=5 3 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 5 12 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f29001258f0 osd=5 4 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 10 client.40526.objecter _send_command 12 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.895+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f290012b120 con 0x7f2900125f70 2024-02-18T00:34:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f290012b120 con 0x7f2900125f70 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f28fcff9700 10 client.40526.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f29001258f0 osd=5 4 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 5 12 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 10 client.40526.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 13 2024-02-18T00:34:48.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6808/3356009390 conn(0x7f290012a440 0x7f290012a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 13 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 6 13 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012b120 osd=6 4 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 10 client.40526.objecter _send_command 13 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.897+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290012aef0 con 0x7f290012a440 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.898+0000 7f28fffff700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6808/3356009390 conn(0x7f290012a440 0x7f290012a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.898+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f290012a440 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.898+0000 7f28fffff700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6808/3356009390 conn(0x7f290012a440 0x7f290012a880 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.899+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290012aef0 con 0x7f290012a440 2024-02-18T00:34:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.899+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:34:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.899+0000 7f28fcff9700 10 client.40526.objecter _finish_command 13 = system:0 2024-02-18T00:34:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.899+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f290012b120 osd=6 4 2024-02-18T00:34:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.899+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 6 13 2024-02-18T00:34:48.960 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 10 client.40526.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 14 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 14 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f290012b120 osd=6 3 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 6 14 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012b120 osd=6 4 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 10 client.40526.objecter _send_command 14 2024-02-18T00:34:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.913+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f290012aef0 con 0x7f290012a440 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f290012aef0 con 0x7f290012a440 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f28fcff9700 10 client.40526.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f290012b120 osd=6 4 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 6 14 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 10 client.40526.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 15 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6812/53025171 conn(0x7f290012d8f0 0x7f290012dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 15 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 7 15 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012aef0 osd=7 4 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 10 client.40526.objecter _send_command 15 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f290012e360 con 0x7f290012d8f0 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.915+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6812/53025171 conn(0x7f290012d8f0 0x7f290012dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.916+0000 7f28fcff9700 10 client.40526.objecter ms_handle_connect 0x7f290012d8f0 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.916+0000 7f28ff7fe700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6812/53025171 conn(0x7f290012d8f0 0x7f290012dd50 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.916+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f290012e360 con 0x7f290012d8f0 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.916+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.916+0000 7f28fcff9700 10 client.40526.objecter _finish_command 15 = system:0 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.916+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f290012aef0 osd=7 4 2024-02-18T00:34:48.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.916+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 7 15 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 10 client.40526.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign -1 16 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 20 client.40526.objecter _get_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_remove -1 16 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 20 client.40526.objecter get_session s=0x7f290012aef0 osd=7 3 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 15 client.40526.objecter _session_command_op_assign 7 16 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012aef0 osd=7 4 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 10 client.40526.objecter _send_command 16 2024-02-18T00:34:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.931+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f290012e360 con 0x7f290012d8f0 2024-02-18T00:34:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.932+0000 7f28fcff9700 1 -- 172.21.15.59:0/596669088 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f290012e360 con 0x7f290012d8f0 2024-02-18T00:34:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.932+0000 7f28fcff9700 10 client.40526.objecter ms_dispatch 0x7f2900001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:34:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.932+0000 7f28fcff9700 10 client.40526.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.932+0000 7f28fcff9700 20 client.40526.objecter put_session s=0x7f290012aef0 osd=7 4 2024-02-18T00:34:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.932+0000 7f28fcff9700 15 client.40526.objecter _session_command_op_remove 7 16 2024-02-18T00:34:48.965 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 librados: watch_flush enter 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 librados: watch_flush exit 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.0 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6801/3700238977 conn(0x7f29000fdea0 msgr2=0x7f29000fe280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6801/3700238977 conn(0x7f29000fdea0 0x7f29000fe280 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290005ee80 osd=0 1 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.1 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6805/26279110 conn(0x7f29000598f0 msgr2=0x7f2900059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6805/26279110 conn(0x7f29000598f0 0x7f2900059cf0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900028540 osd=1 1 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.2 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6809/1153193637 conn(0x7f29000280a0 msgr2=0x7f290011e1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6809/1153193637 conn(0x7f29000280a0 0x7f290011e1c0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900059770 osd=2 1 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.3 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6813/1805700964 conn(0x7f290011f830 msgr2=0x7f290011fc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6813/1805700964 conn(0x7f290011f830 0x7f290011fc90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900027940 osd=3 1 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.4 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2900123010 msgr2=0x7f2900125400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2900123010 0x7f2900125400 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f28fffff700 1 -- 172.21.15.59:0/596669088 reap_dead start 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f2900120b70 osd=4 1 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.5 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2900125f70 msgr2=0x7f2900128360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2900125f70 0x7f2900128360 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f28fffff700 1 -- 172.21.15.59:0/596669088 reap_dead start 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f29001258f0 osd=5 1 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.6 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6808/3356009390 conn(0x7f290012a440 msgr2=0x7f290012a880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f28fffff700 1 -- 172.21.15.59:0/596669088 reap_dead start 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6808/3356009390 conn(0x7f290012a440 0x7f290012a880 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012b120 osd=6 1 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter close_session for osd.7 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6812/53025171 conn(0x7f290012d8f0 msgr2=0x7f290012dd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6812/53025171 conn(0x7f290012d8f0 0x7f290012dd50 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter put_session s=0x7f290012aef0 osd=7 1 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 20 client.40526.objecter shutdown clearing up homeless session... 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 10 client.40526.objecter successfully canceled tick 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6800/3721891797 conn(0x7f28e8062c90 msgr2=0x7f28e8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6800/3721891797 conn(0x7f28e8062c90 0x7f28e8065150 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 rx=0x7f2900061700 tx=0x7f28f0009040).stop 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 msgr2=0x7f2900106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900106e20 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 rx=0x7f28f40048f0 tx=0x7f28f4009cb0).stop 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 shutdown_connections 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6812/53025171 conn(0x7f290012d8f0 0x7f290012dd50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.137:6808/3356009390 conn(0x7f290012a440 0x7f290012a880 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29000f7400 0x7f2900106e20 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 --2- 172.21.15.59:0/596669088 >> v2:172.21.15.59:6800/3721891797 conn(0x7f28e8062c90 0x7f28e8065150 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 >> 172.21.15.59:0/596669088 conn(0x7f29000f2820 msgr2=0x7f29000f40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:48.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 shutdown_connections 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 -- 172.21.15.59:0/596669088 wait complete. 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:48.933+0000 7f2905d1c700 1 librados: shutdown 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:34:48.971 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:34:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:47.607833+0000 mon.a (mon.0) 1508 : cluster [DBG] osdmap e209: 8 total, 8 up, 7 in 2024-02-18T00:34:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:47.704787+0000 mon.b (mon.1) 262 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:47.838888+0000 mgr.y (mgr.24454) 250 : cluster [DBG] pgmap v301: 234 pgs: 1 clean+premerge+peered, 12 active+clean+snaptrim, 8 unknown, 19 active+clean+snaptrim_wait, 194 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:34:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.069995+0000 mon.a (mon.0) 1509 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "19"}]: dispatch 2024-02-18T00:34:49.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.624+0000 7ff2ca7fc700 1 -- 172.21.15.59:0/2871605691 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v211) v1 ==== 113+0+0 (secure 0 0 0) 0x7ff2d0050190 con 0x7ff2d40f8420 2024-02-18T00:34:49.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.630+0000 7ff2db71b700 10 librados: watch_flush enter 2024-02-18T00:34:49.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.618268+0000 mon.a (mon.0) 1510 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:48.618353+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e210: 8 total, 8 up, 7 in 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.622356+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.626691+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.627470+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.628332+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.59:0/1461454752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.628684+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.705843+0000 mon.b (mon.1) 263 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.756811+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.59:0/2871605691' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.757127+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:34:49.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:48.790114+0000 mon.c (mon.2) 157 : audit [DBG] from='client.? 172.21.15.59:0/596669088' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:49.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.636+0000 7ff2db71b700 10 librados: watch_flush exit 2024-02-18T00:34:49.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.636+0000 7ff2db71b700 20 client.40520.objecter shutdown clearing up homeless session... 2024-02-18T00:34:49.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.636+0000 7ff2db71b700 10 client.40520.objecter successfully canceled tick 2024-02-18T00:34:49.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.637+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff2c0062c90 msgr2=0x7ff2c0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:49.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.637+0000 7ff2db71b700 1 --2- 172.21.15.59:0/2871605691 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff2c0062c90 0x7ff2c0065150 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 rx=0x7ff2c4009a00 tx=0x7ff2c400b040).stop 2024-02-18T00:34:49.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.637+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40f8420 msgr2=0x7ff2d410e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:49.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.637+0000 7ff2db71b700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40f8420 0x7ff2d410e0c0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 rx=0x7ff2d0004920 tx=0x7ff2d0009cb0).stop 2024-02-18T00:34:49.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.643+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 shutdown_connections 2024-02-18T00:34:49.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.643+0000 7ff2db71b700 1 --2- 172.21.15.59:0/2871605691 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff2c0062c90 0x7ff2c0065150 unknown :-1 s=CLOSED pgs=3000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:49.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.643+0000 7ff2db71b700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2d40fe510 0x7ff2d4108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:49.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.643+0000 7ff2db71b700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2d40f8420 0x7ff2d410e0c0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:49.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.643+0000 7ff2db71b700 1 --2- 172.21.15.59:0/2871605691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d40fdbb0 0x7ff2d410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:49.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.643+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 >> 172.21.15.59:0/2871605691 conn(0x7ff2d40f3800 msgr2=0x7ff2d40fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:49.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.647+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 shutdown_connections 2024-02-18T00:34:49.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.647+0000 7ff2db71b700 1 -- 172.21.15.59:0/2871605691 wait complete. 2024-02-18T00:34:49.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:49.649+0000 7ff2db71b700 1 librados: shutdown 2024-02-18T00:34:49.650 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.3 to 1 (10000) 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.618268+0000 mon.a (mon.0) 1510 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:48.618353+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e210: 8 total, 8 up, 7 in 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.622356+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.626691+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.627470+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.628332+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.59:0/1461454752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.628684+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.705843+0000 mon.b (mon.1) 263 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:50.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.756811+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.59:0/2871605691' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:34:50.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.757127+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:34:50.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:48.790114+0000 mon.c (mon.2) 157 : audit [DBG] from='client.? 172.21.15.59:0/596669088' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:50.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.618268+0000 mon.a (mon.0) 1510 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "19"}]': finished 2024-02-18T00:34:50.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:48.618353+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e210: 8 total, 8 up, 7 in 2024-02-18T00:34:50.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.622356+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.626691+0000 mon.c (mon.2) 154 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.627470+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.628332+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.59:0/1461454752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.628684+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.705843+0000 mon.b (mon.1) 263 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:50.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.756811+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.59:0/2871605691' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:34:50.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.757127+0000 mon.a (mon.0) 1515 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:34:50.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:48.790114+0000 mon.c (mon.2) 157 : audit [DBG] from='client.? 172.21.15.59:0/596669088' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:34:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.618027+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.618145+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.618262+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.618419+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:49.618478+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e211: 8 total, 8 up, 7 in 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.630604+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.645730+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.647237+0000 mon.a (mon.0) 1522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:49.706894+0000 mon.b (mon.1) 264 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:49.839369+0000 mgr.y (mgr.24454) 251 : cluster [DBG] pgmap v304: 290 pgs: 1 clean+premerge+peered, 12 active+clean+snaptrim, 64 unknown, 19 active+clean+snaptrim_wait, 194 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.621384+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.621486+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:50.621645+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e212: 8 total, 8 up, 7 in 2024-02-18T00:34:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.625263+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.627409+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.618027+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.618145+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.618262+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.618419+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:49.618478+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e211: 8 total, 8 up, 7 in 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.630604+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.645730+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:51.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.647237+0000 mon.a (mon.0) 1522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:51.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:49.706894+0000 mon.b (mon.1) 264 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:51.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:49.839369+0000 mgr.y (mgr.24454) 251 : cluster [DBG] pgmap v304: 290 pgs: 1 clean+premerge+peered, 12 active+clean+snaptrim, 64 unknown, 19 active+clean+snaptrim_wait, 194 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:51.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.621384+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:51.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.621486+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:51.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:50.621645+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e212: 8 total, 8 up, 7 in 2024-02-18T00:34:51.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.625263+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:51.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.627409+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:51.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.618027+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:51.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.618145+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:51.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.618262+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi059-82882-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:51.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.618419+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:34:51.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:49.618478+0000 mon.a (mon.0) 1520 : cluster [DBG] osdmap e211: 8 total, 8 up, 7 in 2024-02-18T00:34:51.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.630604+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]: dispatch 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.645730+0000 mon.c (mon.2) 158 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.647237+0000 mon.a (mon.0) 1522 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:49.706894+0000 mon.b (mon.1) 264 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:49.839369+0000 mgr.y (mgr.24454) 251 : cluster [DBG] pgmap v304: 290 pgs: 1 clean+premerge+peered, 12 active+clean+snaptrim, 64 unknown, 19 active+clean+snaptrim_wait, 194 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.621384+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? 172.21.15.59:0/1448753841' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi059-82877-29"}]': finished 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.621486+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:50.621645+0000 mon.a (mon.0) 1525 : cluster [DBG] osdmap e212: 8 total, 8 up, 7 in 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.625263+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:51.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.627409+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:51.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:34:51.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:34:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:51.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.656248+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.656986+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.657789+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.658129+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.658722+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:51.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.659101+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:50.707938+0000 mon.b (mon.1) 265 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.034972+0000 mgr.y (mgr.24454) 252 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.623375+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]': finished 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.623514+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:51.623631+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e213: 8 total, 8 up, 7 in 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.629208+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.629670+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.629697+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.630441+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.635489+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.59:0/2918754524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:51.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.635902+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.656248+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.656986+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.657789+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.658129+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.658722+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.659101+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:50.707938+0000 mon.b (mon.1) 265 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.034972+0000 mgr.y (mgr.24454) 252 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:51.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.623375+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]': finished 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.623514+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:51.623631+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e213: 8 total, 8 up, 7 in 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.629208+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]: dispatch 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.629670+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.629697+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]: dispatch 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.630441+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.635489+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.59:0/2918754524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:51.942 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.635902+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:51.943 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:51.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.656248+0000 mon.c (mon.2) 160 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.656986+0000 mon.a (mon.0) 1527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.657789+0000 mon.c (mon.2) 161 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.658129+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.658722+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.659101+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:50.707938+0000 mon.b (mon.1) 265 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.034972+0000 mgr.y (mgr.24454) 252 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.623375+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-17"}]': finished 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.623514+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi059-82877-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:51.623631+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e213: 8 total, 8 up, 7 in 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.629208+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.629670+0000 mon.c (mon.2) 164 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.629697+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.630441+0000 mon.a (mon.0) 1534 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.635489+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.59:0/2918754524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:52.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.635902+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:52.044 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:52.076 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:52.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:52.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:52.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:34:52.209 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:34:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:34:52] "GET /metrics HTTP/1.1" 200 196802 "" "Prometheus/2.33.4" 2024-02-18T00:34:52.237 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.708907+0000 mon.b (mon.1) 266 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:51.840050+0000 mgr.y (mgr.24454) 253 : cluster [DBG] pgmap v307: 289 pgs: 1 peering, 12 active+clean+snaptrim, 32 unknown, 19 active+clean+snaptrim_wait, 225 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-18T00:34:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:51.841074+0000 mon.a (mon.0) 1536 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-18T00:34:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:52.620034+0000 mon.a (mon.0) 1537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.708907+0000 mon.b (mon.1) 266 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:51.840050+0000 mgr.y (mgr.24454) 253 : cluster [DBG] pgmap v307: 289 pgs: 1 peering, 12 active+clean+snaptrim, 32 unknown, 19 active+clean+snaptrim_wait, 225 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-18T00:34:53.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:51.841074+0000 mon.a (mon.0) 1536 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-18T00:34:53.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:52.620034+0000 mon.a (mon.0) 1537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.708907+0000 mon.b (mon.1) 266 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:51.840050+0000 mgr.y (mgr.24454) 253 : cluster [DBG] pgmap v307: 289 pgs: 1 peering, 12 active+clean+snaptrim, 32 unknown, 19 active+clean+snaptrim_wait, 225 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-18T00:34:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:51.841074+0000 mon.a (mon.0) 1536 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "18"}]: dispatch 2024-02-18T00:34:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:52.620034+0000 mon.a (mon.0) 1537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:52.628416+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]': finished 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:52.709957+0000 mon.b (mon.1) 267 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:52.712020+0000 mon.c (mon.2) 166 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.077519+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.077632+0000 mon.a (mon.0) 1540 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:53.077703+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e214: 8 total, 8 up, 7 in 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.087871+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.598662+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.598741+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:53.598804+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e215: 8 total, 8 up, 7 in 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.608529+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.609091+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:53.710948+0000 mon.b (mon.1) 268 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:52.628416+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]': finished 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:52.709957+0000 mon.b (mon.1) 267 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:52.712020+0000 mon.c (mon.2) 166 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.077519+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.077632+0000 mon.a (mon.0) 1540 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:53.077703+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e214: 8 total, 8 up, 7 in 2024-02-18T00:34:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.087871+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:54.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.598662+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:54.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.598741+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:54.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:53.598804+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e215: 8 total, 8 up, 7 in 2024-02-18T00:34:54.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.608529+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:54.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.609091+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:54.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:53.710948+0000 mon.b (mon.1) 268 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:54.473 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:34:54.476 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:54.476 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:34:54.480 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 4 1 2024-02-18T00:34:54.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:52.628416+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-17", "mode": "writeback"}]': finished 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:52.709957+0000 mon.b (mon.1) 267 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:52.712020+0000 mon.c (mon.2) 166 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.077519+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi059-82882-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.077632+0000 mon.a (mon.0) 1540 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "18"}]': finished 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:53.077703+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e214: 8 total, 8 up, 7 in 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.087871+0000 mon.a (mon.0) 1542 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.598662+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi059-82877-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.598741+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:53.598804+0000 mon.a (mon.0) 1545 : cluster [DBG] osdmap e215: 8 total, 8 up, 7 in 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.608529+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.609091+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]: dispatch 2024-02-18T00:34:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:53.710948+0000 mon.b (mon.1) 268 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:54.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:54.610 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 -- 172.21.15.59:0/2467974933 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f9610 msgr2=0x7f0994101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 --2- 172.21.15.59:0/2467974933 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f9610 0x7f0994101d20 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 rx=0x7f0990004870 tx=0x7f0990009cb0).stop 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 -- 172.21.15.59:0/2467974933 shutdown_connections 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 --2- 172.21.15.59:0/2467974933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09940f8cb0 0x7f09940f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 --2- 172.21.15.59:0/2467974933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 0x7f0994104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 --2- 172.21.15.59:0/2467974933 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f9610 0x7f0994101d20 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 -- 172.21.15.59:0/2467974933 >> 172.21.15.59:0/2467974933 conn(0x7f09940f2820 msgr2=0x7f09940f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 -- 172.21.15.59:0/2467974933 shutdown_connections 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.205+0000 7f099b8f3700 1 -- 172.21.15.59:0/2467974933 wait complete. 2024-02-18T00:34:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 librados: starting msgr at 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 librados: starting objecter 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 Processor -- start 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 -- start start 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 librados: setting wanted keys 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 librados: calling monclient init 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f8cb0 0x7f099410e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09940f9610 0x7f099410e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 0x7f0994108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0994108840 con 0x7f0994102210 2024-02-18T00:34:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941089b0 con 0x7f09940f9610 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099b8f3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0994108b70 con 0x7f09940f8cb0 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099968f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f8cb0 0x7f099410e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 0x7f0994108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 0x7f0994108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55966/0 (socket says 172.21.15.59:55966) 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f099968f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f8cb0 0x7f099410e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56212/0 (socket says 172.21.15.59:56212) 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 -- 172.21.15.59:0/1948884154 learned_addr learned my addr 172.21.15.59:0/1948884154 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0998e8e700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09940f9610 0x7f099410e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 -- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f8cb0 msgr2=0x7f099410e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f8cb0 0x7f099410e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 -- 172.21.15.59:0/1948884154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09940f9610 msgr2=0x7f099410e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09940f9610 0x7f099410e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.206+0000 7f0999e90700 1 -- 172.21.15.59:0/1948884154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990005040 con 0x7f0994102210 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f099968f700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f8cb0 0x7f099410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:34:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f0998e8e700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09940f9610 0x7f099410e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f0999e90700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 0x7f0994108350 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7f098c00eb70 tx=0x7f098c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f09867fc700 10 client.?.objecter ms_handle_connect 0x7f0994102210 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f09867fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f09867fc700 1 -- 172.21.15.59:0/1948884154 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c009910 con 0x7f0994102210 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0994108df0 con 0x7f0994102210 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f09867fc700 1 -- 172.21.15.59:0/1948884154 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f098c00f040 con 0x7f0994102210 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f09867fc700 1 -- 172.21.15.59:0/1948884154 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c007e30 con 0x7f0994102210 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f099b8f3700 10 client.30720.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0994114fe0 con 0x7f0994102210 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.207+0000 7f099b8f3700 1 librados: init done 2024-02-18T00:34:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.208+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099405fa60 con 0x7f0994102210 2024-02-18T00:34:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 1 -- 172.21.15.59:0/1948884154 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f098c004020 con 0x7f0994102210 2024-02-18T00:34:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 1 --2- 172.21.15.59:0/1948884154 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0980062c90 0x7f0980065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:34:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 1 -- 172.21.15.59:0/1948884154 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(216..216 src has 44..216) v4 ==== 9325+0+0 (secure 0 0 0) 0x7f098c07d450 con 0x7f0994102210 2024-02-18T00:34:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 10 client.30720.objecter ms_dispatch 0x7f0994001550 osd_map(216..216 src has 44..216) v4 2024-02-18T00:34:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 3 client.30720.objecter handle_osd_map got epochs [216,216] > 0 2024-02-18T00:34:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 3 client.30720.objecter handle_osd_map decoding full epoch 216 2024-02-18T00:34:55.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f099968f700 1 --2- 172.21.15.59:0/1948884154 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0980062c90 0x7f0980065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:34:55.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 20 client.30720.objecter dump_active .. 0 homeless 2024-02-18T00:34:55.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f099968f700 1 --2- 172.21.15.59:0/1948884154 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0980062c90 0x7f0980065150 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 rx=0x7f09880041d0 tx=0x7f0988009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:34:55.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.209+0000 7f09867fc700 10 client.30720.objecter ms_handle_connect 0x7f0980062c90 2024-02-18T00:34:55.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.214+0000 7f09867fc700 1 -- 172.21.15.59:0/1948884154 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f098c049f60 con 0x7f0994102210 2024-02-18T00:34:55.217 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:53.840459+0000 mgr.y (mgr.24454) 254 : cluster [DBG] pgmap v310: 265 pgs: 1 peering, 12 active+clean+snaptrim, 8 unknown, 19 active+clean+snaptrim_wait, 225 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-18T00:34:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:54.595296+0000 mon.a (mon.0) 1547 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:54.603884+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]': finished 2024-02-18T00:34:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:54.604033+0000 mon.a (mon.0) 1549 : cluster [DBG] osdmap e216: 8 total, 8 up, 7 in 2024-02-18T00:34:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:54.616967+0000 mon.c (mon.2) 168 : audit [INF] from='client.? 172.21.15.59:0/20605312' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:54.618535+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:54.711906+0000 mon.b (mon.1) 269 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:54.893950+0000 mon.a (mon.0) 1551 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:53.840459+0000 mgr.y (mgr.24454) 254 : cluster [DBG] pgmap v310: 265 pgs: 1 peering, 12 active+clean+snaptrim, 8 unknown, 19 active+clean+snaptrim_wait, 225 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:54.595296+0000 mon.a (mon.0) 1547 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:54.603884+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]': finished 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:54.604033+0000 mon.a (mon.0) 1549 : cluster [DBG] osdmap e216: 8 total, 8 up, 7 in 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:54.616967+0000 mon.c (mon.2) 168 : audit [INF] from='client.? 172.21.15.59:0/20605312' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:54.618535+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:54.711906+0000 mon.b (mon.1) 269 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:55.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:54.893950+0000 mon.a (mon.0) 1551 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:55.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.409+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f099405ee80 con 0x7f0994102210 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:53.840459+0000 mgr.y (mgr.24454) 254 : cluster [DBG] pgmap v310: 265 pgs: 1 peering, 12 active+clean+snaptrim, 8 unknown, 19 active+clean+snaptrim_wait, 225 active+clean; 11 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:54.595296+0000 mon.a (mon.0) 1547 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:54.603884+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-17"}]': finished 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:54.604033+0000 mon.a (mon.0) 1549 : cluster [DBG] osdmap e216: 8 total, 8 up, 7 in 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:54.616967+0000 mon.c (mon.2) 168 : audit [INF] from='client.? 172.21.15.59:0/20605312' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:54.618535+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:54.711906+0000 mon.b (mon.1) 269 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:54.893950+0000 mon.a (mon.0) 1551 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:34:55.574 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:55.577 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:55.578 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:55.588 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:34:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.608+0000 7f09867fc700 1 -- 172.21.15.59:0/1948884154 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v217) v1 ==== 97+0+0 (secure 0 0 0) 0x7f098c050470 con 0x7f0994102210 2024-02-18T00:34:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.615+0000 7f099b8f3700 10 librados: watch_flush enter 2024-02-18T00:34:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.616+0000 7f099b8f3700 10 librados: watch_flush exit 2024-02-18T00:34:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.616+0000 7f099b8f3700 20 client.30720.objecter shutdown clearing up homeless session... 2024-02-18T00:34:55.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.616+0000 7f099b8f3700 10 client.30720.objecter successfully canceled tick 2024-02-18T00:34:55.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0980062c90 msgr2=0x7f0980065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:55.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 --2- 172.21.15.59:0/1948884154 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0980062c90 0x7f0980065150 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 rx=0x7f09880041d0 tx=0x7f0988009040).stop 2024-02-18T00:34:55.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 msgr2=0x7f0994108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:34:55.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 0x7f0994108350 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7f098c00eb70 tx=0x7f098c00b040).stop 2024-02-18T00:34:55.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 shutdown_connections 2024-02-18T00:34:55.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0994102210 0x7f0994108350 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09940f8cb0 0x7f099410e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 --2- 172.21.15.59:0/1948884154 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0980062c90 0x7f0980065150 secure :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 rx=0x7f09880041d0 tx=0x7f0988009040).stop 2024-02-18T00:34:55.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 --2- 172.21.15.59:0/1948884154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09940f9610 0x7f099410e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:34:55.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.620+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 >> 172.21.15.59:0/1948884154 conn(0x7f09940f2820 msgr2=0x7f09940f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:34:55.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.637+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 shutdown_connections 2024-02-18T00:34:55.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.647+0000 7f099b8f3700 1 -- 172.21.15.59:0/1948884154 wait complete. 2024-02-18T00:34:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:34:55.649+0000 7f099b8f3700 1 librados: shutdown 2024-02-18T00:34:55.665 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:34:55.679 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:55.707 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:55.727 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:55.729 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:55.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:55.830 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:55.857 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:55.876 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:55.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:55.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:55.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:56.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:55.409711+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? 172.21.15.59:0/1948884154' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:56.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:55.601747+0000 mon.a (mon.0) 1553 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:56.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:55.607324+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:56.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:55.607474+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? 172.21.15.59:0/1948884154' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:56.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:55.607515+0000 mon.a (mon.0) 1556 : cluster [DBG] osdmap e217: 8 total, 8 up, 7 in 2024-02-18T00:34:56.184 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:55.612437+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:56.184 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:55.712833+0000 mon.b (mon.1) 270 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:56.184 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.075020+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:56.184 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.076315+0000 mon.a (mon.0) 1558 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-18T00:34:56.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:55.409711+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? 172.21.15.59:0/1948884154' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:56.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:55.601747+0000 mon.a (mon.0) 1553 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:56.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:55.607324+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:56.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:55.607474+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? 172.21.15.59:0/1948884154' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:56.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:55.607515+0000 mon.a (mon.0) 1556 : cluster [DBG] osdmap e217: 8 total, 8 up, 7 in 2024-02-18T00:34:56.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:55.612437+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:56.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:55.712833+0000 mon.b (mon.1) 270 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:56.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.075020+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:56.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.076315+0000 mon.a (mon.0) 1558 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-18T00:34:56.468 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:56.469 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:56.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:56.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:55.409711+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? 172.21.15.59:0/1948884154' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:55.601747+0000 mon.a (mon.0) 1553 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:55.607324+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi059-82882-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:55.607474+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? 172.21.15.59:0/1948884154' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:55.607515+0000 mon.a (mon.0) 1556 : cluster [DBG] osdmap e217: 8 total, 8 up, 7 in 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:55.612437+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:55.712833+0000 mon.b (mon.1) 270 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.075020+0000 mon.a (mon.0) 1557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.076315+0000 mon.a (mon.0) 1558 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "17"}]: dispatch 2024-02-18T00:34:56.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:34:56.600 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:56.870 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:34:56.876 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:56.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:34:56.905 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:34:56.977 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:57.009 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:57.033 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:57.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:57.134 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:57.164 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:57.183 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:57.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:57.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:55.841332+0000 mgr.y (mgr.24454) 255 : cluster [DBG] pgmap v313: 256 pgs: 9 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 231 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.615998+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.616079+0000 mon.a (mon.0) 1560 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:56.616164+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e218: 8 total, 8 up, 7 in 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.621109+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.623169+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.629563+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.630253+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:55.841332+0000 mgr.y (mgr.24454) 255 : cluster [DBG] pgmap v313: 256 pgs: 9 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 231 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T00:34:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.615998+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.616079+0000 mon.a (mon.0) 1560 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-18T00:34:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:56.616164+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e218: 8 total, 8 up, 7 in 2024-02-18T00:34:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.621109+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.623169+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.629563+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.630253+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:57.286 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:55.841332+0000 mgr.y (mgr.24454) 255 : cluster [DBG] pgmap v313: 256 pgs: 9 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 231 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.615998+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.616079+0000 mon.a (mon.0) 1560 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "17"}]': finished 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:56.616164+0000 mon.a (mon.0) 1561 : cluster [DBG] osdmap e218: 8 total, 8 up, 7 in 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.621109+0000 mon.c (mon.2) 170 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.623169+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.629563+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.59:0/1864156409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:57.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.630253+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]: dispatch 2024-02-18T00:34:57.313 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:57.332 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:57.334 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:57.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:57.436 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:34:57.463 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:58.212 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:34:58.216 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:58.216 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:34:58.317 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:58.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:56.713890+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.618986+0000 mon.a (mon.0) 1564 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.619199+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:57.619316+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e219: 8 total, 8 up, 7 in 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.632841+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.59:0/4085673095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.633403+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.638758+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.639146+0000 mon.a (mon.0) 1568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.657327+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.657643+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.658262+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.658539+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.658978+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:58.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.659296+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:58.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:57.714836+0000 mon.b (mon.1) 272 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:58.371 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:58.373 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:58.374 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:58.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:56.713890+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.618986+0000 mon.a (mon.0) 1564 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.619199+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:57.619316+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e219: 8 total, 8 up, 7 in 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.632841+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.59:0/4085673095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.633403+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.638758+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.639146+0000 mon.a (mon.0) 1568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.657327+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.657643+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.658262+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.658539+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.658978+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:58.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.659296+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:57.714836+0000 mon.b (mon.1) 272 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:56.713890+0000 mon.b (mon.1) 271 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.618986+0000 mon.a (mon.0) 1564 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.619199+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi059-82877-30"}]': finished 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:57.619316+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e219: 8 total, 8 up, 7 in 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.632841+0000 mon.c (mon.2) 172 : audit [INF] from='client.? 172.21.15.59:0/4085673095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.633403+0000 mon.a (mon.0) 1567 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.638758+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.639146+0000 mon.a (mon.0) 1568 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.657327+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.657643+0000 mon.a (mon.0) 1569 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.658262+0000 mon.c (mon.2) 175 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.658539+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:58.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.658978+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:58.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.659296+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:34:58.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:57.714836+0000 mon.b (mon.1) 272 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:58.474 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:58.504 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:58.525 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:58.527 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:58.528 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:58.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:58.662 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:58.681 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:58.683 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:58.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:34:58.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:34:58.818 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:59.048 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:34:59.048 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:34:59.067 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:34:59.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:59.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:34:59.171 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:34:59.200 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:59.220 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:34:59.223 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:59.223 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:34:59.324 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:57.841846+0000 mgr.y (mgr.24454) 256 : cluster [DBG] pgmap v316: 288 pgs: 9 active+clean+snaptrim, 64 unknown, 16 active+clean+snaptrim_wait, 199 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.601785+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.601895+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.601990+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:58.602036+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e220: 8 total, 8 up, 7 in 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.604913+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.605899+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.606243+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.619131+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:34:59.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:34:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:58.715778+0000 mon.b (mon.1) 273 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:59.352 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:59.371 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:59.374 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:59.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:59.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:57.841846+0000 mgr.y (mgr.24454) 256 : cluster [DBG] pgmap v316: 288 pgs: 9 active+clean+snaptrim, 64 unknown, 16 active+clean+snaptrim_wait, 199 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:34:59.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.601785+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.601895+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.601990+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:58.602036+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e220: 8 total, 8 up, 7 in 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.604913+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.605899+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.606243+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.619131+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:58.715778+0000 mon.b (mon.1) 273 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:59.403 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:57.841846+0000 mgr.y (mgr.24454) 256 : cluster [DBG] pgmap v316: 288 pgs: 9 active+clean+snaptrim, 64 unknown, 16 active+clean+snaptrim_wait, 199 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.601785+0000 mon.a (mon.0) 1572 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi059-82882-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.601895+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.601990+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi059-82877-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:58.602036+0000 mon.a (mon.0) 1575 : cluster [DBG] osdmap e220: 8 total, 8 up, 7 in 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.604913+0000 mon.c (mon.2) 177 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.605899+0000 mon.a (mon.0) 1576 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.606243+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.619131+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:34:59.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:34:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:58.715778+0000 mon.b (mon.1) 273 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:34:59.475 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:59.502 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:59.520 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:59.523 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:59.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:59.624 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:34:59.651 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:59.669 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:34:59.671 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:59.672 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:34:59.772 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:34:59.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-18T00:34:59.798 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:59.817 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:34:59.819 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:59.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:34:59.921 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:34:59.947 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:34:59.965 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:34:59.967 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:34:59.968 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:00.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:00.097 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:00.297 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:00.300 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:00.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:00.401 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:00.428 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:00.449 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:00.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:00.550 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 -- 172.21.15.59:0/4207673776 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f63b0 msgr2=0x7f44c40f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 --2- 172.21.15.59:0/4207673776 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f63b0 0x7f44c40f6830 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 rx=0x7f44b00048a0 tx=0x7f44b0009cb0).stop 2024-02-18T00:35:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 -- 172.21.15.59:0/4207673776 shutdown_connections 2024-02-18T00:35:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 --2- 172.21.15.59:0/4207673776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c40f89b0 0x7f44c40f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 --2- 172.21.15.59:0/4207673776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f6d20 0x7f44c4106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 --2- 172.21.15.59:0/4207673776 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f63b0 0x7f44c40f6830 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 -- 172.21.15.59:0/4207673776 >> 172.21.15.59:0/4207673776 conn(0x7f44c406c370 msgr2=0x7f44c406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 -- 172.21.15.59:0/4207673776 shutdown_connections 2024-02-18T00:35:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.557+0000 7f44cad84700 1 -- 172.21.15.59:0/4207673776 wait complete. 2024-02-18T00:35:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.558+0000 7f44cad84700 1 librados: starting msgr at 2024-02-18T00:35:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.558+0000 7f44cad84700 1 librados: starting objecter 2024-02-18T00:35:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.558+0000 7f44cad84700 1 Processor -- start 2024-02-18T00:35:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.558+0000 7f44cad84700 1 -- start start 2024-02-18T00:35:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.558+0000 7f44cad84700 1 librados: setting wanted keys 2024-02-18T00:35:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.558+0000 7f44cad84700 1 librados: calling monclient init 2024-02-18T00:35:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44cad84700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c40f63b0 0x7f44c40ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44cad84700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f6d20 0x7f44c4100260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44cad84700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f89b0 0x7f44c40faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44cad84700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40fb4f0 con 0x7f44c40f89b0 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44cad84700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40fb660 con 0x7f44c40f63b0 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44cad84700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40fb820 con 0x7f44c40f6d20 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f89b0 0x7f44c40faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f89b0 0x7f44c40faf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40580/0 (socket says 172.21.15.59:40580) 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 -- 172.21.15.59:0/4164520772 learned_addr learned my addr 172.21.15.59:0/4164520772 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c8b20700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c40f63b0 0x7f44c40ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c3fff700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f6d20 0x7f44c4100260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 -- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f6d20 msgr2=0x7f44c4100260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:00.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f6d20 0x7f44c4100260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 -- 172.21.15.59:0/4164520772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c40f63b0 msgr2=0x7f44c40ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c40f63b0 0x7f44c40ffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 -- 172.21.15.59:0/4164520772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0005040 con 0x7f44c40f89b0 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c9321700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f89b0 0x7f44c40faf70 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 rx=0x7f44bc00ec20 tx=0x7f44bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.559+0000 7f44c1ffb700 10 client.?.objecter ms_handle_connect 0x7f44c40f89b0 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44c1ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44c1ffb700 1 -- 172.21.15.59:0/4164520772 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc009950 con 0x7f44c40f89b0 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c40fbaa0 con 0x7f44c40f89b0 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44c1ffb700 1 -- 172.21.15.59:0/4164520772 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44bc00f040 con 0x7f44c40f89b0 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44c1ffb700 1 -- 172.21.15.59:0/4164520772 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc007df0 con 0x7f44c40f89b0 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44cad84700 10 client.30744.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c4105b60 con 0x7f44c40f89b0 2024-02-18T00:35:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.560+0000 7f44cad84700 1 librados: init done 2024-02-18T00:35:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.561+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c40fc8f0 con 0x7f44c40f89b0 2024-02-18T00:35:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.563+0000 7f44c1ffb700 1 -- 172.21.15.59:0/4164520772 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f44bc0092e0 con 0x7f44c40f89b0 2024-02-18T00:35:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.563+0000 7f44c1ffb700 1 --2- 172.21.15.59:0/4164520772 >> v2:172.21.15.59:6800/3721891797 conn(0x7f44b4062c90 0x7f44b4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.563+0000 7f44c1ffb700 1 -- 172.21.15.59:0/4164520772 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(221..221 src has 44..221) v4 ==== 8528+0+0 (secure 0 0 0) 0x7f44bc07d530 con 0x7f44c40f89b0 2024-02-18T00:35:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.563+0000 7f44c1ffb700 10 client.30744.objecter ms_dispatch 0x7f44c4001550 osd_map(221..221 src has 44..221) v4 2024-02-18T00:35:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.563+0000 7f44c1ffb700 3 client.30744.objecter handle_osd_map got epochs [221,221] > 0 2024-02-18T00:35:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.563+0000 7f44c1ffb700 3 client.30744.objecter handle_osd_map decoding full epoch 221 2024-02-18T00:35:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.564+0000 7f44c1ffb700 20 client.30744.objecter dump_active .. 0 homeless 2024-02-18T00:35:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.564+0000 7f44c8b20700 1 --2- 172.21.15.59:0/4164520772 >> v2:172.21.15.59:6800/3721891797 conn(0x7f44b4062c90 0x7f44b4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.564+0000 7f44c1ffb700 10 client.30744.objecter ms_handle_connect 0x7f44b4062c90 2024-02-18T00:35:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.564+0000 7f44c8b20700 1 --2- 172.21.15.59:0/4164520772 >> v2:172.21.15.59:6800/3721891797 conn(0x7f44b4062c90 0x7f44b4065150 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 rx=0x7f44b80048b0 tx=0x7f44b800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.567+0000 7f44c1ffb700 1 -- 172.21.15.59:0/4164520772 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f44bc0508b0 con 0x7f44c40f89b0 2024-02-18T00:35:00.577 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:00.595 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:00.597 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:00.598 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:00.698 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:00.728 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:00.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:34:59.717063+0000 mon.b (mon.1) 274 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:00.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:34:59.842239+0000 mgr.y (mgr.24454) 257 : cluster [DBG] pgmap v319: 255 pgs: 9 active+clean+snaptrim, 32 unknown, 16 active+clean+snaptrim_wait, 198 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:00.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.041219+0000 mon.a (mon.0) 1578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]': finished 2024-02-18T00:35:00.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:00.041323+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e221: 8 total, 8 up, 7 in 2024-02-18T00:35:00.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.041976+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]: dispatch 2024-02-18T00:35:00.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.043248+0000 mon.a (mon.0) 1580 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-18T00:35:00.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.043588+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]: dispatch 2024-02-18T00:35:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:00.791+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7f44c405f950 con 0x7f44c40f89b0 2024-02-18T00:35:00.880 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:00.883 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:34:59.717063+0000 mon.b (mon.1) 274 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:00.883 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:34:59.842239+0000 mgr.y (mgr.24454) 257 : cluster [DBG] pgmap v319: 255 pgs: 9 active+clean+snaptrim, 32 unknown, 16 active+clean+snaptrim_wait, 198 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.041219+0000 mon.a (mon.0) 1578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]': finished 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:00.041323+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e221: 8 total, 8 up, 7 in 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.041976+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]: dispatch 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.043248+0000 mon.a (mon.0) 1580 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.043588+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]: dispatch 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:34:59.717063+0000 mon.b (mon.1) 274 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:34:59.842239+0000 mgr.y (mgr.24454) 257 : cluster [DBG] pgmap v319: 255 pgs: 9 active+clean+snaptrim, 32 unknown, 16 active+clean+snaptrim_wait, 198 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.041219+0000 mon.a (mon.0) 1578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-19"}]': finished 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:00.041323+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e221: 8 total, 8 up, 7 in 2024-02-18T00:35:00.884 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.041976+0000 mon.c (mon.2) 179 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]: dispatch 2024-02-18T00:35:00.885 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.043248+0000 mon.a (mon.0) 1580 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "16"}]: dispatch 2024-02-18T00:35:00.885 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.043588+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]: dispatch 2024-02-18T00:35:00.885 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:00.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:00.986 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:01.015 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:01.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:01.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:01.144 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:01.172 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:01.190 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:01.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:01.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:01.294 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:01.319 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:35:01.319 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:35:01.321 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:01.339 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:01.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:01.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:01.443 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:01.473 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:01.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.611+0000 7f44c1ffb700 1 -- 172.21.15.59:0/4164520772 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v223) v1 ==== 113+0+0 (secure 0 0 0) 0x7f44bc022030 con 0x7f44c40f89b0 2024-02-18T00:35:01.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.618+0000 7f44cad84700 10 librados: watch_flush enter 2024-02-18T00:35:01.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.618+0000 7f44cad84700 10 librados: watch_flush exit 2024-02-18T00:35:01.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.618+0000 7f44cad84700 20 client.30744.objecter shutdown clearing up homeless session... 2024-02-18T00:35:01.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.618+0000 7f44cad84700 10 client.30744.objecter successfully canceled tick 2024-02-18T00:35:01.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 >> v2:172.21.15.59:6800/3721891797 conn(0x7f44b4062c90 msgr2=0x7f44b4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:01.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 --2- 172.21.15.59:0/4164520772 >> v2:172.21.15.59:6800/3721891797 conn(0x7f44b4062c90 0x7f44b4065150 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 rx=0x7f44b80048b0 tx=0x7f44b800b040).stop 2024-02-18T00:35:01.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f89b0 msgr2=0x7f44c40faf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:01.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f89b0 0x7f44c40faf70 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 rx=0x7f44bc00ec20 tx=0x7f44bc00b040).stop 2024-02-18T00:35:01.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 shutdown_connections 2024-02-18T00:35:01.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 --2- 172.21.15.59:0/4164520772 >> v2:172.21.15.59:6800/3721891797 conn(0x7f44b4062c90 0x7f44b4065150 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:01.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44c40f63b0 0x7f44c40ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:01.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44c40f89b0 0x7f44c40faf70 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:01.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 --2- 172.21.15.59:0/4164520772 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f44c40f6d20 0x7f44c4100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:01.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.621+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 >> 172.21.15.59:0/4164520772 conn(0x7f44c406c370 msgr2=0x7f44c40f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:01.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.623+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 shutdown_connections 2024-02-18T00:35:01.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.630+0000 7f44cad84700 1 -- 172.21.15.59:0/4164520772 wait complete. 2024-02-18T00:35:01.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:01.634+0000 7f44cad84700 1 librados: shutdown 2024-02-18T00:35:01.643 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.4 to 1 (10000) 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:00.598952+0000 mon.a (mon.0) 1582 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.612240+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.612343+0000 mon.a (mon.0) 1584 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.612402+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]': finished 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:00.612493+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e222: 8 total, 8 up, 7 in 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.633752+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.59:0/218249582' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.634486+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.655676+0000 mon.c (mon.2) 181 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:01.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.656060+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.718071+0000 mon.b (mon.1) 275 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:00.793535+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? 172.21.15.59:0/4164520772' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:01.045902+0000 mgr.y (mgr.24454) 258 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:00.598952+0000 mon.a (mon.0) 1582 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.612240+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.612343+0000 mon.a (mon.0) 1584 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.612402+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]': finished 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:00.612493+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e222: 8 total, 8 up, 7 in 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.633752+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.59:0/218249582' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.634486+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:01.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.655676+0000 mon.c (mon.2) 181 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:01.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.656060+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:01.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.718071+0000 mon.b (mon.1) 275 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:01.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:00.793535+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? 172.21.15.59:0/4164520772' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:35:01.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:01.045902+0000 mgr.y (mgr.24454) 258 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:00.598952+0000 mon.a (mon.0) 1582 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.612240+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi059-82877-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.612343+0000 mon.a (mon.0) 1584 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "16"}]': finished 2024-02-18T00:35:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.612402+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-19", "mode": "writeback"}]': finished 2024-02-18T00:35:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:00.612493+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e222: 8 total, 8 up, 7 in 2024-02-18T00:35:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.633752+0000 mon.c (mon.2) 180 : audit [INF] from='client.? 172.21.15.59:0/218249582' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.634486+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:02.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.655676+0000 mon.c (mon.2) 181 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:02.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.656060+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:02.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.718071+0000 mon.b (mon.1) 275 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:02.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:00.793535+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? 172.21.15.59:0/4164520772' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:35:02.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:01.045902+0000 mgr.y (mgr.24454) 258 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:02.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:35:02.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:02] "GET /metrics HTTP/1.1" 200 194695 "" "Prometheus/2.33.4" 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:01.612232+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:01.612316+0000 mon.a (mon.0) 1591 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:01.612378+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.59:0/4164520772' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:01.612431+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e223: 8 total, 8 up, 7 in 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:01.617612+0000 mon.c (mon.2) 182 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:01.636231+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:01.719011+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:01.842653+0000 mgr.y (mgr.24454) 259 : cluster [DBG] pgmap v322: 295 pgs: 40 creating+peering, 2 peering, 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 231 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:35:02.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:02.608691+0000 mon.a (mon.0) 1595 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:02.614247+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]': finished 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:02.614366+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e224: 8 total, 8 up, 7 in 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:01.612232+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:01.612316+0000 mon.a (mon.0) 1591 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:01.612378+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.59:0/4164520772' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:01.612431+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e223: 8 total, 8 up, 7 in 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:01.617612+0000 mon.c (mon.2) 182 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:01.636231+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:01.719011+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:01.842653+0000 mgr.y (mgr.24454) 259 : cluster [DBG] pgmap v322: 295 pgs: 40 creating+peering, 2 peering, 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 231 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:02.608691+0000 mon.a (mon.0) 1595 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:02.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:02.614247+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]': finished 2024-02-18T00:35:02.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:02.614366+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e224: 8 total, 8 up, 7 in 2024-02-18T00:35:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:01.612232+0000 mon.a (mon.0) 1590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi059-82882-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:01.612316+0000 mon.a (mon.0) 1591 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:01.612378+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.59:0/4164520772' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:35:03.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:01.612431+0000 mon.a (mon.0) 1593 : cluster [DBG] osdmap e223: 8 total, 8 up, 7 in 2024-02-18T00:35:03.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:01.617612+0000 mon.c (mon.2) 182 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:35:03.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:01.636231+0000 mon.a (mon.0) 1594 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]: dispatch 2024-02-18T00:35:03.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:01.719011+0000 mon.b (mon.1) 276 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:03.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:01.842653+0000 mgr.y (mgr.24454) 259 : cluster [DBG] pgmap v322: 295 pgs: 40 creating+peering, 2 peering, 8 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 231 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:35:03.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:02.608691+0000 mon.a (mon.0) 1595 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:03.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:02.614247+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-19"}]': finished 2024-02-18T00:35:03.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:02.614366+0000 mon.a (mon.0) 1597 : cluster [DBG] osdmap e224: 8 total, 8 up, 7 in 2024-02-18T00:35:03.428 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:03.431 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:03.431 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:03.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:03.561 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:03.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:03.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:03.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:03.937 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:04.035 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:04.040 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:04.040 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:04.141 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:04.172 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:04.193 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:04.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:04.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:04.297 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:02.625050+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:02.629468+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:02.704812+0000 osd.3 (osd.3) 137 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:02.705050+0000 osd.3 (osd.3) 138 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:02.720017+0000 mon.b (mon.1) 277 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:03.604268+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:03.604397+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e225: 8 total, 8 up, 7 in 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:03.612062+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.325 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:03.612803+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.327 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:04.347 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:04.349 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:04.349 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:04.378 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:35:04.378 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:35:04.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:02.625050+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:02.629468+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:02.704812+0000 osd.3 (osd.3) 137 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:02.705050+0000 osd.3 (osd.3) 138 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:02.720017+0000 mon.b (mon.1) 277 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:03.604268+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:03.604397+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e225: 8 total, 8 up, 7 in 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:03.612062+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:03.612803+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:02.625050+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.408 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:02.629468+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:02.704812+0000 osd.3 (osd.3) 137 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:04.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:02.705050+0000 osd.3 (osd.3) 138 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:04.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:02.720017+0000 mon.b (mon.1) 277 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:04.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:03.604268+0000 mon.a (mon.0) 1599 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:04.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:03.604397+0000 mon.a (mon.0) 1600 : cluster [DBG] osdmap e225: 8 total, 8 up, 7 in 2024-02-18T00:35:04.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:03.612062+0000 mon.c (mon.2) 184 : audit [INF] from='client.? 172.21.15.59:0/3774101649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.409 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:03.612803+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]: dispatch 2024-02-18T00:35:04.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:04.482 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:04.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 5 1 2024-02-18T00:35:05.100 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:03.628232+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.59:0/1326867146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:03.628232+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.59:0/1326867146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:03.720951+0000 mon.b (mon.1) 278 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:03.843048+0000 mgr.y (mgr.24454) 260 : cluster [DBG] pgmap v325: 254 pgs: 1 peering, 8 active+clean+snaptrim, 32 unknown, 14 active+clean+snaptrim_wait, 199 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.063387+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.064384+0000 mon.a (mon.0) 1603 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.606241+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.606407+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.606486+0000 mon.a (mon.0) 1606 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-18T00:35:05.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:04.606538+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e226: 8 total, 8 up, 7 in 2024-02-18T00:35:05.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.611518+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.612044+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.644608+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:05.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.645965+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:05.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.646318+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi059-82877-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:05.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.301+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3145832905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780fe510 msgr2=0x7f1f780f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.301+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3145832905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780fe510 0x7f1f780f7d40 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 rx=0x7f1f64004870 tx=0x7f1f64009cb0).stop 2024-02-18T00:35:05.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.301+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3145832905 shutdown_connections 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.301+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3145832905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f780fdbb0 0x7f1f780fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.301+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3145832905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f780f8420 0x7f1f78104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.301+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3145832905 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780fe510 0x7f1f780f7d40 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.301+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3145832905 >> 172.21.15.59:0/3145832905 conn(0x7f1f780f3800 msgr2=0x7f1f780f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3145832905 shutdown_connections 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3145832905 wait complete. 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 librados: starting msgr at 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 librados: starting objecter 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 Processor -- start 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 -- start start 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 librados: setting wanted keys 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.302+0000 7f1f7eb2b700 1 librados: calling monclient init 2024-02-18T00:35:05.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7eb2b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780f8420 0x7f1f7810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7eb2b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f780fdbb0 0x7f1f7810e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7eb2b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f780fe510 0x7f1f78108120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7eb2b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78108610 con 0x7f1f780fdbb0 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7eb2b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78108780 con 0x7f1f780fe510 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7eb2b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78108940 con 0x7f1f780f8420 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780f8420 0x7f1f7810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780f8420 0x7f1f7810df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44936/0 (socket says 172.21.15.59:44936) 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f77fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f780fdbb0 0x7f1f7810e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 -- 172.21.15.59:0/3647302678 learned_addr learned my addr 172.21.15.59:0/3647302678 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7d0c8700 1 --2- 172.21.15.59:0/3647302678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f780fe510 0x7f1f78108120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 -- 172.21.15.59:0/3647302678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f780fe510 msgr2=0x7f1f78108120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f780fe510 0x7f1f78108120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 -- 172.21.15.59:0/3647302678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f780fdbb0 msgr2=0x7f1f7810e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f780fdbb0 0x7f1f7810e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 -- 172.21.15.59:0/3647302678 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700097c0 con 0x7f1f780f8420 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f77fff700 1 --2- 172.21.15.59:0/3647302678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f780fdbb0 0x7f1f7810e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780f8420 0x7f1f7810df80 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 rx=0x7f1f6c00c7c0 tx=0x7f1f6c007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f75ffb700 10 client.?.objecter ms_handle_connect 0x7f1f780f8420 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f75ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c004430 con 0x7f1f780f8420 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f6c00e970 con 0x7f1f780f8420 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f64005040 con 0x7f1f780f8420 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.303+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c013a50 con 0x7f1f780f8420 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.304+0000 7f1f7eb2b700 10 client.40595.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.304+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f78108f80 con 0x7f1f780f8420 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.304+0000 7f1f7eb2b700 1 librados: init done 2024-02-18T00:35:05.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.304+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f1f781070a0 con 0x7f1f780f8420 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1f6c00eae0 con 0x7f1f780f8420 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1f68062c90 0x7f1f68065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(226..226 src has 44..226) v4 ==== 8504+0+0 (secure 0 0 0) 0x7f1f6c07d620 con 0x7f1f780f8420 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 osd_map(226..226 src has 44..226) v4 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 3 client.40595.objecter handle_osd_map got epochs [226,226] > 0 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 3 client.40595.objecter handle_osd_map decoding full epoch 226 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f77fff700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1f68062c90 0x7f1f68065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 20 client.40595.objecter dump_active .. 0 homeless 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.305+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v226) v1 ==== 54+0+16 (secure 0 0 0) 0x7f1f6c084070 con 0x7f1f780f8420 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 1 2024-02-18T00:35:05.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1f780fee00 0x7f1f780ff220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f68062c90 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f77fff700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1f68062c90 0x7f1f68065150 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 rx=0x7f1f640043d0 tx=0x7f1f6400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 1 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 0 1 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78109600 osd=0 4 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 1 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f780ff710 con 0x7f1f780fee00 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7d0c8700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1f780fee00 0x7f1f780ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f780fee00 2024-02-18T00:35:05.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.306+0000 7f1f7d0c8700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1f780fee00 0x7f1f780ff220 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.307+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f780ff710 con 0x7f1f780fee00 2024-02-18T00:35:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.307+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:35:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.307+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 1 = system:0 2024-02-18T00:35:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.307+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78109600 osd=0 4 2024-02-18T00:35:05.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.307+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 0 1 2024-02-18T00:35:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 2 2024-02-18T00:35:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 2 2024-02-18T00:35:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78109600 osd=0 3 2024-02-18T00:35:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 0 2 2024-02-18T00:35:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78109600 osd=0 4 2024-02-18T00:35:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 2 2024-02-18T00:35:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.322+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f780ff8b0 con 0x7f1f780fee00 2024-02-18T00:35:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f780ff8b0 con 0x7f1f780fee00 2024-02-18T00:35:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78109600 osd=0 4 2024-02-18T00:35:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 0 2 2024-02-18T00:35:05.325 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:35:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 3 2024-02-18T00:35:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6805/26279110 conn(0x7f1f78028540 0x7f1f78118ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 3 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 1 3 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f780f5eb0 osd=1 4 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 3 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.323+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f780f6020 con 0x7f1f78028540 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6805/26279110 conn(0x7f1f78028540 0x7f1f78118ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f78028540 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6805/26279110 conn(0x7f1f78028540 0x7f1f78118ec0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f780f6020 con 0x7f1f78028540 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 3 = system:0 2024-02-18T00:35:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f780f5eb0 osd=1 4 2024-02-18T00:35:05.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.324+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 1 3 2024-02-18T00:35:05.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 4 2024-02-18T00:35:05.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 4 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f780f5eb0 osd=1 3 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 1 4 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f780f5eb0 osd=1 4 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 4 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.339+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f7805a040 con 0x7f1f78028540 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f7805a040 con 0x7f1f78028540 2024-02-18T00:35:05.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f780f5eb0 osd=1 4 2024-02-18T00:35:05.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 1 4 2024-02-18T00:35:05.343 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 5 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1f780594d0 0x7f1f7811c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 5 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 2 5 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78059360 osd=2 4 2024-02-18T00:35:05.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 5 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f780280a0 con 0x7f1f780594d0 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f77fff700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1f780594d0 0x7f1f7811c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f780594d0 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.341+0000 7f1f77fff700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1f780594d0 0x7f1f7811c540 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.342+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f780280a0 con 0x7f1f780594d0 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.342+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.342+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 5 = system:0 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.342+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78059360 osd=2 4 2024-02-18T00:35:05.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.342+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 2 5 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 6 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 6 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78059360 osd=2 3 2024-02-18T00:35:05.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 2 6 2024-02-18T00:35:05.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78059360 osd=2 4 2024-02-18T00:35:05.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 6 2024-02-18T00:35:05.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.356+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f7811ca30 con 0x7f1f780594d0 2024-02-18T00:35:05.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f7811ca30 con 0x7f1f780594d0 2024-02-18T00:35:05.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78059360 osd=2 4 2024-02-18T00:35:05.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 2 6 2024-02-18T00:35:05.360 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 7 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1f78027770 0x7f1f7811fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 7 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 3 7 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78027600 osd=3 4 2024-02-18T00:35:05.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 7 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f7811a210 con 0x7f1f78027770 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7d0c8700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1f78027770 0x7f1f7811fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f78027770 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.358+0000 7f1f7d0c8700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1f78027770 0x7f1f7811fdc0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.359+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f7811a210 con 0x7f1f78027770 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.359+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.359+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 7 = system:0 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.359+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78027600 osd=3 4 2024-02-18T00:35:05.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.359+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 3 7 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 8 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 8 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78027600 osd=3 3 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 3 8 2024-02-18T00:35:05.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78027600 osd=3 4 2024-02-18T00:35:05.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 8 2024-02-18T00:35:05.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.373+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f781195f0 con 0x7f1f78027770 2024-02-18T00:35:05.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f781195f0 con 0x7f1f78027770 2024-02-18T00:35:05.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78027600 osd=3 4 2024-02-18T00:35:05.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 3 8 2024-02-18T00:35:05.377 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:35:05.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 9 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1f7811d8b0 0x7f1f78123440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 9 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 4 9 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781195f0 osd=4 4 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 9 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f781197c0 con 0x7f1f7811d8b0 2024-02-18T00:35:05.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.375+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1f7811d8b0 0x7f1f78123440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.376+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f7811d8b0 2024-02-18T00:35:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.376+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1f7811d8b0 0x7f1f78123440 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.377+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f781197c0 con 0x7f1f7811d8b0 2024-02-18T00:35:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.377+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:35:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.377+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 9 = system:0 2024-02-18T00:35:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.377+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f781195f0 osd=4 4 2024-02-18T00:35:05.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.377+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 4 9 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 10 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 10 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f781195f0 osd=4 3 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 4 10 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781195f0 osd=4 4 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 10 2024-02-18T00:35:05.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.391+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f781199e0 con 0x7f1f7811d8b0 2024-02-18T00:35:05.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f781199e0 con 0x7f1f7811d8b0 2024-02-18T00:35:05.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f781195f0 osd=4 4 2024-02-18T00:35:05.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 4 10 2024-02-18T00:35:05.395 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:35:05.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 11 2024-02-18T00:35:05.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1f78123e40 0x7f1f781262b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 11 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 5 11 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781199e0 osd=5 4 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 11 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.393+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f781197c0 con 0x7f1f78123e40 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.394+0000 7f1f77fff700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1f78123e40 0x7f1f781262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.394+0000 7f1f77fff700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1f78123e40 0x7f1f781262b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.394+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f78123e40 2024-02-18T00:35:05.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.395+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f781197c0 con 0x7f1f78123e40 2024-02-18T00:35:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.395+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:35:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.395+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 11 = system:0 2024-02-18T00:35:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.395+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f781199e0 osd=5 4 2024-02-18T00:35:05.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.395+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 5 11 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 12 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 12 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f781199e0 osd=5 3 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 5 12 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781199e0 osd=5 4 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 12 2024-02-18T00:35:05.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.409+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f78119760 con 0x7f1f78123e40 2024-02-18T00:35:05.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f78119760 con 0x7f1f78123e40 2024-02-18T00:35:05.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f781199e0 osd=5 4 2024-02-18T00:35:05.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 5 12 2024-02-18T00:35:05.413 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 13 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1f78128380 0x7f1f781287e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 13 2024-02-18T00:35:05.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 6 13 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78119760 osd=6 4 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 13 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.411+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f78128dd0 con 0x7f1f78128380 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.412+0000 7f1f7d0c8700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1f78128380 0x7f1f781287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.412+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f78128380 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.412+0000 7f1f7d0c8700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1f78128380 0x7f1f781287e0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.413+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f78128dd0 con 0x7f1f78128380 2024-02-18T00:35:05.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.413+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:35:05.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.413+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 13 = system:0 2024-02-18T00:35:05.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.413+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78119760 osd=6 4 2024-02-18T00:35:05.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.413+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 6 13 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 14 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 14 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78119760 osd=6 3 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 6 14 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78119760 osd=6 4 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 14 2024-02-18T00:35:05.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.427+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f78128dd0 con 0x7f1f78128380 2024-02-18T00:35:05.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f78128dd0 con 0x7f1f78128380 2024-02-18T00:35:05.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78119760 osd=6 4 2024-02-18T00:35:05.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 6 14 2024-02-18T00:35:05.431 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 15 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6812/53025171 conn(0x7f1f78126dd0 0x7f1f781271f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 15 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 7 15 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78128dd0 osd=7 4 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 15 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.429+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f78127860 con 0x7f1f78126dd0 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.430+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6812/53025171 conn(0x7f1f78126dd0 0x7f1f781271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.430+0000 7f1f75ffb700 10 client.40595.objecter ms_handle_connect 0x7f1f78126dd0 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.430+0000 7f1f7c8c7700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6812/53025171 conn(0x7f1f78126dd0 0x7f1f781271f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.431+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1f78127860 con 0x7f1f78126dd0 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.431+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:35:05.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.431+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 15 = system:0 2024-02-18T00:35:05.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.431+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78128dd0 osd=7 4 2024-02-18T00:35:05.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.431+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 7 15 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:03.720951+0000 mon.b (mon.1) 278 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:03.843048+0000 mgr.y (mgr.24454) 260 : cluster [DBG] pgmap v325: 254 pgs: 1 peering, 8 active+clean+snaptrim, 32 unknown, 14 active+clean+snaptrim_wait, 199 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.063387+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.064384+0000 mon.a (mon.0) 1603 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.606241+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.606407+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.606486+0000 mon.a (mon.0) 1606 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:04.606538+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e226: 8 total, 8 up, 7 in 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.611518+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.612044+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.441 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.644608+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:05.441 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.645965+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:05.441 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.646318+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi059-82877-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:05.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 10 client.40595.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:05.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign -1 16 2024-02-18T00:35:05.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 20 client.40595.objecter _get_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_remove -1 16 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 20 client.40595.objecter get_session s=0x7f1f78128dd0 osd=7 3 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 15 client.40595.objecter _session_command_op_assign 7 16 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78128dd0 osd=7 4 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 10 client.40595.objecter _send_command 16 2024-02-18T00:35:05.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.447+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1f78127860 con 0x7f1f78126dd0 2024-02-18T00:35:05.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f75ffb700 1 -- 172.21.15.59:0/3647302678 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1f78127860 con 0x7f1f78126dd0 2024-02-18T00:35:05.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f75ffb700 10 client.40595.objecter ms_dispatch 0x7f1f78001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:05.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f75ffb700 10 client.40595.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f75ffb700 20 client.40595.objecter put_session s=0x7f1f78128dd0 osd=7 4 2024-02-18T00:35:05.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f75ffb700 15 client.40595.objecter _session_command_op_remove 7 16 2024-02-18T00:35:05.451 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:35:05.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 librados: watch_flush enter 2024-02-18T00:35:05.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 librados: watch_flush exit 2024-02-18T00:35:05.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.0 2024-02-18T00:35:05.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1f780fee00 msgr2=0x7f1f780ff220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1f780fee00 0x7f1f780ff220 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78109600 osd=0 1 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.1 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6805/26279110 conn(0x7f1f78028540 msgr2=0x7f1f78118ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6805/26279110 conn(0x7f1f78028540 0x7f1f78118ec0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f780f5eb0 osd=1 1 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.2 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1f780594d0 msgr2=0x7f1f7811c540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1f780594d0 0x7f1f7811c540 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78059360 osd=2 1 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.3 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1f78027770 msgr2=0x7f1f7811fdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1f78027770 0x7f1f7811fdc0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78027600 osd=3 1 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.4 2024-02-18T00:35:05.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1f7811d8b0 msgr2=0x7f1f78123440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1f7811d8b0 0x7f1f78123440 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7d0c8700 1 -- 172.21.15.59:0/3647302678 reap_dead start 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781195f0 osd=4 1 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.5 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1f78123e40 msgr2=0x7f1f781262b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1f78123e40 0x7f1f781262b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7d0c8700 1 -- 172.21.15.59:0/3647302678 reap_dead start 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7d0c8700 1 -- 172.21.15.59:0/3647302678 reap_dead start 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f781199e0 osd=5 1 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.6 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1f78128380 msgr2=0x7f1f781287e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1f78128380 0x7f1f781287e0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78119760 osd=6 1 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter close_session for osd.7 2024-02-18T00:35:05.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6812/53025171 conn(0x7f1f78126dd0 msgr2=0x7f1f781271f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6812/53025171 conn(0x7f1f78126dd0 0x7f1f781271f0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter put_session s=0x7f1f78128dd0 osd=7 1 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 20 client.40595.objecter shutdown clearing up homeless session... 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 10 client.40595.objecter successfully canceled tick 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1f68062c90 msgr2=0x7f1f68065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1f68062c90 0x7f1f68065150 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 rx=0x7f1f640043d0 tx=0x7f1f6400b040).stop 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780f8420 msgr2=0x7f1f7810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.449+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780f8420 0x7f1f7810df80 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 rx=0x7f1f6c00c7c0 tx=0x7f1f6c007ef0).stop 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 shutdown_connections 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6812/53025171 conn(0x7f1f78126dd0 0x7f1f781271f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1f78128380 0x7f1f781287e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f780f8420 0x7f1f7810df80 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 --2- 172.21.15.59:0/3647302678 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1f68062c90 0x7f1f68065150 unknown :-1 s=CLOSED pgs=3013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 >> 172.21.15.59:0/3647302678 conn(0x7f1f780f3800 msgr2=0x7f1f780f5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 shutdown_connections 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 -- 172.21.15.59:0/3647302678 wait complete. 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:05.450+0000 7f1f7eb2b700 1 librados: shutdown 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.456 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:35:05.458 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:03.628232+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.59:0/1326867146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:03.720951+0000 mon.b (mon.1) 278 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:03.843048+0000 mgr.y (mgr.24454) 260 : cluster [DBG] pgmap v325: 254 pgs: 1 peering, 8 active+clean+snaptrim, 32 unknown, 14 active+clean+snaptrim_wait, 199 active+clean; 10 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.063387+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.064384+0000 mon.a (mon.0) 1603 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "15"}]: dispatch 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.606241+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi059-82877-31"}]': finished 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.606407+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.606486+0000 mon.a (mon.0) 1606 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "15"}]': finished 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:04.606538+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e226: 8 total, 8 up, 7 in 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.611518+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.612044+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.644608+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.645965+0000 mon.a (mon.0) 1610 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.646318+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi059-82877-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:05.833 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:05.836 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:05.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:05.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-18T00:35:05.938 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:05.993 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:05.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:06.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:06.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:04.722088+0000 mon.b (mon.1) 279 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:06.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.306493+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.59:0/3647302678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:06.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.607617+0000 mon.a (mon.0) 1612 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:06.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.607866+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi059-82877-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:06.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:05.607975+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e227: 8 total, 8 up, 7 in 2024-02-18T00:35:06.102 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.609328+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi059-82877-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:06.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.616093+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.59:0/1293895968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:06.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.617098+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:06.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.632320+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:06.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.632913+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:06.103 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:05.723440+0000 mon.b (mon.1) 280 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:06.124 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:04.722088+0000 mon.b (mon.1) 279 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.306493+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.59:0/3647302678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.607617+0000 mon.a (mon.0) 1612 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.607866+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi059-82877-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:05.607975+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e227: 8 total, 8 up, 7 in 2024-02-18T00:35:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.609328+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi059-82877-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.616093+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.59:0/1293895968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:06.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.617098+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:06.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.632320+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:06.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.632913+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:06.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:05.723440+0000 mon.b (mon.1) 280 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:06.271 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:06.278 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:06.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:06.379 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:06.411 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:04.722088+0000 mon.b (mon.1) 279 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.306493+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.59:0/3647302678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.607617+0000 mon.a (mon.0) 1612 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.607866+0000 mon.a (mon.0) 1613 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi059-82877-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:05.607975+0000 mon.a (mon.0) 1614 : cluster [DBG] osdmap e227: 8 total, 8 up, 7 in 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.609328+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi059-82877-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.616093+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.59:0/1293895968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.617098+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.632320+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.632913+0000 mon.a (mon.0) 1617 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:06.422 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:05.723440+0000 mon.b (mon.1) 280 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:06.431 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:06.434 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:06.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:06.536 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:06.567 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:06.588 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:06.590 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:06.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:06.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:06.724 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:06.752 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:06.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:06.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:06.885 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:06.964 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:06.968 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:06.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:07.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:07.245 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:07.248 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:07.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:07.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:07.829 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:07.835 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:05.844254+0000 mgr.y (mgr.24454) 261 : cluster [DBG] pgmap v328: 318 pgs: 32 creating+peering, 2 clean+premerge+peered, 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 263 active+clean; 9.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:35:07.835 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:06.725320+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:07.835 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.055946+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:07.835 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.056227+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:07.836 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:07.056319+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e228: 8 total, 8 up, 7 in 2024-02-18T00:35:07.836 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.057232+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:07.836 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.059569+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:07.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:07.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:07.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:05.844254+0000 mgr.y (mgr.24454) 261 : cluster [DBG] pgmap v328: 318 pgs: 32 creating+peering, 2 clean+premerge+peered, 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 263 active+clean; 9.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:35:07.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:06.725320+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:07.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.055946+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:07.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.056227+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:07.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:07.056319+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e228: 8 total, 8 up, 7 in 2024-02-18T00:35:07.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.057232+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:07.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.059569+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:07.939 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:07.969 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:05.844254+0000 mgr.y (mgr.24454) 261 : cluster [DBG] pgmap v328: 318 pgs: 32 creating+peering, 2 clean+premerge+peered, 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 263 active+clean; 9.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:35:07.969 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:06.725320+0000 mon.b (mon.1) 281 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:07.970 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.055946+0000 mon.a (mon.0) 1618 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:07.970 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.056227+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:07.970 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:07.056319+0000 mon.a (mon.0) 1620 : cluster [DBG] osdmap e228: 8 total, 8 up, 7 in 2024-02-18T00:35:07.970 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.057232+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:07.970 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.059569+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:07.971 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:07.993 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:07.996 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:07.997 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:08.097 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:08.128 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:08.148 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:08.151 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:08.151 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:08.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:08.287 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:08.300 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:08.315 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:08.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:08.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.325+0000 7f1babee0700 1 -- 172.21.15.59:0/1898292871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba40f6a10 msgr2=0x7f1ba40f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:08.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.325+0000 7f1babee0700 1 --2- 172.21.15.59:0/1898292871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba40f6a10 0x7f1ba40f6df0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 rx=0x7f1ba0004870 tx=0x7f1ba0009cb0).stop 2024-02-18T00:35:08.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.325+0000 7f1babee0700 1 -- 172.21.15.59:0/1898292871 shutdown_connections 2024-02-18T00:35:08.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.325+0000 7f1babee0700 1 --2- 172.21.15.59:0/1898292871 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ba4104570 0x7f1ba4106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:35:08.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.325+0000 7f1babee0700 1 --2- 172.21.15.59:0/1898292871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba40f6a10 0x7f1ba40f6df0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:08.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.325+0000 7f1babee0700 1 --2- 172.21.15.59:0/1898292871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ba40f72e0 0x7f1ba4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:08.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.325+0000 7f1babee0700 1 -- 172.21.15.59:0/1898292871 >> 172.21.15.59:0/1898292871 conn(0x7f1ba40f26f0 msgr2=0x7f1ba40f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:08.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.329+0000 7f1babee0700 1 -- 172.21.15.59:0/1898292871 shutdown_connections 2024-02-18T00:35:08.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.330+0000 7f1babee0700 1 -- 172.21.15.59:0/1898292871 wait complete. 2024-02-18T00:35:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 librados: starting msgr at 2024-02-18T00:35:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 librados: starting objecter 2024-02-18T00:35:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 Processor -- start 2024-02-18T00:35:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 -- start start 2024-02-18T00:35:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 librados: setting wanted keys 2024-02-18T00:35:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 librados: calling monclient init 2024-02-18T00:35:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ba40f6a10 0x7f1ba4110610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:08.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ba40f72e0 0x7f1ba4110b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:08.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba4104570 0x7f1ba410a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:08.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba410ace0 con 0x7f1ba4104570 2024-02-18T00:35:08.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba410ae50 con 0x7f1ba40f6a10 2024-02-18T00:35:08.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.331+0000 7f1babee0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba410b010 con 0x7f1ba40f72e0 2024-02-18T00:35:08.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba4104570 0x7f1ba410a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba4104570 0x7f1ba410a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40678/0 (socket says 172.21.15.59:40678) 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 -- 172.21.15.59:0/1887332574 learned_addr learned my addr 172.21.15.59:0/1887332574 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1ba9c7c700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ba40f6a10 0x7f1ba4110610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 -- 172.21.15.59:0/1887332574 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ba40f72e0 msgr2=0x7f1ba4110b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ba40f72e0 0x7f1ba4110b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 -- 172.21.15.59:0/1887332574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ba40f6a10 msgr2=0x7f1ba4110610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ba40f6a10 0x7f1ba4110610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 -- 172.21.15.59:0/1887332574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba0005040 con 0x7f1ba4104570 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1baa47d700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba4104570 0x7f1ba410a7b0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 rx=0x7f1b9c00bb30 tx=0x7f1b9c0097e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1b9affd700 10 client.?.objecter ms_handle_connect 0x7f1ba4104570 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1b9affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:08.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1b9affd700 1 -- 172.21.15.59:0/1887332574 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00c940 con 0x7f1ba4104570 2024-02-18T00:35:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.332+0000 7f1b9affd700 1 -- 172.21.15.59:0/1887332574 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b9c015420 con 0x7f1ba4104570 2024-02-18T00:35:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.333+0000 7f1b9affd700 1 -- 172.21.15.59:0/1887332574 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0143d0 con 0x7f1ba4104570 2024-02-18T00:35:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.333+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba410b2f0 con 0x7f1ba4104570 2024-02-18T00:35:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.333+0000 7f1babee0700 10 client.30780.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.333+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba4117410 con 0x7f1ba4104570 2024-02-18T00:35:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.333+0000 7f1babee0700 1 librados: init done 2024-02-18T00:35:08.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.333+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba405ee80 con 0x7f1ba4104570 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.335+0000 7f1b9affd700 1 -- 172.21.15.59:0/1887332574 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1b9c012020 con 0x7f1ba4104570 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1b9affd700 1 --2- 172.21.15.59:0/1887332574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b90062c90 0x7f1b90065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1b9affd700 1 -- 172.21.15.59:0/1887332574 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(229..229 src has 44..229) v4 ==== 10072+0+0 (secure 0 0 0) 0x7f1b9c07ddb0 con 0x7f1ba4104570 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1b9affd700 10 client.30780.objecter ms_dispatch 0x7f1ba4001550 osd_map(229..229 src has 44..229) v4 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1b9affd700 3 client.30780.objecter handle_osd_map got epochs [229,229] > 0 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1b9affd700 3 client.30780.objecter handle_osd_map decoding full epoch 229 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1ba9c7c700 1 --2- 172.21.15.59:0/1887332574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b90062c90 0x7f1b90065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1b9affd700 20 client.30780.objecter dump_active .. 0 homeless 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1b9affd700 10 client.30780.objecter ms_handle_connect 0x7f1b90062c90 2024-02-18T00:35:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.336+0000 7f1ba9c7c700 1 --2- 172.21.15.59:0/1887332574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b90062c90 0x7f1b90065150 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 rx=0x7f1ba0004840 tx=0x7f1ba0009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.340+0000 7f1b9affd700 1 -- 172.21.15.59:0/1887332574 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1b9c050aa0 con 0x7f1ba4104570 2024-02-18T00:35:08.350 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:08.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.537+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f1ba410bc90 con 0x7f1ba4104570 2024-02-18T00:35:08.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.622+0000 7f1b9affd700 1 -- 172.21.15.59:0/1887332574 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v230) v1 ==== 91+0+0 (secure 0 0 0) 0x7f1b9c050690 con 0x7f1ba4104570 2024-02-18T00:35:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 10 librados: watch_flush enter 2024-02-18T00:35:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 10 librados: watch_flush exit 2024-02-18T00:35:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 20 client.30780.objecter shutdown clearing up homeless session... 2024-02-18T00:35:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 10 client.30780.objecter successfully canceled tick 2024-02-18T00:35:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b90062c90 msgr2=0x7f1b90065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 1 --2- 172.21.15.59:0/1887332574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b90062c90 0x7f1b90065150 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 rx=0x7f1ba0004840 tx=0x7f1ba0009cb0).stop 2024-02-18T00:35:08.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba4104570 msgr2=0x7f1ba410a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:08.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.629+0000 7f1babee0700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba4104570 0x7f1ba410a7b0 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 rx=0x7f1b9c00bb30 tx=0x7f1b9c0097e0).stop 2024-02-18T00:35:08.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.632+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 shutdown_connections 2024-02-18T00:35:08.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.632+0000 7f1babee0700 1 --2- 172.21.15.59:0/1887332574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b90062c90 0x7f1b90065150 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:08.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.632+0000 7f1babee0700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ba40f6a10 0x7f1ba4110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:08.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.632+0000 7f1babee0700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ba4104570 0x7f1ba410a7b0 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:08.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.632+0000 7f1babee0700 1 --2- 172.21.15.59:0/1887332574 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ba40f72e0 0x7f1ba4110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:35:08.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.632+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 >> 172.21.15.59:0/1887332574 conn(0x7f1ba40f26f0 msgr2=0x7f1ba40f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:08.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.637+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 shutdown_connections 2024-02-18T00:35:08.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.637+0000 7f1babee0700 1 -- 172.21.15.59:0/1887332574 wait complete. 2024-02-18T00:35:08.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:08.637+0000 7f1babee0700 1 librados: shutdown 2024-02-18T00:35:08.639 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:35:08.642 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:08.645 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:08.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:08.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:08.773 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:08.792 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:08.794 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:08.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:08.896 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.617044+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi059-82877-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.617206+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]': finished 2024-02-18T00:35:08.920 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:07.617309+0000 mon.a (mon.0) 1624 : cluster [DBG] osdmap e229: 8 total, 8 up, 7 in 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.618892+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]: dispatch 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.620008+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.59:0/2751030111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.620278+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]: dispatch 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.620854+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.726395+0000 mon.b (mon.1) 282 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:07.844723+0000 mgr.y (mgr.24454) 262 : cluster [DBG] pgmap v331: 357 pgs: 32 creating+peering, 1 clean+premerge+peered, 7 active+clean+snaptrim, 40 unknown, 14 active+clean+snaptrim_wait, 263 active+clean; 9.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:07.846697+0000 mon.a (mon.0) 1627 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-18T00:35:08.921 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.539321+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? 172.21.15.59:0/1887332574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:08.922 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:08.941 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:08.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:08.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:08.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:35:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.617044+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi059-82877-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.617206+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]': finished 2024-02-18T00:35:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:07.617309+0000 mon.a (mon.0) 1624 : cluster [DBG] osdmap e229: 8 total, 8 up, 7 in 2024-02-18T00:35:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.618892+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]: dispatch 2024-02-18T00:35:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.620008+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.59:0/2751030111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.620278+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]: dispatch 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.620854+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.726395+0000 mon.b (mon.1) 282 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:07.844723+0000 mgr.y (mgr.24454) 262 : cluster [DBG] pgmap v331: 357 pgs: 32 creating+peering, 1 clean+premerge+peered, 7 active+clean+snaptrim, 40 unknown, 14 active+clean+snaptrim_wait, 263 active+clean; 9.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:07.846697+0000 mon.a (mon.0) 1627 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.539321+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? 172.21.15.59:0/1887332574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.617044+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi059-82877-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.617206+0000 mon.a (mon.0) 1623 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-21"}]': finished 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:07.617309+0000 mon.a (mon.0) 1624 : cluster [DBG] osdmap e229: 8 total, 8 up, 7 in 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.618892+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]: dispatch 2024-02-18T00:35:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.620008+0000 mon.c (mon.2) 192 : audit [INF] from='client.? 172.21.15.59:0/2751030111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:09.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.620278+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]: dispatch 2024-02-18T00:35:09.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.620854+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:09.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.726395+0000 mon.b (mon.1) 282 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:09.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:07.844723+0000 mgr.y (mgr.24454) 262 : cluster [DBG] pgmap v331: 357 pgs: 32 creating+peering, 1 clean+premerge+peered, 7 active+clean+snaptrim, 40 unknown, 14 active+clean+snaptrim_wait, 263 active+clean; 9.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:09.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:07.846697+0000 mon.a (mon.0) 1627 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "14"}]: dispatch 2024-02-18T00:35:09.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.539321+0000 mon.a (mon.0) 1628 : audit [INF] from='client.? 172.21.15.59:0/1887332574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:09.044 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:09.070 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:09.088 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:09.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:09.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:09.191 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:09.217 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:09.236 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:09.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:09.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:09.339 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:09.368 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:09.389 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:09.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:09.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:09.493 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:09.520 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:09.539 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:09.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:09.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:09.642 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:09.669 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:09.688 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:09.690 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:09.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:09.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:09.820 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:09.943 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:09.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:08.614317+0000 mon.a (mon.0) 1629 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:08.614347+0000 mon.a (mon.0) 1630 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.620147+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]': finished 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.620245+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.620414+0000 mon.a (mon.0) 1633 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.620474+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? 172.21.15.59:0/1887332574' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:08.620521+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e230: 8 total, 8 up, 7 in 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.727309+0000 mon.b (mon.1) 283 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.759310+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:08.759626+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:08.847101+0000 osd.5 (osd.5) 103 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:08.848240+0000 osd.5 (osd.5) 104 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:35:09.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:09.619806+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:09.619878+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e231: 8 total, 8 up, 7 in 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:09.623632+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:09.624027+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:09.624374+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:08.614317+0000 mon.a (mon.0) 1629 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:08.614347+0000 mon.a (mon.0) 1630 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.620147+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]': finished 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.620245+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.620414+0000 mon.a (mon.0) 1633 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-18T00:35:09.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.620474+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? 172.21.15.59:0/1887332574' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:08.620521+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e230: 8 total, 8 up, 7 in 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.727309+0000 mon.b (mon.1) 283 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.759310+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:08.759626+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:08.847101+0000 osd.5 (osd.5) 103 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:08.848240+0000 osd.5 (osd.5) 104 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:09.619806+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:09.619878+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e231: 8 total, 8 up, 7 in 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:09.623632+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:09.624027+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:09.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:09.624374+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:09.949 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:09.949 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:08.614317+0000 mon.a (mon.0) 1629 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:08.614347+0000 mon.a (mon.0) 1630 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.620147+0000 mon.a (mon.0) 1631 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-21", "mode": "writeback"}]': finished 2024-02-18T00:35:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.620245+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi059-82882-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.620414+0000 mon.a (mon.0) 1633 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "14"}]': finished 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.620474+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? 172.21.15.59:0/1887332574' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:08.620521+0000 mon.a (mon.0) 1635 : cluster [DBG] osdmap e230: 8 total, 8 up, 7 in 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.727309+0000 mon.b (mon.1) 283 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.759310+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:08.759626+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:08.847101+0000 osd.5 (osd.5) 103 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:08.848240+0000 osd.5 (osd.5) 104 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:09.619806+0000 mon.a (mon.0) 1637 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:09.619878+0000 mon.a (mon.0) 1638 : cluster [DBG] osdmap e231: 8 total, 8 up, 7 in 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:09.623632+0000 mon.c (mon.2) 194 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:09.624027+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:10.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:09.624374+0000 mon.a (mon.0) 1640 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]: dispatch 2024-02-18T00:35:10.050 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:10.077 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:10.095 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:10.097 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:10.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:10.199 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:10.224 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:10.243 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:10.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:10.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:10.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:10.376 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:09.274232+0000 osd.2 (osd.2) 105 : cluster [DBG] 26.7 scrub starts 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:09.277919+0000 osd.2 (osd.2) 106 : cluster [DBG] 26.7 scrub ok 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:09.728304+0000 mon.b (mon.1) 284 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:09.845218+0000 mgr.y (mgr.24454) 263 : cluster [DBG] pgmap v334: 317 pgs: 32 creating+peering, 1 clean+premerge+peered, 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 263 active+clean; 9.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:09.893641+0000 mon.a (mon.0) 1641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:10.617731+0000 mon.a (mon.0) 1642 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:10.621524+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:10.621594+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]': finished 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:10.621689+0000 mon.a (mon.0) 1645 : cluster [DBG] osdmap e232: 8 total, 8 up, 7 in 2024-02-18T00:35:10.869 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:10.626928+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:09.274232+0000 osd.2 (osd.2) 105 : cluster [DBG] 26.7 scrub starts 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:09.277919+0000 osd.2 (osd.2) 106 : cluster [DBG] 26.7 scrub ok 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:09.728304+0000 mon.b (mon.1) 284 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:09.845218+0000 mgr.y (mgr.24454) 263 : cluster [DBG] pgmap v334: 317 pgs: 32 creating+peering, 1 clean+premerge+peered, 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 263 active+clean; 9.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:09.893641+0000 mon.a (mon.0) 1641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:10.617731+0000 mon.a (mon.0) 1642 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:10.621524+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:10.621594+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]': finished 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:10.621689+0000 mon.a (mon.0) 1645 : cluster [DBG] osdmap e232: 8 total, 8 up, 7 in 2024-02-18T00:35:10.870 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:10.626928+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:09.274232+0000 osd.2 (osd.2) 105 : cluster [DBG] 26.7 scrub starts 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:09.277919+0000 osd.2 (osd.2) 106 : cluster [DBG] 26.7 scrub ok 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:09.728304+0000 mon.b (mon.1) 284 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:09.845218+0000 mgr.y (mgr.24454) 263 : cluster [DBG] pgmap v334: 317 pgs: 32 creating+peering, 1 clean+premerge+peered, 7 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 263 active+clean; 9.7 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:09.893641+0000 mon.a (mon.0) 1641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:10.617731+0000 mon.a (mon.0) 1642 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:10.621524+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:10.621594+0000 mon.a (mon.0) 1644 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-21"}]': finished 2024-02-18T00:35:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:10.621689+0000 mon.a (mon.0) 1645 : cluster [DBG] osdmap e232: 8 total, 8 up, 7 in 2024-02-18T00:35:11.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:10.626928+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]: dispatch 2024-02-18T00:35:11.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:35:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:35:11.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:35:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:35:11.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.502+0000 7f3094810700 1 -- 172.21.15.59:0/3942359778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fdbf0 msgr2=0x7f30900fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.502+0000 7f3094810700 1 --2- 172.21.15.59:0/3942359778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fdbf0 0x7f30900fdfd0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 rx=0x7f3084004870 tx=0x7f3084009cb0).stop 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 -- 172.21.15.59:0/3942359778 shutdown_connections 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 --2- 172.21.15.59:0/3942359778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30900f67e0 0x7f3090104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 --2- 172.21.15.59:0/3942359778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fdbf0 0x7f30900fdfd0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 --2- 172.21.15.59:0/3942359778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30900fe550 0x7f30900f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 -- 172.21.15.59:0/3942359778 >> 172.21.15.59:0/3942359778 conn(0x7f3090070030 msgr2=0x7f30900f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 -- 172.21.15.59:0/3942359778 shutdown_connections 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 -- 172.21.15.59:0/3942359778 wait complete. 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 librados: starting msgr at 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 librados: starting objecter 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 Processor -- start 2024-02-18T00:35:11.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 -- start start 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 librados: setting wanted keys 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.503+0000 7f3094810700 1 librados: calling monclient init 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30900f67e0 0x7f309010e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30900fdbf0 0x7f309010e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fe550 0x7f30901082f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30901087e0 con 0x7f30900f67e0 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3090108950 con 0x7f30900fdbf0 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3090108b10 con 0x7f30900fe550 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308ed9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fe550 0x7f30901082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30900f67e0 0x7f309010e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30900f67e0 0x7f309010e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60522/0 (socket says 172.21.15.59:60522) 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308e59c700 1 -- 172.21.15.59:0/1997911068 learned_addr learned my addr 172.21.15.59:0/1997911068 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308dd9b700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30900fdbf0 0x7f309010e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:11.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308ed9d700 1 -- 172.21.15.59:0/1997911068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30900fdbf0 msgr2=0x7f309010e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308ed9d700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30900fdbf0 0x7f309010e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308ed9d700 1 -- 172.21.15.59:0/1997911068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30900f67e0 msgr2=0x7f309010e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308ed9d700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30900f67e0 0x7f309010e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308ed9d700 1 -- 172.21.15.59:0/1997911068 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080005040 con 0x7f30900fe550 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f308ed9d700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fe550 0x7f30901082f0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 rx=0x7f308800c8e0 tx=0x7f3088004ca0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f307f7fe700 10 client.?.objecter ms_handle_connect 0x7f30900fe550 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f307f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f307f7fe700 1 -- 172.21.15.59:0/1997911068 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308800e4a0 con 0x7f30900fe550 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f307f7fe700 1 -- 172.21.15.59:0/1997911068 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f308800ead0 con 0x7f30900fe550 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3084005040 con 0x7f30900fe550 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f307f7fe700 1 -- 172.21.15.59:0/1997911068 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088012a70 con 0x7f30900fe550 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 10 client.40607.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:11.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3090114f80 con 0x7f30900fe550 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.504+0000 7f3094810700 1 librados: init done 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.505+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309005f990 con 0x7f30900fe550 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f307f7fe700 1 -- 172.21.15.59:0/1997911068 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f308800e600 con 0x7f30900fe550 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f307f7fe700 1 --2- 172.21.15.59:0/1997911068 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3078062c90 0x7f3078065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f307f7fe700 1 -- 172.21.15.59:0/1997911068 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(232..232 src has 44..232) v4 ==== 8618+0+0 (secure 0 0 0) 0x7f308807ce10 con 0x7f30900fe550 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f307f7fe700 10 client.40607.objecter ms_dispatch 0x7f3090001550 osd_map(232..232 src has 44..232) v4 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f307f7fe700 3 client.40607.objecter handle_osd_map got epochs [232,232] > 0 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f307f7fe700 3 client.40607.objecter handle_osd_map decoding full epoch 232 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f308e59c700 1 --2- 172.21.15.59:0/1997911068 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3078062c90 0x7f3078065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.506+0000 7f307f7fe700 20 client.40607.objecter dump_active .. 0 homeless 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.507+0000 7f307f7fe700 10 client.40607.objecter ms_handle_connect 0x7f3078062c90 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.507+0000 7f308e59c700 1 --2- 172.21.15.59:0/1997911068 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3078062c90 0x7f3078065150 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 rx=0x7f3084009a00 tx=0x7f308400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:11.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.510+0000 7f307f7fe700 1 -- 172.21.15.59:0/1997911068 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3088048f60 con 0x7f30900fe550 2024-02-18T00:35:11.556 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:11.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:11.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:11.661 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:11.691 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:11.711 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:11.714 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:11.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:11.739+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f3090107110 con 0x7f30900fe550 2024-02-18T00:35:11.816 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:11.846 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:11.965 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:35:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:12.098 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:12.103 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:12.103 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:12.204 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:12.236 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:12.257 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:12.259 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:12.259 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:12.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:12.390 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:12.410 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:12.413 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:12.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:12.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:12] "GET /metrics HTTP/1.1" 200 195099 "" "Prometheus/2.33.4" 2024-02-18T00:35:12.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:12.544 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:12.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.634+0000 7f307f7fe700 1 -- 172.21.15.59:0/1997911068 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v234) v1 ==== 113+0+0 (secure 0 0 0) 0x7f3088017070 con 0x7f30900fe550 2024-02-18T00:35:12.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 10 librados: watch_flush enter 2024-02-18T00:35:12.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 10 librados: watch_flush exit 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 20 client.40607.objecter shutdown clearing up homeless session... 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 10 client.40607.objecter successfully canceled tick 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3078062c90 msgr2=0x7f3078065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 --2- 172.21.15.59:0/1997911068 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3078062c90 0x7f3078065150 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 rx=0x7f3084009a00 tx=0x7f308400b040).stop 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fe550 msgr2=0x7f30901082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fe550 0x7f30901082f0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 rx=0x7f308800c8e0 tx=0x7f3088004ca0).stop 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 shutdown_connections 2024-02-18T00:35:12.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30900fe550 0x7f30901082f0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:12.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30900f67e0 0x7f309010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:12.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 --2- 172.21.15.59:0/1997911068 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3078062c90 0x7f3078065150 secure :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 rx=0x7f3084009a00 tx=0x7f308400b040).stop 2024-02-18T00:35:12.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 --2- 172.21.15.59:0/1997911068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30900fdbf0 0x7f309010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:12.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.637+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 >> 172.21.15.59:0/1997911068 conn(0x7f3090070030 msgr2=0x7f309006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.641+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 shutdown_connections 2024-02-18T00:35:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.641+0000 7f3094810700 1 -- 172.21.15.59:0/1997911068 wait complete. 2024-02-18T00:35:12.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:12.641+0000 7f3094810700 1 librados: shutdown 2024-02-18T00:35:12.644 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.5 to 1 (10000) 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:10.599646+0000 osd.3 (osd.3) 139 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:10.600494+0000 osd.3 (osd.3) 140 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:10.729403+0000 mon.b (mon.1) 285 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:11.056917+0000 mgr.y (mgr.24454) 264 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:11.623349+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:11.623469+0000 mon.a (mon.0) 1648 : cluster [DBG] osdmap e233: 8 total, 8 up, 7 in 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:10.599646+0000 osd.3 (osd.3) 139 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:35:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:10.600494+0000 osd.3 (osd.3) 140 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:35:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:10.729403+0000 mon.b (mon.1) 285 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:11.056917+0000 mgr.y (mgr.24454) 264 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:11.623349+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:11.623469+0000 mon.a (mon.0) 1648 : cluster [DBG] osdmap e233: 8 total, 8 up, 7 in 2024-02-18T00:35:13.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:10.599646+0000 osd.3 (osd.3) 139 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:35:13.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:10.600494+0000 osd.3 (osd.3) 140 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:35:13.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:10.729403+0000 mon.b (mon.1) 285 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:11.056917+0000 mgr.y (mgr.24454) 264 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:13.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:11.623349+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.59:0/300843906' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi059-82877-32"}]': finished 2024-02-18T00:35:13.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:11.623469+0000 mon.a (mon.0) 1648 : cluster [DBG] osdmap e233: 8 total, 8 up, 7 in 2024-02-18T00:35:13.494 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:13.497 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:13.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:13.596 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:35:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:35:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:35:13.596 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:35:13.596 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:35:13.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:13.640 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:11.493268+0000 osd.6 (osd.6) 117 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:35:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:11.494255+0000 osd.6 (osd.6) 118 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:35:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:11.730465+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:11.741095+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.59:0/1997911068' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:35:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:11.778692+0000 osd.5 (osd.5) 105 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:35:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:11.781531+0000 osd.5 (osd.5) 106 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:35:13.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:11.845688+0000 mgr.y (mgr.24454) 265 : cluster [DBG] pgmap v337: 220 pgs: 1 peering, 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 204 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:12.444157+0000 osd.6 (osd.6) 119 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:12.445059+0000 osd.6 (osd.6) 120 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.530220+0000 mon.a (mon.0) 1649 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.530467+0000 mon.a (mon.0) 1650 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.534867+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.535252+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.535610+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi059-82877-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.627093+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.627160+0000 mon.a (mon.0) 1655 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.627238+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi059-82877-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:12.627299+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e234: 8 total, 8 up, 7 in 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.630886+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi059-82877-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.635371+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.59:0/558528338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.635652+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.636146+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.636609+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:12.731399+0000 mon.b (mon.1) 287 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:12.779534+0000 osd.5 (osd.5) 107 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:12.780356+0000 osd.5 (osd.5) 108 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:11.493268+0000 osd.6 (osd.6) 117 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:11.494255+0000 osd.6 (osd.6) 118 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:11.730465+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:11.741095+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.59:0/1997911068' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:11.778692+0000 osd.5 (osd.5) 105 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:35:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:11.781531+0000 osd.5 (osd.5) 106 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:11.845688+0000 mgr.y (mgr.24454) 265 : cluster [DBG] pgmap v337: 220 pgs: 1 peering, 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 204 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:12.444157+0000 osd.6 (osd.6) 119 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:12.445059+0000 osd.6 (osd.6) 120 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.530220+0000 mon.a (mon.0) 1649 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.530467+0000 mon.a (mon.0) 1650 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.534867+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.535252+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.535610+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi059-82877-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.627093+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.627160+0000 mon.a (mon.0) 1655 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.627238+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi059-82877-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:13.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:12.627299+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e234: 8 total, 8 up, 7 in 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.630886+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi059-82877-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.635371+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.59:0/558528338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.635652+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.636146+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.636609+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:12.731399+0000 mon.b (mon.1) 287 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:12.779534+0000 osd.5 (osd.5) 107 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:35:13.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:12.780356+0000 osd.5 (osd.5) 108 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:35:13.842 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:13.844 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:13.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:13.945 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:13.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:11.493268+0000 osd.6 (osd.6) 117 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:35:13.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:11.494255+0000 osd.6 (osd.6) 118 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:35:13.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:11.730465+0000 mon.b (mon.1) 286 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:11.741095+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.59:0/1997911068' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:35:13.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:11.778692+0000 osd.5 (osd.5) 105 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:35:13.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:11.781531+0000 osd.5 (osd.5) 106 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:35:13.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:11.845688+0000 mgr.y (mgr.24454) 265 : cluster [DBG] pgmap v337: 220 pgs: 1 peering, 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 204 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:12.444157+0000 osd.6 (osd.6) 119 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:12.445059+0000 osd.6 (osd.6) 120 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.530220+0000 mon.a (mon.0) 1649 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.530467+0000 mon.a (mon.0) 1650 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "13"}]: dispatch 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.534867+0000 mon.a (mon.0) 1651 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.535252+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.535610+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi059-82877-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.627093+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.627160+0000 mon.a (mon.0) 1655 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "13"}]': finished 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.627238+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi059-82877-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:12.627299+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e234: 8 total, 8 up, 7 in 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.630886+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi059-82877-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.635371+0000 mon.c (mon.2) 196 : audit [INF] from='client.? 172.21.15.59:0/558528338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.635652+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.636146+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.636609+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:13.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:12.731399+0000 mon.b (mon.1) 287 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:13.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:12.779534+0000 osd.5 (osd.5) 107 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:35:13.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:12.780356+0000 osd.5 (osd.5) 108 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:35:13.974 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:13.992 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:13.995 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:13.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:14.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:14.125 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:13.071741+0000 osd.0 (osd.0) 117 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:35:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:13.072635+0000 osd.0 (osd.0) 118 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:35:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:13.357036+0000 osd.2 (osd.2) 107 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:35:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:13.357838+0000 osd.2 (osd.2) 108 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:35:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:13.449304+0000 osd.1 (osd.1) 119 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:35:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:13.452216+0000 osd.1 (osd.1) 120 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:35:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:13.732566+0000 mon.b (mon.1) 288 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:13.846120+0000 mgr.y (mgr.24454) 266 : cluster [DBG] pgmap v340: 284 pgs: 1 peering, 5 active+clean+snaptrim, 64 unknown, 10 active+clean+snaptrim_wait, 204 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:14.050657+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:14.050776+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.050891+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e235: 8 total, 8 up, 7 in 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:13.071741+0000 osd.0 (osd.0) 117 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:13.072635+0000 osd.0 (osd.0) 118 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:13.357036+0000 osd.2 (osd.2) 107 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:13.357838+0000 osd.2 (osd.2) 108 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:13.449304+0000 osd.1 (osd.1) 119 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:13.452216+0000 osd.1 (osd.1) 120 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:13.732566+0000 mon.b (mon.1) 288 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:13.846120+0000 mgr.y (mgr.24454) 266 : cluster [DBG] pgmap v340: 284 pgs: 1 peering, 5 active+clean+snaptrim, 64 unknown, 10 active+clean+snaptrim_wait, 204 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:14.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:14.050657+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:14.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:14.050776+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:14.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.050891+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e235: 8 total, 8 up, 7 in 2024-02-18T00:35:14.902 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 6 1 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:13.071741+0000 osd.0 (osd.0) 117 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:13.072635+0000 osd.0 (osd.0) 118 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:13.357036+0000 osd.2 (osd.2) 107 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:13.357838+0000 osd.2 (osd.2) 108 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:13.449304+0000 osd.1 (osd.1) 119 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:13.452216+0000 osd.1 (osd.1) 120 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:13.732566+0000 mon.b (mon.1) 288 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:13.846120+0000 mgr.y (mgr.24454) 266 : cluster [DBG] pgmap v340: 284 pgs: 1 peering, 5 active+clean+snaptrim, 64 unknown, 10 active+clean+snaptrim_wait, 204 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:14.050657+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:14.050776+0000 mon.a (mon.0) 1662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:15.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.050891+0000 mon.a (mon.0) 1663 : cluster [DBG] osdmap e235: 8 total, 8 up, 7 in 2024-02-18T00:35:15.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-18T00:35:15.475 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:15.477 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:15.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:15.484 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:15.578 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:15.611 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:15.647 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:15.650 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:15.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:15.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.307434+0000 osd.4 (osd.4) 143 : cluster [DBG] 2.f scrub starts 2024-02-18T00:35:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.310914+0000 osd.4 (osd.4) 144 : cluster [DBG] 2.f scrub ok 2024-02-18T00:35:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.478846+0000 osd.6 (osd.6) 121 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:35:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.479699+0000 osd.6 (osd.6) 122 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:35:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:14.611616+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi059-82877-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.611755+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e236: 8 total, 8 up, 7 in 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:14.613715+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? 172.21.15.59:0/4277233923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:14.631517+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:14.631899+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:14.733856+0000 mon.b (mon.1) 289 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.307434+0000 osd.4 (osd.4) 143 : cluster [DBG] 2.f scrub starts 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.310914+0000 osd.4 (osd.4) 144 : cluster [DBG] 2.f scrub ok 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.478846+0000 osd.6 (osd.6) 121 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.479699+0000 osd.6 (osd.6) 122 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:35:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:14.611616+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi059-82877-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:15.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.611755+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e236: 8 total, 8 up, 7 in 2024-02-18T00:35:15.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:14.613715+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? 172.21.15.59:0/4277233923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:15.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:14.631517+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:15.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:14.631899+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:15.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:14.733856+0000 mon.b (mon.1) 289 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:15.795 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.307434+0000 osd.4 (osd.4) 143 : cluster [DBG] 2.f scrub starts 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.310914+0000 osd.4 (osd.4) 144 : cluster [DBG] 2.f scrub ok 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.478846+0000 osd.6 (osd.6) 121 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.479699+0000 osd.6 (osd.6) 122 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:14.611616+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi059-82877-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.611755+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e236: 8 total, 8 up, 7 in 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:14.613715+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? 172.21.15.59:0/4277233923' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:14.631517+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:16.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:14.631899+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:16.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:14.733856+0000 mon.b (mon.1) 289 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.994810+0000 osd.0 (osd.0) 119 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:14.996072+0000 osd.0 (osd.0) 120 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:15.290805+0000 osd.4 (osd.4) 145 : cluster [DBG] 5.e scrub starts 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:15.294282+0000 osd.4 (osd.4) 146 : cluster [DBG] 5.e scrub ok 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:15.470374+0000 osd.6 (osd.6) 123 : cluster [DBG] 4.a scrub starts 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:15.472538+0000 osd.6 (osd.6) 124 : cluster [DBG] 4.a scrub ok 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:15.614200+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? 172.21.15.59:0/4277233923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:16.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:15.614363+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:15.614422+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e237: 8 total, 8 up, 7 in 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:15.620426+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:15.621907+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:15.735333+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:15.847108+0000 mgr.y (mgr.24454) 267 : cluster [DBG] pgmap v343: 323 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 314 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.105899+0000 mon.a (mon.0) 1672 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.617202+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]': finished 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.617353+0000 mon.a (mon.0) 1674 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:16.617410+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e238: 8 total, 8 up, 7 in 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.622902+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:16.913 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.994810+0000 osd.0 (osd.0) 119 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:14.996072+0000 osd.0 (osd.0) 120 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:15.290805+0000 osd.4 (osd.4) 145 : cluster [DBG] 5.e scrub starts 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:15.294282+0000 osd.4 (osd.4) 146 : cluster [DBG] 5.e scrub ok 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:15.470374+0000 osd.6 (osd.6) 123 : cluster [DBG] 4.a scrub starts 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:15.472538+0000 osd.6 (osd.6) 124 : cluster [DBG] 4.a scrub ok 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:15.614200+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? 172.21.15.59:0/4277233923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:15.614363+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:15.614422+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e237: 8 total, 8 up, 7 in 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:15.620426+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:15.621907+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:15.735333+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:15.847108+0000 mgr.y (mgr.24454) 267 : cluster [DBG] pgmap v343: 323 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 314 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:35:16.914 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.105899+0000 mon.a (mon.0) 1672 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-18T00:35:16.915 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.617202+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]': finished 2024-02-18T00:35:16.915 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.617353+0000 mon.a (mon.0) 1674 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-18T00:35:16.915 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:16.617410+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e238: 8 total, 8 up, 7 in 2024-02-18T00:35:16.915 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.622902+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.994810+0000 osd.0 (osd.0) 119 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:35:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:14.996072+0000 osd.0 (osd.0) 120 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:35:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:15.290805+0000 osd.4 (osd.4) 145 : cluster [DBG] 5.e scrub starts 2024-02-18T00:35:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:15.294282+0000 osd.4 (osd.4) 146 : cluster [DBG] 5.e scrub ok 2024-02-18T00:35:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:15.470374+0000 osd.6 (osd.6) 123 : cluster [DBG] 4.a scrub starts 2024-02-18T00:35:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:15.472538+0000 osd.6 (osd.6) 124 : cluster [DBG] 4.a scrub ok 2024-02-18T00:35:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:15.614200+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? 172.21.15.59:0/4277233923' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:15.614363+0000 mon.a (mon.0) 1669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:15.614422+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e237: 8 total, 8 up, 7 in 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:15.620426+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:15.621907+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:15.735333+0000 mon.b (mon.1) 290 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:15.847108+0000 mgr.y (mgr.24454) 267 : cluster [DBG] pgmap v343: 323 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 314 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.105899+0000 mon.a (mon.0) 1672 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "12"}]: dispatch 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.617202+0000 mon.a (mon.0) 1673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-23"}]': finished 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.617353+0000 mon.a (mon.0) 1674 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "12"}]': finished 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:16.617410+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e238: 8 total, 8 up, 7 in 2024-02-18T00:35:17.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.622902+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:16.543349+0000 osd.3 (osd.3) 141 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:35:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:16.544645+0000 osd.3 (osd.3) 142 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:35:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.629524+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? 172.21.15.59:0/2109412080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.633326+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]: dispatch 2024-02-18T00:35:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.637587+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]: dispatch 2024-02-18T00:35:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:16.736284+0000 mon.b (mon.1) 291 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:17.656 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:16.770420+0000 osd.5 (osd.5) 109 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:35:17.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:16.771997+0000 osd.5 (osd.5) 110 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:35:17.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.612784+0000 mon.a (mon.0) 1679 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:17.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:17.624990+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:17.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:17.625076+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? 172.21.15.59:0/2109412080' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:17.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:17.625239+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]': finished 2024-02-18T00:35:17.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.625315+0000 mon.a (mon.0) 1683 : cluster [DBG] osdmap e239: 8 total, 8 up, 7 in 2024-02-18T00:35:17.657 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:17.627686+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:17.662 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:17.662 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:17.763 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:17.792 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:17.811 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:17.813 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:17.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:17.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:16.543349+0000 osd.3 (osd.3) 141 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:16.544645+0000 osd.3 (osd.3) 142 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.629524+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? 172.21.15.59:0/2109412080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.633326+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]: dispatch 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.637587+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]: dispatch 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:16.736284+0000 mon.b (mon.1) 291 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:16.770420+0000 osd.5 (osd.5) 109 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:16.771997+0000 osd.5 (osd.5) 110 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.612784+0000 mon.a (mon.0) 1679 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:17.933 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:17.624990+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:17.934 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:17.625076+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? 172.21.15.59:0/2109412080' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:17.934 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:17.625239+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]': finished 2024-02-18T00:35:17.934 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.625315+0000 mon.a (mon.0) 1683 : cluster [DBG] osdmap e239: 8 total, 8 up, 7 in 2024-02-18T00:35:17.934 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:17.627686+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:17.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:16.543349+0000 osd.3 (osd.3) 141 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:35:17.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:16.544645+0000 osd.3 (osd.3) 142 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:35:17.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.629524+0000 mon.a (mon.0) 1677 : audit [INF] from='client.? 172.21.15.59:0/2109412080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:17.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.633326+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]: dispatch 2024-02-18T00:35:17.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.637587+0000 mon.a (mon.0) 1678 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]: dispatch 2024-02-18T00:35:17.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:16.736284+0000 mon.b (mon.1) 291 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:16.770420+0000 osd.5 (osd.5) 109 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:16.771997+0000 osd.5 (osd.5) 110 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.612784+0000 mon.a (mon.0) 1679 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:17.624990+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:17.625076+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? 172.21.15.59:0/2109412080' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:17.625239+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-23", "mode": "writeback"}]': finished 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.625315+0000 mon.a (mon.0) 1683 : cluster [DBG] osdmap e239: 8 total, 8 up, 7 in 2024-02-18T00:35:17.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:17.627686+0000 mon.a (mon.0) 1684 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]: dispatch 2024-02-18T00:35:17.942 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:17.960 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:17.962 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:17.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:18.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:18.228 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:18.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:18.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:18.331 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/3011551174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fdbb0 msgr2=0x7f1b140fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/3011551174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fdbb0 0x7f1b140fdf90 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 rx=0x7f1b10004870 tx=0x7f1b10009cb0).stop 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/3011551174 shutdown_connections 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/3011551174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b140f8420 0x7f1b14104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/3011551174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fdbb0 0x7f1b140fdf90 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/3011551174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b140fe510 0x7f1b140f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/3011551174 >> 172.21.15.59:0/3011551174 conn(0x7f1b140f3800 msgr2=0x7f1b140f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/3011551174 shutdown_connections 2024-02-18T00:35:18.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/3011551174 wait complete. 2024-02-18T00:35:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 librados: starting msgr at 2024-02-18T00:35:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 librados: starting objecter 2024-02-18T00:35:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.332+0000 7f1b1c3c5700 1 Processor -- start 2024-02-18T00:35:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 -- start start 2024-02-18T00:35:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 librados: setting wanted keys 2024-02-18T00:35:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 librados: calling monclient init 2024-02-18T00:35:18.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b140f8420 0x7f1b1410e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b140fdbb0 0x7f1b1410e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fe510 0x7f1b14108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14108710 con 0x7f1b140fe510 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14108880 con 0x7f1b140f8420 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1c3c5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b14108a40 con 0x7f1b140fdbb0 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b140fdbb0 0x7f1b1410e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b140fdbb0 0x7f1b1410e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43660/0 (socket says 172.21.15.59:43660) 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 -- 172.21.15.59:0/1868921966 learned_addr learned my addr 172.21.15.59:0/1868921966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fe510 0x7f1b14108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b140f8420 0x7f1b1410e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 -- 172.21.15.59:0/1868921966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b140f8420 msgr2=0x7f1b1410e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 --2- 172.21.15.59:0/1868921966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b140f8420 0x7f1b1410e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 -- 172.21.15.59:0/1868921966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fe510 msgr2=0x7f1b14108220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 --2- 172.21.15.59:0/1868921966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fe510 0x7f1b14108220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 -- 172.21.15.59:0/1868921966 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10005040 con 0x7f1b140fdbb0 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b140fe510 0x7f1b14108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b19960700 1 --2- 172.21.15.59:0/1868921966 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b140fdbb0 0x7f1b1410e570 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 rx=0x7f1b0400e970 tx=0x7f1b0400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b0b7fe700 10 client.?.objecter ms_handle_connect 0x7f1b140fdbb0 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.333+0000 7f1b0b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.334+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004660 con 0x7f1b140fdbb0 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.334+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b14108d20 con 0x7f1b140fdbb0 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.334+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0400f040 con 0x7f1b140fdbb0 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.334+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04014790 con 0x7f1b140fdbb0 2024-02-18T00:35:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.334+0000 7f1b1c3c5700 10 client.40646.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.334+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b141090a0 con 0x7f1b140fdbb0 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.334+0000 7f1b1c3c5700 1 librados: init done 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f1b141070a0 con 0x7f1b140fdbb0 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1b040090d0 con 0x7f1b140fdbb0 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b0b7fe700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b00062c90 0x7f1b00065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(239..239 src has 44..239) v4 ==== 9383+0+0 (secure 0 0 0) 0x7f1b0407d2a0 con 0x7f1b140fdbb0 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 osd_map(239..239 src has 44..239) v4 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b0b7fe700 3 client.40646.objecter handle_osd_map got epochs [239,239] > 0 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b0b7fe700 3 client.40646.objecter handle_osd_map decoding full epoch 239 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.335+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b00062c90 0x7f1b00065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b0b7fe700 20 client.40646.objecter dump_active .. 0 homeless 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v239) v1 ==== 54+0+16 (secure 0 0 0) 0x7f1b04084070 con 0x7f1b140fdbb0 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b00062c90 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b00062c90 0x7f1b00065150 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 rx=0x7f1b100043d0 tx=0x7f1b1000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 1 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1b140fee80 0x7f1b140ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 1 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 0 1 2024-02-18T00:35:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14109700 osd=0 4 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 1 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b140ff750 con 0x7f1b140fee80 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1b140fee80 0x7f1b140ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b140fee80 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.336+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1b140fee80 0x7f1b140ff260 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.337+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b140ff750 con 0x7f1b140fee80 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.337+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.337+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 1 = system:0 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.337+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14109700 osd=0 4 2024-02-18T00:35:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.337+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 0 1 2024-02-18T00:35:18.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 2 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 2 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14109700 osd=0 3 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 0 2 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14109700 osd=0 4 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 2 2024-02-18T00:35:18.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.353+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b140ff920 con 0x7f1b140fee80 2024-02-18T00:35:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b140ff920 con 0x7f1b140fee80 2024-02-18T00:35:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14109700 osd=0 4 2024-02-18T00:35:18.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 0 2 2024-02-18T00:35:18.356 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:35:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 3 2024-02-18T00:35:18.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6805/26279110 conn(0x7f1b14028540 0x7f1b141190f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 3 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 1 3 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b140f5e80 osd=1 4 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 3 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.354+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b140f5ff0 con 0x7f1b14028540 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.355+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6805/26279110 conn(0x7f1b14028540 0x7f1b141190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.355+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b14028540 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.355+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6805/26279110 conn(0x7f1b14028540 0x7f1b141190f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.355+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b140f5ff0 con 0x7f1b14028540 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.355+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:35:18.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.356+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 3 = system:0 2024-02-18T00:35:18.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.356+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b140f5e80 osd=1 4 2024-02-18T00:35:18.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.356+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 1 3 2024-02-18T00:35:18.360 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:18.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.175+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1868341071 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 msgr2=0x7fbb10104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.175+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1868341071 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb10104a20 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 rx=0x7fbb08009be0 tx=0x7fbb08004d10).stop 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1868341071 shutdown_connections 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1868341071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb100fdbb0 0x7fbb100fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1868341071 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb10104a20 unknown :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1868341071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb100fe510 0x7fbb100f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1868341071 >> 172.21.15.59:0/1868341071 conn(0x7fbb100f3800 msgr2=0x7fbb100f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1868341071 shutdown_connections 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1868341071 wait complete. 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 librados: starting msgr at 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 librados: starting objecter 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 Processor -- start 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 -- start start 2024-02-18T00:35:18.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 librados: setting wanted keys 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.176+0000 7fbb14e9c700 1 librados: calling monclient init 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb1006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb100fdbb0 0x7fbb1006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb100fe510 0x7fbb1006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101162b0 con 0x7fbb100fe510 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb10116420 con 0x7fbb100fdbb0 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101165e0 con 0x7fbb100f8420 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb1006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb1006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43656/0 (socket says 172.21.15.59:43656) 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 -- 172.21.15.59:0/1034275778 learned_addr learned my addr 172.21.15.59:0/1034275778 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0ed9d700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb100fe510 0x7fbb1006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0dd9b700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb100fdbb0 0x7fbb1006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 -- 172.21.15.59:0/1034275778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb100fdbb0 msgr2=0x7fbb1006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb100fdbb0 0x7fbb1006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 -- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb100fe510 msgr2=0x7fbb1006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb100fe510 0x7fbb1006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 -- 172.21.15.59:0/1034275778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080097c0 con 0x7fbb100f8420 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0ed9d700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb100fe510 0x7fbb1006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb0e59c700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb1006b0d0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 rx=0x7fbb000027f0 tx=0x7fbb00002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbaff7fe700 10 client.?.objecter ms_handle_connect 0x7fbb100f8420 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbaff7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbaff7fe700 1 -- 172.21.15.59:0/1034275778 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00007b40 con 0x7fbb100f8420 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb101168c0 con 0x7fbb100f8420 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbaff7fe700 1 -- 172.21.15.59:0/1034275778 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb0000f040 con 0x7fbb100f8420 2024-02-18T00:35:18.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbaff7fe700 1 -- 172.21.15.59:0/1034275778 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb000146f0 con 0x7fbb100f8420 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 10 client.40640.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb10116e30 con 0x7fbb100f8420 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.177+0000 7fbb14e9c700 1 librados: init done 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.178+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb10107110 con 0x7fbb100f8420 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbaff7fe700 1 -- 172.21.15.59:0/1034275778 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fbb00009110 con 0x7fbb100f8420 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbaff7fe700 1 --2- 172.21.15.59:0/1034275778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbaf8062c90 0x7fbaf8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbaff7fe700 1 -- 172.21.15.59:0/1034275778 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(239..239 src has 44..239) v4 ==== 9383+0+0 (secure 0 0 0) 0x7fbb0007d4a0 con 0x7fbb100f8420 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbaff7fe700 10 client.40640.objecter ms_dispatch 0x7fbb10001550 osd_map(239..239 src has 44..239) v4 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbaff7fe700 3 client.40640.objecter handle_osd_map got epochs [239,239] > 0 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbaff7fe700 3 client.40640.objecter handle_osd_map decoding full epoch 239 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbb0dd9b700 1 --2- 172.21.15.59:0/1034275778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbaf8062c90 0x7fbaf8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.179+0000 7fbaff7fe700 20 client.40640.objecter dump_active .. 0 homeless 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.180+0000 7fbaff7fe700 10 client.40640.objecter ms_handle_connect 0x7fbaf8062c90 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.180+0000 7fbb0dd9b700 1 --2- 172.21.15.59:0/1034275778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbaf8062c90 0x7fbaf8065150 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 rx=0x7fbb1006c870 tx=0x7fbb0400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.183+0000 7fbaff7fe700 1 -- 172.21.15.59:0/1034275778 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb00050420 con 0x7fbb100f8420 2024-02-18T00:35:18.380 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:18.382 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:18.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 4 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 4 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b140f5e80 osd=1 3 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 1 4 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b140f5e80 osd=1 4 2024-02-18T00:35:18.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 4 2024-02-18T00:35:18.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.386+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b1405a040 con 0x7f1b14028540 2024-02-18T00:35:18.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b1405a040 con 0x7f1b14028540 2024-02-18T00:35:18.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b140f5e80 osd=1 4 2024-02-18T00:35:18.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 1 4 2024-02-18T00:35:18.390 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:35:18.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 5 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1b140594d0 0x7f1b1411c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 5 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 2 5 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14059360 osd=2 4 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 5 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b14028130 con 0x7f1b140594d0 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b19960700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1b140594d0 0x7f1b1411c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b140594d0 2024-02-18T00:35:18.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7f1b19960700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1b140594d0 0x7f1b1411c770 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.388+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fbb1005fa60 con 0x7fbb100f8420 2024-02-18T00:35:18.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.389+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b14028130 con 0x7f1b140594d0 2024-02-18T00:35:18.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.389+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:35:18.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.389+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 5 = system:0 2024-02-18T00:35:18.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.389+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14059360 osd=2 4 2024-02-18T00:35:18.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.389+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 2 5 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 6 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 6 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14059360 osd=2 3 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 2 6 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14059360 osd=2 4 2024-02-18T00:35:18.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 6 2024-02-18T00:35:18.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.403+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b14027600 con 0x7f1b140594d0 2024-02-18T00:35:18.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b14027600 con 0x7f1b140594d0 2024-02-18T00:35:18.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14059360 osd=2 4 2024-02-18T00:35:18.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 2 6 2024-02-18T00:35:18.407 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:35:18.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 7 2024-02-18T00:35:18.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1b1411a2c0 0x7f1b1411fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 7 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 3 7 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14027a10 osd=3 4 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 7 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.405+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b14122a40 con 0x7f1b1411a2c0 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.406+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1b1411a2c0 0x7f1b1411fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.406+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b1411a2c0 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.406+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1b1411a2c0 0x7f1b1411fdf0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.407+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b14122a40 con 0x7f1b1411a2c0 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.407+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.407+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 7 = system:0 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.407+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14027a10 osd=3 4 2024-02-18T00:35:18.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.407+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 3 7 2024-02-18T00:35:18.414 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:18.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 8 2024-02-18T00:35:18.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 8 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14027a10 osd=3 3 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 3 8 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14027a10 osd=3 4 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 8 2024-02-18T00:35:18.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.425+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b14119820 con 0x7f1b1411a2c0 2024-02-18T00:35:18.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.427+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b14119820 con 0x7f1b1411a2c0 2024-02-18T00:35:18.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.427+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.427+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.427+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14027a10 osd=3 4 2024-02-18T00:35:18.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.427+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 3 8 2024-02-18T00:35:18.429 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:35:18.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 9 2024-02-18T00:35:18.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1b141199c0 0x7f1b14121460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 9 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 4 9 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14119820 osd=4 4 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 9 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b1411db40 con 0x7f1b141199c0 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1b141199c0 0x7f1b14121460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1b141199c0 0x7f1b14121460 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.428+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b141199c0 2024-02-18T00:35:18.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.429+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b1411db40 con 0x7f1b141199c0 2024-02-18T00:35:18.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.429+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:35:18.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.429+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 9 = system:0 2024-02-18T00:35:18.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.429+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14119820 osd=4 4 2024-02-18T00:35:18.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.429+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 4 9 2024-02-18T00:35:18.452 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:18.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 10 2024-02-18T00:35:18.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 10 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14119820 osd=4 3 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 4 10 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14119820 osd=4 4 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 10 2024-02-18T00:35:18.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.454+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b1411db40 con 0x7f1b141199c0 2024-02-18T00:35:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b1411db40 con 0x7f1b141199c0 2024-02-18T00:35:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14119820 osd=4 4 2024-02-18T00:35:18.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 4 10 2024-02-18T00:35:18.458 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 11 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1b14121e60 0x7f1b141222c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 11 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 5 11 2024-02-18T00:35:18.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b1411db40 osd=5 4 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 11 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b141227b0 con 0x7f1b14121e60 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.456+0000 7f1b19960700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1b14121e60 0x7f1b141222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.457+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b14121e60 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.457+0000 7f1b19960700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1b14121e60 0x7f1b141222c0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.457+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b141227b0 con 0x7f1b14121e60 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.457+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.457+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 11 = system:0 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.457+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b1411db40 osd=5 4 2024-02-18T00:35:18.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.457+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 5 11 2024-02-18T00:35:18.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 12 2024-02-18T00:35:18.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 12 2024-02-18T00:35:18.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b1411db40 osd=5 3 2024-02-18T00:35:18.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 5 12 2024-02-18T00:35:18.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b1411db40 osd=5 4 2024-02-18T00:35:18.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 12 2024-02-18T00:35:18.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.473+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b141227b0 con 0x7f1b14121e60 2024-02-18T00:35:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.475+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b141227b0 con 0x7f1b14121e60 2024-02-18T00:35:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.475+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.475+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.475+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b1411db40 osd=5 4 2024-02-18T00:35:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.475+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 5 12 2024-02-18T00:35:18.478 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 13 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1b14125150 0x7f1b141255b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 13 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 6 13 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b141227b0 osd=6 4 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 13 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b14127120 con 0x7f1b14125150 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1b14125150 0x7f1b141255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b14125150 2024-02-18T00:35:18.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.476+0000 7f1b1a962700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1b14125150 0x7f1b141255b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.477+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b14127120 con 0x7f1b14125150 2024-02-18T00:35:18.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.477+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:35:18.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.477+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 13 = system:0 2024-02-18T00:35:18.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.477+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b141227b0 osd=6 4 2024-02-18T00:35:18.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.477+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 6 13 2024-02-18T00:35:18.484 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:18.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 14 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 14 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b141227b0 osd=6 3 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 6 14 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b141227b0 osd=6 4 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 14 2024-02-18T00:35:18.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.492+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b14127120 con 0x7f1b14125150 2024-02-18T00:35:18.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b14127120 con 0x7f1b14125150 2024-02-18T00:35:18.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b141227b0 osd=6 4 2024-02-18T00:35:18.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 6 14 2024-02-18T00:35:18.496 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 15 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6812/53025171 conn(0x7f1b1412dd10 0x7f1b1412e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 15 2024-02-18T00:35:18.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 7 15 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14127120 osd=7 4 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 15 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b1412e7c0 con 0x7f1b1412dd10 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6812/53025171 conn(0x7f1b1412dd10 0x7f1b1412e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b0b7fe700 10 client.40646.objecter ms_handle_connect 0x7f1b1412dd10 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.494+0000 7f1b1a161700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6812/53025171 conn(0x7f1b1412dd10 0x7f1b1412e150 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.495+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1b1001a040 con 0x7f1b1412dd10 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.495+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.495+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 15 = system:0 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.495+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14127120 osd=7 4 2024-02-18T00:35:18.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.495+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 7 15 2024-02-18T00:35:18.511 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 10 client.40646.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign -1 16 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 20 client.40646.objecter _get_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_remove -1 16 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 20 client.40646.objecter get_session s=0x7f1b14127120 osd=7 3 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 15 client.40646.objecter _session_command_op_assign 7 16 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14127120 osd=7 4 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 10 client.40646.objecter _send_command 16 2024-02-18T00:35:18.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.510+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f1b141309a0 con 0x7f1b1412dd10 2024-02-18T00:35:18.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b0b7fe700 1 -- 172.21.15.59:0/1868921966 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f1b141309a0 con 0x7f1b1412dd10 2024-02-18T00:35:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b0b7fe700 10 client.40646.objecter ms_dispatch 0x7f1b14001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b0b7fe700 10 client.40646.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b0b7fe700 20 client.40646.objecter put_session s=0x7f1b14127120 osd=7 4 2024-02-18T00:35:18.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b0b7fe700 15 client.40646.objecter _session_command_op_remove 7 16 2024-02-18T00:35:18.514 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 librados: watch_flush enter 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 librados: watch_flush exit 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.0 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1b140fee80 msgr2=0x7f1b140ff260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1b140fee80 0x7f1b140ff260 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14109700 osd=0 1 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.1 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6805/26279110 conn(0x7f1b14028540 msgr2=0x7f1b141190f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6805/26279110 conn(0x7f1b14028540 0x7f1b141190f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b140f5e80 osd=1 1 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.2 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1b140594d0 msgr2=0x7f1b1411c770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1b140594d0 0x7f1b1411c770 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14059360 osd=2 1 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.3 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1b1411a2c0 msgr2=0x7f1b1411fdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1b1411a2c0 0x7f1b1411fdf0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14027a10 osd=3 1 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1a962700 1 -- 172.21.15.59:0/1868921966 reap_dead start 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.4 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1b141199c0 msgr2=0x7f1b14121460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1b141199c0 0x7f1b14121460 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1a962700 1 -- 172.21.15.59:0/1868921966 reap_dead start 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14119820 osd=4 1 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.5 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1b14121e60 msgr2=0x7f1b141222c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1b14121e60 0x7f1b141222c0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b1411db40 osd=5 1 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.6 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1b14125150 msgr2=0x7f1b141255b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1b14125150 0x7f1b141255b0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b141227b0 osd=6 1 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter close_session for osd.7 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6812/53025171 conn(0x7f1b1412dd10 msgr2=0x7f1b1412e150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.137:6812/53025171 conn(0x7f1b1412dd10 0x7f1b1412e150 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter put_session s=0x7f1b14127120 osd=7 1 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 20 client.40646.objecter shutdown clearing up homeless session... 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 10 client.40646.objecter successfully canceled tick 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b00062c90 msgr2=0x7f1b00065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1b00062c90 0x7f1b00065150 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 rx=0x7f1b100043d0 tx=0x7f1b1000b040).stop 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b140fdbb0 msgr2=0x7f1b1410e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 --2- 172.21.15.59:0/1868921966 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b140fdbb0 0x7f1b1410e570 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 rx=0x7f1b0400e970 tx=0x7f1b0400b040).stop 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1a962700 1 -- 172.21.15.59:0/1868921966 reap_dead start 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 shutdown_connections 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 >> 172.21.15.59:0/1868921966 conn(0x7f1b140f3800 msgr2=0x7f1b140f52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 shutdown_connections 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 -- 172.21.15.59:0/1868921966 wait complete. 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.512+0000 7f1b1c3c5700 1 librados: shutdown 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.519 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.520 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:35:18.531 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:18.534 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:18.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:18.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.615+0000 7fbaff7fe700 1 -- 172.21.15.59:0/1034275778 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v240) v1 ==== 101+0+0 (secure 0 0 0) 0x7fbb00050010 con 0x7fbb100f8420 2024-02-18T00:35:18.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.618+0000 7fbb14e9c700 10 librados: watch_flush enter 2024-02-18T00:35:18.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.618+0000 7fbb14e9c700 10 librados: watch_flush exit 2024-02-18T00:35:18.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 20 client.40640.objecter shutdown clearing up homeless session... 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 10 client.40640.objecter successfully canceled tick 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbaf8062c90 msgr2=0x7fbaf8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1034275778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbaf8062c90 0x7fbaf8065150 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 rx=0x7fbb1006c870 tx=0x7fbb0400b040).stop 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 msgr2=0x7fbb1006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb1006b0d0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 rx=0x7fbb000027f0 tx=0x7fbb00002b00).stop 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 shutdown_connections 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb100fe510 0x7fbb1006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbb100f8420 0x7fbb1006b0d0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1034275778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbaf8062c90 0x7fbaf8065150 secure :-1 s=CLOSED pgs=3022 cs=0 l=1 rev1=1 rx=0x7fbb1006c870 tx=0x7fbb0400b040).stop 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 --2- 172.21.15.59:0/1034275778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb100fdbb0 0x7fbb1006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:18.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.619+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 >> 172.21.15.59:0/1034275778 conn(0x7fbb100f3800 msgr2=0x7fbb100f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.626+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 shutdown_connections 2024-02-18T00:35:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.626+0000 7fbb14e9c700 1 -- 172.21.15.59:0/1034275778 wait complete. 2024-02-18T00:35:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:18.626+0000 7fbb14e9c700 1 librados: shutdown 2024-02-18T00:35:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:35:18.634 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:18.667 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:18.789 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:18.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.498362+0000 osd.6 (osd.6) 125 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:35:18.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.499989+0000 osd.6 (osd.6) 126 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:35:18.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.512209+0000 osd.3 (osd.3) 143 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:35:18.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.514017+0000 osd.3 (osd.3) 144 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:17.737074+0000 mon.b (mon.1) 292 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.812927+0000 osd.5 (osd.5) 111 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.815820+0000 osd.5 (osd.5) 112 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:17.847563+0000 mgr.y (mgr.24454) 268 : cluster [DBG] pgmap v346: 347 pgs: 3 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 306 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.336781+0000 mon.c (mon.2) 201 : audit [DBG] from='client.? 172.21.15.59:0/1868921966' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.390419+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.59:0/1034275778' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.390815+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:18.603646+0000 mon.a (mon.0) 1686 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.609444+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.609631+0000 mon.a (mon.0) 1688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:18.609691+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e240: 8 total, 8 up, 7 in 2024-02-18T00:35:18.792 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.621615+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? 172.21.15.59:0/289331174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.498362+0000 osd.6 (osd.6) 125 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.499989+0000 osd.6 (osd.6) 126 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.512209+0000 osd.3 (osd.3) 143 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.514017+0000 osd.3 (osd.3) 144 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:17.737074+0000 mon.b (mon.1) 292 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.812927+0000 osd.5 (osd.5) 111 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.815820+0000 osd.5 (osd.5) 112 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:17.847563+0000 mgr.y (mgr.24454) 268 : cluster [DBG] pgmap v346: 347 pgs: 3 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 306 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:35:18.793 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.336781+0000 mon.c (mon.2) 201 : audit [DBG] from='client.? 172.21.15.59:0/1868921966' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.390419+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.59:0/1034275778' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.390815+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:18.603646+0000 mon.a (mon.0) 1686 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.609444+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.609631+0000 mon.a (mon.0) 1688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:18.609691+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e240: 8 total, 8 up, 7 in 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.621615+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? 172.21.15.59:0/289331174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.643349+0000 mon.b (mon.1) 293 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.643591+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.794 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.644945+0000 mon.b (mon.1) 294 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.645145+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.645973+0000 mon.b (mon.1) 295 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.646210+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.647173+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.647479+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.643349+0000 mon.b (mon.1) 293 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.643591+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.644945+0000 mon.b (mon.1) 294 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.645145+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.795 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.645973+0000 mon.b (mon.1) 295 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:18.796 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.646210+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:18.796 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.647173+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:18.796 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.647479+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:18.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:18.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:18.897 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.498362+0000 osd.6 (osd.6) 125 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.499989+0000 osd.6 (osd.6) 126 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.512209+0000 osd.3 (osd.3) 143 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.514017+0000 osd.3 (osd.3) 144 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:17.737074+0000 mon.b (mon.1) 292 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.812927+0000 osd.5 (osd.5) 111 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.815820+0000 osd.5 (osd.5) 112 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:17.847563+0000 mgr.y (mgr.24454) 268 : cluster [DBG] pgmap v346: 347 pgs: 3 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 306 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.336781+0000 mon.c (mon.2) 201 : audit [DBG] from='client.? 172.21.15.59:0/1868921966' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.390419+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.59:0/1034275778' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:18.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.390815+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:18.603646+0000 mon.a (mon.0) 1686 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.609444+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? 172.21.15.59:0/49510408' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi059-82877-33"}]': finished 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.609631+0000 mon.a (mon.0) 1688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:18.609691+0000 mon.a (mon.0) 1689 : cluster [DBG] osdmap e240: 8 total, 8 up, 7 in 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.621615+0000 mon.a (mon.0) 1690 : audit [INF] from='client.? 172.21.15.59:0/289331174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.643349+0000 mon.b (mon.1) 293 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.643591+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.644945+0000 mon.b (mon.1) 294 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.645145+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.645973+0000 mon.b (mon.1) 295 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.646210+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.647173+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:18.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.647479+0000 mon.a (mon.0) 1694 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:18.943 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:18.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:18.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:19.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:19.076 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:19.224 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:19.232 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:19.233 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:19.334 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:19.362 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:19.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:19.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:19.485 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:19.513 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:19.533 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:19.535 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:19.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:19.636 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:19.682 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:19.685 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:19.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:19.786 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:19.814 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:19.852 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:19.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:19.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:18.492214+0000 osd.6 (osd.6) 127 : cluster [DBG] 25.d scrub starts 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:18.493091+0000 osd.6 (osd.6) 128 : cluster [DBG] 25.d scrub ok 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:18.738321+0000 mon.b (mon.1) 296 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:18.788676+0000 osd.5 (osd.5) 113 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:18.791559+0000 osd.5 (osd.5) 114 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.611800+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.59:0/289331174' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.611925+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:19.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.612039+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.612118+0000 mon.a (mon.0) 1698 : cluster [DBG] osdmap e241: 8 total, 8 up, 7 in 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.622662+0000 mon.b (mon.1) 297 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.628293+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.631469+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.632248+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:18.492214+0000 osd.6 (osd.6) 127 : cluster [DBG] 25.d scrub starts 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:18.493091+0000 osd.6 (osd.6) 128 : cluster [DBG] 25.d scrub ok 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:18.738321+0000 mon.b (mon.1) 296 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:18.788676+0000 osd.5 (osd.5) 113 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:35:19.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:18.791559+0000 osd.5 (osd.5) 114 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.611800+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.59:0/289331174' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.611925+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.612039+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.612118+0000 mon.a (mon.0) 1698 : cluster [DBG] osdmap e241: 8 total, 8 up, 7 in 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.622662+0000 mon.b (mon.1) 297 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.628293+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.631469+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:19.918 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.632248+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:19.955 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:19.985 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:20.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:18.492214+0000 osd.6 (osd.6) 127 : cluster [DBG] 25.d scrub starts 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:18.493091+0000 osd.6 (osd.6) 128 : cluster [DBG] 25.d scrub ok 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:18.738321+0000 mon.b (mon.1) 296 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:18.788676+0000 osd.5 (osd.5) 113 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:18.791559+0000 osd.5 (osd.5) 114 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.611800+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? 172.21.15.59:0/289331174' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.611925+0000 mon.a (mon.0) 1696 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi059-82877-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.612039+0000 mon.a (mon.0) 1697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.612118+0000 mon.a (mon.0) 1698 : cluster [DBG] osdmap e241: 8 total, 8 up, 7 in 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.622662+0000 mon.b (mon.1) 297 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.628293+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.631469+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:20.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.632248+0000 mon.a (mon.0) 1700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]: dispatch 2024-02-18T00:35:20.110 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:20.113 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:20.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:20.214 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:20.242 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:20.264 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:20.267 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:20.267 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:20.368 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:20.400 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:20.422 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:20.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:20.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:20.526 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:20.555 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:20.575 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:20.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:20.578 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:20.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:20.708 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:20.808 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:20.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:20.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:20.910 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:20.942 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:20.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:20.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:21.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:21.095 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:21.193 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:21.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:21.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:21.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:35:21.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:35:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.262593+0000 osd.4 (osd.4) 147 : cluster [DBG] 4.1a deep-scrub starts 2024-02-18T00:35:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.264575+0000 osd.4 (osd.4) 148 : cluster [DBG] 4.1a deep-scrub ok 2024-02-18T00:35:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.509636+0000 osd.3 (osd.3) 145 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:35:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.511008+0000 osd.3 (osd.3) 146 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:35:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:19.739237+0000 mon.b (mon.1) 298 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.806806+0000 osd.5 (osd.5) 115 : cluster [DBG] 5.f scrub starts 2024-02-18T00:35:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.809704+0000 osd.5 (osd.5) 116 : cluster [DBG] 5.f scrub ok 2024-02-18T00:35:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.848114+0000 mgr.y (mgr.24454) 269 : cluster [DBG] pgmap v349: 378 pgs: 3 active+clean+snaptrim, 64 unknown, 6 active+clean+snaptrim_wait, 305 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.915315+0000 osd.1 (osd.1) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:19.915595+0000 osd.1 (osd.1) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:20.109157+0000 mon.a (mon.0) 1701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-02-18T00:35:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:20.608229+0000 mon.a (mon.0) 1702 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:21.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:21.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.262593+0000 osd.4 (osd.4) 147 : cluster [DBG] 4.1a deep-scrub starts 2024-02-18T00:35:21.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.264575+0000 osd.4 (osd.4) 148 : cluster [DBG] 4.1a deep-scrub ok 2024-02-18T00:35:21.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.509636+0000 osd.3 (osd.3) 145 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:35:21.310 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.511008+0000 osd.3 (osd.3) 146 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:19.739237+0000 mon.b (mon.1) 298 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.806806+0000 osd.5 (osd.5) 115 : cluster [DBG] 5.f scrub starts 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.809704+0000 osd.5 (osd.5) 116 : cluster [DBG] 5.f scrub ok 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.848114+0000 mgr.y (mgr.24454) 269 : cluster [DBG] pgmap v349: 378 pgs: 3 active+clean+snaptrim, 64 unknown, 6 active+clean+snaptrim_wait, 305 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.915315+0000 osd.1 (osd.1) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:19.915595+0000 osd.1 (osd.1) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:20.109157+0000 mon.a (mon.0) 1701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:20.608229+0000 mon.a (mon.0) 1702 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.262593+0000 osd.4 (osd.4) 147 : cluster [DBG] 4.1a deep-scrub starts 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.264575+0000 osd.4 (osd.4) 148 : cluster [DBG] 4.1a deep-scrub ok 2024-02-18T00:35:21.311 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.509636+0000 osd.3 (osd.3) 145 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.511008+0000 osd.3 (osd.3) 146 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:19.739237+0000 mon.b (mon.1) 298 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.806806+0000 osd.5 (osd.5) 115 : cluster [DBG] 5.f scrub starts 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.809704+0000 osd.5 (osd.5) 116 : cluster [DBG] 5.f scrub ok 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.848114+0000 mgr.y (mgr.24454) 269 : cluster [DBG] pgmap v349: 378 pgs: 3 active+clean+snaptrim, 64 unknown, 6 active+clean+snaptrim_wait, 305 active+clean; 9.5 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.915315+0000 osd.1 (osd.1) 121 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:19.915595+0000 osd.1 (osd.1) 122 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:20.109157+0000 mon.a (mon.0) 1701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-02-18T00:35:21.312 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:20.608229+0000 mon.a (mon.0) 1702 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:21.325 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:21.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:20.428019+0000 osd.1 (osd.1) 123 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:20.428890+0000 osd.1 (osd.1) 124 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:20.740144+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:21.045021+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]': finished 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:21.045186+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "11"}]': finished 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:21.045268+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e242: 8 total, 8 up, 7 in 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:21.047130+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? 172.21.15.59:0/330719652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:21.066887+0000 mgr.y (mgr.24454) 270 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:21.617480+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:21.617713+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.59:0/330719652' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:22.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:21.617831+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e243: 8 total, 8 up, 7 in 2024-02-18T00:35:22.444 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:22] "GET /metrics HTTP/1.1" 200 195099 "" "Prometheus/2.33.4" 2024-02-18T00:35:22.444 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:20.428019+0000 osd.1 (osd.1) 123 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:35:22.444 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:20.428890+0000 osd.1 (osd.1) 124 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:35:22.444 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:20.740144+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:22.444 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:21.045021+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]': finished 2024-02-18T00:35:22.444 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:21.045186+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "11"}]': finished 2024-02-18T00:35:22.444 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:21.045268+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e242: 8 total, 8 up, 7 in 2024-02-18T00:35:22.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:21.047130+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? 172.21.15.59:0/330719652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:22.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:21.066887+0000 mgr.y (mgr.24454) 270 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:22.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:21.617480+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:22.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:21.617713+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.59:0/330719652' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:22.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:21.617831+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e243: 8 total, 8 up, 7 in 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:20.428019+0000 osd.1 (osd.1) 123 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:20.428890+0000 osd.1 (osd.1) 124 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:20.740144+0000 mon.b (mon.1) 299 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:21.045021+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-23"}]': finished 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:21.045186+0000 mon.a (mon.0) 1704 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi059-82917-1", "var": "pg_num_actual", "val": "11"}]': finished 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:21.045268+0000 mon.a (mon.0) 1705 : cluster [DBG] osdmap e242: 8 total, 8 up, 7 in 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:21.047130+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? 172.21.15.59:0/330719652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:21.066887+0000 mgr.y (mgr.24454) 270 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:21.617480+0000 mon.a (mon.0) 1707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi059-82877-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:21.617713+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? 172.21.15.59:0/330719652' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi059-82882-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:21.617831+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e243: 8 total, 8 up, 7 in 2024-02-18T00:35:22.875 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:22.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:22.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:22.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:23.006 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:23.108 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:23.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:23.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:23.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:21.471933+0000 osd.1 (osd.1) 125 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:35:23.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:21.474953+0000 osd.1 (osd.1) 126 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:21.741282+0000 mon.b (mon.1) 300 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:21.848618+0000 mgr.y (mgr.24454) 271 : cluster [DBG] pgmap v352: 386 pgs: 25 creating+peering, 2 peering, 2 active+clean+snaptrim, 15 unknown, 6 active+clean+snaptrim_wait, 336 active+clean; 9.4 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:22.239449+0000 osd.4 (osd.4) 149 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:22.242400+0000 osd.4 (osd.4) 150 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:22.619264+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e244: 8 total, 8 up, 7 in 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:22.631501+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:22.637276+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:22.742364+0000 mon.b (mon.1) 301 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:21.471933+0000 osd.1 (osd.1) 125 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:21.474953+0000 osd.1 (osd.1) 126 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:35:23.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:21.741282+0000 mon.b (mon.1) 300 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:23.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:21.848618+0000 mgr.y (mgr.24454) 271 : cluster [DBG] pgmap v352: 386 pgs: 25 creating+peering, 2 peering, 2 active+clean+snaptrim, 15 unknown, 6 active+clean+snaptrim_wait, 336 active+clean; 9.4 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T00:35:23.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:22.239449+0000 osd.4 (osd.4) 149 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:35:23.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:22.242400+0000 osd.4 (osd.4) 150 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:35:23.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:22.619264+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e244: 8 total, 8 up, 7 in 2024-02-18T00:35:23.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:22.631501+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:23.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:22.637276+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:23.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:22.742364+0000 mon.b (mon.1) 301 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:23.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:23.241 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:23.314 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:23.317 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:23.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:23.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:23.450 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:23.471 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:23.475 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:23.475 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:21.471933+0000 osd.1 (osd.1) 125 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:21.474953+0000 osd.1 (osd.1) 126 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:21.741282+0000 mon.b (mon.1) 300 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:21.848618+0000 mgr.y (mgr.24454) 271 : cluster [DBG] pgmap v352: 386 pgs: 25 creating+peering, 2 peering, 2 active+clean+snaptrim, 15 unknown, 6 active+clean+snaptrim_wait, 336 active+clean; 9.4 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:22.239449+0000 osd.4 (osd.4) 149 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:22.242400+0000 osd.4 (osd.4) 150 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:22.619264+0000 mon.a (mon.0) 1710 : cluster [DBG] osdmap e244: 8 total, 8 up, 7 in 2024-02-18T00:35:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:22.631501+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:22.637276+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:22.742364+0000 mon.b (mon.1) 301 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:23.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:23.605 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 -- 172.21.15.59:0/2673915960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c0f72d0 msgr2=0x7f506c104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 --2- 172.21.15.59:0/2673915960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c0f72d0 0x7f506c104070 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 rx=0x7f50600048a0 tx=0x7f5060009cb0).stop 2024-02-18T00:35:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 -- 172.21.15.59:0/2673915960 shutdown_connections 2024-02-18T00:35:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 --2- 172.21.15.59:0/2673915960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c104560 0x7f506c106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 --2- 172.21.15.59:0/2673915960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 0x7f506c0f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 --2- 172.21.15.59:0/2673915960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c0f72d0 0x7f506c104070 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 -- 172.21.15.59:0/2673915960 >> 172.21.15.59:0/2673915960 conn(0x7f506c0f26e0 msgr2=0x7f506c0f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 -- 172.21.15.59:0/2673915960 shutdown_connections 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 -- 172.21.15.59:0/2673915960 wait complete. 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 librados: starting msgr at 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.793+0000 7f5070a00700 1 librados: starting objecter 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 Processor -- start 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 -- start start 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 librados: setting wanted keys 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 librados: calling monclient init 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 0x7f506c110390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c0f72d0 0x7f506c110880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c104560 0x7f506c10a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c10aa20 con 0x7f506c0f6a00 2024-02-18T00:35:23.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c10ab90 con 0x7f506c0f72d0 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5070a00700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c10ad50 con 0x7f506c104560 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f506a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 0x7f506c110390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f506a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 0x7f506c110390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59352/0 (socket says 172.21.15.59:59352) 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f506a59c700 1 -- 172.21.15.59:0/2076683132 learned_addr learned my addr 172.21.15.59:0/2076683132 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f506ad9d700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c104560 0x7f506c10a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.794+0000 7f5069d9b700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c0f72d0 0x7f506c110880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f506ad9d700 1 -- 172.21.15.59:0/2076683132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c0f72d0 msgr2=0x7f506c110880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f506ad9d700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c0f72d0 0x7f506c110880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f506ad9d700 1 -- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 msgr2=0x7f506c110390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f506ad9d700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 0x7f506c110390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f506ad9d700 1 -- 172.21.15.59:0/2076683132 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c005040 con 0x7f506c104560 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f506a59c700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 0x7f506c110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f506ad9d700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c104560 0x7f506c10a530 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 rx=0x7f506400d8a0 tx=0x7f5064004db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:23.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f505b7fe700 10 client.?.objecter ms_handle_connect 0x7f506c104560 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f505b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f505b7fe700 1 -- 172.21.15.59:0/2076683132 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064004020 con 0x7f506c104560 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5060005040 con 0x7f506c104560 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f505b7fe700 1 -- 172.21.15.59:0/2076683132 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f506400f040 con 0x7f506c104560 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f505b7fe700 1 -- 172.21.15.59:0/2076683132 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50640036c0 con 0x7f506c104560 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f5070a00700 10 client.40664.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c1171c0 con 0x7f506c104560 2024-02-18T00:35:23.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.795+0000 7f5070a00700 1 librados: init done 2024-02-18T00:35:23.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.796+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c05f950 con 0x7f506c104560 2024-02-18T00:35:23.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f505b7fe700 1 -- 172.21.15.59:0/2076683132 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5064003af0 con 0x7f506c104560 2024-02-18T00:35:23.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f505b7fe700 1 --2- 172.21.15.59:0/2076683132 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5054062c90 0x7f5054065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:23.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f505b7fe700 1 -- 172.21.15.59:0/2076683132 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(245..245 src has 44..245) v4 ==== 9694+0+0 (secure 0 0 0) 0x7f506407d1e0 con 0x7f506c104560 2024-02-18T00:35:23.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f505b7fe700 10 client.40664.objecter ms_dispatch 0x7f506c001550 osd_map(245..245 src has 44..245) v4 2024-02-18T00:35:23.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f505b7fe700 3 client.40664.objecter handle_osd_map got epochs [245,245] > 0 2024-02-18T00:35:23.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f505b7fe700 3 client.40664.objecter handle_osd_map decoding full epoch 245 2024-02-18T00:35:23.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f506a59c700 1 --2- 172.21.15.59:0/2076683132 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5054062c90 0x7f5054065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:23.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.797+0000 7f505b7fe700 20 client.40664.objecter dump_active .. 0 homeless 2024-02-18T00:35:23.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.798+0000 7f505b7fe700 10 client.40664.objecter ms_handle_connect 0x7f5054062c90 2024-02-18T00:35:23.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.798+0000 7f506a59c700 1 --2- 172.21.15.59:0/2076683132 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5054062c90 0x7f5054065150 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 rx=0x7f505c009810 tx=0x7f505c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:23.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.802+0000 7f505b7fe700 1 -- 172.21.15.59:0/2076683132 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5064048f60 con 0x7f506c104560 2024-02-18T00:35:23.941 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:35:23.941 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:35:23.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:23.987+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7f506c05ee80 con 0x7f506c104560 2024-02-18T00:35:24.097 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:22.454981+0000 osd.1 (osd.1) 127 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:22.455937+0000 osd.1 (osd.1) 128 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:23.468748+0000 osd.2 (osd.2) 109 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:23.469657+0000 osd.2 (osd.2) 110 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:23.611249+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:23.611405+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e245: 8 total, 8 up, 7 in 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:23.625969+0000 mon.b (mon.1) 302 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:23.636913+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:23.743563+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:24.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:23.989119+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.59:0/2076683132' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:35:24.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:23.989419+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:35:24.154 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:24.179 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:24.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:24.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:24.286 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:24.319 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:24.339 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:35:24.340 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:24.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:24.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:22.454981+0000 osd.1 (osd.1) 127 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:22.455937+0000 osd.1 (osd.1) 128 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:23.468748+0000 osd.2 (osd.2) 109 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:23.469657+0000 osd.2 (osd.2) 110 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:23.611249+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:23.611405+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e245: 8 total, 8 up, 7 in 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:23.625969+0000 mon.b (mon.1) 302 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:23.636913+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:23.743563+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:24.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:23.989119+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.59:0/2076683132' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:35:24.367 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:23.989419+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:35:24.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:22.454981+0000 osd.1 (osd.1) 127 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:22.455937+0000 osd.1 (osd.1) 128 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:23.468748+0000 osd.2 (osd.2) 109 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:23.469657+0000 osd.2 (osd.2) 110 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:23.611249+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:23.611405+0000 mon.a (mon.0) 1713 : cluster [DBG] osdmap e245: 8 total, 8 up, 7 in 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:23.625969+0000 mon.b (mon.1) 302 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:23.636913+0000 mon.a (mon.0) 1714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:23.743563+0000 mon.b (mon.1) 303 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:23.989119+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.59:0/2076683132' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:35:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:23.989419+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:35:24.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.628+0000 7f505b7fe700 1 -- 172.21.15.59:0/2076683132 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v246) v1 ==== 113+0+0 (secure 0 0 0) 0x7f50640500d0 con 0x7f506c104560 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.632+0000 7f5070a00700 10 librados: watch_flush enter 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 10 librados: watch_flush exit 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 20 client.40664.objecter shutdown clearing up homeless session... 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 10 client.40664.objecter successfully canceled tick 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5054062c90 msgr2=0x7f5054065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 --2- 172.21.15.59:0/2076683132 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5054062c90 0x7f5054065150 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 rx=0x7f505c009810 tx=0x7f505c00b040).stop 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c104560 msgr2=0x7f506c10a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c104560 0x7f506c10a530 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 rx=0x7f506400d8a0 tx=0x7f5064004db0).stop 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 shutdown_connections 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f506c104560 0x7f506c10a530 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f506c0f6a00 0x7f506c110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:24.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 --2- 172.21.15.59:0/2076683132 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5054062c90 0x7f5054065150 unknown :-1 s=CLOSED pgs=3026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 --2- 172.21.15.59:0/2076683132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c0f72d0 0x7f506c110880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.633+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 >> 172.21.15.59:0/2076683132 conn(0x7f506c0f26e0 msgr2=0x7f506c0f4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.634+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 shutdown_connections 2024-02-18T00:35:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.634+0000 7f5070a00700 1 -- 172.21.15.59:0/2076683132 wait complete. 2024-02-18T00:35:24.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:24.634+0000 7f5070a00700 1 librados: shutdown 2024-02-18T00:35:24.637 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.6 to 1 (10000) 2024-02-18T00:35:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:23.481133+0000 osd.6 (osd.6) 129 : cluster [DBG] 62.17 deep-scrub starts 2024-02-18T00:35:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:23.483926+0000 osd.6 (osd.6) 130 : cluster [DBG] 62.17 deep-scrub ok 2024-02-18T00:35:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:23.849073+0000 mgr.y (mgr.24454) 272 : cluster [DBG] pgmap v355: 345 pgs: 1 peering, 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 304 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.621285+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.621385+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:35:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:24.621440+0000 mon.a (mon.0) 1718 : cluster [DBG] osdmap e246: 8 total, 8 up, 7 in 2024-02-18T00:35:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.628560+0000 mon.b (mon.1) 304 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.629036+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.644588+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.645409+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.744459+0000 mon.b (mon.1) 305 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:24.894156+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:23.481133+0000 osd.6 (osd.6) 129 : cluster [DBG] 62.17 deep-scrub starts 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:23.483926+0000 osd.6 (osd.6) 130 : cluster [DBG] 62.17 deep-scrub ok 2024-02-18T00:35:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:23.849073+0000 mgr.y (mgr.24454) 272 : cluster [DBG] pgmap v355: 345 pgs: 1 peering, 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 304 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.621285+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.621385+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:24.621440+0000 mon.a (mon.0) 1718 : cluster [DBG] osdmap e246: 8 total, 8 up, 7 in 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.628560+0000 mon.b (mon.1) 304 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.629036+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.644588+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.645409+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.744459+0000 mon.b (mon.1) 305 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:24.894156+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:23.481133+0000 osd.6 (osd.6) 129 : cluster [DBG] 62.17 deep-scrub starts 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:23.483926+0000 osd.6 (osd.6) 130 : cluster [DBG] 62.17 deep-scrub ok 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:23.849073+0000 mgr.y (mgr.24454) 272 : cluster [DBG] pgmap v355: 345 pgs: 1 peering, 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 304 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.621285+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.621385+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:24.621440+0000 mon.a (mon.0) 1718 : cluster [DBG] osdmap e246: 8 total, 8 up, 7 in 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.628560+0000 mon.b (mon.1) 304 : audit [INF] from='client.? 172.21.15.59:0/2835339393' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.629036+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]: dispatch 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.644588+0000 mon.c (mon.2) 207 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.645409+0000 mon.a (mon.0) 1720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.744459+0000 mon.b (mon.1) 305 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:24.894156+0000 mon.a (mon.0) 1721 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:26.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:26.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:26.232 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:26.264 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:26.285 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:26.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:26.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:26.389 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:26.418 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:26.438 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:26.440 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:26.441 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:26.541 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:26.572 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:24.989708+0000 osd.0 (osd.0) 121 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:35:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:24.990574+0000 osd.0 (osd.0) 122 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:35:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:25.476013+0000 osd.2 (osd.2) 111 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:35:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:25.477604+0000 osd.2 (osd.2) 112 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:35:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.620038+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.620165+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:25.620234+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e247: 8 total, 8 up, 7 in 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.624085+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.624610+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.648916+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.649760+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.650190+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi059-82877-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:25.745471+0000 mon.b (mon.1) 306 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:25.850194+0000 mgr.y (mgr.24454) 273 : cluster [DBG] pgmap v358: 281 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 273 active+clean; 9.3 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:25.952333+0000 osd.0 (osd.0) 123 : cluster [DBG] 26.9 scrub starts 2024-02-18T00:35:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:25.953696+0000 osd.0 (osd.0) 124 : cluster [DBG] 26.9 scrub ok 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:24.989708+0000 osd.0 (osd.0) 121 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:24.990574+0000 osd.0 (osd.0) 122 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:25.476013+0000 osd.2 (osd.2) 111 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:25.477604+0000 osd.2 (osd.2) 112 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.620038+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.620165+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:25.620234+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e247: 8 total, 8 up, 7 in 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.624085+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.624610+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.648916+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:27.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.649760+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:27.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.650190+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi059-82877-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:27.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:25.745471+0000 mon.b (mon.1) 306 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:27.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:25.850194+0000 mgr.y (mgr.24454) 273 : cluster [DBG] pgmap v358: 281 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 273 active+clean; 9.3 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:35:27.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:25.952333+0000 osd.0 (osd.0) 123 : cluster [DBG] 26.9 scrub starts 2024-02-18T00:35:27.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:25.953696+0000 osd.0 (osd.0) 124 : cluster [DBG] 26.9 scrub ok 2024-02-18T00:35:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:24.989708+0000 osd.0 (osd.0) 121 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:35:27.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:24.990574+0000 osd.0 (osd.0) 122 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:25.476013+0000 osd.2 (osd.2) 111 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:25.477604+0000 osd.2 (osd.2) 112 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.620038+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi059-82877-34"}]': finished 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.620165+0000 mon.a (mon.0) 1723 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:25.620234+0000 mon.a (mon.0) 1724 : cluster [DBG] osdmap e247: 8 total, 8 up, 7 in 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.624085+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.624610+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.648916+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.649760+0000 mon.a (mon.0) 1727 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.650190+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi059-82877-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:25.745471+0000 mon.b (mon.1) 306 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:25.850194+0000 mgr.y (mgr.24454) 273 : cluster [DBG] pgmap v358: 281 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 273 active+clean; 9.3 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:35:27.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:25.952333+0000 osd.0 (osd.0) 123 : cluster [DBG] 26.9 scrub starts 2024-02-18T00:35:27.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:25.953696+0000 osd.0 (osd.0) 124 : cluster [DBG] 26.9 scrub ok 2024-02-18T00:35:27.134 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-18T00:35:27.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:26.536605+0000 osd.3 (osd.3) 147 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:26.537456+0000 osd.3 (osd.3) 148 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:26.556397+0000 osd.1 (osd.1) 129 : cluster [DBG] 26.3 scrub starts 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:26.559924+0000 osd.1 (osd.1) 130 : cluster [DBG] 26.3 scrub ok 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:26.625978+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]': finished 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:26.626097+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi059-82877-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:26.626174+0000 mon.a (mon.0) 1731 : cluster [DBG] osdmap e248: 8 total, 8 up, 7 in 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:26.627693+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi059-82877-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:26.628725+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]: dispatch 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:26.631505+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]: dispatch 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:26.746754+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:26.964963+0000 osd.0 (osd.0) 125 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:26.966334+0000 osd.0 (osd.0) 126 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:26.536605+0000 osd.3 (osd.3) 147 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:26.537456+0000 osd.3 (osd.3) 148 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:26.556397+0000 osd.1 (osd.1) 129 : cluster [DBG] 26.3 scrub starts 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:26.559924+0000 osd.1 (osd.1) 130 : cluster [DBG] 26.3 scrub ok 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:26.625978+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]': finished 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:26.626097+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi059-82877-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:28.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:26.626174+0000 mon.a (mon.0) 1731 : cluster [DBG] osdmap e248: 8 total, 8 up, 7 in 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:26.627693+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi059-82877-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:26.628725+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]: dispatch 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:26.631505+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]: dispatch 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:26.746754+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:26.964963+0000 osd.0 (osd.0) 125 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:26.966334+0000 osd.0 (osd.0) 126 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:26.536605+0000 osd.3 (osd.3) 147 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:26.537456+0000 osd.3 (osd.3) 148 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:35:28.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:26.556397+0000 osd.1 (osd.1) 129 : cluster [DBG] 26.3 scrub starts 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:26.559924+0000 osd.1 (osd.1) 130 : cluster [DBG] 26.3 scrub ok 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:26.625978+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-25"}]': finished 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:26.626097+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi059-82877-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:26.626174+0000 mon.a (mon.0) 1731 : cluster [DBG] osdmap e248: 8 total, 8 up, 7 in 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:26.627693+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi059-82877-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:26.628725+0000 mon.c (mon.2) 209 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]: dispatch 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:26.631505+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]: dispatch 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:26.746754+0000 mon.b (mon.1) 307 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:26.964963+0000 osd.0 (osd.0) 125 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:35:28.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:26.966334+0000 osd.0 (osd.0) 126 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:35:28.127 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:28.130 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:28.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:28.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-18T00:35:28.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:28.350 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:28.355 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:28.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:28.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:28.484 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:28.734 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:28.735 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:27.230986+0000 osd.4 (osd.4) 151 : cluster [DBG] 4.d scrub starts 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:27.234457+0000 osd.4 (osd.4) 152 : cluster [DBG] 4.d scrub ok 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:27.519465+0000 osd.3 (osd.3) 149 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:27.520356+0000 osd.3 (osd.3) 150 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:27.622510+0000 mon.a (mon.0) 1734 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:27.748542+0000 mon.b (mon.1) 308 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:27.850592+0000 mgr.y (mgr.24454) 274 : cluster [DBG] pgmap v361: 281 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 241 active+clean; 9.2 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s; 63 B/s, 0 objects/s recovering 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.066218+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]': finished 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.066321+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e249: 8 total, 8 up, 7 in 2024-02-18T00:35:28.741 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.067439+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.59:0/2005742270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.069073+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.086088+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.086370+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:27.230986+0000 osd.4 (osd.4) 151 : cluster [DBG] 4.d scrub starts 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:27.234457+0000 osd.4 (osd.4) 152 : cluster [DBG] 4.d scrub ok 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:27.519465+0000 osd.3 (osd.3) 149 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:27.520356+0000 osd.3 (osd.3) 150 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:27.622510+0000 mon.a (mon.0) 1734 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:28.742 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:27.748542+0000 mon.b (mon.1) 308 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:28.743 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:27.850592+0000 mgr.y (mgr.24454) 274 : cluster [DBG] pgmap v361: 281 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 241 active+clean; 9.2 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s; 63 B/s, 0 objects/s recovering 2024-02-18T00:35:28.743 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.066218+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]': finished 2024-02-18T00:35:28.743 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.066321+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e249: 8 total, 8 up, 7 in 2024-02-18T00:35:28.743 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.067439+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.59:0/2005742270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:28.743 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.069073+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:28.743 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.086088+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:28.743 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.086370+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:28.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:28.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:28.845 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:28.872 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:28.891 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:28.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:27.230986+0000 osd.4 (osd.4) 151 : cluster [DBG] 4.d scrub starts 2024-02-18T00:35:28.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:27.234457+0000 osd.4 (osd.4) 152 : cluster [DBG] 4.d scrub ok 2024-02-18T00:35:28.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:27.519465+0000 osd.3 (osd.3) 149 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:35:28.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:27.520356+0000 osd.3 (osd.3) 150 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:35:28.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:27.622510+0000 mon.a (mon.0) 1734 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:28.893 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:27.748542+0000 mon.b (mon.1) 308 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:28.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:27.850592+0000 mgr.y (mgr.24454) 274 : cluster [DBG] pgmap v361: 281 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 241 active+clean; 9.2 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s; 63 B/s, 0 objects/s recovering 2024-02-18T00:35:28.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.066218+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-25", "mode": "writeback"}]': finished 2024-02-18T00:35:28.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.066321+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e249: 8 total, 8 up, 7 in 2024-02-18T00:35:28.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.067439+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.59:0/2005742270' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:28.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.069073+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:28.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.086088+0000 mon.c (mon.2) 211 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:28.894 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.086370+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:28.895 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:28.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:28.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:29.024 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:29.049 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:29.052 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:29.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:29.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:29.183 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:29.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.374659+0000 osd.6 (osd.6) 131 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:35:29.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.376114+0000 osd.6 (osd.6) 132 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:35:29.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.512022+0000 osd.1 (osd.1) 131 : cluster [DBG] 26.5 deep-scrub starts 2024-02-18T00:35:29.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.514978+0000 osd.1 (osd.1) 132 : cluster [DBG] 26.5 deep-scrub ok 2024-02-18T00:35:29.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.640722+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi059-82877-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:29.664 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.640921+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:29.665 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.640990+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:29.665 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.641050+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e250: 8 total, 8 up, 7 in 2024-02-18T00:35:29.665 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.644935+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:29.665 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.645300+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:29.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 -- 172.21.15.59:0/860898342 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff194102210 msgr2=0x7ff194104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:29.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/860898342 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff194102210 0x7ff194104600 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 rx=0x7ff18c009be0 tx=0x7ff18c004d10).stop 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 -- 172.21.15.59:0/860898342 shutdown_connections 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/860898342 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff194102210 0x7ff194104600 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/860898342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1940f8cb0 0x7ff1940f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/860898342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1940f9610 0x7ff194101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 -- 172.21.15.59:0/860898342 >> 172.21.15.59:0/860898342 conn(0x7ff1940f2820 msgr2=0x7ff1940f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 -- 172.21.15.59:0/860898342 shutdown_connections 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 -- 172.21.15.59:0/860898342 wait complete. 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 librados: starting msgr at 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.764+0000 7ff19b8b9700 1 librados: starting objecter 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 Processor -- start 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 -- start start 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 librados: setting wanted keys 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 librados: calling monclient init 2024-02-18T00:35:29.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1940f8cb0 0x7ff19410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1940f9610 0x7ff19410e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102210 0x7ff194108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff194108510 con 0x7ff1940f9610 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff194108680 con 0x7ff194102210 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff19b8b9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff194108840 con 0x7ff1940f8cb0 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1940f8cb0 0x7ff19410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1940f8cb0 0x7ff19410de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:46332/0 (socket says 172.21.15.59:46332) 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 -- 172.21.15.59:0/2708429398 learned_addr learned my addr 172.21.15.59:0/2708429398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199e56700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102210 0x7ff194108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff198e54700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1940f9610 0x7ff19410e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 -- 172.21.15.59:0/2708429398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102210 msgr2=0x7ff194108020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102210 0x7ff194108020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 -- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1940f9610 msgr2=0x7ff19410e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:29.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1940f9610 0x7ff19410e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.765+0000 7ff199655700 1 -- 172.21.15.59:0/2708429398 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c0097c0 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff199655700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1940f8cb0 0x7ff19410de80 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 rx=0x7ff19000c7c0 tx=0x7ff190007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff18a7fc700 10 client.?.objecter ms_handle_connect 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff18a7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff18a7fc700 1 -- 172.21.15.59:0/2708429398 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190004430 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff18a7fc700 1 -- 172.21.15.59:0/2708429398 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19000e970 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff194108b20 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff18a7fc700 1 -- 172.21.15.59:0/2708429398 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190013a60 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff19b8b9700 10 client.40685.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff194108ea0 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.766+0000 7ff19b8b9700 1 librados: init done 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.767+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19405fa60 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.767+0000 7ff18a7fc700 1 -- 172.21.15.59:0/2708429398 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff19000e4a0 con 0x7ff1940f8cb0 2024-02-18T00:35:29.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.767+0000 7ff18a7fc700 1 --2- 172.21.15.59:0/2708429398 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff180062c90 0x7ff180065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff18a7fc700 1 -- 172.21.15.59:0/2708429398 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(251..251 src has 44..251) v4 ==== 8973+0+0 (secure 0 0 0) 0x7ff19007d260 con 0x7ff1940f8cb0 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff18a7fc700 10 client.40685.objecter ms_dispatch 0x7ff194001550 osd_map(251..251 src has 44..251) v4 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff18a7fc700 3 client.40685.objecter handle_osd_map got epochs [251,251] > 0 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff18a7fc700 3 client.40685.objecter handle_osd_map decoding full epoch 251 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff198e54700 1 --2- 172.21.15.59:0/2708429398 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff180062c90 0x7ff180065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff18a7fc700 20 client.40685.objecter dump_active .. 0 homeless 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff18a7fc700 10 client.40685.objecter ms_handle_connect 0x7ff180062c90 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.768+0000 7ff198e54700 1 --2- 172.21.15.59:0/2708429398 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff180062c90 0x7ff180065150 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 rx=0x7ff1941097c0 tx=0x7ff18400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:29.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.772+0000 7ff18a7fc700 1 -- 172.21.15.59:0/2708429398 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff1900503c0 con 0x7ff1940f8cb0 2024-02-18T00:35:29.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:29.954+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7ff194109500 con 0x7ff1940f8cb0 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:28.749546+0000 mon.b (mon.1) 309 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.831125+0000 osd.5 (osd.5) 117 : cluster [DBG] 25.f scrub starts 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:28.834034+0000 osd.5 (osd.5) 118 : cluster [DBG] 25.f scrub ok 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.374659+0000 osd.6 (osd.6) 131 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.376114+0000 osd.6 (osd.6) 132 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.512022+0000 osd.1 (osd.1) 131 : cluster [DBG] 26.5 deep-scrub starts 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.514978+0000 osd.1 (osd.1) 132 : cluster [DBG] 26.5 deep-scrub ok 2024-02-18T00:35:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.640722+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi059-82877-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.640921+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.640990+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.641050+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e250: 8 total, 8 up, 7 in 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.644935+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.645300+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:28.749546+0000 mon.b (mon.1) 309 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.831125+0000 osd.5 (osd.5) 117 : cluster [DBG] 25.f scrub starts 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:28.834034+0000 osd.5 (osd.5) 118 : cluster [DBG] 25.f scrub ok 2024-02-18T00:35:30.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.374659+0000 osd.6 (osd.6) 131 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.376114+0000 osd.6 (osd.6) 132 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.512022+0000 osd.1 (osd.1) 131 : cluster [DBG] 26.5 deep-scrub starts 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.514978+0000 osd.1 (osd.1) 132 : cluster [DBG] 26.5 deep-scrub ok 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.640722+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi059-82877-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.640921+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi059-82882-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.640990+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.641050+0000 mon.a (mon.0) 1742 : cluster [DBG] osdmap e250: 8 total, 8 up, 7 in 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.644935+0000 mon.c (mon.2) 212 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.645300+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]: dispatch 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:28.749546+0000 mon.b (mon.1) 309 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.831125+0000 osd.5 (osd.5) 117 : cluster [DBG] 25.f scrub starts 2024-02-18T00:35:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:28.834034+0000 osd.5 (osd.5) 118 : cluster [DBG] 25.f scrub ok 2024-02-18T00:35:30.152 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:30.153 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:30.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:29.517603+0000 osd.1 (osd.1) 133 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:35:30.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:29.520628+0000 osd.1 (osd.1) 134 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:35:30.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:29.536984+0000 osd.3 (osd.3) 151 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:35:30.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:29.537901+0000 osd.3 (osd.3) 152 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:35:30.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:29.637533+0000 mon.a (mon.0) 1744 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:30.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.669+0000 7ff18a7fc700 1 -- 172.21.15.59:0/2708429398 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v252) v1 ==== 87+0+0 (secure 0 0 0) 0x7ff19004ffb0 con 0x7ff1940f8cb0 2024-02-18T00:35:30.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.674+0000 7ff19b8b9700 10 librados: watch_flush enter 2024-02-18T00:35:30.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.674+0000 7ff19b8b9700 10 librados: watch_flush exit 2024-02-18T00:35:30.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.674+0000 7ff19b8b9700 20 client.40685.objecter shutdown clearing up homeless session... 2024-02-18T00:35:30.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.674+0000 7ff19b8b9700 10 client.40685.objecter successfully canceled tick 2024-02-18T00:35:30.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.677+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff180062c90 msgr2=0x7ff180065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:30.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.677+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/2708429398 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff180062c90 0x7ff180065150 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 rx=0x7ff1941097c0 tx=0x7ff18400b040).stop 2024-02-18T00:35:30.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.677+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1940f8cb0 msgr2=0x7ff19410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:30.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.677+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1940f8cb0 0x7ff19410de80 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 rx=0x7ff19000c7c0 tx=0x7ff190007ef0).stop 2024-02-18T00:35:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.679+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 shutdown_connections 2024-02-18T00:35:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.680+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/2708429398 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff180062c90 0x7ff180065150 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.680+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102210 0x7ff194108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.680+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1940f8cb0 0x7ff19410de80 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.680+0000 7ff19b8b9700 1 --2- 172.21.15.59:0/2708429398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1940f9610 0x7ff19410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:30.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.680+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 >> 172.21.15.59:0/2708429398 conn(0x7ff1940f2820 msgr2=0x7ff1940f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:30.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.686+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 shutdown_connections 2024-02-18T00:35:30.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.686+0000 7ff19b8b9700 1 -- 172.21.15.59:0/2708429398 wait complete. 2024-02-18T00:35:30.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:30.686+0000 7ff19b8b9700 1 librados: shutdown 2024-02-18T00:35:30.694 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:35:30.732 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:30.737 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:30.737 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:30.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:30.867 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:29.517603+0000 osd.1 (osd.1) 133 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:29.520628+0000 osd.1 (osd.1) 134 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:29.536984+0000 osd.3 (osd.3) 151 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:29.537901+0000 osd.3 (osd.3) 152 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:29.637533+0000 mon.a (mon.0) 1744 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:29.642747+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]': finished 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:29.642913+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e251: 8 total, 8 up, 7 in 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:29.750611+0000 mon.b (mon.1) 310 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:29.851000+0000 mgr.y (mgr.24454) 275 : cluster [DBG] pgmap v364: 257 pgs: 2 active+clean+snaptrim, 8 unknown, 6 active+clean+snaptrim_wait, 241 active+clean; 9.2 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:29.955686+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.59:0/2708429398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:35:31.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:30.124555+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:35:31.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:29.642747+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]': finished 2024-02-18T00:35:31.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:29.642913+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e251: 8 total, 8 up, 7 in 2024-02-18T00:35:31.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:29.750611+0000 mon.b (mon.1) 310 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:31.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:29.851000+0000 mgr.y (mgr.24454) 275 : cluster [DBG] pgmap v364: 257 pgs: 2 active+clean+snaptrim, 8 unknown, 6 active+clean+snaptrim_wait, 241 active+clean; 9.2 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:31.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:29.955686+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.59:0/2708429398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:35:31.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:30.124555+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:29.517603+0000 osd.1 (osd.1) 133 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:29.520628+0000 osd.1 (osd.1) 134 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:29.536984+0000 osd.3 (osd.3) 151 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:29.537901+0000 osd.3 (osd.3) 152 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:29.637533+0000 mon.a (mon.0) 1744 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:29.642747+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-25"}]': finished 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:29.642913+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e251: 8 total, 8 up, 7 in 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:29.750611+0000 mon.b (mon.1) 310 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:29.851000+0000 mgr.y (mgr.24454) 275 : cluster [DBG] pgmap v364: 257 pgs: 2 active+clean+snaptrim, 8 unknown, 6 active+clean+snaptrim_wait, 241 active+clean; 9.2 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:29.955686+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.59:0/2708429398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:35:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:30.124555+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:35:31.067 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:31.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:31.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:31.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:31.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:35:31.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:35:31.553 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:31.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:31.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:31.657 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:31.721 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:31.724 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:31.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:31.825 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:31.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:31.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:31.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.235383+0000 osd.4 (osd.4) 153 : cluster [DBG] 4.18 deep-scrub starts 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.243223+0000 osd.4 (osd.4) 154 : cluster [DBG] 4.18 deep-scrub ok 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.341716+0000 osd.6 (osd.6) 133 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.343435+0000 osd.6 (osd.6) 134 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.473983+0000 osd.1 (osd.1) 135 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.476941+0000 osd.1 (osd.1) 136 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.656727+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:30.666907+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.666981+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e252: 8 total, 8 up, 7 in 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:30.683248+0000 mon.b (mon.1) 311 : audit [INF] from='client.? 172.21.15.59:0/297789645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:30.751676+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.833036+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:30.833165+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.076184+0000 mgr.y (mgr.24454) 276 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.128147+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.128453+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.235383+0000 osd.4 (osd.4) 153 : cluster [DBG] 4.18 deep-scrub starts 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.243223+0000 osd.4 (osd.4) 154 : cluster [DBG] 4.18 deep-scrub ok 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.341716+0000 osd.6 (osd.6) 133 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.343435+0000 osd.6 (osd.6) 134 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.473983+0000 osd.1 (osd.1) 135 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.476941+0000 osd.1 (osd.1) 136 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:35:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.656727+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:30.666907+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.666981+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e252: 8 total, 8 up, 7 in 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:30.683248+0000 mon.b (mon.1) 311 : audit [INF] from='client.? 172.21.15.59:0/297789645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:30.751676+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.833036+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:30.833165+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.076184+0000 mgr.y (mgr.24454) 276 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.128147+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:32.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.128453+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.235383+0000 osd.4 (osd.4) 153 : cluster [DBG] 4.18 deep-scrub starts 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.243223+0000 osd.4 (osd.4) 154 : cluster [DBG] 4.18 deep-scrub ok 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.341716+0000 osd.6 (osd.6) 133 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.343435+0000 osd.6 (osd.6) 134 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.473983+0000 osd.1 (osd.1) 135 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.476941+0000 osd.1 (osd.1) 136 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.656727+0000 mon.a (mon.0) 1748 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:30.666907+0000 mon.a (mon.0) 1749 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.666981+0000 mon.a (mon.0) 1750 : cluster [DBG] osdmap e252: 8 total, 8 up, 7 in 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:30.683248+0000 mon.b (mon.1) 311 : audit [INF] from='client.? 172.21.15.59:0/297789645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:30.751676+0000 mon.b (mon.1) 312 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.833036+0000 osd.5 (osd.5) 119 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:32.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:30.833165+0000 osd.5 (osd.5) 120 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:32.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.076184+0000 mgr.y (mgr.24454) 276 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:32.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.128147+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:32.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.128453+0000 mon.a (mon.0) 1752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:32.008 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:32.503 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:32] "GET /metrics HTTP/1.1" 200 195435 "" "Prometheus/2.33.4" 2024-02-18T00:35:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.672740+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.672849+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:31.672930+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e253: 8 total, 8 up, 7 in 2024-02-18T00:35:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.674982+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.687358+0000 mon.c (mon.2) 214 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.688766+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:31.758327+0000 mon.b (mon.1) 313 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:31.851483+0000 mgr.y (mgr.24454) 277 : cluster [DBG] pgmap v367: 281 pgs: 32 creating+peering, 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.672740+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.672849+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:31.672930+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e253: 8 total, 8 up, 7 in 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.674982+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.687358+0000 mon.c (mon.2) 214 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:33.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.688766+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:33.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:31.758327+0000 mon.b (mon.1) 313 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:33.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:31.851483+0000 mgr.y (mgr.24454) 277 : cluster [DBG] pgmap v367: 281 pgs: 32 creating+peering, 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.672740+0000 mon.a (mon.0) 1753 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.672849+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi059-82882-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:31.672930+0000 mon.a (mon.0) 1755 : cluster [DBG] osdmap e253: 8 total, 8 up, 7 in 2024-02-18T00:35:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.674982+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]: dispatch 2024-02-18T00:35:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.687358+0000 mon.c (mon.2) 214 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.688766+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:31.758327+0000 mon.b (mon.1) 313 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:33.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:31.851483+0000 mgr.y (mgr.24454) 277 : cluster [DBG] pgmap v367: 281 pgs: 32 creating+peering, 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:33.058 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:33.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:33.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:33.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:33.194 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:33.746 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:33.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:33.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 -- 172.21.15.59:0/676531016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fcbb0 msgr2=0x7fcba80fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 --2- 172.21.15.59:0/676531016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fcbb0 0x7fcba80fcf90 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 rx=0x7fcb98004870 tx=0x7fcb98009cb0).stop 2024-02-18T00:35:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 -- 172.21.15.59:0/676531016 shutdown_connections 2024-02-18T00:35:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 --2- 172.21.15.59:0/676531016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcba80f7400 0x7fcba8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 --2- 172.21.15.59:0/676531016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fcbb0 0x7fcba80fcf90 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 --2- 172.21.15.59:0/676531016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcba80fd510 0x7fcba80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 -- 172.21.15.59:0/676531016 >> 172.21.15.59:0/676531016 conn(0x7fcba80f2820 msgr2=0x7fcba80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 -- 172.21.15.59:0/676531016 shutdown_connections 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 -- 172.21.15.59:0/676531016 wait complete. 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 librados: starting msgr at 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 librados: starting objecter 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 Processor -- start 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 -- start start 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 librados: setting wanted keys 2024-02-18T00:35:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.851+0000 7fcbae358700 1 librados: calling monclient init 2024-02-18T00:35:33.854 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbae358700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcba80f7400 0x7fcba80f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbae358700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcba80fcbb0 0x7fcba80f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbae358700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fd510 0x7fcba80f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbae358700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8116710 con 0x7fcba80fd510 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbae358700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8116880 con 0x7fcba80fcbb0 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbae358700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8116a40 con 0x7fcba80f7400 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fd510 0x7fcba80f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fd510 0x7fcba80f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39582/0 (socket says 172.21.15.59:39582) 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 -- 172.21.15.59:0/1639732524 learned_addr learned my addr 172.21.15.59:0/1639732524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcba77fe700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcba80fcbb0 0x7fcba80f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcba7fff700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcba80f7400 0x7fcba80f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 -- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcba80f7400 msgr2=0x7fcba80f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcba80f7400 0x7fcba80f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 -- 172.21.15.59:0/1639732524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcba80fcbb0 msgr2=0x7fcba80f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcba80fcbb0 0x7fcba80f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 -- 172.21.15.59:0/1639732524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00097c0 con 0x7fcba80fd510 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcbac8f5700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fd510 0x7fcba80f9390 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 rx=0x7fcba0001df0 tx=0x7fcba0004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcba57fa700 10 client.?.objecter ms_handle_connect 0x7fcba80fd510 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcba57fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.852+0000 7fcba57fa700 1 -- 172.21.15.59:0/1639732524 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0009a10 con 0x7fcba80fd510 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.853+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb98005040 con 0x7fcba80fd510 2024-02-18T00:35:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.853+0000 7fcba57fa700 1 -- 172.21.15.59:0/1639732524 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba00095a0 con 0x7fcba80fd510 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.853+0000 7fcba57fa700 1 -- 172.21.15.59:0/1639732524 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0007da0 con 0x7fcba80fd510 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.853+0000 7fcbae358700 10 client.30882.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.853+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba8117080 con 0x7fcba80fd510 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.853+0000 7fcbae358700 1 librados: init done 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba805fa60 con 0x7fcba80fd510 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcba57fa700 1 -- 172.21.15.59:0/1639732524 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fcba00090d0 con 0x7fcba80fd510 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcba57fa700 1 --2- 172.21.15.59:0/1639732524 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb90062c90 0x7fcb90065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcba57fa700 1 -- 172.21.15.59:0/1639732524 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(255..255 src has 44..255) v4 ==== 8756+0+0 (secure 0 0 0) 0x7fcba007dea0 con 0x7fcba80fd510 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcba57fa700 10 client.30882.objecter ms_dispatch 0x7fcba8001550 osd_map(255..255 src has 44..255) v4 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcba57fa700 3 client.30882.objecter handle_osd_map got epochs [255,255] > 0 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcba57fa700 3 client.30882.objecter handle_osd_map decoding full epoch 255 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.854+0000 7fcba7fff700 1 --2- 172.21.15.59:0/1639732524 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb90062c90 0x7fcb90065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.855+0000 7fcba57fa700 20 client.30882.objecter dump_active .. 0 homeless 2024-02-18T00:35:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.855+0000 7fcba57fa700 10 client.30882.objecter ms_handle_connect 0x7fcb90062c90 2024-02-18T00:35:33.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.855+0000 7fcba7fff700 1 --2- 172.21.15.59:0/1639732524 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb90062c90 0x7fcb90065150 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 rx=0x7fcb98004ee0 tx=0x7fcb9800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:33.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:33.859+0000 7fcba57fa700 1 -- 172.21.15.59:0/1639732524 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcba00510b0 con 0x7fcba80fd510 2024-02-18T00:35:33.891 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:33.912 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:33.915 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:33.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.680526+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.680741+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:32.680867+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e254: 8 total, 8 up, 7 in 2024-02-18T00:35:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.733538+0000 mon.b (mon.1) 314 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.734025+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.739061+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.739618+0000 mon.b (mon.1) 315 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.740206+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.741940+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.742448+0000 mon.b (mon.1) 316 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.742478+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.742810+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.680526+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.680741+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:32.680867+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e254: 8 total, 8 up, 7 in 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.733538+0000 mon.b (mon.1) 314 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.734025+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.739061+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.739618+0000 mon.b (mon.1) 315 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.740206+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.741940+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.742448+0000 mon.b (mon.1) 316 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.742478+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.742810+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:32.759359+0000 mon.b (mon.1) 317 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.616780+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.616878+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:33.617156+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e255: 8 total, 8 up, 7 in 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.618280+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.619904+0000 mon.b (mon.1) 318 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.622740+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.635417+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.635776+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:32.759359+0000 mon.b (mon.1) 317 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.616780+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.616878+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:33.617156+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e255: 8 total, 8 up, 7 in 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.618280+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.619904+0000 mon.b (mon.1) 318 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.622740+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.635417+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:34.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.635776+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:34.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.680526+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? 172.21.15.59:0/2990802299' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi059-82877-35"}]': finished 2024-02-18T00:35:34.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.680741+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:34.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:32.680867+0000 mon.a (mon.0) 1760 : cluster [DBG] osdmap e254: 8 total, 8 up, 7 in 2024-02-18T00:35:34.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.733538+0000 mon.b (mon.1) 314 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.734025+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.739061+0000 mon.a (mon.0) 1762 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.739618+0000 mon.b (mon.1) 315 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.740206+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.741940+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.742448+0000 mon.b (mon.1) 316 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.742478+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.742810+0000 mon.a (mon.0) 1766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:32.759359+0000 mon.b (mon.1) 317 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.616780+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.616878+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:33.617156+0000 mon.a (mon.0) 1769 : cluster [DBG] osdmap e255: 8 total, 8 up, 7 in 2024-02-18T00:35:34.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.618280+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:34.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.619904+0000 mon.b (mon.1) 318 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.622740+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:34.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.635417+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:34.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.635776+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:34.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.029+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcba80f9e90 con 0x7fcb90062c90 2024-02-18T00:35:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.046+0000 7fcba57fa700 1 -- 172.21.15.59:0/1639732524 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+623955 (secure 0 0 0) 0x7fcba80f9e90 con 0x7fcb90062c90 2024-02-18T00:35:34.057 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 10 librados: watch_flush enter 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 10 librados: watch_flush exit 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 20 client.30882.objecter shutdown clearing up homeless session... 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 10 client.30882.objecter successfully canceled tick 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb90062c90 msgr2=0x7fcb90065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 --2- 172.21.15.59:0/1639732524 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb90062c90 0x7fcb90065150 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 rx=0x7fcb98004ee0 tx=0x7fcb9800b040).stop 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fd510 msgr2=0x7fcba80f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:34.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fd510 0x7fcba80f9390 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 rx=0x7fcba0001df0 tx=0x7fcba0004c50).stop 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 shutdown_connections 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba80fd510 0x7fcba80f9390 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcba80f7400 0x7fcba80f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 --2- 172.21.15.59:0/1639732524 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb90062c90 0x7fcb90065150 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 --2- 172.21.15.59:0/1639732524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcba80fcbb0 0x7fcba80f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 >> 172.21.15.59:0/1639732524 conn(0x7fcba80f2820 msgr2=0x7fcba80f4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 shutdown_connections 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 -- 172.21.15.59:0/1639732524 wait complete. 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:34.051+0000 7fcbae358700 1 librados: shutdown 2024-02-18T00:35:34.064 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:35:35.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 -- 172.21.15.59:0/1177806600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c01022b0 msgr2=0x7fd6c01046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1177806600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c01022b0 0x7fd6c01046a0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 rx=0x7fd6b8009be0 tx=0x7fd6b8004d10).stop 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 -- 172.21.15.59:0/1177806600 shutdown_connections 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1177806600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c00f8d50 0x7fd6c00f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1177806600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c01022b0 0x7fd6c01046a0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1177806600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c00f9620 0x7fd6c0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 -- 172.21.15.59:0/1177806600 >> 172.21.15.59:0/1177806600 conn(0x7fd6c00f2820 msgr2=0x7fd6c00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 -- 172.21.15.59:0/1177806600 shutdown_connections 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.152+0000 7fd6c6251700 1 -- 172.21.15.59:0/1177806600 wait complete. 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.153+0000 7fd6c6251700 1 librados: starting msgr at 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.153+0000 7fd6c6251700 1 librados: starting objecter 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.153+0000 7fd6c6251700 1 Processor -- start 2024-02-18T00:35:35.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.153+0000 7fd6c6251700 1 -- start start 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.153+0000 7fd6c6251700 1 librados: setting wanted keys 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.153+0000 7fd6c6251700 1 librados: calling monclient init 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6c6251700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c00f8d50 0x7fd6c010dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6c6251700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c00f9620 0x7fd6c010e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6c6251700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01022b0 0x7fd6c0108140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6c6251700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0108630 con 0x7fd6c01022b0 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6c6251700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c01087a0 con 0x7fd6c00f9620 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6c6251700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c0108960 con 0x7fd6c00f8d50 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c00f8d50 0x7fd6c010dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c00f8d50 0x7fd6c010dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:46412/0 (socket says 172.21.15.59:46412) 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 -- 172.21.15.59:0/1451674738 learned_addr learned my addr 172.21.15.59:0/1451674738 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6beffd700 1 --2- 172.21.15.59:0/1451674738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c00f9620 0x7fd6c010e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bffff700 1 --2- 172.21.15.59:0/1451674738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01022b0 0x7fd6c0108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 -- 172.21.15.59:0/1451674738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c00f9620 msgr2=0x7fd6c010e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c00f9620 0x7fd6c010e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 -- 172.21.15.59:0/1451674738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01022b0 msgr2=0x7fd6c0108140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c01022b0 0x7fd6c0108140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 -- 172.21.15.59:0/1451674738 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b80097c0 con 0x7fd6c00f8d50 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c00f8d50 0x7fd6c010dfa0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 rx=0x7fd6b00041d0 tx=0x7fd6b0009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bcff9700 10 client.?.objecter ms_handle_connect 0x7fd6c00f8d50 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.154+0000 7fd6bcff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0007b40 con 0x7fd6c00f8d50 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6b000f040 con 0x7fd6c00f8d50 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0014810 con 0x7fd6c00f8d50 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c0108c40 con 0x7fd6c00f8d50 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6c6251700 10 client.40706.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c0114dd0 con 0x7fd6c00f8d50 2024-02-18T00:35:35.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6c6251700 1 librados: init done 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.155+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd6c005fa60 con 0x7fd6c00f8d50 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.156+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd6b0009110 con 0x7fd6c00f8d50 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd6a8062c90 0x7fd6a8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(256..256 src has 44..256) v4 ==== 8985+0+0 (secure 0 0 0) 0x7fd6b007dd20 con 0x7fd6c00f8d50 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 osd_map(256..256 src has 44..256) v4 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 3 client.40706.objecter handle_osd_map got epochs [256,256] > 0 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 3 client.40706.objecter handle_osd_map decoding full epoch 256 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6beffd700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd6a8062c90 0x7fd6a8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 20 client.40706.objecter dump_active .. 0 homeless 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v256) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd6b007e1c0 con 0x7fd6c00f8d50 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 10 client.40706.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 1 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd6c00fc280 0x7fd6c00fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6beffd700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd6a8062c90 0x7fd6a8065150 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 rx=0x7fd6c01098e0 tx=0x7fd6b400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6a8062c90 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 1 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 0 1 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c005ee80 osd=0 4 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 10 client.40706.objecter _send_command 1 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c0107290 con 0x7fd6c00fc280 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.157+0000 7fd6bffff700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd6c00fc280 0x7fd6c00fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.158+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c00fc280 2024-02-18T00:35:35.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.158+0000 7fd6bffff700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd6c00fc280 0x7fd6c00fc660 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.158+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6c0107290 con 0x7fd6c00fc280 2024-02-18T00:35:35.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.158+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:35:35.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.158+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 1 = system:0 2024-02-18T00:35:35.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.158+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c005ee80 osd=0 4 2024-02-18T00:35:35.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.158+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 0 1 2024-02-18T00:35:35.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 10 client.40706.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 2 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 2 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c005ee80 osd=0 3 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 0 2 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c005ee80 osd=0 4 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 10 client.40706.objecter _send_command 2 2024-02-18T00:35:35.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.174+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c0116fd0 con 0x7fd6c00fc280 2024-02-18T00:35:35.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c0116fd0 con 0x7fd6c00fc280 2024-02-18T00:35:35.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c005ee80 osd=0 4 2024-02-18T00:35:35.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 0 2 2024-02-18T00:35:35.178 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 10 client.40706.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 3 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6805/26279110 conn(0x7fd6c00f4ef0 0x7fd6c00273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 3 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 1 3 2024-02-18T00:35:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c00f4d80 osd=1 4 2024-02-18T00:35:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 10 client.40706.objecter _send_command 3 2024-02-18T00:35:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c00f8b50 con 0x7fd6c00f4ef0 2024-02-18T00:35:35.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6805/26279110 conn(0x7fd6c00f4ef0 0x7fd6c00273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c00f4ef0 2024-02-18T00:35:35.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.176+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6805/26279110 conn(0x7fd6c00f4ef0 0x7fd6c00273c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.178+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6c00f8b50 con 0x7fd6c00f4ef0 2024-02-18T00:35:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.178+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:35:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.178+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 3 = system:0 2024-02-18T00:35:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.178+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c00f4d80 osd=1 4 2024-02-18T00:35:35.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.178+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 1 3 2024-02-18T00:35:35.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 10 client.40706.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 4 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 4 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c00f4d80 osd=1 3 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 1 4 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c00f4d80 osd=1 4 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 10 client.40706.objecter _send_command 4 2024-02-18T00:35:35.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.193+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c00278b0 con 0x7fd6c00f4ef0 2024-02-18T00:35:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c00278b0 con 0x7fd6c00f4ef0 2024-02-18T00:35:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c00f4d80 osd=1 4 2024-02-18T00:35:35.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 1 4 2024-02-18T00:35:35.198 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:35:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 10 client.40706.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 5 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd6c0117550 0x7fd6c011e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 5 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 2 5 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01173e0 osd=2 4 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 10 client.40706.objecter _send_command 5 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c0028760 con 0x7fd6c0117550 2024-02-18T00:35:35.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.196+0000 7fd6beffd700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd6c0117550 0x7fd6c011e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.197+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c0117550 2024-02-18T00:35:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.197+0000 7fd6beffd700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd6c0117550 0x7fd6c011e570 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.197+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6c0028760 con 0x7fd6c0117550 2024-02-18T00:35:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.197+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:35:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.197+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 5 = system:0 2024-02-18T00:35:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.197+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c01173e0 osd=2 4 2024-02-18T00:35:35.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.197+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 2 5 2024-02-18T00:35:35.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 10 client.40706.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 6 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 6 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c01173e0 osd=2 3 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 2 6 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01173e0 osd=2 4 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 10 client.40706.objecter _send_command 6 2024-02-18T00:35:35.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.212+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c0027c60 con 0x7fd6c0117550 2024-02-18T00:35:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c0027c60 con 0x7fd6c0117550 2024-02-18T00:35:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c01173e0 osd=2 4 2024-02-18T00:35:35.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 2 6 2024-02-18T00:35:35.217 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:35:35.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 10 client.40706.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 7 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd6c0059e00 0x7fd6c0121bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 7 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 3 7 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0028070 osd=3 4 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 10 client.40706.objecter _send_command 7 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c0124870 con 0x7fd6c0059e00 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.215+0000 7fd6bffff700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd6c0059e00 0x7fd6c0121bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.216+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c0059e00 2024-02-18T00:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.216+0000 7fd6bffff700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd6c0059e00 0x7fd6c0121bf0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.216+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6c0124870 con 0x7fd6c0059e00 2024-02-18T00:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.216+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.216+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 7 = system:0 2024-02-18T00:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.216+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c0028070 osd=3 4 2024-02-18T00:35:35.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.216+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 3 7 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 10 client.40706.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 8 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 8 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c0028070 osd=3 3 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 3 8 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0028070 osd=3 4 2024-02-18T00:35:35.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 10 client.40706.objecter _send_command 8 2024-02-18T00:35:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.231+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c0059360 con 0x7fd6c0059e00 2024-02-18T00:35:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c0059360 con 0x7fd6c0059e00 2024-02-18T00:35:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c0028070 osd=3 4 2024-02-18T00:35:35.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 3 8 2024-02-18T00:35:35.235 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:35:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 10 client.40706.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 9 2024-02-18T00:35:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd6c00594d0 0x7fd6c0123260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 9 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 4 9 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0059360 osd=4 4 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 10 client.40706.objecter _send_command 9 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c011f870 con 0x7fd6c00594d0 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.233+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd6c00594d0 0x7fd6c0123260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.234+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c00594d0 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.234+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd6c00594d0 0x7fd6c0123260 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.234+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6c011f870 con 0x7fd6c00594d0 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.234+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:35:35.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.234+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 9 = system:0 2024-02-18T00:35:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.234+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c0059360 osd=4 4 2024-02-18T00:35:35.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.234+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 4 9 2024-02-18T00:35:35.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 10 client.40706.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 10 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 10 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c0059360 osd=4 3 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 4 10 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0059360 osd=4 4 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 10 client.40706.objecter _send_command 10 2024-02-18T00:35:35.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.249+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c011f870 con 0x7fd6c00594d0 2024-02-18T00:35:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.251+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c011f870 con 0x7fd6c00594d0 2024-02-18T00:35:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.251+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.251+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.251+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c0059360 osd=4 4 2024-02-18T00:35:35.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.251+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 4 10 2024-02-18T00:35:35.254 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.251+0000 7fd6c6251700 10 client.40706.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.251+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 11 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd6c0123c60 0x7fd6c01240a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 11 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 5 11 2024-02-18T00:35:35.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c011f870 osd=5 4 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 10 client.40706.objecter _send_command 11 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c01246b0 con 0x7fd6c0123c60 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6beffd700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd6c0123c60 0x7fd6c01240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c0123c60 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.252+0000 7fd6beffd700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd6c0123c60 0x7fd6c01240a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.253+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6c01246b0 con 0x7fd6c0123c60 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.253+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.253+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 11 = system:0 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.253+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c011f870 osd=5 4 2024-02-18T00:35:35.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.253+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 5 11 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 10 client.40706.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 12 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 12 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c011f870 osd=5 3 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 5 12 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c011f870 osd=5 4 2024-02-18T00:35:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 10 client.40706.objecter _send_command 12 2024-02-18T00:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.267+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c01246b0 con 0x7fd6c0123c60 2024-02-18T00:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.269+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c01246b0 con 0x7fd6c0123c60 2024-02-18T00:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.269+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.269+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.269+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c011f870 osd=5 4 2024-02-18T00:35:35.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.269+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 5 12 2024-02-18T00:35:35.272 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:35:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 10 client.40706.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 13 2024-02-18T00:35:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd6c0128fc0 0x7fd6c0129400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 13 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 6 13 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01246b0 osd=6 4 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 10 client.40706.objecter _send_command 13 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c012b000 con 0x7fd6c0128fc0 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6bffff700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd6c0128fc0 0x7fd6c0129400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c0128fc0 2024-02-18T00:35:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.270+0000 7fd6bffff700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd6c0128fc0 0x7fd6c0129400 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.271+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6c012b000 con 0x7fd6c0128fc0 2024-02-18T00:35:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.271+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:35:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.271+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 13 = system:0 2024-02-18T00:35:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.271+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c01246b0 osd=6 4 2024-02-18T00:35:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.271+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 6 13 2024-02-18T00:35:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 10 client.40706.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 14 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 14 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c01246b0 osd=6 3 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 6 14 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01246b0 osd=6 4 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 10 client.40706.objecter _send_command 14 2024-02-18T00:35:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.286+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c012b000 con 0x7fd6c0128fc0 2024-02-18T00:35:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c012b000 con 0x7fd6c0128fc0 2024-02-18T00:35:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.291 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:35:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c01246b0 osd=6 4 2024-02-18T00:35:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 6 14 2024-02-18T00:35:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 10 client.40706.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 15 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6812/53025171 conn(0x7fd6c012d850 0x7fd6c012dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 15 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 7 15 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c012b000 osd=7 4 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 10 client.40706.objecter _send_command 15 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6c012e300 con 0x7fd6c012d850 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.288+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6812/53025171 conn(0x7fd6c012d850 0x7fd6c012dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.289+0000 7fd6bcff9700 10 client.40706.objecter ms_handle_connect 0x7fd6c012d850 2024-02-18T00:35:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.289+0000 7fd6bf7fe700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6812/53025171 conn(0x7fd6c012d850 0x7fd6c012dc90 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.293+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd6b005c040 con 0x7fd6c012d850 2024-02-18T00:35:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.293+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:35:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.293+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 15 = system:0 2024-02-18T00:35:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.293+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c012b000 osd=7 4 2024-02-18T00:35:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.293+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 7 15 2024-02-18T00:35:35.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 10 client.40706.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:35.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign -1 16 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 20 client.40706.objecter _get_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_remove -1 16 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 20 client.40706.objecter get_session s=0x7fd6c012b000 osd=7 3 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 15 client.40706.objecter _session_command_op_assign 7 16 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c012b000 osd=7 4 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 10 client.40706.objecter _send_command 16 2024-02-18T00:35:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.332+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd6c0130540 con 0x7fd6c012d850 2024-02-18T00:35:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6bcff9700 1 -- 172.21.15.59:0/1451674738 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd6c0130540 con 0x7fd6c012d850 2024-02-18T00:35:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6bcff9700 10 client.40706.objecter ms_dispatch 0x7fd6c0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6bcff9700 10 client.40706.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6bcff9700 20 client.40706.objecter put_session s=0x7fd6c012b000 osd=7 4 2024-02-18T00:35:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6bcff9700 15 client.40706.objecter _session_command_op_remove 7 16 2024-02-18T00:35:35.336 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 librados: watch_flush enter 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 librados: watch_flush exit 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.0 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd6c00fc280 msgr2=0x7fd6c00fc660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd6c00fc280 0x7fd6c00fc660 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c005ee80 osd=0 1 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.1 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6805/26279110 conn(0x7fd6c00f4ef0 msgr2=0x7fd6c00273c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6805/26279110 conn(0x7fd6c00f4ef0 0x7fd6c00273c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c00f4d80 osd=1 1 2024-02-18T00:35:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.2 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd6c0117550 msgr2=0x7fd6c011e570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd6c0117550 0x7fd6c011e570 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01173e0 osd=2 1 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.3 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd6c0059e00 msgr2=0x7fd6c0121bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6bffff700 1 -- 172.21.15.59:0/1451674738 reap_dead start 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd6c0059e00 0x7fd6c0121bf0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0028070 osd=3 1 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.4 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd6c00594d0 msgr2=0x7fd6c0123260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd6c00594d0 0x7fd6c0123260 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c0059360 osd=4 1 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.5 2024-02-18T00:35:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd6c0123c60 msgr2=0x7fd6c01240a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd6c0123c60 0x7fd6c01240a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c011f870 osd=5 1 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.6 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd6c0128fc0 msgr2=0x7fd6c0129400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd6c0128fc0 0x7fd6c0129400 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c01246b0 osd=6 1 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter close_session for osd.7 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6812/53025171 conn(0x7fd6c012d850 msgr2=0x7fd6c012dc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.137:6812/53025171 conn(0x7fd6c012d850 0x7fd6c012dc90 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter put_session s=0x7fd6c012b000 osd=7 1 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 20 client.40706.objecter shutdown clearing up homeless session... 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 10 client.40706.objecter successfully canceled tick 2024-02-18T00:35:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6bffff700 1 -- 172.21.15.59:0/1451674738 reap_dead start 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd6a8062c90 msgr2=0x7fd6a8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd6a8062c90 0x7fd6a8065150 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 rx=0x7fd6c01098e0 tx=0x7fd6b400b040).stop 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c00f8d50 msgr2=0x7fd6c010dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c00f8d50 0x7fd6c010dfa0 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 rx=0x7fd6b00041d0 tx=0x7fd6b0009f60).stop 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 shutdown_connections 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6c00f8d50 0x7fd6c010dfa0 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 --2- 172.21.15.59:0/1451674738 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd6a8062c90 0x7fd6a8065150 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.334+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 >> 172.21.15.59:0/1451674738 conn(0x7fd6c00f2820 msgr2=0x7fd6c00f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.335+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 shutdown_connections 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.335+0000 7fd6c6251700 1 -- 172.21.15.59:0/1451674738 wait complete. 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:35.335+0000 7fd6c6251700 1 librados: shutdown 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:35:35.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:33.760567+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:35.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:33.851940+0000 mgr.y (mgr.24454) 278 : cluster [DBG] pgmap v370: 249 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:35.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:34.030779+0000 mgr.y (mgr.24454) 279 : audit [DBG] from='client.30882 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:35:35.740 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:33.760567+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:35.740 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:33.851940+0000 mgr.y (mgr.24454) 278 : cluster [DBG] pgmap v370: 249 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:35.740 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:34.030779+0000 mgr.y (mgr.24454) 279 : audit [DBG] from='client.30882 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:35:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:33.760567+0000 mon.b (mon.1) 319 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:33.851940+0000 mgr.y (mgr.24454) 278 : cluster [DBG] pgmap v370: 249 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 9.1 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:34.030779+0000 mgr.y (mgr.24454) 279 : audit [DBG] from='client.30882 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:35:36.058 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-02-18T00:35:33.851641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9286,"num_objects":381,"num_object_clones":25,"num_object_copies":1143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":381,"num_whiteouts":1,"num_read":1562,"num_read_kb":1192,"num_write":1286,"num_write_kb":646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":3765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2430,"ondisk_log_size":2430,"up":651,"acting":651,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":768,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2195308,"kb_used_data":28672,"kb_used_omap":0,"kb_used_meta":2166528,"kb_avail":654020756,"statfs":{"total":671965249536,"available":669717254144,"internally_reserved":0,"allocated":29360128,"data_stored":15766472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2218524672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-88,"num_objects":-8,"num_object_clones":-8,"num_object_copies":-24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000890"},"pg_stats":[{"pgid":"166.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"166.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:31.662135+0000","last_change":"2024-02-18T00:35:31.662135+0000","last_active":"2024-02-18T00:35:31.662135+0000","last_peered":"2024-02-18T00:35:31.662135+0000","last_clean":"2024-02-18T00:35:31.662135+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:31.662135+0000","last_undegraded":"2024-02-18T00:35:31.662135+0000","last_fullsized":"2024-02-18T00:35:31.662135+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:31.662135+0000","last_clean_scrub_stamp":"2024-02-18T00:35:31.662135+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":199,"reported_epoch":251,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:29.784264+0000","last_change":"2024-02-18T00:35:29.784264+0000","last_active":"2024-02-18T00:35:29.784264+0000","last_peered":"2024-02-18T00:35:29.784264+0000","last_clean":"2024-02-18T00:35:29.784264+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:35:29.784264+0000","last_undegraded":"2024-02-18T00:35:29.784264+0000","last_fullsized":"2024-02-18T00:35:29.784264+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.1","version":"0'0","reported_seq":173,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:30.477057+0000","last_change":"2024-02-18T00:35:30.477030+0000","last_active":"2024-02-18T00:35:30.477057+0000","last_peered":"2024-02-18T00:35:30.477057+0000","last_clean":"2024-02-18T00:35:30.477057+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:35:30.477057+0000","last_undegraded":"2024-02-18T00:35:30.477057+0000","last_fullsized":"2024-02-18T00:35:30.477057+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.2","version":"231'68","reported_seq":236,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654480+0000","last_change":"2024-02-18T00:35:18.620812+0000","last_active":"2024-02-18T00:35:29.654480+0000","last_peered":"2024-02-18T00:35:29.654480+0000","last_clean":"2024-02-18T00:35:29.654480+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:35:29.654480+0000","last_undegraded":"2024-02-18T00:35:29.654480+0000","last_fullsized":"2024-02-18T00:35:29.654480+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.3","version":"245'73","reported_seq":228,"reported_epoch":251,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:29.653926+0000","last_change":"2024-02-18T00:35:25.776188+0000","last_active":"2024-02-18T00:35:29.653926+0000","last_peered":"2024-02-18T00:35:29.653926+0000","last_clean":"2024-02-18T00:35:29.653926+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:35:29.653926+0000","last_undegraded":"2024-02-18T00:35:29.653926+0000","last_fullsized":"2024-02-18T00:35:29.653926+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.4","version":"230'120","reported_seq":275,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647708+0000","last_change":"2024-02-18T00:35:18.620088+0000","last_active":"2024-02-18T00:35:29.647708+0000","last_peered":"2024-02-18T00:35:29.647708+0000","last_clean":"2024-02-18T00:35:29.647708+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:35:29.647708+0000","last_undegraded":"2024-02-18T00:35:29.647708+0000","last_fullsized":"2024-02-18T00:35:29.647708+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.5","version":"240'90","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651942+0000","last_change":"2024-02-18T00:35:20.627929+0000","last_active":"2024-02-18T00:35:29.651942+0000","last_peered":"2024-02-18T00:35:29.651942+0000","last_clean":"2024-02-18T00:35:29.651942+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:35:29.651942+0000","last_undegraded":"2024-02-18T00:35:29.651942+0000","last_fullsized":"2024-02-18T00:35:29.651942+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.6","version":"243'108","reported_seq":224,"reported_epoch":251,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:29.655441+0000","last_change":"2024-02-18T00:35:23.774061+0000","last_active":"2024-02-18T00:35:29.655441+0000","last_peered":"2024-02-18T00:35:29.655441+0000","last_clean":"2024-02-18T00:35:29.655441+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:35:29.655441+0000","last_undegraded":"2024-02-18T00:35:29.655441+0000","last_fullsized":"2024-02-18T00:35:29.655441+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":108,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":162,"num_objects":16,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":5,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.7","version":"237'60","reported_seq":208,"reported_epoch":251,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:29.669670+0000","last_change":"2024-02-18T00:35:17.767938+0000","last_active":"2024-02-18T00:35:29.669670+0000","last_peered":"2024-02-18T00:35:29.669670+0000","last_clean":"2024-02-18T00:35:29.669670+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:35:29.669670+0000","last_undegraded":"2024-02-18T00:35:29.669670+0000","last_fullsized":"2024-02-18T00:35:29.669670+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":60,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.8","version":"215'68","reported_seq":195,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.666378+0000","last_change":"2024-02-18T00:35:18.617607+0000","last_active":"2024-02-18T00:35:29.666378+0000","last_peered":"2024-02-18T00:35:29.666378+0000","last_clean":"2024-02-18T00:35:29.666378+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:35:29.666378+0000","last_undegraded":"2024-02-18T00:35:29.666378+0000","last_fullsized":"2024-02-18T00:35:29.666378+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.9","version":"229'51","reported_seq":212,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647247+0000","last_change":"2024-02-18T00:35:18.609615+0000","last_active":"2024-02-18T00:35:29.647247+0000","last_peered":"2024-02-18T00:35:29.647247+0000","last_clean":"2024-02-18T00:35:29.647247+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:35:29.647247+0000","last_undegraded":"2024-02-18T00:35:29.647247+0000","last_fullsized":"2024-02-18T00:35:29.647247+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.a","version":"245'34","reported_seq":167,"reported_epoch":251,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:29.671019+0000","last_change":"2024-02-18T00:35:25.776579+0000","last_active":"2024-02-18T00:35:29.671019+0000","last_peered":"2024-02-18T00:35:29.671019+0000","last_clean":"2024-02-18T00:35:29.671019+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:35:29.671019+0000","last_undegraded":"2024-02-18T00:35:29.671019+0000","last_fullsized":"2024-02-18T00:35:29.671019+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"2"}]},{"pgid":"62.b","version":"246'54","reported_seq":193,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651172+0000","last_change":"2024-02-18T00:35:18.633458+0000","last_active":"2024-02-18T00:35:29.651172+0000","last_peered":"2024-02-18T00:35:29.651172+0000","last_clean":"2024-02-18T00:35:29.651172+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:35:29.651172+0000","last_undegraded":"2024-02-18T00:35:29.651172+0000","last_fullsized":"2024-02-18T00:35:29.651172+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.c","version":"213'17","reported_seq":184,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645766+0000","last_change":"2024-02-18T00:35:18.635372+0000","last_active":"2024-02-18T00:35:29.645766+0000","last_peered":"2024-02-18T00:35:29.645766+0000","last_clean":"2024-02-18T00:35:29.645766+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:35:29.645766+0000","last_undegraded":"2024-02-18T00:35:29.645766+0000","last_fullsized":"2024-02-18T00:35:29.645766+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.d","version":"240'23","reported_seq":209,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651840+0000","last_change":"2024-02-18T00:35:20.627940+0000","last_active":"2024-02-18T00:35:29.651840+0000","last_peered":"2024-02-18T00:35:29.651840+0000","last_clean":"2024-02-18T00:35:29.651840+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:35:29.651840+0000","last_undegraded":"2024-02-18T00:35:29.651840+0000","last_fullsized":"2024-02-18T00:35:29.651840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.e","version":"213'68","reported_seq":223,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644268+0000","last_change":"2024-02-18T00:35:18.616186+0000","last_active":"2024-02-18T00:35:29.644268+0000","last_peered":"2024-02-18T00:35:29.644268+0000","last_clean":"2024-02-18T00:35:29.644268+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:35:29.644268+0000","last_undegraded":"2024-02-18T00:35:29.644268+0000","last_fullsized":"2024-02-18T00:35:29.644268+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.f","version":"229'68","reported_seq":217,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655416+0000","last_change":"2024-02-18T00:35:18.612027+0000","last_active":"2024-02-18T00:35:29.655416+0000","last_peered":"2024-02-18T00:35:29.655416+0000","last_clean":"2024-02-18T00:35:29.655416+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:35:29.655416+0000","last_undegraded":"2024-02-18T00:35:29.655416+0000","last_fullsized":"2024-02-18T00:35:29.655416+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.10","version":"209'56","reported_seq":230,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.666254+0000","last_change":"2024-02-18T00:35:18.617390+0000","last_active":"2024-02-18T00:35:29.666254+0000","last_peered":"2024-02-18T00:35:29.666254+0000","last_clean":"2024-02-18T00:35:29.666254+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:35:29.666254+0000","last_undegraded":"2024-02-18T00:35:29.666254+0000","last_fullsized":"2024-02-18T00:35:29.666254+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.11","version":"214'19","reported_seq":188,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.660115+0000","last_change":"2024-02-18T00:35:18.616362+0000","last_active":"2024-02-18T00:35:29.660115+0000","last_peered":"2024-02-18T00:35:29.660115+0000","last_clean":"2024-02-18T00:35:29.660115+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:35:29.660115+0000","last_undegraded":"2024-02-18T00:35:29.660115+0000","last_fullsized":"2024-02-18T00:35:29.660115+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.12","version":"241'60","reported_seq":209,"reported_epoch":251,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:29.671145+0000","last_change":"2024-02-18T00:35:21.771750+0000","last_active":"2024-02-18T00:35:29.671145+0000","last_peered":"2024-02-18T00:35:29.671145+0000","last_clean":"2024-02-18T00:35:29.671145+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:35:29.671145+0000","last_undegraded":"2024-02-18T00:35:29.671145+0000","last_fullsized":"2024-02-18T00:35:29.671145+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":60,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":88,"num_objects":8,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"62.13","version":"210'17","reported_seq":184,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646872+0000","last_change":"2024-02-18T00:35:18.618560+0000","last_active":"2024-02-18T00:35:29.646872+0000","last_peered":"2024-02-18T00:35:29.646872+0000","last_clean":"2024-02-18T00:35:29.646872+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:35:29.646872+0000","last_undegraded":"2024-02-18T00:35:29.646872+0000","last_fullsized":"2024-02-18T00:35:29.646872+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.16","version":"225'22","reported_seq":192,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651115+0000","last_change":"2024-02-18T00:35:20.628234+0000","last_active":"2024-02-18T00:35:29.651115+0000","last_peered":"2024-02-18T00:35:29.651115+0000","last_clean":"2024-02-18T00:35:29.651115+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:35:29.651115+0000","last_undegraded":"2024-02-18T00:35:29.651115+0000","last_fullsized":"2024-02-18T00:35:29.651115+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.17","version":"244'18","reported_seq":200,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655846+0000","last_change":"2024-02-18T00:35:23.484016+0000","last_active":"2024-02-18T00:35:29.655846+0000","last_peered":"2024-02-18T00:35:29.655846+0000","last_clean":"2024-02-18T00:35:29.655846+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:35:29.655846+0000","last_undegraded":"2024-02-18T00:35:29.655846+0000","last_fullsized":"2024-02-18T00:35:29.655846+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"62.15","version":"225'102","reported_seq":263,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644233+0000","last_change":"2024-02-18T00:35:18.633687+0000","last_active":"2024-02-18T00:35:29.644233+0000","last_peered":"2024-02-18T00:35:29.644233+0000","last_clean":"2024-02-18T00:35:29.644233+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:35:29.644233+0000","last_undegraded":"2024-02-18T00:35:29.644233+0000","last_fullsized":"2024-02-18T00:35:29.644233+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.29","version":"0'0","reported_seq":166,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.653051+0000","last_change":"2024-02-18T00:34:15.230999+0000","last_active":"2024-02-18T00:35:29.653051+0000","last_peered":"2024-02-18T00:35:29.653051+0000","last_clean":"2024-02-18T00:35:29.653051+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:35:29.653051+0000","last_undegraded":"2024-02-18T00:35:29.653051+0000","last_fullsized":"2024-02-18T00:35:29.653051+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.230938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.230938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"251'98","reported_seq":249,"reported_epoch":251,"state":"active+clean+snaptrim","last_fresh":"2024-02-18T00:35:29.786073+0000","last_change":"2024-02-18T00:35:25.776196+0000","last_active":"2024-02-18T00:35:29.786073+0000","last_peered":"2024-02-18T00:35:29.786073+0000","last_clean":"2024-02-18T00:35:29.786073+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:35:29.786073+0000","last_undegraded":"2024-02-18T00:35:29.786073+0000","last_fullsized":"2024-02-18T00:35:29.786073+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":98,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"stat_sum":{"num_bytes":150,"num_objects":15,"num_object_clones":7,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":166,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643390+0000","last_change":"2024-02-18T00:34:39.758860+0000","last_active":"2024-02-18T00:35:29.643390+0000","last_peered":"2024-02-18T00:35:29.643390+0000","last_clean":"2024-02-18T00:35:29.643390+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:35:29.643390+0000","last_undegraded":"2024-02-18T00:35:29.643390+0000","last_fullsized":"2024-02-18T00:35:29.643390+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.758794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.758794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":208,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647293+0000","last_change":"2024-02-18T00:35:18.610791+0000","last_active":"2024-02-18T00:35:29.647293+0000","last_peered":"2024-02-18T00:35:29.647293+0000","last_clean":"2024-02-18T00:35:29.647293+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:35:29.647293+0000","last_undegraded":"2024-02-18T00:35:29.647293+0000","last_fullsized":"2024-02-18T00:35:29.647293+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.27","version":"0'0","reported_seq":122,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655785+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:35:29.655785+0000","last_peered":"2024-02-18T00:35:29.655785+0000","last_clean":"2024-02-18T00:35:29.655785+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:35:29.655785+0000","last_undegraded":"2024-02-18T00:35:29.655785+0000","last_fullsized":"2024-02-18T00:35:29.655785+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":263,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654597+0000","last_change":"2024-02-18T00:35:18.620279+0000","last_active":"2024-02-18T00:35:29.654597+0000","last_peered":"2024-02-18T00:35:29.654597+0000","last_clean":"2024-02-18T00:35:29.654597+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:35:29.654597+0000","last_undegraded":"2024-02-18T00:35:29.654597+0000","last_fullsized":"2024-02-18T00:35:29.654597+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.26","version":"0'0","reported_seq":120,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651258+0000","last_change":"2024-02-18T00:34:26.276135+0000","last_active":"2024-02-18T00:35:29.651258+0000","last_peered":"2024-02-18T00:35:29.651258+0000","last_clean":"2024-02-18T00:35:29.651258+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:35:29.651258+0000","last_undegraded":"2024-02-18T00:35:29.651258+0000","last_fullsized":"2024-02-18T00:35:29.651258+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.276071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.276071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":240,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.653063+0000","last_change":"2024-02-18T00:35:20.625861+0000","last_active":"2024-02-18T00:35:29.653063+0000","last_peered":"2024-02-18T00:35:29.653063+0000","last_clean":"2024-02-18T00:35:29.653063+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:35:29.653063+0000","last_undegraded":"2024-02-18T00:35:29.653063+0000","last_fullsized":"2024-02-18T00:35:29.653063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.25","version":"57'1","reported_seq":172,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.652440+0000","last_change":"2024-02-18T00:33:43.346148+0000","last_active":"2024-02-18T00:35:29.652440+0000","last_peered":"2024-02-18T00:35:29.652440+0000","last_clean":"2024-02-18T00:35:29.652440+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:35:29.652440+0000","last_undegraded":"2024-02-18T00:35:29.652440+0000","last_fullsized":"2024-02-18T00:35:29.652440+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:43.346088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.346088+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"213'17","reported_seq":190,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643381+0000","last_change":"2024-02-18T00:35:18.629237+0000","last_active":"2024-02-18T00:35:29.643381+0000","last_peered":"2024-02-18T00:35:29.643381+0000","last_clean":"2024-02-18T00:35:29.643381+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:35:29.643381+0000","last_undegraded":"2024-02-18T00:35:29.643381+0000","last_fullsized":"2024-02-18T00:35:29.643381+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.24","version":"0'0","reported_seq":166,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654358+0000","last_change":"2024-02-18T00:34:17.271924+0000","last_active":"2024-02-18T00:35:29.654358+0000","last_peered":"2024-02-18T00:35:29.654358+0000","last_clean":"2024-02-18T00:35:29.654358+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:35:29.654358+0000","last_undegraded":"2024-02-18T00:35:29.654358+0000","last_fullsized":"2024-02-18T00:35:29.654358+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.271867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.271867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"251'50","reported_seq":207,"reported_epoch":251,"state":"active+clean+snaptrim","last_fresh":"2024-02-18T00:35:29.786716+0000","last_change":"2024-02-18T00:35:29.784294+0000","last_active":"2024-02-18T00:35:29.786716+0000","last_peered":"2024-02-18T00:35:29.786716+0000","last_clean":"2024-02-18T00:35:29.786716+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:35:29.786716+0000","last_undegraded":"2024-02-18T00:35:29.786716+0000","last_fullsized":"2024-02-18T00:35:29.786716+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":50,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":47,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"1"}]},{"pgid":"2.23","version":"0'0","reported_seq":157,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651265+0000","last_change":"2024-02-18T00:33:55.783013+0000","last_active":"2024-02-18T00:35:29.651265+0000","last_peered":"2024-02-18T00:35:29.651265+0000","last_clean":"2024-02-18T00:35:29.651265+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:35:29.651265+0000","last_undegraded":"2024-02-18T00:35:29.651265+0000","last_fullsized":"2024-02-18T00:35:29.651265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.782955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.782955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":179,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.658622+0000","last_change":"2024-02-18T00:35:28.376224+0000","last_active":"2024-02-18T00:35:29.658622+0000","last_peered":"2024-02-18T00:35:29.658622+0000","last_clean":"2024-02-18T00:35:29.658622+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:35:29.658622+0000","last_undegraded":"2024-02-18T00:35:29.658622+0000","last_fullsized":"2024-02-18T00:35:29.658622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.22","version":"0'0","reported_seq":159,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654920+0000","last_change":"2024-02-18T00:33:49.344703+0000","last_active":"2024-02-18T00:35:29.654920+0000","last_peered":"2024-02-18T00:35:29.654920+0000","last_clean":"2024-02-18T00:35:29.654920+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:35:29.654920+0000","last_undegraded":"2024-02-18T00:35:29.654920+0000","last_fullsized":"2024-02-18T00:35:29.654920+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.344637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.344637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":185,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643010+0000","last_change":"2024-02-18T00:35:18.621441+0000","last_active":"2024-02-18T00:35:29.643010+0000","last_peered":"2024-02-18T00:35:29.643010+0000","last_clean":"2024-02-18T00:35:29.643010+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:35:29.643010+0000","last_undegraded":"2024-02-18T00:35:29.643010+0000","last_fullsized":"2024-02-18T00:35:29.643010+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.21","version":"0'0","reported_seq":237,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655238+0000","last_change":"2024-02-18T00:33:48.736504+0000","last_active":"2024-02-18T00:35:29.655238+0000","last_peered":"2024-02-18T00:35:29.655238+0000","last_clean":"2024-02-18T00:35:29.655238+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:35:29.655238+0000","last_undegraded":"2024-02-18T00:35:29.655238+0000","last_fullsized":"2024-02-18T00:35:29.655238+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.834878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.834878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"227'17","reported_seq":189,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:30.343587+0000","last_change":"2024-02-18T00:35:30.343532+0000","last_active":"2024-02-18T00:35:30.343587+0000","last_peered":"2024-02-18T00:35:30.343587+0000","last_clean":"2024-02-18T00:35:30.343587+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:35:30.343587+0000","last_undegraded":"2024-02-18T00:35:30.343587+0000","last_fullsized":"2024-02-18T00:35:30.343587+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"2"},{"start":"c","length":"1"},{"start":"e","length":"1"}]},{"pgid":"2.20","version":"0'0","reported_seq":191,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.641778+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:35:29.641778+0000","last_peered":"2024-02-18T00:35:29.641778+0000","last_clean":"2024-02-18T00:35:29.641778+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:35:29.641778+0000","last_undegraded":"2024-02-18T00:35:29.641778+0000","last_fullsized":"2024-02-18T00:35:29.641778+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643444+0000","last_change":"2024-02-18T00:33:48.727714+0000","last_active":"2024-02-18T00:35:29.643444+0000","last_peered":"2024-02-18T00:35:29.643444+0000","last_clean":"2024-02-18T00:35:29.643444+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:35:29.643444+0000","last_undegraded":"2024-02-18T00:35:29.643444+0000","last_fullsized":"2024-02-18T00:35:29.643444+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.653621+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:35:29.653621+0000","last_peered":"2024-02-18T00:35:29.653621+0000","last_clean":"2024-02-18T00:35:29.653621+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:35:29.653621+0000","last_undegraded":"2024-02-18T00:35:29.653621+0000","last_fullsized":"2024-02-18T00:35:29.653621+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":301,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.642052+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:35:29.642052+0000","last_peered":"2024-02-18T00:35:29.642052+0000","last_clean":"2024-02-18T00:35:29.642052+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:35:29.642052+0000","last_undegraded":"2024-02-18T00:35:29.642052+0000","last_fullsized":"2024-02-18T00:35:29.642052+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":230,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656408+0000","last_change":"2024-02-18T00:34:26.060181+0000","last_active":"2024-02-18T00:35:29.656408+0000","last_peered":"2024-02-18T00:35:29.656408+0000","last_clean":"2024-02-18T00:35:29.656408+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:35:29.656408+0000","last_undegraded":"2024-02-18T00:35:29.656408+0000","last_fullsized":"2024-02-18T00:35:29.656408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.060096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.060096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643588+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:35:29.643588+0000","last_peered":"2024-02-18T00:35:29.643588+0000","last_clean":"2024-02-18T00:35:29.643588+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:35:29.643588+0000","last_undegraded":"2024-02-18T00:35:29.643588+0000","last_fullsized":"2024-02-18T00:35:29.643588+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":159,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.657503+0000","last_change":"2024-02-18T00:35:20.428964+0000","last_active":"2024-02-18T00:35:29.657503+0000","last_peered":"2024-02-18T00:35:29.657503+0000","last_clean":"2024-02-18T00:35:29.657503+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:35:29.657503+0000","last_undegraded":"2024-02-18T00:35:29.657503+0000","last_fullsized":"2024-02-18T00:35:29.657503+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:20.428893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:20.428893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":242,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647440+0000","last_change":"2024-02-18T00:32:50.007673+0000","last_active":"2024-02-18T00:35:29.647440+0000","last_peered":"2024-02-18T00:35:29.647440+0000","last_clean":"2024-02-18T00:35:29.647440+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:35:29.647440+0000","last_undegraded":"2024-02-18T00:35:29.647440+0000","last_fullsized":"2024-02-18T00:35:29.647440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.007594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.007594+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.662821+0000","last_change":"2024-02-18T00:34:24.334917+0000","last_active":"2024-02-18T00:35:29.662821+0000","last_peered":"2024-02-18T00:35:29.662821+0000","last_clean":"2024-02-18T00:35:29.662821+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:35:29.662821+0000","last_undegraded":"2024-02-18T00:35:29.662821+0000","last_fullsized":"2024-02-18T00:35:29.662821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.334814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.334814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.671288+0000","last_change":"2024-02-18T00:33:42.768749+0000","last_active":"2024-02-18T00:35:29.671288+0000","last_peered":"2024-02-18T00:35:29.671288+0000","last_clean":"2024-02-18T00:35:29.671288+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:35:29.671288+0000","last_undegraded":"2024-02-18T00:35:29.671288+0000","last_fullsized":"2024-02-18T00:35:29.671288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.768630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.768630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":221,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647673+0000","last_change":"2024-02-18T00:35:24.990649+0000","last_active":"2024-02-18T00:35:29.647673+0000","last_peered":"2024-02-18T00:35:29.647673+0000","last_clean":"2024-02-18T00:35:29.647673+0000","last_became_active":"2024-02-18T00:33:48.734414+0000","last_became_peered":"2024-02-18T00:33:48.734414+0000","last_unstale":"2024-02-18T00:35:29.647673+0000","last_undegraded":"2024-02-18T00:35:29.647673+0000","last_fullsized":"2024-02-18T00:35:29.647673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.663565+0000","last_change":"2024-02-18T00:33:48.745303+0000","last_active":"2024-02-18T00:35:29.663565+0000","last_peered":"2024-02-18T00:35:29.663565+0000","last_clean":"2024-02-18T00:35:29.663565+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:35:29.663565+0000","last_undegraded":"2024-02-18T00:35:29.663565+0000","last_fullsized":"2024-02-18T00:35:29.663565+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":300,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655514+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:35:29.655514+0000","last_peered":"2024-02-18T00:35:29.655514+0000","last_clean":"2024-02-18T00:35:29.655514+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:35:29.655514+0000","last_undegraded":"2024-02-18T00:35:29.655514+0000","last_fullsized":"2024-02-18T00:35:29.655514+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654735+0000","last_change":"2024-02-18T00:33:46.861306+0000","last_active":"2024-02-18T00:35:29.654735+0000","last_peered":"2024-02-18T00:35:29.654735+0000","last_clean":"2024-02-18T00:35:29.654735+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:35:29.654735+0000","last_undegraded":"2024-02-18T00:35:29.654735+0000","last_fullsized":"2024-02-18T00:35:29.654735+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.861233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.861233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.653922+0000","last_change":"2024-02-18T00:33:53.005012+0000","last_active":"2024-02-18T00:35:29.653922+0000","last_peered":"2024-02-18T00:35:29.653922+0000","last_clean":"2024-02-18T00:35:29.653922+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:35:29.653922+0000","last_undegraded":"2024-02-18T00:35:29.653922+0000","last_fullsized":"2024-02-18T00:35:29.653922+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.004903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.004903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654816+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:35:29.654816+0000","last_peered":"2024-02-18T00:35:29.654816+0000","last_clean":"2024-02-18T00:35:29.654816+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:35:29.654816+0000","last_undegraded":"2024-02-18T00:35:29.654816+0000","last_fullsized":"2024-02-18T00:35:29.654816+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.642340+0000","last_change":"2024-02-18T00:33:02.523582+0000","last_active":"2024-02-18T00:35:29.642340+0000","last_peered":"2024-02-18T00:35:29.642340+0000","last_clean":"2024-02-18T00:35:29.642340+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:35:29.642340+0000","last_undegraded":"2024-02-18T00:35:29.642340+0000","last_fullsized":"2024-02-18T00:35:29.642340+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":250,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.652309+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:35:29.652309+0000","last_peered":"2024-02-18T00:35:29.652309+0000","last_clean":"2024-02-18T00:35:29.652309+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:35:29.652309+0000","last_undegraded":"2024-02-18T00:35:29.652309+0000","last_fullsized":"2024-02-18T00:35:29.652309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648301+0000","last_change":"2024-02-18T00:34:18.301213+0000","last_active":"2024-02-18T00:35:29.648301+0000","last_peered":"2024-02-18T00:35:29.648301+0000","last_clean":"2024-02-18T00:35:29.648301+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:35:29.648301+0000","last_undegraded":"2024-02-18T00:35:29.648301+0000","last_fullsized":"2024-02-18T00:35:29.648301+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.301136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.301136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.657654+0000","last_change":"2024-02-18T00:34:39.453471+0000","last_active":"2024-02-18T00:35:29.657654+0000","last_peered":"2024-02-18T00:35:29.657654+0000","last_clean":"2024-02-18T00:35:29.657654+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:35:29.657654+0000","last_undegraded":"2024-02-18T00:35:29.657654+0000","last_fullsized":"2024-02-18T00:35:29.657654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.453312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.453312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.669848+0000","last_change":"2024-02-18T00:33:45.813625+0000","last_active":"2024-02-18T00:35:29.669848+0000","last_peered":"2024-02-18T00:35:29.669848+0000","last_clean":"2024-02-18T00:35:29.669848+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:35:29.669848+0000","last_undegraded":"2024-02-18T00:35:29.669848+0000","last_fullsized":"2024-02-18T00:35:29.669848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.813514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.813514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647754+0000","last_change":"2024-02-18T00:33:48.733994+0000","last_active":"2024-02-18T00:35:29.647754+0000","last_peered":"2024-02-18T00:35:29.647754+0000","last_clean":"2024-02-18T00:35:29.647754+0000","last_became_active":"2024-02-18T00:33:48.732220+0000","last_became_peered":"2024-02-18T00:33:48.732220+0000","last_unstale":"2024-02-18T00:35:29.647754+0000","last_undegraded":"2024-02-18T00:35:29.647754+0000","last_fullsized":"2024-02-18T00:35:29.647754+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":232,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.649436+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:35:29.649436+0000","last_peered":"2024-02-18T00:35:29.649436+0000","last_clean":"2024-02-18T00:35:29.649436+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:35:29.649436+0000","last_undegraded":"2024-02-18T00:35:29.649436+0000","last_fullsized":"2024-02-18T00:35:29.649436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":228,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.664717+0000","last_change":"2024-02-18T00:34:23.295161+0000","last_active":"2024-02-18T00:35:29.664717+0000","last_peered":"2024-02-18T00:35:29.664717+0000","last_clean":"2024-02-18T00:35:29.664717+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:35:29.664717+0000","last_undegraded":"2024-02-18T00:35:29.664717+0000","last_fullsized":"2024-02-18T00:35:29.664717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.295051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":314,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.657388+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:35:29.657388+0000","last_peered":"2024-02-18T00:35:29.657388+0000","last_clean":"2024-02-18T00:35:29.657388+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:35:29.657388+0000","last_undegraded":"2024-02-18T00:35:29.657388+0000","last_fullsized":"2024-02-18T00:35:29.657388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651648+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:35:29.651648+0000","last_peered":"2024-02-18T00:35:29.651648+0000","last_clean":"2024-02-18T00:35:29.651648+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:35:29.651648+0000","last_undegraded":"2024-02-18T00:35:29.651648+0000","last_fullsized":"2024-02-18T00:35:29.651648+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":159,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.665550+0000","last_change":"2024-02-18T00:35:29.520701+0000","last_active":"2024-02-18T00:35:29.665550+0000","last_peered":"2024-02-18T00:35:29.665550+0000","last_clean":"2024-02-18T00:35:29.665550+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:35:29.665550+0000","last_undegraded":"2024-02-18T00:35:29.665550+0000","last_fullsized":"2024-02-18T00:35:29.665550+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.520632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.520632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":220,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645216+0000","last_change":"2024-02-18T00:33:44.007884+0000","last_active":"2024-02-18T00:35:29.645216+0000","last_peered":"2024-02-18T00:35:29.645216+0000","last_clean":"2024-02-18T00:35:29.645216+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:35:29.645216+0000","last_undegraded":"2024-02-18T00:35:29.645216+0000","last_fullsized":"2024-02-18T00:35:29.645216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:33:44.007804+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:33:44.007804+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":237,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.660750+0000","last_change":"2024-02-18T00:33:48.713658+0000","last_active":"2024-02-18T00:35:29.660750+0000","last_peered":"2024-02-18T00:35:29.660750+0000","last_clean":"2024-02-18T00:35:29.660750+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:35:29.660750+0000","last_undegraded":"2024-02-18T00:35:29.660750+0000","last_fullsized":"2024-02-18T00:35:29.660750+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:33:47.199009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":298,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651567+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:35:29.651567+0000","last_peered":"2024-02-18T00:35:29.651567+0000","last_clean":"2024-02-18T00:35:29.651567+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:35:29.651567+0000","last_undegraded":"2024-02-18T00:35:29.651567+0000","last_fullsized":"2024-02-18T00:35:29.651567+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":208,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643440+0000","last_change":"2024-02-18T00:33:41.057722+0000","last_active":"2024-02-18T00:35:29.643440+0000","last_peered":"2024-02-18T00:35:29.643440+0000","last_clean":"2024-02-18T00:35:29.643440+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:35:29.643440+0000","last_undegraded":"2024-02-18T00:35:29.643440+0000","last_fullsized":"2024-02-18T00:35:29.643440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.057659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.057659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647485+0000","last_change":"2024-02-18T00:33:02.533944+0000","last_active":"2024-02-18T00:35:29.647485+0000","last_peered":"2024-02-18T00:35:29.647485+0000","last_clean":"2024-02-18T00:35:29.647485+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:35:29.647485+0000","last_undegraded":"2024-02-18T00:35:29.647485+0000","last_fullsized":"2024-02-18T00:35:29.647485+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":251,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651344+0000","last_change":"2024-02-18T00:33:45.311343+0000","last_active":"2024-02-18T00:35:29.651344+0000","last_peered":"2024-02-18T00:35:29.651344+0000","last_clean":"2024-02-18T00:35:29.651344+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:35:29.651344+0000","last_undegraded":"2024-02-18T00:35:29.651344+0000","last_fullsized":"2024-02-18T00:35:29.651344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:45.311259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.311259+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":190,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646351+0000","last_change":"2024-02-18T00:34:27.927933+0000","last_active":"2024-02-18T00:35:29.646351+0000","last_peered":"2024-02-18T00:35:29.646351+0000","last_clean":"2024-02-18T00:35:29.646351+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:35:29.646351+0000","last_undegraded":"2024-02-18T00:35:29.646351+0000","last_fullsized":"2024-02-18T00:35:29.646351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.927871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.927871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645396+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:35:29.645396+0000","last_peered":"2024-02-18T00:35:29.645396+0000","last_clean":"2024-02-18T00:35:29.645396+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:35:29.645396+0000","last_undegraded":"2024-02-18T00:35:29.645396+0000","last_fullsized":"2024-02-18T00:35:29.645396+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645989+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:35:29.645989+0000","last_peered":"2024-02-18T00:35:29.645989+0000","last_clean":"2024-02-18T00:35:29.645989+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:35:29.645989+0000","last_undegraded":"2024-02-18T00:35:29.645989+0000","last_fullsized":"2024-02-18T00:35:29.645989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":159,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654222+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:35:29.654222+0000","last_peered":"2024-02-18T00:35:29.654222+0000","last_clean":"2024-02-18T00:35:29.654222+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:35:29.654222+0000","last_undegraded":"2024-02-18T00:35:29.654222+0000","last_fullsized":"2024-02-18T00:35:29.654222+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":299,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643865+0000","last_change":"2024-02-18T00:33:54.753052+0000","last_active":"2024-02-18T00:35:29.643865+0000","last_peered":"2024-02-18T00:35:29.643865+0000","last_clean":"2024-02-18T00:35:29.643865+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:35:29.643865+0000","last_undegraded":"2024-02-18T00:35:29.643865+0000","last_fullsized":"2024-02-18T00:35:29.643865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:33:54.752985+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.752985+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":253,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648345+0000","last_change":"2024-02-18T00:33:53.320429+0000","last_active":"2024-02-18T00:35:29.648345+0000","last_peered":"2024-02-18T00:35:29.648345+0000","last_clean":"2024-02-18T00:35:29.648345+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:35:29.648345+0000","last_undegraded":"2024-02-18T00:35:29.648345+0000","last_fullsized":"2024-02-18T00:35:29.648345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:33:53.320356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.320356+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":237,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656070+0000","last_change":"2024-02-18T00:33:48.672342+0000","last_active":"2024-02-18T00:35:29.656070+0000","last_peered":"2024-02-18T00:35:29.656070+0000","last_clean":"2024-02-18T00:35:29.656070+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:35:29.656070+0000","last_undegraded":"2024-02-18T00:35:29.656070+0000","last_fullsized":"2024-02-18T00:35:29.656070+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.368010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.368010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654018+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:35:29.654018+0000","last_peered":"2024-02-18T00:35:29.654018+0000","last_clean":"2024-02-18T00:35:29.654018+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:35:29.654018+0000","last_undegraded":"2024-02-18T00:35:29.654018+0000","last_fullsized":"2024-02-18T00:35:29.654018+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643143+0000","last_change":"2024-02-18T00:33:02.477932+0000","last_active":"2024-02-18T00:35:29.643143+0000","last_peered":"2024-02-18T00:35:29.643143+0000","last_clean":"2024-02-18T00:35:29.643143+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:35:29.643143+0000","last_undegraded":"2024-02-18T00:35:29.643143+0000","last_fullsized":"2024-02-18T00:35:29.643143+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.649170+0000","last_change":"2024-02-18T00:33:50.287103+0000","last_active":"2024-02-18T00:35:29.649170+0000","last_peered":"2024-02-18T00:35:29.649170+0000","last_clean":"2024-02-18T00:35:29.649170+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:35:29.649170+0000","last_undegraded":"2024-02-18T00:35:29.649170+0000","last_fullsized":"2024-02-18T00:35:29.649170+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.287007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.287007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":320,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643775+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:35:29.643775+0000","last_peered":"2024-02-18T00:35:29.643775+0000","last_clean":"2024-02-18T00:35:29.643775+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:35:29.643775+0000","last_undegraded":"2024-02-18T00:35:29.643775+0000","last_fullsized":"2024-02-18T00:35:29.643775+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":228,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.657585+0000","last_change":"2024-02-18T00:33:51.202714+0000","last_active":"2024-02-18T00:35:29.657585+0000","last_peered":"2024-02-18T00:35:29.657585+0000","last_clean":"2024-02-18T00:35:29.657585+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:35:29.657585+0000","last_undegraded":"2024-02-18T00:35:29.657585+0000","last_fullsized":"2024-02-18T00:35:29.657585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.202648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.202648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648554+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:35:29.648554+0000","last_peered":"2024-02-18T00:35:29.648554+0000","last_clean":"2024-02-18T00:35:29.648554+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:35:29.648554+0000","last_undegraded":"2024-02-18T00:35:29.648554+0000","last_fullsized":"2024-02-18T00:35:29.648554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643263+0000","last_change":"2024-02-18T00:33:04.650781+0000","last_active":"2024-02-18T00:35:29.643263+0000","last_peered":"2024-02-18T00:35:29.643263+0000","last_clean":"2024-02-18T00:35:29.643263+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:35:29.643263+0000","last_undegraded":"2024-02-18T00:35:29.643263+0000","last_fullsized":"2024-02-18T00:35:29.643263+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655000+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:35:29.655000+0000","last_peered":"2024-02-18T00:35:29.655000+0000","last_clean":"2024-02-18T00:35:29.655000+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:35:29.655000+0000","last_undegraded":"2024-02-18T00:35:29.655000+0000","last_fullsized":"2024-02-18T00:35:29.655000+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":296,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647303+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:35:29.647303+0000","last_peered":"2024-02-18T00:35:29.647303+0000","last_clean":"2024-02-18T00:35:29.647303+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:35:29.647303+0000","last_undegraded":"2024-02-18T00:35:29.647303+0000","last_fullsized":"2024-02-18T00:35:29.647303+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":127,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643709+0000","last_change":"2024-02-18T00:34:32.021327+0000","last_active":"2024-02-18T00:35:29.643709+0000","last_peered":"2024-02-18T00:35:29.643709+0000","last_clean":"2024-02-18T00:35:29.643709+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:35:29.643709+0000","last_undegraded":"2024-02-18T00:35:29.643709+0000","last_fullsized":"2024-02-18T00:35:29.643709+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:34:32.021219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.021219+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.660835+0000","last_change":"2024-02-18T00:35:13.452285+0000","last_active":"2024-02-18T00:35:29.660835+0000","last_peered":"2024-02-18T00:35:29.660835+0000","last_clean":"2024-02-18T00:35:29.660835+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:35:29.660835+0000","last_undegraded":"2024-02-18T00:35:29.660835+0000","last_fullsized":"2024-02-18T00:35:29.660835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.452220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.452220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.661195+0000","last_change":"2024-02-18T00:33:04.587776+0000","last_active":"2024-02-18T00:35:29.661195+0000","last_peered":"2024-02-18T00:35:29.661195+0000","last_clean":"2024-02-18T00:35:29.661195+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:35:29.661195+0000","last_undegraded":"2024-02-18T00:35:29.661195+0000","last_fullsized":"2024-02-18T00:35:29.661195+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645512+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:35:29.645512+0000","last_peered":"2024-02-18T00:35:29.645512+0000","last_clean":"2024-02-18T00:35:29.645512+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:35:29.645512+0000","last_undegraded":"2024-02-18T00:35:29.645512+0000","last_fullsized":"2024-02-18T00:35:29.645512+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":234,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646643+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:35:29.646643+0000","last_peered":"2024-02-18T00:35:29.646643+0000","last_clean":"2024-02-18T00:35:29.646643+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:35:29.646643+0000","last_undegraded":"2024-02-18T00:35:29.646643+0000","last_fullsized":"2024-02-18T00:35:29.646643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":190,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.667527+0000","last_change":"2024-02-18T00:34:26.340185+0000","last_active":"2024-02-18T00:35:29.667527+0000","last_peered":"2024-02-18T00:35:29.667527+0000","last_clean":"2024-02-18T00:35:29.667527+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:35:29.667527+0000","last_undegraded":"2024-02-18T00:35:29.667527+0000","last_fullsized":"2024-02-18T00:35:29.667527+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.340120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.340120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":217,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647314+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:35:29.647314+0000","last_peered":"2024-02-18T00:35:29.647314+0000","last_clean":"2024-02-18T00:35:29.647314+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:35:29.647314+0000","last_undegraded":"2024-02-18T00:35:29.647314+0000","last_fullsized":"2024-02-18T00:35:29.647314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.657747+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:35:29.657747+0000","last_peered":"2024-02-18T00:35:29.657747+0000","last_clean":"2024-02-18T00:35:29.657747+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:35:29.657747+0000","last_undegraded":"2024-02-18T00:35:29.657747+0000","last_fullsized":"2024-02-18T00:35:29.657747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646432+0000","last_change":"2024-02-18T00:33:48.731380+0000","last_active":"2024-02-18T00:35:29.646432+0000","last_peered":"2024-02-18T00:35:29.646432+0000","last_clean":"2024-02-18T00:35:29.646432+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:35:29.646432+0000","last_undegraded":"2024-02-18T00:35:29.646432+0000","last_fullsized":"2024-02-18T00:35:29.646432+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654436+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:35:29.654436+0000","last_peered":"2024-02-18T00:35:29.654436+0000","last_clean":"2024-02-18T00:35:29.654436+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:35:29.654436+0000","last_undegraded":"2024-02-18T00:35:29.654436+0000","last_fullsized":"2024-02-18T00:35:29.654436+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":269,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654011+0000","last_change":"2024-02-18T00:34:28.290944+0000","last_active":"2024-02-18T00:35:29.654011+0000","last_peered":"2024-02-18T00:35:29.654011+0000","last_clean":"2024-02-18T00:35:29.654011+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:35:29.654011+0000","last_undegraded":"2024-02-18T00:35:29.654011+0000","last_fullsized":"2024-02-18T00:35:29.654011+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:34:28.290868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.290868+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":153,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646394+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:35:29.646394+0000","last_peered":"2024-02-18T00:35:29.646394+0000","last_clean":"2024-02-18T00:35:29.646394+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:35:29.646394+0000","last_undegraded":"2024-02-18T00:35:29.646394+0000","last_fullsized":"2024-02-18T00:35:29.646394+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":237,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644512+0000","last_change":"2024-02-18T00:34:35.765124+0000","last_active":"2024-02-18T00:35:29.644512+0000","last_peered":"2024-02-18T00:35:29.644512+0000","last_clean":"2024-02-18T00:35:29.644512+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:35:29.644512+0000","last_undegraded":"2024-02-18T00:35:29.644512+0000","last_fullsized":"2024-02-18T00:35:29.644512+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.765059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.765059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644945+0000","last_change":"2024-02-18T00:33:02.549194+0000","last_active":"2024-02-18T00:35:29.644945+0000","last_peered":"2024-02-18T00:35:29.644945+0000","last_clean":"2024-02-18T00:35:29.644945+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:35:29.644945+0000","last_undegraded":"2024-02-18T00:35:29.644945+0000","last_fullsized":"2024-02-18T00:35:29.644945+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:02.549081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.549081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645511+0000","last_change":"2024-02-18T00:34:17.016556+0000","last_active":"2024-02-18T00:35:29.645511+0000","last_peered":"2024-02-18T00:35:29.645511+0000","last_clean":"2024-02-18T00:35:29.645511+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:35:29.645511+0000","last_undegraded":"2024-02-18T00:35:29.645511+0000","last_fullsized":"2024-02-18T00:35:29.645511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.016476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.016476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643692+0000","last_change":"2024-02-18T00:33:04.649428+0000","last_active":"2024-02-18T00:35:29.643692+0000","last_peered":"2024-02-18T00:35:29.643692+0000","last_clean":"2024-02-18T00:35:29.643692+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:35:29.643692+0000","last_undegraded":"2024-02-18T00:35:29.643692+0000","last_fullsized":"2024-02-18T00:35:29.643692+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656602+0000","last_change":"2024-02-18T00:33:49.897870+0000","last_active":"2024-02-18T00:35:29.656602+0000","last_peered":"2024-02-18T00:35:29.656602+0000","last_clean":"2024-02-18T00:35:29.656602+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:35:29.656602+0000","last_undegraded":"2024-02-18T00:35:29.656602+0000","last_fullsized":"2024-02-18T00:35:29.656602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.897812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.897812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":259,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.658314+0000","last_change":"2024-02-18T00:34:25.385634+0000","last_active":"2024-02-18T00:35:29.658314+0000","last_peered":"2024-02-18T00:35:29.658314+0000","last_clean":"2024-02-18T00:35:29.658314+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:35:29.658314+0000","last_undegraded":"2024-02-18T00:35:29.658314+0000","last_fullsized":"2024-02-18T00:35:29.658314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:34:25.385564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.385564+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":228,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655113+0000","last_change":"2024-02-18T00:33:37.809035+0000","last_active":"2024-02-18T00:35:29.655113+0000","last_peered":"2024-02-18T00:35:29.655113+0000","last_clean":"2024-02-18T00:35:29.655113+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:35:29.655113+0000","last_undegraded":"2024-02-18T00:35:29.655113+0000","last_fullsized":"2024-02-18T00:35:29.655113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:37.808923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.808923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":589,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:30.575470+0000","last_change":"2024-02-18T00:33:41.154008+0000","last_active":"2024-02-18T00:35:30.575470+0000","last_peered":"2024-02-18T00:35:30.575470+0000","last_clean":"2024-02-18T00:35:30.575470+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:35:30.575470+0000","last_undegraded":"2024-02-18T00:35:30.575470+0000","last_fullsized":"2024-02-18T00:35:30.575470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:33:41.153868+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.153868+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":367,"num_read_kb":362,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":156,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654153+0000","last_change":"2024-02-18T00:34:29.252896+0000","last_active":"2024-02-18T00:35:29.654153+0000","last_peered":"2024-02-18T00:35:29.654153+0000","last_clean":"2024-02-18T00:35:29.654153+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:35:29.654153+0000","last_undegraded":"2024-02-18T00:35:29.654153+0000","last_fullsized":"2024-02-18T00:35:29.654153+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:34:29.252830+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.252830+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.649207+0000","last_change":"2024-02-18T00:33:48.670413+0000","last_active":"2024-02-18T00:35:29.649207+0000","last_peered":"2024-02-18T00:35:29.649207+0000","last_clean":"2024-02-18T00:35:29.649207+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:35:29.649207+0000","last_undegraded":"2024-02-18T00:35:29.649207+0000","last_fullsized":"2024-02-18T00:35:29.649207+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":214,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643787+0000","last_change":"2024-02-18T00:34:19.925200+0000","last_active":"2024-02-18T00:35:29.643787+0000","last_peered":"2024-02-18T00:35:29.643787+0000","last_clean":"2024-02-18T00:35:29.643787+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:35:29.643787+0000","last_undegraded":"2024-02-18T00:35:29.643787+0000","last_fullsized":"2024-02-18T00:35:29.643787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:19.925111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.925111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":355,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656918+0000","last_change":"2024-02-18T00:33:54.198053+0000","last_active":"2024-02-18T00:35:29.656918+0000","last_peered":"2024-02-18T00:35:29.656918+0000","last_clean":"2024-02-18T00:35:29.656918+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:35:29.656918+0000","last_undegraded":"2024-02-18T00:35:29.656918+0000","last_fullsized":"2024-02-18T00:35:29.656918+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:33:54.197976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.197976+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":228,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644043+0000","last_change":"2024-02-18T00:34:15.045746+0000","last_active":"2024-02-18T00:35:29.644043+0000","last_peered":"2024-02-18T00:35:29.644043+0000","last_clean":"2024-02-18T00:35:29.644043+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:35:29.644043+0000","last_undegraded":"2024-02-18T00:35:29.644043+0000","last_fullsized":"2024-02-18T00:35:29.644043+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.045677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.045677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":217,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.642800+0000","last_change":"2024-02-18T00:33:48.688410+0000","last_active":"2024-02-18T00:35:29.642800+0000","last_peered":"2024-02-18T00:35:29.642800+0000","last_clean":"2024-02-18T00:35:29.642800+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:35:29.642800+0000","last_undegraded":"2024-02-18T00:35:29.642800+0000","last_fullsized":"2024-02-18T00:35:29.642800+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:43.926301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.926301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.650579+0000","last_change":"2024-02-18T00:33:04.719631+0000","last_active":"2024-02-18T00:35:29.650579+0000","last_peered":"2024-02-18T00:35:29.650579+0000","last_clean":"2024-02-18T00:35:29.650579+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:35:29.650579+0000","last_undegraded":"2024-02-18T00:35:29.650579+0000","last_fullsized":"2024-02-18T00:35:29.650579+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":297,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647496+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:35:29.647496+0000","last_peered":"2024-02-18T00:35:29.647496+0000","last_clean":"2024-02-18T00:35:29.647496+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:35:29.647496+0000","last_undegraded":"2024-02-18T00:35:29.647496+0000","last_fullsized":"2024-02-18T00:35:29.647496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":217,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.671516+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:35:29.671516+0000","last_peered":"2024-02-18T00:35:29.671516+0000","last_clean":"2024-02-18T00:35:29.671516+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:35:29.671516+0000","last_undegraded":"2024-02-18T00:35:29.671516+0000","last_fullsized":"2024-02-18T00:35:29.671516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":242,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656992+0000","last_change":"2024-02-18T00:34:31.410145+0000","last_active":"2024-02-18T00:35:29.656992+0000","last_peered":"2024-02-18T00:35:29.656992+0000","last_clean":"2024-02-18T00:35:29.656992+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:35:29.656992+0000","last_undegraded":"2024-02-18T00:35:29.656992+0000","last_fullsized":"2024-02-18T00:35:29.656992+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.410081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.410081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":207,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646211+0000","last_change":"2024-02-18T00:33:48.010937+0000","last_active":"2024-02-18T00:35:29.646211+0000","last_peered":"2024-02-18T00:35:29.646211+0000","last_clean":"2024-02-18T00:35:29.646211+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:35:29.646211+0000","last_undegraded":"2024-02-18T00:35:29.646211+0000","last_fullsized":"2024-02-18T00:35:29.646211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:48.010821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:48.010821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654146+0000","last_change":"2024-02-18T00:33:02.483549+0000","last_active":"2024-02-18T00:35:29.654146+0000","last_peered":"2024-02-18T00:35:29.654146+0000","last_clean":"2024-02-18T00:35:29.654146+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:35:29.654146+0000","last_undegraded":"2024-02-18T00:35:29.654146+0000","last_fullsized":"2024-02-18T00:35:29.654146+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654672+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:35:29.654672+0000","last_peered":"2024-02-18T00:35:29.654672+0000","last_clean":"2024-02-18T00:35:29.654672+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:35:29.654672+0000","last_undegraded":"2024-02-18T00:35:29.654672+0000","last_fullsized":"2024-02-18T00:35:29.654672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":270,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648518+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:35:29.648518+0000","last_peered":"2024-02-18T00:35:29.648518+0000","last_clean":"2024-02-18T00:35:29.648518+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:35:29.648518+0000","last_undegraded":"2024-02-18T00:35:29.648518+0000","last_fullsized":"2024-02-18T00:35:29.648518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":252,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656023+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:35:29.656023+0000","last_peered":"2024-02-18T00:35:29.656023+0000","last_clean":"2024-02-18T00:35:29.656023+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:35:29.656023+0000","last_undegraded":"2024-02-18T00:35:29.656023+0000","last_fullsized":"2024-02-18T00:35:29.656023+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":222,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651777+0000","last_change":"2024-02-18T00:34:26.701231+0000","last_active":"2024-02-18T00:35:29.651777+0000","last_peered":"2024-02-18T00:35:29.651777+0000","last_clean":"2024-02-18T00:35:29.651777+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:35:29.651777+0000","last_undegraded":"2024-02-18T00:35:29.651777+0000","last_fullsized":"2024-02-18T00:35:29.651777+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.701162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.701162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"109'2","reported_seq":187,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.667435+0000","last_change":"2024-02-18T00:34:31.655028+0000","last_active":"2024-02-18T00:35:29.667435+0000","last_peered":"2024-02-18T00:35:29.667435+0000","last_clean":"2024-02-18T00:35:29.667435+0000","last_became_active":"2024-02-18T00:34:31.654426+0000","last_became_peered":"2024-02-18T00:34:31.654426+0000","last_unstale":"2024-02-18T00:35:29.667435+0000","last_undegraded":"2024-02-18T00:35:29.667435+0000","last_fullsized":"2024-02-18T00:35:29.667435+0000","mapping_epoch":149,"log_start":"109'2","ondisk_log_start":"109'2","created":106,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648049+0000","last_change":"2024-02-18T00:33:02.533610+0000","last_active":"2024-02-18T00:35:29.648049+0000","last_peered":"2024-02-18T00:35:29.648049+0000","last_clean":"2024-02-18T00:35:29.648049+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:35:29.648049+0000","last_undegraded":"2024-02-18T00:35:29.648049+0000","last_fullsized":"2024-02-18T00:35:29.648049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":285,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646739+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:35:29.646739+0000","last_peered":"2024-02-18T00:35:29.646739+0000","last_clean":"2024-02-18T00:35:29.646739+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:35:29.646739+0000","last_undegraded":"2024-02-18T00:35:29.646739+0000","last_fullsized":"2024-02-18T00:35:29.646739+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654585+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:35:29.654585+0000","last_peered":"2024-02-18T00:35:29.654585+0000","last_clean":"2024-02-18T00:35:29.654585+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:35:29.654585+0000","last_undegraded":"2024-02-18T00:35:29.654585+0000","last_fullsized":"2024-02-18T00:35:29.654585+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.663412+0000","last_change":"2024-02-18T00:34:25.683506+0000","last_active":"2024-02-18T00:35:29.663412+0000","last_peered":"2024-02-18T00:35:29.663412+0000","last_clean":"2024-02-18T00:35:29.663412+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:35:29.663412+0000","last_undegraded":"2024-02-18T00:35:29.663412+0000","last_fullsized":"2024-02-18T00:35:29.663412+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.683442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.683442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":222,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654343+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:35:29.654343+0000","last_peered":"2024-02-18T00:35:29.654343+0000","last_clean":"2024-02-18T00:35:29.654343+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:35:29.654343+0000","last_undegraded":"2024-02-18T00:35:29.654343+0000","last_fullsized":"2024-02-18T00:35:29.654343+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":186,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647833+0000","last_change":"2024-02-18T00:35:25.953784+0000","last_active":"2024-02-18T00:35:29.647833+0000","last_peered":"2024-02-18T00:35:29.647833+0000","last_clean":"2024-02-18T00:35:29.647833+0000","last_became_active":"2024-02-18T00:34:27.633954+0000","last_became_peered":"2024-02-18T00:34:27.633954+0000","last_unstale":"2024-02-18T00:35:29.647833+0000","last_undegraded":"2024-02-18T00:35:29.647833+0000","last_fullsized":"2024-02-18T00:35:29.647833+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:25.953702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.953702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643463+0000","last_change":"2024-02-18T00:33:04.706011+0000","last_active":"2024-02-18T00:35:29.643463+0000","last_peered":"2024-02-18T00:35:29.643463+0000","last_clean":"2024-02-18T00:35:29.643463+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:35:29.643463+0000","last_undegraded":"2024-02-18T00:35:29.643463+0000","last_fullsized":"2024-02-18T00:35:29.643463+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":279,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.663262+0000","last_change":"2024-02-18T00:33:48.861148+0000","last_active":"2024-02-18T00:35:29.663262+0000","last_peered":"2024-02-18T00:35:29.663262+0000","last_clean":"2024-02-18T00:35:29.663262+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:35:29.663262+0000","last_undegraded":"2024-02-18T00:35:29.663262+0000","last_fullsized":"2024-02-18T00:35:29.663262+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:33:46.897445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.897445+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.653407+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:35:29.653407+0000","last_peered":"2024-02-18T00:35:29.653407+0000","last_clean":"2024-02-18T00:35:29.653407+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:35:29.653407+0000","last_undegraded":"2024-02-18T00:35:29.653407+0000","last_fullsized":"2024-02-18T00:35:29.653407+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644109+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:35:29.644109+0000","last_peered":"2024-02-18T00:35:29.644109+0000","last_clean":"2024-02-18T00:35:29.644109+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:35:29.644109+0000","last_undegraded":"2024-02-18T00:35:29.644109+0000","last_fullsized":"2024-02-18T00:35:29.644109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654223+0000","last_change":"2024-02-18T00:33:55.265441+0000","last_active":"2024-02-18T00:35:29.654223+0000","last_peered":"2024-02-18T00:35:29.654223+0000","last_clean":"2024-02-18T00:35:29.654223+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:35:29.654223+0000","last_undegraded":"2024-02-18T00:35:29.654223+0000","last_fullsized":"2024-02-18T00:35:29.654223+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.265373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.265373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"175'14","reported_seq":186,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643609+0000","last_change":"2024-02-18T00:34:24.618811+0000","last_active":"2024-02-18T00:35:29.643609+0000","last_peered":"2024-02-18T00:35:29.643609+0000","last_clean":"2024-02-18T00:35:29.643609+0000","last_became_active":"2024-02-18T00:34:24.618468+0000","last_became_peered":"2024-02-18T00:34:24.618468+0000","last_unstale":"2024-02-18T00:35:29.643609+0000","last_undegraded":"2024-02-18T00:35:29.643609+0000","last_fullsized":"2024-02-18T00:35:29.643609+0000","mapping_epoch":106,"log_start":"175'14","ondisk_log_start":"175'14","created":106,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655244+0000","last_change":"2024-02-18T00:33:04.730258+0000","last_active":"2024-02-18T00:35:29.655244+0000","last_peered":"2024-02-18T00:35:29.655244+0000","last_clean":"2024-02-18T00:35:29.655244+0000","last_became_active":"2024-02-18T00:33:04.729050+0000","last_became_peered":"2024-02-18T00:33:04.729050+0000","last_unstale":"2024-02-18T00:35:29.655244+0000","last_undegraded":"2024-02-18T00:35:29.655244+0000","last_fullsized":"2024-02-18T00:35:29.655244+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":287,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646343+0000","last_change":"2024-02-18T00:34:30.666844+0000","last_active":"2024-02-18T00:35:29.646343+0000","last_peered":"2024-02-18T00:35:29.646343+0000","last_clean":"2024-02-18T00:35:29.646343+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:35:29.646343+0000","last_undegraded":"2024-02-18T00:35:29.646343+0000","last_fullsized":"2024-02-18T00:35:29.646343+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:34:30.666765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.666765+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":237,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643812+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:35:29.643812+0000","last_peered":"2024-02-18T00:35:29.643812+0000","last_clean":"2024-02-18T00:35:29.643812+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:35:29.643812+0000","last_undegraded":"2024-02-18T00:35:29.643812+0000","last_fullsized":"2024-02-18T00:35:29.643812+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":298,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643622+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:35:29.643622+0000","last_peered":"2024-02-18T00:35:29.643622+0000","last_clean":"2024-02-18T00:35:29.643622+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:35:29.643622+0000","last_undegraded":"2024-02-18T00:35:29.643622+0000","last_fullsized":"2024-02-18T00:35:29.643622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":221,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646553+0000","last_change":"2024-02-18T00:33:53.101420+0000","last_active":"2024-02-18T00:35:29.646553+0000","last_peered":"2024-02-18T00:35:29.646553+0000","last_clean":"2024-02-18T00:35:29.646553+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:35:29.646553+0000","last_undegraded":"2024-02-18T00:35:29.646553+0000","last_fullsized":"2024-02-18T00:35:29.646553+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.101251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.101251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":168,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651112+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:35:29.651112+0000","last_peered":"2024-02-18T00:35:29.651112+0000","last_clean":"2024-02-18T00:35:29.651112+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:35:29.651112+0000","last_undegraded":"2024-02-18T00:35:29.651112+0000","last_fullsized":"2024-02-18T00:35:29.651112+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":264,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643219+0000","last_change":"2024-02-18T00:33:49.276437+0000","last_active":"2024-02-18T00:35:29.643219+0000","last_peered":"2024-02-18T00:35:29.643219+0000","last_clean":"2024-02-18T00:35:29.643219+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:35:29.643219+0000","last_undegraded":"2024-02-18T00:35:29.643219+0000","last_fullsized":"2024-02-18T00:35:29.643219+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:33:40.095425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.095425+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":218,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647374+0000","last_change":"2024-02-18T00:34:15.935045+0000","last_active":"2024-02-18T00:35:29.647374+0000","last_peered":"2024-02-18T00:35:29.647374+0000","last_clean":"2024-02-18T00:35:29.647374+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:35:29.647374+0000","last_undegraded":"2024-02-18T00:35:29.647374+0000","last_fullsized":"2024-02-18T00:35:29.647374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.934970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"150'9","reported_seq":311,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646561+0000","last_change":"2024-02-18T00:34:27.741951+0000","last_active":"2024-02-18T00:35:29.646561+0000","last_peered":"2024-02-18T00:35:29.646561+0000","last_clean":"2024-02-18T00:35:29.646561+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:35:29.646561+0000","last_undegraded":"2024-02-18T00:35:29.646561+0000","last_fullsized":"2024-02-18T00:35:29.646561+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:27.741881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.741881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":208,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656098+0000","last_change":"2024-02-18T00:32:53.114523+0000","last_active":"2024-02-18T00:35:29.656098+0000","last_peered":"2024-02-18T00:35:29.656098+0000","last_clean":"2024-02-18T00:35:29.656098+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:35:29.656098+0000","last_undegraded":"2024-02-18T00:35:29.656098+0000","last_fullsized":"2024-02-18T00:35:29.656098+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.114424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.114424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":159,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645619+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:35:29.645619+0000","last_peered":"2024-02-18T00:35:29.645619+0000","last_clean":"2024-02-18T00:35:29.645619+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:35:29.645619+0000","last_undegraded":"2024-02-18T00:35:29.645619+0000","last_fullsized":"2024-02-18T00:35:29.645619+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":233,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.664805+0000","last_change":"2024-02-18T00:34:32.418786+0000","last_active":"2024-02-18T00:35:29.664805+0000","last_peered":"2024-02-18T00:35:29.664805+0000","last_clean":"2024-02-18T00:35:29.664805+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:35:29.664805+0000","last_undegraded":"2024-02-18T00:35:29.664805+0000","last_fullsized":"2024-02-18T00:35:29.664805+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:34:32.418717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.418717+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654503+0000","last_change":"2024-02-18T00:34:20.312165+0000","last_active":"2024-02-18T00:35:29.654503+0000","last_peered":"2024-02-18T00:35:29.654503+0000","last_clean":"2024-02-18T00:35:29.654503+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:35:29.654503+0000","last_undegraded":"2024-02-18T00:35:29.654503+0000","last_fullsized":"2024-02-18T00:35:29.654503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.312096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.312096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651694+0000","last_change":"2024-02-18T00:33:50.765435+0000","last_active":"2024-02-18T00:35:29.651694+0000","last_peered":"2024-02-18T00:35:29.651694+0000","last_clean":"2024-02-18T00:35:29.651694+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:35:29.651694+0000","last_undegraded":"2024-02-18T00:35:29.651694+0000","last_fullsized":"2024-02-18T00:35:29.651694+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.765368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.765368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":207,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646121+0000","last_change":"2024-02-18T00:33:01.075249+0000","last_active":"2024-02-18T00:35:29.646121+0000","last_peered":"2024-02-18T00:35:29.646121+0000","last_clean":"2024-02-18T00:35:29.646121+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:35:29.646121+0000","last_undegraded":"2024-02-18T00:35:29.646121+0000","last_fullsized":"2024-02-18T00:35:29.646121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.075169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.075169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646466+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:35:29.646466+0000","last_peered":"2024-02-18T00:35:29.646466+0000","last_clean":"2024-02-18T00:35:29.646466+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:35:29.646466+0000","last_undegraded":"2024-02-18T00:35:29.646466+0000","last_fullsized":"2024-02-18T00:35:29.646466+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":236,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648387+0000","last_change":"2024-02-18T00:34:20.362851+0000","last_active":"2024-02-18T00:35:29.648387+0000","last_peered":"2024-02-18T00:35:29.648387+0000","last_clean":"2024-02-18T00:35:29.648387+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:35:29.648387+0000","last_undegraded":"2024-02-18T00:35:29.648387+0000","last_fullsized":"2024-02-18T00:35:29.648387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:34:20.362782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.362782+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.662611+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:35:29.662611+0000","last_peered":"2024-02-18T00:35:29.662611+0000","last_clean":"2024-02-18T00:35:29.662611+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:35:29.662611+0000","last_undegraded":"2024-02-18T00:35:29.662611+0000","last_fullsized":"2024-02-18T00:35:29.662611+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.664889+0000","last_change":"2024-02-18T00:34:34.355483+0000","last_active":"2024-02-18T00:35:29.664889+0000","last_peered":"2024-02-18T00:35:29.664889+0000","last_clean":"2024-02-18T00:35:29.664889+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:35:29.664889+0000","last_undegraded":"2024-02-18T00:35:29.664889+0000","last_fullsized":"2024-02-18T00:35:29.664889+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.355416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.355416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646656+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:35:29.646656+0000","last_peered":"2024-02-18T00:35:29.646656+0000","last_clean":"2024-02-18T00:35:29.646656+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:35:29.646656+0000","last_undegraded":"2024-02-18T00:35:29.646656+0000","last_fullsized":"2024-02-18T00:35:29.646656+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":159,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654108+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:35:29.654108+0000","last_peered":"2024-02-18T00:35:29.654108+0000","last_clean":"2024-02-18T00:35:29.654108+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:35:29.654108+0000","last_undegraded":"2024-02-18T00:35:29.654108+0000","last_fullsized":"2024-02-18T00:35:29.654108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":223,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643617+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:35:29.643617+0000","last_peered":"2024-02-18T00:35:29.643617+0000","last_clean":"2024-02-18T00:35:29.643617+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:35:29.643617+0000","last_undegraded":"2024-02-18T00:35:29.643617+0000","last_fullsized":"2024-02-18T00:35:29.643617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644155+0000","last_change":"2024-02-18T00:33:39.051726+0000","last_active":"2024-02-18T00:35:29.644155+0000","last_peered":"2024-02-18T00:35:29.644155+0000","last_clean":"2024-02-18T00:35:29.644155+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:35:29.644155+0000","last_undegraded":"2024-02-18T00:35:29.644155+0000","last_fullsized":"2024-02-18T00:35:29.644155+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.051653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.051653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.653297+0000","last_change":"2024-02-18T00:33:56.296342+0000","last_active":"2024-02-18T00:35:29.653297+0000","last_peered":"2024-02-18T00:35:29.653297+0000","last_clean":"2024-02-18T00:35:29.653297+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:35:29.653297+0000","last_undegraded":"2024-02-18T00:35:29.653297+0000","last_fullsized":"2024-02-18T00:35:29.653297+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:56.296274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.296274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.671432+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:35:29.671432+0000","last_peered":"2024-02-18T00:35:29.671432+0000","last_clean":"2024-02-18T00:35:29.671432+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:35:29.671432+0000","last_undegraded":"2024-02-18T00:35:29.671432+0000","last_fullsized":"2024-02-18T00:35:29.671432+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647382+0000","last_change":"2024-02-18T00:33:48.714793+0000","last_active":"2024-02-18T00:35:29.647382+0000","last_peered":"2024-02-18T00:35:29.647382+0000","last_clean":"2024-02-18T00:35:29.647382+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:35:29.647382+0000","last_undegraded":"2024-02-18T00:35:29.647382+0000","last_fullsized":"2024-02-18T00:35:29.647382+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":274,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647661+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:35:29.647661+0000","last_peered":"2024-02-18T00:35:29.647661+0000","last_clean":"2024-02-18T00:35:29.647661+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:35:29.647661+0000","last_undegraded":"2024-02-18T00:35:29.647661+0000","last_fullsized":"2024-02-18T00:35:29.647661+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655992+0000","last_change":"2024-02-18T00:33:51.295157+0000","last_active":"2024-02-18T00:35:29.655992+0000","last_peered":"2024-02-18T00:35:29.655992+0000","last_clean":"2024-02-18T00:35:29.655992+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:35:29.655992+0000","last_undegraded":"2024-02-18T00:35:29.655992+0000","last_fullsized":"2024-02-18T00:35:29.655992+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.295093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.295093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647099+0000","last_change":"2024-02-18T00:34:23.374398+0000","last_active":"2024-02-18T00:35:29.647099+0000","last_peered":"2024-02-18T00:35:29.647099+0000","last_clean":"2024-02-18T00:35:29.647099+0000","last_became_active":"2024-02-18T00:33:48.741677+0000","last_became_peered":"2024-02-18T00:33:48.741677+0000","last_unstale":"2024-02-18T00:35:29.647099+0000","last_undegraded":"2024-02-18T00:35:29.647099+0000","last_fullsized":"2024-02-18T00:35:29.647099+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.374328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.374328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651866+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:35:29.651866+0000","last_peered":"2024-02-18T00:35:29.651866+0000","last_clean":"2024-02-18T00:35:29.651866+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:35:29.651866+0000","last_undegraded":"2024-02-18T00:35:29.651866+0000","last_fullsized":"2024-02-18T00:35:29.651866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"175'7","reported_seq":184,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.652015+0000","last_change":"2024-02-18T00:35:03.629983+0000","last_active":"2024-02-18T00:35:29.652015+0000","last_peered":"2024-02-18T00:35:29.652015+0000","last_clean":"2024-02-18T00:35:29.652015+0000","last_became_active":"2024-02-18T00:35:03.629630+0000","last_became_peered":"2024-02-18T00:35:03.629630+0000","last_unstale":"2024-02-18T00:35:29.652015+0000","last_undegraded":"2024-02-18T00:35:29.652015+0000","last_fullsized":"2024-02-18T00:35:29.652015+0000","mapping_epoch":106,"log_start":"175'7","ondisk_log_start":"175'7","created":106,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'7","last_scrub_stamp":"2024-02-18T00:34:32.218733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.218733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":114,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647396+0000","last_change":"2024-02-18T00:33:48.680037+0000","last_active":"2024-02-18T00:35:29.647396+0000","last_peered":"2024-02-18T00:35:29.647396+0000","last_clean":"2024-02-18T00:35:29.647396+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:35:29.647396+0000","last_undegraded":"2024-02-18T00:35:29.647396+0000","last_fullsized":"2024-02-18T00:35:29.647396+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":275,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646962+0000","last_change":"2024-02-18T00:33:56.121942+0000","last_active":"2024-02-18T00:35:29.646962+0000","last_peered":"2024-02-18T00:35:29.646962+0000","last_clean":"2024-02-18T00:35:29.646962+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:35:29.646962+0000","last_undegraded":"2024-02-18T00:35:29.646962+0000","last_fullsized":"2024-02-18T00:35:29.646962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.121872+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":223,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.642159+0000","last_change":"2024-02-18T00:33:48.634995+0000","last_active":"2024-02-18T00:35:29.642159+0000","last_peered":"2024-02-18T00:35:29.642159+0000","last_clean":"2024-02-18T00:35:29.642159+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:35:29.642159+0000","last_undegraded":"2024-02-18T00:35:29.642159+0000","last_fullsized":"2024-02-18T00:35:29.642159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:49.926996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.926996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":319,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656158+0000","last_change":"2024-02-18T00:34:18.318312+0000","last_active":"2024-02-18T00:35:29.656158+0000","last_peered":"2024-02-18T00:35:29.656158+0000","last_clean":"2024-02-18T00:35:29.656158+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:35:29.656158+0000","last_undegraded":"2024-02-18T00:35:29.656158+0000","last_fullsized":"2024-02-18T00:35:29.656158+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:18.318190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.318190+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":132,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647384+0000","last_change":"2024-02-18T00:34:20.995201+0000","last_active":"2024-02-18T00:35:29.647384+0000","last_peered":"2024-02-18T00:35:29.647384+0000","last_clean":"2024-02-18T00:35:29.647384+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:35:29.647384+0000","last_undegraded":"2024-02-18T00:35:29.647384+0000","last_fullsized":"2024-02-18T00:35:29.647384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:34:20.995125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.995125+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1","version":"175'9","reported_seq":179,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.649270+0000","last_change":"2024-02-18T00:34:59.611803+0000","last_active":"2024-02-18T00:35:29.649270+0000","last_peered":"2024-02-18T00:35:29.649270+0000","last_clean":"2024-02-18T00:35:29.649270+0000","last_became_active":"2024-02-18T00:34:59.611429+0000","last_became_peered":"2024-02-18T00:34:59.611429+0000","last_unstale":"2024-02-18T00:35:29.649270+0000","last_undegraded":"2024-02-18T00:35:29.649270+0000","last_fullsized":"2024-02-18T00:35:29.649270+0000","mapping_epoch":106,"log_start":"175'9","ondisk_log_start":"175'9","created":106,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644484+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:35:29.644484+0000","last_peered":"2024-02-18T00:35:29.644484+0000","last_clean":"2024-02-18T00:35:29.644484+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:35:29.644484+0000","last_undegraded":"2024-02-18T00:35:29.644484+0000","last_fullsized":"2024-02-18T00:35:29.644484+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":251,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656251+0000","last_change":"2024-02-18T00:33:52.250914+0000","last_active":"2024-02-18T00:35:29.656251+0000","last_peered":"2024-02-18T00:35:29.656251+0000","last_clean":"2024-02-18T00:35:29.656251+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:35:29.656251+0000","last_undegraded":"2024-02-18T00:35:29.656251+0000","last_fullsized":"2024-02-18T00:35:29.656251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:52.250848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.250848+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646944+0000","last_change":"2024-02-18T00:34:28.731011+0000","last_active":"2024-02-18T00:35:29.646944+0000","last_peered":"2024-02-18T00:35:29.646944+0000","last_clean":"2024-02-18T00:35:29.646944+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:35:29.646944+0000","last_undegraded":"2024-02-18T00:35:29.646944+0000","last_fullsized":"2024-02-18T00:35:29.646944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.730955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.730955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":218,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647918+0000","last_change":"2024-02-18T00:33:48.751672+0000","last_active":"2024-02-18T00:35:29.647918+0000","last_peered":"2024-02-18T00:35:29.647918+0000","last_clean":"2024-02-18T00:35:29.647918+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:35:29.647918+0000","last_undegraded":"2024-02-18T00:35:29.647918+0000","last_fullsized":"2024-02-18T00:35:29.647918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.025296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.025296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.662465+0000","last_change":"2024-02-18T00:32:59.800515+0000","last_active":"2024-02-18T00:35:29.662465+0000","last_peered":"2024-02-18T00:35:29.662465+0000","last_clean":"2024-02-18T00:35:29.662465+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:35:29.662465+0000","last_undegraded":"2024-02-18T00:35:29.662465+0000","last_fullsized":"2024-02-18T00:35:29.662465+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.800355+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":172,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.665636+0000","last_change":"2024-02-18T00:34:55.656536+0000","last_active":"2024-02-18T00:35:29.665636+0000","last_peered":"2024-02-18T00:35:29.665636+0000","last_clean":"2024-02-18T00:35:29.665636+0000","last_became_active":"2024-02-18T00:34:55.656245+0000","last_became_peered":"2024-02-18T00:34:55.656245+0000","last_unstale":"2024-02-18T00:35:29.665636+0000","last_undegraded":"2024-02-18T00:35:29.665636+0000","last_fullsized":"2024-02-18T00:35:29.665636+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":172,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.655117+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:35:29.655117+0000","last_peered":"2024-02-18T00:35:29.655117+0000","last_clean":"2024-02-18T00:35:29.655117+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:35:29.655117+0000","last_undegraded":"2024-02-18T00:35:29.655117+0000","last_fullsized":"2024-02-18T00:35:29.655117+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":247,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654822+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:35:29.654822+0000","last_peered":"2024-02-18T00:35:29.654822+0000","last_clean":"2024-02-18T00:35:29.654822+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:35:29.654822+0000","last_undegraded":"2024-02-18T00:35:29.654822+0000","last_fullsized":"2024-02-18T00:35:29.654822+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":233,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654689+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:35:29.654689+0000","last_peered":"2024-02-18T00:35:29.654689+0000","last_clean":"2024-02-18T00:35:29.654689+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:35:29.654689+0000","last_undegraded":"2024-02-18T00:35:29.654689+0000","last_fullsized":"2024-02-18T00:35:29.654689+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.663458+0000","last_change":"2024-02-18T00:34:18.357435+0000","last_active":"2024-02-18T00:35:29.663458+0000","last_peered":"2024-02-18T00:35:29.663458+0000","last_clean":"2024-02-18T00:35:29.663458+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:35:29.663458+0000","last_undegraded":"2024-02-18T00:35:29.663458+0000","last_fullsized":"2024-02-18T00:35:29.663458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.357369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.357369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651434+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:35:29.651434+0000","last_peered":"2024-02-18T00:35:29.651434+0000","last_clean":"2024-02-18T00:35:29.651434+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:35:29.651434+0000","last_undegraded":"2024-02-18T00:35:29.651434+0000","last_fullsized":"2024-02-18T00:35:29.651434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3","version":"175'14","reported_seq":175,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.659422+0000","last_change":"2024-02-18T00:35:26.560007+0000","last_active":"2024-02-18T00:35:29.659422+0000","last_peered":"2024-02-18T00:35:29.659422+0000","last_clean":"2024-02-18T00:35:29.659422+0000","last_became_active":"2024-02-18T00:35:23.664255+0000","last_became_peered":"2024-02-18T00:35:23.664255+0000","last_unstale":"2024-02-18T00:35:29.659422+0000","last_undegraded":"2024-02-18T00:35:29.659422+0000","last_fullsized":"2024-02-18T00:35:29.659422+0000","mapping_epoch":149,"log_start":"175'14","ondisk_log_start":"175'14","created":106,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'14","last_scrub_stamp":"2024-02-18T00:35:26.559929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.559929+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":512,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":159,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.652063+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:35:29.652063+0000","last_peered":"2024-02-18T00:35:29.652063+0000","last_clean":"2024-02-18T00:35:29.652063+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:35:29.652063+0000","last_undegraded":"2024-02-18T00:35:29.652063+0000","last_fullsized":"2024-02-18T00:35:29.652063+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":255,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644391+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:35:29.644391+0000","last_peered":"2024-02-18T00:35:29.644391+0000","last_clean":"2024-02-18T00:35:29.644391+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:35:29.644391+0000","last_undegraded":"2024-02-18T00:35:29.644391+0000","last_fullsized":"2024-02-18T00:35:29.644391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":239,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651614+0000","last_change":"2024-02-18T00:33:48.748311+0000","last_active":"2024-02-18T00:35:29.651614+0000","last_peered":"2024-02-18T00:35:29.651614+0000","last_clean":"2024-02-18T00:35:29.651614+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:35:29.651614+0000","last_undegraded":"2024-02-18T00:35:29.651614+0000","last_fullsized":"2024-02-18T00:35:29.651614+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:33:45.906516+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.906516+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.649655+0000","last_change":"2024-02-18T00:34:29.236028+0000","last_active":"2024-02-18T00:35:29.649655+0000","last_peered":"2024-02-18T00:35:29.649655+0000","last_clean":"2024-02-18T00:35:29.649655+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:35:29.649655+0000","last_undegraded":"2024-02-18T00:35:29.649655+0000","last_fullsized":"2024-02-18T00:35:29.649655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:29.235953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.235953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":208,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.663164+0000","last_change":"2024-02-18T00:33:38.914876+0000","last_active":"2024-02-18T00:35:29.663164+0000","last_peered":"2024-02-18T00:35:29.663164+0000","last_clean":"2024-02-18T00:35:29.663164+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:35:29.663164+0000","last_undegraded":"2024-02-18T00:35:29.663164+0000","last_fullsized":"2024-02-18T00:35:29.663164+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:38.914691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:38.914691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.4","version":"109'2","reported_seq":190,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654648+0000","last_change":"2024-02-18T00:35:19.638620+0000","last_active":"2024-02-18T00:35:29.654648+0000","last_peered":"2024-02-18T00:35:29.654648+0000","last_clean":"2024-02-18T00:35:29.654648+0000","last_became_active":"2024-02-18T00:35:19.638287+0000","last_became_peered":"2024-02-18T00:35:19.638287+0000","last_unstale":"2024-02-18T00:35:29.654648+0000","last_undegraded":"2024-02-18T00:35:29.654648+0000","last_fullsized":"2024-02-18T00:35:29.654648+0000","mapping_epoch":106,"log_start":"109'2","ondisk_log_start":"109'2","created":106,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.650984+0000","last_change":"2024-02-18T00:33:48.765117+0000","last_active":"2024-02-18T00:35:29.650984+0000","last_peered":"2024-02-18T00:35:29.650984+0000","last_clean":"2024-02-18T00:35:29.650984+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:35:29.650984+0000","last_undegraded":"2024-02-18T00:35:29.650984+0000","last_fullsized":"2024-02-18T00:35:29.650984+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":275,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648231+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:35:29.648231+0000","last_peered":"2024-02-18T00:35:29.648231+0000","last_clean":"2024-02-18T00:35:29.648231+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:35:29.648231+0000","last_undegraded":"2024-02-18T00:35:29.648231+0000","last_fullsized":"2024-02-18T00:35:29.648231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.645671+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:35:29.645671+0000","last_peered":"2024-02-18T00:35:29.645671+0000","last_clean":"2024-02-18T00:35:29.645671+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:35:29.645671+0000","last_undegraded":"2024-02-18T00:35:29.645671+0000","last_fullsized":"2024-02-18T00:35:29.645671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.644126+0000","last_change":"2024-02-18T00:34:25.961381+0000","last_active":"2024-02-18T00:35:29.644126+0000","last_peered":"2024-02-18T00:35:29.644126+0000","last_clean":"2024-02-18T00:35:29.644126+0000","last_became_active":"2024-02-18T00:33:48.736206+0000","last_became_peered":"2024-02-18T00:33:48.736206+0000","last_unstale":"2024-02-18T00:35:29.644126+0000","last_undegraded":"2024-02-18T00:35:29.644126+0000","last_fullsized":"2024-02-18T00:35:29.644126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.961303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.961303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":221,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.647290+0000","last_change":"2024-02-18T00:33:48.649527+0000","last_active":"2024-02-18T00:35:29.647290+0000","last_peered":"2024-02-18T00:35:29.647290+0000","last_clean":"2024-02-18T00:35:29.647290+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:35:29.647290+0000","last_undegraded":"2024-02-18T00:35:29.647290+0000","last_fullsized":"2024-02-18T00:35:29.647290+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.996913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.996913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"249'13","reported_seq":197,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.662315+0000","last_change":"2024-02-18T00:35:28.515044+0000","last_active":"2024-02-18T00:35:29.662315+0000","last_peered":"2024-02-18T00:35:29.662315+0000","last_clean":"2024-02-18T00:35:29.662315+0000","last_became_active":"2024-02-18T00:35:15.634956+0000","last_became_peered":"2024-02-18T00:35:15.634956+0000","last_unstale":"2024-02-18T00:35:29.662315+0000","last_undegraded":"2024-02-18T00:35:29.662315+0000","last_fullsized":"2024-02-18T00:35:29.662315+0000","mapping_epoch":106,"log_start":"175'11","ondisk_log_start":"175'11","created":106,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'13","last_scrub_stamp":"2024-02-18T00:35:28.514983+0000","last_deep_scrub":"249'13","last_deep_scrub_stamp":"2024-02-18T00:35:28.514983+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.514983+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.656245+0000","last_change":"2024-02-18T00:33:04.678891+0000","last_active":"2024-02-18T00:35:29.656245+0000","last_peered":"2024-02-18T00:35:29.656245+0000","last_clean":"2024-02-18T00:35:29.656245+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:35:29.656245+0000","last_undegraded":"2024-02-18T00:35:29.656245+0000","last_fullsized":"2024-02-18T00:35:29.656245+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":245,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648211+0000","last_change":"2024-02-18T00:34:19.325220+0000","last_active":"2024-02-18T00:35:29.648211+0000","last_peered":"2024-02-18T00:35:29.648211+0000","last_clean":"2024-02-18T00:35:29.648211+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:35:29.648211+0000","last_undegraded":"2024-02-18T00:35:29.648211+0000","last_fullsized":"2024-02-18T00:35:29.648211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:34:19.325130+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.325130+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":119,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.654037+0000","last_change":"2024-02-18T00:34:31.241231+0000","last_active":"2024-02-18T00:35:29.654037+0000","last_peered":"2024-02-18T00:35:29.654037+0000","last_clean":"2024-02-18T00:35:29.654037+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:35:29.654037+0000","last_undegraded":"2024-02-18T00:35:29.654037+0000","last_fullsized":"2024-02-18T00:35:29.654037+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.241146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":114,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648338+0000","last_change":"2024-02-18T00:33:48.733307+0000","last_active":"2024-02-18T00:35:29.648338+0000","last_peered":"2024-02-18T00:35:29.648338+0000","last_clean":"2024-02-18T00:35:29.648338+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:35:29.648338+0000","last_undegraded":"2024-02-18T00:35:29.648338+0000","last_fullsized":"2024-02-18T00:35:29.648338+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.801504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.801504+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643085+0000","last_change":"2024-02-18T00:33:54.013575+0000","last_active":"2024-02-18T00:35:29.643085+0000","last_peered":"2024-02-18T00:35:29.643085+0000","last_clean":"2024-02-18T00:35:29.643085+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:35:29.643085+0000","last_undegraded":"2024-02-18T00:35:29.643085+0000","last_fullsized":"2024-02-18T00:35:29.643085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:54.013467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.013467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"175'11","reported_seq":181,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.658852+0000","last_change":"2024-02-18T00:35:11.654395+0000","last_active":"2024-02-18T00:35:29.658852+0000","last_peered":"2024-02-18T00:35:29.658852+0000","last_clean":"2024-02-18T00:35:29.658852+0000","last_became_active":"2024-02-18T00:35:11.654076+0000","last_became_peered":"2024-02-18T00:35:11.654076+0000","last_unstale":"2024-02-18T00:35:29.658852+0000","last_undegraded":"2024-02-18T00:35:29.658852+0000","last_fullsized":"2024-02-18T00:35:29.658852+0000","mapping_epoch":149,"log_start":"175'11","ondisk_log_start":"175'11","created":106,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'11","last_scrub_stamp":"2024-02-18T00:34:35.243061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.243061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":163,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.649256+0000","last_change":"2024-02-18T00:33:48.714160+0000","last_active":"2024-02-18T00:35:29.649256+0000","last_peered":"2024-02-18T00:35:29.649256+0000","last_clean":"2024-02-18T00:35:29.649256+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:35:29.649256+0000","last_undegraded":"2024-02-18T00:35:29.649256+0000","last_fullsized":"2024-02-18T00:35:29.649256+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":253,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:30.243335+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:35:30.243335+0000","last_peered":"2024-02-18T00:35:30.243335+0000","last_clean":"2024-02-18T00:35:30.243335+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:35:30.243335+0000","last_undegraded":"2024-02-18T00:35:30.243335+0000","last_fullsized":"2024-02-18T00:35:30.243335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":219,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.643044+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:35:29.643044+0000","last_peered":"2024-02-18T00:35:29.643044+0000","last_clean":"2024-02-18T00:35:29.643044+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:35:29.643044+0000","last_undegraded":"2024-02-18T00:35:29.643044+0000","last_fullsized":"2024-02-18T00:35:29.643044+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":224,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.660656+0000","last_change":"2024-02-18T00:34:38.428939+0000","last_active":"2024-02-18T00:35:29.660656+0000","last_peered":"2024-02-18T00:35:29.660656+0000","last_clean":"2024-02-18T00:35:29.660656+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:35:29.660656+0000","last_undegraded":"2024-02-18T00:35:29.660656+0000","last_fullsized":"2024-02-18T00:35:29.660656+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.428855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.428855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":213,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648210+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:35:29.648210+0000","last_peered":"2024-02-18T00:35:29.648210+0000","last_clean":"2024-02-18T00:35:29.648210+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:35:29.648210+0000","last_undegraded":"2024-02-18T00:35:29.648210+0000","last_fullsized":"2024-02-18T00:35:29.648210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.7","version":"175'7","reported_seq":195,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.669751+0000","last_change":"2024-02-18T00:35:09.277995+0000","last_active":"2024-02-18T00:35:29.669751+0000","last_peered":"2024-02-18T00:35:29.669751+0000","last_clean":"2024-02-18T00:35:29.669751+0000","last_became_active":"2024-02-18T00:35:07.629918+0000","last_became_peered":"2024-02-18T00:35:07.629918+0000","last_unstale":"2024-02-18T00:35:29.669751+0000","last_undegraded":"2024-02-18T00:35:29.669751+0000","last_fullsized":"2024-02-18T00:35:29.669751+0000","mapping_epoch":106,"log_start":"175'7","ondisk_log_start":"175'7","created":106,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'7","last_scrub_stamp":"2024-02-18T00:35:09.277926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:09.277926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":154,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646803+0000","last_change":"2024-02-18T00:33:04.720986+0000","last_active":"2024-02-18T00:35:29.646803+0000","last_peered":"2024-02-18T00:35:29.646803+0000","last_clean":"2024-02-18T00:35:29.646803+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:35:29.646803+0000","last_undegraded":"2024-02-18T00:35:29.646803+0000","last_fullsized":"2024-02-18T00:35:29.646803+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":299,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.646868+0000","last_change":"2024-02-18T00:33:41.866669+0000","last_active":"2024-02-18T00:35:29.646868+0000","last_peered":"2024-02-18T00:35:29.646868+0000","last_clean":"2024-02-18T00:35:29.646868+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:35:29.646868+0000","last_undegraded":"2024-02-18T00:35:29.646868+0000","last_fullsized":"2024-02-18T00:35:29.646868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:33:41.866531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.866531+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":227,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.650411+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:35:29.650411+0000","last_peered":"2024-02-18T00:35:29.650411+0000","last_clean":"2024-02-18T00:35:29.650411+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:35:29.650411+0000","last_undegraded":"2024-02-18T00:35:29.650411+0000","last_fullsized":"2024-02-18T00:35:29.650411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":215,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.648213+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:35:29.648213+0000","last_peered":"2024-02-18T00:35:29.648213+0000","last_clean":"2024-02-18T00:35:29.648213+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:35:29.648213+0000","last_undegraded":"2024-02-18T00:35:29.648213+0000","last_fullsized":"2024-02-18T00:35:29.648213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":222,"reported_epoch":251,"state":"active+clean","last_fresh":"2024-02-18T00:35:29.651411+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:35:29.651411+0000","last_peered":"2024-02-18T00:35:29.651411+0000","last_clean":"2024-02-18T00:35:29.651411+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:35:29.651411+0000","last_undegraded":"2024-02-18T00:35:29.651411+0000","last_fullsized":"2024-02-18T00:35:29.651411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":166,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":26,"num_pg":11,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":6528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":33,"acting":33,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1437,"num_objects":140,"num_object_clones":25,"num_object_copies":420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":1,"num_read":91,"num_read_kb":1,"num_write":528,"num_write_kb":525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1716224,"data_stored":4344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1709,"ondisk_log_size":1709,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":628,"ondisk_log_size":628,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":367,"num_read_kb":362,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266767,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863462997,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313588,"kb_used_data":4072,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431564,"statfs":{"total":95995035648,"available":95673921536,"internally_reserved":0,"allocated":4169728,"data_stored":2250953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":2.7120000000000002,"15min":1.2250000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":13.935,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.1319999999999999,"15min":0.70399999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":10.433,"5min":10.433,"15min":10.433},"last":0.59199999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":2.944,"15min":1.3100000000000001},"min":{"1min":0.27900000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.2290000000000001,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":2.9129999999999998,"15min":1.2929999999999999},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.899,"5min":71.872,"15min":71.872},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.3520000000000001,"15min":0.78700000000000003},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":11.855,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":1.804,"15min":0.94399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.7769999999999999,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.64500000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":1.347,"15min":0.78300000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.1280000000000001,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":1.228,"15min":0.753},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":5.0069999999999997,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.625}]},{"osd":4,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.63100000000000001,"15min":0.47099999999999997},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.8169999999999999,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":1.091,"15min":0.626},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":5.085,"5min":35.18,"15min":35.18},"last":0.35499999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66300000000000003,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":5.2539999999999996,"5min":10.606,"15min":10.606},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":1.177,"15min":0.66500000000000004},"min":{"1min":0.33500000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.5899999999999999,"5min":35.119,"15min":35.119},"last":0.41899999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60699999999999998,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.441},{"interface":"front","average":{"1min":0.754,"5min":1.089,"15min":0.71499999999999997},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.3620000000000001,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.45200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313632,"kb_used_data":4112,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431520,"statfs":{"total":95995035648,"available":95673876480,"internally_reserved":0,"allocated":4210688,"data_stored":2255581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":1.2529999999999999,"15min":0.623},"min":{"1min":0.24299999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.79600000000000004,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.249},{"interface":"front","average":{"1min":0.33100000000000002,"5min":1.329,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59999999999999998,"5min":43.137,"15min":43.137},"last":0.26000000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.5369999999999999,"15min":0.76100000000000001},"min":{"1min":0.23000000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55300000000000005,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.97599999999999998,"15min":0.58599999999999997},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":17.596,"15min":17.596},"last":0.31}]},{"osd":3,"last update":"Sun Feb 18 00:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.363,"15min":0.76100000000000001},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":19.274000000000001,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":1.877,"5min":1.3620000000000001,"15min":0.78000000000000003},"min":{"1min":0.33700000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":22.989000000000001,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.35899999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":1.1499999999999999,"15min":0.78300000000000003},"min":{"1min":0.39300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88900000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.167,"15min":0.76400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77000000000000002,"5min":12.75,"15min":12.75},"last":0.88100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.099,"15min":0.76400000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":8.9019999999999992,"5min":12.611000000000001,"15min":12.611000000000001},"last":3.8759999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.099,"15min":0.77700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":4.9550000000000001,"5min":17.417000000000002,"15min":17.417000000000002},"last":4.1180000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":1.1970000000000001,"15min":0.81699999999999995},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":12.834,"15min":12.834},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":1.1499999999999999,"15min":0.80600000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90400000000000003,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.54100000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.151,"15min":0.80500000000000005},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92500000000000004,"5min":17.452000000000002,"15min":17.452000000000002},"last":1.109},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.141,"15min":0.81599999999999995},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81699999999999995,"5min":12.805,"15min":12.805},"last":1.5409999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313576,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431576,"statfs":{"total":95995035648,"available":95673933824,"internally_reserved":0,"allocated":4153344,"data_stored":2253957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":2.2639999999999998,"15min":1.155},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":2.524,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.5069999999999999,"15min":0.73599999999999999},"min":{"1min":0.222,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":2.331,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.30499999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":1.827,"15min":0.87},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.577,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":1.8560000000000001,"15min":0.874},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":2.153,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.28999999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":1.171,"15min":0.68799999999999994},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":3.5609999999999999,"5min":19.006,"15min":19.006},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":2.2170000000000001,"15min":1.0760000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.7430000000000001,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.38500000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":1.661,"15min":0.91700000000000004},"min":{"1min":0.32800000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.1850000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.6850000000000001,"15min":0.96699999999999997},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.5680000000000001,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.72699999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":1.617,"15min":0.93200000000000005},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.5,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.8169999999999999,"15min":0.96599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":3.4060000000000001,"5min":57.991,"15min":57.991},"last":0.54200000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":1.2270000000000001,"15min":0.79300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.2830000000000004,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.706,"15min":0.95199999999999996},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.5369999999999999,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":1.8700000000000001,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5940000000000001,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.8029999999999999,"15min":0.99299999999999999},"min":{"1min":0.45500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":3.0249999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.70499999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":94,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313744,"kb_used_data":4224,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431408,"statfs":{"total":95995035648,"available":95673761792,"internally_reserved":0,"allocated":4325376,"data_stored":2254652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.73099999999999998,"15min":0.495},"min":{"1min":0.216,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61899999999999999,"5min":12.144,"15min":12.144},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":1.0580000000000001,"15min":0.60799999999999998},"min":{"1min":0.246,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66500000000000004,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.27400000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.71999999999999997,"15min":0.5},"min":{"1min":0.23899999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":2.3370000000000002,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.82899999999999996,"15min":0.54700000000000004},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.77100000000000002,"5min":18.919,"15min":18.919},"last":0.33500000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.94399999999999995,"15min":0.62},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.3809999999999998,"5min":22.712,"15min":22.712},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.94299999999999995,"15min":0.63300000000000001},"min":{"1min":0.29099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.8529999999999998,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.29899999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.88300000000000001,"15min":0.68700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80500000000000005,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68300000000000005},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79000000000000004,"5min":10.016,"15min":10.016},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80100000000000005,"15min":0.65500000000000003},"min":{"1min":0.35199999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":2.98,"5min":6.766,"15min":6.766},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.113,"5min":0.85999999999999999,"15min":0.70699999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":10.114000000000001,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.79400000000000004,"15min":0.69999999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.196,"5min":6.806,"15min":6.806},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.91300000000000003,"15min":0.75},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.827,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.53500000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.81000000000000005,"15min":0.74099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.89800000000000002,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.622}]}]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":122,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309532,"kb_used_data":4112,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435620,"statfs":{"total":95995035648,"available":95678074880,"internally_reserved":0,"allocated":4210688,"data_stored":2249791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:34:37 2024","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":0.66100000000000003,"15min":0.48499999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":25.082999999999998,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.20899999999999999},{"interface":"front","average":{"1min":1.339,"5min":0.75600000000000001,"15min":0.52900000000000003},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":18.911000000000001,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.309}]},{"osd":1,"last update":"Sun Feb 18 00:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.60299999999999998,"15min":0.48599999999999999},"min":{"1min":0.27100000000000002,"5min":0.223,"15min":0.223},"max":{"1min":4.1550000000000002,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.23899999999999999},{"interface":"front","average":{"1min":1.649,"5min":1.2010000000000001,"15min":0.67200000000000004},"min":{"1min":0.25800000000000001,"5min":0.252,"15min":0.25},"max":{"1min":23.439,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.29599999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:34:37 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.0700000000000001,"15min":0.65200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":23.405000000000001,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.54800000000000004,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":5.3479999999999999,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.28100000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.88200000000000001,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":10.131,"15min":10.131},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.70199999999999996,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72899999999999998,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.70299999999999996,"15min":0.63900000000000001},"min":{"1min":0.35599999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.9670000000000001,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.018,"5min":0.81499999999999995,"15min":0.68100000000000005},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":4.2149999999999999,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.022,"15min":0.73899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.292,"5min":10.189,"15min":10.189},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.72999999999999998,"15min":0.66100000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79100000000000004,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.78100000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.92300000000000004,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80300000000000005,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.54300000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986270,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":317736,"kb_used_data":4120,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93427416,"statfs":{"total":95995035648,"available":95669673984,"internally_reserved":0,"allocated":4218880,"data_stored":2250865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":1.5629999999999999,"15min":0.92600000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65900000000000003,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":1.5089999999999999,"15min":0.88100000000000001},"min":{"1min":0.34100000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.33300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.81200000000000006,"15min":0.65500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67300000000000004,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.435,"5min":1.595,"15min":0.90700000000000003},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.78300000000000003,"15min":0.67300000000000004},"min":{"1min":0.373,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60299999999999998,"5min":11.691000000000001,"15min":11.691000000000001},"last":1.119},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.90200000000000002,"15min":0.68600000000000005},"min":{"1min":0.33100000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1459999999999999,"5min":17.77,"15min":17.77},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.69699999999999995,"15min":0.63600000000000001},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64200000000000002,"5min":7.218,"15min":7.218},"last":1.8939999999999999},{"interface":"front","average":{"1min":0.502,"5min":1.071,"15min":0.72299999999999998},"min":{"1min":0.372,"5min":0.309,"15min":0.309},"max":{"1min":0.65100000000000002,"5min":33.530999999999999,"15min":33.530999999999999},"last":1.349}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.439,"15min":0.38300000000000001},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38},{"interface":"front","average":{"1min":0.499,"5min":0.45900000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55500000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45400000000000001,"15min":0.40799999999999997},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60699999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.497,"15min":0.44900000000000001},"min":{"1min":0.307,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.629,"5min":0.751,"15min":0.751},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72399999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.44}]}]},{"osd":5,"up_from":39,"seq":167503724633,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313500,"kb_used_data":3976,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431652,"statfs":{"total":95995035648,"available":95674011648,"internally_reserved":0,"allocated":4071424,"data_stored":2250673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.55800000000000005,"15min":0.54700000000000004},"min":{"1min":0.29199999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":6.3150000000000004,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.68999999999999995,"15min":0.59799999999999998},"min":{"1min":0.36099999999999999,"5min":0.316,"15min":0.316},"max":{"1min":3.6819999999999999,"5min":12.744,"15min":12.744},"last":0.46899999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.67600000000000005,"15min":0.621},"min":{"1min":0.34000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.016,"5min":0.71899999999999997,"15min":0.61499999999999999},"min":{"1min":0.30099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.4829999999999997,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Feb 18 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.26200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":6.1680000000000001,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.66500000000000004,"15min":0.629},"min":{"1min":0.311,"5min":0.308,"15min":0.308},"max":{"1min":6.1879999999999997,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.57299999999999995}]},{"osd":3,"last update":"Sun Feb 18 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.69099999999999995,"15min":0.60499999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28399999999999997},"max":{"1min":5.9880000000000004,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.626,"15min":0.57999999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":8.8650000000000002,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.436}]},{"osd":4,"last update":"Sun Feb 18 00:34:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.54800000000000004,"15min":0.42099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.69199999999999995,"5min":11.818,"15min":11.818},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.48799999999999999,"15min":0.40799999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54900000000000004,"5min":4.274,"15min":4.274},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.64000000000000001,"15min":0.47799999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52200000000000002,"5min":18.605,"15min":18.605},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.52200000000000002,"15min":0.44},"min":{"1min":0.35599999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57699999999999996,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.45100000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.60099999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":2778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:35:36.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:35:36.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:35:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:34.479621+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:34.479680+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:34.762063+0000 mon.b (mon.1) 320 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.157383+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? 172.21.15.59:0/1451674738' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.495211+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:35.495291+0000 mon.a (mon.0) 1774 : cluster [DBG] osdmap e256: 8 total, 8 up, 7 in 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.495943+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.496400+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.617599+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.617697+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.617794+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]': finished 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:35.617862+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e257: 8 total, 8 up, 7 in 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.626029+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]: dispatch 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.627108+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]: dispatch 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:35.763212+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:36.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:35.852944+0000 mgr.y (mgr.24454) 280 : cluster [DBG] pgmap v373: 265 pgs: 16 creating+peering, 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:34.479621+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:34.479680+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:34.762063+0000 mon.b (mon.1) 320 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.157383+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? 172.21.15.59:0/1451674738' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.495211+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:35.495291+0000 mon.a (mon.0) 1774 : cluster [DBG] osdmap e256: 8 total, 8 up, 7 in 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.495943+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.496400+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.617599+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.617697+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:36.857 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.617794+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]': finished 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:35.617862+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e257: 8 total, 8 up, 7 in 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.626029+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]: dispatch 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.627108+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]: dispatch 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:35.763212+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:35.852944+0000 mgr.y (mgr.24454) 280 : cluster [DBG] pgmap v373: 265 pgs: 16 creating+peering, 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:34.479621+0000 osd.7 (osd.7) 99 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:34.479680+0000 osd.7 (osd.7) 100 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:34.762063+0000 mon.b (mon.1) 320 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.157383+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? 172.21.15.59:0/1451674738' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.495211+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:36.858 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:35.495291+0000 mon.a (mon.0) 1774 : cluster [DBG] osdmap e256: 8 total, 8 up, 7 in 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.495943+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.496400+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.617599+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.617697+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi059-82877-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.617794+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-27"}]': finished 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:35.617862+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e257: 8 total, 8 up, 7 in 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.626029+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]: dispatch 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.627108+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]: dispatch 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:35.763212+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:36.859 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:35.852944+0000 mgr.y (mgr.24454) 280 : cluster [DBG] pgmap v373: 265 pgs: 16 creating+peering, 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:35:36.898 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:36.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:36.902 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:37.002 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:37.034 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:37.056 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:37.059 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:37.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:37.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:37.166 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:35:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:35:37.166 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:35:37.166 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:35:37.190 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:37.192 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:37.215 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:37.219 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:37.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:37.321 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:37.373 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:37.376 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:37.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:37.477 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:37.506 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:37.527 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:37.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:37.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:37.661 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:36.615175+0000 mon.a (mon.0) 1781 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:37.662 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:36.625080+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]': finished 2024-02-18T00:35:37.662 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:36.625165+0000 mon.a (mon.0) 1783 : cluster [DBG] osdmap e258: 8 total, 8 up, 7 in 2024-02-18T00:35:37.662 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:36.764224+0000 mon.b (mon.1) 322 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:37.663 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:37.683 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:37.685 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:37.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:37.786 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:37.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:36.615175+0000 mon.a (mon.0) 1781 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:37.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:36.625080+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]': finished 2024-02-18T00:35:37.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:36.625165+0000 mon.a (mon.0) 1783 : cluster [DBG] osdmap e258: 8 total, 8 up, 7 in 2024-02-18T00:35:37.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:36.764224+0000 mon.b (mon.1) 322 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:37.814 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:36.615175+0000 mon.a (mon.0) 1781 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:37.814 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:36.625080+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-27", "mode": "writeback"}]': finished 2024-02-18T00:35:37.814 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:36.625165+0000 mon.a (mon.0) 1783 : cluster [DBG] osdmap e258: 8 total, 8 up, 7 in 2024-02-18T00:35:37.814 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:36.764224+0000 mon.b (mon.1) 322 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:37.816 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:38.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-18T00:35:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:38.017 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:38.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:38.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:38.146 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:38.344 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:38.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:38.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:38.449 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:38.481 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:38.502 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:38.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:38.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:38.605 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:38.644 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:37.627106+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e259: 8 total, 8 up, 7 in 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:37.632258+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:37.636391+0000 mon.b (mon.1) 323 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:37.639724+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:37.668034+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:37.668358+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:37.765052+0000 mon.b (mon.1) 324 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:37.853433+0000 mgr.y (mgr.24454) 281 : cluster [DBG] pgmap v376: 249 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:35:38.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.615442+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:38.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.615516+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:38.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.615572+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:38.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:38.615618+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e260: 8 total, 8 up, 7 in 2024-02-18T00:35:38.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.618443+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:38.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.618459+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:38.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.619082+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:37.627106+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e259: 8 total, 8 up, 7 in 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:37.632258+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:37.636391+0000 mon.b (mon.1) 323 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:37.639724+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:37.668034+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:37.668358+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:37.765052+0000 mon.b (mon.1) 324 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:37.853433+0000 mgr.y (mgr.24454) 281 : cluster [DBG] pgmap v376: 249 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.615442+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.615516+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.615572+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:38.615618+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e260: 8 total, 8 up, 7 in 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.618443+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.618459+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.619082+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:37.627106+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e259: 8 total, 8 up, 7 in 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:37.632258+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:37.636391+0000 mon.b (mon.1) 323 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:37.639724+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:37.668034+0000 mon.c (mon.2) 219 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:37.668358+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:37.765052+0000 mon.b (mon.1) 324 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:37.853433+0000 mgr.y (mgr.24454) 281 : cluster [DBG] pgmap v376: 249 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.615442+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.615516+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.615572+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:38.615618+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e260: 8 total, 8 up, 7 in 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.618443+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.618459+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]: dispatch 2024-02-18T00:35:39.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.619082+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]: dispatch 2024-02-18T00:35:39.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:35:39.193 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:39.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:39.196 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:39.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:40.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.618565+0000 mon.b (mon.1) 325 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:40.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.622478+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:40.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:38.766134+0000 mon.b (mon.1) 326 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:40.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:39.612900+0000 mon.a (mon.0) 1795 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:40.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.619368+0000 mon.a (mon.0) 1796 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.619440+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]': finished 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.619514+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:39.619568+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e261: 8 total, 8 up, 7 in 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.618565+0000 mon.b (mon.1) 325 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.622478+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:38.766134+0000 mon.b (mon.1) 326 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:39.612900+0000 mon.a (mon.0) 1795 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.619368+0000 mon.a (mon.0) 1796 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.619440+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]': finished 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.619514+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:40.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:39.619568+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e261: 8 total, 8 up, 7 in 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.618565+0000 mon.b (mon.1) 325 : audit [INF] from='client.? 172.21.15.59:0/360131685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.622478+0000 mon.a (mon.0) 1794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]: dispatch 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:38.766134+0000 mon.b (mon.1) 326 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:39.612900+0000 mon.a (mon.0) 1795 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.619368+0000 mon.a (mon.0) 1796 : audit [INF] from='client.? 172.21.15.59:0/4143483286' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-45"}]': finished 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.619440+0000 mon.a (mon.0) 1797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-27"}]': finished 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.619514+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi059-82877-36"}]': finished 2024-02-18T00:35:40.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:39.619568+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e261: 8 total, 8 up, 7 in 2024-02-18T00:35:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.644300+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.645525+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.645853+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.646372+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.646684+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.647060+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi059-82877-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.767184+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:39.853808+0000 mgr.y (mgr.24454) 282 : cluster [DBG] pgmap v379: 249 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:39.898346+0000 mon.a (mon.0) 1806 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.562034+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.59:0/2071244269' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.644300+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.645525+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.645853+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.646372+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.646684+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.647060+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi059-82877-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:41.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.767184+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:41.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:39.853808+0000 mgr.y (mgr.24454) 282 : cluster [DBG] pgmap v379: 249 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:41.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:39.898346+0000 mon.a (mon.0) 1806 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:41.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.562034+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.59:0/2071244269' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:41.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.644300+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.645525+0000 mon.a (mon.0) 1801 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.645853+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.646372+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.646684+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.647060+0000 mon.a (mon.0) 1805 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi059-82877-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.767184+0000 mon.b (mon.1) 327 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:39.853808+0000 mgr.y (mgr.24454) 282 : cluster [DBG] pgmap v379: 249 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 241 active+clean; 9.0 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:39.898346+0000 mon.a (mon.0) 1806 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:41.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.562034+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? 172.21.15.59:0/2071244269' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:35:41.232 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:41.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:41.235 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:41.336 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:41.366 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:41.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:35:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:35:41.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:35:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:35:41.565 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:41.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:41.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:41.672 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:41.701 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:41.721 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:41.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:41.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 -- 172.21.15.59:0/625825050 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 msgr2=0x7f204c0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 --2- 172.21.15.59:0/625825050 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c0fdf90 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 rx=0x7f203c0047b0 tx=0x7f203c009cb0).stop 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 -- 172.21.15.59:0/625825050 shutdown_connections 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 --2- 172.21.15.59:0/625825050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0f8420 0x7f204c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 --2- 172.21.15.59:0/625825050 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c0fdf90 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 --2- 172.21.15.59:0/625825050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f204c0fe510 0x7f204c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 -- 172.21.15.59:0/625825050 >> 172.21.15.59:0/625825050 conn(0x7f204c0f3800 msgr2=0x7f204c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 -- 172.21.15.59:0/625825050 shutdown_connections 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 -- 172.21.15.59:0/625825050 wait complete. 2024-02-18T00:35:41.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 librados: starting msgr at 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.370+0000 7f20522d8700 1 librados: starting objecter 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 Processor -- start 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 -- start start 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 librados: setting wanted keys 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 librados: calling monclient init 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f204c0f8420 0x7f204c110240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0fe510 0x7f204c10a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c10a8d0 con 0x7f204c0fe510 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c10aa40 con 0x7f204c0f8420 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f20522d8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c10ac00 con 0x7f204c0fdbb0 2024-02-18T00:35:41.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f204b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0fe510 0x7f204c10a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0fe510 0x7f204c10a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48424/0 (socket says 172.21.15.59:48424) 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 -- 172.21.15.59:0/2071244269 learned_addr learned my addr 172.21.15.59:0/2071244269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f204b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:54222/0 (socket says 172.21.15.59:54222) 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f204bfff700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f204c0f8420 0x7f204c110240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 -- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 msgr2=0x7f204c110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 -- 172.21.15.59:0/2071244269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f204c0f8420 msgr2=0x7f204c110240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f204c0f8420 0x7f204c110240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f2050875700 1 -- 172.21.15.59:0/2071244269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20440097c0 con 0x7f204c0fe510 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.371+0000 7f204b7fe700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f2050875700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0fe510 0x7f204c10a3e0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 rx=0x7f2044001df0 tx=0x7f2044004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:41.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f2049775700 10 client.?.objecter ms_handle_connect 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f2049775700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f2049775700 1 -- 172.21.15.59:0/2071244269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044009a10 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c005040 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f2049775700 1 -- 172.21.15.59:0/2071244269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20440090d0 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f2049775700 1 -- 172.21.15.59:0/2071244269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044007d30 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f20522d8700 10 client.30906.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c10b240 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.372+0000 7f20522d8700 1 librados: init done 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.373+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c05ee80 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.373+0000 7f2049775700 1 -- 172.21.15.59:0/2071244269 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f204400fb30 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f2049775700 1 --2- 172.21.15.59:0/2071244269 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2034062c90 0x7f2034065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f2049775700 1 -- 172.21.15.59:0/2071244269 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(261..261 src has 44..261) v4 ==== 8217+0+0 (secure 0 0 0) 0x7f204407db10 con 0x7f204c0fe510 2024-02-18T00:35:41.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f2049775700 10 client.30906.objecter ms_dispatch 0x7f204c001550 osd_map(261..261 src has 44..261) v4 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f2049775700 3 client.30906.objecter handle_osd_map got epochs [261,261] > 0 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f2049775700 3 client.30906.objecter handle_osd_map decoding full epoch 261 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f204bfff700 1 --2- 172.21.15.59:0/2071244269 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2034062c90 0x7f2034065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f2049775700 20 client.30906.objecter dump_active .. 0 homeless 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f2049775700 10 client.30906.objecter ms_handle_connect 0x7f2034062c90 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.374+0000 7f204bfff700 1 --2- 172.21.15.59:0/2071244269 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2034062c90 0x7f2034065150 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 rx=0x7f203c005010 tx=0x7f203c007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.378+0000 7f2049775700 1 -- 172.21.15.59:0/2071244269 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2044050f40 con 0x7f204c0fe510 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.560+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f204c107110 con 0x7f204c0fe510 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.643+0000 7f2049775700 1 -- 172.21.15.59:0/2071244269 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v262) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2044050b30 con 0x7f204c0fe510 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 10 librados: watch_flush enter 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 10 librados: watch_flush exit 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 20 client.30906.objecter shutdown clearing up homeless session... 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 10 client.30906.objecter successfully canceled tick 2024-02-18T00:35:41.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2034062c90 msgr2=0x7f2034065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 --2- 172.21.15.59:0/2071244269 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2034062c90 0x7f2034065150 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 rx=0x7f203c005010 tx=0x7f203c007f90).stop 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0fe510 msgr2=0x7f204c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0fe510 0x7f204c10a3e0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 rx=0x7f2044001df0 tx=0x7f2044004d10).stop 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 shutdown_connections 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 --2- 172.21.15.59:0/2071244269 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2034062c90 0x7f2034065150 unknown :-1 s=CLOSED pgs=3037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f204c0f8420 0x7f204c110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f204c0fe510 0x7f204c10a3e0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 --2- 172.21.15.59:0/2071244269 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f204c0fdbb0 0x7f204c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 >> 172.21.15.59:0/2071244269 conn(0x7f204c0f3800 msgr2=0x7f204c103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 shutdown_connections 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 -- 172.21.15.59:0/2071244269 wait complete. 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:40.648+0000 7f20522d8700 1 librados: shutdown 2024-02-18T00:35:41.738 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:35:41.825 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:41.854 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:41.874 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:41.876 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:41.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:41.904 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:41.978 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:42.006 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:42.025 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:42.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:42.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:42.129 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:42.156 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:42.175 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:42.178 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:42.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:42.251 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:42] "GET /metrics HTTP/1.1" 200 190681 "" "Prometheus/2.33.4" 2024-02-18T00:35:42.279 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:42.306 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:42.324 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:42.327 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:42.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:42.428 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:42.455 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:42.473 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:42.476 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:42.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:42.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:42.607 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:42.681 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:42.691 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:42.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:42.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:40.249108+0000 osd.6 (osd.6) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:42.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:40.249292+0000 osd.6 (osd.6) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:42.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:40.631357+0000 mon.a (mon.0) 1808 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:42.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.643750+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.643842+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi059-82877-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.643952+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? 172.21.15.59:0/2071244269' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:40.644001+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e262: 8 total, 8 up, 7 in 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.645439+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi059-82877-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.645841+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.653696+0000 mon.a (mon.0) 1815 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:40.768410+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.087081+0000 mgr.y (mgr.24454) 283 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:42.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.098200+0000 mon.a (mon.0) 1816 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-18T00:35:42.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:40.249108+0000 osd.6 (osd.6) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:42.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:40.249292+0000 osd.6 (osd.6) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:42.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:40.631357+0000 mon.a (mon.0) 1808 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:42.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.643750+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:42.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.643842+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi059-82877-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:42.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.643952+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? 172.21.15.59:0/2071244269' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:35:42.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:40.644001+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e262: 8 total, 8 up, 7 in 2024-02-18T00:35:42.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.645439+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi059-82877-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:42.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.645841+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:42.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.653696+0000 mon.a (mon.0) 1815 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-02-18T00:35:42.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:40.768410+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:42.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.087081+0000 mgr.y (mgr.24454) 283 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:42.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.098200+0000 mon.a (mon.0) 1816 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-18T00:35:42.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.098426+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 0]}]: dispatch 2024-02-18T00:35:42.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.098600+0000 mon.a (mon.0) 1818 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [0, 3]}]: dispatch 2024-02-18T00:35:42.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.098778+0000 mon.a (mon.0) 1819 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1b", "id": [0, 6]}]: dispatch 2024-02-18T00:35:42.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.098945+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [3, 1]}]: dispatch 2024-02-18T00:35:42.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.099109+0000 mon.a (mon.0) 1821 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.9", "id": [0, 4]}]: dispatch 2024-02-18T00:35:42.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.099280+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.11", "id": [0, 2]}]: dispatch 2024-02-18T00:35:42.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.098426+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 0]}]: dispatch 2024-02-18T00:35:42.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.098600+0000 mon.a (mon.0) 1818 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [0, 3]}]: dispatch 2024-02-18T00:35:42.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.098778+0000 mon.a (mon.0) 1819 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1b", "id": [0, 6]}]: dispatch 2024-02-18T00:35:42.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.098945+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [3, 1]}]: dispatch 2024-02-18T00:35:42.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.099109+0000 mon.a (mon.0) 1821 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.9", "id": [0, 4]}]: dispatch 2024-02-18T00:35:42.757 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.099280+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.11", "id": [0, 2]}]: dispatch 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:40.249108+0000 osd.6 (osd.6) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:40.249292+0000 osd.6 (osd.6) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:40.631357+0000 mon.a (mon.0) 1808 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.643750+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi059-82882-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.643842+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi059-82877-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.643952+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? 172.21.15.59:0/2071244269' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:40.644001+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e262: 8 total, 8 up, 7 in 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.645439+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi059-82877-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:42.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.645841+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.653696+0000 mon.a (mon.0) 1815 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:40.768410+0000 mon.b (mon.1) 328 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.087081+0000 mgr.y (mgr.24454) 283 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.098200+0000 mon.a (mon.0) 1816 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.098426+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 0]}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.098600+0000 mon.a (mon.0) 1818 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [0, 3]}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.098778+0000 mon.a (mon.0) 1819 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1b", "id": [0, 6]}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.098945+0000 mon.a (mon.0) 1820 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [3, 1]}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.099109+0000 mon.a (mon.0) 1821 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.9", "id": [0, 4]}]: dispatch 2024-02-18T00:35:42.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.099280+0000 mon.a (mon.0) 1822 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.11", "id": [0, 2]}]: dispatch 2024-02-18T00:35:42.792 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:42.817 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:42.842 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:42.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:42.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:42.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:43.318 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.780+0000 7fb3ba495700 1 -- 172.21.15.59:0/1465718027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f8d30 msgr2=0x7fb3b40f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.780+0000 7fb3ba495700 1 --2- 172.21.15.59:0/1465718027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f8d30 0x7fb3b40f9110 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 rx=0x7fb3a40047b0 tx=0x7fb3a4009cb0).stop 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 -- 172.21.15.59:0/1465718027 shutdown_connections 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 --2- 172.21.15.59:0/1465718027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b4102290 0x7fb3b4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 --2- 172.21.15.59:0/1465718027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f8d30 0x7fb3b40f9110 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 --2- 172.21.15.59:0/1465718027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3b40f9600 0x7fb3b4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 -- 172.21.15.59:0/1465718027 >> 172.21.15.59:0/1465718027 conn(0x7fb3b40f2820 msgr2=0x7fb3b40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 -- 172.21.15.59:0/1465718027 shutdown_connections 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 -- 172.21.15.59:0/1465718027 wait complete. 2024-02-18T00:35:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 librados: starting msgr at 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 librados: starting objecter 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 Processor -- start 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 -- start start 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 librados: setting wanted keys 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.781+0000 7fb3ba495700 1 librados: calling monclient init 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b40f8d30 0x7fb3b40f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f9600 0x7fb3b40f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3b4102290 0x7fb3b40f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4116710 con 0x7fb3b40f9600 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4116880 con 0x7fb3b40f8d30 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4116a40 con 0x7fb3b4102290 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f9600 0x7fb3b40f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f9600 0x7fb3b40f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48444/0 (socket says 172.21.15.59:48444) 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 -- 172.21.15.59:0/3804455069 learned_addr learned my addr 172.21.15.59:0/3804455069 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b8a32700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3b4102290 0x7fb3b40f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b3fff700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b40f8d30 0x7fb3b40f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 -- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3b4102290 msgr2=0x7fb3b40f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3b4102290 0x7fb3b40f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 -- 172.21.15.59:0/3804455069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b40f8d30 msgr2=0x7fb3b40f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b40f8d30 0x7fb3b40f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 -- 172.21.15.59:0/3804455069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a4005040 con 0x7fb3b40f9600 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b37fe700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f9600 0x7fb3b40f71d0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 rx=0x7fb3a800e970 tx=0x7fb3a800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b17fa700 10 client.?.objecter ms_handle_connect 0x7fb3b40f9600 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b17fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b17fa700 1 -- 172.21.15.59:0/3804455069 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80046e0 con 0x7fb3b40f9600 2024-02-18T00:35:43.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b4116d20 con 0x7fb3b40f9600 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 10 client.30912.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b4117200 con 0x7fb3b40f9600 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3ba495700 1 librados: init done 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b17fa700 1 -- 172.21.15.59:0/3804455069 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a800f040 con 0x7fb3b40f9600 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.782+0000 7fb3b17fa700 1 -- 172.21.15.59:0/3804455069 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80146d0 con 0x7fb3b40f9600 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3b17fa700 1 -- 172.21.15.59:0/3804455069 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb3a80090d0 con 0x7fb3b40f9600 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a0004f50 con 0x7fb3b40f9600 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3b17fa700 1 --2- 172.21.15.59:0/3804455069 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb39c062c90 0x7fb39c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3b3fff700 1 --2- 172.21.15.59:0/3804455069 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb39c062c90 0x7fb39c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3b17fa700 1 -- 172.21.15.59:0/3804455069 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(264..264 src has 44..264) v4 ==== 10061+0+0 (secure 0 0 0) 0x7fb3a807d400 con 0x7fb3b40f9600 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3b17fa700 10 client.30912.objecter ms_dispatch 0x7fb3b4001550 osd_map(264..264 src has 44..264) v4 2024-02-18T00:35:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3b17fa700 3 client.30912.objecter handle_osd_map got epochs [264,264] > 0 2024-02-18T00:35:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.784+0000 7fb3b17fa700 3 client.30912.objecter handle_osd_map decoding full epoch 264 2024-02-18T00:35:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.785+0000 7fb3b17fa700 20 client.30912.objecter dump_active .. 0 homeless 2024-02-18T00:35:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.785+0000 7fb3b17fa700 10 client.30912.objecter ms_handle_connect 0x7fb39c062c90 2024-02-18T00:35:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.785+0000 7fb3b3fff700 1 --2- 172.21.15.59:0/3804455069 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb39c062c90 0x7fb39c065150 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 rx=0x7fb3a4005010 tx=0x7fb3a4009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.789+0000 7fb3b17fa700 1 -- 172.21.15.59:0/3804455069 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb3a80501b0 con 0x7fb3b40f9600 2024-02-18T00:35:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.968+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3a0000bf0 con 0x7fb39c062c90 2024-02-18T00:35:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:42.991+0000 7fb3b17fa700 1 -- 172.21.15.59:0/3804455069 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+661647 (secure 0 0 0) 0x7fb3a0000bf0 con 0x7fb39c062c90 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.319+0000 7fb3ba495700 10 librados: watch_flush enter 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.319+0000 7fb3ba495700 10 librados: watch_flush exit 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 20 client.30912.objecter shutdown clearing up homeless session... 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 10 client.30912.objecter successfully canceled tick 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb39c062c90 msgr2=0x7fb39c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 --2- 172.21.15.59:0/3804455069 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb39c062c90 0x7fb39c065150 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 rx=0x7fb3a4005010 tx=0x7fb3a4009cb0).stop 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f9600 msgr2=0x7fb3b40f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f9600 0x7fb3b40f71d0 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 rx=0x7fb3a800e970 tx=0x7fb3a800b040).stop 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 shutdown_connections 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 --2- 172.21.15.59:0/3804455069 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb39c062c90 0x7fb39c065150 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b40f8d30 0x7fb3b40f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3b4102290 0x7fb3b40f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 --2- 172.21.15.59:0/3804455069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3b40f9600 0x7fb3b40f71d0 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:43.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 >> 172.21.15.59:0/3804455069 conn(0x7fb3b40f2820 msgr2=0x7fb3b40f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:43.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 shutdown_connections 2024-02-18T00:35:43.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 -- 172.21.15.59:0/3804455069 wait complete. 2024-02-18T00:35:43.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:43.320+0000 7fb3ba495700 1 librados: shutdown 2024-02-18T00:35:43.338 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:35:43.339 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:43.339 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:43.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:43.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:43.470 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:43.497 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:43.500 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:43.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:43.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:41.135177+0000 osd.2 (osd.2) 113 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:41.135399+0000 osd.2 (osd.2) 114 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:41.769405+0000 mon.b (mon.1) 329 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:41.854198+0000 mgr.y (mgr.24454) 284 : cluster [DBG] pgmap v382: 249 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.511898+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-02-18T00:35:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512003+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-18T00:35:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512054+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 0]}]': finished 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512120+0000 mon.a (mon.0) 1826 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [0, 3]}]': finished 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512218+0000 mon.a (mon.0) 1827 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1b", "id": [0, 6]}]': finished 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512302+0000 mon.a (mon.0) 1828 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [3, 1]}]': finished 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512392+0000 mon.a (mon.0) 1829 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.9", "id": [0, 4]}]': finished 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512477+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.11", "id": [0, 2]}]': finished 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:42.512567+0000 mon.a (mon.0) 1831 : cluster [DBG] osdmap e263: 8 total, 8 up, 7 in 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.512963+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:41.135177+0000 osd.2 (osd.2) 113 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:43.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:41.135399+0000 osd.2 (osd.2) 114 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:41.769405+0000 mon.b (mon.1) 329 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:41.854198+0000 mgr.y (mgr.24454) 284 : cluster [DBG] pgmap v382: 249 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.511898+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512003+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512054+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 0]}]': finished 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512120+0000 mon.a (mon.0) 1826 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [0, 3]}]': finished 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512218+0000 mon.a (mon.0) 1827 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1b", "id": [0, 6]}]': finished 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512302+0000 mon.a (mon.0) 1828 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [3, 1]}]': finished 2024-02-18T00:35:43.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512392+0000 mon.a (mon.0) 1829 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.9", "id": [0, 4]}]': finished 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512477+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.11", "id": [0, 2]}]': finished 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:42.512567+0000 mon.a (mon.0) 1831 : cluster [DBG] osdmap e263: 8 total, 8 up, 7 in 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.512963+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.521425+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.652580+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi059-82877-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.652661+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.652855+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:42.652928+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e264: 8 total, 8 up, 7 in 2024-02-18T00:35:43.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.659533+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:43.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.660682+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:43.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.770329+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:43.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:42.969632+0000 mgr.y (mgr.24454) 285 : audit [DBG] from='client.30912 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:35:43.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.521425+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:43.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.652580+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi059-82877-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:43.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.652661+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:43.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.652855+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:43.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:42.652928+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e264: 8 total, 8 up, 7 in 2024-02-18T00:35:43.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.659533+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:43.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.660682+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:43.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.770329+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:43.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:42.969632+0000 mgr.y (mgr.24454) 285 : audit [DBG] from='client.30912 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:35:43.903 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:43.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:43.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:41.135177+0000 osd.2 (osd.2) 113 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:41.135399+0000 osd.2 (osd.2) 114 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:41.769405+0000 mon.b (mon.1) 329 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:41.854198+0000 mgr.y (mgr.24454) 284 : cluster [DBG] pgmap v382: 249 pgs: 2 active+clean+snaptrim, 32 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.511898+0000 mon.a (mon.0) 1823 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-02-18T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512003+0000 mon.a (mon.0) 1824 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-02-18T00:35:44.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512054+0000 mon.a (mon.0) 1825 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 0]}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512120+0000 mon.a (mon.0) 1826 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.10", "id": [0, 3]}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512218+0000 mon.a (mon.0) 1827 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "5.1b", "id": [0, 6]}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512302+0000 mon.a (mon.0) 1828 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [3, 1]}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512392+0000 mon.a (mon.0) 1829 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.9", "id": [0, 4]}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512477+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.11", "id": [0, 2]}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:42.512567+0000 mon.a (mon.0) 1831 : cluster [DBG] osdmap e263: 8 total, 8 up, 7 in 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.512963+0000 mon.c (mon.2) 221 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.521425+0000 mon.a (mon.0) 1832 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.652580+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi059-82877-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.652661+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi059-82882-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.652855+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:42.652928+0000 mon.a (mon.0) 1836 : cluster [DBG] osdmap e264: 8 total, 8 up, 7 in 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.659533+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:44.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.660682+0000 mon.a (mon.0) 1837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:44.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.770329+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:44.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:42.969632+0000 mgr.y (mgr.24454) 285 : audit [DBG] from='client.30912 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:35:44.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:44.034 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:44.495 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:35:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:43.617675+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:43.617807+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e265: 8 total, 8 up, 7 in 2024-02-18T00:35:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:43.631245+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:43.639755+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:43.771396+0000 mon.b (mon.1) 331 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:45.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:43.854570+0000 mgr.y (mgr.24454) 286 : cluster [DBG] pgmap v385: 265 pgs: 2 active+clean+snaptrim, 48 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.179409+0000 mon.a (mon.0) 1841 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.420260+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:43.617675+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:43.617807+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e265: 8 total, 8 up, 7 in 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:43.631245+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:43.639755+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:43.771396+0000 mon.b (mon.1) 331 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:43.854570+0000 mgr.y (mgr.24454) 286 : cluster [DBG] pgmap v385: 265 pgs: 2 active+clean+snaptrim, 48 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.179409+0000 mon.a (mon.0) 1841 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:35:45.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.420260+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:43.617675+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:43.617807+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e265: 8 total, 8 up, 7 in 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:43.631245+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:43.639755+0000 mon.a (mon.0) 1840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:43.771396+0000 mon.b (mon.1) 331 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:43.854570+0000 mgr.y (mgr.24454) 286 : cluster [DBG] pgmap v385: 265 pgs: 2 active+clean+snaptrim, 48 unknown, 6 active+clean+snaptrim_wait, 209 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.179409+0000 mon.a (mon.0) 1841 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:35:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.420260+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:45.128 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-02-18T00:35:42.649854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9108,"num_objects":368,"num_object_clones":8,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":2,"num_read":1573,"num_read_kb":1202,"num_write":1294,"num_write_kb":653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":3765,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2484,"ondisk_log_size":2484,"up":651,"acting":651,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":779,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2179952,"kb_used_data":29696,"kb_used_omap":0,"kb_used_meta":2150144,"kb_avail":654036112,"statfs":{"total":671965249536,"available":669732978688,"internally_reserved":0,"allocated":30408704,"data_stored":16483071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2201747456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-104,"num_objects":-10,"num_object_clones":-10,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.037155"},"pg_stats":[{"pgid":"169.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"171.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"170.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:42.640373+0000","last_change":"2024-02-18T00:35:42.640373+0000","last_active":"2024-02-18T00:35:42.640373+0000","last_peered":"2024-02-18T00:35:42.640373+0000","last_clean":"2024-02-18T00:35:42.640373+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:42.640373+0000","last_undegraded":"2024-02-18T00:35:42.640373+0000","last_fullsized":"2024-02-18T00:35:42.640373+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:42.640373+0000","last_clean_scrub_stamp":"2024-02-18T00:35:42.640373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"169.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:35:41.637565+0000","last_change":"2024-02-18T00:35:41.637565+0000","last_active":"2024-02-18T00:35:41.637565+0000","last_peered":"2024-02-18T00:35:41.637565+0000","last_clean":"2024-02-18T00:35:41.637565+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:35:41.637565+0000","last_undegraded":"2024-02-18T00:35:41.637565+0000","last_fullsized":"2024-02-18T00:35:41.637565+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:41.637565+0000","last_clean_scrub_stamp":"2024-02-18T00:35:41.637565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":214,"reported_epoch":261,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:39.618203+0000","last_change":"2024-02-18T00:35:37.794347+0000","last_active":"2024-02-18T00:35:39.618203+0000","last_peered":"2024-02-18T00:35:39.618203+0000","last_clean":"2024-02-18T00:35:39.618203+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:35:39.618203+0000","last_undegraded":"2024-02-18T00:35:39.618203+0000","last_fullsized":"2024-02-18T00:35:39.618203+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"}]},{"pgid":"62.1","version":"0'0","reported_seq":188,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627055+0000","last_change":"2024-02-18T00:35:37.631634+0000","last_active":"2024-02-18T00:35:39.627055+0000","last_peered":"2024-02-18T00:35:39.627055+0000","last_clean":"2024-02-18T00:35:39.627055+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:35:39.627055+0000","last_undegraded":"2024-02-18T00:35:39.627055+0000","last_fullsized":"2024-02-18T00:35:39.627055+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.2","version":"231'68","reported_seq":250,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.615525+0000","last_change":"2024-02-18T00:35:37.650537+0000","last_active":"2024-02-18T00:35:38.615525+0000","last_peered":"2024-02-18T00:35:38.615525+0000","last_clean":"2024-02-18T00:35:38.615525+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:35:38.615525+0000","last_undegraded":"2024-02-18T00:35:38.615525+0000","last_fullsized":"2024-02-18T00:35:38.615525+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.3","version":"245'73","reported_seq":243,"reported_epoch":261,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:39.618452+0000","last_change":"2024-02-18T00:35:37.793336+0000","last_active":"2024-02-18T00:35:39.618452+0000","last_peered":"2024-02-18T00:35:39.618452+0000","last_clean":"2024-02-18T00:35:39.618452+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:35:39.618452+0000","last_undegraded":"2024-02-18T00:35:39.618452+0000","last_fullsized":"2024-02-18T00:35:39.618452+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"}]},{"pgid":"62.4","version":"230'120","reported_seq":289,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.620449+0000","last_change":"2024-02-18T00:35:37.643034+0000","last_active":"2024-02-18T00:35:38.620449+0000","last_peered":"2024-02-18T00:35:38.620449+0000","last_clean":"2024-02-18T00:35:38.620449+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:35:38.620449+0000","last_undegraded":"2024-02-18T00:35:38.620449+0000","last_fullsized":"2024-02-18T00:35:38.620449+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.5","version":"240'90","reported_seq":259,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218760+0000","last_change":"2024-02-18T00:35:37.633264+0000","last_active":"2024-02-18T00:35:39.218760+0000","last_peered":"2024-02-18T00:35:39.218760+0000","last_clean":"2024-02-18T00:35:39.218760+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:35:39.218760+0000","last_undegraded":"2024-02-18T00:35:39.218760+0000","last_fullsized":"2024-02-18T00:35:39.218760+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.6","version":"261'120","reported_seq":245,"reported_epoch":261,"state":"active+clean+snaptrim","last_fresh":"2024-02-18T00:35:39.796243+0000","last_change":"2024-02-18T00:35:35.790815+0000","last_active":"2024-02-18T00:35:39.796243+0000","last_peered":"2024-02-18T00:35:39.796243+0000","last_clean":"2024-02-18T00:35:39.796243+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:35:39.796243+0000","last_undegraded":"2024-02-18T00:35:39.796243+0000","last_fullsized":"2024-02-18T00:35:39.796243+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"stat_sum":{"num_bytes":96,"num_objects":10,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":5,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"255'71","reported_seq":231,"reported_epoch":261,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:39.619286+0000","last_change":"2024-02-18T00:35:35.790800+0000","last_active":"2024-02-18T00:35:39.619286+0000","last_peered":"2024-02-18T00:35:39.619286+0000","last_clean":"2024-02-18T00:35:39.619286+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:35:39.619286+0000","last_undegraded":"2024-02-18T00:35:39.619286+0000","last_fullsized":"2024-02-18T00:35:39.619286+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":71,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":52,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"}]},{"pgid":"62.8","version":"215'68","reported_seq":210,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.626981+0000","last_change":"2024-02-18T00:35:37.632402+0000","last_active":"2024-02-18T00:35:39.626981+0000","last_peered":"2024-02-18T00:35:39.626981+0000","last_clean":"2024-02-18T00:35:39.626981+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:35:39.626981+0000","last_undegraded":"2024-02-18T00:35:39.626981+0000","last_fullsized":"2024-02-18T00:35:39.626981+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.9","version":"229'51","reported_seq":226,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217918+0000","last_change":"2024-02-18T00:35:37.632112+0000","last_active":"2024-02-18T00:35:39.217918+0000","last_peered":"2024-02-18T00:35:39.217918+0000","last_clean":"2024-02-18T00:35:39.217918+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:35:39.217918+0000","last_undegraded":"2024-02-18T00:35:39.217918+0000","last_fullsized":"2024-02-18T00:35:39.217918+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.a","version":"255'37","reported_seq":184,"reported_epoch":261,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:39.621266+0000","last_change":"2024-02-18T00:35:37.793776+0000","last_active":"2024-02-18T00:35:39.621266+0000","last_peered":"2024-02-18T00:35:39.621266+0000","last_clean":"2024-02-18T00:35:39.621266+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:35:39.621266+0000","last_undegraded":"2024-02-18T00:35:39.621266+0000","last_fullsized":"2024-02-18T00:35:39.621266+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":37,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"stat_sum":{"num_bytes":35,"num_objects":4,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"}]},{"pgid":"62.b","version":"246'54","reported_seq":207,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.532601+0000","last_change":"2024-02-18T00:35:37.632982+0000","last_active":"2024-02-18T00:35:39.532601+0000","last_peered":"2024-02-18T00:35:39.532601+0000","last_clean":"2024-02-18T00:35:39.532601+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:35:39.532601+0000","last_undegraded":"2024-02-18T00:35:39.532601+0000","last_fullsized":"2024-02-18T00:35:39.532601+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.c","version":"213'17","reported_seq":198,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.530958+0000","last_change":"2024-02-18T00:35:37.634574+0000","last_active":"2024-02-18T00:35:39.530958+0000","last_peered":"2024-02-18T00:35:39.530958+0000","last_clean":"2024-02-18T00:35:39.530958+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:35:39.530958+0000","last_undegraded":"2024-02-18T00:35:39.530958+0000","last_fullsized":"2024-02-18T00:35:39.530958+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.d","version":"259'28","reported_seq":230,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.639119+0000","last_change":"2024-02-18T00:35:39.639403+0000","last_active":"2024-02-18T00:35:39.639119+0000","last_peered":"2024-02-18T00:35:39.639119+0000","last_clean":"2024-02-18T00:35:39.639119+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:35:39.639119+0000","last_undegraded":"2024-02-18T00:35:39.639119+0000","last_fullsized":"2024-02-18T00:35:39.639119+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.e","version":"259'73","reported_seq":241,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644999+0000","last_change":"2024-02-18T00:35:39.645326+0000","last_active":"2024-02-18T00:35:39.644999+0000","last_peered":"2024-02-18T00:35:39.644999+0000","last_clean":"2024-02-18T00:35:39.644999+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:35:39.644999+0000","last_undegraded":"2024-02-18T00:35:39.644999+0000","last_fullsized":"2024-02-18T00:35:39.644999+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.f","version":"229'68","reported_seq":231,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219317+0000","last_change":"2024-02-18T00:35:37.634002+0000","last_active":"2024-02-18T00:35:39.219317+0000","last_peered":"2024-02-18T00:35:39.219317+0000","last_clean":"2024-02-18T00:35:39.219317+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:35:39.219317+0000","last_undegraded":"2024-02-18T00:35:39.219317+0000","last_fullsized":"2024-02-18T00:35:39.219317+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.10","version":"209'56","reported_seq":245,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.626955+0000","last_change":"2024-02-18T00:35:37.632250+0000","last_active":"2024-02-18T00:35:39.626955+0000","last_peered":"2024-02-18T00:35:39.626955+0000","last_clean":"2024-02-18T00:35:39.626955+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:35:39.626955+0000","last_undegraded":"2024-02-18T00:35:39.626955+0000","last_fullsized":"2024-02-18T00:35:39.626955+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.11","version":"214'19","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.626939+0000","last_change":"2024-02-18T00:35:37.631390+0000","last_active":"2024-02-18T00:35:39.626939+0000","last_peered":"2024-02-18T00:35:39.626939+0000","last_clean":"2024-02-18T00:35:39.626939+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:35:39.626939+0000","last_undegraded":"2024-02-18T00:35:39.626939+0000","last_fullsized":"2024-02-18T00:35:39.626939+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.12","version":"257'68","reported_seq":230,"reported_epoch":261,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:39.621396+0000","last_change":"2024-02-18T00:35:37.792805+0000","last_active":"2024-02-18T00:35:39.621396+0000","last_peered":"2024-02-18T00:35:39.621396+0000","last_clean":"2024-02-18T00:35:39.621396+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:35:39.621396+0000","last_undegraded":"2024-02-18T00:35:39.621396+0000","last_fullsized":"2024-02-18T00:35:39.621396+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"}]},{"pgid":"62.13","version":"210'17","reported_seq":198,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.620302+0000","last_change":"2024-02-18T00:35:37.643050+0000","last_active":"2024-02-18T00:35:38.620302+0000","last_peered":"2024-02-18T00:35:38.620302+0000","last_clean":"2024-02-18T00:35:38.620302+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:35:38.620302+0000","last_undegraded":"2024-02-18T00:35:38.620302+0000","last_fullsized":"2024-02-18T00:35:38.620302+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.16","version":"225'22","reported_seq":206,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218996+0000","last_change":"2024-02-18T00:35:37.631924+0000","last_active":"2024-02-18T00:35:39.218996+0000","last_peered":"2024-02-18T00:35:39.218996+0000","last_clean":"2024-02-18T00:35:39.218996+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:35:39.218996+0000","last_undegraded":"2024-02-18T00:35:39.218996+0000","last_fullsized":"2024-02-18T00:35:39.218996+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.17","version":"244'18","reported_seq":214,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.617200+0000","last_change":"2024-02-18T00:35:37.652825+0000","last_active":"2024-02-18T00:35:38.617200+0000","last_peered":"2024-02-18T00:35:38.617200+0000","last_clean":"2024-02-18T00:35:38.617200+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:35:38.617200+0000","last_undegraded":"2024-02-18T00:35:38.617200+0000","last_fullsized":"2024-02-18T00:35:38.617200+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"62.15","version":"225'102","reported_seq":277,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533726+0000","last_change":"2024-02-18T00:35:37.633131+0000","last_active":"2024-02-18T00:35:39.533726+0000","last_peered":"2024-02-18T00:35:39.533726+0000","last_clean":"2024-02-18T00:35:39.533726+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:35:39.533726+0000","last_undegraded":"2024-02-18T00:35:39.533726+0000","last_fullsized":"2024-02-18T00:35:39.533726+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.29","version":"0'0","reported_seq":175,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.615384+0000","last_change":"2024-02-18T00:34:15.230999+0000","last_active":"2024-02-18T00:35:38.615384+0000","last_peered":"2024-02-18T00:35:38.615384+0000","last_clean":"2024-02-18T00:35:38.615384+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:35:38.615384+0000","last_undegraded":"2024-02-18T00:35:38.615384+0000","last_fullsized":"2024-02-18T00:35:38.615384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.230938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.230938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"261'106","reported_seq":270,"reported_epoch":261,"state":"active+clean+snaptrim","last_fresh":"2024-02-18T00:35:39.797422+0000","last_change":"2024-02-18T00:35:37.794536+0000","last_active":"2024-02-18T00:35:39.797422+0000","last_peered":"2024-02-18T00:35:39.797422+0000","last_clean":"2024-02-18T00:35:39.797422+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:35:39.797422+0000","last_undegraded":"2024-02-18T00:35:39.797422+0000","last_fullsized":"2024-02-18T00:35:39.797422+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":106,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":4,"stat_sum":{"num_bytes":106,"num_objects":11,"num_object_clones":3,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":175,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.530735+0000","last_change":"2024-02-18T00:34:39.758860+0000","last_active":"2024-02-18T00:35:39.530735+0000","last_peered":"2024-02-18T00:35:39.530735+0000","last_clean":"2024-02-18T00:35:39.530735+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:35:39.530735+0000","last_undegraded":"2024-02-18T00:35:39.530735+0000","last_fullsized":"2024-02-18T00:35:39.530735+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.758794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.758794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217999+0000","last_change":"2024-02-18T00:35:37.632238+0000","last_active":"2024-02-18T00:35:39.217999+0000","last_peered":"2024-02-18T00:35:39.217999+0000","last_clean":"2024-02-18T00:35:39.217999+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:35:39.217999+0000","last_undegraded":"2024-02-18T00:35:39.217999+0000","last_fullsized":"2024-02-18T00:35:39.217999+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.27","version":"0'0","reported_seq":131,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219223+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:35:39.219223+0000","last_peered":"2024-02-18T00:35:39.219223+0000","last_clean":"2024-02-18T00:35:39.219223+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:35:39.219223+0000","last_undegraded":"2024-02-18T00:35:39.219223+0000","last_fullsized":"2024-02-18T00:35:39.219223+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":277,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.615523+0000","last_change":"2024-02-18T00:35:37.650168+0000","last_active":"2024-02-18T00:35:38.615523+0000","last_peered":"2024-02-18T00:35:38.615523+0000","last_clean":"2024-02-18T00:35:38.615523+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:35:38.615523+0000","last_undegraded":"2024-02-18T00:35:38.615523+0000","last_fullsized":"2024-02-18T00:35:38.615523+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.26","version":"0'0","reported_seq":129,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219211+0000","last_change":"2024-02-18T00:34:26.276135+0000","last_active":"2024-02-18T00:35:39.219211+0000","last_peered":"2024-02-18T00:35:39.219211+0000","last_clean":"2024-02-18T00:35:39.219211+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:35:39.219211+0000","last_undegraded":"2024-02-18T00:35:39.219211+0000","last_fullsized":"2024-02-18T00:35:39.219211+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.276071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.276071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":254,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.615436+0000","last_change":"2024-02-18T00:35:37.648153+0000","last_active":"2024-02-18T00:35:38.615436+0000","last_peered":"2024-02-18T00:35:38.615436+0000","last_clean":"2024-02-18T00:35:38.615436+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:35:38.615436+0000","last_undegraded":"2024-02-18T00:35:38.615436+0000","last_fullsized":"2024-02-18T00:35:38.615436+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.25","version":"57'1","reported_seq":181,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219514+0000","last_change":"2024-02-18T00:33:43.346148+0000","last_active":"2024-02-18T00:35:39.219514+0000","last_peered":"2024-02-18T00:35:39.219514+0000","last_clean":"2024-02-18T00:35:39.219514+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:35:39.219514+0000","last_undegraded":"2024-02-18T00:35:39.219514+0000","last_fullsized":"2024-02-18T00:35:39.219514+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:43.346088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.346088+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"213'17","reported_seq":204,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.530600+0000","last_change":"2024-02-18T00:35:37.628605+0000","last_active":"2024-02-18T00:35:39.530600+0000","last_peered":"2024-02-18T00:35:39.530600+0000","last_clean":"2024-02-18T00:35:39.530600+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:35:39.530600+0000","last_undegraded":"2024-02-18T00:35:39.530600+0000","last_fullsized":"2024-02-18T00:35:39.530600+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.24","version":"0'0","reported_seq":175,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616876+0000","last_change":"2024-02-18T00:34:17.271924+0000","last_active":"2024-02-18T00:35:38.616876+0000","last_peered":"2024-02-18T00:35:38.616876+0000","last_clean":"2024-02-18T00:35:38.616876+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:35:38.616876+0000","last_undegraded":"2024-02-18T00:35:38.616876+0000","last_fullsized":"2024-02-18T00:35:38.616876+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.271867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.271867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":223,"reported_epoch":261,"state":"active+clean+snaptrim_wait","last_fresh":"2024-02-18T00:35:39.619159+0000","last_change":"2024-02-18T00:35:33.788573+0000","last_active":"2024-02-18T00:35:39.619159+0000","last_peered":"2024-02-18T00:35:39.619159+0000","last_clean":"2024-02-18T00:35:39.619159+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:35:39.619159+0000","last_undegraded":"2024-02-18T00:35:39.619159+0000","last_fullsized":"2024-02-18T00:35:39.619159+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1"}]},{"pgid":"2.23","version":"0'0","reported_seq":166,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.532723+0000","last_change":"2024-02-18T00:33:55.783013+0000","last_active":"2024-02-18T00:35:39.532723+0000","last_peered":"2024-02-18T00:35:39.532723+0000","last_clean":"2024-02-18T00:35:39.532723+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:35:39.532723+0000","last_undegraded":"2024-02-18T00:35:39.532723+0000","last_fullsized":"2024-02-18T00:35:39.532723+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.782955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.782955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":193,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616720+0000","last_change":"2024-02-18T00:35:37.652333+0000","last_active":"2024-02-18T00:35:38.616720+0000","last_peered":"2024-02-18T00:35:38.616720+0000","last_clean":"2024-02-18T00:35:38.616720+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:35:38.616720+0000","last_undegraded":"2024-02-18T00:35:38.616720+0000","last_fullsized":"2024-02-18T00:35:38.616720+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.22","version":"0'0","reported_seq":168,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616110+0000","last_change":"2024-02-18T00:33:49.344703+0000","last_active":"2024-02-18T00:35:38.616110+0000","last_peered":"2024-02-18T00:35:38.616110+0000","last_clean":"2024-02-18T00:35:38.616110+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:35:38.616110+0000","last_undegraded":"2024-02-18T00:35:38.616110+0000","last_fullsized":"2024-02-18T00:35:38.616110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.344637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.344637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":200,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643865+0000","last_change":"2024-02-18T00:35:37.659096+0000","last_active":"2024-02-18T00:35:39.643865+0000","last_peered":"2024-02-18T00:35:39.643865+0000","last_clean":"2024-02-18T00:35:39.643865+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:35:39.643865+0000","last_undegraded":"2024-02-18T00:35:39.643865+0000","last_fullsized":"2024-02-18T00:35:39.643865+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.21","version":"0'0","reported_seq":247,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.619133+0000","last_change":"2024-02-18T00:33:48.736504+0000","last_active":"2024-02-18T00:35:39.619133+0000","last_peered":"2024-02-18T00:35:39.619133+0000","last_clean":"2024-02-18T00:35:39.619133+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:35:39.619133+0000","last_undegraded":"2024-02-18T00:35:39.619133+0000","last_fullsized":"2024-02-18T00:35:39.619133+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.834878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.834878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"227'17","reported_seq":203,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616567+0000","last_change":"2024-02-18T00:35:37.652285+0000","last_active":"2024-02-18T00:35:38.616567+0000","last_peered":"2024-02-18T00:35:38.616567+0000","last_clean":"2024-02-18T00:35:38.616567+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:35:38.616567+0000","last_undegraded":"2024-02-18T00:35:38.616567+0000","last_fullsized":"2024-02-18T00:35:38.616567+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"7","length":"1"},{"start":"c","length":"1"},{"start":"e","length":"1"},{"start":"11","length":"1"}]},{"pgid":"2.20","version":"0'0","reported_seq":201,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.642936+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:35:39.642936+0000","last_peered":"2024-02-18T00:35:39.642936+0000","last_clean":"2024-02-18T00:35:39.642936+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:35:39.642936+0000","last_undegraded":"2024-02-18T00:35:39.642936+0000","last_fullsized":"2024-02-18T00:35:39.642936+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.620998+0000","last_change":"2024-02-18T00:33:48.727714+0000","last_active":"2024-02-18T00:35:38.620998+0000","last_peered":"2024-02-18T00:35:38.620998+0000","last_clean":"2024-02-18T00:35:38.620998+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:35:38.620998+0000","last_undegraded":"2024-02-18T00:35:38.620998+0000","last_fullsized":"2024-02-18T00:35:38.620998+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":174,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616545+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:35:38.616545+0000","last_peered":"2024-02-18T00:35:38.616545+0000","last_clean":"2024-02-18T00:35:38.616545+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:35:38.616545+0000","last_undegraded":"2024-02-18T00:35:38.616545+0000","last_fullsized":"2024-02-18T00:35:38.616545+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":311,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643049+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:35:39.643049+0000","last_peered":"2024-02-18T00:35:39.643049+0000","last_clean":"2024-02-18T00:35:39.643049+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:35:39.643049+0000","last_undegraded":"2024-02-18T00:35:39.643049+0000","last_fullsized":"2024-02-18T00:35:39.643049+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":240,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.620269+0000","last_change":"2024-02-18T00:34:26.060181+0000","last_active":"2024-02-18T00:35:39.620269+0000","last_peered":"2024-02-18T00:35:39.620269+0000","last_clean":"2024-02-18T00:35:39.620269+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:35:39.620269+0000","last_undegraded":"2024-02-18T00:35:39.620269+0000","last_fullsized":"2024-02-18T00:35:39.620269+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.060096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.060096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":223,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643409+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:35:39.643409+0000","last_peered":"2024-02-18T00:35:39.643409+0000","last_clean":"2024-02-18T00:35:39.643409+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:35:39.643409+0000","last_undegraded":"2024-02-18T00:35:39.643409+0000","last_fullsized":"2024-02-18T00:35:39.643409+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":169,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627876+0000","last_change":"2024-02-18T00:35:20.428964+0000","last_active":"2024-02-18T00:35:39.627876+0000","last_peered":"2024-02-18T00:35:39.627876+0000","last_clean":"2024-02-18T00:35:39.627876+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:35:39.627876+0000","last_undegraded":"2024-02-18T00:35:39.627876+0000","last_fullsized":"2024-02-18T00:35:39.627876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:20.428893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:20.428893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":251,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621054+0000","last_change":"2024-02-18T00:32:50.007673+0000","last_active":"2024-02-18T00:35:38.621054+0000","last_peered":"2024-02-18T00:35:38.621054+0000","last_clean":"2024-02-18T00:35:38.621054+0000","last_became_active":"2024-02-18T00:29:17.083599+0000","last_became_peered":"2024-02-18T00:29:17.083599+0000","last_unstale":"2024-02-18T00:35:38.621054+0000","last_undegraded":"2024-02-18T00:35:38.621054+0000","last_fullsized":"2024-02-18T00:35:38.621054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:32:50.007594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:32:50.007594+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":229,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628955+0000","last_change":"2024-02-18T00:34:24.334917+0000","last_active":"2024-02-18T00:35:39.628955+0000","last_peered":"2024-02-18T00:35:39.628955+0000","last_clean":"2024-02-18T00:35:39.628955+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:35:39.628955+0000","last_undegraded":"2024-02-18T00:35:39.628955+0000","last_fullsized":"2024-02-18T00:35:39.628955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.334814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.334814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621557+0000","last_change":"2024-02-18T00:33:42.768749+0000","last_active":"2024-02-18T00:35:39.621557+0000","last_peered":"2024-02-18T00:35:39.621557+0000","last_clean":"2024-02-18T00:35:39.621557+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:35:39.621557+0000","last_undegraded":"2024-02-18T00:35:39.621557+0000","last_fullsized":"2024-02-18T00:35:39.621557+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.768630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.768630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":230,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619674+0000","last_change":"2024-02-18T00:35:24.990649+0000","last_active":"2024-02-18T00:35:38.619674+0000","last_peered":"2024-02-18T00:35:38.619674+0000","last_clean":"2024-02-18T00:35:38.619674+0000","last_became_active":"2024-02-18T00:33:48.734414+0000","last_became_peered":"2024-02-18T00:33:48.734414+0000","last_unstale":"2024-02-18T00:35:38.619674+0000","last_undegraded":"2024-02-18T00:35:38.619674+0000","last_fullsized":"2024-02-18T00:35:38.619674+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.534079+0000","last_change":"2024-02-18T00:33:48.745303+0000","last_active":"2024-02-18T00:35:39.534079+0000","last_peered":"2024-02-18T00:35:39.534079+0000","last_clean":"2024-02-18T00:35:39.534079+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:35:39.534079+0000","last_undegraded":"2024-02-18T00:35:39.534079+0000","last_fullsized":"2024-02-18T00:35:39.534079+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":309,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616207+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:35:38.616207+0000","last_peered":"2024-02-18T00:35:38.616207+0000","last_clean":"2024-02-18T00:35:38.616207+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:35:38.616207+0000","last_undegraded":"2024-02-18T00:35:38.616207+0000","last_fullsized":"2024-02-18T00:35:38.616207+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":229,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621595+0000","last_change":"2024-02-18T00:33:46.861306+0000","last_active":"2024-02-18T00:35:39.621595+0000","last_peered":"2024-02-18T00:35:39.621595+0000","last_clean":"2024-02-18T00:35:39.621595+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:35:39.621595+0000","last_undegraded":"2024-02-18T00:35:39.621595+0000","last_fullsized":"2024-02-18T00:35:39.621595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.861233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.861233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.618737+0000","last_change":"2024-02-18T00:33:53.005012+0000","last_active":"2024-02-18T00:35:39.618737+0000","last_peered":"2024-02-18T00:35:39.618737+0000","last_clean":"2024-02-18T00:35:39.618737+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:35:39.618737+0000","last_undegraded":"2024-02-18T00:35:39.618737+0000","last_fullsized":"2024-02-18T00:35:39.618737+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.004903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.004903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":129,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.618780+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:35:39.618780+0000","last_peered":"2024-02-18T00:35:39.618780+0000","last_clean":"2024-02-18T00:35:39.618780+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:35:39.618780+0000","last_undegraded":"2024-02-18T00:35:39.618780+0000","last_fullsized":"2024-02-18T00:35:39.618780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643227+0000","last_change":"2024-02-18T00:33:02.523582+0000","last_active":"2024-02-18T00:35:39.643227+0000","last_peered":"2024-02-18T00:35:39.643227+0000","last_clean":"2024-02-18T00:35:39.643227+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:35:39.643227+0000","last_undegraded":"2024-02-18T00:35:39.643227+0000","last_fullsized":"2024-02-18T00:35:39.643227+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":259,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219387+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:35:39.219387+0000","last_peered":"2024-02-18T00:35:39.219387+0000","last_clean":"2024-02-18T00:35:39.219387+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:35:39.219387+0000","last_undegraded":"2024-02-18T00:35:39.219387+0000","last_fullsized":"2024-02-18T00:35:39.219387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218575+0000","last_change":"2024-02-18T00:34:18.301213+0000","last_active":"2024-02-18T00:35:39.218575+0000","last_peered":"2024-02-18T00:35:39.218575+0000","last_clean":"2024-02-18T00:35:39.218575+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:35:39.218575+0000","last_undegraded":"2024-02-18T00:35:39.218575+0000","last_fullsized":"2024-02-18T00:35:39.218575+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.301136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.301136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627775+0000","last_change":"2024-02-18T00:34:39.453471+0000","last_active":"2024-02-18T00:35:39.627775+0000","last_peered":"2024-02-18T00:35:39.627775+0000","last_clean":"2024-02-18T00:35:39.627775+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:35:39.627775+0000","last_undegraded":"2024-02-18T00:35:39.627775+0000","last_fullsized":"2024-02-18T00:35:39.627775+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.453312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.453312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":223,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.619624+0000","last_change":"2024-02-18T00:33:45.813625+0000","last_active":"2024-02-18T00:35:39.619624+0000","last_peered":"2024-02-18T00:35:39.619624+0000","last_clean":"2024-02-18T00:35:39.619624+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:35:39.619624+0000","last_undegraded":"2024-02-18T00:35:39.619624+0000","last_fullsized":"2024-02-18T00:35:39.619624+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.813514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.813514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619752+0000","last_change":"2024-02-18T00:33:48.733994+0000","last_active":"2024-02-18T00:35:38.619752+0000","last_peered":"2024-02-18T00:35:38.619752+0000","last_clean":"2024-02-18T00:35:38.619752+0000","last_became_active":"2024-02-18T00:33:48.732220+0000","last_became_peered":"2024-02-18T00:33:48.732220+0000","last_unstale":"2024-02-18T00:35:38.619752+0000","last_undegraded":"2024-02-18T00:35:38.619752+0000","last_fullsized":"2024-02-18T00:35:38.619752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":241,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218437+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:35:39.218437+0000","last_peered":"2024-02-18T00:35:39.218437+0000","last_clean":"2024-02-18T00:35:39.218437+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:35:39.218437+0000","last_undegraded":"2024-02-18T00:35:39.218437+0000","last_fullsized":"2024-02-18T00:35:39.218437+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":238,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628628+0000","last_change":"2024-02-18T00:34:23.295161+0000","last_active":"2024-02-18T00:35:39.628628+0000","last_peered":"2024-02-18T00:35:39.628628+0000","last_clean":"2024-02-18T00:35:39.628628+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:35:39.628628+0000","last_undegraded":"2024-02-18T00:35:39.628628+0000","last_fullsized":"2024-02-18T00:35:39.628628+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.295051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":326,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621239+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:35:39.621239+0000","last_peered":"2024-02-18T00:35:39.621239+0000","last_clean":"2024-02-18T00:35:39.621239+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:35:39.621239+0000","last_undegraded":"2024-02-18T00:35:39.621239+0000","last_fullsized":"2024-02-18T00:35:39.621239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219393+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:35:39.219393+0000","last_peered":"2024-02-18T00:35:39.219393+0000","last_clean":"2024-02-18T00:35:39.219393+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:35:39.219393+0000","last_undegraded":"2024-02-18T00:35:39.219393+0000","last_fullsized":"2024-02-18T00:35:39.219393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":169,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628990+0000","last_change":"2024-02-18T00:35:29.520701+0000","last_active":"2024-02-18T00:35:39.628990+0000","last_peered":"2024-02-18T00:35:39.628990+0000","last_clean":"2024-02-18T00:35:39.628990+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:35:39.628990+0000","last_undegraded":"2024-02-18T00:35:39.628990+0000","last_fullsized":"2024-02-18T00:35:39.628990+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.520632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.520632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":230,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644327+0000","last_change":"2024-02-18T00:33:44.007884+0000","last_active":"2024-02-18T00:35:39.644327+0000","last_peered":"2024-02-18T00:35:39.644327+0000","last_clean":"2024-02-18T00:35:39.644327+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:35:39.644327+0000","last_undegraded":"2024-02-18T00:35:39.644327+0000","last_fullsized":"2024-02-18T00:35:39.644327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:33:44.007804+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:33:44.007804+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":247,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627504+0000","last_change":"2024-02-18T00:33:48.713658+0000","last_active":"2024-02-18T00:35:39.627504+0000","last_peered":"2024-02-18T00:35:39.627504+0000","last_clean":"2024-02-18T00:35:39.627504+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:35:39.627504+0000","last_undegraded":"2024-02-18T00:35:39.627504+0000","last_fullsized":"2024-02-18T00:35:39.627504+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:33:47.199009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":309,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219392+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:35:39.219392+0000","last_peered":"2024-02-18T00:35:39.219392+0000","last_clean":"2024-02-18T00:35:39.219392+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:35:39.219392+0000","last_undegraded":"2024-02-18T00:35:39.219392+0000","last_fullsized":"2024-02-18T00:35:39.219392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":218,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644254+0000","last_change":"2024-02-18T00:33:41.057722+0000","last_active":"2024-02-18T00:35:39.644254+0000","last_peered":"2024-02-18T00:35:39.644254+0000","last_clean":"2024-02-18T00:35:39.644254+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:35:39.644254+0000","last_undegraded":"2024-02-18T00:35:39.644254+0000","last_fullsized":"2024-02-18T00:35:39.644254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.057659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.057659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218198+0000","last_change":"2024-02-18T00:33:02.533944+0000","last_active":"2024-02-18T00:35:39.218198+0000","last_peered":"2024-02-18T00:35:39.218198+0000","last_clean":"2024-02-18T00:35:39.218198+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:35:39.218198+0000","last_undegraded":"2024-02-18T00:35:39.218198+0000","last_fullsized":"2024-02-18T00:35:39.218198+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":260,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219196+0000","last_change":"2024-02-18T00:33:45.311343+0000","last_active":"2024-02-18T00:35:39.219196+0000","last_peered":"2024-02-18T00:35:39.219196+0000","last_clean":"2024-02-18T00:35:39.219196+0000","last_became_active":"2024-02-18T00:29:17.087715+0000","last_became_peered":"2024-02-18T00:29:17.087715+0000","last_unstale":"2024-02-18T00:35:39.219196+0000","last_undegraded":"2024-02-18T00:35:39.219196+0000","last_fullsized":"2024-02-18T00:35:39.219196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:45.311259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.311259+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":199,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619225+0000","last_change":"2024-02-18T00:34:27.927933+0000","last_active":"2024-02-18T00:35:38.619225+0000","last_peered":"2024-02-18T00:35:38.619225+0000","last_clean":"2024-02-18T00:35:38.619225+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:35:38.619225+0000","last_undegraded":"2024-02-18T00:35:38.619225+0000","last_fullsized":"2024-02-18T00:35:38.619225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.927871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.927871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644510+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:35:39.644510+0000","last_peered":"2024-02-18T00:35:39.644510+0000","last_clean":"2024-02-18T00:35:39.644510+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:35:39.644510+0000","last_undegraded":"2024-02-18T00:35:39.644510+0000","last_fullsized":"2024-02-18T00:35:39.644510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531228+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:35:39.531228+0000","last_peered":"2024-02-18T00:35:39.531228+0000","last_clean":"2024-02-18T00:35:39.531228+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:35:39.531228+0000","last_undegraded":"2024-02-18T00:35:39.531228+0000","last_fullsized":"2024-02-18T00:35:39.531228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":168,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218897+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:35:39.218897+0000","last_peered":"2024-02-18T00:35:39.218897+0000","last_clean":"2024-02-18T00:35:39.218897+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:35:39.218897+0000","last_undegraded":"2024-02-18T00:35:39.218897+0000","last_fullsized":"2024-02-18T00:35:39.218897+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":310,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531138+0000","last_change":"2024-02-18T00:33:54.753052+0000","last_active":"2024-02-18T00:35:39.531138+0000","last_peered":"2024-02-18T00:35:39.531138+0000","last_clean":"2024-02-18T00:35:39.531138+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:35:39.531138+0000","last_undegraded":"2024-02-18T00:35:39.531138+0000","last_fullsized":"2024-02-18T00:35:39.531138+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:33:54.752985+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.752985+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":262,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218347+0000","last_change":"2024-02-18T00:33:53.320429+0000","last_active":"2024-02-18T00:35:39.218347+0000","last_peered":"2024-02-18T00:35:39.218347+0000","last_clean":"2024-02-18T00:35:39.218347+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:35:39.218347+0000","last_undegraded":"2024-02-18T00:35:39.218347+0000","last_fullsized":"2024-02-18T00:35:39.218347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:33:53.320356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.320356+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.617261+0000","last_change":"2024-02-18T00:33:48.672342+0000","last_active":"2024-02-18T00:35:38.617261+0000","last_peered":"2024-02-18T00:35:38.617261+0000","last_clean":"2024-02-18T00:35:38.617261+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:35:38.617261+0000","last_undegraded":"2024-02-18T00:35:38.617261+0000","last_fullsized":"2024-02-18T00:35:38.617261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.368010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.368010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616270+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:35:38.616270+0000","last_peered":"2024-02-18T00:35:38.616270+0000","last_clean":"2024-02-18T00:35:38.616270+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:35:38.616270+0000","last_undegraded":"2024-02-18T00:35:38.616270+0000","last_fullsized":"2024-02-18T00:35:38.616270+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644012+0000","last_change":"2024-02-18T00:33:02.477932+0000","last_active":"2024-02-18T00:35:39.644012+0000","last_peered":"2024-02-18T00:35:39.644012+0000","last_clean":"2024-02-18T00:35:39.644012+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:35:39.644012+0000","last_undegraded":"2024-02-18T00:35:39.644012+0000","last_fullsized":"2024-02-18T00:35:39.644012+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":233,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218629+0000","last_change":"2024-02-18T00:33:50.287103+0000","last_active":"2024-02-18T00:35:39.218629+0000","last_peered":"2024-02-18T00:35:39.218629+0000","last_clean":"2024-02-18T00:35:39.218629+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:35:39.218629+0000","last_undegraded":"2024-02-18T00:35:39.218629+0000","last_fullsized":"2024-02-18T00:35:39.218629+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.287007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.287007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":329,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531046+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:35:39.531046+0000","last_peered":"2024-02-18T00:35:39.531046+0000","last_clean":"2024-02-18T00:35:39.531046+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:35:39.531046+0000","last_undegraded":"2024-02-18T00:35:39.531046+0000","last_fullsized":"2024-02-18T00:35:39.531046+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":238,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627956+0000","last_change":"2024-02-18T00:33:51.202714+0000","last_active":"2024-02-18T00:35:39.627956+0000","last_peered":"2024-02-18T00:35:39.627956+0000","last_clean":"2024-02-18T00:35:39.627956+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:35:39.627956+0000","last_undegraded":"2024-02-18T00:35:39.627956+0000","last_fullsized":"2024-02-18T00:35:39.627956+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.202648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.202648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218226+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:35:39.218226+0000","last_peered":"2024-02-18T00:35:39.218226+0000","last_clean":"2024-02-18T00:35:39.218226+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:35:39.218226+0000","last_undegraded":"2024-02-18T00:35:39.218226+0000","last_fullsized":"2024-02-18T00:35:39.218226+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531222+0000","last_change":"2024-02-18T00:33:04.650781+0000","last_active":"2024-02-18T00:35:39.531222+0000","last_peered":"2024-02-18T00:35:39.531222+0000","last_clean":"2024-02-18T00:35:39.531222+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:35:39.531222+0000","last_undegraded":"2024-02-18T00:35:39.531222+0000","last_fullsized":"2024-02-18T00:35:39.531222+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":224,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.615959+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:35:38.615959+0000","last_peered":"2024-02-18T00:35:38.615959+0000","last_clean":"2024-02-18T00:35:38.615959+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:35:38.615959+0000","last_undegraded":"2024-02-18T00:35:38.615959+0000","last_fullsized":"2024-02-18T00:35:38.615959+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":305,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.620895+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:35:38.620895+0000","last_peered":"2024-02-18T00:35:38.620895+0000","last_clean":"2024-02-18T00:35:38.620895+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:35:38.620895+0000","last_undegraded":"2024-02-18T00:35:38.620895+0000","last_fullsized":"2024-02-18T00:35:38.620895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621148+0000","last_change":"2024-02-18T00:34:32.021327+0000","last_active":"2024-02-18T00:35:38.621148+0000","last_peered":"2024-02-18T00:35:38.621148+0000","last_clean":"2024-02-18T00:35:38.621148+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:35:38.621148+0000","last_undegraded":"2024-02-18T00:35:38.621148+0000","last_fullsized":"2024-02-18T00:35:38.621148+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:34:32.021219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.021219+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":223,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627589+0000","last_change":"2024-02-18T00:35:13.452285+0000","last_active":"2024-02-18T00:35:39.627589+0000","last_peered":"2024-02-18T00:35:39.627589+0000","last_clean":"2024-02-18T00:35:39.627589+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:35:39.627589+0000","last_undegraded":"2024-02-18T00:35:39.627589+0000","last_fullsized":"2024-02-18T00:35:39.627589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.452220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.452220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533269+0000","last_change":"2024-02-18T00:33:04.587776+0000","last_active":"2024-02-18T00:35:39.533269+0000","last_peered":"2024-02-18T00:35:39.533269+0000","last_clean":"2024-02-18T00:35:39.533269+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:35:39.533269+0000","last_undegraded":"2024-02-18T00:35:39.533269+0000","last_fullsized":"2024-02-18T00:35:39.533269+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644584+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:35:39.644584+0000","last_peered":"2024-02-18T00:35:39.644584+0000","last_clean":"2024-02-18T00:35:39.644584+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:35:39.644584+0000","last_undegraded":"2024-02-18T00:35:39.644584+0000","last_fullsized":"2024-02-18T00:35:39.644584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":243,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621479+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:35:38.621479+0000","last_peered":"2024-02-18T00:35:38.621479+0000","last_clean":"2024-02-18T00:35:38.621479+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:35:38.621479+0000","last_undegraded":"2024-02-18T00:35:38.621479+0000","last_fullsized":"2024-02-18T00:35:38.621479+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":200,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627365+0000","last_change":"2024-02-18T00:34:26.340185+0000","last_active":"2024-02-18T00:35:39.627365+0000","last_peered":"2024-02-18T00:35:39.627365+0000","last_clean":"2024-02-18T00:35:39.627365+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:35:39.627365+0000","last_undegraded":"2024-02-18T00:35:39.627365+0000","last_fullsized":"2024-02-18T00:35:39.627365+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.340120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.340120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":226,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619422+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:35:38.619422+0000","last_peered":"2024-02-18T00:35:38.619422+0000","last_clean":"2024-02-18T00:35:38.619422+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:35:38.619422+0000","last_undegraded":"2024-02-18T00:35:38.619422+0000","last_fullsized":"2024-02-18T00:35:38.619422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":223,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627820+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:35:39.627820+0000","last_peered":"2024-02-18T00:35:39.627820+0000","last_clean":"2024-02-18T00:35:39.627820+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:35:39.627820+0000","last_undegraded":"2024-02-18T00:35:39.627820+0000","last_fullsized":"2024-02-18T00:35:39.627820+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619356+0000","last_change":"2024-02-18T00:33:48.731380+0000","last_active":"2024-02-18T00:35:38.619356+0000","last_peered":"2024-02-18T00:35:38.619356+0000","last_clean":"2024-02-18T00:35:38.619356+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:35:38.619356+0000","last_undegraded":"2024-02-18T00:35:38.619356+0000","last_fullsized":"2024-02-18T00:35:38.619356+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":129,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621459+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:35:39.621459+0000","last_peered":"2024-02-18T00:35:39.621459+0000","last_clean":"2024-02-18T00:35:39.621459+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:35:39.621459+0000","last_undegraded":"2024-02-18T00:35:39.621459+0000","last_fullsized":"2024-02-18T00:35:39.621459+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":278,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616248+0000","last_change":"2024-02-18T00:34:28.290944+0000","last_active":"2024-02-18T00:35:38.616248+0000","last_peered":"2024-02-18T00:35:38.616248+0000","last_clean":"2024-02-18T00:35:38.616248+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:35:38.616248+0000","last_undegraded":"2024-02-18T00:35:38.616248+0000","last_fullsized":"2024-02-18T00:35:38.616248+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:34:28.290868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.290868+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619304+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:35:38.619304+0000","last_peered":"2024-02-18T00:35:38.619304+0000","last_clean":"2024-02-18T00:35:38.619304+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:35:38.619304+0000","last_undegraded":"2024-02-18T00:35:38.619304+0000","last_fullsized":"2024-02-18T00:35:38.619304+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":247,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643686+0000","last_change":"2024-02-18T00:34:35.765124+0000","last_active":"2024-02-18T00:35:39.643686+0000","last_peered":"2024-02-18T00:35:39.643686+0000","last_clean":"2024-02-18T00:35:39.643686+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:35:39.643686+0000","last_undegraded":"2024-02-18T00:35:39.643686+0000","last_fullsized":"2024-02-18T00:35:39.643686+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.765059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.765059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531691+0000","last_change":"2024-02-18T00:33:02.549194+0000","last_active":"2024-02-18T00:35:39.531691+0000","last_peered":"2024-02-18T00:35:39.531691+0000","last_clean":"2024-02-18T00:35:39.531691+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:35:39.531691+0000","last_undegraded":"2024-02-18T00:35:39.531691+0000","last_fullsized":"2024-02-18T00:35:39.531691+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:02.549081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:33:02.549081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":223,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644663+0000","last_change":"2024-02-18T00:34:17.016556+0000","last_active":"2024-02-18T00:35:39.644663+0000","last_peered":"2024-02-18T00:35:39.644663+0000","last_clean":"2024-02-18T00:35:39.644663+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:35:39.644663+0000","last_undegraded":"2024-02-18T00:35:39.644663+0000","last_fullsized":"2024-02-18T00:35:39.644663+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.016476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.016476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619648+0000","last_change":"2024-02-18T00:33:04.649428+0000","last_active":"2024-02-18T00:35:38.619648+0000","last_peered":"2024-02-18T00:35:38.619648+0000","last_clean":"2024-02-18T00:35:38.619648+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:35:38.619648+0000","last_undegraded":"2024-02-18T00:35:38.619648+0000","last_fullsized":"2024-02-18T00:35:38.619648+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":234,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.620453+0000","last_change":"2024-02-18T00:33:49.897870+0000","last_active":"2024-02-18T00:35:39.620453+0000","last_peered":"2024-02-18T00:35:39.620453+0000","last_clean":"2024-02-18T00:35:39.620453+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:35:39.620453+0000","last_undegraded":"2024-02-18T00:35:39.620453+0000","last_fullsized":"2024-02-18T00:35:39.620453+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.897812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.897812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":269,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628107+0000","last_change":"2024-02-18T00:34:25.385634+0000","last_active":"2024-02-18T00:35:39.628107+0000","last_peered":"2024-02-18T00:35:39.628107+0000","last_clean":"2024-02-18T00:35:39.628107+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:35:39.628107+0000","last_undegraded":"2024-02-18T00:35:39.628107+0000","last_fullsized":"2024-02-18T00:35:39.628107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:34:25.385564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.385564+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":238,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.618827+0000","last_change":"2024-02-18T00:33:37.809035+0000","last_active":"2024-02-18T00:35:39.618827+0000","last_peered":"2024-02-18T00:35:39.618827+0000","last_clean":"2024-02-18T00:35:39.618827+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:35:39.618827+0000","last_undegraded":"2024-02-18T00:35:39.618827+0000","last_fullsized":"2024-02-18T00:35:39.618827+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:37.808923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.808923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":609,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:40.592888+0000","last_change":"2024-02-18T00:33:41.154008+0000","last_active":"2024-02-18T00:35:40.592888+0000","last_peered":"2024-02-18T00:35:40.592888+0000","last_clean":"2024-02-18T00:35:40.592888+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:35:40.592888+0000","last_undegraded":"2024-02-18T00:35:40.592888+0000","last_fullsized":"2024-02-18T00:35:40.592888+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:33:41.153868+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.153868+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":377,"num_read_kb":372,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":165,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616497+0000","last_change":"2024-02-18T00:34:29.252896+0000","last_active":"2024-02-18T00:35:38.616497+0000","last_peered":"2024-02-18T00:35:38.616497+0000","last_clean":"2024-02-18T00:35:38.616497+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:35:38.616497+0000","last_undegraded":"2024-02-18T00:35:38.616497+0000","last_fullsized":"2024-02-18T00:35:38.616497+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:34:29.252830+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.252830+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218767+0000","last_change":"2024-02-18T00:33:48.670413+0000","last_active":"2024-02-18T00:35:39.218767+0000","last_peered":"2024-02-18T00:35:39.218767+0000","last_clean":"2024-02-18T00:35:39.218767+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:35:39.218767+0000","last_undegraded":"2024-02-18T00:35:39.218767+0000","last_fullsized":"2024-02-18T00:35:39.218767+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":223,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621206+0000","last_change":"2024-02-18T00:34:19.925200+0000","last_active":"2024-02-18T00:35:38.621206+0000","last_peered":"2024-02-18T00:35:38.621206+0000","last_clean":"2024-02-18T00:35:38.621206+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:35:38.621206+0000","last_undegraded":"2024-02-18T00:35:38.621206+0000","last_fullsized":"2024-02-18T00:35:38.621206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:19.925111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.925111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"64'34","reported_seq":367,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627565+0000","last_change":"2024-02-18T00:33:54.198053+0000","last_active":"2024-02-18T00:35:39.627565+0000","last_peered":"2024-02-18T00:35:39.627565+0000","last_clean":"2024-02-18T00:35:39.627565+0000","last_became_active":"2024-02-18T00:29:17.093434+0000","last_became_peered":"2024-02-18T00:29:17.093434+0000","last_unstale":"2024-02-18T00:35:39.627565+0000","last_undegraded":"2024-02-18T00:35:39.627565+0000","last_fullsized":"2024-02-18T00:35:39.627565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:33:54.197976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.197976+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":238,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643572+0000","last_change":"2024-02-18T00:34:15.045746+0000","last_active":"2024-02-18T00:35:39.643572+0000","last_peered":"2024-02-18T00:35:39.643572+0000","last_clean":"2024-02-18T00:35:39.643572+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:35:39.643572+0000","last_undegraded":"2024-02-18T00:35:39.643572+0000","last_fullsized":"2024-02-18T00:35:39.643572+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.045677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.045677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":226,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.530665+0000","last_change":"2024-02-18T00:33:48.688410+0000","last_active":"2024-02-18T00:35:39.530665+0000","last_peered":"2024-02-18T00:35:39.530665+0000","last_clean":"2024-02-18T00:35:39.530665+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:35:39.530665+0000","last_undegraded":"2024-02-18T00:35:39.530665+0000","last_fullsized":"2024-02-18T00:35:39.530665+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:43.926301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.926301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.623350+0000","last_change":"2024-02-18T00:33:04.719631+0000","last_active":"2024-02-18T00:35:38.623350+0000","last_peered":"2024-02-18T00:35:38.623350+0000","last_clean":"2024-02-18T00:35:38.623350+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:35:38.623350+0000","last_undegraded":"2024-02-18T00:35:38.623350+0000","last_fullsized":"2024-02-18T00:35:38.623350+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":308,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621164+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:35:38.621164+0000","last_peered":"2024-02-18T00:35:38.621164+0000","last_clean":"2024-02-18T00:35:38.621164+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:35:38.621164+0000","last_undegraded":"2024-02-18T00:35:38.621164+0000","last_fullsized":"2024-02-18T00:35:38.621164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":227,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621926+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:35:39.621926+0000","last_peered":"2024-02-18T00:35:39.621926+0000","last_clean":"2024-02-18T00:35:39.621926+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:35:39.621926+0000","last_undegraded":"2024-02-18T00:35:39.621926+0000","last_fullsized":"2024-02-18T00:35:39.621926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":252,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627650+0000","last_change":"2024-02-18T00:34:31.410145+0000","last_active":"2024-02-18T00:35:39.627650+0000","last_peered":"2024-02-18T00:35:39.627650+0000","last_clean":"2024-02-18T00:35:39.627650+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:35:39.627650+0000","last_undegraded":"2024-02-18T00:35:39.627650+0000","last_fullsized":"2024-02-18T00:35:39.627650+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.410081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.410081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":216,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619068+0000","last_change":"2024-02-18T00:33:48.010937+0000","last_active":"2024-02-18T00:35:38.619068+0000","last_peered":"2024-02-18T00:35:38.619068+0000","last_clean":"2024-02-18T00:35:38.619068+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:35:38.619068+0000","last_undegraded":"2024-02-18T00:35:38.619068+0000","last_fullsized":"2024-02-18T00:35:38.619068+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:48.010821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:48.010821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218677+0000","last_change":"2024-02-18T00:33:02.483549+0000","last_active":"2024-02-18T00:35:39.218677+0000","last_peered":"2024-02-18T00:35:39.218677+0000","last_clean":"2024-02-18T00:35:39.218677+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:35:39.218677+0000","last_undegraded":"2024-02-18T00:35:39.218677+0000","last_fullsized":"2024-02-18T00:35:39.218677+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.618637+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:35:39.618637+0000","last_peered":"2024-02-18T00:35:39.618637+0000","last_clean":"2024-02-18T00:35:39.618637+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:35:39.618637+0000","last_undegraded":"2024-02-18T00:35:39.618637+0000","last_fullsized":"2024-02-18T00:35:39.618637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":279,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218404+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:35:39.218404+0000","last_peered":"2024-02-18T00:35:39.218404+0000","last_clean":"2024-02-18T00:35:39.218404+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:35:39.218404+0000","last_undegraded":"2024-02-18T00:35:39.218404+0000","last_fullsized":"2024-02-18T00:35:39.218404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":261,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219524+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:35:39.219524+0000","last_peered":"2024-02-18T00:35:39.219524+0000","last_clean":"2024-02-18T00:35:39.219524+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:35:39.219524+0000","last_undegraded":"2024-02-18T00:35:39.219524+0000","last_fullsized":"2024-02-18T00:35:39.219524+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":231,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533091+0000","last_change":"2024-02-18T00:34:26.701231+0000","last_active":"2024-02-18T00:35:39.533091+0000","last_peered":"2024-02-18T00:35:39.533091+0000","last_clean":"2024-02-18T00:35:39.533091+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:35:39.533091+0000","last_undegraded":"2024-02-18T00:35:39.533091+0000","last_fullsized":"2024-02-18T00:35:39.533091+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.701162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.701162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"109'2","reported_seq":197,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627351+0000","last_change":"2024-02-18T00:34:31.655028+0000","last_active":"2024-02-18T00:35:39.627351+0000","last_peered":"2024-02-18T00:35:39.627351+0000","last_clean":"2024-02-18T00:35:39.627351+0000","last_became_active":"2024-02-18T00:34:31.654426+0000","last_became_peered":"2024-02-18T00:34:31.654426+0000","last_unstale":"2024-02-18T00:35:39.627351+0000","last_undegraded":"2024-02-18T00:35:39.627351+0000","last_fullsized":"2024-02-18T00:35:39.627351+0000","mapping_epoch":149,"log_start":"109'2","ondisk_log_start":"109'2","created":106,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218307+0000","last_change":"2024-02-18T00:33:02.533610+0000","last_active":"2024-02-18T00:35:39.218307+0000","last_peered":"2024-02-18T00:35:39.218307+0000","last_clean":"2024-02-18T00:35:39.218307+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:35:39.218307+0000","last_undegraded":"2024-02-18T00:35:39.218307+0000","last_fullsized":"2024-02-18T00:35:39.218307+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":294,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621672+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:35:38.621672+0000","last_peered":"2024-02-18T00:35:38.621672+0000","last_clean":"2024-02-18T00:35:38.621672+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:35:38.621672+0000","last_undegraded":"2024-02-18T00:35:38.621672+0000","last_fullsized":"2024-02-18T00:35:38.621672+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":229,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.618551+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:35:39.618551+0000","last_peered":"2024-02-18T00:35:39.618551+0000","last_clean":"2024-02-18T00:35:39.618551+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:35:39.618551+0000","last_undegraded":"2024-02-18T00:35:39.618551+0000","last_fullsized":"2024-02-18T00:35:39.618551+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":233,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533948+0000","last_change":"2024-02-18T00:34:25.683506+0000","last_active":"2024-02-18T00:35:39.533948+0000","last_peered":"2024-02-18T00:35:39.533948+0000","last_clean":"2024-02-18T00:35:39.533948+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:35:39.533948+0000","last_undegraded":"2024-02-18T00:35:39.533948+0000","last_fullsized":"2024-02-18T00:35:39.533948+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.683442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.683442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":232,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621446+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:35:39.621446+0000","last_peered":"2024-02-18T00:35:39.621446+0000","last_clean":"2024-02-18T00:35:39.621446+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:35:39.621446+0000","last_undegraded":"2024-02-18T00:35:39.621446+0000","last_fullsized":"2024-02-18T00:35:39.621446+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":195,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619823+0000","last_change":"2024-02-18T00:35:25.953784+0000","last_active":"2024-02-18T00:35:38.619823+0000","last_peered":"2024-02-18T00:35:38.619823+0000","last_clean":"2024-02-18T00:35:38.619823+0000","last_became_active":"2024-02-18T00:34:27.633954+0000","last_became_peered":"2024-02-18T00:34:27.633954+0000","last_unstale":"2024-02-18T00:35:38.619823+0000","last_undegraded":"2024-02-18T00:35:38.619823+0000","last_fullsized":"2024-02-18T00:35:38.619823+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:25.953702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.953702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619476+0000","last_change":"2024-02-18T00:33:04.706011+0000","last_active":"2024-02-18T00:35:38.619476+0000","last_peered":"2024-02-18T00:35:38.619476+0000","last_clean":"2024-02-18T00:35:38.619476+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:35:38.619476+0000","last_undegraded":"2024-02-18T00:35:38.619476+0000","last_fullsized":"2024-02-18T00:35:38.619476+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":288,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533812+0000","last_change":"2024-02-18T00:33:48.861148+0000","last_active":"2024-02-18T00:35:39.533812+0000","last_peered":"2024-02-18T00:35:39.533812+0000","last_clean":"2024-02-18T00:35:39.533812+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:35:39.533812+0000","last_undegraded":"2024-02-18T00:35:39.533812+0000","last_fullsized":"2024-02-18T00:35:39.533812+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:33:46.897445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.897445+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":233,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616281+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:35:38.616281+0000","last_peered":"2024-02-18T00:35:38.616281+0000","last_clean":"2024-02-18T00:35:38.616281+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:35:38.616281+0000","last_undegraded":"2024-02-18T00:35:38.616281+0000","last_fullsized":"2024-02-18T00:35:38.616281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644373+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:35:39.644373+0000","last_peered":"2024-02-18T00:35:39.644373+0000","last_clean":"2024-02-18T00:35:39.644373+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:35:39.644373+0000","last_undegraded":"2024-02-18T00:35:39.644373+0000","last_fullsized":"2024-02-18T00:35:39.644373+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616496+0000","last_change":"2024-02-18T00:33:55.265441+0000","last_active":"2024-02-18T00:35:38.616496+0000","last_peered":"2024-02-18T00:35:38.616496+0000","last_clean":"2024-02-18T00:35:38.616496+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:35:38.616496+0000","last_undegraded":"2024-02-18T00:35:38.616496+0000","last_fullsized":"2024-02-18T00:35:38.616496+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.265373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.265373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.a","version":"175'14","reported_seq":195,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619623+0000","last_change":"2024-02-18T00:34:24.618811+0000","last_active":"2024-02-18T00:35:38.619623+0000","last_peered":"2024-02-18T00:35:38.619623+0000","last_clean":"2024-02-18T00:35:38.619623+0000","last_became_active":"2024-02-18T00:34:24.618468+0000","last_became_peered":"2024-02-18T00:34:24.618468+0000","last_unstale":"2024-02-18T00:35:38.619623+0000","last_undegraded":"2024-02-18T00:35:38.619623+0000","last_fullsized":"2024-02-18T00:35:38.619623+0000","mapping_epoch":106,"log_start":"175'14","ondisk_log_start":"175'14","created":106,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":384,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.619051+0000","last_change":"2024-02-18T00:33:04.730258+0000","last_active":"2024-02-18T00:35:39.619051+0000","last_peered":"2024-02-18T00:35:39.619051+0000","last_clean":"2024-02-18T00:35:39.619051+0000","last_became_active":"2024-02-18T00:33:04.729050+0000","last_became_peered":"2024-02-18T00:33:04.729050+0000","last_unstale":"2024-02-18T00:35:39.619051+0000","last_undegraded":"2024-02-18T00:35:39.619051+0000","last_fullsized":"2024-02-18T00:35:39.619051+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":296,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531389+0000","last_change":"2024-02-18T00:34:30.666844+0000","last_active":"2024-02-18T00:35:39.531389+0000","last_peered":"2024-02-18T00:35:39.531389+0000","last_clean":"2024-02-18T00:35:39.531389+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:35:39.531389+0000","last_undegraded":"2024-02-18T00:35:39.531389+0000","last_fullsized":"2024-02-18T00:35:39.531389+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:34:30.666765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.666765+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":247,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643531+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:35:39.643531+0000","last_peered":"2024-02-18T00:35:39.643531+0000","last_clean":"2024-02-18T00:35:39.643531+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:35:39.643531+0000","last_undegraded":"2024-02-18T00:35:39.643531+0000","last_fullsized":"2024-02-18T00:35:39.643531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":310,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644147+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:35:39.644147+0000","last_peered":"2024-02-18T00:35:39.644147+0000","last_clean":"2024-02-18T00:35:39.644147+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:35:39.644147+0000","last_undegraded":"2024-02-18T00:35:39.644147+0000","last_fullsized":"2024-02-18T00:35:39.644147+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":230,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621396+0000","last_change":"2024-02-18T00:33:53.101420+0000","last_active":"2024-02-18T00:35:38.621396+0000","last_peered":"2024-02-18T00:35:38.621396+0000","last_clean":"2024-02-18T00:35:38.621396+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:35:38.621396+0000","last_undegraded":"2024-02-18T00:35:38.621396+0000","last_fullsized":"2024-02-18T00:35:38.621396+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.101251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.101251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":177,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.615208+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:35:38.615208+0000","last_peered":"2024-02-18T00:35:38.615208+0000","last_clean":"2024-02-18T00:35:38.615208+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:35:38.615208+0000","last_undegraded":"2024-02-18T00:35:38.615208+0000","last_fullsized":"2024-02-18T00:35:38.615208+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":274,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643969+0000","last_change":"2024-02-18T00:33:49.276437+0000","last_active":"2024-02-18T00:35:39.643969+0000","last_peered":"2024-02-18T00:35:39.643969+0000","last_clean":"2024-02-18T00:35:39.643969+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:35:39.643969+0000","last_undegraded":"2024-02-18T00:35:39.643969+0000","last_fullsized":"2024-02-18T00:35:39.643969+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:33:40.095425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.095425+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":227,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.620976+0000","last_change":"2024-02-18T00:34:15.935045+0000","last_active":"2024-02-18T00:35:38.620976+0000","last_peered":"2024-02-18T00:35:38.620976+0000","last_clean":"2024-02-18T00:35:38.620976+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:35:38.620976+0000","last_undegraded":"2024-02-18T00:35:38.620976+0000","last_fullsized":"2024-02-18T00:35:38.620976+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.934970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"150'9","reported_seq":322,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531519+0000","last_change":"2024-02-18T00:34:27.741951+0000","last_active":"2024-02-18T00:35:39.531519+0000","last_peered":"2024-02-18T00:35:39.531519+0000","last_clean":"2024-02-18T00:35:39.531519+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:35:39.531519+0000","last_undegraded":"2024-02-18T00:35:39.531519+0000","last_fullsized":"2024-02-18T00:35:39.531519+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:27.741881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.741881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":217,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219711+0000","last_change":"2024-02-18T00:32:53.114523+0000","last_active":"2024-02-18T00:35:39.219711+0000","last_peered":"2024-02-18T00:35:39.219711+0000","last_clean":"2024-02-18T00:35:39.219711+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:35:39.219711+0000","last_undegraded":"2024-02-18T00:35:39.219711+0000","last_fullsized":"2024-02-18T00:35:39.219711+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:53.114424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:32:53.114424+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":169,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644662+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:35:39.644662+0000","last_peered":"2024-02-18T00:35:39.644662+0000","last_clean":"2024-02-18T00:35:39.644662+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:35:39.644662+0000","last_undegraded":"2024-02-18T00:35:39.644662+0000","last_fullsized":"2024-02-18T00:35:39.644662+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":243,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628772+0000","last_change":"2024-02-18T00:34:32.418786+0000","last_active":"2024-02-18T00:35:39.628772+0000","last_peered":"2024-02-18T00:35:39.628772+0000","last_clean":"2024-02-18T00:35:39.628772+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:35:39.628772+0000","last_undegraded":"2024-02-18T00:35:39.628772+0000","last_fullsized":"2024-02-18T00:35:39.628772+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:34:32.418717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.418717+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":233,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616967+0000","last_change":"2024-02-18T00:34:20.312165+0000","last_active":"2024-02-18T00:35:38.616967+0000","last_peered":"2024-02-18T00:35:38.616967+0000","last_clean":"2024-02-18T00:35:38.616967+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:35:38.616967+0000","last_undegraded":"2024-02-18T00:35:38.616967+0000","last_fullsized":"2024-02-18T00:35:38.616967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.312096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.312096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":224,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533037+0000","last_change":"2024-02-18T00:33:50.765435+0000","last_active":"2024-02-18T00:35:39.533037+0000","last_peered":"2024-02-18T00:35:39.533037+0000","last_clean":"2024-02-18T00:35:39.533037+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:35:39.533037+0000","last_undegraded":"2024-02-18T00:35:39.533037+0000","last_fullsized":"2024-02-18T00:35:39.533037+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.765368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.765368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":216,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.618975+0000","last_change":"2024-02-18T00:33:01.075249+0000","last_active":"2024-02-18T00:35:38.618975+0000","last_peered":"2024-02-18T00:35:38.618975+0000","last_clean":"2024-02-18T00:35:38.618975+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:35:38.618975+0000","last_undegraded":"2024-02-18T00:35:38.618975+0000","last_fullsized":"2024-02-18T00:35:38.618975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.075169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.075169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621310+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:35:38.621310+0000","last_peered":"2024-02-18T00:35:38.621310+0000","last_clean":"2024-02-18T00:35:38.621310+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:35:38.621310+0000","last_undegraded":"2024-02-18T00:35:38.621310+0000","last_fullsized":"2024-02-18T00:35:38.621310+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":245,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219093+0000","last_change":"2024-02-18T00:34:20.362851+0000","last_active":"2024-02-18T00:35:39.219093+0000","last_peered":"2024-02-18T00:35:39.219093+0000","last_clean":"2024-02-18T00:35:39.219093+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:35:39.219093+0000","last_undegraded":"2024-02-18T00:35:39.219093+0000","last_fullsized":"2024-02-18T00:35:39.219093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:34:20.362782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.362782+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":229,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628818+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:35:39.628818+0000","last_peered":"2024-02-18T00:35:39.628818+0000","last_clean":"2024-02-18T00:35:39.628818+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:35:39.628818+0000","last_undegraded":"2024-02-18T00:35:39.628818+0000","last_fullsized":"2024-02-18T00:35:39.628818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628854+0000","last_change":"2024-02-18T00:34:34.355483+0000","last_active":"2024-02-18T00:35:39.628854+0000","last_peered":"2024-02-18T00:35:39.628854+0000","last_clean":"2024-02-18T00:35:39.628854+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:35:39.628854+0000","last_undegraded":"2024-02-18T00:35:39.628854+0000","last_fullsized":"2024-02-18T00:35:39.628854+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.355416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.355416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.530768+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:35:39.530768+0000","last_peered":"2024-02-18T00:35:39.530768+0000","last_clean":"2024-02-18T00:35:39.530768+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:35:39.530768+0000","last_undegraded":"2024-02-18T00:35:39.530768+0000","last_fullsized":"2024-02-18T00:35:39.530768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":168,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616365+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:35:38.616365+0000","last_peered":"2024-02-18T00:35:38.616365+0000","last_clean":"2024-02-18T00:35:38.616365+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:35:38.616365+0000","last_undegraded":"2024-02-18T00:35:38.616365+0000","last_fullsized":"2024-02-18T00:35:38.616365+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":232,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.530886+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:35:39.530886+0000","last_peered":"2024-02-18T00:35:39.530886+0000","last_clean":"2024-02-18T00:35:39.530886+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:35:39.530886+0000","last_undegraded":"2024-02-18T00:35:39.530886+0000","last_fullsized":"2024-02-18T00:35:39.530886+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":229,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644389+0000","last_change":"2024-02-18T00:33:39.051726+0000","last_active":"2024-02-18T00:35:39.644389+0000","last_peered":"2024-02-18T00:35:39.644389+0000","last_clean":"2024-02-18T00:35:39.644389+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:35:39.644389+0000","last_undegraded":"2024-02-18T00:35:39.644389+0000","last_fullsized":"2024-02-18T00:35:39.644389+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.051653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.051653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":233,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616186+0000","last_change":"2024-02-18T00:33:56.296342+0000","last_active":"2024-02-18T00:35:38.616186+0000","last_peered":"2024-02-18T00:35:38.616186+0000","last_clean":"2024-02-18T00:35:38.616186+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:35:38.616186+0000","last_undegraded":"2024-02-18T00:35:38.616186+0000","last_fullsized":"2024-02-18T00:35:38.616186+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:56.296274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.296274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":129,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621713+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:35:39.621713+0000","last_peered":"2024-02-18T00:35:39.621713+0000","last_clean":"2024-02-18T00:35:39.621713+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:35:39.621713+0000","last_undegraded":"2024-02-18T00:35:39.621713+0000","last_fullsized":"2024-02-18T00:35:39.621713+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":173,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.644131+0000","last_change":"2024-02-18T00:33:48.714793+0000","last_active":"2024-02-18T00:35:39.644131+0000","last_peered":"2024-02-18T00:35:39.644131+0000","last_clean":"2024-02-18T00:35:39.644131+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:35:39.644131+0000","last_undegraded":"2024-02-18T00:35:39.644131+0000","last_fullsized":"2024-02-18T00:35:39.644131+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":283,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531376+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:35:39.531376+0000","last_peered":"2024-02-18T00:35:39.531376+0000","last_clean":"2024-02-18T00:35:39.531376+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:35:39.531376+0000","last_undegraded":"2024-02-18T00:35:39.531376+0000","last_fullsized":"2024-02-18T00:35:39.531376+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":228,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.617168+0000","last_change":"2024-02-18T00:33:51.295157+0000","last_active":"2024-02-18T00:35:38.617168+0000","last_peered":"2024-02-18T00:35:38.617168+0000","last_clean":"2024-02-18T00:35:38.617168+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:35:38.617168+0000","last_undegraded":"2024-02-18T00:35:38.617168+0000","last_fullsized":"2024-02-18T00:35:38.617168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.295093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.295093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217784+0000","last_change":"2024-02-18T00:34:23.374398+0000","last_active":"2024-02-18T00:35:39.217784+0000","last_peered":"2024-02-18T00:35:39.217784+0000","last_clean":"2024-02-18T00:35:39.217784+0000","last_became_active":"2024-02-18T00:33:48.741677+0000","last_became_peered":"2024-02-18T00:33:48.741677+0000","last_unstale":"2024-02-18T00:35:39.217784+0000","last_undegraded":"2024-02-18T00:35:39.217784+0000","last_fullsized":"2024-02-18T00:35:39.217784+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.374328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.374328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533136+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:35:39.533136+0000","last_peered":"2024-02-18T00:35:39.533136+0000","last_clean":"2024-02-18T00:35:39.533136+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:35:39.533136+0000","last_undegraded":"2024-02-18T00:35:39.533136+0000","last_fullsized":"2024-02-18T00:35:39.533136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"175'7","reported_seq":193,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616357+0000","last_change":"2024-02-18T00:35:03.629983+0000","last_active":"2024-02-18T00:35:38.616357+0000","last_peered":"2024-02-18T00:35:38.616357+0000","last_clean":"2024-02-18T00:35:38.616357+0000","last_became_active":"2024-02-18T00:35:03.629630+0000","last_became_peered":"2024-02-18T00:35:03.629630+0000","last_unstale":"2024-02-18T00:35:38.616357+0000","last_undegraded":"2024-02-18T00:35:38.616357+0000","last_fullsized":"2024-02-18T00:35:38.616357+0000","mapping_epoch":106,"log_start":"175'7","ondisk_log_start":"175'7","created":106,"last_epoch_clean":225,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'7","last_scrub_stamp":"2024-02-18T00:34:32.218733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.218733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":128,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218286+0000","last_change":"2024-02-18T00:33:48.680037+0000","last_active":"2024-02-18T00:35:39.218286+0000","last_peered":"2024-02-18T00:35:39.218286+0000","last_clean":"2024-02-18T00:35:39.218286+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:35:39.218286+0000","last_undegraded":"2024-02-18T00:35:39.218286+0000","last_fullsized":"2024-02-18T00:35:39.218286+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":284,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.621906+0000","last_change":"2024-02-18T00:33:56.121942+0000","last_active":"2024-02-18T00:35:38.621906+0000","last_peered":"2024-02-18T00:35:38.621906+0000","last_clean":"2024-02-18T00:35:38.621906+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:35:38.621906+0000","last_undegraded":"2024-02-18T00:35:38.621906+0000","last_fullsized":"2024-02-18T00:35:38.621906+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.121872+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":233,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643059+0000","last_change":"2024-02-18T00:33:48.634995+0000","last_active":"2024-02-18T00:35:39.643059+0000","last_peered":"2024-02-18T00:35:39.643059+0000","last_clean":"2024-02-18T00:35:39.643059+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:35:39.643059+0000","last_undegraded":"2024-02-18T00:35:39.643059+0000","last_fullsized":"2024-02-18T00:35:39.643059+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:49.926996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:32:49.926996+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":330,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.617366+0000","last_change":"2024-02-18T00:34:18.318312+0000","last_active":"2024-02-18T00:35:38.617366+0000","last_peered":"2024-02-18T00:35:38.617366+0000","last_clean":"2024-02-18T00:35:38.617366+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:35:38.617366+0000","last_undegraded":"2024-02-18T00:35:38.617366+0000","last_fullsized":"2024-02-18T00:35:38.617366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:18.318190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.318190+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":142,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643900+0000","last_change":"2024-02-18T00:34:20.995201+0000","last_active":"2024-02-18T00:35:39.643900+0000","last_peered":"2024-02-18T00:35:39.643900+0000","last_clean":"2024-02-18T00:35:39.643900+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:35:39.643900+0000","last_undegraded":"2024-02-18T00:35:39.643900+0000","last_fullsized":"2024-02-18T00:35:39.643900+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:34:20.995125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.995125+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1","version":"175'9","reported_seq":188,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218826+0000","last_change":"2024-02-18T00:34:59.611803+0000","last_active":"2024-02-18T00:35:39.218826+0000","last_peered":"2024-02-18T00:35:39.218826+0000","last_clean":"2024-02-18T00:35:39.218826+0000","last_became_active":"2024-02-18T00:34:59.611429+0000","last_became_peered":"2024-02-18T00:34:59.611429+0000","last_unstale":"2024-02-18T00:35:39.218826+0000","last_undegraded":"2024-02-18T00:35:39.218826+0000","last_fullsized":"2024-02-18T00:35:39.218826+0000","mapping_epoch":106,"log_start":"175'9","ondisk_log_start":"175'9","created":106,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531538+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:35:39.531538+0000","last_peered":"2024-02-18T00:35:39.531538+0000","last_clean":"2024-02-18T00:35:39.531538+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:35:39.531538+0000","last_undegraded":"2024-02-18T00:35:39.531538+0000","last_fullsized":"2024-02-18T00:35:39.531538+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":260,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.617425+0000","last_change":"2024-02-18T00:33:52.250914+0000","last_active":"2024-02-18T00:35:38.617425+0000","last_peered":"2024-02-18T00:35:38.617425+0000","last_clean":"2024-02-18T00:35:38.617425+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:35:38.617425+0000","last_undegraded":"2024-02-18T00:35:38.617425+0000","last_fullsized":"2024-02-18T00:35:38.617425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:52.250848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.250848+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":228,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.529664+0000","last_change":"2024-02-18T00:34:28.731011+0000","last_active":"2024-02-18T00:35:39.529664+0000","last_peered":"2024-02-18T00:35:39.529664+0000","last_clean":"2024-02-18T00:35:39.529664+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:35:39.529664+0000","last_undegraded":"2024-02-18T00:35:39.529664+0000","last_fullsized":"2024-02-18T00:35:39.529664+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.730955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.730955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":227,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.622882+0000","last_change":"2024-02-18T00:33:48.751672+0000","last_active":"2024-02-18T00:35:38.622882+0000","last_peered":"2024-02-18T00:35:38.622882+0000","last_clean":"2024-02-18T00:35:38.622882+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:35:38.622882+0000","last_undegraded":"2024-02-18T00:35:38.622882+0000","last_fullsized":"2024-02-18T00:35:38.622882+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.025296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.025296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":223,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628685+0000","last_change":"2024-02-18T00:32:59.800515+0000","last_active":"2024-02-18T00:35:39.628685+0000","last_peered":"2024-02-18T00:35:39.628685+0000","last_clean":"2024-02-18T00:35:39.628685+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:35:39.628685+0000","last_undegraded":"2024-02-18T00:35:39.628685+0000","last_fullsized":"2024-02-18T00:35:39.628685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:32:59.800355+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":182,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.629084+0000","last_change":"2024-02-18T00:34:55.656536+0000","last_active":"2024-02-18T00:35:39.629084+0000","last_peered":"2024-02-18T00:35:39.629084+0000","last_clean":"2024-02-18T00:35:39.629084+0000","last_became_active":"2024-02-18T00:34:55.656245+0000","last_became_peered":"2024-02-18T00:34:55.656245+0000","last_unstale":"2024-02-18T00:35:39.629084+0000","last_undegraded":"2024-02-18T00:35:39.629084+0000","last_fullsized":"2024-02-18T00:35:39.629084+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":182,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.618972+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:35:39.618972+0000","last_peered":"2024-02-18T00:35:39.618972+0000","last_clean":"2024-02-18T00:35:39.618972+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:35:39.618972+0000","last_undegraded":"2024-02-18T00:35:39.618972+0000","last_fullsized":"2024-02-18T00:35:39.618972+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":257,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621637+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:35:39.621637+0000","last_peered":"2024-02-18T00:35:39.621637+0000","last_clean":"2024-02-18T00:35:39.621637+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:35:39.621637+0000","last_undegraded":"2024-02-18T00:35:39.621637+0000","last_fullsized":"2024-02-18T00:35:39.621637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":242,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.615933+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:35:38.615933+0000","last_peered":"2024-02-18T00:35:38.615933+0000","last_clean":"2024-02-18T00:35:38.615933+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:35:38.615933+0000","last_undegraded":"2024-02-18T00:35:38.615933+0000","last_fullsized":"2024-02-18T00:35:38.615933+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":225,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.629086+0000","last_change":"2024-02-18T00:34:18.357435+0000","last_active":"2024-02-18T00:35:39.629086+0000","last_peered":"2024-02-18T00:35:39.629086+0000","last_clean":"2024-02-18T00:35:39.629086+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:35:39.629086+0000","last_undegraded":"2024-02-18T00:35:39.629086+0000","last_fullsized":"2024-02-18T00:35:39.629086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.357369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.357369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.219294+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:35:39.219294+0000","last_peered":"2024-02-18T00:35:39.219294+0000","last_clean":"2024-02-18T00:35:39.219294+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:35:39.219294+0000","last_undegraded":"2024-02-18T00:35:39.219294+0000","last_fullsized":"2024-02-18T00:35:39.219294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3","version":"175'14","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.629791+0000","last_change":"2024-02-18T00:35:26.560007+0000","last_active":"2024-02-18T00:35:39.629791+0000","last_peered":"2024-02-18T00:35:39.629791+0000","last_clean":"2024-02-18T00:35:39.629791+0000","last_became_active":"2024-02-18T00:35:23.664255+0000","last_became_peered":"2024-02-18T00:35:23.664255+0000","last_unstale":"2024-02-18T00:35:39.629791+0000","last_undegraded":"2024-02-18T00:35:39.629791+0000","last_fullsized":"2024-02-18T00:35:39.629791+0000","mapping_epoch":149,"log_start":"175'14","ondisk_log_start":"175'14","created":106,"last_epoch_clean":245,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'14","last_scrub_stamp":"2024-02-18T00:35:26.559929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.559929+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":512,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":168,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616594+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:35:38.616594+0000","last_peered":"2024-02-18T00:35:38.616594+0000","last_clean":"2024-02-18T00:35:38.616594+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:35:38.616594+0000","last_undegraded":"2024-02-18T00:35:38.616594+0000","last_fullsized":"2024-02-18T00:35:38.616594+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":264,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533905+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:35:39.533905+0000","last_peered":"2024-02-18T00:35:39.533905+0000","last_clean":"2024-02-18T00:35:39.533905+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:35:39.533905+0000","last_undegraded":"2024-02-18T00:35:39.533905+0000","last_fullsized":"2024-02-18T00:35:39.533905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":248,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.532993+0000","last_change":"2024-02-18T00:33:48.748311+0000","last_active":"2024-02-18T00:35:39.532993+0000","last_peered":"2024-02-18T00:35:39.532993+0000","last_clean":"2024-02-18T00:35:39.532993+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:35:39.532993+0000","last_undegraded":"2024-02-18T00:35:39.532993+0000","last_fullsized":"2024-02-18T00:35:39.532993+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:33:45.906516+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.906516+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":224,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218477+0000","last_change":"2024-02-18T00:34:29.236028+0000","last_active":"2024-02-18T00:35:39.218477+0000","last_peered":"2024-02-18T00:35:39.218477+0000","last_clean":"2024-02-18T00:35:39.218477+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:35:39.218477+0000","last_undegraded":"2024-02-18T00:35:39.218477+0000","last_fullsized":"2024-02-18T00:35:39.218477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:29.235953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.235953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":217,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.533653+0000","last_change":"2024-02-18T00:33:38.914876+0000","last_active":"2024-02-18T00:35:39.533653+0000","last_peered":"2024-02-18T00:35:39.533653+0000","last_clean":"2024-02-18T00:35:39.533653+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:35:39.533653+0000","last_undegraded":"2024-02-18T00:35:39.533653+0000","last_fullsized":"2024-02-18T00:35:39.533653+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:38.914691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:38.914691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.4","version":"109'2","reported_seq":200,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.621584+0000","last_change":"2024-02-18T00:35:19.638620+0000","last_active":"2024-02-18T00:35:39.621584+0000","last_peered":"2024-02-18T00:35:39.621584+0000","last_clean":"2024-02-18T00:35:39.621584+0000","last_became_active":"2024-02-18T00:35:19.638287+0000","last_became_peered":"2024-02-18T00:35:19.638287+0000","last_unstale":"2024-02-18T00:35:39.621584+0000","last_undegraded":"2024-02-18T00:35:39.621584+0000","last_fullsized":"2024-02-18T00:35:39.621584+0000","mapping_epoch":106,"log_start":"109'2","ondisk_log_start":"109'2","created":106,"last_epoch_clean":241,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.623406+0000","last_change":"2024-02-18T00:33:48.765117+0000","last_active":"2024-02-18T00:35:38.623406+0000","last_peered":"2024-02-18T00:35:38.623406+0000","last_clean":"2024-02-18T00:35:38.623406+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:35:38.623406+0000","last_undegraded":"2024-02-18T00:35:38.623406+0000","last_fullsized":"2024-02-18T00:35:38.623406+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":284,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217852+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:35:39.217852+0000","last_peered":"2024-02-18T00:35:39.217852+0000","last_clean":"2024-02-18T00:35:39.217852+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:35:39.217852+0000","last_undegraded":"2024-02-18T00:35:39.217852+0000","last_fullsized":"2024-02-18T00:35:39.217852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":129,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643746+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:35:39.643746+0000","last_peered":"2024-02-18T00:35:39.643746+0000","last_clean":"2024-02-18T00:35:39.643746+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:35:39.643746+0000","last_undegraded":"2024-02-18T00:35:39.643746+0000","last_fullsized":"2024-02-18T00:35:39.643746+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":234,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643499+0000","last_change":"2024-02-18T00:34:25.961381+0000","last_active":"2024-02-18T00:35:39.643499+0000","last_peered":"2024-02-18T00:35:39.643499+0000","last_clean":"2024-02-18T00:35:39.643499+0000","last_became_active":"2024-02-18T00:33:48.736206+0000","last_became_peered":"2024-02-18T00:33:48.736206+0000","last_unstale":"2024-02-18T00:35:39.643499+0000","last_undegraded":"2024-02-18T00:35:39.643499+0000","last_fullsized":"2024-02-18T00:35:39.643499+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.961303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.961303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":230,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.619477+0000","last_change":"2024-02-18T00:33:48.649527+0000","last_active":"2024-02-18T00:35:38.619477+0000","last_peered":"2024-02-18T00:35:38.619477+0000","last_clean":"2024-02-18T00:35:38.619477+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:35:38.619477+0000","last_undegraded":"2024-02-18T00:35:38.619477+0000","last_fullsized":"2024-02-18T00:35:38.619477+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.996913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.996913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"249'13","reported_seq":207,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.628555+0000","last_change":"2024-02-18T00:35:28.515044+0000","last_active":"2024-02-18T00:35:39.628555+0000","last_peered":"2024-02-18T00:35:39.628555+0000","last_clean":"2024-02-18T00:35:39.628555+0000","last_became_active":"2024-02-18T00:35:15.634956+0000","last_became_peered":"2024-02-18T00:35:15.634956+0000","last_unstale":"2024-02-18T00:35:39.628555+0000","last_undegraded":"2024-02-18T00:35:39.628555+0000","last_fullsized":"2024-02-18T00:35:39.628555+0000","mapping_epoch":106,"log_start":"175'11","ondisk_log_start":"175'11","created":106,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'13","last_scrub_stamp":"2024-02-18T00:35:28.514983+0000","last_deep_scrub":"249'13","last_deep_scrub_stamp":"2024-02-18T00:35:28.514983+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.514983+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.620142+0000","last_change":"2024-02-18T00:33:04.678891+0000","last_active":"2024-02-18T00:35:39.620142+0000","last_peered":"2024-02-18T00:35:39.620142+0000","last_clean":"2024-02-18T00:35:39.620142+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:35:39.620142+0000","last_undegraded":"2024-02-18T00:35:39.620142+0000","last_fullsized":"2024-02-18T00:35:39.620142+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":254,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.218552+0000","last_change":"2024-02-18T00:34:19.325220+0000","last_active":"2024-02-18T00:35:39.218552+0000","last_peered":"2024-02-18T00:35:39.218552+0000","last_clean":"2024-02-18T00:35:39.218552+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:35:39.218552+0000","last_undegraded":"2024-02-18T00:35:39.218552+0000","last_fullsized":"2024-02-18T00:35:39.218552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:34:19.325130+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.325130+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.616358+0000","last_change":"2024-02-18T00:34:31.241231+0000","last_active":"2024-02-18T00:35:38.616358+0000","last_peered":"2024-02-18T00:35:38.616358+0000","last_clean":"2024-02-18T00:35:38.616358+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:35:38.616358+0000","last_undegraded":"2024-02-18T00:35:38.616358+0000","last_fullsized":"2024-02-18T00:35:38.616358+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.241146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217960+0000","last_change":"2024-02-18T00:33:48.733307+0000","last_active":"2024-02-18T00:35:39.217960+0000","last_peered":"2024-02-18T00:35:39.217960+0000","last_clean":"2024-02-18T00:35:39.217960+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:35:39.217960+0000","last_undegraded":"2024-02-18T00:35:39.217960+0000","last_fullsized":"2024-02-18T00:35:39.217960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.801504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.801504+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":223,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.643110+0000","last_change":"2024-02-18T00:33:54.013575+0000","last_active":"2024-02-18T00:35:39.643110+0000","last_peered":"2024-02-18T00:35:39.643110+0000","last_clean":"2024-02-18T00:35:39.643110+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:35:39.643110+0000","last_undegraded":"2024-02-18T00:35:39.643110+0000","last_fullsized":"2024-02-18T00:35:39.643110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:54.013467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.013467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"175'11","reported_seq":190,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.617233+0000","last_change":"2024-02-18T00:35:11.654395+0000","last_active":"2024-02-18T00:35:38.617233+0000","last_peered":"2024-02-18T00:35:38.617233+0000","last_clean":"2024-02-18T00:35:38.617233+0000","last_became_active":"2024-02-18T00:35:11.654076+0000","last_became_peered":"2024-02-18T00:35:11.654076+0000","last_unstale":"2024-02-18T00:35:38.617233+0000","last_undegraded":"2024-02-18T00:35:38.617233+0000","last_fullsized":"2024-02-18T00:35:38.617233+0000","mapping_epoch":149,"log_start":"175'11","ondisk_log_start":"175'11","created":106,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'11","last_scrub_stamp":"2024-02-18T00:34:35.243061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.243061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.534168+0000","last_change":"2024-02-18T00:33:48.714160+0000","last_active":"2024-02-18T00:35:39.534168+0000","last_peered":"2024-02-18T00:35:39.534168+0000","last_clean":"2024-02-18T00:35:39.534168+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:35:39.534168+0000","last_undegraded":"2024-02-18T00:35:39.534168+0000","last_fullsized":"2024-02-18T00:35:39.534168+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":262,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217510+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:35:39.217510+0000","last_peered":"2024-02-18T00:35:39.217510+0000","last_clean":"2024-02-18T00:35:39.217510+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:35:39.217510+0000","last_undegraded":"2024-02-18T00:35:39.217510+0000","last_fullsized":"2024-02-18T00:35:39.217510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":228,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.530219+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:35:39.530219+0000","last_peered":"2024-02-18T00:35:39.530219+0000","last_clean":"2024-02-18T00:35:39.530219+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:35:39.530219+0000","last_undegraded":"2024-02-18T00:35:39.530219+0000","last_fullsized":"2024-02-18T00:35:39.530219+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":234,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.627424+0000","last_change":"2024-02-18T00:34:38.428939+0000","last_active":"2024-02-18T00:35:39.627424+0000","last_peered":"2024-02-18T00:35:39.627424+0000","last_clean":"2024-02-18T00:35:39.627424+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:35:39.627424+0000","last_undegraded":"2024-02-18T00:35:39.627424+0000","last_fullsized":"2024-02-18T00:35:39.627424+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.428855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.428855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":222,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217745+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:35:39.217745+0000","last_peered":"2024-02-18T00:35:39.217745+0000","last_clean":"2024-02-18T00:35:39.217745+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:35:39.217745+0000","last_undegraded":"2024-02-18T00:35:39.217745+0000","last_fullsized":"2024-02-18T00:35:39.217745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.7","version":"175'7","reported_seq":205,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.619371+0000","last_change":"2024-02-18T00:35:09.277995+0000","last_active":"2024-02-18T00:35:39.619371+0000","last_peered":"2024-02-18T00:35:39.619371+0000","last_clean":"2024-02-18T00:35:39.619371+0000","last_became_active":"2024-02-18T00:35:07.629918+0000","last_became_peered":"2024-02-18T00:35:07.629918+0000","last_unstale":"2024-02-18T00:35:39.619371+0000","last_undegraded":"2024-02-18T00:35:39.619371+0000","last_fullsized":"2024-02-18T00:35:39.619371+0000","mapping_epoch":106,"log_start":"175'7","ondisk_log_start":"175'7","created":106,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"175'7","last_scrub_stamp":"2024-02-18T00:35:09.277926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:09.277926+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":256,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531698+0000","last_change":"2024-02-18T00:33:04.720986+0000","last_active":"2024-02-18T00:35:39.531698+0000","last_peered":"2024-02-18T00:35:39.531698+0000","last_clean":"2024-02-18T00:35:39.531698+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:35:39.531698+0000","last_undegraded":"2024-02-18T00:35:39.531698+0000","last_fullsized":"2024-02-18T00:35:39.531698+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":308,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.531020+0000","last_change":"2024-02-18T00:33:41.866669+0000","last_active":"2024-02-18T00:35:39.531020+0000","last_peered":"2024-02-18T00:35:39.531020+0000","last_clean":"2024-02-18T00:35:39.531020+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:35:39.531020+0000","last_undegraded":"2024-02-18T00:35:39.531020+0000","last_fullsized":"2024-02-18T00:35:39.531020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:33:41.866531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.866531+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":236,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:38.623211+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:35:38.623211+0000","last_peered":"2024-02-18T00:35:38.623211+0000","last_clean":"2024-02-18T00:35:38.623211+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:35:38.623211+0000","last_undegraded":"2024-02-18T00:35:38.623211+0000","last_fullsized":"2024-02-18T00:35:38.623211+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":224,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.217905+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:35:39.217905+0000","last_peered":"2024-02-18T00:35:39.217905+0000","last_clean":"2024-02-18T00:35:39.217905+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:35:39.217905+0000","last_undegraded":"2024-02-18T00:35:39.217905+0000","last_fullsized":"2024-02-18T00:35:39.217905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":231,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-02-18T00:35:39.532857+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:35:39.532857+0000","last_peered":"2024-02-18T00:35:39.532857+0000","last_clean":"2024-02-18T00:35:39.532857+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:35:39.532857+0000","last_undegraded":"2024-02-18T00:35:39.532857+0000","last_fullsized":"2024-02-18T00:35:39.532857+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":171,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":170,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":169,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":26,"num_pg":11,"stat_sum":{"num_bytes":2048,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":6528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":33,"acting":33,"num_store_stats":8},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1259,"num_objects":127,"num_object_clones":8,"num_object_copies":381,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":2,"num_read":92,"num_read_kb":1,"num_write":536,"num_write_kb":532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1544192,"data_stored":3810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1763,"ondisk_log_size":1763,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":1364,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":628,"ondisk_log_size":628,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":377,"num_read_kb":372,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266769,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863462999,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313736,"kb_used_data":4216,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431416,"statfs":{"total":95995035648,"available":95673769984,"internally_reserved":0,"allocated":4317184,"data_stored":2355070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":2.7120000000000002,"15min":1.2250000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":13.935,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":1.1319999999999999,"15min":0.70399999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":10.433,"5min":10.433,"15min":10.433},"last":0.53900000000000003}]},{"osd":1,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":2.944,"15min":1.3100000000000001},"min":{"1min":0.27900000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.2290000000000001,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":2.9129999999999998,"15min":1.2929999999999999},"min":{"1min":0.32100000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":7.899,"5min":71.872,"15min":71.872},"last":0.36899999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.3520000000000001,"15min":0.78700000000000003},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":11.855,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":1.804,"15min":0.94399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.7769999999999999,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.499}]},{"osd":3,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":1.347,"15min":0.78300000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":5.1280000000000001,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":1.228,"15min":0.753},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":5.0069999999999997,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.63100000000000001,"15min":0.47099999999999997},"min":{"1min":0.26100000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.8169999999999999,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":1.091,"15min":0.626},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":5.085,"5min":35.18,"15min":35.18},"last":0.38700000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66300000000000003,"15min":0.48899999999999999},"min":{"1min":0.30299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":5.2539999999999996,"5min":10.606,"15min":10.606},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":1.177,"15min":0.66500000000000004},"min":{"1min":0.33500000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":5.5899999999999999,"5min":35.119,"15min":35.119},"last":0.432}]},{"osd":7,"last update":"Sun Feb 18 00:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60699999999999998,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.754,"5min":1.089,"15min":0.71499999999999997},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.3620000000000001,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.47899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313780,"kb_used_data":4256,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431372,"statfs":{"total":95995035648,"available":95673724928,"internally_reserved":0,"allocated":4358144,"data_stored":2355622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":1.2529999999999999,"15min":0.623},"min":{"1min":0.24299999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.79600000000000004,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":1.329,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.59999999999999998,"5min":43.137,"15min":43.137},"last":0.41099999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.5369999999999999,"15min":0.76100000000000001},"min":{"1min":0.23000000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55300000000000005,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.97599999999999998,"15min":0.58599999999999997},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":17.596,"15min":17.596},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:34:41 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.363,"15min":0.76100000000000001},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":19.274000000000001,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":1.877,"5min":1.3620000000000001,"15min":0.78000000000000003},"min":{"1min":0.33700000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":22.989000000000001,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.51700000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":1.1499999999999999,"15min":0.78300000000000003},"min":{"1min":0.39300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88900000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.167,"15min":0.76400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77000000000000002,"5min":12.75,"15min":12.75},"last":0.79800000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.1439999999999999,"15min":0.77700000000000002},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.8759999999999999,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.1499999999999999,"15min":0.79200000000000004},"min":{"1min":0.41999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":4.1180000000000003,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.749}]},{"osd":6,"last update":"Sun Feb 18 00:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":1.1970000000000001,"15min":0.81699999999999995},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96999999999999997,"5min":12.834,"15min":12.834},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":1.1499999999999999,"15min":0.80600000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90400000000000003,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.151,"15min":0.80500000000000005},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92500000000000004,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.141,"15min":0.81599999999999995},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81699999999999995,"5min":12.805,"15min":12.805},"last":0.75600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313736,"kb_used_data":4212,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431416,"statfs":{"total":95995035648,"available":95673769984,"internally_reserved":0,"allocated":4313088,"data_stored":2354019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":2.2639999999999998,"15min":1.155},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":2.524,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.309},{"interface":"front","average":{"1min":0.48799999999999999,"5min":1.5069999999999999,"15min":0.73599999999999999},"min":{"1min":0.222,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":2.331,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.221}]},{"osd":2,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":1.827,"15min":0.87},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.577,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":1.8560000000000001,"15min":0.874},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":2.153,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.26900000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":1.171,"15min":0.68799999999999994},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":3.5609999999999999,"5min":19.006,"15min":19.006},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":2.2170000000000001,"15min":1.0760000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.7430000000000001,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":1.661,"15min":0.91700000000000004},"min":{"1min":0.32800000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.1850000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.6850000000000001,"15min":0.96699999999999997},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.5680000000000001,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.48799999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":1.617,"15min":0.93200000000000005},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.5,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.8169999999999999,"15min":0.96599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":3.4060000000000001,"5min":57.991,"15min":57.991},"last":0.51100000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":1.2270000000000001,"15min":0.79300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.2830000000000004,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.434},{"interface":"front","average":{"1min":0.58199999999999996,"5min":1.706,"15min":0.95199999999999996},"min":{"1min":0.34899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.5369999999999999,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.55400000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":1.8700000000000001,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5940000000000001,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.8029999999999999,"15min":0.99299999999999999},"min":{"1min":0.45500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":3.0249999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.44900000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313856,"kb_used_data":4332,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431296,"statfs":{"total":95995035648,"available":95673647104,"internally_reserved":0,"allocated":4435968,"data_stored":2354576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.73099999999999998,"15min":0.495},"min":{"1min":0.216,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61899999999999999,"5min":12.144,"15min":12.144},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":1.0580000000000001,"15min":0.60799999999999998},"min":{"1min":0.246,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66500000000000004,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.71999999999999997,"15min":0.5},"min":{"1min":0.23899999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":2.3370000000000002,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.82899999999999996,"15min":0.54700000000000004},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.77100000000000002,"5min":18.919,"15min":18.919},"last":0.33700000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.94399999999999995,"15min":0.62},"min":{"1min":0.318,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.3809999999999998,"5min":22.712,"15min":22.712},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.94299999999999995,"15min":0.63300000000000001},"min":{"1min":0.29099999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":4.8529999999999998,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.45300000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.88300000000000001,"15min":0.68700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80500000000000005,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.753},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68300000000000005},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79000000000000004,"5min":10.016,"15min":10.016},"last":0.63100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.78200000000000003,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":6.766,"15min":6.766},"last":0.62},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.85399999999999998,"15min":0.70099999999999996},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78500000000000003,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:34:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.79400000000000004,"15min":0.69999999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.196,"5min":6.806,"15min":6.806},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.91300000000000003,"15min":0.75},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.827,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.72699999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.81000000000000005,"15min":0.74099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.89800000000000002,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.73899999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149797,"num_pgs":126,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309704,"kb_used_data":4280,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435448,"statfs":{"total":95995035648,"available":95677898752,"internally_reserved":0,"allocated":4382720,"data_stored":2353968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:34:37 2024","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":0.66100000000000003,"15min":0.48499999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":25.082999999999998,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.25800000000000001},{"interface":"front","average":{"1min":1.339,"5min":0.75600000000000001,"15min":0.52900000000000003},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":18.911000000000001,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.27400000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:34:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.60299999999999998,"15min":0.48599999999999999},"min":{"1min":0.27100000000000002,"5min":0.223,"15min":0.223},"max":{"1min":4.1550000000000002,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.31},{"interface":"front","average":{"1min":1.649,"5min":1.2010000000000001,"15min":0.67200000000000004},"min":{"1min":0.25800000000000001,"5min":0.252,"15min":0.25},"max":{"1min":23.439,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.24399999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:34:37 2024","interfaces":[{"interface":"back","average":{"1min":1.8779999999999999,"5min":1.0700000000000001,"15min":0.65200000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":23.405000000000001,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.54800000000000004,"15min":0.47399999999999998},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":5.3479999999999999,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.88200000000000001,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":10.131,"15min":10.131},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.70199999999999996,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72899999999999998,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.70299999999999996,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81799999999999995,"15min":0.67800000000000005},"min":{"1min":0.41099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1279999999999999,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.68300000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:34:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.022,"15min":0.73899999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.292,"5min":10.189,"15min":10.189},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.72999999999999998,"15min":0.66100000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79100000000000004,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.78100000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.92300000000000004,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80300000000000005,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.57299999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986272,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301480,"kb_used_data":4256,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443672,"statfs":{"total":95995035648,"available":95686320128,"internally_reserved":0,"allocated":4358144,"data_stored":2354963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":1.5629999999999999,"15min":0.92600000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65900000000000003,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":1.5089999999999999,"15min":0.88100000000000001},"min":{"1min":0.34100000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.68799999999999994}]},{"osd":1,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.81200000000000006,"15min":0.65500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67300000000000004,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.435,"5min":1.595,"15min":0.90700000000000003},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.66900000000000004}]},{"osd":2,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.78300000000000003,"15min":0.67300000000000004},"min":{"1min":0.373,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60299999999999998,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.90200000000000002,"15min":0.68600000000000005},"min":{"1min":0.33100000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1459999999999999,"5min":17.77,"15min":17.77},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.69699999999999995,"15min":0.63600000000000001},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64200000000000002,"5min":7.218,"15min":7.218},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.502,"5min":1.071,"15min":0.72299999999999998},"min":{"1min":0.372,"5min":0.309,"15min":0.309},"max":{"1min":0.65100000000000002,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.60099999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.439,"15min":0.38300000000000001},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.45900000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41099999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45400000000000001,"15min":0.40799999999999997},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60699999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.497,"15min":0.44900000000000001},"min":{"1min":0.307,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.629,"5min":0.751,"15min":0.751},"last":0.53600000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72399999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.433},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.44700000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724635,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313660,"kb_used_data":4144,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431492,"statfs":{"total":95995035648,"available":95673847808,"internally_reserved":0,"allocated":4243456,"data_stored":2354853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57199999999999995,"15min":0.54700000000000004},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.8480000000000001,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.441},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.69899999999999995,"15min":0.59699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.8149999999999999,"5min":12.744,"15min":12.744},"last":0.60799999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.625},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6669999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.73399999999999999,"15min":0.62},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":4.7149999999999999,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.65000000000000002,"15min":0.61299999999999999},"min":{"1min":0.30199999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.7410000000000001,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67100000000000004,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.734,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.57899999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":2.1099999999999999,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63200000000000001,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.546,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.65600000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.54600000000000004,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.621,"5min":11.818,"15min":11.818},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.48799999999999999,"15min":0.41299999999999998},"min":{"1min":0.309,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73299999999999998,"5min":4.274,"15min":4.274},"last":0.55800000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:34:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.64000000000000001,"15min":0.47799999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.52200000000000002,"5min":18.605,"15min":18.605},"last":0.504},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.52200000000000002,"15min":0.44},"min":{"1min":0.35599999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.57699999999999996,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.497}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.60099999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.78400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":2778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:35:45.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph status --format=json 2024-02-18T00:35:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:45.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:45.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:45.389 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:45.419 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:45.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:45.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:45.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:45.722 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:45.722 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.626122+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]': finished 2024-02-18T00:35:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:44.626248+0000 mon.a (mon.0) 1844 : cluster [DBG] osdmap e266: 8 total, 8 up, 7 in 2024-02-18T00:35:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.627580+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]: dispatch 2024-02-18T00:35:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.630638+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]: dispatch 2024-02-18T00:35:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.634176+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.634355+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:44.772391+0000 mon.b (mon.1) 332 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:45.823 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:45.851 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:45.870 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:45.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:45.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:45.974 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:46.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.626122+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]': finished 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:44.626248+0000 mon.a (mon.0) 1844 : cluster [DBG] osdmap e266: 8 total, 8 up, 7 in 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.627580+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]: dispatch 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.630638+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]: dispatch 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.634176+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.634355+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:44.772391+0000 mon.b (mon.1) 332 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.626122+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-29"}]': finished 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:44.626248+0000 mon.a (mon.0) 1844 : cluster [DBG] osdmap e266: 8 total, 8 up, 7 in 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.627580+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]: dispatch 2024-02-18T00:35:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.630638+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]: dispatch 2024-02-18T00:35:46.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.634176+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:46.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.634355+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:46.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:44.772391+0000 mon.b (mon.1) 332 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:46.003 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:46.021 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:46.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:46.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:46.125 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:46.152 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:46.171 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:46.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:46.174 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:46.275 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:46.301 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 -- 172.21.15.59:0/3927774031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 msgr2=0x7fcb34103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 --2- 172.21.15.59:0/3927774031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 0x7fcb34103a20 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 rx=0x7fcb2c009be0 tx=0x7fcb2c004d10).stop 2024-02-18T00:35:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 -- 172.21.15.59:0/3927774031 shutdown_connections 2024-02-18T00:35:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 --2- 172.21.15.59:0/3927774031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 0x7fcb34103a20 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 --2- 172.21.15.59:0/3927774031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340fcbb0 0x7fcb340fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 --2- 172.21.15.59:0/3927774031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb340fd510 0x7fcb340f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 -- 172.21.15.59:0/3927774031 >> 172.21.15.59:0/3927774031 conn(0x7fcb340f2820 msgr2=0x7fcb340f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.302+0000 7fcb39b08700 1 -- 172.21.15.59:0/3927774031 shutdown_connections 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.303+0000 7fcb39b08700 1 -- 172.21.15.59:0/3927774031 wait complete. 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.303+0000 7fcb39b08700 1 librados: starting msgr at 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.303+0000 7fcb39b08700 1 librados: starting objecter 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.303+0000 7fcb39b08700 1 Processor -- start 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.303+0000 7fcb39b08700 1 -- start start 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.303+0000 7fcb39b08700 1 librados: setting wanted keys 2024-02-18T00:35:46.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.303+0000 7fcb39b08700 1 librados: calling monclient init 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb39b08700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 0x7fcb3410e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb39b08700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340fcbb0 0x7fcb3410e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb39b08700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb340fd510 0x7fcb34108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb39b08700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34108720 con 0x7fcb340f7400 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb39b08700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34108890 con 0x7fcb340fd510 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb39b08700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34108a50 con 0x7fcb340fcbb0 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 0x7fcb3410e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 0x7fcb3410e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48452/0 (socket says 172.21.15.59:48452) 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 -- 172.21.15.59:0/4158008229 learned_addr learned my addr 172.21.15.59:0/4158008229 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb33fff700 1 --2- 172.21.15.59:0/4158008229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb340fd510 0x7fcb34108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340fcbb0 0x7fcb3410e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 -- 172.21.15.59:0/4158008229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340fcbb0 msgr2=0x7fcb3410e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340fcbb0 0x7fcb3410e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 -- 172.21.15.59:0/4158008229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb340fd510 msgr2=0x7fcb34108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb340fd510 0x7fcb34108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 -- 172.21.15.59:0/4158008229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24005040 con 0x7fcb340f7400 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340fcbb0 0x7fcb3410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 0x7fcb3410e090 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 rx=0x7fcb240048f0 tx=0x7fcb24009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb30ff9700 10 client.?.objecter ms_handle_connect 0x7fcb340f7400 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.304+0000 7fcb30ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.305+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24007b00 con 0x7fcb340f7400 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.305+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c0097c0 con 0x7fcb340f7400 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.305+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb2400f040 con 0x7fcb340f7400 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.305+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24014970 con 0x7fcb340f7400 2024-02-18T00:35:46.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.305+0000 7fcb39b08700 10 client.30918.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.305+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb34114ec0 con 0x7fcb340f7400 2024-02-18T00:35:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.305+0000 7fcb39b08700 1 librados: init done 2024-02-18T00:35:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.306+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fcb3405fa60 con 0x7fcb340f7400 2024-02-18T00:35:46.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.306+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fcb24010500 con 0x7fcb340f7400 2024-02-18T00:35:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb30ff9700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb1c062c90 0x7fcb1c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(267..267 src has 44..267) v4 ==== 8670+0+0 (secure 0 0 0) 0x7fcb2407cfb0 con 0x7fcb340f7400 2024-02-18T00:35:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 osd_map(267..267 src has 44..267) v4 2024-02-18T00:35:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb30ff9700 3 client.30918.objecter handle_osd_map got epochs [267,267] > 0 2024-02-18T00:35:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb30ff9700 3 client.30918.objecter handle_osd_map decoding full epoch 267 2024-02-18T00:35:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb30ff9700 20 client.30918.objecter dump_active .. 0 homeless 2024-02-18T00:35:46.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v267) v1 ==== 54+0+16 (secure 0 0 0) 0x7fcb2407f600 con 0x7fcb340f7400 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.307+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb1c062c90 0x7fcb1c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 10 client.30918.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 1 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcb340ffed0 0x7fcb341002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 1 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 0 1 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3405ee80 osd=0 4 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 10 client.30918.objecter _send_command 1 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb34106180 con 0x7fcb340ffed0 2024-02-18T00:35:46.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb33fff700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcb340ffed0 0x7fcb341002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb1c062c90 0x7fcb1c065150 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 rx=0x7fcb28009fd0 tx=0x7fcb2800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb1c062c90 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb340ffed0 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.308+0000 7fcb33fff700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcb340ffed0 0x7fcb341002b0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.309+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb34106180 con 0x7fcb340ffed0 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.309+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.309+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 1 = system:0 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.309+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb3405ee80 osd=0 4 2024-02-18T00:35:46.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.309+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 0 1 2024-02-18T00:35:46.320 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:46.323 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:46.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 10 client.30918.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 2 2024-02-18T00:35:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 2 2024-02-18T00:35:46.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb3405ee80 osd=0 3 2024-02-18T00:35:46.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 0 2 2024-02-18T00:35:46.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3405ee80 osd=0 4 2024-02-18T00:35:46.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 10 client.30918.objecter _send_command 2 2024-02-18T00:35:46.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.327+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb340273c0 con 0x7fcb340ffed0 2024-02-18T00:35:46.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb340273c0 con 0x7fcb340ffed0 2024-02-18T00:35:46.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb3405ee80 osd=0 4 2024-02-18T00:35:46.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 0 2 2024-02-18T00:35:46.331 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 10 client.30918.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 3 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6805/26279110 conn(0x7fcb340598f0 0x7fcb34059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 3 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 1 3 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34028710 osd=1 4 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 10 client.30918.objecter _send_command 3 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb3411b7d0 con 0x7fcb340598f0 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.329+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6805/26279110 conn(0x7fcb340598f0 0x7fcb34059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.330+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb340598f0 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.330+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6805/26279110 conn(0x7fcb340598f0 0x7fcb34059cf0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.330+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb3411b7d0 con 0x7fcb340598f0 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.330+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.330+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 3 = system:0 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.330+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb34028710 osd=1 4 2024-02-18T00:35:46.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.330+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 1 3 2024-02-18T00:35:46.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 10 client.30918.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 4 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 4 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb34028710 osd=1 3 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 1 4 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34028710 osd=1 4 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 10 client.30918.objecter _send_command 4 2024-02-18T00:35:46.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.345+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb34059360 con 0x7fcb340598f0 2024-02-18T00:35:46.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.346+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb34059360 con 0x7fcb340598f0 2024-02-18T00:35:46.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.346+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.346+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.346+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb34028710 osd=1 4 2024-02-18T00:35:46.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.346+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 1 4 2024-02-18T00:35:46.348 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:35:46.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 10 client.30918.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 5 2024-02-18T00:35:46.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcb34028270 0x7fcb3411a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 5 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 2 5 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34059770 osd=2 4 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 10 client.30918.objecter _send_command 5 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb3411a6e0 con 0x7fcb34028270 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcb34028270 0x7fcb3411a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb34028270 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.347+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcb34028270 0x7fcb3411a1f0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.348+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb3411a6e0 con 0x7fcb34028270 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.348+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.348+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 5 = system:0 2024-02-18T00:35:46.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.348+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb34059770 osd=2 4 2024-02-18T00:35:46.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.348+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 2 5 2024-02-18T00:35:46.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 10 client.30918.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 6 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 6 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb34059770 osd=2 3 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 2 6 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34059770 osd=2 4 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 10 client.30918.objecter _send_command 6 2024-02-18T00:35:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.362+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb340277d0 con 0x7fcb34028270 2024-02-18T00:35:46.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.363+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb340277d0 con 0x7fcb34028270 2024-02-18T00:35:46.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.363+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.363+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.363+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb34059770 osd=2 4 2024-02-18T00:35:46.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.363+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 2 6 2024-02-18T00:35:46.366 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:35:46.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 10 client.30918.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 7 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcb340f87f0 0x7fcb340f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 7 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 3 7 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3411b0d0 osd=3 4 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 10 client.30918.objecter _send_command 7 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb340fa740 con 0x7fcb340f87f0 2024-02-18T00:35:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb33fff700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcb340f87f0 0x7fcb340f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb340f87f0 2024-02-18T00:35:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.364+0000 7fcb33fff700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcb340f87f0 0x7fcb340f8c70 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.365+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb340fa740 con 0x7fcb340f87f0 2024-02-18T00:35:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.365+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:35:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.365+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 7 = system:0 2024-02-18T00:35:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.365+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb3411b0d0 osd=3 4 2024-02-18T00:35:46.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.365+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 3 7 2024-02-18T00:35:46.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 10 client.30918.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 8 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 8 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb3411b0d0 osd=3 3 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 3 8 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3411b0d0 osd=3 4 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 10 client.30918.objecter _send_command 8 2024-02-18T00:35:46.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.379+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb340fa2e0 con 0x7fcb340f87f0 2024-02-18T00:35:46.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.380+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb340fa2e0 con 0x7fcb340f87f0 2024-02-18T00:35:46.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.380+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.380+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.380+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb3411b0d0 osd=3 4 2024-02-18T00:35:46.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.380+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 3 8 2024-02-18T00:35:46.383 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:35:46.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 10 client.30918.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 9 2024-02-18T00:35:46.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcb340f97e0 0x7fcb340f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 9 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 4 9 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340f9160 osd=4 4 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 10 client.30918.objecter _send_command 9 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb340fa110 con 0x7fcb340f97e0 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcb340f97e0 0x7fcb340f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb340f97e0 2024-02-18T00:35:46.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.381+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcb340f97e0 0x7fcb340f9c20 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.382+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb340fa110 con 0x7fcb340f97e0 2024-02-18T00:35:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.382+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:35:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.382+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 9 = system:0 2024-02-18T00:35:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.382+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb340f9160 osd=4 4 2024-02-18T00:35:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.382+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 4 9 2024-02-18T00:35:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 10 client.30918.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 10 2024-02-18T00:35:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 10 2024-02-18T00:35:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb340f9160 osd=4 3 2024-02-18T00:35:46.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 4 10 2024-02-18T00:35:46.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340f9160 osd=4 4 2024-02-18T00:35:46.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 10 client.30918.objecter _send_command 10 2024-02-18T00:35:46.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.396+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb340fa110 con 0x7fcb340f97e0 2024-02-18T00:35:46.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb340fa110 con 0x7fcb340f97e0 2024-02-18T00:35:46.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb340f9160 osd=4 4 2024-02-18T00:35:46.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 4 10 2024-02-18T00:35:46.400 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:35:46.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 10 client.30918.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 11 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcb34126290 0x7fcb34128680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 11 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 5 11 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340fa110 osd=5 4 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 10 client.30918.objecter _send_command 11 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.398+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb3412b2d0 con 0x7fcb34126290 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.399+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcb34126290 0x7fcb34128680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.399+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb34126290 2024-02-18T00:35:46.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.399+0000 7fcb32ffd700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcb34126290 0x7fcb34128680 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.400+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb3412b2d0 con 0x7fcb34126290 2024-02-18T00:35:46.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.400+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:35:46.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.400+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 11 = system:0 2024-02-18T00:35:46.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.400+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb340fa110 osd=5 4 2024-02-18T00:35:46.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.400+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 5 11 2024-02-18T00:35:46.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 10 client.30918.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 12 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 12 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb340fa110 osd=5 3 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 5 12 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340fa110 osd=5 4 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 10 client.30918.objecter _send_command 12 2024-02-18T00:35:46.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.414+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb34128b70 con 0x7fcb34126290 2024-02-18T00:35:46.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb34128b70 con 0x7fcb34126290 2024-02-18T00:35:46.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb340fa110 osd=5 4 2024-02-18T00:35:46.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 5 12 2024-02-18T00:35:46.418 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 10 client.30918.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 13 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcb341291f0 0x7fcb3412b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 13 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 6 13 2024-02-18T00:35:46.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34128b70 osd=6 4 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 10 client.30918.objecter _send_command 13 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb3412e2c0 con 0x7fcb341291f0 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb33fff700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcb341291f0 0x7fcb3412b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb33fff700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcb341291f0 0x7fcb3412b5e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.416+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb341291f0 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.417+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb3412e2c0 con 0x7fcb341291f0 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.417+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.417+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 13 = system:0 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.417+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb34128b70 osd=6 4 2024-02-18T00:35:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.417+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 6 13 2024-02-18T00:35:46.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 10 client.30918.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 14 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 14 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb34128b70 osd=6 3 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 6 14 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34128b70 osd=6 4 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 10 client.30918.objecter _send_command 14 2024-02-18T00:35:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.434+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb3412e2c0 con 0x7fcb341291f0 2024-02-18T00:35:46.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb3412e2c0 con 0x7fcb341291f0 2024-02-18T00:35:46.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb34128b70 osd=6 4 2024-02-18T00:35:46.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 6 14 2024-02-18T00:35:46.438 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 10 client.30918.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 15 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6812/53025171 conn(0x7fcb3412d650 0x7fcb3412dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 15 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 7 15 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3412e2c0 osd=7 4 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 10 client.30918.objecter _send_command 15 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb34131670 con 0x7fcb3412d650 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.436+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6812/53025171 conn(0x7fcb3412d650 0x7fcb3412dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.437+0000 7fcb30ff9700 10 client.30918.objecter ms_handle_connect 0x7fcb3412d650 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.437+0000 7fcb337fe700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6812/53025171 conn(0x7fcb3412d650 0x7fcb3412dab0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.437+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcb34131670 con 0x7fcb3412d650 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.437+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.437+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 15 = system:0 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.437+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb3412e2c0 osd=7 4 2024-02-18T00:35:46.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.437+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 7 15 2024-02-18T00:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 10 client.30918.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign -1 16 2024-02-18T00:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 20 client.30918.objecter _get_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_remove -1 16 2024-02-18T00:35:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 20 client.30918.objecter get_session s=0x7fcb3412e2c0 osd=7 3 2024-02-18T00:35:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 15 client.30918.objecter _session_command_op_assign 7 16 2024-02-18T00:35:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3412e2c0 osd=7 4 2024-02-18T00:35:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 10 client.30918.objecter _send_command 16 2024-02-18T00:35:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.452+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcb3412bad0 con 0x7fcb3412d650 2024-02-18T00:35:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb30ff9700 1 -- 172.21.15.59:0/4158008229 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcb3412bad0 con 0x7fcb3412d650 2024-02-18T00:35:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb30ff9700 10 client.30918.objecter ms_dispatch 0x7fcb34001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:35:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb30ff9700 10 client.30918.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb30ff9700 20 client.30918.objecter put_session s=0x7fcb3412e2c0 osd=7 4 2024-02-18T00:35:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb30ff9700 15 client.30918.objecter _session_command_op_remove 7 16 2024-02-18T00:35:46.456 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:35:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 librados: watch_flush enter 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 librados: watch_flush exit 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.0 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcb340ffed0 msgr2=0x7fcb341002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcb340ffed0 0x7fcb341002b0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3405ee80 osd=0 1 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.1 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6805/26279110 conn(0x7fcb340598f0 msgr2=0x7fcb34059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6805/26279110 conn(0x7fcb340598f0 0x7fcb34059cf0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34028710 osd=1 1 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.2 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcb34028270 msgr2=0x7fcb3411a1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcb34028270 0x7fcb3411a1f0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34059770 osd=2 1 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.3 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcb340f87f0 msgr2=0x7fcb340f8c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcb340f87f0 0x7fcb340f8c70 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3411b0d0 osd=3 1 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.4 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcb340f97e0 msgr2=0x7fcb340f9c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcb340f97e0 0x7fcb340f9c20 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb33fff700 1 -- 172.21.15.59:0/4158008229 reap_dead start 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340f9160 osd=4 1 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.5 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb33fff700 1 -- 172.21.15.59:0/4158008229 reap_dead start 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcb34126290 msgr2=0x7fcb34128680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcb34126290 0x7fcb34128680 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb340fa110 osd=5 1 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.6 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcb341291f0 msgr2=0x7fcb3412b5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcb341291f0 0x7fcb3412b5e0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb34128b70 osd=6 1 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter close_session for osd.7 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6812/53025171 conn(0x7fcb3412d650 msgr2=0x7fcb3412dab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.137:6812/53025171 conn(0x7fcb3412d650 0x7fcb3412dab0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter put_session s=0x7fcb3412e2c0 osd=7 1 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 20 client.30918.objecter shutdown clearing up homeless session... 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.454+0000 7fcb39b08700 10 client.30918.objecter successfully canceled tick 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb1c062c90 msgr2=0x7fcb1c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcb1c062c90 0x7fcb1c065150 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 rx=0x7fcb28009fd0 tx=0x7fcb2800b040).stop 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 msgr2=0x7fcb3410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 --2- 172.21.15.59:0/4158008229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb340f7400 0x7fcb3410e090 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 rx=0x7fcb240048f0 tx=0x7fcb24009cb0).stop 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb33fff700 1 -- 172.21.15.59:0/4158008229 reap_dead start 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 shutdown_connections 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 >> 172.21.15.59:0/4158008229 conn(0x7fcb340f2820 msgr2=0x7fcb34060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 shutdown_connections 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 -- 172.21.15.59:0/4158008229 wait complete. 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:46.455+0000 7fcb39b08700 1 librados: shutdown 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:35:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:45.621375+0000 mon.a (mon.0) 1848 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.626548+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]': finished 2024-02-18T00:35:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.626629+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.626689+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:45.626732+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e267: 8 total, 8 up, 7 in 2024-02-18T00:35:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.628700+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.628879+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.707119+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.707434+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:45.773437+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:45.855572+0000 mgr.y (mgr.24454) 287 : cluster [DBG] pgmap v388: 249 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 242 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 144 B/s, 5 objects/s recovering 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.308184+0000 mon.a (mon.0) 1856 : audit [DBG] from='client.? 172.21.15.59:0/4158008229' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.312675+0000 mon.a (mon.0) 1857 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.313410+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:45.621375+0000 mon.a (mon.0) 1848 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:46.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.626548+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]': finished 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.626629+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.626689+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:45.626732+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e267: 8 total, 8 up, 7 in 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.628700+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.628879+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.707119+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.707434+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:45.773437+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:45.855572+0000 mgr.y (mgr.24454) 287 : cluster [DBG] pgmap v388: 249 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 242 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 144 B/s, 5 objects/s recovering 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.308184+0000 mon.a (mon.0) 1856 : audit [DBG] from='client.? 172.21.15.59:0/4158008229' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.312675+0000 mon.a (mon.0) 1857 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:35:46.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.313410+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.318912+0000 mon.a (mon.0) 1859 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.627249+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.627320+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.627375+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:46.627433+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e268: 8 total, 8 up, 7 in 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.318912+0000 mon.a (mon.0) 1859 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.627249+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:46.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.627320+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:46.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.627375+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:46.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:46.627433+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e268: 8 total, 8 up, 7 in 2024-02-18T00:35:46.905 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:46.909 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:46.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:47.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:45.621375+0000 mon.a (mon.0) 1848 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:47.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.626548+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-29", "mode": "writeback"}]': finished 2024-02-18T00:35:47.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.626629+0000 mon.a (mon.0) 1850 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:47.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.626689+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:47.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:45.626732+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e267: 8 total, 8 up, 7 in 2024-02-18T00:35:47.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.628700+0000 mon.a (mon.0) 1853 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]: dispatch 2024-02-18T00:35:47.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.628879+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]: dispatch 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.707119+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.707434+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:45.773437+0000 mon.b (mon.1) 333 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:45.855572+0000 mgr.y (mgr.24454) 287 : cluster [DBG] pgmap v388: 249 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 242 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 144 B/s, 5 objects/s recovering 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.308184+0000 mon.a (mon.0) 1856 : audit [DBG] from='client.? 172.21.15.59:0/4158008229' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.312675+0000 mon.a (mon.0) 1857 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.313410+0000 mon.a (mon.0) 1858 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.318912+0000 mon.a (mon.0) 1859 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.627249+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? 172.21.15.59:0/450379973' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi059-82882-46"}]': finished 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.627320+0000 mon.a (mon.0) 1861 : audit [INF] from='client.? 172.21.15.59:0/643487737' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi059-82877-37"}]': finished 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.627375+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:47.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:46.627433+0000 mon.a (mon.0) 1863 : cluster [DBG] osdmap e268: 8 total, 8 up, 7 in 2024-02-18T00:35:47.010 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:47.060 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:47.064 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:47.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:47.165 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:47.193 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:47.214 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:47.216 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:47.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:47.317 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:47.347 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:47.486 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:47.495 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:47.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:47.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:47.599 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:47.633 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:47.656 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:47.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:47.659 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:47.739 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:47.747 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:35:47.748 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.629011+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.629857+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.648650+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.648895+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.649948+0000 mon.b (mon.1) 335 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.650099+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.650289+0000 mon.c (mon.2) 227 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.650554+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.650753+0000 mon.b (mon.1) 336 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.650904+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.651097+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.651346+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.629011+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.629857+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.648650+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.648895+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.649948+0000 mon.b (mon.1) 335 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.650099+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.650289+0000 mon.c (mon.2) 227 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.650554+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.650753+0000 mon.b (mon.1) 336 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.650904+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.651097+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.651346+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.651706+0000 mon.c (mon.2) 229 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.651916+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:46.774550+0000 mon.b (mon.1) 337 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:47.624520+0000 mon.a (mon.0) 1871 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.628789+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]': finished 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.628873+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:47.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.628957+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:47.629033+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e269: 8 total, 8 up, 7 in 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.630851+0000 mon.b (mon.1) 338 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.631321+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.633539+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.633893+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.651706+0000 mon.c (mon.2) 229 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.651916+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:46.774550+0000 mon.b (mon.1) 337 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:47.624520+0000 mon.a (mon.0) 1871 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.628789+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]': finished 2024-02-18T00:35:47.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.628873+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:47.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.628957+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:47.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:47.629033+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e269: 8 total, 8 up, 7 in 2024-02-18T00:35:47.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.630851+0000 mon.b (mon.1) 338 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.631321+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.633539+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.756 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.633893+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.760 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:47.787 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:47.807 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:47.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:47.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:47.910 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:35:47.935 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.629011+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:47.935 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.629857+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.648650+0000 mon.b (mon.1) 334 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.648895+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.649948+0000 mon.b (mon.1) 335 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.650099+0000 mon.a (mon.0) 1866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.650289+0000 mon.c (mon.2) 227 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.650554+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.650753+0000 mon.b (mon.1) 336 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.650904+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.651097+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.651346+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.651706+0000 mon.c (mon.2) 229 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.936 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.651916+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:46.774550+0000 mon.b (mon.1) 337 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:47.624520+0000 mon.a (mon.0) 1871 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.628789+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-29"}]': finished 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.628873+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi059-82882-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.628957+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi059-82877-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:47.629033+0000 mon.a (mon.0) 1875 : cluster [DBG] osdmap e269: 8 total, 8 up, 7 in 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.630851+0000 mon.b (mon.1) 338 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.631321+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.633539+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.633893+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:47.955 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:35:47.958 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:47.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:35:48.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:48.091 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:48.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-18T00:35:48.843 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:48.846 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:48.846 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:48.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:48.976 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:49.224 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:49.226 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:49.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:49.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:35:49.362 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:49.436 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:35:49.440 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:49.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:35:49.541 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:35:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:47.775671+0000 mon.b (mon.1) 339 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:47.855942+0000 mgr.y (mgr.24454) 288 : cluster [DBG] pgmap v391: 249 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 242 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 144 B/s, 5 objects/s recovering 2024-02-18T00:35:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:47.775671+0000 mon.b (mon.1) 339 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:47.855942+0000 mgr.y (mgr.24454) 288 : cluster [DBG] pgmap v391: 249 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 242 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 144 B/s, 5 objects/s recovering 2024-02-18T00:35:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:47.775671+0000 mon.b (mon.1) 339 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:47.855942+0000 mgr.y (mgr.24454) 288 : cluster [DBG] pgmap v391: 249 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 242 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s; 144 B/s, 5 objects/s recovering 2024-02-18T00:35:50.731 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:35:50.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:50.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:35:50.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:50.867 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:48.777078+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:49.495720+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e270: 8 total, 8 up, 7 in 2024-02-18T00:35:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:49.778255+0000 mon.b (mon.1) 341 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:49.856328+0000 mgr.y (mgr.24454) 289 : cluster [DBG] pgmap v393: 217 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 210 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 241 B/s wr, 1 op/s; 136 B/s, 5 objects/s recovering 2024-02-18T00:35:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:49.966895+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:49.967011+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:49.967121+0000 mon.a (mon.0) 1881 : cluster [DBG] osdmap e271: 8 total, 8 up, 7 in 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:49.972101+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:49.972555+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:50.779208+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:48.777078+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:49.495720+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e270: 8 total, 8 up, 7 in 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:49.778255+0000 mon.b (mon.1) 341 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:49.856328+0000 mgr.y (mgr.24454) 289 : cluster [DBG] pgmap v393: 217 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 210 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 241 B/s wr, 1 op/s; 136 B/s, 5 objects/s recovering 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:49.966895+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:49.967011+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:49.967121+0000 mon.a (mon.0) 1881 : cluster [DBG] osdmap e271: 8 total, 8 up, 7 in 2024-02-18T00:35:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:49.972101+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:49.972555+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:51.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:50.779208+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:35:51.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:35:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:48.777078+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:49.495720+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e270: 8 total, 8 up, 7 in 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:49.778255+0000 mon.b (mon.1) 341 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:49.856328+0000 mgr.y (mgr.24454) 289 : cluster [DBG] pgmap v393: 217 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 210 active+clean; 8.9 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 241 B/s wr, 1 op/s; 136 B/s, 5 objects/s recovering 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:49.966895+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi059-82882-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:49.967011+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi059-82877-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:49.967121+0000 mon.a (mon.0) 1881 : cluster [DBG] osdmap e271: 8 total, 8 up, 7 in 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:49.972101+0000 mon.c (mon.2) 231 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:49.972555+0000 mon.a (mon.0) 1882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:50.779208+0000 mon.b (mon.1) 342 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:52.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:35:52] "GET /metrics HTTP/1.1" 200 190681 "" "Prometheus/2.33.4" 2024-02-18T00:35:52.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:50.967554+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:52.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:50.967739+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e272: 8 total, 8 up, 7 in 2024-02-18T00:35:52.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:50.971341+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:52.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:50.972795+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.097965+0000 mgr.y (mgr.24454) 290 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.780341+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:50.967554+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:50.967739+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e272: 8 total, 8 up, 7 in 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:50.971341+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:50.972795+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.097965+0000 mgr.y (mgr.24454) 290 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:52.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.780341+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:50.967554+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:50.967739+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e272: 8 total, 8 up, 7 in 2024-02-18T00:35:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:50.971341+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:50.972795+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.097965+0000 mgr.y (mgr.24454) 290 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:35:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.780341+0000 mon.b (mon.1) 343 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:52.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 -- 172.21.15.59:0/558461086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 msgr2=0x7fa7f4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/558461086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 0x7fa7f4101d20 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 rx=0x7fa7e4004870 tx=0x7fa7e4009cb0).stop 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 -- 172.21.15.59:0/558461086 shutdown_connections 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/558461086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f40f8cb0 0x7fa7f40f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/558461086 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 0x7fa7f4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/558461086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 0x7fa7f4101d20 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 -- 172.21.15.59:0/558461086 >> 172.21.15.59:0/558461086 conn(0x7fa7f40f2820 msgr2=0x7fa7f40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 -- 172.21.15.59:0/558461086 shutdown_connections 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.459+0000 7fa7fb44f700 1 -- 172.21.15.59:0/558461086 wait complete. 2024-02-18T00:35:52.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 librados: starting msgr at 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 librados: starting objecter 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 Processor -- start 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 -- start start 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 librados: setting wanted keys 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 librados: calling monclient init 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f40f8cb0 0x7fa7f410e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 0x7fa7f410e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 0x7fa7f41083f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f41088e0 con 0x7fa7f40f9610 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4108a50 con 0x7fa7f40f8cb0 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7fb44f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f4108c10 con 0x7fa7f4102210 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 0x7fa7f41083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:52.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 0x7fa7f41083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:54564/0 (socket says 172.21.15.59:54564) 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 -- 172.21.15.59:0/615868746 learned_addr learned my addr 172.21.15.59:0/615868746 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f89ea700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 0x7fa7f410e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 -- 172.21.15.59:0/615868746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f40f8cb0 msgr2=0x7fa7f410e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f40f8cb0 0x7fa7f410e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 -- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 msgr2=0x7fa7f410e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 0x7fa7f410e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f99ec700 1 -- 172.21.15.59:0/615868746 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0097c0 con 0x7fa7f4102210 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.460+0000 7fa7f89ea700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 0x7fa7f410e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7f99ec700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 0x7fa7f41083f0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 rx=0x7fa7ec001df0 tx=0x7fa7ec004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7ea7fc700 10 client.?.objecter ms_handle_connect 0x7fa7f4102210 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7ea7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/615868746 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0099f0 con 0x7fa7f4102210 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e4005040 con 0x7fa7f4102210 2024-02-18T00:35:52.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/615868746 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7ec0095a0 con 0x7fa7f4102210 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/615868746 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec015420 con 0x7fa7f4102210 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7fb44f700 10 client.40754.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f4109200 con 0x7fa7f4102210 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.461+0000 7fa7fb44f700 1 librados: init done 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.462+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f405fa60 con 0x7fa7f4102210 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.462+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/615868746 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa7ec0090d0 con 0x7fa7f4102210 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7ea7fc700 1 --2- 172.21.15.59:0/615868746 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/615868746 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(273..273 src has 44..273) v4 ==== 9233+0+0 (secure 0 0 0) 0x7fa7ec07e030 con 0x7fa7f4102210 2024-02-18T00:35:52.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7ea7fc700 10 client.40754.objecter ms_dispatch 0x7fa7f4001550 osd_map(273..273 src has 44..273) v4 2024-02-18T00:35:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7ea7fc700 3 client.40754.objecter handle_osd_map got epochs [273,273] > 0 2024-02-18T00:35:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7ea7fc700 3 client.40754.objecter handle_osd_map decoding full epoch 273 2024-02-18T00:35:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7f91eb700 1 --2- 172.21.15.59:0/615868746 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7ea7fc700 20 client.40754.objecter dump_active .. 0 homeless 2024-02-18T00:35:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7ea7fc700 10 client.40754.objecter ms_handle_connect 0x7fa7e0062c90 2024-02-18T00:35:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.463+0000 7fa7f91eb700 1 --2- 172.21.15.59:0/615868746 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 rx=0x7fa7f0004c60 tx=0x7fa7f000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:52.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.467+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/615868746 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa7ec051060 con 0x7fa7f4102210 2024-02-18T00:35:52.493 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:52.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.360+0000 7fb877d58700 1 -- 172.21.15.59:0/3980230114 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700fe510 msgr2=0x7fb8700f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.360+0000 7fb877d58700 1 --2- 172.21.15.59:0/3980230114 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700fe510 0x7fb8700f7d40 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 rx=0x7fb8600047b0 tx=0x7fb860009cb0).stop 2024-02-18T00:35:52.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 -- 172.21.15.59:0/3980230114 shutdown_connections 2024-02-18T00:35:52.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 --2- 172.21.15.59:0/3980230114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8700f8420 0x7fb870104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 --2- 172.21.15.59:0/3980230114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8700fdbb0 0x7fb8700fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 --2- 172.21.15.59:0/3980230114 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700fe510 0x7fb8700f7d40 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 -- 172.21.15.59:0/3980230114 >> 172.21.15.59:0/3980230114 conn(0x7fb8700f3800 msgr2=0x7fb8700f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 -- 172.21.15.59:0/3980230114 shutdown_connections 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 -- 172.21.15.59:0/3980230114 wait complete. 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 librados: starting msgr at 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 librados: starting objecter 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 Processor -- start 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 -- start start 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 librados: setting wanted keys 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.361+0000 7fb877d58700 1 librados: calling monclient init 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700f8420 0x7fb87006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8700fdbb0 0x7fb87006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8700fe510 0x7fb87006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb870116530 con 0x7fb8700fe510 2024-02-18T00:35:52.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701166a0 con 0x7fb8700fdbb0 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb870116860 con 0x7fb8700f8420 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700f8420 0x7fb87006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700f8420 0x7fb87006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:54542/0 (socket says 172.21.15.59:54542) 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 -- 172.21.15.59:0/928846527 learned_addr learned my addr 172.21.15.59:0/928846527 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb8762f5700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8700fe510 0x7fb87006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb8752f3700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8700fdbb0 0x7fb87006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 -- 172.21.15.59:0/928846527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8700fdbb0 msgr2=0x7fb87006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8700fdbb0 0x7fb87006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 -- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8700fe510 msgr2=0x7fb87006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8700fe510 0x7fb87006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 -- 172.21.15.59:0/928846527 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c005040 con 0x7fb8700f8420 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb875af4700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700f8420 0x7fb87006b0d0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 rx=0x7fb86c009400 tx=0x7fb86c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb866ffd700 10 client.?.objecter ms_handle_connect 0x7fb8700f8420 2024-02-18T00:35:52.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb866ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb866ffd700 1 -- 172.21.15.59:0/928846527 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c007a40 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb860005040 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb866ffd700 1 -- 172.21.15.59:0/928846527 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb86c00f040 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb866ffd700 1 -- 172.21.15.59:0/928846527 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c014aa0 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 10 client.40748.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb870116f20 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.362+0000 7fb877d58700 1 librados: init done 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb870107120 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb866ffd700 1 -- 172.21.15.59:0/928846527 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb86c00e630 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb866ffd700 1 --2- 172.21.15.59:0/928846527 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb85c062c90 0x7fb85c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb866ffd700 1 -- 172.21.15.59:0/928846527 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(273..273 src has 44..273) v4 ==== 9233+0+0 (secure 0 0 0) 0x7fb86c07d2d0 con 0x7fb8700f8420 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb866ffd700 10 client.40748.objecter ms_dispatch 0x7fb870001550 osd_map(273..273 src has 44..273) v4 2024-02-18T00:35:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb866ffd700 3 client.40748.objecter handle_osd_map got epochs [273,273] > 0 2024-02-18T00:35:52.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb866ffd700 3 client.40748.objecter handle_osd_map decoding full epoch 273 2024-02-18T00:35:52.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb8752f3700 1 --2- 172.21.15.59:0/928846527 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb85c062c90 0x7fb85c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:52.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.364+0000 7fb866ffd700 20 client.40748.objecter dump_active .. 0 homeless 2024-02-18T00:35:52.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.365+0000 7fb866ffd700 10 client.40748.objecter ms_handle_connect 0x7fb85c062c90 2024-02-18T00:35:52.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.365+0000 7fb8752f3700 1 --2- 172.21.15.59:0/928846527 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb85c062c90 0x7fb85c065150 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 rx=0x7fb860004e20 tx=0x7fb860007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:52.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.369+0000 7fb866ffd700 1 -- 172.21.15.59:0/928846527 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb86c049f60 con 0x7fb8700f8420 2024-02-18T00:35:52.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.622+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb87005fb10 con 0x7fb8700f8420 2024-02-18T00:35:52.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.622+0000 7fb866ffd700 1 -- 172.21.15.59:0/928846527 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2539 (secure 0 0 0) 0x7fb86c0500d0 con 0x7fb8700f8420 2024-02-18T00:35:52.624 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:35:52.624 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":625,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":273,"num_osds":8,"num_up_osds":8,"osd_up_since":1708216146,"num_in_osds":7,"osd_in_since":1708216427,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":217},{"state_name":"creating+peering","count":48}],"num_pgs":265,"num_pools":12,"num_objects":360,"data_bytes":9051,"bytes_used":2237628416,"bytes_avail":669727621120,"bytes_total":671965249536,"inactive_pgs_ratio":0.18113207817077637,"read_bytes_sec":1210,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.59:8443/","prometheus":"http://172.21.15.59:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-18T00:33:31.796486+0000","services":{"rgw":{"daemons":{"summary":"","24347":{"start_epoch":12,"start_stamp":"2024-02-18T00:29:22.198518+0000","gid":24347,"addr":"172.21.15.59:0/2522576766","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)","ceph_version_short":"16.2.14-548-g81bd20d6","container_hostname":"smithi059","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi059","id":"foo.a","kernel_description":"#1 SMP Mon Feb 5 18:34:56 UTC 2024","kernel_version":"4.18.0-539.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543852","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"e67bc818-d879-487a-b572-f16f51d463f9","zone_name":"default","zonegroup_id":"1a312463-4a64-495f-95ab-5ad353fdc6fb","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-18T00:35:52.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 10 librados: watch_flush enter 2024-02-18T00:35:52.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 10 librados: watch_flush exit 2024-02-18T00:35:52.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 20 client.40748.objecter shutdown clearing up homeless session... 2024-02-18T00:35:52.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 10 client.40748.objecter successfully canceled tick 2024-02-18T00:35:52.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb85c062c90 msgr2=0x7fb85c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 --2- 172.21.15.59:0/928846527 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb85c062c90 0x7fb85c065150 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 rx=0x7fb860004e20 tx=0x7fb860007f90).stop 2024-02-18T00:35:52.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700f8420 msgr2=0x7fb87006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700f8420 0x7fb87006b0d0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 rx=0x7fb86c009400 tx=0x7fb86c009cb0).stop 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 shutdown_connections 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8700fe510 0x7fb87006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8700f8420 0x7fb87006b0d0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 --2- 172.21.15.59:0/928846527 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb85c062c90 0x7fb85c065150 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 --2- 172.21.15.59:0/928846527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8700fdbb0 0x7fb87006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 >> 172.21.15.59:0/928846527 conn(0x7fb8700f3800 msgr2=0x7fb8700f52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 shutdown_connections 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.624+0000 7fb877d58700 1 -- 172.21.15.59:0/928846527 wait complete. 2024-02-18T00:35:52.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.625+0000 7fb877d58700 1 librados: shutdown 2024-02-18T00:35:52.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.661+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fa7f4107290 con 0x7fa7f4102210 2024-02-18T00:35:52.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:35:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.973+0000 7fa7ea7fc700 1 -- 172.21.15.59:0/615868746 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v274) v1 ==== 91+0+0 (secure 0 0 0) 0x7fa7ec050c50 con 0x7fa7f4102210 2024-02-18T00:35:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.976+0000 7fa7fb44f700 10 librados: watch_flush enter 2024-02-18T00:35:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.976+0000 7fa7fb44f700 10 librados: watch_flush exit 2024-02-18T00:35:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.976+0000 7fa7fb44f700 20 client.40754.objecter shutdown clearing up homeless session... 2024-02-18T00:35:52.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.976+0000 7fa7fb44f700 10 client.40754.objecter successfully canceled tick 2024-02-18T00:35:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 msgr2=0x7fa7e0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/615868746 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 secure :-1 s=READY pgs=3043 cs=0 l=1 rev1=1 rx=0x7fa7f0004c60 tx=0x7fa7f000b040).stop 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 msgr2=0x7fa7f41083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 0x7fa7f41083f0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 rx=0x7fa7ec001df0 tx=0x7fa7ec004c50).stop 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 shutdown_connections 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/615868746 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa7e0062c90 0x7fa7e0065150 unknown :-1 s=CLOSED pgs=3043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7f40f8cb0 0x7fa7f410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7f4102210 0x7fa7f41083f0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 --2- 172.21.15.59:0/615868746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7f40f9610 0x7fa7f410e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.979+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 >> 172.21.15.59:0/615868746 conn(0x7fa7f40f2820 msgr2=0x7fa7f40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:52.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.980+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 shutdown_connections 2024-02-18T00:35:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.980+0000 7fa7fb44f700 1 -- 172.21.15.59:0/615868746 wait complete. 2024-02-18T00:35:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:52.980+0000 7fa7fb44f700 1 librados: shutdown 2024-02-18T00:35:52.987 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:35:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:51.856790+0000 mgr.y (mgr.24454) 291 : cluster [DBG] pgmap v396: 265 pgs: 48 creating+peering, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:35:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.974513+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:51.974648+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e273: 8 total, 8 up, 7 in 2024-02-18T00:35:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.978109+0000 mon.b (mon.1) 344 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.978783+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.980478+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.982047+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.984295+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:51.984695+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.623767+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.59:0/928846527' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.662753+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.59:0/615868746' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.663081+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.781472+0000 mon.b (mon.1) 345 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:51.856790+0000 mgr.y (mgr.24454) 291 : cluster [DBG] pgmap v396: 265 pgs: 48 creating+peering, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.974513+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:51.974648+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e273: 8 total, 8 up, 7 in 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.978109+0000 mon.b (mon.1) 344 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.978783+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.980478+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.982047+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.984295+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:51.984695+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.623767+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.59:0/928846527' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.662753+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.59:0/615868746' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.663081+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.781472+0000 mon.b (mon.1) 345 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:51.856790+0000 mgr.y (mgr.24454) 291 : cluster [DBG] pgmap v396: 265 pgs: 48 creating+peering, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.974513+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:51.974648+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e273: 8 total, 8 up, 7 in 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.978109+0000 mon.b (mon.1) 344 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.978783+0000 mon.c (mon.2) 233 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.980478+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.982047+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.984295+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:51.984695+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.623767+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.59:0/928846527' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.662753+0000 mon.c (mon.2) 236 : audit [INF] from='client.? 172.21.15.59:0/615868746' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.663081+0000 mon.a (mon.0) 1891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:35:53.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.781472+0000 mon.b (mon.1) 345 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:53.953 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:53.956 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:53.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:54.058 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:54.091 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:54.112 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:54.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:54.115 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:54.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:54.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:52.968719+0000 mon.a (mon.0) 1892 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:54.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.973593+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]': finished 2024-02-18T00:35:54.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.973691+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:54.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.973790+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:54.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.973870+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:52.973938+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e274: 8 total, 8 up, 7 in 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.975422+0000 mon.b (mon.1) 346 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.976252+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.979203+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.979431+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.979987+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]: dispatch 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:52.985863+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]: dispatch 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.396919+0000 osd.4 (osd.4) 155 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.400435+0000 osd.4 (osd.4) 156 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:35:54.246 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.617447+0000 mon.a (mon.0) 1901 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:52.968719+0000 mon.a (mon.0) 1892 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.973593+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]': finished 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.973691+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.973790+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.973870+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:52.973938+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e274: 8 total, 8 up, 7 in 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.975422+0000 mon.b (mon.1) 346 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.976252+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.979203+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.979431+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.979987+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]: dispatch 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:52.985863+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]: dispatch 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.396919+0000 osd.4 (osd.4) 155 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.400435+0000 osd.4 (osd.4) 156 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:35:54.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.617447+0000 mon.a (mon.0) 1901 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.621770+0000 mon.a (mon.0) 1902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.621853+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.621916+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]': finished 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.621974+0000 mon.a (mon.0) 1905 : cluster [DBG] osdmap e275: 8 total, 8 up, 7 in 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.651939+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.653212+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.653581+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi059-82882-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.654334+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.654823+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:54.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.655147+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi059-82877-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.734453+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.734758+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:53.782609+0000 mon.b (mon.1) 347 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.621770+0000 mon.a (mon.0) 1902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.621853+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.621916+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]': finished 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.621974+0000 mon.a (mon.0) 1905 : cluster [DBG] osdmap e275: 8 total, 8 up, 7 in 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.651939+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.653212+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.653581+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi059-82882-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:54.249 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.654334+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:54.250 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.654823+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:54.250 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.655147+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi059-82877-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:54.250 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.734453+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:54.250 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.734758+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:54.250 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:53.782609+0000 mon.b (mon.1) 347 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:52.968719+0000 mon.a (mon.0) 1892 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.973593+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-31"}]': finished 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.973691+0000 mon.a (mon.0) 1894 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.973790+0000 mon.a (mon.0) 1895 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.973870+0000 mon.a (mon.0) 1896 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:52.973938+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e274: 8 total, 8 up, 7 in 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.975422+0000 mon.b (mon.1) 346 : audit [INF] from='client.? 172.21.15.59:0/2208826121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.976252+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.59:0/2131198647' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.979203+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]: dispatch 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.979431+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]: dispatch 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.979987+0000 mon.c (mon.2) 238 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]: dispatch 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:52.985863+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]: dispatch 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.396919+0000 osd.4 (osd.4) 155 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.400435+0000 osd.4 (osd.4) 156 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.617447+0000 mon.a (mon.0) 1901 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.621770+0000 mon.a (mon.0) 1902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi059-82882-47"}]': finished 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.621853+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi059-82877-38"}]': finished 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.621916+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-31", "mode": "writeback"}]': finished 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.621974+0000 mon.a (mon.0) 1905 : cluster [DBG] osdmap e275: 8 total, 8 up, 7 in 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.651939+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.653212+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:54.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.653581+0000 mon.a (mon.0) 1908 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi059-82882-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.654334+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.654823+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.655147+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi059-82877-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:35:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.734453+0000 mon.c (mon.2) 239 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.734758+0000 mon.a (mon.0) 1912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:35:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:53.782609+0000 mon.b (mon.1) 347 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:54.448 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:54.451 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:54.451 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:54.552 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:54.584 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:54.605 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:54.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:54.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:54.709 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:54.744 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:54.760 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:54.763 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:54.763 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:54.864 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:54.891 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:54.911 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:54.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:54.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:55.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:55.044 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:55.070 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:55.072 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:55.073 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:55.075 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-18T00:35:55.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:35:55.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:35:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.560065+0000 osd.1 (osd.1) 137 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:35:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.563558+0000 osd.1 (osd.1) 138 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:35:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.687075+0000 osd.3 (osd.3) 153 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:35:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.688080+0000 osd.3 (osd.3) 154 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:35:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:53.857409+0000 mgr.y (mgr.24454) 292 : cluster [DBG] pgmap v400: 249 pgs: 32 creating+peering, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.184501+0000 osd.0 (osd.0) 127 : cluster [DBG] 4.b scrub starts 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.188184+0000 osd.0 (osd.0) 128 : cluster [DBG] 4.b scrub ok 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.626136+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi059-82882-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.626244+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi059-82877-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.626308+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.626365+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e276: 8 total, 8 up, 7 in 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.629089+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi059-82877-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.629286+0000 mon.c (mon.2) 240 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.629597+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi059-82882-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.632271+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.663374+0000 osd.2 (osd.2) 115 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.664876+0000 osd.2 (osd.2) 116 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:35:55.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.783907+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:54.898700+0000 mon.a (mon.0) 1920 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.560065+0000 osd.1 (osd.1) 137 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.563558+0000 osd.1 (osd.1) 138 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.687075+0000 osd.3 (osd.3) 153 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.688080+0000 osd.3 (osd.3) 154 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:53.857409+0000 mgr.y (mgr.24454) 292 : cluster [DBG] pgmap v400: 249 pgs: 32 creating+peering, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.184501+0000 osd.0 (osd.0) 127 : cluster [DBG] 4.b scrub starts 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.188184+0000 osd.0 (osd.0) 128 : cluster [DBG] 4.b scrub ok 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.626136+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi059-82882-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.626244+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi059-82877-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.626308+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:55.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.626365+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e276: 8 total, 8 up, 7 in 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.629089+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi059-82877-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.629286+0000 mon.c (mon.2) 240 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.629597+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi059-82882-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.632271+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.663374+0000 osd.2 (osd.2) 115 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.664876+0000 osd.2 (osd.2) 116 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.783907+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:55.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:54.898700+0000 mon.a (mon.0) 1920 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.560065+0000 osd.1 (osd.1) 137 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.563558+0000 osd.1 (osd.1) 138 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.687075+0000 osd.3 (osd.3) 153 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.688080+0000 osd.3 (osd.3) 154 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:53.857409+0000 mgr.y (mgr.24454) 292 : cluster [DBG] pgmap v400: 249 pgs: 32 creating+peering, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.184501+0000 osd.0 (osd.0) 127 : cluster [DBG] 4.b scrub starts 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.188184+0000 osd.0 (osd.0) 128 : cluster [DBG] 4.b scrub ok 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.626136+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi059-82882-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.626244+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi059-82877-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.626308+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.626365+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e276: 8 total, 8 up, 7 in 2024-02-18T00:35:55.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.629089+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi059-82877-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:55.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.629286+0000 mon.c (mon.2) 240 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:55.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.629597+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi059-82882-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:55.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.632271+0000 mon.a (mon.0) 1919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]: dispatch 2024-02-18T00:35:55.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.663374+0000 osd.2 (osd.2) 115 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:35:55.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.664876+0000 osd.2 (osd.2) 116 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:35:55.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.783907+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:55.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:54.898700+0000 mon.a (mon.0) 1920 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:35:55.694 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:55.977 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:35:56.051 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:35:56.055 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:56.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:35:56.156 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:56.187 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:56.209 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:56.212 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:56.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:56.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:35:56.344 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:56.365 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:35:56.367 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:56.368 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:35:56.468 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:35:56.497 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:56.516 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:35:56.519 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:56.519 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:35:56.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:56.670 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.567963+0000 osd.1 (osd.1) 139 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:35:56.670 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.569029+0000 osd.1 (osd.1) 140 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:35:56.670 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.735857+0000 osd.3 (osd.3) 155 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.736816+0000 osd.3 (osd.3) 156 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.835842+0000 osd.5 (osd.5) 121 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:54.836730+0000 osd.5 (osd.5) 122 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.620749+0000 mon.a (mon.0) 1921 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:56.517804+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]': finished 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.567963+0000 osd.1 (osd.1) 139 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.569029+0000 osd.1 (osd.1) 140 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.735857+0000 osd.3 (osd.3) 155 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:35:56.671 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.736816+0000 osd.3 (osd.3) 156 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:35:56.672 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.835842+0000 osd.5 (osd.5) 121 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:35:56.672 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:54.836730+0000 osd.5 (osd.5) 122 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:35:56.672 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.620749+0000 mon.a (mon.0) 1921 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:56.672 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:56.517804+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]': finished 2024-02-18T00:35:56.673 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:56.726 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:35:56.726 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:35:56.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.567963+0000 osd.1 (osd.1) 139 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:35:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.569029+0000 osd.1 (osd.1) 140 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:35:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.735857+0000 osd.3 (osd.3) 155 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:35:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.736816+0000 osd.3 (osd.3) 156 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:35:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.835842+0000 osd.5 (osd.5) 121 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:35:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:54.836730+0000 osd.5 (osd.5) 122 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:35:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.620749+0000 mon.a (mon.0) 1921 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:35:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:56.517804+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-31"}]': finished 2024-02-18T00:35:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.236797+0000 osd.0 (osd.0) 129 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:35:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.237917+0000 osd.0 (osd.0) 130 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:35:58.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.392613+0000 osd.4 (osd.4) 157 : cluster [DBG] 25.b scrub starts 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.394534+0000 osd.6 (osd.6) 137 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.395662+0000 osd.4 (osd.4) 158 : cluster [DBG] 25.b scrub ok 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.396067+0000 osd.6 (osd.6) 138 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.536197+0000 osd.1 (osd.1) 141 : cluster [DBG] 26.8 scrub starts 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.537413+0000 osd.1 (osd.1) 142 : cluster [DBG] 26.8 scrub ok 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:55.785574+0000 mon.b (mon.1) 349 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.858228+0000 mgr.y (mgr.24454) 293 : cluster [DBG] pgmap v403: 249 pgs: 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 526 B/s wr, 4 op/s 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.886271+0000 osd.5 (osd.5) 123 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:55.889239+0000 osd.5 (osd.5) 124 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.190381+0000 osd.0 (osd.0) 131 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.193318+0000 osd.0 (osd.0) 132 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.402183+0000 osd.4 (osd.4) 159 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:35:58.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.405087+0000 osd.4 (osd.4) 160 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.422065+0000 osd.6 (osd.6) 139 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.424852+0000 osd.6 (osd.6) 140 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.517921+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e277: 8 total, 8 up, 7 in 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:56.631438+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi059-82877-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:56.631540+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi059-82882-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.631634+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e278: 8 total, 8 up, 7 in 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.755556+0000 osd.3 (osd.3) 157 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.756381+0000 osd.3 (osd.3) 158 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:56.786753+0000 mon.b (mon.1) 350 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:56.787426+0000 mon.b (mon.1) 351 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.921692+0000 osd.5 (osd.5) 125 : cluster [DBG] 25.c scrub starts 2024-02-18T00:35:58.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:56.924601+0000 osd.5 (osd.5) 126 : cluster [DBG] 25.c scrub ok 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.098241+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.236797+0000 osd.0 (osd.0) 129 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.237917+0000 osd.0 (osd.0) 130 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.392613+0000 osd.4 (osd.4) 157 : cluster [DBG] 25.b scrub starts 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.394534+0000 osd.6 (osd.6) 137 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.395662+0000 osd.4 (osd.4) 158 : cluster [DBG] 25.b scrub ok 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.396067+0000 osd.6 (osd.6) 138 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.536197+0000 osd.1 (osd.1) 141 : cluster [DBG] 26.8 scrub starts 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.537413+0000 osd.1 (osd.1) 142 : cluster [DBG] 26.8 scrub ok 2024-02-18T00:35:58.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:55.785574+0000 mon.b (mon.1) 349 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.858228+0000 mgr.y (mgr.24454) 293 : cluster [DBG] pgmap v403: 249 pgs: 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 526 B/s wr, 4 op/s 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.886271+0000 osd.5 (osd.5) 123 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:55.889239+0000 osd.5 (osd.5) 124 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.190381+0000 osd.0 (osd.0) 131 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.193318+0000 osd.0 (osd.0) 132 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.402183+0000 osd.4 (osd.4) 159 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.405087+0000 osd.4 (osd.4) 160 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.422065+0000 osd.6 (osd.6) 139 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.424852+0000 osd.6 (osd.6) 140 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.517921+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e277: 8 total, 8 up, 7 in 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:56.631438+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi059-82877-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:56.631540+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi059-82882-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.631634+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e278: 8 total, 8 up, 7 in 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.755556+0000 osd.3 (osd.3) 157 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:35:58.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.756381+0000 osd.3 (osd.3) 158 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:56.786753+0000 mon.b (mon.1) 350 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:56.787426+0000 mon.b (mon.1) 351 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.921692+0000 osd.5 (osd.5) 125 : cluster [DBG] 25.c scrub starts 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:56.924601+0000 osd.5 (osd.5) 126 : cluster [DBG] 25.c scrub ok 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.098241+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.236797+0000 osd.0 (osd.0) 129 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.237917+0000 osd.0 (osd.0) 130 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:35:58.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.392613+0000 osd.4 (osd.4) 157 : cluster [DBG] 25.b scrub starts 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.394534+0000 osd.6 (osd.6) 137 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.395662+0000 osd.4 (osd.4) 158 : cluster [DBG] 25.b scrub ok 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.396067+0000 osd.6 (osd.6) 138 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.536197+0000 osd.1 (osd.1) 141 : cluster [DBG] 26.8 scrub starts 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.537413+0000 osd.1 (osd.1) 142 : cluster [DBG] 26.8 scrub ok 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:55.785574+0000 mon.b (mon.1) 349 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.858228+0000 mgr.y (mgr.24454) 293 : cluster [DBG] pgmap v403: 249 pgs: 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 526 B/s wr, 4 op/s 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.886271+0000 osd.5 (osd.5) 123 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:55.889239+0000 osd.5 (osd.5) 124 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.190381+0000 osd.0 (osd.0) 131 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.193318+0000 osd.0 (osd.0) 132 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.402183+0000 osd.4 (osd.4) 159 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:35:58.008 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.405087+0000 osd.4 (osd.4) 160 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.422065+0000 osd.6 (osd.6) 139 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.424852+0000 osd.6 (osd.6) 140 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.517921+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e277: 8 total, 8 up, 7 in 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:56.631438+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi059-82877-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:56.631540+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi059-82882-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.631634+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e278: 8 total, 8 up, 7 in 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.755556+0000 osd.3 (osd.3) 157 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.756381+0000 osd.3 (osd.3) 158 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:56.786753+0000 mon.b (mon.1) 350 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:56.787426+0000 mon.b (mon.1) 351 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.921692+0000 osd.5 (osd.5) 125 : cluster [DBG] 25.c scrub starts 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:56.924601+0000 osd.5 (osd.5) 126 : cluster [DBG] 25.c scrub ok 2024-02-18T00:35:58.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.098241+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T00:35:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:57.223262+0000 osd.0 (osd.0) 133 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:35:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:57.224338+0000 osd.0 (osd.0) 134 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:35:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:57.469673+0000 osd.6 (osd.6) 141 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:35:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:57.471692+0000 osd.6 (osd.6) 142 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:35:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.635979+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]': finished 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:57.636099+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e279: 8 total, 8 up, 7 in 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.649067+0000 mon.b (mon.1) 352 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.649936+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.652426+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.652619+0000 mon.b (mon.1) 354 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.653280+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:57.654550+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:57.858738+0000 mgr.y (mgr.24454) 294 : cluster [DBG] pgmap v406: 265 pgs: 48 unknown, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:57.223262+0000 osd.0 (osd.0) 133 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:35:59.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:57.224338+0000 osd.0 (osd.0) 134 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:57.469673+0000 osd.6 (osd.6) 141 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:57.471692+0000 osd.6 (osd.6) 142 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.635979+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]': finished 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:57.636099+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e279: 8 total, 8 up, 7 in 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.649067+0000 mon.b (mon.1) 352 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.649936+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.652426+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.652619+0000 mon.b (mon.1) 354 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.653280+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:57.654550+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:35:59.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:57.858738+0000 mgr.y (mgr.24454) 294 : cluster [DBG] pgmap v406: 265 pgs: 48 unknown, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:57.223262+0000 osd.0 (osd.0) 133 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:57.224338+0000 osd.0 (osd.0) 134 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:57.469673+0000 osd.6 (osd.6) 141 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:57.471692+0000 osd.6 (osd.6) 142 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.635979+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pg_num","val":"11"}]': finished 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:57.636099+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e279: 8 total, 8 up, 7 in 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.649067+0000 mon.b (mon.1) 352 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.649936+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.652426+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.652619+0000 mon.b (mon.1) 354 : audit [INF] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:35:59.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.653280+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:35:59.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:57.654550+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T00:35:59.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:57.858738+0000 mgr.y (mgr.24454) 294 : cluster [DBG] pgmap v406: 265 pgs: 48 unknown, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:35:59.326 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:59.329 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:59.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:59.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:59.462 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:59.613 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:59.616 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:59.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:59.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:35:59.749 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:59.769 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:35:59.772 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:59.772 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:35:59.782 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:35:59.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:35:59.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.878+0000 7f2754f56700 1 -- 172.21.15.59:0/37873276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27500f73f0 msgr2=0x7f2750104190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:59.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.878+0000 7f2754f56700 1 --2- 172.21.15.59:0/37873276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27500f73f0 0x7f2750104190 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 rx=0x7f2738004870 tx=0x7f2738009cb0).stop 2024-02-18T00:35:59.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.878+0000 7f2754f56700 1 -- 172.21.15.59:0/37873276 shutdown_connections 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.878+0000 7f2754f56700 1 --2- 172.21.15.59:0/37873276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2750104680 0x7f2750106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.878+0000 7f2754f56700 1 --2- 172.21.15.59:0/37873276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 0x7f27500f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.878+0000 7f2754f56700 1 --2- 172.21.15.59:0/37873276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27500f73f0 0x7f2750104190 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.878+0000 7f2754f56700 1 -- 172.21.15.59:0/37873276 >> 172.21.15.59:0/37873276 conn(0x7f27500f2820 msgr2=0x7f27500f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 -- 172.21.15.59:0/37873276 shutdown_connections 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 -- 172.21.15.59:0/37873276 wait complete. 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 librados: starting msgr at 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 librados: starting objecter 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 Processor -- start 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 -- start start 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 librados: setting wanted keys 2024-02-18T00:35:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.879+0000 7f2754f56700 1 librados: calling monclient init 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 0x7f2750110370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27500f73f0 0x7f2750110860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2750104680 0x7f275010a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275010aa00 con 0x7f27500f6b20 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275010ab70 con 0x7f27500f73f0 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275010ad30 con 0x7f2750104680 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 0x7f2750110370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 0x7f2750110370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47928/0 (socket says 172.21.15.59:47928) 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 -- 172.21.15.59:0/2965523497 learned_addr learned my addr 172.21.15.59:0/2965523497 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274ed9d700 1 --2- 172.21.15.59:0/2965523497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2750104680 0x7f275010a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274dd9b700 1 --2- 172.21.15.59:0/2965523497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27500f73f0 0x7f2750110860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 -- 172.21.15.59:0/2965523497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2750104680 msgr2=0x7f275010a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2750104680 0x7f275010a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 -- 172.21.15.59:0/2965523497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27500f73f0 msgr2=0x7f2750110860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27500f73f0 0x7f2750110860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 -- 172.21.15.59:0/2965523497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27480097c0 con 0x7f27500f6b20 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 0x7f2750110370 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 rx=0x7f2748001e70 tx=0x7f2748004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f27477fe700 10 client.?.objecter ms_handle_connect 0x7f27500f6b20 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f27477fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27480099f0 con 0x7f27500f6b20 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2738005040 con 0x7f27500f6b20 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27480090d0 con 0x7f27500f6b20 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748007c70 con 0x7f27500f6b20 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 10 client.30957.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27501171a0 con 0x7f27500f6b20 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.880+0000 7f2754f56700 1 librados: init done 2024-02-18T00:35:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.881+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f275005fa60 con 0x7f27500f6b20 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.882+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f274800fad0 con 0x7f27500f6b20 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.882+0000 7f27477fe700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6800/3721891797 conn(0x7f273c062c90 0x7f273c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(281..281 src has 44..281) v4 ==== 8592+0+0 (secure 0 0 0) 0x7f274807e1d0 con 0x7f27500f6b20 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 osd_map(281..281 src has 44..281) v4 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 3 client.30957.objecter handle_osd_map got epochs [281,281] > 0 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 3 client.30957.objecter handle_osd_map decoding full epoch 281 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f274dd9b700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6800/3721891797 conn(0x7f273c062c90 0x7f273c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 20 client.30957.objecter dump_active .. 0 homeless 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v281) v1 ==== 54+0+16 (secure 0 0 0) 0x7f274807e5f0 con 0x7f27500f6b20 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f273c062c90 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f274dd9b700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6800/3721891797 conn(0x7f273c062c90 0x7f273c065150 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 rx=0x7f2738009a00 tx=0x7f273800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 10 client.30957.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:59.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 1 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6801/3700238977 conn(0x7f27500fe660 0x7f27500fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 1 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 0 1 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275005ee80 osd=0 4 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 10 client.30957.objecter _send_command 1 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27501093f0 con 0x7f27500fe660 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f274ed9d700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6801/3700238977 conn(0x7f27500fe660 0x7f27500fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f274ed9d700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6801/3700238977 conn(0x7f27500fe660 0x7f27500fea40 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.883+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f27500fe660 2024-02-18T00:35:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.884+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f27501093f0 con 0x7f27500fe660 2024-02-18T00:35:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.884+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:35:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.884+0000 7f27477fe700 10 client.30957.objecter _finish_command 1 = system:0 2024-02-18T00:35:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.884+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275005ee80 osd=0 4 2024-02-18T00:35:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.884+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 0 1 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 10 client.30957.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 2 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 2 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275005ee80 osd=0 3 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 0 2 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275005ee80 osd=0 4 2024-02-18T00:35:59.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 10 client.30957.objecter _send_command 2 2024-02-18T00:35:59.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.899+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f27500273c0 con 0x7f27500fe660 2024-02-18T00:35:59.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f27500273c0 con 0x7f27500fe660 2024-02-18T00:35:59.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:59.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f27477fe700 10 client.30957.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:59.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275005ee80 osd=0 4 2024-02-18T00:35:59.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 0 2 2024-02-18T00:35:59.903 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 10 client.30957.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 3 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6805/26279110 conn(0x7f27500598f0 0x7f2750059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 3 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 1 3 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750028710 osd=1 4 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 10 client.30957.objecter _send_command 3 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f275011dab0 con 0x7f27500598f0 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6805/26279110 conn(0x7f27500598f0 0x7f2750059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f27500598f0 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.901+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6805/26279110 conn(0x7f27500598f0 0x7f2750059cf0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.902+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f274805dac0 con 0x7f27500598f0 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.902+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.902+0000 7f27477fe700 10 client.30957.objecter _finish_command 3 = system:0 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.902+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f2750028710 osd=1 4 2024-02-18T00:35:59.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.902+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 1 3 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:58.228824+0000 osd.0 (osd.0) 135 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:58.230111+0000 osd.0 (osd.0) 136 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:58.642299+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:58.642401+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:58.642526+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e280: 8 total, 8 up, 7 in 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:58.645140+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:58.645789+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:58.648616+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:58.648671+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:35:59.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:58.648823+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:35:59.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:58.697939+0000 osd.2 (osd.2) 117 : cluster [DBG] 26.4 scrub starts 2024-02-18T00:35:59.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:58.726405+0000 osd.2 (osd.2) 118 : cluster [DBG] 26.4 scrub ok 2024-02-18T00:35:59.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:58.959559+0000 osd.5 (osd.5) 127 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:35:59.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:35:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:58.960446+0000 osd.5 (osd.5) 128 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:35:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 10 client.30957.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 4 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 4 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f2750028710 osd=1 3 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 1 4 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750028710 osd=1 4 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 10 client.30957.objecter _send_command 4 2024-02-18T00:35:59.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.917+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2750059360 con 0x7f27500598f0 2024-02-18T00:35:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2750059360 con 0x7f27500598f0 2024-02-18T00:35:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f27477fe700 10 client.30957.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f2750028710 osd=1 4 2024-02-18T00:35:59.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 1 4 2024-02-18T00:35:59.921 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:35:59.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 10 client.30957.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:59.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 5 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2750028270 0x7f275011c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 5 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 2 5 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750059770 osd=2 4 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 10 client.30957.objecter _send_command 5 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f275011c9c0 con 0x7f2750028270 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f274dd9b700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2750028270 0x7f275011c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f2750028270 2024-02-18T00:35:59.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.919+0000 7f274dd9b700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2750028270 0x7f275011c4d0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.920+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f275011c9c0 con 0x7f2750028270 2024-02-18T00:35:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.920+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:35:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.920+0000 7f27477fe700 10 client.30957.objecter _finish_command 5 = system:0 2024-02-18T00:35:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.920+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f2750059770 osd=2 4 2024-02-18T00:35:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.920+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 2 5 2024-02-18T00:35:59.927 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:35:59.930 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:35:59.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:35:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 10 client.30957.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 6 2024-02-18T00:35:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 6 2024-02-18T00:35:59.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f2750059770 osd=2 3 2024-02-18T00:35:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 2 6 2024-02-18T00:35:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750059770 osd=2 4 2024-02-18T00:35:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 10 client.30957.objecter _send_command 6 2024-02-18T00:35:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.934+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f27500277d0 con 0x7f2750028270 2024-02-18T00:35:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f27500277d0 con 0x7f2750028270 2024-02-18T00:35:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:59.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f27477fe700 10 client.30957.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f2750059770 osd=2 4 2024-02-18T00:35:59.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 2 6 2024-02-18T00:35:59.938 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 10 client.30957.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 7 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6813/1805700964 conn(0x7f275011fc30 0x7f2750122020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 7 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 3 7 2024-02-18T00:35:59.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750027be0 osd=3 4 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 10 client.30957.objecter _send_command 7 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f275011d8c0 con 0x7f275011fc30 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f274ed9d700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6813/1805700964 conn(0x7f275011fc30 0x7f2750122020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f275011fc30 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.936+0000 7f274ed9d700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6813/1805700964 conn(0x7f275011fc30 0x7f2750122020 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.937+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f275011d8c0 con 0x7f275011fc30 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.937+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.937+0000 7f27477fe700 10 client.30957.objecter _finish_command 7 = system:0 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.937+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f2750027be0 osd=3 4 2024-02-18T00:35:59.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.937+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 3 7 2024-02-18T00:35:59.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 10 client.30957.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:59.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 8 2024-02-18T00:35:59.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 8 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f2750027be0 osd=3 3 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 3 8 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750027be0 osd=3 4 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 10 client.30957.objecter _send_command 8 2024-02-18T00:35:59.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.952+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f275011ce20 con 0x7f275011fc30 2024-02-18T00:35:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f275011ce20 con 0x7f275011fc30 2024-02-18T00:35:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f27477fe700 10 client.30957.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f2750027be0 osd=3 4 2024-02-18T00:35:59.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 3 8 2024-02-18T00:35:59.955 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:35:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 10 client.30957.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 9 2024-02-18T00:35:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6800/2044387056 conn(0x7f275011cfc0 0x7f27501256a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 9 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 4 9 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011ce20 osd=4 4 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 10 client.30957.objecter _send_command 9 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.953+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f275011c0a0 con 0x7f275011cfc0 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.954+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6800/2044387056 conn(0x7f275011cfc0 0x7f27501256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.954+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6800/2044387056 conn(0x7f275011cfc0 0x7f27501256a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.954+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f275011cfc0 2024-02-18T00:35:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.955+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f275011c0a0 con 0x7f275011cfc0 2024-02-18T00:35:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.955+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:35:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.955+0000 7f27477fe700 10 client.30957.objecter _finish_command 9 = system:0 2024-02-18T00:35:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.955+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275011ce20 osd=4 4 2024-02-18T00:35:59.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.955+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 4 9 2024-02-18T00:35:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 10 client.30957.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 10 2024-02-18T00:35:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 10 2024-02-18T00:35:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275011ce20 osd=4 3 2024-02-18T00:35:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 4 10 2024-02-18T00:35:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011ce20 osd=4 4 2024-02-18T00:35:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 10 client.30957.objecter _send_command 10 2024-02-18T00:35:59.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.970+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f275011c0a0 con 0x7f275011cfc0 2024-02-18T00:35:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f275011c0a0 con 0x7f275011cfc0 2024-02-18T00:35:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f27477fe700 10 client.30957.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275011ce20 osd=4 4 2024-02-18T00:35:59.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 4 10 2024-02-18T00:35:59.974 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:35:59.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 10 client.30957.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 11 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6804/2170432263 conn(0x7f27501260a0 0x7f2750128490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 11 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 5 11 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011c0a0 osd=5 4 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 10 client.30957.objecter _send_command 11 2024-02-18T00:35:59.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.972+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f275012b180 con 0x7f27501260a0 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.973+0000 7f274dd9b700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6804/2170432263 conn(0x7f27501260a0 0x7f2750128490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.973+0000 7f274dd9b700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6804/2170432263 conn(0x7f27501260a0 0x7f2750128490 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.973+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f27501260a0 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.974+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f275012b180 con 0x7f27501260a0 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.974+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.974+0000 7f27477fe700 10 client.30957.objecter _finish_command 11 = system:0 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.974+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275011c0a0 osd=5 4 2024-02-18T00:35:59.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.974+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 5 11 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 10 client.30957.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 12 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 12 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275011c0a0 osd=5 3 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 5 12 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011c0a0 osd=5 4 2024-02-18T00:35:59.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 10 client.30957.objecter _send_command 12 2024-02-18T00:35:59.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.988+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f275012b180 con 0x7f27501260a0 2024-02-18T00:35:59.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f275012b180 con 0x7f27501260a0 2024-02-18T00:35:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:35:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f27477fe700 10 client.30957.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:35:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275011c0a0 osd=5 4 2024-02-18T00:35:59.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 5 12 2024-02-18T00:35:59.992 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 10 client.30957.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 13 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6808/3356009390 conn(0x7f275012a4a0 0x7f275012a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012b180 osd=6 3 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012b180 osd=6 3 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012b180 osd=6 3 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 13 2024-02-18T00:35:59.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275012b180 osd=6 3 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 6 13 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012b180 osd=6 4 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 10 client.30957.objecter _send_command 13 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f275012af70 con 0x7f275012a4a0 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.990+0000 7f274ed9d700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6808/3356009390 conn(0x7f275012a4a0 0x7f275012a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.991+0000 7f274ed9d700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6808/3356009390 conn(0x7f275012a4a0 0x7f275012a900 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.991+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f275012a4a0 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.992+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f275012af70 con 0x7f275012a4a0 2024-02-18T00:35:59.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.992+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:35:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.992+0000 7f27477fe700 10 client.30957.objecter _finish_command 13 = system:0 2024-02-18T00:35:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.992+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275012b180 osd=6 4 2024-02-18T00:35:59.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:35:59.992+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 6 13 2024-02-18T00:36:00.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:58.228824+0000 osd.0 (osd.0) 135 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:58.230111+0000 osd.0 (osd.0) 136 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:58.642299+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:58.642401+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:58.642526+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e280: 8 total, 8 up, 7 in 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:58.645140+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:58.645789+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:58.648616+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:58.648671+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:58.648823+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:58.697939+0000 osd.2 (osd.2) 117 : cluster [DBG] 26.4 scrub starts 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:58.726405+0000 osd.2 (osd.2) 118 : cluster [DBG] 26.4 scrub ok 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:58.959559+0000 osd.5 (osd.5) 127 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:36:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:58.960446+0000 osd.5 (osd.5) 128 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:58.228824+0000 osd.0 (osd.0) 135 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:58.230111+0000 osd.0 (osd.0) 136 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:58.642299+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:58.642401+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi059-82917-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:58.642526+0000 mon.a (mon.0) 1934 : cluster [DBG] osdmap e280: 8 total, 8 up, 7 in 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:58.645140+0000 mon.c (mon.2) 242 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:58.645789+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:58.648616+0000 mon.a (mon.0) 1936 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:58.648671+0000 mon.b (mon.1) 355 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:58.648823+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:36:00.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:58.697939+0000 osd.2 (osd.2) 117 : cluster [DBG] 26.4 scrub starts 2024-02-18T00:36:00.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:58.726405+0000 osd.2 (osd.2) 118 : cluster [DBG] 26.4 scrub ok 2024-02-18T00:36:00.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:58.959559+0000 osd.5 (osd.5) 127 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:36:00.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:35:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:58.960446+0000 osd.5 (osd.5) 128 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:36:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 10 client.30957.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 14 2024-02-18T00:36:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012b180 osd=6 3 2024-02-18T00:36:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012b180 osd=6 3 2024-02-18T00:36:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012b180 osd=6 3 2024-02-18T00:36:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 14 2024-02-18T00:36:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275012b180 osd=6 3 2024-02-18T00:36:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 6 14 2024-02-18T00:36:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012b180 osd=6 4 2024-02-18T00:36:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 10 client.30957.objecter _send_command 14 2024-02-18T00:36:00.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.008+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f275012af70 con 0x7f275012a4a0 2024-02-18T00:36:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f275012af70 con 0x7f275012a4a0 2024-02-18T00:36:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f27477fe700 10 client.30957.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275012b180 osd=6 4 2024-02-18T00:36:00.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 6 14 2024-02-18T00:36:00.012 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:36:00.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 10 client.30957.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:00.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 15 2024-02-18T00:36:00.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6812/53025171 conn(0x7f275012d930 0x7f275012dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:00.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 15 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 7 15 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012af70 osd=7 4 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 10 client.30957.objecter _send_command 15 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.010+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27501317a0 con 0x7f275012d930 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.011+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6812/53025171 conn(0x7f275012d930 0x7f275012dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.011+0000 7f27477fe700 10 client.30957.objecter ms_handle_connect 0x7f275012d930 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.011+0000 7f274e59c700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6812/53025171 conn(0x7f275012d930 0x7f275012dd90 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.012+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f27501317a0 con 0x7f275012d930 2024-02-18T00:36:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.012+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:36:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.012+0000 7f27477fe700 10 client.30957.objecter _finish_command 15 = system:0 2024-02-18T00:36:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.012+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275012af70 osd=7 4 2024-02-18T00:36:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.012+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 7 15 2024-02-18T00:36:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 10 client.30957.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign -1 16 2024-02-18T00:36:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 20 client.30957.objecter _get_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 15 client.30957.objecter _session_command_op_remove -1 16 2024-02-18T00:36:00.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 20 client.30957.objecter get_session s=0x7f275012af70 osd=7 3 2024-02-18T00:36:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 15 client.30957.objecter _session_command_op_assign 7 16 2024-02-18T00:36:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012af70 osd=7 4 2024-02-18T00:36:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 10 client.30957.objecter _send_command 16 2024-02-18T00:36:00.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.026+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2750128980 con 0x7f275012d930 2024-02-18T00:36:00.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.028+0000 7f27477fe700 1 -- 172.21.15.59:0/2965523497 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2750128980 con 0x7f275012d930 2024-02-18T00:36:00.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.028+0000 7f27477fe700 10 client.30957.objecter ms_dispatch 0x7f2750001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:00.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.028+0000 7f27477fe700 10 client.30957.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.028+0000 7f27477fe700 20 client.30957.objecter put_session s=0x7f275012af70 osd=7 4 2024-02-18T00:36:00.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.028+0000 7f27477fe700 15 client.30957.objecter _session_command_op_remove 7 16 2024-02-18T00:36:00.031 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:36:00.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.028+0000 7f2754f56700 10 librados: watch_flush enter 2024-02-18T00:36:00.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.028+0000 7f2754f56700 10 librados: watch_flush exit 2024-02-18T00:36:00.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.0 2024-02-18T00:36:00.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6801/3700238977 conn(0x7f27500fe660 msgr2=0x7f27500fea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6801/3700238977 conn(0x7f27500fe660 0x7f27500fea40 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275005ee80 osd=0 1 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.1 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6805/26279110 conn(0x7f27500598f0 msgr2=0x7f2750059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6805/26279110 conn(0x7f27500598f0 0x7f2750059cf0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750028710 osd=1 1 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.2 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2750028270 msgr2=0x7f275011c4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2750028270 0x7f275011c4d0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750059770 osd=2 1 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.3 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6813/1805700964 conn(0x7f275011fc30 msgr2=0x7f2750122020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6813/1805700964 conn(0x7f275011fc30 0x7f2750122020 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f2750027be0 osd=3 1 2024-02-18T00:36:00.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.4 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6800/2044387056 conn(0x7f275011cfc0 msgr2=0x7f27501256a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6800/2044387056 conn(0x7f275011cfc0 0x7f27501256a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f274ed9d700 1 -- 172.21.15.59:0/2965523497 reap_dead start 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011ce20 osd=4 1 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.5 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f274ed9d700 1 -- 172.21.15.59:0/2965523497 reap_dead start 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6804/2170432263 conn(0x7f27501260a0 msgr2=0x7f2750128490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6804/2170432263 conn(0x7f27501260a0 0x7f2750128490 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275011c0a0 osd=5 1 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.6 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6808/3356009390 conn(0x7f275012a4a0 msgr2=0x7f275012a900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6808/3356009390 conn(0x7f275012a4a0 0x7f275012a900 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012b180 osd=6 1 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter close_session for osd.7 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f274ed9d700 1 -- 172.21.15.59:0/2965523497 reap_dead start 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6812/53025171 conn(0x7f275012d930 msgr2=0x7f275012dd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6812/53025171 conn(0x7f275012d930 0x7f275012dd90 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter put_session s=0x7f275012af70 osd=7 1 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 20 client.30957.objecter shutdown clearing up homeless session... 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 10 client.30957.objecter successfully canceled tick 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6800/3721891797 conn(0x7f273c062c90 msgr2=0x7f273c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6800/3721891797 conn(0x7f273c062c90 0x7f273c065150 secure :-1 s=READY pgs=3046 cs=0 l=1 rev1=1 rx=0x7f2738009a00 tx=0x7f273800b040).stop 2024-02-18T00:36:00.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 msgr2=0x7f2750110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 0x7f2750110370 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 rx=0x7f2748001e70 tx=0x7f2748004c50).stop 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 shutdown_connections 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.137:6812/53025171 conn(0x7f275012d930 0x7f275012dd90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> v2:172.21.15.59:6800/3721891797 conn(0x7f273c062c90 0x7f273c065150 unknown :-1 s=CLOSED pgs=3046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 --2- 172.21.15.59:0/2965523497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27500f6b20 0x7f2750110370 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 >> 172.21.15.59:0/2965523497 conn(0x7f27500f2820 msgr2=0x7f27500f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 shutdown_connections 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 -- 172.21.15.59:0/2965523497 wait complete. 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:00.029+0000 7f2754f56700 1 librados: shutdown 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.036 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.037 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.037 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.037 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.037 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:36:00.037 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:00.065 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:00.084 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:00.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:00.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:00.188 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:00.215 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:00.233 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:00.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:00.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:00.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:00.368 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:00.406 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:00.410 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:00.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:00.426 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-18T00:36:00.511 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:00.538 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:00.556 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:00.558 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:00.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:00.659 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:00.687 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:00.699 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:00.706 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:00.710 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:00.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:00.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:00.835 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:59.410371+0000 osd.4 (osd.4) 161 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:36:00.835 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:59.413520+0000 osd.4 (osd.4) 162 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:36:00.835 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.642450+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:00.835 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.642553+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.642604+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:59.642662+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e281: 8 total, 8 up, 7 in 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.647322+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.648186+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.650755+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.653128+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.653368+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:00.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:59.703893+0000 osd.2 (osd.2) 119 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:36:00.837 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:59.704803+0000 osd.2 (osd.2) 120 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:36:00.837 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:35:59.859126+0000 mgr.y (mgr.24454) 295 : cluster [DBG] pgmap v409: 249 pgs: 32 unknown, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:00.837 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:35:59.883345+0000 mon.a (mon.0) 1945 : audit [DBG] from='client.? 172.21.15.59:0/2965523497' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:00.837 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.642525+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:36:00.837 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.642612+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:36:00.837 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.642672+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:00.837 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:00.642747+0000 mon.a (mon.0) 1949 : cluster [DBG] osdmap e282: 8 total, 8 up, 7 in 2024-02-18T00:36:00.838 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:00.935 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:00.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:00.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:59.410371+0000 osd.4 (osd.4) 161 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:36:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:59.413520+0000 osd.4 (osd.4) 162 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.642450+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.642553+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.642604+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:59.642662+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e281: 8 total, 8 up, 7 in 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.647322+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.648186+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.650755+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.653128+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.653368+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:59.703893+0000 osd.2 (osd.2) 119 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:59.704803+0000 osd.2 (osd.2) 120 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:35:59.859126+0000 mgr.y (mgr.24454) 295 : cluster [DBG] pgmap v409: 249 pgs: 32 unknown, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:35:59.883345+0000 mon.a (mon.0) 1945 : audit [DBG] from='client.? 172.21.15.59:0/2965523497' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.642525+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.642612+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.642672+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:00.642747+0000 mon.a (mon.0) 1949 : cluster [DBG] osdmap e282: 8 total, 8 up, 7 in 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:59.410371+0000 osd.4 (osd.4) 161 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:59.413520+0000 osd.4 (osd.4) 162 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.642450+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:01.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.642553+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.642604+0000 mon.a (mon.0) 1940 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:59.642662+0000 mon.a (mon.0) 1941 : cluster [DBG] osdmap e281: 8 total, 8 up, 7 in 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.647322+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]: dispatch 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.648186+0000 mon.c (mon.2) 243 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.650755+0000 mon.b (mon.1) 356 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.653128+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]: dispatch 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.653368+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:59.703893+0000 osd.2 (osd.2) 119 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:59.704803+0000 osd.2 (osd.2) 120 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:35:59.859126+0000 mgr.y (mgr.24454) 295 : cluster [DBG] pgmap v409: 249 pgs: 32 unknown, 217 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:35:59.883345+0000 mon.a (mon.0) 1945 : audit [DBG] from='client.? 172.21.15.59:0/2965523497' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.642525+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? 172.21.15.59:0/1624514835' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi059-82882-48"}]': finished 2024-02-18T00:36:01.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.642612+0000 mon.a (mon.0) 1947 : audit [INF] from='client.? 172.21.15.59:0/105667530' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi059-82877-39"}]': finished 2024-02-18T00:36:01.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.642672+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:01.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:00.642747+0000 mon.a (mon.0) 1949 : cluster [DBG] osdmap e282: 8 total, 8 up, 7 in 2024-02-18T00:36:01.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:01.069 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:01.110 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:36:01.193 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:01.195 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:01.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:01.255 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:01.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:01.324 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:01.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:36:01.573 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:01.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:01.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:01.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:01.708 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:01.884 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:00.405955+0000 osd.4 (osd.4) 163 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:00.408854+0000 osd.4 (osd.4) 164 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.647658+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.649026+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.651928+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.668410+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.669407+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.669605+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.669989+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.670215+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi059-82877-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:00.670452+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi059-82882-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:00.738763+0000 osd.2 (osd.2) 121 : cluster [DBG] 62.a scrub starts 2024-02-18T00:36:01.885 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:00.739954+0000 osd.2 (osd.2) 122 : cluster [DBG] 62.a scrub ok 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:00.757906+0000 osd.3 (osd.3) 159 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:00.758903+0000 osd.3 (osd.3) 160 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.004508+0000 osd.5 (osd.5) 129 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.007401+0000 osd.5 (osd.5) 130 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:01.109083+0000 mgr.y (mgr.24454) 296 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.640391+0000 mon.a (mon.0) 1957 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:01.643981+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]': finished 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:01.644048+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi059-82877-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:01.644111+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi059-82882-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.644147+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e283: 8 total, 8 up, 7 in 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:01.644693+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi059-82877-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:01.645245+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi059-82882-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:01.653985+0000 mon.b (mon.1) 358 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:00.405955+0000 osd.4 (osd.4) 163 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:00.408854+0000 osd.4 (osd.4) 164 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.647658+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]: dispatch 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.649026+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]: dispatch 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.651928+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.668410+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.669407+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.669605+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.669989+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.670215+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi059-82877-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:00.670452+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi059-82882-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:00.738763+0000 osd.2 (osd.2) 121 : cluster [DBG] 62.a scrub starts 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:00.739954+0000 osd.2 (osd.2) 122 : cluster [DBG] 62.a scrub ok 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:00.757906+0000 osd.3 (osd.3) 159 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:00.758903+0000 osd.3 (osd.3) 160 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.004508+0000 osd.5 (osd.5) 129 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.007401+0000 osd.5 (osd.5) 130 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:01.109083+0000 mgr.y (mgr.24454) 296 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.640391+0000 mon.a (mon.0) 1957 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:01.643981+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]': finished 2024-02-18T00:36:01.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:01.644048+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi059-82877-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:01.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:01.644111+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi059-82882-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:01.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.644147+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e283: 8 total, 8 up, 7 in 2024-02-18T00:36:01.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:01.644693+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi059-82877-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:01.645245+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi059-82882-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:01.653985+0000 mon.b (mon.1) 358 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.959 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:01.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:00.405955+0000 osd.4 (osd.4) 163 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:36:01.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:00.408854+0000 osd.4 (osd.4) 164 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:36:01.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.647658+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]: dispatch 2024-02-18T00:36:01.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.649026+0000 mon.a (mon.0) 1950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]: dispatch 2024-02-18T00:36:01.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.651928+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.668410+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.669407+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.669605+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.669989+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.670215+0000 mon.a (mon.0) 1955 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi059-82877-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:00.670452+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi059-82882-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:00.738763+0000 osd.2 (osd.2) 121 : cluster [DBG] 62.a scrub starts 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:00.739954+0000 osd.2 (osd.2) 122 : cluster [DBG] 62.a scrub ok 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:00.757906+0000 osd.3 (osd.3) 159 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:00.758903+0000 osd.3 (osd.3) 160 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:36:01.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.004508+0000 osd.5 (osd.5) 129 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.007401+0000 osd.5 (osd.5) 130 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:01.109083+0000 mgr.y (mgr.24454) 296 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.640391+0000 mon.a (mon.0) 1957 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:01.643981+0000 mon.a (mon.0) 1958 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-33", "mode": "writeback"}]': finished 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:01.644048+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi059-82877-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:01.644111+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi059-82882-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.644147+0000 mon.a (mon.0) 1961 : cluster [DBG] osdmap e283: 8 total, 8 up, 7 in 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:01.644693+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi059-82877-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:01.645245+0000 mon.a (mon.0) 1963 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi059-82882-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:01.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:01.653985+0000 mon.b (mon.1) 358 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:01.962 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:01.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:02.064 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:02.090 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:02.238 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:02.242 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:02.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:02.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:02.372 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:02.496 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:02.496 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:36:02.497 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:36:02.497 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:02.497 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:02.498 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:36:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:02] "GET /metrics HTTP/1.1" 200 190807 "" "Prometheus/2.33.4" 2024-02-18T00:36:02.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:02.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:02.600 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:02.629 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:02.727 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:02.730 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:02.731 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:02.831 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:02.858 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:03.134 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-18T00:36:03.194 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:36:03.206 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:03.209 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:03.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:03.311 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:03.356 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:03.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:03.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:03.442 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:03.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:03.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.448278+0000 osd.6 (osd.6) 143 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.449410+0000 osd.6 (osd.6) 144 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.597085+0000 osd.1 (osd.1) 143 : cluster [DBG] 26.2 scrub starts 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.600101+0000 osd.1 (osd.1) 144 : cluster [DBG] 26.2 scrub ok 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.859644+0000 mgr.y (mgr.24454) 297 : cluster [DBG] pgmap v412: 249 pgs: 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.448278+0000 osd.6 (osd.6) 143 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.449410+0000 osd.6 (osd.6) 144 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.597085+0000 osd.1 (osd.1) 143 : cluster [DBG] 26.2 scrub starts 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.600101+0000 osd.1 (osd.1) 144 : cluster [DBG] 26.2 scrub ok 2024-02-18T00:36:03.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.859644+0000 mgr.y (mgr.24454) 297 : cluster [DBG] pgmap v412: 249 pgs: 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:03.925 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.448278+0000 osd.6 (osd.6) 143 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:36:03.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.449410+0000 osd.6 (osd.6) 144 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:36:03.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.597085+0000 osd.1 (osd.1) 143 : cluster [DBG] 26.2 scrub starts 2024-02-18T00:36:03.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.600101+0000 osd.1 (osd.1) 144 : cluster [DBG] 26.2 scrub ok 2024-02-18T00:36:03.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.859644+0000 mgr.y (mgr.24454) 297 : cluster [DBG] pgmap v412: 249 pgs: 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:04.250 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:04.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:04.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:04.354 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:04.383 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:04.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.742807+0000 osd.4 (osd.4) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:04.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.742996+0000 osd.4 (osd.4) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:04.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.769853+0000 osd.2 (osd.2) 123 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:36:04.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:01.770675+0000 osd.2 (osd.2) 124 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:36:04.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:02.644201+0000 osd.1 (osd.1) 145 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:36:04.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:02.645620+0000 osd.1 (osd.1) 146 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:36:04.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:02.654819+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:02.983601+0000 osd.5 (osd.5) 131 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:02.987054+0000 osd.5 (osd.5) 132 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.384349+0000 osd.0 (osd.0) 137 : cluster [DBG] 26.a scrub starts 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.388984+0000 osd.0 (osd.0) 138 : cluster [DBG] 26.a scrub ok 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.439495+0000 osd.4 (osd.4) 167 : cluster [DBG] 26.1 scrub starts 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.441117+0000 osd.4 (osd.4) 168 : cluster [DBG] 26.1 scrub ok 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.521652+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e284: 8 total, 8 up, 7 in 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:03.625318+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi059-82877-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:03.625414+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi059-82882-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.625518+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e285: 8 total, 8 up, 7 in 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:03.655768+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:03.668045+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:03.668407+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:04.590 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.781985+0000 osd.3 (osd.3) 161 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:36:04.591 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.782796+0000 osd.3 (osd.3) 162 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:36:04.591 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:03.860269+0000 mgr.y (mgr.24454) 298 : cluster [DBG] pgmap v415: 265 pgs: 16 unknown, 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:04.860 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:04.863 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:04.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:04.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 -- 172.21.15.59:0/4062524747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec0f9610 msgr2=0x7f4fec101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:04.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/4062524747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec0f9610 0x7f4fec101d20 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 rx=0x7f4fe0004870 tx=0x7f4fe0009cb0).stop 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 -- 172.21.15.59:0/4062524747 shutdown_connections 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/4062524747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fec0f8cb0 0x7f4fec0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/4062524747 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec102210 0x7f4fec104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/4062524747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec0f9610 0x7f4fec101d20 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 -- 172.21.15.59:0/4062524747 >> 172.21.15.59:0/4062524747 conn(0x7f4fec0f2820 msgr2=0x7f4fec0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.842+0000 7f4ff16b1700 1 -- 172.21.15.59:0/4062524747 shutdown_connections 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.843+0000 7f4ff16b1700 1 -- 172.21.15.59:0/4062524747 wait complete. 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.843+0000 7f4ff16b1700 1 librados: starting msgr at 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.843+0000 7f4ff16b1700 1 librados: starting objecter 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.843+0000 7f4ff16b1700 1 Processor -- start 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 -- start start 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 librados: setting wanted keys 2024-02-18T00:36:04.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 librados: calling monclient init 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fec0f8cb0 0x7f4fec10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec0f9610 0x7f4fec10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec102210 0x7f4fec1083f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec1088e0 con 0x7f4fec102210 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec108a50 con 0x7f4fec0f8cb0 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4ff16b1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec108c10 con 0x7f4fec0f9610 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec0f9610 0x7f4fec10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec0f9610 0x7f4fec10e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:36318/0 (socket says 172.21.15.59:36318) 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4feaffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fec0f8cb0 0x7f4fec10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 -- 172.21.15.59:0/199702220 learned_addr learned my addr 172.21.15.59:0/199702220 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4feb7fe700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec102210 0x7f4fec1083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 -- 172.21.15.59:0/199702220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fec0f8cb0 msgr2=0x7f4fec10e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fec0f8cb0 0x7f4fec10e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 -- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec102210 msgr2=0x7f4fec1083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec102210 0x7f4fec1083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 -- 172.21.15.59:0/199702220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40097c0 con 0x7f4fec0f9610 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4feaffd700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fec0f8cb0 0x7f4fec10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fea7fc700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec0f9610 0x7f4fec10e6d0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 rx=0x7f4fe00043a0 tx=0x7f4fe000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fd3fff700 10 client.?.objecter ms_handle_connect 0x7f4fec0f9610 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.844+0000 7f4fd3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.845+0000 7f4fd3fff700 1 -- 172.21.15.59:0/199702220 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe000e070 con 0x7f4fec0f9610 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.845+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe0005040 con 0x7f4fec0f9610 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.845+0000 7f4fd3fff700 1 -- 172.21.15.59:0/199702220 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fe0009710 con 0x7f4fec0f9610 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.845+0000 7f4fd3fff700 1 -- 172.21.15.59:0/199702220 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0019070 con 0x7f4fec0f9610 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.845+0000 7f4ff16b1700 10 client.40796.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:04.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.845+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec1090b0 con 0x7f4fec0f9610 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.845+0000 7f4ff16b1700 1 librados: init done 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.846+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec1092a0 con 0x7f4fec0f9610 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.846+0000 7f4fd3fff700 1 -- 172.21.15.59:0/199702220 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4fe00090d0 con 0x7f4fec0f9610 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4fd3fff700 1 --2- 172.21.15.59:0/199702220 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fd4062c90 0x7f4fd4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4fd3fff700 1 -- 172.21.15.59:0/199702220 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(286..286 src has 44..286) v4 ==== 9615+0+0 (secure 0 0 0) 0x7f4fe007e170 con 0x7f4fec0f9610 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4fd3fff700 10 client.40796.objecter ms_dispatch 0x7f4fec001550 osd_map(286..286 src has 44..286) v4 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4fd3fff700 3 client.40796.objecter handle_osd_map got epochs [286,286] > 0 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4fd3fff700 3 client.40796.objecter handle_osd_map decoding full epoch 286 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4feaffd700 1 --2- 172.21.15.59:0/199702220 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fd4062c90 0x7f4fd4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4fd3fff700 20 client.40796.objecter dump_active .. 0 homeless 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4fd3fff700 10 client.40796.objecter ms_handle_connect 0x7f4fd4062c90 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.847+0000 7f4feaffd700 1 --2- 172.21.15.59:0/199702220 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fd4062c90 0x7f4fd4065150 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 rx=0x7f4fdc007b70 tx=0x7f4fdc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:04.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.851+0000 7f4fd3fff700 1 -- 172.21.15.59:0/199702220 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4fe00510a0 con 0x7f4fec0f9610 2024-02-18T00:36:04.964 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:04.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.759+0000 7fb958459700 1 -- 172.21.15.59:0/1482529067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 msgr2=0x7fb9500f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:04.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.759+0000 7fb958459700 1 --2- 172.21.15.59:0/1482529067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 0x7fb9500f7d40 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 rx=0x7fb940004870 tx=0x7fb940009cb0).stop 2024-02-18T00:36:04.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 -- 172.21.15.59:0/1482529067 shutdown_connections 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 --2- 172.21.15.59:0/1482529067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9500f8420 0x7fb950104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 --2- 172.21.15.59:0/1482529067 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 0x7fb9500fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 --2- 172.21.15.59:0/1482529067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 0x7fb9500f7d40 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 -- 172.21.15.59:0/1482529067 >> 172.21.15.59:0/1482529067 conn(0x7fb9500f3800 msgr2=0x7fb9500f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 -- 172.21.15.59:0/1482529067 shutdown_connections 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 -- 172.21.15.59:0/1482529067 wait complete. 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 librados: starting msgr at 2024-02-18T00:36:04.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.760+0000 7fb958459700 1 librados: starting objecter 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 Processor -- start 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 -- start start 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 librados: setting wanted keys 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 librados: calling monclient init 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9500f8420 0x7fb95010e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 0x7fb95010e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 0x7fb950108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950108710 con 0x7fb9500fe510 2024-02-18T00:36:04.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950108880 con 0x7fb9500f8420 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb958459700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb950108a40 con 0x7fb9500fdbb0 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb9559f4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 0x7fb95010e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb9559f4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 0x7fb95010e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:36300/0 (socket says 172.21.15.59:36300) 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb9559f4700 1 -- 172.21.15.59:0/96206734 learned_addr learned my addr 172.21.15.59:0/96206734 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb9569f6700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 0x7fb950108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.761+0000 7fb9561f5700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9500f8420 0x7fb95010e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9559f4700 1 -- 172.21.15.59:0/96206734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9500f8420 msgr2=0x7fb95010e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9559f4700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9500f8420 0x7fb95010e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9559f4700 1 -- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 msgr2=0x7fb950108220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9559f4700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 0x7fb950108220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9559f4700 1 -- 172.21.15.59:0/96206734 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb940005040 con 0x7fb9500fdbb0 2024-02-18T00:36:04.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9559f4700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 0x7fb95010e570 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 rx=0x7fb9400043a0 tx=0x7fb94000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9477fe700 10 client.?.objecter ms_handle_connect 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9477fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9477fe700 1 -- 172.21.15.59:0/96206734 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000e070 con 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9477fe700 1 -- 172.21.15.59:0/96206734 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb940009710 con 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb9477fe700 1 -- 172.21.15.59:0/96206734 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940019070 con 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb950108cc0 con 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb958459700 10 client.40790.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb950114eb0 con 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.762+0000 7fb958459700 1 librados: init done 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.763+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95005ee80 con 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 1 -- 172.21.15.59:0/96206734 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb9400090d0 con 0x7fb9500fdbb0 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 1 --2- 172.21.15.59:0/96206734 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb93c062c90 0x7fb93c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:04.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 1 -- 172.21.15.59:0/96206734 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(286..286 src has 44..286) v4 ==== 9615+0+0 (secure 0 0 0) 0x7fb94007e170 con 0x7fb9500fdbb0 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 10 client.40790.objecter ms_dispatch 0x7fb950001550 osd_map(286..286 src has 44..286) v4 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 3 client.40790.objecter handle_osd_map got epochs [286,286] > 0 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 3 client.40790.objecter handle_osd_map decoding full epoch 286 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9561f5700 1 --2- 172.21.15.59:0/96206734 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb93c062c90 0x7fb93c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 20 client.40790.objecter dump_active .. 0 homeless 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9477fe700 10 client.40790.objecter ms_handle_connect 0x7fb93c062c90 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.764+0000 7fb9561f5700 1 --2- 172.21.15.59:0/96206734 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb93c062c90 0x7fb93c065150 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 rx=0x7fb94c004790 tx=0x7fb94c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.772+0000 7fb9477fe700 1 -- 172.21.15.59:0/96206734 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb9400510a0 con 0x7fb9500fdbb0 2024-02-18T00:36:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:04.958+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fb950109790 con 0x7fb9500fdbb0 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.742807+0000 osd.4 (osd.4) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.742996+0000 osd.4 (osd.4) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.769853+0000 osd.2 (osd.2) 123 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:01.770675+0000 osd.2 (osd.2) 124 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:02.644201+0000 osd.1 (osd.1) 145 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:02.645620+0000 osd.1 (osd.1) 146 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:02.654819+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:02.983601+0000 osd.5 (osd.5) 131 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:02.987054+0000 osd.5 (osd.5) 132 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.384349+0000 osd.0 (osd.0) 137 : cluster [DBG] 26.a scrub starts 2024-02-18T00:36:04.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.388984+0000 osd.0 (osd.0) 138 : cluster [DBG] 26.a scrub ok 2024-02-18T00:36:04.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.439495+0000 osd.4 (osd.4) 167 : cluster [DBG] 26.1 scrub starts 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.441117+0000 osd.4 (osd.4) 168 : cluster [DBG] 26.1 scrub ok 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.521652+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e284: 8 total, 8 up, 7 in 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:03.625318+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi059-82877-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:03.625414+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi059-82882-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.625518+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e285: 8 total, 8 up, 7 in 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:03.655768+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.742807+0000 osd.4 (osd.4) 165 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:04.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.742996+0000 osd.4 (osd.4) 166 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:04.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.769853+0000 osd.2 (osd.2) 123 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:36:04.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:01.770675+0000 osd.2 (osd.2) 124 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:36:04.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:02.644201+0000 osd.1 (osd.1) 145 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:02.645620+0000 osd.1 (osd.1) 146 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:02.654819+0000 mon.b (mon.1) 359 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:02.983601+0000 osd.5 (osd.5) 131 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:02.987054+0000 osd.5 (osd.5) 132 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.384349+0000 osd.0 (osd.0) 137 : cluster [DBG] 26.a scrub starts 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.388984+0000 osd.0 (osd.0) 138 : cluster [DBG] 26.a scrub ok 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.439495+0000 osd.4 (osd.4) 167 : cluster [DBG] 26.1 scrub starts 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.441117+0000 osd.4 (osd.4) 168 : cluster [DBG] 26.1 scrub ok 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.521652+0000 mon.a (mon.0) 1964 : cluster [DBG] osdmap e284: 8 total, 8 up, 7 in 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:03.625318+0000 mon.a (mon.0) 1965 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi059-82877-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:03.625414+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi059-82882-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.625518+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e285: 8 total, 8 up, 7 in 2024-02-18T00:36:04.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:03.655768+0000 mon.b (mon.1) 360 : audit [DBG] from='client.? 172.21.15.59:0/1312135677' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T00:36:04.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:03.668045+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:04.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:03.668407+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:04.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.781985+0000 osd.3 (osd.3) 161 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:36:04.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.782796+0000 osd.3 (osd.3) 162 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:36:04.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:03.860269+0000 mgr.y (mgr.24454) 298 : cluster [DBG] pgmap v415: 265 pgs: 16 unknown, 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:04.998 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:03.668045+0000 mon.c (mon.2) 245 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:04.999 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:03.668407+0000 mon.a (mon.0) 1968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:04.999 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.781985+0000 osd.3 (osd.3) 161 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:36:04.999 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.782796+0000 osd.3 (osd.3) 162 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:36:04.999 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:03.860269+0000 mgr.y (mgr.24454) 298 : cluster [DBG] pgmap v415: 265 pgs: 16 unknown, 249 active+clean; 8.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:05.017 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:05.020 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:05.020 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:05.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.024+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fec05ef10 con 0x7f4fd4062c90 2024-02-18T00:36:05.031 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:05.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.042+0000 7f4fd3fff700 1 -- 172.21.15.59:0/199702220 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+635023 (secure 0 0 0) 0x7f4fdc00f040 con 0x7f4fd4062c90 2024-02-18T00:36:05.052 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.047+0000 7f4ff16b1700 10 librados: watch_flush enter 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.047+0000 7f4ff16b1700 10 librados: watch_flush exit 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 20 client.40796.objecter shutdown clearing up homeless session... 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 10 client.40796.objecter successfully canceled tick 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fd4062c90 msgr2=0x7f4fd4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/199702220 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fd4062c90 0x7f4fd4065150 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 rx=0x7f4fdc007b70 tx=0x7f4fdc00b040).stop 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec0f9610 msgr2=0x7f4fec10e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:05.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec0f9610 0x7f4fec10e6d0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 rx=0x7f4fe00043a0 tx=0x7f4fe000b040).stop 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 shutdown_connections 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/199702220 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fd4062c90 0x7f4fd4065150 unknown :-1 s=CLOSED pgs=3051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fec0f8cb0 0x7f4fec10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fec102210 0x7f4fec1083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 --2- 172.21.15.59:0/199702220 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fec0f9610 0x7f4fec10e6d0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 >> 172.21.15.59:0/199702220 conn(0x7f4fec0f2820 msgr2=0x7f4fec0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 shutdown_connections 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 -- 172.21.15.59:0/199702220 wait complete. 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.048+0000 7f4ff16b1700 1 librados: shutdown 2024-02-18T00:36:05.060 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:36:05.103 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:05.103 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:36:05.103 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:36:05.103 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:36:05.103 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:05.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:05.151 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:05.351 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:05.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:05.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:05.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:05.485 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:05.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.642+0000 7fb9477fe700 1 -- 172.21.15.59:0/96206734 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v287) v1 ==== 101+0+0 (secure 0 0 0) 0x7fb940050c90 con 0x7fb9500fdbb0 2024-02-18T00:36:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 10 librados: watch_flush enter 2024-02-18T00:36:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 10 librados: watch_flush exit 2024-02-18T00:36:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 20 client.40790.objecter shutdown clearing up homeless session... 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 10 client.40790.objecter successfully canceled tick 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb93c062c90 msgr2=0x7fb93c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 --2- 172.21.15.59:0/96206734 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb93c062c90 0x7fb93c065150 secure :-1 s=READY pgs=3050 cs=0 l=1 rev1=1 rx=0x7fb94c004790 tx=0x7fb94c00b040).stop 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 msgr2=0x7fb95010e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 0x7fb95010e570 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 rx=0x7fb9400043a0 tx=0x7fb94000b040).stop 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 shutdown_connections 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 --2- 172.21.15.59:0/96206734 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb93c062c90 0x7fb93c065150 unknown :-1 s=CLOSED pgs=3050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9500f8420 0x7fb95010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9500fe510 0x7fb950108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 --2- 172.21.15.59:0/96206734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9500fdbb0 0x7fb95010e570 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 >> 172.21.15.59:0/96206734 conn(0x7fb9500f3800 msgr2=0x7fb9500fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 shutdown_connections 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 -- 172.21.15.59:0/96206734 wait complete. 2024-02-18T00:36:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:05.646+0000 7fb958459700 1 librados: shutdown 2024-02-18T00:36:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.410291+0000 osd.4 (osd.4) 169 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.413197+0000 osd.4 (osd.4) 170 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.428657+0000 osd.0 (osd.0) 139 : cluster [DBG] 25.a scrub starts 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.431704+0000 osd.0 (osd.0) 140 : cluster [DBG] 25.a scrub ok 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.632713+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.632830+0000 mon.a (mon.0) 1970 : cluster [DBG] osdmap e286: 8 total, 8 up, 7 in 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.653782+0000 osd.1 (osd.1) 147 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.656406+0000 osd.1 (osd.1) 148 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.661562+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:05.730 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.661996+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.668812+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.669128+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.670041+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.670351+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.670869+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.671186+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.778902+0000 osd.3 (osd.3) 163 : cluster [DBG] 62.c scrub starts 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:04.780538+0000 osd.3 (osd.3) 164 : cluster [DBG] 62.c scrub ok 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.959098+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.59:0/96206734' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:04.959433+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.025253+0000 mgr.y (mgr.24454) 299 : audit [DBG] from='client.40796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:05.062399+0000 osd.5 (osd.5) 133 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:36:05.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:05.065290+0000 osd.5 (osd.5) 134 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:36:05.733 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:05.736 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:05.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:05.749 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-18T00:36:05.838 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:05.864 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:05.882 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:05.885 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:05.886 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:05.986 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.410291+0000 osd.4 (osd.4) 169 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.413197+0000 osd.4 (osd.4) 170 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.428657+0000 osd.0 (osd.0) 139 : cluster [DBG] 25.a scrub starts 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.431704+0000 osd.0 (osd.0) 140 : cluster [DBG] 25.a scrub ok 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.632713+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.632830+0000 mon.a (mon.0) 1970 : cluster [DBG] osdmap e286: 8 total, 8 up, 7 in 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.653782+0000 osd.1 (osd.1) 147 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.656406+0000 osd.1 (osd.1) 148 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.661562+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.661996+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.668812+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.669128+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.670041+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.670351+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.670869+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.410291+0000 osd.4 (osd.4) 169 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.413197+0000 osd.4 (osd.4) 170 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.428657+0000 osd.0 (osd.0) 139 : cluster [DBG] 25.a scrub starts 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.431704+0000 osd.0 (osd.0) 140 : cluster [DBG] 25.a scrub ok 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.632713+0000 mon.a (mon.0) 1969 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.632830+0000 mon.a (mon.0) 1970 : cluster [DBG] osdmap e286: 8 total, 8 up, 7 in 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.653782+0000 osd.1 (osd.1) 147 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.656406+0000 osd.1 (osd.1) 148 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.661562+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.661996+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.668812+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.669128+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.670041+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.670351+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.670869+0000 mon.c (mon.2) 249 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:06.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.671186+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.778902+0000 osd.3 (osd.3) 163 : cluster [DBG] 62.c scrub starts 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:04.780538+0000 osd.3 (osd.3) 164 : cluster [DBG] 62.c scrub ok 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.959098+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.59:0/96206734' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:04.959433+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.025253+0000 mgr.y (mgr.24454) 299 : audit [DBG] from='client.40796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:05.062399+0000 osd.5 (osd.5) 133 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:05.065290+0000 osd.5 (osd.5) 134 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.671186+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.778902+0000 osd.3 (osd.3) 163 : cluster [DBG] 62.c scrub starts 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:04.780538+0000 osd.3 (osd.3) 164 : cluster [DBG] 62.c scrub ok 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.959098+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.59:0/96206734' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:04.959433+0000 mon.a (mon.0) 1975 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:06.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.025253+0000 mgr.y (mgr.24454) 299 : audit [DBG] from='client.40796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:06.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:05.062399+0000 osd.5 (osd.5) 133 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:36:06.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:05.065290+0000 osd.5 (osd.5) 134 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:36:06.016 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:06.036 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:06.038 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:06.039 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:06.060 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:06.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:06.171 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:05.629894+0000 mon.a (mon.0) 1976 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:36:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.638981+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.639082+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.639185+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:05.639257+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e287: 8 total, 8 up, 7 in 2024-02-18T00:36:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.642723+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.644100+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.644660+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:05.645499+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:05.861096+0000 mgr.y (mgr.24454) 300 : cluster [DBG] pgmap v418: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2 op/s 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:05.629894+0000 mon.a (mon.0) 1976 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.638981+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.639082+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.639185+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:05.639257+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e287: 8 total, 8 up, 7 in 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.642723+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.644100+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.644660+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:06.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:05.645499+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:06.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:05.861096+0000 mgr.y (mgr.24454) 300 : cluster [DBG] pgmap v418: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2 op/s 2024-02-18T00:36:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:05.629894+0000 mon.a (mon.0) 1976 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:36:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.638981+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.639082+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi059-82917-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:06.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.639185+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:05.639257+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e287: 8 total, 8 up, 7 in 2024-02-18T00:36:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.642723+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.644100+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.644660+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:05.645499+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:06.890 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:05.861096+0000 mgr.y (mgr.24454) 300 : cluster [DBG] pgmap v418: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2 op/s 2024-02-18T00:36:07.023 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:07.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:07.027 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:07.128 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:07.159 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:07.178 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:07.181 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:07.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:07.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:07.311 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:07.384 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:07.387 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:07.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:07.488 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:07.515 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:07.535 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:07.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:07.538 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:07.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:07.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.482552+0000 osd.4 (osd.4) 171 : cluster [DBG] 62.16 scrub starts 2024-02-18T00:36:07.647 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.486021+0000 osd.4 (osd.4) 172 : cluster [DBG] 62.16 scrub ok 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.639873+0000 osd.1 (osd.1) 149 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.641180+0000 osd.1 (osd.1) 150 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.724658+0000 osd.3 (osd.3) 165 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.725964+0000 osd.3 (osd.3) 166 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.754652+0000 osd.2 (osd.2) 125 : cluster [DBG] 62.12 deep-scrub starts 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:06.758765+0000 osd.2 (osd.2) 126 : cluster [DBG] 62.12 deep-scrub ok 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.095542+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.095632+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:07.095678+0000 mon.a (mon.0) 1986 : cluster [DBG] osdmap e288: 8 total, 8 up, 7 in 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.096396+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.096627+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:07.648 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.107272+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:07.649 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.107543+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:07.663 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:07.663 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:36:07.663 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:36:07.663 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:36:07.663 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.663 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2059 17.11 e108/104 17:8d70d4b8:test-rados-api-smithi059-82949-16::big:head v 108'2, mlcod=0'0)", 2024-02-18T00:36:07.663 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.678171+0000", 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 183.978602182, 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.062291530999999997, 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.678171+0000", 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.678171+0000", 2024-02-18T00:36:07.664 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.678170+0000", 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027259334 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705430+0000", 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3023e-05 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.665 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705453+0000", 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6519999999999995e-06 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705461+0000", 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5605999999999994e-05 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705537+0000", 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0548e-05 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.666 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705558+0000", 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034859651999999998 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.740417+0000", 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5531000000000002e-05 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.740463+0000", 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.062291530999999997 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.667 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.0.0:148 17.4 e108/104 -19:238a5d4d:::temp_17.4_0_24140_1:head v 0'0, mlcod=108'1)", 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.758875+0000", 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 183.89789854700001, 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025775896999999999, 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.758875+0000", 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.668 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.758875+0000", 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999914 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.758866+0000", 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.016318358000000002 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775185+0000", 2024-02-18T00:36:07.669 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1304000000000001e-05 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775206+0000", 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7670000000000003e-06 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775214+0000", 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4301000000000006e-05 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775288+0000", 2024-02-18T00:36:07.670 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8799000000000001e-05 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775307+0000", 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0093030990000000004 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.784610+0000", 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0865999999999998e-05 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.671 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.784651+0000", 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025775896999999999 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.0.0:149 17.4 e108/104 -19:238a5d4d:::temp_17.4_0_24140_1:head v 0'0, mlcod=108'1)", 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.782907+0000", 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 183.87386630200001, 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017161189, 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.672 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.782907+0000", 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.782907+0000", 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.782906+0000", 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0077003490000000004 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.673 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790606+0000", 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9126e-05 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790625+0000", 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1689999999999992e-06 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790633+0000", 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3750000000000004e-05 2024-02-18T00:36:07.674 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790707+0000", 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4715999999999998e-05 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790742+0000", 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0092875760000000005 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.800029+0000", 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8961999999999997e-05 2024-02-18T00:36:07.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.800068+0000", 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017161189 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.0.0:150 17.4 e108/104 -19:238a5d4d:::temp_17.4_0_24140_1:head v 0'0, mlcod=108'1)", 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.806114+0000", 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 183.85065909400001, 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017799674000000001, 2024-02-18T00:36:07.676 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.806114+0000", 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.806114+0000", 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.806112+0000", 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0082654129999999992 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814378+0000", 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6948000000000001e-05 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814395+0000", 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6560000000000004e-06 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.678 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814404+0000", 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8322999999999999e-05 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814443+0000", 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9939999999999999e-05 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814463+0000", 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0094215979999999998 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.679 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.823884+0000", 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9757e-05 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.823914+0000", 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017799674000000001 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.680 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2069 17.3 17:c1ec3694:test-rados-api-smithi059-82949-17::big:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.014659+0000", 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 183.642114859, 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096479857000000002, 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2069 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.681 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.014659+0000", 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.014659+0000", 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.014658+0000", 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.032078772999999998 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.682 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046736+0000", 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5109999999999999e-05 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046752+0000", 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7400000000000004e-06 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046759+0000", 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0557999999999999e-05 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.683 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046810+0000", 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000123153 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046933+0000", 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000120703 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.047054+0000", 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.059890066999999998 2024-02-18T00:36:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.106944+0000", 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003654839 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.110599+0000", 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00046675800000000001 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111065+0000", 2024-02-18T00:36:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4075000000000002e-05 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111099+0000", 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8964000000000002e-05 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111138+0000", 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096479857000000002 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.686 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:6 22.2 e108/105 22:47a9abb2:::36.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.068708+0000", 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 183.588065726, 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034744510999999999, 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068708+0000", 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.687 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068708+0000", 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068707+0000", 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2602000000000001e-05 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068719+0000", 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7615999999999999e-05 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.688 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068737+0000", 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6810000000000003e-06 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068743+0000", 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6243999999999999e-05 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068799+0000", 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5424999999999999e-05 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.689 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068814+0000", 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034560975000000001 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103375+0000", 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6917999999999998e-05 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103452+0000", 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034744510999999999 2024-02-18T00:36:07.690 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24865.0:4 29.16 29:681f27cd:::2.obj:head [write 0~105] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.757362+0000", 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 182.89941162100001, 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012451744000000001, 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24865", 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2678422317", 2024-02-18T00:36:07.691 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 4 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757362+0000", 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757362+0000", 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.692 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757361+0000", 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6770000000000003e-06 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757366+0000", 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7200000000000006e-06 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757373+0000", 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6960000000000002e-06 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.693 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757379+0000", 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1489999999999997e-05 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757421+0000", 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4278999999999998e-05 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757495+0000", 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000130619 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.694 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757625+0000", 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00034791500000000002 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757973+0000", 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000186467 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.758160+0000", 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011598658 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.695 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.769758+0000", 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2137000000000001e-05 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.769791+0000", 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2926e-05 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.769814+0000", 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012451744000000001 2024-02-18T00:36:07.696 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.15213.0:2 41.1a 41:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:08.335259+0000", 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 179.321514866, 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.54663461800000002, 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.15213", 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1863209286", 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:07.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335259+0000", 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335259+0000", 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335258+0000", 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.045e-06 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335263+0000", 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3730000000000004e-06 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335270+0000", 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1789999999999999e-06 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335275+0000", 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8170999999999998e-05 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335313+0000", 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6689999999999997e-06 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335318+0000", 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.52990910599999996 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.865227+0000", 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000124976 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.865352+0000", 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014595600000000001 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.865498+0000", 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00086496499999999998 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.866363+0000", 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0082699220000000007 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.874633+0000", 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0072001019999999999 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.881833+0000", 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3037000000000002e-05 2024-02-18T00:36:07.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.881866+0000", 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6931999999999999e-05 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.881893+0000", 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.54663461800000002 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.703 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:2 71.15 71:ad551702:::test:head [create,write 0~4] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 159.56800077400001, 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067209718000000002, 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088771+0000", 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7769999999999997e-06 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088778+0000", 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9749999999999996e-06 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088787+0000", 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6979999999999998e-06 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088794+0000", 2024-02-18T00:36:07.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3955999999999997e-05 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088838+0000", 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0620000000000003e-06 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088845+0000", 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042483522000000003 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.707 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131329+0000", 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014866200000000001 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131477+0000", 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019506600000000001 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131673+0000", 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074819399999999999 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.132421+0000", 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010435201999999999 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.142856+0000", 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013070307 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155926+0000", 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2119000000000002e-05 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.709 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155958+0000", 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3989999999999999e-05 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155982+0000", 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067209718000000002 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:4 71.15 71:ad551702:::test:head [cmpext 0~4,write 0~4] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:36:07.710 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.162410+0000", 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 159.49436302199999, 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.01216428, 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 4 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162410+0000", 2024-02-18T00:36:07.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162410+0000", 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162410+0000", 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0389999999999998e-06 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162414+0000", 2024-02-18T00:36:07.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9700000000000002e-06 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162421+0000", 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3809999999999999e-06 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162424+0000", 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3266999999999997e-05 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.713 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162477+0000", 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2444e-05 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162550+0000", 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000219034 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.162769+0000", 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00038272600000000001 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.163152+0000", 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00055287499999999998 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.163704+0000", 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010786726 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.174491+0000", 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8065000000000001e-05 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.174539+0000", 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5352999999999998e-05 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.174575+0000", 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.01216428 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:6 71.15 71:ad551702:::test:head [cmpext 0~4,write 0~4] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:36:07.716 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 159.472835395, 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026666680000000002, 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 6 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:36:07.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183937+0000", 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4240000000000003e-06 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183942+0000", 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2269999999999996e-06 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183949+0000", 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9459999999999997e-06 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183954+0000", 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4397e-05 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183978+0000", 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0843000000000002e-05 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.184019+0000", 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026585647 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.210605+0000", 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026666680000000002 2024-02-18T00:36:07.720 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:7 71.15 71:ad551702:::test:head [delete] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 159.442680806, 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019741999999999999, 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 7 2024-02-18T00:36:07.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214092+0000", 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1780000000000002e-06 2024-02-18T00:36:07.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214097+0000", 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0831e-05 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214108+0000", 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6280000000000004e-06 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214112+0000", 2024-02-18T00:36:07.723 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3321000000000001e-05 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214156+0000", 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5122000000000004e-05 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214221+0000", 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7605e-05 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.724 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214298+0000", 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029685200000000001 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214595+0000", 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019093308999999999 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233688+0000", 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5790000000000003e-05 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233784+0000", 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9073000000000001e-05 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233813+0000", 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1229999999999998e-05 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233835+0000", 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019741999999999999 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.726 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30097.0:2 74.1f 74:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 157.292974883, 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014294953000000001, 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30097", 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3002694168", 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:07.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.728 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363797+0000", 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9099999999999999e-06 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363804+0000", 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1026000000000001e-05 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363815+0000", 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9299999999999997e-06 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363822+0000", 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5386000000000002e-05 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363867+0000", 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0513999999999995e-05 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363948+0000", 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5987000000000001e-05 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,6", 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364044+0000", 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00049711199999999997 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364541+0000", 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000204945 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364746+0000", 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013291124 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378037+0000", 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2733e-05 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378069+0000", 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4020999999999999e-05 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378093+0000", 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014294953000000001 2024-02-18T00:36:07.732 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25220.0:109 62.d 62:b295a385:test-rados-api-smithi059-82957-10::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 156.29147516099999, 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031474052000000002, 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25220", 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 109 2024-02-18T00:36:07.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365297+0000", 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8029999999999997e-06 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365305+0000", 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.093e-05 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365315+0000", 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1999999999999997e-06 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365323+0000", 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3896999999999997e-05 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365367+0000", 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2156000000000004e-05 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365449+0000", 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3950999999999995e-05 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,7", 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365543+0000", 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00031575199999999998 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365858+0000", 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000242795 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.366101+0000", 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.030572499 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396674+0000", 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8595999999999999e-05 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396732+0000", 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9944000000000002e-05 2024-02-18T00:36:07.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396772+0000", 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031474052000000002 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.5.39:15 17.3 17:c1ec3694:test-rados-api-smithi059-82949-17::big:head [assert-version v1,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected e136)", 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.426892+0000", 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 153.22988108000001, 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011501162000000001, 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.739 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426892+0000", 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426892+0000", 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.740 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426890+0000", 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0927999999999999e-05 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426901+0000", 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8774e-05 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426920+0000", 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0669999999999998e-06 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426929+0000", 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1212999999999998e-05 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426990+0000", 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5549000000000005e-05 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.427066+0000", 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011327538999999999 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.438393+0000", 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011501162000000001 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:334 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 152.97377274199999, 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02568811, 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.682999+0000", 2024-02-18T00:36:07.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015711060999999998 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698710+0000", 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7644999999999994e-05 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698778+0000", 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2190000000000004e-06 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.745 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698786+0000", 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3647999999999999e-05 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698840+0000", 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8643000000000001e-05 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698859+0000", 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0097918710000000006 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.708650+0000", 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8368999999999998e-05 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.708689+0000", 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02568811 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:335 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 152.93830840199999, 2024-02-18T00:36:07.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019566244, 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718462+0000", 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0098851159999999993 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728347+0000", 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7931000000000002e-05 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728365+0000", 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7789999999999996e-06 2024-02-18T00:36:07.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728373+0000", 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7117999999999998e-05 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728430+0000", 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9799000000000001e-05 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728450+0000", 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0095431630000000003 2024-02-18T00:36:07.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.737993+0000", 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8220999999999999e-05 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.738031+0000", 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019566244 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.751 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:336 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 152.89967689700001, 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017739901999999998, 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999967 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757093+0000", 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0078518660000000007 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764945+0000", 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1234999999999999e-05 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764966+0000", 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.858e-06 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764974+0000", 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2244000000000003e-05 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.765027+0000", 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1681000000000001e-05 2024-02-18T00:36:07.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.765048+0000", 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0097585759999999997 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.774807+0000", 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9601999999999999e-05 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.774836+0000", 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017739901999999998 2024-02-18T00:36:07.755 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:500 62.c e205/123 62:348d0773:test-rados-api-smithi059-82957-14::foo71:5 v 205'15, mlcod=193'11)", 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 84.006040467999995, 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020248832000000001, 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999866 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650720+0000", 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.51e-05 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650745+0000", 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7669e-05 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650762+0000", 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.289e-06 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650770+0000", 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5961000000000001e-05 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650836+0000", 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1322000000000003e-05 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650867+0000", 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020080569 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670948+0000", 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4106000000000002e-05 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670982+0000", 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020248832000000001 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40613.0:4 155.7 155:e19d0d35:::oid_RmXattrPP_:head [rmxattr attr1 in=5b] snapc 0=[] ondisk+write+known_if_redirected e235)", 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 52.032909687999997, 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013802841, 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:07.760 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40613", 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/558528338", 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 4 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:07.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623861+0000", 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1669e-05 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623873+0000", 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2911999999999999e-05 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623886+0000", 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5609999999999999e-06 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623894+0000", 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3367000000000003e-05 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623978+0000", 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6638999999999994e-05 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.763 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.624074+0000", 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015765399999999999 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.624232+0000", 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001632643 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.625865+0000", 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051082250000000001 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.630973+0000", 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0065934679999999999 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637566+0000", 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8032000000000003e-05 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637624+0000", 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2246000000000002e-05 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:07.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637667+0000", 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013802841 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:07.766 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:07.918 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.482552+0000 osd.4 (osd.4) 171 : cluster [DBG] 62.16 scrub starts 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.486021+0000 osd.4 (osd.4) 172 : cluster [DBG] 62.16 scrub ok 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.639873+0000 osd.1 (osd.1) 149 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.641180+0000 osd.1 (osd.1) 150 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.724658+0000 osd.3 (osd.3) 165 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.725964+0000 osd.3 (osd.3) 166 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.754652+0000 osd.2 (osd.2) 125 : cluster [DBG] 62.12 deep-scrub starts 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:06.758765+0000 osd.2 (osd.2) 126 : cluster [DBG] 62.12 deep-scrub ok 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.095542+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.095632+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:07.095678+0000 mon.a (mon.0) 1986 : cluster [DBG] osdmap e288: 8 total, 8 up, 7 in 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.096396+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:07.919 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.096627+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:07.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.107272+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:07.920 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.107543+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:07.938 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-02-18T00:36:04.629715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7003,"num_objects":344,"num_object_clones":1,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":1594,"num_read_kb":1222,"num_write":1165,"num_write_kb":581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2500,"ondisk_log_size":2500,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":733,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2174216,"kb_used_data":32148,"kb_used_omap":0,"kb_used_meta":2141952,"kb_avail":654041848,"statfs":{"total":671965249536,"available":669738852352,"internally_reserved":0,"allocated":32919552,"data_stored":17995214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2193358848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2048,"num_objects":-16,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-16,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":-132,"num_write_kb":-74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-21,"num_bytes_recovered":-1536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.989199"},"pg_stats":[{"pgid":"177.17","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696781+0000","last_change":"2024-02-18T00:35:58.727894+0000","last_active":"2024-02-18T00:35:59.696781+0000","last_peered":"2024-02-18T00:35:59.696781+0000","last_clean":"2024-02-18T00:35:59.696781+0000","last_became_active":"2024-02-18T00:35:58.727589+0000","last_became_peered":"2024-02-18T00:35:58.727589+0000","last_unstale":"2024-02-18T00:35:59.696781+0000","last_undegraded":"2024-02-18T00:35:59.696781+0000","last_fullsized":"2024-02-18T00:35:59.696781+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"177.16","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696768+0000","last_change":"2024-02-18T00:35:58.663366+0000","last_active":"2024-02-18T00:35:59.696768+0000","last_peered":"2024-02-18T00:35:59.696768+0000","last_clean":"2024-02-18T00:35:59.696768+0000","last_became_active":"2024-02-18T00:35:58.663046+0000","last_became_peered":"2024-02-18T00:35:58.663046+0000","last_unstale":"2024-02-18T00:35:59.696768+0000","last_undegraded":"2024-02-18T00:35:59.696768+0000","last_fullsized":"2024-02-18T00:35:59.696768+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"177.15","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.691693+0000","last_change":"2024-02-18T00:35:58.664401+0000","last_active":"2024-02-18T00:35:58.691693+0000","last_peered":"2024-02-18T00:35:58.691693+0000","last_clean":"2024-02-18T00:35:58.691693+0000","last_became_active":"2024-02-18T00:35:58.664133+0000","last_became_peered":"2024-02-18T00:35:58.664133+0000","last_unstale":"2024-02-18T00:35:58.691693+0000","last_undegraded":"2024-02-18T00:35:58.691693+0000","last_fullsized":"2024-02-18T00:35:58.691693+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"177.14","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697034+0000","last_change":"2024-02-18T00:35:58.726496+0000","last_active":"2024-02-18T00:35:59.697034+0000","last_peered":"2024-02-18T00:35:59.697034+0000","last_clean":"2024-02-18T00:35:59.697034+0000","last_became_active":"2024-02-18T00:35:58.726205+0000","last_became_peered":"2024-02-18T00:35:58.726205+0000","last_unstale":"2024-02-18T00:35:59.697034+0000","last_undegraded":"2024-02-18T00:35:59.697034+0000","last_fullsized":"2024-02-18T00:35:59.697034+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"177.13","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.729805+0000","last_change":"2024-02-18T00:35:58.726991+0000","last_active":"2024-02-18T00:35:58.729805+0000","last_peered":"2024-02-18T00:35:58.729805+0000","last_clean":"2024-02-18T00:35:58.729805+0000","last_became_active":"2024-02-18T00:35:58.726531+0000","last_became_peered":"2024-02-18T00:35:58.726531+0000","last_unstale":"2024-02-18T00:35:58.729805+0000","last_undegraded":"2024-02-18T00:35:58.729805+0000","last_fullsized":"2024-02-18T00:35:58.729805+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"177.12","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.692671+0000","last_change":"2024-02-18T00:35:58.692191+0000","last_active":"2024-02-18T00:35:58.692671+0000","last_peered":"2024-02-18T00:35:58.692671+0000","last_clean":"2024-02-18T00:35:58.692671+0000","last_became_active":"2024-02-18T00:35:58.691876+0000","last_became_peered":"2024-02-18T00:35:58.691876+0000","last_unstale":"2024-02-18T00:35:58.692671+0000","last_undegraded":"2024-02-18T00:35:58.692671+0000","last_fullsized":"2024-02-18T00:35:58.692671+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"177.11","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695742+0000","last_change":"2024-02-18T00:35:58.677912+0000","last_active":"2024-02-18T00:35:59.695742+0000","last_peered":"2024-02-18T00:35:59.695742+0000","last_clean":"2024-02-18T00:35:59.695742+0000","last_became_active":"2024-02-18T00:35:58.677610+0000","last_became_peered":"2024-02-18T00:35:58.677610+0000","last_unstale":"2024-02-18T00:35:59.695742+0000","last_undegraded":"2024-02-18T00:35:59.695742+0000","last_fullsized":"2024-02-18T00:35:59.695742+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"177.10","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.729497+0000","last_change":"2024-02-18T00:35:58.726140+0000","last_active":"2024-02-18T00:35:58.729497+0000","last_peered":"2024-02-18T00:35:58.729497+0000","last_clean":"2024-02-18T00:35:58.729497+0000","last_became_active":"2024-02-18T00:35:58.725853+0000","last_became_peered":"2024-02-18T00:35:58.725853+0000","last_unstale":"2024-02-18T00:35:58.729497+0000","last_undegraded":"2024-02-18T00:35:58.729497+0000","last_fullsized":"2024-02-18T00:35:58.729497+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"177.f","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696899+0000","last_change":"2024-02-18T00:35:58.692929+0000","last_active":"2024-02-18T00:35:59.696899+0000","last_peered":"2024-02-18T00:35:59.696899+0000","last_clean":"2024-02-18T00:35:59.696899+0000","last_became_active":"2024-02-18T00:35:58.692574+0000","last_became_peered":"2024-02-18T00:35:58.692574+0000","last_unstale":"2024-02-18T00:35:59.696899+0000","last_undegraded":"2024-02-18T00:35:59.696899+0000","last_fullsized":"2024-02-18T00:35:59.696899+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"177.e","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.644028+0000","last_change":"2024-02-18T00:35:58.678920+0000","last_active":"2024-02-18T00:35:59.644028+0000","last_peered":"2024-02-18T00:35:59.644028+0000","last_clean":"2024-02-18T00:35:59.644028+0000","last_became_active":"2024-02-18T00:35:58.678652+0000","last_became_peered":"2024-02-18T00:35:58.678652+0000","last_unstale":"2024-02-18T00:35:59.644028+0000","last_undegraded":"2024-02-18T00:35:59.644028+0000","last_fullsized":"2024-02-18T00:35:59.644028+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"177.d","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696630+0000","last_change":"2024-02-18T00:35:58.727908+0000","last_active":"2024-02-18T00:35:59.696630+0000","last_peered":"2024-02-18T00:35:59.696630+0000","last_clean":"2024-02-18T00:35:59.696630+0000","last_became_active":"2024-02-18T00:35:58.727599+0000","last_became_peered":"2024-02-18T00:35:58.727599+0000","last_unstale":"2024-02-18T00:35:59.696630+0000","last_undegraded":"2024-02-18T00:35:59.696630+0000","last_fullsized":"2024-02-18T00:35:59.696630+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"177.c","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.642812+0000","last_change":"2024-02-18T00:35:58.691600+0000","last_active":"2024-02-18T00:35:59.642812+0000","last_peered":"2024-02-18T00:35:59.642812+0000","last_clean":"2024-02-18T00:35:59.642812+0000","last_became_active":"2024-02-18T00:35:58.691288+0000","last_became_peered":"2024-02-18T00:35:58.691288+0000","last_unstale":"2024-02-18T00:35:59.642812+0000","last_undegraded":"2024-02-18T00:35:59.642812+0000","last_fullsized":"2024-02-18T00:35:59.642812+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"177.b","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697737+0000","last_change":"2024-02-18T00:35:58.727815+0000","last_active":"2024-02-18T00:35:59.697737+0000","last_peered":"2024-02-18T00:35:59.697737+0000","last_clean":"2024-02-18T00:35:59.697737+0000","last_became_active":"2024-02-18T00:35:58.727530+0000","last_became_peered":"2024-02-18T00:35:58.727530+0000","last_unstale":"2024-02-18T00:35:59.697737+0000","last_undegraded":"2024-02-18T00:35:59.697737+0000","last_fullsized":"2024-02-18T00:35:59.697737+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"177.a","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696644+0000","last_change":"2024-02-18T00:35:58.660455+0000","last_active":"2024-02-18T00:35:59.696644+0000","last_peered":"2024-02-18T00:35:59.696644+0000","last_clean":"2024-02-18T00:35:59.696644+0000","last_became_active":"2024-02-18T00:35:58.659039+0000","last_became_peered":"2024-02-18T00:35:58.659039+0000","last_unstale":"2024-02-18T00:35:59.696644+0000","last_undegraded":"2024-02-18T00:35:59.696644+0000","last_fullsized":"2024-02-18T00:35:59.696644+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"177.9","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695249+0000","last_change":"2024-02-18T00:35:58.669269+0000","last_active":"2024-02-18T00:35:59.695249+0000","last_peered":"2024-02-18T00:35:59.695249+0000","last_clean":"2024-02-18T00:35:59.695249+0000","last_became_active":"2024-02-18T00:35:58.668944+0000","last_became_peered":"2024-02-18T00:35:58.668944+0000","last_unstale":"2024-02-18T00:35:59.695249+0000","last_undegraded":"2024-02-18T00:35:59.695249+0000","last_fullsized":"2024-02-18T00:35:59.695249+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"177.8","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.694928+0000","last_change":"2024-02-18T00:35:58.662240+0000","last_active":"2024-02-18T00:35:59.694928+0000","last_peered":"2024-02-18T00:35:59.694928+0000","last_clean":"2024-02-18T00:35:59.694928+0000","last_became_active":"2024-02-18T00:35:58.661920+0000","last_became_peered":"2024-02-18T00:35:58.661920+0000","last_unstale":"2024-02-18T00:35:59.694928+0000","last_undegraded":"2024-02-18T00:35:59.694928+0000","last_fullsized":"2024-02-18T00:35:59.694928+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"179.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.0","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.729794+0000","last_change":"2024-02-18T00:35:58.727560+0000","last_active":"2024-02-18T00:35:58.729794+0000","last_peered":"2024-02-18T00:35:58.729794+0000","last_clean":"2024-02-18T00:35:58.729794+0000","last_became_active":"2024-02-18T00:35:58.727246+0000","last_became_peered":"2024-02-18T00:35:58.727246+0000","last_unstale":"2024-02-18T00:35:58.729794+0000","last_undegraded":"2024-02-18T00:35:58.729794+0000","last_fullsized":"2024-02-18T00:35:58.729794+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"179.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.1","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695191+0000","last_change":"2024-02-18T00:35:58.726068+0000","last_active":"2024-02-18T00:35:59.695191+0000","last_peered":"2024-02-18T00:35:59.695191+0000","last_clean":"2024-02-18T00:35:59.695191+0000","last_became_active":"2024-02-18T00:35:58.725816+0000","last_became_peered":"2024-02-18T00:35:58.725816+0000","last_unstale":"2024-02-18T00:35:59.695191+0000","last_undegraded":"2024-02-18T00:35:59.695191+0000","last_fullsized":"2024-02-18T00:35:59.695191+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"179.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.2","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641536+0000","last_change":"2024-02-18T00:35:58.678895+0000","last_active":"2024-02-18T00:35:59.641536+0000","last_peered":"2024-02-18T00:35:59.641536+0000","last_clean":"2024-02-18T00:35:59.641536+0000","last_became_active":"2024-02-18T00:35:58.678600+0000","last_became_peered":"2024-02-18T00:35:58.678600+0000","last_unstale":"2024-02-18T00:35:59.641536+0000","last_undegraded":"2024-02-18T00:35:59.641536+0000","last_fullsized":"2024-02-18T00:35:59.641536+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"179.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.3","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.729832+0000","last_change":"2024-02-18T00:35:58.727567+0000","last_active":"2024-02-18T00:35:58.729832+0000","last_peered":"2024-02-18T00:35:58.729832+0000","last_clean":"2024-02-18T00:35:58.729832+0000","last_became_active":"2024-02-18T00:35:58.727275+0000","last_became_peered":"2024-02-18T00:35:58.727275+0000","last_unstale":"2024-02-18T00:35:58.729832+0000","last_undegraded":"2024-02-18T00:35:58.729832+0000","last_fullsized":"2024-02-18T00:35:58.729832+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"179.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.4","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.727488+0000","last_change":"2024-02-18T00:35:58.681242+0000","last_active":"2024-02-18T00:35:58.727488+0000","last_peered":"2024-02-18T00:35:58.727488+0000","last_clean":"2024-02-18T00:35:58.727488+0000","last_became_active":"2024-02-18T00:35:58.680973+0000","last_became_peered":"2024-02-18T00:35:58.680973+0000","last_unstale":"2024-02-18T00:35:58.727488+0000","last_undegraded":"2024-02-18T00:35:58.727488+0000","last_fullsized":"2024-02-18T00:35:58.727488+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"179.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.5","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697953+0000","last_change":"2024-02-18T00:35:58.662467+0000","last_active":"2024-02-18T00:35:59.697953+0000","last_peered":"2024-02-18T00:35:59.697953+0000","last_clean":"2024-02-18T00:35:59.697953+0000","last_became_active":"2024-02-18T00:35:58.662194+0000","last_became_peered":"2024-02-18T00:35:58.662194+0000","last_unstale":"2024-02-18T00:35:59.697953+0000","last_undegraded":"2024-02-18T00:35:59.697953+0000","last_fullsized":"2024-02-18T00:35:59.697953+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"179.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.6","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696839+0000","last_change":"2024-02-18T00:35:58.667188+0000","last_active":"2024-02-18T00:35:59.696839+0000","last_peered":"2024-02-18T00:35:59.696839+0000","last_clean":"2024-02-18T00:35:59.696839+0000","last_became_active":"2024-02-18T00:35:58.666890+0000","last_became_peered":"2024-02-18T00:35:58.666890+0000","last_unstale":"2024-02-18T00:35:59.696839+0000","last_undegraded":"2024-02-18T00:35:59.696839+0000","last_fullsized":"2024-02-18T00:35:59.696839+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"179.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"178.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:03.620594+0000","last_change":"2024-02-18T00:36:03.620594+0000","last_active":"2024-02-18T00:36:03.620594+0000","last_peered":"2024-02-18T00:36:03.620594+0000","last_clean":"2024-02-18T00:36:03.620594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:03.620594+0000","last_undegraded":"2024-02-18T00:36:03.620594+0000","last_fullsized":"2024-02-18T00:36:03.620594+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:03.620594+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.620594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"177.7","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697924+0000","last_change":"2024-02-18T00:35:58.692415+0000","last_active":"2024-02-18T00:35:59.697924+0000","last_peered":"2024-02-18T00:35:59.697924+0000","last_clean":"2024-02-18T00:35:59.697924+0000","last_became_active":"2024-02-18T00:35:58.692068+0000","last_became_peered":"2024-02-18T00:35:58.692068+0000","last_unstale":"2024-02-18T00:35:59.697924+0000","last_undegraded":"2024-02-18T00:35:59.697924+0000","last_fullsized":"2024-02-18T00:35:59.697924+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"177.1a","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697434+0000","last_change":"2024-02-18T00:35:58.727931+0000","last_active":"2024-02-18T00:35:59.697434+0000","last_peered":"2024-02-18T00:35:59.697434+0000","last_clean":"2024-02-18T00:35:59.697434+0000","last_became_active":"2024-02-18T00:35:58.727456+0000","last_became_peered":"2024-02-18T00:35:58.727456+0000","last_unstale":"2024-02-18T00:35:59.697434+0000","last_undegraded":"2024-02-18T00:35:59.697434+0000","last_fullsized":"2024-02-18T00:35:59.697434+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"177.1b","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695460+0000","last_change":"2024-02-18T00:35:58.666080+0000","last_active":"2024-02-18T00:35:59.695460+0000","last_peered":"2024-02-18T00:35:59.695460+0000","last_clean":"2024-02-18T00:35:59.695460+0000","last_became_active":"2024-02-18T00:35:58.665746+0000","last_became_peered":"2024-02-18T00:35:58.665746+0000","last_unstale":"2024-02-18T00:35:59.695460+0000","last_undegraded":"2024-02-18T00:35:59.695460+0000","last_fullsized":"2024-02-18T00:35:59.695460+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"177.18","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696037+0000","last_change":"2024-02-18T00:35:58.666094+0000","last_active":"2024-02-18T00:35:59.696037+0000","last_peered":"2024-02-18T00:35:59.696037+0000","last_clean":"2024-02-18T00:35:59.696037+0000","last_became_active":"2024-02-18T00:35:58.665782+0000","last_became_peered":"2024-02-18T00:35:58.665782+0000","last_unstale":"2024-02-18T00:35:59.696037+0000","last_undegraded":"2024-02-18T00:35:59.696037+0000","last_fullsized":"2024-02-18T00:35:59.696037+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"177.19","version":"0'0","reported_seq":10,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.727679+0000","last_change":"2024-02-18T00:35:58.692538+0000","last_active":"2024-02-18T00:35:58.727679+0000","last_peered":"2024-02-18T00:35:58.727679+0000","last_clean":"2024-02-18T00:35:58.727679+0000","last_became_active":"2024-02-18T00:35:58.692198+0000","last_became_peered":"2024-02-18T00:35:58.692198+0000","last_unstale":"2024-02-18T00:35:58.727679+0000","last_undegraded":"2024-02-18T00:35:58.727679+0000","last_fullsized":"2024-02-18T00:35:58.727679+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"177.1e","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643940+0000","last_change":"2024-02-18T00:35:58.679009+0000","last_active":"2024-02-18T00:35:59.643940+0000","last_peered":"2024-02-18T00:35:59.643940+0000","last_clean":"2024-02-18T00:35:59.643940+0000","last_became_active":"2024-02-18T00:35:58.678687+0000","last_became_peered":"2024-02-18T00:35:58.678687+0000","last_unstale":"2024-02-18T00:35:59.643940+0000","last_undegraded":"2024-02-18T00:35:59.643940+0000","last_fullsized":"2024-02-18T00:35:59.643940+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"177.1f","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696712+0000","last_change":"2024-02-18T00:35:58.727733+0000","last_active":"2024-02-18T00:35:59.696712+0000","last_peered":"2024-02-18T00:35:59.696712+0000","last_clean":"2024-02-18T00:35:59.696712+0000","last_became_active":"2024-02-18T00:35:58.727441+0000","last_became_peered":"2024-02-18T00:35:58.727441+0000","last_unstale":"2024-02-18T00:35:59.696712+0000","last_undegraded":"2024-02-18T00:35:59.696712+0000","last_fullsized":"2024-02-18T00:35:59.696712+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"177.1c","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696936+0000","last_change":"2024-02-18T00:35:58.664376+0000","last_active":"2024-02-18T00:35:59.696936+0000","last_peered":"2024-02-18T00:35:59.696936+0000","last_clean":"2024-02-18T00:35:59.696936+0000","last_became_active":"2024-02-18T00:35:58.664101+0000","last_became_peered":"2024-02-18T00:35:58.664101+0000","last_unstale":"2024-02-18T00:35:59.696936+0000","last_undegraded":"2024-02-18T00:35:59.696936+0000","last_fullsized":"2024-02-18T00:35:59.696936+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"177.1d","version":"0'0","reported_seq":11,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695844+0000","last_change":"2024-02-18T00:35:58.667025+0000","last_active":"2024-02-18T00:35:59.695844+0000","last_peered":"2024-02-18T00:35:59.695844+0000","last_clean":"2024-02-18T00:35:59.695844+0000","last_became_active":"2024-02-18T00:35:58.666695+0000","last_became_peered":"2024-02-18T00:35:58.666695+0000","last_unstale":"2024-02-18T00:35:59.695844+0000","last_undegraded":"2024-02-18T00:35:59.695844+0000","last_fullsized":"2024-02-18T00:35:59.695844+0000","mapping_epoch":279,"log_start":"0'0","ondisk_log_start":"0'0","created":279,"last_epoch_clean":280,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:35:57.627636+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.627636+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641791+0000","last_change":"2024-02-18T00:35:45.803022+0000","last_active":"2024-02-18T00:35:59.641791+0000","last_peered":"2024-02-18T00:35:59.641791+0000","last_clean":"2024-02-18T00:35:59.641791+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:35:59.641791+0000","last_undegraded":"2024-02-18T00:35:59.641791+0000","last_fullsized":"2024-02-18T00:35:59.641791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"0'0","reported_seq":209,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.663355+0000","last_change":"2024-02-18T00:35:37.631634+0000","last_active":"2024-02-18T00:35:58.663355+0000","last_peered":"2024-02-18T00:35:58.663355+0000","last_clean":"2024-02-18T00:35:58.663355+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:35:58.663355+0000","last_undegraded":"2024-02-18T00:35:58.663355+0000","last_fullsized":"2024-02-18T00:35:58.663355+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":278,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697647+0000","last_change":"2024-02-18T00:35:56.424960+0000","last_active":"2024-02-18T00:35:59.697647+0000","last_peered":"2024-02-18T00:35:59.697647+0000","last_clean":"2024-02-18T00:35:59.697647+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:35:59.697647+0000","last_undegraded":"2024-02-18T00:35:59.697647+0000","last_fullsized":"2024-02-18T00:35:59.697647+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"245'73","reported_seq":272,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643432+0000","last_change":"2024-02-18T00:35:43.802410+0000","last_active":"2024-02-18T00:35:59.643432+0000","last_peered":"2024-02-18T00:35:59.643432+0000","last_clean":"2024-02-18T00:35:59.643432+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:35:59.643432+0000","last_undegraded":"2024-02-18T00:35:59.643432+0000","last_fullsized":"2024-02-18T00:35:59.643432+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"230'120","reported_seq":312,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695609+0000","last_change":"2024-02-18T00:35:37.643034+0000","last_active":"2024-02-18T00:35:59.695609+0000","last_peered":"2024-02-18T00:35:59.695609+0000","last_clean":"2024-02-18T00:35:59.695609+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:35:59.695609+0000","last_undegraded":"2024-02-18T00:35:59.695609+0000","last_fullsized":"2024-02-18T00:35:59.695609+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"240'90","reported_seq":281,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.651205+0000","last_change":"2024-02-18T00:35:37.633264+0000","last_active":"2024-02-18T00:35:58.651205+0000","last_peered":"2024-02-18T00:35:58.651205+0000","last_clean":"2024-02-18T00:35:58.651205+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:35:58.651205+0000","last_undegraded":"2024-02-18T00:35:58.651205+0000","last_fullsized":"2024-02-18T00:35:58.651205+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"267'124","reported_seq":285,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.644250+0000","last_change":"2024-02-18T00:35:47.805967+0000","last_active":"2024-02-18T00:35:59.644250+0000","last_peered":"2024-02-18T00:35:59.644250+0000","last_clean":"2024-02-18T00:35:59.644250+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:35:59.644250+0000","last_undegraded":"2024-02-18T00:35:59.644250+0000","last_fullsized":"2024-02-18T00:35:59.644250+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":124,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"267'73","reported_seq":266,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.642193+0000","last_change":"2024-02-18T00:35:47.805748+0000","last_active":"2024-02-18T00:35:59.642193+0000","last_peered":"2024-02-18T00:35:59.642193+0000","last_clean":"2024-02-18T00:35:59.642193+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:35:59.642193+0000","last_undegraded":"2024-02-18T00:35:59.642193+0000","last_fullsized":"2024-02-18T00:35:59.642193+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"215'68","reported_seq":231,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.668589+0000","last_change":"2024-02-18T00:35:37.632402+0000","last_active":"2024-02-18T00:35:58.668589+0000","last_peered":"2024-02-18T00:35:58.668589+0000","last_clean":"2024-02-18T00:35:58.668589+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:35:58.668589+0000","last_undegraded":"2024-02-18T00:35:58.668589+0000","last_fullsized":"2024-02-18T00:35:58.668589+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"229'51","reported_seq":248,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.651148+0000","last_change":"2024-02-18T00:35:37.632112+0000","last_active":"2024-02-18T00:35:58.651148+0000","last_peered":"2024-02-18T00:35:58.651148+0000","last_clean":"2024-02-18T00:35:58.651148+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:35:58.651148+0000","last_undegraded":"2024-02-18T00:35:58.651148+0000","last_fullsized":"2024-02-18T00:35:58.651148+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"265'39","reported_seq":220,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-18T00:36:00.740026+0000","last_change":"2024-02-18T00:36:00.740026+0000","last_active":"2024-02-18T00:36:00.740026+0000","last_peered":"2024-02-18T00:36:00.740026+0000","last_clean":"2024-02-18T00:36:00.740026+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:36:00.740026+0000","last_undegraded":"2024-02-18T00:36:00.740026+0000","last_fullsized":"2024-02-18T00:36:00.740026+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"246'54","reported_seq":230,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697872+0000","last_change":"2024-02-18T00:35:37.632982+0000","last_active":"2024-02-18T00:35:59.697872+0000","last_peered":"2024-02-18T00:35:59.697872+0000","last_clean":"2024-02-18T00:35:59.697872+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:35:59.697872+0000","last_undegraded":"2024-02-18T00:35:59.697872+0000","last_fullsized":"2024-02-18T00:35:59.697872+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"213'17","reported_seq":221,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697401+0000","last_change":"2024-02-18T00:35:37.634574+0000","last_active":"2024-02-18T00:35:59.697401+0000","last_peered":"2024-02-18T00:35:59.697401+0000","last_clean":"2024-02-18T00:35:59.697401+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:35:59.697401+0000","last_undegraded":"2024-02-18T00:35:59.697401+0000","last_fullsized":"2024-02-18T00:35:59.697401+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'28","reported_seq":252,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.651119+0000","last_change":"2024-02-18T00:35:39.639403+0000","last_active":"2024-02-18T00:35:58.651119+0000","last_peered":"2024-02-18T00:35:58.651119+0000","last_clean":"2024-02-18T00:35:58.651119+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:35:58.651119+0000","last_undegraded":"2024-02-18T00:35:58.651119+0000","last_fullsized":"2024-02-18T00:35:58.651119+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"259'73","reported_seq":264,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696970+0000","last_change":"2024-02-18T00:35:39.645326+0000","last_active":"2024-02-18T00:35:59.696970+0000","last_peered":"2024-02-18T00:35:59.696970+0000","last_clean":"2024-02-18T00:35:59.696970+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:35:59.696970+0000","last_undegraded":"2024-02-18T00:35:59.696970+0000","last_fullsized":"2024-02-18T00:35:59.696970+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":253,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.658157+0000","last_change":"2024-02-18T00:35:37.634002+0000","last_active":"2024-02-18T00:35:58.658157+0000","last_peered":"2024-02-18T00:35:58.658157+0000","last_clean":"2024-02-18T00:35:58.658157+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:35:58.658157+0000","last_undegraded":"2024-02-18T00:35:58.658157+0000","last_fullsized":"2024-02-18T00:35:58.658157+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"209'56","reported_seq":266,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.668499+0000","last_change":"2024-02-18T00:35:37.632250+0000","last_active":"2024-02-18T00:35:58.668499+0000","last_peered":"2024-02-18T00:35:58.668499+0000","last_clean":"2024-02-18T00:35:58.668499+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:35:58.668499+0000","last_undegraded":"2024-02-18T00:35:58.668499+0000","last_fullsized":"2024-02-18T00:35:58.668499+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"214'19","reported_seq":229,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.663257+0000","last_change":"2024-02-18T00:35:53.563686+0000","last_active":"2024-02-18T00:35:58.663257+0000","last_peered":"2024-02-18T00:35:58.663257+0000","last_clean":"2024-02-18T00:35:58.663257+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:35:58.663257+0000","last_undegraded":"2024-02-18T00:35:58.663257+0000","last_fullsized":"2024-02-18T00:35:58.663257+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:35:53.563562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.563562+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"257'68","reported_seq":263,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641448+0000","last_change":"2024-02-18T00:35:47.805379+0000","last_active":"2024-02-18T00:35:59.641448+0000","last_peered":"2024-02-18T00:35:59.641448+0000","last_clean":"2024-02-18T00:35:59.641448+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:35:59.641448+0000","last_undegraded":"2024-02-18T00:35:59.641448+0000","last_fullsized":"2024-02-18T00:35:59.641448+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"210'17","reported_seq":221,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695584+0000","last_change":"2024-02-18T00:35:37.643050+0000","last_active":"2024-02-18T00:35:59.695584+0000","last_peered":"2024-02-18T00:35:59.695584+0000","last_clean":"2024-02-18T00:35:59.695584+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:35:59.695584+0000","last_undegraded":"2024-02-18T00:35:59.695584+0000","last_fullsized":"2024-02-18T00:35:59.695584+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":232,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.651637+0000","last_change":"2024-02-18T00:35:37.631924+0000","last_active":"2024-02-18T00:35:58.651637+0000","last_peered":"2024-02-18T00:35:58.651637+0000","last_clean":"2024-02-18T00:35:58.651637+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:35:58.651637+0000","last_undegraded":"2024-02-18T00:35:58.651637+0000","last_fullsized":"2024-02-18T00:35:58.651637+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"244'18","reported_seq":237,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697148+0000","last_change":"2024-02-18T00:35:37.652825+0000","last_active":"2024-02-18T00:35:59.697148+0000","last_peered":"2024-02-18T00:35:59.697148+0000","last_clean":"2024-02-18T00:35:59.697148+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:35:59.697148+0000","last_undegraded":"2024-02-18T00:35:59.697148+0000","last_fullsized":"2024-02-18T00:35:59.697148+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":300,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696286+0000","last_change":"2024-02-18T00:35:37.633131+0000","last_active":"2024-02-18T00:35:59.696286+0000","last_peered":"2024-02-18T00:35:59.696286+0000","last_clean":"2024-02-18T00:35:59.696286+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:35:59.696286+0000","last_undegraded":"2024-02-18T00:35:59.696286+0000","last_fullsized":"2024-02-18T00:35:59.696286+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":196,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695571+0000","last_change":"2024-02-18T00:34:15.230999+0000","last_active":"2024-02-18T00:35:59.695571+0000","last_peered":"2024-02-18T00:35:59.695571+0000","last_clean":"2024-02-18T00:35:59.695571+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:35:59.695571+0000","last_undegraded":"2024-02-18T00:35:59.695571+0000","last_fullsized":"2024-02-18T00:35:59.695571+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.230938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.230938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":311,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.642971+0000","last_change":"2024-02-18T00:35:47.806131+0000","last_active":"2024-02-18T00:35:59.642971+0000","last_peered":"2024-02-18T00:35:59.642971+0000","last_clean":"2024-02-18T00:35:59.642971+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:35:59.642971+0000","last_undegraded":"2024-02-18T00:35:59.642971+0000","last_fullsized":"2024-02-18T00:35:59.642971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":196,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695798+0000","last_change":"2024-02-18T00:34:39.758860+0000","last_active":"2024-02-18T00:35:59.695798+0000","last_peered":"2024-02-18T00:35:59.695798+0000","last_clean":"2024-02-18T00:35:59.695798+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:35:59.695798+0000","last_undegraded":"2024-02-18T00:35:59.695798+0000","last_fullsized":"2024-02-18T00:35:59.695798+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.758794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.758794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":249,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652780+0000","last_change":"2024-02-18T00:35:53.400527+0000","last_active":"2024-02-18T00:35:58.652780+0000","last_peered":"2024-02-18T00:35:58.652780+0000","last_clean":"2024-02-18T00:35:58.652780+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:35:58.652780+0000","last_undegraded":"2024-02-18T00:35:58.652780+0000","last_fullsized":"2024-02-18T00:35:58.652780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":151,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.658253+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:35:58.658253+0000","last_peered":"2024-02-18T00:35:58.658253+0000","last_clean":"2024-02-18T00:35:58.658253+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:35:58.658253+0000","last_undegraded":"2024-02-18T00:35:58.658253+0000","last_fullsized":"2024-02-18T00:35:58.658253+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":305,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697588+0000","last_change":"2024-02-18T00:35:57.471793+0000","last_active":"2024-02-18T00:35:59.697588+0000","last_peered":"2024-02-18T00:35:59.697588+0000","last_clean":"2024-02-18T00:35:59.697588+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:35:59.697588+0000","last_undegraded":"2024-02-18T00:35:59.697588+0000","last_fullsized":"2024-02-18T00:35:59.697588+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":149,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.656365+0000","last_change":"2024-02-18T00:34:26.276135+0000","last_active":"2024-02-18T00:35:58.656365+0000","last_peered":"2024-02-18T00:35:58.656365+0000","last_clean":"2024-02-18T00:35:58.656365+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:35:58.656365+0000","last_undegraded":"2024-02-18T00:35:58.656365+0000","last_fullsized":"2024-02-18T00:35:58.656365+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.276071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.276071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":282,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695477+0000","last_change":"2024-02-18T00:35:55.396191+0000","last_active":"2024-02-18T00:35:59.695477+0000","last_peered":"2024-02-18T00:35:59.695477+0000","last_clean":"2024-02-18T00:35:59.695477+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:35:59.695477+0000","last_undegraded":"2024-02-18T00:35:59.695477+0000","last_fullsized":"2024-02-18T00:35:59.695477+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":201,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652984+0000","last_change":"2024-02-18T00:33:43.346148+0000","last_active":"2024-02-18T00:35:58.652984+0000","last_peered":"2024-02-18T00:35:58.652984+0000","last_clean":"2024-02-18T00:35:58.652984+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:35:58.652984+0000","last_undegraded":"2024-02-18T00:35:58.652984+0000","last_fullsized":"2024-02-18T00:35:58.652984+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:43.346088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.346088+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"213'17","reported_seq":227,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695701+0000","last_change":"2024-02-18T00:35:37.628605+0000","last_active":"2024-02-18T00:35:59.695701+0000","last_peered":"2024-02-18T00:35:59.695701+0000","last_clean":"2024-02-18T00:35:59.695701+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:35:59.695701+0000","last_undegraded":"2024-02-18T00:35:59.695701+0000","last_fullsized":"2024-02-18T00:35:59.695701+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":196,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696378+0000","last_change":"2024-02-18T00:34:17.271924+0000","last_active":"2024-02-18T00:35:59.696378+0000","last_peered":"2024-02-18T00:35:59.696378+0000","last_clean":"2024-02-18T00:35:59.696378+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:35:59.696378+0000","last_undegraded":"2024-02-18T00:35:59.696378+0000","last_fullsized":"2024-02-18T00:35:59.696378+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.271867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.271867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":256,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.642057+0000","last_change":"2024-02-18T00:35:45.804047+0000","last_active":"2024-02-18T00:35:59.642057+0000","last_peered":"2024-02-18T00:35:59.642057+0000","last_clean":"2024-02-18T00:35:59.642057+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:35:59.642057+0000","last_undegraded":"2024-02-18T00:35:59.642057+0000","last_fullsized":"2024-02-18T00:35:59.642057+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":187,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697920+0000","last_change":"2024-02-18T00:33:55.783013+0000","last_active":"2024-02-18T00:35:59.697920+0000","last_peered":"2024-02-18T00:35:59.697920+0000","last_clean":"2024-02-18T00:35:59.697920+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:35:59.697920+0000","last_undegraded":"2024-02-18T00:35:59.697920+0000","last_fullsized":"2024-02-18T00:35:59.697920+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.782955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.782955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":216,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696752+0000","last_change":"2024-02-18T00:35:37.652333+0000","last_active":"2024-02-18T00:35:59.696752+0000","last_peered":"2024-02-18T00:35:59.696752+0000","last_clean":"2024-02-18T00:35:59.696752+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:35:59.696752+0000","last_undegraded":"2024-02-18T00:35:59.696752+0000","last_fullsized":"2024-02-18T00:35:59.696752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698005+0000","last_change":"2024-02-18T00:33:49.344703+0000","last_active":"2024-02-18T00:35:59.698005+0000","last_peered":"2024-02-18T00:35:59.698005+0000","last_clean":"2024-02-18T00:35:59.698005+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:35:59.698005+0000","last_undegraded":"2024-02-18T00:35:59.698005+0000","last_fullsized":"2024-02-18T00:35:59.698005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.344637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.344637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":222,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698117+0000","last_change":"2024-02-18T00:35:37.659096+0000","last_active":"2024-02-18T00:35:59.698117+0000","last_peered":"2024-02-18T00:35:59.698117+0000","last_clean":"2024-02-18T00:35:59.698117+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:35:59.698117+0000","last_undegraded":"2024-02-18T00:35:59.698117+0000","last_fullsized":"2024-02-18T00:35:59.698117+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":267,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.642268+0000","last_change":"2024-02-18T00:33:48.736504+0000","last_active":"2024-02-18T00:35:59.642268+0000","last_peered":"2024-02-18T00:35:59.642268+0000","last_clean":"2024-02-18T00:35:59.642268+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:35:59.642268+0000","last_undegraded":"2024-02-18T00:35:59.642268+0000","last_fullsized":"2024-02-18T00:35:59.642268+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.834878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.834878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":227,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696236+0000","last_change":"2024-02-18T00:35:37.652285+0000","last_active":"2024-02-18T00:35:59.696236+0000","last_peered":"2024-02-18T00:35:59.696236+0000","last_clean":"2024-02-18T00:35:59.696236+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:35:59.696236+0000","last_undegraded":"2024-02-18T00:35:59.696236+0000","last_fullsized":"2024-02-18T00:35:59.696236+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":221,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695141+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:35:59.695141+0000","last_peered":"2024-02-18T00:35:59.695141+0000","last_clean":"2024-02-18T00:35:59.695141+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:35:59.695141+0000","last_undegraded":"2024-02-18T00:35:59.695141+0000","last_fullsized":"2024-02-18T00:35:59.695141+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":193,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696267+0000","last_change":"2024-02-18T00:33:48.727714+0000","last_active":"2024-02-18T00:35:59.696267+0000","last_peered":"2024-02-18T00:35:59.696267+0000","last_clean":"2024-02-18T00:35:59.696267+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:35:59.696267+0000","last_undegraded":"2024-02-18T00:35:59.696267+0000","last_fullsized":"2024-02-18T00:35:59.696267+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":199,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696137+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:35:59.696137+0000","last_peered":"2024-02-18T00:35:59.696137+0000","last_clean":"2024-02-18T00:35:59.696137+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:35:59.696137+0000","last_undegraded":"2024-02-18T00:35:59.696137+0000","last_fullsized":"2024-02-18T00:35:59.696137+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":331,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695241+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:35:59.695241+0000","last_peered":"2024-02-18T00:35:59.695241+0000","last_clean":"2024-02-18T00:35:59.695241+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:35:59.695241+0000","last_undegraded":"2024-02-18T00:35:59.695241+0000","last_fullsized":"2024-02-18T00:35:59.695241+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":260,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.644600+0000","last_change":"2024-02-18T00:34:26.060181+0000","last_active":"2024-02-18T00:35:59.644600+0000","last_peered":"2024-02-18T00:35:59.644600+0000","last_clean":"2024-02-18T00:35:59.644600+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:35:59.644600+0000","last_undegraded":"2024-02-18T00:35:59.644600+0000","last_fullsized":"2024-02-18T00:35:59.644600+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.060096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.060096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695926+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:35:59.695926+0000","last_peered":"2024-02-18T00:35:59.695926+0000","last_clean":"2024-02-18T00:35:59.695926+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:35:59.695926+0000","last_undegraded":"2024-02-18T00:35:59.695926+0000","last_fullsized":"2024-02-18T00:35:59.695926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":188,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.671023+0000","last_change":"2024-02-18T00:35:20.428964+0000","last_active":"2024-02-18T00:35:58.671023+0000","last_peered":"2024-02-18T00:35:58.671023+0000","last_clean":"2024-02-18T00:35:58.671023+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:35:58.671023+0000","last_undegraded":"2024-02-18T00:35:58.671023+0000","last_fullsized":"2024-02-18T00:35:58.671023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:20.428893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:20.428893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":298,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696383+0000","last_change":"2024-02-18T00:35:54.188262+0000","last_active":"2024-02-18T00:35:59.696383+0000","last_peered":"2024-02-18T00:35:59.696383+0000","last_clean":"2024-02-18T00:35:59.696383+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:35:59.696383+0000","last_undegraded":"2024-02-18T00:35:59.696383+0000","last_fullsized":"2024-02-18T00:35:59.696383+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:35:54.188188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.188188+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":248,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.670876+0000","last_change":"2024-02-18T00:34:24.334917+0000","last_active":"2024-02-18T00:35:58.670876+0000","last_peered":"2024-02-18T00:35:58.670876+0000","last_clean":"2024-02-18T00:35:58.670876+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:35:58.670876+0000","last_undegraded":"2024-02-18T00:35:58.670876+0000","last_fullsized":"2024-02-18T00:35:58.670876+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.334814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.334814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641588+0000","last_change":"2024-02-18T00:33:42.768749+0000","last_active":"2024-02-18T00:35:59.641588+0000","last_peered":"2024-02-18T00:35:59.641588+0000","last_clean":"2024-02-18T00:35:59.641588+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:35:59.641588+0000","last_undegraded":"2024-02-18T00:35:59.641588+0000","last_fullsized":"2024-02-18T00:35:59.641588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.768630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.768630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":260,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695060+0000","last_change":"2024-02-18T00:35:42.683115+0000","last_active":"2024-02-18T00:35:59.695060+0000","last_peered":"2024-02-18T00:35:59.695060+0000","last_clean":"2024-02-18T00:35:59.695060+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:35:59.695060+0000","last_undegraded":"2024-02-18T00:35:59.695060+0000","last_fullsized":"2024-02-18T00:35:59.695060+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":193,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697578+0000","last_change":"2024-02-18T00:33:48.745303+0000","last_active":"2024-02-18T00:35:59.697578+0000","last_peered":"2024-02-18T00:35:59.697578+0000","last_clean":"2024-02-18T00:35:59.697578+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:35:59.697578+0000","last_undegraded":"2024-02-18T00:35:59.697578+0000","last_fullsized":"2024-02-18T00:35:59.697578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":330,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698075+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:35:59.698075+0000","last_peered":"2024-02-18T00:35:59.698075+0000","last_clean":"2024-02-18T00:35:59.698075+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:35:59.698075+0000","last_undegraded":"2024-02-18T00:35:59.698075+0000","last_fullsized":"2024-02-18T00:35:59.698075+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":249,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643579+0000","last_change":"2024-02-18T00:33:46.861306+0000","last_active":"2024-02-18T00:35:59.643579+0000","last_peered":"2024-02-18T00:35:59.643579+0000","last_clean":"2024-02-18T00:35:59.643579+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:35:59.643579+0000","last_undegraded":"2024-02-18T00:35:59.643579+0000","last_fullsized":"2024-02-18T00:35:59.643579+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.861233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.861233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643517+0000","last_change":"2024-02-18T00:33:53.005012+0000","last_active":"2024-02-18T00:35:59.643517+0000","last_peered":"2024-02-18T00:35:59.643517+0000","last_clean":"2024-02-18T00:35:59.643517+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:35:59.643517+0000","last_undegraded":"2024-02-18T00:35:59.643517+0000","last_fullsized":"2024-02-18T00:35:59.643517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.004903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.004903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641652+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:35:59.641652+0000","last_peered":"2024-02-18T00:35:59.641652+0000","last_clean":"2024-02-18T00:35:59.641652+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:35:59.641652+0000","last_undegraded":"2024-02-18T00:35:59.641652+0000","last_fullsized":"2024-02-18T00:35:59.641652+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695569+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:35:59.695569+0000","last_peered":"2024-02-18T00:35:59.695569+0000","last_clean":"2024-02-18T00:35:59.695569+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:35:59.695569+0000","last_undegraded":"2024-02-18T00:35:59.695569+0000","last_fullsized":"2024-02-18T00:35:59.695569+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":279,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652810+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:35:58.652810+0000","last_peered":"2024-02-18T00:35:58.652810+0000","last_clean":"2024-02-18T00:35:58.652810+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:35:58.652810+0000","last_undegraded":"2024-02-18T00:35:58.652810+0000","last_fullsized":"2024-02-18T00:35:58.652810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":148,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.653312+0000","last_change":"2024-02-18T00:34:18.301213+0000","last_active":"2024-02-18T00:35:58.653312+0000","last_peered":"2024-02-18T00:35:58.653312+0000","last_clean":"2024-02-18T00:35:58.653312+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:35:58.653312+0000","last_undegraded":"2024-02-18T00:35:58.653312+0000","last_fullsized":"2024-02-18T00:35:58.653312+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.301136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.301136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":244,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.658596+0000","last_change":"2024-02-18T00:34:39.453471+0000","last_active":"2024-02-18T00:35:58.658596+0000","last_peered":"2024-02-18T00:35:58.658596+0000","last_clean":"2024-02-18T00:35:58.658596+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:35:58.658596+0000","last_undegraded":"2024-02-18T00:35:58.658596+0000","last_fullsized":"2024-02-18T00:35:58.658596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.453312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.453312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.642520+0000","last_change":"2024-02-18T00:33:45.813625+0000","last_active":"2024-02-18T00:35:59.642520+0000","last_peered":"2024-02-18T00:35:59.642520+0000","last_clean":"2024-02-18T00:35:59.642520+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:35:59.642520+0000","last_undegraded":"2024-02-18T00:35:59.642520+0000","last_fullsized":"2024-02-18T00:35:59.642520+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.813514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.813514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":33,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641174+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:35:59.641174+0000","last_peered":"2024-02-18T00:35:59.641174+0000","last_clean":"2024-02-18T00:35:59.641174+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:35:59.641174+0000","last_undegraded":"2024-02-18T00:35:59.641174+0000","last_fullsized":"2024-02-18T00:35:59.641174+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":261,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.651625+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:35:58.651625+0000","last_peered":"2024-02-18T00:35:58.651625+0000","last_clean":"2024-02-18T00:35:58.651625+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:35:58.651625+0000","last_undegraded":"2024-02-18T00:35:58.651625+0000","last_fullsized":"2024-02-18T00:35:58.651625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":257,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.659426+0000","last_change":"2024-02-18T00:34:23.295161+0000","last_active":"2024-02-18T00:35:58.659426+0000","last_peered":"2024-02-18T00:35:58.659426+0000","last_clean":"2024-02-18T00:35:58.659426+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:35:58.659426+0000","last_undegraded":"2024-02-18T00:35:58.659426+0000","last_fullsized":"2024-02-18T00:35:58.659426+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.295051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":350,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643125+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:35:59.643125+0000","last_peered":"2024-02-18T00:35:59.643125+0000","last_clean":"2024-02-18T00:35:59.643125+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:35:59.643125+0000","last_undegraded":"2024-02-18T00:35:59.643125+0000","last_fullsized":"2024-02-18T00:35:59.643125+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":242,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.657382+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:35:58.657382+0000","last_peered":"2024-02-18T00:35:58.657382+0000","last_clean":"2024-02-18T00:35:58.657382+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:35:58.657382+0000","last_undegraded":"2024-02-18T00:35:58.657382+0000","last_fullsized":"2024-02-18T00:35:58.657382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":188,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.659737+0000","last_change":"2024-02-18T00:35:29.520701+0000","last_active":"2024-02-18T00:35:58.659737+0000","last_peered":"2024-02-18T00:35:58.659737+0000","last_clean":"2024-02-18T00:35:58.659737+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:35:58.659737+0000","last_undegraded":"2024-02-18T00:35:58.659737+0000","last_fullsized":"2024-02-18T00:35:58.659737+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.520632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.520632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":250,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698194+0000","last_change":"2024-02-18T00:33:44.007884+0000","last_active":"2024-02-18T00:35:59.698194+0000","last_peered":"2024-02-18T00:35:59.698194+0000","last_clean":"2024-02-18T00:35:59.698194+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:35:59.698194+0000","last_undegraded":"2024-02-18T00:35:59.698194+0000","last_fullsized":"2024-02-18T00:35:59.698194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:33:44.007804+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:33:44.007804+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":266,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.664361+0000","last_change":"2024-02-18T00:33:48.713658+0000","last_active":"2024-02-18T00:35:58.664361+0000","last_peered":"2024-02-18T00:35:58.664361+0000","last_clean":"2024-02-18T00:35:58.664361+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:35:58.664361+0000","last_undegraded":"2024-02-18T00:35:58.664361+0000","last_fullsized":"2024-02-18T00:35:58.664361+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:33:47.199009+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":333,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.657293+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:35:58.657293+0000","last_peered":"2024-02-18T00:35:58.657293+0000","last_clean":"2024-02-18T00:35:58.657293+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:35:58.657293+0000","last_undegraded":"2024-02-18T00:35:58.657293+0000","last_fullsized":"2024-02-18T00:35:58.657293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":238,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698587+0000","last_change":"2024-02-18T00:33:41.057722+0000","last_active":"2024-02-18T00:35:59.698587+0000","last_peered":"2024-02-18T00:35:59.698587+0000","last_clean":"2024-02-18T00:35:59.698587+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:35:59.698587+0000","last_undegraded":"2024-02-18T00:35:59.698587+0000","last_fullsized":"2024-02-18T00:35:59.698587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.057659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:23.987791+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.057659+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":188,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:36:00.408919+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:36:00.408919+0000","last_peered":"2024-02-18T00:36:00.408919+0000","last_clean":"2024-02-18T00:36:00.408919+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:36:00.408919+0000","last_undegraded":"2024-02-18T00:36:00.408919+0000","last_fullsized":"2024-02-18T00:36:00.408919+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":303,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.656464+0000","last_change":"2024-02-18T00:35:43.219869+0000","last_active":"2024-02-18T00:35:58.656464+0000","last_peered":"2024-02-18T00:35:58.656464+0000","last_clean":"2024-02-18T00:35:58.656464+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:35:58.656464+0000","last_undegraded":"2024-02-18T00:35:58.656464+0000","last_fullsized":"2024-02-18T00:35:58.656464+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:45.311259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.311259+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":220,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.694735+0000","last_change":"2024-02-18T00:34:27.927933+0000","last_active":"2024-02-18T00:35:59.694735+0000","last_peered":"2024-02-18T00:35:59.694735+0000","last_clean":"2024-02-18T00:35:59.694735+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:35:59.694735+0000","last_undegraded":"2024-02-18T00:35:59.694735+0000","last_fullsized":"2024-02-18T00:35:59.694735+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.927871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.927871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698318+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:35:59.698318+0000","last_peered":"2024-02-18T00:35:59.698318+0000","last_clean":"2024-02-18T00:35:59.698318+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:35:59.698318+0000","last_undegraded":"2024-02-18T00:35:59.698318+0000","last_fullsized":"2024-02-18T00:35:59.698318+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697566+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:35:59.697566+0000","last_peered":"2024-02-18T00:35:59.697566+0000","last_clean":"2024-02-18T00:35:59.697566+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:35:59.697566+0000","last_undegraded":"2024-02-18T00:35:59.697566+0000","last_fullsized":"2024-02-18T00:35:59.697566+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":188,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652261+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:35:58.652261+0000","last_peered":"2024-02-18T00:35:58.652261+0000","last_clean":"2024-02-18T00:35:58.652261+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:35:58.652261+0000","last_undegraded":"2024-02-18T00:35:58.652261+0000","last_fullsized":"2024-02-18T00:35:58.652261+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":335,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696191+0000","last_change":"2024-02-18T00:33:54.753052+0000","last_active":"2024-02-18T00:35:59.696191+0000","last_peered":"2024-02-18T00:35:59.696191+0000","last_clean":"2024-02-18T00:35:59.696191+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:35:59.696191+0000","last_undegraded":"2024-02-18T00:35:59.696191+0000","last_fullsized":"2024-02-18T00:35:59.696191+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:33:54.752985+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.752985+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":282,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.651438+0000","last_change":"2024-02-18T00:33:53.320429+0000","last_active":"2024-02-18T00:35:58.651438+0000","last_peered":"2024-02-18T00:35:58.651438+0000","last_clean":"2024-02-18T00:35:58.651438+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:35:58.651438+0000","last_undegraded":"2024-02-18T00:35:58.651438+0000","last_fullsized":"2024-02-18T00:35:58.651438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:33:53.320356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.320356+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":267,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697220+0000","last_change":"2024-02-18T00:33:48.672342+0000","last_active":"2024-02-18T00:35:59.697220+0000","last_peered":"2024-02-18T00:35:59.697220+0000","last_clean":"2024-02-18T00:35:59.697220+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:35:59.697220+0000","last_undegraded":"2024-02-18T00:35:59.697220+0000","last_fullsized":"2024-02-18T00:35:59.697220+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.368010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.368010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696088+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:35:59.696088+0000","last_peered":"2024-02-18T00:35:59.696088+0000","last_clean":"2024-02-18T00:35:59.696088+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:35:59.696088+0000","last_undegraded":"2024-02-18T00:35:59.696088+0000","last_fullsized":"2024-02-18T00:35:59.696088+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698416+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:35:59.698416+0000","last_peered":"2024-02-18T00:35:59.698416+0000","last_clean":"2024-02-18T00:35:59.698416+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:35:59.698416+0000","last_undegraded":"2024-02-18T00:35:59.698416+0000","last_fullsized":"2024-02-18T00:35:59.698416+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":253,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652024+0000","last_change":"2024-02-18T00:33:50.287103+0000","last_active":"2024-02-18T00:35:58.652024+0000","last_peered":"2024-02-18T00:35:58.652024+0000","last_clean":"2024-02-18T00:35:58.652024+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:35:58.652024+0000","last_undegraded":"2024-02-18T00:35:58.652024+0000","last_fullsized":"2024-02-18T00:35:58.652024+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.287007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.287007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":350,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696094+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:35:59.696094+0000","last_peered":"2024-02-18T00:35:59.696094+0000","last_clean":"2024-02-18T00:35:59.696094+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:35:59.696094+0000","last_undegraded":"2024-02-18T00:35:59.696094+0000","last_fullsized":"2024-02-18T00:35:59.696094+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":257,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.671105+0000","last_change":"2024-02-18T00:33:51.202714+0000","last_active":"2024-02-18T00:35:58.671105+0000","last_peered":"2024-02-18T00:35:58.671105+0000","last_clean":"2024-02-18T00:35:58.671105+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:35:58.671105+0000","last_undegraded":"2024-02-18T00:35:58.671105+0000","last_fullsized":"2024-02-18T00:35:58.671105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.202648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.202648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":242,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.653370+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:35:58.653370+0000","last_peered":"2024-02-18T00:35:58.653370+0000","last_clean":"2024-02-18T00:35:58.653370+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:35:58.653370+0000","last_undegraded":"2024-02-18T00:35:58.653370+0000","last_fullsized":"2024-02-18T00:35:58.653370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697899+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:35:59.697899+0000","last_peered":"2024-02-18T00:35:59.697899+0000","last_clean":"2024-02-18T00:35:59.697899+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:35:59.697899+0000","last_undegraded":"2024-02-18T00:35:59.697899+0000","last_fullsized":"2024-02-18T00:35:59.697899+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697843+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:35:59.697843+0000","last_peered":"2024-02-18T00:35:59.697843+0000","last_clean":"2024-02-18T00:35:59.697843+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:35:59.697843+0000","last_undegraded":"2024-02-18T00:35:59.697843+0000","last_fullsized":"2024-02-18T00:35:59.697843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":326,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696040+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:35:59.696040+0000","last_peered":"2024-02-18T00:35:59.696040+0000","last_clean":"2024-02-18T00:35:59.696040+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:35:59.696040+0000","last_undegraded":"2024-02-18T00:35:59.696040+0000","last_fullsized":"2024-02-18T00:35:59.696040+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":157,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696501+0000","last_change":"2024-02-18T00:34:32.021327+0000","last_active":"2024-02-18T00:35:59.696501+0000","last_peered":"2024-02-18T00:35:59.696501+0000","last_clean":"2024-02-18T00:35:59.696501+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:35:59.696501+0000","last_undegraded":"2024-02-18T00:35:59.696501+0000","last_fullsized":"2024-02-18T00:35:59.696501+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:34:32.021219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.021219+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":242,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.664451+0000","last_change":"2024-02-18T00:35:13.452285+0000","last_active":"2024-02-18T00:35:58.664451+0000","last_peered":"2024-02-18T00:35:58.664451+0000","last_clean":"2024-02-18T00:35:58.664451+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:35:58.664451+0000","last_undegraded":"2024-02-18T00:35:58.664451+0000","last_fullsized":"2024-02-18T00:35:58.664451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.452220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.452220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":184,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698345+0000","last_change":"2024-02-18T00:33:04.587776+0000","last_active":"2024-02-18T00:35:59.698345+0000","last_peered":"2024-02-18T00:35:59.698345+0000","last_clean":"2024-02-18T00:35:59.698345+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:35:59.698345+0000","last_undegraded":"2024-02-18T00:35:59.698345+0000","last_fullsized":"2024-02-18T00:35:59.698345+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698386+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:35:59.698386+0000","last_peered":"2024-02-18T00:35:59.698386+0000","last_clean":"2024-02-18T00:35:59.698386+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:35:59.698386+0000","last_undegraded":"2024-02-18T00:35:59.698386+0000","last_fullsized":"2024-02-18T00:35:59.698386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":264,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697233+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:35:59.697233+0000","last_peered":"2024-02-18T00:35:59.697233+0000","last_clean":"2024-02-18T00:35:59.697233+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:35:59.697233+0000","last_undegraded":"2024-02-18T00:35:59.697233+0000","last_fullsized":"2024-02-18T00:35:59.697233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":219,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.669273+0000","last_change":"2024-02-18T00:34:26.340185+0000","last_active":"2024-02-18T00:35:58.669273+0000","last_peered":"2024-02-18T00:35:58.669273+0000","last_clean":"2024-02-18T00:35:58.669273+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:35:58.669273+0000","last_undegraded":"2024-02-18T00:35:58.669273+0000","last_fullsized":"2024-02-18T00:35:58.669273+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.340120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.340120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":247,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.694876+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:35:59.694876+0000","last_peered":"2024-02-18T00:35:59.694876+0000","last_clean":"2024-02-18T00:35:59.694876+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:35:59.694876+0000","last_undegraded":"2024-02-18T00:35:59.694876+0000","last_fullsized":"2024-02-18T00:35:59.694876+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":242,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.658684+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:35:58.658684+0000","last_peered":"2024-02-18T00:35:58.658684+0000","last_clean":"2024-02-18T00:35:58.658684+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:35:58.658684+0000","last_undegraded":"2024-02-18T00:35:58.658684+0000","last_fullsized":"2024-02-18T00:35:58.658684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":198,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.694841+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:35:59.694841+0000","last_peered":"2024-02-18T00:35:59.694841+0000","last_clean":"2024-02-18T00:35:59.694841+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:35:59.694841+0000","last_undegraded":"2024-02-18T00:35:59.694841+0000","last_fullsized":"2024-02-18T00:35:59.694841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643264+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:35:59.643264+0000","last_peered":"2024-02-18T00:35:59.643264+0000","last_clean":"2024-02-18T00:35:59.643264+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:35:59.643264+0000","last_undegraded":"2024-02-18T00:35:59.643264+0000","last_fullsized":"2024-02-18T00:35:59.643264+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":299,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697110+0000","last_change":"2024-02-18T00:34:28.290944+0000","last_active":"2024-02-18T00:35:59.697110+0000","last_peered":"2024-02-18T00:35:59.697110+0000","last_clean":"2024-02-18T00:35:59.697110+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:35:59.697110+0000","last_undegraded":"2024-02-18T00:35:59.697110+0000","last_fullsized":"2024-02-18T00:35:59.697110+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:34:28.290868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.290868+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":183,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.694789+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:35:59.694789+0000","last_peered":"2024-02-18T00:35:59.694789+0000","last_clean":"2024-02-18T00:35:59.694789+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:35:59.694789+0000","last_undegraded":"2024-02-18T00:35:59.694789+0000","last_fullsized":"2024-02-18T00:35:59.694789+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":267,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695885+0000","last_change":"2024-02-18T00:34:35.765124+0000","last_active":"2024-02-18T00:35:59.695885+0000","last_peered":"2024-02-18T00:35:59.695885+0000","last_clean":"2024-02-18T00:35:59.695885+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:35:59.695885+0000","last_undegraded":"2024-02-18T00:35:59.695885+0000","last_fullsized":"2024-02-18T00:35:59.695885+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.765059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.765059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":248,"reported_epoch":282,"state":"active+clean","last_fresh":"2024-02-18T00:36:00.758971+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:36:00.758971+0000","last_peered":"2024-02-18T00:36:00.758971+0000","last_clean":"2024-02-18T00:36:00.758971+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:36:00.758971+0000","last_undegraded":"2024-02-18T00:36:00.758971+0000","last_fullsized":"2024-02-18T00:36:00.758971+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698483+0000","last_change":"2024-02-18T00:34:17.016556+0000","last_active":"2024-02-18T00:35:59.698483+0000","last_peered":"2024-02-18T00:35:59.698483+0000","last_clean":"2024-02-18T00:35:59.698483+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:35:59.698483+0000","last_undegraded":"2024-02-18T00:35:59.698483+0000","last_fullsized":"2024-02-18T00:35:59.698483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.016476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.016476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696218+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:35:59.696218+0000","last_peered":"2024-02-18T00:35:59.696218+0000","last_clean":"2024-02-18T00:35:59.696218+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:35:59.696218+0000","last_undegraded":"2024-02-18T00:35:59.696218+0000","last_fullsized":"2024-02-18T00:35:59.696218+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":254,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.644781+0000","last_change":"2024-02-18T00:33:49.897870+0000","last_active":"2024-02-18T00:35:59.644781+0000","last_peered":"2024-02-18T00:35:59.644781+0000","last_clean":"2024-02-18T00:35:59.644781+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:35:59.644781+0000","last_undegraded":"2024-02-18T00:35:59.644781+0000","last_fullsized":"2024-02-18T00:35:59.644781+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.897812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.897812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":288,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.671488+0000","last_change":"2024-02-18T00:34:25.385634+0000","last_active":"2024-02-18T00:35:58.671488+0000","last_peered":"2024-02-18T00:35:58.671488+0000","last_clean":"2024-02-18T00:35:58.671488+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:35:58.671488+0000","last_undegraded":"2024-02-18T00:35:58.671488+0000","last_fullsized":"2024-02-18T00:35:58.671488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:34:25.385564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.385564+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":258,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641937+0000","last_change":"2024-02-18T00:33:37.809035+0000","last_active":"2024-02-18T00:35:59.641937+0000","last_peered":"2024-02-18T00:35:59.641937+0000","last_clean":"2024-02-18T00:35:59.641937+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:35:59.641937+0000","last_undegraded":"2024-02-18T00:35:59.641937+0000","last_fullsized":"2024-02-18T00:35:59.641937+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:37.808923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.808923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":648,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.646365+0000","last_change":"2024-02-18T00:33:41.154008+0000","last_active":"2024-02-18T00:35:59.646365+0000","last_peered":"2024-02-18T00:35:59.646365+0000","last_clean":"2024-02-18T00:35:59.646365+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:35:59.646365+0000","last_undegraded":"2024-02-18T00:35:59.646365+0000","last_fullsized":"2024-02-18T00:35:59.646365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:33:41.153868+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.153868+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":397,"num_read_kb":392,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":186,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697409+0000","last_change":"2024-02-18T00:34:29.252896+0000","last_active":"2024-02-18T00:35:59.697409+0000","last_peered":"2024-02-18T00:35:59.697409+0000","last_clean":"2024-02-18T00:35:59.697409+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:35:59.697409+0000","last_undegraded":"2024-02-18T00:35:59.697409+0000","last_fullsized":"2024-02-18T00:35:59.697409+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:34:29.252830+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.252830+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":192,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652106+0000","last_change":"2024-02-18T00:33:48.670413+0000","last_active":"2024-02-18T00:35:58.652106+0000","last_peered":"2024-02-18T00:35:58.652106+0000","last_clean":"2024-02-18T00:35:58.652106+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:35:58.652106+0000","last_undegraded":"2024-02-18T00:35:58.652106+0000","last_fullsized":"2024-02-18T00:35:58.652106+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":244,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696588+0000","last_change":"2024-02-18T00:34:19.925200+0000","last_active":"2024-02-18T00:35:59.696588+0000","last_peered":"2024-02-18T00:35:59.696588+0000","last_clean":"2024-02-18T00:35:59.696588+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:35:59.696588+0000","last_undegraded":"2024-02-18T00:35:59.696588+0000","last_fullsized":"2024-02-18T00:35:59.696588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:19.925111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.925111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":56,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697024+0000","last_change":"2024-02-18T00:35:44.479802+0000","last_active":"2024-02-18T00:35:59.697024+0000","last_peered":"2024-02-18T00:35:59.697024+0000","last_clean":"2024-02-18T00:35:59.697024+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:35:59.697024+0000","last_undegraded":"2024-02-18T00:35:59.697024+0000","last_fullsized":"2024-02-18T00:35:59.697024+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:33:54.197976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.197976+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":258,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696171+0000","last_change":"2024-02-18T00:34:15.045746+0000","last_active":"2024-02-18T00:35:59.696171+0000","last_peered":"2024-02-18T00:35:59.696171+0000","last_clean":"2024-02-18T00:35:59.696171+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:35:59.696171+0000","last_undegraded":"2024-02-18T00:35:59.696171+0000","last_fullsized":"2024-02-18T00:35:59.696171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.045677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.045677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":247,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697462+0000","last_change":"2024-02-18T00:33:48.688410+0000","last_active":"2024-02-18T00:35:59.697462+0000","last_peered":"2024-02-18T00:35:59.697462+0000","last_clean":"2024-02-18T00:35:59.697462+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:35:59.697462+0000","last_undegraded":"2024-02-18T00:35:59.697462+0000","last_fullsized":"2024-02-18T00:35:59.697462+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:43.926301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.926301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696761+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:35:59.696761+0000","last_peered":"2024-02-18T00:35:59.696761+0000","last_clean":"2024-02-18T00:35:59.696761+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:35:59.696761+0000","last_undegraded":"2024-02-18T00:35:59.696761+0000","last_fullsized":"2024-02-18T00:35:59.696761+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":333,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696480+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:35:59.696480+0000","last_peered":"2024-02-18T00:35:59.696480+0000","last_clean":"2024-02-18T00:35:59.696480+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:35:59.696480+0000","last_undegraded":"2024-02-18T00:35:59.696480+0000","last_fullsized":"2024-02-18T00:35:59.696480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":247,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641713+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:35:59.641713+0000","last_peered":"2024-02-18T00:35:59.641713+0000","last_clean":"2024-02-18T00:35:59.641713+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:35:59.641713+0000","last_undegraded":"2024-02-18T00:35:59.641713+0000","last_fullsized":"2024-02-18T00:35:59.641713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":271,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.658442+0000","last_change":"2024-02-18T00:34:31.410145+0000","last_active":"2024-02-18T00:35:58.658442+0000","last_peered":"2024-02-18T00:35:58.658442+0000","last_clean":"2024-02-18T00:35:58.658442+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:35:58.658442+0000","last_undegraded":"2024-02-18T00:35:58.658442+0000","last_fullsized":"2024-02-18T00:35:58.658442+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.410081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.410081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":237,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.694628+0000","last_change":"2024-02-18T00:33:48.010937+0000","last_active":"2024-02-18T00:35:59.694628+0000","last_peered":"2024-02-18T00:35:59.694628+0000","last_clean":"2024-02-18T00:35:59.694628+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:35:59.694628+0000","last_undegraded":"2024-02-18T00:35:59.694628+0000","last_fullsized":"2024-02-18T00:35:59.694628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:48.010821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:48.010821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":183,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652185+0000","last_change":"2024-02-18T00:33:02.483549+0000","last_active":"2024-02-18T00:35:58.652185+0000","last_peered":"2024-02-18T00:35:58.652185+0000","last_clean":"2024-02-18T00:35:58.652185+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:35:58.652185+0000","last_undegraded":"2024-02-18T00:35:58.652185+0000","last_fullsized":"2024-02-18T00:35:58.652185+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641500+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:35:59.641500+0000","last_peered":"2024-02-18T00:35:59.641500+0000","last_clean":"2024-02-18T00:35:59.641500+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:35:59.641500+0000","last_undegraded":"2024-02-18T00:35:59.641500+0000","last_fullsized":"2024-02-18T00:35:59.641500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":299,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.651485+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:35:58.651485+0000","last_peered":"2024-02-18T00:35:58.651485+0000","last_clean":"2024-02-18T00:35:58.651485+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:35:58.651485+0000","last_undegraded":"2024-02-18T00:35:58.651485+0000","last_fullsized":"2024-02-18T00:35:58.651485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":281,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.658952+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:35:58.658952+0000","last_peered":"2024-02-18T00:35:58.658952+0000","last_clean":"2024-02-18T00:35:58.658952+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:35:58.658952+0000","last_undegraded":"2024-02-18T00:35:58.658952+0000","last_fullsized":"2024-02-18T00:35:58.658952+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":252,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698227+0000","last_change":"2024-02-18T00:34:26.701231+0000","last_active":"2024-02-18T00:35:59.698227+0000","last_peered":"2024-02-18T00:35:59.698227+0000","last_clean":"2024-02-18T00:35:59.698227+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:35:59.698227+0000","last_undegraded":"2024-02-18T00:35:59.698227+0000","last_fullsized":"2024-02-18T00:35:59.698227+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.701162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.701162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":188,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.653056+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:35:58.653056+0000","last_peered":"2024-02-18T00:35:58.653056+0000","last_clean":"2024-02-18T00:35:58.653056+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:35:58.653056+0000","last_undegraded":"2024-02-18T00:35:58.653056+0000","last_fullsized":"2024-02-18T00:35:58.653056+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":315,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697305+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:35:59.697305+0000","last_peered":"2024-02-18T00:35:59.697305+0000","last_clean":"2024-02-18T00:35:59.697305+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:35:59.697305+0000","last_undegraded":"2024-02-18T00:35:59.697305+0000","last_fullsized":"2024-02-18T00:35:59.697305+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":249,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641410+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:35:59.641410+0000","last_peered":"2024-02-18T00:35:59.641410+0000","last_clean":"2024-02-18T00:35:59.641410+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:35:59.641410+0000","last_undegraded":"2024-02-18T00:35:59.641410+0000","last_fullsized":"2024-02-18T00:35:59.641410+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":254,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697421+0000","last_change":"2024-02-18T00:34:25.683506+0000","last_active":"2024-02-18T00:35:59.697421+0000","last_peered":"2024-02-18T00:35:59.697421+0000","last_clean":"2024-02-18T00:35:59.697421+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:35:59.697421+0000","last_undegraded":"2024-02-18T00:35:59.697421+0000","last_fullsized":"2024-02-18T00:35:59.697421+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.683442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.683442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":252,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643185+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:35:59.643185+0000","last_peered":"2024-02-18T00:35:59.643185+0000","last_clean":"2024-02-18T00:35:59.643185+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:35:59.643185+0000","last_undegraded":"2024-02-18T00:35:59.643185+0000","last_fullsized":"2024-02-18T00:35:59.643185+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":184,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695969+0000","last_change":"2024-02-18T00:33:04.706011+0000","last_active":"2024-02-18T00:35:59.695969+0000","last_peered":"2024-02-18T00:35:59.695969+0000","last_clean":"2024-02-18T00:35:59.695969+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:35:59.695969+0000","last_undegraded":"2024-02-18T00:35:59.695969+0000","last_fullsized":"2024-02-18T00:35:59.695969+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":309,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697274+0000","last_change":"2024-02-18T00:33:48.861148+0000","last_active":"2024-02-18T00:35:59.697274+0000","last_peered":"2024-02-18T00:35:59.697274+0000","last_clean":"2024-02-18T00:35:59.697274+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:35:59.697274+0000","last_undegraded":"2024-02-18T00:35:59.697274+0000","last_fullsized":"2024-02-18T00:35:59.697274+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:33:46.897445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.897445+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":254,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695978+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:35:59.695978+0000","last_peered":"2024-02-18T00:35:59.695978+0000","last_clean":"2024-02-18T00:35:59.695978+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:35:59.695978+0000","last_undegraded":"2024-02-18T00:35:59.695978+0000","last_fullsized":"2024-02-18T00:35:59.695978+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698058+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:35:59.698058+0000","last_peered":"2024-02-18T00:35:59.698058+0000","last_clean":"2024-02-18T00:35:59.698058+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:35:59.698058+0000","last_undegraded":"2024-02-18T00:35:59.698058+0000","last_fullsized":"2024-02-18T00:35:59.698058+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697517+0000","last_change":"2024-02-18T00:33:55.265441+0000","last_active":"2024-02-18T00:35:59.697517+0000","last_peered":"2024-02-18T00:35:59.697517+0000","last_clean":"2024-02-18T00:35:59.697517+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:35:59.697517+0000","last_undegraded":"2024-02-18T00:35:59.697517+0000","last_fullsized":"2024-02-18T00:35:59.697517+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.265373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.265373+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":198,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.704886+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:35:59.704886+0000","last_peered":"2024-02-18T00:35:59.704886+0000","last_clean":"2024-02-18T00:35:59.704886+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:35:59.704886+0000","last_undegraded":"2024-02-18T00:35:59.704886+0000","last_fullsized":"2024-02-18T00:35:59.704886+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":317,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697851+0000","last_change":"2024-02-18T00:34:30.666844+0000","last_active":"2024-02-18T00:35:59.697851+0000","last_peered":"2024-02-18T00:35:59.697851+0000","last_clean":"2024-02-18T00:35:59.697851+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:35:59.697851+0000","last_undegraded":"2024-02-18T00:35:59.697851+0000","last_fullsized":"2024-02-18T00:35:59.697851+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:34:30.666765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.666765+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":267,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696088+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:35:59.696088+0000","last_peered":"2024-02-18T00:35:59.696088+0000","last_clean":"2024-02-18T00:35:59.696088+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:35:59.696088+0000","last_undegraded":"2024-02-18T00:35:59.696088+0000","last_fullsized":"2024-02-18T00:35:59.696088+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":334,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698529+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:35:59.698529+0000","last_peered":"2024-02-18T00:35:59.698529+0000","last_clean":"2024-02-18T00:35:59.698529+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:35:59.698529+0000","last_undegraded":"2024-02-18T00:35:59.698529+0000","last_fullsized":"2024-02-18T00:35:59.698529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":251,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697187+0000","last_change":"2024-02-18T00:33:53.101420+0000","last_active":"2024-02-18T00:35:59.697187+0000","last_peered":"2024-02-18T00:35:59.697187+0000","last_clean":"2024-02-18T00:35:59.697187+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:35:59.697187+0000","last_undegraded":"2024-02-18T00:35:59.697187+0000","last_fullsized":"2024-02-18T00:35:59.697187+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.101251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.101251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":198,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695143+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:35:59.695143+0000","last_peered":"2024-02-18T00:35:59.695143+0000","last_clean":"2024-02-18T00:35:59.695143+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:35:59.695143+0000","last_undegraded":"2024-02-18T00:35:59.695143+0000","last_fullsized":"2024-02-18T00:35:59.695143+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":294,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698453+0000","last_change":"2024-02-18T00:33:49.276437+0000","last_active":"2024-02-18T00:35:59.698453+0000","last_peered":"2024-02-18T00:35:59.698453+0000","last_clean":"2024-02-18T00:35:59.698453+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:35:59.698453+0000","last_undegraded":"2024-02-18T00:35:59.698453+0000","last_fullsized":"2024-02-18T00:35:59.698453+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:33:40.095425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.095425+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":248,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696127+0000","last_change":"2024-02-18T00:34:15.935045+0000","last_active":"2024-02-18T00:35:59.696127+0000","last_peered":"2024-02-18T00:35:59.696127+0000","last_clean":"2024-02-18T00:35:59.696127+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:35:59.696127+0000","last_undegraded":"2024-02-18T00:35:59.696127+0000","last_fullsized":"2024-02-18T00:35:59.696127+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.934970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"150'9","reported_seq":347,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698066+0000","last_change":"2024-02-18T00:34:27.741951+0000","last_active":"2024-02-18T00:35:59.698066+0000","last_peered":"2024-02-18T00:35:59.698066+0000","last_clean":"2024-02-18T00:35:59.698066+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:35:59.698066+0000","last_undegraded":"2024-02-18T00:35:59.698066+0000","last_fullsized":"2024-02-18T00:35:59.698066+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:27.741881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.741881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":242,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.413643+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:35:59.413643+0000","last_peered":"2024-02-18T00:35:59.413643+0000","last_clean":"2024-02-18T00:35:59.413643+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:35:59.413643+0000","last_undegraded":"2024-02-18T00:35:59.413643+0000","last_fullsized":"2024-02-18T00:35:59.413643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698545+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:35:59.698545+0000","last_peered":"2024-02-18T00:35:59.698545+0000","last_clean":"2024-02-18T00:35:59.698545+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:35:59.698545+0000","last_undegraded":"2024-02-18T00:35:59.698545+0000","last_fullsized":"2024-02-18T00:35:59.698545+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":262,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.659520+0000","last_change":"2024-02-18T00:34:32.418786+0000","last_active":"2024-02-18T00:35:58.659520+0000","last_peered":"2024-02-18T00:35:58.659520+0000","last_clean":"2024-02-18T00:35:58.659520+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:35:58.659520+0000","last_undegraded":"2024-02-18T00:35:58.659520+0000","last_fullsized":"2024-02-18T00:35:58.659520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:34:32.418717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.418717+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":254,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696547+0000","last_change":"2024-02-18T00:34:20.312165+0000","last_active":"2024-02-18T00:35:59.696547+0000","last_peered":"2024-02-18T00:35:59.696547+0000","last_clean":"2024-02-18T00:35:59.696547+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:35:59.696547+0000","last_undegraded":"2024-02-18T00:35:59.696547+0000","last_fullsized":"2024-02-18T00:35:59.696547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.312096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.312096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":245,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698192+0000","last_change":"2024-02-18T00:33:50.765435+0000","last_active":"2024-02-18T00:35:59.698192+0000","last_peered":"2024-02-18T00:35:59.698192+0000","last_clean":"2024-02-18T00:35:59.698192+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:35:59.698192+0000","last_undegraded":"2024-02-18T00:35:59.698192+0000","last_fullsized":"2024-02-18T00:35:59.698192+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.765368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.765368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":237,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.694628+0000","last_change":"2024-02-18T00:33:01.075249+0000","last_active":"2024-02-18T00:35:59.694628+0000","last_peered":"2024-02-18T00:35:59.694628+0000","last_clean":"2024-02-18T00:35:59.694628+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:35:59.694628+0000","last_undegraded":"2024-02-18T00:35:59.694628+0000","last_fullsized":"2024-02-18T00:35:59.694628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.075169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.075169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":184,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697128+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:35:59.697128+0000","last_peered":"2024-02-18T00:35:59.697128+0000","last_clean":"2024-02-18T00:35:59.697128+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:35:59.697128+0000","last_undegraded":"2024-02-18T00:35:59.697128+0000","last_fullsized":"2024-02-18T00:35:59.697128+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":265,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.653412+0000","last_change":"2024-02-18T00:34:20.362851+0000","last_active":"2024-02-18T00:35:58.653412+0000","last_peered":"2024-02-18T00:35:58.653412+0000","last_clean":"2024-02-18T00:35:58.653412+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:35:58.653412+0000","last_undegraded":"2024-02-18T00:35:58.653412+0000","last_fullsized":"2024-02-18T00:35:58.653412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:34:20.362782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.362782+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":248,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.670800+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:35:58.670800+0000","last_peered":"2024-02-18T00:35:58.670800+0000","last_clean":"2024-02-18T00:35:58.670800+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:35:58.670800+0000","last_undegraded":"2024-02-18T00:35:58.670800+0000","last_fullsized":"2024-02-18T00:35:58.670800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":244,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.659602+0000","last_change":"2024-02-18T00:34:34.355483+0000","last_active":"2024-02-18T00:35:58.659602+0000","last_peered":"2024-02-18T00:35:58.659602+0000","last_clean":"2024-02-18T00:35:58.659602+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:35:58.659602+0000","last_undegraded":"2024-02-18T00:35:58.659602+0000","last_fullsized":"2024-02-18T00:35:58.659602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.355416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.355416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695404+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:35:59.695404+0000","last_peered":"2024-02-18T00:35:59.695404+0000","last_clean":"2024-02-18T00:35:59.695404+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:35:59.695404+0000","last_undegraded":"2024-02-18T00:35:59.695404+0000","last_fullsized":"2024-02-18T00:35:59.695404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697297+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:35:59.697297+0000","last_peered":"2024-02-18T00:35:59.697297+0000","last_clean":"2024-02-18T00:35:59.697297+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:35:59.697297+0000","last_undegraded":"2024-02-18T00:35:59.697297+0000","last_fullsized":"2024-02-18T00:35:59.697297+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":253,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695943+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:35:59.695943+0000","last_peered":"2024-02-18T00:35:59.695943+0000","last_clean":"2024-02-18T00:35:59.695943+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:35:59.695943+0000","last_undegraded":"2024-02-18T00:35:59.695943+0000","last_fullsized":"2024-02-18T00:35:59.695943+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":249,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698132+0000","last_change":"2024-02-18T00:33:39.051726+0000","last_active":"2024-02-18T00:35:59.698132+0000","last_peered":"2024-02-18T00:35:59.698132+0000","last_clean":"2024-02-18T00:35:59.698132+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:35:59.698132+0000","last_undegraded":"2024-02-18T00:35:59.698132+0000","last_fullsized":"2024-02-18T00:35:59.698132+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.051653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.051653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":254,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695891+0000","last_change":"2024-02-18T00:33:56.296342+0000","last_active":"2024-02-18T00:35:59.695891+0000","last_peered":"2024-02-18T00:35:59.695891+0000","last_clean":"2024-02-18T00:35:59.695891+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:35:59.695891+0000","last_undegraded":"2024-02-18T00:35:59.695891+0000","last_fullsized":"2024-02-18T00:35:59.695891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:56.296274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.296274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.641711+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:35:59.641711+0000","last_peered":"2024-02-18T00:35:59.641711+0000","last_clean":"2024-02-18T00:35:59.641711+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:35:59.641711+0000","last_undegraded":"2024-02-18T00:35:59.641711+0000","last_fullsized":"2024-02-18T00:35:59.641711+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":198,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697690+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:35:59.697690+0000","last_peered":"2024-02-18T00:35:59.697690+0000","last_clean":"2024-02-18T00:35:59.697690+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:35:59.697690+0000","last_undegraded":"2024-02-18T00:35:59.697690+0000","last_fullsized":"2024-02-18T00:35:59.697690+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":304,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697693+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:35:59.697693+0000","last_peered":"2024-02-18T00:35:59.697693+0000","last_clean":"2024-02-18T00:35:59.697693+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:35:59.697693+0000","last_undegraded":"2024-02-18T00:35:59.697693+0000","last_fullsized":"2024-02-18T00:35:59.697693+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":249,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697192+0000","last_change":"2024-02-18T00:33:51.295157+0000","last_active":"2024-02-18T00:35:59.697192+0000","last_peered":"2024-02-18T00:35:59.697192+0000","last_clean":"2024-02-18T00:35:59.697192+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:35:59.697192+0000","last_undegraded":"2024-02-18T00:35:59.697192+0000","last_fullsized":"2024-02-18T00:35:59.697192+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.295093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.295093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":157,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.650542+0000","last_change":"2024-02-18T00:35:42.661117+0000","last_active":"2024-02-18T00:35:58.650542+0000","last_peered":"2024-02-18T00:35:58.650542+0000","last_clean":"2024-02-18T00:35:58.650542+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:35:58.650542+0000","last_undegraded":"2024-02-18T00:35:58.650542+0000","last_fullsized":"2024-02-18T00:35:58.650542+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.374328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.374328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698276+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:35:59.698276+0000","last_peered":"2024-02-18T00:35:59.698276+0000","last_clean":"2024-02-18T00:35:59.698276+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:35:59.698276+0000","last_undegraded":"2024-02-18T00:35:59.698276+0000","last_fullsized":"2024-02-18T00:35:59.698276+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":148,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652875+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:35:58.652875+0000","last_peered":"2024-02-18T00:35:58.652875+0000","last_clean":"2024-02-18T00:35:58.652875+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:35:58.652875+0000","last_undegraded":"2024-02-18T00:35:58.652875+0000","last_fullsized":"2024-02-18T00:35:58.652875+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":305,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697465+0000","last_change":"2024-02-18T00:33:56.121942+0000","last_active":"2024-02-18T00:35:59.697465+0000","last_peered":"2024-02-18T00:35:59.697465+0000","last_clean":"2024-02-18T00:35:59.697465+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:35:59.697465+0000","last_undegraded":"2024-02-18T00:35:59.697465+0000","last_fullsized":"2024-02-18T00:35:59.697465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.121872+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":258,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695337+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:35:59.695337+0000","last_peered":"2024-02-18T00:35:59.695337+0000","last_clean":"2024-02-18T00:35:59.695337+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:35:59.695337+0000","last_undegraded":"2024-02-18T00:35:59.695337+0000","last_fullsized":"2024-02-18T00:35:59.695337+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":355,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697317+0000","last_change":"2024-02-18T00:34:18.318312+0000","last_active":"2024-02-18T00:35:59.697317+0000","last_peered":"2024-02-18T00:35:59.697317+0000","last_clean":"2024-02-18T00:35:59.697317+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:35:59.697317+0000","last_undegraded":"2024-02-18T00:35:59.697317+0000","last_fullsized":"2024-02-18T00:35:59.697317+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:18.318190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.318190+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":162,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697632+0000","last_change":"2024-02-18T00:34:20.995201+0000","last_active":"2024-02-18T00:35:59.697632+0000","last_peered":"2024-02-18T00:35:59.697632+0000","last_clean":"2024-02-18T00:35:59.697632+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:35:59.697632+0000","last_undegraded":"2024-02-18T00:35:59.697632+0000","last_fullsized":"2024-02-18T00:35:59.697632+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:34:20.995125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.995125+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696859+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:35:59.696859+0000","last_peered":"2024-02-18T00:35:59.696859+0000","last_clean":"2024-02-18T00:35:59.696859+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:35:59.696859+0000","last_undegraded":"2024-02-18T00:35:59.696859+0000","last_fullsized":"2024-02-18T00:35:59.696859+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":281,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697290+0000","last_change":"2024-02-18T00:33:52.250914+0000","last_active":"2024-02-18T00:35:59.697290+0000","last_peered":"2024-02-18T00:35:59.697290+0000","last_clean":"2024-02-18T00:35:59.697290+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:35:59.697290+0000","last_undegraded":"2024-02-18T00:35:59.697290+0000","last_fullsized":"2024-02-18T00:35:59.697290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:52.250848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:52.250848+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":249,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695512+0000","last_change":"2024-02-18T00:34:28.731011+0000","last_active":"2024-02-18T00:35:59.695512+0000","last_peered":"2024-02-18T00:35:59.695512+0000","last_clean":"2024-02-18T00:35:59.695512+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:35:59.695512+0000","last_undegraded":"2024-02-18T00:35:59.695512+0000","last_fullsized":"2024-02-18T00:35:59.695512+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.730955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.730955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":248,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697509+0000","last_change":"2024-02-18T00:33:48.751672+0000","last_active":"2024-02-18T00:35:59.697509+0000","last_peered":"2024-02-18T00:35:59.697509+0000","last_clean":"2024-02-18T00:35:59.697509+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:35:59.697509+0000","last_undegraded":"2024-02-18T00:35:59.697509+0000","last_fullsized":"2024-02-18T00:35:59.697509+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.025296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.025296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":247,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.670711+0000","last_change":"2024-02-18T00:35:54.569108+0000","last_active":"2024-02-18T00:35:58.670711+0000","last_peered":"2024-02-18T00:35:58.670711+0000","last_clean":"2024-02-18T00:35:58.670711+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:35:58.670711+0000","last_undegraded":"2024-02-18T00:35:58.670711+0000","last_fullsized":"2024-02-18T00:35:58.670711+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.569033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.569033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":202,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.642092+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:35:59.642092+0000","last_peered":"2024-02-18T00:35:59.642092+0000","last_clean":"2024-02-18T00:35:59.642092+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:35:59.642092+0000","last_undegraded":"2024-02-18T00:35:59.642092+0000","last_fullsized":"2024-02-18T00:35:59.642092+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":277,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.643662+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:35:59.643662+0000","last_peered":"2024-02-18T00:35:59.643662+0000","last_clean":"2024-02-18T00:35:59.643662+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:35:59.643662+0000","last_undegraded":"2024-02-18T00:35:59.643662+0000","last_fullsized":"2024-02-18T00:35:59.643662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":263,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697840+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:35:59.697840+0000","last_peered":"2024-02-18T00:35:59.697840+0000","last_clean":"2024-02-18T00:35:59.697840+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:35:59.697840+0000","last_undegraded":"2024-02-18T00:35:59.697840+0000","last_fullsized":"2024-02-18T00:35:59.697840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":244,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.670956+0000","last_change":"2024-02-18T00:34:18.357435+0000","last_active":"2024-02-18T00:35:58.670956+0000","last_peered":"2024-02-18T00:35:58.670956+0000","last_clean":"2024-02-18T00:35:58.670956+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:35:58.670956+0000","last_undegraded":"2024-02-18T00:35:58.670956+0000","last_fullsized":"2024-02-18T00:35:58.670956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.357369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.357369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":242,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.656558+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:35:58.656558+0000","last_peered":"2024-02-18T00:35:58.656558+0000","last_clean":"2024-02-18T00:35:58.656558+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:35:58.656558+0000","last_undegraded":"2024-02-18T00:35:58.656558+0000","last_fullsized":"2024-02-18T00:35:58.656558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695839+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:35:59.695839+0000","last_peered":"2024-02-18T00:35:59.695839+0000","last_clean":"2024-02-18T00:35:59.695839+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:35:59.695839+0000","last_undegraded":"2024-02-18T00:35:59.695839+0000","last_fullsized":"2024-02-18T00:35:59.695839+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":285,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696436+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:35:59.696436+0000","last_peered":"2024-02-18T00:35:59.696436+0000","last_clean":"2024-02-18T00:35:59.696436+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:35:59.696436+0000","last_undegraded":"2024-02-18T00:35:59.696436+0000","last_fullsized":"2024-02-18T00:35:59.696436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":269,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698121+0000","last_change":"2024-02-18T00:33:48.748311+0000","last_active":"2024-02-18T00:35:59.698121+0000","last_peered":"2024-02-18T00:35:59.698121+0000","last_clean":"2024-02-18T00:35:59.698121+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:35:59.698121+0000","last_undegraded":"2024-02-18T00:35:59.698121+0000","last_fullsized":"2024-02-18T00:35:59.698121+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:33:45.906516+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.906516+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":244,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652053+0000","last_change":"2024-02-18T00:34:29.236028+0000","last_active":"2024-02-18T00:35:58.652053+0000","last_peered":"2024-02-18T00:35:58.652053+0000","last_clean":"2024-02-18T00:35:58.652053+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:35:58.652053+0000","last_undegraded":"2024-02-18T00:35:58.652053+0000","last_fullsized":"2024-02-18T00:35:58.652053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:29.235953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.235953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":238,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697143+0000","last_change":"2024-02-18T00:33:38.914876+0000","last_active":"2024-02-18T00:35:59.697143+0000","last_peered":"2024-02-18T00:35:59.697143+0000","last_clean":"2024-02-18T00:35:59.697143+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:35:59.697143+0000","last_undegraded":"2024-02-18T00:35:59.697143+0000","last_fullsized":"2024-02-18T00:35:59.697143+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:38.914691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:38.914691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":198,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696842+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:35:59.696842+0000","last_peered":"2024-02-18T00:35:59.696842+0000","last_clean":"2024-02-18T00:35:59.696842+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:35:59.696842+0000","last_undegraded":"2024-02-18T00:35:59.696842+0000","last_fullsized":"2024-02-18T00:35:59.696842+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":304,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652460+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:35:58.652460+0000","last_peered":"2024-02-18T00:35:58.652460+0000","last_clean":"2024-02-18T00:35:58.652460+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:35:58.652460+0000","last_undegraded":"2024-02-18T00:35:58.652460+0000","last_fullsized":"2024-02-18T00:35:58.652460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697418+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:35:59.697418+0000","last_peered":"2024-02-18T00:35:59.697418+0000","last_clean":"2024-02-18T00:35:59.697418+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:35:59.697418+0000","last_undegraded":"2024-02-18T00:35:59.697418+0000","last_fullsized":"2024-02-18T00:35:59.697418+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":263,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695799+0000","last_change":"2024-02-18T00:35:42.660706+0000","last_active":"2024-02-18T00:35:59.695799+0000","last_peered":"2024-02-18T00:35:59.695799+0000","last_clean":"2024-02-18T00:35:59.695799+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:35:59.695799+0000","last_undegraded":"2024-02-18T00:35:59.695799+0000","last_fullsized":"2024-02-18T00:35:59.695799+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.961303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.961303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":251,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.695021+0000","last_change":"2024-02-18T00:33:48.649527+0000","last_active":"2024-02-18T00:35:59.695021+0000","last_peered":"2024-02-18T00:35:59.695021+0000","last_clean":"2024-02-18T00:35:59.695021+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:35:59.695021+0000","last_undegraded":"2024-02-18T00:35:59.695021+0000","last_fullsized":"2024-02-18T00:35:59.695021+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.996913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.996913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":184,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.644471+0000","last_change":"2024-02-18T00:33:04.678891+0000","last_active":"2024-02-18T00:35:59.644471+0000","last_peered":"2024-02-18T00:35:59.644471+0000","last_clean":"2024-02-18T00:35:59.644471+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:35:59.644471+0000","last_undegraded":"2024-02-18T00:35:59.644471+0000","last_fullsized":"2024-02-18T00:35:59.644471+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":274,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.653205+0000","last_change":"2024-02-18T00:34:19.325220+0000","last_active":"2024-02-18T00:35:58.653205+0000","last_peered":"2024-02-18T00:35:58.653205+0000","last_clean":"2024-02-18T00:35:58.653205+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:35:58.653205+0000","last_undegraded":"2024-02-18T00:35:58.653205+0000","last_fullsized":"2024-02-18T00:35:58.653205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:34:19.325130+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.325130+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":149,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697208+0000","last_change":"2024-02-18T00:34:31.241231+0000","last_active":"2024-02-18T00:35:59.697208+0000","last_peered":"2024-02-18T00:35:59.697208+0000","last_clean":"2024-02-18T00:35:59.697208+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:35:59.697208+0000","last_undegraded":"2024-02-18T00:35:59.697208+0000","last_fullsized":"2024-02-18T00:35:59.697208+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.241146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":143,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.653161+0000","last_change":"2024-02-18T00:33:48.733307+0000","last_active":"2024-02-18T00:35:58.653161+0000","last_peered":"2024-02-18T00:35:58.653161+0000","last_clean":"2024-02-18T00:35:58.653161+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:35:58.653161+0000","last_undegraded":"2024-02-18T00:35:58.653161+0000","last_fullsized":"2024-02-18T00:35:58.653161+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.801504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.801504+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":243,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696103+0000","last_change":"2024-02-18T00:33:54.013575+0000","last_active":"2024-02-18T00:35:59.696103+0000","last_peered":"2024-02-18T00:35:59.696103+0000","last_clean":"2024-02-18T00:35:59.696103+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:35:59.696103+0000","last_undegraded":"2024-02-18T00:35:59.696103+0000","last_fullsized":"2024-02-18T00:35:59.696103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:54.013467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.013467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":198,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696690+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:35:59.696690+0000","last_peered":"2024-02-18T00:35:59.696690+0000","last_clean":"2024-02-18T00:35:59.696690+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:35:59.696690+0000","last_undegraded":"2024-02-18T00:35:59.696690+0000","last_fullsized":"2024-02-18T00:35:59.696690+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":282,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.650450+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:35:58.650450+0000","last_peered":"2024-02-18T00:35:58.650450+0000","last_clean":"2024-02-18T00:35:58.650450+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:35:58.650450+0000","last_undegraded":"2024-02-18T00:35:58.650450+0000","last_fullsized":"2024-02-18T00:35:58.650450+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":249,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697752+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:35:59.697752+0000","last_peered":"2024-02-18T00:35:59.697752+0000","last_clean":"2024-02-18T00:35:59.697752+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:35:59.697752+0000","last_undegraded":"2024-02-18T00:35:59.697752+0000","last_fullsized":"2024-02-18T00:35:59.697752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":253,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.664277+0000","last_change":"2024-02-18T00:34:38.428939+0000","last_active":"2024-02-18T00:35:58.664277+0000","last_peered":"2024-02-18T00:35:58.664277+0000","last_clean":"2024-02-18T00:35:58.664277+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:35:58.664277+0000","last_undegraded":"2024-02-18T00:35:58.664277+0000","last_fullsized":"2024-02-18T00:35:58.664277+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.428855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.428855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":242,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.652370+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:35:58.652370+0000","last_peered":"2024-02-18T00:35:58.652370+0000","last_clean":"2024-02-18T00:35:58.652370+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:35:58.652370+0000","last_undegraded":"2024-02-18T00:35:58.652370+0000","last_fullsized":"2024-02-18T00:35:58.652370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":189,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698320+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:35:59.698320+0000","last_peered":"2024-02-18T00:35:59.698320+0000","last_clean":"2024-02-18T00:35:59.698320+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:35:59.698320+0000","last_undegraded":"2024-02-18T00:35:59.698320+0000","last_fullsized":"2024-02-18T00:35:59.698320+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":329,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.697456+0000","last_change":"2024-02-18T00:33:41.866669+0000","last_active":"2024-02-18T00:35:59.697456+0000","last_peered":"2024-02-18T00:35:59.697456+0000","last_clean":"2024-02-18T00:35:59.697456+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:35:59.697456+0000","last_undegraded":"2024-02-18T00:35:59.697456+0000","last_fullsized":"2024-02-18T00:35:59.697456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:33:41.866531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.866531+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":257,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.696549+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:35:59.696549+0000","last_peered":"2024-02-18T00:35:59.696549+0000","last_clean":"2024-02-18T00:35:59.696549+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:35:59.696549+0000","last_undegraded":"2024-02-18T00:35:59.696549+0000","last_fullsized":"2024-02-18T00:35:59.696549+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":244,"reported_epoch":280,"state":"active+clean","last_fresh":"2024-02-18T00:35:58.650793+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:35:58.650793+0000","last_peered":"2024-02-18T00:35:58.650793+0000","last_clean":"2024-02-18T00:35:58.650793+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:35:58.650793+0000","last_undegraded":"2024-02-18T00:35:58.650793+0000","last_fullsized":"2024-02-18T00:35:58.650793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":252,"reported_epoch":281,"state":"active+clean","last_fresh":"2024-02-18T00:35:59.698032+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:35:59.698032+0000","last_peered":"2024-02-18T00:35:59.698032+0000","last_clean":"2024-02-18T00:35:59.698032+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:35:59.698032+0000","last_undegraded":"2024-02-18T00:35:59.698032+0000","last_fullsized":"2024-02-18T00:35:59.698032+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":179,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":178,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":177,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1202,"num_objects":119,"num_object_clones":1,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":539,"num_write_kb":534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1470464,"data_stored":3639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1780,"ondisk_log_size":1780,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":397,"num_read_kb":392,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266773,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463003,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301800,"kb_used_data":4572,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443352,"statfs":{"total":95995035648,"available":95685992448,"internally_reserved":0,"allocated":4681728,"data_stored":2568194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":2.7280000000000002,"15min":1.234},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.7879999999999998,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":1.1619999999999999,"15min":0.71899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.1130000000000004,"5min":10.433,"15min":10.433},"last":0.28899999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":2.9590000000000001,"15min":1.3169999999999999},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":2.8330000000000002,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":2.9409999999999998,"15min":1.306},"min":{"1min":0.36899999999999999,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":3.6030000000000002,"5min":71.872,"15min":71.872},"last":0.34100000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":1.389,"15min":0.80000000000000004},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.316},"max":{"1min":3.657,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.31},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.839,"15min":0.95899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":4.1600000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.39500000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":1.383,"15min":0.79900000000000004},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.5670000000000002,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.284,"15min":0.77200000000000002},"min":{"1min":0.438,"5min":0.309,"15min":0.309},"max":{"1min":3.694,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.37}]},{"osd":4,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.49099999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.4359999999999999,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1890000000000001,"15min":0.66200000000000003},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.907,"5min":35.18,"15min":35.18},"last":0.26100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":4.476,"5min":10.606,"15min":10.606},"last":0.378},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.2310000000000001,"15min":0.68799999999999994},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6769999999999996,"5min":35.119,"15min":35.119},"last":0.40200000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60699999999999998,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.754,"5min":1.089,"15min":0.71499999999999997},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.3620000000000001,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.35999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314100,"kb_used_data":4580,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431052,"statfs":{"total":95995035648,"available":95673397248,"internally_reserved":0,"allocated":4689920,"data_stored":2568149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":1.2290000000000001,"15min":0.625},"min":{"1min":0.19900000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.47899999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":1.3140000000000001,"15min":0.65900000000000003},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46999999999999997,"5min":43.137,"15min":43.137},"last":0.42899999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.5369999999999999,"15min":0.76100000000000001},"min":{"1min":0.23000000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55300000000000005,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.97599999999999998,"15min":0.58599999999999997},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":17.596,"15min":17.596},"last":0.374}]},{"osd":3,"last update":"Sun Feb 18 00:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":1.355,"15min":0.755},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51700000000000002,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.432,"5min":1.3580000000000001,"15min":0.77600000000000002},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:35:04 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":1.1499999999999999,"15min":0.78300000000000003},"min":{"1min":0.39300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88900000000000001,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.167,"15min":0.76400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77000000000000002,"5min":12.75,"15min":12.75},"last":0.56699999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.1439999999999999,"15min":0.77700000000000002},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.8759999999999999,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.1499999999999999,"15min":0.79200000000000004},"min":{"1min":0.41999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":4.1180000000000003,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.53400000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.1779999999999999,"15min":0.81399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":12.834,"15min":12.834},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.149,"15min":0.80300000000000005},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81100000000000005,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.151,"15min":0.80500000000000005},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92500000000000004,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.141,"15min":0.81599999999999995},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81699999999999995,"5min":12.805,"15min":12.805},"last":0.60299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314096,"kb_used_data":4576,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431056,"statfs":{"total":95995035648,"available":95673401344,"internally_reserved":0,"allocated":4685824,"data_stored":2567261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":2.2679999999999998,"15min":1.1359999999999999},"min":{"1min":0.25700000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.0820000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":1.5089999999999999,"15min":0.73999999999999999},"min":{"1min":0.221,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":1.179,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.39400000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":1.827,"15min":0.87},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.577,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":1.8560000000000001,"15min":0.874},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":2.153,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.38400000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.1659999999999999,"15min":0.68100000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55400000000000005,"5min":19.006,"15min":19.006},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.442,"5min":2.222,"15min":1.071},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2130000000000001,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.434}]},{"osd":4,"last update":"Sun Feb 18 00:35:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":1.661,"15min":0.91700000000000004},"min":{"1min":0.32800000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":3.1850000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":1.6850000000000001,"15min":0.96699999999999997},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.5680000000000001,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.60999999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":1.617,"15min":0.93200000000000005},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.5,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.8169999999999999,"15min":0.96599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":3.4060000000000001,"5min":57.991,"15min":57.991},"last":0.69299999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.2270000000000001,"15min":0.79300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.712,"15min":0.95399999999999996},"min":{"1min":0.41799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.28,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":1.8700000000000001,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5940000000000001,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.8029999999999999,"15min":0.99299999999999999},"min":{"1min":0.45500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":3.0249999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.71799999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411439,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314204,"kb_used_data":4680,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430948,"statfs":{"total":95995035648,"available":95673290752,"internally_reserved":0,"allocated":4792320,"data_stored":2577968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.73099999999999998,"15min":0.495},"min":{"1min":0.216,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61899999999999999,"5min":12.144,"15min":12.144},"last":0.22},{"interface":"front","average":{"1min":0.35699999999999998,"5min":1.0580000000000001,"15min":0.60799999999999998},"min":{"1min":0.246,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66500000000000004,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.436}]},{"osd":1,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.71999999999999997,"15min":0.5},"min":{"1min":0.23899999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":2.3370000000000002,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.82899999999999996,"15min":0.54700000000000004},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.77100000000000002,"5min":18.919,"15min":18.919},"last":0.311}]},{"osd":3,"last update":"Sun Feb 18 00:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.92400000000000004,"15min":0.61599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71799999999999997,"5min":22.712,"15min":22.712},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.92300000000000004,"15min":0.627},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.629,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.372}]},{"osd":4,"last update":"Sun Feb 18 00:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.88300000000000001,"15min":0.68700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80500000000000005,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68300000000000005},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79000000000000004,"5min":10.016,"15min":10.016},"last":0.38800000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.78200000000000003,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":6.766,"15min":6.766},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.85399999999999998,"15min":0.70099999999999996},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78500000000000003,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.36699999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.79600000000000004,"15min":0.69299999999999995},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":6.806,"15min":6.806},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.90800000000000003,"15min":0.74099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.40100000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.81000000000000005,"15min":0.74099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.89800000000000002,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.47999999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149801,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310060,"kb_used_data":4636,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435092,"statfs":{"total":95995035648,"available":95677534208,"internally_reserved":0,"allocated":4747264,"data_stored":2577277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.65200000000000002,"15min":0.47799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53100000000000003,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.435},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.75700000000000001,"15min":0.52500000000000002},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.304,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.47299999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.60499999999999998,"15min":0.48199999999999998},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.0800000000000001,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.34699999999999998,"5min":1.1990000000000001,"15min":0.66800000000000004},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65400000000000003,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":1.0700000000000001,"15min":0.64800000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.117,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.54300000000000004,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.53000000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.88200000000000001,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":10.131,"15min":10.131},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.70199999999999996,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72899999999999998,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.71299999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.70299999999999996,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81799999999999995,"15min":0.67800000000000005},"min":{"1min":0.41099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1279999999999999,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.92700000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":1.0189999999999999,"15min":0.73599999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86199999999999999,"5min":10.189,"15min":10.189},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.73599999999999999,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84999999999999998,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.82399999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.78100000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.92300000000000004,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80300000000000005,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.88100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986276,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301824,"kb_used_data":4592,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443328,"statfs":{"total":95995035648,"available":95685967872,"internally_reserved":0,"allocated":4702208,"data_stored":2568038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":1.5629999999999999,"15min":0.92600000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65900000000000003,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":1.5089999999999999,"15min":0.88100000000000001},"min":{"1min":0.34100000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.63400000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.81200000000000006,"15min":0.65500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67300000000000004,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.435,"5min":1.595,"15min":0.90700000000000003},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.82699999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.78300000000000003,"15min":0.67300000000000004},"min":{"1min":0.373,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60299999999999998,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.755},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.90200000000000002,"15min":0.68600000000000005},"min":{"1min":0.33100000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1459999999999999,"5min":17.77,"15min":17.77},"last":0.67300000000000004}]},{"osd":3,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.69699999999999995,"15min":0.63600000000000001},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64200000000000002,"5min":7.218,"15min":7.218},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.502,"5min":1.071,"15min":0.72299999999999998},"min":{"1min":0.372,"5min":0.309,"15min":0.309},"max":{"1min":0.65100000000000002,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.439,"15min":0.38300000000000001},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.45900000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.311}]},{"osd":6,"last update":"Sun Feb 18 00:35:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45400000000000001,"15min":0.40799999999999997},"min":{"1min":0.26100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.60699999999999998,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.497,"15min":0.44900000000000001},"min":{"1min":0.307,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.629,"5min":0.751,"15min":0.751},"last":0.39400000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72399999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42199999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724639,"num_pgs":103,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318132,"kb_used_data":4512,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93427020,"statfs":{"total":95995035648,"available":95669268480,"internally_reserved":0,"allocated":4620288,"data_stored":2568327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57199999999999995,"15min":0.54700000000000004},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.8480000000000001,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.69899999999999995,"15min":0.59699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.8149999999999999,"5min":12.744,"15min":12.744},"last":0.47399999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.625},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6669999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.73399999999999999,"15min":0.62},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":4.7149999999999999,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.44800000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.65000000000000002,"15min":0.61299999999999999},"min":{"1min":0.30199999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.7410000000000001,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67100000000000004,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.734,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.49399999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":2.1099999999999999,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.505},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63200000000000001,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.546,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.54600000000000004,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.621,"5min":11.818,"15min":11.818},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.48799999999999999,"15min":0.41299999999999998},"min":{"1min":0.309,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73299999999999998,"5min":4.274,"15min":4.274},"last":0.40100000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.64000000000000001,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":18.605,"15min":18.605},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52500000000000002,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65600000000000003,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.437}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.60099999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.41599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196608,"data_stored":495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":177,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":177,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":177,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":177,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":177,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":177,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":177,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:36:07.981 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:07.981 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:07.982 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.482552+0000 osd.4 (osd.4) 171 : cluster [DBG] 62.16 scrub starts 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.486021+0000 osd.4 (osd.4) 172 : cluster [DBG] 62.16 scrub ok 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.639873+0000 osd.1 (osd.1) 149 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.641180+0000 osd.1 (osd.1) 150 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.724658+0000 osd.3 (osd.3) 165 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.725964+0000 osd.3 (osd.3) 166 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.754652+0000 osd.2 (osd.2) 125 : cluster [DBG] 62.12 deep-scrub starts 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:06.758765+0000 osd.2 (osd.2) 126 : cluster [DBG] 62.12 deep-scrub ok 2024-02-18T00:36:07.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.095542+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:07.989 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.095632+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:07.989 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:07.095678+0000 mon.a (mon.0) 1986 : cluster [DBG] osdmap e288: 8 total, 8 up, 7 in 2024-02-18T00:36:07.989 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.096396+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]: dispatch 2024-02-18T00:36:07.989 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.096627+0000 mon.a (mon.0) 1988 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]: dispatch 2024-02-18T00:36:07.989 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.107272+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:07.989 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.107543+0000 mon.a (mon.0) 1989 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:08.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:08.111 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:08.210 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:08.214 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:08.214 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:08.286 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-18T00:36:08.315 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:08.356 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:08.378 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:08.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1311386213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9c00f8420 msgr2=0x7fb9c0104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1311386213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9c00f8420 0x7fb9c0104a20 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 rx=0x7fb9b8009be0 tx=0x7fb9b8004d10).stop 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1311386213 shutdown_connections 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1311386213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9c00fdbb0 0x7fb9c00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1311386213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9c00f8420 0x7fb9c0104a20 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1311386213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00fe510 0x7fb9c00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1311386213 >> 172.21.15.59:0/1311386213 conn(0x7fb9c00f3800 msgr2=0x7fb9c00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:08.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1311386213 shutdown_connections 2024-02-18T00:36:08.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1311386213 wait complete. 2024-02-18T00:36:08.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 librados: starting msgr at 2024-02-18T00:36:08.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.453+0000 7fb9c56fb700 1 librados: starting objecter 2024-02-18T00:36:08.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 Processor -- start 2024-02-18T00:36:08.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 -- start start 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 librados: setting wanted keys 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 librados: calling monclient init 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00f8420 0x7fb9c010df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9c00fdbb0 0x7fb9c010e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9c00fe510 0x7fb9c01080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01085a0 con 0x7fb9c00f8420 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0108710 con 0x7fb9c00fe510 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9c56fb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c01088d0 con 0x7fb9c00fdbb0 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9beffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00f8420 0x7fb9c010df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9beffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00f8420 0x7fb9c010df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48070/0 (socket says 172.21.15.59:48070) 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9beffd700 1 -- 172.21.15.59:0/1469589477 learned_addr learned my addr 172.21.15.59:0/1469589477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.454+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9c00fe510 0x7fb9c01080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9be7fc700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9c00fdbb0 0x7fb9c010e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9beffd700 1 -- 172.21.15.59:0/1469589477 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9c00fdbb0 msgr2=0x7fb9c010e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb9c00fdbb0 0x7fb9c010e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9beffd700 1 -- 172.21.15.59:0/1469589477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9c00fe510 msgr2=0x7fb9c01080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9c00fe510 0x7fb9c01080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9beffd700 1 -- 172.21.15.59:0/1469589477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8005040 con 0x7fb9c00f8420 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9c00fe510 0x7fb9c01080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00f8420 0x7fb9c010df10 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 rx=0x7fb9b000e970 tx=0x7fb9b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9b7fff700 10 client.?.objecter ms_handle_connect 0x7fb9c00f8420 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9b7fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b000ee60 con 0x7fb9c00f8420 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9b000f040 con 0x7fb9c00f8420 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0007ea0 con 0x7fb9c00f8420 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b80097c0 con 0x7fb9c00f8420 2024-02-18T00:36:08.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9c56fb700 10 client.31005.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:08.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c0108f10 con 0x7fb9c00f8420 2024-02-18T00:36:08.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.455+0000 7fb9c56fb700 1 librados: init done 2024-02-18T00:36:08.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.456+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fb9c0107110 con 0x7fb9c00f8420 2024-02-18T00:36:08.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.457+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb9b000cb10 con 0x7fb9c00f8420 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb9ac062c90 0x7fb9ac065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(289..289 src has 44..289) v4 ==== 8867+0+0 (secure 0 0 0) 0x7fb9b007d450 con 0x7fb9c00f8420 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 osd_map(289..289 src has 44..289) v4 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 3 client.31005.objecter handle_osd_map got epochs [289,289] > 0 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 3 client.31005.objecter handle_osd_map decoding full epoch 289 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9be7fc700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb9ac062c90 0x7fb9ac065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 20 client.31005.objecter dump_active .. 0 homeless 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v289) v1 ==== 54+0+16 (secure 0 0 0) 0x7fb9b007d8f0 con 0x7fb9c00f8420 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9ac062c90 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9be7fc700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb9ac062c90 0x7fb9ac065150 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 rx=0x7fb9c0109850 tx=0x7fb9a8009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 1 2024-02-18T00:36:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb9c00598f0 0x7fb9c0059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 1 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 0 1 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0109590 osd=0 4 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 10 client.31005.objecter _send_command 1 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c005fa60 con 0x7fb9c00598f0 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.458+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb9c00598f0 0x7fb9c0059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.459+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c00598f0 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.459+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb9c00598f0 0x7fb9c0059cd0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.459+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9c005fa60 con 0x7fb9c00598f0 2024-02-18T00:36:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.459+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:36:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.459+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 1 = system:0 2024-02-18T00:36:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.459+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c0109590 osd=0 4 2024-02-18T00:36:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.459+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 0 1 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 2 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 2 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c0109590 osd=0 3 2024-02-18T00:36:08.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 0 2 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0109590 osd=0 4 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 10 client.31005.objecter _send_command 2 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.474+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c005fbd0 con 0x7fb9c00598f0 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c005fbd0 con 0x7fb9c00598f0 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c0109590 osd=0 4 2024-02-18T00:36:08.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 0 2 2024-02-18T00:36:08.477 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 3 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6805/26279110 conn(0x7fb9c0028540 0x7fb9c011b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 3 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 1 3 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c00595a0 osd=1 4 2024-02-18T00:36:08.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 10 client.31005.objecter _send_command 3 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c0059710 con 0x7fb9c0028540 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6805/26279110 conn(0x7fb9c0028540 0x7fb9c011b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6805/26279110 conn(0x7fb9c0028540 0x7fb9c011b1f0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.476+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c0028540 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.477+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9c0059710 con 0x7fb9c0028540 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.477+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.477+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 3 = system:0 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.477+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c00595a0 osd=1 4 2024-02-18T00:36:08.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.477+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 1 3 2024-02-18T00:36:08.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:08.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 4 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 4 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c00595a0 osd=1 3 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 1 4 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c00595a0 osd=1 4 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 10 client.31005.objecter _send_command 4 2024-02-18T00:36:08.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.492+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c006cf20 con 0x7fb9c0028540 2024-02-18T00:36:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.493+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c006cf20 con 0x7fb9c0028540 2024-02-18T00:36:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.493+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.493+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.493+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c00595a0 osd=1 4 2024-02-18T00:36:08.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.493+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 1 4 2024-02-18T00:36:08.495 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:36:08.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 5 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb9c011bd60 0x7fb9c011e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 5 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 2 5 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c011b6e0 osd=2 4 2024-02-18T00:36:08.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 10 client.31005.objecter _send_command 5 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c0120da0 con 0x7fb9c011bd60 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9be7fc700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb9c011bd60 0x7fb9c011e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c011bd60 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.494+0000 7fb9be7fc700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb9c011bd60 0x7fb9c011e150 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.495+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9c0120da0 con 0x7fb9c011bd60 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.495+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.495+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 5 = system:0 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.495+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c011b6e0 osd=2 4 2024-02-18T00:36:08.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.495+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 2 5 2024-02-18T00:36:08.509 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:08.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 6 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 6 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c011b6e0 osd=2 3 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 2 6 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c011b6e0 osd=2 4 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 10 client.31005.objecter _send_command 6 2024-02-18T00:36:08.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.510+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c00feb90 con 0x7fb9c011bd60 2024-02-18T00:36:08.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c00feb90 con 0x7fb9c011bd60 2024-02-18T00:36:08.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c011b6e0 osd=2 4 2024-02-18T00:36:08.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 2 6 2024-02-18T00:36:08.513 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:36:08.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 7 2024-02-18T00:36:08.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb9c011f7c0 0x7fb9c011fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 7 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 3 7 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0027b90 osd=3 4 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 10 client.31005.objecter _send_command 7 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c0028300 con 0x7fb9c011f7c0 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb9c011f7c0 0x7fb9c011fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb9c011f7c0 0x7fb9c011fc00 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.512+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c011f7c0 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.513+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9c0028300 con 0x7fb9c011f7c0 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.513+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:36:08.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.513+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 7 = system:0 2024-02-18T00:36:08.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.513+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c0027b90 osd=3 4 2024-02-18T00:36:08.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.513+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 3 7 2024-02-18T00:36:08.529 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 8 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 8 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c0027b90 osd=3 3 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 3 8 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0027b90 osd=3 4 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 10 client.31005.objecter _send_command 8 2024-02-18T00:36:08.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.529+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c01208c0 con 0x7fb9c011f7c0 2024-02-18T00:36:08.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c01208c0 con 0x7fb9c011f7c0 2024-02-18T00:36:08.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c0027b90 osd=3 4 2024-02-18T00:36:08.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 3 8 2024-02-18T00:36:08.532 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:36:08.532 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:08.533 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 9 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb9c0122fd0 0x7fb9c01253c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 9 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 4 9 2024-02-18T00:36:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c01208c0 osd=4 4 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 10 client.31005.objecter _send_command 9 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c0120b20 con 0x7fb9c0122fd0 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.531+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb9c0122fd0 0x7fb9c01253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.532+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c0122fd0 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.532+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb9c0122fd0 0x7fb9c01253c0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.533+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9c0120b20 con 0x7fb9c0122fd0 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.533+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.533+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 9 = system:0 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.533+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c01208c0 osd=4 4 2024-02-18T00:36:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.533+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 4 9 2024-02-18T00:36:08.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 10 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 10 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c01208c0 osd=4 3 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 4 10 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c01208c0 osd=4 4 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 10 client.31005.objecter _send_command 10 2024-02-18T00:36:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.548+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c0120b20 con 0x7fb9c0122fd0 2024-02-18T00:36:08.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c0120b20 con 0x7fb9c0122fd0 2024-02-18T00:36:08.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c01208c0 osd=4 4 2024-02-18T00:36:08.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 4 10 2024-02-18T00:36:08.552 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 11 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb9c0125e00 0x7fb9c0126240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 11 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 5 11 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0120b20 osd=5 4 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 10 client.31005.objecter _send_command 11 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.550+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c0127e90 con 0x7fb9c0125e00 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.551+0000 7fb9be7fc700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb9c0125e00 0x7fb9c0126240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.551+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c0125e00 2024-02-18T00:36:08.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.551+0000 7fb9be7fc700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb9c0125e00 0x7fb9c0126240 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.552+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9a800f040 con 0x7fb9c0125e00 2024-02-18T00:36:08.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.552+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:36:08.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.552+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 11 = system:0 2024-02-18T00:36:08.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.552+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c0120b20 osd=5 4 2024-02-18T00:36:08.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.552+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 5 11 2024-02-18T00:36:08.558 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:08.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 12 2024-02-18T00:36:08.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 12 2024-02-18T00:36:08.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c0120b20 osd=5 3 2024-02-18T00:36:08.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 5 12 2024-02-18T00:36:08.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0120b20 osd=5 4 2024-02-18T00:36:08.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 10 client.31005.objecter _send_command 12 2024-02-18T00:36:08.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.566+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c012a050 con 0x7fb9c0125e00 2024-02-18T00:36:08.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c012a050 con 0x7fb9c0125e00 2024-02-18T00:36:08.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c0120b20 osd=5 4 2024-02-18T00:36:08.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 5 12 2024-02-18T00:36:08.569 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:36:08.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 13 2024-02-18T00:36:08.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb9c012a770 0x7fb9c012abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 13 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 6 13 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012a050 osd=6 4 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 10 client.31005.objecter _send_command 13 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c012e480 con 0x7fb9c012a770 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.568+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb9c012a770 0x7fb9c012abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.569+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c012a770 2024-02-18T00:36:08.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.569+0000 7fb9bf7fe700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb9c012a770 0x7fb9c012abd0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.569+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9c012e480 con 0x7fb9c012a770 2024-02-18T00:36:08.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.569+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:36:08.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.569+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 13 = system:0 2024-02-18T00:36:08.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.569+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c012a050 osd=6 4 2024-02-18T00:36:08.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.569+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 6 13 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 14 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 14 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c012a050 osd=6 3 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 6 14 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012a050 osd=6 4 2024-02-18T00:36:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 10 client.31005.objecter _send_command 14 2024-02-18T00:36:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.584+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c012e480 con 0x7fb9c012a770 2024-02-18T00:36:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.585+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c012e480 con 0x7fb9c012a770 2024-02-18T00:36:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.585+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.585+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.585+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c012a050 osd=6 4 2024-02-18T00:36:08.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.585+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 6 14 2024-02-18T00:36:08.588 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 15 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6812/53025171 conn(0x7fb9c012d7a0 0x7fb9c012dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 15 2024-02-18T00:36:08.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 7 15 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012e480 osd=7 4 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 10 client.31005.objecter _send_command 15 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c012e220 con 0x7fb9c012d7a0 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6812/53025171 conn(0x7fb9c012d7a0 0x7fb9c012dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9b7fff700 10 client.31005.objecter ms_handle_connect 0x7fb9c012d7a0 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.586+0000 7fb9beffd700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6812/53025171 conn(0x7fb9c012d7a0 0x7fb9c012dc00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.587+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb9c012e220 con 0x7fb9c012d7a0 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.587+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.587+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 15 = system:0 2024-02-18T00:36:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.587+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c012e480 osd=7 4 2024-02-18T00:36:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.587+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 7 15 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 10 client.31005.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign -1 16 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 20 client.31005.objecter _get_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_remove -1 16 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 20 client.31005.objecter get_session s=0x7fb9c012e480 osd=7 3 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 15 client.31005.objecter _session_command_op_assign 7 16 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012e480 osd=7 4 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 10 client.31005.objecter _send_command 16 2024-02-18T00:36:08.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.601+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb9c012e220 con 0x7fb9c012d7a0 2024-02-18T00:36:08.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9b7fff700 1 -- 172.21.15.59:0/1469589477 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb9c012e220 con 0x7fb9c012d7a0 2024-02-18T00:36:08.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9b7fff700 10 client.31005.objecter ms_dispatch 0x7fb9c0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:08.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9b7fff700 10 client.31005.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9b7fff700 20 client.31005.objecter put_session s=0x7fb9c012e480 osd=7 4 2024-02-18T00:36:08.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9b7fff700 15 client.31005.objecter _session_command_op_remove 7 16 2024-02-18T00:36:08.604 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 librados: watch_flush enter 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 librados: watch_flush exit 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.0 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb9c00598f0 msgr2=0x7fb9c0059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb9c00598f0 0x7fb9c0059cd0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0109590 osd=0 1 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.1 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6805/26279110 conn(0x7fb9c0028540 msgr2=0x7fb9c011b1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6805/26279110 conn(0x7fb9c0028540 0x7fb9c011b1f0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c00595a0 osd=1 1 2024-02-18T00:36:08.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.2 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb9c011bd60 msgr2=0x7fb9c011e150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb9c011bd60 0x7fb9c011e150 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c011b6e0 osd=2 1 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.3 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb9c011f7c0 msgr2=0x7fb9c011fc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9bf7fe700 1 -- 172.21.15.59:0/1469589477 reap_dead start 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb9c011f7c0 0x7fb9c011fc00 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0027b90 osd=3 1 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.4 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb9c0122fd0 msgr2=0x7fb9c01253c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb9c0122fd0 0x7fb9c01253c0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c01208c0 osd=4 1 2024-02-18T00:36:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.5 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb9c0125e00 msgr2=0x7fb9c0126240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb9c0125e00 0x7fb9c0126240 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c0120b20 osd=5 1 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.6 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb9c012a770 msgr2=0x7fb9c012abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb9c012a770 0x7fb9c012abd0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012a050 osd=6 1 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter close_session for osd.7 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6812/53025171 conn(0x7fb9c012d7a0 msgr2=0x7fb9c012dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.137:6812/53025171 conn(0x7fb9c012d7a0 0x7fb9c012dc00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter put_session s=0x7fb9c012e480 osd=7 1 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 20 client.31005.objecter shutdown clearing up homeless session... 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 10 client.31005.objecter successfully canceled tick 2024-02-18T00:36:08.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9bf7fe700 1 -- 172.21.15.59:0/1469589477 reap_dead start 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb9ac062c90 msgr2=0x7fb9ac065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb9ac062c90 0x7fb9ac065150 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 rx=0x7fb9c0109850 tx=0x7fb9a8009cb0).stop 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00f8420 msgr2=0x7fb9c010df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.603+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00f8420 0x7fb9c010df10 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 rx=0x7fb9b000e970 tx=0x7fb9b000b040).stop 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.604+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 shutdown_connections 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.604+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9c00f8420 0x7fb9c010df10 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.604+0000 7fb9c56fb700 1 --2- 172.21.15.59:0/1469589477 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb9ac062c90 0x7fb9ac065150 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.604+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 >> 172.21.15.59:0/1469589477 conn(0x7fb9c00f3800 msgr2=0x7fb9c00fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.604+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 shutdown_connections 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.604+0000 7fb9c56fb700 1 -- 172.21.15.59:0/1469589477 wait complete. 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:08.604+0000 7fb9c56fb700 1 librados: shutdown 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.609 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:36:08.634 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:08.662 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:08.783 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAit: : entry=12 expected=12 2024-02-18T00:36:08.783 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:d83876eb:::4:head 2024-02-18T00:36:08.783 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:d83876eb:::4:head expected=26:d83876eb:::4:head 2024-02-18T00:36:08.783 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:d83876eb:::4:head -> 4 2024-02-18T00:36:08.783 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=4 expected=4 2024-02-18T00:36:08.783 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:cfc208b3:::3:head 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:cfc208b3:::3:head expected=26:cfc208b3:::3:head 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:cfc208b3:::3:head -> 3 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=3 expected=3 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:c4fdafeb:::6:head 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:c4fdafeb:::6:head expected=26:c4fdafeb:::6:head 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:c4fdafeb:::6:head -> 6 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=6 expected=6 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:bd63b0f1:::8:head 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:bd63b0f1:::8:head expected=26:bd63b0f1:::8:head 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:bd63b0f1:::8:head -> 8 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=8 expected=8 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:b29083e3:::5:head 2024-02-18T00:36:08.784 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:b29083e3:::5:head expected=26:b29083e3:::5:head 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:b29083e3:::5:head -> 5 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=5 expected=5 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:02547ec2:::1:head 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:02547ec2:::1:head expected=26:02547ec2:::1:head 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:02547ec2:::1:head -> 1 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=1 expected=1 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:566253c9:::13:head 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:566253c9:::13:head expected=26:566253c9:::13:head 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:566253c9:::13:head -> 13 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=13 expected=13 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : seek to 26:52ea6a34:::10:head 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : cursor()=26:52ea6a34:::10:head expected=26:52ea6a34:::10:head 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: > 26:52ea6a34:::10:head -> 10 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: : entry=10 expected=10 2024-02-18T00:36:08.785 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (223 ms) 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (68412 ms) 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: split 0/5 -> MIN 26:33333333::::head 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: split 1/5 -> 26:33333333::::head 26:66666666::::head 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: split 2/5 -> 26:66666666::::head 26:99999999::::head 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: split 3/5 -> 26:99999999::::head 26:cccccccc::::head 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: split 4/5 -> 26:cccccccc::::head MAX 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (110036 ms) 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] 7 tests from LibRadosList (180766 ms total) 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-18T00:36:08.786 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1095 ms) 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset foo1,foo2,foo3 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo1 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo2 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo3 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset foo1,foo4,foo5 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo4 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo5 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo1 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset foo6,foo7 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo7 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: foo6 2024-02-18T00:36:08.787 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns1:foo4 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns1:foo5 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns2:foo7 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns2:foo6 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: ns1:foo1 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: :foo1 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: :foo2 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: :foo3 2024-02-18T00:36:08.788 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (30 ms) 2024-02-18T00:36:08.804 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:08.807 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:08.807 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:08.908 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:08.929 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:36:08.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:07.452678+0000 osd.4 (osd.4) 173 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:36:08.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:07.453609+0000 osd.4 (osd.4) 174 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:07.531917+0000 osd.6 (osd.6) 145 : cluster [DBG] 2.22 deep-scrub starts 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:07.532773+0000 osd.6 (osd.6) 146 : cluster [DBG] 2.22 deep-scrub ok 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.654342+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.654426+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.654479+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.654541+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:07.654604+0000 mon.a (mon.0) 1994 : cluster [DBG] osdmap e289: 8 total, 8 up, 7 in 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.660184+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.660602+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.680078+0000 mon.b (mon.1) 361 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.680492+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.681182+0000 mon.b (mon.1) 362 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.938 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.681316+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.682048+0000 mon.b (mon.1) 363 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:08.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:07.682388+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:08.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:07.861421+0000 mgr.y (mgr.24454) 301 : cluster [DBG] pgmap v421: 246 pgs: 8 unknown, 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2 op/s 2024-02-18T00:36:08.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.051155+0000 osd.5 (osd.5) 135 : cluster [DBG] 6.d deep-scrub starts 2024-02-18T00:36:08.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.054201+0000 osd.5 (osd.5) 136 : cluster [DBG] 6.d deep-scrub ok 2024-02-18T00:36:08.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:08.457054+0000 mon.a (mon.0) 1999 : audit [DBG] from='client.? 172.21.15.59:0/1469589477' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:08.939 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:07.452678+0000 osd.4 (osd.4) 173 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:07.453609+0000 osd.4 (osd.4) 174 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:07.531917+0000 osd.6 (osd.6) 145 : cluster [DBG] 2.22 deep-scrub starts 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:07.532773+0000 osd.6 (osd.6) 146 : cluster [DBG] 2.22 deep-scrub ok 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.654342+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.654426+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.654479+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.654541+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:07.654604+0000 mon.a (mon.0) 1994 : cluster [DBG] osdmap e289: 8 total, 8 up, 7 in 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.660184+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.660602+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.680078+0000 mon.b (mon.1) 361 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.680492+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.681182+0000 mon.b (mon.1) 362 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.681316+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.682048+0000 mon.b (mon.1) 363 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:07.682388+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:07.861421+0000 mgr.y (mgr.24454) 301 : cluster [DBG] pgmap v421: 246 pgs: 8 unknown, 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2 op/s 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.051155+0000 osd.5 (osd.5) 135 : cluster [DBG] 6.d deep-scrub starts 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.054201+0000 osd.5 (osd.5) 136 : cluster [DBG] 6.d deep-scrub ok 2024-02-18T00:36:08.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:08.457054+0000 mon.a (mon.0) 1999 : audit [DBG] from='client.? 172.21.15.59:0/1469589477' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:08.941 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:07.452678+0000 osd.4 (osd.4) 173 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:36:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:07.453609+0000 osd.4 (osd.4) 174 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:36:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:07.531917+0000 osd.6 (osd.6) 145 : cluster [DBG] 2.22 deep-scrub starts 2024-02-18T00:36:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:07.532773+0000 osd.6 (osd.6) 146 : cluster [DBG] 2.22 deep-scrub ok 2024-02-18T00:36:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.654342+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi059-82917-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.654426+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? 172.21.15.59:0/3784803143' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi059-82877-40"}]': finished 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.654479+0000 mon.a (mon.0) 1992 : audit [INF] from='client.? 172.21.15.59:0/3697991996' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi059-82882-49"}]': finished 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.654541+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:07.654604+0000 mon.a (mon.0) 1994 : cluster [DBG] osdmap e289: 8 total, 8 up, 7 in 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.660184+0000 mon.c (mon.2) 253 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.660602+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]: dispatch 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.680078+0000 mon.b (mon.1) 361 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.680492+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.681182+0000 mon.b (mon.1) 362 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.681316+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.682048+0000 mon.b (mon.1) 363 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:09.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:07.682388+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:09.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:07.861421+0000 mgr.y (mgr.24454) 301 : cluster [DBG] pgmap v421: 246 pgs: 8 unknown, 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2 op/s 2024-02-18T00:36:09.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.051155+0000 osd.5 (osd.5) 135 : cluster [DBG] 6.d deep-scrub starts 2024-02-18T00:36:09.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.054201+0000 osd.5 (osd.5) 136 : cluster [DBG] 6.d deep-scrub ok 2024-02-18T00:36:09.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:08.457054+0000 mon.a (mon.0) 1999 : audit [DBG] from='client.? 172.21.15.59:0/1469589477' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:09.064 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:09.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:09.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:09.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:09.206 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:09.553 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:09.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:09.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:09.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:09.687 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:09.951 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.416858+0000 osd.4 (osd.4) 175 : cluster [DBG] 62.d scrub starts 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.420337+0000 osd.4 (osd.4) 176 : cluster [DBG] 62.d scrub ok 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.455388+0000 osd.0 (osd.0) 141 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.457422+0000 osd.0 (osd.0) 142 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.651894+0000 mon.a (mon.0) 2000 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.683923+0000 osd.1 (osd.1) 151 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.716769+0000 osd.1 (osd.1) 152 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:08.726063+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:08.726215+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:08.726323+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e290: 8 total, 8 up, 7 in 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:08.731694+0000 mon.b (mon.1) 364 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:08.731734+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.59:0/4033699682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:09.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:08.732220+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:08.739744+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.410872+0000 osd.4 (osd.4) 177 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.412128+0000 osd.4 (osd.4) 178 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.416858+0000 osd.4 (osd.4) 175 : cluster [DBG] 62.d scrub starts 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.420337+0000 osd.4 (osd.4) 176 : cluster [DBG] 62.d scrub ok 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.455388+0000 osd.0 (osd.0) 141 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.457422+0000 osd.0 (osd.0) 142 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.651894+0000 mon.a (mon.0) 2000 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.683923+0000 osd.1 (osd.1) 151 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:36:09.953 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.716769+0000 osd.1 (osd.1) 152 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:08.726063+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:08.726215+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:08.726323+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e290: 8 total, 8 up, 7 in 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:08.731694+0000 mon.b (mon.1) 364 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:08.731734+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.59:0/4033699682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:08.732220+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:08.739744+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.410872+0000 osd.4 (osd.4) 177 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:36:09.954 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.412128+0000 osd.4 (osd.4) 178 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:36:09.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.416858+0000 osd.4 (osd.4) 175 : cluster [DBG] 62.d scrub starts 2024-02-18T00:36:09.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.420337+0000 osd.4 (osd.4) 176 : cluster [DBG] 62.d scrub ok 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.455388+0000 osd.0 (osd.0) 141 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.457422+0000 osd.0 (osd.0) 142 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.651894+0000 mon.a (mon.0) 2000 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.683923+0000 osd.1 (osd.1) 151 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.716769+0000 osd.1 (osd.1) 152 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:08.726063+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-33"}]': finished 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:08.726215+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi059-82877-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:08.726323+0000 mon.a (mon.0) 2003 : cluster [DBG] osdmap e290: 8 total, 8 up, 7 in 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:08.731694+0000 mon.b (mon.1) 364 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:08.731734+0000 mon.c (mon.2) 254 : audit [INF] from='client.? 172.21.15.59:0/4033699682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:08.732220+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:08.739744+0000 mon.a (mon.0) 2005 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:09.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.410872+0000 osd.4 (osd.4) 177 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:36:09.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.412128+0000 osd.4 (osd.4) 178 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:36:10.040 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:10.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:10.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:10.144 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:10.173 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:10.212 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:10.214 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:10.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:10.247 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:36:10.247 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:36:10.316 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:10.344 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:10.492 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:10.494 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:10.495 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:10.595 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:10.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.438556+0000 osd.0 (osd.0) 143 : cluster [DBG] 62.e scrub starts 2024-02-18T00:36:10.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.440366+0000 osd.0 (osd.0) 144 : cluster [DBG] 62.e scrub ok 2024-02-18T00:36:10.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.528223+0000 osd.6 (osd.6) 147 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:36:10.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.529897+0000 osd.6 (osd.6) 148 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.646981+0000 osd.1 (osd.1) 153 : cluster [DBG] 2.6 deep-scrub starts 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.647924+0000 osd.1 (osd.1) 154 : cluster [DBG] 2.6 deep-scrub ok 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.720822+0000 osd.2 (osd.2) 127 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.722330+0000 osd.2 (osd.2) 128 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:09.730592+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:09.861785+0000 mgr.y (mgr.24454) 302 : cluster [DBG] pgmap v424: 238 pgs: 32 unknown, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.170394+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e291: 8 total, 8 up, 7 in 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.171638+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.175891+0000 mon.a (mon.0) 2008 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.177485+0000 mon.a (mon.0) 2009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.381645+0000 osd.4 (osd.4) 179 : cluster [DBG] 62.5 scrub starts 2024-02-18T00:36:10.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.385122+0000 osd.4 (osd.4) 180 : cluster [DBG] 62.5 scrub ok 2024-02-18T00:36:10.925 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:10.926 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-18T00:36:10.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:10.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:10.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.438556+0000 osd.0 (osd.0) 143 : cluster [DBG] 62.e scrub starts 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.440366+0000 osd.0 (osd.0) 144 : cluster [DBG] 62.e scrub ok 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.528223+0000 osd.6 (osd.6) 147 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.529897+0000 osd.6 (osd.6) 148 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.646981+0000 osd.1 (osd.1) 153 : cluster [DBG] 2.6 deep-scrub starts 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.647924+0000 osd.1 (osd.1) 154 : cluster [DBG] 2.6 deep-scrub ok 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.720822+0000 osd.2 (osd.2) 127 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.722330+0000 osd.2 (osd.2) 128 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:09.730592+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:09.861785+0000 mgr.y (mgr.24454) 302 : cluster [DBG] pgmap v424: 238 pgs: 32 unknown, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.170394+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e291: 8 total, 8 up, 7 in 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.171638+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.175891+0000 mon.a (mon.0) 2008 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.177485+0000 mon.a (mon.0) 2009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.381645+0000 osd.4 (osd.4) 179 : cluster [DBG] 62.5 scrub starts 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.385122+0000 osd.4 (osd.4) 180 : cluster [DBG] 62.5 scrub ok 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.438556+0000 osd.0 (osd.0) 143 : cluster [DBG] 62.e scrub starts 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.440366+0000 osd.0 (osd.0) 144 : cluster [DBG] 62.e scrub ok 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.528223+0000 osd.6 (osd.6) 147 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.529897+0000 osd.6 (osd.6) 148 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.646981+0000 osd.1 (osd.1) 153 : cluster [DBG] 2.6 deep-scrub starts 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.647924+0000 osd.1 (osd.1) 154 : cluster [DBG] 2.6 deep-scrub ok 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.720822+0000 osd.2 (osd.2) 127 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:36:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.722330+0000 osd.2 (osd.2) 128 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:09.730592+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi059-82882-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:09.861785+0000 mgr.y (mgr.24454) 302 : cluster [DBG] pgmap v424: 238 pgs: 32 unknown, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.170394+0000 mon.a (mon.0) 2007 : cluster [DBG] osdmap e291: 8 total, 8 up, 7 in 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.171638+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.175891+0000 mon.a (mon.0) 2008 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.177485+0000 mon.a (mon.0) 2009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.381645+0000 osd.4 (osd.4) 179 : cluster [DBG] 62.5 scrub starts 2024-02-18T00:36:11.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.385122+0000 osd.4 (osd.4) 180 : cluster [DBG] 62.5 scrub ok 2024-02-18T00:36:11.029 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:11.055 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:11.074 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:11.077 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:11.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:11.178 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:11.206 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:36:11.206 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:36:11.208 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:11.228 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:11.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:11.231 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:11.242 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:11.332 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:11.360 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:11.508 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:11.512 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:11.513 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:11.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:36:11.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:11.642 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 -- 172.21.15.59:0/2949448932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fcac0 msgr2=0x7fe65c0fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 --2- 172.21.15.59:0/2949448932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fcac0 0x7fe65c0fcea0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 rx=0x7fe64c004870 tx=0x7fe64c009cb0).stop 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 -- 172.21.15.59:0/2949448932 shutdown_connections 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 --2- 172.21.15.59:0/2949448932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c0f7330 0x7fe65c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 --2- 172.21.15.59:0/2949448932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fcac0 0x7fe65c0fcea0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 --2- 172.21.15.59:0/2949448932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe65c0fd420 0x7fe65c0f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 -- 172.21.15.59:0/2949448932 >> 172.21.15.59:0/2949448932 conn(0x7fe65c0f2710 msgr2=0x7fe65c0f4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:11.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.835+0000 7fe663ae4700 1 -- 172.21.15.59:0/2949448932 shutdown_connections 2024-02-18T00:36:11.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.836+0000 7fe663ae4700 1 -- 172.21.15.59:0/2949448932 wait complete. 2024-02-18T00:36:11.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.836+0000 7fe663ae4700 1 librados: starting msgr at 2024-02-18T00:36:11.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.836+0000 7fe663ae4700 1 librados: starting objecter 2024-02-18T00:36:11.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.836+0000 7fe663ae4700 1 Processor -- start 2024-02-18T00:36:11.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.836+0000 7fe663ae4700 1 -- start start 2024-02-18T00:36:11.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.836+0000 7fe663ae4700 1 librados: setting wanted keys 2024-02-18T00:36:11.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.836+0000 7fe663ae4700 1 librados: calling monclient init 2024-02-18T00:36:11.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe663ae4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe65c0f7330 0x7fe65c10d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:11.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe663ae4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c0fcac0 0x7fe65c10d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:11.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe663ae4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fd420 0x7fe65c1071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:11.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe663ae4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c1076e0 con 0x7fe65c0fcac0 2024-02-18T00:36:11.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe663ae4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c107850 con 0x7fe65c0f7330 2024-02-18T00:36:11.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe663ae4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c107a10 con 0x7fe65c0fd420 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe66107f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c0fcac0 0x7fe65c10d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe661880700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe65c0f7330 0x7fe65c10d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe66107f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c0fcac0 0x7fe65c10d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54286/0 (socket says 172.21.15.59:54286) 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe66107f700 1 -- 172.21.15.59:0/108331976 learned_addr learned my addr 172.21.15.59:0/108331976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe662081700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fd420 0x7fe65c1071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe661880700 1 -- 172.21.15.59:0/108331976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fd420 msgr2=0x7fe65c1071f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe661880700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fd420 0x7fe65c1071f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe661880700 1 -- 172.21.15.59:0/108331976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c0fcac0 msgr2=0x7fe65c10d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe661880700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c0fcac0 0x7fe65c10d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe661880700 1 -- 172.21.15.59:0/108331976 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64c005040 con 0x7fe65c0f7330 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe661880700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe65c0f7330 0x7fe65c10d050 secure :-1 s=READY pgs=4060 cs=0 l=1 rev1=1 rx=0x7fe64c004870 tx=0x7fe64c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe652f78700 10 client.?.objecter ms_handle_connect 0x7fe65c0f7330 2024-02-18T00:36:11.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe652f78700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe662081700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fd420 0x7fe65c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe652f78700 1 -- 172.21.15.59:0/108331976 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00e070 con 0x7fe65c0f7330 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c107c90 con 0x7fe65c0f7330 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe652f78700 1 -- 172.21.15.59:0/108331976 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe64c013da0 con 0x7fe65c0f7330 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.837+0000 7fe652f78700 1 -- 172.21.15.59:0/108331976 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c019070 con 0x7fe65c0f7330 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.838+0000 7fe663ae4700 10 client.43363.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.838+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c108040 con 0x7fe65c0f7330 2024-02-18T00:36:11.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.838+0000 7fe663ae4700 1 librados: init done 2024-02-18T00:36:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.839+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c0606d0 con 0x7fe65c0f7330 2024-02-18T00:36:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.839+0000 7fe652f78700 1 -- 172.21.15.59:0/108331976 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe64c0090d0 con 0x7fe65c0f7330 2024-02-18T00:36:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.840+0000 7fe652f78700 1 --2- 172.21.15.59:0/108331976 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe648062c90 0x7fe648065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.840+0000 7fe652f78700 1 -- 172.21.15.59:0/108331976 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(293..293 src has 44..293) v4 ==== 9144+0+0 (secure 0 0 0) 0x7fe64c07dc80 con 0x7fe65c0f7330 2024-02-18T00:36:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.840+0000 7fe652f78700 10 client.43363.objecter ms_dispatch 0x7fe65c001550 osd_map(293..293 src has 44..293) v4 2024-02-18T00:36:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.840+0000 7fe652f78700 3 client.43363.objecter handle_osd_map got epochs [293,293] > 0 2024-02-18T00:36:11.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.840+0000 7fe652f78700 3 client.43363.objecter handle_osd_map decoding full epoch 293 2024-02-18T00:36:11.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.840+0000 7fe652f78700 20 client.43363.objecter dump_active .. 0 homeless 2024-02-18T00:36:11.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.841+0000 7fe66107f700 1 --2- 172.21.15.59:0/108331976 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe648062c90 0x7fe648065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:11.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.841+0000 7fe66107f700 1 --2- 172.21.15.59:0/108331976 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe648062c90 0x7fe648065150 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 rx=0x7fe65c108990 tx=0x7fe65800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:11.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.841+0000 7fe652f78700 10 client.43363.objecter ms_handle_connect 0x7fe648062c90 2024-02-18T00:36:11.844 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:11.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:11.844+0000 7fe652f78700 1 -- 172.21.15.59:0/108331976 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe64c050d80 con 0x7fe65c0f7330 2024-02-18T00:36:11.847 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:11.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:11.918 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:11.948 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.524396+0000 osd.6 (osd.6) 149 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.525296+0000 osd.6 (osd.6) 150 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.737891+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.738027+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.738119+0000 mon.a (mon.0) 2012 : cluster [DBG] osdmap e292: 8 total, 8 up, 7 in 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.739238+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.739545+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.739664+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.740131+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.769451+0000 osd.3 (osd.3) 167 : cluster [DBG] 62.18 scrub starts 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:10.770633+0000 osd.3 (osd.3) 168 : cluster [DBG] 62.18 scrub ok 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.774840+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.776075+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:10.776379+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:11.119907+0000 mgr.y (mgr.24454) 303 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.524396+0000 osd.6 (osd.6) 149 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.525296+0000 osd.6 (osd.6) 150 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.737891+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.738027+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:12.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.738119+0000 mon.a (mon.0) 2012 : cluster [DBG] osdmap e292: 8 total, 8 up, 7 in 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.739238+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.739545+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.739664+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.740131+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.769451+0000 osd.3 (osd.3) 167 : cluster [DBG] 62.18 scrub starts 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:10.770633+0000 osd.3 (osd.3) 168 : cluster [DBG] 62.18 scrub ok 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.774840+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.776075+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:10.776379+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:12.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:11.119907+0000 mgr.y (mgr.24454) 303 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.524396+0000 osd.6 (osd.6) 149 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.525296+0000 osd.6 (osd.6) 150 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.737891+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi059-82877-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.738027+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.738119+0000 mon.a (mon.0) 2012 : cluster [DBG] osdmap e292: 8 total, 8 up, 7 in 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.739238+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.59:0/2880867683' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.739545+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.739664+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]: dispatch 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.740131+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.769451+0000 osd.3 (osd.3) 167 : cluster [DBG] 62.18 scrub starts 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:10.770633+0000 osd.3 (osd.3) 168 : cluster [DBG] 62.18 scrub ok 2024-02-18T00:36:12.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.774840+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:12.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.776075+0000 mon.a (mon.0) 2016 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:12.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:10.776379+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:12.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:11.119907+0000 mgr.y (mgr.24454) 303 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:12.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:12.047+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fe65c05ee80 con 0x7fe65c0f7330 2024-02-18T00:36:12.051 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:12.055 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:12.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:12.157 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:12.184 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:12.383 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:12.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:12.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:12.486 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:12.497 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:12] "GET /metrics HTTP/1.1" 200 188512 "" "Prometheus/2.33.4" 2024-02-18T00:36:12.516 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:12.909 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:36:12.909 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.345266+0000 osd.0 (osd.0) 145 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.346191+0000 osd.0 (osd.0) 146 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.550748+0000 osd.6 (osd.6) 151 : cluster [DBG] 2.17 deep-scrub starts 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.551593+0000 osd.6 (osd.6) 152 : cluster [DBG] 2.17 deep-scrub ok 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.596716+0000 osd.0 (osd.0) 147 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.596872+0000 osd.0 (osd.0) 148 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:11.743318+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:12.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:11.743402+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:11.743598+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.743654+0000 mon.a (mon.0) 2021 : cluster [DBG] osdmap e293: 8 total, 8 up, 7 in 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:11.746707+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:11.747244+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:11.748894+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi059-82882-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.807398+0000 osd.3 (osd.3) 169 : cluster [DBG] 62.b scrub starts 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.809917+0000 osd.3 (osd.3) 170 : cluster [DBG] 62.b scrub ok 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.862406+0000 mgr.y (mgr.24454) 304 : cluster [DBG] pgmap v427: 246 pgs: 40 creating+peering, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.869865+0000 osd.3 (osd.3) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:11.870052+0000 osd.3 (osd.3) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:12.048630+0000 mon.b (mon.1) 365 : audit [INF] from='client.? 172.21.15.59:0/108331976' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:36:12.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:12.245755+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:36:13.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.194+0000 7fe652f78700 1 -- 172.21.15.59:0/108331976 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v294) v1 ==== 87+0+0 (secure 0 0 0) 0x7fe64c023030 con 0x7fe65c0f7330 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.197+0000 7fe663ae4700 10 librados: watch_flush enter 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 10 librados: watch_flush exit 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 20 client.43363.objecter shutdown clearing up homeless session... 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 10 client.43363.objecter successfully canceled tick 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe648062c90 msgr2=0x7fe648065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 --2- 172.21.15.59:0/108331976 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe648062c90 0x7fe648065150 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 rx=0x7fe65c108990 tx=0x7fe65800b040).stop 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe65c0f7330 msgr2=0x7fe65c10d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:13.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe65c0f7330 0x7fe65c10d050 secure :-1 s=READY pgs=4060 cs=0 l=1 rev1=1 rx=0x7fe64c004870 tx=0x7fe64c00b040).stop 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 shutdown_connections 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 --2- 172.21.15.59:0/108331976 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe648062c90 0x7fe648065150 unknown :-1 s=CLOSED pgs=3057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe65c0f7330 0x7fe65c10d050 unknown :-1 s=CLOSED pgs=4060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe65c0fd420 0x7fe65c1071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 --2- 172.21.15.59:0/108331976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe65c0fcac0 0x7fe65c10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 >> 172.21.15.59:0/108331976 conn(0x7fe65c0f2710 msgr2=0x7fe65c0f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 shutdown_connections 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 -- 172.21.15.59:0/108331976 wait complete. 2024-02-18T00:36:13.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:13.198+0000 7fe663ae4700 1 librados: shutdown 2024-02-18T00:36:13.204 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:36:13.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.345266+0000 osd.0 (osd.0) 145 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:36:13.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.346191+0000 osd.0 (osd.0) 146 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.550748+0000 osd.6 (osd.6) 151 : cluster [DBG] 2.17 deep-scrub starts 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.551593+0000 osd.6 (osd.6) 152 : cluster [DBG] 2.17 deep-scrub ok 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.596716+0000 osd.0 (osd.0) 147 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.596872+0000 osd.0 (osd.0) 148 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:11.743318+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:11.743402+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:11.743598+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.743654+0000 mon.a (mon.0) 2021 : cluster [DBG] osdmap e293: 8 total, 8 up, 7 in 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:11.746707+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:11.747244+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:13.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:11.748894+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi059-82882-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:13.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.807398+0000 osd.3 (osd.3) 169 : cluster [DBG] 62.b scrub starts 2024-02-18T00:36:13.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.809917+0000 osd.3 (osd.3) 170 : cluster [DBG] 62.b scrub ok 2024-02-18T00:36:13.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.862406+0000 mgr.y (mgr.24454) 304 : cluster [DBG] pgmap v427: 246 pgs: 40 creating+peering, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T00:36:13.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.869865+0000 osd.3 (osd.3) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:13.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:11.870052+0000 osd.3 (osd.3) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:13.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:12.048630+0000 mon.b (mon.1) 365 : audit [INF] from='client.? 172.21.15.59:0/108331976' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:36:13.229 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:12.245755+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:36:13.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.345266+0000 osd.0 (osd.0) 145 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:36:13.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.346191+0000 osd.0 (osd.0) 146 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:36:13.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.550748+0000 osd.6 (osd.6) 151 : cluster [DBG] 2.17 deep-scrub starts 2024-02-18T00:36:13.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.551593+0000 osd.6 (osd.6) 152 : cluster [DBG] 2.17 deep-scrub ok 2024-02-18T00:36:13.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.596716+0000 osd.0 (osd.0) 147 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:13.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.596872+0000 osd.0 (osd.0) 148 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:13.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:11.743318+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi059-82917-2"}]': finished 2024-02-18T00:36:13.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:11.743402+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:11.743598+0000 mon.a (mon.0) 2020 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.743654+0000 mon.a (mon.0) 2021 : cluster [DBG] osdmap e293: 8 total, 8 up, 7 in 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:11.746707+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:11.747244+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:11.748894+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi059-82882-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.807398+0000 osd.3 (osd.3) 169 : cluster [DBG] 62.b scrub starts 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.809917+0000 osd.3 (osd.3) 170 : cluster [DBG] 62.b scrub ok 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.862406+0000 mgr.y (mgr.24454) 304 : cluster [DBG] pgmap v427: 246 pgs: 40 creating+peering, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.869865+0000 osd.3 (osd.3) 171 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:13.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:11.870052+0000 osd.3 (osd.3) 172 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:13.232 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:12.048630+0000 mon.b (mon.1) 365 : audit [INF] from='client.? 172.21.15.59:0/108331976' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:36:13.232 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:12.245755+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:36:13.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-18T00:36:13.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:12.517872+0000 osd.6 (osd.6) 153 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:36:13.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:12.518835+0000 osd.6 (osd.6) 154 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:36:13.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:12.740668+0000 mon.a (mon.0) 2025 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:13.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:12.757379+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:12.762566+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.193125+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:13.193234+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e294: 8 total, 8 up, 7 in 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.194801+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.59:0/549447909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.195264+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.195729+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.196416+0000 mon.b (mon.1) 366 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.196615+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.633474+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi059-82882-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.633642+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]': finished 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.633712+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:13.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.633827+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:12.517872+0000 osd.6 (osd.6) 153 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:12.518835+0000 osd.6 (osd.6) 154 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:12.740668+0000 mon.a (mon.0) 2025 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:12.757379+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:12.762566+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.193125+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:13.193234+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e294: 8 total, 8 up, 7 in 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.194801+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.59:0/549447909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.195264+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:13.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.195729+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.196416+0000 mon.b (mon.1) 366 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.196615+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.633474+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi059-82882-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.633642+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]': finished 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.633712+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.633827+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:13.633884+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e295: 8 total, 8 up, 7 in 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.636096+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]: dispatch 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.636191+0000 mon.b (mon.1) 367 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.638011+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]: dispatch 2024-02-18T00:36:13.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.638323+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:13.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:13.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:13.873 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:13.974 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:14.000 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:12.517872+0000 osd.6 (osd.6) 153 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:36:14.000 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:12.518835+0000 osd.6 (osd.6) 154 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:36:14.000 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:12.740668+0000 mon.a (mon.0) 2025 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:14.000 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:12.757379+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:14.000 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:12.762566+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:14.000 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.193125+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:13.193234+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e294: 8 total, 8 up, 7 in 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.194801+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.59:0/549447909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.195264+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.195729+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.196416+0000 mon.b (mon.1) 366 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.196615+0000 mon.a (mon.0) 2031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.633474+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi059-82882-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.633642+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-35"}]': finished 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.633712+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82917-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.633827+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:13.633884+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e295: 8 total, 8 up, 7 in 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.636096+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]: dispatch 2024-02-18T00:36:14.001 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.636191+0000 mon.b (mon.1) 367 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:14.002 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.638011+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]: dispatch 2024-02-18T00:36:14.002 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.638323+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:14.002 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.659451+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.59:0/549447909' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T00:36:14.002 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:13.659942+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T00:36:14.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.659451+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.59:0/549447909' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:13.659942+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T00:36:14.003 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:13.633884+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e295: 8 total, 8 up, 7 in 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.636096+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]: dispatch 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.636191+0000 mon.b (mon.1) 367 : audit [INF] from='client.? 172.21.15.59:0/593184153' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.638011+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]: dispatch 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.638323+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]: dispatch 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.659451+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.59:0/549447909' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T00:36:14.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:13.659942+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T00:36:14.125 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:14.127 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:14.127 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:14.228 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:14.255 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:14.352 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:14.354 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:14.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:14.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:14.512 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:14.514 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:14.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:14.615 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:14.649 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-18T00:36:14.649 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 1 0 2024-02-18T00:36:14.649 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 10 0 2024-02-18T00:36:14.649 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 13 0 2024-02-18T00:36:14.649 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 7 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 14 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 0 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 15 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 11 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 5 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 8 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 6 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 3 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 4 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 12 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 9 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 2 0 2024-02-18T00:36:14.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (45 ms) 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] 3 tests from LibRadosListEC (1170 ms total) 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2888 ms) 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] 1 test from LibRadosListNP (2888 ms total) 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [----------] Global test environment tear-down 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [==========] 11 tests from 3 test suites ran. (203720 ms total) 2024-02-18T00:36:14.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_list: [ PASSED ] 11 tests. 2024-02-18T00:36:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83000 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83000 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82992 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82992 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83056 2024-02-18T00:36:14.653 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83056 2024-02-18T00:36:14.654 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:36:14.654 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82880 2024-02-18T00:36:14.654 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82880 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout:ete 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6943 ms) 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6990 ms) 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7031 ms) 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7006 ms) 2024-02-18T00:36:14.656 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (6978 ms) 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7469 ms) 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6534 ms) 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7012 ms) 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7067 ms) 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6927 ms) 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7005 ms) 2024-02-18T00:36:14.657 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6997 ms) 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7021 ms) 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7010 ms) 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6994 ms) 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111665 ms total) 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [----------] Global test environment tear-down 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (203762 ms total) 2024-02-18T00:36:14.658 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio: [ PASSED ] 41 tests. 2024-02-18T00:36:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:14.676 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:14.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:14.777 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:14.803 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:14.875 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:14.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:14.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:14.984 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:15.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:13.862867+0000 mgr.y (mgr.24454) 305 : cluster [DBG] pgmap v430: 278 pgs: 40 unknown, 32 creating+peering, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:15.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:14.627902+0000 mon.a (mon.0) 2040 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:14.632514+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]': finished 2024-02-18T00:36:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:14.632598+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:14.632753+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-18T00:36:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:14.632837+0000 mon.a (mon.0) 2044 : cluster [DBG] osdmap e296: 8 total, 8 up, 7 in 2024-02-18T00:36:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:14.682826+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:14.683130+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:15.011 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:13.862867+0000 mgr.y (mgr.24454) 305 : cluster [DBG] pgmap v430: 278 pgs: 40 unknown, 32 creating+peering, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:14.627902+0000 mon.a (mon.0) 2040 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:14.632514+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]': finished 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:14.632598+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:14.632753+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:14.632837+0000 mon.a (mon.0) 2044 : cluster [DBG] osdmap e296: 8 total, 8 up, 7 in 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:14.682826+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:15.060 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:14.683130+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:13.862867+0000 mgr.y (mgr.24454) 305 : cluster [DBG] pgmap v430: 278 pgs: 40 unknown, 32 creating+peering, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:14.627902+0000 mon.a (mon.0) 2040 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:14.632514+0000 mon.a (mon.0) 2041 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-35", "mode": "writeback"}]': finished 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:14.632598+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi059-82877-41"}]': finished 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:14.632753+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi059-82917-3","pool2":"test-rados-api-smithi059-82917-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:14.632837+0000 mon.a (mon.0) 2044 : cluster [DBG] osdmap e296: 8 total, 8 up, 7 in 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:14.682826+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:15.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:14.683130+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:15.208 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:15.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:15.211 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:15.312 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:15.346 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 -- 172.21.15.59:0/1247783371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800f7400 msgr2=0x7fac80103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 --2- 172.21.15.59:0/1247783371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800f7400 0x7fac80103a20 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 rx=0x7fac78004870 tx=0x7fac78009cb0).stop 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 -- 172.21.15.59:0/1247783371 shutdown_connections 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 --2- 172.21.15.59:0/1247783371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800f7400 0x7fac80103a20 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 --2- 172.21.15.59:0/1247783371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac800fcbb0 0x7fac800fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 --2- 172.21.15.59:0/1247783371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac800fd510 0x7fac800f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 -- 172.21.15.59:0/1247783371 >> 172.21.15.59:0/1247783371 conn(0x7fac800f2820 msgr2=0x7fac800f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 -- 172.21.15.59:0/1247783371 shutdown_connections 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.551+0000 7fac87726700 1 -- 172.21.15.59:0/1247783371 wait complete. 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 librados: starting msgr at 2024-02-18T00:36:15.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 librados: starting objecter 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 Processor -- start 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 -- start start 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 librados: setting wanted keys 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 librados: calling monclient init 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac800f7400 0x7fac8010dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800fcbb0 0x7fac8010e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac800fd510 0x7fac80108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac80108570 con 0x7fac800fd510 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac801086e0 con 0x7fac800f7400 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac87726700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac801088a0 con 0x7fac800fcbb0 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800fcbb0 0x7fac8010e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac85cc3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac800fd510 0x7fac80108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:15.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800fcbb0 0x7fac8010e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60572/0 (socket says 172.21.15.59:60572) 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 -- 172.21.15.59:0/1842996496 learned_addr learned my addr 172.21.15.59:0/1842996496 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac854c2700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac800f7400 0x7fac8010dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 -- 172.21.15.59:0/1842996496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac800f7400 msgr2=0x7fac8010dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac800f7400 0x7fac8010dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 -- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac800fd510 msgr2=0x7fac80108080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac800fd510 0x7fac80108080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac84cc1700 1 -- 172.21.15.59:0/1842996496 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac700097c0 con 0x7fac800fcbb0 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.552+0000 7fac85cc3700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac800fd510 0x7fac80108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac84cc1700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800fcbb0 0x7fac8010e3d0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 rx=0x7fac740027f0 tx=0x7fac74002b00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac7e7fc700 10 client.?.objecter ms_handle_connect 0x7fac800fcbb0 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac7e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac7e7fc700 1 -- 172.21.15.59:0/1842996496 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74007b40 con 0x7fac800fcbb0 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac78005040 con 0x7fac800fcbb0 2024-02-18T00:36:15.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac7e7fc700 1 -- 172.21.15.59:0/1842996496 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac7400f040 con 0x7fac800fcbb0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac7e7fc700 1 -- 172.21.15.59:0/1842996496 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7400d7e0 con 0x7fac800fcbb0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac87726700 10 client.40838.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac80114d10 con 0x7fac800fcbb0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.553+0000 7fac87726700 1 librados: init done 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.554+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8005fa60 con 0x7fac800fcbb0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.554+0000 7fac7e7fc700 1 -- 172.21.15.59:0/1842996496 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fac7400dc10 con 0x7fac800fcbb0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac7e7fc700 1 --2- 172.21.15.59:0/1842996496 >> v2:172.21.15.59:6800/3721891797 conn(0x7fac68062c90 0x7fac68065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac7e7fc700 1 -- 172.21.15.59:0/1842996496 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(296..296 src has 44..296) v4 ==== 8871+0+0 (secure 0 0 0) 0x7fac7407d370 con 0x7fac800fcbb0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac7e7fc700 10 client.40838.objecter ms_dispatch 0x7fac80001550 osd_map(296..296 src has 44..296) v4 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac7e7fc700 3 client.40838.objecter handle_osd_map got epochs [296,296] > 0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac7e7fc700 3 client.40838.objecter handle_osd_map decoding full epoch 296 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac854c2700 1 --2- 172.21.15.59:0/1842996496 >> v2:172.21.15.59:6800/3721891797 conn(0x7fac68062c90 0x7fac68065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac7e7fc700 20 client.40838.objecter dump_active .. 0 homeless 2024-02-18T00:36:15.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac7e7fc700 10 client.40838.objecter ms_handle_connect 0x7fac68062c90 2024-02-18T00:36:15.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.555+0000 7fac854c2700 1 --2- 172.21.15.59:0/1842996496 >> v2:172.21.15.59:6800/3721891797 conn(0x7fac68062c90 0x7fac68065150 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 rx=0x7fac70001450 tx=0x7fac7000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:15.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.559+0000 7fac7e7fc700 1 -- 172.21.15.59:0/1842996496 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fac740504b0 con 0x7fac800fcbb0 2024-02-18T00:36:15.597 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:15.601 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:15.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:15.702 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:15.719 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:36:15.719 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:36:15.733 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:15.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.749+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac801060e0 con 0x7fac68062c90 2024-02-18T00:36:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.767+0000 7fac7e7fc700 1 -- 172.21.15.59:0/1842996496 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+597561 (secure 0 0 0) 0x7fac801060e0 con 0x7fac68062c90 2024-02-18T00:36:15.776 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 10 librados: watch_flush enter 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 10 librados: watch_flush exit 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 20 client.40838.objecter shutdown clearing up homeless session... 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 10 client.40838.objecter successfully canceled tick 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 >> v2:172.21.15.59:6800/3721891797 conn(0x7fac68062c90 msgr2=0x7fac68065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 --2- 172.21.15.59:0/1842996496 >> v2:172.21.15.59:6800/3721891797 conn(0x7fac68062c90 0x7fac68065150 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 rx=0x7fac70001450 tx=0x7fac7000b040).stop 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800fcbb0 msgr2=0x7fac8010e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:15.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800fcbb0 0x7fac8010e3d0 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 rx=0x7fac740027f0 tx=0x7fac74002b00).stop 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 shutdown_connections 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 --2- 172.21.15.59:0/1842996496 >> v2:172.21.15.59:6800/3721891797 conn(0x7fac68062c90 0x7fac68065150 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac800f7400 0x7fac8010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac800fd510 0x7fac80108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 --2- 172.21.15.59:0/1842996496 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fac800fcbb0 0x7fac8010e3d0 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 >> 172.21.15.59:0/1842996496 conn(0x7fac800f2820 msgr2=0x7fac80060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 shutdown_connections 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 -- 172.21.15.59:0/1842996496 wait complete. 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:15.771+0000 7fac87726700 1 librados: shutdown 2024-02-18T00:36:15.783 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:36:16.266 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:16.273 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:16.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:16.358 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-18T00:36:16.374 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:16.429 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:16.450 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:16.452 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:16.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:16.553 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:16.601 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:16.603 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:16.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:16.632 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:16.705 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:16.732 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:16.784 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:16.789 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:16.790 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:16.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:15.633665+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:16.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:15.633731+0000 mon.a (mon.0) 2047 : cluster [DBG] osdmap e297: 8 total, 8 up, 7 in 2024-02-18T00:36:16.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:15.635164+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:16.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:15.639076+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:16.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:15.639370+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:16.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:15.750983+0000 mgr.y (mgr.24454) 306 : audit [DBG] from='client.40838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:16.884 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:15.863880+0000 mgr.y (mgr.24454) 307 : cluster [DBG] pgmap v433: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-18T00:36:16.891 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:16.920 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:16.939 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:16.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:16.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:15.633665+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:15.633731+0000 mon.a (mon.0) 2047 : cluster [DBG] osdmap e297: 8 total, 8 up, 7 in 2024-02-18T00:36:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:15.635164+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:15.639076+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:15.639370+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:15.750983+0000 mgr.y (mgr.24454) 306 : audit [DBG] from='client.40838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:15.863880+0000 mgr.y (mgr.24454) 307 : cluster [DBG] pgmap v433: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-18T00:36:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:15.633665+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:15.633731+0000 mon.a (mon.0) 2047 : cluster [DBG] osdmap e297: 8 total, 8 up, 7 in 2024-02-18T00:36:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:15.635164+0000 mon.a (mon.0) 2048 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:15.639076+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:15.639370+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]: dispatch 2024-02-18T00:36:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:15.750983+0000 mgr.y (mgr.24454) 306 : audit [DBG] from='client.40838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:15.863880+0000 mgr.y (mgr.24454) 307 : cluster [DBG] pgmap v433: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-18T00:36:17.043 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:17.069 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:17.083 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:36:17.106 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:17.109 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:17.109 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:17.148 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-02-18T00:36:15.631195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7005,"num_objects":346,"num_object_clones":2,"num_object_copies":1038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":346,"num_whiteouts":0,"num_read":1607,"num_read_kb":1235,"num_write":1168,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2512,"ondisk_log_size":2512,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":751,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2167128,"kb_used_data":33248,"kb_used_omap":0,"kb_used_meta":2133760,"kb_avail":654048936,"statfs":{"total":671965249536,"available":669746110464,"internally_reserved":0,"allocated":34045952,"data_stored":18765326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2184970240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.890400"},"pg_stats":[{"pgid":"183.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.737065+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"183.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.738889+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.745935+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"183.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.742946+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"183.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.744035+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"183.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.739327+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.745587+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"183.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.740400+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.736468+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"183.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.738494+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.735553+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"183.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.743370+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"183.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.742474+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"183.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.738679+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.743980+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"183.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.742097+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"183.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.749886+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"183.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.743601+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"183.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.739034+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.744409+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"183.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.740785+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.745563+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"183.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.750914+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"183.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.750157+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"183.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.739597+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.747212+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"183.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.748246+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"183.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.745522+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"183.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.743933+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"183.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.736062+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"183.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.741376+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"183.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:10.727571+0000","last_change":"2024-02-18T00:36:10.743382+0000","last_active":"2024-02-18T00:36:10.727571+0000","last_peered":"2024-02-18T00:36:10.727571+0000","last_clean":"2024-02-18T00:36:10.727571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:10.727571+0000","last_undegraded":"2024-02-18T00:36:10.727571+0000","last_fullsized":"2024-02-18T00:36:10.727571+0000","mapping_epoch":292,"log_start":"0'0","ondisk_log_start":"0'0","created":292,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:10.727571+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.727571+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":262,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731179+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:36:09.731179+0000","last_peered":"2024-02-18T00:36:09.731179+0000","last_clean":"2024-02-18T00:36:09.731179+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:36:09.731179+0000","last_undegraded":"2024-02-18T00:36:09.731179+0000","last_fullsized":"2024-02-18T00:36:09.731179+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.1","version":"0'0","reported_seq":224,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.743590+0000","last_change":"2024-02-18T00:36:06.658623+0000","last_active":"2024-02-18T00:36:09.743590+0000","last_peered":"2024-02-18T00:36:09.743590+0000","last_clean":"2024-02-18T00:36:09.743590+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:36:09.743590+0000","last_undegraded":"2024-02-18T00:36:09.743590+0000","last_fullsized":"2024-02-18T00:36:09.743590+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.2","version":"231'68","reported_seq":292,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.745278+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:36:09.745278+0000","last_peered":"2024-02-18T00:36:09.745278+0000","last_clean":"2024-02-18T00:36:09.745278+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:36:09.745278+0000","last_undegraded":"2024-02-18T00:36:09.745278+0000","last_fullsized":"2024-02-18T00:36:09.745278+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.3","version":"245'73","reported_seq":286,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731342+0000","last_change":"2024-02-18T00:36:06.648070+0000","last_active":"2024-02-18T00:36:09.731342+0000","last_peered":"2024-02-18T00:36:09.731342+0000","last_clean":"2024-02-18T00:36:09.731342+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:36:09.731342+0000","last_undegraded":"2024-02-18T00:36:09.731342+0000","last_fullsized":"2024-02-18T00:36:09.731342+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.4","version":"230'120","reported_seq":326,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741486+0000","last_change":"2024-02-18T00:36:06.647751+0000","last_active":"2024-02-18T00:36:09.741486+0000","last_peered":"2024-02-18T00:36:09.741486+0000","last_clean":"2024-02-18T00:36:09.741486+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:36:09.741486+0000","last_undegraded":"2024-02-18T00:36:09.741486+0000","last_fullsized":"2024-02-18T00:36:09.741486+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.5","version":"240'90","reported_seq":301,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:10.385220+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:36:10.385220+0000","last_peered":"2024-02-18T00:36:10.385220+0000","last_clean":"2024-02-18T00:36:10.385220+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:36:10.385220+0000","last_undegraded":"2024-02-18T00:36:10.385220+0000","last_fullsized":"2024-02-18T00:36:10.385220+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":90,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.6","version":"267'124","reported_seq":299,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.734539+0000","last_change":"2024-02-18T00:36:06.654594+0000","last_active":"2024-02-18T00:36:09.734539+0000","last_peered":"2024-02-18T00:36:09.734539+0000","last_clean":"2024-02-18T00:36:09.734539+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:36:09.734539+0000","last_undegraded":"2024-02-18T00:36:09.734539+0000","last_fullsized":"2024-02-18T00:36:09.734539+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":124,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.7","version":"267'73","reported_seq":280,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732377+0000","last_change":"2024-02-18T00:36:06.651545+0000","last_active":"2024-02-18T00:36:09.732377+0000","last_peered":"2024-02-18T00:36:09.732377+0000","last_clean":"2024-02-18T00:36:09.732377+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:36:09.732377+0000","last_undegraded":"2024-02-18T00:36:09.732377+0000","last_fullsized":"2024-02-18T00:36:09.732377+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.8","version":"215'68","reported_seq":251,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.740574+0000","last_change":"2024-02-18T00:36:06.662037+0000","last_active":"2024-02-18T00:36:09.740574+0000","last_peered":"2024-02-18T00:36:09.740574+0000","last_clean":"2024-02-18T00:36:09.740574+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:36:09.740574+0000","last_undegraded":"2024-02-18T00:36:09.740574+0000","last_fullsized":"2024-02-18T00:36:09.740574+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-18T00:36:02.645624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.645624+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.9","version":"229'51","reported_seq":268,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.730410+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:36:09.730410+0000","last_peered":"2024-02-18T00:36:09.730410+0000","last_clean":"2024-02-18T00:36:09.730410+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:36:09.730410+0000","last_undegraded":"2024-02-18T00:36:09.730410+0000","last_fullsized":"2024-02-18T00:36:09.730410+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.a","version":"265'39","reported_seq":234,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.736086+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:36:09.736086+0000","last_peered":"2024-02-18T00:36:09.736086+0000","last_clean":"2024-02-18T00:36:09.736086+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:36:09.736086+0000","last_undegraded":"2024-02-18T00:36:09.736086+0000","last_fullsized":"2024-02-18T00:36:09.736086+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.b","version":"246'54","reported_seq":244,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.761684+0000","last_change":"2024-02-18T00:36:06.668541+0000","last_active":"2024-02-18T00:36:09.761684+0000","last_peered":"2024-02-18T00:36:09.761684+0000","last_clean":"2024-02-18T00:36:09.761684+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:36:09.761684+0000","last_undegraded":"2024-02-18T00:36:09.761684+0000","last_fullsized":"2024-02-18T00:36:09.761684+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.c","version":"213'17","reported_seq":240,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.760774+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:36:09.760774+0000","last_peered":"2024-02-18T00:36:09.760774+0000","last_clean":"2024-02-18T00:36:09.760774+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:36:09.760774+0000","last_undegraded":"2024-02-18T00:36:09.760774+0000","last_fullsized":"2024-02-18T00:36:09.760774+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.d","version":"259'28","reported_seq":272,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731886+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:36:09.731886+0000","last_peered":"2024-02-18T00:36:09.731886+0000","last_clean":"2024-02-18T00:36:09.731886+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:36:09.731886+0000","last_undegraded":"2024-02-18T00:36:09.731886+0000","last_fullsized":"2024-02-18T00:36:09.731886+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.e","version":"288'81","reported_seq":292,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748491+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:36:09.748491+0000","last_peered":"2024-02-18T00:36:09.748491+0000","last_clean":"2024-02-18T00:36:09.748491+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:36:09.748491+0000","last_undegraded":"2024-02-18T00:36:09.748491+0000","last_fullsized":"2024-02-18T00:36:09.748491+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":81,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.f","version":"229'68","reported_seq":268,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733147+0000","last_change":"2024-02-18T00:36:06.652954+0000","last_active":"2024-02-18T00:36:09.733147+0000","last_peered":"2024-02-18T00:36:09.733147+0000","last_clean":"2024-02-18T00:36:09.733147+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:36:09.733147+0000","last_undegraded":"2024-02-18T00:36:09.733147+0000","last_fullsized":"2024-02-18T00:36:09.733147+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.10","version":"209'56","reported_seq":286,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.740515+0000","last_change":"2024-02-18T00:36:06.661736+0000","last_active":"2024-02-18T00:36:09.740515+0000","last_peered":"2024-02-18T00:36:09.740515+0000","last_clean":"2024-02-18T00:36:09.740515+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:36:09.740515+0000","last_undegraded":"2024-02-18T00:36:09.740515+0000","last_fullsized":"2024-02-18T00:36:09.740515+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'56","last_scrub_stamp":"2024-02-18T00:36:04.656413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.656413+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.11","version":"214'19","reported_seq":244,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.743510+0000","last_change":"2024-02-18T00:36:06.658579+0000","last_active":"2024-02-18T00:36:09.743510+0000","last_peered":"2024-02-18T00:36:09.743510+0000","last_clean":"2024-02-18T00:36:09.743510+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:36:09.743510+0000","last_undegraded":"2024-02-18T00:36:09.743510+0000","last_fullsized":"2024-02-18T00:36:09.743510+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:35:53.563562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.563562+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.12","version":"288'72","reported_seq":286,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.736137+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:36:09.736137+0000","last_peered":"2024-02-18T00:36:09.736137+0000","last_clean":"2024-02-18T00:36:09.736137+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:36:09.736137+0000","last_undegraded":"2024-02-18T00:36:09.736137+0000","last_fullsized":"2024-02-18T00:36:09.736137+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.13","version":"210'17","reported_seq":235,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742237+0000","last_change":"2024-02-18T00:36:06.649777+0000","last_active":"2024-02-18T00:36:09.742237+0000","last_peered":"2024-02-18T00:36:09.742237+0000","last_clean":"2024-02-18T00:36:09.742237+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:36:09.742237+0000","last_undegraded":"2024-02-18T00:36:09.742237+0000","last_fullsized":"2024-02-18T00:36:09.742237+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.16","version":"275'24","reported_seq":252,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733573+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:36:09.733573+0000","last_peered":"2024-02-18T00:36:09.733573+0000","last_clean":"2024-02-18T00:36:09.733573+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:36:09.733573+0000","last_undegraded":"2024-02-18T00:36:09.733573+0000","last_fullsized":"2024-02-18T00:36:09.733573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.17","version":"244'18","reported_seq":251,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749596+0000","last_change":"2024-02-18T00:36:06.659238+0000","last_active":"2024-02-18T00:36:09.749596+0000","last_peered":"2024-02-18T00:36:09.749596+0000","last_clean":"2024-02-18T00:36:09.749596+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:36:09.749596+0000","last_undegraded":"2024-02-18T00:36:09.749596+0000","last_fullsized":"2024-02-18T00:36:09.749596+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"62.15","version":"225'102","reported_seq":314,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.774412+0000","last_change":"2024-02-18T00:36:06.663924+0000","last_active":"2024-02-18T00:36:09.774412+0000","last_peered":"2024-02-18T00:36:09.774412+0000","last_clean":"2024-02-18T00:36:09.774412+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:36:09.774412+0000","last_undegraded":"2024-02-18T00:36:09.774412+0000","last_fullsized":"2024-02-18T00:36:09.774412+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.29","version":"0'0","reported_seq":206,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739364+0000","last_change":"2024-02-18T00:34:15.230999+0000","last_active":"2024-02-18T00:36:09.739364+0000","last_peered":"2024-02-18T00:36:09.739364+0000","last_clean":"2024-02-18T00:36:09.739364+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:36:09.739364+0000","last_undegraded":"2024-02-18T00:36:09.739364+0000","last_fullsized":"2024-02-18T00:36:09.739364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.230938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.230938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":325,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731198+0000","last_change":"2024-02-18T00:36:06.651013+0000","last_active":"2024-02-18T00:36:09.731198+0000","last_peered":"2024-02-18T00:36:09.731198+0000","last_clean":"2024-02-18T00:36:09.731198+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:36:09.731198+0000","last_undegraded":"2024-02-18T00:36:09.731198+0000","last_fullsized":"2024-02-18T00:36:09.731198+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.28","version":"0'0","reported_seq":206,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.753227+0000","last_change":"2024-02-18T00:34:39.758860+0000","last_active":"2024-02-18T00:36:09.753227+0000","last_peered":"2024-02-18T00:36:09.753227+0000","last_clean":"2024-02-18T00:36:09.753227+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:36:09.753227+0000","last_undegraded":"2024-02-18T00:36:09.753227+0000","last_fullsized":"2024-02-18T00:36:09.753227+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.758794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.758794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731098+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:36:09.731098+0000","last_peered":"2024-02-18T00:36:09.731098+0000","last_clean":"2024-02-18T00:36:09.731098+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:36:09.731098+0000","last_undegraded":"2024-02-18T00:36:09.731098+0000","last_fullsized":"2024-02-18T00:36:09.731098+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.27","version":"0'0","reported_seq":162,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733223+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:36:09.733223+0000","last_peered":"2024-02-18T00:36:09.733223+0000","last_clean":"2024-02-18T00:36:09.733223+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:36:09.733223+0000","last_undegraded":"2024-02-18T00:36:09.733223+0000","last_fullsized":"2024-02-18T00:36:09.733223+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":319,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.745165+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:36:09.745165+0000","last_peered":"2024-02-18T00:36:09.745165+0000","last_clean":"2024-02-18T00:36:09.745165+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:36:09.745165+0000","last_undegraded":"2024-02-18T00:36:09.745165+0000","last_fullsized":"2024-02-18T00:36:09.745165+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.26","version":"0'0","reported_seq":160,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733655+0000","last_change":"2024-02-18T00:34:26.276135+0000","last_active":"2024-02-18T00:36:09.733655+0000","last_peered":"2024-02-18T00:36:09.733655+0000","last_clean":"2024-02-18T00:36:09.733655+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:36:09.733655+0000","last_undegraded":"2024-02-18T00:36:09.733655+0000","last_fullsized":"2024-02-18T00:36:09.733655+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.276071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.276071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":296,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742814+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:36:09.742814+0000","last_peered":"2024-02-18T00:36:09.742814+0000","last_clean":"2024-02-18T00:36:09.742814+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:36:09.742814+0000","last_undegraded":"2024-02-18T00:36:09.742814+0000","last_fullsized":"2024-02-18T00:36:09.742814+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.25","version":"57'1","reported_seq":212,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732674+0000","last_change":"2024-02-18T00:33:43.346148+0000","last_active":"2024-02-18T00:36:09.732674+0000","last_peered":"2024-02-18T00:36:09.732674+0000","last_clean":"2024-02-18T00:36:09.732674+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:36:09.732674+0000","last_undegraded":"2024-02-18T00:36:09.732674+0000","last_fullsized":"2024-02-18T00:36:09.732674+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:43.346088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.346088+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"213'17","reported_seq":246,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-02-18T00:36:10.770717+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:36:10.770717+0000","last_peered":"2024-02-18T00:36:10.770717+0000","last_clean":"2024-02-18T00:36:10.770717+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:36:10.770717+0000","last_undegraded":"2024-02-18T00:36:10.770717+0000","last_fullsized":"2024-02-18T00:36:10.770717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.24","version":"0'0","reported_seq":206,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742895+0000","last_change":"2024-02-18T00:34:17.271924+0000","last_active":"2024-02-18T00:36:09.742895+0000","last_peered":"2024-02-18T00:36:09.742895+0000","last_clean":"2024-02-18T00:36:09.742895+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:36:09.742895+0000","last_undegraded":"2024-02-18T00:36:09.742895+0000","last_fullsized":"2024-02-18T00:36:09.742895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.271867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.271867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":270,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732274+0000","last_change":"2024-02-18T00:36:06.651247+0000","last_active":"2024-02-18T00:36:09.732274+0000","last_peered":"2024-02-18T00:36:09.732274+0000","last_clean":"2024-02-18T00:36:09.732274+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:36:09.732274+0000","last_undegraded":"2024-02-18T00:36:09.732274+0000","last_fullsized":"2024-02-18T00:36:09.732274+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.23","version":"0'0","reported_seq":197,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.761789+0000","last_change":"2024-02-18T00:33:55.783013+0000","last_active":"2024-02-18T00:36:09.761789+0000","last_peered":"2024-02-18T00:36:09.761789+0000","last_clean":"2024-02-18T00:36:09.761789+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:36:09.761789+0000","last_undegraded":"2024-02-18T00:36:09.761789+0000","last_fullsized":"2024-02-18T00:36:09.761789+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.782955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.782955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":230,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744567+0000","last_change":"2024-02-18T00:36:06.659436+0000","last_active":"2024-02-18T00:36:09.744567+0000","last_peered":"2024-02-18T00:36:09.744567+0000","last_clean":"2024-02-18T00:36:09.744567+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:36:09.744567+0000","last_undegraded":"2024-02-18T00:36:09.744567+0000","last_fullsized":"2024-02-18T00:36:09.744567+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.22","version":"0'0","reported_seq":204,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749184+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:36:09.749184+0000","last_peered":"2024-02-18T00:36:09.749184+0000","last_clean":"2024-02-18T00:36:09.749184+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:36:09.749184+0000","last_undegraded":"2024-02-18T00:36:09.749184+0000","last_fullsized":"2024-02-18T00:36:09.749184+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":241,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742511+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:36:09.742511+0000","last_peered":"2024-02-18T00:36:09.742511+0000","last_clean":"2024-02-18T00:36:09.742511+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:36:09.742511+0000","last_undegraded":"2024-02-18T00:36:09.742511+0000","last_fullsized":"2024-02-18T00:36:09.742511+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.21","version":"0'0","reported_seq":277,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732359+0000","last_change":"2024-02-18T00:33:48.736504+0000","last_active":"2024-02-18T00:36:09.732359+0000","last_peered":"2024-02-18T00:36:09.732359+0000","last_clean":"2024-02-18T00:36:09.732359+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:36:09.732359+0000","last_undegraded":"2024-02-18T00:36:09.732359+0000","last_fullsized":"2024-02-18T00:36:09.732359+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.834878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.834878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":241,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742764+0000","last_change":"2024-02-18T00:36:06.657624+0000","last_active":"2024-02-18T00:36:09.742764+0000","last_peered":"2024-02-18T00:36:09.742764+0000","last_clean":"2024-02-18T00:36:09.742764+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:36:09.742764+0000","last_undegraded":"2024-02-18T00:36:09.742764+0000","last_fullsized":"2024-02-18T00:36:09.742764+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.20","version":"0'0","reported_seq":231,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.738803+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:36:09.738803+0000","last_peered":"2024-02-18T00:36:09.738803+0000","last_clean":"2024-02-18T00:36:09.738803+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:36:09.738803+0000","last_undegraded":"2024-02-18T00:36:09.738803+0000","last_fullsized":"2024-02-18T00:36:09.738803+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741983+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:36:09.741983+0000","last_peered":"2024-02-18T00:36:09.741983+0000","last_clean":"2024-02-18T00:36:09.741983+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:36:09.741983+0000","last_undegraded":"2024-02-18T00:36:09.741983+0000","last_fullsized":"2024-02-18T00:36:09.741983+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":211,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739765+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:36:09.739765+0000","last_peered":"2024-02-18T00:36:09.739765+0000","last_clean":"2024-02-18T00:36:09.739765+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:36:09.739765+0000","last_undegraded":"2024-02-18T00:36:09.739765+0000","last_fullsized":"2024-02-18T00:36:09.739765+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":341,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.738893+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:36:09.738893+0000","last_peered":"2024-02-18T00:36:09.738893+0000","last_clean":"2024-02-18T00:36:09.738893+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:36:09.738893+0000","last_undegraded":"2024-02-18T00:36:09.738893+0000","last_fullsized":"2024-02-18T00:36:09.738893+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":270,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.735242+0000","last_change":"2024-02-18T00:34:26.060181+0000","last_active":"2024-02-18T00:36:09.735242+0000","last_peered":"2024-02-18T00:36:09.735242+0000","last_clean":"2024-02-18T00:36:09.735242+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:36:09.735242+0000","last_undegraded":"2024-02-18T00:36:09.735242+0000","last_fullsized":"2024-02-18T00:36:09.735242+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.060096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.060096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739509+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:36:09.739509+0000","last_peered":"2024-02-18T00:36:09.739509+0000","last_clean":"2024-02-18T00:36:09.739509+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:36:09.739509+0000","last_undegraded":"2024-02-18T00:36:09.739509+0000","last_fullsized":"2024-02-18T00:36:09.739509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746775+0000","last_change":"2024-02-18T00:35:20.428964+0000","last_active":"2024-02-18T00:36:09.746775+0000","last_peered":"2024-02-18T00:36:09.746775+0000","last_clean":"2024-02-18T00:36:09.746775+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:36:09.746775+0000","last_undegraded":"2024-02-18T00:36:09.746775+0000","last_fullsized":"2024-02-18T00:36:09.746775+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:20.428893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:20.428893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":308,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742604+0000","last_change":"2024-02-18T00:35:54.188262+0000","last_active":"2024-02-18T00:36:09.742604+0000","last_peered":"2024-02-18T00:36:09.742604+0000","last_clean":"2024-02-18T00:36:09.742604+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:36:09.742604+0000","last_undegraded":"2024-02-18T00:36:09.742604+0000","last_fullsized":"2024-02-18T00:36:09.742604+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:35:54.188188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.188188+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":259,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749201+0000","last_change":"2024-02-18T00:34:24.334917+0000","last_active":"2024-02-18T00:36:09.749201+0000","last_peered":"2024-02-18T00:36:09.749201+0000","last_clean":"2024-02-18T00:36:09.749201+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:36:09.749201+0000","last_undegraded":"2024-02-18T00:36:09.749201+0000","last_fullsized":"2024-02-18T00:36:09.749201+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.334814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.334814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.736262+0000","last_change":"2024-02-18T00:33:42.768749+0000","last_active":"2024-02-18T00:36:09.736262+0000","last_peered":"2024-02-18T00:36:09.736262+0000","last_clean":"2024-02-18T00:36:09.736262+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:36:09.736262+0000","last_undegraded":"2024-02-18T00:36:09.736262+0000","last_fullsized":"2024-02-18T00:36:09.736262+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.768630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.768630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":270,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.737126+0000","last_change":"2024-02-18T00:35:42.683115+0000","last_active":"2024-02-18T00:36:09.737126+0000","last_peered":"2024-02-18T00:36:09.737126+0000","last_clean":"2024-02-18T00:36:09.737126+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:36:09.737126+0000","last_undegraded":"2024-02-18T00:36:09.737126+0000","last_fullsized":"2024-02-18T00:36:09.737126+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.770023+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:36:09.770023+0000","last_peered":"2024-02-18T00:36:09.770023+0000","last_clean":"2024-02-18T00:36:09.770023+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:36:09.770023+0000","last_undegraded":"2024-02-18T00:36:09.770023+0000","last_fullsized":"2024-02-18T00:36:09.770023+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":340,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749356+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:36:09.749356+0000","last_peered":"2024-02-18T00:36:09.749356+0000","last_clean":"2024-02-18T00:36:09.749356+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:36:09.749356+0000","last_undegraded":"2024-02-18T00:36:09.749356+0000","last_fullsized":"2024-02-18T00:36:09.749356+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":259,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731555+0000","last_change":"2024-02-18T00:33:46.861306+0000","last_active":"2024-02-18T00:36:09.731555+0000","last_peered":"2024-02-18T00:36:09.731555+0000","last_clean":"2024-02-18T00:36:09.731555+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:36:09.731555+0000","last_undegraded":"2024-02-18T00:36:09.731555+0000","last_fullsized":"2024-02-18T00:36:09.731555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.861233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.861233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731341+0000","last_change":"2024-02-18T00:33:53.005012+0000","last_active":"2024-02-18T00:36:09.731341+0000","last_peered":"2024-02-18T00:36:09.731341+0000","last_clean":"2024-02-18T00:36:09.731341+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:36:09.731341+0000","last_undegraded":"2024-02-18T00:36:09.731341+0000","last_fullsized":"2024-02-18T00:36:09.731341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.004903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.004903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731597+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:36:09.731597+0000","last_peered":"2024-02-18T00:36:09.731597+0000","last_clean":"2024-02-18T00:36:09.731597+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:36:09.731597+0000","last_undegraded":"2024-02-18T00:36:09.731597+0000","last_fullsized":"2024-02-18T00:36:09.731597+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739110+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:36:09.739110+0000","last_peered":"2024-02-18T00:36:09.739110+0000","last_clean":"2024-02-18T00:36:09.739110+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:36:09.739110+0000","last_undegraded":"2024-02-18T00:36:09.739110+0000","last_fullsized":"2024-02-18T00:36:09.739110+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":290,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732520+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:36:09.732520+0000","last_peered":"2024-02-18T00:36:09.732520+0000","last_clean":"2024-02-18T00:36:09.732520+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:36:09.732520+0000","last_undegraded":"2024-02-18T00:36:09.732520+0000","last_fullsized":"2024-02-18T00:36:09.732520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731529+0000","last_change":"2024-02-18T00:34:18.301213+0000","last_active":"2024-02-18T00:36:09.731529+0000","last_peered":"2024-02-18T00:36:09.731529+0000","last_clean":"2024-02-18T00:36:09.731529+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:36:09.731529+0000","last_undegraded":"2024-02-18T00:36:09.731529+0000","last_fullsized":"2024-02-18T00:36:09.731529+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.301136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.301136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744869+0000","last_change":"2024-02-18T00:34:39.453471+0000","last_active":"2024-02-18T00:36:09.744869+0000","last_peered":"2024-02-18T00:36:09.744869+0000","last_clean":"2024-02-18T00:36:09.744869+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:36:09.744869+0000","last_undegraded":"2024-02-18T00:36:09.744869+0000","last_fullsized":"2024-02-18T00:36:09.744869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.453312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.453312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732644+0000","last_change":"2024-02-18T00:33:45.813625+0000","last_active":"2024-02-18T00:36:09.732644+0000","last_peered":"2024-02-18T00:36:09.732644+0000","last_clean":"2024-02-18T00:36:09.732644+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:36:09.732644+0000","last_undegraded":"2024-02-18T00:36:09.732644+0000","last_fullsized":"2024-02-18T00:36:09.732644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.813514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.813514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":43,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.730575+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:36:09.730575+0000","last_peered":"2024-02-18T00:36:09.730575+0000","last_clean":"2024-02-18T00:36:09.730575+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:36:09.730575+0000","last_undegraded":"2024-02-18T00:36:09.730575+0000","last_fullsized":"2024-02-18T00:36:09.730575+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":272,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732137+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:36:09.732137+0000","last_peered":"2024-02-18T00:36:09.732137+0000","last_clean":"2024-02-18T00:36:09.732137+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:36:09.732137+0000","last_undegraded":"2024-02-18T00:36:09.732137+0000","last_fullsized":"2024-02-18T00:36:09.732137+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":268,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744683+0000","last_change":"2024-02-18T00:34:23.295161+0000","last_active":"2024-02-18T00:36:09.744683+0000","last_peered":"2024-02-18T00:36:09.744683+0000","last_clean":"2024-02-18T00:36:09.744683+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:36:09.744683+0000","last_undegraded":"2024-02-18T00:36:09.744683+0000","last_fullsized":"2024-02-18T00:36:09.744683+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.295051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":362,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731921+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:36:09.731921+0000","last_peered":"2024-02-18T00:36:09.731921+0000","last_clean":"2024-02-18T00:36:09.731921+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:36:09.731921+0000","last_undegraded":"2024-02-18T00:36:09.731921+0000","last_fullsized":"2024-02-18T00:36:09.731921+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733832+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:36:09.733832+0000","last_peered":"2024-02-18T00:36:09.733832+0000","last_clean":"2024-02-18T00:36:09.733832+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:36:09.733832+0000","last_undegraded":"2024-02-18T00:36:09.733832+0000","last_fullsized":"2024-02-18T00:36:09.733832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.745110+0000","last_change":"2024-02-18T00:35:29.520701+0000","last_active":"2024-02-18T00:36:09.745110+0000","last_peered":"2024-02-18T00:36:09.745110+0000","last_clean":"2024-02-18T00:36:09.745110+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:36:09.745110+0000","last_undegraded":"2024-02-18T00:36:09.745110+0000","last_fullsized":"2024-02-18T00:36:09.745110+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.520632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.520632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":260,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.745984+0000","last_change":"2024-02-18T00:33:44.007884+0000","last_active":"2024-02-18T00:36:09.745984+0000","last_peered":"2024-02-18T00:36:09.745984+0000","last_clean":"2024-02-18T00:36:09.745984+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:36:09.745984+0000","last_undegraded":"2024-02-18T00:36:09.745984+0000","last_fullsized":"2024-02-18T00:36:09.745984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:33:44.007804+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:33:44.007804+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":282,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.743856+0000","last_change":"2024-02-18T00:36:08.716901+0000","last_active":"2024-02-18T00:36:09.743856+0000","last_peered":"2024-02-18T00:36:09.743856+0000","last_clean":"2024-02-18T00:36:09.743856+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:36:09.743856+0000","last_undegraded":"2024-02-18T00:36:09.743856+0000","last_fullsized":"2024-02-18T00:36:09.743856+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.716777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.716777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":346,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733837+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:36:09.733837+0000","last_peered":"2024-02-18T00:36:09.733837+0000","last_clean":"2024-02-18T00:36:09.733837+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:36:09.733837+0000","last_undegraded":"2024-02-18T00:36:09.733837+0000","last_fullsized":"2024-02-18T00:36:09.733837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742959+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:36:09.742959+0000","last_peered":"2024-02-18T00:36:09.742959+0000","last_clean":"2024-02-18T00:36:09.742959+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:36:09.742959+0000","last_undegraded":"2024-02-18T00:36:09.742959+0000","last_fullsized":"2024-02-18T00:36:09.742959+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731275+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:36:09.731275+0000","last_peered":"2024-02-18T00:36:09.731275+0000","last_clean":"2024-02-18T00:36:09.731275+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:36:09.731275+0000","last_undegraded":"2024-02-18T00:36:09.731275+0000","last_fullsized":"2024-02-18T00:36:09.731275+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":314,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733681+0000","last_change":"2024-02-18T00:35:43.219869+0000","last_active":"2024-02-18T00:36:09.733681+0000","last_peered":"2024-02-18T00:36:09.733681+0000","last_clean":"2024-02-18T00:36:09.733681+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:36:09.733681+0000","last_undegraded":"2024-02-18T00:36:09.733681+0000","last_fullsized":"2024-02-18T00:36:09.733681+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:45.311259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.311259+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":230,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746384+0000","last_change":"2024-02-18T00:34:27.927933+0000","last_active":"2024-02-18T00:36:09.746384+0000","last_peered":"2024-02-18T00:36:09.746384+0000","last_clean":"2024-02-18T00:36:09.746384+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:36:09.746384+0000","last_undegraded":"2024-02-18T00:36:09.746384+0000","last_fullsized":"2024-02-18T00:36:09.746384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.927871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.927871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746144+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:36:09.746144+0000","last_peered":"2024-02-18T00:36:09.746144+0000","last_clean":"2024-02-18T00:36:09.746144+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:36:09.746144+0000","last_undegraded":"2024-02-18T00:36:09.746144+0000","last_fullsized":"2024-02-18T00:36:09.746144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.760957+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:36:09.760957+0000","last_peered":"2024-02-18T00:36:09.760957+0000","last_clean":"2024-02-18T00:36:09.760957+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:36:09.760957+0000","last_undegraded":"2024-02-18T00:36:09.760957+0000","last_fullsized":"2024-02-18T00:36:09.760957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732284+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:36:09.732284+0000","last_peered":"2024-02-18T00:36:09.732284+0000","last_clean":"2024-02-18T00:36:09.732284+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:36:09.732284+0000","last_undegraded":"2024-02-18T00:36:09.732284+0000","last_fullsized":"2024-02-18T00:36:09.732284+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":347,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.758082+0000","last_change":"2024-02-18T00:33:54.753052+0000","last_active":"2024-02-18T00:36:09.758082+0000","last_peered":"2024-02-18T00:36:09.758082+0000","last_clean":"2024-02-18T00:36:09.758082+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:36:09.758082+0000","last_undegraded":"2024-02-18T00:36:09.758082+0000","last_fullsized":"2024-02-18T00:36:09.758082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:33:54.752985+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.752985+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":293,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.730797+0000","last_change":"2024-02-18T00:33:53.320429+0000","last_active":"2024-02-18T00:36:09.730797+0000","last_peered":"2024-02-18T00:36:09.730797+0000","last_clean":"2024-02-18T00:36:09.730797+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:36:09.730797+0000","last_undegraded":"2024-02-18T00:36:09.730797+0000","last_fullsized":"2024-02-18T00:36:09.730797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:33:53.320356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.320356+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":282,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749770+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:36:09.749770+0000","last_peered":"2024-02-18T00:36:09.749770+0000","last_clean":"2024-02-18T00:36:09.749770+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:36:09.749770+0000","last_undegraded":"2024-02-18T00:36:09.749770+0000","last_fullsized":"2024-02-18T00:36:09.749770+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741322+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:36:09.741322+0000","last_peered":"2024-02-18T00:36:09.741322+0000","last_clean":"2024-02-18T00:36:09.741322+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:36:09.741322+0000","last_undegraded":"2024-02-18T00:36:09.741322+0000","last_fullsized":"2024-02-18T00:36:09.741322+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742591+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:36:09.742591+0000","last_peered":"2024-02-18T00:36:09.742591+0000","last_clean":"2024-02-18T00:36:09.742591+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:36:09.742591+0000","last_undegraded":"2024-02-18T00:36:09.742591+0000","last_fullsized":"2024-02-18T00:36:09.742591+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731334+0000","last_change":"2024-02-18T00:33:50.287103+0000","last_active":"2024-02-18T00:36:09.731334+0000","last_peered":"2024-02-18T00:36:09.731334+0000","last_clean":"2024-02-18T00:36:09.731334+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:36:09.731334+0000","last_undegraded":"2024-02-18T00:36:09.731334+0000","last_fullsized":"2024-02-18T00:36:09.731334+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.287007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.287007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":360,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.757953+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:36:09.757953+0000","last_peered":"2024-02-18T00:36:09.757953+0000","last_clean":"2024-02-18T00:36:09.757953+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:36:09.757953+0000","last_undegraded":"2024-02-18T00:36:09.757953+0000","last_fullsized":"2024-02-18T00:36:09.757953+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":273,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746852+0000","last_change":"2024-02-18T00:36:09.647995+0000","last_active":"2024-02-18T00:36:09.746852+0000","last_peered":"2024-02-18T00:36:09.746852+0000","last_clean":"2024-02-18T00:36:09.746852+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:36:09.746852+0000","last_undegraded":"2024-02-18T00:36:09.746852+0000","last_fullsized":"2024-02-18T00:36:09.746852+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.647928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733471+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:36:09.733471+0000","last_peered":"2024-02-18T00:36:09.733471+0000","last_clean":"2024-02-18T00:36:09.733471+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:36:09.733471+0000","last_undegraded":"2024-02-18T00:36:09.733471+0000","last_fullsized":"2024-02-18T00:36:09.733471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.770475+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:36:09.770475+0000","last_peered":"2024-02-18T00:36:09.770475+0000","last_clean":"2024-02-18T00:36:09.770475+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:36:09.770475+0000","last_undegraded":"2024-02-18T00:36:09.770475+0000","last_fullsized":"2024-02-18T00:36:09.770475+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748951+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:36:09.748951+0000","last_peered":"2024-02-18T00:36:09.748951+0000","last_clean":"2024-02-18T00:36:09.748951+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:36:09.748951+0000","last_undegraded":"2024-02-18T00:36:09.748951+0000","last_fullsized":"2024-02-18T00:36:09.748951+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":336,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742490+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:36:09.742490+0000","last_peered":"2024-02-18T00:36:09.742490+0000","last_clean":"2024-02-18T00:36:09.742490+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:36:09.742490+0000","last_undegraded":"2024-02-18T00:36:09.742490+0000","last_fullsized":"2024-02-18T00:36:09.742490+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":167,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742121+0000","last_change":"2024-02-18T00:34:32.021327+0000","last_active":"2024-02-18T00:36:09.742121+0000","last_peered":"2024-02-18T00:36:09.742121+0000","last_clean":"2024-02-18T00:36:09.742121+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:36:09.742121+0000","last_undegraded":"2024-02-18T00:36:09.742121+0000","last_fullsized":"2024-02-18T00:36:09.742121+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:34:32.021219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.021219+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.743930+0000","last_change":"2024-02-18T00:35:13.452285+0000","last_active":"2024-02-18T00:36:09.743930+0000","last_peered":"2024-02-18T00:36:09.743930+0000","last_clean":"2024-02-18T00:36:09.743930+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:36:09.743930+0000","last_undegraded":"2024-02-18T00:36:09.743930+0000","last_fullsized":"2024-02-18T00:36:09.743930+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.452220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.452220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.762797+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:36:09.762797+0000","last_peered":"2024-02-18T00:36:09.762797+0000","last_clean":"2024-02-18T00:36:09.762797+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:36:09.762797+0000","last_undegraded":"2024-02-18T00:36:09.762797+0000","last_fullsized":"2024-02-18T00:36:09.762797+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746220+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:36:09.746220+0000","last_peered":"2024-02-18T00:36:09.746220+0000","last_clean":"2024-02-18T00:36:09.746220+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:36:09.746220+0000","last_undegraded":"2024-02-18T00:36:09.746220+0000","last_fullsized":"2024-02-18T00:36:09.746220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":274,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748857+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:36:09.748857+0000","last_peered":"2024-02-18T00:36:09.748857+0000","last_clean":"2024-02-18T00:36:09.748857+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:36:09.748857+0000","last_undegraded":"2024-02-18T00:36:09.748857+0000","last_fullsized":"2024-02-18T00:36:09.748857+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741133+0000","last_change":"2024-02-18T00:34:26.340185+0000","last_active":"2024-02-18T00:36:09.741133+0000","last_peered":"2024-02-18T00:36:09.741133+0000","last_clean":"2024-02-18T00:36:09.741133+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:36:09.741133+0000","last_undegraded":"2024-02-18T00:36:09.741133+0000","last_fullsized":"2024-02-18T00:36:09.741133+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.340120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.340120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":257,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.736855+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:36:09.736855+0000","last_peered":"2024-02-18T00:36:09.736855+0000","last_clean":"2024-02-18T00:36:09.736855+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:36:09.736855+0000","last_undegraded":"2024-02-18T00:36:09.736855+0000","last_fullsized":"2024-02-18T00:36:09.736855+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744952+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:36:09.744952+0000","last_peered":"2024-02-18T00:36:09.744952+0000","last_clean":"2024-02-18T00:36:09.744952+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:36:09.744952+0000","last_undegraded":"2024-02-18T00:36:09.744952+0000","last_fullsized":"2024-02-18T00:36:09.744952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746540+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:36:09.746540+0000","last_peered":"2024-02-18T00:36:09.746540+0000","last_clean":"2024-02-18T00:36:09.746540+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:36:09.746540+0000","last_undegraded":"2024-02-18T00:36:09.746540+0000","last_fullsized":"2024-02-18T00:36:09.746540+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731359+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:36:09.731359+0000","last_peered":"2024-02-18T00:36:09.731359+0000","last_clean":"2024-02-18T00:36:09.731359+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:36:09.731359+0000","last_undegraded":"2024-02-18T00:36:09.731359+0000","last_fullsized":"2024-02-18T00:36:09.731359+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":309,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744599+0000","last_change":"2024-02-18T00:34:28.290944+0000","last_active":"2024-02-18T00:36:09.744599+0000","last_peered":"2024-02-18T00:36:09.744599+0000","last_clean":"2024-02-18T00:36:09.744599+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:36:09.744599+0000","last_undegraded":"2024-02-18T00:36:09.744599+0000","last_fullsized":"2024-02-18T00:36:09.744599+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:34:28.290868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.290868+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":193,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746462+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:36:09.746462+0000","last_peered":"2024-02-18T00:36:09.746462+0000","last_clean":"2024-02-18T00:36:09.746462+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:36:09.746462+0000","last_undegraded":"2024-02-18T00:36:09.746462+0000","last_fullsized":"2024-02-18T00:36:09.746462+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":277,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.747031+0000","last_change":"2024-02-18T00:34:35.765124+0000","last_active":"2024-02-18T00:36:09.747031+0000","last_peered":"2024-02-18T00:36:09.747031+0000","last_clean":"2024-02-18T00:36:09.747031+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:36:09.747031+0000","last_undegraded":"2024-02-18T00:36:09.747031+0000","last_fullsized":"2024-02-18T00:36:09.747031+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.765059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.765059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":258,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.753485+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:36:09.753485+0000","last_peered":"2024-02-18T00:36:09.753485+0000","last_clean":"2024-02-18T00:36:09.753485+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:36:09.753485+0000","last_undegraded":"2024-02-18T00:36:09.753485+0000","last_fullsized":"2024-02-18T00:36:09.753485+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746279+0000","last_change":"2024-02-18T00:34:17.016556+0000","last_active":"2024-02-18T00:36:09.746279+0000","last_peered":"2024-02-18T00:36:09.746279+0000","last_clean":"2024-02-18T00:36:09.746279+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:36:09.746279+0000","last_undegraded":"2024-02-18T00:36:09.746279+0000","last_fullsized":"2024-02-18T00:36:09.746279+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.016476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.016476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.734530+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:36:09.734530+0000","last_peered":"2024-02-18T00:36:09.734530+0000","last_clean":"2024-02-18T00:36:09.734530+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:36:09.734530+0000","last_undegraded":"2024-02-18T00:36:09.734530+0000","last_fullsized":"2024-02-18T00:36:09.734530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.735001+0000","last_change":"2024-02-18T00:33:49.897870+0000","last_active":"2024-02-18T00:36:09.735001+0000","last_peered":"2024-02-18T00:36:09.735001+0000","last_clean":"2024-02-18T00:36:09.735001+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:36:09.735001+0000","last_undegraded":"2024-02-18T00:36:09.735001+0000","last_fullsized":"2024-02-18T00:36:09.735001+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.897812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.897812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":299,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.761693+0000","last_change":"2024-02-18T00:34:25.385634+0000","last_active":"2024-02-18T00:36:09.761693+0000","last_peered":"2024-02-18T00:36:09.761693+0000","last_clean":"2024-02-18T00:36:09.761693+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:36:09.761693+0000","last_undegraded":"2024-02-18T00:36:09.761693+0000","last_fullsized":"2024-02-18T00:36:09.761693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:34:25.385564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.385564+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":268,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732087+0000","last_change":"2024-02-18T00:33:37.809035+0000","last_active":"2024-02-18T00:36:09.732087+0000","last_peered":"2024-02-18T00:36:09.732087+0000","last_clean":"2024-02-18T00:36:09.732087+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:36:09.732087+0000","last_undegraded":"2024-02-18T00:36:09.732087+0000","last_fullsized":"2024-02-18T00:36:09.732087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:37.808923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.808923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":674,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:10.653006+0000","last_change":"2024-02-18T00:36:06.641279+0000","last_active":"2024-02-18T00:36:10.653006+0000","last_peered":"2024-02-18T00:36:10.653006+0000","last_clean":"2024-02-18T00:36:10.653006+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:36:10.653006+0000","last_undegraded":"2024-02-18T00:36:10.653006+0000","last_fullsized":"2024-02-18T00:36:10.653006+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:36:06.641185+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.641185+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":402,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":196,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744807+0000","last_change":"2024-02-18T00:34:29.252896+0000","last_active":"2024-02-18T00:36:09.744807+0000","last_peered":"2024-02-18T00:36:09.744807+0000","last_clean":"2024-02-18T00:36:09.744807+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:36:09.744807+0000","last_undegraded":"2024-02-18T00:36:09.744807+0000","last_fullsized":"2024-02-18T00:36:09.744807+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:34:29.252830+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.252830+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731444+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:36:09.731444+0000","last_peered":"2024-02-18T00:36:09.731444+0000","last_clean":"2024-02-18T00:36:09.731444+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:36:09.731444+0000","last_undegraded":"2024-02-18T00:36:09.731444+0000","last_fullsized":"2024-02-18T00:36:09.731444+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":254,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742121+0000","last_change":"2024-02-18T00:34:19.925200+0000","last_active":"2024-02-18T00:36:09.742121+0000","last_peered":"2024-02-18T00:36:09.742121+0000","last_clean":"2024-02-18T00:36:09.742121+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:36:09.742121+0000","last_undegraded":"2024-02-18T00:36:09.742121+0000","last_fullsized":"2024-02-18T00:36:09.742121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:19.925111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.925111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":68,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741434+0000","last_change":"2024-02-18T00:35:44.479802+0000","last_active":"2024-02-18T00:36:09.741434+0000","last_peered":"2024-02-18T00:36:09.741434+0000","last_clean":"2024-02-18T00:36:09.741434+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:36:09.741434+0000","last_undegraded":"2024-02-18T00:36:09.741434+0000","last_fullsized":"2024-02-18T00:36:09.741434+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:33:54.197976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.197976+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":268,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739664+0000","last_change":"2024-02-18T00:34:15.045746+0000","last_active":"2024-02-18T00:36:09.739664+0000","last_peered":"2024-02-18T00:36:09.739664+0000","last_clean":"2024-02-18T00:36:09.739664+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:36:09.739664+0000","last_undegraded":"2024-02-18T00:36:09.739664+0000","last_fullsized":"2024-02-18T00:36:09.739664+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.045677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.045677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":257,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.767880+0000","last_change":"2024-02-18T00:33:48.688410+0000","last_active":"2024-02-18T00:36:09.767880+0000","last_peered":"2024-02-18T00:36:09.767880+0000","last_clean":"2024-02-18T00:36:09.767880+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:36:09.767880+0000","last_undegraded":"2024-02-18T00:36:09.767880+0000","last_fullsized":"2024-02-18T00:36:09.767880+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:43.926301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.926301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.745481+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:36:09.745481+0000","last_peered":"2024-02-18T00:36:09.745481+0000","last_clean":"2024-02-18T00:36:09.745481+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:36:09.745481+0000","last_undegraded":"2024-02-18T00:36:09.745481+0000","last_fullsized":"2024-02-18T00:36:09.745481+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":345,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742690+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:36:09.742690+0000","last_peered":"2024-02-18T00:36:09.742690+0000","last_clean":"2024-02-18T00:36:09.742690+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:36:09.742690+0000","last_undegraded":"2024-02-18T00:36:09.742690+0000","last_fullsized":"2024-02-18T00:36:09.742690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":257,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.736387+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:36:09.736387+0000","last_peered":"2024-02-18T00:36:09.736387+0000","last_clean":"2024-02-18T00:36:09.736387+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:36:09.736387+0000","last_undegraded":"2024-02-18T00:36:09.736387+0000","last_fullsized":"2024-02-18T00:36:09.736387+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":282,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744722+0000","last_change":"2024-02-18T00:34:31.410145+0000","last_active":"2024-02-18T00:36:09.744722+0000","last_peered":"2024-02-18T00:36:09.744722+0000","last_clean":"2024-02-18T00:36:09.744722+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:36:09.744722+0000","last_undegraded":"2024-02-18T00:36:09.744722+0000","last_fullsized":"2024-02-18T00:36:09.744722+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.410081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.410081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":247,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746194+0000","last_change":"2024-02-18T00:33:48.010937+0000","last_active":"2024-02-18T00:36:09.746194+0000","last_peered":"2024-02-18T00:36:09.746194+0000","last_clean":"2024-02-18T00:36:09.746194+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:36:09.746194+0000","last_undegraded":"2024-02-18T00:36:09.746194+0000","last_fullsized":"2024-02-18T00:36:09.746194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:48.010821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:48.010821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732243+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:36:09.732243+0000","last_peered":"2024-02-18T00:36:09.732243+0000","last_clean":"2024-02-18T00:36:09.732243+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:36:09.732243+0000","last_undegraded":"2024-02-18T00:36:09.732243+0000","last_fullsized":"2024-02-18T00:36:09.732243+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731453+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:36:09.731453+0000","last_peered":"2024-02-18T00:36:09.731453+0000","last_clean":"2024-02-18T00:36:09.731453+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:36:09.731453+0000","last_undegraded":"2024-02-18T00:36:09.731453+0000","last_fullsized":"2024-02-18T00:36:09.731453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":310,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.730888+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:36:09.730888+0000","last_peered":"2024-02-18T00:36:09.730888+0000","last_clean":"2024-02-18T00:36:09.730888+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:36:09.730888+0000","last_undegraded":"2024-02-18T00:36:09.730888+0000","last_fullsized":"2024-02-18T00:36:09.730888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":292,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733313+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:36:09.733313+0000","last_peered":"2024-02-18T00:36:09.733313+0000","last_clean":"2024-02-18T00:36:09.733313+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:36:09.733313+0000","last_undegraded":"2024-02-18T00:36:09.733313+0000","last_fullsized":"2024-02-18T00:36:09.733313+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":262,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.762450+0000","last_change":"2024-02-18T00:34:26.701231+0000","last_active":"2024-02-18T00:36:09.762450+0000","last_peered":"2024-02-18T00:36:09.762450+0000","last_clean":"2024-02-18T00:36:09.762450+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:36:09.762450+0000","last_undegraded":"2024-02-18T00:36:09.762450+0000","last_fullsized":"2024-02-18T00:36:09.762450+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.701162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.701162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731499+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:36:09.731499+0000","last_peered":"2024-02-18T00:36:09.731499+0000","last_clean":"2024-02-18T00:36:09.731499+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:36:09.731499+0000","last_undegraded":"2024-02-18T00:36:09.731499+0000","last_fullsized":"2024-02-18T00:36:09.731499+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":325,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748993+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:36:09.748993+0000","last_peered":"2024-02-18T00:36:09.748993+0000","last_clean":"2024-02-18T00:36:09.748993+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:36:09.748993+0000","last_undegraded":"2024-02-18T00:36:09.748993+0000","last_fullsized":"2024-02-18T00:36:09.748993+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":259,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731357+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:36:09.731357+0000","last_peered":"2024-02-18T00:36:09.731357+0000","last_clean":"2024-02-18T00:36:09.731357+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:36:09.731357+0000","last_undegraded":"2024-02-18T00:36:09.731357+0000","last_fullsized":"2024-02-18T00:36:09.731357+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.769875+0000","last_change":"2024-02-18T00:34:25.683506+0000","last_active":"2024-02-18T00:36:09.769875+0000","last_peered":"2024-02-18T00:36:09.769875+0000","last_clean":"2024-02-18T00:36:09.769875+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:36:09.769875+0000","last_undegraded":"2024-02-18T00:36:09.769875+0000","last_fullsized":"2024-02-18T00:36:09.769875+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.683442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.683442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":262,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731355+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:36:09.731355+0000","last_peered":"2024-02-18T00:36:09.731355+0000","last_clean":"2024-02-18T00:36:09.731355+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:36:09.731355+0000","last_undegraded":"2024-02-18T00:36:09.731355+0000","last_fullsized":"2024-02-18T00:36:09.731355+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.734402+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:36:09.734402+0000","last_peered":"2024-02-18T00:36:09.734402+0000","last_clean":"2024-02-18T00:36:09.734402+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:36:09.734402+0000","last_undegraded":"2024-02-18T00:36:09.734402+0000","last_fullsized":"2024-02-18T00:36:09.734402+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":319,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.769718+0000","last_change":"2024-02-18T00:33:48.861148+0000","last_active":"2024-02-18T00:36:09.769718+0000","last_peered":"2024-02-18T00:36:09.769718+0000","last_clean":"2024-02-18T00:36:09.769718+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:36:09.769718+0000","last_undegraded":"2024-02-18T00:36:09.769718+0000","last_fullsized":"2024-02-18T00:36:09.769718+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:33:46.897445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.897445+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739621+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:36:09.739621+0000","last_peered":"2024-02-18T00:36:09.739621+0000","last_clean":"2024-02-18T00:36:09.739621+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:36:09.739621+0000","last_undegraded":"2024-02-18T00:36:09.739621+0000","last_fullsized":"2024-02-18T00:36:09.739621+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.740143+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:36:09.740143+0000","last_peered":"2024-02-18T00:36:09.740143+0000","last_clean":"2024-02-18T00:36:09.740143+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:36:09.740143+0000","last_undegraded":"2024-02-18T00:36:09.740143+0000","last_fullsized":"2024-02-18T00:36:09.740143+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":164,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:10.525390+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:36:10.525390+0000","last_peered":"2024-02-18T00:36:10.525390+0000","last_clean":"2024-02-18T00:36:10.525390+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:36:10.525390+0000","last_undegraded":"2024-02-18T00:36:10.525390+0000","last_fullsized":"2024-02-18T00:36:10.525390+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732275+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:36:09.732275+0000","last_peered":"2024-02-18T00:36:09.732275+0000","last_clean":"2024-02-18T00:36:09.732275+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:36:09.732275+0000","last_undegraded":"2024-02-18T00:36:09.732275+0000","last_fullsized":"2024-02-18T00:36:09.732275+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":327,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.761292+0000","last_change":"2024-02-18T00:34:30.666844+0000","last_active":"2024-02-18T00:36:09.761292+0000","last_peered":"2024-02-18T00:36:09.761292+0000","last_clean":"2024-02-18T00:36:09.761292+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:36:09.761292+0000","last_undegraded":"2024-02-18T00:36:09.761292+0000","last_fullsized":"2024-02-18T00:36:09.761292+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:34:30.666765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.666765+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":277,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739589+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:36:09.739589+0000","last_peered":"2024-02-18T00:36:09.739589+0000","last_clean":"2024-02-18T00:36:09.739589+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:36:09.739589+0000","last_undegraded":"2024-02-18T00:36:09.739589+0000","last_fullsized":"2024-02-18T00:36:09.739589+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":346,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742817+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:36:09.742817+0000","last_peered":"2024-02-18T00:36:09.742817+0000","last_clean":"2024-02-18T00:36:09.742817+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:36:09.742817+0000","last_undegraded":"2024-02-18T00:36:09.742817+0000","last_fullsized":"2024-02-18T00:36:09.742817+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":261,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748773+0000","last_change":"2024-02-18T00:33:53.101420+0000","last_active":"2024-02-18T00:36:09.748773+0000","last_peered":"2024-02-18T00:36:09.748773+0000","last_clean":"2024-02-18T00:36:09.748773+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:36:09.748773+0000","last_undegraded":"2024-02-18T00:36:09.748773+0000","last_fullsized":"2024-02-18T00:36:09.748773+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.101251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.101251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739881+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:36:09.739881+0000","last_peered":"2024-02-18T00:36:09.739881+0000","last_clean":"2024-02-18T00:36:09.739881+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:36:09.739881+0000","last_undegraded":"2024-02-18T00:36:09.739881+0000","last_fullsized":"2024-02-18T00:36:09.739881+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":309,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742676+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:36:09.742676+0000","last_peered":"2024-02-18T00:36:09.742676+0000","last_clean":"2024-02-18T00:36:09.742676+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:36:09.742676+0000","last_undegraded":"2024-02-18T00:36:09.742676+0000","last_fullsized":"2024-02-18T00:36:09.742676+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":258,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.742541+0000","last_change":"2024-02-18T00:34:15.935045+0000","last_active":"2024-02-18T00:36:09.742541+0000","last_peered":"2024-02-18T00:36:09.742541+0000","last_clean":"2024-02-18T00:36:09.742541+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:36:09.742541+0000","last_undegraded":"2024-02-18T00:36:09.742541+0000","last_fullsized":"2024-02-18T00:36:09.742541+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.934970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"150'9","reported_seq":359,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.769747+0000","last_change":"2024-02-18T00:34:27.741951+0000","last_active":"2024-02-18T00:36:09.769747+0000","last_peered":"2024-02-18T00:36:09.769747+0000","last_clean":"2024-02-18T00:36:09.769747+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:36:09.769747+0000","last_undegraded":"2024-02-18T00:36:09.769747+0000","last_fullsized":"2024-02-18T00:36:09.769747+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:27.741881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.741881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733387+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:36:09.733387+0000","last_peered":"2024-02-18T00:36:09.733387+0000","last_clean":"2024-02-18T00:36:09.733387+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:36:09.733387+0000","last_undegraded":"2024-02-18T00:36:09.733387+0000","last_fullsized":"2024-02-18T00:36:09.733387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746292+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:36:09.746292+0000","last_peered":"2024-02-18T00:36:09.746292+0000","last_clean":"2024-02-18T00:36:09.746292+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:36:09.746292+0000","last_undegraded":"2024-02-18T00:36:09.746292+0000","last_fullsized":"2024-02-18T00:36:09.746292+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":273,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744803+0000","last_change":"2024-02-18T00:34:32.418786+0000","last_active":"2024-02-18T00:36:09.744803+0000","last_peered":"2024-02-18T00:36:09.744803+0000","last_clean":"2024-02-18T00:36:09.744803+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:36:09.744803+0000","last_undegraded":"2024-02-18T00:36:09.744803+0000","last_fullsized":"2024-02-18T00:36:09.744803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:34:32.418717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.418717+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.743036+0000","last_change":"2024-02-18T00:34:20.312165+0000","last_active":"2024-02-18T00:36:09.743036+0000","last_peered":"2024-02-18T00:36:09.743036+0000","last_clean":"2024-02-18T00:36:09.743036+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:36:09.743036+0000","last_undegraded":"2024-02-18T00:36:09.743036+0000","last_fullsized":"2024-02-18T00:36:09.743036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.312096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.312096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.762340+0000","last_change":"2024-02-18T00:33:50.765435+0000","last_active":"2024-02-18T00:36:09.762340+0000","last_peered":"2024-02-18T00:36:09.762340+0000","last_clean":"2024-02-18T00:36:09.762340+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:36:09.762340+0000","last_undegraded":"2024-02-18T00:36:09.762340+0000","last_fullsized":"2024-02-18T00:36:09.762340+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.765368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.765368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":247,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746102+0000","last_change":"2024-02-18T00:33:01.075249+0000","last_active":"2024-02-18T00:36:09.746102+0000","last_peered":"2024-02-18T00:36:09.746102+0000","last_clean":"2024-02-18T00:36:09.746102+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:36:09.746102+0000","last_undegraded":"2024-02-18T00:36:09.746102+0000","last_fullsized":"2024-02-18T00:36:09.746102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.075169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.075169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":194,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748686+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:36:09.748686+0000","last_peered":"2024-02-18T00:36:09.748686+0000","last_clean":"2024-02-18T00:36:09.748686+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:36:09.748686+0000","last_undegraded":"2024-02-18T00:36:09.748686+0000","last_fullsized":"2024-02-18T00:36:09.748686+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":276,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731592+0000","last_change":"2024-02-18T00:34:20.362851+0000","last_active":"2024-02-18T00:36:09.731592+0000","last_peered":"2024-02-18T00:36:09.731592+0000","last_clean":"2024-02-18T00:36:09.731592+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:36:09.731592+0000","last_undegraded":"2024-02-18T00:36:09.731592+0000","last_fullsized":"2024-02-18T00:36:09.731592+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:34:20.362782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.362782+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":259,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749032+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:36:09.749032+0000","last_peered":"2024-02-18T00:36:09.749032+0000","last_clean":"2024-02-18T00:36:09.749032+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:36:09.749032+0000","last_undegraded":"2024-02-18T00:36:09.749032+0000","last_fullsized":"2024-02-18T00:36:09.749032+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744923+0000","last_change":"2024-02-18T00:34:34.355483+0000","last_active":"2024-02-18T00:36:09.744923+0000","last_peered":"2024-02-18T00:36:09.744923+0000","last_clean":"2024-02-18T00:36:09.744923+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:36:09.744923+0000","last_undegraded":"2024-02-18T00:36:09.744923+0000","last_fullsized":"2024-02-18T00:36:09.744923+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.355416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.355416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.752671+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:36:09.752671+0000","last_peered":"2024-02-18T00:36:09.752671+0000","last_clean":"2024-02-18T00:36:09.752671+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:36:09.752671+0000","last_undegraded":"2024-02-18T00:36:09.752671+0000","last_fullsized":"2024-02-18T00:36:09.752671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749120+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:36:09.749120+0000","last_peered":"2024-02-18T00:36:09.749120+0000","last_clean":"2024-02-18T00:36:09.749120+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:36:09.749120+0000","last_undegraded":"2024-02-18T00:36:09.749120+0000","last_fullsized":"2024-02-18T00:36:09.749120+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":263,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.761382+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:36:09.761382+0000","last_peered":"2024-02-18T00:36:09.761382+0000","last_clean":"2024-02-18T00:36:09.761382+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:36:09.761382+0000","last_undegraded":"2024-02-18T00:36:09.761382+0000","last_fullsized":"2024-02-18T00:36:09.761382+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.740225+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:36:09.740225+0000","last_peered":"2024-02-18T00:36:09.740225+0000","last_clean":"2024-02-18T00:36:09.740225+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:36:09.740225+0000","last_undegraded":"2024-02-18T00:36:09.740225+0000","last_fullsized":"2024-02-18T00:36:09.740225+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.739544+0000","last_change":"2024-02-18T00:33:56.296342+0000","last_active":"2024-02-18T00:36:09.739544+0000","last_peered":"2024-02-18T00:36:09.739544+0000","last_clean":"2024-02-18T00:36:09.739544+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:36:09.739544+0000","last_undegraded":"2024-02-18T00:36:09.739544+0000","last_fullsized":"2024-02-18T00:36:09.739544+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:56.296274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.296274+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.736344+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:36:09.736344+0000","last_peered":"2024-02-18T00:36:09.736344+0000","last_clean":"2024-02-18T00:36:09.736344+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:36:09.736344+0000","last_undegraded":"2024-02-18T00:36:09.736344+0000","last_fullsized":"2024-02-18T00:36:09.736344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741985+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:36:09.741985+0000","last_peered":"2024-02-18T00:36:09.741985+0000","last_clean":"2024-02-18T00:36:09.741985+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:36:09.741985+0000","last_undegraded":"2024-02-18T00:36:09.741985+0000","last_fullsized":"2024-02-18T00:36:09.741985+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":314,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.757329+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:36:09.757329+0000","last_peered":"2024-02-18T00:36:09.757329+0000","last_clean":"2024-02-18T00:36:09.757329+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:36:09.757329+0000","last_undegraded":"2024-02-18T00:36:09.757329+0000","last_fullsized":"2024-02-18T00:36:09.757329+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":259,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749759+0000","last_change":"2024-02-18T00:33:51.295157+0000","last_active":"2024-02-18T00:36:09.749759+0000","last_peered":"2024-02-18T00:36:09.749759+0000","last_clean":"2024-02-18T00:36:09.749759+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:36:09.749759+0000","last_undegraded":"2024-02-18T00:36:09.749759+0000","last_fullsized":"2024-02-18T00:36:09.749759+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:51.295093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:51.295093+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":168,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.730669+0000","last_change":"2024-02-18T00:35:42.661117+0000","last_active":"2024-02-18T00:36:09.730669+0000","last_peered":"2024-02-18T00:36:09.730669+0000","last_clean":"2024-02-18T00:36:09.730669+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:36:09.730669+0000","last_undegraded":"2024-02-18T00:36:09.730669+0000","last_fullsized":"2024-02-18T00:36:09.730669+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.374328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.374328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.762559+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:36:09.762559+0000","last_peered":"2024-02-18T00:36:09.762559+0000","last_clean":"2024-02-18T00:36:09.762559+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:36:09.762559+0000","last_undegraded":"2024-02-18T00:36:09.762559+0000","last_fullsized":"2024-02-18T00:36:09.762559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731178+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:36:09.731178+0000","last_peered":"2024-02-18T00:36:09.731178+0000","last_clean":"2024-02-18T00:36:09.731178+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:36:09.731178+0000","last_undegraded":"2024-02-18T00:36:09.731178+0000","last_fullsized":"2024-02-18T00:36:09.731178+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":315,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749180+0000","last_change":"2024-02-18T00:33:56.121942+0000","last_active":"2024-02-18T00:36:09.749180+0000","last_peered":"2024-02-18T00:36:09.749180+0000","last_clean":"2024-02-18T00:36:09.749180+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:36:09.749180+0000","last_undegraded":"2024-02-18T00:36:09.749180+0000","last_fullsized":"2024-02-18T00:36:09.749180+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.121872+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":268,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.738973+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:36:09.738973+0000","last_peered":"2024-02-18T00:36:09.738973+0000","last_clean":"2024-02-18T00:36:09.738973+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:36:09.738973+0000","last_undegraded":"2024-02-18T00:36:09.738973+0000","last_fullsized":"2024-02-18T00:36:09.738973+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":367,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749865+0000","last_change":"2024-02-18T00:34:18.318312+0000","last_active":"2024-02-18T00:36:09.749865+0000","last_peered":"2024-02-18T00:36:09.749865+0000","last_clean":"2024-02-18T00:36:09.749865+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:36:09.749865+0000","last_undegraded":"2024-02-18T00:36:09.749865+0000","last_fullsized":"2024-02-18T00:36:09.749865+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:18.318190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.318190+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":172,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741905+0000","last_change":"2024-02-18T00:34:20.995201+0000","last_active":"2024-02-18T00:36:09.741905+0000","last_peered":"2024-02-18T00:36:09.741905+0000","last_clean":"2024-02-18T00:36:09.741905+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:36:09.741905+0000","last_undegraded":"2024-02-18T00:36:09.741905+0000","last_fullsized":"2024-02-18T00:36:09.741905+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:34:20.995125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.995125+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.753335+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:36:09.753335+0000","last_peered":"2024-02-18T00:36:09.753335+0000","last_clean":"2024-02-18T00:36:09.753335+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:36:09.753335+0000","last_undegraded":"2024-02-18T00:36:09.753335+0000","last_fullsized":"2024-02-18T00:36:09.753335+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":296,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749952+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:36:09.749952+0000","last_peered":"2024-02-18T00:36:09.749952+0000","last_clean":"2024-02-18T00:36:09.749952+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:36:09.749952+0000","last_undegraded":"2024-02-18T00:36:09.749952+0000","last_fullsized":"2024-02-18T00:36:09.749952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":259,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.756389+0000","last_change":"2024-02-18T00:34:28.731011+0000","last_active":"2024-02-18T00:36:09.756389+0000","last_peered":"2024-02-18T00:36:09.756389+0000","last_clean":"2024-02-18T00:36:09.756389+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:36:09.756389+0000","last_undegraded":"2024-02-18T00:36:09.756389+0000","last_fullsized":"2024-02-18T00:36:09.756389+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.730955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.730955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":258,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749292+0000","last_change":"2024-02-18T00:33:48.751672+0000","last_active":"2024-02-18T00:36:09.749292+0000","last_peered":"2024-02-18T00:36:09.749292+0000","last_clean":"2024-02-18T00:36:09.749292+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:36:09.749292+0000","last_undegraded":"2024-02-18T00:36:09.749292+0000","last_fullsized":"2024-02-18T00:36:09.749292+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.025296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.025296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":258,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748878+0000","last_change":"2024-02-18T00:35:54.569108+0000","last_active":"2024-02-18T00:36:09.748878+0000","last_peered":"2024-02-18T00:36:09.748878+0000","last_clean":"2024-02-18T00:36:09.748878+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:36:09.748878+0000","last_undegraded":"2024-02-18T00:36:09.748878+0000","last_fullsized":"2024-02-18T00:36:09.748878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.569033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.569033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":212,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732195+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:36:09.732195+0000","last_peered":"2024-02-18T00:36:09.732195+0000","last_clean":"2024-02-18T00:36:09.732195+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:36:09.732195+0000","last_undegraded":"2024-02-18T00:36:09.732195+0000","last_fullsized":"2024-02-18T00:36:09.732195+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":287,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731639+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:36:09.731639+0000","last_peered":"2024-02-18T00:36:09.731639+0000","last_clean":"2024-02-18T00:36:09.731639+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:36:09.731639+0000","last_undegraded":"2024-02-18T00:36:09.731639+0000","last_fullsized":"2024-02-18T00:36:09.731639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":273,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.748854+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:36:09.748854+0000","last_peered":"2024-02-18T00:36:09.748854+0000","last_clean":"2024-02-18T00:36:09.748854+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:36:09.748854+0000","last_undegraded":"2024-02-18T00:36:09.748854+0000","last_fullsized":"2024-02-18T00:36:09.748854+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.749345+0000","last_change":"2024-02-18T00:34:18.357435+0000","last_active":"2024-02-18T00:36:09.749345+0000","last_peered":"2024-02-18T00:36:09.749345+0000","last_clean":"2024-02-18T00:36:09.749345+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:36:09.749345+0000","last_undegraded":"2024-02-18T00:36:09.749345+0000","last_fullsized":"2024-02-18T00:36:09.749345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.357369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.357369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.733744+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:36:09.733744+0000","last_peered":"2024-02-18T00:36:09.733744+0000","last_clean":"2024-02-18T00:36:09.733744+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:36:09.733744+0000","last_undegraded":"2024-02-18T00:36:09.733744+0000","last_fullsized":"2024-02-18T00:36:09.733744+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741023+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:36:09.741023+0000","last_peered":"2024-02-18T00:36:09.741023+0000","last_clean":"2024-02-18T00:36:09.741023+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:36:09.741023+0000","last_undegraded":"2024-02-18T00:36:09.741023+0000","last_fullsized":"2024-02-18T00:36:09.741023+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":295,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.774490+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:36:09.774490+0000","last_peered":"2024-02-18T00:36:09.774490+0000","last_clean":"2024-02-18T00:36:09.774490+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:36:09.774490+0000","last_undegraded":"2024-02-18T00:36:09.774490+0000","last_fullsized":"2024-02-18T00:36:09.774490+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":279,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.762236+0000","last_change":"2024-02-18T00:33:48.748311+0000","last_active":"2024-02-18T00:36:09.762236+0000","last_peered":"2024-02-18T00:36:09.762236+0000","last_clean":"2024-02-18T00:36:09.762236+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:36:09.762236+0000","last_undegraded":"2024-02-18T00:36:09.762236+0000","last_fullsized":"2024-02-18T00:36:09.762236+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:33:45.906516+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.906516+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732150+0000","last_change":"2024-02-18T00:34:29.236028+0000","last_active":"2024-02-18T00:36:09.732150+0000","last_peered":"2024-02-18T00:36:09.732150+0000","last_clean":"2024-02-18T00:36:09.732150+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:36:09.732150+0000","last_undegraded":"2024-02-18T00:36:09.732150+0000","last_fullsized":"2024-02-18T00:36:09.732150+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:29.235953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.235953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":248,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.770063+0000","last_change":"2024-02-18T00:33:38.914876+0000","last_active":"2024-02-18T00:36:09.770063+0000","last_peered":"2024-02-18T00:36:09.770063+0000","last_clean":"2024-02-18T00:36:09.770063+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:36:09.770063+0000","last_undegraded":"2024-02-18T00:36:09.770063+0000","last_fullsized":"2024-02-18T00:36:09.770063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:38.914691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:38.914691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.745559+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:36:09.745559+0000","last_peered":"2024-02-18T00:36:09.745559+0000","last_clean":"2024-02-18T00:36:09.745559+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:36:09.745559+0000","last_undegraded":"2024-02-18T00:36:09.745559+0000","last_fullsized":"2024-02-18T00:36:09.745559+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":315,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731976+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:36:09.731976+0000","last_peered":"2024-02-18T00:36:09.731976+0000","last_clean":"2024-02-18T00:36:09.731976+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:36:09.731976+0000","last_undegraded":"2024-02-18T00:36:09.731976+0000","last_fullsized":"2024-02-18T00:36:09.731976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.741763+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:36:09.741763+0000","last_peered":"2024-02-18T00:36:09.741763+0000","last_clean":"2024-02-18T00:36:09.741763+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:36:09.741763+0000","last_undegraded":"2024-02-18T00:36:09.741763+0000","last_fullsized":"2024-02-18T00:36:09.741763+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":273,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746969+0000","last_change":"2024-02-18T00:35:42.660706+0000","last_active":"2024-02-18T00:36:09.746969+0000","last_peered":"2024-02-18T00:36:09.746969+0000","last_clean":"2024-02-18T00:36:09.746969+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:36:09.746969+0000","last_undegraded":"2024-02-18T00:36:09.746969+0000","last_fullsized":"2024-02-18T00:36:09.746969+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.961303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.961303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":261,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.746726+0000","last_change":"2024-02-18T00:33:48.649527+0000","last_active":"2024-02-18T00:36:09.746726+0000","last_peered":"2024-02-18T00:36:09.746726+0000","last_clean":"2024-02-18T00:36:09.746726+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:36:09.746726+0000","last_undegraded":"2024-02-18T00:36:09.746726+0000","last_fullsized":"2024-02-18T00:36:09.746726+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.996913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.996913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.734754+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:36:09.734754+0000","last_peered":"2024-02-18T00:36:09.734754+0000","last_clean":"2024-02-18T00:36:09.734754+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:36:09.734754+0000","last_undegraded":"2024-02-18T00:36:09.734754+0000","last_fullsized":"2024-02-18T00:36:09.734754+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":285,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731438+0000","last_change":"2024-02-18T00:34:19.325220+0000","last_active":"2024-02-18T00:36:09.731438+0000","last_peered":"2024-02-18T00:36:09.731438+0000","last_clean":"2024-02-18T00:36:09.731438+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:36:09.731438+0000","last_undegraded":"2024-02-18T00:36:09.731438+0000","last_fullsized":"2024-02-18T00:36:09.731438+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:34:19.325130+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.325130+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":159,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.744694+0000","last_change":"2024-02-18T00:34:31.241231+0000","last_active":"2024-02-18T00:36:09.744694+0000","last_peered":"2024-02-18T00:36:09.744694+0000","last_clean":"2024-02-18T00:36:09.744694+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:36:09.744694+0000","last_undegraded":"2024-02-18T00:36:09.744694+0000","last_fullsized":"2024-02-18T00:36:09.744694+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.241146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":154,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.732108+0000","last_change":"2024-02-18T00:33:48.733307+0000","last_active":"2024-02-18T00:36:09.732108+0000","last_peered":"2024-02-18T00:36:09.732108+0000","last_clean":"2024-02-18T00:36:09.732108+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:36:09.732108+0000","last_undegraded":"2024-02-18T00:36:09.732108+0000","last_fullsized":"2024-02-18T00:36:09.732108+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.801504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.801504+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.738872+0000","last_change":"2024-02-18T00:33:54.013575+0000","last_active":"2024-02-18T00:36:09.738872+0000","last_peered":"2024-02-18T00:36:09.738872+0000","last_clean":"2024-02-18T00:36:09.738872+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:36:09.738872+0000","last_undegraded":"2024-02-18T00:36:09.738872+0000","last_fullsized":"2024-02-18T00:36:09.738872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:54.013467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.013467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":208,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.774627+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:36:09.774627+0000","last_peered":"2024-02-18T00:36:09.774627+0000","last_clean":"2024-02-18T00:36:09.774627+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:36:09.774627+0000","last_undegraded":"2024-02-18T00:36:09.774627+0000","last_fullsized":"2024-02-18T00:36:09.774627+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":293,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.730545+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:36:09.730545+0000","last_peered":"2024-02-18T00:36:09.730545+0000","last_clean":"2024-02-18T00:36:09.730545+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:36:09.730545+0000","last_undegraded":"2024-02-18T00:36:09.730545+0000","last_fullsized":"2024-02-18T00:36:09.730545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":259,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.770369+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:36:09.770369+0000","last_peered":"2024-02-18T00:36:09.770369+0000","last_clean":"2024-02-18T00:36:09.770369+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:36:09.770369+0000","last_undegraded":"2024-02-18T00:36:09.770369+0000","last_fullsized":"2024-02-18T00:36:09.770369+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":264,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.743769+0000","last_change":"2024-02-18T00:34:38.428939+0000","last_active":"2024-02-18T00:36:09.743769+0000","last_peered":"2024-02-18T00:36:09.743769+0000","last_clean":"2024-02-18T00:36:09.743769+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:36:09.743769+0000","last_undegraded":"2024-02-18T00:36:09.743769+0000","last_fullsized":"2024-02-18T00:36:09.743769+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.428855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.428855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":253,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.731890+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:36:09.731890+0000","last_peered":"2024-02-18T00:36:09.731890+0000","last_clean":"2024-02-18T00:36:09.731890+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:36:09.731890+0000","last_undegraded":"2024-02-18T00:36:09.731890+0000","last_fullsized":"2024-02-18T00:36:09.731890+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":199,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.769844+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:36:09.769844+0000","last_peered":"2024-02-18T00:36:09.769844+0000","last_clean":"2024-02-18T00:36:09.769844+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:36:09.769844+0000","last_undegraded":"2024-02-18T00:36:09.769844+0000","last_fullsized":"2024-02-18T00:36:09.769844+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":339,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.753286+0000","last_change":"2024-02-18T00:33:41.866669+0000","last_active":"2024-02-18T00:36:09.753286+0000","last_peered":"2024-02-18T00:36:09.753286+0000","last_clean":"2024-02-18T00:36:09.753286+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:36:09.753286+0000","last_undegraded":"2024-02-18T00:36:09.753286+0000","last_fullsized":"2024-02-18T00:36:09.753286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:33:41.866531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.866531+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":267,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.745288+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:36:09.745288+0000","last_peered":"2024-02-18T00:36:09.745288+0000","last_clean":"2024-02-18T00:36:09.745288+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:36:09.745288+0000","last_undegraded":"2024-02-18T00:36:09.745288+0000","last_fullsized":"2024-02-18T00:36:09.745288+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":255,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.730824+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:36:09.730824+0000","last_peered":"2024-02-18T00:36:09.730824+0000","last_clean":"2024-02-18T00:36:09.730824+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:36:09.730824+0000","last_undegraded":"2024-02-18T00:36:09.730824+0000","last_fullsized":"2024-02-18T00:36:09.730824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":262,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-02-18T00:36:09.761966+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:36:09.761966+0000","last_peered":"2024-02-18T00:36:09.761966+0000","last_clean":"2024-02-18T00:36:09.761966+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:36:09.761966+0000","last_undegraded":"2024-02-18T00:36:09.761966+0000","last_fullsized":"2024-02-18T00:36:09.761966+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":183,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":1204,"num_objects":121,"num_object_clones":2,"num_object_copies":363,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":542,"num_write_kb":537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1495040,"data_stored":3645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1792,"ondisk_log_size":1792,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":402,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266775,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463005,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301956,"kb_used_data":4724,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443196,"statfs":{"total":95995035648,"available":95685832704,"internally_reserved":0,"allocated":4837376,"data_stored":2677888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":2.7280000000000002,"15min":1.234},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.7879999999999998,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":1.1619999999999999,"15min":0.71899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.1130000000000004,"5min":10.433,"15min":10.433},"last":0.60999999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":2.9590000000000001,"15min":1.3169999999999999},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":2.8330000000000002,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.496},{"interface":"front","average":{"1min":0.67400000000000004,"5min":2.9409999999999998,"15min":1.306},"min":{"1min":0.36899999999999999,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":3.6030000000000002,"5min":71.872,"15min":71.872},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":1.389,"15min":0.80000000000000004},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.316},"max":{"1min":3.657,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.437},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.839,"15min":0.95899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":4.1600000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.58199999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":1.383,"15min":0.79900000000000004},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.5670000000000002,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.284,"15min":0.77200000000000002},"min":{"1min":0.438,"5min":0.309,"15min":0.309},"max":{"1min":3.694,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.49099999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.4359999999999999,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1890000000000001,"15min":0.66200000000000003},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.907,"5min":35.18,"15min":35.18},"last":0.32200000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":4.476,"5min":10.606,"15min":10.606},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.2310000000000001,"15min":0.68799999999999994},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6769999999999996,"5min":35.119,"15min":35.119},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.56299999999999994,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60699999999999998,"5min":5.1230000000000002,"15min":5.1230000000000002},"last":1.389},{"interface":"front","average":{"1min":0.754,"5min":1.089,"15min":0.71499999999999997},"min":{"1min":0.36899999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.3620000000000001,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.52900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":117,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318348,"kb_used_data":4728,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93426804,"statfs":{"total":95995035648,"available":95669047296,"internally_reserved":0,"allocated":4841472,"data_stored":2677969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":1.2290000000000001,"15min":0.625},"min":{"1min":0.19900000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.47899999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.441},{"interface":"front","average":{"1min":0.33400000000000002,"5min":1.3140000000000001,"15min":0.65900000000000003},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46999999999999997,"5min":43.137,"15min":43.137},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.5369999999999999,"15min":0.76100000000000001},"min":{"1min":0.23000000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55300000000000005,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.97599999999999998,"15min":0.58599999999999997},"min":{"1min":0.27100000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":17.596,"15min":17.596},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":1.355,"15min":0.755},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51700000000000002,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.497},{"interface":"front","average":{"1min":0.432,"5min":1.3580000000000001,"15min":0.77600000000000002},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Feb 18 00:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.1339999999999999,"15min":0.78000000000000003},"min":{"1min":0.29999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1659999999999999,"15min":0.76600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88100000000000001,"5min":12.75,"15min":12.75},"last":0.79000000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.1439999999999999,"15min":0.77700000000000002},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.8759999999999999,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.1499999999999999,"15min":0.79200000000000004},"min":{"1min":0.41999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":4.1180000000000003,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.78000000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.1779999999999999,"15min":0.81399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":12.834,"15min":12.834},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.149,"15min":0.80300000000000005},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81100000000000005,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":1.151,"15min":0.80500000000000005},"min":{"1min":0.48299999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92500000000000004,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.64600000000000002,"5min":1.141,"15min":0.81599999999999995},"min":{"1min":0.502,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81699999999999995,"5min":12.805,"15min":12.805},"last":0.75900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314256,"kb_used_data":4732,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430896,"statfs":{"total":95995035648,"available":95673237504,"internally_reserved":0,"allocated":4845568,"data_stored":2677083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":2.2679999999999998,"15min":1.1359999999999999},"min":{"1min":0.25700000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.0820000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":1.5089999999999999,"15min":0.73999999999999999},"min":{"1min":0.221,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":1.179,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.371}]},{"osd":2,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":1.827,"15min":0.87},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":3.577,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":1.8560000000000001,"15min":0.874},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":2.153,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.26800000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.1659999999999999,"15min":0.68100000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55400000000000005,"5min":19.006,"15min":19.006},"last":0.255},{"interface":"front","average":{"1min":0.442,"5min":2.222,"15min":1.071},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2130000000000001,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.32300000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":1.6499999999999999,"15min":0.91400000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.63900000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":1.6970000000000001,"15min":0.96799999999999997},"min":{"1min":0.42199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2969999999999999,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.64300000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":1.617,"15min":0.93200000000000005},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.5,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.8169999999999999,"15min":0.96599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":3.4060000000000001,"5min":57.991,"15min":57.991},"last":0.61699999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.2270000000000001,"15min":0.79300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.712,"15min":0.95399999999999996},"min":{"1min":0.41799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.28,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:35:14 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":1.8700000000000001,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.5940000000000001,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":1.8029999999999999,"15min":0.99299999999999999},"min":{"1min":0.45500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":3.0249999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.66300000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314344,"kb_used_data":4828,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430808,"statfs":{"total":95995035648,"available":95673147392,"internally_reserved":0,"allocated":4943872,"data_stored":2687831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.73099999999999998,"15min":0.495},"min":{"1min":0.216,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.61899999999999999,"5min":12.144,"15min":12.144},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35699999999999998,"5min":1.0580000000000001,"15min":0.60799999999999998},"min":{"1min":0.246,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.66500000000000004,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.312}]},{"osd":1,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.71999999999999997,"15min":0.5},"min":{"1min":0.23899999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":2.3370000000000002,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.82899999999999996,"15min":0.54700000000000004},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.77100000000000002,"5min":18.919,"15min":18.919},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.92400000000000004,"15min":0.61599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71799999999999997,"5min":22.712,"15min":22.712},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.92300000000000004,"15min":0.627},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.629,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.372}]},{"osd":4,"last update":"Sun Feb 18 00:35:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.88300000000000001,"15min":0.68700000000000006},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.80500000000000005,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.85599999999999998,"15min":0.68300000000000005},"min":{"1min":0.33700000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79000000000000004,"5min":10.016,"15min":10.016},"last":0.52000000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.78200000000000003,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":6.766,"15min":6.766},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.85399999999999998,"15min":0.70099999999999996},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78500000000000003,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.79600000000000004,"15min":0.69299999999999995},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":6.806,"15min":6.806},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.90800000000000003,"15min":0.74099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:35:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.81000000000000005,"15min":0.74099999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.89800000000000002,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.81499999999999995,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.59399999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314352,"kb_used_data":4828,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430800,"statfs":{"total":95995035648,"available":95673139200,"internally_reserved":0,"allocated":4943872,"data_stored":2688294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.65200000000000002,"15min":0.47799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53100000000000003,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.75700000000000001,"15min":0.52500000000000002},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.304,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.60499999999999998,"15min":0.48199999999999998},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.0800000000000001,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":1.1990000000000001,"15min":0.66800000000000004},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65400000000000003,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.436}]},{"osd":2,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":1.0700000000000001,"15min":0.64800000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.117,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.378},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.54300000000000004,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:35:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.88200000000000001,"15min":0.67800000000000005},"min":{"1min":0.379,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.747,"5min":10.131,"15min":10.131},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.70199999999999996,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72899999999999998,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.70299999999999996,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81799999999999995,"15min":0.67800000000000005},"min":{"1min":0.41099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1279999999999999,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.624}]},{"osd":6,"last update":"Sun Feb 18 00:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":1.0189999999999999,"15min":0.73599999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86199999999999999,"5min":10.189,"15min":10.189},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.73599999999999999,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84999999999999998,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.78100000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.92300000000000004,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80300000000000005,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.59799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986278,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301992,"kb_used_data":4756,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443160,"statfs":{"total":95995035648,"available":95685795840,"internally_reserved":0,"allocated":4870144,"data_stored":2678370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":1.5629999999999999,"15min":0.92600000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65900000000000003,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":1.5089999999999999,"15min":0.88100000000000001},"min":{"1min":0.34100000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.66500000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.42299999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.81200000000000006,"15min":0.65500000000000003},"min":{"1min":0.39300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67300000000000004,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.438},{"interface":"front","average":{"1min":0.435,"5min":1.595,"15min":0.90700000000000003},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":0.63100000000000001,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.78300000000000003,"15min":0.67300000000000004},"min":{"1min":0.373,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60299999999999998,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.90200000000000002,"15min":0.68600000000000005},"min":{"1min":0.33100000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.1459999999999999,"5min":17.77,"15min":17.77},"last":0.44500000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.69699999999999995,"15min":0.63600000000000001},"min":{"1min":0.35199999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.64200000000000002,"5min":7.218,"15min":7.218},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.502,"5min":1.071,"15min":0.72299999999999998},"min":{"1min":0.372,"5min":0.309,"15min":0.309},"max":{"1min":0.65100000000000002,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.40999999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.439,"15min":0.38300000000000001},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.45900000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.34499999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46700000000000003,"15min":0.41599999999999998},"min":{"1min":0.32400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70299999999999996,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.45600000000000002},"min":{"1min":0.34000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.751,"15min":0.751},"last":0.45900000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.50700000000000001,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72399999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50600000000000001,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.36499999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724641,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301880,"kb_used_data":4652,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443272,"statfs":{"total":95995035648,"available":95685910528,"internally_reserved":0,"allocated":4763648,"data_stored":2677891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57199999999999995,"15min":0.54700000000000004},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.8480000000000001,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.69899999999999995,"15min":0.59699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.8149999999999999,"5min":12.744,"15min":12.744},"last":0.54500000000000004}]},{"osd":1,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.625},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6669999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.73399999999999999,"15min":0.62},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":4.7149999999999999,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.51400000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.65000000000000002,"15min":0.61299999999999999},"min":{"1min":0.30199999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.7410000000000001,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67100000000000004,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.734,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.59599999999999997}]},{"osd":3,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":2.1099999999999999,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63200000000000001,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.546,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.503}]},{"osd":4,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.54600000000000004,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.621,"5min":11.818,"15min":11.818},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.443,"5min":0.48799999999999999,"15min":0.41299999999999998},"min":{"1min":0.309,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73299999999999998,"5min":4.274,"15min":4.274},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.64000000000000001,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":18.605,"15min":18.605},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52500000000000002,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65600000000000003,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:35:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.48999999999999999,"15min":0.46000000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70499999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.626},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.33100000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.60099999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":204800,"data_stored":497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":183,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":183,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":183,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":183,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":183,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":183,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":183,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:36:17.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph status --format=json 2024-02-18T00:36:17.210 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:17.239 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:17.297 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:17.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:17.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:17.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:17.440 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:17.462 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:17.464 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:17.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:17.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:17.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 -- 172.21.15.59:0/617965582 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fdbb0 msgr2=0x7f4e080fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/617965582 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fdbb0 0x7f4e080fdf90 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 rx=0x7f4dfc004870 tx=0x7f4dfc009cb0).stop 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 -- 172.21.15.59:0/617965582 shutdown_connections 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/617965582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e080f8420 0x7f4e08104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/617965582 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fdbb0 0x7f4e080fdf90 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/617965582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e080fe510 0x7f4e080f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 -- 172.21.15.59:0/617965582 >> 172.21.15.59:0/617965582 conn(0x7f4e080f3800 msgr2=0x7f4e080f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 -- 172.21.15.59:0/617965582 shutdown_connections 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.692+0000 7f4e0edfb700 1 -- 172.21.15.59:0/617965582 wait complete. 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 librados: starting msgr at 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 librados: starting objecter 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 Processor -- start 2024-02-18T00:36:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 -- start start 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 librados: setting wanted keys 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 librados: calling monclient init 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e080f8420 0x7f4e0806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e080fdbb0 0x7f4e0806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fe510 0x7f4e0806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081164f0 con 0x7f4e080fdbb0 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08116660 con 0x7f4e080f8420 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0edfb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08116820 con 0x7f4e080fe510 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fe510 0x7f4e0806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fe510 0x7f4e0806bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60586/0 (socket says 172.21.15.59:60586) 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 learned_addr learned my addr 172.21.15.59:0/2488760239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e07fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e080fdbb0 0x7f4e0806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e080f8420 0x7f4e0806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e080f8420 msgr2=0x7f4e0806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 --2- 172.21.15.59:0/2488760239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e080f8420 0x7f4e0806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e080fdbb0 msgr2=0x7f4e0806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 --2- 172.21.15.59:0/2488760239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e080fdbb0 0x7f4e0806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.693+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc005040 con 0x7f4e080fe510 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e0d398700 1 --2- 172.21.15.59:0/2488760239 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fe510 0x7f4e0806bab0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 rx=0x7f4e0000db50 tx=0x7f4e00004f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e05ffb700 10 client.?.objecter ms_handle_connect 0x7f4e080fe510 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e05ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000b750 con 0x7f4e080fe510 2024-02-18T00:36:17.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e08116b00 con 0x7f4e080fe510 2024-02-18T00:36:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0000f040 con 0x7f4e080fe510 2024-02-18T00:36:17.722 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:36:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00013a40 con 0x7f4e080fe510 2024-02-18T00:36:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e0edfb700 10 client.40844.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e08116ff0 con 0x7f4e080fe510 2024-02-18T00:36:17.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.694+0000 7f4e0edfb700 1 librados: init done 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.695+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f4e081070d0 con 0x7f4e080fe510 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.696+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4e0000b8b0 con 0x7f4e080fe510 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4df8062c90 0x7f4df8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(299..299 src has 44..299) v4 ==== 7634+0+0 (secure 0 0 0) 0x7f4e0007cd20 con 0x7f4e080fe510 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 osd_map(299..299 src has 44..299) v4 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 3 client.40844.objecter handle_osd_map got epochs [299,299] > 0 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 3 client.40844.objecter handle_osd_map decoding full epoch 299 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4df8062c90 0x7f4df8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 20 client.40844.objecter dump_active .. 0 homeless 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v299) v1 ==== 54+0+16 (secure 0 0 0) 0x7f4e0007f280 con 0x7f4e080fe510 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 1 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4e080fee60 0x7f4e080ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 1 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 0 1 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0806c5b0 osd=0 4 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 10 client.40844.objecter _send_command 1 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e080ff750 con 0x7f4e080fee60 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e07fff700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4e080fee60 0x7f4e080ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4df8062c90 2024-02-18T00:36:17.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4df8062c90 0x7f4df8065150 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 rx=0x7f4dfc004540 tx=0x7f4dfc009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e080fee60 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.697+0000 7f4e07fff700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4e080fee60 0x7f4e080ff260 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.698+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e080ff750 con 0x7f4e080fee60 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.698+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.698+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 1 = system:0 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.698+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0806c5b0 osd=0 4 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.698+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 0 1 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 2 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 2 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0806c5b0 osd=0 3 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 0 2 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0806c5b0 osd=0 4 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 10 client.40844.objecter _send_command 2 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.713+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e080ff8f0 con 0x7f4e080fee60 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e080ff8f0 con 0x7f4e080fee60 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0806c5b0 osd=0 4 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 0 2 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 3 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6805/26279110 conn(0x7f4e08028540 0x7f4e0811b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 3 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 1 3 2024-02-18T00:36:17.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080f5f30 osd=1 4 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 10 client.40844.objecter _send_command 3 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e080f60a0 con 0x7f4e08028540 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6805/26279110 conn(0x7f4e08028540 0x7f4e0811b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e08028540 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.715+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6805/26279110 conn(0x7f4e08028540 0x7f4e0811b260 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.716+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e080f60a0 con 0x7f4e08028540 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.716+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.716+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 3 = system:0 2024-02-18T00:36:17.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.716+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e080f5f30 osd=1 4 2024-02-18T00:36:17.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.716+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 1 3 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 4 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 4 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e080f5f30 osd=1 3 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 1 4 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080f5f30 osd=1 4 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 10 client.40844.objecter _send_command 4 2024-02-18T00:36:17.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.732+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e0805a040 con 0x7f4e08028540 2024-02-18T00:36:17.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e0805a040 con 0x7f4e08028540 2024-02-18T00:36:17.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e080f5f30 osd=1 4 2024-02-18T00:36:17.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 1 4 2024-02-18T00:36:17.736 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 5 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4e080594d0 0x7f4e0811e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 5 2024-02-18T00:36:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 2 5 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08059360 osd=2 4 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 10 client.40844.objecter _send_command 5 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e080280a0 con 0x7f4e080594d0 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.734+0000 7f4e07fff700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4e080594d0 0x7f4e0811e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.735+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e080594d0 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.735+0000 7f4e07fff700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4e080594d0 0x7f4e0811e8e0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.735+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e080280a0 con 0x7f4e080594d0 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.735+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.735+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 5 = system:0 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.735+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e08059360 osd=2 4 2024-02-18T00:36:17.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.735+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 2 5 2024-02-18T00:36:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:17.749 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:16.631588+0000 mon.a (mon.0) 2050 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:17.749 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:16.642502+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:17.749 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:16.642568+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]': finished 2024-02-18T00:36:17.749 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:16.642633+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e298: 8 total, 8 up, 7 in 2024-02-18T00:36:17.750 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:16.644923+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:17.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:16.631588+0000 mon.a (mon.0) 2050 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:17.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:16.642502+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:17.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:16.642568+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]': finished 2024-02-18T00:36:17.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:16.642633+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e298: 8 total, 8 up, 7 in 2024-02-18T00:36:17.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:16.644923+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:17.751 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:17.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 6 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 6 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e08059360 osd=2 3 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 2 6 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08059360 osd=2 4 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 10 client.40844.objecter _send_command 6 2024-02-18T00:36:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.752+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e08028380 con 0x7f4e080594d0 2024-02-18T00:36:17.756 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:36:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e08028380 con 0x7f4e080594d0 2024-02-18T00:36:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e08059360 osd=2 4 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 2 6 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 7 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4e0811c430 0x7f4e08121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 7 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 3 7 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080278d0 osd=3 4 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 10 client.40844.objecter _send_command 7 2024-02-18T00:36:17.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.754+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e08124bb0 con 0x7f4e0811c430 2024-02-18T00:36:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.758+0000 7f4e0d398700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4e0811c430 0x7f4e08121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.759+0000 7f4e0d398700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4e0811c430 0x7f4e08121f60 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.759+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e0811c430 2024-02-18T00:36:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.760+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e08124bb0 con 0x7f4e0811c430 2024-02-18T00:36:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.760+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:36:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.760+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 7 = system:0 2024-02-18T00:36:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.760+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e080278d0 osd=3 4 2024-02-18T00:36:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.760+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 3 7 2024-02-18T00:36:17.772 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 8 2024-02-18T00:36:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 8 2024-02-18T00:36:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e080278d0 osd=3 3 2024-02-18T00:36:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 3 8 2024-02-18T00:36:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080278d0 osd=3 4 2024-02-18T00:36:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 10 client.40844.objecter _send_command 8 2024-02-18T00:36:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.778+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e0811b990 con 0x7f4e0811c430 2024-02-18T00:36:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.779+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e0811b990 con 0x7f4e0811c430 2024-02-18T00:36:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.779+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.779+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.779+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e080278d0 osd=3 4 2024-02-18T00:36:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.779+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 3 8 2024-02-18T00:36:17.782 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:36:17.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 9 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4e0811fab0 0x7f4e081235d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 9 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 4 9 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811b990 osd=4 4 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 10 client.40844.objecter _send_command 9 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e0811bca0 con 0x7f4e0811fab0 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4e0811fab0 0x7f4e081235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e0811fab0 2024-02-18T00:36:17.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.780+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4e0811fab0 0x7f4e081235d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.781+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e0811bca0 con 0x7f4e0811fab0 2024-02-18T00:36:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.781+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:36:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.781+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 9 = system:0 2024-02-18T00:36:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.781+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0811b990 osd=4 4 2024-02-18T00:36:17.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.781+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 4 9 2024-02-18T00:36:17.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 10 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 10 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0811b990 osd=4 3 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 4 10 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811b990 osd=4 4 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 10 client.40844.objecter _send_command 10 2024-02-18T00:36:17.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.797+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e0811bca0 con 0x7f4e0811fab0 2024-02-18T00:36:17.803 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:36:17.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e0811bca0 con 0x7f4e0811fab0 2024-02-18T00:36:17.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0811b990 osd=4 4 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 4 10 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 11 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4e08123fd0 0x7f4e08124450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 11 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 5 11 2024-02-18T00:36:17.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811bca0 osd=5 4 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 10 client.40844.objecter _send_command 11 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.799+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e0812b4d0 con 0x7f4e08123fd0 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.800+0000 7f4e07fff700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4e08123fd0 0x7f4e08124450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.800+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e08123fd0 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.800+0000 7f4e07fff700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4e08123fd0 0x7f4e08124450 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.801+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e0812b4d0 con 0x7f4e08123fd0 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.801+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.801+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 11 = system:0 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.801+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0811bca0 osd=5 4 2024-02-18T00:36:17.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.801+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 5 11 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 12 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 12 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0811bca0 osd=5 3 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 5 12 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811bca0 osd=5 4 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 10 client.40844.objecter _send_command 12 2024-02-18T00:36:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.817+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e08128d70 con 0x7f4e08123fd0 2024-02-18T00:36:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e08128d70 con 0x7f4e08123fd0 2024-02-18T00:36:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0811bca0 osd=5 4 2024-02-18T00:36:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 5 12 2024-02-18T00:36:17.821 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:36:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 13 2024-02-18T00:36:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4e08129420 0x7f4e0812b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 13 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 6 13 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08128d70 osd=6 4 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 10 client.40844.objecter _send_command 13 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e0812e5f0 con 0x7f4e08129420 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.819+0000 7f4e0d398700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4e08129420 0x7f4e0812b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.820+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e08129420 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.820+0000 7f4e0d398700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4e08129420 0x7f4e0812b850 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.820+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e0812e5f0 con 0x7f4e08129420 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.820+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.820+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 13 = system:0 2024-02-18T00:36:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.820+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e08128d70 osd=6 4 2024-02-18T00:36:17.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.820+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 6 13 2024-02-18T00:36:17.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 14 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 14 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e08128d70 osd=6 3 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 6 14 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08128d70 osd=6 4 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 10 client.40844.objecter _send_command 14 2024-02-18T00:36:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.835+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e0812e5f0 con 0x7f4e08129420 2024-02-18T00:36:17.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e0812e5f0 con 0x7f4e08129420 2024-02-18T00:36:17.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e08128d70 osd=6 4 2024-02-18T00:36:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 6 14 2024-02-18T00:36:17.839 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 15 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6812/53025171 conn(0x7f4e0812d960 0x7f4e0812ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 15 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 7 15 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0812e5f0 osd=7 4 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 10 client.40844.objecter _send_command 15 2024-02-18T00:36:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e081319d0 con 0x7f4e0812d960 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6812/53025171 conn(0x7f4e0812d960 0x7f4e0812ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e05ffb700 10 client.40844.objecter ms_handle_connect 0x7f4e0812d960 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.837+0000 7f4e0cb97700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6812/53025171 conn(0x7f4e0812d960 0x7f4e0812ddc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.838+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4e081319d0 con 0x7f4e0812d960 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.838+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.838+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 15 = system:0 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.838+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0812e5f0 osd=7 4 2024-02-18T00:36:17.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.838+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 7 15 2024-02-18T00:36:17.852 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 10 client.40844.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign -1 16 2024-02-18T00:36:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 20 client.40844.objecter _get_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_remove -1 16 2024-02-18T00:36:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 20 client.40844.objecter get_session s=0x7f4e0812e5f0 osd=7 3 2024-02-18T00:36:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 15 client.40844.objecter _session_command_op_assign 7 16 2024-02-18T00:36:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0812e5f0 osd=7 4 2024-02-18T00:36:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 10 client.40844.objecter _send_command 16 2024-02-18T00:36:17.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.853+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4e0812be00 con 0x7f4e0812d960 2024-02-18T00:36:17.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.854+0000 7f4e05ffb700 1 -- 172.21.15.59:0/2488760239 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4e0812be00 con 0x7f4e0812d960 2024-02-18T00:36:17.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.854+0000 7f4e05ffb700 10 client.40844.objecter ms_dispatch 0x7f4e08001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:17.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.854+0000 7f4e05ffb700 10 client.40844.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.854+0000 7f4e05ffb700 20 client.40844.objecter put_session s=0x7f4e0812e5f0 osd=7 4 2024-02-18T00:36:17.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.854+0000 7f4e05ffb700 15 client.40844.objecter _session_command_op_remove 7 16 2024-02-18T00:36:17.857 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:36:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 librados: watch_flush enter 2024-02-18T00:36:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 librados: watch_flush exit 2024-02-18T00:36:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.0 2024-02-18T00:36:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4e080fee60 msgr2=0x7f4e080ff260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4e080fee60 0x7f4e080ff260 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0806c5b0 osd=0 1 2024-02-18T00:36:17.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.1 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6805/26279110 conn(0x7f4e08028540 msgr2=0x7f4e0811b260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6805/26279110 conn(0x7f4e08028540 0x7f4e0811b260 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080f5f30 osd=1 1 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.2 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4e080594d0 msgr2=0x7f4e0811e8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4e080594d0 0x7f4e0811e8e0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08059360 osd=2 1 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.3 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4e0811c430 msgr2=0x7f4e08121f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4e0811c430 0x7f4e08121f60 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e080278d0 osd=3 1 2024-02-18T00:36:17.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.4 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4e0811fab0 msgr2=0x7f4e081235d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4e0811fab0 0x7f4e081235d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811b990 osd=4 1 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.5 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4e08123fd0 msgr2=0x7f4e08124450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4e08123fd0 0x7f4e08124450 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0811bca0 osd=5 1 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.6 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4e08129420 msgr2=0x7f4e0812b850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4e08129420 0x7f4e0812b850 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e08128d70 osd=6 1 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter close_session for osd.7 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6812/53025171 conn(0x7f4e0812d960 msgr2=0x7f4e0812ddc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.137:6812/53025171 conn(0x7f4e0812d960 0x7f4e0812ddc0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter put_session s=0x7f4e0812e5f0 osd=7 1 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 20 client.40844.objecter shutdown clearing up homeless session... 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 10 client.40844.objecter successfully canceled tick 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 reap_dead start 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 reap_dead start 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 reap_dead start 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 reap_dead start 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0d398700 1 -- 172.21.15.59:0/2488760239 reap_dead start 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4df8062c90 msgr2=0x7f4df8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4df8062c90 0x7f4df8065150 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 rx=0x7f4dfc004540 tx=0x7f4dfc009cb0).stop 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fe510 msgr2=0x7f4e0806bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fe510 0x7f4e0806bab0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 rx=0x7f4e0000db50 tx=0x7f4e00004f00).stop 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 shutdown_connections 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4e080fe510 0x7f4e0806bab0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 --2- 172.21.15.59:0/2488760239 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4df8062c90 0x7f4df8065150 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 >> 172.21.15.59:0/2488760239 conn(0x7f4e080f3800 msgr2=0x7f4e080f53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 shutdown_connections 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 -- 172.21.15.59:0/2488760239 wait complete. 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:17.855+0000 7f4e0edfb700 1 librados: shutdown 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.862 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:36:17.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:16.631588+0000 mon.a (mon.0) 2050 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:17.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:16.642502+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:17.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:16.642568+0000 mon.a (mon.0) 2052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-35"}]': finished 2024-02-18T00:36:17.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:16.642633+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e298: 8 total, 8 up, 7 in 2024-02-18T00:36:17.870 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:16.644923+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]: dispatch 2024-02-18T00:36:17.881 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:17.902 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:17.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:17.906 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:18.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:18.037 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:18.187 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:18.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:18.190 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:18.291 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:18.320 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:18.375 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:18.376 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:36:18.376 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:36:18.376 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:18.391 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:18.393 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:18.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:18.494 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:18.523 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:18.568 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:18.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:18.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:18.717 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:18.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:18.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:18.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:18.854 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:17.649591+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:18.854 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:17.649682+0000 mon.a (mon.0) 2056 : cluster [DBG] osdmap e299: 8 total, 8 up, 7 in 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:17.697986+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.59:0/2488760239' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:17.864249+0000 mgr.y (mgr.24454) 308 : cluster [DBG] pgmap v436: 206 pgs: 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:18.131985+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:18.133154+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:18.133444+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:17.649591+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:17.649682+0000 mon.a (mon.0) 2056 : cluster [DBG] osdmap e299: 8 total, 8 up, 7 in 2024-02-18T00:36:18.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:17.697986+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.59:0/2488760239' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:18.856 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:17.864249+0000 mgr.y (mgr.24454) 308 : cluster [DBG] pgmap v436: 206 pgs: 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:36:18.856 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:18.131985+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:18.856 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:18.133154+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:18.856 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:18.133444+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:18.857 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:18.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:17.649591+0000 mon.a (mon.0) 2055 : audit [INF] from='client.? 172.21.15.59:0/872697826' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi059-82882-51"}]': finished 2024-02-18T00:36:18.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:17.649682+0000 mon.a (mon.0) 2056 : cluster [DBG] osdmap e299: 8 total, 8 up, 7 in 2024-02-18T00:36:18.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:17.697986+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.59:0/2488760239' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:18.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:17.864249+0000 mgr.y (mgr.24454) 308 : cluster [DBG] pgmap v436: 206 pgs: 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:36:18.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:18.131985+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:18.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:18.133154+0000 mon.a (mon.0) 2058 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:18.985 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:18.133444+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:19.002 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-18T00:36:19.208 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:19.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:19.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:19.265 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:19.313 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:19.340 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:19.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:19.492 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:19.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:19.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:19.624 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:19.723 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:19.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:19.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:19.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:19.855 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:20.394 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:18.653457+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:18.653502+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e300: 8 total, 8 up, 7 in 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:18.658340+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:18.660818+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi059-82882-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:18.661853+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:18.653457+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:18.653502+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e300: 8 total, 8 up, 7 in 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:18.658340+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:18.660818+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi059-82882-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:20.395 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:18.661853+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:20.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:18.653457+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:20.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:18.653502+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e300: 8 total, 8 up, 7 in 2024-02-18T00:36:20.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:18.658340+0000 mon.c (mon.2) 266 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:20.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:18.660818+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi059-82882-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:20.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:18.661853+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:20.948 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:36:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:36:20.948 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:36:20.948 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:21.205 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:21.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:21.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:21.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:36:21.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:36:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:19.864913+0000 mgr.y (mgr.24454) 309 : cluster [DBG] pgmap v439: 238 pgs: 32 unknown, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:20.094672+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:20.094788+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e301: 8 total, 8 up, 7 in 2024-02-18T00:36:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:20.108589+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:20.108844+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:20.658469+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi059-82882-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:20.658562+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:20.658618+0000 mon.a (mon.0) 2069 : cluster [DBG] osdmap e302: 8 total, 8 up, 7 in 2024-02-18T00:36:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:20.660178+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:21.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:20.660861+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:21.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:19.864913+0000 mgr.y (mgr.24454) 309 : cluster [DBG] pgmap v439: 238 pgs: 32 unknown, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:20.094672+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:20.094788+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e301: 8 total, 8 up, 7 in 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:20.108589+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:20.108844+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:20.658469+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi059-82882-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:20.658562+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:20.658618+0000 mon.a (mon.0) 2069 : cluster [DBG] osdmap e302: 8 total, 8 up, 7 in 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:20.660178+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:20.660861+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:19.864913+0000 mgr.y (mgr.24454) 309 : cluster [DBG] pgmap v439: 238 pgs: 32 unknown, 206 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:20.094672+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:20.094788+0000 mon.a (mon.0) 2065 : cluster [DBG] osdmap e301: 8 total, 8 up, 7 in 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:20.108589+0000 mon.c (mon.2) 267 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:20.108844+0000 mon.a (mon.0) 2066 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:20.658469+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi059-82882-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:20.658562+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:20.658618+0000 mon.a (mon.0) 2069 : cluster [DBG] osdmap e302: 8 total, 8 up, 7 in 2024-02-18T00:36:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:20.660178+0000 mon.c (mon.2) 268 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:21.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:20.660861+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:21.592 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:21.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:21.674 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-18T00:36:21.693 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:21.744 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:21.770 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:21.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:21.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:21.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:21.902 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:21.924 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:21.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:21.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:21.959 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:22.028 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:22.059 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:22.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:21.130666+0000 mgr.y (mgr.24454) 310 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:22.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:21.660760+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]': finished 2024-02-18T00:36:22.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:21.660918+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e303: 8 total, 8 up, 7 in 2024-02-18T00:36:22.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:21.665093+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]: dispatch 2024-02-18T00:36:22.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:21.665412+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]: dispatch 2024-02-18T00:36:22.208 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:21.130666+0000 mgr.y (mgr.24454) 310 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:21.660760+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]': finished 2024-02-18T00:36:22.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:21.660918+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e303: 8 total, 8 up, 7 in 2024-02-18T00:36:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:21.665093+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]: dispatch 2024-02-18T00:36:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:21.665412+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]: dispatch 2024-02-18T00:36:22.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:21.130666+0000 mgr.y (mgr.24454) 310 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:22.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:21.660760+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-37"}]': finished 2024-02-18T00:36:22.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:21.660918+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e303: 8 total, 8 up, 7 in 2024-02-18T00:36:22.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:21.665093+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]: dispatch 2024-02-18T00:36:22.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:21.665412+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]: dispatch 2024-02-18T00:36:22.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:22.211 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:22.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/3875778132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fd510 msgr2=0x7f2b280f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/3875778132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fd510 0x7f2b280f6d40 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 rx=0x7f2b1c004870 tx=0x7f2b1c009cb0).stop 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/3875778132 shutdown_connections 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/3875778132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b280fcbb0 0x7f2b280fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/3875778132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b280f7400 0x7f2b28103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/3875778132 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fd510 0x7f2b280f6d40 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/3875778132 >> 172.21.15.59:0/3875778132 conn(0x7f2b280f2820 msgr2=0x7f2b280f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/3875778132 shutdown_connections 2024-02-18T00:36:22.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/3875778132 wait complete. 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 librados: starting msgr at 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 librados: starting objecter 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 Processor -- start 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.215+0000 7f2b2e5f6700 1 -- start start 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 librados: setting wanted keys 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 librados: calling monclient init 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b280f7400 0x7f2b2810d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fcbb0 0x7f2b2810d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b280fd510 0x7f2b28107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b281076f0 con 0x7f2b280fd510 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28107860 con 0x7f2b280f7400 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2e5f6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28107a20 con 0x7f2b280fcbb0 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fcbb0 0x7f2b2810d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b2cb93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b280fd510 0x7f2b28107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fcbb0 0x7f2b2810d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49774/0 (socket says 172.21.15.59:49774) 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 -- 172.21.15.59:0/303829099 learned_addr learned my addr 172.21.15.59:0/303829099 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b27fff700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b280f7400 0x7f2b2810d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 -- 172.21.15.59:0/303829099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b280f7400 msgr2=0x7f2b2810d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b280f7400 0x7f2b2810d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 -- 172.21.15.59:0/303829099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b280fd510 msgr2=0x7f2b28107200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b280fd510 0x7f2b28107200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.216+0000 7f2b277fe700 1 -- 172.21.15.59:0/303829099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c005040 con 0x7f2b280fcbb0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b257fa700 10 client.?.objecter ms_handle_connect 0x7f2b280fcbb0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b277fe700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fcbb0 0x7f2b2810d550 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 rx=0x7f2b1c009eb0 tx=0x7f2b1c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b257fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b257fa700 1 -- 172.21.15.59:0/303829099 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00e070 con 0x7f2b280fcbb0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b257fa700 1 -- 172.21.15.59:0/303829099 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1c013da0 con 0x7f2b280fcbb0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b257fa700 1 -- 172.21.15.59:0/303829099 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c019070 con 0x7f2b280fcbb0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b28107ca0 con 0x7f2b280fcbb0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b2e5f6700 10 client.40856.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b28113e90 con 0x7f2b280fcbb0 2024-02-18T00:36:22.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.217+0000 7f2b2e5f6700 1 librados: init done 2024-02-18T00:36:22.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.218+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b281080e0 con 0x7f2b280fcbb0 2024-02-18T00:36:22.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.218+0000 7f2b257fa700 1 -- 172.21.15.59:0/303829099 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2b1c01e420 con 0x7f2b280fcbb0 2024-02-18T00:36:22.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b257fa700 1 --2- 172.21.15.59:0/303829099 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2b10062c90 0x7f2b10065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b257fa700 1 -- 172.21.15.59:0/303829099 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(303..303 src has 44..303) v4 ==== 8855+0+0 (secure 0 0 0) 0x7f2b1c07dfe0 con 0x7f2b280fcbb0 2024-02-18T00:36:22.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b257fa700 10 client.40856.objecter ms_dispatch 0x7f2b28001550 osd_map(303..303 src has 44..303) v4 2024-02-18T00:36:22.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b257fa700 3 client.40856.objecter handle_osd_map got epochs [303,303] > 0 2024-02-18T00:36:22.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b257fa700 3 client.40856.objecter handle_osd_map decoding full epoch 303 2024-02-18T00:36:22.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b27fff700 1 --2- 172.21.15.59:0/303829099 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2b10062c90 0x7f2b10065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b257fa700 20 client.40856.objecter dump_active .. 0 homeless 2024-02-18T00:36:22.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b257fa700 10 client.40856.objecter ms_handle_connect 0x7f2b10062c90 2024-02-18T00:36:22.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.219+0000 7f2b27fff700 1 --2- 172.21.15.59:0/303829099 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2b10062c90 0x7f2b10065150 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 rx=0x7f2b18004e60 tx=0x7f2b1800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:22.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.223+0000 7f2b257fa700 1 -- 172.21.15.59:0/303829099 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2b1c0acf60 con 0x7f2b280fcbb0 2024-02-18T00:36:22.312 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:22.340 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:22.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.464+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2b2805eeb0 con 0x7f2b280fcbb0 2024-02-18T00:36:22.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.465+0000 7f2b257fa700 1 -- 172.21.15.59:0/303829099 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2451 (secure 0 0 0) 0x7f2b1c023070 con 0x7f2b280fcbb0 2024-02-18T00:36:22.467 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:36:22.467 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":655,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":303,"num_osds":8,"num_up_osds":8,"osd_up_since":1708216146,"num_in_osds":7,"osd_in_since":1708216427,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":206},{"state_name":"unknown","count":32}],"num_pgs":238,"num_pools":9,"num_objects":347,"data_bytes":7013,"bytes_used":2207244288,"bytes_avail":669758005248,"bytes_total":671965249536,"unknown_pgs_ratio":0.13445378839969635},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.59:8443/","prometheus":"http://172.21.15.59:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-18T00:33:31.796486+0000","services":{"rgw":{"daemons":{"summary":"","24347":{"start_epoch":12,"start_stamp":"2024-02-18T00:29:22.198518+0000","gid":24347,"addr":"172.21.15.59:0/2522576766","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)","ceph_version_short":"16.2.14-548-g81bd20d6","container_hostname":"smithi059","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi059","id":"foo.a","kernel_description":"#1 SMP Mon Feb 5 18:34:56 UTC 2024","kernel_version":"4.18.0-539.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543852","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"e67bc818-d879-487a-b572-f16f51d463f9","zone_name":"default","zonegroup_id":"1a312463-4a64-495f-95ab-5ad353fdc6fb","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 10 librados: watch_flush enter 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 10 librados: watch_flush exit 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 20 client.40856.objecter shutdown clearing up homeless session... 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 10 client.40856.objecter successfully canceled tick 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2b10062c90 msgr2=0x7f2b10065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/303829099 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2b10062c90 0x7f2b10065150 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 rx=0x7f2b18004e60 tx=0x7f2b1800b040).stop 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fcbb0 msgr2=0x7f2b2810d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fcbb0 0x7f2b2810d550 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 rx=0x7f2b1c009eb0 tx=0x7f2b1c00b040).stop 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 shutdown_connections 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/303829099 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2b10062c90 0x7f2b10065150 unknown :-1 s=CLOSED pgs=3061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b280f7400 0x7f2b2810d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b280fd510 0x7f2b28107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 --2- 172.21.15.59:0/303829099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b280fcbb0 0x7f2b2810d550 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 >> 172.21.15.59:0/303829099 conn(0x7f2b280f2820 msgr2=0x7f2b280f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 shutdown_connections 2024-02-18T00:36:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 -- 172.21.15.59:0/303829099 wait complete. 2024-02-18T00:36:22.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.467+0000 7f2b2e5f6700 1 librados: shutdown 2024-02-18T00:36:22.492 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:36:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:22] "GET /metrics HTTP/1.1" 200 188512 "" "Prometheus/2.33.4" 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.522+0000 7f37bc31e700 1 -- 172.21.15.59:0/306763051 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 msgr2=0x7f37b40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.522+0000 7f37bc31e700 1 --2- 172.21.15.59:0/306763051 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 0x7f37b40f7d40 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 rx=0x7f37a4004870 tx=0x7f37a4009cb0).stop 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 -- 172.21.15.59:0/306763051 shutdown_connections 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 --2- 172.21.15.59:0/306763051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37b40fdbb0 0x7f37b40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 --2- 172.21.15.59:0/306763051 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40f8420 0x7f37b4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 --2- 172.21.15.59:0/306763051 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 0x7f37b40f7d40 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 -- 172.21.15.59:0/306763051 >> 172.21.15.59:0/306763051 conn(0x7f37b40f3800 msgr2=0x7f37b40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 -- 172.21.15.59:0/306763051 shutdown_connections 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 -- 172.21.15.59:0/306763051 wait complete. 2024-02-18T00:36:22.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 librados: starting msgr at 2024-02-18T00:36:22.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 librados: starting objecter 2024-02-18T00:36:22.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 Processor -- start 2024-02-18T00:36:22.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 -- start start 2024-02-18T00:36:22.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 librados: setting wanted keys 2024-02-18T00:36:22.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.523+0000 7f37bc31e700 1 librados: calling monclient init 2024-02-18T00:36:22.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37b40f8420 0x7f37b406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40fdbb0 0x7f37b406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 0x7f37b406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4116560 con 0x7f37b40fdbb0 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b41166d0 con 0x7f37b40f8420 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4116890 con 0x7f37b40fe510 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40fdbb0 0x7f37b406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40fdbb0 0x7f37b406b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37602/0 (socket says 172.21.15.59:37602) 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 -- 172.21.15.59:0/3626307759 learned_addr learned my addr 172.21.15.59:0/3626307759 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ba8bb700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 0x7f37b406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ba0ba700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37b40f8420 0x7f37b406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 -- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 msgr2=0x7f37b406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 0x7f37b406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 -- 172.21.15.59:0/3626307759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37b40f8420 msgr2=0x7f37b406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37b40f8420 0x7f37b406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 -- 172.21.15.59:0/3626307759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0097c0 con 0x7f37b40fdbb0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ba8bb700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 0x7f37b406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37b98b9700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40fdbb0 0x7f37b406b5c0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 rx=0x7f37a40043a0 tx=0x7f37a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ab7fe700 10 client.?.objecter ms_handle_connect 0x7f37b40fdbb0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ab7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ab7fe700 1 -- 172.21.15.59:0/3626307759 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a400e070 con 0x7f37b40fdbb0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a4005040 con 0x7f37b40fdbb0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ab7fe700 1 -- 172.21.15.59:0/3626307759 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37a4009710 con 0x7f37b40fdbb0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37ab7fe700 1 -- 172.21.15.59:0/3626307759 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4019070 con 0x7f37b40fdbb0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 10 client.31050.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b4116e60 con 0x7f37b40fdbb0 2024-02-18T00:36:22.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.524+0000 7f37bc31e700 1 librados: init done 2024-02-18T00:36:22.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.525+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b41070a0 con 0x7f37b40fdbb0 2024-02-18T00:36:22.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ab7fe700 1 -- 172.21.15.59:0/3626307759 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f37a40090d0 con 0x7f37b40fdbb0 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ab7fe700 1 --2- 172.21.15.59:0/3626307759 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37a0062c90 0x7f37a0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ab7fe700 1 -- 172.21.15.59:0/3626307759 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(303..303 src has 44..303) v4 ==== 8855+0+0 (secure 0 0 0) 0x7f37a407df10 con 0x7f37b40fdbb0 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ab7fe700 10 client.31050.objecter ms_dispatch 0x7f37b4001550 osd_map(303..303 src has 44..303) v4 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ab7fe700 3 client.31050.objecter handle_osd_map got epochs [303,303] > 0 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ab7fe700 3 client.31050.objecter handle_osd_map decoding full epoch 303 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ba0ba700 1 --2- 172.21.15.59:0/3626307759 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37a0062c90 0x7f37a0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.526+0000 7f37ab7fe700 20 client.31050.objecter dump_active .. 0 homeless 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.527+0000 7f37ab7fe700 10 client.31050.objecter ms_handle_connect 0x7f37a0062c90 2024-02-18T00:36:22.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.527+0000 7f37ba0ba700 1 --2- 172.21.15.59:0/3626307759 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37a0062c90 0x7f37a0065150 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f37b00092f0 tx=0x7f37b000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:22.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.531+0000 7f37ab7fe700 1 -- 172.21.15.59:0/3626307759 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f37a4051130 con 0x7f37b40fdbb0 2024-02-18T00:36:22.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:22.742+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f37b405fa60 con 0x7f37b40fdbb0 2024-02-18T00:36:22.789 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:22.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:22.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:22.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:22.923 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:23.164 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:21.865363+0000 mgr.y (mgr.24454) 311 : cluster [DBG] pgmap v442: 246 pgs: 8 creating+peering, 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:23.164 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:22.466318+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? 172.21.15.59:0/303829099' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:36:23.164 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:22.657285+0000 mon.a (mon.0) 2074 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:23.164 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:22.663607+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]': finished 2024-02-18T00:36:23.165 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:22.663711+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e304: 8 total, 8 up, 7 in 2024-02-18T00:36:23.165 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:22.669870+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:23.165 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:22.743900+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.59:0/3626307759' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:23.165 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:22.768443+0000 mon.c (mon.2) 271 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:23.165 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:22.768748+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:23.170 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:23.173 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:23.173 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:23.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:23.306 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-18T00:36:23.306 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:36:23.307 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:23.327 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:23.330 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:23.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:23.354 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:21.865363+0000 mgr.y (mgr.24454) 311 : cluster [DBG] pgmap v442: 246 pgs: 8 creating+peering, 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:23.354 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:22.466318+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? 172.21.15.59:0/303829099' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:22.657285+0000 mon.a (mon.0) 2074 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:22.663607+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]': finished 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:22.663711+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e304: 8 total, 8 up, 7 in 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:22.669870+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:22.743900+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.59:0/3626307759' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:22.768443+0000 mon.c (mon.2) 271 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:22.768748+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:21.865363+0000 mgr.y (mgr.24454) 311 : cluster [DBG] pgmap v442: 246 pgs: 8 creating+peering, 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:22.466318+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? 172.21.15.59:0/303829099' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:36:23.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:22.657285+0000 mon.a (mon.0) 2074 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:36:23.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:22.663607+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-37", "mode": "writeback"}]': finished 2024-02-18T00:36:23.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:22.663711+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e304: 8 total, 8 up, 7 in 2024-02-18T00:36:23.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:22.669870+0000 mon.a (mon.0) 2077 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:23.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:22.743900+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.59:0/3626307759' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:23.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:22.768443+0000 mon.c (mon.2) 271 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:23.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:22.768748+0000 mon.a (mon.0) 2079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:23.432 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:23.462 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:23.481 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:23.484 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:23.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:23.583 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:36:23.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:23.612 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:23.616 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:23.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.632+0000 7f37ab7fe700 1 -- 172.21.15.59:0/3626307759 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v305) v1 ==== 97+0+0 (secure 0 0 0) 0x7f37a4050d20 con 0x7f37b40fdbb0 2024-02-18T00:36:23.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.636+0000 7f37bc31e700 10 librados: watch_flush enter 2024-02-18T00:36:23.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.636+0000 7f37bc31e700 10 librados: watch_flush exit 2024-02-18T00:36:23.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.636+0000 7f37bc31e700 20 client.31050.objecter shutdown clearing up homeless session... 2024-02-18T00:36:23.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.636+0000 7f37bc31e700 10 client.31050.objecter successfully canceled tick 2024-02-18T00:36:23.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.639+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37a0062c90 msgr2=0x7f37a0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:23.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.639+0000 7f37bc31e700 1 --2- 172.21.15.59:0/3626307759 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37a0062c90 0x7f37a0065150 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f37b00092f0 tx=0x7f37b000b040).stop 2024-02-18T00:36:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.639+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40fdbb0 msgr2=0x7f37b406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:23.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.639+0000 7f37bc31e700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40fdbb0 0x7f37b406b5c0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 rx=0x7f37a40043a0 tx=0x7f37a400b040).stop 2024-02-18T00:36:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.642+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 shutdown_connections 2024-02-18T00:36:23.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.642+0000 7f37bc31e700 1 --2- 172.21.15.59:0/3626307759 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37a0062c90 0x7f37a0065150 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.642+0000 7f37bc31e700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37b40f8420 0x7f37b406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.642+0000 7f37bc31e700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37b40fe510 0x7f37b406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.642+0000 7f37bc31e700 1 --2- 172.21.15.59:0/3626307759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37b40fdbb0 0x7f37b406b5c0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.642+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 >> 172.21.15.59:0/3626307759 conn(0x7f37b40f3800 msgr2=0x7f37b40f53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:23.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.642+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 shutdown_connections 2024-02-18T00:36:23.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.645+0000 7f37bc31e700 1 -- 172.21.15.59:0/3626307759 wait complete. 2024-02-18T00:36:23.647 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:23.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:23.645+0000 7f37bc31e700 1 librados: shutdown 2024-02-18T00:36:23.648 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:36:23.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:23.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:23.709 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:36:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:36:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:36:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24829.0:2 15.f 15:f318d819:test-rados-api-smithi059-82989-2::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:36:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.398997+0000", 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 202.306777735, 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.049483261000000001, 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24829", 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3512082368", 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.398997+0000", 2024-02-18T00:36:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.398997+0000", 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.398994+0000", 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9949999999999994e-06 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.399003+0000", 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3013000000000001e-05 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.399016+0000", 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3659999999999992e-06 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.399024+0000", 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001370217 2024-02-18T00:36:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.400395+0000", 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0260000000000007e-06 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.400403+0000", 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.032272268 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.432675+0000", 2024-02-18T00:36:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000105389 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.432780+0000", 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8295999999999995e-05 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 4,7", 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.432879+0000", 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001211431 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.434090+0000", 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0055690369999999998 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.439659+0000", 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0087210759999999995 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.448380+0000", 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8251999999999997e-05 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.448438+0000", 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1529000000000002e-05 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.448480+0000", 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.049483261000000001 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24817.0:2 14.19 14:9a945d6d:test-rados-api-smithi059-82981-2::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.406655+0000", 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 202.29911920399999, 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.148903963, 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24817", 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/521262486", 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406655+0000", 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406655+0000", 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406653+0000", 2024-02-18T00:36:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7129999999999994e-06 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406662+0000", 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.577e-06 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406671+0000", 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5109999999999997e-06 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406679+0000", 2024-02-18T00:36:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1348999999999997e-05 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406750+0000", 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.418e-06 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406758+0000", 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.089051853 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.495809+0000", 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7888999999999997e-05 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.495907+0000", 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000140634 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.496048+0000", 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0083311749999999997 2024-02-18T00:36:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.504379+0000", 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020820595000000001 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.525200+0000", 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.030250745999999998 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.555451+0000", 2024-02-18T00:36:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4838999999999994e-05 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.555515+0000", 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3755999999999999e-05 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.555559+0000", 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.148903963 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24766.0:2 12.4 12:216d7194:test-rados-api-smithi059-82901-3::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.406697+0000", 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 202.29907729600001, 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.16439246499999999, 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24766", 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1346273899", 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406697+0000", 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406697+0000", 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406696+0000", 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2810000000000002e-06 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406701+0000", 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1720000000000004e-06 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406707+0000", 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2470000000000002e-06 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.406711+0000", 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0018499429999999999 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.408561+0000", 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8899999999999998e-06 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.408566+0000", 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.14697418300000001 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.555540+0000", 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000104386 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.555644+0000", 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014483700000000001 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,3", 2024-02-18T00:36:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.555789+0000", 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0015021240000000001 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.557291+0000", 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0056437579999999996 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.562935+0000", 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0080680070000000003 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.571003+0000", 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5513000000000003e-05 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.571049+0000", 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0905999999999998e-05 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.571090+0000", 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.16439246499999999 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2 17.6 17:600fd1a8:test-rados-api-smithi059-82949-3::aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:head [write 0~7] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.411275+0000", 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 202.294499207, 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.121058095, 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:36:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411275+0000", 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411275+0000", 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411273+0000", 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1194e-05 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411284+0000", 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0293e-05 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411294+0000", 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2660000000000004e-06 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411301+0000", 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5239000000000002e-05 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411377+0000", 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3100000000000001e-06 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.411385+0000", 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.082779501000000005 2024-02-18T00:36:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.494164+0000", 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019172699999999999 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.494356+0000", 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00021107100000000001 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,1", 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.494567+0000", 2024-02-18T00:36:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.009565044 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.504132+0000", 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021056782 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.525189+0000", 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0068930160000000001 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.532082+0000", 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2215000000000001e-05 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.532134+0000", 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000198987 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.532333+0000", 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.121058095 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24730.0:2 7.c 7:32c737ec:test-rados-api-smithi059-82893-2::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.420363+0000", 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 202.28541124099999, 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.152481588, 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24730", 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1230350963", 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.420363+0000", 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.420363+0000", 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.420361+0000", 2024-02-18T00:36:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0970000000000006e-06 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.420369+0000", 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7319999999999993e-06 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.420379+0000", 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8830000000000006e-06 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.420387+0000", 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00071904800000000002 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.421106+0000", 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2949999999999994e-06 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.421114+0000", 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.13593359799999999 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.557048+0000", 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011107600000000001 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.557159+0000", 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014904600000000001 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.557308+0000", 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00109789 2024-02-18T00:36:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.558406+0000", 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000249273 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.558655+0000", 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.01413883 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.572794+0000", 2024-02-18T00:36:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9513e-05 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.572824+0000", 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1046000000000001e-05 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.572845+0000", 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.152481588 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24841.0:2 16.d 16:b4e7e639:test-rados-api-smithi059-83061-3::testobj:head [write 0~8] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.424673+0000", 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 202.28110138299999, 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.146420884, 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24841", 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1293437097", 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424673+0000", 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424673+0000", 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424671+0000", 2024-02-18T00:36:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1099999999999999e-06 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424678+0000", 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0719e-05 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424688+0000", 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.401e-06 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424696+0000", 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1282e-05 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424757+0000", 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5939999999999996e-06 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.424765+0000", 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.062247648000000003 2024-02-18T00:36:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.487012+0000", 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012895199999999999 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.487141+0000", 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014848500000000001 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.487290+0000", 2024-02-18T00:36:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051783319999999999 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.492468+0000", 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.063695747999999996 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.556164+0000", 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014833331 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.570997+0000", 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3043e-05 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.571050+0000", 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3775e-05 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.571094+0000", 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.146420884 2024-02-18T00:36:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2069 17.3 e108/104 17:c1ec3694:test-rados-api-smithi059-82949-17::big:head v 108'1, mlcod=0'0)", 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.047180+0000", 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 199.65859472299999, 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.063788946999999999, 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.047180+0000", 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.047180+0000", 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.047178+0000", 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014896659 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062075+0000", 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8931999999999999e-05 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062114+0000", 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.291e-05 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062127+0000", 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4231999999999998e-05 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062181+0000", 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1489e-05 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062203+0000", 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.048738034 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.110941+0000", 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7817000000000002e-05 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.110969+0000", 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.063788946999999999 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:20 22.1 22:83282cb0:::1.obj2:head [write 0~90] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.153790+0000", 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 199.551984215, 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.047448522, 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 20 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153790+0000", 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153790+0000", 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153789+0000", 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9100000000000002e-06 2024-02-18T00:36:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153795+0000", 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9150000000000002e-06 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153804+0000", 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5639999999999998e-06 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153808+0000", 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4737e-05 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153833+0000", 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1096999999999999e-05 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.153924+0000", 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011127499999999999 2024-02-18T00:36:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154035+0000", 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00049524499999999995 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154530+0000", 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000242017 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154772+0000", 2024-02-18T00:36:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.046363580000000001 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.201136+0000", 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7405999999999999e-05 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.201193+0000", 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5201999999999999e-05 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.201239+0000", 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.047448522 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2074 17.1c e108/104 17:38a558b5:test-rados-api-smithi059-82949-17::big3:head v 108'2, mlcod=0'0)", 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.249369+0000", 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 199.45640548200001, 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.072395273999999996, 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.249369+0000", 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.249369+0000", 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.249368+0000", 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042574984000000003 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.291943+0000", 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2796e-05 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.291966+0000", 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0291e-05 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.291976+0000", 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6464000000000006e-05 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.292043+0000", 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0069999999999999e-05 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.292063+0000", 2024-02-18T00:36:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.029671771 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321735+0000", 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9620000000000001e-05 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321764+0000", 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.072395273999999996 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:9 66.12 66:4993f5f1:test-rados-api-smithi059-82981-8::foo:head [delete] snapc 3=[3,2] ondisk+write+peerstat_old+known_if_redirected e130)", 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.041099+0000", 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 175.66467545099999, 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099041255999999994, 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 9 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041099+0000", 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041099+0000", 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041097+0000", 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0064999999999999e-05 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041107+0000", 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4542000000000001e-05 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041121+0000", 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6449999999999994e-06 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041130+0000", 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0083999999999997e-05 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041180+0000", 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9109000000000003e-05 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041279+0000", 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015191400000000001 2024-02-18T00:36:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.041431+0000", 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012258410000000001 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.042657+0000", 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021696099999999999 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.064353+0000", 2024-02-18T00:36:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.075678602999999997 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.140032+0000", 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0707e-05 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.140092+0000", 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7871000000000003e-05 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.140140+0000", 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099041255999999994 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:28 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 0~128] snapc a=[a] ondisk+write+known_if_redirected e136)", 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.222686+0000", 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 169.483088044, 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065099489999999996, 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 28 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222686+0000", 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222686+0000", 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222685+0000", 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2450000000000001e-06 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222692+0000", 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2241e-05 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222704+0000", 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2179999999999996e-06 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222710+0000", 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5794999999999998e-05 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222756+0000", 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8658000000000003e-05 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222855+0000", 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016973699999999999 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.223025+0000", 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00068663000000000003 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.223711+0000", 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051602410000000003 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.228871+0000", 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.058841457 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.287713+0000", 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7420999999999999e-05 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.287760+0000", 2024-02-18T00:36:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5474999999999999e-05 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.287786+0000", 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065099489999999996 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:29 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 256~128] snapc a=[a] ondisk+write+known_if_redirected e136)", 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.288196+0000", 2024-02-18T00:36:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 169.417578265, 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.070967524000000004, 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 29 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288196+0000", 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288196+0000", 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288195+0000", 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8059999999999995e-06 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288202+0000", 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1141000000000001e-05 2024-02-18T00:36:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288214+0000", 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.922e-06 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288220+0000", 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6390000000000002e-05 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288257+0000", 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0473000000000003e-05 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288307+0000", 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6013000000000004e-05 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288403+0000", 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00042509999999999998 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288828+0000", 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012590334999999999 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.301419+0000", 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.057660309 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.359079+0000", 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5225000000000002e-05 2024-02-18T00:36:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.359124+0000", 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9319000000000001e-05 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.359164+0000", 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.070967524000000004 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.5.0:246 17.19 e136/104 -19:9dd58b98:::temp_17.19_0_24256_2:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.397742+0000", 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 169.30803199900001, 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.023808631, 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.397742+0000", 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.397742+0000", 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.397740+0000", 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0053101390000000002 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403050+0000", 2024-02-18T00:36:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4936e-05 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403075+0000", 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1406999999999999e-05 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403086+0000", 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4187999999999997e-05 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403141+0000", 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6684999999999999e-05 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403167+0000", 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018339692000000001 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.421507+0000", 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4008999999999998e-05 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.421551+0000", 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.023808631 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.5.0:247 17.19 e136/104 -19:9dd58b98:::temp_17.19_0_24256_2:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.421187+0000", 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 169.284587101, 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.023175525999999998, 2024-02-18T00:36:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.421187+0000", 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.421187+0000", 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.421185+0000", 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0053502189999999998 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426535+0000", 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7154000000000001e-05 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426563+0000", 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7772000000000001e-05 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426590+0000", 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4081e-05 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426634+0000", 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0268e-05 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.426655+0000", 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017638622999999999 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.444293+0000", 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9444999999999997e-05 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.444363+0000", 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.023175525999999998 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:35 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 128~128] snapc b=[b,a] ondisk+write+known_if_redirected e138)", 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:36.604817+0000", 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 167.100956978, 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.083436367999999997, 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 35 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604817+0000", 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604817+0000", 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604816+0000", 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5280000000000003e-06 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604824+0000", 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2872e-05 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604837+0000", 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0390000000000005e-06 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604845+0000", 2024-02-18T00:36:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1752000000000003e-05 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604887+0000", 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6851000000000001e-05 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604954+0000", 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013683599999999999 2024-02-18T00:36:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.605091+0000", 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00042250399999999998 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.605513+0000", 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034101531999999997 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.639615+0000", 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.048551496 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.688166+0000", 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5745999999999999e-05 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.688212+0000", 2024-02-18T00:36:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1726e-05 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.688254+0000", 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.083436367999999997 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:42 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 128~128] snapc c=[c,b,a] ondisk+write+known_if_redirected e139)", 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:37.597494+0000", 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 166.108279958, 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065026311000000003, 2024-02-18T00:36:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 42 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597494+0000", 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597494+0000", 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597493+0000", 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8929999999999994e-06 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597502+0000", 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1167e-05 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597513+0000", 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7940000000000003e-06 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597521+0000", 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014479580000000001 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.612000+0000", 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0443000000000003e-05 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.612071+0000", 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020784500000000001 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.612279+0000", 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024797320000000001 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.614758+0000", 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015656066999999999 2024-02-18T00:36:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.630414+0000", 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.032035302000000002 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.662450+0000", 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4518999999999999e-05 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.662484+0000", 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6553000000000001e-05 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.662521+0000", 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065026311000000003 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30748.0:9 78.2s0 78:5be0bd51:test-rados-api-smithi059-82973-12::foo:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected e141)", 2024-02-18T00:36:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:39.311751+0000", 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 164.394023041, 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.038382069999999997, 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30748", 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/510272679", 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 9 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311751+0000", 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311751+0000", 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311750+0000", 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5599999999999996e-06 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311757+0000", 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0591000000000001e-05 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311768+0000", 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0659999999999998e-06 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311775+0000", 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000115534 2024-02-18T00:36:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311890+0000", 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000107452 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311998+0000", 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00065203199999999998 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.312650+0000", 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058148760000000001 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.318465+0000", 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031589093999999998 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.350054+0000", 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9618999999999998e-05 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.350133+0000", 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.038382069999999997 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30876.0:1 167.6s0 167:68f45781:::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e257)", 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:35.626513+0000", 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.079260896000001, 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0131212380000001, 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30876", 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/4143483286", 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626513+0000", 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626513+0000", 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626512+0000", 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0230000000000004e-06 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626519+0000", 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0539999999999995e-06 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626527+0000", 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7159999999999997e-06 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626533+0000", 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8000000000000002e-05 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626571+0000", 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.484e-06 2024-02-18T00:36:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626575+0000", 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0085786489999999 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.635154+0000", 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015310399999999999 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.635307+0000", 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000564885 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.635872+0000", 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00052139999999999999 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.636393+0000", 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0031555540000000001 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.639549+0000", 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6174999999999994e-05 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.639635+0000", 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0131212380000001 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30897.0:1 171.7s0 171:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:42.665607+0000", 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 41.040167220000001, 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.96649063599999996, 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30897", 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/450379973", 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665607+0000", 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665607+0000", 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665605+0000", 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3909999999999998e-06 2024-02-18T00:36:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665613+0000", 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.218e-05 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665625+0000", 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5990000000000002e-06 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665634+0000", 2024-02-18T00:36:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5861e-05 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665680+0000", 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9500000000000001e-06 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665683+0000", 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.96409705300000004 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.629780+0000", 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000143356 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.629923+0000", 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00055660099999999999 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.630480+0000", 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00054972099999999998 2024-02-18T00:36:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.631029+0000", 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00098787500000000004 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.632017+0000", 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0391999999999999e-05 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.632098+0000", 2024-02-18T00:36:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.96649063599999996 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.2.18:31 62.17 62:ee291d87:test-rados-api-smithi059-82957-38::foo:head [copy-from ver 2] snapc 0=[] ondisk+write+ignore_overlay+enforce_snapc+known_if_redirected e304)", 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.99610609000000006, 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027920693, 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:36:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709665+0000", 2024-02-18T00:36:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1831000000000001e-05 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709677+0000", 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7572e-05 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709695+0000", 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1678000000000001e-05 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:36:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709717+0000", 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0013999999999998e-05 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709777+0000", 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015211700000000001 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709929+0000", 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014904904 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 3,5", 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.724834+0000", 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0095493409999999994 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.734383+0000", 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4509999999999997e-06 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.734389+0000", 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003096742 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737486+0000", 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6187000000000003e-05 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737542+0000", 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6628000000000001e-05 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737589+0000", 2024-02-18T00:36:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027920693 2024-02-18T00:36:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:36:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:36:23.833 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:23.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:23.861 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:23.912 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:23.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:23.915 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:24.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:24.045 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:24.231 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:23.627216+0000 mon.a (mon.0) 2080 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:23.633166+0000 mon.a (mon.0) 2081 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:23.633247+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.59:0/3626307759' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:23.633306+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:23.633364+0000 mon.a (mon.0) 2084 : cluster [DBG] osdmap e305: 8 total, 8 up, 7 in 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:23.635442+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:23.636361+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:23.638122+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:23.627216+0000 mon.a (mon.0) 2080 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:23.633166+0000 mon.a (mon.0) 2081 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:23.633247+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.59:0/3626307759' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:23.633306+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:23.633364+0000 mon.a (mon.0) 2084 : cluster [DBG] osdmap e305: 8 total, 8 up, 7 in 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:23.635442+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:23.636361+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:23.638122+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:23.627216+0000 mon.a (mon.0) 2080 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:23.633166+0000 mon.a (mon.0) 2081 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:23.633247+0000 mon.a (mon.0) 2082 : audit [INF] from='client.? 172.21.15.59:0/3626307759' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:23.633306+0000 mon.a (mon.0) 2083 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:23.633364+0000 mon.a (mon.0) 2084 : cluster [DBG] osdmap e305: 8 total, 8 up, 7 in 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:23.635442+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]: dispatch 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:23.636361+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:23.638122+0000 mon.a (mon.0) 2086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]: dispatch 2024-02-18T00:36:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:24.774 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:24.776 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:24.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:24.879 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:25.005 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:25.007 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:25.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:25.109 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:25.136 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:25.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:23.865772+0000 mgr.y (mgr.24454) 312 : cluster [DBG] pgmap v445: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:24.630042+0000 mon.a (mon.0) 2087 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.637053+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.637145+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]': finished 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:24.637217+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e306: 8 total, 8 up, 7 in 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.661751+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.662146+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.663270+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.663587+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.664063+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.664388+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:25.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:24.899474+0000 mon.a (mon.0) 2094 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:25.337 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:25.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:25.353 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:25.438 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:25.464 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:23.865772+0000 mgr.y (mgr.24454) 312 : cluster [DBG] pgmap v445: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:24.630042+0000 mon.a (mon.0) 2087 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.637053+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.637145+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]': finished 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:24.637217+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e306: 8 total, 8 up, 7 in 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.661751+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.662146+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.663270+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.663587+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.664063+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.664388+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:24.899474+0000 mon.a (mon.0) 2094 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:23.865772+0000 mgr.y (mgr.24454) 312 : cluster [DBG] pgmap v445: 238 pgs: 238 active+clean; 6.8 KiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:24.630042+0000 mon.a (mon.0) 2087 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.637053+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.59:0/2265555398' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi059-82882-52"}]': finished 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.637145+0000 mon.a (mon.0) 2089 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-37"}]': finished 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:24.637217+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e306: 8 total, 8 up, 7 in 2024-02-18T00:36:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.661751+0000 mon.c (mon.2) 273 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.662146+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.663270+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.663587+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:25.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.664063+0000 mon.c (mon.2) 275 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:25.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.664388+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:25.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:24.899474+0000 mon.a (mon.0) 2094 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:26.112 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:26.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:26.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:26.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:26.243 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:26.320 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:26.323 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:26.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:26.360 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:26.360 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:36:26.360 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:36:26.360 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:26.424 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:26.452 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:25.639285+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:25.639403+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e307: 8 total, 8 up, 7 in 2024-02-18T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:25.641534+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:25.642120+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:26.644 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:25.866487+0000 mgr.y (mgr.24454) 313 : cluster [DBG] pgmap v448: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-18T00:36:26.649 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:25.639285+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:26.649 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:25.639403+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e307: 8 total, 8 up, 7 in 2024-02-18T00:36:26.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:25.641534+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:26.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:25.642120+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:26.650 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:25.866487+0000 mgr.y (mgr.24454) 313 : cluster [DBG] pgmap v448: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-18T00:36:26.651 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:26.656 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:26.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:26.758 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:26.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:36:26.785 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:27.001 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:25.639285+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi059-82882-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:25.639403+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e307: 8 total, 8 up, 7 in 2024-02-18T00:36:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:25.641534+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:25.642120+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:25.866487+0000 mgr.y (mgr.24454) 313 : cluster [DBG] pgmap v448: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-18T00:36:27.062 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-18T00:36:27.134 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:27.137 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:27.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:27.238 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:27.264 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:27.289 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:27.292 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:27.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:27.377 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:27.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:27.422 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:27.870 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:27.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:27.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:27.954 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:36:27.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 -- 172.21.15.59:0/1476893305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800f8420 msgr2=0x7efc80104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:27.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 --2- 172.21.15.59:0/1476893305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800f8420 0x7efc80104a20 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 rx=0x7efc78009be0 tx=0x7efc78004d10).stop 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 -- 172.21.15.59:0/1476893305 shutdown_connections 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 --2- 172.21.15.59:0/1476893305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800f8420 0x7efc80104a20 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 --2- 172.21.15.59:0/1476893305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800fdbb0 0x7efc800fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 --2- 172.21.15.59:0/1476893305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc800fe510 0x7efc800f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 -- 172.21.15.59:0/1476893305 >> 172.21.15.59:0/1476893305 conn(0x7efc800f3800 msgr2=0x7efc800f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 -- 172.21.15.59:0/1476893305 shutdown_connections 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.536+0000 7efc84922700 1 -- 172.21.15.59:0/1476893305 wait complete. 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 librados: starting msgr at 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 librados: starting objecter 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 Processor -- start 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 -- start start 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 librados: setting wanted keys 2024-02-18T00:36:27.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 librados: calling monclient init 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800f8420 0x7efc8010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc800fdbb0 0x7efc8010e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800fe510 0x7efc80107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc80108480 con 0x7efc800f8420 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc801085f0 con 0x7efc800fdbb0 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc84922700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc801087b0 con 0x7efc800fe510 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc7e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800f8420 0x7efc8010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc7ed9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800fe510 0x7efc80107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc7e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800f8420 0x7efc8010ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37646/0 (socket says 172.21.15.59:37646) 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.537+0000 7efc7e59c700 1 -- 172.21.15.59:0/3804835473 learned_addr learned my addr 172.21.15.59:0/3804835473 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7dd9b700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc800fdbb0 0x7efc8010e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7e59c700 1 -- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800fe510 msgr2=0x7efc80107f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:27.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7e59c700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800fe510 0x7efc80107f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7e59c700 1 -- 172.21.15.59:0/3804835473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc800fdbb0 msgr2=0x7efc8010e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7e59c700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc800fdbb0 0x7efc8010e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7e59c700 1 -- 172.21.15.59:0/3804835473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68005040 con 0x7efc800f8420 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7ed9d700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800fe510 0x7efc80107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc7e59c700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800f8420 0x7efc8010ddf0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7efc680048f0 tx=0x7efc68009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc777fe700 10 client.?.objecter ms_handle_connect 0x7efc800f8420 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc777fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc777fe700 1 -- 172.21.15.59:0/3804835473 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68007b00 con 0x7efc800f8420 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc780097c0 con 0x7efc800f8420 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc777fe700 1 -- 172.21.15.59:0/3804835473 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6800f040 con 0x7efc800f8420 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc84922700 10 client.31062.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc80108df0 con 0x7efc800f8420 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc84922700 1 librados: init done 2024-02-18T00:36:27.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.538+0000 7efc777fe700 1 -- 172.21.15.59:0/3804835473 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc680149a0 con 0x7efc800f8420 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.539+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc80107110 con 0x7efc800f8420 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc777fe700 1 -- 172.21.15.59:0/3804835473 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efc6800e580 con 0x7efc800f8420 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc777fe700 1 --2- 172.21.15.59:0/3804835473 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc6c062c90 0x7efc6c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc777fe700 1 -- 172.21.15.59:0/3804835473 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(308..308 src has 44..308) v4 ==== 8401+0+0 (secure 0 0 0) 0x7efc6807d330 con 0x7efc800f8420 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc777fe700 10 client.31062.objecter ms_dispatch 0x7efc80001550 osd_map(308..308 src has 44..308) v4 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc777fe700 3 client.31062.objecter handle_osd_map got epochs [308,308] > 0 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc777fe700 3 client.31062.objecter handle_osd_map decoding full epoch 308 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc7dd9b700 1 --2- 172.21.15.59:0/3804835473 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc6c062c90 0x7efc6c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.540+0000 7efc777fe700 20 client.31062.objecter dump_active .. 0 homeless 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.541+0000 7efc777fe700 10 client.31062.objecter ms_handle_connect 0x7efc6c062c90 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.541+0000 7efc7dd9b700 1 --2- 172.21.15.59:0/3804835473 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc6c062c90 0x7efc6c065150 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 rx=0x7efc80109730 tx=0x7efc70009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.544+0000 7efc777fe700 1 -- 172.21.15.59:0/3804835473 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efc68050640 con 0x7efc800f8420 2024-02-18T00:36:27.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.733+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc8005fb50 con 0x7efc6c062c90 2024-02-18T00:36:27.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.749+0000 7efc777fe700 1 -- 172.21.15.59:0/3804835473 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+612932 (secure 0 0 0) 0x7efc8005fb50 con 0x7efc6c062c90 2024-02-18T00:36:27.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 10 librados: watch_flush enter 2024-02-18T00:36:27.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 10 librados: watch_flush exit 2024-02-18T00:36:27.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 20 client.31062.objecter shutdown clearing up homeless session... 2024-02-18T00:36:27.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 10 client.31062.objecter successfully canceled tick 2024-02-18T00:36:27.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc6c062c90 msgr2=0x7efc6c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 --2- 172.21.15.59:0/3804835473 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc6c062c90 0x7efc6c065150 secure :-1 s=READY pgs=3064 cs=0 l=1 rev1=1 rx=0x7efc80109730 tx=0x7efc70009040).stop 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800f8420 msgr2=0x7efc8010ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800f8420 0x7efc8010ddf0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7efc680048f0 tx=0x7efc68009cb0).stop 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 shutdown_connections 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc800fe510 0x7efc80107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc800f8420 0x7efc8010ddf0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 --2- 172.21.15.59:0/3804835473 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc6c062c90 0x7efc6c065150 unknown :-1 s=CLOSED pgs=3064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 --2- 172.21.15.59:0/3804835473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc800fdbb0 0x7efc8010e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 >> 172.21.15.59:0/3804835473 conn(0x7efc800f3800 msgr2=0x7efc800f5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 shutdown_connections 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 -- 172.21.15.59:0/3804835473 wait complete. 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:27.955+0000 7efc84922700 1 librados: shutdown 2024-02-18T00:36:27.972 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:36:27.975 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:27.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:27.083425+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e308: 8 total, 8 up, 7 in 2024-02-18T00:36:27.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:27.084923+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:27.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:27.085413+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:27.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:27.083425+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e308: 8 total, 8 up, 7 in 2024-02-18T00:36:27.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:27.084923+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:27.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:27.085413+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:28.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:27.083425+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e308: 8 total, 8 up, 7 in 2024-02-18T00:36:28.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:27.084923+0000 mon.c (mon.2) 277 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:28.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:27.085413+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:28.006 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:27.650827+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:27.651066+0000 mon.a (mon.0) 2101 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:27.651154+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e309: 8 total, 8 up, 7 in 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:27.652806+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:27.657710+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:27.734459+0000 mgr.y (mgr.24454) 314 : audit [DBG] from='client.31062 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:27.866899+0000 mgr.y (mgr.24454) 315 : cluster [DBG] pgmap v451: 246 pgs: 40 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T00:36:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:27.650827+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:27.651066+0000 mon.a (mon.0) 2101 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:27.651154+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e309: 8 total, 8 up, 7 in 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:27.652806+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:27.657710+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:27.734459+0000 mgr.y (mgr.24454) 314 : audit [DBG] from='client.31062 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:27.866899+0000 mgr.y (mgr.24454) 315 : cluster [DBG] pgmap v451: 246 pgs: 40 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:27.650827+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi059-82882-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:29.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:27.651066+0000 mon.a (mon.0) 2101 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:27.651154+0000 mon.a (mon.0) 2102 : cluster [DBG] osdmap e309: 8 total, 8 up, 7 in 2024-02-18T00:36:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:27.652806+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:27.657710+0000 mon.a (mon.0) 2103 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:27.734459+0000 mgr.y (mgr.24454) 314 : audit [DBG] from='client.31062 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:27.866899+0000 mgr.y (mgr.24454) 315 : cluster [DBG] pgmap v451: 246 pgs: 40 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T00:36:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:29.113 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-02-18T00:36:27.648241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007021,"num_objects":348,"num_object_clones":2,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":348,"num_whiteouts":0,"num_read":1622,"num_read_kb":1250,"num_write":1174,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2518,"ondisk_log_size":2518,"up":618,"acting":618,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":702,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2168284,"kb_used_data":46696,"kb_used_omap":0,"kb_used_meta":2121472,"kb_avail":654047780,"statfs":{"total":671965249536,"available":669744926720,"internally_reserved":0,"allocated":47816704,"data_stored":31817661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2172387328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4000008,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":3907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017998"},"pg_stats":[{"pgid":"188.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"189.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:27.642635+0000","last_change":"2024-02-18T00:36:27.642635+0000","last_active":"2024-02-18T00:36:27.642635+0000","last_peered":"2024-02-18T00:36:27.642635+0000","last_clean":"2024-02-18T00:36:27.642635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:27.642635+0000","last_undegraded":"2024-02-18T00:36:27.642635+0000","last_fullsized":"2024-02-18T00:36:27.642635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:27.642635+0000","last_clean_scrub_stamp":"2024-02-18T00:36:27.642635+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"188.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:36:26.636261+0000","last_change":"2024-02-18T00:36:26.636261+0000","last_active":"2024-02-18T00:36:26.636261+0000","last_peered":"2024-02-18T00:36:26.636261+0000","last_clean":"2024-02-18T00:36:26.636261+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:26.636261+0000","last_undegraded":"2024-02-18T00:36:26.636261+0000","last_fullsized":"2024-02-18T00:36:26.636261+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:26.636261+0000","last_clean_scrub_stamp":"2024-02-18T00:36:26.636261+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":278,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651356+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:36:24.651356+0000","last_peered":"2024-02-18T00:36:24.651356+0000","last_clean":"2024-02-18T00:36:24.651356+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:36:24.651356+0000","last_undegraded":"2024-02-18T00:36:24.651356+0000","last_fullsized":"2024-02-18T00:36:24.651356+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"0'0","reported_seq":240,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.658607+0000","last_change":"2024-02-18T00:36:06.658623+0000","last_active":"2024-02-18T00:36:24.658607+0000","last_peered":"2024-02-18T00:36:24.658607+0000","last_clean":"2024-02-18T00:36:24.658607+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:36:24.658607+0000","last_undegraded":"2024-02-18T00:36:24.658607+0000","last_fullsized":"2024-02-18T00:36:24.658607+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":308,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648355+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:36:24.648355+0000","last_peered":"2024-02-18T00:36:24.648355+0000","last_clean":"2024-02-18T00:36:24.648355+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:36:24.648355+0000","last_undegraded":"2024-02-18T00:36:24.648355+0000","last_fullsized":"2024-02-18T00:36:24.648355+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"245'73","reported_seq":302,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652108+0000","last_change":"2024-02-18T00:36:06.648070+0000","last_active":"2024-02-18T00:36:24.652108+0000","last_peered":"2024-02-18T00:36:24.652108+0000","last_clean":"2024-02-18T00:36:24.652108+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:36:24.652108+0000","last_undegraded":"2024-02-18T00:36:24.652108+0000","last_fullsized":"2024-02-18T00:36:24.652108+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"230'120","reported_seq":341,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636676+0000","last_change":"2024-02-18T00:36:06.647751+0000","last_active":"2024-02-18T00:36:24.636676+0000","last_peered":"2024-02-18T00:36:24.636676+0000","last_clean":"2024-02-18T00:36:24.636676+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:36:24.636676+0000","last_undegraded":"2024-02-18T00:36:24.636676+0000","last_fullsized":"2024-02-18T00:36:24.636676+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":322,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645863+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:36:24.645863+0000","last_peered":"2024-02-18T00:36:24.645863+0000","last_clean":"2024-02-18T00:36:24.645863+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:36:24.645863+0000","last_undegraded":"2024-02-18T00:36:24.645863+0000","last_fullsized":"2024-02-18T00:36:24.645863+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"267'124","reported_seq":315,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652558+0000","last_change":"2024-02-18T00:36:06.654594+0000","last_active":"2024-02-18T00:36:24.652558+0000","last_peered":"2024-02-18T00:36:24.652558+0000","last_clean":"2024-02-18T00:36:24.652558+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:36:24.652558+0000","last_undegraded":"2024-02-18T00:36:24.652558+0000","last_fullsized":"2024-02-18T00:36:24.652558+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":124,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"267'73","reported_seq":296,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652728+0000","last_change":"2024-02-18T00:36:06.651545+0000","last_active":"2024-02-18T00:36:24.652728+0000","last_peered":"2024-02-18T00:36:24.652728+0000","last_clean":"2024-02-18T00:36:24.652728+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:36:24.652728+0000","last_undegraded":"2024-02-18T00:36:24.652728+0000","last_fullsized":"2024-02-18T00:36:24.652728+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"215'68","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.658579+0000","last_change":"2024-02-18T00:36:06.662037+0000","last_active":"2024-02-18T00:36:24.658579+0000","last_peered":"2024-02-18T00:36:24.658579+0000","last_clean":"2024-02-18T00:36:24.658579+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:36:24.658579+0000","last_undegraded":"2024-02-18T00:36:24.658579+0000","last_fullsized":"2024-02-18T00:36:24.658579+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-18T00:36:02.645624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.645624+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"229'51","reported_seq":284,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645338+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:36:24.645338+0000","last_peered":"2024-02-18T00:36:24.645338+0000","last_clean":"2024-02-18T00:36:24.645338+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:36:24.645338+0000","last_undegraded":"2024-02-18T00:36:24.645338+0000","last_fullsized":"2024-02-18T00:36:24.645338+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"265'39","reported_seq":250,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651837+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:36:24.651837+0000","last_peered":"2024-02-18T00:36:24.651837+0000","last_clean":"2024-02-18T00:36:24.651837+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:36:24.651837+0000","last_undegraded":"2024-02-18T00:36:24.651837+0000","last_fullsized":"2024-02-18T00:36:24.651837+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"246'54","reported_seq":264,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642585+0000","last_change":"2024-02-18T00:36:11.810050+0000","last_active":"2024-02-18T00:36:24.642585+0000","last_peered":"2024-02-18T00:36:24.642585+0000","last_clean":"2024-02-18T00:36:24.642585+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:36:24.642585+0000","last_undegraded":"2024-02-18T00:36:24.642585+0000","last_fullsized":"2024-02-18T00:36:24.642585+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'54","last_scrub_stamp":"2024-02-18T00:36:11.809925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.809925+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"213'17","reported_seq":255,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636605+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:36:24.636605+0000","last_peered":"2024-02-18T00:36:24.636605+0000","last_clean":"2024-02-18T00:36:24.636605+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:36:24.636605+0000","last_undegraded":"2024-02-18T00:36:24.636605+0000","last_fullsized":"2024-02-18T00:36:24.636605+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'28","reported_seq":288,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645735+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:36:24.645735+0000","last_peered":"2024-02-18T00:36:24.645735+0000","last_clean":"2024-02-18T00:36:24.645735+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:36:24.645735+0000","last_undegraded":"2024-02-18T00:36:24.645735+0000","last_fullsized":"2024-02-18T00:36:24.645735+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"288'81","reported_seq":307,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.638621+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:36:24.638621+0000","last_peered":"2024-02-18T00:36:24.638621+0000","last_clean":"2024-02-18T00:36:24.638621+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:36:24.638621+0000","last_undegraded":"2024-02-18T00:36:24.638621+0000","last_fullsized":"2024-02-18T00:36:24.638621+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":81,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":284,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646523+0000","last_change":"2024-02-18T00:36:06.652954+0000","last_active":"2024-02-18T00:36:24.646523+0000","last_peered":"2024-02-18T00:36:24.646523+0000","last_clean":"2024-02-18T00:36:24.646523+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:36:24.646523+0000","last_undegraded":"2024-02-18T00:36:24.646523+0000","last_fullsized":"2024-02-18T00:36:24.646523+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"209'56","reported_seq":302,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.658471+0000","last_change":"2024-02-18T00:36:06.661736+0000","last_active":"2024-02-18T00:36:24.658471+0000","last_peered":"2024-02-18T00:36:24.658471+0000","last_clean":"2024-02-18T00:36:24.658471+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:36:24.658471+0000","last_undegraded":"2024-02-18T00:36:24.658471+0000","last_fullsized":"2024-02-18T00:36:24.658471+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'56","last_scrub_stamp":"2024-02-18T00:36:04.656413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.656413+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"214'19","reported_seq":260,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.658550+0000","last_change":"2024-02-18T00:36:06.658579+0000","last_active":"2024-02-18T00:36:24.658550+0000","last_peered":"2024-02-18T00:36:24.658550+0000","last_clean":"2024-02-18T00:36:24.658550+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:36:24.658550+0000","last_undegraded":"2024-02-18T00:36:24.658550+0000","last_fullsized":"2024-02-18T00:36:24.658550+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:35:53.563562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.563562+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"288'72","reported_seq":302,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652308+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:36:24.652308+0000","last_peered":"2024-02-18T00:36:24.652308+0000","last_clean":"2024-02-18T00:36:24.652308+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:36:24.652308+0000","last_undegraded":"2024-02-18T00:36:24.652308+0000","last_fullsized":"2024-02-18T00:36:24.652308+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"210'17","reported_seq":250,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.638801+0000","last_change":"2024-02-18T00:36:06.649777+0000","last_active":"2024-02-18T00:36:24.638801+0000","last_peered":"2024-02-18T00:36:24.638801+0000","last_clean":"2024-02-18T00:36:24.638801+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:36:24.638801+0000","last_undegraded":"2024-02-18T00:36:24.638801+0000","last_fullsized":"2024-02-18T00:36:24.638801+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646299+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:36:24.646299+0000","last_peered":"2024-02-18T00:36:24.646299+0000","last_clean":"2024-02-18T00:36:24.646299+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:36:24.646299+0000","last_undegraded":"2024-02-18T00:36:24.646299+0000","last_fullsized":"2024-02-18T00:36:24.646299+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"304'19","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648808+0000","last_change":"2024-02-18T00:36:06.659238+0000","last_active":"2024-02-18T00:36:24.648808+0000","last_peered":"2024-02-18T00:36:24.648808+0000","last_clean":"2024-02-18T00:36:24.648808+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:36:24.648808+0000","last_undegraded":"2024-02-18T00:36:24.648808+0000","last_fullsized":"2024-02-18T00:36:24.648808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000019,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":329,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636891+0000","last_change":"2024-02-18T00:36:06.663924+0000","last_active":"2024-02-18T00:36:24.636891+0000","last_peered":"2024-02-18T00:36:24.636891+0000","last_clean":"2024-02-18T00:36:24.636891+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:36:24.636891+0000","last_undegraded":"2024-02-18T00:36:24.636891+0000","last_fullsized":"2024-02-18T00:36:24.636891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":221,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.647992+0000","last_change":"2024-02-18T00:34:15.230999+0000","last_active":"2024-02-18T00:36:24.647992+0000","last_peered":"2024-02-18T00:36:24.647992+0000","last_clean":"2024-02-18T00:36:24.647992+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:36:24.647992+0000","last_undegraded":"2024-02-18T00:36:24.647992+0000","last_fullsized":"2024-02-18T00:36:24.647992+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.230938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.230938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":341,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652048+0000","last_change":"2024-02-18T00:36:06.651013+0000","last_active":"2024-02-18T00:36:24.652048+0000","last_peered":"2024-02-18T00:36:24.652048+0000","last_clean":"2024-02-18T00:36:24.652048+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:36:24.652048+0000","last_undegraded":"2024-02-18T00:36:24.652048+0000","last_fullsized":"2024-02-18T00:36:24.652048+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":220,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636232+0000","last_change":"2024-02-18T00:34:39.758860+0000","last_active":"2024-02-18T00:36:24.636232+0000","last_peered":"2024-02-18T00:36:24.636232+0000","last_clean":"2024-02-18T00:36:24.636232+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:36:24.636232+0000","last_undegraded":"2024-02-18T00:36:24.636232+0000","last_fullsized":"2024-02-18T00:36:24.636232+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.758794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.758794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":280,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645778+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:36:24.645778+0000","last_peered":"2024-02-18T00:36:24.645778+0000","last_clean":"2024-02-18T00:36:24.645778+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:36:24.645778+0000","last_undegraded":"2024-02-18T00:36:24.645778+0000","last_fullsized":"2024-02-18T00:36:24.645778+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":177,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646533+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:36:24.646533+0000","last_peered":"2024-02-18T00:36:24.646533+0000","last_clean":"2024-02-18T00:36:24.646533+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:36:24.646533+0000","last_undegraded":"2024-02-18T00:36:24.646533+0000","last_fullsized":"2024-02-18T00:36:24.646533+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":335,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648106+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:36:24.648106+0000","last_peered":"2024-02-18T00:36:24.648106+0000","last_clean":"2024-02-18T00:36:24.648106+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:36:24.648106+0000","last_undegraded":"2024-02-18T00:36:24.648106+0000","last_fullsized":"2024-02-18T00:36:24.648106+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":175,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646351+0000","last_change":"2024-02-18T00:34:26.276135+0000","last_active":"2024-02-18T00:36:24.646351+0000","last_peered":"2024-02-18T00:36:24.646351+0000","last_clean":"2024-02-18T00:36:24.646351+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:36:24.646351+0000","last_undegraded":"2024-02-18T00:36:24.646351+0000","last_fullsized":"2024-02-18T00:36:24.646351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.276071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.276071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":312,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648008+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:36:24.648008+0000","last_peered":"2024-02-18T00:36:24.648008+0000","last_clean":"2024-02-18T00:36:24.648008+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:36:24.648008+0000","last_undegraded":"2024-02-18T00:36:24.648008+0000","last_fullsized":"2024-02-18T00:36:24.648008+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":227,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646308+0000","last_change":"2024-02-18T00:33:43.346148+0000","last_active":"2024-02-18T00:36:24.646308+0000","last_peered":"2024-02-18T00:36:24.646308+0000","last_clean":"2024-02-18T00:36:24.646308+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:36:24.646308+0000","last_undegraded":"2024-02-18T00:36:24.646308+0000","last_fullsized":"2024-02-18T00:36:24.646308+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:43.346088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.346088+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"213'17","reported_seq":261,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636217+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:36:24.636217+0000","last_peered":"2024-02-18T00:36:24.636217+0000","last_clean":"2024-02-18T00:36:24.636217+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:36:24.636217+0000","last_undegraded":"2024-02-18T00:36:24.636217+0000","last_fullsized":"2024-02-18T00:36:24.636217+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":221,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648766+0000","last_change":"2024-02-18T00:34:17.271924+0000","last_active":"2024-02-18T00:36:24.648766+0000","last_peered":"2024-02-18T00:36:24.648766+0000","last_clean":"2024-02-18T00:36:24.648766+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:36:24.648766+0000","last_undegraded":"2024-02-18T00:36:24.648766+0000","last_fullsized":"2024-02-18T00:36:24.648766+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.271867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.271867+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":286,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652752+0000","last_change":"2024-02-18T00:36:06.651247+0000","last_active":"2024-02-18T00:36:24.652752+0000","last_peered":"2024-02-18T00:36:24.652752+0000","last_clean":"2024-02-18T00:36:24.652752+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:36:24.652752+0000","last_undegraded":"2024-02-18T00:36:24.652752+0000","last_fullsized":"2024-02-18T00:36:24.652752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":211,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642553+0000","last_change":"2024-02-18T00:33:55.783013+0000","last_active":"2024-02-18T00:36:24.642553+0000","last_peered":"2024-02-18T00:36:24.642553+0000","last_clean":"2024-02-18T00:36:24.642553+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:36:24.642553+0000","last_undegraded":"2024-02-18T00:36:24.642553+0000","last_fullsized":"2024-02-18T00:36:24.642553+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.782955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.782955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":246,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648761+0000","last_change":"2024-02-18T00:36:06.659436+0000","last_active":"2024-02-18T00:36:24.648761+0000","last_peered":"2024-02-18T00:36:24.648761+0000","last_clean":"2024-02-18T00:36:24.648761+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:36:24.648761+0000","last_undegraded":"2024-02-18T00:36:24.648761+0000","last_fullsized":"2024-02-18T00:36:24.648761+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":219,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648983+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:36:24.648983+0000","last_peered":"2024-02-18T00:36:24.648983+0000","last_clean":"2024-02-18T00:36:24.648983+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:36:24.648983+0000","last_undegraded":"2024-02-18T00:36:24.648983+0000","last_fullsized":"2024-02-18T00:36:24.648983+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":256,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641242+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:36:24.641242+0000","last_peered":"2024-02-18T00:36:24.641242+0000","last_clean":"2024-02-18T00:36:24.641242+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:36:24.641242+0000","last_undegraded":"2024-02-18T00:36:24.641242+0000","last_fullsized":"2024-02-18T00:36:24.641242+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":292,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651970+0000","last_change":"2024-02-18T00:33:48.736504+0000","last_active":"2024-02-18T00:36:24.651970+0000","last_peered":"2024-02-18T00:36:24.651970+0000","last_clean":"2024-02-18T00:36:24.651970+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:36:24.651970+0000","last_undegraded":"2024-02-18T00:36:24.651970+0000","last_fullsized":"2024-02-18T00:36:24.651970+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.834878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.834878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":257,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648605+0000","last_change":"2024-02-18T00:36:06.657624+0000","last_active":"2024-02-18T00:36:24.648605+0000","last_peered":"2024-02-18T00:36:24.648605+0000","last_clean":"2024-02-18T00:36:24.648605+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:36:24.648605+0000","last_undegraded":"2024-02-18T00:36:24.648605+0000","last_fullsized":"2024-02-18T00:36:24.648605+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":245,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640282+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:36:24.640282+0000","last_peered":"2024-02-18T00:36:24.640282+0000","last_clean":"2024-02-18T00:36:24.640282+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:36:24.640282+0000","last_undegraded":"2024-02-18T00:36:24.640282+0000","last_fullsized":"2024-02-18T00:36:24.640282+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":222,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640242+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:36:24.640242+0000","last_peered":"2024-02-18T00:36:24.640242+0000","last_clean":"2024-02-18T00:36:24.640242+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:36:24.640242+0000","last_undegraded":"2024-02-18T00:36:24.640242+0000","last_fullsized":"2024-02-18T00:36:24.640242+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":229,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648650+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:36:24.648650+0000","last_peered":"2024-02-18T00:36:24.648650+0000","last_clean":"2024-02-18T00:36:24.648650+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:36:24.648650+0000","last_undegraded":"2024-02-18T00:36:24.648650+0000","last_fullsized":"2024-02-18T00:36:24.648650+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":355,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640390+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:36:24.640390+0000","last_peered":"2024-02-18T00:36:24.640390+0000","last_clean":"2024-02-18T00:36:24.640390+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:36:24.640390+0000","last_undegraded":"2024-02-18T00:36:24.640390+0000","last_fullsized":"2024-02-18T00:36:24.640390+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":285,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652771+0000","last_change":"2024-02-18T00:34:26.060181+0000","last_active":"2024-02-18T00:36:24.652771+0000","last_peered":"2024-02-18T00:36:24.652771+0000","last_clean":"2024-02-18T00:36:24.652771+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:36:24.652771+0000","last_undegraded":"2024-02-18T00:36:24.652771+0000","last_fullsized":"2024-02-18T00:36:24.652771+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.060096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.060096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640660+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:36:24.640660+0000","last_peered":"2024-02-18T00:36:24.640660+0000","last_clean":"2024-02-18T00:36:24.640660+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:36:24.640660+0000","last_undegraded":"2024-02-18T00:36:24.640660+0000","last_fullsized":"2024-02-18T00:36:24.640660+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.661080+0000","last_change":"2024-02-18T00:35:20.428964+0000","last_active":"2024-02-18T00:36:24.661080+0000","last_peered":"2024-02-18T00:36:24.661080+0000","last_clean":"2024-02-18T00:36:24.661080+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:36:24.661080+0000","last_undegraded":"2024-02-18T00:36:24.661080+0000","last_fullsized":"2024-02-18T00:36:24.661080+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:20.428893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:20.428893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":322,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640085+0000","last_change":"2024-02-18T00:35:54.188262+0000","last_active":"2024-02-18T00:36:24.640085+0000","last_peered":"2024-02-18T00:36:24.640085+0000","last_clean":"2024-02-18T00:36:24.640085+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:36:24.640085+0000","last_undegraded":"2024-02-18T00:36:24.640085+0000","last_fullsized":"2024-02-18T00:36:24.640085+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:35:54.188188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.188188+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":274,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.660456+0000","last_change":"2024-02-18T00:34:24.334917+0000","last_active":"2024-02-18T00:36:24.660456+0000","last_peered":"2024-02-18T00:36:24.660456+0000","last_clean":"2024-02-18T00:36:24.660456+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:36:24.660456+0000","last_undegraded":"2024-02-18T00:36:24.660456+0000","last_fullsized":"2024-02-18T00:36:24.660456+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.334814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.334814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651965+0000","last_change":"2024-02-18T00:33:42.768749+0000","last_active":"2024-02-18T00:36:24.651965+0000","last_peered":"2024-02-18T00:36:24.651965+0000","last_clean":"2024-02-18T00:36:24.651965+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:36:24.651965+0000","last_undegraded":"2024-02-18T00:36:24.651965+0000","last_fullsized":"2024-02-18T00:36:24.651965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.768630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.768630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":284,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639222+0000","last_change":"2024-02-18T00:35:42.683115+0000","last_active":"2024-02-18T00:36:24.639222+0000","last_peered":"2024-02-18T00:36:24.639222+0000","last_clean":"2024-02-18T00:36:24.639222+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:36:24.639222+0000","last_undegraded":"2024-02-18T00:36:24.639222+0000","last_fullsized":"2024-02-18T00:36:24.639222+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":222,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.644278+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:36:24.644278+0000","last_peered":"2024-02-18T00:36:24.644278+0000","last_clean":"2024-02-18T00:36:24.644278+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:36:24.644278+0000","last_undegraded":"2024-02-18T00:36:24.644278+0000","last_fullsized":"2024-02-18T00:36:24.644278+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":355,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649366+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:36:24.649366+0000","last_peered":"2024-02-18T00:36:24.649366+0000","last_clean":"2024-02-18T00:36:24.649366+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:36:24.649366+0000","last_undegraded":"2024-02-18T00:36:24.649366+0000","last_fullsized":"2024-02-18T00:36:24.649366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":274,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652675+0000","last_change":"2024-02-18T00:33:46.861306+0000","last_active":"2024-02-18T00:36:24.652675+0000","last_peered":"2024-02-18T00:36:24.652675+0000","last_clean":"2024-02-18T00:36:24.652675+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:36:24.652675+0000","last_undegraded":"2024-02-18T00:36:24.652675+0000","last_fullsized":"2024-02-18T00:36:24.652675+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.861233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.861233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652167+0000","last_change":"2024-02-18T00:33:53.005012+0000","last_active":"2024-02-18T00:36:24.652167+0000","last_peered":"2024-02-18T00:36:24.652167+0000","last_clean":"2024-02-18T00:36:24.652167+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:36:24.652167+0000","last_undegraded":"2024-02-18T00:36:24.652167+0000","last_fullsized":"2024-02-18T00:36:24.652167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.004903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.004903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":174,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651699+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:36:24.651699+0000","last_peered":"2024-02-18T00:36:24.651699+0000","last_clean":"2024-02-18T00:36:24.651699+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:36:24.651699+0000","last_undegraded":"2024-02-18T00:36:24.651699+0000","last_fullsized":"2024-02-18T00:36:24.651699+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640562+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:36:24.640562+0000","last_peered":"2024-02-18T00:36:24.640562+0000","last_clean":"2024-02-18T00:36:24.640562+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:36:24.640562+0000","last_undegraded":"2024-02-18T00:36:24.640562+0000","last_fullsized":"2024-02-18T00:36:24.640562+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":305,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646206+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:36:24.646206+0000","last_peered":"2024-02-18T00:36:24.646206+0000","last_clean":"2024-02-18T00:36:24.646206+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:36:24.646206+0000","last_undegraded":"2024-02-18T00:36:24.646206+0000","last_fullsized":"2024-02-18T00:36:24.646206+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":174,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646079+0000","last_change":"2024-02-18T00:34:18.301213+0000","last_active":"2024-02-18T00:36:24.646079+0000","last_peered":"2024-02-18T00:36:24.646079+0000","last_clean":"2024-02-18T00:36:24.646079+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:36:24.646079+0000","last_undegraded":"2024-02-18T00:36:24.646079+0000","last_fullsized":"2024-02-18T00:36:24.646079+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.301136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.301136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659699+0000","last_change":"2024-02-18T00:34:39.453471+0000","last_active":"2024-02-18T00:36:24.659699+0000","last_peered":"2024-02-18T00:36:24.659699+0000","last_clean":"2024-02-18T00:36:24.659699+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:36:24.659699+0000","last_undegraded":"2024-02-18T00:36:24.659699+0000","last_fullsized":"2024-02-18T00:36:24.659699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.453312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.453312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652893+0000","last_change":"2024-02-18T00:33:45.813625+0000","last_active":"2024-02-18T00:36:24.652893+0000","last_peered":"2024-02-18T00:36:24.652893+0000","last_clean":"2024-02-18T00:36:24.652893+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:36:24.652893+0000","last_undegraded":"2024-02-18T00:36:24.652893+0000","last_fullsized":"2024-02-18T00:36:24.652893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.813514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.813514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":58,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651393+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:36:24.651393+0000","last_peered":"2024-02-18T00:36:24.651393+0000","last_clean":"2024-02-18T00:36:24.651393+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:36:24.651393+0000","last_undegraded":"2024-02-18T00:36:24.651393+0000","last_fullsized":"2024-02-18T00:36:24.651393+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":287,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645832+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:36:24.645832+0000","last_peered":"2024-02-18T00:36:24.645832+0000","last_clean":"2024-02-18T00:36:24.645832+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:36:24.645832+0000","last_undegraded":"2024-02-18T00:36:24.645832+0000","last_fullsized":"2024-02-18T00:36:24.645832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":283,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659167+0000","last_change":"2024-02-18T00:34:23.295161+0000","last_active":"2024-02-18T00:36:24.659167+0000","last_peered":"2024-02-18T00:36:24.659167+0000","last_clean":"2024-02-18T00:36:24.659167+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:36:24.659167+0000","last_undegraded":"2024-02-18T00:36:24.659167+0000","last_fullsized":"2024-02-18T00:36:24.659167+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.295051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":380,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652531+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:36:24.652531+0000","last_peered":"2024-02-18T00:36:24.652531+0000","last_clean":"2024-02-18T00:36:24.652531+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:36:24.652531+0000","last_undegraded":"2024-02-18T00:36:24.652531+0000","last_fullsized":"2024-02-18T00:36:24.652531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646509+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:36:24.646509+0000","last_peered":"2024-02-18T00:36:24.646509+0000","last_clean":"2024-02-18T00:36:24.646509+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:36:24.646509+0000","last_undegraded":"2024-02-18T00:36:24.646509+0000","last_fullsized":"2024-02-18T00:36:24.646509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659334+0000","last_change":"2024-02-18T00:35:29.520701+0000","last_active":"2024-02-18T00:36:24.659334+0000","last_peered":"2024-02-18T00:36:24.659334+0000","last_clean":"2024-02-18T00:36:24.659334+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:36:24.659334+0000","last_undegraded":"2024-02-18T00:36:24.659334+0000","last_fullsized":"2024-02-18T00:36:24.659334+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.520632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.520632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":274,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641438+0000","last_change":"2024-02-18T00:33:44.007884+0000","last_active":"2024-02-18T00:36:24.641438+0000","last_peered":"2024-02-18T00:36:24.641438+0000","last_clean":"2024-02-18T00:36:24.641438+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:36:24.641438+0000","last_undegraded":"2024-02-18T00:36:24.641438+0000","last_fullsized":"2024-02-18T00:36:24.641438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:33:44.007804+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:33:44.007804+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":297,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.658981+0000","last_change":"2024-02-18T00:36:08.716901+0000","last_active":"2024-02-18T00:36:24.658981+0000","last_peered":"2024-02-18T00:36:24.658981+0000","last_clean":"2024-02-18T00:36:24.658981+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:36:24.658981+0000","last_undegraded":"2024-02-18T00:36:24.658981+0000","last_fullsized":"2024-02-18T00:36:24.658981+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.716777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.716777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":364,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646508+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:36:24.646508+0000","last_peered":"2024-02-18T00:36:24.646508+0000","last_clean":"2024-02-18T00:36:24.646508+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:36:24.646508+0000","last_undegraded":"2024-02-18T00:36:24.646508+0000","last_fullsized":"2024-02-18T00:36:24.646508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641493+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:36:24.641493+0000","last_peered":"2024-02-18T00:36:24.641493+0000","last_clean":"2024-02-18T00:36:24.641493+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:36:24.641493+0000","last_undegraded":"2024-02-18T00:36:24.641493+0000","last_fullsized":"2024-02-18T00:36:24.641493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645900+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:36:24.645900+0000","last_peered":"2024-02-18T00:36:24.645900+0000","last_clean":"2024-02-18T00:36:24.645900+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:36:24.645900+0000","last_undegraded":"2024-02-18T00:36:24.645900+0000","last_fullsized":"2024-02-18T00:36:24.645900+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":329,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646389+0000","last_change":"2024-02-18T00:35:43.219869+0000","last_active":"2024-02-18T00:36:24.646389+0000","last_peered":"2024-02-18T00:36:24.646389+0000","last_clean":"2024-02-18T00:36:24.646389+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:36:24.646389+0000","last_undegraded":"2024-02-18T00:36:24.646389+0000","last_fullsized":"2024-02-18T00:36:24.646389+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:45.311259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.311259+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":244,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636268+0000","last_change":"2024-02-18T00:34:27.927933+0000","last_active":"2024-02-18T00:36:24.636268+0000","last_peered":"2024-02-18T00:36:24.636268+0000","last_clean":"2024-02-18T00:36:24.636268+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:36:24.636268+0000","last_undegraded":"2024-02-18T00:36:24.636268+0000","last_fullsized":"2024-02-18T00:36:24.636268+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.927871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.927871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":269,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641565+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:36:24.641565+0000","last_peered":"2024-02-18T00:36:24.641565+0000","last_clean":"2024-02-18T00:36:24.641565+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:36:24.641565+0000","last_undegraded":"2024-02-18T00:36:24.641565+0000","last_fullsized":"2024-02-18T00:36:24.641565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636767+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:36:24.636767+0000","last_peered":"2024-02-18T00:36:24.636767+0000","last_clean":"2024-02-18T00:36:24.636767+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:36:24.636767+0000","last_undegraded":"2024-02-18T00:36:24.636767+0000","last_fullsized":"2024-02-18T00:36:24.636767+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646088+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:36:24.646088+0000","last_peered":"2024-02-18T00:36:24.646088+0000","last_clean":"2024-02-18T00:36:24.646088+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:36:24.646088+0000","last_undegraded":"2024-02-18T00:36:24.646088+0000","last_fullsized":"2024-02-18T00:36:24.646088+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":364,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636672+0000","last_change":"2024-02-18T00:33:54.753052+0000","last_active":"2024-02-18T00:36:24.636672+0000","last_peered":"2024-02-18T00:36:24.636672+0000","last_clean":"2024-02-18T00:36:24.636672+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:36:24.636672+0000","last_undegraded":"2024-02-18T00:36:24.636672+0000","last_fullsized":"2024-02-18T00:36:24.636672+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:33:54.752985+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.752985+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":308,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645476+0000","last_change":"2024-02-18T00:33:53.320429+0000","last_active":"2024-02-18T00:36:24.645476+0000","last_peered":"2024-02-18T00:36:24.645476+0000","last_clean":"2024-02-18T00:36:24.645476+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:36:24.645476+0000","last_undegraded":"2024-02-18T00:36:24.645476+0000","last_fullsized":"2024-02-18T00:36:24.645476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:33:53.320356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.320356+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":297,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648991+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:36:24.648991+0000","last_peered":"2024-02-18T00:36:24.648991+0000","last_clean":"2024-02-18T00:36:24.648991+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:36:24.648991+0000","last_undegraded":"2024-02-18T00:36:24.648991+0000","last_fullsized":"2024-02-18T00:36:24.648991+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":174,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648362+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:36:24.648362+0000","last_peered":"2024-02-18T00:36:24.648362+0000","last_clean":"2024-02-18T00:36:24.648362+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:36:24.648362+0000","last_undegraded":"2024-02-18T00:36:24.648362+0000","last_fullsized":"2024-02-18T00:36:24.648362+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641233+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:36:24.641233+0000","last_peered":"2024-02-18T00:36:24.641233+0000","last_clean":"2024-02-18T00:36:24.641233+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:36:24.641233+0000","last_undegraded":"2024-02-18T00:36:24.641233+0000","last_fullsized":"2024-02-18T00:36:24.641233+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":279,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645631+0000","last_change":"2024-02-18T00:33:50.287103+0000","last_active":"2024-02-18T00:36:24.645631+0000","last_peered":"2024-02-18T00:36:24.645631+0000","last_clean":"2024-02-18T00:36:24.645631+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:36:24.645631+0000","last_undegraded":"2024-02-18T00:36:24.645631+0000","last_fullsized":"2024-02-18T00:36:24.645631+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.287007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.287007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":374,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636568+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:36:24.636568+0000","last_peered":"2024-02-18T00:36:24.636568+0000","last_clean":"2024-02-18T00:36:24.636568+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:36:24.636568+0000","last_undegraded":"2024-02-18T00:36:24.636568+0000","last_fullsized":"2024-02-18T00:36:24.636568+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":288,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.661090+0000","last_change":"2024-02-18T00:36:09.647995+0000","last_active":"2024-02-18T00:36:24.661090+0000","last_peered":"2024-02-18T00:36:24.661090+0000","last_clean":"2024-02-18T00:36:24.661090+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:36:24.661090+0000","last_undegraded":"2024-02-18T00:36:24.661090+0000","last_fullsized":"2024-02-18T00:36:24.661090+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.647928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645741+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:36:24.645741+0000","last_peered":"2024-02-18T00:36:24.645741+0000","last_clean":"2024-02-18T00:36:24.645741+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:36:24.645741+0000","last_undegraded":"2024-02-18T00:36:24.645741+0000","last_fullsized":"2024-02-18T00:36:24.645741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.643205+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:36:24.643205+0000","last_peered":"2024-02-18T00:36:24.643205+0000","last_clean":"2024-02-18T00:36:24.643205+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:36:24.643205+0000","last_undegraded":"2024-02-18T00:36:24.643205+0000","last_fullsized":"2024-02-18T00:36:24.643205+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648880+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:36:24.648880+0000","last_peered":"2024-02-18T00:36:24.648880+0000","last_clean":"2024-02-18T00:36:24.648880+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:36:24.648880+0000","last_undegraded":"2024-02-18T00:36:24.648880+0000","last_fullsized":"2024-02-18T00:36:24.648880+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":350,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639185+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:36:24.639185+0000","last_peered":"2024-02-18T00:36:24.639185+0000","last_clean":"2024-02-18T00:36:24.639185+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:36:24.639185+0000","last_undegraded":"2024-02-18T00:36:24.639185+0000","last_fullsized":"2024-02-18T00:36:24.639185+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":181,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640392+0000","last_change":"2024-02-18T00:34:32.021327+0000","last_active":"2024-02-18T00:36:24.640392+0000","last_peered":"2024-02-18T00:36:24.640392+0000","last_clean":"2024-02-18T00:36:24.640392+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:36:24.640392+0000","last_undegraded":"2024-02-18T00:36:24.640392+0000","last_fullsized":"2024-02-18T00:36:24.640392+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:34:32.021219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.021219+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659129+0000","last_change":"2024-02-18T00:35:13.452285+0000","last_active":"2024-02-18T00:36:24.659129+0000","last_peered":"2024-02-18T00:36:24.659129+0000","last_clean":"2024-02-18T00:36:24.659129+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:36:24.659129+0000","last_undegraded":"2024-02-18T00:36:24.659129+0000","last_fullsized":"2024-02-18T00:36:24.659129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.452220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.452220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.643046+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:36:24.643046+0000","last_peered":"2024-02-18T00:36:24.643046+0000","last_clean":"2024-02-18T00:36:24.643046+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:36:24.643046+0000","last_undegraded":"2024-02-18T00:36:24.643046+0000","last_fullsized":"2024-02-18T00:36:24.643046+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":269,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641610+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:36:24.641610+0000","last_peered":"2024-02-18T00:36:24.641610+0000","last_clean":"2024-02-18T00:36:24.641610+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:36:24.641610+0000","last_undegraded":"2024-02-18T00:36:24.641610+0000","last_fullsized":"2024-02-18T00:36:24.641610+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":288,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.638840+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:36:24.638840+0000","last_peered":"2024-02-18T00:36:24.638840+0000","last_clean":"2024-02-18T00:36:24.638840+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:36:24.638840+0000","last_undegraded":"2024-02-18T00:36:24.638840+0000","last_fullsized":"2024-02-18T00:36:24.638840+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":245,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659423+0000","last_change":"2024-02-18T00:34:26.340185+0000","last_active":"2024-02-18T00:36:24.659423+0000","last_peered":"2024-02-18T00:36:24.659423+0000","last_clean":"2024-02-18T00:36:24.659423+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:36:24.659423+0000","last_undegraded":"2024-02-18T00:36:24.659423+0000","last_fullsized":"2024-02-18T00:36:24.659423+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.340120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.340120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":271,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.638728+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:36:24.638728+0000","last_peered":"2024-02-18T00:36:24.638728+0000","last_clean":"2024-02-18T00:36:24.638728+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:36:24.638728+0000","last_undegraded":"2024-02-18T00:36:24.638728+0000","last_fullsized":"2024-02-18T00:36:24.638728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659763+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:36:24.659763+0000","last_peered":"2024-02-18T00:36:24.659763+0000","last_clean":"2024-02-18T00:36:24.659763+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:36:24.659763+0000","last_undegraded":"2024-02-18T00:36:24.659763+0000","last_fullsized":"2024-02-18T00:36:24.659763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":222,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636347+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:36:24.636347+0000","last_peered":"2024-02-18T00:36:24.636347+0000","last_clean":"2024-02-18T00:36:24.636347+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:36:24.636347+0000","last_undegraded":"2024-02-18T00:36:24.636347+0000","last_fullsized":"2024-02-18T00:36:24.636347+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":174,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652384+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:36:24.652384+0000","last_peered":"2024-02-18T00:36:24.652384+0000","last_clean":"2024-02-18T00:36:24.652384+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:36:24.652384+0000","last_undegraded":"2024-02-18T00:36:24.652384+0000","last_fullsized":"2024-02-18T00:36:24.652384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":324,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649257+0000","last_change":"2024-02-18T00:34:28.290944+0000","last_active":"2024-02-18T00:36:24.649257+0000","last_peered":"2024-02-18T00:36:24.649257+0000","last_clean":"2024-02-18T00:36:24.649257+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:36:24.649257+0000","last_undegraded":"2024-02-18T00:36:24.649257+0000","last_fullsized":"2024-02-18T00:36:24.649257+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:34:28.290868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.290868+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":207,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636328+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:36:24.636328+0000","last_peered":"2024-02-18T00:36:24.636328+0000","last_clean":"2024-02-18T00:36:24.636328+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:36:24.636328+0000","last_undegraded":"2024-02-18T00:36:24.636328+0000","last_fullsized":"2024-02-18T00:36:24.636328+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":291,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640903+0000","last_change":"2024-02-18T00:34:35.765124+0000","last_active":"2024-02-18T00:36:24.640903+0000","last_peered":"2024-02-18T00:36:24.640903+0000","last_clean":"2024-02-18T00:36:24.640903+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:36:24.640903+0000","last_undegraded":"2024-02-18T00:36:24.640903+0000","last_fullsized":"2024-02-18T00:36:24.640903+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.765059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.765059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":272,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.643085+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:36:24.643085+0000","last_peered":"2024-02-18T00:36:24.643085+0000","last_clean":"2024-02-18T00:36:24.643085+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:36:24.643085+0000","last_undegraded":"2024-02-18T00:36:24.643085+0000","last_fullsized":"2024-02-18T00:36:24.643085+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641664+0000","last_change":"2024-02-18T00:34:17.016556+0000","last_active":"2024-02-18T00:36:24.641664+0000","last_peered":"2024-02-18T00:36:24.641664+0000","last_clean":"2024-02-18T00:36:24.641664+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:36:24.641664+0000","last_undegraded":"2024-02-18T00:36:24.641664+0000","last_fullsized":"2024-02-18T00:36:24.641664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.016476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.016476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639139+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:36:24.639139+0000","last_peered":"2024-02-18T00:36:24.639139+0000","last_clean":"2024-02-18T00:36:24.639139+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:36:24.639139+0000","last_undegraded":"2024-02-18T00:36:24.639139+0000","last_fullsized":"2024-02-18T00:36:24.639139+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":279,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652915+0000","last_change":"2024-02-18T00:33:49.897870+0000","last_active":"2024-02-18T00:36:24.652915+0000","last_peered":"2024-02-18T00:36:24.652915+0000","last_clean":"2024-02-18T00:36:24.652915+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:36:24.652915+0000","last_undegraded":"2024-02-18T00:36:24.652915+0000","last_fullsized":"2024-02-18T00:36:24.652915+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.897812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.897812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":314,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.661130+0000","last_change":"2024-02-18T00:34:25.385634+0000","last_active":"2024-02-18T00:36:24.661130+0000","last_peered":"2024-02-18T00:36:24.661130+0000","last_clean":"2024-02-18T00:36:24.661130+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:36:24.661130+0000","last_undegraded":"2024-02-18T00:36:24.661130+0000","last_fullsized":"2024-02-18T00:36:24.661130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:34:25.385564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.385564+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":283,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651784+0000","last_change":"2024-02-18T00:33:37.809035+0000","last_active":"2024-02-18T00:36:24.651784+0000","last_peered":"2024-02-18T00:36:24.651784+0000","last_clean":"2024-02-18T00:36:24.651784+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:36:24.651784+0000","last_undegraded":"2024-02-18T00:36:24.651784+0000","last_fullsized":"2024-02-18T00:36:24.651784+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:37.808923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.808923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":704,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:25.673076+0000","last_change":"2024-02-18T00:36:06.641279+0000","last_active":"2024-02-18T00:36:25.673076+0000","last_peered":"2024-02-18T00:36:25.673076+0000","last_clean":"2024-02-18T00:36:25.673076+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:36:25.673076+0000","last_undegraded":"2024-02-18T00:36:25.673076+0000","last_fullsized":"2024-02-18T00:36:25.673076+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:36:06.641185+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.641185+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":422,"num_read_kb":417,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":211,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649369+0000","last_change":"2024-02-18T00:34:29.252896+0000","last_active":"2024-02-18T00:36:24.649369+0000","last_peered":"2024-02-18T00:36:24.649369+0000","last_clean":"2024-02-18T00:36:24.649369+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:36:24.649369+0000","last_undegraded":"2024-02-18T00:36:24.649369+0000","last_fullsized":"2024-02-18T00:36:24.649369+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:34:29.252830+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.252830+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":223,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645731+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:36:24.645731+0000","last_peered":"2024-02-18T00:36:24.645731+0000","last_clean":"2024-02-18T00:36:24.645731+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:36:24.645731+0000","last_undegraded":"2024-02-18T00:36:24.645731+0000","last_fullsized":"2024-02-18T00:36:24.645731+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640409+0000","last_change":"2024-02-18T00:34:19.925200+0000","last_active":"2024-02-18T00:36:24.640409+0000","last_peered":"2024-02-18T00:36:24.640409+0000","last_clean":"2024-02-18T00:36:24.640409+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:36:24.640409+0000","last_undegraded":"2024-02-18T00:36:24.640409+0000","last_fullsized":"2024-02-18T00:36:24.640409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:19.925111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.925111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":85,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640872+0000","last_change":"2024-02-18T00:35:44.479802+0000","last_active":"2024-02-18T00:36:24.640872+0000","last_peered":"2024-02-18T00:36:24.640872+0000","last_clean":"2024-02-18T00:36:24.640872+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:36:24.640872+0000","last_undegraded":"2024-02-18T00:36:24.640872+0000","last_fullsized":"2024-02-18T00:36:24.640872+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:33:54.197976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.197976+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":282,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640752+0000","last_change":"2024-02-18T00:34:15.045746+0000","last_active":"2024-02-18T00:36:24.640752+0000","last_peered":"2024-02-18T00:36:24.640752+0000","last_clean":"2024-02-18T00:36:24.640752+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:36:24.640752+0000","last_undegraded":"2024-02-18T00:36:24.640752+0000","last_fullsized":"2024-02-18T00:36:24.640752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.045677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.045677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":271,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642926+0000","last_change":"2024-02-18T00:33:48.688410+0000","last_active":"2024-02-18T00:36:24.642926+0000","last_peered":"2024-02-18T00:36:24.642926+0000","last_clean":"2024-02-18T00:36:24.642926+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:36:24.642926+0000","last_undegraded":"2024-02-18T00:36:24.642926+0000","last_fullsized":"2024-02-18T00:36:24.642926+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:43.926301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.926301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639961+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:36:24.639961+0000","last_peered":"2024-02-18T00:36:24.639961+0000","last_clean":"2024-02-18T00:36:24.639961+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:36:24.639961+0000","last_undegraded":"2024-02-18T00:36:24.639961+0000","last_fullsized":"2024-02-18T00:36:24.639961+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":362,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639943+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:36:24.639943+0000","last_peered":"2024-02-18T00:36:24.639943+0000","last_clean":"2024-02-18T00:36:24.639943+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:36:24.639943+0000","last_undegraded":"2024-02-18T00:36:24.639943+0000","last_fullsized":"2024-02-18T00:36:24.639943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":272,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652111+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:36:24.652111+0000","last_peered":"2024-02-18T00:36:24.652111+0000","last_clean":"2024-02-18T00:36:24.652111+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:36:24.652111+0000","last_undegraded":"2024-02-18T00:36:24.652111+0000","last_fullsized":"2024-02-18T00:36:24.652111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":297,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659094+0000","last_change":"2024-02-18T00:34:31.410145+0000","last_active":"2024-02-18T00:36:24.659094+0000","last_peered":"2024-02-18T00:36:24.659094+0000","last_clean":"2024-02-18T00:36:24.659094+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:36:24.659094+0000","last_undegraded":"2024-02-18T00:36:24.659094+0000","last_fullsized":"2024-02-18T00:36:24.659094+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.410081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.410081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":261,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636171+0000","last_change":"2024-02-18T00:33:48.010937+0000","last_active":"2024-02-18T00:36:24.636171+0000","last_peered":"2024-02-18T00:36:24.636171+0000","last_clean":"2024-02-18T00:36:24.636171+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:36:24.636171+0000","last_undegraded":"2024-02-18T00:36:24.636171+0000","last_fullsized":"2024-02-18T00:36:24.636171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:48.010821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:48.010821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645992+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:36:24.645992+0000","last_peered":"2024-02-18T00:36:24.645992+0000","last_clean":"2024-02-18T00:36:24.645992+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:36:24.645992+0000","last_undegraded":"2024-02-18T00:36:24.645992+0000","last_fullsized":"2024-02-18T00:36:24.645992+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651570+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:36:24.651570+0000","last_peered":"2024-02-18T00:36:24.651570+0000","last_clean":"2024-02-18T00:36:24.651570+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:36:24.651570+0000","last_undegraded":"2024-02-18T00:36:24.651570+0000","last_fullsized":"2024-02-18T00:36:24.651570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":325,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645505+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:36:24.645505+0000","last_peered":"2024-02-18T00:36:24.645505+0000","last_clean":"2024-02-18T00:36:24.645505+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:36:24.645505+0000","last_undegraded":"2024-02-18T00:36:24.645505+0000","last_fullsized":"2024-02-18T00:36:24.645505+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":307,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646694+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:36:24.646694+0000","last_peered":"2024-02-18T00:36:24.646694+0000","last_clean":"2024-02-18T00:36:24.646694+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:36:24.646694+0000","last_undegraded":"2024-02-18T00:36:24.646694+0000","last_fullsized":"2024-02-18T00:36:24.646694+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":276,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642894+0000","last_change":"2024-02-18T00:34:26.701231+0000","last_active":"2024-02-18T00:36:24.642894+0000","last_peered":"2024-02-18T00:36:24.642894+0000","last_clean":"2024-02-18T00:36:24.642894+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:36:24.642894+0000","last_undegraded":"2024-02-18T00:36:24.642894+0000","last_fullsized":"2024-02-18T00:36:24.642894+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.701162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.701162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645970+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:36:24.645970+0000","last_peered":"2024-02-18T00:36:24.645970+0000","last_clean":"2024-02-18T00:36:24.645970+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:36:24.645970+0000","last_undegraded":"2024-02-18T00:36:24.645970+0000","last_fullsized":"2024-02-18T00:36:24.645970+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":339,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.638972+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:36:24.638972+0000","last_peered":"2024-02-18T00:36:24.638972+0000","last_clean":"2024-02-18T00:36:24.638972+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:36:24.638972+0000","last_undegraded":"2024-02-18T00:36:24.638972+0000","last_fullsized":"2024-02-18T00:36:24.638972+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":274,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651563+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:36:24.651563+0000","last_peered":"2024-02-18T00:36:24.651563+0000","last_clean":"2024-02-18T00:36:24.651563+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:36:24.651563+0000","last_undegraded":"2024-02-18T00:36:24.651563+0000","last_fullsized":"2024-02-18T00:36:24.651563+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":278,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.644205+0000","last_change":"2024-02-18T00:34:25.683506+0000","last_active":"2024-02-18T00:36:24.644205+0000","last_peered":"2024-02-18T00:36:24.644205+0000","last_clean":"2024-02-18T00:36:24.644205+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:36:24.644205+0000","last_undegraded":"2024-02-18T00:36:24.644205+0000","last_fullsized":"2024-02-18T00:36:24.644205+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.683442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.683442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":277,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652287+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:36:24.652287+0000","last_peered":"2024-02-18T00:36:24.652287+0000","last_clean":"2024-02-18T00:36:24.652287+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:36:24.652287+0000","last_undegraded":"2024-02-18T00:36:24.652287+0000","last_fullsized":"2024-02-18T00:36:24.652287+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639107+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:36:24.639107+0000","last_peered":"2024-02-18T00:36:24.639107+0000","last_clean":"2024-02-18T00:36:24.639107+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:36:24.639107+0000","last_undegraded":"2024-02-18T00:36:24.639107+0000","last_fullsized":"2024-02-18T00:36:24.639107+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":333,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.644088+0000","last_change":"2024-02-18T00:33:48.861148+0000","last_active":"2024-02-18T00:36:24.644088+0000","last_peered":"2024-02-18T00:36:24.644088+0000","last_clean":"2024-02-18T00:36:24.644088+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:36:24.644088+0000","last_undegraded":"2024-02-18T00:36:24.644088+0000","last_fullsized":"2024-02-18T00:36:24.644088+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:33:46.897445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.897445+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":279,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648257+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:36:24.648257+0000","last_peered":"2024-02-18T00:36:24.648257+0000","last_clean":"2024-02-18T00:36:24.648257+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:36:24.648257+0000","last_undegraded":"2024-02-18T00:36:24.648257+0000","last_fullsized":"2024-02-18T00:36:24.648257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":269,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641171+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:36:24.641171+0000","last_peered":"2024-02-18T00:36:24.641171+0000","last_clean":"2024-02-18T00:36:24.641171+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:36:24.641171+0000","last_undegraded":"2024-02-18T00:36:24.641171+0000","last_fullsized":"2024-02-18T00:36:24.641171+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":179,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649401+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:36:24.649401+0000","last_peered":"2024-02-18T00:36:24.649401+0000","last_clean":"2024-02-18T00:36:24.649401+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:36:24.649401+0000","last_undegraded":"2024-02-18T00:36:24.649401+0000","last_fullsized":"2024-02-18T00:36:24.649401+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":223,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651896+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:36:24.651896+0000","last_peered":"2024-02-18T00:36:24.651896+0000","last_clean":"2024-02-18T00:36:24.651896+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:36:24.651896+0000","last_undegraded":"2024-02-18T00:36:24.651896+0000","last_fullsized":"2024-02-18T00:36:24.651896+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":341,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640291+0000","last_change":"2024-02-18T00:34:30.666844+0000","last_active":"2024-02-18T00:36:24.640291+0000","last_peered":"2024-02-18T00:36:24.640291+0000","last_clean":"2024-02-18T00:36:24.640291+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:36:24.640291+0000","last_undegraded":"2024-02-18T00:36:24.640291+0000","last_fullsized":"2024-02-18T00:36:24.640291+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:34:30.666765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.666765+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":291,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640701+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:36:24.640701+0000","last_peered":"2024-02-18T00:36:24.640701+0000","last_clean":"2024-02-18T00:36:24.640701+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:36:24.640701+0000","last_undegraded":"2024-02-18T00:36:24.640701+0000","last_fullsized":"2024-02-18T00:36:24.640701+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":363,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641471+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:36:24.641471+0000","last_peered":"2024-02-18T00:36:24.641471+0000","last_clean":"2024-02-18T00:36:24.641471+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:36:24.641471+0000","last_undegraded":"2024-02-18T00:36:24.641471+0000","last_fullsized":"2024-02-18T00:36:24.641471+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":275,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.638841+0000","last_change":"2024-02-18T00:33:53.101420+0000","last_active":"2024-02-18T00:36:24.638841+0000","last_peered":"2024-02-18T00:36:24.638841+0000","last_clean":"2024-02-18T00:36:24.638841+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:36:24.638841+0000","last_undegraded":"2024-02-18T00:36:24.638841+0000","last_fullsized":"2024-02-18T00:36:24.638841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.101251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.101251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":223,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.647992+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:36:24.647992+0000","last_peered":"2024-02-18T00:36:24.647992+0000","last_clean":"2024-02-18T00:36:24.647992+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:36:24.647992+0000","last_undegraded":"2024-02-18T00:36:24.647992+0000","last_fullsized":"2024-02-18T00:36:24.647992+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":323,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641397+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:36:24.641397+0000","last_peered":"2024-02-18T00:36:24.641397+0000","last_clean":"2024-02-18T00:36:24.641397+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:36:24.641397+0000","last_undegraded":"2024-02-18T00:36:24.641397+0000","last_fullsized":"2024-02-18T00:36:24.641397+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":272,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639206+0000","last_change":"2024-02-18T00:34:15.935045+0000","last_active":"2024-02-18T00:36:24.639206+0000","last_peered":"2024-02-18T00:36:24.639206+0000","last_clean":"2024-02-18T00:36:24.639206+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:36:24.639206+0000","last_undegraded":"2024-02-18T00:36:24.639206+0000","last_fullsized":"2024-02-18T00:36:24.639206+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.934970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"150'9","reported_seq":376,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640422+0000","last_change":"2024-02-18T00:34:27.741951+0000","last_active":"2024-02-18T00:36:24.640422+0000","last_peered":"2024-02-18T00:36:24.640422+0000","last_clean":"2024-02-18T00:36:24.640422+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:36:24.640422+0000","last_undegraded":"2024-02-18T00:36:24.640422+0000","last_fullsized":"2024-02-18T00:36:24.640422+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:27.741881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.741881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646987+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:36:24.646987+0000","last_peered":"2024-02-18T00:36:24.646987+0000","last_clean":"2024-02-18T00:36:24.646987+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:36:24.646987+0000","last_undegraded":"2024-02-18T00:36:24.646987+0000","last_fullsized":"2024-02-18T00:36:24.646987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641698+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:36:24.641698+0000","last_peered":"2024-02-18T00:36:24.641698+0000","last_clean":"2024-02-18T00:36:24.641698+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:36:24.641698+0000","last_undegraded":"2024-02-18T00:36:24.641698+0000","last_fullsized":"2024-02-18T00:36:24.641698+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":288,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659154+0000","last_change":"2024-02-18T00:34:32.418786+0000","last_active":"2024-02-18T00:36:24.659154+0000","last_peered":"2024-02-18T00:36:24.659154+0000","last_clean":"2024-02-18T00:36:24.659154+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:36:24.659154+0000","last_undegraded":"2024-02-18T00:36:24.659154+0000","last_fullsized":"2024-02-18T00:36:24.659154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:34:32.418717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.418717+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":279,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649130+0000","last_change":"2024-02-18T00:34:20.312165+0000","last_active":"2024-02-18T00:36:24.649130+0000","last_peered":"2024-02-18T00:36:24.649130+0000","last_clean":"2024-02-18T00:36:24.649130+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:36:24.649130+0000","last_undegraded":"2024-02-18T00:36:24.649130+0000","last_fullsized":"2024-02-18T00:36:24.649130+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.312096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.312096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":269,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642848+0000","last_change":"2024-02-18T00:33:50.765435+0000","last_active":"2024-02-18T00:36:24.642848+0000","last_peered":"2024-02-18T00:36:24.642848+0000","last_clean":"2024-02-18T00:36:24.642848+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:36:24.642848+0000","last_undegraded":"2024-02-18T00:36:24.642848+0000","last_fullsized":"2024-02-18T00:36:24.642848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.765368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.765368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":266,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636155+0000","last_change":"2024-02-18T00:36:11.346267+0000","last_active":"2024-02-18T00:36:24.636155+0000","last_peered":"2024-02-18T00:36:24.636155+0000","last_clean":"2024-02-18T00:36:24.636155+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:36:24.636155+0000","last_undegraded":"2024-02-18T00:36:24.636155+0000","last_fullsized":"2024-02-18T00:36:24.636155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.346197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.346197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":208,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.638718+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:36:24.638718+0000","last_peered":"2024-02-18T00:36:24.638718+0000","last_clean":"2024-02-18T00:36:24.638718+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:36:24.638718+0000","last_undegraded":"2024-02-18T00:36:24.638718+0000","last_fullsized":"2024-02-18T00:36:24.638718+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":291,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646146+0000","last_change":"2024-02-18T00:34:20.362851+0000","last_active":"2024-02-18T00:36:24.646146+0000","last_peered":"2024-02-18T00:36:24.646146+0000","last_clean":"2024-02-18T00:36:24.646146+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:36:24.646146+0000","last_undegraded":"2024-02-18T00:36:24.646146+0000","last_fullsized":"2024-02-18T00:36:24.646146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:34:20.362782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.362782+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":274,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.660321+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:36:24.660321+0000","last_peered":"2024-02-18T00:36:24.660321+0000","last_clean":"2024-02-18T00:36:24.660321+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:36:24.660321+0000","last_undegraded":"2024-02-18T00:36:24.660321+0000","last_fullsized":"2024-02-18T00:36:24.660321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.659250+0000","last_change":"2024-02-18T00:34:34.355483+0000","last_active":"2024-02-18T00:36:24.659250+0000","last_peered":"2024-02-18T00:36:24.659250+0000","last_clean":"2024-02-18T00:36:24.659250+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:36:24.659250+0000","last_undegraded":"2024-02-18T00:36:24.659250+0000","last_fullsized":"2024-02-18T00:36:24.659250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.355416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.355416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640436+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:36:24.640436+0000","last_peered":"2024-02-18T00:36:24.640436+0000","last_clean":"2024-02-18T00:36:24.640436+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:36:24.640436+0000","last_undegraded":"2024-02-18T00:36:24.640436+0000","last_fullsized":"2024-02-18T00:36:24.640436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649328+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:36:24.649328+0000","last_peered":"2024-02-18T00:36:24.649328+0000","last_clean":"2024-02-18T00:36:24.649328+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:36:24.649328+0000","last_undegraded":"2024-02-18T00:36:24.649328+0000","last_fullsized":"2024-02-18T00:36:24.649328+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":277,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636335+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:36:24.636335+0000","last_peered":"2024-02-18T00:36:24.636335+0000","last_clean":"2024-02-18T00:36:24.636335+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:36:24.636335+0000","last_undegraded":"2024-02-18T00:36:24.636335+0000","last_fullsized":"2024-02-18T00:36:24.636335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":278,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641249+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:36:24.641249+0000","last_peered":"2024-02-18T00:36:24.641249+0000","last_clean":"2024-02-18T00:36:24.641249+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:36:24.641249+0000","last_undegraded":"2024-02-18T00:36:24.641249+0000","last_fullsized":"2024-02-18T00:36:24.641249+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":284,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648225+0000","last_change":"2024-02-18T00:36:12.518914+0000","last_active":"2024-02-18T00:36:24.648225+0000","last_peered":"2024-02-18T00:36:24.648225+0000","last_clean":"2024-02-18T00:36:24.648225+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:36:24.648225+0000","last_undegraded":"2024-02-18T00:36:24.648225+0000","last_fullsized":"2024-02-18T00:36:24.648225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:12.518838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:12.518838+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":174,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652060+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:36:24.652060+0000","last_peered":"2024-02-18T00:36:24.652060+0000","last_clean":"2024-02-18T00:36:24.652060+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:36:24.652060+0000","last_undegraded":"2024-02-18T00:36:24.652060+0000","last_fullsized":"2024-02-18T00:36:24.652060+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":222,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641307+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:36:24.641307+0000","last_peered":"2024-02-18T00:36:24.641307+0000","last_clean":"2024-02-18T00:36:24.641307+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:36:24.641307+0000","last_undegraded":"2024-02-18T00:36:24.641307+0000","last_fullsized":"2024-02-18T00:36:24.641307+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":328,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641176+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:36:24.641176+0000","last_peered":"2024-02-18T00:36:24.641176+0000","last_clean":"2024-02-18T00:36:24.641176+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:36:24.641176+0000","last_undegraded":"2024-02-18T00:36:24.641176+0000","last_fullsized":"2024-02-18T00:36:24.641176+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":279,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648932+0000","last_change":"2024-02-18T00:36:11.551695+0000","last_active":"2024-02-18T00:36:24.648932+0000","last_peered":"2024-02-18T00:36:24.648932+0000","last_clean":"2024-02-18T00:36:24.648932+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:36:24.648932+0000","last_undegraded":"2024-02-18T00:36:24.648932+0000","last_fullsized":"2024-02-18T00:36:24.648932+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.551602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":183,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645272+0000","last_change":"2024-02-18T00:35:42.661117+0000","last_active":"2024-02-18T00:36:24.645272+0000","last_peered":"2024-02-18T00:36:24.645272+0000","last_clean":"2024-02-18T00:36:24.645272+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:36:24.645272+0000","last_undegraded":"2024-02-18T00:36:24.645272+0000","last_fullsized":"2024-02-18T00:36:24.645272+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.374328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.374328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642950+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:36:24.642950+0000","last_peered":"2024-02-18T00:36:24.642950+0000","last_clean":"2024-02-18T00:36:24.642950+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:36:24.642950+0000","last_undegraded":"2024-02-18T00:36:24.642950+0000","last_fullsized":"2024-02-18T00:36:24.642950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":174,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645878+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:36:24.645878+0000","last_peered":"2024-02-18T00:36:24.645878+0000","last_clean":"2024-02-18T00:36:24.645878+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:36:24.645878+0000","last_undegraded":"2024-02-18T00:36:24.645878+0000","last_fullsized":"2024-02-18T00:36:24.645878+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":329,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639089+0000","last_change":"2024-02-18T00:33:56.121942+0000","last_active":"2024-02-18T00:36:24.639089+0000","last_peered":"2024-02-18T00:36:24.639089+0000","last_clean":"2024-02-18T00:36:24.639089+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:36:24.639089+0000","last_undegraded":"2024-02-18T00:36:24.639089+0000","last_fullsized":"2024-02-18T00:36:24.639089+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.121872+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":282,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640392+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:36:24.640392+0000","last_peered":"2024-02-18T00:36:24.640392+0000","last_clean":"2024-02-18T00:36:24.640392+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:36:24.640392+0000","last_undegraded":"2024-02-18T00:36:24.640392+0000","last_fullsized":"2024-02-18T00:36:24.640392+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":385,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649000+0000","last_change":"2024-02-18T00:34:18.318312+0000","last_active":"2024-02-18T00:36:24.649000+0000","last_peered":"2024-02-18T00:36:24.649000+0000","last_clean":"2024-02-18T00:36:24.649000+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:36:24.649000+0000","last_undegraded":"2024-02-18T00:36:24.649000+0000","last_fullsized":"2024-02-18T00:36:24.649000+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:18.318190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.318190+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":186,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641293+0000","last_change":"2024-02-18T00:34:20.995201+0000","last_active":"2024-02-18T00:36:24.641293+0000","last_peered":"2024-02-18T00:36:24.641293+0000","last_clean":"2024-02-18T00:36:24.641293+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:36:24.641293+0000","last_undegraded":"2024-02-18T00:36:24.641293+0000","last_fullsized":"2024-02-18T00:36:24.641293+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:34:20.995125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.995125+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":173,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642731+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:36:24.642731+0000","last_peered":"2024-02-18T00:36:24.642731+0000","last_clean":"2024-02-18T00:36:24.642731+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:36:24.642731+0000","last_undegraded":"2024-02-18T00:36:24.642731+0000","last_fullsized":"2024-02-18T00:36:24.642731+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":311,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649129+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:36:24.649129+0000","last_peered":"2024-02-18T00:36:24.649129+0000","last_clean":"2024-02-18T00:36:24.649129+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:36:24.649129+0000","last_undegraded":"2024-02-18T00:36:24.649129+0000","last_fullsized":"2024-02-18T00:36:24.649129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":273,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640828+0000","last_change":"2024-02-18T00:34:28.731011+0000","last_active":"2024-02-18T00:36:24.640828+0000","last_peered":"2024-02-18T00:36:24.640828+0000","last_clean":"2024-02-18T00:36:24.640828+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:36:24.640828+0000","last_undegraded":"2024-02-18T00:36:24.640828+0000","last_fullsized":"2024-02-18T00:36:24.640828+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.730955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.730955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":272,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639212+0000","last_change":"2024-02-18T00:33:48.751672+0000","last_active":"2024-02-18T00:36:24.639212+0000","last_peered":"2024-02-18T00:36:24.639212+0000","last_clean":"2024-02-18T00:36:24.639212+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:36:24.639212+0000","last_undegraded":"2024-02-18T00:36:24.639212+0000","last_fullsized":"2024-02-18T00:36:24.639212+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.025296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.025296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":273,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.660176+0000","last_change":"2024-02-18T00:35:54.569108+0000","last_active":"2024-02-18T00:36:24.660176+0000","last_peered":"2024-02-18T00:36:24.660176+0000","last_clean":"2024-02-18T00:36:24.660176+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:36:24.660176+0000","last_undegraded":"2024-02-18T00:36:24.660176+0000","last_fullsized":"2024-02-18T00:36:24.660176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.569033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.569033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":227,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.651874+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:36:24.651874+0000","last_peered":"2024-02-18T00:36:24.651874+0000","last_clean":"2024-02-18T00:36:24.651874+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:36:24.651874+0000","last_undegraded":"2024-02-18T00:36:24.651874+0000","last_fullsized":"2024-02-18T00:36:24.651874+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":302,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652837+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:36:24.652837+0000","last_peered":"2024-02-18T00:36:24.652837+0000","last_clean":"2024-02-18T00:36:24.652837+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:36:24.652837+0000","last_undegraded":"2024-02-18T00:36:24.652837+0000","last_fullsized":"2024-02-18T00:36:24.652837+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":288,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648805+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:36:24.648805+0000","last_peered":"2024-02-18T00:36:24.648805+0000","last_clean":"2024-02-18T00:36:24.648805+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:36:24.648805+0000","last_undegraded":"2024-02-18T00:36:24.648805+0000","last_fullsized":"2024-02-18T00:36:24.648805+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.660681+0000","last_change":"2024-02-18T00:34:18.357435+0000","last_active":"2024-02-18T00:36:24.660681+0000","last_peered":"2024-02-18T00:36:24.660681+0000","last_clean":"2024-02-18T00:36:24.660681+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:36:24.660681+0000","last_undegraded":"2024-02-18T00:36:24.660681+0000","last_fullsized":"2024-02-18T00:36:24.660681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.357369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.357369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646429+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:36:24.646429+0000","last_peered":"2024-02-18T00:36:24.646429+0000","last_clean":"2024-02-18T00:36:24.646429+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:36:24.646429+0000","last_undegraded":"2024-02-18T00:36:24.646429+0000","last_fullsized":"2024-02-18T00:36:24.646429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.648502+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:36:24.648502+0000","last_peered":"2024-02-18T00:36:24.648502+0000","last_clean":"2024-02-18T00:36:24.648502+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:36:24.648502+0000","last_undegraded":"2024-02-18T00:36:24.648502+0000","last_fullsized":"2024-02-18T00:36:24.648502+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":309,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640119+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:36:24.640119+0000","last_peered":"2024-02-18T00:36:24.640119+0000","last_clean":"2024-02-18T00:36:24.640119+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:36:24.640119+0000","last_undegraded":"2024-02-18T00:36:24.640119+0000","last_fullsized":"2024-02-18T00:36:24.640119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":293,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642795+0000","last_change":"2024-02-18T00:33:48.748311+0000","last_active":"2024-02-18T00:36:24.642795+0000","last_peered":"2024-02-18T00:36:24.642795+0000","last_clean":"2024-02-18T00:36:24.642795+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:36:24.642795+0000","last_undegraded":"2024-02-18T00:36:24.642795+0000","last_fullsized":"2024-02-18T00:36:24.642795+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:33:45.906516+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.906516+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646051+0000","last_change":"2024-02-18T00:34:29.236028+0000","last_active":"2024-02-18T00:36:24.646051+0000","last_peered":"2024-02-18T00:36:24.646051+0000","last_clean":"2024-02-18T00:36:24.646051+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:36:24.646051+0000","last_undegraded":"2024-02-18T00:36:24.646051+0000","last_fullsized":"2024-02-18T00:36:24.646051+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:29.235953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.235953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":262,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.644014+0000","last_change":"2024-02-18T00:33:38.914876+0000","last_active":"2024-02-18T00:36:24.644014+0000","last_peered":"2024-02-18T00:36:24.644014+0000","last_clean":"2024-02-18T00:36:24.644014+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:36:24.644014+0000","last_undegraded":"2024-02-18T00:36:24.644014+0000","last_fullsized":"2024-02-18T00:36:24.644014+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:38.914691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:38.914691+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":222,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639972+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:36:24.639972+0000","last_peered":"2024-02-18T00:36:24.639972+0000","last_clean":"2024-02-18T00:36:24.639972+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:36:24.639972+0000","last_undegraded":"2024-02-18T00:36:24.639972+0000","last_fullsized":"2024-02-18T00:36:24.639972+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":330,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645498+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:36:24.645498+0000","last_peered":"2024-02-18T00:36:24.645498+0000","last_clean":"2024-02-18T00:36:24.645498+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:36:24.645498+0000","last_undegraded":"2024-02-18T00:36:24.645498+0000","last_fullsized":"2024-02-18T00:36:24.645498+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":173,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.641194+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:36:24.641194+0000","last_peered":"2024-02-18T00:36:24.641194+0000","last_clean":"2024-02-18T00:36:24.641194+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:36:24.641194+0000","last_undegraded":"2024-02-18T00:36:24.641194+0000","last_fullsized":"2024-02-18T00:36:24.641194+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":287,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640892+0000","last_change":"2024-02-18T00:35:42.660706+0000","last_active":"2024-02-18T00:36:24.640892+0000","last_peered":"2024-02-18T00:36:24.640892+0000","last_clean":"2024-02-18T00:36:24.640892+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:36:24.640892+0000","last_undegraded":"2024-02-18T00:36:24.640892+0000","last_fullsized":"2024-02-18T00:36:24.640892+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.961303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.961303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":275,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.636514+0000","last_change":"2024-02-18T00:33:48.649527+0000","last_active":"2024-02-18T00:36:24.636514+0000","last_peered":"2024-02-18T00:36:24.636514+0000","last_clean":"2024-02-18T00:36:24.636514+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:36:24.636514+0000","last_undegraded":"2024-02-18T00:36:24.636514+0000","last_fullsized":"2024-02-18T00:36:24.636514+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.996913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.996913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.652678+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:36:24.652678+0000","last_peered":"2024-02-18T00:36:24.652678+0000","last_clean":"2024-02-18T00:36:24.652678+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:36:24.652678+0000","last_undegraded":"2024-02-18T00:36:24.652678+0000","last_fullsized":"2024-02-18T00:36:24.652678+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":300,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.646061+0000","last_change":"2024-02-18T00:34:19.325220+0000","last_active":"2024-02-18T00:36:24.646061+0000","last_peered":"2024-02-18T00:36:24.646061+0000","last_clean":"2024-02-18T00:36:24.646061+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:36:24.646061+0000","last_undegraded":"2024-02-18T00:36:24.646061+0000","last_fullsized":"2024-02-18T00:36:24.646061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:34:19.325130+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.325130+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":174,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.649278+0000","last_change":"2024-02-18T00:34:31.241231+0000","last_active":"2024-02-18T00:36:24.649278+0000","last_peered":"2024-02-18T00:36:24.649278+0000","last_clean":"2024-02-18T00:36:24.649278+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:36:24.649278+0000","last_undegraded":"2024-02-18T00:36:24.649278+0000","last_fullsized":"2024-02-18T00:36:24.649278+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.241146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":169,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645613+0000","last_change":"2024-02-18T00:33:48.733307+0000","last_active":"2024-02-18T00:36:24.645613+0000","last_peered":"2024-02-18T00:36:24.645613+0000","last_clean":"2024-02-18T00:36:24.645613+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:36:24.645613+0000","last_undegraded":"2024-02-18T00:36:24.645613+0000","last_fullsized":"2024-02-18T00:36:24.645613+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.801504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.801504+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":267,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640555+0000","last_change":"2024-02-18T00:33:54.013575+0000","last_active":"2024-02-18T00:36:24.640555+0000","last_peered":"2024-02-18T00:36:24.640555+0000","last_clean":"2024-02-18T00:36:24.640555+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:36:24.640555+0000","last_undegraded":"2024-02-18T00:36:24.640555+0000","last_fullsized":"2024-02-18T00:36:24.640555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:54.013467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.013467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":222,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640430+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:36:24.640430+0000","last_peered":"2024-02-18T00:36:24.640430+0000","last_clean":"2024-02-18T00:36:24.640430+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:36:24.640430+0000","last_undegraded":"2024-02-18T00:36:24.640430+0000","last_fullsized":"2024-02-18T00:36:24.640430+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":308,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645267+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:36:24.645267+0000","last_peered":"2024-02-18T00:36:24.645267+0000","last_clean":"2024-02-18T00:36:24.645267+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:36:24.645267+0000","last_undegraded":"2024-02-18T00:36:24.645267+0000","last_fullsized":"2024-02-18T00:36:24.645267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":273,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.643110+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:36:24.643110+0000","last_peered":"2024-02-18T00:36:24.643110+0000","last_clean":"2024-02-18T00:36:24.643110+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:36:24.643110+0000","last_undegraded":"2024-02-18T00:36:24.643110+0000","last_fullsized":"2024-02-18T00:36:24.643110+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":279,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.658891+0000","last_change":"2024-02-18T00:34:38.428939+0000","last_active":"2024-02-18T00:36:24.658891+0000","last_peered":"2024-02-18T00:36:24.658891+0000","last_clean":"2024-02-18T00:36:24.658891+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:36:24.658891+0000","last_undegraded":"2024-02-18T00:36:24.658891+0000","last_fullsized":"2024-02-18T00:36:24.658891+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.428855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.428855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":268,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645493+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:36:24.645493+0000","last_peered":"2024-02-18T00:36:24.645493+0000","last_clean":"2024-02-18T00:36:24.645493+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:36:24.645493+0000","last_undegraded":"2024-02-18T00:36:24.645493+0000","last_fullsized":"2024-02-18T00:36:24.645493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":213,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640636+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:36:24.640636+0000","last_peered":"2024-02-18T00:36:24.640636+0000","last_clean":"2024-02-18T00:36:24.640636+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:36:24.640636+0000","last_undegraded":"2024-02-18T00:36:24.640636+0000","last_fullsized":"2024-02-18T00:36:24.640636+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":353,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.640751+0000","last_change":"2024-02-18T00:33:41.866669+0000","last_active":"2024-02-18T00:36:24.640751+0000","last_peered":"2024-02-18T00:36:24.640751+0000","last_clean":"2024-02-18T00:36:24.640751+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:36:24.640751+0000","last_undegraded":"2024-02-18T00:36:24.640751+0000","last_fullsized":"2024-02-18T00:36:24.640751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:33:41.866531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.866531+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":281,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.639124+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:36:24.639124+0000","last_peered":"2024-02-18T00:36:24.639124+0000","last_clean":"2024-02-18T00:36:24.639124+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:36:24.639124+0000","last_undegraded":"2024-02-18T00:36:24.639124+0000","last_fullsized":"2024-02-18T00:36:24.639124+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":270,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.645477+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:36:24.645477+0000","last_peered":"2024-02-18T00:36:24.645477+0000","last_clean":"2024-02-18T00:36:24.645477+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:36:24.645477+0000","last_undegraded":"2024-02-18T00:36:24.645477+0000","last_fullsized":"2024-02-18T00:36:24.645477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":276,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-02-18T00:36:24.642699+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:36:24.642699+0000","last_peered":"2024-02-18T00:36:24.642699+0000","last_clean":"2024-02-18T00:36:24.642699+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:36:24.642699+0000","last_undegraded":"2024-02-18T00:36:24.642699+0000","last_fullsized":"2024-02-18T00:36:24.642699+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001220,"num_objects":123,"num_object_clones":2,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":548,"num_write_kb":4449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13512704,"data_stored":12003693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1798,"ondisk_log_size":1798,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":189,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":188,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":422,"num_read_kb":417,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266778,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463008,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306112,"kb_used_data":8880,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93439040,"statfs":{"total":95995035648,"available":95681576960,"internally_reserved":0,"allocated":9093120,"data_stored":6829936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":2.7280000000000002,"15min":1.234},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.7879999999999998,"5min":69.194999999999993,"15min":69.194999999999993},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":1.1619999999999999,"15min":0.71899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.1130000000000004,"5min":10.433,"15min":10.433},"last":1}]},{"osd":1,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":2.9590000000000001,"15min":1.3169999999999999},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":2.8330000000000002,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":2.9409999999999998,"15min":1.306},"min":{"1min":0.36899999999999999,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":3.6030000000000002,"5min":71.872,"15min":71.872},"last":1.216}]},{"osd":2,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":1.389,"15min":0.80000000000000004},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.316},"max":{"1min":3.657,"5min":39.939999999999998,"15min":39.939999999999998},"last":1.141},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.839,"15min":0.95899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":4.1600000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":1.0429999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":1.383,"15min":0.79900000000000004},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.5670000000000002,"5min":21.219000000000001,"15min":21.219000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.284,"15min":0.77200000000000002},"min":{"1min":0.438,"5min":0.309,"15min":0.309},"max":{"1min":3.694,"5min":14.007999999999999,"15min":14.007999999999999},"last":1.0149999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.49099999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.4359999999999999,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1890000000000001,"15min":0.66200000000000003},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.907,"5min":35.18,"15min":35.18},"last":0.439}]},{"osd":5,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":4.476,"5min":10.606,"15min":10.606},"last":0.371},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.2310000000000001,"15min":0.68799999999999994},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6769999999999996,"5min":35.119,"15min":35.119},"last":0.39200000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.63200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4770000000000003,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":1.119,"15min":0.72299999999999998},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.8730000000000002,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.46200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":107,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302212,"kb_used_data":4976,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442940,"statfs":{"total":95995035648,"available":95685570560,"internally_reserved":0,"allocated":5095424,"data_stored":2830009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":1.2290000000000001,"15min":0.625},"min":{"1min":0.19900000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.47899999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.252},{"interface":"front","average":{"1min":0.33400000000000002,"5min":1.3140000000000001,"15min":0.65900000000000003},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46999999999999997,"5min":43.137,"15min":43.137},"last":0.315}]},{"osd":2,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":1.5289999999999999,"15min":0.75900000000000001},"min":{"1min":0.20699999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51900000000000002,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.95999999999999996,"15min":0.58599999999999997},"min":{"1min":0.28499999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":17.596,"15min":17.596},"last":0.32900000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":1.355,"15min":0.755},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51700000000000002,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.432,"5min":1.3580000000000001,"15min":0.77600000000000002},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.1339999999999999,"15min":0.78000000000000003},"min":{"1min":0.29999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1659999999999999,"15min":0.76600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88100000000000001,"5min":12.75,"15min":12.75},"last":0.40999999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.1439999999999999,"15min":0.77700000000000002},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.8759999999999999,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.1499999999999999,"15min":0.79200000000000004},"min":{"1min":0.41999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":4.1180000000000003,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.46700000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.1779999999999999,"15min":0.81399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":12.834,"15min":12.834},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.149,"15min":0.80300000000000005},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81100000000000005,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.55900000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":1.147,"15min":0.80500000000000005},"min":{"1min":0.41899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.109,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":1.1459999999999999,"15min":0.81499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5409999999999999,"5min":12.805,"15min":12.805},"last":0.52900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314508,"kb_used_data":4984,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430644,"statfs":{"total":95995035648,"available":95672979456,"internally_reserved":0,"allocated":5103616,"data_stored":2829131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":2.2679999999999998,"15min":1.1359999999999999},"min":{"1min":0.25700000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.0820000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":1.5089999999999999,"15min":0.73999999999999999},"min":{"1min":0.221,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":1.179,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.42899999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":1.8240000000000001,"15min":0.872},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.2430000000000001,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.311,"5min":1.8280000000000001,"15min":0.87},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":0.40000000000000002,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.36599999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.1659999999999999,"15min":0.68100000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55400000000000005,"5min":19.006,"15min":19.006},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.442,"5min":2.222,"15min":1.071},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2130000000000001,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":1.6499999999999999,"15min":0.91400000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.63900000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.627,"5min":1.6970000000000001,"15min":0.96799999999999997},"min":{"1min":0.42199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2969999999999999,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.53900000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":1.617,"15min":0.93200000000000005},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.5,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":1.8169999999999999,"15min":0.96599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":3.4060000000000001,"5min":57.991,"15min":57.991},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.2270000000000001,"15min":0.79300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.441},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.712,"15min":0.95399999999999996},"min":{"1min":0.41799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.28,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.57599999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.8680000000000001,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3129999999999999,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.628},{"interface":"front","average":{"1min":0.56799999999999995,"5min":1.798,"15min":0.99099999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.45900000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314588,"kb_used_data":5072,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430564,"statfs":{"total":95995035648,"available":95672897536,"internally_reserved":0,"allocated":5193728,"data_stored":2838272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.71599999999999997,"15min":0.49099999999999999},"min":{"1min":0.22,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46100000000000002,"5min":12.144,"15min":12.144},"last":0.249},{"interface":"front","average":{"1min":0.35799999999999998,"5min":1.0529999999999999,"15min":0.60699999999999998},"min":{"1min":0.221,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45800000000000002,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.34999999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.70899999999999996,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47099999999999997,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.82099999999999995,"15min":0.54500000000000004},"min":{"1min":0.29499999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48499999999999999,"5min":18.919,"15min":18.919},"last":0.379}]},{"osd":3,"last update":"Sun Feb 18 00:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.92400000000000004,"15min":0.61599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71799999999999997,"5min":22.712,"15min":22.712},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.92300000000000004,"15min":0.627},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.629,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.307}]},{"osd":4,"last update":"Sun Feb 18 00:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.88400000000000001,"15min":0.68300000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.85399999999999998,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":10.016,"15min":10.016},"last":0.40899999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.78200000000000003,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":6.766,"15min":6.766},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.85399999999999998,"15min":0.70099999999999996},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78500000000000003,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.436}]},{"osd":6,"last update":"Sun Feb 18 00:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.79600000000000004,"15min":0.69299999999999995},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":6.806,"15min":6.806},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.90800000000000003,"15min":0.74099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.47099999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.81200000000000006,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.89700000000000002,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.48299999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149806,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318488,"kb_used_data":8968,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93426664,"statfs":{"total":95995035648,"available":95668903936,"internally_reserved":0,"allocated":9183232,"data_stored":6829956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.65200000000000002,"15min":0.47799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53100000000000003,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.75700000000000001,"15min":0.52500000000000002},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.304,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.60499999999999998,"15min":0.48199999999999998},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.0800000000000001,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":1.1990000000000001,"15min":0.66800000000000004},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65400000000000003,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.40899999999999997}]},{"osd":2,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":1.0700000000000001,"15min":0.64800000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.117,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.54300000000000004,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.88500000000000001,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.96899999999999997,"5min":10.131,"15min":10.131},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.70599999999999996,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71299999999999997,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.70299999999999996,"15min":0.63400000000000001},"min":{"1min":0.371,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76100000000000001,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81799999999999995,"15min":0.67800000000000005},"min":{"1min":0.41099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1279999999999999,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":1.0189999999999999,"15min":0.73599999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86199999999999999,"5min":10.189,"15min":10.189},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.73599999999999999,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84999999999999998,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:35:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.78100000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73499999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.92300000000000004,"15min":0.70399999999999996},"min":{"1min":0.42499999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80300000000000005,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.75800000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986281,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306340,"kb_used_data":5008,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438812,"statfs":{"total":95995035648,"available":95681343488,"internally_reserved":0,"allocated":5128192,"data_stored":2830418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":1.556,"15min":0.91800000000000004},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.67800000000000005,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":1.506,"15min":0.877},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72499999999999998,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.47099999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.81499999999999995,"15min":0.65200000000000002},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":1.595,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":0.82699999999999996,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.60699999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.78900000000000003,"15min":0.66900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.119,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.90500000000000003,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70299999999999996,"5min":17.77,"15min":17.77},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.70599999999999996,"15min":0.63400000000000001},"min":{"1min":0.34899999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":1.8939999999999999,"5min":7.218,"15min":7.218},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.077,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.349,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:35:30 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.439,"15min":0.38300000000000001},"min":{"1min":0.29399999999999998,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.70399999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.433},{"interface":"front","average":{"1min":0.499,"5min":0.45900000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.68999999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.51900000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46700000000000003,"15min":0.41599999999999998},"min":{"1min":0.32400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70299999999999996,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.45600000000000002},"min":{"1min":0.34000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.751,"15min":0.751},"last":0.48999999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.68999999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.623},{"interface":"front","average":{"1min":0.5,"5min":0.505,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73599999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.57499999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724644,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306036,"kb_used_data":8808,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93439116,"statfs":{"total":95995035648,"available":95681654784,"internally_reserved":0,"allocated":9019392,"data_stored":6829939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57199999999999995,"15min":0.54700000000000004},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.8480000000000001,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.69899999999999995,"15min":0.59699999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.8149999999999999,"5min":12.744,"15min":12.744},"last":0.504}]},{"osd":1,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.69899999999999995,"15min":0.625},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":4.6669999999999998,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.73399999999999999,"15min":0.62},"min":{"1min":0.318,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":4.7149999999999999,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.65000000000000002,"15min":0.61299999999999999},"min":{"1min":0.30199999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.7410000000000001,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.67100000000000004,"15min":0.626},"min":{"1min":0.38900000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.734,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69799999999999995,"15min":0.60699999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":2.1099999999999999,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.63200000000000001,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.546,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:35:31 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.54600000000000004,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.621,"5min":11.818,"15min":11.818},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.48799999999999999,"15min":0.41299999999999998},"min":{"1min":0.309,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.73299999999999998,"5min":4.274,"15min":4.274},"last":0.35899999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.64000000000000001,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":18.605,"15min":18.605},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52500000000000002,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65600000000000003,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.33100000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.51100000000000001,"15min":0.46899999999999997},"min":{"1min":0.33300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.56799999999999995,"15min":0.502},"min":{"1min":0.318,"5min":0.309,"15min":0.309},"max":{"1min":4.649,"5min":4.649,"15min":4.649},"last":0.41499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4272128,"data_stored":4000669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4112384,"data_stored":4000278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4000505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:36:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:36:29.904 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:36:29.904 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:36:29.904 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:36:29.904 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:36:29.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:28.656088+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:29.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:28.656267+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e310: 8 total, 8 up, 7 in 2024-02-18T00:36:29.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:28.662538+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:36:29.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:28.677986+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:36:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:28.656088+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:28.656267+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e310: 8 total, 8 up, 7 in 2024-02-18T00:36:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:28.662538+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:36:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:28.677986+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:36:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:28.656088+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:28.656267+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e310: 8 total, 8 up, 7 in 2024-02-18T00:36:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:28.662538+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:36:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:28.677986+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:36:30.562 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:29.675195+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:29.675319+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e311: 8 total, 8 up, 7 in 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:29.677385+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:29.677917+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:29.678512+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:29.678940+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:29.867485+0000 mgr.y (mgr.24454) 316 : cluster [DBG] pgmap v454: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:30.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:29.675195+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:36:30.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:29.675319+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e311: 8 total, 8 up, 7 in 2024-02-18T00:36:30.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:29.677385+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:30.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:29.677917+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:30.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:29.678512+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:30.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:29.678940+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:30.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:29.867485+0000 mgr.y (mgr.24454) 316 : cluster [DBG] pgmap v454: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:30.836 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:36:30.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 -- 172.21.15.59:0/1448992562 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff4080f9670 msgr2=0x7ff408101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 --2- 172.21.15.59:0/1448992562 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff4080f9670 0x7ff408101d80 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 rx=0x7ff3f8004870 tx=0x7ff3f8009cb0).stop 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 -- 172.21.15.59:0/1448992562 shutdown_connections 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 --2- 172.21.15.59:0/1448992562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4080f8d10 0x7ff4080f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 --2- 172.21.15.59:0/1448992562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff408102270 0x7ff408104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 --2- 172.21.15.59:0/1448992562 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff4080f9670 0x7ff408101d80 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 -- 172.21.15.59:0/1448992562 >> 172.21.15.59:0/1448992562 conn(0x7ff4080f2820 msgr2=0x7ff4080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 -- 172.21.15.59:0/1448992562 shutdown_connections 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.907+0000 7ff40f886700 1 -- 172.21.15.59:0/1448992562 wait complete. 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 librados: starting msgr at 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 librados: starting objecter 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 Processor -- start 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 -- start start 2024-02-18T00:36:30.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 librados: setting wanted keys 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 librados: calling monclient init 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4080f8d10 0x7ff4081103b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080f9670 0x7ff4081108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff408102270 0x7ff40810a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40810aa40 con 0x7ff4080f9670 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40810abb0 con 0x7ff4080f8d10 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40f886700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40810ad70 con 0x7ff408102270 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40de23700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff408102270 0x7ff40810a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40ce21700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080f9670 0x7ff4081108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40de23700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff408102270 0x7ff40810a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59502/0 (socket says 172.21.15.59:59502) 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40de23700 1 -- 172.21.15.59:0/1911668586 learned_addr learned my addr 172.21.15.59:0/1911668586 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.908+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4080f8d10 0x7ff4081103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40de23700 1 -- 172.21.15.59:0/1911668586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4080f8d10 msgr2=0x7ff4081103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40de23700 1 --2- 172.21.15.59:0/1911668586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4080f8d10 0x7ff4081103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:30.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40de23700 1 -- 172.21.15.59:0/1911668586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080f9670 msgr2=0x7ff4081108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40de23700 1 --2- 172.21.15.59:0/1911668586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080f9670 0x7ff4081108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40de23700 1 -- 172.21.15.59:0/1911668586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4000097c0 con 0x7ff408102270 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40ce21700 1 --2- 172.21.15.59:0/1911668586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080f9670 0x7ff4081108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40de23700 1 --2- 172.21.15.59:0/1911668586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff408102270 0x7ff40810a550 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 rx=0x7ff400001df0 tx=0x7ff400004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff3fe7fc700 10 client.?.objecter ms_handle_connect 0x7ff408102270 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff3fe7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000099f0 con 0x7ff408102270 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4000090d0 con 0x7ff408102270 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f8005040 con 0x7ff408102270 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400007d20 con 0x7ff408102270 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40f886700 10 client.40880.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40810b3b0 con 0x7ff408102270 2024-02-18T00:36:30.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.909+0000 7ff40f886700 1 librados: init done 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.910+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff40805fa60 con 0x7ff408102270 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.910+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff40000fb30 con 0x7ff408102270 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff3f4062c90 0x7ff3f4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(312..312 src has 44..312) v4 ==== 8134+0+0 (secure 0 0 0) 0x7ff40007d9a0 con 0x7ff408102270 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 osd_map(312..312 src has 44..312) v4 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 3 client.40880.objecter handle_osd_map got epochs [312,312] > 0 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 3 client.40880.objecter handle_osd_map decoding full epoch 312 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff3f4062c90 0x7ff3f4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 20 client.40880.objecter dump_active .. 0 homeless 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v312) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff400082220 con 0x7ff408102270 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff3f4062c90 0x7ff3f4065150 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 rx=0x7ff404004dc0 tx=0x7ff40400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff3f4062c90 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 10 client.40880.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 1 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff4080fe640 0x7ff4080fea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 1 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 0 1 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40805ee80 osd=0 4 2024-02-18T00:36:30.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 10 client.40880.objecter _send_command 1 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4081072a0 con 0x7ff4080fe640 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.911+0000 7ff40ce21700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff4080fe640 0x7ff4080fea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.912+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff4080fe640 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.912+0000 7ff40ce21700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff4080fe640 0x7ff4080fea20 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.912+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff4081072a0 con 0x7ff4080fe640 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.912+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.912+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 1 = system:0 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.912+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40805ee80 osd=0 4 2024-02-18T00:36:30.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.912+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 0 1 2024-02-18T00:36:30.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 10 client.40880.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:30.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 2 2024-02-18T00:36:30.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 2 2024-02-18T00:36:30.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40805ee80 osd=0 3 2024-02-18T00:36:30.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 0 2 2024-02-18T00:36:30.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40805ee80 osd=0 4 2024-02-18T00:36:30.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 10 client.40880.objecter _send_command 2 2024-02-18T00:36:30.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.927+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff4080273c0 con 0x7ff4080fe640 2024-02-18T00:36:30.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff4080273c0 con 0x7ff4080fe640 2024-02-18T00:36:30.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:30.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:30.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40805ee80 osd=0 4 2024-02-18T00:36:30.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 0 2 2024-02-18T00:36:30.931 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:36:30.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 10 client.40880.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:30.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 3 2024-02-18T00:36:30.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6805/26279110 conn(0x7ff4080598f0 0x7ff408059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 3 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 1 3 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408028710 osd=1 4 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 10 client.40880.objecter _send_command 3 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff40811da20 con 0x7ff4080598f0 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.929+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6805/26279110 conn(0x7ff4080598f0 0x7ff408059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.930+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6805/26279110 conn(0x7ff4080598f0 0x7ff408059cd0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.930+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff4080598f0 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.930+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff40811da20 con 0x7ff4080598f0 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.930+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.930+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 3 = system:0 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.930+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff408028710 osd=1 4 2024-02-18T00:36:30.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.930+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 1 3 2024-02-18T00:36:30.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 10 client.40880.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:30.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 4 2024-02-18T00:36:30.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 4 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff408028710 osd=1 3 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 1 4 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408028710 osd=1 4 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 10 client.40880.objecter _send_command 4 2024-02-18T00:36:30.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.945+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff408059360 con 0x7ff4080598f0 2024-02-18T00:36:30.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff408059360 con 0x7ff4080598f0 2024-02-18T00:36:30.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:30.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:30.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff408028710 osd=1 4 2024-02-18T00:36:30.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 1 4 2024-02-18T00:36:30.949 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:36:30.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 10 client.40880.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:30.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 5 2024-02-18T00:36:30.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff408028270 0x7ff40811c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 5 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 2 5 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408059770 osd=2 4 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 10 client.40880.objecter _send_command 5 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.946+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff40811c930 con 0x7ff408028270 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff40ce21700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff408028270 0x7ff40811c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff408028270 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff40ce21700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff408028270 0x7ff40811c440 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff40811c930 con 0x7ff408028270 2024-02-18T00:36:30.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:36:30.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 5 = system:0 2024-02-18T00:36:30.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff408059770 osd=2 4 2024-02-18T00:36:30.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.947+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 2 5 2024-02-18T00:36:30.957 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:30.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:30.961 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 10 client.40880.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 6 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 6 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff408059770 osd=2 3 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 2 6 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408059770 osd=2 4 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 10 client.40880.objecter _send_command 6 2024-02-18T00:36:30.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.963+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff4080277d0 con 0x7ff408028270 2024-02-18T00:36:30.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff4080277d0 con 0x7ff408028270 2024-02-18T00:36:30.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:30.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:30.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff408059770 osd=2 4 2024-02-18T00:36:30.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 2 6 2024-02-18T00:36:30.967 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 10 client.40880.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 7 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff40811fba0 0x7ff408121f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 7 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 3 7 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408027be0 osd=3 4 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 10 client.40880.objecter _send_command 7 2024-02-18T00:36:30.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff40811d830 con 0x7ff40811fba0 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40de23700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff40811fba0 0x7ff408121f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff40811fba0 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.965+0000 7ff40de23700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff40811fba0 0x7ff408121f90 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.966+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff40811d830 con 0x7ff40811fba0 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.966+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.966+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 7 = system:0 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.966+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff408027be0 osd=3 4 2024-02-18T00:36:30.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.966+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 3 7 2024-02-18T00:36:30.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 10 client.40880.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:30.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 8 2024-02-18T00:36:30.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 8 2024-02-18T00:36:30.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff408027be0 osd=3 3 2024-02-18T00:36:30.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 3 8 2024-02-18T00:36:30.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408027be0 osd=3 4 2024-02-18T00:36:30.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 10 client.40880.objecter _send_command 8 2024-02-18T00:36:30.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.980+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff40811cd90 con 0x7ff40811fba0 2024-02-18T00:36:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff40811cd90 con 0x7ff40811fba0 2024-02-18T00:36:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff408027be0 osd=3 4 2024-02-18T00:36:30.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 3 8 2024-02-18T00:36:30.985 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:36:30.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 10 client.40880.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 9 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff40811bfa0 0x7ff408125610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 9 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 4 9 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811cb50 osd=4 4 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 10 client.40880.objecter _send_command 9 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff40811ccc0 con 0x7ff40811bfa0 2024-02-18T00:36:30.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.982+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff40811bfa0 0x7ff408125610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:30.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.983+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff40811bfa0 2024-02-18T00:36:30.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.983+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff40811bfa0 0x7ff408125610 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:30.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.983+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff40811ccc0 con 0x7ff40811bfa0 2024-02-18T00:36:30.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.983+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:36:30.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.983+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 9 = system:0 2024-02-18T00:36:30.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.983+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40811cb50 osd=4 4 2024-02-18T00:36:30.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.983+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 4 9 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 10 client.40880.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 10 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 10 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40811cb50 osd=4 3 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 4 10 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811cb50 osd=4 4 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 10 client.40880.objecter _send_command 10 2024-02-18T00:36:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:30.998+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff40811ccc0 con 0x7ff40811bfa0 2024-02-18T00:36:31.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff40811ccc0 con 0x7ff40811bfa0 2024-02-18T00:36:31.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:31.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40811cb50 osd=4 4 2024-02-18T00:36:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 4 10 2024-02-18T00:36:31.002 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 10 client.40880.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 11 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff408126010 0x7ff408128480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 11 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 5 11 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811ccc0 osd=5 4 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 10 client.40880.objecter _send_command 11 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff40812b250 con 0x7ff408126010 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.000+0000 7ff40ce21700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff408126010 0x7ff408128480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.001+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff408126010 2024-02-18T00:36:31.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.001+0000 7ff40ce21700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff408126010 0x7ff408128480 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:29.675195+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:36:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:29.675319+0000 mon.a (mon.0) 2108 : cluster [DBG] osdmap e311: 8 total, 8 up, 7 in 2024-02-18T00:36:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:29.677385+0000 mon.c (mon.2) 280 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:29.677917+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:29.678512+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:29.678940+0000 mon.a (mon.0) 2110 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:31.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:29.867485+0000 mgr.y (mgr.24454) 316 : cluster [DBG] pgmap v454: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:31.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.001+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff40812b250 con 0x7ff408126010 2024-02-18T00:36:31.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.001+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:36:31.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.001+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 11 = system:0 2024-02-18T00:36:31.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.001+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40811ccc0 osd=5 4 2024-02-18T00:36:31.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.001+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 5 11 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 10 client.40880.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 12 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 12 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40811ccc0 osd=5 3 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 5 12 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811ccc0 osd=5 4 2024-02-18T00:36:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 10 client.40880.objecter _send_command 12 2024-02-18T00:36:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.016+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff40812b250 con 0x7ff408126010 2024-02-18T00:36:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff40812b250 con 0x7ff408126010 2024-02-18T00:36:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40811ccc0 osd=5 4 2024-02-18T00:36:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 5 12 2024-02-18T00:36:31.020 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 10 client.40880.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 13 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff40812a550 0x7ff40812a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 13 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 6 13 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b250 osd=6 4 2024-02-18T00:36:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 10 client.40880.objecter _send_command 13 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff40812b040 con 0x7ff40812a550 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40de23700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff40812a550 0x7ff40812a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff40812a550 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.018+0000 7ff40de23700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff40812a550 0x7ff40812a9d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.019+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff40812b040 con 0x7ff40812a550 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.019+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.019+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 13 = system:0 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.019+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40812b250 osd=6 4 2024-02-18T00:36:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.019+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 6 13 2024-02-18T00:36:31.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 10 client.40880.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:31.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 14 2024-02-18T00:36:31.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 14 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40812b250 osd=6 3 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 6 14 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b250 osd=6 4 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 10 client.40880.objecter _send_command 14 2024-02-18T00:36:31.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.034+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff40812b040 con 0x7ff40812a550 2024-02-18T00:36:31.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.035+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff40812b040 con 0x7ff40812a550 2024-02-18T00:36:31.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.035+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:31.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.035+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.035+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40812b250 osd=6 4 2024-02-18T00:36:31.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.035+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 6 14 2024-02-18T00:36:31.037 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:36:31.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.035+0000 7ff40f886700 10 client.40880.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:31.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 15 2024-02-18T00:36:31.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6812/53025171 conn(0x7ff40812d9c0 0x7ff40812de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:31.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 15 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 7 15 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b040 osd=7 4 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 10 client.40880.objecter _send_command 15 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff40812e4c0 con 0x7ff40812d9c0 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6812/53025171 conn(0x7ff40812d9c0 0x7ff40812de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff3fe7fc700 10 client.40880.objecter ms_handle_connect 0x7ff40812d9c0 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.036+0000 7ff40d622700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6812/53025171 conn(0x7ff40812d9c0 0x7ff40812de20 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.037+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff40812e4c0 con 0x7ff40812d9c0 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.037+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.037+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 15 = system:0 2024-02-18T00:36:31.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.037+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40812b040 osd=7 4 2024-02-18T00:36:31.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.037+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 7 15 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 10 client.40880.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign -1 16 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 20 client.40880.objecter _get_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 15 client.40880.objecter _session_command_op_remove -1 16 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 20 client.40880.objecter get_session s=0x7ff40812b040 osd=7 3 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 15 client.40880.objecter _session_command_op_assign 7 16 2024-02-18T00:36:31.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b040 osd=7 4 2024-02-18T00:36:31.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 10 client.40880.objecter _send_command 16 2024-02-18T00:36:31.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.051+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff40812e4c0 con 0x7ff40812d9c0 2024-02-18T00:36:31.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff3fe7fc700 1 -- 172.21.15.59:0/1911668586 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff40812e4c0 con 0x7ff40812d9c0 2024-02-18T00:36:31.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff3fe7fc700 10 client.40880.objecter ms_dispatch 0x7ff408001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:31.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff3fe7fc700 10 client.40880.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff3fe7fc700 20 client.40880.objecter put_session s=0x7ff40812b040 osd=7 4 2024-02-18T00:36:31.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff3fe7fc700 15 client.40880.objecter _session_command_op_remove 7 16 2024-02-18T00:36:31.055 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 librados: watch_flush enter 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 librados: watch_flush exit 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 client.40880.objecter close_session for osd.0 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff4080fe640 msgr2=0x7ff4080fea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff4080fe640 0x7ff4080fea20 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40805ee80 osd=0 1 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 client.40880.objecter close_session for osd.1 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6805/26279110 conn(0x7ff4080598f0 msgr2=0x7ff408059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6805/26279110 conn(0x7ff4080598f0 0x7ff408059cd0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408028710 osd=1 1 2024-02-18T00:36:31.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 client.40880.objecter close_session for osd.2 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff408028270 msgr2=0x7ff40811c440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff408028270 0x7ff40811c440 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408059770 osd=2 1 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 client.40880.objecter close_session for osd.3 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff40811fba0 msgr2=0x7ff408121f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff40811fba0 0x7ff408121f90 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff408027be0 osd=3 1 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 client.40880.objecter close_session for osd.4 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff40811bfa0 msgr2=0x7ff408125610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff40811bfa0 0x7ff408125610 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40de23700 1 -- 172.21.15.59:0/1911668586 reap_dead start 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811cb50 osd=4 1 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 10 client.40880.objecter close_session for osd.5 2024-02-18T00:36:31.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff408126010 msgr2=0x7ff408128480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff408126010 0x7ff408128480 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40811ccc0 osd=5 1 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 10 client.40880.objecter close_session for osd.6 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff40812a550 msgr2=0x7ff40812a9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff40812a550 0x7ff40812a9d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b250 osd=6 1 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 10 client.40880.objecter close_session for osd.7 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.053+0000 7ff40de23700 1 -- 172.21.15.59:0/1911668586 reap_dead start 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6812/53025171 conn(0x7ff40812d9c0 msgr2=0x7ff40812de20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.137:6812/53025171 conn(0x7ff40812d9c0 0x7ff40812de20 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 20 client.40880.objecter put_session s=0x7ff40812b040 osd=7 1 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 20 client.40880.objecter shutdown clearing up homeless session... 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 10 client.40880.objecter successfully canceled tick 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40de23700 1 -- 172.21.15.59:0/1911668586 reap_dead start 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff3f4062c90 msgr2=0x7ff3f4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff3f4062c90 0x7ff3f4065150 secure :-1 s=READY pgs=3065 cs=0 l=1 rev1=1 rx=0x7ff404004dc0 tx=0x7ff40400b040).stop 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff408102270 msgr2=0x7ff40810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff408102270 0x7ff40810a550 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 rx=0x7ff400001df0 tx=0x7ff400004c50).stop 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 shutdown_connections 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff408102270 0x7ff40810a550 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 --2- 172.21.15.59:0/1911668586 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff3f4062c90 0x7ff3f4065150 unknown :-1 s=CLOSED pgs=3065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 >> 172.21.15.59:0/1911668586 conn(0x7ff4080f2820 msgr2=0x7ff4080f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 shutdown_connections 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 -- 172.21.15.59:0/1911668586 wait complete. 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:31.054+0000 7ff40f886700 1 librados: shutdown 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.061 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.061 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:36:31.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:31.327 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:36:31.327 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:30.681624+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:30.681750+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:30.681811+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e312: 8 total, 8 up, 7 in 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:30.683339+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:30.683683+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:30.684491+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:30.684755+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:30.912133+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? 172.21.15.59:0/1911668586' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:32.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.140888+0000 mgr.y (mgr.24454) 317 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:30.681624+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:30.681750+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:30.681811+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e312: 8 total, 8 up, 7 in 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:30.683339+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:30.683683+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:30.684491+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:30.684755+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:30.912133+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? 172.21.15.59:0/1911668586' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:32.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.140888+0000 mgr.y (mgr.24454) 317 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:30.681624+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:30.681750+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:30.681811+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e312: 8 total, 8 up, 7 in 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:30.683339+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:30.683683+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:30.684491+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.59:0/458568879' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:30.684755+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]: dispatch 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:30.912133+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? 172.21.15.59:0/1911668586' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.140888+0000 mgr.y (mgr.24454) 317 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:32.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:36:32.420 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:32] "GET /metrics HTTP/1.1" 200 188638 "" "Prometheus/2.33.4" 2024-02-18T00:36:32.441 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:32.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:32.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:32.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:32.573 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:36:32.573 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:36:32.574 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:32.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.683741+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.683797+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:31.683852+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e313: 8 total, 8 up, 7 in 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.710767+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.711126+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.711836+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.712085+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.712525+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:31.712762+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:32.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:31.868030+0000 mgr.y (mgr.24454) 318 : cluster [DBG] pgmap v457: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:32.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:32.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:32.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.683741+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:36:32.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.683797+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:32.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:31.683852+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e313: 8 total, 8 up, 7 in 2024-02-18T00:36:32.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.710767+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.711126+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.711836+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.712085+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:32.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.712525+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:32.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:31.712762+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:32.942 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:31.868030+0000 mgr.y (mgr.24454) 318 : cluster [DBG] pgmap v457: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.683741+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:36:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.683797+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi059-82882-53"}]': finished 2024-02-18T00:36:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:31.683852+0000 mon.a (mon.0) 2118 : cluster [DBG] osdmap e313: 8 total, 8 up, 7 in 2024-02-18T00:36:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.710767+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.711126+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.711836+0000 mon.c (mon.2) 286 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:33.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.712085+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.712525+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:31.712762+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:33.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:31.868030+0000 mgr.y (mgr.24454) 318 : cluster [DBG] pgmap v457: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:33.007 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:33.037 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:33.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-18T00:36:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:33.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:33.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:33.391 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.415+0000 7f3f13b8b700 1 -- 172.21.15.59:0/3063664805 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f0c0f8420 msgr2=0x7f3f0c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.415+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/3063664805 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f0c0f8420 0x7f3f0c104a20 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f3f04009be0 tx=0x7f3f04004d10).stop 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.415+0000 7f3f13b8b700 1 -- 172.21.15.59:0/3063664805 shutdown_connections 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.415+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/3063664805 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f0c0f8420 0x7f3f0c104a20 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.415+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/3063664805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0fdbb0 0x7f3f0c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.415+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/3063664805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f0c0fe510 0x7f3f0c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.415+0000 7f3f13b8b700 1 -- 172.21.15.59:0/3063664805 >> 172.21.15.59:0/3063664805 conn(0x7f3f0c0f3800 msgr2=0x7f3f0c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:33.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 -- 172.21.15.59:0/3063664805 shutdown_connections 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 -- 172.21.15.59:0/3063664805 wait complete. 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 librados: starting msgr at 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 librados: starting objecter 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 Processor -- start 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 -- start start 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 librados: setting wanted keys 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.416+0000 7f3f13b8b700 1 librados: calling monclient init 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f13b8b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0f8420 0x7f3f0c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f13b8b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f0c0fdbb0 0x7f3f0c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f13b8b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f0c0fe510 0x7f3f0c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f13b8b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c1163c0 con 0x7f3f0c0f8420 2024-02-18T00:36:33.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f13b8b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c116530 con 0x7f3f0c0fe510 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f13b8b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c1166f0 con 0x7f3f0c0fdbb0 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0f8420 0x7f3f0c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0f8420 0x7f3f0c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51204/0 (socket says 172.21.15.59:51204) 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 -- 172.21.15.59:0/2272039423 learned_addr learned my addr 172.21.15.59:0/2272039423 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f12128700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f0c0fe510 0x7f3f0c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 -- 172.21.15.59:0/2272039423 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f0c0fdbb0 msgr2=0x7f3f0c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f0c0fdbb0 0x7f3f0c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 -- 172.21.15.59:0/2272039423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f0c0fe510 msgr2=0x7f3f0c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f0c0fe510 0x7f3f0c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 -- 172.21.15.59:0/2272039423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08005040 con 0x7f3f0c0f8420 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f12128700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f0c0fe510 0x7f3f0c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f11927700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0f8420 0x7f3f0c06b0d0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7f3f080048f0 tx=0x7f3f08009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:33.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f02ffd700 10 client.?.objecter ms_handle_connect 0x7f3f0c0f8420 2024-02-18T00:36:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f02ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:33.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f02ffd700 1 -- 172.21.15.59:0/2272039423 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08007b00 con 0x7f3f0c0f8420 2024-02-18T00:36:33.422 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f02ffd700 1 -- 172.21.15.59:0/2272039423 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0800f040 con 0x7f3f0c0f8420 2024-02-18T00:36:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f040097c0 con 0x7f3f0c0f8420 2024-02-18T00:36:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.417+0000 7f3f02ffd700 1 -- 172.21.15.59:0/2272039423 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08014aa0 con 0x7f3f0c0f8420 2024-02-18T00:36:33.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.418+0000 7f3f13b8b700 10 client.31080.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.418+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c116db0 con 0x7f3f0c0f8420 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.418+0000 7f3f13b8b700 1 librados: init done 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.419+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c05ee80 con 0x7f3f0c0f8420 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.419+0000 7f3f02ffd700 1 -- 172.21.15.59:0/2272039423 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f3f0800e660 con 0x7f3f0c0f8420 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.419+0000 7f3f02ffd700 1 --2- 172.21.15.59:0/2272039423 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3ef8062c90 0x7f3ef8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.419+0000 7f3f02ffd700 1 -- 172.21.15.59:0/2272039423 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(314..314 src has 44..314) v4 ==== 8311+0+0 (secure 0 0 0) 0x7f3f0807cfc0 con 0x7f3f0c0f8420 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.420+0000 7f3f02ffd700 10 client.31080.objecter ms_dispatch 0x7f3f0c001550 osd_map(314..314 src has 44..314) v4 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.420+0000 7f3f02ffd700 3 client.31080.objecter handle_osd_map got epochs [314,314] > 0 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.420+0000 7f3f02ffd700 3 client.31080.objecter handle_osd_map decoding full epoch 314 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.420+0000 7f3f11126700 1 --2- 172.21.15.59:0/2272039423 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3ef8062c90 0x7f3ef8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.420+0000 7f3f02ffd700 20 client.31080.objecter dump_active .. 0 homeless 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.420+0000 7f3f11126700 1 --2- 172.21.15.59:0/2272039423 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3ef8062c90 0x7f3ef8065150 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 rx=0x7f3f0c06c870 tx=0x7f3efc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:33.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.420+0000 7f3f02ffd700 10 client.31080.objecter ms_handle_connect 0x7f3ef8062c90 2024-02-18T00:36:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:33.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.424+0000 7f3f02ffd700 1 -- 172.21.15.59:0/2272039423 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3f08049f60 con 0x7f3f0c0f8420 2024-02-18T00:36:33.436 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:33.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:33.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:33.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:33.567 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:33.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:33.626+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f3f0c05fa60 con 0x7f3f0c0f8420 2024-02-18T00:36:33.719 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:33.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:33.723 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:33.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:33.978 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:33.981 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:33.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:34.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.086+0000 7f3f02ffd700 1 -- 172.21.15.59:0/2272039423 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v315) v1 ==== 91+0+0 (secure 0 0 0) 0x7f3f08050390 con 0x7f3f0c0f8420 2024-02-18T00:36:34.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.088+0000 7f3f13b8b700 10 librados: watch_flush enter 2024-02-18T00:36:34.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.088+0000 7f3f13b8b700 10 librados: watch_flush exit 2024-02-18T00:36:34.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.088+0000 7f3f13b8b700 20 client.31080.objecter shutdown clearing up homeless session... 2024-02-18T00:36:34.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.088+0000 7f3f13b8b700 10 client.31080.objecter successfully canceled tick 2024-02-18T00:36:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3ef8062c90 msgr2=0x7f3ef8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/2272039423 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3ef8062c90 0x7f3ef8065150 secure :-1 s=READY pgs=3067 cs=0 l=1 rev1=1 rx=0x7f3f0c06c870 tx=0x7f3efc00b040).stop 2024-02-18T00:36:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0f8420 msgr2=0x7f3f0c06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0f8420 0x7f3f0c06b0d0 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7f3f080048f0 tx=0x7f3f08009cb0).stop 2024-02-18T00:36:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 shutdown_connections 2024-02-18T00:36:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/2272039423 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3ef8062c90 0x7f3ef8065150 unknown :-1 s=CLOSED pgs=3067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f0c0fe510 0x7f3f0c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f0c0f8420 0x7f3f0c06b0d0 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 --2- 172.21.15.59:0/2272039423 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f0c0fdbb0 0x7f3f0c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:36:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 >> 172.21.15.59:0/2272039423 conn(0x7f3f0c0f3800 msgr2=0x7f3f0c0f5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 shutdown_connections 2024-02-18T00:36:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 -- 172.21.15.59:0/2272039423 wait complete. 2024-02-18T00:36:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:34.089+0000 7f3f13b8b700 1 librados: shutdown 2024-02-18T00:36:34.093 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:32.699744+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:32.699819+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e314: 8 total, 8 up, 7 in 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:32.705242+0000 mon.c (mon.2) 288 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:32.705572+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:32.719902+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:32.720246+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:32.720755+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]: dispatch 2024-02-18T00:36:34.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:32.720991+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:33.627637+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? 172.21.15.59:0/2272039423' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:33.630658+0000 mon.a (mon.0) 2128 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:32.699744+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:32.699819+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e314: 8 total, 8 up, 7 in 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:32.705242+0000 mon.c (mon.2) 288 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:32.705572+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:32.719902+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:32.720246+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:32.720755+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:32.720991+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]: dispatch 2024-02-18T00:36:34.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:33.627637+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? 172.21.15.59:0/2272039423' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:36:34.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:33.630658+0000 mon.a (mon.0) 2128 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:34.116 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:32.699744+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi059-82882-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:32.699819+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e314: 8 total, 8 up, 7 in 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:32.705242+0000 mon.c (mon.2) 288 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:32.705572+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:32.719902+0000 mon.c (mon.2) 289 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:32.720246+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:32.720755+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]: dispatch 2024-02-18T00:36:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:32.720991+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]: dispatch 2024-02-18T00:36:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:33.627637+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? 172.21.15.59:0/2272039423' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:36:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:33.630658+0000 mon.a (mon.0) 2128 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:36:34.565 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:34.567 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:34.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:34.649 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [==========] Running 72 tests from 4 test suites. 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: seed 82957 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (226 ms) 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (16360 ms) 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (13 ms) 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (16599 ms total) 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: 2024-02-18T00:36:34.650 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (8159 ms) 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (9987 ms) 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (12302 ms) 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: my_snaps [4] 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: my_snaps [5,4] 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: my_snaps [6,5,4] 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: promoting some heads 2024-02-18T00:36:34.651 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: promoting from clones for snap 7 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: waiting for scrubs... 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: done waiting 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46031 ms) 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9990 ms) 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7998 ms) 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8004 ms) 2024-02-18T00:36:34.652 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8012 ms) 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10003 ms) 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (9044 ms) 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (9977 ms) 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8844 ms) 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (7145 ms) 2024-02-18T00:36:34.653 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13530 ms) 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7480 ms) 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (7989 ms) 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-18T00:36:34.654 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: ok, hit_set contains 188:602f83fe:::foo:head 2024-02-18T00:36:34.668 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:34.700 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:34.721 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:34.724 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:34.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:34.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:34.852 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:34.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:36:35.202 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:33.848608+0000 osd.5 (osd.5) 137 : cluster [DBG] 4.f scrub starts 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:33.853321+0000 osd.5 (osd.5) 138 : cluster [DBG] 4.f scrub ok 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:33.868394+0000 mgr.y (mgr.24454) 319 : cluster [DBG] pgmap v460: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.015550+0000 osd.3 (osd.3) 173 : cluster [DBG] 62.15 scrub starts 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.017060+0000 osd.3 (osd.3) 174 : cluster [DBG] 62.15 scrub ok 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:34.087213+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]': finished 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:34.087332+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? 172.21.15.59:0/2272039423' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.087408+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e315: 8 total, 8 up, 7 in 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:34.643078+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.643208+0000 mon.a (mon.0) 2133 : cluster [DBG] osdmap e316: 8 total, 8 up, 7 in 2024-02-18T00:36:35.203 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:33.848608+0000 osd.5 (osd.5) 137 : cluster [DBG] 4.f scrub starts 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:33.853321+0000 osd.5 (osd.5) 138 : cluster [DBG] 4.f scrub ok 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:33.868394+0000 mgr.y (mgr.24454) 319 : cluster [DBG] pgmap v460: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.015550+0000 osd.3 (osd.3) 173 : cluster [DBG] 62.15 scrub starts 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.017060+0000 osd.3 (osd.3) 174 : cluster [DBG] 62.15 scrub ok 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:34.087213+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]': finished 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:34.087332+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? 172.21.15.59:0/2272039423' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.087408+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e315: 8 total, 8 up, 7 in 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:34.643078+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:35.204 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.643208+0000 mon.a (mon.0) 2133 : cluster [DBG] osdmap e316: 8 total, 8 up, 7 in 2024-02-18T00:36:35.211 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:35.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:35.312 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:35.344 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:35.365 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:33.848608+0000 osd.5 (osd.5) 137 : cluster [DBG] 4.f scrub starts 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:33.853321+0000 osd.5 (osd.5) 138 : cluster [DBG] 4.f scrub ok 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:33.868394+0000 mgr.y (mgr.24454) 319 : cluster [DBG] pgmap v460: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.015550+0000 osd.3 (osd.3) 173 : cluster [DBG] 62.15 scrub starts 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.017060+0000 osd.3 (osd.3) 174 : cluster [DBG] 62.15 scrub ok 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:34.087213+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-39"}]': finished 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:34.087332+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? 172.21.15.59:0/2272039423' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.087408+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e315: 8 total, 8 up, 7 in 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:34.643078+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi059-82882-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:35.367 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.643208+0000 mon.a (mon.0) 2133 : cluster [DBG] osdmap e316: 8 total, 8 up, 7 in 2024-02-18T00:36:35.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:35.368 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:35.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:36.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.204213+0000 osd.4 (osd.4) 181 : cluster [DBG] 62.f scrub starts 2024-02-18T00:36:36.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.207835+0000 osd.4 (osd.4) 182 : cluster [DBG] 62.f scrub ok 2024-02-18T00:36:36.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.579129+0000 osd.2 (osd.2) 129 : cluster [DBG] 62.3 scrub starts 2024-02-18T00:36:36.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.581298+0000 osd.2 (osd.2) 130 : cluster [DBG] 62.3 scrub ok 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.798913+0000 osd.1 (osd.1) 155 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:34.800320+0000 osd.1 (osd.1) 156 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.053701+0000 osd.3 (osd.3) 175 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.055136+0000 osd.3 (osd.3) 176 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.643262+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e317: 8 total, 8 up, 7 in 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:35.644849+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:35.647246+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.204213+0000 osd.4 (osd.4) 181 : cluster [DBG] 62.f scrub starts 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.207835+0000 osd.4 (osd.4) 182 : cluster [DBG] 62.f scrub ok 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.579129+0000 osd.2 (osd.2) 129 : cluster [DBG] 62.3 scrub starts 2024-02-18T00:36:36.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.581298+0000 osd.2 (osd.2) 130 : cluster [DBG] 62.3 scrub ok 2024-02-18T00:36:36.346 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.798913+0000 osd.1 (osd.1) 155 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:36:36.346 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:34.800320+0000 osd.1 (osd.1) 156 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:36:36.346 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.053701+0000 osd.3 (osd.3) 175 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:36:36.346 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.055136+0000 osd.3 (osd.3) 176 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:36:36.346 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.643262+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e317: 8 total, 8 up, 7 in 2024-02-18T00:36:36.346 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:35.644849+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:36.346 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:35.647246+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:36.349 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:36.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:36.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:36.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:36.484 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.204213+0000 osd.4 (osd.4) 181 : cluster [DBG] 62.f scrub starts 2024-02-18T00:36:36.484 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.207835+0000 osd.4 (osd.4) 182 : cluster [DBG] 62.f scrub ok 2024-02-18T00:36:36.484 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.579129+0000 osd.2 (osd.2) 129 : cluster [DBG] 62.3 scrub starts 2024-02-18T00:36:36.484 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.581298+0000 osd.2 (osd.2) 130 : cluster [DBG] 62.3 scrub ok 2024-02-18T00:36:36.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.798913+0000 osd.1 (osd.1) 155 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:36:36.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:34.800320+0000 osd.1 (osd.1) 156 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:36:36.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.053701+0000 osd.3 (osd.3) 175 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:36:36.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.055136+0000 osd.3 (osd.3) 176 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:36:36.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.643262+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e317: 8 total, 8 up, 7 in 2024-02-18T00:36:36.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:35.644849+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:36.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:35.647246+0000 mon.a (mon.0) 2135 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:36.486 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:36.508 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:36.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:36.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:36.612 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:36.643 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:36.678 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:36.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:36.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:36.781 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:36.808 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:36.828 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:36.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:36.831 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:36.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:36.962 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 -- 172.21.15.59:0/1719290321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00fcbb0 msgr2=0x7ff7f00fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/1719290321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00fcbb0 0x7ff7f00fcf90 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 rx=0x7ff7e8009be0 tx=0x7ff7e8004d10).stop 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 -- 172.21.15.59:0/1719290321 shutdown_connections 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/1719290321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f00f7400 0x7ff7f0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/1719290321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00fcbb0 0x7ff7f00fcf90 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/1719290321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fd510 0x7ff7f00f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 -- 172.21.15.59:0/1719290321 >> 172.21.15.59:0/1719290321 conn(0x7ff7f00f2820 msgr2=0x7ff7f00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 -- 172.21.15.59:0/1719290321 shutdown_connections 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.104+0000 7ff7f63aa700 1 -- 172.21.15.59:0/1719290321 wait complete. 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.105+0000 7ff7f63aa700 1 librados: starting msgr at 2024-02-18T00:36:37.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.105+0000 7ff7f63aa700 1 librados: starting objecter 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.105+0000 7ff7f63aa700 1 Processor -- start 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.105+0000 7ff7f63aa700 1 -- start start 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.105+0000 7ff7f63aa700 1 librados: setting wanted keys 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.105+0000 7ff7f63aa700 1 librados: calling monclient init 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00f7400 0x7ff7f010d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fcbb0 0x7ff7f010d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f00fd510 0x7ff7f0107380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0107870 con 0x7ff7f00f7400 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f01079e0 con 0x7ff7f00fd510 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0107ba0 con 0x7ff7f00fcbb0 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ef7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fcbb0 0x7ff7f010d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00f7400 0x7ff7f010d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:37.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ef7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fcbb0 0x7ff7f010d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59560/0 (socket says 172.21.15.59:59560) 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ef7fe700 1 -- 172.21.15.59:0/3606035506 learned_addr learned my addr 172.21.15.59:0/3606035506 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f4947700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f00fd510 0x7ff7f0107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7effff700 1 -- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fcbb0 msgr2=0x7ff7f010d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7effff700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fcbb0 0x7ff7f010d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7effff700 1 -- 172.21.15.59:0/3606035506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f00fd510 msgr2=0x7ff7f0107380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7effff700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f00fd510 0x7ff7f0107380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7effff700 1 -- 172.21.15.59:0/3606035506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e80097c0 con 0x7ff7f00f7400 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ef7fe700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fcbb0 0x7ff7f010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7effff700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00f7400 0x7ff7f010d190 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 rx=0x7ff7e8009fd0 tx=0x7ff7e8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ed7fa700 10 client.?.objecter ms_handle_connect 0x7ff7f00f7400 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ed7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ed7fa700 1 -- 172.21.15.59:0/3606035506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800e040 con 0x7ff7f00f7400 2024-02-18T00:36:37.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ed7fa700 1 -- 172.21.15.59:0/3606035506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7e8018040 con 0x7ff7f00f7400 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f0107e20 con 0x7ff7f00f7400 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7ed7fa700 1 -- 172.21.15.59:0/3606035506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8013bc0 con 0x7ff7f00f7400 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 10 client.31086.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f0113fc0 con 0x7ff7f00f7400 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.106+0000 7ff7f63aa700 1 librados: init done 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.108+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f0108260 con 0x7ff7f00f7400 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.108+0000 7ff7ed7fa700 1 -- 172.21.15.59:0/3606035506 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff7e8007910 con 0x7ff7f00f7400 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.108+0000 7ff7ed7fa700 1 --2- 172.21.15.59:0/3606035506 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff7d8062c90 0x7ff7d8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ef7fe700 1 --2- 172.21.15.59:0/3606035506 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff7d8062c90 0x7ff7d8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ed7fa700 1 -- 172.21.15.59:0/3606035506 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(318..318 src has 44..318) v4 ==== 8406+0+0 (secure 0 0 0) 0x7ff7e807ec30 con 0x7ff7f00f7400 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ed7fa700 10 client.31086.objecter ms_dispatch 0x7ff7f0001550 osd_map(318..318 src has 44..318) v4 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ed7fa700 3 client.31086.objecter handle_osd_map got epochs [318,318] > 0 2024-02-18T00:36:37.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ed7fa700 3 client.31086.objecter handle_osd_map decoding full epoch 318 2024-02-18T00:36:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ed7fa700 20 client.31086.objecter dump_active .. 0 homeless 2024-02-18T00:36:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ed7fa700 10 client.31086.objecter ms_handle_connect 0x7ff7d8062c90 2024-02-18T00:36:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.109+0000 7ff7ef7fe700 1 --2- 172.21.15.59:0/3606035506 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff7d8062c90 0x7ff7d8065150 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 rx=0x7ff7f0108ad0 tx=0x7ff7e400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:37.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.113+0000 7ff7ed7fa700 1 -- 172.21.15.59:0/3606035506 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff7e8051fa0 con 0x7ff7f00f7400 2024-02-18T00:36:37.161 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:37.165 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:37.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.459845+0000 osd.6 (osd.6) 155 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:36:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.460699+0000 osd.6 (osd.6) 156 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:36:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.807395+0000 osd.1 (osd.1) 157 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:36:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.808377+0000 osd.1 (osd.1) 158 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:36:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:35.869441+0000 mgr.y (mgr.24454) 320 : cluster [DBG] pgmap v463: 246 pgs: 32 creating+peering, 214 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:36.460887+0000 osd.0 (osd.0) 149 : cluster [DBG] 62.4 scrub starts 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:36.464356+0000 osd.0 (osd.0) 150 : cluster [DBG] 62.4 scrub ok 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:36.500148+0000 osd.2 (osd.2) 131 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:36.500902+0000 osd.2 (osd.2) 132 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:36.640211+0000 mon.a (mon.0) 2136 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:36.649473+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:36.649620+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e318: 8 total, 8 up, 7 in 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:36.654816+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:36.655526+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:36.663589+0000 mon.c (mon.2) 293 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi059-82957-6","var": "pg_num","format": "json"}]: dispatch 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:36.664008+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:36.664633+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.459845+0000 osd.6 (osd.6) 155 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.460699+0000 osd.6 (osd.6) 156 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.807395+0000 osd.1 (osd.1) 157 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.808377+0000 osd.1 (osd.1) 158 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:35.869441+0000 mgr.y (mgr.24454) 320 : cluster [DBG] pgmap v463: 246 pgs: 32 creating+peering, 214 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:36.460887+0000 osd.0 (osd.0) 149 : cluster [DBG] 62.4 scrub starts 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:36.464356+0000 osd.0 (osd.0) 150 : cluster [DBG] 62.4 scrub ok 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:36.500148+0000 osd.2 (osd.2) 131 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:36:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:36.500902+0000 osd.2 (osd.2) 132 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:36.640211+0000 mon.a (mon.0) 2136 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:36.649473+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:36.649620+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e318: 8 total, 8 up, 7 in 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:36.654816+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:36.655526+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:36.663589+0000 mon.c (mon.2) 293 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi059-82957-6","var": "pg_num","format": "json"}]: dispatch 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:36.664008+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:37.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:36.664633+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:37.266 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:37.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.292+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7f0106150 con 0x7ff7d8062c90 2024-02-18T00:36:37.296 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:37.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.308+0000 7ff7ed7fa700 1 -- 172.21.15.59:0/3606035506 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+596897 (secure 0 0 0) 0x7ff7f0106150 con 0x7ff7d8062c90 2024-02-18T00:36:37.319 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:37.321 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:37.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:37.322 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:36:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 10 librados: watch_flush enter 2024-02-18T00:36:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 10 librados: watch_flush exit 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 20 client.31086.objecter shutdown clearing up homeless session... 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 10 client.31086.objecter successfully canceled tick 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff7d8062c90 msgr2=0x7ff7d8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/3606035506 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff7d8062c90 0x7ff7d8065150 secure :-1 s=READY pgs=3068 cs=0 l=1 rev1=1 rx=0x7ff7f0108ad0 tx=0x7ff7e400b040).stop 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00f7400 msgr2=0x7ff7f010d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00f7400 0x7ff7f010d190 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 rx=0x7ff7e8009fd0 tx=0x7ff7e8004d10).stop 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 shutdown_connections 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/3606035506 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff7d8062c90 0x7ff7d8065150 unknown :-1 s=CLOSED pgs=3068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f00fd510 0x7ff7f0107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7f00f7400 0x7ff7f010d190 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 --2- 172.21.15.59:0/3606035506 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7f00fcbb0 0x7ff7f010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 >> 172.21.15.59:0/3606035506 conn(0x7ff7f00f2820 msgr2=0x7ff7f00fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 shutdown_connections 2024-02-18T00:36:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 -- 172.21.15.59:0/3606035506 wait complete. 2024-02-18T00:36:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:37.313+0000 7ff7f63aa700 1 librados: shutdown 2024-02-18T00:36:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:36:37.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.459845+0000 osd.6 (osd.6) 155 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.460699+0000 osd.6 (osd.6) 156 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.807395+0000 osd.1 (osd.1) 157 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.808377+0000 osd.1 (osd.1) 158 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:35.869441+0000 mgr.y (mgr.24454) 320 : cluster [DBG] pgmap v463: 246 pgs: 32 creating+peering, 214 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:36.460887+0000 osd.0 (osd.0) 149 : cluster [DBG] 62.4 scrub starts 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:36.464356+0000 osd.0 (osd.0) 150 : cluster [DBG] 62.4 scrub ok 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:36.500148+0000 osd.2 (osd.2) 131 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:36.500902+0000 osd.2 (osd.2) 132 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:36.640211+0000 mon.a (mon.0) 2136 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:36.649473+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:36.649620+0000 mon.a (mon.0) 2138 : cluster [DBG] osdmap e318: 8 total, 8 up, 7 in 2024-02-18T00:36:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:36.654816+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:36.655526+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:36.663589+0000 mon.c (mon.2) 293 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi059-82957-6","var": "pg_num","format": "json"}]: dispatch 2024-02-18T00:36:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:36.664008+0000 mon.c (mon.2) 294 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:36.664633+0000 mon.a (mon.0) 2140 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:37.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:36:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:37.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:37.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:37.903 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:37.938 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:37.960 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:37.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:37.963 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:38.006 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:38.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:37.293579+0000 mgr.y (mgr.24454) 321 : audit [DBG] from='client.31086 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:37.528851+0000 osd.2 (osd.2) 133 : cluster [DBG] 62.7 scrub starts 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:37.532312+0000 osd.2 (osd.2) 134 : cluster [DBG] 62.7 scrub ok 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:37.653264+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:37.653340+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:37.653401+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e319: 8 total, 8 up, 7 in 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:37.655418+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T00:36:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:37.655671+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:38.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:37.655783+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T00:36:38.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:37.656196+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:38.291 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:38.295 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:38.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:38.343 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:37.293579+0000 mgr.y (mgr.24454) 321 : audit [DBG] from='client.31086 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:38.343 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:37.528851+0000 osd.2 (osd.2) 133 : cluster [DBG] 62.7 scrub starts 2024-02-18T00:36:38.343 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:37.532312+0000 osd.2 (osd.2) 134 : cluster [DBG] 62.7 scrub ok 2024-02-18T00:36:38.343 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:37.653264+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:38.343 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:37.653340+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:38.343 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:37.653401+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e319: 8 total, 8 up, 7 in 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:37.655418+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:37.655671+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:37.655783+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:37.656196+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:37.293579+0000 mgr.y (mgr.24454) 321 : audit [DBG] from='client.31086 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:37.528851+0000 osd.2 (osd.2) 133 : cluster [DBG] 62.7 scrub starts 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:37.532312+0000 osd.2 (osd.2) 134 : cluster [DBG] 62.7 scrub ok 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:37.653264+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:37.653340+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:38.344 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:37.653401+0000 mon.a (mon.0) 2143 : cluster [DBG] osdmap e319: 8 total, 8 up, 7 in 2024-02-18T00:36:38.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:37.655418+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T00:36:38.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:37.655671+0000 mon.c (mon.2) 296 : audit [INF] from='client.? 172.21.15.59:0/926924076' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:38.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:37.655783+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T00:36:38.345 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:37.656196+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]: dispatch 2024-02-18T00:36:38.396 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:38.428 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:38.450 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:38.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:38.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:38.554 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:38.583 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:38.602 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:38.605 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:38.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:38.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:38.742 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:38.842 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:36:38.842 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:36:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:37.503604+0000 osd.6 (osd.6) 157 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:36:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:37.504564+0000 osd.6 (osd.6) 158 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:36:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:37.869811+0000 mgr.y (mgr.24454) 322 : cluster [DBG] pgmap v466: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:38.580988+0000 osd.2 (osd.2) 135 : cluster [DBG] 62.14 scrub starts 2024-02-18T00:36:39.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:38.583051+0000 osd.2 (osd.2) 136 : cluster [DBG] 62.14 scrub ok 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.660800+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]': finished 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.660884+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:38.660938+0000 mon.a (mon.0) 2148 : cluster [DBG] osdmap e320: 8 total, 8 up, 7 in 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.662983+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.663482+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.686578+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.686909+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.687350+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.687649+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.688027+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:38.688371+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:39.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:37.503604+0000 osd.6 (osd.6) 157 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:37.504564+0000 osd.6 (osd.6) 158 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:37.869811+0000 mgr.y (mgr.24454) 322 : cluster [DBG] pgmap v466: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:38.580988+0000 osd.2 (osd.2) 135 : cluster [DBG] 62.14 scrub starts 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:38.583051+0000 osd.2 (osd.2) 136 : cluster [DBG] 62.14 scrub ok 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.660800+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]': finished 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.660884+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:38.660938+0000 mon.a (mon.0) 2148 : cluster [DBG] osdmap e320: 8 total, 8 up, 7 in 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.662983+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.663482+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.686578+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.686909+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.687350+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.687649+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.688027+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:38.688371+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:37.503604+0000 osd.6 (osd.6) 157 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:37.504564+0000 osd.6 (osd.6) 158 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:37.869811+0000 mgr.y (mgr.24454) 322 : cluster [DBG] pgmap v466: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:38.580988+0000 osd.2 (osd.2) 135 : cluster [DBG] 62.14 scrub starts 2024-02-18T00:36:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:38.583051+0000 osd.2 (osd.2) 136 : cluster [DBG] 62.14 scrub ok 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.660800+0000 mon.a (mon.0) 2146 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_count","val": "8"}]': finished 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.660884+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi059-82882-54"}]': finished 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:38.660938+0000 mon.a (mon.0) 2148 : cluster [DBG] osdmap e320: 8 total, 8 up, 7 in 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.662983+0000 mon.c (mon.2) 297 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.663482+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.686578+0000 mon.c (mon.2) 298 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.686909+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.687350+0000 mon.c (mon.2) 299 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.687649+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.688027+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:39.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:38.688371+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:40.069 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-18T00:36:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:38.555443+0000 osd.6 (osd.6) 159 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:36:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:38.557289+0000 osd.6 (osd.6) 160 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:36:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:38.880489+0000 osd.1 (osd.1) 159 : cluster [DBG] 2.a scrub starts 2024-02-18T00:36:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:38.883383+0000 osd.1 (osd.1) 160 : cluster [DBG] 2.a scrub ok 2024-02-18T00:36:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:39.164586+0000 osd.4 (osd.4) 183 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:39.168075+0000 osd.4 (osd.4) 184 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:39.662418+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:39.662533+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:39.662623+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e321: 8 total, 8 up, 7 in 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:39.664867+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:39.665226+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:39.665358+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:39.666217+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:38.555443+0000 osd.6 (osd.6) 159 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:38.557289+0000 osd.6 (osd.6) 160 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:36:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:38.880489+0000 osd.1 (osd.1) 159 : cluster [DBG] 2.a scrub starts 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:38.883383+0000 osd.1 (osd.1) 160 : cluster [DBG] 2.a scrub ok 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:39.164586+0000 osd.4 (osd.4) 183 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:39.168075+0000 osd.4 (osd.4) 184 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:39.662418+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:39.662533+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:39.662623+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e321: 8 total, 8 up, 7 in 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:39.664867+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:39.665226+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:39.665358+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:40.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:39.666217+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:38.555443+0000 osd.6 (osd.6) 159 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:38.557289+0000 osd.6 (osd.6) 160 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:38.880489+0000 osd.1 (osd.1) 159 : cluster [DBG] 2.a scrub starts 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:38.883383+0000 osd.1 (osd.1) 160 : cluster [DBG] 2.a scrub ok 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:39.164586+0000 osd.4 (osd.4) 183 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:39.168075+0000 osd.4 (osd.4) 184 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:39.662418+0000 mon.a (mon.0) 2153 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:39.662533+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi059-82882-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:39.662623+0000 mon.a (mon.0) 2155 : cluster [DBG] osdmap e321: 8 total, 8 up, 7 in 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:39.664867+0000 mon.c (mon.2) 301 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:39.665226+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T00:36:40.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:39.665358+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:40.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:39.666217+0000 mon.a (mon.0) 2157 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T00:36:41.193 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-02-18T00:36:36.647147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007021,"num_objects":348,"num_object_clones":2,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":348,"num_whiteouts":0,"num_read":1632,"num_read_kb":1260,"num_write":1174,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2518,"ondisk_log_size":2518,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":687,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2173472,"kb_used_data":47780,"kb_used_omap":0,"kb_used_meta":2125568,"kb_avail":654042592,"statfs":{"total":671965249536,"available":669739614208,"internally_reserved":0,"allocated":48926720,"data_stored":32463478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2176581632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.950507"},"pg_stats":[{"pgid":"191.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.642951+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.650814+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,3],"acting":[1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.645987+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.646243+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.644005+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.644858+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.645779+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.642825+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.649647+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.650471+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.646198+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[6,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.645027+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.642818+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.645327+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.645614+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.646041+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"191.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.647662+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.642905+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.655146+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,0],"acting":[1,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"191.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.648903+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.645371+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.648440+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.644824+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.648572+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.645405+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"191.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.646636+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.647790+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"191.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.647269+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"191.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.644525+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"191.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.642829+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.642823+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"191.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-02-18T00:36:35.638287+0000","last_change":"2024-02-18T00:36:35.644944+0000","last_active":"2024-02-18T00:36:35.638287+0000","last_peered":"2024-02-18T00:36:35.638287+0000","last_clean":"2024-02-18T00:36:35.638287+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:36:35.638287+0000","last_undegraded":"2024-02-18T00:36:35.638287+0000","last_fullsized":"2024-02-18T00:36:35.638287+0000","mapping_epoch":317,"log_start":"0'0","ondisk_log_start":"0'0","created":317,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:35.638287+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.638287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":288,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645037+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:36:34.645037+0000","last_peered":"2024-02-18T00:36:34.645037+0000","last_clean":"2024-02-18T00:36:34.645037+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:36:34.645037+0000","last_undegraded":"2024-02-18T00:36:34.645037+0000","last_fullsized":"2024-02-18T00:36:34.645037+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"0'0","reported_seq":250,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801380+0000","last_change":"2024-02-18T00:36:06.658623+0000","last_active":"2024-02-18T00:36:34.801380+0000","last_peered":"2024-02-18T00:36:34.801380+0000","last_clean":"2024-02-18T00:36:34.801380+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:36:34.801380+0000","last_undegraded":"2024-02-18T00:36:34.801380+0000","last_fullsized":"2024-02-18T00:36:34.801380+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":318,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.651421+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:36:34.651421+0000","last_peered":"2024-02-18T00:36:34.651421+0000","last_clean":"2024-02-18T00:36:34.651421+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:36:34.651421+0000","last_undegraded":"2024-02-18T00:36:34.651421+0000","last_fullsized":"2024-02-18T00:36:34.651421+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"245'73","reported_seq":317,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643791+0000","last_change":"2024-02-18T00:36:34.581378+0000","last_active":"2024-02-18T00:36:34.643791+0000","last_peered":"2024-02-18T00:36:34.643791+0000","last_clean":"2024-02-18T00:36:34.643791+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:36:34.643791+0000","last_undegraded":"2024-02-18T00:36:34.643791+0000","last_fullsized":"2024-02-18T00:36:34.643791+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'73","last_scrub_stamp":"2024-02-18T00:36:34.581302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.581302+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"230'120","reported_seq":351,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.650385+0000","last_change":"2024-02-18T00:36:06.647751+0000","last_active":"2024-02-18T00:36:34.650385+0000","last_peered":"2024-02-18T00:36:34.650385+0000","last_clean":"2024-02-18T00:36:34.650385+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:36:34.650385+0000","last_undegraded":"2024-02-18T00:36:34.650385+0000","last_fullsized":"2024-02-18T00:36:34.650385+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":332,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645105+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:36:34.645105+0000","last_peered":"2024-02-18T00:36:34.645105+0000","last_clean":"2024-02-18T00:36:34.645105+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:36:34.645105+0000","last_undegraded":"2024-02-18T00:36:34.645105+0000","last_fullsized":"2024-02-18T00:36:34.645105+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"267'124","reported_seq":325,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644977+0000","last_change":"2024-02-18T00:36:06.654594+0000","last_active":"2024-02-18T00:36:34.644977+0000","last_peered":"2024-02-18T00:36:34.644977+0000","last_clean":"2024-02-18T00:36:34.644977+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:36:34.644977+0000","last_undegraded":"2024-02-18T00:36:34.644977+0000","last_fullsized":"2024-02-18T00:36:34.644977+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":124,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"267'73","reported_seq":306,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646822+0000","last_change":"2024-02-18T00:36:06.651545+0000","last_active":"2024-02-18T00:36:34.646822+0000","last_peered":"2024-02-18T00:36:34.646822+0000","last_clean":"2024-02-18T00:36:34.646822+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:36:34.646822+0000","last_undegraded":"2024-02-18T00:36:34.646822+0000","last_fullsized":"2024-02-18T00:36:34.646822+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"215'68","reported_seq":277,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801544+0000","last_change":"2024-02-18T00:36:06.662037+0000","last_active":"2024-02-18T00:36:34.801544+0000","last_peered":"2024-02-18T00:36:34.801544+0000","last_clean":"2024-02-18T00:36:34.801544+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:36:34.801544+0000","last_undegraded":"2024-02-18T00:36:34.801544+0000","last_fullsized":"2024-02-18T00:36:34.801544+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-18T00:36:02.645624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.645624+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"229'51","reported_seq":294,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644561+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:36:34.644561+0000","last_peered":"2024-02-18T00:36:34.644561+0000","last_clean":"2024-02-18T00:36:34.644561+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:36:34.644561+0000","last_undegraded":"2024-02-18T00:36:34.644561+0000","last_fullsized":"2024-02-18T00:36:34.644561+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"265'39","reported_seq":260,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643393+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:36:34.643393+0000","last_peered":"2024-02-18T00:36:34.643393+0000","last_clean":"2024-02-18T00:36:34.643393+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:36:34.643393+0000","last_undegraded":"2024-02-18T00:36:34.643393+0000","last_fullsized":"2024-02-18T00:36:34.643393+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"246'54","reported_seq":274,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.658411+0000","last_change":"2024-02-18T00:36:11.810050+0000","last_active":"2024-02-18T00:36:34.658411+0000","last_peered":"2024-02-18T00:36:34.658411+0000","last_clean":"2024-02-18T00:36:34.658411+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:36:34.658411+0000","last_undegraded":"2024-02-18T00:36:34.658411+0000","last_fullsized":"2024-02-18T00:36:34.658411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'54","last_scrub_stamp":"2024-02-18T00:36:11.809925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.809925+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"213'17","reported_seq":265,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656719+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:36:34.656719+0000","last_peered":"2024-02-18T00:36:34.656719+0000","last_clean":"2024-02-18T00:36:34.656719+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:36:34.656719+0000","last_undegraded":"2024-02-18T00:36:34.656719+0000","last_fullsized":"2024-02-18T00:36:34.656719+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'28","reported_seq":298,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644838+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:36:34.644838+0000","last_peered":"2024-02-18T00:36:34.644838+0000","last_clean":"2024-02-18T00:36:34.644838+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:36:34.644838+0000","last_undegraded":"2024-02-18T00:36:34.644838+0000","last_fullsized":"2024-02-18T00:36:34.644838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"288'81","reported_seq":317,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645573+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:36:34.645573+0000","last_peered":"2024-02-18T00:36:34.645573+0000","last_clean":"2024-02-18T00:36:34.645573+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:36:34.645573+0000","last_undegraded":"2024-02-18T00:36:34.645573+0000","last_fullsized":"2024-02-18T00:36:34.645573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":81,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":299,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645656+0000","last_change":"2024-02-18T00:36:34.207908+0000","last_active":"2024-02-18T00:36:34.645656+0000","last_peered":"2024-02-18T00:36:34.645656+0000","last_clean":"2024-02-18T00:36:34.645656+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:36:34.645656+0000","last_undegraded":"2024-02-18T00:36:34.645656+0000","last_fullsized":"2024-02-18T00:36:34.645656+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:36:34.207840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.207840+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"209'56","reported_seq":312,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801446+0000","last_change":"2024-02-18T00:36:06.661736+0000","last_active":"2024-02-18T00:36:34.801446+0000","last_peered":"2024-02-18T00:36:34.801446+0000","last_clean":"2024-02-18T00:36:34.801446+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:36:34.801446+0000","last_undegraded":"2024-02-18T00:36:34.801446+0000","last_fullsized":"2024-02-18T00:36:34.801446+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'56","last_scrub_stamp":"2024-02-18T00:36:04.656413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.656413+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"214'19","reported_seq":270,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801325+0000","last_change":"2024-02-18T00:36:06.658579+0000","last_active":"2024-02-18T00:36:34.801325+0000","last_peered":"2024-02-18T00:36:34.801325+0000","last_clean":"2024-02-18T00:36:34.801325+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:36:34.801325+0000","last_undegraded":"2024-02-18T00:36:34.801325+0000","last_fullsized":"2024-02-18T00:36:34.801325+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:35:53.563562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.563562+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"288'72","reported_seq":312,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643486+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:36:34.643486+0000","last_peered":"2024-02-18T00:36:34.643486+0000","last_clean":"2024-02-18T00:36:34.643486+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:36:34.643486+0000","last_undegraded":"2024-02-18T00:36:34.643486+0000","last_fullsized":"2024-02-18T00:36:34.643486+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"210'17","reported_seq":260,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.648375+0000","last_change":"2024-02-18T00:36:06.649777+0000","last_active":"2024-02-18T00:36:34.648375+0000","last_peered":"2024-02-18T00:36:34.648375+0000","last_clean":"2024-02-18T00:36:34.648375+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:36:34.648375+0000","last_undegraded":"2024-02-18T00:36:34.648375+0000","last_fullsized":"2024-02-18T00:36:34.648375+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646287+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:36:34.646287+0000","last_peered":"2024-02-18T00:36:34.646287+0000","last_clean":"2024-02-18T00:36:34.646287+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:36:34.646287+0000","last_undegraded":"2024-02-18T00:36:34.646287+0000","last_fullsized":"2024-02-18T00:36:34.646287+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"304'19","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656734+0000","last_change":"2024-02-18T00:36:06.659238+0000","last_active":"2024-02-18T00:36:34.656734+0000","last_peered":"2024-02-18T00:36:34.656734+0000","last_clean":"2024-02-18T00:36:34.656734+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:36:34.656734+0000","last_undegraded":"2024-02-18T00:36:34.656734+0000","last_fullsized":"2024-02-18T00:36:34.656734+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000019,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":344,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654942+0000","last_change":"2024-02-18T00:36:34.017135+0000","last_active":"2024-02-18T00:36:34.654942+0000","last_peered":"2024-02-18T00:36:34.654942+0000","last_clean":"2024-02-18T00:36:34.654942+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:36:34.654942+0000","last_undegraded":"2024-02-18T00:36:34.654942+0000","last_fullsized":"2024-02-18T00:36:34.654942+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:36:34.017065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.017065+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":231,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656252+0000","last_change":"2024-02-18T00:34:15.230999+0000","last_active":"2024-02-18T00:36:34.656252+0000","last_peered":"2024-02-18T00:36:34.656252+0000","last_clean":"2024-02-18T00:36:34.656252+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:36:34.656252+0000","last_undegraded":"2024-02-18T00:36:34.656252+0000","last_fullsized":"2024-02-18T00:36:34.656252+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.230938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.230938+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":351,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643758+0000","last_change":"2024-02-18T00:36:06.651013+0000","last_active":"2024-02-18T00:36:34.643758+0000","last_peered":"2024-02-18T00:36:34.643758+0000","last_clean":"2024-02-18T00:36:34.643758+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:36:34.643758+0000","last_undegraded":"2024-02-18T00:36:34.643758+0000","last_fullsized":"2024-02-18T00:36:34.643758+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":230,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.652947+0000","last_change":"2024-02-18T00:34:39.758860+0000","last_active":"2024-02-18T00:36:34.652947+0000","last_peered":"2024-02-18T00:36:34.652947+0000","last_clean":"2024-02-18T00:36:34.652947+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:36:34.652947+0000","last_undegraded":"2024-02-18T00:36:34.652947+0000","last_fullsized":"2024-02-18T00:36:34.652947+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.758794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.758794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":290,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644570+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:36:34.644570+0000","last_peered":"2024-02-18T00:36:34.644570+0000","last_clean":"2024-02-18T00:36:34.644570+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:36:34.644570+0000","last_undegraded":"2024-02-18T00:36:34.644570+0000","last_fullsized":"2024-02-18T00:36:34.644570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":187,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645939+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:36:34.645939+0000","last_peered":"2024-02-18T00:36:34.645939+0000","last_clean":"2024-02-18T00:36:34.645939+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:36:34.645939+0000","last_undegraded":"2024-02-18T00:36:34.645939+0000","last_fullsized":"2024-02-18T00:36:34.645939+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":345,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.651332+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:36:34.651332+0000","last_peered":"2024-02-18T00:36:34.651332+0000","last_clean":"2024-02-18T00:36:34.651332+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:36:34.651332+0000","last_undegraded":"2024-02-18T00:36:34.651332+0000","last_fullsized":"2024-02-18T00:36:34.651332+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":185,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646435+0000","last_change":"2024-02-18T00:34:26.276135+0000","last_active":"2024-02-18T00:36:34.646435+0000","last_peered":"2024-02-18T00:36:34.646435+0000","last_clean":"2024-02-18T00:36:34.646435+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:36:34.646435+0000","last_undegraded":"2024-02-18T00:36:34.646435+0000","last_fullsized":"2024-02-18T00:36:34.646435+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.276071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:38.977852+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.276071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":322,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.655897+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:36:34.655897+0000","last_peered":"2024-02-18T00:36:34.655897+0000","last_clean":"2024-02-18T00:36:34.655897+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:36:34.655897+0000","last_undegraded":"2024-02-18T00:36:34.655897+0000","last_fullsized":"2024-02-18T00:36:34.655897+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":237,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.647495+0000","last_change":"2024-02-18T00:33:43.346148+0000","last_active":"2024-02-18T00:36:34.647495+0000","last_peered":"2024-02-18T00:36:34.647495+0000","last_clean":"2024-02-18T00:36:34.647495+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:36:34.647495+0000","last_undegraded":"2024-02-18T00:36:34.647495+0000","last_fullsized":"2024-02-18T00:36:34.647495+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:33:43.346088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.346088+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"213'17","reported_seq":271,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.652861+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:36:34.652861+0000","last_peered":"2024-02-18T00:36:34.652861+0000","last_clean":"2024-02-18T00:36:34.652861+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:36:34.652861+0000","last_undegraded":"2024-02-18T00:36:34.652861+0000","last_fullsized":"2024-02-18T00:36:34.652861+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":236,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:35.460813+0000","last_change":"2024-02-18T00:36:35.460784+0000","last_active":"2024-02-18T00:36:35.460813+0000","last_peered":"2024-02-18T00:36:35.460813+0000","last_clean":"2024-02-18T00:36:35.460813+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:36:35.460813+0000","last_undegraded":"2024-02-18T00:36:35.460813+0000","last_fullsized":"2024-02-18T00:36:35.460813+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.460703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.460703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":296,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646737+0000","last_change":"2024-02-18T00:36:06.651247+0000","last_active":"2024-02-18T00:36:34.646737+0000","last_peered":"2024-02-18T00:36:34.646737+0000","last_clean":"2024-02-18T00:36:34.646737+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:36:34.646737+0000","last_undegraded":"2024-02-18T00:36:34.646737+0000","last_fullsized":"2024-02-18T00:36:34.646737+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:33:19.952766+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":221,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.658492+0000","last_change":"2024-02-18T00:33:55.783013+0000","last_active":"2024-02-18T00:36:34.658492+0000","last_peered":"2024-02-18T00:36:34.658492+0000","last_clean":"2024-02-18T00:36:34.658492+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:36:34.658492+0000","last_undegraded":"2024-02-18T00:36:34.658492+0000","last_fullsized":"2024-02-18T00:36:34.658492+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:55.782955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:55.782955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":256,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.655617+0000","last_change":"2024-02-18T00:36:06.659436+0000","last_active":"2024-02-18T00:36:34.655617+0000","last_peered":"2024-02-18T00:36:34.655617+0000","last_clean":"2024-02-18T00:36:34.655617+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:36:34.655617+0000","last_undegraded":"2024-02-18T00:36:34.655617+0000","last_fullsized":"2024-02-18T00:36:34.655617+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":229,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657476+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:36:34.657476+0000","last_peered":"2024-02-18T00:36:34.657476+0000","last_clean":"2024-02-18T00:36:34.657476+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:36:34.657476+0000","last_undegraded":"2024-02-18T00:36:34.657476+0000","last_fullsized":"2024-02-18T00:36:34.657476+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":265,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653831+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:36:34.653831+0000","last_peered":"2024-02-18T00:36:34.653831+0000","last_clean":"2024-02-18T00:36:34.653831+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:36:34.653831+0000","last_undegraded":"2024-02-18T00:36:34.653831+0000","last_fullsized":"2024-02-18T00:36:34.653831+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":302,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643868+0000","last_change":"2024-02-18T00:33:48.736504+0000","last_active":"2024-02-18T00:36:34.643868+0000","last_peered":"2024-02-18T00:36:34.643868+0000","last_clean":"2024-02-18T00:36:34.643868+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:36:34.643868+0000","last_undegraded":"2024-02-18T00:36:34.643868+0000","last_fullsized":"2024-02-18T00:36:34.643868+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:40.834878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:40.834878+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":267,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.652056+0000","last_change":"2024-02-18T00:36:06.657624+0000","last_active":"2024-02-18T00:36:34.652056+0000","last_peered":"2024-02-18T00:36:34.652056+0000","last_clean":"2024-02-18T00:36:34.652056+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:36:34.652056+0000","last_undegraded":"2024-02-18T00:36:34.652056+0000","last_fullsized":"2024-02-18T00:36:34.652056+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":254,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.651852+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:36:34.651852+0000","last_peered":"2024-02-18T00:36:34.651852+0000","last_clean":"2024-02-18T00:36:34.651852+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:36:34.651852+0000","last_undegraded":"2024-02-18T00:36:34.651852+0000","last_fullsized":"2024-02-18T00:36:34.651852+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":232,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.642942+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:36:34.642942+0000","last_peered":"2024-02-18T00:36:34.642942+0000","last_clean":"2024-02-18T00:36:34.642942+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:36:34.642942+0000","last_undegraded":"2024-02-18T00:36:34.642942+0000","last_fullsized":"2024-02-18T00:36:34.642942+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":241,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657977+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:36:34.657977+0000","last_peered":"2024-02-18T00:36:34.657977+0000","last_clean":"2024-02-18T00:36:34.657977+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:36:34.657977+0000","last_undegraded":"2024-02-18T00:36:34.657977+0000","last_fullsized":"2024-02-18T00:36:34.657977+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":364,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.651967+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:36:34.651967+0000","last_peered":"2024-02-18T00:36:34.651967+0000","last_clean":"2024-02-18T00:36:34.651967+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:36:34.651967+0000","last_undegraded":"2024-02-18T00:36:34.651967+0000","last_fullsized":"2024-02-18T00:36:34.651967+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":295,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645800+0000","last_change":"2024-02-18T00:34:26.060181+0000","last_active":"2024-02-18T00:36:34.645800+0000","last_peered":"2024-02-18T00:36:34.645800+0000","last_clean":"2024-02-18T00:36:34.645800+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:36:34.645800+0000","last_undegraded":"2024-02-18T00:36:34.645800+0000","last_fullsized":"2024-02-18T00:36:34.645800+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.060096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.060096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":276,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654412+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:36:34.654412+0000","last_peered":"2024-02-18T00:36:34.654412+0000","last_clean":"2024-02-18T00:36:34.654412+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:36:34.654412+0000","last_undegraded":"2024-02-18T00:36:34.654412+0000","last_fullsized":"2024-02-18T00:36:34.654412+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.805261+0000","last_change":"2024-02-18T00:35:20.428964+0000","last_active":"2024-02-18T00:36:34.805261+0000","last_peered":"2024-02-18T00:36:34.805261+0000","last_clean":"2024-02-18T00:36:34.805261+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:36:34.805261+0000","last_undegraded":"2024-02-18T00:36:34.805261+0000","last_fullsized":"2024-02-18T00:36:34.805261+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:20.428893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:20.428893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":332,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.648907+0000","last_change":"2024-02-18T00:35:54.188262+0000","last_active":"2024-02-18T00:36:34.648907+0000","last_peered":"2024-02-18T00:36:34.648907+0000","last_clean":"2024-02-18T00:36:34.648907+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:36:34.648907+0000","last_undegraded":"2024-02-18T00:36:34.648907+0000","last_fullsized":"2024-02-18T00:36:34.648907+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:35:54.188188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.188188+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":284,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.799877+0000","last_change":"2024-02-18T00:34:24.334917+0000","last_active":"2024-02-18T00:36:34.799877+0000","last_peered":"2024-02-18T00:36:34.799877+0000","last_clean":"2024-02-18T00:36:34.799877+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:36:34.799877+0000","last_undegraded":"2024-02-18T00:36:34.799877+0000","last_fullsized":"2024-02-18T00:36:34.799877+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.334814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.334814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643639+0000","last_change":"2024-02-18T00:33:42.768749+0000","last_active":"2024-02-18T00:36:34.643639+0000","last_peered":"2024-02-18T00:36:34.643639+0000","last_clean":"2024-02-18T00:36:34.643639+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:36:34.643639+0000","last_undegraded":"2024-02-18T00:36:34.643639+0000","last_fullsized":"2024-02-18T00:36:34.643639+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:42.768630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:42.768630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":294,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.650013+0000","last_change":"2024-02-18T00:35:42.683115+0000","last_active":"2024-02-18T00:36:34.650013+0000","last_peered":"2024-02-18T00:36:34.650013+0000","last_clean":"2024-02-18T00:36:34.650013+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:36:34.650013+0000","last_undegraded":"2024-02-18T00:36:34.650013+0000","last_fullsized":"2024-02-18T00:36:34.650013+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":232,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656804+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:36:34.656804+0000","last_peered":"2024-02-18T00:36:34.656804+0000","last_clean":"2024-02-18T00:36:34.656804+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:36:34.656804+0000","last_undegraded":"2024-02-18T00:36:34.656804+0000","last_fullsized":"2024-02-18T00:36:34.656804+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":365,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657633+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:36:34.657633+0000","last_peered":"2024-02-18T00:36:34.657633+0000","last_clean":"2024-02-18T00:36:34.657633+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:36:34.657633+0000","last_undegraded":"2024-02-18T00:36:34.657633+0000","last_fullsized":"2024-02-18T00:36:34.657633+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":284,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644551+0000","last_change":"2024-02-18T00:33:46.861306+0000","last_active":"2024-02-18T00:36:34.644551+0000","last_peered":"2024-02-18T00:36:34.644551+0000","last_clean":"2024-02-18T00:36:34.644551+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:36:34.644551+0000","last_undegraded":"2024-02-18T00:36:34.644551+0000","last_fullsized":"2024-02-18T00:36:34.644551+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.861233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.861233+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643874+0000","last_change":"2024-02-18T00:33:53.005012+0000","last_active":"2024-02-18T00:36:34.643874+0000","last_peered":"2024-02-18T00:36:34.643874+0000","last_clean":"2024-02-18T00:36:34.643874+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:36:34.643874+0000","last_undegraded":"2024-02-18T00:36:34.643874+0000","last_fullsized":"2024-02-18T00:36:34.643874+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.004903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.004903+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":184,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646048+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:36:34.646048+0000","last_peered":"2024-02-18T00:36:34.646048+0000","last_clean":"2024-02-18T00:36:34.646048+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:36:34.646048+0000","last_undegraded":"2024-02-18T00:36:34.646048+0000","last_fullsized":"2024-02-18T00:36:34.646048+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":222,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.652845+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:36:34.652845+0000","last_peered":"2024-02-18T00:36:34.652845+0000","last_clean":"2024-02-18T00:36:34.652845+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:36:34.652845+0000","last_undegraded":"2024-02-18T00:36:34.652845+0000","last_fullsized":"2024-02-18T00:36:34.652845+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":315,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.647402+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:36:34.647402+0000","last_peered":"2024-02-18T00:36:34.647402+0000","last_clean":"2024-02-18T00:36:34.647402+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:36:34.647402+0000","last_undegraded":"2024-02-18T00:36:34.647402+0000","last_fullsized":"2024-02-18T00:36:34.647402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":184,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645534+0000","last_change":"2024-02-18T00:34:18.301213+0000","last_active":"2024-02-18T00:36:34.645534+0000","last_peered":"2024-02-18T00:36:34.645534+0000","last_clean":"2024-02-18T00:36:34.645534+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:36:34.645534+0000","last_undegraded":"2024-02-18T00:36:34.645534+0000","last_fullsized":"2024-02-18T00:36:34.645534+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.301136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.301136+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.804473+0000","last_change":"2024-02-18T00:34:39.453471+0000","last_active":"2024-02-18T00:36:34.804473+0000","last_peered":"2024-02-18T00:36:34.804473+0000","last_clean":"2024-02-18T00:36:34.804473+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:36:34.804473+0000","last_undegraded":"2024-02-18T00:36:34.804473+0000","last_fullsized":"2024-02-18T00:36:34.804473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.453312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.453312+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.647483+0000","last_change":"2024-02-18T00:33:45.813625+0000","last_active":"2024-02-18T00:36:34.647483+0000","last_peered":"2024-02-18T00:36:34.647483+0000","last_clean":"2024-02-18T00:36:34.647483+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:36:34.647483+0000","last_undegraded":"2024-02-18T00:36:34.647483+0000","last_fullsized":"2024-02-18T00:36:34.647483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:45.813514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.813514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":68,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.642072+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:36:34.642072+0000","last_peered":"2024-02-18T00:36:34.642072+0000","last_clean":"2024-02-18T00:36:34.642072+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:36:34.642072+0000","last_undegraded":"2024-02-18T00:36:34.642072+0000","last_fullsized":"2024-02-18T00:36:34.642072+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":297,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645353+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:36:34.645353+0000","last_peered":"2024-02-18T00:36:34.645353+0000","last_clean":"2024-02-18T00:36:34.645353+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:36:34.645353+0000","last_undegraded":"2024-02-18T00:36:34.645353+0000","last_fullsized":"2024-02-18T00:36:34.645353+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":293,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801616+0000","last_change":"2024-02-18T00:34:23.295161+0000","last_active":"2024-02-18T00:36:34.801616+0000","last_peered":"2024-02-18T00:36:34.801616+0000","last_clean":"2024-02-18T00:36:34.801616+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:36:34.801616+0000","last_undegraded":"2024-02-18T00:36:34.801616+0000","last_fullsized":"2024-02-18T00:36:34.801616+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.295051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.295051+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":392,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645245+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:36:34.645245+0000","last_peered":"2024-02-18T00:36:34.645245+0000","last_clean":"2024-02-18T00:36:34.645245+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:36:34.645245+0000","last_undegraded":"2024-02-18T00:36:34.645245+0000","last_fullsized":"2024-02-18T00:36:34.645245+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.647330+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:36:34.647330+0000","last_peered":"2024-02-18T00:36:34.647330+0000","last_clean":"2024-02-18T00:36:34.647330+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:36:34.647330+0000","last_undegraded":"2024-02-18T00:36:34.647330+0000","last_fullsized":"2024-02-18T00:36:34.647330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.802755+0000","last_change":"2024-02-18T00:35:29.520701+0000","last_active":"2024-02-18T00:36:34.802755+0000","last_peered":"2024-02-18T00:36:34.802755+0000","last_clean":"2024-02-18T00:36:34.802755+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:36:34.802755+0000","last_undegraded":"2024-02-18T00:36:34.802755+0000","last_fullsized":"2024-02-18T00:36:34.802755+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.520632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.520632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":288,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659267+0000","last_change":"2024-02-18T00:36:33.853442+0000","last_active":"2024-02-18T00:36:34.659267+0000","last_peered":"2024-02-18T00:36:34.659267+0000","last_clean":"2024-02-18T00:36:34.659267+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:36:34.659267+0000","last_undegraded":"2024-02-18T00:36:34.659267+0000","last_fullsized":"2024-02-18T00:36:34.659267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:36:33.853337+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:36:33.853337+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":307,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801617+0000","last_change":"2024-02-18T00:36:08.716901+0000","last_active":"2024-02-18T00:36:34.801617+0000","last_peered":"2024-02-18T00:36:34.801617+0000","last_clean":"2024-02-18T00:36:34.801617+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:36:34.801617+0000","last_undegraded":"2024-02-18T00:36:34.801617+0000","last_fullsized":"2024-02-18T00:36:34.801617+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.716777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.716777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":376,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.647322+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:36:34.647322+0000","last_peered":"2024-02-18T00:36:34.647322+0000","last_clean":"2024-02-18T00:36:34.647322+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:36:34.647322+0000","last_undegraded":"2024-02-18T00:36:34.647322+0000","last_fullsized":"2024-02-18T00:36:34.647322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":276,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657180+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:36:34.657180+0000","last_peered":"2024-02-18T00:36:34.657180+0000","last_clean":"2024-02-18T00:36:34.657180+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:36:34.657180+0000","last_undegraded":"2024-02-18T00:36:34.657180+0000","last_fullsized":"2024-02-18T00:36:34.657180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644706+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:36:34.644706+0000","last_peered":"2024-02-18T00:36:34.644706+0000","last_clean":"2024-02-18T00:36:34.644706+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:36:34.644706+0000","last_undegraded":"2024-02-18T00:36:34.644706+0000","last_fullsized":"2024-02-18T00:36:34.644706+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":339,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646427+0000","last_change":"2024-02-18T00:35:43.219869+0000","last_active":"2024-02-18T00:36:34.646427+0000","last_peered":"2024-02-18T00:36:34.646427+0000","last_clean":"2024-02-18T00:36:34.646427+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:36:34.646427+0000","last_undegraded":"2024-02-18T00:36:34.646427+0000","last_fullsized":"2024-02-18T00:36:34.646427+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:33:45.311259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.311259+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":254,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644461+0000","last_change":"2024-02-18T00:34:27.927933+0000","last_active":"2024-02-18T00:36:34.644461+0000","last_peered":"2024-02-18T00:36:34.644461+0000","last_clean":"2024-02-18T00:36:34.644461+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:36:34.644461+0000","last_undegraded":"2024-02-18T00:36:34.644461+0000","last_fullsized":"2024-02-18T00:36:34.644461+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.927871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.927871+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659632+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:36:34.659632+0000","last_peered":"2024-02-18T00:36:34.659632+0000","last_clean":"2024-02-18T00:36:34.659632+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:36:34.659632+0000","last_undegraded":"2024-02-18T00:36:34.659632+0000","last_fullsized":"2024-02-18T00:36:34.659632+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":277,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656857+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:36:34.656857+0000","last_peered":"2024-02-18T00:36:34.656857+0000","last_clean":"2024-02-18T00:36:34.656857+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:36:34.656857+0000","last_undegraded":"2024-02-18T00:36:34.656857+0000","last_fullsized":"2024-02-18T00:36:34.656857+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645669+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:36:34.645669+0000","last_peered":"2024-02-18T00:36:34.645669+0000","last_clean":"2024-02-18T00:36:34.645669+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:36:34.645669+0000","last_undegraded":"2024-02-18T00:36:34.645669+0000","last_fullsized":"2024-02-18T00:36:34.645669+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":376,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653377+0000","last_change":"2024-02-18T00:33:54.753052+0000","last_active":"2024-02-18T00:36:34.653377+0000","last_peered":"2024-02-18T00:36:34.653377+0000","last_clean":"2024-02-18T00:36:34.653377+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:36:34.653377+0000","last_undegraded":"2024-02-18T00:36:34.653377+0000","last_fullsized":"2024-02-18T00:36:34.653377+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:33:54.752985+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:30:22.716226+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.752985+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":318,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644928+0000","last_change":"2024-02-18T00:33:53.320429+0000","last_active":"2024-02-18T00:36:34.644928+0000","last_peered":"2024-02-18T00:36:34.644928+0000","last_clean":"2024-02-18T00:36:34.644928+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:36:34.644928+0000","last_undegraded":"2024-02-18T00:36:34.644928+0000","last_fullsized":"2024-02-18T00:36:34.644928+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:33:53.320356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.320356+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":307,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656960+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:36:34.656960+0000","last_peered":"2024-02-18T00:36:34.656960+0000","last_clean":"2024-02-18T00:36:34.656960+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:36:34.656960+0000","last_undegraded":"2024-02-18T00:36:34.656960+0000","last_fullsized":"2024-02-18T00:36:34.656960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":184,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654206+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:36:34.654206+0000","last_peered":"2024-02-18T00:36:34.654206+0000","last_clean":"2024-02-18T00:36:34.654206+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:36:34.654206+0000","last_undegraded":"2024-02-18T00:36:34.654206+0000","last_fullsized":"2024-02-18T00:36:34.654206+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":222,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656799+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:36:34.656799+0000","last_peered":"2024-02-18T00:36:34.656799+0000","last_clean":"2024-02-18T00:36:34.656799+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:36:34.656799+0000","last_undegraded":"2024-02-18T00:36:34.656799+0000","last_fullsized":"2024-02-18T00:36:34.656799+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":289,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645261+0000","last_change":"2024-02-18T00:33:50.287103+0000","last_active":"2024-02-18T00:36:34.645261+0000","last_peered":"2024-02-18T00:36:34.645261+0000","last_clean":"2024-02-18T00:36:34.645261+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:36:34.645261+0000","last_undegraded":"2024-02-18T00:36:34.645261+0000","last_fullsized":"2024-02-18T00:36:34.645261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.287007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.287007+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":384,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653266+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:36:34.653266+0000","last_peered":"2024-02-18T00:36:34.653266+0000","last_clean":"2024-02-18T00:36:34.653266+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:36:34.653266+0000","last_undegraded":"2024-02-18T00:36:34.653266+0000","last_fullsized":"2024-02-18T00:36:34.653266+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":298,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.805437+0000","last_change":"2024-02-18T00:36:09.647995+0000","last_active":"2024-02-18T00:36:34.805437+0000","last_peered":"2024-02-18T00:36:34.805437+0000","last_clean":"2024-02-18T00:36:34.805437+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:36:34.805437+0000","last_undegraded":"2024-02-18T00:36:34.805437+0000","last_fullsized":"2024-02-18T00:36:34.805437+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.647928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645809+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:36:34.645809+0000","last_peered":"2024-02-18T00:36:34.645809+0000","last_clean":"2024-02-18T00:36:34.645809+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:36:34.645809+0000","last_undegraded":"2024-02-18T00:36:34.645809+0000","last_fullsized":"2024-02-18T00:36:34.645809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":223,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654634+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:36:34.654634+0000","last_peered":"2024-02-18T00:36:34.654634+0000","last_clean":"2024-02-18T00:36:34.654634+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:36:34.654634+0000","last_undegraded":"2024-02-18T00:36:34.654634+0000","last_fullsized":"2024-02-18T00:36:34.654634+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657216+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:36:34.657216+0000","last_peered":"2024-02-18T00:36:34.657216+0000","last_clean":"2024-02-18T00:36:34.657216+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:36:34.657216+0000","last_undegraded":"2024-02-18T00:36:34.657216+0000","last_fullsized":"2024-02-18T00:36:34.657216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":360,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.648764+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:36:34.648764+0000","last_peered":"2024-02-18T00:36:34.648764+0000","last_clean":"2024-02-18T00:36:34.648764+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:36:34.648764+0000","last_undegraded":"2024-02-18T00:36:34.648764+0000","last_fullsized":"2024-02-18T00:36:34.648764+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":191,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643204+0000","last_change":"2024-02-18T00:34:32.021327+0000","last_active":"2024-02-18T00:36:34.643204+0000","last_peered":"2024-02-18T00:36:34.643204+0000","last_clean":"2024-02-18T00:36:34.643204+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:36:34.643204+0000","last_undegraded":"2024-02-18T00:36:34.643204+0000","last_fullsized":"2024-02-18T00:36:34.643204+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:34:32.021219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.021219+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.802183+0000","last_change":"2024-02-18T00:35:13.452285+0000","last_active":"2024-02-18T00:36:34.802183+0000","last_peered":"2024-02-18T00:36:34.802183+0000","last_clean":"2024-02-18T00:36:34.802183+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:36:34.802183+0000","last_undegraded":"2024-02-18T00:36:34.802183+0000","last_fullsized":"2024-02-18T00:36:34.802183+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.452220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.452220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":223,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659304+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:36:34.659304+0000","last_peered":"2024-02-18T00:36:34.659304+0000","last_clean":"2024-02-18T00:36:34.659304+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:36:34.659304+0000","last_undegraded":"2024-02-18T00:36:34.659304+0000","last_fullsized":"2024-02-18T00:36:34.659304+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659818+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:36:34.659818+0000","last_peered":"2024-02-18T00:36:34.659818+0000","last_clean":"2024-02-18T00:36:34.659818+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:36:34.659818+0000","last_undegraded":"2024-02-18T00:36:34.659818+0000","last_fullsized":"2024-02-18T00:36:34.659818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":298,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645925+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:36:34.645925+0000","last_peered":"2024-02-18T00:36:34.645925+0000","last_clean":"2024-02-18T00:36:34.645925+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:36:34.645925+0000","last_undegraded":"2024-02-18T00:36:34.645925+0000","last_fullsized":"2024-02-18T00:36:34.645925+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":260,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.802570+0000","last_change":"2024-02-18T00:36:34.800419+0000","last_active":"2024-02-18T00:36:34.802570+0000","last_peered":"2024-02-18T00:36:34.802570+0000","last_clean":"2024-02-18T00:36:34.802570+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:36:34.802570+0000","last_undegraded":"2024-02-18T00:36:34.802570+0000","last_fullsized":"2024-02-18T00:36:34.802570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:34.800327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.800327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":281,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.649289+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:36:34.649289+0000","last_peered":"2024-02-18T00:36:34.649289+0000","last_clean":"2024-02-18T00:36:34.649289+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:36:34.649289+0000","last_undegraded":"2024-02-18T00:36:34.649289+0000","last_fullsized":"2024-02-18T00:36:34.649289+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.804495+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:36:34.804495+0000","last_peered":"2024-02-18T00:36:34.804495+0000","last_clean":"2024-02-18T00:36:34.804495+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:36:34.804495+0000","last_undegraded":"2024-02-18T00:36:34.804495+0000","last_fullsized":"2024-02-18T00:36:34.804495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":232,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644577+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:36:34.644577+0000","last_peered":"2024-02-18T00:36:34.644577+0000","last_clean":"2024-02-18T00:36:34.644577+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:36:34.644577+0000","last_undegraded":"2024-02-18T00:36:34.644577+0000","last_fullsized":"2024-02-18T00:36:34.644577+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":184,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644450+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:36:34.644450+0000","last_peered":"2024-02-18T00:36:34.644450+0000","last_clean":"2024-02-18T00:36:34.644450+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:36:34.644450+0000","last_undegraded":"2024-02-18T00:36:34.644450+0000","last_fullsized":"2024-02-18T00:36:34.644450+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":334,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656108+0000","last_change":"2024-02-18T00:34:28.290944+0000","last_active":"2024-02-18T00:36:34.656108+0000","last_peered":"2024-02-18T00:36:34.656108+0000","last_clean":"2024-02-18T00:36:34.656108+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:36:34.656108+0000","last_undegraded":"2024-02-18T00:36:34.656108+0000","last_fullsized":"2024-02-18T00:36:34.656108+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:34:28.290868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.290868+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":217,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644546+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:36:34.644546+0000","last_peered":"2024-02-18T00:36:34.644546+0000","last_clean":"2024-02-18T00:36:34.644546+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:36:34.644546+0000","last_undegraded":"2024-02-18T00:36:34.644546+0000","last_fullsized":"2024-02-18T00:36:34.644546+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":300,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654251+0000","last_change":"2024-02-18T00:34:35.765124+0000","last_active":"2024-02-18T00:36:34.654251+0000","last_peered":"2024-02-18T00:36:34.654251+0000","last_clean":"2024-02-18T00:36:34.654251+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:36:34.654251+0000","last_undegraded":"2024-02-18T00:36:34.654251+0000","last_fullsized":"2024-02-18T00:36:34.654251+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.765059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.765059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":282,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653988+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:36:34.653988+0000","last_peered":"2024-02-18T00:36:34.653988+0000","last_clean":"2024-02-18T00:36:34.653988+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:36:34.653988+0000","last_undegraded":"2024-02-18T00:36:34.653988+0000","last_fullsized":"2024-02-18T00:36:34.653988+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":276,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659909+0000","last_change":"2024-02-18T00:34:17.016556+0000","last_active":"2024-02-18T00:36:34.659909+0000","last_peered":"2024-02-18T00:36:34.659909+0000","last_clean":"2024-02-18T00:36:34.659909+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:36:34.659909+0000","last_undegraded":"2024-02-18T00:36:34.659909+0000","last_fullsized":"2024-02-18T00:36:34.659909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:17.016476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:34:17.016476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":223,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643906+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:36:34.643906+0000","last_peered":"2024-02-18T00:36:34.643906+0000","last_clean":"2024-02-18T00:36:34.643906+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:36:34.643906+0000","last_undegraded":"2024-02-18T00:36:34.643906+0000","last_fullsized":"2024-02-18T00:36:34.643906+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":289,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646458+0000","last_change":"2024-02-18T00:33:49.897870+0000","last_active":"2024-02-18T00:36:34.646458+0000","last_peered":"2024-02-18T00:36:34.646458+0000","last_clean":"2024-02-18T00:36:34.646458+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:36:34.646458+0000","last_undegraded":"2024-02-18T00:36:34.646458+0000","last_fullsized":"2024-02-18T00:36:34.646458+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:49.897812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:49.897812+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":324,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.806748+0000","last_change":"2024-02-18T00:34:25.385634+0000","last_active":"2024-02-18T00:36:34.806748+0000","last_peered":"2024-02-18T00:36:34.806748+0000","last_clean":"2024-02-18T00:36:34.806748+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:36:34.806748+0000","last_undegraded":"2024-02-18T00:36:34.806748+0000","last_fullsized":"2024-02-18T00:36:34.806748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:34:25.385564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.385564+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":293,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643066+0000","last_change":"2024-02-18T00:33:37.809035+0000","last_active":"2024-02-18T00:36:34.643066+0000","last_peered":"2024-02-18T00:36:34.643066+0000","last_clean":"2024-02-18T00:36:34.643066+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:36:34.643066+0000","last_undegraded":"2024-02-18T00:36:34.643066+0000","last_fullsized":"2024-02-18T00:36:34.643066+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:37.808923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:33:37.808923+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":724,"reported_epoch":317,"state":"active+clean","last_fresh":"2024-02-18T00:36:35.651591+0000","last_change":"2024-02-18T00:36:06.641279+0000","last_active":"2024-02-18T00:36:35.651591+0000","last_peered":"2024-02-18T00:36:35.651591+0000","last_clean":"2024-02-18T00:36:35.651591+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:36:35.651591+0000","last_undegraded":"2024-02-18T00:36:35.651591+0000","last_fullsized":"2024-02-18T00:36:35.651591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:36:06.641185+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.641185+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":432,"num_read_kb":427,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":221,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656222+0000","last_change":"2024-02-18T00:34:29.252896+0000","last_active":"2024-02-18T00:36:34.656222+0000","last_peered":"2024-02-18T00:36:34.656222+0000","last_clean":"2024-02-18T00:36:34.656222+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:36:34.656222+0000","last_undegraded":"2024-02-18T00:36:34.656222+0000","last_fullsized":"2024-02-18T00:36:34.656222+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:34:29.252830+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.252830+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":233,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645325+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:36:34.645325+0000","last_peered":"2024-02-18T00:36:34.645325+0000","last_clean":"2024-02-18T00:36:34.645325+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:36:34.645325+0000","last_undegraded":"2024-02-18T00:36:34.645325+0000","last_fullsized":"2024-02-18T00:36:34.645325+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643332+0000","last_change":"2024-02-18T00:34:19.925200+0000","last_active":"2024-02-18T00:36:34.643332+0000","last_peered":"2024-02-18T00:36:34.643332+0000","last_clean":"2024-02-18T00:36:34.643332+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:36:34.643332+0000","last_undegraded":"2024-02-18T00:36:34.643332+0000","last_fullsized":"2024-02-18T00:36:34.643332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:19.925111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.925111+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":96,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653213+0000","last_change":"2024-02-18T00:35:44.479802+0000","last_active":"2024-02-18T00:36:34.653213+0000","last_peered":"2024-02-18T00:36:34.653213+0000","last_clean":"2024-02-18T00:36:34.653213+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:36:34.653213+0000","last_undegraded":"2024-02-18T00:36:34.653213+0000","last_fullsized":"2024-02-18T00:36:34.653213+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'34","last_scrub_stamp":"2024-02-18T00:33:54.197976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.197976+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":291,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654627+0000","last_change":"2024-02-18T00:34:15.045746+0000","last_active":"2024-02-18T00:36:34.654627+0000","last_peered":"2024-02-18T00:36:34.654627+0000","last_clean":"2024-02-18T00:36:34.654627+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:36:34.654627+0000","last_undegraded":"2024-02-18T00:36:34.654627+0000","last_fullsized":"2024-02-18T00:36:34.654627+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.045677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.045677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":281,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653739+0000","last_change":"2024-02-18T00:33:48.688410+0000","last_active":"2024-02-18T00:36:34.653739+0000","last_peered":"2024-02-18T00:36:34.653739+0000","last_clean":"2024-02-18T00:36:34.653739+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:36:34.653739+0000","last_undegraded":"2024-02-18T00:36:34.653739+0000","last_fullsized":"2024-02-18T00:36:34.653739+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:43.926301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:43.926301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":223,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.649096+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:36:34.649096+0000","last_peered":"2024-02-18T00:36:34.649096+0000","last_clean":"2024-02-18T00:36:34.649096+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:36:34.649096+0000","last_undegraded":"2024-02-18T00:36:34.649096+0000","last_fullsized":"2024-02-18T00:36:34.649096+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":374,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.649025+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:36:34.649025+0000","last_peered":"2024-02-18T00:36:34.649025+0000","last_clean":"2024-02-18T00:36:34.649025+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:36:34.649025+0000","last_undegraded":"2024-02-18T00:36:34.649025+0000","last_fullsized":"2024-02-18T00:36:34.649025+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":282,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643897+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:36:34.643897+0000","last_peered":"2024-02-18T00:36:34.643897+0000","last_clean":"2024-02-18T00:36:34.643897+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:36:34.643897+0000","last_undegraded":"2024-02-18T00:36:34.643897+0000","last_fullsized":"2024-02-18T00:36:34.643897+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":307,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.804389+0000","last_change":"2024-02-18T00:34:31.410145+0000","last_active":"2024-02-18T00:36:34.804389+0000","last_peered":"2024-02-18T00:36:34.804389+0000","last_clean":"2024-02-18T00:36:34.804389+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:36:34.804389+0000","last_undegraded":"2024-02-18T00:36:34.804389+0000","last_fullsized":"2024-02-18T00:36:34.804389+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.410081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.410081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":271,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644313+0000","last_change":"2024-02-18T00:33:48.010937+0000","last_active":"2024-02-18T00:36:34.644313+0000","last_peered":"2024-02-18T00:36:34.644313+0000","last_clean":"2024-02-18T00:36:34.644313+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:36:34.644313+0000","last_undegraded":"2024-02-18T00:36:34.644313+0000","last_fullsized":"2024-02-18T00:36:34.644313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:48.010821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:48.010821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645565+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:36:34.645565+0000","last_peered":"2024-02-18T00:36:34.645565+0000","last_clean":"2024-02-18T00:36:34.645565+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:36:34.645565+0000","last_undegraded":"2024-02-18T00:36:34.645565+0000","last_fullsized":"2024-02-18T00:36:34.645565+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645383+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:36:34.645383+0000","last_peered":"2024-02-18T00:36:34.645383+0000","last_clean":"2024-02-18T00:36:34.645383+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:36:34.645383+0000","last_undegraded":"2024-02-18T00:36:34.645383+0000","last_fullsized":"2024-02-18T00:36:34.645383+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":335,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645158+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:36:34.645158+0000","last_peered":"2024-02-18T00:36:34.645158+0000","last_clean":"2024-02-18T00:36:34.645158+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:36:34.645158+0000","last_undegraded":"2024-02-18T00:36:34.645158+0000","last_fullsized":"2024-02-18T00:36:34.645158+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":317,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646205+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:36:34.646205+0000","last_peered":"2024-02-18T00:36:34.646205+0000","last_clean":"2024-02-18T00:36:34.646205+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:36:34.646205+0000","last_undegraded":"2024-02-18T00:36:34.646205+0000","last_fullsized":"2024-02-18T00:36:34.646205+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":286,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659142+0000","last_change":"2024-02-18T00:34:26.701231+0000","last_active":"2024-02-18T00:36:34.659142+0000","last_peered":"2024-02-18T00:36:34.659142+0000","last_clean":"2024-02-18T00:36:34.659142+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:36:34.659142+0000","last_undegraded":"2024-02-18T00:36:34.659142+0000","last_fullsized":"2024-02-18T00:36:34.659142+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.701162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.701162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644830+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:36:34.644830+0000","last_peered":"2024-02-18T00:36:34.644830+0000","last_clean":"2024-02-18T00:36:34.644830+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:36:34.644830+0000","last_undegraded":"2024-02-18T00:36:34.644830+0000","last_fullsized":"2024-02-18T00:36:34.644830+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":349,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646014+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:36:34.646014+0000","last_peered":"2024-02-18T00:36:34.646014+0000","last_clean":"2024-02-18T00:36:34.646014+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:36:34.646014+0000","last_undegraded":"2024-02-18T00:36:34.646014+0000","last_fullsized":"2024-02-18T00:36:34.646014+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":284,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645324+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:36:34.645324+0000","last_peered":"2024-02-18T00:36:34.645324+0000","last_clean":"2024-02-18T00:36:34.645324+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:36:34.645324+0000","last_undegraded":"2024-02-18T00:36:34.645324+0000","last_fullsized":"2024-02-18T00:36:34.645324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":288,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656726+0000","last_change":"2024-02-18T00:34:25.683506+0000","last_active":"2024-02-18T00:36:34.656726+0000","last_peered":"2024-02-18T00:36:34.656726+0000","last_clean":"2024-02-18T00:36:34.656726+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:36:34.656726+0000","last_undegraded":"2024-02-18T00:36:34.656726+0000","last_fullsized":"2024-02-18T00:36:34.656726+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.683442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.683442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":287,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644400+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:36:34.644400+0000","last_peered":"2024-02-18T00:36:34.644400+0000","last_clean":"2024-02-18T00:36:34.644400+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:36:34.644400+0000","last_undegraded":"2024-02-18T00:36:34.644400+0000","last_fullsized":"2024-02-18T00:36:34.644400+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":223,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643738+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:36:34.643738+0000","last_peered":"2024-02-18T00:36:34.643738+0000","last_clean":"2024-02-18T00:36:34.643738+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:36:34.643738+0000","last_undegraded":"2024-02-18T00:36:34.643738+0000","last_fullsized":"2024-02-18T00:36:34.643738+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":343,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656642+0000","last_change":"2024-02-18T00:33:48.861148+0000","last_active":"2024-02-18T00:36:34.656642+0000","last_peered":"2024-02-18T00:36:34.656642+0000","last_clean":"2024-02-18T00:36:34.656642+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:36:34.656642+0000","last_undegraded":"2024-02-18T00:36:34.656642+0000","last_fullsized":"2024-02-18T00:36:34.656642+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:33:46.897445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.897445+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":289,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657816+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:36:34.657816+0000","last_peered":"2024-02-18T00:36:34.657816+0000","last_clean":"2024-02-18T00:36:34.657816+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:36:34.657816+0000","last_undegraded":"2024-02-18T00:36:34.657816+0000","last_fullsized":"2024-02-18T00:36:34.657816+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.655276+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:36:34.655276+0000","last_peered":"2024-02-18T00:36:34.655276+0000","last_clean":"2024-02-18T00:36:34.655276+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:36:34.655276+0000","last_undegraded":"2024-02-18T00:36:34.655276+0000","last_fullsized":"2024-02-18T00:36:34.655276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":189,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656920+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:36:34.656920+0000","last_peered":"2024-02-18T00:36:34.656920+0000","last_clean":"2024-02-18T00:36:34.656920+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:36:34.656920+0000","last_undegraded":"2024-02-18T00:36:34.656920+0000","last_fullsized":"2024-02-18T00:36:34.656920+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":233,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643785+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:36:34.643785+0000","last_peered":"2024-02-18T00:36:34.643785+0000","last_clean":"2024-02-18T00:36:34.643785+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:36:34.643785+0000","last_undegraded":"2024-02-18T00:36:34.643785+0000","last_fullsized":"2024-02-18T00:36:34.643785+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":351,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657107+0000","last_change":"2024-02-18T00:34:30.666844+0000","last_active":"2024-02-18T00:36:34.657107+0000","last_peered":"2024-02-18T00:36:34.657107+0000","last_clean":"2024-02-18T00:36:34.657107+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:36:34.657107+0000","last_undegraded":"2024-02-18T00:36:34.657107+0000","last_fullsized":"2024-02-18T00:36:34.657107+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:34:30.666765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.666765+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":300,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654510+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:36:34.654510+0000","last_peered":"2024-02-18T00:36:34.654510+0000","last_clean":"2024-02-18T00:36:34.654510+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:36:34.654510+0000","last_undegraded":"2024-02-18T00:36:34.654510+0000","last_fullsized":"2024-02-18T00:36:34.654510+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":374,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657032+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:36:34.657032+0000","last_peered":"2024-02-18T00:36:34.657032+0000","last_clean":"2024-02-18T00:36:34.657032+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:36:34.657032+0000","last_undegraded":"2024-02-18T00:36:34.657032+0000","last_fullsized":"2024-02-18T00:36:34.657032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":285,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645842+0000","last_change":"2024-02-18T00:33:53.101420+0000","last_active":"2024-02-18T00:36:34.645842+0000","last_peered":"2024-02-18T00:36:34.645842+0000","last_clean":"2024-02-18T00:36:34.645842+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:36:34.645842+0000","last_undegraded":"2024-02-18T00:36:34.645842+0000","last_fullsized":"2024-02-18T00:36:34.645842+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:53.101251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:53.101251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":233,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.650903+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:36:34.650903+0000","last_peered":"2024-02-18T00:36:34.650903+0000","last_clean":"2024-02-18T00:36:34.650903+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:36:34.650903+0000","last_undegraded":"2024-02-18T00:36:34.650903+0000","last_fullsized":"2024-02-18T00:36:34.650903+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":332,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656890+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:36:34.656890+0000","last_peered":"2024-02-18T00:36:34.656890+0000","last_clean":"2024-02-18T00:36:34.656890+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:36:34.656890+0000","last_undegraded":"2024-02-18T00:36:34.656890+0000","last_fullsized":"2024-02-18T00:36:34.656890+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":282,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.648820+0000","last_change":"2024-02-18T00:34:15.935045+0000","last_active":"2024-02-18T00:36:34.648820+0000","last_peered":"2024-02-18T00:36:34.648820+0000","last_clean":"2024-02-18T00:36:34.648820+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:36:34.648820+0000","last_undegraded":"2024-02-18T00:36:34.648820+0000","last_fullsized":"2024-02-18T00:36:34.648820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.934970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"150'9","reported_seq":388,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657317+0000","last_change":"2024-02-18T00:34:27.741951+0000","last_active":"2024-02-18T00:36:34.657317+0000","last_peered":"2024-02-18T00:36:34.657317+0000","last_clean":"2024-02-18T00:36:34.657317+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:36:34.657317+0000","last_undegraded":"2024-02-18T00:36:34.657317+0000","last_fullsized":"2024-02-18T00:36:34.657317+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:27.741881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.741881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.647161+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:36:34.647161+0000","last_peered":"2024-02-18T00:36:34.647161+0000","last_clean":"2024-02-18T00:36:34.647161+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:36:34.647161+0000","last_undegraded":"2024-02-18T00:36:34.647161+0000","last_fullsized":"2024-02-18T00:36:34.647161+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":222,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659985+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:36:34.659985+0000","last_peered":"2024-02-18T00:36:34.659985+0000","last_clean":"2024-02-18T00:36:34.659985+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:36:34.659985+0000","last_undegraded":"2024-02-18T00:36:34.659985+0000","last_fullsized":"2024-02-18T00:36:34.659985+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":298,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801707+0000","last_change":"2024-02-18T00:34:32.418786+0000","last_active":"2024-02-18T00:36:34.801707+0000","last_peered":"2024-02-18T00:36:34.801707+0000","last_clean":"2024-02-18T00:36:34.801707+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:36:34.801707+0000","last_undegraded":"2024-02-18T00:36:34.801707+0000","last_fullsized":"2024-02-18T00:36:34.801707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:34:32.418717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.418717+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":289,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.652345+0000","last_change":"2024-02-18T00:34:20.312165+0000","last_active":"2024-02-18T00:36:34.652345+0000","last_peered":"2024-02-18T00:36:34.652345+0000","last_clean":"2024-02-18T00:36:34.652345+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:36:34.652345+0000","last_undegraded":"2024-02-18T00:36:34.652345+0000","last_fullsized":"2024-02-18T00:36:34.652345+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.312096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.312096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":279,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659040+0000","last_change":"2024-02-18T00:33:50.765435+0000","last_active":"2024-02-18T00:36:34.659040+0000","last_peered":"2024-02-18T00:36:34.659040+0000","last_clean":"2024-02-18T00:36:34.659040+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:36:34.659040+0000","last_undegraded":"2024-02-18T00:36:34.659040+0000","last_fullsized":"2024-02-18T00:36:34.659040+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:50.765368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:50.765368+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":276,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644228+0000","last_change":"2024-02-18T00:36:11.346267+0000","last_active":"2024-02-18T00:36:34.644228+0000","last_peered":"2024-02-18T00:36:34.644228+0000","last_clean":"2024-02-18T00:36:34.644228+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:36:34.644228+0000","last_undegraded":"2024-02-18T00:36:34.644228+0000","last_fullsized":"2024-02-18T00:36:34.644228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.346197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.346197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":218,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645758+0000","last_change":"2024-02-18T00:33:04.661856+0000","last_active":"2024-02-18T00:36:34.645758+0000","last_peered":"2024-02-18T00:36:34.645758+0000","last_clean":"2024-02-18T00:36:34.645758+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:36:34.645758+0000","last_undegraded":"2024-02-18T00:36:34.645758+0000","last_fullsized":"2024-02-18T00:36:34.645758+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:33:01.316512+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":301,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645655+0000","last_change":"2024-02-18T00:34:20.362851+0000","last_active":"2024-02-18T00:36:34.645655+0000","last_peered":"2024-02-18T00:36:34.645655+0000","last_clean":"2024-02-18T00:36:34.645655+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:36:34.645655+0000","last_undegraded":"2024-02-18T00:36:34.645655+0000","last_fullsized":"2024-02-18T00:36:34.645655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:34:20.362782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.362782+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":284,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.799989+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:36:34.799989+0000","last_peered":"2024-02-18T00:36:34.799989+0000","last_clean":"2024-02-18T00:36:34.799989+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:36:34.799989+0000","last_undegraded":"2024-02-18T00:36:34.799989+0000","last_fullsized":"2024-02-18T00:36:34.799989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801973+0000","last_change":"2024-02-18T00:34:34.355483+0000","last_active":"2024-02-18T00:36:34.801973+0000","last_peered":"2024-02-18T00:36:34.801973+0000","last_clean":"2024-02-18T00:36:34.801973+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:36:34.801973+0000","last_undegraded":"2024-02-18T00:36:34.801973+0000","last_fullsized":"2024-02-18T00:36:34.801973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.355416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.355416+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":277,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.652900+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:36:34.652900+0000","last_peered":"2024-02-18T00:36:34.652900+0000","last_clean":"2024-02-18T00:36:34.652900+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:36:34.652900+0000","last_undegraded":"2024-02-18T00:36:34.652900+0000","last_fullsized":"2024-02-18T00:36:34.652900+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656059+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:36:34.656059+0000","last_peered":"2024-02-18T00:36:34.656059+0000","last_clean":"2024-02-18T00:36:34.656059+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:36:34.656059+0000","last_undegraded":"2024-02-18T00:36:34.656059+0000","last_fullsized":"2024-02-18T00:36:34.656059+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":287,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653091+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:36:34.653091+0000","last_peered":"2024-02-18T00:36:34.653091+0000","last_clean":"2024-02-18T00:36:34.653091+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:36:34.653091+0000","last_undegraded":"2024-02-18T00:36:34.653091+0000","last_fullsized":"2024-02-18T00:36:34.653091+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":287,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.655287+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:36:34.655287+0000","last_peered":"2024-02-18T00:36:34.655287+0000","last_clean":"2024-02-18T00:36:34.655287+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:36:34.655287+0000","last_undegraded":"2024-02-18T00:36:34.655287+0000","last_fullsized":"2024-02-18T00:36:34.655287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":294,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657719+0000","last_change":"2024-02-18T00:36:12.518914+0000","last_active":"2024-02-18T00:36:34.657719+0000","last_peered":"2024-02-18T00:36:34.657719+0000","last_clean":"2024-02-18T00:36:34.657719+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:36:34.657719+0000","last_undegraded":"2024-02-18T00:36:34.657719+0000","last_fullsized":"2024-02-18T00:36:34.657719+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:12.518838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:12.518838+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":184,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643814+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:36:34.643814+0000","last_peered":"2024-02-18T00:36:34.643814+0000","last_clean":"2024-02-18T00:36:34.643814+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:36:34.643814+0000","last_undegraded":"2024-02-18T00:36:34.643814+0000","last_fullsized":"2024-02-18T00:36:34.643814+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":231,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656619+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:36:34.656619+0000","last_peered":"2024-02-18T00:36:34.656619+0000","last_clean":"2024-02-18T00:36:34.656619+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:36:34.656619+0000","last_undegraded":"2024-02-18T00:36:34.656619+0000","last_fullsized":"2024-02-18T00:36:34.656619+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":338,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653523+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:36:34.653523+0000","last_peered":"2024-02-18T00:36:34.653523+0000","last_clean":"2024-02-18T00:36:34.653523+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:36:34.653523+0000","last_undegraded":"2024-02-18T00:36:34.653523+0000","last_fullsized":"2024-02-18T00:36:34.653523+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":289,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.656879+0000","last_change":"2024-02-18T00:36:11.551695+0000","last_active":"2024-02-18T00:36:34.656879+0000","last_peered":"2024-02-18T00:36:34.656879+0000","last_clean":"2024-02-18T00:36:34.656879+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:36:34.656879+0000","last_undegraded":"2024-02-18T00:36:34.656879+0000","last_fullsized":"2024-02-18T00:36:34.656879+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.551602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":193,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644291+0000","last_change":"2024-02-18T00:35:42.661117+0000","last_active":"2024-02-18T00:36:34.644291+0000","last_peered":"2024-02-18T00:36:34.644291+0000","last_clean":"2024-02-18T00:36:34.644291+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:36:34.644291+0000","last_undegraded":"2024-02-18T00:36:34.644291+0000","last_fullsized":"2024-02-18T00:36:34.644291+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:23.374328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.374328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":277,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.659225+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:36:34.659225+0000","last_peered":"2024-02-18T00:36:34.659225+0000","last_clean":"2024-02-18T00:36:34.659225+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:36:34.659225+0000","last_undegraded":"2024-02-18T00:36:34.659225+0000","last_fullsized":"2024-02-18T00:36:34.659225+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":184,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644687+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:36:34.644687+0000","last_peered":"2024-02-18T00:36:34.644687+0000","last_clean":"2024-02-18T00:36:34.644687+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:36:34.644687+0000","last_undegraded":"2024-02-18T00:36:34.644687+0000","last_fullsized":"2024-02-18T00:36:34.644687+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":339,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646229+0000","last_change":"2024-02-18T00:33:56.121942+0000","last_active":"2024-02-18T00:36:34.646229+0000","last_peered":"2024-02-18T00:36:34.646229+0000","last_clean":"2024-02-18T00:36:34.646229+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:36:34.646229+0000","last_undegraded":"2024-02-18T00:36:34.646229+0000","last_fullsized":"2024-02-18T00:36:34.646229+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:33:56.121872+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":291,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.652056+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:36:34.652056+0000","last_peered":"2024-02-18T00:36:34.652056+0000","last_clean":"2024-02-18T00:36:34.652056+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:36:34.652056+0000","last_undegraded":"2024-02-18T00:36:34.652056+0000","last_fullsized":"2024-02-18T00:36:34.652056+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":397,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657054+0000","last_change":"2024-02-18T00:34:18.318312+0000","last_active":"2024-02-18T00:36:34.657054+0000","last_peered":"2024-02-18T00:36:34.657054+0000","last_clean":"2024-02-18T00:36:34.657054+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:36:34.657054+0000","last_undegraded":"2024-02-18T00:36:34.657054+0000","last_fullsized":"2024-02-18T00:36:34.657054+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:18.318190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.318190+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":195,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.658857+0000","last_change":"2024-02-18T00:34:20.995201+0000","last_active":"2024-02-18T00:36:34.658857+0000","last_peered":"2024-02-18T00:36:34.658857+0000","last_clean":"2024-02-18T00:36:34.658857+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:36:34.658857+0000","last_undegraded":"2024-02-18T00:36:34.658857+0000","last_fullsized":"2024-02-18T00:36:34.658857+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:34:20.995125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.995125+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":183,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653261+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:36:34.653261+0000","last_peered":"2024-02-18T00:36:34.653261+0000","last_clean":"2024-02-18T00:36:34.653261+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:36:34.653261+0000","last_undegraded":"2024-02-18T00:36:34.653261+0000","last_fullsized":"2024-02-18T00:36:34.653261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":321,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657146+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:36:34.657146+0000","last_peered":"2024-02-18T00:36:34.657146+0000","last_clean":"2024-02-18T00:36:34.657146+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:36:34.657146+0000","last_undegraded":"2024-02-18T00:36:34.657146+0000","last_fullsized":"2024-02-18T00:36:34.657146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":283,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653180+0000","last_change":"2024-02-18T00:34:28.731011+0000","last_active":"2024-02-18T00:36:34.653180+0000","last_peered":"2024-02-18T00:36:34.653180+0000","last_clean":"2024-02-18T00:36:34.653180+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:36:34.653180+0000","last_undegraded":"2024-02-18T00:36:34.653180+0000","last_fullsized":"2024-02-18T00:36:34.653180+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.730955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.730955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":282,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.646313+0000","last_change":"2024-02-18T00:33:48.751672+0000","last_active":"2024-02-18T00:36:34.646313+0000","last_peered":"2024-02-18T00:36:34.646313+0000","last_clean":"2024-02-18T00:36:34.646313+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:36:34.646313+0000","last_undegraded":"2024-02-18T00:36:34.646313+0000","last_fullsized":"2024-02-18T00:36:34.646313+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:46.025296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:33:46.025296+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":283,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.807385+0000","last_change":"2024-02-18T00:35:54.569108+0000","last_active":"2024-02-18T00:36:34.807385+0000","last_peered":"2024-02-18T00:36:34.807385+0000","last_clean":"2024-02-18T00:36:34.807385+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:36:34.807385+0000","last_undegraded":"2024-02-18T00:36:34.807385+0000","last_fullsized":"2024-02-18T00:36:34.807385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.569033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.569033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":237,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.643696+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:36:34.643696+0000","last_peered":"2024-02-18T00:36:34.643696+0000","last_clean":"2024-02-18T00:36:34.643696+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:36:34.643696+0000","last_undegraded":"2024-02-18T00:36:34.643696+0000","last_fullsized":"2024-02-18T00:36:34.643696+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":312,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644595+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:36:34.644595+0000","last_peered":"2024-02-18T00:36:34.644595+0000","last_clean":"2024-02-18T00:36:34.644595+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:36:34.644595+0000","last_undegraded":"2024-02-18T00:36:34.644595+0000","last_fullsized":"2024-02-18T00:36:34.644595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":298,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.651633+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:36:34.651633+0000","last_peered":"2024-02-18T00:36:34.651633+0000","last_clean":"2024-02-18T00:36:34.651633+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:36:34.651633+0000","last_undegraded":"2024-02-18T00:36:34.651633+0000","last_fullsized":"2024-02-18T00:36:34.651633+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.809256+0000","last_change":"2024-02-18T00:34:18.357435+0000","last_active":"2024-02-18T00:36:34.809256+0000","last_peered":"2024-02-18T00:36:34.809256+0000","last_clean":"2024-02-18T00:36:34.809256+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:36:34.809256+0000","last_undegraded":"2024-02-18T00:36:34.809256+0000","last_fullsized":"2024-02-18T00:36:34.809256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:18.357369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.357369+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.647245+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:36:34.647245+0000","last_peered":"2024-02-18T00:36:34.647245+0000","last_clean":"2024-02-18T00:36:34.647245+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:36:34.647245+0000","last_undegraded":"2024-02-18T00:36:34.647245+0000","last_fullsized":"2024-02-18T00:36:34.647245+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.651377+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:36:34.651377+0000","last_peered":"2024-02-18T00:36:34.651377+0000","last_clean":"2024-02-18T00:36:34.651377+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:36:34.651377+0000","last_undegraded":"2024-02-18T00:36:34.651377+0000","last_fullsized":"2024-02-18T00:36:34.651377+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":319,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.655080+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:36:34.655080+0000","last_peered":"2024-02-18T00:36:34.655080+0000","last_clean":"2024-02-18T00:36:34.655080+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:36:34.655080+0000","last_undegraded":"2024-02-18T00:36:34.655080+0000","last_fullsized":"2024-02-18T00:36:34.655080+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":303,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.658964+0000","last_change":"2024-02-18T00:33:48.748311+0000","last_active":"2024-02-18T00:36:34.658964+0000","last_peered":"2024-02-18T00:36:34.658964+0000","last_clean":"2024-02-18T00:36:34.658964+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:36:34.658964+0000","last_undegraded":"2024-02-18T00:36:34.658964+0000","last_fullsized":"2024-02-18T00:36:34.658964+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:33:45.906516+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:33:45.906516+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645721+0000","last_change":"2024-02-18T00:34:29.236028+0000","last_active":"2024-02-18T00:36:34.645721+0000","last_peered":"2024-02-18T00:36:34.645721+0000","last_clean":"2024-02-18T00:36:34.645721+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:36:34.645721+0000","last_undegraded":"2024-02-18T00:36:34.645721+0000","last_fullsized":"2024-02-18T00:36:34.645721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:29.235953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.235953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":277,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:35.055276+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:36:35.055276+0000","last_peered":"2024-02-18T00:36:35.055276+0000","last_clean":"2024-02-18T00:36:35.055276+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:36:35.055276+0000","last_undegraded":"2024-02-18T00:36:35.055276+0000","last_fullsized":"2024-02-18T00:36:35.055276+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":232,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.649180+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:36:34.649180+0000","last_peered":"2024-02-18T00:36:34.649180+0000","last_clean":"2024-02-18T00:36:34.649180+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:36:34.649180+0000","last_undegraded":"2024-02-18T00:36:34.649180+0000","last_fullsized":"2024-02-18T00:36:34.649180+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":340,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645281+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:36:34.645281+0000","last_peered":"2024-02-18T00:36:34.645281+0000","last_clean":"2024-02-18T00:36:34.645281+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:36:34.645281+0000","last_undegraded":"2024-02-18T00:36:34.645281+0000","last_fullsized":"2024-02-18T00:36:34.645281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":182,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654423+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:36:34.654423+0000","last_peered":"2024-02-18T00:36:34.654423+0000","last_clean":"2024-02-18T00:36:34.654423+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:36:34.654423+0000","last_undegraded":"2024-02-18T00:36:34.654423+0000","last_fullsized":"2024-02-18T00:36:34.654423+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":296,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654231+0000","last_change":"2024-02-18T00:35:42.660706+0000","last_active":"2024-02-18T00:36:34.654231+0000","last_peered":"2024-02-18T00:36:34.654231+0000","last_clean":"2024-02-18T00:36:34.654231+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:36:34.654231+0000","last_undegraded":"2024-02-18T00:36:34.654231+0000","last_fullsized":"2024-02-18T00:36:34.654231+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:25.961303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:25.961303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":285,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.649364+0000","last_change":"2024-02-18T00:33:48.649527+0000","last_active":"2024-02-18T00:36:34.649364+0000","last_peered":"2024-02-18T00:36:34.649364+0000","last_clean":"2024-02-18T00:36:34.649364+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:36:34.649364+0000","last_undegraded":"2024-02-18T00:36:34.649364+0000","last_fullsized":"2024-02-18T00:36:34.649364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:41.996913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.996913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":224,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645214+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:36:34.645214+0000","last_peered":"2024-02-18T00:36:34.645214+0000","last_clean":"2024-02-18T00:36:34.645214+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:36:34.645214+0000","last_undegraded":"2024-02-18T00:36:34.645214+0000","last_fullsized":"2024-02-18T00:36:34.645214+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":310,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645159+0000","last_change":"2024-02-18T00:34:19.325220+0000","last_active":"2024-02-18T00:36:34.645159+0000","last_peered":"2024-02-18T00:36:34.645159+0000","last_clean":"2024-02-18T00:36:34.645159+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:36:34.645159+0000","last_undegraded":"2024-02-18T00:36:34.645159+0000","last_fullsized":"2024-02-18T00:36:34.645159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:34:19.325130+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:34:19.325130+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":184,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.655971+0000","last_change":"2024-02-18T00:34:31.241231+0000","last_active":"2024-02-18T00:36:34.655971+0000","last_peered":"2024-02-18T00:36:34.655971+0000","last_clean":"2024-02-18T00:36:34.655971+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:36:34.655971+0000","last_undegraded":"2024-02-18T00:36:34.655971+0000","last_fullsized":"2024-02-18T00:36:34.655971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:34:31.241146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":179,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645611+0000","last_change":"2024-02-18T00:33:48.733307+0000","last_active":"2024-02-18T00:36:34.645611+0000","last_peered":"2024-02-18T00:36:34.645611+0000","last_clean":"2024-02-18T00:36:34.645611+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:36:34.645611+0000","last_undegraded":"2024-02-18T00:36:34.645611+0000","last_fullsized":"2024-02-18T00:36:34.645611+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:39.801504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:33:39.801504+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":276,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653129+0000","last_change":"2024-02-18T00:33:54.013575+0000","last_active":"2024-02-18T00:36:34.653129+0000","last_peered":"2024-02-18T00:36:34.653129+0000","last_clean":"2024-02-18T00:36:34.653129+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:36:34.653129+0000","last_undegraded":"2024-02-18T00:36:34.653129+0000","last_fullsized":"2024-02-18T00:36:34.653129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:33:54.013467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:33:54.013467+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":232,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.655349+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:36:34.655349+0000","last_peered":"2024-02-18T00:36:34.655349+0000","last_clean":"2024-02-18T00:36:34.655349+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:36:34.655349+0000","last_undegraded":"2024-02-18T00:36:34.655349+0000","last_fullsized":"2024-02-18T00:36:34.655349+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":318,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644171+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:36:34.644171+0000","last_peered":"2024-02-18T00:36:34.644171+0000","last_clean":"2024-02-18T00:36:34.644171+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:36:34.644171+0000","last_undegraded":"2024-02-18T00:36:34.644171+0000","last_fullsized":"2024-02-18T00:36:34.644171+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":283,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.654439+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:36:34.654439+0000","last_peered":"2024-02-18T00:36:34.654439+0000","last_clean":"2024-02-18T00:36:34.654439+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:36:34.654439+0000","last_undegraded":"2024-02-18T00:36:34.654439+0000","last_fullsized":"2024-02-18T00:36:34.654439+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":289,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.801550+0000","last_change":"2024-02-18T00:34:38.428939+0000","last_active":"2024-02-18T00:36:34.801550+0000","last_peered":"2024-02-18T00:36:34.801550+0000","last_clean":"2024-02-18T00:36:34.801550+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:36:34.801550+0000","last_undegraded":"2024-02-18T00:36:34.801550+0000","last_fullsized":"2024-02-18T00:36:34.801550+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.428855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.428855+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":278,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.645457+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:36:34.645457+0000","last_peered":"2024-02-18T00:36:34.645457+0000","last_clean":"2024-02-18T00:36:34.645457+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:36:34.645457+0000","last_undegraded":"2024-02-18T00:36:34.645457+0000","last_fullsized":"2024-02-18T00:36:34.645457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":223,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.657553+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:36:34.657553+0000","last_peered":"2024-02-18T00:36:34.657553+0000","last_clean":"2024-02-18T00:36:34.657553+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:36:34.657553+0000","last_undegraded":"2024-02-18T00:36:34.657553+0000","last_fullsized":"2024-02-18T00:36:34.657553+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":363,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.653088+0000","last_change":"2024-02-18T00:33:41.866669+0000","last_active":"2024-02-18T00:36:34.653088+0000","last_peered":"2024-02-18T00:36:34.653088+0000","last_clean":"2024-02-18T00:36:34.653088+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:36:34.653088+0000","last_undegraded":"2024-02-18T00:36:34.653088+0000","last_fullsized":"2024-02-18T00:36:34.653088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:33:41.866531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:33:41.866531+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":291,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.648882+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:36:34.648882+0000","last_peered":"2024-02-18T00:36:34.648882+0000","last_clean":"2024-02-18T00:36:34.648882+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:36:34.648882+0000","last_undegraded":"2024-02-18T00:36:34.648882+0000","last_fullsized":"2024-02-18T00:36:34.648882+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":280,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.644528+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:36:34.644528+0000","last_peered":"2024-02-18T00:36:34.644528+0000","last_clean":"2024-02-18T00:36:34.644528+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:36:34.644528+0000","last_undegraded":"2024-02-18T00:36:34.644528+0000","last_fullsized":"2024-02-18T00:36:34.644528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":286,"reported_epoch":316,"state":"active+clean","last_fresh":"2024-02-18T00:36:34.658628+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:36:34.658628+0000","last_peered":"2024-02-18T00:36:34.658628+0000","last_clean":"2024-02-18T00:36:34.658628+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:36:34.658628+0000","last_undegraded":"2024-02-18T00:36:34.658628+0000","last_fullsized":"2024-02-18T00:36:34.658628+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001220,"num_objects":123,"num_object_clones":2,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":548,"num_write_kb":4449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13512704,"data_stored":12003693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1798,"ondisk_log_size":1798,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":191,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":432,"num_read_kb":427,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266780,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463010,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306276,"kb_used_data":9040,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93438876,"statfs":{"total":95995035648,"available":95681409024,"internally_reserved":0,"allocated":9256960,"data_stored":6924800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":2.7280000000000002,"15min":1.234},"min":{"1min":0.253,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":2.7879999999999998,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":1.1619999999999999,"15min":0.71899999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.1130000000000004,"5min":10.433,"15min":10.433},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":2.9590000000000001,"15min":1.3169999999999999},"min":{"1min":0.28799999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":2.8330000000000002,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":2.9409999999999998,"15min":1.306},"min":{"1min":0.36899999999999999,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":3.6030000000000002,"5min":71.872,"15min":71.872},"last":0.438}]},{"osd":2,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":1.389,"15min":0.80000000000000004},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.316},"max":{"1min":3.657,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.839,"15min":0.95899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":4.1600000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.45500000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":1.383,"15min":0.79900000000000004},"min":{"1min":0.35699999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.5670000000000002,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.284,"15min":0.77200000000000002},"min":{"1min":0.438,"5min":0.309,"15min":0.309},"max":{"1min":3.694,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.67800000000000005,"15min":0.49099999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.4359999999999999,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.1890000000000001,"15min":0.66200000000000003},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":7.907,"5min":35.18,"15min":35.18},"last":0.42099999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:35:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71099999999999997,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":4.476,"5min":10.606,"15min":10.606},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.2310000000000001,"15min":0.68799999999999994},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6769999999999996,"5min":35.119,"15min":35.119},"last":0.34100000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.63200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4770000000000003,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":1.119,"15min":0.72299999999999998},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.8730000000000002,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302352,"kb_used_data":5116,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442800,"statfs":{"total":95995035648,"available":95685427200,"internally_reserved":0,"allocated":5238784,"data_stored":2915437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":1.2290000000000001,"15min":0.625},"min":{"1min":0.19900000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.47899999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":1.3140000000000001,"15min":0.65900000000000003},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46999999999999997,"5min":43.137,"15min":43.137},"last":0.30499999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":1.5289999999999999,"15min":0.75900000000000001},"min":{"1min":0.20699999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51900000000000002,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.95999999999999996,"15min":0.58599999999999997},"min":{"1min":0.28499999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":17.596,"15min":17.596},"last":0.38200000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:35:44 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":1.355,"15min":0.755},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51700000000000002,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.432,"5min":1.3580000000000001,"15min":0.77600000000000002},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Sun Feb 18 00:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.1339999999999999,"15min":0.78000000000000003},"min":{"1min":0.29999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1659999999999999,"15min":0.76600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88100000000000001,"5min":12.75,"15min":12.75},"last":0.53500000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:35:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":1.1439999999999999,"15min":0.77700000000000002},"min":{"1min":0.40300000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.8759999999999999,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":1.1499999999999999,"15min":0.79200000000000004},"min":{"1min":0.41999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":4.1180000000000003,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.45700000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.1779999999999999,"15min":0.81399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":12.834,"15min":12.834},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.149,"15min":0.80300000000000005},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81100000000000005,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":1.147,"15min":0.80500000000000005},"min":{"1min":0.41899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.109,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":1.1459999999999999,"15min":0.81499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5409999999999999,"5min":12.805,"15min":12.805},"last":0.503}]}]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":103,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314660,"kb_used_data":5144,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430492,"statfs":{"total":95995035648,"available":95672823808,"internally_reserved":0,"allocated":5267456,"data_stored":2923995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":2.2679999999999998,"15min":1.1359999999999999},"min":{"1min":0.25700000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.0820000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.312},{"interface":"front","average":{"1min":0.38500000000000001,"5min":1.5089999999999999,"15min":0.73999999999999999},"min":{"1min":0.221,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":1.179,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.245}]},{"osd":2,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":1.8240000000000001,"15min":0.872},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.2430000000000001,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.311,"5min":1.8280000000000001,"15min":0.87},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":0.40000000000000002,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.27900000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":1.1659999999999999,"15min":0.68100000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.55400000000000005,"5min":19.006,"15min":19.006},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.442,"5min":2.222,"15min":1.071},"min":{"1min":0.30299999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.2130000000000001,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":1.6499999999999999,"15min":0.91400000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.63900000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.627,"5min":1.6970000000000001,"15min":0.96799999999999997},"min":{"1min":0.42199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2969999999999999,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.42899999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.613,"15min":0.93100000000000005},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.262,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":1.8069999999999999,"15min":0.96599999999999997},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":57.991,"15min":57.991},"last":0.66000000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:35:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.2270000000000001,"15min":0.79300000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77600000000000002,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.712,"15min":0.95399999999999996},"min":{"1min":0.41799999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.28,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.63800000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.8680000000000001,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3129999999999999,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":1.798,"15min":0.99099999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.69599999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318840,"kb_used_data":5216,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93426312,"statfs":{"total":95995035648,"available":95668543488,"internally_reserved":0,"allocated":5341184,"data_stored":2924341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.71599999999999997,"15min":0.49099999999999999},"min":{"1min":0.22,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46100000000000002,"5min":12.144,"15min":12.144},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":1.0529999999999999,"15min":0.60699999999999998},"min":{"1min":0.221,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45800000000000002,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.28599999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.70899999999999996,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47099999999999997,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.82099999999999995,"15min":0.54500000000000004},"min":{"1min":0.29499999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48499999999999999,"5min":18.919,"15min":18.919},"last":0.64300000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:35:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.92400000000000004,"15min":0.61599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71799999999999997,"5min":22.712,"15min":22.712},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.92300000000000004,"15min":0.627},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.629,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.32600000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.88400000000000001,"15min":0.68300000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.502},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.85399999999999998,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":10.016,"15min":10.016},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:35:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.78200000000000003,"15min":0.65100000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78800000000000003,"5min":6.766,"15min":6.766},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.85399999999999998,"15min":0.70099999999999996},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78500000000000003,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.51400000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:35:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.79600000000000004,"15min":0.69299999999999995},"min":{"1min":0.39400000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":6.806,"15min":6.806},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.90800000000000003,"15min":0.74099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72699999999999998,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.81200000000000006,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.89700000000000002,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.495}]}]},{"osd":3,"up_from":26,"seq":111669149808,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318652,"kb_used_data":9128,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93426500,"statfs":{"total":95995035648,"available":95668736000,"internally_reserved":0,"allocated":9347072,"data_stored":6924820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.65200000000000002,"15min":0.47799999999999998},"min":{"1min":0.20899999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.53100000000000003,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.245},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.75700000000000001,"15min":0.52500000000000002},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.304,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.60499999999999998,"15min":0.48199999999999998},"min":{"1min":0.23899999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.0800000000000001,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":1.1990000000000001,"15min":0.66800000000000004},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.65400000000000003,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:35:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":1.0700000000000001,"15min":0.64800000000000002},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.117,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.54300000000000004,"15min":0.46800000000000003},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52200000000000002,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.88500000000000001,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.96899999999999997,"5min":10.131,"15min":10.131},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.70599999999999996,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71299999999999997,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.47999999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.70099999999999996,"15min":0.63100000000000001},"min":{"1min":0.42199999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66400000000000003,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.81799999999999995,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.92700000000000005,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.56299999999999994}]},{"osd":6,"last update":"Sun Feb 18 00:35:53 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":1.0189999999999999,"15min":0.73599999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86199999999999999,"5min":10.189,"15min":10.189},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.73599999999999999,"15min":0.65900000000000003},"min":{"1min":0.46800000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84999999999999998,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.79900000000000004,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1599999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.93000000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.39800000000000002},"max":{"1min":0.88100000000000001,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.53500000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986283,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306492,"kb_used_data":5168,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438660,"statfs":{"total":95995035648,"available":95681187840,"internally_reserved":0,"allocated":5292032,"data_stored":2925282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":1.556,"15min":0.91800000000000004},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.67800000000000005,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.442},{"interface":"front","average":{"1min":0.51200000000000001,"5min":1.506,"15min":0.877},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72499999999999998,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.81499999999999995,"15min":0.65200000000000002},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.373},{"interface":"front","average":{"1min":0.53200000000000003,"5min":1.595,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":0.82699999999999996,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.47699999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.78900000000000003,"15min":0.66900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.119,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.90500000000000003,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70299999999999996,"5min":17.77,"15min":17.77},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.70599999999999996,"15min":0.63400000000000001},"min":{"1min":0.34899999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":1.8939999999999999,"5min":7.218,"15min":7.218},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.077,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.349,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.49099999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.436,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.66700000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.45100000000000001,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.39800000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46700000000000003,"15min":0.41599999999999998},"min":{"1min":0.32400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70299999999999996,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.45600000000000002},"min":{"1min":0.34000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.751,"15min":0.751},"last":0.45400000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.68999999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.505,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73599999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.503}]}]},{"osd":5,"up_from":39,"seq":167503724646,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306200,"kb_used_data":8968,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93438952,"statfs":{"total":95995035648,"available":95681486848,"internally_reserved":0,"allocated":9183232,"data_stored":6924803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.29799999999999999,"5min":0.217,"15min":0.217},"max":{"1min":2.2549999999999999,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.623},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.76000000000000001,"15min":0.61499999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.8680000000000003,"5min":12.744,"15min":12.744},"last":0.39900000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.70599999999999996,"15min":0.625},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.9850000000000001,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.629},"min":{"1min":0.35199999999999998,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":4.2919999999999998,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.61899999999999999},"min":{"1min":0.32200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":3.2959999999999998,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.71999999999999997,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":4.4359999999999999,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.621},"min":{"1min":0.33200000000000002,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":3.4550000000000001,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.68400000000000005,"15min":0.59799999999999998},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.3920000000000003,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.59999999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.44400000000000001},"min":{"1min":0.31900000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.6859999999999999,"5min":11.818,"15min":11.818},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.501,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.5680000000000001,"5min":4.274,"15min":4.274},"last":0.69599999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:35:48 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.64000000000000001,"15min":0.47999999999999998},"min":{"1min":0.313,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":18.605,"15min":18.605},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.52500000000000002,"15min":0.44500000000000001},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65600000000000003,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.51100000000000001,"15min":0.46899999999999997},"min":{"1min":0.33300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.56799999999999995,"15min":0.502},"min":{"1min":0.318,"5min":0.309,"15min":0.309},"max":{"1min":4.649,"5min":4.649,"15min":4.649},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4272128,"data_stored":4000669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4112384,"data_stored":4000278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4000505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":191,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:36:41.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph status --format=json 2024-02-18T00:36:41.203 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:41.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:41.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:41.216 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:41.262 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:41.307 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:41.340 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9008 ms) 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg_num = 32 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 0 ls 1708216601,0 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 1 ls 1708216601,0 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 2 ls 1708216601,0 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 3 ls 1708216601,0 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 4 ls 1708216601,0 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 5 ls 1708216601,0 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 6 ls 1708216601,0 2024-02-18T00:36:41.388 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 7 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 8 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 9 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 10 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 11 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 12 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 13 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 14 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 15 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 16 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 17 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 18 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 19 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 20 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 21 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 22 ls 1708216601,0 2024-02-18T00:36:41.389 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 23 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 24 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 25 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 26 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 27 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 28 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 29 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 30 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg 31 ls 1708216601,0 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: pg_num = 32 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6cac518f:::0:head 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:02547ec2:::1:head 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f905c69b:::2:head 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:cfc208b3:::3:head 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d83876eb:::4:head 2024-02-18T00:36:41.390 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b29083e3:::5:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c4fdafeb:::6:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:5c6b0b28:::7:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:bd63b0f1:::8:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:e960b815:::9:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:52ea6a34:::10:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:89d3ae78:::11:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:de5d7c5f:::12:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:566253c9:::13:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:62a1935d:::14:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:863748b0:::15:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:3958e169:::16:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:4d4dabf9:::17:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:8391935d:::18:head 2024-02-18T00:36:41.391 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:28883081:::19:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:69259c59:::20:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:4bdb80b7:::21:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a11c5d71:::22:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:271af37b:::23:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:95b121be:::24:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:58d1031b:::25:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:0a050783:::26:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c709704c:::27:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:cbe56eaf:::28:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:86b4b162:::29:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:70d89383:::30:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:dd450c7c:::31:head 2024-02-18T00:36:41.392 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6d5729b1:::32:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c388f3fb:::33:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:56cfea31:::34:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:9dbc1bf7:::35:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:40b74ccd:::36:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:4d5aaf42:::37:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:920f362c:::38:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6cc53222:::39:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:9cad833f:::40:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:1ea84d41:::41:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c4480ef6:::42:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a694361e:::43:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d1bd33e9:::44:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:ddc2cd5d:::45:head 2024-02-18T00:36:41.393 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:2b782207:::46:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:7b187fca:::47:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:90ecdf6f:::48:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a5ed95fe:::49:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:ea0eaa55:::50:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f33ef17b:::51:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a0d1b2f6:::52:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:60c5229e:::53:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:edcbc575:::54:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:102cf253:::55:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:efb7fb0b:::56:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:50d0a326:::57:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d4dc5daf:::58:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:3a130462:::59:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:ec87ed71:::60:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d5bc9454:::61:head 2024-02-18T00:36:41.394 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:3ddfe313:::62:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:7c2816b9:::63:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:47e00e4d:::64:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c6410c18:::65:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b48ed237:::66:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:cd63ad31:::67:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b179e92b:::68:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:0d9f741a:::69:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6d3352ae:::70:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c6d5c19e:::71:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:bc4729c3:::72:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:77e930b9:::73:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:0abeecfd:::74:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b7c37e15:::75:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b6378398:::76:head 2024-02-18T00:36:41.395 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:02bd68de:::77:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:cc795d2d:::78:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:630d4fea:::79:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:e0d29ef5:::80:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:fd6f13d2:::81:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:606461d5:::82:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:eadbdc43:::83:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:8761d0bb:::84:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:9ef0186f:::85:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:e0d41294:::86:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:961de695:::87:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:1423148f:::88:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:633a8fa2:::89:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a8653809:::90:head 2024-02-18T00:36:41.396 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:3dac8b33:::91:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:35aad435:::92:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f6dcc343:::93:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:dbbdad87:::94:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:1cb48ce0:::95:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:03cd461c:::96:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:17a4ea99:::97:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:9993c9a7:::98:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6394211c:::99:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:94c7ae57:::100:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6fdee5bb:::101:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:9a477fd1:::102:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:eb850916:::103:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:affc56b9:::104:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b42dc814:::105:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f319f8f0:::106:head 2024-02-18T00:36:41.397 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:9a40b9de:::107:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:8b524f28:::108:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:e3de589f:::109:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:90f90a5b:::110:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a7b4f1d7:::111:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:af51766e:::112:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b6f90bd1:::113:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:e0261208:::114:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c9569ef7:::115:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:61bebe50:::116:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:fe93412b:::117:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d3d38bee:::118:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:3100ba0c:::119:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d0560ada:::120:head 2024-02-18T00:36:41.398 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f0ea8b35:::121:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:766f231a:::122:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a07a2582:::123:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:bd7c6b3a:::124:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:fb2ddaff:::125:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:4408e1fe:::126:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:ee1df7a7:::127:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c3002909:::128:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:4f48ffa9:::129:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:edf38733:::130:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c08425c0:::131:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:5f902d98:::132:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:41ea2c93:::133:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:813cee13:::134:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:0131818d:::135:head 2024-02-18T00:36:41.399 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:26ba5a85:::136:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:381b8a5a:::137:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:28797e47:::138:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:bfca7f22:::139:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:36807075:::140:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:80b03975:::141:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:5c15709b:::142:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f39ea15e:::143:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:ea992956:::144:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:48887b1c:::145:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:9f24a9dd:::146:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:987f100b:::147:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d2dd3581:::148:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:7fed1808:::149:head 2024-02-18T00:36:41.400 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c80b70e9:::150:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:85ed90f9:::151:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:36428b24:::152:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d044c34a:::153:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:7c18bf58:::154:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d1c21232:::155:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a7a3c575:::156:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:87da0633:::157:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d5ac3822:::158:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:3f20522d:::159:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6ca26563:::160:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:532ce135:::161:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:c78863e6:::162:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:7462ddf6:::163:head 2024-02-18T00:36:41.401 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:5d165639:::164:head 2024-02-18T00:36:41.402 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f43765fc:::165:head 2024-02-18T00:36:41.402 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:41.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:39.870256+0000 mgr.y (mgr.24454) 323 : cluster [DBG] pgmap v469: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:41.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.097450+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-18T00:36:41.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.097532+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e322: 8 total, 8 up, 7 in 2024-02-18T00:36:41.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:41.403 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:41.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:39.870256+0000 mgr.y (mgr.24454) 323 : cluster [DBG] pgmap v469: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:41.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.097450+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-18T00:36:41.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.097532+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e322: 8 total, 8 up, 7 in 2024-02-18T00:36:41.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:36:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:36:41.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:36:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:36:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:39.870256+0000 mgr.y (mgr.24454) 323 : cluster [DBG] pgmap v469: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.097450+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-18T00:36:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.097532+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e322: 8 total, 8 up, 7 in 2024-02-18T00:36:41.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:41.552 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:41.555 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:41.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:41.656 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:41.693 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 -- 172.21.15.59:0/3990629802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fd510 msgr2=0x7fd8bc0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/3990629802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fd510 0x7fd8bc0f6d40 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 rx=0x7fd8a4004870 tx=0x7fd8a4009cb0).stop 2024-02-18T00:36:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 -- 172.21.15.59:0/3990629802 shutdown_connections 2024-02-18T00:36:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/3990629802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8bc0fcbb0 0x7fd8bc0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/3990629802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8bc0f7400 0x7fd8bc103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/3990629802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fd510 0x7fd8bc0f6d40 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 -- 172.21.15.59:0/3990629802 >> 172.21.15.59:0/3990629802 conn(0x7fd8bc0f2820 msgr2=0x7fd8bc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 -- 172.21.15.59:0/3990629802 shutdown_connections 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.951+0000 7fd8c0cba700 1 -- 172.21.15.59:0/3990629802 wait complete. 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 librados: starting msgr at 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 librados: starting objecter 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 Processor -- start 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 -- start start 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 librados: setting wanted keys 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 librados: calling monclient init 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8bc0f7400 0x7fd8bc0f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fcbb0 0x7fd8bc0f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:41.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8bc0fd510 0x7fd8bc0f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc116710 con 0x7fd8bc0f7400 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc116880 con 0x7fd8bc0fd510 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.952+0000 7fd8c0cba700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc116a40 con 0x7fd8bc0fcbb0 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8bc0f7400 0x7fd8bc0f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8bc0f7400 0x7fd8bc0f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38482/0 (socket says 172.21.15.59:38482) 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 -- 172.21.15.59:0/1617065937 learned_addr learned my addr 172.21.15.59:0/1617065937 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fcbb0 0x7fd8bc0f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8bad9d700 1 --2- 172.21.15.59:0/1617065937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8bc0fd510 0x7fd8bc0f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 -- 172.21.15.59:0/1617065937 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fcbb0 msgr2=0x7fd8bc0f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fcbb0 0x7fd8bc0f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 -- 172.21.15.59:0/1617065937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8bc0fd510 msgr2=0x7fd8bc0f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8bc0fd510 0x7fd8bc0f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 -- 172.21.15.59:0/1617065937 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4005040 con 0x7fd8bc0f7400 2024-02-18T00:36:41.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd8bc0fcbb0 0x7fd8bc0f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8bc0f7400 0x7fd8bc0f89b0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 rx=0x7fd8ac00db80 tx=0x7fd8ac004e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8b37fe700 10 client.?.objecter ms_handle_connect 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8b37fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac007e70 con 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a4005040 con 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8ac00f040 con 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8c0cba700 10 client.31098.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc117080 con 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8c0cba700 1 librados: init done 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.953+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac003640 con 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.954+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd8bc05fa60 con 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd8ac00b750 con 0x7fd8bc0f7400 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd8a8062c90 0x7fd8a8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:41.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(323..323 src has 44..323) v4 ==== 8831+0+0 (secure 0 0 0) 0x7fd8ac07db50 con 0x7fd8bc0f7400 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 osd_map(323..323 src has 44..323) v4 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 3 client.31098.objecter handle_osd_map got epochs [323,323] > 0 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 3 client.31098.objecter handle_osd_map decoding full epoch 323 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd8a8062c90 0x7fd8a8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 20 client.31098.objecter dump_active .. 0 homeless 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.955+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v323) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd8ac07e0a0 con 0x7fd8bc0f7400 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8a8062c90 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd8a8062c90 0x7fd8a8065150 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 rx=0x7fd8a4004ee0 tx=0x7fd8a400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 1 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd8bc0ffed0 0x7fd8bc1002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:41.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 1 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 0 1 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc05ee80 osd=0 4 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 10 client.31098.objecter _send_command 1 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc106120 con 0x7fd8bc0ffed0 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8bad9d700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd8bc0ffed0 0x7fd8bc1002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc0ffed0 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.956+0000 7fd8bad9d700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd8bc0ffed0 0x7fd8bc1002b0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.957+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc106120 con 0x7fd8bc0ffed0 2024-02-18T00:36:41.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.957+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:36:41.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.957+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 1 = system:0 2024-02-18T00:36:41.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.957+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc05ee80 osd=0 4 2024-02-18T00:36:41.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.957+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 0 1 2024-02-18T00:36:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 2 2024-02-18T00:36:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 2 2024-02-18T00:36:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc05ee80 osd=0 3 2024-02-18T00:36:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 0 2 2024-02-18T00:36:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc05ee80 osd=0 4 2024-02-18T00:36:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 10 client.31098.objecter _send_command 2 2024-02-18T00:36:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.971+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc0273c0 con 0x7fd8bc0ffed0 2024-02-18T00:36:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc0273c0 con 0x7fd8bc0ffed0 2024-02-18T00:36:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc05ee80 osd=0 4 2024-02-18T00:36:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 0 2 2024-02-18T00:36:41.976 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 3 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6805/26279110 conn(0x7fd8bc0598f0 0x7fd8bc059d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 3 2024-02-18T00:36:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 1 3 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc028710 osd=1 4 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 10 client.31098.objecter _send_command 3 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc11d990 con 0x7fd8bc0598f0 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.973+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6805/26279110 conn(0x7fd8bc0598f0 0x7fd8bc059d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.974+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6805/26279110 conn(0x7fd8bc0598f0 0x7fd8bc059d60 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.974+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc0598f0 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.974+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc11d990 con 0x7fd8bc0598f0 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.974+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.974+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 3 = system:0 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.974+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc028710 osd=1 4 2024-02-18T00:36:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.974+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 1 3 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 4 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 4 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc028710 osd=1 3 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 1 4 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc028710 osd=1 4 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 10 client.31098.objecter _send_command 4 2024-02-18T00:36:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.989+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc059360 con 0x7fd8bc0598f0 2024-02-18T00:36:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.990+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc059360 con 0x7fd8bc0598f0 2024-02-18T00:36:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.990+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.990+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.990+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc028710 osd=1 4 2024-02-18T00:36:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.990+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 1 4 2024-02-18T00:36:41.993 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:36:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 5 2024-02-18T00:36:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd8bc028270 0x7fd8bc11c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 5 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 2 5 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc059770 osd=2 4 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 10 client.31098.objecter _send_command 5 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc11c8a0 con 0x7fd8bc028270 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd8bc028270 0x7fd8bc11c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc028270 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.991+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd8bc028270 0x7fd8bc11c3b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:41.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.992+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc11c8a0 con 0x7fd8bc028270 2024-02-18T00:36:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.992+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:36:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.992+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 5 = system:0 2024-02-18T00:36:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.992+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc059770 osd=2 4 2024-02-18T00:36:41.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:41.992+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 2 5 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 6 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 6 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc059770 osd=2 3 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 2 6 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc059770 osd=2 4 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 10 client.31098.objecter _send_command 6 2024-02-18T00:36:42.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.006+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc0277d0 con 0x7fd8bc028270 2024-02-18T00:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc0277d0 con 0x7fd8bc028270 2024-02-18T00:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:42.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc059770 osd=2 4 2024-02-18T00:36:42.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 2 6 2024-02-18T00:36:42.010 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 7 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd8bc11fb10 0x7fd8bc121f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 7 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 3 7 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11d290 osd=3 4 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 10 client.31098.objecter _send_command 7 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc124b50 con 0x7fd8bc11fb10 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8bad9d700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd8bc11fb10 0x7fd8bc121f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc11fb10 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.008+0000 7fd8bad9d700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd8bc11fb10 0x7fd8bc121f00 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.009+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc124b50 con 0x7fd8bc11fb10 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.009+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.009+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 7 = system:0 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.009+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc11d290 osd=3 4 2024-02-18T00:36:42.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.009+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 3 7 2024-02-18T00:36:42.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 8 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 8 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc11d290 osd=3 3 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 3 8 2024-02-18T00:36:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11d290 osd=3 4 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 10 client.31098.objecter _send_command 8 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.024+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc11cd00 con 0x7fd8bc11fb10 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.025+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc11cd00 con 0x7fd8bc11fb10 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.025+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.025+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.025+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc11d290 osd=3 4 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.025+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 3 8 2024-02-18T00:36:42.029 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:36:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 9 2024-02-18T00:36:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd8bc11cea0 0x7fd8bc123570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 9 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 4 9 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11cd00 osd=4 4 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 10 client.31098.objecter _send_command 9 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc11c090 con 0x7fd8bc11cea0 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd8bc11cea0 0x7fd8bc123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc11cea0 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.026+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd8bc11cea0 0x7fd8bc123570 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.027+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc11c090 con 0x7fd8bc11cea0 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.027+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.027+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 9 = system:0 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.027+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc11cd00 osd=4 4 2024-02-18T00:36:42.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.027+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 4 9 2024-02-18T00:36:42.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 10 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 10 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc11cd00 osd=4 3 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 4 10 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11cd00 osd=4 4 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 10 client.31098.objecter _send_command 10 2024-02-18T00:36:42.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.043+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc11c090 con 0x7fd8bc11cea0 2024-02-18T00:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc11c090 con 0x7fd8bc11cea0 2024-02-18T00:36:42.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc11cd00 osd=4 4 2024-02-18T00:36:42.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 4 10 2024-02-18T00:36:42.047 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 11 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd8bc123f70 0x7fd8bc1243d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 11 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 5 11 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11c090 osd=5 4 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 10 client.31098.objecter _send_command 11 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc1249b0 con 0x7fd8bc123f70 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd8bc123f70 0x7fd8bc1243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc123f70 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.045+0000 7fd8b9d9b700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd8bc123f70 0x7fd8bc1243d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.046+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc1249b0 con 0x7fd8bc123f70 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.046+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.046+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 11 = system:0 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.046+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc11c090 osd=5 4 2024-02-18T00:36:42.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.046+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 5 11 2024-02-18T00:36:42.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:42.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 12 2024-02-18T00:36:42.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 12 2024-02-18T00:36:42.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc11c090 osd=5 3 2024-02-18T00:36:42.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 5 12 2024-02-18T00:36:42.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11c090 osd=5 4 2024-02-18T00:36:42.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 10 client.31098.objecter _send_command 12 2024-02-18T00:36:42.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.060+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc1249b0 con 0x7fd8bc123f70 2024-02-18T00:36:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.062+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc1249b0 con 0x7fd8bc123f70 2024-02-18T00:36:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.062+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.062+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.062+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc11c090 osd=5 4 2024-02-18T00:36:42.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.062+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 5 12 2024-02-18T00:36:42.065 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:36:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 13 2024-02-18T00:36:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd8bc1291f0 0x7fd8bc129650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 13 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 6 13 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc1249b0 osd=6 4 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 10 client.31098.objecter _send_command 13 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc12b240 con 0x7fd8bc1291f0 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8bad9d700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd8bc1291f0 0x7fd8bc129650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc1291f0 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.063+0000 7fd8bad9d700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd8bc1291f0 0x7fd8bc129650 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:42.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.064+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc12b240 con 0x7fd8bc1291f0 2024-02-18T00:36:42.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.064+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:36:42.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.064+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 13 = system:0 2024-02-18T00:36:42.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.064+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc1249b0 osd=6 4 2024-02-18T00:36:42.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.064+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 6 13 2024-02-18T00:36:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 14 2024-02-18T00:36:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 14 2024-02-18T00:36:42.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc1249b0 osd=6 3 2024-02-18T00:36:42.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 6 14 2024-02-18T00:36:42.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc1249b0 osd=6 4 2024-02-18T00:36:42.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 10 client.31098.objecter _send_command 14 2024-02-18T00:36:42.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.078+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc12b240 con 0x7fd8bc1291f0 2024-02-18T00:36:42.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc12b240 con 0x7fd8bc1291f0 2024-02-18T00:36:42.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:42.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc1249b0 osd=6 4 2024-02-18T00:36:42.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 6 14 2024-02-18T00:36:42.082 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:36:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 15 2024-02-18T00:36:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6812/53025171 conn(0x7fd8bc12d9e0 0x7fd8bc12de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 15 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 7 15 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc12b240 osd=7 4 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 10 client.31098.objecter _send_command 15 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.080+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8bc12e450 con 0x7fd8bc12d9e0 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.081+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6812/53025171 conn(0x7fd8bc12d9e0 0x7fd8bc12de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.081+0000 7fd8b37fe700 10 client.31098.objecter ms_handle_connect 0x7fd8bc12d9e0 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.081+0000 7fd8ba59c700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6812/53025171 conn(0x7fd8bc12d9e0 0x7fd8bc12de40 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.082+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd8bc12e450 con 0x7fd8bc12d9e0 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.082+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.082+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 15 = system:0 2024-02-18T00:36:42.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.082+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc12b240 osd=7 4 2024-02-18T00:36:42.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.082+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 7 15 2024-02-18T00:36:42.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 10 client.31098.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign -1 16 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 20 client.31098.objecter _get_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_remove -1 16 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 20 client.31098.objecter get_session s=0x7fd8bc12b240 osd=7 3 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 15 client.31098.objecter _session_command_op_assign 7 16 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc12b240 osd=7 4 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 10 client.31098.objecter _send_command 16 2024-02-18T00:36:42.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.096+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd8bc12e450 con 0x7fd8bc12d9e0 2024-02-18T00:36:42.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8b37fe700 1 -- 172.21.15.59:0/1617065937 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd8bc12e450 con 0x7fd8bc12d9e0 2024-02-18T00:36:42.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8b37fe700 10 client.31098.objecter ms_dispatch 0x7fd8bc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:36:42.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8b37fe700 10 client.31098.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8b37fe700 20 client.31098.objecter put_session s=0x7fd8bc12b240 osd=7 4 2024-02-18T00:36:42.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8b37fe700 15 client.31098.objecter _session_command_op_remove 7 16 2024-02-18T00:36:42.100 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 librados: watch_flush enter 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 librados: watch_flush exit 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.0 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd8bc0ffed0 msgr2=0x7fd8bc1002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd8bc0ffed0 0x7fd8bc1002b0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc05ee80 osd=0 1 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.1 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6805/26279110 conn(0x7fd8bc0598f0 msgr2=0x7fd8bc059d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6805/26279110 conn(0x7fd8bc0598f0 0x7fd8bc059d60 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc028710 osd=1 1 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.2 2024-02-18T00:36:42.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd8bc028270 msgr2=0x7fd8bc11c3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd8bc028270 0x7fd8bc11c3b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc059770 osd=2 1 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.3 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd8bc11fb10 msgr2=0x7fd8bc121f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd8bc11fb10 0x7fd8bc121f00 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11d290 osd=3 1 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.4 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd8bc11cea0 msgr2=0x7fd8bc123570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd8bc11cea0 0x7fd8bc123570 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8bad9d700 1 -- 172.21.15.59:0/1617065937 reap_dead start 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11cd00 osd=4 1 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.5 2024-02-18T00:36:42.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd8bc123f70 msgr2=0x7fd8bc1243d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd8bc123f70 0x7fd8bc1243d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc11c090 osd=5 1 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.6 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd8bc1291f0 msgr2=0x7fd8bc129650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd8bc1291f0 0x7fd8bc129650 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc1249b0 osd=6 1 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter close_session for osd.7 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6812/53025171 conn(0x7fd8bc12d9e0 msgr2=0x7fd8bc12de40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.137:6812/53025171 conn(0x7fd8bc12d9e0 0x7fd8bc12de40 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter put_session s=0x7fd8bc12b240 osd=7 1 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 20 client.31098.objecter shutdown clearing up homeless session... 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 10 client.31098.objecter successfully canceled tick 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd8a8062c90 msgr2=0x7fd8a8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd8a8062c90 0x7fd8a8065150 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 rx=0x7fd8a4004ee0 tx=0x7fd8a400b040).stop 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8bad9d700 1 -- 172.21.15.59:0/1617065937 reap_dead start 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8bc0f7400 msgr2=0x7fd8bc0f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 --2- 172.21.15.59:0/1617065937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8bc0f7400 0x7fd8bc0f89b0 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 rx=0x7fd8ac00db80 tx=0x7fd8ac004e60).stop 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8bad9d700 1 -- 172.21.15.59:0/1617065937 reap_dead start 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8bad9d700 1 -- 172.21.15.59:0/1617065937 reap_dead start 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8bad9d700 1 -- 172.21.15.59:0/1617065937 reap_dead start 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 shutdown_connections 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 >> 172.21.15.59:0/1617065937 conn(0x7fd8bc0f2820 msgr2=0x7fd8bc0f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.098+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 shutdown_connections 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.099+0000 7fd8c0cba700 1 -- 172.21.15.59:0/1617065937 wait complete. 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:42.099+0000 7fd8c0cba700 1 librados: shutdown 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.105 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.106 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.106 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.106 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.106 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:42] "GET /metrics HTTP/1.1" 200 188787 "" "Prometheus/2.33.4" 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:40.823240+0000 osd.5 (osd.5) 139 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:40.825047+0000 osd.5 (osd.5) 140 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.150859+0000 mgr.y (mgr.24454) 324 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.386794+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi059-82957-41","var": "pg_num","format": "json"}]: dispatch 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.507122+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.507483+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:42.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.508030+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]: dispatch 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.508351+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]: dispatch 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.620690+0000 osd.2 (osd.2) 137 : cluster [DBG] 62.1f scrub starts 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.624176+0000 osd.2 (osd.2) 138 : cluster [DBG] 62.1f scrub ok 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.667961+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.668198+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]': finished 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.668249+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e323: 8 total, 8 up, 7 in 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:41.955927+0000 mon.a (mon.0) 2165 : audit [DBG] from='client.? 172.21.15.59:0/1617065937' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:40.823240+0000 osd.5 (osd.5) 139 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:40.825047+0000 osd.5 (osd.5) 140 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.150859+0000 mgr.y (mgr.24454) 324 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.386794+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi059-82957-41","var": "pg_num","format": "json"}]: dispatch 2024-02-18T00:36:42.370 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.507122+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.507483+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.508030+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]: dispatch 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.508351+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]: dispatch 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.620690+0000 osd.2 (osd.2) 137 : cluster [DBG] 62.1f scrub starts 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.624176+0000 osd.2 (osd.2) 138 : cluster [DBG] 62.1f scrub ok 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.667961+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.668198+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]': finished 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.668249+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e323: 8 total, 8 up, 7 in 2024-02-18T00:36:42.371 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:41.955927+0000 mon.a (mon.0) 2165 : audit [DBG] from='client.? 172.21.15.59:0/1617065937' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:40.823240+0000 osd.5 (osd.5) 139 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:40.825047+0000 osd.5 (osd.5) 140 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.150859+0000 mgr.y (mgr.24454) 324 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.386794+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi059-82957-41","var": "pg_num","format": "json"}]: dispatch 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.507122+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.507483+0000 mon.a (mon.0) 2160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.508030+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]: dispatch 2024-02-18T00:36:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.508351+0000 mon.a (mon.0) 2161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]: dispatch 2024-02-18T00:36:42.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.620690+0000 osd.2 (osd.2) 137 : cluster [DBG] 62.1f scrub starts 2024-02-18T00:36:42.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.624176+0000 osd.2 (osd.2) 138 : cluster [DBG] 62.1f scrub ok 2024-02-18T00:36:42.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.667961+0000 mon.a (mon.0) 2162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi059-82882-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:42.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.668198+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-41"}]': finished 2024-02-18T00:36:42.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.668249+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e323: 8 total, 8 up, 7 in 2024-02-18T00:36:42.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:41.955927+0000 mon.a (mon.0) 2165 : audit [DBG] from='client.? 172.21.15.59:0/1617065937' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.268476+0000 osd.1 (osd.1) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.268769+0000 osd.1 (osd.1) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.797679+0000 osd.5 (osd.5) 141 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:36:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.801139+0000 osd.5 (osd.5) 142 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:36:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:41.870679+0000 mgr.y (mgr.24454) 325 : cluster [DBG] pgmap v472: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.668617+0000 mon.a (mon.0) 2166 : cluster [DBG] osdmap e324: 8 total, 8 up, 7 in 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.268476+0000 osd.1 (osd.1) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.268769+0000 osd.1 (osd.1) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.797679+0000 osd.5 (osd.5) 141 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.801139+0000 osd.5 (osd.5) 142 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:41.870679+0000 mgr.y (mgr.24454) 325 : cluster [DBG] pgmap v472: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.668617+0000 mon.a (mon.0) 2166 : cluster [DBG] osdmap e324: 8 total, 8 up, 7 in 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.268476+0000 osd.1 (osd.1) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.268769+0000 osd.1 (osd.1) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.797679+0000 osd.5 (osd.5) 141 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:36:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.801139+0000 osd.5 (osd.5) 142 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:36:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:41.870679+0000 mgr.y (mgr.24454) 325 : cluster [DBG] pgmap v472: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.668617+0000 mon.a (mon.0) 2166 : cluster [DBG] osdmap e324: 8 total, 8 up, 7 in 2024-02-18T00:36:44.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.354211+0000 osd.0 (osd.0) 151 : cluster [DBG] 25.e scrub starts 2024-02-18T00:36:44.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.355111+0000 osd.0 (osd.0) 152 : cluster [DBG] 25.e scrub ok 2024-02-18T00:36:44.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.641784+0000 osd.6 (osd.6) 161 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:36:44.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.643428+0000 osd.6 (osd.6) 162 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:36:44.379 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.825559+0000 osd.5 (osd.5) 143 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.828537+0000 osd.5 (osd.5) 144 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.854663+0000 osd.1 (osd.1) 163 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:42.857554+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.279752+0000 osd.4 (osd.4) 185 : cluster [DBG] 4.e scrub starts 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.283226+0000 osd.4 (osd.4) 186 : cluster [DBG] 4.e scrub ok 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.341560+0000 osd.0 (osd.0) 153 : cluster [DBG] 62.13 scrub starts 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.345055+0000 osd.0 (osd.0) 154 : cluster [DBG] 62.13 scrub ok 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.633130+0000 mon.a (mon.0) 2167 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.639015+0000 mon.a (mon.0) 2168 : cluster [DBG] osdmap e325: 8 total, 8 up, 7 in 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:43.646523+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:43.646742+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:43.646903+0000 mon.a (mon.0) 2169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:44.380 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:43.647406+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.354211+0000 osd.0 (osd.0) 151 : cluster [DBG] 25.e scrub starts 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.355111+0000 osd.0 (osd.0) 152 : cluster [DBG] 25.e scrub ok 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.641784+0000 osd.6 (osd.6) 161 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.643428+0000 osd.6 (osd.6) 162 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.825559+0000 osd.5 (osd.5) 143 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.828537+0000 osd.5 (osd.5) 144 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.854663+0000 osd.1 (osd.1) 163 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:42.857554+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.279752+0000 osd.4 (osd.4) 185 : cluster [DBG] 4.e scrub starts 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.283226+0000 osd.4 (osd.4) 186 : cluster [DBG] 4.e scrub ok 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.341560+0000 osd.0 (osd.0) 153 : cluster [DBG] 62.13 scrub starts 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.345055+0000 osd.0 (osd.0) 154 : cluster [DBG] 62.13 scrub ok 2024-02-18T00:36:44.381 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.633130+0000 mon.a (mon.0) 2167 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:36:44.382 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.639015+0000 mon.a (mon.0) 2168 : cluster [DBG] osdmap e325: 8 total, 8 up, 7 in 2024-02-18T00:36:44.382 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:43.646523+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:44.382 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:43.646742+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:44.382 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:43.646903+0000 mon.a (mon.0) 2169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:44.382 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:43.647406+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.354211+0000 osd.0 (osd.0) 151 : cluster [DBG] 25.e scrub starts 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.355111+0000 osd.0 (osd.0) 152 : cluster [DBG] 25.e scrub ok 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.641784+0000 osd.6 (osd.6) 161 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.643428+0000 osd.6 (osd.6) 162 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.825559+0000 osd.5 (osd.5) 143 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.828537+0000 osd.5 (osd.5) 144 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.854663+0000 osd.1 (osd.1) 163 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:42.857554+0000 osd.1 (osd.1) 164 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:36:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.279752+0000 osd.4 (osd.4) 185 : cluster [DBG] 4.e scrub starts 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.283226+0000 osd.4 (osd.4) 186 : cluster [DBG] 4.e scrub ok 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.341560+0000 osd.0 (osd.0) 153 : cluster [DBG] 62.13 scrub starts 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.345055+0000 osd.0 (osd.0) 154 : cluster [DBG] 62.13 scrub ok 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.633130+0000 mon.a (mon.0) 2167 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.639015+0000 mon.a (mon.0) 2168 : cluster [DBG] osdmap e325: 8 total, 8 up, 7 in 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:43.646523+0000 mon.c (mon.2) 306 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:43.646742+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:43.646903+0000 mon.a (mon.0) 2169 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:44.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:43.647406+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:36:44.757 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:36:44.757 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:36:44.757 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:36:44.757 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:36:44.757 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:36:45.301 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:36:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.836762+0000 osd.1 (osd.1) 165 : cluster [DBG] 2.d scrub starts 2024-02-18T00:36:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.837698+0000 osd.1 (osd.1) 166 : cluster [DBG] 2.d scrub ok 2024-02-18T00:36:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.871064+0000 mgr.y (mgr.24454) 326 : cluster [DBG] pgmap v475: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.932364+0000 osd.3 (osd.3) 177 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:36:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:43.933411+0000 osd.3 (osd.3) 178 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:44.365880+0000 osd.0 (osd.0) 155 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:44.367025+0000 osd.0 (osd.0) 156 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:44.645847+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:44.645972+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:44.646053+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e326: 8 total, 8 up, 7 in 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:44.647716+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:44.653108+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:44.656213+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:44.658088+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:45.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.836762+0000 osd.1 (osd.1) 165 : cluster [DBG] 2.d scrub starts 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.837698+0000 osd.1 (osd.1) 166 : cluster [DBG] 2.d scrub ok 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.871064+0000 mgr.y (mgr.24454) 326 : cluster [DBG] pgmap v475: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.932364+0000 osd.3 (osd.3) 177 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:43.933411+0000 osd.3 (osd.3) 178 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:44.365880+0000 osd.0 (osd.0) 155 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:44.367025+0000 osd.0 (osd.0) 156 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:44.645847+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:44.645972+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:44.646053+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e326: 8 total, 8 up, 7 in 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:44.647716+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:44.653108+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:44.656213+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:45.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:44.658088+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.836762+0000 osd.1 (osd.1) 165 : cluster [DBG] 2.d scrub starts 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.837698+0000 osd.1 (osd.1) 166 : cluster [DBG] 2.d scrub ok 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.871064+0000 mgr.y (mgr.24454) 326 : cluster [DBG] pgmap v475: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.932364+0000 osd.3 (osd.3) 177 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:43.933411+0000 osd.3 (osd.3) 178 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:44.365880+0000 osd.0 (osd.0) 155 : cluster [DBG] 6.16 scrub starts 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:44.367025+0000 osd.0 (osd.0) 156 : cluster [DBG] 6.16 scrub ok 2024-02-18T00:36:45.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:44.645847+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:44.645972+0000 mon.a (mon.0) 2172 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:36:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:44.646053+0000 mon.a (mon.0) 2173 : cluster [DBG] osdmap e326: 8 total, 8 up, 7 in 2024-02-18T00:36:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:44.647716+0000 mon.c (mon.2) 308 : audit [INF] from='client.? 172.21.15.59:0/2706507942' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:44.653108+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]: dispatch 2024-02-18T00:36:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:44.656213+0000 mon.c (mon.2) 309 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:45.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:44.658088+0000 mon.a (mon.0) 2175 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:36:45.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-18T00:36:46.258 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:46.262 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:46.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:46.281 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:46.363 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:46.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:44.616426+0000 osd.6 (osd.6) 163 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:36:46.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:44.617438+0000 osd.6 (osd.6) 164 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:36:46.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.307253+0000 osd.4 (osd.4) 187 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:36:46.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.308294+0000 osd.4 (osd.4) 188 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:36:46.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.411554+0000 osd.0 (osd.0) 157 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:36:46.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.412402+0000 osd.0 (osd.0) 158 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.626764+0000 osd.2 (osd.2) 139 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.627697+0000 osd.2 (osd.2) 140 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.646725+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.646781+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.646821+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e327: 8 total, 8 up, 7 in 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.649983+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.650313+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.666545+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.666837+0000 mon.a (mon.0) 2180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.667433+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.667676+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.668054+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:46.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:45.668291+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:46.395 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:46.416 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:46.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:46.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:44.616426+0000 osd.6 (osd.6) 163 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:44.617438+0000 osd.6 (osd.6) 164 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.307253+0000 osd.4 (osd.4) 187 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.308294+0000 osd.4 (osd.4) 188 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.411554+0000 osd.0 (osd.0) 157 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.412402+0000 osd.0 (osd.0) 158 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.626764+0000 osd.2 (osd.2) 139 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.627697+0000 osd.2 (osd.2) 140 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.646725+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.646781+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.646821+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e327: 8 total, 8 up, 7 in 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.649983+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.650313+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.666545+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.666837+0000 mon.a (mon.0) 2180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.667433+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.667676+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.668054+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:46.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:45.668291+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:44.616426+0000 osd.6 (osd.6) 163 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:44.617438+0000 osd.6 (osd.6) 164 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.307253+0000 osd.4 (osd.4) 187 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.308294+0000 osd.4 (osd.4) 188 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.411554+0000 osd.0 (osd.0) 157 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.412402+0000 osd.0 (osd.0) 158 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.626764+0000 osd.2 (osd.2) 139 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.627697+0000 osd.2 (osd.2) 140 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.646725+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi059-82882-55"}]': finished 2024-02-18T00:36:46.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.646781+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.646821+0000 mon.a (mon.0) 2178 : cluster [DBG] osdmap e327: 8 total, 8 up, 7 in 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.649983+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.650313+0000 mon.a (mon.0) 2179 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.666545+0000 mon.c (mon.2) 311 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.666837+0000 mon.a (mon.0) 2180 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.667433+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.667676+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.668054+0000 mon.c (mon.2) 313 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:46.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:45.668291+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:46.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:46.548 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:46.646 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:46.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:46.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:46.752 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:46.778 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:46.797 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:46.799 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:46.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:46.901 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:46.927 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:46.946 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:46.948 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:46.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:47.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:47.078 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.772693+0000 osd.5 (osd.5) 145 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:36:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.773691+0000 osd.5 (osd.5) 146 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:36:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.859021+0000 osd.1 (osd.1) 167 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:36:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.861963+0000 osd.1 (osd.1) 168 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:36:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:45.871903+0000 mgr.y (mgr.24454) 327 : cluster [DBG] pgmap v478: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.324292+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:46.368403+0000 osd.0 (osd.0) 159 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:46.369495+0000 osd.0 (osd.0) 160 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.653594+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]': finished 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.653683+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:46.653759+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e328: 8 total, 8 up, 7 in 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.654927+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.656064+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.659009+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.659697+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.758743+0000 mon.a (mon.0) 2189 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.759423+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:36:47.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:46.763811+0000 mon.a (mon.0) 2191 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.772693+0000 osd.5 (osd.5) 145 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.773691+0000 osd.5 (osd.5) 146 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.859021+0000 osd.1 (osd.1) 167 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.861963+0000 osd.1 (osd.1) 168 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:45.871903+0000 mgr.y (mgr.24454) 327 : cluster [DBG] pgmap v478: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.324292+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:46.368403+0000 osd.0 (osd.0) 159 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:46.369495+0000 osd.0 (osd.0) 160 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.653594+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]': finished 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.653683+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:46.653759+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e328: 8 total, 8 up, 7 in 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.654927+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:47.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.656064+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:47.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.659009+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:36:47.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.659697+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:36:47.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.758743+0000 mon.a (mon.0) 2189 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:36:47.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.759423+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:36:47.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:46.763811+0000 mon.a (mon.0) 2191 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.772693+0000 osd.5 (osd.5) 145 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:36:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.773691+0000 osd.5 (osd.5) 146 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:36:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.859021+0000 osd.1 (osd.1) 167 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:36:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.861963+0000 osd.1 (osd.1) 168 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:36:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:45.871903+0000 mgr.y (mgr.24454) 327 : cluster [DBG] pgmap v478: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.324292+0000 mon.a (mon.0) 2183 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:46.368403+0000 osd.0 (osd.0) 159 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:46.369495+0000 osd.0 (osd.0) 160 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.653594+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_count","val": "3"}]': finished 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.653683+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi059-82882-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:46.653759+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e328: 8 total, 8 up, 7 in 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.654927+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.656064+0000 mon.a (mon.0) 2187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.659009+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.659697+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:36:47.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.758743+0000 mon.a (mon.0) 2189 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:36:47.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.759423+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:36:47.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:46.763811+0000 mon.a (mon.0) 2191 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:47.766 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:47.777 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:47.828 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:47.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:47.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:47.932 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:47.965 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:47.986 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:47.989 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:47.989 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:48.090 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:48.122 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:48.146 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:48.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:48.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:48.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:46.939344+0000 osd.3 (osd.3) 179 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:36:48.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:46.942578+0000 osd.3 (osd.3) 180 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:36:48.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:48.090726+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]': finished 2024-02-18T00:36:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:48.090810+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e329: 8 total, 8 up, 7 in 2024-02-18T00:36:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:48.091860+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:36:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:48.092213+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:36:48.277 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:48.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:46.939344+0000 osd.3 (osd.3) 179 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:36:48.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:46.942578+0000 osd.3 (osd.3) 180 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:48.090726+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]': finished 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:48.090810+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e329: 8 total, 8 up, 7 in 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:48.091860+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:48.092213+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:46.939344+0000 osd.3 (osd.3) 179 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:46.942578+0000 osd.3 (osd.3) 180 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:48.090726+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_period","val": "3"}]': finished 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:48.090810+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e329: 8 total, 8 up, 7 in 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:48.091860+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:36:48.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:48.092213+0000 mon.a (mon.0) 2194 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:36:48.476 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:48.479 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:48.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:48.581 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:48.613 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:48.633 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:48.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:48.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:48.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:48.798 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:48.801 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:48.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:48.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.818+0000 7fadec331700 1 -- 172.21.15.59:0/1728903327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade4102270 msgr2=0x7fade4104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:48.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.818+0000 7fadec331700 1 --2- 172.21.15.59:0/1728903327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade4102270 0x7fade4104660 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 rx=0x7faddc009c30 tx=0x7faddc004c50).stop 2024-02-18T00:36:48.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.818+0000 7fadec331700 1 -- 172.21.15.59:0/1728903327 shutdown_connections 2024-02-18T00:36:48.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.818+0000 7fadec331700 1 --2- 172.21.15.59:0/1728903327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade4102270 0x7fade4104660 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:48.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.818+0000 7fadec331700 1 --2- 172.21.15.59:0/1728903327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade40f8d10 0x7fade40f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:48.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.818+0000 7fadec331700 1 --2- 172.21.15.59:0/1728903327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade40f9670 0x7fade4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:48.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.818+0000 7fadec331700 1 -- 172.21.15.59:0/1728903327 >> 172.21.15.59:0/1728903327 conn(0x7fade40f2820 msgr2=0x7fade40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:48.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.820+0000 7fadec331700 1 -- 172.21.15.59:0/1728903327 shutdown_connections 2024-02-18T00:36:48.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.820+0000 7fadec331700 1 -- 172.21.15.59:0/1728903327 wait complete. 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 librados: starting msgr at 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 librados: starting objecter 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 Processor -- start 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 -- start start 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 librados: setting wanted keys 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 librados: calling monclient init 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade40f8d10 0x7fade410e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade40f9670 0x7fade410e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:48.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade4102270 0x7fade4108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade41086f0 con 0x7fade4102270 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade4108860 con 0x7fade40f8d10 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadec331700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4108a20 con 0x7fade40f9670 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fadea8ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade4102270 0x7fade4108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.821+0000 7fade98cc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade40f9670 0x7fade410e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade40f9670 0x7fade410e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:41390/0 (socket says 172.21.15.59:41390) 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fadea8ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade4102270 0x7fade4108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38506/0 (socket says 172.21.15.59:38506) 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 -- 172.21.15.59:0/4097974245 learned_addr learned my addr 172.21.15.59:0/4097974245 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fadea0cd700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade40f8d10 0x7fade410e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 -- 172.21.15.59:0/4097974245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade40f8d10 msgr2=0x7fade410e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade40f8d10 0x7fade410e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 -- 172.21.15.59:0/4097974245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade4102270 msgr2=0x7fade4108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:48.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade4102270 0x7fade4108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:48.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 -- 172.21.15.59:0/4097974245 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0005040 con 0x7fade40f9670 2024-02-18T00:36:48.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fade98cc700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade40f9670 0x7fade410e550 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 rx=0x7fadd400e990 tx=0x7fadd400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:48.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7faddb7fe700 10 client.?.objecter ms_handle_connect 0x7fade40f9670 2024-02-18T00:36:48.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7faddb7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:48.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7faddb7fe700 1 -- 172.21.15.59:0/4097974245 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400ee80 con 0x7fade40f9670 2024-02-18T00:36:48.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc0097c0 con 0x7fade40f9670 2024-02-18T00:36:48.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7faddb7fe700 1 -- 172.21.15.59:0/4097974245 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadd400f040 con 0x7fade40f9670 2024-02-18T00:36:48.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7faddb7fe700 1 -- 172.21.15.59:0/4097974245 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400c840 con 0x7fade40f9670 2024-02-18T00:36:48.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fadec331700 10 client.40919.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:48.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade4108fb0 con 0x7fade40f9670 2024-02-18T00:36:48.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.822+0000 7fadec331700 1 librados: init done 2024-02-18T00:36:48.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.823+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade405fa60 con 0x7fade40f9670 2024-02-18T00:36:48.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 1 -- 172.21.15.59:0/4097974245 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fadd400cc70 con 0x7fade40f9670 2024-02-18T00:36:48.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 1 --2- 172.21.15.59:0/4097974245 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadd0062c90 0x7fadd0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:48.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 1 -- 172.21.15.59:0/4097974245 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(330..330 src has 44..330) v4 ==== 8841+0+0 (secure 0 0 0) 0x7fadd407d060 con 0x7fade40f9670 2024-02-18T00:36:48.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 10 client.40919.objecter ms_dispatch 0x7fade4001550 osd_map(330..330 src has 44..330) v4 2024-02-18T00:36:48.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 3 client.40919.objecter handle_osd_map got epochs [330,330] > 0 2024-02-18T00:36:48.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 3 client.40919.objecter handle_osd_map decoding full epoch 330 2024-02-18T00:36:48.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7fadea0cd700 1 --2- 172.21.15.59:0/4097974245 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadd0062c90 0x7fadd0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:48.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 20 client.40919.objecter dump_active .. 0 homeless 2024-02-18T00:36:48.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7faddb7fe700 10 client.40919.objecter ms_handle_connect 0x7fadd0062c90 2024-02-18T00:36:48.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.824+0000 7fadea0cd700 1 --2- 172.21.15.59:0/4097974245 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadd0062c90 0x7fadd0065150 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 rx=0x7fade00047e0 tx=0x7fade0009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:48.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:48.828+0000 7faddb7fe700 1 -- 172.21.15.59:0/4097974245 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fadd4049f60 con 0x7fade40f9670 2024-02-18T00:36:48.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:49.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.011+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fade41072a0 con 0x7fade40f9670 2024-02-18T00:36:49.129 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:49.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:47.872341+0000 mgr.y (mgr.24454) 328 : cluster [DBG] pgmap v481: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:49.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:48.663576+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:49.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:48.663682+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:36:49.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:48.663748+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e330: 8 total, 8 up, 7 in 2024-02-18T00:36:49.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:48.668697+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:36:49.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:48.673238+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:36:49.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:49.013198+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.59:0/4097974245' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:49.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:49.013502+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:49.134 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:49.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:49.235 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:49.266 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:49.287 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:49.289 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:49.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:49.390 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:49.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:47.872341+0000 mgr.y (mgr.24454) 328 : cluster [DBG] pgmap v481: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:49.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:48.663576+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:49.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:48.663682+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:36:49.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:48.663748+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e330: 8 total, 8 up, 7 in 2024-02-18T00:36:49.417 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:48.668697+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:36:49.418 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:48.673238+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:36:49.418 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:49.013198+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.59:0/4097974245' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:49.418 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:49.013502+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:49.419 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:49.438 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:49.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:49.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:47.872341+0000 mgr.y (mgr.24454) 328 : cluster [DBG] pgmap v481: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:48.663576+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi059-82882-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:48.663682+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:48.663748+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e330: 8 total, 8 up, 7 in 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:48.668697+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:48.673238+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:49.013198+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.59:0/4097974245' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:49.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:49.013502+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:36:49.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:49.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.667+0000 7faddb7fe700 1 -- 172.21.15.59:0/4097974245 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v331) v1 ==== 101+0+0 (secure 0 0 0) 0x7fadd4050220 con 0x7fade40f9670 2024-02-18T00:36:49.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 10 librados: watch_flush enter 2024-02-18T00:36:49.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 10 librados: watch_flush exit 2024-02-18T00:36:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 20 client.40919.objecter shutdown clearing up homeless session... 2024-02-18T00:36:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 10 client.40919.objecter successfully canceled tick 2024-02-18T00:36:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadd0062c90 msgr2=0x7fadd0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 --2- 172.21.15.59:0/4097974245 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadd0062c90 0x7fadd0065150 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 rx=0x7fade00047e0 tx=0x7fade0009cb0).stop 2024-02-18T00:36:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade40f9670 msgr2=0x7fade410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:49.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade40f9670 0x7fade410e550 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 rx=0x7fadd400e990 tx=0x7fadd400b040).stop 2024-02-18T00:36:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 shutdown_connections 2024-02-18T00:36:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 --2- 172.21.15.59:0/4097974245 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadd0062c90 0x7fadd0065150 unknown :-1 s=CLOSED pgs=3072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade40f8d10 0x7fade410e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fade4102270 0x7fade4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 --2- 172.21.15.59:0/4097974245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fade40f9670 0x7fade410e550 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 >> 172.21.15.59:0/4097974245 conn(0x7fade40f2820 msgr2=0x7fade40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 shutdown_connections 2024-02-18T00:36:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.670+0000 7fadec331700 1 -- 172.21.15.59:0/4097974245 wait complete. 2024-02-18T00:36:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:49.671+0000 7fadec331700 1 librados: shutdown 2024-02-18T00:36:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.057293+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.057540+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.275646+0000 osd.4 (osd.4) 189 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.276970+0000 osd.4 (osd.4) 190 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.660380+0000 mon.a (mon.0) 2200 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:49.667039+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:49.667168+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.667224+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e331: 8 total, 8 up, 7 in 2024-02-18T00:36:50.180 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.675365+0000 osd.2 (osd.2) 143 : cluster [DBG] 2.c deep-scrub starts 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.680410+0000 osd.2 (osd.2) 144 : cluster [DBG] 2.c deep-scrub ok 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.057293+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.057540+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.275646+0000 osd.4 (osd.4) 189 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.276970+0000 osd.4 (osd.4) 190 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.660380+0000 mon.a (mon.0) 2200 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:49.667039+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:49.667168+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.667224+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e331: 8 total, 8 up, 7 in 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.675365+0000 osd.2 (osd.2) 143 : cluster [DBG] 2.c deep-scrub starts 2024-02-18T00:36:50.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.680410+0000 osd.2 (osd.2) 144 : cluster [DBG] 2.c deep-scrub ok 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.057293+0000 osd.2 (osd.2) 141 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.057540+0000 osd.2 (osd.2) 142 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.275646+0000 osd.4 (osd.4) 189 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.276970+0000 osd.4 (osd.4) 190 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.660380+0000 mon.a (mon.0) 2200 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:49.667039+0000 mon.a (mon.0) 2201 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:49.667168+0000 mon.a (mon.0) 2202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.667224+0000 mon.a (mon.0) 2203 : cluster [DBG] osdmap e331: 8 total, 8 up, 7 in 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.675365+0000 osd.2 (osd.2) 143 : cluster [DBG] 2.c deep-scrub starts 2024-02-18T00:36:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.680410+0000 osd.2 (osd.2) 144 : cluster [DBG] 2.c deep-scrub ok 2024-02-18T00:36:50.623 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:50.627 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:50.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:50.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:51.109 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:51.112 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:51.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:51.213 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:51.245 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:51.269 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:51.269 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:51.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:51.396 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.723697+0000 osd.5 (osd.5) 147 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:36:51.396 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.727153+0000 osd.5 (osd.5) 148 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.872918+0000 mgr.y (mgr.24454) 329 : cluster [DBG] pgmap v484: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.933756+0000 osd.1 (osd.1) 169 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.936640+0000 osd.1 (osd.1) 170 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.975079+0000 osd.3 (osd.3) 181 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:49.975952+0000 osd.3 (osd.3) 182 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:50.260926+0000 osd.4 (osd.4) 191 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:50.262597+0000 osd.4 (osd.4) 192 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:50.668186+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e332: 8 total, 8 up, 7 in 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:50.671220+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:51.397 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:50.675038+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.723697+0000 osd.5 (osd.5) 147 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.727153+0000 osd.5 (osd.5) 148 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.872918+0000 mgr.y (mgr.24454) 329 : cluster [DBG] pgmap v484: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.933756+0000 osd.1 (osd.1) 169 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.936640+0000 osd.1 (osd.1) 170 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.975079+0000 osd.3 (osd.3) 181 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:49.975952+0000 osd.3 (osd.3) 182 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:50.260926+0000 osd.4 (osd.4) 191 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:50.262597+0000 osd.4 (osd.4) 192 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:50.668186+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e332: 8 total, 8 up, 7 in 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:50.671220+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:51.398 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:50.675038+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:51.399 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:51.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:36:51.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:36:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.723697+0000 osd.5 (osd.5) 147 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:36:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.727153+0000 osd.5 (osd.5) 148 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:36:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.872918+0000 mgr.y (mgr.24454) 329 : cluster [DBG] pgmap v484: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.933756+0000 osd.1 (osd.1) 169 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.936640+0000 osd.1 (osd.1) 170 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.975079+0000 osd.3 (osd.3) 181 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:49.975952+0000 osd.3 (osd.3) 182 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:50.260926+0000 osd.4 (osd.4) 191 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:50.262597+0000 osd.4 (osd.4) 192 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:50.668186+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e332: 8 total, 8 up, 7 in 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:50.671220+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:51.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:50.675038+0000 mon.a (mon.0) 2205 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:51.949 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:51.952 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:51.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:52.053 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:52.085 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:52.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:52.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:52.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:50.757039+0000 osd.6 (osd.6) 165 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:36:52.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:50.759012+0000 osd.6 (osd.6) 166 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:36:52.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:50.969320+0000 osd.3 (osd.3) 183 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:36:52.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:50.970189+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:36:52.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:51.161659+0000 mgr.y (mgr.24454) 330 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:51.299253+0000 osd.4 (osd.4) 193 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:51.302911+0000 osd.4 (osd.4) 194 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:51.669902+0000 mon.a (mon.0) 2206 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:51.669976+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e333: 8 total, 8 up, 7 in 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:51.676268+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:51.676551+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:50.757039+0000 osd.6 (osd.6) 165 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:50.759012+0000 osd.6 (osd.6) 166 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:50.969320+0000 osd.3 (osd.3) 183 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:36:52.182 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:50.970189+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:36:52.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:51.161659+0000 mgr.y (mgr.24454) 330 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:52.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:51.299253+0000 osd.4 (osd.4) 193 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:36:52.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:51.302911+0000 osd.4 (osd.4) 194 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:36:52.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:51.669902+0000 mon.a (mon.0) 2206 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:52.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:51.669976+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e333: 8 total, 8 up, 7 in 2024-02-18T00:36:52.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:51.676268+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:52.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:51.676551+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:52.210 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:52.241 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:52.260 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:52.263 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:52.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:52.365 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:50.757039+0000 osd.6 (osd.6) 165 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:36:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:50.759012+0000 osd.6 (osd.6) 166 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:36:52.392 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:50.969320+0000 osd.3 (osd.3) 183 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:36:52.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:50.970189+0000 osd.3 (osd.3) 184 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:36:52.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:51.161659+0000 mgr.y (mgr.24454) 330 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:36:52.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:51.299253+0000 osd.4 (osd.4) 193 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:36:52.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:51.302911+0000 osd.4 (osd.4) 194 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:36:52.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:51.669902+0000 mon.a (mon.0) 2206 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:52.393 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:51.669976+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e333: 8 total, 8 up, 7 in 2024-02-18T00:36:52.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:51.676268+0000 mon.c (mon.2) 320 : audit [INF] from='client.? 172.21.15.59:0/1248208728' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:52.394 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:51.676551+0000 mon.a (mon.0) 2208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]: dispatch 2024-02-18T00:36:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:52.413 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:52.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:52.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:52.465 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:36:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:36:52] "GET /metrics HTTP/1.1" 200 188787 "" "Prometheus/2.33.4" 2024-02-18T00:36:52.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:52.549 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:52.999 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:36:53.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:53.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:36:53.103 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:53.132 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:53.151 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:53.154 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:53.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:51.873502+0000 mgr.y (mgr.24454) 331 : cluster [DBG] pgmap v487: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:36:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:51.947017+0000 osd.3 (osd.3) 185 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:36:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:51.948903+0000 osd.3 (osd.3) 186 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:36:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.340072+0000 osd.4 (osd.4) 195 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:36:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.343553+0000 osd.4 (osd.4) 196 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:36:53.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.371202+0000 osd.0 (osd.0) 161 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.371967+0000 osd.0 (osd.0) 162 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:52.675730+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.675800+0000 mon.a (mon.0) 2210 : cluster [DBG] osdmap e334: 8 total, 8 up, 7 in 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:52.695282+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:52.696191+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:52.696509+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi059-82882-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:51.873502+0000 mgr.y (mgr.24454) 331 : cluster [DBG] pgmap v487: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:51.947017+0000 osd.3 (osd.3) 185 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:51.948903+0000 osd.3 (osd.3) 186 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:36:53.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.340072+0000 osd.4 (osd.4) 195 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.343553+0000 osd.4 (osd.4) 196 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.371202+0000 osd.0 (osd.0) 161 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.371967+0000 osd.0 (osd.0) 162 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:52.675730+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.675800+0000 mon.a (mon.0) 2210 : cluster [DBG] osdmap e334: 8 total, 8 up, 7 in 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:52.695282+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:52.696191+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:53.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:52.696509+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi059-82882-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:53.256 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:53.283 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:53.301 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:53.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:53.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:53.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:53.437 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:51.873502+0000 mgr.y (mgr.24454) 331 : cluster [DBG] pgmap v487: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:51.947017+0000 osd.3 (osd.3) 185 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:51.948903+0000 osd.3 (osd.3) 186 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.340072+0000 osd.4 (osd.4) 195 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.343553+0000 osd.4 (osd.4) 196 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.371202+0000 osd.0 (osd.0) 161 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.371967+0000 osd.0 (osd.0) 162 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:52.675730+0000 mon.a (mon.0) 2209 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi059-82882-56"}]': finished 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.675800+0000 mon.a (mon.0) 2210 : cluster [DBG] osdmap e334: 8 total, 8 up, 7 in 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:52.695282+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:52.696191+0000 mon.a (mon.0) 2212 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:52.696509+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi059-82882-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:36:53.586 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:53.590 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:53.590 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:53.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:53.718 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:53.737 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:53.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:53.740 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:53.747 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:36:53.747 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24818.0:6 27.11 27:8dd16f86:::obj3:head [stat] snapc 0=[] ondisk+read+known_if_redirected e107)", 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 230.70642419800001, 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7480210029999999, 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24818", 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1844909298", 2024-02-18T00:36:53.748 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:53.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921757+0000", 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2109999999999997e-06 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921762+0000", 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9610000000000005e-06 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921770+0000", 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0260000000000002e-06 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921777+0000", 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4104000000000003e-05 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921821+0000", 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921827+0000", 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7478305119999999 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.669658+0000", 2024-02-18T00:36:53.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012113200000000001 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.669779+0000", 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7480210029999999 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:2061 17.4 17:238a5d4d:test-rados-api-smithi059-82949-16::big.copy:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:36:53.753 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 229.886218472, 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.082181671999999997, 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2061 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741963+0000", 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2389999999999999e-06 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741969+0000", 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0874e-05 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741980+0000", 2024-02-18T00:36:53.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1250000000000001e-06 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741985+0000", 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0633999999999998e-05 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.742025+0000", 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0579000000000001e-05 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.742116+0000", 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069955422000000003 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,4", 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812071+0000", 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011236616 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.823308+0000", 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026764999999999997 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.823576+0000", 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000464523 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824040+0000", 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.995e-05 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824100+0000", 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5300000000000003e-05 2024-02-18T00:36:53.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824146+0000", 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.082181671999999997 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2070 17.15 e108/104 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head v 108'1, mlcod=0'0)", 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 229.491359665, 2024-02-18T00:36:53.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070392237999999996, 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136822+0000", 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044622728 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181444+0000", 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5222e-05 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181469+0000", 2024-02-18T00:36:53.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0170000000000002e-06 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181478+0000", 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.237e-05 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181551+0000", 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9718999999999999e-05 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181571+0000", 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025600378999999999 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.207171+0000", 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3922999999999999e-05 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.207215+0000", 2024-02-18T00:36:53.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070392237999999996 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.25025.0:40 22.c e108/105 22:33eb54d8:::39.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 229.35912719699999, 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032619941, 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.765 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269053+0000", 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7693000000000002e-05 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269071+0000", 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9028000000000001e-05 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269090+0000", 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5519999999999999e-06 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269097+0000", 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0609999999999998e-05 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269148+0000", 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1269000000000001e-05 2024-02-18T00:36:53.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269189+0000", 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032451386999999998 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.301640+0000", 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4557000000000001e-05 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.301675+0000", 2024-02-18T00:36:53.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032619941 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(24.2 epoch 109/105 rep_tid 111 entries 109'2 (0'0) error 24:4061fbc0:test-rados-api-smithi059-82933-5::foo:head by client.24806.0:10 0.000000 -17 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 229.06029614799999, 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051717428000000003, 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:36:53.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0070000000000003e-06 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567895+0000", 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1107999999999999e-05 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567906+0000", 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0720000000000006e-06 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567915+0000", 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023667508 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.591582+0000", 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028021134999999999 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.619604+0000", 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051717428000000003 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.773 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24817.0:5 14.6 14:6787766c:test-rados-api-smithi059-82981-3::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e110)", 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:05.574197+0000", 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 228.05398515499999, 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024407564, 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24817", 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/521262486", 2024-02-18T00:36:53.774 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 5 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574197+0000", 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574197+0000", 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574195+0000", 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4889999999999995e-06 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574204+0000", 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1002000000000001e-05 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574215+0000", 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0979999999999996e-06 2024-02-18T00:36:53.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574222+0000", 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8387999999999998e-05 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574270+0000", 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014009 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574410+0000", 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0001639 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.574574+0000", 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022677427999999999 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.597252+0000", 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00037822300000000001 2024-02-18T00:36:53.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.597630+0000", 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00088009299999999998 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.598510+0000", 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1109000000000001e-05 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.598561+0000", 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3699999999999998e-05 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.598605+0000", 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024407564 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.26167.0:11 66.17 66:e89f79a2:test-rados-api-smithi059-82981-9::foo:head [write 0~128] snapc 6=[6] ondisk+write+known_if_redirected e131)", 2024-02-18T00:36:53.780 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 204.321994905, 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062117686999999998, 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.26167", 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306186+0000", 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6010000000000001e-06 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306195+0000", 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2559e-05 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306208+0000", 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6709999999999999e-06 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306215+0000", 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019698599999999999 2024-02-18T00:36:53.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306412+0000", 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8329999999999996e-06 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306419+0000", 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012823485000000001 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.319243+0000", 2024-02-18T00:36:53.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012152899999999999 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.319364+0000", 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010919998 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.330284+0000", 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011843823 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.342128+0000", 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2046000000000003e-05 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.342180+0000", 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026025063000000001 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368205+0000", 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5210000000000002e-05 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368260+0000", 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4642999999999999e-05 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368305+0000", 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062117686999999998 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.787 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.29425.0:2 73.6 73:602f83fe:::foo:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 204.320569232, 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.075318273000000005, 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.29425", 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/4028720008", 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307611+0000", 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0359999999999995e-06 2024-02-18T00:36:53.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307620+0000", 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1882000000000001e-05 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307632+0000", 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7689999999999996e-06 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307640+0000", 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030238405999999999 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.337878+0000", 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6550000000000004e-06 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.337886+0000", 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028640344000000002 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366526+0000", 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000118879 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366645+0000", 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014485400000000001 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366790+0000", 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003334025 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.370124+0000", 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6338999999999997e-05 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.370181+0000", 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012666067 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382847+0000", 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6785e-05 2024-02-18T00:36:53.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382893+0000", 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7939999999999999e-05 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382931+0000", 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.075318273000000005 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.26167.0:16 66.17 66:e89f79a2:test-rados-api-smithi059-82981-9::foo:head [write 128~128] snapc 7=[7,6] ondisk+write+known_if_redirected e132)", 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 203.308511818, 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059589835000000001, 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.26167", 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 16 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319668+0000", 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1216000000000001e-05 2024-02-18T00:36:53.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319679+0000", 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010448810000000001 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320724+0000", 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0662999999999999e-05 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320735+0000", 2024-02-18T00:36:53.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4954999999999998e-05 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320790+0000", 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320798+0000", 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021397759999999998 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342195+0000", 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4319000000000003e-05 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342290+0000", 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000201516 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342491+0000", 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0097928909999999997 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.352284+0000", 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013119486 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.365404+0000", 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013735841 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379139+0000", 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3324999999999993e-05 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379213+0000", 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7633999999999998e-05 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379260+0000", 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059589835000000001 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.801 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.7.0:212 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 199.10483161799999, 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083227508000000006, 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523349+0000", 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064971570000000006 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588321+0000", 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2592999999999999e-05 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588343+0000", 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4349999999999997e-06 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588352+0000", 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000863093 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.589215+0000", 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6400000000000001e-05 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.589241+0000", 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017286283 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.606528+0000", 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0519000000000002e-05 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.606578+0000", 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083227508000000006 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.805 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.7.0:213 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.588417+0000", 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 199.039765171, 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025918482999999999, 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588417+0000", 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588417+0000", 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967296 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588417+0000", 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011778965000000001 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600196+0000", 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1806e-05 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600228+0000", 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1321000000000001e-05 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600239+0000", 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041404400000000002 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600653+0000", 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9181999999999999e-05 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600692+0000", 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013593381 2024-02-18T00:36:53.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.614286+0000", 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9991000000000003e-05 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.614336+0000", 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025918482999999999 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.7.0:214 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 199.027923305, 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047186580999999998, 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:36:53.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600257+0000", 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024532449000000001 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624790+0000", 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9515000000000001e-05 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624819+0000", 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0849000000000001e-05 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624830+0000", 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025507999999999998 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.625085+0000", 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4088e-05 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.625109+0000", 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022278747000000002 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.647388+0000", 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7408999999999999e-05 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.647446+0000", 2024-02-18T00:36:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047186580999999998 2024-02-18T00:36:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42808.0:2 99.6 99:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e156)", 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.04680789099999, 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041101081999999997, 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42808", 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3195038165", 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:53.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581373+0000", 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4339999999999992e-06 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581381+0000", 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8351000000000001e-05 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581399+0000", 2024-02-18T00:36:53.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3760000000000008e-06 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581408+0000", 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2623e-05 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581470+0000", 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.43e-06 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581478+0000", 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013746548000000001 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.595224+0000", 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2150000000000004e-06 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.595232+0000", 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021220474 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616452+0000", 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106791 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616559+0000", 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000140035 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,4", 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616699+0000", 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001518248 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.618217+0000", 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015478130000000001 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.619765+0000", 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026234829999999998 2024-02-18T00:36:53.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622388+0000", 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9582000000000001e-05 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622438+0000", 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7565999999999999e-05 2024-02-18T00:36:53.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622475+0000", 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041101081999999997 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.40966.0:11 94.5s0 94:a3e1d3f7:test-rados-api-smithi059-82973-16::foo:head [write 0~8192] snapc 6=[6] ondisk+write+known_if_redirected e156)", 2024-02-18T00:36:53.827 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:54.590355+0000", 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 179.03782710499999, 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030761625000000001, 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.40966", 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/4226878741", 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11 2024-02-18T00:36:53.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590355+0000", 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590355+0000", 2024-02-18T00:36:53.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590353+0000", 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3351000000000001e-05 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590377+0000", 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0685e-05 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590387+0000", 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9300000000000003e-06 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590395+0000", 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8125999999999999e-05 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590454+0000", 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8790000000000003e-06 2024-02-18T00:36:53.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.590461+0000", 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023922219000000002 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.614384+0000", 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105207 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.614489+0000", 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026637999999999998 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.614755+0000", 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00035821699999999998 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.615113+0000", 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0059641429999999999 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.621078+0000", 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9252999999999998e-05 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.621117+0000", 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030761625000000001 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.834 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42916.0:1 114.5s0 114:a1464ac0:test-rados-api-smithi059-82981-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e176)", 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 158.235984462, 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996, 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42916", 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2278224535", 2024-02-18T00:36:53.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392195+0000", 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1378e-05 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392206+0000", 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3589e-05 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392220+0000", 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1283e-05 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392231+0000", 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0343999999999997e-05 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392292+0000", 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3749999999999999e-06 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392295+0000", 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.58736239099999998 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979657+0000", 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103571 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979761+0000", 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000375254 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.980136+0000", 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061387799999999999 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.986275+0000", 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032303779999999999 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989505+0000", 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7123000000000003e-05 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989552+0000", 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42910.0:2 115.13 115:ce638d0e:test-rados-api-smithi059-82949-26::foo:head [writefull 0~8388608] snapc 0=[] ondisk+write+known_if_redirected e178)", 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 156.64823239, 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003, 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42910", 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/955395839", 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979948+0000", 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0051422639999999997 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985090+0000", 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1109e-05 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985101+0000", 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2439999999999993e-06 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985110+0000", 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032139199999999999 2024-02-18T00:36:53.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985431+0000", 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115649 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985547+0000", 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014544800000000001 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,3", 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985692+0000", 2024-02-18T00:36:53.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016096787000000001 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.001789+0000", 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016631943 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018421+0000", 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016582200000000001 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018587+0000", 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4758999999999994e-05 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018651+0000", 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8247000000000002e-05 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018690+0000", 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003 2024-02-18T00:36:53.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:588 62.9 e205/123 62:9441d9b2:test-rados-api-smithi059-82957-14::foo82:4 v 205'35, mlcod=193'31)", 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 129.957214725, 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999, 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670966+0000", 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2614e-05 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670979+0000", 2024-02-18T00:36:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1616000000000001e-05 2024-02-18T00:36:53.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671000+0000", 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9030000000000004e-06 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671008+0000", 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3940000000000001e-05 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671072+0000", 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0826999999999999e-05 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671093+0000", 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034117472000000003 2024-02-18T00:36:53.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705210+0000", 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5481000000000005e-05 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705276+0000", 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:589 62.9 e205/123 62:93987215:test-rados-api-smithi059-82957-14::foo76:4 v 205'37, mlcod=193'31)", 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 129.95672892600001, 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001, 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.109e-06 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671460+0000", 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.027e-05 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671470+0000", 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.993e-06 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671476+0000", 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2646999999999999e-05 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671529+0000", 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0522000000000001e-05 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671549+0000", 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036671306000000001 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708221+0000", 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8853000000000002e-05 2024-02-18T00:36:53.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708259+0000", 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30969.0:1 179.5s0 179:b3610dda:::oid_RoundTripPP2_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.990532948000002, 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001, 2024-02-18T00:36:53.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30969", 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3697991996", 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637648+0000", 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2819999999999999e-06 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637655+0000", 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9309999999999997e-06 2024-02-18T00:36:53.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637664+0000", 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.153e-06 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637671+0000", 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2398000000000003e-05 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637713+0000", 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2790000000000001e-06 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637716+0000", 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0156700160000001 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653386+0000", 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131438 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653517+0000", 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043157100000000001 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653949+0000", 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031255889999999998 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657074+0000", 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2801999999999999e-05 2024-02-18T00:36:53.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657147+0000", 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000247073 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657394+0000", 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30966.0:1 178.6s0 178:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 49.989814518999999, 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001, 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30966", 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3784803143", 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:36:53.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638367+0000", 2024-02-18T00:36:53.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.622e-06 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638372+0000", 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7880000000000001e-06 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638381+0000", 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7909999999999999e-06 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638388+0000", 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.171e-05 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638420+0000", 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6700000000000001e-06 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638422+0000", 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0205109640000001 2024-02-18T00:36:53.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.658932+0000", 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014527400000000001 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659078+0000", 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043918800000000002 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659517+0000", 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00089794300000000001 2024-02-18T00:36:53.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660415+0000", 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9174e-05 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660474+0000", 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024629900000000001 2024-02-18T00:36:53.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660720+0000", 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:36:53.871 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:36:53.872 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:36:53.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:36:53.898 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:53.917 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:36:53.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:53.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:36:53.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:36:54.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:54.048 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:54.122 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:54.124 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:54.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.695+0000 7f09b001d700 1 -- 172.21.15.59:0/3553643168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 msgr2=0x7f09a80f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.695+0000 7f09b001d700 1 --2- 172.21.15.59:0/3553643168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 0x7f09a80f7d40 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 rx=0x7f099c0047b0 tx=0x7f099c009cb0).stop 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.695+0000 7f09b001d700 1 -- 172.21.15.59:0/3553643168 shutdown_connections 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.695+0000 7f09b001d700 1 --2- 172.21.15.59:0/3553643168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a80f8420 0x7f09a8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.695+0000 7f09b001d700 1 --2- 172.21.15.59:0/3553643168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a80fdbb0 0x7f09a80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.695+0000 7f09b001d700 1 --2- 172.21.15.59:0/3553643168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 0x7f09a80f7d40 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.695+0000 7f09b001d700 1 -- 172.21.15.59:0/3553643168 >> 172.21.15.59:0/3553643168 conn(0x7f09a80f3800 msgr2=0x7f09a80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 -- 172.21.15.59:0/3553643168 shutdown_connections 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 -- 172.21.15.59:0/3553643168 wait complete. 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 librados: starting msgr at 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 librados: starting objecter 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 Processor -- start 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 -- start start 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 librados: setting wanted keys 2024-02-18T00:36:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 librados: calling monclient init 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a80f8420 0x7f09a810e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a80fdbb0 0x7f09a810e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 0x7f09a8108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8108730 con 0x7f09a80fdbb0 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81088a0 con 0x7f09a80f8420 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09b001d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a8108a60 con 0x7f09a80fe510 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.696+0000 7f09ad5b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a80fdbb0 0x7f09a810e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ae5ba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 0x7f09a8108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ad5b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a80fdbb0 0x7f09a810e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55340/0 (socket says 172.21.15.59:55340) 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09addb9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a80f8420 0x7f09a810e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ad5b8700 1 -- 172.21.15.59:0/2682371979 learned_addr learned my addr 172.21.15.59:0/2682371979 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ae5ba700 1 -- 172.21.15.59:0/2682371979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a80f8420 msgr2=0x7f09a810e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ae5ba700 1 --2- 172.21.15.59:0/2682371979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a80f8420 0x7f09a810e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ae5ba700 1 -- 172.21.15.59:0/2682371979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a80fdbb0 msgr2=0x7f09a810e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ae5ba700 1 --2- 172.21.15.59:0/2682371979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a80fdbb0 0x7f09a810e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ae5ba700 1 -- 172.21.15.59:0/2682371979 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a4005040 con 0x7f09a80fe510 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ad5b8700 1 --2- 172.21.15.59:0/2682371979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09a80fdbb0 0x7f09a810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09ae5ba700 1 --2- 172.21.15.59:0/2682371979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 0x7f09a8108240 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 rx=0x7f09a000db00 tx=0x7f09a000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f099affd700 10 client.?.objecter ms_handle_connect 0x7f09a80fe510 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f099affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a000dee0 con 0x7f09a80fe510 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c005040 con 0x7f09a80fe510 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09a0014070 con 0x7f09a80fe510 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0007ae0 con 0x7f09a80fe510 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09b001d700 10 client.40931.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a81090c0 con 0x7f09a80fe510 2024-02-18T00:36:54.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.697+0000 7f09b001d700 1 librados: init done 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.698+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f09a81070d0 con 0x7f09a80fe510 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f09a000b5a0 con 0x7f09a80fe510 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0994062c90 0x7f0994065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(335..335 src has 44..335) v4 ==== 8341+0+0 (secure 0 0 0) 0x7f09a007d260 con 0x7f09a80fe510 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 osd_map(335..335 src has 44..335) v4 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 3 client.40931.objecter handle_osd_map got epochs [335,335] > 0 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 3 client.40931.objecter handle_osd_map decoding full epoch 335 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0994062c90 0x7f0994065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 20 client.40931.objecter dump_active .. 0 homeless 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.699+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v335) v1 ==== 54+0+16 (secure 0 0 0) 0x7f09a0084070 con 0x7f09a80fe510 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f0994062c90 2024-02-18T00:36:54.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0994062c90 0x7f0994065150 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 rx=0x7f09a4014b10 tx=0x7f09a400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 10 client.40931.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 1 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6801/3700238977 conn(0x7f09a80fee00 0x7f09a80ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 1 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 0 1 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8109720 osd=0 4 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 10 client.40931.objecter _send_command 1 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a806cfb0 con 0x7f09a80fee00 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09ad5b8700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6801/3700238977 conn(0x7f09a80fee00 0x7f09a80ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a80fee00 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.700+0000 7f09ad5b8700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6801/3700238977 conn(0x7f09a80fee00 0x7f09a80ff1e0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.701+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a806cfb0 con 0x7f09a80fee00 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.701+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.701+0000 7f099affd700 10 client.40931.objecter _finish_command 1 = system:0 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.701+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8109720 osd=0 4 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.701+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 0 1 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 10 client.40931.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 2 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 2 2024-02-18T00:36:54.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8109720 osd=0 3 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 0 2 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8109720 osd=0 4 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 10 client.40931.objecter _send_command 2 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.717+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a80ff910 con 0x7f09a80fee00 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f099c019040 con 0x7f09a80fee00 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f099affd700 10 client.40931.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8109720 osd=0 4 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 0 2 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 10 client.40931.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 3 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6805/26279110 conn(0x7f09a80598f0 0x7f09a8059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 3 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 1 3 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8028540 osd=1 4 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 10 client.40931.objecter _send_command 3 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.718+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a80286b0 con 0x7f09a80598f0 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.719+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6805/26279110 conn(0x7f09a80598f0 0x7f09a8059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.719+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6805/26279110 conn(0x7f09a80598f0 0x7f09a8059cd0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.719+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a80598f0 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:36:54.225 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:36:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.720+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a80286b0 con 0x7f09a80598f0 2024-02-18T00:36:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.720+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:36:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.720+0000 7f099affd700 10 client.40931.objecter _finish_command 3 = system:0 2024-02-18T00:36:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.720+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8028540 osd=1 4 2024-02-18T00:36:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.720+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 1 3 2024-02-18T00:36:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 10 client.40931.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 4 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 4 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8028540 osd=1 3 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 1 4 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8028540 osd=1 4 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 10 client.40931.objecter _send_command 4 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.734+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a8059360 con 0x7f09a80598f0 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f09a8059360 con 0x7f09a80598f0 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f099affd700 10 client.40931.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8028540 osd=1 4 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 1 4 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 10 client.40931.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 5 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6809/1153193637 conn(0x7f09a80280a0 0x7f09a811a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 5 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 2 5 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8059770 osd=2 4 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 10 client.40931.objecter _send_command 5 2024-02-18T00:36:54.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a811ccc0 con 0x7f09a80280a0 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.736+0000 7f09ad5b8700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6809/1153193637 conn(0x7f09a80280a0 0x7f09a811a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.737+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a80280a0 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.737+0000 7f09ad5b8700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6809/1153193637 conn(0x7f09a80280a0 0x7f09a811a070 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.737+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a811ccc0 con 0x7f09a80280a0 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.737+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.737+0000 7f099affd700 10 client.40931.objecter _finish_command 5 = system:0 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.737+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8059770 osd=2 4 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.737+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 2 5 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 10 client.40931.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 6 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 6 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8059770 osd=2 3 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 2 6 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8059770 osd=2 4 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 10 client.40931.objecter _send_command 6 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.752+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a8027660 con 0x7f09a80280a0 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.753+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f09a8027660 con 0x7f09a80280a0 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.753+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.753+0000 7f099affd700 10 client.40931.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.753+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8059770 osd=2 4 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.753+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 2 6 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 10 client.40931.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 7 2024-02-18T00:36:54.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6813/1805700964 conn(0x7f09a811b6e0 0x7f09a811bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 7 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 3 7 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a81176b0 osd=3 4 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 10 client.40931.objecter _send_command 7 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a8117d30 con 0x7f09a811b6e0 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09ae5ba700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6813/1805700964 conn(0x7f09a811b6e0 0x7f09a811bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a811b6e0 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.754+0000 7f09ae5ba700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6813/1805700964 conn(0x7f09a811b6e0 0x7f09a811bb20 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.755+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a8117d30 con 0x7f09a811b6e0 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.755+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.755+0000 7f099affd700 10 client.40931.objecter _finish_command 7 = system:0 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.755+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a81176b0 osd=3 4 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.755+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 3 7 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 10 client.40931.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 8 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 8 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a81176b0 osd=3 3 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 3 8 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a81176b0 osd=3 4 2024-02-18T00:36:54.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 10 client.40931.objecter _send_command 8 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.769+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a811c7e0 con 0x7f09a811b6e0 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.770+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f09a811c7e0 con 0x7f09a811b6e0 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.770+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.770+0000 7f099affd700 10 client.40931.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.770+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a81176b0 osd=3 4 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.770+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 3 8 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 10 client.40931.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 9 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6800/2044387056 conn(0x7f09a811ee40 0x7f09a8121230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 9 2024-02-18T00:36:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 4 9 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c010 osd=4 4 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 10 client.40931.objecter _send_command 9 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a811c6c0 con 0x7f09a811ee40 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6800/2044387056 conn(0x7f09a811ee40 0x7f09a8121230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6800/2044387056 conn(0x7f09a811ee40 0x7f09a8121230 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.771+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a811ee40 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.772+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a811c6c0 con 0x7f09a811ee40 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.772+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.772+0000 7f099affd700 10 client.40931.objecter _finish_command 9 = system:0 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.772+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a811c010 osd=4 4 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.772+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 4 9 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 10 client.40931.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 10 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 10 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a811c010 osd=4 3 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 4 10 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c010 osd=4 4 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 10 client.40931.objecter _send_command 10 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.787+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a811c6c0 con 0x7f09a811ee40 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f09a811c6c0 con 0x7f09a811ee40 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f099affd700 10 client.40931.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a811c010 osd=4 4 2024-02-18T00:36:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 4 10 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 10 client.40931.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 11 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6804/2170432263 conn(0x7f09a8121c30 0x7f09a8124060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 11 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 5 11 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c6c0 osd=5 4 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 10 client.40931.objecter _send_command 11 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a8126e30 con 0x7f09a8121c30 2024-02-18T00:36:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09ad5b8700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6804/2170432263 conn(0x7f09a8121c30 0x7f09a8124060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f09ad5b8700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6804/2170432263 conn(0x7f09a8121c30 0x7f09a8124060 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.789+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a8121c30 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.790+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a8126e30 con 0x7f09a8121c30 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.790+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.790+0000 7f099affd700 10 client.40931.objecter _finish_command 11 = system:0 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.790+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a811c6c0 osd=5 4 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.790+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 5 11 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 10 client.40931.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 12 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 12 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a811c6c0 osd=5 3 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 5 12 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c6c0 osd=5 4 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 10 client.40931.objecter _send_command 12 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.804+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a8126e30 con 0x7f09a8121c30 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f09a8126e30 con 0x7f09a8121c30 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f099affd700 10 client.40931.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a811c6c0 osd=5 4 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 5 12 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 10 client.40931.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 13 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6808/3356009390 conn(0x7f09a8126130 0x7f09a81265b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 13 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 6 13 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126e30 osd=6 4 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 10 client.40931.objecter _send_command 13 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a8126c20 con 0x7f09a8126130 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.806+0000 7f09ae5ba700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6808/3356009390 conn(0x7f09a8126130 0x7f09a81265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.807+0000 7f09ae5ba700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6808/3356009390 conn(0x7f09a8126130 0x7f09a81265b0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.807+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a8126130 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.808+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a8126c20 con 0x7f09a8126130 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.808+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:36:54.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.808+0000 7f099affd700 10 client.40931.objecter _finish_command 13 = system:0 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.808+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8126e30 osd=6 4 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.808+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 6 13 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 10 client.40931.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 14 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 14 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8126e30 osd=6 3 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 6 14 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126e30 osd=6 4 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 10 client.40931.objecter _send_command 14 2024-02-18T00:36:54.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.822+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a8126c20 con 0x7f09a8126130 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f09a8126c20 con 0x7f09a8126130 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f099affd700 10 client.40931.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8126e30 osd=6 4 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 6 14 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 10 client.40931.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 15 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6812/53025171 conn(0x7f09a81295a0 0x7f09a8129a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 15 2024-02-18T00:36:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 7 15 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126c20 osd=7 4 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 10 client.40931.objecter _send_command 15 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a812a050 con 0x7f09a81295a0 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6812/53025171 conn(0x7f09a81295a0 0x7f09a8129a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f09addb9700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6812/53025171 conn(0x7f09a81295a0 0x7f09a8129a00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.824+0000 7f099affd700 10 client.40931.objecter ms_handle_connect 0x7f09a81295a0 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.825+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09a812a050 con 0x7f09a81295a0 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.825+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.825+0000 7f099affd700 10 client.40931.objecter _finish_command 15 = system:0 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.825+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8126c20 osd=7 4 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.825+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 7 15 2024-02-18T00:36:54.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 10 client.40931.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign -1 16 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 20 client.40931.objecter _get_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 15 client.40931.objecter _session_command_op_remove -1 16 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 20 client.40931.objecter get_session s=0x7f09a8126c20 osd=7 3 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 15 client.40931.objecter _session_command_op_assign 7 16 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126c20 osd=7 4 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 10 client.40931.objecter _send_command 16 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.839+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f09a812a050 con 0x7f09a81295a0 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f099affd700 1 -- 172.21.15.59:0/2682371979 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f09a812a050 con 0x7f09a81295a0 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f099affd700 10 client.40931.objecter ms_dispatch 0x7f09a8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:36:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f099affd700 10 client.40931.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f099affd700 20 client.40931.objecter put_session s=0x7f09a8126c20 osd=7 4 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f099affd700 15 client.40931.objecter _session_command_op_remove 7 16 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 librados: watch_flush enter 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 librados: watch_flush exit 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.0 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6801/3700238977 conn(0x7f09a80fee00 msgr2=0x7f09a80ff1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6801/3700238977 conn(0x7f09a80fee00 0x7f09a80ff1e0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8109720 osd=0 1 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.1 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6805/26279110 conn(0x7f09a80598f0 msgr2=0x7f09a8059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6805/26279110 conn(0x7f09a80598f0 0x7f09a8059cd0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8028540 osd=1 1 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.2 2024-02-18T00:36:54.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6809/1153193637 conn(0x7f09a80280a0 msgr2=0x7f09a811a070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6809/1153193637 conn(0x7f09a80280a0 0x7f09a811a070 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8059770 osd=2 1 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.3 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6813/1805700964 conn(0x7f09a811b6e0 msgr2=0x7f09a811bb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6813/1805700964 conn(0x7f09a811b6e0 0x7f09a811bb20 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09ae5ba700 1 -- 172.21.15.59:0/2682371979 reap_dead start 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a81176b0 osd=3 1 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.4 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6800/2044387056 conn(0x7f09a811ee40 msgr2=0x7f09a8121230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6800/2044387056 conn(0x7f09a811ee40 0x7f09a8121230 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c010 osd=4 1 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.5 2024-02-18T00:36:54.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6804/2170432263 conn(0x7f09a8121c30 msgr2=0x7f09a8124060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09ae5ba700 1 -- 172.21.15.59:0/2682371979 reap_dead start 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6804/2170432263 conn(0x7f09a8121c30 0x7f09a8124060 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a811c6c0 osd=5 1 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.6 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6808/3356009390 conn(0x7f09a8126130 msgr2=0x7f09a81265b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6808/3356009390 conn(0x7f09a8126130 0x7f09a81265b0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126e30 osd=6 1 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter close_session for osd.7 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6812/53025171 conn(0x7f09a81295a0 msgr2=0x7f09a8129a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.137:6812/53025171 conn(0x7f09a81295a0 0x7f09a8129a00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter put_session s=0x7f09a8126c20 osd=7 1 2024-02-18T00:36:54.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 20 client.40931.objecter shutdown clearing up homeless session... 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 10 client.40931.objecter successfully canceled tick 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0994062c90 msgr2=0x7f0994065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0994062c90 0x7f0994065150 secure :-1 s=READY pgs=3074 cs=0 l=1 rev1=1 rx=0x7f09a4014b10 tx=0x7f09a400b040).stop 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 msgr2=0x7f09a8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 --2- 172.21.15.59:0/2682371979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09a80fe510 0x7f09a8108240 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 rx=0x7f09a000db00 tx=0x7f09a000b040).stop 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09ae5ba700 1 -- 172.21.15.59:0/2682371979 reap_dead start 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 shutdown_connections 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 >> 172.21.15.59:0/2682371979 conn(0x7f09a80f3800 msgr2=0x7f09a80f52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 shutdown_connections 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 -- 172.21.15.59:0/2682371979 wait complete. 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:53.842+0000 7f09b001d700 1 librados: shutdown 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.247 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:36:54.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:54.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.758028+0000 osd.6 (osd.6) 167 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.758900+0000 osd.6 (osd.6) 168 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.926490+0000 osd.3 (osd.3) 187 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:52.928067+0000 osd.3 (osd.3) 188 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.342976+0000 osd.4 (osd.4) 197 : cluster [DBG] 2.26 deep-scrub starts 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.345868+0000 osd.4 (osd.4) 198 : cluster [DBG] 2.26 deep-scrub ok 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.400238+0000 osd.0 (osd.0) 163 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.401815+0000 osd.0 (osd.0) 164 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:53.643837+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi059-82882-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.643921+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e335: 8 total, 8 up, 7 in 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:53.646555+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi059-82882-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.647729+0000 osd.2 (osd.2) 145 : cluster [DBG] 62.6 scrub starts 2024-02-18T00:36:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.649672+0000 osd.2 (osd.2) 146 : cluster [DBG] 62.6 scrub ok 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:53.700425+0000 mon.c (mon.2) 321 : audit [DBG] from='client.? 172.21.15.59:0/2682371979' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.758028+0000 osd.6 (osd.6) 167 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.758900+0000 osd.6 (osd.6) 168 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.926490+0000 osd.3 (osd.3) 187 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:52.928067+0000 osd.3 (osd.3) 188 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.342976+0000 osd.4 (osd.4) 197 : cluster [DBG] 2.26 deep-scrub starts 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.345868+0000 osd.4 (osd.4) 198 : cluster [DBG] 2.26 deep-scrub ok 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.400238+0000 osd.0 (osd.0) 163 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.401815+0000 osd.0 (osd.0) 164 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:36:54.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:53.643837+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi059-82882-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:54.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.643921+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e335: 8 total, 8 up, 7 in 2024-02-18T00:36:54.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:53.646555+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi059-82882-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:54.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.647729+0000 osd.2 (osd.2) 145 : cluster [DBG] 62.6 scrub starts 2024-02-18T00:36:54.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.649672+0000 osd.2 (osd.2) 146 : cluster [DBG] 62.6 scrub ok 2024-02-18T00:36:54.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:53.700425+0000 mon.c (mon.2) 321 : audit [DBG] from='client.? 172.21.15.59:0/2682371979' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.758028+0000 osd.6 (osd.6) 167 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.758900+0000 osd.6 (osd.6) 168 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.926490+0000 osd.3 (osd.3) 187 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:52.928067+0000 osd.3 (osd.3) 188 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.342976+0000 osd.4 (osd.4) 197 : cluster [DBG] 2.26 deep-scrub starts 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.345868+0000 osd.4 (osd.4) 198 : cluster [DBG] 2.26 deep-scrub ok 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.400238+0000 osd.0 (osd.0) 163 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:36:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.401815+0000 osd.0 (osd.0) 164 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:36:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:53.643837+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi059-82882-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:36:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.643921+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e335: 8 total, 8 up, 7 in 2024-02-18T00:36:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:53.646555+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi059-82882-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.647729+0000 osd.2 (osd.2) 145 : cluster [DBG] 62.6 scrub starts 2024-02-18T00:36:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.649672+0000 osd.2 (osd.2) 146 : cluster [DBG] 62.6 scrub ok 2024-02-18T00:36:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:53.700425+0000 mon.c (mon.2) 321 : audit [DBG] from='client.? 172.21.15.59:0/2682371979' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:36:54.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.226+0000 7fb83161c700 1 -- 172.21.15.59:0/1146510983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 msgr2=0x7fb82c101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.226+0000 7fb83161c700 1 --2- 172.21.15.59:0/1146510983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c101d20 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 rx=0x7fb820004870 tx=0x7fb820009cb0).stop 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 -- 172.21.15.59:0/1146510983 shutdown_connections 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 --2- 172.21.15.59:0/1146510983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb82c0f8cb0 0x7fb82c0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 --2- 172.21.15.59:0/1146510983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb82c102210 0x7fb82c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 --2- 172.21.15.59:0/1146510983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c101d20 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 -- 172.21.15.59:0/1146510983 >> 172.21.15.59:0/1146510983 conn(0x7fb82c0f2820 msgr2=0x7fb82c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 -- 172.21.15.59:0/1146510983 shutdown_connections 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 -- 172.21.15.59:0/1146510983 wait complete. 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 librados: starting msgr at 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 librados: starting objecter 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 Processor -- start 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 -- start start 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 librados: setting wanted keys 2024-02-18T00:36:54.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.227+0000 7fb83161c700 1 librados: calling monclient init 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb82c0f8cb0 0x7fb82c0f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c0f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb82c102210 0x7fb82c0f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c1167a0 con 0x7fb82c102210 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c116910 con 0x7fb82c0f8cb0 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c116ad0 con 0x7fb82c0f9610 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c0f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb82c102210 0x7fb82c0f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c0f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:45684/0 (socket says 172.21.15.59:45684) 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 -- 172.21.15.59:0/2525928967 learned_addr learned my addr 172.21.15.59:0/2525928967 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82affd700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb82c0f8cb0 0x7fb82c0f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 -- 172.21.15.59:0/2525928967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb82c0f8cb0 msgr2=0x7fb82c0f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb82c0f8cb0 0x7fb82c0f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 -- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb82c102210 msgr2=0x7fb82c0f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb82c102210 0x7fb82c0f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 -- 172.21.15.59:0/2525928967 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8240097c0 con 0x7fb82c0f9610 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82b7fe700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb82c102210 0x7fb82c0f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb82a7fc700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c0f7150 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 rx=0x7fb8200043a0 tx=0x7fb82000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb813fff700 10 client.?.objecter ms_handle_connect 0x7fb82c0f9610 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb813fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb813fff700 1 -- 172.21.15.59:0/2525928967 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82000e070 con 0x7fb82c0f9610 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb820005040 con 0x7fb82c0f9610 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb813fff700 1 -- 172.21.15.59:0/2525928967 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb820009710 con 0x7fb82c0f9610 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb813fff700 1 -- 172.21.15.59:0/2525928967 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820019070 con 0x7fb82c0f9610 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 10 client.40937.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c116fc0 con 0x7fb82c0f9610 2024-02-18T00:36:54.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.228+0000 7fb83161c700 1 librados: init done 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.229+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c05fa60 con 0x7fb82c0f9610 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 1 -- 172.21.15.59:0/2525928967 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb8200283f0 con 0x7fb82c0f9610 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 1 --2- 172.21.15.59:0/2525928967 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb814062c90 0x7fb814065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 1 -- 172.21.15.59:0/2525928967 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(335..335 src has 44..335) v4 ==== 8341+0+0 (secure 0 0 0) 0x7fb82007dcc0 con 0x7fb82c0f9610 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 10 client.40937.objecter ms_dispatch 0x7fb82c001550 osd_map(335..335 src has 44..335) v4 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 3 client.40937.objecter handle_osd_map got epochs [335,335] > 0 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 3 client.40937.objecter handle_osd_map decoding full epoch 335 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb82affd700 1 --2- 172.21.15.59:0/2525928967 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb814062c90 0x7fb814065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 20 client.40937.objecter dump_active .. 0 homeless 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb813fff700 10 client.40937.objecter ms_handle_connect 0x7fb814062c90 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.230+0000 7fb82affd700 1 --2- 172.21.15.59:0/2525928967 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb814062c90 0x7fb814065150 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 rx=0x7fb81c004790 tx=0x7fb81c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:36:54.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.234+0000 7fb813fff700 1 -- 172.21.15.59:0/2525928967 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb8200511c0 con 0x7fb82c0f9610 2024-02-18T00:36:54.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.480+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb82c0f8140 con 0x7fb82c0f9610 2024-02-18T00:36:54.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.481+0000 7fb813fff700 1 -- 172.21.15.59:0/2525928967 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2353 (secure 0 0 0) 0x7fb820050db0 con 0x7fb82c0f9610 2024-02-18T00:36:54.581 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:36:54.581 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":687,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":335,"num_osds":8,"num_up_osds":8,"osd_up_since":1708216146,"num_in_osds":7,"osd_in_since":1708216427,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":238}],"num_pgs":238,"num_pools":9,"num_objects":349,"data_bytes":4007022,"bytes_used":2206380032,"bytes_avail":669758869504,"bytes_total":671965249536,"read_bytes_sec":1279,"write_bytes_sec":511,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.59:8443/","prometheus":"http://172.21.15.59:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-18T00:33:31.796486+0000","services":{"rgw":{"daemons":{"summary":"","24347":{"start_epoch":12,"start_stamp":"2024-02-18T00:29:22.198518+0000","gid":24347,"addr":"172.21.15.59:0/2522576766","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)","ceph_version_short":"16.2.14-548-g81bd20d6","container_hostname":"smithi059","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi059","id":"foo.a","kernel_description":"#1 SMP Mon Feb 5 18:34:56 UTC 2024","kernel_version":"4.18.0-539.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543852","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"e67bc818-d879-487a-b572-f16f51d463f9","zone_name":"default","zonegroup_id":"1a312463-4a64-495f-95ab-5ad353fdc6fb","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 10 librados: watch_flush enter 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 10 librados: watch_flush exit 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 20 client.40937.objecter shutdown clearing up homeless session... 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 10 client.40937.objecter successfully canceled tick 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb814062c90 msgr2=0x7fb814065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 --2- 172.21.15.59:0/2525928967 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb814062c90 0x7fb814065150 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 rx=0x7fb81c004790 tx=0x7fb81c00b040).stop 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 msgr2=0x7fb82c0f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c0f7150 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 rx=0x7fb8200043a0 tx=0x7fb82000b040).stop 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 shutdown_connections 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 --2- 172.21.15.59:0/2525928967 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb814062c90 0x7fb814065150 unknown :-1 s=CLOSED pgs=3075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb82c0f8cb0 0x7fb82c0f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb82c102210 0x7fb82c0f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 --2- 172.21.15.59:0/2525928967 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb82c0f9610 0x7fb82c0f7150 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:36:54.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 >> 172.21.15.59:0/2525928967 conn(0x7fb82c0f2820 msgr2=0x7fb82c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:36:54.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 shutdown_connections 2024-02-18T00:36:54.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 -- 172.21.15.59:0/2525928967 wait complete. 2024-02-18T00:36:54.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:36:54.483+0000 7fb83161c700 1 librados: shutdown 2024-02-18T00:36:54.829 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:54.832 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:54.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:54.933 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:54.964 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:54.985 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:54.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:54.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:55.089 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:55.119 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.873951+0000 mgr.y (mgr.24454) 332 : cluster [DBG] pgmap v490: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.909208+0000 osd.3 (osd.3) 189 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.910329+0000 osd.3 (osd.3) 190 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.922252+0000 osd.1 (osd.1) 171 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:53.925178+0000 osd.1 (osd.1) 172 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:54.482324+0000 mon.c (mon.2) 322 : audit [DBG] from='client.? 172.21.15.59:0/2525928967' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:55.081120+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e336: 8 total, 8 up, 7 in 2024-02-18T00:36:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:55.086768+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.873951+0000 mgr.y (mgr.24454) 332 : cluster [DBG] pgmap v490: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.909208+0000 osd.3 (osd.3) 189 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.910329+0000 osd.3 (osd.3) 190 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.922252+0000 osd.1 (osd.1) 171 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:53.925178+0000 osd.1 (osd.1) 172 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:54.482324+0000 mon.c (mon.2) 322 : audit [DBG] from='client.? 172.21.15.59:0/2525928967' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:55.081120+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e336: 8 total, 8 up, 7 in 2024-02-18T00:36:55.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:55.086768+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.873951+0000 mgr.y (mgr.24454) 332 : cluster [DBG] pgmap v490: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.909208+0000 osd.3 (osd.3) 189 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.910329+0000 osd.3 (osd.3) 190 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.922252+0000 osd.1 (osd.1) 171 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:53.925178+0000 osd.1 (osd.1) 172 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:54.482324+0000 mon.c (mon.2) 322 : audit [DBG] from='client.? 172.21.15.59:0/2525928967' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:55.081120+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e336: 8 total, 8 up, 7 in 2024-02-18T00:36:55.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:55.086768+0000 mon.a (mon.0) 2218 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:36:55.468 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:55.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:55.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:55.573 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:55.605 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:55.626 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:55.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:55.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:55.730 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:55.759 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:55.781 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:55.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:55.882 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:55.910 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:55.932 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:55.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:56.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:56.040 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-18T00:36:56.062 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:56.507 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:56.511 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:56.516 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:56.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:56.534 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-18T00:36:56.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:36:56.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:56.646 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:55.651555+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi059-82882-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:55.651648+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e337: 8 total, 8 up, 7 in 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:55.860269+0000 osd.3 (osd.3) 191 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:55.861039+0000 osd.3 (osd.3) 192 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:55.874865+0000 mgr.y (mgr.24454) 333 : cluster [DBG] pgmap v493: 246 pgs: 8 creating+peering, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:55.967125+0000 osd.1 (osd.1) 173 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:55.970620+0000 osd.1 (osd.1) 174 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:55.651555+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi059-82882-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:36:56.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:55.651648+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e337: 8 total, 8 up, 7 in 2024-02-18T00:36:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:55.860269+0000 osd.3 (osd.3) 191 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:36:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:55.861039+0000 osd.3 (osd.3) 192 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:36:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:55.874865+0000 mgr.y (mgr.24454) 333 : cluster [DBG] pgmap v493: 246 pgs: 8 creating+peering, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T00:36:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:55.967125+0000 osd.1 (osd.1) 173 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:36:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:55.970620+0000 osd.1 (osd.1) 174 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:36:56.855 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:56.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:56.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:56.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:56.990 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:55.651555+0000 mon.a (mon.0) 2219 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi059-82882-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:36:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:55.651648+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e337: 8 total, 8 up, 7 in 2024-02-18T00:36:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:55.860269+0000 osd.3 (osd.3) 191 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:36:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:55.861039+0000 osd.3 (osd.3) 192 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:36:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:55.874865+0000 mgr.y (mgr.24454) 333 : cluster [DBG] pgmap v493: 246 pgs: 8 creating+peering, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T00:36:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:55.967125+0000 osd.1 (osd.1) 173 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:36:57.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:55.970620+0000 osd.1 (osd.1) 174 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:36:57.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:36:57.839 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:57.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:56.301747+0000 osd.0 (osd.0) 165 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:36:57.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:56.302610+0000 osd.0 (osd.0) 166 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:36:57.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:56.600416+0000 osd.2 (osd.2) 147 : cluster [DBG] 5.a scrub starts 2024-02-18T00:36:57.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:56.601721+0000 osd.2 (osd.2) 148 : cluster [DBG] 5.a scrub ok 2024-02-18T00:36:57.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:56.661555+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e338: 8 total, 8 up, 7 in 2024-02-18T00:36:57.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:56.849141+0000 osd.3 (osd.3) 193 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:36:57.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:56.850323+0000 osd.3 (osd.3) 194 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:36:57.842 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:56.301747+0000 osd.0 (osd.0) 165 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:36:57.842 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:56.302610+0000 osd.0 (osd.0) 166 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:36:57.842 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:56.600416+0000 osd.2 (osd.2) 147 : cluster [DBG] 5.a scrub starts 2024-02-18T00:36:57.842 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:56.601721+0000 osd.2 (osd.2) 148 : cluster [DBG] 5.a scrub ok 2024-02-18T00:36:57.842 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:56.661555+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e338: 8 total, 8 up, 7 in 2024-02-18T00:36:57.842 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:56.849141+0000 osd.3 (osd.3) 193 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:36:57.842 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:56.850323+0000 osd.3 (osd.3) 194 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:36:57.842 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:57.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:57.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:36:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:56.301747+0000 osd.0 (osd.0) 165 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:36:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:56.302610+0000 osd.0 (osd.0) 166 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:36:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:56.600416+0000 osd.2 (osd.2) 147 : cluster [DBG] 5.a scrub starts 2024-02-18T00:36:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:56.601721+0000 osd.2 (osd.2) 148 : cluster [DBG] 5.a scrub ok 2024-02-18T00:36:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:56.661555+0000 mon.a (mon.0) 2221 : cluster [DBG] osdmap e338: 8 total, 8 up, 7 in 2024-02-18T00:36:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:56.849141+0000 osd.3 (osd.3) 193 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:36:58.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:56.850323+0000 osd.3 (osd.3) 194 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:36:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:58.172 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:36:58.176 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:58.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:36:58.208 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:36:58.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:36:58.305 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:36:58.506 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:58.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:36:58.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:36:58.635 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.321049+0000 osd.0 (osd.0) 167 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.321927+0000 osd.0 (osd.0) 168 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.631913+0000 osd.2 (osd.2) 149 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.633125+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.665356+0000 mon.a (mon.0) 2222 : cluster [DBG] osdmap e339: 8 total, 8 up, 7 in 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:57.666849+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.789100+0000 osd.5 (osd.5) 149 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.791340+0000 osd.5 (osd.5) 150 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:57.875229+0000 mgr.y (mgr.24454) 334 : cluster [DBG] pgmap v496: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T00:36:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.106803+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.106957+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.321049+0000 osd.0 (osd.0) 167 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.321927+0000 osd.0 (osd.0) 168 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.631913+0000 osd.2 (osd.2) 149 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.633125+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.665356+0000 mon.a (mon.0) 2222 : cluster [DBG] osdmap e339: 8 total, 8 up, 7 in 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:57.666849+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.789100+0000 osd.5 (osd.5) 149 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.791340+0000 osd.5 (osd.5) 150 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:57.875229+0000 mgr.y (mgr.24454) 334 : cluster [DBG] pgmap v496: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T00:36:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.106803+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:58.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.106957+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:58.833 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:36:58.836 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:58.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:36:58.937 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.321049+0000 osd.0 (osd.0) 167 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.321927+0000 osd.0 (osd.0) 168 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.631913+0000 osd.2 (osd.2) 149 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.633125+0000 osd.2 (osd.2) 150 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.665356+0000 mon.a (mon.0) 2222 : cluster [DBG] osdmap e339: 8 total, 8 up, 7 in 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:57.666849+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.789100+0000 osd.5 (osd.5) 149 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:36:58.968 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.791340+0000 osd.5 (osd.5) 150 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:36:58.969 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:57.875229+0000 mgr.y (mgr.24454) 334 : cluster [DBG] pgmap v496: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T00:36:58.969 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.106803+0000 osd.5 (osd.5) 151 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:36:58.969 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.106957+0000 osd.5 (osd.5) 152 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:36:58.971 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:58.992 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:36:58.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:58.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:36:59.096 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:36:59.125 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:59.145 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:36:59.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:59.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:36:59.249 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:36:59.277 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:59.296 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:36:59.299 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:36:59.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:36:59.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:36:59.430 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:36:59.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.279043+0000 osd.4 (osd.4) 199 : cluster [DBG] 2.b scrub starts 2024-02-18T00:36:59.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.282669+0000 osd.4 (osd.4) 200 : cluster [DBG] 2.b scrub ok 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.599144+0000 osd.2 (osd.2) 151 : cluster [DBG] 6.f scrub starts 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.602457+0000 osd.2 (osd.2) 152 : cluster [DBG] 6.f scrub ok 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:58.670274+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.670338+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e340: 8 total, 8 up, 7 in 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:58.673329+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.816614+0000 osd.3 (osd.3) 195 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.819498+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:58.997587+0000 osd.1 (osd.1) 175 : cluster [DBG] 5.c scrub starts 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.000687+0000 osd.1 (osd.1) 176 : cluster [DBG] 5.c scrub ok 2024-02-18T00:36:59.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.279043+0000 osd.4 (osd.4) 199 : cluster [DBG] 2.b scrub starts 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.282669+0000 osd.4 (osd.4) 200 : cluster [DBG] 2.b scrub ok 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.599144+0000 osd.2 (osd.2) 151 : cluster [DBG] 6.f scrub starts 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.602457+0000 osd.2 (osd.2) 152 : cluster [DBG] 6.f scrub ok 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:58.670274+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.670338+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e340: 8 total, 8 up, 7 in 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:58.673329+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.816614+0000 osd.3 (osd.3) 195 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.819498+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:58.997587+0000 osd.1 (osd.1) 175 : cluster [DBG] 5.c scrub starts 2024-02-18T00:36:59.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:36:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.000687+0000 osd.1 (osd.1) 176 : cluster [DBG] 5.c scrub ok 2024-02-18T00:37:00.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.279043+0000 osd.4 (osd.4) 199 : cluster [DBG] 2.b scrub starts 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.282669+0000 osd.4 (osd.4) 200 : cluster [DBG] 2.b scrub ok 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.599144+0000 osd.2 (osd.2) 151 : cluster [DBG] 6.f scrub starts 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.602457+0000 osd.2 (osd.2) 152 : cluster [DBG] 6.f scrub ok 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:58.670274+0000 mon.a (mon.0) 2224 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.670338+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e340: 8 total, 8 up, 7 in 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:58.673329+0000 mon.a (mon.0) 2226 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]: dispatch 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.816614+0000 osd.3 (osd.3) 195 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.819498+0000 osd.3 (osd.3) 196 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:58.997587+0000 osd.1 (osd.1) 175 : cluster [DBG] 5.c scrub starts 2024-02-18T00:37:00.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:36:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.000687+0000 osd.1 (osd.1) 176 : cluster [DBG] 5.c scrub ok 2024-02-18T00:37:00.691 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.242888+0000 osd.4 (osd.4) 201 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:37:00.691 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.243696+0000 osd.4 (osd.4) 202 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.379699+0000 osd.7 (osd.7) 101 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.379786+0000 osd.7 (osd.7) 102 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:59.677092+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.677170+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e341: 8 total, 8 up, 7 in 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:59.696694+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:59.697527+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:36:59.697879+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.828294+0000 osd.3 (osd.3) 197 : cluster [DBG] 5.14 deep-scrub starts 2024-02-18T00:37:00.692 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.830493+0000 osd.3 (osd.3) 198 : cluster [DBG] 5.14 deep-scrub ok 2024-02-18T00:37:01.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.875755+0000 mgr.y (mgr.24454) 335 : cluster [DBG] pgmap v499: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.976547+0000 osd.1 (osd.1) 177 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:36:59.979558+0000 osd.1 (osd.1) 178 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.242888+0000 osd.4 (osd.4) 201 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.243696+0000 osd.4 (osd.4) 202 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.379699+0000 osd.7 (osd.7) 101 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.379786+0000 osd.7 (osd.7) 102 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:59.677092+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:37:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.677170+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e341: 8 total, 8 up, 7 in 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:59.696694+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:59.697527+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:36:59.697879+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.828294+0000 osd.3 (osd.3) 197 : cluster [DBG] 5.14 deep-scrub starts 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.830493+0000 osd.3 (osd.3) 198 : cluster [DBG] 5.14 deep-scrub ok 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.875755+0000 mgr.y (mgr.24454) 335 : cluster [DBG] pgmap v499: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.976547+0000 osd.1 (osd.1) 177 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:36:59.979558+0000 osd.1 (osd.1) 178 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:37:01.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.242888+0000 osd.4 (osd.4) 201 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.243696+0000 osd.4 (osd.4) 202 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.379699+0000 osd.7 (osd.7) 101 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.379786+0000 osd.7 (osd.7) 102 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:59.677092+0000 mon.a (mon.0) 2227 : audit [INF] from='client.? 172.21.15.59:0/791466082' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi059-82882-57"}]': finished 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.677170+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e341: 8 total, 8 up, 7 in 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:59.696694+0000 mon.a (mon.0) 2229 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:59.697527+0000 mon.a (mon.0) 2230 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:36:59.697879+0000 mon.a (mon.0) 2231 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.828294+0000 osd.3 (osd.3) 197 : cluster [DBG] 5.14 deep-scrub starts 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.830493+0000 osd.3 (osd.3) 198 : cluster [DBG] 5.14 deep-scrub ok 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.875755+0000 mgr.y (mgr.24454) 335 : cluster [DBG] pgmap v499: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.976547+0000 osd.1 (osd.1) 177 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:37:01.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:36:59.979558+0000 osd.1 (osd.1) 178 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:37:01.468 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:01.483 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:01.491 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:01.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:01.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:37:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:37:01.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:37:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:37:01.593 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:01.625 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:01.646 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:01.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:01.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:01.750 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:01.777 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:01.796 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:01.799 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:01.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:01.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:01.931 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:02.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.669460+0000 osd.2 (osd.2) 153 : cluster [DBG] 5.b scrub starts 2024-02-18T00:37:02.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.670643+0000 osd.2 (osd.2) 154 : cluster [DBG] 5.b scrub ok 2024-02-18T00:37:02.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:00.690535+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:02.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.690627+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e342: 8 total, 8 up, 7 in 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:00.691681+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi059-82882-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.802574+0000 osd.3 (osd.3) 199 : cluster [DBG] 6.b deep-scrub starts 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.804645+0000 osd.3 (osd.3) 200 : cluster [DBG] 6.b deep-scrub ok 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.818564+0000 osd.5 (osd.5) 153 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.820276+0000 osd.5 (osd.5) 154 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.946976+0000 osd.1 (osd.1) 179 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:00.950124+0000 osd.1 (osd.1) 180 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:01.172096+0000 mgr.y (mgr.24454) 336 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:02.358 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:02] "GET /metrics HTTP/1.1" 200 188935 "" "Prometheus/2.33.4" 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.669460+0000 osd.2 (osd.2) 153 : cluster [DBG] 5.b scrub starts 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.670643+0000 osd.2 (osd.2) 154 : cluster [DBG] 5.b scrub ok 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:00.690535+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.690627+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e342: 8 total, 8 up, 7 in 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:00.691681+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi059-82882-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.802574+0000 osd.3 (osd.3) 199 : cluster [DBG] 6.b deep-scrub starts 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.804645+0000 osd.3 (osd.3) 200 : cluster [DBG] 6.b deep-scrub ok 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.818564+0000 osd.5 (osd.5) 153 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.820276+0000 osd.5 (osd.5) 154 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.946976+0000 osd.1 (osd.1) 179 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:00.950124+0000 osd.1 (osd.1) 180 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:37:02.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:01.172096+0000 mgr.y (mgr.24454) 336 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:02.447 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:37:02.447 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.669460+0000 osd.2 (osd.2) 153 : cluster [DBG] 5.b scrub starts 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.670643+0000 osd.2 (osd.2) 154 : cluster [DBG] 5.b scrub ok 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:00.690535+0000 mon.a (mon.0) 2232 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.690627+0000 mon.a (mon.0) 2233 : cluster [DBG] osdmap e342: 8 total, 8 up, 7 in 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:00.691681+0000 mon.a (mon.0) 2234 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi059-82882-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.802574+0000 osd.3 (osd.3) 199 : cluster [DBG] 6.b deep-scrub starts 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.804645+0000 osd.3 (osd.3) 200 : cluster [DBG] 6.b deep-scrub ok 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.818564+0000 osd.5 (osd.5) 153 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.820276+0000 osd.5 (osd.5) 154 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.946976+0000 osd.1 (osd.1) 179 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:00.950124+0000 osd.1 (osd.1) 180 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:37:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:01.172096+0000 mgr.y (mgr.24454) 336 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.863+0000 7fea65709700 1 -- 172.21.15.59:0/3617015433 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600fcbb0 msgr2=0x7fea600f63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.863+0000 7fea65709700 1 --2- 172.21.15.59:0/3617015433 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600fcbb0 0x7fea600f63e0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 rx=0x7fea54004870 tx=0x7fea54009cb0).stop 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 -- 172.21.15.59:0/3617015433 shutdown_connections 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 --2- 172.21.15.59:0/3617015433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea600f6ac0 0x7fea601030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 --2- 172.21.15.59:0/3617015433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 0x7fea600fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 --2- 172.21.15.59:0/3617015433 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600fcbb0 0x7fea600f63e0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 -- 172.21.15.59:0/3617015433 >> 172.21.15.59:0/3617015433 conn(0x7fea6006ffc0 msgr2=0x7fea600f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 -- 172.21.15.59:0/3617015433 shutdown_connections 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 -- 172.21.15.59:0/3617015433 wait complete. 2024-02-18T00:37:02.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 librados: starting msgr at 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 librados: starting objecter 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 Processor -- start 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 -- start start 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 librados: setting wanted keys 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.864+0000 7fea65709700 1 librados: calling monclient init 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600f6ac0 0x7fea6010e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 0x7fea6010e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea600fcbb0 0x7fea601081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea601086c0 con 0x7fea600fc250 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60108830 con 0x7fea600fcbb0 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea601089f0 con 0x7fea600f6ac0 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 0x7fea6010e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600f6ac0 0x7fea6010e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:02.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 0x7fea6010e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58404/0 (socket says 172.21.15.59:58404) 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600f6ac0 0x7fea6010e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43770/0 (socket says 172.21.15.59:43770) 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 -- 172.21.15.59:0/3123661642 learned_addr learned my addr 172.21.15.59:0/3123661642 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5f7fe700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea600fcbb0 0x7fea601081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 -- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600f6ac0 msgr2=0x7fea6010e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600f6ac0 0x7fea6010e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 -- 172.21.15.59:0/3123661642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea600fcbb0 msgr2=0x7fea601081d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea600fcbb0 0x7fea601081d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 -- 172.21.15.59:0/3123661642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50005040 con 0x7fea600fc250 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5effd700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600f6ac0 0x7fea6010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea5e7fc700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 0x7fea6010e520 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 rx=0x7fea54009eb0 tx=0x7fea5400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea47fff700 10 client.?.objecter ms_handle_connect 0x7fea600fc250 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea47fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:02.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea47fff700 1 -- 172.21.15.59:0/3123661642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400e070 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea54005040 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea47fff700 1 -- 172.21.15.59:0/3123661642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea54013da0 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea47fff700 1 -- 172.21.15.59:0/3123661642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54019070 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 10 client.31140.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea60108f60 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.865+0000 7fea65709700 1 librados: init done 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.866+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6005f9b0 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea47fff700 1 -- 172.21.15.59:0/3123661642 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fea5401e420 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea47fff700 1 --2- 172.21.15.59:0/3123661642 >> v2:172.21.15.59:6800/3721891797 conn(0x7fea48062c90 0x7fea48065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea47fff700 1 -- 172.21.15.59:0/3123661642 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(344..344 src has 44..344) v4 ==== 8859+0+0 (secure 0 0 0) 0x7fea5407dff0 con 0x7fea600fc250 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea47fff700 10 client.31140.objecter ms_dispatch 0x7fea60001550 osd_map(344..344 src has 44..344) v4 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea47fff700 3 client.31140.objecter handle_osd_map got epochs [344,344] > 0 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea47fff700 3 client.31140.objecter handle_osd_map decoding full epoch 344 2024-02-18T00:37:02.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea5effd700 1 --2- 172.21.15.59:0/3123661642 >> v2:172.21.15.59:6800/3721891797 conn(0x7fea48062c90 0x7fea48065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:02.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.867+0000 7fea47fff700 20 client.31140.objecter dump_active .. 0 homeless 2024-02-18T00:37:02.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.868+0000 7fea47fff700 10 client.31140.objecter ms_handle_connect 0x7fea48062c90 2024-02-18T00:37:02.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.868+0000 7fea5effd700 1 --2- 172.21.15.59:0/3123661642 >> v2:172.21.15.59:6800/3721891797 conn(0x7fea48062c90 0x7fea48065150 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 rx=0x7fea50004d80 tx=0x7fea5000c040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:02.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:02.872+0000 7fea47fff700 1 -- 172.21.15.59:0/3123661642 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fea540acf60 con 0x7fea600fc250 2024-02-18T00:37:03.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.056+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fea60104e70 con 0x7fea600fc250 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:01.249368+0000 osd.0 (osd.0) 169 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:01.250766+0000 osd.0 (osd.0) 170 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:01.773882+0000 osd.3 (osd.3) 201 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:01.774791+0000 osd.3 (osd.3) 202 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:01.876168+0000 mgr.y (mgr.24454) 337 : cluster [DBG] pgmap v502: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:01.883543+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:01.918231+0000 osd.1 (osd.1) 181 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:01.919078+0000 osd.1 (osd.1) 182 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:02.136970+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e343: 8 total, 8 up, 7 in 2024-02-18T00:37:03.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:02.138671+0000 mon.a (mon.0) 2236 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:02.141796+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]: dispatch 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:02.143054+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]: dispatch 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:02.626448+0000 osd.2 (osd.2) 155 : cluster [DBG] 2.e scrub starts 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:02.627344+0000 osd.2 (osd.2) 156 : cluster [DBG] 2.e scrub ok 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:02.704464+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi059-82882-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:02.704578+0000 mon.a (mon.0) 2239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]': finished 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:02.704646+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e344: 8 total, 8 up, 7 in 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:03.058193+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? 172.21.15.59:0/3123661642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:01.249368+0000 osd.0 (osd.0) 169 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:01.250766+0000 osd.0 (osd.0) 170 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:01.773882+0000 osd.3 (osd.3) 201 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:37:03.403 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:01.774791+0000 osd.3 (osd.3) 202 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:01.876168+0000 mgr.y (mgr.24454) 337 : cluster [DBG] pgmap v502: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:01.883543+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:01.918231+0000 osd.1 (osd.1) 181 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:01.919078+0000 osd.1 (osd.1) 182 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:02.136970+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e343: 8 total, 8 up, 7 in 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:02.138671+0000 mon.a (mon.0) 2236 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:02.141796+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]: dispatch 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:02.143054+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]: dispatch 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:02.626448+0000 osd.2 (osd.2) 155 : cluster [DBG] 2.e scrub starts 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:02.627344+0000 osd.2 (osd.2) 156 : cluster [DBG] 2.e scrub ok 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:02.704464+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi059-82882-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:03.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:02.704578+0000 mon.a (mon.0) 2239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]': finished 2024-02-18T00:37:03.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:02.704646+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e344: 8 total, 8 up, 7 in 2024-02-18T00:37:03.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:03.058193+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? 172.21.15.59:0/3123661642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:03.415 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:03.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:03.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:01.249368+0000 osd.0 (osd.0) 169 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:01.250766+0000 osd.0 (osd.0) 170 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:01.773882+0000 osd.3 (osd.3) 201 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:01.774791+0000 osd.3 (osd.3) 202 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:01.876168+0000 mgr.y (mgr.24454) 337 : cluster [DBG] pgmap v502: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:01.883543+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:01.918231+0000 osd.1 (osd.1) 181 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:01.919078+0000 osd.1 (osd.1) 182 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:37:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:02.136970+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e343: 8 total, 8 up, 7 in 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:02.138671+0000 mon.a (mon.0) 2236 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:02.141796+0000 mon.c (mon.2) 324 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]: dispatch 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:02.143054+0000 mon.a (mon.0) 2237 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]: dispatch 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:02.626448+0000 osd.2 (osd.2) 155 : cluster [DBG] 2.e scrub starts 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:02.627344+0000 osd.2 (osd.2) 156 : cluster [DBG] 2.e scrub ok 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:02.704464+0000 mon.a (mon.0) 2238 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi059-82882-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:02.704578+0000 mon.a (mon.0) 2239 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-43"}]': finished 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:02.704646+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e344: 8 total, 8 up, 7 in 2024-02-18T00:37:03.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:03.058193+0000 mon.a (mon.0) 2241 : audit [INF] from='client.? 172.21.15.59:0/3123661642' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:03.519 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:03.551 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:03.572 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:03.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:03.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:03.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.647+0000 7fea47fff700 1 -- 172.21.15.59:0/3123661642 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v345) v1 ==== 87+0+0 (secure 0 0 0) 0x7fea54023030 con 0x7fea600fc250 2024-02-18T00:37:03.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.650+0000 7fea65709700 10 librados: watch_flush enter 2024-02-18T00:37:03.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.651+0000 7fea65709700 10 librados: watch_flush exit 2024-02-18T00:37:03.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.651+0000 7fea65709700 20 client.31140.objecter shutdown clearing up homeless session... 2024-02-18T00:37:03.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.651+0000 7fea65709700 10 client.31140.objecter successfully canceled tick 2024-02-18T00:37:03.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.651+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 >> v2:172.21.15.59:6800/3721891797 conn(0x7fea48062c90 msgr2=0x7fea48065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:03.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.651+0000 7fea65709700 1 --2- 172.21.15.59:0/3123661642 >> v2:172.21.15.59:6800/3721891797 conn(0x7fea48062c90 0x7fea48065150 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 rx=0x7fea50004d80 tx=0x7fea5000c040).stop 2024-02-18T00:37:03.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.652+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 msgr2=0x7fea6010e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:03.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.652+0000 7fea65709700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 0x7fea6010e520 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 rx=0x7fea54009eb0 tx=0x7fea5400b040).stop 2024-02-18T00:37:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.657+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 shutdown_connections 2024-02-18T00:37:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.657+0000 7fea65709700 1 --2- 172.21.15.59:0/3123661642 >> v2:172.21.15.59:6800/3721891797 conn(0x7fea48062c90 0x7fea48065150 unknown :-1 s=CLOSED pgs=3077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.657+0000 7fea65709700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea600fcbb0 0x7fea601081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.657+0000 7fea65709700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea600f6ac0 0x7fea6010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.657+0000 7fea65709700 1 --2- 172.21.15.59:0/3123661642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea600fc250 0x7fea6010e520 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:03.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.657+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 >> 172.21.15.59:0/3123661642 conn(0x7fea6006ffc0 msgr2=0x7fea6006dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:03.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.661+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 shutdown_connections 2024-02-18T00:37:03.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.661+0000 7fea65709700 1 -- 172.21.15.59:0/3123661642 wait complete. 2024-02-18T00:37:03.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.661+0000 7fea65709700 1 librados: shutdown 2024-02-18T00:37:03.665 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:37:03.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:03.719 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:03.236462+0000 osd.0 (osd.0) 171 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:03.239443+0000 osd.0 (osd.0) 172 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:03.245372+0000 osd.4 (osd.4) 203 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:03.248273+0000 osd.4 (osd.4) 204 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:03.641097+0000 mon.a (mon.0) 2242 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:03.646672+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? 172.21.15.59:0/3123661642' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:03.646740+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e345: 8 total, 8 up, 7 in 2024-02-18T00:37:04.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:03.236462+0000 osd.0 (osd.0) 171 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:37:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:03.239443+0000 osd.0 (osd.0) 172 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:37:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:03.245372+0000 osd.4 (osd.4) 203 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:37:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:03.248273+0000 osd.4 (osd.4) 204 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:37:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:03.641097+0000 mon.a (mon.0) 2242 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:03.646672+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? 172.21.15.59:0/3123661642' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:37:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:03.646740+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e345: 8 total, 8 up, 7 in 2024-02-18T00:37:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:03.236462+0000 osd.0 (osd.0) 171 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:37:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:03.239443+0000 osd.0 (osd.0) 172 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:37:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:03.245372+0000 osd.4 (osd.4) 203 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:37:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:03.248273+0000 osd.4 (osd.4) 204 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:37:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:03.641097+0000 mon.a (mon.0) 2242 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:03.646672+0000 mon.a (mon.0) 2243 : audit [INF] from='client.? 172.21.15.59:0/3123661642' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:37:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:03.646740+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e345: 8 total, 8 up, 7 in 2024-02-18T00:37:04.736 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-18T00:37:04.875 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:04.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:04.877 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:04.978 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 -- 172.21.15.59:0/973629796 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740fdbf0 msgr2=0x7f9a740fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 --2- 172.21.15.59:0/973629796 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740fdbf0 0x7f9a740fdfd0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 rx=0x7f9a64004870 tx=0x7f9a64009cb0).stop 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 -- 172.21.15.59:0/973629796 shutdown_connections 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 --2- 172.21.15.59:0/973629796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a740f67e0 0x7f9a74104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 --2- 172.21.15.59:0/973629796 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740fdbf0 0x7f9a740fdfd0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 --2- 172.21.15.59:0/973629796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 0x7f9a740f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 -- 172.21.15.59:0/973629796 >> 172.21.15.59:0/973629796 conn(0x7f9a74070030 msgr2=0x7f9a740f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 -- 172.21.15.59:0/973629796 shutdown_connections 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 -- 172.21.15.59:0/973629796 wait complete. 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 librados: starting msgr at 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 librados: starting objecter 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.244+0000 7f9a7887a700 1 Processor -- start 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 -- start start 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 librados: setting wanted keys 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 librados: calling monclient init 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740f67e0 0x7f9a741104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a740fdbf0 0x7f9a74110990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 0x7f9a7410a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410ab30 con 0x7f9a740fe550 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410ac70 con 0x7f9a740fdbf0 2024-02-18T00:37:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7887a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410ae30 con 0x7f9a740f67e0 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 0x7f9a7410a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 0x7f9a7410a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58432/0 (socket says 172.21.15.59:58432) 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 -- 172.21.15.59:0/2039377508 learned_addr learned my addr 172.21.15.59:0/2039377508 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a69d9b700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a740fdbf0 0x7f9a74110990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a7259c700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740f67e0 0x7f9a741104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 -- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740f67e0 msgr2=0x7f9a741104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740f67e0 0x7f9a741104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 -- 172.21.15.59:0/2039377508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a740fdbf0 msgr2=0x7f9a74110990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a740fdbf0 0x7f9a74110990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 -- 172.21.15.59:0/2039377508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64005040 con 0x7f9a740fe550 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a69d9b700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a740fdbf0 0x7f9a74110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a72d9d700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 0x7f9a7410a640 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 rx=0x7f9a6c013a60 tx=0x7f9a6c0143c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:05.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a6bfff700 10 client.?.objecter ms_handle_connect 0x7f9a740fe550 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.245+0000 7f9a6bfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.246+0000 7f9a6bfff700 1 -- 172.21.15.59:0/2039377508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c014430 con 0x7f9a740fe550 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.246+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7410b110 con 0x7f9a740fe550 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.246+0000 7f9a6bfff700 1 -- 172.21.15.59:0/2039377508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a6c0117b0 con 0x7f9a740fe550 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.246+0000 7f9a6bfff700 1 -- 172.21.15.59:0/2039377508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c01cd60 con 0x7f9a740fe550 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.246+0000 7f9a7887a700 10 client.31146.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.246+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a740fea10 con 0x7f9a740fe550 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.246+0000 7f9a7887a700 1 librados: init done 2024-02-18T00:37:05.006 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.247+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7405f990 con 0x7f9a740fe550 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.247+0000 7f9a6bfff700 1 -- 172.21.15.59:0/2039377508 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9a6c0112e0 con 0x7f9a740fe550 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a6bfff700 1 --2- 172.21.15.59:0/2039377508 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9a60062c90 0x7f9a60065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a6bfff700 1 -- 172.21.15.59:0/2039377508 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(344..344 src has 44..344) v4 ==== 8859+0+0 (secure 0 0 0) 0x7f9a6c086430 con 0x7f9a740fe550 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a6bfff700 10 client.31146.objecter ms_dispatch 0x7f9a74001550 osd_map(344..344 src has 44..344) v4 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a6bfff700 3 client.31146.objecter handle_osd_map got epochs [344,344] > 0 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a6bfff700 3 client.31146.objecter handle_osd_map decoding full epoch 344 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a7259c700 1 --2- 172.21.15.59:0/2039377508 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9a60062c90 0x7f9a60065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a6bfff700 20 client.31146.objecter dump_active .. 0 homeless 2024-02-18T00:37:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a6bfff700 10 client.31146.objecter ms_handle_connect 0x7f9a60062c90 2024-02-18T00:37:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.248+0000 7f9a7259c700 1 --2- 172.21.15.59:0/2039377508 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9a60062c90 0x7f9a60065150 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7f9a64004ee0 tx=0x7f9a6400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.252+0000 7f9a6bfff700 1 -- 172.21.15.59:0/2039377508 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9a6c0595e0 con 0x7f9a740fe550 2024-02-18T00:37:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.427+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a741071b0 con 0x7f9a60062c90 2024-02-18T00:37:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:03.444+0000 7f9a6bfff700 1 -- 172.21.15.59:0/2039377508 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+616671 (secure 0 0 0) 0x7f9a741071b0 con 0x7f9a60062c90 2024-02-18T00:37:05.013 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:05.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.004+0000 7f9a7887a700 10 librados: watch_flush enter 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.004+0000 7f9a7887a700 10 librados: watch_flush exit 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.004+0000 7f9a7887a700 20 client.31146.objecter shutdown clearing up homeless session... 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.004+0000 7f9a7887a700 10 client.31146.objecter successfully canceled tick 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9a60062c90 msgr2=0x7f9a60065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 --2- 172.21.15.59:0/2039377508 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9a60062c90 0x7f9a60065150 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7f9a64004ee0 tx=0x7f9a6400b040).stop 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 msgr2=0x7f9a7410a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 0x7f9a7410a640 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 rx=0x7f9a6c013a60 tx=0x7f9a6c0143c0).stop 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 shutdown_connections 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a740fe550 0x7f9a7410a640 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a740f67e0 0x7f9a741104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 --2- 172.21.15.59:0/2039377508 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9a60062c90 0x7f9a60065150 unknown :-1 s=CLOSED pgs=3078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 --2- 172.21.15.59:0/2039377508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a740fdbf0 0x7f9a74110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:05.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 >> 172.21.15.59:0/2039377508 conn(0x7f9a74070030 msgr2=0x7f9a74102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:05.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 shutdown_connections 2024-02-18T00:37:05.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 -- 172.21.15.59:0/2039377508 wait complete. 2024-02-18T00:37:05.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:05.005+0000 7f9a7887a700 1 librados: shutdown 2024-02-18T00:37:05.022 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:37:05.031 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:05.034 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:05.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:05.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:05.166 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:05.187 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:05.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:05.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:05.290 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:05.317 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:03.428459+0000 mgr.y (mgr.24454) 338 : audit [DBG] from='client.31146 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:05.317 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:03.876527+0000 mgr.y (mgr.24454) 339 : cluster [DBG] pgmap v505: 214 pgs: 8 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:05.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:04.665177+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e346: 8 total, 8 up, 7 in 2024-02-18T00:37:05.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:04.672883+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:05.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:04.676022+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:05.318 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:04.677558+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:05.319 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:05.339 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:05.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:05.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:05.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:03.428459+0000 mgr.y (mgr.24454) 338 : audit [DBG] from='client.31146 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:05.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:03.876527+0000 mgr.y (mgr.24454) 339 : cluster [DBG] pgmap v505: 214 pgs: 8 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:05.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:04.665177+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e346: 8 total, 8 up, 7 in 2024-02-18T00:37:05.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:04.672883+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:05.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:04.676022+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:05.391 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:04.677558+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:05.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:03.428459+0000 mgr.y (mgr.24454) 338 : audit [DBG] from='client.31146 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:05.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:03.876527+0000 mgr.y (mgr.24454) 339 : cluster [DBG] pgmap v505: 214 pgs: 8 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:05.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:04.665177+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e346: 8 total, 8 up, 7 in 2024-02-18T00:37:05.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:04.672883+0000 mon.c (mon.2) 325 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:05.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:04.676022+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:05.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:04.677558+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:05.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:05.471 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:05.490 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:05.493 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:05.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:05.594 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:05.620 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:05.639 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:05.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:05.642 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:05.742 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:05.771 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:05.824 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:05.827 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:05.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:05.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:05.955 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:06.503 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-02-18T00:37:02.705897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007274,"num_objects":352,"num_object_clones":2,"num_object_copies":1056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":0,"num_read":1657,"num_read_kb":1285,"num_write":1189,"num_write_kb":4511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":252,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2534,"ondisk_log_size":2534,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":715,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2155724,"kb_used_data":50540,"kb_used_omap":0,"kb_used_meta":2105088,"kb_avail":654060340,"statfs":{"total":671965249536,"available":669757788160,"internally_reserved":0,"allocated":51752960,"data_stored":34091563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2155610112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":84,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":84,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.038306"},"pg_stats":[{"pgid":"193.1f","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279298+0000","last_change":"2024-02-18T00:36:44.668499+0000","last_active":"2024-02-18T00:37:00.279298+0000","last_peered":"2024-02-18T00:37:00.279298+0000","last_clean":"2024-02-18T00:37:00.279298+0000","last_became_active":"2024-02-18T00:36:44.668206+0000","last_became_peered":"2024-02-18T00:36:44.668206+0000","last_unstale":"2024-02-18T00:37:00.279298+0000","last_undegraded":"2024-02-18T00:37:00.279298+0000","last_fullsized":"2024-02-18T00:37:00.279298+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"193.1e","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673171+0000","last_change":"2024-02-18T00:36:44.666227+0000","last_active":"2024-02-18T00:36:58.673171+0000","last_peered":"2024-02-18T00:36:58.673171+0000","last_clean":"2024-02-18T00:36:58.673171+0000","last_became_active":"2024-02-18T00:36:44.665912+0000","last_became_peered":"2024-02-18T00:36:44.665912+0000","last_unstale":"2024-02-18T00:36:58.673171+0000","last_undegraded":"2024-02-18T00:36:58.673171+0000","last_fullsized":"2024-02-18T00:36:58.673171+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"193.1d","version":"0'0","reported_seq":23,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827621+0000","last_change":"2024-02-18T00:36:44.661347+0000","last_active":"2024-02-18T00:36:58.827621+0000","last_peered":"2024-02-18T00:36:58.827621+0000","last_clean":"2024-02-18T00:36:58.827621+0000","last_became_active":"2024-02-18T00:36:44.660986+0000","last_became_peered":"2024-02-18T00:36:44.660986+0000","last_unstale":"2024-02-18T00:36:58.827621+0000","last_undegraded":"2024-02-18T00:36:58.827621+0000","last_fullsized":"2024-02-18T00:36:58.827621+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"193.1c","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695467+0000","last_change":"2024-02-18T00:36:44.666286+0000","last_active":"2024-02-18T00:36:59.695467+0000","last_peered":"2024-02-18T00:36:59.695467+0000","last_clean":"2024-02-18T00:36:59.695467+0000","last_became_active":"2024-02-18T00:36:44.665923+0000","last_became_peered":"2024-02-18T00:36:44.665923+0000","last_unstale":"2024-02-18T00:36:59.695467+0000","last_undegraded":"2024-02-18T00:36:59.695467+0000","last_fullsized":"2024-02-18T00:36:59.695467+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"193.1b","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278641+0000","last_change":"2024-02-18T00:36:44.667486+0000","last_active":"2024-02-18T00:37:00.278641+0000","last_peered":"2024-02-18T00:37:00.278641+0000","last_clean":"2024-02-18T00:37:00.278641+0000","last_became_active":"2024-02-18T00:36:44.667167+0000","last_became_peered":"2024-02-18T00:36:44.667167+0000","last_unstale":"2024-02-18T00:37:00.278641+0000","last_undegraded":"2024-02-18T00:37:00.278641+0000","last_fullsized":"2024-02-18T00:37:00.278641+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"193.1a","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.676248+0000","last_change":"2024-02-18T00:36:44.661975+0000","last_active":"2024-02-18T00:36:59.676248+0000","last_peered":"2024-02-18T00:36:59.676248+0000","last_clean":"2024-02-18T00:36:59.676248+0000","last_became_active":"2024-02-18T00:36:44.661620+0000","last_became_peered":"2024-02-18T00:36:44.661620+0000","last_unstale":"2024-02-18T00:36:59.676248+0000","last_undegraded":"2024-02-18T00:36:59.676248+0000","last_fullsized":"2024-02-18T00:36:59.676248+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"193.19","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278090+0000","last_change":"2024-02-18T00:36:44.668689+0000","last_active":"2024-02-18T00:37:00.278090+0000","last_peered":"2024-02-18T00:37:00.278090+0000","last_clean":"2024-02-18T00:37:00.278090+0000","last_became_active":"2024-02-18T00:36:44.667990+0000","last_became_peered":"2024-02-18T00:36:44.667990+0000","last_unstale":"2024-02-18T00:37:00.278090+0000","last_undegraded":"2024-02-18T00:37:00.278090+0000","last_fullsized":"2024-02-18T00:37:00.278090+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"193.18","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673595+0000","last_change":"2024-02-18T00:36:44.666494+0000","last_active":"2024-02-18T00:36:58.673595+0000","last_peered":"2024-02-18T00:36:58.673595+0000","last_clean":"2024-02-18T00:36:58.673595+0000","last_became_active":"2024-02-18T00:36:44.666163+0000","last_became_peered":"2024-02-18T00:36:44.666163+0000","last_unstale":"2024-02-18T00:36:58.673595+0000","last_undegraded":"2024-02-18T00:36:58.673595+0000","last_fullsized":"2024-02-18T00:36:58.673595+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"193.17","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677481+0000","last_change":"2024-02-18T00:36:44.662530+0000","last_active":"2024-02-18T00:36:59.677481+0000","last_peered":"2024-02-18T00:36:59.677481+0000","last_clean":"2024-02-18T00:36:59.677481+0000","last_became_active":"2024-02-18T00:36:44.662048+0000","last_became_peered":"2024-02-18T00:36:44.662048+0000","last_unstale":"2024-02-18T00:36:59.677481+0000","last_undegraded":"2024-02-18T00:36:59.677481+0000","last_fullsized":"2024-02-18T00:36:59.677481+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"193.16","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695639+0000","last_change":"2024-02-18T00:36:44.663865+0000","last_active":"2024-02-18T00:36:59.695639+0000","last_peered":"2024-02-18T00:36:59.695639+0000","last_clean":"2024-02-18T00:36:59.695639+0000","last_became_active":"2024-02-18T00:36:44.663533+0000","last_became_peered":"2024-02-18T00:36:44.663533+0000","last_unstale":"2024-02-18T00:36:59.695639+0000","last_undegraded":"2024-02-18T00:36:59.695639+0000","last_fullsized":"2024-02-18T00:36:59.695639+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"193.15","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.676368+0000","last_change":"2024-02-18T00:36:44.671584+0000","last_active":"2024-02-18T00:36:59.676368+0000","last_peered":"2024-02-18T00:36:59.676368+0000","last_clean":"2024-02-18T00:36:59.676368+0000","last_became_active":"2024-02-18T00:36:44.671231+0000","last_became_peered":"2024-02-18T00:36:44.671231+0000","last_unstale":"2024-02-18T00:36:59.676368+0000","last_undegraded":"2024-02-18T00:36:59.676368+0000","last_fullsized":"2024-02-18T00:36:59.676368+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"193.14","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678378+0000","last_change":"2024-02-18T00:36:44.663418+0000","last_active":"2024-02-18T00:36:59.678378+0000","last_peered":"2024-02-18T00:36:59.678378+0000","last_clean":"2024-02-18T00:36:59.678378+0000","last_became_active":"2024-02-18T00:36:44.663099+0000","last_became_peered":"2024-02-18T00:36:44.663099+0000","last_unstale":"2024-02-18T00:36:59.678378+0000","last_undegraded":"2024-02-18T00:36:59.678378+0000","last_fullsized":"2024-02-18T00:36:59.678378+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"193.13","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.670778+0000","last_change":"2024-02-18T00:36:44.663566+0000","last_active":"2024-02-18T00:36:58.670778+0000","last_peered":"2024-02-18T00:36:58.670778+0000","last_clean":"2024-02-18T00:36:58.670778+0000","last_became_active":"2024-02-18T00:36:44.663282+0000","last_became_peered":"2024-02-18T00:36:44.663282+0000","last_unstale":"2024-02-18T00:36:58.670778+0000","last_undegraded":"2024-02-18T00:36:58.670778+0000","last_fullsized":"2024-02-18T00:36:58.670778+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"193.12","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674434+0000","last_change":"2024-02-18T00:36:44.669542+0000","last_active":"2024-02-18T00:36:58.674434+0000","last_peered":"2024-02-18T00:36:58.674434+0000","last_clean":"2024-02-18T00:36:58.674434+0000","last_became_active":"2024-02-18T00:36:44.669219+0000","last_became_peered":"2024-02-18T00:36:44.669219+0000","last_unstale":"2024-02-18T00:36:58.674434+0000","last_undegraded":"2024-02-18T00:36:58.674434+0000","last_fullsized":"2024-02-18T00:36:58.674434+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"193.11","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672702+0000","last_change":"2024-02-18T00:36:44.671536+0000","last_active":"2024-02-18T00:36:58.672702+0000","last_peered":"2024-02-18T00:36:58.672702+0000","last_clean":"2024-02-18T00:36:58.672702+0000","last_became_active":"2024-02-18T00:36:44.671235+0000","last_became_peered":"2024-02-18T00:36:44.671235+0000","last_unstale":"2024-02-18T00:36:58.672702+0000","last_undegraded":"2024-02-18T00:36:58.672702+0000","last_fullsized":"2024-02-18T00:36:58.672702+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"193.10","version":"0'0","reported_seq":25,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.670799+0000","last_change":"2024-02-18T00:36:44.666417+0000","last_active":"2024-02-18T00:36:59.670799+0000","last_peered":"2024-02-18T00:36:59.670799+0000","last_clean":"2024-02-18T00:36:59.670799+0000","last_became_active":"2024-02-18T00:36:44.666123+0000","last_became_peered":"2024-02-18T00:36:44.666123+0000","last_unstale":"2024-02-18T00:36:59.670799+0000","last_undegraded":"2024-02-18T00:36:59.670799+0000","last_fullsized":"2024-02-18T00:36:59.670799+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"193.f","version":"0'0","reported_seq":23,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828773+0000","last_change":"2024-02-18T00:36:44.664660+0000","last_active":"2024-02-18T00:36:58.828773+0000","last_peered":"2024-02-18T00:36:58.828773+0000","last_clean":"2024-02-18T00:36:58.828773+0000","last_became_active":"2024-02-18T00:36:44.664383+0000","last_became_peered":"2024-02-18T00:36:44.664383+0000","last_unstale":"2024-02-18T00:36:58.828773+0000","last_undegraded":"2024-02-18T00:36:58.828773+0000","last_fullsized":"2024-02-18T00:36:58.828773+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"193.e","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695867+0000","last_change":"2024-02-18T00:36:44.657629+0000","last_active":"2024-02-18T00:36:59.695867+0000","last_peered":"2024-02-18T00:36:59.695867+0000","last_clean":"2024-02-18T00:36:59.695867+0000","last_became_active":"2024-02-18T00:36:44.657322+0000","last_became_peered":"2024-02-18T00:36:44.657322+0000","last_unstale":"2024-02-18T00:36:59.695867+0000","last_undegraded":"2024-02-18T00:36:59.695867+0000","last_fullsized":"2024-02-18T00:36:59.695867+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"193.d","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695728+0000","last_change":"2024-02-18T00:36:44.672308+0000","last_active":"2024-02-18T00:36:59.695728+0000","last_peered":"2024-02-18T00:36:59.695728+0000","last_clean":"2024-02-18T00:36:59.695728+0000","last_became_active":"2024-02-18T00:36:44.672003+0000","last_became_peered":"2024-02-18T00:36:44.672003+0000","last_unstale":"2024-02-18T00:36:59.695728+0000","last_undegraded":"2024-02-18T00:36:59.695728+0000","last_fullsized":"2024-02-18T00:36:59.695728+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"193.c","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278741+0000","last_change":"2024-02-18T00:36:44.667886+0000","last_active":"2024-02-18T00:37:00.278741+0000","last_peered":"2024-02-18T00:37:00.278741+0000","last_clean":"2024-02-18T00:37:00.278741+0000","last_became_active":"2024-02-18T00:36:44.667582+0000","last_became_peered":"2024-02-18T00:36:44.667582+0000","last_unstale":"2024-02-18T00:37:00.278741+0000","last_undegraded":"2024-02-18T00:37:00.278741+0000","last_fullsized":"2024-02-18T00:37:00.278741+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"193.b","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677890+0000","last_change":"2024-02-18T00:36:44.661993+0000","last_active":"2024-02-18T00:36:59.677890+0000","last_peered":"2024-02-18T00:36:59.677890+0000","last_clean":"2024-02-18T00:36:59.677890+0000","last_became_active":"2024-02-18T00:36:44.661668+0000","last_became_peered":"2024-02-18T00:36:44.661668+0000","last_unstale":"2024-02-18T00:36:59.677890+0000","last_undegraded":"2024-02-18T00:36:59.677890+0000","last_fullsized":"2024-02-18T00:36:59.677890+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"193.a","version":"0'0","reported_seq":23,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827025+0000","last_change":"2024-02-18T00:36:44.671469+0000","last_active":"2024-02-18T00:36:58.827025+0000","last_peered":"2024-02-18T00:36:58.827025+0000","last_clean":"2024-02-18T00:36:58.827025+0000","last_became_active":"2024-02-18T00:36:44.671113+0000","last_became_peered":"2024-02-18T00:36:44.671113+0000","last_unstale":"2024-02-18T00:36:58.827025+0000","last_undegraded":"2024-02-18T00:36:58.827025+0000","last_fullsized":"2024-02-18T00:36:58.827025+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"193.9","version":"0'0","reported_seq":23,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.826780+0000","last_change":"2024-02-18T00:36:44.656505+0000","last_active":"2024-02-18T00:36:58.826780+0000","last_peered":"2024-02-18T00:36:58.826780+0000","last_clean":"2024-02-18T00:36:58.826780+0000","last_became_active":"2024-02-18T00:36:44.656176+0000","last_became_peered":"2024-02-18T00:36:44.656176+0000","last_unstale":"2024-02-18T00:36:58.826780+0000","last_undegraded":"2024-02-18T00:36:58.826780+0000","last_fullsized":"2024-02-18T00:36:58.826780+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"193.8","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.694855+0000","last_change":"2024-02-18T00:36:44.667708+0000","last_active":"2024-02-18T00:36:59.694855+0000","last_peered":"2024-02-18T00:36:59.694855+0000","last_clean":"2024-02-18T00:36:59.694855+0000","last_became_active":"2024-02-18T00:36:44.666967+0000","last_became_peered":"2024-02-18T00:36:44.666967+0000","last_unstale":"2024-02-18T00:36:59.694855+0000","last_undegraded":"2024-02-18T00:36:59.694855+0000","last_fullsized":"2024-02-18T00:36:59.694855+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1],"acting":[6,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"196.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.6","version":"341'15","reported_seq":39,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.696224+0000","last_change":"2024-02-18T00:36:44.663865+0000","last_active":"2024-02-18T00:37:00.696224+0000","last_peered":"2024-02-18T00:37:00.696224+0000","last_clean":"2024-02-18T00:37:00.696224+0000","last_became_active":"2024-02-18T00:36:44.663556+0000","last_became_peered":"2024-02-18T00:36:44.663556+0000","last_unstale":"2024-02-18T00:37:00.696224+0000","last_undegraded":"2024-02-18T00:37:00.696224+0000","last_fullsized":"2024-02-18T00:37:00.696224+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":253,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":252,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"196.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.7","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695976+0000","last_change":"2024-02-18T00:36:44.663809+0000","last_active":"2024-02-18T00:36:59.695976+0000","last_peered":"2024-02-18T00:36:59.695976+0000","last_clean":"2024-02-18T00:36:59.695976+0000","last_became_active":"2024-02-18T00:36:44.663437+0000","last_became_peered":"2024-02-18T00:36:44.663437+0000","last_unstale":"2024-02-18T00:36:59.695976+0000","last_undegraded":"2024-02-18T00:36:59.695976+0000","last_fullsized":"2024-02-18T00:36:59.695976+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"196.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.4","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674320+0000","last_change":"2024-02-18T00:36:44.666104+0000","last_active":"2024-02-18T00:36:58.674320+0000","last_peered":"2024-02-18T00:36:58.674320+0000","last_clean":"2024-02-18T00:36:58.674320+0000","last_became_active":"2024-02-18T00:36:44.665761+0000","last_became_peered":"2024-02-18T00:36:44.665761+0000","last_unstale":"2024-02-18T00:36:58.674320+0000","last_undegraded":"2024-02-18T00:36:58.674320+0000","last_fullsized":"2024-02-18T00:36:58.674320+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"196.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.5","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672447+0000","last_change":"2024-02-18T00:36:44.662876+0000","last_active":"2024-02-18T00:36:58.672447+0000","last_peered":"2024-02-18T00:36:58.672447+0000","last_clean":"2024-02-18T00:36:58.672447+0000","last_became_active":"2024-02-18T00:36:44.662563+0000","last_became_peered":"2024-02-18T00:36:44.662563+0000","last_unstale":"2024-02-18T00:36:58.672447+0000","last_undegraded":"2024-02-18T00:36:58.672447+0000","last_fullsized":"2024-02-18T00:36:58.672447+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"196.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.2","version":"0'0","reported_seq":23,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827087+0000","last_change":"2024-02-18T00:36:44.663344+0000","last_active":"2024-02-18T00:36:58.827087+0000","last_peered":"2024-02-18T00:36:58.827087+0000","last_clean":"2024-02-18T00:36:58.827087+0000","last_became_active":"2024-02-18T00:36:44.662909+0000","last_became_peered":"2024-02-18T00:36:44.662909+0000","last_unstale":"2024-02-18T00:36:58.827087+0000","last_undegraded":"2024-02-18T00:36:58.827087+0000","last_fullsized":"2024-02-18T00:36:58.827087+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"196.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.3","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.670828+0000","last_change":"2024-02-18T00:36:44.660458+0000","last_active":"2024-02-18T00:36:58.670828+0000","last_peered":"2024-02-18T00:36:58.670828+0000","last_clean":"2024-02-18T00:36:58.670828+0000","last_became_active":"2024-02-18T00:36:44.660105+0000","last_became_peered":"2024-02-18T00:36:44.660105+0000","last_unstale":"2024-02-18T00:36:58.670828+0000","last_undegraded":"2024-02-18T00:36:58.670828+0000","last_fullsized":"2024-02-18T00:36:58.670828+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"196.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.0","version":"0'0","reported_seq":24,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.694884+0000","last_change":"2024-02-18T00:36:44.667766+0000","last_active":"2024-02-18T00:36:59.694884+0000","last_peered":"2024-02-18T00:36:59.694884+0000","last_clean":"2024-02-18T00:36:59.694884+0000","last_became_active":"2024-02-18T00:36:44.667380+0000","last_became_peered":"2024-02-18T00:36:44.667380+0000","last_unstale":"2024-02-18T00:36:59.694884+0000","last_undegraded":"2024-02-18T00:36:59.694884+0000","last_fullsized":"2024-02-18T00:36:59.694884+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"196.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:02.687757+0000","last_change":"2024-02-18T00:37:02.687757+0000","last_active":"2024-02-18T00:37:02.687757+0000","last_peered":"2024-02-18T00:37:02.687757+0000","last_clean":"2024-02-18T00:37:02.687757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:02.687757+0000","last_undegraded":"2024-02-18T00:37:02.687757+0000","last_fullsized":"2024-02-18T00:37:02.687757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:02.687757+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.687757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"193.1","version":"0'0","reported_seq":24,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672734+0000","last_change":"2024-02-18T00:36:44.671393+0000","last_active":"2024-02-18T00:36:58.672734+0000","last_peered":"2024-02-18T00:36:58.672734+0000","last_clean":"2024-02-18T00:36:58.672734+0000","last_became_active":"2024-02-18T00:36:44.671110+0000","last_became_peered":"2024-02-18T00:36:44.671110+0000","last_unstale":"2024-02-18T00:36:58.672734+0000","last_undegraded":"2024-02-18T00:36:58.672734+0000","last_fullsized":"2024-02-18T00:36:58.672734+0000","mapping_epoch":325,"log_start":"0'0","ondisk_log_start":"0'0","created":325,"last_epoch_clean":326,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:43.633508+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.633508+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":312,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671975+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:36:58.671975+0000","last_peered":"2024-02-18T00:36:58.671975+0000","last_clean":"2024-02-18T00:36:58.671975+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:36:58.671975+0000","last_undegraded":"2024-02-18T00:36:58.671975+0000","last_fullsized":"2024-02-18T00:36:58.671975+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"0'0","reported_seq":274,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672893+0000","last_change":"2024-02-18T00:36:06.658623+0000","last_active":"2024-02-18T00:36:58.672893+0000","last_peered":"2024-02-18T00:36:58.672893+0000","last_clean":"2024-02-18T00:36:58.672893+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:36:58.672893+0000","last_undegraded":"2024-02-18T00:36:58.672893+0000","last_fullsized":"2024-02-18T00:36:58.672893+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":342,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695211+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:36:59.695211+0000","last_peered":"2024-02-18T00:36:59.695211+0000","last_clean":"2024-02-18T00:36:59.695211+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:36:59.695211+0000","last_undegraded":"2024-02-18T00:36:59.695211+0000","last_fullsized":"2024-02-18T00:36:59.695211+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"245'73","reported_seq":341,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.670908+0000","last_change":"2024-02-18T00:36:34.581378+0000","last_active":"2024-02-18T00:36:58.670908+0000","last_peered":"2024-02-18T00:36:58.670908+0000","last_clean":"2024-02-18T00:36:58.670908+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:36:58.670908+0000","last_undegraded":"2024-02-18T00:36:58.670908+0000","last_fullsized":"2024-02-18T00:36:58.670908+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'73","last_scrub_stamp":"2024-02-18T00:36:34.581302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.581302+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"230'120","reported_seq":380,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278881+0000","last_change":"2024-02-18T00:36:36.464430+0000","last_active":"2024-02-18T00:37:00.278881+0000","last_peered":"2024-02-18T00:37:00.278881+0000","last_clean":"2024-02-18T00:37:00.278881+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:37:00.278881+0000","last_undegraded":"2024-02-18T00:37:00.278881+0000","last_fullsized":"2024-02-18T00:37:00.278881+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'120","last_scrub_stamp":"2024-02-18T00:36:36.464361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.464361+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":357,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671546+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:36:59.671546+0000","last_peered":"2024-02-18T00:36:59.671546+0000","last_clean":"2024-02-18T00:36:59.671546+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:36:59.671546+0000","last_undegraded":"2024-02-18T00:36:59.671546+0000","last_fullsized":"2024-02-18T00:36:59.671546+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"267'124","reported_seq":354,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673566+0000","last_change":"2024-02-18T00:36:53.649744+0000","last_active":"2024-02-18T00:36:58.673566+0000","last_peered":"2024-02-18T00:36:58.673566+0000","last_clean":"2024-02-18T00:36:58.673566+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:36:58.673566+0000","last_undegraded":"2024-02-18T00:36:58.673566+0000","last_fullsized":"2024-02-18T00:36:58.673566+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'124","last_scrub_stamp":"2024-02-18T00:36:53.649677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.649677+0000","log_size":124,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"267'73","reported_seq":335,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673034+0000","last_change":"2024-02-18T00:36:37.532389+0000","last_active":"2024-02-18T00:36:58.673034+0000","last_peered":"2024-02-18T00:36:58.673034+0000","last_clean":"2024-02-18T00:36:58.673034+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:36:58.673034+0000","last_undegraded":"2024-02-18T00:36:58.673034+0000","last_fullsized":"2024-02-18T00:36:58.673034+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'73","last_scrub_stamp":"2024-02-18T00:36:37.532316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.532316+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":22,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"215'68","reported_seq":301,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673976+0000","last_change":"2024-02-18T00:36:06.662037+0000","last_active":"2024-02-18T00:36:58.673976+0000","last_peered":"2024-02-18T00:36:58.673976+0000","last_clean":"2024-02-18T00:36:58.673976+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:36:58.673976+0000","last_undegraded":"2024-02-18T00:36:58.673976+0000","last_fullsized":"2024-02-18T00:36:58.673976+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-18T00:36:02.645624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.645624+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"229'51","reported_seq":319,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671098+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:36:59.671098+0000","last_peered":"2024-02-18T00:36:59.671098+0000","last_clean":"2024-02-18T00:36:59.671098+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:36:59.671098+0000","last_undegraded":"2024-02-18T00:36:59.671098+0000","last_fullsized":"2024-02-18T00:36:59.671098+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"265'39","reported_seq":284,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671369+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:36:58.671369+0000","last_peered":"2024-02-18T00:36:58.671369+0000","last_clean":"2024-02-18T00:36:58.671369+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:36:58.671369+0000","last_undegraded":"2024-02-18T00:36:58.671369+0000","last_fullsized":"2024-02-18T00:36:58.671369+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"246'54","reported_seq":298,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678123+0000","last_change":"2024-02-18T00:36:11.810050+0000","last_active":"2024-02-18T00:36:59.678123+0000","last_peered":"2024-02-18T00:36:59.678123+0000","last_clean":"2024-02-18T00:36:59.678123+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:36:59.678123+0000","last_undegraded":"2024-02-18T00:36:59.678123+0000","last_fullsized":"2024-02-18T00:36:59.678123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'54","last_scrub_stamp":"2024-02-18T00:36:11.809925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.809925+0000","log_size":54,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":46,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":17,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"213'17","reported_seq":289,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678678+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:36:59.678678+0000","last_peered":"2024-02-18T00:36:59.678678+0000","last_clean":"2024-02-18T00:36:59.678678+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:36:59.678678+0000","last_undegraded":"2024-02-18T00:36:59.678678+0000","last_fullsized":"2024-02-18T00:36:59.678678+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"259'28","reported_seq":323,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671602+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:36:59.671602+0000","last_peered":"2024-02-18T00:36:59.671602+0000","last_clean":"2024-02-18T00:36:59.671602+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:36:59.671602+0000","last_undegraded":"2024-02-18T00:36:59.671602+0000","last_fullsized":"2024-02-18T00:36:59.671602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"288'81","reported_seq":341,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279389+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:37:00.279389+0000","last_peered":"2024-02-18T00:37:00.279389+0000","last_clean":"2024-02-18T00:37:00.279389+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:37:00.279389+0000","last_undegraded":"2024-02-18T00:37:00.279389+0000","last_fullsized":"2024-02-18T00:37:00.279389+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":81,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":51,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":324,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672094+0000","last_change":"2024-02-18T00:36:34.207908+0000","last_active":"2024-02-18T00:36:59.672094+0000","last_peered":"2024-02-18T00:36:59.672094+0000","last_clean":"2024-02-18T00:36:59.672094+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:36:59.672094+0000","last_undegraded":"2024-02-18T00:36:59.672094+0000","last_fullsized":"2024-02-18T00:36:59.672094+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:36:34.207840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.207840+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"209'56","reported_seq":336,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673922+0000","last_change":"2024-02-18T00:36:06.661736+0000","last_active":"2024-02-18T00:36:58.673922+0000","last_peered":"2024-02-18T00:36:58.673922+0000","last_clean":"2024-02-18T00:36:58.673922+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:36:58.673922+0000","last_undegraded":"2024-02-18T00:36:58.673922+0000","last_fullsized":"2024-02-18T00:36:58.673922+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'56","last_scrub_stamp":"2024-02-18T00:36:04.656413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.656413+0000","log_size":56,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":33,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":17,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"214'19","reported_seq":294,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672842+0000","last_change":"2024-02-18T00:36:06.658579+0000","last_active":"2024-02-18T00:36:58.672842+0000","last_peered":"2024-02-18T00:36:58.672842+0000","last_clean":"2024-02-18T00:36:58.672842+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:36:58.672842+0000","last_undegraded":"2024-02-18T00:36:58.672842+0000","last_fullsized":"2024-02-18T00:36:58.672842+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:35:53.563562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.563562+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"288'72","reported_seq":336,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671454+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:36:58.671454+0000","last_peered":"2024-02-18T00:36:58.671454+0000","last_clean":"2024-02-18T00:36:58.671454+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:36:58.671454+0000","last_undegraded":"2024-02-18T00:36:58.671454+0000","last_fullsized":"2024-02-18T00:36:58.671454+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"210'17","reported_seq":289,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278753+0000","last_change":"2024-02-18T00:36:43.345182+0000","last_active":"2024-02-18T00:37:00.278753+0000","last_peered":"2024-02-18T00:37:00.278753+0000","last_clean":"2024-02-18T00:37:00.278753+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:37:00.278753+0000","last_undegraded":"2024-02-18T00:37:00.278753+0000","last_fullsized":"2024-02-18T00:37:00.278753+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'17","last_scrub_stamp":"2024-02-18T00:36:43.345064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.345064+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":303,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672260+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:36:59.672260+0000","last_peered":"2024-02-18T00:36:59.672260+0000","last_clean":"2024-02-18T00:36:59.672260+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:36:59.672260+0000","last_undegraded":"2024-02-18T00:36:59.672260+0000","last_fullsized":"2024-02-18T00:36:59.672260+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"304'19","reported_seq":302,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696202+0000","last_change":"2024-02-18T00:36:06.659238+0000","last_active":"2024-02-18T00:36:59.696202+0000","last_peered":"2024-02-18T00:36:59.696202+0000","last_clean":"2024-02-18T00:36:59.696202+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:36:59.696202+0000","last_undegraded":"2024-02-18T00:36:59.696202+0000","last_fullsized":"2024-02-18T00:36:59.696202+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000019,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":368,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.676649+0000","last_change":"2024-02-18T00:36:34.017135+0000","last_active":"2024-02-18T00:36:59.676649+0000","last_peered":"2024-02-18T00:36:59.676649+0000","last_clean":"2024-02-18T00:36:59.676649+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:36:59.676649+0000","last_undegraded":"2024-02-18T00:36:59.676649+0000","last_fullsized":"2024-02-18T00:36:59.676649+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:36:34.017065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.017065+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":260,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695051+0000","last_change":"2024-02-18T00:36:37.504644+0000","last_active":"2024-02-18T00:36:59.695051+0000","last_peered":"2024-02-18T00:36:59.695051+0000","last_clean":"2024-02-18T00:36:59.695051+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:36:59.695051+0000","last_undegraded":"2024-02-18T00:36:59.695051+0000","last_fullsized":"2024-02-18T00:36:59.695051+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:37.504568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.504568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":380,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672076+0000","last_change":"2024-02-18T00:36:38.583124+0000","last_active":"2024-02-18T00:36:58.672076+0000","last_peered":"2024-02-18T00:36:58.672076+0000","last_clean":"2024-02-18T00:36:58.672076+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:36:58.672076+0000","last_undegraded":"2024-02-18T00:36:58.672076+0000","last_fullsized":"2024-02-18T00:36:58.672076+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'112","last_scrub_stamp":"2024-02-18T00:36:38.583056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.583056+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":254,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677489+0000","last_change":"2024-02-18T00:34:39.758860+0000","last_active":"2024-02-18T00:36:59.677489+0000","last_peered":"2024-02-18T00:36:59.677489+0000","last_clean":"2024-02-18T00:36:59.677489+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:36:59.677489+0000","last_undegraded":"2024-02-18T00:36:59.677489+0000","last_fullsized":"2024-02-18T00:36:59.677489+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.758794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.758794+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":315,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671313+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:36:59.671313+0000","last_peered":"2024-02-18T00:36:59.671313+0000","last_clean":"2024-02-18T00:36:59.671313+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:36:59.671313+0000","last_undegraded":"2024-02-18T00:36:59.671313+0000","last_fullsized":"2024-02-18T00:36:59.671313+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":212,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672108+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:36:59.672108+0000","last_peered":"2024-02-18T00:36:59.672108+0000","last_clean":"2024-02-18T00:36:59.672108+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:36:59.672108+0000","last_undegraded":"2024-02-18T00:36:59.672108+0000","last_fullsized":"2024-02-18T00:36:59.672108+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":369,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695162+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:36:59.695162+0000","last_peered":"2024-02-18T00:36:59.695162+0000","last_clean":"2024-02-18T00:36:59.695162+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:36:59.695162+0000","last_undegraded":"2024-02-18T00:36:59.695162+0000","last_fullsized":"2024-02-18T00:36:59.695162+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":215,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672324+0000","last_change":"2024-02-18T00:36:53.345938+0000","last_active":"2024-02-18T00:36:59.672324+0000","last_peered":"2024-02-18T00:36:59.672324+0000","last_clean":"2024-02-18T00:36:59.672324+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:36:59.672324+0000","last_undegraded":"2024-02-18T00:36:59.672324+0000","last_fullsized":"2024-02-18T00:36:59.672324+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.345872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":346,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695063+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:36:59.695063+0000","last_peered":"2024-02-18T00:36:59.695063+0000","last_clean":"2024-02-18T00:36:59.695063+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:36:59.695063+0000","last_undegraded":"2024-02-18T00:36:59.695063+0000","last_fullsized":"2024-02-18T00:36:59.695063+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":267,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672046+0000","last_change":"2024-02-18T00:36:39.168150+0000","last_active":"2024-02-18T00:36:59.672046+0000","last_peered":"2024-02-18T00:36:59.672046+0000","last_clean":"2024-02-18T00:36:59.672046+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:36:59.672046+0000","last_undegraded":"2024-02-18T00:36:59.672046+0000","last_fullsized":"2024-02-18T00:36:59.672046+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:36:39.168079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:39.168079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"213'17","reported_seq":295,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677404+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:36:59.677404+0000","last_peered":"2024-02-18T00:36:59.677404+0000","last_clean":"2024-02-18T00:36:59.677404+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:36:59.677404+0000","last_undegraded":"2024-02-18T00:36:59.677404+0000","last_fullsized":"2024-02-18T00:36:59.677404+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":260,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695663+0000","last_change":"2024-02-18T00:36:35.460784+0000","last_active":"2024-02-18T00:36:59.695663+0000","last_peered":"2024-02-18T00:36:59.695663+0000","last_clean":"2024-02-18T00:36:59.695663+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:36:59.695663+0000","last_undegraded":"2024-02-18T00:36:59.695663+0000","last_fullsized":"2024-02-18T00:36:59.695663+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.460703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.460703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":325,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672950+0000","last_change":"2024-02-18T00:36:41.624250+0000","last_active":"2024-02-18T00:36:58.672950+0000","last_peered":"2024-02-18T00:36:58.672950+0000","last_clean":"2024-02-18T00:36:58.672950+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:36:58.672950+0000","last_undegraded":"2024-02-18T00:36:58.672950+0000","last_fullsized":"2024-02-18T00:36:58.672950+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:36:41.624180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.624180+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":250,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678228+0000","last_change":"2024-02-18T00:36:43.933486+0000","last_active":"2024-02-18T00:36:59.678228+0000","last_peered":"2024-02-18T00:36:59.678228+0000","last_clean":"2024-02-18T00:36:59.678228+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:36:59.678228+0000","last_undegraded":"2024-02-18T00:36:59.678228+0000","last_fullsized":"2024-02-18T00:36:59.678228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.933414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.933414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":280,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696092+0000","last_change":"2024-02-18T00:36:06.659436+0000","last_active":"2024-02-18T00:36:59.696092+0000","last_peered":"2024-02-18T00:36:59.696092+0000","last_clean":"2024-02-18T00:36:59.696092+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:36:59.696092+0000","last_undegraded":"2024-02-18T00:36:59.696092+0000","last_fullsized":"2024-02-18T00:36:59.696092+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":253,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696037+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:36:59.696037+0000","last_peered":"2024-02-18T00:36:59.696037+0000","last_clean":"2024-02-18T00:36:59.696037+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:36:59.696037+0000","last_undegraded":"2024-02-18T00:36:59.696037+0000","last_fullsized":"2024-02-18T00:36:59.696037+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":288,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827834+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:36:58.827834+0000","last_peered":"2024-02-18T00:36:58.827834+0000","last_clean":"2024-02-18T00:36:58.827834+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:36:58.827834+0000","last_undegraded":"2024-02-18T00:36:58.827834+0000","last_fullsized":"2024-02-18T00:36:58.827834+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":331,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671765+0000","last_change":"2024-02-18T00:36:45.627771+0000","last_active":"2024-02-18T00:36:58.671765+0000","last_peered":"2024-02-18T00:36:58.671765+0000","last_clean":"2024-02-18T00:36:58.671765+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:36:58.671765+0000","last_undegraded":"2024-02-18T00:36:58.671765+0000","last_fullsized":"2024-02-18T00:36:58.671765+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.627700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.627700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":291,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695562+0000","last_change":"2024-02-18T00:36:06.657624+0000","last_active":"2024-02-18T00:36:59.695562+0000","last_peered":"2024-02-18T00:36:59.695562+0000","last_clean":"2024-02-18T00:36:59.695562+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:36:59.695562+0000","last_undegraded":"2024-02-18T00:36:59.695562+0000","last_fullsized":"2024-02-18T00:36:59.695562+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":277,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.826686+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:36:58.826686+0000","last_peered":"2024-02-18T00:36:58.826686+0000","last_clean":"2024-02-18T00:36:58.826686+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:36:58.826686+0000","last_undegraded":"2024-02-18T00:36:58.826686+0000","last_fullsized":"2024-02-18T00:36:58.826686+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":256,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279127+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:37:00.279127+0000","last_peered":"2024-02-18T00:37:00.279127+0000","last_clean":"2024-02-18T00:37:00.279127+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:37:00.279127+0000","last_undegraded":"2024-02-18T00:37:00.279127+0000","last_fullsized":"2024-02-18T00:37:00.279127+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":270,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695407+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:36:59.695407+0000","last_peered":"2024-02-18T00:36:59.695407+0000","last_clean":"2024-02-18T00:36:59.695407+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:36:59.695407+0000","last_undegraded":"2024-02-18T00:36:59.695407+0000","last_fullsized":"2024-02-18T00:36:59.695407+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":387,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.826760+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:36:58.826760+0000","last_peered":"2024-02-18T00:36:58.826760+0000","last_clean":"2024-02-18T00:36:58.826760+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:36:58.826760+0000","last_undegraded":"2024-02-18T00:36:58.826760+0000","last_fullsized":"2024-02-18T00:36:58.826760+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":319,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673847+0000","last_change":"2024-02-18T00:34:26.060181+0000","last_active":"2024-02-18T00:36:58.673847+0000","last_peered":"2024-02-18T00:36:58.673847+0000","last_clean":"2024-02-18T00:36:58.673847+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:36:58.673847+0000","last_undegraded":"2024-02-18T00:36:58.673847+0000","last_fullsized":"2024-02-18T00:36:58.673847+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.060096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.060096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":299,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827463+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:36:58.827463+0000","last_peered":"2024-02-18T00:36:58.827463+0000","last_clean":"2024-02-18T00:36:58.827463+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:36:58.827463+0000","last_undegraded":"2024-02-18T00:36:58.827463+0000","last_fullsized":"2024-02-18T00:36:58.827463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":248,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673520+0000","last_change":"2024-02-18T00:35:20.428964+0000","last_active":"2024-02-18T00:36:58.673520+0000","last_peered":"2024-02-18T00:36:58.673520+0000","last_clean":"2024-02-18T00:36:58.673520+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:36:58.673520+0000","last_undegraded":"2024-02-18T00:36:58.673520+0000","last_fullsized":"2024-02-18T00:36:58.673520+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:20.428893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:20.428893+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":356,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279097+0000","last_change":"2024-02-18T00:35:54.188262+0000","last_active":"2024-02-18T00:37:00.279097+0000","last_peered":"2024-02-18T00:37:00.279097+0000","last_clean":"2024-02-18T00:37:00.279097+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:37:00.279097+0000","last_undegraded":"2024-02-18T00:37:00.279097+0000","last_fullsized":"2024-02-18T00:37:00.279097+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:35:54.188188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.188188+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":313,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673644+0000","last_change":"2024-02-18T00:36:43.837775+0000","last_active":"2024-02-18T00:36:58.673644+0000","last_peered":"2024-02-18T00:36:58.673644+0000","last_clean":"2024-02-18T00:36:58.673644+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:36:58.673644+0000","last_undegraded":"2024-02-18T00:36:58.673644+0000","last_fullsized":"2024-02-18T00:36:58.673644+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.837702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.837702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":309,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671582+0000","last_change":"2024-02-18T00:36:56.601833+0000","last_active":"2024-02-18T00:36:58.671582+0000","last_peered":"2024-02-18T00:36:58.671582+0000","last_clean":"2024-02-18T00:36:58.671582+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:36:58.671582+0000","last_undegraded":"2024-02-18T00:36:58.671582+0000","last_fullsized":"2024-02-18T00:36:58.671582+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.601727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.601727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":318,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278470+0000","last_change":"2024-02-18T00:35:42.683115+0000","last_active":"2024-02-18T00:37:00.278470+0000","last_peered":"2024-02-18T00:37:00.278470+0000","last_clean":"2024-02-18T00:37:00.278470+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:37:00.278470+0000","last_undegraded":"2024-02-18T00:37:00.278470+0000","last_fullsized":"2024-02-18T00:37:00.278470+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":256,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678108+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:36:59.678108+0000","last_peered":"2024-02-18T00:36:59.678108+0000","last_clean":"2024-02-18T00:36:59.678108+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:36:59.678108+0000","last_undegraded":"2024-02-18T00:36:59.678108+0000","last_fullsized":"2024-02-18T00:36:59.678108+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":389,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696118+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:36:59.696118+0000","last_peered":"2024-02-18T00:36:59.696118+0000","last_clean":"2024-02-18T00:36:59.696118+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:36:59.696118+0000","last_undegraded":"2024-02-18T00:36:59.696118+0000","last_fullsized":"2024-02-18T00:36:59.696118+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":313,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672341+0000","last_change":"2024-02-18T00:36:49.680480+0000","last_active":"2024-02-18T00:36:58.672341+0000","last_peered":"2024-02-18T00:36:58.672341+0000","last_clean":"2024-02-18T00:36:58.672341+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:36:58.672341+0000","last_undegraded":"2024-02-18T00:36:58.672341+0000","last_fullsized":"2024-02-18T00:36:58.672341+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.680414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":309,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.670726+0000","last_change":"2024-02-18T00:37:00.670726+0000","last_active":"2024-02-18T00:37:00.670726+0000","last_peered":"2024-02-18T00:37:00.670726+0000","last_clean":"2024-02-18T00:37:00.670726+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:37:00.670726+0000","last_undegraded":"2024-02-18T00:37:00.670726+0000","last_fullsized":"2024-02-18T00:37:00.670726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.670646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.670646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":208,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672485+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:36:58.672485+0000","last_peered":"2024-02-18T00:36:58.672485+0000","last_clean":"2024-02-18T00:36:58.672485+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:36:58.672485+0000","last_undegraded":"2024-02-18T00:36:58.672485+0000","last_fullsized":"2024-02-18T00:36:58.672485+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":245,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.826937+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:36:58.826937+0000","last_peered":"2024-02-18T00:36:58.826937+0000","last_clean":"2024-02-18T00:36:58.826937+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:36:58.826937+0000","last_undegraded":"2024-02-18T00:36:58.826937+0000","last_fullsized":"2024-02-18T00:36:58.826937+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":340,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671986+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:36:59.671986+0000","last_peered":"2024-02-18T00:36:59.671986+0000","last_clean":"2024-02-18T00:36:59.671986+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:36:59.671986+0000","last_undegraded":"2024-02-18T00:36:59.671986+0000","last_fullsized":"2024-02-18T00:36:59.671986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":214,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671642+0000","last_change":"2024-02-18T00:36:58.282770+0000","last_active":"2024-02-18T00:36:59.671642+0000","last_peered":"2024-02-18T00:36:59.671642+0000","last_clean":"2024-02-18T00:36:59.671642+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:36:59.671642+0000","last_undegraded":"2024-02-18T00:36:59.671642+0000","last_fullsized":"2024-02-18T00:36:59.671642+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.282673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.282673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":309,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.000774+0000","last_change":"2024-02-18T00:36:59.000753+0000","last_active":"2024-02-18T00:36:59.000774+0000","last_peered":"2024-02-18T00:36:59.000774+0000","last_clean":"2024-02-18T00:36:59.000774+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:36:59.000774+0000","last_undegraded":"2024-02-18T00:36:59.000774+0000","last_fullsized":"2024-02-18T00:36:59.000774+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.000690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.000690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":307,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673286+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:36:58.673286+0000","last_peered":"2024-02-18T00:36:58.673286+0000","last_clean":"2024-02-18T00:36:58.673286+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:36:58.673286+0000","last_undegraded":"2024-02-18T00:36:58.673286+0000","last_fullsized":"2024-02-18T00:36:58.673286+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":92,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671107+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:36:58.671107+0000","last_peered":"2024-02-18T00:36:58.671107+0000","last_clean":"2024-02-18T00:36:58.671107+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:36:58.671107+0000","last_undegraded":"2024-02-18T00:36:58.671107+0000","last_fullsized":"2024-02-18T00:36:58.671107+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":322,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671523+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:36:59.671523+0000","last_peered":"2024-02-18T00:36:59.671523+0000","last_clean":"2024-02-18T00:36:59.671523+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:36:59.671523+0000","last_undegraded":"2024-02-18T00:36:59.671523+0000","last_fullsized":"2024-02-18T00:36:59.671523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":322,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674658+0000","last_change":"2024-02-18T00:36:38.883486+0000","last_active":"2024-02-18T00:36:58.674658+0000","last_peered":"2024-02-18T00:36:58.674658+0000","last_clean":"2024-02-18T00:36:58.674658+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:36:58.674658+0000","last_undegraded":"2024-02-18T00:36:58.674658+0000","last_fullsized":"2024-02-18T00:36:58.674658+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:38.883391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.883391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":421,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672729+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:36:58.672729+0000","last_peered":"2024-02-18T00:36:58.672729+0000","last_clean":"2024-02-18T00:36:58.672729+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:36:58.672729+0000","last_undegraded":"2024-02-18T00:36:58.672729+0000","last_fullsized":"2024-02-18T00:36:58.672729+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":303,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672496+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:36:59.672496+0000","last_peered":"2024-02-18T00:36:59.672496+0000","last_clean":"2024-02-18T00:36:59.672496+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:36:59.672496+0000","last_undegraded":"2024-02-18T00:36:59.672496+0000","last_fullsized":"2024-02-18T00:36:59.672496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":248,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674793+0000","last_change":"2024-02-18T00:35:29.520701+0000","last_active":"2024-02-18T00:36:58.674793+0000","last_peered":"2024-02-18T00:36:58.674793+0000","last_clean":"2024-02-18T00:36:58.674793+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:36:58.674793+0000","last_undegraded":"2024-02-18T00:36:58.674793+0000","last_fullsized":"2024-02-18T00:36:58.674793+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.520632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.520632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":311,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828266+0000","last_change":"2024-02-18T00:36:33.853442+0000","last_active":"2024-02-18T00:36:58.828266+0000","last_peered":"2024-02-18T00:36:58.828266+0000","last_clean":"2024-02-18T00:36:58.828266+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:36:58.828266+0000","last_undegraded":"2024-02-18T00:36:58.828266+0000","last_fullsized":"2024-02-18T00:36:58.828266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:36:33.853337+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:36:33.853337+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":331,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673206+0000","last_change":"2024-02-18T00:36:08.716901+0000","last_active":"2024-02-18T00:36:58.673206+0000","last_peered":"2024-02-18T00:36:58.673206+0000","last_clean":"2024-02-18T00:36:58.673206+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:36:58.673206+0000","last_undegraded":"2024-02-18T00:36:58.673206+0000","last_fullsized":"2024-02-18T00:36:58.673206+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.716777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.716777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":406,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672493+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:36:59.672493+0000","last_peered":"2024-02-18T00:36:59.672493+0000","last_clean":"2024-02-18T00:36:59.672493+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:36:59.672493+0000","last_undegraded":"2024-02-18T00:36:59.672493+0000","last_fullsized":"2024-02-18T00:36:59.672493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":299,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828148+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:36:58.828148+0000","last_peered":"2024-02-18T00:36:58.828148+0000","last_clean":"2024-02-18T00:36:58.828148+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:36:58.828148+0000","last_undegraded":"2024-02-18T00:36:58.828148+0000","last_fullsized":"2024-02-18T00:36:58.828148+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":249,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671406+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:36:59.671406+0000","last_peered":"2024-02-18T00:36:59.671406+0000","last_clean":"2024-02-18T00:36:59.671406+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:36:59.671406+0000","last_undegraded":"2024-02-18T00:36:59.671406+0000","last_fullsized":"2024-02-18T00:36:59.671406+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":369,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672358+0000","last_change":"2024-02-18T00:36:43.283359+0000","last_active":"2024-02-18T00:36:59.672358+0000","last_peered":"2024-02-18T00:36:59.672358+0000","last_clean":"2024-02-18T00:36:59.672358+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:36:59.672358+0000","last_undegraded":"2024-02-18T00:36:59.672358+0000","last_fullsized":"2024-02-18T00:36:59.672358+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:43.283235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.283235+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":283,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278058+0000","last_change":"2024-02-18T00:36:57.321994+0000","last_active":"2024-02-18T00:37:00.278058+0000","last_peered":"2024-02-18T00:37:00.278058+0000","last_clean":"2024-02-18T00:37:00.278058+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:37:00.278058+0000","last_undegraded":"2024-02-18T00:37:00.278058+0000","last_fullsized":"2024-02-18T00:37:00.278058+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.321930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.321930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":301,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828538+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:36:58.828538+0000","last_peered":"2024-02-18T00:36:58.828538+0000","last_clean":"2024-02-18T00:36:58.828538+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:36:58.828538+0000","last_undegraded":"2024-02-18T00:36:58.828538+0000","last_fullsized":"2024-02-18T00:36:58.828538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":301,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678830+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:36:59.678830+0000","last_peered":"2024-02-18T00:36:59.678830+0000","last_clean":"2024-02-18T00:36:59.678830+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:36:59.678830+0000","last_undegraded":"2024-02-18T00:36:59.678830+0000","last_fullsized":"2024-02-18T00:36:59.678830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":249,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671695+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:36:59.671695+0000","last_peered":"2024-02-18T00:36:59.671695+0000","last_clean":"2024-02-18T00:36:59.671695+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:36:59.671695+0000","last_undegraded":"2024-02-18T00:36:59.671695+0000","last_fullsized":"2024-02-18T00:36:59.671695+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":410,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677860+0000","last_change":"2024-02-18T00:36:53.910404+0000","last_active":"2024-02-18T00:36:59.677860+0000","last_peered":"2024-02-18T00:36:59.677860+0000","last_clean":"2024-02-18T00:36:59.677860+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:36:59.677860+0000","last_undegraded":"2024-02-18T00:36:59.677860+0000","last_fullsized":"2024-02-18T00:36:59.677860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.910333+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":348,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671275+0000","last_change":"2024-02-18T00:36:50.262681+0000","last_active":"2024-02-18T00:36:59.671275+0000","last_peered":"2024-02-18T00:36:59.671275+0000","last_clean":"2024-02-18T00:36:59.671275+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:36:59.671275+0000","last_undegraded":"2024-02-18T00:36:59.671275+0000","last_fullsized":"2024-02-18T00:36:59.671275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:36:50.262602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.262602+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":331,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696326+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:36:59.696326+0000","last_peered":"2024-02-18T00:36:59.696326+0000","last_clean":"2024-02-18T00:36:59.696326+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:36:59.696326+0000","last_undegraded":"2024-02-18T00:36:59.696326+0000","last_fullsized":"2024-02-18T00:36:59.696326+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":208,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695787+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:36:59.695787+0000","last_peered":"2024-02-18T00:36:59.695787+0000","last_clean":"2024-02-18T00:36:59.695787+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:36:59.695787+0000","last_undegraded":"2024-02-18T00:36:59.695787+0000","last_fullsized":"2024-02-18T00:36:59.695787+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":245,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827975+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:36:58.827975+0000","last_peered":"2024-02-18T00:36:58.827975+0000","last_clean":"2024-02-18T00:36:58.827975+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:36:58.827975+0000","last_undegraded":"2024-02-18T00:36:58.827975+0000","last_fullsized":"2024-02-18T00:36:58.827975+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":319,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671838+0000","last_change":"2024-02-18T00:36:45.308400+0000","last_active":"2024-02-18T00:36:59.671838+0000","last_peered":"2024-02-18T00:36:59.671838+0000","last_clean":"2024-02-18T00:36:59.671838+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:36:59.671838+0000","last_undegraded":"2024-02-18T00:36:59.671838+0000","last_fullsized":"2024-02-18T00:36:59.671838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.308300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.308300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":408,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677772+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:36:59.677772+0000","last_peered":"2024-02-18T00:36:59.677772+0000","last_clean":"2024-02-18T00:36:59.677772+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:36:59.677772+0000","last_undegraded":"2024-02-18T00:36:59.677772+0000","last_fullsized":"2024-02-18T00:36:59.677772+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":322,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673595+0000","last_change":"2024-02-18T00:36:09.647995+0000","last_active":"2024-02-18T00:36:58.673595+0000","last_peered":"2024-02-18T00:36:58.673595+0000","last_clean":"2024-02-18T00:36:58.673595+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:36:58.673595+0000","last_undegraded":"2024-02-18T00:36:58.673595+0000","last_fullsized":"2024-02-18T00:36:58.673595+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.647928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":303,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671762+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:36:59.671762+0000","last_peered":"2024-02-18T00:36:59.671762+0000","last_clean":"2024-02-18T00:36:59.671762+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:36:59.671762+0000","last_undegraded":"2024-02-18T00:36:59.671762+0000","last_fullsized":"2024-02-18T00:36:59.671762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":247,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678965+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:36:59.678965+0000","last_peered":"2024-02-18T00:36:59.678965+0000","last_clean":"2024-02-18T00:36:59.678965+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:36:59.678965+0000","last_undegraded":"2024-02-18T00:36:59.678965+0000","last_fullsized":"2024-02-18T00:36:59.678965+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":304,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695784+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:36:59.695784+0000","last_peered":"2024-02-18T00:36:59.695784+0000","last_clean":"2024-02-18T00:36:59.695784+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:36:59.695784+0000","last_undegraded":"2024-02-18T00:36:59.695784+0000","last_fullsized":"2024-02-18T00:36:59.695784+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":384,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278974+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:37:00.278974+0000","last_peered":"2024-02-18T00:37:00.278974+0000","last_clean":"2024-02-18T00:37:00.278974+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:37:00.278974+0000","last_undegraded":"2024-02-18T00:37:00.278974+0000","last_fullsized":"2024-02-18T00:37:00.278974+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":215,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279320+0000","last_change":"2024-02-18T00:34:32.021327+0000","last_active":"2024-02-18T00:37:00.279320+0000","last_peered":"2024-02-18T00:37:00.279320+0000","last_clean":"2024-02-18T00:37:00.279320+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:37:00.279320+0000","last_undegraded":"2024-02-18T00:37:00.279320+0000","last_fullsized":"2024-02-18T00:37:00.279320+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:34:32.021219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.021219+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":307,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.979666+0000","last_change":"2024-02-18T00:36:59.979666+0000","last_active":"2024-02-18T00:36:59.979666+0000","last_peered":"2024-02-18T00:36:59.979666+0000","last_clean":"2024-02-18T00:36:59.979666+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:36:59.979666+0000","last_undegraded":"2024-02-18T00:36:59.979666+0000","last_fullsized":"2024-02-18T00:36:59.979666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.979565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.979565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":247,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.679160+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:36:59.679160+0000","last_peered":"2024-02-18T00:36:59.679160+0000","last_clean":"2024-02-18T00:36:59.679160+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:36:59.679160+0000","last_undegraded":"2024-02-18T00:36:59.679160+0000","last_fullsized":"2024-02-18T00:36:59.679160+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":301,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828677+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:36:58.828677+0000","last_peered":"2024-02-18T00:36:58.828677+0000","last_clean":"2024-02-18T00:36:58.828677+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:36:58.828677+0000","last_undegraded":"2024-02-18T00:36:58.828677+0000","last_fullsized":"2024-02-18T00:36:58.828677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":322,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279556+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:37:00.279556+0000","last_peered":"2024-02-18T00:37:00.279556+0000","last_clean":"2024-02-18T00:37:00.279556+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:37:00.279556+0000","last_undegraded":"2024-02-18T00:37:00.279556+0000","last_fullsized":"2024-02-18T00:37:00.279556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":284,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674351+0000","last_change":"2024-02-18T00:36:34.800419+0000","last_active":"2024-02-18T00:36:58.674351+0000","last_peered":"2024-02-18T00:36:58.674351+0000","last_clean":"2024-02-18T00:36:58.674351+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:36:58.674351+0000","last_undegraded":"2024-02-18T00:36:58.674351+0000","last_fullsized":"2024-02-18T00:36:58.674351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:34.800327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.800327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":305,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.277793+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:37:00.277793+0000","last_peered":"2024-02-18T00:37:00.277793+0000","last_clean":"2024-02-18T00:37:00.277793+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:37:00.277793+0000","last_undegraded":"2024-02-18T00:37:00.277793+0000","last_fullsized":"2024-02-18T00:37:00.277793+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":302,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674662+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:36:58.674662+0000","last_peered":"2024-02-18T00:36:58.674662+0000","last_clean":"2024-02-18T00:36:58.674662+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:36:58.674662+0000","last_undegraded":"2024-02-18T00:36:58.674662+0000","last_fullsized":"2024-02-18T00:36:58.674662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":256,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278123+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:37:00.278123+0000","last_peered":"2024-02-18T00:37:00.278123+0000","last_clean":"2024-02-18T00:37:00.278123+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:37:00.278123+0000","last_undegraded":"2024-02-18T00:37:00.278123+0000","last_fullsized":"2024-02-18T00:37:00.278123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":208,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672248+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:36:58.672248+0000","last_peered":"2024-02-18T00:36:58.672248+0000","last_clean":"2024-02-18T00:36:58.672248+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:36:58.672248+0000","last_undegraded":"2024-02-18T00:36:58.672248+0000","last_fullsized":"2024-02-18T00:36:58.672248+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":363,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696194+0000","last_change":"2024-02-18T00:36:50.759133+0000","last_active":"2024-02-18T00:36:59.696194+0000","last_peered":"2024-02-18T00:36:59.696194+0000","last_clean":"2024-02-18T00:36:59.696194+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:36:59.696194+0000","last_undegraded":"2024-02-18T00:36:59.696194+0000","last_fullsized":"2024-02-18T00:36:59.696194+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:36:50.759019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.759019+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":241,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278177+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:37:00.278177+0000","last_peered":"2024-02-18T00:37:00.278177+0000","last_clean":"2024-02-18T00:37:00.278177+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:37:00.278177+0000","last_undegraded":"2024-02-18T00:37:00.278177+0000","last_fullsized":"2024-02-18T00:37:00.278177+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":323,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827351+0000","last_change":"2024-02-18T00:34:35.765124+0000","last_active":"2024-02-18T00:36:58.827351+0000","last_peered":"2024-02-18T00:36:58.827351+0000","last_clean":"2024-02-18T00:36:58.827351+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:36:58.827351+0000","last_undegraded":"2024-02-18T00:36:58.827351+0000","last_fullsized":"2024-02-18T00:36:58.827351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.765059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.765059+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":306,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.676719+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:36:59.676719+0000","last_peered":"2024-02-18T00:36:59.676719+0000","last_clean":"2024-02-18T00:36:59.676719+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:36:59.676719+0000","last_undegraded":"2024-02-18T00:36:59.676719+0000","last_fullsized":"2024-02-18T00:36:59.676719+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":304,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828778+0000","last_change":"2024-02-18T00:36:42.828611+0000","last_active":"2024-02-18T00:36:58.828778+0000","last_peered":"2024-02-18T00:36:58.828778+0000","last_clean":"2024-02-18T00:36:58.828778+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:36:58.828778+0000","last_undegraded":"2024-02-18T00:36:58.828778+0000","last_fullsized":"2024-02-18T00:36:58.828778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.828542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.828542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":247,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278761+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:37:00.278761+0000","last_peered":"2024-02-18T00:37:00.278761+0000","last_clean":"2024-02-18T00:37:00.278761+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:37:00.278761+0000","last_undegraded":"2024-02-18T00:37:00.278761+0000","last_fullsized":"2024-02-18T00:37:00.278761+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":318,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673946+0000","last_change":"2024-02-18T00:36:57.633211+0000","last_active":"2024-02-18T00:36:58.673946+0000","last_peered":"2024-02-18T00:36:58.673946+0000","last_clean":"2024-02-18T00:36:58.673946+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:36:58.673946+0000","last_undegraded":"2024-02-18T00:36:58.673946+0000","last_fullsized":"2024-02-18T00:36:58.673946+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.633128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.633128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":353,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674179+0000","last_change":"2024-02-18T00:36:45.862082+0000","last_active":"2024-02-18T00:36:58.674179+0000","last_peered":"2024-02-18T00:36:58.674179+0000","last_clean":"2024-02-18T00:36:58.674179+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:36:58.674179+0000","last_undegraded":"2024-02-18T00:36:58.674179+0000","last_fullsized":"2024-02-18T00:36:58.674179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:36:45.861971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.861971+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":322,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671481+0000","last_change":"2024-02-18T00:36:36.500976+0000","last_active":"2024-02-18T00:36:58.671481+0000","last_peered":"2024-02-18T00:36:58.671481+0000","last_clean":"2024-02-18T00:36:58.671481+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:36:58.671481+0000","last_undegraded":"2024-02-18T00:36:58.671481+0000","last_fullsized":"2024-02-18T00:36:58.671481+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:36.500906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.500906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":773,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.734009+0000","last_change":"2024-02-18T00:36:06.641279+0000","last_active":"2024-02-18T00:36:59.734009+0000","last_peered":"2024-02-18T00:36:59.734009+0000","last_clean":"2024-02-18T00:36:59.734009+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:36:59.734009+0000","last_undegraded":"2024-02-18T00:36:59.734009+0000","last_fullsized":"2024-02-18T00:36:59.734009+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:36:06.641185+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.641185+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":457,"num_read_kb":452,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":250,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696302+0000","last_change":"2024-02-18T00:36:42.643527+0000","last_active":"2024-02-18T00:36:59.696302+0000","last_peered":"2024-02-18T00:36:59.696302+0000","last_clean":"2024-02-18T00:36:59.696302+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:36:59.696302+0000","last_undegraded":"2024-02-18T00:36:59.696302+0000","last_fullsized":"2024-02-18T00:36:59.696302+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:36:42.643433+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.643433+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":258,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671792+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:36:59.671792+0000","last_peered":"2024-02-18T00:36:59.671792+0000","last_clean":"2024-02-18T00:36:59.671792+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:36:59.671792+0000","last_undegraded":"2024-02-18T00:36:59.671792+0000","last_fullsized":"2024-02-18T00:36:59.671792+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":307,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279325+0000","last_change":"2024-02-18T00:36:56.302683+0000","last_active":"2024-02-18T00:37:00.279325+0000","last_peered":"2024-02-18T00:37:00.279325+0000","last_clean":"2024-02-18T00:37:00.279325+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:37:00.279325+0000","last_undegraded":"2024-02-18T00:37:00.279325+0000","last_fullsized":"2024-02-18T00:37:00.279325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.302615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.302615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":129,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827393+0000","last_change":"2024-02-18T00:36:41.801219+0000","last_active":"2024-02-18T00:36:58.827393+0000","last_peered":"2024-02-18T00:36:58.827393+0000","last_clean":"2024-02-18T00:36:58.827393+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:36:58.827393+0000","last_undegraded":"2024-02-18T00:36:58.827393+0000","last_fullsized":"2024-02-18T00:36:58.827393+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-18T00:36:41.801144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.801144+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":319,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827647+0000","last_change":"2024-02-18T00:36:57.791451+0000","last_active":"2024-02-18T00:36:58.827647+0000","last_peered":"2024-02-18T00:36:58.827647+0000","last_clean":"2024-02-18T00:36:58.827647+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:36:58.827647+0000","last_undegraded":"2024-02-18T00:36:58.827647+0000","last_fullsized":"2024-02-18T00:36:58.827647+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.791347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.791347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":310,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678472+0000","last_change":"2024-02-18T00:36:49.976027+0000","last_active":"2024-02-18T00:36:59.678472+0000","last_peered":"2024-02-18T00:36:59.678472+0000","last_clean":"2024-02-18T00:36:59.678472+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:36:59.678472+0000","last_undegraded":"2024-02-18T00:36:59.678472+0000","last_fullsized":"2024-02-18T00:36:59.678472+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.975957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.975957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":247,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279703+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:37:00.279703+0000","last_peered":"2024-02-18T00:37:00.279703+0000","last_clean":"2024-02-18T00:37:00.279703+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:37:00.279703+0000","last_undegraded":"2024-02-18T00:37:00.279703+0000","last_fullsized":"2024-02-18T00:37:00.279703+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":403,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279147+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:37:00.279147+0000","last_peered":"2024-02-18T00:37:00.279147+0000","last_clean":"2024-02-18T00:37:00.279147+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:37:00.279147+0000","last_undegraded":"2024-02-18T00:37:00.279147+0000","last_fullsized":"2024-02-18T00:37:00.279147+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":306,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671794+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:36:58.671794+0000","last_peered":"2024-02-18T00:36:58.671794+0000","last_clean":"2024-02-18T00:36:58.671794+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:36:58.671794+0000","last_undegraded":"2024-02-18T00:36:58.671794+0000","last_fullsized":"2024-02-18T00:36:58.671794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":336,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674535+0000","last_change":"2024-02-18T00:36:53.925296+0000","last_active":"2024-02-18T00:36:58.674535+0000","last_peered":"2024-02-18T00:36:58.674535+0000","last_clean":"2024-02-18T00:36:58.674535+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:36:58.674535+0000","last_undegraded":"2024-02-18T00:36:58.674535+0000","last_fullsized":"2024-02-18T00:36:58.674535+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.925187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.925187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":300,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.277981+0000","last_change":"2024-02-18T00:36:46.369602+0000","last_active":"2024-02-18T00:37:00.277981+0000","last_peered":"2024-02-18T00:37:00.277981+0000","last_clean":"2024-02-18T00:37:00.277981+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:37:00.277981+0000","last_undegraded":"2024-02-18T00:37:00.277981+0000","last_fullsized":"2024-02-18T00:37:00.277981+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:46.369503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.369503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":249,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671643+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:36:59.671643+0000","last_peered":"2024-02-18T00:36:59.671643+0000","last_clean":"2024-02-18T00:36:59.671643+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:36:59.671643+0000","last_undegraded":"2024-02-18T00:36:59.671643+0000","last_fullsized":"2024-02-18T00:36:59.671643+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":304,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672336+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:36:58.672336+0000","last_peered":"2024-02-18T00:36:58.672336+0000","last_clean":"2024-02-18T00:36:58.672336+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:36:58.672336+0000","last_undegraded":"2024-02-18T00:36:58.672336+0000","last_fullsized":"2024-02-18T00:36:58.672336+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":360,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671305+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:36:59.671305+0000","last_peered":"2024-02-18T00:36:59.671305+0000","last_clean":"2024-02-18T00:36:59.671305+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:36:59.671305+0000","last_undegraded":"2024-02-18T00:36:59.671305+0000","last_fullsized":"2024-02-18T00:36:59.671305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":342,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672278+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:36:59.672278+0000","last_peered":"2024-02-18T00:36:59.672278+0000","last_clean":"2024-02-18T00:36:59.672278+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:36:59.672278+0000","last_undegraded":"2024-02-18T00:36:59.672278+0000","last_fullsized":"2024-02-18T00:36:59.672278+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":310,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678956+0000","last_change":"2024-02-18T00:34:26.701231+0000","last_active":"2024-02-18T00:36:59.678956+0000","last_peered":"2024-02-18T00:36:59.678956+0000","last_clean":"2024-02-18T00:36:59.678956+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:36:59.678956+0000","last_undegraded":"2024-02-18T00:36:59.678956+0000","last_fullsized":"2024-02-18T00:36:59.678956+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.701162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.701162+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":249,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671444+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:36:59.671444+0000","last_peered":"2024-02-18T00:36:59.671444+0000","last_clean":"2024-02-18T00:36:59.671444+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:36:59.671444+0000","last_undegraded":"2024-02-18T00:36:59.671444+0000","last_fullsized":"2024-02-18T00:36:59.671444+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":373,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279610+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:37:00.279610+0000","last_peered":"2024-02-18T00:37:00.279610+0000","last_clean":"2024-02-18T00:37:00.279610+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:37:00.279610+0000","last_undegraded":"2024-02-18T00:37:00.279610+0000","last_fullsized":"2024-02-18T00:37:00.279610+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":308,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672254+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:36:58.672254+0000","last_peered":"2024-02-18T00:36:58.672254+0000","last_clean":"2024-02-18T00:36:58.672254+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:36:58.672254+0000","last_undegraded":"2024-02-18T00:36:58.672254+0000","last_fullsized":"2024-02-18T00:36:58.672254+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":317,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678050+0000","last_change":"2024-02-18T00:36:55.861115+0000","last_active":"2024-02-18T00:36:59.678050+0000","last_peered":"2024-02-18T00:36:59.678050+0000","last_clean":"2024-02-18T00:36:59.678050+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:36:59.678050+0000","last_undegraded":"2024-02-18T00:36:59.678050+0000","last_fullsized":"2024-02-18T00:36:59.678050+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:55.861044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.861044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":311,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672199+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:36:58.672199+0000","last_peered":"2024-02-18T00:36:58.672199+0000","last_clean":"2024-02-18T00:36:58.672199+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:36:58.672199+0000","last_undegraded":"2024-02-18T00:36:58.672199+0000","last_fullsized":"2024-02-18T00:36:58.672199+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":247,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278651+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:37:00.278651+0000","last_peered":"2024-02-18T00:37:00.278651+0000","last_clean":"2024-02-18T00:37:00.278651+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:37:00.278651+0000","last_undegraded":"2024-02-18T00:37:00.278651+0000","last_fullsized":"2024-02-18T00:37:00.278651+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":372,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677903+0000","last_change":"2024-02-18T00:36:51.948976+0000","last_active":"2024-02-18T00:36:59.677903+0000","last_peered":"2024-02-18T00:36:59.677903+0000","last_clean":"2024-02-18T00:36:59.677903+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:36:59.677903+0000","last_undegraded":"2024-02-18T00:36:59.677903+0000","last_fullsized":"2024-02-18T00:36:59.677903+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:36:51.948907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.948907+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":313,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695252+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:36:59.695252+0000","last_peered":"2024-02-18T00:36:59.695252+0000","last_clean":"2024-02-18T00:36:59.695252+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:36:59.695252+0000","last_undegraded":"2024-02-18T00:36:59.695252+0000","last_fullsized":"2024-02-18T00:36:59.695252+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":301,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827763+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:36:58.827763+0000","last_peered":"2024-02-18T00:36:58.827763+0000","last_clean":"2024-02-18T00:36:58.827763+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:36:58.827763+0000","last_undegraded":"2024-02-18T00:36:58.827763+0000","last_fullsized":"2024-02-18T00:36:58.827763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":213,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696335+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:36:59.696335+0000","last_peered":"2024-02-18T00:36:59.696335+0000","last_clean":"2024-02-18T00:36:59.696335+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:36:59.696335+0000","last_undegraded":"2024-02-18T00:36:59.696335+0000","last_fullsized":"2024-02-18T00:36:59.696335+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":257,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671691+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:36:58.671691+0000","last_peered":"2024-02-18T00:36:58.671691+0000","last_clean":"2024-02-18T00:36:58.671691+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:36:58.671691+0000","last_undegraded":"2024-02-18T00:36:58.671691+0000","last_fullsized":"2024-02-18T00:36:58.671691+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":380,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.679095+0000","last_change":"2024-02-18T00:36:58.819567+0000","last_active":"2024-02-18T00:36:59.679095+0000","last_peered":"2024-02-18T00:36:59.679095+0000","last_clean":"2024-02-18T00:36:59.679095+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:36:59.679095+0000","last_undegraded":"2024-02-18T00:36:59.679095+0000","last_fullsized":"2024-02-18T00:36:59.679095+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:36:58.819503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.819503+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":323,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827525+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:36:58.827525+0000","last_peered":"2024-02-18T00:36:58.827525+0000","last_clean":"2024-02-18T00:36:58.827525+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:36:58.827525+0000","last_undegraded":"2024-02-18T00:36:58.827525+0000","last_fullsized":"2024-02-18T00:36:58.827525+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":402,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828100+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:36:58.828100+0000","last_peered":"2024-02-18T00:36:58.828100+0000","last_clean":"2024-02-18T00:36:58.828100+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:36:58.828100+0000","last_undegraded":"2024-02-18T00:36:58.828100+0000","last_fullsized":"2024-02-18T00:36:58.828100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":314,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279579+0000","last_change":"2024-02-18T00:36:44.367116+0000","last_active":"2024-02-18T00:37:00.279579+0000","last_peered":"2024-02-18T00:37:00.279579+0000","last_clean":"2024-02-18T00:37:00.279579+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:37:00.279579+0000","last_undegraded":"2024-02-18T00:37:00.279579+0000","last_fullsized":"2024-02-18T00:37:00.279579+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.367032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.367032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":257,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695042+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:36:59.695042+0000","last_peered":"2024-02-18T00:36:59.695042+0000","last_clean":"2024-02-18T00:36:59.695042+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:36:59.695042+0000","last_undegraded":"2024-02-18T00:36:59.695042+0000","last_fullsized":"2024-02-18T00:36:59.695042+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":355,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827971+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:36:58.827971+0000","last_peered":"2024-02-18T00:36:58.827971+0000","last_clean":"2024-02-18T00:36:58.827971+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:36:58.827971+0000","last_undegraded":"2024-02-18T00:36:58.827971+0000","last_fullsized":"2024-02-18T00:36:58.827971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":306,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279043+0000","last_change":"2024-02-18T00:34:15.935045+0000","last_active":"2024-02-18T00:37:00.279043+0000","last_peered":"2024-02-18T00:37:00.279043+0000","last_clean":"2024-02-18T00:37:00.279043+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:37:00.279043+0000","last_undegraded":"2024-02-18T00:37:00.279043+0000","last_fullsized":"2024-02-18T00:37:00.279043+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:15.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:34:15.934970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"341'10","reported_seq":423,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.830592+0000","last_change":"2024-02-18T00:36:59.830567+0000","last_active":"2024-02-18T00:36:59.830592+0000","last_peered":"2024-02-18T00:36:59.830592+0000","last_clean":"2024-02-18T00:36:59.830592+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:36:59.830592+0000","last_undegraded":"2024-02-18T00:36:59.830592+0000","last_fullsized":"2024-02-18T00:36:59.830592+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'10","last_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_deep_scrub":"341'10","last_deep_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.830497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":303,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672357+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:36:59.672357+0000","last_peered":"2024-02-18T00:36:59.672357+0000","last_clean":"2024-02-18T00:36:59.672357+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:36:59.672357+0000","last_undegraded":"2024-02-18T00:36:59.672357+0000","last_fullsized":"2024-02-18T00:36:59.672357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":245,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.828839+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:36:58.828839+0000","last_peered":"2024-02-18T00:36:58.828839+0000","last_clean":"2024-02-18T00:36:58.828839+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:36:58.828839+0000","last_undegraded":"2024-02-18T00:36:58.828839+0000","last_fullsized":"2024-02-18T00:36:58.828839+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":327,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674706+0000","last_change":"2024-02-18T00:36:55.970729+0000","last_active":"2024-02-18T00:36:58.674706+0000","last_peered":"2024-02-18T00:36:58.674706+0000","last_clean":"2024-02-18T00:36:58.674706+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:36:58.674706+0000","last_undegraded":"2024-02-18T00:36:58.674706+0000","last_fullsized":"2024-02-18T00:36:58.674706+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:36:55.970628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.970628+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":318,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695757+0000","last_change":"2024-02-18T00:36:44.617522+0000","last_active":"2024-02-18T00:36:59.695757+0000","last_peered":"2024-02-18T00:36:59.695757+0000","last_clean":"2024-02-18T00:36:59.695757+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:36:59.695757+0000","last_undegraded":"2024-02-18T00:36:59.695757+0000","last_fullsized":"2024-02-18T00:36:59.695757+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.617443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.617443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":308,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678860+0000","last_change":"2024-02-18T00:36:50.970267+0000","last_active":"2024-02-18T00:36:59.678860+0000","last_peered":"2024-02-18T00:36:59.678860+0000","last_clean":"2024-02-18T00:36:59.678860+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:36:59.678860+0000","last_undegraded":"2024-02-18T00:36:59.678860+0000","last_fullsized":"2024-02-18T00:36:59.678860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:50.970192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.970192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":300,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.277960+0000","last_change":"2024-02-18T00:36:11.346267+0000","last_active":"2024-02-18T00:37:00.277960+0000","last_peered":"2024-02-18T00:37:00.277960+0000","last_clean":"2024-02-18T00:37:00.277960+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:37:00.277960+0000","last_undegraded":"2024-02-18T00:37:00.277960+0000","last_fullsized":"2024-02-18T00:37:00.277960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.346197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.346197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":247,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279486+0000","last_change":"2024-02-18T00:36:42.355194+0000","last_active":"2024-02-18T00:37:00.279486+0000","last_peered":"2024-02-18T00:37:00.279486+0000","last_clean":"2024-02-18T00:37:00.279486+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:37:00.279486+0000","last_undegraded":"2024-02-18T00:37:00.279486+0000","last_fullsized":"2024-02-18T00:37:00.279486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.355114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.355114+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":331,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671746+0000","last_change":"2024-02-18T00:36:52.343624+0000","last_active":"2024-02-18T00:36:59.671746+0000","last_peered":"2024-02-18T00:36:59.671746+0000","last_clean":"2024-02-18T00:36:59.671746+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:36:59.671746+0000","last_undegraded":"2024-02-18T00:36:59.671746+0000","last_fullsized":"2024-02-18T00:36:59.671746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:36:52.343556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.343556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":308,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673510+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:36:58.673510+0000","last_peered":"2024-02-18T00:36:58.673510+0000","last_clean":"2024-02-18T00:36:58.673510+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:36:58.673510+0000","last_undegraded":"2024-02-18T00:36:58.673510+0000","last_fullsized":"2024-02-18T00:36:58.673510+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":309,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.674758+0000","last_change":"2024-02-18T00:36:42.857622+0000","last_active":"2024-02-18T00:36:58.674758+0000","last_peered":"2024-02-18T00:36:58.674758+0000","last_clean":"2024-02-18T00:36:58.674758+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:36:58.674758+0000","last_undegraded":"2024-02-18T00:36:58.674758+0000","last_fullsized":"2024-02-18T00:36:58.674758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.857557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.857557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":301,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678438+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:36:59.678438+0000","last_peered":"2024-02-18T00:36:59.678438+0000","last_clean":"2024-02-18T00:36:59.678438+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:36:59.678438+0000","last_undegraded":"2024-02-18T00:36:59.678438+0000","last_fullsized":"2024-02-18T00:36:59.678438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":248,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696270+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:36:59.696270+0000","last_peered":"2024-02-18T00:36:59.696270+0000","last_clean":"2024-02-18T00:36:59.696270+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:36:59.696270+0000","last_undegraded":"2024-02-18T00:36:59.696270+0000","last_fullsized":"2024-02-18T00:36:59.696270+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":311,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677620+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:36:59.677620+0000","last_peered":"2024-02-18T00:36:59.677620+0000","last_clean":"2024-02-18T00:36:59.677620+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:36:59.677620+0000","last_undegraded":"2024-02-18T00:36:59.677620+0000","last_fullsized":"2024-02-18T00:36:59.677620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":310,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.826521+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:36:58.826521+0000","last_peered":"2024-02-18T00:36:58.826521+0000","last_clean":"2024-02-18T00:36:58.826521+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:36:58.826521+0000","last_undegraded":"2024-02-18T00:36:58.826521+0000","last_fullsized":"2024-02-18T00:36:58.826521+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":318,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695237+0000","last_change":"2024-02-18T00:36:12.518914+0000","last_active":"2024-02-18T00:36:59.695237+0000","last_peered":"2024-02-18T00:36:59.695237+0000","last_clean":"2024-02-18T00:36:59.695237+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:36:59.695237+0000","last_undegraded":"2024-02-18T00:36:59.695237+0000","last_fullsized":"2024-02-18T00:36:59.695237+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:12.518838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:12.518838+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":208,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671715+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:36:58.671715+0000","last_peered":"2024-02-18T00:36:58.671715+0000","last_clean":"2024-02-18T00:36:58.671715+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:36:58.671715+0000","last_undegraded":"2024-02-18T00:36:58.671715+0000","last_fullsized":"2024-02-18T00:36:58.671715+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":254,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827970+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:36:58.827970+0000","last_peered":"2024-02-18T00:36:58.827970+0000","last_clean":"2024-02-18T00:36:58.827970+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:36:58.827970+0000","last_undegraded":"2024-02-18T00:36:58.827970+0000","last_fullsized":"2024-02-18T00:36:58.827970+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":362,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678869+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:36:59.678869+0000","last_peered":"2024-02-18T00:36:59.678869+0000","last_clean":"2024-02-18T00:36:59.678869+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:36:59.678869+0000","last_undegraded":"2024-02-18T00:36:59.678869+0000","last_fullsized":"2024-02-18T00:36:59.678869+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":313,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696301+0000","last_change":"2024-02-18T00:36:11.551695+0000","last_active":"2024-02-18T00:36:59.696301+0000","last_peered":"2024-02-18T00:36:59.696301+0000","last_clean":"2024-02-18T00:36:59.696301+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:36:59.696301+0000","last_undegraded":"2024-02-18T00:36:59.696301+0000","last_fullsized":"2024-02-18T00:36:59.696301+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.551602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":223,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.670978+0000","last_change":"2024-02-18T00:36:59.243785+0000","last_active":"2024-02-18T00:36:59.670978+0000","last_peered":"2024-02-18T00:36:59.670978+0000","last_clean":"2024-02-18T00:36:59.670978+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:36:59.670978+0000","last_undegraded":"2024-02-18T00:36:59.670978+0000","last_fullsized":"2024-02-18T00:36:59.670978+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.243699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.243699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":301,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.679043+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:36:59.679043+0000","last_peered":"2024-02-18T00:36:59.679043+0000","last_clean":"2024-02-18T00:36:59.679043+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:36:59.679043+0000","last_undegraded":"2024-02-18T00:36:59.679043+0000","last_fullsized":"2024-02-18T00:36:59.679043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":209,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671364+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:36:59.671364+0000","last_peered":"2024-02-18T00:36:59.671364+0000","last_clean":"2024-02-18T00:36:59.671364+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:36:59.671364+0000","last_undegraded":"2024-02-18T00:36:59.671364+0000","last_fullsized":"2024-02-18T00:36:59.671364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":368,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279727+0000","last_change":"2024-02-18T00:36:53.401897+0000","last_active":"2024-02-18T00:37:00.279727+0000","last_peered":"2024-02-18T00:37:00.279727+0000","last_clean":"2024-02-18T00:37:00.279727+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:37:00.279727+0000","last_undegraded":"2024-02-18T00:37:00.279727+0000","last_fullsized":"2024-02-18T00:37:00.279727+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:36:53.401819+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.401819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":314,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.826892+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:36:58.826892+0000","last_peered":"2024-02-18T00:36:58.826892+0000","last_clean":"2024-02-18T00:36:58.826892+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:36:58.826892+0000","last_undegraded":"2024-02-18T00:36:58.826892+0000","last_fullsized":"2024-02-18T00:36:58.826892+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":431,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696364+0000","last_change":"2024-02-18T00:36:38.557419+0000","last_active":"2024-02-18T00:36:59.696364+0000","last_peered":"2024-02-18T00:36:59.696364+0000","last_clean":"2024-02-18T00:36:59.696364+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:36:59.696364+0000","last_undegraded":"2024-02-18T00:36:59.696364+0000","last_fullsized":"2024-02-18T00:36:59.696364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:36:38.557298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.557298+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":223,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827848+0000","last_change":"2024-02-18T00:36:49.727241+0000","last_active":"2024-02-18T00:36:58.827848+0000","last_peered":"2024-02-18T00:36:58.827848+0000","last_clean":"2024-02-18T00:36:58.827848+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:36:58.827848+0000","last_undegraded":"2024-02-18T00:36:58.827848+0000","last_fullsized":"2024-02-18T00:36:58.827848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:36:49.727157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.727157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":207,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.676529+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:36:59.676529+0000","last_peered":"2024-02-18T00:36:59.676529+0000","last_clean":"2024-02-18T00:36:59.676529+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:36:59.676529+0000","last_undegraded":"2024-02-18T00:36:59.676529+0000","last_fullsized":"2024-02-18T00:36:59.676529+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":345,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696417+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:36:59.696417+0000","last_peered":"2024-02-18T00:36:59.696417+0000","last_clean":"2024-02-18T00:36:59.696417+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:36:59.696417+0000","last_undegraded":"2024-02-18T00:36:59.696417+0000","last_fullsized":"2024-02-18T00:36:59.696417+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":312,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678609+0000","last_change":"2024-02-18T00:36:56.850409+0000","last_active":"2024-02-18T00:36:59.678609+0000","last_peered":"2024-02-18T00:36:59.678609+0000","last_clean":"2024-02-18T00:36:59.678609+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:36:59.678609+0000","last_undegraded":"2024-02-18T00:36:59.678609+0000","last_fullsized":"2024-02-18T00:36:59.678609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.850326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.850326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":311,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279758+0000","last_change":"2024-02-18T00:36:52.372043+0000","last_active":"2024-02-18T00:37:00.279758+0000","last_peered":"2024-02-18T00:37:00.279758+0000","last_clean":"2024-02-18T00:37:00.279758+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:37:00.279758+0000","last_undegraded":"2024-02-18T00:37:00.279758+0000","last_fullsized":"2024-02-18T00:37:00.279758+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.371970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.371970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":307,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673292+0000","last_change":"2024-02-18T00:35:54.569108+0000","last_active":"2024-02-18T00:36:58.673292+0000","last_peered":"2024-02-18T00:36:58.673292+0000","last_clean":"2024-02-18T00:36:58.673292+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:36:58.673292+0000","last_undegraded":"2024-02-18T00:36:58.673292+0000","last_fullsized":"2024-02-18T00:36:58.673292+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.569033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.569033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":261,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.671615+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:36:58.671615+0000","last_peered":"2024-02-18T00:36:58.671615+0000","last_clean":"2024-02-18T00:36:58.671615+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:36:58.671615+0000","last_undegraded":"2024-02-18T00:36:58.671615+0000","last_fullsized":"2024-02-18T00:36:58.671615+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":336,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.672387+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:36:58.672387+0000","last_peered":"2024-02-18T00:36:58.672387+0000","last_clean":"2024-02-18T00:36:58.672387+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:36:58.672387+0000","last_undegraded":"2024-02-18T00:36:58.672387+0000","last_fullsized":"2024-02-18T00:36:58.672387+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":322,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695693+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:36:59.695693+0000","last_peered":"2024-02-18T00:36:59.695693+0000","last_clean":"2024-02-18T00:36:59.695693+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:36:59.695693+0000","last_undegraded":"2024-02-18T00:36:59.695693+0000","last_fullsized":"2024-02-18T00:36:59.695693+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":309,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673788+0000","last_change":"2024-02-18T00:36:35.808459+0000","last_active":"2024-02-18T00:36:58.673788+0000","last_peered":"2024-02-18T00:36:58.673788+0000","last_clean":"2024-02-18T00:36:58.673788+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:36:58.673788+0000","last_undegraded":"2024-02-18T00:36:58.673788+0000","last_fullsized":"2024-02-18T00:36:58.673788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.808381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.808381+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":303,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.672407+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:36:59.672407+0000","last_peered":"2024-02-18T00:36:59.672407+0000","last_clean":"2024-02-18T00:36:59.672407+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:36:59.672407+0000","last_undegraded":"2024-02-18T00:36:59.672407+0000","last_fullsized":"2024-02-18T00:36:59.672407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":248,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.695640+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:36:59.695640+0000","last_peered":"2024-02-18T00:36:59.695640+0000","last_clean":"2024-02-18T00:36:59.695640+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:36:59.695640+0000","last_undegraded":"2024-02-18T00:36:59.695640+0000","last_fullsized":"2024-02-18T00:36:59.695640+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":343,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.676791+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:36:59.676791+0000","last_peered":"2024-02-18T00:36:59.676791+0000","last_clean":"2024-02-18T00:36:59.676791+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:36:59.676791+0000","last_undegraded":"2024-02-18T00:36:59.676791+0000","last_fullsized":"2024-02-18T00:36:59.676791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":332,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678760+0000","last_change":"2024-02-18T00:36:52.928195+0000","last_active":"2024-02-18T00:36:59.678760+0000","last_peered":"2024-02-18T00:36:59.678760+0000","last_clean":"2024-02-18T00:36:59.678760+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:36:59.678760+0000","last_undegraded":"2024-02-18T00:36:59.678760+0000","last_fullsized":"2024-02-18T00:36:59.678760+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:36:52.928076+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.928076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":305,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671527+0000","last_change":"2024-02-18T00:34:29.236028+0000","last_active":"2024-02-18T00:36:59.671527+0000","last_peered":"2024-02-18T00:36:59.671527+0000","last_clean":"2024-02-18T00:36:59.671527+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:36:59.671527+0000","last_undegraded":"2024-02-18T00:36:59.671527+0000","last_fullsized":"2024-02-18T00:36:59.671527+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:29.235953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:29.235953+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":301,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677804+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:36:59.677804+0000","last_peered":"2024-02-18T00:36:59.677804+0000","last_clean":"2024-02-18T00:36:59.677804+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:36:59.677804+0000","last_undegraded":"2024-02-18T00:36:59.677804+0000","last_fullsized":"2024-02-18T00:36:59.677804+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":256,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279701+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:37:00.279701+0000","last_peered":"2024-02-18T00:37:00.279701+0000","last_clean":"2024-02-18T00:37:00.279701+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:37:00.279701+0000","last_undegraded":"2024-02-18T00:37:00.279701+0000","last_fullsized":"2024-02-18T00:37:00.279701+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":365,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671391+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:36:59.671391+0000","last_peered":"2024-02-18T00:36:59.671391+0000","last_clean":"2024-02-18T00:36:59.671391+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:36:59.671391+0000","last_undegraded":"2024-02-18T00:36:59.671391+0000","last_fullsized":"2024-02-18T00:36:59.671391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":205,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827680+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:36:58.827680+0000","last_peered":"2024-02-18T00:36:58.827680+0000","last_clean":"2024-02-18T00:36:58.827680+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:36:58.827680+0000","last_undegraded":"2024-02-18T00:36:58.827680+0000","last_fullsized":"2024-02-18T00:36:58.827680+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":324,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827356+0000","last_change":"2024-02-18T00:36:45.773797+0000","last_active":"2024-02-18T00:36:58.827356+0000","last_peered":"2024-02-18T00:36:58.827356+0000","last_clean":"2024-02-18T00:36:58.827356+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:36:58.827356+0000","last_undegraded":"2024-02-18T00:36:58.827356+0000","last_fullsized":"2024-02-18T00:36:58.827356+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.773695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.773695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":314,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.278322+0000","last_change":"2024-02-18T00:36:45.412472+0000","last_active":"2024-02-18T00:37:00.278322+0000","last_peered":"2024-02-18T00:37:00.278322+0000","last_clean":"2024-02-18T00:37:00.278322+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:37:00.278322+0000","last_undegraded":"2024-02-18T00:37:00.278322+0000","last_fullsized":"2024-02-18T00:37:00.278322+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.412406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.412406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":248,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673736+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:36:58.673736+0000","last_peered":"2024-02-18T00:36:58.673736+0000","last_clean":"2024-02-18T00:36:58.673736+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:36:58.673736+0000","last_undegraded":"2024-02-18T00:36:58.673736+0000","last_fullsized":"2024-02-18T00:36:58.673736+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":340,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671540+0000","last_change":"2024-02-18T00:36:51.303043+0000","last_active":"2024-02-18T00:36:59.671540+0000","last_peered":"2024-02-18T00:36:59.671540+0000","last_clean":"2024-02-18T00:36:59.671540+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:36:59.671540+0000","last_undegraded":"2024-02-18T00:36:59.671540+0000","last_fullsized":"2024-02-18T00:36:59.671540+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:36:51.302920+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.302920+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":213,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.696188+0000","last_change":"2024-02-18T00:36:52.758965+0000","last_active":"2024-02-18T00:36:59.696188+0000","last_peered":"2024-02-18T00:36:59.696188+0000","last_clean":"2024-02-18T00:36:59.696188+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:36:59.696188+0000","last_undegraded":"2024-02-18T00:36:59.696188+0000","last_fullsized":"2024-02-18T00:36:59.696188+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.758904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.758904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":209,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671548+0000","last_change":"2024-02-18T00:36:49.277109+0000","last_active":"2024-02-18T00:36:59.671548+0000","last_peered":"2024-02-18T00:36:59.671548+0000","last_clean":"2024-02-18T00:36:59.671548+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:36:59.671548+0000","last_undegraded":"2024-02-18T00:36:59.671548+0000","last_fullsized":"2024-02-18T00:36:59.671548+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.276979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.276979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":304,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.827114+0000","last_change":"2024-02-18T00:36:40.825174+0000","last_active":"2024-02-18T00:36:58.827114+0000","last_peered":"2024-02-18T00:36:58.827114+0000","last_clean":"2024-02-18T00:36:58.827114+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:36:58.827114+0000","last_undegraded":"2024-02-18T00:36:58.827114+0000","last_fullsized":"2024-02-18T00:36:58.827114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:40.825055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:36:40.825055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":256,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.677097+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:36:59.677097+0000","last_peered":"2024-02-18T00:36:59.677097+0000","last_clean":"2024-02-18T00:36:59.677097+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:36:59.677097+0000","last_undegraded":"2024-02-18T00:36:59.677097+0000","last_fullsized":"2024-02-18T00:36:59.677097+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":343,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.670925+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:36:59.670925+0000","last_peered":"2024-02-18T00:36:59.670925+0000","last_clean":"2024-02-18T00:36:59.670925+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:36:59.670925+0000","last_undegraded":"2024-02-18T00:36:59.670925+0000","last_fullsized":"2024-02-18T00:36:59.670925+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":307,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678752+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:36:59.678752+0000","last_peered":"2024-02-18T00:36:59.678752+0000","last_clean":"2024-02-18T00:36:59.678752+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:36:59.678752+0000","last_undegraded":"2024-02-18T00:36:59.678752+0000","last_fullsized":"2024-02-18T00:36:59.678752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":318,"reported_epoch":340,"state":"active+clean","last_fresh":"2024-02-18T00:36:58.673111+0000","last_change":"2024-02-18T00:36:49.936731+0000","last_active":"2024-02-18T00:36:58.673111+0000","last_peered":"2024-02-18T00:36:58.673111+0000","last_clean":"2024-02-18T00:36:58.673111+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:36:58.673111+0000","last_undegraded":"2024-02-18T00:36:58.673111+0000","last_fullsized":"2024-02-18T00:36:58.673111+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.936644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.936644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":303,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671878+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:36:59.671878+0000","last_peered":"2024-02-18T00:36:59.671878+0000","last_clean":"2024-02-18T00:36:59.671878+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:36:59.671878+0000","last_undegraded":"2024-02-18T00:36:59.671878+0000","last_fullsized":"2024-02-18T00:36:59.671878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":247,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.679420+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:36:59.679420+0000","last_peered":"2024-02-18T00:36:59.679420+0000","last_clean":"2024-02-18T00:36:59.679420+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:36:59.679420+0000","last_undegraded":"2024-02-18T00:36:59.679420+0000","last_fullsized":"2024-02-18T00:36:59.679420+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":392,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678562+0000","last_change":"2024-02-18T00:36:46.942689+0000","last_active":"2024-02-18T00:36:59.678562+0000","last_peered":"2024-02-18T00:36:59.678562+0000","last_clean":"2024-02-18T00:36:59.678562+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:36:59.678562+0000","last_undegraded":"2024-02-18T00:36:59.678562+0000","last_fullsized":"2024-02-18T00:36:59.678562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:36:46.942583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.942583+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":315,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:37:00.279544+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:37:00.279544+0000","last_peered":"2024-02-18T00:37:00.279544+0000","last_clean":"2024-02-18T00:37:00.279544+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:37:00.279544+0000","last_undegraded":"2024-02-18T00:37:00.279544+0000","last_fullsized":"2024-02-18T00:37:00.279544+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":305,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.671140+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:36:59.671140+0000","last_peered":"2024-02-18T00:36:59.671140+0000","last_clean":"2024-02-18T00:36:59.671140+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:36:59.671140+0000","last_undegraded":"2024-02-18T00:36:59.671140+0000","last_fullsized":"2024-02-18T00:36:59.671140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":310,"reported_epoch":341,"state":"active+clean","last_fresh":"2024-02-18T00:36:59.678388+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:36:59.678388+0000","last_peered":"2024-02-18T00:36:59.678388+0000","last_clean":"2024-02-18T00:36:59.678388+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:36:59.678388+0000","last_undegraded":"2024-02-18T00:36:59.678388+0000","last_fullsized":"2024-02-18T00:36:59.678388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":196,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":193,"num_pg":32,"stat_sum":{"num_bytes":253,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":252,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15,"ondisk_log_size":15,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001220,"num_objects":123,"num_object_clones":2,"num_object_copies":369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":548,"num_write_kb":4449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13512704,"data_stored":12003693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1798,"ondisk_log_size":1798,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":457,"num_read_kb":452,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266785,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463015,"num_pgs":103,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310768,"kb_used_data":9444,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93434384,"statfs":{"total":95995035648,"available":95676809216,"internally_reserved":0,"allocated":9670656,"data_stored":7158825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":2.722,"15min":1.2370000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.1799999999999999,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.1679999999999999,"15min":0.72299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1,"5min":10.433,"15min":10.433},"last":0.71099999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":2.9569999999999999,"15min":1.3200000000000001},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.97199999999999998,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":2.9529999999999998,"15min":1.3109999999999999},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":1.216,"5min":71.872,"15min":71.872},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.395,"15min":0.80300000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.141,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.8560000000000001,"15min":0.96799999999999997},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.9930000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.749}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.391,"15min":0.80300000000000005},"min":{"1min":0.29999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2629999999999999,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.2929999999999999,"15min":0.77600000000000002},"min":{"1min":0.37,"5min":0.309,"15min":0.309},"max":{"1min":1.0149999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68500000000000005,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.496,"5min":1.1910000000000001,"15min":0.66900000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85099999999999998,"5min":35.18,"15min":35.18},"last":0.40100000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.73199999999999998,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.673,"5min":10.606,"15min":10.606},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.496,"5min":1.224,"15min":0.69399999999999995},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70899999999999996,"5min":35.119,"15min":35.119},"last":0.502}]},{"osd":7,"last update":"Sun Feb 18 00:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.63200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4770000000000003,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":1.119,"15min":0.72299999999999998},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.8730000000000002,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302756,"kb_used_data":5524,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442396,"statfs":{"total":95995035648,"available":95685013504,"internally_reserved":0,"allocated":5656576,"data_stored":3158645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":1.2290000000000001,"15min":0.625},"min":{"1min":0.19900000000000001,"5min":0.193,"15min":0.193},"max":{"1min":0.47899999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":1.3140000000000001,"15min":0.65900000000000003},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46999999999999997,"5min":43.137,"15min":43.137},"last":0.27000000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":1.5289999999999999,"15min":0.75900000000000001},"min":{"1min":0.20699999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51900000000000002,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.95999999999999996,"15min":0.58599999999999997},"min":{"1min":0.28499999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":17.596,"15min":17.596},"last":0.39300000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":1.357,"15min":0.751},"min":{"1min":0.26500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56999999999999995,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":1.3500000000000001,"15min":0.77200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71699999999999997,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:36:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":1.1339999999999999,"15min":0.78000000000000003},"min":{"1min":0.29999999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":1.1659999999999999,"15min":0.76600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88100000000000001,"5min":12.75,"15min":12.75},"last":0.495}]},{"osd":5,"last update":"Sun Feb 18 00:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":1.1319999999999999,"15min":0.77500000000000002},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85299999999999998,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":1.1379999999999999,"15min":0.78900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.58899999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:35:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":1.1779999999999999,"15min":0.81399999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":12.834,"15min":12.834},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.149,"15min":0.80300000000000005},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81100000000000005,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.46700000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":1.147,"15min":0.80500000000000005},"min":{"1min":0.41899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.109,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":1.1459999999999999,"15min":0.81499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5409999999999999,"5min":12.805,"15min":12.805},"last":0.57899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738500,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302748,"kb_used_data":5520,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442404,"statfs":{"total":95995035648,"available":95685021696,"internally_reserved":0,"allocated":5652480,"data_stored":3148510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":2.2749999999999999,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.2330000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":1.5589999999999999,"15min":0.75900000000000001},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":4.577,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":1.8240000000000001,"15min":0.872},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.2430000000000001,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.311,"5min":1.8280000000000001,"15min":0.87},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":0.40000000000000002,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.376}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.149,"15min":0.67600000000000005},"min":{"1min":0.255,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.497,"5min":19.006,"15min":19.006},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":2.2610000000000001,"15min":1.0780000000000001},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":4.4489999999999998,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:36:02 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":1.6499999999999999,"15min":0.91400000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.63900000000000001,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.442},{"interface":"front","average":{"1min":0.627,"5min":1.6970000000000001,"15min":0.96799999999999997},"min":{"1min":0.42199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2969999999999999,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.49299999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.613,"15min":0.93100000000000005},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.262,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":1.8069999999999999,"15min":0.96599999999999997},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":57.991,"15min":57.991},"last":0.53700000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.2270000000000001,"15min":0.79500000000000004},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.746,"15min":0.96799999999999997},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.343,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.58199999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.8680000000000001,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3129999999999999,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":1.798,"15min":0.99099999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.56499999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":302848,"kb_used_data":5620,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442304,"statfs":{"total":95995035648,"available":95684919296,"internally_reserved":0,"allocated":5754880,"data_stored":3158366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.71599999999999997,"15min":0.49099999999999999},"min":{"1min":0.22,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46100000000000002,"5min":12.144,"15min":12.144},"last":0.253},{"interface":"front","average":{"1min":0.35799999999999998,"5min":1.0529999999999999,"15min":0.60699999999999998},"min":{"1min":0.221,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45800000000000002,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.70899999999999996,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47099999999999997,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.82099999999999995,"15min":0.54500000000000004},"min":{"1min":0.29499999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48499999999999999,"5min":18.919,"15min":18.919},"last":0.32100000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.91400000000000003,"15min":0.61099999999999999},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.503,"5min":22.712,"15min":22.712},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.91000000000000003,"15min":0.621},"min":{"1min":0.30499999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50700000000000001,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:36:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.88400000000000001,"15min":0.68300000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.85399999999999998,"15min":0.67900000000000005},"min":{"1min":0.38800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":10.016,"15min":10.016},"last":0.52400000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.77300000000000002,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":6.766,"15min":6.766},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.84099999999999997,"15min":0.69399999999999995},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78700000000000003,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.95999999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.79400000000000004,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":6.806,"15min":6.806},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.89700000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.92100000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.81200000000000006,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.89700000000000002,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.93100000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149813,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319032,"kb_used_data":9516,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93426120,"statfs":{"total":95995035648,"available":95668346880,"internally_reserved":0,"allocated":9744384,"data_stored":7158592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.65900000000000003,"15min":0.47699999999999998},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54100000000000004,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.76000000000000001,"15min":0.52400000000000002},"min":{"1min":0.251,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51900000000000002,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.60699999999999998,"15min":0.47999999999999998},"min":{"1min":0.28899999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.54400000000000004,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.315},{"interface":"front","average":{"1min":0.40100000000000002,"5min":1.206,"15min":0.66800000000000004},"min":{"1min":0.25600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59399999999999997,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.34300000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":1.0720000000000001,"15min":0.64800000000000002},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71499999999999997,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.33300000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:36:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.88500000000000001,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.96899999999999997,"5min":10.131,"15min":10.131},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.70599999999999996,"15min":0.627},"min":{"1min":0.41399999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71299999999999997,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.70099999999999996,"15min":0.63100000000000001},"min":{"1min":0.42199999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66400000000000003,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.81799999999999995,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.92700000000000005,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:36:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":1.05,"15min":0.749},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.8940000000000001,"5min":10.189,"15min":10.189},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.74099999999999999,"15min":0.66400000000000003},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.749}]},{"osd":7,"last update":"Sun Feb 18 00:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.79900000000000004,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1599999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.93000000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.39800000000000002},"max":{"1min":0.88100000000000001,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.64600000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986288,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306900,"kb_used_data":5572,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438252,"statfs":{"total":95995035648,"available":95680770048,"internally_reserved":0,"allocated":5705728,"data_stored":3159307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":1.556,"15min":0.91800000000000004},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.67800000000000005,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":1.506,"15min":0.877},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72499999999999998,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.79300000000000004}]},{"osd":1,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.81499999999999995,"15min":0.65200000000000002},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78200000000000003,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":1.595,"15min":0.90500000000000003},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.27100000000000002},"max":{"1min":0.82699999999999996,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.72999999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.78900000000000003,"15min":0.66900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.119,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.90500000000000003,"15min":0.68400000000000005},"min":{"1min":0.44,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70299999999999996,"5min":17.77,"15min":17.77},"last":0.77300000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:36:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.70599999999999996,"15min":0.63400000000000001},"min":{"1min":0.34899999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":1.8939999999999999,"5min":7.218,"15min":7.218},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":1.077,"15min":0.72399999999999998},"min":{"1min":0.33300000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.349,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.436,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.66700000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.45100000000000001,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.26500000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:36:06 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.46700000000000003,"15min":0.41599999999999998},"min":{"1min":0.32400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70299999999999996,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.50900000000000001,"15min":0.45600000000000002},"min":{"1min":0.34000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.751,"15min":0.751},"last":0.32600000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.68999999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.505,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73599999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.34899999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724651,"num_pgs":94,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310672,"kb_used_data":9344,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93434480,"statfs":{"total":95995035648,"available":95676907520,"internally_reserved":0,"allocated":9568256,"data_stored":7149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.29799999999999999,"5min":0.217,"15min":0.217},"max":{"1min":2.2549999999999999,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.76000000000000001,"15min":0.61499999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.8680000000000003,"5min":12.744,"15min":12.744},"last":0.434}]},{"osd":1,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.70599999999999996,"15min":0.625},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.9850000000000001,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.629},"min":{"1min":0.35199999999999998,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":4.2919999999999998,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.61899999999999999},"min":{"1min":0.32200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":3.2959999999999998,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.71999999999999997,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":4.4359999999999999,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.65200000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.621},"min":{"1min":0.33200000000000002,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":3.4550000000000001,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.627},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.68400000000000005,"15min":0.59799999999999998},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.3920000000000003,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.44400000000000001},"min":{"1min":0.31900000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.6859999999999999,"5min":11.818,"15min":11.818},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.501,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.5680000000000001,"5min":4.274,"15min":4.274},"last":0.499}]},{"osd":6,"last update":"Sun Feb 18 00:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":12.362,"5min":18.605,"15min":18.605},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.46,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.51100000000000001,"15min":0.46899999999999997},"min":{"1min":0.33300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.56799999999999995,"15min":0.502},"min":{"1min":0.318,"5min":0.309,"15min":0.309},"max":{"1min":4.649,"5min":4.649,"15min":4.649},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4272128,"data_stored":4000669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4112384,"data_stored":4000278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":4000505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":193,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":193,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":193,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":193,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":193,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":193,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":193,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:37:06.514 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:06.514 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:06.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:06.616 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:06.647 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:06.669 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:06.672 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:06.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:06.703 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:05.668006+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:05.668119+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:05.668229+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e347: 8 total, 8 up, 7 in 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:05.675879+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:05.714383+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:05.714774+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:05.877453+0000 mgr.y (mgr.24454) 340 : cluster [DBG] pgmap v508: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:05.668006+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:05.668119+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:06.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:05.668229+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e347: 8 total, 8 up, 7 in 2024-02-18T00:37:06.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:05.675879+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:06.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:05.714383+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:06.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:05.714774+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:06.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:05.877453+0000 mgr.y (mgr.24454) 340 : cluster [DBG] pgmap v508: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:37:06.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:06.802 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:06.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:37:07.000 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:07.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:05.668006+0000 mon.a (mon.0) 2248 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:05.668119+0000 mon.a (mon.0) 2249 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:05.668229+0000 mon.a (mon.0) 2250 : cluster [DBG] osdmap e347: 8 total, 8 up, 7 in 2024-02-18T00:37:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:05.675879+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]: dispatch 2024-02-18T00:37:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:05.714383+0000 mon.c (mon.2) 326 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:05.714774+0000 mon.a (mon.0) 2252 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:07.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:05.877453+0000 mgr.y (mgr.24454) 340 : cluster [DBG] pgmap v508: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:37:07.005 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:07.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:07.106 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:07.139 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:07.161 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:07.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:07.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:07.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:07.296 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:07.316 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:07.319 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:07.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:07.420 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:07.449 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:06.354906+0000 osd.6 (osd.6) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:06.355081+0000 osd.6 (osd.6) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:06.674667+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:06.674757+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:06.674826+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e348: 8 total, 8 up, 7 in 2024-02-18T00:37:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:06.677837+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:06.678488+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:06.354906+0000 osd.6 (osd.6) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:06.355081+0000 osd.6 (osd.6) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:06.674667+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:06.674757+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:06.674826+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e348: 8 total, 8 up, 7 in 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:06.677837+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:06.678488+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:08.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:06.354906+0000 osd.6 (osd.6) 169 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:06.355081+0000 osd.6 (osd.6) 170 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:06.674667+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? 172.21.15.59:0/78627635' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi059-82882-58"}]': finished 2024-02-18T00:37:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:06.674757+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:06.674826+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e348: 8 total, 8 up, 7 in 2024-02-18T00:37:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:06.677837+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:06.678488+0000 mon.a (mon.0) 2256 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:08.869 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:37:08.870 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:37:08.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:07.680174+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]': finished 2024-02-18T00:37:08.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:07.680325+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e349: 8 total, 8 up, 7 in 2024-02-18T00:37:08.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:07.682922+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.59:0/292127905' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:08.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:07.683725+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:08.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:07.684490+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]: dispatch 2024-02-18T00:37:08.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:07.685015+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]: dispatch 2024-02-18T00:37:08.893 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:07.877923+0000 mgr.y (mgr.24454) 341 : cluster [DBG] pgmap v511: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:37:08.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:07.680174+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]': finished 2024-02-18T00:37:08.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:07.680325+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e349: 8 total, 8 up, 7 in 2024-02-18T00:37:08.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:07.682922+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.59:0/292127905' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:08.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:07.683725+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:08.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:07.684490+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]: dispatch 2024-02-18T00:37:08.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:07.685015+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]: dispatch 2024-02-18T00:37:08.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:07.877923+0000 mgr.y (mgr.24454) 341 : cluster [DBG] pgmap v511: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:37:08.900 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:08.903 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:08.904 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:07.680174+0000 mon.a (mon.0) 2257 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-45"}]': finished 2024-02-18T00:37:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:07.680325+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e349: 8 total, 8 up, 7 in 2024-02-18T00:37:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:07.682922+0000 mon.c (mon.2) 328 : audit [INF] from='client.? 172.21.15.59:0/292127905' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:07.683725+0000 mon.a (mon.0) 2259 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:07.684490+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]: dispatch 2024-02-18T00:37:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:07.685015+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]: dispatch 2024-02-18T00:37:09.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:07.877923+0000 mgr.y (mgr.24454) 341 : cluster [DBG] pgmap v511: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:37:09.004 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:09.055 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:09.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:09.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:09.150 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.151+0000 7fa8ac93b700 1 -- 172.21.15.59:0/519551406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a80f9670 msgr2=0x7fa8a8101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.151+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/519551406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a80f9670 0x7fa8a8101d80 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 rx=0x7fa89c004870 tx=0x7fa89c009cb0).stop 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 -- 172.21.15.59:0/519551406 shutdown_connections 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/519551406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a8102270 0x7fa8a8104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/519551406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a80f8d10 0x7fa8a80f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/519551406 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a80f9670 0x7fa8a8101d80 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 -- 172.21.15.59:0/519551406 >> 172.21.15.59:0/519551406 conn(0x7fa8a80f2820 msgr2=0x7fa8a80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 -- 172.21.15.59:0/519551406 shutdown_connections 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 -- 172.21.15.59:0/519551406 wait complete. 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 librados: starting msgr at 2024-02-18T00:37:09.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 librados: starting objecter 2024-02-18T00:37:09.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.152+0000 7fa8ac93b700 1 Processor -- start 2024-02-18T00:37:09.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 -- start start 2024-02-18T00:37:09.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 librados: setting wanted keys 2024-02-18T00:37:09.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 librados: calling monclient init 2024-02-18T00:37:09.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a80f8d10 0x7fa8a810e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a80f9670 0x7fa8a810e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a8102270 0x7fa8a8108430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a8108920 con 0x7fa8a80f9670 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a8108a90 con 0x7fa8a80f8d10 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8ac93b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a8108c00 con 0x7fa8a8102270 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a5d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a80f9670 0x7fa8a810e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a5d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a80f9670 0x7fa8a810e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58472/0 (socket says 172.21.15.59:58472) 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a6d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a8102270 0x7fa8a8108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a5d9b700 1 -- 172.21.15.59:0/2038043615 learned_addr learned my addr 172.21.15.59:0/2038043615 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a80f8d10 0x7fa8a810e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a6d9d700 1 -- 172.21.15.59:0/2038043615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a80f8d10 msgr2=0x7fa8a810e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a6d9d700 1 --2- 172.21.15.59:0/2038043615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8a80f8d10 0x7fa8a810e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a6d9d700 1 -- 172.21.15.59:0/2038043615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a80f9670 msgr2=0x7fa8a810e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a6d9d700 1 --2- 172.21.15.59:0/2038043615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a80f9670 0x7fa8a810e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a6d9d700 1 -- 172.21.15.59:0/2038043615 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa898005040 con 0x7fa8a8102270 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.153+0000 7fa8a5d9b700 1 --2- 172.21.15.59:0/2038043615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8a80f9670 0x7fa8a810e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8a6d9d700 1 --2- 172.21.15.59:0/2038043615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a8102270 0x7fa8a8108430 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 rx=0x7fa8a000d8a0 tx=0x7fa8a0004db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8977fe700 10 client.?.objecter ms_handle_connect 0x7fa8a8102270 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8977fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004020 con 0x7fa8a8102270 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8a000f040 con 0x7fa8a8102270 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c005040 con 0x7fa8a8102270 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00036c0 con 0x7fa8a8102270 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8ac93b700 10 client.40967.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a81091f0 con 0x7fa8a8102270 2024-02-18T00:37:09.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.154+0000 7fa8ac93b700 1 librados: init done 2024-02-18T00:37:09.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.155+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa8a805fa60 con 0x7fa8a8102270 2024-02-18T00:37:09.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:09.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.155+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa8a0003af0 con 0x7fa8a8102270 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8977fe700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa890062c90 0x7fa890065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(350..350 src has 44..350) v4 ==== 8419+0+0 (secure 0 0 0) 0x7fa8a007cdc0 con 0x7fa8a8102270 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 osd_map(350..350 src has 44..350) v4 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8977fe700 3 client.40967.objecter handle_osd_map got epochs [350,350] > 0 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8977fe700 3 client.40967.objecter handle_osd_map decoding full epoch 350 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8977fe700 20 client.40967.objecter dump_active .. 0 homeless 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v350) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa8a007f450 con 0x7fa8a8102270 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 1 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa8a80fc200 0x7fa8a80fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 1 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 0 1 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a805ee80 osd=0 4 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 10 client.40967.objecter _send_command 1 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.156+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a81072a0 con 0x7fa8a80fc200 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.157+0000 7fa8a5d9b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa8a80fc200 0x7fa8a80fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.157+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a80fc200 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.157+0000 7fa8a5d9b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa8a80fc200 0x7fa8a80fc5e0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.157+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa890062c90 0x7fa890065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.157+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa890062c90 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.157+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa890062c90 0x7fa890065150 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 rx=0x7fa898009350 tx=0x7fa89800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.158+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa8a81072a0 con 0x7fa8a80fc200 2024-02-18T00:37:09.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.158+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:37:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.158+0000 7fa8977fe700 10 client.40967.objecter _finish_command 1 = system:0 2024-02-18T00:37:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.158+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a805ee80 osd=0 4 2024-02-18T00:37:09.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.158+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 0 1 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 2 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 2 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a805ee80 osd=0 3 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 0 2 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a805ee80 osd=0 4 2024-02-18T00:37:09.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 10 client.40967.objecter _send_command 2 2024-02-18T00:37:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.185+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a80273c0 con 0x7fa8a80fc200 2024-02-18T00:37:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a80273c0 con 0x7fa8a80fc200 2024-02-18T00:37:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8977fe700 10 client.40967.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a805ee80 osd=0 4 2024-02-18T00:37:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 0 2 2024-02-18T00:37:09.189 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:37:09.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6805/26279110 conn(0x7fa8a80598f0 0x7fa8a8059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 1 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8028710 osd=1 4 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 10 client.40967.objecter _send_command 3 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a80f8b10 con 0x7fa8a80598f0 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.187+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6805/26279110 conn(0x7fa8a80598f0 0x7fa8a8059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.188+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6805/26279110 conn(0x7fa8a80598f0 0x7fa8a8059cd0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.188+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a80598f0 2024-02-18T00:37:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.188+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa8a80f8b10 con 0x7fa8a80598f0 2024-02-18T00:37:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.188+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:37:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.188+0000 7fa8977fe700 10 client.40967.objecter _finish_command 3 = system:0 2024-02-18T00:37:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.188+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a8028710 osd=1 4 2024-02-18T00:37:09.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.188+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 1 3 2024-02-18T00:37:09.192 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:09.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 4 2024-02-18T00:37:09.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 4 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a8028710 osd=1 3 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 1 4 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8028710 osd=1 4 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 10 client.40967.objecter _send_command 4 2024-02-18T00:37:09.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.206+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a8118250 con 0x7fa8a80598f0 2024-02-18T00:37:09.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a8118250 con 0x7fa8a80598f0 2024-02-18T00:37:09.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8977fe700 10 client.40967.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a8028710 osd=1 4 2024-02-18T00:37:09.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 1 4 2024-02-18T00:37:09.210 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 5 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa8a80594d0 0x7fa8a811ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 5 2024-02-18T00:37:09.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 2 5 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8059360 osd=2 4 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 10 client.40967.objecter _send_command 5 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.208+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a8028270 con 0x7fa8a80594d0 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.209+0000 7fa8a5d9b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa8a80594d0 0x7fa8a811ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.209+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a80594d0 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.209+0000 7fa8a5d9b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa8a80594d0 0x7fa8a811ead0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.210+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa8a8028270 con 0x7fa8a80594d0 2024-02-18T00:37:09.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.210+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:37:09.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.210+0000 7fa8977fe700 10 client.40967.objecter _finish_command 5 = system:0 2024-02-18T00:37:09.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.210+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a8059360 osd=2 4 2024-02-18T00:37:09.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.210+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 2 5 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 6 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 6 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a8059360 osd=2 3 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 2 6 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8059360 osd=2 4 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 10 client.40967.objecter _send_command 6 2024-02-18T00:37:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.224+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a811efc0 con 0x7fa8a80594d0 2024-02-18T00:37:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a811efc0 con 0x7fa8a80594d0 2024-02-18T00:37:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8977fe700 10 client.40967.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a8059360 osd=2 4 2024-02-18T00:37:09.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 2 6 2024-02-18T00:37:09.228 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 7 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa8a8027940 0x7fa8a8122350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 7 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 3 7 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a80277d0 osd=3 4 2024-02-18T00:37:09.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 10 client.40967.objecter _send_command 7 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a8117db0 con 0x7fa8a8027940 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8a6d9d700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa8a8027940 0x7fa8a8122350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a8027940 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.226+0000 7fa8a6d9d700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa8a8027940 0x7fa8a8122350 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.227+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa8a8117db0 con 0x7fa8a8027940 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.227+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.227+0000 7fa8977fe700 10 client.40967.objecter _finish_command 7 = system:0 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.227+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a80277d0 osd=3 4 2024-02-18T00:37:09.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.227+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 3 7 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 8 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 8 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a80277d0 osd=3 3 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 3 8 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a80277d0 osd=3 4 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 10 client.40967.objecter _send_command 8 2024-02-18T00:37:09.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.241+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a8122840 con 0x7fa8a8027940 2024-02-18T00:37:09.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a8122840 con 0x7fa8a8027940 2024-02-18T00:37:09.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8977fe700 10 client.40967.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a80277d0 osd=3 4 2024-02-18T00:37:09.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 3 8 2024-02-18T00:37:09.245 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 9 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa8a8122ec0 0x7fa8a8125310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 9 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 4 9 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8122840 osd=4 4 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 10 client.40967.objecter _send_command 9 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a81280a0 con 0x7fa8a8122ec0 2024-02-18T00:37:09.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa8a8122ec0 0x7fa8a8125310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a8122ec0 2024-02-18T00:37:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.243+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa8a8122ec0 0x7fa8a8125310 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.244+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa8a81280a0 con 0x7fa8a8122ec0 2024-02-18T00:37:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.244+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:37:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.244+0000 7fa8977fe700 10 client.40967.objecter _finish_command 9 = system:0 2024-02-18T00:37:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.244+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a8122840 osd=4 4 2024-02-18T00:37:09.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.244+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 4 9 2024-02-18T00:37:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 10 2024-02-18T00:37:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 10 2024-02-18T00:37:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a8122840 osd=4 3 2024-02-18T00:37:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 4 10 2024-02-18T00:37:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8122840 osd=4 4 2024-02-18T00:37:09.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 10 client.40967.objecter _send_command 10 2024-02-18T00:37:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.259+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a81280a0 con 0x7fa8a8122ec0 2024-02-18T00:37:09.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a81280a0 con 0x7fa8a8122ec0 2024-02-18T00:37:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8977fe700 10 client.40967.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a8122840 osd=4 4 2024-02-18T00:37:09.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 4 10 2024-02-18T00:37:09.264 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:37:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 11 2024-02-18T00:37:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa8a8127420 0x7fa8a8127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 11 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 5 11 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a81280a0 osd=5 4 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 10 client.40967.objecter _send_command 11 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a8127ef0 con 0x7fa8a8127420 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.261+0000 7fa8a5d9b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa8a8127420 0x7fa8a8127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.262+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a8127420 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.262+0000 7fa8a5d9b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa8a8127420 0x7fa8a8127880 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.262+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa89c01a040 con 0x7fa8a8127420 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.262+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.262+0000 7fa8977fe700 10 client.40967.objecter _finish_command 11 = system:0 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.262+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a81280a0 osd=5 4 2024-02-18T00:37:09.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.262+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 5 11 2024-02-18T00:37:09.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 12 2024-02-18T00:37:09.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 12 2024-02-18T00:37:09.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a81280a0 osd=5 3 2024-02-18T00:37:09.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 5 12 2024-02-18T00:37:09.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a81280a0 osd=5 4 2024-02-18T00:37:09.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 10 client.40967.objecter _send_command 12 2024-02-18T00:37:09.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.277+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a811f160 con 0x7fa8a8127420 2024-02-18T00:37:09.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a811f160 con 0x7fa8a8127420 2024-02-18T00:37:09.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8977fe700 10 client.40967.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a81280a0 osd=5 4 2024-02-18T00:37:09.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 5 12 2024-02-18T00:37:09.281 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:37:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 13 2024-02-18T00:37:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa8a811f870 0x7fa8a811fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 13 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 6 13 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a811f160 osd=6 4 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 10 client.40967.objecter _send_command 13 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a812e710 con 0x7fa8a811f870 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8a6d9d700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa8a811f870 0x7fa8a811fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a811f870 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.279+0000 7fa8a6d9d700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa8a811f870 0x7fa8a811fcd0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.280+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa8a812e710 con 0x7fa8a811f870 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.280+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:37:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.280+0000 7fa8977fe700 10 client.40967.objecter _finish_command 13 = system:0 2024-02-18T00:37:09.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.280+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a811f160 osd=6 4 2024-02-18T00:37:09.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.280+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 6 13 2024-02-18T00:37:09.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.294+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 14 2024-02-18T00:37:09.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 14 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a811f160 osd=6 3 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 6 14 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a811f160 osd=6 4 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 10 client.40967.objecter _send_command 14 2024-02-18T00:37:09.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.295+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a812d590 con 0x7fa8a811f870 2024-02-18T00:37:09.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.296+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a812d590 con 0x7fa8a811f870 2024-02-18T00:37:09.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.296+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.296+0000 7fa8977fe700 10 client.40967.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.296+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a811f160 osd=6 4 2024-02-18T00:37:09.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.296+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 6 14 2024-02-18T00:37:09.299 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:37:09.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:09.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 15 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6812/53025171 conn(0x7fa8a812dc10 0x7fa8a812e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 15 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 7 15 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a812d590 osd=7 4 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 10 client.40967.objecter _send_command 15 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8a812e710 con 0x7fa8a812dc10 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6812/53025171 conn(0x7fa8a812dc10 0x7fa8a812e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:09.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8a659c700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6812/53025171 conn(0x7fa8a812dc10 0x7fa8a812e070 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:09.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.297+0000 7fa8977fe700 10 client.40967.objecter ms_handle_connect 0x7fa8a812dc10 2024-02-18T00:37:09.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.298+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa8a812e710 con 0x7fa8a812dc10 2024-02-18T00:37:09.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.298+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:37:09.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.298+0000 7fa8977fe700 10 client.40967.objecter _finish_command 15 = system:0 2024-02-18T00:37:09.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.298+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a812d590 osd=7 4 2024-02-18T00:37:09.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.298+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 7 15 2024-02-18T00:37:09.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 10 client.40967.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:09.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign -1 16 2024-02-18T00:37:09.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 20 client.40967.objecter _get_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_remove -1 16 2024-02-18T00:37:09.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 20 client.40967.objecter get_session s=0x7fa8a812d590 osd=7 3 2024-02-18T00:37:09.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 15 client.40967.objecter _session_command_op_assign 7 16 2024-02-18T00:37:09.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a812d590 osd=7 4 2024-02-18T00:37:09.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 10 client.40967.objecter _send_command 16 2024-02-18T00:37:09.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.313+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa8a812e710 con 0x7fa8a812dc10 2024-02-18T00:37:09.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8977fe700 1 -- 172.21.15.59:0/2038043615 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa8a812e710 con 0x7fa8a812dc10 2024-02-18T00:37:09.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8977fe700 10 client.40967.objecter ms_dispatch 0x7fa8a8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:09.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8977fe700 10 client.40967.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8977fe700 20 client.40967.objecter put_session s=0x7fa8a812d590 osd=7 4 2024-02-18T00:37:09.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8977fe700 15 client.40967.objecter _session_command_op_remove 7 16 2024-02-18T00:37:09.317 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:37:09.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 librados: watch_flush enter 2024-02-18T00:37:09.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 librados: watch_flush exit 2024-02-18T00:37:09.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.0 2024-02-18T00:37:09.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa8a80fc200 msgr2=0x7fa8a80fc5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa8a80fc200 0x7fa8a80fc5e0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a805ee80 osd=0 1 2024-02-18T00:37:09.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.1 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6805/26279110 conn(0x7fa8a80598f0 msgr2=0x7fa8a8059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6805/26279110 conn(0x7fa8a80598f0 0x7fa8a8059cd0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8028710 osd=1 1 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.2 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa8a80594d0 msgr2=0x7fa8a811ead0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa8a80594d0 0x7fa8a811ead0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8059360 osd=2 1 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.3 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa8a8027940 msgr2=0x7fa8a8122350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa8a8027940 0x7fa8a8122350 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a80277d0 osd=3 1 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.4 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa8a8122ec0 msgr2=0x7fa8a8125310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8a6d9d700 1 -- 172.21.15.59:0/2038043615 reap_dead start 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa8a8122ec0 0x7fa8a8125310 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8a6d9d700 1 -- 172.21.15.59:0/2038043615 reap_dead start 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a8122840 osd=4 1 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.5 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa8a8127420 msgr2=0x7fa8a8127880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa8a8127420 0x7fa8a8127880 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a81280a0 osd=5 1 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.6 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa8a811f870 msgr2=0x7fa8a811fcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa8a811f870 0x7fa8a811fcd0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a811f160 osd=6 1 2024-02-18T00:37:09.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter close_session for osd.7 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6812/53025171 conn(0x7fa8a812dc10 msgr2=0x7fa8a812e070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.137:6812/53025171 conn(0x7fa8a812dc10 0x7fa8a812e070 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter put_session s=0x7fa8a812d590 osd=7 1 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 20 client.40967.objecter shutdown clearing up homeless session... 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 10 client.40967.objecter successfully canceled tick 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa890062c90 msgr2=0x7fa890065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa890062c90 0x7fa890065150 secure :-1 s=READY pgs=3080 cs=0 l=1 rev1=1 rx=0x7fa898009350 tx=0x7fa89800b040).stop 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a8102270 msgr2=0x7fa8a8108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a8102270 0x7fa8a8108430 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 rx=0x7fa8a000d8a0 tx=0x7fa8a0004db0).stop 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8a6d9d700 1 -- 172.21.15.59:0/2038043615 reap_dead start 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 shutdown_connections 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 --2- 172.21.15.59:0/2038043615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa8a8102270 0x7fa8a8108430 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 >> 172.21.15.59:0/2038043615 conn(0x7fa8a80f2820 msgr2=0x7fa8a80f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:09.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 shutdown_connections 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 -- 172.21.15.59:0/2038043615 wait complete. 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:09.315+0000 7fa8ac93b700 1 librados: shutdown 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.322 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:37:09.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:37:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:08.715374+0000 mon.a (mon.0) 2261 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:08.742276+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:08.742394+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]': finished 2024-02-18T00:37:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:08.742523+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e350: 8 total, 8 up, 7 in 2024-02-18T00:37:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:08.745536+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:08.746279+0000 mon.a (mon.0) 2265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.157283+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.59:0/2038043615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:08.715374+0000 mon.a (mon.0) 2261 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:08.742276+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:08.742394+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]': finished 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:08.742523+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e350: 8 total, 8 up, 7 in 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:08.745536+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:08.746279+0000 mon.a (mon.0) 2265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:10.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.157283+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.59:0/2038043615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:08.715374+0000 mon.a (mon.0) 2261 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:08.742276+0000 mon.a (mon.0) 2262 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi059-82882-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:08.742394+0000 mon.a (mon.0) 2263 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-45", "mode": "writeback"}]': finished 2024-02-18T00:37:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:08.742523+0000 mon.a (mon.0) 2264 : cluster [DBG] osdmap e350: 8 total, 8 up, 7 in 2024-02-18T00:37:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:08.745536+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:08.746279+0000 mon.a (mon.0) 2265 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:10.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.157283+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.59:0/2038043615' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:10.442 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:10.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:10.445 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:10.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:10.575 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:10.648 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:10.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:10.651 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:10.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:10.782 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.743508+0000 mon.a (mon.0) 2266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:09.743603+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e351: 8 total, 8 up, 7 in 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.746319+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.746841+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.765733+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.766036+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.766557+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.766819+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.767244+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.767464+0000 mon.a (mon.0) 2271 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:09.878310+0000 mgr.y (mgr.24454) 342 : cluster [DBG] pgmap v514: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:09.900341+0000 mon.a (mon.0) 2272 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:10.783 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.743508+0000 mon.a (mon.0) 2266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:09.743603+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e351: 8 total, 8 up, 7 in 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.746319+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.746841+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.765733+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.766036+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.766557+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.766819+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.767244+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.767464+0000 mon.a (mon.0) 2271 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:09.878310+0000 mgr.y (mgr.24454) 342 : cluster [DBG] pgmap v514: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:10.784 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:09.900341+0000 mon.a (mon.0) 2272 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:10.785 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.743508+0000 mon.a (mon.0) 2266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:09.743603+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e351: 8 total, 8 up, 7 in 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.746319+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.746841+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.765733+0000 mon.c (mon.2) 333 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.766036+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.766557+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.766819+0000 mon.a (mon.0) 2270 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.767244+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:11.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.767464+0000 mon.a (mon.0) 2271 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:11.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:09.878310+0000 mgr.y (mgr.24454) 342 : cluster [DBG] pgmap v514: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:11.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:09.900341+0000 mon.a (mon.0) 2272 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:11.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-18T00:37:11.133 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:11.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:11.137 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:11.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:11.270 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:11.294 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:11.297 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:11.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:11.398 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:11.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:37:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:37:11.427 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:37:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:37:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:11.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:11.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:11.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:11.584 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:11.658 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:11.660 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:11.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:11.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:12.141 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:12.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:12.242 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:12.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:37:12.274 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:12.295 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:12.298 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:12.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:12.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:12.424 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:10.756277+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:10.756347+0000 mon.a (mon.0) 2274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:10.756418+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e352: 8 total, 8 up, 7 in 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:10.760141+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:10.762285+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:10.763139+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:10.763406+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:11.182983+0000 mgr.y (mgr.24454) 343 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:12.425 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:10.756277+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:10.756347+0000 mon.a (mon.0) 2274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:10.756418+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e352: 8 total, 8 up, 7 in 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:10.760141+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:10.762285+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:10.763139+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:10.763406+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:11.182983+0000 mgr.y (mgr.24454) 343 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:12.426 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:37:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:12] "GET /metrics HTTP/1.1" 200 189027 "" "Prometheus/2.33.4" 2024-02-18T00:37:12.427 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:12.449 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:12.451 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:12.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:10.756277+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:37:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:10.756347+0000 mon.a (mon.0) 2274 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi059-82882-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:10.756418+0000 mon.a (mon.0) 2275 : cluster [DBG] osdmap e352: 8 total, 8 up, 7 in 2024-02-18T00:37:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:10.760141+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:10.762285+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:10.763139+0000 mon.c (mon.2) 337 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:10.763406+0000 mon.a (mon.0) 2277 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:12.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:11.182983+0000 mgr.y (mgr.24454) 343 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:12.552 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:12.584 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:12.608 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:12.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 -- 172.21.15.59:0/3216091831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f9610 msgr2=0x7fa6e4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3216091831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f9610 0x7fa6e4101d20 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 rx=0x7fa6d4004870 tx=0x7fa6d4009cb0).stop 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 -- 172.21.15.59:0/3216091831 shutdown_connections 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3216091831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6e4102210 0x7fa6e4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3216091831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e40f8cb0 0x7fa6e40f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3216091831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f9610 0x7fa6e4101d20 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 -- 172.21.15.59:0/3216091831 >> 172.21.15.59:0/3216091831 conn(0x7fa6e40f2820 msgr2=0x7fa6e40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 -- 172.21.15.59:0/3216091831 shutdown_connections 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 -- 172.21.15.59:0/3216091831 wait complete. 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 librados: starting msgr at 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 librados: starting objecter 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.678+0000 7fa6eb965700 1 Processor -- start 2024-02-18T00:37:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 -- start start 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 librados: setting wanted keys 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 librados: calling monclient init 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f8cb0 0x7fa6e40f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6e40f9610 0x7fa6e40f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e4102210 0x7fa6e40f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41166d0 con 0x7fa6e4102210 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4116840 con 0x7fa6e40f9610 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6eb965700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4116a00 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f8cb0 0x7fa6e40f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9f02700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e4102210 0x7fa6e40f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9f02700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e4102210 0x7fa6e40f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48366/0 (socket says 172.21.15.59:48366) 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9f02700 1 -- 172.21.15.59:0/3421145120 learned_addr learned my addr 172.21.15.59:0/3421145120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f8cb0 0x7fa6e40f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:41200/0 (socket says 172.21.15.59:41200) 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e8f00700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6e40f9610 0x7fa6e40f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 -- 172.21.15.59:0/3421145120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6e40f9610 msgr2=0x7fa6e40f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6e40f9610 0x7fa6e40f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 -- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e4102210 msgr2=0x7fa6e40f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e4102210 0x7fa6e40f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 -- 172.21.15.59:0/3421145120 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0005040 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9f02700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e4102210 0x7fa6e40f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6e9701700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f8cb0 0x7fa6e40f6c60 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 rx=0x7fa6e0004920 tx=0x7fa6e0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6da7fc700 10 client.?.objecter ms_handle_connect 0x7fa6e40f8cb0 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6da7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.679+0000 7fa6da7fc700 1 -- 172.21.15.59:0/3421145120 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000e030 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.680+0000 7fa6da7fc700 1 -- 172.21.15.59:0/3421145120 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6e000f040 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.680+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d4005040 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.680+0000 7fa6eb965700 10 client.40979.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.680+0000 7fa6da7fc700 1 -- 172.21.15.59:0/3421145120 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0014850 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.680+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e4117040 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.680+0000 7fa6eb965700 1 librados: init done 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e405fa60 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6da7fc700 1 -- 172.21.15.59:0/3421145120 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa6e0010500 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6da7fc700 1 --2- 172.21.15.59:0/3421145120 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa6d0062c90 0x7fa6d0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6da7fc700 1 -- 172.21.15.59:0/3421145120 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(353..353 src has 44..353) v4 ==== 8448+0+0 (secure 0 0 0) 0x7fa6e007d440 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6da7fc700 10 client.40979.objecter ms_dispatch 0x7fa6e4001550 osd_map(353..353 src has 44..353) v4 2024-02-18T00:37:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6da7fc700 3 client.40979.objecter handle_osd_map got epochs [353,353] > 0 2024-02-18T00:37:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6da7fc700 3 client.40979.objecter handle_osd_map decoding full epoch 353 2024-02-18T00:37:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6e8f00700 1 --2- 172.21.15.59:0/3421145120 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa6d0062c90 0x7fa6d0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.681+0000 7fa6da7fc700 20 client.40979.objecter dump_active .. 0 homeless 2024-02-18T00:37:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.682+0000 7fa6da7fc700 10 client.40979.objecter ms_handle_connect 0x7fa6d0062c90 2024-02-18T00:37:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.682+0000 7fa6e8f00700 1 --2- 172.21.15.59:0/3421145120 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa6d0062c90 0x7fa6d0065150 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 rx=0x7fa6d4004ee0 tx=0x7fa6d400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:12.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.685+0000 7fa6da7fc700 1 -- 172.21.15.59:0/3421145120 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa6e0050790 con 0x7fa6e40f8cb0 2024-02-18T00:37:12.714 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:12.743 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:12.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:12.770 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:12.810 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:12.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:12.900 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:12.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:12.909+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fa6e40f80f0 con 0x7fa6e40f8cb0 2024-02-18T00:37:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:13.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:13.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:13.202 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:13.232 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:13.252 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:13.255 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:13.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:13.356 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:13.385 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:13.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:11.751035+0000 mon.a (mon.0) 2278 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:13.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:11.878659+0000 mgr.y (mgr.24454) 344 : cluster [DBG] pgmap v517: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:13.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.187986+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:37:13.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:12.188093+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e353: 8 total, 8 up, 7 in 2024-02-18T00:37:13.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.189074+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:13.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.194015+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:13.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.763743+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.763840+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:12.763898+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e354: 8 total, 8 up, 7 in 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.766921+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.767246+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.911423+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.59:0/3421145120' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:12.911736+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:11.751035+0000 mon.a (mon.0) 2278 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:11.878659+0000 mgr.y (mgr.24454) 344 : cluster [DBG] pgmap v517: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.187986+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:37:13.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:12.188093+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e353: 8 total, 8 up, 7 in 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.189074+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.194015+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.763743+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.763840+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:12.763898+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e354: 8 total, 8 up, 7 in 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.766921+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.767246+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.911423+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.59:0/3421145120' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:13.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:12.911736+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:11.751035+0000 mon.a (mon.0) 2278 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:11.878659+0000 mgr.y (mgr.24454) 344 : cluster [DBG] pgmap v517: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.187986+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:12.188093+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e353: 8 total, 8 up, 7 in 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.189074+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.194015+0000 mon.a (mon.0) 2281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.763743+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi059-82882-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.763840+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:12.763898+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e354: 8 total, 8 up, 7 in 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.766921+0000 mon.c (mon.2) 339 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:37:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.767246+0000 mon.a (mon.0) 2285 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:37:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.911423+0000 mon.c (mon.2) 340 : audit [INF] from='client.? 172.21.15.59:0/3421145120' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:12.911736+0000 mon.a (mon.0) 2286 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.658+0000 7fa6da7fc700 1 -- 172.21.15.59:0/3421145120 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v355) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa6e0050380 con 0x7fa6e40f8cb0 2024-02-18T00:37:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 10 librados: watch_flush enter 2024-02-18T00:37:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 10 librados: watch_flush exit 2024-02-18T00:37:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 20 client.40979.objecter shutdown clearing up homeless session... 2024-02-18T00:37:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 10 client.40979.objecter successfully canceled tick 2024-02-18T00:37:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa6d0062c90 msgr2=0x7fa6d0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3421145120 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa6d0062c90 0x7fa6d0065150 secure :-1 s=READY pgs=3082 cs=0 l=1 rev1=1 rx=0x7fa6d4004ee0 tx=0x7fa6d400b040).stop 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f8cb0 msgr2=0x7fa6e40f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f8cb0 0x7fa6e40f6c60 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 rx=0x7fa6e0004920 tx=0x7fa6e0009cb0).stop 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 shutdown_connections 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e4102210 0x7fa6e40f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e40f8cb0 0x7fa6e40f6c60 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3421145120 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa6d0062c90 0x7fa6d0065150 unknown :-1 s=CLOSED pgs=3082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 --2- 172.21.15.59:0/3421145120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6e40f9610 0x7fa6e40f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 >> 172.21.15.59:0/3421145120 conn(0x7fa6e40f2820 msgr2=0x7fa6e40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 shutdown_connections 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 -- 172.21.15.59:0/3421145120 wait complete. 2024-02-18T00:37:13.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:13.661+0000 7fa6eb965700 1 librados: shutdown 2024-02-18T00:37:13.665 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:37:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:13.644160+0000 mon.a (mon.0) 2287 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:13.652751+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T00:37:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:13.652909+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:37:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:13.652992+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e355: 8 total, 8 up, 7 in 2024-02-18T00:37:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:13.658766+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:37:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:13.659314+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:37:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:13.644160+0000 mon.a (mon.0) 2287 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:13.652751+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T00:37:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:13.652909+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:37:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:13.652992+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e355: 8 total, 8 up, 7 in 2024-02-18T00:37:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:13.658766+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:37:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:13.659314+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:37:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:13.644160+0000 mon.a (mon.0) 2287 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:13.652751+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T00:37:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:13.652909+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:37:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:13.652992+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e355: 8 total, 8 up, 7 in 2024-02-18T00:37:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:13.658766+0000 mon.c (mon.2) 341 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:37:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:13.659314+0000 mon.a (mon.0) 2291 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:37:14.666 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:14.670 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:14.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:14.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:14.803 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:13.879172+0000 mgr.y (mgr.24454) 345 : cluster [DBG] pgmap v520: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:13.993937+0000 osd.3 (osd.3) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:13.994245+0000 osd.3 (osd.3) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:14.657076+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T00:37:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:14.657207+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e356: 8 total, 8 up, 7 in 2024-02-18T00:37:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:14.660511+0000 mon.c (mon.2) 342 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:15.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:14.663706+0000 mon.a (mon.0) 2294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:14.710796+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:14.711235+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:13.879172+0000 mgr.y (mgr.24454) 345 : cluster [DBG] pgmap v520: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:13.993937+0000 osd.3 (osd.3) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:13.994245+0000 osd.3 (osd.3) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:14.657076+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:14.657207+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e356: 8 total, 8 up, 7 in 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:14.660511+0000 mon.c (mon.2) 342 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:14.663706+0000 mon.a (mon.0) 2294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:14.710796+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:15.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:14.711235+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:13.879172+0000 mgr.y (mgr.24454) 345 : cluster [DBG] pgmap v520: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:13.993937+0000 osd.3 (osd.3) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:13.994245+0000 osd.3 (osd.3) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:14.657076+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:14.657207+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e356: 8 total, 8 up, 7 in 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:14.660511+0000 mon.c (mon.2) 342 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:14.663706+0000 mon.a (mon.0) 2294 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:14.710796+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:14.711235+0000 mon.a (mon.0) 2295 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:16.452 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:16.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:16.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:16.556 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:16.589 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:16.610 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:16.613 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:16.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:bo.RoundTripAppendPP (3005 ms) 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3026 ms) 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2979 ms) 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3011 ms) 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3413 ms) 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio (129114 ms total) 2024-02-18T00:37:16.677 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2585 ms) 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9005 ms) 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (15002 ms) 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3029 ms) 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29621 ms total) 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3048 ms) 2024-02-18T00:37:16.678 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3048 ms total) 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13919 ms) 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6996 ms) 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7023 ms) 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7011 ms) 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3099 ms) 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-18T00:37:16.679 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7357 ms) 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6526 ms) 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7052 ms) 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (6974 ms) 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (6982 ms) 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-18T00:37:16.680 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7029 ms) 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7001 ms) 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (6998 ms) 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3068 ms) 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-18T00:37:16.681 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (6928 ms) 2024-02-18T00:37:16.714 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:16.749 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:15.660672+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:16.749 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:15.660777+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:15.660850+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e357: 8 total, 8 up, 7 in 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:15.665209+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:15.665531+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:15.665888+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:15.666725+0000 mon.a (mon.0) 2300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:15.880389+0000 mgr.y (mgr.24454) 346 : cluster [DBG] pgmap v523: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:15.660672+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:15.660777+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:16.750 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:15.660850+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e357: 8 total, 8 up, 7 in 2024-02-18T00:37:16.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:15.665209+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:16.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:15.665531+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:16.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:15.665888+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:16.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:15.666725+0000 mon.a (mon.0) 2300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:16.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:15.880389+0000 mgr.y (mgr.24454) 346 : cluster [DBG] pgmap v523: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:16.753 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:15.660672+0000 mon.a (mon.0) 2296 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:15.660777+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:15.660850+0000 mon.a (mon.0) 2298 : cluster [DBG] osdmap e357: 8 total, 8 up, 7 in 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:15.665209+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:15.665531+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.59:0/1800225797' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:15.665888+0000 mon.a (mon.0) 2299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:15.666725+0000 mon.a (mon.0) 2300 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]: dispatch 2024-02-18T00:37:17.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:15.880389+0000 mgr.y (mgr.24454) 346 : cluster [DBG] pgmap v523: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 -- 172.21.15.59:0/1627258936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f73d0 msgr2=0x7f7720104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 --2- 172.21.15.59:0/1627258936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f73d0 0x7f7720104170 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 rx=0x7f7714004870 tx=0x7f7714009cb0).stop 2024-02-18T00:37:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 -- 172.21.15.59:0/1627258936 shutdown_connections 2024-02-18T00:37:17.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 --2- 172.21.15.59:0/1627258936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7720104660 0x7f7720106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 --2- 172.21.15.59:0/1627258936 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77200f6b00 0x7f77200f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 --2- 172.21.15.59:0/1627258936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f73d0 0x7f7720104170 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 -- 172.21.15.59:0/1627258936 >> 172.21.15.59:0/1627258936 conn(0x7f77200f2820 msgr2=0x7f77200f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 -- 172.21.15.59:0/1627258936 shutdown_connections 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 -- 172.21.15.59:0/1627258936 wait complete. 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 librados: starting msgr at 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 librados: starting objecter 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.116+0000 7f7725f10700 1 Processor -- start 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 -- start start 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 librados: setting wanted keys 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 librados: calling monclient init 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f6b00 0x7f7720110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77200f73d0 0x7f7720110950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7720104660 0x7f772010a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772010ab90 con 0x7f77200f6b00 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772010acd0 con 0x7f77200f73d0 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f7725f10700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772010ae90 con 0x7f7720104660 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f771f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f6b00 0x7f7720110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f771ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7720104660 0x7f772010a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f771f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f6b00 0x7f7720110460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48396/0 (socket says 172.21.15.59:48396) 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f771ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7720104660 0x7f772010a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:41232/0 (socket says 172.21.15.59:41232) 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f771f7fe700 1 -- 172.21.15.59:0/733987802 learned_addr learned my addr 172.21.15.59:0/733987802 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:17.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.117+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77200f73d0 0x7f7720110950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771f7fe700 1 -- 172.21.15.59:0/733987802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7720104660 msgr2=0x7f772010a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7720104660 0x7f772010a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771f7fe700 1 -- 172.21.15.59:0/733987802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77200f73d0 msgr2=0x7f7720110950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77200f73d0 0x7f7720110950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771f7fe700 1 -- 172.21.15.59:0/733987802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714005040 con 0x7f77200f6b00 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771ffff700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7720104660 0x7f772010a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77200f73d0 0x7f7720110950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f6b00 0x7f7720110460 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 rx=0x7f771000b530 tx=0x7f77100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771cff9700 10 client.?.objecter ms_handle_connect 0x7f77200f6b00 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771cff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710002320 con 0x7f77200f6b00 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772010b170 con 0x7f77200f6b00 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7710002950 con 0x7f77200f6b00 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710012950 con 0x7f77200f6b00 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f7725f10700 10 client.31182.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77200f8d10 con 0x7f77200f6b00 2024-02-18T00:37:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.118+0000 7f7725f10700 1 librados: init done 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f7710002480 con 0x7f77200f6b00 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7708062c90 0x7f7708065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(358..358 src has 44..358) v4 ==== 8019+0+0 (secure 0 0 0) 0x7f771007d1d0 con 0x7f77200f6b00 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 osd_map(358..358 src has 44..358) v4 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 3 client.31182.objecter handle_osd_map got epochs [358,358] > 0 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 3 client.31182.objecter handle_osd_map decoding full epoch 358 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7708062c90 0x7f7708065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f772005fa60 con 0x7f77200f6b00 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 20 client.31182.objecter dump_active .. 0 homeless 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.120+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v358) v1 ==== 54+0+16 (secure 0 0 0) 0x7f7710050700 con 0x7f77200f6b00 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7708062c90 0x7f7708065150 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 rx=0x7f7714004ee0 tx=0x7f771400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f7708062c90 2024-02-18T00:37:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 10 client.31182.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 1 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7720028540 0x7f77200fe130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 1 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 0 1 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772005ee80 osd=0 4 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 10 client.31182.objecter _send_command 1 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7720109420 con 0x7f7720028540 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f771ffff700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7720028540 0x7f77200fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f7720028540 2024-02-18T00:37:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.121+0000 7f771ffff700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7720028540 0x7f77200fe130 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.122+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7720109420 con 0x7f7720028540 2024-02-18T00:37:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.122+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:37:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.122+0000 7f771cff9700 10 client.31182.objecter _finish_command 1 = system:0 2024-02-18T00:37:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.122+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f772005ee80 osd=0 4 2024-02-18T00:37:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.122+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 0 1 2024-02-18T00:37:17.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 10 client.31182.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 2 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 2 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f772005ee80 osd=0 3 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 0 2 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772005ee80 osd=0 4 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 10 client.31182.objecter _send_command 2 2024-02-18T00:37:17.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.138+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f772005a040 con 0x7f7720028540 2024-02-18T00:37:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f772005a040 con 0x7f7720028540 2024-02-18T00:37:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f771cff9700 10 client.31182.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f772005ee80 osd=0 4 2024-02-18T00:37:17.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 0 2 2024-02-18T00:37:17.141 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:37:17.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f7725f10700 10 client.31182.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 3 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.139+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6805/26279110 conn(0x7f7720027b90 0x7f7720027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 3 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 1 3 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77200ff7a0 osd=1 4 2024-02-18T00:37:17.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 10 client.31182.objecter _send_command 3 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77200ff910 con 0x7f7720027b90 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6805/26279110 conn(0x7f7720027b90 0x7f7720027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f7720027b90 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6805/26279110 conn(0x7f7720027b90 0x7f7720027f70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f77200ff910 con 0x7f7720027b90 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771cff9700 10 client.31182.objecter _finish_command 3 = system:0 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f77200ff7a0 osd=1 4 2024-02-18T00:37:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.140+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 1 3 2024-02-18T00:37:17.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 10 client.31182.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 4 2024-02-18T00:37:17.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 4 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f77200ff7a0 osd=1 3 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 1 4 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77200ff7a0 osd=1 4 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 10 client.31182.objecter _send_command 4 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.155+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f77200fcfb0 con 0x7f7720027b90 2024-02-18T00:37:17.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f77200fcfb0 con 0x7f7720027b90 2024-02-18T00:37:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f771cff9700 10 client.31182.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f77200ff7a0 osd=1 4 2024-02-18T00:37:17.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 1 4 2024-02-18T00:37:17.159 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 10 client.31182.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 5 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6809/1153193637 conn(0x7f77200ff300 0x7f772011a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 5 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 2 5 2024-02-18T00:37:17.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720027840 osd=2 4 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 10 client.31182.objecter _send_command 5 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f772011d040 con 0x7f77200ff300 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.157+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6809/1153193637 conn(0x7f77200ff300 0x7f772011a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.158+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f77200ff300 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.158+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6809/1153193637 conn(0x7f77200ff300 0x7f772011a3f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.158+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f772011d040 con 0x7f77200ff300 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.158+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.158+0000 7f771cff9700 10 client.31182.objecter _finish_command 5 = system:0 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.158+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f7720027840 osd=2 4 2024-02-18T00:37:17.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.158+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 2 5 2024-02-18T00:37:17.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 10 client.31182.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 6 2024-02-18T00:37:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 6 2024-02-18T00:37:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f7720027840 osd=2 3 2024-02-18T00:37:17.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 2 6 2024-02-18T00:37:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720027840 osd=2 4 2024-02-18T00:37:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 10 client.31182.objecter _send_command 6 2024-02-18T00:37:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.173+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f77200fe8c0 con 0x7f77200ff300 2024-02-18T00:37:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f77200fe8c0 con 0x7f77200ff300 2024-02-18T00:37:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f771cff9700 10 client.31182.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f7720027840 osd=2 4 2024-02-18T00:37:17.178 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:37:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 2 6 2024-02-18T00:37:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 10 client.31182.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 7 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6813/1805700964 conn(0x7f772011ba60 0x7f772011be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 7 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 3 7 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720117a30 osd=3 4 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 10 client.31182.objecter _send_command 7 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77201181c0 con 0x7f772011ba60 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.174+0000 7f771ffff700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6813/1805700964 conn(0x7f772011ba60 0x7f772011be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.175+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f772011ba60 2024-02-18T00:37:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.175+0000 7f771ffff700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6813/1805700964 conn(0x7f772011ba60 0x7f772011be80 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.175+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f77201181c0 con 0x7f772011ba60 2024-02-18T00:37:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.175+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:37:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.175+0000 7f771cff9700 10 client.31182.objecter _finish_command 7 = system:0 2024-02-18T00:37:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.175+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f7720117a30 osd=3 4 2024-02-18T00:37:17.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.175+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 3 7 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 10 client.31182.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 8 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 8 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f7720117a30 osd=3 3 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 3 8 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720117a30 osd=3 4 2024-02-18T00:37:17.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 10 client.31182.objecter _send_command 8 2024-02-18T00:37:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.190+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f772011cb40 con 0x7f772011ba60 2024-02-18T00:37:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f772011cb40 con 0x7f772011ba60 2024-02-18T00:37:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771cff9700 10 client.31182.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f7720117a30 osd=3 4 2024-02-18T00:37:17.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 3 8 2024-02-18T00:37:17.194 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:37:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 10 client.31182.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 9 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6800/2044387056 conn(0x7f772011f1c0 0x7f77201215b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 9 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 4 9 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011c370 osd=4 4 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 10 client.31182.objecter _send_command 9 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f772011cb40 con 0x7f772011f1c0 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6800/2044387056 conn(0x7f772011f1c0 0x7f77201215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6800/2044387056 conn(0x7f772011f1c0 0x7f77201215b0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.192+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f772011f1c0 2024-02-18T00:37:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.193+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f772011cb40 con 0x7f772011f1c0 2024-02-18T00:37:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.193+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:37:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.193+0000 7f771cff9700 10 client.31182.objecter _finish_command 9 = system:0 2024-02-18T00:37:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.193+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f772011c370 osd=4 4 2024-02-18T00:37:17.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.193+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 4 9 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 10 client.31182.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 10 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 10 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f772011c370 osd=4 3 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 4 10 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011c370 osd=4 4 2024-02-18T00:37:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 10 client.31182.objecter _send_command 10 2024-02-18T00:37:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.208+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f772011cb40 con 0x7f772011f1c0 2024-02-18T00:37:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.210+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f772011cb40 con 0x7f772011f1c0 2024-02-18T00:37:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.210+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.210+0000 7f771cff9700 10 client.31182.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.210+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f772011c370 osd=4 4 2024-02-18T00:37:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.210+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 4 10 2024-02-18T00:37:17.212 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:37:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 10 client.31182.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 11 2024-02-18T00:37:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7720121fb0 0x7f77201223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 11 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 5 11 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011cb40 osd=5 4 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 10 client.31182.objecter _send_command 11 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7720124070 con 0x7f7720121fb0 2024-02-18T00:37:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7720121fb0 0x7f77201223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f7720121fb0 2024-02-18T00:37:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.211+0000 7f771effd700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7720121fb0 0x7f77201223f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.212+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7720124070 con 0x7f7720121fb0 2024-02-18T00:37:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.212+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:37:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.212+0000 7f771cff9700 10 client.31182.objecter _finish_command 11 = system:0 2024-02-18T00:37:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.212+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f772011cb40 osd=5 4 2024-02-18T00:37:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.212+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 5 11 2024-02-18T00:37:17.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 10 client.31182.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 12 2024-02-18T00:37:17.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 12 2024-02-18T00:37:17.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f772011cb40 osd=5 3 2024-02-18T00:37:17.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 5 12 2024-02-18T00:37:17.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011cb40 osd=5 4 2024-02-18T00:37:17.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 10 client.31182.objecter _send_command 12 2024-02-18T00:37:17.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.227+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7720124070 con 0x7f7720121fb0 2024-02-18T00:37:17.230 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:37:17.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7720124070 con 0x7f7720121fb0 2024-02-18T00:37:17.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f771cff9700 10 client.31182.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f772011cb40 osd=5 4 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 5 12 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 10 client.31182.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 13 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7720126820 0x7f7720126c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 13 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 6 13 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720124070 osd=6 4 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 10 client.31182.objecter _send_command 13 2024-02-18T00:37:17.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77201272b0 con 0x7f7720126820 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.229+0000 7f771ffff700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7720126820 0x7f7720126c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.230+0000 7f771ffff700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7720126820 0x7f7720126c60 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.230+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f7720126820 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.231+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f77201272b0 con 0x7f7720126820 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.231+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.231+0000 7f771cff9700 10 client.31182.objecter _finish_command 13 = system:0 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.231+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f7720124070 osd=6 4 2024-02-18T00:37:17.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.231+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 6 13 2024-02-18T00:37:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 10 client.31182.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 14 2024-02-18T00:37:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 14 2024-02-18T00:37:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f7720124070 osd=6 3 2024-02-18T00:37:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 6 14 2024-02-18T00:37:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720124070 osd=6 4 2024-02-18T00:37:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 10 client.31182.objecter _send_command 14 2024-02-18T00:37:17.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.245+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f77201272b0 con 0x7f7720126820 2024-02-18T00:37:17.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f77201272b0 con 0x7f7720126820 2024-02-18T00:37:17.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771cff9700 10 client.31182.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f7720124070 osd=6 4 2024-02-18T00:37:17.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 6 14 2024-02-18T00:37:17.250 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:37:17.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 10 client.31182.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:17.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 15 2024-02-18T00:37:17.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6812/53025171 conn(0x7f7720129a60 0x7f7720129ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 15 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 7 15 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77201272b0 osd=7 4 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 10 client.31182.objecter _send_command 15 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f772012a4c0 con 0x7f7720129a60 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6812/53025171 conn(0x7f7720129a60 0x7f7720129ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771cff9700 10 client.31182.objecter ms_handle_connect 0x7f7720129a60 2024-02-18T00:37:17.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.248+0000 7f771f7fe700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6812/53025171 conn(0x7f7720129a60 0x7f7720129ea0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:17.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.249+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f772012a4c0 con 0x7f7720129a60 2024-02-18T00:37:17.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.249+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:37:17.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.249+0000 7f771cff9700 10 client.31182.objecter _finish_command 15 = system:0 2024-02-18T00:37:17.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.249+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f77201272b0 osd=7 4 2024-02-18T00:37:17.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.249+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 7 15 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 10 client.31182.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign -1 16 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 20 client.31182.objecter _get_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 15 client.31182.objecter _session_command_op_remove -1 16 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 20 client.31182.objecter get_session s=0x7f77201272b0 osd=7 3 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 15 client.31182.objecter _session_command_op_assign 7 16 2024-02-18T00:37:17.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77201272b0 osd=7 4 2024-02-18T00:37:17.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 10 client.31182.objecter _send_command 16 2024-02-18T00:37:17.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.264+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f772012a4c0 con 0x7f7720129a60 2024-02-18T00:37:17.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f771cff9700 1 -- 172.21.15.59:0/733987802 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f772012a4c0 con 0x7f7720129a60 2024-02-18T00:37:17.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f771cff9700 10 client.31182.objecter ms_dispatch 0x7f7720001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:17.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f771cff9700 10 client.31182.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f771cff9700 20 client.31182.objecter put_session s=0x7f77201272b0 osd=7 4 2024-02-18T00:37:17.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f771cff9700 15 client.31182.objecter _session_command_op_remove 7 16 2024-02-18T00:37:17.268 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:37:17.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 librados: watch_flush enter 2024-02-18T00:37:17.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 librados: watch_flush exit 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.0 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7720028540 msgr2=0x7f77200fe130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7720028540 0x7f77200fe130 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772005ee80 osd=0 1 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.1 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6805/26279110 conn(0x7f7720027b90 msgr2=0x7f7720027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6805/26279110 conn(0x7f7720027b90 0x7f7720027f70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77200ff7a0 osd=1 1 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.2 2024-02-18T00:37:17.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6809/1153193637 conn(0x7f77200ff300 msgr2=0x7f772011a3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6809/1153193637 conn(0x7f77200ff300 0x7f772011a3f0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720027840 osd=2 1 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.3 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6813/1805700964 conn(0x7f772011ba60 msgr2=0x7f772011be80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f771ffff700 1 -- 172.21.15.59:0/733987802 reap_dead start 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6813/1805700964 conn(0x7f772011ba60 0x7f772011be80 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720117a30 osd=3 1 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.4 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6800/2044387056 conn(0x7f772011f1c0 msgr2=0x7f77201215b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6800/2044387056 conn(0x7f772011f1c0 0x7f77201215b0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011c370 osd=4 1 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.5 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7720121fb0 msgr2=0x7f77201223f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7720121fb0 0x7f77201223f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f772011cb40 osd=5 1 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.6 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7720126820 msgr2=0x7f7720126c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7720126820 0x7f7720126c60 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f7720124070 osd=6 1 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter close_session for osd.7 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6812/53025171 conn(0x7f7720129a60 msgr2=0x7f7720129ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.137:6812/53025171 conn(0x7f7720129a60 0x7f7720129ea0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter put_session s=0x7f77201272b0 osd=7 1 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 20 client.31182.objecter shutdown clearing up homeless session... 2024-02-18T00:37:17.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 10 client.31182.objecter successfully canceled tick 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f771ffff700 1 -- 172.21.15.59:0/733987802 reap_dead start 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7708062c90 msgr2=0x7f7708065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7708062c90 0x7f7708065150 secure :-1 s=READY pgs=3084 cs=0 l=1 rev1=1 rx=0x7f7714004ee0 tx=0x7f771400b040).stop 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f6b00 msgr2=0x7f7720110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f6b00 0x7f7720110460 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 rx=0x7f771000b530 tx=0x7f77100090f0).stop 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 shutdown_connections 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7708062c90 0x7f7708065150 unknown :-1 s=CLOSED pgs=3084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 --2- 172.21.15.59:0/733987802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77200f6b00 0x7f7720110460 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 >> 172.21.15.59:0/733987802 conn(0x7f77200f2820 msgr2=0x7f77200f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.266+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 shutdown_connections 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.267+0000 7f7725f10700 1 -- 172.21.15.59:0/733987802 wait complete. 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:17.267+0000 7f7725f10700 1 librados: shutdown 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.274 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.274 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.274 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.274 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.274 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.274 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:37:17.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:37:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:17.790 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:17.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:17.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:17.921 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:17.974 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:17.976 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:16.582748+0000 osd.4 (osd.4) 205 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:17.976 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:16.583000+0000 osd.4 (osd.4) 206 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:17.976 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.670376+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]': finished 2024-02-18T00:37:17.976 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.670481+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:16.670548+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e358: 8 total, 8 up, 7 in 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.697323+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.698278+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.698703+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.708177+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.708520+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.709112+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:16.709498+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:17.121950+0000 mon.a (mon.0) 2309 : audit [DBG] from='client.? 172.21.15.59:0/733987802' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:17.977 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:16.582748+0000 osd.4 (osd.4) 205 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:16.583000+0000 osd.4 (osd.4) 206 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.670376+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]': finished 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.670481+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:16.670548+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e358: 8 total, 8 up, 7 in 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.697323+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.698278+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.698703+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.708177+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.708520+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.709112+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:16.709498+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:17.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:17.121950+0000 mon.a (mon.0) 2309 : audit [DBG] from='client.? 172.21.15.59:0/733987802' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:17.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:16.582748+0000 osd.4 (osd.4) 205 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:16.583000+0000 osd.4 (osd.4) 206 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.670376+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]': finished 2024-02-18T00:37:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.670481+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi059-82882-60"}]': finished 2024-02-18T00:37:18.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:16.670548+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e358: 8 total, 8 up, 7 in 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.697323+0000 mon.a (mon.0) 2304 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.698278+0000 mon.a (mon.0) 2305 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.698703+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.708177+0000 mon.c (mon.2) 346 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.708520+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.709112+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:16.709498+0000 mon.a (mon.0) 2308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-45"}]: dispatch 2024-02-18T00:37:18.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:17.121950+0000 mon.a (mon.0) 2309 : audit [DBG] from='client.? 172.21.15.59:0/733987802' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:18.080 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:18.113 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:18.134 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:18.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:18.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:18.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:18.268 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:18.289 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:18.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:18.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:18.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:18.423 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:18.781 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:18.784 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:18.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:18.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:18.913 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:18.941 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:18.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:17.666960+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:18.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:17.667060+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e359: 8 total, 8 up, 7 in 2024-02-18T00:37:18.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:17.676483+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi059-82882-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:18.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:17.880727+0000 mgr.y (mgr.24454) 347 : cluster [DBG] pgmap v526: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:18.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:17.666960+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:18.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:17.667060+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e359: 8 total, 8 up, 7 in 2024-02-18T00:37:18.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:17.676483+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi059-82882-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:18.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:17.880727+0000 mgr.y (mgr.24454) 347 : cluster [DBG] pgmap v526: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:18.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:18.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:19.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:17.666960+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:17.667060+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e359: 8 total, 8 up, 7 in 2024-02-18T00:37:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:17.676483+0000 mon.a (mon.0) 2312 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi059-82882-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:17.880727+0000 mgr.y (mgr.24454) 347 : cluster [DBG] pgmap v526: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:19.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:19.078 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:19.190 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:19.192 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:19.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:19.293 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:19.326 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:19.346 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:19.349 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:19.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:19.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:19.483 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:19.565 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:37:19.565 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:37:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:19.145020+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e360: 8 total, 8 up, 7 in 2024-02-18T00:37:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:19.147441+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:19.147812+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:19.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:19.145020+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e360: 8 total, 8 up, 7 in 2024-02-18T00:37:19.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:19.147441+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:19.147812+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:19.145020+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e360: 8 total, 8 up, 7 in 2024-02-18T00:37:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:19.147441+0000 mon.c (mon.2) 348 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:19.147812+0000 mon.a (mon.0) 2314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:20.233 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:20.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:20.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:20.254 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-18T00:37:20.338 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:20.369 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:20.391 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:20.393 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:20.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:20.495 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:20.522 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:20.645 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:20.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:20.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:20.749 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:20.781 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:20.801 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:20.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:20.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 -- 172.21.15.59:0/2756071093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 msgr2=0x7efc780fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 --2- 172.21.15.59:0/2756071093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780fdfb0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 rx=0x7efc680047b0 tx=0x7efc68009cb0).stop 2024-02-18T00:37:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 -- 172.21.15.59:0/2756071093 shutdown_connections 2024-02-18T00:37:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 --2- 172.21.15.59:0/2756071093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc780f6a90 0x7efc78104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 --2- 172.21.15.59:0/2756071093 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780fdfb0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 --2- 172.21.15.59:0/2756071093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc780fe530 0x7efc780f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 -- 172.21.15.59:0/2756071093 >> 172.21.15.59:0/2756071093 conn(0x7efc7806c370 msgr2=0x7efc7806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 -- 172.21.15.59:0/2756071093 shutdown_connections 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.576+0000 7efc7e73a700 1 -- 172.21.15.59:0/2756071093 wait complete. 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 librados: starting msgr at 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 librados: starting objecter 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 Processor -- start 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 -- start start 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 librados: setting wanted keys 2024-02-18T00:37:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 librados: calling monclient init 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc780f6a90 0x7efc780f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc780fe530 0x7efc780f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc78116630 con 0x7efc780fe530 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc781167a0 con 0x7efc780f6a90 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc7e73a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc78116960 con 0x7efc780fdbd0 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.577+0000 7efc777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc7ccd7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc780fe530 0x7efc780f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:42236/0 (socket says 172.21.15.59:42236) 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 -- 172.21.15.59:0/3666767653 learned_addr learned my addr 172.21.15.59:0/3666767653 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc77fff700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc780f6a90 0x7efc780f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 -- 172.21.15.59:0/3666767653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc780f6a90 msgr2=0x7efc780f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc780f6a90 0x7efc780f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 -- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc780fe530 msgr2=0x7efc780f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc780fe530 0x7efc780f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 -- 172.21.15.59:0/3666767653 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c005040 con 0x7efc780fdbd0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc7ccd7700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc780fe530 0x7efc780f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc777fe700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780f7b70 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 rx=0x7efc6c009400 tx=0x7efc6c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc757fa700 10 client.?.objecter ms_handle_connect 0x7efc780fdbd0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc757fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc757fa700 1 -- 172.21.15.59:0/3666767653 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c007a40 con 0x7efc780fdbd0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc68005040 con 0x7efc780fdbd0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc757fa700 1 -- 172.21.15.59:0/3666767653 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6c00f040 con 0x7efc780fdbd0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc757fa700 1 -- 172.21.15.59:0/3666767653 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c014aa0 con 0x7efc780fdbd0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc7e73a700 10 client.40991.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc78116ff0 con 0x7efc780fdbd0 2024-02-18T00:37:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.578+0000 7efc7e73a700 1 librados: init done 2024-02-18T00:37:20.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.579+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7805fa40 con 0x7efc780fdbd0 2024-02-18T00:37:20.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 1 -- 172.21.15.59:0/3666767653 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efc6c00e630 con 0x7efc780fdbd0 2024-02-18T00:37:20.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 1 --2- 172.21.15.59:0/3666767653 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc60062c90 0x7efc60065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:20.827 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:37:20.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 1 -- 172.21.15.59:0/3666767653 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(360..360 src has 180..360) v4 ==== 8405+0+0 (secure 0 0 0) 0x7efc6c07cf90 con 0x7efc780fdbd0 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 10 client.40991.objecter ms_dispatch 0x7efc78001550 osd_map(360..360 src has 180..360) v4 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 3 client.40991.objecter handle_osd_map got epochs [360,360] > 0 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 3 client.40991.objecter handle_osd_map decoding full epoch 360 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc77fff700 1 --2- 172.21.15.59:0/3666767653 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc60062c90 0x7efc60065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 20 client.40991.objecter dump_active .. 0 homeless 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc757fa700 10 client.40991.objecter ms_handle_connect 0x7efc60062c90 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.580+0000 7efc77fff700 1 --2- 172.21.15.59:0/3666767653 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc60062c90 0x7efc60065150 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 rx=0x7efc68005010 tx=0x7efc68009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.584+0000 7efc757fa700 1 -- 172.21.15.59:0/3666767653 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efc6c049f60 con 0x7efc780fdbd0 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.771+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc780f8b60 con 0x7efc60062c90 2024-02-18T00:37:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:19.788+0000 7efc757fa700 1 -- 172.21.15.59:0/3666767653 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+612878 (secure 0 0 0) 0x7efc780f8b60 con 0x7efc60062c90 2024-02-18T00:37:20.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 10 librados: watch_flush enter 2024-02-18T00:37:20.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 10 librados: watch_flush exit 2024-02-18T00:37:20.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 20 client.40991.objecter shutdown clearing up homeless session... 2024-02-18T00:37:20.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 10 client.40991.objecter successfully canceled tick 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc60062c90 msgr2=0x7efc60065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 --2- 172.21.15.59:0/3666767653 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc60062c90 0x7efc60065150 secure :-1 s=READY pgs=3085 cs=0 l=1 rev1=1 rx=0x7efc68005010 tx=0x7efc68009cb0).stop 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 msgr2=0x7efc780f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780f7b70 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 rx=0x7efc6c009400 tx=0x7efc6c009cb0).stop 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 shutdown_connections 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 --2- 172.21.15.59:0/3666767653 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc60062c90 0x7efc60065150 unknown :-1 s=CLOSED pgs=3085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc780f6a90 0x7efc780f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc780fe530 0x7efc780f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 --2- 172.21.15.59:0/3666767653 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc780fdbd0 0x7efc780f7b70 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 >> 172.21.15.59:0/3666767653 conn(0x7efc7806c370 msgr2=0x7efc780fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 shutdown_connections 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.825+0000 7efc7e73a700 1 -- 172.21.15.59:0/3666767653 wait complete. 2024-02-18T00:37:20.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:20.826+0000 7efc7e73a700 1 librados: shutdown 2024-02-18T00:37:20.842 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:37:20.842 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:20.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:19.689177+0000 mon.a (mon.0) 2315 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi059-82882-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:20.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:19.689396+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:20.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:19.689464+0000 mon.a (mon.0) 2317 : cluster [DBG] osdmap e361: 8 total, 8 up, 7 in 2024-02-18T00:37:20.854 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:19.717967+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:20.854 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:19.718340+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:20.854 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:19.772545+0000 mgr.y (mgr.24454) 348 : audit [DBG] from='client.40991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:20.854 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:19.881078+0000 mgr.y (mgr.24454) 349 : cluster [DBG] pgmap v529: 246 pgs: 40 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:20.854 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:19.689177+0000 mon.a (mon.0) 2315 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi059-82882-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:20.854 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:19.689396+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:20.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:19.689464+0000 mon.a (mon.0) 2317 : cluster [DBG] osdmap e361: 8 total, 8 up, 7 in 2024-02-18T00:37:20.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:19.717967+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:20.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:19.718340+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:20.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:19.772545+0000 mgr.y (mgr.24454) 348 : audit [DBG] from='client.40991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:20.855 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:19.881078+0000 mgr.y (mgr.24454) 349 : cluster [DBG] pgmap v529: 246 pgs: 40 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:20.905 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:20.937 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:20.958 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:20.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:19.689177+0000 mon.a (mon.0) 2315 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi059-82882-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:20.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:19.689396+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:20.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:19.689464+0000 mon.a (mon.0) 2317 : cluster [DBG] osdmap e361: 8 total, 8 up, 7 in 2024-02-18T00:37:20.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:19.717967+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:20.960 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:19.718340+0000 mon.a (mon.0) 2318 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:20.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:19.772545+0000 mgr.y (mgr.24454) 348 : audit [DBG] from='client.40991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:20.961 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:19.881078+0000 mgr.y (mgr.24454) 349 : cluster [DBG] pgmap v529: 246 pgs: 40 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:20.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:20.962 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:21.062 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:21.093 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:21.112 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:21.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:21.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:21.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:21.246 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:21.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:37:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:37:21.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:37:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:37:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:21.497 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:21.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:21.598 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:21.629 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:21.650 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:21.653 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:21.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:20.701829+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:20.701918+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e362: 8 total, 8 up, 7 in 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:20.707071+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:20.707957+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:21.193906+0000 mgr.y (mgr.24454) 350 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:20.701829+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:20.701918+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e362: 8 total, 8 up, 7 in 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:20.707071+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:21.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:20.707957+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:21.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:21.193906+0000 mgr.y (mgr.24454) 350 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:21.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:21.784 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:21.933 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:21.935 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:21.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:21.936 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:20.701829+0000 mon.a (mon.0) 2319 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:20.701918+0000 mon.a (mon.0) 2320 : cluster [DBG] osdmap e362: 8 total, 8 up, 7 in 2024-02-18T00:37:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:20.707071+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:20.707957+0000 mon.a (mon.0) 2321 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:21.193906+0000 mgr.y (mgr.24454) 350 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:22.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:22.064 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:22.263 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:22.265 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:22.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:22.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:22.394 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:22] "GET /metrics HTTP/1.1" 200 189027 "" "Prometheus/2.33.4" 2024-02-18T00:37:22.396 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:22.664 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:37:22.888 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2024-02-18T00:37:19.686556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007181,"num_objects":368,"num_object_clones":2,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":0,"num_read":1674,"num_read_kb":1301,"num_write":1194,"num_write_kb":4516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2539,"ondisk_log_size":2539,"up":618,"acting":618,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":730,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2161764,"kb_used_data":52476,"kb_used_omap":0,"kb_used_meta":2109184,"kb_avail":654054300,"statfs":{"total":671965249536,"available":669751603200,"internally_reserved":0,"allocated":53735424,"data_stored":35106832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2159804416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028494"},"pg_stats":[{"pgid":"200.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"201.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:19.679718+0000","last_change":"2024-02-18T00:37:19.679718+0000","last_active":"2024-02-18T00:37:19.679718+0000","last_peered":"2024-02-18T00:37:19.679718+0000","last_clean":"2024-02-18T00:37:19.679718+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:19.679718+0000","last_undegraded":"2024-02-18T00:37:19.679718+0000","last_fullsized":"2024-02-18T00:37:19.679718+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:19.679718+0000","last_clean_scrub_stamp":"2024-02-18T00:37:19.679718+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"200.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:37:18.672560+0000","last_change":"2024-02-18T00:37:18.672560+0000","last_active":"2024-02-18T00:37:18.672560+0000","last_peered":"2024-02-18T00:37:18.672560+0000","last_clean":"2024-02-18T00:37:18.672560+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:37:18.672560+0000","last_undegraded":"2024-02-18T00:37:18.672560+0000","last_fullsized":"2024-02-18T00:37:18.672560+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:18.672560+0000","last_clean_scrub_stamp":"2024-02-18T00:37:18.672560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":328,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.659542+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:37:14.659542+0000","last_peered":"2024-02-18T00:37:14.659542+0000","last_clean":"2024-02-18T00:37:14.659542+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:37:14.659542+0000","last_undegraded":"2024-02-18T00:37:14.659542+0000","last_fullsized":"2024-02-18T00:37:14.659542+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"347'2","reported_seq":291,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.824948+0000","last_change":"2024-02-18T00:36:06.658623+0000","last_active":"2024-02-18T00:37:13.824948+0000","last_peered":"2024-02-18T00:37:13.824948+0000","last_clean":"2024-02-18T00:37:13.824948+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:37:13.824948+0000","last_undegraded":"2024-02-18T00:37:13.824948+0000","last_fullsized":"2024-02-18T00:37:13.824948+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:30.476947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.476947+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":356,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698231+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:37:13.698231+0000","last_peered":"2024-02-18T00:37:13.698231+0000","last_clean":"2024-02-18T00:37:13.698231+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:37:13.698231+0000","last_undegraded":"2024-02-18T00:37:13.698231+0000","last_fullsized":"2024-02-18T00:37:13.698231+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"347'75","reported_seq":359,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657549+0000","last_change":"2024-02-18T00:36:34.581378+0000","last_active":"2024-02-18T00:37:14.657549+0000","last_peered":"2024-02-18T00:37:14.657549+0000","last_clean":"2024-02-18T00:37:14.657549+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:37:14.657549+0000","last_undegraded":"2024-02-18T00:37:14.657549+0000","last_fullsized":"2024-02-18T00:37:14.657549+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'73","last_scrub_stamp":"2024-02-18T00:36:34.581302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.581302+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"230'120","reported_seq":395,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656670+0000","last_change":"2024-02-18T00:36:36.464430+0000","last_active":"2024-02-18T00:37:14.656670+0000","last_peered":"2024-02-18T00:37:14.656670+0000","last_clean":"2024-02-18T00:37:14.656670+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:37:14.656670+0000","last_undegraded":"2024-02-18T00:37:14.656670+0000","last_fullsized":"2024-02-18T00:37:14.656670+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'120","last_scrub_stamp":"2024-02-18T00:36:36.464361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.464361+0000","log_size":120,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":36,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":372,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657531+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:37:14.657531+0000","last_peered":"2024-02-18T00:37:14.657531+0000","last_clean":"2024-02-18T00:37:14.657531+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:37:14.657531+0000","last_undegraded":"2024-02-18T00:37:14.657531+0000","last_fullsized":"2024-02-18T00:37:14.657531+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"347'125","reported_seq":371,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660483+0000","last_change":"2024-02-18T00:36:53.649744+0000","last_active":"2024-02-18T00:37:14.660483+0000","last_peered":"2024-02-18T00:37:14.660483+0000","last_clean":"2024-02-18T00:37:14.660483+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:37:14.660483+0000","last_undegraded":"2024-02-18T00:37:14.660483+0000","last_fullsized":"2024-02-18T00:37:14.660483+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'124","last_scrub_stamp":"2024-02-18T00:36:53.649677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.649677+0000","log_size":125,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":38,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"347'74","reported_seq":352,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.663021+0000","last_change":"2024-02-18T00:36:37.532389+0000","last_active":"2024-02-18T00:37:14.663021+0000","last_peered":"2024-02-18T00:37:14.663021+0000","last_clean":"2024-02-18T00:37:14.663021+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:37:14.663021+0000","last_undegraded":"2024-02-18T00:37:14.663021+0000","last_fullsized":"2024-02-18T00:37:14.663021+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'73","last_scrub_stamp":"2024-02-18T00:36:37.532316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.532316+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":23,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'70","reported_seq":318,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.824883+0000","last_change":"2024-02-18T00:36:06.662037+0000","last_active":"2024-02-18T00:37:13.824883+0000","last_peered":"2024-02-18T00:37:13.824883+0000","last_clean":"2024-02-18T00:37:13.824883+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:37:13.824883+0000","last_undegraded":"2024-02-18T00:37:13.824883+0000","last_fullsized":"2024-02-18T00:37:13.824883+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-18T00:36:02.645624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.645624+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"347'52","reported_seq":335,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656167+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:37:14.656167+0000","last_peered":"2024-02-18T00:37:14.656167+0000","last_clean":"2024-02-18T00:37:14.656167+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:37:14.656167+0000","last_undegraded":"2024-02-18T00:37:14.656167+0000","last_fullsized":"2024-02-18T00:37:14.656167+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'41","reported_seq":302,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656537+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:37:14.656537+0000","last_peered":"2024-02-18T00:37:14.656537+0000","last_clean":"2024-02-18T00:37:14.656537+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:37:14.656537+0000","last_undegraded":"2024-02-18T00:37:14.656537+0000","last_fullsized":"2024-02-18T00:37:14.656537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"347'55","reported_seq":317,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.697918+0000","last_change":"2024-02-18T00:36:11.810050+0000","last_active":"2024-02-18T00:37:14.697918+0000","last_peered":"2024-02-18T00:37:14.697918+0000","last_clean":"2024-02-18T00:37:14.697918+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:37:14.697918+0000","last_undegraded":"2024-02-18T00:37:14.697918+0000","last_fullsized":"2024-02-18T00:37:14.697918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'54","last_scrub_stamp":"2024-02-18T00:36:11.809925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.809925+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":18,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"213'17","reported_seq":303,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655131+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:37:13.655131+0000","last_peered":"2024-02-18T00:37:13.655131+0000","last_clean":"2024-02-18T00:37:13.655131+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:37:13.655131+0000","last_undegraded":"2024-02-18T00:37:13.655131+0000","last_fullsized":"2024-02-18T00:37:13.655131+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"347'29","reported_seq":339,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657422+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:37:14.657422+0000","last_peered":"2024-02-18T00:37:14.657422+0000","last_clean":"2024-02-18T00:37:14.657422+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:37:14.657422+0000","last_undegraded":"2024-02-18T00:37:14.657422+0000","last_fullsized":"2024-02-18T00:37:14.657422+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"347'83","reported_seq":358,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660845+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:37:14.660845+0000","last_peered":"2024-02-18T00:37:14.660845+0000","last_clean":"2024-02-18T00:37:14.660845+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:37:14.660845+0000","last_undegraded":"2024-02-18T00:37:14.660845+0000","last_fullsized":"2024-02-18T00:37:14.660845+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":67,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":339,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658327+0000","last_change":"2024-02-18T00:36:34.207908+0000","last_active":"2024-02-18T00:37:14.658327+0000","last_peered":"2024-02-18T00:37:14.658327+0000","last_clean":"2024-02-18T00:37:14.658327+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:37:14.658327+0000","last_undegraded":"2024-02-18T00:37:14.658327+0000","last_fullsized":"2024-02-18T00:37:14.658327+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:36:34.207840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.207840+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'57","reported_seq":352,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.824876+0000","last_change":"2024-02-18T00:36:06.661736+0000","last_active":"2024-02-18T00:37:13.824876+0000","last_peered":"2024-02-18T00:37:13.824876+0000","last_clean":"2024-02-18T00:37:13.824876+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:37:13.824876+0000","last_undegraded":"2024-02-18T00:37:13.824876+0000","last_fullsized":"2024-02-18T00:37:13.824876+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'56","last_scrub_stamp":"2024-02-18T00:36:04.656413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.656413+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"214'19","reported_seq":309,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.825328+0000","last_change":"2024-02-18T00:36:06.658579+0000","last_active":"2024-02-18T00:37:13.825328+0000","last_peered":"2024-02-18T00:37:13.825328+0000","last_clean":"2024-02-18T00:37:13.825328+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:37:13.825328+0000","last_undegraded":"2024-02-18T00:37:13.825328+0000","last_fullsized":"2024-02-18T00:37:13.825328+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:35:53.563562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.563562+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"347'74","reported_seq":354,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656649+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:37:14.656649+0000","last_peered":"2024-02-18T00:37:14.656649+0000","last_clean":"2024-02-18T00:37:14.656649+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:37:14.656649+0000","last_undegraded":"2024-02-18T00:37:14.656649+0000","last_fullsized":"2024-02-18T00:37:14.656649+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"210'17","reported_seq":304,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.659860+0000","last_change":"2024-02-18T00:36:43.345182+0000","last_active":"2024-02-18T00:37:14.659860+0000","last_peered":"2024-02-18T00:37:14.659860+0000","last_clean":"2024-02-18T00:37:14.659860+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:37:14.659860+0000","last_undegraded":"2024-02-18T00:37:14.659860+0000","last_fullsized":"2024-02-18T00:37:14.659860+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'17","last_scrub_stamp":"2024-02-18T00:36:43.345064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.345064+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":318,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657629+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:37:14.657629+0000","last_peered":"2024-02-18T00:37:14.657629+0000","last_clean":"2024-02-18T00:37:14.657629+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:37:14.657629+0000","last_undegraded":"2024-02-18T00:37:14.657629+0000","last_fullsized":"2024-02-18T00:37:14.657629+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"347'20","reported_seq":317,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699543+0000","last_change":"2024-02-18T00:36:06.659238+0000","last_active":"2024-02-18T00:37:13.699543+0000","last_peered":"2024-02-18T00:37:13.699543+0000","last_clean":"2024-02-18T00:37:13.699543+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:37:13.699543+0000","last_undegraded":"2024-02-18T00:37:13.699543+0000","last_fullsized":"2024-02-18T00:37:13.699543+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000027,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7,"num_write_kb":3913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":382,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654940+0000","last_change":"2024-02-18T00:36:34.017135+0000","last_active":"2024-02-18T00:37:13.654940+0000","last_peered":"2024-02-18T00:37:13.654940+0000","last_clean":"2024-02-18T00:37:13.654940+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:37:13.654940+0000","last_undegraded":"2024-02-18T00:37:13.654940+0000","last_fullsized":"2024-02-18T00:37:13.654940+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:36:34.017065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.017065+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":274,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698070+0000","last_change":"2024-02-18T00:36:37.504644+0000","last_active":"2024-02-18T00:37:13.698070+0000","last_peered":"2024-02-18T00:37:13.698070+0000","last_clean":"2024-02-18T00:37:13.698070+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:37:13.698070+0000","last_undegraded":"2024-02-18T00:37:13.698070+0000","last_fullsized":"2024-02-18T00:37:13.698070+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:37.504568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.504568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":396,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.662624+0000","last_change":"2024-02-18T00:36:38.583124+0000","last_active":"2024-02-18T00:37:14.662624+0000","last_peered":"2024-02-18T00:37:14.662624+0000","last_clean":"2024-02-18T00:37:14.662624+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:37:14.662624+0000","last_undegraded":"2024-02-18T00:37:14.662624+0000","last_fullsized":"2024-02-18T00:37:14.662624+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'112","last_scrub_stamp":"2024-02-18T00:36:38.583056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.583056+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":273,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654418+0000","last_change":"2024-02-18T00:37:01.774863+0000","last_active":"2024-02-18T00:37:13.654418+0000","last_peered":"2024-02-18T00:37:13.654418+0000","last_clean":"2024-02-18T00:37:13.654418+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:37:13.654418+0000","last_undegraded":"2024-02-18T00:37:13.654418+0000","last_fullsized":"2024-02-18T00:37:13.654418+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.774795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.774795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":330,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656920+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:37:14.656920+0000","last_peered":"2024-02-18T00:37:14.656920+0000","last_clean":"2024-02-18T00:37:14.656920+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:37:14.656920+0000","last_undegraded":"2024-02-18T00:37:14.656920+0000","last_fullsized":"2024-02-18T00:37:14.656920+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":227,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658338+0000","last_change":"2024-02-18T00:34:27.300646+0000","last_active":"2024-02-18T00:37:14.658338+0000","last_peered":"2024-02-18T00:37:14.658338+0000","last_clean":"2024-02-18T00:37:14.658338+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:37:14.658338+0000","last_undegraded":"2024-02-18T00:37:14.658338+0000","last_fullsized":"2024-02-18T00:37:14.658338+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:27.300572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:27.300572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":383,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698503+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:37:13.698503+0000","last_peered":"2024-02-18T00:37:13.698503+0000","last_clean":"2024-02-18T00:37:13.698503+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:37:13.698503+0000","last_undegraded":"2024-02-18T00:37:13.698503+0000","last_fullsized":"2024-02-18T00:37:13.698503+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":230,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657809+0000","last_change":"2024-02-18T00:36:53.345938+0000","last_active":"2024-02-18T00:37:14.657809+0000","last_peered":"2024-02-18T00:37:14.657809+0000","last_clean":"2024-02-18T00:37:14.657809+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:37:14.657809+0000","last_undegraded":"2024-02-18T00:37:14.657809+0000","last_fullsized":"2024-02-18T00:37:14.657809+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.345872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":360,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.697993+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:37:13.697993+0000","last_peered":"2024-02-18T00:37:13.697993+0000","last_clean":"2024-02-18T00:37:13.697993+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:37:13.697993+0000","last_undegraded":"2024-02-18T00:37:13.697993+0000","last_fullsized":"2024-02-18T00:37:13.697993+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":282,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658500+0000","last_change":"2024-02-18T00:36:39.168150+0000","last_active":"2024-02-18T00:37:14.658500+0000","last_peered":"2024-02-18T00:37:14.658500+0000","last_clean":"2024-02-18T00:37:14.658500+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:37:14.658500+0000","last_undegraded":"2024-02-18T00:37:14.658500+0000","last_fullsized":"2024-02-18T00:37:14.658500+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:36:39.168079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:39.168079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"347'18","reported_seq":310,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654388+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:37:13.654388+0000","last_peered":"2024-02-18T00:37:13.654388+0000","last_clean":"2024-02-18T00:37:13.654388+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:37:13.654388+0000","last_undegraded":"2024-02-18T00:37:13.654388+0000","last_fullsized":"2024-02-18T00:37:13.654388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":274,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699213+0000","last_change":"2024-02-18T00:36:35.460784+0000","last_active":"2024-02-18T00:37:13.699213+0000","last_peered":"2024-02-18T00:37:13.699213+0000","last_clean":"2024-02-18T00:37:13.699213+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:37:13.699213+0000","last_undegraded":"2024-02-18T00:37:13.699213+0000","last_fullsized":"2024-02-18T00:37:13.699213+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.460703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.460703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":341,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.662937+0000","last_change":"2024-02-18T00:36:41.624250+0000","last_active":"2024-02-18T00:37:14.662937+0000","last_peered":"2024-02-18T00:37:14.662937+0000","last_clean":"2024-02-18T00:37:14.662937+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:37:14.662937+0000","last_undegraded":"2024-02-18T00:37:14.662937+0000","last_fullsized":"2024-02-18T00:37:14.662937+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:36:41.624180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.624180+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":264,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655299+0000","last_change":"2024-02-18T00:36:43.933486+0000","last_active":"2024-02-18T00:37:13.655299+0000","last_peered":"2024-02-18T00:37:13.655299+0000","last_clean":"2024-02-18T00:37:13.655299+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:37:13.655299+0000","last_undegraded":"2024-02-18T00:37:13.655299+0000","last_fullsized":"2024-02-18T00:37:13.655299+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.933414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.933414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":294,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699121+0000","last_change":"2024-02-18T00:36:06.659436+0000","last_active":"2024-02-18T00:37:13.699121+0000","last_peered":"2024-02-18T00:37:13.699121+0000","last_clean":"2024-02-18T00:37:13.699121+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:37:13.699121+0000","last_undegraded":"2024-02-18T00:37:13.699121+0000","last_fullsized":"2024-02-18T00:37:13.699121+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:35:28.376119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.376119+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":267,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699307+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:37:13.699307+0000","last_peered":"2024-02-18T00:37:13.699307+0000","last_clean":"2024-02-18T00:37:13.699307+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:37:13.699307+0000","last_undegraded":"2024-02-18T00:37:13.699307+0000","last_fullsized":"2024-02-18T00:37:13.699307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"199'17","reported_seq":303,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655666+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:37:13.655666+0000","last_peered":"2024-02-18T00:37:13.655666+0000","last_clean":"2024-02-18T00:37:13.655666+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:37:13.655666+0000","last_undegraded":"2024-02-18T00:37:13.655666+0000","last_fullsized":"2024-02-18T00:37:13.655666+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":11,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":347,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658684+0000","last_change":"2024-02-18T00:36:45.627771+0000","last_active":"2024-02-18T00:37:14.658684+0000","last_peered":"2024-02-18T00:37:14.658684+0000","last_clean":"2024-02-18T00:37:14.658684+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:37:14.658684+0000","last_undegraded":"2024-02-18T00:37:14.658684+0000","last_fullsized":"2024-02-18T00:37:14.658684+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.627700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.627700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":305,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699102+0000","last_change":"2024-02-18T00:36:06.657624+0000","last_active":"2024-02-18T00:37:13.699102+0000","last_peered":"2024-02-18T00:37:13.699102+0000","last_clean":"2024-02-18T00:37:13.699102+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:37:13.699102+0000","last_undegraded":"2024-02-18T00:37:13.699102+0000","last_fullsized":"2024-02-18T00:37:13.699102+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"227'17","last_scrub_stamp":"2024-02-18T00:35:30.343441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.343441+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":292,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655375+0000","last_change":"2024-02-18T00:35:08.848358+0000","last_active":"2024-02-18T00:37:13.655375+0000","last_peered":"2024-02-18T00:37:13.655375+0000","last_clean":"2024-02-18T00:37:13.655375+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:37:13.655375+0000","last_undegraded":"2024-02-18T00:37:13.655375+0000","last_fullsized":"2024-02-18T00:37:13.655375+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:08.848248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:08.848248+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":271,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.659805+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:37:14.659805+0000","last_peered":"2024-02-18T00:37:14.659805+0000","last_clean":"2024-02-18T00:37:14.659805+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:37:14.659805+0000","last_undegraded":"2024-02-18T00:37:14.659805+0000","last_fullsized":"2024-02-18T00:37:14.659805+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":287,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698431+0000","last_change":"2024-02-18T00:34:23.258526+0000","last_active":"2024-02-18T00:37:13.698431+0000","last_peered":"2024-02-18T00:37:13.698431+0000","last_clean":"2024-02-18T00:37:13.698431+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:37:13.698431+0000","last_undegraded":"2024-02-18T00:37:13.698431+0000","last_fullsized":"2024-02-18T00:37:13.698431+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:23.258454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:23.258454+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":402,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655469+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:37:13.655469+0000","last_peered":"2024-02-18T00:37:13.655469+0000","last_clean":"2024-02-18T00:37:13.655469+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:37:13.655469+0000","last_undegraded":"2024-02-18T00:37:13.655469+0000","last_fullsized":"2024-02-18T00:37:13.655469+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":340,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660819+0000","last_change":"2024-02-18T00:37:02.627426+0000","last_active":"2024-02-18T00:37:14.660819+0000","last_peered":"2024-02-18T00:37:14.660819+0000","last_clean":"2024-02-18T00:37:14.660819+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:37:14.660819+0000","last_undegraded":"2024-02-18T00:37:14.660819+0000","last_fullsized":"2024-02-18T00:37:14.660819+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.627349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.627349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":314,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.657099+0000","last_change":"2024-02-18T00:34:30.876447+0000","last_active":"2024-02-18T00:37:13.657099+0000","last_peered":"2024-02-18T00:37:13.657099+0000","last_clean":"2024-02-18T00:37:13.657099+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:37:13.657099+0000","last_undegraded":"2024-02-18T00:37:13.657099+0000","last_fullsized":"2024-02-18T00:37:13.657099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.876371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.876371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":268,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.827453+0000","last_change":"2024-02-18T00:37:01.919166+0000","last_active":"2024-02-18T00:37:13.827453+0000","last_peered":"2024-02-18T00:37:13.827453+0000","last_clean":"2024-02-18T00:37:13.827453+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:37:13.827453+0000","last_undegraded":"2024-02-18T00:37:13.827453+0000","last_fullsized":"2024-02-18T00:37:13.827453+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.919082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.919082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":371,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660990+0000","last_change":"2024-02-18T00:35:54.188262+0000","last_active":"2024-02-18T00:37:14.660990+0000","last_peered":"2024-02-18T00:37:14.660990+0000","last_clean":"2024-02-18T00:37:14.660990+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:37:14.660990+0000","last_undegraded":"2024-02-18T00:37:14.660990+0000","last_fullsized":"2024-02-18T00:37:14.660990+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:35:54.188188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.188188+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":328,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.828627+0000","last_change":"2024-02-18T00:36:43.837775+0000","last_active":"2024-02-18T00:37:13.828627+0000","last_peered":"2024-02-18T00:37:13.828627+0000","last_clean":"2024-02-18T00:37:13.828627+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:37:13.828627+0000","last_undegraded":"2024-02-18T00:37:13.828627+0000","last_fullsized":"2024-02-18T00:37:13.828627+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.837702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.837702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":325,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656803+0000","last_change":"2024-02-18T00:36:56.601833+0000","last_active":"2024-02-18T00:37:14.656803+0000","last_peered":"2024-02-18T00:37:14.656803+0000","last_clean":"2024-02-18T00:37:14.656803+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:37:14.656803+0000","last_undegraded":"2024-02-18T00:37:14.656803+0000","last_fullsized":"2024-02-18T00:37:14.656803+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.601727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.601727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":333,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661653+0000","last_change":"2024-02-18T00:35:42.683115+0000","last_active":"2024-02-18T00:37:14.661653+0000","last_peered":"2024-02-18T00:37:14.661653+0000","last_clean":"2024-02-18T00:37:14.661653+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:37:14.661653+0000","last_undegraded":"2024-02-18T00:37:14.661653+0000","last_fullsized":"2024-02-18T00:37:14.661653+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":270,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658539+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:37:13.658539+0000","last_peered":"2024-02-18T00:37:13.658539+0000","last_clean":"2024-02-18T00:37:13.658539+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:37:13.658539+0000","last_undegraded":"2024-02-18T00:37:13.658539+0000","last_fullsized":"2024-02-18T00:37:13.658539+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":403,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699427+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:37:13.699427+0000","last_peered":"2024-02-18T00:37:13.699427+0000","last_clean":"2024-02-18T00:37:13.699427+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:37:13.699427+0000","last_undegraded":"2024-02-18T00:37:13.699427+0000","last_fullsized":"2024-02-18T00:37:13.699427+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":329,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.663110+0000","last_change":"2024-02-18T00:36:49.680480+0000","last_active":"2024-02-18T00:37:14.663110+0000","last_peered":"2024-02-18T00:37:14.663110+0000","last_clean":"2024-02-18T00:37:14.663110+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:37:14.663110+0000","last_undegraded":"2024-02-18T00:37:14.663110+0000","last_fullsized":"2024-02-18T00:37:14.663110+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.680414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":325,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657622+0000","last_change":"2024-02-18T00:37:00.670726+0000","last_active":"2024-02-18T00:37:14.657622+0000","last_peered":"2024-02-18T00:37:14.657622+0000","last_clean":"2024-02-18T00:37:14.657622+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:37:14.657622+0000","last_undegraded":"2024-02-18T00:37:14.657622+0000","last_fullsized":"2024-02-18T00:37:14.657622+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.670646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.670646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":224,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660035+0000","last_change":"2024-02-18T00:34:34.229441+0000","last_active":"2024-02-18T00:37:14.660035+0000","last_peered":"2024-02-18T00:37:14.660035+0000","last_clean":"2024-02-18T00:37:14.660035+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:37:14.660035+0000","last_undegraded":"2024-02-18T00:37:14.660035+0000","last_fullsized":"2024-02-18T00:37:14.660035+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.229327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.229327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":260,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655655+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:37:13.655655+0000","last_peered":"2024-02-18T00:37:13.655655+0000","last_clean":"2024-02-18T00:37:13.655655+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:37:13.655655+0000","last_undegraded":"2024-02-18T00:37:13.655655+0000","last_fullsized":"2024-02-18T00:37:13.655655+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":355,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658374+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:37:14.658374+0000","last_peered":"2024-02-18T00:37:14.658374+0000","last_clean":"2024-02-18T00:37:14.658374+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:37:14.658374+0000","last_undegraded":"2024-02-18T00:37:14.658374+0000","last_fullsized":"2024-02-18T00:37:14.658374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":229,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657376+0000","last_change":"2024-02-18T00:36:58.282770+0000","last_active":"2024-02-18T00:37:14.657376+0000","last_peered":"2024-02-18T00:37:14.657376+0000","last_clean":"2024-02-18T00:37:14.657376+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:37:14.657376+0000","last_undegraded":"2024-02-18T00:37:14.657376+0000","last_fullsized":"2024-02-18T00:37:14.657376+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.282673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.282673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":324,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.827181+0000","last_change":"2024-02-18T00:36:59.000753+0000","last_active":"2024-02-18T00:37:13.827181+0000","last_peered":"2024-02-18T00:37:13.827181+0000","last_clean":"2024-02-18T00:37:13.827181+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:37:13.827181+0000","last_undegraded":"2024-02-18T00:37:13.827181+0000","last_fullsized":"2024-02-18T00:37:13.827181+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.000690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.000690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":323,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.663313+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:37:14.663313+0000","last_peered":"2024-02-18T00:37:14.663313+0000","last_clean":"2024-02-18T00:37:14.663313+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:37:14.663313+0000","last_undegraded":"2024-02-18T00:37:14.663313+0000","last_fullsized":"2024-02-18T00:37:14.663313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":108,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657945+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:37:14.657945+0000","last_peered":"2024-02-18T00:37:14.657945+0000","last_clean":"2024-02-18T00:37:14.657945+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:37:14.657945+0000","last_undegraded":"2024-02-18T00:37:14.657945+0000","last_fullsized":"2024-02-18T00:37:14.657945+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":337,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657993+0000","last_change":"2024-02-18T00:34:24.329061+0000","last_active":"2024-02-18T00:37:14.657993+0000","last_peered":"2024-02-18T00:37:14.657993+0000","last_clean":"2024-02-18T00:37:14.657993+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:37:14.657993+0000","last_undegraded":"2024-02-18T00:37:14.657993+0000","last_fullsized":"2024-02-18T00:37:14.657993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:34:24.328990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.328990+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":337,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.826370+0000","last_change":"2024-02-18T00:36:38.883486+0000","last_active":"2024-02-18T00:37:13.826370+0000","last_peered":"2024-02-18T00:37:13.826370+0000","last_clean":"2024-02-18T00:37:13.826370+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:37:13.826370+0000","last_undegraded":"2024-02-18T00:37:13.826370+0000","last_fullsized":"2024-02-18T00:37:13.826370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:38.883391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.883391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":440,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657117+0000","last_change":"2024-02-18T00:34:33.256541+0000","last_active":"2024-02-18T00:37:14.657117+0000","last_peered":"2024-02-18T00:37:14.657117+0000","last_clean":"2024-02-18T00:37:14.657117+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:37:14.657117+0000","last_undegraded":"2024-02-18T00:37:14.657117+0000","last_fullsized":"2024-02-18T00:37:14.657117+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:34:33.256472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:33.256472+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":318,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658127+0000","last_change":"2024-02-18T00:34:22.358185+0000","last_active":"2024-02-18T00:37:14.658127+0000","last_peered":"2024-02-18T00:37:14.658127+0000","last_clean":"2024-02-18T00:37:14.658127+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:37:14.658127+0000","last_undegraded":"2024-02-18T00:37:14.658127+0000","last_fullsized":"2024-02-18T00:37:14.658127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.358103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.358103+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":268,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.826702+0000","last_change":"2024-02-18T00:37:00.950276+0000","last_active":"2024-02-18T00:37:13.826702+0000","last_peered":"2024-02-18T00:37:13.826702+0000","last_clean":"2024-02-18T00:37:13.826702+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:37:13.826702+0000","last_undegraded":"2024-02-18T00:37:13.826702+0000","last_fullsized":"2024-02-18T00:37:13.826702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.950130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.950130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":326,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658261+0000","last_change":"2024-02-18T00:36:33.853442+0000","last_active":"2024-02-18T00:37:13.658261+0000","last_peered":"2024-02-18T00:37:13.658261+0000","last_clean":"2024-02-18T00:37:13.658261+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:37:13.658261+0000","last_undegraded":"2024-02-18T00:37:13.658261+0000","last_fullsized":"2024-02-18T00:37:13.658261+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:36:33.853337+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:36:33.853337+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":346,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.825500+0000","last_change":"2024-02-18T00:36:08.716901+0000","last_active":"2024-02-18T00:37:13.825500+0000","last_peered":"2024-02-18T00:37:13.825500+0000","last_clean":"2024-02-18T00:37:13.825500+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:37:13.825500+0000","last_undegraded":"2024-02-18T00:37:13.825500+0000","last_fullsized":"2024-02-18T00:37:13.825500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.716777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.716777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":424,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658038+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:37:14.658038+0000","last_peered":"2024-02-18T00:37:14.658038+0000","last_clean":"2024-02-18T00:37:14.658038+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:37:14.658038+0000","last_undegraded":"2024-02-18T00:37:14.658038+0000","last_fullsized":"2024-02-18T00:37:14.658038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":314,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.656763+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:37:13.656763+0000","last_peered":"2024-02-18T00:37:13.656763+0000","last_clean":"2024-02-18T00:37:13.656763+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:37:13.656763+0000","last_undegraded":"2024-02-18T00:37:13.656763+0000","last_fullsized":"2024-02-18T00:37:13.656763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":264,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657036+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:37:14.657036+0000","last_peered":"2024-02-18T00:37:14.657036+0000","last_clean":"2024-02-18T00:37:14.657036+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:37:14.657036+0000","last_undegraded":"2024-02-18T00:37:14.657036+0000","last_fullsized":"2024-02-18T00:37:14.657036+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":384,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657862+0000","last_change":"2024-02-18T00:36:43.283359+0000","last_active":"2024-02-18T00:37:14.657862+0000","last_peered":"2024-02-18T00:37:14.657862+0000","last_clean":"2024-02-18T00:37:14.657862+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:37:14.657862+0000","last_undegraded":"2024-02-18T00:37:14.657862+0000","last_fullsized":"2024-02-18T00:37:14.657862+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:43.283235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.283235+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":298,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656786+0000","last_change":"2024-02-18T00:36:57.321994+0000","last_active":"2024-02-18T00:37:14.656786+0000","last_peered":"2024-02-18T00:37:14.656786+0000","last_clean":"2024-02-18T00:37:14.656786+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:37:14.656786+0000","last_undegraded":"2024-02-18T00:37:14.656786+0000","last_fullsized":"2024-02-18T00:37:14.656786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.321930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.321930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":316,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658620+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:37:13.658620+0000","last_peered":"2024-02-18T00:37:13.658620+0000","last_clean":"2024-02-18T00:37:13.658620+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:37:13.658620+0000","last_undegraded":"2024-02-18T00:37:13.658620+0000","last_fullsized":"2024-02-18T00:37:13.658620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":315,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655195+0000","last_change":"2024-02-18T00:34:35.700278+0000","last_active":"2024-02-18T00:37:13.655195+0000","last_peered":"2024-02-18T00:37:13.655195+0000","last_clean":"2024-02-18T00:37:13.655195+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:37:13.655195+0000","last_undegraded":"2024-02-18T00:37:13.655195+0000","last_fullsized":"2024-02-18T00:37:13.655195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.700187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.700187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":264,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657960+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:37:14.657960+0000","last_peered":"2024-02-18T00:37:14.657960+0000","last_clean":"2024-02-18T00:37:14.657960+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:37:14.657960+0000","last_undegraded":"2024-02-18T00:37:14.657960+0000","last_fullsized":"2024-02-18T00:37:14.657960+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":427,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654654+0000","last_change":"2024-02-18T00:36:53.910404+0000","last_active":"2024-02-18T00:37:13.654654+0000","last_peered":"2024-02-18T00:37:13.654654+0000","last_clean":"2024-02-18T00:37:13.654654+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:37:13.654654+0000","last_undegraded":"2024-02-18T00:37:13.654654+0000","last_fullsized":"2024-02-18T00:37:13.654654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.910333+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":363,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656545+0000","last_change":"2024-02-18T00:36:50.262681+0000","last_active":"2024-02-18T00:37:14.656545+0000","last_peered":"2024-02-18T00:37:14.656545+0000","last_clean":"2024-02-18T00:37:14.656545+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:37:14.656545+0000","last_undegraded":"2024-02-18T00:37:14.656545+0000","last_fullsized":"2024-02-18T00:37:14.656545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:36:50.262602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.262602+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":345,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699717+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:37:13.699717+0000","last_peered":"2024-02-18T00:37:13.699717+0000","last_clean":"2024-02-18T00:37:13.699717+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:37:13.699717+0000","last_undegraded":"2024-02-18T00:37:13.699717+0000","last_fullsized":"2024-02-18T00:37:13.699717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":222,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698929+0000","last_change":"2024-02-18T00:34:26.298022+0000","last_active":"2024-02-18T00:37:13.698929+0000","last_peered":"2024-02-18T00:37:13.698929+0000","last_clean":"2024-02-18T00:37:13.698929+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:37:13.698929+0000","last_undegraded":"2024-02-18T00:37:13.698929+0000","last_fullsized":"2024-02-18T00:37:13.698929+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:26.297951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:26.297951+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":260,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655750+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:37:13.655750+0000","last_peered":"2024-02-18T00:37:13.655750+0000","last_clean":"2024-02-18T00:37:13.655750+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:37:13.655750+0000","last_undegraded":"2024-02-18T00:37:13.655750+0000","last_fullsized":"2024-02-18T00:37:13.655750+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":334,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656897+0000","last_change":"2024-02-18T00:36:45.308400+0000","last_active":"2024-02-18T00:37:14.656897+0000","last_peered":"2024-02-18T00:37:14.656897+0000","last_clean":"2024-02-18T00:37:14.656897+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:37:14.656897+0000","last_undegraded":"2024-02-18T00:37:14.656897+0000","last_fullsized":"2024-02-18T00:37:14.656897+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.308300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.308300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":422,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654647+0000","last_change":"2024-02-18T00:34:36.698972+0000","last_active":"2024-02-18T00:37:13.654647+0000","last_peered":"2024-02-18T00:37:13.654647+0000","last_clean":"2024-02-18T00:37:13.654647+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:37:13.654647+0000","last_undegraded":"2024-02-18T00:37:13.654647+0000","last_fullsized":"2024-02-18T00:37:13.654647+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:34:36.698876+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.698876+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":337,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.827530+0000","last_change":"2024-02-18T00:36:09.647995+0000","last_active":"2024-02-18T00:37:13.827530+0000","last_peered":"2024-02-18T00:37:13.827530+0000","last_clean":"2024-02-18T00:37:13.827530+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:37:13.827530+0000","last_undegraded":"2024-02-18T00:37:13.827530+0000","last_fullsized":"2024-02-18T00:37:13.827530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.647928+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":318,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656850+0000","last_change":"2024-02-18T00:34:34.207362+0000","last_active":"2024-02-18T00:37:14.656850+0000","last_peered":"2024-02-18T00:37:14.656850+0000","last_clean":"2024-02-18T00:37:14.656850+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:37:14.656850+0000","last_undegraded":"2024-02-18T00:37:14.656850+0000","last_fullsized":"2024-02-18T00:37:14.656850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:34.207297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.207297+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":261,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.659474+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:37:13.659474+0000","last_peered":"2024-02-18T00:37:13.659474+0000","last_clean":"2024-02-18T00:37:13.659474+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:37:13.659474+0000","last_undegraded":"2024-02-18T00:37:13.659474+0000","last_fullsized":"2024-02-18T00:37:13.659474+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":318,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698591+0000","last_change":"2024-02-18T00:34:24.262632+0000","last_active":"2024-02-18T00:37:13.698591+0000","last_peered":"2024-02-18T00:37:13.698591+0000","last_clean":"2024-02-18T00:37:13.698591+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:37:13.698591+0000","last_undegraded":"2024-02-18T00:37:13.698591+0000","last_fullsized":"2024-02-18T00:37:13.698591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:24.262540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:24.262540+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":399,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660372+0000","last_change":"2024-02-18T00:34:36.038145+0000","last_active":"2024-02-18T00:37:14.660372+0000","last_peered":"2024-02-18T00:37:14.660372+0000","last_clean":"2024-02-18T00:37:14.660372+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:37:14.660372+0000","last_undegraded":"2024-02-18T00:37:14.660372+0000","last_fullsized":"2024-02-18T00:37:14.660372+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:34:36.037987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:36.037987+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":235,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660060+0000","last_change":"2024-02-18T00:37:01.250860+0000","last_active":"2024-02-18T00:37:14.660060+0000","last_peered":"2024-02-18T00:37:14.660060+0000","last_clean":"2024-02-18T00:37:14.660060+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:37:14.660060+0000","last_undegraded":"2024-02-18T00:37:14.660060+0000","last_fullsized":"2024-02-18T00:37:14.660060+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:37:01.250771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.250771+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":322,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.825587+0000","last_change":"2024-02-18T00:36:59.979666+0000","last_active":"2024-02-18T00:37:13.825587+0000","last_peered":"2024-02-18T00:37:13.825587+0000","last_clean":"2024-02-18T00:37:13.825587+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:37:13.825587+0000","last_undegraded":"2024-02-18T00:37:13.825587+0000","last_fullsized":"2024-02-18T00:37:13.825587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.979565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.979565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":261,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655934+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:37:13.655934+0000","last_peered":"2024-02-18T00:37:13.655934+0000","last_clean":"2024-02-18T00:37:13.655934+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:37:13.655934+0000","last_undegraded":"2024-02-18T00:37:13.655934+0000","last_fullsized":"2024-02-18T00:37:13.655934+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":316,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658795+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:37:13.658795+0000","last_peered":"2024-02-18T00:37:13.658795+0000","last_clean":"2024-02-18T00:37:13.658795+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:37:13.658795+0000","last_undegraded":"2024-02-18T00:37:13.658795+0000","last_fullsized":"2024-02-18T00:37:13.658795+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":337,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661244+0000","last_change":"2024-02-18T00:34:39.020123+0000","last_active":"2024-02-18T00:37:14.661244+0000","last_peered":"2024-02-18T00:37:14.661244+0000","last_clean":"2024-02-18T00:37:14.661244+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:37:14.661244+0000","last_undegraded":"2024-02-18T00:37:14.661244+0000","last_fullsized":"2024-02-18T00:37:14.661244+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:34:39.020052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.020052+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":299,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.825197+0000","last_change":"2024-02-18T00:36:34.800419+0000","last_active":"2024-02-18T00:37:13.825197+0000","last_peered":"2024-02-18T00:37:13.825197+0000","last_clean":"2024-02-18T00:37:13.825197+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:37:13.825197+0000","last_undegraded":"2024-02-18T00:37:13.825197+0000","last_fullsized":"2024-02-18T00:37:13.825197+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:34.800327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.800327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":320,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661329+0000","last_change":"2024-02-18T00:34:22.917665+0000","last_active":"2024-02-18T00:37:14.661329+0000","last_peered":"2024-02-18T00:37:14.661329+0000","last_clean":"2024-02-18T00:37:14.661329+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:37:14.661329+0000","last_undegraded":"2024-02-18T00:37:14.661329+0000","last_fullsized":"2024-02-18T00:37:14.661329+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:22.917597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:34:22.917597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":317,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.827282+0000","last_change":"2024-02-18T00:35:21.475025+0000","last_active":"2024-02-18T00:37:13.827282+0000","last_peered":"2024-02-18T00:37:13.827282+0000","last_clean":"2024-02-18T00:37:13.827282+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:37:13.827282+0000","last_undegraded":"2024-02-18T00:37:13.827282+0000","last_fullsized":"2024-02-18T00:37:13.827282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:21.474956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:21.474956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":271,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656958+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:37:14.656958+0000","last_peered":"2024-02-18T00:37:14.656958+0000","last_clean":"2024-02-18T00:37:14.656958+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:37:14.656958+0000","last_undegraded":"2024-02-18T00:37:14.656958+0000","last_fullsized":"2024-02-18T00:37:14.656958+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":224,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.662923+0000","last_change":"2024-02-18T00:34:35.259853+0000","last_active":"2024-02-18T00:37:14.662923+0000","last_peered":"2024-02-18T00:37:14.662923+0000","last_clean":"2024-02-18T00:37:14.662923+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:37:14.662923+0000","last_undegraded":"2024-02-18T00:37:14.662923+0000","last_fullsized":"2024-02-18T00:37:14.662923+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.259766+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":377,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699047+0000","last_change":"2024-02-18T00:36:50.759133+0000","last_active":"2024-02-18T00:37:13.699047+0000","last_peered":"2024-02-18T00:37:13.699047+0000","last_clean":"2024-02-18T00:37:13.699047+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:37:13.699047+0000","last_undegraded":"2024-02-18T00:37:13.699047+0000","last_fullsized":"2024-02-18T00:37:13.699047+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:36:50.759019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.759019+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":256,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656872+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:37:14.656872+0000","last_peered":"2024-02-18T00:37:14.656872+0000","last_clean":"2024-02-18T00:37:14.656872+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:37:14.656872+0000","last_undegraded":"2024-02-18T00:37:14.656872+0000","last_fullsized":"2024-02-18T00:37:14.656872+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":343,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.657695+0000","last_change":"2024-02-18T00:37:00.820469+0000","last_active":"2024-02-18T00:37:13.657695+0000","last_peered":"2024-02-18T00:37:13.657695+0000","last_clean":"2024-02-18T00:37:13.657695+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:37:13.657695+0000","last_undegraded":"2024-02-18T00:37:13.657695+0000","last_fullsized":"2024-02-18T00:37:13.657695+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.820283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.820283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":320,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654964+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:37:13.654964+0000","last_peered":"2024-02-18T00:37:13.654964+0000","last_clean":"2024-02-18T00:37:13.654964+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:37:13.654964+0000","last_undegraded":"2024-02-18T00:37:13.654964+0000","last_fullsized":"2024-02-18T00:37:13.654964+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":319,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658893+0000","last_change":"2024-02-18T00:36:42.828611+0000","last_active":"2024-02-18T00:37:13.658893+0000","last_peered":"2024-02-18T00:37:13.658893+0000","last_clean":"2024-02-18T00:37:13.658893+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:37:13.658893+0000","last_undegraded":"2024-02-18T00:37:13.658893+0000","last_fullsized":"2024-02-18T00:37:13.658893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.828542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.828542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":262,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.662591+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:37:14.662591+0000","last_peered":"2024-02-18T00:37:14.662591+0000","last_clean":"2024-02-18T00:37:14.662591+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:37:14.662591+0000","last_undegraded":"2024-02-18T00:37:14.662591+0000","last_fullsized":"2024-02-18T00:37:14.662591+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":334,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661016+0000","last_change":"2024-02-18T00:36:57.633211+0000","last_active":"2024-02-18T00:37:14.661016+0000","last_peered":"2024-02-18T00:37:14.661016+0000","last_clean":"2024-02-18T00:37:14.661016+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:37:14.661016+0000","last_undegraded":"2024-02-18T00:37:14.661016+0000","last_fullsized":"2024-02-18T00:37:14.661016+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.633128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.633128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":368,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.828727+0000","last_change":"2024-02-18T00:36:45.862082+0000","last_active":"2024-02-18T00:37:13.828727+0000","last_peered":"2024-02-18T00:37:13.828727+0000","last_clean":"2024-02-18T00:37:13.828727+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:37:13.828727+0000","last_undegraded":"2024-02-18T00:37:13.828727+0000","last_fullsized":"2024-02-18T00:37:13.828727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:36:45.861971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.861971+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":338,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658429+0000","last_change":"2024-02-18T00:36:36.500976+0000","last_active":"2024-02-18T00:37:14.658429+0000","last_peered":"2024-02-18T00:37:14.658429+0000","last_clean":"2024-02-18T00:37:14.658429+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:37:14.658429+0000","last_undegraded":"2024-02-18T00:37:14.658429+0000","last_fullsized":"2024-02-18T00:37:14.658429+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:36.500906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.500906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":802,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.669192+0000","last_change":"2024-02-18T00:36:06.641279+0000","last_active":"2024-02-18T00:37:14.669192+0000","last_peered":"2024-02-18T00:37:14.669192+0000","last_clean":"2024-02-18T00:37:14.669192+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:37:14.669192+0000","last_undegraded":"2024-02-18T00:37:14.669192+0000","last_fullsized":"2024-02-18T00:37:14.669192+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:36:06.641185+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.641185+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":471,"num_read_kb":466,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":264,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699386+0000","last_change":"2024-02-18T00:36:42.643527+0000","last_active":"2024-02-18T00:37:13.699386+0000","last_peered":"2024-02-18T00:37:13.699386+0000","last_clean":"2024-02-18T00:37:13.699386+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:37:13.699386+0000","last_undegraded":"2024-02-18T00:37:13.699386+0000","last_fullsized":"2024-02-18T00:37:13.699386+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:36:42.643433+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.643433+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":273,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656917+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:37:14.656917+0000","last_peered":"2024-02-18T00:37:14.656917+0000","last_clean":"2024-02-18T00:37:14.656917+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:37:14.656917+0000","last_undegraded":"2024-02-18T00:37:14.656917+0000","last_fullsized":"2024-02-18T00:37:14.656917+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":322,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660197+0000","last_change":"2024-02-18T00:36:56.302683+0000","last_active":"2024-02-18T00:37:14.660197+0000","last_peered":"2024-02-18T00:37:14.660197+0000","last_clean":"2024-02-18T00:37:14.660197+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:37:14.660197+0000","last_undegraded":"2024-02-18T00:37:14.660197+0000","last_fullsized":"2024-02-18T00:37:14.660197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.302615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.302615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":147,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658031+0000","last_change":"2024-02-18T00:36:41.801219+0000","last_active":"2024-02-18T00:37:13.658031+0000","last_peered":"2024-02-18T00:37:13.658031+0000","last_clean":"2024-02-18T00:37:13.658031+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:37:13.658031+0000","last_undegraded":"2024-02-18T00:37:13.658031+0000","last_fullsized":"2024-02-18T00:37:13.658031+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-18T00:36:41.801144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.801144+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":334,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.657283+0000","last_change":"2024-02-18T00:36:57.791451+0000","last_active":"2024-02-18T00:37:13.657283+0000","last_peered":"2024-02-18T00:37:13.657283+0000","last_clean":"2024-02-18T00:37:13.657283+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:37:13.657283+0000","last_undegraded":"2024-02-18T00:37:13.657283+0000","last_fullsized":"2024-02-18T00:37:13.657283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.791347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.791347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":324,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.659030+0000","last_change":"2024-02-18T00:36:49.976027+0000","last_active":"2024-02-18T00:37:13.659030+0000","last_peered":"2024-02-18T00:37:13.659030+0000","last_clean":"2024-02-18T00:37:13.659030+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:37:13.659030+0000","last_undegraded":"2024-02-18T00:37:13.659030+0000","last_fullsized":"2024-02-18T00:37:13.659030+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.975957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.975957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":262,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658394+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:37:14.658394+0000","last_peered":"2024-02-18T00:37:14.658394+0000","last_clean":"2024-02-18T00:37:14.658394+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:37:14.658394+0000","last_undegraded":"2024-02-18T00:37:14.658394+0000","last_fullsized":"2024-02-18T00:37:14.658394+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":421,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661087+0000","last_change":"2024-02-18T00:34:18.980677+0000","last_active":"2024-02-18T00:37:14.661087+0000","last_peered":"2024-02-18T00:37:14.661087+0000","last_clean":"2024-02-18T00:37:14.661087+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:37:14.661087+0000","last_undegraded":"2024-02-18T00:37:14.661087+0000","last_fullsized":"2024-02-18T00:37:14.661087+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:18.980613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:18.980613+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":322,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658790+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:37:14.658790+0000","last_peered":"2024-02-18T00:37:14.658790+0000","last_clean":"2024-02-18T00:37:14.658790+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:37:14.658790+0000","last_undegraded":"2024-02-18T00:37:14.658790+0000","last_fullsized":"2024-02-18T00:37:14.658790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":351,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.827017+0000","last_change":"2024-02-18T00:36:53.925296+0000","last_active":"2024-02-18T00:37:13.827017+0000","last_peered":"2024-02-18T00:37:13.827017+0000","last_clean":"2024-02-18T00:37:13.827017+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:37:13.827017+0000","last_undegraded":"2024-02-18T00:37:13.827017+0000","last_fullsized":"2024-02-18T00:37:13.827017+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.925187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.925187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":315,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656635+0000","last_change":"2024-02-18T00:36:46.369602+0000","last_active":"2024-02-18T00:37:14.656635+0000","last_peered":"2024-02-18T00:37:14.656635+0000","last_clean":"2024-02-18T00:37:14.656635+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:37:14.656635+0000","last_undegraded":"2024-02-18T00:37:14.656635+0000","last_fullsized":"2024-02-18T00:37:14.656635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:46.369503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.369503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":264,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657826+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:37:14.657826+0000","last_peered":"2024-02-18T00:37:14.657826+0000","last_clean":"2024-02-18T00:37:14.657826+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:37:14.657826+0000","last_undegraded":"2024-02-18T00:37:14.657826+0000","last_fullsized":"2024-02-18T00:37:14.657826+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":320,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.659896+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:37:14.659896+0000","last_peered":"2024-02-18T00:37:14.659896+0000","last_clean":"2024-02-18T00:37:14.659896+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:37:14.659896+0000","last_undegraded":"2024-02-18T00:37:14.659896+0000","last_fullsized":"2024-02-18T00:37:14.659896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":375,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656658+0000","last_change":"2024-02-18T00:34:32.170285+0000","last_active":"2024-02-18T00:37:14.656658+0000","last_peered":"2024-02-18T00:37:14.656658+0000","last_clean":"2024-02-18T00:37:14.656658+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:37:14.656658+0000","last_undegraded":"2024-02-18T00:37:14.656658+0000","last_fullsized":"2024-02-18T00:37:14.656658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:34:32.170194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:32.170194+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":357,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658509+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:37:14.658509+0000","last_peered":"2024-02-18T00:37:14.658509+0000","last_clean":"2024-02-18T00:37:14.658509+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:37:14.658509+0000","last_undegraded":"2024-02-18T00:37:14.658509+0000","last_fullsized":"2024-02-18T00:37:14.658509+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":329,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655792+0000","last_change":"2024-02-18T00:37:00.804782+0000","last_active":"2024-02-18T00:37:13.655792+0000","last_peered":"2024-02-18T00:37:13.655792+0000","last_clean":"2024-02-18T00:37:13.655792+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:37:13.655792+0000","last_undegraded":"2024-02-18T00:37:13.655792+0000","last_fullsized":"2024-02-18T00:37:13.655792+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.804656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":264,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657392+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:37:14.657392+0000","last_peered":"2024-02-18T00:37:14.657392+0000","last_clean":"2024-02-18T00:37:14.657392+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:37:14.657392+0000","last_undegraded":"2024-02-18T00:37:14.657392+0000","last_fullsized":"2024-02-18T00:37:14.657392+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":388,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661368+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:37:14.661368+0000","last_peered":"2024-02-18T00:37:14.661368+0000","last_clean":"2024-02-18T00:37:14.661368+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:37:14.661368+0000","last_undegraded":"2024-02-18T00:37:14.661368+0000","last_fullsized":"2024-02-18T00:37:14.661368+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":324,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.659822+0000","last_change":"2024-02-18T00:34:20.999562+0000","last_active":"2024-02-18T00:37:14.659822+0000","last_peered":"2024-02-18T00:37:14.659822+0000","last_clean":"2024-02-18T00:37:14.659822+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:37:14.659822+0000","last_undegraded":"2024-02-18T00:37:14.659822+0000","last_fullsized":"2024-02-18T00:37:14.659822+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:20.999471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:34:20.999471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":331,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658411+0000","last_change":"2024-02-18T00:36:55.861115+0000","last_active":"2024-02-18T00:37:13.658411+0000","last_peered":"2024-02-18T00:37:13.658411+0000","last_clean":"2024-02-18T00:37:13.658411+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:37:13.658411+0000","last_undegraded":"2024-02-18T00:37:13.658411+0000","last_fullsized":"2024-02-18T00:37:13.658411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:55.861044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.861044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":327,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.662838+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:37:14.662838+0000","last_peered":"2024-02-18T00:37:14.662838+0000","last_clean":"2024-02-18T00:37:14.662838+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:37:14.662838+0000","last_undegraded":"2024-02-18T00:37:14.662838+0000","last_fullsized":"2024-02-18T00:37:14.662838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":262,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.662453+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:37:14.662453+0000","last_peered":"2024-02-18T00:37:14.662453+0000","last_clean":"2024-02-18T00:37:14.662453+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:37:14.662453+0000","last_undegraded":"2024-02-18T00:37:14.662453+0000","last_fullsized":"2024-02-18T00:37:14.662453+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":386,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658279+0000","last_change":"2024-02-18T00:36:51.948976+0000","last_active":"2024-02-18T00:37:13.658279+0000","last_peered":"2024-02-18T00:37:13.658279+0000","last_clean":"2024-02-18T00:37:13.658279+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:37:13.658279+0000","last_undegraded":"2024-02-18T00:37:13.658279+0000","last_fullsized":"2024-02-18T00:37:13.658279+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:36:51.948907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.948907+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":327,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698284+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:37:13.698284+0000","last_peered":"2024-02-18T00:37:13.698284+0000","last_clean":"2024-02-18T00:37:13.698284+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:37:13.698284+0000","last_undegraded":"2024-02-18T00:37:13.698284+0000","last_fullsized":"2024-02-18T00:37:13.698284+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":316,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.657451+0000","last_change":"2024-02-18T00:35:17.815896+0000","last_active":"2024-02-18T00:37:13.657451+0000","last_peered":"2024-02-18T00:37:13.657451+0000","last_clean":"2024-02-18T00:37:13.657451+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:37:13.657451+0000","last_undegraded":"2024-02-18T00:37:13.657451+0000","last_fullsized":"2024-02-18T00:37:13.657451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.815824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.815824+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":227,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699313+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:37:13.699313+0000","last_peered":"2024-02-18T00:37:13.699313+0000","last_clean":"2024-02-18T00:37:13.699313+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:37:13.699313+0000","last_undegraded":"2024-02-18T00:37:13.699313+0000","last_fullsized":"2024-02-18T00:37:13.699313+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":273,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658619+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:37:14.658619+0000","last_peered":"2024-02-18T00:37:14.658619+0000","last_clean":"2024-02-18T00:37:14.658619+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:37:14.658619+0000","last_undegraded":"2024-02-18T00:37:14.658619+0000","last_fullsized":"2024-02-18T00:37:14.658619+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":394,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655355+0000","last_change":"2024-02-18T00:36:58.819567+0000","last_active":"2024-02-18T00:37:13.655355+0000","last_peered":"2024-02-18T00:37:13.655355+0000","last_clean":"2024-02-18T00:37:13.655355+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:37:13.655355+0000","last_undegraded":"2024-02-18T00:37:13.655355+0000","last_fullsized":"2024-02-18T00:37:13.655355+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:36:58.819503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.819503+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":338,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.657194+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:37:13.657194+0000","last_peered":"2024-02-18T00:37:13.657194+0000","last_clean":"2024-02-18T00:37:13.657194+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:37:13.657194+0000","last_undegraded":"2024-02-18T00:37:13.657194+0000","last_fullsized":"2024-02-18T00:37:13.657194+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":420,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.656583+0000","last_change":"2024-02-18T00:34:34.779619+0000","last_active":"2024-02-18T00:37:13.656583+0000","last_peered":"2024-02-18T00:37:13.656583+0000","last_clean":"2024-02-18T00:37:13.656583+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:37:13.656583+0000","last_undegraded":"2024-02-18T00:37:13.656583+0000","last_fullsized":"2024-02-18T00:37:13.656583+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:34:34.779531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:34.779531+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":329,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661123+0000","last_change":"2024-02-18T00:36:44.367116+0000","last_active":"2024-02-18T00:37:14.661123+0000","last_peered":"2024-02-18T00:37:14.661123+0000","last_clean":"2024-02-18T00:37:14.661123+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:37:14.661123+0000","last_undegraded":"2024-02-18T00:37:14.661123+0000","last_fullsized":"2024-02-18T00:37:14.661123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.367032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.367032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":271,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698240+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:37:13.698240+0000","last_peered":"2024-02-18T00:37:13.698240+0000","last_clean":"2024-02-18T00:37:13.698240+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:37:13.698240+0000","last_undegraded":"2024-02-18T00:37:13.698240+0000","last_fullsized":"2024-02-18T00:37:13.698240+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":370,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655842+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:37:13.655842+0000","last_peered":"2024-02-18T00:37:13.655842+0000","last_clean":"2024-02-18T00:37:13.655842+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:37:13.655842+0000","last_undegraded":"2024-02-18T00:37:13.655842+0000","last_fullsized":"2024-02-18T00:37:13.655842+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":326,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660467+0000","last_change":"2024-02-18T00:37:03.239555+0000","last_active":"2024-02-18T00:37:14.660467+0000","last_peered":"2024-02-18T00:37:14.660467+0000","last_clean":"2024-02-18T00:37:14.660467+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:37:14.660467+0000","last_undegraded":"2024-02-18T00:37:14.660467+0000","last_fullsized":"2024-02-18T00:37:14.660467+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.239451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.239451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"341'10","reported_seq":440,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655472+0000","last_change":"2024-02-18T00:36:59.830567+0000","last_active":"2024-02-18T00:37:13.655472+0000","last_peered":"2024-02-18T00:37:13.655472+0000","last_clean":"2024-02-18T00:37:13.655472+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:37:13.655472+0000","last_undegraded":"2024-02-18T00:37:13.655472+0000","last_fullsized":"2024-02-18T00:37:13.655472+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'10","last_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_deep_scrub":"341'10","last_deep_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.830497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":318,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658646+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:37:14.658646+0000","last_peered":"2024-02-18T00:37:14.658646+0000","last_clean":"2024-02-18T00:37:14.658646+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:37:14.658646+0000","last_undegraded":"2024-02-18T00:37:14.658646+0000","last_fullsized":"2024-02-18T00:37:14.658646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":260,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658981+0000","last_change":"2024-02-18T00:35:28.834120+0000","last_active":"2024-02-18T00:37:13.658981+0000","last_peered":"2024-02-18T00:37:13.658981+0000","last_clean":"2024-02-18T00:37:13.658981+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:37:13.658981+0000","last_undegraded":"2024-02-18T00:37:13.658981+0000","last_fullsized":"2024-02-18T00:37:13.658981+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:28.834040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:28.834040+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":342,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.826460+0000","last_change":"2024-02-18T00:36:55.970729+0000","last_active":"2024-02-18T00:37:13.826460+0000","last_peered":"2024-02-18T00:37:13.826460+0000","last_clean":"2024-02-18T00:37:13.826460+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:37:13.826460+0000","last_undegraded":"2024-02-18T00:37:13.826460+0000","last_fullsized":"2024-02-18T00:37:13.826460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:36:55.970628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.970628+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":332,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699404+0000","last_change":"2024-02-18T00:36:44.617522+0000","last_active":"2024-02-18T00:37:13.699404+0000","last_peered":"2024-02-18T00:37:13.699404+0000","last_clean":"2024-02-18T00:37:13.699404+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:37:13.699404+0000","last_undegraded":"2024-02-18T00:37:13.699404+0000","last_fullsized":"2024-02-18T00:37:13.699404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.617443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.617443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":322,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655726+0000","last_change":"2024-02-18T00:36:50.970267+0000","last_active":"2024-02-18T00:37:13.655726+0000","last_peered":"2024-02-18T00:37:13.655726+0000","last_clean":"2024-02-18T00:37:13.655726+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:37:13.655726+0000","last_undegraded":"2024-02-18T00:37:13.655726+0000","last_fullsized":"2024-02-18T00:37:13.655726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:50.970192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.970192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":315,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656537+0000","last_change":"2024-02-18T00:36:11.346267+0000","last_active":"2024-02-18T00:37:14.656537+0000","last_peered":"2024-02-18T00:37:14.656537+0000","last_clean":"2024-02-18T00:37:14.656537+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:37:14.656537+0000","last_undegraded":"2024-02-18T00:37:14.656537+0000","last_fullsized":"2024-02-18T00:37:14.656537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.346197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.346197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":262,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661027+0000","last_change":"2024-02-18T00:36:42.355194+0000","last_active":"2024-02-18T00:37:14.661027+0000","last_peered":"2024-02-18T00:37:14.661027+0000","last_clean":"2024-02-18T00:37:14.661027+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:37:14.661027+0000","last_undegraded":"2024-02-18T00:37:14.661027+0000","last_fullsized":"2024-02-18T00:37:14.661027+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.355114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.355114+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":346,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657500+0000","last_change":"2024-02-18T00:36:52.343624+0000","last_active":"2024-02-18T00:37:14.657500+0000","last_peered":"2024-02-18T00:37:14.657500+0000","last_clean":"2024-02-18T00:37:14.657500+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:37:14.657500+0000","last_undegraded":"2024-02-18T00:37:14.657500+0000","last_fullsized":"2024-02-18T00:37:14.657500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:36:52.343556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.343556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":323,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.827493+0000","last_change":"2024-02-18T00:35:22.456015+0000","last_active":"2024-02-18T00:37:13.827493+0000","last_peered":"2024-02-18T00:37:13.827493+0000","last_clean":"2024-02-18T00:37:13.827493+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:37:13.827493+0000","last_undegraded":"2024-02-18T00:37:13.827493+0000","last_fullsized":"2024-02-18T00:37:13.827493+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.455942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.455942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":324,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.826550+0000","last_change":"2024-02-18T00:36:42.857622+0000","last_active":"2024-02-18T00:37:13.826550+0000","last_peered":"2024-02-18T00:37:13.826550+0000","last_clean":"2024-02-18T00:37:13.826550+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:37:13.826550+0000","last_undegraded":"2024-02-18T00:37:13.826550+0000","last_fullsized":"2024-02-18T00:37:13.826550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.857557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.857557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":315,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654625+0000","last_change":"2024-02-18T00:35:10.600567+0000","last_active":"2024-02-18T00:37:13.654625+0000","last_peered":"2024-02-18T00:37:13.654625+0000","last_clean":"2024-02-18T00:37:13.654625+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:37:13.654625+0000","last_undegraded":"2024-02-18T00:37:13.654625+0000","last_fullsized":"2024-02-18T00:37:13.654625+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:10.600497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:10.600497+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":262,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699169+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:37:13.699169+0000","last_peered":"2024-02-18T00:37:13.699169+0000","last_clean":"2024-02-18T00:37:13.699169+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:37:13.699169+0000","last_undegraded":"2024-02-18T00:37:13.699169+0000","last_fullsized":"2024-02-18T00:37:13.699169+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":325,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654524+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:37:13.654524+0000","last_peered":"2024-02-18T00:37:13.654524+0000","last_clean":"2024-02-18T00:37:13.654524+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:37:13.654524+0000","last_undegraded":"2024-02-18T00:37:13.654524+0000","last_fullsized":"2024-02-18T00:37:13.654524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":325,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.657477+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:37:13.657477+0000","last_peered":"2024-02-18T00:37:13.657477+0000","last_clean":"2024-02-18T00:37:13.657477+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:37:13.657477+0000","last_undegraded":"2024-02-18T00:37:13.657477+0000","last_fullsized":"2024-02-18T00:37:13.657477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":332,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698273+0000","last_change":"2024-02-18T00:36:12.518914+0000","last_active":"2024-02-18T00:37:13.698273+0000","last_peered":"2024-02-18T00:37:13.698273+0000","last_clean":"2024-02-18T00:37:13.698273+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:37:13.698273+0000","last_undegraded":"2024-02-18T00:37:13.698273+0000","last_fullsized":"2024-02-18T00:37:13.698273+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:12.518838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:12.518838+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":224,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658955+0000","last_change":"2024-02-18T00:34:28.115212+0000","last_active":"2024-02-18T00:37:14.658955+0000","last_peered":"2024-02-18T00:37:14.658955+0000","last_clean":"2024-02-18T00:37:14.658955+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:37:14.658955+0000","last_undegraded":"2024-02-18T00:37:14.658955+0000","last_fullsized":"2024-02-18T00:37:14.658955+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:28.115143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:28.115143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":269,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.659087+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:37:13.659087+0000","last_peered":"2024-02-18T00:37:13.659087+0000","last_clean":"2024-02-18T00:37:13.659087+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:37:13.659087+0000","last_undegraded":"2024-02-18T00:37:13.659087+0000","last_fullsized":"2024-02-18T00:37:13.659087+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":376,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655110+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:37:13.655110+0000","last_peered":"2024-02-18T00:37:13.655110+0000","last_clean":"2024-02-18T00:37:13.655110+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:37:13.655110+0000","last_undegraded":"2024-02-18T00:37:13.655110+0000","last_fullsized":"2024-02-18T00:37:13.655110+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":327,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699666+0000","last_change":"2024-02-18T00:36:11.551695+0000","last_active":"2024-02-18T00:37:13.699666+0000","last_peered":"2024-02-18T00:37:13.699666+0000","last_clean":"2024-02-18T00:37:13.699666+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:37:13.699666+0000","last_undegraded":"2024-02-18T00:37:13.699666+0000","last_fullsized":"2024-02-18T00:37:13.699666+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.551602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":238,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656108+0000","last_change":"2024-02-18T00:36:59.243785+0000","last_active":"2024-02-18T00:37:14.656108+0000","last_peered":"2024-02-18T00:37:14.656108+0000","last_clean":"2024-02-18T00:37:14.656108+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:37:14.656108+0000","last_undegraded":"2024-02-18T00:37:14.656108+0000","last_fullsized":"2024-02-18T00:37:14.656108+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.243699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.243699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":315,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655839+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:37:13.655839+0000","last_peered":"2024-02-18T00:37:13.655839+0000","last_clean":"2024-02-18T00:37:13.655839+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:37:13.655839+0000","last_undegraded":"2024-02-18T00:37:13.655839+0000","last_fullsized":"2024-02-18T00:37:13.655839+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":224,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657000+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:37:14.657000+0000","last_peered":"2024-02-18T00:37:14.657000+0000","last_clean":"2024-02-18T00:37:14.657000+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:37:14.657000+0000","last_undegraded":"2024-02-18T00:37:14.657000+0000","last_fullsized":"2024-02-18T00:37:14.657000+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":383,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661665+0000","last_change":"2024-02-18T00:36:53.401897+0000","last_active":"2024-02-18T00:37:14.661665+0000","last_peered":"2024-02-18T00:37:14.661665+0000","last_clean":"2024-02-18T00:37:14.661665+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:37:14.661665+0000","last_undegraded":"2024-02-18T00:37:14.661665+0000","last_fullsized":"2024-02-18T00:37:14.661665+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:36:53.401819+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.401819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":329,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655913+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:37:13.655913+0000","last_peered":"2024-02-18T00:37:13.655913+0000","last_clean":"2024-02-18T00:37:13.655913+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:37:13.655913+0000","last_undegraded":"2024-02-18T00:37:13.655913+0000","last_fullsized":"2024-02-18T00:37:13.655913+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":448,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699794+0000","last_change":"2024-02-18T00:36:38.557419+0000","last_active":"2024-02-18T00:37:13.699794+0000","last_peered":"2024-02-18T00:37:13.699794+0000","last_clean":"2024-02-18T00:37:13.699794+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:37:13.699794+0000","last_undegraded":"2024-02-18T00:37:13.699794+0000","last_fullsized":"2024-02-18T00:37:13.699794+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:36:38.557298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.557298+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":238,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.659029+0000","last_change":"2024-02-18T00:36:49.727241+0000","last_active":"2024-02-18T00:37:13.659029+0000","last_peered":"2024-02-18T00:37:13.659029+0000","last_clean":"2024-02-18T00:37:13.659029+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:37:13.659029+0000","last_undegraded":"2024-02-18T00:37:13.659029+0000","last_fullsized":"2024-02-18T00:37:13.659029+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:36:49.727157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.727157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":221,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654759+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:37:13.654759+0000","last_peered":"2024-02-18T00:37:13.654759+0000","last_clean":"2024-02-18T00:37:13.654759+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:37:13.654759+0000","last_undegraded":"2024-02-18T00:37:13.654759+0000","last_fullsized":"2024-02-18T00:37:13.654759+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":359,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699904+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:37:13.699904+0000","last_peered":"2024-02-18T00:37:13.699904+0000","last_clean":"2024-02-18T00:37:13.699904+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:37:13.699904+0000","last_undegraded":"2024-02-18T00:37:13.699904+0000","last_fullsized":"2024-02-18T00:37:13.699904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":326,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655240+0000","last_change":"2024-02-18T00:36:56.850409+0000","last_active":"2024-02-18T00:37:13.655240+0000","last_peered":"2024-02-18T00:37:13.655240+0000","last_clean":"2024-02-18T00:37:13.655240+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:37:13.655240+0000","last_undegraded":"2024-02-18T00:37:13.655240+0000","last_fullsized":"2024-02-18T00:37:13.655240+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.850326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.850326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":326,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.661766+0000","last_change":"2024-02-18T00:36:52.372043+0000","last_active":"2024-02-18T00:37:14.661766+0000","last_peered":"2024-02-18T00:37:14.661766+0000","last_clean":"2024-02-18T00:37:14.661766+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:37:14.661766+0000","last_undegraded":"2024-02-18T00:37:14.661766+0000","last_fullsized":"2024-02-18T00:37:14.661766+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.371970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.371970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":322,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.827348+0000","last_change":"2024-02-18T00:35:54.569108+0000","last_active":"2024-02-18T00:37:13.827348+0000","last_peered":"2024-02-18T00:37:13.827348+0000","last_clean":"2024-02-18T00:37:13.827348+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:37:13.827348+0000","last_undegraded":"2024-02-18T00:37:13.827348+0000","last_fullsized":"2024-02-18T00:37:13.827348+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.569033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.569033+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":277,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658554+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:37:14.658554+0000","last_peered":"2024-02-18T00:37:14.658554+0000","last_clean":"2024-02-18T00:37:14.658554+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:37:14.658554+0000","last_undegraded":"2024-02-18T00:37:14.658554+0000","last_fullsized":"2024-02-18T00:37:14.658554+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":352,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.663203+0000","last_change":"2024-02-18T00:34:30.154271+0000","last_active":"2024-02-18T00:37:14.663203+0000","last_peered":"2024-02-18T00:37:14.663203+0000","last_clean":"2024-02-18T00:37:14.663203+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:37:14.663203+0000","last_undegraded":"2024-02-18T00:37:14.663203+0000","last_fullsized":"2024-02-18T00:37:14.663203+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:34:30.154200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.154200+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":336,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698499+0000","last_change":"2024-02-18T00:35:12.445167+0000","last_active":"2024-02-18T00:37:13.698499+0000","last_peered":"2024-02-18T00:37:13.698499+0000","last_clean":"2024-02-18T00:37:13.698499+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:37:13.698499+0000","last_undegraded":"2024-02-18T00:37:13.698499+0000","last_fullsized":"2024-02-18T00:37:13.698499+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.445068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.445068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":324,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.828559+0000","last_change":"2024-02-18T00:36:35.808459+0000","last_active":"2024-02-18T00:37:13.828559+0000","last_peered":"2024-02-18T00:37:13.828559+0000","last_clean":"2024-02-18T00:37:13.828559+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:37:13.828559+0000","last_undegraded":"2024-02-18T00:37:13.828559+0000","last_fullsized":"2024-02-18T00:37:13.828559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.808381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.808381+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":318,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657912+0000","last_change":"2024-02-18T00:34:30.225694+0000","last_active":"2024-02-18T00:37:14.657912+0000","last_peered":"2024-02-18T00:37:14.657912+0000","last_clean":"2024-02-18T00:37:14.657912+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:37:14.657912+0000","last_undegraded":"2024-02-18T00:37:14.657912+0000","last_fullsized":"2024-02-18T00:37:14.657912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:34:30.225625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":262,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.698932+0000","last_change":"2024-02-18T00:35:17.500109+0000","last_active":"2024-02-18T00:37:13.698932+0000","last_peered":"2024-02-18T00:37:13.698932+0000","last_clean":"2024-02-18T00:37:13.698932+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:37:13.698932+0000","last_undegraded":"2024-02-18T00:37:13.698932+0000","last_fullsized":"2024-02-18T00:37:13.698932+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:17.499993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.499993+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":357,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655114+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:37:13.655114+0000","last_peered":"2024-02-18T00:37:13.655114+0000","last_clean":"2024-02-18T00:37:13.655114+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:37:13.655114+0000","last_undegraded":"2024-02-18T00:37:13.655114+0000","last_fullsized":"2024-02-18T00:37:13.655114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":346,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655652+0000","last_change":"2024-02-18T00:36:52.928195+0000","last_active":"2024-02-18T00:37:13.655652+0000","last_peered":"2024-02-18T00:37:13.655652+0000","last_clean":"2024-02-18T00:37:13.655652+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:37:13.655652+0000","last_undegraded":"2024-02-18T00:37:13.655652+0000","last_fullsized":"2024-02-18T00:37:13.655652+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:36:52.928076+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.928076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":325,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657641+0000","last_change":"2024-02-18T00:37:03.248349+0000","last_active":"2024-02-18T00:37:14.657641+0000","last_peered":"2024-02-18T00:37:14.657641+0000","last_clean":"2024-02-18T00:37:14.657641+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:37:14.657641+0000","last_undegraded":"2024-02-18T00:37:14.657641+0000","last_fullsized":"2024-02-18T00:37:14.657641+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.248277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.248277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":315,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658190+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:37:13.658190+0000","last_peered":"2024-02-18T00:37:13.658190+0000","last_clean":"2024-02-18T00:37:13.658190+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:37:13.658190+0000","last_undegraded":"2024-02-18T00:37:13.658190+0000","last_fullsized":"2024-02-18T00:37:13.658190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":271,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658478+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:37:14.658478+0000","last_peered":"2024-02-18T00:37:14.658478+0000","last_clean":"2024-02-18T00:37:14.658478+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:37:14.658478+0000","last_undegraded":"2024-02-18T00:37:14.658478+0000","last_fullsized":"2024-02-18T00:37:14.658478+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":380,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656584+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:37:14.656584+0000","last_peered":"2024-02-18T00:37:14.656584+0000","last_clean":"2024-02-18T00:37:14.656584+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:37:14.656584+0000","last_undegraded":"2024-02-18T00:37:14.656584+0000","last_fullsized":"2024-02-18T00:37:14.656584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":220,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.658971+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:37:13.658971+0000","last_peered":"2024-02-18T00:37:13.658971+0000","last_clean":"2024-02-18T00:37:13.658971+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:37:13.658971+0000","last_undegraded":"2024-02-18T00:37:13.658971+0000","last_fullsized":"2024-02-18T00:37:13.658971+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":339,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.657614+0000","last_change":"2024-02-18T00:36:45.773797+0000","last_active":"2024-02-18T00:37:13.657614+0000","last_peered":"2024-02-18T00:37:13.657614+0000","last_clean":"2024-02-18T00:37:13.657614+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:37:13.657614+0000","last_undegraded":"2024-02-18T00:37:13.657614+0000","last_fullsized":"2024-02-18T00:37:13.657614+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.773695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.773695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":329,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657197+0000","last_change":"2024-02-18T00:36:45.412472+0000","last_active":"2024-02-18T00:37:14.657197+0000","last_peered":"2024-02-18T00:37:14.657197+0000","last_clean":"2024-02-18T00:37:14.657197+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:37:14.657197+0000","last_undegraded":"2024-02-18T00:37:14.657197+0000","last_fullsized":"2024-02-18T00:37:14.657197+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.412406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.412406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":264,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.660689+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:37:14.660689+0000","last_peered":"2024-02-18T00:37:14.660689+0000","last_clean":"2024-02-18T00:37:14.660689+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:37:14.660689+0000","last_undegraded":"2024-02-18T00:37:14.660689+0000","last_fullsized":"2024-02-18T00:37:14.660689+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":355,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.657239+0000","last_change":"2024-02-18T00:36:51.303043+0000","last_active":"2024-02-18T00:37:14.657239+0000","last_peered":"2024-02-18T00:37:14.657239+0000","last_clean":"2024-02-18T00:37:14.657239+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:37:14.657239+0000","last_undegraded":"2024-02-18T00:37:14.657239+0000","last_fullsized":"2024-02-18T00:37:14.657239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:36:51.302920+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.302920+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":227,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.699109+0000","last_change":"2024-02-18T00:36:52.758965+0000","last_active":"2024-02-18T00:37:13.699109+0000","last_peered":"2024-02-18T00:37:13.699109+0000","last_clean":"2024-02-18T00:37:13.699109+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:37:13.699109+0000","last_undegraded":"2024-02-18T00:37:13.699109+0000","last_fullsized":"2024-02-18T00:37:13.699109+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.758904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.758904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":224,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656698+0000","last_change":"2024-02-18T00:36:49.277109+0000","last_active":"2024-02-18T00:37:14.656698+0000","last_peered":"2024-02-18T00:37:14.656698+0000","last_clean":"2024-02-18T00:37:14.656698+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:37:14.656698+0000","last_undegraded":"2024-02-18T00:37:14.656698+0000","last_fullsized":"2024-02-18T00:37:14.656698+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.276979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.276979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":319,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.656590+0000","last_change":"2024-02-18T00:36:40.825174+0000","last_active":"2024-02-18T00:37:13.656590+0000","last_peered":"2024-02-18T00:37:13.656590+0000","last_clean":"2024-02-18T00:37:13.656590+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:37:13.656590+0000","last_undegraded":"2024-02-18T00:37:13.656590+0000","last_fullsized":"2024-02-18T00:37:13.656590+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:40.825055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:36:40.825055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":270,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.656067+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:37:13.656067+0000","last_peered":"2024-02-18T00:37:13.656067+0000","last_clean":"2024-02-18T00:37:13.656067+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:37:13.656067+0000","last_undegraded":"2024-02-18T00:37:13.656067+0000","last_fullsized":"2024-02-18T00:37:13.656067+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":358,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656092+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:37:14.656092+0000","last_peered":"2024-02-18T00:37:14.656092+0000","last_clean":"2024-02-18T00:37:14.656092+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:37:14.656092+0000","last_undegraded":"2024-02-18T00:37:14.656092+0000","last_fullsized":"2024-02-18T00:37:14.656092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":321,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.659281+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:37:13.659281+0000","last_peered":"2024-02-18T00:37:13.659281+0000","last_clean":"2024-02-18T00:37:13.659281+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:37:13.659281+0000","last_undegraded":"2024-02-18T00:37:13.659281+0000","last_fullsized":"2024-02-18T00:37:13.659281+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":333,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.825416+0000","last_change":"2024-02-18T00:36:49.936731+0000","last_active":"2024-02-18T00:37:13.825416+0000","last_peered":"2024-02-18T00:37:13.825416+0000","last_clean":"2024-02-18T00:37:13.825416+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:37:13.825416+0000","last_undegraded":"2024-02-18T00:37:13.825416+0000","last_fullsized":"2024-02-18T00:37:13.825416+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.936644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.936644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":318,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656502+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:37:14.656502+0000","last_peered":"2024-02-18T00:37:14.656502+0000","last_clean":"2024-02-18T00:37:14.656502+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:37:14.656502+0000","last_undegraded":"2024-02-18T00:37:14.656502+0000","last_fullsized":"2024-02-18T00:37:14.656502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":261,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655628+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:37:13.655628+0000","last_peered":"2024-02-18T00:37:13.655628+0000","last_clean":"2024-02-18T00:37:13.655628+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:37:13.655628+0000","last_undegraded":"2024-02-18T00:37:13.655628+0000","last_fullsized":"2024-02-18T00:37:13.655628+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":406,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.654767+0000","last_change":"2024-02-18T00:36:46.942689+0000","last_active":"2024-02-18T00:37:13.654767+0000","last_peered":"2024-02-18T00:37:13.654767+0000","last_clean":"2024-02-18T00:37:13.654767+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:37:13.654767+0000","last_undegraded":"2024-02-18T00:37:13.654767+0000","last_fullsized":"2024-02-18T00:37:13.654767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:36:46.942583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.942583+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":330,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.658163+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:37:14.658163+0000","last_peered":"2024-02-18T00:37:14.658163+0000","last_clean":"2024-02-18T00:37:14.658163+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:37:14.658163+0000","last_undegraded":"2024-02-18T00:37:14.658163+0000","last_fullsized":"2024-02-18T00:37:14.658163+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":320,"reported_epoch":356,"state":"active+clean","last_fresh":"2024-02-18T00:37:14.656363+0000","last_change":"2024-02-18T00:34:39.366241+0000","last_active":"2024-02-18T00:37:14.656363+0000","last_peered":"2024-02-18T00:37:14.656363+0000","last_clean":"2024-02-18T00:37:14.656363+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:37:14.656363+0000","last_undegraded":"2024-02-18T00:37:14.656363+0000","last_fullsized":"2024-02-18T00:37:14.656363+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:39.366159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:34:39.366159+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":324,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-02-18T00:37:13.655439+0000","last_change":"2024-02-18T00:34:38.743498+0000","last_active":"2024-02-18T00:37:13.655439+0000","last_peered":"2024-02-18T00:37:13.655439+0000","last_clean":"2024-02-18T00:37:13.655439+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:37:13.655439+0000","last_undegraded":"2024-02-18T00:37:13.655439+0000","last_fullsized":"2024-02-18T00:37:13.655439+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:38.743393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:38.743393+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":201,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":200,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001380,"num_objects":143,"num_object_clones":2,"num_object_copies":429,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":99,"num_read_kb":6,"num_write":568,"num_write_kb":4469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13758464,"data_stored":12004173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1818,"ondisk_log_size":1818,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":471,"num_read_kb":466,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266788,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463018,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311028,"kb_used_data":9704,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93434124,"statfs":{"total":95995035648,"available":95676542976,"internally_reserved":0,"allocated":9936896,"data_stored":7301066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":2.722,"15min":1.2370000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.1799999999999999,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.1679999999999999,"15min":0.72299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1,"5min":10.433,"15min":10.433},"last":0.57199999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":2.9569999999999999,"15min":1.3200000000000001},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.97199999999999998,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":2.9529999999999998,"15min":1.3109999999999999},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":1.216,"5min":71.872,"15min":71.872},"last":0.65500000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.395,"15min":0.80300000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.141,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.8560000000000001,"15min":0.96799999999999997},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.9930000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.66700000000000004}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.391,"15min":0.80300000000000005},"min":{"1min":0.29999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2629999999999999,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.2929999999999999,"15min":0.77600000000000002},"min":{"1min":0.37,"5min":0.309,"15min":0.309},"max":{"1min":1.0149999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.55400000000000005}]},{"osd":4,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68500000000000005,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.496,"5min":1.1910000000000001,"15min":0.66900000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85099999999999998,"5min":35.18,"15min":35.18},"last":0.46700000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.73199999999999998,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.673,"5min":10.606,"15min":10.606},"last":0.441},{"interface":"front","average":{"1min":0.496,"5min":1.224,"15min":0.69399999999999995},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70899999999999996,"5min":35.119,"15min":35.119},"last":0.60199999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:36:23 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.63200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.4770000000000003,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":1.119,"15min":0.72299999999999998},"min":{"1min":0.35999999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.8730000000000002,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.71799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307132,"kb_used_data":5804,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438020,"statfs":{"total":95995035648,"available":95680532480,"internally_reserved":0,"allocated":5943296,"data_stored":3301147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":1.228,"15min":0.63},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47399999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":1.3160000000000001,"15min":0.66300000000000003},"min":{"1min":0.253,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56999999999999995,"5min":43.137,"15min":43.137},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":1.5289999999999999,"15min":0.75900000000000001},"min":{"1min":0.20699999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51900000000000002,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.95999999999999996,"15min":0.58599999999999997},"min":{"1min":0.28499999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.53400000000000003,"5min":17.596,"15min":17.596},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":1.357,"15min":0.751},"min":{"1min":0.26500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56999999999999995,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.37},{"interface":"front","average":{"1min":0.42299999999999999,"5min":1.3500000000000001,"15min":0.77200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71699999999999997,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":1.1319999999999999,"15min":0.77700000000000002},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73999999999999999,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":1.1719999999999999,"15min":0.77100000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.9790000000000001,"5min":12.75,"15min":12.75},"last":0.69399999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:36:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":1.1319999999999999,"15min":0.77500000000000002},"min":{"1min":0.40400000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85299999999999998,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":1.1379999999999999,"15min":0.78900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.71199999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:37:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":1.165,"15min":0.81000000000000005},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":12.834,"15min":12.834},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.1399999999999999,"15min":0.80000000000000004},"min":{"1min":0.44700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83399999999999996,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:36:21 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":1.147,"15min":0.80500000000000005},"min":{"1min":0.41899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.109,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":1.1459999999999999,"15min":0.81499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.5409999999999999,"5min":12.805,"15min":12.805},"last":0.75900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738503,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303044,"kb_used_data":5812,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442108,"statfs":{"total":95995035648,"available":95684718592,"internally_reserved":0,"allocated":5951488,"data_stored":3300269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":2.2749999999999999,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.2330000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.253},{"interface":"front","average":{"1min":0.61799999999999999,"5min":1.5589999999999999,"15min":0.75900000000000001},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":4.577,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.30499999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":1.8240000000000001,"15min":0.872},"min":{"1min":0.255,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.2430000000000001,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.311,"5min":1.8280000000000001,"15min":0.87},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.224},"max":{"1min":0.40000000000000002,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.27900000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.149,"15min":0.67600000000000005},"min":{"1min":0.255,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.497,"5min":19.006,"15min":19.006},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":2.2610000000000001,"15min":1.0780000000000001},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":4.4489999999999998,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:37:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.651,"15min":0.91600000000000004},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.77500000000000002,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.434},{"interface":"front","average":{"1min":0.78600000000000003,"5min":1.7370000000000001,"15min":0.97999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.2919999999999998,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.56599999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:36:34 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":1.613,"15min":0.93100000000000005},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.262,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":1.8069999999999999,"15min":0.96599999999999997},"min":{"1min":0.41199999999999998,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":57.991,"15min":57.991},"last":0.54200000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.2270000000000001,"15min":0.79500000000000004},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.746,"15min":0.96799999999999997},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.343,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.52900000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:36:14 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":1.8680000000000001,"15min":1.03},"min":{"1min":0.45400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3129999999999999,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":1.798,"15min":0.99099999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.498}]}]},{"osd":2,"up_from":18,"seq":77309411454,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303128,"kb_used_data":5900,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442024,"statfs":{"total":95995035648,"available":95684632576,"internally_reserved":0,"allocated":6041600,"data_stored":3300647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.71599999999999997,"15min":0.49099999999999999},"min":{"1min":0.22,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46100000000000002,"5min":12.144,"15min":12.144},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":1.0529999999999999,"15min":0.60699999999999998},"min":{"1min":0.221,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.45800000000000002,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.29599999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.70899999999999996,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.47099999999999997,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.316},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.82099999999999995,"15min":0.54500000000000004},"min":{"1min":0.29499999999999998,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48499999999999999,"5min":18.919,"15min":18.919},"last":0.36799999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.91400000000000003,"15min":0.61099999999999999},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.503,"5min":22.712,"15min":22.712},"last":0.379},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.91000000000000003,"15min":0.621},"min":{"1min":0.30499999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50700000000000001,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.89000000000000001,"15min":0.68500000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.875,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.84499999999999997,"15min":0.67600000000000005},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76000000000000001,"5min":10.016,"15min":10.016},"last":0.64500000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.77300000000000002,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":6.766,"15min":6.766},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.84099999999999997,"15min":0.69399999999999995},"min":{"1min":0.36699999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78700000000000003,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.68600000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.79400000000000004,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":6.806,"15min":6.806},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.89700000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:36:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.81200000000000006,"15min":0.73099999999999998},"min":{"1min":0.40899999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.75},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.89700000000000002,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.73699999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149816,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319316,"kb_used_data":9800,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93425836,"statfs":{"total":95995035648,"available":95668056064,"internally_reserved":0,"allocated":10035200,"data_stored":7301102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.65900000000000003,"15min":0.47699999999999998},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54100000000000004,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.76000000000000001,"15min":0.52400000000000002},"min":{"1min":0.251,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51900000000000002,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.34699999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.60699999999999998,"15min":0.47999999999999998},"min":{"1min":0.28899999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.54400000000000004,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":1.206,"15min":0.66800000000000004},"min":{"1min":0.25600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59399999999999997,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.379}]},{"osd":2,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":1.0720000000000001,"15min":0.64800000000000002},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71499999999999997,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.36299999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.91800000000000004,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.8410000000000002,"5min":10.131,"15min":10.131},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.70399999999999996,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.86499999999999999,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.52600000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:36:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.70099999999999996,"15min":0.63100000000000001},"min":{"1min":0.42199999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66400000000000003,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.81799999999999995,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.92700000000000005,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.67900000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:36:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":1.05,"15min":0.749},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.8940000000000001,"5min":10.189,"15min":10.189},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.74099999999999999,"15min":0.66400000000000003},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.622}]},{"osd":7,"last update":"Sun Feb 18 00:36:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.79900000000000004,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1599999999999999,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.93000000000000005,"15min":0.70699999999999996},"min":{"1min":0.47999999999999998,"5min":0.42499999999999999,"15min":0.39800000000000002},"max":{"1min":0.88100000000000001,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.68899999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986291,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307164,"kb_used_data":5836,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93437988,"statfs":{"total":95995035648,"available":95680499712,"internally_reserved":0,"allocated":5976064,"data_stored":3301556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":1.542,"15min":0.91100000000000003},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.92600000000000005,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.498,"5min":1.4950000000000001,"15min":0.872},"min":{"1min":0.28199999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.48399999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.80900000000000005,"15min":0.64500000000000002},"min":{"1min":0.33900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89800000000000002,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":1.581,"15min":0.90100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.88,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.58199999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.78000000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.022,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.90000000000000002,"15min":0.68100000000000005},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80200000000000005,"5min":17.77,"15min":17.77},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.68999999999999995,"15min":0.627},"min":{"1min":0.318,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.82199999999999995,"5min":7.218,"15min":7.218},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":1.091,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.309,"15min":0.309},"max":{"1min":1.2350000000000001,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.436,"15min":0.38900000000000001},"min":{"1min":0.253,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.66700000000000004,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.45100000000000001,"15min":0.39900000000000002},"min":{"1min":0.27100000000000002,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.435}]},{"osd":6,"last update":"Sun Feb 18 00:37:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45600000000000002,"15min":0.41899999999999998},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.621},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.30099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65700000000000003,"5min":0.751,"15min":0.751},"last":0.56999999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:36:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.68999999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.5,"5min":0.505,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73599999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59499999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724654,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310952,"kb_used_data":9620,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93434200,"statfs":{"total":95995035648,"available":95676620800,"internally_reserved":0,"allocated":9850880,"data_stored":7301045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57299999999999995,"15min":0.54700000000000004},"min":{"1min":0.29799999999999999,"5min":0.217,"15min":0.217},"max":{"1min":2.2549999999999999,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.76000000000000001,"15min":0.61499999999999999},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.8680000000000003,"5min":12.744,"15min":12.744},"last":0.40899999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.70599999999999996,"15min":0.625},"min":{"1min":0.33200000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.9850000000000001,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.629},"min":{"1min":0.35199999999999998,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":4.2919999999999998,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.61899999999999999},"min":{"1min":0.32200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":3.2959999999999998,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.71999999999999997,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.308,"15min":0.308},"max":{"1min":4.4359999999999999,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73899999999999999,"15min":0.621},"min":{"1min":0.33200000000000002,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":3.4550000000000001,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.502},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.68400000000000005,"15min":0.59799999999999998},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.3920000000000003,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:36:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58699999999999997,"15min":0.44400000000000001},"min":{"1min":0.31900000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.6859999999999999,"5min":11.818,"15min":11.818},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.501,"15min":0.42299999999999999},"min":{"1min":0.35099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.5680000000000001,"5min":4.274,"15min":4.274},"last":0.44700000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":12.362,"5min":18.605,"15min":18.605},"last":0.377},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.46,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:36:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.51100000000000001,"15min":0.46899999999999997},"min":{"1min":0.33300000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.56799999999999995,"15min":0.502},"min":{"1min":0.318,"5min":0.309,"15min":0.309},"max":{"1min":4.649,"5min":4.649,"15min":4.649},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4313088,"data_stored":4000749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4132864,"data_stored":4000318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4235264,"data_stored":4000561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:37:22.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph status --format=json 2024-02-18T00:37:22.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:21.697874+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]': finished 2024-02-18T00:37:22.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:21.697978+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e363: 8 total, 8 up, 7 in 2024-02-18T00:37:22.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:21.699577+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]: dispatch 2024-02-18T00:37:22.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:21.704430+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:22.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:21.707298+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]: dispatch 2024-02-18T00:37:22.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:21.881461+0000 mgr.y (mgr.24454) 351 : cluster [DBG] pgmap v532: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:37:22.991 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:21.697874+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]': finished 2024-02-18T00:37:22.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:21.697978+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e363: 8 total, 8 up, 7 in 2024-02-18T00:37:22.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:21.699577+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]: dispatch 2024-02-18T00:37:22.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:21.704430+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:22.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:21.707298+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]: dispatch 2024-02-18T00:37:22.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:21.881461+0000 mgr.y (mgr.24454) 351 : cluster [DBG] pgmap v532: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:37:23.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:21.697874+0000 mon.a (mon.0) 2322 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-47"}]': finished 2024-02-18T00:37:23.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:21.697978+0000 mon.a (mon.0) 2323 : cluster [DBG] osdmap e363: 8 total, 8 up, 7 in 2024-02-18T00:37:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:21.699577+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]: dispatch 2024-02-18T00:37:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:21.704430+0000 mon.a (mon.0) 2324 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:21.707298+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]: dispatch 2024-02-18T00:37:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:21.881461+0000 mgr.y (mgr.24454) 351 : cluster [DBG] pgmap v532: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:22.700866+0000 mon.a (mon.0) 2326 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:22.716738+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:22.716807+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]': finished 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:22.716880+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e364: 8 total, 8 up, 7 in 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:22.718063+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:23.658387+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:23.658460+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e365: 8 total, 8 up, 7 in 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:23.677434+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:23.678164+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:23.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:23.678530+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi059-82882-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:22.700866+0000 mon.a (mon.0) 2326 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:22.716738+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:22.716807+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]': finished 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:22.716880+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e364: 8 total, 8 up, 7 in 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:22.718063+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:23.658387+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:23.658460+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e365: 8 total, 8 up, 7 in 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:23.677434+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:23.678164+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:23.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:23.678530+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi059-82882-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:22.700866+0000 mon.a (mon.0) 2326 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:22.716738+0000 mon.a (mon.0) 2327 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:22.716807+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-47", "mode": "readproxy"}]': finished 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:22.716880+0000 mon.a (mon.0) 2329 : cluster [DBG] osdmap e364: 8 total, 8 up, 7 in 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:22.718063+0000 mon.a (mon.0) 2330 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]: dispatch 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:23.658387+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? 172.21.15.59:0/397807707' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi059-82882-61"}]': finished 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:23.658460+0000 mon.a (mon.0) 2332 : cluster [DBG] osdmap e365: 8 total, 8 up, 7 in 2024-02-18T00:37:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:23.677434+0000 mon.a (mon.0) 2333 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:24.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:23.678164+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:24.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:23.678530+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi059-82882-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:24.146 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:24.150 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:24.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:24.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:24.280 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:24.717 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:23.882062+0000 mgr.y (mgr.24454) 352 : cluster [DBG] pgmap v535: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:37:24.717 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:24.657740+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi059-82882-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:24.717 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:24.657820+0000 mon.a (mon.0) 2337 : cluster [DBG] osdmap e366: 8 total, 8 up, 7 in 2024-02-18T00:37:24.717 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:24.660440+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi059-82882-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:25.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:23.882062+0000 mgr.y (mgr.24454) 352 : cluster [DBG] pgmap v535: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:24.657740+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi059-82882-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:24.657820+0000 mon.a (mon.0) 2337 : cluster [DBG] osdmap e366: 8 total, 8 up, 7 in 2024-02-18T00:37:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:24.660440+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi059-82882-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:23.882062+0000 mgr.y (mgr.24454) 352 : cluster [DBG] pgmap v535: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:37:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:24.657740+0000 mon.a (mon.0) 2336 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi059-82882-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:24.657820+0000 mon.a (mon.0) 2337 : cluster [DBG] osdmap e366: 8 total, 8 up, 7 in 2024-02-18T00:37:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:24.660440+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi059-82882-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:25.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.846+0000 7f84e11e2700 1 -- 172.21.15.59:0/1267774256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f8cb0 msgr2=0x7f84dc0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:25.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.846+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1267774256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f8cb0 0x7f84dc0f9090 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 rx=0x7f84d00047b0 tx=0x7f84d0009cb0).stop 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 -- 172.21.15.59:0/1267774256 shutdown_connections 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1267774256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84dc102210 0x7f84dc104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1267774256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f8cb0 0x7f84dc0f9090 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1267774256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84dc0f9610 0x7f84dc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 -- 172.21.15.59:0/1267774256 >> 172.21.15.59:0/1267774256 conn(0x7f84dc0f2820 msgr2=0x7f84dc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 -- 172.21.15.59:0/1267774256 shutdown_connections 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 -- 172.21.15.59:0/1267774256 wait complete. 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 librados: starting msgr at 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 librados: starting objecter 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.847+0000 7f84e11e2700 1 Processor -- start 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 -- start start 2024-02-18T00:37:25.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 librados: setting wanted keys 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 librados: calling monclient init 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84dc0f8cb0 0x7f84dc0f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f9610 0x7f84dc0f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84dc102210 0x7f84dc0f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1166b0 con 0x7f84dc0f8cb0 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc116820 con 0x7f84dc102210 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84e11e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1169e0 con 0x7f84dc0f9610 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f9610 0x7f84dc0f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f9610 0x7f84dc0f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:42290/0 (socket says 172.21.15.59:42290) 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 -- 172.21.15.59:0/1925700430 learned_addr learned my addr 172.21.15.59:0/1925700430 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84dad9d700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84dc0f8cb0 0x7f84dc0f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:25.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84db59e700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84dc102210 0x7f84dc0f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 -- 172.21.15.59:0/1925700430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84dc102210 msgr2=0x7f84dc0f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84dc102210 0x7f84dc0f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 -- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84dc0f8cb0 msgr2=0x7f84dc0f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84dc0f8cb0 0x7f84dc0f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 -- 172.21.15.59:0/1925700430 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc005040 con 0x7f84dc0f9610 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84da59c700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f9610 0x7f84dc0f7150 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7f84cc009400 tx=0x7f84cc009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84c3fff700 10 client.?.objecter ms_handle_connect 0x7f84dc0f9610 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84c3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.848+0000 7f84c3fff700 1 -- 172.21.15.59:0/1925700430 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc007a40 con 0x7f84dc0f9610 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.849+0000 7f84c3fff700 1 -- 172.21.15.59:0/1925700430 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84cc00f040 con 0x7f84dc0f9610 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.849+0000 7f84c3fff700 1 -- 172.21.15.59:0/1925700430 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc014a70 con 0x7f84dc0f9610 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.849+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d0005040 con 0x7f84dc0f9610 2024-02-18T00:37:25.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.849+0000 7f84e11e2700 10 client.41003.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.849+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc116fd0 con 0x7f84dc0f9610 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.849+0000 7f84e11e2700 1 librados: init done 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.849+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc05fa60 con 0x7f84dc0f9610 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84c3fff700 1 -- 172.21.15.59:0/1925700430 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f84cc010530 con 0x7f84dc0f9610 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84c3fff700 1 --2- 172.21.15.59:0/1925700430 >> v2:172.21.15.59:6800/3721891797 conn(0x7f84c4062c90 0x7f84c4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84c3fff700 1 -- 172.21.15.59:0/1925700430 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(366..366 src has 180..366) v4 ==== 8315+0+0 (secure 0 0 0) 0x7f84cc07cec0 con 0x7f84dc0f9610 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84c3fff700 10 client.41003.objecter ms_dispatch 0x7f84dc001550 osd_map(366..366 src has 180..366) v4 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84c3fff700 3 client.41003.objecter handle_osd_map got epochs [366,366] > 0 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84c3fff700 3 client.41003.objecter handle_osd_map decoding full epoch 366 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84dad9d700 1 --2- 172.21.15.59:0/1925700430 >> v2:172.21.15.59:6800/3721891797 conn(0x7f84c4062c90 0x7f84c4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.850+0000 7f84c3fff700 20 client.41003.objecter dump_active .. 0 homeless 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.851+0000 7f84c3fff700 10 client.41003.objecter ms_handle_connect 0x7f84c4062c90 2024-02-18T00:37:25.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.851+0000 7f84dad9d700 1 --2- 172.21.15.59:0/1925700430 >> v2:172.21.15.59:6800/3721891797 conn(0x7f84c4062c90 0x7f84c4065150 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 rx=0x7f84d0005010 tx=0x7f84d0009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:25.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:24.855+0000 7f84c3fff700 1 -- 172.21.15.59:0/1925700430 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f84cc049f60 con 0x7f84dc0f9610 2024-02-18T00:37:25.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:25.037+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f84dc1072c0 con 0x7f84dc0f9610 2024-02-18T00:37:25.330 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:25.331 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:25.336 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:25.336 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:25.437 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:25.469 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:25.491 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:25.493 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:25.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:25.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:25.629 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.100+0000 7f84c3fff700 1 -- 172.21.15.59:0/1925700430 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v367) v1 ==== 91+0+0 (secure 0 0 0) 0x7f84cc050290 con 0x7f84dc0f9610 2024-02-18T00:37:26.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 10 librados: watch_flush enter 2024-02-18T00:37:26.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 10 librados: watch_flush exit 2024-02-18T00:37:26.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 20 client.41003.objecter shutdown clearing up homeless session... 2024-02-18T00:37:26.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 10 client.41003.objecter successfully canceled tick 2024-02-18T00:37:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 >> v2:172.21.15.59:6800/3721891797 conn(0x7f84c4062c90 msgr2=0x7f84c4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1925700430 >> v2:172.21.15.59:6800/3721891797 conn(0x7f84c4062c90 0x7f84c4065150 secure :-1 s=READY pgs=3087 cs=0 l=1 rev1=1 rx=0x7f84d0005010 tx=0x7f84d0009cb0).stop 2024-02-18T00:37:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f9610 msgr2=0x7f84dc0f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f9610 0x7f84dc0f7150 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7f84cc009400 tx=0x7f84cc009cb0).stop 2024-02-18T00:37:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 shutdown_connections 2024-02-18T00:37:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1925700430 >> v2:172.21.15.59:6800/3721891797 conn(0x7f84c4062c90 0x7f84c4065150 unknown :-1 s=CLOSED pgs=3087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84dc102210 0x7f84dc0f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84dc0f8cb0 0x7f84dc0f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 --2- 172.21.15.59:0/1925700430 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84dc0f9610 0x7f84dc0f7150 secure :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 rx=0x7f84cc009400 tx=0x7f84cc009cb0).stop 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 >> 172.21.15.59:0/1925700430 conn(0x7f84dc0f2820 msgr2=0x7f84dc0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 shutdown_connections 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 -- 172.21.15.59:0/1925700430 wait complete. 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:26.103+0000 7f84e11e2700 1 librados: shutdown 2024-02-18T00:37:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:37:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:25.038254+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.59:0/1925700430' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:37:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:25.038605+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:37:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:25.655039+0000 mon.a (mon.0) 2340 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:26.100323+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:37:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:25.038254+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.59:0/1925700430' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:37:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:25.038605+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:37:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:25.655039+0000 mon.a (mon.0) 2340 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:26.100323+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:37:26.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:25.038254+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.59:0/1925700430' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:37:26.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:25.038605+0000 mon.a (mon.0) 2339 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:37:26.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:25.655039+0000 mon.a (mon.0) 2340 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:26.100323+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:37:27.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:25.864798+0000 osd.1 (osd.1) 183 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:37:27.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:25.868059+0000 osd.1 (osd.1) 184 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:37:27.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:25.882862+0000 mgr.y (mgr.24454) 353 : cluster [DBG] pgmap v538: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:27.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:26.100394+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e367: 8 total, 8 up, 7 in 2024-02-18T00:37:27.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:26.203929+0000 osd.0 (osd.0) 173 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:26.204912+0000 osd.0 (osd.0) 174 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:26.664724+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi059-82882-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:26.664826+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e368: 8 total, 8 up, 7 in 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:25.864798+0000 osd.1 (osd.1) 183 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:25.868059+0000 osd.1 (osd.1) 184 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:25.882862+0000 mgr.y (mgr.24454) 353 : cluster [DBG] pgmap v538: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:26.100394+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e367: 8 total, 8 up, 7 in 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:26.203929+0000 osd.0 (osd.0) 173 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:26.204912+0000 osd.0 (osd.0) 174 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:26.664724+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi059-82882-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:27.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:26.664826+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e368: 8 total, 8 up, 7 in 2024-02-18T00:37:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:25.864798+0000 osd.1 (osd.1) 183 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:37:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:25.868059+0000 osd.1 (osd.1) 184 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:37:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:25.882862+0000 mgr.y (mgr.24454) 353 : cluster [DBG] pgmap v538: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:26.100394+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e367: 8 total, 8 up, 7 in 2024-02-18T00:37:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:26.203929+0000 osd.0 (osd.0) 173 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:37:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:26.204912+0000 osd.0 (osd.0) 174 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:37:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:26.664724+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi059-82882-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:27.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:26.664826+0000 mon.a (mon.0) 2344 : cluster [DBG] osdmap e368: 8 total, 8 up, 7 in 2024-02-18T00:37:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:27.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:27.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:27.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:27.794 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:28.145 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:28.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:28.147 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:28.248 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:28.280 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:28.302 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:28.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:28.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:28.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:28.435 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:28.459 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:28.462 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:28.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:28.467 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:37:28.467 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:37:28.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:28.593 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:28.612 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:28.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:28.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:28.727 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.127288+0000 osd.4 (osd.4) 207 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.130248+0000 osd.4 (osd.4) 208 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.356720+0000 osd.2 (osd.2) 157 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.357605+0000 osd.2 (osd.2) 158 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.639214+0000 osd.5 (osd.5) 155 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.642657+0000 osd.5 (osd.5) 156 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.666550+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e369: 8 total, 8 up, 7 in 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.732021+0000 osd.6 (osd.6) 171 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.733602+0000 osd.6 (osd.6) 172 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.829172+0000 osd.1 (osd.1) 185 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.830220+0000 osd.1 (osd.1) 186 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:37:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:27.883231+0000 mgr.y (mgr.24454) 354 : cluster [DBG] pgmap v541: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.127288+0000 osd.4 (osd.4) 207 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.130248+0000 osd.4 (osd.4) 208 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.356720+0000 osd.2 (osd.2) 157 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.357605+0000 osd.2 (osd.2) 158 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.639214+0000 osd.5 (osd.5) 155 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.642657+0000 osd.5 (osd.5) 156 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.666550+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e369: 8 total, 8 up, 7 in 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.732021+0000 osd.6 (osd.6) 171 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.733602+0000 osd.6 (osd.6) 172 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.829172+0000 osd.1 (osd.1) 185 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:37:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.830220+0000 osd.1 (osd.1) 186 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:37:28.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:27.883231+0000 mgr.y (mgr.24454) 354 : cluster [DBG] pgmap v541: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:28.755 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:28.852 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:28.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:28.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:28.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:28.985 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:29.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.127288+0000 osd.4 (osd.4) 207 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.130248+0000 osd.4 (osd.4) 208 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.356720+0000 osd.2 (osd.2) 157 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.357605+0000 osd.2 (osd.2) 158 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.639214+0000 osd.5 (osd.5) 155 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.642657+0000 osd.5 (osd.5) 156 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.666550+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e369: 8 total, 8 up, 7 in 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.732021+0000 osd.6 (osd.6) 171 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.733602+0000 osd.6 (osd.6) 172 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.829172+0000 osd.1 (osd.1) 185 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.830220+0000 osd.1 (osd.1) 186 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:37:29.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:27.883231+0000 mgr.y (mgr.24454) 354 : cluster [DBG] pgmap v541: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:37:29.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:37:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:28.392818+0000 osd.0 (osd.0) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:28.392977+0000 osd.0 (osd.0) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:28.672088+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e370: 8 total, 8 up, 7 in 2024-02-18T00:37:30.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:28.676795+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:28.392818+0000 osd.0 (osd.0) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:28.392977+0000 osd.0 (osd.0) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:30.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:28.672088+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e370: 8 total, 8 up, 7 in 2024-02-18T00:37:30.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:28.676795+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:30.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:28.392818+0000 osd.0 (osd.0) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:37:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:28.392977+0000 osd.0 (osd.0) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:37:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:28.672088+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e370: 8 total, 8 up, 7 in 2024-02-18T00:37:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:28.676795+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:30.535 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:30.539 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:30.540 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:30.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-18T00:37:30.641 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:30.849 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.138396+0000 osd.0 (osd.0) 177 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:37:30.849 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.141895+0000 osd.0 (osd.0) 178 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:37:30.849 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.173918+0000 osd.4 (osd.4) 209 : cluster [DBG] 4.c scrub starts 2024-02-18T00:37:30.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.177364+0000 osd.4 (osd.4) 210 : cluster [DBG] 4.c scrub ok 2024-02-18T00:37:30.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.373860+0000 osd.2 (osd.2) 159 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:37:30.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.374706+0000 osd.2 (osd.2) 160 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:37:30.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:29.673743+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:30.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.673812+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e371: 8 total, 8 up, 7 in 2024-02-18T00:37:30.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:29.678400+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:30.850 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.755649+0000 osd.1 (osd.1) 187 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:37:30.851 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.759110+0000 osd.1 (osd.1) 188 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:37:30.851 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:29.883691+0000 mgr.y (mgr.24454) 355 : cluster [DBG] pgmap v544: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:30.851 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:30.677552+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:30.851 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:30.677627+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e372: 8 total, 8 up, 7 in 2024-02-18T00:37:30.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.138396+0000 osd.0 (osd.0) 177 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:37:30.851 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.141895+0000 osd.0 (osd.0) 178 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.173918+0000 osd.4 (osd.4) 209 : cluster [DBG] 4.c scrub starts 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.177364+0000 osd.4 (osd.4) 210 : cluster [DBG] 4.c scrub ok 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.373860+0000 osd.2 (osd.2) 159 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.374706+0000 osd.2 (osd.2) 160 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:29.673743+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.673812+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e371: 8 total, 8 up, 7 in 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:29.678400+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.755649+0000 osd.1 (osd.1) 187 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.759110+0000 osd.1 (osd.1) 188 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:29.883691+0000 mgr.y (mgr.24454) 355 : cluster [DBG] pgmap v544: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:30.677552+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:30.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:30.677627+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e372: 8 total, 8 up, 7 in 2024-02-18T00:37:30.883 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:30.887 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:30.887 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:30.988 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.138396+0000 osd.0 (osd.0) 177 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:37:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.141895+0000 osd.0 (osd.0) 178 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.173918+0000 osd.4 (osd.4) 209 : cluster [DBG] 4.c scrub starts 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.177364+0000 osd.4 (osd.4) 210 : cluster [DBG] 4.c scrub ok 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.373860+0000 osd.2 (osd.2) 159 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.374706+0000 osd.2 (osd.2) 160 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:29.673743+0000 mon.a (mon.0) 2348 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.673812+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e371: 8 total, 8 up, 7 in 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:29.678400+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]: dispatch 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.755649+0000 osd.1 (osd.1) 187 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.759110+0000 osd.1 (osd.1) 188 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:29.883691+0000 mgr.y (mgr.24454) 355 : cluster [DBG] pgmap v544: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:30.677552+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? 172.21.15.59:0/4265475043' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi059-82882-62"}]': finished 2024-02-18T00:37:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:30.677627+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e372: 8 total, 8 up, 7 in 2024-02-18T00:37:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:31.056 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:31.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:31.059 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:31.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 -- 172.21.15.59:0/3068414371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 msgr2=0x7fb0fc104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 --2- 172.21.15.59:0/3068414371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 0x7fb0fc104600 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 rx=0x7fb0f00047b0 tx=0x7fb0f0009cb0).stop 2024-02-18T00:37:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 -- 172.21.15.59:0/3068414371 shutdown_connections 2024-02-18T00:37:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 --2- 172.21.15.59:0/3068414371 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 0x7fb0fc104600 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 --2- 172.21.15.59:0/3068414371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f8cb0 0x7fb0fc0f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 --2- 172.21.15.59:0/3068414371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0fc0f9610 0x7fb0fc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 -- 172.21.15.59:0/3068414371 >> 172.21.15.59:0/3068414371 conn(0x7fb0fc0f2820 msgr2=0x7fb0fc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 -- 172.21.15.59:0/3068414371 shutdown_connections 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.165+0000 7fb101a75700 1 -- 172.21.15.59:0/3068414371 wait complete. 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.166+0000 7fb101a75700 1 librados: starting msgr at 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.166+0000 7fb101a75700 1 librados: starting objecter 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.166+0000 7fb101a75700 1 Processor -- start 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.166+0000 7fb101a75700 1 -- start start 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.166+0000 7fb101a75700 1 librados: setting wanted keys 2024-02-18T00:37:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.166+0000 7fb101a75700 1 librados: calling monclient init 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb101a75700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0fc0f8cb0 0x7fb0fc10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb101a75700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f9610 0x7fb0fc10e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb101a75700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 0x7fb0fc1080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb101a75700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1085a0 con 0x7fb0fc0f9610 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb101a75700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc108710 con 0x7fb0fc0f8cb0 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb101a75700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1088d0 con 0x7fb0fc102210 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f9610 0x7fb0fc10e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f9610 0x7fb0fc10e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60030/0 (socket says 172.21.15.59:60030) 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 -- 172.21.15.59:0/4066675366 learned_addr learned my addr 172.21.15.59:0/4066675366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0fb7fe700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0fc0f8cb0 0x7fb0fc10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0fbfff700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 0x7fb0fc1080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 -- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 msgr2=0x7fb0fc1080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 0x7fb0fc1080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 -- 172.21.15.59:0/4066675366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0fc0f8cb0 msgr2=0x7fb0fc10df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0fc0f8cb0 0x7fb0fc10df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 -- 172.21.15.59:0/4066675366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40097c0 con 0x7fb0fc0f9610 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0fbfff700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 0x7fb0fc1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0faffd700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f9610 0x7fb0fc10e400 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 rx=0x7fb0ec00c7c0 tx=0x7fb0ec007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0f8f74700 10 client.?.objecter ms_handle_connect 0x7fb0fc0f9610 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0f8f74700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0f8f74700 1 -- 172.21.15.59:0/4066675366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec004430 con 0x7fb0fc0f9610 2024-02-18T00:37:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.167+0000 7fb0f8f74700 1 -- 172.21.15.59:0/4066675366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0ec00e970 con 0x7fb0fc0f9610 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.168+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f0005040 con 0x7fb0fc0f9610 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.168+0000 7fb0f8f74700 1 -- 172.21.15.59:0/4066675366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec013a60 con 0x7fb0fc0f9610 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.168+0000 7fb101a75700 10 client.31212.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.168+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc114d40 con 0x7fb0fc0f9610 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.168+0000 7fb101a75700 1 librados: init done 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.169+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc05fa60 con 0x7fb0fc0f9610 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.169+0000 7fb0f8f74700 1 -- 172.21.15.59:0/4066675366 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb0ec00eae0 con 0x7fb0fc0f9610 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0f8f74700 1 --2- 172.21.15.59:0/4066675366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb0e4062c90 0x7fb0e4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0f8f74700 1 -- 172.21.15.59:0/4066675366 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(372..372 src has 180..372) v4 ==== 8027+0+0 (secure 0 0 0) 0x7fb0ec07ce10 con 0x7fb0fc0f9610 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0f8f74700 10 client.31212.objecter ms_dispatch 0x7fb0fc001550 osd_map(372..372 src has 180..372) v4 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0f8f74700 3 client.31212.objecter handle_osd_map got epochs [372,372] > 0 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0f8f74700 3 client.31212.objecter handle_osd_map decoding full epoch 372 2024-02-18T00:37:31.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0fb7fe700 1 --2- 172.21.15.59:0/4066675366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb0e4062c90 0x7fb0e4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:31.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0f8f74700 20 client.31212.objecter dump_active .. 0 homeless 2024-02-18T00:37:31.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0fb7fe700 1 --2- 172.21.15.59:0/4066675366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb0e4062c90 0x7fb0e4065150 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 rx=0x7fb0f4004fd0 tx=0x7fb0f4004d10).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:31.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.170+0000 7fb0f8f74700 10 client.31212.objecter ms_handle_connect 0x7fb0e4062c90 2024-02-18T00:37:31.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.174+0000 7fb0f8f74700 1 -- 172.21.15.59:0/4066675366 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb0ec050340 con 0x7fb0fc0f9610 2024-02-18T00:37:31.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:37:31.369 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:37:31.371 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:31.392 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:31.395 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:31.395 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:31.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.424+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb0fc05ee80 con 0x7fb0fc0f9610 2024-02-18T00:37:31.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.425+0000 7fb0f8f74700 1 -- 172.21.15.59:0/4066675366 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2518 (secure 0 0 0) 0x7fb0ec04ff30 con 0x7fb0fc0f9610 2024-02-18T00:37:31.426 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:37:31.426 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","health":{"status":"HEALTH_WARN","checks":{"CACHE_POOL_NO_HIT_SET":{"severity":"HEALTH_WARN","summary":{"message":"1 cache pools are missing hit_sets","count":1},"muted":false},"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":723,"monmap":{"epoch":3,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":372,"num_osds":8,"num_up_osds":8,"osd_up_since":1708216146,"num_in_osds":7,"osd_in_since":1708216427,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":238}],"num_pgs":238,"num_pools":9,"num_objects":369,"data_bytes":4007189,"bytes_used":2209509376,"bytes_avail":669755740160,"bytes_total":671965249536},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.59:8443/","prometheus":"http://172.21.15.59:9283/"}},"servicemap":{"epoch":23,"modified":"2024-02-18T00:33:31.796486+0000","services":{"rgw":{"daemons":{"summary":"","24347":{"start_epoch":12,"start_stamp":"2024-02-18T00:29:22.198518+0000","gid":24347,"addr":"172.21.15.59:0/2522576766","metadata":{"arch":"x86_64","ceph_release":"pacific","ceph_version":"ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)","ceph_version_short":"16.2.14-548-g81bd20d6","container_hostname":"smithi059","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 8","distro_version":"8","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi059","id":"foo.a","kernel_description":"#1 SMP Mon Feb 5 18:34:56 UTC 2024","kernel_version":"4.18.0-539.el8.x86_64","mem_swap_kb":"0","mem_total_kb":"32543852","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"e67bc818-d879-487a-b572-f16f51d463f9","zone_name":"default","zonegroup_id":"1a312463-4a64-495f-95ab-5ad353fdc6fb","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 10 librados: watch_flush enter 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 10 librados: watch_flush exit 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 20 client.31212.objecter shutdown clearing up homeless session... 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 10 client.31212.objecter successfully canceled tick 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb0e4062c90 msgr2=0x7fb0e4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 --2- 172.21.15.59:0/4066675366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb0e4062c90 0x7fb0e4065150 secure :-1 s=READY pgs=3089 cs=0 l=1 rev1=1 rx=0x7fb0f4004fd0 tx=0x7fb0f4004d10).stop 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f9610 msgr2=0x7fb0fc10e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:31.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f9610 0x7fb0fc10e400 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 rx=0x7fb0ec00c7c0 tx=0x7fb0ec007ef0).stop 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 shutdown_connections 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 --2- 172.21.15.59:0/4066675366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb0e4062c90 0x7fb0e4065150 unknown :-1 s=CLOSED pgs=3089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0fc0f8cb0 0x7fb0fc10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0fc102210 0x7fb0fc1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 --2- 172.21.15.59:0/4066675366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0fc0f9610 0x7fb0fc10e400 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 >> 172.21.15.59:0/4066675366 conn(0x7fb0fc0f2820 msgr2=0x7fb0fc0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 shutdown_connections 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 -- 172.21.15.59:0/4066675366 wait complete. 2024-02-18T00:37:31.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:31.427+0000 7fb101a75700 1 librados: shutdown 2024-02-18T00:37:31.496 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:31.525 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:31.546 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:31.548 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:31.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:31.650 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:31.679 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:31.703 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:31.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:30.701914+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:30.702275+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:30.702916+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:30.703189+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:30.703636+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:30.703897+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:30.845977+0000 osd.3 (osd.3) 205 : cluster [DBG] 6.c scrub starts 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:30.848958+0000 osd.3 (osd.3) 206 : cluster [DBG] 6.c scrub ok 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:31.201210+0000 mgr.y (mgr.24454) 356 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:31.425144+0000 mon.a (mon.0) 2356 : audit [DBG] from='client.? 172.21.15.59:0/4066675366' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:30.701914+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:30.702275+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:30.702916+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:30.703189+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:30.703636+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:30.703897+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:30.845977+0000 osd.3 (osd.3) 205 : cluster [DBG] 6.c scrub starts 2024-02-18T00:37:31.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:30.848958+0000 osd.3 (osd.3) 206 : cluster [DBG] 6.c scrub ok 2024-02-18T00:37:31.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:31.201210+0000 mgr.y (mgr.24454) 356 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:31.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:31.425144+0000 mon.a (mon.0) 2356 : audit [DBG] from='client.? 172.21.15.59:0/4066675366' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:37:31.797 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:31.804 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:31.831 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:31.850 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:31.852 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:31.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:31.953 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:31.978 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:30.701914+0000 mon.c (mon.2) 353 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.978 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:30.702275+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:30.702916+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:30.703189+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:30.703636+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:30.703897+0000 mon.a (mon.0) 2355 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:30.845977+0000 osd.3 (osd.3) 205 : cluster [DBG] 6.c scrub starts 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:30.848958+0000 osd.3 (osd.3) 206 : cluster [DBG] 6.c scrub ok 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:31.201210+0000 mgr.y (mgr.24454) 356 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:31.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:31.425144+0000 mon.a (mon.0) 2356 : audit [DBG] from='client.? 172.21.15.59:0/4066675366' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-02-18T00:37:31.980 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:31.999 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:32.001 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:32.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:32.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 -- 172.21.15.59:0/1638286229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 msgr2=0x7ff8180fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 --2- 172.21.15.59:0/1638286229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff8180fdf90 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7ff80c0047b0 tx=0x7ff80c009cb0).stop 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 -- 172.21.15.59:0/1638286229 shutdown_connections 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 --2- 172.21.15.59:0/1638286229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8180f8420 0x7ff818104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 --2- 172.21.15.59:0/1638286229 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff8180fdf90 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 --2- 172.21.15.59:0/1638286229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8180fe510 0x7ff8180f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 -- 172.21.15.59:0/1638286229 >> 172.21.15.59:0/1638286229 conn(0x7ff8180f3800 msgr2=0x7ff8180f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 -- 172.21.15.59:0/1638286229 shutdown_connections 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.017+0000 7ff81ce37700 1 -- 172.21.15.59:0/1638286229 wait complete. 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.018+0000 7ff81ce37700 1 librados: starting msgr at 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.018+0000 7ff81ce37700 1 librados: starting objecter 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.018+0000 7ff81ce37700 1 Processor -- start 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 -- start start 2024-02-18T00:37:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 librados: setting wanted keys 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 librados: calling monclient init 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8180f8420 0x7ff81810e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff81810e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8180fe510 0x7ff818108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff818108800 con 0x7ff8180fe510 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff818108970 con 0x7ff8180f8420 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81ce37700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff818108b30 con 0x7ff8180fdbb0 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff81810e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff816d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8180fe510 0x7ff818108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff816d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8180fe510 0x7ff818108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60058/0 (socket says 172.21.15.59:60058) 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff81810e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43164/0 (socket says 172.21.15.59:43164) 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff816d9d700 1 -- 172.21.15.59:0/2910222106 learned_addr learned my addr 172.21.15.59:0/2910222106 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:32.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8180f8420 0x7ff81810e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 -- 172.21.15.59:0/2910222106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8180f8420 msgr2=0x7ff81810e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 --2- 172.21.15.59:0/2910222106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8180f8420 0x7ff81810e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 -- 172.21.15.59:0/2910222106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8180fe510 msgr2=0x7ff818108310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 --2- 172.21.15.59:0/2910222106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8180fe510 0x7ff818108310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 -- 172.21.15.59:0/2910222106 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808005040 con 0x7ff8180fdbb0 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff816d9d700 1 --2- 172.21.15.59:0/2910222106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8180fe510 0x7ff818108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff815d9b700 1 --2- 172.21.15.59:0/2910222106 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff81810e660 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 rx=0x7ff8080091f0 tx=0x7ff808009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff8077fe700 10 client.?.objecter ms_handle_connect 0x7ff8180fdbb0 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.019+0000 7ff8077fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80800e030 con 0x7ff8180fdbb0 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff80800f040 con 0x7ff8180fdbb0 2024-02-18T00:37:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c005040 con 0x7ff8180fdbb0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8080147e0 con 0x7ff8180fdbb0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff81ce37700 10 client.41021.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff818109170 con 0x7ff8180fdbb0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff81ce37700 1 librados: init done 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.020+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff81805ee80 con 0x7ff8180fdbb0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.021+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff8080104f0 con 0x7ff8180fdbb0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.021+0000 7ff8077fe700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff800062c90 0x7ff800065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(373..373 src has 180..373) v4 ==== 8317+0+0 (secure 0 0 0) 0x7ff80807d370 con 0x7ff8180fdbb0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 osd_map(373..373 src has 180..373) v4 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 3 client.41021.objecter handle_osd_map got epochs [373,373] > 0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 3 client.41021.objecter handle_osd_map decoding full epoch 373 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff800062c90 0x7ff800065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 20 client.41021.objecter dump_active .. 0 homeless 2024-02-18T00:37:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v373) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff808084070 con 0x7ff8180fdbb0 2024-02-18T00:37:32.046 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:37:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff800062c90 2024-02-18T00:37:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 10 client.41021.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 1 2024-02-18T00:37:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff800062c90 0x7ff800065150 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 rx=0x7ff80c005010 tx=0x7ff80c007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8180fee60 0x7ff8180ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 1 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 0 1 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818107110 osd=0 4 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 10 client.41021.objecter _send_command 1 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8181097f0 con 0x7ff8180fee60 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff816d9d700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8180fee60 0x7ff8180ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff816d9d700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8180fee60 0x7ff8180ff240 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.022+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff8180fee60 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.023+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8181097f0 con 0x7ff8180fee60 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.023+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.023+0000 7ff8077fe700 10 client.41021.objecter _finish_command 1 = system:0 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.023+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818107110 osd=0 4 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.023+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 0 1 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 10 client.41021.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 2 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 2 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818107110 osd=0 3 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 0 2 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818107110 osd=0 4 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 10 client.41021.objecter _send_command 2 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.038+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8180273c0 con 0x7ff8180fee60 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8180273c0 con 0x7ff8180fee60 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff8077fe700 10 client.41021.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818107110 osd=0 4 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 0 2 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 10 client.41021.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 3 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8180598f0 0x7ff818059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 3 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 1 3 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818028710 osd=1 4 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 10 client.41021.objecter _send_command 3 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.039+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff81811b780 con 0x7ff8180598f0 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8180598f0 0x7ff818059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff8180598f0 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8180598f0 0x7ff818059cd0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff81811b780 con 0x7ff8180598f0 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:37:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff8077fe700 10 client.41021.objecter _finish_command 3 = system:0 2024-02-18T00:37:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818028710 osd=1 4 2024-02-18T00:37:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.040+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 1 3 2024-02-18T00:37:32.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 10 client.41021.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 4 2024-02-18T00:37:32.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 4 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818028710 osd=1 3 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 1 4 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818028710 osd=1 4 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 10 client.41021.objecter _send_command 4 2024-02-18T00:37:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.056+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff818059360 con 0x7ff8180598f0 2024-02-18T00:37:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.057+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff818059360 con 0x7ff8180598f0 2024-02-18T00:37:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.057+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.057+0000 7ff8077fe700 10 client.41021.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.057+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818028710 osd=1 4 2024-02-18T00:37:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.057+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 1 4 2024-02-18T00:37:32.059 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:37:32.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 10 client.41021.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 5 2024-02-18T00:37:32.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff818028270 0x7ff81811a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 5 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 2 5 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818059770 osd=2 4 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 10 client.41021.objecter _send_command 5 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff81811a690 con 0x7ff818028270 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff815d9b700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff818028270 0x7ff81811a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff818028270 2024-02-18T00:37:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.058+0000 7ff815d9b700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff818028270 0x7ff81811a1a0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.059+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff81811a690 con 0x7ff818028270 2024-02-18T00:37:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.059+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:37:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.059+0000 7ff8077fe700 10 client.41021.objecter _finish_command 5 = system:0 2024-02-18T00:37:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.059+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818059770 osd=2 4 2024-02-18T00:37:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.059+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 2 5 2024-02-18T00:37:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 10 client.41021.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 6 2024-02-18T00:37:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 6 2024-02-18T00:37:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818059770 osd=2 3 2024-02-18T00:37:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 2 6 2024-02-18T00:37:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818059770 osd=2 4 2024-02-18T00:37:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 10 client.41021.objecter _send_command 6 2024-02-18T00:37:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.075+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8180277d0 con 0x7ff818028270 2024-02-18T00:37:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.077+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8180277d0 con 0x7ff818028270 2024-02-18T00:37:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.077+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.077+0000 7ff8077fe700 10 client.41021.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.077+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818059770 osd=2 4 2024-02-18T00:37:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.077+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 2 6 2024-02-18T00:37:32.079 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 10 client.41021.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 7 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff81811d900 0x7ff81811fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 7 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 3 7 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818027be0 osd=3 4 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 10 client.41021.objecter _send_command 7 2024-02-18T00:37:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff818122940 con 0x7ff81811d900 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff816d9d700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff81811d900 0x7ff81811fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff81811d900 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.078+0000 7ff816d9d700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff81811d900 0x7ff81811fcf0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.079+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff818122940 con 0x7ff81811d900 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.079+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.079+0000 7ff8077fe700 10 client.41021.objecter _finish_command 7 = system:0 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.079+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818027be0 osd=3 4 2024-02-18T00:37:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.079+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 3 7 2024-02-18T00:37:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 10 client.41021.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 8 2024-02-18T00:37:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 8 2024-02-18T00:37:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818027be0 osd=3 3 2024-02-18T00:37:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 3 8 2024-02-18T00:37:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818027be0 osd=3 4 2024-02-18T00:37:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 10 client.41021.objecter _send_command 8 2024-02-18T00:37:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.094+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff81811aaf0 con 0x7ff81811d900 2024-02-18T00:37:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff81811aaf0 con 0x7ff81811d900 2024-02-18T00:37:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff8077fe700 10 client.41021.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818027be0 osd=3 4 2024-02-18T00:37:32.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 3 8 2024-02-18T00:37:32.097 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:37:32.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 10 client.41021.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 9 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff81811ac90 0x7ff8181201e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 9 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 4 9 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff81811aaf0 osd=4 4 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 10 client.41021.objecter _send_command 9 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff818119d00 con 0x7ff81811ac90 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.096+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff81811ac90 0x7ff8181201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.097+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff81811ac90 0x7ff8181201e0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.097+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff81811ac90 2024-02-18T00:37:32.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.097+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff818119d00 con 0x7ff81811ac90 2024-02-18T00:37:32.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.097+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:37:32.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.097+0000 7ff8077fe700 10 client.41021.objecter _finish_command 9 = system:0 2024-02-18T00:37:32.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.097+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff81811aaf0 osd=4 4 2024-02-18T00:37:32.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.098+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 4 9 2024-02-18T00:37:32.103 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:32.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 10 client.41021.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 10 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 10 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff81811aaf0 osd=4 3 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 4 10 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff81811aaf0 osd=4 4 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 10 client.41021.objecter _send_command 10 2024-02-18T00:37:32.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.114+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff818119d00 con 0x7ff81811ac90 2024-02-18T00:37:32.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.115+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff818119d00 con 0x7ff81811ac90 2024-02-18T00:37:32.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.115+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.115+0000 7ff8077fe700 10 client.41021.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.115+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff81811aaf0 osd=4 4 2024-02-18T00:37:32.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.115+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 4 10 2024-02-18T00:37:32.117 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:37:32.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 10 client.41021.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 11 2024-02-18T00:37:32.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff818120be0 0x7ff818121020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 11 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 5 11 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818119d00 osd=5 4 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 10 client.41021.objecter _send_command 11 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff818122690 con 0x7ff818120be0 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff815d9b700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff818120be0 0x7ff818121020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff818120be0 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.116+0000 7ff815d9b700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff818120be0 0x7ff818121020 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.117+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff818122690 con 0x7ff818120be0 2024-02-18T00:37:32.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.117+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:37:32.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.117+0000 7ff8077fe700 10 client.41021.objecter _finish_command 11 = system:0 2024-02-18T00:37:32.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.117+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818119d00 osd=5 4 2024-02-18T00:37:32.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.117+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 5 11 2024-02-18T00:37:32.133 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 10 client.41021.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 12 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 12 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818119d00 osd=5 3 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 5 12 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818119d00 osd=5 4 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 10 client.41021.objecter _send_command 12 2024-02-18T00:37:32.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.133+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff818121510 con 0x7ff818120be0 2024-02-18T00:37:32.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff818121510 con 0x7ff818120be0 2024-02-18T00:37:32.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff8077fe700 10 client.41021.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818119d00 osd=5 4 2024-02-18T00:37:32.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 5 12 2024-02-18T00:37:32.137 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:37:32.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 10 client.41021.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 13 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff818121b90 0x7ff818121ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 13 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 6 13 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818121510 osd=6 4 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 10 client.41021.objecter _send_command 13 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff818122550 con 0x7ff818121b90 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff816d9d700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff818121b90 0x7ff818121ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff818121b90 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.135+0000 7ff816d9d700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff818121b90 0x7ff818121ff0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.136+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff818122550 con 0x7ff818121b90 2024-02-18T00:37:32.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.136+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:37:32.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.136+0000 7ff8077fe700 10 client.41021.objecter _finish_command 13 = system:0 2024-02-18T00:37:32.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.136+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818121510 osd=6 4 2024-02-18T00:37:32.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.136+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 6 13 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 10 client.41021.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 14 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 14 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818121510 osd=6 3 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 6 14 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818121510 osd=6 4 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 10 client.41021.objecter _send_command 14 2024-02-18T00:37:32.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.152+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff818122550 con 0x7ff818121b90 2024-02-18T00:37:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff818122550 con 0x7ff818121b90 2024-02-18T00:37:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff8077fe700 10 client.41021.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818121510 osd=6 4 2024-02-18T00:37:32.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 6 14 2024-02-18T00:37:32.155 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:37:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 10 client.41021.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 15 2024-02-18T00:37:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6812/53025171 conn(0x7ff81812c5a0 0x7ff81812ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 15 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 7 15 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818122550 osd=7 4 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 10 client.41021.objecter _send_command 15 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff818131810 con 0x7ff81812c5a0 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6812/53025171 conn(0x7ff81812c5a0 0x7ff81812ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff8077fe700 10 client.41021.objecter ms_handle_connect 0x7ff81812c5a0 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.154+0000 7ff81659c700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6812/53025171 conn(0x7ff81812c5a0 0x7ff81812ea10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.155+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff818131810 con 0x7ff81812c5a0 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.155+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.155+0000 7ff8077fe700 10 client.41021.objecter _finish_command 15 = system:0 2024-02-18T00:37:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.155+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818122550 osd=7 4 2024-02-18T00:37:32.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.155+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 7 15 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 10 client.41021.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign -1 16 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 20 client.41021.objecter _get_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_remove -1 16 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 20 client.41021.objecter get_session s=0x7ff818122550 osd=7 3 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 15 client.41021.objecter _session_command_op_assign 7 16 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818122550 osd=7 4 2024-02-18T00:37:32.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 10 client.41021.objecter _send_command 16 2024-02-18T00:37:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.170+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff818131810 con 0x7ff81812c5a0 2024-02-18T00:37:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff8077fe700 1 -- 172.21.15.59:0/2910222106 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff818131810 con 0x7ff81812c5a0 2024-02-18T00:37:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff8077fe700 10 client.41021.objecter ms_dispatch 0x7ff818001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:32.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff8077fe700 10 client.41021.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff8077fe700 20 client.41021.objecter put_session s=0x7ff818122550 osd=7 4 2024-02-18T00:37:32.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff8077fe700 15 client.41021.objecter _session_command_op_remove 7 16 2024-02-18T00:37:32.173 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:37:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 librados: watch_flush enter 2024-02-18T00:37:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 librados: watch_flush exit 2024-02-18T00:37:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.0 2024-02-18T00:37:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8180fee60 msgr2=0x7ff8180ff240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8180fee60 0x7ff8180ff240 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818107110 osd=0 1 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.1 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8180598f0 msgr2=0x7ff818059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8180598f0 0x7ff818059cd0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818028710 osd=1 1 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.2 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff818028270 msgr2=0x7ff81811a1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff818028270 0x7ff81811a1a0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818059770 osd=2 1 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.3 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff81811d900 msgr2=0x7ff81811fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff81811d900 0x7ff81811fcf0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818027be0 osd=3 1 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.4 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff81811ac90 msgr2=0x7ff8181201e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff81811ac90 0x7ff8181201e0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff81811aaf0 osd=4 1 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.5 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff818120be0 msgr2=0x7ff818121020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff818120be0 0x7ff818121020 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff816d9d700 1 -- 172.21.15.59:0/2910222106 reap_dead start 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff816d9d700 1 -- 172.21.15.59:0/2910222106 reap_dead start 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818119d00 osd=5 1 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.6 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff818121b90 msgr2=0x7ff818121ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff818121b90 0x7ff818121ff0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff816d9d700 1 -- 172.21.15.59:0/2910222106 reap_dead start 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818121510 osd=6 1 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter close_session for osd.7 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6812/53025171 conn(0x7ff81812c5a0 msgr2=0x7ff81812ea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6812/53025171 conn(0x7ff81812c5a0 0x7ff81812ea10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter put_session s=0x7ff818122550 osd=7 1 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 20 client.41021.objecter shutdown clearing up homeless session... 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 10 client.41021.objecter successfully canceled tick 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff800062c90 msgr2=0x7ff800065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff800062c90 0x7ff800065150 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 rx=0x7ff80c005010 tx=0x7ff80c007f90).stop 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 msgr2=0x7ff81810e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff81810e660 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 rx=0x7ff8080091f0 tx=0x7ff808009cb0).stop 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 shutdown_connections 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.137:6812/53025171 conn(0x7ff81812c5a0 0x7ff81812ea10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8180fdbb0 0x7ff81810e660 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 --2- 172.21.15.59:0/2910222106 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff800062c90 0x7ff800065150 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 >> 172.21.15.59:0/2910222106 conn(0x7ff8180f3800 msgr2=0x7ff8180fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.172+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 shutdown_connections 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.173+0000 7ff81ce37700 1 -- 172.21.15.59:0/2910222106 wait complete. 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:32.173+0000 7ff81ce37700 1 librados: shutdown 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:37:32.383 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:32] "GET /metrics HTTP/1.1" 200 189152 "" "Prometheus/2.33.4" 2024-02-18T00:37:32.582 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:32.585 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:32.585 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:32.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:32.708 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.124443+0000 osd.4 (osd.4) 211 : cluster [DBG] 6.e scrub starts 2024-02-18T00:37:32.708 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.127336+0000 osd.4 (osd.4) 212 : cluster [DBG] 6.e scrub ok 2024-02-18T00:37:32.708 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.671654+0000 osd.6 (osd.6) 173 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.672695+0000 osd.6 (osd.6) 174 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:31.692948+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.693037+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e373: 8 total, 8 up, 7 in 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:31.694452+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:31.701020+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.813808+0000 osd.1 (osd.1) 189 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.817275+0000 osd.1 (osd.1) 190 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:31.884127+0000 mgr.y (mgr.24454) 357 : cluster [DBG] pgmap v547: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:32.709 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:32.021795+0000 mon.c (mon.2) 357 : audit [DBG] from='client.? 172.21.15.59:0/2910222106' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.124443+0000 osd.4 (osd.4) 211 : cluster [DBG] 6.e scrub starts 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.127336+0000 osd.4 (osd.4) 212 : cluster [DBG] 6.e scrub ok 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.671654+0000 osd.6 (osd.6) 173 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.672695+0000 osd.6 (osd.6) 174 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:31.692948+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.693037+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e373: 8 total, 8 up, 7 in 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:31.694452+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:31.701020+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.813808+0000 osd.1 (osd.1) 189 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.817275+0000 osd.1 (osd.1) 190 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:37:33.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:31.884127+0000 mgr.y (mgr.24454) 357 : cluster [DBG] pgmap v547: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:33.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:32.021795+0000 mon.c (mon.2) 357 : audit [DBG] from='client.? 172.21.15.59:0/2910222106' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:33.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.124443+0000 osd.4 (osd.4) 211 : cluster [DBG] 6.e scrub starts 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.127336+0000 osd.4 (osd.4) 212 : cluster [DBG] 6.e scrub ok 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.671654+0000 osd.6 (osd.6) 173 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.672695+0000 osd.6 (osd.6) 174 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:31.692948+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi059-82882-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.693037+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e373: 8 total, 8 up, 7 in 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:31.694452+0000 mon.c (mon.2) 356 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:31.701020+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.813808+0000 osd.1 (osd.1) 189 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.817275+0000 osd.1 (osd.1) 190 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:31.884127+0000 mgr.y (mgr.24454) 357 : cluster [DBG] pgmap v547: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:32.021795+0000 mon.c (mon.2) 357 : audit [DBG] from='client.? 172.21.15.59:0/2910222106' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:33.400 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-02-18T00:37:33.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:37:33.453 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:33.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:33.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:33.556 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:33.585 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:34.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:32.400754+0000 osd.2 (osd.2) 161 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:37:34.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:32.401615+0000 osd.2 (osd.2) 162 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:37:34.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:32.763853+0000 osd.1 (osd.1) 191 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:37:34.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:32.766196+0000 osd.1 (osd.1) 192 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:37:34.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:32.952502+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.145582+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e374: 8 total, 8 up, 7 in 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:33.146600+0000 mon.a (mon.0) 2361 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:33.658420+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:33.658495+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.658538+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e375: 8 total, 8 up, 7 in 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:33.666340+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:33.670057+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:32.400754+0000 osd.2 (osd.2) 161 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:32.401615+0000 osd.2 (osd.2) 162 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:37:34.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:32.763853+0000 osd.1 (osd.1) 191 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:32.766196+0000 osd.1 (osd.1) 192 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:32.952502+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.145582+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e374: 8 total, 8 up, 7 in 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:33.146600+0000 mon.a (mon.0) 2361 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:33.658420+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:33.658495+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.658538+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e375: 8 total, 8 up, 7 in 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:33.666340+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.145 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:33.670057+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.251 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:34.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:32.400754+0000 osd.2 (osd.2) 161 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:37:34.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:32.401615+0000 osd.2 (osd.2) 162 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:32.763853+0000 osd.1 (osd.1) 191 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:32.766196+0000 osd.1 (osd.1) 192 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:32.952502+0000 mon.c (mon.2) 358 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.145582+0000 mon.a (mon.0) 2360 : cluster [DBG] osdmap e374: 8 total, 8 up, 7 in 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:33.146600+0000 mon.a (mon.0) 2361 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:33.658420+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi059-82882-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:33.658495+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.658538+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e375: 8 total, 8 up, 7 in 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:33.666340+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:33.670057+0000 mon.a (mon.0) 2365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.116243+0000 osd.4 (osd.4) 213 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.117169+0000 osd.4 (osd.4) 214 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.615898+0000 osd.5 (osd.5) 157 : cluster [DBG] 6.a scrub starts 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.616869+0000 osd.5 (osd.5) 158 : cluster [DBG] 6.a scrub ok 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.617891+0000 osd.6 (osd.6) 175 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.619404+0000 osd.6 (osd.6) 176 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:33.884638+0000 mgr.y (mgr.24454) 358 : cluster [DBG] pgmap v550: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:34.119817+0000 osd.4 (osd.4) 215 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:34.122755+0000 osd.4 (osd.4) 216 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:34.655183+0000 mon.a (mon.0) 2366 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:34.660837+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]': finished 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:34.660939+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e376: 8 total, 8 up, 7 in 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:34.689318+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.961 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:34.689600+0000 mon.a (mon.0) 2369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:34.690097+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:34.690448+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.116243+0000 osd.4 (osd.4) 213 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.117169+0000 osd.4 (osd.4) 214 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.615898+0000 osd.5 (osd.5) 157 : cluster [DBG] 6.a scrub starts 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.616869+0000 osd.5 (osd.5) 158 : cluster [DBG] 6.a scrub ok 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.617891+0000 osd.6 (osd.6) 175 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.619404+0000 osd.6 (osd.6) 176 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:33.884638+0000 mgr.y (mgr.24454) 358 : cluster [DBG] pgmap v550: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:34.119817+0000 osd.4 (osd.4) 215 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:34.122755+0000 osd.4 (osd.4) 216 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:37:34.962 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:34.655183+0000 mon.a (mon.0) 2366 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:34.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:34.660837+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]': finished 2024-02-18T00:37:34.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:34.660939+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e376: 8 total, 8 up, 7 in 2024-02-18T00:37:34.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:34.689318+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:34.689600+0000 mon.a (mon.0) 2369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:34.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:34.690097+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:34.963 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:34.690448+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:35.236 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:35.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:35.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.116243+0000 osd.4 (osd.4) 213 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.117169+0000 osd.4 (osd.4) 214 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.615898+0000 osd.5 (osd.5) 157 : cluster [DBG] 6.a scrub starts 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.616869+0000 osd.5 (osd.5) 158 : cluster [DBG] 6.a scrub ok 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.617891+0000 osd.6 (osd.6) 175 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.619404+0000 osd.6 (osd.6) 176 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:33.884638+0000 mgr.y (mgr.24454) 358 : cluster [DBG] pgmap v550: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:34.119817+0000 osd.4 (osd.4) 215 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:34.122755+0000 osd.4 (osd.4) 216 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:34.655183+0000 mon.a (mon.0) 2366 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:34.660837+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]': finished 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:34.660939+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e376: 8 total, 8 up, 7 in 2024-02-18T00:37:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:34.689318+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:34.689600+0000 mon.a (mon.0) 2369 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:34.690097+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:35.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:34.690448+0000 mon.a (mon.0) 2370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-47"}]: dispatch 2024-02-18T00:37:35.340 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:35.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:35.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:35.600 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:35.622 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:37:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:37:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:37:35.622 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:37:35.622 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:37:35.630 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:35.653 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:35.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:35.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:35.785 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:35.934 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:35.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:35.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:36.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:36.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:34.858925+0000 osd.3 (osd.3) 207 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:37:36.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:34.860210+0000 osd.3 (osd.3) 208 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:37:36.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:35.661824+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e377: 8 total, 8 up, 7 in 2024-02-18T00:37:36.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:35.672412+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:36.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:35.673066+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:36.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:34.858925+0000 osd.3 (osd.3) 207 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:37:36.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:34.860210+0000 osd.3 (osd.3) 208 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:37:36.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:35.661824+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e377: 8 total, 8 up, 7 in 2024-02-18T00:37:36.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:35.672412+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:36.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:35.673066+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:34.858925+0000 osd.3 (osd.3) 207 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:37:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:34.860210+0000 osd.3 (osd.3) 208 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:37:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:35.661824+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e377: 8 total, 8 up, 7 in 2024-02-18T00:37:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:35.672412+0000 mon.c (mon.2) 362 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:35.673066+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:36.724 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:37:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 -- 172.21.15.59:0/4207133859 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40fe510 msgr2=0x7f08b40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 --2- 172.21.15.59:0/4207133859 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40fe510 0x7f08b40f7d40 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 rx=0x7f089c004870 tx=0x7f089c009cb0).stop 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 -- 172.21.15.59:0/4207133859 shutdown_connections 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 --2- 172.21.15.59:0/4207133859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b40fdbb0 0x7f08b40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 --2- 172.21.15.59:0/4207133859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b40f8420 0x7f08b4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 --2- 172.21.15.59:0/4207133859 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40fe510 0x7f08b40f7d40 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 -- 172.21.15.59:0/4207133859 >> 172.21.15.59:0/4207133859 conn(0x7f08b40f3800 msgr2=0x7f08b40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 -- 172.21.15.59:0/4207133859 shutdown_connections 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.436+0000 7f08b94d9700 1 -- 172.21.15.59:0/4207133859 wait complete. 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 librados: starting msgr at 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 librados: starting objecter 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 Processor -- start 2024-02-18T00:37:37.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 -- start start 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 librados: setting wanted keys 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 librados: calling monclient init 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40f8420 0x7f08b406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b40fdbb0 0x7f08b406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b40fe510 0x7f08b406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4116560 con 0x7f08b40fe510 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b41166d0 con 0x7f08b40fdbb0 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b94d9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4116890 con 0x7f08b40f8420 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40f8420 0x7f08b406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b40fe510 0x7f08b406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40f8420 0x7f08b406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43176/0 (socket says 172.21.15.59:43176) 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b2ffd700 1 -- 172.21.15.59:0/3775472183 learned_addr learned my addr 172.21.15.59:0/3775472183 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.437+0000 7f08b27fc700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b40fdbb0 0x7f08b406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b37fe700 1 -- 172.21.15.59:0/3775472183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40f8420 msgr2=0x7f08b406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b37fe700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40f8420 0x7f08b406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b37fe700 1 -- 172.21.15.59:0/3775472183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b40fdbb0 msgr2=0x7f08b406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b37fe700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b40fdbb0 0x7f08b406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b37fe700 1 -- 172.21.15.59:0/3775472183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4005040 con 0x7f08b40fe510 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b37fe700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b40fe510 0x7f08b406bab0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 rx=0x7f08a40048f0 tx=0x7f08a4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08abfff700 10 client.?.objecter ms_handle_connect 0x7f08b40fe510 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08abfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08abfff700 1 -- 172.21.15.59:0/3775472183 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400e030 con 0x7f08b40fe510 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c005040 con 0x7f08b40fe510 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08abfff700 1 -- 172.21.15.59:0/3775472183 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a400f040 con 0x7f08b40fe510 2024-02-18T00:37:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b94d9700 10 client.31224.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b4116f50 con 0x7f08b40fe510 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08b94d9700 1 librados: init done 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.438+0000 7f08abfff700 1 -- 172.21.15.59:0/3775472183 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40147c0 con 0x7f08b40fe510 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.439+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b41070a0 con 0x7f08b40fe510 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.439+0000 7f08abfff700 1 -- 172.21.15.59:0/3775472183 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f08a4010500 con 0x7f08b40fe510 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08abfff700 1 --2- 172.21.15.59:0/3775472183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08a0062c90 0x7f08a0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08abfff700 1 -- 172.21.15.59:0/3775472183 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(378..378 src has 180..378) v4 ==== 8115+0+0 (secure 0 0 0) 0x7f08a407d180 con 0x7f08b40fe510 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08abfff700 10 client.31224.objecter ms_dispatch 0x7f08b4001550 osd_map(378..378 src has 180..378) v4 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08abfff700 3 client.31224.objecter handle_osd_map got epochs [378,378] > 0 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08abfff700 3 client.31224.objecter handle_osd_map decoding full epoch 378 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08b2ffd700 1 --2- 172.21.15.59:0/3775472183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08a0062c90 0x7f08a0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08abfff700 20 client.31224.objecter dump_active .. 0 homeless 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08abfff700 10 client.31224.objecter ms_handle_connect 0x7f08a0062c90 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.440+0000 7f08b2ffd700 1 --2- 172.21.15.59:0/3775472183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08a0062c90 0x7f08a0065150 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 rx=0x7f08ac0023e0 tx=0x7f08ac00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.444+0000 7f08abfff700 1 -- 172.21.15.59:0/3775472183 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f08a4050650 con 0x7f08b40fe510 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:35.211304+0000 osd.0 (osd.0) 179 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:35.212723+0000 osd.0 (osd.0) 180 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:35.837973+0000 osd.3 (osd.3) 209 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:35.841041+0000 osd.3 (osd.3) 210 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:35.885436+0000 mgr.y (mgr.24454) 359 : cluster [DBG] pgmap v553: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:36.118145+0000 osd.4 (osd.4) 217 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:36.119286+0000 osd.4 (osd.4) 218 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:36.666303+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:36.666455+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e378: 8 total, 8 up, 7 in 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:36.667923+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:36.668965+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:36.673721+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:37.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:36.674545+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:35.211304+0000 osd.0 (osd.0) 179 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:35.212723+0000 osd.0 (osd.0) 180 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:35.837973+0000 osd.3 (osd.3) 209 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:35.841041+0000 osd.3 (osd.3) 210 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:35.885436+0000 mgr.y (mgr.24454) 359 : cluster [DBG] pgmap v553: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:36.118145+0000 osd.4 (osd.4) 217 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:36.119286+0000 osd.4 (osd.4) 218 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:36.666303+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:36.666455+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e378: 8 total, 8 up, 7 in 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:36.667923+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:37.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:36.668965+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:36.673721+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:36.674545+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:35.211304+0000 osd.0 (osd.0) 179 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:35.212723+0000 osd.0 (osd.0) 180 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:35.837973+0000 osd.3 (osd.3) 209 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:35.841041+0000 osd.3 (osd.3) 210 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:35.885436+0000 mgr.y (mgr.24454) 359 : cluster [DBG] pgmap v553: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:36.118145+0000 osd.4 (osd.4) 217 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:37:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:36.119286+0000 osd.4 (osd.4) 218 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:37:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:36.666303+0000 mon.a (mon.0) 2373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:36.666455+0000 mon.a (mon.0) 2374 : cluster [DBG] osdmap e378: 8 total, 8 up, 7 in 2024-02-18T00:37:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:36.667923+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:36.668965+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:36.673721+0000 mon.c (mon.2) 364 : audit [INF] from='client.? 172.21.15.59:0/1940889306' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:37.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:36.674545+0000 mon.a (mon.0) 2376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]: dispatch 2024-02-18T00:37:37.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.629+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f08b405fa90 con 0x7f08b40fe510 2024-02-18T00:37:37.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.668+0000 7f08abfff700 1 -- 172.21.15.59:0/3775472183 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v379) v1 ==== 101+0+0 (secure 0 0 0) 0x7f08a4050240 con 0x7f08b40fe510 2024-02-18T00:37:37.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 10 librados: watch_flush enter 2024-02-18T00:37:37.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 10 librados: watch_flush exit 2024-02-18T00:37:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 20 client.31224.objecter shutdown clearing up homeless session... 2024-02-18T00:37:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 10 client.31224.objecter successfully canceled tick 2024-02-18T00:37:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08a0062c90 msgr2=0x7f08a0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 1 --2- 172.21.15.59:0/3775472183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08a0062c90 0x7f08a0065150 secure :-1 s=READY pgs=3091 cs=0 l=1 rev1=1 rx=0x7f08ac0023e0 tx=0x7f08ac00b040).stop 2024-02-18T00:37:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b40fe510 msgr2=0x7f08b406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:37.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.674+0000 7f08b94d9700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b40fe510 0x7f08b406bab0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 rx=0x7f08a40048f0 tx=0x7f08a4009cb0).stop 2024-02-18T00:37:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.676+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 shutdown_connections 2024-02-18T00:37:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.676+0000 7f08b94d9700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b40fe510 0x7f08b406bab0 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.676+0000 7f08b94d9700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08b40f8420 0x7f08b406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.676+0000 7f08b94d9700 1 --2- 172.21.15.59:0/3775472183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08a0062c90 0x7f08a0065150 secure :-1 s=CLOSED pgs=3091 cs=0 l=1 rev1=1 rx=0x7f08ac0023e0 tx=0x7f08ac00b040).stop 2024-02-18T00:37:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.676+0000 7f08b94d9700 1 --2- 172.21.15.59:0/3775472183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b40fdbb0 0x7f08b406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.676+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 >> 172.21.15.59:0/3775472183 conn(0x7f08b40f3800 msgr2=0x7f08b40f53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:37.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.679+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 shutdown_connections 2024-02-18T00:37:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.683+0000 7f08b94d9700 1 -- 172.21.15.59:0/3775472183 wait complete. 2024-02-18T00:37:37.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:37.683+0000 7f08b94d9700 1 librados: shutdown 2024-02-18T00:37:37.693 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:37:38.049 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:38.053 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:38.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:38.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-18T00:37:38.154 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:38.186 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:38.208 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:38.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:38.211 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:38.278 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:38.312 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:38.319 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:38.341 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:38.360 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:38.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:38.363 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:38.464 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:38.490 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:36.614693+0000 osd.6 (osd.6) 177 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:36.615563+0000 osd.6 (osd.6) 178 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:36.815871+0000 osd.1 (osd.1) 193 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:36.817715+0000 osd.1 (osd.1) 194 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.278135+0000 osd.0 (osd.0) 181 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.281670+0000 osd.0 (osd.0) 182 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.475807+0000 osd.2 (osd.2) 163 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.476672+0000 osd.2 (osd.2) 164 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.631327+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? 172.21.15.59:0/3775472183' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.663784+0000 mon.a (mon.0) 2378 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.669541+0000 mon.a (mon.0) 2379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:38.491 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.669728+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.669804+0000 mon.a (mon.0) 2381 : audit [INF] from='client.? 172.21.15.59:0/3775472183' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.669837+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e379: 8 total, 8 up, 7 in 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.698043+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.698406+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.704846+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.705294+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.705762+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.705969+0000 mon.a (mon.0) 2385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.706383+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:38.492 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:37.706604+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:38.493 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:36.614693+0000 osd.6 (osd.6) 177 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:36.615563+0000 osd.6 (osd.6) 178 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:36.815871+0000 osd.1 (osd.1) 193 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:36.817715+0000 osd.1 (osd.1) 194 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.278135+0000 osd.0 (osd.0) 181 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.281670+0000 osd.0 (osd.0) 182 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.475807+0000 osd.2 (osd.2) 163 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.476672+0000 osd.2 (osd.2) 164 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.631327+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? 172.21.15.59:0/3775472183' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.663784+0000 mon.a (mon.0) 2378 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.669541+0000 mon.a (mon.0) 2379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.669728+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.669804+0000 mon.a (mon.0) 2381 : audit [INF] from='client.? 172.21.15.59:0/3775472183' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.669837+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e379: 8 total, 8 up, 7 in 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.698043+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.698406+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.704846+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.705294+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.705762+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.705969+0000 mon.a (mon.0) 2385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.706383+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:37.706604+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:36.614693+0000 osd.6 (osd.6) 177 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:37:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:36.615563+0000 osd.6 (osd.6) 178 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:36.815871+0000 osd.1 (osd.1) 193 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:36.817715+0000 osd.1 (osd.1) 194 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.278135+0000 osd.0 (osd.0) 181 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.281670+0000 osd.0 (osd.0) 182 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.475807+0000 osd.2 (osd.2) 163 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.476672+0000 osd.2 (osd.2) 164 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.631327+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? 172.21.15.59:0/3775472183' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.663784+0000 mon.a (mon.0) 2378 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.669541+0000 mon.a (mon.0) 2379 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.669728+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi059-82882-63"}]': finished 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.669804+0000 mon.a (mon.0) 2381 : audit [INF] from='client.? 172.21.15.59:0/3775472183' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.669837+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e379: 8 total, 8 up, 7 in 2024-02-18T00:37:38.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.698043+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:38.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.698406+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:37:38.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.704846+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.705294+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.705762+0000 mon.c (mon.2) 367 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.705969+0000 mon.a (mon.0) 2385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:38.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.706383+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:38.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:37.706604+0000 mon.a (mon.0) 2386 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:38.510 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:38.513 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:38.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:38.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:38.646 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:38.801 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:38.808 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:38.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:38.909 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:38.939 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.617389+0000 osd.6 (osd.6) 179 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:37:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.618661+0000 osd.6 (osd.6) 180 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:37:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.725866+0000 osd.5 (osd.5) 159 : cluster [DBG] 2.20 deep-scrub starts 2024-02-18T00:37:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.726856+0000 osd.5 (osd.5) 160 : cluster [DBG] 2.20 deep-scrub ok 2024-02-18T00:37:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.773310+0000 osd.1 (osd.1) 195 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:37:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.774400+0000 osd.1 (osd.1) 196 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.847648+0000 osd.3 (osd.3) 211 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.848519+0000 osd.3 (osd.3) 212 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:37.885841+0000 mgr.y (mgr.24454) 360 : cluster [DBG] pgmap v556: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:38.432284+0000 osd.2 (osd.2) 165 : cluster [DBG] 5.d scrub starts 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:38.433872+0000 osd.2 (osd.2) 166 : cluster [DBG] 5.d scrub ok 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:38.674316+0000 mon.a (mon.0) 2387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:38.674401+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:38.674463+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e380: 8 total, 8 up, 7 in 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:38.678699+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:38.679049+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:38.679860+0000 mon.a (mon.0) 2390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:38.680332+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.617389+0000 osd.6 (osd.6) 179 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.618661+0000 osd.6 (osd.6) 180 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.725866+0000 osd.5 (osd.5) 159 : cluster [DBG] 2.20 deep-scrub starts 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.726856+0000 osd.5 (osd.5) 160 : cluster [DBG] 2.20 deep-scrub ok 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.773310+0000 osd.1 (osd.1) 195 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.774400+0000 osd.1 (osd.1) 196 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.847648+0000 osd.3 (osd.3) 211 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.848519+0000 osd.3 (osd.3) 212 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:37.885841+0000 mgr.y (mgr.24454) 360 : cluster [DBG] pgmap v556: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:38.432284+0000 osd.2 (osd.2) 165 : cluster [DBG] 5.d scrub starts 2024-02-18T00:37:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:38.433872+0000 osd.2 (osd.2) 166 : cluster [DBG] 5.d scrub ok 2024-02-18T00:37:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:38.674316+0000 mon.a (mon.0) 2387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:38.674401+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:38.674463+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e380: 8 total, 8 up, 7 in 2024-02-18T00:37:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:38.678699+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:38.679049+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:38.679860+0000 mon.a (mon.0) 2390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:38.680332+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.617389+0000 osd.6 (osd.6) 179 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:37:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.618661+0000 osd.6 (osd.6) 180 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:37:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.725866+0000 osd.5 (osd.5) 159 : cluster [DBG] 2.20 deep-scrub starts 2024-02-18T00:37:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.726856+0000 osd.5 (osd.5) 160 : cluster [DBG] 2.20 deep-scrub ok 2024-02-18T00:37:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.773310+0000 osd.1 (osd.1) 195 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:37:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.774400+0000 osd.1 (osd.1) 196 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.847648+0000 osd.3 (osd.3) 211 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.848519+0000 osd.3 (osd.3) 212 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:37.885841+0000 mgr.y (mgr.24454) 360 : cluster [DBG] pgmap v556: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:38.432284+0000 osd.2 (osd.2) 165 : cluster [DBG] 5.d scrub starts 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:38.433872+0000 osd.2 (osd.2) 166 : cluster [DBG] 5.d scrub ok 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:38.674316+0000 mon.a (mon.0) 2387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:38.674401+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi059-82882-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:38.674463+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e380: 8 total, 8 up, 7 in 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:38.678699+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:38.679049+0000 mon.c (mon.2) 370 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:38.679860+0000 mon.a (mon.0) 2390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:39.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:38.680332+0000 mon.a (mon.0) 2391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:39.833 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:40.192 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:40.193 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:40.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:40.327 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:40.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:38.586558+0000 osd.6 (osd.6) 181 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:37:40.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:38.588185+0000 osd.6 (osd.6) 182 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:37:40.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:38.681610+0000 osd.5 (osd.5) 161 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:38.687125+0000 osd.5 (osd.5) 162 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:39.124120+0000 osd.4 (osd.4) 219 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:39.126391+0000 osd.4 (osd.4) 220 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:39.383870+0000 osd.2 (osd.2) 167 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:39.385670+0000 osd.2 (osd.2) 168 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.116451+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]': finished 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:40.116534+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e381: 8 total, 8 up, 7 in 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.117122+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]: dispatch 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.118005+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]: dispatch 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.123925+0000 mon.a (mon.0) 2395 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:40.458 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:38.586558+0000 osd.6 (osd.6) 181 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:38.588185+0000 osd.6 (osd.6) 182 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:38.681610+0000 osd.5 (osd.5) 161 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:38.687125+0000 osd.5 (osd.5) 162 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:39.124120+0000 osd.4 (osd.4) 219 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:39.126391+0000 osd.4 (osd.4) 220 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:39.383870+0000 osd.2 (osd.2) 167 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:39.385670+0000 osd.2 (osd.2) 168 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.116451+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]': finished 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:40.116534+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e381: 8 total, 8 up, 7 in 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.117122+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]: dispatch 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.118005+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]: dispatch 2024-02-18T00:37:40.459 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.123925+0000 mon.a (mon.0) 2395 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:40.476 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:40.478 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:40.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:40.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:38.586558+0000 osd.6 (osd.6) 181 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:38.588185+0000 osd.6 (osd.6) 182 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:38.681610+0000 osd.5 (osd.5) 161 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:38.687125+0000 osd.5 (osd.5) 162 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:39.124120+0000 osd.4 (osd.4) 219 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:39.126391+0000 osd.4 (osd.4) 220 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:39.383870+0000 osd.2 (osd.2) 167 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:39.385670+0000 osd.2 (osd.2) 168 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.116451+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-49"}]': finished 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:40.116534+0000 mon.a (mon.0) 2393 : cluster [DBG] osdmap e381: 8 total, 8 up, 7 in 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.117122+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]: dispatch 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.118005+0000 mon.a (mon.0) 2394 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]: dispatch 2024-02-18T00:37:40.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.123925+0000 mon.a (mon.0) 2395 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:40.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:40.608 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:41.087 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:41.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:41.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:41.191 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:41.219 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:41.318 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:41.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:41.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:41.322 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:37:41.387 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:39.636195+0000 osd.5 (osd.5) 163 : cluster [DBG] 25.f scrub starts 2024-02-18T00:37:41.387 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:39.639228+0000 osd.5 (osd.5) 164 : cluster [DBG] 25.f scrub ok 2024-02-18T00:37:41.387 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:39.886229+0000 mgr.y (mgr.24454) 361 : cluster [DBG] pgmap v559: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:41.387 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:40.672300+0000 mon.a (mon.0) 2396 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.681461+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.681556+0000 mon.a (mon.0) 2398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]': finished 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:40.681630+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e382: 8 total, 8 up, 7 in 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.722057+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:40.722381+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:39.636195+0000 osd.5 (osd.5) 163 : cluster [DBG] 25.f scrub starts 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:39.639228+0000 osd.5 (osd.5) 164 : cluster [DBG] 25.f scrub ok 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:39.886229+0000 mgr.y (mgr.24454) 361 : cluster [DBG] pgmap v559: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:41.388 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:40.672300+0000 mon.a (mon.0) 2396 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:41.389 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.681461+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:41.389 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.681556+0000 mon.a (mon.0) 2398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]': finished 2024-02-18T00:37:41.389 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:40.681630+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e382: 8 total, 8 up, 7 in 2024-02-18T00:37:41.389 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.722057+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:41.389 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:40.722381+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:41.423 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:41.452 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:37:41.453 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:37:41.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:39.636195+0000 osd.5 (osd.5) 163 : cluster [DBG] 25.f scrub starts 2024-02-18T00:37:41.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:39.639228+0000 osd.5 (osd.5) 164 : cluster [DBG] 25.f scrub ok 2024-02-18T00:37:41.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:39.886229+0000 mgr.y (mgr.24454) 361 : cluster [DBG] pgmap v559: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:41.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:40.672300+0000 mon.a (mon.0) 2396 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:37:41.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.681461+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi059-82882-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:41.453 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.681556+0000 mon.a (mon.0) 2398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-49", "mode": "writeback"}]': finished 2024-02-18T00:37:41.454 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:40.681630+0000 mon.a (mon.0) 2399 : cluster [DBG] osdmap e382: 8 total, 8 up, 7 in 2024-02-18T00:37:41.454 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.722057+0000 mon.c (mon.2) 372 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:41.454 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:40.722381+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:37:41.455 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:41.477 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:41.479 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:41.480 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:41.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:41.608 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:40.561254+0000 osd.6 (osd.6) 183 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:37:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:40.562300+0000 osd.6 (osd.6) 184 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:37:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:40.706429+0000 osd.1 (osd.1) 197 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:37:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:40.707355+0000 osd.1 (osd.1) 198 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:37:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:41.212191+0000 mgr.y (mgr.24454) 362 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:41.682876+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:37:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.682956+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e383: 8 total, 8 up, 7 in 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:41.686876+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:41.687606+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:40.561254+0000 osd.6 (osd.6) 183 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:40.562300+0000 osd.6 (osd.6) 184 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:40.706429+0000 osd.1 (osd.1) 197 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:40.707355+0000 osd.1 (osd.1) 198 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:41.212191+0000 mgr.y (mgr.24454) 362 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:41.682876+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.682956+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e383: 8 total, 8 up, 7 in 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:41.686876+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:41.687606+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:42.358 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:42.361 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:42.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:42.463 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:42.489 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:37:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:42] "GET /metrics HTTP/1.1" 200 189273 "" "Prometheus/2.33.4" 2024-02-18T00:37:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:40.561254+0000 osd.6 (osd.6) 183 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:40.562300+0000 osd.6 (osd.6) 184 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:40.706429+0000 osd.1 (osd.1) 197 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:40.707355+0000 osd.1 (osd.1) 198 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:41.212191+0000 mgr.y (mgr.24454) 362 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:41.682876+0000 mon.a (mon.0) 2401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.682956+0000 mon.a (mon.0) 2402 : cluster [DBG] osdmap e383: 8 total, 8 up, 7 in 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:41.686876+0000 mon.c (mon.2) 373 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:41.687606+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:37:42.697 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:42.700 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:42.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:42.801 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:42.833 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:42.854 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:42.856 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:42.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:42.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.948+0000 7f46ca189700 1 -- 172.21.15.59:0/428114521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c4102270 msgr2=0x7f46c4104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:42.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.948+0000 7f46ca189700 1 --2- 172.21.15.59:0/428114521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c4102270 0x7f46c4104660 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 rx=0x7f46bc009be0 tx=0x7f46bc004d10).stop 2024-02-18T00:37:42.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 -- 172.21.15.59:0/428114521 shutdown_connections 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 --2- 172.21.15.59:0/428114521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c40f8d10 0x7f46c40f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 --2- 172.21.15.59:0/428114521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c4102270 0x7f46c4104660 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 --2- 172.21.15.59:0/428114521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c40f9670 0x7f46c4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 -- 172.21.15.59:0/428114521 >> 172.21.15.59:0/428114521 conn(0x7f46c40f2820 msgr2=0x7f46c40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 -- 172.21.15.59:0/428114521 shutdown_connections 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 -- 172.21.15.59:0/428114521 wait complete. 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 librados: starting msgr at 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 librados: starting objecter 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 Processor -- start 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 -- start start 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 librados: setting wanted keys 2024-02-18T00:37:42.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.949+0000 7f46ca189700 1 librados: calling monclient init 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46ca189700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c40f8d10 0x7f46c40f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46ca189700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c40f9670 0x7f46c40f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46ca189700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c4102270 0x7f46c40f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46ca189700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4116450 con 0x7f46c4102270 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46ca189700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41165c0 con 0x7f46c40f9670 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46ca189700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4116780 con 0x7f46c40f8d10 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c4102270 0x7f46c40f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c40f8d10 0x7f46c40f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c4102270 0x7f46c40f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42222/0 (socket says 172.21.15.59:42222) 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c2ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c40f9670 0x7f46c40f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c40f8d10 0x7f46c40f6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59632/0 (socket says 172.21.15.59:59632) 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c3fff700 1 -- 172.21.15.59:0/2287857513 learned_addr learned my addr 172.21.15.59:0/2287857513 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 -- 172.21.15.59:0/2287857513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c40f9670 msgr2=0x7f46c40f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:42.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c40f9670 0x7f46c40f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 -- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c4102270 msgr2=0x7f46c40f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c4102270 0x7f46c40f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 -- 172.21.15.59:0/2287857513 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b8005040 con 0x7f46c40f8d10 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c3fff700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c4102270 0x7f46c40f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c37fe700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c40f8d10 0x7f46c40f6cc0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 rx=0x7f46b4002780 tx=0x7f46b4009e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c0ff9700 10 client.?.objecter ms_handle_connect 0x7f46c40f8d10 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c0ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c0ff9700 1 -- 172.21.15.59:0/2287857513 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4007b40 con 0x7f46c40f8d10 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c0ff9700 1 -- 172.21.15.59:0/2287857513 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46b400f040 con 0x7f46c40f8d10 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc0097c0 con 0x7f46c40f8d10 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.950+0000 7f46c0ff9700 1 -- 172.21.15.59:0/2287857513 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4014850 con 0x7f46c40f8d10 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.951+0000 7f46ca189700 10 client.41039.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.951+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c4116dc0 con 0x7f46c40f8d10 2024-02-18T00:37:42.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.951+0000 7f46ca189700 1 librados: init done 2024-02-18T00:37:42.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c405fa60 con 0x7f46c40f8d10 2024-02-18T00:37:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c0ff9700 1 -- 172.21.15.59:0/2287857513 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f46b4007ca0 con 0x7f46c40f8d10 2024-02-18T00:37:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c0ff9700 1 --2- 172.21.15.59:0/2287857513 >> v2:172.21.15.59:6800/3721891797 conn(0x7f46ac062c90 0x7f46ac065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c0ff9700 1 -- 172.21.15.59:0/2287857513 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(384..384 src has 180..384) v4 ==== 8410+0+0 (secure 0 0 0) 0x7f46b407d310 con 0x7f46c40f8d10 2024-02-18T00:37:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c0ff9700 10 client.41039.objecter ms_dispatch 0x7f46c4001550 osd_map(384..384 src has 180..384) v4 2024-02-18T00:37:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c0ff9700 3 client.41039.objecter handle_osd_map got epochs [384,384] > 0 2024-02-18T00:37:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c0ff9700 3 client.41039.objecter handle_osd_map decoding full epoch 384 2024-02-18T00:37:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c2ffd700 1 --2- 172.21.15.59:0/2287857513 >> v2:172.21.15.59:6800/3721891797 conn(0x7f46ac062c90 0x7f46ac065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.952+0000 7f46c0ff9700 20 client.41039.objecter dump_active .. 0 homeless 2024-02-18T00:37:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.953+0000 7f46c0ff9700 10 client.41039.objecter ms_handle_connect 0x7f46ac062c90 2024-02-18T00:37:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.953+0000 7f46c2ffd700 1 --2- 172.21.15.59:0/2287857513 >> v2:172.21.15.59:6800/3721891797 conn(0x7f46ac062c90 0x7f46ac065150 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 rx=0x7f46c40f8460 tx=0x7f46b800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:42.958 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:42.957+0000 7f46c0ff9700 1 -- 172.21.15.59:0/2287857513 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f46b4050680 con 0x7f46c40f8d10 2024-02-18T00:37:42.987 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:43.007 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:43.010 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:43.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:43.111 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:43.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.125+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46c40f8210 con 0x7f46ac062c90 2024-02-18T00:37:43.145 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:43.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.144+0000 7f46c0ff9700 1 -- 172.21.15.59:0/2287857513 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+598099 (secure 0 0 0) 0x7f46c40f8210 con 0x7f46ac062c90 2024-02-18T00:37:43.154 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 10 librados: watch_flush enter 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 10 librados: watch_flush exit 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 20 client.41039.objecter shutdown clearing up homeless session... 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 10 client.41039.objecter successfully canceled tick 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 >> v2:172.21.15.59:6800/3721891797 conn(0x7f46ac062c90 msgr2=0x7f46ac065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 --2- 172.21.15.59:0/2287857513 >> v2:172.21.15.59:6800/3721891797 conn(0x7f46ac062c90 0x7f46ac065150 secure :-1 s=READY pgs=3093 cs=0 l=1 rev1=1 rx=0x7f46c40f8460 tx=0x7f46b800b040).stop 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c40f8d10 msgr2=0x7f46c40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c40f8d10 0x7f46c40f6cc0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 rx=0x7f46b4002780 tx=0x7f46b4009e40).stop 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 shutdown_connections 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46c4102270 0x7f46c40f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:43.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f46c40f8d10 0x7f46c40f6cc0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 --2- 172.21.15.59:0/2287857513 >> v2:172.21.15.59:6800/3721891797 conn(0x7f46ac062c90 0x7f46ac065150 unknown :-1 s=CLOSED pgs=3093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 --2- 172.21.15.59:0/2287857513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46c40f9670 0x7f46c40f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 >> 172.21.15.59:0/2287857513 conn(0x7f46c40f2820 msgr2=0x7f46c40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 shutdown_connections 2024-02-18T00:37:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 -- 172.21.15.59:0/2287857513 wait complete. 2024-02-18T00:37:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:43.149+0000 7f46ca189700 1 librados: shutdown 2024-02-18T00:37:43.162 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:37:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.573976+0000 osd.6 (osd.6) 185 : cluster [DBG] 62.17 scrub starts 2024-02-18T00:37:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.575809+0000 osd.6 (osd.6) 186 : cluster [DBG] 62.17 scrub ok 2024-02-18T00:37:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.636502+0000 osd.5 (osd.5) 165 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:37:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.637545+0000 osd.5 (osd.5) 166 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:37:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.744365+0000 osd.1 (osd.1) 199 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:37:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.747284+0000 osd.1 (osd.1) 200 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:37:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.879775+0000 osd.3 (osd.3) 213 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.880852+0000 osd.3 (osd.3) 214 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:41.886616+0000 mgr.y (mgr.24454) 363 : cluster [DBG] pgmap v562: 246 pgs: 8 creating+peering, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:42.182645+0000 osd.4 (osd.4) 221 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:42.183564+0000 osd.4 (osd.4) 222 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:42.353762+0000 osd.0 (osd.0) 183 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:42.354754+0000 osd.0 (osd.0) 184 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:42.688979+0000 mon.a (mon.0) 2404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:42.689081+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e384: 8 total, 8 up, 7 in 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:42.693945+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:42.694382+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:42.696537+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:42.696834+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.573976+0000 osd.6 (osd.6) 185 : cluster [DBG] 62.17 scrub starts 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.575809+0000 osd.6 (osd.6) 186 : cluster [DBG] 62.17 scrub ok 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.636502+0000 osd.5 (osd.5) 165 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.637545+0000 osd.5 (osd.5) 166 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.744365+0000 osd.1 (osd.1) 199 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.747284+0000 osd.1 (osd.1) 200 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.879775+0000 osd.3 (osd.3) 213 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.880852+0000 osd.3 (osd.3) 214 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:41.886616+0000 mgr.y (mgr.24454) 363 : cluster [DBG] pgmap v562: 246 pgs: 8 creating+peering, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:42.182645+0000 osd.4 (osd.4) 221 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:42.183564+0000 osd.4 (osd.4) 222 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:37:43.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:42.353762+0000 osd.0 (osd.0) 183 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:42.354754+0000 osd.0 (osd.0) 184 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:42.688979+0000 mon.a (mon.0) 2404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:42.689081+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e384: 8 total, 8 up, 7 in 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:42.693945+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:42.694382+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:42.696537+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:42.696834+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:43.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.573976+0000 osd.6 (osd.6) 185 : cluster [DBG] 62.17 scrub starts 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.575809+0000 osd.6 (osd.6) 186 : cluster [DBG] 62.17 scrub ok 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.636502+0000 osd.5 (osd.5) 165 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.637545+0000 osd.5 (osd.5) 166 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.744365+0000 osd.1 (osd.1) 199 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.747284+0000 osd.1 (osd.1) 200 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.879775+0000 osd.3 (osd.3) 213 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.880852+0000 osd.3 (osd.3) 214 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:41.886616+0000 mgr.y (mgr.24454) 363 : cluster [DBG] pgmap v562: 246 pgs: 8 creating+peering, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:42.182645+0000 osd.4 (osd.4) 221 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:42.183564+0000 osd.4 (osd.4) 222 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:37:43.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:42.353762+0000 osd.0 (osd.0) 183 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:37:43.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:42.354754+0000 osd.0 (osd.0) 184 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:37:43.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:42.688979+0000 mon.a (mon.0) 2404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:37:43.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:42.689081+0000 mon.a (mon.0) 2405 : cluster [DBG] osdmap e384: 8 total, 8 up, 7 in 2024-02-18T00:37:43.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:42.693945+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:43.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:42.694382+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:37:43.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:42.696537+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:43.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:42.696834+0000 mon.a (mon.0) 2407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:43.898 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:43.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:43.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:44.002 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:44.034 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:44.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:44.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:44.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:44.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:43.127235+0000 mgr.y (mgr.24454) 364 : audit [DBG] from='client.41039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:44.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.157712+0000 osd.4 (osd.4) 223 : cluster [DBG] 5.e scrub starts 2024-02-18T00:37:44.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.161179+0000 osd.4 (osd.4) 224 : cluster [DBG] 5.e scrub ok 2024-02-18T00:37:44.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.307988+0000 osd.2 (osd.2) 169 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:37:44.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.309483+0000 osd.2 (osd.2) 170 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:37:44.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.655746+0000 mon.a (mon.0) 2408 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:44.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:43.660414+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:43.660489+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.660532+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e385: 8 total, 8 up, 7 in 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:43.662555+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:43.662665+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:43.668536+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:43.668965+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:43.127235+0000 mgr.y (mgr.24454) 364 : audit [DBG] from='client.41039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.157712+0000 osd.4 (osd.4) 223 : cluster [DBG] 5.e scrub starts 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.161179+0000 osd.4 (osd.4) 224 : cluster [DBG] 5.e scrub ok 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.307988+0000 osd.2 (osd.2) 169 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:37:44.188 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.309483+0000 osd.2 (osd.2) 170 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.655746+0000 mon.a (mon.0) 2408 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:43.660414+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:43.660489+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.660532+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e385: 8 total, 8 up, 7 in 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:43.662555+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:43.662665+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:43.668536+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:44.189 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:43.668965+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:44.190 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:44.245 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:44.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:44.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:44.350 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:44.380 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:43.127235+0000 mgr.y (mgr.24454) 364 : audit [DBG] from='client.41039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.157712+0000 osd.4 (osd.4) 223 : cluster [DBG] 5.e scrub starts 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.161179+0000 osd.4 (osd.4) 224 : cluster [DBG] 5.e scrub ok 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.307988+0000 osd.2 (osd.2) 169 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.309483+0000 osd.2 (osd.2) 170 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.655746+0000 mon.a (mon.0) 2408 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:43.660414+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:43.660489+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.660532+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e385: 8 total, 8 up, 7 in 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:43.662555+0000 mon.c (mon.2) 376 : audit [INF] from='client.? 172.21.15.59:0/3621370462' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:43.662665+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:43.668536+0000 mon.a (mon.0) 2412 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]: dispatch 2024-02-18T00:37:44.381 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:43.668965+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:37:44.382 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:44.405 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:44.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:44.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:44.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:44.538 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:44.558 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:44.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:44.561 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:44.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:44.700 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:44.927 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":563,"stamp":"2024-02-18T00:37:42.691431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007253,"num_objects":377,"num_object_clones":2,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":1708,"num_read_kb":1331,"num_write":1201,"num_write_kb":4521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":4,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":2,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2550,"ondisk_log_size":2550,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":686,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2158632,"kb_used_data":41200,"kb_used_omap":0,"kb_used_meta":2117376,"kb_avail":654057432,"statfs":{"total":671965249536,"available":669754810368,"internally_reserved":0,"allocated":42188800,"data_stored":27032296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2168193024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":64,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":4,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":2,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019079"},"pg_stats":[{"pgid":"204.1f","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676979+0000","last_change":"2024-02-18T00:37:37.691057+0000","last_active":"2024-02-18T00:37:39.676979+0000","last_peered":"2024-02-18T00:37:39.676979+0000","last_clean":"2024-02-18T00:37:39.676979+0000","last_became_active":"2024-02-18T00:37:37.690668+0000","last_became_peered":"2024-02-18T00:37:37.690668+0000","last_unstale":"2024-02-18T00:37:39.676979+0000","last_undegraded":"2024-02-18T00:37:39.676979+0000","last_fullsized":"2024-02-18T00:37:39.676979+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.1e","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310267+0000","last_change":"2024-02-18T00:37:37.686442+0000","last_active":"2024-02-18T00:37:40.310267+0000","last_peered":"2024-02-18T00:37:40.310267+0000","last_clean":"2024-02-18T00:37:40.310267+0000","last_became_active":"2024-02-18T00:37:37.686132+0000","last_became_peered":"2024-02-18T00:37:37.686132+0000","last_unstale":"2024-02-18T00:37:40.310267+0000","last_undegraded":"2024-02-18T00:37:40.310267+0000","last_fullsized":"2024-02-18T00:37:40.310267+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.1d","version":"382'2","reported_seq":15,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.707054+0000","last_change":"2024-02-18T00:37:37.691189+0000","last_active":"2024-02-18T00:37:40.707054+0000","last_peered":"2024-02-18T00:37:40.707054+0000","last_clean":"2024-02-18T00:37:40.707054+0000","last_became_active":"2024-02-18T00:37:37.690876+0000","last_became_peered":"2024-02-18T00:37:37.690876+0000","last_unstale":"2024-02-18T00:37:40.707054+0000","last_undegraded":"2024-02-18T00:37:40.707054+0000","last_fullsized":"2024-02-18T00:37:40.707054+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":1,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.1c","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312867+0000","last_change":"2024-02-18T00:37:37.690281+0000","last_active":"2024-02-18T00:37:40.312867+0000","last_peered":"2024-02-18T00:37:40.312867+0000","last_clean":"2024-02-18T00:37:40.312867+0000","last_became_active":"2024-02-18T00:37:37.689959+0000","last_became_peered":"2024-02-18T00:37:37.689959+0000","last_unstale":"2024-02-18T00:37:40.312867+0000","last_undegraded":"2024-02-18T00:37:40.312867+0000","last_fullsized":"2024-02-18T00:37:40.312867+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.1b","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311494+0000","last_change":"2024-02-18T00:37:37.690607+0000","last_active":"2024-02-18T00:37:40.311494+0000","last_peered":"2024-02-18T00:37:40.311494+0000","last_clean":"2024-02-18T00:37:40.311494+0000","last_became_active":"2024-02-18T00:37:37.690143+0000","last_became_peered":"2024-02-18T00:37:37.690143+0000","last_unstale":"2024-02-18T00:37:40.311494+0000","last_undegraded":"2024-02-18T00:37:40.311494+0000","last_fullsized":"2024-02-18T00:37:40.311494+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.1a","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674100+0000","last_change":"2024-02-18T00:37:37.690951+0000","last_active":"2024-02-18T00:37:39.674100+0000","last_peered":"2024-02-18T00:37:39.674100+0000","last_clean":"2024-02-18T00:37:39.674100+0000","last_became_active":"2024-02-18T00:37:37.690575+0000","last_became_peered":"2024-02-18T00:37:37.690575+0000","last_unstale":"2024-02-18T00:37:39.674100+0000","last_undegraded":"2024-02-18T00:37:39.674100+0000","last_fullsized":"2024-02-18T00:37:39.674100+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.19","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313670+0000","last_change":"2024-02-18T00:37:37.685040+0000","last_active":"2024-02-18T00:37:40.313670+0000","last_peered":"2024-02-18T00:37:40.313670+0000","last_clean":"2024-02-18T00:37:40.313670+0000","last_became_active":"2024-02-18T00:37:37.684783+0000","last_became_peered":"2024-02-18T00:37:37.684783+0000","last_unstale":"2024-02-18T00:37:40.313670+0000","last_undegraded":"2024-02-18T00:37:40.313670+0000","last_fullsized":"2024-02-18T00:37:40.313670+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.18","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309777+0000","last_change":"2024-02-18T00:37:37.691121+0000","last_active":"2024-02-18T00:37:40.309777+0000","last_peered":"2024-02-18T00:37:40.309777+0000","last_clean":"2024-02-18T00:37:40.309777+0000","last_became_active":"2024-02-18T00:37:37.690776+0000","last_became_peered":"2024-02-18T00:37:37.690776+0000","last_unstale":"2024-02-18T00:37:40.309777+0000","last_undegraded":"2024-02-18T00:37:40.309777+0000","last_fullsized":"2024-02-18T00:37:40.309777+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.17","version":"382'1","reported_seq":14,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.708861+0000","last_change":"2024-02-18T00:37:37.684600+0000","last_active":"2024-02-18T00:37:40.708861+0000","last_peered":"2024-02-18T00:37:40.708861+0000","last_clean":"2024-02-18T00:37:40.708861+0000","last_became_active":"2024-02-18T00:37:37.684352+0000","last_became_peered":"2024-02-18T00:37:37.684352+0000","last_unstale":"2024-02-18T00:37:40.708861+0000","last_undegraded":"2024-02-18T00:37:40.708861+0000","last_fullsized":"2024-02-18T00:37:40.708861+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.16","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310356+0000","last_change":"2024-02-18T00:37:37.686206+0000","last_active":"2024-02-18T00:37:40.310356+0000","last_peered":"2024-02-18T00:37:40.310356+0000","last_clean":"2024-02-18T00:37:40.310356+0000","last_became_active":"2024-02-18T00:37:37.685931+0000","last_became_peered":"2024-02-18T00:37:37.685931+0000","last_unstale":"2024-02-18T00:37:40.310356+0000","last_undegraded":"2024-02-18T00:37:40.310356+0000","last_fullsized":"2024-02-18T00:37:40.310356+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.15","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675823+0000","last_change":"2024-02-18T00:37:37.683379+0000","last_active":"2024-02-18T00:37:39.675823+0000","last_peered":"2024-02-18T00:37:39.675823+0000","last_clean":"2024-02-18T00:37:39.675823+0000","last_became_active":"2024-02-18T00:37:37.683099+0000","last_became_peered":"2024-02-18T00:37:37.683099+0000","last_unstale":"2024-02-18T00:37:39.675823+0000","last_undegraded":"2024-02-18T00:37:39.675823+0000","last_fullsized":"2024-02-18T00:37:39.675823+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.14","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310437+0000","last_change":"2024-02-18T00:37:37.689283+0000","last_active":"2024-02-18T00:37:40.310437+0000","last_peered":"2024-02-18T00:37:40.310437+0000","last_clean":"2024-02-18T00:37:40.310437+0000","last_became_active":"2024-02-18T00:37:37.688954+0000","last_became_peered":"2024-02-18T00:37:37.688954+0000","last_unstale":"2024-02-18T00:37:40.310437+0000","last_undegraded":"2024-02-18T00:37:40.310437+0000","last_fullsized":"2024-02-18T00:37:40.310437+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.13","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675469+0000","last_change":"2024-02-18T00:37:37.682492+0000","last_active":"2024-02-18T00:37:39.675469+0000","last_peered":"2024-02-18T00:37:39.675469+0000","last_clean":"2024-02-18T00:37:39.675469+0000","last_became_active":"2024-02-18T00:37:37.682181+0000","last_became_peered":"2024-02-18T00:37:37.682181+0000","last_unstale":"2024-02-18T00:37:39.675469+0000","last_undegraded":"2024-02-18T00:37:39.675469+0000","last_fullsized":"2024-02-18T00:37:39.675469+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.12","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675709+0000","last_change":"2024-02-18T00:37:37.681706+0000","last_active":"2024-02-18T00:37:39.675709+0000","last_peered":"2024-02-18T00:37:39.675709+0000","last_clean":"2024-02-18T00:37:39.675709+0000","last_became_active":"2024-02-18T00:37:37.681258+0000","last_became_peered":"2024-02-18T00:37:37.681258+0000","last_unstale":"2024-02-18T00:37:39.675709+0000","last_undegraded":"2024-02-18T00:37:39.675709+0000","last_fullsized":"2024-02-18T00:37:39.675709+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.11","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674887+0000","last_change":"2024-02-18T00:37:37.679936+0000","last_active":"2024-02-18T00:37:39.674887+0000","last_peered":"2024-02-18T00:37:39.674887+0000","last_clean":"2024-02-18T00:37:39.674887+0000","last_became_active":"2024-02-18T00:37:37.679607+0000","last_became_peered":"2024-02-18T00:37:37.679607+0000","last_unstale":"2024-02-18T00:37:39.674887+0000","last_undegraded":"2024-02-18T00:37:39.674887+0000","last_fullsized":"2024-02-18T00:37:39.674887+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"204.10","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309530+0000","last_change":"2024-02-18T00:37:37.689865+0000","last_active":"2024-02-18T00:37:40.309530+0000","last_peered":"2024-02-18T00:37:40.309530+0000","last_clean":"2024-02-18T00:37:40.309530+0000","last_became_active":"2024-02-18T00:37:37.689544+0000","last_became_peered":"2024-02-18T00:37:37.689544+0000","last_unstale":"2024-02-18T00:37:40.309530+0000","last_undegraded":"2024-02-18T00:37:40.309530+0000","last_fullsized":"2024-02-18T00:37:40.309530+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.f","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311340+0000","last_change":"2024-02-18T00:37:37.691260+0000","last_active":"2024-02-18T00:37:40.311340+0000","last_peered":"2024-02-18T00:37:40.311340+0000","last_clean":"2024-02-18T00:37:40.311340+0000","last_became_active":"2024-02-18T00:37:37.690980+0000","last_became_peered":"2024-02-18T00:37:37.690980+0000","last_unstale":"2024-02-18T00:37:40.311340+0000","last_undegraded":"2024-02-18T00:37:40.311340+0000","last_fullsized":"2024-02-18T00:37:40.311340+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.e","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676136+0000","last_change":"2024-02-18T00:37:37.681239+0000","last_active":"2024-02-18T00:37:39.676136+0000","last_peered":"2024-02-18T00:37:39.676136+0000","last_clean":"2024-02-18T00:37:39.676136+0000","last_became_active":"2024-02-18T00:37:37.681019+0000","last_became_peered":"2024-02-18T00:37:37.681019+0000","last_unstale":"2024-02-18T00:37:39.676136+0000","last_undegraded":"2024-02-18T00:37:39.676136+0000","last_fullsized":"2024-02-18T00:37:39.676136+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.d","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311314+0000","last_change":"2024-02-18T00:37:37.684496+0000","last_active":"2024-02-18T00:37:40.311314+0000","last_peered":"2024-02-18T00:37:40.311314+0000","last_clean":"2024-02-18T00:37:40.311314+0000","last_became_active":"2024-02-18T00:37:37.684149+0000","last_became_peered":"2024-02-18T00:37:37.684149+0000","last_unstale":"2024-02-18T00:37:40.311314+0000","last_undegraded":"2024-02-18T00:37:40.311314+0000","last_fullsized":"2024-02-18T00:37:40.311314+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"204.c","version":"382'1","reported_seq":14,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.698667+0000","last_change":"2024-02-18T00:37:37.689105+0000","last_active":"2024-02-18T00:37:40.698667+0000","last_peered":"2024-02-18T00:37:40.698667+0000","last_clean":"2024-02-18T00:37:40.698667+0000","last_became_active":"2024-02-18T00:37:37.688822+0000","last_became_peered":"2024-02-18T00:37:37.688822+0000","last_unstale":"2024-02-18T00:37:40.698667+0000","last_undegraded":"2024-02-18T00:37:40.698667+0000","last_fullsized":"2024-02-18T00:37:40.698667+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"204.b","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311581+0000","last_change":"2024-02-18T00:37:37.690442+0000","last_active":"2024-02-18T00:37:40.311581+0000","last_peered":"2024-02-18T00:37:40.311581+0000","last_clean":"2024-02-18T00:37:40.311581+0000","last_became_active":"2024-02-18T00:37:37.689811+0000","last_became_peered":"2024-02-18T00:37:37.689811+0000","last_unstale":"2024-02-18T00:37:40.311581+0000","last_undegraded":"2024-02-18T00:37:40.311581+0000","last_fullsized":"2024-02-18T00:37:40.311581+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.a","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675815+0000","last_change":"2024-02-18T00:37:37.681540+0000","last_active":"2024-02-18T00:37:39.675815+0000","last_peered":"2024-02-18T00:37:39.675815+0000","last_clean":"2024-02-18T00:37:39.675815+0000","last_became_active":"2024-02-18T00:37:37.680851+0000","last_became_peered":"2024-02-18T00:37:37.680851+0000","last_unstale":"2024-02-18T00:37:39.675815+0000","last_undegraded":"2024-02-18T00:37:39.675815+0000","last_fullsized":"2024-02-18T00:37:39.675815+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.9","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.673911+0000","last_change":"2024-02-18T00:37:37.683135+0000","last_active":"2024-02-18T00:37:39.673911+0000","last_peered":"2024-02-18T00:37:39.673911+0000","last_clean":"2024-02-18T00:37:39.673911+0000","last_became_active":"2024-02-18T00:37:37.682836+0000","last_became_peered":"2024-02-18T00:37:37.682836+0000","last_unstale":"2024-02-18T00:37:39.673911+0000","last_undegraded":"2024-02-18T00:37:39.673911+0000","last_fullsized":"2024-02-18T00:37:39.673911+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.8","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309599+0000","last_change":"2024-02-18T00:37:37.686676+0000","last_active":"2024-02-18T00:37:40.309599+0000","last_peered":"2024-02-18T00:37:40.309599+0000","last_clean":"2024-02-18T00:37:40.309599+0000","last_became_active":"2024-02-18T00:37:37.686332+0000","last_became_peered":"2024-02-18T00:37:37.686332+0000","last_unstale":"2024-02-18T00:37:40.309599+0000","last_undegraded":"2024-02-18T00:37:40.309599+0000","last_fullsized":"2024-02-18T00:37:40.309599+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.7","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676395+0000","last_change":"2024-02-18T00:37:37.681879+0000","last_active":"2024-02-18T00:37:39.676395+0000","last_peered":"2024-02-18T00:37:39.676395+0000","last_clean":"2024-02-18T00:37:39.676395+0000","last_became_active":"2024-02-18T00:37:37.681577+0000","last_became_peered":"2024-02-18T00:37:37.681577+0000","last_unstale":"2024-02-18T00:37:39.676395+0000","last_undegraded":"2024-02-18T00:37:39.676395+0000","last_fullsized":"2024-02-18T00:37:39.676395+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.6","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677641+0000","last_change":"2024-02-18T00:37:37.683674+0000","last_active":"2024-02-18T00:37:39.677641+0000","last_peered":"2024-02-18T00:37:39.677641+0000","last_clean":"2024-02-18T00:37:39.677641+0000","last_became_active":"2024-02-18T00:37:37.683363+0000","last_became_peered":"2024-02-18T00:37:37.683363+0000","last_unstale":"2024-02-18T00:37:39.677641+0000","last_undegraded":"2024-02-18T00:37:39.677641+0000","last_fullsized":"2024-02-18T00:37:39.677641+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.5","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311025+0000","last_change":"2024-02-18T00:37:37.688299+0000","last_active":"2024-02-18T00:37:40.311025+0000","last_peered":"2024-02-18T00:37:40.311025+0000","last_clean":"2024-02-18T00:37:40.311025+0000","last_became_active":"2024-02-18T00:37:37.688016+0000","last_became_peered":"2024-02-18T00:37:37.688016+0000","last_unstale":"2024-02-18T00:37:40.311025+0000","last_undegraded":"2024-02-18T00:37:40.311025+0000","last_fullsized":"2024-02-18T00:37:40.311025+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.4","version":"382'2","reported_seq":15,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.696097+0000","last_change":"2024-02-18T00:37:37.692443+0000","last_active":"2024-02-18T00:37:40.696097+0000","last_peered":"2024-02-18T00:37:40.696097+0000","last_clean":"2024-02-18T00:37:40.696097+0000","last_became_active":"2024-02-18T00:37:37.691353+0000","last_became_peered":"2024-02-18T00:37:37.691353+0000","last_unstale":"2024-02-18T00:37:40.696097+0000","last_undegraded":"2024-02-18T00:37:40.696097+0000","last_fullsized":"2024-02-18T00:37:40.696097+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":1,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.3","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310775+0000","last_change":"2024-02-18T00:37:37.687815+0000","last_active":"2024-02-18T00:37:40.310775+0000","last_peered":"2024-02-18T00:37:40.310775+0000","last_clean":"2024-02-18T00:37:40.310775+0000","last_became_active":"2024-02-18T00:37:37.687486+0000","last_became_peered":"2024-02-18T00:37:37.687486+0000","last_unstale":"2024-02-18T00:37:40.310775+0000","last_undegraded":"2024-02-18T00:37:40.310775+0000","last_fullsized":"2024-02-18T00:37:40.310775+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"204.2","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310798+0000","last_change":"2024-02-18T00:37:37.691352+0000","last_active":"2024-02-18T00:37:40.310798+0000","last_peered":"2024-02-18T00:37:40.310798+0000","last_clean":"2024-02-18T00:37:40.310798+0000","last_became_active":"2024-02-18T00:37:37.690874+0000","last_became_peered":"2024-02-18T00:37:37.690874+0000","last_unstale":"2024-02-18T00:37:40.310798+0000","last_undegraded":"2024-02-18T00:37:40.310798+0000","last_fullsized":"2024-02-18T00:37:40.310798+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.1","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309701+0000","last_change":"2024-02-18T00:37:37.686342+0000","last_active":"2024-02-18T00:37:40.309701+0000","last_peered":"2024-02-18T00:37:40.309701+0000","last_clean":"2024-02-18T00:37:40.309701+0000","last_became_active":"2024-02-18T00:37:37.686026+0000","last_became_peered":"2024-02-18T00:37:37.686026+0000","last_unstale":"2024-02-18T00:37:40.309701+0000","last_undegraded":"2024-02-18T00:37:40.309701+0000","last_fullsized":"2024-02-18T00:37:40.309701+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.0","version":"0'0","reported_seq":12,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309788+0000","last_change":"2024-02-18T00:37:37.694899+0000","last_active":"2024-02-18T00:37:40.309788+0000","last_peered":"2024-02-18T00:37:40.309788+0000","last_clean":"2024-02-18T00:37:40.309788+0000","last_became_active":"2024-02-18T00:37:37.694553+0000","last_became_peered":"2024-02-18T00:37:37.694553+0000","last_unstale":"2024-02-18T00:37:40.309788+0000","last_undegraded":"2024-02-18T00:37:40.309788+0000","last_fullsized":"2024-02-18T00:37:40.309788+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":353,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.673386+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:37:39.673386+0000","last_peered":"2024-02-18T00:37:39.673386+0000","last_clean":"2024-02-18T00:37:39.673386+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:37:39.673386+0000","last_undegraded":"2024-02-18T00:37:39.673386+0000","last_fullsized":"2024-02-18T00:37:39.673386+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"347'2","reported_seq":321,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675398+0000","last_change":"2024-02-18T00:37:29.759198+0000","last_active":"2024-02-18T00:37:39.675398+0000","last_peered":"2024-02-18T00:37:39.675398+0000","last_clean":"2024-02-18T00:37:39.675398+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:37:39.675398+0000","last_undegraded":"2024-02-18T00:37:39.675398+0000","last_fullsized":"2024-02-18T00:37:39.675398+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'2","last_scrub_stamp":"2024-02-18T00:37:29.759114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.759114+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":381,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313361+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:37:40.313361+0000","last_peered":"2024-02-18T00:37:40.313361+0000","last_clean":"2024-02-18T00:37:40.313361+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:37:40.313361+0000","last_undegraded":"2024-02-18T00:37:40.313361+0000","last_fullsized":"2024-02-18T00:37:40.313361+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"347'75","reported_seq":384,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.678420+0000","last_change":"2024-02-18T00:36:34.581378+0000","last_active":"2024-02-18T00:37:39.678420+0000","last_peered":"2024-02-18T00:37:39.678420+0000","last_clean":"2024-02-18T00:37:39.678420+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:37:39.678420+0000","last_undegraded":"2024-02-18T00:37:39.678420+0000","last_fullsized":"2024-02-18T00:37:39.678420+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'73","last_scrub_stamp":"2024-02-18T00:36:34.581302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.581302+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"379'121","reported_seq":423,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.693318+0000","last_change":"2024-02-18T00:36:36.464430+0000","last_active":"2024-02-18T00:37:40.693318+0000","last_peered":"2024-02-18T00:37:40.693318+0000","last_clean":"2024-02-18T00:37:40.693318+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:37:40.693318+0000","last_undegraded":"2024-02-18T00:37:40.693318+0000","last_fullsized":"2024-02-18T00:37:40.693318+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'120","last_scrub_stamp":"2024-02-18T00:36:36.464361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.464361+0000","log_size":121,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":397,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675978+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:37:39.675978+0000","last_peered":"2024-02-18T00:37:39.675978+0000","last_clean":"2024-02-18T00:37:39.675978+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:37:39.675978+0000","last_undegraded":"2024-02-18T00:37:39.675978+0000","last_fullsized":"2024-02-18T00:37:39.675978+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"347'125","reported_seq":396,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677727+0000","last_change":"2024-02-18T00:36:53.649744+0000","last_active":"2024-02-18T00:37:39.677727+0000","last_peered":"2024-02-18T00:37:39.677727+0000","last_clean":"2024-02-18T00:37:39.677727+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:37:39.677727+0000","last_undegraded":"2024-02-18T00:37:39.677727+0000","last_fullsized":"2024-02-18T00:37:39.677727+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'124","last_scrub_stamp":"2024-02-18T00:36:53.649677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.649677+0000","log_size":125,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":38,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"347'74","reported_seq":377,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676571+0000","last_change":"2024-02-18T00:36:37.532389+0000","last_active":"2024-02-18T00:37:39.676571+0000","last_peered":"2024-02-18T00:37:39.676571+0000","last_clean":"2024-02-18T00:37:39.676571+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:37:39.676571+0000","last_undegraded":"2024-02-18T00:37:39.676571+0000","last_fullsized":"2024-02-18T00:37:39.676571+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'73","last_scrub_stamp":"2024-02-18T00:36:37.532316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.532316+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":23,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'70","reported_seq":343,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674232+0000","last_change":"2024-02-18T00:36:06.662037+0000","last_active":"2024-02-18T00:37:39.674232+0000","last_peered":"2024-02-18T00:37:39.674232+0000","last_clean":"2024-02-18T00:37:39.674232+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:37:39.674232+0000","last_undegraded":"2024-02-18T00:37:39.674232+0000","last_fullsized":"2024-02-18T00:37:39.674232+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'68","last_scrub_stamp":"2024-02-18T00:36:02.645624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.645624+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"347'52","reported_seq":360,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675130+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:37:39.675130+0000","last_peered":"2024-02-18T00:37:39.675130+0000","last_clean":"2024-02-18T00:37:39.675130+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:37:39.675130+0000","last_undegraded":"2024-02-18T00:37:39.675130+0000","last_fullsized":"2024-02-18T00:37:39.675130+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'41","reported_seq":327,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674208+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:37:39.674208+0000","last_peered":"2024-02-18T00:37:39.674208+0000","last_clean":"2024-02-18T00:37:39.674208+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:37:39.674208+0000","last_undegraded":"2024-02-18T00:37:39.674208+0000","last_fullsized":"2024-02-18T00:37:39.674208+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"347'55","reported_seq":342,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312792+0000","last_change":"2024-02-18T00:36:11.810050+0000","last_active":"2024-02-18T00:37:40.312792+0000","last_peered":"2024-02-18T00:37:40.312792+0000","last_clean":"2024-02-18T00:37:40.312792+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:37:40.312792+0000","last_undegraded":"2024-02-18T00:37:40.312792+0000","last_fullsized":"2024-02-18T00:37:40.312792+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'54","last_scrub_stamp":"2024-02-18T00:36:11.809925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.809925+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":18,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"379'18","reported_seq":331,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.697277+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:37:40.697277+0000","last_peered":"2024-02-18T00:37:40.697277+0000","last_clean":"2024-02-18T00:37:40.697277+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:37:40.697277+0000","last_undegraded":"2024-02-18T00:37:40.697277+0000","last_fullsized":"2024-02-18T00:37:40.697277+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"347'29","reported_seq":364,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675825+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:37:39.675825+0000","last_peered":"2024-02-18T00:37:39.675825+0000","last_clean":"2024-02-18T00:37:39.675825+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:37:39.675825+0000","last_undegraded":"2024-02-18T00:37:39.675825+0000","last_fullsized":"2024-02-18T00:37:39.675825+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"347'83","reported_seq":383,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310620+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:37:40.310620+0000","last_peered":"2024-02-18T00:37:40.310620+0000","last_clean":"2024-02-18T00:37:40.310620+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:37:40.310620+0000","last_undegraded":"2024-02-18T00:37:40.310620+0000","last_fullsized":"2024-02-18T00:37:40.310620+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":67,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":364,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676121+0000","last_change":"2024-02-18T00:36:34.207908+0000","last_active":"2024-02-18T00:37:39.676121+0000","last_peered":"2024-02-18T00:37:39.676121+0000","last_clean":"2024-02-18T00:37:39.676121+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:37:39.676121+0000","last_undegraded":"2024-02-18T00:37:39.676121+0000","last_fullsized":"2024-02-18T00:37:39.676121+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:36:34.207840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.207840+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'57","reported_seq":382,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674115+0000","last_change":"2024-02-18T00:37:36.817791+0000","last_active":"2024-02-18T00:37:39.674115+0000","last_peered":"2024-02-18T00:37:39.674115+0000","last_clean":"2024-02-18T00:37:39.674115+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:37:39.674115+0000","last_undegraded":"2024-02-18T00:37:39.674115+0000","last_fullsized":"2024-02-18T00:37:39.674115+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'57","last_scrub_stamp":"2024-02-18T00:37:36.817719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.817719+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"214'19","reported_seq":339,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675392+0000","last_change":"2024-02-18T00:37:31.817355+0000","last_active":"2024-02-18T00:37:39.675392+0000","last_peered":"2024-02-18T00:37:39.675392+0000","last_clean":"2024-02-18T00:37:39.675392+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:37:39.675392+0000","last_undegraded":"2024-02-18T00:37:39.675392+0000","last_fullsized":"2024-02-18T00:37:39.675392+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:37:31.817281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.817281+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"347'74","reported_seq":379,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674272+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:37:39.674272+0000","last_peered":"2024-02-18T00:37:39.674272+0000","last_clean":"2024-02-18T00:37:39.674272+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:37:39.674272+0000","last_undegraded":"2024-02-18T00:37:39.674272+0000","last_fullsized":"2024-02-18T00:37:39.674272+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"361'18","reported_seq":331,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311759+0000","last_change":"2024-02-18T00:36:43.345182+0000","last_active":"2024-02-18T00:37:40.311759+0000","last_peered":"2024-02-18T00:37:40.311759+0000","last_clean":"2024-02-18T00:37:40.311759+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:37:40.311759+0000","last_undegraded":"2024-02-18T00:37:40.311759+0000","last_fullsized":"2024-02-18T00:37:40.311759+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'17","last_scrub_stamp":"2024-02-18T00:36:43.345064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.345064+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":343,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676455+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:37:39.676455+0000","last_peered":"2024-02-18T00:37:39.676455+0000","last_clean":"2024-02-18T00:37:39.676455+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:37:39.676455+0000","last_undegraded":"2024-02-18T00:37:39.676455+0000","last_fullsized":"2024-02-18T00:37:39.676455+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"379'21","reported_seq":345,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.707771+0000","last_change":"2024-02-18T00:36:06.659238+0000","last_active":"2024-02-18T00:37:40.707771+0000","last_peered":"2024-02-18T00:37:40.707771+0000","last_clean":"2024-02-18T00:37:40.707771+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:37:40.707771+0000","last_undegraded":"2024-02-18T00:37:40.707771+0000","last_fullsized":"2024-02-18T00:37:40.707771+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'18","last_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.483932+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000035,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":3914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":407,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311922+0000","last_change":"2024-02-18T00:36:34.017135+0000","last_active":"2024-02-18T00:37:40.311922+0000","last_peered":"2024-02-18T00:37:40.311922+0000","last_clean":"2024-02-18T00:37:40.311922+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:37:40.311922+0000","last_undegraded":"2024-02-18T00:37:40.311922+0000","last_fullsized":"2024-02-18T00:37:40.311922+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:36:34.017065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.017065+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":299,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310209+0000","last_change":"2024-02-18T00:36:37.504644+0000","last_active":"2024-02-18T00:37:40.310209+0000","last_peered":"2024-02-18T00:37:40.310209+0000","last_clean":"2024-02-18T00:37:40.310209+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:37:40.310209+0000","last_undegraded":"2024-02-18T00:37:40.310209+0000","last_fullsized":"2024-02-18T00:37:40.310209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:37.504568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.504568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":421,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676991+0000","last_change":"2024-02-18T00:36:38.583124+0000","last_active":"2024-02-18T00:37:39.676991+0000","last_peered":"2024-02-18T00:37:39.676991+0000","last_clean":"2024-02-18T00:37:39.676991+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:37:39.676991+0000","last_undegraded":"2024-02-18T00:37:39.676991+0000","last_fullsized":"2024-02-18T00:37:39.676991+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'112","last_scrub_stamp":"2024-02-18T00:36:38.583056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.583056+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":298,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310161+0000","last_change":"2024-02-18T00:37:01.774863+0000","last_active":"2024-02-18T00:37:40.310161+0000","last_peered":"2024-02-18T00:37:40.310161+0000","last_clean":"2024-02-18T00:37:40.310161+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:37:40.310161+0000","last_undegraded":"2024-02-18T00:37:40.310161+0000","last_fullsized":"2024-02-18T00:37:40.310161+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.774795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.774795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":355,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675566+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:37:39.675566+0000","last_peered":"2024-02-18T00:37:39.675566+0000","last_clean":"2024-02-18T00:37:39.675566+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:37:39.675566+0000","last_undegraded":"2024-02-18T00:37:39.675566+0000","last_fullsized":"2024-02-18T00:37:39.675566+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":257,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676181+0000","last_change":"2024-02-18T00:37:27.130328+0000","last_active":"2024-02-18T00:37:39.676181+0000","last_peered":"2024-02-18T00:37:39.676181+0000","last_clean":"2024-02-18T00:37:39.676181+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:37:39.676181+0000","last_undegraded":"2024-02-18T00:37:39.676181+0000","last_fullsized":"2024-02-18T00:37:39.676181+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.130252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.130252+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":408,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311187+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:37:40.311187+0000","last_peered":"2024-02-18T00:37:40.311187+0000","last_clean":"2024-02-18T00:37:40.311187+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:37:40.311187+0000","last_undegraded":"2024-02-18T00:37:40.311187+0000","last_fullsized":"2024-02-18T00:37:40.311187+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":255,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676270+0000","last_change":"2024-02-18T00:36:53.345938+0000","last_active":"2024-02-18T00:37:39.676270+0000","last_peered":"2024-02-18T00:37:39.676270+0000","last_clean":"2024-02-18T00:37:39.676270+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:37:39.676270+0000","last_undegraded":"2024-02-18T00:37:39.676270+0000","last_fullsized":"2024-02-18T00:37:39.676270+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.345872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":385,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310097+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:37:40.310097+0000","last_peered":"2024-02-18T00:37:40.310097+0000","last_clean":"2024-02-18T00:37:40.310097+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:37:40.310097+0000","last_undegraded":"2024-02-18T00:37:40.310097+0000","last_fullsized":"2024-02-18T00:37:40.310097+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":307,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676647+0000","last_change":"2024-02-18T00:36:39.168150+0000","last_active":"2024-02-18T00:37:39.676647+0000","last_peered":"2024-02-18T00:37:39.676647+0000","last_clean":"2024-02-18T00:37:39.676647+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:37:39.676647+0000","last_undegraded":"2024-02-18T00:37:39.676647+0000","last_fullsized":"2024-02-18T00:37:39.676647+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:36:39.168079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:39.168079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"347'18","reported_seq":335,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310062+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:37:40.310062+0000","last_peered":"2024-02-18T00:37:40.310062+0000","last_clean":"2024-02-18T00:37:40.310062+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:37:40.310062+0000","last_undegraded":"2024-02-18T00:37:40.310062+0000","last_fullsized":"2024-02-18T00:37:40.310062+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":299,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311477+0000","last_change":"2024-02-18T00:36:35.460784+0000","last_active":"2024-02-18T00:37:40.311477+0000","last_peered":"2024-02-18T00:37:40.311477+0000","last_clean":"2024-02-18T00:37:40.311477+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:37:40.311477+0000","last_undegraded":"2024-02-18T00:37:40.311477+0000","last_fullsized":"2024-02-18T00:37:40.311477+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.460703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.460703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":366,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676520+0000","last_change":"2024-02-18T00:36:41.624250+0000","last_active":"2024-02-18T00:37:39.676520+0000","last_peered":"2024-02-18T00:37:39.676520+0000","last_clean":"2024-02-18T00:37:39.676520+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:37:39.676520+0000","last_undegraded":"2024-02-18T00:37:39.676520+0000","last_fullsized":"2024-02-18T00:37:39.676520+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:36:41.624180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.624180+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":289,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312872+0000","last_change":"2024-02-18T00:36:43.933486+0000","last_active":"2024-02-18T00:37:40.312872+0000","last_peered":"2024-02-18T00:37:40.312872+0000","last_clean":"2024-02-18T00:37:40.312872+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:37:40.312872+0000","last_undegraded":"2024-02-18T00:37:40.312872+0000","last_fullsized":"2024-02-18T00:37:40.312872+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.933414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.933414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":324,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312126+0000","last_change":"2024-02-18T00:37:37.618760+0000","last_active":"2024-02-18T00:37:40.312126+0000","last_peered":"2024-02-18T00:37:40.312126+0000","last_clean":"2024-02-18T00:37:40.312126+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:37:40.312126+0000","last_undegraded":"2024-02-18T00:37:40.312126+0000","last_fullsized":"2024-02-18T00:37:40.312126+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:37:37.618667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.618667+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":292,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312861+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:37:40.312861+0000","last_peered":"2024-02-18T00:37:40.312861+0000","last_clean":"2024-02-18T00:37:40.312861+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:37:40.312861+0000","last_undegraded":"2024-02-18T00:37:40.312861+0000","last_fullsized":"2024-02-18T00:37:40.312861+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"379'18","reported_seq":331,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.702913+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:37:40.702913+0000","last_peered":"2024-02-18T00:37:40.702913+0000","last_clean":"2024-02-18T00:37:40.702913+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:37:40.702913+0000","last_undegraded":"2024-02-18T00:37:40.702913+0000","last_fullsized":"2024-02-18T00:37:40.702913+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":372,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675265+0000","last_change":"2024-02-18T00:36:45.627771+0000","last_active":"2024-02-18T00:37:39.675265+0000","last_peered":"2024-02-18T00:37:39.675265+0000","last_clean":"2024-02-18T00:37:39.675265+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:37:39.675265+0000","last_undegraded":"2024-02-18T00:37:39.675265+0000","last_fullsized":"2024-02-18T00:37:39.675265+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.627700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.627700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":335,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311341+0000","last_change":"2024-02-18T00:37:38.588259+0000","last_active":"2024-02-18T00:37:40.311341+0000","last_peered":"2024-02-18T00:37:40.311341+0000","last_clean":"2024-02-18T00:37:40.311341+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:37:40.311341+0000","last_undegraded":"2024-02-18T00:37:40.311341+0000","last_fullsized":"2024-02-18T00:37:40.311341+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'18","last_scrub_stamp":"2024-02-18T00:37:38.588190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.588190+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":322,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309846+0000","last_change":"2024-02-18T00:37:37.726927+0000","last_active":"2024-02-18T00:37:40.309846+0000","last_peered":"2024-02-18T00:37:40.309846+0000","last_clean":"2024-02-18T00:37:40.309846+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:37:40.309846+0000","last_undegraded":"2024-02-18T00:37:40.309846+0000","last_fullsized":"2024-02-18T00:37:40.309846+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.726861+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":296,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311815+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:37:40.311815+0000","last_peered":"2024-02-18T00:37:40.311815+0000","last_clean":"2024-02-18T00:37:40.311815+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:37:40.311815+0000","last_undegraded":"2024-02-18T00:37:40.311815+0000","last_fullsized":"2024-02-18T00:37:40.311815+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":322,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311015+0000","last_change":"2024-02-18T00:37:33.619485+0000","last_active":"2024-02-18T00:37:40.311015+0000","last_peered":"2024-02-18T00:37:40.311015+0000","last_clean":"2024-02-18T00:37:40.311015+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:37:40.311015+0000","last_undegraded":"2024-02-18T00:37:40.311015+0000","last_fullsized":"2024-02-18T00:37:40.311015+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:33.619410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.619410+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":427,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310199+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:37:40.310199+0000","last_peered":"2024-02-18T00:37:40.310199+0000","last_clean":"2024-02-18T00:37:40.310199+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:37:40.310199+0000","last_undegraded":"2024-02-18T00:37:40.310199+0000","last_fullsized":"2024-02-18T00:37:40.310199+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":365,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.678002+0000","last_change":"2024-02-18T00:37:02.627426+0000","last_active":"2024-02-18T00:37:39.678002+0000","last_peered":"2024-02-18T00:37:39.678002+0000","last_clean":"2024-02-18T00:37:39.678002+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:37:39.678002+0000","last_undegraded":"2024-02-18T00:37:39.678002+0000","last_fullsized":"2024-02-18T00:37:39.678002+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.627349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.627349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":344,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311932+0000","last_change":"2024-02-18T00:37:33.616952+0000","last_active":"2024-02-18T00:37:40.311932+0000","last_peered":"2024-02-18T00:37:40.311932+0000","last_clean":"2024-02-18T00:37:40.311932+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:37:40.311932+0000","last_undegraded":"2024-02-18T00:37:40.311932+0000","last_fullsized":"2024-02-18T00:37:40.311932+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.616874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.616874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":293,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676472+0000","last_change":"2024-02-18T00:37:01.919166+0000","last_active":"2024-02-18T00:37:39.676472+0000","last_peered":"2024-02-18T00:37:39.676472+0000","last_clean":"2024-02-18T00:37:39.676472+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:37:39.676472+0000","last_undegraded":"2024-02-18T00:37:39.676472+0000","last_fullsized":"2024-02-18T00:37:39.676472+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.919082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.919082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":396,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312268+0000","last_change":"2024-02-18T00:35:54.188262+0000","last_active":"2024-02-18T00:37:40.312268+0000","last_peered":"2024-02-18T00:37:40.312268+0000","last_clean":"2024-02-18T00:37:40.312268+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:37:40.312268+0000","last_undegraded":"2024-02-18T00:37:40.312268+0000","last_fullsized":"2024-02-18T00:37:40.312268+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:35:54.188188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.188188+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":353,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676505+0000","last_change":"2024-02-18T00:36:43.837775+0000","last_active":"2024-02-18T00:37:39.676505+0000","last_peered":"2024-02-18T00:37:39.676505+0000","last_clean":"2024-02-18T00:37:39.676505+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:37:39.676505+0000","last_undegraded":"2024-02-18T00:37:39.676505+0000","last_fullsized":"2024-02-18T00:37:39.676505+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.837702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.837702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":350,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674419+0000","last_change":"2024-02-18T00:36:56.601833+0000","last_active":"2024-02-18T00:37:39.674419+0000","last_peered":"2024-02-18T00:37:39.674419+0000","last_clean":"2024-02-18T00:37:39.674419+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:37:39.674419+0000","last_undegraded":"2024-02-18T00:37:39.674419+0000","last_fullsized":"2024-02-18T00:37:39.674419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.601727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.601727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":358,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310234+0000","last_change":"2024-02-18T00:35:42.683115+0000","last_active":"2024-02-18T00:37:40.310234+0000","last_peered":"2024-02-18T00:37:40.310234+0000","last_clean":"2024-02-18T00:37:40.310234+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:37:40.310234+0000","last_undegraded":"2024-02-18T00:37:40.310234+0000","last_fullsized":"2024-02-18T00:37:40.310234+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:24.990579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:24.990579+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":295,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311622+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:37:40.311622+0000","last_peered":"2024-02-18T00:37:40.311622+0000","last_clean":"2024-02-18T00:37:40.311622+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:37:40.311622+0000","last_undegraded":"2024-02-18T00:37:40.311622+0000","last_fullsized":"2024-02-18T00:37:40.311622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":428,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312945+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:37:40.312945+0000","last_peered":"2024-02-18T00:37:40.312945+0000","last_clean":"2024-02-18T00:37:40.312945+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:37:40.312945+0000","last_undegraded":"2024-02-18T00:37:40.312945+0000","last_fullsized":"2024-02-18T00:37:40.312945+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":354,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677271+0000","last_change":"2024-02-18T00:36:49.680480+0000","last_active":"2024-02-18T00:37:39.677271+0000","last_peered":"2024-02-18T00:37:39.677271+0000","last_clean":"2024-02-18T00:37:39.677271+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:37:39.677271+0000","last_undegraded":"2024-02-18T00:37:39.677271+0000","last_fullsized":"2024-02-18T00:37:39.677271+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.680414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":350,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.678449+0000","last_change":"2024-02-18T00:37:00.670726+0000","last_active":"2024-02-18T00:37:39.678449+0000","last_peered":"2024-02-18T00:37:39.678449+0000","last_clean":"2024-02-18T00:37:39.678449+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:37:39.678449+0000","last_undegraded":"2024-02-18T00:37:39.678449+0000","last_fullsized":"2024-02-18T00:37:39.678449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.670646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.670646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":254,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.673764+0000","last_change":"2024-02-18T00:37:37.476740+0000","last_active":"2024-02-18T00:37:39.673764+0000","last_peered":"2024-02-18T00:37:39.673764+0000","last_clean":"2024-02-18T00:37:39.673764+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:37:39.673764+0000","last_undegraded":"2024-02-18T00:37:39.673764+0000","last_fullsized":"2024-02-18T00:37:39.673764+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.476676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.476676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":285,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310116+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:37:40.310116+0000","last_peered":"2024-02-18T00:37:40.310116+0000","last_clean":"2024-02-18T00:37:40.310116+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:37:40.310116+0000","last_undegraded":"2024-02-18T00:37:40.310116+0000","last_fullsized":"2024-02-18T00:37:40.310116+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":380,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676506+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:37:39.676506+0000","last_peered":"2024-02-18T00:37:39.676506+0000","last_clean":"2024-02-18T00:37:39.676506+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:37:39.676506+0000","last_undegraded":"2024-02-18T00:37:39.676506+0000","last_fullsized":"2024-02-18T00:37:39.676506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":254,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675903+0000","last_change":"2024-02-18T00:36:58.282770+0000","last_active":"2024-02-18T00:37:39.675903+0000","last_peered":"2024-02-18T00:37:39.675903+0000","last_clean":"2024-02-18T00:37:39.675903+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:37:39.675903+0000","last_undegraded":"2024-02-18T00:37:39.675903+0000","last_fullsized":"2024-02-18T00:37:39.675903+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.282673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.282673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":349,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677763+0000","last_change":"2024-02-18T00:36:59.000753+0000","last_active":"2024-02-18T00:37:39.677763+0000","last_peered":"2024-02-18T00:37:39.677763+0000","last_clean":"2024-02-18T00:37:39.677763+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:37:39.677763+0000","last_undegraded":"2024-02-18T00:37:39.677763+0000","last_fullsized":"2024-02-18T00:37:39.677763+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.000690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.000690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":348,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676782+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:37:39.676782+0000","last_peered":"2024-02-18T00:37:39.676782+0000","last_clean":"2024-02-18T00:37:39.676782+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:37:39.676782+0000","last_undegraded":"2024-02-18T00:37:39.676782+0000","last_fullsized":"2024-02-18T00:37:39.676782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":133,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.673586+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:37:39.673586+0000","last_peered":"2024-02-18T00:37:39.673586+0000","last_clean":"2024-02-18T00:37:39.673586+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:37:39.673586+0000","last_undegraded":"2024-02-18T00:37:39.673586+0000","last_fullsized":"2024-02-18T00:37:39.673586+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":367,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676088+0000","last_change":"2024-02-18T00:37:29.177448+0000","last_active":"2024-02-18T00:37:39.676088+0000","last_peered":"2024-02-18T00:37:39.676088+0000","last_clean":"2024-02-18T00:37:39.676088+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:37:39.676088+0000","last_undegraded":"2024-02-18T00:37:39.676088+0000","last_fullsized":"2024-02-18T00:37:39.676088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:37:29.177369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.177369+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":362,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676350+0000","last_change":"2024-02-18T00:36:38.883486+0000","last_active":"2024-02-18T00:37:39.676350+0000","last_peered":"2024-02-18T00:37:39.676350+0000","last_clean":"2024-02-18T00:37:39.676350+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:37:39.676350+0000","last_undegraded":"2024-02-18T00:37:39.676350+0000","last_fullsized":"2024-02-18T00:37:39.676350+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:38.883391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.883391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":475,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676209+0000","last_change":"2024-02-18T00:37:38.433946+0000","last_active":"2024-02-18T00:37:39.676209+0000","last_peered":"2024-02-18T00:37:39.676209+0000","last_clean":"2024-02-18T00:37:39.676209+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:37:39.676209+0000","last_undegraded":"2024-02-18T00:37:39.676209+0000","last_fullsized":"2024-02-18T00:37:39.676209+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:38.433877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.433877+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":348,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676714+0000","last_change":"2024-02-18T00:37:31.127432+0000","last_active":"2024-02-18T00:37:39.676714+0000","last_peered":"2024-02-18T00:37:39.676714+0000","last_clean":"2024-02-18T00:37:39.676714+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:37:39.676714+0000","last_undegraded":"2024-02-18T00:37:39.676714+0000","last_fullsized":"2024-02-18T00:37:39.676714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.127341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.127341+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":293,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676693+0000","last_change":"2024-02-18T00:37:00.950276+0000","last_active":"2024-02-18T00:37:39.676693+0000","last_peered":"2024-02-18T00:37:39.676693+0000","last_clean":"2024-02-18T00:37:39.676693+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:37:39.676693+0000","last_undegraded":"2024-02-18T00:37:39.676693+0000","last_fullsized":"2024-02-18T00:37:39.676693+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.950130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.950130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":351,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311930+0000","last_change":"2024-02-18T00:36:33.853442+0000","last_active":"2024-02-18T00:37:40.311930+0000","last_peered":"2024-02-18T00:37:40.311930+0000","last_clean":"2024-02-18T00:37:40.311930+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:37:40.311930+0000","last_undegraded":"2024-02-18T00:37:40.311930+0000","last_fullsized":"2024-02-18T00:37:40.311930+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:36:33.853337+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:36:33.853337+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":371,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675696+0000","last_change":"2024-02-18T00:36:08.716901+0000","last_active":"2024-02-18T00:37:39.675696+0000","last_peered":"2024-02-18T00:37:39.675696+0000","last_clean":"2024-02-18T00:37:39.675696+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:37:39.675696+0000","last_undegraded":"2024-02-18T00:37:39.675696+0000","last_fullsized":"2024-02-18T00:37:39.675696+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.716777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.716777+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":454,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676685+0000","last_change":"2024-02-18T00:35:15.294420+0000","last_active":"2024-02-18T00:37:39.676685+0000","last_peered":"2024-02-18T00:37:39.676685+0000","last_clean":"2024-02-18T00:37:39.676685+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:37:39.676685+0000","last_undegraded":"2024-02-18T00:37:39.676685+0000","last_fullsized":"2024-02-18T00:37:39.676685+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:35:15.294291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.294291+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":339,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312011+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:37:40.312011+0000","last_peered":"2024-02-18T00:37:40.312011+0000","last_clean":"2024-02-18T00:37:40.312011+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:37:40.312011+0000","last_undegraded":"2024-02-18T00:37:40.312011+0000","last_fullsized":"2024-02-18T00:37:40.312011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":289,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675687+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:37:39.675687+0000","last_peered":"2024-02-18T00:37:39.675687+0000","last_clean":"2024-02-18T00:37:39.675687+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:37:39.675687+0000","last_undegraded":"2024-02-18T00:37:39.675687+0000","last_fullsized":"2024-02-18T00:37:39.675687+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":409,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676458+0000","last_change":"2024-02-18T00:36:43.283359+0000","last_active":"2024-02-18T00:37:39.676458+0000","last_peered":"2024-02-18T00:37:39.676458+0000","last_clean":"2024-02-18T00:37:39.676458+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:37:39.676458+0000","last_undegraded":"2024-02-18T00:37:39.676458+0000","last_fullsized":"2024-02-18T00:37:39.676458+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:43.283235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.283235+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":323,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309829+0000","last_change":"2024-02-18T00:36:57.321994+0000","last_active":"2024-02-18T00:37:40.309829+0000","last_peered":"2024-02-18T00:37:40.309829+0000","last_clean":"2024-02-18T00:37:40.309829+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:37:40.309829+0000","last_undegraded":"2024-02-18T00:37:40.309829+0000","last_fullsized":"2024-02-18T00:37:40.309829+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.321930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.321930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":341,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312500+0000","last_change":"2024-02-18T00:35:19.809776+0000","last_active":"2024-02-18T00:37:40.312500+0000","last_peered":"2024-02-18T00:37:40.312500+0000","last_clean":"2024-02-18T00:37:40.312500+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:37:40.312500+0000","last_undegraded":"2024-02-18T00:37:40.312500+0000","last_fullsized":"2024-02-18T00:37:40.312500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:19.809707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.809707+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":345,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313110+0000","last_change":"2024-02-18T00:37:30.849071+0000","last_active":"2024-02-18T00:37:40.313110+0000","last_peered":"2024-02-18T00:37:40.313110+0000","last_clean":"2024-02-18T00:37:40.313110+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:37:40.313110+0000","last_undegraded":"2024-02-18T00:37:40.313110+0000","last_fullsized":"2024-02-18T00:37:40.313110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:30.848966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:30.848966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":289,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676472+0000","last_change":"2024-02-18T00:35:22.242488+0000","last_active":"2024-02-18T00:37:39.676472+0000","last_peered":"2024-02-18T00:37:39.676472+0000","last_clean":"2024-02-18T00:37:39.676472+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:37:39.676472+0000","last_undegraded":"2024-02-18T00:37:39.676472+0000","last_fullsized":"2024-02-18T00:37:39.676472+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:22.242404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:22.242404+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":457,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310537+0000","last_change":"2024-02-18T00:36:53.910404+0000","last_active":"2024-02-18T00:37:40.310537+0000","last_peered":"2024-02-18T00:37:40.310537+0000","last_clean":"2024-02-18T00:37:40.310537+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:37:40.310537+0000","last_undegraded":"2024-02-18T00:37:40.310537+0000","last_fullsized":"2024-02-18T00:37:40.310537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.910333+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":388,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675342+0000","last_change":"2024-02-18T00:36:50.262681+0000","last_active":"2024-02-18T00:37:39.675342+0000","last_peered":"2024-02-18T00:37:39.675342+0000","last_clean":"2024-02-18T00:37:39.675342+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:37:39.675342+0000","last_undegraded":"2024-02-18T00:37:39.675342+0000","last_fullsized":"2024-02-18T00:37:39.675342+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:36:50.262602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.262602+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":370,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313441+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:37:40.313441+0000","last_peered":"2024-02-18T00:37:40.313441+0000","last_clean":"2024-02-18T00:37:40.313441+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:37:40.313441+0000","last_undegraded":"2024-02-18T00:37:40.313441+0000","last_fullsized":"2024-02-18T00:37:40.313441+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":252,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311517+0000","last_change":"2024-02-18T00:37:27.733729+0000","last_active":"2024-02-18T00:37:40.311517+0000","last_peered":"2024-02-18T00:37:40.311517+0000","last_clean":"2024-02-18T00:37:40.311517+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:37:40.311517+0000","last_undegraded":"2024-02-18T00:37:40.311517+0000","last_fullsized":"2024-02-18T00:37:40.311517+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.733609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.733609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":285,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311533+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:37:40.311533+0000","last_peered":"2024-02-18T00:37:40.311533+0000","last_clean":"2024-02-18T00:37:40.311533+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:37:40.311533+0000","last_undegraded":"2024-02-18T00:37:40.311533+0000","last_fullsized":"2024-02-18T00:37:40.311533+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":359,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675533+0000","last_change":"2024-02-18T00:36:45.308400+0000","last_active":"2024-02-18T00:37:39.675533+0000","last_peered":"2024-02-18T00:37:39.675533+0000","last_clean":"2024-02-18T00:37:39.675533+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:37:39.675533+0000","last_undegraded":"2024-02-18T00:37:39.675533+0000","last_fullsized":"2024-02-18T00:37:39.675533+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.308300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.308300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":452,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310452+0000","last_change":"2024-02-18T00:37:35.841167+0000","last_active":"2024-02-18T00:37:40.310452+0000","last_peered":"2024-02-18T00:37:40.310452+0000","last_clean":"2024-02-18T00:37:40.310452+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:37:40.310452+0000","last_undegraded":"2024-02-18T00:37:40.310452+0000","last_fullsized":"2024-02-18T00:37:40.310452+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:37:35.841050+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.841050+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":367,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676545+0000","last_change":"2024-02-18T00:37:32.766303+0000","last_active":"2024-02-18T00:37:39.676545+0000","last_peered":"2024-02-18T00:37:39.676545+0000","last_clean":"2024-02-18T00:37:39.676545+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:37:39.676545+0000","last_undegraded":"2024-02-18T00:37:39.676545+0000","last_fullsized":"2024-02-18T00:37:39.676545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.766203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.766203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":348,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675835+0000","last_change":"2024-02-18T00:37:33.117248+0000","last_active":"2024-02-18T00:37:39.675835+0000","last_peered":"2024-02-18T00:37:39.675835+0000","last_clean":"2024-02-18T00:37:39.675835+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:37:39.675835+0000","last_undegraded":"2024-02-18T00:37:39.675835+0000","last_fullsized":"2024-02-18T00:37:39.675835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.117173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.117173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":286,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311045+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:37:40.311045+0000","last_peered":"2024-02-18T00:37:40.311045+0000","last_clean":"2024-02-18T00:37:40.311045+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:37:40.311045+0000","last_undegraded":"2024-02-18T00:37:40.311045+0000","last_fullsized":"2024-02-18T00:37:40.311045+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":348,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312718+0000","last_change":"2024-02-18T00:37:31.672777+0000","last_active":"2024-02-18T00:37:40.312718+0000","last_peered":"2024-02-18T00:37:40.312718+0000","last_clean":"2024-02-18T00:37:40.312718+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:37:40.312718+0000","last_undegraded":"2024-02-18T00:37:40.312718+0000","last_fullsized":"2024-02-18T00:37:40.312718+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.672699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.672699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":429,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312100+0000","last_change":"2024-02-18T00:37:37.281742+0000","last_active":"2024-02-18T00:37:40.312100+0000","last_peered":"2024-02-18T00:37:40.312100+0000","last_clean":"2024-02-18T00:37:40.312100+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:37:40.312100+0000","last_undegraded":"2024-02-18T00:37:40.312100+0000","last_fullsized":"2024-02-18T00:37:40.312100+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:37:37.281674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.281674+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":260,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312049+0000","last_change":"2024-02-18T00:37:01.250860+0000","last_active":"2024-02-18T00:37:40.312049+0000","last_peered":"2024-02-18T00:37:40.312049+0000","last_clean":"2024-02-18T00:37:40.312049+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:37:40.312049+0000","last_undegraded":"2024-02-18T00:37:40.312049+0000","last_fullsized":"2024-02-18T00:37:40.312049+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:37:01.250771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.250771+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":347,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675791+0000","last_change":"2024-02-18T00:36:59.979666+0000","last_active":"2024-02-18T00:37:39.675791+0000","last_peered":"2024-02-18T00:37:39.675791+0000","last_clean":"2024-02-18T00:37:39.675791+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:37:39.675791+0000","last_undegraded":"2024-02-18T00:37:39.675791+0000","last_fullsized":"2024-02-18T00:37:39.675791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.979565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.979565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":286,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313574+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:37:40.313574+0000","last_peered":"2024-02-18T00:37:40.313574+0000","last_clean":"2024-02-18T00:37:40.313574+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:37:40.313574+0000","last_undegraded":"2024-02-18T00:37:40.313574+0000","last_fullsized":"2024-02-18T00:37:40.313574+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":341,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312586+0000","last_change":"2024-02-18T00:35:11.781618+0000","last_active":"2024-02-18T00:37:40.312586+0000","last_peered":"2024-02-18T00:37:40.312586+0000","last_clean":"2024-02-18T00:37:40.312586+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:37:40.312586+0000","last_undegraded":"2024-02-18T00:37:40.312586+0000","last_fullsized":"2024-02-18T00:37:40.312586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.781535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.781535+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":367,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310973+0000","last_change":"2024-02-18T00:37:35.212798+0000","last_active":"2024-02-18T00:37:40.310973+0000","last_peered":"2024-02-18T00:37:40.310973+0000","last_clean":"2024-02-18T00:37:40.310973+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:37:40.310973+0000","last_undegraded":"2024-02-18T00:37:40.310973+0000","last_fullsized":"2024-02-18T00:37:40.310973+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:37:35.212727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.212727+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":324,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674883+0000","last_change":"2024-02-18T00:36:34.800419+0000","last_active":"2024-02-18T00:37:39.674883+0000","last_peered":"2024-02-18T00:37:39.674883+0000","last_clean":"2024-02-18T00:37:39.674883+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:37:39.674883+0000","last_undegraded":"2024-02-18T00:37:39.674883+0000","last_fullsized":"2024-02-18T00:37:39.674883+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:34.800327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.800327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":350,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309447+0000","last_change":"2024-02-18T00:37:26.204990+0000","last_active":"2024-02-18T00:37:40.309447+0000","last_peered":"2024-02-18T00:37:40.309447+0000","last_clean":"2024-02-18T00:37:40.309447+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:37:40.309447+0000","last_undegraded":"2024-02-18T00:37:40.309447+0000","last_fullsized":"2024-02-18T00:37:40.309447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:26.204918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:37:26.204918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":347,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677876+0000","last_change":"2024-02-18T00:37:25.868179+0000","last_active":"2024-02-18T00:37:39.677876+0000","last_peered":"2024-02-18T00:37:39.677876+0000","last_clean":"2024-02-18T00:37:39.677876+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:37:39.677876+0000","last_undegraded":"2024-02-18T00:37:39.677876+0000","last_fullsized":"2024-02-18T00:37:39.677876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:25.868065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:25.868065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":296,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309986+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:37:40.309986+0000","last_peered":"2024-02-18T00:37:40.309986+0000","last_clean":"2024-02-18T00:37:40.309986+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:37:40.309986+0000","last_undegraded":"2024-02-18T00:37:40.309986+0000","last_fullsized":"2024-02-18T00:37:40.309986+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":254,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677174+0000","last_change":"2024-02-18T00:37:32.401689+0000","last_active":"2024-02-18T00:37:39.677174+0000","last_peered":"2024-02-18T00:37:39.677174+0000","last_clean":"2024-02-18T00:37:39.677174+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:37:39.677174+0000","last_undegraded":"2024-02-18T00:37:39.677174+0000","last_fullsized":"2024-02-18T00:37:39.677174+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.401618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.401618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":402,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311562+0000","last_change":"2024-02-18T00:36:50.759133+0000","last_active":"2024-02-18T00:37:40.311562+0000","last_peered":"2024-02-18T00:37:40.311562+0000","last_clean":"2024-02-18T00:37:40.311562+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:37:40.311562+0000","last_undegraded":"2024-02-18T00:37:40.311562+0000","last_fullsized":"2024-02-18T00:37:40.311562+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:36:50.759019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.759019+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":281,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309908+0000","last_change":"2024-02-18T00:35:14.996144+0000","last_active":"2024-02-18T00:37:40.309908+0000","last_peered":"2024-02-18T00:37:40.309908+0000","last_clean":"2024-02-18T00:37:40.309908+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:37:40.309908+0000","last_undegraded":"2024-02-18T00:37:40.309908+0000","last_fullsized":"2024-02-18T00:37:40.309908+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.996075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.996075+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":368,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311084+0000","last_change":"2024-02-18T00:37:00.820469+0000","last_active":"2024-02-18T00:37:40.311084+0000","last_peered":"2024-02-18T00:37:40.311084+0000","last_clean":"2024-02-18T00:37:40.311084+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:37:40.311084+0000","last_undegraded":"2024-02-18T00:37:40.311084+0000","last_fullsized":"2024-02-18T00:37:40.311084+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.820283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.820283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":345,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312083+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:37:40.312083+0000","last_peered":"2024-02-18T00:37:40.312083+0000","last_clean":"2024-02-18T00:37:40.312083+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:37:40.312083+0000","last_undegraded":"2024-02-18T00:37:40.312083+0000","last_fullsized":"2024-02-18T00:37:40.312083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":344,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312632+0000","last_change":"2024-02-18T00:36:42.828611+0000","last_active":"2024-02-18T00:37:40.312632+0000","last_peered":"2024-02-18T00:37:40.312632+0000","last_clean":"2024-02-18T00:37:40.312632+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:37:40.312632+0000","last_undegraded":"2024-02-18T00:37:40.312632+0000","last_fullsized":"2024-02-18T00:37:40.312632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.828542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.828542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":287,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310962+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:37:40.310962+0000","last_peered":"2024-02-18T00:37:40.310962+0000","last_clean":"2024-02-18T00:37:40.310962+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:37:40.310962+0000","last_undegraded":"2024-02-18T00:37:40.310962+0000","last_fullsized":"2024-02-18T00:37:40.310962+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":359,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.678214+0000","last_change":"2024-02-18T00:36:57.633211+0000","last_active":"2024-02-18T00:37:39.678214+0000","last_peered":"2024-02-18T00:37:39.678214+0000","last_clean":"2024-02-18T00:37:39.678214+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:37:39.678214+0000","last_undegraded":"2024-02-18T00:37:39.678214+0000","last_fullsized":"2024-02-18T00:37:39.678214+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.633128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.633128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":393,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677245+0000","last_change":"2024-02-18T00:36:45.862082+0000","last_active":"2024-02-18T00:37:39.677245+0000","last_peered":"2024-02-18T00:37:39.677245+0000","last_clean":"2024-02-18T00:37:39.677245+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:37:39.677245+0000","last_undegraded":"2024-02-18T00:37:39.677245+0000","last_fullsized":"2024-02-18T00:37:39.677245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:36:45.861971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.861971+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674248+0000","last_change":"2024-02-18T00:36:36.500976+0000","last_active":"2024-02-18T00:37:39.674248+0000","last_peered":"2024-02-18T00:37:39.674248+0000","last_clean":"2024-02-18T00:37:39.674248+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:37:39.674248+0000","last_undegraded":"2024-02-18T00:37:39.674248+0000","last_fullsized":"2024-02-18T00:37:39.674248+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:36.500906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.500906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":857,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.817011+0000","last_change":"2024-02-18T00:37:37.774470+0000","last_active":"2024-02-18T00:37:39.817011+0000","last_peered":"2024-02-18T00:37:39.817011+0000","last_clean":"2024-02-18T00:37:39.817011+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:37:39.817011+0000","last_undegraded":"2024-02-18T00:37:39.817011+0000","last_fullsized":"2024-02-18T00:37:39.817011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:37:37.774404+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.774404+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":491,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":289,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311873+0000","last_change":"2024-02-18T00:36:42.643527+0000","last_active":"2024-02-18T00:37:40.311873+0000","last_peered":"2024-02-18T00:37:40.311873+0000","last_clean":"2024-02-18T00:37:40.311873+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:37:40.311873+0000","last_undegraded":"2024-02-18T00:37:40.311873+0000","last_fullsized":"2024-02-18T00:37:40.311873+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:36:42.643433+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.643433+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":298,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675537+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:37:39.675537+0000","last_peered":"2024-02-18T00:37:39.675537+0000","last_clean":"2024-02-18T00:37:39.675537+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:37:39.675537+0000","last_undegraded":"2024-02-18T00:37:39.675537+0000","last_fullsized":"2024-02-18T00:37:39.675537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":347,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312127+0000","last_change":"2024-02-18T00:36:56.302683+0000","last_active":"2024-02-18T00:37:40.312127+0000","last_peered":"2024-02-18T00:37:40.312127+0000","last_clean":"2024-02-18T00:37:40.312127+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:37:40.312127+0000","last_undegraded":"2024-02-18T00:37:40.312127+0000","last_fullsized":"2024-02-18T00:37:40.312127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.302615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.302615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":177,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310556+0000","last_change":"2024-02-18T00:36:41.801219+0000","last_active":"2024-02-18T00:37:40.310556+0000","last_peered":"2024-02-18T00:37:40.310556+0000","last_clean":"2024-02-18T00:37:40.310556+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:37:40.310556+0000","last_undegraded":"2024-02-18T00:37:40.310556+0000","last_fullsized":"2024-02-18T00:37:40.310556+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-18T00:36:41.801144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.801144+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":359,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312109+0000","last_change":"2024-02-18T00:36:57.791451+0000","last_active":"2024-02-18T00:37:40.312109+0000","last_peered":"2024-02-18T00:37:40.312109+0000","last_clean":"2024-02-18T00:37:40.312109+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:37:40.312109+0000","last_undegraded":"2024-02-18T00:37:40.312109+0000","last_fullsized":"2024-02-18T00:37:40.312109+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.791347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.791347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":349,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310627+0000","last_change":"2024-02-18T00:36:49.976027+0000","last_active":"2024-02-18T00:37:40.310627+0000","last_peered":"2024-02-18T00:37:40.310627+0000","last_clean":"2024-02-18T00:37:40.310627+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:37:40.310627+0000","last_undegraded":"2024-02-18T00:37:40.310627+0000","last_fullsized":"2024-02-18T00:37:40.310627+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.975957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.975957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":287,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311566+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:37:40.311566+0000","last_peered":"2024-02-18T00:37:40.311566+0000","last_clean":"2024-02-18T00:37:40.311566+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:37:40.311566+0000","last_undegraded":"2024-02-18T00:37:40.311566+0000","last_fullsized":"2024-02-18T00:37:40.311566+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":456,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312345+0000","last_change":"2024-02-18T00:37:29.141966+0000","last_active":"2024-02-18T00:37:40.312345+0000","last_peered":"2024-02-18T00:37:40.312345+0000","last_clean":"2024-02-18T00:37:40.312345+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:37:40.312345+0000","last_undegraded":"2024-02-18T00:37:40.312345+0000","last_fullsized":"2024-02-18T00:37:40.312345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:29.141901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.141901+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":347,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.678125+0000","last_change":"2024-02-18T00:34:40.244840+0000","last_active":"2024-02-18T00:37:39.678125+0000","last_peered":"2024-02-18T00:37:39.678125+0000","last_clean":"2024-02-18T00:37:39.678125+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:37:39.678125+0000","last_undegraded":"2024-02-18T00:37:39.678125+0000","last_fullsized":"2024-02-18T00:37:39.678125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:34:40.244768+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:34:40.244768+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":376,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677567+0000","last_change":"2024-02-18T00:36:53.925296+0000","last_active":"2024-02-18T00:37:39.677567+0000","last_peered":"2024-02-18T00:37:39.677567+0000","last_clean":"2024-02-18T00:37:39.677567+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:37:39.677567+0000","last_undegraded":"2024-02-18T00:37:39.677567+0000","last_fullsized":"2024-02-18T00:37:39.677567+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.925187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.925187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":340,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309692+0000","last_change":"2024-02-18T00:36:46.369602+0000","last_active":"2024-02-18T00:37:40.309692+0000","last_peered":"2024-02-18T00:37:40.309692+0000","last_clean":"2024-02-18T00:37:40.309692+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:37:40.309692+0000","last_undegraded":"2024-02-18T00:37:40.309692+0000","last_fullsized":"2024-02-18T00:37:40.309692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:46.369503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.369503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":289,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676506+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:37:39.676506+0000","last_peered":"2024-02-18T00:37:39.676506+0000","last_clean":"2024-02-18T00:37:39.676506+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:37:39.676506+0000","last_undegraded":"2024-02-18T00:37:39.676506+0000","last_fullsized":"2024-02-18T00:37:39.676506+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":345,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.673666+0000","last_change":"2024-02-18T00:35:13.357912+0000","last_active":"2024-02-18T00:37:39.673666+0000","last_peered":"2024-02-18T00:37:39.673666+0000","last_clean":"2024-02-18T00:37:39.673666+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:37:39.673666+0000","last_undegraded":"2024-02-18T00:37:39.673666+0000","last_fullsized":"2024-02-18T00:37:39.673666+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.357841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.357841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":405,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675326+0000","last_change":"2024-02-18T00:37:39.126531+0000","last_active":"2024-02-18T00:37:39.675326+0000","last_peered":"2024-02-18T00:37:39.675326+0000","last_clean":"2024-02-18T00:37:39.675326+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:37:39.675326+0000","last_undegraded":"2024-02-18T00:37:39.675326+0000","last_fullsized":"2024-02-18T00:37:39.675326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:37:39.126398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.126398+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":382,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676417+0000","last_change":"2024-02-18T00:35:14.311040+0000","last_active":"2024-02-18T00:37:39.676417+0000","last_peered":"2024-02-18T00:37:39.676417+0000","last_clean":"2024-02-18T00:37:39.676417+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:37:39.676417+0000","last_undegraded":"2024-02-18T00:37:39.676417+0000","last_fullsized":"2024-02-18T00:37:39.676417+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:35:14.310921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.310921+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":354,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313369+0000","last_change":"2024-02-18T00:37:00.804782+0000","last_active":"2024-02-18T00:37:40.313369+0000","last_peered":"2024-02-18T00:37:40.313369+0000","last_clean":"2024-02-18T00:37:40.313369+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:37:40.313369+0000","last_undegraded":"2024-02-18T00:37:40.313369+0000","last_fullsized":"2024-02-18T00:37:40.313369+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.804656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":289,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675690+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:37:39.675690+0000","last_peered":"2024-02-18T00:37:39.675690+0000","last_clean":"2024-02-18T00:37:39.675690+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:37:39.675690+0000","last_undegraded":"2024-02-18T00:37:39.675690+0000","last_fullsized":"2024-02-18T00:37:39.675690+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":413,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311055+0000","last_change":"2024-02-18T00:35:26.966401+0000","last_active":"2024-02-18T00:37:40.311055+0000","last_peered":"2024-02-18T00:37:40.311055+0000","last_clean":"2024-02-18T00:37:40.311055+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:37:40.311055+0000","last_undegraded":"2024-02-18T00:37:40.311055+0000","last_fullsized":"2024-02-18T00:37:40.311055+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:35:26.966338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.966338+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":354,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.673647+0000","last_change":"2024-02-18T00:37:27.357689+0000","last_active":"2024-02-18T00:37:39.673647+0000","last_peered":"2024-02-18T00:37:39.673647+0000","last_clean":"2024-02-18T00:37:39.673647+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:37:39.673647+0000","last_undegraded":"2024-02-18T00:37:39.673647+0000","last_fullsized":"2024-02-18T00:37:39.673647+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.357610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.357610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":356,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312425+0000","last_change":"2024-02-18T00:36:55.861115+0000","last_active":"2024-02-18T00:37:40.312425+0000","last_peered":"2024-02-18T00:37:40.312425+0000","last_clean":"2024-02-18T00:37:40.312425+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:37:40.312425+0000","last_undegraded":"2024-02-18T00:37:40.312425+0000","last_fullsized":"2024-02-18T00:37:40.312425+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:55.861044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.861044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":352,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677122+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:37:39.677122+0000","last_peered":"2024-02-18T00:37:39.677122+0000","last_clean":"2024-02-18T00:37:39.677122+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:37:39.677122+0000","last_undegraded":"2024-02-18T00:37:39.677122+0000","last_fullsized":"2024-02-18T00:37:39.677122+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":287,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310838+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:37:40.310838+0000","last_peered":"2024-02-18T00:37:40.310838+0000","last_clean":"2024-02-18T00:37:40.310838+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:37:40.310838+0000","last_undegraded":"2024-02-18T00:37:40.310838+0000","last_fullsized":"2024-02-18T00:37:40.310838+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":411,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311464+0000","last_change":"2024-02-18T00:36:51.948976+0000","last_active":"2024-02-18T00:37:40.311464+0000","last_peered":"2024-02-18T00:37:40.311464+0000","last_clean":"2024-02-18T00:37:40.311464+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:37:40.311464+0000","last_undegraded":"2024-02-18T00:37:40.311464+0000","last_fullsized":"2024-02-18T00:37:40.311464+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:36:51.948907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.948907+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":352,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310508+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:37:40.310508+0000","last_peered":"2024-02-18T00:37:40.310508+0000","last_clean":"2024-02-18T00:37:40.310508+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:37:40.310508+0000","last_undegraded":"2024-02-18T00:37:40.310508+0000","last_fullsized":"2024-02-18T00:37:40.310508+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":346,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312127+0000","last_change":"2024-02-18T00:37:38.687198+0000","last_active":"2024-02-18T00:37:40.312127+0000","last_peered":"2024-02-18T00:37:40.312127+0000","last_clean":"2024-02-18T00:37:40.312127+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:37:40.312127+0000","last_undegraded":"2024-02-18T00:37:40.312127+0000","last_fullsized":"2024-02-18T00:37:40.312127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:38.687129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.687129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":252,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311972+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:37:40.311972+0000","last_peered":"2024-02-18T00:37:40.311972+0000","last_clean":"2024-02-18T00:37:40.311972+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:37:40.311972+0000","last_undegraded":"2024-02-18T00:37:40.311972+0000","last_fullsized":"2024-02-18T00:37:40.311972+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":298,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674364+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:37:39.674364+0000","last_peered":"2024-02-18T00:37:39.674364+0000","last_clean":"2024-02-18T00:37:39.674364+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:37:39.674364+0000","last_undegraded":"2024-02-18T00:37:39.674364+0000","last_fullsized":"2024-02-18T00:37:39.674364+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":419,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313365+0000","last_change":"2024-02-18T00:36:58.819567+0000","last_active":"2024-02-18T00:37:40.313365+0000","last_peered":"2024-02-18T00:37:40.313365+0000","last_clean":"2024-02-18T00:37:40.313365+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:37:40.313365+0000","last_undegraded":"2024-02-18T00:37:40.313365+0000","last_fullsized":"2024-02-18T00:37:40.313365+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:36:58.819503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.819503+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":363,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312028+0000","last_change":"2024-02-18T00:35:12.780429+0000","last_active":"2024-02-18T00:37:40.312028+0000","last_peered":"2024-02-18T00:37:40.312028+0000","last_clean":"2024-02-18T00:37:40.312028+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:37:40.312028+0000","last_undegraded":"2024-02-18T00:37:40.312028+0000","last_fullsized":"2024-02-18T00:37:40.312028+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:12.780361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:12.780361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":455,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311824+0000","last_change":"2024-02-18T00:37:27.642761+0000","last_active":"2024-02-18T00:37:40.311824+0000","last_peered":"2024-02-18T00:37:40.311824+0000","last_clean":"2024-02-18T00:37:40.311824+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:37:40.311824+0000","last_undegraded":"2024-02-18T00:37:40.311824+0000","last_fullsized":"2024-02-18T00:37:40.311824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:27.642661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.642661+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":354,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310895+0000","last_change":"2024-02-18T00:36:44.367116+0000","last_active":"2024-02-18T00:37:40.310895+0000","last_peered":"2024-02-18T00:37:40.310895+0000","last_clean":"2024-02-18T00:37:40.310895+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:37:40.310895+0000","last_undegraded":"2024-02-18T00:37:40.310895+0000","last_fullsized":"2024-02-18T00:37:40.310895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.367032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.367032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":296,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309987+0000","last_change":"2024-02-18T00:35:11.494338+0000","last_active":"2024-02-18T00:37:40.309987+0000","last_peered":"2024-02-18T00:37:40.309987+0000","last_clean":"2024-02-18T00:37:40.309987+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:37:40.309987+0000","last_undegraded":"2024-02-18T00:37:40.309987+0000","last_fullsized":"2024-02-18T00:37:40.309987+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:11.494259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:11.494259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":395,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311638+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:37:40.311638+0000","last_peered":"2024-02-18T00:37:40.311638+0000","last_clean":"2024-02-18T00:37:40.311638+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:37:40.311638+0000","last_undegraded":"2024-02-18T00:37:40.311638+0000","last_fullsized":"2024-02-18T00:37:40.311638+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":351,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312184+0000","last_change":"2024-02-18T00:37:03.239555+0000","last_active":"2024-02-18T00:37:40.312184+0000","last_peered":"2024-02-18T00:37:40.312184+0000","last_clean":"2024-02-18T00:37:40.312184+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:37:40.312184+0000","last_undegraded":"2024-02-18T00:37:40.312184+0000","last_fullsized":"2024-02-18T00:37:40.312184+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.239451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.239451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"341'10","reported_seq":470,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313625+0000","last_change":"2024-02-18T00:36:59.830567+0000","last_active":"2024-02-18T00:37:40.313625+0000","last_peered":"2024-02-18T00:37:40.313625+0000","last_clean":"2024-02-18T00:37:40.313625+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:37:40.313625+0000","last_undegraded":"2024-02-18T00:37:40.313625+0000","last_fullsized":"2024-02-18T00:37:40.313625+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'10","last_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_deep_scrub":"341'10","last_deep_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.830497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":343,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676615+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:37:39.676615+0000","last_peered":"2024-02-18T00:37:39.676615+0000","last_clean":"2024-02-18T00:37:39.676615+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:37:39.676615+0000","last_undegraded":"2024-02-18T00:37:39.676615+0000","last_fullsized":"2024-02-18T00:37:39.676615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":290,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312674+0000","last_change":"2024-02-18T00:37:39.639314+0000","last_active":"2024-02-18T00:37:40.312674+0000","last_peered":"2024-02-18T00:37:40.312674+0000","last_clean":"2024-02-18T00:37:40.312674+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:37:40.312674+0000","last_undegraded":"2024-02-18T00:37:40.312674+0000","last_fullsized":"2024-02-18T00:37:40.312674+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:39.639232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.639232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":367,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676443+0000","last_change":"2024-02-18T00:36:55.970729+0000","last_active":"2024-02-18T00:37:39.676443+0000","last_peered":"2024-02-18T00:37:39.676443+0000","last_clean":"2024-02-18T00:37:39.676443+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:37:39.676443+0000","last_undegraded":"2024-02-18T00:37:39.676443+0000","last_fullsized":"2024-02-18T00:37:39.676443+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:36:55.970628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.970628+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":357,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312087+0000","last_change":"2024-02-18T00:36:44.617522+0000","last_active":"2024-02-18T00:37:40.312087+0000","last_peered":"2024-02-18T00:37:40.312087+0000","last_clean":"2024-02-18T00:37:40.312087+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:37:40.312087+0000","last_undegraded":"2024-02-18T00:37:40.312087+0000","last_fullsized":"2024-02-18T00:37:40.312087+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.617443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.617443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":347,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313293+0000","last_change":"2024-02-18T00:36:50.970267+0000","last_active":"2024-02-18T00:37:40.313293+0000","last_peered":"2024-02-18T00:37:40.313293+0000","last_clean":"2024-02-18T00:37:40.313293+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:37:40.313293+0000","last_undegraded":"2024-02-18T00:37:40.313293+0000","last_fullsized":"2024-02-18T00:37:40.313293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:50.970192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.970192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":340,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310364+0000","last_change":"2024-02-18T00:36:11.346267+0000","last_active":"2024-02-18T00:37:40.310364+0000","last_peered":"2024-02-18T00:37:40.310364+0000","last_clean":"2024-02-18T00:37:40.310364+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:37:40.310364+0000","last_undegraded":"2024-02-18T00:37:40.310364+0000","last_fullsized":"2024-02-18T00:37:40.310364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.346197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.346197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":287,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310814+0000","last_change":"2024-02-18T00:36:42.355194+0000","last_active":"2024-02-18T00:37:40.310814+0000","last_peered":"2024-02-18T00:37:40.310814+0000","last_clean":"2024-02-18T00:37:40.310814+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:37:40.310814+0000","last_undegraded":"2024-02-18T00:37:40.310814+0000","last_fullsized":"2024-02-18T00:37:40.310814+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.355114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.355114+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":371,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676023+0000","last_change":"2024-02-18T00:36:52.343624+0000","last_active":"2024-02-18T00:37:39.676023+0000","last_peered":"2024-02-18T00:37:39.676023+0000","last_clean":"2024-02-18T00:37:39.676023+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:37:39.676023+0000","last_undegraded":"2024-02-18T00:37:39.676023+0000","last_fullsized":"2024-02-18T00:37:39.676023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:36:52.343556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.343556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":353,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676358+0000","last_change":"2024-02-18T00:37:27.830300+0000","last_active":"2024-02-18T00:37:39.676358+0000","last_peered":"2024-02-18T00:37:39.676358+0000","last_clean":"2024-02-18T00:37:39.676358+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:37:39.676358+0000","last_undegraded":"2024-02-18T00:37:39.676358+0000","last_fullsized":"2024-02-18T00:37:39.676358+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.830225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.830225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":349,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676536+0000","last_change":"2024-02-18T00:36:42.857622+0000","last_active":"2024-02-18T00:37:39.676536+0000","last_peered":"2024-02-18T00:37:39.676536+0000","last_clean":"2024-02-18T00:37:39.676536+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:37:39.676536+0000","last_undegraded":"2024-02-18T00:37:39.676536+0000","last_fullsized":"2024-02-18T00:37:39.676536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.857557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.857557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":345,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313850+0000","last_change":"2024-02-18T00:37:37.848598+0000","last_active":"2024-02-18T00:37:40.313850+0000","last_peered":"2024-02-18T00:37:40.313850+0000","last_clean":"2024-02-18T00:37:40.313850+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:37:40.313850+0000","last_undegraded":"2024-02-18T00:37:40.313850+0000","last_fullsized":"2024-02-18T00:37:40.313850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.848524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.848524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":287,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311767+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:37:40.311767+0000","last_peered":"2024-02-18T00:37:40.311767+0000","last_clean":"2024-02-18T00:37:40.311767+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:37:40.311767+0000","last_undegraded":"2024-02-18T00:37:40.311767+0000","last_fullsized":"2024-02-18T00:37:40.311767+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":350,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310305+0000","last_change":"2024-02-18T00:35:16.544731+0000","last_active":"2024-02-18T00:37:40.310305+0000","last_peered":"2024-02-18T00:37:40.310305+0000","last_clean":"2024-02-18T00:37:40.310305+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:37:40.310305+0000","last_undegraded":"2024-02-18T00:37:40.310305+0000","last_fullsized":"2024-02-18T00:37:40.310305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:35:16.544650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.544650+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":350,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312225+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:37:40.312225+0000","last_peered":"2024-02-18T00:37:40.312225+0000","last_clean":"2024-02-18T00:37:40.312225+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:37:40.312225+0000","last_undegraded":"2024-02-18T00:37:40.312225+0000","last_fullsized":"2024-02-18T00:37:40.312225+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":357,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310428+0000","last_change":"2024-02-18T00:36:12.518914+0000","last_active":"2024-02-18T00:37:40.310428+0000","last_peered":"2024-02-18T00:37:40.310428+0000","last_clean":"2024-02-18T00:37:40.310428+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:37:40.310428+0000","last_undegraded":"2024-02-18T00:37:40.310428+0000","last_fullsized":"2024-02-18T00:37:40.310428+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:12.518838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:12.518838+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":254,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.678034+0000","last_change":"2024-02-18T00:37:29.374778+0000","last_active":"2024-02-18T00:37:39.678034+0000","last_peered":"2024-02-18T00:37:39.678034+0000","last_clean":"2024-02-18T00:37:39.678034+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:37:39.678034+0000","last_undegraded":"2024-02-18T00:37:39.678034+0000","last_fullsized":"2024-02-18T00:37:39.678034+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:29.374712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.374712+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":294,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312206+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:37:40.312206+0000","last_peered":"2024-02-18T00:37:40.312206+0000","last_clean":"2024-02-18T00:37:40.312206+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:37:40.312206+0000","last_undegraded":"2024-02-18T00:37:40.312206+0000","last_fullsized":"2024-02-18T00:37:40.312206+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":401,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.314118+0000","last_change":"2024-02-18T00:35:19.511086+0000","last_active":"2024-02-18T00:37:40.314118+0000","last_peered":"2024-02-18T00:37:40.314118+0000","last_clean":"2024-02-18T00:37:40.314118+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:37:40.314118+0000","last_undegraded":"2024-02-18T00:37:40.314118+0000","last_fullsized":"2024-02-18T00:37:40.314118+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:19.511013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.511013+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":352,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313416+0000","last_change":"2024-02-18T00:36:11.551695+0000","last_active":"2024-02-18T00:37:40.313416+0000","last_peered":"2024-02-18T00:37:40.313416+0000","last_clean":"2024-02-18T00:37:40.313416+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:37:40.313416+0000","last_undegraded":"2024-02-18T00:37:40.313416+0000","last_fullsized":"2024-02-18T00:37:40.313416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.551602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":263,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674957+0000","last_change":"2024-02-18T00:36:59.243785+0000","last_active":"2024-02-18T00:37:39.674957+0000","last_peered":"2024-02-18T00:37:39.674957+0000","last_clean":"2024-02-18T00:37:39.674957+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:37:39.674957+0000","last_undegraded":"2024-02-18T00:37:39.674957+0000","last_fullsized":"2024-02-18T00:37:39.674957+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.243699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.243699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":340,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313445+0000","last_change":"2024-02-18T00:35:29.537971+0000","last_active":"2024-02-18T00:37:40.313445+0000","last_peered":"2024-02-18T00:37:40.313445+0000","last_clean":"2024-02-18T00:37:40.313445+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:37:40.313445+0000","last_undegraded":"2024-02-18T00:37:40.313445+0000","last_fullsized":"2024-02-18T00:37:40.313445+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:29.537905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:35:29.537905+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":249,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675551+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:37:39.675551+0000","last_peered":"2024-02-18T00:37:39.675551+0000","last_clean":"2024-02-18T00:37:39.675551+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:37:39.675551+0000","last_undegraded":"2024-02-18T00:37:39.675551+0000","last_fullsized":"2024-02-18T00:37:39.675551+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":408,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311257+0000","last_change":"2024-02-18T00:36:53.401897+0000","last_active":"2024-02-18T00:37:40.311257+0000","last_peered":"2024-02-18T00:37:40.311257+0000","last_clean":"2024-02-18T00:37:40.311257+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:37:40.311257+0000","last_undegraded":"2024-02-18T00:37:40.311257+0000","last_fullsized":"2024-02-18T00:37:40.311257+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:36:53.401819+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.401819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":354,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.309941+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:37:40.309941+0000","last_peered":"2024-02-18T00:37:40.309941+0000","last_clean":"2024-02-18T00:37:40.309941+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:37:40.309941+0000","last_undegraded":"2024-02-18T00:37:40.309941+0000","last_fullsized":"2024-02-18T00:37:40.309941+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":478,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313478+0000","last_change":"2024-02-18T00:36:38.557419+0000","last_active":"2024-02-18T00:37:40.313478+0000","last_peered":"2024-02-18T00:37:40.313478+0000","last_clean":"2024-02-18T00:37:40.313478+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:37:40.313478+0000","last_undegraded":"2024-02-18T00:37:40.313478+0000","last_fullsized":"2024-02-18T00:37:40.313478+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:36:38.557298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.557298+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":263,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312114+0000","last_change":"2024-02-18T00:36:49.727241+0000","last_active":"2024-02-18T00:37:40.312114+0000","last_peered":"2024-02-18T00:37:40.312114+0000","last_clean":"2024-02-18T00:37:40.312114+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:37:40.312114+0000","last_undegraded":"2024-02-18T00:37:40.312114+0000","last_fullsized":"2024-02-18T00:37:40.312114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:36:49.727157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.727157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":246,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311899+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:37:40.311899+0000","last_peered":"2024-02-18T00:37:40.311899+0000","last_clean":"2024-02-18T00:37:40.311899+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:37:40.311899+0000","last_undegraded":"2024-02-18T00:37:40.311899+0000","last_fullsized":"2024-02-18T00:37:40.311899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":384,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313504+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:37:40.313504+0000","last_peered":"2024-02-18T00:37:40.313504+0000","last_clean":"2024-02-18T00:37:40.313504+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:37:40.313504+0000","last_undegraded":"2024-02-18T00:37:40.313504+0000","last_fullsized":"2024-02-18T00:37:40.313504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":351,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313964+0000","last_change":"2024-02-18T00:36:56.850409+0000","last_active":"2024-02-18T00:37:40.313964+0000","last_peered":"2024-02-18T00:37:40.313964+0000","last_clean":"2024-02-18T00:37:40.313964+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:37:40.313964+0000","last_undegraded":"2024-02-18T00:37:40.313964+0000","last_fullsized":"2024-02-18T00:37:40.313964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.850326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.850326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":351,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311336+0000","last_change":"2024-02-18T00:36:52.372043+0000","last_active":"2024-02-18T00:37:40.311336+0000","last_peered":"2024-02-18T00:37:40.311336+0000","last_clean":"2024-02-18T00:37:40.311336+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:37:40.311336+0000","last_undegraded":"2024-02-18T00:37:40.311336+0000","last_fullsized":"2024-02-18T00:37:40.311336+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.371970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.371970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":352,"reported_epoch":382,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.707426+0000","last_change":"2024-02-18T00:37:40.707426+0000","last_active":"2024-02-18T00:37:40.707426+0000","last_peered":"2024-02-18T00:37:40.707426+0000","last_clean":"2024-02-18T00:37:40.707426+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:37:40.707426+0000","last_undegraded":"2024-02-18T00:37:40.707426+0000","last_fullsized":"2024-02-18T00:37:40.707426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.707359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.707359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":302,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.674363+0000","last_change":"2024-02-18T00:35:25.477704+0000","last_active":"2024-02-18T00:37:39.674363+0000","last_peered":"2024-02-18T00:37:39.674363+0000","last_clean":"2024-02-18T00:37:39.674363+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:37:39.674363+0000","last_undegraded":"2024-02-18T00:37:39.674363+0000","last_fullsized":"2024-02-18T00:37:39.674363+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:35:25.477612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:25.477612+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":382,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677313+0000","last_change":"2024-02-18T00:37:39.385746+0000","last_active":"2024-02-18T00:37:39.677313+0000","last_peered":"2024-02-18T00:37:39.677313+0000","last_clean":"2024-02-18T00:37:39.677313+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:37:39.677313+0000","last_undegraded":"2024-02-18T00:37:39.677313+0000","last_fullsized":"2024-02-18T00:37:39.677313+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:39.385675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.385675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":368,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312666+0000","last_change":"2024-02-18T00:37:36.615647+0000","last_active":"2024-02-18T00:37:40.312666+0000","last_peered":"2024-02-18T00:37:40.312666+0000","last_clean":"2024-02-18T00:37:40.312666+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:37:40.312666+0000","last_undegraded":"2024-02-18T00:37:40.312666+0000","last_fullsized":"2024-02-18T00:37:40.312666+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.615568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.615568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":349,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676673+0000","last_change":"2024-02-18T00:36:35.808459+0000","last_active":"2024-02-18T00:37:39.676673+0000","last_peered":"2024-02-18T00:37:39.676673+0000","last_clean":"2024-02-18T00:37:39.676673+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:37:39.676673+0000","last_undegraded":"2024-02-18T00:37:39.676673+0000","last_fullsized":"2024-02-18T00:37:39.676673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.808381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.808381+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":348,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676617+0000","last_change":"2024-02-18T00:37:34.122873+0000","last_active":"2024-02-18T00:37:39.676617+0000","last_peered":"2024-02-18T00:37:39.676617+0000","last_clean":"2024-02-18T00:37:39.676617+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:37:39.676617+0000","last_undegraded":"2024-02-18T00:37:39.676617+0000","last_fullsized":"2024-02-18T00:37:39.676617+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.122764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.122764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":292,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.562421+0000","last_change":"2024-02-18T00:37:40.562387+0000","last_active":"2024-02-18T00:37:40.562421+0000","last_peered":"2024-02-18T00:37:40.562421+0000","last_clean":"2024-02-18T00:37:40.562421+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:37:40.562421+0000","last_undegraded":"2024-02-18T00:37:40.562421+0000","last_fullsized":"2024-02-18T00:37:40.562421+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.562306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.562306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":382,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312058+0000","last_change":"2024-02-18T00:35:17.514102+0000","last_active":"2024-02-18T00:37:40.312058+0000","last_peered":"2024-02-18T00:37:40.312058+0000","last_clean":"2024-02-18T00:37:40.312058+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:37:40.312058+0000","last_undegraded":"2024-02-18T00:37:40.312058+0000","last_fullsized":"2024-02-18T00:37:40.312058+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:35:17.514023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:17.514023+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":371,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313218+0000","last_change":"2024-02-18T00:36:52.928195+0000","last_active":"2024-02-18T00:37:40.313218+0000","last_peered":"2024-02-18T00:37:40.313218+0000","last_clean":"2024-02-18T00:37:40.313218+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:37:40.313218+0000","last_undegraded":"2024-02-18T00:37:40.313218+0000","last_fullsized":"2024-02-18T00:37:40.313218+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:36:52.928076+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.928076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":350,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.676230+0000","last_change":"2024-02-18T00:37:03.248349+0000","last_active":"2024-02-18T00:37:39.676230+0000","last_peered":"2024-02-18T00:37:39.676230+0000","last_clean":"2024-02-18T00:37:39.676230+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:37:39.676230+0000","last_undegraded":"2024-02-18T00:37:39.676230+0000","last_fullsized":"2024-02-18T00:37:39.676230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.248277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.248277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":340,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311429+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:37:40.311429+0000","last_peered":"2024-02-18T00:37:40.311429+0000","last_clean":"2024-02-18T00:37:40.311429+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:37:40.311429+0000","last_undegraded":"2024-02-18T00:37:40.311429+0000","last_fullsized":"2024-02-18T00:37:40.311429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":296,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311615+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:37:40.311615+0000","last_peered":"2024-02-18T00:37:40.311615+0000","last_clean":"2024-02-18T00:37:40.311615+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:37:40.311615+0000","last_undegraded":"2024-02-18T00:37:40.311615+0000","last_fullsized":"2024-02-18T00:37:40.311615+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":405,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675611+0000","last_change":"2024-02-18T00:35:19.264658+0000","last_active":"2024-02-18T00:37:39.675611+0000","last_peered":"2024-02-18T00:37:39.675611+0000","last_clean":"2024-02-18T00:37:39.675611+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:37:39.675611+0000","last_undegraded":"2024-02-18T00:37:39.675611+0000","last_fullsized":"2024-02-18T00:37:39.675611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:35:19.264580+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":245,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311944+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:37:40.311944+0000","last_peered":"2024-02-18T00:37:40.311944+0000","last_clean":"2024-02-18T00:37:40.311944+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:37:40.311944+0000","last_undegraded":"2024-02-18T00:37:40.311944+0000","last_fullsized":"2024-02-18T00:37:40.311944+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":364,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311010+0000","last_change":"2024-02-18T00:36:45.773797+0000","last_active":"2024-02-18T00:37:40.311010+0000","last_peered":"2024-02-18T00:37:40.311010+0000","last_clean":"2024-02-18T00:37:40.311010+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:37:40.311010+0000","last_undegraded":"2024-02-18T00:37:40.311010+0000","last_fullsized":"2024-02-18T00:37:40.311010+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.773695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.773695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":354,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310188+0000","last_change":"2024-02-18T00:36:45.412472+0000","last_active":"2024-02-18T00:37:40.310188+0000","last_peered":"2024-02-18T00:37:40.310188+0000","last_clean":"2024-02-18T00:37:40.310188+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:37:40.310188+0000","last_undegraded":"2024-02-18T00:37:40.310188+0000","last_fullsized":"2024-02-18T00:37:40.310188+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.412406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.412406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":289,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.677866+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:37:39.677866+0000","last_peered":"2024-02-18T00:37:39.677866+0000","last_clean":"2024-02-18T00:37:39.677866+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:37:39.677866+0000","last_undegraded":"2024-02-18T00:37:39.677866+0000","last_fullsized":"2024-02-18T00:37:39.677866+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":380,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675871+0000","last_change":"2024-02-18T00:36:51.303043+0000","last_active":"2024-02-18T00:37:39.675871+0000","last_peered":"2024-02-18T00:37:39.675871+0000","last_clean":"2024-02-18T00:37:39.675871+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:37:39.675871+0000","last_undegraded":"2024-02-18T00:37:39.675871+0000","last_fullsized":"2024-02-18T00:37:39.675871+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:36:51.302920+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.302920+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":252,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311659+0000","last_change":"2024-02-18T00:36:52.758965+0000","last_active":"2024-02-18T00:37:40.311659+0000","last_peered":"2024-02-18T00:37:40.311659+0000","last_clean":"2024-02-18T00:37:40.311659+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:37:40.311659+0000","last_undegraded":"2024-02-18T00:37:40.311659+0000","last_fullsized":"2024-02-18T00:37:40.311659+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.758904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.758904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":249,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675587+0000","last_change":"2024-02-18T00:36:49.277109+0000","last_active":"2024-02-18T00:37:39.675587+0000","last_peered":"2024-02-18T00:37:39.675587+0000","last_clean":"2024-02-18T00:37:39.675587+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:37:39.675587+0000","last_undegraded":"2024-02-18T00:37:39.675587+0000","last_fullsized":"2024-02-18T00:37:39.675587+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.276979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.276979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":344,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310456+0000","last_change":"2024-02-18T00:36:40.825174+0000","last_active":"2024-02-18T00:37:40.310456+0000","last_peered":"2024-02-18T00:37:40.310456+0000","last_clean":"2024-02-18T00:37:40.310456+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:37:40.310456+0000","last_undegraded":"2024-02-18T00:37:40.310456+0000","last_fullsized":"2024-02-18T00:37:40.310456+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:40.825055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:36:40.825055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":295,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.312312+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:37:40.312312+0000","last_peered":"2024-02-18T00:37:40.312312+0000","last_clean":"2024-02-18T00:37:40.312312+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:37:40.312312+0000","last_undegraded":"2024-02-18T00:37:40.312312+0000","last_fullsized":"2024-02-18T00:37:40.312312+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":383,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675015+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:37:39.675015+0000","last_peered":"2024-02-18T00:37:39.675015+0000","last_clean":"2024-02-18T00:37:39.675015+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:37:39.675015+0000","last_undegraded":"2024-02-18T00:37:39.675015+0000","last_fullsized":"2024-02-18T00:37:39.675015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":346,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.310862+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:37:40.310862+0000","last_peered":"2024-02-18T00:37:40.310862+0000","last_clean":"2024-02-18T00:37:40.310862+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:37:40.310862+0000","last_undegraded":"2024-02-18T00:37:40.310862+0000","last_fullsized":"2024-02-18T00:37:40.310862+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":358,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675602+0000","last_change":"2024-02-18T00:36:49.936731+0000","last_active":"2024-02-18T00:37:39.675602+0000","last_peered":"2024-02-18T00:37:39.675602+0000","last_clean":"2024-02-18T00:37:39.675602+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:37:39.675602+0000","last_undegraded":"2024-02-18T00:37:39.675602+0000","last_fullsized":"2024-02-18T00:37:39.675602+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.936644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.936644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":343,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675350+0000","last_change":"2024-02-18T00:34:35.250611+0000","last_active":"2024-02-18T00:37:39.675350+0000","last_peered":"2024-02-18T00:37:39.675350+0000","last_clean":"2024-02-18T00:37:39.675350+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:37:39.675350+0000","last_undegraded":"2024-02-18T00:37:39.675350+0000","last_fullsized":"2024-02-18T00:37:39.675350+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:34:35.250532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:34:35.250532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":286,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313737+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:37:40.313737+0000","last_peered":"2024-02-18T00:37:40.313737+0000","last_clean":"2024-02-18T00:37:40.313737+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:37:40.313737+0000","last_undegraded":"2024-02-18T00:37:40.313737+0000","last_fullsized":"2024-02-18T00:37:40.313737+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":431,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313925+0000","last_change":"2024-02-18T00:36:46.942689+0000","last_active":"2024-02-18T00:37:40.313925+0000","last_peered":"2024-02-18T00:37:40.313925+0000","last_clean":"2024-02-18T00:37:40.313925+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:37:40.313925+0000","last_undegraded":"2024-02-18T00:37:40.313925+0000","last_fullsized":"2024-02-18T00:37:40.313925+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:36:46.942583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.942583+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":355,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.311399+0000","last_change":"2024-02-18T00:35:13.072713+0000","last_active":"2024-02-18T00:37:40.311399+0000","last_peered":"2024-02-18T00:37:40.311399+0000","last_clean":"2024-02-18T00:37:40.311399+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:37:40.311399+0000","last_undegraded":"2024-02-18T00:37:40.311399+0000","last_fullsized":"2024-02-18T00:37:40.311399+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:13.072639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:13.072639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":350,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:39.675194+0000","last_change":"2024-02-18T00:37:36.119369+0000","last_active":"2024-02-18T00:37:39.675194+0000","last_peered":"2024-02-18T00:37:39.675194+0000","last_clean":"2024-02-18T00:37:39.675194+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:37:39.675194+0000","last_undegraded":"2024-02-18T00:37:39.675194+0000","last_fullsized":"2024-02-18T00:37:39.675194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.119291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.119291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":354,"reported_epoch":381,"state":"active+clean","last_fresh":"2024-02-18T00:37:40.313006+0000","last_change":"2024-02-18T00:37:34.860293+0000","last_active":"2024-02-18T00:37:40.313006+0000","last_peered":"2024-02-18T00:37:40.313006+0000","last_clean":"2024-02-18T00:37:40.313006+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:37:40.313006+0000","last_undegraded":"2024-02-18T00:37:40.313006+0000","last_fullsized":"2024-02-18T00:37:40.313006+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.860214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.860214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":204,"num_pg":32,"stat_sum":{"num_bytes":32,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":4,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":2,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":96,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001420,"num_objects":148,"num_object_clones":2,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":104,"num_read_kb":7,"num_write":573,"num_write_kb":4474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13819904,"data_stored":12004293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1823,"ondisk_log_size":1823,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":491,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266793,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463023,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309416,"kb_used_data":8096,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435736,"statfs":{"total":95995035648,"available":95678193664,"internally_reserved":0,"allocated":8290304,"data_stored":6147829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":2.722,"15min":1.2370000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.1799999999999999,"5min":69.194999999999993,"15min":69.194999999999993},"last":4.0129999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.1679999999999999,"15min":0.72299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1,"5min":10.433,"15min":10.433},"last":1.7170000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":2.9569999999999999,"15min":1.3200000000000001},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.97199999999999998,"5min":76.108999999999995,"15min":76.108999999999995},"last":1.788},{"interface":"front","average":{"1min":0.56399999999999995,"5min":2.9529999999999998,"15min":1.3109999999999999},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":1.216,"5min":71.872,"15min":71.872},"last":1.1519999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.395,"15min":0.80300000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.141,"5min":39.939999999999998,"15min":39.939999999999998},"last":1.196},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.8560000000000001,"15min":0.96799999999999997},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.9930000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":3.1600000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.391,"15min":0.80300000000000005},"min":{"1min":0.29999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2629999999999999,"5min":21.219000000000001,"15min":21.219000000000001},"last":8.3420000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.2929999999999999,"15min":0.77600000000000002},"min":{"1min":0.37,"5min":0.309,"15min":0.309},"max":{"1min":1.0149999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":1.506}]},{"osd":4,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68500000000000005,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.496,"5min":1.1910000000000001,"15min":0.66900000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85099999999999998,"5min":35.18,"15min":35.18},"last":0.54800000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.73199999999999998,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.673,"5min":10.606,"15min":10.606},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.496,"5min":1.224,"15min":0.69399999999999995},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70899999999999996,"5min":35.119,"15min":35.119},"last":0.45600000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63500000000000001,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73299999999999998,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.1220000000000001,"15min":0.72699999999999998},"min":{"1min":0.42799999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.745,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.50800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305524,"kb_used_data":4204,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439628,"statfs":{"total":95995035648,"available":95682179072,"internally_reserved":0,"allocated":4304896,"data_stored":2157331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":1.228,"15min":0.63},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47399999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.437},{"interface":"front","average":{"1min":0.35799999999999998,"5min":1.3160000000000001,"15min":0.66300000000000003},"min":{"1min":0.253,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56999999999999995,"5min":43.137,"15min":43.137},"last":0.35999999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":1.528,"15min":0.75900000000000001},"min":{"1min":0.22900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.78300000000000003,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.375},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.95699999999999996,"15min":0.58599999999999997},"min":{"1min":0.27000000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.623,"5min":17.596,"15min":17.596},"last":0.46500000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:36:45 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":1.357,"15min":0.751},"min":{"1min":0.26500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56999999999999995,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":1.3500000000000001,"15min":0.77200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71699999999999997,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.48599999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":1.1319999999999999,"15min":0.77700000000000002},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73999999999999999,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":1.1719999999999999,"15min":0.77100000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.9790000000000001,"5min":12.75,"15min":12.75},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:37:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.127,"15min":0.77600000000000002},"min":{"1min":0.40000000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.129,"15min":0.78800000000000003},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80200000000000005,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.622}]},{"osd":6,"last update":"Sun Feb 18 00:37:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":1.165,"15min":0.81000000000000005},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":12.834,"15min":12.834},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.1399999999999999,"15min":0.80000000000000004},"min":{"1min":0.44700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83399999999999996,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":1.143,"15min":0.80400000000000005},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":1.123,"15min":0.81699999999999995},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.0859999999999999,"5min":12.805,"15min":12.805},"last":0.70799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301428,"kb_used_data":4204,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443724,"statfs":{"total":95995035648,"available":95686373376,"internally_reserved":0,"allocated":4304896,"data_stored":2146725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":2.2749999999999999,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.2330000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":1.5589999999999999,"15min":0.75900000000000001},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":4.577,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.91700000000000004}]},{"osd":2,"last update":"Sun Feb 18 00:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.8759999999999999,"15min":0.88900000000000001},"min":{"1min":0.27400000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":4.54,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.40699999999999997,"5min":1.8380000000000001,"15min":0.871},"min":{"1min":0.25,"5min":0.23100000000000001,"15min":0.224},"max":{"1min":0.94899999999999995,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.434}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.149,"15min":0.67600000000000005},"min":{"1min":0.255,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.497,"5min":19.006,"15min":19.006},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":2.2610000000000001,"15min":1.0780000000000001},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":4.4489999999999998,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.86699999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:37:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.651,"15min":0.91600000000000004},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.77500000000000002,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":1.7370000000000001,"15min":0.97999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.2919999999999998,"5min":90.070999999999998,"15min":90.070999999999998},"last":1.018}]},{"osd":5,"last update":"Sun Feb 18 00:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.673,"15min":0.95199999999999996},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.3760000000000003,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.8220000000000001,"15min":0.97499999999999998},"min":{"1min":0.42699999999999999,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":1.5640000000000001,"5min":57.991,"15min":57.991},"last":0.46700000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.2270000000000001,"15min":0.79500000000000004},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.746,"15min":0.96799999999999997},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.343,"5min":89.989999999999995,"15min":89.989999999999995},"last":1.046}]},{"osd":7,"last update":"Sun Feb 18 00:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.9059999999999999,"15min":1.0449999999999999},"min":{"1min":0.42099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.4100000000000001,"5min":90.025000000000006,"15min":90.025000000000006},"last":1.073},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.798,"15min":0.99299999999999999},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.83899999999999997,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.502}]}]},{"osd":2,"up_from":18,"seq":77309411459,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301504,"kb_used_data":4280,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443648,"statfs":{"total":95995035648,"available":95686295552,"internally_reserved":0,"allocated":4382720,"data_stored":2147079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.70699999999999996,"15min":0.48899999999999999},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46100000000000002,"5min":12.144,"15min":12.144},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.056,"15min":0.60699999999999998},"min":{"1min":0.27000000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56200000000000006,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.55800000000000005}]},{"osd":1,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.69699999999999995,"15min":0.495},"min":{"1min":0.28499999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.81399999999999995,"15min":0.54400000000000004},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.64300000000000002,"5min":18.919,"15min":18.919},"last":0.49199999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:36:47 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.91400000000000003,"15min":0.61099999999999999},"min":{"1min":0.33200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.503,"5min":22.712,"15min":22.712},"last":0.436},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.91000000000000003,"15min":0.621},"min":{"1min":0.30499999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50700000000000001,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.89000000000000001,"15min":0.68500000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.875,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.623},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.84499999999999997,"15min":0.67600000000000005},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76000000000000001,"5min":10.016,"15min":10.016},"last":0.52100000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.75800000000000001,"15min":0.65000000000000002},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92800000000000005,"5min":6.766,"15min":6.766},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.95999999999999996,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.56699999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.79400000000000004,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":6.806,"15min":6.806},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.89700000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.58999999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.80900000000000005,"15min":0.72399999999999998},"min":{"1min":0.44500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.90000000000000002,"15min":0.746},"min":{"1min":0.495,"5min":0.372,"15min":0.372},"max":{"1min":0.96399999999999997,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.55000000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149821,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":321792,"kb_used_data":8184,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93423360,"statfs":{"total":95995035648,"available":95665520640,"internally_reserved":0,"allocated":8380416,"data_stored":6137528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.65900000000000003,"15min":0.47699999999999998},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54100000000000004,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.76000000000000001,"15min":0.52400000000000002},"min":{"1min":0.251,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51900000000000002,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.60699999999999998,"15min":0.47999999999999998},"min":{"1min":0.28899999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.54400000000000004,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":1.206,"15min":0.66800000000000004},"min":{"1min":0.25600000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59399999999999997,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:36:43 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":1.0720000000000001,"15min":0.64800000000000002},"min":{"1min":0.28000000000000003,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.71499999999999997,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.505},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.55100000000000005,"15min":0.46800000000000003},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.23999999999999999},"max":{"1min":0.58299999999999996,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.91800000000000004,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.8410000000000002,"5min":10.131,"15min":10.131},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.70399999999999996,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.86499999999999999,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.77500000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:37:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.69499999999999995,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.81000000000000005,"15min":0.67800000000000005},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78800000000000003,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:36:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":1.05,"15min":0.749},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.8940000000000001,"5min":10.189,"15min":10.189},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.74099999999999999,"15min":0.66400000000000003},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:37:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.80000000000000004,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89300000000000002,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.96199999999999997,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.39800000000000002},"max":{"1min":3.9289999999999998,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.83199999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986296,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309632,"kb_used_data":4216,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435520,"statfs":{"total":95995035648,"available":95677972480,"internally_reserved":0,"allocated":4317184,"data_stored":2147988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":1.542,"15min":0.91100000000000003},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.92600000000000005,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.498,"5min":1.4950000000000001,"15min":0.872},"min":{"1min":0.28199999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.53000000000000003}]},{"osd":1,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.80900000000000005,"15min":0.64500000000000002},"min":{"1min":0.33900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89800000000000002,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":1.581,"15min":0.90100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.88,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.60199999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.78000000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.022,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.90000000000000002,"15min":0.68100000000000005},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80200000000000005,"5min":17.77,"15min":17.77},"last":0.495}]},{"osd":3,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.68999999999999995,"15min":0.627},"min":{"1min":0.318,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.82199999999999995,"5min":7.218,"15min":7.218},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":1.091,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.309,"15min":0.309},"max":{"1min":1.2350000000000001,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.435,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42799999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:37:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45600000000000002,"15min":0.41899999999999998},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.30099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65700000000000003,"5min":0.751,"15min":0.751},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.76900000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.34899999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73499999999999999,"5min":0.79700000000000004,"15min":0.83299999999999996},"last":0.63300000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724659,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309336,"kb_used_data":8016,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435816,"statfs":{"total":95995035648,"available":95678275584,"internally_reserved":0,"allocated":8208384,"data_stored":6147816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.55100000000000005},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":1.4630000000000001,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.624},"min":{"1min":0.29099999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":4.4779999999999998,"5min":12.744,"15min":12.744},"last":0.50900000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.71599999999999997,"15min":0.624},"min":{"1min":0.33800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.151,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.77100000000000002,"15min":0.63100000000000001},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":1.127,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.46000000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67000000000000004,"15min":0.61399999999999999},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.77000000000000002,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.72899999999999998,"15min":0.64200000000000002},"min":{"1min":0.41999999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.1000000000000001,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.435}]},{"osd":3,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.73599999999999999,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83099999999999996,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.39100000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.58999999999999997,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.67200000000000004,"5min":11.818,"15min":11.818},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50600000000000001,"15min":0.433},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72999999999999998,"5min":4.274,"15min":4.274},"last":0.51500000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":12.362,"5min":18.605,"15min":18.605},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.46,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.48699999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.47399999999999998},"min":{"1min":0.376,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68899999999999995,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56399999999999995,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.70299999999999996,"5min":4.649,"15min":4.649},"last":0.499}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4329472,"data_stored":4000781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4145152,"data_stored":4000342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4247552,"data_stored":4000585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":32,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":24,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:37:45.171 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:37:45.172 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:37:45.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.881103+0000 osd.3 (osd.3) 215 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:37:45.181 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.883167+0000 osd.3 (osd.3) 216 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:43.887019+0000 mgr.y (mgr.24454) 365 : cluster [DBG] pgmap v565: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.196944+0000 osd.4 (osd.4) 225 : cluster [DBG] 25.1c deep-scrub starts 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.199937+0000 osd.4 (osd.4) 226 : cluster [DBG] 25.1c deep-scrub ok 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.242554+0000 mon.a (mon.0) 2414 : audit [DBG] from='client.? 172.21.15.59:0/1768803730' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.334575+0000 osd.2 (osd.2) 171 : cluster [DBG] 5.8 deep-scrub starts 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.335384+0000 osd.2 (osd.2) 172 : cluster [DBG] 5.8 deep-scrub ok 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.672178+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.672265+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.672332+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e386: 8 total, 8 up, 7 in 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.675060+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.675700+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:37:45.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.695429+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.695812+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.696416+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.696644+0000 mon.a (mon.0) 2420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.697078+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:44.697383+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.881103+0000 osd.3 (osd.3) 215 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.883167+0000 osd.3 (osd.3) 216 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:43.887019+0000 mgr.y (mgr.24454) 365 : cluster [DBG] pgmap v565: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T00:37:45.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.196944+0000 osd.4 (osd.4) 225 : cluster [DBG] 25.1c deep-scrub starts 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.199937+0000 osd.4 (osd.4) 226 : cluster [DBG] 25.1c deep-scrub ok 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.242554+0000 mon.a (mon.0) 2414 : audit [DBG] from='client.? 172.21.15.59:0/1768803730' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.334575+0000 osd.2 (osd.2) 171 : cluster [DBG] 5.8 deep-scrub starts 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.335384+0000 osd.2 (osd.2) 172 : cluster [DBG] 5.8 deep-scrub ok 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.672178+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.672265+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.672332+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e386: 8 total, 8 up, 7 in 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.675060+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.675700+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.695429+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.695812+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.696416+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.184 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.696644+0000 mon.a (mon.0) 2420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.697078+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:45.185 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:44.697383+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 -- 172.21.15.59:0/3740649700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fd510 msgr2=0x7f4ae80f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/3740649700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fd510 0x7f4ae80f6d40 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 rx=0x7f4ad4004870 tx=0x7f4ad4009cb0).stop 2024-02-18T00:37:45.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 -- 172.21.15.59:0/3740649700 shutdown_connections 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/3740649700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ae80f7400 0x7f4ae8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/3740649700 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ae80fcbb0 0x7f4ae80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/3740649700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fd510 0x7f4ae80f6d40 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 -- 172.21.15.59:0/3740649700 >> 172.21.15.59:0/3740649700 conn(0x7f4ae80f2820 msgr2=0x7f4ae80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 -- 172.21.15.59:0/3740649700 shutdown_connections 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 -- 172.21.15.59:0/3740649700 wait complete. 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 librados: starting msgr at 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 librados: starting objecter 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 Processor -- start 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 -- start start 2024-02-18T00:37:45.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 librados: setting wanted keys 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.238+0000 7f4aeeaad700 1 librados: calling monclient init 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aeeaad700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ae80f7400 0x7f4ae8109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aeeaad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fcbb0 0x7f4ae8109880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aeeaad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ae80fd510 0x7f4ae8109e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aeeaad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae8114580 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aeeaad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae81146f0 con 0x7f4ae80fd510 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aeeaad700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae81148b0 con 0x7f4ae80f7400 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aec849700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ae80f7400 0x7f4ae8109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fcbb0 0x7f4ae8109880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aec849700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ae80f7400 0x7f4ae8109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59650/0 (socket says 172.21.15.59:59650) 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aec849700 1 -- 172.21.15.59:0/1768803730 learned_addr learned my addr 172.21.15.59:0/1768803730 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aed04a700 1 --2- 172.21.15.59:0/1768803730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ae80fd510 0x7f4ae8109e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae7fff700 1 -- 172.21.15.59:0/1768803730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ae80f7400 msgr2=0x7f4ae8109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae7fff700 1 --2- 172.21.15.59:0/1768803730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ae80f7400 0x7f4ae8109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae7fff700 1 -- 172.21.15.59:0/1768803730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ae80fd510 msgr2=0x7f4ae8109e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae7fff700 1 --2- 172.21.15.59:0/1768803730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ae80fd510 0x7f4ae8109e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae7fff700 1 -- 172.21.15.59:0/1768803730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc005040 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae7fff700 1 --2- 172.21.15.59:0/1768803730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fcbb0 0x7f4ae8109880 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f4ad4004870 tx=0x7f4ad400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae5f76700 10 client.?.objecter ms_handle_connect 0x7f4ae80fcbb0 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae5f76700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400e070 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad4005040 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ad4013da0 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.239+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4019070 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.240+0000 7f4aeeaad700 10 client.31242.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.240+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae8114da0 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.240+0000 7f4aeeaad700 1 librados: init done 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.241+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f4ae805fa60 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.241+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4ad401e420 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4ad8062c90 0x7f4ad8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(385..385 src has 180..385) v4 ==== 8154+0+0 (secure 0 0 0) 0x7f4ad407dfb0 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 osd_map(385..385 src has 180..385) v4 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 3 client.31242.objecter handle_osd_map got epochs [385,385] > 0 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 3 client.31242.objecter handle_osd_map decoding full epoch 385 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4ad8062c90 0x7f4ad8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 20 client.31242.objecter dump_active .. 0 homeless 2024-02-18T00:37:45.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v385) v1 ==== 54+0+16 (secure 0 0 0) 0x7f4ad407e490 con 0x7f4ae80fcbb0 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ad8062c90 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.242+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4ad8062c90 0x7f4ad8065150 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 rx=0x7f4adc007fd0 tx=0x7f4adc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 1 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4ae8028540 0x7f4ae80fd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 1 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 0 1 2024-02-18T00:37:45.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae805ee80 osd=0 4 2024-02-18T00:37:45.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 10 client.31242.objecter _send_command 1 2024-02-18T00:37:45.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae81060e0 con 0x7f4ae8028540 2024-02-18T00:37:45.286 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:37:45.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aed04a700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4ae8028540 0x7f4ae80fd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae8028540 2024-02-18T00:37:45.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.243+0000 7f4aed04a700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4ae8028540 0x7f4ae80fd9b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.244+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae81060e0 con 0x7f4ae8028540 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.244+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.244+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 1 = system:0 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.244+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae805ee80 osd=0 4 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.244+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 0 1 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 2 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 2 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae805ee80 osd=0 3 2024-02-18T00:37:45.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 0 2 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae805ee80 osd=0 4 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 10 client.31242.objecter _send_command 2 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.264+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae805a040 con 0x7f4ae8028540 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae805a040 con 0x7f4ae8028540 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae805ee80 osd=0 4 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 0 2 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 3 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6805/26279110 conn(0x7f4ae8027b90 0x7f4ae8027f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 3 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 1 3 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae80ff020 osd=1 4 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 10 client.31242.objecter _send_command 3 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae80ff190 con 0x7f4ae8027b90 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.266+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6805/26279110 conn(0x7f4ae8027b90 0x7f4ae8027f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.267+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae8027b90 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.267+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6805/26279110 conn(0x7f4ae8027b90 0x7f4ae8027f90 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.267+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae80ff190 con 0x7f4ae8027b90 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.267+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:37:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.267+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 3 = system:0 2024-02-18T00:37:45.290 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:37:45.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.267+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae80ff020 osd=1 4 2024-02-18T00:37:45.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.267+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 1 3 2024-02-18T00:37:45.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 4 2024-02-18T00:37:45.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 4 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae80ff020 osd=1 3 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 1 4 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae80ff020 osd=1 4 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 10 client.31242.objecter _send_command 4 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.284+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae8027600 con 0x7f4ae8027b90 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae8027600 con 0x7f4ae8027b90 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae80ff020 osd=1 4 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 1 4 2024-02-18T00:37:45.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 5 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4ae80feb80 0x7f4ae811c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 5 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 2 5 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8027a10 osd=2 4 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 10 client.31242.objecter _send_command 5 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.285+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae811edc0 con 0x7f4ae80feb80 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae7fff700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4ae80feb80 0x7f4ae811c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae80feb80 2024-02-18T00:37:45.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae7fff700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4ae80feb80 0x7f4ae811c170 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae811edc0 con 0x7f4ae80feb80 2024-02-18T00:37:45.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:37:45.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 5 = system:0 2024-02-18T00:37:45.293 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:37:45.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae8027a10 osd=2 4 2024-02-18T00:37:45.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.286+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 2 5 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 6 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 6 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae8027a10 osd=2 3 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 2 6 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8027a10 osd=2 4 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 10 client.31242.objecter _send_command 6 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.302+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae80fe140 con 0x7f4ae80feb80 2024-02-18T00:37:45.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae80fe140 con 0x7f4ae80feb80 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae8027a10 osd=2 4 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 2 6 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 7 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4ae8107180 0x7f4ae811e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 7 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 3 7 2024-02-18T00:37:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81197b0 osd=3 4 2024-02-18T00:37:45.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 10 client.31242.objecter _send_command 7 2024-02-18T00:37:45.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.303+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae811ec50 con 0x7f4ae8107180 2024-02-18T00:37:45.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4aed04a700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4ae8107180 0x7f4ae811e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae8107180 2024-02-18T00:37:45.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4aed04a700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4ae8107180 0x7f4ae811e670 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.296 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:37:45.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae811ec50 con 0x7f4ae8107180 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 7 = system:0 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae81197b0 osd=3 4 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.304+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 3 7 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 8 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 8 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae81197b0 osd=3 3 2024-02-18T00:37:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 3 8 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81197b0 osd=3 4 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 10 client.31242.objecter _send_command 8 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.319+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae81075a0 con 0x7f4ae8107180 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae81075a0 con 0x7f4ae8107180 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae81197b0 osd=3 4 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 3 8 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 9 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4ae8107cb0 0x7f4ae81080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 9 2024-02-18T00:37:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 4 9 2024-02-18T00:37:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81075a0 osd=4 4 2024-02-18T00:37:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 10 client.31242.objecter _send_command 9 2024-02-18T00:37:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae81086d0 con 0x7f4ae8107cb0 2024-02-18T00:37:45.299 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4ae8107cb0 0x7f4ae81080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae8107cb0 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.321+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4ae8107cb0 0x7f4ae81080f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.322+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae81086d0 con 0x7f4ae8107cb0 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.322+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.322+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 9 = system:0 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.322+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae81075a0 osd=4 4 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.322+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 4 9 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 10 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 10 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae81075a0 osd=4 3 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 4 10 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81075a0 osd=4 4 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 10 client.31242.objecter _send_command 10 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.338+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae81086d0 con 0x7f4ae8107cb0 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae81086d0 con 0x7f4ae8107cb0 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae81075a0 osd=4 4 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 4 10 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 11 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4ae8108e80 0x7f4ae81284e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 11 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 5 11 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81086d0 osd=5 4 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 10 client.31242.objecter _send_command 11 2024-02-18T00:37:45.302 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.340+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae812b220 con 0x7f4ae8108e80 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.341+0000 7f4ae7fff700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4ae8108e80 0x7f4ae81284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.341+0000 7f4ae7fff700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4ae8108e80 0x7f4ae81284e0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.341+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae8108e80 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.342+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae812b220 con 0x7f4ae8108e80 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.342+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.342+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 11 = system:0 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.342+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae81086d0 osd=5 4 2024-02-18T00:37:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.342+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 5 11 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 12 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 12 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae81086d0 osd=5 3 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 5 12 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81086d0 osd=5 4 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 10 client.31242.objecter _send_command 12 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.356+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae812b220 con 0x7f4ae8108e80 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae812b220 con 0x7f4ae8108e80 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae81086d0 osd=5 4 2024-02-18T00:37:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 5 12 2024-02-18T00:37:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 13 2024-02-18T00:37:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4ae812a5b0 0x7f4ae812aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.305 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:37:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 13 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 6 13 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae812b220 osd=6 4 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 10 client.31242.objecter _send_command 13 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae812e5d0 con 0x7f4ae812a5b0 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aed04a700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4ae812a5b0 0x7f4ae812aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae812a5b0 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.359+0000 7f4aed04a700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4ae812a5b0 0x7f4ae812aa30 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.360+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae812e5d0 con 0x7f4ae812a5b0 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.360+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:37:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.360+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 13 = system:0 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.360+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae812b220 osd=6 4 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.360+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 6 13 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 14 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 14 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae812b220 osd=6 3 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 6 14 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae812b220 osd=6 4 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 10 client.31242.objecter _send_command 14 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.375+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae8128a00 con 0x7f4ae812a5b0 2024-02-18T00:37:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.376+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae8128a00 con 0x7f4ae812a5b0 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.376+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.376+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.376+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae812b220 osd=6 4 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.376+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 6 14 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 15 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6812/53025171 conn(0x7f4ae81290b0 0x7f4ae8129510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.308 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:37:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 15 2024-02-18T00:37:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 7 15 2024-02-18T00:37:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8128a00 osd=7 4 2024-02-18T00:37:45.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 10 client.31242.objecter _send_command 15 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ae8129a00 con 0x7f4ae81290b0 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6812/53025171 conn(0x7f4ae81290b0 0x7f4ae8129510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4aec849700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6812/53025171 conn(0x7f4ae81290b0 0x7f4ae8129510 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.377+0000 7f4ae5f76700 10 client.31242.objecter ms_handle_connect 0x7f4ae81290b0 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.378+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4ae8129a00 con 0x7f4ae81290b0 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.378+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.378+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 15 = system:0 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.378+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae8128a00 osd=7 4 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.378+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 7 15 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 10 client.31242.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign -1 16 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 20 client.31242.objecter _get_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_remove -1 16 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 20 client.31242.objecter get_session s=0x7f4ae8128a00 osd=7 3 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 15 client.31242.objecter _session_command_op_assign 7 16 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8128a00 osd=7 4 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 10 client.31242.objecter _send_command 16 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.392+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f4ae8129a00 con 0x7f4ae81290b0 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4ae5f76700 1 -- 172.21.15.59:0/1768803730 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f4ae8129a00 con 0x7f4ae81290b0 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4ae5f76700 10 client.31242.objecter ms_dispatch 0x7f4ae8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4ae5f76700 10 client.31242.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4ae5f76700 20 client.31242.objecter put_session s=0x7f4ae8128a00 osd=7 4 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4ae5f76700 15 client.31242.objecter _session_command_op_remove 7 16 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4aeeaad700 10 librados: watch_flush enter 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4aeeaad700 10 librados: watch_flush exit 2024-02-18T00:37:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.0 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.394+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4ae8028540 msgr2=0x7f4ae80fd9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4ae8028540 0x7f4ae80fd9b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae805ee80 osd=0 1 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.1 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6805/26279110 conn(0x7f4ae8027b90 msgr2=0x7f4ae8027f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6805/26279110 conn(0x7f4ae8027b90 0x7f4ae8027f90 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae80ff020 osd=1 1 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.2 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4ae80feb80 msgr2=0x7f4ae811c170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4ae80feb80 0x7f4ae811c170 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8027a10 osd=2 1 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.3 2024-02-18T00:37:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4ae8107180 msgr2=0x7f4ae811e670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4ae8107180 0x7f4ae811e670 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81197b0 osd=3 1 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.4 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4ae8107cb0 msgr2=0x7f4ae81080f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4ae8107cb0 0x7f4ae81080f0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aed04a700 1 -- 172.21.15.59:0/1768803730 reap_dead start 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81075a0 osd=4 1 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.5 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4ae8108e80 msgr2=0x7f4ae81284e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4ae8108e80 0x7f4ae81284e0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae81086d0 osd=5 1 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.6 2024-02-18T00:37:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4ae812a5b0 msgr2=0x7f4ae812aa30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4ae812a5b0 0x7f4ae812aa30 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae812b220 osd=6 1 2024-02-18T00:37:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter close_session for osd.7 2024-02-18T00:37:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6812/53025171 conn(0x7f4ae81290b0 msgr2=0x7f4ae8129510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.137:6812/53025171 conn(0x7f4ae81290b0 0x7f4ae8129510 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter put_session s=0x7f4ae8128a00 osd=7 1 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 20 client.31242.objecter shutdown clearing up homeless session... 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 10 client.31242.objecter successfully canceled tick 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aed04a700 1 -- 172.21.15.59:0/1768803730 reap_dead start 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aed04a700 1 -- 172.21.15.59:0/1768803730 reap_dead start 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aed04a700 1 -- 172.21.15.59:0/1768803730 reap_dead start 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aed04a700 1 -- 172.21.15.59:0/1768803730 reap_dead start 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4ad8062c90 msgr2=0x7f4ad8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4ad8062c90 0x7f4ad8065150 secure :-1 s=READY pgs=3094 cs=0 l=1 rev1=1 rx=0x7f4adc007fd0 tx=0x7f4adc00b040).stop 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fcbb0 msgr2=0x7f4ae8109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fcbb0 0x7f4ae8109880 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f4ad4004870 tx=0x7f4ad400b040).stop 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 shutdown_connections 2024-02-18T00:37:45.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80fcbb0 0x7f4ae8109880 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 --2- 172.21.15.59:0/1768803730 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4ad8062c90 0x7f4ad8065150 unknown :-1 s=CLOSED pgs=3094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 >> 172.21.15.59:0/1768803730 conn(0x7f4ae80f2820 msgr2=0x7f4ae80f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 shutdown_connections 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 -- 172.21.15.59:0/1768803730 wait complete. 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:44.395+0000 7f4aeeaad700 1 librados: shutdown 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.316 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:37:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.881103+0000 osd.3 (osd.3) 215 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.883167+0000 osd.3 (osd.3) 216 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:43.887019+0000 mgr.y (mgr.24454) 365 : cluster [DBG] pgmap v565: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.196944+0000 osd.4 (osd.4) 225 : cluster [DBG] 25.1c deep-scrub starts 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.199937+0000 osd.4 (osd.4) 226 : cluster [DBG] 25.1c deep-scrub ok 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.242554+0000 mon.a (mon.0) 2414 : audit [DBG] from='client.? 172.21.15.59:0/1768803730' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.334575+0000 osd.2 (osd.2) 171 : cluster [DBG] 5.8 deep-scrub starts 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.335384+0000 osd.2 (osd.2) 172 : cluster [DBG] 5.8 deep-scrub ok 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.672178+0000 mon.a (mon.0) 2415 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi059-82882-64"}]': finished 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.672265+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.672332+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e386: 8 total, 8 up, 7 in 2024-02-18T00:37:45.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.675060+0000 mon.c (mon.2) 378 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:37:45.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.675700+0000 mon.a (mon.0) 2418 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:37:45.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.695429+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.695812+0000 mon.a (mon.0) 2419 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.696416+0000 mon.c (mon.2) 380 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.696644+0000 mon.a (mon.0) 2420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:45.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.697078+0000 mon.c (mon.2) 381 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:45.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:44.697383+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:37:46.050 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:46.052 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:46.053 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:46.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:46.184 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:46.267 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:46.284 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:46.287 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.681858+0000 osd.1 (osd.1) 201 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:37:46.287 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.685584+0000 osd.1 (osd.1) 202 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:37:46.287 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.868889+0000 osd.3 (osd.3) 217 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:44.869858+0000 osd.3 (osd.3) 218 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.182584+0000 osd.4 (osd.4) 227 : cluster [DBG] 2.f scrub starts 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.186133+0000 osd.4 (osd.4) 228 : cluster [DBG] 2.f scrub ok 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.453138+0000 osd.0 (osd.0) 185 : cluster [DBG] 2.1f deep-scrub starts 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.454065+0000 osd.0 (osd.0) 186 : cluster [DBG] 2.1f deep-scrub ok 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:45.675021+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]': finished 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:45.675113+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.675195+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e387: 8 total, 8 up, 7 in 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:45.679074+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:46.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:45.681099+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.681858+0000 osd.1 (osd.1) 201 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.685584+0000 osd.1 (osd.1) 202 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.868889+0000 osd.3 (osd.3) 217 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:44.869858+0000 osd.3 (osd.3) 218 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.182584+0000 osd.4 (osd.4) 227 : cluster [DBG] 2.f scrub starts 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.186133+0000 osd.4 (osd.4) 228 : cluster [DBG] 2.f scrub ok 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.453138+0000 osd.0 (osd.0) 185 : cluster [DBG] 2.1f deep-scrub starts 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.454065+0000 osd.0 (osd.0) 186 : cluster [DBG] 2.1f deep-scrub ok 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:45.675021+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]': finished 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:45.675113+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.675195+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e387: 8 total, 8 up, 7 in 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:45.679074+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:46.289 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:45.681099+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:46.290 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:46.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:46.391 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:46.420 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.681858+0000 osd.1 (osd.1) 201 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.685584+0000 osd.1 (osd.1) 202 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.868889+0000 osd.3 (osd.3) 217 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:44.869858+0000 osd.3 (osd.3) 218 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.182584+0000 osd.4 (osd.4) 227 : cluster [DBG] 2.f scrub starts 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.186133+0000 osd.4 (osd.4) 228 : cluster [DBG] 2.f scrub ok 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.453138+0000 osd.0 (osd.0) 185 : cluster [DBG] 2.1f deep-scrub starts 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.454065+0000 osd.0 (osd.0) 186 : cluster [DBG] 2.1f deep-scrub ok 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:45.675021+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-49","var": "target_max_objects","val": "1"}]': finished 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:45.675113+0000 mon.a (mon.0) 2423 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi059-82882-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.675195+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e387: 8 total, 8 up, 7 in 2024-02-18T00:37:46.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:45.679074+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:45.681099+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:46.568 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:46.571 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:46.571 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:46.672 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:46.702 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:46.723 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:46.725 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:46.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:46.826 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:46.855 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:46.875 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:46.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:46.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:46.978 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:47.005 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:47.024 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:47.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:47.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:47.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:47.162 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:47.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-18T00:37:47.184 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:47.187 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:47.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:47.288 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:47.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.836910+0000 osd.3 (osd.3) 219 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:37:47.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.838517+0000 osd.3 (osd.3) 220 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:37:47.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:45.888094+0000 mgr.y (mgr.24454) 366 : cluster [DBG] pgmap v568: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:37:47.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:46.470871+0000 osd.0 (osd.0) 187 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:37:47.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:46.472222+0000 osd.0 (osd.0) 188 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:37:47.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:46.671046+0000 mon.a (mon.0) 2426 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T00:37:47.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:47.111714+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e388: 8 total, 8 up, 7 in 2024-02-18T00:37:47.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:47.112144+0000 mon.a (mon.0) 2428 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:37:47.314 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:47.334 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:47.337 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:47.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:47.438 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:47.466 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:47.487 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:47.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:47.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.836910+0000 osd.3 (osd.3) 219 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.838517+0000 osd.3 (osd.3) 220 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:45.888094+0000 mgr.y (mgr.24454) 366 : cluster [DBG] pgmap v568: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:46.470871+0000 osd.0 (osd.0) 187 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:46.472222+0000 osd.0 (osd.0) 188 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:46.671046+0000 mon.a (mon.0) 2426 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:47.111714+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e388: 8 total, 8 up, 7 in 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:47.112144+0000 mon.a (mon.0) 2428 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.836910+0000 osd.3 (osd.3) 219 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.838517+0000 osd.3 (osd.3) 220 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:37:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:45.888094+0000 mgr.y (mgr.24454) 366 : cluster [DBG] pgmap v568: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:37:47.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:46.470871+0000 osd.0 (osd.0) 187 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:37:47.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:46.472222+0000 osd.0 (osd.0) 188 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:37:47.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:46.671046+0000 mon.a (mon.0) 2426 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T00:37:47.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:47.111714+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e388: 8 total, 8 up, 7 in 2024-02-18T00:37:47.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:47.112144+0000 mon.a (mon.0) 2428 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:37:47.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:37:47.616 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:47.690 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:37:47.696 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:47.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:37:47.797 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:47.824 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:47.842 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:47.845 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:47.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:47.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:37:47.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:48.010 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:48.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:46.638818+0000 osd.6 (osd.6) 187 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:46.639896+0000 osd.6 (osd.6) 188 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:47.451642+0000 mon.a (mon.0) 2429 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:47.452301+0000 mon.a (mon.0) 2430 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:47.456596+0000 mon.a (mon.0) 2431 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:47.487927+0000 osd.0 (osd.0) 189 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:47.488818+0000 osd.0 (osd.0) 190 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:37:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:47.680039+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:47.680140+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e389: 8 total, 8 up, 7 in 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:46.638818+0000 osd.6 (osd.6) 187 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:46.639896+0000 osd.6 (osd.6) 188 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:47.451642+0000 mon.a (mon.0) 2429 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:47.452301+0000 mon.a (mon.0) 2430 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:47.456596+0000 mon.a (mon.0) 2431 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:47.487927+0000 osd.0 (osd.0) 189 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:47.488818+0000 osd.0 (osd.0) 190 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:47.680039+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:47.680140+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e389: 8 total, 8 up, 7 in 2024-02-18T00:37:48.417 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:46.638818+0000 osd.6 (osd.6) 187 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:46.639896+0000 osd.6 (osd.6) 188 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:47.451642+0000 mon.a (mon.0) 2429 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:47.452301+0000 mon.a (mon.0) 2430 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:47.456596+0000 mon.a (mon.0) 2431 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:47.487927+0000 osd.0 (osd.0) 189 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:47.488818+0000 osd.0 (osd.0) 190 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:47.680039+0000 mon.a (mon.0) 2432 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi059-82882-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:47.680140+0000 mon.a (mon.0) 2433 : cluster [DBG] osdmap e389: 8 total, 8 up, 7 in 2024-02-18T00:37:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:48.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:48.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:49.064 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:49.093 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:49.113 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:49.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:49.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:49.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:49.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:47.571128+0000 osd.5 (osd.5) 167 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:37:49.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:47.574000+0000 osd.5 (osd.5) 168 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:47.888461+0000 mgr.y (mgr.24454) 367 : cluster [DBG] pgmap v571: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:48.215084+0000 osd.4 (osd.4) 229 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:48.216923+0000 osd.4 (osd.4) 230 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:48.393517+0000 osd.2 (osd.2) 173 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:48.395201+0000 osd.2 (osd.2) 174 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:48.692984+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e390: 8 total, 8 up, 7 in 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:47.571128+0000 osd.5 (osd.5) 167 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:37:49.228 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:47.574000+0000 osd.5 (osd.5) 168 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:37:49.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:47.888461+0000 mgr.y (mgr.24454) 367 : cluster [DBG] pgmap v571: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:37:49.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:48.215084+0000 osd.4 (osd.4) 229 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:37:49.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:48.216923+0000 osd.4 (osd.4) 230 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:37:49.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:48.393517+0000 osd.2 (osd.2) 173 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:37:49.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:48.395201+0000 osd.2 (osd.2) 174 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:37:49.229 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:48.692984+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e390: 8 total, 8 up, 7 in 2024-02-18T00:37:49.248 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:49.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 -- 172.21.15.59:0/2246816874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0680fcbb0 msgr2=0x7ff0680fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:49.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 --2- 172.21.15.59:0/2246816874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0680fcbb0 0x7ff0680fcf90 secure :-1 s=READY pgs=4062 cs=0 l=1 rev1=1 rx=0x7ff0580047b0 tx=0x7ff058009cb0).stop 2024-02-18T00:37:49.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 -- 172.21.15.59:0/2246816874 shutdown_connections 2024-02-18T00:37:49.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 --2- 172.21.15.59:0/2246816874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0680fcbb0 0x7ff0680fcf90 unknown :-1 s=CLOSED pgs=4062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:49.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 --2- 172.21.15.59:0/2246816874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680f7400 0x7ff068103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 --2- 172.21.15.59:0/2246816874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 0x7ff0680f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 -- 172.21.15.59:0/2246816874 >> 172.21.15.59:0/2246816874 conn(0x7ff0680f2820 msgr2=0x7ff0680f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 -- 172.21.15.59:0/2246816874 shutdown_connections 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 -- 172.21.15.59:0/2246816874 wait complete. 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 librados: starting msgr at 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 librados: starting objecter 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.204+0000 7ff06e900700 1 Processor -- start 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 -- start start 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 librados: setting wanted keys 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 librados: calling monclient init 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0680f7400 0x7ff06810e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:49.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680fcbb0 0x7ff06810e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 0x7ff068108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068108940 con 0x7ff0680fd510 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff068108ab0 con 0x7ff0680f7400 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06e900700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff068108c70 con 0x7ff0680fcbb0 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff0677fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680fcbb0 0x7ff06810e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06ce9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 0x7ff068108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06ce9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 0x7ff068108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42278/0 (socket says 172.21.15.59:42278) 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff0677fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680fcbb0 0x7ff06810e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59682/0 (socket says 172.21.15.59:59682) 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06ce9d700 1 -- 172.21.15.59:0/1412888649 learned_addr learned my addr 172.21.15.59:0/1412888649 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:49.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff0677fe700 1 -- 172.21.15.59:0/1412888649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0680f7400 msgr2=0x7ff06810e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff0677fe700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0680f7400 0x7ff06810e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff0677fe700 1 -- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 msgr2=0x7ff068108450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff0677fe700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 0x7ff068108450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff0677fe700 1 -- 172.21.15.59:0/1412888649 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058005040 con 0x7ff0680fcbb0 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.205+0000 7ff06ce9d700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 0x7ff068108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff0677fe700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680fcbb0 0x7ff06810e7a0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 rx=0x7ff05c009a00 tx=0x7ff05c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff0657fa700 10 client.?.objecter ms_handle_connect 0x7ff0680fcbb0 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff0657fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff0657fa700 1 -- 172.21.15.59:0/1412888649 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c004630 con 0x7ff0680fcbb0 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff068108f50 con 0x7ff0680fcbb0 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff0657fa700 1 -- 172.21.15.59:0/1412888649 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff05c00f040 con 0x7ff0680fcbb0 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff0657fa700 1 -- 172.21.15.59:0/1412888649 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c014830 con 0x7ff0680fcbb0 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff06e900700 10 client.41057.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:49.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0681092d0 con 0x7ff0680fcbb0 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.206+0000 7ff06e900700 1 librados: init done 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff0657fa700 1 -- 172.21.15.59:0/1412888649 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff05c01d3b0 con 0x7ff0680fcbb0 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06805fa60 con 0x7ff0680fcbb0 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff0657fa700 1 --2- 172.21.15.59:0/1412888649 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff050062c90 0x7ff050065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff0657fa700 1 -- 172.21.15.59:0/1412888649 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(390..390 src has 180..390) v4 ==== 8861+0+0 (secure 0 0 0) 0x7ff05c07d100 con 0x7ff0680fcbb0 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff0657fa700 10 client.41057.objecter ms_dispatch 0x7ff068001550 osd_map(390..390 src has 180..390) v4 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff0657fa700 3 client.41057.objecter handle_osd_map got epochs [390,390] > 0 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff0657fa700 3 client.41057.objecter handle_osd_map decoding full epoch 390 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.208+0000 7ff0657fa700 20 client.41057.objecter dump_active .. 0 homeless 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.209+0000 7ff067fff700 1 --2- 172.21.15.59:0/1412888649 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff050062c90 0x7ff050065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:49.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.209+0000 7ff0657fa700 10 client.41057.objecter ms_handle_connect 0x7ff050062c90 2024-02-18T00:37:49.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.209+0000 7ff067fff700 1 --2- 172.21.15.59:0/1412888649 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff050062c90 0x7ff050065150 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 rx=0x7ff058004650 tx=0x7ff058009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:49.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.213+0000 7ff0657fa700 1 -- 172.21.15.59:0/1412888649 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff05c04af60 con 0x7ff0680fcbb0 2024-02-18T00:37:49.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.405+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7ff068107280 con 0x7ff0680fcbb0 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:47.571128+0000 osd.5 (osd.5) 167 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:47.574000+0000 osd.5 (osd.5) 168 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:47.888461+0000 mgr.y (mgr.24454) 367 : cluster [DBG] pgmap v571: 246 pgs: 8 unknown, 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:48.215084+0000 osd.4 (osd.4) 229 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:48.216923+0000 osd.4 (osd.4) 230 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:48.393517+0000 osd.2 (osd.2) 173 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:48.395201+0000 osd.2 (osd.2) 174 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:37:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:48.692984+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e390: 8 total, 8 up, 7 in 2024-02-18T00:37:49.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.697+0000 7ff0657fa700 1 -- 172.21.15.59:0/1412888649 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v391) v1 ==== 87+0+0 (secure 0 0 0) 0x7ff05c019070 con 0x7ff0680fcbb0 2024-02-18T00:37:49.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 10 librados: watch_flush enter 2024-02-18T00:37:49.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 10 librados: watch_flush exit 2024-02-18T00:37:49.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 20 client.41057.objecter shutdown clearing up homeless session... 2024-02-18T00:37:49.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 10 client.41057.objecter successfully canceled tick 2024-02-18T00:37:49.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff050062c90 msgr2=0x7ff050065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 --2- 172.21.15.59:0/1412888649 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff050062c90 0x7ff050065150 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 rx=0x7ff058004650 tx=0x7ff058009cb0).stop 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680fcbb0 msgr2=0x7ff06810e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680fcbb0 0x7ff06810e7a0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 rx=0x7ff05c009a00 tx=0x7ff05c00b040).stop 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 shutdown_connections 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 --2- 172.21.15.59:0/1412888649 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff050062c90 0x7ff050065150 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0680f7400 0x7ff06810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0680fd510 0x7ff068108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 --2- 172.21.15.59:0/1412888649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0680fcbb0 0x7ff06810e7a0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.700+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 >> 172.21.15.59:0/1412888649 conn(0x7ff0680f2820 msgr2=0x7ff0680f4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.701+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 shutdown_connections 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.701+0000 7ff06e900700 1 -- 172.21.15.59:0/1412888649 wait complete. 2024-02-18T00:37:49.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:49.701+0000 7ff06e900700 1 librados: shutdown 2024-02-18T00:37:49.704 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:37:49.717 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:49.720 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:49.900 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:49.902 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:49.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:50.004 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:50.035 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:50.057 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:50.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:50.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:50.161 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:37:50.190 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:50.210 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:37:50.213 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:50.213 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:37:50.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:50.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:48.524884+0000 osd.5 (osd.5) 169 : cluster [DBG] 5.f scrub starts 2024-02-18T00:37:50.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:48.527776+0000 osd.5 (osd.5) 170 : cluster [DBG] 5.f scrub ok 2024-02-18T00:37:50.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:49.406571+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.59:0/1412888649' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:50.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:49.406851+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:50.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:49.519605+0000 osd.0 (osd.0) 191 : cluster [DBG] 4.b scrub starts 2024-02-18T00:37:50.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:49.523117+0000 osd.0 (osd.0) 192 : cluster [DBG] 4.b scrub ok 2024-02-18T00:37:50.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:49.688120+0000 mon.a (mon.0) 2436 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:50.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:49.692554+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:37:50.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:49.692630+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e391: 8 total, 8 up, 7 in 2024-02-18T00:37:50.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:49.696727+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:50.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:49.697053+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:50.361 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:50.363 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:50.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:50.465 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:50.493 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:48.524884+0000 osd.5 (osd.5) 169 : cluster [DBG] 5.f scrub starts 2024-02-18T00:37:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:48.527776+0000 osd.5 (osd.5) 170 : cluster [DBG] 5.f scrub ok 2024-02-18T00:37:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:49.406571+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.59:0/1412888649' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:49.406851+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:49.519605+0000 osd.0 (osd.0) 191 : cluster [DBG] 4.b scrub starts 2024-02-18T00:37:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:49.523117+0000 osd.0 (osd.0) 192 : cluster [DBG] 4.b scrub ok 2024-02-18T00:37:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:49.688120+0000 mon.a (mon.0) 2436 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:49.692554+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:37:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:49.692630+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e391: 8 total, 8 up, 7 in 2024-02-18T00:37:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:49.696727+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:50.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:49.697053+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:50.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:48.524884+0000 osd.5 (osd.5) 169 : cluster [DBG] 5.f scrub starts 2024-02-18T00:37:50.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:48.527776+0000 osd.5 (osd.5) 170 : cluster [DBG] 5.f scrub ok 2024-02-18T00:37:50.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:49.406571+0000 mon.c (mon.2) 383 : audit [INF] from='client.? 172.21.15.59:0/1412888649' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:50.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:49.406851+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:37:50.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:49.519605+0000 osd.0 (osd.0) 191 : cluster [DBG] 4.b scrub starts 2024-02-18T00:37:50.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:49.523117+0000 osd.0 (osd.0) 192 : cluster [DBG] 4.b scrub ok 2024-02-18T00:37:50.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:49.688120+0000 mon.a (mon.0) 2436 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:37:50.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:49.692554+0000 mon.a (mon.0) 2437 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:37:50.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:49.692630+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e391: 8 total, 8 up, 7 in 2024-02-18T00:37:50.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:49.696727+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:50.506 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:49.697053+0000 mon.a (mon.0) 2439 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:50.512 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:50.514 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:50.515 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:50.616 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:50.650 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:51.100 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:51.103 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:51.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:51.204 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:51.234 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:51.358 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:51.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:49.889078+0000 mgr.y (mgr.24454) 368 : cluster [DBG] pgmap v574: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:51.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:50.698541+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:51.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:50.698618+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e392: 8 total, 8 up, 7 in 2024-02-18T00:37:51.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:50.701039+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:51.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:50.701537+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:51.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:49.889078+0000 mgr.y (mgr.24454) 368 : cluster [DBG] pgmap v574: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:51.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:50.698541+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:51.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:50.698618+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e392: 8 total, 8 up, 7 in 2024-02-18T00:37:51.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:50.701039+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:51.361 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:50.701537+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:51.362 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:51.362 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:51.463 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:51.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:37:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:37:51.494 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:49.889078+0000 mgr.y (mgr.24454) 368 : cluster [DBG] pgmap v574: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:37:51.494 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:50.698541+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:51.494 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:50.698618+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e392: 8 total, 8 up, 7 in 2024-02-18T00:37:51.494 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:50.701039+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.59:0/267555837' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:51.494 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:50.701537+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]: dispatch 2024-02-18T00:37:51.494 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:37:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:37:51.495 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:51.517 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:51.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:51.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:51.621 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:37:51.651 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:37:51.673 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:51.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:37:51.706 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-18T00:37:51.706 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (6986 ms) 2024-02-18T00:37:51.706 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-18T00:37:51.707 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7021 ms) 2024-02-18T00:37:51.707 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-18T00:37:51.707 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7001 ms) 2024-02-18T00:37:51.707 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-18T00:37:51.707 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (6991 ms) 2024-02-18T00:37:51.707 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-18T00:37:51.708 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7027 ms) 2024-02-18T00:37:51.708 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (138991 ms total) 2024-02-18T00:37:51.708 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: 2024-02-18T00:37:51.708 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-18T00:37:51.708 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [==========] 56 tests from 4 test suites ran. (300774 ms total) 2024-02-18T00:37:51.708 INFO:tasks.workunit.client.0.smithi059.stdout: api_aio_pp: [ PASSED ] 56 tests. 2024-02-18T00:37:51.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82896 2024-02-18T00:37:51.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82896 2024-02-18T00:37:51.715 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82960 2024-02-18T00:37:51.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82960 2024-02-18T00:37:51.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82944 2024-02-18T00:37:51.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82944 2024-02-18T00:37:51.716 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.717 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82920 2024-02-18T00:37:51.717 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82920 2024-02-18T00:37:51.717 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.717 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83016 2024-02-18T00:37:51.717 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83016 2024-02-18T00:37:51.717 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.717 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82888 2024-02-18T00:37:51.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82888 2024-02-18T00:37:51.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83032 2024-02-18T00:37:51.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83032 2024-02-18T00:37:51.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82984 2024-02-18T00:37:51.718 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82984 2024-02-18T00:37:51.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83064 2024-02-18T00:37:51.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83064 2024-02-18T00:37:51.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83024 2024-02-18T00:37:51.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83024 2024-02-18T00:37:51.719 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83080 2024-02-18T00:37:51.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83080 2024-02-18T00:37:51.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82872 2024-02-18T00:37:51.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82872 2024-02-18T00:37:51.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.720 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83072 2024-02-18T00:37:51.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83072 2024-02-18T00:37:51.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=83040 2024-02-18T00:37:51.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 83040 2024-02-18T00:37:51.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ for t in "${!pids[@]}" 2024-02-18T00:37:51.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ pid=82952 2024-02-18T00:37:51.721 INFO:tasks.workunit.client.0.smithi059.stderr:+ wait 82952 2024-02-18T00:37:51.774 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:51.802 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:51.820 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:51.823 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:51.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:51.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:51.955 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:52.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:51.220669+0000 mgr.y (mgr.24454) 369 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:52.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:51.700902+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:52.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:51.700986+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e393: 8 total, 8 up, 7 in 2024-02-18T00:37:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:51.220669+0000 mgr.y (mgr.24454) 369 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:51.700902+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:52.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:51.700986+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e393: 8 total, 8 up, 7 in 2024-02-18T00:37:52.408 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:52.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:52.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:52.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:37:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:37:52] "GET /metrics HTTP/1.1" 200 189273 "" "Prometheus/2.33.4" 2024-02-18T00:37:52.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:51.220669+0000 mgr.y (mgr.24454) 369 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:37:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:51.700902+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi059-82882-65"}]': finished 2024-02-18T00:37:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:51.700986+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e393: 8 total, 8 up, 7 in 2024-02-18T00:37:52.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:51.889592+0000 mgr.y (mgr.24454) 370 : cluster [DBG] pgmap v577: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:51.889592+0000 mgr.y (mgr.24454) 370 : cluster [DBG] pgmap v577: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:51.889592+0000 mgr.y (mgr.24454) 370 : cluster [DBG] pgmap v577: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:37:53.797 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:53.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:53.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:53.887 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:37:53.887 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:37:53.902 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:53.934 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:53.956 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:53.959 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:53.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:54.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:54.091 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:54.111 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:54.113 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:54.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:54.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:37:54.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:37:54.243 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:54.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-18T00:37:54.923 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:53.890213+0000 mgr.y (mgr.24454) 371 : cluster [DBG] pgmap v578: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 984 B/s rd, 0 op/s 2024-02-18T00:37:54.923 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:53.890213+0000 mgr.y (mgr.24454) 371 : cluster [DBG] pgmap v578: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 984 B/s rd, 0 op/s 2024-02-18T00:37:55.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:53.890213+0000 mgr.y (mgr.24454) 371 : cluster [DBG] pgmap v578: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 984 B/s rd, 0 op/s 2024-02-18T00:37:55.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 -- 172.21.15.59:0/1314684806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0fd510 msgr2=0x7f62cc0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:55.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1314684806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0fd510 0x7f62cc0f6d40 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 rx=0x7f62c0004f90 tx=0x7f62c0004990).stop 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 -- 172.21.15.59:0/1314684806 shutdown_connections 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1314684806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc0f7400 0x7f62cc103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1314684806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62cc0fcbb0 0x7f62cc0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1314684806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0fd510 0x7f62cc0f6d40 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 -- 172.21.15.59:0/1314684806 >> 172.21.15.59:0/1314684806 conn(0x7f62cc0f2820 msgr2=0x7f62cc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 -- 172.21.15.59:0/1314684806 shutdown_connections 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 -- 172.21.15.59:0/1314684806 wait complete. 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 librados: starting msgr at 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.204+0000 7f62d1f91700 1 librados: starting objecter 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 Processor -- start 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 -- start start 2024-02-18T00:37:55.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 librados: setting wanted keys 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 librados: calling monclient init 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0f7400 0x7f62cc10c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc0fcbb0 0x7f62cc10c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62cc0fd510 0x7f62cc1061a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc106690 con 0x7f62cc0fd510 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc106800 con 0x7f62cc0fcbb0 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62d1f91700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc1069c0 con 0x7f62cc0f7400 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0f7400 0x7f62cc10c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62cc0fd510 0x7f62cc1061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0f7400 0x7f62cc10c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56594/0 (socket says 172.21.15.59:56594) 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 -- 172.21.15.59:0/1384092172 learned_addr learned my addr 172.21.15.59:0/1384092172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62caffd700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc0fcbb0 0x7f62cc10c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 -- 172.21.15.59:0/1384092172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc0fcbb0 msgr2=0x7f62cc10c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc0fcbb0 0x7f62cc10c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 -- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62cc0fd510 msgr2=0x7f62cc1061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62cc0fd510 0x7f62cc1061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cb7fe700 1 -- 172.21.15.59:0/1384092172 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc005040 con 0x7f62cc0f7400 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62caffd700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc0fcbb0 0x7f62cc10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.205+0000 7f62cbfff700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62cc0fd510 0x7f62cc1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62cb7fe700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0f7400 0x7f62cc10c000 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 rx=0x7f62bc0048f0 tx=0x7f62bc009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62c8ff9700 10 client.?.objecter ms_handle_connect 0x7f62cc0f7400 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62c8ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62c8ff9700 1 -- 172.21.15.59:0/1384092172 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc007b00 con 0x7f62cc0f7400 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62c8ff9700 1 -- 172.21.15.59:0/1384092172 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62bc00f040 con 0x7f62cc0f7400 2024-02-18T00:37:55.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c0005040 con 0x7f62cc0f7400 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62c8ff9700 1 -- 172.21.15.59:0/1384092172 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc014a00 con 0x7f62cc0f7400 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62d1f91700 10 client.41063.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc107000 con 0x7f62cc0f7400 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.206+0000 7f62d1f91700 1 librados: init done 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.207+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc05fa60 con 0x7f62cc0f7400 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.207+0000 7f62c8ff9700 1 -- 172.21.15.59:0/1384092172 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f62bc0133c0 con 0x7f62cc0f7400 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.207+0000 7f62c8ff9700 1 --2- 172.21.15.59:0/1384092172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f62b4062c90 0x7f62b4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62c8ff9700 1 -- 172.21.15.59:0/1384092172 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(393..393 src has 180..393) v4 ==== 8053+0+0 (secure 0 0 0) 0x7f62bc07cdb0 con 0x7f62cc0f7400 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62c8ff9700 10 client.41063.objecter ms_dispatch 0x7f62cc001550 osd_map(393..393 src has 180..393) v4 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62c8ff9700 3 client.41063.objecter handle_osd_map got epochs [393,393] > 0 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62c8ff9700 3 client.41063.objecter handle_osd_map decoding full epoch 393 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62caffd700 1 --2- 172.21.15.59:0/1384092172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f62b4062c90 0x7f62b4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:55.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62c8ff9700 20 client.41063.objecter dump_active .. 0 homeless 2024-02-18T00:37:55.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62c8ff9700 10 client.41063.objecter ms_handle_connect 0x7f62b4062c90 2024-02-18T00:37:55.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.208+0000 7f62caffd700 1 --2- 172.21.15.59:0/1384092172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f62b4062c90 0x7f62b4065150 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 rx=0x7f62c0004a80 tx=0x7f62c0004990).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:55.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.212+0000 7f62c8ff9700 1 -- 172.21.15.59:0/1384092172 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f62bc050260 con 0x7f62cc0f7400 2024-02-18T00:37:55.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.380+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62cc107680 con 0x7f62b4062c90 2024-02-18T00:37:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.397+0000 7f62c8ff9700 1 -- 172.21.15.59:0/1384092172 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+598153 (secure 0 0 0) 0x7f62cc107680 con 0x7f62b4062c90 2024-02-18T00:37:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:37:55.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 10 librados: watch_flush enter 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 10 librados: watch_flush exit 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 20 client.41063.objecter shutdown clearing up homeless session... 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 10 client.41063.objecter successfully canceled tick 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f62b4062c90 msgr2=0x7f62b4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1384092172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f62b4062c90 0x7f62b4065150 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 rx=0x7f62c0004a80 tx=0x7f62c0004990).stop 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0f7400 msgr2=0x7f62cc10c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0f7400 0x7f62cc10c000 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 rx=0x7f62bc0048f0 tx=0x7f62bc009cb0).stop 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 shutdown_connections 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62cc0fd510 0x7f62cc1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62cc0f7400 0x7f62cc10c000 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1384092172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f62b4062c90 0x7f62b4065150 unknown :-1 s=CLOSED pgs=3097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 --2- 172.21.15.59:0/1384092172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62cc0fcbb0 0x7f62cc10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:55.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 >> 172.21.15.59:0/1384092172 conn(0x7f62cc0f2820 msgr2=0x7f62cc0f43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:55.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 shutdown_connections 2024-02-18T00:37:55.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 -- 172.21.15.59:0/1384092172 wait complete. 2024-02-18T00:37:55.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:55.401+0000 7f62d1f91700 1 librados: shutdown 2024-02-18T00:37:55.414 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:37:56.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:54.901369+0000 mon.a (mon.0) 2445 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:56.164 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:55.691035+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:56.165 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:55.691377+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:56.165 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:54.901369+0000 mon.a (mon.0) 2445 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:56.165 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:55.691035+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:56.165 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:55.691377+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:56.200 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:37:56.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:56.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:37:56.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:54.901369+0000 mon.a (mon.0) 2445 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:37:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:55.691035+0000 mon.c (mon.2) 386 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:55.691377+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:56.304 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:56.337 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:56.358 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:56.360 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:56.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:56.462 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:56.492 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:56.513 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:56.515 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:56.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:56.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:56.655 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:56.914 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:55.382274+0000 mgr.y (mgr.24454) 372 : audit [DBG] from='client.41063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:56.915 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:55.891389+0000 mgr.y (mgr.24454) 373 : cluster [DBG] pgmap v579: 238 pgs: 238 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-18T00:37:56.915 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:55.908014+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:55.908089+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e394: 8 total, 8 up, 7 in 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:55.908873+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:55.909264+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:55.382274+0000 mgr.y (mgr.24454) 372 : audit [DBG] from='client.41063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:55.891389+0000 mgr.y (mgr.24454) 373 : cluster [DBG] pgmap v579: 238 pgs: 238 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:55.908014+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:55.908089+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e394: 8 total, 8 up, 7 in 2024-02-18T00:37:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:55.908873+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:55.909264+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:57.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:55.382274+0000 mgr.y (mgr.24454) 372 : audit [DBG] from='client.41063 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:37:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:55.891389+0000 mgr.y (mgr.24454) 373 : cluster [DBG] pgmap v579: 238 pgs: 238 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-18T00:37:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:55.908014+0000 mon.a (mon.0) 2447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:37:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:55.908089+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e394: 8 total, 8 up, 7 in 2024-02-18T00:37:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:55.908873+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:57.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:55.909264+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 -- 172.21.15.59:0/2365579194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40fe510 msgr2=0x7f5bd40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/2365579194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40fe510 0x7f5bd40f7d40 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 rx=0x7f5bc8004870 tx=0x7f5bc8009cb0).stop 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 -- 172.21.15.59:0/2365579194 shutdown_connections 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/2365579194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40fdbb0 0x7f5bd40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/2365579194 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5bd40f8420 0x7f5bd4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/2365579194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40fe510 0x7f5bd40f7d40 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 -- 172.21.15.59:0/2365579194 >> 172.21.15.59:0/2365579194 conn(0x7f5bd40f3800 msgr2=0x7f5bd40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 -- 172.21.15.59:0/2365579194 shutdown_connections 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.572+0000 7f5bd9b69700 1 -- 172.21.15.59:0/2365579194 wait complete. 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 librados: starting msgr at 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 librados: starting objecter 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 Processor -- start 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 -- start start 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 librados: setting wanted keys 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 librados: calling monclient init 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40f8420 0x7f5bd406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.573+0000 7f5bd9b69700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5bd40fdbb0 0x7f5bd406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd9b69700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40fe510 0x7f5bd406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd9b69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd41164a0 con 0x7f5bd40f8420 2024-02-18T00:37:57.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd9b69700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4116610 con 0x7f5bd40fe510 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd9b69700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd41167d0 con 0x7f5bd40fdbb0 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40f8420 0x7f5bd406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40f8420 0x7f5bd406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40878/0 (socket says 172.21.15.59:40878) 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5bd40fdbb0 0x7f5bd406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 -- 172.21.15.59:0/3182060748 learned_addr learned my addr 172.21.15.59:0/3182060748 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd3fff700 1 --2- 172.21.15.59:0/3182060748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40fe510 0x7f5bd406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 -- 172.21.15.59:0/3182060748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5bd40fdbb0 msgr2=0x7f5bd406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5bd40fdbb0 0x7f5bd406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 -- 172.21.15.59:0/3182060748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40fe510 msgr2=0x7f5bd406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bd40fe510 0x7f5bd406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 -- 172.21.15.59:0/3182060748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc8005040 con 0x7f5bd40f8420 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd2ffd700 1 --2- 172.21.15.59:0/3182060748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5bd40fdbb0 0x7f5bd406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:37:57.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40f8420 0x7f5bd406b0d0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7f5bc40041d0 tx=0x7f5bc4009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd0ff9700 10 client.?.objecter ms_handle_connect 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd0ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4007b40 con 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd4116ab0 con 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bc400f040 con 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4014730 con 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.574+0000 7f5bd9b69700 10 client.31266.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.575+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd4116ff0 con 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.575+0000 7f5bd9b69700 1 librados: init done 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.575+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f5bd405ee80 con 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5bc4009110 con 0x7f5bd40f8420 2024-02-18T00:37:57.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bbc062c90 0x7f5bbc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(395..395 src has 180..395) v4 ==== 8019+0+0 (secure 0 0 0) 0x7f5bc407d190 con 0x7f5bd40f8420 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 osd_map(395..395 src has 180..395) v4 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 3 client.31266.objecter handle_osd_map got epochs [395,395] > 0 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 3 client.31266.objecter handle_osd_map decoding full epoch 395 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd2ffd700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bbc062c90 0x7f5bbc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 20 client.31266.objecter dump_active .. 0 homeless 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.576+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v395) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5bc4084070 con 0x7f5bd40f8420 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd2ffd700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bbc062c90 0x7f5bbc065150 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 rx=0x7f5bc80043d0 tx=0x7f5bc800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bbc062c90 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 1 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5bd40fee00 0x7f5bd40ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 1 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 0 1 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4107060 osd=0 4 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 10 client.31266.objecter _send_command 1 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd406c5b0 con 0x7f5bd40fee00 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd3fff700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5bd40fee00 0x7f5bd40ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd40fee00 2024-02-18T00:37:57.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.577+0000 7f5bd3fff700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5bd40fee00 0x7f5bd40ff200 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.578+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd406c5b0 con 0x7f5bd40fee00 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.578+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.578+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 1 = system:0 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.578+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4107060 osd=0 4 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.578+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 0 1 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 2 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 2 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4107060 osd=0 3 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 0 2 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4107060 osd=0 4 2024-02-18T00:37:57.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 10 client.31266.objecter _send_command 2 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.593+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd40273c0 con 0x7f5bd40fee00 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd40273c0 con 0x7f5bd40fee00 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4107060 osd=0 4 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 0 2 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:37:57.706 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 3 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6805/26279110 conn(0x7f5bd40598f0 0x7f5bd4059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 3 2024-02-18T00:37:57.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 1 3 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4028710 osd=1 4 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 10 client.31266.objecter _send_command 3 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.594+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd411d9c0 con 0x7f5bd40598f0 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6805/26279110 conn(0x7f5bd40598f0 0x7f5bd4059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd40598f0 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6805/26279110 conn(0x7f5bd40598f0 0x7f5bd4059d10 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd411d9c0 con 0x7f5bd40598f0 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 3 = system:0 2024-02-18T00:37:57.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4028710 osd=1 4 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.595+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 1 3 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 4 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 4 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4028710 osd=1 3 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 1 4 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4028710 osd=1 4 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 10 client.31266.objecter _send_command 4 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.610+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd4059360 con 0x7f5bd40598f0 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd4059360 con 0x7f5bd40598f0 2024-02-18T00:37:57.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4028710 osd=1 4 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 1 4 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 5 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5bd411c3e0 0x7f5bd411c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 5 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 2 5 2024-02-18T00:37:57.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4027d60 osd=2 4 2024-02-18T00:37:57.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 10 client.31266.objecter _send_command 5 2024-02-18T00:37:57.711 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:37:57.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd40283e0 con 0x7f5bd411c3e0 2024-02-18T00:37:57.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd2ffd700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5bd411c3e0 0x7f5bd411c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd411c3e0 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.612+0000 7f5bd2ffd700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5bd411c3e0 0x7f5bd411c7e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.613+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd40283e0 con 0x7f5bd411c3e0 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.613+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.613+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 5 = system:0 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.613+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4027d60 osd=2 4 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.613+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 2 5 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 6 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 6 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4027d60 osd=2 3 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 2 6 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4027d60 osd=2 4 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 10 client.31266.objecter _send_command 6 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.627+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd411d4a0 con 0x7f5bd411c3e0 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd411d4a0 con 0x7f5bd411c3e0 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4027d60 osd=2 4 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 2 6 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 7 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5bd4027940 0x7f5bd4122060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 7 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 3 7 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd40277d0 osd=3 4 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 10 client.31266.objecter _send_command 7 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd4124cb0 con 0x7f5bd4027940 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd3fff700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5bd4027940 0x7f5bd4122060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd4027940 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.629+0000 7f5bd3fff700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5bd4027940 0x7f5bd4122060 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.630+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd4124cb0 con 0x7f5bd4027940 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.630+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.630+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 7 = system:0 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.630+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd40277d0 osd=3 4 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.630+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 3 7 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 8 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 8 2024-02-18T00:37:57.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd40277d0 osd=3 3 2024-02-18T00:37:57.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 3 8 2024-02-18T00:37:57.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd40277d0 osd=3 4 2024-02-18T00:37:57.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 10 client.31266.objecter _send_command 8 2024-02-18T00:37:57.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.644+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd411cf10 con 0x7f5bd4027940 2024-02-18T00:37:57.716 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd411cf10 con 0x7f5bd4027940 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd40277d0 osd=3 4 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 3 8 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 9 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bd411d080 0x7f5bd41236d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 9 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 4 9 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411cf10 osd=4 4 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 10 client.31266.objecter _send_command 9 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd411c070 con 0x7f5bd411d080 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bd411d080 0x7f5bd41236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd411d080 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.646+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bd411d080 0x7f5bd41236d0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.647+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd411c070 con 0x7f5bd411d080 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.647+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.647+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 9 = system:0 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.647+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd411cf10 osd=4 4 2024-02-18T00:37:57.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.647+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 4 9 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 10 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 10 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd411cf10 osd=4 3 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 4 10 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411cf10 osd=4 4 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 10 client.31266.objecter _send_command 10 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.662+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd411c070 con 0x7f5bd411d080 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd411c070 con 0x7f5bd411d080 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd411cf10 osd=4 4 2024-02-18T00:37:57.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 4 10 2024-02-18T00:37:57.720 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:37:57.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 11 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5bd41240d0 0x7f5bd4124550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 11 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 5 11 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411c070 osd=5 4 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 10 client.31266.objecter _send_command 11 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.664+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd4124ab0 con 0x7f5bd41240d0 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.665+0000 7f5bd2ffd700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5bd41240d0 0x7f5bd4124550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.665+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd41240d0 2024-02-18T00:37:57.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.665+0000 7f5bd2ffd700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5bd41240d0 0x7f5bd4124550 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.666+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd4124ab0 con 0x7f5bd41240d0 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.666+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.666+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 11 = system:0 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.666+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd411c070 osd=5 4 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.666+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 5 11 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 12 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 12 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd411c070 osd=5 3 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 5 12 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411c070 osd=5 4 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 10 client.31266.objecter _send_command 12 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.680+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd4124ab0 con 0x7f5bd41240d0 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd4124ab0 con 0x7f5bd41240d0 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd411c070 osd=5 4 2024-02-18T00:37:57.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 5 12 2024-02-18T00:37:57.724 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:37:57.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 13 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5bd4129350 0x7f5bd41297b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 13 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 6 13 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4124ab0 osd=6 4 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 10 client.31266.objecter _send_command 13 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.682+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd412b380 con 0x7f5bd4129350 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.683+0000 7f5bd3fff700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5bd4129350 0x7f5bd41297b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.683+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd4129350 2024-02-18T00:37:57.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.683+0000 7f5bd3fff700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5bd4129350 0x7f5bd41297b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.684+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd412b380 con 0x7f5bd4129350 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.684+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.684+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 13 = system:0 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.684+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4124ab0 osd=6 4 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.684+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 6 13 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 14 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 14 2024-02-18T00:37:57.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd4124ab0 osd=6 3 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 6 14 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4124ab0 osd=6 4 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 10 client.31266.objecter _send_command 14 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.699+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd412b380 con 0x7f5bd4129350 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.701+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd412b380 con 0x7f5bd4129350 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.701+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.701+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.701+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd4124ab0 osd=6 4 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.701+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 6 14 2024-02-18T00:37:57.727 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 15 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6812/53025171 conn(0x7f5bd412a270 0x7f5bd412a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 15 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 7 15 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd412b380 osd=7 4 2024-02-18T00:37:57.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 10 client.31266.objecter _send_command 15 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bd412acf0 con 0x7f5bd412a270 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6812/53025171 conn(0x7f5bd412a270 0x7f5bd412a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd0ff9700 10 client.31266.objecter ms_handle_connect 0x7f5bd412a270 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.702+0000 7f5bd37fe700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6812/53025171 conn(0x7f5bd412a270 0x7f5bd412a6d0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.703+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bd412acf0 con 0x7f5bd412a270 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.703+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.703+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 15 = system:0 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.703+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd412b380 osd=7 4 2024-02-18T00:37:57.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.703+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 7 15 2024-02-18T00:37:57.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:57.732 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:57.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:37:57.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 10 client.31266.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:37:57.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign -1 16 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 20 client.31266.objecter _get_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_remove -1 16 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 20 client.31266.objecter get_session s=0x7f5bd412b380 osd=7 3 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 15 client.31266.objecter _session_command_op_assign 7 16 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd412b380 osd=7 4 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 10 client.31266.objecter _send_command 16 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.718+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5bd412acf0 con 0x7f5bd412a270 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.720+0000 7f5bd0ff9700 1 -- 172.21.15.59:0/3182060748 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5bd412acf0 con 0x7f5bd412a270 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.720+0000 7f5bd0ff9700 10 client.31266.objecter ms_dispatch 0x7f5bd4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:37:57.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.720+0000 7f5bd0ff9700 10 client.31266.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.720+0000 7f5bd0ff9700 20 client.31266.objecter put_session s=0x7f5bd412b380 osd=7 4 2024-02-18T00:37:57.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.720+0000 7f5bd0ff9700 15 client.31266.objecter _session_command_op_remove 7 16 2024-02-18T00:37:57.734 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 librados: watch_flush enter 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 librados: watch_flush exit 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.0 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5bd40fee00 msgr2=0x7f5bd40ff200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5bd40fee00 0x7f5bd40ff200 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4107060 osd=0 1 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.1 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6805/26279110 conn(0x7f5bd40598f0 msgr2=0x7f5bd4059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6805/26279110 conn(0x7f5bd40598f0 0x7f5bd4059d10 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4028710 osd=1 1 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.2 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5bd411c3e0 msgr2=0x7f5bd411c7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5bd411c3e0 0x7f5bd411c7e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4027d60 osd=2 1 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.3 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5bd4027940 msgr2=0x7f5bd4122060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5bd4027940 0x7f5bd4122060 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd40277d0 osd=3 1 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.4 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd3fff700 1 -- 172.21.15.59:0/3182060748 reap_dead start 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bd411d080 msgr2=0x7f5bd41236d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bd411d080 0x7f5bd41236d0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd3fff700 1 -- 172.21.15.59:0/3182060748 reap_dead start 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411cf10 osd=4 1 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.5 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5bd41240d0 msgr2=0x7f5bd4124550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5bd41240d0 0x7f5bd4124550 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd411c070 osd=5 1 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.6 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5bd4129350 msgr2=0x7f5bd41297b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5bd4129350 0x7f5bd41297b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd4124ab0 osd=6 1 2024-02-18T00:37:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter close_session for osd.7 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6812/53025171 conn(0x7f5bd412a270 msgr2=0x7f5bd412a6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.137:6812/53025171 conn(0x7f5bd412a270 0x7f5bd412a6d0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter put_session s=0x7f5bd412b380 osd=7 1 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 20 client.31266.objecter shutdown clearing up homeless session... 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 10 client.31266.objecter successfully canceled tick 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bbc062c90 msgr2=0x7f5bbc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bbc062c90 0x7f5bbc065150 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 rx=0x7f5bc80043d0 tx=0x7f5bc800b040).stop 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd3fff700 1 -- 172.21.15.59:0/3182060748 reap_dead start 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40f8420 msgr2=0x7f5bd406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40f8420 0x7f5bd406b0d0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7f5bc40041d0 tx=0x7f5bc4009f60).stop 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 shutdown_connections 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 --2- 172.21.15.59:0/3182060748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5bd40f8420 0x7f5bd406b0d0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:37:57.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 >> 172.21.15.59:0/3182060748 conn(0x7f5bd40f3800 msgr2=0x7f5bd40f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 shutdown_connections 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 -- 172.21.15.59:0/3182060748 wait complete. 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:37:57.721+0000 7f5bd9b69700 1 librados: shutdown 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:37:57.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:57.861 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:57.922 INFO:tasks.workunit.client.0.smithi059.stdout:4c720e9:::166:head 2024-02-18T00:37:57.922 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:e694b040:::167:head 2024-02-18T00:37:57.922 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:afa38db2:::168:head 2024-02-18T00:37:57.922 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:77ba9f53:::169:head 2024-02-18T00:37:57.922 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:87495034:::170:head 2024-02-18T00:37:57.922 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:7c96bf0e:::171:head 2024-02-18T00:37:57.922 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:dbe346cc:::172:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:e943ec24:::173:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f97a9c0c:::174:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:6f26e74d:::175:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:4f95e106:::176:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:0e6f2f8f:::177:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:05db05f1:::178:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:38a78d66:::179:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:d095610b:::180:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:a1a9d709:::181:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:1e5d39db:::182:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:f7df4fb9:::183:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:03a7f161:::184:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:ba70721e:::185:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:28e5662d:::186:head 2024-02-18T00:37:57.923 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:973d52de:::187:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:4303eb1c:::188:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:b990b48e:::189:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:29b8165b:::190:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:3547f197:::191:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:7e260936:::192:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:1abec7b1:::193:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:10fdda93:::194:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:15817eea:::195:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:770bab57:::196:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:ed9e13e7:::197:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:71471a8f:::198:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: checking for 191:10fb1d02:::199:head 2024-02-18T00:37:57.924 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8021 ms) 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: first is 1708216610 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,1708216615,1708216616,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,1708216615,1708216616,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,1708216615,1708216616,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,1708216615,1708216616,1708216618,1708216619,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,1708216615,1708216616,1708216618,1708216619,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216610,1708216612,1708216613,1708216615,1708216616,1708216618,1708216619,0 2024-02-18T00:37:57.925 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708216613,1708216615,1708216616,1708216618,1708216619,1708216621,1708216622,0 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: first now 1708216613, trimmed 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20982 ms) 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: foo0 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14019 ms) 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (18002 ms) 2024-02-18T00:37:57.926 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-18T00:37:58.199 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:56.909169+0000 mon.a (mon.0) 2450 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]': finished 2024-02-18T00:37:58.199 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:56.909259+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e395: 8 total, 8 up, 7 in 2024-02-18T00:37:58.199 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:56.942690+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:58.199 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:56.943054+0000 mon.a (mon.0) 2452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:58.199 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:56.943583+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:58.199 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:56.943822+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:57.577521+0000 mon.a (mon.0) 2454 : audit [DBG] from='client.? 172.21.15.59:0/3182060748' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:56.909169+0000 mon.a (mon.0) 2450 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]': finished 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:56.909259+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e395: 8 total, 8 up, 7 in 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:56.942690+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:56.943054+0000 mon.a (mon.0) 2452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:56.943583+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:56.943822+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:58.200 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:57.577521+0000 mon.a (mon.0) 2454 : audit [DBG] from='client.? 172.21.15.59:0/3182060748' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:58.219 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":578,"stamp":"2024-02-18T00:37:53.889787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007237,"num_objects":375,"num_object_clones":2,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":1718,"num_read_kb":1341,"num_write":1203,"num_write_kb":4521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":2,"num_evict_kb":2,"num_promote":4,"num_flush_mode_high":2,"num_flush_mode_low":0,"num_evict_mode_some":2,"num_evict_mode_full":0,"num_objects_pinned":2,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2552,"ondisk_log_size":2552,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":720,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2163988,"kb_used_data":42332,"kb_used_omap":0,"kb_used_meta":2121600,"kb_avail":654052076,"statfs":{"total":671965249536,"available":669749325824,"internally_reserved":0,"allocated":43347968,"data_stored":27688560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2172518400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.198835"},"pg_stats":[{"pgid":"204.1f","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694699+0000","last_change":"2024-02-18T00:37:37.691057+0000","last_active":"2024-02-18T00:37:49.694699+0000","last_peered":"2024-02-18T00:37:49.694699+0000","last_clean":"2024-02-18T00:37:49.694699+0000","last_became_active":"2024-02-18T00:37:37.690668+0000","last_became_peered":"2024-02-18T00:37:37.690668+0000","last_unstale":"2024-02-18T00:37:49.694699+0000","last_undegraded":"2024-02-18T00:37:49.694699+0000","last_fullsized":"2024-02-18T00:37:49.694699+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.1e","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849347+0000","last_change":"2024-02-18T00:37:37.686442+0000","last_active":"2024-02-18T00:37:48.849347+0000","last_peered":"2024-02-18T00:37:48.849347+0000","last_clean":"2024-02-18T00:37:48.849347+0000","last_became_active":"2024-02-18T00:37:37.686132+0000","last_became_peered":"2024-02-18T00:37:37.686132+0000","last_unstale":"2024-02-18T00:37:48.849347+0000","last_undegraded":"2024-02-18T00:37:48.849347+0000","last_fullsized":"2024-02-18T00:37:48.849347+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.1d","version":"382'2","reported_seq":24,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724097+0000","last_change":"2024-02-18T00:37:37.691189+0000","last_active":"2024-02-18T00:37:48.724097+0000","last_peered":"2024-02-18T00:37:48.724097+0000","last_clean":"2024-02-18T00:37:48.724097+0000","last_became_active":"2024-02-18T00:37:37.690876+0000","last_became_peered":"2024-02-18T00:37:37.690876+0000","last_unstale":"2024-02-18T00:37:48.724097+0000","last_undegraded":"2024-02-18T00:37:48.724097+0000","last_fullsized":"2024-02-18T00:37:48.724097+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":1,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":1,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.1c","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849220+0000","last_change":"2024-02-18T00:37:37.690281+0000","last_active":"2024-02-18T00:37:48.849220+0000","last_peered":"2024-02-18T00:37:48.849220+0000","last_clean":"2024-02-18T00:37:48.849220+0000","last_became_active":"2024-02-18T00:37:37.689959+0000","last_became_peered":"2024-02-18T00:37:37.689959+0000","last_unstale":"2024-02-18T00:37:48.849220+0000","last_undegraded":"2024-02-18T00:37:48.849220+0000","last_fullsized":"2024-02-18T00:37:48.849220+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.1b","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755905+0000","last_change":"2024-02-18T00:37:37.690607+0000","last_active":"2024-02-18T00:37:48.755905+0000","last_peered":"2024-02-18T00:37:48.755905+0000","last_clean":"2024-02-18T00:37:48.755905+0000","last_became_active":"2024-02-18T00:37:37.690143+0000","last_became_peered":"2024-02-18T00:37:37.690143+0000","last_unstale":"2024-02-18T00:37:48.755905+0000","last_undegraded":"2024-02-18T00:37:48.755905+0000","last_fullsized":"2024-02-18T00:37:48.755905+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.1a","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692109+0000","last_change":"2024-02-18T00:37:37.690951+0000","last_active":"2024-02-18T00:37:49.692109+0000","last_peered":"2024-02-18T00:37:49.692109+0000","last_clean":"2024-02-18T00:37:49.692109+0000","last_became_active":"2024-02-18T00:37:37.690575+0000","last_became_peered":"2024-02-18T00:37:37.690575+0000","last_unstale":"2024-02-18T00:37:49.692109+0000","last_undegraded":"2024-02-18T00:37:49.692109+0000","last_fullsized":"2024-02-18T00:37:49.692109+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1],"acting":[2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.19","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848243+0000","last_change":"2024-02-18T00:37:37.685040+0000","last_active":"2024-02-18T00:37:48.848243+0000","last_peered":"2024-02-18T00:37:48.848243+0000","last_clean":"2024-02-18T00:37:48.848243+0000","last_became_active":"2024-02-18T00:37:37.684783+0000","last_became_peered":"2024-02-18T00:37:37.684783+0000","last_unstale":"2024-02-18T00:37:48.848243+0000","last_undegraded":"2024-02-18T00:37:48.848243+0000","last_fullsized":"2024-02-18T00:37:48.848243+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.18","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722814+0000","last_change":"2024-02-18T00:37:37.691121+0000","last_active":"2024-02-18T00:37:48.722814+0000","last_peered":"2024-02-18T00:37:48.722814+0000","last_clean":"2024-02-18T00:37:48.722814+0000","last_became_active":"2024-02-18T00:37:37.690776+0000","last_became_peered":"2024-02-18T00:37:37.690776+0000","last_unstale":"2024-02-18T00:37:48.722814+0000","last_undegraded":"2024-02-18T00:37:48.722814+0000","last_fullsized":"2024-02-18T00:37:48.722814+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.17","version":"387'2","reported_seq":25,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691898+0000","last_change":"2024-02-18T00:37:37.684600+0000","last_active":"2024-02-18T00:37:49.691898+0000","last_peered":"2024-02-18T00:37:49.691898+0000","last_clean":"2024-02-18T00:37:49.691898+0000","last_became_active":"2024-02-18T00:37:37.684352+0000","last_became_peered":"2024-02-18T00:37:37.684352+0000","last_unstale":"2024-02-18T00:37:49.691898+0000","last_undegraded":"2024-02-18T00:37:49.691898+0000","last_fullsized":"2024-02-18T00:37:49.691898+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":1,"num_evict_kb":1,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.16","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849695+0000","last_change":"2024-02-18T00:37:37.686206+0000","last_active":"2024-02-18T00:37:48.849695+0000","last_peered":"2024-02-18T00:37:48.849695+0000","last_clean":"2024-02-18T00:37:48.849695+0000","last_became_active":"2024-02-18T00:37:37.685931+0000","last_became_peered":"2024-02-18T00:37:37.685931+0000","last_unstale":"2024-02-18T00:37:48.849695+0000","last_undegraded":"2024-02-18T00:37:48.849695+0000","last_fullsized":"2024-02-18T00:37:48.849695+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.15","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694503+0000","last_change":"2024-02-18T00:37:37.683379+0000","last_active":"2024-02-18T00:37:49.694503+0000","last_peered":"2024-02-18T00:37:49.694503+0000","last_clean":"2024-02-18T00:37:49.694503+0000","last_became_active":"2024-02-18T00:37:37.683099+0000","last_became_peered":"2024-02-18T00:37:37.683099+0000","last_unstale":"2024-02-18T00:37:49.694503+0000","last_undegraded":"2024-02-18T00:37:49.694503+0000","last_fullsized":"2024-02-18T00:37:49.694503+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.14","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.756840+0000","last_change":"2024-02-18T00:37:37.689283+0000","last_active":"2024-02-18T00:37:48.756840+0000","last_peered":"2024-02-18T00:37:48.756840+0000","last_clean":"2024-02-18T00:37:48.756840+0000","last_became_active":"2024-02-18T00:37:37.688954+0000","last_became_peered":"2024-02-18T00:37:37.688954+0000","last_unstale":"2024-02-18T00:37:48.756840+0000","last_undegraded":"2024-02-18T00:37:48.756840+0000","last_fullsized":"2024-02-18T00:37:48.756840+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.13","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693449+0000","last_change":"2024-02-18T00:37:37.682492+0000","last_active":"2024-02-18T00:37:49.693449+0000","last_peered":"2024-02-18T00:37:49.693449+0000","last_clean":"2024-02-18T00:37:49.693449+0000","last_became_active":"2024-02-18T00:37:37.682181+0000","last_became_peered":"2024-02-18T00:37:37.682181+0000","last_unstale":"2024-02-18T00:37:49.693449+0000","last_undegraded":"2024-02-18T00:37:49.693449+0000","last_fullsized":"2024-02-18T00:37:49.693449+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.12","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693046+0000","last_change":"2024-02-18T00:37:37.681706+0000","last_active":"2024-02-18T00:37:49.693046+0000","last_peered":"2024-02-18T00:37:49.693046+0000","last_clean":"2024-02-18T00:37:49.693046+0000","last_became_active":"2024-02-18T00:37:37.681258+0000","last_became_peered":"2024-02-18T00:37:37.681258+0000","last_unstale":"2024-02-18T00:37:49.693046+0000","last_undegraded":"2024-02-18T00:37:49.693046+0000","last_fullsized":"2024-02-18T00:37:49.693046+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.11","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950989+0000","last_change":"2024-02-18T00:37:37.679936+0000","last_active":"2024-02-18T00:37:49.950989+0000","last_peered":"2024-02-18T00:37:49.950989+0000","last_clean":"2024-02-18T00:37:49.950989+0000","last_became_active":"2024-02-18T00:37:37.679607+0000","last_became_peered":"2024-02-18T00:37:37.679607+0000","last_unstale":"2024-02-18T00:37:49.950989+0000","last_undegraded":"2024-02-18T00:37:49.950989+0000","last_fullsized":"2024-02-18T00:37:49.950989+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"204.10","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755064+0000","last_change":"2024-02-18T00:37:37.689865+0000","last_active":"2024-02-18T00:37:48.755064+0000","last_peered":"2024-02-18T00:37:48.755064+0000","last_clean":"2024-02-18T00:37:48.755064+0000","last_became_active":"2024-02-18T00:37:37.689544+0000","last_became_peered":"2024-02-18T00:37:37.689544+0000","last_unstale":"2024-02-18T00:37:48.755064+0000","last_undegraded":"2024-02-18T00:37:48.755064+0000","last_fullsized":"2024-02-18T00:37:48.755064+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.f","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726443+0000","last_change":"2024-02-18T00:37:37.691260+0000","last_active":"2024-02-18T00:37:48.726443+0000","last_peered":"2024-02-18T00:37:48.726443+0000","last_clean":"2024-02-18T00:37:48.726443+0000","last_became_active":"2024-02-18T00:37:37.690980+0000","last_became_peered":"2024-02-18T00:37:37.690980+0000","last_unstale":"2024-02-18T00:37:48.726443+0000","last_undegraded":"2024-02-18T00:37:48.726443+0000","last_fullsized":"2024-02-18T00:37:48.726443+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.e","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694275+0000","last_change":"2024-02-18T00:37:37.681239+0000","last_active":"2024-02-18T00:37:49.694275+0000","last_peered":"2024-02-18T00:37:49.694275+0000","last_clean":"2024-02-18T00:37:49.694275+0000","last_became_active":"2024-02-18T00:37:37.681019+0000","last_became_peered":"2024-02-18T00:37:37.681019+0000","last_unstale":"2024-02-18T00:37:49.694275+0000","last_undegraded":"2024-02-18T00:37:49.694275+0000","last_fullsized":"2024-02-18T00:37:49.694275+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.d","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724155+0000","last_change":"2024-02-18T00:37:37.684496+0000","last_active":"2024-02-18T00:37:48.724155+0000","last_peered":"2024-02-18T00:37:48.724155+0000","last_clean":"2024-02-18T00:37:48.724155+0000","last_became_active":"2024-02-18T00:37:37.684149+0000","last_became_peered":"2024-02-18T00:37:37.684149+0000","last_unstale":"2024-02-18T00:37:48.724155+0000","last_undegraded":"2024-02-18T00:37:48.724155+0000","last_fullsized":"2024-02-18T00:37:48.724155+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"204.c","version":"387'2","reported_seq":24,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722865+0000","last_change":"2024-02-18T00:37:37.689105+0000","last_active":"2024-02-18T00:37:48.722865+0000","last_peered":"2024-02-18T00:37:48.722865+0000","last_clean":"2024-02-18T00:37:48.722865+0000","last_became_active":"2024-02-18T00:37:37.688822+0000","last_became_peered":"2024-02-18T00:37:37.688822+0000","last_unstale":"2024-02-18T00:37:48.722865+0000","last_undegraded":"2024-02-18T00:37:48.722865+0000","last_fullsized":"2024-02-18T00:37:48.722865+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":1,"num_evict_kb":1,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"204.b","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755986+0000","last_change":"2024-02-18T00:37:37.690442+0000","last_active":"2024-02-18T00:37:48.755986+0000","last_peered":"2024-02-18T00:37:48.755986+0000","last_clean":"2024-02-18T00:37:48.755986+0000","last_became_active":"2024-02-18T00:37:37.689811+0000","last_became_peered":"2024-02-18T00:37:37.689811+0000","last_unstale":"2024-02-18T00:37:48.755986+0000","last_undegraded":"2024-02-18T00:37:48.755986+0000","last_fullsized":"2024-02-18T00:37:48.755986+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.a","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693199+0000","last_change":"2024-02-18T00:37:37.681540+0000","last_active":"2024-02-18T00:37:49.693199+0000","last_peered":"2024-02-18T00:37:49.693199+0000","last_clean":"2024-02-18T00:37:49.693199+0000","last_became_active":"2024-02-18T00:37:37.680851+0000","last_became_peered":"2024-02-18T00:37:37.680851+0000","last_unstale":"2024-02-18T00:37:49.693199+0000","last_undegraded":"2024-02-18T00:37:49.693199+0000","last_fullsized":"2024-02-18T00:37:49.693199+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.9","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691785+0000","last_change":"2024-02-18T00:37:37.683135+0000","last_active":"2024-02-18T00:37:49.691785+0000","last_peered":"2024-02-18T00:37:49.691785+0000","last_clean":"2024-02-18T00:37:49.691785+0000","last_became_active":"2024-02-18T00:37:37.682836+0000","last_became_peered":"2024-02-18T00:37:37.682836+0000","last_unstale":"2024-02-18T00:37:49.691785+0000","last_undegraded":"2024-02-18T00:37:49.691785+0000","last_fullsized":"2024-02-18T00:37:49.691785+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"204.8","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755123+0000","last_change":"2024-02-18T00:37:37.686676+0000","last_active":"2024-02-18T00:37:48.755123+0000","last_peered":"2024-02-18T00:37:48.755123+0000","last_clean":"2024-02-18T00:37:48.755123+0000","last_became_active":"2024-02-18T00:37:37.686332+0000","last_became_peered":"2024-02-18T00:37:37.686332+0000","last_unstale":"2024-02-18T00:37:48.755123+0000","last_undegraded":"2024-02-18T00:37:48.755123+0000","last_fullsized":"2024-02-18T00:37:48.755123+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"204.7","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691940+0000","last_change":"2024-02-18T00:37:37.681879+0000","last_active":"2024-02-18T00:37:49.691940+0000","last_peered":"2024-02-18T00:37:49.691940+0000","last_clean":"2024-02-18T00:37:49.691940+0000","last_became_active":"2024-02-18T00:37:37.681577+0000","last_became_peered":"2024-02-18T00:37:37.681577+0000","last_unstale":"2024-02-18T00:37:49.691940+0000","last_undegraded":"2024-02-18T00:37:49.691940+0000","last_fullsized":"2024-02-18T00:37:49.691940+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.6","version":"0'0","reported_seq":22,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694163+0000","last_change":"2024-02-18T00:37:37.683674+0000","last_active":"2024-02-18T00:37:49.694163+0000","last_peered":"2024-02-18T00:37:49.694163+0000","last_clean":"2024-02-18T00:37:49.694163+0000","last_became_active":"2024-02-18T00:37:37.683363+0000","last_became_peered":"2024-02-18T00:37:37.683363+0000","last_unstale":"2024-02-18T00:37:49.694163+0000","last_undegraded":"2024-02-18T00:37:49.694163+0000","last_fullsized":"2024-02-18T00:37:49.694163+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.5","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724180+0000","last_change":"2024-02-18T00:37:37.688299+0000","last_active":"2024-02-18T00:37:48.724180+0000","last_peered":"2024-02-18T00:37:48.724180+0000","last_clean":"2024-02-18T00:37:48.724180+0000","last_became_active":"2024-02-18T00:37:37.688016+0000","last_became_peered":"2024-02-18T00:37:37.688016+0000","last_unstale":"2024-02-18T00:37:48.724180+0000","last_undegraded":"2024-02-18T00:37:48.724180+0000","last_fullsized":"2024-02-18T00:37:48.724180+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,3],"acting":[6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.4","version":"382'2","reported_seq":25,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693558+0000","last_change":"2024-02-18T00:37:37.692443+0000","last_active":"2024-02-18T00:37:49.693558+0000","last_peered":"2024-02-18T00:37:49.693558+0000","last_clean":"2024-02-18T00:37:49.693558+0000","last_became_active":"2024-02-18T00:37:37.691353+0000","last_became_peered":"2024-02-18T00:37:37.691353+0000","last_unstale":"2024-02-18T00:37:49.693558+0000","last_undegraded":"2024-02-18T00:37:49.693558+0000","last_fullsized":"2024-02-18T00:37:49.693558+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":1,"num_flush_mode_low":0,"num_evict_mode_some":1,"num_evict_mode_full":0,"num_objects_pinned":1,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"204.3","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724354+0000","last_change":"2024-02-18T00:37:37.687815+0000","last_active":"2024-02-18T00:37:48.724354+0000","last_peered":"2024-02-18T00:37:48.724354+0000","last_clean":"2024-02-18T00:37:48.724354+0000","last_became_active":"2024-02-18T00:37:37.687486+0000","last_became_peered":"2024-02-18T00:37:37.687486+0000","last_unstale":"2024-02-18T00:37:48.724354+0000","last_undegraded":"2024-02-18T00:37:48.724354+0000","last_fullsized":"2024-02-18T00:37:48.724354+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"204.2","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725747+0000","last_change":"2024-02-18T00:37:37.691352+0000","last_active":"2024-02-18T00:37:48.725747+0000","last_peered":"2024-02-18T00:37:48.725747+0000","last_clean":"2024-02-18T00:37:48.725747+0000","last_became_active":"2024-02-18T00:37:37.690874+0000","last_became_peered":"2024-02-18T00:37:37.690874+0000","last_unstale":"2024-02-18T00:37:48.725747+0000","last_undegraded":"2024-02-18T00:37:48.725747+0000","last_fullsized":"2024-02-18T00:37:48.725747+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"204.1","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848863+0000","last_change":"2024-02-18T00:37:37.686342+0000","last_active":"2024-02-18T00:37:48.848863+0000","last_peered":"2024-02-18T00:37:48.848863+0000","last_clean":"2024-02-18T00:37:48.848863+0000","last_became_active":"2024-02-18T00:37:37.686026+0000","last_became_peered":"2024-02-18T00:37:37.686026+0000","last_unstale":"2024-02-18T00:37:48.848863+0000","last_undegraded":"2024-02-18T00:37:48.848863+0000","last_fullsized":"2024-02-18T00:37:48.848863+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"204.0","version":"0'0","reported_seq":21,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723467+0000","last_change":"2024-02-18T00:37:37.694899+0000","last_active":"2024-02-18T00:37:48.723467+0000","last_peered":"2024-02-18T00:37:48.723467+0000","last_clean":"2024-02-18T00:37:48.723467+0000","last_became_active":"2024-02-18T00:37:37.694553+0000","last_became_peered":"2024-02-18T00:37:37.694553+0000","last_unstale":"2024-02-18T00:37:48.723467+0000","last_undegraded":"2024-02-18T00:37:48.723467+0000","last_fullsized":"2024-02-18T00:37:48.723467+0000","mapping_epoch":378,"log_start":"0'0","ondisk_log_start":"0'0","created":378,"last_epoch_clean":379,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:36.658826+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.658826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":363,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.690888+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:37:49.690888+0000","last_peered":"2024-02-18T00:37:49.690888+0000","last_clean":"2024-02-18T00:37:49.690888+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:37:49.690888+0000","last_undegraded":"2024-02-18T00:37:49.690888+0000","last_fullsized":"2024-02-18T00:37:49.690888+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"347'2","reported_seq":331,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692084+0000","last_change":"2024-02-18T00:37:29.759198+0000","last_active":"2024-02-18T00:37:49.692084+0000","last_peered":"2024-02-18T00:37:49.692084+0000","last_clean":"2024-02-18T00:37:49.692084+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:37:49.692084+0000","last_undegraded":"2024-02-18T00:37:49.692084+0000","last_fullsized":"2024-02-18T00:37:49.692084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'2","last_scrub_stamp":"2024-02-18T00:37:29.759114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.759114+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":390,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725882+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:37:48.725882+0000","last_peered":"2024-02-18T00:37:48.725882+0000","last_clean":"2024-02-18T00:37:48.725882+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:37:48.725882+0000","last_undegraded":"2024-02-18T00:37:48.725882+0000","last_fullsized":"2024-02-18T00:37:48.725882+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"347'75","reported_seq":394,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691857+0000","last_change":"2024-02-18T00:36:34.581378+0000","last_active":"2024-02-18T00:37:49.691857+0000","last_peered":"2024-02-18T00:37:49.691857+0000","last_clean":"2024-02-18T00:37:49.691857+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:37:49.691857+0000","last_undegraded":"2024-02-18T00:37:49.691857+0000","last_fullsized":"2024-02-18T00:37:49.691857+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'73","last_scrub_stamp":"2024-02-18T00:36:34.581302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.581302+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"379'121","reported_seq":432,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.754435+0000","last_change":"2024-02-18T00:36:36.464430+0000","last_active":"2024-02-18T00:37:48.754435+0000","last_peered":"2024-02-18T00:37:48.754435+0000","last_clean":"2024-02-18T00:37:48.754435+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:37:48.754435+0000","last_undegraded":"2024-02-18T00:37:48.754435+0000","last_fullsized":"2024-02-18T00:37:48.754435+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'120","last_scrub_stamp":"2024-02-18T00:36:36.464361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.464361+0000","log_size":121,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":37,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":407,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950817+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:37:49.950817+0000","last_peered":"2024-02-18T00:37:49.950817+0000","last_clean":"2024-02-18T00:37:49.950817+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:37:49.950817+0000","last_undegraded":"2024-02-18T00:37:49.950817+0000","last_fullsized":"2024-02-18T00:37:49.950817+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"347'125","reported_seq":406,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694315+0000","last_change":"2024-02-18T00:36:53.649744+0000","last_active":"2024-02-18T00:37:49.694315+0000","last_peered":"2024-02-18T00:37:49.694315+0000","last_clean":"2024-02-18T00:37:49.694315+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:37:49.694315+0000","last_undegraded":"2024-02-18T00:37:49.694315+0000","last_fullsized":"2024-02-18T00:37:49.694315+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'124","last_scrub_stamp":"2024-02-18T00:36:53.649677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.649677+0000","log_size":125,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":38,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"347'74","reported_seq":387,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692074+0000","last_change":"2024-02-18T00:36:37.532389+0000","last_active":"2024-02-18T00:37:49.692074+0000","last_peered":"2024-02-18T00:37:49.692074+0000","last_clean":"2024-02-18T00:37:49.692074+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:37:49.692074+0000","last_undegraded":"2024-02-18T00:37:49.692074+0000","last_fullsized":"2024-02-18T00:37:49.692074+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'73","last_scrub_stamp":"2024-02-18T00:36:37.532316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.532316+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":23,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'70","reported_seq":358,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694846+0000","last_change":"2024-02-18T00:37:44.685708+0000","last_active":"2024-02-18T00:37:49.694846+0000","last_peered":"2024-02-18T00:37:49.694846+0000","last_clean":"2024-02-18T00:37:49.694846+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:37:49.694846+0000","last_undegraded":"2024-02-18T00:37:49.694846+0000","last_fullsized":"2024-02-18T00:37:49.694846+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-18T00:37:44.685592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.685592+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"347'52","reported_seq":370,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951295+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:37:49.951295+0000","last_peered":"2024-02-18T00:37:49.951295+0000","last_clean":"2024-02-18T00:37:49.951295+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:37:49.951295+0000","last_undegraded":"2024-02-18T00:37:49.951295+0000","last_fullsized":"2024-02-18T00:37:49.951295+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'41","reported_seq":337,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692278+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:37:49.692278+0000","last_peered":"2024-02-18T00:37:49.692278+0000","last_clean":"2024-02-18T00:37:49.692278+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:37:49.692278+0000","last_undegraded":"2024-02-18T00:37:49.692278+0000","last_fullsized":"2024-02-18T00:37:49.692278+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"347'55","reported_seq":351,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.847521+0000","last_change":"2024-02-18T00:36:11.810050+0000","last_active":"2024-02-18T00:37:48.847521+0000","last_peered":"2024-02-18T00:37:48.847521+0000","last_clean":"2024-02-18T00:37:48.847521+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:37:48.847521+0000","last_undegraded":"2024-02-18T00:37:48.847521+0000","last_fullsized":"2024-02-18T00:37:48.847521+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'54","last_scrub_stamp":"2024-02-18T00:36:11.809925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.809925+0000","log_size":55,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":2,"num_write":18,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"379'18","reported_seq":340,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849344+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:37:48.849344+0000","last_peered":"2024-02-18T00:37:48.849344+0000","last_clean":"2024-02-18T00:37:48.849344+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:37:48.849344+0000","last_undegraded":"2024-02-18T00:37:48.849344+0000","last_fullsized":"2024-02-18T00:37:48.849344+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"347'29","reported_seq":374,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950700+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:37:49.950700+0000","last_peered":"2024-02-18T00:37:49.950700+0000","last_clean":"2024-02-18T00:37:49.950700+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:37:49.950700+0000","last_undegraded":"2024-02-18T00:37:49.950700+0000","last_fullsized":"2024-02-18T00:37:49.950700+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"347'83","reported_seq":392,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.758980+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:37:48.758980+0000","last_peered":"2024-02-18T00:37:48.758980+0000","last_clean":"2024-02-18T00:37:48.758980+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:37:48.758980+0000","last_undegraded":"2024-02-18T00:37:48.758980+0000","last_fullsized":"2024-02-18T00:37:48.758980+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":67,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":374,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951100+0000","last_change":"2024-02-18T00:36:34.207908+0000","last_active":"2024-02-18T00:37:49.951100+0000","last_peered":"2024-02-18T00:37:49.951100+0000","last_clean":"2024-02-18T00:37:49.951100+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:37:49.951100+0000","last_undegraded":"2024-02-18T00:37:49.951100+0000","last_fullsized":"2024-02-18T00:37:49.951100+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:36:34.207840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.207840+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'57","reported_seq":392,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694742+0000","last_change":"2024-02-18T00:37:36.817791+0000","last_active":"2024-02-18T00:37:49.694742+0000","last_peered":"2024-02-18T00:37:49.694742+0000","last_clean":"2024-02-18T00:37:49.694742+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:37:49.694742+0000","last_undegraded":"2024-02-18T00:37:49.694742+0000","last_fullsized":"2024-02-18T00:37:49.694742+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'57","last_scrub_stamp":"2024-02-18T00:37:36.817719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.817719+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"214'19","reported_seq":349,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691994+0000","last_change":"2024-02-18T00:37:31.817355+0000","last_active":"2024-02-18T00:37:49.691994+0000","last_peered":"2024-02-18T00:37:49.691994+0000","last_clean":"2024-02-18T00:37:49.691994+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:37:49.691994+0000","last_undegraded":"2024-02-18T00:37:49.691994+0000","last_fullsized":"2024-02-18T00:37:49.691994+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:37:31.817281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.817281+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"347'74","reported_seq":389,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692341+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:37:49.692341+0000","last_peered":"2024-02-18T00:37:49.692341+0000","last_clean":"2024-02-18T00:37:49.692341+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:37:49.692341+0000","last_undegraded":"2024-02-18T00:37:49.692341+0000","last_fullsized":"2024-02-18T00:37:49.692341+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"361'18","reported_seq":340,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.756135+0000","last_change":"2024-02-18T00:36:43.345182+0000","last_active":"2024-02-18T00:37:48.756135+0000","last_peered":"2024-02-18T00:37:48.756135+0000","last_clean":"2024-02-18T00:37:48.756135+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:37:48.756135+0000","last_undegraded":"2024-02-18T00:37:48.756135+0000","last_fullsized":"2024-02-18T00:37:48.756135+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'17","last_scrub_stamp":"2024-02-18T00:36:43.345064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.345064+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":353,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950343+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:37:49.950343+0000","last_peered":"2024-02-18T00:37:49.950343+0000","last_clean":"2024-02-18T00:37:49.950343+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:37:49.950343+0000","last_undegraded":"2024-02-18T00:37:49.950343+0000","last_fullsized":"2024-02-18T00:37:49.950343+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"379'21","reported_seq":359,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726614+0000","last_change":"2024-02-18T00:37:41.575899+0000","last_active":"2024-02-18T00:37:48.726614+0000","last_peered":"2024-02-18T00:37:48.726614+0000","last_clean":"2024-02-18T00:37:48.726614+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:37:48.726614+0000","last_undegraded":"2024-02-18T00:37:48.726614+0000","last_fullsized":"2024-02-18T00:37:48.726614+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'21","last_scrub_stamp":"2024-02-18T00:37:41.575814+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.575814+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000035,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":3914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":416,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849477+0000","last_change":"2024-02-18T00:36:34.017135+0000","last_active":"2024-02-18T00:37:48.849477+0000","last_peered":"2024-02-18T00:37:48.849477+0000","last_clean":"2024-02-18T00:37:48.849477+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:37:48.849477+0000","last_undegraded":"2024-02-18T00:37:48.849477+0000","last_fullsized":"2024-02-18T00:37:48.849477+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:36:34.017065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.017065+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":308,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722249+0000","last_change":"2024-02-18T00:36:37.504644+0000","last_active":"2024-02-18T00:37:48.722249+0000","last_peered":"2024-02-18T00:37:48.722249+0000","last_clean":"2024-02-18T00:37:48.722249+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:37:48.722249+0000","last_undegraded":"2024-02-18T00:37:48.722249+0000","last_fullsized":"2024-02-18T00:37:48.722249+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:37.504568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.504568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":431,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693652+0000","last_change":"2024-02-18T00:36:38.583124+0000","last_active":"2024-02-18T00:37:49.693652+0000","last_peered":"2024-02-18T00:37:49.693652+0000","last_clean":"2024-02-18T00:37:49.693652+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:37:49.693652+0000","last_undegraded":"2024-02-18T00:37:49.693652+0000","last_fullsized":"2024-02-18T00:37:49.693652+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'112","last_scrub_stamp":"2024-02-18T00:36:38.583056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.583056+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":307,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848318+0000","last_change":"2024-02-18T00:37:01.774863+0000","last_active":"2024-02-18T00:37:48.848318+0000","last_peered":"2024-02-18T00:37:48.848318+0000","last_clean":"2024-02-18T00:37:48.848318+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:37:48.848318+0000","last_undegraded":"2024-02-18T00:37:48.848318+0000","last_fullsized":"2024-02-18T00:37:48.848318+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.774795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.774795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":365,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950391+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:37:49.950391+0000","last_peered":"2024-02-18T00:37:49.950391+0000","last_clean":"2024-02-18T00:37:49.950391+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:37:49.950391+0000","last_undegraded":"2024-02-18T00:37:49.950391+0000","last_fullsized":"2024-02-18T00:37:49.950391+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":267,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951448+0000","last_change":"2024-02-18T00:37:27.130328+0000","last_active":"2024-02-18T00:37:49.951448+0000","last_peered":"2024-02-18T00:37:49.951448+0000","last_clean":"2024-02-18T00:37:49.951448+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:37:49.951448+0000","last_undegraded":"2024-02-18T00:37:49.951448+0000","last_fullsized":"2024-02-18T00:37:49.951448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.130252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.130252+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":417,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725838+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:37:48.725838+0000","last_peered":"2024-02-18T00:37:48.725838+0000","last_clean":"2024-02-18T00:37:48.725838+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:37:48.725838+0000","last_undegraded":"2024-02-18T00:37:48.725838+0000","last_fullsized":"2024-02-18T00:37:48.725838+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":265,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950496+0000","last_change":"2024-02-18T00:36:53.345938+0000","last_active":"2024-02-18T00:37:49.950496+0000","last_peered":"2024-02-18T00:37:49.950496+0000","last_clean":"2024-02-18T00:37:49.950496+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:37:49.950496+0000","last_undegraded":"2024-02-18T00:37:49.950496+0000","last_fullsized":"2024-02-18T00:37:49.950496+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.345872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":394,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722165+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:37:48.722165+0000","last_peered":"2024-02-18T00:37:48.722165+0000","last_clean":"2024-02-18T00:37:48.722165+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:37:48.722165+0000","last_undegraded":"2024-02-18T00:37:48.722165+0000","last_fullsized":"2024-02-18T00:37:48.722165+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":317,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951601+0000","last_change":"2024-02-18T00:36:39.168150+0000","last_active":"2024-02-18T00:37:49.951601+0000","last_peered":"2024-02-18T00:37:49.951601+0000","last_clean":"2024-02-18T00:37:49.951601+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:37:49.951601+0000","last_undegraded":"2024-02-18T00:37:49.951601+0000","last_fullsized":"2024-02-18T00:37:49.951601+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:36:39.168079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:39.168079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"347'18","reported_seq":344,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848422+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:37:48.848422+0000","last_peered":"2024-02-18T00:37:48.848422+0000","last_clean":"2024-02-18T00:37:48.848422+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:37:48.848422+0000","last_undegraded":"2024-02-18T00:37:48.848422+0000","last_fullsized":"2024-02-18T00:37:48.848422+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":308,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724911+0000","last_change":"2024-02-18T00:36:35.460784+0000","last_active":"2024-02-18T00:37:48.724911+0000","last_peered":"2024-02-18T00:37:48.724911+0000","last_clean":"2024-02-18T00:37:48.724911+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:37:48.724911+0000","last_undegraded":"2024-02-18T00:37:48.724911+0000","last_fullsized":"2024-02-18T00:37:48.724911+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.460703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.460703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":376,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692057+0000","last_change":"2024-02-18T00:36:41.624250+0000","last_active":"2024-02-18T00:37:49.692057+0000","last_peered":"2024-02-18T00:37:49.692057+0000","last_clean":"2024-02-18T00:37:49.692057+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:37:49.692057+0000","last_undegraded":"2024-02-18T00:37:49.692057+0000","last_fullsized":"2024-02-18T00:37:49.692057+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:36:41.624180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.624180+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":298,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848874+0000","last_change":"2024-02-18T00:36:43.933486+0000","last_active":"2024-02-18T00:37:48.848874+0000","last_peered":"2024-02-18T00:37:48.848874+0000","last_clean":"2024-02-18T00:37:48.848874+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:37:48.848874+0000","last_undegraded":"2024-02-18T00:37:48.848874+0000","last_fullsized":"2024-02-18T00:37:48.848874+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.933414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.933414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":333,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726463+0000","last_change":"2024-02-18T00:37:37.618760+0000","last_active":"2024-02-18T00:37:48.726463+0000","last_peered":"2024-02-18T00:37:48.726463+0000","last_clean":"2024-02-18T00:37:48.726463+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:37:48.726463+0000","last_undegraded":"2024-02-18T00:37:48.726463+0000","last_fullsized":"2024-02-18T00:37:48.726463+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:37:37.618667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.618667+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":301,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726084+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:37:48.726084+0000","last_peered":"2024-02-18T00:37:48.726084+0000","last_clean":"2024-02-18T00:37:48.726084+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:37:48.726084+0000","last_undegraded":"2024-02-18T00:37:48.726084+0000","last_fullsized":"2024-02-18T00:37:48.726084+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"379'18","reported_seq":340,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724225+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:37:48.724225+0000","last_peered":"2024-02-18T00:37:48.724225+0000","last_clean":"2024-02-18T00:37:48.724225+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:37:48.724225+0000","last_undegraded":"2024-02-18T00:37:48.724225+0000","last_fullsized":"2024-02-18T00:37:48.724225+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":382,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693077+0000","last_change":"2024-02-18T00:36:45.627771+0000","last_active":"2024-02-18T00:37:49.693077+0000","last_peered":"2024-02-18T00:37:49.693077+0000","last_clean":"2024-02-18T00:37:49.693077+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:37:49.693077+0000","last_undegraded":"2024-02-18T00:37:49.693077+0000","last_fullsized":"2024-02-18T00:37:49.693077+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.627700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.627700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":344,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724782+0000","last_change":"2024-02-18T00:37:38.588259+0000","last_active":"2024-02-18T00:37:48.724782+0000","last_peered":"2024-02-18T00:37:48.724782+0000","last_clean":"2024-02-18T00:37:48.724782+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:37:48.724782+0000","last_undegraded":"2024-02-18T00:37:48.724782+0000","last_fullsized":"2024-02-18T00:37:48.724782+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'18","last_scrub_stamp":"2024-02-18T00:37:38.588190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.588190+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":331,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723577+0000","last_change":"2024-02-18T00:37:37.726927+0000","last_active":"2024-02-18T00:37:48.723577+0000","last_peered":"2024-02-18T00:37:48.723577+0000","last_clean":"2024-02-18T00:37:48.723577+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:37:48.723577+0000","last_undegraded":"2024-02-18T00:37:48.723577+0000","last_fullsized":"2024-02-18T00:37:48.723577+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.726861+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":305,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.754530+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:37:48.754530+0000","last_peered":"2024-02-18T00:37:48.754530+0000","last_clean":"2024-02-18T00:37:48.754530+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:37:48.754530+0000","last_undegraded":"2024-02-18T00:37:48.754530+0000","last_fullsized":"2024-02-18T00:37:48.754530+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":333,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722663+0000","last_change":"2024-02-18T00:37:33.619485+0000","last_active":"2024-02-18T00:37:48.722663+0000","last_peered":"2024-02-18T00:37:48.722663+0000","last_clean":"2024-02-18T00:37:48.722663+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:37:48.722663+0000","last_undegraded":"2024-02-18T00:37:48.722663+0000","last_fullsized":"2024-02-18T00:37:48.722663+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:33.619410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.619410+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":436,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723646+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:37:48.723646+0000","last_peered":"2024-02-18T00:37:48.723646+0000","last_clean":"2024-02-18T00:37:48.723646+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:37:48.723646+0000","last_undegraded":"2024-02-18T00:37:48.723646+0000","last_fullsized":"2024-02-18T00:37:48.723646+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":375,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694744+0000","last_change":"2024-02-18T00:37:02.627426+0000","last_active":"2024-02-18T00:37:49.694744+0000","last_peered":"2024-02-18T00:37:49.694744+0000","last_clean":"2024-02-18T00:37:49.694744+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:37:49.694744+0000","last_undegraded":"2024-02-18T00:37:49.694744+0000","last_fullsized":"2024-02-18T00:37:49.694744+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.627349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.627349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":353,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723998+0000","last_change":"2024-02-18T00:37:33.616952+0000","last_active":"2024-02-18T00:37:48.723998+0000","last_peered":"2024-02-18T00:37:48.723998+0000","last_clean":"2024-02-18T00:37:48.723998+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:37:48.723998+0000","last_undegraded":"2024-02-18T00:37:48.723998+0000","last_fullsized":"2024-02-18T00:37:48.723998+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.616874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.616874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":303,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694576+0000","last_change":"2024-02-18T00:37:01.919166+0000","last_active":"2024-02-18T00:37:49.694576+0000","last_peered":"2024-02-18T00:37:49.694576+0000","last_clean":"2024-02-18T00:37:49.694576+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:37:49.694576+0000","last_undegraded":"2024-02-18T00:37:49.694576+0000","last_fullsized":"2024-02-18T00:37:49.694576+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.919082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.919082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":410,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.523224+0000","last_change":"2024-02-18T00:37:49.523198+0000","last_active":"2024-02-18T00:37:49.523224+0000","last_peered":"2024-02-18T00:37:49.523224+0000","last_clean":"2024-02-18T00:37:49.523224+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:37:49.523224+0000","last_undegraded":"2024-02-18T00:37:49.523224+0000","last_fullsized":"2024-02-18T00:37:49.523224+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:49.523121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:49.523121+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":363,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.698112+0000","last_change":"2024-02-18T00:36:43.837775+0000","last_active":"2024-02-18T00:37:49.698112+0000","last_peered":"2024-02-18T00:37:49.698112+0000","last_clean":"2024-02-18T00:37:49.698112+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:37:49.698112+0000","last_undegraded":"2024-02-18T00:37:49.698112+0000","last_fullsized":"2024-02-18T00:37:49.698112+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.837702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.837702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":360,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692431+0000","last_change":"2024-02-18T00:36:56.601833+0000","last_active":"2024-02-18T00:37:49.692431+0000","last_peered":"2024-02-18T00:37:49.692431+0000","last_clean":"2024-02-18T00:37:49.692431+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:37:49.692431+0000","last_undegraded":"2024-02-18T00:37:49.692431+0000","last_fullsized":"2024-02-18T00:37:49.692431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.601727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.601727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":372,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755747+0000","last_change":"2024-02-18T00:37:47.488889+0000","last_active":"2024-02-18T00:37:48.755747+0000","last_peered":"2024-02-18T00:37:48.755747+0000","last_clean":"2024-02-18T00:37:48.755747+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:37:48.755747+0000","last_undegraded":"2024-02-18T00:37:48.755747+0000","last_fullsized":"2024-02-18T00:37:48.755747+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.488821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.488821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":304,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850438+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:37:48.850438+0000","last_peered":"2024-02-18T00:37:48.850438+0000","last_clean":"2024-02-18T00:37:48.850438+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:37:48.850438+0000","last_undegraded":"2024-02-18T00:37:48.850438+0000","last_fullsized":"2024-02-18T00:37:48.850438+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":437,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726152+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:37:48.726152+0000","last_peered":"2024-02-18T00:37:48.726152+0000","last_clean":"2024-02-18T00:37:48.726152+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:37:48.726152+0000","last_undegraded":"2024-02-18T00:37:48.726152+0000","last_fullsized":"2024-02-18T00:37:48.726152+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":364,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694726+0000","last_change":"2024-02-18T00:36:49.680480+0000","last_active":"2024-02-18T00:37:49.694726+0000","last_peered":"2024-02-18T00:37:49.694726+0000","last_clean":"2024-02-18T00:37:49.694726+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:37:49.694726+0000","last_undegraded":"2024-02-18T00:37:49.694726+0000","last_fullsized":"2024-02-18T00:37:49.694726+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.680414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":360,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691847+0000","last_change":"2024-02-18T00:37:00.670726+0000","last_active":"2024-02-18T00:37:49.691847+0000","last_peered":"2024-02-18T00:37:49.691847+0000","last_clean":"2024-02-18T00:37:49.691847+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:37:49.691847+0000","last_undegraded":"2024-02-18T00:37:49.691847+0000","last_fullsized":"2024-02-18T00:37:49.691847+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.670646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.670646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":264,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691231+0000","last_change":"2024-02-18T00:37:37.476740+0000","last_active":"2024-02-18T00:37:49.691231+0000","last_peered":"2024-02-18T00:37:49.691231+0000","last_clean":"2024-02-18T00:37:49.691231+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:37:49.691231+0000","last_undegraded":"2024-02-18T00:37:49.691231+0000","last_fullsized":"2024-02-18T00:37:49.691231+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.476676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.476676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":294,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723866+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:37:48.723866+0000","last_peered":"2024-02-18T00:37:48.723866+0000","last_clean":"2024-02-18T00:37:48.723866+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:37:48.723866+0000","last_undegraded":"2024-02-18T00:37:48.723866+0000","last_fullsized":"2024-02-18T00:37:48.723866+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":390,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951547+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:37:49.951547+0000","last_peered":"2024-02-18T00:37:49.951547+0000","last_clean":"2024-02-18T00:37:49.951547+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:37:49.951547+0000","last_undegraded":"2024-02-18T00:37:49.951547+0000","last_fullsized":"2024-02-18T00:37:49.951547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":264,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.952213+0000","last_change":"2024-02-18T00:36:58.282770+0000","last_active":"2024-02-18T00:37:49.952213+0000","last_peered":"2024-02-18T00:37:49.952213+0000","last_clean":"2024-02-18T00:37:49.952213+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:37:49.952213+0000","last_undegraded":"2024-02-18T00:37:49.952213+0000","last_fullsized":"2024-02-18T00:37:49.952213+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.282673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.282673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":359,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694241+0000","last_change":"2024-02-18T00:36:59.000753+0000","last_active":"2024-02-18T00:37:49.694241+0000","last_peered":"2024-02-18T00:37:49.694241+0000","last_clean":"2024-02-18T00:37:49.694241+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:37:49.694241+0000","last_undegraded":"2024-02-18T00:37:49.694241+0000","last_fullsized":"2024-02-18T00:37:49.694241+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.000690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.000690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":358,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693282+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:37:49.693282+0000","last_peered":"2024-02-18T00:37:49.693282+0000","last_clean":"2024-02-18T00:37:49.693282+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:37:49.693282+0000","last_undegraded":"2024-02-18T00:37:49.693282+0000","last_fullsized":"2024-02-18T00:37:49.693282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":143,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692382+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:37:49.692382+0000","last_peered":"2024-02-18T00:37:49.692382+0000","last_clean":"2024-02-18T00:37:49.692382+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:37:49.692382+0000","last_undegraded":"2024-02-18T00:37:49.692382+0000","last_fullsized":"2024-02-18T00:37:49.692382+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":377,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.949967+0000","last_change":"2024-02-18T00:37:29.177448+0000","last_active":"2024-02-18T00:37:49.949967+0000","last_peered":"2024-02-18T00:37:49.949967+0000","last_clean":"2024-02-18T00:37:49.949967+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:37:49.949967+0000","last_undegraded":"2024-02-18T00:37:49.949967+0000","last_fullsized":"2024-02-18T00:37:49.949967+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:37:29.177369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.177369+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":372,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693540+0000","last_change":"2024-02-18T00:36:38.883486+0000","last_active":"2024-02-18T00:37:49.693540+0000","last_peered":"2024-02-18T00:37:49.693540+0000","last_clean":"2024-02-18T00:37:49.693540+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:37:49.693540+0000","last_undegraded":"2024-02-18T00:37:49.693540+0000","last_fullsized":"2024-02-18T00:37:49.693540+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:38.883391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.883391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":487,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691635+0000","last_change":"2024-02-18T00:37:38.433946+0000","last_active":"2024-02-18T00:37:49.691635+0000","last_peered":"2024-02-18T00:37:49.691635+0000","last_clean":"2024-02-18T00:37:49.691635+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:37:49.691635+0000","last_undegraded":"2024-02-18T00:37:49.691635+0000","last_fullsized":"2024-02-18T00:37:49.691635+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:38.433877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.433877+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":358,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950994+0000","last_change":"2024-02-18T00:37:31.127432+0000","last_active":"2024-02-18T00:37:49.950994+0000","last_peered":"2024-02-18T00:37:49.950994+0000","last_clean":"2024-02-18T00:37:49.950994+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:37:49.950994+0000","last_undegraded":"2024-02-18T00:37:49.950994+0000","last_fullsized":"2024-02-18T00:37:49.950994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.127341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.127341+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":303,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693818+0000","last_change":"2024-02-18T00:37:00.950276+0000","last_active":"2024-02-18T00:37:49.693818+0000","last_peered":"2024-02-18T00:37:49.693818+0000","last_clean":"2024-02-18T00:37:49.693818+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:37:49.693818+0000","last_undegraded":"2024-02-18T00:37:49.693818+0000","last_fullsized":"2024-02-18T00:37:49.693818+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.950130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.950130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":360,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724817+0000","last_change":"2024-02-18T00:36:33.853442+0000","last_active":"2024-02-18T00:37:48.724817+0000","last_peered":"2024-02-18T00:37:48.724817+0000","last_clean":"2024-02-18T00:37:48.724817+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:37:48.724817+0000","last_undegraded":"2024-02-18T00:37:48.724817+0000","last_fullsized":"2024-02-18T00:37:48.724817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:36:33.853337+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:36:33.853337+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":386,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692443+0000","last_change":"2024-02-18T00:37:41.747387+0000","last_active":"2024-02-18T00:37:49.692443+0000","last_peered":"2024-02-18T00:37:49.692443+0000","last_clean":"2024-02-18T00:37:49.692443+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:37:49.692443+0000","last_undegraded":"2024-02-18T00:37:49.692443+0000","last_fullsized":"2024-02-18T00:37:49.692443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.747293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.747293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":471,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950872+0000","last_change":"2024-02-18T00:37:43.161256+0000","last_active":"2024-02-18T00:37:49.950872+0000","last_peered":"2024-02-18T00:37:49.950872+0000","last_clean":"2024-02-18T00:37:49.950872+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:37:49.950872+0000","last_undegraded":"2024-02-18T00:37:49.950872+0000","last_fullsized":"2024-02-18T00:37:49.950872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:43.161186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.161186+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":348,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724598+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:37:48.724598+0000","last_peered":"2024-02-18T00:37:48.724598+0000","last_clean":"2024-02-18T00:37:48.724598+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:37:48.724598+0000","last_undegraded":"2024-02-18T00:37:48.724598+0000","last_fullsized":"2024-02-18T00:37:48.724598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":299,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950574+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:37:49.950574+0000","last_peered":"2024-02-18T00:37:49.950574+0000","last_clean":"2024-02-18T00:37:49.950574+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:37:49.950574+0000","last_undegraded":"2024-02-18T00:37:49.950574+0000","last_fullsized":"2024-02-18T00:37:49.950574+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":419,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950594+0000","last_change":"2024-02-18T00:36:43.283359+0000","last_active":"2024-02-18T00:37:49.950594+0000","last_peered":"2024-02-18T00:37:49.950594+0000","last_clean":"2024-02-18T00:37:49.950594+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:37:49.950594+0000","last_undegraded":"2024-02-18T00:37:49.950594+0000","last_fullsized":"2024-02-18T00:37:49.950594+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:43.283235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.283235+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":332,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.754211+0000","last_change":"2024-02-18T00:36:57.321994+0000","last_active":"2024-02-18T00:37:48.754211+0000","last_peered":"2024-02-18T00:37:48.754211+0000","last_clean":"2024-02-18T00:37:48.754211+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:37:48.754211+0000","last_undegraded":"2024-02-18T00:37:48.754211+0000","last_fullsized":"2024-02-18T00:37:48.754211+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.321930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.321930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":355,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725015+0000","last_change":"2024-02-18T00:37:48.527852+0000","last_active":"2024-02-18T00:37:48.725015+0000","last_peered":"2024-02-18T00:37:48.725015+0000","last_clean":"2024-02-18T00:37:48.725015+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:37:48.725015+0000","last_undegraded":"2024-02-18T00:37:48.725015+0000","last_fullsized":"2024-02-18T00:37:48.725015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:48.527779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.527779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":354,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850208+0000","last_change":"2024-02-18T00:37:30.849071+0000","last_active":"2024-02-18T00:37:48.850208+0000","last_peered":"2024-02-18T00:37:48.850208+0000","last_clean":"2024-02-18T00:37:48.850208+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:37:48.850208+0000","last_undegraded":"2024-02-18T00:37:48.850208+0000","last_fullsized":"2024-02-18T00:37:48.850208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:30.848966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:30.848966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":304,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950356+0000","last_change":"2024-02-18T00:37:44.200088+0000","last_active":"2024-02-18T00:37:49.950356+0000","last_peered":"2024-02-18T00:37:49.950356+0000","last_clean":"2024-02-18T00:37:49.950356+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:37:49.950356+0000","last_undegraded":"2024-02-18T00:37:49.950356+0000","last_fullsized":"2024-02-18T00:37:49.950356+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.199947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":468,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848586+0000","last_change":"2024-02-18T00:36:53.910404+0000","last_active":"2024-02-18T00:37:48.848586+0000","last_peered":"2024-02-18T00:37:48.848586+0000","last_clean":"2024-02-18T00:37:48.848586+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:37:48.848586+0000","last_undegraded":"2024-02-18T00:37:48.848586+0000","last_fullsized":"2024-02-18T00:37:48.848586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.910333+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":398,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951627+0000","last_change":"2024-02-18T00:36:50.262681+0000","last_active":"2024-02-18T00:37:49.951627+0000","last_peered":"2024-02-18T00:37:49.951627+0000","last_clean":"2024-02-18T00:37:49.951627+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:37:49.951627+0000","last_undegraded":"2024-02-18T00:37:49.951627+0000","last_fullsized":"2024-02-18T00:37:49.951627+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:36:50.262602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.262602+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":379,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726715+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:37:48.726715+0000","last_peered":"2024-02-18T00:37:48.726715+0000","last_clean":"2024-02-18T00:37:48.726715+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:37:48.726715+0000","last_undegraded":"2024-02-18T00:37:48.726715+0000","last_fullsized":"2024-02-18T00:37:48.726715+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":261,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725146+0000","last_change":"2024-02-18T00:37:27.733729+0000","last_active":"2024-02-18T00:37:48.725146+0000","last_peered":"2024-02-18T00:37:48.725146+0000","last_clean":"2024-02-18T00:37:48.725146+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:37:48.725146+0000","last_undegraded":"2024-02-18T00:37:48.725146+0000","last_fullsized":"2024-02-18T00:37:48.725146+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.733609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.733609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":294,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724272+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:37:48.724272+0000","last_peered":"2024-02-18T00:37:48.724272+0000","last_clean":"2024-02-18T00:37:48.724272+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:37:48.724272+0000","last_undegraded":"2024-02-18T00:37:48.724272+0000","last_fullsized":"2024-02-18T00:37:48.724272+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":369,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951858+0000","last_change":"2024-02-18T00:36:45.308400+0000","last_active":"2024-02-18T00:37:49.951858+0000","last_peered":"2024-02-18T00:37:49.951858+0000","last_clean":"2024-02-18T00:37:49.951858+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:37:49.951858+0000","last_undegraded":"2024-02-18T00:37:49.951858+0000","last_fullsized":"2024-02-18T00:37:49.951858+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.308300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.308300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":461,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848566+0000","last_change":"2024-02-18T00:37:35.841167+0000","last_active":"2024-02-18T00:37:48.848566+0000","last_peered":"2024-02-18T00:37:48.848566+0000","last_clean":"2024-02-18T00:37:48.848566+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:37:48.848566+0000","last_undegraded":"2024-02-18T00:37:48.848566+0000","last_fullsized":"2024-02-18T00:37:48.848566+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:37:35.841050+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.841050+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":377,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694890+0000","last_change":"2024-02-18T00:37:32.766303+0000","last_active":"2024-02-18T00:37:49.694890+0000","last_peered":"2024-02-18T00:37:49.694890+0000","last_clean":"2024-02-18T00:37:49.694890+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:37:49.694890+0000","last_undegraded":"2024-02-18T00:37:49.694890+0000","last_fullsized":"2024-02-18T00:37:49.694890+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.766203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.766203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":358,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951894+0000","last_change":"2024-02-18T00:37:33.117248+0000","last_active":"2024-02-18T00:37:49.951894+0000","last_peered":"2024-02-18T00:37:49.951894+0000","last_clean":"2024-02-18T00:37:49.951894+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:37:49.951894+0000","last_undegraded":"2024-02-18T00:37:49.951894+0000","last_fullsized":"2024-02-18T00:37:49.951894+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.117173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.117173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":295,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850253+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:37:48.850253+0000","last_peered":"2024-02-18T00:37:48.850253+0000","last_clean":"2024-02-18T00:37:48.850253+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:37:48.850253+0000","last_undegraded":"2024-02-18T00:37:48.850253+0000","last_fullsized":"2024-02-18T00:37:48.850253+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":357,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726007+0000","last_change":"2024-02-18T00:37:31.672777+0000","last_active":"2024-02-18T00:37:48.726007+0000","last_peered":"2024-02-18T00:37:48.726007+0000","last_clean":"2024-02-18T00:37:48.726007+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:37:48.726007+0000","last_undegraded":"2024-02-18T00:37:48.726007+0000","last_fullsized":"2024-02-18T00:37:48.726007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.672699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.672699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":438,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.756507+0000","last_change":"2024-02-18T00:37:37.281742+0000","last_active":"2024-02-18T00:37:48.756507+0000","last_peered":"2024-02-18T00:37:48.756507+0000","last_clean":"2024-02-18T00:37:48.756507+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:37:48.756507+0000","last_undegraded":"2024-02-18T00:37:48.756507+0000","last_fullsized":"2024-02-18T00:37:48.756507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:37:37.281674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.281674+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":269,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.754333+0000","last_change":"2024-02-18T00:37:01.250860+0000","last_active":"2024-02-18T00:37:48.754333+0000","last_peered":"2024-02-18T00:37:48.754333+0000","last_clean":"2024-02-18T00:37:48.754333+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:37:48.754333+0000","last_undegraded":"2024-02-18T00:37:48.754333+0000","last_fullsized":"2024-02-18T00:37:48.754333+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:37:01.250771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.250771+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":357,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692528+0000","last_change":"2024-02-18T00:36:59.979666+0000","last_active":"2024-02-18T00:37:49.692528+0000","last_peered":"2024-02-18T00:37:49.692528+0000","last_clean":"2024-02-18T00:37:49.692528+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:37:49.692528+0000","last_undegraded":"2024-02-18T00:37:49.692528+0000","last_fullsized":"2024-02-18T00:37:49.692528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.979565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.979565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":295,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849737+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:37:48.849737+0000","last_peered":"2024-02-18T00:37:48.849737+0000","last_clean":"2024-02-18T00:37:48.849737+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:37:48.849737+0000","last_undegraded":"2024-02-18T00:37:48.849737+0000","last_fullsized":"2024-02-18T00:37:48.849737+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":355,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725088+0000","last_change":"2024-02-18T00:37:47.574086+0000","last_active":"2024-02-18T00:37:48.725088+0000","last_peered":"2024-02-18T00:37:48.725088+0000","last_clean":"2024-02-18T00:37:48.725088+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:37:48.725088+0000","last_undegraded":"2024-02-18T00:37:48.725088+0000","last_fullsized":"2024-02-18T00:37:48.725088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.574003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.574003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":376,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.759286+0000","last_change":"2024-02-18T00:37:35.212798+0000","last_active":"2024-02-18T00:37:48.759286+0000","last_peered":"2024-02-18T00:37:48.759286+0000","last_clean":"2024-02-18T00:37:48.759286+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:37:48.759286+0000","last_undegraded":"2024-02-18T00:37:48.759286+0000","last_fullsized":"2024-02-18T00:37:48.759286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:37:35.212727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.212727+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":334,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.695818+0000","last_change":"2024-02-18T00:36:34.800419+0000","last_active":"2024-02-18T00:37:49.695818+0000","last_peered":"2024-02-18T00:37:49.695818+0000","last_clean":"2024-02-18T00:37:49.695818+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:37:49.695818+0000","last_undegraded":"2024-02-18T00:37:49.695818+0000","last_fullsized":"2024-02-18T00:37:49.695818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:34.800327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.800327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":359,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755465+0000","last_change":"2024-02-18T00:37:26.204990+0000","last_active":"2024-02-18T00:37:48.755465+0000","last_peered":"2024-02-18T00:37:48.755465+0000","last_clean":"2024-02-18T00:37:48.755465+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:37:48.755465+0000","last_undegraded":"2024-02-18T00:37:48.755465+0000","last_fullsized":"2024-02-18T00:37:48.755465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:26.204918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:37:26.204918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":357,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694330+0000","last_change":"2024-02-18T00:37:25.868179+0000","last_active":"2024-02-18T00:37:49.694330+0000","last_peered":"2024-02-18T00:37:49.694330+0000","last_clean":"2024-02-18T00:37:49.694330+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:37:49.694330+0000","last_undegraded":"2024-02-18T00:37:49.694330+0000","last_fullsized":"2024-02-18T00:37:49.694330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:25.868065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:25.868065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":305,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.754129+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:37:48.754129+0000","last_peered":"2024-02-18T00:37:48.754129+0000","last_clean":"2024-02-18T00:37:48.754129+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:37:48.754129+0000","last_undegraded":"2024-02-18T00:37:48.754129+0000","last_fullsized":"2024-02-18T00:37:48.754129+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":264,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694523+0000","last_change":"2024-02-18T00:37:32.401689+0000","last_active":"2024-02-18T00:37:49.694523+0000","last_peered":"2024-02-18T00:37:49.694523+0000","last_clean":"2024-02-18T00:37:49.694523+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:37:49.694523+0000","last_undegraded":"2024-02-18T00:37:49.694523+0000","last_fullsized":"2024-02-18T00:37:49.694523+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.401618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.401618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":411,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724622+0000","last_change":"2024-02-18T00:36:50.759133+0000","last_active":"2024-02-18T00:37:48.724622+0000","last_peered":"2024-02-18T00:37:48.724622+0000","last_clean":"2024-02-18T00:37:48.724622+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:37:48.724622+0000","last_undegraded":"2024-02-18T00:37:48.724622+0000","last_fullsized":"2024-02-18T00:37:48.724622+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:36:50.759019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.759019+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":295,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.754087+0000","last_change":"2024-02-18T00:37:42.354830+0000","last_active":"2024-02-18T00:37:48.754087+0000","last_peered":"2024-02-18T00:37:48.754087+0000","last_clean":"2024-02-18T00:37:48.754087+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:37:48.754087+0000","last_undegraded":"2024-02-18T00:37:48.754087+0000","last_fullsized":"2024-02-18T00:37:48.754087+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.354758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.354758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":377,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724570+0000","last_change":"2024-02-18T00:37:00.820469+0000","last_active":"2024-02-18T00:37:48.724570+0000","last_peered":"2024-02-18T00:37:48.724570+0000","last_clean":"2024-02-18T00:37:48.724570+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:37:48.724570+0000","last_undegraded":"2024-02-18T00:37:48.724570+0000","last_fullsized":"2024-02-18T00:37:48.724570+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.820283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.820283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":354,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848773+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:37:48.848773+0000","last_peered":"2024-02-18T00:37:48.848773+0000","last_clean":"2024-02-18T00:37:48.848773+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:37:48.848773+0000","last_undegraded":"2024-02-18T00:37:48.848773+0000","last_fullsized":"2024-02-18T00:37:48.848773+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":353,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725134+0000","last_change":"2024-02-18T00:36:42.828611+0000","last_active":"2024-02-18T00:37:48.725134+0000","last_peered":"2024-02-18T00:37:48.725134+0000","last_clean":"2024-02-18T00:37:48.725134+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:37:48.725134+0000","last_undegraded":"2024-02-18T00:37:48.725134+0000","last_fullsized":"2024-02-18T00:37:48.725134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.828542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.828542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":296,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.756305+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:37:48.756305+0000","last_peered":"2024-02-18T00:37:48.756305+0000","last_clean":"2024-02-18T00:37:48.756305+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:37:48.756305+0000","last_undegraded":"2024-02-18T00:37:48.756305+0000","last_fullsized":"2024-02-18T00:37:48.756305+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":369,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694949+0000","last_change":"2024-02-18T00:36:57.633211+0000","last_active":"2024-02-18T00:37:49.694949+0000","last_peered":"2024-02-18T00:37:49.694949+0000","last_clean":"2024-02-18T00:37:49.694949+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:37:49.694949+0000","last_undegraded":"2024-02-18T00:37:49.694949+0000","last_fullsized":"2024-02-18T00:37:49.694949+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.633128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.633128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":403,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.695390+0000","last_change":"2024-02-18T00:36:45.862082+0000","last_active":"2024-02-18T00:37:49.695390+0000","last_peered":"2024-02-18T00:37:49.695390+0000","last_clean":"2024-02-18T00:37:49.695390+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:37:49.695390+0000","last_undegraded":"2024-02-18T00:37:49.695390+0000","last_fullsized":"2024-02-18T00:37:49.695390+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:36:45.861971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.861971+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":373,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692821+0000","last_change":"2024-02-18T00:36:36.500976+0000","last_active":"2024-02-18T00:37:49.692821+0000","last_peered":"2024-02-18T00:37:49.692821+0000","last_clean":"2024-02-18T00:37:49.692821+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:37:49.692821+0000","last_undegraded":"2024-02-18T00:37:49.692821+0000","last_fullsized":"2024-02-18T00:37:49.692821+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:36.500906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.500906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":877,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.833523+0000","last_change":"2024-02-18T00:37:37.774470+0000","last_active":"2024-02-18T00:37:49.833523+0000","last_peered":"2024-02-18T00:37:49.833523+0000","last_clean":"2024-02-18T00:37:49.833523+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:37:49.833523+0000","last_undegraded":"2024-02-18T00:37:49.833523+0000","last_fullsized":"2024-02-18T00:37:49.833523+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:37:37.774404+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.774404+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":501,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":298,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724879+0000","last_change":"2024-02-18T00:36:42.643527+0000","last_active":"2024-02-18T00:37:48.724879+0000","last_peered":"2024-02-18T00:37:48.724879+0000","last_clean":"2024-02-18T00:37:48.724879+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:37:48.724879+0000","last_undegraded":"2024-02-18T00:37:48.724879+0000","last_fullsized":"2024-02-18T00:37:48.724879+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:36:42.643433+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.643433+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":308,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951987+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:37:49.951987+0000","last_peered":"2024-02-18T00:37:49.951987+0000","last_clean":"2024-02-18T00:37:49.951987+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:37:49.951987+0000","last_undegraded":"2024-02-18T00:37:49.951987+0000","last_fullsized":"2024-02-18T00:37:49.951987+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":356,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.758029+0000","last_change":"2024-02-18T00:36:56.302683+0000","last_active":"2024-02-18T00:37:48.758029+0000","last_peered":"2024-02-18T00:37:48.758029+0000","last_clean":"2024-02-18T00:37:48.758029+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:37:48.758029+0000","last_undegraded":"2024-02-18T00:37:48.758029+0000","last_fullsized":"2024-02-18T00:37:48.758029+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.302615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.302615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":188,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722775+0000","last_change":"2024-02-18T00:36:41.801219+0000","last_active":"2024-02-18T00:37:48.722775+0000","last_peered":"2024-02-18T00:37:48.722775+0000","last_clean":"2024-02-18T00:37:48.722775+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:37:48.722775+0000","last_undegraded":"2024-02-18T00:37:48.722775+0000","last_fullsized":"2024-02-18T00:37:48.722775+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-18T00:36:41.801144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.801144+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":368,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724156+0000","last_change":"2024-02-18T00:36:57.791451+0000","last_active":"2024-02-18T00:37:48.724156+0000","last_peered":"2024-02-18T00:37:48.724156+0000","last_clean":"2024-02-18T00:37:48.724156+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:37:48.724156+0000","last_undegraded":"2024-02-18T00:37:48.724156+0000","last_fullsized":"2024-02-18T00:37:48.724156+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.791347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.791347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":358,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849970+0000","last_change":"2024-02-18T00:36:49.976027+0000","last_active":"2024-02-18T00:37:48.849970+0000","last_peered":"2024-02-18T00:37:48.849970+0000","last_clean":"2024-02-18T00:37:48.849970+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:37:48.849970+0000","last_undegraded":"2024-02-18T00:37:48.849970+0000","last_fullsized":"2024-02-18T00:37:48.849970+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.975957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.975957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":296,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.760093+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:37:48.760093+0000","last_peered":"2024-02-18T00:37:48.760093+0000","last_clean":"2024-02-18T00:37:48.760093+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:37:48.760093+0000","last_undegraded":"2024-02-18T00:37:48.760093+0000","last_fullsized":"2024-02-18T00:37:48.760093+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":467,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.757374+0000","last_change":"2024-02-18T00:37:29.141966+0000","last_active":"2024-02-18T00:37:48.757374+0000","last_peered":"2024-02-18T00:37:48.757374+0000","last_clean":"2024-02-18T00:37:48.757374+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:37:48.757374+0000","last_undegraded":"2024-02-18T00:37:48.757374+0000","last_fullsized":"2024-02-18T00:37:48.757374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:29.141901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.141901+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":362,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692642+0000","last_change":"2024-02-18T00:37:43.309558+0000","last_active":"2024-02-18T00:37:49.692642+0000","last_peered":"2024-02-18T00:37:49.692642+0000","last_clean":"2024-02-18T00:37:49.692642+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:37:49.692642+0000","last_undegraded":"2024-02-18T00:37:49.692642+0000","last_fullsized":"2024-02-18T00:37:49.692642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:37:43.309488+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.309488+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":386,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694070+0000","last_change":"2024-02-18T00:36:53.925296+0000","last_active":"2024-02-18T00:37:49.694070+0000","last_peered":"2024-02-18T00:37:49.694070+0000","last_clean":"2024-02-18T00:37:49.694070+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:37:49.694070+0000","last_undegraded":"2024-02-18T00:37:49.694070+0000","last_fullsized":"2024-02-18T00:37:49.694070+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.925187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.925187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":349,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755286+0000","last_change":"2024-02-18T00:36:46.369602+0000","last_active":"2024-02-18T00:37:48.755286+0000","last_peered":"2024-02-18T00:37:48.755286+0000","last_clean":"2024-02-18T00:37:48.755286+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:37:48.755286+0000","last_undegraded":"2024-02-18T00:37:48.755286+0000","last_fullsized":"2024-02-18T00:37:48.755286+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:46.369503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.369503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":299,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950262+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:37:49.950262+0000","last_peered":"2024-02-18T00:37:49.950262+0000","last_clean":"2024-02-18T00:37:49.950262+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:37:49.950262+0000","last_undegraded":"2024-02-18T00:37:49.950262+0000","last_fullsized":"2024-02-18T00:37:49.950262+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":360,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691122+0000","last_change":"2024-02-18T00:37:44.335458+0000","last_active":"2024-02-18T00:37:49.691122+0000","last_peered":"2024-02-18T00:37:49.691122+0000","last_clean":"2024-02-18T00:37:49.691122+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:37:49.691122+0000","last_undegraded":"2024-02-18T00:37:49.691122+0000","last_fullsized":"2024-02-18T00:37:49.691122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":415,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951718+0000","last_change":"2024-02-18T00:37:39.126531+0000","last_active":"2024-02-18T00:37:49.951718+0000","last_peered":"2024-02-18T00:37:49.951718+0000","last_clean":"2024-02-18T00:37:49.951718+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:37:49.951718+0000","last_undegraded":"2024-02-18T00:37:49.951718+0000","last_fullsized":"2024-02-18T00:37:49.951718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:37:39.126398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.126398+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":397,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951753+0000","last_change":"2024-02-18T00:37:45.186258+0000","last_active":"2024-02-18T00:37:49.951753+0000","last_peered":"2024-02-18T00:37:49.951753+0000","last_clean":"2024-02-18T00:37:49.951753+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:37:49.951753+0000","last_undegraded":"2024-02-18T00:37:49.951753+0000","last_fullsized":"2024-02-18T00:37:49.951753+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:37:45.186143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.186143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":363,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.847397+0000","last_change":"2024-02-18T00:37:00.804782+0000","last_active":"2024-02-18T00:37:48.847397+0000","last_peered":"2024-02-18T00:37:48.847397+0000","last_clean":"2024-02-18T00:37:48.847397+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:37:48.847397+0000","last_undegraded":"2024-02-18T00:37:48.847397+0000","last_fullsized":"2024-02-18T00:37:48.847397+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.804656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":299,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951240+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:37:49.951240+0000","last_peered":"2024-02-18T00:37:49.951240+0000","last_clean":"2024-02-18T00:37:49.951240+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:37:49.951240+0000","last_undegraded":"2024-02-18T00:37:49.951240+0000","last_fullsized":"2024-02-18T00:37:49.951240+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":427,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.759320+0000","last_change":"2024-02-18T00:37:46.472303+0000","last_active":"2024-02-18T00:37:48.759320+0000","last_peered":"2024-02-18T00:37:48.759320+0000","last_clean":"2024-02-18T00:37:48.759320+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:37:48.759320+0000","last_undegraded":"2024-02-18T00:37:48.759320+0000","last_fullsized":"2024-02-18T00:37:48.759320+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:37:46.472227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.472227+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":364,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.691095+0000","last_change":"2024-02-18T00:37:27.357689+0000","last_active":"2024-02-18T00:37:49.691095+0000","last_peered":"2024-02-18T00:37:49.691095+0000","last_clean":"2024-02-18T00:37:49.691095+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:37:49.691095+0000","last_undegraded":"2024-02-18T00:37:49.691095+0000","last_fullsized":"2024-02-18T00:37:49.691095+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.357610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.357610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":365,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850295+0000","last_change":"2024-02-18T00:36:55.861115+0000","last_active":"2024-02-18T00:37:48.850295+0000","last_peered":"2024-02-18T00:37:48.850295+0000","last_clean":"2024-02-18T00:37:48.850295+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:37:48.850295+0000","last_undegraded":"2024-02-18T00:37:48.850295+0000","last_fullsized":"2024-02-18T00:37:48.850295+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:55.861044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.861044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":362,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694445+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:37:49.694445+0000","last_peered":"2024-02-18T00:37:49.694445+0000","last_clean":"2024-02-18T00:37:49.694445+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:37:49.694445+0000","last_undegraded":"2024-02-18T00:37:49.694445+0000","last_fullsized":"2024-02-18T00:37:49.694445+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":296,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.756222+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:37:48.756222+0000","last_peered":"2024-02-18T00:37:48.756222+0000","last_clean":"2024-02-18T00:37:48.756222+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:37:48.756222+0000","last_undegraded":"2024-02-18T00:37:48.756222+0000","last_fullsized":"2024-02-18T00:37:48.756222+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":420,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849882+0000","last_change":"2024-02-18T00:36:51.948976+0000","last_active":"2024-02-18T00:37:48.849882+0000","last_peered":"2024-02-18T00:37:48.849882+0000","last_clean":"2024-02-18T00:37:48.849882+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:37:48.849882+0000","last_undegraded":"2024-02-18T00:37:48.849882+0000","last_fullsized":"2024-02-18T00:37:48.849882+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:36:51.948907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.948907+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":361,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722517+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:37:48.722517+0000","last_peered":"2024-02-18T00:37:48.722517+0000","last_clean":"2024-02-18T00:37:48.722517+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:37:48.722517+0000","last_undegraded":"2024-02-18T00:37:48.722517+0000","last_fullsized":"2024-02-18T00:37:48.722517+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":355,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722929+0000","last_change":"2024-02-18T00:37:38.687198+0000","last_active":"2024-02-18T00:37:48.722929+0000","last_peered":"2024-02-18T00:37:48.722929+0000","last_clean":"2024-02-18T00:37:48.722929+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:37:48.722929+0000","last_undegraded":"2024-02-18T00:37:48.722929+0000","last_fullsized":"2024-02-18T00:37:48.722929+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:38.687129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.687129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":261,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724964+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:37:48.724964+0000","last_peered":"2024-02-18T00:37:48.724964+0000","last_clean":"2024-02-18T00:37:48.724964+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:37:48.724964+0000","last_undegraded":"2024-02-18T00:37:48.724964+0000","last_fullsized":"2024-02-18T00:37:48.724964+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":308,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693017+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:37:49.693017+0000","last_peered":"2024-02-18T00:37:49.693017+0000","last_clean":"2024-02-18T00:37:49.693017+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:37:49.693017+0000","last_undegraded":"2024-02-18T00:37:49.693017+0000","last_fullsized":"2024-02-18T00:37:49.693017+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":428,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850321+0000","last_change":"2024-02-18T00:36:58.819567+0000","last_active":"2024-02-18T00:37:48.850321+0000","last_peered":"2024-02-18T00:37:48.850321+0000","last_clean":"2024-02-18T00:37:48.850321+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:37:48.850321+0000","last_undegraded":"2024-02-18T00:37:48.850321+0000","last_fullsized":"2024-02-18T00:37:48.850321+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:36:58.819503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.819503+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":377,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724079+0000","last_change":"2024-02-18T00:37:41.637636+0000","last_active":"2024-02-18T00:37:48.724079+0000","last_peered":"2024-02-18T00:37:48.724079+0000","last_clean":"2024-02-18T00:37:48.724079+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:37:48.724079+0000","last_undegraded":"2024-02-18T00:37:48.724079+0000","last_fullsized":"2024-02-18T00:37:48.724079+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.637550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.637550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":466,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724453+0000","last_change":"2024-02-18T00:37:27.642761+0000","last_active":"2024-02-18T00:37:48.724453+0000","last_peered":"2024-02-18T00:37:48.724453+0000","last_clean":"2024-02-18T00:37:48.724453+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:37:48.724453+0000","last_undegraded":"2024-02-18T00:37:48.724453+0000","last_fullsized":"2024-02-18T00:37:48.724453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:27.642661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.642661+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":363,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.757623+0000","last_change":"2024-02-18T00:36:44.367116+0000","last_active":"2024-02-18T00:37:48.757623+0000","last_peered":"2024-02-18T00:37:48.757623+0000","last_clean":"2024-02-18T00:37:48.757623+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:37:48.757623+0000","last_undegraded":"2024-02-18T00:37:48.757623+0000","last_fullsized":"2024-02-18T00:37:48.757623+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.367032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.367032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":310,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723079+0000","last_change":"2024-02-18T00:37:46.639988+0000","last_active":"2024-02-18T00:37:48.723079+0000","last_peered":"2024-02-18T00:37:48.723079+0000","last_clean":"2024-02-18T00:37:48.723079+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:37:48.723079+0000","last_undegraded":"2024-02-18T00:37:48.723079+0000","last_fullsized":"2024-02-18T00:37:48.723079+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:46.639901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.639901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":404,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724336+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:37:48.724336+0000","last_peered":"2024-02-18T00:37:48.724336+0000","last_clean":"2024-02-18T00:37:48.724336+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:37:48.724336+0000","last_undegraded":"2024-02-18T00:37:48.724336+0000","last_fullsized":"2024-02-18T00:37:48.724336+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":360,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.757268+0000","last_change":"2024-02-18T00:37:03.239555+0000","last_active":"2024-02-18T00:37:48.757268+0000","last_peered":"2024-02-18T00:37:48.757268+0000","last_clean":"2024-02-18T00:37:48.757268+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:37:48.757268+0000","last_undegraded":"2024-02-18T00:37:48.757268+0000","last_fullsized":"2024-02-18T00:37:48.757268+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.239451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.239451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"341'10","reported_seq":481,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850446+0000","last_change":"2024-02-18T00:36:59.830567+0000","last_active":"2024-02-18T00:37:48.850446+0000","last_peered":"2024-02-18T00:37:48.850446+0000","last_clean":"2024-02-18T00:37:48.850446+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:37:48.850446+0000","last_undegraded":"2024-02-18T00:37:48.850446+0000","last_fullsized":"2024-02-18T00:37:48.850446+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'10","last_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_deep_scrub":"341'10","last_deep_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.830497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":353,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.952025+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:37:49.952025+0000","last_peered":"2024-02-18T00:37:49.952025+0000","last_clean":"2024-02-18T00:37:49.952025+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:37:49.952025+0000","last_undegraded":"2024-02-18T00:37:49.952025+0000","last_fullsized":"2024-02-18T00:37:49.952025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":299,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725177+0000","last_change":"2024-02-18T00:37:39.639314+0000","last_active":"2024-02-18T00:37:48.725177+0000","last_peered":"2024-02-18T00:37:48.725177+0000","last_clean":"2024-02-18T00:37:48.725177+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:37:48.725177+0000","last_undegraded":"2024-02-18T00:37:48.725177+0000","last_fullsized":"2024-02-18T00:37:48.725177+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:39.639232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.639232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":377,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693615+0000","last_change":"2024-02-18T00:36:55.970729+0000","last_active":"2024-02-18T00:37:49.693615+0000","last_peered":"2024-02-18T00:37:49.693615+0000","last_clean":"2024-02-18T00:37:49.693615+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:37:49.693615+0000","last_undegraded":"2024-02-18T00:37:49.693615+0000","last_fullsized":"2024-02-18T00:37:49.693615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:36:55.970628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.970628+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":366,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725026+0000","last_change":"2024-02-18T00:36:44.617522+0000","last_active":"2024-02-18T00:37:48.725026+0000","last_peered":"2024-02-18T00:37:48.725026+0000","last_clean":"2024-02-18T00:37:48.725026+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:37:48.725026+0000","last_undegraded":"2024-02-18T00:37:48.725026+0000","last_fullsized":"2024-02-18T00:37:48.725026+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.617443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.617443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":356,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849239+0000","last_change":"2024-02-18T00:36:50.970267+0000","last_active":"2024-02-18T00:37:48.849239+0000","last_peered":"2024-02-18T00:37:48.849239+0000","last_clean":"2024-02-18T00:37:48.849239+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:37:48.849239+0000","last_undegraded":"2024-02-18T00:37:48.849239+0000","last_fullsized":"2024-02-18T00:37:48.849239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:50.970192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.970192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":349,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755205+0000","last_change":"2024-02-18T00:36:11.346267+0000","last_active":"2024-02-18T00:37:48.755205+0000","last_peered":"2024-02-18T00:37:48.755205+0000","last_clean":"2024-02-18T00:37:48.755205+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:37:48.755205+0000","last_undegraded":"2024-02-18T00:37:48.755205+0000","last_fullsized":"2024-02-18T00:37:48.755205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.346197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.346197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":296,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.759184+0000","last_change":"2024-02-18T00:36:42.355194+0000","last_active":"2024-02-18T00:37:48.759184+0000","last_peered":"2024-02-18T00:37:48.759184+0000","last_clean":"2024-02-18T00:37:48.759184+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:37:48.759184+0000","last_undegraded":"2024-02-18T00:37:48.759184+0000","last_fullsized":"2024-02-18T00:37:48.759184+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.355114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.355114+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":381,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.952186+0000","last_change":"2024-02-18T00:36:52.343624+0000","last_active":"2024-02-18T00:37:49.952186+0000","last_peered":"2024-02-18T00:37:49.952186+0000","last_clean":"2024-02-18T00:37:49.952186+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:37:49.952186+0000","last_undegraded":"2024-02-18T00:37:49.952186+0000","last_fullsized":"2024-02-18T00:37:49.952186+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:36:52.343556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.343556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":363,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.695145+0000","last_change":"2024-02-18T00:37:27.830300+0000","last_active":"2024-02-18T00:37:49.695145+0000","last_peered":"2024-02-18T00:37:49.695145+0000","last_clean":"2024-02-18T00:37:49.695145+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:37:49.695145+0000","last_undegraded":"2024-02-18T00:37:49.695145+0000","last_fullsized":"2024-02-18T00:37:49.695145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.830225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.830225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":359,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.693631+0000","last_change":"2024-02-18T00:36:42.857622+0000","last_active":"2024-02-18T00:37:49.693631+0000","last_peered":"2024-02-18T00:37:49.693631+0000","last_clean":"2024-02-18T00:37:49.693631+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:37:49.693631+0000","last_undegraded":"2024-02-18T00:37:49.693631+0000","last_fullsized":"2024-02-18T00:37:49.693631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.857557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.857557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":354,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849092+0000","last_change":"2024-02-18T00:37:37.848598+0000","last_active":"2024-02-18T00:37:48.849092+0000","last_peered":"2024-02-18T00:37:48.849092+0000","last_clean":"2024-02-18T00:37:48.849092+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:37:48.849092+0000","last_undegraded":"2024-02-18T00:37:48.849092+0000","last_fullsized":"2024-02-18T00:37:48.849092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.848524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.848524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":296,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724787+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:37:48.724787+0000","last_peered":"2024-02-18T00:37:48.724787+0000","last_clean":"2024-02-18T00:37:48.724787+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:37:48.724787+0000","last_undegraded":"2024-02-18T00:37:48.724787+0000","last_fullsized":"2024-02-18T00:37:48.724787+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":364,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848467+0000","last_change":"2024-02-18T00:37:41.880926+0000","last_active":"2024-02-18T00:37:48.848467+0000","last_peered":"2024-02-18T00:37:48.848467+0000","last_clean":"2024-02-18T00:37:48.848467+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:37:48.848467+0000","last_undegraded":"2024-02-18T00:37:48.848467+0000","last_fullsized":"2024-02-18T00:37:48.848467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:37:41.880855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.880855+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":359,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723007+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:37:48.723007+0000","last_peered":"2024-02-18T00:37:48.723007+0000","last_clean":"2024-02-18T00:37:48.723007+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:37:48.723007+0000","last_undegraded":"2024-02-18T00:37:48.723007+0000","last_fullsized":"2024-02-18T00:37:48.723007+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":366,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.722443+0000","last_change":"2024-02-18T00:36:12.518914+0000","last_active":"2024-02-18T00:37:48.722443+0000","last_peered":"2024-02-18T00:37:48.722443+0000","last_clean":"2024-02-18T00:37:48.722443+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:37:48.722443+0000","last_undegraded":"2024-02-18T00:37:48.722443+0000","last_fullsized":"2024-02-18T00:37:48.722443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:12.518838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:12.518838+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":264,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692549+0000","last_change":"2024-02-18T00:37:29.374778+0000","last_active":"2024-02-18T00:37:49.692549+0000","last_peered":"2024-02-18T00:37:49.692549+0000","last_clean":"2024-02-18T00:37:49.692549+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:37:49.692549+0000","last_undegraded":"2024-02-18T00:37:49.692549+0000","last_fullsized":"2024-02-18T00:37:49.692549+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:29.374712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.374712+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":303,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723290+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:37:48.723290+0000","last_peered":"2024-02-18T00:37:48.723290+0000","last_clean":"2024-02-18T00:37:48.723290+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:37:48.723290+0000","last_undegraded":"2024-02-18T00:37:48.723290+0000","last_fullsized":"2024-02-18T00:37:48.723290+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":415,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849499+0000","last_change":"2024-02-18T00:37:45.838598+0000","last_active":"2024-02-18T00:37:48.849499+0000","last_peered":"2024-02-18T00:37:48.849499+0000","last_clean":"2024-02-18T00:37:48.849499+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:37:48.849499+0000","last_undegraded":"2024-02-18T00:37:48.849499+0000","last_fullsized":"2024-02-18T00:37:48.849499+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:37:45.838523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.838523+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":361,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726654+0000","last_change":"2024-02-18T00:36:11.551695+0000","last_active":"2024-02-18T00:37:48.726654+0000","last_peered":"2024-02-18T00:37:48.726654+0000","last_clean":"2024-02-18T00:37:48.726654+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:37:48.726654+0000","last_undegraded":"2024-02-18T00:37:48.726654+0000","last_fullsized":"2024-02-18T00:37:48.726654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.551602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":273,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.949132+0000","last_change":"2024-02-18T00:36:59.243785+0000","last_active":"2024-02-18T00:37:49.949132+0000","last_peered":"2024-02-18T00:37:49.949132+0000","last_clean":"2024-02-18T00:37:49.949132+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:37:49.949132+0000","last_undegraded":"2024-02-18T00:37:49.949132+0000","last_fullsized":"2024-02-18T00:37:49.949132+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.243699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.243699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":354,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849390+0000","last_change":"2024-02-18T00:37:44.869935+0000","last_active":"2024-02-18T00:37:48.849390+0000","last_peered":"2024-02-18T00:37:48.849390+0000","last_clean":"2024-02-18T00:37:48.849390+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:37:48.849390+0000","last_undegraded":"2024-02-18T00:37:48.849390+0000","last_fullsized":"2024-02-18T00:37:48.849390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.869862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.869862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":259,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950485+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:37:49.950485+0000","last_peered":"2024-02-18T00:37:49.950485+0000","last_clean":"2024-02-18T00:37:49.950485+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:37:49.950485+0000","last_undegraded":"2024-02-18T00:37:49.950485+0000","last_fullsized":"2024-02-18T00:37:49.950485+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":417,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.759443+0000","last_change":"2024-02-18T00:36:53.401897+0000","last_active":"2024-02-18T00:37:48.759443+0000","last_peered":"2024-02-18T00:37:48.759443+0000","last_clean":"2024-02-18T00:37:48.759443+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:37:48.759443+0000","last_undegraded":"2024-02-18T00:37:48.759443+0000","last_fullsized":"2024-02-18T00:37:48.759443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:36:53.401819+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.401819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":363,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723708+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:37:48.723708+0000","last_peered":"2024-02-18T00:37:48.723708+0000","last_clean":"2024-02-18T00:37:48.723708+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:37:48.723708+0000","last_undegraded":"2024-02-18T00:37:48.723708+0000","last_fullsized":"2024-02-18T00:37:48.723708+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":489,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726709+0000","last_change":"2024-02-18T00:36:38.557419+0000","last_active":"2024-02-18T00:37:48.726709+0000","last_peered":"2024-02-18T00:37:48.726709+0000","last_clean":"2024-02-18T00:37:48.726709+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:37:48.726709+0000","last_undegraded":"2024-02-18T00:37:48.726709+0000","last_fullsized":"2024-02-18T00:37:48.726709+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:36:38.557298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.557298+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":272,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723226+0000","last_change":"2024-02-18T00:36:49.727241+0000","last_active":"2024-02-18T00:37:48.723226+0000","last_peered":"2024-02-18T00:37:48.723226+0000","last_clean":"2024-02-18T00:37:48.723226+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:37:48.723226+0000","last_undegraded":"2024-02-18T00:37:48.723226+0000","last_fullsized":"2024-02-18T00:37:48.723226+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:36:49.727157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.727157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":255,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848666+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:37:48.848666+0000","last_peered":"2024-02-18T00:37:48.848666+0000","last_clean":"2024-02-18T00:37:48.848666+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:37:48.848666+0000","last_undegraded":"2024-02-18T00:37:48.848666+0000","last_fullsized":"2024-02-18T00:37:48.848666+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":393,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.726767+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:37:48.726767+0000","last_peered":"2024-02-18T00:37:48.726767+0000","last_clean":"2024-02-18T00:37:48.726767+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:37:48.726767+0000","last_undegraded":"2024-02-18T00:37:48.726767+0000","last_fullsized":"2024-02-18T00:37:48.726767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":360,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849226+0000","last_change":"2024-02-18T00:36:56.850409+0000","last_active":"2024-02-18T00:37:48.849226+0000","last_peered":"2024-02-18T00:37:48.849226+0000","last_clean":"2024-02-18T00:37:48.849226+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:37:48.849226+0000","last_undegraded":"2024-02-18T00:37:48.849226+0000","last_fullsized":"2024-02-18T00:37:48.849226+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.850326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.850326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":360,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.757512+0000","last_change":"2024-02-18T00:36:52.372043+0000","last_active":"2024-02-18T00:37:48.757512+0000","last_peered":"2024-02-18T00:37:48.757512+0000","last_clean":"2024-02-18T00:37:48.757512+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:37:48.757512+0000","last_undegraded":"2024-02-18T00:37:48.757512+0000","last_fullsized":"2024-02-18T00:37:48.757512+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.371970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.371970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":362,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694838+0000","last_change":"2024-02-18T00:37:40.707426+0000","last_active":"2024-02-18T00:37:49.694838+0000","last_peered":"2024-02-18T00:37:49.694838+0000","last_clean":"2024-02-18T00:37:49.694838+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:37:49.694838+0000","last_undegraded":"2024-02-18T00:37:49.694838+0000","last_fullsized":"2024-02-18T00:37:49.694838+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.707359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.707359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":317,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692941+0000","last_change":"2024-02-18T00:37:48.395278+0000","last_active":"2024-02-18T00:37:49.692941+0000","last_peered":"2024-02-18T00:37:49.692941+0000","last_clean":"2024-02-18T00:37:49.692941+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:37:49.692941+0000","last_undegraded":"2024-02-18T00:37:49.692941+0000","last_fullsized":"2024-02-18T00:37:49.692941+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:37:48.395205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.395205+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":392,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694808+0000","last_change":"2024-02-18T00:37:39.385746+0000","last_active":"2024-02-18T00:37:49.694808+0000","last_peered":"2024-02-18T00:37:49.694808+0000","last_clean":"2024-02-18T00:37:49.694808+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:37:49.694808+0000","last_undegraded":"2024-02-18T00:37:49.694808+0000","last_fullsized":"2024-02-18T00:37:49.694808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:39.385675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.385675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":377,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.725965+0000","last_change":"2024-02-18T00:37:36.615647+0000","last_active":"2024-02-18T00:37:48.725965+0000","last_peered":"2024-02-18T00:37:48.725965+0000","last_clean":"2024-02-18T00:37:48.725965+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:37:48.725965+0000","last_undegraded":"2024-02-18T00:37:48.725965+0000","last_fullsized":"2024-02-18T00:37:48.725965+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.615568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.615568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":359,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.698220+0000","last_change":"2024-02-18T00:36:35.808459+0000","last_active":"2024-02-18T00:37:49.698220+0000","last_peered":"2024-02-18T00:37:49.698220+0000","last_clean":"2024-02-18T00:37:49.698220+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:37:49.698220+0000","last_undegraded":"2024-02-18T00:37:49.698220+0000","last_fullsized":"2024-02-18T00:37:49.698220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.808381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.808381+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":358,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950703+0000","last_change":"2024-02-18T00:37:34.122873+0000","last_active":"2024-02-18T00:37:49.950703+0000","last_peered":"2024-02-18T00:37:49.950703+0000","last_clean":"2024-02-18T00:37:49.950703+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:37:49.950703+0000","last_undegraded":"2024-02-18T00:37:49.950703+0000","last_fullsized":"2024-02-18T00:37:49.950703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.122764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.122764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":301,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723687+0000","last_change":"2024-02-18T00:37:40.562387+0000","last_active":"2024-02-18T00:37:48.723687+0000","last_peered":"2024-02-18T00:37:48.723687+0000","last_clean":"2024-02-18T00:37:48.723687+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:37:48.723687+0000","last_undegraded":"2024-02-18T00:37:48.723687+0000","last_fullsized":"2024-02-18T00:37:48.723687+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.562306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.562306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":396,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849573+0000","last_change":"2024-02-18T00:37:43.883242+0000","last_active":"2024-02-18T00:37:48.849573+0000","last_peered":"2024-02-18T00:37:48.849573+0000","last_clean":"2024-02-18T00:37:48.849573+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:37:48.849573+0000","last_undegraded":"2024-02-18T00:37:48.849573+0000","last_fullsized":"2024-02-18T00:37:48.849573+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:37:43.883172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.883172+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":380,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849135+0000","last_change":"2024-02-18T00:36:52.928195+0000","last_active":"2024-02-18T00:37:48.849135+0000","last_peered":"2024-02-18T00:37:48.849135+0000","last_clean":"2024-02-18T00:37:48.849135+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:37:48.849135+0000","last_undegraded":"2024-02-18T00:37:48.849135+0000","last_fullsized":"2024-02-18T00:37:48.849135+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:36:52.928076+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.928076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":360,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950059+0000","last_change":"2024-02-18T00:37:03.248349+0000","last_active":"2024-02-18T00:37:49.950059+0000","last_peered":"2024-02-18T00:37:49.950059+0000","last_clean":"2024-02-18T00:37:49.950059+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:37:49.950059+0000","last_undegraded":"2024-02-18T00:37:49.950059+0000","last_fullsized":"2024-02-18T00:37:49.950059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.248277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.248277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":349,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849794+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:37:48.849794+0000","last_peered":"2024-02-18T00:37:48.849794+0000","last_clean":"2024-02-18T00:37:48.849794+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:37:48.849794+0000","last_undegraded":"2024-02-18T00:37:48.849794+0000","last_fullsized":"2024-02-18T00:37:48.849794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":305,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.760138+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:37:48.760138+0000","last_peered":"2024-02-18T00:37:48.760138+0000","last_clean":"2024-02-18T00:37:48.760138+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:37:48.760138+0000","last_undegraded":"2024-02-18T00:37:48.760138+0000","last_fullsized":"2024-02-18T00:37:48.760138+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":420,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950963+0000","last_change":"2024-02-18T00:37:48.216997+0000","last_active":"2024-02-18T00:37:49.950963+0000","last_peered":"2024-02-18T00:37:49.950963+0000","last_clean":"2024-02-18T00:37:49.950963+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:37:49.950963+0000","last_undegraded":"2024-02-18T00:37:49.950963+0000","last_fullsized":"2024-02-18T00:37:49.950963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:37:48.216929+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.216929+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":254,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723142+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:37:48.723142+0000","last_peered":"2024-02-18T00:37:48.723142+0000","last_clean":"2024-02-18T00:37:48.723142+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:37:48.723142+0000","last_undegraded":"2024-02-18T00:37:48.723142+0000","last_fullsized":"2024-02-18T00:37:48.723142+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":373,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724512+0000","last_change":"2024-02-18T00:36:45.773797+0000","last_active":"2024-02-18T00:37:48.724512+0000","last_peered":"2024-02-18T00:37:48.724512+0000","last_clean":"2024-02-18T00:37:48.724512+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:37:48.724512+0000","last_undegraded":"2024-02-18T00:37:48.724512+0000","last_fullsized":"2024-02-18T00:37:48.724512+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.773695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.773695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":363,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.755579+0000","last_change":"2024-02-18T00:36:45.412472+0000","last_active":"2024-02-18T00:37:48.755579+0000","last_peered":"2024-02-18T00:37:48.755579+0000","last_clean":"2024-02-18T00:37:48.755579+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:37:48.755579+0000","last_undegraded":"2024-02-18T00:37:48.755579+0000","last_fullsized":"2024-02-18T00:37:48.755579+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.412406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.412406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":299,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.694521+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:37:49.694521+0000","last_peered":"2024-02-18T00:37:49.694521+0000","last_clean":"2024-02-18T00:37:49.694521+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:37:49.694521+0000","last_undegraded":"2024-02-18T00:37:49.694521+0000","last_fullsized":"2024-02-18T00:37:49.694521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":390,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951401+0000","last_change":"2024-02-18T00:36:51.303043+0000","last_active":"2024-02-18T00:37:49.951401+0000","last_peered":"2024-02-18T00:37:49.951401+0000","last_clean":"2024-02-18T00:37:49.951401+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:37:49.951401+0000","last_undegraded":"2024-02-18T00:37:49.951401+0000","last_fullsized":"2024-02-18T00:37:49.951401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:36:51.302920+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.302920+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":261,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.724704+0000","last_change":"2024-02-18T00:36:52.758965+0000","last_active":"2024-02-18T00:37:48.724704+0000","last_peered":"2024-02-18T00:37:48.724704+0000","last_clean":"2024-02-18T00:37:48.724704+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:37:48.724704+0000","last_undegraded":"2024-02-18T00:37:48.724704+0000","last_fullsized":"2024-02-18T00:37:48.724704+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.758904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.758904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":259,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.951918+0000","last_change":"2024-02-18T00:36:49.277109+0000","last_active":"2024-02-18T00:37:49.951918+0000","last_peered":"2024-02-18T00:37:49.951918+0000","last_clean":"2024-02-18T00:37:49.951918+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:37:49.951918+0000","last_undegraded":"2024-02-18T00:37:49.951918+0000","last_fullsized":"2024-02-18T00:37:49.951918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.276979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.276979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":353,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.723796+0000","last_change":"2024-02-18T00:36:40.825174+0000","last_active":"2024-02-18T00:37:48.723796+0000","last_peered":"2024-02-18T00:37:48.723796+0000","last_clean":"2024-02-18T00:37:48.723796+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:37:48.723796+0000","last_undegraded":"2024-02-18T00:37:48.723796+0000","last_fullsized":"2024-02-18T00:37:48.723796+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:40.825055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:36:40.825055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":304,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849742+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:37:48.849742+0000","last_peered":"2024-02-18T00:37:48.849742+0000","last_clean":"2024-02-18T00:37:48.849742+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:37:48.849742+0000","last_undegraded":"2024-02-18T00:37:48.849742+0000","last_fullsized":"2024-02-18T00:37:48.849742+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":393,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.949034+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:37:49.949034+0000","last_peered":"2024-02-18T00:37:49.949034+0000","last_clean":"2024-02-18T00:37:49.949034+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:37:49.949034+0000","last_undegraded":"2024-02-18T00:37:49.949034+0000","last_fullsized":"2024-02-18T00:37:49.949034+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":355,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850140+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:37:48.850140+0000","last_peered":"2024-02-18T00:37:48.850140+0000","last_clean":"2024-02-18T00:37:48.850140+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:37:48.850140+0000","last_undegraded":"2024-02-18T00:37:48.850140+0000","last_fullsized":"2024-02-18T00:37:48.850140+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":368,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.692362+0000","last_change":"2024-02-18T00:36:49.936731+0000","last_active":"2024-02-18T00:37:49.692362+0000","last_peered":"2024-02-18T00:37:49.692362+0000","last_clean":"2024-02-18T00:37:49.692362+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:37:49.692362+0000","last_undegraded":"2024-02-18T00:37:49.692362+0000","last_fullsized":"2024-02-18T00:37:49.692362+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.936644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.936644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":358,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.950842+0000","last_change":"2024-02-18T00:37:42.183653+0000","last_active":"2024-02-18T00:37:49.950842+0000","last_peered":"2024-02-18T00:37:49.950842+0000","last_clean":"2024-02-18T00:37:49.950842+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:37:49.950842+0000","last_undegraded":"2024-02-18T00:37:49.950842+0000","last_fullsized":"2024-02-18T00:37:49.950842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.183568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.183568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":295,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.850684+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:37:48.850684+0000","last_peered":"2024-02-18T00:37:48.850684+0000","last_clean":"2024-02-18T00:37:48.850684+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:37:48.850684+0000","last_undegraded":"2024-02-18T00:37:48.850684+0000","last_fullsized":"2024-02-18T00:37:48.850684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":440,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.849200+0000","last_change":"2024-02-18T00:36:46.942689+0000","last_active":"2024-02-18T00:37:48.849200+0000","last_peered":"2024-02-18T00:37:48.849200+0000","last_clean":"2024-02-18T00:37:48.849200+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:37:48.849200+0000","last_undegraded":"2024-02-18T00:37:48.849200+0000","last_fullsized":"2024-02-18T00:37:48.849200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:36:46.942583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.942583+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":371,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.759918+0000","last_change":"2024-02-18T00:37:45.454134+0000","last_active":"2024-02-18T00:37:48.759918+0000","last_peered":"2024-02-18T00:37:48.759918+0000","last_clean":"2024-02-18T00:37:48.759918+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:37:48.759918+0000","last_undegraded":"2024-02-18T00:37:48.759918+0000","last_fullsized":"2024-02-18T00:37:48.759918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.454069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":360,"reported_epoch":391,"state":"active+clean","last_fresh":"2024-02-18T00:37:49.949365+0000","last_change":"2024-02-18T00:37:36.119369+0000","last_active":"2024-02-18T00:37:49.949365+0000","last_peered":"2024-02-18T00:37:49.949365+0000","last_clean":"2024-02-18T00:37:49.949365+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:37:49.949365+0000","last_undegraded":"2024-02-18T00:37:49.949365+0000","last_fullsized":"2024-02-18T00:37:49.949365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.119291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.119291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":363,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-02-18T00:37:48.848965+0000","last_change":"2024-02-18T00:37:34.860293+0000","last_active":"2024-02-18T00:37:48.848965+0000","last_peered":"2024-02-18T00:37:48.848965+0000","last_clean":"2024-02-18T00:37:48.848965+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:37:48.848965+0000","last_undegraded":"2024-02-18T00:37:48.848965+0000","last_fullsized":"2024-02-18T00:37:48.848965+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.860214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.860214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":204,"num_pg":32,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":2,"num_evict_kb":2,"num_promote":4,"num_flush_mode_high":2,"num_flush_mode_low":0,"num_evict_mode_some":2,"num_evict_mode_full":0,"num_objects_pinned":2,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":48,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001420,"num_objects":148,"num_object_clones":2,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":104,"num_read_kb":7,"num_write":573,"num_write_kb":4474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13819904,"data_stored":12004293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1823,"ondisk_log_size":1823,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":506,"num_read_kb":501,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266795,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463025,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309580,"kb_used_data":8260,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435572,"statfs":{"total":95995035648,"available":95678025728,"internally_reserved":0,"allocated":8458240,"data_stored":6241623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":2.722,"15min":1.2370000000000001},"min":{"1min":0.27600000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.1799999999999999,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.1679999999999999,"15min":0.72299999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1,"5min":10.433,"15min":10.433},"last":0.61299999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":2.9569999999999999,"15min":1.3200000000000001},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.97199999999999998,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.498},{"interface":"front","average":{"1min":0.56399999999999995,"5min":2.9529999999999998,"15min":1.3109999999999999},"min":{"1min":0.34100000000000003,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":1.216,"5min":71.872,"15min":71.872},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":1.395,"15min":0.80300000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.141,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.8560000000000001,"15min":0.96799999999999997},"min":{"1min":0.39500000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.9930000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.64900000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.391,"15min":0.80300000000000005},"min":{"1min":0.29999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2629999999999999,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":1.2929999999999999,"15min":0.77600000000000002},"min":{"1min":0.37,"5min":0.309,"15min":0.309},"max":{"1min":1.0149999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.68500000000000005,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.496,"5min":1.1910000000000001,"15min":0.66900000000000004},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85099999999999998,"5min":35.18,"15min":35.18},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.73199999999999998,"15min":0.52400000000000002},"min":{"1min":0.317,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.673,"5min":10.606,"15min":10.606},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.496,"5min":1.224,"15min":0.69399999999999995},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70899999999999996,"5min":35.119,"15min":35.119},"last":0.44}]},{"osd":7,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63500000000000001,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73299999999999998,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.1220000000000001,"15min":0.72699999999999998},"min":{"1min":0.42799999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.745,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.47699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574984,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305676,"kb_used_data":4356,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439476,"statfs":{"total":95995035648,"available":95682023424,"internally_reserved":0,"allocated":4460544,"data_stored":2249764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:02 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":1.228,"15min":0.63},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.47399999999999998,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":1.3160000000000001,"15min":0.66300000000000003},"min":{"1min":0.253,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56999999999999995,"5min":43.137,"15min":43.137},"last":0.88900000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":1.528,"15min":0.75900000000000001},"min":{"1min":0.22900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.78300000000000003,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.441},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.95699999999999996,"15min":0.58599999999999997},"min":{"1min":0.27000000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.623,"5min":17.596,"15min":17.596},"last":0.83199999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":1.3440000000000001,"15min":0.747},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.92800000000000005,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":1.3400000000000001,"15min":0.76600000000000001},"min":{"1min":0.30199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61499999999999999,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.61499999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":1.1319999999999999,"15min":0.77700000000000002},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73999999999999999,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.874},{"interface":"front","average":{"1min":0.63300000000000001,"5min":1.1719999999999999,"15min":0.77100000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.9790000000000001,"5min":12.75,"15min":12.75},"last":0.81999999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:37:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.127,"15min":0.77600000000000002},"min":{"1min":0.40000000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.63},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.129,"15min":0.78800000000000003},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80200000000000005,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.69499999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:37:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":1.165,"15min":0.81000000000000005},"min":{"1min":0.40500000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":12.834,"15min":12.834},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":1.1399999999999999,"15min":0.80000000000000004},"min":{"1min":0.44700000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83399999999999996,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":1.143,"15min":0.80400000000000005},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":1.123,"15min":0.81699999999999995},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.0859999999999999,"5min":12.805,"15min":12.805},"last":0.76000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738510,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301592,"kb_used_data":4368,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443560,"statfs":{"total":95995035648,"available":95686205440,"internally_reserved":0,"allocated":4472832,"data_stored":2240519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":2.2749999999999999,"15min":1.1180000000000001},"min":{"1min":0.21299999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.2330000000000001,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.378},{"interface":"front","average":{"1min":0.61799999999999999,"5min":1.5589999999999999,"15min":0.75900000000000001},"min":{"1min":0.23000000000000001,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":4.577,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.33900000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.8759999999999999,"15min":0.88900000000000001},"min":{"1min":0.27400000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":4.54,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":1.8380000000000001,"15min":0.871},"min":{"1min":0.25,"5min":0.23100000000000001,"15min":0.224},"max":{"1min":0.94899999999999995,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":1.149,"15min":0.67600000000000005},"min":{"1min":0.255,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.497,"5min":19.006,"15min":19.006},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":2.2610000000000001,"15min":1.0780000000000001},"min":{"1min":0.27200000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":4.4489999999999998,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:37:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.651,"15min":0.91600000000000004},"min":{"1min":0.40200000000000002,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.77500000000000002,"5min":57.914000000000001,"15min":57.914000000000001},"last":0.435},{"interface":"front","average":{"1min":0.78600000000000003,"5min":1.7370000000000001,"15min":0.97999999999999998},"min":{"1min":0.32800000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.2919999999999998,"5min":90.070999999999998,"15min":90.070999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.673,"15min":0.95199999999999996},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.3760000000000003,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.8220000000000001,"15min":0.97499999999999998},"min":{"1min":0.42699999999999999,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":1.5640000000000001,"5min":57.991,"15min":57.991},"last":0.47599999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:36:55 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.2270000000000001,"15min":0.79500000000000004},"min":{"1min":0.441,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76200000000000001,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.746,"15min":0.96799999999999997},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":4.343,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.44400000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.9059999999999999,"15min":1.0449999999999999},"min":{"1min":0.42099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.4100000000000001,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.798,"15min":0.99299999999999999},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.83899999999999997,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.501}]}]},{"osd":2,"up_from":18,"seq":77309411461,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301672,"kb_used_data":4448,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443480,"statfs":{"total":95995035648,"available":95686123520,"internally_reserved":0,"allocated":4554752,"data_stored":2240881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.70699999999999996,"15min":0.48899999999999999},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46100000000000002,"5min":12.144,"15min":12.144},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.056,"15min":0.60699999999999998},"min":{"1min":0.27000000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56200000000000006,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.69699999999999995,"15min":0.495},"min":{"1min":0.28499999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.81399999999999995,"15min":0.54400000000000004},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.64300000000000002,"5min":18.919,"15min":18.919},"last":0.35499999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.89900000000000002,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.5,"5min":22.712,"15min":22.712},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.88500000000000001,"15min":0.61699999999999999},"min":{"1min":0.33500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51100000000000001,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.89000000000000001,"15min":0.68500000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.875,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.503},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.84499999999999997,"15min":0.67600000000000005},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76000000000000001,"5min":10.016,"15min":10.016},"last":0.41299999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.75800000000000001,"15min":0.65000000000000002},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92800000000000005,"5min":6.766,"15min":6.766},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.95999999999999996,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.437}]},{"osd":6,"last update":"Sun Feb 18 00:36:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.79400000000000004,"15min":0.68700000000000006},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":6.806,"15min":6.806},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.89700000000000002,"15min":0.73399999999999999},"min":{"1min":0.40100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.47899999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.80900000000000005,"15min":0.72399999999999998},"min":{"1min":0.44500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.90000000000000002,"15min":0.746},"min":{"1min":0.495,"5min":0.372,"15min":0.372},"max":{"1min":0.96399999999999997,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.48999999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149823,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":326176,"kb_used_data":8344,"kb_used_omap":0,"kb_used_meta":317824,"kb_avail":93418976,"statfs":{"total":95995035648,"available":95661031424,"internally_reserved":0,"allocated":8544256,"data_stored":6232389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":325451776},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.65800000000000003,"15min":0.47699999999999998},"min":{"1min":0.254,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.82499999999999996,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.249},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.76600000000000001,"15min":0.52600000000000002},"min":{"1min":0.27900000000000003,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.2110000000000001,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.59999999999999998,"15min":0.47699999999999998},"min":{"1min":0.27300000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.59299999999999997,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.274,"15min":0.69599999999999995},"min":{"1min":0.26100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":8.0779999999999994,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.28000000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":1.107,"15min":0.66000000000000003},"min":{"1min":0.30299999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.0190000000000001,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.315},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.54600000000000004,"15min":0.46899999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":1.4430000000000001,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.91800000000000004,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.8410000000000002,"5min":10.131,"15min":10.131},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.70399999999999996,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.86499999999999999,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.42399999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:37:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.69499999999999995,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.81000000000000005,"15min":0.67800000000000005},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78800000000000003,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.53500000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:36:54 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":1.05,"15min":0.749},"min":{"1min":0.38800000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.8940000000000001,"5min":10.189,"15min":10.189},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.74099999999999999,"15min":0.66400000000000003},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92700000000000005,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.46000000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:37:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.80000000000000004,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89300000000000002,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.96199999999999997,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.39800000000000002},"max":{"1min":3.9289999999999998,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.437}]}]},{"osd":4,"up_from":31,"seq":133143986298,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309792,"kb_used_data":4376,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435360,"statfs":{"total":95995035648,"available":95677808640,"internally_reserved":0,"allocated":4481024,"data_stored":2241774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":1.542,"15min":0.91100000000000003},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.92600000000000005,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.5},{"interface":"front","average":{"1min":0.498,"5min":1.4950000000000001,"15min":0.872},"min":{"1min":0.28199999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.40300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.80900000000000005,"15min":0.64500000000000002},"min":{"1min":0.33900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89800000000000002,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":1.581,"15min":0.90100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.88,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.41099999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.78000000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.022,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.90000000000000002,"15min":0.68100000000000005},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80200000000000005,"5min":17.77,"15min":17.77},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.68999999999999995,"15min":0.627},"min":{"1min":0.318,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.82199999999999995,"5min":7.218,"15min":7.218},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":1.091,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.309,"15min":0.309},"max":{"1min":1.2350000000000001,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.47399999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.435,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.372}]},{"osd":6,"last update":"Sun Feb 18 00:37:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45600000000000002,"15min":0.41899999999999998},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.30099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65700000000000003,"5min":0.751,"15min":0.751},"last":0.41899999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.76900000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.34899999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73499999999999999,"5min":0.79700000000000004,"15min":0.83299999999999996},"last":0.46300000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724661,"num_pgs":102,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309500,"kb_used_data":8180,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435652,"statfs":{"total":95995035648,"available":95678107648,"internally_reserved":0,"allocated":8376320,"data_stored":6241610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.55100000000000005},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":1.4630000000000001,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.624},"min":{"1min":0.29099999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":4.4779999999999998,"5min":12.744,"15min":12.744},"last":0.69899999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.71599999999999997,"15min":0.624},"min":{"1min":0.33800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.151,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.77100000000000002,"15min":0.63100000000000001},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":1.127,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.54900000000000004}]},{"osd":2,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67000000000000004,"15min":0.61399999999999999},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.77000000000000002,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.72899999999999998,"15min":0.64200000000000002},"min":{"1min":0.41999999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.1000000000000001,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.66300000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.73599999999999999,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.501},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83099999999999996,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.628}]},{"osd":4,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.58999999999999997,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.67200000000000004,"5min":11.818,"15min":11.818},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50600000000000001,"15min":0.433},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72999999999999998,"5min":4.274,"15min":4.274},"last":0.35999999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:36:50 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73499999999999999,"15min":0.52000000000000002},"min":{"1min":0.34100000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":12.362,"5min":18.605,"15min":18.605},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52800000000000002,"15min":0.45400000000000001},"min":{"1min":0.309,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.46,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.40699999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.47399999999999998},"min":{"1min":0.376,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68899999999999995,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56399999999999995,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.70299999999999996,"5min":4.649,"15min":4.649},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":212992,"data_stored":523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4329472,"data_stored":4000781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4145152,"data_stored":4000342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4247552,"data_stored":4000585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":204,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:37:58.229 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-18T00:37:58.229 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:37:58.229 INFO:tasks.thrashosds.thrasher:Growing pool iscsi 2024-02-18T00:37:58.230 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:37:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:56.909169+0000 mon.a (mon.0) 2450 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]': finished 2024-02-18T00:37:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:56.909259+0000 mon.a (mon.0) 2451 : cluster [DBG] osdmap e395: 8 total, 8 up, 7 in 2024-02-18T00:37:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:56.942690+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:56.943054+0000 mon.a (mon.0) 2452 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:37:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:56.943583+0000 mon.c (mon.2) 389 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:56.943822+0000 mon.a (mon.0) 2453 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-49"}]: dispatch 2024-02-18T00:37:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:57.577521+0000 mon.a (mon.0) 2454 : audit [DBG] from='client.? 172.21.15.59:0/3182060748' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:37:58.440 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:58.444 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:58.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:58.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:58.889 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:58.892 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:37:58.927 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:58.931 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:58.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:59.032 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:37:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:59.233 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:57.891893+0000 mgr.y (mgr.24454) 374 : cluster [DBG] pgmap v582: 238 pgs: 238 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 992 B/s rd, 0 op/s 2024-02-18T00:37:59.233 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:57.912465+0000 mon.a (mon.0) 2455 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T00:37:59.233 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:57.920815+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e396: 8 total, 8 up, 7 in 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:58.668877+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e397: 8 total, 8 up, 7 in 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:58.672991+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:58.673547+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:57.891893+0000 mgr.y (mgr.24454) 374 : cluster [DBG] pgmap v582: 238 pgs: 238 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 992 B/s rd, 0 op/s 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:57.912465+0000 mon.a (mon.0) 2455 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:57.920815+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e396: 8 total, 8 up, 7 in 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:58.668877+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e397: 8 total, 8 up, 7 in 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:58.672991+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:59.234 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:37:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:58.673547+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:59.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:57.891893+0000 mgr.y (mgr.24454) 374 : cluster [DBG] pgmap v582: 238 pgs: 238 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 992 B/s rd, 0 op/s 2024-02-18T00:37:59.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:57.912465+0000 mon.a (mon.0) 2455 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T00:37:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:57.920815+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e396: 8 total, 8 up, 7 in 2024-02-18T00:37:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:58.668877+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e397: 8 total, 8 up, 7 in 2024-02-18T00:37:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:58.672991+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:59.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:37:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:58.673547+0000 mon.a (mon.0) 2458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:37:59.259 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:37:59.262 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:59.262 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:37:59.363 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:37:59.395 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:59.416 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:37:59.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:59.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:37:59.520 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:59.550 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:59.570 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:59.573 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:59.573 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:59.674 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:37:59.709 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:59.729 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:37:59.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:59.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:37:59.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:37:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:37:59.961 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:37:59.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:37:59.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:00.065 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:00.093 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:00.113 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:00.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:00.116 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:00.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:00.243 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:00.261 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:00.264 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:00.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:00.365 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:00.392 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:00.409 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:00.411 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:00.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:00.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:00.546 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:00.571 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:00.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:00.575 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:00.676 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:00.702 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:00.720 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:00.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:00.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:00.824 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:00.851 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:00.869 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:00.872 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:00.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:00.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:59.669288+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:00.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:59.669399+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e398: 8 total, 8 up, 7 in 2024-02-18T00:38:00.897 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:59.706597+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:59.706876+0000 mon.a (mon.0) 2461 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:59.707311+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-51"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:37:59.707526+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-51"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:37:59.892474+0000 mgr.y (mgr.24454) 375 : cluster [DBG] pgmap v586: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:59.669288+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:59.669399+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e398: 8 total, 8 up, 7 in 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:59.706597+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:59.706876+0000 mon.a (mon.0) 2461 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:59.707311+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-51"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:37:59.707526+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-51"}]: dispatch 2024-02-18T00:38:00.898 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:37:59.892474+0000 mgr.y (mgr.24454) 375 : cluster [DBG] pgmap v586: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:00.919 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:38:00.973 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:59.669288+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:59.669399+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e398: 8 total, 8 up, 7 in 2024-02-18T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:59.706597+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:59.706876+0000 mon.a (mon.0) 2461 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:59.707311+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-51"}]: dispatch 2024-02-18T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:37:59.707526+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-51"}]: dispatch 2024-02-18T00:38:00.998 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:37:59.892474+0000 mgr.y (mgr.24454) 375 : cluster [DBG] pgmap v586: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:00.999 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:01.021 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:01.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:01.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:01.155 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:01.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:38:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:38:01.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:38:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:38:01.405 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:01.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:01.408 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:01.508 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:01.537 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:01.557 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:01.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:01.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:01.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:01.706 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:01.724 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:01.727 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:01.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:01.829 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:01.896 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:01.898 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:01.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:02.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:00.676283+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e399: 8 total, 8 up, 7 in 2024-02-18T00:38:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:01.228786+0000 mgr.y (mgr.24454) 376 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:02.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:00.676283+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e399: 8 total, 8 up, 7 in 2024-02-18T00:38:02.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:01.228786+0000 mgr.y (mgr.24454) 376 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:02.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:00.676283+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e399: 8 total, 8 up, 7 in 2024-02-18T00:38:02.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:01.228786+0000 mgr.y (mgr.24454) 376 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:02.028 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:02.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 -- 172.21.15.59:0/2129562118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0fe540 msgr2=0x7f4ecc0f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/2129562118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0fe540 0x7f4ecc0f60c0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7f4ec0004870 tx=0x7f4ec0009cb0).stop 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 -- 172.21.15.59:0/2129562118 shutdown_connections 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/2129562118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ecc0f67a0 0x7f4ecc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/2129562118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ecc0fdbe0 0x7f4ecc0fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/2129562118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0fe540 0x7f4ecc0f60c0 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 -- 172.21.15.59:0/2129562118 >> 172.21.15.59:0/2129562118 conn(0x7f4ecc06ffc0 msgr2=0x7f4ecc0f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 -- 172.21.15.59:0/2129562118 shutdown_connections 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 -- 172.21.15.59:0/2129562118 wait complete. 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 librados: starting msgr at 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 librados: starting objecter 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.079+0000 7f4ed17e4700 1 Processor -- start 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 -- start start 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 librados: setting wanted keys 2024-02-18T00:38:02.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 librados: calling monclient init 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0f67a0 0x7f4ecc10e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ecc0fdbe0 0x7f4ecc10e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ecc0fe540 0x7f4ecc108300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1087f0 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc108960 con 0x7f4ecc0fdbe0 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ed17e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc108b20 con 0x7f4ecc0fe540 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0f67a0 0x7f4ecc10e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0f67a0 0x7f4ecc10e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39048/0 (socket says 172.21.15.59:39048) 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 -- 172.21.15.59:0/661133989 learned_addr learned my addr 172.21.15.59:0/661133989 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecb7fe700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ecc0fe540 0x7f4ecc108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4eca7fc700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ecc0fdbe0 0x7f4ecc10e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:02.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 -- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ecc0fe540 msgr2=0x7f4ecc108300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ecc0fe540 0x7f4ecc108300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 -- 172.21.15.59:0/661133989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ecc0fdbe0 msgr2=0x7f4ecc10e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ecc0fdbe0 0x7f4ecc10e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 -- 172.21.15.59:0/661133989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc005040 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4ecaffd700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0f67a0 0x7f4ecc10e160 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 rx=0x7f4ebc0048f0 tx=0x7f4ebc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4eb3fff700 10 client.?.objecter ms_handle_connect 0x7f4ecc0f67a0 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.080+0000 7f4eb3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.081+0000 7f4eb3fff700 1 -- 172.21.15.59:0/661133989 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc007b00 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.081+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec0005040 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.081+0000 7f4ed17e4700 10 client.31272.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.081+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc114f90 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.081+0000 7f4ed17e4700 1 librados: init done 2024-02-18T00:38:02.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.081+0000 7f4eb3fff700 1 -- 172.21.15.59:0/661133989 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ebc00f040 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.081+0000 7f4eb3fff700 1 -- 172.21.15.59:0/661133989 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc014910 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.082+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb8004f50 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.082+0000 7f4eb3fff700 1 -- 172.21.15.59:0/661133989 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4ebc010710 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.082+0000 7f4eb3fff700 1 --2- 172.21.15.59:0/661133989 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4eb4062c90 0x7f4eb4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eb3fff700 1 -- 172.21.15.59:0/661133989 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(400..400 src has 180..400) v4 ==== 8006+0+0 (secure 0 0 0) 0x7f4ebc07ccd0 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eb3fff700 10 client.31272.objecter ms_dispatch 0x7f4ecc001550 osd_map(400..400 src has 180..400) v4 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eb3fff700 3 client.31272.objecter handle_osd_map got epochs [400,400] > 0 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eb3fff700 3 client.31272.objecter handle_osd_map decoding full epoch 400 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eca7fc700 1 --2- 172.21.15.59:0/661133989 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4eb4062c90 0x7f4eb4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eb3fff700 20 client.31272.objecter dump_active .. 0 homeless 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eb3fff700 10 client.31272.objecter ms_handle_connect 0x7f4eb4062c90 2024-02-18T00:38:02.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.083+0000 7f4eca7fc700 1 --2- 172.21.15.59:0/661133989 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4eb4062c90 0x7f4eb4065150 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 rx=0x7f4ec0004ee0 tx=0x7f4ec000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.087+0000 7f4eb3fff700 1 -- 172.21.15.59:0/661133989 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4ebc049f60 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:02.105 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:02.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:02.206 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:02.223 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:02.232 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:02.251 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:02.253 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:02.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:02.254 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:02.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.290+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f4eb8005200 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.354 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:02.381 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:02] "GET /metrics HTTP/1.1" 200 189381 "" "Prometheus/2.33.4" 2024-02-18T00:38:02.383 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:02.525 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:02.527 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:02.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:02.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:02.658 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:02.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.685+0000 7f4eb3fff700 1 -- 172.21.15.59:0/661133989 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v401) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4ebc050210 con 0x7f4ecc0f67a0 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.694+0000 7f4ed17e4700 10 librados: watch_flush enter 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.695+0000 7f4ed17e4700 10 librados: watch_flush exit 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.695+0000 7f4ed17e4700 20 client.31272.objecter shutdown clearing up homeless session... 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.695+0000 7f4ed17e4700 10 client.31272.objecter successfully canceled tick 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.695+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4eb4062c90 msgr2=0x7f4eb4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.695+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/661133989 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4eb4062c90 0x7f4eb4065150 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 rx=0x7f4ec0004ee0 tx=0x7f4ec000b040).stop 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.695+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0f67a0 msgr2=0x7f4ecc10e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:02.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.695+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0f67a0 0x7f4ecc10e160 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 rx=0x7f4ebc0048f0 tx=0x7f4ebc009cb0).stop 2024-02-18T00:38:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.702+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 shutdown_connections 2024-02-18T00:38:02.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.702+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ecc0fe540 0x7f4ecc108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.702+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ecc0f67a0 0x7f4ecc10e160 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.702+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/661133989 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4eb4062c90 0x7f4eb4065150 unknown :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.702+0000 7f4ed17e4700 1 --2- 172.21.15.59:0/661133989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ecc0fdbe0 0x7f4ecc10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.702+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 >> 172.21.15.59:0/661133989 conn(0x7f4ecc06ffc0 msgr2=0x7f4ecc0fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.704+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 shutdown_connections 2024-02-18T00:38:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.705+0000 7f4ed17e4700 1 -- 172.21.15.59:0/661133989 wait complete. 2024-02-18T00:38:02.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:02.705+0000 7f4ed17e4700 1 librados: shutdown 2024-02-18T00:38:02.709 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:38:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:01.679416+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e400: 8 total, 8 up, 7 in 2024-02-18T00:38:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:01.682023+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:01.682732+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:01.893057+0000 mgr.y (mgr.24454) 377 : cluster [DBG] pgmap v589: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 515 B/s wr, 1 op/s 2024-02-18T00:38:03.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.292229+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? 172.21.15.59:0/661133989' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:03.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:01.679416+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e400: 8 total, 8 up, 7 in 2024-02-18T00:38:03.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:01.682023+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:03.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:01.682732+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:03.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:01.893057+0000 mgr.y (mgr.24454) 377 : cluster [DBG] pgmap v589: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 515 B/s wr, 1 op/s 2024-02-18T00:38:03.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.292229+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? 172.21.15.59:0/661133989' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:03.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:01.679416+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e400: 8 total, 8 up, 7 in 2024-02-18T00:38:03.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:01.682023+0000 mon.c (mon.2) 393 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:01.682732+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:01.893057+0000 mgr.y (mgr.24454) 377 : cluster [DBG] pgmap v589: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 515 B/s wr, 1 op/s 2024-02-18T00:38:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.292229+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? 172.21.15.59:0/661133989' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:02.677205+0000 mon.a (mon.0) 2467 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.685166+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.685330+0000 mon.a (mon.0) 2469 : audit [INF] from='client.? 172.21.15.59:0/661133989' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:02.685381+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e401: 8 total, 8 up, 7 in 2024-02-18T00:38:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.694106+0000 mon.c (mon.2) 394 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.732709+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.733085+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.733526+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-53"}]: dispatch 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:02.733772+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-53"}]: dispatch 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:02.677205+0000 mon.a (mon.0) 2467 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.685166+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.685330+0000 mon.a (mon.0) 2469 : audit [INF] from='client.? 172.21.15.59:0/661133989' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:02.685381+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e401: 8 total, 8 up, 7 in 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.694106+0000 mon.c (mon.2) 394 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.732709+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.733085+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.733526+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-53"}]: dispatch 2024-02-18T00:38:04.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:02.733772+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-53"}]: dispatch 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:02.677205+0000 mon.a (mon.0) 2467 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.685166+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.685330+0000 mon.a (mon.0) 2469 : audit [INF] from='client.? 172.21.15.59:0/661133989' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:02.685381+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e401: 8 total, 8 up, 7 in 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.694106+0000 mon.c (mon.2) 394 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.732709+0000 mon.c (mon.2) 395 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.733085+0000 mon.a (mon.0) 2471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:04.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.733526+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-53"}]: dispatch 2024-02-18T00:38:04.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:02.733772+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-53"}]: dispatch 2024-02-18T00:38:04.267 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:38:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:38:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:38:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:38:04.268 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:38:04.510 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:04.513 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:04.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:04.615 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:04.671 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:04.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:04.674 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:04.699 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:03.689418+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e402: 8 total, 8 up, 7 in 2024-02-18T00:38:04.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:04.806 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:05.001 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:03.893414+0000 mgr.y (mgr.24454) 378 : cluster [DBG] pgmap v592: 206 pgs: 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:03.689418+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e402: 8 total, 8 up, 7 in 2024-02-18T00:38:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:03.893414+0000 mgr.y (mgr.24454) 378 : cluster [DBG] pgmap v592: 206 pgs: 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:03.689418+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e402: 8 total, 8 up, 7 in 2024-02-18T00:38:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:03.893414+0000 mgr.y (mgr.24454) 378 : cluster [DBG] pgmap v592: 206 pgs: 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:05.255 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:05.258 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:05.258 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:05.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:05.388 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:04.700872+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e403: 8 total, 8 up, 7 in 2024-02-18T00:38:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:04.709053+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:04.711604+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:05.036593+0000 osd.1 (osd.1) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:05.036792+0000 osd.1 (osd.1) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:04.700872+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e403: 8 total, 8 up, 7 in 2024-02-18T00:38:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:04.709053+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:04.711604+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:05.036593+0000 osd.1 (osd.1) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:06.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:05.036792+0000 osd.1 (osd.1) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:06.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:04.700872+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e403: 8 total, 8 up, 7 in 2024-02-18T00:38:06.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:04.709053+0000 mon.c (mon.2) 397 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:06.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:04.711604+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:06.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:05.036593+0000 osd.1 (osd.1) 203 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:06.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:05.036792+0000 osd.1 (osd.1) 204 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:06.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:06.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:06.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:06.876 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:06.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/483852312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0fe510 msgr2=0x7f5ebc0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/483852312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0fe510 0x7f5ebc0f7d40 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 rx=0x7f5ea80047b0 tx=0x7f5ea8009cb0).stop 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/483852312 shutdown_connections 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/483852312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ebc0fdbb0 0x7f5ebc0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/483852312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0f8420 0x7f5ebc104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/483852312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0fe510 0x7f5ebc0f7d40 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/483852312 >> 172.21.15.59:0/483852312 conn(0x7f5ebc0f3800 msgr2=0x7f5ebc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/483852312 shutdown_connections 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/483852312 wait complete. 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 librados: starting msgr at 2024-02-18T00:38:06.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 librados: starting objecter 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 Processor -- start 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 -- start start 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 librados: setting wanted keys 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.990+0000 7f5ec2b9e700 1 librados: calling monclient init 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec2b9e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0f8420 0x7f5ebc10e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec2b9e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ebc0fdbb0 0x7f5ebc10e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec2b9e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0fe510 0x7f5ebc108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec2b9e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc108720 con 0x7f5ebc0f8420 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec2b9e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc108890 con 0x7f5ebc0fdbb0 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec2b9e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc108a00 con 0x7f5ebc0fe510 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec113b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0fe510 0x7f5ebc108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec093a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0f8420 0x7f5ebc10e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:06.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec113b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0fe510 0x7f5ebc108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:32870/0 (socket says 172.21.15.59:32870) 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec113b700 1 -- 172.21.15.59:0/747252199 learned_addr learned my addr 172.21.15.59:0/747252199 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ebbfff700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ebc0fdbb0 0x7f5ebc10e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec093a700 1 -- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0fe510 msgr2=0x7f5ebc108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec093a700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0fe510 0x7f5ebc108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec093a700 1 -- 172.21.15.59:0/747252199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ebc0fdbb0 msgr2=0x7f5ebc10e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec093a700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ebc0fdbb0 0x7f5ebc10e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec093a700 1 -- 172.21.15.59:0/747252199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40097c0 con 0x7f5ebc0f8420 2024-02-18T00:38:06.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec113b700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0fe510 0x7f5ebc108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec093a700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0f8420 0x7f5ebc10e050 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 rx=0x7f5eb0002780 tx=0x7f5eb0009e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5eb9ffb700 10 client.?.objecter ms_handle_connect 0x7f5ebc0f8420 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5eb9ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5eb9ffb700 1 -- 172.21.15.59:0/747252199 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0007b40 con 0x7f5ebc0f8420 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea8005040 con 0x7f5ebc0f8420 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5eb9ffb700 1 -- 172.21.15.59:0/747252199 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5eb000f040 con 0x7f5ebc0f8420 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.991+0000 7f5eb9ffb700 1 -- 172.21.15.59:0/747252199 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0014850 con 0x7f5ebc0f8420 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.992+0000 7f5ec2b9e700 10 client.31278.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.992+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ebc109090 con 0x7f5ebc0f8420 2024-02-18T00:38:06.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.992+0000 7f5ec2b9e700 1 librados: init done 2024-02-18T00:38:06.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ebc05ee80 con 0x7f5ebc0f8420 2024-02-18T00:38:06.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5eb9ffb700 1 -- 172.21.15.59:0/747252199 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5eb0007ca0 con 0x7f5ebc0f8420 2024-02-18T00:38:06.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5eb9ffb700 1 --2- 172.21.15.59:0/747252199 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5eac062c90 0x7f5eac065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5eb9ffb700 1 -- 172.21.15.59:0/747252199 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(405..405 src has 180..405) v4 ==== 8019+0+0 (secure 0 0 0) 0x7f5eb007d150 con 0x7f5ebc0f8420 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5ebbfff700 1 --2- 172.21.15.59:0/747252199 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5eac062c90 0x7f5eac065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5eb9ffb700 10 client.31278.objecter ms_dispatch 0x7f5ebc001550 osd_map(405..405 src has 180..405) v4 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5eb9ffb700 3 client.31278.objecter handle_osd_map got epochs [405,405] > 0 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.993+0000 7f5eb9ffb700 3 client.31278.objecter handle_osd_map decoding full epoch 405 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.994+0000 7f5eb9ffb700 20 client.31278.objecter dump_active .. 0 homeless 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.994+0000 7f5eb9ffb700 10 client.31278.objecter ms_handle_connect 0x7f5eac062c90 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.994+0000 7f5ebbfff700 1 --2- 172.21.15.59:0/747252199 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5eac062c90 0x7f5eac065150 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 rx=0x7f5ebc1099d0 tx=0x7f5ea8007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:06.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:06.998+0000 7f5eb9ffb700 1 -- 172.21.15.59:0/747252199 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5eb0050680 con 0x7f5ebc0f8420 2024-02-18T00:38:07.001 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:07.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:05.700059+0000 mon.a (mon.0) 2476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:07.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:05.700147+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e404: 8 total, 8 up, 7 in 2024-02-18T00:38:07.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:05.894576+0000 mgr.y (mgr.24454) 379 : cluster [DBG] pgmap v595: 238 pgs: 10 creating+activating, 228 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-18T00:38:07.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:05.700059+0000 mon.a (mon.0) 2476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:07.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:05.700147+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e404: 8 total, 8 up, 7 in 2024-02-18T00:38:07.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:05.894576+0000 mgr.y (mgr.24454) 379 : cluster [DBG] pgmap v595: 238 pgs: 10 creating+activating, 228 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-18T00:38:07.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:05.700059+0000 mon.a (mon.0) 2476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:07.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:05.700147+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e404: 8 total, 8 up, 7 in 2024-02-18T00:38:07.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:05.894576+0000 mgr.y (mgr.24454) 379 : cluster [DBG] pgmap v595: 238 pgs: 10 creating+activating, 228 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-18T00:38:07.005 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:07.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:07.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:07.136 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:07.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.179+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ebc05fa60 con 0x7f5eac062c90 2024-02-18T00:38:07.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.196+0000 7f5eb9ffb700 1 -- 172.21.15.59:0/747252199 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+597783 (secure 0 0 0) 0x7f5ebc05fa60 con 0x7f5eac062c90 2024-02-18T00:38:07.207 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:38:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 10 librados: watch_flush enter 2024-02-18T00:38:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 10 librados: watch_flush exit 2024-02-18T00:38:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 20 client.31278.objecter shutdown clearing up homeless session... 2024-02-18T00:38:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 10 client.31278.objecter successfully canceled tick 2024-02-18T00:38:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5eac062c90 msgr2=0x7f5eac065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/747252199 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5eac062c90 0x7f5eac065150 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 rx=0x7f5ebc1099d0 tx=0x7f5ea8007f90).stop 2024-02-18T00:38:07.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0f8420 msgr2=0x7f5ebc10e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:07.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0f8420 0x7f5ebc10e050 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 rx=0x7f5eb0002780 tx=0x7f5eb0009e40).stop 2024-02-18T00:38:07.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 shutdown_connections 2024-02-18T00:38:07.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ebc0fe510 0x7f5ebc108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:07.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ebc0f8420 0x7f5ebc10e050 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:07.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/747252199 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5eac062c90 0x7f5eac065150 unknown :-1 s=CLOSED pgs=3100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:07.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.199+0000 7f5ec2b9e700 1 --2- 172.21.15.59:0/747252199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ebc0fdbb0 0x7f5ebc10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:07.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.200+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 >> 172.21.15.59:0/747252199 conn(0x7f5ebc0f3800 msgr2=0x7f5ebc0f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:07.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.200+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 shutdown_connections 2024-02-18T00:38:07.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.200+0000 7f5ec2b9e700 1 -- 172.21.15.59:0/747252199 wait complete. 2024-02-18T00:38:07.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:07.200+0000 7f5ec2b9e700 1 librados: shutdown 2024-02-18T00:38:07.216 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:38:07.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:06.715558+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e405: 8 total, 8 up, 7 in 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:06.717825+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:06.719466+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:06.720009+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-55"}]: dispatch 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:06.720295+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-55"}]: dispatch 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:07.180798+0000 mgr.y (mgr.24454) 380 : audit [DBG] from='client.31278 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:06.715558+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e405: 8 total, 8 up, 7 in 2024-02-18T00:38:08.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:06.717825+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:06.719466+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:06.720009+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-55"}]: dispatch 2024-02-18T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:06.720295+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-55"}]: dispatch 2024-02-18T00:38:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:07.180798+0000 mgr.y (mgr.24454) 380 : audit [DBG] from='client.31278 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:38:08.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:06.715558+0000 mon.a (mon.0) 2478 : cluster [DBG] osdmap e405: 8 total, 8 up, 7 in 2024-02-18T00:38:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:06.717825+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:06.719466+0000 mon.a (mon.0) 2479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:06.720009+0000 mon.c (mon.2) 399 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-55"}]: dispatch 2024-02-18T00:38:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:06.720295+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-55"}]: dispatch 2024-02-18T00:38:08.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:07.180798+0000 mgr.y (mgr.24454) 380 : audit [DBG] from='client.31278 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:38:08.178 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":596,"stamp":"2024-02-18T00:38:06.712347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007256,"num_objects":380,"num_object_clones":2,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":380,"num_whiteouts":0,"num_read":1735,"num_read_kb":1356,"num_write":1212,"num_write_kb":4532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2562,"ondisk_log_size":2562,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":671,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2148592,"kb_used_data":43448,"kb_used_omap":0,"kb_used_meta":2105088,"kb_avail":654067472,"statfs":{"total":671965249536,"available":669765091328,"internally_reserved":0,"allocated":44490752,"data_stored":28419108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2155610112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030125"},"pg_stats":[{"pgid":"209.1f","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.714454+0000","last_change":"2024-02-18T00:38:05.714454+0000","last_active":"2024-02-18T00:38:05.714454+0000","last_peered":"2024-02-18T00:38:05.714454+0000","last_clean":"2024-02-18T00:38:05.714454+0000","last_became_active":"2024-02-18T00:38:05.714144+0000","last_became_peered":"2024-02-18T00:38:05.714144+0000","last_unstale":"2024-02-18T00:38:05.714454+0000","last_undegraded":"2024-02-18T00:38:05.714454+0000","last_fullsized":"2024-02-18T00:38:05.714454+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"209.1e","version":"404'2","reported_seq":12,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.732732+0000","last_change":"2024-02-18T00:38:05.715465+0000","last_active":"2024-02-18T00:38:05.732732+0000","last_peered":"2024-02-18T00:38:05.732732+0000","last_clean":"2024-02-18T00:38:05.732732+0000","last_became_active":"2024-02-18T00:38:05.715145+0000","last_became_peered":"2024-02-18T00:38:05.715145+0000","last_unstale":"2024-02-18T00:38:05.732732+0000","last_undegraded":"2024-02-18T00:38:05.732732+0000","last_fullsized":"2024-02-18T00:38:05.732732+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"209.1d","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.718421+0000","last_change":"2024-02-18T00:38:05.718421+0000","last_active":"2024-02-18T00:38:05.718421+0000","last_peered":"2024-02-18T00:38:05.718421+0000","last_clean":"2024-02-18T00:38:05.718421+0000","last_became_active":"2024-02-18T00:38:05.718119+0000","last_became_peered":"2024-02-18T00:38:05.718119+0000","last_unstale":"2024-02-18T00:38:05.718421+0000","last_undegraded":"2024-02-18T00:38:05.718421+0000","last_fullsized":"2024-02-18T00:38:05.718421+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"209.1c","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.724992+0000","last_change":"2024-02-18T00:38:05.724992+0000","last_active":"2024-02-18T00:38:05.724992+0000","last_peered":"2024-02-18T00:38:05.724992+0000","last_clean":"2024-02-18T00:38:05.724992+0000","last_became_active":"2024-02-18T00:38:05.724644+0000","last_became_peered":"2024-02-18T00:38:05.724644+0000","last_unstale":"2024-02-18T00:38:05.724992+0000","last_undegraded":"2024-02-18T00:38:05.724992+0000","last_fullsized":"2024-02-18T00:38:05.724992+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"209.1b","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.713974+0000","last_change":"2024-02-18T00:38:05.713974+0000","last_active":"2024-02-18T00:38:05.713974+0000","last_peered":"2024-02-18T00:38:05.713974+0000","last_clean":"2024-02-18T00:38:05.713974+0000","last_became_active":"2024-02-18T00:38:05.713532+0000","last_became_peered":"2024-02-18T00:38:05.713532+0000","last_unstale":"2024-02-18T00:38:05.713974+0000","last_undegraded":"2024-02-18T00:38:05.713974+0000","last_fullsized":"2024-02-18T00:38:05.713974+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"209.1a","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.713838+0000","last_change":"2024-02-18T00:38:05.713838+0000","last_active":"2024-02-18T00:38:05.713838+0000","last_peered":"2024-02-18T00:38:05.713838+0000","last_clean":"2024-02-18T00:38:05.713838+0000","last_became_active":"2024-02-18T00:38:05.713489+0000","last_became_peered":"2024-02-18T00:38:05.713489+0000","last_unstale":"2024-02-18T00:38:05.713838+0000","last_undegraded":"2024-02-18T00:38:05.713838+0000","last_fullsized":"2024-02-18T00:38:05.713838+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"209.19","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.703955+0000","last_change":"2024-02-18T00:38:05.703955+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.703955+0000","last_undegraded":"2024-02-18T00:38:05.703955+0000","last_fullsized":"2024-02-18T00:38:05.703955+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":["0","1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"209.18","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.720136+0000","last_change":"2024-02-18T00:38:05.720136+0000","last_active":"2024-02-18T00:38:05.720136+0000","last_peered":"2024-02-18T00:38:05.720136+0000","last_clean":"2024-02-18T00:38:05.720136+0000","last_became_active":"2024-02-18T00:38:05.719852+0000","last_became_peered":"2024-02-18T00:38:05.719852+0000","last_unstale":"2024-02-18T00:38:05.720136+0000","last_undegraded":"2024-02-18T00:38:05.720136+0000","last_fullsized":"2024-02-18T00:38:05.720136+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"209.17","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.719515+0000","last_change":"2024-02-18T00:38:05.719515+0000","last_active":"2024-02-18T00:38:05.719515+0000","last_peered":"2024-02-18T00:38:05.719515+0000","last_clean":"2024-02-18T00:38:05.719515+0000","last_became_active":"2024-02-18T00:38:05.719241+0000","last_became_peered":"2024-02-18T00:38:05.719241+0000","last_unstale":"2024-02-18T00:38:05.719515+0000","last_undegraded":"2024-02-18T00:38:05.719515+0000","last_fullsized":"2024-02-18T00:38:05.719515+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"209.16","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.712195+0000","last_change":"2024-02-18T00:38:05.712195+0000","last_active":"2024-02-18T00:38:05.712195+0000","last_peered":"2024-02-18T00:38:05.712195+0000","last_clean":"2024-02-18T00:38:05.712195+0000","last_became_active":"2024-02-18T00:38:05.711842+0000","last_became_peered":"2024-02-18T00:38:05.711842+0000","last_unstale":"2024-02-18T00:38:05.712195+0000","last_undegraded":"2024-02-18T00:38:05.712195+0000","last_fullsized":"2024-02-18T00:38:05.712195+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"209.15","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.706407+0000","last_change":"2024-02-18T00:38:05.706407+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.706407+0000","last_undegraded":"2024-02-18T00:38:05.706407+0000","last_fullsized":"2024-02-18T00:38:05.706407+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":["6","3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"209.14","version":"404'2","reported_seq":12,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.730563+0000","last_change":"2024-02-18T00:38:05.714710+0000","last_active":"2024-02-18T00:38:05.730563+0000","last_peered":"2024-02-18T00:38:05.730563+0000","last_clean":"2024-02-18T00:38:05.730563+0000","last_became_active":"2024-02-18T00:38:05.714223+0000","last_became_peered":"2024-02-18T00:38:05.714223+0000","last_unstale":"2024-02-18T00:38:05.730563+0000","last_undegraded":"2024-02-18T00:38:05.730563+0000","last_fullsized":"2024-02-18T00:38:05.730563+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"209.13","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.713808+0000","last_change":"2024-02-18T00:38:05.713808+0000","last_active":"2024-02-18T00:38:05.713808+0000","last_peered":"2024-02-18T00:38:05.713808+0000","last_clean":"2024-02-18T00:38:05.713808+0000","last_became_active":"2024-02-18T00:38:05.713128+0000","last_became_peered":"2024-02-18T00:38:05.713128+0000","last_unstale":"2024-02-18T00:38:05.713808+0000","last_undegraded":"2024-02-18T00:38:05.713808+0000","last_fullsized":"2024-02-18T00:38:05.713808+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"209.12","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.715958+0000","last_change":"2024-02-18T00:38:05.715958+0000","last_active":"2024-02-18T00:38:05.715958+0000","last_peered":"2024-02-18T00:38:05.715958+0000","last_clean":"2024-02-18T00:38:05.715958+0000","last_became_active":"2024-02-18T00:38:05.715658+0000","last_became_peered":"2024-02-18T00:38:05.715658+0000","last_unstale":"2024-02-18T00:38:05.715958+0000","last_undegraded":"2024-02-18T00:38:05.715958+0000","last_fullsized":"2024-02-18T00:38:05.715958+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"209.11","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.699652+0000","last_change":"2024-02-18T00:38:05.699652+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.699652+0000","last_undegraded":"2024-02-18T00:38:05.699652+0000","last_fullsized":"2024-02-18T00:38:05.699652+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":["4","5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"209.10","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.702969+0000","last_change":"2024-02-18T00:38:05.702969+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.702969+0000","last_undegraded":"2024-02-18T00:38:05.702969+0000","last_fullsized":"2024-02-18T00:38:05.702969+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":["0","3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"209.f","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.699259+0000","last_change":"2024-02-18T00:38:05.699259+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.699259+0000","last_undegraded":"2024-02-18T00:38:05.699259+0000","last_fullsized":"2024-02-18T00:38:05.699259+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":["4","0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"209.e","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.713769+0000","last_change":"2024-02-18T00:38:05.713769+0000","last_active":"2024-02-18T00:38:05.713769+0000","last_peered":"2024-02-18T00:38:05.713769+0000","last_clean":"2024-02-18T00:38:05.713769+0000","last_became_active":"2024-02-18T00:38:05.713441+0000","last_became_peered":"2024-02-18T00:38:05.713441+0000","last_unstale":"2024-02-18T00:38:05.713769+0000","last_undegraded":"2024-02-18T00:38:05.713769+0000","last_fullsized":"2024-02-18T00:38:05.713769+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"209.d","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.719555+0000","last_change":"2024-02-18T00:38:05.719555+0000","last_active":"2024-02-18T00:38:05.719555+0000","last_peered":"2024-02-18T00:38:05.719555+0000","last_clean":"2024-02-18T00:38:05.719555+0000","last_became_active":"2024-02-18T00:38:05.719232+0000","last_became_peered":"2024-02-18T00:38:05.719232+0000","last_unstale":"2024-02-18T00:38:05.719555+0000","last_undegraded":"2024-02-18T00:38:05.719555+0000","last_fullsized":"2024-02-18T00:38:05.719555+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"209.c","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.714072+0000","last_change":"2024-02-18T00:38:05.714072+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.714072+0000","last_undegraded":"2024-02-18T00:38:05.714072+0000","last_fullsized":"2024-02-18T00:38:05.714072+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":["3","2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"209.b","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.699975+0000","last_change":"2024-02-18T00:38:05.699975+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.699975+0000","last_undegraded":"2024-02-18T00:38:05.699975+0000","last_fullsized":"2024-02-18T00:38:05.699975+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":["2","3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"209.a","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.715503+0000","last_change":"2024-02-18T00:38:05.715503+0000","last_active":"2024-02-18T00:38:05.715503+0000","last_peered":"2024-02-18T00:38:05.715503+0000","last_clean":"2024-02-18T00:38:05.715503+0000","last_became_active":"2024-02-18T00:38:05.715201+0000","last_became_peered":"2024-02-18T00:38:05.715201+0000","last_unstale":"2024-02-18T00:38:05.715503+0000","last_undegraded":"2024-02-18T00:38:05.715503+0000","last_fullsized":"2024-02-18T00:38:05.715503+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"209.9","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.715131+0000","last_change":"2024-02-18T00:38:05.715131+0000","last_active":"2024-02-18T00:38:05.715131+0000","last_peered":"2024-02-18T00:38:05.715131+0000","last_clean":"2024-02-18T00:38:05.715131+0000","last_became_active":"2024-02-18T00:38:05.714806+0000","last_became_peered":"2024-02-18T00:38:05.714806+0000","last_unstale":"2024-02-18T00:38:05.715131+0000","last_undegraded":"2024-02-18T00:38:05.715131+0000","last_fullsized":"2024-02-18T00:38:05.715131+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"209.8","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.720490+0000","last_change":"2024-02-18T00:38:05.720490+0000","last_active":"2024-02-18T00:38:05.720490+0000","last_peered":"2024-02-18T00:38:05.720490+0000","last_clean":"2024-02-18T00:38:05.720490+0000","last_became_active":"2024-02-18T00:38:05.720224+0000","last_became_peered":"2024-02-18T00:38:05.720224+0000","last_unstale":"2024-02-18T00:38:05.720490+0000","last_undegraded":"2024-02-18T00:38:05.720490+0000","last_fullsized":"2024-02-18T00:38:05.720490+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"209.7","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.714558+0000","last_change":"2024-02-18T00:38:05.714558+0000","last_active":"2024-02-18T00:38:05.714558+0000","last_peered":"2024-02-18T00:38:05.714558+0000","last_clean":"2024-02-18T00:38:05.714558+0000","last_became_active":"2024-02-18T00:38:05.714212+0000","last_became_peered":"2024-02-18T00:38:05.714212+0000","last_unstale":"2024-02-18T00:38:05.714558+0000","last_undegraded":"2024-02-18T00:38:05.714558+0000","last_fullsized":"2024-02-18T00:38:05.714558+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"209.6","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.702115+0000","last_change":"2024-02-18T00:38:05.702115+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.702115+0000","last_undegraded":"2024-02-18T00:38:05.702115+0000","last_fullsized":"2024-02-18T00:38:05.702115+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":["2","1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"209.5","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.699182+0000","last_change":"2024-02-18T00:38:05.699182+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.699182+0000","last_undegraded":"2024-02-18T00:38:05.699182+0000","last_fullsized":"2024-02-18T00:38:05.699182+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":["4","5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"209.4","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.718274+0000","last_change":"2024-02-18T00:38:05.718274+0000","last_active":"2024-02-18T00:38:05.718274+0000","last_peered":"2024-02-18T00:38:05.718274+0000","last_clean":"2024-02-18T00:38:05.718274+0000","last_became_active":"2024-02-18T00:38:05.715982+0000","last_became_peered":"2024-02-18T00:38:05.715982+0000","last_unstale":"2024-02-18T00:38:05.718274+0000","last_undegraded":"2024-02-18T00:38:05.718274+0000","last_fullsized":"2024-02-18T00:38:05.718274+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"209.3","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.714363+0000","last_change":"2024-02-18T00:38:05.714363+0000","last_active":"2024-02-18T00:38:05.714363+0000","last_peered":"2024-02-18T00:38:05.714363+0000","last_clean":"2024-02-18T00:38:05.714363+0000","last_became_active":"2024-02-18T00:38:05.713981+0000","last_became_peered":"2024-02-18T00:38:05.713981+0000","last_unstale":"2024-02-18T00:38:05.714363+0000","last_undegraded":"2024-02-18T00:38:05.714363+0000","last_fullsized":"2024-02-18T00:38:05.714363+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"209.2","version":"0'0","reported_seq":3,"reported_epoch":404,"state":"creating+activating","last_fresh":"2024-02-18T00:38:05.705799+0000","last_change":"2024-02-18T00:38:05.705799+0000","last_active":"2024-02-18T00:38:04.687174+0000","last_peered":"2024-02-18T00:38:04.687174+0000","last_clean":"2024-02-18T00:38:04.687174+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:38:05.705799+0000","last_undegraded":"2024-02-18T00:38:05.705799+0000","last_fullsized":"2024-02-18T00:38:05.705799+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":["6","3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"209.1","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.712861+0000","last_change":"2024-02-18T00:38:05.712861+0000","last_active":"2024-02-18T00:38:05.712861+0000","last_peered":"2024-02-18T00:38:05.712861+0000","last_clean":"2024-02-18T00:38:05.712861+0000","last_became_active":"2024-02-18T00:38:05.712401+0000","last_became_peered":"2024-02-18T00:38:05.712401+0000","last_unstale":"2024-02-18T00:38:05.712861+0000","last_undegraded":"2024-02-18T00:38:05.712861+0000","last_fullsized":"2024-02-18T00:38:05.712861+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"209.0","version":"0'0","reported_seq":9,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.723193+0000","last_change":"2024-02-18T00:38:05.723193+0000","last_active":"2024-02-18T00:38:05.723193+0000","last_peered":"2024-02-18T00:38:05.723193+0000","last_clean":"2024-02-18T00:38:05.723193+0000","last_became_active":"2024-02-18T00:38:05.722910+0000","last_became_peered":"2024-02-18T00:38:05.722910+0000","last_unstale":"2024-02-18T00:38:05.723193+0000","last_undegraded":"2024-02-18T00:38:05.723193+0000","last_fullsized":"2024-02-18T00:38:05.723193+0000","mapping_epoch":403,"log_start":"0'0","ondisk_log_start":"0'0","created":403,"last_epoch_clean":404,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:04.687174+0000","last_clean_scrub_stamp":"2024-02-18T00:38:04.687174+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":374,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698488+0000","last_change":"2024-02-18T00:36:09.722458+0000","last_active":"2024-02-18T00:38:04.698488+0000","last_peered":"2024-02-18T00:38:04.698488+0000","last_clean":"2024-02-18T00:38:04.698488+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:38:04.698488+0000","last_undegraded":"2024-02-18T00:38:04.698488+0000","last_fullsized":"2024-02-18T00:38:04.698488+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:36:09.722339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.722339+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"401'6","reported_seq":348,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709909+0000","last_change":"2024-02-18T00:37:29.759198+0000","last_active":"2024-02-18T00:38:04.709909+0000","last_peered":"2024-02-18T00:38:04.709909+0000","last_clean":"2024-02-18T00:38:04.709909+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:38:04.709909+0000","last_undegraded":"2024-02-18T00:38:04.709909+0000","last_fullsized":"2024-02-18T00:38:04.709909+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'2","last_scrub_stamp":"2024-02-18T00:37:29.759114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.759114+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"231'68","reported_seq":402,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703804+0000","last_change":"2024-02-18T00:36:06.657872+0000","last_active":"2024-02-18T00:38:04.703804+0000","last_peered":"2024-02-18T00:38:04.703804+0000","last_clean":"2024-02-18T00:38:04.703804+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:38:04.703804+0000","last_undegraded":"2024-02-18T00:38:04.703804+0000","last_fullsized":"2024-02-18T00:38:04.703804+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'68","last_scrub_stamp":"2024-02-18T00:35:56.424859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.424859+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"347'75","reported_seq":405,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700722+0000","last_change":"2024-02-18T00:36:34.581378+0000","last_active":"2024-02-18T00:38:04.700722+0000","last_peered":"2024-02-18T00:38:04.700722+0000","last_clean":"2024-02-18T00:38:04.700722+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:38:04.700722+0000","last_undegraded":"2024-02-18T00:38:04.700722+0000","last_fullsized":"2024-02-18T00:38:04.700722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'73","last_scrub_stamp":"2024-02-18T00:36:34.581302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.581302+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"404'123","reported_seq":447,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.724407+0000","last_change":"2024-02-18T00:36:36.464430+0000","last_active":"2024-02-18T00:38:05.724407+0000","last_peered":"2024-02-18T00:38:05.724407+0000","last_clean":"2024-02-18T00:38:05.724407+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:38:05.724407+0000","last_undegraded":"2024-02-18T00:38:05.724407+0000","last_fullsized":"2024-02-18T00:38:05.724407+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'120","last_scrub_stamp":"2024-02-18T00:36:36.464361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.464361+0000","log_size":123,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":39,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":419,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699178+0000","last_change":"2024-02-18T00:36:10.385196+0000","last_active":"2024-02-18T00:38:04.699178+0000","last_peered":"2024-02-18T00:38:04.699178+0000","last_clean":"2024-02-18T00:38:04.699178+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:38:04.699178+0000","last_undegraded":"2024-02-18T00:38:04.699178+0000","last_fullsized":"2024-02-18T00:38:04.699178+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'90","last_scrub_stamp":"2024-02-18T00:36:10.385126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.385126+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"404'127","reported_seq":419,"reported_epoch":404,"state":"active+clean","last_fresh":"2024-02-18T00:38:05.725945+0000","last_change":"2024-02-18T00:36:53.649744+0000","last_active":"2024-02-18T00:38:05.725945+0000","last_peered":"2024-02-18T00:38:05.725945+0000","last_clean":"2024-02-18T00:38:05.725945+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:38:05.725945+0000","last_undegraded":"2024-02-18T00:38:05.725945+0000","last_fullsized":"2024-02-18T00:38:05.725945+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'124","last_scrub_stamp":"2024-02-18T00:36:53.649677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.649677+0000","log_size":127,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":95,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":39,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"347'74","reported_seq":398,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.707182+0000","last_change":"2024-02-18T00:36:37.532389+0000","last_active":"2024-02-18T00:38:04.707182+0000","last_peered":"2024-02-18T00:38:04.707182+0000","last_clean":"2024-02-18T00:38:04.707182+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:38:04.707182+0000","last_undegraded":"2024-02-18T00:38:04.707182+0000","last_fullsized":"2024-02-18T00:38:04.707182+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'73","last_scrub_stamp":"2024-02-18T00:36:37.532316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.532316+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":23,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'70","reported_seq":370,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.708746+0000","last_change":"2024-02-18T00:37:44.685708+0000","last_active":"2024-02-18T00:38:04.708746+0000","last_peered":"2024-02-18T00:38:04.708746+0000","last_clean":"2024-02-18T00:38:04.708746+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:38:04.708746+0000","last_undegraded":"2024-02-18T00:38:04.708746+0000","last_fullsized":"2024-02-18T00:38:04.708746+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-18T00:37:44.685592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.685592+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"347'52","reported_seq":382,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698428+0000","last_change":"2024-02-18T00:36:09.412200+0000","last_active":"2024-02-18T00:38:04.698428+0000","last_peered":"2024-02-18T00:38:04.698428+0000","last_clean":"2024-02-18T00:38:04.698428+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:38:04.698428+0000","last_undegraded":"2024-02-18T00:38:04.698428+0000","last_fullsized":"2024-02-18T00:38:04.698428+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'51","last_scrub_stamp":"2024-02-18T00:36:09.412132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.412132+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'41","reported_seq":348,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699338+0000","last_change":"2024-02-18T00:36:06.648992+0000","last_active":"2024-02-18T00:38:04.699338+0000","last_peered":"2024-02-18T00:38:04.699338+0000","last_clean":"2024-02-18T00:38:04.699338+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:38:04.699338+0000","last_undegraded":"2024-02-18T00:38:04.699338+0000","last_fullsized":"2024-02-18T00:38:04.699338+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'39","last_scrub_stamp":"2024-02-18T00:36:00.739958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.739958+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"401'59","reported_seq":367,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705299+0000","last_change":"2024-02-18T00:36:11.810050+0000","last_active":"2024-02-18T00:38:04.705299+0000","last_peered":"2024-02-18T00:38:04.705299+0000","last_clean":"2024-02-18T00:38:04.705299+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:38:04.705299+0000","last_undegraded":"2024-02-18T00:38:04.705299+0000","last_fullsized":"2024-02-18T00:38:04.705299+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"246'54","last_scrub_stamp":"2024-02-18T00:36:11.809925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.809925+0000","log_size":59,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"379'18","reported_seq":351,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704574+0000","last_change":"2024-02-18T00:36:06.668857+0000","last_active":"2024-02-18T00:38:04.704574+0000","last_peered":"2024-02-18T00:38:04.704574+0000","last_clean":"2024-02-18T00:38:04.704574+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:38:04.704574+0000","last_undegraded":"2024-02-18T00:38:04.704574+0000","last_fullsized":"2024-02-18T00:38:04.704574+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:04.780547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.780547+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"347'29","reported_seq":386,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699081+0000","last_change":"2024-02-18T00:36:08.420415+0000","last_active":"2024-02-18T00:38:04.699081+0000","last_peered":"2024-02-18T00:38:04.699081+0000","last_clean":"2024-02-18T00:38:04.699081+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:38:04.699081+0000","last_undegraded":"2024-02-18T00:38:04.699081+0000","last_fullsized":"2024-02-18T00:38:04.699081+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'28","last_scrub_stamp":"2024-02-18T00:36:08.420343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.420343+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"347'83","reported_seq":405,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705679+0000","last_change":"2024-02-18T00:36:09.440443+0000","last_active":"2024-02-18T00:38:04.705679+0000","last_peered":"2024-02-18T00:38:04.705679+0000","last_clean":"2024-02-18T00:38:04.705679+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:38:04.705679+0000","last_undegraded":"2024-02-18T00:38:04.705679+0000","last_fullsized":"2024-02-18T00:38:04.705679+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'81","last_scrub_stamp":"2024-02-18T00:36:09.440371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.440371+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":67,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":386,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700695+0000","last_change":"2024-02-18T00:36:34.207908+0000","last_active":"2024-02-18T00:38:04.700695+0000","last_peered":"2024-02-18T00:38:04.700695+0000","last_clean":"2024-02-18T00:38:04.700695+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:38:04.700695+0000","last_undegraded":"2024-02-18T00:38:04.700695+0000","last_fullsized":"2024-02-18T00:38:04.700695+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:36:34.207840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.207840+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'57","reported_seq":404,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.708720+0000","last_change":"2024-02-18T00:37:36.817791+0000","last_active":"2024-02-18T00:38:04.708720+0000","last_peered":"2024-02-18T00:38:04.708720+0000","last_clean":"2024-02-18T00:38:04.708720+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:38:04.708720+0000","last_undegraded":"2024-02-18T00:38:04.708720+0000","last_fullsized":"2024-02-18T00:38:04.708720+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'57","last_scrub_stamp":"2024-02-18T00:37:36.817719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.817719+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"398'21","reported_seq":363,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709818+0000","last_change":"2024-02-18T00:37:31.817355+0000","last_active":"2024-02-18T00:38:04.709818+0000","last_peered":"2024-02-18T00:38:04.709818+0000","last_clean":"2024-02-18T00:38:04.709818+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:38:04.709818+0000","last_undegraded":"2024-02-18T00:38:04.709818+0000","last_fullsized":"2024-02-18T00:38:04.709818+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:37:31.817281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.817281+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"347'74","reported_seq":400,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699424+0000","last_change":"2024-02-18T00:36:06.758841+0000","last_active":"2024-02-18T00:38:04.699424+0000","last_peered":"2024-02-18T00:38:04.699424+0000","last_clean":"2024-02-18T00:38:04.699424+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:38:04.699424+0000","last_undegraded":"2024-02-18T00:38:04.699424+0000","last_fullsized":"2024-02-18T00:38:04.699424+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"288'72","last_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.758769+0000","log_size":74,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"361'18","reported_seq":353,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700975+0000","last_change":"2024-02-18T00:36:43.345182+0000","last_active":"2024-02-18T00:38:04.700975+0000","last_peered":"2024-02-18T00:38:04.700975+0000","last_clean":"2024-02-18T00:38:04.700975+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:38:04.700975+0000","last_undegraded":"2024-02-18T00:38:04.700975+0000","last_fullsized":"2024-02-18T00:38:04.700975+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'17","last_scrub_stamp":"2024-02-18T00:36:43.345064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.345064+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":365,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700386+0000","last_change":"2024-02-18T00:36:06.652733+0000","last_active":"2024-02-18T00:38:04.700386+0000","last_peered":"2024-02-18T00:38:04.700386+0000","last_clean":"2024-02-18T00:38:04.700386+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:38:04.700386+0000","last_undegraded":"2024-02-18T00:38:04.700386+0000","last_fullsized":"2024-02-18T00:38:04.700386+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:36:06.486028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.486028+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"379'21","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702492+0000","last_change":"2024-02-18T00:37:41.575899+0000","last_active":"2024-02-18T00:38:04.702492+0000","last_peered":"2024-02-18T00:38:04.702492+0000","last_clean":"2024-02-18T00:38:04.702492+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:38:04.702492+0000","last_undegraded":"2024-02-18T00:38:04.702492+0000","last_fullsized":"2024-02-18T00:38:04.702492+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'21","last_scrub_stamp":"2024-02-18T00:37:41.575814+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.575814+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000035,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":3914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":427,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703345+0000","last_change":"2024-02-18T00:36:34.017135+0000","last_active":"2024-02-18T00:38:04.703345+0000","last_peered":"2024-02-18T00:38:04.703345+0000","last_clean":"2024-02-18T00:38:04.703345+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:38:04.703345+0000","last_undegraded":"2024-02-18T00:38:04.703345+0000","last_fullsized":"2024-02-18T00:38:04.703345+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:36:34.017065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.017065+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":320,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702523+0000","last_change":"2024-02-18T00:36:37.504644+0000","last_active":"2024-02-18T00:38:04.702523+0000","last_peered":"2024-02-18T00:38:04.702523+0000","last_clean":"2024-02-18T00:38:04.702523+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:38:04.702523+0000","last_undegraded":"2024-02-18T00:38:04.702523+0000","last_fullsized":"2024-02-18T00:38:04.702523+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:37.504568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.504568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"265'112","reported_seq":442,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698300+0000","last_change":"2024-02-18T00:36:38.583124+0000","last_active":"2024-02-18T00:38:04.698300+0000","last_peered":"2024-02-18T00:38:04.698300+0000","last_clean":"2024-02-18T00:38:04.698300+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:38:04.698300+0000","last_undegraded":"2024-02-18T00:38:04.698300+0000","last_fullsized":"2024-02-18T00:38:04.698300+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'112","last_scrub_stamp":"2024-02-18T00:36:38.583056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.583056+0000","log_size":112,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":76,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":318,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.711344+0000","last_change":"2024-02-18T00:37:01.774863+0000","last_active":"2024-02-18T00:38:04.711344+0000","last_peered":"2024-02-18T00:38:04.711344+0000","last_clean":"2024-02-18T00:38:04.711344+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:38:04.711344+0000","last_undegraded":"2024-02-18T00:38:04.711344+0000","last_fullsized":"2024-02-18T00:38:04.711344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.774795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.774795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":377,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698633+0000","last_change":"2024-02-18T00:36:06.651042+0000","last_active":"2024-02-18T00:38:04.698633+0000","last_peered":"2024-02-18T00:38:04.698633+0000","last_clean":"2024-02-18T00:38:04.698633+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:38:04.698633+0000","last_undegraded":"2024-02-18T00:38:04.698633+0000","last_fullsized":"2024-02-18T00:38:04.698633+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:35:53.400439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.400439+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":279,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700785+0000","last_change":"2024-02-18T00:37:27.130328+0000","last_active":"2024-02-18T00:38:04.700785+0000","last_peered":"2024-02-18T00:38:04.700785+0000","last_clean":"2024-02-18T00:38:04.700785+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:38:04.700785+0000","last_undegraded":"2024-02-18T00:38:04.700785+0000","last_fullsized":"2024-02-18T00:38:04.700785+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.130252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.130252+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":429,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702444+0000","last_change":"2024-02-18T00:36:06.657946+0000","last_active":"2024-02-18T00:38:04.702444+0000","last_peered":"2024-02-18T00:38:04.702444+0000","last_clean":"2024-02-18T00:38:04.702444+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:38:04.702444+0000","last_undegraded":"2024-02-18T00:38:04.702444+0000","last_fullsized":"2024-02-18T00:38:04.702444+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:35:57.471698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.471698+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":277,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700519+0000","last_change":"2024-02-18T00:36:53.345938+0000","last_active":"2024-02-18T00:38:04.700519+0000","last_peered":"2024-02-18T00:38:04.700519+0000","last_clean":"2024-02-18T00:38:04.700519+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:38:04.700519+0000","last_undegraded":"2024-02-18T00:38:04.700519+0000","last_fullsized":"2024-02-18T00:38:04.700519+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.345872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":406,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702437+0000","last_change":"2024-02-18T00:36:06.657455+0000","last_active":"2024-02-18T00:38:04.702437+0000","last_peered":"2024-02-18T00:38:04.702437+0000","last_clean":"2024-02-18T00:38:04.702437+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:38:04.702437+0000","last_undegraded":"2024-02-18T00:38:04.702437+0000","last_fullsized":"2024-02-18T00:38:04.702437+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:35:55.396078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.396078+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":329,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700057+0000","last_change":"2024-02-18T00:36:39.168150+0000","last_active":"2024-02-18T00:38:04.700057+0000","last_peered":"2024-02-18T00:38:04.700057+0000","last_clean":"2024-02-18T00:38:04.700057+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:38:04.700057+0000","last_undegraded":"2024-02-18T00:38:04.700057+0000","last_fullsized":"2024-02-18T00:38:04.700057+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:36:39.168079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:39.168079+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"347'18","reported_seq":355,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710719+0000","last_change":"2024-02-18T00:36:10.770717+0000","last_active":"2024-02-18T00:38:04.710719+0000","last_peered":"2024-02-18T00:38:04.710719+0000","last_clean":"2024-02-18T00:38:04.710719+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:38:04.710719+0000","last_undegraded":"2024-02-18T00:38:04.710719+0000","last_fullsized":"2024-02-18T00:38:04.710719+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'17","last_scrub_stamp":"2024-02-18T00:36:10.770640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.770640+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":320,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703050+0000","last_change":"2024-02-18T00:36:35.460784+0000","last_active":"2024-02-18T00:38:04.703050+0000","last_peered":"2024-02-18T00:38:04.703050+0000","last_clean":"2024-02-18T00:38:04.703050+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:38:04.703050+0000","last_undegraded":"2024-02-18T00:38:04.703050+0000","last_fullsized":"2024-02-18T00:38:04.703050+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.460703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.460703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":387,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706952+0000","last_change":"2024-02-18T00:36:41.624250+0000","last_active":"2024-02-18T00:38:04.706952+0000","last_peered":"2024-02-18T00:38:04.706952+0000","last_clean":"2024-02-18T00:38:04.706952+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:38:04.706952+0000","last_undegraded":"2024-02-18T00:38:04.706952+0000","last_fullsized":"2024-02-18T00:38:04.706952+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:36:41.624180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.624180+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":309,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705417+0000","last_change":"2024-02-18T00:36:43.933486+0000","last_active":"2024-02-18T00:38:04.705417+0000","last_peered":"2024-02-18T00:38:04.705417+0000","last_clean":"2024-02-18T00:38:04.705417+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:38:04.705417+0000","last_undegraded":"2024-02-18T00:38:04.705417+0000","last_fullsized":"2024-02-18T00:38:04.705417+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.933414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.933414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":345,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703935+0000","last_change":"2024-02-18T00:37:37.618760+0000","last_active":"2024-02-18T00:38:04.703935+0000","last_peered":"2024-02-18T00:38:04.703935+0000","last_clean":"2024-02-18T00:38:04.703935+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:38:04.703935+0000","last_undegraded":"2024-02-18T00:38:04.703935+0000","last_fullsized":"2024-02-18T00:38:04.703935+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:37:37.618667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.618667+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":313,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702924+0000","last_change":"2024-02-18T00:36:07.532848+0000","last_active":"2024-02-18T00:38:04.702924+0000","last_peered":"2024-02-18T00:38:04.702924+0000","last_clean":"2024-02-18T00:38:04.702924+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:38:04.702924+0000","last_undegraded":"2024-02-18T00:38:04.702924+0000","last_fullsized":"2024-02-18T00:38:04.702924+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.532776+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"379'18","reported_seq":352,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648142+0000","last_change":"2024-02-18T00:36:06.724820+0000","last_active":"2024-02-18T00:38:04.648142+0000","last_peered":"2024-02-18T00:38:04.648142+0000","last_clean":"2024-02-18T00:38:04.648142+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:38:04.648142+0000","last_undegraded":"2024-02-18T00:38:04.648142+0000","last_fullsized":"2024-02-18T00:38:04.648142+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"199'17","last_scrub_stamp":"2024-02-18T00:36:01.007406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.007406+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":393,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700554+0000","last_change":"2024-02-18T00:36:45.627771+0000","last_active":"2024-02-18T00:38:04.700554+0000","last_peered":"2024-02-18T00:38:04.700554+0000","last_clean":"2024-02-18T00:38:04.700554+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:38:04.700554+0000","last_undegraded":"2024-02-18T00:38:04.700554+0000","last_fullsized":"2024-02-18T00:38:04.700554+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.627700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.627700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":356,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702937+0000","last_change":"2024-02-18T00:37:38.588259+0000","last_active":"2024-02-18T00:38:04.702937+0000","last_peered":"2024-02-18T00:38:04.702937+0000","last_clean":"2024-02-18T00:38:04.702937+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:38:04.702937+0000","last_undegraded":"2024-02-18T00:38:04.702937+0000","last_fullsized":"2024-02-18T00:38:04.702937+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'18","last_scrub_stamp":"2024-02-18T00:37:38.588190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.588190+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":343,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.644615+0000","last_change":"2024-02-18T00:37:37.726927+0000","last_active":"2024-02-18T00:38:04.644615+0000","last_peered":"2024-02-18T00:38:04.644615+0000","last_clean":"2024-02-18T00:38:04.644615+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:38:04.644615+0000","last_undegraded":"2024-02-18T00:38:04.644615+0000","last_fullsized":"2024-02-18T00:38:04.644615+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.726861+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":318,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703420+0000","last_change":"2024-02-18T00:36:08.457508+0000","last_active":"2024-02-18T00:38:04.703420+0000","last_peered":"2024-02-18T00:38:04.703420+0000","last_clean":"2024-02-18T00:38:04.703420+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:38:04.703420+0000","last_undegraded":"2024-02-18T00:38:04.703420+0000","last_fullsized":"2024-02-18T00:38:04.703420+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.457426+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.457426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":348,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703109+0000","last_change":"2024-02-18T00:37:33.619485+0000","last_active":"2024-02-18T00:38:04.703109+0000","last_peered":"2024-02-18T00:38:04.703109+0000","last_clean":"2024-02-18T00:38:04.703109+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:38:04.703109+0000","last_undegraded":"2024-02-18T00:38:04.703109+0000","last_fullsized":"2024-02-18T00:38:04.703109+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:33.619410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.619410+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":448,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.644435+0000","last_change":"2024-02-18T00:35:16.772079+0000","last_active":"2024-02-18T00:38:04.644435+0000","last_peered":"2024-02-18T00:38:04.644435+0000","last_clean":"2024-02-18T00:38:04.644435+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:38:04.644435+0000","last_undegraded":"2024-02-18T00:38:04.644435+0000","last_fullsized":"2024-02-18T00:38:04.644435+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:35:16.772003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:16.772003+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":386,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698786+0000","last_change":"2024-02-18T00:37:02.627426+0000","last_active":"2024-02-18T00:38:04.698786+0000","last_peered":"2024-02-18T00:38:04.698786+0000","last_clean":"2024-02-18T00:38:04.698786+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:38:04.698786+0000","last_undegraded":"2024-02-18T00:38:04.698786+0000","last_fullsized":"2024-02-18T00:38:04.698786+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.627349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.627349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":365,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646437+0000","last_change":"2024-02-18T00:37:33.616952+0000","last_active":"2024-02-18T00:38:04.646437+0000","last_peered":"2024-02-18T00:38:04.646437+0000","last_clean":"2024-02-18T00:38:04.646437+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:38:04.646437+0000","last_undegraded":"2024-02-18T00:38:04.646437+0000","last_fullsized":"2024-02-18T00:38:04.646437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.616874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.616874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":315,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709642+0000","last_change":"2024-02-18T00:37:01.919166+0000","last_active":"2024-02-18T00:38:04.709642+0000","last_peered":"2024-02-18T00:38:04.709642+0000","last_clean":"2024-02-18T00:38:04.709642+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:38:04.709642+0000","last_undegraded":"2024-02-18T00:38:04.709642+0000","last_fullsized":"2024-02-18T00:38:04.709642+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.919082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.919082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":423,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701509+0000","last_change":"2024-02-18T00:37:49.523198+0000","last_active":"2024-02-18T00:38:04.701509+0000","last_peered":"2024-02-18T00:38:04.701509+0000","last_clean":"2024-02-18T00:38:04.701509+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:38:04.701509+0000","last_undegraded":"2024-02-18T00:38:04.701509+0000","last_fullsized":"2024-02-18T00:38:04.701509+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:49.523121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:49.523121+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":375,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709897+0000","last_change":"2024-02-18T00:36:43.837775+0000","last_active":"2024-02-18T00:38:04.709897+0000","last_peered":"2024-02-18T00:38:04.709897+0000","last_clean":"2024-02-18T00:38:04.709897+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:38:04.709897+0000","last_undegraded":"2024-02-18T00:38:04.709897+0000","last_fullsized":"2024-02-18T00:38:04.709897+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.837702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.837702+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699565+0000","last_change":"2024-02-18T00:36:56.601833+0000","last_active":"2024-02-18T00:38:04.699565+0000","last_peered":"2024-02-18T00:38:04.699565+0000","last_clean":"2024-02-18T00:38:04.699565+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:38:04.699565+0000","last_undegraded":"2024-02-18T00:38:04.699565+0000","last_fullsized":"2024-02-18T00:38:04.699565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.601727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.601727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":385,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701592+0000","last_change":"2024-02-18T00:37:47.488889+0000","last_active":"2024-02-18T00:38:04.701592+0000","last_peered":"2024-02-18T00:38:04.701592+0000","last_clean":"2024-02-18T00:38:04.701592+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:38:04.701592+0000","last_undegraded":"2024-02-18T00:38:04.701592+0000","last_fullsized":"2024-02-18T00:38:04.701592+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.488821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.488821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":315,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706078+0000","last_change":"2024-02-18T00:36:03.782871+0000","last_active":"2024-02-18T00:38:04.706078+0000","last_peered":"2024-02-18T00:38:04.706078+0000","last_clean":"2024-02-18T00:38:04.706078+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:38:04.706078+0000","last_undegraded":"2024-02-18T00:38:04.706078+0000","last_fullsized":"2024-02-18T00:38:04.706078+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:03.782800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:03.782800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":449,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701350+0000","last_change":"2024-02-18T00:35:15.472630+0000","last_active":"2024-02-18T00:38:04.701350+0000","last_peered":"2024-02-18T00:38:04.701350+0000","last_clean":"2024-02-18T00:38:04.701350+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:38:04.701350+0000","last_undegraded":"2024-02-18T00:38:04.701350+0000","last_fullsized":"2024-02-18T00:38:04.701350+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:35:15.472544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:15.472544+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":375,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700621+0000","last_change":"2024-02-18T00:36:49.680480+0000","last_active":"2024-02-18T00:38:04.700621+0000","last_peered":"2024-02-18T00:38:04.700621+0000","last_clean":"2024-02-18T00:38:04.700621+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:38:04.700621+0000","last_undegraded":"2024-02-18T00:38:04.700621+0000","last_fullsized":"2024-02-18T00:38:04.700621+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.680414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700807+0000","last_change":"2024-02-18T00:37:00.670726+0000","last_active":"2024-02-18T00:38:04.700807+0000","last_peered":"2024-02-18T00:38:04.700807+0000","last_clean":"2024-02-18T00:38:04.700807+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:38:04.700807+0000","last_undegraded":"2024-02-18T00:38:04.700807+0000","last_fullsized":"2024-02-18T00:38:04.700807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.670646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.670646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":275,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698992+0000","last_change":"2024-02-18T00:37:37.476740+0000","last_active":"2024-02-18T00:38:04.698992+0000","last_peered":"2024-02-18T00:38:04.698992+0000","last_clean":"2024-02-18T00:38:04.698992+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:38:04.698992+0000","last_undegraded":"2024-02-18T00:38:04.698992+0000","last_fullsized":"2024-02-18T00:38:04.698992+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.476676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.476676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":306,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646102+0000","last_change":"2024-02-18T00:35:54.836804+0000","last_active":"2024-02-18T00:38:04.646102+0000","last_peered":"2024-02-18T00:38:04.646102+0000","last_clean":"2024-02-18T00:38:04.646102+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:38:04.646102+0000","last_undegraded":"2024-02-18T00:38:04.646102+0000","last_fullsized":"2024-02-18T00:38:04.646102+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.836737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.836737+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":402,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699920+0000","last_change":"2024-02-18T00:35:27.234543+0000","last_active":"2024-02-18T00:38:04.699920+0000","last_peered":"2024-02-18T00:38:04.699920+0000","last_clean":"2024-02-18T00:38:04.699920+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:38:04.699920+0000","last_undegraded":"2024-02-18T00:38:04.699920+0000","last_fullsized":"2024-02-18T00:38:04.699920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:35:27.234463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.234463+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":276,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698889+0000","last_change":"2024-02-18T00:36:58.282770+0000","last_active":"2024-02-18T00:38:04.698889+0000","last_peered":"2024-02-18T00:38:04.698889+0000","last_clean":"2024-02-18T00:38:04.698889+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:38:04.698889+0000","last_undegraded":"2024-02-18T00:38:04.698889+0000","last_fullsized":"2024-02-18T00:38:04.698889+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.282673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.282673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.711082+0000","last_change":"2024-02-18T00:36:59.000753+0000","last_active":"2024-02-18T00:38:04.711082+0000","last_peered":"2024-02-18T00:38:04.711082+0000","last_clean":"2024-02-18T00:38:04.711082+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:38:04.711082+0000","last_undegraded":"2024-02-18T00:38:04.711082+0000","last_fullsized":"2024-02-18T00:38:04.711082+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.000690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.000690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":369,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.707399+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:38:04.707399+0000","last_peered":"2024-02-18T00:38:04.707399+0000","last_clean":"2024-02-18T00:38:04.707399+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:38:04.707399+0000","last_undegraded":"2024-02-18T00:38:04.707399+0000","last_fullsized":"2024-02-18T00:38:04.707399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":154,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698669+0000","last_change":"2024-02-18T00:35:54.664943+0000","last_active":"2024-02-18T00:38:04.698669+0000","last_peered":"2024-02-18T00:38:04.698669+0000","last_clean":"2024-02-18T00:38:04.698669+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:38:04.698669+0000","last_undegraded":"2024-02-18T00:38:04.698669+0000","last_fullsized":"2024-02-18T00:38:04.698669+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.664879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.664879+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":389,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698894+0000","last_change":"2024-02-18T00:37:29.177448+0000","last_active":"2024-02-18T00:38:04.698894+0000","last_peered":"2024-02-18T00:38:04.698894+0000","last_clean":"2024-02-18T00:38:04.698894+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:38:04.698894+0000","last_undegraded":"2024-02-18T00:38:04.698894+0000","last_fullsized":"2024-02-18T00:38:04.698894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:37:29.177369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.177369+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":384,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709734+0000","last_change":"2024-02-18T00:36:38.883486+0000","last_active":"2024-02-18T00:38:04.709734+0000","last_peered":"2024-02-18T00:38:04.709734+0000","last_clean":"2024-02-18T00:38:04.709734+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:38:04.709734+0000","last_undegraded":"2024-02-18T00:38:04.709734+0000","last_fullsized":"2024-02-18T00:38:04.709734+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:38.883391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.883391+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":501,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700056+0000","last_change":"2024-02-18T00:37:38.433946+0000","last_active":"2024-02-18T00:38:04.700056+0000","last_peered":"2024-02-18T00:38:04.700056+0000","last_clean":"2024-02-18T00:38:04.700056+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:38:04.700056+0000","last_undegraded":"2024-02-18T00:38:04.700056+0000","last_fullsized":"2024-02-18T00:38:04.700056+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:38.433877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.433877+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":370,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700905+0000","last_change":"2024-02-18T00:37:31.127432+0000","last_active":"2024-02-18T00:38:04.700905+0000","last_peered":"2024-02-18T00:38:04.700905+0000","last_clean":"2024-02-18T00:38:04.700905+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:38:04.700905+0000","last_undegraded":"2024-02-18T00:38:04.700905+0000","last_fullsized":"2024-02-18T00:38:04.700905+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.127341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.127341+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":315,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710063+0000","last_change":"2024-02-18T00:37:00.950276+0000","last_active":"2024-02-18T00:38:04.710063+0000","last_peered":"2024-02-18T00:38:04.710063+0000","last_clean":"2024-02-18T00:38:04.710063+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:38:04.710063+0000","last_undegraded":"2024-02-18T00:38:04.710063+0000","last_fullsized":"2024-02-18T00:38:04.710063+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.950130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.950130+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":372,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648143+0000","last_change":"2024-02-18T00:36:33.853442+0000","last_active":"2024-02-18T00:38:04.648143+0000","last_peered":"2024-02-18T00:38:04.648143+0000","last_clean":"2024-02-18T00:38:04.648143+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:38:04.648143+0000","last_undegraded":"2024-02-18T00:38:04.648143+0000","last_fullsized":"2024-02-18T00:38:04.648143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:36:33.853337+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:36:33.853337+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":398,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710272+0000","last_change":"2024-02-18T00:37:41.747387+0000","last_active":"2024-02-18T00:38:04.710272+0000","last_peered":"2024-02-18T00:38:04.710272+0000","last_clean":"2024-02-18T00:38:04.710272+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:38:04.710272+0000","last_undegraded":"2024-02-18T00:38:04.710272+0000","last_fullsized":"2024-02-18T00:38:04.710272+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.747293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.747293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":486,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700829+0000","last_change":"2024-02-18T00:37:43.161256+0000","last_active":"2024-02-18T00:38:04.700829+0000","last_peered":"2024-02-18T00:38:04.700829+0000","last_clean":"2024-02-18T00:38:04.700829+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:38:04.700829+0000","last_undegraded":"2024-02-18T00:38:04.700829+0000","last_fullsized":"2024-02-18T00:38:04.700829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:43.161186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.161186+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":360,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648670+0000","last_change":"2024-02-18T00:36:08.054282+0000","last_active":"2024-02-18T00:38:04.648670+0000","last_peered":"2024-02-18T00:38:04.648670+0000","last_clean":"2024-02-18T00:38:04.648670+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:38:04.648670+0000","last_undegraded":"2024-02-18T00:38:04.648670+0000","last_fullsized":"2024-02-18T00:38:04.648670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:36:08.054205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":311,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698735+0000","last_change":"2024-02-18T00:36:00.408919+0000","last_active":"2024-02-18T00:38:04.698735+0000","last_peered":"2024-02-18T00:38:04.698735+0000","last_clean":"2024-02-18T00:38:04.698735+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:38:04.698735+0000","last_undegraded":"2024-02-18T00:38:04.698735+0000","last_fullsized":"2024-02-18T00:38:04.698735+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.408858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.408858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":431,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700611+0000","last_change":"2024-02-18T00:36:43.283359+0000","last_active":"2024-02-18T00:38:04.700611+0000","last_peered":"2024-02-18T00:38:04.700611+0000","last_clean":"2024-02-18T00:38:04.700611+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:38:04.700611+0000","last_undegraded":"2024-02-18T00:38:04.700611+0000","last_fullsized":"2024-02-18T00:38:04.700611+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:43.283235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.283235+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":345,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700030+0000","last_change":"2024-02-18T00:36:57.321994+0000","last_active":"2024-02-18T00:38:04.700030+0000","last_peered":"2024-02-18T00:38:04.700030+0000","last_clean":"2024-02-18T00:38:04.700030+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:38:04.700030+0000","last_undegraded":"2024-02-18T00:38:04.700030+0000","last_fullsized":"2024-02-18T00:38:04.700030+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.321930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.321930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":367,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648539+0000","last_change":"2024-02-18T00:37:48.527852+0000","last_active":"2024-02-18T00:38:04.648539+0000","last_peered":"2024-02-18T00:38:04.648539+0000","last_clean":"2024-02-18T00:38:04.648539+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:38:04.648539+0000","last_undegraded":"2024-02-18T00:38:04.648539+0000","last_fullsized":"2024-02-18T00:38:04.648539+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:48.527779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.527779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":365,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704789+0000","last_change":"2024-02-18T00:37:30.849071+0000","last_active":"2024-02-18T00:38:04.704789+0000","last_peered":"2024-02-18T00:38:04.704789+0000","last_clean":"2024-02-18T00:38:04.704789+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:38:04.704789+0000","last_undegraded":"2024-02-18T00:38:04.704789+0000","last_fullsized":"2024-02-18T00:38:04.704789+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:30.848966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:30.848966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":316,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699155+0000","last_change":"2024-02-18T00:37:44.200088+0000","last_active":"2024-02-18T00:38:04.699155+0000","last_peered":"2024-02-18T00:38:04.699155+0000","last_clean":"2024-02-18T00:38:04.699155+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:38:04.699155+0000","last_undegraded":"2024-02-18T00:38:04.699155+0000","last_fullsized":"2024-02-18T00:38:04.699155+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.199947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":482,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.711883+0000","last_change":"2024-02-18T00:36:53.910404+0000","last_active":"2024-02-18T00:38:04.711883+0000","last_peered":"2024-02-18T00:38:04.711883+0000","last_clean":"2024-02-18T00:38:04.711883+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:38:04.711883+0000","last_undegraded":"2024-02-18T00:38:04.711883+0000","last_fullsized":"2024-02-18T00:38:04.711883+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.910333+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":410,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698699+0000","last_change":"2024-02-18T00:36:50.262681+0000","last_active":"2024-02-18T00:38:04.698699+0000","last_peered":"2024-02-18T00:38:04.698699+0000","last_clean":"2024-02-18T00:38:04.698699+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:38:04.698699+0000","last_undegraded":"2024-02-18T00:38:04.698699+0000","last_fullsized":"2024-02-18T00:38:04.698699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:36:50.262602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.262602+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":391,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702616+0000","last_change":"2024-02-18T00:36:01.449526+0000","last_active":"2024-02-18T00:38:04.702616+0000","last_peered":"2024-02-18T00:38:04.702616+0000","last_clean":"2024-02-18T00:38:04.702616+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:38:04.702616+0000","last_undegraded":"2024-02-18T00:38:04.702616+0000","last_fullsized":"2024-02-18T00:38:04.702616+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.449418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.449418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":273,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703720+0000","last_change":"2024-02-18T00:37:27.733729+0000","last_active":"2024-02-18T00:38:04.703720+0000","last_peered":"2024-02-18T00:38:04.703720+0000","last_clean":"2024-02-18T00:38:04.703720+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:38:04.703720+0000","last_undegraded":"2024-02-18T00:38:04.703720+0000","last_fullsized":"2024-02-18T00:38:04.703720+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.733609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.733609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":306,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648230+0000","last_change":"2024-02-18T00:35:55.889369+0000","last_active":"2024-02-18T00:38:04.648230+0000","last_peered":"2024-02-18T00:38:04.648230+0000","last_clean":"2024-02-18T00:38:04.648230+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:38:04.648230+0000","last_undegraded":"2024-02-18T00:38:04.648230+0000","last_fullsized":"2024-02-18T00:38:04.648230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.889249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.889249+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":381,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700034+0000","last_change":"2024-02-18T00:36:45.308400+0000","last_active":"2024-02-18T00:38:04.700034+0000","last_peered":"2024-02-18T00:38:04.700034+0000","last_clean":"2024-02-18T00:38:04.700034+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:38:04.700034+0000","last_undegraded":"2024-02-18T00:38:04.700034+0000","last_fullsized":"2024-02-18T00:38:04.700034+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.308300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.308300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":472,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.711760+0000","last_change":"2024-02-18T00:37:35.841167+0000","last_active":"2024-02-18T00:38:04.711760+0000","last_peered":"2024-02-18T00:38:04.711760+0000","last_clean":"2024-02-18T00:38:04.711760+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:38:04.711760+0000","last_undegraded":"2024-02-18T00:38:04.711760+0000","last_fullsized":"2024-02-18T00:38:04.711760+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:37:35.841050+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.841050+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":389,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709732+0000","last_change":"2024-02-18T00:37:32.766303+0000","last_active":"2024-02-18T00:38:04.709732+0000","last_peered":"2024-02-18T00:38:04.709732+0000","last_clean":"2024-02-18T00:38:04.709732+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:38:04.709732+0000","last_undegraded":"2024-02-18T00:38:04.709732+0000","last_fullsized":"2024-02-18T00:38:04.709732+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.766203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.766203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":370,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699172+0000","last_change":"2024-02-18T00:37:33.117248+0000","last_active":"2024-02-18T00:38:04.699172+0000","last_peered":"2024-02-18T00:38:04.699172+0000","last_clean":"2024-02-18T00:38:04.699172+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:38:04.699172+0000","last_undegraded":"2024-02-18T00:38:04.699172+0000","last_fullsized":"2024-02-18T00:38:04.699172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.117173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.117173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":306,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.713076+0000","last_change":"2024-02-18T00:35:54.736925+0000","last_active":"2024-02-18T00:38:04.713076+0000","last_peered":"2024-02-18T00:38:04.713076+0000","last_clean":"2024-02-18T00:38:04.713076+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:38:04.713076+0000","last_undegraded":"2024-02-18T00:38:04.713076+0000","last_fullsized":"2024-02-18T00:38:04.713076+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:54.736823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:54.736823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":369,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702747+0000","last_change":"2024-02-18T00:37:31.672777+0000","last_active":"2024-02-18T00:38:04.702747+0000","last_peered":"2024-02-18T00:38:04.702747+0000","last_clean":"2024-02-18T00:38:04.702747+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:38:04.702747+0000","last_undegraded":"2024-02-18T00:38:04.702747+0000","last_fullsized":"2024-02-18T00:38:04.702747+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.672699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.672699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":451,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701344+0000","last_change":"2024-02-18T00:37:37.281742+0000","last_active":"2024-02-18T00:38:04.701344+0000","last_peered":"2024-02-18T00:38:04.701344+0000","last_clean":"2024-02-18T00:38:04.701344+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:38:04.701344+0000","last_undegraded":"2024-02-18T00:38:04.701344+0000","last_fullsized":"2024-02-18T00:38:04.701344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:37:37.281674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.281674+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":282,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703684+0000","last_change":"2024-02-18T00:37:01.250860+0000","last_active":"2024-02-18T00:38:04.703684+0000","last_peered":"2024-02-18T00:38:04.703684+0000","last_clean":"2024-02-18T00:38:04.703684+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:38:04.703684+0000","last_undegraded":"2024-02-18T00:38:04.703684+0000","last_fullsized":"2024-02-18T00:38:04.703684+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:37:01.250771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.250771+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":369,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710364+0000","last_change":"2024-02-18T00:36:59.979666+0000","last_active":"2024-02-18T00:38:04.710364+0000","last_peered":"2024-02-18T00:38:04.710364+0000","last_clean":"2024-02-18T00:38:04.710364+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:38:04.710364+0000","last_undegraded":"2024-02-18T00:38:04.710364+0000","last_fullsized":"2024-02-18T00:38:04.710364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.979565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.979565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":306,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706327+0000","last_change":"2024-02-18T00:36:06.726081+0000","last_active":"2024-02-18T00:38:04.706327+0000","last_peered":"2024-02-18T00:38:04.706327+0000","last_clean":"2024-02-18T00:38:04.706327+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:38:04.706327+0000","last_undegraded":"2024-02-18T00:38:04.706327+0000","last_fullsized":"2024-02-18T00:38:04.706327+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:06.725972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:06.725972+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":367,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648702+0000","last_change":"2024-02-18T00:37:47.574086+0000","last_active":"2024-02-18T00:38:04.648702+0000","last_peered":"2024-02-18T00:38:04.648702+0000","last_clean":"2024-02-18T00:38:04.648702+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:38:04.648702+0000","last_undegraded":"2024-02-18T00:38:04.648702+0000","last_fullsized":"2024-02-18T00:38:04.648702+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.574003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.574003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":389,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706061+0000","last_change":"2024-02-18T00:37:35.212798+0000","last_active":"2024-02-18T00:38:04.706061+0000","last_peered":"2024-02-18T00:38:04.706061+0000","last_clean":"2024-02-18T00:38:04.706061+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:38:04.706061+0000","last_undegraded":"2024-02-18T00:38:04.706061+0000","last_fullsized":"2024-02-18T00:38:04.706061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:37:35.212727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.212727+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":346,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709089+0000","last_change":"2024-02-18T00:36:34.800419+0000","last_active":"2024-02-18T00:38:04.709089+0000","last_peered":"2024-02-18T00:38:04.709089+0000","last_clean":"2024-02-18T00:38:04.709089+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:38:04.709089+0000","last_undegraded":"2024-02-18T00:38:04.709089+0000","last_fullsized":"2024-02-18T00:38:04.709089+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:34.800327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.800327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":372,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701324+0000","last_change":"2024-02-18T00:37:26.204990+0000","last_active":"2024-02-18T00:38:04.701324+0000","last_peered":"2024-02-18T00:38:04.701324+0000","last_clean":"2024-02-18T00:38:04.701324+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:38:04.701324+0000","last_undegraded":"2024-02-18T00:38:04.701324+0000","last_fullsized":"2024-02-18T00:38:04.701324+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:26.204918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:37:26.204918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":369,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.711171+0000","last_change":"2024-02-18T00:37:25.868179+0000","last_active":"2024-02-18T00:38:04.711171+0000","last_peered":"2024-02-18T00:38:04.711171+0000","last_clean":"2024-02-18T00:38:04.711171+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:38:04.711171+0000","last_undegraded":"2024-02-18T00:38:04.711171+0000","last_fullsized":"2024-02-18T00:38:04.711171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:25.868065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:25.868065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":318,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700215+0000","last_change":"2024-02-18T00:35:55.237987+0000","last_active":"2024-02-18T00:38:04.700215+0000","last_peered":"2024-02-18T00:38:04.700215+0000","last_clean":"2024-02-18T00:38:04.700215+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:38:04.700215+0000","last_undegraded":"2024-02-18T00:38:04.700215+0000","last_fullsized":"2024-02-18T00:38:04.700215+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.237921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.237921+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":275,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700427+0000","last_change":"2024-02-18T00:37:32.401689+0000","last_active":"2024-02-18T00:38:04.700427+0000","last_peered":"2024-02-18T00:38:04.700427+0000","last_clean":"2024-02-18T00:38:04.700427+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:38:04.700427+0000","last_undegraded":"2024-02-18T00:38:04.700427+0000","last_fullsized":"2024-02-18T00:38:04.700427+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.401618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.401618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":423,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704050+0000","last_change":"2024-02-18T00:36:50.759133+0000","last_active":"2024-02-18T00:38:04.704050+0000","last_peered":"2024-02-18T00:38:04.704050+0000","last_clean":"2024-02-18T00:38:04.704050+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:38:04.704050+0000","last_undegraded":"2024-02-18T00:38:04.704050+0000","last_fullsized":"2024-02-18T00:38:04.704050+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:36:50.759019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.759019+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":308,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700110+0000","last_change":"2024-02-18T00:37:42.354830+0000","last_active":"2024-02-18T00:38:04.700110+0000","last_peered":"2024-02-18T00:38:04.700110+0000","last_clean":"2024-02-18T00:38:04.700110+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:38:04.700110+0000","last_undegraded":"2024-02-18T00:38:04.700110+0000","last_fullsized":"2024-02-18T00:38:04.700110+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.354758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.354758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":389,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.647609+0000","last_change":"2024-02-18T00:37:00.820469+0000","last_active":"2024-02-18T00:38:04.647609+0000","last_peered":"2024-02-18T00:38:04.647609+0000","last_clean":"2024-02-18T00:38:04.647609+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:38:04.647609+0000","last_undegraded":"2024-02-18T00:38:04.647609+0000","last_fullsized":"2024-02-18T00:38:04.647609+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.820283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.820283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":365,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701993+0000","last_change":"2024-02-18T00:36:00.758971+0000","last_active":"2024-02-18T00:38:04.701993+0000","last_peered":"2024-02-18T00:38:04.701993+0000","last_clean":"2024-02-18T00:38:04.701993+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:38:04.701993+0000","last_undegraded":"2024-02-18T00:38:04.701993+0000","last_fullsized":"2024-02-18T00:38:04.701993+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:00.758910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:36:00.758910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":365,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.650499+0000","last_change":"2024-02-18T00:36:42.828611+0000","last_active":"2024-02-18T00:38:04.650499+0000","last_peered":"2024-02-18T00:38:04.650499+0000","last_clean":"2024-02-18T00:38:04.650499+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:38:04.650499+0000","last_undegraded":"2024-02-18T00:38:04.650499+0000","last_fullsized":"2024-02-18T00:38:04.650499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.828542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.828542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":309,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705079+0000","last_change":"2024-02-18T00:35:56.193396+0000","last_active":"2024-02-18T00:38:04.705079+0000","last_peered":"2024-02-18T00:38:04.705079+0000","last_clean":"2024-02-18T00:38:04.705079+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:38:04.705079+0000","last_undegraded":"2024-02-18T00:38:04.705079+0000","last_fullsized":"2024-02-18T00:38:04.705079+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.193321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.193321+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":380,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703165+0000","last_change":"2024-02-18T00:36:57.633211+0000","last_active":"2024-02-18T00:38:04.703165+0000","last_peered":"2024-02-18T00:38:04.703165+0000","last_clean":"2024-02-18T00:38:04.703165+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:38:04.703165+0000","last_undegraded":"2024-02-18T00:38:04.703165+0000","last_fullsized":"2024-02-18T00:38:04.703165+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.633128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.633128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":415,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710652+0000","last_change":"2024-02-18T00:36:45.862082+0000","last_active":"2024-02-18T00:38:04.710652+0000","last_peered":"2024-02-18T00:38:04.710652+0000","last_clean":"2024-02-18T00:38:04.710652+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:38:04.710652+0000","last_undegraded":"2024-02-18T00:38:04.710652+0000","last_fullsized":"2024-02-18T00:38:04.710652+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:36:45.861971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.861971+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":384,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700245+0000","last_change":"2024-02-18T00:36:36.500976+0000","last_active":"2024-02-18T00:38:04.700245+0000","last_peered":"2024-02-18T00:38:04.700245+0000","last_clean":"2024-02-18T00:38:04.700245+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:38:04.700245+0000","last_undegraded":"2024-02-18T00:38:04.700245+0000","last_fullsized":"2024-02-18T00:38:04.700245+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:36.500906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.500906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":904,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.859431+0000","last_change":"2024-02-18T00:37:37.774470+0000","last_active":"2024-02-18T00:38:04.859431+0000","last_peered":"2024-02-18T00:38:04.859431+0000","last_clean":"2024-02-18T00:38:04.859431+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:38:04.859431+0000","last_undegraded":"2024-02-18T00:38:04.859431+0000","last_fullsized":"2024-02-18T00:38:04.859431+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:37:37.774404+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.774404+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":521,"num_read_kb":516,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":310,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704271+0000","last_change":"2024-02-18T00:36:42.643527+0000","last_active":"2024-02-18T00:38:04.704271+0000","last_peered":"2024-02-18T00:38:04.704271+0000","last_clean":"2024-02-18T00:38:04.704271+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:38:04.704271+0000","last_undegraded":"2024-02-18T00:38:04.704271+0000","last_fullsized":"2024-02-18T00:38:04.704271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:36:42.643433+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.643433+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":320,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700109+0000","last_change":"2024-02-18T00:36:07.453678+0000","last_active":"2024-02-18T00:38:04.700109+0000","last_peered":"2024-02-18T00:38:04.700109+0000","last_clean":"2024-02-18T00:38:04.700109+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:38:04.700109+0000","last_undegraded":"2024-02-18T00:38:04.700109+0000","last_fullsized":"2024-02-18T00:38:04.700109+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:07.453614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:07.453614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":369,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703779+0000","last_change":"2024-02-18T00:36:56.302683+0000","last_active":"2024-02-18T00:38:04.703779+0000","last_peered":"2024-02-18T00:38:04.703779+0000","last_clean":"2024-02-18T00:38:04.703779+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:38:04.703779+0000","last_undegraded":"2024-02-18T00:38:04.703779+0000","last_fullsized":"2024-02-18T00:38:04.703779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.302615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.302615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":203,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646409+0000","last_change":"2024-02-18T00:36:41.801219+0000","last_active":"2024-02-18T00:38:04.646409+0000","last_peered":"2024-02-18T00:38:04.646409+0000","last_clean":"2024-02-18T00:38:04.646409+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:38:04.646409+0000","last_undegraded":"2024-02-18T00:38:04.646409+0000","last_fullsized":"2024-02-18T00:38:04.646409+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-18T00:36:41.801144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.801144+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":380,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646645+0000","last_change":"2024-02-18T00:36:57.791451+0000","last_active":"2024-02-18T00:38:04.646645+0000","last_peered":"2024-02-18T00:38:04.646645+0000","last_clean":"2024-02-18T00:38:04.646645+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:38:04.646645+0000","last_undegraded":"2024-02-18T00:38:04.646645+0000","last_fullsized":"2024-02-18T00:38:04.646645+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.791347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.791347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":369,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.712681+0000","last_change":"2024-02-18T00:36:49.976027+0000","last_active":"2024-02-18T00:38:04.712681+0000","last_peered":"2024-02-18T00:38:04.712681+0000","last_clean":"2024-02-18T00:38:04.712681+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:38:04.712681+0000","last_undegraded":"2024-02-18T00:38:04.712681+0000","last_fullsized":"2024-02-18T00:38:04.712681+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.975957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.975957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":309,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705132+0000","last_change":"2024-02-18T00:35:57.224409+0000","last_active":"2024-02-18T00:38:04.705132+0000","last_peered":"2024-02-18T00:38:04.705132+0000","last_clean":"2024-02-18T00:38:04.705132+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:38:04.705132+0000","last_undegraded":"2024-02-18T00:38:04.705132+0000","last_fullsized":"2024-02-18T00:38:04.705132+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:57.224342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:57.224342+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":483,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701596+0000","last_change":"2024-02-18T00:37:29.141966+0000","last_active":"2024-02-18T00:38:04.701596+0000","last_peered":"2024-02-18T00:38:04.701596+0000","last_clean":"2024-02-18T00:38:04.701596+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:38:04.701596+0000","last_undegraded":"2024-02-18T00:38:04.701596+0000","last_fullsized":"2024-02-18T00:38:04.701596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:29.141901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.141901+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":373,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699733+0000","last_change":"2024-02-18T00:37:43.309558+0000","last_active":"2024-02-18T00:38:04.699733+0000","last_peered":"2024-02-18T00:38:04.699733+0000","last_clean":"2024-02-18T00:38:04.699733+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:38:04.699733+0000","last_undegraded":"2024-02-18T00:38:04.699733+0000","last_fullsized":"2024-02-18T00:38:04.699733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:37:43.309488+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.309488+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":398,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710748+0000","last_change":"2024-02-18T00:36:53.925296+0000","last_active":"2024-02-18T00:38:04.710748+0000","last_peered":"2024-02-18T00:38:04.710748+0000","last_clean":"2024-02-18T00:38:04.710748+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:38:04.710748+0000","last_undegraded":"2024-02-18T00:38:04.710748+0000","last_fullsized":"2024-02-18T00:38:04.710748+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.925187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.925187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":362,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699891+0000","last_change":"2024-02-18T00:36:46.369602+0000","last_active":"2024-02-18T00:38:04.699891+0000","last_peered":"2024-02-18T00:38:04.699891+0000","last_clean":"2024-02-18T00:38:04.699891+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:38:04.699891+0000","last_undegraded":"2024-02-18T00:38:04.699891+0000","last_fullsized":"2024-02-18T00:38:04.699891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:46.369503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.369503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":311,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699064+0000","last_change":"2024-02-18T00:36:04.413265+0000","last_active":"2024-02-18T00:38:04.699064+0000","last_peered":"2024-02-18T00:38:04.699064+0000","last_clean":"2024-02-18T00:38:04.699064+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:38:04.699064+0000","last_undegraded":"2024-02-18T00:38:04.699064+0000","last_fullsized":"2024-02-18T00:38:04.699064+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.413200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.413200+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698864+0000","last_change":"2024-02-18T00:37:44.335458+0000","last_active":"2024-02-18T00:38:04.698864+0000","last_peered":"2024-02-18T00:38:04.698864+0000","last_clean":"2024-02-18T00:38:04.698864+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:38:04.698864+0000","last_undegraded":"2024-02-18T00:38:04.698864+0000","last_fullsized":"2024-02-18T00:38:04.698864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":427,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699854+0000","last_change":"2024-02-18T00:37:39.126531+0000","last_active":"2024-02-18T00:38:04.699854+0000","last_peered":"2024-02-18T00:38:04.699854+0000","last_clean":"2024-02-18T00:38:04.699854+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:38:04.699854+0000","last_undegraded":"2024-02-18T00:38:04.699854+0000","last_fullsized":"2024-02-18T00:38:04.699854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:37:39.126398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.126398+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":409,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700985+0000","last_change":"2024-02-18T00:37:45.186258+0000","last_active":"2024-02-18T00:38:04.700985+0000","last_peered":"2024-02-18T00:38:04.700985+0000","last_clean":"2024-02-18T00:38:04.700985+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:38:04.700985+0000","last_undegraded":"2024-02-18T00:38:04.700985+0000","last_fullsized":"2024-02-18T00:38:04.700985+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:37:45.186143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.186143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":374,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705984+0000","last_change":"2024-02-18T00:37:00.804782+0000","last_active":"2024-02-18T00:38:04.705984+0000","last_peered":"2024-02-18T00:38:04.705984+0000","last_clean":"2024-02-18T00:38:04.705984+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:38:04.705984+0000","last_undegraded":"2024-02-18T00:38:04.705984+0000","last_fullsized":"2024-02-18T00:38:04.705984+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.804656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":311,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698773+0000","last_change":"2024-02-18T00:35:55.395754+0000","last_active":"2024-02-18T00:38:04.698773+0000","last_peered":"2024-02-18T00:38:04.698773+0000","last_clean":"2024-02-18T00:38:04.698773+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:38:04.698773+0000","last_undegraded":"2024-02-18T00:38:04.698773+0000","last_fullsized":"2024-02-18T00:38:04.698773+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:55.395666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:55.395666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":440,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706160+0000","last_change":"2024-02-18T00:37:46.472303+0000","last_active":"2024-02-18T00:38:04.706160+0000","last_peered":"2024-02-18T00:38:04.706160+0000","last_clean":"2024-02-18T00:38:04.706160+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:38:04.706160+0000","last_undegraded":"2024-02-18T00:38:04.706160+0000","last_fullsized":"2024-02-18T00:38:04.706160+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:37:46.472227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.472227+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":375,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698787+0000","last_change":"2024-02-18T00:37:27.357689+0000","last_active":"2024-02-18T00:38:04.698787+0000","last_peered":"2024-02-18T00:38:04.698787+0000","last_clean":"2024-02-18T00:38:04.698787+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:38:04.698787+0000","last_undegraded":"2024-02-18T00:38:04.698787+0000","last_fullsized":"2024-02-18T00:38:04.698787+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.357610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.357610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":376,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705872+0000","last_change":"2024-02-18T00:36:55.861115+0000","last_active":"2024-02-18T00:38:04.705872+0000","last_peered":"2024-02-18T00:38:04.705872+0000","last_clean":"2024-02-18T00:38:04.705872+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:38:04.705872+0000","last_undegraded":"2024-02-18T00:38:04.705872+0000","last_fullsized":"2024-02-18T00:38:04.705872+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:55.861044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.861044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":373,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700349+0000","last_change":"2024-02-18T00:35:23.469731+0000","last_active":"2024-02-18T00:38:04.700349+0000","last_peered":"2024-02-18T00:38:04.700349+0000","last_clean":"2024-02-18T00:38:04.700349+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:38:04.700349+0000","last_undegraded":"2024-02-18T00:38:04.700349+0000","last_fullsized":"2024-02-18T00:38:04.700349+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:23.469666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:35:23.469666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":309,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704939+0000","last_change":"2024-02-18T00:36:04.431769+0000","last_active":"2024-02-18T00:38:04.704939+0000","last_peered":"2024-02-18T00:38:04.704939+0000","last_clean":"2024-02-18T00:38:04.704939+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:38:04.704939+0000","last_undegraded":"2024-02-18T00:38:04.704939+0000","last_fullsized":"2024-02-18T00:38:04.704939+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:04.431708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:04.431708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":431,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705668+0000","last_change":"2024-02-18T00:36:51.948976+0000","last_active":"2024-02-18T00:38:04.705668+0000","last_peered":"2024-02-18T00:38:04.705668+0000","last_clean":"2024-02-18T00:38:04.705668+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:38:04.705668+0000","last_undegraded":"2024-02-18T00:38:04.705668+0000","last_fullsized":"2024-02-18T00:38:04.705668+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:36:51.948907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.948907+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":373,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703021+0000","last_change":"2024-02-18T00:35:14.479779+0000","last_active":"2024-02-18T00:38:04.703021+0000","last_peered":"2024-02-18T00:38:04.703021+0000","last_clean":"2024-02-18T00:38:04.703021+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:38:04.703021+0000","last_undegraded":"2024-02-18T00:38:04.703021+0000","last_fullsized":"2024-02-18T00:38:04.703021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:14.479703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:14.479703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":367,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646266+0000","last_change":"2024-02-18T00:37:38.687198+0000","last_active":"2024-02-18T00:38:04.646266+0000","last_peered":"2024-02-18T00:38:04.646266+0000","last_clean":"2024-02-18T00:38:04.646266+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:38:04.646266+0000","last_undegraded":"2024-02-18T00:38:04.646266+0000","last_fullsized":"2024-02-18T00:38:04.646266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:38.687129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.687129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":273,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704324+0000","last_change":"2024-02-18T00:36:10.525364+0000","last_active":"2024-02-18T00:38:04.704324+0000","last_peered":"2024-02-18T00:38:04.704324+0000","last_clean":"2024-02-18T00:38:04.704324+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:38:04.704324+0000","last_undegraded":"2024-02-18T00:38:04.704324+0000","last_fullsized":"2024-02-18T00:38:04.704324+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:10.525299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:10.525299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":319,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700470+0000","last_change":"2024-02-18T00:35:59.704864+0000","last_active":"2024-02-18T00:38:04.700470+0000","last_peered":"2024-02-18T00:38:04.700470+0000","last_clean":"2024-02-18T00:38:04.700470+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:38:04.700470+0000","last_undegraded":"2024-02-18T00:38:04.700470+0000","last_fullsized":"2024-02-18T00:38:04.700470+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.704806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.704806+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":439,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705592+0000","last_change":"2024-02-18T00:36:58.819567+0000","last_active":"2024-02-18T00:38:04.705592+0000","last_peered":"2024-02-18T00:38:04.705592+0000","last_clean":"2024-02-18T00:38:04.705592+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:38:04.705592+0000","last_undegraded":"2024-02-18T00:38:04.705592+0000","last_fullsized":"2024-02-18T00:38:04.705592+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:36:58.819503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.819503+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":389,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646540+0000","last_change":"2024-02-18T00:37:41.637636+0000","last_active":"2024-02-18T00:38:04.646540+0000","last_peered":"2024-02-18T00:38:04.646540+0000","last_clean":"2024-02-18T00:38:04.646540+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:38:04.646540+0000","last_undegraded":"2024-02-18T00:38:04.646540+0000","last_fullsized":"2024-02-18T00:38:04.646540+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.637550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.637550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":481,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648509+0000","last_change":"2024-02-18T00:37:27.642761+0000","last_active":"2024-02-18T00:38:04.648509+0000","last_peered":"2024-02-18T00:38:04.648509+0000","last_clean":"2024-02-18T00:38:04.648509+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:38:04.648509+0000","last_undegraded":"2024-02-18T00:38:04.648509+0000","last_fullsized":"2024-02-18T00:38:04.648509+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:27.642661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.642661+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":376,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705976+0000","last_change":"2024-02-18T00:36:44.367116+0000","last_active":"2024-02-18T00:38:04.705976+0000","last_peered":"2024-02-18T00:38:04.705976+0000","last_clean":"2024-02-18T00:38:04.705976+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:38:04.705976+0000","last_undegraded":"2024-02-18T00:38:04.705976+0000","last_fullsized":"2024-02-18T00:38:04.705976+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.367032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.367032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":322,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701571+0000","last_change":"2024-02-18T00:37:46.639988+0000","last_active":"2024-02-18T00:38:04.701571+0000","last_peered":"2024-02-18T00:38:04.701571+0000","last_clean":"2024-02-18T00:38:04.701571+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:38:04.701571+0000","last_undegraded":"2024-02-18T00:38:04.701571+0000","last_fullsized":"2024-02-18T00:38:04.701571+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:46.639901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.639901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":416,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.648340+0000","last_change":"2024-02-18T00:36:02.987137+0000","last_active":"2024-02-18T00:38:04.648340+0000","last_peered":"2024-02-18T00:38:04.648340+0000","last_clean":"2024-02-18T00:38:04.648340+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:38:04.648340+0000","last_undegraded":"2024-02-18T00:38:04.648340+0000","last_fullsized":"2024-02-18T00:38:04.648340+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:36:02.987058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:02.987058+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":373,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701424+0000","last_change":"2024-02-18T00:37:03.239555+0000","last_active":"2024-02-18T00:38:04.701424+0000","last_peered":"2024-02-18T00:38:04.701424+0000","last_clean":"2024-02-18T00:38:04.701424+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:38:04.701424+0000","last_undegraded":"2024-02-18T00:38:04.701424+0000","last_fullsized":"2024-02-18T00:38:04.701424+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.239451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.239451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"341'10","reported_seq":495,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705900+0000","last_change":"2024-02-18T00:36:59.830567+0000","last_active":"2024-02-18T00:38:04.705900+0000","last_peered":"2024-02-18T00:38:04.705900+0000","last_clean":"2024-02-18T00:38:04.705900+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:38:04.705900+0000","last_undegraded":"2024-02-18T00:38:04.705900+0000","last_fullsized":"2024-02-18T00:38:04.705900+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'10","last_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_deep_scrub":"341'10","last_deep_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.830497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":365,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.701144+0000","last_change":"2024-02-18T00:35:59.413614+0000","last_active":"2024-02-18T00:38:04.701144+0000","last_peered":"2024-02-18T00:38:04.701144+0000","last_clean":"2024-02-18T00:38:04.701144+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:38:04.701144+0000","last_undegraded":"2024-02-18T00:38:04.701144+0000","last_fullsized":"2024-02-18T00:38:04.701144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:59.413524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:35:59.413524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":311,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.650587+0000","last_change":"2024-02-18T00:37:39.639314+0000","last_active":"2024-02-18T00:38:04.650587+0000","last_peered":"2024-02-18T00:38:04.650587+0000","last_clean":"2024-02-18T00:38:04.650587+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:38:04.650587+0000","last_undegraded":"2024-02-18T00:38:04.650587+0000","last_fullsized":"2024-02-18T00:38:04.650587+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:39.639232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.639232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":389,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709829+0000","last_change":"2024-02-18T00:36:55.970729+0000","last_active":"2024-02-18T00:38:04.709829+0000","last_peered":"2024-02-18T00:38:04.709829+0000","last_clean":"2024-02-18T00:38:04.709829+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:38:04.709829+0000","last_undegraded":"2024-02-18T00:38:04.709829+0000","last_fullsized":"2024-02-18T00:38:04.709829+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:36:55.970628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.970628+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":378,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703202+0000","last_change":"2024-02-18T00:36:44.617522+0000","last_active":"2024-02-18T00:38:04.703202+0000","last_peered":"2024-02-18T00:38:04.703202+0000","last_clean":"2024-02-18T00:38:04.703202+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:38:04.703202+0000","last_undegraded":"2024-02-18T00:38:04.703202+0000","last_fullsized":"2024-02-18T00:38:04.703202+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.617443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.617443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":367,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705855+0000","last_change":"2024-02-18T00:36:50.970267+0000","last_active":"2024-02-18T00:38:04.705855+0000","last_peered":"2024-02-18T00:38:04.705855+0000","last_clean":"2024-02-18T00:38:04.705855+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:38:04.705855+0000","last_undegraded":"2024-02-18T00:38:04.705855+0000","last_fullsized":"2024-02-18T00:38:04.705855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:50.970192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.970192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":362,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699811+0000","last_change":"2024-02-18T00:36:11.346267+0000","last_active":"2024-02-18T00:38:04.699811+0000","last_peered":"2024-02-18T00:38:04.699811+0000","last_clean":"2024-02-18T00:38:04.699811+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:38:04.699811+0000","last_undegraded":"2024-02-18T00:38:04.699811+0000","last_fullsized":"2024-02-18T00:38:04.699811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.346197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.346197+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":309,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705882+0000","last_change":"2024-02-18T00:36:42.355194+0000","last_active":"2024-02-18T00:38:04.705882+0000","last_peered":"2024-02-18T00:38:04.705882+0000","last_clean":"2024-02-18T00:38:04.705882+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:38:04.705882+0000","last_undegraded":"2024-02-18T00:38:04.705882+0000","last_fullsized":"2024-02-18T00:38:04.705882+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.355114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.355114+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":393,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698957+0000","last_change":"2024-02-18T00:36:52.343624+0000","last_active":"2024-02-18T00:38:04.698957+0000","last_peered":"2024-02-18T00:38:04.698957+0000","last_clean":"2024-02-18T00:38:04.698957+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:38:04.698957+0000","last_undegraded":"2024-02-18T00:38:04.698957+0000","last_fullsized":"2024-02-18T00:38:04.698957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:36:52.343556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.343556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":375,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709750+0000","last_change":"2024-02-18T00:37:27.830300+0000","last_active":"2024-02-18T00:38:04.709750+0000","last_peered":"2024-02-18T00:38:04.709750+0000","last_clean":"2024-02-18T00:38:04.709750+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:38:04.709750+0000","last_undegraded":"2024-02-18T00:38:04.709750+0000","last_fullsized":"2024-02-18T00:38:04.709750+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.830225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.830225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709908+0000","last_change":"2024-02-18T00:36:42.857622+0000","last_active":"2024-02-18T00:38:04.709908+0000","last_peered":"2024-02-18T00:38:04.709908+0000","last_clean":"2024-02-18T00:38:04.709908+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:38:04.709908+0000","last_undegraded":"2024-02-18T00:38:04.709908+0000","last_fullsized":"2024-02-18T00:38:04.709908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.857557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.857557+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":365,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704977+0000","last_change":"2024-02-18T00:37:37.848598+0000","last_active":"2024-02-18T00:38:04.704977+0000","last_peered":"2024-02-18T00:38:04.704977+0000","last_clean":"2024-02-18T00:38:04.704977+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:38:04.704977+0000","last_undegraded":"2024-02-18T00:38:04.704977+0000","last_fullsized":"2024-02-18T00:38:04.704977+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.848524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.848524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":308,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704147+0000","last_change":"2024-02-18T00:35:18.493169+0000","last_active":"2024-02-18T00:38:04.704147+0000","last_peered":"2024-02-18T00:38:04.704147+0000","last_clean":"2024-02-18T00:38:04.704147+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:38:04.704147+0000","last_undegraded":"2024-02-18T00:38:04.704147+0000","last_fullsized":"2024-02-18T00:38:04.704147+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.493095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.493095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":375,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.713236+0000","last_change":"2024-02-18T00:37:41.880926+0000","last_active":"2024-02-18T00:38:04.713236+0000","last_peered":"2024-02-18T00:38:04.713236+0000","last_clean":"2024-02-18T00:38:04.713236+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:38:04.713236+0000","last_undegraded":"2024-02-18T00:38:04.713236+0000","last_fullsized":"2024-02-18T00:38:04.713236+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:37:41.880855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.880855+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":371,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646366+0000","last_change":"2024-02-18T00:36:05.065367+0000","last_active":"2024-02-18T00:38:04.646366+0000","last_peered":"2024-02-18T00:38:04.646366+0000","last_clean":"2024-02-18T00:38:04.646366+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:38:04.646366+0000","last_undegraded":"2024-02-18T00:38:04.646366+0000","last_fullsized":"2024-02-18T00:38:04.646366+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:05.065294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:36:05.065294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":378,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702777+0000","last_change":"2024-02-18T00:36:12.518914+0000","last_active":"2024-02-18T00:38:04.702777+0000","last_peered":"2024-02-18T00:38:04.702777+0000","last_clean":"2024-02-18T00:38:04.702777+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:38:04.702777+0000","last_undegraded":"2024-02-18T00:38:04.702777+0000","last_fullsized":"2024-02-18T00:38:04.702777+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:12.518838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:12.518838+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":275,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698535+0000","last_change":"2024-02-18T00:37:29.374778+0000","last_active":"2024-02-18T00:38:04.698535+0000","last_peered":"2024-02-18T00:38:04.698535+0000","last_clean":"2024-02-18T00:38:04.698535+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:38:04.698535+0000","last_undegraded":"2024-02-18T00:38:04.698535+0000","last_fullsized":"2024-02-18T00:38:04.698535+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:29.374712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.374712+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":315,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646980+0000","last_change":"2024-02-18T00:35:56.924678+0000","last_active":"2024-02-18T00:38:04.646980+0000","last_peered":"2024-02-18T00:38:04.646980+0000","last_clean":"2024-02-18T00:38:04.646980+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:38:04.646980+0000","last_undegraded":"2024-02-18T00:38:04.646980+0000","last_fullsized":"2024-02-18T00:38:04.646980+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.924605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.924605+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":426,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703419+0000","last_change":"2024-02-18T00:37:45.838598+0000","last_active":"2024-02-18T00:38:04.703419+0000","last_peered":"2024-02-18T00:38:04.703419+0000","last_clean":"2024-02-18T00:38:04.703419+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:38:04.703419+0000","last_undegraded":"2024-02-18T00:38:04.703419+0000","last_fullsized":"2024-02-18T00:38:04.703419+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:37:45.838523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.838523+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":373,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703329+0000","last_change":"2024-02-18T00:36:11.551695+0000","last_active":"2024-02-18T00:38:04.703329+0000","last_peered":"2024-02-18T00:38:04.703329+0000","last_clean":"2024-02-18T00:38:04.703329+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:38:04.703329+0000","last_undegraded":"2024-02-18T00:38:04.703329+0000","last_fullsized":"2024-02-18T00:38:04.703329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:36:11.551602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":285,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698210+0000","last_change":"2024-02-18T00:36:59.243785+0000","last_active":"2024-02-18T00:38:04.698210+0000","last_peered":"2024-02-18T00:38:04.698210+0000","last_clean":"2024-02-18T00:38:04.698210+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:38:04.698210+0000","last_undegraded":"2024-02-18T00:38:04.698210+0000","last_fullsized":"2024-02-18T00:38:04.698210+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.243699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.243699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":365,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706106+0000","last_change":"2024-02-18T00:37:44.869935+0000","last_active":"2024-02-18T00:38:04.706106+0000","last_peered":"2024-02-18T00:38:04.706106+0000","last_clean":"2024-02-18T00:38:04.706106+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:38:04.706106+0000","last_undegraded":"2024-02-18T00:38:04.706106+0000","last_fullsized":"2024-02-18T00:38:04.706106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.869862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.869862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":271,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698683+0000","last_change":"2024-02-18T00:35:56.405166+0000","last_active":"2024-02-18T00:38:04.698683+0000","last_peered":"2024-02-18T00:38:04.698683+0000","last_clean":"2024-02-18T00:38:04.698683+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:38:04.698683+0000","last_undegraded":"2024-02-18T00:38:04.698683+0000","last_fullsized":"2024-02-18T00:38:04.698683+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.405090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.405090+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":430,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706358+0000","last_change":"2024-02-18T00:36:53.401897+0000","last_active":"2024-02-18T00:38:04.706358+0000","last_peered":"2024-02-18T00:38:04.706358+0000","last_clean":"2024-02-18T00:38:04.706358+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:38:04.706358+0000","last_undegraded":"2024-02-18T00:38:04.706358+0000","last_fullsized":"2024-02-18T00:38:04.706358+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:36:53.401819+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.401819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":375,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.644225+0000","last_change":"2024-02-18T00:35:58.960530+0000","last_active":"2024-02-18T00:38:04.644225+0000","last_peered":"2024-02-18T00:38:04.644225+0000","last_clean":"2024-02-18T00:38:04.644225+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:38:04.644225+0000","last_undegraded":"2024-02-18T00:38:04.644225+0000","last_fullsized":"2024-02-18T00:38:04.644225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.960452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.960452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":504,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703364+0000","last_change":"2024-02-18T00:36:38.557419+0000","last_active":"2024-02-18T00:38:04.703364+0000","last_peered":"2024-02-18T00:38:04.703364+0000","last_clean":"2024-02-18T00:38:04.703364+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:38:04.703364+0000","last_undegraded":"2024-02-18T00:38:04.703364+0000","last_fullsized":"2024-02-18T00:38:04.703364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:36:38.557298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.557298+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":284,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646930+0000","last_change":"2024-02-18T00:36:49.727241+0000","last_active":"2024-02-18T00:38:04.646930+0000","last_peered":"2024-02-18T00:38:04.646930+0000","last_clean":"2024-02-18T00:38:04.646930+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:38:04.646930+0000","last_undegraded":"2024-02-18T00:38:04.646930+0000","last_fullsized":"2024-02-18T00:38:04.646930+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:36:49.727157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.727157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":266,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.711486+0000","last_change":"2024-02-18T00:35:27.520427+0000","last_active":"2024-02-18T00:38:04.711486+0000","last_peered":"2024-02-18T00:38:04.711486+0000","last_clean":"2024-02-18T00:38:04.711486+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:38:04.711486+0000","last_undegraded":"2024-02-18T00:38:04.711486+0000","last_fullsized":"2024-02-18T00:38:04.711486+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:27.520359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:27.520359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":405,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703417+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:38:04.703417+0000","last_peered":"2024-02-18T00:38:04.703417+0000","last_clean":"2024-02-18T00:38:04.703417+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:38:04.703417+0000","last_undegraded":"2024-02-18T00:38:04.703417+0000","last_fullsized":"2024-02-18T00:38:04.703417+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706079+0000","last_change":"2024-02-18T00:36:56.850409+0000","last_active":"2024-02-18T00:38:04.706079+0000","last_peered":"2024-02-18T00:38:04.706079+0000","last_clean":"2024-02-18T00:38:04.706079+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:38:04.706079+0000","last_undegraded":"2024-02-18T00:38:04.706079+0000","last_fullsized":"2024-02-18T00:38:04.706079+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.850326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.850326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":373,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.706434+0000","last_change":"2024-02-18T00:36:52.372043+0000","last_active":"2024-02-18T00:38:04.706434+0000","last_peered":"2024-02-18T00:38:04.706434+0000","last_clean":"2024-02-18T00:38:04.706434+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:38:04.706434+0000","last_undegraded":"2024-02-18T00:38:04.706434+0000","last_fullsized":"2024-02-18T00:38:04.706434+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.371970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.371970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":374,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.709564+0000","last_change":"2024-02-18T00:37:40.707426+0000","last_active":"2024-02-18T00:38:04.709564+0000","last_peered":"2024-02-18T00:38:04.709564+0000","last_clean":"2024-02-18T00:38:04.709564+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:38:04.709564+0000","last_undegraded":"2024-02-18T00:38:04.709564+0000","last_fullsized":"2024-02-18T00:38:04.709564+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.707359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.707359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":328,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700383+0000","last_change":"2024-02-18T00:37:48.395278+0000","last_active":"2024-02-18T00:38:04.700383+0000","last_peered":"2024-02-18T00:38:04.700383+0000","last_clean":"2024-02-18T00:38:04.700383+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:38:04.700383+0000","last_undegraded":"2024-02-18T00:38:04.700383+0000","last_fullsized":"2024-02-18T00:38:04.700383+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:37:48.395205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.395205+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":403,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700706+0000","last_change":"2024-02-18T00:37:39.385746+0000","last_active":"2024-02-18T00:38:04.700706+0000","last_peered":"2024-02-18T00:38:04.700706+0000","last_clean":"2024-02-18T00:38:04.700706+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:38:04.700706+0000","last_undegraded":"2024-02-18T00:38:04.700706+0000","last_fullsized":"2024-02-18T00:38:04.700706+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:39.385675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.385675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":389,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702646+0000","last_change":"2024-02-18T00:37:36.615647+0000","last_active":"2024-02-18T00:38:04.702646+0000","last_peered":"2024-02-18T00:38:04.702646+0000","last_clean":"2024-02-18T00:38:04.702646+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:38:04.702646+0000","last_undegraded":"2024-02-18T00:38:04.702646+0000","last_fullsized":"2024-02-18T00:38:04.702646+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.615568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.615568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":371,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710046+0000","last_change":"2024-02-18T00:36:35.808459+0000","last_active":"2024-02-18T00:38:04.710046+0000","last_peered":"2024-02-18T00:38:04.710046+0000","last_clean":"2024-02-18T00:38:04.710046+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:38:04.710046+0000","last_undegraded":"2024-02-18T00:38:04.710046+0000","last_fullsized":"2024-02-18T00:38:04.710046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.808381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.808381+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":370,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700694+0000","last_change":"2024-02-18T00:37:34.122873+0000","last_active":"2024-02-18T00:38:04.700694+0000","last_peered":"2024-02-18T00:38:04.700694+0000","last_clean":"2024-02-18T00:38:04.700694+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:38:04.700694+0000","last_undegraded":"2024-02-18T00:38:04.700694+0000","last_fullsized":"2024-02-18T00:38:04.700694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.122764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.122764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":313,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702161+0000","last_change":"2024-02-18T00:37:40.562387+0000","last_active":"2024-02-18T00:38:04.702161+0000","last_peered":"2024-02-18T00:38:04.702161+0000","last_clean":"2024-02-18T00:38:04.702161+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:38:04.702161+0000","last_undegraded":"2024-02-18T00:38:04.702161+0000","last_fullsized":"2024-02-18T00:38:04.702161+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.562306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.562306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":407,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.703557+0000","last_change":"2024-02-18T00:37:43.883242+0000","last_active":"2024-02-18T00:38:04.703557+0000","last_peered":"2024-02-18T00:38:04.703557+0000","last_clean":"2024-02-18T00:38:04.703557+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:38:04.703557+0000","last_undegraded":"2024-02-18T00:38:04.703557+0000","last_fullsized":"2024-02-18T00:38:04.703557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:37:43.883172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.883172+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":391,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704682+0000","last_change":"2024-02-18T00:36:52.928195+0000","last_active":"2024-02-18T00:38:04.704682+0000","last_peered":"2024-02-18T00:38:04.704682+0000","last_clean":"2024-02-18T00:38:04.704682+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:38:04.704682+0000","last_undegraded":"2024-02-18T00:38:04.704682+0000","last_fullsized":"2024-02-18T00:38:04.704682+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:36:52.928076+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.928076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":372,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699021+0000","last_change":"2024-02-18T00:37:03.248349+0000","last_active":"2024-02-18T00:38:04.699021+0000","last_peered":"2024-02-18T00:38:04.699021+0000","last_clean":"2024-02-18T00:38:04.699021+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:38:04.699021+0000","last_undegraded":"2024-02-18T00:38:04.699021+0000","last_fullsized":"2024-02-18T00:38:04.699021+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.248277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.248277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":360,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705532+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:38:04.705532+0000","last_peered":"2024-02-18T00:38:04.705532+0000","last_clean":"2024-02-18T00:38:04.705532+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:38:04.705532+0000","last_undegraded":"2024-02-18T00:38:04.705532+0000","last_fullsized":"2024-02-18T00:38:04.705532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":318,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705217+0000","last_change":"2024-02-18T00:35:58.230215+0000","last_active":"2024-02-18T00:38:04.705217+0000","last_peered":"2024-02-18T00:38:04.705217+0000","last_clean":"2024-02-18T00:38:04.705217+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:38:04.705217+0000","last_undegraded":"2024-02-18T00:38:04.705217+0000","last_fullsized":"2024-02-18T00:38:04.705217+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:58.230116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:58.230116+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":432,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698787+0000","last_change":"2024-02-18T00:37:48.216997+0000","last_active":"2024-02-18T00:38:04.698787+0000","last_peered":"2024-02-18T00:38:04.698787+0000","last_clean":"2024-02-18T00:38:04.698787+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:38:04.698787+0000","last_undegraded":"2024-02-18T00:38:04.698787+0000","last_fullsized":"2024-02-18T00:38:04.698787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:37:48.216929+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.216929+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":266,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.646812+0000","last_change":"2024-02-18T00:35:18.791644+0000","last_active":"2024-02-18T00:38:04.646812+0000","last_peered":"2024-02-18T00:38:04.646812+0000","last_clean":"2024-02-18T00:38:04.646812+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:38:04.646812+0000","last_undegraded":"2024-02-18T00:38:04.646812+0000","last_fullsized":"2024-02-18T00:38:04.646812+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:18.791563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:35:18.791563+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":385,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.647507+0000","last_change":"2024-02-18T00:36:45.773797+0000","last_active":"2024-02-18T00:38:04.647507+0000","last_peered":"2024-02-18T00:38:04.647507+0000","last_clean":"2024-02-18T00:38:04.647507+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:38:04.647507+0000","last_undegraded":"2024-02-18T00:38:04.647507+0000","last_fullsized":"2024-02-18T00:38:04.647507+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.773695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.773695+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":376,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.700453+0000","last_change":"2024-02-18T00:36:45.412472+0000","last_active":"2024-02-18T00:38:04.700453+0000","last_peered":"2024-02-18T00:38:04.700453+0000","last_clean":"2024-02-18T00:38:04.700453+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:38:04.700453+0000","last_undegraded":"2024-02-18T00:38:04.700453+0000","last_fullsized":"2024-02-18T00:38:04.700453+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.412406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.412406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":310,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.702905+0000","last_change":"2024-02-18T00:36:01.770742+0000","last_active":"2024-02-18T00:38:04.702905+0000","last_peered":"2024-02-18T00:38:04.702905+0000","last_clean":"2024-02-18T00:38:04.702905+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:38:04.702905+0000","last_undegraded":"2024-02-18T00:38:04.702905+0000","last_fullsized":"2024-02-18T00:38:04.702905+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:01.770679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:01.770679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":402,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698862+0000","last_change":"2024-02-18T00:36:51.303043+0000","last_active":"2024-02-18T00:38:04.698862+0000","last_peered":"2024-02-18T00:38:04.698862+0000","last_clean":"2024-02-18T00:38:04.698862+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:38:04.698862+0000","last_undegraded":"2024-02-18T00:38:04.698862+0000","last_fullsized":"2024-02-18T00:38:04.698862+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:36:51.302920+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.302920+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":273,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704096+0000","last_change":"2024-02-18T00:36:52.758965+0000","last_active":"2024-02-18T00:38:04.704096+0000","last_peered":"2024-02-18T00:38:04.704096+0000","last_clean":"2024-02-18T00:38:04.704096+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:38:04.704096+0000","last_undegraded":"2024-02-18T00:38:04.704096+0000","last_fullsized":"2024-02-18T00:38:04.704096+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.758904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.758904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":271,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.699762+0000","last_change":"2024-02-18T00:36:49.277109+0000","last_active":"2024-02-18T00:38:04.699762+0000","last_peered":"2024-02-18T00:38:04.699762+0000","last_clean":"2024-02-18T00:38:04.699762+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:38:04.699762+0000","last_undegraded":"2024-02-18T00:38:04.699762+0000","last_fullsized":"2024-02-18T00:38:04.699762+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.276979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.276979+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":365,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.645848+0000","last_change":"2024-02-18T00:36:40.825174+0000","last_active":"2024-02-18T00:38:04.645848+0000","last_peered":"2024-02-18T00:38:04.645848+0000","last_clean":"2024-02-18T00:38:04.645848+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:38:04.645848+0000","last_undegraded":"2024-02-18T00:38:04.645848+0000","last_fullsized":"2024-02-18T00:38:04.645848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:40.825055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:36:40.825055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":315,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704142+0000","last_change":"2024-02-18T00:35:53.688172+0000","last_active":"2024-02-18T00:38:04.704142+0000","last_peered":"2024-02-18T00:38:04.704142+0000","last_clean":"2024-02-18T00:38:04.704142+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:38:04.704142+0000","last_undegraded":"2024-02-18T00:38:04.704142+0000","last_fullsized":"2024-02-18T00:38:04.704142+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:53.688084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:53.688084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":405,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698092+0000","last_change":"2024-02-18T00:35:30.243304+0000","last_active":"2024-02-18T00:38:04.698092+0000","last_peered":"2024-02-18T00:38:04.698092+0000","last_clean":"2024-02-18T00:38:04.698092+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:38:04.698092+0000","last_undegraded":"2024-02-18T00:38:04.698092+0000","last_fullsized":"2024-02-18T00:38:04.698092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:35:30.243229+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":366,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.712895+0000","last_change":"2024-02-18T00:35:26.537533+0000","last_active":"2024-02-18T00:38:04.712895+0000","last_peered":"2024-02-18T00:38:04.712895+0000","last_clean":"2024-02-18T00:38:04.712895+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:38:04.712895+0000","last_undegraded":"2024-02-18T00:38:04.712895+0000","last_fullsized":"2024-02-18T00:38:04.712895+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:26.537460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:35:26.537460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":380,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.710192+0000","last_change":"2024-02-18T00:36:49.936731+0000","last_active":"2024-02-18T00:38:04.710192+0000","last_peered":"2024-02-18T00:38:04.710192+0000","last_clean":"2024-02-18T00:38:04.710192+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:38:04.710192+0000","last_undegraded":"2024-02-18T00:38:04.710192+0000","last_fullsized":"2024-02-18T00:38:04.710192+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.936644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.936644+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":370,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698686+0000","last_change":"2024-02-18T00:37:42.183653+0000","last_active":"2024-02-18T00:38:04.698686+0000","last_peered":"2024-02-18T00:38:04.698686+0000","last_clean":"2024-02-18T00:38:04.698686+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:38:04.698686+0000","last_undegraded":"2024-02-18T00:38:04.698686+0000","last_fullsized":"2024-02-18T00:38:04.698686+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.183568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.183568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":306,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.712564+0000","last_change":"2024-02-18T00:35:56.756453+0000","last_active":"2024-02-18T00:38:04.712564+0000","last_peered":"2024-02-18T00:38:04.712564+0000","last_clean":"2024-02-18T00:38:04.712564+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:38:04.712564+0000","last_undegraded":"2024-02-18T00:38:04.712564+0000","last_fullsized":"2024-02-18T00:38:04.712564+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:35:56.756384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:35:56.756384+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":451,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.705953+0000","last_change":"2024-02-18T00:36:46.942689+0000","last_active":"2024-02-18T00:38:04.705953+0000","last_peered":"2024-02-18T00:38:04.705953+0000","last_clean":"2024-02-18T00:38:04.705953+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:38:04.705953+0000","last_undegraded":"2024-02-18T00:38:04.705953+0000","last_fullsized":"2024-02-18T00:38:04.705953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:36:46.942583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.942583+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":384,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.704931+0000","last_change":"2024-02-18T00:37:45.454134+0000","last_active":"2024-02-18T00:38:04.704931+0000","last_peered":"2024-02-18T00:38:04.704931+0000","last_clean":"2024-02-18T00:38:04.704931+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:38:04.704931+0000","last_undegraded":"2024-02-18T00:38:04.704931+0000","last_fullsized":"2024-02-18T00:38:04.704931+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.454069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":372,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.698754+0000","last_change":"2024-02-18T00:37:36.119369+0000","last_active":"2024-02-18T00:38:04.698754+0000","last_peered":"2024-02-18T00:38:04.698754+0000","last_clean":"2024-02-18T00:38:04.698754+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:38:04.698754+0000","last_undegraded":"2024-02-18T00:38:04.698754+0000","last_fullsized":"2024-02-18T00:38:04.698754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.119291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.119291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":374,"reported_epoch":403,"state":"active+clean","last_fresh":"2024-02-18T00:38:04.713327+0000","last_change":"2024-02-18T00:37:34.860293+0000","last_active":"2024-02-18T00:38:04.713327+0000","last_peered":"2024-02-18T00:38:04.713327+0000","last_clean":"2024-02-18T00:38:04.713327+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:38:04.713327+0000","last_undegraded":"2024-02-18T00:38:04.713327+0000","last_fullsized":"2024-02-18T00:38:04.713327+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.860214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.860214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":209,"num_pg":32,"stat_sum":{"num_bytes":10,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":6},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001445,"num_objects":153,"num_object_clones":2,"num_object_copies":459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":106,"num_read_kb":9,"num_write":582,"num_write_kb":4483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13856768,"data_stored":12004368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1837,"ondisk_log_size":1837,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":521,"num_read_kb":516,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266798,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463028,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309732,"kb_used_data":8412,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435420,"statfs":{"total":95995035648,"available":95677870080,"internally_reserved":0,"allocated":8613888,"data_stored":6344616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":2.7639999999999998,"15min":1.252},"min":{"1min":0.38500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":4.0129999999999999,"5min":69.194999999999993,"15min":69.194999999999993},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":1.1830000000000001,"15min":0.73099999999999998},"min":{"1min":0.38500000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.7170000000000001,"5min":10.433,"15min":10.433},"last":0.64600000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":2.968,"15min":1.325},"min":{"1min":0.314,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":1.788,"5min":76.108999999999995,"15min":76.108999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":2.9670000000000001,"15min":1.3149999999999999},"min":{"1min":0.36099999999999999,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":1.1519999999999999,"5min":71.872,"15min":71.872},"last":0.61199999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":1.3999999999999999,"15min":0.80600000000000005},"min":{"1min":0.39100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":1.196,"5min":39.939999999999998,"15min":39.939999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.8959999999999999,"15min":0.98199999999999998},"min":{"1min":0.40500000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":3.1600000000000001,"5min":39.673000000000002,"15min":39.673000000000002},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.476,"15min":0.83599999999999997},"min":{"1min":0.33800000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":8.3420000000000005,"5min":21.219000000000001,"15min":21.219000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.302,"15min":0.78200000000000003},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.506,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.67300000000000004,"15min":0.5},"min":{"1min":0.28399999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.59499999999999997,"5min":8.1479999999999997,"15min":8.1479999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":1.1839999999999999,"15min":0.67600000000000005},"min":{"1min":0.35199999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.626,"5min":35.18,"15min":35.18},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:37:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.72299999999999998,"15min":0.52800000000000002},"min":{"1min":0.29399999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67800000000000005,"5min":10.606,"15min":10.606},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":1.214,"15min":0.70099999999999996},"min":{"1min":0.35999999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66900000000000004,"5min":35.119,"15min":35.119},"last":0.60399999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63500000000000001,"15min":0.55000000000000004},"min":{"1min":0.33200000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73299999999999998,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.1220000000000001,"15min":0.72699999999999998},"min":{"1min":0.42799999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.745,"5min":32.985999999999997,"15min":32.985999999999997},"last":0.443}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305824,"kb_used_data":4504,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439328,"statfs":{"total":95995035648,"available":95681871872,"internally_reserved":0,"allocated":4612096,"data_stored":2344376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":1.218,"15min":0.63200000000000001},"min":{"1min":0.22,"5min":0.193,"15min":0.193},"max":{"1min":0.51800000000000002,"5min":32.106999999999999,"15min":32.106999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":1.2989999999999999,"15min":0.67800000000000005},"min":{"1min":0.23999999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":2.7040000000000002,"5min":43.137,"15min":43.137},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":1.528,"15min":0.75900000000000001},"min":{"1min":0.22900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.78300000000000003,"5min":49.201000000000001,"15min":49.201000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.95699999999999996,"15min":0.58599999999999997},"min":{"1min":0.27000000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.623,"5min":17.596,"15min":17.596},"last":0.502}]},{"osd":3,"last update":"Sun Feb 18 00:37:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":1.3440000000000001,"15min":0.747},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.92800000000000005,"5min":37.524000000000001,"15min":37.524000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.41499999999999998,"5min":1.3400000000000001,"15min":0.76600000000000001},"min":{"1min":0.30199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61499999999999999,"5min":32.036000000000001,"15min":32.036000000000001},"last":0.48399999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:37:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":1.1319999999999999,"15min":0.77700000000000002},"min":{"1min":0.41799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73999999999999999,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":1.1719999999999999,"15min":0.77100000000000002},"min":{"1min":0.40999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.9790000000000001,"5min":12.75,"15min":12.75},"last":0.63800000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:37:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":1.127,"15min":0.77600000000000002},"min":{"1min":0.40000000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82899999999999996,"5min":12.611000000000001,"15min":12.611000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":1.129,"15min":0.78800000000000003},"min":{"1min":0.45900000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80200000000000005,"5min":17.417000000000002,"15min":17.417000000000002},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":1.1359999999999999,"15min":0.81100000000000005},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.0430000000000001,"5min":12.834,"15min":12.834},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":1.1240000000000001,"15min":0.79700000000000004},"min":{"1min":0.42199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.871,"5min":12.651999999999999,"15min":12.651999999999999},"last":0.75900000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":1.143,"15min":0.80400000000000005},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77000000000000002,"5min":17.452000000000002,"15min":17.452000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":1.123,"15min":0.81699999999999995},"min":{"1min":0.47799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.0859999999999999,"5min":12.805,"15min":12.805},"last":0.60199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738513,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301752,"kb_used_data":4528,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443400,"statfs":{"total":95995035648,"available":95686041600,"internally_reserved":0,"allocated":4636672,"data_stored":2343527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":2.2229999999999999,"15min":1.0980000000000001},"min":{"1min":0.218,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.55900000000000005,"5min":133.38999999999999,"15min":133.38999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.5760000000000001,"15min":0.77200000000000002},"min":{"1min":0.24399999999999999,"5min":0.217,"15min":0.20999999999999999},"max":{"1min":2.7999999999999998,"5min":90.623000000000005,"15min":90.623000000000005},"last":0.307}]},{"osd":2,"last update":"Sun Feb 18 00:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":1.8759999999999999,"15min":0.88900000000000001},"min":{"1min":0.27400000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":4.54,"5min":90.850999999999999,"15min":90.850999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":1.8380000000000001,"15min":0.871},"min":{"1min":0.25,"5min":0.23100000000000001,"15min":0.224},"max":{"1min":0.94899999999999995,"5min":80.537000000000006,"15min":80.537000000000006},"last":0.28599999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":1.135,"15min":0.67400000000000004},"min":{"1min":0.23999999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1359999999999999,"5min":19.006,"15min":19.006},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.45000000000000001,"5min":2.254,"15min":1.0740000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93799999999999994,"5min":146.71899999999999,"15min":146.71899999999999},"last":0.33800000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:38:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.89400000000000002,"15min":0.92000000000000004},"min":{"1min":0.379,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":1.5269999999999999,"5min":12.988,"15min":57.914000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.88500000000000001,"15min":0.97999999999999998},"min":{"1min":0.39300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.018,"5min":11.551,"15min":90.070999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.673,"15min":0.95199999999999996},"min":{"1min":0.41199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":4.3760000000000003,"5min":90.049000000000007,"15min":90.049000000000007},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.8220000000000001,"15min":0.97499999999999998},"min":{"1min":0.42699999999999999,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":1.5640000000000001,"5min":57.991,"15min":57.991},"last":0.47499999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":1.222,"15min":0.79700000000000004},"min":{"1min":0.46200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85899999999999999,"5min":19.039000000000001,"15min":19.039000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.629,"5min":1.742,"15min":0.97099999999999997},"min":{"1min":0.44400000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.046,"5min":89.989999999999995,"15min":89.989999999999995},"last":0.68300000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:37:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.9059999999999999,"15min":1.0449999999999999},"min":{"1min":0.42099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.4100000000000001,"5min":90.025000000000006,"15min":90.025000000000006},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":1.798,"15min":0.99299999999999999},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.83899999999999997,"5min":57.962000000000003,"15min":57.962000000000003},"last":0.57999999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411464,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305916,"kb_used_data":4596,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439236,"statfs":{"total":95995035648,"available":95681777664,"internally_reserved":0,"allocated":4706304,"data_stored":2343866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.70699999999999996,"15min":0.48899999999999999},"min":{"1min":0.253,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46100000000000002,"5min":12.144,"15min":12.144},"last":0.379},{"interface":"front","average":{"1min":0.35299999999999998,"5min":1.056,"15min":0.60699999999999998},"min":{"1min":0.27000000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56200000000000006,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.253}]},{"osd":1,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.69699999999999995,"15min":0.495},"min":{"1min":0.28499999999999998,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.44700000000000001,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.81399999999999995,"15min":0.54400000000000004},"min":{"1min":0.29799999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.64300000000000002,"5min":18.919,"15min":18.919},"last":0.35899999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:37:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.89900000000000002,"15min":0.60699999999999998},"min":{"1min":0.308,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.5,"5min":22.712,"15min":22.712},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.88500000000000001,"15min":0.61699999999999999},"min":{"1min":0.33500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51100000000000001,"5min":22.359000000000002,"15min":22.359000000000002},"last":0.311}]},{"osd":4,"last update":"Sun Feb 18 00:37:13 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.89000000000000001,"15min":0.68500000000000005},"min":{"1min":0.41999999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.875,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.84499999999999997,"15min":0.67600000000000005},"min":{"1min":0.37,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76000000000000001,"5min":10.016,"15min":10.016},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:37:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.75800000000000001,"15min":0.65000000000000002},"min":{"1min":0.40400000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92800000000000005,"5min":6.766,"15min":6.766},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.84499999999999997,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.95999999999999996,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.77300000000000002,"15min":0.68300000000000005},"min":{"1min":0.38300000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":6.806,"15min":6.806},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.88,"15min":0.72999999999999998},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92100000000000004,"5min":18.861000000000001,"15min":18.861000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:37:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.80900000000000005,"15min":0.72399999999999998},"min":{"1min":0.44500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83699999999999997,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.90000000000000002,"15min":0.746},"min":{"1min":0.495,"5min":0.372,"15min":0.372},"max":{"1min":0.96399999999999997,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.73199999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149826,"num_pgs":118,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305760,"kb_used_data":8536,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93439392,"statfs":{"total":95995035648,"available":95681937408,"internally_reserved":0,"allocated":8740864,"data_stored":6353330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.65800000000000003,"15min":0.47699999999999998},"min":{"1min":0.254,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.82499999999999996,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.76600000000000001,"15min":0.52600000000000002},"min":{"1min":0.27900000000000003,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.2110000000000001,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.59999999999999998,"15min":0.47699999999999998},"min":{"1min":0.27300000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.59299999999999997,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":1.274,"15min":0.69599999999999995},"min":{"1min":0.26100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":8.0779999999999994,"5min":23.606999999999999,"15min":23.606999999999999},"last":0.28999999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":1.107,"15min":0.66000000000000003},"min":{"1min":0.30299999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":4.0190000000000001,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.54600000000000004,"15min":0.46899999999999997},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":1.4430000000000001,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.91800000000000004,"15min":0.69199999999999995},"min":{"1min":0.42499999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.8410000000000002,"5min":10.131,"15min":10.131},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.70399999999999996,"15min":0.628},"min":{"1min":0.39500000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.86499999999999999,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:37:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.69499999999999995,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.81000000000000005,"15min":0.67800000000000005},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78800000000000003,"5min":6.7750000000000004,"15min":6.7750000000000004},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:37:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":1.0329999999999999,"15min":0.74399999999999999},"min":{"1min":0.377,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75900000000000001,"5min":10.189,"15min":10.189},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.73899999999999999,"15min":0.66300000000000003},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88100000000000001,"5min":6.4279999999999999,"15min":6.4279999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:37:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.80000000000000004,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89300000000000002,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.96199999999999997,"15min":0.72199999999999998},"min":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.39800000000000002},"max":{"1min":3.9289999999999998,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.72599999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986301,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309952,"kb_used_data":4536,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435200,"statfs":{"total":95995035648,"available":95677644800,"internally_reserved":0,"allocated":4644864,"data_stored":2344785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":1.542,"15min":0.91100000000000003},"min":{"1min":0.312,"5min":0.31,"15min":0.31},"max":{"1min":0.92600000000000005,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.498,"5min":1.4950000000000001,"15min":0.872},"min":{"1min":0.28199999999999997,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79300000000000004,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.46300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.80900000000000005,"15min":0.64500000000000002},"min":{"1min":0.33900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89800000000000002,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":1.581,"15min":0.90100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.88,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.78000000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.022,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.90000000000000002,"15min":0.68100000000000005},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80200000000000005,"5min":17.77,"15min":17.77},"last":0.68400000000000005}]},{"osd":3,"last update":"Sun Feb 18 00:37:14 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.68999999999999995,"15min":0.627},"min":{"1min":0.318,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.82199999999999995,"5min":7.218,"15min":7.218},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":1.091,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.309,"15min":0.309},"max":{"1min":1.2350000000000001,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.435,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.246,"15min":0.20300000000000001},"max":{"1min":0.59899999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.40799999999999997},"min":{"1min":0.26500000000000001,"5min":0.248,"15min":0.24299999999999999},"max":{"1min":0.66200000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.34200000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:37:08 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45600000000000002,"15min":0.41899999999999998},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.76100000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.30099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.65700000000000003,"5min":0.751,"15min":0.751},"last":0.35299999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50600000000000001,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.76900000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.498,"15min":0.48599999999999999},"min":{"1min":0.34899999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73499999999999999,"5min":0.79700000000000004,"15min":0.83299999999999996},"last":0.432}]}]},{"osd":5,"up_from":39,"seq":167503724664,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309656,"kb_used_data":8336,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435496,"statfs":{"total":95995035648,"available":95677947904,"internally_reserved":0,"allocated":8536064,"data_stored":6344608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59499999999999997,"15min":0.55100000000000005},"min":{"1min":0.253,"5min":0.217,"15min":0.217},"max":{"1min":1.4630000000000001,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.624},"min":{"1min":0.29099999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":4.4779999999999998,"5min":12.744,"15min":12.744},"last":0.63}]},{"osd":1,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.71599999999999997,"15min":0.624},"min":{"1min":0.33800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.151,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.77100000000000002,"15min":0.63100000000000001},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":1.127,"5min":6.4829999999999997,"15min":6.4829999999999997},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.67000000000000004,"15min":0.61399999999999999},"min":{"1min":0.311,"5min":0.251,"15min":0.251},"max":{"1min":0.77000000000000002,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.72899999999999998,"15min":0.64200000000000002},"min":{"1min":0.41999999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.1000000000000001,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.67300000000000004}]},{"osd":3,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.73599999999999999,"15min":0.61899999999999999},"min":{"1min":0.35599999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.80300000000000005,"5min":5.9880000000000004,"15min":5.9880000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.67800000000000005,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83099999999999996,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.58999999999999997,"15min":0.45200000000000001},"min":{"1min":0.28699999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.67200000000000004,"5min":11.818,"15min":11.818},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.50600000000000001,"15min":0.433},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.72999999999999998,"5min":4.274,"15min":4.274},"last":0.56100000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:37:54 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.73499999999999999,"15min":0.52700000000000002},"min":{"1min":0.27500000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.71299999999999997,"5min":18.605,"15min":18.605},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52800000000000002,"15min":0.46500000000000002},"min":{"1min":0.39100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":4.2089999999999996,"15min":4.2089999999999996},"last":0.51100000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:37:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.47399999999999998},"min":{"1min":0.376,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68899999999999995,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56399999999999995,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.70299999999999996,"5min":4.649,"15min":4.649},"last":0.66400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4333568,"data_stored":4000786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4149248,"data_stored":4000347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4247552,"data_stored":4000585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":209,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":209,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":209,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":209,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":209,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":209,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":10,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:38:08.393 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:08.395 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:08.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:08.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:38:08.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:08.565 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:08.568 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:08.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:08.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:08.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.697+0000 7efd9e5f6700 1 -- 172.21.15.59:0/2572761174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980fcbb0 msgr2=0x7efd980fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.697+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/2572761174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980fcbb0 0x7efd980fcf90 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 rx=0x7efd880047b0 tx=0x7efd88009cb0).stop 2024-02-18T00:38:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.697+0000 7efd9e5f6700 1 -- 172.21.15.59:0/2572761174 shutdown_connections 2024-02-18T00:38:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.697+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/2572761174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd980f7400 0x7efd98103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.697+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/2572761174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980fcbb0 0x7efd980fcf90 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.697+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/2572761174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd980fd510 0x7efd980f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.697+0000 7efd9e5f6700 1 -- 172.21.15.59:0/2572761174 >> 172.21.15.59:0/2572761174 conn(0x7efd980f2820 msgr2=0x7efd980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:08.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.698+0000 7efd9e5f6700 1 -- 172.21.15.59:0/2572761174 shutdown_connections 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.698+0000 7efd9e5f6700 1 -- 172.21.15.59:0/2572761174 wait complete. 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.698+0000 7efd9e5f6700 1 librados: starting msgr at 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.698+0000 7efd9e5f6700 1 librados: starting objecter 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 Processor -- start 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 -- start start 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 librados: setting wanted keys 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 librados: calling monclient init 2024-02-18T00:38:08.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980f7400 0x7efd98103420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd980fcbb0 0x7efd98103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd980fd510 0x7efd980fda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd980fe0d0 con 0x7efd980f7400 2024-02-18T00:38:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd980fe240 con 0x7efd980fcbb0 2024-02-18T00:38:08.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.699+0000 7efd9e5f6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd980fe400 con 0x7efd980fd510 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd9cb93700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd980fd510 0x7efd980fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd97fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980f7400 0x7efd98103420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd9cb93700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd980fd510 0x7efd980fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:32882/0 (socket says 172.21.15.59:32882) 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd9cb93700 1 -- 172.21.15.59:0/824067075 learned_addr learned my addr 172.21.15.59:0/824067075 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd97fff700 1 -- 172.21.15.59:0/824067075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd980fd510 msgr2=0x7efd980fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd980fd510 0x7efd980fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd97fff700 1 -- 172.21.15.59:0/824067075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd980fcbb0 msgr2=0x7efd98103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd980fcbb0 0x7efd98103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd97fff700 1 -- 172.21.15.59:0/824067075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88005040 con 0x7efd980f7400 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980f7400 0x7efd98103420 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7efd880047b0 tx=0x7efd88009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd957fa700 10 client.?.objecter ms_handle_connect 0x7efd980f7400 2024-02-18T00:38:08.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.700+0000 7efd957fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:08.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.701+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88018070 con 0x7efd980f7400 2024-02-18T00:38:08.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.701+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd88013da0 con 0x7efd980f7400 2024-02-18T00:38:08.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.701+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8801dc10 con 0x7efd980f7400 2024-02-18T00:38:08.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.701+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd980fe620 con 0x7efd980f7400 2024-02-18T00:38:08.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.701+0000 7efd9e5f6700 10 client.31284.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:08.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.701+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd98112de0 con 0x7efd980f7400 2024-02-18T00:38:08.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.701+0000 7efd9e5f6700 1 librados: init done 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.702+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7efd9805fa60 con 0x7efd980f7400 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.704+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efd8801d420 con 0x7efd980f7400 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.704+0000 7efd957fa700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd80062ce0 0x7efd800651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.704+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(407..407 src has 180..407) v4 ==== 8006+0+0 (secure 0 0 0) 0x7efd8807e580 con 0x7efd980f7400 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.704+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 osd_map(407..407 src has 180..407) v4 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.704+0000 7efd957fa700 3 client.31284.objecter handle_osd_map got epochs [407,407] > 0 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.704+0000 7efd957fa700 3 client.31284.objecter handle_osd_map decoding full epoch 407 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.705+0000 7efd957fa700 20 client.31284.objecter dump_active .. 0 homeless 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.706+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v407) v1 ==== 54+0+16 (secure 0 0 0) 0x7efd88085070 con 0x7efd980f7400 2024-02-18T00:38:08.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.706+0000 7efd977fe700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd80062ce0 0x7efd800651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.707+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd80062ce0 2024-02-18T00:38:08.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.707+0000 7efd977fe700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd80062ce0 0x7efd800651a0 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 rx=0x7efd9001ae50 tx=0x7efd9001c040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 1 2024-02-18T00:38:08.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd98065000 0x7efd980653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 1 2024-02-18T00:38:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 0 1 2024-02-18T00:38:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9805ee80 osd=0 4 2024-02-18T00:38:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 10 client.31284.objecter _send_command 1 2024-02-18T00:38:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.708+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd98064490 con 0x7efd98065000 2024-02-18T00:38:08.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.710+0000 7efd9cb93700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd98065000 0x7efd980653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.710+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd98065000 2024-02-18T00:38:08.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.710+0000 7efd9cb93700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd98065000 0x7efd980653e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.712+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd98064490 con 0x7efd98065000 2024-02-18T00:38:08.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.712+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:38:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.712+0000 7efd957fa700 10 client.31284.objecter _finish_command 1 = system:0 2024-02-18T00:38:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.712+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd9805ee80 osd=0 4 2024-02-18T00:38:08.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.712+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 0 1 2024-02-18T00:38:08.729 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:08.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 2 2024-02-18T00:38:08.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 2 2024-02-18T00:38:08.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd9805ee80 osd=0 3 2024-02-18T00:38:08.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 0 2 2024-02-18T00:38:08.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9805ee80 osd=0 4 2024-02-18T00:38:08.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 10 client.31284.objecter _send_command 2 2024-02-18T00:38:08.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.734+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd980273c0 con 0x7efd98065000 2024-02-18T00:38:08.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.740+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd980273c0 con 0x7efd98065000 2024-02-18T00:38:08.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.740+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.740+0000 7efd957fa700 10 client.31284.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.740+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd9805ee80 osd=0 4 2024-02-18T00:38:08.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.740+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 0 2 2024-02-18T00:38:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 3 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6805/26279110 conn(0x7efd980598f0 0x7efd98059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 3 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 1 3 2024-02-18T00:38:08.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98028710 osd=1 4 2024-02-18T00:38:08.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 10 client.31284.objecter _send_command 3 2024-02-18T00:38:08.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.741+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd981196f0 con 0x7efd980598f0 2024-02-18T00:38:08.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.746+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6805/26279110 conn(0x7efd980598f0 0x7efd98059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.746+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6805/26279110 conn(0x7efd980598f0 0x7efd98059cf0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.748+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd980598f0 2024-02-18T00:38:08.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.748+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd981196f0 con 0x7efd980598f0 2024-02-18T00:38:08.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.748+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:38:08.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.748+0000 7efd957fa700 10 client.31284.objecter _finish_command 3 = system:0 2024-02-18T00:38:08.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.748+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98028710 osd=1 4 2024-02-18T00:38:08.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.748+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 1 3 2024-02-18T00:38:08.757 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:08.757 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:08.763 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:08.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:08.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.768+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 4 2024-02-18T00:38:08.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 4 2024-02-18T00:38:08.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98028710 osd=1 3 2024-02-18T00:38:08.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 1 4 2024-02-18T00:38:08.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98028710 osd=1 4 2024-02-18T00:38:08.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 10 client.31284.objecter _send_command 4 2024-02-18T00:38:08.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.769+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd98059360 con 0x7efd980598f0 2024-02-18T00:38:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd98059360 con 0x7efd980598f0 2024-02-18T00:38:08.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd957fa700 10 client.31284.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98028710 osd=1 4 2024-02-18T00:38:08.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 1 4 2024-02-18T00:38:08.773 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 5 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd98028270 0x7efd98118110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 5 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 2 5 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98059740 osd=2 4 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 10 client.31284.objecter _send_command 5 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd98118600 con 0x7efd98028270 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.771+0000 7efd977fe700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd98028270 0x7efd98118110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.772+0000 7efd977fe700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd98028270 0x7efd98118110 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.772+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd98028270 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.772+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd98118600 con 0x7efd98028270 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.772+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.772+0000 7efd957fa700 10 client.31284.objecter _finish_command 5 = system:0 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.772+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98059740 osd=2 4 2024-02-18T00:38:08.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.772+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 2 5 2024-02-18T00:38:08.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 6 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 6 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98059740 osd=2 3 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 2 6 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98059740 osd=2 4 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 10 client.31284.objecter _send_command 6 2024-02-18T00:38:08.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.788+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd980277d0 con 0x7efd98028270 2024-02-18T00:38:08.791 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:38:08.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.789+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd980277d0 con 0x7efd98028270 2024-02-18T00:38:08.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.789+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.789+0000 7efd957fa700 10 client.31284.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.789+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98059740 osd=2 4 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.789+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 2 6 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 7 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd9811b870 0x7efd9811dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 7 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 3 7 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98027bb0 osd=3 4 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 10 client.31284.objecter _send_command 7 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd98119500 con 0x7efd9811b870 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9cb93700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd9811b870 0x7efd9811dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd9cb93700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd9811b870 0x7efd9811dc60 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.790+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd9811b870 2024-02-18T00:38:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.791+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd98119500 con 0x7efd9811b870 2024-02-18T00:38:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.791+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:38:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.791+0000 7efd957fa700 10 client.31284.objecter _finish_command 7 = system:0 2024-02-18T00:38:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.791+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98027bb0 osd=3 4 2024-02-18T00:38:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.791+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 3 7 2024-02-18T00:38:08.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 8 2024-02-18T00:38:08.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 8 2024-02-18T00:38:08.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98027bb0 osd=3 3 2024-02-18T00:38:08.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 3 8 2024-02-18T00:38:08.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98027bb0 osd=3 4 2024-02-18T00:38:08.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 10 client.31284.objecter _send_command 8 2024-02-18T00:38:08.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.806+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd98118a60 con 0x7efd9811b870 2024-02-18T00:38:08.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd98118a60 con 0x7efd9811b870 2024-02-18T00:38:08.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd957fa700 10 client.31284.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98027bb0 osd=3 4 2024-02-18T00:38:08.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 3 8 2024-02-18T00:38:08.810 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:38:08.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 9 2024-02-18T00:38:08.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd98118c00 0x7efd9811f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 9 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 4 9 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98118a60 osd=4 4 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 10 client.31284.objecter _send_command 9 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd98117e50 con 0x7efd98118c00 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.808+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd98118c00 0x7efd9811f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.809+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd98118c00 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.809+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd98118c00 0x7efd9811f2d0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.809+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd98117e50 con 0x7efd98118c00 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.809+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.809+0000 7efd957fa700 10 client.31284.objecter _finish_command 9 = system:0 2024-02-18T00:38:08.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.810+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98118a60 osd=4 4 2024-02-18T00:38:08.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.810+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 4 9 2024-02-18T00:38:08.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 10 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 10 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98118a60 osd=4 3 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 4 10 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98118a60 osd=4 4 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 10 client.31284.objecter _send_command 10 2024-02-18T00:38:08.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.830+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd98117e50 con 0x7efd98118c00 2024-02-18T00:38:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd98117e50 con 0x7efd98118c00 2024-02-18T00:38:08.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd957fa700 10 client.31284.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98118a60 osd=4 4 2024-02-18T00:38:08.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 4 10 2024-02-18T00:38:08.834 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 11 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd9811fcd0 0x7efd98120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 11 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 5 11 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98117e50 osd=5 4 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 10 client.31284.objecter _send_command 11 2024-02-18T00:38:08.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd98127060 con 0x7efd9811fcd0 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.832+0000 7efd977fe700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd9811fcd0 0x7efd98120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.833+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd9811fcd0 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.833+0000 7efd977fe700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd9811fcd0 0x7efd98120130 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.833+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd98127060 con 0x7efd9811fcd0 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.833+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.833+0000 7efd957fa700 10 client.31284.objecter _finish_command 11 = system:0 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.833+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98117e50 osd=5 4 2024-02-18T00:38:08.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.833+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 5 11 2024-02-18T00:38:08.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 12 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 12 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98117e50 osd=5 3 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 5 12 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98117e50 osd=5 4 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 10 client.31284.objecter _send_command 12 2024-02-18T00:38:08.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.848+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd98124900 con 0x7efd9811fcd0 2024-02-18T00:38:08.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd98124900 con 0x7efd9811fcd0 2024-02-18T00:38:08.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd957fa700 10 client.31284.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98117e50 osd=5 4 2024-02-18T00:38:08.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 5 12 2024-02-18T00:38:08.853 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 13 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd98124f80 0x7efd98127370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 13 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 6 13 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98124900 osd=6 4 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 10 client.31284.objecter _send_command 13 2024-02-18T00:38:08.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd9812a120 con 0x7efd98124f80 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9cb93700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd98124f80 0x7efd98127370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd98124f80 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.850+0000 7efd9cb93700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd98124f80 0x7efd98127370 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.851+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd9812a120 con 0x7efd98124f80 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.851+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.851+0000 7efd957fa700 10 client.31284.objecter _finish_command 13 = system:0 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.851+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98124900 osd=6 4 2024-02-18T00:38:08.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.851+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 6 13 2024-02-18T00:38:08.864 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 14 2024-02-18T00:38:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 14 2024-02-18T00:38:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd98124900 osd=6 3 2024-02-18T00:38:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 6 14 2024-02-18T00:38:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98124900 osd=6 4 2024-02-18T00:38:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 10 client.31284.objecter _send_command 14 2024-02-18T00:38:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.866+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd9812a120 con 0x7efd98124f80 2024-02-18T00:38:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd9812a120 con 0x7efd98124f80 2024-02-18T00:38:08.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd957fa700 10 client.31284.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd98124900 osd=6 4 2024-02-18T00:38:08.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 6 14 2024-02-18T00:38:08.870 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:38:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 15 2024-02-18T00:38:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6812/53025171 conn(0x7efd98129420 0x7efd981298a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 15 2024-02-18T00:38:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 7 15 2024-02-18T00:38:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9812a120 osd=7 4 2024-02-18T00:38:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 10 client.31284.objecter _send_command 15 2024-02-18T00:38:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd98129ec0 con 0x7efd98129420 2024-02-18T00:38:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6812/53025171 conn(0x7efd98129420 0x7efd981298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:08.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd957fa700 10 client.31284.objecter ms_handle_connect 0x7efd98129420 2024-02-18T00:38:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.868+0000 7efd97fff700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6812/53025171 conn(0x7efd98129420 0x7efd981298a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.869+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd98129ec0 con 0x7efd98129420 2024-02-18T00:38:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.869+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:38:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.869+0000 7efd957fa700 10 client.31284.objecter _finish_command 15 = system:0 2024-02-18T00:38:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.869+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd9812a120 osd=7 4 2024-02-18T00:38:08.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.869+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 7 15 2024-02-18T00:38:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 10 client.31284.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign -1 16 2024-02-18T00:38:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 20 client.31284.objecter _get_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_remove -1 16 2024-02-18T00:38:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 20 client.31284.objecter get_session s=0x7efd9812a120 osd=7 3 2024-02-18T00:38:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 15 client.31284.objecter _session_command_op_assign 7 16 2024-02-18T00:38:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9812a120 osd=7 4 2024-02-18T00:38:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 10 client.31284.objecter _send_command 16 2024-02-18T00:38:08.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.886+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd98129ec0 con 0x7efd98129420 2024-02-18T00:38:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.888+0000 7efd957fa700 1 -- 172.21.15.59:0/824067075 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd98129ec0 con 0x7efd98129420 2024-02-18T00:38:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.888+0000 7efd957fa700 10 client.31284.objecter ms_dispatch 0x7efd98001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.888+0000 7efd957fa700 10 client.31284.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.888+0000 7efd957fa700 20 client.31284.objecter put_session s=0x7efd9812a120 osd=7 4 2024-02-18T00:38:08.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.888+0000 7efd957fa700 15 client.31284.objecter _session_command_op_remove 7 16 2024-02-18T00:38:08.891 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 librados: watch_flush enter 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 librados: watch_flush exit 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.0 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd98065000 msgr2=0x7efd980653e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd98065000 0x7efd980653e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9805ee80 osd=0 1 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.1 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6805/26279110 conn(0x7efd980598f0 msgr2=0x7efd98059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6805/26279110 conn(0x7efd980598f0 0x7efd98059cf0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98028710 osd=1 1 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.2 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd98028270 msgr2=0x7efd98118110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd98028270 0x7efd98118110 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98059740 osd=2 1 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.3 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd9811b870 msgr2=0x7efd9811dc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd9811b870 0x7efd9811dc60 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9cb93700 1 -- 172.21.15.59:0/824067075 reap_dead start 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98027bb0 osd=3 1 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.4 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd98118c00 msgr2=0x7efd9811f2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd98118c00 0x7efd9811f2d0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9cb93700 1 -- 172.21.15.59:0/824067075 reap_dead start 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98118a60 osd=4 1 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.5 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd9811fcd0 msgr2=0x7efd98120130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd9811fcd0 0x7efd98120130 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98117e50 osd=5 1 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.6 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd98124f80 msgr2=0x7efd98127370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd98124f80 0x7efd98127370 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd98124900 osd=6 1 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter close_session for osd.7 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6812/53025171 conn(0x7efd98129420 msgr2=0x7efd981298a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.137:6812/53025171 conn(0x7efd98129420 0x7efd981298a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter put_session s=0x7efd9812a120 osd=7 1 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 20 client.31284.objecter shutdown clearing up homeless session... 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 10 client.31284.objecter successfully canceled tick 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd80062ce0 msgr2=0x7efd800651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd80062ce0 0x7efd800651a0 secure :-1 s=READY pgs=3101 cs=0 l=1 rev1=1 rx=0x7efd9001ae50 tx=0x7efd9001c040).stop 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980f7400 msgr2=0x7efd98103420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 --2- 172.21.15.59:0/824067075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd980f7400 0x7efd98103420 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7efd880047b0 tx=0x7efd88009cb0).stop 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9cb93700 1 -- 172.21.15.59:0/824067075 reap_dead start 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 shutdown_connections 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 >> 172.21.15.59:0/824067075 conn(0x7efd980f2820 msgr2=0x7efd980f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 shutdown_connections 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 -- 172.21.15.59:0/824067075 wait complete. 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:08.889+0000 7efd9e5f6700 1 librados: shutdown 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.895 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:38:08.896 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:07.722059+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e406: 8 total, 8 up, 7 in 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:07.894934+0000 mgr.y (mgr.24454) 381 : cluster [DBG] pgmap v598: 206 pgs: 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:08.670007+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e407: 8 total, 8 up, 7 in 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:08.679553+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:08.688502+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:08.706909+0000 mon.a (mon.0) 2484 : audit [DBG] from='client.? 172.21.15.59:0/824067075' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:07.722059+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e406: 8 total, 8 up, 7 in 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:07.894934+0000 mgr.y (mgr.24454) 381 : cluster [DBG] pgmap v598: 206 pgs: 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:08.670007+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e407: 8 total, 8 up, 7 in 2024-02-18T00:38:09.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:08.679553+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:08.688502+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:08.706909+0000 mon.a (mon.0) 2484 : audit [DBG] from='client.? 172.21.15.59:0/824067075' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:09.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:07.722059+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e406: 8 total, 8 up, 7 in 2024-02-18T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:07.894934+0000 mgr.y (mgr.24454) 381 : cluster [DBG] pgmap v598: 206 pgs: 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:08.670007+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e407: 8 total, 8 up, 7 in 2024-02-18T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:08.679553+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:08.688502+0000 mon.a (mon.0) 2483 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:08.706909+0000 mon.a (mon.0) 2484 : audit [DBG] from='client.? 172.21.15.59:0/824067075' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:09.745 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:09.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:09.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:09.849 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:09.880 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:09.902 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:09.905 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:09.905 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:10.006 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:10.037 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:10.054 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:10.057 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:10.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:10.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:10.186 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:10.205 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:10.207 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:10.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:10.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:10.325 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:10.342 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:10.364 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:10.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:10.369 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:10.470 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:10.497 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:10.515 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:10.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:10.518 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:10.619 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:10.648 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:10.664 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:10.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:10.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:09.671079+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:09.671187+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e408: 8 total, 8 up, 7 in 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:09.677178+0000 mon.c (mon.2) 401 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:09.718099+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:09.718462+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:09.718921+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-57"}]: dispatch 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:09.719215+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-57"}]: dispatch 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:09.895291+0000 mgr.y (mgr.24454) 382 : cluster [DBG] pgmap v601: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:10.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:09.902112+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:10.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:10.797 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:09.671079+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:09.671187+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e408: 8 total, 8 up, 7 in 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:09.677178+0000 mon.c (mon.2) 401 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:09.718099+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:09.718462+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:09.718921+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-57"}]: dispatch 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:09.719215+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-57"}]: dispatch 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:09.895291+0000 mgr.y (mgr.24454) 382 : cluster [DBG] pgmap v601: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:09.902112+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:09.671079+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:09.671187+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e408: 8 total, 8 up, 7 in 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:09.677178+0000 mon.c (mon.2) 401 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:09.718099+0000 mon.c (mon.2) 402 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:09.718462+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:09.718921+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-57"}]: dispatch 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:09.719215+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-57"}]: dispatch 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:09.895291+0000 mgr.y (mgr.24454) 382 : cluster [DBG] pgmap v601: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:11.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:09.902112+0000 mon.a (mon.0) 2489 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:11.048 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:11.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:11.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:11.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:11.179 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:11.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:38:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:38:11.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:38:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:38:12.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:10.678471+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e409: 8 total, 8 up, 7 in 2024-02-18T00:38:12.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:11.230856+0000 mgr.y (mgr.24454) 383 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:10.678471+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e409: 8 total, 8 up, 7 in 2024-02-18T00:38:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:11.230856+0000 mgr.y (mgr.24454) 383 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:12.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:10.678471+0000 mon.a (mon.0) 2490 : cluster [DBG] osdmap e409: 8 total, 8 up, 7 in 2024-02-18T00:38:12.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:11.230856+0000 mgr.y (mgr.24454) 383 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:12.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:38:12.431 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:12] "GET /metrics HTTP/1.1" 200 189469 "" "Prometheus/2.33.4" 2024-02-18T00:38:12.434 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:12.437 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:12.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:12.538 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:12.570 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:12.591 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:12.594 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:12.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:12.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:12.745 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:12.771 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:12.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:12.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:12.903 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:11.683562+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e410: 8 total, 8 up, 7 in 2024-02-18T00:38:12.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:11.694663+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:12.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:11.695120+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:12.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:11.895725+0000 mgr.y (mgr.24454) 384 : cluster [DBG] pgmap v604: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:38:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:12.922 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:12.925 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:12.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:11.683562+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e410: 8 total, 8 up, 7 in 2024-02-18T00:38:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:11.694663+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:11.695120+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:11.895725+0000 mgr.y (mgr.24454) 384 : cluster [DBG] pgmap v604: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:38:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:11.683562+0000 mon.a (mon.0) 2491 : cluster [DBG] osdmap e410: 8 total, 8 up, 7 in 2024-02-18T00:38:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:11.694663+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:11.695120+0000 mon.a (mon.0) 2492 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:11.895725+0000 mgr.y (mgr.24454) 384 : cluster [DBG] pgmap v604: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:38:13.024 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:38:13.025 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:38:13.026 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:13.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:13.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:13.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:13.189 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:38:13.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 0 1 2024-02-18T00:38:13.207 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:13.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 -- 172.21.15.59:0/2824240798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f73d0 msgr2=0x7f7130104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:13.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 --2- 172.21.15.59:0/2824240798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f73d0 0x7f7130104170 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f7124004870 tx=0x7f7124009cb0).stop 2024-02-18T00:38:13.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 -- 172.21.15.59:0/2824240798 shutdown_connections 2024-02-18T00:38:13.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 --2- 172.21.15.59:0/2824240798 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71300f6b00 0x7f71300f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 --2- 172.21.15.59:0/2824240798 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7130104660 0x7f7130106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 --2- 172.21.15.59:0/2824240798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f73d0 0x7f7130104170 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 -- 172.21.15.59:0/2824240798 >> 172.21.15.59:0/2824240798 conn(0x7f71300f2820 msgr2=0x7f71300f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 -- 172.21.15.59:0/2824240798 shutdown_connections 2024-02-18T00:38:13.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.287+0000 7f71359a2700 1 -- 172.21.15.59:0/2824240798 wait complete. 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 librados: starting msgr at 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 librados: starting objecter 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 Processor -- start 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 -- start start 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 librados: setting wanted keys 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 librados: calling monclient init 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f6b00 0x7f71301103d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71300f73d0 0x7f71301108c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7130104660 0x7f713010a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:13.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713010aa60 con 0x7f71300f6b00 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713010abd0 con 0x7f7130104660 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f71359a2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713010ad90 con 0x7f71300f73d0 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.288+0000 7f712effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f6b00 0x7f71301103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f6b00 0x7f71301103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55838/0 (socket says 172.21.15.59:55838) 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 -- 172.21.15.59:0/1230711139 learned_addr learned my addr 172.21.15.59:0/1230711139 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712f7fe700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7130104660 0x7f713010a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712e7fc700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71300f73d0 0x7f71301108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 -- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71300f73d0 msgr2=0x7f71301108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71300f73d0 0x7f71301108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 -- 172.21.15.59:0/1230711139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7130104660 msgr2=0x7f713010a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7130104660 0x7f713010a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 -- 172.21.15.59:0/1230711139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124005040 con 0x7f71300f6b00 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712e7fc700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71300f73d0 0x7f71301108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f712effd700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f6b00 0x7f71301103d0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 rx=0x7f71200027f0 tx=0x7f7120002b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f7117fff700 10 client.?.objecter ms_handle_connect 0x7f71300f6b00 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f7117fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f7117fff700 1 -- 172.21.15.59:0/1230711139 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120007b40 con 0x7f71300f6b00 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713010b070 con 0x7f71300f6b00 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f7117fff700 1 -- 172.21.15.59:0/1230711139 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712000f040 con 0x7f71300f6b00 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f71359a2700 10 client.31290.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f7117fff700 1 -- 172.21.15.59:0/1230711139 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000d7e0 con 0x7f71300f6b00 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713010b3f0 con 0x7f71300f6b00 2024-02-18T00:38:13.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.289+0000 7f71359a2700 1 librados: init done 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.290+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713005fa60 con 0x7f71300f6b00 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.291+0000 7f7117fff700 1 -- 172.21.15.59:0/1230711139 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f712000dc10 con 0x7f71300f6b00 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.291+0000 7f7117fff700 1 --2- 172.21.15.59:0/1230711139 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7118062c90 0x7f7118065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.291+0000 7f7117fff700 1 -- 172.21.15.59:0/1230711139 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(411..411 src has 180..411) v4 ==== 8019+0+0 (secure 0 0 0) 0x7f712007d030 con 0x7f71300f6b00 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.291+0000 7f7117fff700 10 client.31290.objecter ms_dispatch 0x7f7130001550 osd_map(411..411 src has 180..411) v4 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.291+0000 7f7117fff700 3 client.31290.objecter handle_osd_map got epochs [411,411] > 0 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.291+0000 7f7117fff700 3 client.31290.objecter handle_osd_map decoding full epoch 411 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.291+0000 7f712e7fc700 1 --2- 172.21.15.59:0/1230711139 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7118062c90 0x7f7118065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.292+0000 7f7117fff700 20 client.31290.objecter dump_active .. 0 homeless 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.292+0000 7f7117fff700 10 client.31290.objecter ms_handle_connect 0x7f7118062c90 2024-02-18T00:38:13.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.292+0000 7f712e7fc700 1 --2- 172.21.15.59:0/1230711139 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7118062c90 0x7f7118065150 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 rx=0x7f7124009da0 tx=0x7f7124007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:13.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.296+0000 7f7117fff700 1 -- 172.21.15.59:0/1230711139 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7120050500 con 0x7f71300f6b00 2024-02-18T00:38:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:13.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:13.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:13.413 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:13.442 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:13.461 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:13.464 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:13.464 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:13.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.479+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f713005ee80 con 0x7f71300f6b00 2024-02-18T00:38:13.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:13.592 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:13.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.692+0000 7f7117fff700 1 -- 172.21.15.59:0/1230711139 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v412) v1 ==== 91+0+0 (secure 0 0 0) 0x7f713005ee80 con 0x7f71300f6b00 2024-02-18T00:38:13.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.695+0000 7f71359a2700 10 librados: watch_flush enter 2024-02-18T00:38:13.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 10 librados: watch_flush exit 2024-02-18T00:38:13.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 20 client.31290.objecter shutdown clearing up homeless session... 2024-02-18T00:38:13.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 10 client.31290.objecter successfully canceled tick 2024-02-18T00:38:13.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7118062c90 msgr2=0x7f7118065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:13.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 --2- 172.21.15.59:0/1230711139 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7118062c90 0x7f7118065150 secure :-1 s=READY pgs=3102 cs=0 l=1 rev1=1 rx=0x7f7124009da0 tx=0x7f7124007f90).stop 2024-02-18T00:38:13.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f6b00 msgr2=0x7f71301103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:13.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f6b00 0x7f71301103d0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 rx=0x7f71200027f0 tx=0x7f7120002b00).stop 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 shutdown_connections 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 --2- 172.21.15.59:0/1230711139 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7118062c90 0x7f7118065150 unknown :-1 s=CLOSED pgs=3102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7130104660 0x7f713010a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71300f6b00 0x7f71301103d0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 --2- 172.21.15.59:0/1230711139 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f71300f73d0 0x7f71301108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 >> 172.21.15.59:0/1230711139 conn(0x7f71300f2820 msgr2=0x7f71300f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 shutdown_connections 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 -- 172.21.15.59:0/1230711139 wait complete. 2024-02-18T00:38:13.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:13.697+0000 7f71359a2700 1 librados: shutdown 2024-02-18T00:38:13.701 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:38:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:12.688919+0000 mon.a (mon.0) 2493 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:12.689009+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e411: 8 total, 8 up, 7 in 2024-02-18T00:38:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:12.693137+0000 mon.c (mon.2) 405 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:12.696571+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:12.701650+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:13.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:13.480990+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.59:0/1230711139' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:12.688919+0000 mon.a (mon.0) 2493 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:12.689009+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e411: 8 total, 8 up, 7 in 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:12.693137+0000 mon.c (mon.2) 405 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:12.696571+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:12.701650+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:13.480990+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.59:0/1230711139' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:12.688919+0000 mon.a (mon.0) 2493 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:12.689009+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e411: 8 total, 8 up, 7 in 2024-02-18T00:38:14.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:12.693137+0000 mon.c (mon.2) 405 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:14.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:12.696571+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:14.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:12.701650+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:14.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:13.480990+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? 172.21.15.59:0/1230711139' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:14.027 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-18T00:38:14.041 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:14.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:14.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:14.145 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:14.171 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:14.190 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:14.192 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:14.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:14.293 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:14.298 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:14.321 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:14.571 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:14.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:14.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:14.675 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:14.704 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:13.686001+0000 mon.a (mon.0) 2497 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:13.693127+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:13.693223+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? 172.21.15.59:0/1230711139' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:13.693266+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e412: 8 total, 8 up, 7 in 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:13.724940+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:13.725289+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:13.725774+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-59"}]: dispatch 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:13.726008+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-59"}]: dispatch 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:13.744833+0000 osd.6 (osd.6) 189 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:13.745548+0000 osd.6 (osd.6) 190 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:38:14.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:13.896302+0000 mgr.y (mgr.24454) 385 : cluster [DBG] pgmap v607: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:13.686001+0000 mon.a (mon.0) 2497 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:13.693127+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:13.693223+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? 172.21.15.59:0/1230711139' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:13.693266+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e412: 8 total, 8 up, 7 in 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:13.724940+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:13.725289+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:13.725774+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-59"}]: dispatch 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:13.726008+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-59"}]: dispatch 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:13.744833+0000 osd.6 (osd.6) 189 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:13.745548+0000 osd.6 (osd.6) 190 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:38:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:13.896302+0000 mgr.y (mgr.24454) 385 : cluster [DBG] pgmap v607: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:38:14.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:13.686001+0000 mon.a (mon.0) 2497 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:14.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:13.693127+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:14.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:13.693223+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? 172.21.15.59:0/1230711139' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:38:14.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:13.693266+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e412: 8 total, 8 up, 7 in 2024-02-18T00:38:14.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:13.724940+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:14.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:13.725289+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:14.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:13.725774+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-59"}]: dispatch 2024-02-18T00:38:14.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:13.726008+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-59"}]: dispatch 2024-02-18T00:38:14.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:13.744833+0000 osd.6 (osd.6) 189 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:38:14.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:13.745548+0000 osd.6 (osd.6) 190 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:38:14.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:13.896302+0000 mgr.y (mgr.24454) 385 : cluster [DBG] pgmap v607: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T00:38:14.923 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:14.923 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:15.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:14.256626+0000 osd.4 (osd.4) 231 : cluster [DBG] 4.d scrub starts 2024-02-18T00:38:15.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:14.260069+0000 osd.4 (osd.4) 232 : cluster [DBG] 4.d scrub ok 2024-02-18T00:38:15.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:14.699578+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e413: 8 total, 8 up, 7 in 2024-02-18T00:38:15.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:14.256626+0000 osd.4 (osd.4) 231 : cluster [DBG] 4.d scrub starts 2024-02-18T00:38:15.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:14.260069+0000 osd.4 (osd.4) 232 : cluster [DBG] 4.d scrub ok 2024-02-18T00:38:15.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:14.699578+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e413: 8 total, 8 up, 7 in 2024-02-18T00:38:15.906 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:14.256626+0000 osd.4 (osd.4) 231 : cluster [DBG] 4.d scrub starts 2024-02-18T00:38:15.906 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:14.260069+0000 osd.4 (osd.4) 232 : cluster [DBG] 4.d scrub ok 2024-02-18T00:38:15.906 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:14.699578+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e413: 8 total, 8 up, 7 in 2024-02-18T00:38:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:15.958 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:15.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:16.059 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:16.069 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:38:16.069 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:38:16.090 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:16.240 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:16.243 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:16.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:16.345 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:16.376 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:16.769 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-18T00:38:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:16.826 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:16.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:16.854 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:38:16.928 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:16.954 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:16.973 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:16.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:15.533014+0000 osd.1 (osd.1) 205 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:38:16.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:15.533978+0000 osd.1 (osd.1) 206 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:38:16.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:15.657136+0000 osd.6 (osd.6) 191 : cluster [DBG] 25.d scrub starts 2024-02-18T00:38:16.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:15.658200+0000 osd.6 (osd.6) 192 : cluster [DBG] 25.d scrub ok 2024-02-18T00:38:16.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:15.705464+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e414: 8 total, 8 up, 7 in 2024-02-18T00:38:16.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:15.712294+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:16.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:15.712792+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:16.976 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:15.897376+0000 mgr.y (mgr.24454) 386 : cluster [DBG] pgmap v610: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:16.976 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:16.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:15.533014+0000 osd.1 (osd.1) 205 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:15.533978+0000 osd.1 (osd.1) 206 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:15.657136+0000 osd.6 (osd.6) 191 : cluster [DBG] 25.d scrub starts 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:15.658200+0000 osd.6 (osd.6) 192 : cluster [DBG] 25.d scrub ok 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:15.705464+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e414: 8 total, 8 up, 7 in 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:15.712294+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:15.712792+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:15.897376+0000 mgr.y (mgr.24454) 386 : cluster [DBG] pgmap v610: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:15.533014+0000 osd.1 (osd.1) 205 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:15.533978+0000 osd.1 (osd.1) 206 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:15.657136+0000 osd.6 (osd.6) 191 : cluster [DBG] 25.d scrub starts 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:15.658200+0000 osd.6 (osd.6) 192 : cluster [DBG] 25.d scrub ok 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:15.705464+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e414: 8 total, 8 up, 7 in 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:15.712294+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:15.712792+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:15.897376+0000 mgr.y (mgr.24454) 386 : cluster [DBG] pgmap v610: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:17.076 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:17.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:17.105 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:17.353 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:17.356 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:17.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:17.457 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:17.484 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:17.536 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:17.539 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:17.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:17.640 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:17.667 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:17.738 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:38:17.738 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:38:17.738 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:38:17.738 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:38:17.738 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:38:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.626+0000 7f2ed91ea700 1 -- 172.21.15.59:0/3363853623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fcab0 msgr2=0x7f2ed40fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.626+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/3363853623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fcab0 0x7f2ed40fce90 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 rx=0x7f2ec8004870 tx=0x7f2ec8009cb0).stop 2024-02-18T00:38:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.626+0000 7f2ed91ea700 1 -- 172.21.15.59:0/3363853623 shutdown_connections 2024-02-18T00:38:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.626+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/3363853623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed40f7320 0x7f2ed4103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.626+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/3363853623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fcab0 0x7f2ed40fce90 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.626+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/3363853623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed40fd410 0x7f2ed40f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.626+0000 7f2ed91ea700 1 -- 172.21.15.59:0/3363853623 >> 172.21.15.59:0/3363853623 conn(0x7f2ed40f26e0 msgr2=0x7f2ed40f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.627+0000 7f2ed91ea700 1 -- 172.21.15.59:0/3363853623 shutdown_connections 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.627+0000 7f2ed91ea700 1 -- 172.21.15.59:0/3363853623 wait complete. 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.627+0000 7f2ed91ea700 1 librados: starting msgr at 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.627+0000 7f2ed91ea700 1 librados: starting objecter 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.627+0000 7f2ed91ea700 1 Processor -- start 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 -- start start 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 librados: setting wanted keys 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 librados: calling monclient init 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed40f7320 0x7f2ed40f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed40fcab0 0x7f2ed40f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fd410 0x7f2ed40f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4116530 con 0x7f2ed40f7320 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed41166a0 con 0x7f2ed40fcab0 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed4116860 con 0x7f2ed40fd410 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed40f7320 0x7f2ed40f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed359e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fd410 0x7f2ed40f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed40f7320 0x7f2ed40f88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55864/0 (socket says 172.21.15.59:55864) 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 -- 172.21.15.59:0/2293065036 learned_addr learned my addr 172.21.15.59:0/2293065036 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed259c700 1 --2- 172.21.15.59:0/2293065036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed40fcab0 0x7f2ed40f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 -- 172.21.15.59:0/2293065036 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fd410 msgr2=0x7f2ed40f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fd410 0x7f2ed40f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 -- 172.21.15.59:0/2293065036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed40fcab0 msgr2=0x7f2ed40f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed40fcab0 0x7f2ed40f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 -- 172.21.15.59:0/2293065036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec4005040 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed359e700 1 --2- 172.21.15.59:0/2293065036 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ed40fd410 0x7f2ed40f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed40f7320 0x7f2ed40f88b0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 rx=0x7f2ec80043a0 tx=0x7f2ec800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ebbfff700 10 client.?.objecter ms_handle_connect 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ebbfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800e070 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ec8009710 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.628+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec8005040 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.629+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8019070 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.629+0000 7f2ed91ea700 10 client.31296.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.629+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed4116e30 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.629+0000 7f2ed91ea700 1 librados: init done 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.629+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2ed405f9d0 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.630+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2ec80090d0 con 0x7f2ed40f7320 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.630+0000 7f2ebbfff700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ebc062c90 0x7f2ebc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.630+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(415..415 src has 180..415) v4 ==== 8035+0+0 (secure 0 0 0) 0x7f2ec807db10 con 0x7f2ed40f7320 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.630+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 osd_map(415..415 src has 180..415) v4 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.630+0000 7f2ebbfff700 3 client.31296.objecter handle_osd_map got epochs [415,415] > 0 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.630+0000 7f2ebbfff700 3 client.31296.objecter handle_osd_map decoding full epoch 415 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.630+0000 7f2ed259c700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ebc062c90 0x7f2ebc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ebbfff700 20 client.31296.objecter dump_active .. 0 homeless 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v415) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2ec8082220 con 0x7f2ed40f7320 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ebc062c90 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed259c700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ebc062c90 0x7f2ebc065150 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 rx=0x7f2ed40fa050 tx=0x7f2ec400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 1 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2ed40ffd70 0x7f2ed4100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 1 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 0 1 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed405ee80 osd=0 4 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 10 client.31296.objecter _send_command 1 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed4105f60 con 0x7f2ed40ffd70 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed359e700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2ed40ffd70 0x7f2ed4100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed40ffd70 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.631+0000 7f2ed359e700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2ed40ffd70 0x7f2ed4100170 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.632+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed4105f60 con 0x7f2ed40ffd70 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.632+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.632+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 1 = system:0 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.632+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed405ee80 osd=0 4 2024-02-18T00:38:17.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.632+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 0 1 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 2 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 2 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed405ee80 osd=0 3 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 0 2 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed405ee80 osd=0 4 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 10 client.31296.objecter _send_command 2 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.647+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed40273c0 con 0x7f2ed40ffd70 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed40273c0 con 0x7f2ed40ffd70 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed405ee80 osd=0 4 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 0 2 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6805/26279110 conn(0x7f2ed40598f0 0x7f2ed4059d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 1 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4028710 osd=1 4 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 10 client.31296.objecter _send_command 3 2024-02-18T00:38:17.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed411d740 con 0x7f2ed40598f0 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.649+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6805/26279110 conn(0x7f2ed40598f0 0x7f2ed4059d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.650+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed40598f0 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.650+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6805/26279110 conn(0x7f2ed40598f0 0x7f2ed4059d30 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.650+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed411d740 con 0x7f2ed40598f0 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.650+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.650+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 3 = system:0 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.650+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4028710 osd=1 4 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.650+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 1 3 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 4 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 4 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4028710 osd=1 3 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 1 4 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4028710 osd=1 4 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 10 client.31296.objecter _send_command 4 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.665+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed4059360 con 0x7f2ed40598f0 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed4059360 con 0x7f2ed40598f0 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4028710 osd=1 4 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 1 4 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 5 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2ed4028270 0x7f2ed411c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 5 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 2 5 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4059770 osd=2 4 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 10 client.31296.objecter _send_command 5 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed411c650 con 0x7f2ed4028270 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed259c700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2ed4028270 0x7f2ed411c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed4028270 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.667+0000 7f2ed259c700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2ed4028270 0x7f2ed411c160 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.668+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed411c650 con 0x7f2ed4028270 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.668+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.668+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 5 = system:0 2024-02-18T00:38:17.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.668+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4059770 osd=2 4 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.668+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 2 5 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 6 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 6 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4059770 osd=2 3 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 2 6 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4059770 osd=2 4 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 10 client.31296.objecter _send_command 6 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.682+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed4027830 con 0x7f2ed4028270 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed4027830 con 0x7f2ed4028270 2024-02-18T00:38:17.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4059770 osd=2 4 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 2 6 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 7 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2ed411f8c0 0x7f2ed4121cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 7 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 3 7 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411d040 osd=3 4 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 10 client.31296.objecter _send_command 7 2024-02-18T00:38:17.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed41249f0 con 0x7f2ed411f8c0 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.684+0000 7f2ed359e700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2ed411f8c0 0x7f2ed4121cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.685+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed411f8c0 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.685+0000 7f2ed359e700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2ed411f8c0 0x7f2ed4121cb0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.685+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed41249f0 con 0x7f2ed411f8c0 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.685+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.685+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 7 = system:0 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.685+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed411d040 osd=3 4 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.685+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 3 7 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 8 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 8 2024-02-18T00:38:17.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed411d040 osd=3 3 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 3 8 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411d040 osd=3 4 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 10 client.31296.objecter _send_command 8 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.700+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed411cb70 con 0x7f2ed411f8c0 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed411cb70 con 0x7f2ed411f8c0 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed411d040 osd=3 4 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 3 8 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 9 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2ed411bcc0 0x7f2ed41221d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 9 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 4 9 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411c930 osd=4 4 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 10 client.31296.objecter _send_command 9 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.701+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed41226c0 con 0x7f2ed411bcc0 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.702+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2ed411bcc0 0x7f2ed41221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.702+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2ed411bcc0 0x7f2ed41221d0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.702+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed411bcc0 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.703+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed41226c0 con 0x7f2ed411bcc0 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.703+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.703+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 9 = system:0 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.703+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed411c930 osd=4 4 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.703+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 4 9 2024-02-18T00:38:17.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 10 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 10 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed411c930 osd=4 3 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 4 10 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411c930 osd=4 4 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 10 client.31296.objecter _send_command 10 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.722+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed41226c0 con 0x7f2ed411bcc0 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.723+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed41226c0 con 0x7f2ed411bcc0 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.723+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.723+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.723+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed411c930 osd=4 4 2024-02-18T00:38:17.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.723+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 4 10 2024-02-18T00:38:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 11 2024-02-18T00:38:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2ed4123f70 0x7f2ed41243b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 11 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 5 11 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed41226c0 osd=5 4 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 10 client.31296.objecter _send_command 11 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed412b390 con 0x7f2ed4123f70 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed259c700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2ed4123f70 0x7f2ed41243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed4123f70 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.724+0000 7f2ed259c700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2ed4123f70 0x7f2ed41243b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.726+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed412b390 con 0x7f2ed4123f70 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.726+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.726+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 11 = system:0 2024-02-18T00:38:17.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.726+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed41226c0 osd=5 4 2024-02-18T00:38:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.726+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 5 11 2024-02-18T00:38:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 12 2024-02-18T00:38:17.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 12 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed41226c0 osd=5 3 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 5 12 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed41226c0 osd=5 4 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 10 client.31296.objecter _send_command 12 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.744+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed4128c30 con 0x7f2ed4123f70 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed4128c30 con 0x7f2ed4123f70 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed41226c0 osd=5 4 2024-02-18T00:38:17.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 5 12 2024-02-18T00:38:17.762 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:38:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 13 2024-02-18T00:38:17.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2ed41292f0 0x7f2ed4129730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 13 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 6 13 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4128c30 osd=6 4 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 10 client.31296.objecter _send_command 13 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed412e790 con 0x7f2ed41292f0 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.746+0000 7f2ed359e700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2ed41292f0 0x7f2ed4129730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.747+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed41292f0 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.747+0000 7f2ed359e700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2ed41292f0 0x7f2ed4129730 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.747+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed412e790 con 0x7f2ed41292f0 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.747+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.747+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 13 = system:0 2024-02-18T00:38:17.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.747+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4128c30 osd=6 4 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.747+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 6 13 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 14 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 14 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4128c30 osd=6 3 2024-02-18T00:38:17.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 6 14 2024-02-18T00:38:17.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4128c30 osd=6 4 2024-02-18T00:38:17.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 10 client.31296.objecter _send_command 14 2024-02-18T00:38:17.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.762+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed4129d10 con 0x7f2ed41292f0 2024-02-18T00:38:17.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed4129d10 con 0x7f2ed41292f0 2024-02-18T00:38:17.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4128c30 osd=6 4 2024-02-18T00:38:17.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 6 14 2024-02-18T00:38:17.766 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:38:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 15 2024-02-18T00:38:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6812/53025171 conn(0x7f2ed412a390 0x7f2ed412a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 15 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 7 15 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4129d10 osd=7 4 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 10 client.31296.objecter _send_command 15 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ed412ade0 con 0x7f2ed412a390 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.764+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6812/53025171 conn(0x7f2ed412a390 0x7f2ed412a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.765+0000 7f2ebbfff700 10 client.31296.objecter ms_handle_connect 0x7f2ed412a390 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.765+0000 7f2ed2d9d700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6812/53025171 conn(0x7f2ed412a390 0x7f2ed412a7f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.765+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2ed412ade0 con 0x7f2ed412a390 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.765+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.765+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 15 = system:0 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.765+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4129d10 osd=7 4 2024-02-18T00:38:17.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.765+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 7 15 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 10 client.31296.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign -1 16 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 20 client.31296.objecter _get_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_remove -1 16 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 20 client.31296.objecter get_session s=0x7f2ed4129d10 osd=7 3 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 15 client.31296.objecter _session_command_op_assign 7 16 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4129d10 osd=7 4 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 10 client.31296.objecter _send_command 16 2024-02-18T00:38:17.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.783+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2ed412ade0 con 0x7f2ed412a390 2024-02-18T00:38:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ebbfff700 1 -- 172.21.15.59:0/2293065036 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2ed412ade0 con 0x7f2ed412a390 2024-02-18T00:38:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ebbfff700 10 client.31296.objecter ms_dispatch 0x7f2ed4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ebbfff700 10 client.31296.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ebbfff700 20 client.31296.objecter put_session s=0x7f2ed4129d10 osd=7 4 2024-02-18T00:38:17.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ebbfff700 15 client.31296.objecter _session_command_op_remove 7 16 2024-02-18T00:38:17.787 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 librados: watch_flush enter 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 librados: watch_flush exit 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.0 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2ed40ffd70 msgr2=0x7f2ed4100170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2ed40ffd70 0x7f2ed4100170 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed405ee80 osd=0 1 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.1 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6805/26279110 conn(0x7f2ed40598f0 msgr2=0x7f2ed4059d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6805/26279110 conn(0x7f2ed40598f0 0x7f2ed4059d30 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4028710 osd=1 1 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.2 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2ed4028270 msgr2=0x7f2ed411c160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2ed4028270 0x7f2ed411c160 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4059770 osd=2 1 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.3 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2ed411f8c0 msgr2=0x7f2ed4121cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2ed411f8c0 0x7f2ed4121cb0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed359e700 1 -- 172.21.15.59:0/2293065036 reap_dead start 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411d040 osd=3 1 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.4 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2ed411bcc0 msgr2=0x7f2ed41221d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2ed411bcc0 0x7f2ed41221d0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed411c930 osd=4 1 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.5 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed359e700 1 -- 172.21.15.59:0/2293065036 reap_dead start 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2ed4123f70 msgr2=0x7f2ed41243b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2ed4123f70 0x7f2ed41243b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed41226c0 osd=5 1 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.6 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2ed41292f0 msgr2=0x7f2ed4129730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2ed41292f0 0x7f2ed4129730 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4128c30 osd=6 1 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter close_session for osd.7 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6812/53025171 conn(0x7f2ed412a390 msgr2=0x7f2ed412a7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.137:6812/53025171 conn(0x7f2ed412a390 0x7f2ed412a7f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter put_session s=0x7f2ed4129d10 osd=7 1 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 20 client.31296.objecter shutdown clearing up homeless session... 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.785+0000 7f2ed91ea700 10 client.31296.objecter successfully canceled tick 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ebc062c90 msgr2=0x7f2ebc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2ebc062c90 0x7f2ebc065150 secure :-1 s=READY pgs=3103 cs=0 l=1 rev1=1 rx=0x7f2ed40fa050 tx=0x7f2ec400b040).stop 2024-02-18T00:38:17.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed40f7320 msgr2=0x7f2ed40f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 --2- 172.21.15.59:0/2293065036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ed40f7320 0x7f2ed40f88b0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 rx=0x7f2ec80043a0 tx=0x7f2ec800b040).stop 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed359e700 1 -- 172.21.15.59:0/2293065036 reap_dead start 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 shutdown_connections 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 >> 172.21.15.59:0/2293065036 conn(0x7f2ed40f26e0 msgr2=0x7f2ed4060e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 shutdown_connections 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 -- 172.21.15.59:0/2293065036 wait complete. 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:17.786+0000 7f2ed91ea700 1 librados: shutdown 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.792 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.793 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.793 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:38:17.915 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:16.268209+0000 osd.2 (osd.2) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:16.268452+0000 osd.2 (osd.2) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:16.493924+0000 osd.1 (osd.1) 207 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:16.494800+0000 osd.1 (osd.1) 208 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:16.714850+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:16.714971+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e415: 8 total, 8 up, 7 in 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:16.721112+0000 mon.c (mon.2) 410 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:16.724518+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:16.725186+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:17.917 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:17.631725+0000 mon.a (mon.0) 2509 : audit [DBG] from='client.? 172.21.15.59:0/2293065036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:17.918 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:17.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:16.268209+0000 osd.2 (osd.2) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:16.268452+0000 osd.2 (osd.2) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:16.493924+0000 osd.1 (osd.1) 207 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:16.494800+0000 osd.1 (osd.1) 208 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:16.714850+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:16.714971+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e415: 8 total, 8 up, 7 in 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:16.721112+0000 mon.c (mon.2) 410 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:16.724518+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:16.725186+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:17.631725+0000 mon.a (mon.0) 2509 : audit [DBG] from='client.? 172.21.15.59:0/2293065036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:16.268209+0000 osd.2 (osd.2) 175 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:16.268452+0000 osd.2 (osd.2) 176 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:16.493924+0000 osd.1 (osd.1) 207 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:16.494800+0000 osd.1 (osd.1) 208 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:16.714850+0000 mon.a (mon.0) 2506 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:16.714971+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e415: 8 total, 8 up, 7 in 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:16.721112+0000 mon.c (mon.2) 410 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:16.724518+0000 mon.c (mon.2) 411 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:16.725186+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:18.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:17.631725+0000 mon.a (mon.0) 2509 : audit [DBG] from='client.? 172.21.15.59:0/2293065036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:18.019 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:18.046 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:18.497 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:18.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:18.598 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:18.625 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:18.749 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.267836+0000 osd.4 (osd.4) 233 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.269045+0000 osd.4 (osd.4) 234 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.656931+0000 osd.5 (osd.5) 171 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.658621+0000 osd.5 (osd.5) 172 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:17.715528+0000 mon.a (mon.0) 2510 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.715600+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e416: 8 total, 8 up, 7 in 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.775043+0000 osd.3 (osd.3) 221 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.776235+0000 osd.3 (osd.3) 222 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:17.897747+0000 mgr.y (mgr.24454) 387 : cluster [DBG] pgmap v613: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:18.288925+0000 osd.4 (osd.4) 235 : cluster [DBG] 25.b scrub starts 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:18.292193+0000 osd.4 (osd.4) 236 : cluster [DBG] 25.b scrub ok 2024-02-18T00:38:18.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:18.678362+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e417: 8 total, 8 up, 7 in 2024-02-18T00:38:18.753 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:18.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:18.854 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:18.883 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:18.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 -- 172.21.15.59:0/3827611265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f8cb0 msgr2=0x7fc5cc0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 --2- 172.21.15.59:0/3827611265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f8cb0 0x7fc5cc0f9090 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 rx=0x7fc5bc0047b0 tx=0x7fc5bc009cb0).stop 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 -- 172.21.15.59:0/3827611265 shutdown_connections 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 --2- 172.21.15.59:0/3827611265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5cc102210 0x7fc5cc104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 --2- 172.21.15.59:0/3827611265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f8cb0 0x7fc5cc0f9090 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 --2- 172.21.15.59:0/3827611265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f9610 0x7fc5cc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 -- 172.21.15.59:0/3827611265 >> 172.21.15.59:0/3827611265 conn(0x7fc5cc0f2820 msgr2=0x7fc5cc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 -- 172.21.15.59:0/3827611265 shutdown_connections 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.921+0000 7fc5d3762700 1 -- 172.21.15.59:0/3827611265 wait complete. 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 librados: starting msgr at 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 librados: starting objecter 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 Processor -- start 2024-02-18T00:38:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 -- start start 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 librados: setting wanted keys 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 librados: calling monclient init 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f8cb0 0x7fc5cc10e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f9610 0x7fc5cc10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5cc102210 0x7fc5cc1083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc108890 con 0x7fc5cc0f8cb0 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc108a00 con 0x7fc5cc102210 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d3762700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc108bc0 con 0x7fc5cc0f9610 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d14fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f8cb0 0x7fc5cc10e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d0cfd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f9610 0x7fc5cc10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d14fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f8cb0 0x7fc5cc10e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55880/0 (socket says 172.21.15.59:55880) 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d0cfd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f9610 0x7fc5cc10e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49742/0 (socket says 172.21.15.59:49742) 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d14fe700 1 -- 172.21.15.59:0/226006379 learned_addr learned my addr 172.21.15.59:0/226006379 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:18.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.922+0000 7fc5d1cff700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5cc102210 0x7fc5cc1083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d0cfd700 1 -- 172.21.15.59:0/226006379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5cc102210 msgr2=0x7fc5cc1083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d0cfd700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5cc102210 0x7fc5cc1083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d0cfd700 1 -- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f8cb0 msgr2=0x7fc5cc10e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d0cfd700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f8cb0 0x7fc5cc10e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d0cfd700 1 -- 172.21.15.59:0/226006379 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8005040 con 0x7fc5cc0f9610 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d14fe700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f8cb0 0x7fc5cc10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d0cfd700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f9610 0x7fc5cc10e6f0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 rx=0x7fc5c80091f0 tx=0x7fc5c8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5c27fc700 10 client.?.objecter ms_handle_connect 0x7fc5cc0f9610 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5c27fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5c27fc700 1 -- 172.21.15.59:0/226006379 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800e030 con 0x7fc5cc0f9610 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5c27fc700 1 -- 172.21.15.59:0/226006379 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5c800f040 con 0x7fc5cc0f9610 2024-02-18T00:38:18.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc005040 con 0x7fc5cc0f9610 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5c27fc700 1 -- 172.21.15.59:0/226006379 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80147e0 con 0x7fc5cc0f9610 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d3762700 10 client.41105.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc115030 con 0x7fc5cc0f9610 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.923+0000 7fc5d3762700 1 librados: init done 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.924+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5cc05fa60 con 0x7fc5cc0f9610 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.924+0000 7fc5c27fc700 1 -- 172.21.15.59:0/226006379 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc5c80104f0 con 0x7fc5cc0f9610 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5c27fc700 1 --2- 172.21.15.59:0/226006379 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5b8062c90 0x7fc5b8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5c27fc700 1 -- 172.21.15.59:0/226006379 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(417..417 src has 180..417) v4 ==== 8035+0+0 (secure 0 0 0) 0x7fc5c807d180 con 0x7fc5cc0f9610 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5c27fc700 10 client.41105.objecter ms_dispatch 0x7fc5cc001550 osd_map(417..417 src has 180..417) v4 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5c27fc700 3 client.41105.objecter handle_osd_map got epochs [417,417] > 0 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5c27fc700 3 client.41105.objecter handle_osd_map decoding full epoch 417 2024-02-18T00:38:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5d14fe700 1 --2- 172.21.15.59:0/226006379 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5b8062c90 0x7fc5b8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5c27fc700 20 client.41105.objecter dump_active .. 0 homeless 2024-02-18T00:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5c27fc700 10 client.41105.objecter ms_handle_connect 0x7fc5b8062c90 2024-02-18T00:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.925+0000 7fc5d14fe700 1 --2- 172.21.15.59:0/226006379 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5b8062c90 0x7fc5b8065150 secure :-1 s=READY pgs=3104 cs=0 l=1 rev1=1 rx=0x7fc5bc009f20 tx=0x7fc5bc009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:18.929+0000 7fc5c27fc700 1 -- 172.21.15.59:0/226006379 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc5c8050740 con 0x7fc5cc0f9610 2024-02-18T00:38:18.984 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.267836+0000 osd.4 (osd.4) 233 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.269045+0000 osd.4 (osd.4) 234 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.656931+0000 osd.5 (osd.5) 171 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.658621+0000 osd.5 (osd.5) 172 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:17.715528+0000 mon.a (mon.0) 2510 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.715600+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e416: 8 total, 8 up, 7 in 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.775043+0000 osd.3 (osd.3) 221 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.776235+0000 osd.3 (osd.3) 222 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:17.897747+0000 mgr.y (mgr.24454) 387 : cluster [DBG] pgmap v613: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:18.288925+0000 osd.4 (osd.4) 235 : cluster [DBG] 25.b scrub starts 2024-02-18T00:38:19.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:18.292193+0000 osd.4 (osd.4) 236 : cluster [DBG] 25.b scrub ok 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:18.678362+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e417: 8 total, 8 up, 7 in 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.267836+0000 osd.4 (osd.4) 233 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.269045+0000 osd.4 (osd.4) 234 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.656931+0000 osd.5 (osd.5) 171 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.658621+0000 osd.5 (osd.5) 172 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:17.715528+0000 mon.a (mon.0) 2510 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.715600+0000 mon.a (mon.0) 2511 : cluster [DBG] osdmap e416: 8 total, 8 up, 7 in 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.775043+0000 osd.3 (osd.3) 221 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:38:19.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.776235+0000 osd.3 (osd.3) 222 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:38:19.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:17.897747+0000 mgr.y (mgr.24454) 387 : cluster [DBG] pgmap v613: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:38:19.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:18.288925+0000 osd.4 (osd.4) 235 : cluster [DBG] 25.b scrub starts 2024-02-18T00:38:19.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:18.292193+0000 osd.4 (osd.4) 236 : cluster [DBG] 25.b scrub ok 2024-02-18T00:38:19.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:18.678362+0000 mon.a (mon.0) 2512 : cluster [DBG] osdmap e417: 8 total, 8 up, 7 in 2024-02-18T00:38:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:19.008 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:19.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:19.009 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:38:19.009 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:38:19.009 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:38:19.009 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:38:19.009 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:38:19.109 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:19.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.126+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) v1 -- 0x7fc5cc107290 con 0x7fc5cc0f9610 2024-02-18T00:38:19.136 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:19.209 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:19.212 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:19.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:19.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:19.345 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:19.648 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-18T00:38:19.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.674+0000 7fc5c27fc700 1 -- 172.21.15.59:0/226006379 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v418) v1 ==== 113+0+0 (secure 0 0 0) 0x7fc5c8050330 con 0x7fc5cc0f9610 2024-02-18T00:38:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 10 librados: watch_flush enter 2024-02-18T00:38:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 10 librados: watch_flush exit 2024-02-18T00:38:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 20 client.41105.objecter shutdown clearing up homeless session... 2024-02-18T00:38:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 10 client.41105.objecter successfully canceled tick 2024-02-18T00:38:19.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5b8062c90 msgr2=0x7fc5b8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 --2- 172.21.15.59:0/226006379 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5b8062c90 0x7fc5b8065150 secure :-1 s=READY pgs=3104 cs=0 l=1 rev1=1 rx=0x7fc5bc009f20 tx=0x7fc5bc009cb0).stop 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f9610 msgr2=0x7fc5cc10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f9610 0x7fc5cc10e6f0 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 rx=0x7fc5c80091f0 tx=0x7fc5c8009cb0).stop 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 shutdown_connections 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 --2- 172.21.15.59:0/226006379 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5b8062c90 0x7fc5b8065150 unknown :-1 s=CLOSED pgs=3104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5cc102210 0x7fc5cc1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5cc0f8cb0 0x7fc5cc10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 --2- 172.21.15.59:0/226006379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5cc0f9610 0x7fc5cc10e6f0 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 >> 172.21.15.59:0/226006379 conn(0x7fc5cc0f2820 msgr2=0x7fc5cc0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:19.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 shutdown_connections 2024-02-18T00:38:19.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 -- 172.21.15.59:0/226006379 wait complete. 2024-02-18T00:38:19.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:19.677+0000 7fc5d3762700 1 librados: shutdown 2024-02-18T00:38:19.681 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.0 to 1 (10000) 2024-02-18T00:38:19.915 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:18.210435+0000 osd.2 (osd.2) 177 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:18.211322+0000 osd.2 (osd.2) 178 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:19.127985+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.59:0/226006379' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:19.128320+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.249690+0000 osd.4 (osd.4) 237 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.252804+0000 osd.4 (osd.4) 238 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:19.674690+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:38:20.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.674768+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e418: 8 total, 8 up, 7 in 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:18.210435+0000 osd.2 (osd.2) 177 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:18.211322+0000 osd.2 (osd.2) 178 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:19.127985+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.59:0/226006379' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:19.128320+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.249690+0000 osd.4 (osd.4) 237 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.252804+0000 osd.4 (osd.4) 238 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:19.674690+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:38:20.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.674768+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e418: 8 total, 8 up, 7 in 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:18.210435+0000 osd.2 (osd.2) 177 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:18.211322+0000 osd.2 (osd.2) 178 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:19.127985+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.59:0/226006379' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:19.128320+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.249690+0000 osd.4 (osd.4) 237 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.252804+0000 osd.4 (osd.4) 238 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:19.674690+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:38:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.674768+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e418: 8 total, 8 up, 7 in 2024-02-18T00:38:20.501 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:20.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:20.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:20.605 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:20.633 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:20.671 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:20.673 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:20.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:20.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:20.804 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.533159+0000 osd.1 (osd.1) 209 : cluster [DBG] 2.a scrub starts 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.536087+0000 osd.1 (osd.1) 210 : cluster [DBG] 2.a scrub ok 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.634886+0000 osd.0 (osd.0) 193 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.635813+0000 osd.0 (osd.0) 194 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:19.898240+0000 mgr.y (mgr.24454) 388 : cluster [DBG] pgmap v616: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.219034+0000 osd.4 (osd.4) 239 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.222266+0000 osd.4 (osd.4) 240 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:38:20.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.676895+0000 mon.a (mon.0) 2516 : cluster [DBG] osdmap e419: 8 total, 8 up, 7 in 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.533159+0000 osd.1 (osd.1) 209 : cluster [DBG] 2.a scrub starts 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.536087+0000 osd.1 (osd.1) 210 : cluster [DBG] 2.a scrub ok 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.634886+0000 osd.0 (osd.0) 193 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.635813+0000 osd.0 (osd.0) 194 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:19.898240+0000 mgr.y (mgr.24454) 388 : cluster [DBG] pgmap v616: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.219034+0000 osd.4 (osd.4) 239 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.222266+0000 osd.4 (osd.4) 240 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:38:21.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.676895+0000 mon.a (mon.0) 2516 : cluster [DBG] osdmap e419: 8 total, 8 up, 7 in 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.533159+0000 osd.1 (osd.1) 209 : cluster [DBG] 2.a scrub starts 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.536087+0000 osd.1 (osd.1) 210 : cluster [DBG] 2.a scrub ok 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.634886+0000 osd.0 (osd.0) 193 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.635813+0000 osd.0 (osd.0) 194 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:19.898240+0000 mgr.y (mgr.24454) 388 : cluster [DBG] pgmap v616: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.219034+0000 osd.4 (osd.4) 239 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.222266+0000 osd.4 (osd.4) 240 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:38:21.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.676895+0000 mon.a (mon.0) 2516 : cluster [DBG] osdmap e419: 8 total, 8 up, 7 in 2024-02-18T00:38:21.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 1 1 2024-02-18T00:38:21.153 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:21.157 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:21.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:21.169 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:38:21.259 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:21.286 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:21.452 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:38:21.485 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:21.489 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:21.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:21.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:21.619 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:21.717 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:21.719 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:21.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2059 17.11 e108/104 17:8d70d4b8:test-rados-api-smithi059-82949-16::big:head v 108'2, mlcod=0'0)", 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.678171+0000", 2024-02-18T00:38:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 318.04816050199997, 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.062291530999999997, 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.678171+0000", 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.678171+0000", 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.678170+0000", 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027259334 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705430+0000", 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3023e-05 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705453+0000", 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6519999999999995e-06 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705461+0000", 2024-02-18T00:38:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5605999999999994e-05 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705537+0000", 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0548e-05 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.705558+0000", 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034859651999999998 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.740417+0000", 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5531000000000002e-05 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.740463+0000", 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.062291530999999997 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.0.0:148 17.4 e108/104 -19:238a5d4d:::temp_17.4_0_24140_1:head v 0'0, mlcod=108'1)", 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.758875+0000", 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 317.96745686700001, 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025775896999999999, 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.758875+0000", 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.758875+0000", 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999914 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.758866+0000", 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.016318358000000002 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775185+0000", 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1304000000000001e-05 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775206+0000", 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7670000000000003e-06 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775214+0000", 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4301000000000006e-05 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775288+0000", 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8799000000000001e-05 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.775307+0000", 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0093030990000000004 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.784610+0000", 2024-02-18T00:38:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0865999999999998e-05 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.784651+0000", 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025775896999999999 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.0.0:149 17.4 e108/104 -19:238a5d4d:::temp_17.4_0_24140_1:head v 0'0, mlcod=108'1)", 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.782907+0000", 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 317.94342462200001, 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017161189, 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.782907+0000", 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.782907+0000", 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.782906+0000", 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0077003490000000004 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790606+0000", 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9126e-05 2024-02-18T00:38:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790625+0000", 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1689999999999992e-06 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790633+0000", 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3750000000000004e-05 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790707+0000", 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4715999999999998e-05 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.790742+0000", 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0092875760000000005 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.800029+0000", 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8961999999999997e-05 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.800068+0000", 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017161189 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.0.0:150 17.4 e108/104 -19:238a5d4d:::temp_17.4_0_24140_1:head v 0'0, mlcod=108'1)", 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.806114+0000", 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 317.92021741399998, 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017799674000000001, 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.806114+0000", 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.806114+0000", 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.806112+0000", 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0082654129999999992 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814378+0000", 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6948000000000001e-05 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814395+0000", 2024-02-18T00:38:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6560000000000004e-06 2024-02-18T00:38:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814404+0000", 2024-02-18T00:38:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8322999999999999e-05 2024-02-18T00:38:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814443+0000", 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9939999999999999e-05 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.814463+0000", 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0094215979999999998 2024-02-18T00:38:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.823884+0000", 2024-02-18T00:38:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9757e-05 2024-02-18T00:38:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.823914+0000", 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017799674000000001 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2069 17.3 17:c1ec3694:test-rados-api-smithi059-82949-17::big:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.014659+0000", 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 317.711673179, 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096479857000000002, 2024-02-18T00:38:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2069 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.014659+0000", 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.014659+0000", 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.014658+0000", 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.032078772999999998 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046736+0000", 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5109999999999999e-05 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046752+0000", 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7400000000000004e-06 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046759+0000", 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0557999999999999e-05 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046810+0000", 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000123153 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.046933+0000", 2024-02-18T00:38:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000120703 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.047054+0000", 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.059890066999999998 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.106944+0000", 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003654839 2024-02-18T00:38:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.110599+0000", 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00046675800000000001 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111065+0000", 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4075000000000002e-05 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111099+0000", 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8964000000000002e-05 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111138+0000", 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096479857000000002 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:6 22.2 e108/105 22:47a9abb2:::36.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.068708+0000", 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 317.65762404600002, 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034744510999999999, 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068708+0000", 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068708+0000", 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068707+0000", 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2602000000000001e-05 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068719+0000", 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7615999999999999e-05 2024-02-18T00:38:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068737+0000", 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6810000000000003e-06 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068743+0000", 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6243999999999999e-05 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068799+0000", 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5424999999999999e-05 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068814+0000", 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034560975000000001 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103375+0000", 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6917999999999998e-05 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103452+0000", 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034744510999999999 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24865.0:4 29.16 29:681f27cd:::2.obj:head [write 0~105] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.757362+0000", 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 316.96896994100001, 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012451744000000001, 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24865", 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2678422317", 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 4 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757362+0000", 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757362+0000", 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757361+0000", 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6770000000000003e-06 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757366+0000", 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7200000000000006e-06 2024-02-18T00:38:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757373+0000", 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6960000000000002e-06 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757379+0000", 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1489999999999997e-05 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757421+0000", 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4278999999999998e-05 2024-02-18T00:38:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757495+0000", 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000130619 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757625+0000", 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00034791500000000002 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.757973+0000", 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000186467 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.758160+0000", 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011598658 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.769758+0000", 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2137000000000001e-05 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.769791+0000", 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2926e-05 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.769814+0000", 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012451744000000001 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.15213.0:2 41.1a 41:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:08.335259+0000", 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 313.39107318600003, 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.54663461800000002, 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.15213", 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1863209286", 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335259+0000", 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335259+0000", 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335258+0000", 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.045e-06 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335263+0000", 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3730000000000004e-06 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335270+0000", 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1789999999999999e-06 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335275+0000", 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8170999999999998e-05 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335313+0000", 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6689999999999997e-06 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.335318+0000", 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.52990910599999996 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.865227+0000", 2024-02-18T00:38:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000124976 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.865352+0000", 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014595600000000001 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.865498+0000", 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00086496499999999998 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.866363+0000", 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0082699220000000007 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.874633+0000", 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0072001019999999999 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.881833+0000", 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3037000000000002e-05 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.881866+0000", 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6931999999999999e-05 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:08.881893+0000", 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.54663461800000002 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:2 71.15 71:ad551702:::test:head [create,write 0~4] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 293.63755909399998, 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067209718000000002, 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088771+0000", 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7769999999999997e-06 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088778+0000", 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9749999999999996e-06 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088787+0000", 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6979999999999998e-06 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088794+0000", 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3955999999999997e-05 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088838+0000", 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0620000000000003e-06 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088845+0000", 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042483522000000003 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131329+0000", 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014866200000000001 2024-02-18T00:38:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131477+0000", 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019506600000000001 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131673+0000", 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074819399999999999 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.132421+0000", 2024-02-18T00:38:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010435201999999999 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.142856+0000", 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013070307 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155926+0000", 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2119000000000002e-05 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155958+0000", 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3989999999999999e-05 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155982+0000", 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067209718000000002 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:6 71.15 71:ad551702:::test:head [cmpext 0~4,write 0~4] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:38:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 293.542393715, 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026666680000000002, 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 6 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183937+0000", 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4240000000000003e-06 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183942+0000", 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2269999999999996e-06 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183949+0000", 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9459999999999997e-06 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183954+0000", 2024-02-18T00:38:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4397e-05 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183978+0000", 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0843000000000002e-05 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.184019+0000", 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026585647 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.210605+0000", 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026666680000000002 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:7 71.15 71:ad551702:::test:head [delete] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 293.512239126, 2024-02-18T00:38:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019741999999999999, 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 7 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214092+0000", 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1780000000000002e-06 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214097+0000", 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0831e-05 2024-02-18T00:38:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214108+0000", 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6280000000000004e-06 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214112+0000", 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3321000000000001e-05 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214156+0000", 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5122000000000004e-05 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214221+0000", 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7605e-05 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214298+0000", 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029685200000000001 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214595+0000", 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019093308999999999 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233688+0000", 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5790000000000003e-05 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233784+0000", 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9073000000000001e-05 2024-02-18T00:38:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233813+0000", 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1229999999999998e-05 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233835+0000", 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019741999999999999 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30097.0:2 74.1f 74:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 291.362533203, 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014294953000000001, 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30097", 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3002694168", 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:38:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363797+0000", 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9099999999999999e-06 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363804+0000", 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1026000000000001e-05 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363815+0000", 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9299999999999997e-06 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363822+0000", 2024-02-18T00:38:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5386000000000002e-05 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363867+0000", 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0513999999999995e-05 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363948+0000", 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5987000000000001e-05 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,6", 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364044+0000", 2024-02-18T00:38:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00049711199999999997 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364541+0000", 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000204945 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364746+0000", 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013291124 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378037+0000", 2024-02-18T00:38:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2733e-05 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378069+0000", 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4020999999999999e-05 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378093+0000", 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014294953000000001 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25220.0:109 62.d 62:b295a385:test-rados-api-smithi059-82957-10::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 290.36103348099999, 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031474052000000002, 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25220", 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 109 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365297+0000", 2024-02-18T00:38:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8029999999999997e-06 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365305+0000", 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.093e-05 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365315+0000", 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1999999999999997e-06 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365323+0000", 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3896999999999997e-05 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365367+0000", 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2156000000000004e-05 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365449+0000", 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3950999999999995e-05 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,7", 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365543+0000", 2024-02-18T00:38:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00031575199999999998 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365858+0000", 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000242795 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.366101+0000", 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.030572499 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396674+0000", 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8595999999999999e-05 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396732+0000", 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9944000000000002e-05 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396772+0000", 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031474052000000002 2024-02-18T00:38:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:334 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 287.04333106199999, 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02568811, 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.682999+0000", 2024-02-18T00:38:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015711060999999998 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698710+0000", 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7644999999999994e-05 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698778+0000", 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2190000000000004e-06 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698786+0000", 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3647999999999999e-05 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698840+0000", 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8643000000000001e-05 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698859+0000", 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0097918710000000006 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.708650+0000", 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8368999999999998e-05 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.708689+0000", 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02568811 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:335 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:38:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 287.00786672200002, 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019566244, 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:38:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718462+0000", 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0098851159999999993 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728347+0000", 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7931000000000002e-05 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728365+0000", 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7789999999999996e-06 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728373+0000", 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7117999999999998e-05 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728430+0000", 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9799000000000001e-05 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728450+0000", 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0095431630000000003 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.737993+0000", 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8220999999999999e-05 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.738031+0000", 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019566244 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:336 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 286.969235217, 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017739901999999998, 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999967 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757093+0000", 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0078518660000000007 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764945+0000", 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1234999999999999e-05 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764966+0000", 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.858e-06 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764974+0000", 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2244000000000003e-05 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.765027+0000", 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1681000000000001e-05 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.765048+0000", 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0097585759999999997 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.774807+0000", 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9601999999999999e-05 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.774836+0000", 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017739901999999998 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:500 62.c e205/123 62:348d0773:test-rados-api-smithi059-82957-14::foo71:5 v 205'15, mlcod=193'11)", 2024-02-18T00:38:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 218.07559878800001, 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020248832000000001, 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999866 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650720+0000", 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.51e-05 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650745+0000", 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7669e-05 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650762+0000", 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.289e-06 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650770+0000", 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5961000000000001e-05 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650836+0000", 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1322000000000003e-05 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650867+0000", 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020080569 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670948+0000", 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4106000000000002e-05 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670982+0000", 2024-02-18T00:38:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020248832000000001 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40613.0:4 155.7 155:e19d0d35:::oid_RmXattrPP_:head [rmxattr attr1 in=5b] snapc 0=[] ondisk+write+known_if_redirected e235)", 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 186.10246800799999, 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013802841, 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40613", 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/558528338", 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 4 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:38:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623861+0000", 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1669e-05 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623873+0000", 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2911999999999999e-05 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623886+0000", 2024-02-18T00:38:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5609999999999999e-06 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623894+0000", 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3367000000000003e-05 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623978+0000", 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6638999999999994e-05 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.624074+0000", 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015765399999999999 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.624232+0000", 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001632643 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.625865+0000", 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051082250000000001 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.630973+0000", 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0065934679999999999 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637566+0000", 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8032000000000003e-05 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637624+0000", 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2246000000000002e-05 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637667+0000", 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013802841 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40868.0:1 189.6s0 189:72c88bc2:::oid_IsCompletePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e309)", 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 114.06785975699999, 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.008077509, 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.831 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40868", 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/458568879", 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658470+0000", 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9140000000000008e-06 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658478+0000", 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0781999999999999e-05 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658489+0000", 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3030000000000002e-06 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658496+0000", 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051365309999999997 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.663633+0000", 2024-02-18T00:38:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4510000000000001e-06 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.663636+0000", 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0011838420000001 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.664820+0000", 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014268199999999999 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.664963+0000", 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047133100000000002 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.665434+0000", 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00069780299999999999 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666132+0000", 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00035506600000000002 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666487+0000", 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2547999999999998e-05 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666549+0000", 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.008077509 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.41033.0:1 205.3s0 205:cbb9903e:::test_obj:head [omap-set-vals,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e382)", 2024-02-18T00:38:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 41.037357485999998, 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3087259229999999, 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.41033", 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3621370462", 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:38:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688972+0000", 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.114e-06 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688979+0000", 2024-02-18T00:38:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8130000000000007e-06 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688989+0000", 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7789999999999996e-06 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688997+0000", 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0035324459999999998 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.692529+0000", 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4070000000000003e-06 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.692534+0000", 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.302913354 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.995447+0000", 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015382699999999999 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.995601+0000", 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020991709999999999 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.997700+0000", 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3087259229999999 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:21.842 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:38:21.843 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.205551+0000 osd.2 (osd.2) 179 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.206472+0000 osd.2 (osd.2) 180 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.585414+0000 osd.1 (osd.1) 211 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.588647+0000 osd.1 (osd.1) 212 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.695228+0000 osd.6 (osd.6) 193 : cluster [DBG] 4.a scrub starts 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:20.697259+0000 osd.6 (osd.6) 194 : cluster [DBG] 4.a scrub ok 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:21.241719+0000 mgr.y (mgr.24454) 389 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:21.678791+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e420: 8 total, 8 up, 7 in 2024-02-18T00:38:21.844 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.205551+0000 osd.2 (osd.2) 179 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:38:21.845 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.206472+0000 osd.2 (osd.2) 180 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:38:21.845 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.585414+0000 osd.1 (osd.1) 211 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:38:21.845 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.588647+0000 osd.1 (osd.1) 212 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:38:21.845 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.695228+0000 osd.6 (osd.6) 193 : cluster [DBG] 4.a scrub starts 2024-02-18T00:38:21.845 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:20.697259+0000 osd.6 (osd.6) 194 : cluster [DBG] 4.a scrub ok 2024-02-18T00:38:21.845 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:21.241719+0000 mgr.y (mgr.24454) 389 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:21.845 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:21.678791+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e420: 8 total, 8 up, 7 in 2024-02-18T00:38:21.875 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:22.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.205551+0000 osd.2 (osd.2) 179 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:38:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.206472+0000 osd.2 (osd.2) 180 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:38:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.585414+0000 osd.1 (osd.1) 211 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:38:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.588647+0000 osd.1 (osd.1) 212 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:38:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.695228+0000 osd.6 (osd.6) 193 : cluster [DBG] 4.a scrub starts 2024-02-18T00:38:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:20.697259+0000 osd.6 (osd.6) 194 : cluster [DBG] 4.a scrub ok 2024-02-18T00:38:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:21.241719+0000 mgr.y (mgr.24454) 389 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:21.678791+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e420: 8 total, 8 up, 7 in 2024-02-18T00:38:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 -- 172.21.15.59:0/1757852216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fd510 msgr2=0x7f7cdc0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 --2- 172.21.15.59:0/1757852216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fd510 0x7f7cdc0f6d40 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 rx=0x7f7ccc004870 tx=0x7f7ccc009cb0).stop 2024-02-18T00:38:22.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 -- 172.21.15.59:0/1757852216 shutdown_connections 2024-02-18T00:38:22.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 --2- 172.21.15.59:0/1757852216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cdc0fcbb0 0x7f7cdc0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 --2- 172.21.15.59:0/1757852216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7cdc0f7400 0x7f7cdc103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 --2- 172.21.15.59:0/1757852216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fd510 0x7f7cdc0f6d40 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 -- 172.21.15.59:0/1757852216 >> 172.21.15.59:0/1757852216 conn(0x7f7cdc0f2820 msgr2=0x7f7cdc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:22.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 -- 172.21.15.59:0/1757852216 shutdown_connections 2024-02-18T00:38:22.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 -- 172.21.15.59:0/1757852216 wait complete. 2024-02-18T00:38:22.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 librados: starting msgr at 2024-02-18T00:38:22.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.158+0000 7f7ce17db700 1 librados: starting objecter 2024-02-18T00:38:22.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 Processor -- start 2024-02-18T00:38:22.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 -- start start 2024-02-18T00:38:22.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 librados: setting wanted keys 2024-02-18T00:38:22.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 librados: calling monclient init 2024-02-18T00:38:22.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7cdc0f7400 0x7f7cdc10e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:22.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fcbb0 0x7f7cdc10e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:22.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cdc0fd510 0x7f7cdc1082e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:22.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc1087d0 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc108940 con 0x7f7cdc0fd510 2024-02-18T00:38:22.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7ce17db700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc108b00 con 0x7f7cdc0f7400 2024-02-18T00:38:22.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fcbb0 0x7f7cdc10e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:22.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cdaffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7cdc0f7400 0x7f7cdc10e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:22.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fcbb0 0x7f7cdc10e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47180/0 (socket says 172.21.15.59:47180) 2024-02-18T00:38:22.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 -- 172.21.15.59:0/3142753491 learned_addr learned my addr 172.21.15.59:0/3142753491 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:22.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cdb7fe700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cdc0fd510 0x7f7cdc1082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:22.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 -- 172.21.15.59:0/3142753491 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7cdc0f7400 msgr2=0x7f7cdc10e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:22.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7cdc0f7400 0x7f7cdc10e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 -- 172.21.15.59:0/3142753491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cdc0fd510 msgr2=0x7f7cdc1082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cdc0fd510 0x7f7cdc1082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.159+0000 7f7cda7fc700 1 -- 172.21.15.59:0/3142753491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc005040 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7cda7fc700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fcbb0 0x7f7cdc10e630 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 rx=0x7f7ccc009eb0 tx=0x7f7ccc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7cd3fff700 10 client.?.objecter ms_handle_connect 0x7f7cdc0fcbb0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7cd3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7cd3fff700 1 -- 172.21.15.59:0/3142753491 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00e070 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc108d80 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7cd3fff700 1 -- 172.21.15.59:0/3142753491 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ccc013da0 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7cd3fff700 1 -- 172.21.15.59:0/3142753491 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc019070 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7ce17db700 10 client.31308.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc114f70 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.160+0000 7f7ce17db700 1 librados: init done 2024-02-18T00:38:22.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.161+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc05fa60 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cd3fff700 1 -- 172.21.15.59:0/3142753491 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f7ccc01e420 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cd3fff700 1 --2- 172.21.15.59:0/3142753491 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cc8062c90 0x7f7cc8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cd3fff700 1 -- 172.21.15.59:0/3142753491 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(420..420 src has 180..420) v4 ==== 8091+0+0 (secure 0 0 0) 0x7f7ccc07e010 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cd3fff700 10 client.31308.objecter ms_dispatch 0x7f7cdc001550 osd_map(420..420 src has 180..420) v4 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cd3fff700 3 client.31308.objecter handle_osd_map got epochs [420,420] > 0 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cd3fff700 3 client.31308.objecter handle_osd_map decoding full epoch 420 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cdaffd700 1 --2- 172.21.15.59:0/3142753491 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cc8062c90 0x7f7cc8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.162+0000 7f7cd3fff700 20 client.31308.objecter dump_active .. 0 homeless 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.163+0000 7f7cd3fff700 10 client.31308.objecter ms_handle_connect 0x7f7cc8062c90 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.163+0000 7f7cdaffd700 1 --2- 172.21.15.59:0/3142753491 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cc8062c90 0x7f7cc8065150 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 rx=0x7f7cc4004dc0 tx=0x7f7cc400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:22.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.166+0000 7f7cd3fff700 1 -- 172.21.15.59:0/3142753491 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7ccc0515a0 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.225 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:22.230 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:22.231 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:22.263 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:22.332 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:22.360 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:22.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.367+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f7cdc106120 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.430 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-18T00:38:22.458 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:38:22.484 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:22] "GET /metrics HTTP/1.1" 200 189469 "" "Prometheus/2.33.4" 2024-02-18T00:38:22.559 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:22.561 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:22.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:22.663 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:22.691 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:22.712 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:22.715 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:22.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:22.748 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:22.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.749+0000 7f7cd3fff700 1 -- 172.21.15.59:0/3142753491 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v421) v1 ==== 101+0+0 (secure 0 0 0) 0x7f7ccc023070 con 0x7f7cdc0fcbb0 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 10 librados: watch_flush enter 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 10 librados: watch_flush exit 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 20 client.31308.objecter shutdown clearing up homeless session... 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 10 client.31308.objecter successfully canceled tick 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cc8062c90 msgr2=0x7f7cc8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 1 --2- 172.21.15.59:0/3142753491 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cc8062c90 0x7f7cc8065150 secure :-1 s=READY pgs=3105 cs=0 l=1 rev1=1 rx=0x7f7cc4004dc0 tx=0x7f7cc400b040).stop 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fcbb0 msgr2=0x7f7cdc10e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:22.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.752+0000 7f7ce17db700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fcbb0 0x7f7cdc10e630 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 rx=0x7f7ccc009eb0 tx=0x7f7ccc00b040).stop 2024-02-18T00:38:22.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 shutdown_connections 2024-02-18T00:38:22.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 --2- 172.21.15.59:0/3142753491 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7cc8062c90 0x7f7cc8065150 unknown :-1 s=CLOSED pgs=3105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cdc0fd510 0x7f7cdc1082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7cdc0f7400 0x7f7cdc10e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 --2- 172.21.15.59:0/3142753491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cdc0fcbb0 0x7f7cdc10e630 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:22.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 >> 172.21.15.59:0/3142753491 conn(0x7f7cdc0f2820 msgr2=0x7f7cdc0f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:22.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 shutdown_connections 2024-02-18T00:38:22.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 -- 172.21.15.59:0/3142753491 wait complete. 2024-02-18T00:38:22.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:22.754+0000 7f7ce17db700 1 librados: shutdown 2024-02-18T00:38:22.756 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:38:22.817 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:22.845 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:21.519008+0000 osd.5 (osd.5) 173 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:21.519145+0000 osd.5 (osd.5) 174 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:21.898592+0000 mgr.y (mgr.24454) 390 : cluster [DBG] pgmap v619: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:22.368582+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? 172.21.15.59:0/3142753491' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:21.519008+0000 osd.5 (osd.5) 173 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:21.519145+0000 osd.5 (osd.5) 174 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:21.898592+0000 mgr.y (mgr.24454) 390 : cluster [DBG] pgmap v619: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-18T00:38:23.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:22.368582+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? 172.21.15.59:0/3142753491' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:23.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:21.519008+0000 osd.5 (osd.5) 173 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:21.519145+0000 osd.5 (osd.5) 174 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:21.898592+0000 mgr.y (mgr.24454) 390 : cluster [DBG] pgmap v619: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-18T00:38:23.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:22.368582+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? 172.21.15.59:0/3142753491' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.637769+0000 osd.5 (osd.5) 175 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.640714+0000 osd.5 (osd.5) 176 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.717055+0000 osd.6 (osd.6) 195 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.718619+0000 osd.6 (osd.6) 196 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.738754+0000 mon.a (mon.0) 2519 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:22.750620+0000 mon.a (mon.0) 2520 : audit [INF] from='client.? 172.21.15.59:0/3142753491' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.750686+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e421: 8 total, 8 up, 7 in 2024-02-18T00:38:24.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.775373+0000 osd.3 (osd.3) 223 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:22.776324+0000 osd.3 (osd.3) 224 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.637769+0000 osd.5 (osd.5) 175 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.640714+0000 osd.5 (osd.5) 176 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.717055+0000 osd.6 (osd.6) 195 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.718619+0000 osd.6 (osd.6) 196 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.738754+0000 mon.a (mon.0) 2519 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:22.750620+0000 mon.a (mon.0) 2520 : audit [INF] from='client.? 172.21.15.59:0/3142753491' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.750686+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e421: 8 total, 8 up, 7 in 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.775373+0000 osd.3 (osd.3) 223 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:38:24.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:22.776324+0000 osd.3 (osd.3) 224 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.637769+0000 osd.5 (osd.5) 175 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.640714+0000 osd.5 (osd.5) 176 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.717055+0000 osd.6 (osd.6) 195 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.718619+0000 osd.6 (osd.6) 196 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.738754+0000 mon.a (mon.0) 2519 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:22.750620+0000 mon.a (mon.0) 2520 : audit [INF] from='client.? 172.21.15.59:0/3142753491' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.750686+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e421: 8 total, 8 up, 7 in 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.775373+0000 osd.3 (osd.3) 223 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:38:24.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:22.776324+0000 osd.3 (osd.3) 224 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:38:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 -- 172.21.15.59:0/3544107183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd140f9640 msgr2=0x7efd14101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 --2- 172.21.15.59:0/3544107183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd140f9640 0x7efd14101de0 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 rx=0x7efd04004870 tx=0x7efd04009cb0).stop 2024-02-18T00:38:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 -- 172.21.15.59:0/3544107183 shutdown_connections 2024-02-18T00:38:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 --2- 172.21.15.59:0/3544107183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd140f8d70 0x7efd140f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 --2- 172.21.15.59:0/3544107183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd141022d0 0x7efd141046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 --2- 172.21.15.59:0/3544107183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd140f9640 0x7efd14101de0 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 -- 172.21.15.59:0/3544107183 >> 172.21.15.59:0/3544107183 conn(0x7efd140f2820 msgr2=0x7efd140f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 -- 172.21.15.59:0/3544107183 shutdown_connections 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.509+0000 7efd1be23700 1 -- 172.21.15.59:0/3544107183 wait complete. 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.510+0000 7efd1be23700 1 librados: starting msgr at 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.510+0000 7efd1be23700 1 librados: starting objecter 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.510+0000 7efd1be23700 1 Processor -- start 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.510+0000 7efd1be23700 1 -- start start 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.510+0000 7efd1be23700 1 librados: setting wanted keys 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.510+0000 7efd1be23700 1 librados: calling monclient init 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd140f8d70 0x7efd1410e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd140f9640 0x7efd1410e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd141022d0 0x7efd141083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd141088c0 con 0x7efd141022d0 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd14108a30 con 0x7efd140f8d70 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd14108ba0 con 0x7efd140f9640 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd140f9640 0x7efd1410e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1a3c0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd141022d0 0x7efd141083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd140f9640 0x7efd1410e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59564/0 (socket says 172.21.15.59:59564) 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 -- 172.21.15.59:0/3420486287 learned_addr learned my addr 172.21.15.59:0/3420486287 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd19bbf700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd140f8d70 0x7efd1410e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 -- 172.21.15.59:0/3420486287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd140f8d70 msgr2=0x7efd1410e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd140f8d70 0x7efd1410e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 -- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd141022d0 msgr2=0x7efd141083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd141022d0 0x7efd141083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 -- 172.21.15.59:0/3420486287 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0097c0 con 0x7efd140f9640 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1a3c0700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd141022d0 0x7efd141083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd193be700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd140f9640 0x7efd1410e6e0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 rx=0x7efd04009eb0 tx=0x7efd0400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd0affd700 10 client.?.objecter ms_handle_connect 0x7efd140f9640 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd0affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd0affd700 1 -- 172.21.15.59:0/3420486287 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0400e070 con 0x7efd140f9640 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd0affd700 1 -- 172.21.15.59:0/3420486287 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd04013da0 con 0x7efd140f9640 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd04005040 con 0x7efd140f9640 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd0affd700 1 -- 172.21.15.59:0/3420486287 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04019070 con 0x7efd140f9640 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 10 client.41117.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd14109090 con 0x7efd140f9640 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.511+0000 7efd1be23700 1 librados: init done 2024-02-18T00:38:24.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.512+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd141092b0 con 0x7efd140f9640 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 1 -- 172.21.15.59:0/3420486287 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efd0401e420 con 0x7efd140f9640 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 1 --2- 172.21.15.59:0/3420486287 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd00062c90 0x7efd00065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 1 -- 172.21.15.59:0/3420486287 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(421..421 src has 180..421) v4 ==== 8063+0+0 (secure 0 0 0) 0x7efd0407dc90 con 0x7efd140f9640 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 10 client.41117.objecter ms_dispatch 0x7efd14001550 osd_map(421..421 src has 180..421) v4 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 3 client.41117.objecter handle_osd_map got epochs [421,421] > 0 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 3 client.41117.objecter handle_osd_map decoding full epoch 421 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd19bbf700 1 --2- 172.21.15.59:0/3420486287 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd00062c90 0x7efd00065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 20 client.41117.objecter dump_active .. 0 homeless 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd0affd700 10 client.41117.objecter ms_handle_connect 0x7efd00062c90 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.513+0000 7efd19bbf700 1 --2- 172.21.15.59:0/3420486287 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd00062c90 0x7efd00065150 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 rx=0x7efd10004dc0 tx=0x7efd1000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:24.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.517+0000 7efd0affd700 1 -- 172.21.15.59:0/3420486287 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efd04051280 con 0x7efd140f9640 2024-02-18T00:38:24.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:24.706+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7efd14107290 con 0x7efd140f9640 2024-02-18T00:38:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:23.608948+0000 osd.5 (osd.5) 177 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:38:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:23.612078+0000 osd.5 (osd.5) 178 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:38:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:23.710012+0000 osd.7 (osd.7) 103 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:23.710081+0000 osd.7 (osd.7) 104 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:23.899230+0000 mgr.y (mgr.24454) 391 : cluster [DBG] pgmap v621: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-02-18T00:38:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.125281+0000 osd.4 (osd.4) 241 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:38:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.128166+0000 osd.4 (osd.4) 242 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:38:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.675371+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e422: 8 total, 8 up, 7 in 2024-02-18T00:38:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:24.707421+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.59:0/3420486287' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:38:25.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:24.707705+0000 mon.a (mon.0) 2523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:38:25.013 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:38:25.013 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:38:25.013 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:38:25.013 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:23.608948+0000 osd.5 (osd.5) 177 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:23.612078+0000 osd.5 (osd.5) 178 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:23.710012+0000 osd.7 (osd.7) 103 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:23.710081+0000 osd.7 (osd.7) 104 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:23.899230+0000 mgr.y (mgr.24454) 391 : cluster [DBG] pgmap v621: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.125281+0000 osd.4 (osd.4) 241 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.128166+0000 osd.4 (osd.4) 242 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:38:25.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.675371+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e422: 8 total, 8 up, 7 in 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:24.707421+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.59:0/3420486287' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:24.707705+0000 mon.a (mon.0) 2523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:23.608948+0000 osd.5 (osd.5) 177 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:23.612078+0000 osd.5 (osd.5) 178 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:23.710012+0000 osd.7 (osd.7) 103 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:23.710081+0000 osd.7 (osd.7) 104 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:23.899230+0000 mgr.y (mgr.24454) 391 : cluster [DBG] pgmap v621: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.6 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.125281+0000 osd.4 (osd.4) 241 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.128166+0000 osd.4 (osd.4) 242 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.675371+0000 mon.a (mon.0) 2522 : cluster [DBG] osdmap e422: 8 total, 8 up, 7 in 2024-02-18T00:38:25.093 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:24.707421+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.59:0/3420486287' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:38:25.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:24.707705+0000 mon.a (mon.0) 2523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:38:25.097 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:25.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:25.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:25.153 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:25.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:25.460 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:25.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:25.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:25.564 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:25.601 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:25.616 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-18T00:38:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:25.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:25.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:25.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.680+0000 7efd0affd700 1 -- 172.21.15.59:0/3420486287 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v423) v1 ==== 113+0+0 (secure 0 0 0) 0x7efd04023030 con 0x7efd140f9640 2024-02-18T00:38:25.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 10 librados: watch_flush enter 2024-02-18T00:38:25.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 10 librados: watch_flush exit 2024-02-18T00:38:25.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 20 client.41117.objecter shutdown clearing up homeless session... 2024-02-18T00:38:25.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 10 client.41117.objecter successfully canceled tick 2024-02-18T00:38:25.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd00062c90 msgr2=0x7efd00065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:25.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 --2- 172.21.15.59:0/3420486287 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd00062c90 0x7efd00065150 secure :-1 s=READY pgs=3106 cs=0 l=1 rev1=1 rx=0x7efd10004dc0 tx=0x7efd1000b040).stop 2024-02-18T00:38:25.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd140f9640 msgr2=0x7efd1410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd140f9640 0x7efd1410e6e0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 rx=0x7efd04009eb0 tx=0x7efd0400b040).stop 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 shutdown_connections 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 --2- 172.21.15.59:0/3420486287 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd00062c90 0x7efd00065150 unknown :-1 s=CLOSED pgs=3106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd140f8d70 0x7efd1410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd141022d0 0x7efd141083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 --2- 172.21.15.59:0/3420486287 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd140f9640 0x7efd1410e6e0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 >> 172.21.15.59:0/3420486287 conn(0x7efd140f2820 msgr2=0x7efd140f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 shutdown_connections 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 -- 172.21.15.59:0/3420486287 wait complete. 2024-02-18T00:38:25.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:25.683+0000 7efd1be23700 1 librados: shutdown 2024-02-18T00:38:25.687 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.1 to 1 (10000) 2024-02-18T00:38:25.726 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:25.752 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:25.772 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:25.775 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:25.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:25.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:25.925 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:25.928 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:25.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.598144+0000 osd.1 (osd.1) 213 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.601305+0000 osd.1 (osd.1) 214 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.644884+0000 osd.6 (osd.6) 197 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.645745+0000 osd.6 (osd.6) 198 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.767387+0000 osd.3 (osd.3) 225 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:24.768667+0000 osd.3 (osd.3) 226 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:24.901662+0000 mon.a (mon.0) 2524 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.162792+0000 osd.4 (osd.4) 243 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.166245+0000 osd.4 (osd.4) 244 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:38:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:25.680096+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.680200+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e423: 8 total, 8 up, 7 in 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.598144+0000 osd.1 (osd.1) 213 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.601305+0000 osd.1 (osd.1) 214 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.644884+0000 osd.6 (osd.6) 197 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.645745+0000 osd.6 (osd.6) 198 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.767387+0000 osd.3 (osd.3) 225 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:24.768667+0000 osd.3 (osd.3) 226 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:24.901662+0000 mon.a (mon.0) 2524 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.162792+0000 osd.4 (osd.4) 243 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.166245+0000 osd.4 (osd.4) 244 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:38:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:25.680096+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:38:26.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.680200+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e423: 8 total, 8 up, 7 in 2024-02-18T00:38:26.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:26.059 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.598144+0000 osd.1 (osd.1) 213 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.601305+0000 osd.1 (osd.1) 214 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.644884+0000 osd.6 (osd.6) 197 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.645745+0000 osd.6 (osd.6) 198 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.767387+0000 osd.3 (osd.3) 225 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:24.768667+0000 osd.3 (osd.3) 226 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:24.901662+0000 mon.a (mon.0) 2524 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.162792+0000 osd.4 (osd.4) 243 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:38:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.166245+0000 osd.4 (osd.4) 244 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:38:26.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:25.680096+0000 mon.a (mon.0) 2525 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:38:26.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.680200+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e423: 8 total, 8 up, 7 in 2024-02-18T00:38:26.909 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 2 1 2024-02-18T00:38:26.910 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:26.914 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:26.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:27.015 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:27.044 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:27.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:38:27.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.643870+0000 osd.1 (osd.1) 215 : cluster [DBG] 2.d scrub starts 2024-02-18T00:38:27.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.644787+0000 osd.1 (osd.1) 216 : cluster [DBG] 2.d scrub ok 2024-02-18T00:38:27.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.646227+0000 osd.0 (osd.0) 195 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:38:27.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.649116+0000 osd.0 (osd.0) 196 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:38:27.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.729359+0000 osd.3 (osd.3) 227 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:38:27.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.732105+0000 osd.3 (osd.3) 228 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:38:27.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:25.900145+0000 mgr.y (mgr.24454) 392 : cluster [DBG] pgmap v624: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:27.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:26.146240+0000 osd.4 (osd.4) 245 : cluster [DBG] 25.13 deep-scrub starts 2024-02-18T00:38:27.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:26.149164+0000 osd.4 (osd.4) 246 : cluster [DBG] 25.13 deep-scrub ok 2024-02-18T00:38:27.117 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:27.121 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:27.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.643870+0000 osd.1 (osd.1) 215 : cluster [DBG] 2.d scrub starts 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.644787+0000 osd.1 (osd.1) 216 : cluster [DBG] 2.d scrub ok 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.646227+0000 osd.0 (osd.0) 195 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.649116+0000 osd.0 (osd.0) 196 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.729359+0000 osd.3 (osd.3) 227 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.732105+0000 osd.3 (osd.3) 228 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:25.900145+0000 mgr.y (mgr.24454) 392 : cluster [DBG] pgmap v624: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:26.146240+0000 osd.4 (osd.4) 245 : cluster [DBG] 25.13 deep-scrub starts 2024-02-18T00:38:27.143 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:26.149164+0000 osd.4 (osd.4) 246 : cluster [DBG] 25.13 deep-scrub ok 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.643870+0000 osd.1 (osd.1) 215 : cluster [DBG] 2.d scrub starts 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.644787+0000 osd.1 (osd.1) 216 : cluster [DBG] 2.d scrub ok 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.646227+0000 osd.0 (osd.0) 195 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.649116+0000 osd.0 (osd.0) 196 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.729359+0000 osd.3 (osd.3) 227 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.732105+0000 osd.3 (osd.3) 228 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:25.900145+0000 mgr.y (mgr.24454) 392 : cluster [DBG] pgmap v624: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:26.146240+0000 osd.4 (osd.4) 245 : cluster [DBG] 25.13 deep-scrub starts 2024-02-18T00:38:27.144 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:26.149164+0000 osd.4 (osd.4) 246 : cluster [DBG] 25.13 deep-scrub ok 2024-02-18T00:38:27.222 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:27.249 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:27.373 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:27.376 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:27.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:27.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:27.506 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:27.637 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:38:27.637 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:38:27.637 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:38:27.637 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:38:27.637 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:38:27.859 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:27.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:27.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:27.964 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:27.994 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:28.068 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:28.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:28.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:28.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:28.201 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.135186+0000 osd.4 (osd.4) 247 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:38:28.201 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.136745+0000 osd.4 (osd.4) 248 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:38:28.201 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.135186+0000 osd.4 (osd.4) 247 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:38:28.201 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.136745+0000 osd.4 (osd.4) 248 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:38:28.203 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:28.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.135186+0000 osd.4 (osd.4) 247 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:38:28.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.136745+0000 osd.4 (osd.4) 248 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:38:28.291 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-18T00:38:28.556 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:29.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.175+0000 7f299cbfa700 1 -- 172.21.15.59:0/2609644848 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 msgr2=0x7f29980f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.175+0000 7f299cbfa700 1 --2- 172.21.15.59:0/2609644848 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f29980f6e80 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 rx=0x7f29880047b0 tx=0x7f2988009cb0).stop 2024-02-18T00:38:29.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 -- 172.21.15.59:0/2609644848 shutdown_connections 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 --2- 172.21.15.59:0/2609644848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2998104600 0x7f29981069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 --2- 172.21.15.59:0/2609644848 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f29980f6e80 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 --2- 172.21.15.59:0/2609644848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29980f7370 0x7f2998104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 -- 172.21.15.59:0/2609644848 >> 172.21.15.59:0/2609644848 conn(0x7f29980f2820 msgr2=0x7f29980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 -- 172.21.15.59:0/2609644848 shutdown_connections 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 -- 172.21.15.59:0/2609644848 wait complete. 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 librados: starting msgr at 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 librados: starting objecter 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 Processor -- start 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 -- start start 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 librados: setting wanted keys 2024-02-18T00:38:29.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.176+0000 7f299cbfa700 1 librados: calling monclient init 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299cbfa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f2998110380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299cbfa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29980f7370 0x7f2998110870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299cbfa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2998104600 0x7f299810a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299cbfa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299810aa10 con 0x7f29980f7370 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299cbfa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299810ab80 con 0x7f2998104600 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299cbfa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299810ad40 con 0x7f29980f6aa0 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f2998110380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f2995d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29980f7370 0x7f2998110870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f2998110380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:59578/0 (socket says 172.21.15.59:59578) 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f2995d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29980f7370 0x7f2998110870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47210/0 (socket says 172.21.15.59:47210) 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 -- 172.21.15.59:0/4076807136 learned_addr learned my addr 172.21.15.59:0/4076807136 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:29.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f2996d9d700 1 --2- 172.21.15.59:0/4076807136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2998104600 0x7f299810a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 -- 172.21.15.59:0/4076807136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2998104600 msgr2=0x7f299810a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2998104600 0x7f299810a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 -- 172.21.15.59:0/4076807136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29980f7370 msgr2=0x7f2998110870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29980f7370 0x7f2998110870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 -- 172.21.15.59:0/4076807136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988005040 con 0x7f29980f6aa0 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f2995d9b700 1 --2- 172.21.15.59:0/4076807136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29980f7370 0x7f2998110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f2998110380 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 rx=0x7f2988009b10 tx=0x7f2988009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f29877fe700 10 client.?.objecter ms_handle_connect 0x7f29980f6aa0 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.177+0000 7f29877fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.178+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988018070 con 0x7f29980f6aa0 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.178+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299810afc0 con 0x7f29980f6aa0 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.178+0000 7f299cbfa700 10 client.41123.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.178+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29981171b0 con 0x7f29980f6aa0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.178+0000 7f299cbfa700 1 librados: init done 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.178+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f298801d420 con 0x7f29980f6aa0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.178+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298801c3c0 con 0x7f29980f6aa0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.179+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2978004f30 con 0x7f29980f6aa0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.179+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f298801c7f0 con 0x7f29980f6aa0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2980062c90 0x7f2980065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(424..424 src has 180..424) v4 ==== 8063+0+0 (secure 0 0 0) 0x7f298807e350 con 0x7f29980f6aa0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 osd_map(424..424 src has 180..424) v4 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 3 client.41123.objecter handle_osd_map got epochs [424,424] > 0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 3 client.41123.objecter handle_osd_map decoding full epoch 424 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f2995d9b700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2980062c90 0x7f2980065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 20 client.41123.objecter dump_active .. 0 homeless 2024-02-18T00:38:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v424) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2988085070 con 0x7f29980f6aa0 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 10 client.41123.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 1 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2978004560 0x7f29780049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 1 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 0 1 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f29780051d0 osd=0 4 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 10 client.41123.objecter _send_command 1 2024-02-18T00:38:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2978008960 con 0x7f2978004560 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f2995d9b700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2980062c90 0x7f2980065150 secure :-1 s=READY pgs=3107 cs=0 l=1 rev1=1 rx=0x7f299810bcc0 tx=0x7f298c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f2980062c90 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.180+0000 7f2996d9d700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2978004560 0x7f29780049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.181+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f2978004560 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.181+0000 7f2996d9d700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2978004560 0x7f29780049a0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.181+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2978008960 con 0x7f2978004560 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.181+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.181+0000 7f29877fe700 10 client.41123.objecter _finish_command 1 = system:0 2024-02-18T00:38:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.181+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f29780051d0 osd=0 4 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.181+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 0 1 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 10 client.41123.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 2 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 2 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f29780051d0 osd=0 3 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 0 2 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f29780051d0 osd=0 4 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 10 client.41123.objecter _send_command 2 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.196+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2978003fa0 con 0x7f2978004560 2024-02-18T00:38:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2978003fa0 con 0x7f2978004560 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f29877fe700 10 client.41123.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f29780051d0 osd=0 4 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 0 2 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 10 client.41123.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 3 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6805/26279110 conn(0x7f2978002740 0x7f2978002bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 3 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 1 3 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007fb0 osd=1 4 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 10 client.41123.objecter _send_command 3 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2978008700 con 0x7f2978002740 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.198+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6805/26279110 conn(0x7f2978002740 0x7f2978002bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.199+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f2978002740 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.199+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6805/26279110 conn(0x7f2978002740 0x7f2978002bd0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.199+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2978008700 con 0x7f2978002740 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.199+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.199+0000 7f29877fe700 10 client.41123.objecter _finish_command 3 = system:0 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.199+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978007fb0 osd=1 4 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.199+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 1 3 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 10 client.41123.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 4 2024-02-18T00:38:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 4 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978007fb0 osd=1 3 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 1 4 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007fb0 osd=1 4 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 10 client.41123.objecter _send_command 4 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.215+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2978003890 con 0x7f2978002740 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2978003890 con 0x7f2978002740 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f29877fe700 10 client.41123.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978007fb0 osd=1 4 2024-02-18T00:38:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 1 4 2024-02-18T00:38:29.220 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:38:29.220 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:38:29.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 10 client.41123.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 5 2024-02-18T00:38:29.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6809/1153193637 conn(0x7f297800aff0 0x7f297800d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 5 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 2 5 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007ae0 osd=2 4 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 10 client.41123.objecter _send_command 5 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2978010030 con 0x7f297800aff0 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f2995d9b700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6809/1153193637 conn(0x7f297800aff0 0x7f297800d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f297800aff0 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.217+0000 7f2995d9b700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6809/1153193637 conn(0x7f297800aff0 0x7f297800d3e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.218+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2978010030 con 0x7f297800aff0 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.218+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.218+0000 7f29877fe700 10 client.41123.objecter _finish_command 5 = system:0 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.218+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978007ae0 osd=2 4 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.218+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 2 5 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 10 client.41123.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 6 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 6 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978007ae0 osd=2 3 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 2 6 2024-02-18T00:38:29.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007ae0 osd=2 4 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 10 client.41123.objecter _send_command 6 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.232+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2978003300 con 0x7f297800aff0 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2978003300 con 0x7f297800aff0 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f29877fe700 10 client.41123.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978007ae0 osd=2 4 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 2 6 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 10 client.41123.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 7 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6813/1805700964 conn(0x7f297800ea50 0x7f297800eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 7 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 3 7 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978001bb0 osd=3 4 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 10 client.41123.objecter _send_command 7 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29780022a0 con 0x7f297800ea50 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.234+0000 7f2996d9d700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6813/1805700964 conn(0x7f297800ea50 0x7f297800eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.235+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f297800ea50 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.235+0000 7f2996d9d700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6813/1805700964 conn(0x7f297800ea50 0x7f297800eee0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.235+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f29780022a0 con 0x7f297800ea50 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.235+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.235+0000 7f29877fe700 10 client.41123.objecter _finish_command 7 = system:0 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.235+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978001bb0 osd=3 4 2024-02-18T00:38:29.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.235+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 3 7 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 10 client.41123.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 8 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 8 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978001bb0 osd=3 3 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 3 8 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978001bb0 osd=3 4 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 10 client.41123.objecter _send_command 8 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.249+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f297800fba0 con 0x7f297800ea50 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f297800fba0 con 0x7f297800ea50 2024-02-18T00:38:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f29877fe700 10 client.41123.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.226 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:38:29.226 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:38:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978001bb0 osd=3 4 2024-02-18T00:38:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 3 8 2024-02-18T00:38:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 10 client.41123.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 9 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6800/2044387056 conn(0x7f29780121b0 0x7f29780145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 9 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 4 9 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800f3d0 osd=4 4 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 10 client.41123.objecter _send_command 9 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f297800fba0 con 0x7f29780121b0 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.251+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6800/2044387056 conn(0x7f29780121b0 0x7f29780145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.252+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6800/2044387056 conn(0x7f29780121b0 0x7f29780145a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.252+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f29780121b0 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.253+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f297800fba0 con 0x7f29780121b0 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.253+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.253+0000 7f29877fe700 10 client.41123.objecter _finish_command 9 = system:0 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.253+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f297800f3d0 osd=4 4 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.253+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 4 9 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 10 client.41123.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 10 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 10 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f297800f3d0 osd=4 3 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 4 10 2024-02-18T00:38:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800f3d0 osd=4 4 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 10 client.41123.objecter _send_command 10 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.267+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f297800fde0 con 0x7f29780121b0 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f297800fde0 con 0x7f29780121b0 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f29877fe700 10 client.41123.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f297800f3d0 osd=4 4 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 4 10 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 10 client.41123.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 11 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2978014fa0 0x7f29780153e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 11 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 5 11 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800fde0 osd=5 4 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 10 client.41123.objecter _send_command 11 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.269+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2978017060 con 0x7f2978014fa0 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.270+0000 7f2995d9b700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2978014fa0 0x7f29780153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.230 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.270+0000 7f2995d9b700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2978014fa0 0x7f29780153e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.270+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f2978014fa0 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.271+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2978017060 con 0x7f2978014fa0 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.271+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.271+0000 7f29877fe700 10 client.41123.objecter _finish_command 11 = system:0 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.271+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f297800fde0 osd=5 4 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.271+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 5 11 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 10 client.41123.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 12 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 12 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f297800fde0 osd=5 3 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 5 12 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800fde0 osd=5 4 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 10 client.41123.objecter _send_command 12 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.285+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f2978017060 con 0x7f2978014fa0 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f2978017060 con 0x7f2978014fa0 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f29877fe700 10 client.41123.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f297800fde0 osd=5 4 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 5 12 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 10 client.41123.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 13 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2978019800 0x7f2978019c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 13 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 6 13 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978017060 osd=6 4 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 10 client.41123.objecter _send_command 13 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f297801a2c0 con 0x7f2978019800 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.287+0000 7f2996d9d700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2978019800 0x7f2978019c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.288+0000 7f2996d9d700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2978019800 0x7f2978019c20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.288+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f2978019800 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.289+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f297801a2c0 con 0x7f2978019800 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.289+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:38:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.289+0000 7f29877fe700 10 client.41123.objecter _finish_command 13 = system:0 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.289+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978017060 osd=6 4 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.289+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 6 13 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 10 client.41123.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 14 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:38:29.234 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 14 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f2978017060 osd=6 3 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 6 14 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978017060 osd=6 4 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 10 client.41123.objecter _send_command 14 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.303+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f297801a2c0 con 0x7f2978019800 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f297801a2c0 con 0x7f2978019800 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f29877fe700 10 client.41123.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f2978017060 osd=6 4 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 6 14 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 10 client.41123.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 15 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6812/53025171 conn(0x7f297801ca80 0x7f297801cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 15 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 7 15 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297801a2c0 osd=7 4 2024-02-18T00:38:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 10 client.41123.objecter _send_command 15 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f297801d550 con 0x7f297801ca80 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.305+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6812/53025171 conn(0x7f297801ca80 0x7f297801cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.306+0000 7f29877fe700 10 client.41123.objecter ms_handle_connect 0x7f297801ca80 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.306+0000 7f299659c700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6812/53025171 conn(0x7f297801ca80 0x7f297801cee0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.306+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f297801d550 con 0x7f297801ca80 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.306+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.306+0000 7f29877fe700 10 client.41123.objecter _finish_command 15 = system:0 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.306+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f297801a2c0 osd=7 4 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.306+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 7 15 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 10 client.41123.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign -1 16 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 20 client.41123.objecter _get_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_remove -1 16 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 20 client.41123.objecter get_session s=0x7f297801a2c0 osd=7 3 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 15 client.41123.objecter _session_command_op_assign 7 16 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297801a2c0 osd=7 4 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 10 client.41123.objecter _send_command 16 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.321+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f297801d550 con 0x7f297801ca80 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f29877fe700 1 -- 172.21.15.59:0/4076807136 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f297801d550 con 0x7f297801ca80 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f29877fe700 10 client.41123.objecter ms_dispatch 0x7f2998001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f29877fe700 10 client.41123.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f29877fe700 20 client.41123.objecter put_session s=0x7f297801a2c0 osd=7 4 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f29877fe700 15 client.41123.objecter _session_command_op_remove 7 16 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 librados: watch_flush enter 2024-02-18T00:38:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 librados: watch_flush exit 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.0 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2978004560 msgr2=0x7f29780049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2978004560 0x7f29780049a0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f29780051d0 osd=0 1 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.1 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6805/26279110 conn(0x7f2978002740 msgr2=0x7f2978002bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6805/26279110 conn(0x7f2978002740 0x7f2978002bd0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007fb0 osd=1 1 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.2 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6809/1153193637 conn(0x7f297800aff0 msgr2=0x7f297800d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6809/1153193637 conn(0x7f297800aff0 0x7f297800d3e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978007ae0 osd=2 1 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.3 2024-02-18T00:38:29.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6813/1805700964 conn(0x7f297800ea50 msgr2=0x7f297800eee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6813/1805700964 conn(0x7f297800ea50 0x7f297800eee0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978001bb0 osd=3 1 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.4 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6800/2044387056 conn(0x7f29780121b0 msgr2=0x7f29780145a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6800/2044387056 conn(0x7f29780121b0 0x7f29780145a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f2996d9d700 1 -- 172.21.15.59:0/4076807136 reap_dead start 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800f3d0 osd=4 1 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.5 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2978014fa0 msgr2=0x7f29780153e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2978014fa0 0x7f29780153e0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f2996d9d700 1 -- 172.21.15.59:0/4076807136 reap_dead start 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297800fde0 osd=5 1 2024-02-18T00:38:29.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.6 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2978019800 msgr2=0x7f2978019c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2978019800 0x7f2978019c20 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f2978017060 osd=6 1 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter close_session for osd.7 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6812/53025171 conn(0x7f297801ca80 msgr2=0x7f297801cee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.137:6812/53025171 conn(0x7f297801ca80 0x7f297801cee0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter put_session s=0x7f297801a2c0 osd=7 1 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f2996d9d700 1 -- 172.21.15.59:0/4076807136 reap_dead start 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 20 client.41123.objecter shutdown clearing up homeless session... 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 10 client.41123.objecter successfully canceled tick 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2980062c90 msgr2=0x7f2980065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2980062c90 0x7f2980065150 secure :-1 s=READY pgs=3107 cs=0 l=1 rev1=1 rx=0x7f299810bcc0 tx=0x7f298c00b040).stop 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 msgr2=0x7f2998110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f2998110380 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 rx=0x7f2988009b10 tx=0x7f2988009cb0).stop 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 shutdown_connections 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29980f6aa0 0x7f2998110380 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 --2- 172.21.15.59:0/4076807136 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2980062c90 0x7f2980065150 unknown :-1 s=CLOSED pgs=3107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 >> 172.21.15.59:0/4076807136 conn(0x7f29980f2820 msgr2=0x7f29980f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 shutdown_connections 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 -- 172.21.15.59:0/4076807136 wait complete. 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:28.323+0000 7f299cbfa700 1 librados: shutdown 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.704407+0000 osd.0 (osd.0) 197 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.705438+0000 osd.0 (osd.0) 198 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.876797+0000 osd.6 (osd.6) 199 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.876949+0000 osd.6 (osd.6) 200 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.900550+0000 mgr.y (mgr.24454) 393 : cluster [DBG] pgmap v625: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:27.917963+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e424: 8 total, 8 up, 7 in 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.135587+0000 osd.4 (osd.4) 249 : cluster [DBG] 62.d scrub starts 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.139050+0000 osd.4 (osd.4) 250 : cluster [DBG] 62.d scrub ok 2024-02-18T00:38:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:28.181202+0000 mon.c (mon.2) 414 : audit [DBG] from='client.? 172.21.15.59:0/4076807136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.677077+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e425: 8 total, 8 up, 7 in 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.704407+0000 osd.0 (osd.0) 197 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.705438+0000 osd.0 (osd.0) 198 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.876797+0000 osd.6 (osd.6) 199 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.876949+0000 osd.6 (osd.6) 200 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.900550+0000 mgr.y (mgr.24454) 393 : cluster [DBG] pgmap v625: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:27.917963+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e424: 8 total, 8 up, 7 in 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.135587+0000 osd.4 (osd.4) 249 : cluster [DBG] 62.d scrub starts 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.139050+0000 osd.4 (osd.4) 250 : cluster [DBG] 62.d scrub ok 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:28.181202+0000 mon.c (mon.2) 414 : audit [DBG] from='client.? 172.21.15.59:0/4076807136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.677077+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e425: 8 total, 8 up, 7 in 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.704407+0000 osd.0 (osd.0) 197 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.705438+0000 osd.0 (osd.0) 198 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.876797+0000 osd.6 (osd.6) 199 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.876949+0000 osd.6 (osd.6) 200 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.900550+0000 mgr.y (mgr.24454) 393 : cluster [DBG] pgmap v625: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:27.917963+0000 mon.a (mon.0) 2527 : cluster [DBG] osdmap e424: 8 total, 8 up, 7 in 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.135587+0000 osd.4 (osd.4) 249 : cluster [DBG] 62.d scrub starts 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.139050+0000 osd.4 (osd.4) 250 : cluster [DBG] 62.d scrub ok 2024-02-18T00:38:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:28.181202+0000 mon.c (mon.2) 414 : audit [DBG] from='client.? 172.21.15.59:0/4076807136' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:29.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.677077+0000 mon.a (mon.0) 2528 : cluster [DBG] osdmap e425: 8 total, 8 up, 7 in 2024-02-18T00:38:29.729 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:29.730 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:29.757 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:29.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:29.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:29.888 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:30.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.667281+0000 osd.1 (osd.1) 217 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:38:30.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.669043+0000 osd.1 (osd.1) 218 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:38:30.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.690763+0000 osd.0 (osd.0) 199 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:38:30.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.691951+0000 osd.0 (osd.0) 200 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.805942+0000 osd.3 (osd.3) 229 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:28.807224+0000 osd.3 (osd.3) 230 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:29.088235+0000 osd.4 (osd.4) 251 : cluster [DBG] 25.19 deep-scrub starts 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:29.090646+0000 osd.4 (osd.4) 252 : cluster [DBG] 25.19 deep-scrub ok 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.667281+0000 osd.1 (osd.1) 217 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.669043+0000 osd.1 (osd.1) 218 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.690763+0000 osd.0 (osd.0) 199 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:38:30.052 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.691951+0000 osd.0 (osd.0) 200 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:38:30.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.805942+0000 osd.3 (osd.3) 229 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:38:30.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:28.807224+0000 osd.3 (osd.3) 230 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:38:30.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:29.088235+0000 osd.4 (osd.4) 251 : cluster [DBG] 25.19 deep-scrub starts 2024-02-18T00:38:30.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:29.090646+0000 osd.4 (osd.4) 252 : cluster [DBG] 25.19 deep-scrub ok 2024-02-18T00:38:30.088 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:30.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:30.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:30.192 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:30.221 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:30.246 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:38:30.246 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:38:30.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.667281+0000 osd.1 (osd.1) 217 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:38:30.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.669043+0000 osd.1 (osd.1) 218 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:38:30.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.690763+0000 osd.0 (osd.0) 199 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:38:30.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.691951+0000 osd.0 (osd.0) 200 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:38:30.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.805942+0000 osd.3 (osd.3) 229 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:38:30.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:28.807224+0000 osd.3 (osd.3) 230 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:38:30.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:29.088235+0000 osd.4 (osd.4) 251 : cluster [DBG] 25.19 deep-scrub starts 2024-02-18T00:38:30.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:29.090646+0000 osd.4 (osd.4) 252 : cluster [DBG] 25.19 deep-scrub ok 2024-02-18T00:38:30.344 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:30.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:30.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:30.448 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:30.481 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:30.580 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:30.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:30.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:30.684 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:30.711 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:30.877 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:30.881 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:30.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:30.888 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-18T00:38:30.982 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:31.013 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:31.192 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:31.246 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:29.542289+0000 osd.5 (osd.5) 179 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:38:31.246 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:29.545777+0000 osd.5 (osd.5) 180 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:38:31.246 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:29.677696+0000 osd.6 (osd.6) 201 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:38:31.246 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:29.678572+0000 osd.6 (osd.6) 202 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:38:31.246 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:29.901088+0000 mgr.y (mgr.24454) 394 : cluster [DBG] pgmap v628: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:31.246 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:29.542289+0000 osd.5 (osd.5) 179 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:29.545777+0000 osd.5 (osd.5) 180 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:29.677696+0000 osd.6 (osd.6) 201 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:29.678572+0000 osd.6 (osd.6) 202 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:29.901088+0000 mgr.y (mgr.24454) 394 : cluster [DBG] pgmap v628: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:29.542289+0000 osd.5 (osd.5) 179 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:29.545777+0000 osd.5 (osd.5) 180 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:29.677696+0000 osd.6 (osd.6) 201 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:38:31.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:29.678572+0000 osd.6 (osd.6) 202 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:38:31.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:29.901088+0000 mgr.y (mgr.24454) 394 : cluster [DBG] pgmap v628: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:31.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:38:31.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:30.698798+0000 osd.0 (osd.0) 201 : cluster [DBG] 62.e scrub starts 2024-02-18T00:38:31.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:30.700344+0000 osd.0 (osd.0) 202 : cluster [DBG] 62.e scrub ok 2024-02-18T00:38:31.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:30.789579+0000 osd.3 (osd.3) 231 : cluster [DBG] 62.c scrub starts 2024-02-18T00:38:31.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:30.791389+0000 osd.3 (osd.3) 232 : cluster [DBG] 62.c scrub ok 2024-02-18T00:38:31.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.183130+0000 osd.3 (osd.3) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:31.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.183406+0000 osd.3 (osd.3) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:31.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:31.244652+0000 mgr.y (mgr.24454) 395 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:30.698798+0000 osd.0 (osd.0) 201 : cluster [DBG] 62.e scrub starts 2024-02-18T00:38:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:30.700344+0000 osd.0 (osd.0) 202 : cluster [DBG] 62.e scrub ok 2024-02-18T00:38:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:30.789579+0000 osd.3 (osd.3) 231 : cluster [DBG] 62.c scrub starts 2024-02-18T00:38:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:30.791389+0000 osd.3 (osd.3) 232 : cluster [DBG] 62.c scrub ok 2024-02-18T00:38:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.183130+0000 osd.3 (osd.3) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.183406+0000 osd.3 (osd.3) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:31.244652+0000 mgr.y (mgr.24454) 395 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:30.698798+0000 osd.0 (osd.0) 201 : cluster [DBG] 62.e scrub starts 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:30.700344+0000 osd.0 (osd.0) 202 : cluster [DBG] 62.e scrub ok 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:30.789579+0000 osd.3 (osd.3) 231 : cluster [DBG] 62.c scrub starts 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:30.791389+0000 osd.3 (osd.3) 232 : cluster [DBG] 62.c scrub ok 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.183130+0000 osd.3 (osd.3) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.183406+0000 osd.3 (osd.3) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:31.244652+0000 mgr.y (mgr.24454) 395 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:32.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:32] "GET /metrics HTTP/1.1" 200 189625 "" "Prometheus/2.33.4" 2024-02-18T00:38:32.759 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:38:32.759 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:38:33.060 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.654883+0000 osd.1 (osd.1) 219 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:38:33.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.657774+0000 osd.1 (osd.1) 220 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:38:33.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.706801+0000 osd.6 (osd.6) 203 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:38:33.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.708774+0000 osd.6 (osd.6) 204 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:38:33.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.740785+0000 osd.3 (osd.3) 235 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:38:33.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.741668+0000 osd.3 (osd.3) 236 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:38:33.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.902300+0000 mgr.y (mgr.24454) 396 : cluster [DBG] pgmap v629: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:33.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:31.929443+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e426: 8 total, 8 up, 7 in 2024-02-18T00:38:33.064 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:33.067 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:33.067 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:33.168 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:33.198 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.654883+0000 osd.1 (osd.1) 219 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.657774+0000 osd.1 (osd.1) 220 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.706801+0000 osd.6 (osd.6) 203 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.708774+0000 osd.6 (osd.6) 204 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.740785+0000 osd.3 (osd.3) 235 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.741668+0000 osd.3 (osd.3) 236 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.902300+0000 mgr.y (mgr.24454) 396 : cluster [DBG] pgmap v629: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:31.929443+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e426: 8 total, 8 up, 7 in 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.654883+0000 osd.1 (osd.1) 219 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.657774+0000 osd.1 (osd.1) 220 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.706801+0000 osd.6 (osd.6) 203 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.708774+0000 osd.6 (osd.6) 204 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.740785+0000 osd.3 (osd.3) 235 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.741668+0000 osd.3 (osd.3) 236 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.902300+0000 mgr.y (mgr.24454) 396 : cluster [DBG] pgmap v629: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:33.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:31.929443+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e426: 8 total, 8 up, 7 in 2024-02-18T00:38:33.350 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:33.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:33.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:33.368 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-18T00:38:33.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:33.481 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:33.501 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:33.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:33.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:33.605 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:33.634 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:33.654 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:33.657 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:33.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:33.758 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:33.785 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:33.983 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:33.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:33.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 -- 172.21.15.59:0/2674776661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fe550 msgr2=0x7fe6a00f6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 --2- 172.21.15.59:0/2674776661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fe550 0x7fe6a00f6120 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 rx=0x7fe694004870 tx=0x7fe694009cb0).stop 2024-02-18T00:38:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 -- 172.21.15.59:0/2674776661 shutdown_connections 2024-02-18T00:38:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 --2- 172.21.15.59:0/2674776661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a00fdbf0 0x7fe6a00fdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 --2- 172.21.15.59:0/2674776661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00f67e0 0x7fe6a0104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 --2- 172.21.15.59:0/2674776661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fe550 0x7fe6a00f6120 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 -- 172.21.15.59:0/2674776661 >> 172.21.15.59:0/2674776661 conn(0x7fe6a0070030 msgr2=0x7fe6a00f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 -- 172.21.15.59:0/2674776661 shutdown_connections 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 -- 172.21.15.59:0/2674776661 wait complete. 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 librados: starting msgr at 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.905+0000 7fe6a8457700 1 librados: starting objecter 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 Processor -- start 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 -- start start 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 librados: setting wanted keys 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 librados: calling monclient init 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a00f67e0 0x7fe6a00f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fdbf0 0x7fe6a00f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00fe550 0x7fe6a00f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a01165b0 con 0x7fe6a00fe550 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a0116720 con 0x7fe6a00f67e0 2024-02-18T00:38:34.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a8457700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a01168e0 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fdbf0 0x7fe6a00f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a69f4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00fe550 0x7fe6a00f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fdbf0 0x7fe6a00f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:53068/0 (socket says 172.21.15.59:53068) 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 -- 172.21.15.59:0/431704852 learned_addr learned my addr 172.21.15.59:0/431704852 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a69f4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00fe550 0x7fe6a00f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44928/0 (socket says 172.21.15.59:44928) 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a61f3700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a00f67e0 0x7fe6a00f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 -- 172.21.15.59:0/431704852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a00f67e0 msgr2=0x7fe6a00f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a00f67e0 0x7fe6a00f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 -- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00fe550 msgr2=0x7fe6a00f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00fe550 0x7fe6a00f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a59f2700 1 -- 172.21.15.59:0/431704852 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900097c0 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.906+0000 7fe6a69f4700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00fe550 0x7fe6a00f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:34.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe6a59f2700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fdbf0 0x7fe6a00f78e0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 rx=0x7fe694009eb0 tx=0x7fe69400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe69f7fe700 10 client.?.objecter ms_handle_connect 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe69f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe69f7fe700 1 -- 172.21.15.59:0/431704852 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69400e070 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe69f7fe700 1 -- 172.21.15.59:0/431704852 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe694013da0 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe69f7fe700 1 -- 172.21.15.59:0/431704852 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe694019070 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe694005040 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe6a8457700 10 client.41129.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a0116d80 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.907+0000 7fe6a8457700 1 librados: init done 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.908+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a005f990 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe69f7fe700 1 -- 172.21.15.59:0/431704852 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe69401e420 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe69f7fe700 1 --2- 172.21.15.59:0/431704852 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe688062c90 0x7fe688065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe69f7fe700 1 -- 172.21.15.59:0/431704852 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(427..427 src has 180..427) v4 ==== 8063+0+0 (secure 0 0 0) 0x7fe69407dc90 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe69f7fe700 10 client.41129.objecter ms_dispatch 0x7fe6a0001550 osd_map(427..427 src has 180..427) v4 2024-02-18T00:38:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe69f7fe700 3 client.41129.objecter handle_osd_map got epochs [427,427] > 0 2024-02-18T00:38:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe69f7fe700 3 client.41129.objecter handle_osd_map decoding full epoch 427 2024-02-18T00:38:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe6a61f3700 1 --2- 172.21.15.59:0/431704852 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe688062c90 0x7fe688065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.909+0000 7fe69f7fe700 20 client.41129.objecter dump_active .. 0 homeless 2024-02-18T00:38:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.910+0000 7fe69f7fe700 10 client.41129.objecter ms_handle_connect 0x7fe688062c90 2024-02-18T00:38:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.910+0000 7fe6a61f3700 1 --2- 172.21.15.59:0/431704852 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe688062c90 0x7fe688065150 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 rx=0x7fe6a0060c00 tx=0x7fe69800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:34.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.913+0000 7fe69f7fe700 1 -- 172.21.15.59:0/431704852 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe694051280 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.089 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/2938615742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0fd510 msgr2=0x7fb7ec0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/2938615742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0fd510 0x7fb7ec0f6d40 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 rx=0x7fb7d8004870 tx=0x7fb7d8009cb0).stop 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/2938615742 shutdown_connections 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/2938615742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec0f7400 0x7fb7ec103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/2938615742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 0x7fb7ec0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/2938615742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0fd510 0x7fb7ec0f6d40 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/2938615742 >> 172.21.15.59:0/2938615742 conn(0x7fb7ec0f2820 msgr2=0x7fb7ec0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/2938615742 shutdown_connections 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.984+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/2938615742 wait complete. 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 librados: starting msgr at 2024-02-18T00:38:34.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 librados: starting objecter 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 Processor -- start 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 -- start start 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 librados: setting wanted keys 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 librados: calling monclient init 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0f7400 0x7fb7ec1092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 0x7fb7ec1097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec0fd510 0x7fb7ec109da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec1144b0 con 0x7fb7ec0f7400 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec114620 con 0x7fb7ec0fd510 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f2d1d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec1147e0 con 0x7fb7ec0fcbb0 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f0ab9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0f7400 0x7fb7ec1092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 0x7fb7ec1097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f0ab9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0f7400 0x7fb7ec1092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44952/0 (socket says 172.21.15.59:44952) 2024-02-18T00:38:34.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 0x7fb7ec1097c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:53076/0 (socket says 172.21.15.59:53076) 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f0ab9700 1 -- 172.21.15.59:0/16481304 learned_addr learned my addr 172.21.15.59:0/16481304 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.985+0000 7fb7f12ba700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec0fd510 0x7fb7ec109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f0ab9700 1 -- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 msgr2=0x7fb7ec1097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f0ab9700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 0x7fb7ec1097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f0ab9700 1 -- 172.21.15.59:0/16481304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec0fd510 msgr2=0x7fb7ec109da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f0ab9700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec0fd510 0x7fb7ec109da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f0ab9700 1 -- 172.21.15.59:0/16481304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0005040 con 0x7fb7ec0f7400 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7ebfff700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 0x7fb7ec1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f0ab9700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0f7400 0x7fb7ec1092d0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 rx=0x7fb7e00048f0 tx=0x7fb7e0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7e9ffb700 10 client.?.objecter ms_handle_connect 0x7fb7ec0f7400 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7e9ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7e9ffb700 1 -- 172.21.15.59:0/16481304 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0007b00 con 0x7fb7ec0f7400 2024-02-18T00:38:34.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7e9ffb700 1 -- 172.21.15.59:0/16481304 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7e000f040 con 0x7fb7ec0f7400 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d8005040 con 0x7fb7ec0f7400 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7e9ffb700 1 -- 172.21.15.59:0/16481304 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0014990 con 0x7fb7ec0f7400 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f2d1d700 10 client.31332.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec114e70 con 0x7fb7ec0f7400 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.986+0000 7fb7f2d1d700 1 librados: init done 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.987+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec05fa60 con 0x7fb7ec0f7400 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.987+0000 7fb7e9ffb700 1 -- 172.21.15.59:0/16481304 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb7e0010500 con 0x7fb7ec0f7400 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7e9ffb700 1 --2- 172.21.15.59:0/16481304 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb7dc062c90 0x7fb7dc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7e9ffb700 1 -- 172.21.15.59:0/16481304 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(427..427 src has 180..427) v4 ==== 8063+0+0 (secure 0 0 0) 0x7fb7e007cd30 con 0x7fb7ec0f7400 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7ebfff700 1 --2- 172.21.15.59:0/16481304 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb7dc062c90 0x7fb7dc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7e9ffb700 10 client.31332.objecter ms_dispatch 0x7fb7ec001550 osd_map(427..427 src has 180..427) v4 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7e9ffb700 3 client.31332.objecter handle_osd_map got epochs [427,427] > 0 2024-02-18T00:38:34.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7e9ffb700 3 client.31332.objecter handle_osd_map decoding full epoch 427 2024-02-18T00:38:34.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7e9ffb700 20 client.31332.objecter dump_active .. 0 homeless 2024-02-18T00:38:34.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7e9ffb700 10 client.31332.objecter ms_handle_connect 0x7fb7dc062c90 2024-02-18T00:38:34.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.988+0000 7fb7ebfff700 1 --2- 172.21.15.59:0/16481304 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb7dc062c90 0x7fb7dc065150 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 rx=0x7fb7d8004ee0 tx=0x7fb7d800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:34.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:33.992+0000 7fb7e9ffb700 1 -- 172.21.15.59:0/16481304 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb7e0049f60 con 0x7fb7ec0f7400 2024-02-18T00:38:34.117 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:34.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.120+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fe6a005ee80 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.196+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) v1 -- 0x7fb7ec106140 con 0x7fb7ec0f7400 2024-02-18T00:38:34.215 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:34.218 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:34.218 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:34.319 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:34.351 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:34.370 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:34.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:34.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:34.474 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:34.500 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:34.572 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:34.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:34.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:34.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:34.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.696+0000 7fb7e9ffb700 1 -- 172.21.15.59:0/16481304 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v428) v1 ==== 113+0+0 (secure 0 0 0) 0x7fb7e00501e0 con 0x7fb7ec0f7400 2024-02-18T00:38:34.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fe69f7fe700 1 -- 172.21.15.59:0/431704852 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v428) v1 ==== 87+0+0 (secure 0 0 0) 0x7fe694023030 con 0x7fe6a00fdbf0 2024-02-18T00:38:34.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.699+0000 7fb7f2d1d700 10 librados: watch_flush enter 2024-02-18T00:38:34.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fb7f2d1d700 10 librados: watch_flush exit 2024-02-18T00:38:34.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fb7f2d1d700 20 client.31332.objecter shutdown clearing up homeless session... 2024-02-18T00:38:34.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fb7f2d1d700 10 client.31332.objecter successfully canceled tick 2024-02-18T00:38:34.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb7dc062c90 msgr2=0x7fb7dc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/16481304 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb7dc062c90 0x7fb7dc065150 secure :-1 s=READY pgs=3109 cs=0 l=1 rev1=1 rx=0x7fb7d8004ee0 tx=0x7fb7d800b040).stop 2024-02-18T00:38:34.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0f7400 msgr2=0x7fb7ec1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.700+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0f7400 0x7fb7ec1092d0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 rx=0x7fb7e00048f0 tx=0x7fb7e0009cb0).stop 2024-02-18T00:38:34.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 shutdown_connections 2024-02-18T00:38:34.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/16481304 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb7dc062c90 0x7fb7dc065150 unknown :-1 s=CLOSED pgs=3109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7ec0fd510 0x7fb7ec109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ec0f7400 0x7fb7ec1092d0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 --2- 172.21.15.59:0/16481304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7ec0fcbb0 0x7fb7ec1097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 >> 172.21.15.59:0/16481304 conn(0x7fb7ec0f2820 msgr2=0x7fb7ec0f43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 shutdown_connections 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 -- 172.21.15.59:0/16481304 wait complete. 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.701+0000 7fb7f2d1d700 1 librados: shutdown 2024-02-18T00:38:34.704 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.2 to 1 (10000) 2024-02-18T00:38:34.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.702+0000 7fe6a8457700 10 librados: watch_flush enter 2024-02-18T00:38:34.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.702+0000 7fe6a8457700 10 librados: watch_flush exit 2024-02-18T00:38:34.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.702+0000 7fe6a8457700 20 client.41129.objecter shutdown clearing up homeless session... 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.702+0000 7fe6a8457700 10 client.41129.objecter successfully canceled tick 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe688062c90 msgr2=0x7fe688065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 --2- 172.21.15.59:0/431704852 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe688062c90 0x7fe688065150 secure :-1 s=READY pgs=3108 cs=0 l=1 rev1=1 rx=0x7fe6a0060c00 tx=0x7fe69800b040).stop 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fdbf0 msgr2=0x7fe6a00f78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fdbf0 0x7fe6a00f78e0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 rx=0x7fe694009eb0 tx=0x7fe69400b040).stop 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 shutdown_connections 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 --2- 172.21.15.59:0/431704852 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe688062c90 0x7fe688065150 unknown :-1 s=CLOSED pgs=3108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a00f67e0 0x7fe6a00f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a00fe550 0x7fe6a00f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 --2- 172.21.15.59:0/431704852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe6a00fdbf0 0x7fe6a00f78e0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 >> 172.21.15.59:0/431704852 conn(0x7fe6a0070030 msgr2=0x7fe6a006d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 shutdown_connections 2024-02-18T00:38:34.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 -- 172.21.15.59:0/431704852 wait complete. 2024-02-18T00:38:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:34.703+0000 7fe6a8457700 1 librados: shutdown 2024-02-18T00:38:34.707 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:38:34.715 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:34.740 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:34.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:34.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:34.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.311078+0000 osd.2 (osd.2) 181 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:38:34.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.312076+0000 osd.2 (osd.2) 182 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.483447+0000 osd.5 (osd.5) 181 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.486419+0000 osd.5 (osd.5) 182 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.644472+0000 osd.1 (osd.1) 221 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.647951+0000 osd.1 (osd.1) 222 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.678141+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e427: 8 total, 8 up, 7 in 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:33.902644+0000 mgr.y (mgr.24454) 397 : cluster [DBG] pgmap v632: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 855 B/s rd, 0 op/s 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:34.122105+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.59:0/431704852' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:34.122492+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:38:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:34.197953+0000 mon.a (mon.0) 2532 : audit [INF] from='client.? 172.21.15.59:0/16481304' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:38:34.845 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:34.871 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:34.995 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:34.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:34.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.311078+0000 osd.2 (osd.2) 181 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.312076+0000 osd.2 (osd.2) 182 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.483447+0000 osd.5 (osd.5) 181 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.486419+0000 osd.5 (osd.5) 182 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.644472+0000 osd.1 (osd.1) 221 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.647951+0000 osd.1 (osd.1) 222 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.678141+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e427: 8 total, 8 up, 7 in 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:33.902644+0000 mgr.y (mgr.24454) 397 : cluster [DBG] pgmap v632: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 855 B/s rd, 0 op/s 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:34.122105+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.59:0/431704852' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:34.122492+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:38:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:34.197953+0000 mon.a (mon.0) 2532 : audit [INF] from='client.? 172.21.15.59:0/16481304' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.311078+0000 osd.2 (osd.2) 181 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.312076+0000 osd.2 (osd.2) 182 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.483447+0000 osd.5 (osd.5) 181 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.486419+0000 osd.5 (osd.5) 182 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.644472+0000 osd.1 (osd.1) 221 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.647951+0000 osd.1 (osd.1) 222 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.678141+0000 mon.a (mon.0) 2530 : cluster [DBG] osdmap e427: 8 total, 8 up, 7 in 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:33.902644+0000 mgr.y (mgr.24454) 397 : cluster [DBG] pgmap v632: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 855 B/s rd, 0 op/s 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:34.122105+0000 mon.c (mon.2) 415 : audit [INF] from='client.? 172.21.15.59:0/431704852' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:34.122492+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:38:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:34.197953+0000 mon.a (mon.0) 2532 : audit [INF] from='client.? 172.21.15.59:0/16481304' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:38:35.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:38:35.100 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:35.128 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:35.471 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:38:35.471 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:38:35.477 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:35.479 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:35.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:35.581 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:35.610 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:35.732 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:35.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:35.736 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:34.681316+0000 mon.a (mon.0) 2533 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:34.695541+0000 osd.0 (osd.0) 203 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:38:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:34.695895+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:38:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:34.695979+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.59:0/16481304' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:38:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:34.696044+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e428: 8 total, 8 up, 7 in 2024-02-18T00:38:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:34.718677+0000 osd.3 (osd.3) 237 : cluster [DBG] 62.b deep-scrub starts 2024-02-18T00:38:35.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:34.725711+0000 osd.3 (osd.3) 238 : cluster [DBG] 62.b deep-scrub ok 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:34.681316+0000 mon.a (mon.0) 2533 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:34.695541+0000 osd.0 (osd.0) 203 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:34.695895+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:34.695979+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.59:0/16481304' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:34.696044+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e428: 8 total, 8 up, 7 in 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:34.718677+0000 osd.3 (osd.3) 237 : cluster [DBG] 62.b deep-scrub starts 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:34.725711+0000 osd.3 (osd.3) 238 : cluster [DBG] 62.b deep-scrub ok 2024-02-18T00:38:35.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:34.681316+0000 mon.a (mon.0) 2533 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:34.695541+0000 osd.0 (osd.0) 203 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:38:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:34.695895+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:38:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:34.695979+0000 mon.a (mon.0) 2535 : audit [INF] from='client.? 172.21.15.59:0/16481304' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:38:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:34.696044+0000 mon.a (mon.0) 2536 : cluster [DBG] osdmap e428: 8 total, 8 up, 7 in 2024-02-18T00:38:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:34.718677+0000 osd.3 (osd.3) 237 : cluster [DBG] 62.b deep-scrub starts 2024-02-18T00:38:35.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:34.725711+0000 osd.3 (osd.3) 238 : cluster [DBG] 62.b deep-scrub ok 2024-02-18T00:38:35.836 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:35.864 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:36.212 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:36.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:36.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:36.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-18T00:38:36.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 3 1 2024-02-18T00:38:36.299 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:36.316 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:36.343 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:36.361 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:36.364 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:36.364 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:36.465 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:36.492 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:36.528 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:36.617 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:36.620 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:36.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:36.722 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:36.749 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:35.903494+0000 mgr.y (mgr.24454) 398 : cluster [DBG] pgmap v634: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:38:36.750 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:35.903494+0000 mgr.y (mgr.24454) 398 : cluster [DBG] pgmap v634: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:38:36.752 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:36.826 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:36.828 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:36.829 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:36.929 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:36.955 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:35.903494+0000 mgr.y (mgr.24454) 398 : cluster [DBG] pgmap v634: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:38:36.958 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:36.996 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:36.998 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:36.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:37.100 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:37.126 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:37.198 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:37.324 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:37.326 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:37.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:37.427 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:37.455 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:37.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:38:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:37.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:37.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:37.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:37.934 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:38.182 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:38.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:38.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:38.256 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:38:38.256 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:38:38.287 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:38.319 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:38.517 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:38.519 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:38.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:38.621 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:38.648 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:38.688 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:38.693 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:38.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:38.795 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:38.821 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:39.070 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:39.073 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:39.074 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:39.084 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-18T00:38:39.175 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:39.206 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:37.903863+0000 mgr.y (mgr.24454) 399 : cluster [DBG] pgmap v635: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:38.679431+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e429: 8 total, 8 up, 7 in 2024-02-18T00:38:39.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:37.903863+0000 mgr.y (mgr.24454) 399 : cluster [DBG] pgmap v635: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:39.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:38.679431+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e429: 8 total, 8 up, 7 in 2024-02-18T00:38:39.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:37.903863+0000 mgr.y (mgr.24454) 399 : cluster [DBG] pgmap v635: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:39.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:38.679431+0000 mon.a (mon.0) 2537 : cluster [DBG] osdmap e429: 8 total, 8 up, 7 in 2024-02-18T00:38:39.388 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:39.755 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:39.759 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:39.759 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:39.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:39.890 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:39.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 -- 172.21.15.59:0/637381010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 msgr2=0x7f1aa0101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:39.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 --2- 172.21.15.59:0/637381010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa0101d20 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 rx=0x7f1a90004870 tx=0x7f1a90009cb0).stop 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 -- 172.21.15.59:0/637381010 shutdown_connections 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 --2- 172.21.15.59:0/637381010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1aa0102210 0x7f1aa0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 --2- 172.21.15.59:0/637381010 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aa00f8cb0 0x7f1aa00f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 --2- 172.21.15.59:0/637381010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa0101d20 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 -- 172.21.15.59:0/637381010 >> 172.21.15.59:0/637381010 conn(0x7f1aa00f2820 msgr2=0x7f1aa00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 -- 172.21.15.59:0/637381010 shutdown_connections 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 -- 172.21.15.59:0/637381010 wait complete. 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 librados: starting msgr at 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.992+0000 7f1aa819b700 1 librados: starting objecter 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 Processor -- start 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 -- start start 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 librados: setting wanted keys 2024-02-18T00:38:39.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 librados: calling monclient init 2024-02-18T00:38:39.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aa00f8cb0 0x7f1aa010e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:39.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa010e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:39.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1aa0102210 0x7f1aa0108430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:39.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0108920 con 0x7f1aa00f9610 2024-02-18T00:38:39.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0108a90 con 0x7f1aa0102210 2024-02-18T00:38:39.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa819b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0108c00 con 0x7f1aa00f8cb0 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5736700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa010e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5f37700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aa00f8cb0 0x7f1aa010e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5736700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa010e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46204/0 (socket says 172.21.15.59:46204) 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5736700 1 -- 172.21.15.59:0/1907854029 learned_addr learned my addr 172.21.15.59:0/1907854029 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa6738700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1aa0102210 0x7f1aa0108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5f37700 1 -- 172.21.15.59:0/1907854029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1aa0102210 msgr2=0x7f1aa0108430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5f37700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1aa0102210 0x7f1aa0108430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:39.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5f37700 1 -- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 msgr2=0x7f1aa010e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5f37700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa010e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5f37700 1 -- 172.21.15.59:0/1907854029 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90005040 con 0x7f1aa00f8cb0 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5736700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa010e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1aa5f37700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aa00f8cb0 0x7f1aa010e250 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 rx=0x7f1a9c00e8f0 tx=0x7f1a9c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.993+0000 7f1a96ffd700 10 client.?.objecter ms_handle_connect 0x7f1aa00f8cb0 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1a96ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1a96ffd700 1 -- 172.21.15.59:0/1907854029 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c004620 con 0x7f1aa00f8cb0 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1a96ffd700 1 -- 172.21.15.59:0/1907854029 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a9c00f040 con 0x7f1aa00f8cb0 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa0108ee0 con 0x7f1aa00f8cb0 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1a96ffd700 1 -- 172.21.15.59:0/1907854029 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c0146f0 con 0x7f1aa00f8cb0 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1aa819b700 10 client.41141.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:39.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa01150c0 con 0x7f1aa00f8cb0 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.994+0000 7f1aa819b700 1 librados: init done 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.995+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa005fa60 con 0x7f1aa00f8cb0 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.995+0000 7f1a96ffd700 1 -- 172.21.15.59:0/1907854029 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1a9c0090d0 con 0x7f1aa00f8cb0 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1a96ffd700 1 --2- 172.21.15.59:0/1907854029 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1a8c062c90 0x7f1a8c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1a96ffd700 1 -- 172.21.15.59:0/1907854029 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(429..429 src has 180..429) v4 ==== 8063+0+0 (secure 0 0 0) 0x7f1a9c07ce50 con 0x7f1aa00f8cb0 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1a96ffd700 10 client.41141.objecter ms_dispatch 0x7f1aa0001550 osd_map(429..429 src has 180..429) v4 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1a96ffd700 3 client.41141.objecter handle_osd_map got epochs [429,429] > 0 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1a96ffd700 3 client.41141.objecter handle_osd_map decoding full epoch 429 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1aa5736700 1 --2- 172.21.15.59:0/1907854029 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1a8c062c90 0x7f1a8c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1a96ffd700 20 client.41141.objecter dump_active .. 0 homeless 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1aa5736700 1 --2- 172.21.15.59:0/1907854029 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1a8c062c90 0x7f1a8c065150 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 rx=0x7f1a90009bc0 tx=0x7f1a9000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:39.996+0000 7f1a96ffd700 10 client.41141.objecter ms_handle_connect 0x7f1a8c062c90 2024-02-18T00:38:40.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.000+0000 7f1a96ffd700 1 -- 172.21.15.59:0/1907854029 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1a9c0504b0 con 0x7f1aa00f8cb0 2024-02-18T00:38:40.016 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:40.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:40.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:40.123 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:40.151 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.187+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f1aa005ee80 con 0x7f1aa00f8cb0 2024-02-18T00:38:40.204 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:40.207 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:40.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:40.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.308+0000 7efc9c295700 1 -- 172.21.15.59:0/1533639954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940f8420 msgr2=0x7efc94104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.308+0000 7efc9c295700 1 --2- 172.21.15.59:0/1533639954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940f8420 0x7efc94104a20 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 rx=0x7efc840047b0 tx=0x7efc84009cb0).stop 2024-02-18T00:38:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.308+0000 7efc9c295700 1 -- 172.21.15.59:0/1533639954 shutdown_connections 2024-02-18T00:38:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.308+0000 7efc9c295700 1 --2- 172.21.15.59:0/1533639954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc940fdbb0 0x7efc940fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.308+0000 7efc9c295700 1 --2- 172.21.15.59:0/1533639954 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940f8420 0x7efc94104a20 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.308+0000 7efc9c295700 1 --2- 172.21.15.59:0/1533639954 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 0x7efc940f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.308+0000 7efc9c295700 1 -- 172.21.15.59:0/1533639954 >> 172.21.15.59:0/1533639954 conn(0x7efc940f3800 msgr2=0x7efc940f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 -- 172.21.15.59:0/1533639954 shutdown_connections 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 -- 172.21.15.59:0/1533639954 wait complete. 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 librados: starting msgr at 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 librados: starting objecter 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 Processor -- start 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 -- start start 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 librados: setting wanted keys 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 librados: calling monclient init 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc940f8420 0x7efc94110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940fdbb0 0x7efc94110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 0x7efc9410a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9410a6a0 con 0x7efc940fdbb0 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9410a810 con 0x7efc940f8420 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9c295700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9410a9d0 con 0x7efc940fe510 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9a832700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 0x7efc9410a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc99830700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940fdbb0 0x7efc94110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.309+0000 7efc9a832700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 0x7efc9410a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:53220/0 (socket says 172.21.15.59:53220) 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a832700 1 -- 172.21.15.59:0/4035561099 learned_addr learned my addr 172.21.15.59:0/4035561099 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc99830700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940fdbb0 0x7efc94110500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46228/0 (socket says 172.21.15.59:46228) 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc940f8420 0x7efc94110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a832700 1 -- 172.21.15.59:0/4035561099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc940f8420 msgr2=0x7efc94110010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a832700 1 --2- 172.21.15.59:0/4035561099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efc940f8420 0x7efc94110010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a832700 1 -- 172.21.15.59:0/4035561099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940fdbb0 msgr2=0x7efc94110500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a832700 1 --2- 172.21.15.59:0/4035561099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940fdbb0 0x7efc94110500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a832700 1 -- 172.21.15.59:0/4035561099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc88005040 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc99830700 1 --2- 172.21.15.59:0/4035561099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efc940fdbb0 0x7efc94110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9a832700 1 --2- 172.21.15.59:0/4035561099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 0x7efc9410a1b0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 rx=0x7efc840044e0 tx=0x7efc84007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc93779700 10 client.?.objecter ms_handle_connect 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc93779700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84018070 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc84005040 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc84009710 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8401c3d0 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9c295700 10 client.41147.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9410ae70 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9c295700 1 librados: init done 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.311+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7efc9405ee80 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efc8401d470 con 0x7efc940fe510 2024-02-18T00:38:40.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc7c062c90 0x7efc7c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(429..429 src has 180..429) v4 ==== 8063+0+0 (secure 0 0 0) 0x7efc8407e6e0 con 0x7efc940fe510 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 osd_map(429..429 src has 180..429) v4 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 3 client.41147.objecter handle_osd_map got epochs [429,429] > 0 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 3 client.41147.objecter handle_osd_map decoding full epoch 429 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc7c062c90 0x7efc7c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 20 client.41147.objecter dump_active .. 0 homeless 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.312+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v429) v1 ==== 54+0+16 (secure 0 0 0) 0x7efc8407ebc0 con 0x7efc940fe510 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 10 client.41147.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 1 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6801/3700238977 conn(0x7efc940f9d00 0x7efc940fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc7c062c90 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc7c062c90 0x7efc7c065150 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 rx=0x7efc8c007fb0 tx=0x7efc8c009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 1 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 0 1 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94107110 osd=0 4 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 10 client.41147.objecter _send_command 1 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9410b690 con 0x7efc940f9d00 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc99830700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6801/3700238977 conn(0x7efc940f9d00 0x7efc940fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc940f9d00 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.313+0000 7efc99830700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6801/3700238977 conn(0x7efc940f9d00 0x7efc940fa0e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.314+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9410b690 con 0x7efc940f9d00 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.314+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:38:40.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.314+0000 7efc93779700 10 client.41147.objecter _finish_command 1 = system:0 2024-02-18T00:38:40.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.314+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94107110 osd=0 4 2024-02-18T00:38:40.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.314+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 0 1 2024-02-18T00:38:40.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 10 client.41147.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 2 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 2 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94107110 osd=0 3 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 0 2 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94107110 osd=0 4 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 10 client.41147.objecter _send_command 2 2024-02-18T00:38:40.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.329+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc940273c0 con 0x7efc940f9d00 2024-02-18T00:38:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc940273c0 con 0x7efc940f9d00 2024-02-18T00:38:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc93779700 10 client.41147.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94107110 osd=0 4 2024-02-18T00:38:40.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 0 2 2024-02-18T00:38:40.333 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:38:40.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 10 client.41147.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6805/26279110 conn(0x7efc940598f0 0x7efc94059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 1 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94028710 osd=1 4 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 10 client.41147.objecter _send_command 3 2024-02-18T00:38:40.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9411d5c0 con 0x7efc940598f0 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6805/26279110 conn(0x7efc940598f0 0x7efc94059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc940598f0 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.331+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6805/26279110 conn(0x7efc940598f0 0x7efc94059cd0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.332+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9411d5c0 con 0x7efc940598f0 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.332+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.332+0000 7efc93779700 10 client.41147.objecter _finish_command 3 = system:0 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.332+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94028710 osd=1 4 2024-02-18T00:38:40.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.332+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 1 3 2024-02-18T00:38:40.337 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 10 client.41147.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 4 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 4 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94028710 osd=1 3 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 1 4 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94028710 osd=1 4 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 10 client.41147.objecter _send_command 4 2024-02-18T00:38:40.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.349+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc94059360 con 0x7efc940598f0 2024-02-18T00:38:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc94059360 con 0x7efc940598f0 2024-02-18T00:38:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 10 client.41147.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94028710 osd=1 4 2024-02-18T00:38:40.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 1 4 2024-02-18T00:38:40.352 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 10 client.41147.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 5 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6809/1153193637 conn(0x7efc94028270 0x7efc9411bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 5 2024-02-18T00:38:40.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 2 5 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94059770 osd=2 4 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 10 client.41147.objecter _send_command 5 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9411c4d0 con 0x7efc94028270 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc99830700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6809/1153193637 conn(0x7efc94028270 0x7efc9411bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc94028270 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc99830700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6809/1153193637 conn(0x7efc94028270 0x7efc9411bfe0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9411c4d0 con 0x7efc94028270 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 10 client.41147.objecter _finish_command 5 = system:0 2024-02-18T00:38:40.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94059770 osd=2 4 2024-02-18T00:38:40.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.351+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 2 5 2024-02-18T00:38:40.356 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:40.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:40.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 10 client.41147.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 6 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 6 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94059770 osd=2 3 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 2 6 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94059770 osd=2 4 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 10 client.41147.objecter _send_command 6 2024-02-18T00:38:40.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.367+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc940277d0 con 0x7efc94028270 2024-02-18T00:38:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc940277d0 con 0x7efc94028270 2024-02-18T00:38:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc93779700 10 client.41147.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94059770 osd=2 4 2024-02-18T00:38:40.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 2 6 2024-02-18T00:38:40.371 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 10 client.41147.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 7 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6813/1805700964 conn(0x7efc9411f740 0x7efc94121b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 7 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 3 7 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94027b80 osd=3 4 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 10 client.41147.objecter _send_command 7 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9411d3d0 con 0x7efc9411f740 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9a832700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6813/1805700964 conn(0x7efc9411f740 0x7efc94121b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc9411f740 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.369+0000 7efc9a832700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6813/1805700964 conn(0x7efc9411f740 0x7efc94121b30 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.370+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9411d3d0 con 0x7efc9411f740 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.370+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.370+0000 7efc93779700 10 client.41147.objecter _finish_command 7 = system:0 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.370+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94027b80 osd=3 4 2024-02-18T00:38:40.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.370+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 3 7 2024-02-18T00:38:40.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 10 client.41147.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 8 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 8 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc94027b80 osd=3 3 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 3 8 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94027b80 osd=3 4 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 10 client.41147.objecter _send_command 8 2024-02-18T00:38:40.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.392+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc9411c930 con 0x7efc9411f740 2024-02-18T00:38:40.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.393+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc9411c930 con 0x7efc9411f740 2024-02-18T00:38:40.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.393+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.393+0000 7efc93779700 10 client.41147.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.393+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc94027b80 osd=3 4 2024-02-18T00:38:40.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.393+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 3 8 2024-02-18T00:38:40.395 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:38:40.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 10 client.41147.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 9 2024-02-18T00:38:40.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6800/2044387056 conn(0x7efc9411bb40 0x7efc941251b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 9 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 4 9 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411c930 osd=4 4 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 10 client.41147.objecter _send_command 9 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9411cbf0 con 0x7efc9411bb40 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6800/2044387056 conn(0x7efc9411bb40 0x7efc941251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc9411bb40 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6800/2044387056 conn(0x7efc9411bb40 0x7efc941251b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9411cbf0 con 0x7efc9411bb40 2024-02-18T00:38:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:38:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc93779700 10 client.41147.objecter _finish_command 9 = system:0 2024-02-18T00:38:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9411c930 osd=4 4 2024-02-18T00:38:40.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.395+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 4 9 2024-02-18T00:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 10 client.41147.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 10 2024-02-18T00:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 10 2024-02-18T00:38:40.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9411c930 osd=4 3 2024-02-18T00:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 4 10 2024-02-18T00:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411c930 osd=4 4 2024-02-18T00:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 10 client.41147.objecter _send_command 10 2024-02-18T00:38:40.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.410+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc9411cbf0 con 0x7efc9411bb40 2024-02-18T00:38:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc9411cbf0 con 0x7efc9411bb40 2024-02-18T00:38:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc93779700 10 client.41147.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9411c930 osd=4 4 2024-02-18T00:38:40.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 4 10 2024-02-18T00:38:40.415 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 10 client.41147.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 11 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6804/2170432263 conn(0x7efc94125bb0 0x7efc94128000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 11 2024-02-18T00:38:40.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 5 11 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411cbf0 osd=5 4 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 10 client.41147.objecter _send_command 11 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9412ae00 con 0x7efc94125bb0 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc99830700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6804/2170432263 conn(0x7efc94125bb0 0x7efc94128000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc99830700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6804/2170432263 conn(0x7efc94125bb0 0x7efc94128000 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.413+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc94125bb0 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.414+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9412ae00 con 0x7efc94125bb0 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.414+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.414+0000 7efc93779700 10 client.41147.objecter _finish_command 11 = system:0 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.414+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9411cbf0 osd=5 4 2024-02-18T00:38:40.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.414+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 5 11 2024-02-18T00:38:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 10 client.41147.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 12 2024-02-18T00:38:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 12 2024-02-18T00:38:40.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9411cbf0 osd=5 3 2024-02-18T00:38:40.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 5 12 2024-02-18T00:38:40.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411cbf0 osd=5 4 2024-02-18T00:38:40.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 10 client.41147.objecter _send_command 12 2024-02-18T00:38:40.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.428+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc9412ae00 con 0x7efc94125bb0 2024-02-18T00:38:40.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc9412ae00 con 0x7efc94125bb0 2024-02-18T00:38:40.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc93779700 10 client.41147.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9411cbf0 osd=5 4 2024-02-18T00:38:40.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 5 12 2024-02-18T00:38:40.433 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:38:40.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 10 client.41147.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 13 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6808/3356009390 conn(0x7efc9412a100 0x7efc9412a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 13 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 6 13 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ae00 osd=6 4 2024-02-18T00:38:40.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 10 client.41147.objecter _send_command 13 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9412ac20 con 0x7efc9412a100 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9a832700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6808/3356009390 conn(0x7efc9412a100 0x7efc9412a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc9a832700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6808/3356009390 conn(0x7efc9412a100 0x7efc9412a580 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.431+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc9412a100 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.433+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9412ac20 con 0x7efc9412a100 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.433+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.433+0000 7efc93779700 10 client.41147.objecter _finish_command 13 = system:0 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.433+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9412ae00 osd=6 4 2024-02-18T00:38:40.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.433+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 6 13 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 10 client.41147.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 14 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 14 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9412ae00 osd=6 3 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 6 14 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ae00 osd=6 4 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 10 client.41147.objecter _send_command 14 2024-02-18T00:38:40.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.446+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc9412ac20 con 0x7efc9412a100 2024-02-18T00:38:40.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc9412ac20 con 0x7efc9412a100 2024-02-18T00:38:40.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 10 client.41147.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9412ae00 osd=6 4 2024-02-18T00:38:40.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 6 14 2024-02-18T00:38:40.451 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:38:40.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 10 client.41147.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:40.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 15 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6812/53025171 conn(0x7efc9412d540 0x7efc9412d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 15 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 7 15 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ac20 osd=7 4 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 10 client.41147.objecter _send_command 15 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc9412e010 con 0x7efc9412d540 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6812/53025171 conn(0x7efc9412d540 0x7efc9412d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:40.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 10 client.41147.objecter ms_handle_connect 0x7efc9412d540 2024-02-18T00:38:40.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc9a031700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6812/53025171 conn(0x7efc9412d540 0x7efc9412d9a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:40.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efc9412e010 con 0x7efc9412d540 2024-02-18T00:38:40.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:38:40.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 10 client.41147.objecter _finish_command 15 = system:0 2024-02-18T00:38:40.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9412ac20 osd=7 4 2024-02-18T00:38:40.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.449+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 7 15 2024-02-18T00:38:40.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 10 client.41147.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign -1 16 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 20 client.41147.objecter _get_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 15 client.41147.objecter _session_command_op_remove -1 16 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 20 client.41147.objecter get_session s=0x7efc9412ac20 osd=7 3 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 15 client.41147.objecter _session_command_op_assign 7 16 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ac20 osd=7 4 2024-02-18T00:38:40.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 10 client.41147.objecter _send_command 16 2024-02-18T00:38:40.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.464+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7efc9412e010 con 0x7efc9412d540 2024-02-18T00:38:40.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc93779700 1 -- 172.21.15.59:0/4035561099 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7efc9412e010 con 0x7efc9412d540 2024-02-18T00:38:40.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc93779700 10 client.41147.objecter ms_dispatch 0x7efc94001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:38:40.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc93779700 10 client.41147.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc93779700 20 client.41147.objecter put_session s=0x7efc9412ac20 osd=7 4 2024-02-18T00:38:40.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc93779700 15 client.41147.objecter _session_command_op_remove 7 16 2024-02-18T00:38:40.468 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:38:40.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 librados: watch_flush enter 2024-02-18T00:38:40.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 librados: watch_flush exit 2024-02-18T00:38:40.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.0 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6801/3700238977 conn(0x7efc940f9d00 msgr2=0x7efc940fa0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6801/3700238977 conn(0x7efc940f9d00 0x7efc940fa0e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94107110 osd=0 1 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.1 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6805/26279110 conn(0x7efc940598f0 msgr2=0x7efc94059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6805/26279110 conn(0x7efc940598f0 0x7efc94059cd0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94028710 osd=1 1 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.2 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6809/1153193637 conn(0x7efc94028270 msgr2=0x7efc9411bfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6809/1153193637 conn(0x7efc94028270 0x7efc9411bfe0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94059770 osd=2 1 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.3 2024-02-18T00:38:40.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6813/1805700964 conn(0x7efc9411f740 msgr2=0x7efc94121b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6813/1805700964 conn(0x7efc9411f740 0x7efc94121b30 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc94027b80 osd=3 1 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.4 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6800/2044387056 conn(0x7efc9411bb40 msgr2=0x7efc941251b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6800/2044387056 conn(0x7efc9411bb40 0x7efc941251b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411c930 osd=4 1 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.5 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6804/2170432263 conn(0x7efc94125bb0 msgr2=0x7efc94128000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6804/2170432263 conn(0x7efc94125bb0 0x7efc94128000 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9411cbf0 osd=5 1 2024-02-18T00:38:40.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.6 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6808/3356009390 conn(0x7efc9412a100 msgr2=0x7efc9412a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9a832700 1 -- 172.21.15.59:0/4035561099 reap_dead start 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6808/3356009390 conn(0x7efc9412a100 0x7efc9412a580 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9a832700 1 -- 172.21.15.59:0/4035561099 reap_dead start 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ae00 osd=6 1 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter close_session for osd.7 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6812/53025171 conn(0x7efc9412d540 msgr2=0x7efc9412d9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9a832700 1 -- 172.21.15.59:0/4035561099 reap_dead start 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6812/53025171 conn(0x7efc9412d540 0x7efc9412d9a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter put_session s=0x7efc9412ac20 osd=7 1 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 20 client.41147.objecter shutdown clearing up homeless session... 2024-02-18T00:38:40.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 10 client.41147.objecter successfully canceled tick 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc7c062c90 msgr2=0x7efc7c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc7c062c90 0x7efc7c065150 secure :-1 s=READY pgs=3111 cs=0 l=1 rev1=1 rx=0x7efc8c007fb0 tx=0x7efc8c009040).stop 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 msgr2=0x7efc9410a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 0x7efc9410a1b0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 rx=0x7efc840044e0 tx=0x7efc84007f90).stop 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 shutdown_connections 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.137:6812/53025171 conn(0x7efc9412d540 0x7efc9412d9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efc940fe510 0x7efc9410a1b0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 --2- 172.21.15.59:0/4035561099 >> v2:172.21.15.59:6800/3721891797 conn(0x7efc7c062c90 0x7efc7c065150 unknown :-1 s=CLOSED pgs=3111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 >> 172.21.15.59:0/4035561099 conn(0x7efc940f3800 msgr2=0x7efc940f5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 shutdown_connections 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 -- 172.21.15.59:0/4035561099 wait complete. 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.467+0000 7efc9c295700 1 librados: shutdown 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.473 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.474 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:38:40.488 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:40.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.907+0000 7f1a96ffd700 1 -- 172.21.15.59:0/1907854029 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v430) v1 ==== 113+0+0 (secure 0 0 0) 0x7f1a9c019070 con 0x7f1aa00f8cb0 2024-02-18T00:38:40.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.909+0000 7f1aa819b700 10 librados: watch_flush enter 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.909+0000 7f1aa819b700 10 librados: watch_flush exit 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.909+0000 7f1aa819b700 20 client.41141.objecter shutdown clearing up homeless session... 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.909+0000 7f1aa819b700 10 client.41141.objecter successfully canceled tick 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1a8c062c90 msgr2=0x7f1a8c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 --2- 172.21.15.59:0/1907854029 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1a8c062c90 0x7f1a8c065150 secure :-1 s=READY pgs=3110 cs=0 l=1 rev1=1 rx=0x7f1a90009bc0 tx=0x7f1a9000b040).stop 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aa00f8cb0 msgr2=0x7f1aa010e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aa00f8cb0 0x7f1aa010e250 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 rx=0x7f1a9c00e8f0 tx=0x7f1a9c00b040).stop 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 shutdown_connections 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 --2- 172.21.15.59:0/1907854029 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1a8c062c90 0x7f1a8c065150 unknown :-1 s=CLOSED pgs=3110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1aa0102210 0x7f1aa0108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aa00f8cb0 0x7f1aa010e250 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 --2- 172.21.15.59:0/1907854029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aa00f9610 0x7f1aa010e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:40.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 >> 172.21.15.59:0/1907854029 conn(0x7f1aa00f2820 msgr2=0x7f1aa00f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:40.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 shutdown_connections 2024-02-18T00:38:40.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 -- 172.21.15.59:0/1907854029 wait complete. 2024-02-18T00:38:40.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:40.910+0000 7f1aa819b700 1 librados: shutdown 2024-02-18T00:38:40.912 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.3 to 1 (10000) 2024-02-18T00:38:40.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:39.902328+0000 mon.a (mon.0) 2538 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:40.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:39.904207+0000 mgr.y (mgr.24454) 400 : cluster [DBG] pgmap v637: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:40.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:40.188843+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.59:0/1907854029' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:38:40.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:40.189135+0000 mon.a (mon.0) 2539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:38:40.926 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:40.313005+0000 mon.c (mon.2) 417 : audit [DBG] from='client.? 172.21.15.59:0/4035561099' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:40.932 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:39.902328+0000 mon.a (mon.0) 2538 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:40.932 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:39.904207+0000 mgr.y (mgr.24454) 400 : cluster [DBG] pgmap v637: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:40.932 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:40.188843+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.59:0/1907854029' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:38:40.932 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:40.189135+0000 mon.a (mon.0) 2539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:38:40.932 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:40.313005+0000 mon.c (mon.2) 417 : audit [DBG] from='client.? 172.21.15.59:0/4035561099' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:40.933 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:39.902328+0000 mon.a (mon.0) 2538 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:40.933 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:39.904207+0000 mgr.y (mgr.24454) 400 : cluster [DBG] pgmap v637: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:38:40.933 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:40.188843+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.59:0/1907854029' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:38:40.933 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:40.189135+0000 mon.a (mon.0) 2539 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:38:40.933 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:40.313005+0000 mon.c (mon.2) 417 : audit [DBG] from='client.? 172.21.15.59:0/4035561099' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:41.152 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:38:41.152 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:38:41.153 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:38:41.153 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:38:41.498 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:38:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:38:41.498 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:38:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:38:41.890 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-18T00:38:42.159 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:42.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:40.905049+0000 mon.a (mon.0) 2540 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:38:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:40.905103+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e430: 8 total, 8 up, 7 in 2024-02-18T00:38:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:41.255417+0000 mgr.y (mgr.24454) 401 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:40.905049+0000 mon.a (mon.0) 2540 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:38:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:40.905103+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e430: 8 total, 8 up, 7 in 2024-02-18T00:38:42.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:41.255417+0000 mgr.y (mgr.24454) 401 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:40.905049+0000 mon.a (mon.0) 2540 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:38:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:40.905103+0000 mon.a (mon.0) 2541 : cluster [DBG] osdmap e430: 8 total, 8 up, 7 in 2024-02-18T00:38:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:41.255417+0000 mgr.y (mgr.24454) 401 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:42.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:38:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:42] "GET /metrics HTTP/1.1" 200 189635 "" "Prometheus/2.33.4" 2024-02-18T00:38:42.720 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 4 1 2024-02-18T00:38:42.739 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:42.742 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:42.743 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:42.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:42.872 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:42.892 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:42.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:42.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:42.996 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:43.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:41.904582+0000 mgr.y (mgr.24454) 402 : cluster [DBG] pgmap v639: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:38:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:41.904582+0000 mgr.y (mgr.24454) 402 : cluster [DBG] pgmap v639: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:38:43.023 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:41.904582+0000 mgr.y (mgr.24454) 402 : cluster [DBG] pgmap v639: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:38:43.672 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:43.675 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:43.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:43.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:43.807 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:43.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 -- 172.21.15.59:0/810048792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000f7400 msgr2=0x7f9000103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 --2- 172.21.15.59:0/810048792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000f7400 0x7f9000103a20 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 rx=0x7f8ff8009be0 tx=0x7f8ff8004d10).stop 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 -- 172.21.15.59:0/810048792 shutdown_connections 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 --2- 172.21.15.59:0/810048792 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000f7400 0x7f9000103a20 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 --2- 172.21.15.59:0/810048792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90000fcbb0 0x7f90000fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 --2- 172.21.15.59:0/810048792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90000fd510 0x7f90000f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 -- 172.21.15.59:0/810048792 >> 172.21.15.59:0/810048792 conn(0x7f90000f2820 msgr2=0x7f90000f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 -- 172.21.15.59:0/810048792 shutdown_connections 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 -- 172.21.15.59:0/810048792 wait complete. 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 librados: starting msgr at 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 librados: starting objecter 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 Processor -- start 2024-02-18T00:38:43.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 -- start start 2024-02-18T00:38:43.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 librados: setting wanted keys 2024-02-18T00:38:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.790+0000 7f9005a32700 1 librados: calling monclient init 2024-02-18T00:38:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f9005a32700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90000f7400 0x7f900010ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f9005a32700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000fcbb0 0x7f900010d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f9005a32700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90000fd510 0x7f9000107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f9005a32700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000107510 con 0x7f90000f7400 2024-02-18T00:38:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f9005a32700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000107680 con 0x7f90000fd510 2024-02-18T00:38:43.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f9005a32700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90001077c0 con 0x7f90000fcbb0 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffe7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000fcbb0 0x7f900010d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffeffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90000f7400 0x7f900010ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffe7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000fcbb0 0x7f900010d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:53244/0 (socket says 172.21.15.59:53244) 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffe7fc700 1 -- 172.21.15.59:0/1540287238 learned_addr learned my addr 172.21.15.59:0/1540287238 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8fff7fe700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90000fd510 0x7f9000107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffeffd700 1 -- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000fcbb0 msgr2=0x7f900010d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffeffd700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000fcbb0 0x7f900010d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffeffd700 1 -- 172.21.15.59:0/1540287238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90000fd510 msgr2=0x7f9000107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffeffd700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90000fd510 0x7f9000107020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffeffd700 1 -- 172.21.15.59:0/1540287238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff80097c0 con 0x7f90000f7400 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffe7fc700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000fcbb0 0x7f900010d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8ffeffd700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90000f7400 0x7f900010ce80 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7f8ff000e8f0 tx=0x7f8ff000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:43.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8fe7fff700 10 client.?.objecter ms_handle_connect 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8fe7fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.791+0000 7f8fe7fff700 1 -- 172.21.15.59:0/1540287238 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0004620 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.792+0000 7f8fe7fff700 1 -- 172.21.15.59:0/1540287238 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ff000f040 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.792+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9000107aa0 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.792+0000 7f8fe7fff700 1 -- 172.21.15.59:0/1540287238 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0014830 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.792+0000 7f9005a32700 10 client.31350.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.792+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9000060db0 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.792+0000 7f9005a32700 1 librados: init done 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.793+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900005fa60 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.793+0000 7f8fe7fff700 1 -- 172.21.15.59:0/1540287238 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8ff00090d0 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8fe7fff700 1 --2- 172.21.15.59:0/1540287238 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8fe8062c90 0x7f8fe8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8fe7fff700 1 -- 172.21.15.59:0/1540287238 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(431..431 src has 180..431) v4 ==== 8091+0+0 (secure 0 0 0) 0x7f8ff007cb60 con 0x7f90000f7400 2024-02-18T00:38:43.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8fe7fff700 10 client.31350.objecter ms_dispatch 0x7f9000001550 osd_map(431..431 src has 180..431) v4 2024-02-18T00:38:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8fe7fff700 3 client.31350.objecter handle_osd_map got epochs [431,431] > 0 2024-02-18T00:38:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8fe7fff700 3 client.31350.objecter handle_osd_map decoding full epoch 431 2024-02-18T00:38:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8ffe7fc700 1 --2- 172.21.15.59:0/1540287238 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8fe8062c90 0x7f8fe8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8fe7fff700 20 client.31350.objecter dump_active .. 0 homeless 2024-02-18T00:38:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8fe7fff700 10 client.31350.objecter ms_handle_connect 0x7f8fe8062c90 2024-02-18T00:38:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.794+0000 7f8ffe7fc700 1 --2- 172.21.15.59:0/1540287238 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8fe8062c90 0x7f8fe8065150 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 rx=0x7f8ff4004fd0 tx=0x7f8ff400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:43.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:43.798+0000 7f8fe7fff700 1 -- 172.21.15.59:0/1540287238 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8ff004af60 con 0x7f90000f7400 2024-02-18T00:38:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:43.833 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:43.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:43.865 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:43.931 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:42.917334+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e431: 8 total, 8 up, 7 in 2024-02-18T00:38:43.934 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:43.957 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:38:43.957 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:38:43.957 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:38:43.957 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:38:43.957 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:38:43.967 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:43.985 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:43.987 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:43.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:43.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:38:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.005+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f9000108500 con 0x7f90000f7400 2024-02-18T00:38:44.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:44.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:42.917334+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e431: 8 total, 8 up, 7 in 2024-02-18T00:38:44.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:42.917334+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e431: 8 total, 8 up, 7 in 2024-02-18T00:38:44.116 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:44.140 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:44.144 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:44.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:44.245 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:44.273 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:44.295 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:44.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:44.396 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:44.425 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:44.524 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:44.526 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:44.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:44.528 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-18T00:38:44.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:44.660 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:44.784 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:44.787 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:44.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:44.807 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:38:44.889 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:44.915 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:44.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.939+0000 7f8fe7fff700 1 -- 172.21.15.59:0/1540287238 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v432) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8ff0019070 con 0x7f90000f7400 2024-02-18T00:38:44.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 10 librados: watch_flush enter 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 10 librados: watch_flush exit 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 20 client.31350.objecter shutdown clearing up homeless session... 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 10 client.31350.objecter successfully canceled tick 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8fe8062c90 msgr2=0x7f8fe8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 --2- 172.21.15.59:0/1540287238 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8fe8062c90 0x7f8fe8065150 secure :-1 s=READY pgs=3112 cs=0 l=1 rev1=1 rx=0x7f8ff4004fd0 tx=0x7f8ff400b040).stop 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90000f7400 msgr2=0x7f900010ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90000f7400 0x7f900010ce80 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7f8ff000e8f0 tx=0x7f8ff000b040).stop 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 shutdown_connections 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 --2- 172.21.15.59:0/1540287238 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8fe8062c90 0x7f8fe8065150 unknown :-1 s=CLOSED pgs=3112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90000fd510 0x7f9000107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90000f7400 0x7f900010ce80 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:44.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 --2- 172.21.15.59:0/1540287238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90000fcbb0 0x7f900010d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 >> 172.21.15.59:0/1540287238 conn(0x7f90000f2820 msgr2=0x7f90000f4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 shutdown_connections 2024-02-18T00:38:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 -- 172.21.15.59:0/1540287238 wait complete. 2024-02-18T00:38:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:44.942+0000 7f9005a32700 1 librados: shutdown 2024-02-18T00:38:44.944 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:38:45.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:43.905033+0000 mgr.y (mgr.24454) 403 : cluster [DBG] pgmap v641: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:38:45.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:44.006034+0000 mon.a (mon.0) 2543 : audit [INF] from='client.? 172.21.15.59:0/1540287238' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:45.064 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:43.905033+0000 mgr.y (mgr.24454) 403 : cluster [DBG] pgmap v641: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:38:45.064 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:44.006034+0000 mon.a (mon.0) 2543 : audit [INF] from='client.? 172.21.15.59:0/1540287238' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:45.113 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:45.117 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:45.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:45.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:45.248 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:43.905033+0000 mgr.y (mgr.24454) 403 : cluster [DBG] pgmap v641: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:38:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:44.006034+0000 mon.a (mon.0) 2543 : audit [INF] from='client.? 172.21.15.59:0/1540287238' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:38:46.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:44.930571+0000 mon.a (mon.0) 2544 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:46.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:44.938114+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.59:0/1540287238' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:46.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:44.938200+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e432: 8 total, 8 up, 7 in 2024-02-18T00:38:46.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:44.930571+0000 mon.a (mon.0) 2544 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:46.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:44.938114+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.59:0/1540287238' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:46.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:44.938200+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e432: 8 total, 8 up, 7 in 2024-02-18T00:38:46.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:44.930571+0000 mon.a (mon.0) 2544 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:46.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:44.938114+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.59:0/1540287238' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:38:46.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:44.938200+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e432: 8 total, 8 up, 7 in 2024-02-18T00:38:46.515 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:38:46.515 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24865.0:2 29.1a 29:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.365742+0000", 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 345.14671630700002, 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1529440639999999, 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24865", 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2678422317", 2024-02-18T00:38:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365742+0000", 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365742+0000", 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.517 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365740+0000", 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9699999999999994e-06 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365750+0000", 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4426e-05 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365774+0000", 2024-02-18T00:38:46.518 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.149e-06 2024-02-18T00:38:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.365783+0000", 2024-02-18T00:38:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0027213300000000001 2024-02-18T00:38:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.519 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.368505+0000", 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4719999999999995e-06 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.368513+0000", 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1456162480000001 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.514130+0000", 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012966900000000001 2024-02-18T00:38:46.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.514259+0000", 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014994800000000001 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.514409+0000", 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016852989999999999 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.516094+0000", 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002244242 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518339+0000", 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00025869000000000001 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518597+0000", 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8600999999999998e-05 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.522 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518646+0000", 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0485000000000003e-05 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:02.518686+0000", 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1529440639999999 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24805.0:2 13.18 13:1e6cc557:test-rados-api-smithi059-82957-2::foo:head [undirty] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.401592+0000", 2024-02-18T00:38:46.523 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 345.11086655499997, 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.17911491299999999, 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24805", 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1794179512", 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401592+0000", 2024-02-18T00:38:46.524 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401592+0000", 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401590+0000", 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0469999999999999e-06 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401597+0000", 2024-02-18T00:38:46.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9949999999999994e-06 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401606+0000", 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2190000000000001e-06 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.401612+0000", 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001212917 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.526 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.402825+0000", 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8909999999999995e-06 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.402833+0000", 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.060490740000000001 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.463323+0000", 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012003899999999999 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.463443+0000", 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11726376199999999 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.580707+0000", 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.17911491299999999 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24829.0:2 15.f e106/104 15:f318d819:test-rados-api-smithi059-82989-2::foo:head v 106'1, mlcod=0'0)", 2024-02-18T00:38:46.528 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 345.07871066500002, 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011160974000000001, 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:38:46.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433748+0000", 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4830000000000005e-06 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433757+0000", 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1699999999999997e-06 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433765+0000", 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.888e-06 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433772+0000", 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016121700000000001 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433933+0000", 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6643e-05 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.531 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.433950+0000", 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010933965 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.444884+0000", 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5137e-05 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.444909+0000", 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011160974000000001 2024-02-18T00:38:46.532 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24805.0:3 13.18 13:1e6cc557:test-rados-api-smithi059-82957-2::foo:head [create] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.582528+0000", 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 344.92993117100002, 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011408494, 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24805", 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1794179512", 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 3 2024-02-18T00:38:46.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582528+0000", 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582528+0000", 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582526+0000", 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5890000000000002e-06 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582532+0000", 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0080000000000008e-06 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582541+0000", 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4349999999999999e-06 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582545+0000", 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6727e-05 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582572+0000", 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9898999999999998e-05 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.536 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582622+0000", 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8269000000000005e-05 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,4", 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.582710+0000", 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030607300000000002 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.583016+0000", 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000163521 2024-02-18T00:38:46.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.583180+0000", 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010674022999999999 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.593854+0000", 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7469999999999998e-05 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.593901+0000", 2024-02-18T00:38:46.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4650000000000002e-05 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.593936+0000", 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011408494 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24766.0:23 12.d 12:b6d4da3a:test-rados-api-smithi059-82901-8::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:01.619767+0000", 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 344.89269158500002, 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0068538059999999996, 2024-02-18T00:38:46.539 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24766", 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1346273899", 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 23 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619767+0000", 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.540 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619767+0000", 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619766+0000", 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5389999999999996e-06 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619776+0000", 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4699999999999999e-06 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619782+0000", 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.709e-06 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619786+0000", 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5612999999999999e-05 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619822+0000", 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8061999999999997e-05 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.619920+0000", 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013012500000000001 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.620050+0000", 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00036723700000000002 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.620417+0000", 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00046972000000000002 2024-02-18T00:38:46.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.620887+0000", 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0056526720000000001 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.626539+0000", 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4203999999999999e-05 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.626584+0000", 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7239e-05 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:01.626621+0000", 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0068538059999999996 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25003.0:1 22.1a e107/105 22:5c6776db:::0.obj:head v 107'5, mlcod=107'3)", 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.436676+0000", 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 343.07578300500001, 2024-02-18T00:38:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012922347000000001, 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436676+0000", 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436676+0000", 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436675+0000", 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0831e-05 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436685+0000", 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0688e-05 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436696+0000", 2024-02-18T00:38:46.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6769999999999995e-06 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.436704+0000", 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0079016120000000006 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.444605+0000", 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4494999999999998e-05 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.444630+0000", 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049239829999999998 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.449554+0000", 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4261000000000002e-05 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.449598+0000", 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012922347000000001 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:3 22.d 22:b27a9ef8:::17.obj2:head [write 0~255] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.054788+0000", 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.45767074999998, 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0077752630000000001, 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 3 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.550 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054788+0000", 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054788+0000", 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054787+0000", 2024-02-18T00:38:46.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8359999999999992e-06 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054794+0000", 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3886999999999999e-05 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054808+0000", 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3310000000000004e-06 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054817+0000", 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7200999999999995e-05 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.054894+0000", 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020316 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.055097+0000", 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026296499999999999 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.553 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 4,6", 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.055360+0000", 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0030637559999999999 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.058424+0000", 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00054202500000000002 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.058966+0000", 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003509968 2024-02-18T00:38:46.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062476+0000", 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0881999999999999e-05 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062517+0000", 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6555999999999999e-05 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.062563+0000", 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0077752630000000001 2024-02-18T00:38:46.555 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:6 22.2 e108/105 22:47a9abb2:::36.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.068716+0000", 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.443742966, 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034977148999999999, 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068716+0000", 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068716+0000", 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068714+0000", 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.577e-05 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068730+0000", 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3647e-05 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068744+0000", 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8710000000000002e-06 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068750+0000", 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5821e-05 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068805+0000", 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1854000000000001e-05 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.068827+0000", 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034833981999999999 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103661+0000", 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1584000000000001e-05 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.103693+0000", 2024-02-18T00:38:46.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034977148999999999 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2070 17.15 e108/104 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head v 108'1, mlcod=0'0)", 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.136909+0000", 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.375549697, 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.066111226999999995, 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136909+0000", 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136909+0000", 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136907+0000", 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017153972999999999 2024-02-18T00:38:46.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154061+0000", 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6050999999999999e-05 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154077+0000", 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4019999999999996e-06 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154084+0000", 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2176999999999998e-05 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154147+0000", 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.448e-05 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.154161+0000", 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.048825701999999999 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.202987+0000", 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3439000000000003e-05 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.203020+0000", 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.066111226999999995 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.565 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:20 22.1 e108/105 22:83282cb0:::1.obj2:head v 108'14, mlcod=108'12)", 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.158484+0000", 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.35397479199997, 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042476860999999998, 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158484+0000", 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.566 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158484+0000", 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158483+0000", 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3191e-05 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158496+0000", 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6535999999999999e-05 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158512+0000", 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8220000000000002e-06 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158522+0000", 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0906999999999999e-05 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158573+0000", 2024-02-18T00:38:46.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0628e-05 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.158594+0000", 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042318445000000003 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.200912+0000", 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8708999999999997e-05 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.569 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.200961+0000", 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042476860999999998 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.24823.0:2073 17.15 e108/104 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head v 108'4, mlcod=108'2)", 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.29558227299998, 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0089590280000000008, 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.570 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.216876+0000", 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00083967099999999995 2024-02-18T00:38:46.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217715+0000", 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9789e-05 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217735+0000", 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0609999999999998e-06 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217742+0000", 2024-02-18T00:38:46.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2679000000000002e-05 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217795+0000", 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2875e-05 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.217818+0000", 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0079678040000000002 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.573 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225786+0000", 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9818999999999999e-05 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225835+0000", 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0089590280000000008 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:32 22.8 22:12e893d3:::7.obj2:head [write 0~135] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:38:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.219993+0000", 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.29246554700001, 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.006045323, 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 32 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.219993+0000", 2024-02-18T00:38:46.575 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.219993+0000", 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.219991+0000", 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0264e-05 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.576 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220001+0000", 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6555999999999999e-05 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220018+0000", 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0730000000000001e-05 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220028+0000", 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9993000000000001e-05 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220068+0000", 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010894 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220177+0000", 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000111055 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,4", 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.220288+0000", 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074838599999999997 2024-02-18T00:38:46.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.221037+0000", 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000118282 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.221155+0000", 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047722520000000003 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225927+0000", 2024-02-18T00:38:46.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8940000000000002e-05 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.225986+0000", 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2339000000000002e-05 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.226038+0000", 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.006045323 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.580 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:40 22.c 22:33eb54d8:::39.obj2:head [write 0~150] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.268821+0000", 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.24363773499999, 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.033196955, 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 40 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.581 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268821+0000", 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268821+0000", 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268820+0000", 2024-02-18T00:38:46.582 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.49e-06 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268825+0000", 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0638999999999999e-05 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268836+0000", 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1050000000000003e-06 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.583 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268841+0000", 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5590999999999999e-05 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268877+0000", 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1233000000000001e-05 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.268948+0000", 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4674999999999997e-05 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,3", 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.269043+0000", 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029585499999999998 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.269338+0000", 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.030906822 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.300245+0000", 2024-02-18T00:38:46.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001703366 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.301949+0000", 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7935000000000002e-05 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.301987+0000", 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1309999999999997e-05 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302018+0000", 2024-02-18T00:38:46.586 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.033196955 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.25025.0:41 22.e e108/105 22:7639cdab:::46.obj2:head v 108'9, mlcod=108'7)", 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.302685+0000", 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 342.20977373699998, 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018774604, 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.587 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302685+0000", 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302685+0000", 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302684+0000", 2024-02-18T00:38:46.588 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0800000000000006e-06 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302692+0000", 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4999999999999999e-06 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302701+0000", 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8539999999999999e-06 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.589 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302705+0000", 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1548000000000003e-05 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302746+0000", 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3828e-05 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302770+0000", 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018660823 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.590 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321431+0000", 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8781000000000001e-05 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321460+0000", 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018774604 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.591 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25052.0:14 22.a 22:54367c00:::22.obj:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e109)", 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.891467+0000", 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 341.62099183100003, 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0070211850000000001, 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25052", 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1321287304", 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 14 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.592 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891467+0000", 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891467+0000", 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891466+0000", 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3839999999999999e-06 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.593 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891471+0000", 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2650000000000002e-06 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891480+0000", 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4100000000000001e-06 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891484+0000", 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9938999999999999e-05 2024-02-18T00:38:46.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891514+0000", 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0243e-05 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891564+0000", 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7618000000000002e-05 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891632+0000", 2024-02-18T00:38:46.595 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00022058700000000001 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.891853+0000", 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00024304200000000001 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.892096+0000", 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0063404799999999999 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.596 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.898436+0000", 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7296000000000001e-05 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.898463+0000", 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4664999999999998e-05 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.898488+0000", 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0070211850000000001 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.597 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25073.0:3 37.f 37:f4b50b61:::1.obj:head [write 0~198] snapc 0=[] ondisk+write+known_if_redirected e112)", 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:07.512122+0000", 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 339.00033677599998, 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020910159000000001, 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25073", 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1549676353", 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 3 2024-02-18T00:38:46.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512122+0000", 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512122+0000", 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512121+0000", 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3009999999999997e-06 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512126+0000", 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6290000000000001e-06 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512134+0000", 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0050000000000001e-06 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.600 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512140+0000", 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5095000000000002e-05 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512175+0000", 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000106903 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512282+0000", 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015298800000000001 2024-02-18T00:38:46.601 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512435+0000", 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00044063199999999999 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.512875+0000", 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00018640599999999999 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.513062+0000", 2024-02-18T00:38:46.602 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019895076000000001 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.532957+0000", 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7423999999999999e-05 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.532994+0000", 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7999e-05 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.603 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:07.533032+0000", 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020910159000000001 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.15264.0:19 60.10 60:0b42fc87:test-rados-api-smithi059-82973-8::foo:head [delete] snapc 7=[7,6] ondisk+write+peerstat_old+known_if_redirected e130)", 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.023202+0000", 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 318.48925700799998, 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.043374965000000001, 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.604 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.15264", 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/310625823", 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 19 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023202+0000", 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.605 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023202+0000", 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023200+0000", 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6118999999999999e-05 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023216+0000", 2024-02-18T00:38:46.606 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.452e-05 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023230+0000", 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9570000000000008e-06 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023239+0000", 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5216999999999997e-05 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.607 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023294+0000", 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2409999999999995e-06 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.023303+0000", 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026498173 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.049801+0000", 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8725000000000002e-05 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.608 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.049890+0000", 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000145439 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.050035+0000", 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001137379 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.051172+0000", 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029129900000000002 2024-02-18T00:38:46.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.051464+0000", 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015008858999999999 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.066473+0000", 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8291999999999997e-05 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.066531+0000", 2024-02-18T00:38:46.610 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5847999999999999e-05 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.066577+0000", 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.043374965000000001 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30748.0:1 78.4s0 78:3cba11e7:test-rados-api-smithi059-82973-10::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e134)", 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:32.611721+0000", 2024-02-18T00:38:46.611 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 313.90073809099999, 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6967851380000001, 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30748", 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/510272679", 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611721+0000", 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611721+0000", 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611719+0000", 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.153e-05 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611731+0000", 2024-02-18T00:38:46.613 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1849999999999998e-06 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611738+0000", 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6439999999999997e-06 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611744+0000", 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4613000000000002e-05 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611778+0000", 2024-02-18T00:38:46.614 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.396e-06 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:32.611781+0000", 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6366086819999999 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.248389+0000", 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012518499999999999 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.615 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.248515+0000", 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030601100000000002 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.248821+0000", 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047360799999999998 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.249294+0000", 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.059161694000000001 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.616 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.308456+0000", 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9768999999999997e-05 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.308506+0000", 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6967851380000001 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.617 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2079 17.c 17:36e2cd47:test-rados-api-smithi059-82949-17::big2.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e136)", 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.511355+0000", 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 312.001104064, 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.14033522800000001, 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2079 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511355+0000", 2024-02-18T00:38:46.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511355+0000", 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511353+0000", 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3270000000000003e-06 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.619 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511360+0000", 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2292e-05 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511373+0000", 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7450000000000001e-06 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511378+0000", 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0411999999999999e-05 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511419+0000", 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010539099999999999 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.511524+0000", 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.069820673999999999 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,5", 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.581345+0000", 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0099325850000000007 2024-02-18T00:38:46.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.591278+0000", 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056372900000000001 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.591841+0000", 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.059783511999999997 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.651625+0000", 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7512999999999998e-05 2024-02-18T00:38:46.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.651662+0000", 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7617999999999999e-05 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.651690+0000", 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.14033522800000001 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.623 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.33712.0:7 82.c 82:3197012c:::foo_rmxattr:head [setxattr attr2 (12) in=5b] snapc 0=[] ondisk+write+known_if_redirected e138)", 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:36.773678+0000", 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 309.73878060099997, 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058598080000000002, 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.33712", 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3967728574", 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 7 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.624 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773678+0000", 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773678+0000", 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773677+0000", 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9460000000000001e-06 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773684+0000", 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0764000000000001e-05 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773695+0000", 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9930000000000003e-06 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773700+0000", 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1032000000000002e-05 2024-02-18T00:38:46.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773751+0000", 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4580000000000007e-05 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773835+0000", 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014667200000000001 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,4", 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.773982+0000", 2024-02-18T00:38:46.627 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00032207600000000002 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.774304+0000", 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017701699999999999 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.774481+0000", 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049669709999999997 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.628 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.779448+0000", 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9666000000000002e-05 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.779498+0000", 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0355999999999998e-05 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.779538+0000", 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058598080000000002 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.629 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.630 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.630 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:38:46.630 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:38:46.630 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:38:46.631 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:46.631 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:46.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:46.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:46.766 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:46.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 -- 172.21.15.59:0/2709224562 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580fdbb0 msgr2=0x7fe5580fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:46.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 --2- 172.21.15.59:0/2709224562 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580fdbb0 0x7fe5580fdf90 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 rx=0x7fe54c0047b0 tx=0x7fe54c009cb0).stop 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 -- 172.21.15.59:0/2709224562 shutdown_connections 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 --2- 172.21.15.59:0/2709224562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5580f8420 0x7fe558104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 --2- 172.21.15.59:0/2709224562 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580fdbb0 0x7fe5580fdf90 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 --2- 172.21.15.59:0/2709224562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5580fe510 0x7fe5580f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 -- 172.21.15.59:0/2709224562 >> 172.21.15.59:0/2709224562 conn(0x7fe5580f3800 msgr2=0x7fe5580f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 -- 172.21.15.59:0/2709224562 shutdown_connections 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.828+0000 7fe55cf31700 1 -- 172.21.15.59:0/2709224562 wait complete. 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 librados: starting msgr at 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 librados: starting objecter 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 Processor -- start 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 -- start start 2024-02-18T00:38:46.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 librados: setting wanted keys 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 librados: calling monclient init 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580f8420 0x7fe55806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5580fdbb0 0x7fe55806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5580fe510 0x7fe55806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe558116580 con 0x7fe5580fe510 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5581166f0 con 0x7fe5580fdbb0 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55cf31700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5581168b0 con 0x7fe5580f8420 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580f8420 0x7fe55806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580f8420 0x7fe55806b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:53266/0 (socket says 172.21.15.59:53266) 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe556d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5580fe510 0x7fe55806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.829+0000 7fe55659c700 1 -- 172.21.15.59:0/1625750842 learned_addr learned my addr 172.21.15.59:0/1625750842 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:46.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe555d9b700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5580fdbb0 0x7fe55806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55659c700 1 -- 172.21.15.59:0/1625750842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5580fdbb0 msgr2=0x7fe55806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55659c700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5580fdbb0 0x7fe55806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55659c700 1 -- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5580fe510 msgr2=0x7fe55806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55659c700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5580fe510 0x7fe55806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55659c700 1 -- 172.21.15.59:0/1625750842 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe548005040 con 0x7fe5580f8420 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe556d9d700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5580fe510 0x7fe55806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55659c700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580f8420 0x7fe55806b0d0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 rx=0x7fe54c0044e0 tx=0x7fe54c007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe5477fe700 10 client.?.objecter ms_handle_connect 0x7fe5580f8420 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe5477fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe5477fe700 1 -- 172.21.15.59:0/1625750842 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c018070 con 0x7fe5580f8420 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe5477fe700 1 -- 172.21.15.59:0/1625750842 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe54c0092e0 con 0x7fe5580f8420 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe5477fe700 1 -- 172.21.15.59:0/1625750842 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c01dc50 con 0x7fe5580f8420 2024-02-18T00:38:46.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c005040 con 0x7fe5580f8420 2024-02-18T00:38:46.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55cf31700 10 client.41159.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:46.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe558116dd0 con 0x7fe5580f8420 2024-02-18T00:38:46.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.830+0000 7fe55cf31700 1 librados: init done 2024-02-18T00:38:46.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.831+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55805ee80 con 0x7fe5580f8420 2024-02-18T00:38:46.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 1 -- 172.21.15.59:0/1625750842 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe54c01d420 con 0x7fe5580f8420 2024-02-18T00:38:46.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 1 --2- 172.21.15.59:0/1625750842 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe540062c90 0x7fe540065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:46.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 1 -- 172.21.15.59:0/1625750842 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(432..432 src has 180..432) v4 ==== 8063+0+0 (secure 0 0 0) 0x7fe54c07e410 con 0x7fe5580f8420 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 10 client.41159.objecter ms_dispatch 0x7fe558001550 osd_map(432..432 src has 180..432) v4 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 3 client.41159.objecter handle_osd_map got epochs [432,432] > 0 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 3 client.41159.objecter handle_osd_map decoding full epoch 432 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe555d9b700 1 --2- 172.21.15.59:0/1625750842 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe540062c90 0x7fe540065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 20 client.41159.objecter dump_active .. 0 homeless 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe555d9b700 1 --2- 172.21.15.59:0/1625750842 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe540062c90 0x7fe540065150 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 rx=0x7fe55806c870 tx=0x7fe54800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.832+0000 7fe5477fe700 10 client.41159.objecter ms_handle_connect 0x7fe540062c90 2024-02-18T00:38:46.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:46.836+0000 7fe5477fe700 1 -- 172.21.15.59:0/1625750842 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe54c0519c0 con 0x7fe5580f8420 2024-02-18T00:38:46.865 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:46.868 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:46.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:46.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:47.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.043+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7fe55805fa60 con 0x7fe5580f8420 2024-02-18T00:38:47.060 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:47.063 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:47.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:47.165 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:47.194 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:45.906110+0000 mgr.y (mgr.24454) 404 : cluster [DBG] pgmap v643: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-18T00:38:47.194 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:46.364881+0000 osd.4 (osd.4) 253 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:47.194 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:46.365240+0000 osd.4 (osd.4) 254 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:47.197 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:47.251 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:45.906110+0000 mgr.y (mgr.24454) 404 : cluster [DBG] pgmap v643: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-18T00:38:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:46.364881+0000 osd.4 (osd.4) 253 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:46.365240+0000 osd.4 (osd.4) 254 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:45.906110+0000 mgr.y (mgr.24454) 404 : cluster [DBG] pgmap v643: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-02-18T00:38:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:46.364881+0000 osd.4 (osd.4) 253 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:46.365240+0000 osd.4 (osd.4) 254 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:47.254 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:47.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:47.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-18T00:38:47.356 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:47.409 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:47.409 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:47.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:47.577 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:47.597 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:47.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:47.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:47.725 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:47.750 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:47.752 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:47.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:47.854 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:47.880 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:47.898 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:47.901 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:47.901 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:47.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.955+0000 7fe5477fe700 1 -- 172.21.15.59:0/1625750842 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v434) v1 ==== 113+0+0 (secure 0 0 0) 0x7fe54c0515b0 con 0x7fe5580f8420 2024-02-18T00:38:47.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.957+0000 7fe55cf31700 10 librados: watch_flush enter 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.957+0000 7fe55cf31700 10 librados: watch_flush exit 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.957+0000 7fe55cf31700 20 client.41159.objecter shutdown clearing up homeless session... 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.957+0000 7fe55cf31700 10 client.41159.objecter successfully canceled tick 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe540062c90 msgr2=0x7fe540065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 --2- 172.21.15.59:0/1625750842 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe540062c90 0x7fe540065150 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 rx=0x7fe55806c870 tx=0x7fe54800b040).stop 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580f8420 msgr2=0x7fe55806b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580f8420 0x7fe55806b0d0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 rx=0x7fe54c0044e0 tx=0x7fe54c007f90).stop 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 shutdown_connections 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5580fe510 0x7fe55806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5580f8420 0x7fe55806b0d0 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 --2- 172.21.15.59:0/1625750842 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe540062c90 0x7fe540065150 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:47.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 --2- 172.21.15.59:0/1625750842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5580fdbb0 0x7fe55806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:47.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 >> 172.21.15.59:0/1625750842 conn(0x7fe5580f3800 msgr2=0x7fe5580f5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:47.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 shutdown_connections 2024-02-18T00:38:47.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 -- 172.21.15.59:0/1625750842 wait complete. 2024-02-18T00:38:47.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:47.958+0000 7fe55cf31700 1 librados: shutdown 2024-02-18T00:38:47.960 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.4 to 1 (10000) 2024-02-18T00:38:48.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:38:48.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:46.940698+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e433: 8 total, 8 up, 7 in 2024-02-18T00:38:48.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:47.043738+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.59:0/1625750842' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:38:48.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:47.044066+0000 mon.a (mon.0) 2548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:38:48.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:47.460957+0000 mon.a (mon.0) 2549 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:38:48.033 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:48.056 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:38:48.059 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:48.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:38:48.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:48.186 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:48.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:48.208 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:46.940698+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e433: 8 total, 8 up, 7 in 2024-02-18T00:38:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:47.043738+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.59:0/1625750842' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:38:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:47.044066+0000 mon.a (mon.0) 2548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:38:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:47.460957+0000 mon.a (mon.0) 2549 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:38:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:46.940698+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e433: 8 total, 8 up, 7 in 2024-02-18T00:38:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:47.043738+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.59:0/1625750842' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:38:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:47.044066+0000 mon.a (mon.0) 2548 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:38:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:47.460957+0000 mon.a (mon.0) 2549 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:38:48.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:38:48.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:48.365 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:48.515 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:48.521 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:48.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:48.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:48.650 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:48.998 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:49.002 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 5 1 2024-02-18T00:38:49.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:49.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:49.017 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:49.031 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:49.104 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:49.133 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:49.153 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:49.155 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:49.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:49.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:47.906721+0000 mgr.y (mgr.24454) 405 : cluster [DBG] pgmap v645: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:38:49.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:47.954318+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:38:49.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:47.954402+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e434: 8 total, 8 up, 7 in 2024-02-18T00:38:49.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:48.683091+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e435: 8 total, 8 up, 7 in 2024-02-18T00:38:49.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:47.906721+0000 mgr.y (mgr.24454) 405 : cluster [DBG] pgmap v645: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:38:49.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:47.954318+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:38:49.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:47.954402+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e434: 8 total, 8 up, 7 in 2024-02-18T00:38:49.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:48.683091+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e435: 8 total, 8 up, 7 in 2024-02-18T00:38:49.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:47.906721+0000 mgr.y (mgr.24454) 405 : cluster [DBG] pgmap v645: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:38:49.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:47.954318+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:38:49.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:47.954402+0000 mon.a (mon.0) 2551 : cluster [DBG] osdmap e434: 8 total, 8 up, 7 in 2024-02-18T00:38:49.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:48.683091+0000 mon.a (mon.0) 2552 : cluster [DBG] osdmap e435: 8 total, 8 up, 7 in 2024-02-18T00:38:49.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:49.283 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:49.301 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:49.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:49.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:49.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:49.435 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:49.662 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:49.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:49.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:49.767 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:49.795 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:49.813 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:49.815 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:49.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:49.917 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:49.942 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:49.961 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:49.964 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:49.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:50.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 -- 172.21.15.59:0/766551416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 msgr2=0x7ff91c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 --2- 172.21.15.59:0/766551416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 0x7ff91c0f7d40 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7ff9080047b0 tx=0x7ff908009cb0).stop 2024-02-18T00:38:50.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 -- 172.21.15.59:0/766551416 shutdown_connections 2024-02-18T00:38:50.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 --2- 172.21.15.59:0/766551416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff91c0f8420 0x7ff91c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 --2- 172.21.15.59:0/766551416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91c0fdbb0 0x7ff91c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 --2- 172.21.15.59:0/766551416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 0x7ff91c0f7d40 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 -- 172.21.15.59:0/766551416 >> 172.21.15.59:0/766551416 conn(0x7ff91c0f3800 msgr2=0x7ff91c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 -- 172.21.15.59:0/766551416 shutdown_connections 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 -- 172.21.15.59:0/766551416 wait complete. 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 librados: starting msgr at 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.975+0000 7ff923046700 1 librados: starting objecter 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 Processor -- start 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 -- start start 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 librados: setting wanted keys 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 librados: calling monclient init 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff91c0f8420 0x7ff91c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91c0fdbb0 0x7ff91c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 0x7ff91c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c116550 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1166c0 con 0x7ff91c0f8420 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff923046700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c116880 con 0x7ff91c0fe510 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91c0fdbb0 0x7ff91c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff9215e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 0x7ff91c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91c0fdbb0 0x7ff91c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48392/0 (socket says 172.21.15.59:48392) 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff9215e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 0x7ff91c06bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:58338/0 (socket says 172.21.15.59:58338) 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 -- 172.21.15.59:0/856680595 learned_addr learned my addr 172.21.15.59:0/856680595 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff91c0f8420 0x7ff91c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 -- 172.21.15.59:0/856680595 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 msgr2=0x7ff91c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 --2- 172.21.15.59:0/856680595 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 0x7ff91c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 -- 172.21.15.59:0/856680595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff91c0f8420 msgr2=0x7ff91c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 --2- 172.21.15.59:0/856680595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff91c0f8420 0x7ff91c06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 -- 172.21.15.59:0/856680595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff910005040 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff9215e3700 1 --2- 172.21.15.59:0/856680595 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff91c0fe510 0x7ff91c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff91bfff700 1 --2- 172.21.15.59:0/856680595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91c0fdbb0 0x7ff91c06b5c0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 rx=0x7ff9080044e0 tx=0x7ff908007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff919ffb700 10 client.?.objecter ms_handle_connect 0x7ff91c0fdbb0 2024-02-18T00:38:50.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.976+0000 7ff919ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.977+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908018070 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.977+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff908005040 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.977+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9080092e0 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.977+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90801dc50 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.054 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:38:50.054 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:38:50.055 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:38:50.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.977+0000 7ff923046700 10 client.31362.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:50.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.977+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c116df0 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.977+0000 7ff923046700 1 librados: init done 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.978+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff91c1070d0 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.978+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff90801d420 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.978+0000 7ff919ffb700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff90c062c90 0x7ff90c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(435..435 src has 180..435) v4 ==== 8063+0+0 (secure 0 0 0) 0x7ff90807e4a0 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 osd_map(435..435 src has 180..435) v4 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 3 client.31362.objecter handle_osd_map got epochs [435,435] > 0 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 3 client.31362.objecter handle_osd_map decoding full epoch 435 2024-02-18T00:38:50.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff90c062c90 0x7ff90c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 20 client.31362.objecter dump_active .. 0 homeless 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v435) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff908085070 con 0x7ff91c0fdbb0 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff90c062c90 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff90c062c90 0x7ff90c065150 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 rx=0x7ff910004870 tx=0x7ff91000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 10 client.31362.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 1 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff91c0fee00 0x7ff91c0ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 1 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 0 1 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c06c5b0 osd=0 4 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 10 client.31362.objecter _send_command 1 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c0ff730 con 0x7ff91c0fee00 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff9215e3700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff91c0fee00 0x7ff91c0ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c0fee00 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.979+0000 7ff9215e3700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff91c0fee00 0x7ff91c0ff240 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.980+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c0ff730 con 0x7ff91c0fee00 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.980+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.980+0000 7ff919ffb700 10 client.31362.objecter _finish_command 1 = system:0 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.980+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c06c5b0 osd=0 4 2024-02-18T00:38:50.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.980+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 0 1 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 10 client.31362.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 2 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 2 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c06c5b0 osd=0 3 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 0 2 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c06c5b0 osd=0 4 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 10 client.31362.objecter _send_command 2 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.995+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c0ff920 con 0x7ff91c0fee00 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c0ff920 con 0x7ff91c0fee00 2024-02-18T00:38:50.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff919ffb700 10 client.31362.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c06c5b0 osd=0 4 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 0 2 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 10 client.31362.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 3 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6805/26279110 conn(0x7ff91c028540 0x7ff91c11b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 3 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 1 3 2024-02-18T00:38:50.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c0f5e70 osd=1 4 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 10 client.31362.objecter _send_command 3 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c0f5fe0 con 0x7ff91c028540 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6805/26279110 conn(0x7ff91c028540 0x7ff91c11b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6805/26279110 conn(0x7ff91c028540 0x7ff91c11b030 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.997+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c028540 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.998+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c0f5fe0 con 0x7ff91c028540 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.998+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.998+0000 7ff919ffb700 10 client.31362.objecter _finish_command 3 = system:0 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.998+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c0f5e70 osd=1 4 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:49.998+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 1 3 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 10 client.31362.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 4 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 4 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c0f5e70 osd=1 3 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 1 4 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c0f5e70 osd=1 4 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 10 client.31362.objecter _send_command 4 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.013+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c05a040 con 0x7ff91c028540 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.014+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c05a040 con 0x7ff91c028540 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.014+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.014+0000 7ff919ffb700 10 client.31362.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.014+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c0f5e70 osd=1 4 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.014+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 1 4 2024-02-18T00:38:50.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 10 client.31362.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 5 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff91c0594d0 0x7ff91c11e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 5 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 2 5 2024-02-18T00:38:50.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c059360 osd=2 4 2024-02-18T00:38:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 10 client.31362.objecter _send_command 5 2024-02-18T00:38:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c0281b0 con 0x7ff91c0594d0 2024-02-18T00:38:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff91bfff700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff91c0594d0 0x7ff91c11e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c0594d0 2024-02-18T00:38:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.015+0000 7ff91bfff700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff91c0594d0 0x7ff91c11e6b0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.016+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c0281b0 con 0x7ff91c0594d0 2024-02-18T00:38:50.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.016+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.016+0000 7ff919ffb700 10 client.31362.objecter _finish_command 5 = system:0 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.016+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c059360 osd=2 4 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.016+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 2 5 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 10 client.31362.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 6 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 6 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c059360 osd=2 3 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 2 6 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c059360 osd=2 4 2024-02-18T00:38:50.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 10 client.31362.objecter _send_command 6 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.031+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c027600 con 0x7ff91c0594d0 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.032+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c027600 con 0x7ff91c0594d0 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.032+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.032+0000 7ff919ffb700 10 client.31362.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.032+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c059360 osd=2 4 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.032+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 2 6 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 10 client.31362.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 7 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff91c11c200 0x7ff91c121d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 7 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 3 7 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c027a10 osd=3 4 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 10 client.31362.objecter _send_command 7 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c124a50 con 0x7ff91c11c200 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff9215e3700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff91c11c200 0x7ff91c121d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c11c200 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.033+0000 7ff9215e3700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff91c11c200 0x7ff91c121d30 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.034+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c124a50 con 0x7ff91c11c200 2024-02-18T00:38:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.034+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.034+0000 7ff919ffb700 10 client.31362.objecter _finish_command 7 = system:0 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.034+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c027a10 osd=3 4 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.034+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 3 7 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 10 client.31362.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 8 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 8 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c027a10 osd=3 3 2024-02-18T00:38:50.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 3 8 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c027a10 osd=3 4 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 10 client.31362.objecter _send_command 8 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.049+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c11b760 con 0x7ff91c11c200 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c11b760 con 0x7ff91c11c200 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff919ffb700 10 client.31362.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c027a10 osd=3 4 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 3 8 2024-02-18T00:38:50.069 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:38:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 10 client.31362.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 9 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff91c11f880 0x7ff91c1233a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 9 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 4 9 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b520 osd=4 4 2024-02-18T00:38:50.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 10 client.31362.objecter _send_command 9 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.050+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c11b6c0 con 0x7ff91c11f880 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.051+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff91c11f880 0x7ff91c1233a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.051+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c11f880 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.051+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff91c11f880 0x7ff91c1233a0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.052+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c11b6c0 con 0x7ff91c11f880 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.052+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.052+0000 7ff919ffb700 10 client.31362.objecter _finish_command 9 = system:0 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.052+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c11b520 osd=4 4 2024-02-18T00:38:50.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.052+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 4 9 2024-02-18T00:38:50.073 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:50.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 10 client.31362.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 10 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 10 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c11b520 osd=4 3 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 4 10 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b520 osd=4 4 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 10 client.31362.objecter _send_command 10 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.071+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c11b6c0 con 0x7ff91c11f880 2024-02-18T00:38:50.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c11b6c0 con 0x7ff91c11f880 2024-02-18T00:38:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff919ffb700 10 client.31362.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c11b520 osd=4 4 2024-02-18T00:38:50.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 4 10 2024-02-18T00:38:50.076 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 10 client.31362.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 11 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff91c123e00 0x7ff91c124220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 11 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 5 11 2024-02-18T00:38:50.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b6c0 osd=5 4 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 10 client.31362.objecter _send_command 11 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c12b3a0 con 0x7ff91c123e00 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.073+0000 7ff91bfff700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff91c123e00 0x7ff91c124220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.074+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c123e00 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.074+0000 7ff91bfff700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff91c123e00 0x7ff91c124220 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.074+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c12b3a0 con 0x7ff91c123e00 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.074+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.074+0000 7ff919ffb700 10 client.31362.objecter _finish_command 11 = system:0 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.074+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c11b6c0 osd=5 4 2024-02-18T00:38:50.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.074+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 5 11 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 10 client.31362.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 12 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 12 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c11b6c0 osd=5 3 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 5 12 2024-02-18T00:38:50.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b6c0 osd=5 4 2024-02-18T00:38:50.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 10 client.31362.objecter _send_command 12 2024-02-18T00:38:50.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.089+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c128c40 con 0x7ff91c123e00 2024-02-18T00:38:50.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c128c40 con 0x7ff91c123e00 2024-02-18T00:38:50.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff919ffb700 10 client.31362.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c11b6c0 osd=5 4 2024-02-18T00:38:50.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 5 12 2024-02-18T00:38:50.092 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:38:50.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 10 client.31362.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 13 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff91c1292c0 0x7ff91c12b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 13 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 6 13 2024-02-18T00:38:50.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c128c40 osd=6 4 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 10 client.31362.objecter _send_command 13 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.091+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c12e500 con 0x7ff91c1292c0 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.092+0000 7ff9215e3700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff91c1292c0 0x7ff91c12b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.092+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c1292c0 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.092+0000 7ff9215e3700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff91c1292c0 0x7ff91c12b730 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.093+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c12e500 con 0x7ff91c1292c0 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.093+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.093+0000 7ff919ffb700 10 client.31362.objecter _finish_command 13 = system:0 2024-02-18T00:38:50.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.093+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c128c40 osd=6 4 2024-02-18T00:38:50.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.093+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 6 13 2024-02-18T00:38:50.098 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:49.122278+0000 mon.a (mon.0) 2553 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:38:50.098 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:49.123018+0000 mon.a (mon.0) 2554 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:38:50.098 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:49.127956+0000 mon.a (mon.0) 2555 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:50.100 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 10 client.31362.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 14 2024-02-18T00:38:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 14 2024-02-18T00:38:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c128c40 osd=6 3 2024-02-18T00:38:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 6 14 2024-02-18T00:38:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c128c40 osd=6 4 2024-02-18T00:38:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 10 client.31362.objecter _send_command 14 2024-02-18T00:38:50.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.107+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c12e500 con 0x7ff91c1292c0 2024-02-18T00:38:50.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c12e500 con 0x7ff91c1292c0 2024-02-18T00:38:50.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff919ffb700 10 client.31362.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c128c40 osd=6 4 2024-02-18T00:38:50.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 6 14 2024-02-18T00:38:50.112 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:38:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 10 client.31362.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 15 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6812/53025171 conn(0x7ff91c12d800 0x7ff91c12dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 15 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 7 15 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c12e500 osd=7 4 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 10 client.31362.objecter _send_command 15 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.110+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff91c12e2f0 con 0x7ff91c12d800 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.111+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6812/53025171 conn(0x7ff91c12d800 0x7ff91c12dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:50.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.111+0000 7ff919ffb700 10 client.31362.objecter ms_handle_connect 0x7ff91c12d800 2024-02-18T00:38:50.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.111+0000 7ff920de2700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6812/53025171 conn(0x7ff91c12d800 0x7ff91c12dc80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:50.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.112+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff91c12e2f0 con 0x7ff91c12d800 2024-02-18T00:38:50.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.112+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:38:50.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.112+0000 7ff919ffb700 10 client.31362.objecter _finish_command 15 = system:0 2024-02-18T00:38:50.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.112+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c12e500 osd=7 4 2024-02-18T00:38:50.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.112+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 7 15 2024-02-18T00:38:50.120 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:50.123 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:50.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:50.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 10 client.31362.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:38:50.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign -1 16 2024-02-18T00:38:50.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 20 client.31362.objecter _get_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 15 client.31362.objecter _session_command_op_remove -1 16 2024-02-18T00:38:50.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 20 client.31362.objecter get_session s=0x7ff91c12e500 osd=7 3 2024-02-18T00:38:50.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 15 client.31362.objecter _session_command_op_assign 7 16 2024-02-18T00:38:50.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c12e500 osd=7 4 2024-02-18T00:38:50.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 10 client.31362.objecter _send_command 16 2024-02-18T00:38:50.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.126+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff91c12e2f0 con 0x7ff91c12d800 2024-02-18T00:38:50.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff919ffb700 1 -- 172.21.15.59:0/856680595 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff91c12e2f0 con 0x7ff91c12d800 2024-02-18T00:38:50.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff919ffb700 10 client.31362.objecter ms_dispatch 0x7ff91c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:38:50.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff919ffb700 10 client.31362.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff919ffb700 20 client.31362.objecter put_session s=0x7ff91c12e500 osd=7 4 2024-02-18T00:38:50.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff919ffb700 15 client.31362.objecter _session_command_op_remove 7 16 2024-02-18T00:38:50.129 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:38:50.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 librados: watch_flush enter 2024-02-18T00:38:50.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 librados: watch_flush exit 2024-02-18T00:38:50.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.0 2024-02-18T00:38:50.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff91c0fee00 msgr2=0x7ff91c0ff240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff91c0fee00 0x7ff91c0ff240 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c06c5b0 osd=0 1 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.1 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6805/26279110 conn(0x7ff91c028540 msgr2=0x7ff91c11b030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6805/26279110 conn(0x7ff91c028540 0x7ff91c11b030 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c0f5e70 osd=1 1 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.2 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff91c0594d0 msgr2=0x7ff91c11e6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff91c0594d0 0x7ff91c11e6b0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c059360 osd=2 1 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.3 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff91c11c200 msgr2=0x7ff91c121d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff91c11c200 0x7ff91c121d30 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c027a10 osd=3 1 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff9215e3700 1 -- 172.21.15.59:0/856680595 reap_dead start 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.4 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff91c11f880 msgr2=0x7ff91c1233a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff91c11f880 0x7ff91c1233a0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b520 osd=4 1 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff9215e3700 1 -- 172.21.15.59:0/856680595 reap_dead start 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.5 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff91c123e00 msgr2=0x7ff91c124220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff91c123e00 0x7ff91c124220 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c11b6c0 osd=5 1 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.6 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff91c1292c0 msgr2=0x7ff91c12b730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff91c1292c0 0x7ff91c12b730 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c128c40 osd=6 1 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter close_session for osd.7 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6812/53025171 conn(0x7ff91c12d800 msgr2=0x7ff91c12dc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.137:6812/53025171 conn(0x7ff91c12d800 0x7ff91c12dc80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter put_session s=0x7ff91c12e500 osd=7 1 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 20 client.31362.objecter shutdown clearing up homeless session... 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.128+0000 7ff923046700 10 client.31362.objecter successfully canceled tick 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff90c062c90 msgr2=0x7ff90c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff90c062c90 0x7ff90c065150 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 rx=0x7ff910004870 tx=0x7ff91000b040).stop 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91c0fdbb0 msgr2=0x7ff91c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 --2- 172.21.15.59:0/856680595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff91c0fdbb0 0x7ff91c06b5c0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 rx=0x7ff9080044e0 tx=0x7ff908007f90).stop 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff9215e3700 1 -- 172.21.15.59:0/856680595 reap_dead start 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 shutdown_connections 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 >> 172.21.15.59:0/856680595 conn(0x7ff91c0f3800 msgr2=0x7ff91c0f52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 shutdown_connections 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 -- 172.21.15.59:0/856680595 wait complete. 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:50.129+0000 7ff923046700 1 librados: shutdown 2024-02-18T00:38:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.135 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:38:50.225 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:50.251 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:50.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:49.122278+0000 mon.a (mon.0) 2553 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:38:50.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:49.123018+0000 mon.a (mon.0) 2554 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:38:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:49.127956+0000 mon.a (mon.0) 2555 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:49.122278+0000 mon.a (mon.0) 2553 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:38:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:49.123018+0000 mon.a (mon.0) 2554 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:38:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:49.127956+0000 mon.a (mon.0) 2555 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:38:50.269 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:50.272 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:50.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:50.373 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:50.404 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:50.603 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:50.605 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:50.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:50.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:50.736 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:51.062 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:51.086 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:51.089 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:51.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:51.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:51.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:49.907142+0000 mgr.y (mgr.24454) 406 : cluster [DBG] pgmap v648: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:38:51.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:49.978905+0000 mon.a (mon.0) 2556 : audit [DBG] from='client.? 172.21.15.59:0/856680595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:51.218 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:49.907142+0000 mgr.y (mgr.24454) 406 : cluster [DBG] pgmap v648: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:38:51.218 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:49.978905+0000 mon.a (mon.0) 2556 : audit [DBG] from='client.? 172.21.15.59:0/856680595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:51.220 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:51.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:38:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:38:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:49.907142+0000 mgr.y (mgr.24454) 406 : cluster [DBG] pgmap v648: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:38:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:49.978905+0000 mon.a (mon.0) 2556 : audit [DBG] from='client.? 172.21.15.59:0/856680595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:38:51.370 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:51.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:51.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:51.474 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:51.506 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:51.527 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:51.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:38:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:38:51.530 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:51.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:51.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:51.660 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:51.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:51.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:51.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:51.813 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:52.012 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:52.014 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:52.014 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:52.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:52.145 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:52.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:51.266211+0000 mgr.y (mgr.24454) 407 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:52.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:51.266211+0000 mgr.y (mgr.24454) 407 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:52.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:38:52] "GET /metrics HTTP/1.1" 200 189635 "" "Prometheus/2.33.4" 2024-02-18T00:38:52.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:51.266211+0000 mgr.y (mgr.24454) 407 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:38:52.271 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:52.282 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:52.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:52.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:52.413 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:52.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:52.765 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:52.866 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:52.894 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:53.035 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:51.908036+0000 mgr.y (mgr.24454) 408 : cluster [DBG] pgmap v649: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:52.940394+0000 mon.c (mon.2) 419 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:52.940802+0000 mon.a (mon.0) 2557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:52.941325+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-61"}]: dispatch 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:52.941604+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-61"}]: dispatch 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:51.908036+0000 mgr.y (mgr.24454) 408 : cluster [DBG] pgmap v649: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:52.940394+0000 mon.c (mon.2) 419 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:52.940802+0000 mon.a (mon.0) 2557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:52.941325+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-61"}]: dispatch 2024-02-18T00:38:53.036 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:52.941604+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-61"}]: dispatch 2024-02-18T00:38:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:53.047 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:53.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:53.148 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:53.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:53.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:51.908036+0000 mgr.y (mgr.24454) 408 : cluster [DBG] pgmap v649: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:38:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:52.940394+0000 mon.c (mon.2) 419 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:52.940802+0000 mon.a (mon.0) 2557 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:38:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:52.941325+0000 mon.c (mon.2) 420 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-61"}]: dispatch 2024-02-18T00:38:53.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:52.941604+0000 mon.a (mon.0) 2558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-61"}]: dispatch 2024-02-18T00:38:53.304 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:53.334 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:53.354 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:53.354 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:38:53.355 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:38:53.357 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:53.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:53.458 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:53.487 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:53.505 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:53.509 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:53.509 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:53.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:53.639 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:54.018 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:54.020 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:54.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:54.121 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:54.153 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:54.174 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:54.177 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:54.177 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:54.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:54.308 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:54.327 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:54.330 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:54.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:54.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:54.460 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:54.712 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:54.715 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:54.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:54.816 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:54.847 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:54.867 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:54.869 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:54.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:54.970 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:55.004 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:55.023 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:55.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:55.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:55.127 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:38:55.155 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:55.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:53.712189+0000 osd.0 (osd.0) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:53.712371+0000 osd.0 (osd.0) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:53.908440+0000 mgr.y (mgr.24454) 409 : cluster [DBG] pgmap v650: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 734 B/s rd, 0 op/s 2024-02-18T00:38:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:53.969446+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e436: 8 total, 8 up, 7 in 2024-02-18T00:38:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:53.712189+0000 osd.0 (osd.0) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:53.712371+0000 osd.0 (osd.0) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:53.908440+0000 mgr.y (mgr.24454) 409 : cluster [DBG] pgmap v650: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 734 B/s rd, 0 op/s 2024-02-18T00:38:55.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:53.969446+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e436: 8 total, 8 up, 7 in 2024-02-18T00:38:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:53.712189+0000 osd.0 (osd.0) 204 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:38:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:53.712371+0000 osd.0 (osd.0) 205 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:38:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:53.908440+0000 mgr.y (mgr.24454) 409 : cluster [DBG] pgmap v650: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 734 B/s rd, 0 op/s 2024-02-18T00:38:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:53.969446+0000 mon.a (mon.0) 2559 : cluster [DBG] osdmap e436: 8 total, 8 up, 7 in 2024-02-18T00:38:55.303 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:38:55.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:55.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:38:55.409 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:55.439 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:55.462 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:55.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:55.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:55.595 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:55.698 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:55.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:55.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:55.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/3318978136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fdbb0 msgr2=0x7f6fd40fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/3318978136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fdbb0 0x7f6fd40fdf90 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 rx=0x7f6fc40047b0 tx=0x7f6fc4009cb0).stop 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/3318978136 shutdown_connections 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/3318978136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fd40f8420 0x7f6fd4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/3318978136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fdbb0 0x7f6fd40fdf90 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/3318978136 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fe510 0x7f6fd40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/3318978136 >> 172.21.15.59:0/3318978136 conn(0x7f6fd40f3800 msgr2=0x7f6fd40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/3318978136 shutdown_connections 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.794+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/3318978136 wait complete. 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 librados: starting msgr at 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 librados: starting objecter 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 Processor -- start 2024-02-18T00:38:55.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 -- start start 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 librados: setting wanted keys 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 librados: calling monclient init 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fd40f8420 0x7f6fd410e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fdbb0 0x7f6fd410e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fe510 0x7f6fd4108300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41087f0 con 0x7f6fd40fe510 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd4108960 con 0x7f6fd40f8420 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fdbdd4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd4108b20 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fd936f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fdbb0 0x7f6fd410e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fda371700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fe510 0x7f6fd4108300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fd936f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fdbb0 0x7f6fd410e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:58360/0 (socket says 172.21.15.59:58360) 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fda371700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fe510 0x7f6fd4108300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48406/0 (socket says 172.21.15.59:48406) 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.795+0000 7f6fd936f700 1 -- 172.21.15.59:0/2790311769 learned_addr learned my addr 172.21.15.59:0/2790311769 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:55.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd9b70700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fd40f8420 0x7f6fd410e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd936f700 1 -- 172.21.15.59:0/2790311769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fd40f8420 msgr2=0x7f6fd410e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd936f700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fd40f8420 0x7f6fd410e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd936f700 1 -- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fe510 msgr2=0x7f6fd4108300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd936f700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fe510 0x7f6fd4108300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd936f700 1 -- 172.21.15.59:0/2790311769 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0005040 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd9b70700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fd40f8420 0x7f6fd410e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fda371700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fe510 0x7f6fd4108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fd936f700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fdbb0 0x7f6fd410e650 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 rx=0x7f6fd0009400 tx=0x7f6fd0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fcaffd700 10 client.?.objecter ms_handle_connect 0x7f6fd40fdbb0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fcaffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fcaffd700 1 -- 172.21.15.59:0/2790311769 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0007a40 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc4005040 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fcaffd700 1 -- 172.21.15.59:0/2790311769 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fd000f040 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fcaffd700 1 -- 172.21.15.59:0/2790311769 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0014aa0 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fdbdd4700 10 client.41171.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd4109160 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.796+0000 7f6fdbdd4700 1 librados: init done 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.797+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd405ee80 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.797+0000 7f6fcaffd700 1 -- 172.21.15.59:0/2790311769 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6fd000e630 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fcaffd700 1 --2- 172.21.15.59:0/2790311769 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6fc0062c90 0x7f6fc0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fcaffd700 1 -- 172.21.15.59:0/2790311769 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(437..437 src has 180..437) v4 ==== 8022+0+0 (secure 0 0 0) 0x7f6fd007ce10 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fcaffd700 10 client.41171.objecter ms_dispatch 0x7f6fd4001550 osd_map(437..437 src has 180..437) v4 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fcaffd700 3 client.41171.objecter handle_osd_map got epochs [437,437] > 0 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fcaffd700 3 client.41171.objecter handle_osd_map decoding full epoch 437 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fd9b70700 1 --2- 172.21.15.59:0/2790311769 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6fc0062c90 0x7f6fc0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fcaffd700 20 client.41171.objecter dump_active .. 0 homeless 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fcaffd700 10 client.41171.objecter ms_handle_connect 0x7f6fc0062c90 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.798+0000 7f6fd9b70700 1 --2- 172.21.15.59:0/2790311769 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6fc0062c90 0x7f6fc0065150 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 rx=0x7f6fc4005010 tx=0x7f6fc4007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:55.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.802+0000 7f6fcaffd700 1 -- 172.21.15.59:0/2790311769 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6fd0049f60 con 0x7f6fd40fdbb0 2024-02-18T00:38:55.831 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:55.852 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:55.855 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:55.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:55.956 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:55.981 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:54.975574+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e437: 8 total, 8 up, 7 in 2024-02-18T00:38:55.981 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:54.979719+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:55.981 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:54.980328+0000 mon.a (mon.0) 2561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:55.982 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:54.975574+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e437: 8 total, 8 up, 7 in 2024-02-18T00:38:55.982 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:54.979719+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:55.982 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:54.980328+0000 mon.a (mon.0) 2561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:55.987 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:56.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:55.999+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f6fd4109870 con 0x7f6fd40fdbb0 2024-02-18T00:38:56.012 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:56.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:56.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:56.028 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:38:56.116 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:56.145 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:56.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:54.975574+0000 mon.a (mon.0) 2560 : cluster [DBG] osdmap e437: 8 total, 8 up, 7 in 2024-02-18T00:38:56.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:54.979719+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:56.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:54.980328+0000 mon.a (mon.0) 2561 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:38:56.400 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:38:56.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-18T00:38:56.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:56.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:38:56.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:38:56.536 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:56.557 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:38:56.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:56.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:38:56.661 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:38:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 -- 172.21.15.59:0/2731648849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 msgr2=0x7f94c8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 --2- 172.21.15.59:0/2731648849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 0x7f94c8104a20 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 rx=0x7f94c0009be0 tx=0x7f94c0004d10).stop 2024-02-18T00:38:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 -- 172.21.15.59:0/2731648849 shutdown_connections 2024-02-18T00:38:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 --2- 172.21.15.59:0/2731648849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94c80fdbb0 0x7f94c80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 --2- 172.21.15.59:0/2731648849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 0x7f94c8104a20 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 --2- 172.21.15.59:0/2731648849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 0x7f94c80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 -- 172.21.15.59:0/2731648849 >> 172.21.15.59:0/2731648849 conn(0x7f94c80f3800 msgr2=0x7f94c80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 -- 172.21.15.59:0/2731648849 shutdown_connections 2024-02-18T00:38:56.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.674+0000 7f94ce790700 1 -- 172.21.15.59:0/2731648849 wait complete. 2024-02-18T00:38:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.675+0000 7f94ce790700 1 librados: starting msgr at 2024-02-18T00:38:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.675+0000 7f94ce790700 1 librados: starting objecter 2024-02-18T00:38:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.675+0000 7f94ce790700 1 Processor -- start 2024-02-18T00:38:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.675+0000 7f94ce790700 1 -- start start 2024-02-18T00:38:56.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.675+0000 7f94ce790700 1 librados: setting wanted keys 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.675+0000 7f94ce790700 1 librados: calling monclient init 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ce790700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 0x7f94c810dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ce790700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94c80fdbb0 0x7f94c810e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ce790700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 0x7f94c8108190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ce790700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c8108680 con 0x7f94c80fe510 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ce790700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c81087f0 con 0x7f94c80fdbb0 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ce790700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c81089b0 con 0x7f94c80f8420 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 0x7f94c8108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 0x7f94c8108190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48434/0 (socket says 172.21.15.59:48434) 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 -- 172.21.15.59:0/249716814 learned_addr learned my addr 172.21.15.59:0/249716814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94c7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 0x7f94c810dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94c77fe700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94c80fdbb0 0x7f94c810e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 -- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 msgr2=0x7f94c810dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:56.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 0x7f94c810dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 -- 172.21.15.59:0/249716814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94c80fdbb0 msgr2=0x7f94c810e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94c80fdbb0 0x7f94c810e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 -- 172.21.15.59:0/249716814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c00097c0 con 0x7f94c80fe510 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94c7fff700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 0x7f94c810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.676+0000 7f94ccd2d700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 0x7f94c8108190 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7f94c0004e00 tx=0x7f94c0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94c57fa700 10 client.?.objecter ms_handle_connect 0x7f94c80fe510 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94c57fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:38:56.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94c57fa700 1 -- 172.21.15.59:0/249716814 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000e040 con 0x7f94c80fe510 2024-02-18T00:38:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c8108c30 con 0x7f94c80fe510 2024-02-18T00:38:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94ce790700 10 client.31374.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:38:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c806af10 con 0x7f94c80fe510 2024-02-18T00:38:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94ce790700 1 librados: init done 2024-02-18T00:38:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94c57fa700 1 -- 172.21.15.59:0/249716814 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c0018040 con 0x7f94c80fe510 2024-02-18T00:38:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.677+0000 7f94c57fa700 1 -- 172.21.15.59:0/249716814 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0013ea0 con 0x7f94c80fe510 2024-02-18T00:38:56.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.678+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4004f50 con 0x7f94c80fe510 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.679+0000 7f94c57fa700 1 -- 172.21.15.59:0/249716814 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f94c0013420 con 0x7f94c80fe510 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.679+0000 7f94c57fa700 1 --2- 172.21.15.59:0/249716814 >> v2:172.21.15.59:6800/3721891797 conn(0x7f94b0062ce0 0x7f94b00651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.679+0000 7f94c57fa700 1 -- 172.21.15.59:0/249716814 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(438..438 src has 180..438) v4 ==== 8035+0+0 (secure 0 0 0) 0x7f94c007e160 con 0x7f94c80fe510 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.679+0000 7f94c57fa700 10 client.31374.objecter ms_dispatch 0x7f94c8001550 osd_map(438..438 src has 180..438) v4 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.679+0000 7f94c57fa700 3 client.31374.objecter handle_osd_map got epochs [438,438] > 0 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.679+0000 7f94c57fa700 3 client.31374.objecter handle_osd_map decoding full epoch 438 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.679+0000 7f94c57fa700 20 client.31374.objecter dump_active .. 0 homeless 2024-02-18T00:38:56.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.681+0000 7f94c7fff700 1 --2- 172.21.15.59:0/249716814 >> v2:172.21.15.59:6800/3721891797 conn(0x7f94b0062ce0 0x7f94b00651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:38:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.681+0000 7f94c57fa700 10 client.31374.objecter ms_handle_connect 0x7f94b0062ce0 2024-02-18T00:38:56.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.681+0000 7f94c7fff700 1 --2- 172.21.15.59:0/249716814 >> v2:172.21.15.59:6800/3721891797 conn(0x7f94b0062ce0 0x7f94b00651a0 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 rx=0x7f94b8007bf0 tx=0x7f94b800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:38:56.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.684+0000 7f94c57fa700 1 -- 172.21.15.59:0/249716814 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f94c00516f0 con 0x7f94c80fe510 2024-02-18T00:38:56.691 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:56.712 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:38:56.714 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:56.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:38:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:56.816 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:38:56.845 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:56.865 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:38:56.867 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:56.868 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:38:56.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.890+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f94b4005200 con 0x7f94c80fe510 2024-02-18T00:38:56.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:38:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.982+0000 7f94c57fa700 1 -- 172.21.15.59:0/249716814 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v439) v1 ==== 113+0+0 (secure 0 0 0) 0x7f94c0021030 con 0x7f94c80fe510 2024-02-18T00:38:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.984+0000 7f6fcaffd700 1 -- 172.21.15.59:0/2790311769 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v439) v1 ==== 91+0+0 (secure 0 0 0) 0x7f6fd0050340 con 0x7f6fd40fdbb0 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.985+0000 7f94ce790700 10 librados: watch_flush enter 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f94ce790700 10 librados: watch_flush exit 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f94ce790700 20 client.31374.objecter shutdown clearing up homeless session... 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f94ce790700 10 client.31374.objecter successfully canceled tick 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.987+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 >> v2:172.21.15.59:6800/3721891797 conn(0x7f94b0062ce0 msgr2=0x7f94b00651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.987+0000 7f94ce790700 1 --2- 172.21.15.59:0/249716814 >> v2:172.21.15.59:6800/3721891797 conn(0x7f94b0062ce0 0x7f94b00651a0 secure :-1 s=READY pgs=3116 cs=0 l=1 rev1=1 rx=0x7f94b8007bf0 tx=0x7f94b800b040).stop 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.987+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 msgr2=0x7f94c8108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.987+0000 7f94ce790700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 0x7f94c8108190 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7f94c0004e00 tx=0x7f94c0004d10).stop 2024-02-18T00:38:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f6fdbdd4700 10 librados: watch_flush enter 2024-02-18T00:38:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f6fdbdd4700 10 librados: watch_flush exit 2024-02-18T00:38:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f6fdbdd4700 20 client.41171.objecter shutdown clearing up homeless session... 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f6fdbdd4700 10 client.41171.objecter successfully canceled tick 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.986+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6fc0062c90 msgr2=0x7f6fc0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.987+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/2790311769 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6fc0062c90 0x7f6fc0065150 secure :-1 s=READY pgs=3115 cs=0 l=1 rev1=1 rx=0x7f6fc4005010 tx=0x7f6fc4007f90).stop 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.987+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fdbb0 msgr2=0x7f6fd410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.987+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fdbb0 0x7f6fd410e650 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 rx=0x7f6fd0009400 tx=0x7f6fd0009cb0).stop 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 shutdown_connections 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/2790311769 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6fc0062c90 0x7f6fc0065150 unknown :-1 s=CLOSED pgs=3115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fd40f8420 0x7f6fd410e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd40fe510 0x7f6fd4108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f6fdbdd4700 1 --2- 172.21.15.59:0/2790311769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40fdbb0 0x7f6fd410e650 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 >> 172.21.15.59:0/2790311769 conn(0x7f6fd40f3800 msgr2=0x7f6fd40fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 shutdown_connections 2024-02-18T00:38:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f94ce790700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94c80fe510 0x7f94c8108190 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f94ce790700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94c80f8420 0x7f94c810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f94ce790700 1 --2- 172.21.15.59:0/249716814 >> v2:172.21.15.59:6800/3721891797 conn(0x7f94b0062ce0 0x7f94b00651a0 unknown :-1 s=CLOSED pgs=3116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f94ce790700 1 --2- 172.21.15.59:0/249716814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94c80fdbb0 0x7f94c810e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:38:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.988+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 >> 172.21.15.59:0/249716814 conn(0x7f94c80f3800 msgr2=0x7f94c80f5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:38:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.992+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 shutdown_connections 2024-02-18T00:38:56.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.992+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 shutdown_connections 2024-02-18T00:38:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.993+0000 7f6fdbdd4700 1 -- 172.21.15.59:0/2790311769 wait complete. 2024-02-18T00:38:56.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.994+0000 7f6fdbdd4700 1 librados: shutdown 2024-02-18T00:38:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:38:56.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.995+0000 7f94ce790700 1 -- 172.21.15.59:0/249716814 wait complete. 2024-02-18T00:38:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:38:57.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:38:56.999+0000 7f94ce790700 1 librados: shutdown 2024-02-18T00:38:57.009 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.5 to 1 (10000) 2024-02-18T00:38:57.011 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:57.050 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:38:57.053 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:57.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:38:57.154 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:57.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:55.909423+0000 mgr.y (mgr.24454) 410 : cluster [DBG] pgmap v653: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:55.982147+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:55.982314+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e438: 8 total, 8 up, 7 in 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:55.986493+0000 mon.c (mon.2) 422 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:55.995993+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:55.996433+0000 mon.a (mon.0) 2564 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:56.004927+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.59:0/2790311769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:56.005496+0000 mon.a (mon.0) 2565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:57.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:56.892065+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.59:0/249716814' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:38:57.182 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:57.202 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:57.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:57.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:55.909423+0000 mgr.y (mgr.24454) 410 : cluster [DBG] pgmap v653: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:55.982147+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:55.982314+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e438: 8 total, 8 up, 7 in 2024-02-18T00:38:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:55.986493+0000 mon.c (mon.2) 422 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:57.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:55.995993+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:55.996433+0000 mon.a (mon.0) 2564 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:56.004927+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.59:0/2790311769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:56.005496+0000 mon.a (mon.0) 2565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:56.892065+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.59:0/249716814' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:55.909423+0000 mgr.y (mgr.24454) 410 : cluster [DBG] pgmap v653: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:55.982147+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:55.982314+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e438: 8 total, 8 up, 7 in 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:55.986493+0000 mon.c (mon.2) 422 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:38:57.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:55.995993+0000 mon.c (mon.2) 423 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:57.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:55.996433+0000 mon.a (mon.0) 2564 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:38:57.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:56.004927+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.59:0/2790311769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:57.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:56.005496+0000 mon.a (mon.0) 2565 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:38:57.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:56.892065+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.59:0/249716814' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:38:57.305 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:38:57.332 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:57.351 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:38:57.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:38:57.354 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:38:57.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:38:57.483 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:56.977832+0000 mon.a (mon.0) 2567 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:56.982375+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:56.982461+0000 mon.a (mon.0) 2569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:38:56.982525+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.59:0/249716814' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:56.982589+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e439: 8 total, 8 up, 7 in 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.346670+0000 osd.4 (osd.4) 255 : cluster [DBG] 62.16 scrub starts 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.350160+0000 osd.4 (osd.4) 256 : cluster [DBG] 62.16 scrub ok 2024-02-18T00:38:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.420406+0000 osd.5 (osd.5) 183 : cluster [DBG] 6.d scrub starts 2024-02-18T00:38:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.423401+0000 osd.5 (osd.5) 184 : cluster [DBG] 6.d scrub ok 2024-02-18T00:38:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.505563+0000 osd.2 (osd.2) 183 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:38:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.506734+0000 osd.2 (osd.2) 184 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:38:58.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:56.977832+0000 mon.a (mon.0) 2567 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:58.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:56.982375+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:58.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:56.982461+0000 mon.a (mon.0) 2569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:38:58.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:38:56.982525+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.59:0/249716814' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:56.982589+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e439: 8 total, 8 up, 7 in 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.346670+0000 osd.4 (osd.4) 255 : cluster [DBG] 62.16 scrub starts 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.350160+0000 osd.4 (osd.4) 256 : cluster [DBG] 62.16 scrub ok 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.420406+0000 osd.5 (osd.5) 183 : cluster [DBG] 6.d scrub starts 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.423401+0000 osd.5 (osd.5) 184 : cluster [DBG] 6.d scrub ok 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.505563+0000 osd.2 (osd.2) 183 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.506734+0000 osd.2 (osd.2) 184 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:56.977832+0000 mon.a (mon.0) 2567 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:38:58.400 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:56.982375+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:56.982461+0000 mon.a (mon.0) 2569 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:38:56.982525+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.59:0/249716814' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:56.982589+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e439: 8 total, 8 up, 7 in 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.346670+0000 osd.4 (osd.4) 255 : cluster [DBG] 62.16 scrub starts 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.350160+0000 osd.4 (osd.4) 256 : cluster [DBG] 62.16 scrub ok 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.420406+0000 osd.5 (osd.5) 183 : cluster [DBG] 6.d scrub starts 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.423401+0000 osd.5 (osd.5) 184 : cluster [DBG] 6.d scrub ok 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.505563+0000 osd.2 (osd.2) 183 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:38:58.401 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.506734+0000 osd.2 (osd.2) 184 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.680993+0000 osd.3 (osd.3) 239 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.682087+0000 osd.3 (osd.3) 240 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:57.909883+0000 mgr.y (mgr.24454) 411 : cluster [DBG] pgmap v656: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:58.006549+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e440: 8 total, 8 up, 7 in 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:58.686692+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e441: 8 total, 8 up, 7 in 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.680993+0000 osd.3 (osd.3) 239 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.682087+0000 osd.3 (osd.3) 240 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:38:59.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:57.909883+0000 mgr.y (mgr.24454) 411 : cluster [DBG] pgmap v656: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:59.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:58.006549+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e440: 8 total, 8 up, 7 in 2024-02-18T00:38:59.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:38:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:58.686692+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e441: 8 total, 8 up, 7 in 2024-02-18T00:38:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.680993+0000 osd.3 (osd.3) 239 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:38:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.682087+0000 osd.3 (osd.3) 240 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:38:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:57.909883+0000 mgr.y (mgr.24454) 411 : cluster [DBG] pgmap v656: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:38:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:58.006549+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e440: 8 total, 8 up, 7 in 2024-02-18T00:38:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:38:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:58.686692+0000 mon.a (mon.0) 2573 : cluster [DBG] osdmap e441: 8 total, 8 up, 7 in 2024-02-18T00:38:59.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 6 1 2024-02-18T00:39:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:58.466679+0000 osd.5 (osd.5) 185 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:39:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:58.470134+0000 osd.5 (osd.5) 186 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:39:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:58.747489+0000 osd.6 (osd.6) 205 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:39:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:58.749671+0000 osd.6 (osd.6) 206 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:39:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.310168+0000 osd.4 (osd.4) 257 : cluster [DBG] 62.5 scrub starts 2024-02-18T00:39:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.313622+0000 osd.4 (osd.4) 258 : cluster [DBG] 62.5 scrub ok 2024-02-18T00:39:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.686994+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e442: 8 total, 8 up, 7 in 2024-02-18T00:39:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:58.466679+0000 osd.5 (osd.5) 185 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:39:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:58.470134+0000 osd.5 (osd.5) 186 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:39:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:58.747489+0000 osd.6 (osd.6) 205 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:39:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:58.749671+0000 osd.6 (osd.6) 206 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:39:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.310168+0000 osd.4 (osd.4) 257 : cluster [DBG] 62.5 scrub starts 2024-02-18T00:39:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.313622+0000 osd.4 (osd.4) 258 : cluster [DBG] 62.5 scrub ok 2024-02-18T00:39:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.686994+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e442: 8 total, 8 up, 7 in 2024-02-18T00:39:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:58.466679+0000 osd.5 (osd.5) 185 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:39:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:58.470134+0000 osd.5 (osd.5) 186 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:39:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:58.747489+0000 osd.6 (osd.6) 205 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:39:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:58.749671+0000 osd.6 (osd.6) 206 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:39:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.310168+0000 osd.4 (osd.4) 257 : cluster [DBG] 62.5 scrub starts 2024-02-18T00:39:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.313622+0000 osd.4 (osd.4) 258 : cluster [DBG] 62.5 scrub ok 2024-02-18T00:39:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.686994+0000 mon.a (mon.0) 2574 : cluster [DBG] osdmap e442: 8 total, 8 up, 7 in 2024-02-18T00:39:01.278 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:39:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:39:01.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.473117+0000 osd.5 (osd.5) 187 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:39:01.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.474121+0000 osd.5 (osd.5) 188 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:39:01.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.771451+0000 osd.3 (osd.3) 241 : cluster [DBG] 62.18 scrub starts 2024-02-18T00:39:01.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.775760+0000 osd.3 (osd.3) 242 : cluster [DBG] 62.18 scrub ok 2024-02-18T00:39:01.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:38:59.910438+0000 mgr.y (mgr.24454) 412 : cluster [DBG] pgmap v660: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.473117+0000 osd.5 (osd.5) 187 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:39:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.474121+0000 osd.5 (osd.5) 188 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:39:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.771451+0000 osd.3 (osd.3) 241 : cluster [DBG] 62.18 scrub starts 2024-02-18T00:39:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.775760+0000 osd.3 (osd.3) 242 : cluster [DBG] 62.18 scrub ok 2024-02-18T00:39:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:38:59.910438+0000 mgr.y (mgr.24454) 412 : cluster [DBG] pgmap v660: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:01.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.473117+0000 osd.5 (osd.5) 187 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:39:01.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.474121+0000 osd.5 (osd.5) 188 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:39:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.771451+0000 osd.3 (osd.3) 241 : cluster [DBG] 62.18 scrub starts 2024-02-18T00:39:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.775760+0000 osd.3 (osd.3) 242 : cluster [DBG] 62.18 scrub ok 2024-02-18T00:39:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:38:59.910438+0000 mgr.y (mgr.24454) 412 : cluster [DBG] pgmap v660: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:01.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:39:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:39:02.206 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:02.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:02.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:02.311 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:02.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:00.785414+0000 osd.6 (osd.6) 207 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:39:02.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:00.786465+0000 osd.6 (osd.6) 208 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:39:02.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:01.277185+0000 mgr.y (mgr.24454) 413 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:02.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:01.508734+0000 osd.2 (osd.2) 185 : cluster [DBG] 62.12 scrub starts 2024-02-18T00:39:02.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:01.510991+0000 osd.2 (osd.2) 186 : cluster [DBG] 62.12 scrub ok 2024-02-18T00:39:02.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:01.602511+0000 osd.0 (osd.0) 206 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:39:02.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:01.603547+0000 osd.0 (osd.0) 207 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:39:02.343 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:02.364 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:02.367 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:02.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:02.468 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:02.499 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:02.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:02] "GET /metrics HTTP/1.1" 200 187176 "" "Prometheus/2.33.4" 2024-02-18T00:39:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:00.785414+0000 osd.6 (osd.6) 207 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:39:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:00.786465+0000 osd.6 (osd.6) 208 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:39:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:01.277185+0000 mgr.y (mgr.24454) 413 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:01.508734+0000 osd.2 (osd.2) 185 : cluster [DBG] 62.12 scrub starts 2024-02-18T00:39:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:01.510991+0000 osd.2 (osd.2) 186 : cluster [DBG] 62.12 scrub ok 2024-02-18T00:39:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:01.602511+0000 osd.0 (osd.0) 206 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:39:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:01.603547+0000 osd.0 (osd.0) 207 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:39:02.503 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:00.785414+0000 osd.6 (osd.6) 207 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:39:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:00.786465+0000 osd.6 (osd.6) 208 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:39:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:01.277185+0000 mgr.y (mgr.24454) 413 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:01.508734+0000 osd.2 (osd.2) 185 : cluster [DBG] 62.12 scrub starts 2024-02-18T00:39:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:01.510991+0000 osd.2 (osd.2) 186 : cluster [DBG] 62.12 scrub ok 2024-02-18T00:39:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:01.602511+0000 osd.0 (osd.0) 206 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:39:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:01.603547+0000 osd.0 (osd.0) 207 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:39:02.520 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:02.522 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:02.523 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:02.561 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:39:02.562 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:39:02.623 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:02.652 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:02.671 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:02.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:02.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:02.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:02.808 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:02.828 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:02.831 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:02.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:02.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:02.961 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:03.111 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:01.511261+0000 osd.5 (osd.5) 189 : cluster [DBG] 25.c scrub starts 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:01.514187+0000 osd.5 (osd.5) 190 : cluster [DBG] 25.c scrub ok 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:01.911444+0000 mgr.y (mgr.24454) 414 : cluster [DBG] pgmap v661: 238 pgs: 1 active+clean+snaptrim, 237 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.376946+0000 osd.4 (osd.4) 259 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.380405+0000 osd.4 (osd.4) 260 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.512450+0000 osd.2 (osd.2) 187 : cluster [DBG] 62.a scrub starts 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.513912+0000 osd.2 (osd.2) 188 : cluster [DBG] 62.a scrub ok 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.633755+0000 osd.0 (osd.0) 208 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:39:03.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.635097+0000 osd.0 (osd.0) 209 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:01.511261+0000 osd.5 (osd.5) 189 : cluster [DBG] 25.c scrub starts 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:01.514187+0000 osd.5 (osd.5) 190 : cluster [DBG] 25.c scrub ok 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:01.911444+0000 mgr.y (mgr.24454) 414 : cluster [DBG] pgmap v661: 238 pgs: 1 active+clean+snaptrim, 237 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.376946+0000 osd.4 (osd.4) 259 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.380405+0000 osd.4 (osd.4) 260 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.512450+0000 osd.2 (osd.2) 187 : cluster [DBG] 62.a scrub starts 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.513912+0000 osd.2 (osd.2) 188 : cluster [DBG] 62.a scrub ok 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.633755+0000 osd.0 (osd.0) 208 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:39:03.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.635097+0000 osd.0 (osd.0) 209 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:39:03.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:03.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:03.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:03.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:03.289 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:03.289 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:03.390 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:01.511261+0000 osd.5 (osd.5) 189 : cluster [DBG] 25.c scrub starts 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:01.514187+0000 osd.5 (osd.5) 190 : cluster [DBG] 25.c scrub ok 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:01.911444+0000 mgr.y (mgr.24454) 414 : cluster [DBG] pgmap v661: 238 pgs: 1 active+clean+snaptrim, 237 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.376946+0000 osd.4 (osd.4) 259 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.380405+0000 osd.4 (osd.4) 260 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.512450+0000 osd.2 (osd.2) 187 : cluster [DBG] 62.a scrub starts 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.513912+0000 osd.2 (osd.2) 188 : cluster [DBG] 62.a scrub ok 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.633755+0000 osd.0 (osd.0) 208 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:39:03.416 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.635097+0000 osd.0 (osd.0) 209 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:39:03.418 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:03.437 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:03.439 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:03.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:03.540 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:03.568 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:03.587 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:03.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:03.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:03.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:03.718 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:03.737 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:03.739 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:03.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:03.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:03.870 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:03.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:39:04.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.595829+0000 osd.1 (osd.1) 223 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:39:04.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.598737+0000 osd.1 (osd.1) 224 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:39:04.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.672292+0000 osd.3 (osd.3) 243 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:39:04.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:02.673084+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:39:04.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.338034+0000 osd.4 (osd.4) 261 : cluster [DBG] 4.e scrub starts 2024-02-18T00:39:04.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.341498+0000 osd.4 (osd.4) 262 : cluster [DBG] 4.e scrub ok 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.502025+0000 osd.2 (osd.2) 189 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.503252+0000 osd.2 (osd.2) 190 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.686686+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e443: 8 total, 8 up, 7 in 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.595829+0000 osd.1 (osd.1) 223 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.598737+0000 osd.1 (osd.1) 224 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.672292+0000 osd.3 (osd.3) 243 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:02.673084+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.338034+0000 osd.4 (osd.4) 261 : cluster [DBG] 4.e scrub starts 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.341498+0000 osd.4 (osd.4) 262 : cluster [DBG] 4.e scrub ok 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.502025+0000 osd.2 (osd.2) 189 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:39:04.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.503252+0000 osd.2 (osd.2) 190 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:39:04.239 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.686686+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e443: 8 total, 8 up, 7 in 2024-02-18T00:39:04.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-18T00:39:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.595829+0000 osd.1 (osd.1) 223 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:39:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.598737+0000 osd.1 (osd.1) 224 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:39:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.672292+0000 osd.3 (osd.3) 243 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:39:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:02.673084+0000 osd.3 (osd.3) 244 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:39:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.338034+0000 osd.4 (osd.4) 261 : cluster [DBG] 4.e scrub starts 2024-02-18T00:39:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.341498+0000 osd.4 (osd.4) 262 : cluster [DBG] 4.e scrub ok 2024-02-18T00:39:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.502025+0000 osd.2 (osd.2) 189 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:39:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.503252+0000 osd.2 (osd.2) 190 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:39:04.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.686686+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e443: 8 total, 8 up, 7 in 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 -- 172.21.15.59:0/3489114102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 msgr2=0x7fc1880f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/3489114102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f6d40 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 rx=0x7fc170004870 tx=0x7fc170009cb0).stop 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 -- 172.21.15.59:0/3489114102 shutdown_connections 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/3489114102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1880fcbb0 0x7fc1880fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/3489114102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1880f7400 0x7fc188103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/3489114102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f6d40 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 -- 172.21.15.59:0/3489114102 >> 172.21.15.59:0/3489114102 conn(0x7fc1880f2820 msgr2=0x7fc1880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:04.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 -- 172.21.15.59:0/3489114102 shutdown_connections 2024-02-18T00:39:04.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 -- 172.21.15.59:0/3489114102 wait complete. 2024-02-18T00:39:04.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 librados: starting msgr at 2024-02-18T00:39:04.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.616+0000 7fc18cf0e700 1 librados: starting objecter 2024-02-18T00:39:04.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 Processor -- start 2024-02-18T00:39:04.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 -- start start 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 librados: setting wanted keys 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 librados: calling monclient init 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1880f7400 0x7fc1880f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1880fcbb0 0x7fc1880f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1881166b0 con 0x7fc1880fd510 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc188116820 con 0x7fc1880fcbb0 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18cf0e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1881169e0 con 0x7fc1880f7400 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1880f7400 0x7fc1880f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc186d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1880f7400 0x7fc1880f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43186/0 (socket says 172.21.15.59:43186) 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc186d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44880/0 (socket says 172.21.15.59:44880) 2024-02-18T00:39:04.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc18659c700 1 -- 172.21.15.59:0/2565218054 learned_addr learned my addr 172.21.15.59:0/2565218054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc17ffff700 1 --2- 172.21.15.59:0/2565218054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1880fcbb0 0x7fc1880f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc186d9d700 1 -- 172.21.15.59:0/2565218054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1880f7400 msgr2=0x7fc1880f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc186d9d700 1 --2- 172.21.15.59:0/2565218054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1880f7400 0x7fc1880f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc186d9d700 1 -- 172.21.15.59:0/2565218054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1880fcbb0 msgr2=0x7fc1880f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc186d9d700 1 --2- 172.21.15.59:0/2565218054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1880fcbb0 0x7fc1880f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.617+0000 7fc186d9d700 1 -- 172.21.15.59:0/2565218054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800097c0 con 0x7fc1880fd510 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc186d9d700 1 --2- 172.21.15.59:0/2565218054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f9390 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 rx=0x7fc180001cb0 tx=0x7fc180004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc17f7fe700 10 client.?.objecter ms_handle_connect 0x7fc1880fd510 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc17f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18000fa30 con 0x7fc1880fd510 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1800095a0 con 0x7fc1880fd510 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc170005040 con 0x7fc1880fd510 2024-02-18T00:39:04.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180007d50 con 0x7fc1880fd510 2024-02-18T00:39:04.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc18cf0e700 10 client.31380.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:04.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc188117020 con 0x7fc1880fd510 2024-02-18T00:39:04.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.618+0000 7fc18cf0e700 1 librados: init done 2024-02-18T00:39:04.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.619+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fc18805fa60 con 0x7fc1880fd510 2024-02-18T00:39:04.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc1800090d0 con 0x7fc1880fd510 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc174062c90 0x7fc174065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(443..443 src has 180..443) v4 ==== 8063+0+0 (secure 0 0 0) 0x7fc18007d990 con 0x7fc1880fd510 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 osd_map(443..443 src has 180..443) v4 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 3 client.31380.objecter handle_osd_map got epochs [443,443] > 0 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 3 client.31380.objecter handle_osd_map decoding full epoch 443 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc174062c90 0x7fc174065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 20 client.31380.objecter dump_active .. 0 homeless 2024-02-18T00:39:04.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.620+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v443) v1 ==== 54+0+16 (secure 0 0 0) 0x7fc180082220 con 0x7fc1880fd510 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 1 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc174062c90 0x7fc174065150 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 rx=0x7fc178004e80 tx=0x7fc17800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc174062c90 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc1880ffed0 0x7fc1881002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 1 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 0 1 2024-02-18T00:39:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18805ee80 osd=0 4 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 10 client.31380.objecter _send_command 1 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc188106120 con 0x7fc1880ffed0 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc17ffff700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc1880ffed0 0x7fc1881002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc17ffff700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc1880ffed0 0x7fc1881002f0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.621+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc1880ffed0 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.622+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc188106120 con 0x7fc1880ffed0 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.622+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.622+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 1 = system:0 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.622+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18805ee80 osd=0 4 2024-02-18T00:39:04.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.622+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 0 1 2024-02-18T00:39:04.638 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 2 2024-02-18T00:39:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 2 2024-02-18T00:39:04.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18805ee80 osd=0 3 2024-02-18T00:39:04.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 0 2 2024-02-18T00:39:04.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18805ee80 osd=0 4 2024-02-18T00:39:04.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 10 client.31380.objecter _send_command 2 2024-02-18T00:39:04.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.637+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc1880273c0 con 0x7fc1880ffed0 2024-02-18T00:39:04.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.639+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc1880273c0 con 0x7fc1880ffed0 2024-02-18T00:39:04.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.639+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.639+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.639+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18805ee80 osd=0 4 2024-02-18T00:39:04.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.639+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 0 2 2024-02-18T00:39:04.641 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:39:04.642 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:04.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:04.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 3 2024-02-18T00:39:04.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6805/26279110 conn(0x7fc1880598f0 0x7fc188059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 3 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 1 3 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc1880286e0 osd=1 4 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 10 client.31380.objecter _send_command 3 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc18811d990 con 0x7fc1880598f0 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6805/26279110 conn(0x7fc1880598f0 0x7fc188059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc1880598f0 2024-02-18T00:39:04.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.640+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6805/26279110 conn(0x7fc1880598f0 0x7fc188059d10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.641+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc18811d990 con 0x7fc1880598f0 2024-02-18T00:39:04.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.641+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:39:04.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.641+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 3 = system:0 2024-02-18T00:39:04.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.641+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc1880286e0 osd=1 4 2024-02-18T00:39:04.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.641+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 1 3 2024-02-18T00:39:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 4 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 4 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc1880286e0 osd=1 3 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 1 4 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc1880286e0 osd=1 4 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 10 client.31380.objecter _send_command 4 2024-02-18T00:39:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.658+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc188059360 con 0x7fc1880598f0 2024-02-18T00:39:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.659+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc188059360 con 0x7fc1880598f0 2024-02-18T00:39:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.659+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.659+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.659+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc1880286e0 osd=1 4 2024-02-18T00:39:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.659+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 1 4 2024-02-18T00:39:04.662 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:39:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 5 2024-02-18T00:39:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc188028240 0x7fc18811c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 5 2024-02-18T00:39:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 2 5 2024-02-18T00:39:04.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188059770 osd=2 4 2024-02-18T00:39:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 10 client.31380.objecter _send_command 5 2024-02-18T00:39:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc18811c8a0 con 0x7fc188028240 2024-02-18T00:39:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc17ffff700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc188028240 0x7fc18811c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc188028240 2024-02-18T00:39:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.660+0000 7fc17ffff700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc188028240 0x7fc18811c3b0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.661+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc18811c8a0 con 0x7fc188028240 2024-02-18T00:39:04.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.661+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:39:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.661+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 5 = system:0 2024-02-18T00:39:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.661+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc188059770 osd=2 4 2024-02-18T00:39:04.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.661+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 2 5 2024-02-18T00:39:04.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 6 2024-02-18T00:39:04.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 6 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc188059770 osd=2 3 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 2 6 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188059770 osd=2 4 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 10 client.31380.objecter _send_command 6 2024-02-18T00:39:04.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.678+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc1880277a0 con 0x7fc188028240 2024-02-18T00:39:04.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc1880277a0 con 0x7fc188028240 2024-02-18T00:39:04.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc188059770 osd=2 4 2024-02-18T00:39:04.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 2 6 2024-02-18T00:39:04.682 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 7 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc18811fb10 0x7fc188121f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 7 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 3 7 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188027bb0 osd=3 4 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 10 client.31380.objecter _send_command 7 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc188124b50 con 0x7fc18811fb10 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc186d9d700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc18811fb10 0x7fc188121f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc18811fb10 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.680+0000 7fc186d9d700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc18811fb10 0x7fc188121f00 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.681+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc188124b50 con 0x7fc18811fb10 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.681+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.681+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 7 = system:0 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.681+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc188027bb0 osd=3 4 2024-02-18T00:39:04.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.681+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 3 7 2024-02-18T00:39:04.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 8 2024-02-18T00:39:04.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 8 2024-02-18T00:39:04.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc188027bb0 osd=3 3 2024-02-18T00:39:04.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 3 8 2024-02-18T00:39:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188027bb0 osd=3 4 2024-02-18T00:39:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 10 client.31380.objecter _send_command 8 2024-02-18T00:39:04.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.696+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc18811cd00 con 0x7fc18811fb10 2024-02-18T00:39:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc18811cd00 con 0x7fc18811fb10 2024-02-18T00:39:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc188027bb0 osd=3 4 2024-02-18T00:39:04.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 3 8 2024-02-18T00:39:04.702 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:39:04.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 9 2024-02-18T00:39:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc18811bf10 0x7fc188123570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 9 2024-02-18T00:39:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 4 9 2024-02-18T00:39:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cd00 osd=4 4 2024-02-18T00:39:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 10 client.31380.objecter _send_command 9 2024-02-18T00:39:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc18811cff0 con 0x7fc18811bf10 2024-02-18T00:39:04.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.698+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc18811bf10 0x7fc188123570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.699+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc18811bf10 2024-02-18T00:39:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.699+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc18811bf10 0x7fc188123570 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.700+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc18811cff0 con 0x7fc18811bf10 2024-02-18T00:39:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.700+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:39:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.700+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 9 = system:0 2024-02-18T00:39:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.700+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18811cd00 osd=4 4 2024-02-18T00:39:04.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.700+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 4 9 2024-02-18T00:39:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 10 2024-02-18T00:39:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 10 2024-02-18T00:39:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18811cd00 osd=4 3 2024-02-18T00:39:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 4 10 2024-02-18T00:39:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cd00 osd=4 4 2024-02-18T00:39:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 10 client.31380.objecter _send_command 10 2024-02-18T00:39:04.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.718+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc18811cff0 con 0x7fc18811bf10 2024-02-18T00:39:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc18811cff0 con 0x7fc18811bf10 2024-02-18T00:39:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18811cd00 osd=4 4 2024-02-18T00:39:04.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 4 10 2024-02-18T00:39:04.723 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:39:04.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 11 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc188123f70 0x7fc1881243b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 11 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 5 11 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cff0 osd=5 4 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 10 client.31380.objecter _send_command 11 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.720+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1881249d0 con 0x7fc188123f70 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.721+0000 7fc17ffff700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc188123f70 0x7fc1881243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.721+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc188123f70 2024-02-18T00:39:04.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.721+0000 7fc17ffff700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc188123f70 0x7fc1881243b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.722+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc17001a040 con 0x7fc188123f70 2024-02-18T00:39:04.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.722+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:39:04.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.722+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 11 = system:0 2024-02-18T00:39:04.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.722+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18811cff0 osd=5 4 2024-02-18T00:39:04.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.722+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 5 11 2024-02-18T00:39:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 12 2024-02-18T00:39:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 12 2024-02-18T00:39:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18811cff0 osd=5 3 2024-02-18T00:39:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 5 12 2024-02-18T00:39:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cff0 osd=5 4 2024-02-18T00:39:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 10 client.31380.objecter _send_command 12 2024-02-18T00:39:04.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.737+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc188128ce0 con 0x7fc188123f70 2024-02-18T00:39:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc188128ce0 con 0x7fc188123f70 2024-02-18T00:39:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18811cff0 osd=5 4 2024-02-18T00:39:04.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 5 12 2024-02-18T00:39:04.741 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 13 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc18812aae0 0x7fc18812af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 13 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 6 13 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812a460 osd=6 4 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 10 client.31380.objecter _send_command 13 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc18812e6b0 con 0x7fc18812aae0 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.739+0000 7fc186d9d700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc18812aae0 0x7fc18812af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.740+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc18812aae0 2024-02-18T00:39:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.740+0000 7fc186d9d700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc18812aae0 0x7fc18812af40 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.740+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc18812e6b0 con 0x7fc18812aae0 2024-02-18T00:39:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.740+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:39:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.740+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 13 = system:0 2024-02-18T00:39:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.740+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18812a460 osd=6 4 2024-02-18T00:39:04.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.740+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 6 13 2024-02-18T00:39:04.743 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:04.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 14 2024-02-18T00:39:04.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 14 2024-02-18T00:39:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18812a460 osd=6 3 2024-02-18T00:39:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 6 14 2024-02-18T00:39:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812a460 osd=6 4 2024-02-18T00:39:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 10 client.31380.objecter _send_command 14 2024-02-18T00:39:04.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.755+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc18812e6b0 con 0x7fc18812aae0 2024-02-18T00:39:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc18812e6b0 con 0x7fc18812aae0 2024-02-18T00:39:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18812a460 osd=6 4 2024-02-18T00:39:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 6 14 2024-02-18T00:39:04.759 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:39:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 15 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6812/53025171 conn(0x7fc18812daa0 0x7fc18812df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 15 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 7 15 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812e6b0 osd=7 4 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 10 client.31380.objecter _send_command 15 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc188131a60 con 0x7fc18812daa0 2024-02-18T00:39:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.757+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6812/53025171 conn(0x7fc18812daa0 0x7fc18812df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.758+0000 7fc17f7fe700 10 client.31380.objecter ms_handle_connect 0x7fc18812daa0 2024-02-18T00:39:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.758+0000 7fc18659c700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6812/53025171 conn(0x7fc18812daa0 0x7fc18812df00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.758+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc188131a60 con 0x7fc18812daa0 2024-02-18T00:39:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.758+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:39:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.758+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 15 = system:0 2024-02-18T00:39:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.758+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18812e6b0 osd=7 4 2024-02-18T00:39:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.758+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 7 15 2024-02-18T00:39:04.772 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:04.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 10 client.31380.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:04.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign -1 16 2024-02-18T00:39:04.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 20 client.31380.objecter _get_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_remove -1 16 2024-02-18T00:39:04.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 20 client.31380.objecter get_session s=0x7fc18812e6b0 osd=7 3 2024-02-18T00:39:04.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 15 client.31380.objecter _session_command_op_assign 7 16 2024-02-18T00:39:04.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812e6b0 osd=7 4 2024-02-18T00:39:04.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 10 client.31380.objecter _send_command 16 2024-02-18T00:39:04.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.773+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fc188128da0 con 0x7fc18812daa0 2024-02-18T00:39:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc17f7fe700 1 -- 172.21.15.59:0/2565218054 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fc188128da0 con 0x7fc18812daa0 2024-02-18T00:39:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc17f7fe700 10 client.31380.objecter ms_dispatch 0x7fc188001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc17f7fe700 10 client.31380.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc17f7fe700 20 client.31380.objecter put_session s=0x7fc18812e6b0 osd=7 4 2024-02-18T00:39:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc17f7fe700 15 client.31380.objecter _session_command_op_remove 7 16 2024-02-18T00:39:04.778 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:39:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 librados: watch_flush enter 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 librados: watch_flush exit 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.0 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc1880ffed0 msgr2=0x7fc1881002f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc1880ffed0 0x7fc1881002f0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18805ee80 osd=0 1 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.1 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6805/26279110 conn(0x7fc1880598f0 msgr2=0x7fc188059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6805/26279110 conn(0x7fc1880598f0 0x7fc188059d10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc1880286e0 osd=1 1 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.2 2024-02-18T00:39:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc188028240 msgr2=0x7fc18811c3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc188028240 0x7fc18811c3b0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188059770 osd=2 1 2024-02-18T00:39:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.3 2024-02-18T00:39:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc18811fb10 msgr2=0x7fc188121f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc18811fb10 0x7fc188121f00 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc186d9d700 1 -- 172.21.15.59:0/2565218054 reap_dead start 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc188027bb0 osd=3 1 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.4 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc18811bf10 msgr2=0x7fc188123570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc18811bf10 0x7fc188123570 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cd00 osd=4 1 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.5 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc188123f70 msgr2=0x7fc1881243b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc188123f70 0x7fc1881243b0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18811cff0 osd=5 1 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.6 2024-02-18T00:39:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc18812aae0 msgr2=0x7fc18812af40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc18812aae0 0x7fc18812af40 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812a460 osd=6 1 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter close_session for osd.7 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6812/53025171 conn(0x7fc18812daa0 msgr2=0x7fc18812df00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.137:6812/53025171 conn(0x7fc18812daa0 0x7fc18812df00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter put_session s=0x7fc18812e6b0 osd=7 1 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 20 client.31380.objecter shutdown clearing up homeless session... 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 10 client.31380.objecter successfully canceled tick 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc186d9d700 1 -- 172.21.15.59:0/2565218054 reap_dead start 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc174062c90 msgr2=0x7fc174065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc174062c90 0x7fc174065150 secure :-1 s=READY pgs=3117 cs=0 l=1 rev1=1 rx=0x7fc178004e80 tx=0x7fc17800b040).stop 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 msgr2=0x7fc1880f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.775+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f9390 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 rx=0x7fc180001cb0 tx=0x7fc180004c50).stop 2024-02-18T00:39:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.776+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 shutdown_connections 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.776+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc174062c90 0x7fc174065150 unknown :-1 s=CLOSED pgs=3117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.776+0000 7fc18cf0e700 1 --2- 172.21.15.59:0/2565218054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1880fd510 0x7fc1880f9390 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.776+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 >> 172.21.15.59:0/2565218054 conn(0x7fc1880f2820 msgr2=0x7fc188060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.776+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 shutdown_connections 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.776+0000 7fc18cf0e700 1 -- 172.21.15.59:0/2565218054 wait complete. 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:04.776+0000 7fc18cf0e700 1 librados: shutdown 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:39:04.793 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:04.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:04.796 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:04.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:04.928 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:04.947 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:04.949 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:04.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:05.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:05.079 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:05.202 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:05.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:05.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:05.308 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:05.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.661940+0000 osd.3 (osd.3) 245 : cluster [DBG] 62.15 scrub starts 2024-02-18T00:39:05.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.663863+0000 osd.3 (osd.3) 246 : cluster [DBG] 62.15 scrub ok 2024-02-18T00:39:05.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:03.911883+0000 mgr.y (mgr.24454) 415 : cluster [DBG] pgmap v663: 238 pgs: 1 active+clean+snaptrim, 237 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-18T00:39:05.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:04.621419+0000 mon.a (mon.0) 2576 : audit [DBG] from='client.? 172.21.15.59:0/2565218054' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:05.336 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:05.355 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:05.359 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:05.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:05.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:05.488 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.661940+0000 osd.3 (osd.3) 245 : cluster [DBG] 62.15 scrub starts 2024-02-18T00:39:05.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.663863+0000 osd.3 (osd.3) 246 : cluster [DBG] 62.15 scrub ok 2024-02-18T00:39:05.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:03.911883+0000 mgr.y (mgr.24454) 415 : cluster [DBG] pgmap v663: 238 pgs: 1 active+clean+snaptrim, 237 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-18T00:39:05.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:04.621419+0000 mon.a (mon.0) 2576 : audit [DBG] from='client.? 172.21.15.59:0/2565218054' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:05.489 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.661940+0000 osd.3 (osd.3) 245 : cluster [DBG] 62.15 scrub starts 2024-02-18T00:39:05.489 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.663863+0000 osd.3 (osd.3) 246 : cluster [DBG] 62.15 scrub ok 2024-02-18T00:39:05.489 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:03.911883+0000 mgr.y (mgr.24454) 415 : cluster [DBG] pgmap v663: 238 pgs: 1 active+clean+snaptrim, 237 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-18T00:39:05.489 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:04.621419+0000 mon.a (mon.0) 2576 : audit [DBG] from='client.? 172.21.15.59:0/2565218054' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:05.491 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:05.513 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:05.515 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:05.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:05.617 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:05.645 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:05.663 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:05.666 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:05.666 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:05.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 -- 172.21.15.59:0/1011013654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c0f8de0 msgr2=0x7f228c0f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 --2- 172.21.15.59:0/1011013654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c0f8de0 0x7f228c0f91c0 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 rx=0x7f2288004870 tx=0x7f2288009cb0).stop 2024-02-18T00:39:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 -- 172.21.15.59:0/1011013654 shutdown_connections 2024-02-18T00:39:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 --2- 172.21.15.59:0/1011013654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c102180 0x7f228c104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 --2- 172.21.15.59:0/1011013654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c0f8de0 0x7f228c0f91c0 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 --2- 172.21.15.59:0/1011013654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f228c0f96b0 0x7f228c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 -- 172.21.15.59:0/1011013654 >> 172.21.15.59:0/1011013654 conn(0x7f228c0f26f0 msgr2=0x7f228c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 -- 172.21.15.59:0/1011013654 shutdown_connections 2024-02-18T00:39:05.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.782+0000 7f2294414700 1 -- 172.21.15.59:0/1011013654 wait complete. 2024-02-18T00:39:05.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 librados: starting msgr at 2024-02-18T00:39:05.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 librados: starting objecter 2024-02-18T00:39:05.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 Processor -- start 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 -- start start 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 librados: setting wanted keys 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 librados: calling monclient init 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f228c0f8de0 0x7f228c0f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c0f96b0 0x7f228c0f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.783+0000 7f2294414700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102180 0x7f228c0f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f2294414700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c116730 con 0x7f228c102180 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f2294414700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c1168a0 con 0x7f228c0f96b0 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f2294414700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c116a60 con 0x7f228c0f8de0 2024-02-18T00:39:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102180 0x7f228c0f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102180 0x7f228c0f75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44898/0 (socket says 172.21.15.59:44898) 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 -- 172.21.15.59:0/2091219458 learned_addr learned my addr 172.21.15.59:0/2091219458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22921b0700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f228c0f8de0 0x7f228c0f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22919af700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c0f96b0 0x7f228c0f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 -- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f228c0f8de0 msgr2=0x7f228c0f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f228c0f8de0 0x7f228c0f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 -- 172.21.15.59:0/2091219458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c0f96b0 msgr2=0x7f228c0f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c0f96b0 0x7f228c0f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 -- 172.21.15.59:0/2091219458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288005040 con 0x7f228c102180 2024-02-18T00:39:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22919af700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c0f96b0 0x7f228c0f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22929b1700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102180 0x7f228c0f75b0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 rx=0x7f228400db80 tx=0x7f2284004ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22837fe700 10 client.?.objecter ms_handle_connect 0x7f228c102180 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22837fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22837fe700 1 -- 172.21.15.59:0/2091219458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228400b750 con 0x7f228c102180 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22837fe700 1 -- 172.21.15.59:0/2091219458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f228400f040 con 0x7f228c102180 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c116d40 con 0x7f228c102180 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f22837fe700 1 -- 172.21.15.59:0/2091219458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284013b00 con 0x7f228c102180 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f2294414700 10 client.31386.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c117230 con 0x7f228c102180 2024-02-18T00:39:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.784+0000 7f2294414700 1 librados: init done 2024-02-18T00:39:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.785+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c05ee80 con 0x7f228c102180 2024-02-18T00:39:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.786+0000 7f22837fe700 1 -- 172.21.15.59:0/2091219458 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f228400b8b0 con 0x7f228c102180 2024-02-18T00:39:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.786+0000 7f22837fe700 1 --2- 172.21.15.59:0/2091219458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2278062c90 0x7f2278065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22921b0700 1 --2- 172.21.15.59:0/2091219458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2278062c90 0x7f2278065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22837fe700 1 -- 172.21.15.59:0/2091219458 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(443..443 src has 180..443) v4 ==== 8063+0+0 (secure 0 0 0) 0x7f228407ce80 con 0x7f228c102180 2024-02-18T00:39:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22837fe700 10 client.31386.objecter ms_dispatch 0x7f228c001550 osd_map(443..443 src has 180..443) v4 2024-02-18T00:39:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22837fe700 3 client.31386.objecter handle_osd_map got epochs [443,443] > 0 2024-02-18T00:39:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22837fe700 3 client.31386.objecter handle_osd_map decoding full epoch 443 2024-02-18T00:39:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22837fe700 20 client.31386.objecter dump_active .. 0 homeless 2024-02-18T00:39:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22837fe700 10 client.31386.objecter ms_handle_connect 0x7f2278062c90 2024-02-18T00:39:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.787+0000 7f22921b0700 1 --2- 172.21.15.59:0/2091219458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2278062c90 0x7f2278065150 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 rx=0x7f2288005010 tx=0x7f2288016040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:05.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.791+0000 7f22837fe700 1 -- 172.21.15.59:0/2091219458 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2284050330 con 0x7f228c102180 2024-02-18T00:39:05.798 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:05.818 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:05.820 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:05.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:05.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:05.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:05.979+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7f228c0f80b0 con 0x7f228c102180 2024-02-18T00:39:06.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.023+0000 7f22837fe700 1 -- 172.21.15.59:0/2091219458 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v444) v1 ==== 113+0+0 (secure 0 0 0) 0x7f2284018070 con 0x7f228c102180 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.024+0000 7f2294414700 10 librados: watch_flush enter 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 10 librados: watch_flush exit 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 20 client.31386.objecter shutdown clearing up homeless session... 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 10 client.31386.objecter successfully canceled tick 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2278062c90 msgr2=0x7f2278065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 --2- 172.21.15.59:0/2091219458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2278062c90 0x7f2278065150 secure :-1 s=READY pgs=3118 cs=0 l=1 rev1=1 rx=0x7f2288005010 tx=0x7f2288016040).stop 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102180 msgr2=0x7f228c0f75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102180 0x7f228c0f75b0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 rx=0x7f228400db80 tx=0x7f2284004ea0).stop 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 shutdown_connections 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102180 0x7f228c0f75b0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f228c0f8de0 0x7f228c0f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:06.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 --2- 172.21.15.59:0/2091219458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2278062c90 0x7f2278065150 unknown :-1 s=CLOSED pgs=3118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 --2- 172.21.15.59:0/2091219458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f228c0f96b0 0x7f228c0f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 >> 172.21.15.59:0/2091219458 conn(0x7f228c0f26f0 msgr2=0x7f228c0f32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 shutdown_connections 2024-02-18T00:39:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 -- 172.21.15.59:0/2091219458 wait complete. 2024-02-18T00:39:06.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:06.025+0000 7f2294414700 1 librados: shutdown 2024-02-18T00:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.6 to 1 (10000) 2024-02-18T00:39:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:06.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:04.583080+0000 osd.1 (osd.1) 225 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:39:06.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:04.585953+0000 osd.1 (osd.1) 226 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:39:06.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:05.980660+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? 172.21.15.59:0/2091219458' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:39:06.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:06.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:06.052 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:04.583080+0000 osd.1 (osd.1) 225 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:39:06.052 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:04.585953+0000 osd.1 (osd.1) 226 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:39:06.052 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:05.980660+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? 172.21.15.59:0/2091219458' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:39:06.153 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:06.180 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:06.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:04.583080+0000 osd.1 (osd.1) 225 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:39:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:04.585953+0000 osd.1 (osd.1) 226 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:39:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:05.980660+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? 172.21.15.59:0/2091219458' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:39:07.030 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:07.033 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:07.033 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:07.134 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:07.166 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:07.187 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:07.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:07.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:05.912762+0000 mgr.y (mgr.24454) 416 : cluster [DBG] pgmap v664: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-18T00:39:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:06.024195+0000 mon.a (mon.0) 2578 : audit [INF] from='client.? 172.21.15.59:0/2091219458' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:39:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:06.024239+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e444: 8 total, 8 up, 7 in 2024-02-18T00:39:07.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:05.912762+0000 mgr.y (mgr.24454) 416 : cluster [DBG] pgmap v664: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-18T00:39:07.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:06.024195+0000 mon.a (mon.0) 2578 : audit [INF] from='client.? 172.21.15.59:0/2091219458' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:39:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:06.024239+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e444: 8 total, 8 up, 7 in 2024-02-18T00:39:07.291 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:07.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:05.912762+0000 mgr.y (mgr.24454) 416 : cluster [DBG] pgmap v664: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-18T00:39:07.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:06.024195+0000 mon.a (mon.0) 2578 : audit [INF] from='client.? 172.21.15.59:0/2091219458' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:39:07.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:06.024239+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e444: 8 total, 8 up, 7 in 2024-02-18T00:39:07.321 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:07.342 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:07.345 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:07.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:07.446 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:07.474 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:07.494 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:07.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:07.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:07.569 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:07.595 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-18T00:39:07.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:39:07.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:07.611 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:07.890 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:07.892 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:07.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:07.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:08.021 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:08.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:06.561481+0000 osd.5 (osd.5) 191 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:39:08.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:06.562562+0000 osd.5 (osd.5) 192 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:39:08.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:06.695333+0000 osd.3 (osd.3) 247 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:39:08.248 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:06.697311+0000 osd.3 (osd.3) 248 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:39:08.248 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:06.775395+0000 osd.6 (osd.6) 209 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:39:08.248 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:06.776393+0000 osd.6 (osd.6) 210 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:39:08.248 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:07.343546+0000 osd.4 (osd.4) 263 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:39:08.248 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:07.344480+0000 osd.4 (osd.4) 264 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:39:08.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:06.561481+0000 osd.5 (osd.5) 191 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:39:08.248 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:06.562562+0000 osd.5 (osd.5) 192 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:39:08.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:06.695333+0000 osd.3 (osd.3) 247 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:39:08.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:06.697311+0000 osd.3 (osd.3) 248 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:39:08.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:06.775395+0000 osd.6 (osd.6) 209 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:39:08.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:06.776393+0000 osd.6 (osd.6) 210 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:39:08.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:07.343546+0000 osd.4 (osd.4) 263 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:39:08.249 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:07.344480+0000 osd.4 (osd.4) 264 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:06.561481+0000 osd.5 (osd.5) 191 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:06.562562+0000 osd.5 (osd.5) 192 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:06.695333+0000 osd.3 (osd.3) 247 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:06.697311+0000 osd.3 (osd.3) 248 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:06.775395+0000 osd.6 (osd.6) 209 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:06.776393+0000 osd.6 (osd.6) 210 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:07.343546+0000 osd.4 (osd.4) 263 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:39:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:07.344480+0000 osd.4 (osd.4) 264 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:39:08.969 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:39:09.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:07.651950+0000 osd.3 (osd.3) 249 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:39:09.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:07.653629+0000 osd.3 (osd.3) 250 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:39:09.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:07.913141+0000 mgr.y (mgr.24454) 417 : cluster [DBG] pgmap v666: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-18T00:39:09.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:08.305766+0000 osd.4 (osd.4) 265 : cluster [DBG] 62.f scrub starts 2024-02-18T00:39:09.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:08.309288+0000 osd.4 (osd.4) 266 : cluster [DBG] 62.f scrub ok 2024-02-18T00:39:09.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:08.688346+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e445: 8 total, 8 up, 7 in 2024-02-18T00:39:09.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:07.651950+0000 osd.3 (osd.3) 249 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:39:09.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:07.653629+0000 osd.3 (osd.3) 250 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:39:09.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:07.913141+0000 mgr.y (mgr.24454) 417 : cluster [DBG] pgmap v666: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-18T00:39:09.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:08.305766+0000 osd.4 (osd.4) 265 : cluster [DBG] 62.f scrub starts 2024-02-18T00:39:09.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:08.309288+0000 osd.4 (osd.4) 266 : cluster [DBG] 62.f scrub ok 2024-02-18T00:39:09.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:08.688346+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e445: 8 total, 8 up, 7 in 2024-02-18T00:39:09.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:07.651950+0000 osd.3 (osd.3) 249 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:39:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:07.653629+0000 osd.3 (osd.3) 250 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:39:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:07.913141+0000 mgr.y (mgr.24454) 417 : cluster [DBG] pgmap v666: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-18T00:39:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:08.305766+0000 osd.4 (osd.4) 265 : cluster [DBG] 62.f scrub starts 2024-02-18T00:39:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:08.309288+0000 osd.4 (osd.4) 266 : cluster [DBG] 62.f scrub ok 2024-02-18T00:39:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:08.688346+0000 mon.a (mon.0) 2580 : cluster [DBG] osdmap e445: 8 total, 8 up, 7 in 2024-02-18T00:39:09.734 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:10.073 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:10.076 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:10.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:10.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:10.209 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:10.429 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:09.586618+0000 osd.0 (osd.0) 210 : cluster [DBG] 25.a scrub starts 2024-02-18T00:39:10.429 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:09.589597+0000 osd.0 (osd.0) 211 : cluster [DBG] 25.a scrub ok 2024-02-18T00:39:10.429 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:09.706293+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:10.429 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:09.706703+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:10.429 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:09.707375+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-63"}]: dispatch 2024-02-18T00:39:10.429 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:09.707650+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-63"}]: dispatch 2024-02-18T00:39:10.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:09.586618+0000 osd.0 (osd.0) 210 : cluster [DBG] 25.a scrub starts 2024-02-18T00:39:10.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:09.589597+0000 osd.0 (osd.0) 211 : cluster [DBG] 25.a scrub ok 2024-02-18T00:39:10.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:09.706293+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:10.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:09.706703+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:10.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:09.707375+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-63"}]: dispatch 2024-02-18T00:39:10.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:09.707650+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-63"}]: dispatch 2024-02-18T00:39:10.445 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24818.0:6 27.11 27:8dd16f86:::obj3:head [stat] snapc 0=[] ondisk+read+known_if_redirected e107)", 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 367.483558458, 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7480210029999999, 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24818", 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1844909298", 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921757+0000", 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2109999999999997e-06 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921762+0000", 2024-02-18T00:39:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9610000000000005e-06 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921770+0000", 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0260000000000002e-06 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921777+0000", 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4104000000000003e-05 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921821+0000", 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921827+0000", 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7478305119999999 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.669658+0000", 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012113200000000001 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.669779+0000", 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7480210029999999 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:2061 17.4 17:238a5d4d:test-rados-api-smithi059-82949-16::big.copy:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 366.66335273200002, 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.082181671999999997, 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2061 2024-02-18T00:39:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741963+0000", 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2389999999999999e-06 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741969+0000", 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0874e-05 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741980+0000", 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1250000000000001e-06 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741985+0000", 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0633999999999998e-05 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.742025+0000", 2024-02-18T00:39:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0579000000000001e-05 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.742116+0000", 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069955422000000003 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,4", 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812071+0000", 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011236616 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.823308+0000", 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026764999999999997 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.823576+0000", 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000464523 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824040+0000", 2024-02-18T00:39:10.457 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.995e-05 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824100+0000", 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5300000000000003e-05 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824146+0000", 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.082181671999999997 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.458 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2070 17.15 e108/104 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head v 108'1, mlcod=0'0)", 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 366.26849392499997, 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070392237999999996, 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.459 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136822+0000", 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044622728 2024-02-18T00:39:10.460 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181444+0000", 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5222e-05 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181469+0000", 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0170000000000002e-06 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.461 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181478+0000", 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.237e-05 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181551+0000", 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9718999999999999e-05 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.462 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181571+0000", 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025600378999999999 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.207171+0000", 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3922999999999999e-05 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.207215+0000", 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070392237999999996 2024-02-18T00:39:10.463 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.25025.0:40 22.c e108/105 22:33eb54d8:::39.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 366.13626145699999, 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032619941, 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.464 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.465 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269053+0000", 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7693000000000002e-05 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269071+0000", 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9028000000000001e-05 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269090+0000", 2024-02-18T00:39:10.466 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5519999999999999e-06 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269097+0000", 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0609999999999998e-05 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269148+0000", 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1269000000000001e-05 2024-02-18T00:39:10.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269189+0000", 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032451386999999998 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.301640+0000", 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4557000000000001e-05 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.468 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.301675+0000", 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032619941 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(24.2 epoch 109/105 rep_tid 111 entries 109'2 (0'0) error 24:4061fbc0:test-rados-api-smithi059-82933-5::foo:head by client.24806.0:10 0.000000 -17 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 365.83743040799999, 2024-02-18T00:39:10.469 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051717428000000003, 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.470 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0070000000000003e-06 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567895+0000", 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1107999999999999e-05 2024-02-18T00:39:10.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567906+0000", 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0720000000000006e-06 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567915+0000", 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023667508 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.472 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.591582+0000", 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028021134999999999 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.619604+0000", 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051717428000000003 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.473 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.26167.0:11 66.17 66:e89f79a2:test-rados-api-smithi059-82981-9::foo:head [write 0~128] snapc 6=[6] ondisk+write+known_if_redirected e131)", 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 341.09912916500002, 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062117686999999998, 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.26167", 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.474 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.475 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306186+0000", 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6010000000000001e-06 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306195+0000", 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2559e-05 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306208+0000", 2024-02-18T00:39:10.476 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6709999999999999e-06 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306215+0000", 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019698599999999999 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306412+0000", 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8329999999999996e-06 2024-02-18T00:39:10.477 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306419+0000", 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012823485000000001 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.319243+0000", 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012152899999999999 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.478 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.319364+0000", 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010919998 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.330284+0000", 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011843823 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.479 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.342128+0000", 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2046000000000003e-05 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.342180+0000", 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026025063000000001 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368205+0000", 2024-02-18T00:39:10.480 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5210000000000002e-05 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368260+0000", 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4642999999999999e-05 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368305+0000", 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062117686999999998 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.481 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.29425.0:2 73.6 73:602f83fe:::foo:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 341.09770349199999, 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.075318273000000005, 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.29425", 2024-02-18T00:39:10.482 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/4028720008", 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:39:10.483 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307611+0000", 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0359999999999995e-06 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307620+0000", 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1882000000000001e-05 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.484 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307632+0000", 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7689999999999996e-06 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307640+0000", 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030238405999999999 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.485 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.337878+0000", 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6550000000000004e-06 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.337886+0000", 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028640344000000002 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366526+0000", 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000118879 2024-02-18T00:39:10.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366645+0000", 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014485400000000001 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366790+0000", 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003334025 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:10.487 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.370124+0000", 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6338999999999997e-05 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.370181+0000", 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012666067 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382847+0000", 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6785e-05 2024-02-18T00:39:10.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382893+0000", 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7939999999999999e-05 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382931+0000", 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.075318273000000005 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.489 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.26167.0:16 66.17 66:e89f79a2:test-rados-api-smithi059-82981-9::foo:head [write 128~128] snapc 7=[7,6] ondisk+write+known_if_redirected e132)", 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 340.08564607800002, 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059589835000000001, 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.26167", 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 16 2024-02-18T00:39:10.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.491 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319668+0000", 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1216000000000001e-05 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319679+0000", 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010448810000000001 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.492 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320724+0000", 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0662999999999999e-05 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320735+0000", 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4954999999999998e-05 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320790+0000", 2024-02-18T00:39:10.493 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320798+0000", 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021397759999999998 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342195+0000", 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4319000000000003e-05 2024-02-18T00:39:10.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342290+0000", 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000201516 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342491+0000", 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0097928909999999997 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.495 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.352284+0000", 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013119486 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.365404+0000", 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013735841 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.496 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379139+0000", 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3324999999999993e-05 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379213+0000", 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7633999999999998e-05 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379260+0000", 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059589835000000001 2024-02-18T00:39:10.497 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.7.0:212 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 335.88196587800002, 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083227508000000006, 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.499 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523349+0000", 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064971570000000006 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588321+0000", 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2592999999999999e-05 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588343+0000", 2024-02-18T00:39:10.500 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4349999999999997e-06 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588352+0000", 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000863093 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.589215+0000", 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6400000000000001e-05 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.501 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.589241+0000", 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017286283 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.606528+0000", 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0519000000000002e-05 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.502 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.606578+0000", 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083227508000000006 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.7.0:214 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 335.80505756500003, 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047186580999999998, 2024-02-18T00:39:10.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:39:10.504 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600257+0000", 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024532449000000001 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624790+0000", 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9515000000000001e-05 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.505 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624819+0000", 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0849000000000001e-05 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624830+0000", 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025507999999999998 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.506 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.625085+0000", 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4088e-05 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.625109+0000", 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022278747000000002 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.647388+0000", 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7408999999999999e-05 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.507 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.647446+0000", 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047186580999999998 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42808.0:2 99.6 99:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e156)", 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:39:10.508 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 315.82394215099998, 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041101081999999997, 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42808", 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3195038165", 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:39:10.509 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581373+0000", 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4339999999999992e-06 2024-02-18T00:39:10.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581381+0000", 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8351000000000001e-05 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581399+0000", 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3760000000000008e-06 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.511 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581408+0000", 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2623e-05 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581470+0000", 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.43e-06 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:39:10.512 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581478+0000", 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013746548000000001 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.595224+0000", 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2150000000000004e-06 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.595232+0000", 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021220474 2024-02-18T00:39:10.513 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616452+0000", 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106791 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616559+0000", 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000140035 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.514 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,4", 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616699+0000", 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001518248 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.618217+0000", 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015478130000000001 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.619765+0000", 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026234829999999998 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622388+0000", 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9582000000000001e-05 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622438+0000", 2024-02-18T00:39:10.516 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7565999999999999e-05 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622475+0000", 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041101081999999997 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.517 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42916.0:1 114.5s0 114:a1464ac0:test-rados-api-smithi059-82981-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e176)", 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 295.013118722, 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996, 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42916", 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2278224535", 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.518 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.519 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392195+0000", 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1378e-05 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392206+0000", 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3589e-05 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392220+0000", 2024-02-18T00:39:10.520 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1283e-05 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392231+0000", 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0343999999999997e-05 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392292+0000", 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3749999999999999e-06 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.521 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392295+0000", 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.58736239099999998 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979657+0000", 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103571 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.522 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979761+0000", 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000375254 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.980136+0000", 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061387799999999999 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.986275+0000", 2024-02-18T00:39:10.523 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032303779999999999 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989505+0000", 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7123000000000003e-05 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989552+0000", 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996 2024-02-18T00:39:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42910.0:2 115.13 115:ce638d0e:test-rados-api-smithi059-82949-26::foo:head [writefull 0~8388608] snapc 0=[] ondisk+write+known_if_redirected e178)", 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 293.42536665, 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003, 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.525 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42910", 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/955395839", 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.526 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979948+0000", 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0051422639999999997 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985090+0000", 2024-02-18T00:39:10.527 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1109e-05 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985101+0000", 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2439999999999993e-06 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985110+0000", 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032139199999999999 2024-02-18T00:39:10.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985431+0000", 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115649 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985547+0000", 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014544800000000001 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.529 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,3", 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985692+0000", 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016096787000000001 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.001789+0000", 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016631943 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.530 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018421+0000", 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016582200000000001 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018587+0000", 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4758999999999994e-05 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.531 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018651+0000", 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8247000000000002e-05 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018690+0000", 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.532 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:588 62.9 e205/123 62:9441d9b2:test-rados-api-smithi059-82957-14::foo82:4 v 205'35, mlcod=193'31)", 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 266.734348985, 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999, 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:39:10.533 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670966+0000", 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2614e-05 2024-02-18T00:39:10.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670979+0000", 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1616000000000001e-05 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671000+0000", 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9030000000000004e-06 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.535 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671008+0000", 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3940000000000001e-05 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671072+0000", 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0826999999999999e-05 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.536 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671093+0000", 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034117472000000003 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705210+0000", 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5481000000000005e-05 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705276+0000", 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999 2024-02-18T00:39:10.537 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:589 62.9 e205/123 62:93987215:test-rados-api-smithi059-82957-14::foo76:4 v 205'37, mlcod=193'31)", 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 266.73386318600001, 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001, 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.538 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.539 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.109e-06 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671460+0000", 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.027e-05 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.540 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671470+0000", 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.993e-06 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671476+0000", 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2646999999999999e-05 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671529+0000", 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0522000000000001e-05 2024-02-18T00:39:10.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671549+0000", 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036671306000000001 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708221+0000", 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8853000000000002e-05 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.542 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708259+0000", 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30969.0:1 179.5s0 179:b3610dda:::oid_RoundTripPP2_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 186.76766720800001, 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001, 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30969", 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3697991996", 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637648+0000", 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2819999999999999e-06 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.545 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637655+0000", 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9309999999999997e-06 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637664+0000", 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.153e-06 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637671+0000", 2024-02-18T00:39:10.546 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2398000000000003e-05 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637713+0000", 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2790000000000001e-06 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637716+0000", 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0156700160000001 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.547 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653386+0000", 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131438 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653517+0000", 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043157100000000001 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:39:10.548 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653949+0000", 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031255889999999998 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657074+0000", 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2801999999999999e-05 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657147+0000", 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000247073 2024-02-18T00:39:10.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657394+0000", 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30966.0:1 178.6s0 178:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:39:10.550 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 186.76694877899999, 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001, 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30966", 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3784803143", 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:39:10.551 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638367+0000", 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.622e-06 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.552 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638372+0000", 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7880000000000001e-06 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638381+0000", 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7909999999999999e-06 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.553 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638388+0000", 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.171e-05 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638420+0000", 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6700000000000001e-06 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638422+0000", 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0205109640000001 2024-02-18T00:39:10.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.658932+0000", 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014527400000000001 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659078+0000", 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043918800000000002 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:39:10.555 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659517+0000", 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00089794300000000001 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660415+0000", 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9174e-05 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660474+0000", 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024629900000000001 2024-02-18T00:39:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660720+0000", 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.40973.0:1 199.2s0 199:482bd08b:::oid_SimpleStatPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e354)", 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 117.630148108, 2024-02-18T00:39:10.557 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0539473699999999, 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.40973", 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1800225797", 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:39:10.558 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775166+0000", 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0146000000000001e-05 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.559 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775176+0000", 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2602000000000001e-05 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775189+0000", 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4409999999999999e-06 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.560 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775198+0000", 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9956e-05 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775258+0000", 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7239999999999998e-06 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.561 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775262+0000", 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.050233969 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.825496+0000", 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000114114 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.825610+0000", 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045617599999999998 2024-02-18T00:39:10.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.826066+0000", 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049755600000000002 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.826564+0000", 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024990279999999999 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.563 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.829063+0000", 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3276000000000003e-05 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.829116+0000", 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0539473699999999 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.564 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.41009.0:1 203.4s0 203:311cbe94:::oid_ExecuteClassPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 96.734065006999998, 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.014258211, 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.41009", 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1940889306", 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.565 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.566 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671249+0000", 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2910000000000007e-06 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671257+0000", 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0814e-05 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671268+0000", 2024-02-18T00:39:10.567 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5390000000000002e-06 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671276+0000", 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9815999999999998e-05 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671336+0000", 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.568 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671339+0000", 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.011917613 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683257+0000", 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016619700000000001 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683423+0000", 2024-02-18T00:39:10.569 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00048756299999999999 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683910+0000", 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00099101599999999995 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.684901+0000", 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00052922899999999998 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.570 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.685431+0000", 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9234999999999996e-05 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.685510+0000", 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.014258211 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.571 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25220.0:5782 209.10 209:08000000:%01:::head [pgnls start_epoch 404] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e404)", 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 64.670318034999994, 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.965927443, 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25220", 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 5782 2024-02-18T00:39:10.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.573 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734998+0000", 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7509999999999997e-06 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735003+0000", 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0363000000000001e-05 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:10.574 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735014+0000", 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8989999999999998e-06 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735020+0000", 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2460000000000003e-05 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735072+0000", 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8979999999999996e-06 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.575 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735080+0000", 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96567722099999997 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700757+0000", 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4641000000000002e-05 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.576 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700802+0000", 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012411200000000001 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700926+0000", 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.965927443 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:10.577 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:39:10.578 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:09.586618+0000 osd.0 (osd.0) 210 : cluster [DBG] 25.a scrub starts 2024-02-18T00:39:10.578 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:09.589597+0000 osd.0 (osd.0) 211 : cluster [DBG] 25.a scrub ok 2024-02-18T00:39:10.578 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:09.706293+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:10.578 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:09.706703+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:10.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:09.707375+0000 mon.c (mon.2) 426 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-63"}]: dispatch 2024-02-18T00:39:10.579 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:09.707650+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-63"}]: dispatch 2024-02-18T00:39:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:11.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:11.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:11.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:11.295 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:11.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:09.534627+0000 osd.5 (osd.5) 193 : cluster [DBG] 4.f scrub starts 2024-02-18T00:39:11.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:09.538130+0000 osd.5 (osd.5) 194 : cluster [DBG] 4.f scrub ok 2024-02-18T00:39:11.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:09.866180+0000 osd.6 (osd.6) 211 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:39:11.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:09.867256+0000 osd.6 (osd.6) 212 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:39:11.489 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:09.913574+0000 mgr.y (mgr.24454) 418 : cluster [DBG] pgmap v668: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 821 B/s rd, 0 op/s 2024-02-18T00:39:11.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:10.052900+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e446: 8 total, 8 up, 7 in 2024-02-18T00:39:11.490 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:09.534627+0000 osd.5 (osd.5) 193 : cluster [DBG] 4.f scrub starts 2024-02-18T00:39:11.490 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:09.538130+0000 osd.5 (osd.5) 194 : cluster [DBG] 4.f scrub ok 2024-02-18T00:39:11.490 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:09.866180+0000 osd.6 (osd.6) 211 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:39:11.490 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:09.867256+0000 osd.6 (osd.6) 212 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:39:11.490 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:09.913574+0000 mgr.y (mgr.24454) 418 : cluster [DBG] pgmap v668: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 821 B/s rd, 0 op/s 2024-02-18T00:39:11.490 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:10.052900+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e446: 8 total, 8 up, 7 in 2024-02-18T00:39:11.494 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:11.500 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:11.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:11.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:39:11.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:39:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:09.534627+0000 osd.5 (osd.5) 193 : cluster [DBG] 4.f scrub starts 2024-02-18T00:39:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:09.538130+0000 osd.5 (osd.5) 194 : cluster [DBG] 4.f scrub ok 2024-02-18T00:39:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:09.866180+0000 osd.6 (osd.6) 211 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:39:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:09.867256+0000 osd.6 (osd.6) 212 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:39:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:09.913574+0000 mgr.y (mgr.24454) 418 : cluster [DBG] pgmap v668: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 821 B/s rd, 0 op/s 2024-02-18T00:39:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:10.052900+0000 mon.a (mon.0) 2583 : cluster [DBG] osdmap e446: 8 total, 8 up, 7 in 2024-02-18T00:39:11.602 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:11.634 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:11.656 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:11.658 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:11.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:11.760 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:11.790 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:11.812 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:11.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:11.815 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:11.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:11.946 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:11.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 -- 172.21.15.59:0/3715579666 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd40102210 msgr2=0x7fbd40104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 --2- 172.21.15.59:0/3715579666 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd40102210 0x7fbd40104600 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 rx=0x7fbd38009c30 tx=0x7fbd38004c50).stop 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 -- 172.21.15.59:0/3715579666 shutdown_connections 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 --2- 172.21.15.59:0/3715579666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd400f8cb0 0x7fbd400f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 --2- 172.21.15.59:0/3715579666 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd40102210 0x7fbd40104600 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 --2- 172.21.15.59:0/3715579666 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd400f9610 0x7fbd40101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 -- 172.21.15.59:0/3715579666 >> 172.21.15.59:0/3715579666 conn(0x7fbd400f2820 msgr2=0x7fbd400f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 -- 172.21.15.59:0/3715579666 shutdown_connections 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.988+0000 7fbd4504b700 1 -- 172.21.15.59:0/3715579666 wait complete. 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 librados: starting msgr at 2024-02-18T00:39:11.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 librados: starting objecter 2024-02-18T00:39:11.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 Processor -- start 2024-02-18T00:39:11.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 -- start start 2024-02-18T00:39:11.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 librados: setting wanted keys 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 librados: calling monclient init 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd400f8cb0 0x7fbd400f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd400f9610 0x7fbd400f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd40102210 0x7fbd400f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.989+0000 7fbd4504b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd40116520 con 0x7fbd40102210 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd4504b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd40116690 con 0x7fbd400f9610 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd4504b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd40116850 con 0x7fbd400f8cb0 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd40102210 0x7fbd400f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd40102210 0x7fbd400f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57692/0 (socket says 172.21.15.59:57692) 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3f59e700 1 -- 172.21.15.59:0/1340891732 learned_addr learned my addr 172.21.15.59:0/1340891732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3ed9d700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd400f8cb0 0x7fbd400f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3e59c700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd400f9610 0x7fbd400f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:11.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3ed9d700 1 -- 172.21.15.59:0/1340891732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd400f9610 msgr2=0x7fbd400f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3ed9d700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd400f9610 0x7fbd400f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3ed9d700 1 -- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd40102210 msgr2=0x7fbd400f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3ed9d700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd40102210 0x7fbd400f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3ed9d700 1 -- 172.21.15.59:0/1340891732 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd380097c0 con 0x7fbd400f8cb0 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3e59c700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd400f9610 0x7fbd400f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3f59e700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd40102210 0x7fbd400f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd3ed9d700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd400f8cb0 0x7fbd400f6c60 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 rx=0x7fbd3400b4e0 tx=0x7fbd34009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd27fff700 10 client.?.objecter ms_handle_connect 0x7fbd400f8cb0 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd27fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd27fff700 1 -- 172.21.15.59:0/1340891732 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34002320 con 0x7fbd400f8cb0 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd27fff700 1 -- 172.21.15.59:0/1340891732 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd34002950 con 0x7fbd400f8cb0 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd40116b30 con 0x7fbd400f8cb0 2024-02-18T00:39:11.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd27fff700 1 -- 172.21.15.59:0/1340891732 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340129c0 con 0x7fbd400f8cb0 2024-02-18T00:39:11.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd4504b700 10 client.41195.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:11.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd40117020 con 0x7fbd400f8cb0 2024-02-18T00:39:11.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.990+0000 7fbd4504b700 1 librados: init done 2024-02-18T00:39:11.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.991+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4005fa60 con 0x7fbd400f8cb0 2024-02-18T00:39:11.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.992+0000 7fbd27fff700 1 -- 172.21.15.59:0/1340891732 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fbd34002480 con 0x7fbd400f8cb0 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.992+0000 7fbd27fff700 1 --2- 172.21.15.59:0/1340891732 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbd28062c90 0x7fbd28065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.992+0000 7fbd27fff700 1 -- 172.21.15.59:0/1340891732 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(447..447 src has 180..447) v4 ==== 8022+0+0 (secure 0 0 0) 0x7fbd3407d120 con 0x7fbd400f8cb0 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.992+0000 7fbd27fff700 10 client.41195.objecter ms_dispatch 0x7fbd40001550 osd_map(447..447 src has 180..447) v4 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.992+0000 7fbd27fff700 3 client.41195.objecter handle_osd_map got epochs [447,447] > 0 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.992+0000 7fbd27fff700 3 client.41195.objecter handle_osd_map decoding full epoch 447 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.992+0000 7fbd3e59c700 1 --2- 172.21.15.59:0/1340891732 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbd28062c90 0x7fbd28065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.993+0000 7fbd27fff700 20 client.41195.objecter dump_active .. 0 homeless 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.993+0000 7fbd27fff700 10 client.41195.objecter ms_handle_connect 0x7fbd28062c90 2024-02-18T00:39:11.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.993+0000 7fbd3e59c700 1 --2- 172.21.15.59:0/1340891732 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbd28062c90 0x7fbd28065150 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 rx=0x7fbd30009f20 tx=0x7fbd3000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:11.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:11.997+0000 7fbd27fff700 1 -- 172.21.15.59:0/1340891732 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbd34050650 con 0x7fbd400f8cb0 2024-02-18T00:39:12.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:12.197+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fbd400f8140 con 0x7fbd400f8cb0 2024-02-18T00:39:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:10.581457+0000 osd.5 (osd.5) 195 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:39:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:10.582423+0000 osd.5 (osd.5) 196 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:39:12.208 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:10.825677+0000 osd.6 (osd.6) 213 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:10.827598+0000 osd.6 (osd.6) 214 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.058267+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e447: 8 total, 8 up, 7 in 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:11.064647+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:11.065107+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:11.283036+0000 mgr.y (mgr.24454) 419 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.323114+0000 osd.4 (osd.4) 267 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.324808+0000 osd.4 (osd.4) 268 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.642207+0000 osd.0 (osd.0) 212 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.643588+0000 osd.0 (osd.0) 213 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:39:12.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:10.581457+0000 osd.5 (osd.5) 195 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:10.582423+0000 osd.5 (osd.5) 196 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:10.825677+0000 osd.6 (osd.6) 213 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:10.827598+0000 osd.6 (osd.6) 214 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.058267+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e447: 8 total, 8 up, 7 in 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:11.064647+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:11.065107+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:11.283036+0000 mgr.y (mgr.24454) 419 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.323114+0000 osd.4 (osd.4) 267 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.324808+0000 osd.4 (osd.4) 268 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.642207+0000 osd.0 (osd.0) 212 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:39:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.643588+0000 osd.0 (osd.0) 213 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:39:12.494 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:39:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:12] "GET /metrics HTTP/1.1" 200 189761 "" "Prometheus/2.33.4" 2024-02-18T00:39:12.497 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:12.500 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:12.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:10.581457+0000 osd.5 (osd.5) 195 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:39:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:10.582423+0000 osd.5 (osd.5) 196 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:10.825677+0000 osd.6 (osd.6) 213 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:10.827598+0000 osd.6 (osd.6) 214 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.058267+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e447: 8 total, 8 up, 7 in 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:11.064647+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:11.065107+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:11.283036+0000 mgr.y (mgr.24454) 419 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.323114+0000 osd.4 (osd.4) 267 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.324808+0000 osd.4 (osd.4) 268 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.642207+0000 osd.0 (osd.0) 212 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:39:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.643588+0000 osd.0 (osd.0) 213 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:39:12.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-18T00:39:12.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:12.630 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:12.658 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:12.661 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:12.661 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:12.762 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:12.816 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:12.818 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:12.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:12.920 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:12.947 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:13.021 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:13.022 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:13.023 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:13.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:13.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.072+0000 7fbd27fff700 1 -- 172.21.15.59:0/1340891732 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v449) v1 ==== 101+0+0 (secure 0 0 0) 0x7fbd34050240 con 0x7fbd400f8cb0 2024-02-18T00:39:13.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 10 librados: watch_flush enter 2024-02-18T00:39:13.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 10 librados: watch_flush exit 2024-02-18T00:39:13.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 20 client.41195.objecter shutdown clearing up homeless session... 2024-02-18T00:39:13.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 10 client.41195.objecter successfully canceled tick 2024-02-18T00:39:13.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbd28062c90 msgr2=0x7fbd28065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 --2- 172.21.15.59:0/1340891732 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbd28062c90 0x7fbd28065150 secure :-1 s=READY pgs=3119 cs=0 l=1 rev1=1 rx=0x7fbd30009f20 tx=0x7fbd3000b040).stop 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd400f8cb0 msgr2=0x7fbd400f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd400f8cb0 0x7fbd400f6c60 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 rx=0x7fbd3400b4e0 tx=0x7fbd34009060).stop 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 shutdown_connections 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd40102210 0x7fbd400f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd400f8cb0 0x7fbd400f6c60 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 --2- 172.21.15.59:0/1340891732 >> v2:172.21.15.59:6800/3721891797 conn(0x7fbd28062c90 0x7fbd28065150 unknown :-1 s=CLOSED pgs=3119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 --2- 172.21.15.59:0/1340891732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd400f9610 0x7fbd400f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.074+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 >> 172.21.15.59:0/1340891732 conn(0x7fbd400f2820 msgr2=0x7fbd400f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.075+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 shutdown_connections 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.075+0000 7fbd4504b700 1 -- 172.21.15.59:0/1340891732 wait complete. 2024-02-18T00:39:13.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:13.075+0000 7fbd4504b700 1 librados: shutdown 2024-02-18T00:39:13.078 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:39:13.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:13.154 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.508747+0000 osd.1 (osd.1) 227 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.511590+0000 osd.1 (osd.1) 228 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.794540+0000 osd.6 (osd.6) 215 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.795504+0000 osd.6 (osd.6) 216 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:11.914127+0000 mgr.y (mgr.24454) 420 : cluster [DBG] pgmap v671: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:12.065462+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:12.065613+0000 mon.a (mon.0) 2587 : cluster [DBG] osdmap e448: 8 total, 8 up, 7 in 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:12.070313+0000 mon.c (mon.2) 428 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:13.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:12.198365+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.59:0/1340891732' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:12.198641+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.508747+0000 osd.1 (osd.1) 227 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.511590+0000 osd.1 (osd.1) 228 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.794540+0000 osd.6 (osd.6) 215 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.795504+0000 osd.6 (osd.6) 216 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:11.914127+0000 mgr.y (mgr.24454) 420 : cluster [DBG] pgmap v671: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:12.065462+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:12.065613+0000 mon.a (mon.0) 2587 : cluster [DBG] osdmap e448: 8 total, 8 up, 7 in 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:12.070313+0000 mon.c (mon.2) 428 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:12.198365+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.59:0/1340891732' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:13.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:12.198641+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:13.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:13.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:13.408 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:13.439 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.508747+0000 osd.1 (osd.1) 227 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:39:13.439 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.511590+0000 osd.1 (osd.1) 228 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:39:13.439 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.794540+0000 osd.6 (osd.6) 215 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:39:13.439 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.795504+0000 osd.6 (osd.6) 216 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:39:13.439 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:11.914127+0000 mgr.y (mgr.24454) 420 : cluster [DBG] pgmap v671: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:13.439 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:12.065462+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:13.440 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:12.065613+0000 mon.a (mon.0) 2587 : cluster [DBG] osdmap e448: 8 total, 8 up, 7 in 2024-02-18T00:39:13.440 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:12.070313+0000 mon.c (mon.2) 428 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:13.440 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:12.198365+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.59:0/1340891732' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:13.440 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:12.198641+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:13.441 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:13.462 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:13.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:13.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:13.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:13.595 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:14.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:13.061640+0000 mon.a (mon.0) 2589 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:13.071367+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:13.071434+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e449: 8 total, 8 up, 7 in 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:13.101811+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:13.102117+0000 mon.a (mon.0) 2592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:13.102585+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-65"}]: dispatch 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:13.102796+0000 mon.a (mon.0) 2593 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-65"}]: dispatch 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:13.641944+0000 osd.0 (osd.0) 214 : cluster [DBG] 62.4 scrub starts 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:13.645420+0000 osd.0 (osd.0) 215 : cluster [DBG] 62.4 scrub ok 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:13.061640+0000 mon.a (mon.0) 2589 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:13.071367+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:13.071434+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e449: 8 total, 8 up, 7 in 2024-02-18T00:39:14.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:13.101811+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:13.102117+0000 mon.a (mon.0) 2592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:13.102585+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-65"}]: dispatch 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:13.102796+0000 mon.a (mon.0) 2593 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-65"}]: dispatch 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:13.641944+0000 osd.0 (osd.0) 214 : cluster [DBG] 62.4 scrub starts 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:13.645420+0000 osd.0 (osd.0) 215 : cluster [DBG] 62.4 scrub ok 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:13.061640+0000 mon.a (mon.0) 2589 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:13.071367+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:13.071434+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e449: 8 total, 8 up, 7 in 2024-02-18T00:39:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:13.101811+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:13.102117+0000 mon.a (mon.0) 2592 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:13.102585+0000 mon.c (mon.2) 431 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-65"}]: dispatch 2024-02-18T00:39:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:13.102796+0000 mon.a (mon.0) 2593 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-65"}]: dispatch 2024-02-18T00:39:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:13.641944+0000 osd.0 (osd.0) 214 : cluster [DBG] 62.4 scrub starts 2024-02-18T00:39:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:13.645420+0000 osd.0 (osd.0) 215 : cluster [DBG] 62.4 scrub ok 2024-02-18T00:39:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:14.547 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:14.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:14.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:14.679 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:15.128 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:15.131 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:15.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:15.232 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:15.286 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:15.288 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:15.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:13.914741+0000 mgr.y (mgr.24454) 421 : cluster [DBG] pgmap v674: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:15.332 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:14.074204+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e450: 8 total, 8 up, 7 in 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:14.405045+0000 osd.2 (osd.2) 191 : cluster [DBG] 62.3 scrub starts 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:14.406962+0000 osd.2 (osd.2) 192 : cluster [DBG] 62.3 scrub ok 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:14.632267+0000 osd.0 (osd.0) 216 : cluster [DBG] 6.16 deep-scrub starts 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:14.633698+0000 osd.0 (osd.0) 217 : cluster [DBG] 6.16 deep-scrub ok 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:13.914741+0000 mgr.y (mgr.24454) 421 : cluster [DBG] pgmap v674: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:14.074204+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e450: 8 total, 8 up, 7 in 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:14.405045+0000 osd.2 (osd.2) 191 : cluster [DBG] 62.3 scrub starts 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:14.406962+0000 osd.2 (osd.2) 192 : cluster [DBG] 62.3 scrub ok 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:14.632267+0000 osd.0 (osd.0) 216 : cluster [DBG] 6.16 deep-scrub starts 2024-02-18T00:39:15.333 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:14.633698+0000 osd.0 (osd.0) 217 : cluster [DBG] 6.16 deep-scrub ok 2024-02-18T00:39:15.390 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:15.419 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:15.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:13.914741+0000 mgr.y (mgr.24454) 421 : cluster [DBG] pgmap v674: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:14.074204+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e450: 8 total, 8 up, 7 in 2024-02-18T00:39:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:14.405045+0000 osd.2 (osd.2) 191 : cluster [DBG] 62.3 scrub starts 2024-02-18T00:39:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:14.406962+0000 osd.2 (osd.2) 192 : cluster [DBG] 62.3 scrub ok 2024-02-18T00:39:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:14.632267+0000 osd.0 (osd.0) 216 : cluster [DBG] 6.16 deep-scrub starts 2024-02-18T00:39:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:14.633698+0000 osd.0 (osd.0) 217 : cluster [DBG] 6.16 deep-scrub ok 2024-02-18T00:39:15.517 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:15.527 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:15.527 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:15.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:15.661 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:15.682 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:15.685 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:15.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:15.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 -- 172.21.15.59:0/669997323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 msgr2=0x7f719c0f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 --2- 172.21.15.59:0/669997323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 0x7f719c0f63b0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 rx=0x7f718c004870 tx=0x7f718c009cb0).stop 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 -- 172.21.15.59:0/669997323 shutdown_connections 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 --2- 172.21.15.59:0/669997323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c0fdbd0 0x7f719c0fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 --2- 172.21.15.59:0/669997323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0f6a90 0x7f719c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 --2- 172.21.15.59:0/669997323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 0x7f719c0f63b0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 -- 172.21.15.59:0/669997323 >> 172.21.15.59:0/669997323 conn(0x7f719c06c370 msgr2=0x7f719c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 -- 172.21.15.59:0/669997323 shutdown_connections 2024-02-18T00:39:15.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 -- 172.21.15.59:0/669997323 wait complete. 2024-02-18T00:39:15.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 librados: starting msgr at 2024-02-18T00:39:15.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 librados: starting objecter 2024-02-18T00:39:15.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.715+0000 7f71a1500700 1 Processor -- start 2024-02-18T00:39:15.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 -- start start 2024-02-18T00:39:15.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 librados: setting wanted keys 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 librados: calling monclient init 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c0f6a90 0x7f719c0f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0fdbd0 0x7f719c0f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 0x7f719c0f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c116550 con 0x7f719c0fe530 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1166c0 con 0x7f719c0f6a90 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f71a1500700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c116880 con 0x7f719c0fdbd0 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0fdbd0 0x7f719c0f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 0x7f719c0f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0fdbd0 0x7f719c0f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:35788/0 (socket says 172.21.15.59:35788) 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 -- 172.21.15.59:0/1297110420 learned_addr learned my addr 172.21.15.59:0/1297110420 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:15.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719affd700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c0f6a90 0x7f719c0f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 -- 172.21.15.59:0/1297110420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c0f6a90 msgr2=0x7f719c0f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c0f6a90 0x7f719c0f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 -- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 msgr2=0x7f719c0f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 0x7f719c0f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 -- 172.21.15.59:0/1297110420 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940097c0 con 0x7f719c0fdbd0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719b7fe700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 0x7f719c0f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.716+0000 7f719a7fc700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0fdbd0 0x7f719c0f7b70 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 rx=0x7f718c009eb0 tx=0x7f718c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f7193fff700 10 client.?.objecter ms_handle_connect 0x7f719c0fdbd0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f7193fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f7193fff700 1 -- 172.21.15.59:0/1297110420 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c00e070 con 0x7f719c0fdbd0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c005040 con 0x7f719c0fdbd0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f7193fff700 1 -- 172.21.15.59:0/1297110420 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f718c013da0 con 0x7f719c0fdbd0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f7193fff700 1 -- 172.21.15.59:0/1297110420 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c019070 con 0x7f719c0fdbd0 2024-02-18T00:39:15.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f71a1500700 10 client.41201.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:15.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c116d70 con 0x7f719c0fdbd0 2024-02-18T00:39:15.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.717+0000 7f71a1500700 1 librados: init done 2024-02-18T00:39:15.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.718+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c05f930 con 0x7f719c0fdbd0 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.718+0000 7f7193fff700 1 -- 172.21.15.59:0/1297110420 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f718c01e420 con 0x7f719c0fdbd0 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.718+0000 7f7193fff700 1 --2- 172.21.15.59:0/1297110420 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7188062c90 0x7f7188065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f7193fff700 1 -- 172.21.15.59:0/1297110420 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(451..451 src has 180..451) v4 ==== 8022+0+0 (secure 0 0 0) 0x7f718c07dc70 con 0x7f719c0fdbd0 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f7193fff700 10 client.41201.objecter ms_dispatch 0x7f719c001550 osd_map(451..451 src has 180..451) v4 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f7193fff700 3 client.41201.objecter handle_osd_map got epochs [451,451] > 0 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f7193fff700 3 client.41201.objecter handle_osd_map decoding full epoch 451 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f719affd700 1 --2- 172.21.15.59:0/1297110420 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7188062c90 0x7f7188065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f7193fff700 20 client.41201.objecter dump_active .. 0 homeless 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f7193fff700 10 client.41201.objecter ms_handle_connect 0x7f7188062c90 2024-02-18T00:39:15.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.719+0000 7f719affd700 1 --2- 172.21.15.59:0/1297110420 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7188062c90 0x7f7188065150 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 rx=0x7f7184004d20 tx=0x7f7184009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:15.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.723+0000 7f7193fff700 1 -- 172.21.15.59:0/1297110420 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f718c051280 con 0x7f719c0fdbd0 2024-02-18T00:39:15.786 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:15.815 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:15.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.894+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f719c107140 con 0x7f7188062c90 2024-02-18T00:39:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.910+0000 7f7193fff700 1 -- 172.21.15.59:0/1297110420 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+594601 (secure 0 0 0) 0x7f719c107140 con 0x7f7188062c90 2024-02-18T00:39:15.920 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:39:15.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 10 librados: watch_flush enter 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 10 librados: watch_flush exit 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 20 client.41201.objecter shutdown clearing up homeless session... 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 10 client.41201.objecter successfully canceled tick 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7188062c90 msgr2=0x7f7188065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 --2- 172.21.15.59:0/1297110420 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7188062c90 0x7f7188065150 secure :-1 s=READY pgs=3120 cs=0 l=1 rev1=1 rx=0x7f7184004d20 tx=0x7f7184009040).stop 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0fdbd0 msgr2=0x7f719c0f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0fdbd0 0x7f719c0f7b70 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 rx=0x7f718c009eb0 tx=0x7f718c00b040).stop 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 shutdown_connections 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 --2- 172.21.15.59:0/1297110420 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7188062c90 0x7f7188065150 unknown :-1 s=CLOSED pgs=3120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f719c0f6a90 0x7f719c0f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f719c0fe530 0x7f719c0f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 --2- 172.21.15.59:0/1297110420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f719c0fdbd0 0x7f719c0f7b70 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:15.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.914+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 >> 172.21.15.59:0/1297110420 conn(0x7f719c06c370 msgr2=0x7f719c0f58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:15.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.915+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 shutdown_connections 2024-02-18T00:39:15.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.915+0000 7f71a1500700 1 -- 172.21.15.59:0/1297110420 wait complete. 2024-02-18T00:39:15.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:15.915+0000 7f71a1500700 1 librados: shutdown 2024-02-18T00:39:15.928 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:39:16.272 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:16.275 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:16.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:16.376 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:16.391 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:14.569926+0000 osd.3 (osd.3) 251 : cluster [DBG] 6.6 deep-scrub starts 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:14.570913+0000 osd.3 (osd.3) 252 : cluster [DBG] 6.6 deep-scrub ok 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:15.080410+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e451: 8 total, 8 up, 7 in 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:15.089235+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:15.089718+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:14.569926+0000 osd.3 (osd.3) 251 : cluster [DBG] 6.6 deep-scrub starts 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:14.570913+0000 osd.3 (osd.3) 252 : cluster [DBG] 6.6 deep-scrub ok 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:15.080410+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e451: 8 total, 8 up, 7 in 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:15.089235+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:16.392 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:15.089718+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:16.410 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:16.436 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:16.440 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:16.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:16.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:14.569926+0000 osd.3 (osd.3) 251 : cluster [DBG] 6.6 deep-scrub starts 2024-02-18T00:39:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:14.570913+0000 osd.3 (osd.3) 252 : cluster [DBG] 6.6 deep-scrub ok 2024-02-18T00:39:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:15.080410+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e451: 8 total, 8 up, 7 in 2024-02-18T00:39:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:15.089235+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:15.089718+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:16.541 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:16.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:39:16.573 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:16.595 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:16.598 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:16.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:16.699 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:16.729 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:16.998 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:17.022 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":676,"stamp":"2024-02-18T00:39:15.077684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007298,"num_objects":385,"num_object_clones":3,"num_object_copies":1155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":385,"num_whiteouts":0,"num_read":1796,"num_read_kb":1419,"num_write":1227,"num_write_kb":4547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2612,"ondisk_log_size":2612,"up":618,"acting":618,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":714,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2164696,"kb_used_data":47264,"kb_used_omap":0,"kb_used_meta":2117376,"kb_avail":654051368,"statfs":{"total":671965249536,"available":669748600832,"internally_reserved":0,"allocated":48398336,"data_stored":30950259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2168193024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022003"},"pg_stats":[{"pgid":"215.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"215.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:39:15.072097+0000","last_change":"2024-02-18T00:39:15.072097+0000","last_active":"2024-02-18T00:39:15.072097+0000","last_peered":"2024-02-18T00:39:15.072097+0000","last_clean":"2024-02-18T00:39:15.072097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:39:15.072097+0000","last_undegraded":"2024-02-18T00:39:15.072097+0000","last_fullsized":"2024-02-18T00:39:15.072097+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:15.072097+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.072097+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":441,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687235+0000","last_change":"2024-02-18T00:38:59.771459+0000","last_active":"2024-02-18T00:39:08.687235+0000","last_peered":"2024-02-18T00:39:08.687235+0000","last_clean":"2024-02-18T00:39:08.687235+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:39:08.687235+0000","last_undegraded":"2024-02-18T00:39:08.687235+0000","last_fullsized":"2024-02-18T00:39:08.687235+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:38:57.506738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.506738+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"401'6","reported_seq":410,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688736+0000","last_change":"2024-02-18T00:38:59.769009+0000","last_active":"2024-02-18T00:39:08.688736+0000","last_peered":"2024-02-18T00:39:08.688736+0000","last_clean":"2024-02-18T00:39:08.688736+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:39:08.688736+0000","last_undegraded":"2024-02-18T00:39:08.688736+0000","last_fullsized":"2024-02-18T00:39:08.688736+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'2","last_scrub_stamp":"2024-02-18T00:37:29.759114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.759114+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"442'87","reported_seq":485,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781051+0000","last_change":"2024-02-18T00:39:01.765858+0000","last_active":"2024-02-18T00:39:08.781051+0000","last_peered":"2024-02-18T00:39:08.781051+0000","last_clean":"2024-02-18T00:39:08.781051+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:39:08.781051+0000","last_undegraded":"2024-02-18T00:39:08.781051+0000","last_fullsized":"2024-02-18T00:39:08.781051+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'85","last_scrub_stamp":"2024-02-18T00:38:58.749675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:58.749675+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":72,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":27,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"347'75","reported_seq":467,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687619+0000","last_change":"2024-02-18T00:38:59.772646+0000","last_active":"2024-02-18T00:39:08.687619+0000","last_peered":"2024-02-18T00:39:08.687619+0000","last_clean":"2024-02-18T00:39:08.687619+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:39:08.687619+0000","last_undegraded":"2024-02-18T00:39:08.687619+0000","last_fullsized":"2024-02-18T00:39:08.687619+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"245'73","last_scrub_stamp":"2024-02-18T00:36:34.581302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:34.581302+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"404'123","reported_seq":509,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463378+0000","last_change":"2024-02-18T00:38:59.691483+0000","last_active":"2024-02-18T00:39:09.463378+0000","last_peered":"2024-02-18T00:39:09.463378+0000","last_clean":"2024-02-18T00:39:09.463378+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:39:09.463378+0000","last_undegraded":"2024-02-18T00:39:09.463378+0000","last_fullsized":"2024-02-18T00:39:09.463378+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"230'120","last_scrub_stamp":"2024-02-18T00:36:36.464361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.464361+0000","log_size":123,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":39,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":485,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689699+0000","last_change":"2024-02-18T00:38:59.691992+0000","last_active":"2024-02-18T00:39:08.689699+0000","last_peered":"2024-02-18T00:39:08.689699+0000","last_clean":"2024-02-18T00:39:08.689699+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:39:08.689699+0000","last_undegraded":"2024-02-18T00:39:08.689699+0000","last_fullsized":"2024-02-18T00:39:08.689699+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'95","last_scrub_stamp":"2024-02-18T00:38:59.313626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.313626+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"404'127","reported_seq":481,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689897+0000","last_change":"2024-02-18T00:38:59.774582+0000","last_active":"2024-02-18T00:39:08.689897+0000","last_peered":"2024-02-18T00:39:08.689897+0000","last_clean":"2024-02-18T00:39:08.689897+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:39:08.689897+0000","last_undegraded":"2024-02-18T00:39:08.689897+0000","last_fullsized":"2024-02-18T00:39:08.689897+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'124","last_scrub_stamp":"2024-02-18T00:36:53.649677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.649677+0000","log_size":127,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":95,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":39,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"412'76","reported_seq":462,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689659+0000","last_change":"2024-02-18T00:38:59.774911+0000","last_active":"2024-02-18T00:39:08.689659+0000","last_peered":"2024-02-18T00:39:08.689659+0000","last_clean":"2024-02-18T00:39:08.689659+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:39:08.689659+0000","last_undegraded":"2024-02-18T00:39:08.689659+0000","last_fullsized":"2024-02-18T00:39:08.689659+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'73","last_scrub_stamp":"2024-02-18T00:36:37.532316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.532316+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":24,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'70","reported_seq":432,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690059+0000","last_change":"2024-02-18T00:38:59.769741+0000","last_active":"2024-02-18T00:39:08.690059+0000","last_peered":"2024-02-18T00:39:08.690059+0000","last_clean":"2024-02-18T00:39:08.690059+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:39:08.690059+0000","last_undegraded":"2024-02-18T00:39:08.690059+0000","last_fullsized":"2024-02-18T00:39:08.690059+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-18T00:37:44.685592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.685592+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"347'52","reported_seq":448,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688522+0000","last_change":"2024-02-18T00:38:59.690323+0000","last_active":"2024-02-18T00:39:08.688522+0000","last_peered":"2024-02-18T00:39:08.688522+0000","last_clean":"2024-02-18T00:39:08.688522+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:39:08.688522+0000","last_undegraded":"2024-02-18T00:39:08.688522+0000","last_fullsized":"2024-02-18T00:39:08.688522+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'52","last_scrub_stamp":"2024-02-18T00:38:27.136753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:27.136753+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'41","reported_seq":415,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687700+0000","last_change":"2024-02-18T00:39:02.513997+0000","last_active":"2024-02-18T00:39:08.687700+0000","last_peered":"2024-02-18T00:39:08.687700+0000","last_clean":"2024-02-18T00:39:08.687700+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:39:08.687700+0000","last_undegraded":"2024-02-18T00:39:08.687700+0000","last_fullsized":"2024-02-18T00:39:08.687700+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'41","last_scrub_stamp":"2024-02-18T00:39:02.513916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.513916+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"428'66","reported_seq":440,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462651+0000","last_change":"2024-02-18T00:38:59.690125+0000","last_active":"2024-02-18T00:39:09.462651+0000","last_peered":"2024-02-18T00:39:09.462651+0000","last_clean":"2024-02-18T00:39:09.462651+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:39:09.462651+0000","last_undegraded":"2024-02-18T00:39:09.462651+0000","last_fullsized":"2024-02-18T00:39:09.462651+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"428'66","last_scrub_stamp":"2024-02-18T00:38:34.725717+0000","last_deep_scrub":"428'66","last_deep_scrub_stamp":"2024-02-18T00:38:34.725717+0000","last_clean_scrub_stamp":"2024-02-18T00:38:34.725717+0000","log_size":66,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"379'18","reported_seq":417,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461922+0000","last_change":"2024-02-18T00:38:59.689614+0000","last_active":"2024-02-18T00:39:09.461922+0000","last_peered":"2024-02-18T00:39:09.461922+0000","last_clean":"2024-02-18T00:39:09.461922+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:39:09.461922+0000","last_undegraded":"2024-02-18T00:39:09.461922+0000","last_fullsized":"2024-02-18T00:39:09.461922+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'18","last_scrub_stamp":"2024-02-18T00:38:30.791397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:30.791397+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"347'29","reported_seq":452,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689395+0000","last_change":"2024-02-18T00:38:59.691571+0000","last_active":"2024-02-18T00:39:08.689395+0000","last_peered":"2024-02-18T00:39:08.689395+0000","last_clean":"2024-02-18T00:39:08.689395+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:39:08.689395+0000","last_undegraded":"2024-02-18T00:39:08.689395+0000","last_fullsized":"2024-02-18T00:39:08.689395+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'29","last_scrub_stamp":"2024-02-18T00:38:28.139054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.139054+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"347'83","reported_seq":472,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.464421+0000","last_change":"2024-02-18T00:38:59.691183+0000","last_active":"2024-02-18T00:39:09.464421+0000","last_peered":"2024-02-18T00:39:09.464421+0000","last_clean":"2024-02-18T00:39:09.464421+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:39:09.464421+0000","last_undegraded":"2024-02-18T00:39:09.464421+0000","last_fullsized":"2024-02-18T00:39:09.464421+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'83","last_scrub_stamp":"2024-02-18T00:38:30.700348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:30.700348+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":67,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"229'68","reported_seq":452,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689722+0000","last_change":"2024-02-18T00:39:08.309384+0000","last_active":"2024-02-18T00:39:08.689722+0000","last_peered":"2024-02-18T00:39:08.689722+0000","last_clean":"2024-02-18T00:39:08.689722+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:39:08.689722+0000","last_undegraded":"2024-02-18T00:39:08.689722+0000","last_fullsized":"2024-02-18T00:39:08.689722+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:39:08.309293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:08.309293+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'57","reported_seq":466,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689942+0000","last_change":"2024-02-18T00:38:59.769435+0000","last_active":"2024-02-18T00:39:08.689942+0000","last_peered":"2024-02-18T00:39:08.689942+0000","last_clean":"2024-02-18T00:39:08.689942+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:39:08.689942+0000","last_undegraded":"2024-02-18T00:39:08.689942+0000","last_fullsized":"2024-02-18T00:39:08.689942+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'57","last_scrub_stamp":"2024-02-18T00:37:36.817719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.817719+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"398'21","reported_seq":425,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688529+0000","last_change":"2024-02-18T00:38:59.768999+0000","last_active":"2024-02-18T00:39:08.688529+0000","last_peered":"2024-02-18T00:39:08.688529+0000","last_clean":"2024-02-18T00:39:08.688529+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:39:08.688529+0000","last_undegraded":"2024-02-18T00:39:08.688529+0000","last_fullsized":"2024-02-18T00:39:08.688529+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:37:31.817281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.817281+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"408'77","reported_seq":470,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687871+0000","last_change":"2024-02-18T00:39:01.511082+0000","last_active":"2024-02-18T00:39:08.687871+0000","last_peered":"2024-02-18T00:39:08.687871+0000","last_clean":"2024-02-18T00:39:08.687871+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:39:08.687871+0000","last_undegraded":"2024-02-18T00:39:08.687871+0000","last_fullsized":"2024-02-18T00:39:08.687871+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"408'77","last_scrub_stamp":"2024-02-18T00:39:01.510999+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.510999+0000","log_size":77,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":25,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"361'18","reported_seq":415,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463164+0000","last_change":"2024-02-18T00:38:59.691780+0000","last_active":"2024-02-18T00:39:09.463164+0000","last_peered":"2024-02-18T00:39:09.463164+0000","last_clean":"2024-02-18T00:39:09.463164+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:39:09.463164+0000","last_undegraded":"2024-02-18T00:39:09.463164+0000","last_fullsized":"2024-02-18T00:39:09.463164+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'17","last_scrub_stamp":"2024-02-18T00:36:43.345064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.345064+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":431,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690321+0000","last_change":"2024-02-18T00:38:59.692296+0000","last_active":"2024-02-18T00:39:08.690321+0000","last_peered":"2024-02-18T00:39:08.690321+0000","last_clean":"2024-02-18T00:39:08.690321+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:39:08.690321+0000","last_undegraded":"2024-02-18T00:39:08.690321+0000","last_fullsized":"2024-02-18T00:39:08.690321+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:38:57.350170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.350170+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"379'21","reported_seq":432,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781883+0000","last_change":"2024-02-18T00:38:59.764345+0000","last_active":"2024-02-18T00:39:08.781883+0000","last_peered":"2024-02-18T00:39:08.781883+0000","last_clean":"2024-02-18T00:39:08.781883+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:39:08.781883+0000","last_undegraded":"2024-02-18T00:39:08.781883+0000","last_fullsized":"2024-02-18T00:39:08.781883+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'21","last_scrub_stamp":"2024-02-18T00:37:41.575814+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.575814+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000035,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":3914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":493,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460350+0000","last_change":"2024-02-18T00:39:03.663944+0000","last_active":"2024-02-18T00:39:09.460350+0000","last_peered":"2024-02-18T00:39:09.460350+0000","last_clean":"2024-02-18T00:39:09.460350+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:39:09.460350+0000","last_undegraded":"2024-02-18T00:39:09.460350+0000","last_fullsized":"2024-02-18T00:39:09.460350+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:39:03.663869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.663869+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":361,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780375+0000","last_change":"2024-02-18T00:36:37.504644+0000","last_active":"2024-02-18T00:39:08.780375+0000","last_peered":"2024-02-18T00:39:08.780375+0000","last_clean":"2024-02-18T00:39:08.780375+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:39:08.780375+0000","last_undegraded":"2024-02-18T00:39:08.780375+0000","last_fullsized":"2024-02-18T00:39:08.780375+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:37.504568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.504568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"412'114","reported_seq":506,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688175+0000","last_change":"2024-02-18T00:38:59.774118+0000","last_active":"2024-02-18T00:39:08.688175+0000","last_peered":"2024-02-18T00:39:08.688175+0000","last_clean":"2024-02-18T00:39:08.688175+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:39:08.688175+0000","last_undegraded":"2024-02-18T00:39:08.688175+0000","last_fullsized":"2024-02-18T00:39:08.688175+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'112","last_scrub_stamp":"2024-02-18T00:36:38.583056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.583056+0000","log_size":114,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":359,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.464037+0000","last_change":"2024-02-18T00:37:01.774863+0000","last_active":"2024-02-18T00:39:09.464037+0000","last_peered":"2024-02-18T00:39:09.464037+0000","last_clean":"2024-02-18T00:39:09.464037+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:39:09.464037+0000","last_undegraded":"2024-02-18T00:39:09.464037+0000","last_fullsized":"2024-02-18T00:39:09.464037+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.774795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.774795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":443,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688883+0000","last_change":"2024-02-18T00:38:59.691596+0000","last_active":"2024-02-18T00:39:08.688883+0000","last_peered":"2024-02-18T00:39:08.688883+0000","last_clean":"2024-02-18T00:39:08.688883+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:39:08.688883+0000","last_undegraded":"2024-02-18T00:39:08.688883+0000","last_fullsized":"2024-02-18T00:39:08.688883+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:38:25.166249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.166249+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":320,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689694+0000","last_change":"2024-02-18T00:37:27.130328+0000","last_active":"2024-02-18T00:39:08.689694+0000","last_peered":"2024-02-18T00:39:08.689694+0000","last_clean":"2024-02-18T00:39:08.689694+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:39:08.689694+0000","last_undegraded":"2024-02-18T00:39:08.689694+0000","last_fullsized":"2024-02-18T00:39:08.689694+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.130252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.130252+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":495,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780832+0000","last_change":"2024-02-18T00:38:59.761571+0000","last_active":"2024-02-18T00:39:08.780832+0000","last_peered":"2024-02-18T00:39:08.780832+0000","last_clean":"2024-02-18T00:39:08.780832+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:39:08.780832+0000","last_undegraded":"2024-02-18T00:39:08.780832+0000","last_fullsized":"2024-02-18T00:39:08.780832+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:38:31.708778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:31.708778+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":318,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690371+0000","last_change":"2024-02-18T00:36:53.345938+0000","last_active":"2024-02-18T00:39:08.690371+0000","last_peered":"2024-02-18T00:39:08.690371+0000","last_clean":"2024-02-18T00:39:08.690371+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:39:08.690371+0000","last_undegraded":"2024-02-18T00:39:08.690371+0000","last_fullsized":"2024-02-18T00:39:08.690371+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.345872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":472,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780389+0000","last_change":"2024-02-18T00:38:59.761613+0000","last_active":"2024-02-18T00:39:08.780389+0000","last_peered":"2024-02-18T00:39:08.780389+0000","last_clean":"2024-02-18T00:39:08.780389+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:39:08.780389+0000","last_undegraded":"2024-02-18T00:39:08.780389+0000","last_fullsized":"2024-02-18T00:39:08.780389+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:38:22.718624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.718624+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":375,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690376+0000","last_change":"2024-02-18T00:39:02.380493+0000","last_active":"2024-02-18T00:39:08.690376+0000","last_peered":"2024-02-18T00:39:08.690376+0000","last_clean":"2024-02-18T00:39:08.690376+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:39:08.690376+0000","last_undegraded":"2024-02-18T00:39:08.690376+0000","last_fullsized":"2024-02-18T00:39:08.690376+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:39:02.380409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.380409+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"347'18","reported_seq":421,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.464212+0000","last_change":"2024-02-18T00:38:59.775856+0000","last_active":"2024-02-18T00:39:09.464212+0000","last_peered":"2024-02-18T00:39:09.464212+0000","last_clean":"2024-02-18T00:39:09.464212+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:39:09.464212+0000","last_undegraded":"2024-02-18T00:39:09.464212+0000","last_fullsized":"2024-02-18T00:39:09.464212+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'18","last_scrub_stamp":"2024-02-18T00:38:59.775767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.775767+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":361,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.779861+0000","last_change":"2024-02-18T00:36:35.460784+0000","last_active":"2024-02-18T00:39:08.779861+0000","last_peered":"2024-02-18T00:39:08.779861+0000","last_clean":"2024-02-18T00:39:08.779861+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:39:08.779861+0000","last_undegraded":"2024-02-18T00:39:08.779861+0000","last_fullsized":"2024-02-18T00:39:08.779861+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.460703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.460703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":449,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689531+0000","last_change":"2024-02-18T00:38:59.774788+0000","last_active":"2024-02-18T00:39:08.689531+0000","last_peered":"2024-02-18T00:39:08.689531+0000","last_clean":"2024-02-18T00:39:08.689531+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:39:08.689531+0000","last_undegraded":"2024-02-18T00:39:08.689531+0000","last_fullsized":"2024-02-18T00:39:08.689531+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:36:41.624180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.624180+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":350,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462770+0000","last_change":"2024-02-18T00:36:43.933486+0000","last_active":"2024-02-18T00:39:09.462770+0000","last_peered":"2024-02-18T00:39:09.462770+0000","last_clean":"2024-02-18T00:39:09.462770+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:39:09.462770+0000","last_undegraded":"2024-02-18T00:39:09.462770+0000","last_fullsized":"2024-02-18T00:39:09.462770+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:43.933414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:43.933414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780025+0000","last_change":"2024-02-18T00:38:59.763127+0000","last_active":"2024-02-18T00:39:08.780025+0000","last_peered":"2024-02-18T00:39:08.780025+0000","last_clean":"2024-02-18T00:39:08.780025+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:39:08.780025+0000","last_undegraded":"2024-02-18T00:39:08.780025+0000","last_fullsized":"2024-02-18T00:39:08.780025+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:37:37.618667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.618667+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":359,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781544+0000","last_change":"2024-02-18T00:38:29.678682+0000","last_active":"2024-02-18T00:39:08.781544+0000","last_peered":"2024-02-18T00:39:08.781544+0000","last_clean":"2024-02-18T00:39:08.781544+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:39:08.781544+0000","last_undegraded":"2024-02-18T00:39:08.781544+0000","last_fullsized":"2024-02-18T00:39:08.781544+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:29.678581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:38:29.678581+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"431'39","reported_seq":441,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462070+0000","last_change":"2024-02-18T00:38:59.766313+0000","last_active":"2024-02-18T00:39:09.462070+0000","last_peered":"2024-02-18T00:39:09.462070+0000","last_clean":"2024-02-18T00:39:09.462070+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:39:09.462070+0000","last_undegraded":"2024-02-18T00:39:09.462070+0000","last_fullsized":"2024-02-18T00:39:09.462070+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"431'39","last_scrub_stamp":"2024-02-18T00:38:58.470138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:58.470138+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":27,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":435,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688056+0000","last_change":"2024-02-18T00:36:45.627771+0000","last_active":"2024-02-18T00:39:08.688056+0000","last_peered":"2024-02-18T00:39:08.688056+0000","last_clean":"2024-02-18T00:39:08.688056+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:39:08.688056+0000","last_undegraded":"2024-02-18T00:39:08.688056+0000","last_fullsized":"2024-02-18T00:39:08.688056+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.627700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.627700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":417,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781268+0000","last_change":"2024-02-18T00:38:59.762397+0000","last_active":"2024-02-18T00:39:08.781268+0000","last_peered":"2024-02-18T00:39:08.781268+0000","last_clean":"2024-02-18T00:39:08.781268+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:39:08.781268+0000","last_undegraded":"2024-02-18T00:39:08.781268+0000","last_fullsized":"2024-02-18T00:39:08.781268+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'18","last_scrub_stamp":"2024-02-18T00:37:38.588190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.588190+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":384,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460774+0000","last_change":"2024-02-18T00:37:37.726927+0000","last_active":"2024-02-18T00:39:09.460774+0000","last_peered":"2024-02-18T00:39:09.460774+0000","last_clean":"2024-02-18T00:39:09.460774+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:39:09.460774+0000","last_undegraded":"2024-02-18T00:39:09.460774+0000","last_fullsized":"2024-02-18T00:39:09.460774+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.726861+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":365,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.464059+0000","last_change":"2024-02-18T00:38:28.692032+0000","last_active":"2024-02-18T00:39:09.464059+0000","last_peered":"2024-02-18T00:39:09.464059+0000","last_clean":"2024-02-18T00:39:09.464059+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:39:09.464059+0000","last_undegraded":"2024-02-18T00:39:09.464059+0000","last_fullsized":"2024-02-18T00:39:09.464059+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.691955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":402,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780639+0000","last_change":"2024-02-18T00:37:33.619485+0000","last_active":"2024-02-18T00:39:08.780639+0000","last_peered":"2024-02-18T00:39:08.780639+0000","last_clean":"2024-02-18T00:39:08.780639+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:39:08.780639+0000","last_undegraded":"2024-02-18T00:39:08.780639+0000","last_fullsized":"2024-02-18T00:39:08.780639+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:33.619410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.619410+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":494,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460887+0000","last_change":"2024-02-18T00:38:17.658701+0000","last_active":"2024-02-18T00:39:09.460887+0000","last_peered":"2024-02-18T00:39:09.460887+0000","last_clean":"2024-02-18T00:39:09.460887+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:39:09.460887+0000","last_undegraded":"2024-02-18T00:39:09.460887+0000","last_fullsized":"2024-02-18T00:39:09.460887+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:38:17.658626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:17.658626+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":428,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690104+0000","last_change":"2024-02-18T00:37:02.627426+0000","last_active":"2024-02-18T00:39:08.690104+0000","last_peered":"2024-02-18T00:39:08.690104+0000","last_clean":"2024-02-18T00:39:08.690104+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:39:08.690104+0000","last_undegraded":"2024-02-18T00:39:08.690104+0000","last_fullsized":"2024-02-18T00:39:08.690104+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.627349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.627349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461307+0000","last_change":"2024-02-18T00:37:33.616952+0000","last_active":"2024-02-18T00:39:09.461307+0000","last_peered":"2024-02-18T00:39:09.461307+0000","last_clean":"2024-02-18T00:39:09.461307+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:39:09.461307+0000","last_undegraded":"2024-02-18T00:39:09.461307+0000","last_fullsized":"2024-02-18T00:39:09.461307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.616874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.616874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":359,"reported_epoch":446,"state":"active+clean","last_fresh":"2024-02-18T00:39:10.526875+0000","last_change":"2024-02-18T00:39:10.533484+0000","last_active":"2024-02-18T00:39:10.526875+0000","last_peered":"2024-02-18T00:39:10.526875+0000","last_clean":"2024-02-18T00:39:10.526875+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:39:10.526875+0000","last_undegraded":"2024-02-18T00:39:10.526875+0000","last_fullsized":"2024-02-18T00:39:10.526875+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.919082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.919082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":465,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463720+0000","last_change":"2024-02-18T00:37:49.523198+0000","last_active":"2024-02-18T00:39:09.463720+0000","last_peered":"2024-02-18T00:39:09.463720+0000","last_clean":"2024-02-18T00:39:09.463720+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:39:09.463720+0000","last_undegraded":"2024-02-18T00:39:09.463720+0000","last_fullsized":"2024-02-18T00:39:09.463720+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:49.523121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:49.523121+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":422,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.691081+0000","last_change":"2024-02-18T00:38:25.644859+0000","last_active":"2024-02-18T00:39:08.691081+0000","last_peered":"2024-02-18T00:39:08.691081+0000","last_clean":"2024-02-18T00:39:08.691081+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:39:08.691081+0000","last_undegraded":"2024-02-18T00:39:08.691081+0000","last_fullsized":"2024-02-18T00:39:08.691081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.644791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.644791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":413,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688036+0000","last_change":"2024-02-18T00:36:56.601833+0000","last_active":"2024-02-18T00:39:08.688036+0000","last_peered":"2024-02-18T00:39:08.688036+0000","last_clean":"2024-02-18T00:39:08.688036+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:39:08.688036+0000","last_undegraded":"2024-02-18T00:39:08.688036+0000","last_fullsized":"2024-02-18T00:39:08.688036+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.601727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.601727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":427,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462313+0000","last_change":"2024-02-18T00:37:47.488889+0000","last_active":"2024-02-18T00:39:09.462313+0000","last_peered":"2024-02-18T00:39:09.462313+0000","last_clean":"2024-02-18T00:39:09.462313+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:39:09.462313+0000","last_undegraded":"2024-02-18T00:39:09.462313+0000","last_fullsized":"2024-02-18T00:39:09.462313+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.488821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.488821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":361,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462087+0000","last_change":"2024-02-18T00:38:25.732225+0000","last_active":"2024-02-18T00:39:09.462087+0000","last_peered":"2024-02-18T00:39:09.462087+0000","last_clean":"2024-02-18T00:39:09.462087+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:39:09.462087+0000","last_undegraded":"2024-02-18T00:39:09.462087+0000","last_fullsized":"2024-02-18T00:39:09.462087+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.732112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.732112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":495,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.779785+0000","last_change":"2024-02-18T00:38:20.697327+0000","last_active":"2024-02-18T00:39:08.779785+0000","last_peered":"2024-02-18T00:39:08.779785+0000","last_clean":"2024-02-18T00:39:08.779785+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:39:08.779785+0000","last_undegraded":"2024-02-18T00:39:08.779785+0000","last_fullsized":"2024-02-18T00:39:08.779785+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:38:20.697264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.697264+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":417,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688696+0000","last_change":"2024-02-18T00:36:49.680480+0000","last_active":"2024-02-18T00:39:08.688696+0000","last_peered":"2024-02-18T00:39:08.688696+0000","last_clean":"2024-02-18T00:39:08.688696+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:39:08.688696+0000","last_undegraded":"2024-02-18T00:39:08.688696+0000","last_fullsized":"2024-02-18T00:39:08.688696+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.680414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":413,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687748+0000","last_change":"2024-02-18T00:37:00.670726+0000","last_active":"2024-02-18T00:39:08.687748+0000","last_peered":"2024-02-18T00:39:08.687748+0000","last_clean":"2024-02-18T00:39:08.687748+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:39:08.687748+0000","last_undegraded":"2024-02-18T00:39:08.687748+0000","last_fullsized":"2024-02-18T00:39:08.687748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.670646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.670646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":317,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687658+0000","last_change":"2024-02-18T00:37:37.476740+0000","last_active":"2024-02-18T00:39:08.687658+0000","last_peered":"2024-02-18T00:39:08.687658+0000","last_clean":"2024-02-18T00:39:08.687658+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:39:08.687658+0000","last_undegraded":"2024-02-18T00:39:08.687658+0000","last_fullsized":"2024-02-18T00:39:08.687658+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.476676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.476676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":352,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461090+0000","last_change":"2024-02-18T00:38:59.474203+0000","last_active":"2024-02-18T00:39:09.461090+0000","last_peered":"2024-02-18T00:39:09.461090+0000","last_clean":"2024-02-18T00:39:09.461090+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:39:09.461090+0000","last_undegraded":"2024-02-18T00:39:09.461090+0000","last_fullsized":"2024-02-18T00:39:09.461090+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:59.474124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.474124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":448,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690263+0000","last_change":"2024-02-18T00:38:14.260160+0000","last_active":"2024-02-18T00:39:08.690263+0000","last_peered":"2024-02-18T00:39:08.690263+0000","last_clean":"2024-02-18T00:39:08.690263+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:39:08.690263+0000","last_undegraded":"2024-02-18T00:39:08.690263+0000","last_fullsized":"2024-02-18T00:39:08.690263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:38:14.260073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:14.260073+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":317,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689947+0000","last_change":"2024-02-18T00:36:58.282770+0000","last_active":"2024-02-18T00:39:08.689947+0000","last_peered":"2024-02-18T00:39:08.689947+0000","last_clean":"2024-02-18T00:39:08.689947+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:39:08.689947+0000","last_undegraded":"2024-02-18T00:39:08.689947+0000","last_fullsized":"2024-02-18T00:39:08.689947+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.282673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.282673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":413,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690462+0000","last_change":"2024-02-18T00:36:59.000753+0000","last_active":"2024-02-18T00:39:08.690462+0000","last_peered":"2024-02-18T00:39:08.690462+0000","last_clean":"2024-02-18T00:39:08.690462+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:39:08.690462+0000","last_undegraded":"2024-02-18T00:39:08.690462+0000","last_fullsized":"2024-02-18T00:39:08.690462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.000690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.000690+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689785+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:39:08.689785+0000","last_peered":"2024-02-18T00:39:08.689785+0000","last_clean":"2024-02-18T00:39:08.689785+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:39:08.689785+0000","last_undegraded":"2024-02-18T00:39:08.689785+0000","last_fullsized":"2024-02-18T00:39:08.689785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":201,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687215+0000","last_change":"2024-02-18T00:38:33.312167+0000","last_active":"2024-02-18T00:39:08.687215+0000","last_peered":"2024-02-18T00:39:08.687215+0000","last_clean":"2024-02-18T00:39:08.687215+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:39:08.687215+0000","last_undegraded":"2024-02-18T00:39:08.687215+0000","last_fullsized":"2024-02-18T00:39:08.687215+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:33.312079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.312079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":430,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689322+0000","last_change":"2024-02-18T00:37:29.177448+0000","last_active":"2024-02-18T00:39:08.689322+0000","last_peered":"2024-02-18T00:39:08.689322+0000","last_clean":"2024-02-18T00:39:08.689322+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:39:08.689322+0000","last_undegraded":"2024-02-18T00:39:08.689322+0000","last_fullsized":"2024-02-18T00:39:08.689322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:37:29.177369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.177369+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":431,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688738+0000","last_change":"2024-02-18T00:38:19.536204+0000","last_active":"2024-02-18T00:39:08.688738+0000","last_peered":"2024-02-18T00:39:08.688738+0000","last_clean":"2024-02-18T00:39:08.688738+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:39:08.688738+0000","last_undegraded":"2024-02-18T00:39:08.688738+0000","last_fullsized":"2024-02-18T00:39:08.688738+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:19.536095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:38:19.536095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":556,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689575+0000","last_change":"2024-02-18T00:37:38.433946+0000","last_active":"2024-02-18T00:39:08.689575+0000","last_peered":"2024-02-18T00:39:08.689575+0000","last_clean":"2024-02-18T00:39:08.689575+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:39:08.689575+0000","last_undegraded":"2024-02-18T00:39:08.689575+0000","last_fullsized":"2024-02-18T00:39:08.689575+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:38.433877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.433877+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690599+0000","last_change":"2024-02-18T00:37:31.127432+0000","last_active":"2024-02-18T00:39:08.690599+0000","last_peered":"2024-02-18T00:39:08.690599+0000","last_clean":"2024-02-18T00:39:08.690599+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:39:08.690599+0000","last_undegraded":"2024-02-18T00:39:08.690599+0000","last_fullsized":"2024-02-18T00:39:08.690599+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.127341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.127341+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":362,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689040+0000","last_change":"2024-02-18T00:38:20.588735+0000","last_active":"2024-02-18T00:39:08.689040+0000","last_peered":"2024-02-18T00:39:08.689040+0000","last_clean":"2024-02-18T00:39:08.689040+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:39:08.689040+0000","last_undegraded":"2024-02-18T00:39:08.689040+0000","last_fullsized":"2024-02-18T00:39:08.689040+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.588654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.588654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":418,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.538243+0000","last_change":"2024-02-18T00:39:09.538209+0000","last_active":"2024-02-18T00:39:09.538243+0000","last_peered":"2024-02-18T00:39:09.538243+0000","last_clean":"2024-02-18T00:39:09.538243+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:39:09.538243+0000","last_undegraded":"2024-02-18T00:39:09.538243+0000","last_fullsized":"2024-02-18T00:39:09.538243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:39:09.538136+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.538136+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":440,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689054+0000","last_change":"2024-02-18T00:37:41.747387+0000","last_active":"2024-02-18T00:39:08.689054+0000","last_peered":"2024-02-18T00:39:08.689054+0000","last_clean":"2024-02-18T00:39:08.689054+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:39:08.689054+0000","last_undegraded":"2024-02-18T00:39:08.689054+0000","last_fullsized":"2024-02-18T00:39:08.689054+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.747293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.747293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":540,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690558+0000","last_change":"2024-02-18T00:37:43.161256+0000","last_active":"2024-02-18T00:39:08.690558+0000","last_peered":"2024-02-18T00:39:08.690558+0000","last_clean":"2024-02-18T00:39:08.690558+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:39:08.690558+0000","last_undegraded":"2024-02-18T00:39:08.690558+0000","last_fullsized":"2024-02-18T00:39:08.690558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:43.161186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.161186+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462228+0000","last_change":"2024-02-18T00:38:57.423470+0000","last_active":"2024-02-18T00:39:09.462228+0000","last_peered":"2024-02-18T00:39:09.462228+0000","last_clean":"2024-02-18T00:39:09.462228+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:39:09.462228+0000","last_undegraded":"2024-02-18T00:39:09.462228+0000","last_fullsized":"2024-02-18T00:39:09.462228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:57.423405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.423405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":357,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689584+0000","last_change":"2024-02-18T00:38:26.149235+0000","last_active":"2024-02-18T00:39:08.689584+0000","last_peered":"2024-02-18T00:39:08.689584+0000","last_clean":"2024-02-18T00:39:08.689584+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:39:08.689584+0000","last_undegraded":"2024-02-18T00:39:08.689584+0000","last_fullsized":"2024-02-18T00:39:08.689584+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:38:26.149169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":477,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690452+0000","last_change":"2024-02-18T00:39:03.341590+0000","last_active":"2024-02-18T00:39:08.690452+0000","last_peered":"2024-02-18T00:39:08.690452+0000","last_clean":"2024-02-18T00:39:08.690452+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:39:08.690452+0000","last_undegraded":"2024-02-18T00:39:08.690452+0000","last_fullsized":"2024-02-18T00:39:08.690452+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:39:03.341502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.341502+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":387,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.459895+0000","last_change":"2024-02-18T00:36:57.321994+0000","last_active":"2024-02-18T00:39:09.459895+0000","last_peered":"2024-02-18T00:39:09.459895+0000","last_clean":"2024-02-18T00:39:09.459895+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:39:09.459895+0000","last_undegraded":"2024-02-18T00:39:09.459895+0000","last_fullsized":"2024-02-18T00:39:09.459895+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.321930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.321930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":408,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462632+0000","last_change":"2024-02-18T00:37:48.527852+0000","last_active":"2024-02-18T00:39:09.462632+0000","last_peered":"2024-02-18T00:39:09.462632+0000","last_clean":"2024-02-18T00:39:09.462632+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:39:09.462632+0000","last_undegraded":"2024-02-18T00:39:09.462632+0000","last_fullsized":"2024-02-18T00:39:09.462632+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:48.527779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.527779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462107+0000","last_change":"2024-02-18T00:37:30.849071+0000","last_active":"2024-02-18T00:39:09.462107+0000","last_peered":"2024-02-18T00:39:09.462107+0000","last_clean":"2024-02-18T00:39:09.462107+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:39:09.462107+0000","last_undegraded":"2024-02-18T00:39:09.462107+0000","last_fullsized":"2024-02-18T00:39:09.462107+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:30.848966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:30.848966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":357,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689610+0000","last_change":"2024-02-18T00:37:44.200088+0000","last_active":"2024-02-18T00:39:08.689610+0000","last_peered":"2024-02-18T00:39:08.689610+0000","last_clean":"2024-02-18T00:39:08.689610+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:39:08.689610+0000","last_undegraded":"2024-02-18T00:39:08.689610+0000","last_fullsized":"2024-02-18T00:39:08.689610+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.199947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":536,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.465412+0000","last_change":"2024-02-18T00:36:53.910404+0000","last_active":"2024-02-18T00:39:09.465412+0000","last_peered":"2024-02-18T00:39:09.465412+0000","last_clean":"2024-02-18T00:39:09.465412+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:39:09.465412+0000","last_undegraded":"2024-02-18T00:39:09.465412+0000","last_fullsized":"2024-02-18T00:39:09.465412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.910333+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":451,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688718+0000","last_change":"2024-02-18T00:36:50.262681+0000","last_active":"2024-02-18T00:39:08.688718+0000","last_peered":"2024-02-18T00:39:08.688718+0000","last_clean":"2024-02-18T00:39:08.688718+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:39:08.688718+0000","last_undegraded":"2024-02-18T00:39:08.688718+0000","last_fullsized":"2024-02-18T00:39:08.688718+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:36:50.262602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.262602+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":437,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781978+0000","last_change":"2024-02-18T00:38:24.645826+0000","last_active":"2024-02-18T00:39:08.781978+0000","last_peered":"2024-02-18T00:39:08.781978+0000","last_clean":"2024-02-18T00:39:08.781978+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:39:08.781978+0000","last_undegraded":"2024-02-18T00:39:08.781978+0000","last_fullsized":"2024-02-18T00:39:08.781978+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.645751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.645751+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":314,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781012+0000","last_change":"2024-02-18T00:37:27.733729+0000","last_active":"2024-02-18T00:39:08.781012+0000","last_peered":"2024-02-18T00:39:08.781012+0000","last_clean":"2024-02-18T00:39:08.781012+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:39:08.781012+0000","last_undegraded":"2024-02-18T00:39:08.781012+0000","last_fullsized":"2024-02-18T00:39:08.781012+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.733609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.733609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":352,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461968+0000","last_change":"2024-02-18T00:38:23.612159+0000","last_active":"2024-02-18T00:39:09.461968+0000","last_peered":"2024-02-18T00:39:09.461968+0000","last_clean":"2024-02-18T00:39:09.461968+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:39:09.461968+0000","last_undegraded":"2024-02-18T00:39:09.461968+0000","last_fullsized":"2024-02-18T00:39:09.461968+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:23.612082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:23.612082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":422,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689408+0000","last_change":"2024-02-18T00:36:45.308400+0000","last_active":"2024-02-18T00:39:08.689408+0000","last_peered":"2024-02-18T00:39:08.689408+0000","last_clean":"2024-02-18T00:39:08.689408+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:39:08.689408+0000","last_undegraded":"2024-02-18T00:39:08.689408+0000","last_fullsized":"2024-02-18T00:39:08.689408+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.308300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.308300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":513,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463780+0000","last_change":"2024-02-18T00:37:35.841167+0000","last_active":"2024-02-18T00:39:09.463780+0000","last_peered":"2024-02-18T00:39:09.463780+0000","last_clean":"2024-02-18T00:39:09.463780+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:39:09.463780+0000","last_undegraded":"2024-02-18T00:39:09.463780+0000","last_fullsized":"2024-02-18T00:39:09.463780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:37:35.841050+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.841050+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":431,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690990+0000","last_change":"2024-02-18T00:37:32.766303+0000","last_active":"2024-02-18T00:39:08.690990+0000","last_peered":"2024-02-18T00:39:08.690990+0000","last_clean":"2024-02-18T00:39:08.690990+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:39:08.690990+0000","last_undegraded":"2024-02-18T00:39:08.690990+0000","last_fullsized":"2024-02-18T00:39:08.690990+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.766203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.766203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689086+0000","last_change":"2024-02-18T00:37:33.117248+0000","last_active":"2024-02-18T00:39:08.689086+0000","last_peered":"2024-02-18T00:39:08.689086+0000","last_clean":"2024-02-18T00:39:08.689086+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:39:08.689086+0000","last_undegraded":"2024-02-18T00:39:08.689086+0000","last_fullsized":"2024-02-18T00:39:08.689086+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.117173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.117173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":352,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462716+0000","last_change":"2024-02-18T00:38:28.807296+0000","last_active":"2024-02-18T00:39:09.462716+0000","last_peered":"2024-02-18T00:39:09.462716+0000","last_clean":"2024-02-18T00:39:09.462716+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:39:09.462716+0000","last_undegraded":"2024-02-18T00:39:09.462716+0000","last_fullsized":"2024-02-18T00:39:09.462716+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:28.807229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.807229+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":410,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781397+0000","last_change":"2024-02-18T00:37:31.672777+0000","last_active":"2024-02-18T00:39:08.781397+0000","last_peered":"2024-02-18T00:39:08.781397+0000","last_clean":"2024-02-18T00:39:08.781397+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:39:08.781397+0000","last_undegraded":"2024-02-18T00:39:08.781397+0000","last_fullsized":"2024-02-18T00:39:08.781397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.672699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.672699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":493,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462657+0000","last_change":"2024-02-18T00:37:37.281742+0000","last_active":"2024-02-18T00:39:09.462657+0000","last_peered":"2024-02-18T00:39:09.462657+0000","last_clean":"2024-02-18T00:39:09.462657+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:39:09.462657+0000","last_undegraded":"2024-02-18T00:39:09.462657+0000","last_fullsized":"2024-02-18T00:39:09.462657+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:37:37.281674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.281674+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":324,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460055+0000","last_change":"2024-02-18T00:37:01.250860+0000","last_active":"2024-02-18T00:39:09.460055+0000","last_peered":"2024-02-18T00:39:09.460055+0000","last_clean":"2024-02-18T00:39:09.460055+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:39:09.460055+0000","last_undegraded":"2024-02-18T00:39:09.460055+0000","last_fullsized":"2024-02-18T00:39:09.460055+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:37:01.250771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.250771+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":416,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689122+0000","last_change":"2024-02-18T00:39:02.598810+0000","last_active":"2024-02-18T00:39:08.689122+0000","last_peered":"2024-02-18T00:39:08.689122+0000","last_clean":"2024-02-18T00:39:08.689122+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:39:08.689122+0000","last_undegraded":"2024-02-18T00:39:08.689122+0000","last_fullsized":"2024-02-18T00:39:08.689122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.598741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.598741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":352,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463351+0000","last_change":"2024-02-18T00:38:57.682162+0000","last_active":"2024-02-18T00:39:09.463351+0000","last_peered":"2024-02-18T00:39:09.463351+0000","last_clean":"2024-02-18T00:39:09.463351+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:39:09.463351+0000","last_undegraded":"2024-02-18T00:39:09.463351+0000","last_fullsized":"2024-02-18T00:39:09.463351+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:57.682091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.682091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":408,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462698+0000","last_change":"2024-02-18T00:37:47.574086+0000","last_active":"2024-02-18T00:39:09.462698+0000","last_peered":"2024-02-18T00:39:09.462698+0000","last_clean":"2024-02-18T00:39:09.462698+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:39:09.462698+0000","last_undegraded":"2024-02-18T00:39:09.462698+0000","last_fullsized":"2024-02-18T00:39:09.462698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.574003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.574003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":431,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460723+0000","last_change":"2024-02-18T00:37:35.212798+0000","last_active":"2024-02-18T00:39:09.460723+0000","last_peered":"2024-02-18T00:39:09.460723+0000","last_clean":"2024-02-18T00:39:09.460723+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:39:09.460723+0000","last_undegraded":"2024-02-18T00:39:09.460723+0000","last_fullsized":"2024-02-18T00:39:09.460723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:37:35.212727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.212727+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":393,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690837+0000","last_change":"2024-02-18T00:38:16.494876+0000","last_active":"2024-02-18T00:39:08.690837+0000","last_peered":"2024-02-18T00:39:08.690837+0000","last_clean":"2024-02-18T00:39:08.690837+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:39:08.690837+0000","last_undegraded":"2024-02-18T00:39:08.690837+0000","last_fullsized":"2024-02-18T00:39:08.690837+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:16.494804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:16.494804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":414,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462042+0000","last_change":"2024-02-18T00:37:26.204990+0000","last_active":"2024-02-18T00:39:09.462042+0000","last_peered":"2024-02-18T00:39:09.462042+0000","last_clean":"2024-02-18T00:39:09.462042+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:39:09.462042+0000","last_undegraded":"2024-02-18T00:39:09.462042+0000","last_fullsized":"2024-02-18T00:39:09.462042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:26.204918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:37:26.204918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690537+0000","last_change":"2024-02-18T00:37:25.868179+0000","last_active":"2024-02-18T00:39:08.690537+0000","last_peered":"2024-02-18T00:39:08.690537+0000","last_clean":"2024-02-18T00:39:08.690537+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:39:08.690537+0000","last_undegraded":"2024-02-18T00:39:08.690537+0000","last_fullsized":"2024-02-18T00:39:08.690537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:25.868065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:25.868065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":365,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.459763+0000","last_change":"2024-02-18T00:38:19.635886+0000","last_active":"2024-02-18T00:39:09.459763+0000","last_peered":"2024-02-18T00:39:09.459763+0000","last_clean":"2024-02-18T00:39:09.459763+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:39:09.459763+0000","last_undegraded":"2024-02-18T00:39:09.459763+0000","last_fullsized":"2024-02-18T00:39:09.459763+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:19.635822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:19.635822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":317,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688504+0000","last_change":"2024-02-18T00:37:32.401689+0000","last_active":"2024-02-18T00:39:08.688504+0000","last_peered":"2024-02-18T00:39:08.688504+0000","last_clean":"2024-02-18T00:39:08.688504+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:39:08.688504+0000","last_undegraded":"2024-02-18T00:39:08.688504+0000","last_fullsized":"2024-02-18T00:39:08.688504+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.401618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.401618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":464,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.779919+0000","last_change":"2024-02-18T00:36:50.759133+0000","last_active":"2024-02-18T00:39:08.779919+0000","last_peered":"2024-02-18T00:39:08.779919+0000","last_clean":"2024-02-18T00:39:08.779919+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:39:08.779919+0000","last_undegraded":"2024-02-18T00:39:08.779919+0000","last_fullsized":"2024-02-18T00:39:08.779919+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:36:50.759019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.759019+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":350,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.459774+0000","last_change":"2024-02-18T00:37:42.354830+0000","last_active":"2024-02-18T00:39:09.459774+0000","last_peered":"2024-02-18T00:39:09.459774+0000","last_clean":"2024-02-18T00:39:09.459774+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:39:09.459774+0000","last_undegraded":"2024-02-18T00:39:09.459774+0000","last_fullsized":"2024-02-18T00:39:09.459774+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.354758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.354758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":430,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461772+0000","last_change":"2024-02-18T00:37:00.820469+0000","last_active":"2024-02-18T00:39:09.461772+0000","last_peered":"2024-02-18T00:39:09.461772+0000","last_clean":"2024-02-18T00:39:09.461772+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:39:09.461772+0000","last_undegraded":"2024-02-18T00:39:09.461772+0000","last_fullsized":"2024-02-18T00:39:09.461772+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.820283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.820283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461464+0000","last_change":"2024-02-18T00:39:02.673159+0000","last_active":"2024-02-18T00:39:09.461464+0000","last_peered":"2024-02-18T00:39:09.461464+0000","last_clean":"2024-02-18T00:39:09.461464+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:39:09.461464+0000","last_undegraded":"2024-02-18T00:39:09.461464+0000","last_fullsized":"2024-02-18T00:39:09.461464+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.673088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.673088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462747+0000","last_change":"2024-02-18T00:36:42.828611+0000","last_active":"2024-02-18T00:39:09.462747+0000","last_peered":"2024-02-18T00:39:09.462747+0000","last_clean":"2024-02-18T00:39:09.462747+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:39:09.462747+0000","last_undegraded":"2024-02-18T00:39:09.462747+0000","last_fullsized":"2024-02-18T00:39:09.462747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.828542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.828542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":356,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463223+0000","last_change":"2024-02-18T00:38:25.649197+0000","last_active":"2024-02-18T00:39:09.463223+0000","last_peered":"2024-02-18T00:39:09.463223+0000","last_clean":"2024-02-18T00:39:09.463223+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:39:09.463223+0000","last_undegraded":"2024-02-18T00:39:09.463223+0000","last_fullsized":"2024-02-18T00:39:09.463223+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.649119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.649119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":422,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690237+0000","last_change":"2024-02-18T00:36:57.633211+0000","last_active":"2024-02-18T00:39:08.690237+0000","last_peered":"2024-02-18T00:39:08.690237+0000","last_clean":"2024-02-18T00:39:08.690237+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:39:08.690237+0000","last_undegraded":"2024-02-18T00:39:08.690237+0000","last_fullsized":"2024-02-18T00:39:08.690237+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.633128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.633128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":462,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.691382+0000","last_change":"2024-02-18T00:38:28.669127+0000","last_active":"2024-02-18T00:39:08.691382+0000","last_peered":"2024-02-18T00:39:08.691382+0000","last_clean":"2024-02-18T00:39:08.691382+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:39:08.691382+0000","last_undegraded":"2024-02-18T00:39:08.691382+0000","last_fullsized":"2024-02-18T00:39:08.691382+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:38:28.669051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.669051+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":426,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687898+0000","last_change":"2024-02-18T00:36:36.500976+0000","last_active":"2024-02-18T00:39:08.687898+0000","last_peered":"2024-02-18T00:39:08.687898+0000","last_clean":"2024-02-18T00:39:08.687898+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:39:08.687898+0000","last_undegraded":"2024-02-18T00:39:08.687898+0000","last_fullsized":"2024-02-18T00:39:08.687898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:36.500906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.500906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1011,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.967319+0000","last_change":"2024-02-18T00:37:37.774470+0000","last_active":"2024-02-18T00:39:09.967319+0000","last_peered":"2024-02-18T00:39:09.967319+0000","last_clean":"2024-02-18T00:39:09.967319+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:39:09.967319+0000","last_undegraded":"2024-02-18T00:39:09.967319+0000","last_fullsized":"2024-02-18T00:39:09.967319+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:37:37.774404+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.774404+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":586,"num_read_kb":581,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":351,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.779900+0000","last_change":"2024-02-18T00:36:42.643527+0000","last_active":"2024-02-18T00:39:08.779900+0000","last_peered":"2024-02-18T00:39:08.779900+0000","last_clean":"2024-02-18T00:39:08.779900+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:39:08.779900+0000","last_undegraded":"2024-02-18T00:39:08.779900+0000","last_fullsized":"2024-02-18T00:39:08.779900+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:36:42.643433+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.643433+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":366,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690102+0000","last_change":"2024-02-18T00:38:29.090754+0000","last_active":"2024-02-18T00:39:08.690102+0000","last_peered":"2024-02-18T00:39:08.690102+0000","last_clean":"2024-02-18T00:39:08.690102+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:39:08.690102+0000","last_undegraded":"2024-02-18T00:39:08.690102+0000","last_fullsized":"2024-02-18T00:39:08.690102+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_clean_scrub_stamp":"2024-02-18T00:38:29.090653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.464334+0000","last_change":"2024-02-18T00:36:56.302683+0000","last_active":"2024-02-18T00:39:09.464334+0000","last_peered":"2024-02-18T00:39:09.464334+0000","last_clean":"2024-02-18T00:39:09.464334+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:39:09.464334+0000","last_undegraded":"2024-02-18T00:39:09.464334+0000","last_fullsized":"2024-02-18T00:39:09.464334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.302615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.302615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":257,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461677+0000","last_change":"2024-02-18T00:36:41.801219+0000","last_active":"2024-02-18T00:39:09.461677+0000","last_peered":"2024-02-18T00:39:09.461677+0000","last_clean":"2024-02-18T00:39:09.461677+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:39:09.461677+0000","last_undegraded":"2024-02-18T00:39:09.461677+0000","last_fullsized":"2024-02-18T00:39:09.461677+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-18T00:36:41.801144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.801144+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":421,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461441+0000","last_change":"2024-02-18T00:36:57.791451+0000","last_active":"2024-02-18T00:39:09.461441+0000","last_peered":"2024-02-18T00:39:09.461441+0000","last_clean":"2024-02-18T00:39:09.461441+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:39:09.461441+0000","last_undegraded":"2024-02-18T00:39:09.461441+0000","last_fullsized":"2024-02-18T00:39:09.461441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.791347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.791347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":410,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461131+0000","last_change":"2024-02-18T00:36:49.976027+0000","last_active":"2024-02-18T00:39:09.461131+0000","last_peered":"2024-02-18T00:39:09.461131+0000","last_clean":"2024-02-18T00:39:09.461131+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:39:09.461131+0000","last_undegraded":"2024-02-18T00:39:09.461131+0000","last_fullsized":"2024-02-18T00:39:09.461131+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.975957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.975957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":356,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.464624+0000","last_change":"2024-02-18T00:39:02.635218+0000","last_active":"2024-02-18T00:39:09.464624+0000","last_peered":"2024-02-18T00:39:09.464624+0000","last_clean":"2024-02-18T00:39:09.464624+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:39:09.464624+0000","last_undegraded":"2024-02-18T00:39:09.464624+0000","last_fullsized":"2024-02-18T00:39:09.464624+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.635104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.635104+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":538,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463877+0000","last_change":"2024-02-18T00:37:29.141966+0000","last_active":"2024-02-18T00:39:09.463877+0000","last_peered":"2024-02-18T00:39:09.463877+0000","last_clean":"2024-02-18T00:39:09.463877+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:39:09.463877+0000","last_undegraded":"2024-02-18T00:39:09.463877+0000","last_fullsized":"2024-02-18T00:39:09.463877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:29.141901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.141901+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":415,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688245+0000","last_change":"2024-02-18T00:37:43.309558+0000","last_active":"2024-02-18T00:39:08.688245+0000","last_peered":"2024-02-18T00:39:08.688245+0000","last_clean":"2024-02-18T00:39:08.688245+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:39:08.688245+0000","last_undegraded":"2024-02-18T00:39:08.688245+0000","last_fullsized":"2024-02-18T00:39:08.688245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:37:43.309488+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.309488+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":445,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690298+0000","last_change":"2024-02-18T00:38:31.657857+0000","last_active":"2024-02-18T00:39:08.690298+0000","last_peered":"2024-02-18T00:39:08.690298+0000","last_clean":"2024-02-18T00:39:08.690298+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:39:08.690298+0000","last_undegraded":"2024-02-18T00:39:08.690298+0000","last_fullsized":"2024-02-18T00:39:08.690298+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:31.657778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:31.657778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":404,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461826+0000","last_change":"2024-02-18T00:36:46.369602+0000","last_active":"2024-02-18T00:39:09.461826+0000","last_peered":"2024-02-18T00:39:09.461826+0000","last_clean":"2024-02-18T00:39:09.461826+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:39:09.461826+0000","last_undegraded":"2024-02-18T00:39:09.461826+0000","last_fullsized":"2024-02-18T00:39:09.461826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:46.369503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.369503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":357,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689534+0000","last_change":"2024-02-18T00:38:24.128263+0000","last_active":"2024-02-18T00:39:08.689534+0000","last_peered":"2024-02-18T00:39:08.689534+0000","last_clean":"2024-02-18T00:39:08.689534+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:39:08.689534+0000","last_undegraded":"2024-02-18T00:39:08.689534+0000","last_fullsized":"2024-02-18T00:39:08.689534+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.128170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.128170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":413,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687530+0000","last_change":"2024-02-18T00:37:44.335458+0000","last_active":"2024-02-18T00:39:08.687530+0000","last_peered":"2024-02-18T00:39:08.687530+0000","last_clean":"2024-02-18T00:39:08.687530+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:39:08.687530+0000","last_undegraded":"2024-02-18T00:39:08.687530+0000","last_fullsized":"2024-02-18T00:39:08.687530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":468,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688733+0000","last_change":"2024-02-18T00:37:39.126531+0000","last_active":"2024-02-18T00:39:08.688733+0000","last_peered":"2024-02-18T00:39:08.688733+0000","last_clean":"2024-02-18T00:39:08.688733+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:39:08.688733+0000","last_undegraded":"2024-02-18T00:39:08.688733+0000","last_fullsized":"2024-02-18T00:39:08.688733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:37:39.126398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.126398+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":450,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689900+0000","last_change":"2024-02-18T00:37:45.186258+0000","last_active":"2024-02-18T00:39:08.689900+0000","last_peered":"2024-02-18T00:39:08.689900+0000","last_clean":"2024-02-18T00:39:08.689900+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:39:08.689900+0000","last_undegraded":"2024-02-18T00:39:08.689900+0000","last_fullsized":"2024-02-18T00:39:08.689900+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:37:45.186143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.186143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":415,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461874+0000","last_change":"2024-02-18T00:37:00.804782+0000","last_active":"2024-02-18T00:39:09.461874+0000","last_peered":"2024-02-18T00:39:09.461874+0000","last_clean":"2024-02-18T00:39:09.461874+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:39:09.461874+0000","last_undegraded":"2024-02-18T00:39:09.461874+0000","last_fullsized":"2024-02-18T00:39:09.461874+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.804656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":357,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689693+0000","last_change":"2024-02-18T00:38:18.292304+0000","last_active":"2024-02-18T00:39:08.689693+0000","last_peered":"2024-02-18T00:39:08.689693+0000","last_clean":"2024-02-18T00:39:08.689693+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:39:08.689693+0000","last_undegraded":"2024-02-18T00:39:08.689693+0000","last_fullsized":"2024-02-18T00:39:08.689693+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:18.292215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:18.292215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":482,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460595+0000","last_change":"2024-02-18T00:37:46.472303+0000","last_active":"2024-02-18T00:39:09.460595+0000","last_peered":"2024-02-18T00:39:09.460595+0000","last_clean":"2024-02-18T00:39:09.460595+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:39:09.460595+0000","last_undegraded":"2024-02-18T00:39:09.460595+0000","last_fullsized":"2024-02-18T00:39:09.460595+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:37:46.472227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.472227+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":417,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687789+0000","last_change":"2024-02-18T00:37:27.357689+0000","last_active":"2024-02-18T00:39:08.687789+0000","last_peered":"2024-02-18T00:39:08.687789+0000","last_clean":"2024-02-18T00:39:08.687789+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:39:08.687789+0000","last_undegraded":"2024-02-18T00:39:08.687789+0000","last_fullsized":"2024-02-18T00:39:08.687789+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.357610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.357610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":417,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462210+0000","last_change":"2024-02-18T00:36:55.861115+0000","last_active":"2024-02-18T00:39:09.462210+0000","last_peered":"2024-02-18T00:39:09.462210+0000","last_clean":"2024-02-18T00:39:09.462210+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:39:09.462210+0000","last_undegraded":"2024-02-18T00:39:09.462210+0000","last_fullsized":"2024-02-18T00:39:09.462210+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:55.861044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.861044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":422,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688430+0000","last_change":"2024-02-18T00:38:18.211396+0000","last_active":"2024-02-18T00:39:08.688430+0000","last_peered":"2024-02-18T00:39:08.688430+0000","last_clean":"2024-02-18T00:39:08.688430+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:39:08.688430+0000","last_undegraded":"2024-02-18T00:39:08.688430+0000","last_fullsized":"2024-02-18T00:39:08.688430+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:18.211326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:38:18.211326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":356,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.589701+0000","last_change":"2024-02-18T00:39:09.589674+0000","last_active":"2024-02-18T00:39:09.589701+0000","last_peered":"2024-02-18T00:39:09.589701+0000","last_clean":"2024-02-18T00:39:09.589701+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:39:09.589701+0000","last_undegraded":"2024-02-18T00:39:09.589701+0000","last_fullsized":"2024-02-18T00:39:09.589701+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:09.589601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.589601+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":477,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462336+0000","last_change":"2024-02-18T00:39:07.653709+0000","last_active":"2024-02-18T00:39:09.462336+0000","last_peered":"2024-02-18T00:39:09.462336+0000","last_clean":"2024-02-18T00:39:09.462336+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:39:09.462336+0000","last_undegraded":"2024-02-18T00:39:09.462336+0000","last_fullsized":"2024-02-18T00:39:09.462336+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:39:07.653633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.653633+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":419,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780550+0000","last_change":"2024-02-18T00:38:13.745638+0000","last_active":"2024-02-18T00:39:08.780550+0000","last_peered":"2024-02-18T00:39:08.780550+0000","last_clean":"2024-02-18T00:39:08.780550+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:39:08.780550+0000","last_undegraded":"2024-02-18T00:39:08.780550+0000","last_fullsized":"2024-02-18T00:39:08.780550+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:13.745551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:13.745551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":408,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462300+0000","last_change":"2024-02-18T00:37:38.687198+0000","last_active":"2024-02-18T00:39:09.462300+0000","last_peered":"2024-02-18T00:39:09.462300+0000","last_clean":"2024-02-18T00:39:09.462300+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:39:09.462300+0000","last_undegraded":"2024-02-18T00:39:09.462300+0000","last_fullsized":"2024-02-18T00:39:09.462300+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:38.687129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.687129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":319,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781715+0000","last_change":"2024-02-18T00:39:00.786552+0000","last_active":"2024-02-18T00:39:08.781715+0000","last_peered":"2024-02-18T00:39:08.781715+0000","last_clean":"2024-02-18T00:39:08.781715+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:39:08.781715+0000","last_undegraded":"2024-02-18T00:39:08.781715+0000","last_fullsized":"2024-02-18T00:39:08.781715+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:00.786470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:00.786470+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":366,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688016+0000","last_change":"2024-02-18T00:38:20.206553+0000","last_active":"2024-02-18T00:39:08.688016+0000","last_peered":"2024-02-18T00:39:08.688016+0000","last_clean":"2024-02-18T00:39:08.688016+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:39:08.688016+0000","last_undegraded":"2024-02-18T00:39:08.688016+0000","last_fullsized":"2024-02-18T00:39:08.688016+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.206476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.206476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":480,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461039+0000","last_change":"2024-02-18T00:36:58.819567+0000","last_active":"2024-02-18T00:39:09.461039+0000","last_peered":"2024-02-18T00:39:09.461039+0000","last_clean":"2024-02-18T00:39:09.461039+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:39:09.461039+0000","last_undegraded":"2024-02-18T00:39:09.461039+0000","last_fullsized":"2024-02-18T00:39:09.461039+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:36:58.819503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.819503+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":430,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461357+0000","last_change":"2024-02-18T00:37:41.637636+0000","last_active":"2024-02-18T00:39:09.461357+0000","last_peered":"2024-02-18T00:39:09.461357+0000","last_clean":"2024-02-18T00:39:09.461357+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:39:09.461357+0000","last_undegraded":"2024-02-18T00:39:09.461357+0000","last_fullsized":"2024-02-18T00:39:09.461357+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.637550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.637550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":535,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462216+0000","last_change":"2024-02-18T00:37:27.642761+0000","last_active":"2024-02-18T00:39:09.462216+0000","last_peered":"2024-02-18T00:39:09.462216+0000","last_clean":"2024-02-18T00:39:09.462216+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:39:09.462216+0000","last_undegraded":"2024-02-18T00:39:09.462216+0000","last_fullsized":"2024-02-18T00:39:09.462216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:27.642661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.642661+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":418,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460841+0000","last_change":"2024-02-18T00:36:44.367116+0000","last_active":"2024-02-18T00:39:09.460841+0000","last_peered":"2024-02-18T00:39:09.460841+0000","last_clean":"2024-02-18T00:39:09.460841+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:39:09.460841+0000","last_undegraded":"2024-02-18T00:39:09.460841+0000","last_fullsized":"2024-02-18T00:39:09.460841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.367032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.367032+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":363,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.779787+0000","last_change":"2024-02-18T00:37:46.639988+0000","last_active":"2024-02-18T00:39:08.779787+0000","last_peered":"2024-02-18T00:39:08.779787+0000","last_clean":"2024-02-18T00:39:08.779787+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:39:08.779787+0000","last_undegraded":"2024-02-18T00:39:08.779787+0000","last_fullsized":"2024-02-18T00:39:08.779787+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:46.639901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.639901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":462,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462080+0000","last_change":"2024-02-18T00:38:29.545852+0000","last_active":"2024-02-18T00:39:09.462080+0000","last_peered":"2024-02-18T00:39:09.462080+0000","last_clean":"2024-02-18T00:39:09.462080+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:39:09.462080+0000","last_undegraded":"2024-02-18T00:39:09.462080+0000","last_fullsized":"2024-02-18T00:39:09.462080+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:38:29.545783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:29.545783+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":415,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463617+0000","last_change":"2024-02-18T00:37:03.239555+0000","last_active":"2024-02-18T00:39:09.463617+0000","last_peered":"2024-02-18T00:39:09.463617+0000","last_clean":"2024-02-18T00:39:09.463617+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:39:09.463617+0000","last_undegraded":"2024-02-18T00:39:09.463617+0000","last_fullsized":"2024-02-18T00:39:09.463617+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.239451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.239451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"341'10","reported_seq":549,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460755+0000","last_change":"2024-02-18T00:36:59.830567+0000","last_active":"2024-02-18T00:39:09.460755+0000","last_peered":"2024-02-18T00:39:09.460755+0000","last_clean":"2024-02-18T00:39:09.460755+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:39:09.460755+0000","last_undegraded":"2024-02-18T00:39:09.460755+0000","last_fullsized":"2024-02-18T00:39:09.460755+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'10","last_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_deep_scrub":"341'10","last_deep_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.830497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690193+0000","last_change":"2024-02-18T00:38:20.222342+0000","last_active":"2024-02-18T00:39:08.690193+0000","last_peered":"2024-02-18T00:39:08.690193+0000","last_clean":"2024-02-18T00:39:08.690193+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:39:08.690193+0000","last_undegraded":"2024-02-18T00:39:08.690193+0000","last_fullsized":"2024-02-18T00:39:08.690193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.222269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.222269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":352,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462768+0000","last_change":"2024-02-18T00:37:39.639314+0000","last_active":"2024-02-18T00:39:09.462768+0000","last_peered":"2024-02-18T00:39:09.462768+0000","last_clean":"2024-02-18T00:39:09.462768+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:39:09.462768+0000","last_undegraded":"2024-02-18T00:39:09.462768+0000","last_fullsized":"2024-02-18T00:39:09.462768+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:39.639232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.639232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":436,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688823+0000","last_change":"2024-02-18T00:38:33.648037+0000","last_active":"2024-02-18T00:39:08.688823+0000","last_peered":"2024-02-18T00:39:08.688823+0000","last_clean":"2024-02-18T00:39:08.688823+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:39:08.688823+0000","last_undegraded":"2024-02-18T00:39:08.688823+0000","last_fullsized":"2024-02-18T00:39:08.688823+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:38:33.647958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.647958+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":419,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781545+0000","last_change":"2024-02-18T00:36:44.617522+0000","last_active":"2024-02-18T00:39:08.781545+0000","last_peered":"2024-02-18T00:39:08.781545+0000","last_clean":"2024-02-18T00:39:08.781545+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:39:08.781545+0000","last_undegraded":"2024-02-18T00:39:08.781545+0000","last_fullsized":"2024-02-18T00:39:08.781545+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.617443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.617443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":408,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463080+0000","last_change":"2024-02-18T00:36:50.970267+0000","last_active":"2024-02-18T00:39:09.463080+0000","last_peered":"2024-02-18T00:39:09.463080+0000","last_clean":"2024-02-18T00:39:09.463080+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:39:09.463080+0000","last_undegraded":"2024-02-18T00:39:09.463080+0000","last_fullsized":"2024-02-18T00:39:09.463080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:50.970192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.970192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":414,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462400+0000","last_change":"2024-02-18T00:39:01.603631+0000","last_active":"2024-02-18T00:39:09.462400+0000","last_peered":"2024-02-18T00:39:09.462400+0000","last_clean":"2024-02-18T00:39:09.462400+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:39:09.462400+0000","last_undegraded":"2024-02-18T00:39:09.462400+0000","last_fullsized":"2024-02-18T00:39:09.462400+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:01.603551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.603551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":351,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.464581+0000","last_change":"2024-02-18T00:36:42.355194+0000","last_active":"2024-02-18T00:39:09.464581+0000","last_peered":"2024-02-18T00:39:09.464581+0000","last_clean":"2024-02-18T00:39:09.464581+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:39:09.464581+0000","last_undegraded":"2024-02-18T00:39:09.464581+0000","last_fullsized":"2024-02-18T00:39:09.464581+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.355114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.355114+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":434,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690012+0000","last_change":"2024-02-18T00:36:52.343624+0000","last_active":"2024-02-18T00:39:08.690012+0000","last_peered":"2024-02-18T00:39:08.690012+0000","last_clean":"2024-02-18T00:39:08.690012+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:39:08.690012+0000","last_undegraded":"2024-02-18T00:39:08.690012+0000","last_fullsized":"2024-02-18T00:39:08.690012+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:36:52.343556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.343556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":417,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690921+0000","last_change":"2024-02-18T00:37:27.830300+0000","last_active":"2024-02-18T00:39:08.690921+0000","last_peered":"2024-02-18T00:39:08.690921+0000","last_clean":"2024-02-18T00:39:08.690921+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:39:08.690921+0000","last_undegraded":"2024-02-18T00:39:08.690921+0000","last_fullsized":"2024-02-18T00:39:08.690921+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.830225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.830225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":418,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688903+0000","last_change":"2024-02-18T00:38:24.601383+0000","last_active":"2024-02-18T00:39:08.688903+0000","last_peered":"2024-02-18T00:39:08.688903+0000","last_clean":"2024-02-18T00:39:08.688903+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:39:08.688903+0000","last_undegraded":"2024-02-18T00:39:08.688903+0000","last_fullsized":"2024-02-18T00:39:08.688903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.601308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.601308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462823+0000","last_change":"2024-02-18T00:37:37.848598+0000","last_active":"2024-02-18T00:39:09.462823+0000","last_peered":"2024-02-18T00:39:09.462823+0000","last_clean":"2024-02-18T00:39:09.462823+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:39:09.462823+0000","last_undegraded":"2024-02-18T00:39:09.462823+0000","last_fullsized":"2024-02-18T00:39:09.462823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.848524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.848524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":354,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781627+0000","last_change":"2024-02-18T00:38:15.658298+0000","last_active":"2024-02-18T00:39:08.781627+0000","last_peered":"2024-02-18T00:39:08.781627+0000","last_clean":"2024-02-18T00:39:08.781627+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:39:08.781627+0000","last_undegraded":"2024-02-18T00:39:08.781627+0000","last_fullsized":"2024-02-18T00:39:08.781627+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:15.658206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:15.658206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":416,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463896+0000","last_change":"2024-02-18T00:37:41.880926+0000","last_active":"2024-02-18T00:39:09.463896+0000","last_peered":"2024-02-18T00:39:09.463896+0000","last_clean":"2024-02-18T00:39:09.463896+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:39:09.463896+0000","last_undegraded":"2024-02-18T00:39:09.463896+0000","last_fullsized":"2024-02-18T00:39:09.463896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:37:41.880855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.880855+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":417,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462372+0000","last_change":"2024-02-18T00:38:33.486555+0000","last_active":"2024-02-18T00:39:09.462372+0000","last_peered":"2024-02-18T00:39:09.462372+0000","last_clean":"2024-02-18T00:39:09.462372+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:39:09.462372+0000","last_undegraded":"2024-02-18T00:39:09.462372+0000","last_fullsized":"2024-02-18T00:39:09.462372+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:33.486426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.486426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":424,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.867363+0000","last_change":"2024-02-18T00:39:09.867335+0000","last_active":"2024-02-18T00:39:09.867363+0000","last_peered":"2024-02-18T00:39:09.867363+0000","last_clean":"2024-02-18T00:39:09.867363+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:39:09.867363+0000","last_undegraded":"2024-02-18T00:39:09.867363+0000","last_fullsized":"2024-02-18T00:39:09.867363+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:09.867259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.867259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":317,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688190+0000","last_change":"2024-02-18T00:37:29.374778+0000","last_active":"2024-02-18T00:39:08.688190+0000","last_peered":"2024-02-18T00:39:08.688190+0000","last_clean":"2024-02-18T00:39:08.688190+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:39:08.688190+0000","last_undegraded":"2024-02-18T00:39:08.688190+0000","last_fullsized":"2024-02-18T00:39:08.688190+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:29.374712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.374712+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":361,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460410+0000","last_change":"2024-02-18T00:39:01.514281+0000","last_active":"2024-02-18T00:39:09.460410+0000","last_peered":"2024-02-18T00:39:09.460410+0000","last_clean":"2024-02-18T00:39:09.460410+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:39:09.460410+0000","last_undegraded":"2024-02-18T00:39:09.460410+0000","last_fullsized":"2024-02-18T00:39:09.460410+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:01.514191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.514191+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":467,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463457+0000","last_change":"2024-02-18T00:37:45.838598+0000","last_active":"2024-02-18T00:39:09.463457+0000","last_peered":"2024-02-18T00:39:09.463457+0000","last_clean":"2024-02-18T00:39:09.463457+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:39:09.463457+0000","last_undegraded":"2024-02-18T00:39:09.463457+0000","last_fullsized":"2024-02-18T00:39:09.463457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:37:45.838523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.838523+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":419,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781890+0000","last_change":"2024-02-18T00:39:06.776476+0000","last_active":"2024-02-18T00:39:08.781890+0000","last_peered":"2024-02-18T00:39:08.781890+0000","last_clean":"2024-02-18T00:39:08.781890+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:39:08.781890+0000","last_undegraded":"2024-02-18T00:39:08.781890+0000","last_fullsized":"2024-02-18T00:39:08.781890+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:06.776398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.776398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":326,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688218+0000","last_change":"2024-02-18T00:36:59.243785+0000","last_active":"2024-02-18T00:39:08.688218+0000","last_peered":"2024-02-18T00:39:08.688218+0000","last_clean":"2024-02-18T00:39:08.688218+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:39:08.688218+0000","last_undegraded":"2024-02-18T00:39:08.688218+0000","last_fullsized":"2024-02-18T00:39:08.688218+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:59.243699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.243699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463213+0000","last_change":"2024-02-18T00:37:44.869935+0000","last_active":"2024-02-18T00:39:09.463213+0000","last_peered":"2024-02-18T00:39:09.463213+0000","last_clean":"2024-02-18T00:39:09.463213+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:39:09.463213+0000","last_undegraded":"2024-02-18T00:39:09.463213+0000","last_fullsized":"2024-02-18T00:39:09.463213+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.869862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.869862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":317,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688938+0000","last_change":"2024-02-18T00:38:19.252878+0000","last_active":"2024-02-18T00:39:08.688938+0000","last_peered":"2024-02-18T00:39:08.688938+0000","last_clean":"2024-02-18T00:39:08.688938+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:39:08.688938+0000","last_undegraded":"2024-02-18T00:39:08.688938+0000","last_fullsized":"2024-02-18T00:39:08.688938+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:19.252808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:19.252808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":472,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460608+0000","last_change":"2024-02-18T00:36:53.401897+0000","last_active":"2024-02-18T00:39:09.460608+0000","last_peered":"2024-02-18T00:39:09.460608+0000","last_clean":"2024-02-18T00:39:09.460608+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:39:09.460608+0000","last_undegraded":"2024-02-18T00:39:09.460608+0000","last_fullsized":"2024-02-18T00:39:09.460608+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:36:53.401819+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.401819+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":421,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460877+0000","last_change":"2024-02-18T00:39:06.562645+0000","last_active":"2024-02-18T00:39:09.460877+0000","last_peered":"2024-02-18T00:39:09.460877+0000","last_clean":"2024-02-18T00:39:09.460877+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:39:09.460877+0000","last_undegraded":"2024-02-18T00:39:09.460877+0000","last_fullsized":"2024-02-18T00:39:09.460877+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:06.562567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.562567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":558,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780004+0000","last_change":"2024-02-18T00:36:38.557419+0000","last_active":"2024-02-18T00:39:08.780004+0000","last_peered":"2024-02-18T00:39:08.780004+0000","last_clean":"2024-02-18T00:39:08.780004+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:39:08.780004+0000","last_undegraded":"2024-02-18T00:39:08.780004+0000","last_fullsized":"2024-02-18T00:39:08.780004+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:36:38.557298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.557298+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":325,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462018+0000","last_change":"2024-02-18T00:36:49.727241+0000","last_active":"2024-02-18T00:39:09.462018+0000","last_peered":"2024-02-18T00:39:09.462018+0000","last_clean":"2024-02-18T00:39:09.462018+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:39:09.462018+0000","last_undegraded":"2024-02-18T00:39:09.462018+0000","last_fullsized":"2024-02-18T00:39:09.462018+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:36:49.727157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.727157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":312,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460760+0000","last_change":"2024-02-18T00:38:17.776297+0000","last_active":"2024-02-18T00:39:09.460760+0000","last_peered":"2024-02-18T00:39:09.460760+0000","last_clean":"2024-02-18T00:39:09.460760+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:39:09.460760+0000","last_undegraded":"2024-02-18T00:39:09.460760+0000","last_fullsized":"2024-02-18T00:39:09.460760+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:17.776238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:17.776238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":446,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.782013+0000","last_change":"2024-02-18T00:36:09.529971+0000","last_active":"2024-02-18T00:39:08.782013+0000","last_peered":"2024-02-18T00:39:08.782013+0000","last_clean":"2024-02-18T00:39:08.782013+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:39:08.782013+0000","last_undegraded":"2024-02-18T00:39:08.782013+0000","last_fullsized":"2024-02-18T00:39:08.782013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:36:09.529903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:09.529903+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":412,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462938+0000","last_change":"2024-02-18T00:36:56.850409+0000","last_active":"2024-02-18T00:39:09.462938+0000","last_peered":"2024-02-18T00:39:09.462938+0000","last_clean":"2024-02-18T00:39:09.462938+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:39:09.462938+0000","last_undegraded":"2024-02-18T00:39:09.462938+0000","last_fullsized":"2024-02-18T00:39:09.462938+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.850326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.850326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":415,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460433+0000","last_change":"2024-02-18T00:36:52.372043+0000","last_active":"2024-02-18T00:39:09.460433+0000","last_peered":"2024-02-18T00:39:09.460433+0000","last_clean":"2024-02-18T00:39:09.460433+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:39:09.460433+0000","last_undegraded":"2024-02-18T00:39:09.460433+0000","last_fullsized":"2024-02-18T00:39:09.460433+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.371970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.371970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":416,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690788+0000","last_change":"2024-02-18T00:37:40.707426+0000","last_active":"2024-02-18T00:39:08.690788+0000","last_peered":"2024-02-18T00:39:08.690788+0000","last_clean":"2024-02-18T00:39:08.690788+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:39:08.690788+0000","last_undegraded":"2024-02-18T00:39:08.690788+0000","last_fullsized":"2024-02-18T00:39:08.690788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.707359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.707359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":370,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.687976+0000","last_change":"2024-02-18T00:37:48.395278+0000","last_active":"2024-02-18T00:39:08.687976+0000","last_peered":"2024-02-18T00:39:08.687976+0000","last_clean":"2024-02-18T00:39:08.687976+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:39:08.687976+0000","last_undegraded":"2024-02-18T00:39:08.687976+0000","last_fullsized":"2024-02-18T00:39:08.687976+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:37:48.395205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.395205+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":445,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688773+0000","last_change":"2024-02-18T00:37:39.385746+0000","last_active":"2024-02-18T00:39:08.688773+0000","last_peered":"2024-02-18T00:39:08.688773+0000","last_clean":"2024-02-18T00:39:08.688773+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:39:08.688773+0000","last_undegraded":"2024-02-18T00:39:08.688773+0000","last_fullsized":"2024-02-18T00:39:08.688773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:39.385675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.385675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":430,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781306+0000","last_change":"2024-02-18T00:37:36.615647+0000","last_active":"2024-02-18T00:39:08.781306+0000","last_peered":"2024-02-18T00:39:08.781306+0000","last_clean":"2024-02-18T00:39:08.781306+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:39:08.781306+0000","last_undegraded":"2024-02-18T00:39:08.781306+0000","last_fullsized":"2024-02-18T00:39:08.781306+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.615568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.615568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":418,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.691234+0000","last_change":"2024-02-18T00:38:15.534086+0000","last_active":"2024-02-18T00:39:08.691234+0000","last_peered":"2024-02-18T00:39:08.691234+0000","last_clean":"2024-02-18T00:39:08.691234+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:39:08.691234+0000","last_undegraded":"2024-02-18T00:39:08.691234+0000","last_fullsized":"2024-02-18T00:39:08.691234+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:15.533986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:38:15.533986+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690457+0000","last_change":"2024-02-18T00:37:34.122873+0000","last_active":"2024-02-18T00:39:08.690457+0000","last_peered":"2024-02-18T00:39:08.690457+0000","last_clean":"2024-02-18T00:39:08.690457+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:39:08.690457+0000","last_undegraded":"2024-02-18T00:39:08.690457+0000","last_fullsized":"2024-02-18T00:39:08.690457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.122764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.122764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":354,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.780607+0000","last_change":"2024-02-18T00:37:40.562387+0000","last_active":"2024-02-18T00:39:08.780607+0000","last_peered":"2024-02-18T00:39:08.780607+0000","last_clean":"2024-02-18T00:39:08.780607+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:39:08.780607+0000","last_undegraded":"2024-02-18T00:39:08.780607+0000","last_fullsized":"2024-02-18T00:39:08.780607+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.562306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.562306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":448,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460080+0000","last_change":"2024-02-18T00:37:43.883242+0000","last_active":"2024-02-18T00:39:09.460080+0000","last_peered":"2024-02-18T00:39:09.460080+0000","last_clean":"2024-02-18T00:39:09.460080+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:39:09.460080+0000","last_undegraded":"2024-02-18T00:39:09.460080+0000","last_fullsized":"2024-02-18T00:39:09.460080+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:37:43.883172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.883172+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":432,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462996+0000","last_change":"2024-02-18T00:36:52.928195+0000","last_active":"2024-02-18T00:39:09.462996+0000","last_peered":"2024-02-18T00:39:09.462996+0000","last_clean":"2024-02-18T00:39:09.462996+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:39:09.462996+0000","last_undegraded":"2024-02-18T00:39:09.462996+0000","last_fullsized":"2024-02-18T00:39:09.462996+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:36:52.928076+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.928076+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":413,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689415+0000","last_change":"2024-02-18T00:37:03.248349+0000","last_active":"2024-02-18T00:39:08.689415+0000","last_peered":"2024-02-18T00:39:08.689415+0000","last_clean":"2024-02-18T00:39:08.689415+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:39:08.689415+0000","last_undegraded":"2024-02-18T00:39:08.689415+0000","last_fullsized":"2024-02-18T00:39:08.689415+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.248277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.248277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":401,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.463201+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:39:09.463201+0000","last_peered":"2024-02-18T00:39:09.463201+0000","last_clean":"2024-02-18T00:39:09.463201+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:39:09.463201+0000","last_undegraded":"2024-02-18T00:39:09.463201+0000","last_fullsized":"2024-02-18T00:39:09.463201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":365,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460065+0000","last_change":"2024-02-18T00:38:27.705513+0000","last_active":"2024-02-18T00:39:09.460065+0000","last_peered":"2024-02-18T00:39:09.460065+0000","last_clean":"2024-02-18T00:39:09.460065+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:39:09.460065+0000","last_undegraded":"2024-02-18T00:39:09.460065+0000","last_fullsized":"2024-02-18T00:39:09.460065+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:27.705442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:27.705442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":473,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688854+0000","last_change":"2024-02-18T00:37:48.216997+0000","last_active":"2024-02-18T00:39:08.688854+0000","last_peered":"2024-02-18T00:39:08.688854+0000","last_clean":"2024-02-18T00:39:08.688854+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:39:08.688854+0000","last_undegraded":"2024-02-18T00:39:08.688854+0000","last_fullsized":"2024-02-18T00:39:08.688854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:37:48.216929+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.216929+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":312,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461927+0000","last_change":"2024-02-18T00:38:22.640818+0000","last_active":"2024-02-18T00:39:09.461927+0000","last_peered":"2024-02-18T00:39:09.461927+0000","last_clean":"2024-02-18T00:39:09.461927+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:39:09.461927+0000","last_undegraded":"2024-02-18T00:39:09.461927+0000","last_fullsized":"2024-02-18T00:39:09.461927+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:22.640720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.640720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":431,"reported_epoch":446,"state":"active+clean","last_fresh":"2024-02-18T00:39:10.582509+0000","last_change":"2024-02-18T00:39:10.582509+0000","last_active":"2024-02-18T00:39:10.582509+0000","last_peered":"2024-02-18T00:39:10.582509+0000","last_clean":"2024-02-18T00:39:10.582509+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:39:10.582509+0000","last_undegraded":"2024-02-18T00:39:10.582509+0000","last_fullsized":"2024-02-18T00:39:10.582509+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:10.582430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:10.582430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":418,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462092+0000","last_change":"2024-02-18T00:36:45.412472+0000","last_active":"2024-02-18T00:39:09.462092+0000","last_peered":"2024-02-18T00:39:09.462092+0000","last_clean":"2024-02-18T00:39:09.462092+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:39:09.462092+0000","last_undegraded":"2024-02-18T00:39:09.462092+0000","last_fullsized":"2024-02-18T00:39:09.462092+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.412406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.412406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":357,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.690036+0000","last_change":"2024-02-18T00:39:03.503323+0000","last_active":"2024-02-18T00:39:08.690036+0000","last_peered":"2024-02-18T00:39:08.690036+0000","last_clean":"2024-02-18T00:39:08.690036+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:39:08.690036+0000","last_undegraded":"2024-02-18T00:39:08.690036+0000","last_fullsized":"2024-02-18T00:39:08.690036+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:03.503256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.503256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":443,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.689843+0000","last_change":"2024-02-18T00:36:51.303043+0000","last_active":"2024-02-18T00:39:08.689843+0000","last_peered":"2024-02-18T00:39:08.689843+0000","last_clean":"2024-02-18T00:39:08.689843+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:39:08.689843+0000","last_undegraded":"2024-02-18T00:39:08.689843+0000","last_fullsized":"2024-02-18T00:39:08.689843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:36:51.302920+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.302920+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":314,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.781583+0000","last_change":"2024-02-18T00:36:52.758965+0000","last_active":"2024-02-18T00:39:08.781583+0000","last_peered":"2024-02-18T00:39:08.781583+0000","last_clean":"2024-02-18T00:39:08.781583+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:39:08.781583+0000","last_undegraded":"2024-02-18T00:39:08.781583+0000","last_fullsized":"2024-02-18T00:39:08.781583+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.758904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.758904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":317,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688929+0000","last_change":"2024-02-18T00:39:07.344557+0000","last_active":"2024-02-18T00:39:08.688929+0000","last_peered":"2024-02-18T00:39:08.688929+0000","last_clean":"2024-02-18T00:39:08.688929+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:39:08.688929+0000","last_undegraded":"2024-02-18T00:39:08.688929+0000","last_fullsized":"2024-02-18T00:39:08.688929+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:07.344483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.344483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":406,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461210+0000","last_change":"2024-02-18T00:36:40.825174+0000","last_active":"2024-02-18T00:39:09.461210+0000","last_peered":"2024-02-18T00:39:09.461210+0000","last_clean":"2024-02-18T00:39:09.461210+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:39:09.461210+0000","last_undegraded":"2024-02-18T00:39:09.461210+0000","last_fullsized":"2024-02-18T00:39:09.461210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:40.825055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:36:40.825055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":361,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462734+0000","last_change":"2024-02-18T00:38:24.768769+0000","last_active":"2024-02-18T00:39:09.462734+0000","last_peered":"2024-02-18T00:39:09.462734+0000","last_clean":"2024-02-18T00:39:09.462734+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:39:09.462734+0000","last_undegraded":"2024-02-18T00:39:09.462734+0000","last_fullsized":"2024-02-18T00:39:09.462734+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.768673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.768673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":451,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688214+0000","last_change":"2024-02-18T00:38:17.269122+0000","last_active":"2024-02-18T00:39:08.688214+0000","last_peered":"2024-02-18T00:39:08.688214+0000","last_clean":"2024-02-18T00:39:08.688214+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:39:08.688214+0000","last_undegraded":"2024-02-18T00:39:08.688214+0000","last_fullsized":"2024-02-18T00:39:08.688214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:38:17.269050+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:38:17.269050+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":412,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462603+0000","last_change":"2024-02-18T00:38:22.776453+0000","last_active":"2024-02-18T00:39:09.462603+0000","last_peered":"2024-02-18T00:39:09.462603+0000","last_clean":"2024-02-18T00:39:09.462603+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:39:09.462603+0000","last_undegraded":"2024-02-18T00:39:09.462603+0000","last_fullsized":"2024-02-18T00:39:09.462603+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:22.776331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.776331+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":427,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688998+0000","last_change":"2024-02-18T00:39:04.586023+0000","last_active":"2024-02-18T00:39:08.688998+0000","last_peered":"2024-02-18T00:39:08.688998+0000","last_clean":"2024-02-18T00:39:08.688998+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:39:08.688998+0000","last_undegraded":"2024-02-18T00:39:08.688998+0000","last_fullsized":"2024-02-18T00:39:08.688998+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:04.585957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:04.585957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":411,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688695+0000","last_change":"2024-02-18T00:37:42.183653+0000","last_active":"2024-02-18T00:39:08.688695+0000","last_peered":"2024-02-18T00:39:08.688695+0000","last_clean":"2024-02-18T00:39:08.688695+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:39:08.688695+0000","last_undegraded":"2024-02-18T00:39:08.688695+0000","last_fullsized":"2024-02-18T00:39:08.688695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.183568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.183568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":352,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462629+0000","last_change":"2024-02-18T00:38:31.741746+0000","last_active":"2024-02-18T00:39:09.462629+0000","last_peered":"2024-02-18T00:39:09.462629+0000","last_clean":"2024-02-18T00:39:09.462629+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:39:09.462629+0000","last_undegraded":"2024-02-18T00:39:09.462629+0000","last_fullsized":"2024-02-18T00:39:09.462629+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:31.741672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:31.741672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":497,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.461222+0000","last_change":"2024-02-18T00:39:06.697405+0000","last_active":"2024-02-18T00:39:09.461222+0000","last_peered":"2024-02-18T00:39:09.461222+0000","last_clean":"2024-02-18T00:39:09.461222+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:39:09.461222+0000","last_undegraded":"2024-02-18T00:39:09.461222+0000","last_fullsized":"2024-02-18T00:39:09.461222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:39:06.697317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.697317+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":426,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.460175+0000","last_change":"2024-02-18T00:37:45.454134+0000","last_active":"2024-02-18T00:39:09.460175+0000","last_peered":"2024-02-18T00:39:09.460175+0000","last_clean":"2024-02-18T00:39:09.460175+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:39:09.460175+0000","last_undegraded":"2024-02-18T00:39:09.460175+0000","last_fullsized":"2024-02-18T00:39:09.460175+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.454069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":413,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:08.688390+0000","last_change":"2024-02-18T00:37:36.119369+0000","last_active":"2024-02-18T00:39:08.688390+0000","last_peered":"2024-02-18T00:39:08.688390+0000","last_clean":"2024-02-18T00:39:08.688390+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:39:08.688390+0000","last_undegraded":"2024-02-18T00:39:08.688390+0000","last_fullsized":"2024-02-18T00:39:08.688390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.119291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.119291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":415,"reported_epoch":445,"state":"active+clean","last_fresh":"2024-02-18T00:39:09.462100+0000","last_change":"2024-02-18T00:37:34.860293+0000","last_active":"2024-02-18T00:39:09.462100+0000","last_peered":"2024-02-18T00:39:09.462100+0000","last_clean":"2024-02-18T00:39:09.462100+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:39:09.462100+0000","last_undegraded":"2024-02-18T00:39:09.462100+0000","last_fullsized":"2024-02-18T00:39:09.462100+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.860214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.860214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":215,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001497,"num_objects":160,"num_object_clones":3,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":106,"num_read_kb":9,"num_write":601,"num_write_kb":4500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":9,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13942784,"data_stored":12004530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1891,"ondisk_log_size":1891,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":586,"num_read_kb":581,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266811,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463041,"num_pgs":101,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314372,"kb_used_data":8956,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93430780,"statfs":{"total":95995035648,"available":95673118720,"internally_reserved":0,"allocated":9170944,"data_stored":6700416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.67600000000000005,"15min":1.2529999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69099999999999995,"5min":13.935,"15min":69.194999999999993},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.66300000000000003,"15min":0.73199999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64600000000000002,"5min":10.433,"15min":10.433},"last":0.378}]},{"osd":1,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57899999999999996,"15min":1.327},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.68300000000000005,"5min":5.2290000000000001,"15min":76.108999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.61399999999999999,"15min":1.3160000000000001},"min":{"1min":0.32800000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.68000000000000005,"5min":7.899,"15min":71.872},"last":0.63600000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.65800000000000003,"15min":0.80700000000000005},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.70799999999999996,"5min":11.855,"15min":39.939999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.63100000000000001,"15min":0.98299999999999998},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":4.1600000000000001,"15min":39.673000000000002},"last":0.621}]},{"osd":3,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.69899999999999995,"15min":0.83599999999999997},"min":{"1min":0.372,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63300000000000001,"5min":8.3420000000000005,"15min":21.219000000000001},"last":0.496},{"interface":"front","average":{"1min":0.505,"5min":0.64000000000000001,"15min":0.78100000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69199999999999995,"5min":5.0069999999999997,"15min":14.007999999999999},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.49099999999999999,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61099999999999999,"5min":4.4359999999999999,"15min":8.1479999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.28000000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70299999999999996,"5min":7.907,"15min":35.18},"last":0.60799999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.57199999999999995,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.628,"5min":5.2539999999999996,"15min":10.606},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.57499999999999996,"15min":0.70199999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67200000000000004,"5min":5.5899999999999999,"15min":35.119},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66500000000000004,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.65000000000000002,"15min":0.72599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":5.7030000000000003,"15min":32.985999999999997},"last":0.54900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":106,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306364,"kb_used_data":5044,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438788,"statfs":{"total":95995035648,"available":95681318912,"internally_reserved":0,"allocated":5165056,"data_stored":2708363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34100000000000003,"15min":0.63600000000000001},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.193},"max":{"1min":0.42899999999999999,"5min":0.79600000000000004,"15min":32.106999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38600000000000001,"15min":0.68300000000000005},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":2.7040000000000002,"15min":43.137},"last":0.442}]},{"osd":2,"last update":"Sun Feb 18 00:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.96299999999999997,"15min":0.75800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":19.241,"15min":49.201000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.77400000000000002,"15min":0.58899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.217},"max":{"1min":0.83199999999999996,"5min":10.654,"15min":17.596},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Feb 18 00:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.67300000000000004,"15min":0.74399999999999999},"min":{"1min":0.28199999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68200000000000005,"5min":19.274000000000001,"15min":37.524000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.71399999999999997,"15min":0.76200000000000001},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59799999999999998,"5min":22.989000000000001,"15min":32.036000000000001},"last":0.59499999999999997}]},{"osd":4,"last update":"Sun Feb 18 00:38:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":1.127,"15min":0.77300000000000002},"min":{"1min":0.41399999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.874,"5min":17.486999999999998,"15min":17.486999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":1.1259999999999999,"15min":0.77400000000000002},"min":{"1min":0.38600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81999999999999995,"5min":12.75,"15min":12.75},"last":0.51400000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.439,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":8.9019999999999992,"15min":12.611000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.72399999999999998,"15min":0.78400000000000003},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":4.9550000000000001,"15min":17.417000000000002},"last":0.63100000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.81000000000000005},"min":{"1min":0.45400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81299999999999994,"5min":2.0430000000000001,"15min":12.834},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.79400000000000004},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81299999999999994,"5min":0.90400000000000003,"15min":12.651999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.95299999999999996,"15min":0.80200000000000005},"min":{"1min":0.46300000000000002,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.83999999999999997,"5min":9.0830000000000002,"15min":17.452000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97999999999999998,"15min":0.81399999999999995},"min":{"1min":0.442,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":9.3610000000000007,"15min":12.805},"last":0.58299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738526,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306396,"kb_used_data":5076,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438756,"statfs":{"total":95995035648,"available":95681286144,"internally_reserved":0,"allocated":5197824,"data_stored":2707534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.379,"15min":1.0780000000000001},"min":{"1min":0.19500000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.52400000000000002,"5min":2.524,"15min":133.38999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.48499999999999999,"15min":0.77700000000000002},"min":{"1min":0.26300000000000001,"5min":0.221,"15min":0.20999999999999999},"max":{"1min":0.82199999999999995,"5min":4.577,"15min":90.623000000000005},"last":0.39200000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.97299999999999998,"15min":0.89400000000000002},"min":{"1min":0.27500000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94799999999999995,"5min":21.388999999999999,"15min":90.850999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.91700000000000004,"15min":0.873},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.482,"5min":23.388999999999999,"15min":80.537000000000006},"last":0.376}]},{"osd":3,"last update":"Sun Feb 18 00:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.66800000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56200000000000006,"5min":3.5609999999999999,"15min":19.006},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.47299999999999998,"15min":1.069},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.90300000000000002,"5min":4.4489999999999998,"15min":146.71899999999999},"last":0.41599999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55700000000000005,"15min":0.91800000000000004},"min":{"1min":0.36499999999999999,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.68999999999999995,"5min":3.1850000000000001,"15min":57.914000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63700000000000001,"15min":0.97899999999999998},"min":{"1min":0.44500000000000001,"5min":0.318,"15min":0.30399999999999999},"max":{"1min":0.97999999999999998,"5min":4.2919999999999998,"15min":90.070999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.88100000000000001,"15min":0.95199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32000000000000001},"max":{"1min":1.056,"5min":11.579000000000001,"15min":90.049000000000007},"last":0.504},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":12.952999999999999,"15min":57.991},"last":0.51800000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.626,"15min":0.79700000000000004},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":5.2830000000000004,"15min":19.039000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.97399999999999998},"min":{"1min":0.42799999999999999,"5min":0.34899999999999998,"15min":0.33500000000000002},"max":{"1min":1.083,"5min":4.343,"15min":89.989999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":1.0580000000000001,"15min":1.048},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.073,"5min":13.24,"15min":90.025000000000006},"last":0.629},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.044,"15min":0.997},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":1.603,"5min":13.019,"15min":57.962000000000003},"last":0.55300000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411477,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306464,"kb_used_data":5144,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438688,"statfs":{"total":95995035648,"available":95681216512,"internally_reserved":0,"allocated":5267456,"data_stored":2707869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.57599999999999996,"15min":0.48799999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.46899999999999997,"5min":10.087999999999999,"15min":12.144},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.92000000000000004,"15min":0.60499999999999998},"min":{"1min":0.253,"5min":0.221,"15min":0.20999999999999999},"max":{"1min":0.55800000000000005,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.28699999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.58899999999999997,"15min":0.49399999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.19700000000000001},"max":{"1min":0.502,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.77300000000000002,"15min":0.54300000000000004},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51500000000000001,"5min":18.919,"15min":18.919},"last":0.35299999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.46600000000000003,"15min":0.60099999999999998},"min":{"1min":0.29499999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.48899999999999999,"5min":3.3809999999999998,"15min":22.712},"last":0.378},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47399999999999998,"15min":0.61199999999999999},"min":{"1min":0.29099999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57099999999999995,"5min":4.8529999999999998,"15min":22.359000000000002},"last":0.41399999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.871,"15min":0.68400000000000005},"min":{"1min":0.433,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85499999999999998,"5min":18.812000000000001,"15min":18.812000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.83799999999999997,"15min":0.67400000000000004},"min":{"1min":0.39200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.75800000000000001,"5min":10.016,"15min":10.016},"last":0.626}]},{"osd":5,"last update":"Sun Feb 18 00:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83799999999999997,"5min":2.98,"15min":6.766},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.72299999999999998,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.55200000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.67700000000000005},"min":{"1min":0.41299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":1.196,"15min":6.806},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.72599999999999998},"min":{"1min":0.441,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76100000000000001,"5min":4.827,"15min":18.861000000000001},"last":0.54500000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76200000000000001,"15min":0.71799999999999997},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.79400000000000004,"15min":0.74199999999999999},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.372},"max":{"1min":0.85799999999999998,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.68999999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149839,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306292,"kb_used_data":9068,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93438860,"statfs":{"total":95995035648,"available":95681392640,"internally_reserved":0,"allocated":9285632,"data_stored":6708666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.59799999999999998,"15min":0.47299999999999998},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54900000000000004,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.216},{"interface":"front","average":{"1min":0.379,"5min":0.58299999999999996,"15min":0.52400000000000002},"min":{"1min":0.23699999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57599999999999996,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.26600000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":4.1550000000000002,"15min":6.8479999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.25,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59899999999999998,"5min":23.439,"15min":23.606999999999999},"last":0.28599999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.745,"15min":0.65800000000000003},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56399999999999995,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.47199999999999998,"15min":0.46700000000000003},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":0.66000000000000003,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.254}]},{"osd":4,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.874,"15min":0.68799999999999994},"min":{"1min":0.39700000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70699999999999996,"5min":10.131,"15min":10.131},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.65700000000000003,"15min":0.627},"min":{"1min":0.41199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84799999999999998,"5min":5.3049999999999997,"15min":5.4630000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":3.9670000000000001,"15min":7.4249999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.32400000000000001},"max":{"1min":0.71699999999999997,"5min":4.2149999999999999,"15min":6.7750000000000004},"last":0.443}]},{"osd":6,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61699999999999999,"15min":0.745},"min":{"1min":0.377,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92300000000000004,"5min":3.8940000000000001,"15min":10.189},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.66100000000000003},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":6.4279999999999999},"last":0.437}]},{"osd":7,"last update":"Sun Feb 18 00:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.77600000000000002,"15min":0.66500000000000004},"min":{"1min":0.44,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.90000000000000002,"15min":0.72099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.51100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986314,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310496,"kb_used_data":5080,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93434656,"statfs":{"total":95995035648,"available":95677087744,"internally_reserved":0,"allocated":5201920,"data_stored":2708780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":1.4830000000000001,"15min":0.90300000000000002},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70799999999999996,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.497,"5min":1.46,"15min":0.86599999999999999},"min":{"1min":0.31900000000000001,"5min":0.28199999999999997,"15min":0.27800000000000002},"max":{"1min":0.68600000000000005,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.50600000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.746,"15min":0.64000000000000001},"min":{"1min":0.29199999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64000000000000001,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.498,"5min":1.569,"15min":0.89700000000000002},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.66500000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.51600000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.76700000000000002,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.872,"15min":0.67600000000000005},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68400000000000005,"5min":17.77,"15min":17.77},"last":0.64800000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.67100000000000004,"15min":0.621},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.65000000000000002,"5min":7.218,"15min":7.218},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":1.0569999999999999,"15min":0.72999999999999998},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.70499999999999996,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:38:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.40400000000000003},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.20300000000000001},"max":{"1min":0.60999999999999999,"5min":0.70399999999999996,"15min":0.72299999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.44700000000000001,"15min":0.41299999999999998},"min":{"1min":0.313,"5min":0.26500000000000001,"15min":0.24299999999999999},"max":{"1min":0.60999999999999999,"5min":0.68999999999999995,"15min":0.77900000000000003},"last":0.436}]},{"osd":6,"last update":"Sun Feb 18 00:38:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.441,"15min":0.42399999999999999},"min":{"1min":0.27300000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.66100000000000003,"5min":0.70299999999999996,"15min":0.76100000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.46400000000000002},"min":{"1min":0.28299999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.69299999999999995,"5min":0.751,"15min":0.751},"last":0.57899999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.76900000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.48899999999999999},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.83299999999999996},"last":0.62}]}]},{"osd":5,"up_from":39,"seq":167503724677,"num_pgs":94,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314312,"kb_used_data":8896,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93430840,"statfs":{"total":95995035648,"available":95673180160,"internally_reserved":0,"allocated":9109504,"data_stored":6708631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.624,"15min":0.55300000000000005},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.217},"max":{"1min":2.157,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.1859999999999999,"5min":6.8680000000000003,"15min":12.744},"last":0.58299999999999996}]},{"osd":1,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.79400000000000004,"15min":0.65900000000000003},"min":{"1min":0.32300000000000001,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":13.25,"5min":13.25,"15min":13.25},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.81000000000000005,"15min":0.66700000000000004},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":13.262,"5min":13.262,"15min":13.262},"last":0.60599999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.64500000000000002,"15min":0.60799999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.251},"max":{"1min":0.79400000000000004,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.503},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.745,"15min":0.65200000000000002},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.308},"max":{"1min":5.0659999999999998,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.65200000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77200000000000002,"15min":0.63600000000000001},"min":{"1min":0.32000000000000001,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":7.641,"5min":7.641,"15min":7.641},"last":0.378},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.753,"15min":0.61599999999999999},"min":{"1min":0.39100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.4580000000000002,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.48399999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47899999999999998,"15min":0.45900000000000002},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.92600000000000005,"5min":4.6859999999999999,"15min":11.818},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47799999999999998,"15min":0.443},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.84899999999999998,"5min":1.5680000000000001,"15min":4.274},"last":0.57299999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.53600000000000003},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.5939999999999999,"5min":12.362,"15min":18.605},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.88600000000000001,"5min":1.46,"15min":4.2089999999999996},"last":0.47099999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51800000000000002,"15min":0.48299999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.5269999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.621},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.51300000000000001},"min":{"1min":0.35599999999999998,"5min":0.318,"15min":0.309},"max":{"1min":0.76000000000000001,"5min":4.649,"15min":4.649},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4345856,"data_stored":4000802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4173824,"data_stored":4000393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4272128,"data_stored":4000631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:39:17.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:15.556829+0000 osd.3 (osd.3) 253 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:15.558356+0000 osd.3 (osd.3) 254 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:15.895486+0000 mgr.y (mgr.24454) 422 : audit [DBG] from='client.41201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:15.915618+0000 mgr.y (mgr.24454) 423 : cluster [DBG] pgmap v677: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:16.090843+0000 mon.a (mon.0) 2597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:16.090951+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e452: 8 total, 8 up, 7 in 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:16.097613+0000 mon.c (mon.2) 433 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:16.134187+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:16.134506+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:16.134916+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-67"}]: dispatch 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:16.135143+0000 mon.a (mon.0) 2600 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-67"}]: dispatch 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:16.383257+0000 osd.4 (osd.4) 269 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:16.384114+0000 osd.4 (osd.4) 270 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:15.556829+0000 osd.3 (osd.3) 253 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:15.558356+0000 osd.3 (osd.3) 254 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:15.895486+0000 mgr.y (mgr.24454) 422 : audit [DBG] from='client.41201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:15.915618+0000 mgr.y (mgr.24454) 423 : cluster [DBG] pgmap v677: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:16.090843+0000 mon.a (mon.0) 2597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:17.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:16.090951+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e452: 8 total, 8 up, 7 in 2024-02-18T00:39:17.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:16.097613+0000 mon.c (mon.2) 433 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:17.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:16.134187+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:17.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:16.134506+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:17.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:16.134916+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-67"}]: dispatch 2024-02-18T00:39:17.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:16.135143+0000 mon.a (mon.0) 2600 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-67"}]: dispatch 2024-02-18T00:39:17.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:16.383257+0000 osd.4 (osd.4) 269 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:39:17.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:16.384114+0000 osd.4 (osd.4) 270 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:15.556829+0000 osd.3 (osd.3) 253 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:15.558356+0000 osd.3 (osd.3) 254 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:15.895486+0000 mgr.y (mgr.24454) 422 : audit [DBG] from='client.41201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:15.915618+0000 mgr.y (mgr.24454) 423 : cluster [DBG] pgmap v677: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:16.090843+0000 mon.a (mon.0) 2597 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:16.090951+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e452: 8 total, 8 up, 7 in 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:16.097613+0000 mon.c (mon.2) 433 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:16.134187+0000 mon.c (mon.2) 434 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:16.134506+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:17.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:16.134916+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-67"}]: dispatch 2024-02-18T00:39:17.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:16.135143+0000 mon.a (mon.0) 2600 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-67"}]: dispatch 2024-02-18T00:39:17.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:16.383257+0000 osd.4 (osd.4) 269 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:39:17.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:16.384114+0000 osd.4 (osd.4) 270 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:39:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:17.783 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:17.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:17.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:17.915 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 -- 172.21.15.59:0/2267180608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 msgr2=0x7f37e8103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 --2- 172.21.15.59:0/2267180608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8103a20 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 rx=0x7f37e0009be0 tx=0x7f37e0004d10).stop 2024-02-18T00:39:18.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 -- 172.21.15.59:0/2267180608 shutdown_connections 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 --2- 172.21.15.59:0/2267180608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e80fcbb0 0x7f37e80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 --2- 172.21.15.59:0/2267180608 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8103a20 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 --2- 172.21.15.59:0/2267180608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e80fd510 0x7f37e80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 -- 172.21.15.59:0/2267180608 >> 172.21.15.59:0/2267180608 conn(0x7f37e80f2820 msgr2=0x7f37e80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 -- 172.21.15.59:0/2267180608 shutdown_connections 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 -- 172.21.15.59:0/2267180608 wait complete. 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 librados: starting msgr at 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.759+0000 7f37eeac6700 1 librados: starting objecter 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 Processor -- start 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 -- start start 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 librados: setting wanted keys 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 librados: calling monclient init 2024-02-18T00:39:18.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e80fcbb0 0x7f37e8107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e80fd510 0x7f37e81078f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e8116590 con 0x7f37e80fd510 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e81166d0 con 0x7f37e80fcbb0 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37eeac6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e8116840 con 0x7f37e80f7400 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:35796/0 (socket says 172.21.15.59:35796) 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 -- 172.21.15.59:0/3157902168 learned_addr learned my addr 172.21.15.59:0/3157902168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ed063700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e80fd510 0x7f37e81078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e80fcbb0 0x7f37e8107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 -- 172.21.15.59:0/3157902168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e80fcbb0 msgr2=0x7f37e8107400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e80fcbb0 0x7f37e8107400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 -- 172.21.15.59:0/3157902168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e80fd510 msgr2=0x7f37e81078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37e80fd510 0x7f37e81078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 -- 172.21.15.59:0/3157902168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e00097c0 con 0x7f37e80f7400 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37e80fcbb0 0x7f37e8107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8106f10 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 rx=0x7f37dc0041d0 tx=0x7f37dc009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:39:18.292 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37e5f76700 10 client.?.objecter ms_handle_connect 0x7f37e80f7400 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.760+0000 7f37e5f76700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc007b40 con 0x7f37e80f7400 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37dc00f040 con 0x7f37e80f7400 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc014810 con 0x7f37e80f7400 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e8116b20 con 0x7f37e80f7400 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37eeac6700 10 client.41207.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:18.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e8116f90 con 0x7f37e80f7400 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37eeac6700 1 librados: init done 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.761+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f37e805fa60 con 0x7f37e80f7400 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.762+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f37dc009110 con 0x7f37e80f7400 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.762+0000 7f37e5f76700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37d8062c90 0x7f37d8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.762+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(453..453 src has 180..453) v4 ==== 7650+0+0 (secure 0 0 0) 0x7f37dc07cfb0 con 0x7f37e80f7400 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.762+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 osd_map(453..453 src has 180..453) v4 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.762+0000 7f37e5f76700 3 client.41207.objecter handle_osd_map got epochs [453,453] > 0 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.762+0000 7f37e5f76700 3 client.41207.objecter handle_osd_map decoding full epoch 453 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.762+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37d8062c90 0x7f37d8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37e5f76700 20 client.41207.objecter dump_active .. 0 homeless 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v453) v1 ==== 54+0+16 (secure 0 0 0) 0x7f37dc084070 con 0x7f37e80f7400 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37d8062c90 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37d8062c90 0x7f37d8065150 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 rx=0x7f37d4007fd0 tx=0x7f37d400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 10 client.41207.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 1 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6801/3700238977 conn(0x7f37e80fdea0 0x7f37e80fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 1 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 0 1 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e805ee80 osd=0 4 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 10 client.41207.objecter _send_command 1 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e81061c0 con 0x7f37e80fdea0 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37ed063700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6801/3700238977 conn(0x7f37e80fdea0 0x7f37e80fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e80fdea0 2024-02-18T00:39:18.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.763+0000 7f37ed063700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6801/3700238977 conn(0x7f37e80fdea0 0x7f37e80fe280 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.764+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e81061c0 con 0x7f37e80fdea0 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.764+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.764+0000 7f37e5f76700 10 client.41207.objecter _finish_command 1 = system:0 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.764+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e805ee80 osd=0 4 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.764+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 0 1 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 10 client.41207.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 2 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 2 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e805ee80 osd=0 3 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 0 2 2024-02-18T00:39:18.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e805ee80 osd=0 4 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 10 client.41207.objecter _send_command 2 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.781+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e80fe9b0 con 0x7f37e80fdea0 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e0019040 con 0x7f37e80fdea0 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37e5f76700 10 client.41207.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e805ee80 osd=0 4 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 0 2 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 10 client.41207.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 3 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6805/26279110 conn(0x7f37e80598f0 0x7f37e8059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 3 2024-02-18T00:39:18.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 1 3 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8028540 osd=1 4 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 10 client.41207.objecter _send_command 3 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e80286b0 con 0x7f37e80598f0 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6805/26279110 conn(0x7f37e80598f0 0x7f37e8059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e80598f0 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.783+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6805/26279110 conn(0x7f37e80598f0 0x7f37e8059cf0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.784+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e80286b0 con 0x7f37e80598f0 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.784+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.784+0000 7f37e5f76700 10 client.41207.objecter _finish_command 3 = system:0 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.784+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8028540 osd=1 4 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.784+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 1 3 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 10 client.41207.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 4 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 4 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8028540 osd=1 3 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 1 4 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8028540 osd=1 4 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 10 client.41207.objecter _send_command 4 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.799+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e8059360 con 0x7f37e80598f0 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e8059360 con 0x7f37e80598f0 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e5f76700 10 client.41207.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8028540 osd=1 4 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 1 4 2024-02-18T00:39:18.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 10 client.41207.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 5 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6809/1153193637 conn(0x7f37e80280a0 0x7f37e811e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 5 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 2 5 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8059770 osd=2 4 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 10 client.41207.objecter _send_command 5 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e8121040 con 0x7f37e80280a0 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6809/1153193637 conn(0x7f37e80280a0 0x7f37e811e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e80280a0 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.801+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6809/1153193637 conn(0x7f37e80280a0 0x7f37e811e3f0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.802+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e8121040 con 0x7f37e80280a0 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.802+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.802+0000 7f37e5f76700 10 client.41207.objecter _finish_command 5 = system:0 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.802+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8059770 osd=2 4 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.802+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 2 5 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 10 client.41207.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 6 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.301 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 6 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8059770 osd=2 3 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 2 6 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8059770 osd=2 4 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 10 client.41207.objecter _send_command 6 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.817+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e8027660 con 0x7f37e80280a0 2024-02-18T00:39:18.302 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e8027660 con 0x7f37e80280a0 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37e5f76700 10 client.41207.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8059770 osd=2 4 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 2 6 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 10 client.41207.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 7 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6813/1805700964 conn(0x7f37e811fa60 0x7f37e811fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 7 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 3 7 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e811ba30 osd=3 4 2024-02-18T00:39:18.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 10 client.41207.objecter _send_command 7 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e811c1a0 con 0x7f37e811fa60 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37ed063700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6813/1805700964 conn(0x7f37e811fa60 0x7f37e811fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.819+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e811fa60 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.820+0000 7f37ed063700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6813/1805700964 conn(0x7f37e811fa60 0x7f37e811fee0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.820+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e811c1a0 con 0x7f37e811fa60 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.820+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.820+0000 7f37e5f76700 10 client.41207.objecter _finish_command 7 = system:0 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.820+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e811ba30 osd=3 4 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.820+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 3 7 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 10 client.41207.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 8 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 8 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e811ba30 osd=3 3 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 3 8 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e811ba30 osd=3 4 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 10 client.41207.objecter _send_command 8 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.835+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e8120ba0 con 0x7f37e811fa60 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.836+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e8120ba0 con 0x7f37e811fa60 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.836+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.836+0000 7f37e5f76700 10 client.41207.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.836+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e811ba30 osd=3 4 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.836+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 3 8 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 10 client.41207.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 9 2024-02-18T00:39:18.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6800/2044387056 conn(0x7f37e8123270 0x7f37e8125660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 9 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 4 9 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120ba0 osd=4 4 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 10 client.41207.objecter _send_command 9 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e8120e00 con 0x7f37e8123270 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6800/2044387056 conn(0x7f37e8123270 0x7f37e8125660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e8123270 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.837+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6800/2044387056 conn(0x7f37e8123270 0x7f37e8125660 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.838+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e8120e00 con 0x7f37e8123270 2024-02-18T00:39:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.838+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.838+0000 7f37e5f76700 10 client.41207.objecter _finish_command 9 = system:0 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.838+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8120ba0 osd=4 4 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.838+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 4 9 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 10 client.41207.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 10 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 10 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8120ba0 osd=4 3 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 4 10 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120ba0 osd=4 4 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 10 client.41207.objecter _send_command 10 2024-02-18T00:39:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.853+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e8120e00 con 0x7f37e8123270 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e8120e00 con 0x7f37e8123270 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37e5f76700 10 client.41207.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8120ba0 osd=4 4 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 4 10 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 10 client.41207.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 11 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6804/2170432263 conn(0x7f37e8126060 0x7f37e81284d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 11 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 5 11 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120e00 osd=5 4 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 10 client.41207.objecter _send_command 11 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.855+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e812b2a0 con 0x7f37e8126060 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.856+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6804/2170432263 conn(0x7f37e8126060 0x7f37e81284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.856+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e8126060 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.856+0000 7f37e7fff700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6804/2170432263 conn(0x7f37e8126060 0x7f37e81284d0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.857+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e812b2a0 con 0x7f37e8126060 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.857+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.857+0000 7f37e5f76700 10 client.41207.objecter _finish_command 11 = system:0 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.857+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8120e00 osd=5 4 2024-02-18T00:39:18.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.857+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 5 11 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 10 client.41207.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 12 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 12 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8120e00 osd=5 3 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 5 12 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120e00 osd=5 4 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 10 client.41207.objecter _send_command 12 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.873+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e812b2a0 con 0x7f37e8126060 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e812b2a0 con 0x7f37e8126060 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37e5f76700 10 client.41207.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8120e00 osd=5 4 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 5 12 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 10 client.41207.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 13 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6808/3356009390 conn(0x7f37e812a5e0 0x7f37e812aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 13 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 6 13 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e812b2a0 osd=6 4 2024-02-18T00:39:18.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 10 client.41207.objecter _send_command 13 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e812e6d0 con 0x7f37e812a5e0 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.875+0000 7f37ed063700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6808/3356009390 conn(0x7f37e812a5e0 0x7f37e812aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.876+0000 7f37ed063700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6808/3356009390 conn(0x7f37e812a5e0 0x7f37e812aa40 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.876+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e812a5e0 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.877+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e812e6d0 con 0x7f37e812a5e0 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.877+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.877+0000 7f37e5f76700 10 client.41207.objecter _finish_command 13 = system:0 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.877+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e812b2a0 osd=6 4 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.877+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 6 13 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 10 client.41207.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 14 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 14 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e812b2a0 osd=6 3 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 6 14 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e812b2a0 osd=6 4 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 10 client.41207.objecter _send_command 14 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.892+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e8128a80 con 0x7f37e812a5e0 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e8128a80 con 0x7f37e812a5e0 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37e5f76700 10 client.41207.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e812b2a0 osd=6 4 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 6 14 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 10 client.41207.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:18.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 15 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6812/53025171 conn(0x7f37e8129100 0x7f37e8129540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 15 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 7 15 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8128a80 osd=7 4 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 10 client.41207.objecter _send_command 15 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37e8129b90 con 0x7f37e8129100 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.894+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6812/53025171 conn(0x7f37e8129100 0x7f37e8129540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.895+0000 7f37e5f76700 10 client.41207.objecter ms_handle_connect 0x7f37e8129100 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.895+0000 7f37ec862700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6812/53025171 conn(0x7f37e8129100 0x7f37e8129540 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:18.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.895+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f37e8129b90 con 0x7f37e8129100 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.895+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.895+0000 7f37e5f76700 10 client.41207.objecter _finish_command 15 = system:0 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.895+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8128a80 osd=7 4 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.895+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 7 15 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 10 client.41207.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign -1 16 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 20 client.41207.objecter _get_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_remove -1 16 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 20 client.41207.objecter get_session s=0x7f37e8128a80 osd=7 3 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 15 client.41207.objecter _session_command_op_assign 7 16 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8128a80 osd=7 4 2024-02-18T00:39:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 10 client.41207.objecter _send_command 16 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.911+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f37e8129b90 con 0x7f37e8129100 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37e5f76700 1 -- 172.21.15.59:0/3157902168 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f37e8129b90 con 0x7f37e8129100 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37e5f76700 10 client.41207.objecter ms_dispatch 0x7f37e8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37e5f76700 10 client.41207.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37e5f76700 20 client.41207.objecter put_session s=0x7f37e8128a80 osd=7 4 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37e5f76700 15 client.41207.objecter _session_command_op_remove 7 16 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 10 librados: watch_flush enter 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 10 librados: watch_flush exit 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.0 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6801/3700238977 conn(0x7f37e80fdea0 msgr2=0x7f37e80fe280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6801/3700238977 conn(0x7f37e80fdea0 0x7f37e80fe280 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e805ee80 osd=0 1 2024-02-18T00:39:18.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.1 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6805/26279110 conn(0x7f37e80598f0 msgr2=0x7f37e8059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6805/26279110 conn(0x7f37e80598f0 0x7f37e8059cf0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8028540 osd=1 1 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.2 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6809/1153193637 conn(0x7f37e80280a0 msgr2=0x7f37e811e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6809/1153193637 conn(0x7f37e80280a0 0x7f37e811e3f0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8059770 osd=2 1 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.3 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6813/1805700964 conn(0x7f37e811fa60 msgr2=0x7f37e811fee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37ed063700 1 -- 172.21.15.59:0/3157902168 reap_dead start 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.913+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6813/1805700964 conn(0x7f37e811fa60 0x7f37e811fee0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e811ba30 osd=3 1 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.4 2024-02-18T00:39:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6800/2044387056 conn(0x7f37e8123270 msgr2=0x7f37e8125660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6800/2044387056 conn(0x7f37e8123270 0x7f37e8125660 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120ba0 osd=4 1 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.5 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6804/2170432263 conn(0x7f37e8126060 msgr2=0x7f37e81284d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6804/2170432263 conn(0x7f37e8126060 0x7f37e81284d0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8120e00 osd=5 1 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.6 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6808/3356009390 conn(0x7f37e812a5e0 msgr2=0x7f37e812aa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6808/3356009390 conn(0x7f37e812a5e0 0x7f37e812aa40 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e812b2a0 osd=6 1 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 10 client.41207.objecter close_session for osd.7 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6812/53025171 conn(0x7f37e8129100 msgr2=0x7f37e8129540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.137:6812/53025171 conn(0x7f37e8129100 0x7f37e8129540 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 20 client.41207.objecter put_session s=0x7f37e8128a80 osd=7 1 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 20 client.41207.objecter shutdown clearing up homeless session... 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 10 client.41207.objecter successfully canceled tick 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37ed063700 1 -- 172.21.15.59:0/3157902168 reap_dead start 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37d8062c90 msgr2=0x7f37d8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37d8062c90 0x7f37d8065150 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 rx=0x7f37d4007fd0 tx=0x7f37d400b040).stop 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 msgr2=0x7f37e8106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8106f10 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 rx=0x7f37dc0041d0 tx=0x7f37dc009f60).stop 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 shutdown_connections 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f37e80f7400 0x7f37e8106f10 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 --2- 172.21.15.59:0/3157902168 >> v2:172.21.15.59:6800/3721891797 conn(0x7f37d8062c90 0x7f37d8065150 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 >> 172.21.15.59:0/3157902168 conn(0x7f37e80f2820 msgr2=0x7f37e80f40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 shutdown_connections 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 -- 172.21.15.59:0/3157902168 wait complete. 2024-02-18T00:39:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:17.914+0000 7f37eeac6700 1 librados: shutdown 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:39:18.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:17.093517+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e453: 8 total, 8 up, 7 in 2024-02-18T00:39:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:17.763710+0000 mon.c (mon.2) 436 : audit [DBG] from='client.? 172.21.15.59:0/3157902168' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:17.093517+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e453: 8 total, 8 up, 7 in 2024-02-18T00:39:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:17.763710+0000 mon.c (mon.2) 436 : audit [DBG] from='client.? 172.21.15.59:0/3157902168' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:17.093517+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e453: 8 total, 8 up, 7 in 2024-02-18T00:39:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:17.763710+0000 mon.c (mon.2) 436 : audit [DBG] from='client.? 172.21.15.59:0/3157902168' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:17.832502+0000 osd.6 (osd.6) 217 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:39:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:17.834135+0000 osd.6 (osd.6) 218 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:39:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:17.915974+0000 mgr.y (mgr.24454) 424 : cluster [DBG] pgmap v680: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:39:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:18.107355+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e454: 8 total, 8 up, 7 in 2024-02-18T00:39:19.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:18.108615+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:18.109413+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:18.517679+0000 osd.0 (osd.0) 218 : cluster [DBG] 62.13 scrub starts 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:18.521147+0000 osd.0 (osd.0) 219 : cluster [DBG] 62.13 scrub ok 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:18.692871+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:18.693076+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e455: 8 total, 8 up, 7 in 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:18.716110+0000 mon.c (mon.2) 438 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:18.719044+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:18.719377+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:17.832502+0000 osd.6 (osd.6) 217 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:17.834135+0000 osd.6 (osd.6) 218 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:39:19.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:17.915974+0000 mgr.y (mgr.24454) 424 : cluster [DBG] pgmap v680: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:18.107355+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e454: 8 total, 8 up, 7 in 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:18.108615+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:18.109413+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:18.517679+0000 osd.0 (osd.0) 218 : cluster [DBG] 62.13 scrub starts 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:18.521147+0000 osd.0 (osd.0) 219 : cluster [DBG] 62.13 scrub ok 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:18.692871+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:18.693076+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e455: 8 total, 8 up, 7 in 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:18.716110+0000 mon.c (mon.2) 438 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:19.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:18.719044+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:18.719377+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:17.832502+0000 osd.6 (osd.6) 217 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:17.834135+0000 osd.6 (osd.6) 218 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:17.915974+0000 mgr.y (mgr.24454) 424 : cluster [DBG] pgmap v680: 206 pgs: 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:18.107355+0000 mon.a (mon.0) 2602 : cluster [DBG] osdmap e454: 8 total, 8 up, 7 in 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:18.108615+0000 mon.c (mon.2) 437 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:18.109413+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:18.517679+0000 osd.0 (osd.0) 218 : cluster [DBG] 62.13 scrub starts 2024-02-18T00:39:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:18.521147+0000 osd.0 (osd.0) 219 : cluster [DBG] 62.13 scrub ok 2024-02-18T00:39:19.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:18.692871+0000 mon.a (mon.0) 2604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:19.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:18.693076+0000 mon.a (mon.0) 2605 : cluster [DBG] osdmap e455: 8 total, 8 up, 7 in 2024-02-18T00:39:19.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:18.716110+0000 mon.c (mon.2) 438 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:19.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:18.719044+0000 mon.c (mon.2) 439 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:39:19.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:18.719377+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:39:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:18.856632+0000 osd.6 (osd.6) 219 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:39:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:18.857986+0000 osd.6 (osd.6) 220 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:39:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:19.315998+0000 osd.4 (osd.4) 271 : cluster [DBG] 2.b deep-scrub starts 2024-02-18T00:39:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:19.319006+0000 osd.4 (osd.4) 272 : cluster [DBG] 2.b deep-scrub ok 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:19.709912+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:19.710007+0000 mon.a (mon.0) 2608 : cluster [DBG] osdmap e456: 8 total, 8 up, 7 in 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:18.856632+0000 osd.6 (osd.6) 219 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:18.857986+0000 osd.6 (osd.6) 220 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:19.315998+0000 osd.4 (osd.4) 271 : cluster [DBG] 2.b deep-scrub starts 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:19.319006+0000 osd.4 (osd.4) 272 : cluster [DBG] 2.b deep-scrub ok 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:19.709912+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:39:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:19.710007+0000 mon.a (mon.0) 2608 : cluster [DBG] osdmap e456: 8 total, 8 up, 7 in 2024-02-18T00:39:20.329 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:20.350 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:39:20.350 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:39:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:18.856632+0000 osd.6 (osd.6) 219 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:39:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:18.857986+0000 osd.6 (osd.6) 220 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:39:20.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:19.315998+0000 osd.4 (osd.4) 271 : cluster [DBG] 2.b deep-scrub starts 2024-02-18T00:39:20.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:19.319006+0000 osd.4 (osd.4) 272 : cluster [DBG] 2.b deep-scrub ok 2024-02-18T00:39:20.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:19.709912+0000 mon.a (mon.0) 2607 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:39:20.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:19.710007+0000 mon.a (mon.0) 2608 : cluster [DBG] osdmap e456: 8 total, 8 up, 7 in 2024-02-18T00:39:21.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:19.916400+0000 mgr.y (mgr.24454) 425 : cluster [DBG] pgmap v684: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:20.712903+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e457: 8 total, 8 up, 7 in 2024-02-18T00:39:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:19.916400+0000 mgr.y (mgr.24454) 425 : cluster [DBG] pgmap v684: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:20.712903+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e457: 8 total, 8 up, 7 in 2024-02-18T00:39:21.471 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:21.474 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:21.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:21.500 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-18T00:39:21.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:39:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:39:21.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:19.916400+0000 mgr.y (mgr.24454) 425 : cluster [DBG] pgmap v684: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:21.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:20.712903+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e457: 8 total, 8 up, 7 in 2024-02-18T00:39:21.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:39:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:39:21.575 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:21.614 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:21.689 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:21.692 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:21.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:21.703 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:39:21.793 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:21.822 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:22.025 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:22.025 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:22.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:22.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:22.129 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:22.161 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:22.183 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:22.185 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:22.186 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:22.287 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:22.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:20.446799+0000 osd.1 (osd.1) 229 : cluster [DBG] 5.c scrub starts 2024-02-18T00:39:22.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:20.450042+0000 osd.1 (osd.1) 230 : cluster [DBG] 5.c scrub ok 2024-02-18T00:39:22.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:20.628301+0000 osd.3 (osd.3) 255 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:39:22.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:20.629415+0000 osd.3 (osd.3) 256 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:39:22.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:21.294069+0000 mgr.y (mgr.24454) 426 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:22.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:21.297215+0000 osd.4 (osd.4) 273 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:39:22.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:21.298544+0000 osd.4 (osd.4) 274 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:21.434608+0000 osd.2 (osd.2) 193 : cluster [DBG] 62.7 deep-scrub starts 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:21.441375+0000 osd.2 (osd.2) 194 : cluster [DBG] 62.7 deep-scrub ok 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:22] "GET /metrics HTTP/1.1" 200 189761 "" "Prometheus/2.33.4" 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:20.446799+0000 osd.1 (osd.1) 229 : cluster [DBG] 5.c scrub starts 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:20.450042+0000 osd.1 (osd.1) 230 : cluster [DBG] 5.c scrub ok 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:20.628301+0000 osd.3 (osd.3) 255 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:20.629415+0000 osd.3 (osd.3) 256 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:21.294069+0000 mgr.y (mgr.24454) 426 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:22.313 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:21.297215+0000 osd.4 (osd.4) 273 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:39:22.314 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:21.298544+0000 osd.4 (osd.4) 274 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:39:22.314 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:21.434608+0000 osd.2 (osd.2) 193 : cluster [DBG] 62.7 deep-scrub starts 2024-02-18T00:39:22.314 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:21.441375+0000 osd.2 (osd.2) 194 : cluster [DBG] 62.7 deep-scrub ok 2024-02-18T00:39:22.314 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:22.337 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:22.339 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:22.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:22.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:22.469 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:20.446799+0000 osd.1 (osd.1) 229 : cluster [DBG] 5.c scrub starts 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:20.450042+0000 osd.1 (osd.1) 230 : cluster [DBG] 5.c scrub ok 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:20.628301+0000 osd.3 (osd.3) 255 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:20.629415+0000 osd.3 (osd.3) 256 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:21.294069+0000 mgr.y (mgr.24454) 426 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:21.297215+0000 osd.4 (osd.4) 273 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:21.298544+0000 osd.4 (osd.4) 274 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:21.434608+0000 osd.2 (osd.2) 193 : cluster [DBG] 62.7 deep-scrub starts 2024-02-18T00:39:22.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:21.441375+0000 osd.2 (osd.2) 194 : cluster [DBG] 62.7 deep-scrub ok 2024-02-18T00:39:23.017 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:23.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:23.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:23.121 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:23.149 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:23.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:21.414955+0000 osd.1 (osd.1) 231 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:21.416245+0000 osd.1 (osd.1) 232 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:21.917229+0000 mgr.y (mgr.24454) 427 : cluster [DBG] pgmap v686: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:22.313516+0000 osd.4 (osd.4) 275 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:22.316424+0000 osd.4 (osd.4) 276 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:21.414955+0000 osd.1 (osd.1) 231 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:21.416245+0000 osd.1 (osd.1) 232 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:21.917229+0000 mgr.y (mgr.24454) 427 : cluster [DBG] pgmap v686: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:22.313516+0000 osd.4 (osd.4) 275 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:39:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:22.316424+0000 osd.4 (osd.4) 276 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:39:23.307 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:23.310 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:23.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:23.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:23.440 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:21.414955+0000 osd.1 (osd.1) 231 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:39:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:21.416245+0000 osd.1 (osd.1) 232 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:39:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:21.917229+0000 mgr.y (mgr.24454) 427 : cluster [DBG] pgmap v686: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-02-18T00:39:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:22.313516+0000 osd.4 (osd.4) 275 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:39:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:22.316424+0000 osd.4 (osd.4) 276 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:39:23.694 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:23.701 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:23.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:23.803 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:23.832 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:23.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 -- 172.21.15.59:0/2693476050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fdbb0 msgr2=0x7f88d80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:23.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 --2- 172.21.15.59:0/2693476050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fdbb0 0x7f88d80fdf90 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 rx=0x7f88d4004870 tx=0x7f88d4009cb0).stop 2024-02-18T00:39:23.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 -- 172.21.15.59:0/2693476050 shutdown_connections 2024-02-18T00:39:23.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 --2- 172.21.15.59:0/2693476050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d80f8420 0x7f88d8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 --2- 172.21.15.59:0/2693476050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fdbb0 0x7f88d80fdf90 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 --2- 172.21.15.59:0/2693476050 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80fe510 0x7f88d80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 -- 172.21.15.59:0/2693476050 >> 172.21.15.59:0/2693476050 conn(0x7f88d80f3800 msgr2=0x7f88d80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 -- 172.21.15.59:0/2693476050 shutdown_connections 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.558+0000 7f88df56e700 1 -- 172.21.15.59:0/2693476050 wait complete. 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 librados: starting msgr at 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 librados: starting objecter 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 Processor -- start 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 -- start start 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 librados: setting wanted keys 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 librados: calling monclient init 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80f8420 0x7f88d810e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d80fdbb0 0x7f88d810e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fe510 0x7f88d8108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d8108770 con 0x7f88d80fe510 2024-02-18T00:39:23.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d81088e0 con 0x7f88d80fdbb0 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88df56e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d8108aa0 con 0x7f88d80f8420 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88dd30a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80f8420 0x7f88d810e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88ddb0b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fe510 0x7f88d8108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88dd30a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80f8420 0x7f88d810e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:36376/0 (socket says 172.21.15.59:36376) 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88ddb0b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fe510 0x7f88d8108280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54152/0 (socket says 172.21.15.59:54152) 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.559+0000 7f88dd30a700 1 -- 172.21.15.59:0/61852265 learned_addr learned my addr 172.21.15.59:0/61852265 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88dcb09700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d80fdbb0 0x7f88d810e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ddb0b700 1 -- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80f8420 msgr2=0x7f88d810e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ddb0b700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80f8420 0x7f88d810e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ddb0b700 1 -- 172.21.15.59:0/61852265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d80fdbb0 msgr2=0x7f88d810e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ddb0b700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d80fdbb0 0x7f88d810e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ddb0b700 1 -- 172.21.15.59:0/61852265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8005040 con 0x7f88d80fe510 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88dd30a700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80f8420 0x7f88d810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88dcb09700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d80fdbb0 0x7f88d810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ddb0b700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fe510 0x7f88d8108280 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 rx=0x7f88d000c7a0 tx=0x7f88d0007d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ce7fc700 10 client.?.objecter ms_handle_connect 0x7f88d80fe510 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ce7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ce7fc700 1 -- 172.21.15.59:0/61852265 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0010510 con 0x7f88d80fe510 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d4005040 con 0x7f88d80fe510 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ce7fc700 1 -- 172.21.15.59:0/61852265 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88d0010b40 con 0x7f88d80fe510 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88ce7fc700 1 -- 172.21.15.59:0/61852265 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d0012b10 con 0x7f88d80fe510 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88df56e700 10 client.31410.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d8109090 con 0x7f88d80fe510 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.560+0000 7f88df56e700 1 librados: init done 2024-02-18T00:39:23.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.561+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d805ee80 con 0x7f88d80fe510 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 1 -- 172.21.15.59:0/61852265 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f88d0010670 con 0x7f88d80fe510 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 1 --2- 172.21.15.59:0/61852265 >> v2:172.21.15.59:6800/3721891797 conn(0x7f88c4062c90 0x7f88c4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 1 -- 172.21.15.59:0/61852265 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(457..457 src has 180..457) v4 ==== 8035+0+0 (secure 0 0 0) 0x7f88d007d200 con 0x7f88d80fe510 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 10 client.31410.objecter ms_dispatch 0x7f88d8001550 osd_map(457..457 src has 180..457) v4 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 3 client.31410.objecter handle_osd_map got epochs [457,457] > 0 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 3 client.31410.objecter handle_osd_map decoding full epoch 457 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88dd30a700 1 --2- 172.21.15.59:0/61852265 >> v2:172.21.15.59:6800/3721891797 conn(0x7f88c4062c90 0x7f88c4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 20 client.31410.objecter dump_active .. 0 homeless 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88ce7fc700 10 client.31410.objecter ms_handle_connect 0x7f88c4062c90 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.562+0000 7f88dd30a700 1 --2- 172.21.15.59:0/61852265 >> v2:172.21.15.59:6800/3721891797 conn(0x7f88c4062c90 0x7f88c4065150 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 rx=0x7f88d4009a00 tx=0x7f88d400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:23.838 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:39:23.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.567+0000 7f88ce7fc700 1 -- 172.21.15.59:0/61852265 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f88d00507d0 con 0x7f88d80fe510 2024-02-18T00:39:23.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.740+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88d8109860 con 0x7f88c4062c90 2024-02-18T00:39:23.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.758+0000 7f88ce7fc700 1 -- 172.21.15.59:0/61852265 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+598609 (secure 0 0 0) 0x7f88d8109860 con 0x7f88c4062c90 2024-02-18T00:39:23.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.835+0000 7f88df56e700 10 librados: watch_flush enter 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.835+0000 7f88df56e700 10 librados: watch_flush exit 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.835+0000 7f88df56e700 20 client.31410.objecter shutdown clearing up homeless session... 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.835+0000 7f88df56e700 10 client.31410.objecter successfully canceled tick 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 >> v2:172.21.15.59:6800/3721891797 conn(0x7f88c4062c90 msgr2=0x7f88c4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 --2- 172.21.15.59:0/61852265 >> v2:172.21.15.59:6800/3721891797 conn(0x7f88c4062c90 0x7f88c4065150 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 rx=0x7f88d4009a00 tx=0x7f88d400b040).stop 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fe510 msgr2=0x7f88d8108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fe510 0x7f88d8108280 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 rx=0x7f88d000c7a0 tx=0x7f88d0007d10).stop 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 shutdown_connections 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d80fe510 0x7f88d8108280 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d80f8420 0x7f88d810e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 --2- 172.21.15.59:0/61852265 >> v2:172.21.15.59:6800/3721891797 conn(0x7f88c4062c90 0x7f88c4065150 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 --2- 172.21.15.59:0/61852265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88d80fdbb0 0x7f88d810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 >> 172.21.15.59:0/61852265 conn(0x7f88d80f3800 msgr2=0x7f88d80fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 shutdown_connections 2024-02-18T00:39:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 -- 172.21.15.59:0/61852265 wait complete. 2024-02-18T00:39:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:23.836+0000 7f88df56e700 1 librados: shutdown 2024-02-18T00:39:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:39:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:23.854 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:23.855 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:23.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:23.986 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:24.234 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:24.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:22.367004+0000 osd.1 (osd.1) 233 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:39:24.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:22.368180+0000 osd.1 (osd.1) 234 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:39:24.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:23.324897+0000 osd.4 (osd.4) 277 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:39:24.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:23.328348+0000 osd.4 (osd.4) 278 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:39:24.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:22.367004+0000 osd.1 (osd.1) 233 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:39:24.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:22.368180+0000 osd.1 (osd.1) 234 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:39:24.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:23.324897+0000 osd.4 (osd.4) 277 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:39:24.238 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:23.328348+0000 osd.4 (osd.4) 278 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:39:24.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:24.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:24.340 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:24.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:22.367004+0000 osd.1 (osd.1) 233 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:39:24.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:22.368180+0000 osd.1 (osd.1) 234 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:39:24.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:23.324897+0000 osd.4 (osd.4) 277 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:39:24.370 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:23.328348+0000 osd.4 (osd.4) 278 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:39:24.372 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:24.394 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:24.396 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:24.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:24.498 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:24.529 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:24.566 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:24.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:24.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:24.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:24.703 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:24.726 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:24.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:24.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:24.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:24.857 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:23.741791+0000 mgr.y (mgr.24454) 428 : audit [DBG] from='client.31410 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:39:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:23.917610+0000 mgr.y (mgr.24454) 429 : cluster [DBG] pgmap v687: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 5 op/s 2024-02-18T00:39:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:24.902789+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:23.741791+0000 mgr.y (mgr.24454) 428 : audit [DBG] from='client.31410 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:39:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:23.917610+0000 mgr.y (mgr.24454) 429 : cluster [DBG] pgmap v687: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 5 op/s 2024-02-18T00:39:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:24.902789+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:23.741791+0000 mgr.y (mgr.24454) 428 : audit [DBG] from='client.31410 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:39:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:23.917610+0000 mgr.y (mgr.24454) 429 : cluster [DBG] pgmap v687: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 5 op/s 2024-02-18T00:39:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:24.902789+0000 mon.a (mon.0) 2610 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:25.567 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":686,"stamp":"2024-02-18T00:39:21.916552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4007344,"num_objects":397,"num_object_clones":5,"num_object_copies":1191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":397,"num_whiteouts":0,"num_read":1814,"num_read_kb":1431,"num_write":1251,"num_write_kb":4565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":2807,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":13,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2648,"ondisk_log_size":2648,"up":714,"acting":714,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":700,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2178040,"kb_used_data":48320,"kb_used_omap":0,"kb_used_meta":2129664,"kb_avail":654038024,"statfs":{"total":671965249536,"available":669734936576,"internally_reserved":0,"allocated":49479680,"data_stored":31560083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2180775936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35,"num_objects":10,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":12,"num_read_kb":7,"num_write":22,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000789"},"pg_stats":[{"pgid":"216.1f","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726810+0000","last_change":"2024-02-18T00:39:18.713362+0000","last_active":"2024-02-18T00:39:19.726810+0000","last_peered":"2024-02-18T00:39:19.726810+0000","last_clean":"2024-02-18T00:39:19.726810+0000","last_became_active":"2024-02-18T00:39:18.713060+0000","last_became_peered":"2024-02-18T00:39:18.713060+0000","last_unstale":"2024-02-18T00:39:19.726810+0000","last_undegraded":"2024-02-18T00:39:19.726810+0000","last_fullsized":"2024-02-18T00:39:19.726810+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"216.1e","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709932+0000","last_change":"2024-02-18T00:39:18.706628+0000","last_active":"2024-02-18T00:39:19.709932+0000","last_peered":"2024-02-18T00:39:19.709932+0000","last_clean":"2024-02-18T00:39:19.709932+0000","last_became_active":"2024-02-18T00:39:18.706338+0000","last_became_peered":"2024-02-18T00:39:18.706338+0000","last_unstale":"2024-02-18T00:39:19.709932+0000","last_undegraded":"2024-02-18T00:39:19.709932+0000","last_fullsized":"2024-02-18T00:39:19.709932+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"216.1d","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716946+0000","last_change":"2024-02-18T00:39:18.711427+0000","last_active":"2024-02-18T00:39:19.716946+0000","last_peered":"2024-02-18T00:39:19.716946+0000","last_clean":"2024-02-18T00:39:19.716946+0000","last_became_active":"2024-02-18T00:39:18.711140+0000","last_became_peered":"2024-02-18T00:39:18.711140+0000","last_unstale":"2024-02-18T00:39:19.716946+0000","last_undegraded":"2024-02-18T00:39:19.716946+0000","last_fullsized":"2024-02-18T00:39:19.716946+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"216.1c","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719747+0000","last_change":"2024-02-18T00:39:18.713682+0000","last_active":"2024-02-18T00:39:19.719747+0000","last_peered":"2024-02-18T00:39:19.719747+0000","last_clean":"2024-02-18T00:39:19.719747+0000","last_became_active":"2024-02-18T00:39:18.713332+0000","last_became_peered":"2024-02-18T00:39:18.713332+0000","last_unstale":"2024-02-18T00:39:19.719747+0000","last_undegraded":"2024-02-18T00:39:19.719747+0000","last_fullsized":"2024-02-18T00:39:19.719747+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"216.1b","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719506+0000","last_change":"2024-02-18T00:39:18.710334+0000","last_active":"2024-02-18T00:39:19.719506+0000","last_peered":"2024-02-18T00:39:19.719506+0000","last_clean":"2024-02-18T00:39:19.719506+0000","last_became_active":"2024-02-18T00:39:18.709866+0000","last_became_peered":"2024-02-18T00:39:18.709866+0000","last_unstale":"2024-02-18T00:39:19.719506+0000","last_undegraded":"2024-02-18T00:39:19.719506+0000","last_fullsized":"2024-02-18T00:39:19.719506+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"216.1a","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719308+0000","last_change":"2024-02-18T00:39:18.710312+0000","last_active":"2024-02-18T00:39:19.719308+0000","last_peered":"2024-02-18T00:39:19.719308+0000","last_clean":"2024-02-18T00:39:19.719308+0000","last_became_active":"2024-02-18T00:39:18.709975+0000","last_became_peered":"2024-02-18T00:39:18.709975+0000","last_unstale":"2024-02-18T00:39:19.719308+0000","last_undegraded":"2024-02-18T00:39:19.719308+0000","last_fullsized":"2024-02-18T00:39:19.719308+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"216.19","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719207+0000","last_change":"2024-02-18T00:39:18.705849+0000","last_active":"2024-02-18T00:39:19.719207+0000","last_peered":"2024-02-18T00:39:19.719207+0000","last_clean":"2024-02-18T00:39:19.719207+0000","last_became_active":"2024-02-18T00:39:18.705435+0000","last_became_peered":"2024-02-18T00:39:18.705435+0000","last_unstale":"2024-02-18T00:39:19.719207+0000","last_undegraded":"2024-02-18T00:39:19.719207+0000","last_fullsized":"2024-02-18T00:39:19.719207+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"216.18","version":"457'3","reported_seq":14,"reported_epoch":457,"state":"active+clean","last_fresh":"2024-02-18T00:39:20.727283+0000","last_change":"2024-02-18T00:39:18.716907+0000","last_active":"2024-02-18T00:39:20.727283+0000","last_peered":"2024-02-18T00:39:20.727283+0000","last_clean":"2024-02-18T00:39:20.727283+0000","last_became_active":"2024-02-18T00:39:18.716693+0000","last_became_peered":"2024-02-18T00:39:18.716693+0000","last_unstale":"2024-02-18T00:39:20.727283+0000","last_undegraded":"2024-02-18T00:39:20.727283+0000","last_fullsized":"2024-02-18T00:39:20.727283+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"216.17","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709092+0000","last_change":"2024-02-18T00:39:18.704831+0000","last_active":"2024-02-18T00:39:19.709092+0000","last_peered":"2024-02-18T00:39:19.709092+0000","last_clean":"2024-02-18T00:39:19.709092+0000","last_became_active":"2024-02-18T00:39:18.704528+0000","last_became_peered":"2024-02-18T00:39:18.704528+0000","last_unstale":"2024-02-18T00:39:19.709092+0000","last_undegraded":"2024-02-18T00:39:19.709092+0000","last_fullsized":"2024-02-18T00:39:19.709092+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"216.16","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710623+0000","last_change":"2024-02-18T00:39:18.706372+0000","last_active":"2024-02-18T00:39:19.710623+0000","last_peered":"2024-02-18T00:39:19.710623+0000","last_clean":"2024-02-18T00:39:19.710623+0000","last_became_active":"2024-02-18T00:39:18.706068+0000","last_became_peered":"2024-02-18T00:39:18.706068+0000","last_unstale":"2024-02-18T00:39:19.710623+0000","last_undegraded":"2024-02-18T00:39:19.710623+0000","last_fullsized":"2024-02-18T00:39:19.710623+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"216.15","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720146+0000","last_change":"2024-02-18T00:39:18.713282+0000","last_active":"2024-02-18T00:39:19.720146+0000","last_peered":"2024-02-18T00:39:19.720146+0000","last_clean":"2024-02-18T00:39:19.720146+0000","last_became_active":"2024-02-18T00:39:18.712967+0000","last_became_peered":"2024-02-18T00:39:18.712967+0000","last_unstale":"2024-02-18T00:39:19.720146+0000","last_undegraded":"2024-02-18T00:39:19.720146+0000","last_fullsized":"2024-02-18T00:39:19.720146+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"216.14","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719869+0000","last_change":"2024-02-18T00:39:18.705198+0000","last_active":"2024-02-18T00:39:19.719869+0000","last_peered":"2024-02-18T00:39:19.719869+0000","last_clean":"2024-02-18T00:39:19.719869+0000","last_became_active":"2024-02-18T00:39:18.704835+0000","last_became_peered":"2024-02-18T00:39:18.704835+0000","last_unstale":"2024-02-18T00:39:19.719869+0000","last_undegraded":"2024-02-18T00:39:19.719869+0000","last_fullsized":"2024-02-18T00:39:19.719869+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"216.13","version":"457'2","reported_seq":13,"reported_epoch":457,"state":"active+clean","last_fresh":"2024-02-18T00:39:20.720565+0000","last_change":"2024-02-18T00:39:18.710148+0000","last_active":"2024-02-18T00:39:20.720565+0000","last_peered":"2024-02-18T00:39:20.720565+0000","last_clean":"2024-02-18T00:39:20.720565+0000","last_became_active":"2024-02-18T00:39:18.709479+0000","last_became_peered":"2024-02-18T00:39:18.709479+0000","last_unstale":"2024-02-18T00:39:20.720565+0000","last_undegraded":"2024-02-18T00:39:20.720565+0000","last_fullsized":"2024-02-18T00:39:20.720565+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"216.12","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719365+0000","last_change":"2024-02-18T00:39:18.706801+0000","last_active":"2024-02-18T00:39:19.719365+0000","last_peered":"2024-02-18T00:39:19.719365+0000","last_clean":"2024-02-18T00:39:19.719365+0000","last_became_active":"2024-02-18T00:39:18.706372+0000","last_became_peered":"2024-02-18T00:39:18.706372+0000","last_unstale":"2024-02-18T00:39:19.719365+0000","last_undegraded":"2024-02-18T00:39:19.719365+0000","last_fullsized":"2024-02-18T00:39:19.719365+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"216.11","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.715867+0000","last_change":"2024-02-18T00:39:18.716379+0000","last_active":"2024-02-18T00:39:19.715867+0000","last_peered":"2024-02-18T00:39:19.715867+0000","last_clean":"2024-02-18T00:39:19.715867+0000","last_became_active":"2024-02-18T00:39:18.716019+0000","last_became_peered":"2024-02-18T00:39:18.716019+0000","last_unstale":"2024-02-18T00:39:19.715867+0000","last_undegraded":"2024-02-18T00:39:19.715867+0000","last_fullsized":"2024-02-18T00:39:19.715867+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"216.10","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719237+0000","last_change":"2024-02-18T00:39:18.707089+0000","last_active":"2024-02-18T00:39:19.719237+0000","last_peered":"2024-02-18T00:39:19.719237+0000","last_clean":"2024-02-18T00:39:19.719237+0000","last_became_active":"2024-02-18T00:39:18.706463+0000","last_became_peered":"2024-02-18T00:39:18.706463+0000","last_unstale":"2024-02-18T00:39:19.719237+0000","last_undegraded":"2024-02-18T00:39:19.719237+0000","last_fullsized":"2024-02-18T00:39:19.719237+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"216.f","version":"456'1","reported_seq":12,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.733375+0000","last_change":"2024-02-18T00:39:18.708962+0000","last_active":"2024-02-18T00:39:19.733375+0000","last_peered":"2024-02-18T00:39:19.733375+0000","last_clean":"2024-02-18T00:39:19.733375+0000","last_became_active":"2024-02-18T00:39:18.708556+0000","last_became_peered":"2024-02-18T00:39:18.708556+0000","last_unstale":"2024-02-18T00:39:19.733375+0000","last_undegraded":"2024-02-18T00:39:19.733375+0000","last_fullsized":"2024-02-18T00:39:19.733375+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"216.e","version":"456'2","reported_seq":13,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.739218+0000","last_change":"2024-02-18T00:39:18.708889+0000","last_active":"2024-02-18T00:39:19.739218+0000","last_peered":"2024-02-18T00:39:19.739218+0000","last_clean":"2024-02-18T00:39:19.739218+0000","last_became_active":"2024-02-18T00:39:18.708567+0000","last_became_peered":"2024-02-18T00:39:18.708567+0000","last_unstale":"2024-02-18T00:39:19.739218+0000","last_undegraded":"2024-02-18T00:39:19.739218+0000","last_fullsized":"2024-02-18T00:39:19.739218+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"216.d","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720075+0000","last_change":"2024-02-18T00:39:18.709914+0000","last_active":"2024-02-18T00:39:19.720075+0000","last_peered":"2024-02-18T00:39:19.720075+0000","last_clean":"2024-02-18T00:39:19.720075+0000","last_became_active":"2024-02-18T00:39:18.709439+0000","last_became_peered":"2024-02-18T00:39:18.709439+0000","last_unstale":"2024-02-18T00:39:19.720075+0000","last_undegraded":"2024-02-18T00:39:19.720075+0000","last_fullsized":"2024-02-18T00:39:19.720075+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"216.c","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.708913+0000","last_change":"2024-02-18T00:39:18.716321+0000","last_active":"2024-02-18T00:39:19.708913+0000","last_peered":"2024-02-18T00:39:19.708913+0000","last_clean":"2024-02-18T00:39:19.708913+0000","last_became_active":"2024-02-18T00:39:18.715999+0000","last_became_peered":"2024-02-18T00:39:18.715999+0000","last_unstale":"2024-02-18T00:39:19.708913+0000","last_undegraded":"2024-02-18T00:39:19.708913+0000","last_fullsized":"2024-02-18T00:39:19.708913+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"216.b","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709400+0000","last_change":"2024-02-18T00:39:18.710084+0000","last_active":"2024-02-18T00:39:19.709400+0000","last_peered":"2024-02-18T00:39:19.709400+0000","last_clean":"2024-02-18T00:39:19.709400+0000","last_became_active":"2024-02-18T00:39:18.709790+0000","last_became_peered":"2024-02-18T00:39:18.709790+0000","last_unstale":"2024-02-18T00:39:19.709400+0000","last_undegraded":"2024-02-18T00:39:19.709400+0000","last_fullsized":"2024-02-18T00:39:19.709400+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"216.a","version":"456'3","reported_seq":14,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.734867+0000","last_change":"2024-02-18T00:39:18.706837+0000","last_active":"2024-02-18T00:39:19.734867+0000","last_peered":"2024-02-18T00:39:19.734867+0000","last_clean":"2024-02-18T00:39:19.734867+0000","last_became_active":"2024-02-18T00:39:18.706540+0000","last_became_peered":"2024-02-18T00:39:18.706540+0000","last_unstale":"2024-02-18T00:39:19.734867+0000","last_undegraded":"2024-02-18T00:39:19.734867+0000","last_fullsized":"2024-02-18T00:39:19.734867+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"216.9","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711016+0000","last_change":"2024-02-18T00:39:18.714557+0000","last_active":"2024-02-18T00:39:19.711016+0000","last_peered":"2024-02-18T00:39:19.711016+0000","last_clean":"2024-02-18T00:39:19.711016+0000","last_became_active":"2024-02-18T00:39:18.714250+0000","last_became_peered":"2024-02-18T00:39:18.714250+0000","last_unstale":"2024-02-18T00:39:19.711016+0000","last_undegraded":"2024-02-18T00:39:19.711016+0000","last_fullsized":"2024-02-18T00:39:19.711016+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"216.8","version":"456'1","reported_seq":12,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.730763+0000","last_change":"2024-02-18T00:39:18.702730+0000","last_active":"2024-02-18T00:39:19.730763+0000","last_peered":"2024-02-18T00:39:19.730763+0000","last_clean":"2024-02-18T00:39:19.730763+0000","last_became_active":"2024-02-18T00:39:18.702410+0000","last_became_peered":"2024-02-18T00:39:18.702410+0000","last_unstale":"2024-02-18T00:39:19.730763+0000","last_undegraded":"2024-02-18T00:39:19.730763+0000","last_fullsized":"2024-02-18T00:39:19.730763+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"216.7","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709224+0000","last_change":"2024-02-18T00:39:18.711785+0000","last_active":"2024-02-18T00:39:19.709224+0000","last_peered":"2024-02-18T00:39:19.709224+0000","last_clean":"2024-02-18T00:39:19.709224+0000","last_became_active":"2024-02-18T00:39:18.711504+0000","last_became_peered":"2024-02-18T00:39:18.711504+0000","last_unstale":"2024-02-18T00:39:19.709224+0000","last_undegraded":"2024-02-18T00:39:19.709224+0000","last_fullsized":"2024-02-18T00:39:19.709224+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"216.6","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710625+0000","last_change":"2024-02-18T00:39:18.707390+0000","last_active":"2024-02-18T00:39:19.710625+0000","last_peered":"2024-02-18T00:39:19.710625+0000","last_clean":"2024-02-18T00:39:19.710625+0000","last_became_active":"2024-02-18T00:39:18.707030+0000","last_became_peered":"2024-02-18T00:39:18.707030+0000","last_unstale":"2024-02-18T00:39:19.710625+0000","last_undegraded":"2024-02-18T00:39:19.710625+0000","last_fullsized":"2024-02-18T00:39:19.710625+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"216.5","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709280+0000","last_change":"2024-02-18T00:39:18.711118+0000","last_active":"2024-02-18T00:39:19.709280+0000","last_peered":"2024-02-18T00:39:19.709280+0000","last_clean":"2024-02-18T00:39:19.709280+0000","last_became_active":"2024-02-18T00:39:18.710809+0000","last_became_peered":"2024-02-18T00:39:18.710809+0000","last_unstale":"2024-02-18T00:39:19.709280+0000","last_undegraded":"2024-02-18T00:39:19.709280+0000","last_fullsized":"2024-02-18T00:39:19.709280+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"216.4","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710203+0000","last_change":"2024-02-18T00:39:18.707163+0000","last_active":"2024-02-18T00:39:19.710203+0000","last_peered":"2024-02-18T00:39:19.710203+0000","last_clean":"2024-02-18T00:39:19.710203+0000","last_became_active":"2024-02-18T00:39:18.706841+0000","last_became_peered":"2024-02-18T00:39:18.706841+0000","last_unstale":"2024-02-18T00:39:19.710203+0000","last_undegraded":"2024-02-18T00:39:19.710203+0000","last_fullsized":"2024-02-18T00:39:19.710203+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"216.3","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710169+0000","last_change":"2024-02-18T00:39:18.711502+0000","last_active":"2024-02-18T00:39:19.710169+0000","last_peered":"2024-02-18T00:39:19.710169+0000","last_clean":"2024-02-18T00:39:19.710169+0000","last_became_active":"2024-02-18T00:39:18.711066+0000","last_became_peered":"2024-02-18T00:39:18.711066+0000","last_unstale":"2024-02-18T00:39:19.710169+0000","last_undegraded":"2024-02-18T00:39:19.710169+0000","last_fullsized":"2024-02-18T00:39:19.710169+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"216.2","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.725922+0000","last_change":"2024-02-18T00:39:18.715985+0000","last_active":"2024-02-18T00:39:19.725922+0000","last_peered":"2024-02-18T00:39:19.725922+0000","last_clean":"2024-02-18T00:39:19.725922+0000","last_became_active":"2024-02-18T00:39:18.715681+0000","last_became_peered":"2024-02-18T00:39:18.715681+0000","last_unstale":"2024-02-18T00:39:19.725922+0000","last_undegraded":"2024-02-18T00:39:19.725922+0000","last_fullsized":"2024-02-18T00:39:19.725922+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"216.1","version":"0'0","reported_seq":11,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.725797+0000","last_change":"2024-02-18T00:39:18.715122+0000","last_active":"2024-02-18T00:39:19.725797+0000","last_peered":"2024-02-18T00:39:19.725797+0000","last_clean":"2024-02-18T00:39:19.725797+0000","last_became_active":"2024-02-18T00:39:18.714899+0000","last_became_peered":"2024-02-18T00:39:18.714899+0000","last_unstale":"2024-02-18T00:39:19.725797+0000","last_undegraded":"2024-02-18T00:39:19.725797+0000","last_fullsized":"2024-02-18T00:39:19.725797+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"216.0","version":"456'2","reported_seq":13,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.737002+0000","last_change":"2024-02-18T00:39:18.707287+0000","last_active":"2024-02-18T00:39:19.737002+0000","last_peered":"2024-02-18T00:39:19.737002+0000","last_clean":"2024-02-18T00:39:19.737002+0000","last_became_active":"2024-02-18T00:39:18.706850+0000","last_became_peered":"2024-02-18T00:39:18.706850+0000","last_unstale":"2024-02-18T00:39:19.737002+0000","last_undegraded":"2024-02-18T00:39:19.737002+0000","last_fullsized":"2024-02-18T00:39:19.737002+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:18.097533+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.097533+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.0","version":"249'68","reported_seq":452,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.708649+0000","last_change":"2024-02-18T00:38:59.771459+0000","last_active":"2024-02-18T00:39:19.708649+0000","last_peered":"2024-02-18T00:39:19.708649+0000","last_clean":"2024-02-18T00:39:19.708649+0000","last_became_active":"2024-02-18T00:33:48.718673+0000","last_became_peered":"2024-02-18T00:33:48.718673+0000","last_unstale":"2024-02-18T00:39:19.708649+0000","last_undegraded":"2024-02-18T00:39:19.708649+0000","last_fullsized":"2024-02-18T00:39:19.708649+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:38:57.506738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.506738+0000","log_size":68,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1","version":"401'6","reported_seq":421,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.725985+0000","last_change":"2024-02-18T00:38:59.769009+0000","last_active":"2024-02-18T00:39:19.725985+0000","last_peered":"2024-02-18T00:39:19.725985+0000","last_clean":"2024-02-18T00:39:19.725985+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:39:19.725985+0000","last_undegraded":"2024-02-18T00:39:19.725985+0000","last_fullsized":"2024-02-18T00:39:19.725985+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'2","last_scrub_stamp":"2024-02-18T00:37:29.759114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.759114+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"442'87","reported_seq":496,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716050+0000","last_change":"2024-02-18T00:39:01.765858+0000","last_active":"2024-02-18T00:39:19.716050+0000","last_peered":"2024-02-18T00:39:19.716050+0000","last_clean":"2024-02-18T00:39:19.716050+0000","last_became_active":"2024-02-18T00:33:48.728886+0000","last_became_peered":"2024-02-18T00:33:48.728886+0000","last_unstale":"2024-02-18T00:39:19.716050+0000","last_undegraded":"2024-02-18T00:39:19.716050+0000","last_fullsized":"2024-02-18T00:39:19.716050+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'85","last_scrub_stamp":"2024-02-18T00:38:58.749675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:58.749675+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":72,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":27,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"347'75","reported_seq":483,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709757+0000","last_change":"2024-02-18T00:39:14.407030+0000","last_active":"2024-02-18T00:39:19.709757+0000","last_peered":"2024-02-18T00:39:19.709757+0000","last_clean":"2024-02-18T00:39:19.709757+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:39:19.709757+0000","last_undegraded":"2024-02-18T00:39:19.709757+0000","last_fullsized":"2024-02-18T00:39:19.709757+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'75","last_scrub_stamp":"2024-02-18T00:39:14.406966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.406966+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":65,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"404'123","reported_seq":525,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710026+0000","last_change":"2024-02-18T00:39:13.645506+0000","last_active":"2024-02-18T00:39:19.710026+0000","last_peered":"2024-02-18T00:39:19.710026+0000","last_clean":"2024-02-18T00:39:19.710026+0000","last_became_active":"2024-02-18T00:33:48.723829+0000","last_became_peered":"2024-02-18T00:33:48.723829+0000","last_unstale":"2024-02-18T00:39:19.710026+0000","last_undegraded":"2024-02-18T00:39:19.710026+0000","last_fullsized":"2024-02-18T00:39:19.710026+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"404'123","last_scrub_stamp":"2024-02-18T00:39:13.645424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:13.645424+0000","log_size":123,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":39,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"296'95","reported_seq":496,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710537+0000","last_change":"2024-02-18T00:38:59.691992+0000","last_active":"2024-02-18T00:39:19.710537+0000","last_peered":"2024-02-18T00:39:19.710537+0000","last_clean":"2024-02-18T00:39:19.710537+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:39:19.710537+0000","last_undegraded":"2024-02-18T00:39:19.710537+0000","last_fullsized":"2024-02-18T00:39:19.710537+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'95","last_scrub_stamp":"2024-02-18T00:38:59.313626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.313626+0000","log_size":95,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":68,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":32,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"404'127","reported_seq":492,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710947+0000","last_change":"2024-02-18T00:38:59.774582+0000","last_active":"2024-02-18T00:39:19.710947+0000","last_peered":"2024-02-18T00:39:19.710947+0000","last_clean":"2024-02-18T00:39:19.710947+0000","last_became_active":"2024-02-18T00:33:48.726447+0000","last_became_peered":"2024-02-18T00:33:48.726447+0000","last_unstale":"2024-02-18T00:39:19.710947+0000","last_undegraded":"2024-02-18T00:39:19.710947+0000","last_fullsized":"2024-02-18T00:39:19.710947+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'124","last_scrub_stamp":"2024-02-18T00:36:53.649677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.649677+0000","log_size":127,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":95,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":39,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1],"acting":[2,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.7","version":"412'76","reported_seq":475,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710411+0000","last_change":"2024-02-18T00:39:17.413159+0000","last_active":"2024-02-18T00:39:19.710411+0000","last_peered":"2024-02-18T00:39:19.710411+0000","last_clean":"2024-02-18T00:39:19.710411+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:39:19.710411+0000","last_undegraded":"2024-02-18T00:39:19.710411+0000","last_fullsized":"2024-02-18T00:39:19.710411+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'73","last_scrub_stamp":"2024-02-18T00:36:37.532316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:37.532316+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":24,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'70","reported_seq":443,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.725848+0000","last_change":"2024-02-18T00:38:59.769741+0000","last_active":"2024-02-18T00:39:19.725848+0000","last_peered":"2024-02-18T00:39:19.725848+0000","last_clean":"2024-02-18T00:39:19.725848+0000","last_became_active":"2024-02-18T00:33:48.710758+0000","last_became_peered":"2024-02-18T00:33:48.710758+0000","last_unstale":"2024-02-18T00:39:19.725848+0000","last_undegraded":"2024-02-18T00:39:19.725848+0000","last_fullsized":"2024-02-18T00:39:19.725848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-18T00:37:44.685592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.685592+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.9","version":"347'52","reported_seq":459,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709425+0000","last_change":"2024-02-18T00:38:59.690323+0000","last_active":"2024-02-18T00:39:19.709425+0000","last_peered":"2024-02-18T00:39:19.709425+0000","last_clean":"2024-02-18T00:39:19.709425+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:39:19.709425+0000","last_undegraded":"2024-02-18T00:39:19.709425+0000","last_fullsized":"2024-02-18T00:39:19.709425+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'52","last_scrub_stamp":"2024-02-18T00:38:27.136753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:27.136753+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"347'41","reported_seq":426,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.708813+0000","last_change":"2024-02-18T00:39:02.513997+0000","last_active":"2024-02-18T00:39:19.708813+0000","last_peered":"2024-02-18T00:39:19.708813+0000","last_clean":"2024-02-18T00:39:19.708813+0000","last_became_active":"2024-02-18T00:33:48.674785+0000","last_became_peered":"2024-02-18T00:33:48.674785+0000","last_unstale":"2024-02-18T00:39:19.708813+0000","last_undegraded":"2024-02-18T00:39:19.708813+0000","last_fullsized":"2024-02-18T00:39:19.708813+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'41","last_scrub_stamp":"2024-02-18T00:39:02.513916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.513916+0000","log_size":41,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":43,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":14,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.b","version":"428'66","reported_seq":451,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719652+0000","last_change":"2024-02-18T00:38:59.690125+0000","last_active":"2024-02-18T00:39:19.719652+0000","last_peered":"2024-02-18T00:39:19.719652+0000","last_clean":"2024-02-18T00:39:19.719652+0000","last_became_active":"2024-02-18T00:33:48.742248+0000","last_became_peered":"2024-02-18T00:33:48.742248+0000","last_unstale":"2024-02-18T00:39:19.719652+0000","last_undegraded":"2024-02-18T00:39:19.719652+0000","last_fullsized":"2024-02-18T00:39:19.719652+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"428'66","last_scrub_stamp":"2024-02-18T00:38:34.725717+0000","last_deep_scrub":"428'66","last_deep_scrub_stamp":"2024-02-18T00:38:34.725717+0000","last_clean_scrub_stamp":"2024-02-18T00:38:34.725717+0000","log_size":66,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":59,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":20,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.c","version":"379'18","reported_seq":428,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720099+0000","last_change":"2024-02-18T00:38:59.689614+0000","last_active":"2024-02-18T00:39:19.720099+0000","last_peered":"2024-02-18T00:39:19.720099+0000","last_clean":"2024-02-18T00:39:19.720099+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:39:19.720099+0000","last_undegraded":"2024-02-18T00:39:19.720099+0000","last_fullsized":"2024-02-18T00:39:19.720099+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'18","last_scrub_stamp":"2024-02-18T00:38:30.791397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:30.791397+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"347'29","reported_seq":463,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710495+0000","last_change":"2024-02-18T00:38:59.691571+0000","last_active":"2024-02-18T00:39:19.710495+0000","last_peered":"2024-02-18T00:39:19.710495+0000","last_clean":"2024-02-18T00:39:19.710495+0000","last_became_active":"2024-02-18T00:33:48.741211+0000","last_became_peered":"2024-02-18T00:33:48.741211+0000","last_unstale":"2024-02-18T00:39:19.710495+0000","last_undegraded":"2024-02-18T00:39:19.710495+0000","last_fullsized":"2024-02-18T00:39:19.710495+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'29","last_scrub_stamp":"2024-02-18T00:38:28.139054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.139054+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"347'83","reported_seq":483,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711691+0000","last_change":"2024-02-18T00:38:59.691183+0000","last_active":"2024-02-18T00:39:19.711691+0000","last_peered":"2024-02-18T00:39:19.711691+0000","last_clean":"2024-02-18T00:39:19.711691+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:39:19.711691+0000","last_undegraded":"2024-02-18T00:39:19.711691+0000","last_fullsized":"2024-02-18T00:39:19.711691+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'83","last_scrub_stamp":"2024-02-18T00:38:30.700348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:30.700348+0000","log_size":83,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":67,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"452'73","reported_seq":469,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710983+0000","last_change":"2024-02-18T00:39:08.309384+0000","last_active":"2024-02-18T00:39:19.710983+0000","last_peered":"2024-02-18T00:39:19.710983+0000","last_clean":"2024-02-18T00:39:19.710983+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:39:19.710983+0000","last_undegraded":"2024-02-18T00:39:19.710983+0000","last_fullsized":"2024-02-18T00:39:19.710983+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:39:08.309293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:08.309293+0000","log_size":73,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":54,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":23,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"347'57","reported_seq":477,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.725748+0000","last_change":"2024-02-18T00:38:59.769435+0000","last_active":"2024-02-18T00:39:19.725748+0000","last_peered":"2024-02-18T00:39:19.725748+0000","last_clean":"2024-02-18T00:39:19.725748+0000","last_became_active":"2024-02-18T00:33:48.680606+0000","last_became_peered":"2024-02-18T00:33:48.680606+0000","last_unstale":"2024-02-18T00:39:19.725748+0000","last_undegraded":"2024-02-18T00:39:19.725748+0000","last_fullsized":"2024-02-18T00:39:19.725748+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'57","last_scrub_stamp":"2024-02-18T00:37:36.817719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.817719+0000","log_size":57,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":41,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":18,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":8,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"398'21","reported_seq":436,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.725933+0000","last_change":"2024-02-18T00:38:59.768999+0000","last_active":"2024-02-18T00:39:19.725933+0000","last_peered":"2024-02-18T00:39:19.725933+0000","last_clean":"2024-02-18T00:39:19.725933+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:39:19.725933+0000","last_undegraded":"2024-02-18T00:39:19.725933+0000","last_fullsized":"2024-02-18T00:39:19.725933+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"214'19","last_scrub_stamp":"2024-02-18T00:37:31.817281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.817281+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"408'77","reported_seq":481,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.708937+0000","last_change":"2024-02-18T00:39:01.511082+0000","last_active":"2024-02-18T00:39:19.708937+0000","last_peered":"2024-02-18T00:39:19.708937+0000","last_clean":"2024-02-18T00:39:19.708937+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:39:19.708937+0000","last_undegraded":"2024-02-18T00:39:19.708937+0000","last_fullsized":"2024-02-18T00:39:19.708937+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"408'77","last_scrub_stamp":"2024-02-18T00:39:01.510999+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.510999+0000","log_size":77,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":25,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"361'18","reported_seq":431,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710493+0000","last_change":"2024-02-18T00:39:18.521234+0000","last_active":"2024-02-18T00:39:19.710493+0000","last_peered":"2024-02-18T00:39:19.710493+0000","last_clean":"2024-02-18T00:39:19.710493+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:39:19.710493+0000","last_undegraded":"2024-02-18T00:39:19.710493+0000","last_fullsized":"2024-02-18T00:39:19.710493+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"361'18","last_scrub_stamp":"2024-02-18T00:39:18.521162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.521162+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"275'24","reported_seq":442,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710927+0000","last_change":"2024-02-18T00:38:59.692296+0000","last_active":"2024-02-18T00:39:19.710927+0000","last_peered":"2024-02-18T00:39:19.710927+0000","last_clean":"2024-02-18T00:39:19.710927+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:39:19.710927+0000","last_undegraded":"2024-02-18T00:39:19.710927+0000","last_fullsized":"2024-02-18T00:39:19.710927+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:38:57.350170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.350170+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"457'32","reported_seq":453,"reported_epoch":457,"state":"active+clean","last_fresh":"2024-02-18T00:39:20.730795+0000","last_change":"2024-02-18T00:38:59.764345+0000","last_active":"2024-02-18T00:39:20.730795+0000","last_peered":"2024-02-18T00:39:20.730795+0000","last_clean":"2024-02-18T00:39:20.730795+0000","last_became_active":"2024-02-18T00:33:48.671138+0000","last_became_peered":"2024-02-18T00:33:48.671138+0000","last_unstale":"2024-02-18T00:39:20.730795+0000","last_undegraded":"2024-02-18T00:39:20.730795+0000","last_fullsized":"2024-02-18T00:39:20.730795+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'21","last_scrub_stamp":"2024-02-18T00:37:41.575814+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.575814+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000046,"num_objects":6,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":13,"num_write_kb":3919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"225'102","reported_seq":504,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720371+0000","last_change":"2024-02-18T00:39:03.663944+0000","last_active":"2024-02-18T00:39:19.720371+0000","last_peered":"2024-02-18T00:39:19.720371+0000","last_clean":"2024-02-18T00:39:19.720371+0000","last_became_active":"2024-02-18T00:33:48.712233+0000","last_became_peered":"2024-02-18T00:33:48.712233+0000","last_unstale":"2024-02-18T00:39:19.720371+0000","last_undegraded":"2024-02-18T00:39:19.720371+0000","last_fullsized":"2024-02-18T00:39:19.720371+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:39:03.663869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.663869+0000","log_size":102,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":66,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":30,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":377,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716004+0000","last_change":"2024-02-18T00:39:18.858081+0000","last_active":"2024-02-18T00:39:19.716004+0000","last_peered":"2024-02-18T00:39:19.716004+0000","last_clean":"2024-02-18T00:39:19.716004+0000","last_became_active":"2024-02-18T00:33:48.629817+0000","last_became_peered":"2024-02-18T00:33:48.629817+0000","last_unstale":"2024-02-18T00:39:19.716004+0000","last_undegraded":"2024-02-18T00:39:19.716004+0000","last_fullsized":"2024-02-18T00:39:19.716004+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.857990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.857990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"412'114","reported_seq":517,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709070+0000","last_change":"2024-02-18T00:38:59.774118+0000","last_active":"2024-02-18T00:39:19.709070+0000","last_peered":"2024-02-18T00:39:19.709070+0000","last_clean":"2024-02-18T00:39:19.709070+0000","last_became_active":"2024-02-18T00:33:48.674179+0000","last_became_peered":"2024-02-18T00:33:48.674179+0000","last_unstale":"2024-02-18T00:39:19.709070+0000","last_undegraded":"2024-02-18T00:39:19.709070+0000","last_fullsized":"2024-02-18T00:39:19.709070+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"265'112","last_scrub_stamp":"2024-02-18T00:36:38.583056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:38.583056+0000","log_size":114,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":84,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":35,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":370,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719369+0000","last_change":"2024-02-18T00:37:01.774863+0000","last_active":"2024-02-18T00:39:19.719369+0000","last_peered":"2024-02-18T00:39:19.719369+0000","last_clean":"2024-02-18T00:39:19.719369+0000","last_became_active":"2024-02-18T00:33:48.701035+0000","last_became_peered":"2024-02-18T00:33:48.701035+0000","last_unstale":"2024-02-18T00:39:19.719369+0000","last_undegraded":"2024-02-18T00:39:19.719369+0000","last_fullsized":"2024-02-18T00:39:19.719369+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:01.774795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.774795+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"223'34","reported_seq":454,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709663+0000","last_change":"2024-02-18T00:38:59.691596+0000","last_active":"2024-02-18T00:39:19.709663+0000","last_peered":"2024-02-18T00:39:19.709663+0000","last_clean":"2024-02-18T00:39:19.709663+0000","last_became_active":"2024-02-18T00:33:48.679822+0000","last_became_peered":"2024-02-18T00:33:48.679822+0000","last_unstale":"2024-02-18T00:39:19.709663+0000","last_undegraded":"2024-02-18T00:39:19.709663+0000","last_fullsized":"2024-02-18T00:39:19.709663+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:38:25.166249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.166249+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":331,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710978+0000","last_change":"2024-02-18T00:37:27.130328+0000","last_active":"2024-02-18T00:39:19.710978+0000","last_peered":"2024-02-18T00:39:19.710978+0000","last_clean":"2024-02-18T00:39:19.710978+0000","last_became_active":"2024-02-18T00:33:48.733227+0000","last_became_peered":"2024-02-18T00:33:48.733227+0000","last_unstale":"2024-02-18T00:39:19.710978+0000","last_undegraded":"2024-02-18T00:39:19.710978+0000","last_fullsized":"2024-02-18T00:39:19.710978+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.130252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.130252+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.1a","version":"229'103","reported_seq":506,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.715994+0000","last_change":"2024-02-18T00:38:59.761571+0000","last_active":"2024-02-18T00:39:19.715994+0000","last_peered":"2024-02-18T00:39:19.715994+0000","last_clean":"2024-02-18T00:39:19.715994+0000","last_became_active":"2024-02-18T00:33:48.649794+0000","last_became_peered":"2024-02-18T00:33:48.649794+0000","last_unstale":"2024-02-18T00:39:19.715994+0000","last_undegraded":"2024-02-18T00:39:19.715994+0000","last_fullsized":"2024-02-18T00:39:19.715994+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'103","last_scrub_stamp":"2024-02-18T00:38:31.708778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:31.708778+0000","log_size":103,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":31,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":329,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711056+0000","last_change":"2024-02-18T00:36:53.345938+0000","last_active":"2024-02-18T00:39:19.711056+0000","last_peered":"2024-02-18T00:39:19.711056+0000","last_clean":"2024-02-18T00:39:19.711056+0000","last_became_active":"2024-02-18T00:33:48.636710+0000","last_became_peered":"2024-02-18T00:33:48.636710+0000","last_unstale":"2024-02-18T00:39:19.711056+0000","last_undegraded":"2024-02-18T00:39:19.711056+0000","last_fullsized":"2024-02-18T00:39:19.711056+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.345872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.19","version":"240'75","reported_seq":483,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.715850+0000","last_change":"2024-02-18T00:38:59.761613+0000","last_active":"2024-02-18T00:39:19.715850+0000","last_peered":"2024-02-18T00:39:19.715850+0000","last_clean":"2024-02-18T00:39:19.715850+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:39:19.715850+0000","last_undegraded":"2024-02-18T00:39:19.715850+0000","last_fullsized":"2024-02-18T00:39:19.715850+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:38:22.718624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.718624+0000","log_size":75,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":57,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":24,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":386,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711016+0000","last_change":"2024-02-18T00:39:02.380493+0000","last_active":"2024-02-18T00:39:19.711016+0000","last_peered":"2024-02-18T00:39:19.711016+0000","last_clean":"2024-02-18T00:39:19.711016+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:39:19.711016+0000","last_undegraded":"2024-02-18T00:39:19.711016+0000","last_fullsized":"2024-02-18T00:39:19.711016+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:39:02.380409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.380409+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"347'18","reported_seq":432,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719356+0000","last_change":"2024-02-18T00:38:59.775856+0000","last_active":"2024-02-18T00:39:19.719356+0000","last_peered":"2024-02-18T00:39:19.719356+0000","last_clean":"2024-02-18T00:39:19.719356+0000","last_became_active":"2024-02-18T00:33:48.692593+0000","last_became_peered":"2024-02-18T00:33:48.692593+0000","last_unstale":"2024-02-18T00:39:19.719356+0000","last_undegraded":"2024-02-18T00:39:19.719356+0000","last_fullsized":"2024-02-18T00:39:19.719356+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'18","last_scrub_stamp":"2024-02-18T00:38:59.775767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.775767+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":377,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716841+0000","last_change":"2024-02-18T00:39:11.795600+0000","last_active":"2024-02-18T00:39:19.716841+0000","last_peered":"2024-02-18T00:39:19.716841+0000","last_clean":"2024-02-18T00:39:19.716841+0000","last_became_active":"2024-02-18T00:33:48.729184+0000","last_became_peered":"2024-02-18T00:33:48.729184+0000","last_unstale":"2024-02-18T00:39:19.716841+0000","last_undegraded":"2024-02-18T00:39:19.716841+0000","last_fullsized":"2024-02-18T00:39:19.716841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:11.795514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.795514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"253'52","reported_seq":460,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710302+0000","last_change":"2024-02-18T00:38:59.774788+0000","last_active":"2024-02-18T00:39:19.710302+0000","last_peered":"2024-02-18T00:39:19.710302+0000","last_clean":"2024-02-18T00:39:19.710302+0000","last_became_active":"2024-02-18T00:33:48.736941+0000","last_became_peered":"2024-02-18T00:33:48.736941+0000","last_unstale":"2024-02-18T00:39:19.710302+0000","last_undegraded":"2024-02-18T00:39:19.710302+0000","last_fullsized":"2024-02-18T00:39:19.710302+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:36:41.624180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.624180+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":366,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:20.629532+0000","last_change":"2024-02-18T00:39:20.629493+0000","last_active":"2024-02-18T00:39:20.629532+0000","last_peered":"2024-02-18T00:39:20.629532+0000","last_clean":"2024-02-18T00:39:20.629532+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:39:20.629532+0000","last_undegraded":"2024-02-18T00:39:20.629532+0000","last_fullsized":"2024-02-18T00:39:20.629532+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:20.629419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:20.629419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717002+0000","last_change":"2024-02-18T00:38:59.763127+0000","last_active":"2024-02-18T00:39:19.717002+0000","last_peered":"2024-02-18T00:39:19.717002+0000","last_clean":"2024-02-18T00:39:19.717002+0000","last_became_active":"2024-02-18T00:33:48.732374+0000","last_became_peered":"2024-02-18T00:33:48.732374+0000","last_unstale":"2024-02-18T00:39:19.717002+0000","last_undegraded":"2024-02-18T00:39:19.717002+0000","last_fullsized":"2024-02-18T00:39:19.717002+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:37:37.618667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.618667+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":370,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716767+0000","last_change":"2024-02-18T00:38:29.678682+0000","last_active":"2024-02-18T00:39:19.716767+0000","last_peered":"2024-02-18T00:39:19.716767+0000","last_clean":"2024-02-18T00:39:19.716767+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:39:19.716767+0000","last_undegraded":"2024-02-18T00:39:19.716767+0000","last_fullsized":"2024-02-18T00:39:19.716767+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:29.678581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:38:29.678581+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"449'45","reported_seq":458,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720113+0000","last_change":"2024-02-18T00:38:59.766313+0000","last_active":"2024-02-18T00:39:19.720113+0000","last_peered":"2024-02-18T00:39:19.720113+0000","last_clean":"2024-02-18T00:39:19.720113+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:39:19.720113+0000","last_undegraded":"2024-02-18T00:39:19.720113+0000","last_fullsized":"2024-02-18T00:39:19.720113+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"431'39","last_scrub_stamp":"2024-02-18T00:38:58.470138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:58.470138+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":446,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711953+0000","last_change":"2024-02-18T00:36:45.627771+0000","last_active":"2024-02-18T00:39:19.711953+0000","last_peered":"2024-02-18T00:39:19.711953+0000","last_clean":"2024-02-18T00:39:19.711953+0000","last_became_active":"2024-02-18T00:33:48.735637+0000","last_became_peered":"2024-02-18T00:33:48.735637+0000","last_unstale":"2024-02-18T00:39:19.711953+0000","last_undegraded":"2024-02-18T00:39:19.711953+0000","last_fullsized":"2024-02-18T00:39:19.711953+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.627700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.627700+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"267'18","reported_seq":428,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716638+0000","last_change":"2024-02-18T00:38:59.762397+0000","last_active":"2024-02-18T00:39:19.716638+0000","last_peered":"2024-02-18T00:39:19.716638+0000","last_clean":"2024-02-18T00:39:19.716638+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:39:19.716638+0000","last_undegraded":"2024-02-18T00:39:19.716638+0000","last_fullsized":"2024-02-18T00:39:19.716638+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'18","last_scrub_stamp":"2024-02-18T00:37:38.588190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.588190+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":395,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719114+0000","last_change":"2024-02-18T00:37:37.726927+0000","last_active":"2024-02-18T00:39:19.719114+0000","last_peered":"2024-02-18T00:39:19.719114+0000","last_clean":"2024-02-18T00:39:19.719114+0000","last_became_active":"2024-02-18T00:33:48.628919+0000","last_became_peered":"2024-02-18T00:33:48.628919+0000","last_unstale":"2024-02-18T00:39:19.719114+0000","last_undegraded":"2024-02-18T00:39:19.719114+0000","last_fullsized":"2024-02-18T00:39:19.719114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:37.726861+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.726861+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":376,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709759+0000","last_change":"2024-02-18T00:38:28.692032+0000","last_active":"2024-02-18T00:39:19.709759+0000","last_peered":"2024-02-18T00:39:19.709759+0000","last_clean":"2024-02-18T00:39:19.709759+0000","last_became_active":"2024-02-18T00:33:48.727212+0000","last_became_peered":"2024-02-18T00:33:48.727212+0000","last_unstale":"2024-02-18T00:39:19.709759+0000","last_undegraded":"2024-02-18T00:39:19.709759+0000","last_fullsized":"2024-02-18T00:39:19.709759+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.691955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"150'9","reported_seq":415,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716606+0000","last_change":"2024-02-18T00:37:33.619485+0000","last_active":"2024-02-18T00:39:19.716606+0000","last_peered":"2024-02-18T00:39:19.716606+0000","last_clean":"2024-02-18T00:39:19.716606+0000","last_became_active":"2024-02-18T00:33:48.649515+0000","last_became_peered":"2024-02-18T00:33:48.649515+0000","last_unstale":"2024-02-18T00:39:19.716606+0000","last_undegraded":"2024-02-18T00:39:19.716606+0000","last_fullsized":"2024-02-18T00:39:19.716606+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:33.619410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.619410+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"64'27","reported_seq":505,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719193+0000","last_change":"2024-02-18T00:38:17.658701+0000","last_active":"2024-02-18T00:39:19.719193+0000","last_peered":"2024-02-18T00:39:19.719193+0000","last_clean":"2024-02-18T00:39:19.719193+0000","last_became_active":"2024-02-18T00:33:48.634612+0000","last_became_peered":"2024-02-18T00:33:48.634612+0000","last_unstale":"2024-02-18T00:39:19.719193+0000","last_undegraded":"2024-02-18T00:39:19.719193+0000","last_fullsized":"2024-02-18T00:39:19.719193+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'27","last_scrub_stamp":"2024-02-18T00:38:17.658626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:17.658626+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":439,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711461+0000","last_change":"2024-02-18T00:37:02.627426+0000","last_active":"2024-02-18T00:39:19.711461+0000","last_peered":"2024-02-18T00:39:19.711461+0000","last_clean":"2024-02-18T00:39:19.711461+0000","last_became_active":"2024-02-18T00:33:48.690989+0000","last_became_peered":"2024-02-18T00:33:48.690989+0000","last_unstale":"2024-02-18T00:39:19.711461+0000","last_undegraded":"2024-02-18T00:39:19.711461+0000","last_fullsized":"2024-02-18T00:39:19.711461+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:02.627349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:02.627349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719782+0000","last_change":"2024-02-18T00:37:33.616952+0000","last_active":"2024-02-18T00:39:19.719782+0000","last_peered":"2024-02-18T00:39:19.719782+0000","last_clean":"2024-02-18T00:39:19.719782+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:39:19.719782+0000","last_undegraded":"2024-02-18T00:39:19.719782+0000","last_fullsized":"2024-02-18T00:39:19.719782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.616874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.616874+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":375,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.727141+0000","last_change":"2024-02-18T00:39:11.511714+0000","last_active":"2024-02-18T00:39:19.727141+0000","last_peered":"2024-02-18T00:39:19.727141+0000","last_clean":"2024-02-18T00:39:19.727141+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:39:19.727141+0000","last_undegraded":"2024-02-18T00:39:19.727141+0000","last_fullsized":"2024-02-18T00:39:19.727141+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:11.511597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.511597+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"64'15","reported_seq":476,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711068+0000","last_change":"2024-02-18T00:37:49.523198+0000","last_active":"2024-02-18T00:39:19.711068+0000","last_peered":"2024-02-18T00:39:19.711068+0000","last_clean":"2024-02-18T00:39:19.711068+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:39:19.711068+0000","last_undegraded":"2024-02-18T00:39:19.711068+0000","last_fullsized":"2024-02-18T00:39:19.711068+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:49.523121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:49.523121+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":433,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.727730+0000","last_change":"2024-02-18T00:38:25.644859+0000","last_active":"2024-02-18T00:39:19.727730+0000","last_peered":"2024-02-18T00:39:19.727730+0000","last_clean":"2024-02-18T00:39:19.727730+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:39:19.727730+0000","last_undegraded":"2024-02-18T00:39:19.727730+0000","last_fullsized":"2024-02-18T00:39:19.727730+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.644791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.644791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":424,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709087+0000","last_change":"2024-02-18T00:36:56.601833+0000","last_active":"2024-02-18T00:39:19.709087+0000","last_peered":"2024-02-18T00:39:19.709087+0000","last_clean":"2024-02-18T00:39:19.709087+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:39:19.709087+0000","last_undegraded":"2024-02-18T00:39:19.709087+0000","last_fullsized":"2024-02-18T00:39:19.709087+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.601727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.601727+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":438,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711269+0000","last_change":"2024-02-18T00:37:47.488889+0000","last_active":"2024-02-18T00:39:19.711269+0000","last_peered":"2024-02-18T00:39:19.711269+0000","last_clean":"2024-02-18T00:39:19.711269+0000","last_became_active":"2024-02-18T00:35:42.682832+0000","last_became_peered":"2024-02-18T00:35:42.682832+0000","last_unstale":"2024-02-18T00:39:19.711269+0000","last_undegraded":"2024-02-18T00:39:19.711269+0000","last_fullsized":"2024-02-18T00:39:19.711269+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.488821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.488821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":372,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721469+0000","last_change":"2024-02-18T00:38:25.732225+0000","last_active":"2024-02-18T00:39:19.721469+0000","last_peered":"2024-02-18T00:39:19.721469+0000","last_clean":"2024-02-18T00:39:19.721469+0000","last_became_active":"2024-02-18T00:33:48.744984+0000","last_became_peered":"2024-02-18T00:33:48.744984+0000","last_unstale":"2024-02-18T00:39:19.721469+0000","last_undegraded":"2024-02-18T00:39:19.721469+0000","last_fullsized":"2024-02-18T00:39:19.721469+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.732112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.732112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"64'26","reported_seq":506,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716880+0000","last_change":"2024-02-18T00:38:20.697327+0000","last_active":"2024-02-18T00:39:19.716880+0000","last_peered":"2024-02-18T00:39:19.716880+0000","last_clean":"2024-02-18T00:39:19.716880+0000","last_became_active":"2024-02-18T00:33:48.687487+0000","last_became_peered":"2024-02-18T00:33:48.687487+0000","last_unstale":"2024-02-18T00:39:19.716880+0000","last_undegraded":"2024-02-18T00:39:19.716880+0000","last_fullsized":"2024-02-18T00:39:19.716880+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'26","last_scrub_stamp":"2024-02-18T00:38:20.697264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.697264+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":428,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711278+0000","last_change":"2024-02-18T00:36:49.680480+0000","last_active":"2024-02-18T00:39:19.711278+0000","last_peered":"2024-02-18T00:39:19.711278+0000","last_clean":"2024-02-18T00:39:19.711278+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:39:19.711278+0000","last_undegraded":"2024-02-18T00:39:19.711278+0000","last_fullsized":"2024-02-18T00:39:19.711278+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.680414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":424,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709837+0000","last_change":"2024-02-18T00:37:00.670726+0000","last_active":"2024-02-18T00:39:19.709837+0000","last_peered":"2024-02-18T00:39:19.709837+0000","last_clean":"2024-02-18T00:39:19.709837+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:39:19.709837+0000","last_undegraded":"2024-02-18T00:39:19.709837+0000","last_fullsized":"2024-02-18T00:39:19.709837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.670646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.670646+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":328,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711096+0000","last_change":"2024-02-18T00:37:37.476740+0000","last_active":"2024-02-18T00:39:19.711096+0000","last_peered":"2024-02-18T00:39:19.711096+0000","last_clean":"2024-02-18T00:39:19.711096+0000","last_became_active":"2024-02-18T00:33:48.694854+0000","last_became_peered":"2024-02-18T00:33:48.694854+0000","last_unstale":"2024-02-18T00:39:19.711096+0000","last_undegraded":"2024-02-18T00:39:19.711096+0000","last_fullsized":"2024-02-18T00:39:19.711096+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.476676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.476676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":363,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719332+0000","last_change":"2024-02-18T00:38:59.474203+0000","last_active":"2024-02-18T00:39:19.719332+0000","last_peered":"2024-02-18T00:39:19.719332+0000","last_clean":"2024-02-18T00:39:19.719332+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:39:19.719332+0000","last_undegraded":"2024-02-18T00:39:19.719332+0000","last_fullsized":"2024-02-18T00:39:19.719332+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:59.474124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.474124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"64'17","reported_seq":459,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710938+0000","last_change":"2024-02-18T00:38:14.260160+0000","last_active":"2024-02-18T00:39:19.710938+0000","last_peered":"2024-02-18T00:39:19.710938+0000","last_clean":"2024-02-18T00:39:19.710938+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:39:19.710938+0000","last_undegraded":"2024-02-18T00:39:19.710938+0000","last_fullsized":"2024-02-18T00:39:19.710938+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:38:14.260073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:14.260073+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":333,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710182+0000","last_change":"2024-02-18T00:39:19.319094+0000","last_active":"2024-02-18T00:39:19.710182+0000","last_peered":"2024-02-18T00:39:19.710182+0000","last_clean":"2024-02-18T00:39:19.710182+0000","last_became_active":"2024-02-18T00:33:48.668682+0000","last_became_peered":"2024-02-18T00:33:48.668682+0000","last_unstale":"2024-02-18T00:39:19.710182+0000","last_undegraded":"2024-02-18T00:39:19.710182+0000","last_fullsized":"2024-02-18T00:39:19.710182+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:39:19.319010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":429,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:20.450167+0000","last_change":"2024-02-18T00:39:20.450122+0000","last_active":"2024-02-18T00:39:20.450167+0000","last_peered":"2024-02-18T00:39:20.450167+0000","last_clean":"2024-02-18T00:39:20.450167+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:39:20.450167+0000","last_undegraded":"2024-02-18T00:39:20.450167+0000","last_fullsized":"2024-02-18T00:39:20.450167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:20.450046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:20.450046+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710676+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:39:19.710676+0000","last_peered":"2024-02-18T00:39:19.710676+0000","last_clean":"2024-02-18T00:39:19.710676+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:39:19.710676+0000","last_undegraded":"2024-02-18T00:39:19.710676+0000","last_fullsized":"2024-02-18T00:39:19.710676+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":212,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711103+0000","last_change":"2024-02-18T00:38:33.312167+0000","last_active":"2024-02-18T00:39:19.711103+0000","last_peered":"2024-02-18T00:39:19.711103+0000","last_clean":"2024-02-18T00:39:19.711103+0000","last_became_active":"2024-02-18T00:35:42.669171+0000","last_became_peered":"2024-02-18T00:35:42.669171+0000","last_unstale":"2024-02-18T00:39:19.711103+0000","last_undegraded":"2024-02-18T00:39:19.711103+0000","last_fullsized":"2024-02-18T00:39:19.711103+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:33.312079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.312079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"64'10","reported_seq":441,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710447+0000","last_change":"2024-02-18T00:37:29.177448+0000","last_active":"2024-02-18T00:39:19.710447+0000","last_peered":"2024-02-18T00:39:19.710447+0000","last_clean":"2024-02-18T00:39:19.710447+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:39:19.710447+0000","last_undegraded":"2024-02-18T00:39:19.710447+0000","last_fullsized":"2024-02-18T00:39:19.710447+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'10","last_scrub_stamp":"2024-02-18T00:37:29.177369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.177369+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":442,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726462+0000","last_change":"2024-02-18T00:38:19.536204+0000","last_active":"2024-02-18T00:39:19.726462+0000","last_peered":"2024-02-18T00:39:19.726462+0000","last_clean":"2024-02-18T00:39:19.726462+0000","last_became_active":"2024-02-18T00:33:48.704471+0000","last_became_peered":"2024-02-18T00:33:48.704471+0000","last_unstale":"2024-02-18T00:39:19.726462+0000","last_undegraded":"2024-02-18T00:39:19.726462+0000","last_fullsized":"2024-02-18T00:39:19.726462+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:19.536095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:38:19.536095+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,5],"acting":[1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"150'9","reported_seq":569,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709454+0000","last_change":"2024-02-18T00:37:38.433946+0000","last_active":"2024-02-18T00:39:19.709454+0000","last_peered":"2024-02-18T00:39:19.709454+0000","last_clean":"2024-02-18T00:39:19.709454+0000","last_became_active":"2024-02-18T00:33:48.745677+0000","last_became_peered":"2024-02-18T00:33:48.745677+0000","last_unstale":"2024-02-18T00:39:19.709454+0000","last_undegraded":"2024-02-18T00:39:19.709454+0000","last_fullsized":"2024-02-18T00:39:19.709454+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:38.433877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.433877+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711287+0000","last_change":"2024-02-18T00:37:31.127432+0000","last_active":"2024-02-18T00:39:19.711287+0000","last_peered":"2024-02-18T00:39:19.711287+0000","last_clean":"2024-02-18T00:39:19.711287+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:39:19.711287+0000","last_undegraded":"2024-02-18T00:39:19.711287+0000","last_fullsized":"2024-02-18T00:39:19.711287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.127341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.127341+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":373,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726679+0000","last_change":"2024-02-18T00:38:20.588735+0000","last_active":"2024-02-18T00:39:19.726679+0000","last_peered":"2024-02-18T00:39:19.726679+0000","last_clean":"2024-02-18T00:39:19.726679+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:39:19.726679+0000","last_undegraded":"2024-02-18T00:39:19.726679+0000","last_fullsized":"2024-02-18T00:39:19.726679+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.588654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.588654+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"82'22","reported_seq":429,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720690+0000","last_change":"2024-02-18T00:39:09.538209+0000","last_active":"2024-02-18T00:39:19.720690+0000","last_peered":"2024-02-18T00:39:19.720690+0000","last_clean":"2024-02-18T00:39:19.720690+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:39:19.720690+0000","last_undegraded":"2024-02-18T00:39:19.720690+0000","last_fullsized":"2024-02-18T00:39:19.720690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:39:09.538136+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.538136+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":451,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726299+0000","last_change":"2024-02-18T00:37:41.747387+0000","last_active":"2024-02-18T00:39:19.726299+0000","last_peered":"2024-02-18T00:39:19.726299+0000","last_clean":"2024-02-18T00:39:19.726299+0000","last_became_active":"2024-02-18T00:33:48.712598+0000","last_became_peered":"2024-02-18T00:33:48.712598+0000","last_unstale":"2024-02-18T00:39:19.726299+0000","last_undegraded":"2024-02-18T00:39:19.726299+0000","last_fullsized":"2024-02-18T00:39:19.726299+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.747293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.747293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":553,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711232+0000","last_change":"2024-02-18T00:37:43.161256+0000","last_active":"2024-02-18T00:39:19.711232+0000","last_peered":"2024-02-18T00:39:19.711232+0000","last_clean":"2024-02-18T00:39:19.711232+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:39:19.711232+0000","last_undegraded":"2024-02-18T00:39:19.711232+0000","last_fullsized":"2024-02-18T00:39:19.711232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:43.161186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.161186+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720551+0000","last_change":"2024-02-18T00:38:57.423470+0000","last_active":"2024-02-18T00:39:19.720551+0000","last_peered":"2024-02-18T00:39:19.720551+0000","last_clean":"2024-02-18T00:39:19.720551+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:39:19.720551+0000","last_undegraded":"2024-02-18T00:39:19.720551+0000","last_fullsized":"2024-02-18T00:39:19.720551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:57.423405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.423405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":368,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709766+0000","last_change":"2024-02-18T00:38:26.149235+0000","last_active":"2024-02-18T00:39:19.709766+0000","last_peered":"2024-02-18T00:39:19.709766+0000","last_clean":"2024-02-18T00:39:19.709766+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:39:19.709766+0000","last_undegraded":"2024-02-18T00:39:19.709766+0000","last_fullsized":"2024-02-18T00:39:19.709766+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:38:26.149169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"64'17","reported_seq":488,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711079+0000","last_change":"2024-02-18T00:39:03.341590+0000","last_active":"2024-02-18T00:39:19.711079+0000","last_peered":"2024-02-18T00:39:19.711079+0000","last_clean":"2024-02-18T00:39:19.711079+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:39:19.711079+0000","last_undegraded":"2024-02-18T00:39:19.711079+0000","last_fullsized":"2024-02-18T00:39:19.711079+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:39:03.341502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.341502+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":398,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.712411+0000","last_change":"2024-02-18T00:36:57.321994+0000","last_active":"2024-02-18T00:39:19.712411+0000","last_peered":"2024-02-18T00:39:19.712411+0000","last_clean":"2024-02-18T00:39:19.712411+0000","last_became_active":"2024-02-18T00:33:48.736513+0000","last_became_peered":"2024-02-18T00:33:48.736513+0000","last_unstale":"2024-02-18T00:39:19.712411+0000","last_undegraded":"2024-02-18T00:39:19.712411+0000","last_fullsized":"2024-02-18T00:39:19.712411+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.321930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.321930+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":419,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720937+0000","last_change":"2024-02-18T00:37:48.527852+0000","last_active":"2024-02-18T00:39:19.720937+0000","last_peered":"2024-02-18T00:39:19.720937+0000","last_clean":"2024-02-18T00:39:19.720937+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:39:19.720937+0000","last_undegraded":"2024-02-18T00:39:19.720937+0000","last_fullsized":"2024-02-18T00:39:19.720937+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:48.527779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.527779+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720216+0000","last_change":"2024-02-18T00:37:30.849071+0000","last_active":"2024-02-18T00:39:19.720216+0000","last_peered":"2024-02-18T00:39:19.720216+0000","last_clean":"2024-02-18T00:39:19.720216+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:39:19.720216+0000","last_undegraded":"2024-02-18T00:39:19.720216+0000","last_fullsized":"2024-02-18T00:39:19.720216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:30.848966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:30.848966+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":368,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710618+0000","last_change":"2024-02-18T00:37:44.200088+0000","last_active":"2024-02-18T00:39:19.710618+0000","last_peered":"2024-02-18T00:39:19.710618+0000","last_clean":"2024-02-18T00:39:19.710618+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:39:19.710618+0000","last_undegraded":"2024-02-18T00:39:19.710618+0000","last_fullsized":"2024-02-18T00:39:19.710618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.199947+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":549,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719754+0000","last_change":"2024-02-18T00:36:53.910404+0000","last_active":"2024-02-18T00:39:19.719754+0000","last_peered":"2024-02-18T00:39:19.719754+0000","last_clean":"2024-02-18T00:39:19.719754+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:39:19.719754+0000","last_undegraded":"2024-02-18T00:39:19.719754+0000","last_fullsized":"2024-02-18T00:39:19.719754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:36:53.910333+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"64'20","reported_seq":467,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709733+0000","last_change":"2024-02-18T00:39:11.324891+0000","last_active":"2024-02-18T00:39:19.709733+0000","last_peered":"2024-02-18T00:39:19.709733+0000","last_clean":"2024-02-18T00:39:19.709733+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:39:19.709733+0000","last_undegraded":"2024-02-18T00:39:19.709733+0000","last_fullsized":"2024-02-18T00:39:19.709733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:39:11.324814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.324814+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":448,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717375+0000","last_change":"2024-02-18T00:38:24.645826+0000","last_active":"2024-02-18T00:39:19.717375+0000","last_peered":"2024-02-18T00:39:19.717375+0000","last_clean":"2024-02-18T00:39:19.717375+0000","last_became_active":"2024-02-18T00:33:48.671673+0000","last_became_peered":"2024-02-18T00:33:48.671673+0000","last_unstale":"2024-02-18T00:39:19.717375+0000","last_undegraded":"2024-02-18T00:39:19.717375+0000","last_fullsized":"2024-02-18T00:39:19.717375+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.645751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.645751+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":325,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716844+0000","last_change":"2024-02-18T00:37:27.733729+0000","last_active":"2024-02-18T00:39:19.716844+0000","last_peered":"2024-02-18T00:39:19.716844+0000","last_clean":"2024-02-18T00:39:19.716844+0000","last_became_active":"2024-02-18T00:33:48.671981+0000","last_became_peered":"2024-02-18T00:33:48.671981+0000","last_unstale":"2024-02-18T00:39:19.716844+0000","last_undegraded":"2024-02-18T00:39:19.716844+0000","last_fullsized":"2024-02-18T00:39:19.716844+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.733609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.733609+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":363,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720207+0000","last_change":"2024-02-18T00:38:23.612159+0000","last_active":"2024-02-18T00:39:19.720207+0000","last_peered":"2024-02-18T00:39:19.720207+0000","last_clean":"2024-02-18T00:39:19.720207+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:39:19.720207+0000","last_undegraded":"2024-02-18T00:39:19.720207+0000","last_fullsized":"2024-02-18T00:39:19.720207+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:23.612082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:23.612082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":433,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710021+0000","last_change":"2024-02-18T00:36:45.308400+0000","last_active":"2024-02-18T00:39:19.710021+0000","last_peered":"2024-02-18T00:39:19.710021+0000","last_clean":"2024-02-18T00:39:19.710021+0000","last_became_active":"2024-02-18T00:33:48.741051+0000","last_became_peered":"2024-02-18T00:33:48.741051+0000","last_unstale":"2024-02-18T00:39:19.710021+0000","last_undegraded":"2024-02-18T00:39:19.710021+0000","last_fullsized":"2024-02-18T00:39:19.710021+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.308300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.308300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"94'40","reported_seq":524,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719662+0000","last_change":"2024-02-18T00:37:35.841167+0000","last_active":"2024-02-18T00:39:19.719662+0000","last_peered":"2024-02-18T00:39:19.719662+0000","last_clean":"2024-02-18T00:39:19.719662+0000","last_became_active":"2024-02-18T00:33:48.734649+0000","last_became_peered":"2024-02-18T00:33:48.734649+0000","last_unstale":"2024-02-18T00:39:19.719662+0000","last_undegraded":"2024-02-18T00:39:19.719662+0000","last_fullsized":"2024-02-18T00:39:19.719662+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'40","last_scrub_stamp":"2024-02-18T00:37:35.841050+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.841050+0000","log_size":40,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":442,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.727325+0000","last_change":"2024-02-18T00:37:32.766303+0000","last_active":"2024-02-18T00:39:19.727325+0000","last_peered":"2024-02-18T00:39:19.727325+0000","last_clean":"2024-02-18T00:39:19.727325+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:39:19.727325+0000","last_undegraded":"2024-02-18T00:39:19.727325+0000","last_fullsized":"2024-02-18T00:39:19.727325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.766203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.766203+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710371+0000","last_change":"2024-02-18T00:37:33.117248+0000","last_active":"2024-02-18T00:39:19.710371+0000","last_peered":"2024-02-18T00:39:19.710371+0000","last_clean":"2024-02-18T00:39:19.710371+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:39:19.710371+0000","last_undegraded":"2024-02-18T00:39:19.710371+0000","last_fullsized":"2024-02-18T00:39:19.710371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:33.117173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:33.117173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":363,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721234+0000","last_change":"2024-02-18T00:38:28.807296+0000","last_active":"2024-02-18T00:39:19.721234+0000","last_peered":"2024-02-18T00:39:19.721234+0000","last_clean":"2024-02-18T00:39:19.721234+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:39:19.721234+0000","last_undegraded":"2024-02-18T00:39:19.721234+0000","last_fullsized":"2024-02-18T00:39:19.721234+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:28.807229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.807229+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":421,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716320+0000","last_change":"2024-02-18T00:37:31.672777+0000","last_active":"2024-02-18T00:39:19.716320+0000","last_peered":"2024-02-18T00:39:19.716320+0000","last_clean":"2024-02-18T00:39:19.716320+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:39:19.716320+0000","last_undegraded":"2024-02-18T00:39:19.716320+0000","last_fullsized":"2024-02-18T00:39:19.716320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:31.672699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:31.672699+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"64'25","reported_seq":504,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710893+0000","last_change":"2024-02-18T00:37:37.281742+0000","last_active":"2024-02-18T00:39:19.710893+0000","last_peered":"2024-02-18T00:39:19.710893+0000","last_clean":"2024-02-18T00:39:19.710893+0000","last_became_active":"2024-02-18T00:33:48.749086+0000","last_became_peered":"2024-02-18T00:33:48.749086+0000","last_unstale":"2024-02-18T00:39:19.710893+0000","last_undegraded":"2024-02-18T00:39:19.710893+0000","last_fullsized":"2024-02-18T00:39:19.710893+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'25","last_scrub_stamp":"2024-02-18T00:37:37.281674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.281674+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":335,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710092+0000","last_change":"2024-02-18T00:37:01.250860+0000","last_active":"2024-02-18T00:39:19.710092+0000","last_peered":"2024-02-18T00:39:19.710092+0000","last_clean":"2024-02-18T00:39:19.710092+0000","last_became_active":"2024-02-18T00:33:48.733877+0000","last_became_peered":"2024-02-18T00:33:48.733877+0000","last_unstale":"2024-02-18T00:39:19.710092+0000","last_undegraded":"2024-02-18T00:39:19.710092+0000","last_fullsized":"2024-02-18T00:39:19.710092+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:37:01.250771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.250771+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":427,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726345+0000","last_change":"2024-02-18T00:39:02.598810+0000","last_active":"2024-02-18T00:39:19.726345+0000","last_peered":"2024-02-18T00:39:19.726345+0000","last_clean":"2024-02-18T00:39:19.726345+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:39:19.726345+0000","last_undegraded":"2024-02-18T00:39:19.726345+0000","last_fullsized":"2024-02-18T00:39:19.726345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.598741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.598741+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":363,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720472+0000","last_change":"2024-02-18T00:38:57.682162+0000","last_active":"2024-02-18T00:39:19.720472+0000","last_peered":"2024-02-18T00:39:19.720472+0000","last_clean":"2024-02-18T00:39:19.720472+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:39:19.720472+0000","last_undegraded":"2024-02-18T00:39:19.720472+0000","last_fullsized":"2024-02-18T00:39:19.720472+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:57.682091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.682091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":419,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721033+0000","last_change":"2024-02-18T00:37:47.574086+0000","last_active":"2024-02-18T00:39:19.721033+0000","last_peered":"2024-02-18T00:39:19.721033+0000","last_clean":"2024-02-18T00:39:19.721033+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:39:19.721033+0000","last_undegraded":"2024-02-18T00:39:19.721033+0000","last_fullsized":"2024-02-18T00:39:19.721033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.574003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.574003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"64'12","reported_seq":442,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711914+0000","last_change":"2024-02-18T00:37:35.212798+0000","last_active":"2024-02-18T00:39:19.711914+0000","last_peered":"2024-02-18T00:39:19.711914+0000","last_clean":"2024-02-18T00:39:19.711914+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:39:19.711914+0000","last_undegraded":"2024-02-18T00:39:19.711914+0000","last_fullsized":"2024-02-18T00:39:19.711914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:37:35.212727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:35.212727+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":404,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726390+0000","last_change":"2024-02-18T00:38:16.494876+0000","last_active":"2024-02-18T00:39:19.726390+0000","last_peered":"2024-02-18T00:39:19.726390+0000","last_clean":"2024-02-18T00:39:19.726390+0000","last_became_active":"2024-02-18T00:33:48.734878+0000","last_became_peered":"2024-02-18T00:33:48.734878+0000","last_unstale":"2024-02-18T00:39:19.726390+0000","last_undegraded":"2024-02-18T00:39:19.726390+0000","last_fullsized":"2024-02-18T00:39:19.726390+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:16.494804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:16.494804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":425,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710984+0000","last_change":"2024-02-18T00:37:26.204990+0000","last_active":"2024-02-18T00:39:19.710984+0000","last_peered":"2024-02-18T00:39:19.710984+0000","last_clean":"2024-02-18T00:39:19.710984+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:39:19.710984+0000","last_undegraded":"2024-02-18T00:39:19.710984+0000","last_fullsized":"2024-02-18T00:39:19.710984+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:26.204918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:37:26.204918+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726613+0000","last_change":"2024-02-18T00:37:25.868179+0000","last_active":"2024-02-18T00:39:19.726613+0000","last_peered":"2024-02-18T00:39:19.726613+0000","last_clean":"2024-02-18T00:39:19.726613+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:39:19.726613+0000","last_undegraded":"2024-02-18T00:39:19.726613+0000","last_fullsized":"2024-02-18T00:39:19.726613+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:25.868065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:25.868065+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":376,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.712578+0000","last_change":"2024-02-18T00:38:19.635886+0000","last_active":"2024-02-18T00:39:19.712578+0000","last_peered":"2024-02-18T00:39:19.712578+0000","last_clean":"2024-02-18T00:39:19.712578+0000","last_became_active":"2024-02-18T00:33:48.730916+0000","last_became_peered":"2024-02-18T00:33:48.730916+0000","last_unstale":"2024-02-18T00:39:19.712578+0000","last_undegraded":"2024-02-18T00:39:19.712578+0000","last_fullsized":"2024-02-18T00:39:19.712578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:19.635822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:19.635822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":328,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711105+0000","last_change":"2024-02-18T00:37:32.401689+0000","last_active":"2024-02-18T00:39:19.711105+0000","last_peered":"2024-02-18T00:39:19.711105+0000","last_clean":"2024-02-18T00:39:19.711105+0000","last_became_active":"2024-02-18T00:33:48.738424+0000","last_became_peered":"2024-02-18T00:33:48.738424+0000","last_unstale":"2024-02-18T00:39:19.711105+0000","last_undegraded":"2024-02-18T00:39:19.711105+0000","last_fullsized":"2024-02-18T00:39:19.711105+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:32.401618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:37:32.401618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.5","version":"64'14","reported_seq":475,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717031+0000","last_change":"2024-02-18T00:36:50.759133+0000","last_active":"2024-02-18T00:39:19.717031+0000","last_peered":"2024-02-18T00:39:19.717031+0000","last_clean":"2024-02-18T00:39:19.717031+0000","last_became_active":"2024-02-18T00:33:48.732763+0000","last_became_peered":"2024-02-18T00:33:48.732763+0000","last_unstale":"2024-02-18T00:39:19.717031+0000","last_undegraded":"2024-02-18T00:39:19.717031+0000","last_fullsized":"2024-02-18T00:39:19.717031+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'14","last_scrub_stamp":"2024-02-18T00:36:50.759019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.759019+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":361,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.712491+0000","last_change":"2024-02-18T00:37:42.354830+0000","last_active":"2024-02-18T00:39:19.712491+0000","last_peered":"2024-02-18T00:39:19.712491+0000","last_clean":"2024-02-18T00:39:19.712491+0000","last_became_active":"2024-02-18T00:33:48.730471+0000","last_became_peered":"2024-02-18T00:33:48.730471+0000","last_unstale":"2024-02-18T00:39:19.712491+0000","last_undegraded":"2024-02-18T00:39:19.712491+0000","last_fullsized":"2024-02-18T00:39:19.712491+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.354758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.354758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":441,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719769+0000","last_change":"2024-02-18T00:37:00.820469+0000","last_active":"2024-02-18T00:39:19.719769+0000","last_peered":"2024-02-18T00:39:19.719769+0000","last_clean":"2024-02-18T00:39:19.719769+0000","last_became_active":"2024-02-18T00:33:48.670349+0000","last_became_peered":"2024-02-18T00:33:48.670349+0000","last_unstale":"2024-02-18T00:39:19.719769+0000","last_undegraded":"2024-02-18T00:39:19.719769+0000","last_fullsized":"2024-02-18T00:39:19.719769+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.820283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.820283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719563+0000","last_change":"2024-02-18T00:39:02.673159+0000","last_active":"2024-02-18T00:39:19.719563+0000","last_peered":"2024-02-18T00:39:19.719563+0000","last_clean":"2024-02-18T00:39:19.719563+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:39:19.719563+0000","last_undegraded":"2024-02-18T00:39:19.719563+0000","last_fullsized":"2024-02-18T00:39:19.719563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.673088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.673088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721023+0000","last_change":"2024-02-18T00:36:42.828611+0000","last_active":"2024-02-18T00:39:19.721023+0000","last_peered":"2024-02-18T00:39:19.721023+0000","last_clean":"2024-02-18T00:39:19.721023+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:39:19.721023+0000","last_undegraded":"2024-02-18T00:39:19.721023+0000","last_fullsized":"2024-02-18T00:39:19.721023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.828542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.828542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":367,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709872+0000","last_change":"2024-02-18T00:38:25.649197+0000","last_active":"2024-02-18T00:39:19.709872+0000","last_peered":"2024-02-18T00:39:19.709872+0000","last_clean":"2024-02-18T00:39:19.709872+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:39:19.709872+0000","last_undegraded":"2024-02-18T00:39:19.709872+0000","last_fullsized":"2024-02-18T00:39:19.709872+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.649119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.649119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":433,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711569+0000","last_change":"2024-02-18T00:36:57.633211+0000","last_active":"2024-02-18T00:39:19.711569+0000","last_peered":"2024-02-18T00:39:19.711569+0000","last_clean":"2024-02-18T00:39:19.711569+0000","last_became_active":"2024-02-18T00:33:48.633849+0000","last_became_peered":"2024-02-18T00:33:48.633849+0000","last_unstale":"2024-02-18T00:39:19.711569+0000","last_undegraded":"2024-02-18T00:39:19.711569+0000","last_fullsized":"2024-02-18T00:39:19.711569+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.633128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.633128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"64'19","reported_seq":473,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.727142+0000","last_change":"2024-02-18T00:38:28.669127+0000","last_active":"2024-02-18T00:39:19.727142+0000","last_peered":"2024-02-18T00:39:19.727142+0000","last_clean":"2024-02-18T00:39:19.727142+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:39:19.727142+0000","last_undegraded":"2024-02-18T00:39:19.727142+0000","last_fullsized":"2024-02-18T00:39:19.727142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'19","last_scrub_stamp":"2024-02-18T00:38:28.669051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.669051+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":437,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711667+0000","last_change":"2024-02-18T00:36:36.500976+0000","last_active":"2024-02-18T00:39:19.711667+0000","last_peered":"2024-02-18T00:39:19.711667+0000","last_clean":"2024-02-18T00:39:19.711667+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:39:19.711667+0000","last_undegraded":"2024-02-18T00:39:19.711667+0000","last_fullsized":"2024-02-18T00:39:19.711667+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:36.500906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:36.500906+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1032,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.986186+0000","last_change":"2024-02-18T00:37:37.774470+0000","last_active":"2024-02-18T00:39:19.986186+0000","last_peered":"2024-02-18T00:39:19.986186+0000","last_clean":"2024-02-18T00:39:19.986186+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:39:19.986186+0000","last_undegraded":"2024-02-18T00:39:19.986186+0000","last_fullsized":"2024-02-18T00:39:19.986186+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:37:37.774404+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.774404+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":591,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":362,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717249+0000","last_change":"2024-02-18T00:36:42.643527+0000","last_active":"2024-02-18T00:39:19.717249+0000","last_peered":"2024-02-18T00:39:19.717249+0000","last_clean":"2024-02-18T00:39:19.717249+0000","last_became_active":"2024-02-18T00:33:48.682232+0000","last_became_peered":"2024-02-18T00:33:48.682232+0000","last_unstale":"2024-02-18T00:39:19.717249+0000","last_undegraded":"2024-02-18T00:39:19.717249+0000","last_fullsized":"2024-02-18T00:39:19.717249+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:36:42.643433+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.643433+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":377,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710130+0000","last_change":"2024-02-18T00:38:29.090754+0000","last_active":"2024-02-18T00:39:19.710130+0000","last_peered":"2024-02-18T00:39:19.710130+0000","last_clean":"2024-02-18T00:39:19.710130+0000","last_became_active":"2024-02-18T00:33:48.669940+0000","last_became_peered":"2024-02-18T00:33:48.669940+0000","last_unstale":"2024-02-18T00:39:19.710130+0000","last_undegraded":"2024-02-18T00:39:19.710130+0000","last_fullsized":"2024-02-18T00:39:19.710130+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_clean_scrub_stamp":"2024-02-18T00:38:29.090653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710191+0000","last_change":"2024-02-18T00:36:56.302683+0000","last_active":"2024-02-18T00:39:19.710191+0000","last_peered":"2024-02-18T00:39:19.710191+0000","last_clean":"2024-02-18T00:39:19.710191+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:39:19.710191+0000","last_undegraded":"2024-02-18T00:39:19.710191+0000","last_fullsized":"2024-02-18T00:39:19.710191+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.302615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.302615+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"264'35","reported_seq":270,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719883+0000","last_change":"2024-02-18T00:36:41.801219+0000","last_active":"2024-02-18T00:39:19.719883+0000","last_peered":"2024-02-18T00:39:19.719883+0000","last_clean":"2024-02-18T00:39:19.719883+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:39:19.719883+0000","last_undegraded":"2024-02-18T00:39:19.719883+0000","last_fullsized":"2024-02-18T00:39:19.719883+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'35","last_scrub_stamp":"2024-02-18T00:36:41.801144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:41.801144+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":432,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719924+0000","last_change":"2024-02-18T00:36:57.791451+0000","last_active":"2024-02-18T00:39:19.719924+0000","last_peered":"2024-02-18T00:39:19.719924+0000","last_clean":"2024-02-18T00:39:19.719924+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:39:19.719924+0000","last_undegraded":"2024-02-18T00:39:19.719924+0000","last_fullsized":"2024-02-18T00:39:19.719924+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:57.791347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:36:57.791347+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":426,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720620+0000","last_change":"2024-02-18T00:39:14.571036+0000","last_active":"2024-02-18T00:39:19.720620+0000","last_peered":"2024-02-18T00:39:19.720620+0000","last_clean":"2024-02-18T00:39:19.720620+0000","last_became_active":"2024-02-18T00:33:48.688100+0000","last_became_peered":"2024-02-18T00:33:48.688100+0000","last_unstale":"2024-02-18T00:39:19.720620+0000","last_undegraded":"2024-02-18T00:39:19.720620+0000","last_fullsized":"2024-02-18T00:39:19.720620+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.570919+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":367,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710477+0000","last_change":"2024-02-18T00:39:02.635218+0000","last_active":"2024-02-18T00:39:19.710477+0000","last_peered":"2024-02-18T00:39:19.710477+0000","last_clean":"2024-02-18T00:39:19.710477+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:39:19.710477+0000","last_undegraded":"2024-02-18T00:39:19.710477+0000","last_fullsized":"2024-02-18T00:39:19.710477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.635104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.635104+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":551,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711170+0000","last_change":"2024-02-18T00:37:29.141966+0000","last_active":"2024-02-18T00:39:19.711170+0000","last_peered":"2024-02-18T00:39:19.711170+0000","last_clean":"2024-02-18T00:39:19.711170+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:39:19.711170+0000","last_undegraded":"2024-02-18T00:39:19.711170+0000","last_fullsized":"2024-02-18T00:39:19.711170+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:29.141901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.141901+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"102'22","reported_seq":426,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710620+0000","last_change":"2024-02-18T00:37:43.309558+0000","last_active":"2024-02-18T00:39:19.710620+0000","last_peered":"2024-02-18T00:39:19.710620+0000","last_clean":"2024-02-18T00:39:19.710620+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:39:19.710620+0000","last_undegraded":"2024-02-18T00:39:19.710620+0000","last_fullsized":"2024-02-18T00:39:19.710620+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:37:43.309488+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.309488+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":456,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726413+0000","last_change":"2024-02-18T00:38:31.657857+0000","last_active":"2024-02-18T00:39:19.726413+0000","last_peered":"2024-02-18T00:39:19.726413+0000","last_clean":"2024-02-18T00:39:19.726413+0000","last_became_active":"2024-02-18T00:33:48.759033+0000","last_became_peered":"2024-02-18T00:33:48.759033+0000","last_unstale":"2024-02-18T00:39:19.726413+0000","last_undegraded":"2024-02-18T00:39:19.726413+0000","last_fullsized":"2024-02-18T00:39:19.726413+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:31.657778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:31.657778+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":415,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.712269+0000","last_change":"2024-02-18T00:36:46.369602+0000","last_active":"2024-02-18T00:39:19.712269+0000","last_peered":"2024-02-18T00:39:19.712269+0000","last_clean":"2024-02-18T00:39:19.712269+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:39:19.712269+0000","last_undegraded":"2024-02-18T00:39:19.712269+0000","last_fullsized":"2024-02-18T00:39:19.712269+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:46.369503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:46.369503+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":368,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710608+0000","last_change":"2024-02-18T00:38:24.128263+0000","last_active":"2024-02-18T00:39:19.710608+0000","last_peered":"2024-02-18T00:39:19.710608+0000","last_clean":"2024-02-18T00:39:19.710608+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:39:19.710608+0000","last_undegraded":"2024-02-18T00:39:19.710608+0000","last_fullsized":"2024-02-18T00:39:19.710608+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.128170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.128170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":424,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710918+0000","last_change":"2024-02-18T00:37:44.335458+0000","last_active":"2024-02-18T00:39:19.710918+0000","last_peered":"2024-02-18T00:39:19.710918+0000","last_clean":"2024-02-18T00:39:19.710918+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:39:19.710918+0000","last_undegraded":"2024-02-18T00:39:19.710918+0000","last_fullsized":"2024-02-18T00:39:19.710918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"64'24","reported_seq":479,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709822+0000","last_change":"2024-02-18T00:37:39.126531+0000","last_active":"2024-02-18T00:39:19.709822+0000","last_peered":"2024-02-18T00:39:19.709822+0000","last_clean":"2024-02-18T00:39:19.709822+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:39:19.709822+0000","last_undegraded":"2024-02-18T00:39:19.709822+0000","last_fullsized":"2024-02-18T00:39:19.709822+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'24","last_scrub_stamp":"2024-02-18T00:37:39.126398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.126398+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":461,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711178+0000","last_change":"2024-02-18T00:37:45.186258+0000","last_active":"2024-02-18T00:39:19.711178+0000","last_peered":"2024-02-18T00:39:19.711178+0000","last_clean":"2024-02-18T00:39:19.711178+0000","last_became_active":"2024-02-18T00:33:48.733942+0000","last_became_peered":"2024-02-18T00:33:48.733942+0000","last_unstale":"2024-02-18T00:39:19.711178+0000","last_undegraded":"2024-02-18T00:39:19.711178+0000","last_fullsized":"2024-02-18T00:39:19.711178+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:37:45.186143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.186143+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":426,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720289+0000","last_change":"2024-02-18T00:37:00.804782+0000","last_active":"2024-02-18T00:39:19.720289+0000","last_peered":"2024-02-18T00:39:19.720289+0000","last_clean":"2024-02-18T00:39:19.720289+0000","last_became_active":"2024-02-18T00:33:48.710872+0000","last_became_peered":"2024-02-18T00:33:48.710872+0000","last_unstale":"2024-02-18T00:39:19.720289+0000","last_undegraded":"2024-02-18T00:39:19.720289+0000","last_fullsized":"2024-02-18T00:39:19.720289+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:37:00.804656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":368,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709802+0000","last_change":"2024-02-18T00:38:18.292304+0000","last_active":"2024-02-18T00:39:19.709802+0000","last_peered":"2024-02-18T00:39:19.709802+0000","last_clean":"2024-02-18T00:39:19.709802+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:39:19.709802+0000","last_undegraded":"2024-02-18T00:39:19.709802+0000","last_fullsized":"2024-02-18T00:39:19.709802+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:18.292215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:18.292215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"64'21","reported_seq":493,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711962+0000","last_change":"2024-02-18T00:37:46.472303+0000","last_active":"2024-02-18T00:39:19.711962+0000","last_peered":"2024-02-18T00:39:19.711962+0000","last_clean":"2024-02-18T00:39:19.711962+0000","last_became_active":"2024-02-18T00:33:48.749588+0000","last_became_peered":"2024-02-18T00:33:48.749588+0000","last_unstale":"2024-02-18T00:39:19.711962+0000","last_undegraded":"2024-02-18T00:39:19.711962+0000","last_fullsized":"2024-02-18T00:39:19.711962+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'21","last_scrub_stamp":"2024-02-18T00:37:46.472227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.472227+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":428,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710733+0000","last_change":"2024-02-18T00:37:27.357689+0000","last_active":"2024-02-18T00:39:19.710733+0000","last_peered":"2024-02-18T00:39:19.710733+0000","last_clean":"2024-02-18T00:39:19.710733+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:39:19.710733+0000","last_undegraded":"2024-02-18T00:39:19.710733+0000","last_fullsized":"2024-02-18T00:39:19.710733+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.357610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.357610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":428,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721422+0000","last_change":"2024-02-18T00:36:55.861115+0000","last_active":"2024-02-18T00:39:19.721422+0000","last_peered":"2024-02-18T00:39:19.721422+0000","last_clean":"2024-02-18T00:39:19.721422+0000","last_became_active":"2024-02-18T00:33:48.736974+0000","last_became_peered":"2024-02-18T00:33:48.736974+0000","last_unstale":"2024-02-18T00:39:19.721422+0000","last_undegraded":"2024-02-18T00:39:19.721422+0000","last_fullsized":"2024-02-18T00:39:19.721422+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:55.861044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:55.861044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":433,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711014+0000","last_change":"2024-02-18T00:38:18.211396+0000","last_active":"2024-02-18T00:39:19.711014+0000","last_peered":"2024-02-18T00:39:19.711014+0000","last_clean":"2024-02-18T00:39:19.711014+0000","last_became_active":"2024-02-18T00:33:48.738018+0000","last_became_peered":"2024-02-18T00:33:48.738018+0000","last_unstale":"2024-02-18T00:39:19.711014+0000","last_undegraded":"2024-02-18T00:39:19.711014+0000","last_fullsized":"2024-02-18T00:39:19.711014+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:18.211326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:38:18.211326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":367,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709960+0000","last_change":"2024-02-18T00:39:09.589674+0000","last_active":"2024-02-18T00:39:19.709960+0000","last_peered":"2024-02-18T00:39:19.709960+0000","last_clean":"2024-02-18T00:39:19.709960+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:39:19.709960+0000","last_undegraded":"2024-02-18T00:39:19.709960+0000","last_fullsized":"2024-02-18T00:39:19.709960+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:09.589601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.589601+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"64'12","reported_seq":488,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721316+0000","last_change":"2024-02-18T00:39:07.653709+0000","last_active":"2024-02-18T00:39:19.721316+0000","last_peered":"2024-02-18T00:39:19.721316+0000","last_clean":"2024-02-18T00:39:19.721316+0000","last_became_active":"2024-02-18T00:33:48.710812+0000","last_became_peered":"2024-02-18T00:33:48.710812+0000","last_unstale":"2024-02-18T00:39:19.721316+0000","last_undegraded":"2024-02-18T00:39:19.721316+0000","last_fullsized":"2024-02-18T00:39:19.721316+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:39:07.653633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.653633+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":430,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716189+0000","last_change":"2024-02-18T00:38:13.745638+0000","last_active":"2024-02-18T00:39:19.716189+0000","last_peered":"2024-02-18T00:39:19.716189+0000","last_clean":"2024-02-18T00:39:19.716189+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:39:19.716189+0000","last_undegraded":"2024-02-18T00:39:19.716189+0000","last_fullsized":"2024-02-18T00:39:19.716189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:13.745551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:13.745551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":419,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720843+0000","last_change":"2024-02-18T00:37:38.687198+0000","last_active":"2024-02-18T00:39:19.720843+0000","last_peered":"2024-02-18T00:39:19.720843+0000","last_clean":"2024-02-18T00:39:19.720843+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:39:19.720843+0000","last_undegraded":"2024-02-18T00:39:19.720843+0000","last_fullsized":"2024-02-18T00:39:19.720843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:38.687129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.687129+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":330,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717346+0000","last_change":"2024-02-18T00:39:00.786552+0000","last_active":"2024-02-18T00:39:19.717346+0000","last_peered":"2024-02-18T00:39:19.717346+0000","last_clean":"2024-02-18T00:39:19.717346+0000","last_became_active":"2024-02-18T00:33:48.635878+0000","last_became_peered":"2024-02-18T00:33:48.635878+0000","last_unstale":"2024-02-18T00:39:19.717346+0000","last_undegraded":"2024-02-18T00:39:19.717346+0000","last_fullsized":"2024-02-18T00:39:19.717346+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:00.786470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:00.786470+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":377,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711879+0000","last_change":"2024-02-18T00:38:20.206553+0000","last_active":"2024-02-18T00:39:19.711879+0000","last_peered":"2024-02-18T00:39:19.711879+0000","last_clean":"2024-02-18T00:39:19.711879+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:39:19.711879+0000","last_undegraded":"2024-02-18T00:39:19.711879+0000","last_fullsized":"2024-02-18T00:39:19.711879+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.206476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.206476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"64'22","reported_seq":491,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720416+0000","last_change":"2024-02-18T00:36:58.819567+0000","last_active":"2024-02-18T00:39:19.720416+0000","last_peered":"2024-02-18T00:39:19.720416+0000","last_clean":"2024-02-18T00:39:19.720416+0000","last_became_active":"2024-02-18T00:33:48.714530+0000","last_became_peered":"2024-02-18T00:33:48.714530+0000","last_unstale":"2024-02-18T00:39:19.720416+0000","last_undegraded":"2024-02-18T00:39:19.720416+0000","last_fullsized":"2024-02-18T00:39:19.720416+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'22","last_scrub_stamp":"2024-02-18T00:36:58.819503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.819503+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":441,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719834+0000","last_change":"2024-02-18T00:37:41.637636+0000","last_active":"2024-02-18T00:39:19.719834+0000","last_peered":"2024-02-18T00:39:19.719834+0000","last_clean":"2024-02-18T00:39:19.719834+0000","last_became_active":"2024-02-18T00:33:48.734832+0000","last_became_peered":"2024-02-18T00:33:48.734832+0000","last_unstale":"2024-02-18T00:39:19.719834+0000","last_undegraded":"2024-02-18T00:39:19.719834+0000","last_fullsized":"2024-02-18T00:39:19.719834+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:41.637550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.637550+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"64'8","reported_seq":548,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720519+0000","last_change":"2024-02-18T00:37:27.642761+0000","last_active":"2024-02-18T00:39:19.720519+0000","last_peered":"2024-02-18T00:39:19.720519+0000","last_clean":"2024-02-18T00:39:19.720519+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:39:19.720519+0000","last_undegraded":"2024-02-18T00:39:19.720519+0000","last_fullsized":"2024-02-18T00:39:19.720519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:27.642661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.642661+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":434,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711864+0000","last_change":"2024-02-18T00:39:14.633803+0000","last_active":"2024-02-18T00:39:19.711864+0000","last_peered":"2024-02-18T00:39:19.711864+0000","last_clean":"2024-02-18T00:39:19.711864+0000","last_became_active":"2024-02-18T00:33:48.734989+0000","last_became_peered":"2024-02-18T00:33:48.734989+0000","last_unstale":"2024-02-18T00:39:19.711864+0000","last_undegraded":"2024-02-18T00:39:19.711864+0000","last_fullsized":"2024-02-18T00:39:19.711864+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:14.633705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.633705+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.633705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":374,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.715657+0000","last_change":"2024-02-18T00:37:46.639988+0000","last_active":"2024-02-18T00:39:19.715657+0000","last_peered":"2024-02-18T00:39:19.715657+0000","last_clean":"2024-02-18T00:39:19.715657+0000","last_became_active":"2024-02-18T00:33:48.730309+0000","last_became_peered":"2024-02-18T00:33:48.730309+0000","last_unstale":"2024-02-18T00:39:19.715657+0000","last_undegraded":"2024-02-18T00:39:19.715657+0000","last_fullsized":"2024-02-18T00:39:19.715657+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:46.639901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:46.639901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"64'15","reported_seq":473,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720342+0000","last_change":"2024-02-18T00:38:29.545852+0000","last_active":"2024-02-18T00:39:19.720342+0000","last_peered":"2024-02-18T00:39:19.720342+0000","last_clean":"2024-02-18T00:39:19.720342+0000","last_became_active":"2024-02-18T00:33:48.668364+0000","last_became_peered":"2024-02-18T00:33:48.668364+0000","last_unstale":"2024-02-18T00:39:19.720342+0000","last_undegraded":"2024-02-18T00:39:19.720342+0000","last_fullsized":"2024-02-18T00:39:19.720342+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:38:29.545783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:29.545783+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":426,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710976+0000","last_change":"2024-02-18T00:37:03.239555+0000","last_active":"2024-02-18T00:39:19.710976+0000","last_peered":"2024-02-18T00:39:19.710976+0000","last_clean":"2024-02-18T00:39:19.710976+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:39:19.710976+0000","last_undegraded":"2024-02-18T00:39:19.710976+0000","last_fullsized":"2024-02-18T00:39:19.710976+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.239451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.239451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"341'10","reported_seq":562,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720604+0000","last_change":"2024-02-18T00:36:59.830567+0000","last_active":"2024-02-18T00:39:19.720604+0000","last_peered":"2024-02-18T00:39:19.720604+0000","last_clean":"2024-02-18T00:39:19.720604+0000","last_became_active":"2024-02-18T00:33:48.713355+0000","last_became_peered":"2024-02-18T00:33:48.713355+0000","last_unstale":"2024-02-18T00:39:19.720604+0000","last_undegraded":"2024-02-18T00:39:19.720604+0000","last_fullsized":"2024-02-18T00:39:19.720604+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"341'10","last_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_deep_scrub":"341'10","last_deep_scrub_stamp":"2024-02-18T00:36:59.830497+0000","last_clean_scrub_stamp":"2024-02-18T00:36:59.830497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711287+0000","last_change":"2024-02-18T00:38:20.222342+0000","last_active":"2024-02-18T00:39:19.711287+0000","last_peered":"2024-02-18T00:39:19.711287+0000","last_clean":"2024-02-18T00:39:19.711287+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:39:19.711287+0000","last_undegraded":"2024-02-18T00:39:19.711287+0000","last_fullsized":"2024-02-18T00:39:19.711287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.222269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.222269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":363,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721093+0000","last_change":"2024-02-18T00:37:39.639314+0000","last_active":"2024-02-18T00:39:19.721093+0000","last_peered":"2024-02-18T00:39:19.721093+0000","last_clean":"2024-02-18T00:39:19.721093+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:39:19.721093+0000","last_undegraded":"2024-02-18T00:39:19.721093+0000","last_fullsized":"2024-02-18T00:39:19.721093+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:39.639232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.639232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"64'9","reported_seq":447,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726581+0000","last_change":"2024-02-18T00:38:33.648037+0000","last_active":"2024-02-18T00:39:19.726581+0000","last_peered":"2024-02-18T00:39:19.726581+0000","last_clean":"2024-02-18T00:39:19.726581+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:39:19.726581+0000","last_undegraded":"2024-02-18T00:39:19.726581+0000","last_fullsized":"2024-02-18T00:39:19.726581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'9","last_scrub_stamp":"2024-02-18T00:38:33.647958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.647958+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":430,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717234+0000","last_change":"2024-02-18T00:36:44.617522+0000","last_active":"2024-02-18T00:39:19.717234+0000","last_peered":"2024-02-18T00:39:19.717234+0000","last_clean":"2024-02-18T00:39:19.717234+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:39:19.717234+0000","last_undegraded":"2024-02-18T00:39:19.717234+0000","last_fullsized":"2024-02-18T00:39:19.717234+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:44.617443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:36:44.617443+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":419,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720168+0000","last_change":"2024-02-18T00:36:50.970267+0000","last_active":"2024-02-18T00:39:19.720168+0000","last_peered":"2024-02-18T00:39:19.720168+0000","last_clean":"2024-02-18T00:39:19.720168+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:39:19.720168+0000","last_undegraded":"2024-02-18T00:39:19.720168+0000","last_fullsized":"2024-02-18T00:39:19.720168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:50.970192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:50.970192+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":425,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710372+0000","last_change":"2024-02-18T00:39:01.603631+0000","last_active":"2024-02-18T00:39:19.710372+0000","last_peered":"2024-02-18T00:39:19.710372+0000","last_clean":"2024-02-18T00:39:19.710372+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:39:19.710372+0000","last_undegraded":"2024-02-18T00:39:19.710372+0000","last_fullsized":"2024-02-18T00:39:19.710372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:01.603551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.603551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":362,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711815+0000","last_change":"2024-02-18T00:36:42.355194+0000","last_active":"2024-02-18T00:39:19.711815+0000","last_peered":"2024-02-18T00:39:19.711815+0000","last_clean":"2024-02-18T00:39:19.711815+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:39:19.711815+0000","last_undegraded":"2024-02-18T00:39:19.711815+0000","last_fullsized":"2024-02-18T00:39:19.711815+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:42.355114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:36:42.355114+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"64'11","reported_seq":445,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710251+0000","last_change":"2024-02-18T00:36:52.343624+0000","last_active":"2024-02-18T00:39:19.710251+0000","last_peered":"2024-02-18T00:39:19.710251+0000","last_clean":"2024-02-18T00:39:19.710251+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:39:19.710251+0000","last_undegraded":"2024-02-18T00:39:19.710251+0000","last_fullsized":"2024-02-18T00:39:19.710251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'11","last_scrub_stamp":"2024-02-18T00:36:52.343556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.343556+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":428,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.727588+0000","last_change":"2024-02-18T00:37:27.830300+0000","last_active":"2024-02-18T00:39:19.727588+0000","last_peered":"2024-02-18T00:39:19.727588+0000","last_clean":"2024-02-18T00:39:19.727588+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:39:19.727588+0000","last_undegraded":"2024-02-18T00:39:19.727588+0000","last_fullsized":"2024-02-18T00:39:19.727588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.830225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.830225+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":429,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726555+0000","last_change":"2024-02-18T00:38:24.601383+0000","last_active":"2024-02-18T00:39:19.726555+0000","last_peered":"2024-02-18T00:39:19.726555+0000","last_clean":"2024-02-18T00:39:19.726555+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:39:19.726555+0000","last_undegraded":"2024-02-18T00:39:19.726555+0000","last_fullsized":"2024-02-18T00:39:19.726555+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.601308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.601308+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719326+0000","last_change":"2024-02-18T00:37:37.848598+0000","last_active":"2024-02-18T00:39:19.719326+0000","last_peered":"2024-02-18T00:39:19.719326+0000","last_clean":"2024-02-18T00:39:19.719326+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:39:19.719326+0000","last_undegraded":"2024-02-18T00:39:19.719326+0000","last_fullsized":"2024-02-18T00:39:19.719326+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:37.848524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:37.848524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":365,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717144+0000","last_change":"2024-02-18T00:38:15.658298+0000","last_active":"2024-02-18T00:39:19.717144+0000","last_peered":"2024-02-18T00:39:19.717144+0000","last_clean":"2024-02-18T00:39:19.717144+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:39:19.717144+0000","last_undegraded":"2024-02-18T00:39:19.717144+0000","last_fullsized":"2024-02-18T00:39:19.717144+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:15.658206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:15.658206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"64'4","reported_seq":427,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719482+0000","last_change":"2024-02-18T00:37:41.880926+0000","last_active":"2024-02-18T00:39:19.719482+0000","last_peered":"2024-02-18T00:39:19.719482+0000","last_clean":"2024-02-18T00:39:19.719482+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:39:19.719482+0000","last_undegraded":"2024-02-18T00:39:19.719482+0000","last_fullsized":"2024-02-18T00:39:19.719482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'4","last_scrub_stamp":"2024-02-18T00:37:41.880855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:41.880855+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":428,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720628+0000","last_change":"2024-02-18T00:38:33.486555+0000","last_active":"2024-02-18T00:39:19.720628+0000","last_peered":"2024-02-18T00:39:19.720628+0000","last_clean":"2024-02-18T00:39:19.720628+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:39:19.720628+0000","last_undegraded":"2024-02-18T00:39:19.720628+0000","last_fullsized":"2024-02-18T00:39:19.720628+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:33.486426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.486426+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":435,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716141+0000","last_change":"2024-02-18T00:39:09.867335+0000","last_active":"2024-02-18T00:39:19.716141+0000","last_peered":"2024-02-18T00:39:19.716141+0000","last_clean":"2024-02-18T00:39:19.716141+0000","last_became_active":"2024-02-18T00:33:48.729150+0000","last_became_peered":"2024-02-18T00:33:48.729150+0000","last_unstale":"2024-02-18T00:39:19.716141+0000","last_undegraded":"2024-02-18T00:39:19.716141+0000","last_fullsized":"2024-02-18T00:39:19.716141+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:09.867259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.867259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":328,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710519+0000","last_change":"2024-02-18T00:37:29.374778+0000","last_active":"2024-02-18T00:39:19.710519+0000","last_peered":"2024-02-18T00:39:19.710519+0000","last_clean":"2024-02-18T00:39:19.710519+0000","last_became_active":"2024-02-18T00:33:48.675558+0000","last_became_peered":"2024-02-18T00:33:48.675558+0000","last_unstale":"2024-02-18T00:39:19.710519+0000","last_undegraded":"2024-02-18T00:39:19.710519+0000","last_fullsized":"2024-02-18T00:39:19.710519+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:29.374712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.374712+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":372,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720359+0000","last_change":"2024-02-18T00:39:01.514281+0000","last_active":"2024-02-18T00:39:19.720359+0000","last_peered":"2024-02-18T00:39:19.720359+0000","last_clean":"2024-02-18T00:39:19.720359+0000","last_became_active":"2024-02-18T00:33:48.714438+0000","last_became_peered":"2024-02-18T00:33:48.714438+0000","last_unstale":"2024-02-18T00:39:19.720359+0000","last_undegraded":"2024-02-18T00:39:19.720359+0000","last_fullsized":"2024-02-18T00:39:19.720359+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:01.514191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.514191+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"64'17","reported_seq":478,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719704+0000","last_change":"2024-02-18T00:37:45.838598+0000","last_active":"2024-02-18T00:39:19.719704+0000","last_peered":"2024-02-18T00:39:19.719704+0000","last_clean":"2024-02-18T00:39:19.719704+0000","last_became_active":"2024-02-18T00:33:48.666869+0000","last_became_peered":"2024-02-18T00:33:48.666869+0000","last_unstale":"2024-02-18T00:39:19.719704+0000","last_undegraded":"2024-02-18T00:39:19.719704+0000","last_fullsized":"2024-02-18T00:39:19.719704+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:37:45.838523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.838523+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":430,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717329+0000","last_change":"2024-02-18T00:39:06.776476+0000","last_active":"2024-02-18T00:39:19.717329+0000","last_peered":"2024-02-18T00:39:19.717329+0000","last_clean":"2024-02-18T00:39:19.717329+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:39:19.717329+0000","last_undegraded":"2024-02-18T00:39:19.717329+0000","last_fullsized":"2024-02-18T00:39:19.717329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:06.776398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.776398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":342,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709565+0000","last_change":"2024-02-18T00:39:16.384193+0000","last_active":"2024-02-18T00:39:19.709565+0000","last_peered":"2024-02-18T00:39:19.709565+0000","last_clean":"2024-02-18T00:39:19.709565+0000","last_became_active":"2024-02-18T00:35:42.660798+0000","last_became_peered":"2024-02-18T00:35:42.660798+0000","last_unstale":"2024-02-18T00:39:19.709565+0000","last_undegraded":"2024-02-18T00:39:19.709565+0000","last_fullsized":"2024-02-18T00:39:19.709565+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:16.384118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:16.384118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720290+0000","last_change":"2024-02-18T00:37:44.869935+0000","last_active":"2024-02-18T00:39:19.720290+0000","last_peered":"2024-02-18T00:39:19.720290+0000","last_clean":"2024-02-18T00:39:19.720290+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:39:19.720290+0000","last_undegraded":"2024-02-18T00:39:19.720290+0000","last_fullsized":"2024-02-18T00:39:19.720290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.869862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.869862+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":328,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709703+0000","last_change":"2024-02-18T00:38:19.252878+0000","last_active":"2024-02-18T00:39:19.709703+0000","last_peered":"2024-02-18T00:39:19.709703+0000","last_clean":"2024-02-18T00:39:19.709703+0000","last_became_active":"2024-02-18T00:33:48.679819+0000","last_became_peered":"2024-02-18T00:33:48.679819+0000","last_unstale":"2024-02-18T00:39:19.709703+0000","last_undegraded":"2024-02-18T00:39:19.709703+0000","last_fullsized":"2024-02-18T00:39:19.709703+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:19.252808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:19.252808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"157'22","reported_seq":488,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.712077+0000","last_change":"2024-02-18T00:39:11.643668+0000","last_active":"2024-02-18T00:39:19.712077+0000","last_peered":"2024-02-18T00:39:19.712077+0000","last_clean":"2024-02-18T00:39:19.712077+0000","last_became_active":"2024-02-18T00:33:48.736618+0000","last_became_peered":"2024-02-18T00:33:48.736618+0000","last_unstale":"2024-02-18T00:39:19.712077+0000","last_undegraded":"2024-02-18T00:39:19.712077+0000","last_fullsized":"2024-02-18T00:39:19.712077+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:39:11.643593+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.643593+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":432,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719229+0000","last_change":"2024-02-18T00:39:06.562645+0000","last_active":"2024-02-18T00:39:19.719229+0000","last_peered":"2024-02-18T00:39:19.719229+0000","last_clean":"2024-02-18T00:39:19.719229+0000","last_became_active":"2024-02-18T00:33:48.634508+0000","last_became_peered":"2024-02-18T00:33:48.634508+0000","last_unstale":"2024-02-18T00:39:19.719229+0000","last_undegraded":"2024-02-18T00:39:19.719229+0000","last_fullsized":"2024-02-18T00:39:19.719229+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:06.562567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.562567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"150'9","reported_seq":576,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717430+0000","last_change":"2024-02-18T00:39:17.834219+0000","last_active":"2024-02-18T00:39:19.717430+0000","last_peered":"2024-02-18T00:39:19.717430+0000","last_clean":"2024-02-18T00:39:19.717430+0000","last_became_active":"2024-02-18T00:33:48.682416+0000","last_became_peered":"2024-02-18T00:33:48.682416+0000","last_unstale":"2024-02-18T00:39:19.717430+0000","last_undegraded":"2024-02-18T00:39:19.717430+0000","last_fullsized":"2024-02-18T00:39:19.717430+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:39:17.834141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:17.834141+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":336,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720358+0000","last_change":"2024-02-18T00:36:49.727241+0000","last_active":"2024-02-18T00:39:19.720358+0000","last_peered":"2024-02-18T00:39:19.720358+0000","last_clean":"2024-02-18T00:39:19.720358+0000","last_became_active":"2024-02-18T00:33:48.673538+0000","last_became_peered":"2024-02-18T00:33:48.673538+0000","last_unstale":"2024-02-18T00:39:19.720358+0000","last_undegraded":"2024-02-18T00:39:19.720358+0000","last_fullsized":"2024-02-18T00:39:19.720358+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:36:49.727157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:49.727157+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":323,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719380+0000","last_change":"2024-02-18T00:38:17.776297+0000","last_active":"2024-02-18T00:39:19.719380+0000","last_peered":"2024-02-18T00:39:19.719380+0000","last_clean":"2024-02-18T00:39:19.719380+0000","last_became_active":"2024-02-18T00:33:48.698481+0000","last_became_peered":"2024-02-18T00:33:48.698481+0000","last_unstale":"2024-02-18T00:39:19.719380+0000","last_undegraded":"2024-02-18T00:39:19.719380+0000","last_fullsized":"2024-02-18T00:39:19.719380+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:17.776238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:17.776238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"64'17","reported_seq":462,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717450+0000","last_change":"2024-02-18T00:39:10.827687+0000","last_active":"2024-02-18T00:39:19.717450+0000","last_peered":"2024-02-18T00:39:19.717450+0000","last_clean":"2024-02-18T00:39:19.717450+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:39:19.717450+0000","last_undegraded":"2024-02-18T00:39:19.717450+0000","last_fullsized":"2024-02-18T00:39:19.717450+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:39:10.827603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:10.827603+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":423,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719468+0000","last_change":"2024-02-18T00:36:56.850409+0000","last_active":"2024-02-18T00:39:19.719468+0000","last_peered":"2024-02-18T00:39:19.719468+0000","last_clean":"2024-02-18T00:39:19.719468+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:39:19.719468+0000","last_undegraded":"2024-02-18T00:39:19.719468+0000","last_fullsized":"2024-02-18T00:39:19.719468+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:56.850326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:36:56.850326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":426,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.712124+0000","last_change":"2024-02-18T00:36:52.372043+0000","last_active":"2024-02-18T00:39:19.712124+0000","last_peered":"2024-02-18T00:39:19.712124+0000","last_clean":"2024-02-18T00:39:19.712124+0000","last_became_active":"2024-02-18T00:33:48.737364+0000","last_became_peered":"2024-02-18T00:33:48.737364+0000","last_unstale":"2024-02-18T00:39:19.712124+0000","last_undegraded":"2024-02-18T00:39:19.712124+0000","last_fullsized":"2024-02-18T00:39:19.712124+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.371970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.371970+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":427,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.727506+0000","last_change":"2024-02-18T00:37:40.707426+0000","last_active":"2024-02-18T00:39:19.727506+0000","last_peered":"2024-02-18T00:39:19.727506+0000","last_clean":"2024-02-18T00:39:19.727506+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:39:19.727506+0000","last_undegraded":"2024-02-18T00:39:19.727506+0000","last_fullsized":"2024-02-18T00:39:19.727506+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.707359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.707359+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":381,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711803+0000","last_change":"2024-02-18T00:37:48.395278+0000","last_active":"2024-02-18T00:39:19.711803+0000","last_peered":"2024-02-18T00:39:19.711803+0000","last_clean":"2024-02-18T00:39:19.711803+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:39:19.711803+0000","last_undegraded":"2024-02-18T00:39:19.711803+0000","last_fullsized":"2024-02-18T00:39:19.711803+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:37:48.395205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.395205+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"64'15","reported_seq":456,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711434+0000","last_change":"2024-02-18T00:37:39.385746+0000","last_active":"2024-02-18T00:39:19.711434+0000","last_peered":"2024-02-18T00:39:19.711434+0000","last_clean":"2024-02-18T00:39:19.711434+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:39:19.711434+0000","last_undegraded":"2024-02-18T00:39:19.711434+0000","last_fullsized":"2024-02-18T00:39:19.711434+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:39.385675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.385675+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":441,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716525+0000","last_change":"2024-02-18T00:37:36.615647+0000","last_active":"2024-02-18T00:39:19.716525+0000","last_peered":"2024-02-18T00:39:19.716525+0000","last_clean":"2024-02-18T00:39:19.716525+0000","last_became_active":"2024-02-18T00:33:48.669355+0000","last_became_peered":"2024-02-18T00:33:48.669355+0000","last_unstale":"2024-02-18T00:39:19.716525+0000","last_undegraded":"2024-02-18T00:39:19.716525+0000","last_fullsized":"2024-02-18T00:39:19.716525+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.615568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.615568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":429,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.727875+0000","last_change":"2024-02-18T00:38:15.534086+0000","last_active":"2024-02-18T00:39:19.727875+0000","last_peered":"2024-02-18T00:39:19.727875+0000","last_clean":"2024-02-18T00:39:19.727875+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:39:19.727875+0000","last_undegraded":"2024-02-18T00:39:19.727875+0000","last_fullsized":"2024-02-18T00:39:19.727875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:15.533986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:38:15.533986+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711146+0000","last_change":"2024-02-18T00:37:34.122873+0000","last_active":"2024-02-18T00:39:19.711146+0000","last_peered":"2024-02-18T00:39:19.711146+0000","last_clean":"2024-02-18T00:39:19.711146+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:39:19.711146+0000","last_undegraded":"2024-02-18T00:39:19.711146+0000","last_fullsized":"2024-02-18T00:39:19.711146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.122764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.122764+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":365,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.716389+0000","last_change":"2024-02-18T00:37:40.562387+0000","last_active":"2024-02-18T00:39:19.716389+0000","last_peered":"2024-02-18T00:39:19.716389+0000","last_clean":"2024-02-18T00:39:19.716389+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:39:19.716389+0000","last_undegraded":"2024-02-18T00:39:19.716389+0000","last_fullsized":"2024-02-18T00:39:19.716389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:40.562306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:40.562306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"64'18","reported_seq":459,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720644+0000","last_change":"2024-02-18T00:37:43.883242+0000","last_active":"2024-02-18T00:39:19.720644+0000","last_peered":"2024-02-18T00:39:19.720644+0000","last_clean":"2024-02-18T00:39:19.720644+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:39:19.720644+0000","last_undegraded":"2024-02-18T00:39:19.720644+0000","last_fullsized":"2024-02-18T00:39:19.720644+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:37:43.883172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.883172+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":448,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720162+0000","last_change":"2024-02-18T00:39:15.558432+0000","last_active":"2024-02-18T00:39:19.720162+0000","last_peered":"2024-02-18T00:39:19.720162+0000","last_clean":"2024-02-18T00:39:19.720162+0000","last_became_active":"2024-02-18T00:33:48.669040+0000","last_became_peered":"2024-02-18T00:33:48.669040+0000","last_unstale":"2024-02-18T00:39:19.720162+0000","last_undegraded":"2024-02-18T00:39:19.720162+0000","last_fullsized":"2024-02-18T00:39:19.720162+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:39:15.558360+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.558360+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":424,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710496+0000","last_change":"2024-02-18T00:37:03.248349+0000","last_active":"2024-02-18T00:39:19.710496+0000","last_peered":"2024-02-18T00:39:19.710496+0000","last_clean":"2024-02-18T00:39:19.710496+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:39:19.710496+0000","last_undegraded":"2024-02-18T00:39:19.710496+0000","last_fullsized":"2024-02-18T00:39:19.710496+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:03.248277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:03.248277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":412,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721280+0000","last_change":"2024-02-18T00:36:35.055243+0000","last_active":"2024-02-18T00:39:19.721280+0000","last_peered":"2024-02-18T00:39:19.721280+0000","last_clean":"2024-02-18T00:39:19.721280+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:39:19.721280+0000","last_undegraded":"2024-02-18T00:39:19.721280+0000","last_fullsized":"2024-02-18T00:39:19.721280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:35.055144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:35.055144+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":376,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710560+0000","last_change":"2024-02-18T00:38:27.705513+0000","last_active":"2024-02-18T00:39:19.710560+0000","last_peered":"2024-02-18T00:39:19.710560+0000","last_clean":"2024-02-18T00:39:19.710560+0000","last_became_active":"2024-02-18T00:33:48.763544+0000","last_became_peered":"2024-02-18T00:33:48.763544+0000","last_unstale":"2024-02-18T00:39:19.710560+0000","last_undegraded":"2024-02-18T00:39:19.710560+0000","last_fullsized":"2024-02-18T00:39:19.710560+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:27.705442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:27.705442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"105'28","reported_seq":484,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710093+0000","last_change":"2024-02-18T00:37:48.216997+0000","last_active":"2024-02-18T00:39:19.710093+0000","last_peered":"2024-02-18T00:39:19.710093+0000","last_clean":"2024-02-18T00:39:19.710093+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:39:19.710093+0000","last_undegraded":"2024-02-18T00:39:19.710093+0000","last_fullsized":"2024-02-18T00:39:19.710093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"105'28","last_scrub_stamp":"2024-02-18T00:37:48.216929+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.216929+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":323,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720261+0000","last_change":"2024-02-18T00:38:22.640818+0000","last_active":"2024-02-18T00:39:19.720261+0000","last_peered":"2024-02-18T00:39:19.720261+0000","last_clean":"2024-02-18T00:39:19.720261+0000","last_became_active":"2024-02-18T00:33:48.673216+0000","last_became_peered":"2024-02-18T00:33:48.673216+0000","last_unstale":"2024-02-18T00:39:19.720261+0000","last_undegraded":"2024-02-18T00:39:19.720261+0000","last_fullsized":"2024-02-18T00:39:19.720261+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:22.640720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.640720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":442,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719748+0000","last_change":"2024-02-18T00:39:10.582509+0000","last_active":"2024-02-18T00:39:19.719748+0000","last_peered":"2024-02-18T00:39:19.719748+0000","last_clean":"2024-02-18T00:39:19.719748+0000","last_became_active":"2024-02-18T00:35:42.660154+0000","last_became_peered":"2024-02-18T00:35:42.660154+0000","last_unstale":"2024-02-18T00:39:19.719748+0000","last_undegraded":"2024-02-18T00:39:19.719748+0000","last_fullsized":"2024-02-18T00:39:19.719748+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:10.582430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:10.582430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":429,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.712780+0000","last_change":"2024-02-18T00:36:45.412472+0000","last_active":"2024-02-18T00:39:19.712780+0000","last_peered":"2024-02-18T00:39:19.712780+0000","last_clean":"2024-02-18T00:39:19.712780+0000","last_became_active":"2024-02-18T00:33:48.649090+0000","last_became_peered":"2024-02-18T00:33:48.649090+0000","last_unstale":"2024-02-18T00:39:19.712780+0000","last_undegraded":"2024-02-18T00:39:19.712780+0000","last_fullsized":"2024-02-18T00:39:19.712780+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:45.412406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:45.412406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":368,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.711222+0000","last_change":"2024-02-18T00:39:03.503323+0000","last_active":"2024-02-18T00:39:19.711222+0000","last_peered":"2024-02-18T00:39:19.711222+0000","last_clean":"2024-02-18T00:39:19.711222+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:39:19.711222+0000","last_undegraded":"2024-02-18T00:39:19.711222+0000","last_fullsized":"2024-02-18T00:39:19.711222+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:03.503256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.503256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"82'14","reported_seq":454,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710085+0000","last_change":"2024-02-18T00:36:51.303043+0000","last_active":"2024-02-18T00:39:19.710085+0000","last_peered":"2024-02-18T00:39:19.710085+0000","last_clean":"2024-02-18T00:39:19.710085+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:39:19.710085+0000","last_undegraded":"2024-02-18T00:39:19.710085+0000","last_fullsized":"2024-02-18T00:39:19.710085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'14","last_scrub_stamp":"2024-02-18T00:36:51.302920+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:36:51.302920+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":325,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.717416+0000","last_change":"2024-02-18T00:36:52.758965+0000","last_active":"2024-02-18T00:39:19.717416+0000","last_peered":"2024-02-18T00:39:19.717416+0000","last_clean":"2024-02-18T00:39:19.717416+0000","last_became_active":"2024-02-18T00:33:48.729205+0000","last_became_peered":"2024-02-18T00:33:48.729205+0000","last_unstale":"2024-02-18T00:39:19.717416+0000","last_undegraded":"2024-02-18T00:39:19.717416+0000","last_fullsized":"2024-02-18T00:39:19.717416+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:52.758904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:36:52.758904+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":328,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710258+0000","last_change":"2024-02-18T00:39:07.344557+0000","last_active":"2024-02-18T00:39:19.710258+0000","last_peered":"2024-02-18T00:39:19.710258+0000","last_clean":"2024-02-18T00:39:19.710258+0000","last_became_active":"2024-02-18T00:33:48.732991+0000","last_became_peered":"2024-02-18T00:33:48.732991+0000","last_unstale":"2024-02-18T00:39:19.710258+0000","last_undegraded":"2024-02-18T00:39:19.710258+0000","last_fullsized":"2024-02-18T00:39:19.710258+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:07.344483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.344483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":417,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719500+0000","last_change":"2024-02-18T00:36:40.825174+0000","last_active":"2024-02-18T00:39:19.719500+0000","last_peered":"2024-02-18T00:39:19.719500+0000","last_clean":"2024-02-18T00:39:19.719500+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:39:19.719500+0000","last_undegraded":"2024-02-18T00:39:19.719500+0000","last_fullsized":"2024-02-18T00:39:19.719500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:40.825055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:36:40.825055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":372,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721114+0000","last_change":"2024-02-18T00:38:24.768769+0000","last_active":"2024-02-18T00:39:19.721114+0000","last_peered":"2024-02-18T00:39:19.721114+0000","last_clean":"2024-02-18T00:39:19.721114+0000","last_became_active":"2024-02-18T00:33:48.694192+0000","last_became_peered":"2024-02-18T00:33:48.694192+0000","last_unstale":"2024-02-18T00:39:19.721114+0000","last_undegraded":"2024-02-18T00:39:19.721114+0000","last_fullsized":"2024-02-18T00:39:19.721114+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.768673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.768673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"251'20","reported_seq":462,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709460+0000","last_change":"2024-02-18T00:38:17.269122+0000","last_active":"2024-02-18T00:39:19.709460+0000","last_peered":"2024-02-18T00:39:19.709460+0000","last_clean":"2024-02-18T00:39:19.709460+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:39:19.709460+0000","last_undegraded":"2024-02-18T00:39:19.709460+0000","last_fullsized":"2024-02-18T00:39:19.709460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:38:17.269050+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:38:17.269050+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":423,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.721124+0000","last_change":"2024-02-18T00:38:22.776453+0000","last_active":"2024-02-18T00:39:19.721124+0000","last_peered":"2024-02-18T00:39:19.721124+0000","last_clean":"2024-02-18T00:39:19.721124+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:39:19.721124+0000","last_undegraded":"2024-02-18T00:39:19.721124+0000","last_fullsized":"2024-02-18T00:39:19.721124+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:22.776331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.776331+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":438,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.726233+0000","last_change":"2024-02-18T00:39:04.586023+0000","last_active":"2024-02-18T00:39:19.726233+0000","last_peered":"2024-02-18T00:39:19.726233+0000","last_clean":"2024-02-18T00:39:19.726233+0000","last_became_active":"2024-02-18T00:33:48.710373+0000","last_became_peered":"2024-02-18T00:33:48.710373+0000","last_unstale":"2024-02-18T00:39:19.726233+0000","last_undegraded":"2024-02-18T00:39:19.726233+0000","last_fullsized":"2024-02-18T00:39:19.726233+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:04.585957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:04.585957+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":422,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.710067+0000","last_change":"2024-02-18T00:37:42.183653+0000","last_active":"2024-02-18T00:39:19.710067+0000","last_peered":"2024-02-18T00:39:19.710067+0000","last_clean":"2024-02-18T00:39:19.710067+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:39:19.710067+0000","last_undegraded":"2024-02-18T00:39:19.710067+0000","last_fullsized":"2024-02-18T00:39:19.710067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:42.183568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:42.183568+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":363,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.720741+0000","last_change":"2024-02-18T00:38:31.741746+0000","last_active":"2024-02-18T00:39:19.720741+0000","last_peered":"2024-02-18T00:39:19.720741+0000","last_clean":"2024-02-18T00:39:19.720741+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:39:19.720741+0000","last_undegraded":"2024-02-18T00:39:19.720741+0000","last_fullsized":"2024-02-18T00:39:19.720741+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:31.741672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:31.741672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"64'39","reported_seq":508,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719430+0000","last_change":"2024-02-18T00:39:06.697405+0000","last_active":"2024-02-18T00:39:19.719430+0000","last_peered":"2024-02-18T00:39:19.719430+0000","last_clean":"2024-02-18T00:39:19.719430+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:39:19.719430+0000","last_undegraded":"2024-02-18T00:39:19.719430+0000","last_fullsized":"2024-02-18T00:39:19.719430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:39:06.697317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.697317+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":437,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709723+0000","last_change":"2024-02-18T00:37:45.454134+0000","last_active":"2024-02-18T00:39:19.709723+0000","last_peered":"2024-02-18T00:39:19.709723+0000","last_clean":"2024-02-18T00:39:19.709723+0000","last_became_active":"2024-02-18T00:33:48.762452+0000","last_became_peered":"2024-02-18T00:33:48.762452+0000","last_unstale":"2024-02-18T00:39:19.709723+0000","last_undegraded":"2024-02-18T00:39:19.709723+0000","last_fullsized":"2024-02-18T00:39:19.709723+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.454069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":424,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.709687+0000","last_change":"2024-02-18T00:37:36.119369+0000","last_active":"2024-02-18T00:39:19.709687+0000","last_peered":"2024-02-18T00:39:19.709687+0000","last_clean":"2024-02-18T00:39:19.709687+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:39:19.709687+0000","last_undegraded":"2024-02-18T00:39:19.709687+0000","last_fullsized":"2024-02-18T00:39:19.709687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:36.119291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:36.119291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":426,"reported_epoch":456,"state":"active+clean","last_fresh":"2024-02-18T00:39:19.719913+0000","last_change":"2024-02-18T00:37:34.860293+0000","last_active":"2024-02-18T00:39:19.719913+0000","last_peered":"2024-02-18T00:39:19.719913+0000","last_clean":"2024-02-18T00:39:19.719913+0000","last_became_active":"2024-02-18T00:33:48.699666+0000","last_became_peered":"2024-02-18T00:33:48.699666+0000","last_unstale":"2024-02-18T00:39:19.719913+0000","last_undegraded":"2024-02-18T00:39:19.719913+0000","last_fullsized":"2024-02-18T00:39:19.719913+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:34.860214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:34.860214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":216,"num_pg":32,"stat_sum":{"num_bytes":14,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":1,"num_write":14,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":42,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":96,"acting":96,"num_store_stats":7},{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4001529,"num_objects":165,"num_object_clones":5,"num_object_copies":495,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":165,"num_whiteouts":0,"num_read":108,"num_read_kb":10,"num_write":611,"num_write_kb":4510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":33,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":13,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14004224,"data_stored":12004680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1913,"ondisk_log_size":1913,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":1942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":12360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":629,"ondisk_log_size":629,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":591,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":450,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":0,"seq":227633266813,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":47,"seq":201863463043,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314536,"kb_used_data":9120,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93430616,"statfs":{"total":95995035648,"available":95672950784,"internally_reserved":0,"allocated":9338880,"data_stored":6794592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.67600000000000005,"15min":1.2529999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69099999999999995,"5min":13.935,"15min":69.194999999999993},"last":0.436},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.66300000000000003,"15min":0.73199999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64600000000000002,"5min":10.433,"15min":10.433},"last":0.57199999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57899999999999996,"15min":1.327},"min":{"1min":0.29399999999999998,"5min":0.27900000000000003,"15min":0.26000000000000001},"max":{"1min":0.68300000000000005,"5min":5.2290000000000001,"15min":76.108999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.61399999999999999,"15min":1.3160000000000001},"min":{"1min":0.32800000000000001,"5min":0.32100000000000001,"15min":0.28100000000000003},"max":{"1min":0.68000000000000005,"5min":7.899,"15min":71.872},"last":0.48599999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.65800000000000003,"15min":0.80700000000000005},"min":{"1min":0.32700000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.70799999999999996,"5min":11.855,"15min":39.939999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.63100000000000001,"15min":0.98299999999999998},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":4.1600000000000001,"15min":39.673000000000002},"last":0.39200000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.69899999999999995,"15min":0.83599999999999997},"min":{"1min":0.372,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.63300000000000001,"5min":8.3420000000000005,"15min":21.219000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.64000000000000001,"15min":0.78100000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.69199999999999995,"5min":5.0069999999999997,"15min":14.007999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.49099999999999999,"15min":0.501},"min":{"1min":0.29799999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61099999999999999,"5min":4.4359999999999999,"15min":8.1479999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.59699999999999998,"15min":0.67800000000000005},"min":{"1min":0.28000000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70299999999999996,"5min":7.907,"15min":35.18},"last":0.50600000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.57199999999999995,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.27900000000000003,"15min":0.27500000000000002},"max":{"1min":0.628,"5min":5.2539999999999996,"15min":10.606},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.57499999999999996,"15min":0.70199999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67200000000000004,"5min":5.5899999999999999,"15min":35.119},"last":0.53400000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:38:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.66500000000000004,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.65000000000000002,"15min":0.72599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69699999999999995,"5min":5.7030000000000003,"15min":32.985999999999997},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":109,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310596,"kb_used_data":5180,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93434556,"statfs":{"total":95995035648,"available":95676985344,"internally_reserved":0,"allocated":5304320,"data_stored":2794286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34100000000000003,"15min":0.63600000000000001},"min":{"1min":0.26800000000000002,"5min":0.19900000000000001,"15min":0.193},"max":{"1min":0.42899999999999999,"5min":0.79600000000000004,"15min":32.106999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38600000000000001,"15min":0.68300000000000005},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.52600000000000002,"5min":2.7040000000000002,"15min":43.137},"last":0.48699999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.96299999999999997,"15min":0.75800000000000001},"min":{"1min":0.27500000000000002,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53700000000000003,"5min":19.241,"15min":49.201000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.77400000000000002,"15min":0.58899999999999997},"min":{"1min":0.27900000000000003,"5min":0.27000000000000002,"15min":0.217},"max":{"1min":0.83199999999999996,"5min":10.654,"15min":17.596},"last":0.51500000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.67300000000000004,"15min":0.74399999999999999},"min":{"1min":0.28199999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68200000000000005,"5min":19.274000000000001,"15min":37.524000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.71399999999999997,"15min":0.76200000000000001},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59799999999999998,"5min":22.989000000000001,"15min":32.036000000000001},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Feb 18 00:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.77200000000000002},"min":{"1min":0.45200000000000001,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.81399999999999995,"5min":0.88900000000000001,"15min":17.486999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.77600000000000002},"min":{"1min":0.40699999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89000000000000001,"5min":1.9790000000000001,"15min":12.75},"last":0.89000000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.439,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":8.9019999999999992,"15min":12.611000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.72399999999999998,"15min":0.78400000000000003},"min":{"1min":0.42199999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":4.9550000000000001,"15min":17.417000000000002},"last":0.90100000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.81000000000000005},"min":{"1min":0.45400000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81299999999999994,"5min":2.0430000000000001,"15min":12.834},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.79400000000000004},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81299999999999994,"5min":0.90400000000000003,"15min":12.651999999999999},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.95299999999999996,"15min":0.80200000000000005},"min":{"1min":0.46300000000000002,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.83999999999999997,"5min":9.0830000000000002,"15min":17.452000000000002},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97999999999999998,"15min":0.81399999999999995},"min":{"1min":0.442,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80900000000000005,"5min":9.3610000000000007,"15min":12.805},"last":0.91500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738528,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306540,"kb_used_data":5220,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438612,"statfs":{"total":95995035648,"available":95681138688,"internally_reserved":0,"allocated":5345280,"data_stored":2793461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.379,"15min":1.0780000000000001},"min":{"1min":0.19500000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.52400000000000002,"5min":2.524,"15min":133.38999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.48499999999999999,"15min":0.77700000000000002},"min":{"1min":0.26300000000000001,"5min":0.221,"15min":0.20999999999999999},"max":{"1min":0.82199999999999995,"5min":4.577,"15min":90.623000000000005},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.97299999999999998,"15min":0.89400000000000002},"min":{"1min":0.27500000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.94799999999999995,"5min":21.388999999999999,"15min":90.850999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.91700000000000004,"15min":0.873},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.482,"5min":23.388999999999999,"15min":80.537000000000006},"last":0.28000000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.40300000000000002,"15min":0.66800000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56200000000000006,"5min":3.5609999999999999,"15min":19.006},"last":0.252},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.47299999999999998,"15min":1.069},"min":{"1min":0.27300000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.90300000000000002,"5min":4.4489999999999998,"15min":146.71899999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:39:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.55700000000000005,"15min":0.91800000000000004},"min":{"1min":0.36499999999999999,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.68999999999999995,"5min":3.1850000000000001,"15min":57.914000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63700000000000001,"15min":0.97899999999999998},"min":{"1min":0.44500000000000001,"5min":0.318,"15min":0.30399999999999999},"max":{"1min":0.97999999999999998,"5min":4.2919999999999998,"15min":90.070999999999998},"last":0.70799999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.88100000000000001,"15min":0.95199999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32000000000000001},"max":{"1min":1.056,"5min":11.579000000000001,"15min":90.049000000000007},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":12.952999999999999,"15min":57.991},"last":0.495}]},{"osd":6,"last update":"Sun Feb 18 00:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.626,"15min":0.79700000000000004},"min":{"1min":0.38100000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":5.2830000000000004,"15min":19.039000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.97399999999999998},"min":{"1min":0.42799999999999999,"5min":0.34899999999999998,"15min":0.33500000000000002},"max":{"1min":1.083,"5min":4.343,"15min":89.989999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":1.0580000000000001,"15min":1.048},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.073,"5min":13.24,"15min":90.025000000000006},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":1.044,"15min":0.997},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":1.603,"5min":13.019,"15min":57.962000000000003},"last":0.52600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411479,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306608,"kb_used_data":5288,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438544,"statfs":{"total":95995035648,"available":95681069056,"internally_reserved":0,"allocated":5414912,"data_stored":2793804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.57599999999999996,"15min":0.48799999999999999},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.46899999999999997,"5min":10.087999999999999,"15min":12.144},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.92000000000000004,"15min":0.60499999999999998},"min":{"1min":0.253,"5min":0.221,"15min":0.20999999999999999},"max":{"1min":0.55800000000000005,"5min":18.891999999999999,"15min":18.891999999999999},"last":0.245}]},{"osd":1,"last update":"Sun Feb 18 00:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.58899999999999997,"15min":0.49399999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.19700000000000001},"max":{"1min":0.502,"5min":10.138999999999999,"15min":10.138999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.373,"5min":0.77300000000000002,"15min":0.54300000000000004},"min":{"1min":0.24199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.51500000000000001,"5min":18.919,"15min":18.919},"last":0.28699999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.46600000000000003,"15min":0.60099999999999998},"min":{"1min":0.29499999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.48899999999999999,"5min":3.3809999999999998,"15min":22.712},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47399999999999998,"15min":0.61199999999999999},"min":{"1min":0.29099999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.57099999999999995,"5min":4.8529999999999998,"15min":22.359000000000002},"last":0.38700000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55800000000000005,"15min":0.68300000000000005},"min":{"1min":0.37,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77600000000000002,"5min":0.875,"15min":18.812000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54100000000000004,"15min":0.67200000000000004},"min":{"1min":0.36099999999999999,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":10.016},"last":0.69199999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:38:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.61099999999999999,"15min":0.64900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83799999999999997,"5min":2.98,"15min":6.766},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.42399999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.72299999999999998,"5min":10.114000000000001,"15min":10.114000000000001},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56899999999999995,"15min":0.67700000000000005},"min":{"1min":0.41299999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.749,"5min":1.196,"15min":6.806},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61299999999999999,"15min":0.72599999999999998},"min":{"1min":0.441,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76100000000000001,"5min":4.827,"15min":18.861000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:38:25 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.76200000000000001,"15min":0.71799999999999997},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.89300000000000002,"5min":6.8300000000000001,"15min":6.8300000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.79400000000000004,"15min":0.74199999999999999},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.372},"max":{"1min":0.85799999999999998,"5min":10.058999999999999,"15min":10.058999999999999},"last":0.58999999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149841,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":310548,"kb_used_data":9228,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93434604,"statfs":{"total":95995035648,"available":95677034496,"internally_reserved":0,"allocated":9449472,"data_stored":6794633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.59799999999999998,"15min":0.47299999999999998},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54900000000000004,"5min":25.082999999999998,"15min":25.082999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.379,"5min":0.58299999999999996,"15min":0.52400000000000002},"min":{"1min":0.23699999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57599999999999996,"5min":18.911000000000001,"15min":18.911000000000001},"last":0.39000000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.44700000000000001,"15min":0.47499999999999998},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.55600000000000005,"5min":4.1550000000000002,"15min":6.8479999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.377,"5min":0.72099999999999997,"15min":0.69399999999999995},"min":{"1min":0.25,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59899999999999998,"5min":23.439,"15min":23.606999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.745,"15min":0.65800000000000003},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56399999999999995,"5min":23.405000000000001,"15min":23.405000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.47199999999999998,"15min":0.46700000000000003},"min":{"1min":0.27700000000000002,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":0.66000000000000003,"5min":5.3479999999999999,"15min":5.3479999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Feb 18 00:39:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59499999999999997,"15min":0.68600000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.95499999999999996,"5min":3.8410000000000002,"15min":10.131},"last":0.753},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57199999999999995,"15min":0.627},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":5.4630000000000001},"last":0.80900000000000005}]},{"osd":5,"last update":"Sun Feb 18 00:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.59999999999999998,"15min":0.623},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":3.9670000000000001,"15min":7.4249999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.41499999999999998,"5min":0.35999999999999999,"15min":0.32400000000000001},"max":{"1min":0.71699999999999997,"5min":4.2149999999999999,"15min":6.7750000000000004},"last":0.76300000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61699999999999999,"15min":0.745},"min":{"1min":0.377,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.92300000000000004,"5min":3.8940000000000001,"15min":10.189},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61099999999999999,"15min":0.66100000000000003},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":6.4279999999999999},"last":0.85599999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.77600000000000002,"15min":0.66500000000000004},"min":{"1min":0.44,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76800000000000002,"5min":6.8220000000000001,"15min":6.8220000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.90000000000000002,"15min":0.72099999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":10.164999999999999,"15min":10.164999999999999},"last":0.66300000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986316,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314744,"kb_used_data":5232,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430408,"statfs":{"total":95995035648,"available":95672737792,"internally_reserved":0,"allocated":5357568,"data_stored":2794719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":1.4830000000000001,"15min":0.90300000000000002},"min":{"1min":0.33900000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.70799999999999996,"5min":77.581999999999994,"15min":77.581999999999994},"last":0.247},{"interface":"front","average":{"1min":0.497,"5min":1.46,"15min":0.86599999999999999},"min":{"1min":0.31900000000000001,"5min":0.28199999999999997,"15min":0.27800000000000002},"max":{"1min":0.68600000000000005,"5min":78.063999999999993,"15min":78.063999999999993},"last":0.41799999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.746,"15min":0.64000000000000001},"min":{"1min":0.29199999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64000000000000001,"5min":9.3160000000000007,"15min":9.3160000000000007},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.498,"5min":1.569,"15min":0.89700000000000002},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.66500000000000004,"5min":81.242000000000004,"15min":81.242000000000004},"last":0.30099999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.76700000000000002,"15min":0.65700000000000003},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67100000000000004,"5min":11.691000000000001,"15min":11.691000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.872,"15min":0.67600000000000005},"min":{"1min":0.33500000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68400000000000005,"5min":17.77,"15min":17.77},"last":0.35899999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:38:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.67100000000000004,"15min":0.621},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.28299999999999997},"max":{"1min":0.65000000000000002,"5min":7.218,"15min":7.218},"last":0.318},{"interface":"front","average":{"1min":0.55200000000000005,"5min":1.0569999999999999,"15min":0.72999999999999998},"min":{"1min":0.40500000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.70499999999999996,"5min":33.530999999999999,"15min":33.530999999999999},"last":0.33300000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:38:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.40400000000000003},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.20300000000000001},"max":{"1min":0.60999999999999999,"5min":0.70399999999999996,"15min":0.72299999999999998},"last":0.371},{"interface":"front","average":{"1min":0.438,"5min":0.44700000000000001,"15min":0.41299999999999998},"min":{"1min":0.313,"5min":0.26500000000000001,"15min":0.24299999999999999},"max":{"1min":0.60999999999999999,"5min":0.68999999999999995,"15min":0.77900000000000003},"last":0.38400000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45100000000000001,"15min":0.42599999999999999},"min":{"1min":0.30599999999999999,"5min":0.26100000000000001,"15min":0.23699999999999999},"max":{"1min":0.56499999999999995,"5min":0.70299999999999996,"15min":0.76100000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.49199999999999999,"15min":0.46700000000000003},"min":{"1min":0.36399999999999999,"5min":0.28299999999999997,"15min":0.26000000000000001},"max":{"1min":0.59299999999999997,"5min":0.69299999999999995,"15min":0.751},"last":0.40799999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:38:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.50700000000000001,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.71299999999999997,"5min":0.72399999999999998,"15min":0.76900000000000002},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.495,"15min":0.48899999999999999},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.63300000000000001,"5min":0.73599999999999999,"15min":0.83299999999999996},"last":0.503}]}]},{"osd":5,"up_from":39,"seq":167503724679,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":314468,"kb_used_data":9052,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93430684,"statfs":{"total":95995035648,"available":95673020416,"internally_reserved":0,"allocated":9269248,"data_stored":6794588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.624,"15min":0.55300000000000005},"min":{"1min":0.30499999999999999,"5min":0.217,"15min":0.217},"max":{"1min":2.157,"5min":6.3150000000000004,"15min":6.3150000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.68200000000000005,"15min":0.625},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.1859999999999999,"5min":6.8680000000000003,"15min":12.744},"last":0.436}]},{"osd":1,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.79400000000000004,"15min":0.65900000000000003},"min":{"1min":0.32300000000000001,"5min":0.316,"15min":0.27400000000000002},"max":{"1min":13.25,"5min":13.25,"15min":13.25},"last":0.36399999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.81000000000000005,"15min":0.66700000000000004},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.29699999999999999},"max":{"1min":13.262,"5min":13.262,"15min":13.262},"last":0.40600000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.64500000000000002,"15min":0.60799999999999998},"min":{"1min":0.27800000000000002,"5min":0.26200000000000001,"15min":0.251},"max":{"1min":0.79400000000000004,"5min":6.1680000000000001,"15min":6.1680000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.745,"15min":0.65200000000000002},"min":{"1min":0.36799999999999999,"5min":0.311,"15min":0.308},"max":{"1min":5.0659999999999998,"5min":6.1879999999999997,"15min":6.1879999999999997},"last":0.46700000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77200000000000002,"15min":0.63600000000000001},"min":{"1min":0.32000000000000001,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":7.641,"5min":7.641,"15min":7.641},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.753,"15min":0.61599999999999999},"min":{"1min":0.39100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.4580000000000002,"5min":8.8650000000000002,"15min":8.8650000000000002},"last":0.41599999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:38:33 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47899999999999998,"15min":0.45900000000000002},"min":{"1min":0.26000000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.92600000000000005,"5min":4.6859999999999999,"15min":11.818},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47799999999999998,"15min":0.443},"min":{"1min":0.34100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.84899999999999998,"5min":1.5680000000000001,"15min":4.274},"last":0.34899999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.53600000000000003},"min":{"1min":0.315,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":2.5939999999999999,"5min":12.362,"15min":18.605},"last":0.379},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.498,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.88600000000000001,"5min":1.46,"15min":4.2089999999999996},"last":0.42399999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:38:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51800000000000002,"15min":0.48299999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.5269999999999999,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.51300000000000001},"min":{"1min":0.35599999999999998,"5min":0.318,"15min":0.309},"max":{"1min":0.76000000000000001,"5min":4.649,"15min":4.649},"last":0.45100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4366336,"data_stored":4000852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4190208,"data_stored":4000433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4288512,"data_stored":4000671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":33,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":216,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":216,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":2,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":216,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":4,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":216,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":216,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":216,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":216,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":4,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:39:25.578 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-18T00:39:25.578 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:39:25.578 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-02-18T00:39:25.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd in 7 2024-02-18T00:39:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:25.827 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:25.827 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:25.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:25.958 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:26.056 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:26.059 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:26.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:26.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:26.189 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:26.327 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 -- 172.21.15.59:0/2892568626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f9270 msgr2=0x7f54100f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 --2- 172.21.15.59:0/2892568626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f9270 0x7f54100f6c50 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 rx=0x7f5404004870 tx=0x7f5404009cb0).stop 2024-02-18T00:39:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 -- 172.21.15.59:0/2892568626 shutdown_connections 2024-02-18T00:39:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 --2- 172.21.15.59:0/2892568626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54100f89a0 0x7f54100f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:26.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 --2- 172.21.15.59:0/2892568626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54100f7330 0x7f5410103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 --2- 172.21.15.59:0/2892568626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f9270 0x7f54100f6c50 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 -- 172.21.15.59:0/2892568626 >> 172.21.15.59:0/2892568626 conn(0x7f54100f26f0 msgr2=0x7f54100f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 -- 172.21.15.59:0/2892568626 shutdown_connections 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.330+0000 7f54154e5700 1 -- 172.21.15.59:0/2892568626 wait complete. 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 librados: starting msgr at 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 librados: starting objecter 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 Processor -- start 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 -- start start 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 librados: setting wanted keys 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 librados: calling monclient init 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54100f7330 0x7f54100f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f89a0 0x7f54100f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:26.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54100f9270 0x7f54100fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.331+0000 7f54154e5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54101165b0 con 0x7f54100f7330 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f54154e5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410116720 con 0x7f54100f9270 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f54154e5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54101168e0 con 0x7f54100f89a0 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f89a0 0x7f54100f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f89a0 0x7f54100f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:36388/0 (socket says 172.21.15.59:36388) 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 -- 172.21.15.59:0/758402323 learned_addr learned my addr 172.21.15.59:0/758402323 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540effd700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54100f7330 0x7f54100f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540f7fe700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54100f9270 0x7f54100fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 -- 172.21.15.59:0/758402323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54100f9270 msgr2=0x7f54100fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54100f9270 0x7f54100fa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 -- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54100f7330 msgr2=0x7f54100f99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:26.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54100f7330 0x7f54100f99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 -- 172.21.15.59:0/758402323 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54080097c0 con 0x7f54100f89a0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540effd700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54100f7330 0x7f54100f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f540e7fc700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f89a0 0x7f54100f9e90 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 rx=0x7f5404009eb0 tx=0x7f540400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f53f7fff700 10 client.?.objecter ms_handle_connect 0x7f54100f89a0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f53f7fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f53f7fff700 1 -- 172.21.15.59:0/758402323 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400e070 con 0x7f54100f89a0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5404005040 con 0x7f54100f89a0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.332+0000 7f53f7fff700 1 -- 172.21.15.59:0/758402323 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5404013da0 con 0x7f54100f89a0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.333+0000 7f53f7fff700 1 -- 172.21.15.59:0/758402323 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404019070 con 0x7f54100f89a0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.333+0000 7f54154e5700 10 client.41219.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.333+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5410116eb0 con 0x7f54100f89a0 2024-02-18T00:39:26.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.333+0000 7f54154e5700 1 librados: init done 2024-02-18T00:39:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.334+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541005ee80 con 0x7f54100f89a0 2024-02-18T00:39:26.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.334+0000 7f53f7fff700 1 -- 172.21.15.59:0/758402323 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f540401e420 con 0x7f54100f89a0 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f53f7fff700 1 --2- 172.21.15.59:0/758402323 >> v2:172.21.15.59:6800/3721891797 conn(0x7f53f8062c90 0x7f53f8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f53f7fff700 1 -- 172.21.15.59:0/758402323 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(458..458 src has 180..458) v4 ==== 8035+0+0 (secure 0 0 0) 0x7f540407ddb0 con 0x7f54100f89a0 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f53f7fff700 10 client.41219.objecter ms_dispatch 0x7f5410001550 osd_map(458..458 src has 180..458) v4 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f53f7fff700 3 client.41219.objecter handle_osd_map got epochs [458,458] > 0 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f53f7fff700 3 client.41219.objecter handle_osd_map decoding full epoch 458 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f540effd700 1 --2- 172.21.15.59:0/758402323 >> v2:172.21.15.59:6800/3721891797 conn(0x7f53f8062c90 0x7f53f8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f53f7fff700 20 client.41219.objecter dump_active .. 0 homeless 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f53f7fff700 10 client.41219.objecter ms_handle_connect 0x7f53f8062c90 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.335+0000 7f540effd700 1 --2- 172.21.15.59:0/758402323 >> v2:172.21.15.59:6800/3721891797 conn(0x7f53f8062c90 0x7f53f8065150 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 rx=0x7f54000041d0 tx=0x7f5400009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:26.339 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:26.340 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:24.876198+0000 osd.6 (osd.6) 221 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:24.877388+0000 osd.6 (osd.6) 222 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:25.272887+0000 osd.2 (osd.2) 195 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:25.273217+0000 osd.2 (osd.2) 196 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:25.387106+0000 osd.2 (osd.2) 197 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:25.388146+0000 osd.2 (osd.2) 198 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:24.876198+0000 osd.6 (osd.6) 221 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:24.877388+0000 osd.6 (osd.6) 222 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:25.272887+0000 osd.2 (osd.2) 195 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:26.341 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:25.273217+0000 osd.2 (osd.2) 196 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:26.342 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:25.387106+0000 osd.2 (osd.2) 197 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:39:26.342 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:25.388146+0000 osd.2 (osd.2) 198 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:39:26.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.339+0000 7f53f7fff700 1 -- 172.21.15.59:0/758402323 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f54040512d0 con 0x7f54100f89a0 2024-02-18T00:39:26.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:26.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:26.444 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:26.472 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:24.876198+0000 osd.6 (osd.6) 221 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:39:26.472 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:24.877388+0000 osd.6 (osd.6) 222 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:39:26.472 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:25.272887+0000 osd.2 (osd.2) 195 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:26.472 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:25.273217+0000 osd.2 (osd.2) 196 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:26.472 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:25.387106+0000 osd.2 (osd.2) 197 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:39:26.472 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:25.388146+0000 osd.2 (osd.2) 198 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:39:26.473 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:26.493 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:26.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:26.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:26.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:26.530+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f54100fae80 con 0x7f54100f89a0 2024-02-18T00:39:26.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:26.628 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:26.878 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:26.880 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:26.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:26.982 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:27.014 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:27.035 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:27.038 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:27.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.137+0000 7f53f7fff700 1 -- 172.21.15.59:0/758402323 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v459) v1 ==== 87+0+0 (secure 0 0 0) 0x7f5404023030 con 0x7f54100f89a0 2024-02-18T00:39:27.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:27.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.139+0000 7f54154e5700 10 librados: watch_flush enter 2024-02-18T00:39:27.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.140+0000 7f54154e5700 10 librados: watch_flush exit 2024-02-18T00:39:27.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.140+0000 7f54154e5700 20 client.41219.objecter shutdown clearing up homeless session... 2024-02-18T00:39:27.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.140+0000 7f54154e5700 10 client.41219.objecter successfully canceled tick 2024-02-18T00:39:27.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.140+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 >> v2:172.21.15.59:6800/3721891797 conn(0x7f53f8062c90 msgr2=0x7f53f8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:27.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.140+0000 7f54154e5700 1 --2- 172.21.15.59:0/758402323 >> v2:172.21.15.59:6800/3721891797 conn(0x7f53f8062c90 0x7f53f8065150 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 rx=0x7f54000041d0 tx=0x7f5400009040).stop 2024-02-18T00:39:27.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.140+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f89a0 msgr2=0x7f54100f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:27.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.140+0000 7f54154e5700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f89a0 0x7f54100f9e90 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 rx=0x7f5404009eb0 tx=0x7f540400b040).stop 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.141+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 shutdown_connections 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.141+0000 7f54154e5700 1 --2- 172.21.15.59:0/758402323 >> v2:172.21.15.59:6800/3721891797 conn(0x7f53f8062c90 0x7f53f8065150 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.141+0000 7f54154e5700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54100f9270 0x7f54100fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.141+0000 7f54154e5700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54100f7330 0x7f54100f99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.141+0000 7f54154e5700 1 --2- 172.21.15.59:0/758402323 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54100f89a0 0x7f54100f9e90 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.141+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 >> 172.21.15.59:0/758402323 conn(0x7f54100f26f0 msgr2=0x7f5410060fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.142+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 shutdown_connections 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.142+0000 7f54154e5700 1 -- 172.21.15.59:0/758402323 wait complete. 2024-02-18T00:39:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:27.142+0000 7f54154e5700 1 librados: shutdown 2024-02-18T00:39:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:39:27.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:25.691884+0000 osd.3 (osd.3) 257 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:39:27.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:25.693170+0000 osd.3 (osd.3) 258 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:39:27.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:25.918668+0000 mgr.y (mgr.24454) 430 : cluster [DBG] pgmap v688: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s rd, 2.2 KiB/s wr, 104 op/s 2024-02-18T00:39:27.166 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:26.133183+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e458: 8 total, 8 up, 7 in 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:26.532186+0000 mon.c (mon.2) 440 : audit [INF] from='client.? 172.21.15.59:0/758402323' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:26.532477+0000 mon.a (mon.0) 2612 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:25.691884+0000 osd.3 (osd.3) 257 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:25.693170+0000 osd.3 (osd.3) 258 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:25.918668+0000 mgr.y (mgr.24454) 430 : cluster [DBG] pgmap v688: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s rd, 2.2 KiB/s wr, 104 op/s 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:26.133183+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e458: 8 total, 8 up, 7 in 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:26.532186+0000 mon.c (mon.2) 440 : audit [INF] from='client.? 172.21.15.59:0/758402323' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:39:27.167 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:26.532477+0000 mon.a (mon.0) 2612 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:39:27.173 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:25.691884+0000 osd.3 (osd.3) 257 : cluster [DBG] 6.1f scrub starts 2024-02-18T00:39:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:25.693170+0000 osd.3 (osd.3) 258 : cluster [DBG] 6.1f scrub ok 2024-02-18T00:39:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:25.918668+0000 mgr.y (mgr.24454) 430 : cluster [DBG] pgmap v688: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s rd, 2.2 KiB/s wr, 104 op/s 2024-02-18T00:39:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:26.133183+0000 mon.a (mon.0) 2611 : cluster [DBG] osdmap e458: 8 total, 8 up, 7 in 2024-02-18T00:39:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:26.532186+0000 mon.c (mon.2) 440 : audit [INF] from='client.? 172.21.15.59:0/758402323' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:39:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:26.532477+0000 mon.a (mon.0) 2612 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:39:27.824 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:27.827 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:27.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:27.929 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:27.961 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:27.983 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:27.985 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:27.986 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:28.087 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:28.119 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:28.140 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:28.144 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:28.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:28.245 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:28.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:26.354448+0000 osd.1 (osd.1) 235 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:39:28.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:26.359124+0000 osd.1 (osd.1) 236 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:39:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:27.127828+0000 mon.a (mon.0) 2613 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:27.137028+0000 mon.a (mon.0) 2614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:39:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:27.137095+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e459: 8 total, 8 up, 7 in 2024-02-18T00:39:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:27.513387+0000 osd.7 (osd.7) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:27.513484+0000 osd.7 (osd.7) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:28.275 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:28.296 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:28.298 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:28.298 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:28.369 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:28.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:28.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:26.354448+0000 osd.1 (osd.1) 235 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:39:28.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:26.359124+0000 osd.1 (osd.1) 236 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:39:28.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:27.127828+0000 mon.a (mon.0) 2613 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:28.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:27.137028+0000 mon.a (mon.0) 2614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:39:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:27.137095+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e459: 8 total, 8 up, 7 in 2024-02-18T00:39:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:27.513387+0000 osd.7 (osd.7) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:28.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:27.513484+0000 osd.7 (osd.7) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:28.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:26.354448+0000 osd.1 (osd.1) 235 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:39:28.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:26.359124+0000 osd.1 (osd.1) 236 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:39:28.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:27.127828+0000 mon.a (mon.0) 2613 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:28.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:27.137028+0000 mon.a (mon.0) 2614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:39:28.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:27.137095+0000 mon.a (mon.0) 2615 : cluster [DBG] osdmap e459: 8 total, 8 up, 7 in 2024-02-18T00:39:28.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:27.513387+0000 osd.7 (osd.7) 105 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:28.406 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:27.513484+0000 osd.7 (osd.7) 106 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:28.432 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:27.919058+0000 mgr.y (mgr.24454) 431 : cluster [DBG] pgmap v691: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s rd, 2.2 KiB/s wr, 104 op/s 2024-02-18T00:39:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:28.141060+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e460: 8 total, 8 up, 7 in 2024-02-18T00:39:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:28.696468+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e461: 8 total, 8 up, 7 in 2024-02-18T00:39:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:27.919058+0000 mgr.y (mgr.24454) 431 : cluster [DBG] pgmap v691: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s rd, 2.2 KiB/s wr, 104 op/s 2024-02-18T00:39:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:28.141060+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e460: 8 total, 8 up, 7 in 2024-02-18T00:39:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:28.696468+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e461: 8 total, 8 up, 7 in 2024-02-18T00:39:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:27.919058+0000 mgr.y (mgr.24454) 431 : cluster [DBG] pgmap v691: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s rd, 2.2 KiB/s wr, 104 op/s 2024-02-18T00:39:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:28.141060+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e460: 8 total, 8 up, 7 in 2024-02-18T00:39:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:28.696468+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e461: 8 total, 8 up, 7 in 2024-02-18T00:39:29.511 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:29.515 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:29.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:29.617 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:29.649 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:29.671 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:29.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:29.674 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:29.775 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:29.805 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:29.825 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:29.827 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:29.828 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:29.929 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:29.962 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:29.852378+0000 mon.c (mon.2) 441 : audit [DBG] from='client.? 172.21.15.59:0/4206166739' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:29.852378+0000 mon.c (mon.2) 441 : audit [DBG] from='client.? 172.21.15.59:0/4206166739' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:30.311 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:30.313 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:30.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:30.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:39:30.414 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:30.442 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:29.852378+0000 mon.c (mon.2) 441 : audit [DBG] from='client.? 172.21.15.59:0/4206166739' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:30.444 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:30.464 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:30.467 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:30.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:30.568 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:30.595 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:30.618 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:30.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:30.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:30.722 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:30.753 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:30.880 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:39:30.880 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:39:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:39:30.881 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:39:30.881 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:39:30.883 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:30.886 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:30.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:30.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:31.365 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:31.367 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:31.368 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:31.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:31.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:29.919494+0000 mgr.y (mgr.24454) 432 : cluster [DBG] pgmap v694: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:31.500 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:29.919494+0000 mgr.y (mgr.24454) 432 : cluster [DBG] pgmap v694: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:31.502 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:31.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:39:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:39:31.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:39:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:39:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:29.919494+0000 mgr.y (mgr.24454) 432 : cluster [DBG] pgmap v694: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:39:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:31.305087+0000 mgr.y (mgr.24454) 433 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:32.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:39:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:32] "GET /metrics HTTP/1.1" 200 189884 "" "Prometheus/2.33.4" 2024-02-18T00:39:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:31.305087+0000 mgr.y (mgr.24454) 433 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:32.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:31.305087+0000 mgr.y (mgr.24454) 433 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/1160010078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 msgr2=0x7ff8e80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/1160010078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 0x7ff8e80fdf90 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7ff8d8004870 tx=0x7ff8d8009cb0).stop 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/1160010078 shutdown_connections 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/1160010078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8e80f8420 0x7ff8e8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/1160010078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 0x7ff8e80fdf90 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/1160010078 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80fe510 0x7ff8e80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/1160010078 >> 172.21.15.59:0/1160010078 conn(0x7ff8e80f3800 msgr2=0x7ff8e80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/1160010078 shutdown_connections 2024-02-18T00:39:33.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.847+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/1160010078 wait complete. 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 librados: starting msgr at 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 librados: starting objecter 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 Processor -- start 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 -- start start 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 librados: setting wanted keys 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 librados: calling monclient init 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80f8420 0x7ff8e810e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 0x7ff8e810e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8e80fe510 0x7ff8e8108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8108770 con 0x7ff8e80fdbb0 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e81088e0 con 0x7ff8e80fe510 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.848+0000 7ff8ee8e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8108aa0 con 0x7ff8e80f8420 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80f8420 0x7ff8e810e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80f8420 0x7ff8e810e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:58664/0 (socket says 172.21.15.59:58664) 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 -- 172.21.15.59:0/4206166739 learned_addr learned my addr 172.21.15.59:0/4206166739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 0x7ff8e810e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8ece86700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8e80fe510 0x7ff8e8108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 -- 172.21.15.59:0/4206166739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8e80fe510 msgr2=0x7ff8e8108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8e80fe510 0x7ff8e8108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 -- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 msgr2=0x7ff8e810e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 0x7ff8e810e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 -- 172.21.15.59:0/4206166739 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc005040 con 0x7ff8e80f8420 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e80fdbb0 0x7ff8e810e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80f8420 0x7ff8e810e040 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7ff8d8009eb0 tx=0x7ff8d800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e57fa700 10 client.?.objecter ms_handle_connect 0x7ff8e80f8420 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e57fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800e070 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d8005040 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8d8013da0 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8019070 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8ee8e9700 10 client.41225.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e8109010 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.849+0000 7ff8ee8e9700 1 librados: init done 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.850+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff8e8107060 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff8d801e420 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff8d0062c90 0x7ff8d0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(461..461 src has 180..461) v4 ==== 8035+0+0 (secure 0 0 0) 0x7ff8d807dc20 con 0x7ff8e80f8420 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 osd_map(461..461 src has 180..461) v4 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 3 client.41225.objecter handle_osd_map got epochs [461,461] > 0 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 3 client.41225.objecter handle_osd_map decoding full epoch 461 2024-02-18T00:39:33.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff8d0062c90 0x7ff8d0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 20 client.41225.objecter dump_active .. 0 homeless 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v461) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff8d80802f0 con 0x7ff8e80f8420 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 1 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8d0062c90 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff8d0062c90 0x7ff8d0065150 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 rx=0x7ff8e8109980 tx=0x7ff8dc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8e80fee60 0x7ff8e80ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 1 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 0 1 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81096c0 osd=0 4 2024-02-18T00:39:33.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 1 2024-02-18T00:39:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.851+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e80ff730 con 0x7ff8e80fee60 2024-02-18T00:39:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8ece86700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8e80fee60 0x7ff8e80ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8ece86700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8e80fee60 0x7ff8e80ff240 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e80fee60 2024-02-18T00:39:33.243 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e80ff730 con 0x7ff8e80fee60 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 1 = system:0 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e81096c0 osd=0 4 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.852+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 0 1 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 2 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 2 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e81096c0 osd=0 3 2024-02-18T00:39:33.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 0 2 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81096c0 osd=0 4 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 2 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.867+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e80ff920 con 0x7ff8e80fee60 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e80ff920 con 0x7ff8e80fee60 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e81096c0 osd=0 4 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 0 2 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 3 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8e8028540 0x7ff8e8118ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 3 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 1 3 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80fd4e0 osd=1 4 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 3 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e80fd650 con 0x7ff8e8028540 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8e8028540 0x7ff8e8118ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e8028540 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.869+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8e8028540 0x7ff8e8118ef0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.870+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e80fd650 con 0x7ff8e8028540 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.870+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.870+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 3 = system:0 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.870+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e80fd4e0 osd=1 4 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.870+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 1 3 2024-02-18T00:39:33.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 4 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 4 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e80fd4e0 osd=1 3 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 1 4 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80fd4e0 osd=1 4 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 4 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.885+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e805a040 con 0x7ff8e8028540 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e805a040 con 0x7ff8e8028540 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e80fd4e0 osd=1 4 2024-02-18T00:39:33.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 1 4 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 5 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff8e80594d0 0x7ff8e811c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 5 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 2 5 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8059360 osd=2 4 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 5 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e80280a0 con 0x7ff8e80594d0 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff8e80594d0 0x7ff8e811c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e80594d0 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.887+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff8e80594d0 0x7ff8e811c570 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.249 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.888+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e80280a0 con 0x7ff8e80594d0 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.888+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.888+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 5 = system:0 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.888+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8059360 osd=2 4 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.888+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 2 5 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 6 2024-02-18T00:39:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 6 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8059360 osd=2 3 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 2 6 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8059360 osd=2 4 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 6 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.902+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e8028330 con 0x7ff8e80594d0 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e8028330 con 0x7ff8e80594d0 2024-02-18T00:39:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8059360 osd=2 4 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 2 6 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 7 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff8e811a0c0 0x7ff8e811fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 7 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 3 7 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80278e0 osd=3 4 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 7 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e8122880 con 0x7ff8e811a0c0 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ece86700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff8e811a0c0 0x7ff8e811fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e811a0c0 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.904+0000 7ff8ece86700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff8e811a0c0 0x7ff8e811fbf0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.905+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e8122880 con 0x7ff8e811a0c0 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.905+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.905+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 7 = system:0 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.905+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e80278e0 osd=3 4 2024-02-18T00:39:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.905+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 3 7 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 8 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 8 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e80278e0 osd=3 3 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 3 8 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80278e0 osd=3 4 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 8 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.920+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e8119620 con 0x7ff8e811a0c0 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e8119620 con 0x7ff8e811a0c0 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e80278e0 osd=3 4 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 3 8 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 9 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff8e811d740 0x7ff8e8121260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 9 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 4 9 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81193e0 osd=4 4 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 9 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.921+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e8119550 con 0x7ff8e811d740 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.922+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff8e811d740 0x7ff8e8121260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.922+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff8e811d740 0x7ff8e8121260 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.922+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e811d740 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.923+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e8119550 con 0x7ff8e811d740 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.923+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.923+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 9 = system:0 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.923+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e81193e0 osd=4 4 2024-02-18T00:39:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.923+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 4 9 2024-02-18T00:39:33.257 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:39:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 10 2024-02-18T00:39:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 10 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e81193e0 osd=4 3 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 4 10 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81193e0 osd=4 4 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 10 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.939+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e8119550 con 0x7ff8e811d740 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e8119550 con 0x7ff8e811d740 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e81193e0 osd=4 4 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 4 10 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 11 2024-02-18T00:39:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff8e8121cc0 0x7ff8e8122100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 11 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 5 11 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8119550 osd=5 4 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 11 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.941+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e8129220 con 0x7ff8e8121cc0 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.942+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff8e8121cc0 0x7ff8e8122100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.942+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e8121cc0 2024-02-18T00:39:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.942+0000 7ff8e77fe700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff8e8121cc0 0x7ff8e8122100 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.943+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e8129220 con 0x7ff8e8121cc0 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.943+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.943+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 11 = system:0 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.943+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8119550 osd=5 4 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.943+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 5 11 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 12 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 12 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8119550 osd=5 3 2024-02-18T00:39:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 5 12 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8119550 osd=5 4 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 12 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.958+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e8126ac0 con 0x7ff8e8121cc0 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e8126ac0 con 0x7ff8e8121cc0 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8119550 osd=5 4 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 5 12 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 13 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff8e8127140 0x7ff8e8127580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 13 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 6 13 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8126ac0 osd=6 4 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 13 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e8129180 con 0x7ff8e8127140 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.960+0000 7ff8ece86700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff8e8127140 0x7ff8e8127580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.961+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e8127140 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.961+0000 7ff8ece86700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff8e8127140 0x7ff8e8127580 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.961+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e8129180 con 0x7ff8e8127140 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.961+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.961+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 13 = system:0 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.961+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8126ac0 osd=6 4 2024-02-18T00:39:33.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.961+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 6 13 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 14 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 14 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8126ac0 osd=6 3 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 6 14 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8126ac0 osd=6 4 2024-02-18T00:39:33.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 14 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.977+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e8129180 con 0x7ff8e8127140 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.978+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e8129180 con 0x7ff8e8127140 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.978+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8126ac0 osd=6 4 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 6 14 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 15 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6812/53025171 conn(0x7ff8e812b8e0 0x7ff8e812bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 15 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 7 15 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8129180 osd=7 4 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 15 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e812c230 con 0x7ff8e812b8e0 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6812/53025171 conn(0x7ff8e812b8e0 0x7ff8e812bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8e7fff700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6812/53025171 conn(0x7ff8e812b8e0 0x7ff8e812bd40 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.979+0000 7ff8e57fa700 10 client.41225.objecter ms_handle_connect 0x7ff8e812b8e0 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.980+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff8e812c230 con 0x7ff8e812b8e0 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.980+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.980+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 15 = system:0 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.980+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8129180 osd=7 4 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.980+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 7 15 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 10 client.41225.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign -1 16 2024-02-18T00:39:33.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 20 client.41225.objecter _get_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_remove -1 16 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 20 client.41225.objecter get_session s=0x7ff8e8129180 osd=7 3 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 15 client.41225.objecter _session_command_op_assign 7 16 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8129180 osd=7 4 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 10 client.41225.objecter _send_command 16 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.995+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff8e812c230 con 0x7ff8e812b8e0 2024-02-18T00:39:33.267 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:39:33.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.996+0000 7ff8e57fa700 1 -- 172.21.15.59:0/4206166739 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff8e812c230 con 0x7ff8e812b8e0 2024-02-18T00:39:33.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.996+0000 7ff8e57fa700 10 client.41225.objecter ms_dispatch 0x7ff8e8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:33.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.996+0000 7ff8e57fa700 10 client.41225.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8e57fa700 20 client.41225.objecter put_session s=0x7ff8e8129180 osd=7 4 2024-02-18T00:39:33.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8e57fa700 15 client.41225.objecter _session_command_op_remove 7 16 2024-02-18T00:39:33.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 librados: watch_flush enter 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 librados: watch_flush exit 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.0 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8e80fee60 msgr2=0x7ff8e80ff240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff8e80fee60 0x7ff8e80ff240 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81096c0 osd=0 1 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.1 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8e8028540 msgr2=0x7ff8e8118ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6805/26279110 conn(0x7ff8e8028540 0x7ff8e8118ef0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80fd4e0 osd=1 1 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.2 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff8e80594d0 msgr2=0x7ff8e811c570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff8e80594d0 0x7ff8e811c570 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8059360 osd=2 1 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.3 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff8e811a0c0 msgr2=0x7ff8e811fbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ece86700 1 -- 172.21.15.59:0/4206166739 reap_dead start 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff8e811a0c0 0x7ff8e811fbf0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e80278e0 osd=3 1 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.4 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff8e811d740 msgr2=0x7ff8e8121260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff8e811d740 0x7ff8e8121260 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e81193e0 osd=4 1 2024-02-18T00:39:33.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.5 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff8e8121cc0 msgr2=0x7ff8e8122100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff8e8121cc0 0x7ff8e8122100 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8119550 osd=5 1 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.6 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff8e8127140 msgr2=0x7ff8e8127580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff8e8127140 0x7ff8e8127580 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8126ac0 osd=6 1 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter close_session for osd.7 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6812/53025171 conn(0x7ff8e812b8e0 msgr2=0x7ff8e812bd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.137:6812/53025171 conn(0x7ff8e812b8e0 0x7ff8e812bd40 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter put_session s=0x7ff8e8129180 osd=7 1 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 20 client.41225.objecter shutdown clearing up homeless session... 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 10 client.41225.objecter successfully canceled tick 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ece86700 1 -- 172.21.15.59:0/4206166739 reap_dead start 2024-02-18T00:39:33.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff8d0062c90 msgr2=0x7ff8d0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff8d0062c90 0x7ff8d0065150 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 rx=0x7ff8e8109980 tx=0x7ff8dc00b040).stop 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80f8420 msgr2=0x7ff8e810e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80f8420 0x7ff8e810e040 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 rx=0x7ff8d8009eb0 tx=0x7ff8d800b040).stop 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 shutdown_connections 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e80f8420 0x7ff8e810e040 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 --2- 172.21.15.59:0/4206166739 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff8d0062c90 0x7ff8d0065150 unknown :-1 s=CLOSED pgs=3124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 >> 172.21.15.59:0/4206166739 conn(0x7ff8e80f3800 msgr2=0x7ff8e80fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 shutdown_connections 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 -- 172.21.15.59:0/4206166739 wait complete. 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:29.997+0000 7ff8ee8e9700 1 librados: shutdown 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.272 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.273 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:39:33.493 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:31.547309+0000 osd.1 (osd.1) 237 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:33.493 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:31.547501+0000 osd.1 (osd.1) 238 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:33.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:31.920504+0000 mgr.y (mgr.24454) 434 : cluster [DBG] pgmap v695: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 530 B/s wr, 3 op/s 2024-02-18T00:39:33.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:32.156893+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e462: 8 total, 8 up, 7 in 2024-02-18T00:39:33.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:31.547309+0000 osd.1 (osd.1) 237 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:33.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:31.547501+0000 osd.1 (osd.1) 238 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:33.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:31.920504+0000 mgr.y (mgr.24454) 434 : cluster [DBG] pgmap v695: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 530 B/s wr, 3 op/s 2024-02-18T00:39:33.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:32.156893+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e462: 8 total, 8 up, 7 in 2024-02-18T00:39:33.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:31.547309+0000 osd.1 (osd.1) 237 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:31.547501+0000 osd.1 (osd.1) 238 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:31.920504+0000 mgr.y (mgr.24454) 434 : cluster [DBG] pgmap v695: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 530 B/s wr, 3 op/s 2024-02-18T00:39:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:32.156893+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e462: 8 total, 8 up, 7 in 2024-02-18T00:39:33.516 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-18T00:39:33.540 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:33.543 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:33.544 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:33.645 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:33.678 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:33.704 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:33.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:33.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:34.083 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:34.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:34.185 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:34.218 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:34.239 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:34.242 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:34.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:34.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:34.375 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 -- 172.21.15.59:0/545722411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80fcbb0 msgr2=0x7fc9a80fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 --2- 172.21.15.59:0/545722411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80fcbb0 0x7fc9a80fcf90 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 rx=0x7fc998004870 tx=0x7fc998009cb0).stop 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 -- 172.21.15.59:0/545722411 shutdown_connections 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 --2- 172.21.15.59:0/545722411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80f7400 0x7fc9a8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 --2- 172.21.15.59:0/545722411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80fcbb0 0x7fc9a80fcf90 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 --2- 172.21.15.59:0/545722411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a80fd510 0x7fc9a80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 -- 172.21.15.59:0/545722411 >> 172.21.15.59:0/545722411 conn(0x7fc9a80f2820 msgr2=0x7fc9a80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 -- 172.21.15.59:0/545722411 shutdown_connections 2024-02-18T00:39:34.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.824+0000 7fc9b07da700 1 -- 172.21.15.59:0/545722411 wait complete. 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 librados: starting msgr at 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 librados: starting objecter 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 Processor -- start 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 -- start start 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 librados: setting wanted keys 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 librados: calling monclient init 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80f7400 0x7fc9a8110350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a80fcbb0 0x7fc9a8110840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80fd510 0x7fc9a810a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a810a9e0 con 0x7fc9a80fd510 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a810ab50 con 0x7fc9a80fcbb0 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9b07da700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a810ad10 con 0x7fc9a80f7400 2024-02-18T00:39:34.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9ae576700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80f7400 0x7fc9a8110350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9aed77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80fd510 0x7fc9a810a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9ae576700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80f7400 0x7fc9a8110350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:58680/0 (socket says 172.21.15.59:58680) 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9ae576700 1 -- 172.21.15.59:0/1371258947 learned_addr learned my addr 172.21.15.59:0/1371258947 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9add75700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a80fcbb0 0x7fc9a8110840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9ae576700 1 -- 172.21.15.59:0/1371258947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a80fcbb0 msgr2=0x7fc9a8110840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.825+0000 7fc9ae576700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a80fcbb0 0x7fc9a8110840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9ae576700 1 -- 172.21.15.59:0/1371258947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80fd510 msgr2=0x7fc9a810a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9ae576700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80fd510 0x7fc9a810a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9ae576700 1 -- 172.21.15.59:0/1371258947 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a00097c0 con 0x7fc9a80f7400 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9add75700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a80fcbb0 0x7fc9a8110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9ae576700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80f7400 0x7fc9a8110350 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 rx=0x7fc9980043a0 tx=0x7fc998009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc99f7fe700 10 client.?.objecter ms_handle_connect 0x7fc9a80f7400 2024-02-18T00:39:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc99f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc99f7fe700 1 -- 172.21.15.59:0/1371258947 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800e030 con 0x7fc9a80f7400 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc99f7fe700 1 -- 172.21.15.59:0/1371258947 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc998013da0 con 0x7fc9a80f7400 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc998005040 con 0x7fc9a80f7400 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc99f7fe700 1 -- 172.21.15.59:0/1371258947 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998018070 con 0x7fc9a80f7400 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9b07da700 10 client.41231.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a810b1b0 con 0x7fc9a80f7400 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.826+0000 7fc9b07da700 1 librados: init done 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a810b3a0 con 0x7fc9a80f7400 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc99f7fe700 1 -- 172.21.15.59:0/1371258947 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc99801d420 con 0x7fc9a80f7400 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc99f7fe700 1 --2- 172.21.15.59:0/1371258947 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc994062c90 0x7fc994065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:34.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc99f7fe700 1 -- 172.21.15.59:0/1371258947 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(463..463 src has 180..463) v4 ==== 8035+0+0 (secure 0 0 0) 0x7fc99807e460 con 0x7fc9a80f7400 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc99f7fe700 10 client.41231.objecter ms_dispatch 0x7fc9a8001550 osd_map(463..463 src has 180..463) v4 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc99f7fe700 3 client.41231.objecter handle_osd_map got epochs [463,463] > 0 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc99f7fe700 3 client.41231.objecter handle_osd_map decoding full epoch 463 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.827+0000 7fc9add75700 1 --2- 172.21.15.59:0/1371258947 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc994062c90 0x7fc994065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.828+0000 7fc99f7fe700 20 client.41231.objecter dump_active .. 0 homeless 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.828+0000 7fc99f7fe700 10 client.41231.objecter ms_handle_connect 0x7fc994062c90 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.828+0000 7fc9add75700 1 --2- 172.21.15.59:0/1371258947 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc994062c90 0x7fc994065150 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 rx=0x7fc9a810bc90 tx=0x7fc9a4009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:34.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:34.832+0000 7fc99f7fe700 1 -- 172.21.15.59:0/1371258947 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc998051a90 con 0x7fc9a80f7400 2024-02-18T00:39:34.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:33.695861+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e463: 8 total, 8 up, 7 in 2024-02-18T00:39:34.949 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:33.920942+0000 mgr.y (mgr.24454) 435 : cluster [DBG] pgmap v698: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 531 B/s wr, 3 op/s 2024-02-18T00:39:34.949 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:33.695861+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e463: 8 total, 8 up, 7 in 2024-02-18T00:39:34.949 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:33.920942+0000 mgr.y (mgr.24454) 435 : cluster [DBG] pgmap v698: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 531 B/s wr, 3 op/s 2024-02-18T00:39:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:33.695861+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e463: 8 total, 8 up, 7 in 2024-02-18T00:39:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:33.920942+0000 mgr.y (mgr.24454) 435 : cluster [DBG] pgmap v698: 238 pgs: 238 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 531 B/s wr, 3 op/s 2024-02-18T00:39:35.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.015+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd in", "ids": ["7"]} v 0) v1 -- 0x7fc9a805fa60 con 0x7fc9a80f7400 2024-02-18T00:39:35.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:39:35.016+0000 7f6151a5e700 -1 mon.a@0(leader).osd e463 definitely_dead 0 2024-02-18T00:39:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.718+0000 7fc99f7fe700 1 -- 172.21.15.59:0/1371258947 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["7"]}]=0 marked in osd.7. v464) v1 ==== 85+0+0 (secure 0 0 0) 0x7fc9a805fa60 con 0x7fc9a80f7400 2024-02-18T00:39:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.722+0000 7fc9b07da700 10 librados: watch_flush enter 2024-02-18T00:39:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.734+0000 7fc9b07da700 10 librados: watch_flush exit 2024-02-18T00:39:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.734+0000 7fc9b07da700 20 client.41231.objecter shutdown clearing up homeless session... 2024-02-18T00:39:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.734+0000 7fc9b07da700 10 client.41231.objecter successfully canceled tick 2024-02-18T00:39:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.738+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc994062c90 msgr2=0x7fc994065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.738+0000 7fc9b07da700 1 --2- 172.21.15.59:0/1371258947 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc994062c90 0x7fc994065150 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 rx=0x7fc9a810bc90 tx=0x7fc9a4009040).stop 2024-02-18T00:39:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.738+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80f7400 msgr2=0x7fc9a8110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.738+0000 7fc9b07da700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80f7400 0x7fc9a8110350 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 rx=0x7fc9980043a0 tx=0x7fc998009cb0).stop 2024-02-18T00:39:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.739+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 shutdown_connections 2024-02-18T00:39:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.739+0000 7fc9b07da700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9a80fd510 0x7fc9a810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:35.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.739+0000 7fc9b07da700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9a80f7400 0x7fc9a8110350 secure :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 rx=0x7fc9980043a0 tx=0x7fc998009cb0).stop 2024-02-18T00:39:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.739+0000 7fc9b07da700 1 --2- 172.21.15.59:0/1371258947 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc994062c90 0x7fc994065150 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.739+0000 7fc9b07da700 1 --2- 172.21.15.59:0/1371258947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a80fcbb0 0x7fc9a8110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:35.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.739+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 >> 172.21.15.59:0/1371258947 conn(0x7fc9a80f2820 msgr2=0x7fc9a80f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.741+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 shutdown_connections 2024-02-18T00:39:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.741+0000 7fc9b07da700 1 -- 172.21.15.59:0/1371258947 wait complete. 2024-02-18T00:39:35.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:35.741+0000 7fc9b07da700 1 librados: shutdown 2024-02-18T00:39:35.749 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:35.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:35.753 INFO:teuthology.orchestra.run.smithi059.stderr:marked in osd.7. 2024-02-18T00:39:35.851 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:35.884 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:35.904 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:35.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:35.907 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:36.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:35.017215+0000 mon.c (mon.2) 442 : audit [DBG] from='client.? 172.21.15.59:0/1371258947' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-18T00:39:36.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:35.017510+0000 mon.a (mon.0) 2620 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-18T00:39:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:35.017215+0000 mon.c (mon.2) 442 : audit [DBG] from='client.? 172.21.15.59:0/1371258947' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-18T00:39:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:35.017510+0000 mon.a (mon.0) 2620 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-18T00:39:36.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:35.017215+0000 mon.c (mon.2) 442 : audit [DBG] from='client.? 172.21.15.59:0/1371258947' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-18T00:39:36.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:35.017510+0000 mon.a (mon.0) 2620 : audit [DBG] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["7"]}]: dispatch 2024-02-18T00:39:36.008 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:36.058 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:36.061 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:36.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:36.163 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:36.192 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:36.212 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:36.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:36.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:36.317 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:36.346 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:36.593 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:36.596 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:36.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:36.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:36.738 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:37.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:35.712569+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-02-18T00:39:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:35.712662+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-18T00:39:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:35.922160+0000 mgr.y (mgr.24454) 436 : cluster [DBG] pgmap v700: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:37.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:35.712569+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-02-18T00:39:37.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:35.712662+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-18T00:39:37.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:35.922160+0000 mgr.y (mgr.24454) 436 : cluster [DBG] pgmap v700: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:37.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:35.712569+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["7"]}]': finished 2024-02-18T00:39:37.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:35.712662+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-18T00:39:37.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:35.922160+0000 mgr.y (mgr.24454) 436 : cluster [DBG] pgmap v700: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:37.201 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:37.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:37.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:37.305 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:37.333 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:37.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:39:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:37.463 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:37.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:37.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:37.594 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:37.721 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:37.729 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:37.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:37.830 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:37.856 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:37.875 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:37.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:37.878 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:37.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:36.716636+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-18T00:39:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:37.715889+0000 mon.a (mon.0) 2624 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-18T00:39:38.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:36.716636+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-18T00:39:38.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:37.715889+0000 mon.a (mon.0) 2624 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-18T00:39:38.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:36.716636+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-18T00:39:38.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:37.715889+0000 mon.a (mon.0) 2624 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-18T00:39:38.007 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:38.106 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:38.111 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:38.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:38.112 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-02-18T00:39:38.212 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:38.239 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 -- 172.21.15.59:0/913558811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fe510 msgr2=0x7fb3740f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 --2- 172.21.15.59:0/913558811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fe510 0x7fb3740f7d40 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 rx=0x7fb364004870 tx=0x7fb364009cb0).stop 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 -- 172.21.15.59:0/913558811 shutdown_connections 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 --2- 172.21.15.59:0/913558811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740fdbb0 0x7fb3740fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 --2- 172.21.15.59:0/913558811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740f8420 0x7fb374104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 --2- 172.21.15.59:0/913558811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fe510 0x7fb3740f7d40 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 -- 172.21.15.59:0/913558811 >> 172.21.15.59:0/913558811 conn(0x7fb3740f3800 msgr2=0x7fb3740f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 -- 172.21.15.59:0/913558811 shutdown_connections 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 -- 172.21.15.59:0/913558811 wait complete. 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 librados: starting msgr at 2024-02-18T00:39:38.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.829+0000 7fb37a119700 1 librados: starting objecter 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 Processor -- start 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 -- start start 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 librados: setting wanted keys 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 librados: calling monclient init 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740f8420 0x7fb37410e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fdbb0 0x7fb37410e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740fe510 0x7fb3741081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3741086e0 con 0x7fb3740fdbb0 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb374108850 con 0x7fb3740f8420 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb37a119700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb374108a10 con 0x7fb3740fe510 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb373fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740fe510 0x7fb3741081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb3737fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740f8420 0x7fb37410e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb372ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fdbb0 0x7fb37410e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb3737fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740f8420 0x7fb37410e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:51558/0 (socket says 172.21.15.59:51558) 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb373fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740fe510 0x7fb3741081f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:58700/0 (socket says 172.21.15.59:58700) 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.830+0000 7fb3737fe700 1 -- 172.21.15.59:0/946029276 learned_addr learned my addr 172.21.15.59:0/946029276 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb3737fe700 1 -- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740fe510 msgr2=0x7fb3741081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb3737fe700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740fe510 0x7fb3741081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb3737fe700 1 -- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fdbb0 msgr2=0x7fb37410e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb3737fe700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fdbb0 0x7fb37410e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb3737fe700 1 -- 172.21.15.59:0/946029276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c0097c0 con 0x7fb3740f8420 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb372ffd700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fdbb0 0x7fb37410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb373fff700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740fe510 0x7fb3741081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:38.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb3737fe700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740f8420 0x7fb37410e050 secure :-1 s=READY pgs=4063 cs=0 l=1 rev1=1 rx=0x7fb36800e970 tx=0x7fb36800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb370ff9700 10 client.?.objecter ms_handle_connect 0x7fb3740f8420 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb370ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb370ff9700 1 -- 172.21.15.59:0/946029276 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368004660 con 0x7fb3740f8420 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb364005040 con 0x7fb3740f8420 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb370ff9700 1 -- 172.21.15.59:0/946029276 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb36800f040 con 0x7fb3740f8420 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb370ff9700 1 -- 172.21.15.59:0/946029276 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368014790 con 0x7fb3740f8420 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb37a119700 10 client.43705.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb374114e80 con 0x7fb3740f8420 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.831+0000 7fb37a119700 1 librados: init done 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.832+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37405ee80 con 0x7fb3740f8420 2024-02-18T00:39:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.832+0000 7fb370ff9700 1 -- 172.21.15.59:0/946029276 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb368014bc0 con 0x7fb3740f8420 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb370ff9700 1 --2- 172.21.15.59:0/946029276 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb35c062c90 0x7fb35c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb370ff9700 1 -- 172.21.15.59:0/946029276 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(466..466 src has 180..466) v4 ==== 8126+0+0 (secure 0 0 0) 0x7fb36807cfe0 con 0x7fb3740f8420 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb372ffd700 1 --2- 172.21.15.59:0/946029276 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb35c062c90 0x7fb35c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb370ff9700 10 client.43705.objecter ms_dispatch 0x7fb374001550 osd_map(466..466 src has 180..466) v4 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb370ff9700 3 client.43705.objecter handle_osd_map got epochs [466,466] > 0 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb370ff9700 3 client.43705.objecter handle_osd_map decoding full epoch 466 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb370ff9700 20 client.43705.objecter dump_active .. 0 homeless 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb370ff9700 10 client.43705.objecter ms_handle_connect 0x7fb35c062c90 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.833+0000 7fb372ffd700 1 --2- 172.21.15.59:0/946029276 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb35c062c90 0x7fb35c065150 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 rx=0x7fb364004ee0 tx=0x7fb36400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:37.837+0000 7fb370ff9700 1 -- 172.21.15.59:0/946029276 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb368050540 con 0x7fb3740f8420 2024-02-18T00:39:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.018+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fb3741070a0 con 0x7fb3740f8420 2024-02-18T00:39:38.256 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:38.257 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:38.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:38.261 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:38.292 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:38.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:38.389 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:38.590 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:38.595 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:38.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:38.696 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:38.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.702+0000 7fb370ff9700 1 -- 172.21.15.59:0/946029276 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v467) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb368019030 con 0x7fb3740f8420 2024-02-18T00:39:38.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.705+0000 7fb37a119700 10 librados: watch_flush enter 2024-02-18T00:39:38.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.708+0000 7fb37a119700 10 librados: watch_flush exit 2024-02-18T00:39:38.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.708+0000 7fb37a119700 20 client.43705.objecter shutdown clearing up homeless session... 2024-02-18T00:39:38.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.708+0000 7fb37a119700 10 client.43705.objecter successfully canceled tick 2024-02-18T00:39:38.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb35c062c90 msgr2=0x7fb35c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:38.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 --2- 172.21.15.59:0/946029276 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb35c062c90 0x7fb35c065150 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 rx=0x7fb364004ee0 tx=0x7fb36400b040).stop 2024-02-18T00:39:38.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740f8420 msgr2=0x7fb37410e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:38.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740f8420 0x7fb37410e050 secure :-1 s=READY pgs=4063 cs=0 l=1 rev1=1 rx=0x7fb36800e970 tx=0x7fb36800b040).stop 2024-02-18T00:39:38.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 shutdown_connections 2024-02-18T00:39:38.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 --2- 172.21.15.59:0/946029276 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb35c062c90 0x7fb35c065150 unknown :-1 s=CLOSED pgs=3126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3740f8420 0x7fb37410e050 unknown :-1 s=CLOSED pgs=4063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3740fe510 0x7fb3741081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 --2- 172.21.15.59:0/946029276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3740fdbb0 0x7fb37410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:38.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 >> 172.21.15.59:0/946029276 conn(0x7fb3740f3800 msgr2=0x7fb3740fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:38.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 shutdown_connections 2024-02-18T00:39:38.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 -- 172.21.15.59:0/946029276 wait complete. 2024-02-18T00:39:38.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:38.709+0000 7fb37a119700 1 librados: shutdown 2024-02-18T00:39:38.712 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:39:38.725 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:38.745 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:38.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:38.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:38.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:38.874 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:38.892 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:38.894 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:38.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:38.995 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:38.997 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:37.922517+0000 mgr.y (mgr.24454) 437 : cluster [DBG] pgmap v703: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:38.997 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:38.019642+0000 mon.b (mon.1) 368 : audit [INF] from='client.? 172.21.15.59:0/946029276' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:38.997 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:38.019822+0000 mon.a (mon.0) 2625 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:38.997 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:38.693217+0000 mon.a (mon.0) 2626 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:38.997 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:38.696661+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:38.997 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:38.696788+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-18T00:39:38.997 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:37.922517+0000 mgr.y (mgr.24454) 437 : cluster [DBG] pgmap v703: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:38.998 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:38.019642+0000 mon.b (mon.1) 368 : audit [INF] from='client.? 172.21.15.59:0/946029276' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:38.998 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:38.019822+0000 mon.a (mon.0) 2625 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:38.998 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:38.693217+0000 mon.a (mon.0) 2626 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:38.998 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:38.696661+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:38.998 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:38.696788+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-18T00:39:39.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:37.922517+0000 mgr.y (mgr.24454) 437 : cluster [DBG] pgmap v703: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:39.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:38.019642+0000 mon.b (mon.1) 368 : audit [INF] from='client.? 172.21.15.59:0/946029276' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:39.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:38.019822+0000 mon.a (mon.0) 2625 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:39.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:38.693217+0000 mon.a (mon.0) 2626 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:39.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:38.696661+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:39.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:38.696788+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-18T00:39:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:39.040 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:39.042 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:39.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:39.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:39.212 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:39.215 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:39.215 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:39.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:39.345 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:40.194 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:40.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:40.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:40.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:40.325 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:40.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:40.780 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:40.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:40.881 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:40.909 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:41.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:39.699062+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-18T00:39:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:39.902820+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:39.922881+0000 mgr.y (mgr.24454) 438 : cluster [DBG] pgmap v706: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:39.699062+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-18T00:39:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:39.902820+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:39.922881+0000 mgr.y (mgr.24454) 438 : cluster [DBG] pgmap v706: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:41.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:39.699062+0000 mon.a (mon.0) 2629 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-18T00:39:41.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:39.902820+0000 mon.a (mon.0) 2630 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:41.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:39.922881+0000 mgr.y (mgr.24454) 438 : cluster [DBG] pgmap v706: 238 pgs: 99 peering, 139 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:41.359 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:41.362 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:41.362 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:41.463 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:41.494 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:39:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:39:41.494 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:39:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:39:41.496 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:41.520 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:41.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:41.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:41.650 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:41.676 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:41.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:41.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:41.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:40.760409+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:41.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:40.760714+0000 mon.a (mon.0) 2631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:41.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:40.761157+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-69"}]: dispatch 2024-02-18T00:39:41.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:40.761395+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-69"}]: dispatch 2024-02-18T00:39:41.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:41.307763+0000 mgr.y (mgr.24454) 439 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:41.782 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:41.812 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:40.760409+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:40.760714+0000 mon.a (mon.0) 2631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:40.761157+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-69"}]: dispatch 2024-02-18T00:39:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:40.761395+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-69"}]: dispatch 2024-02-18T00:39:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:41.307763+0000 mgr.y (mgr.24454) 439 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:40.760409+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:40.760714+0000 mon.a (mon.0) 2631 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:42.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:40.761157+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-69"}]: dispatch 2024-02-18T00:39:42.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:40.761395+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-69"}]: dispatch 2024-02-18T00:39:42.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:41.307763+0000 mgr.y (mgr.24454) 439 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:42.361 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:42.364 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:42] "GET /metrics HTTP/1.1" 200 189889 "" "Prometheus/2.33.4" 2024-02-18T00:39:42.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:42.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:42.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:42.496 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:42.784 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:39:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:39:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:39:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:39:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:975 17.f 17:f7837858:test-rados-api-smithi059-82949-15::foo:head [setxattr a463 in=4b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:39:42.784 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.393216+0000", 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 399.67556499699998, 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.085327403999999996, 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393216+0000", 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393216+0000", 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393215+0000", 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5339999999999999e-06 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393221+0000", 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7600000000000008e-06 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393229+0000", 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0729999999999998e-06 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393234+0000", 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9961999999999999e-05 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393264+0000", 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9594e-05 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393313+0000", 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1745999999999999e-05 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393405+0000", 2024-02-18T00:39:42.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001648465 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.395053+0000", 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013003799999999999 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.395183+0000", 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083269832000000002 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478453+0000", 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0034999999999998e-05 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478503+0000", 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0154999999999998e-05 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478543+0000", 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.085327403999999996 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24847.0:2 23.c 23:3096c917:test-rados-api-smithi059-83005-2::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.472340+0000", 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 399.59644092899998, 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.074309046000000004, 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24847", 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1733075084", 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.791 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472340+0000", 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472340+0000", 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472339+0000", 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.401e-06 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472346+0000", 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2798e-05 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472359+0000", 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1589999999999993e-06 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472367+0000", 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0065191090000000004 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478886+0000", 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5880000000000003e-06 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478893+0000", 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055446065000000003 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534339+0000", 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120016 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534459+0000", 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014479200000000001 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534603+0000", 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053728430000000004 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.539976+0000", 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045922500000000001 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.540436+0000", 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061339020000000001 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.546569+0000", 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3572000000000002e-05 2024-02-18T00:39:42.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.546613+0000", 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6047000000000002e-05 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.546649+0000", 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.074309046000000004 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.797 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:977 17.f 17:f7837858:test-rados-api-smithi059-82949-15::foo:head [setxattr a464 in=4b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.482200+0000", 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 399.58658101700001, 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051936980000000001, 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482200+0000", 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482200+0000", 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482198+0000", 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5739999999999998e-06 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482206+0000", 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0079e-05 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482216+0000", 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2129999999999996e-06 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482222+0000", 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2865999999999999e-05 2024-02-18T00:39:42.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482265+0000", 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7281999999999995e-05 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482362+0000", 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015163199999999999 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:39:42.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482514+0000", 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00288076 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.485395+0000", 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025177637999999999 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.510572+0000", 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023472665 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534045+0000", 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8717000000000001e-05 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534094+0000", 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3095e-05 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534137+0000", 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051936980000000001 2024-02-18T00:39:42.803 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:1921 17.f 17:f7837858:test-rados-api-smithi059-82949-15::foo:head [setxattr a936 in=4b] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.441237+0000", 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 398.62754370099998, 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044349531999999997, 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:39:42.804 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1921 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441237+0000", 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441237+0000", 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:42.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441236+0000", 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1530000000000003e-06 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441245+0000", 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.575e-05 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441261+0000", 2024-02-18T00:39:42.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4599999999999997e-06 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441268+0000", 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2765999999999997e-05 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441311+0000", 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9658000000000001e-05 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441401+0000", 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000165329 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441566+0000", 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0079216749999999996 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.449488+0000", 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010947795999999999 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.460436+0000", 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025060289999999999 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.485496+0000", 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1431000000000003e-05 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.485547+0000", 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9501e-05 2024-02-18T00:39:42.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.485587+0000", 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044349531999999997 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2059 17.11 e108/104 17:8d70d4b8:test-rados-api-smithi059-82949-16::big:head v 108'2, mlcod=0'0)", 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.674229+0000", 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 398.39455201999999, 2024-02-18T00:39:42.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062844539000000005, 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.674229+0000", 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.674229+0000", 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.674227+0000", 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016560637 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690788+0000", 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.747e-05 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690815+0000", 2024-02-18T00:39:42.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.164e-05 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690827+0000", 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4209999999999998e-05 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690881+0000", 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3047999999999999e-05 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690914+0000", 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046112126000000003 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.737026+0000", 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7259999999999998e-05 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.737073+0000", 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062844539000000005 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.814 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25025.0:6 22.2 22:47a9abb2:::36.obj2:head [write 0~171] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.068256+0000", 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 398.00052500100003, 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.035616041000000001, 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25025", 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6 2024-02-18T00:39:42.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068256+0000", 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068256+0000", 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068255+0000", 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9129999999999997e-06 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068262+0000", 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0930999999999999e-05 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068273+0000", 2024-02-18T00:39:42.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6259999999999997e-06 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068279+0000", 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0180999999999999e-05 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068330+0000", 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0109000000000002e-05 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068410+0000", 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000111469 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4,7", 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068521+0000", 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00047116299999999997 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068992+0000", 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034599035 2024-02-18T00:39:42.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103591+0000", 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000171175 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103762+0000", 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640999999999999e-05 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103820+0000", 2024-02-18T00:39:42.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1907999999999998e-05 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103872+0000", 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.035616041000000001 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:2074 17.1c 17:38a558b5:test-rados-api-smithi059-82949-17::big3:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.227584+0000", 2024-02-18T00:39:42.821 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 397.84119657999997, 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.094293116999999996, 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2074 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.227584+0000", 2024-02-18T00:39:42.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.227584+0000", 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.227583+0000", 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021037292999999999 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248620+0000", 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4719999999999999e-05 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248665+0000", 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7345000000000001e-05 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248682+0000", 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3851999999999997e-05 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248756+0000", 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000160879 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248917+0000", 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017516499999999999 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,6", 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.249092+0000", 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057748466999999998 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.306841+0000", 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003428871 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.310269+0000", 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011530294999999999 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.321800+0000", 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3850000000000003e-05 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.321834+0000", 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3900000000000003e-05 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.321878+0000", 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.094293116999999996 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(23.a epoch 109/105 rep_tid 112 entries 109'4 (0'0) error 23:5074b981:test-rados-api-smithi059-83005-4::foo:head by client.24847.0:15 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 108'2)", 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 397.48544587700002, 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071963654000000002, 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6400000000000003e-06 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583343+0000", 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1528999999999999e-05 2024-02-18T00:39:42.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583355+0000", 2024-02-18T00:39:42.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1840000000000005e-06 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583364+0000", 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016310387999999999 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.599675+0000", 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055624172 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.655299+0000", 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071963654000000002 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25073.0:2 37.1a 37:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e110)", 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:05.519257+0000", 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 396.54952397099999, 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9908002250000001, 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.833 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25073", 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1549676353", 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519257+0000", 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519257+0000", 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519255+0000", 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519264+0000", 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1605e-05 2024-02-18T00:39:42.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519275+0000", 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1300000000000001e-06 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519284+0000", 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033962747000000001 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.836 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.553246+0000", 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8060000000000001e-06 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.553255+0000", 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.947313686 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.500569+0000", 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011259100000000001 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.500681+0000", 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014137700000000001 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,4", 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.500823+0000", 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039772840000000002 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.504800+0000", 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000815236 2024-02-18T00:39:42.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.505615+0000", 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043505820000000004 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.509966+0000", 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0556000000000002e-05 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.510016+0000", 2024-02-18T00:39:42.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0825000000000003e-05 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.510057+0000", 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9908002250000001 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.3.0:334 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.683941+0000", 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 367.384839597, 2024-02-18T00:39:42.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048476497, 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.683941+0000", 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.683941+0000", 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.683939+0000", 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013830864999999999 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.697770+0000", 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6985e-05 2024-02-18T00:39:42.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.697797+0000", 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0740000000000001e-05 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.697808+0000", 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00057474900000000005 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.698383+0000", 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3714e-05 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.698406+0000", 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033983909 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.732390+0000", 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7523000000000001e-05 2024-02-18T00:39:42.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.732418+0000", 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048476497 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.3.0:335 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.721154+0000", 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 367.34762662399999, 2024-02-18T00:39:42.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064423909000000001, 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.721154+0000", 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.721154+0000", 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999914 2024-02-18T00:39:42.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.721146+0000", 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028492059 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749638+0000", 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7851000000000002e-05 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749666+0000", 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.223e-05 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749678+0000", 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0841000000000003e-05 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749729+0000", 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2019999999999999e-05 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749751+0000", 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.035739964999999999 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.785491+0000", 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7519e-05 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.785578+0000", 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064423909000000001 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.849 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25220.0:573 62.8 62:16784af7:test-rados-api-smithi059-82957-14::foo0:head [writefull 0~11] snapc 5=[5,4] ondisk+write+known_if_redirected e157)", 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:55.580624+0000", 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 346.48815667100001, 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039479232000000003, 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25220", 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 573 2024-02-18T00:39:42.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580624+0000", 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580624+0000", 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.851 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580621+0000", 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7116999999999999e-05 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580638+0000", 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6254999999999998e-05 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580665+0000", 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0737000000000001e-05 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580675+0000", 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7051999999999994e-05 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580742+0000", 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6139999999999992e-06 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580751+0000", 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022686853 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.603438+0000", 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6570000000000005e-05 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.603535+0000", 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000201819 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.603736+0000", 2024-02-18T00:39:42.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022144249999999999 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.605951+0000", 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012419968999999999 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.618371+0000", 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016625450000000001 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.620033+0000", 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0306999999999999e-05 2024-02-18T00:39:42.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.620074+0000", 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9947e-05 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.620104+0000", 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039479232000000003 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "MOSDPGPushReply(26.1b 161/160 [PushReplyOp(26:d83876eb:::4:head)])", 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:59.978530+0000", 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 342.09025077799998, 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089631667999999998, 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978530+0000", 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978530+0000", 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978529+0000", 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6989999999999997e-06 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978536+0000", 2024-02-18T00:39:42.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0166000000000001e-05 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978546+0000", 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.074e-06 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978551+0000", 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089439904000000001 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:00.067991+0000", 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000170688 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:00.068162+0000", 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089631667999999998 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24862.0:1065 26.d 26:b0000000::::head [pgnls start_epoch 173] snapc 0=[] RETRY=1 ondisk+retry+read+ignore_overlay+known_if_redirected e173)", 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:12.650773+0000", 2024-02-18T00:39:42.860 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 329.418008047, 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96835666399999998, 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24862", 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1312135677", 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650773+0000", 2024-02-18T00:39:42.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650773+0000", 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650771+0000", 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1140000000000006e-06 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650779+0000", 2024-02-18T00:39:42.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012827730000000001 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.652062+0000", 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1474000000000001e-05 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.652073+0000", 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031773589999999998 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.655251+0000", 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2709e-05 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.655264+0000", 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96304593400000005 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.618309+0000", 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6347999999999997e-05 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.618356+0000", 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00077375499999999999 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619130+0000", 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96835666399999998 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.865 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24862.0:1065 26.d 26:b0000000::::head [pgnls start_epoch 173] snapc 0=[] RETRY=2 ondisk+retry+read+ignore_overlay+known_if_redirected e174)", 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:12.659255+0000", 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 329.40952596300002, 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96029274099999995, 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24862", 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1312135677", 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659255+0000", 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659255+0000", 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659252+0000", 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2879e-05 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659265+0000", 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6169000000000001e-05 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659281+0000", 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6491000000000001e-05 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659308+0000", 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9378e-05 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659367+0000", 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1171e-05 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659378+0000", 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.95979067600000001 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619169+0000", 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-02-18T00:39:42.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619207+0000", 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034093200000000001 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619548+0000", 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96029274099999995 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30543.0:2 117.13 117:cbb9903e:::test_obj:head [write-same] snapc 0=[] ondisk+write+known_if_redirected e178)", 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:16.963477+0000", 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 325.10530363399999, 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047047512999999999, 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30543", 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/481047181", 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963477+0000", 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963477+0000", 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963476+0000", 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-02-18T00:39:42.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963484+0000", 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1267000000000001e-05 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963495+0000", 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8210000000000002e-06 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963503+0000", 2024-02-18T00:39:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00052219300000000001 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.964025+0000", 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1019999999999997e-06 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.964033+0000", 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020100052 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984133+0000", 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125695 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984259+0000", 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017865900000000001 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,3", 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984438+0000", 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00053644399999999996 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984974+0000", 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017858202 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.002832+0000", 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074311840000000004 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.010264+0000", 2024-02-18T00:39:42.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5982e-05 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.010320+0000", 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020525200000000001 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.010525+0000", 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047047512999999999 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.0.0:492 62.13 e210/123 62:cdd2dcb4:test-rados-api-smithi059-82957-14::foo49:7 v 210'17, mlcod=201'13)", 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:48.642857+0000", 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 293.425923748, 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038104723, 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642857+0000", 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642857+0000", 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642856+0000", 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9489999999999996e-06 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642864+0000", 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9909999999999997e-06 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642874+0000", 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.199e-06 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642881+0000", 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1041000000000001e-05 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642942+0000", 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3368000000000001e-05 2024-02-18T00:39:42.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642966+0000", 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.037956973999999997 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.680923+0000", 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9112000000000001e-05 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.680962+0000", 2024-02-18T00:39:42.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038104723 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30798.0:1 156.5s0 156:ab97ba35:::nonexistent:head [read 0~128] snapc 0=[] ondisk+read+known_if_redirected e236)", 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:35:14.627697+0000", 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 267.44108420700002, 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.011055861, 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30798", 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/49510408", 2024-02-18T00:39:42.882 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627697+0000", 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627697+0000", 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.883 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627695+0000", 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5110000000000003e-06 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627701+0000", 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6500000000000008e-06 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627711+0000", 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627717+0000", 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0540999999999997e-05 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627757+0000", 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8550000000000002e-06 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627760+0000", 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.010886739 2024-02-18T00:39:42.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:15.638647+0000", 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105617 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:15.638753+0000", 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.011055861 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.43222.0:1 168.6s0 168:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e257)", 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:35:35.628336+0000", 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 246.44044451299999, 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.016274704, 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.43222", 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/360131685", 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.887 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628336+0000", 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628336+0000", 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.888 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628335+0000", 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3909999999999998e-06 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628343+0000", 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8959999999999998e-06 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628353+0000", 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0339999999999997e-06 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628361+0000", 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068555099999999996 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.629047+0000", 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1219999999999999e-06 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.629050+0000", 2024-02-18T00:39:42.890 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0131460990000001 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.642196+0000", 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012572500000000001 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.642322+0000", 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041691800000000001 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.891 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.642738+0000", 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00063511000000000004 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.643374+0000", 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001199117 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.644573+0000", 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8461000000000003e-05 2024-02-18T00:39:42.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.644611+0000", 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.016274704 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31038.0:1 187.2s0 187:5c55be4c:::oid_RoundTripAppendPP_:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:20.660730+0000", 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 201.40805078, 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2238795689999999, 2024-02-18T00:39:42.893 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31038", 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2265555398", 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660730+0000", 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.894 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660730+0000", 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660728+0000", 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4997999999999999e-05 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660743+0000", 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0403e-05 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660753+0000", 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.268e-06 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660760+0000", 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0030131009999999998 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.663773+0000", 2024-02-18T00:39:42.896 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6430000000000001e-06 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.663776+0000", 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2185161870000001 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.882293+0000", 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134227 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.897 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.882427+0000", 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00042525700000000001 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.882852+0000", 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078715700000000003 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.883639+0000", 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00090266100000000002 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.884542+0000", 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7854999999999993e-05 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.884610+0000", 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2238795689999999 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:39:42.899 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:39:43.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:41.705069+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:41.923278+0000 mgr.y (mgr.24454) 440 : cluster [DBG] pgmap v708: 206 pgs: 1 active+recovering+undersized+remapped, 205 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 729 B/s rd, 19 KiB/s wr, 18 op/s; 7/1170 objects misplaced (0.598%); 929 KiB/s, 102 objects/s recovering 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.529994+0000 mon.a (mon.0) 2634 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.530325+0000 mon.a (mon.0) 2635 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 1]}]: dispatch 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.530526+0000 mon.a (mon.0) 2636 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.530678+0000 mon.a (mon.0) 2637 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.530842+0000 mon.a (mon.0) 2638 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.530995+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.8", "id": [7, 5]}]: dispatch 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:41.705069+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-18T00:39:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:41.923278+0000 mgr.y (mgr.24454) 440 : cluster [DBG] pgmap v708: 206 pgs: 1 active+recovering+undersized+remapped, 205 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 729 B/s rd, 19 KiB/s wr, 18 op/s; 7/1170 objects misplaced (0.598%); 929 KiB/s, 102 objects/s recovering 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.529994+0000 mon.a (mon.0) 2634 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.530325+0000 mon.a (mon.0) 2635 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 1]}]: dispatch 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.530526+0000 mon.a (mon.0) 2636 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.530678+0000 mon.a (mon.0) 2637 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.530842+0000 mon.a (mon.0) 2638 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.530995+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.8", "id": [7, 5]}]: dispatch 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:41.705069+0000 mon.a (mon.0) 2633 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:41.923278+0000 mgr.y (mgr.24454) 440 : cluster [DBG] pgmap v708: 206 pgs: 1 active+recovering+undersized+remapped, 205 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 729 B/s rd, 19 KiB/s wr, 18 op/s; 7/1170 objects misplaced (0.598%); 929 KiB/s, 102 objects/s recovering 2024-02-18T00:39:43.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.529994+0000 mon.a (mon.0) 2634 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.530325+0000 mon.a (mon.0) 2635 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 1]}]: dispatch 2024-02-18T00:39:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.530526+0000 mon.a (mon.0) 2636 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.530678+0000 mon.a (mon.0) 2637 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.530842+0000 mon.a (mon.0) 2638 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]: dispatch 2024-02-18T00:39:43.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.530995+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.8", "id": [7, 5]}]: dispatch 2024-02-18T00:39:43.112 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:39:43.113 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:39:43.113 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-02-18T00:39:43.113 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-18T00:39:43.113 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-18T00:39:43.113 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-02-18T00:39:43.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:39:43.385 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:39:43.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-18T00:39:43.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.717124+0000 mon.a (mon.0) 2640 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-02-18T00:39:43.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.717219+0000 mon.a (mon.0) 2641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 1]}]': finished 2024-02-18T00:39:43.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.717282+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-02-18T00:39:43.735 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.717124+0000 mon.a (mon.0) 2640 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-02-18T00:39:43.735 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.717219+0000 mon.a (mon.0) 2641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 1]}]': finished 2024-02-18T00:39:43.735 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.717282+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-02-18T00:39:43.735 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.717344+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-02-18T00:39:43.735 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.717413+0000 mon.a (mon.0) 2644 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.717506+0000 mon.a (mon.0) 2645 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.8", "id": [7, 5]}]': finished 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:42.717575+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.719966+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:42.720541+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:43.465947+0000 osd.5 (osd.5) 197 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:43.466062+0000 osd.5 (osd.5) 198 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:43.700938+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:43.701092+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-18T00:39:43.736 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:43.707236+0000 mon.c (mon.2) 446 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:43.747 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:43.750 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:43.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:43.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.751+0000 7fada67bd700 1 -- 172.21.15.59:0/3506171119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada00f9610 msgr2=0x7fada0101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.751+0000 7fada67bd700 1 --2- 172.21.15.59:0/3506171119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada00f9610 0x7fada0101d20 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 rx=0x7fad940047b0 tx=0x7fad94009cb0).stop 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.752+0000 7fada67bd700 1 -- 172.21.15.59:0/3506171119 shutdown_connections 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.752+0000 7fada67bd700 1 --2- 172.21.15.59:0/3506171119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada00f8cb0 0x7fada00f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.752+0000 7fada67bd700 1 --2- 172.21.15.59:0/3506171119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada0102210 0x7fada0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.752+0000 7fada67bd700 1 --2- 172.21.15.59:0/3506171119 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada00f9610 0x7fada0101d20 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.752+0000 7fada67bd700 1 -- 172.21.15.59:0/3506171119 >> 172.21.15.59:0/3506171119 conn(0x7fada00f2820 msgr2=0x7fada00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.752+0000 7fada67bd700 1 -- 172.21.15.59:0/3506171119 shutdown_connections 2024-02-18T00:39:43.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.753+0000 7fada67bd700 1 -- 172.21.15.59:0/3506171119 wait complete. 2024-02-18T00:39:43.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.753+0000 7fada67bd700 1 librados: starting msgr at 2024-02-18T00:39:43.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.753+0000 7fada67bd700 1 librados: starting objecter 2024-02-18T00:39:43.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.753+0000 7fada67bd700 1 Processor -- start 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.753+0000 7fada67bd700 1 -- start start 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.753+0000 7fada67bd700 1 librados: setting wanted keys 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.753+0000 7fada67bd700 1 librados: calling monclient init 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fada67bd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada00f8cb0 0x7fada0114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fada67bd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00f9610 0x7fada0114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fada67bd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada0102210 0x7fada0100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fada67bd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0101050 con 0x7fada00f9610 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fada67bd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01011c0 con 0x7fada00f8cb0 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fada67bd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada0101380 con 0x7fada0102210 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fad9f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00f9610 0x7fada0114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fad9f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00f9610 0x7fada0114d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46798/0 (socket says 172.21.15.59:46798) 2024-02-18T00:39:43.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fada4d5a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada0102210 0x7fada0100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fad9f7fe700 1 -- 172.21.15.59:0/4242678918 learned_addr learned my addr 172.21.15.59:0/4242678918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada00f8cb0 0x7fada0114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fad9f7fe700 1 -- 172.21.15.59:0/4242678918 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada0102210 msgr2=0x7fada0100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.754+0000 7fad9f7fe700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada0102210 0x7fada0100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9f7fe700 1 -- 172.21.15.59:0/4242678918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada00f8cb0 msgr2=0x7fada0114880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9f7fe700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada00f8cb0 0x7fada0114880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9f7fe700 1 -- 172.21.15.59:0/4242678918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad980097c0 con 0x7fada00f9610 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fada4d5a700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fada0102210 0x7fada0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fada00f8cb0 0x7fada0114880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9f7fe700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00f9610 0x7fada0114d70 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7fad94009b10 tx=0x7fad94009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9d7fa700 10 client.?.objecter ms_handle_connect 0x7fada00f9610 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9d7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:43.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94018070 con 0x7fada00f9610 2024-02-18T00:39:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad94005040 con 0x7fada00f9610 2024-02-18T00:39:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fada67bd700 10 client.31440.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada0101810 con 0x7fada00f9610 2024-02-18T00:39:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fada67bd700 1 librados: init done 2024-02-18T00:39:43.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad94013da0 con 0x7fada00f9610 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.755+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9401dc10 con 0x7fada00f9610 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.756+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fad8c004f30 con 0x7fada00f9610 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fad94009070 con 0x7fada00f9610 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad88062c90 0x7fad88065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(471..471 src has 180..471) v4 ==== 8151+0+0 (secure 0 0 0) 0x7fad9407e5e0 con 0x7fada00f9610 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 osd_map(471..471 src has 180..471) v4 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 3 client.31440.objecter handle_osd_map got epochs [471,471] > 0 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 3 client.31440.objecter handle_osd_map decoding full epoch 471 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad88062c90 0x7fad88065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 20 client.31440.objecter dump_active .. 0 homeless 2024-02-18T00:39:43.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v471) v1 ==== 54+0+16 (secure 0 0 0) 0x7fad94085070 con 0x7fada00f9610 2024-02-18T00:39:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fada67bd700 10 client.31440.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 1 2024-02-18T00:39:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad8c004560 0x7fad8c0049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.757+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 1 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 0 1 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c0051d0 osd=0 4 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada67bd700 10 client.31440.objecter _send_command 1 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c008960 con 0x7fad8c004560 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada4d5a700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad8c004560 0x7fad8c0049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fada4d5a700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad8c004560 0x7fad8c0049a0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c004560 2024-02-18T00:39:43.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad88062c90 2024-02-18T00:39:43.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.758+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad88062c90 0x7fad88065150 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 rx=0x7fad90007b70 tx=0x7fad9000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.759+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c008960 con 0x7fad8c004560 2024-02-18T00:39:43.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.759+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:39:43.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.759+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 1 = system:0 2024-02-18T00:39:43.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.759+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c0051d0 osd=0 4 2024-02-18T00:39:43.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.759+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 0 1 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 10 client.31440.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 2 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 2 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c0051d0 osd=0 3 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 0 2 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c0051d0 osd=0 4 2024-02-18T00:39:43.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 10 client.31440.objecter _send_command 2 2024-02-18T00:39:43.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.775+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c003fa0 con 0x7fad8c004560 2024-02-18T00:39:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c003fa0 con 0x7fad8c004560 2024-02-18T00:39:43.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c0051d0 osd=0 4 2024-02-18T00:39:43.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 0 2 2024-02-18T00:39:43.780 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 10 client.31440.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6805/26279110 conn(0x7fad8c002740 0x7fad8c002bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 1 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007fb0 osd=1 4 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 10 client.31440.objecter _send_command 3 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c008700 con 0x7fad8c002740 2024-02-18T00:39:43.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.778+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6805/26279110 conn(0x7fad8c002740 0x7fad8c002bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.779+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c002740 2024-02-18T00:39:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.779+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6805/26279110 conn(0x7fad8c002740 0x7fad8c002bd0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.779+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c008700 con 0x7fad8c002740 2024-02-18T00:39:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.779+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:39:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.779+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 3 = system:0 2024-02-18T00:39:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.779+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c007fb0 osd=1 4 2024-02-18T00:39:43.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.779+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 1 3 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 10 client.31440.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 4 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 4 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c007fb0 osd=1 3 2024-02-18T00:39:43.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 1 4 2024-02-18T00:39:43.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007fb0 osd=1 4 2024-02-18T00:39:43.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 10 client.31440.objecter _send_command 4 2024-02-18T00:39:43.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.803+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c003890 con 0x7fad8c002740 2024-02-18T00:39:43.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c003890 con 0x7fad8c002740 2024-02-18T00:39:43.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c007fb0 osd=1 4 2024-02-18T00:39:43.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 1 4 2024-02-18T00:39:43.808 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 10 client.31440.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 5 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6809/1153193637 conn(0x7fad8c00aff0 0x7fad8c00d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 5 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 2 5 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007ae0 osd=2 4 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 10 client.31440.objecter _send_command 5 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.806+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c010030 con 0x7fad8c00aff0 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.807+0000 7fad9f7fe700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6809/1153193637 conn(0x7fad8c00aff0 0x7fad8c00d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.807+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c00aff0 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.807+0000 7fad9f7fe700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6809/1153193637 conn(0x7fad8c00aff0 0x7fad8c00d3e0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.808+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c010030 con 0x7fad8c00aff0 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.808+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.808+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 5 = system:0 2024-02-18T00:39:43.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.808+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c007ae0 osd=2 4 2024-02-18T00:39:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.808+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 2 5 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 10 client.31440.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 6 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 6 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c007ae0 osd=2 3 2024-02-18T00:39:43.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 2 6 2024-02-18T00:39:43.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007ae0 osd=2 4 2024-02-18T00:39:43.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 10 client.31440.objecter _send_command 6 2024-02-18T00:39:43.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.828+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c003360 con 0x7fad8c00aff0 2024-02-18T00:39:43.833 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c003360 con 0x7fad8c00aff0 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c007ae0 osd=2 4 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 2 6 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fada67bd700 10 client.31440.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 7 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.831+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6813/1805700964 conn(0x7fad8c00ea50 0x7fad8c00eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 7 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 3 7 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c001bb0 osd=3 4 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 10 client.31440.objecter _send_command 7 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.832+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c002230 con 0x7fad8c00ea50 2024-02-18T00:39:43.836 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:43.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.836+0000 7fada4d5a700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6813/1805700964 conn(0x7fad8c00ea50 0x7fad8c00eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.836+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c00ea50 2024-02-18T00:39:43.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.836+0000 7fada4d5a700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6813/1805700964 conn(0x7fad8c00ea50 0x7fad8c00eec0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.837+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c002230 con 0x7fad8c00ea50 2024-02-18T00:39:43.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.837+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:39:43.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.837+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 7 = system:0 2024-02-18T00:39:43.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.837+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c001bb0 osd=3 4 2024-02-18T00:39:43.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.837+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 3 7 2024-02-18T00:39:43.852 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 10 client.31440.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 8 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 8 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c001bb0 osd=3 3 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 3 8 2024-02-18T00:39:43.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c001bb0 osd=3 4 2024-02-18T00:39:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 10 client.31440.objecter _send_command 8 2024-02-18T00:39:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.853+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c00fb80 con 0x7fad8c00ea50 2024-02-18T00:39:43.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c00fb80 con 0x7fad8c00ea50 2024-02-18T00:39:43.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c001bb0 osd=3 4 2024-02-18T00:39:43.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 3 8 2024-02-18T00:39:43.859 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:39:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 10 client.31440.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 9 2024-02-18T00:39:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6800/2044387056 conn(0x7fad8c0121e0 0x7fad8c0145d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 9 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 4 9 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c00fb80 osd=4 4 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 10 client.31440.objecter _send_command 9 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c017220 con 0x7fad8c0121e0 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6800/2044387056 conn(0x7fad8c0121e0 0x7fad8c0145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c0121e0 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.857+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6800/2044387056 conn(0x7fad8c0121e0 0x7fad8c0145d0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.858+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c017220 con 0x7fad8c0121e0 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.858+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:39:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.858+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 9 = system:0 2024-02-18T00:39:43.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.858+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c00fb80 osd=4 4 2024-02-18T00:39:43.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.858+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 4 9 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.717124+0000 mon.a (mon.0) 2640 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 2]}]': finished 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.717219+0000 mon.a (mon.0) 2641 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 1]}]': finished 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.717282+0000 mon.a (mon.0) 2642 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.717344+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.717413+0000 mon.a (mon.0) 2644 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.717506+0000 mon.a (mon.0) 2645 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.8", "id": [7, 5]}]': finished 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:42.717575+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.719966+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:43.880 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:42.720541+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:43.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:43.465947+0000 osd.5 (osd.5) 197 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:43.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:43.466062+0000 osd.5 (osd.5) 198 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:43.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:43.700938+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:43.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:43.701092+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-18T00:39:43.881 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:43.707236+0000 mon.c (mon.2) 446 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:43.882 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 10 client.31440.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 10 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 10 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c00fb80 osd=4 3 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 4 10 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c00fb80 osd=4 4 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 10 client.31440.objecter _send_command 10 2024-02-18T00:39:43.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.881+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c014ac0 con 0x7fad8c0121e0 2024-02-18T00:39:43.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c014ac0 con 0x7fad8c0121e0 2024-02-18T00:39:43.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c00fb80 osd=4 4 2024-02-18T00:39:43.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 4 10 2024-02-18T00:39:43.885 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:39:43.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 10 client.31440.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 11 2024-02-18T00:39:43.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6804/2170432263 conn(0x7fad8c016820 0x7fad8c016c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.886 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 11 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 5 11 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014ac0 osd=5 4 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 10 client.31440.objecter _send_command 11 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.883+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c01a5d0 con 0x7fad8c016820 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.884+0000 7fad9f7fe700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6804/2170432263 conn(0x7fad8c016820 0x7fad8c016c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.884+0000 7fad9f7fe700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6804/2170432263 conn(0x7fad8c016820 0x7fad8c016c60 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.884+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c016820 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.885+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c01a5d0 con 0x7fad8c016820 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.885+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:39:43.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.885+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 11 = system:0 2024-02-18T00:39:43.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.885+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c014ac0 osd=5 4 2024-02-18T00:39:43.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.885+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 5 11 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 10 client.31440.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 12 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 12 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c014ac0 osd=5 3 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 5 12 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014ac0 osd=5 4 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 10 client.31440.objecter _send_command 12 2024-02-18T00:39:43.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.899+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c014c60 con 0x7fad8c016820 2024-02-18T00:39:43.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c014c60 con 0x7fad8c016820 2024-02-18T00:39:43.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c014ac0 osd=5 4 2024-02-18T00:39:43.903 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 5 12 2024-02-18T00:39:43.903 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 10 client.31440.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 13 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6808/3356009390 conn(0x7fad8c015340 0x7fad8c0157a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 13 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 6 13 2024-02-18T00:39:43.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014c60 osd=6 4 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 10 client.31440.objecter _send_command 13 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.901+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c016110 con 0x7fad8c015340 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.902+0000 7fada4d5a700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6808/3356009390 conn(0x7fad8c015340 0x7fad8c0157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.902+0000 7fada4d5a700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6808/3356009390 conn(0x7fad8c015340 0x7fad8c0157a0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.902+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c015340 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.903+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c016110 con 0x7fad8c015340 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.903+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.903+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 13 = system:0 2024-02-18T00:39:43.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.903+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c014c60 osd=6 4 2024-02-18T00:39:43.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.903+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 6 13 2024-02-18T00:39:43.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 10 client.31440.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 14 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 14 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c014c60 osd=6 3 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 6 14 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014c60 osd=6 4 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 10 client.31440.objecter _send_command 14 2024-02-18T00:39:43.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.918+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c016110 con 0x7fad8c015340 2024-02-18T00:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c016110 con 0x7fad8c015340 2024-02-18T00:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c014c60 osd=6 4 2024-02-18T00:39:43.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 6 14 2024-02-18T00:39:43.922 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 10 client.31440.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 15 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6812/53025171 conn(0x7fad8c01b970 0x7fad8c01dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 15 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 7 15 2024-02-18T00:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c016110 osd=7 4 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 10 client.31440.objecter _send_command 15 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad8c020b50 con 0x7fad8c01b970 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.920+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6812/53025171 conn(0x7fad8c01b970 0x7fad8c01dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.921+0000 7fad9d7fa700 10 client.31440.objecter ms_handle_connect 0x7fad8c01b970 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.921+0000 7fad9ffff700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6812/53025171 conn(0x7fad8c01b970 0x7fad8c01dd80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.921+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fad8c020b50 con 0x7fad8c01b970 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.921+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.921+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 15 = system:0 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.921+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c016110 osd=7 4 2024-02-18T00:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.921+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 7 15 2024-02-18T00:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 10 client.31440.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign -1 16 2024-02-18T00:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 20 client.31440.objecter _get_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 15 client.31440.objecter _session_command_op_remove -1 16 2024-02-18T00:39:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 20 client.31440.objecter get_session s=0x7fad8c016110 osd=7 3 2024-02-18T00:39:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 15 client.31440.objecter _session_command_op_assign 7 16 2024-02-18T00:39:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c016110 osd=7 4 2024-02-18T00:39:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 10 client.31440.objecter _send_command 16 2024-02-18T00:39:43.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.936+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fad8c020b50 con 0x7fad8c01b970 2024-02-18T00:39:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.938+0000 7fad9d7fa700 1 -- 172.21.15.59:0/4242678918 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fad8c020b50 con 0x7fad8c01b970 2024-02-18T00:39:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.938+0000 7fad9d7fa700 10 client.31440.objecter ms_dispatch 0x7fada0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:39:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.938+0000 7fad9d7fa700 10 client.31440.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.938+0000 7fad9d7fa700 20 client.31440.objecter put_session s=0x7fad8c016110 osd=7 4 2024-02-18T00:39:43.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.938+0000 7fad9d7fa700 15 client.31440.objecter _session_command_op_remove 7 16 2024-02-18T00:39:43.940 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 librados: watch_flush enter 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 librados: watch_flush exit 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.0 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad8c004560 msgr2=0x7fad8c0049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6801/3700238977 conn(0x7fad8c004560 0x7fad8c0049a0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c0051d0 osd=0 1 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.1 2024-02-18T00:39:43.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6805/26279110 conn(0x7fad8c002740 msgr2=0x7fad8c002bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6805/26279110 conn(0x7fad8c002740 0x7fad8c002bd0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007fb0 osd=1 1 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.2 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6809/1153193637 conn(0x7fad8c00aff0 msgr2=0x7fad8c00d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6809/1153193637 conn(0x7fad8c00aff0 0x7fad8c00d3e0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c007ae0 osd=2 1 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.3 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6813/1805700964 conn(0x7fad8c00ea50 msgr2=0x7fad8c00eec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6813/1805700964 conn(0x7fad8c00ea50 0x7fad8c00eec0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada4d5a700 1 -- 172.21.15.59:0/4242678918 reap_dead start 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c001bb0 osd=3 1 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.4 2024-02-18T00:39:43.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6800/2044387056 conn(0x7fad8c0121e0 msgr2=0x7fad8c0145d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6800/2044387056 conn(0x7fad8c0121e0 0x7fad8c0145d0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c00fb80 osd=4 1 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.5 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6804/2170432263 conn(0x7fad8c016820 msgr2=0x7fad8c016c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6804/2170432263 conn(0x7fad8c016820 0x7fad8c016c60 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014ac0 osd=5 1 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.6 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6808/3356009390 conn(0x7fad8c015340 msgr2=0x7fad8c0157a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6808/3356009390 conn(0x7fad8c015340 0x7fad8c0157a0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c014c60 osd=6 1 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter close_session for osd.7 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6812/53025171 conn(0x7fad8c01b970 msgr2=0x7fad8c01dd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.137:6812/53025171 conn(0x7fad8c01b970 0x7fad8c01dd80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter put_session s=0x7fad8c016110 osd=7 1 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 20 client.31440.objecter shutdown clearing up homeless session... 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 10 client.31440.objecter successfully canceled tick 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada4d5a700 1 -- 172.21.15.59:0/4242678918 reap_dead start 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad88062c90 msgr2=0x7fad88065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad88062c90 0x7fad88065150 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 rx=0x7fad90007b70 tx=0x7fad9000b040).stop 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00f9610 msgr2=0x7fada0114d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00f9610 0x7fada0114d70 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7fad94009b10 tx=0x7fad94009cb0).stop 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 shutdown_connections 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fada00f9610 0x7fada0114d70 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 --2- 172.21.15.59:0/4242678918 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad88062c90 0x7fad88065150 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 >> 172.21.15.59:0/4242678918 conn(0x7fada00f2820 msgr2=0x7fada00f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 shutdown_connections 2024-02-18T00:39:43.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 -- 172.21.15.59:0/4242678918 wait complete. 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:43.939+0000 7fada67bd700 1 librados: shutdown 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:39:43.946 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:39:44.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.717344+0000 mon.a (mon.0) 2643 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [7, 2]}]': finished 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.717413+0000 mon.a (mon.0) 2644 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [7, 2]}]': finished 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.717506+0000 mon.a (mon.0) 2645 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "62.8", "id": [7, 5]}]': finished 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:42.717575+0000 mon.a (mon.0) 2646 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.719966+0000 mon.c (mon.2) 445 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:42.720541+0000 mon.a (mon.0) 2647 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:43.465947+0000 osd.5 (osd.5) 197 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:43.466062+0000 osd.5 (osd.5) 198 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:43.700938+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:43.701092+0000 mon.a (mon.0) 2649 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-18T00:39:44.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:43.707236+0000 mon.c (mon.2) 446 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:44.133 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:44.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:44.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:44.302 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:44.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:44.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:44.406 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:44.434 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:44.632 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:44.635 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:44.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:44.736 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:44.766 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:44.793 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:44.797 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:44.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:44.898 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:44.926 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:44.936 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:43.758325+0000 mon.a (mon.0) 2650 : audit [DBG] from='client.? 172.21.15.59:0/4242678918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:44.936 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:43.907306+0000 osd.3 (osd.3) 259 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:44.936 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:43.907571+0000 osd.3 (osd.3) 260 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:44.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:43.923635+0000 mgr.y (mgr.24454) 441 : cluster [DBG] pgmap v711: 238 pgs: 32 unknown, 1 active+recovering+undersized+remapped, 205 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 726 B/s rd, 19 KiB/s wr, 18 op/s; 7/1170 objects misplaced (0.598%); 925 KiB/s, 101 objects/s recovering 2024-02-18T00:39:44.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:44.703266+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-18T00:39:44.937 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:43.758325+0000 mon.a (mon.0) 2650 : audit [DBG] from='client.? 172.21.15.59:0/4242678918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:44.937 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:43.907306+0000 osd.3 (osd.3) 259 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:44.937 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:43.907571+0000 osd.3 (osd.3) 260 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:44.937 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:43.923635+0000 mgr.y (mgr.24454) 441 : cluster [DBG] pgmap v711: 238 pgs: 32 unknown, 1 active+recovering+undersized+remapped, 205 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 726 B/s rd, 19 KiB/s wr, 18 op/s; 7/1170 objects misplaced (0.598%); 925 KiB/s, 101 objects/s recovering 2024-02-18T00:39:44.937 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:44.703266+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-18T00:39:44.945 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:44.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:44.947 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:43.758325+0000 mon.a (mon.0) 2650 : audit [DBG] from='client.? 172.21.15.59:0/4242678918' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:43.907306+0000 osd.3 (osd.3) 259 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:43.907571+0000 osd.3 (osd.3) 260 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:43.923635+0000 mgr.y (mgr.24454) 441 : cluster [DBG] pgmap v711: 238 pgs: 32 unknown, 1 active+recovering+undersized+remapped, 205 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 726 B/s rd, 19 KiB/s wr, 18 op/s; 7/1170 objects misplaced (0.598%); 925 KiB/s, 101 objects/s recovering 2024-02-18T00:39:45.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:44.703266+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-18T00:39:45.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:45.079 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:45.138 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:39:45.139 INFO:teuthology.orchestra.run.smithi137.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-18T00:39:45.139 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:39:45.634 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:45.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:45.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:45.738 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:45.767 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:45.861 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:45.139108+0000 osd.6 (osd.6) 223 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:45.862 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:45.139260+0000 osd.6 (osd.6) 224 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:45.862 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:45.702794+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-18T00:39:46.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:45.139108+0000 osd.6 (osd.6) 223 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:45.139260+0000 osd.6 (osd.6) 224 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:46.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:45.702794+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-18T00:39:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:45.139108+0000 osd.6 (osd.6) 223 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:45.139260+0000 osd.6 (osd.6) 224 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:46.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:45.702794+0000 mon.a (mon.0) 2652 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-18T00:39:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:46.220 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:46.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:46.322 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:46.378 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:46.381 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:46.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:46.482 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:46.513 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:46.764 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:46.767 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:45.924783+0000 mgr.y (mgr.24454) 442 : cluster [DBG] pgmap v714: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s; 12 B/s, 2 objects/s recovering 2024-02-18T00:39:46.768 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:45.924783+0000 mgr.y (mgr.24454) 442 : cluster [DBG] pgmap v714: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s; 12 B/s, 2 objects/s recovering 2024-02-18T00:39:46.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:46.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:46.871 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:46.904 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:46.926 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:46.928 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:46.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:47.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:45.924783+0000 mgr.y (mgr.24454) 442 : cluster [DBG] pgmap v714: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s; 12 B/s, 2 objects/s recovering 2024-02-18T00:39:47.030 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:47.060 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:47.080 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:47.083 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:47.083 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:47.184 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:47.213 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:47.232 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:47.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:47.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:47.337 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:47.364 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:47.383 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:47.386 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:47.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:47.487 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:47.518 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 -- 172.21.15.59:0/1040554441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 msgr2=0x7fadd00f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 --2- 172.21.15.59:0/1040554441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd00f9090 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 rx=0x7fadc00047b0 tx=0x7fadc0009cb0).stop 2024-02-18T00:39:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 -- 172.21.15.59:0/1040554441 shutdown_connections 2024-02-18T00:39:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 --2- 172.21.15.59:0/1040554441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadd0102210 0x7fadd0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 --2- 172.21.15.59:0/1040554441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd00f9090 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 --2- 172.21.15.59:0/1040554441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd00f9610 0x7fadd0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 -- 172.21.15.59:0/1040554441 >> 172.21.15.59:0/1040554441 conn(0x7fadd00f2820 msgr2=0x7fadd00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 -- 172.21.15.59:0/1040554441 shutdown_connections 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.443+0000 7fadd4cad700 1 -- 172.21.15.59:0/1040554441 wait complete. 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 librados: starting msgr at 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 librados: starting objecter 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 Processor -- start 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 -- start start 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 librados: setting wanted keys 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 librados: calling monclient init 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd010e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd00f9610 0x7fadd010e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadd0102210 0x7fadd0108450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:47.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0108940 con 0x7fadd00f9610 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0108ab0 con 0x7fadd0102210 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.444+0000 7fadd4cad700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0108c70 con 0x7fadd00f8cb0 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd010e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadcdd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd00f9610 0x7fadd010e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd010e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56872/0 (socket says 172.21.15.59:56872) 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 -- 172.21.15.59:0/160638611 learned_addr learned my addr 172.21.15.59:0/160638611 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadced9d700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadd0102210 0x7fadd0108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 -- 172.21.15.59:0/160638611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadd0102210 msgr2=0x7fadd0108450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadd0102210 0x7fadd0108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 -- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd00f9610 msgr2=0x7fadd010e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd00f9610 0x7fadd010e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 -- 172.21.15.59:0/160638611 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4005040 con 0x7fadd00f8cb0 2024-02-18T00:39:47.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadcdd9b700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd00f9610 0x7fadd010e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadce59c700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd010e2b0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 rx=0x7fadc0009b10 tx=0x7fadc0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadbf7fe700 10 client.?.objecter ms_handle_connect 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadbf7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadbf7fe700 1 -- 172.21.15.59:0/160638611 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0018070 con 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadbf7fe700 1 -- 172.21.15.59:0/160638611 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc0013e70 con 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc0005040 con 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadbf7fe700 1 -- 172.21.15.59:0/160638611 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc001ddb0 con 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadd4cad700 10 client.41249.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd0109110 con 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.445+0000 7fadd4cad700 1 librados: init done 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.446+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0109300 con 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 1 -- 172.21.15.59:0/160638611 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fadc001d420 con 0x7fadd00f8cb0 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 1 --2- 172.21.15.59:0/160638611 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadb8062c90 0x7fadb8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:47.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 1 -- 172.21.15.59:0/160638611 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(473..473 src has 180..473) v4 ==== 8151+0+0 (secure 0 0 0) 0x7fadc007e640 con 0x7fadd00f8cb0 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 10 client.41249.objecter ms_dispatch 0x7fadd0001550 osd_map(473..473 src has 180..473) v4 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 3 client.41249.objecter handle_osd_map got epochs [473,473] > 0 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 3 client.41249.objecter handle_osd_map decoding full epoch 473 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadcdd9b700 1 --2- 172.21.15.59:0/160638611 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadb8062c90 0x7fadb8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 20 client.41249.objecter dump_active .. 0 homeless 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadbf7fe700 10 client.41249.objecter ms_handle_connect 0x7fadb8062c90 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.447+0000 7fadcdd9b700 1 --2- 172.21.15.59:0/160638611 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadb8062c90 0x7fadb8065150 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 rx=0x7fadd0109bf0 tx=0x7fadc400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.451+0000 7fadbf7fe700 1 -- 172.21.15.59:0/160638611 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fadc0051af0 con 0x7fadd00f8cb0 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.639+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fadd0107290 con 0x7fadd00f8cb0 2024-02-18T00:39:47.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.749+0000 7fadbf7fe700 1 -- 172.21.15.59:0/160638611 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v474) v1 ==== 91+0+0 (secure 0 0 0) 0x7fadc00516e0 con 0x7fadd00f8cb0 2024-02-18T00:39:47.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 10 librados: watch_flush enter 2024-02-18T00:39:47.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 10 librados: watch_flush exit 2024-02-18T00:39:47.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 20 client.41249.objecter shutdown clearing up homeless session... 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 10 client.41249.objecter successfully canceled tick 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadb8062c90 msgr2=0x7fadb8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 --2- 172.21.15.59:0/160638611 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadb8062c90 0x7fadb8065150 secure :-1 s=READY pgs=3128 cs=0 l=1 rev1=1 rx=0x7fadd0109bf0 tx=0x7fadc400b040).stop 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 msgr2=0x7fadd010e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd010e2b0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 rx=0x7fadc0009b10 tx=0x7fadc0009cb0).stop 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 shutdown_connections 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 --2- 172.21.15.59:0/160638611 >> v2:172.21.15.59:6800/3721891797 conn(0x7fadb8062c90 0x7fadb8065150 unknown :-1 s=CLOSED pgs=3128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fadd0102210 0x7fadd0108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fadd00f8cb0 0x7fadd010e2b0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.752+0000 7fadd4cad700 1 --2- 172.21.15.59:0/160638611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd00f9610 0x7fadd010e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.753+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 >> 172.21.15.59:0/160638611 conn(0x7fadd00f2820 msgr2=0x7fadd00f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.753+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 shutdown_connections 2024-02-18T00:39:47.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.753+0000 7fadd4cad700 1 -- 172.21.15.59:0/160638611 wait complete. 2024-02-18T00:39:47.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:47.753+0000 7fadd4cad700 1 librados: shutdown 2024-02-18T00:39:47.760 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:39:47.770 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:47.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:47.640714+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.59:0/160638611' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:39:47.772 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:47.641045+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:39:47.772 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:47.640714+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.59:0/160638611' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:39:47.773 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:47.641045+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:39:47.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:47.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:47.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:47.902 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:47.922 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:47.924 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:47.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:48.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:47.640714+0000 mon.c (mon.2) 447 : audit [INF] from='client.? 172.21.15.59:0/160638611' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:39:48.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:47.641045+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:39:48.025 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:48.056 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:48.501 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:39:48.501 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:39:48.505 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:48.508 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:48.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:48.610 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:48.640 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:48.660 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:48.663 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:48.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:48.746 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:39:48.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:48.820 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:49.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:47.741419+0000 mon.a (mon.0) 2654 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:47.749937+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:39:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:47.750011+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-18T00:39:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:47.925280+0000 mgr.y (mgr.24454) 443 : cluster [DBG] pgmap v716: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s; 12 B/s, 2 objects/s recovering 2024-02-18T00:39:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:48.701503+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-18T00:39:49.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:47.741419+0000 mon.a (mon.0) 2654 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:49.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:47.749937+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:39:49.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:47.750011+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-18T00:39:49.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:47.925280+0000 mgr.y (mgr.24454) 443 : cluster [DBG] pgmap v716: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s; 12 B/s, 2 objects/s recovering 2024-02-18T00:39:49.118 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:48.701503+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-18T00:39:49.118 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:47.741419+0000 mon.a (mon.0) 2654 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:39:49.118 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:47.749937+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:39:49.119 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:47.750011+0000 mon.a (mon.0) 2656 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-18T00:39:49.119 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:47.925280+0000 mgr.y (mgr.24454) 443 : cluster [DBG] pgmap v716: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s; 12 B/s, 2 objects/s recovering 2024-02-18T00:39:49.119 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:48.701503+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-18T00:39:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:49.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:49.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:49.277 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:49.306 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:49.330 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:49.333 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:49.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:49.434 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:49.468 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:49.619 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:49.625 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:49.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:49.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:49.754 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:50.055 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:48.226314+0000 osd.0 (osd.0) 220 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:48.227097+0000 osd.0 (osd.0) 221 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.135204+0000 mon.a (mon.0) 2658 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.421994+0000 mon.a (mon.0) 2659 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.422649+0000 mon.a (mon.0) 2660 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.423175+0000 mon.a (mon.0) 2661 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.423682+0000 mon.a (mon.0) 2662 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.428130+0000 mon.a (mon.0) 2663 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.572070+0000 mon.a (mon.0) 2664 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.572734+0000 mon.a (mon.0) 2665 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.573283+0000 mon.a (mon.0) 2666 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.573824+0000 mon.a (mon.0) 2667 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.056 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.580270+0000 mon.a (mon.0) 2668 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.581046+0000 mon.a (mon.0) 2669 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.581688+0000 mon.a (mon.0) 2670 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:49.586605+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:48.226314+0000 osd.0 (osd.0) 220 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:48.227097+0000 osd.0 (osd.0) 221 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.135204+0000 mon.a (mon.0) 2658 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.421994+0000 mon.a (mon.0) 2659 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.422649+0000 mon.a (mon.0) 2660 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.423175+0000 mon.a (mon.0) 2661 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.423682+0000 mon.a (mon.0) 2662 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.057 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.428130+0000 mon.a (mon.0) 2663 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.572070+0000 mon.a (mon.0) 2664 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.572734+0000 mon.a (mon.0) 2665 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.573283+0000 mon.a (mon.0) 2666 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.573824+0000 mon.a (mon.0) 2667 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.580270+0000 mon.a (mon.0) 2668 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.581046+0000 mon.a (mon.0) 2669 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.581688+0000 mon.a (mon.0) 2670 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:39:50.058 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:49.586605+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.074 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-18T00:39:50.105 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:50.107 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:50.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:50.112 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:50.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:50.238 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:48.226314+0000 osd.0 (osd.0) 220 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:48.227097+0000 osd.0 (osd.0) 221 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.135204+0000 mon.a (mon.0) 2658 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.421994+0000 mon.a (mon.0) 2659 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.422649+0000 mon.a (mon.0) 2660 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.423175+0000 mon.a (mon.0) 2661 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.423682+0000 mon.a (mon.0) 2662 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.428130+0000 mon.a (mon.0) 2663 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.572070+0000 mon.a (mon.0) 2664 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.572734+0000 mon.a (mon.0) 2665 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.573283+0000 mon.a (mon.0) 2666 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.573824+0000 mon.a (mon.0) 2667 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T00:39:50.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.580270+0000 mon.a (mon.0) 2668 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.581046+0000 mon.a (mon.0) 2669 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:39:50.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.581688+0000 mon.a (mon.0) 2670 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:39:50.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:49.586605+0000 mon.a (mon.0) 2671 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:50.437 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:50.442 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:50.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:50.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:50.881 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:39:50.881 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:39:50.881 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-02-18T00:39:50.881 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-18T00:39:50.881 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-18T00:39:50.881 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-02-18T00:39:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:49.205805+0000 osd.0 (osd.0) 222 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:39:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:49.207310+0000 osd.0 (osd.0) 223 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:39:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:39:49.424164+0000 mgr.y (mgr.24454) 444 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 3256M 2024-02-18T00:39:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cephadm 2024-02-18T00:39:49.574269+0000 mgr.y (mgr.24454) 445 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 3257M 2024-02-18T00:39:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:49.925755+0000 mgr.y (mgr.24454) 446 : cluster [DBG] pgmap v718: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 5 op/s; 9 B/s, 2 objects/s recovering 2024-02-18T00:39:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:50.124682+0000 osd.2 (osd.2) 199 : cluster [DBG] 2.c scrub starts 2024-02-18T00:39:51.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:50.125731+0000 osd.2 (osd.2) 200 : cluster [DBG] 2.c scrub ok 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:50.156272+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:50.307399+0000 osd.4 (osd.4) 279 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:50.310864+0000 osd.4 (osd.4) 280 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:49.205805+0000 osd.0 (osd.0) 222 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:49.207310+0000 osd.0 (osd.0) 223 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:39:49.424164+0000 mgr.y (mgr.24454) 444 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 3256M 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cephadm 2024-02-18T00:39:49.574269+0000 mgr.y (mgr.24454) 445 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 3257M 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:49.925755+0000 mgr.y (mgr.24454) 446 : cluster [DBG] pgmap v718: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 5 op/s; 9 B/s, 2 objects/s recovering 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:50.124682+0000 osd.2 (osd.2) 199 : cluster [DBG] 2.c scrub starts 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:50.125731+0000 osd.2 (osd.2) 200 : cluster [DBG] 2.c scrub ok 2024-02-18T00:39:51.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:50.156272+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:51.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:50.307399+0000 osd.4 (osd.4) 279 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:39:51.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:50.310864+0000 osd.4 (osd.4) 280 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:39:51.062 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:51.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:39:51.158 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:39:51.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:39:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:49.205805+0000 osd.0 (osd.0) 222 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:49.207310+0000 osd.0 (osd.0) 223 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:39:49.424164+0000 mgr.y (mgr.24454) 444 : cephadm [INF] Adjusting osd_memory_target on smithi137 to 3256M 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cephadm 2024-02-18T00:39:49.574269+0000 mgr.y (mgr.24454) 445 : cephadm [INF] Adjusting osd_memory_target on smithi059 to 3257M 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:49.925755+0000 mgr.y (mgr.24454) 446 : cluster [DBG] pgmap v718: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.3 KiB/s wr, 5 op/s; 9 B/s, 2 objects/s recovering 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:50.124682+0000 osd.2 (osd.2) 199 : cluster [DBG] 2.c scrub starts 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:50.125731+0000 osd.2 (osd.2) 200 : cluster [DBG] 2.c scrub ok 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:50.156272+0000 mon.a (mon.0) 2672 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:50.307399+0000 osd.4 (osd.4) 279 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:39:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:50.310864+0000 osd.4 (osd.4) 280 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:39:51.422 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:51.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:51.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:51.526 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:51.554 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:51.607 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:51.609 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:39:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:39:51.611 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:51.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:51.714 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:51.741 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:51.989 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:51.991 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:50.934829+0000 osd.6 (osd.6) 225 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:39:51.991 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:50.936741+0000 osd.6 (osd.6) 226 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:39:51.991 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:51.317999+0000 mgr.y (mgr.24454) 447 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:51.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:51.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:52.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:52.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:50.934829+0000 osd.6 (osd.6) 225 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:39:52.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:50.936741+0000 osd.6 (osd.6) 226 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:39:52.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:51.317999+0000 mgr.y (mgr.24454) 447 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:52.120 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:50.934829+0000 osd.6 (osd.6) 225 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:39:52.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:50.936741+0000 osd.6 (osd.6) 226 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:39:52.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:51.317999+0000 mgr.y (mgr.24454) 447 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:39:52.123 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:52.371 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:39:52] "GET /metrics HTTP/1.1" 200 189889 "" "Prometheus/2.33.4" 2024-02-18T00:39:52.372 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:52.375 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:52.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:52.477 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:52.507 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:52.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 -- 172.21.15.59:0/681219505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0fdc90 msgr2=0x7fd64c0fe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 --2- 172.21.15.59:0/681219505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0fdc90 0x7fd64c0fe070 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 rx=0x7fd63c0047b0 tx=0x7fd63c009cb0).stop 2024-02-18T00:39:52.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 -- 172.21.15.59:0/681219505 shutdown_connections 2024-02-18T00:39:52.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 --2- 172.21.15.59:0/681219505 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd64c0f6880 0x7fd64c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 --2- 172.21.15.59:0/681219505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0fdc90 0x7fd64c0fe070 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 --2- 172.21.15.59:0/681219505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd64c0fe560 0x7fd64c0f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 -- 172.21.15.59:0/681219505 >> 172.21.15.59:0/681219505 conn(0x7fd64c06ffc0 msgr2=0x7fd64c0f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 -- 172.21.15.59:0/681219505 shutdown_connections 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 -- 172.21.15.59:0/681219505 wait complete. 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 librados: starting msgr at 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 librados: starting objecter 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.594+0000 7fd651270700 1 Processor -- start 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 -- start start 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 librados: setting wanted keys 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 librados: calling monclient init 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0f6880 0x7fd64c10e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd64c0fdc90 0x7fd64c10e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd64c0fe560 0x7fd64c108340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c108830 con 0x7fd64c0f6880 2024-02-18T00:39:52.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1089a0 con 0x7fd64c0fe560 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd651270700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c108b60 con 0x7fd64c0fdc90 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd64affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0f6880 0x7fd64c10e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd64a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd64c0fdc90 0x7fd64c10e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd64affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0f6880 0x7fd64c10e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56848/0 (socket says 172.21.15.59:56848) 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd64a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd64c0fdc90 0x7fd64c10e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:34710/0 (socket says 172.21.15.59:34710) 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd64affd700 1 -- 172.21.15.59:0/3617652940 learned_addr learned my addr 172.21.15.59:0/3617652940 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.595+0000 7fd64b7fe700 1 --2- 172.21.15.59:0/3617652940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd64c0fe560 0x7fd64c108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd64affd700 1 -- 172.21.15.59:0/3617652940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd64c0fdc90 msgr2=0x7fd64c10e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd64c0fdc90 0x7fd64c10e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd64affd700 1 -- 172.21.15.59:0/3617652940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd64c0fe560 msgr2=0x7fd64c108340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd64c0fe560 0x7fd64c108340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd64affd700 1 -- 172.21.15.59:0/3617652940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6440097c0 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd64a7fc700 1 --2- 172.21.15.59:0/3617652940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd64c0fdc90 0x7fd64c10e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0f6880 0x7fd64c10e1a0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 rx=0x7fd63c009440 tx=0x7fd63c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd633fff700 10 client.?.objecter ms_handle_connect 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd633fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c019070 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63c0134c0 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c005040 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c014b30 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd651270700 10 client.31452.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c109080 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.596+0000 7fd651270700 1 librados: init done 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.597+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd64c05f9b0 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd63c013ac0 con 0x7fd64c0f6880 2024-02-18T00:39:52.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd634062c90 0x7fd634065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(475..475 src has 180..475) v4 ==== 8151+0+0 (secure 0 0 0) 0x7fd63c07e450 con 0x7fd64c0f6880 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 osd_map(475..475 src has 180..475) v4 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 3 client.31452.objecter handle_osd_map got epochs [475,475] > 0 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 3 client.31452.objecter handle_osd_map decoding full epoch 475 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd64a7fc700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd634062c90 0x7fd634065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 20 client.31452.objecter dump_active .. 0 homeless 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v475) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd63c07e930 con 0x7fd64c0f6880 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 10 client.31452.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 1 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd634062c90 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd64a7fc700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd634062c90 0x7fd634065150 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 rx=0x7fd64c109ae0 tx=0x7fd640009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd64c0fee80 0x7fd64c0ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 1 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 0 1 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c05ee80 osd=0 4 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 10 client.31452.objecter _send_command 1 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.598+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c1071a0 con 0x7fd64c0fee80 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd64b7fe700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd64c0fee80 0x7fd64c0ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c0fee80 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd64b7fe700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd64c0fee80 0x7fd64c0ff260 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c1071a0 con 0x7fd64c0fee80 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:39:52.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd633fff700 10 client.31452.objecter _finish_command 1 = system:0 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c05ee80 osd=0 4 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.599+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 0 1 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 10 client.31452.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 2 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 2 2024-02-18T00:39:52.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c05ee80 osd=0 3 2024-02-18T00:39:52.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 0 2 2024-02-18T00:39:52.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c05ee80 osd=0 4 2024-02-18T00:39:52.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 10 client.31452.objecter _send_command 2 2024-02-18T00:39:52.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.614+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c0ff990 con 0x7fd64c0fee80 2024-02-18T00:39:52.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64400f040 con 0x7fd64c0fee80 2024-02-18T00:39:52.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd633fff700 10 client.31452.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c05ee80 osd=0 4 2024-02-18T00:39:52.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 0 2 2024-02-18T00:39:52.618 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 10 client.31452.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6805/26279110 conn(0x7fd64c0598f0 0x7fd64c059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 1 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c028540 osd=1 4 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 10 client.31452.objecter _send_command 3 2024-02-18T00:39:52.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c0286b0 con 0x7fd64c0598f0 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6805/26279110 conn(0x7fd64c0598f0 0x7fd64c059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c0598f0 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.616+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6805/26279110 conn(0x7fd64c0598f0 0x7fd64c059d10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.617+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c0286b0 con 0x7fd64c0598f0 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.617+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.617+0000 7fd633fff700 10 client.31452.objecter _finish_command 3 = system:0 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.617+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c028540 osd=1 4 2024-02-18T00:39:52.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.617+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 1 3 2024-02-18T00:39:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:52.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 10 client.31452.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 4 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 4 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c028540 osd=1 3 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 1 4 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c028540 osd=1 4 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 10 client.31452.objecter _send_command 4 2024-02-18T00:39:52.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.632+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c059360 con 0x7fd64c0598f0 2024-02-18T00:39:52.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64c059360 con 0x7fd64c0598f0 2024-02-18T00:39:52.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd633fff700 10 client.31452.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c028540 osd=1 4 2024-02-18T00:39:52.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 1 4 2024-02-18T00:39:52.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:52.636 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:39:52.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:52.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 10 client.31452.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 5 2024-02-18T00:39:52.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd64c11a170 0x7fd64c11c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 5 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 2 5 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027b90 osd=2 4 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 10 client.31452.objecter _send_command 5 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.633+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c028210 con 0x7fd64c11a170 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd64a7fc700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd64c11a170 0x7fd64c11c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c11a170 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd64a7fc700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd64c11a170 0x7fd64c11c560 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c028210 con 0x7fd64c11a170 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd633fff700 10 client.31452.objecter _finish_command 5 = system:0 2024-02-18T00:39:52.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c027b90 osd=2 4 2024-02-18T00:39:52.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.634+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 2 5 2024-02-18T00:39:52.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 10 client.31452.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 6 2024-02-18T00:39:52.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 6 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c027b90 osd=2 3 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 2 6 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027b90 osd=2 4 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 10 client.31452.objecter _send_command 6 2024-02-18T00:39:52.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.649+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c027600 con 0x7fd64c11a170 2024-02-18T00:39:52.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64c027600 con 0x7fd64c11a170 2024-02-18T00:39:52.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd633fff700 10 client.31452.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c027b90 osd=2 4 2024-02-18T00:39:52.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 2 6 2024-02-18T00:39:52.652 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:39:52.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 10 client.31452.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 7 2024-02-18T00:39:52.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd64c119cd0 0x7fd64c11fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 7 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 3 7 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027a10 osd=3 4 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 10 client.31452.objecter _send_command 7 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.650+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c122860 con 0x7fd64c119cd0 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd64b7fe700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd64c119cd0 0x7fd64c11fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c119cd0 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd64b7fe700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd64c119cd0 0x7fd64c11fbe0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c122860 con 0x7fd64c119cd0 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:39:52.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd633fff700 10 client.31452.objecter _finish_command 7 = system:0 2024-02-18T00:39:52.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c027a10 osd=3 4 2024-02-18T00:39:52.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.651+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 3 7 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 10 client.31452.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 8 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 8 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c027a10 osd=3 3 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 3 8 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027a10 osd=3 4 2024-02-18T00:39:52.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 10 client.31452.objecter _send_command 8 2024-02-18T00:39:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.666+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c119230 con 0x7fd64c119cd0 2024-02-18T00:39:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64c119230 con 0x7fd64c119cd0 2024-02-18T00:39:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd633fff700 10 client.31452.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c027a10 osd=3 4 2024-02-18T00:39:52.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 3 8 2024-02-18T00:39:52.670 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:39:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 10 client.31452.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 9 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd64c1193e0 0x7fd64c121250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 9 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 4 9 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c119230 osd=4 4 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 10 client.31452.objecter _send_command 9 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c11d950 con 0x7fd64c1193e0 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd64c1193e0 0x7fd64c121250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c1193e0 2024-02-18T00:39:52.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.668+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd64c1193e0 0x7fd64c121250 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.669+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c11d950 con 0x7fd64c1193e0 2024-02-18T00:39:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.669+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:39:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.669+0000 7fd633fff700 10 client.31452.objecter _finish_command 9 = system:0 2024-02-18T00:39:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.669+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c119230 osd=4 4 2024-02-18T00:39:52.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.669+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 4 9 2024-02-18T00:39:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 10 client.31452.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 10 2024-02-18T00:39:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 10 2024-02-18T00:39:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c119230 osd=4 3 2024-02-18T00:39:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 4 10 2024-02-18T00:39:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c119230 osd=4 4 2024-02-18T00:39:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 10 client.31452.objecter _send_command 10 2024-02-18T00:39:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.684+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c11d910 con 0x7fd64c1193e0 2024-02-18T00:39:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.685+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64c11d910 con 0x7fd64c1193e0 2024-02-18T00:39:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.685+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.685+0000 7fd633fff700 10 client.31452.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.685+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c119230 osd=4 4 2024-02-18T00:39:52.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.685+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 4 10 2024-02-18T00:39:52.688 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:39:52.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 10 client.31452.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 11 2024-02-18T00:39:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd64c121c50 0x7fd64c1220b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 11 2024-02-18T00:39:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 5 11 2024-02-18T00:39:52.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c11d910 osd=5 4 2024-02-18T00:39:52.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 10 client.31452.objecter _send_command 11 2024-02-18T00:39:52.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c129200 con 0x7fd64c121c50 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd64a7fc700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd64c121c50 0x7fd64c1220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c121c50 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.686+0000 7fd64a7fc700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd64c121c50 0x7fd64c1220b0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.687+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c129200 con 0x7fd64c121c50 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.687+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.687+0000 7fd633fff700 10 client.31452.objecter _finish_command 11 = system:0 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.687+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c11d910 osd=5 4 2024-02-18T00:39:52.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.687+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 5 11 2024-02-18T00:39:52.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 10 client.31452.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 12 2024-02-18T00:39:52.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 12 2024-02-18T00:39:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c11d910 osd=5 3 2024-02-18T00:39:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 5 12 2024-02-18T00:39:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c11d910 osd=5 4 2024-02-18T00:39:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 10 client.31452.objecter _send_command 12 2024-02-18T00:39:52.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.702+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c0f72b0 con 0x7fd64c121c50 2024-02-18T00:39:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.703+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64c0f72b0 con 0x7fd64c121c50 2024-02-18T00:39:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.703+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.703+0000 7fd633fff700 10 client.31452.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.703+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c11d910 osd=5 4 2024-02-18T00:39:52.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.703+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 5 12 2024-02-18T00:39:52.708 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 10 client.31452.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 13 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd64c0f7930 0x7fd64c0f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 13 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 6 13 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c0f72b0 osd=6 4 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 10 client.31452.objecter _send_command 13 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c12e510 con 0x7fd64c0f7930 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd64b7fe700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd64c0f7930 0x7fd64c0f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c0f7930 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.704+0000 7fd64b7fe700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd64c0f7930 0x7fd64c0f7d70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.705+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c12e510 con 0x7fd64c0f7930 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.705+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.705+0000 7fd633fff700 10 client.31452.objecter _finish_command 13 = system:0 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.705+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c0f72b0 osd=6 4 2024-02-18T00:39:52.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.705+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 6 13 2024-02-18T00:39:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 10 client.31452.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 14 2024-02-18T00:39:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 14 2024-02-18T00:39:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c0f72b0 osd=6 3 2024-02-18T00:39:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 6 14 2024-02-18T00:39:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c0f72b0 osd=6 4 2024-02-18T00:39:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 10 client.31452.objecter _send_command 14 2024-02-18T00:39:52.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.719+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c12e510 con 0x7fd64c0f7930 2024-02-18T00:39:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64c12e510 con 0x7fd64c0f7930 2024-02-18T00:39:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd633fff700 10 client.31452.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c0f72b0 osd=6 4 2024-02-18T00:39:52.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 6 14 2024-02-18T00:39:52.724 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:39:52.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 10 client.31452.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 15 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6812/53025171 conn(0x7fd64c0f8910 0x7fd64c0f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 15 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 7 15 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c12e510 osd=7 4 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 10 client.31452.objecter _send_command 15 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.721+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd64c1318c0 con 0x7fd64c0f8910 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.722+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6812/53025171 conn(0x7fd64c0f8910 0x7fd64c0f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.722+0000 7fd64affd700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6812/53025171 conn(0x7fd64c0f8910 0x7fd64c0f8d50 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.722+0000 7fd633fff700 10 client.31452.objecter ms_handle_connect 0x7fd64c0f8910 2024-02-18T00:39:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.723+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd64c1318c0 con 0x7fd64c0f8910 2024-02-18T00:39:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.723+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:39:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.723+0000 7fd633fff700 10 client.31452.objecter _finish_command 15 = system:0 2024-02-18T00:39:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.723+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c12e510 osd=7 4 2024-02-18T00:39:52.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.723+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 7 15 2024-02-18T00:39:52.737 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:52.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 10 client.31452.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:39:52.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign -1 16 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 20 client.31452.objecter _get_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 15 client.31452.objecter _session_command_op_remove -1 16 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 20 client.31452.objecter get_session s=0x7fd64c12e510 osd=7 3 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 15 client.31452.objecter _session_command_op_assign 7 16 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c12e510 osd=7 4 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 10 client.31452.objecter _send_command 16 2024-02-18T00:39:52.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.738+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd64c12cec0 con 0x7fd64c0f8910 2024-02-18T00:39:52.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.739+0000 7fd633fff700 1 -- 172.21.15.59:0/3617652940 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd64c12cec0 con 0x7fd64c0f8910 2024-02-18T00:39:52.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.739+0000 7fd633fff700 10 client.31452.objecter ms_dispatch 0x7fd64c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:39:52.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.739+0000 7fd633fff700 10 client.31452.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.739+0000 7fd633fff700 20 client.31452.objecter put_session s=0x7fd64c12e510 osd=7 4 2024-02-18T00:39:52.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.739+0000 7fd633fff700 15 client.31452.objecter _session_command_op_remove 7 16 2024-02-18T00:39:52.742 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 librados: watch_flush enter 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 librados: watch_flush exit 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.0 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd64c0fee80 msgr2=0x7fd64c0ff260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd64c0fee80 0x7fd64c0ff260 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c05ee80 osd=0 1 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.1 2024-02-18T00:39:52.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6805/26279110 conn(0x7fd64c0598f0 msgr2=0x7fd64c059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6805/26279110 conn(0x7fd64c0598f0 0x7fd64c059d10 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c028540 osd=1 1 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.2 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd64c11a170 msgr2=0x7fd64c11c560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd64c11a170 0x7fd64c11c560 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027b90 osd=2 1 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.3 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd64c119cd0 msgr2=0x7fd64c11fbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd64c119cd0 0x7fd64c11fbe0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd64b7fe700 1 -- 172.21.15.59:0/3617652940 reap_dead start 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c027a10 osd=3 1 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.4 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd64c1193e0 msgr2=0x7fd64c121250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd64c1193e0 0x7fd64c121250 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c119230 osd=4 1 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.5 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd64b7fe700 1 -- 172.21.15.59:0/3617652940 reap_dead start 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd64c121c50 msgr2=0x7fd64c1220b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd64c121c50 0x7fd64c1220b0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c11d910 osd=5 1 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.6 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd64c0f7930 msgr2=0x7fd64c0f7d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd64c0f7930 0x7fd64c0f7d70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c0f72b0 osd=6 1 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter close_session for osd.7 2024-02-18T00:39:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6812/53025171 conn(0x7fd64c0f8910 msgr2=0x7fd64c0f8d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.137:6812/53025171 conn(0x7fd64c0f8910 0x7fd64c0f8d50 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter put_session s=0x7fd64c12e510 osd=7 1 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 20 client.31452.objecter shutdown clearing up homeless session... 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 10 client.31452.objecter successfully canceled tick 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd634062c90 msgr2=0x7fd634065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd634062c90 0x7fd634065150 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 rx=0x7fd64c109ae0 tx=0x7fd640009040).stop 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0f6880 msgr2=0x7fd64c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 --2- 172.21.15.59:0/3617652940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd64c0f6880 0x7fd64c10e1a0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 rx=0x7fd63c009440 tx=0x7fd63c00b040).stop 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd64b7fe700 1 -- 172.21.15.59:0/3617652940 reap_dead start 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 shutdown_connections 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 >> 172.21.15.59:0/3617652940 conn(0x7fd64c06ffc0 msgr2=0x7fd64c0fb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 shutdown_connections 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 -- 172.21.15.59:0/3617652940 wait complete. 2024-02-18T00:39:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:52.740+0000 7fd651270700 1 librados: shutdown 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.748 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:39:52.768 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:52.916 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:52.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:52.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:53.020 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:51.744880+0000 osd.7 (osd.7) 107 : cluster [DBG] 62.14 scrub starts 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:51.746708+0000 osd.7 (osd.7) 108 : cluster [DBG] 62.14 scrub ok 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:51.926799+0000 mgr.y (mgr.24454) 448 : cluster [DBG] pgmap v719: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s; 8 B/s, 1 objects/s recovering 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:52.598563+0000 mon.a (mon.0) 2673 : audit [DBG] from='client.? 172.21.15.59:0/3617652940' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:51.744880+0000 osd.7 (osd.7) 107 : cluster [DBG] 62.14 scrub starts 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:51.746708+0000 osd.7 (osd.7) 108 : cluster [DBG] 62.14 scrub ok 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:51.926799+0000 mgr.y (mgr.24454) 448 : cluster [DBG] pgmap v719: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s; 8 B/s, 1 objects/s recovering 2024-02-18T00:39:53.049 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:52.598563+0000 mon.a (mon.0) 2673 : audit [DBG] from='client.? 172.21.15.59:0/3617652940' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:53.051 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:53.124 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:53.127 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:53.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:53.165 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:39:53.165 INFO:teuthology.orchestra.run.smithi137.stdout: "success": "filestore_inject_stall = '3' (not observed, change may require restart) " 2024-02-18T00:39:53.166 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:39:53.190 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:51.744880+0000 osd.7 (osd.7) 107 : cluster [DBG] 62.14 scrub starts 2024-02-18T00:39:53.191 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:51.746708+0000 osd.7 (osd.7) 108 : cluster [DBG] 62.14 scrub ok 2024-02-18T00:39:53.191 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:51.926799+0000 mgr.y (mgr.24454) 448 : cluster [DBG] pgmap v719: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s; 8 B/s, 1 objects/s recovering 2024-02-18T00:39:53.191 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:52.598563+0000 mon.a (mon.0) 2673 : audit [DBG] from='client.? 172.21.15.59:0/3617652940' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:39:53.228 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:53.258 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:53.406 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:53.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:53.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:53.509 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:53.537 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:53.838 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:53.889 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:53.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:53.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:54.018 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.228521+0000 osd.4 (osd.4) 281 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.231442+0000 osd.4 (osd.4) 282 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.686555+0000 osd.7 (osd.7) 109 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.687959+0000 osd.7 (osd.7) 110 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.228521+0000 osd.4 (osd.4) 281 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.231442+0000 osd.4 (osd.4) 282 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.686555+0000 osd.7 (osd.7) 109 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:39:54.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.687959+0000 osd.7 (osd.7) 110 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:39:54.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.228521+0000 osd.4 (osd.4) 281 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:39:54.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.231442+0000 osd.4 (osd.4) 282 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:39:54.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.686555+0000 osd.7 (osd.7) 109 : cluster [DBG] 6.1c scrub starts 2024-02-18T00:39:54.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.687959+0000 osd.7 (osd.7) 110 : cluster [DBG] 6.1c scrub ok 2024-02-18T00:39:54.368 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:54.371 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:54.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:54.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:54.501 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:54.631 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:54.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:54.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:54.735 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:54.767 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:54.789 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:54.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:54.792 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:54.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.575297+0000 osd.5 (osd.5) 199 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:39:54.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.576272+0000 osd.5 (osd.5) 200 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:39:54.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.927284+0000 mgr.y (mgr.24454) 449 : cluster [DBG] pgmap v720: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:39:54.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.939688+0000 osd.6 (osd.6) 227 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:39:54.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:53.940526+0000 osd.6 (osd.6) 228 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:39:54.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:54.921 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:54.940 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:54.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:54.943 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:55.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:55.078 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:55.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.575297+0000 osd.5 (osd.5) 199 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:39:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.576272+0000 osd.5 (osd.5) 200 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:39:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.927284+0000 mgr.y (mgr.24454) 449 : cluster [DBG] pgmap v720: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:39:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.939688+0000 osd.6 (osd.6) 227 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:39:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:53.940526+0000 osd.6 (osd.6) 228 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:39:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.575297+0000 osd.5 (osd.5) 199 : cluster [DBG] 6.19 scrub starts 2024-02-18T00:39:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.576272+0000 osd.5 (osd.5) 200 : cluster [DBG] 6.19 scrub ok 2024-02-18T00:39:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.927284+0000 mgr.y (mgr.24454) 449 : cluster [DBG] pgmap v720: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:39:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.939688+0000 osd.6 (osd.6) 227 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:39:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:53.940526+0000 osd.6 (osd.6) 228 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:39:55.931 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:55.934 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:55.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:56.035 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:56.067 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:56.088 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:56.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:56.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:56.192 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:39:56.216 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:39:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:39:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:39:56.217 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:39:56.217 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.472585+0000 osd.1 (osd.1) 239 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.475696+0000 osd.1 (osd.1) 240 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.714355+0000 osd.0 (osd.0) 224 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.714595+0000 osd.0 (osd.0) 225 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.728650+0000 osd.7 (osd.7) 111 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.729669+0000 osd.7 (osd.7) 112 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.854572+0000 osd.3 (osd.3) 261 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.855448+0000 osd.3 (osd.3) 262 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:39:56.220 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.889164+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.963692+0000 osd.6 (osd.6) 229 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:54.964672+0000 osd.6 (osd.6) 230 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:55.162286+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:55.195648+0000 osd.4 (osd.4) 283 : cluster [DBG] 4.c scrub starts 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:55.199124+0000 osd.4 (osd.4) 284 : cluster [DBG] 4.c scrub ok 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:55.743350+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:55.743656+0000 mon.a (mon.0) 2676 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:55.744125+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-71"}]: dispatch 2024-02-18T00:39:56.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:55.744373+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-71"}]: dispatch 2024-02-18T00:39:56.222 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.472585+0000 osd.1 (osd.1) 239 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.475696+0000 osd.1 (osd.1) 240 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.714355+0000 osd.0 (osd.0) 224 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.714595+0000 osd.0 (osd.0) 225 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.728650+0000 osd.7 (osd.7) 111 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.729669+0000 osd.7 (osd.7) 112 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.854572+0000 osd.3 (osd.3) 261 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.855448+0000 osd.3 (osd.3) 262 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.889164+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.963692+0000 osd.6 (osd.6) 229 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:39:56.239 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:54.964672+0000 osd.6 (osd.6) 230 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:55.162286+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:55.195648+0000 osd.4 (osd.4) 283 : cluster [DBG] 4.c scrub starts 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:55.199124+0000 osd.4 (osd.4) 284 : cluster [DBG] 4.c scrub ok 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:55.743350+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:55.743656+0000 mon.a (mon.0) 2676 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:55.744125+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-71"}]: dispatch 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:55.744373+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-71"}]: dispatch 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.472585+0000 osd.1 (osd.1) 239 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.475696+0000 osd.1 (osd.1) 240 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.714355+0000 osd.0 (osd.0) 224 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:56.240 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.714595+0000 osd.0 (osd.0) 225 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.728650+0000 osd.7 (osd.7) 111 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.729669+0000 osd.7 (osd.7) 112 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.854572+0000 osd.3 (osd.3) 261 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.855448+0000 osd.3 (osd.3) 262 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.889164+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.963692+0000 osd.6 (osd.6) 229 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:54.964672+0000 osd.6 (osd.6) 230 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:55.162286+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:55.195648+0000 osd.4 (osd.4) 283 : cluster [DBG] 4.c scrub starts 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:55.199124+0000 osd.4 (osd.4) 284 : cluster [DBG] 4.c scrub ok 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:55.743350+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:55.743656+0000 mon.a (mon.0) 2676 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:55.744125+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-71"}]: dispatch 2024-02-18T00:39:56.241 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:55.744373+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-71"}]: dispatch 2024-02-18T00:39:56.242 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:39:56.245 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:56.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:39:56.346 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:56.374 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:56.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:39:56.393 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:56.396 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:56.396 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:56.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:56.535 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:56.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 -- 172.21.15.59:0/2927934722 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500fd510 msgr2=0x7f5d500f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 --2- 172.21.15.59:0/2927934722 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500fd510 0x7f5d500f6d40 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 rx=0x7f5d44004870 tx=0x7f5d44009cb0).stop 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 -- 172.21.15.59:0/2927934722 shutdown_connections 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 --2- 172.21.15.59:0/2927934722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d500fcbb0 0x7f5d500fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 --2- 172.21.15.59:0/2927934722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d500f7400 0x7f5d50103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 --2- 172.21.15.59:0/2927934722 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500fd510 0x7f5d500f6d40 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 -- 172.21.15.59:0/2927934722 >> 172.21.15.59:0/2927934722 conn(0x7f5d500f2820 msgr2=0x7f5d500f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 -- 172.21.15.59:0/2927934722 shutdown_connections 2024-02-18T00:39:56.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.900+0000 7f5d56886700 1 -- 172.21.15.59:0/2927934722 wait complete. 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 librados: starting msgr at 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 librados: starting objecter 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 Processor -- start 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 -- start start 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 librados: setting wanted keys 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 librados: calling monclient init 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500f7400 0x7f5d50107140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d500fcbb0 0x7f5d50107630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:56.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d500fd510 0x7f5d50107b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:56.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d501167c0 con 0x7f5d500fd510 2024-02-18T00:39:56.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50116900 con 0x7f5d500fcbb0 2024-02-18T00:39:56.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d56886700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50116a70 con 0x7f5d500f7400 2024-02-18T00:39:56.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d54e23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d500fd510 0x7f5d50107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:56.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.901+0000 7f5d4ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500f7400 0x7f5d50107140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:56.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d54e23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d500fd510 0x7f5d50107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56866/0 (socket says 172.21.15.59:56866) 2024-02-18T00:39:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500f7400 0x7f5d50107140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:34734/0 (socket says 172.21.15.59:34734) 2024-02-18T00:39:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d54e23700 1 -- 172.21.15.59:0/3015841278 learned_addr learned my addr 172.21.15.59:0/3015841278 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:39:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4f7fe700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d500fcbb0 0x7f5d50107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4ffff700 1 -- 172.21.15.59:0/3015841278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d500fcbb0 msgr2=0x7f5d50107630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4ffff700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d500fcbb0 0x7f5d50107630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4ffff700 1 -- 172.21.15.59:0/3015841278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d500fd510 msgr2=0x7f5d50107b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:56.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4ffff700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d500fd510 0x7f5d50107b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4ffff700 1 -- 172.21.15.59:0/3015841278 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44005040 con 0x7f5d500f7400 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4f7fe700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d500fcbb0 0x7f5d50107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4ffff700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500f7400 0x7f5d50107140 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 rx=0x7f5d4000b4e0 tx=0x7f5d40009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4d7fa700 10 client.?.objecter ms_handle_connect 0x7f5d500f7400 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4d7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4d7fa700 1 -- 172.21.15.59:0/3015841278 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40002320 con 0x7f5d500f7400 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d50116d50 con 0x7f5d500f7400 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4d7fa700 1 -- 172.21.15.59:0/3015841278 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d40002950 con 0x7f5d500f7400 2024-02-18T00:39:56.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d4d7fa700 1 -- 172.21.15.59:0/3015841278 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d400129c0 con 0x7f5d500f7400 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d56886700 10 client.41261.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d50117230 con 0x7f5d500f7400 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.902+0000 7f5d56886700 1 librados: init done 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.903+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5005fa60 con 0x7f5d500f7400 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.903+0000 7f5d4d7fa700 1 -- 172.21.15.59:0/3015841278 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5d40002480 con 0x7f5d500f7400 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4d7fa700 1 --2- 172.21.15.59:0/3015841278 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d38062c90 0x7f5d38065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4d7fa700 1 -- 172.21.15.59:0/3015841278 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(477..477 src has 180..477) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f5d4007d070 con 0x7f5d500f7400 2024-02-18T00:39:56.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4d7fa700 10 client.41261.objecter ms_dispatch 0x7f5d50001550 osd_map(477..477 src has 180..477) v4 2024-02-18T00:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4d7fa700 3 client.41261.objecter handle_osd_map got epochs [477,477] > 0 2024-02-18T00:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4d7fa700 3 client.41261.objecter handle_osd_map decoding full epoch 477 2024-02-18T00:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4f7fe700 1 --2- 172.21.15.59:0/3015841278 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d38062c90 0x7f5d38065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4d7fa700 20 client.41261.objecter dump_active .. 0 homeless 2024-02-18T00:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4d7fa700 10 client.41261.objecter ms_handle_connect 0x7f5d38062c90 2024-02-18T00:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.904+0000 7f5d4f7fe700 1 --2- 172.21.15.59:0/3015841278 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d38062c90 0x7f5d38065150 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 rx=0x7f5d44004ee0 tx=0x7f5d4400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:39:56.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:56.908+0000 7f5d4d7fa700 1 -- 172.21.15.59:0/3015841278 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5d400506a0 con 0x7f5d500f7400 2024-02-18T00:39:56.992 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:56.995 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:56.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:56.999 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:57.096 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.098+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f5d50061440 con 0x7f5d500f7400 2024-02-18T00:39:57.129 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:57.151 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:57.153 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:57.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:57.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.174+0000 7f5d4d7fa700 1 -- 172.21.15.59:0/3015841278 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v478) v1 ==== 101+0+0 (secure 0 0 0) 0x7f5d40050290 con 0x7f5d500f7400 2024-02-18T00:39:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.177+0000 7f5d56886700 10 librados: watch_flush enter 2024-02-18T00:39:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.177+0000 7f5d56886700 10 librados: watch_flush exit 2024-02-18T00:39:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.177+0000 7f5d56886700 20 client.41261.objecter shutdown clearing up homeless session... 2024-02-18T00:39:57.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.177+0000 7f5d56886700 10 client.41261.objecter successfully canceled tick 2024-02-18T00:39:57.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.179+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d38062c90 msgr2=0x7f5d38065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:57.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.179+0000 7f5d56886700 1 --2- 172.21.15.59:0/3015841278 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d38062c90 0x7f5d38065150 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 rx=0x7f5d44004ee0 tx=0x7f5d4400b040).stop 2024-02-18T00:39:57.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.179+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500f7400 msgr2=0x7f5d50107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:39:57.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.179+0000 7f5d56886700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500f7400 0x7f5d50107140 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 rx=0x7f5d4000b4e0 tx=0x7f5d40009060).stop 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 shutdown_connections 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d500fd510 0x7f5d50107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5d500f7400 0x7f5d50107140 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 --2- 172.21.15.59:0/3015841278 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d38062c90 0x7f5d38065150 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 --2- 172.21.15.59:0/3015841278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d500fcbb0 0x7f5d50107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 >> 172.21.15.59:0/3015841278 conn(0x7f5d500f2820 msgr2=0x7f5d500fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 shutdown_connections 2024-02-18T00:39:57.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 -- 172.21.15.59:0/3015841278 wait complete. 2024-02-18T00:39:57.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:39:57.180+0000 7f5d56886700 1 librados: shutdown 2024-02-18T00:39:57.184 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:55.461274+0000 osd.1 (osd.1) 241 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:55.464732+0000 osd.1 (osd.1) 242 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:55.720646+0000 osd.7 (osd.7) 113 : cluster [DBG] 62.6 scrub starts 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:55.722595+0000 osd.7 (osd.7) 114 : cluster [DBG] 62.6 scrub ok 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:55.928144+0000 mgr.y (mgr.24454) 450 : cluster [DBG] pgmap v722: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:56.170118+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:56.695922+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:57.100121+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.59:0/3015841278' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:57.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:57.100440+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:55.461274+0000 osd.1 (osd.1) 241 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:55.464732+0000 osd.1 (osd.1) 242 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:55.720646+0000 osd.7 (osd.7) 113 : cluster [DBG] 62.6 scrub starts 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:55.722595+0000 osd.7 (osd.7) 114 : cluster [DBG] 62.6 scrub ok 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:55.928144+0000 mgr.y (mgr.24454) 450 : cluster [DBG] pgmap v722: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:56.170118+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:56.695922+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:57.100121+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.59:0/3015841278' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:57.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:57.100440+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:57.255 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:39:57.283 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:39:57.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:57.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:39:57.406 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:57.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:55.461274+0000 osd.1 (osd.1) 241 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:39:57.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:55.464732+0000 osd.1 (osd.1) 242 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:39:57.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:55.720646+0000 osd.7 (osd.7) 113 : cluster [DBG] 62.6 scrub starts 2024-02-18T00:39:57.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:55.722595+0000 osd.7 (osd.7) 114 : cluster [DBG] 62.6 scrub ok 2024-02-18T00:39:57.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:55.928144+0000 mgr.y (mgr.24454) 450 : cluster [DBG] pgmap v722: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:39:57.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:56.170118+0000 mon.a (mon.0) 2678 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-18T00:39:57.433 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:56.695922+0000 osd.7 (osd.7) 115 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:39:57.433 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:57.100121+0000 mon.c (mon.2) 450 : audit [INF] from='client.? 172.21.15.59:0/3015841278' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:57.433 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:57.100440+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:39:57.433 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:57.453 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:57.455 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:57.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:57.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:39:57.588 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:57.640 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:39:57.642 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:57.643 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:39:57.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:39:57.773 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:57.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:57.796 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:39:57.796 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:39:57.897 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:57.930 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:57.951 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:57.954 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:57.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:58.055 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:58.086 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:58.106 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:58.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:58.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:58.239 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:56.617268+0000 osd.5 (osd.5) 201 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:39:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:56.621785+0000 osd.5 (osd.5) 202 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:39:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:56.697148+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:39:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.165023+0000 mon.a (mon.0) 2680 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:39:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:57.172055+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.172112+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-18T00:39:58.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:57.178732+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:58.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:57.179650+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:58.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.210252+0000 osd.4 (osd.4) 285 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:39:58.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.211161+0000 osd.4 (osd.4) 286 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:39:58.424 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:56.617268+0000 osd.5 (osd.5) 201 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:56.621785+0000 osd.5 (osd.5) 202 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:56.697148+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.165023+0000 mon.a (mon.0) 2680 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:57.172055+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.172112+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:57.178732+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:57.179650+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.210252+0000 osd.4 (osd.4) 285 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:39:58.425 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.211161+0000 osd.4 (osd.4) 286 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:56.617268+0000 osd.5 (osd.5) 201 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:56.621785+0000 osd.5 (osd.5) 202 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:56.697148+0000 osd.7 (osd.7) 116 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.165023+0000 mon.a (mon.0) 2680 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:57.172055+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.172112+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:57.178732+0000 mon.c (mon.2) 451 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:58.426 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:57.179650+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:39:58.427 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.210252+0000 osd.4 (osd.4) 285 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:39:58.427 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.211161+0000 osd.4 (osd.4) 286 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:39:58.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-18T00:39:58.788 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:58.791 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:58.791 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:58.887 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:39:58.887 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:39:58.887 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-02-18T00:39:58.888 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd test-reweight-by-utilization 110 1.0 2024-02-18T00:39:58.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:58.925 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:58.945 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:58.948 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:58.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:59.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:39:59.077 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:39:59.181 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:59.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:39:59.283 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:39:59.313 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:59.333 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:39:59.336 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:59.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:39:59.438 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:39:59.464 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.389172+0000 osd.1 (osd.1) 243 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:39:59.464 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.390265+0000 osd.1 (osd.1) 244 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:39:59.464 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.578752+0000 osd.5 (osd.5) 203 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.582206+0000 osd.5 (osd.5) 204 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:57.928561+0000 mgr.y (mgr.24454) 451 : cluster [DBG] pgmap v725: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:58.181981+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:58.182109+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:39:58.186555+0000 mon.c (mon.2) 452 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:58.635487+0000 osd.4 (osd.4) 287 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:58.635736+0000 osd.4 (osd.4) 288 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:58.670338+0000 osd.7 (osd.7) 117 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:39:59.465 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:39:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:58.671322+0000 osd.7 (osd.7) 118 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:39:59.466 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:39:59.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:59.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.389172+0000 osd.1 (osd.1) 243 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.390265+0000 osd.1 (osd.1) 244 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.578752+0000 osd.5 (osd.5) 203 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.582206+0000 osd.5 (osd.5) 204 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:57.928561+0000 mgr.y (mgr.24454) 451 : cluster [DBG] pgmap v725: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:58.181981+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:58.182109+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-18T00:39:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:39:58.186555+0000 mon.c (mon.2) 452 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:59.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:58.635487+0000 osd.4 (osd.4) 287 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:59.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:58.635736+0000 osd.4 (osd.4) 288 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:59.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:58.670338+0000 osd.7 (osd.7) 117 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:39:59.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:58.671322+0000 osd.7 (osd.7) 118 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:39:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.389172+0000 osd.1 (osd.1) 243 : cluster [DBG] 6.1d scrub starts 2024-02-18T00:39:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.390265+0000 osd.1 (osd.1) 244 : cluster [DBG] 6.1d scrub ok 2024-02-18T00:39:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.578752+0000 osd.5 (osd.5) 203 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.582206+0000 osd.5 (osd.5) 204 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:57.928561+0000 mgr.y (mgr.24454) 451 : cluster [DBG] pgmap v725: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:58.181981+0000 mon.a (mon.0) 2684 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:58.182109+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:39:58.186555+0000 mon.c (mon.2) 452 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:58.635487+0000 osd.4 (osd.4) 287 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:58.635736+0000 osd.4 (osd.4) 288 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:58.670338+0000 osd.7 (osd.7) 117 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:39:59.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:39:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:58.671322+0000 osd.7 (osd.7) 118 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:39:59.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:39:59.618 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:39:59.868 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:39:59.874 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:39:59.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:39:59.877 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:39:59.975 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:00.007 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:00.029 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:00.032 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:00.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:00.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:00.188 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:00.193 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:00.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:00.254 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:40:00.226+0000 7ff591ade700 -1 snap_mapper.add_oid found existing snaps mapped on 62:1c09f51f:test-rados-api-smithi059-82957-74::foo:24, removing 2024-02-18T00:40:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:58.894627+0000 osd.6 (osd.6) 231 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:40:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:58.895680+0000 osd.6 (osd.6) 232 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:40:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:59.190519+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-18T00:40:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:00.000142+0000 mon.a (mon.0) 2687 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-18T00:40:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:00.000180+0000 mon.a (mon.0) 2688 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-18T00:40:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:00.000198+0000 mon.a (mon.0) 2689 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-18T00:40:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:00.000213+0000 mon.a (mon.0) 2690 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-18T00:40:00.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:00.324 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:00.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:40:00.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:58.894627+0000 osd.6 (osd.6) 231 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:40:00.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:58.895680+0000 osd.6 (osd.6) 232 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:40:00.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:59.190519+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-18T00:40:00.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:00.000142+0000 mon.a (mon.0) 2687 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-18T00:40:00.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:00.000180+0000 mon.a (mon.0) 2688 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-18T00:40:00.456 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:00.000198+0000 mon.a (mon.0) 2689 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-18T00:40:00.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:00.000213+0000 mon.a (mon.0) 2690 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-18T00:40:00.457 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2[57675]: debug 2024-02-18T00:40:00.226+0000 7f0e55e8f700 -1 snap_mapper.add_oid found existing snaps mapped on 62:1c09f51f:test-rados-api-smithi059-82957-74::foo:24, removing 2024-02-18T00:40:00.457 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:40:00.226+0000 7fe33517b700 -1 snap_mapper.add_oid found existing snaps mapped on 62:1c09f51f:test-rados-api-smithi059-82957-74::foo:24, removing 2024-02-18T00:40:00.457 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:58.894627+0000 osd.6 (osd.6) 231 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:40:00.458 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:58.895680+0000 osd.6 (osd.6) 232 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:40:00.458 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:59.190519+0000 mon.a (mon.0) 2686 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-18T00:40:00.458 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:00.000142+0000 mon.a (mon.0) 2687 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-18T00:40:00.458 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:00.000180+0000 mon.a (mon.0) 2688 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-18T00:40:00.458 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:00.000198+0000 mon.a (mon.0) 2689 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-18T00:40:00.458 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:00.000213+0000 mon.a (mon.0) 2690 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-18T00:40:00.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 -- 172.21.15.59:0/2294707783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fd510 msgr2=0x7f54740f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2294707783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fd510 0x7f54740f6d40 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 rx=0x7f5464004870 tx=0x7f5464009cb0).stop 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 -- 172.21.15.59:0/2294707783 shutdown_connections 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2294707783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54740f7400 0x7f5474103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2294707783 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54740fcbb0 0x7f54740fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2294707783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fd510 0x7f54740f6d40 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 -- 172.21.15.59:0/2294707783 >> 172.21.15.59:0/2294707783 conn(0x7f54740f2820 msgr2=0x7f54740f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 -- 172.21.15.59:0/2294707783 shutdown_connections 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 -- 172.21.15.59:0/2294707783 wait complete. 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 librados: starting msgr at 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.489+0000 7f547b8e3700 1 librados: starting objecter 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 Processor -- start 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 -- start start 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 librados: setting wanted keys 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 librados: calling monclient init 2024-02-18T00:40:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54740f7400 0x7f547410e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fcbb0 0x7f547410e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54740fd510 0x7f5474108390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5474108880 con 0x7f54740fcbb0 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54741089f0 con 0x7f54740f7400 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547b8e3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5474108bb0 con 0x7f54740fd510 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5478e7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fcbb0 0x7f547410e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5478e7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fcbb0 0x7f547410e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46758/0 (socket says 172.21.15.59:46758) 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5478e7e700 1 -- 172.21.15.59:0/2493469396 learned_addr learned my addr 172.21.15.59:0/2493469396 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54740f7400 0x7f547410e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54740fd510 0x7f5474108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54740f7400 msgr2=0x7f547410e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54740f7400 0x7f547410e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fcbb0 msgr2=0x7f547410e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fcbb0 0x7f547410e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470005040 con 0x7f54740fd510 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5478e7e700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54740fcbb0 0x7f547410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54740f7400 0x7f547410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54740fd510 0x7f5474108390 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7f546c00d8d0 tx=0x7f546c004db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f546a7fc700 10 client.?.objecter ms_handle_connect 0x7f54740fd510 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.490+0000 7f546a7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.491+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c004020 con 0x7f54740fd510 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.491+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546c00f040 con 0x7f54740fd510 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.491+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5464005040 con 0x7f54740fd510 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.491+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c003430 con 0x7f54740fd510 2024-02-18T00:40:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.491+0000 7f547b8e3700 10 client.41267.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.491+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54741091f0 con 0x7f54740fd510 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.491+0000 7f547b8e3700 1 librados: init done 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.492+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f547405fa60 con 0x7f54740fd510 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.492+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f546c004180 con 0x7f54740fd510 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.492+0000 7f546a7fc700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5460062c90 0x7f5460065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(481..481 src has 180..481) v4 ==== 8151+0+0 (secure 0 0 0) 0x7f546c07cc60 con 0x7f54740fd510 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 osd_map(481..481 src has 180..481) v4 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 3 client.41267.objecter handle_osd_map got epochs [481,481] > 0 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 3 client.41267.objecter handle_osd_map decoding full epoch 481 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5460062c90 0x7f5460065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 20 client.41267.objecter dump_active .. 0 homeless 2024-02-18T00:40:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v481) v1 ==== 54+0+16 (secure 0 0 0) 0x7f546c07f280 con 0x7f54740fd510 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 10 client.41267.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 1 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6801/3700238977 conn(0x7f54740ffed0 0x7f54741002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 1 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 0 1 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547405ee80 osd=0 4 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 10 client.41267.objecter _send_command 1 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54741060e0 con 0x7f54740ffed0 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f5460062c90 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5460062c90 0x7f5460065150 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 rx=0x7f5470009760 tx=0x7f547000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f5478e7e700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6801/3700238977 conn(0x7f54740ffed0 0x7f54741002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f54740ffed0 2024-02-18T00:40:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.493+0000 7f5478e7e700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6801/3700238977 conn(0x7f54740ffed0 0x7f54741002b0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.494+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54741060e0 con 0x7f54740ffed0 2024-02-18T00:40:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.494+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:40:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.494+0000 7f546a7fc700 10 client.41267.objecter _finish_command 1 = system:0 2024-02-18T00:40:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.494+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547405ee80 osd=0 4 2024-02-18T00:40:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.494+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 0 1 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 10 client.41267.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 2 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 2 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547405ee80 osd=0 3 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 0 2 2024-02-18T00:40:00.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547405ee80 osd=0 4 2024-02-18T00:40:00.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 10 client.41267.objecter _send_command 2 2024-02-18T00:40:00.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.509+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54740273c0 con 0x7f54740ffed0 2024-02-18T00:40:00.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54740273c0 con 0x7f54740ffed0 2024-02-18T00:40:00.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f546a7fc700 10 client.41267.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547405ee80 osd=0 4 2024-02-18T00:40:00.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 0 2 2024-02-18T00:40:00.513 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:40:00.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 10 client.41267.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 3 2024-02-18T00:40:00.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6805/26279110 conn(0x7f54740598f0 0x7f5474059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 3 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 1 3 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474028710 osd=1 4 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 10 client.41267.objecter _send_command 3 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.511+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f547411b800 con 0x7f54740598f0 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6805/26279110 conn(0x7f54740598f0 0x7f5474059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f54740598f0 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6805/26279110 conn(0x7f54740598f0 0x7f5474059cd0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f547411b800 con 0x7f54740598f0 2024-02-18T00:40:00.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:40:00.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f546a7fc700 10 client.41267.objecter _finish_command 3 = system:0 2024-02-18T00:40:00.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474028710 osd=1 4 2024-02-18T00:40:00.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.512+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 1 3 2024-02-18T00:40:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:00.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:00.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:00.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 10 client.41267.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 4 2024-02-18T00:40:00.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 4 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474028710 osd=1 3 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 1 4 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474028710 osd=1 4 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 10 client.41267.objecter _send_command 4 2024-02-18T00:40:00.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.529+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5474059360 con 0x7f54740598f0 2024-02-18T00:40:00.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5474059360 con 0x7f54740598f0 2024-02-18T00:40:00.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f546a7fc700 10 client.41267.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474028710 osd=1 4 2024-02-18T00:40:00.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 1 4 2024-02-18T00:40:00.532 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:40:00.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 10 client.41267.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 5 2024-02-18T00:40:00.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5474028270 0x7f547411a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 5 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 2 5 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.530+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474059770 osd=2 4 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.531+0000 7f547b8e3700 10 client.41267.objecter _send_command 5 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.531+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f547411a710 con 0x7f5474028270 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.531+0000 7f5478e7e700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5474028270 0x7f547411a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.531+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f5474028270 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.531+0000 7f5478e7e700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5474028270 0x7f547411a220 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.532+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f547411a710 con 0x7f5474028270 2024-02-18T00:40:00.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.532+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:40:00.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.532+0000 7f546a7fc700 10 client.41267.objecter _finish_command 5 = system:0 2024-02-18T00:40:00.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.532+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474059770 osd=2 4 2024-02-18T00:40:00.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.532+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 2 5 2024-02-18T00:40:00.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 10 client.41267.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 6 2024-02-18T00:40:00.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 6 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474059770 osd=2 3 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 2 6 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474059770 osd=2 4 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 10 client.41267.objecter _send_command 6 2024-02-18T00:40:00.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.547+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f54740277d0 con 0x7f5474028270 2024-02-18T00:40:00.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.549+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f54740277d0 con 0x7f5474028270 2024-02-18T00:40:00.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.549+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.549+0000 7f546a7fc700 10 client.41267.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.549+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474059770 osd=2 4 2024-02-18T00:40:00.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.549+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 2 6 2024-02-18T00:40:00.551 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:40:00.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 10 client.41267.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 7 2024-02-18T00:40:00.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6813/1805700964 conn(0x7f547411d980 0x7f547411fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 7 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 3 7 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474027be0 osd=3 4 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 10 client.41267.objecter _send_command 7 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f547411b680 con 0x7f547411d980 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6813/1805700964 conn(0x7f547411d980 0x7f547411fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f547411d980 2024-02-18T00:40:00.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.550+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6813/1805700964 conn(0x7f547411d980 0x7f547411fd70 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.551+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f546c07f260 con 0x7f547411d980 2024-02-18T00:40:00.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.551+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:40:00.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.551+0000 7f546a7fc700 10 client.41267.objecter _finish_command 7 = system:0 2024-02-18T00:40:00.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.551+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474027be0 osd=3 4 2024-02-18T00:40:00.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.551+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 3 7 2024-02-18T00:40:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 10 client.41267.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 8 2024-02-18T00:40:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 8 2024-02-18T00:40:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474027be0 osd=3 3 2024-02-18T00:40:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 3 8 2024-02-18T00:40:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474027be0 osd=3 4 2024-02-18T00:40:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 10 client.41267.objecter _send_command 8 2024-02-18T00:40:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.566+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f547411ab70 con 0x7f547411d980 2024-02-18T00:40:00.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f547411ab70 con 0x7f547411d980 2024-02-18T00:40:00.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f546a7fc700 10 client.41267.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474027be0 osd=3 4 2024-02-18T00:40:00.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 3 8 2024-02-18T00:40:00.571 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 10 client.41267.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 9 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5474119d80 0x7f54741213e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 9 2024-02-18T00:40:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 4 9 2024-02-18T00:40:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ab90 osd=4 4 2024-02-18T00:40:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 10 client.41267.objecter _send_command 9 2024-02-18T00:40:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f547411ae70 con 0x7f5474119d80 2024-02-18T00:40:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.569+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5474119d80 0x7f54741213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.570+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f5474119d80 2024-02-18T00:40:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.570+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5474119d80 0x7f54741213e0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.570+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f547411ae70 con 0x7f5474119d80 2024-02-18T00:40:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.570+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:40:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.570+0000 7f546a7fc700 10 client.41267.objecter _finish_command 9 = system:0 2024-02-18T00:40:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.570+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547411ab90 osd=4 4 2024-02-18T00:40:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.570+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 4 9 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 10 client.41267.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 10 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 10 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547411ab90 osd=4 3 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 4 10 2024-02-18T00:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ab90 osd=4 4 2024-02-18T00:40:00.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 10 client.41267.objecter _send_command 10 2024-02-18T00:40:00.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.585+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f547411ae70 con 0x7f5474119d80 2024-02-18T00:40:00.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f547411ae70 con 0x7f5474119d80 2024-02-18T00:40:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f546a7fc700 10 client.41267.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547411ab90 osd=4 4 2024-02-18T00:40:00.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 4 10 2024-02-18T00:40:00.589 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:40:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 10 client.41267.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 11 2024-02-18T00:40:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5474121de0 0x7f5474122240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 11 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 5 11 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ae70 osd=5 4 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 10 client.41267.objecter _send_command 11 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.587+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54741291f0 con 0x7f5474121de0 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.588+0000 7f5478e7e700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5474121de0 0x7f5474122240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.588+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f5474121de0 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.588+0000 7f5478e7e700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5474121de0 0x7f5474122240 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.589+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f54741291f0 con 0x7f5474121de0 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.589+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.589+0000 7f546a7fc700 10 client.41267.objecter _finish_command 11 = system:0 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.589+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547411ae70 osd=5 4 2024-02-18T00:40:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.589+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 5 11 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 10 client.41267.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 12 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 12 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547411ae70 osd=5 3 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 5 12 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ae70 osd=5 4 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 10 client.41267.objecter _send_command 12 2024-02-18T00:40:00.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.603+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5474126a90 con 0x7f5474121de0 2024-02-18T00:40:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5474126a90 con 0x7f5474121de0 2024-02-18T00:40:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f546a7fc700 10 client.41267.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547411ae70 osd=5 4 2024-02-18T00:40:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 5 12 2024-02-18T00:40:00.607 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:40:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 10 client.41267.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 13 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5474127170 0x7f5474129580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 13 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 6 13 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474126a90 osd=6 4 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 10 client.41267.objecter _send_command 13 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.605+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f547412c350 con 0x7f5474127170 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.606+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5474127170 0x7f5474129580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.606+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f5474127170 2024-02-18T00:40:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.606+0000 7f5479e80700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5474127170 0x7f5474129580 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.607+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f547412c350 con 0x7f5474127170 2024-02-18T00:40:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.607+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:40:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.607+0000 7f546a7fc700 10 client.41267.objecter _finish_command 13 = system:0 2024-02-18T00:40:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.607+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474126a90 osd=6 4 2024-02-18T00:40:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.607+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 6 13 2024-02-18T00:40:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 10 client.41267.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 14 2024-02-18T00:40:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 14 2024-02-18T00:40:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f5474126a90 osd=6 3 2024-02-18T00:40:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 6 14 2024-02-18T00:40:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474126a90 osd=6 4 2024-02-18T00:40:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 10 client.41267.objecter _send_command 14 2024-02-18T00:40:00.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.621+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f547412c350 con 0x7f5474127170 2024-02-18T00:40:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f547412c350 con 0x7f5474127170 2024-02-18T00:40:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f546a7fc700 10 client.41267.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f5474126a90 osd=6 4 2024-02-18T00:40:00.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 6 14 2024-02-18T00:40:00.625 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:40:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 10 client.41267.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 15 2024-02-18T00:40:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6812/53025171 conn(0x7f547412b650 0x7f547412bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 15 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 7 15 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547412c350 osd=7 4 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 10 client.41267.objecter _send_command 15 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.623+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f547412c140 con 0x7f547412b650 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.624+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6812/53025171 conn(0x7f547412b650 0x7f547412bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.624+0000 7f547967f700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6812/53025171 conn(0x7f547412b650 0x7f547412bad0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.624+0000 7f546a7fc700 10 client.41267.objecter ms_handle_connect 0x7f547412b650 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.625+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f547412c140 con 0x7f547412b650 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.625+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:40:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.625+0000 7f546a7fc700 10 client.41267.objecter _finish_command 15 = system:0 2024-02-18T00:40:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.625+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547412c350 osd=7 4 2024-02-18T00:40:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.625+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 7 15 2024-02-18T00:40:00.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 10 client.41267.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign -1 16 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 20 client.41267.objecter _get_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_remove -1 16 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 20 client.41267.objecter get_session s=0x7f547412c350 osd=7 3 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 15 client.41267.objecter _session_command_op_assign 7 16 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547412c350 osd=7 4 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 10 client.41267.objecter _send_command 16 2024-02-18T00:40:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.642+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f547412c140 con 0x7f547412b650 2024-02-18T00:40:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f546a7fc700 1 -- 172.21.15.59:0/2493469396 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f547412c140 con 0x7f547412b650 2024-02-18T00:40:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f546a7fc700 10 client.41267.objecter ms_dispatch 0x7f5474001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f546a7fc700 10 client.41267.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f546a7fc700 20 client.41267.objecter put_session s=0x7f547412c350 osd=7 4 2024-02-18T00:40:00.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f546a7fc700 15 client.41267.objecter _session_command_op_remove 7 16 2024-02-18T00:40:00.646 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:40:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 librados: watch_flush enter 2024-02-18T00:40:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 librados: watch_flush exit 2024-02-18T00:40:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.0 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6801/3700238977 conn(0x7f54740ffed0 msgr2=0x7f54741002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6801/3700238977 conn(0x7f54740ffed0 0x7f54741002b0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547405ee80 osd=0 1 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.1 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6805/26279110 conn(0x7f54740598f0 msgr2=0x7f5474059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6805/26279110 conn(0x7f54740598f0 0x7f5474059cd0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474028710 osd=1 1 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.2 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5474028270 msgr2=0x7f547411a220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5474028270 0x7f547411a220 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474059770 osd=2 1 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.3 2024-02-18T00:40:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6813/1805700964 conn(0x7f547411d980 msgr2=0x7f547411fd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6813/1805700964 conn(0x7f547411d980 0x7f547411fd70 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474027be0 osd=3 1 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.4 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5474119d80 msgr2=0x7f54741213e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5474119d80 0x7f54741213e0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ab90 osd=4 1 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.5 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5474121de0 msgr2=0x7f5474122240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5474121de0 0x7f5474122240 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547411ae70 osd=5 1 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.6 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5474127170 msgr2=0x7f5474129580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5474127170 0x7f5474129580 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f5474126a90 osd=6 1 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter close_session for osd.7 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6812/53025171 conn(0x7f547412b650 msgr2=0x7f547412bad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.137:6812/53025171 conn(0x7f547412b650 0x7f547412bad0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter put_session s=0x7f547412c350 osd=7 1 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 20 client.41267.objecter shutdown clearing up homeless session... 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 10 client.41267.objecter successfully canceled tick 2024-02-18T00:40:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 reap_dead start 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 reap_dead start 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 reap_dead start 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 reap_dead start 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5460062c90 msgr2=0x7f5460065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5460062c90 0x7f5460065150 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 rx=0x7f5470009760 tx=0x7f547000b040).stop 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f5479e80700 1 -- 172.21.15.59:0/2493469396 reap_dead start 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54740fd510 msgr2=0x7f5474108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54740fd510 0x7f5474108390 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 rx=0x7f546c00d8d0 tx=0x7f546c004db0).stop 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 shutdown_connections 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 --2- 172.21.15.59:0/2493469396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f54740fd510 0x7f5474108390 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.644+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 >> 172.21.15.59:0/2493469396 conn(0x7f54740f2820 msgr2=0x7f5474060e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.645+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 shutdown_connections 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.645+0000 7f547b8e3700 1 -- 172.21.15.59:0/2493469396 wait complete. 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:00.645+0000 7f547b8e3700 1 librados: shutdown 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.653 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:40:00.656 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:00.675 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:00.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:00.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:00.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:00.808 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:00.829 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:00.832 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:00.832 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:00.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:01.065 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:01.065 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:01.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:01.166 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:01.194 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:01.214 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:01.216 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:01.216 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:01.317 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:01.329 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:40:01.329 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:39:59.929216+0000 mgr.y (mgr.24454) 452 : cluster [DBG] pgmap v728: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:01.329 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:00.198081+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-18T00:40:01.329 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:00.243036+0000 mon.c (mon.2) 453 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:01.329 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:00.243368+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:01.330 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:00.243801+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-73"}]: dispatch 2024-02-18T00:40:01.330 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:00.244034+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-73"}]: dispatch 2024-02-18T00:40:01.330 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:00.493845+0000 mon.c (mon.2) 455 : audit [DBG] from='client.? 172.21.15.59:0/2493469396' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:01.343 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:01.362 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:01.364 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:01.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:01.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:01.493 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:39:59.929216+0000 mgr.y (mgr.24454) 452 : cluster [DBG] pgmap v728: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:00.198081+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:00.243036+0000 mon.c (mon.2) 453 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:00.243368+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:00.243801+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-73"}]: dispatch 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:00.244034+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-73"}]: dispatch 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:00.493845+0000 mon.c (mon.2) 455 : audit [DBG] from='client.? 172.21.15.59:0/2493469396' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:39:59.929216+0000 mgr.y (mgr.24454) 452 : cluster [DBG] pgmap v728: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:00.198081+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:00.243036+0000 mon.c (mon.2) 453 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:01.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:00.243368+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:01.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:00.243801+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-73"}]: dispatch 2024-02-18T00:40:01.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:00.244034+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-73"}]: dispatch 2024-02-18T00:40:01.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:00.493845+0000 mon.c (mon.2) 455 : audit [DBG] from='client.? 172.21.15.59:0/2493469396' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:01.495 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:01.620 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:01.623 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:01.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:01.725 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:01.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:40:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:40:01.752 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:01.771 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:01.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:01.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:01.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:01.900 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:01.919 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:01.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:01.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:02.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:02.052 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:02.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:02.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:02.304 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:02.331 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:02.350 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:02.352 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:02.352 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:02.453 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:02.460 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:02] "GET /metrics HTTP/1.1" 200 190020 "" "Prometheus/2.33.4" 2024-02-18T00:40:02.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:00.514777+0000 osd.5 (osd.5) 205 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:40:02.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:00.515560+0000 osd.5 (osd.5) 206 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:40:02.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:01.197616+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-18T00:40:02.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:01.328748+0000 mgr.y (mgr.24454) 453 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:02.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:00.514777+0000 osd.5 (osd.5) 205 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:40:02.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:00.515560+0000 osd.5 (osd.5) 206 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:40:02.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:01.197616+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-18T00:40:02.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:01.328748+0000 mgr.y (mgr.24454) 453 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:02.478 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:00.514777+0000 osd.5 (osd.5) 205 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:40:02.478 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:00.515560+0000 osd.5 (osd.5) 206 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:40:02.478 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:01.197616+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-18T00:40:02.478 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:01.328748+0000 mgr.y (mgr.24454) 453 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:02.480 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:02.499 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:02.501 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:02.501 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:02.602 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:02.628 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:02.646 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:02.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:02.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:02.750 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:02.776 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:02.795 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:02.797 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:02.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:02.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:02.925 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:02.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:02.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:03.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:03.078 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:03.118 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:03.121 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:03.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:03.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:03.224 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:01.447393+0000 osd.1 (osd.1) 245 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:40:03.224 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:01.450317+0000 osd.1 (osd.1) 246 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:40:03.224 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:01.929677+0000 mgr.y (mgr.24454) 454 : cluster [DBG] pgmap v731: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:40:03.224 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:01.993667+0000 osd.2 (osd.2) 201 : cluster [DBG] 62.1f scrub starts 2024-02-18T00:40:03.224 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:01.997225+0000 osd.2 (osd.2) 202 : cluster [DBG] 62.1f scrub ok 2024-02-18T00:40:03.224 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:02.213267+0000 mon.a (mon.0) 2695 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-18T00:40:03.224 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:02.214738+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:03.225 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:02.215382+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:03.225 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:02.836711+0000 osd.6 (osd.6) 233 : cluster [DBG] 62.17 scrub starts 2024-02-18T00:40:03.225 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:02.838257+0000 osd.6 (osd.6) 234 : cluster [DBG] 62.17 scrub ok 2024-02-18T00:40:03.229 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:01.447393+0000 osd.1 (osd.1) 245 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:01.450317+0000 osd.1 (osd.1) 246 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:01.929677+0000 mgr.y (mgr.24454) 454 : cluster [DBG] pgmap v731: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:01.993667+0000 osd.2 (osd.2) 201 : cluster [DBG] 62.1f scrub starts 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:01.997225+0000 osd.2 (osd.2) 202 : cluster [DBG] 62.1f scrub ok 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:02.213267+0000 mon.a (mon.0) 2695 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:02.214738+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:02.215382+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:02.836711+0000 osd.6 (osd.6) 233 : cluster [DBG] 62.17 scrub starts 2024-02-18T00:40:03.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:02.838257+0000 osd.6 (osd.6) 234 : cluster [DBG] 62.17 scrub ok 2024-02-18T00:40:03.259 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:01.447393+0000 osd.1 (osd.1) 245 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:01.450317+0000 osd.1 (osd.1) 246 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:01.929677+0000 mgr.y (mgr.24454) 454 : cluster [DBG] pgmap v731: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:01.993667+0000 osd.2 (osd.2) 201 : cluster [DBG] 62.1f scrub starts 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:01.997225+0000 osd.2 (osd.2) 202 : cluster [DBG] 62.1f scrub ok 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:02.213267+0000 mon.a (mon.0) 2695 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:02.214738+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:02.215382+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:02.836711+0000 osd.6 (osd.6) 233 : cluster [DBG] 62.17 scrub starts 2024-02-18T00:40:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:02.838257+0000 osd.6 (osd.6) 234 : cluster [DBG] 62.17 scrub ok 2024-02-18T00:40:03.508 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:03.511 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:03.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:03.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:03.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:40:03.641 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:02.437925+0000 osd.1 (osd.1) 247 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:02.439075+0000 osd.1 (osd.1) 248 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:02.729643+0000 osd.7 (osd.7) 119 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:02.730681+0000 osd.7 (osd.7) 120 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:03.214056+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.214260+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:03.225096+0000 mon.c (mon.2) 457 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.705700+0000 mon.a (mon.0) 2699 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.813119+0000 osd.6 (osd.6) 235 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:40:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.814472+0000 osd.6 (osd.6) 236 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:02.437925+0000 osd.1 (osd.1) 247 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:02.439075+0000 osd.1 (osd.1) 248 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:02.729643+0000 osd.7 (osd.7) 119 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:02.730681+0000 osd.7 (osd.7) 120 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:03.214056+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.214260+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:03.225096+0000 mon.c (mon.2) 457 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.705700+0000 mon.a (mon.0) 2699 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.813119+0000 osd.6 (osd.6) 235 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:40:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.814472+0000 osd.6 (osd.6) 236 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:02.437925+0000 osd.1 (osd.1) 247 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:02.439075+0000 osd.1 (osd.1) 248 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:02.729643+0000 osd.7 (osd.7) 119 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:02.730681+0000 osd.7 (osd.7) 120 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:03.214056+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.214260+0000 mon.a (mon.0) 2698 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:03.225096+0000 mon.c (mon.2) 457 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.705700+0000 mon.a (mon.0) 2699 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.813119+0000 osd.6 (osd.6) 235 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:40:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.814472+0000 osd.6 (osd.6) 236 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:40:05.460 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:40:05.460 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.464440+0000 osd.1 (osd.1) 249 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.466761+0000 osd.1 (osd.1) 250 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.520972+0000 osd.5 (osd.5) 207 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.521794+0000 osd.5 (osd.5) 208 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.930371+0000 mgr.y (mgr.24454) 455 : cluster [DBG] pgmap v735: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.978252+0000 osd.3 (osd.3) 263 : cluster [DBG] 6.b scrub starts 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:03.980628+0000 osd.3 (osd.3) 264 : cluster [DBG] 6.b scrub ok 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:04.188791+0000 osd.4 (osd.4) 289 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:04.189752+0000 osd.4 (osd.4) 290 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:40:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:04.254048+0000 osd.0 (osd.0) 226 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:04.255346+0000 osd.0 (osd.0) 227 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:04.713734+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:04.735698+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.4"}]: dispatch 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.464440+0000 osd.1 (osd.1) 249 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.466761+0000 osd.1 (osd.1) 250 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.520972+0000 osd.5 (osd.5) 207 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.521794+0000 osd.5 (osd.5) 208 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.930371+0000 mgr.y (mgr.24454) 455 : cluster [DBG] pgmap v735: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.978252+0000 osd.3 (osd.3) 263 : cluster [DBG] 6.b scrub starts 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:03.980628+0000 osd.3 (osd.3) 264 : cluster [DBG] 6.b scrub ok 2024-02-18T00:40:05.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:04.188791+0000 osd.4 (osd.4) 289 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:40:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:04.189752+0000 osd.4 (osd.4) 290 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:40:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:04.254048+0000 osd.0 (osd.0) 226 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:40:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:04.255346+0000 osd.0 (osd.0) 227 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:40:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:04.713734+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-18T00:40:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:04.735698+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.4"}]: dispatch 2024-02-18T00:40:05.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.464440+0000 osd.1 (osd.1) 249 : cluster [DBG] 62.10 scrub starts 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.466761+0000 osd.1 (osd.1) 250 : cluster [DBG] 62.10 scrub ok 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.520972+0000 osd.5 (osd.5) 207 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.521794+0000 osd.5 (osd.5) 208 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.930371+0000 mgr.y (mgr.24454) 455 : cluster [DBG] pgmap v735: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.978252+0000 osd.3 (osd.3) 263 : cluster [DBG] 6.b scrub starts 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:03.980628+0000 osd.3 (osd.3) 264 : cluster [DBG] 6.b scrub ok 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:04.188791+0000 osd.4 (osd.4) 289 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:04.189752+0000 osd.4 (osd.4) 290 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:04.254048+0000 osd.0 (osd.0) 226 : cluster [DBG] 6.18 scrub starts 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:04.255346+0000 osd.0 (osd.0) 227 : cluster [DBG] 6.18 scrub ok 2024-02-18T00:40:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:04.713734+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-18T00:40:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:04.735698+0000 mon.c (mon.2) 458 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.4"}]: dispatch 2024-02-18T00:40:05.917 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:05.921 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:05.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:06.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:06.053 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:06.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:04.496608+0000 osd.5 (osd.5) 209 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:40:06.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:04.499791+0000 osd.5 (osd.5) 210 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:40:06.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:04.735913+0000 mgr.y (mgr.24454) 456 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.4"}]: dispatch 2024-02-18T00:40:06.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:04.496608+0000 osd.5 (osd.5) 209 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:40:06.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:04.499791+0000 osd.5 (osd.5) 210 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:40:06.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:04.735913+0000 mgr.y (mgr.24454) 456 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.4"}]: dispatch 2024-02-18T00:40:06.501 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:06.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:04.496608+0000 osd.5 (osd.5) 209 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:40:06.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:04.499791+0000 osd.5 (osd.5) 210 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:40:06.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:04.735913+0000 mgr.y (mgr.24454) 456 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.4"}]: dispatch 2024-02-18T00:40:06.504 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:06.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:06.508 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-18T00:40:06.520 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:06.606 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:06.633 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:06.838 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:06.843 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:06.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:06.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:06.975 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:05.405728+0000 osd.1 (osd.1) 251 : cluster [DBG] 2.a scrub starts 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:05.408644+0000 osd.1 (osd.1) 252 : cluster [DBG] 2.a scrub ok 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:05.931229+0000 mgr.y (mgr.24454) 457 : cluster [DBG] pgmap v737: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.036020+0000 osd.2 (osd.2) 203 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.036928+0000 osd.2 (osd.2) 204 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.821532+0000 osd.6 (osd.6) 237 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.822426+0000 osd.6 (osd.6) 238 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:40:07.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:05.405728+0000 osd.1 (osd.1) 251 : cluster [DBG] 2.a scrub starts 2024-02-18T00:40:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:05.408644+0000 osd.1 (osd.1) 252 : cluster [DBG] 2.a scrub ok 2024-02-18T00:40:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:05.931229+0000 mgr.y (mgr.24454) 457 : cluster [DBG] pgmap v737: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T00:40:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.036020+0000 osd.2 (osd.2) 203 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:40:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.036928+0000 osd.2 (osd.2) 204 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:40:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.821532+0000 osd.6 (osd.6) 237 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:40:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.822426+0000 osd.6 (osd.6) 238 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:40:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:05.405728+0000 osd.1 (osd.1) 251 : cluster [DBG] 2.a scrub starts 2024-02-18T00:40:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:05.408644+0000 osd.1 (osd.1) 252 : cluster [DBG] 2.a scrub ok 2024-02-18T00:40:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:05.931229+0000 mgr.y (mgr.24454) 457 : cluster [DBG] pgmap v737: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T00:40:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.036020+0000 osd.2 (osd.2) 203 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:40:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.036928+0000 osd.2 (osd.2) 204 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:40:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.821532+0000 osd.6 (osd.6) 237 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:40:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.822426+0000 osd.6 (osd.6) 238 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:40:07.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 -- 172.21.15.59:0/3680905235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80f7400 msgr2=0x7f29a8103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:07.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 --2- 172.21.15.59:0/3680905235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80f7400 0x7f29a8103a20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 rx=0x7f29a0009c30 tx=0x7f29a0004c50).stop 2024-02-18T00:40:07.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 -- 172.21.15.59:0/3680905235 shutdown_connections 2024-02-18T00:40:07.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 --2- 172.21.15.59:0/3680905235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29a80fcbb0 0x7f29a80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 --2- 172.21.15.59:0/3680905235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80f7400 0x7f29a8103a20 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 --2- 172.21.15.59:0/3680905235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fd510 0x7f29a80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 -- 172.21.15.59:0/3680905235 >> 172.21.15.59:0/3680905235 conn(0x7f29a80f2820 msgr2=0x7f29a80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 -- 172.21.15.59:0/3680905235 shutdown_connections 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.463+0000 7f29ada29700 1 -- 172.21.15.59:0/3680905235 wait complete. 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.464+0000 7f29ada29700 1 librados: starting msgr at 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.464+0000 7f29ada29700 1 librados: starting objecter 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.464+0000 7f29ada29700 1 Processor -- start 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.464+0000 7f29ada29700 1 -- start start 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.464+0000 7f29ada29700 1 librados: setting wanted keys 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.464+0000 7f29ada29700 1 librados: calling monclient init 2024-02-18T00:40:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.464+0000 7f29ada29700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29a80f7400 0x7f29a8109210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fcbb0 0x7f29a8109700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80fd510 0x7f29a8109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a81143f0 con 0x7f29a80fcbb0 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8114560 con 0x7f29a80f7400 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8114720 con 0x7f29a80fd510 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a67fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fcbb0 0x7f29a8109700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80fd510 0x7f29a8109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a67fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fcbb0 0x7f29a8109700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46774/0 (socket says 172.21.15.59:46774) 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a67fc700 1 -- 172.21.15.59:0/2087084258 learned_addr learned my addr 172.21.15.59:0/2087084258 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80fd510 0x7f29a8109ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:52868/0 (socket says 172.21.15.59:52868) 2024-02-18T00:40:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a6ffd700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29a80f7400 0x7f29a8109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 714 (0%) 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:avg 89.25 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 7.3101 -> 7.3101 (expected baseline 8.83707) 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.5 with 82 -> 82 pgs (0.918768 -> 0.918768 * mean) 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.7 with 102 -> 102 pgs (1.14286 -> 1.14286 * mean) 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:oload 110 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 1 2024-02-18T00:40:07.788 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0033 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0037 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 -- 172.21.15.59:0/2087084258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29a80f7400 msgr2=0x7f29a8109210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29a80f7400 0x7f29a8109210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 -- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fcbb0 msgr2=0x7f29a8109700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fcbb0 0x7f29a8109700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 -- 172.21.15.59:0/2087084258 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2998005040 con 0x7f29a80fd510 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a67fc700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fcbb0 0x7f29a8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:07.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29a77fe700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80fd510 0x7f29a8109ce0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 rx=0x7f29a0009ad0 tx=0x7f29a0004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f298ffff700 10 client.?.objecter ms_handle_connect 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f298ffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f298ffff700 1 -- 172.21.15.59:0/2087084258 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000e030 con 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f298ffff700 1 -- 172.21.15.59:0/2087084258 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a0004020 con 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a00097c0 con 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f298ffff700 1 -- 172.21.15.59:0/2087084258 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0018070 con 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 10 client.41273.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a8114c10 con 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.465+0000 7f29ada29700 1 librados: init done 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.466+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a805fa60 con 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 1 -- 172.21.15.59:0/2087084258 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f29a0003c90 con 0x7f29a80fd510 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 1 --2- 172.21.15.59:0/2087084258 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2990062c90 0x7f2990065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:07.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 1 -- 172.21.15.59:0/2087084258 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(486..486 src has 180..486) v4 ==== 8151+0+0 (secure 0 0 0) 0x7f29a007e190 con 0x7f29a80fd510 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 10 client.41273.objecter ms_dispatch 0x7f29a8001550 osd_map(486..486 src has 180..486) v4 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 3 client.41273.objecter handle_osd_map got epochs [486,486] > 0 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 3 client.41273.objecter handle_osd_map decoding full epoch 486 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f29a6ffd700 1 --2- 172.21.15.59:0/2087084258 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2990062c90 0x7f2990065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 20 client.41273.objecter dump_active .. 0 homeless 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f29a6ffd700 1 --2- 172.21.15.59:0/2087084258 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2990062c90 0x7f2990065150 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 rx=0x7f299c004ee0 tx=0x7f299c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.467+0000 7f298ffff700 10 client.41273.objecter ms_handle_connect 0x7f2990062c90 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.471+0000 7f298ffff700 1 -- 172.21.15.59:0/2087084258 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f29a0051640 con 0x7f29a80fd510 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.642+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}) v1 -- 0x7f29a8061550 con 0x7f2990062c90 2024-02-18T00:40:07.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.643+0000 7f298ffff700 1 -- 172.21.15.59:0/2087084258 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+294 (secure 0 0 0) 0x7f29a8061550 con 0x7f2990062c90 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 10 librados: watch_flush enter 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 10 librados: watch_flush exit 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 20 client.41273.objecter shutdown clearing up homeless session... 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 10 client.41273.objecter successfully canceled tick 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2990062c90 msgr2=0x7f2990065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 --2- 172.21.15.59:0/2087084258 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2990062c90 0x7f2990065150 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 rx=0x7f299c004ee0 tx=0x7f299c00b040).stop 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80fd510 msgr2=0x7f29a8109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80fd510 0x7f29a8109ce0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 rx=0x7f29a0009ad0 tx=0x7f29a0004c50).stop 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 shutdown_connections 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 --2- 172.21.15.59:0/2087084258 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2990062c90 0x7f2990065150 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29a80f7400 0x7f29a8109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29a80fd510 0x7f29a8109ce0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 --2- 172.21.15.59:0/2087084258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29a80fcbb0 0x7f29a8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 >> 172.21.15.59:0/2087084258 conn(0x7f29a80f2820 msgr2=0x7f29a80f4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:07.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 shutdown_connections 2024-02-18T00:40:07.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 -- 172.21.15.59:0/2087084258 wait complete. 2024-02-18T00:40:07.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:07.645+0000 7f29ada29700 1 librados: shutdown 2024-02-18T00:40:07.793 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-02-18T00:40:08.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.570140+0000 osd.5 (osd.5) 211 : cluster [DBG] 25.f deep-scrub starts 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.573138+0000 osd.5 (osd.5) 212 : cluster [DBG] 25.f deep-scrub ok 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.959211+0000 osd.3 (osd.3) 265 : cluster [DBG] 5.14 deep-scrub starts 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:06.960499+0000 osd.3 (osd.3) 266 : cluster [DBG] 5.14 deep-scrub ok 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:07.239837+0000 osd.0 (osd.0) 228 : cluster [DBG] 25.e deep-scrub starts 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:07.241095+0000 osd.0 (osd.0) 229 : cluster [DBG] 25.e deep-scrub ok 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.570140+0000 osd.5 (osd.5) 211 : cluster [DBG] 25.f deep-scrub starts 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.573138+0000 osd.5 (osd.5) 212 : cluster [DBG] 25.f deep-scrub ok 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.959211+0000 osd.3 (osd.3) 265 : cluster [DBG] 5.14 deep-scrub starts 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:06.960499+0000 osd.3 (osd.3) 266 : cluster [DBG] 5.14 deep-scrub ok 2024-02-18T00:40:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:07.239837+0000 osd.0 (osd.0) 228 : cluster [DBG] 25.e deep-scrub starts 2024-02-18T00:40:08.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:07.241095+0000 osd.0 (osd.0) 229 : cluster [DBG] 25.e deep-scrub ok 2024-02-18T00:40:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.570140+0000 osd.5 (osd.5) 211 : cluster [DBG] 25.f deep-scrub starts 2024-02-18T00:40:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.573138+0000 osd.5 (osd.5) 212 : cluster [DBG] 25.f deep-scrub ok 2024-02-18T00:40:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.959211+0000 osd.3 (osd.3) 265 : cluster [DBG] 5.14 deep-scrub starts 2024-02-18T00:40:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:06.960499+0000 osd.3 (osd.3) 266 : cluster [DBG] 5.14 deep-scrub ok 2024-02-18T00:40:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:07.239837+0000 osd.0 (osd.0) 228 : cluster [DBG] 25.e deep-scrub starts 2024-02-18T00:40:08.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:07.241095+0000 osd.0 (osd.0) 229 : cluster [DBG] 25.e deep-scrub ok 2024-02-18T00:40:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1256602572 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 msgr2=0x7f5db40f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1256602572 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 0x7f5db40f7d40 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 rx=0x7f5da0004870 tx=0x7f5da0009cb0).stop 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1256602572 shutdown_connections 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1256602572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5db40fdbb0 0x7f5db40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1256602572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 0x7f5db4104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1256602572 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 0x7f5db40f7d40 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1256602572 >> 172.21.15.59:0/1256602572 conn(0x7f5db40f3800 msgr2=0x7f5db40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1256602572 shutdown_connections 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.357+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1256602572 wait complete. 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 librados: starting msgr at 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 librados: starting objecter 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 Processor -- start 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 -- start start 2024-02-18T00:40:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 librados: setting wanted keys 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 librados: calling monclient init 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 0x7f5db410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5db40fdbb0 0x7f5db410e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 0x7f5db41081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41086b0 con 0x7f5db40f8420 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4108820 con 0x7f5db40fdbb0 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5dbb16b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41089e0 con 0x7f5db40fe510 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5db8f07700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 0x7f5db410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5db8f07700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 0x7f5db410e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46800/0 (socket says 172.21.15.59:46800) 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5db8f07700 1 -- 172.21.15.59:0/1618205178 learned_addr learned my addr 172.21.15.59:0/1618205178 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.358+0000 7f5db9708700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 0x7f5db41081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db3fff700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5db40fdbb0 0x7f5db410e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db8f07700 1 -- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 msgr2=0x7f5db41081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db8f07700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 0x7f5db41081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db8f07700 1 -- 172.21.15.59:0/1618205178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5db40fdbb0 msgr2=0x7f5db410e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db8f07700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5db40fdbb0 0x7f5db410e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db8f07700 1 -- 172.21.15.59:0/1618205178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0097c0 con 0x7f5db40f8420 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db9708700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 0x7f5db41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db8f07700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 0x7f5db410e020 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7f5da80027f0 tx=0x7f5da8002b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db1ffb700 10 client.?.objecter ms_handle_connect 0x7f5db40f8420 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db1ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db1ffb700 1 -- 172.21.15.59:0/1618205178 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da8007b40 con 0x7f5db40f8420 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da0005040 con 0x7f5db40f8420 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db1ffb700 1 -- 172.21.15.59:0/1618205178 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5da800f040 con 0x7f5db40f8420 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5db1ffb700 1 -- 172.21.15.59:0/1618205178 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da80146f0 con 0x7f5db40f8420 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5dbb16b700 10 client.31476.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4109020 con 0x7f5db40f8420 2024-02-18T00:40:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.359+0000 7f5dbb16b700 1 librados: init done 2024-02-18T00:40:09.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.360+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db405ee80 con 0x7f5db40f8420 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 1 -- 172.21.15.59:0/1618205178 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5da8009110 con 0x7f5db40f8420 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 1 --2- 172.21.15.59:0/1618205178 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5da4062c90 0x7f5da4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 1 -- 172.21.15.59:0/1618205178 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(487..487 src has 180..487) v4 ==== 8151+0+0 (secure 0 0 0) 0x7f5da807cfd0 con 0x7f5db40f8420 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 10 client.31476.objecter ms_dispatch 0x7f5db4001550 osd_map(487..487 src has 180..487) v4 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 3 client.31476.objecter handle_osd_map got epochs [487,487] > 0 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 3 client.31476.objecter handle_osd_map decoding full epoch 487 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db3fff700 1 --2- 172.21.15.59:0/1618205178 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5da4062c90 0x7f5da4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 20 client.31476.objecter dump_active .. 0 homeless 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db1ffb700 10 client.31476.objecter ms_handle_connect 0x7f5da4062c90 2024-02-18T00:40:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.361+0000 7f5db3fff700 1 --2- 172.21.15.59:0/1618205178 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5da4062c90 0x7f5da4065150 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 rx=0x7f5da0009a00 tx=0x7f5da000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:09.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.365+0000 7f5db1ffb700 1 -- 172.21.15.59:0/1618205178 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5da8050420 con 0x7f5db40f8420 2024-02-18T00:40:09.422 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:09.427 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:09.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:09.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:09.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:07.474808+0000 osd.1 (osd.1) 253 : cluster [DBG] 2.d scrub starts 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:07.475727+0000 osd.1 (osd.1) 254 : cluster [DBG] 2.d scrub ok 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:07.643246+0000 mgr.y (mgr.24454) 458 : audit [DBG] from='client.41273 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:07.767083+0000 osd.7 (osd.7) 121 : cluster [DBG] 219.4 deep-scrub starts 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:07.769361+0000 osd.7 (osd.7) 122 : cluster [DBG] 219.4 deep-scrub ok 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:07.931645+0000 mgr.y (mgr.24454) 459 : cluster [DBG] pgmap v738: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 894 B/s rd, 894 B/s wr, 2 op/s 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.707758+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:07.474808+0000 osd.1 (osd.1) 253 : cluster [DBG] 2.d scrub starts 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:07.475727+0000 osd.1 (osd.1) 254 : cluster [DBG] 2.d scrub ok 2024-02-18T00:40:09.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:07.643246+0000 mgr.y (mgr.24454) 458 : audit [DBG] from='client.41273 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:07.767083+0000 osd.7 (osd.7) 121 : cluster [DBG] 219.4 deep-scrub starts 2024-02-18T00:40:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:07.769361+0000 osd.7 (osd.7) 122 : cluster [DBG] 219.4 deep-scrub ok 2024-02-18T00:40:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:07.931645+0000 mgr.y (mgr.24454) 459 : cluster [DBG] pgmap v738: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 894 B/s rd, 894 B/s wr, 2 op/s 2024-02-18T00:40:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.707758+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-18T00:40:09.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:07.474808+0000 osd.1 (osd.1) 253 : cluster [DBG] 2.d scrub starts 2024-02-18T00:40:09.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:07.475727+0000 osd.1 (osd.1) 254 : cluster [DBG] 2.d scrub ok 2024-02-18T00:40:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:07.643246+0000 mgr.y (mgr.24454) 458 : audit [DBG] from='client.41273 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:07.767083+0000 osd.7 (osd.7) 121 : cluster [DBG] 219.4 deep-scrub starts 2024-02-18T00:40:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:07.769361+0000 osd.7 (osd.7) 122 : cluster [DBG] 219.4 deep-scrub ok 2024-02-18T00:40:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:07.931645+0000 mgr.y (mgr.24454) 459 : cluster [DBG] pgmap v738: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 894 B/s rd, 894 B/s wr, 2 op/s 2024-02-18T00:40:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.707758+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-18T00:40:09.533 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:09.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:09.567+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f5db405fa60 con 0x7f5db40f8420 2024-02-18T00:40:09.587 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:09.590 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:09.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:09.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:09.719 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:09.739 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:09.742 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:09.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:09.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:09.871 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:09.890 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:09.893 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:09.893 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:09.980 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-02-18T00:40:09.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd test-reweight-by-utilization 110 3.0 2024-02-18T00:40:09.991 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:09.994 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:10.021 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:10.041 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:10.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:10.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:10.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:10.173 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:10.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.249+0000 7f5db1ffb700 1 -- 172.21.15.59:0/1618205178 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v488) v1 ==== 87+0+0 (secure 0 0 0) 0x7f5da8050010 con 0x7f5db40f8420 2024-02-18T00:40:10.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 10 librados: watch_flush enter 2024-02-18T00:40:10.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 10 librados: watch_flush exit 2024-02-18T00:40:10.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 20 client.31476.objecter shutdown clearing up homeless session... 2024-02-18T00:40:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 10 client.31476.objecter successfully canceled tick 2024-02-18T00:40:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5da4062c90 msgr2=0x7f5da4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1618205178 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5da4062c90 0x7f5da4065150 secure :-1 s=READY pgs=3133 cs=0 l=1 rev1=1 rx=0x7f5da0009a00 tx=0x7f5da000b040).stop 2024-02-18T00:40:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 msgr2=0x7f5db410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:10.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 0x7f5db410e020 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7f5da80027f0 tx=0x7f5da8002b00).stop 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 shutdown_connections 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5db40fe510 0x7f5db41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5db40f8420 0x7f5db410e020 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1618205178 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5da4062c90 0x7f5da4065150 unknown :-1 s=CLOSED pgs=3133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 --2- 172.21.15.59:0/1618205178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5db40fdbb0 0x7f5db410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 >> 172.21.15.59:0/1618205178 conn(0x7f5db40f3800 msgr2=0x7f5db40f53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 shutdown_connections 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 -- 172.21.15.59:0/1618205178 wait complete. 2024-02-18T00:40:10.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:10.252+0000 7f5dbb16b700 1 librados: shutdown 2024-02-18T00:40:10.255 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:40:10.272 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:10.276 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:10.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:10.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:10.408 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.428334+0000 osd.1 (osd.1) 255 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.429562+0000 osd.1 (osd.1) 256 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.614427+0000 osd.5 (osd.5) 213 : cluster [DBG] 5.f scrub starts 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.617344+0000 osd.5 (osd.5) 214 : cluster [DBG] 5.f scrub ok 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.780943+0000 osd.7 (osd.7) 123 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.782661+0000 osd.7 (osd.7) 124 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.960680+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:08.961537+0000 osd.3 (osd.3) 268 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:09.568078+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? 172.21.15.59:0/1618205178' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:09.722322+0000 osd.6 (osd.6) 239 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:09.723971+0000 osd.6 (osd.6) 240 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:40:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:09.904173+0000 mon.a (mon.0) 2703 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.428334+0000 osd.1 (osd.1) 255 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.429562+0000 osd.1 (osd.1) 256 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.614427+0000 osd.5 (osd.5) 213 : cluster [DBG] 5.f scrub starts 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.617344+0000 osd.5 (osd.5) 214 : cluster [DBG] 5.f scrub ok 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.780943+0000 osd.7 (osd.7) 123 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.782661+0000 osd.7 (osd.7) 124 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.960680+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:08.961537+0000 osd.3 (osd.3) 268 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:09.568078+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? 172.21.15.59:0/1618205178' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:09.722322+0000 osd.6 (osd.6) 239 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:40:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:09.723971+0000 osd.6 (osd.6) 240 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:09.904173+0000 mon.a (mon.0) 2703 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.428334+0000 osd.1 (osd.1) 255 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.429562+0000 osd.1 (osd.1) 256 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.614427+0000 osd.5 (osd.5) 213 : cluster [DBG] 5.f scrub starts 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.617344+0000 osd.5 (osd.5) 214 : cluster [DBG] 5.f scrub ok 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.780943+0000 osd.7 (osd.7) 123 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.782661+0000 osd.7 (osd.7) 124 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:40:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.960680+0000 osd.3 (osd.3) 267 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:40:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:08.961537+0000 osd.3 (osd.3) 268 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:40:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:09.568078+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? 172.21.15.59:0/1618205178' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:40:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:09.722322+0000 osd.6 (osd.6) 239 : cluster [DBG] 62.1c scrub starts 2024-02-18T00:40:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:09.723971+0000 osd.6 (osd.6) 240 : cluster [DBG] 62.1c scrub ok 2024-02-18T00:40:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:09.904173+0000 mon.a (mon.0) 2703 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:10.506 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:10.508 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:10.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:10.610 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:10.658 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:10.661 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:10.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:10.762 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:10.791 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:09.451029+0000 osd.1 (osd.1) 257 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:09.454044+0000 osd.1 (osd.1) 258 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:09.664875+0000 osd.5 (osd.5) 215 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:09.668902+0000 osd.5 (osd.5) 216 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:09.931976+0000 mgr.y (mgr.24454) 460 : cluster [DBG] pgmap v740: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 822 B/s rd, 822 B/s wr, 2 op/s 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:10.235039+0000 mon.a (mon.0) 2704 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:11.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:10.249080+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.59:0/1618205178' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:40:11.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:10.249129+0000 mon.a (mon.0) 2706 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:09.451029+0000 osd.1 (osd.1) 257 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:09.454044+0000 osd.1 (osd.1) 258 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:09.664875+0000 osd.5 (osd.5) 215 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:09.668902+0000 osd.5 (osd.5) 216 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:09.931976+0000 mgr.y (mgr.24454) 460 : cluster [DBG] pgmap v740: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 822 B/s rd, 822 B/s wr, 2 op/s 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:10.235039+0000 mon.a (mon.0) 2704 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:10.249080+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.59:0/1618205178' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:10.249129+0000 mon.a (mon.0) 2706 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:09.451029+0000 osd.1 (osd.1) 257 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:40:11.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:09.454044+0000 osd.1 (osd.1) 258 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:40:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:09.664875+0000 osd.5 (osd.5) 215 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:40:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:09.668902+0000 osd.5 (osd.5) 216 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:40:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:09.931976+0000 mgr.y (mgr.24454) 460 : cluster [DBG] pgmap v740: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 822 B/s rd, 822 B/s wr, 2 op/s 2024-02-18T00:40:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:10.235039+0000 mon.a (mon.0) 2704 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:10.249080+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? 172.21.15.59:0/1618205178' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:40:11.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:10.249129+0000 mon.a (mon.0) 2706 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-18T00:40:11.640 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:11.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:11.643 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:11.744 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:11.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:40:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:40:11.776 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:11.796 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:11.799 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:11.800 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:11.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:11.930 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:12.029 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:12.037 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:12.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:12.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:12.167 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:12.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:10.632640+0000 osd.5 (osd.5) 217 : cluster [DBG] 6.a scrub starts 2024-02-18T00:40:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:10.634589+0000 osd.5 (osd.5) 218 : cluster [DBG] 6.a scrub ok 2024-02-18T00:40:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:11.332855+0000 mgr.y (mgr.24454) 461 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:12.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:40:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:12] "GET /metrics HTTP/1.1" 200 190123 "" "Prometheus/2.33.4" 2024-02-18T00:40:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:10.632640+0000 osd.5 (osd.5) 217 : cluster [DBG] 6.a scrub starts 2024-02-18T00:40:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:10.634589+0000 osd.5 (osd.5) 218 : cluster [DBG] 6.a scrub ok 2024-02-18T00:40:12.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:11.332855+0000 mgr.y (mgr.24454) 461 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:10.632640+0000 osd.5 (osd.5) 217 : cluster [DBG] 6.a scrub starts 2024-02-18T00:40:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:10.634589+0000 osd.5 (osd.5) 218 : cluster [DBG] 6.a scrub ok 2024-02-18T00:40:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:11.332855+0000 mgr.y (mgr.24454) 461 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:12.917 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:12.922 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:12.922 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:13.023 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:13.056 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:13.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:13.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:13.180 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:13.210 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:13.229 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:13.232 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:13.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:13.333 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:13.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:11.444939+0000 osd.1 (osd.1) 259 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:40:13.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:11.447188+0000 osd.1 (osd.1) 260 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:40:13.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:11.932851+0000 mgr.y (mgr.24454) 462 : cluster [DBG] pgmap v742: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 709 B/s wr, 2 op/s 2024-02-18T00:40:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:13.381 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:13.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:13.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:13.485 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:13.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:11.444939+0000 osd.1 (osd.1) 259 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:40:13.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:11.447188+0000 osd.1 (osd.1) 260 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:40:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:11.932851+0000 mgr.y (mgr.24454) 462 : cluster [DBG] pgmap v742: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 709 B/s wr, 2 op/s 2024-02-18T00:40:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:11.444939+0000 osd.1 (osd.1) 259 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:40:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:11.447188+0000 osd.1 (osd.1) 260 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:40:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:11.932851+0000 mgr.y (mgr.24454) 462 : cluster [DBG] pgmap v742: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 709 B/s wr, 2 op/s 2024-02-18T00:40:13.520 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:13.540 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:13.542 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:13.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 -- 172.21.15.59:0/402823369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0fe510 msgr2=0x7fd84c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 --2- 172.21.15.59:0/402823369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0fe510 0x7fd84c0f7d40 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 rx=0x7fd840004870 tx=0x7fd840009cb0).stop 2024-02-18T00:40:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 -- 172.21.15.59:0/402823369 shutdown_connections 2024-02-18T00:40:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 --2- 172.21.15.59:0/402823369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd84c0f8420 0x7fd84c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 --2- 172.21.15.59:0/402823369 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd84c0fdbb0 0x7fd84c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 --2- 172.21.15.59:0/402823369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0fe510 0x7fd84c0f7d40 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 -- 172.21.15.59:0/402823369 >> 172.21.15.59:0/402823369 conn(0x7fd84c0f3800 msgr2=0x7fd84c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:13.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 -- 172.21.15.59:0/402823369 shutdown_connections 2024-02-18T00:40:13.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 -- 172.21.15.59:0/402823369 wait complete. 2024-02-18T00:40:13.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 librados: starting msgr at 2024-02-18T00:40:13.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.558+0000 7fd850cdd700 1 librados: starting objecter 2024-02-18T00:40:13.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 Processor -- start 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 -- start start 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 librados: setting wanted keys 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 librados: calling monclient init 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0f8420 0x7fd84c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd84c0fdbb0 0x7fd84c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd84c0fe510 0x7fd84c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c1164b0 con 0x7fd84c0f8420 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c116620 con 0x7fd84c0fdbb0 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd850cdd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c1167e0 con 0x7fd84c0fe510 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd84ad9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd84c0fe510 0x7fd84c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd84a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0f8420 0x7fd84c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd84ad9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd84c0fe510 0x7fd84c06bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:50018/0 (socket says 172.21.15.59:50018) 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 learned_addr learned my addr 172.21.15.59:0/4267698177 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.559+0000 7fd849d9b700 1 --2- 172.21.15.59:0/4267698177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd84c0fdbb0 0x7fd84c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd84c0fdbb0 msgr2=0x7fd84c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd84ad9d700 1 --2- 172.21.15.59:0/4267698177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd84c0fdbb0 0x7fd84c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0f8420 msgr2=0x7fd84c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd84ad9d700 1 --2- 172.21.15.59:0/4267698177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0f8420 0x7fd84c06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c005040 con 0x7fd84c0fe510 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd84c0f8420 0x7fd84c06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd84ad9d700 1 --2- 172.21.15.59:0/4267698177 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd84c0fe510 0x7fd84c06bab0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 rx=0x7fd84400c960 tx=0x7fd844004d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd83b7fe700 10 client.?.objecter ms_handle_connect 0x7fd84c0fe510 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd83b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400a520 con 0x7fd84c0fe510 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd84400ab50 con 0x7fd84c0fe510 2024-02-18T00:40:13.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd840005040 con 0x7fd84c0fe510 2024-02-18T00:40:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844012ba0 con 0x7fd84c0fe510 2024-02-18T00:40:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd850cdd700 10 client.41285.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c116ea0 con 0x7fd84c0fe510 2024-02-18T00:40:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.560+0000 7fd850cdd700 1 librados: init done 2024-02-18T00:40:13.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.561+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd84c1070a0 con 0x7fd84c0fe510 2024-02-18T00:40:13.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd84400a680 con 0x7fd84c0fe510 2024-02-18T00:40:13.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd834062c90 0x7fd834065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd834062c90 0x7fd834065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(488..488 src has 180..488) v4 ==== 8151+0+0 (secure 0 0 0) 0x7fd84407cdf0 con 0x7fd84c0fe510 2024-02-18T00:40:13.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 osd_map(488..488 src has 180..488) v4 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 3 client.41285.objecter handle_osd_map got epochs [488,488] > 0 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 3 client.41285.objecter handle_osd_map decoding full epoch 488 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 20 client.41285.objecter dump_active .. 0 homeless 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v488) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd84407f4c0 con 0x7fd84c0fe510 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd834062c90 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd834062c90 0x7fd834065150 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 rx=0x7fd83c009760 tx=0x7fd83c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 10 client.41285.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 1 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd84c0fee00 0x7fd84c0ff220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 1 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 0 1 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c06c5b0 osd=0 4 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 10 client.41285.objecter _send_command 1 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.562+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c0ff710 con 0x7fd84c0fee00 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd849d9b700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd84c0fee00 0x7fd84c0ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd849d9b700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd84c0fee00 0x7fd84c0ff220 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c0fee00 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c0ff710 con 0x7fd84c0fee00 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 1 = system:0 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c06c5b0 osd=0 4 2024-02-18T00:40:13.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.563+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 0 1 2024-02-18T00:40:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 10 client.41285.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 2 2024-02-18T00:40:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 2 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c06c5b0 osd=0 3 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 0 2 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c06c5b0 osd=0 4 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 10 client.41285.objecter _send_command 2 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.578+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c0ff900 con 0x7fd84c0fee00 2024-02-18T00:40:13.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c0ff900 con 0x7fd84c0fee00 2024-02-18T00:40:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c06c5b0 osd=0 4 2024-02-18T00:40:13.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 0 2 2024-02-18T00:40:13.581 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 10 client.41285.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 3 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6805/26279110 conn(0x7fd84c028540 0x7fd84c11b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 3 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 1 3 2024-02-18T00:40:13.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c0fd400 osd=1 4 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 10 client.41285.objecter _send_command 3 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c0fd570 con 0x7fd84c028540 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.580+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6805/26279110 conn(0x7fd84c028540 0x7fd84c11b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.581+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6805/26279110 conn(0x7fd84c028540 0x7fd84c11b110 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.581+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c028540 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.581+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c0fd570 con 0x7fd84c028540 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.581+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.581+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 3 = system:0 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.581+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c0fd400 osd=1 4 2024-02-18T00:40:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.581+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 1 3 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 10 client.41285.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 4 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 4 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c0fd400 osd=1 3 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 1 4 2024-02-18T00:40:13.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c0fd400 osd=1 4 2024-02-18T00:40:13.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 10 client.41285.objecter _send_command 4 2024-02-18T00:40:13.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.596+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c05a040 con 0x7fd84c028540 2024-02-18T00:40:13.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c05a040 con 0x7fd84c028540 2024-02-18T00:40:13.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c0fd400 osd=1 4 2024-02-18T00:40:13.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 1 4 2024-02-18T00:40:13.599 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 10 client.41285.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 5 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd84c0594d0 0x7fd84c11e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 5 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 2 5 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c059360 osd=2 4 2024-02-18T00:40:13.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 10 client.41285.objecter _send_command 5 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c0280a0 con 0x7fd84c0594d0 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd849d9b700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd84c0594d0 0x7fd84c11e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c0594d0 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.598+0000 7fd849d9b700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd84c0594d0 0x7fd84c11e790 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.599+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c0280a0 con 0x7fd84c0594d0 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.599+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.599+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 5 = system:0 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.599+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c059360 osd=2 4 2024-02-18T00:40:13.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.599+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 2 5 2024-02-18T00:40:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 10 client.41285.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 6 2024-02-18T00:40:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 6 2024-02-18T00:40:13.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c059360 osd=2 3 2024-02-18T00:40:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 2 6 2024-02-18T00:40:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c059360 osd=2 4 2024-02-18T00:40:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 10 client.41285.objecter _send_command 6 2024-02-18T00:40:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.613+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c028320 con 0x7fd84c0594d0 2024-02-18T00:40:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c028320 con 0x7fd84c0594d0 2024-02-18T00:40:13.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c059360 osd=2 4 2024-02-18T00:40:13.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 2 6 2024-02-18T00:40:13.616 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 10 client.41285.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 7 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd84c11c2e0 0x7fd84c121e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 7 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 3 7 2024-02-18T00:40:13.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c027860 osd=3 4 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 10 client.41285.objecter _send_command 7 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c124a60 con 0x7fd84c11c2e0 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd84ad9d700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd84c11c2e0 0x7fd84c121e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c11c2e0 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.615+0000 7fd84ad9d700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd84c11c2e0 0x7fd84c121e10 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.616+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c124a60 con 0x7fd84c11c2e0 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.616+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.616+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 7 = system:0 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.616+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c027860 osd=3 4 2024-02-18T00:40:13.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.616+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 3 7 2024-02-18T00:40:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 10 client.41285.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 8 2024-02-18T00:40:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 8 2024-02-18T00:40:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c027860 osd=3 3 2024-02-18T00:40:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 3 8 2024-02-18T00:40:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c027860 osd=3 4 2024-02-18T00:40:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 10 client.41285.objecter _send_command 8 2024-02-18T00:40:13.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.630+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c11b840 con 0x7fd84c11c2e0 2024-02-18T00:40:13.632 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:40:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c11b840 con 0x7fd84c11c2e0 2024-02-18T00:40:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c027860 osd=3 4 2024-02-18T00:40:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 3 8 2024-02-18T00:40:13.633 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:40:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 10 client.41285.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 9 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd84c11f960 0x7fd84c123480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 9 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 4 9 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11b840 osd=4 4 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 10 client.41285.objecter _send_command 9 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c11bb10 con 0x7fd84c11f960 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.632+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd84c11f960 0x7fd84c123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.633+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c11f960 2024-02-18T00:40:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.633+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd84c11f960 0x7fd84c123480 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.633+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c11bb10 con 0x7fd84c11f960 2024-02-18T00:40:13.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.633+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:40:13.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.633+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 9 = system:0 2024-02-18T00:40:13.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.633+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c11b840 osd=4 4 2024-02-18T00:40:13.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.633+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 4 9 2024-02-18T00:40:13.644 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:13.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 10 client.41285.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 10 2024-02-18T00:40:13.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 10 2024-02-18T00:40:13.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c11b840 osd=4 3 2024-02-18T00:40:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 4 10 2024-02-18T00:40:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11b840 osd=4 4 2024-02-18T00:40:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 10 client.41285.objecter _send_command 10 2024-02-18T00:40:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.648+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c11bb10 con 0x7fd84c11f960 2024-02-18T00:40:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c11bb10 con 0x7fd84c11f960 2024-02-18T00:40:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c11b840 osd=4 4 2024-02-18T00:40:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 4 10 2024-02-18T00:40:13.651 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:40:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 10 client.41285.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 11 2024-02-18T00:40:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd84c123e80 0x7fd84c1242a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 11 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 5 11 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11bb10 osd=5 4 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 10 client.41285.objecter _send_command 11 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.650+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c12b350 con 0x7fd84c123e80 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.651+0000 7fd849d9b700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd84c123e80 0x7fd84c1242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.651+0000 7fd849d9b700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd84c123e80 0x7fd84c1242a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.651+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c123e80 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.652+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c12b350 con 0x7fd84c123e80 2024-02-18T00:40:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.652+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:40:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.652+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 11 = system:0 2024-02-18T00:40:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.652+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c11bb10 osd=5 4 2024-02-18T00:40:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.652+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 5 11 2024-02-18T00:40:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 10 client.41285.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 12 2024-02-18T00:40:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 12 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c11bb10 osd=5 3 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 5 12 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11bb10 osd=5 4 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 10 client.41285.objecter _send_command 12 2024-02-18T00:40:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.668+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c128bf0 con 0x7fd84c123e80 2024-02-18T00:40:13.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.669+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c128bf0 con 0x7fd84c123e80 2024-02-18T00:40:13.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.669+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.669+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.669+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c11bb10 osd=5 4 2024-02-18T00:40:13.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.669+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 5 12 2024-02-18T00:40:13.671 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:40:13.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 10 client.41285.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 13 2024-02-18T00:40:13.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd84c1292d0 0x7fd84c12b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 13 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 6 13 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c128bf0 osd=6 4 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 10 client.41285.objecter _send_command 13 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c12e4b0 con 0x7fd84c1292d0 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd84ad9d700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd84c1292d0 0x7fd84c12b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c1292d0 2024-02-18T00:40:13.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.670+0000 7fd84ad9d700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd84c1292d0 0x7fd84c12b6e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.671+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c12e4b0 con 0x7fd84c1292d0 2024-02-18T00:40:13.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.671+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:40:13.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.671+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 13 = system:0 2024-02-18T00:40:13.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.671+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c128bf0 osd=6 4 2024-02-18T00:40:13.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.671+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 6 13 2024-02-18T00:40:13.673 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:13.692 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:13.694 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:13.695 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:13.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 10 client.41285.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 14 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 14 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c128bf0 osd=6 3 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 6 14 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c128bf0 osd=6 4 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 10 client.41285.objecter _send_command 14 2024-02-18T00:40:13.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.702+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c12e4b0 con 0x7fd84c1292d0 2024-02-18T00:40:13.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c12e4b0 con 0x7fd84c1292d0 2024-02-18T00:40:13.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c128bf0 osd=6 4 2024-02-18T00:40:13.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 6 14 2024-02-18T00:40:13.705 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:40:13.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 10 client.41285.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:13.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 15 2024-02-18T00:40:13.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6812/53025171 conn(0x7fd84c12d7b0 0x7fd84c12dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:13.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 15 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 7 15 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c12e4b0 osd=7 4 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 10 client.41285.objecter _send_command 15 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84c12e2a0 con 0x7fd84c12d7b0 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.704+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6812/53025171 conn(0x7fd84c12d7b0 0x7fd84c12dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.705+0000 7fd84a59c700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6812/53025171 conn(0x7fd84c12d7b0 0x7fd84c12dc30 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.705+0000 7fd83b7fe700 10 client.41285.objecter ms_handle_connect 0x7fd84c12d7b0 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.705+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd84c12e2a0 con 0x7fd84c12d7b0 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.705+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.705+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 15 = system:0 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.705+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c12e4b0 osd=7 4 2024-02-18T00:40:13.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.705+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 7 15 2024-02-18T00:40:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 10 client.41285.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign -1 16 2024-02-18T00:40:13.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 20 client.41285.objecter _get_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_remove -1 16 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 20 client.41285.objecter get_session s=0x7fd84c12e4b0 osd=7 3 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 15 client.41285.objecter _session_command_op_assign 7 16 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c12e4b0 osd=7 4 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 10 client.41285.objecter _send_command 16 2024-02-18T00:40:13.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.726+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd84c12e2a0 con 0x7fd84c12d7b0 2024-02-18T00:40:13.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd83b7fe700 1 -- 172.21.15.59:0/4267698177 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd84c12e2a0 con 0x7fd84c12d7b0 2024-02-18T00:40:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd83b7fe700 10 client.41285.objecter ms_dispatch 0x7fd84c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd83b7fe700 10 client.41285.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd83b7fe700 20 client.41285.objecter put_session s=0x7fd84c12e4b0 osd=7 4 2024-02-18T00:40:13.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd83b7fe700 15 client.41285.objecter _session_command_op_remove 7 16 2024-02-18T00:40:13.730 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:40:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 librados: watch_flush enter 2024-02-18T00:40:13.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 librados: watch_flush exit 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.0 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd84c0fee00 msgr2=0x7fd84c0ff220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd84c0fee00 0x7fd84c0ff220 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c06c5b0 osd=0 1 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.1 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6805/26279110 conn(0x7fd84c028540 msgr2=0x7fd84c11b110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6805/26279110 conn(0x7fd84c028540 0x7fd84c11b110 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c0fd400 osd=1 1 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.2 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd84c0594d0 msgr2=0x7fd84c11e790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd84c0594d0 0x7fd84c11e790 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c059360 osd=2 1 2024-02-18T00:40:13.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.3 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd84c11c2e0 msgr2=0x7fd84c121e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd84c11c2e0 0x7fd84c121e10 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c027860 osd=3 1 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.4 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd84c11f960 msgr2=0x7fd84c123480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd84c11f960 0x7fd84c123480 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11b840 osd=4 1 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.5 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd84c123e80 msgr2=0x7fd84c1242a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd84c123e80 0x7fd84c1242a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c11bb10 osd=5 1 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.6 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd84c1292d0 msgr2=0x7fd84c12b6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd84c1292d0 0x7fd84c12b6e0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c128bf0 osd=6 1 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter close_session for osd.7 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6812/53025171 conn(0x7fd84c12d7b0 msgr2=0x7fd84c12dc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.137:6812/53025171 conn(0x7fd84c12d7b0 0x7fd84c12dc30 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter put_session s=0x7fd84c12e4b0 osd=7 1 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 20 client.41285.objecter shutdown clearing up homeless session... 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.729+0000 7fd850cdd700 10 client.41285.objecter successfully canceled tick 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.730+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd834062c90 msgr2=0x7fd834065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.730+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd834062c90 0x7fd834065150 secure :-1 s=READY pgs=3134 cs=0 l=1 rev1=1 rx=0x7fd83c009760 tx=0x7fd83c00b040).stop 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.730+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd84c0fe510 msgr2=0x7fd84c06bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.730+0000 7fd850cdd700 1 --2- 172.21.15.59:0/4267698177 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd84c0fe510 0x7fd84c06bab0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 rx=0x7fd84400c960 tx=0x7fd844004d60).stop 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 reap_dead start 2024-02-18T00:40:13.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 reap_dead start 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 reap_dead start 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 reap_dead start 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 reap_dead start 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd84ad9d700 1 -- 172.21.15.59:0/4267698177 reap_dead start 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 shutdown_connections 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 >> 172.21.15.59:0/4267698177 conn(0x7fd84c0f3800 msgr2=0x7fd84c0fb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 shutdown_connections 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd850cdd700 1 -- 172.21.15.59:0/4267698177 wait complete. 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:13.731+0000 7fd850cdd700 1 librados: shutdown 2024-02-18T00:40:13.735 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.736 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:40:13.796 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:13.823 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:13.842 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:13.844 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:13.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:13.947 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:13.973 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:13.991 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:13.994 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:13.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:14.065 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:14.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:14.125 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:14.368 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:13.562370+0000 mon.c (mon.2) 459 : audit [DBG] from='client.? 172.21.15.59:0/4267698177' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:14.368 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:13.713122+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-18T00:40:14.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:13.562370+0000 mon.c (mon.2) 459 : audit [DBG] from='client.? 172.21.15.59:0/4267698177' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:14.368 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:13.713122+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-18T00:40:14.374 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:14.377 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:14.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:14.478 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:13.562370+0000 mon.c (mon.2) 459 : audit [DBG] from='client.? 172.21.15.59:0/4267698177' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:13.713122+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-18T00:40:14.506 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:14.525 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:14.528 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:14.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:14.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:14.659 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:15.208 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:15.210 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:15.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:15.311 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:15.341 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:15.379 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:15.381 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:15.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:15.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:15.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:13.933518+0000 mgr.y (mgr.24454) 463 : cluster [DBG] pgmap v744: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:40:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:13.933518+0000 mgr.y (mgr.24454) 463 : cluster [DBG] pgmap v744: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:40:15.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:13.933518+0000 mgr.y (mgr.24454) 463 : cluster [DBG] pgmap v744: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:40:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:15.960 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:15.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:15.963 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:16.064 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:16.097 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:16.117 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:16.121 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:16.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:16.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:16.252 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:16.377 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:16.379 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:16.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:16.380 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:40:16.381 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:40:16.381 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:40:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:40:16.480 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:16.509 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:16.632 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:16.634 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:16.635 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:16.736 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:16.765 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:17.108 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:17.115 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:17.124 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:17.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:17.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:40:17.226 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:17.281 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:17.283 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:17.284 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:17.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:17.387 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:15.934382+0000 mgr.y (mgr.24454) 464 : cluster [DBG] pgmap v745: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:40:17.388 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:15.934382+0000 mgr.y (mgr.24454) 464 : cluster [DBG] pgmap v745: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:40:17.418 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.639+0000 7fe916852700 1 -- 172.21.15.59:0/475729901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 msgr2=0x7fe9100fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.639+0000 7fe916852700 1 --2- 172.21.15.59:0/475729901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe9100fdf90 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 rx=0x7fe900004870 tx=0x7fe900009cb0).stop 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.639+0000 7fe916852700 1 -- 172.21.15.59:0/475729901 shutdown_connections 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.639+0000 7fe916852700 1 --2- 172.21.15.59:0/475729901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9100f8420 0x7fe910104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.639+0000 7fe916852700 1 --2- 172.21.15.59:0/475729901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe9100fdf90 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.639+0000 7fe916852700 1 --2- 172.21.15.59:0/475729901 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9100fe510 0x7fe9100f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.639+0000 7fe916852700 1 -- 172.21.15.59:0/475729901 >> 172.21.15.59:0/475729901 conn(0x7fe9100f3800 msgr2=0x7fe9100f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 -- 172.21.15.59:0/475729901 shutdown_connections 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 -- 172.21.15.59:0/475729901 wait complete. 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 librados: starting msgr at 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 librados: starting objecter 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 Processor -- start 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 -- start start 2024-02-18T00:40:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 librados: setting wanted keys 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 librados: calling monclient init 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9100f8420 0x7fe91010e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe91010e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9100fe510 0x7fe910108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910108770 con 0x7fe9100fdbb0 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9101088e0 con 0x7fe9100fe510 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.640+0000 7fe916852700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe910108aa0 con 0x7fe9100f8420 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe91010e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9100f8420 0x7fe91010e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe91010e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38024/0 (socket says 172.21.15.59:38024) 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90f7fe700 1 -- 172.21.15.59:0/3985945038 learned_addr learned my addr 172.21.15.59:0/3985945038 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:17.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe914def700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9100fe510 0x7fe910108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90ffff700 1 -- 172.21.15.59:0/3985945038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9100fe510 msgr2=0x7fe910108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90ffff700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9100fe510 0x7fe910108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90ffff700 1 -- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 msgr2=0x7fe91010e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90ffff700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe91010e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90ffff700 1 -- 172.21.15.59:0/3985945038 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8005040 con 0x7fe9100f8420 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90f7fe700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe91010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90ffff700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9100f8420 0x7fe91010e040 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 rx=0x7fe900004870 tx=0x7fe90000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90d7fa700 10 client.?.objecter ms_handle_connect 0x7fe9100f8420 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90d7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90d7fa700 1 -- 172.21.15.59:0/3985945038 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90000e070 con 0x7fe9100f8420 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe900005040 con 0x7fe9100f8420 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90d7fa700 1 -- 172.21.15.59:0/3985945038 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe900013da0 con 0x7fe9100f8420 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe90d7fa700 1 -- 172.21.15.59:0/3985945038 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe900019070 con 0x7fe9100f8420 2024-02-18T00:40:17.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe916852700 10 client.41291.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe910109010 con 0x7fe9100f8420 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.641+0000 7fe916852700 1 librados: init done 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.642+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe910107060 con 0x7fe9100f8420 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 1 -- 172.21.15.59:0/3985945038 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe90001e420 con 0x7fe9100f8420 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 1 --2- 172.21.15.59:0/3985945038 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe8fc062c90 0x7fe8fc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 1 -- 172.21.15.59:0/3985945038 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(489..489 src has 180..489) v4 ==== 8151+0+0 (secure 0 0 0) 0x7fe90007daa0 con 0x7fe9100f8420 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 10 client.41291.objecter ms_dispatch 0x7fe910001550 osd_map(489..489 src has 180..489) v4 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 3 client.41291.objecter handle_osd_map got epochs [489,489] > 0 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 3 client.41291.objecter handle_osd_map decoding full epoch 489 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90f7fe700 1 --2- 172.21.15.59:0/3985945038 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe8fc062c90 0x7fe8fc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 20 client.41291.objecter dump_active .. 0 homeless 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90d7fa700 10 client.41291.objecter ms_handle_connect 0x7fe8fc062c90 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.643+0000 7fe90f7fe700 1 --2- 172.21.15.59:0/3985945038 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe8fc062c90 0x7fe8fc065150 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 rx=0x7fe910109980 tx=0x7fe8f800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.647+0000 7fe90d7fa700 1 -- 172.21.15.59:0/3985945038 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe900051030 con 0x7fe9100f8420 2024-02-18T00:40:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:17.677 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:17.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:17.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:15.934382+0000 mgr.y (mgr.24454) 464 : cluster [DBG] pgmap v745: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:40:17.778 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:17.808 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:17.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.834+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}) v1 -- 0x7fe91005fa60 con 0x7fe8fc062c90 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.835+0000 7fe90d7fa700 1 -- 172.21.15.59:0/3985945038 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+294 (secure 0 0 0) 0x7fe91005fa60 con 0x7fe8fc062c90 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 714 (0%) 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stdout:avg 89.25 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 7.3101 -> 7.3101 (expected baseline 8.83707) 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.5 with 82 -> 82 pgs (0.918768 -> 0.918768 * mean) 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.7 with 102 -> 102 pgs (1.14286 -> 1.14286 * mean) 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:40:17.837 INFO:teuthology.orchestra.run.smithi059.stdout:oload 110 2024-02-18T00:40:17.838 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 3 2024-02-18T00:40:17.838 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-02-18T00:40:17.838 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0033 2024-02-18T00:40:17.838 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0037 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 10 librados: watch_flush enter 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 10 librados: watch_flush exit 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 20 client.41291.objecter shutdown clearing up homeless session... 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 10 client.41291.objecter successfully canceled tick 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe8fc062c90 msgr2=0x7fe8fc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 --2- 172.21.15.59:0/3985945038 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe8fc062c90 0x7fe8fc065150 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 rx=0x7fe910109980 tx=0x7fe8f800b040).stop 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9100f8420 msgr2=0x7fe91010e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:17.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9100f8420 0x7fe91010e040 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 rx=0x7fe900004870 tx=0x7fe90000b040).stop 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 shutdown_connections 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 --2- 172.21.15.59:0/3985945038 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe8fc062c90 0x7fe8fc065150 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9100fe510 0x7fe910108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9100f8420 0x7fe91010e040 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 --2- 172.21.15.59:0/3985945038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9100fdbb0 0x7fe91010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 >> 172.21.15.59:0/3985945038 conn(0x7fe9100f3800 msgr2=0x7fe9100fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 shutdown_connections 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 -- 172.21.15.59:0/3985945038 wait complete. 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:17.837+0000 7fe916852700 1 librados: shutdown 2024-02-18T00:40:17.840 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-02-18T00:40:18.662 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:18.665 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:18.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:18.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:18.799 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:18.829 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:18.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:18.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:18.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:18.967 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:17.836015+0000 mgr.y (mgr.24454) 465 : audit [DBG] from='client.41291 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:19.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:17.934981+0000 mgr.y (mgr.24454) 466 : cluster [DBG] pgmap v746: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:19.654 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:17.836015+0000 mgr.y (mgr.24454) 465 : audit [DBG] from='client.41291 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:19.654 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:17.934981+0000 mgr.y (mgr.24454) 466 : cluster [DBG] pgmap v746: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:19.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-18T00:40:19.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:17.836015+0000 mgr.y (mgr.24454) 465 : audit [DBG] from='client.41291 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:19.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:17.934981+0000 mgr.y (mgr.24454) 466 : cluster [DBG] pgmap v746: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:20.021 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-02-18T00:40:20.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight-by-utilization 110 3.0 2024-02-18T00:40:20.026 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:20.029 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:20.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:20.038 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:20.130 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:20.161 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:20.181 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:20.184 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:20.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:20.286 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:20.313 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:20.366 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:20.369 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:20.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:20.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:20.500 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:20.699 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:20.703 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:20.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:20.805 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:20.838 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:20.859 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:20.861 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:20.862 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:20.962 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:20.992 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:21.015 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:21.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:21.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:21.146 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:21.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:40:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:40:21.400 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:21.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:19.935623+0000 mgr.y (mgr.24454) 467 : cluster [DBG] pgmap v747: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:40:21.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:21.188548+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? 172.21.15.59:0/4225294405' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:40:21.402 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:19.935623+0000 mgr.y (mgr.24454) 467 : cluster [DBG] pgmap v747: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:40:21.402 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:21.188548+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? 172.21.15.59:0/4225294405' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:40:21.402 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:21.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:21.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 -- 172.21.15.59:0/2172937808 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 msgr2=0x7fda140f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/2172937808 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 0x7fda140f7d40 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 rx=0x7fda080047b0 tx=0x7fda08009cb0).stop 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 -- 172.21.15.59:0/2172937808 shutdown_connections 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/2172937808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda140fdbb0 0x7fda140fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/2172937808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 0x7fda14104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/2172937808 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 0x7fda140f7d40 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 -- 172.21.15.59:0/2172937808 >> 172.21.15.59:0/2172937808 conn(0x7fda140f3800 msgr2=0x7fda140f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.985+0000 7fda1a3cd700 1 -- 172.21.15.59:0/2172937808 shutdown_connections 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 -- 172.21.15.59:0/2172937808 wait complete. 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 librados: starting msgr at 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 librados: starting objecter 2024-02-18T00:40:21.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 Processor -- start 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 -- start start 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 librados: setting wanted keys 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 librados: calling monclient init 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.986+0000 7fda1a3cd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 0x7fda1410e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda140fdbb0 0x7fda1410e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 0x7fda14108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda14108750 con 0x7fda140f8420 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda141088c0 con 0x7fda140fdbb0 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda14108a80 con 0x7fda140fe510 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 0x7fda1410e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1896a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 0x7fda14108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 0x7fda1410e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36846/0 (socket says 172.21.15.59:36846) 2024-02-18T00:40:21.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 -- 172.21.15.59:0/4225294405 learned_addr learned my addr 172.21.15.59:0/4225294405 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda137fe700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda140fdbb0 0x7fda1410e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 -- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 msgr2=0x7fda14108260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 0x7fda14108260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 -- 172.21.15.59:0/4225294405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda140fdbb0 msgr2=0x7fda1410e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda140fdbb0 0x7fda1410e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 -- 172.21.15.59:0/4225294405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c0097c0 con 0x7fda140f8420 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1896a700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 0x7fda14108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda13fff700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 0x7fda1410e0c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7fda0400e970 tx=0x7fda0400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda117fa700 10 client.?.objecter ms_handle_connect 0x7fda140f8420 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda117fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda117fa700 1 -- 172.21.15.59:0/4225294405 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04004660 con 0x7fda140f8420 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda08005040 con 0x7fda140f8420 2024-02-18T00:40:21.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda117fa700 1 -- 172.21.15.59:0/4225294405 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0400f040 con 0x7fda140f8420 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda117fa700 1 -- 172.21.15.59:0/4225294405 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04014790 con 0x7fda140f8420 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 10 client.31494.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda14114ef0 con 0x7fda140f8420 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.987+0000 7fda1a3cd700 1 librados: init done 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.988+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1405ee80 con 0x7fda140f8420 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda117fa700 1 -- 172.21.15.59:0/4225294405 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fda040090d0 con 0x7fda140f8420 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda117fa700 1 --2- 172.21.15.59:0/4225294405 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd9fc062c90 0x7fd9fc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda117fa700 1 -- 172.21.15.59:0/4225294405 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(489..489 src has 180..489) v4 ==== 8151+0+0 (secure 0 0 0) 0x7fda0407cd90 con 0x7fda140f8420 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda117fa700 10 client.31494.objecter ms_dispatch 0x7fda14001550 osd_map(489..489 src has 180..489) v4 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda117fa700 3 client.31494.objecter handle_osd_map got epochs [489,489] > 0 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda117fa700 3 client.31494.objecter handle_osd_map decoding full epoch 489 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda137fe700 1 --2- 172.21.15.59:0/4225294405 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd9fc062c90 0x7fd9fc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.989+0000 7fda117fa700 20 client.31494.objecter dump_active .. 0 homeless 2024-02-18T00:40:21.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.990+0000 7fda117fa700 10 client.31494.objecter ms_handle_connect 0x7fd9fc062c90 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.990+0000 7fda137fe700 1 --2- 172.21.15.59:0/4225294405 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd9fc062c90 0x7fd9fc065150 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 rx=0x7fda14109a00 tx=0x7fda08007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:20.993+0000 7fda117fa700 1 -- 172.21.15.59:0/4225294405 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fda0404af60 con 0x7fda140f8420 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.187+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fda14109840 con 0x7fda140f8420 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.264+0000 7fda117fa700 1 -- 172.21.15.59:0/4225294405 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v490) v1 ==== 97+0+0 (secure 0 0 0) 0x7fda04019070 con 0x7fda140f8420 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 10 librados: watch_flush enter 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 10 librados: watch_flush exit 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 20 client.31494.objecter shutdown clearing up homeless session... 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 10 client.31494.objecter successfully canceled tick 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd9fc062c90 msgr2=0x7fd9fc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/4225294405 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd9fc062c90 0x7fd9fc065150 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 rx=0x7fda14109a00 tx=0x7fda08007f90).stop 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 msgr2=0x7fda1410e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 0x7fda1410e0c0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7fda0400e970 tx=0x7fda0400b040).stop 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 shutdown_connections 2024-02-18T00:40:21.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda140fe510 0x7fda14108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda140f8420 0x7fda1410e0c0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/4225294405 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd9fc062c90 0x7fd9fc065150 unknown :-1 s=CLOSED pgs=3136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 --2- 172.21.15.59:0/4225294405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fda140fdbb0 0x7fda1410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.266+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 >> 172.21.15.59:0/4225294405 conn(0x7fda140f3800 msgr2=0x7fda140f5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.267+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 shutdown_connections 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.267+0000 7fda1a3cd700 1 -- 172.21.15.59:0/4225294405 wait complete. 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:21.267+0000 7fda1a3cd700 1 librados: shutdown 2024-02-18T00:40:21.498 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:40:21.503 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:21.532 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:40:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:40:21.533 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:19.935623+0000 mgr.y (mgr.24454) 467 : cluster [DBG] pgmap v747: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:40:21.533 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:21.188548+0000 mon.a (mon.0) 2708 : audit [INF] from='client.? 172.21.15.59:0/4225294405' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:40:21.534 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:21.558 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:21.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:21.660 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:21.690 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:21.728 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:21.728 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:21.734 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:21.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:21.835 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:21.867 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:21.889 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:21.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:21.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:21.993 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:22.023 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:22.044 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:22.047 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:22.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:22.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:22.178 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:22.326 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:22.328 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:22.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:22.430 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:22.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:21.258866+0000 mon.a (mon.0) 2709 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:22.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:21.263061+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? 172.21.15.59:0/4225294405' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:40:22.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:21.263129+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-18T00:40:22.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:21.343792+0000 mgr.y (mgr.24454) 468 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:22.483 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:22] "GET /metrics HTTP/1.1" 200 190123 "" "Prometheus/2.33.4" 2024-02-18T00:40:22.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:21.258866+0000 mon.a (mon.0) 2709 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:22.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:21.263061+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? 172.21.15.59:0/4225294405' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:40:22.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:21.263129+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-18T00:40:22.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:21.343792+0000 mgr.y (mgr.24454) 468 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:22.483 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:22.486 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:22.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:22.588 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:22.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:21.258866+0000 mon.a (mon.0) 2709 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:22.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:21.263061+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? 172.21.15.59:0/4225294405' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:40:22.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:21.263129+0000 mon.a (mon.0) 2711 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-18T00:40:22.615 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:21.343792+0000 mgr.y (mgr.24454) 468 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:22.617 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:22.636 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:22.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:22.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:22.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:22.771 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:22.866 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:22.871 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:22.875 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:22.875 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:22.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:23.006 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:23.155 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:23.160 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:23.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:23.261 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:23.291 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:21.936096+0000 mgr.y (mgr.24454) 469 : cluster [DBG] pgmap v749: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:23.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:21.936096+0000 mgr.y (mgr.24454) 469 : cluster [DBG] pgmap v749: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:23.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:21.936096+0000 mgr.y (mgr.24454) 469 : cluster [DBG] pgmap v749: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:24.339 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:40:24.942 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:24.944 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:23.716503+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-18T00:40:24.944 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:23.936504+0000 mgr.y (mgr.24454) 470 : cluster [DBG] pgmap v751: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:40:24.944 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:23.716503+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-18T00:40:24.944 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:23.936504+0000 mgr.y (mgr.24454) 470 : cluster [DBG] pgmap v751: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:40:24.945 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:40:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2[57675]: debug 2024-02-18T00:40:24.748+0000 7f0e4fe83700 -1 snap_mapper.add_oid found existing snaps mapped on 219:24fa6925:test-rados-api-smithi059-82957-76::foo:2, removing 2024-02-18T00:40:24.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:24.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:23.716503+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-18T00:40:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:23.936504+0000 mgr.y (mgr.24454) 470 : cluster [DBG] pgmap v751: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:40:25.004 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:40:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:40:24.747+0000 7ff5982eb700 -1 snap_mapper.add_oid found existing snaps mapped on 219:24fa6925:test-rados-api-smithi059-82957-76::foo:2, removing 2024-02-18T00:40:25.004 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:40:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:40:24.748+0000 7fc4d9625700 -1 snap_mapper.add_oid found existing snaps mapped on 219:24fa6925:test-rados-api-smithi059-82957-76::foo:2, removing 2024-02-18T00:40:25.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:25.074 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:24.750850+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:24.751132+0000 mon.a (mon.0) 2713 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:24.751748+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-75"}]: dispatch 2024-02-18T00:40:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:24.752072+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-75"}]: dispatch 2024-02-18T00:40:26.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:24.903873+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:26.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:24.750850+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:24.751132+0000 mon.a (mon.0) 2713 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:24.751748+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-75"}]: dispatch 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:24.752072+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-75"}]: dispatch 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:24.903873+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:24.750850+0000 mon.c (mon.2) 460 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:24.751132+0000 mon.a (mon.0) 2713 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:24.751748+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-75"}]: dispatch 2024-02-18T00:40:26.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:24.752072+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-75"}]: dispatch 2024-02-18T00:40:26.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:24.903873+0000 mon.a (mon.0) 2715 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 -- 172.21.15.59:0/4038874353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 msgr2=0x7fe1600f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 --2- 172.21.15.59:0/4038874353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f9090 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 rx=0x7fe150004870 tx=0x7fe150009cb0).stop 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 -- 172.21.15.59:0/4038874353 shutdown_connections 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 --2- 172.21.15.59:0/4038874353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe160102210 0x7fe160104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 --2- 172.21.15.59:0/4038874353 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f9090 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 --2- 172.21.15.59:0/4038874353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1600f9610 0x7fe160101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 -- 172.21.15.59:0/4038874353 >> 172.21.15.59:0/4038874353 conn(0x7fe1600f2820 msgr2=0x7fe1600f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 -- 172.21.15.59:0/4038874353 shutdown_connections 2024-02-18T00:40:26.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 -- 172.21.15.59:0/4038874353 wait complete. 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 librados: starting msgr at 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 librados: starting objecter 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.075+0000 7fe1658c3700 1 Processor -- start 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 -- start start 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 librados: setting wanted keys 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 librados: calling monclient init 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1600f9610 0x7fe1600f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe160102210 0x7fe1600f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160116710 con 0x7fe1600f9610 2024-02-18T00:40:26.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160116880 con 0x7fe160102210 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe1658c3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe160116a40 con 0x7fe1600f8cb0 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1600f9610 0x7fe1600f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:36634/0 (socket says 172.21.15.59:36634) 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 -- 172.21.15.59:0/2815702821 learned_addr learned my addr 172.21.15.59:0/2815702821 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe160102210 0x7fe1600f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 -- 172.21.15.59:0/2815702821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe160102210 msgr2=0x7fe1600f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe160102210 0x7fe1600f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 -- 172.21.15.59:0/2815702821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1600f9610 msgr2=0x7fe1600f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1600f9610 0x7fe1600f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 -- 172.21.15.59:0/2815702821 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150005040 con 0x7fe1600f8cb0 2024-02-18T00:40:26.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe160102210 0x7fe1600f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f6c60 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 rx=0x7fe15000b1a0 tx=0x7fe150007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.076+0000 7fe147fff700 10 client.?.objecter ms_handle_connect 0x7fe1600f8cb0 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe147fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15000e030 con 0x7fe1600f8cb0 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe160116cc0 con 0x7fe1600f8cb0 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe1658c3700 10 client.41303.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1601170c0 con 0x7fe1600f8cb0 2024-02-18T00:40:26.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe1658c3700 1 librados: init done 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:40:26.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe150004490 con 0x7fe1600f8cb0 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.077+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150018070 con 0x7fe1600f8cb0 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.078+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe14c004f30 con 0x7fe1600f8cb0 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.078+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe150027440 con 0x7fe1600f8cb0 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe148062c90 0x7fe148065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(492..492 src has 180..492) v4 ==== 7766+0+0 (secure 0 0 0) 0x7fe15007e3d0 con 0x7fe1600f8cb0 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 osd_map(492..492 src has 180..492) v4 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 3 client.41303.objecter handle_osd_map got epochs [492,492] > 0 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 3 client.41303.objecter handle_osd_map decoding full epoch 492 2024-02-18T00:40:26.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe15e7fc700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe148062c90 0x7fe148065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 20 client.41303.objecter dump_active .. 0 homeless 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v492) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe150085070 con 0x7fe1600f8cb0 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 10 client.41303.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 1 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe14c004560 0x7fe14c0049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 1 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 0 1 2024-02-18T00:40:26.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0051d0 osd=0 4 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 10 client.41303.objecter _send_command 1 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c008960 con 0x7fe14c004560 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe15e7fc700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe148062c90 0x7fe148065150 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 rx=0x7fe1600f8400 tx=0x7fe154009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe148062c90 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.079+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe14c004560 0x7fe14c0049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.080+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c004560 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.080+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe14c004560 0x7fe14c0049a0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.080+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe14c008960 con 0x7fe14c004560 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.080+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.080+0000 7fe147fff700 10 client.41303.objecter _finish_command 1 = system:0 2024-02-18T00:40:26.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.080+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c0051d0 osd=0 4 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.080+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 0 1 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 10 client.41303.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 2 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 2 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c0051d0 osd=0 3 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 0 2 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0051d0 osd=0 4 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 10 client.41303.objecter _send_command 2 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.095+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c003fa0 con 0x7fe14c004560 2024-02-18T00:40:26.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c003fa0 con 0x7fe14c004560 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe147fff700 10 client.41303.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c0051d0 osd=0 4 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 0 2 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 10 client.41303.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 3 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6805/26279110 conn(0x7fe14c002740 0x7fe14c002bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 3 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 1 3 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007fb0 osd=1 4 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 10 client.41303.objecter _send_command 3 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c008700 con 0x7fe14c002740 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6805/26279110 conn(0x7fe14c002740 0x7fe14c002bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6805/26279110 conn(0x7fe14c002740 0x7fe14c002bd0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.097+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c002740 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.098+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe14c008700 con 0x7fe14c002740 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.098+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.098+0000 7fe147fff700 10 client.41303.objecter _finish_command 3 = system:0 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.098+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c007fb0 osd=1 4 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.098+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 1 3 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 10 client.41303.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 4 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 4 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c007fb0 osd=1 3 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 1 4 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007fb0 osd=1 4 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 10 client.41303.objecter _send_command 4 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.113+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c003890 con 0x7fe14c002740 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c003890 con 0x7fe14c002740 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe147fff700 10 client.41303.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c007fb0 osd=1 4 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 1 4 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 10 client.41303.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 5 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe14c00aff0 0x7fe14c00d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 5 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 2 5 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007ae0 osd=2 4 2024-02-18T00:40:26.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 10 client.41303.objecter _send_command 5 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c007e30 con 0x7fe14c00aff0 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.114+0000 7fe15e7fc700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe14c00aff0 0x7fe14c00d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.115+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c00aff0 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.115+0000 7fe15e7fc700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe14c00aff0 0x7fe14c00d3e0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.115+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe15400fb30 con 0x7fe14c00aff0 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.115+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.115+0000 7fe147fff700 10 client.41303.objecter _finish_command 5 = system:0 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.115+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c007ae0 osd=2 4 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.115+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 2 5 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 10 client.41303.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 6 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 6 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c007ae0 osd=2 3 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 2 6 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007ae0 osd=2 4 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 10 client.41303.objecter _send_command 6 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.130+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c003360 con 0x7fe14c00aff0 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.131+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c003360 con 0x7fe14c00aff0 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.131+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.131+0000 7fe147fff700 10 client.41303.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.131+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c007ae0 osd=2 4 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.131+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 2 6 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 10 client.41303.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 7 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe14c00ea50 0x7fe14c00ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 7 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 3 7 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c001bb0 osd=3 4 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 10 client.41303.objecter _send_command 7 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c002330 con 0x7fe14c00ea50 2024-02-18T00:40:26.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe14c00ea50 0x7fe14c00ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe14c00ea50 0x7fe14c00ef00 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.132+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c00ea50 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.133+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe14c002330 con 0x7fe14c00ea50 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.133+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.133+0000 7fe147fff700 10 client.41303.objecter _finish_command 7 = system:0 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.133+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c001bb0 osd=3 4 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.133+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 3 7 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 10 client.41303.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 8 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 8 2024-02-18T00:40:26.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c001bb0 osd=3 3 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 3 8 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c001bb0 osd=3 4 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 10 client.41303.objecter _send_command 8 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.158+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c00fbc0 con 0x7fe14c00ea50 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.160+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c00fbc0 con 0x7fe14c00ea50 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.160+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.160+0000 7fe147fff700 10 client.41303.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.160+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c001bb0 osd=3 4 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.160+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 3 8 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 10 client.41303.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 9 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe14c0120f0 0x7fe14c0144e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 9 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 4 9 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c00f3f0 osd=4 4 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 10 client.41303.objecter _send_command 9 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c017130 con 0x7fe14c0120f0 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe14c0120f0 0x7fe14c0144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c0120f0 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.161+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe14c0120f0 0x7fe14c0144e0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.162+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe14c017130 con 0x7fe14c0120f0 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.162+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.162+0000 7fe147fff700 10 client.41303.objecter _finish_command 9 = system:0 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.162+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c00f3f0 osd=4 4 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.162+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 4 9 2024-02-18T00:40:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 10 client.41303.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 10 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 10 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c00f3f0 osd=4 3 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 4 10 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c00f3f0 osd=4 4 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 10 client.41303.objecter _send_command 10 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.179+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c0149d0 con 0x7fe14c0120f0 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c0149d0 con 0x7fe14c0120f0 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe147fff700 10 client.41303.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c00f3f0 osd=4 4 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 4 10 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 10 client.41303.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 11 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe14c0150b0 0x7fe14c0174a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 11 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 5 11 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0149d0 osd=5 4 2024-02-18T00:40:26.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 10 client.41303.objecter _send_command 11 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c01a290 con 0x7fe14c0150b0 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.181+0000 7fe15e7fc700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe14c0150b0 0x7fe14c0174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.182+0000 7fe15e7fc700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe14c0150b0 0x7fe14c0174a0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.182+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c0150b0 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.183+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe14c01a290 con 0x7fe14c0150b0 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.183+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.183+0000 7fe147fff700 10 client.41303.objecter _finish_command 11 = system:0 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.183+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c0149d0 osd=5 4 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.183+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 5 11 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 10 client.41303.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 12 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 12 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c0149d0 osd=5 3 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 5 12 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0149d0 osd=5 4 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 10 client.41303.objecter _send_command 12 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.201+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c01a250 con 0x7fe14c0150b0 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c01a250 con 0x7fe14c0150b0 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe147fff700 10 client.41303.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c0149d0 osd=5 4 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 5 12 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 10 client.41303.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 13 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe14c0195b0 0x7fe14c019a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 13 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 6 13 2024-02-18T00:40:26.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c01a250 osd=6 4 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 10 client.41303.objecter _send_command 13 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.203+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c01d5f0 con 0x7fe14c0195b0 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.204+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe14c0195b0 0x7fe14c019a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.204+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c0195b0 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.204+0000 7fe15f7fe700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe14c0195b0 0x7fe14c019a30 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.205+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe14c01d5f0 con 0x7fe14c0195b0 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.205+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.205+0000 7fe147fff700 10 client.41303.objecter _finish_command 13 = system:0 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.205+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c01a250 osd=6 4 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.205+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 6 13 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 10 client.41303.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 14 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 14 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c01a250 osd=6 3 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 6 14 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c01a250 osd=6 4 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 10 client.41303.objecter _send_command 14 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.220+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c017a30 con 0x7fe14c0195b0 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c017a30 con 0x7fe14c0195b0 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe147fff700 10 client.41303.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c01a250 osd=6 4 2024-02-18T00:40:26.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 6 14 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 10 client.41303.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 15 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6812/53025171 conn(0x7fe14c018140 0x7fe14c0185a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 15 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 7 15 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c017a30 osd=7 4 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 10 client.41303.objecter _send_command 15 2024-02-18T00:40:26.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.222+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c018c10 con 0x7fe14c018140 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.223+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6812/53025171 conn(0x7fe14c018140 0x7fe14c0185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.223+0000 7fe147fff700 10 client.41303.objecter ms_handle_connect 0x7fe14c018140 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.223+0000 7fe15effd700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6812/53025171 conn(0x7fe14c018140 0x7fe14c0185a0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.224+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe14c018c10 con 0x7fe14c018140 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.224+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.224+0000 7fe147fff700 10 client.41303.objecter _finish_command 15 = system:0 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.224+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c017a30 osd=7 4 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.224+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 7 15 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 10 client.41303.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign -1 16 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 20 client.41303.objecter _get_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_remove -1 16 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 20 client.41303.objecter get_session s=0x7fe14c017a30 osd=7 3 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 15 client.41303.objecter _session_command_op_assign 7 16 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c017a30 osd=7 4 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 10 client.41303.objecter _send_command 16 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.238+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fe14c018c10 con 0x7fe14c018140 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.240+0000 7fe147fff700 1 -- 172.21.15.59:0/2815702821 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fe14c018c10 con 0x7fe14c018140 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.240+0000 7fe147fff700 10 client.41303.objecter ms_dispatch 0x7fe160001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.240+0000 7fe147fff700 10 client.41303.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.240+0000 7fe147fff700 20 client.41303.objecter put_session s=0x7fe14c017a30 osd=7 4 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.240+0000 7fe147fff700 15 client.41303.objecter _session_command_op_remove 7 16 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 librados: watch_flush enter 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 librados: watch_flush exit 2024-02-18T00:40:26.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.0 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe14c004560 msgr2=0x7fe14c0049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe14c004560 0x7fe14c0049a0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0051d0 osd=0 1 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.1 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6805/26279110 conn(0x7fe14c002740 msgr2=0x7fe14c002bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6805/26279110 conn(0x7fe14c002740 0x7fe14c002bd0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007fb0 osd=1 1 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.2 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe14c00aff0 msgr2=0x7fe14c00d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe14c00aff0 0x7fe14c00d3e0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c007ae0 osd=2 1 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.3 2024-02-18T00:40:26.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe14c00ea50 msgr2=0x7fe14c00ef00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe14c00ea50 0x7fe14c00ef00 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c001bb0 osd=3 1 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.4 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe14c0120f0 msgr2=0x7fe14c0144e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe14c0120f0 0x7fe14c0144e0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe15f7fe700 1 -- 172.21.15.59:0/2815702821 reap_dead start 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c00f3f0 osd=4 1 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.5 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe14c0150b0 msgr2=0x7fe14c0174a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe14c0150b0 0x7fe14c0174a0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe15f7fe700 1 -- 172.21.15.59:0/2815702821 reap_dead start 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c0149d0 osd=5 1 2024-02-18T00:40:26.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.6 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe14c0195b0 msgr2=0x7fe14c019a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe14c0195b0 0x7fe14c019a30 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c01a250 osd=6 1 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter close_session for osd.7 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6812/53025171 conn(0x7fe14c018140 msgr2=0x7fe14c0185a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.137:6812/53025171 conn(0x7fe14c018140 0x7fe14c0185a0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter put_session s=0x7fe14c017a30 osd=7 1 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 20 client.41303.objecter shutdown clearing up homeless session... 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 10 client.41303.objecter successfully canceled tick 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe15f7fe700 1 -- 172.21.15.59:0/2815702821 reap_dead start 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe148062c90 msgr2=0x7fe148065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe148062c90 0x7fe148065150 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 rx=0x7fe1600f8400 tx=0x7fe154009040).stop 2024-02-18T00:40:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 msgr2=0x7fe1600f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f6c60 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 rx=0x7fe15000b1a0 tx=0x7fe150007f90).stop 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 shutdown_connections 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1600f8cb0 0x7fe1600f6c60 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 --2- 172.21.15.59:0/2815702821 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe148062c90 0x7fe148065150 unknown :-1 s=CLOSED pgs=3137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 >> 172.21.15.59:0/2815702821 conn(0x7fe1600f2820 msgr2=0x7fe1600f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 shutdown_connections 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 -- 172.21.15.59:0/2815702821 wait complete. 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.241+0000 7fe1658c3700 1 librados: shutdown 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.288 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:40:26.289 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:26.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.498+0000 7ff0b068f700 1 -- 172.21.15.59:0/425175983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80fe510 msgr2=0x7ff0a80f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.498+0000 7ff0b068f700 1 --2- 172.21.15.59:0/425175983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80fe510 0x7ff0a80f7d40 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 rx=0x7ff09c004870 tx=0x7ff09c009cb0).stop 2024-02-18T00:40:26.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.498+0000 7ff0b068f700 1 -- 172.21.15.59:0/425175983 shutdown_connections 2024-02-18T00:40:26.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.498+0000 7ff0b068f700 1 --2- 172.21.15.59:0/425175983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0a80f8420 0x7ff0a8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.498+0000 7ff0b068f700 1 --2- 172.21.15.59:0/425175983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fdbb0 0x7ff0a80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.498+0000 7ff0b068f700 1 --2- 172.21.15.59:0/425175983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80fe510 0x7ff0a80f7d40 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.498+0000 7ff0b068f700 1 -- 172.21.15.59:0/425175983 >> 172.21.15.59:0/425175983 conn(0x7ff0a80f3800 msgr2=0x7ff0a80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 -- 172.21.15.59:0/425175983 shutdown_connections 2024-02-18T00:40:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 -- 172.21.15.59:0/425175983 wait complete. 2024-02-18T00:40:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 librados: starting msgr at 2024-02-18T00:40:26.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 librados: starting objecter 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 Processor -- start 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 -- start start 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 librados: setting wanted keys 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.499+0000 7ff0b068f700 1 librados: calling monclient init 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0b068f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80f8420 0x7ff0a810e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0b068f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0a80fdbb0 0x7ff0a810e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0b068f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fe510 0x7ff0a8108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0b068f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8108710 con 0x7ff0a80fe510 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0b068f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8108880 con 0x7ff0a80fdbb0 2024-02-18T00:40:26.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0b068f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8108a40 con 0x7ff0a80f8420 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fe510 0x7ff0a8108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fe510 0x7ff0a8108220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36872/0 (socket says 172.21.15.59:36872) 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 -- 172.21.15.59:0/2915185451 learned_addr learned my addr 172.21.15.59:0/2915185451 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0ae42b700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80f8420 0x7ff0a810e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0adc2a700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0a80fdbb0 0x7ff0a810e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 -- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80f8420 msgr2=0x7ff0a810e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80f8420 0x7ff0a810e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 -- 172.21.15.59:0/2915185451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0a80fdbb0 msgr2=0x7ff0a810e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0a80fdbb0 0x7ff0a810e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 -- 172.21.15.59:0/2915185451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098005040 con 0x7ff0a80fe510 2024-02-18T00:40:26.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0ae42b700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80f8420 0x7ff0a810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0aec2c700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fe510 0x7ff0a8108220 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 rx=0x7ff0a000ec20 tx=0x7ff0a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0a77fe700 10 client.?.objecter ms_handle_connect 0x7ff0a80fe510 2024-02-18T00:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0a77fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.500+0000 7ff0a77fe700 1 -- 172.21.15.59:0/2915185451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0009950 con 0x7ff0a80fe510 2024-02-18T00:40:26.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.501+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c005040 con 0x7ff0a80fe510 2024-02-18T00:40:26.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.501+0000 7ff0a77fe700 1 -- 172.21.15.59:0/2915185451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a000f040 con 0x7ff0a80fe510 2024-02-18T00:40:26.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.501+0000 7ff0a77fe700 1 -- 172.21.15.59:0/2915185451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0007df0 con 0x7ff0a80fe510 2024-02-18T00:40:26.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.501+0000 7ff0b068f700 10 client.31503.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:26.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.501+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a8109020 con 0x7ff0a80fe510 2024-02-18T00:40:26.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.501+0000 7ff0b068f700 1 librados: init done 2024-02-18T00:40:26.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.502+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a805ee80 con 0x7ff0a80fe510 2024-02-18T00:40:26.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.502+0000 7ff0a77fe700 1 -- 172.21.15.59:0/2915185451 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff0a00092e0 con 0x7ff0a80fe510 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0a77fe700 1 --2- 172.21.15.59:0/2915185451 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff090062c90 0x7ff090065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0a77fe700 1 -- 172.21.15.59:0/2915185451 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(492..492 src has 180..492) v4 ==== 7766+0+0 (secure 0 0 0) 0x7ff0a007cd80 con 0x7ff0a80fe510 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0a77fe700 10 client.31503.objecter ms_dispatch 0x7ff0a8001550 osd_map(492..492 src has 180..492) v4 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0a77fe700 3 client.31503.objecter handle_osd_map got epochs [492,492] > 0 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0a77fe700 3 client.31503.objecter handle_osd_map decoding full epoch 492 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0ae42b700 1 --2- 172.21.15.59:0/2915185451 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff090062c90 0x7ff090065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0a77fe700 20 client.31503.objecter dump_active .. 0 homeless 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0a77fe700 10 client.31503.objecter ms_handle_connect 0x7ff090062c90 2024-02-18T00:40:26.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.503+0000 7ff0ae42b700 1 --2- 172.21.15.59:0/2915185451 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff090062c90 0x7ff090065150 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 rx=0x7ff098009760 tx=0x7ff09800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:26.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.507+0000 7ff0a77fe700 1 -- 172.21.15.59:0/2915185451 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff0a0049f60 con 0x7ff0a80fe510 2024-02-18T00:40:26.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.684+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}) v1 -- 0x7ff0a805fa60 con 0x7ff090062c90 2024-02-18T00:40:26.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.684+0000 7ff0a77fe700 1 -- 172.21.15.59:0/2915185451 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+292 (secure 0 0 0) 0x7ff0a805fa60 con 0x7ff090062c90 2024-02-18T00:40:26.686 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 618 (0%) 2024-02-18T00:40:26.686 INFO:teuthology.orchestra.run.smithi059.stdout:avg 77.25 2024-02-18T00:40:26.686 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 6.0156 -> 6.0156 (expected baseline 8.22154) 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.2 with 70 -> 70 pgs (0.906149 -> 0.906149 * mean) 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.3 with 87 -> 87 pgs (1.12621 -> 1.12621 * mean) 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout:oload 110 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 3 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0033 2024-02-18T00:40:26.687 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0037 2024-02-18T00:40:26.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 10 librados: watch_flush enter 2024-02-18T00:40:26.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 10 librados: watch_flush exit 2024-02-18T00:40:26.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 20 client.31503.objecter shutdown clearing up homeless session... 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 10 client.31503.objecter successfully canceled tick 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff090062c90 msgr2=0x7ff090065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 --2- 172.21.15.59:0/2915185451 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff090062c90 0x7ff090065150 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 rx=0x7ff098009760 tx=0x7ff09800b040).stop 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fe510 msgr2=0x7ff0a8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fe510 0x7ff0a8108220 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 rx=0x7ff0a000ec20 tx=0x7ff0a000b040).stop 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 shutdown_connections 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0a80fe510 0x7ff0a8108220 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0a80f8420 0x7ff0a810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 --2- 172.21.15.59:0/2915185451 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff090062c90 0x7ff090065150 unknown :-1 s=CLOSED pgs=3138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 --2- 172.21.15.59:0/2915185451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0a80fdbb0 0x7ff0a810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:26.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 >> 172.21.15.59:0/2915185451 conn(0x7ff0a80f3800 msgr2=0x7ff0a80fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 shutdown_connections 2024-02-18T00:40:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.686+0000 7ff0b068f700 1 -- 172.21.15.59:0/2915185451 wait complete. 2024-02-18T00:40:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:26.687+0000 7ff0b068f700 1 librados: shutdown 2024-02-18T00:40:26.690 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-02-18T00:40:27.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:25.728643+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-18T00:40:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:25.937956+0000 mgr.y (mgr.24454) 471 : cluster [DBG] pgmap v753: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-02-18T00:40:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:26.080052+0000 mon.c (mon.2) 462 : audit [DBG] from='client.? 172.21.15.59:0/2815702821' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:25.728643+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-18T00:40:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:25.937956+0000 mgr.y (mgr.24454) 471 : cluster [DBG] pgmap v753: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-02-18T00:40:27.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:26.080052+0000 mon.c (mon.2) 462 : audit [DBG] from='client.? 172.21.15.59:0/2815702821' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:27.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:25.728643+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-18T00:40:27.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:25.937956+0000 mgr.y (mgr.24454) 471 : cluster [DBG] pgmap v753: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-02-18T00:40:27.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:26.080052+0000 mon.c (mon.2) 462 : audit [DBG] from='client.? 172.21.15.59:0/2815702821' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:27.890 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:40:27.890 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:40:27.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:26.685411+0000 mgr.y (mgr.24454) 472 : audit [DBG] from='client.31503 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:27.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:26.748466+0000 mon.a (mon.0) 2717 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-18T00:40:27.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:26.752433+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:27.912 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:26.753099+0000 mon.a (mon.0) 2718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:27.912 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:26.685411+0000 mgr.y (mgr.24454) 472 : audit [DBG] from='client.31503 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:27.912 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:26.748466+0000 mon.a (mon.0) 2717 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-18T00:40:27.913 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:26.752433+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:27.913 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:26.753099+0000 mon.a (mon.0) 2718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:28.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:26.685411+0000 mgr.y (mgr.24454) 472 : audit [DBG] from='client.31503 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:26.748466+0000 mon.a (mon.0) 2717 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-18T00:40:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:26.752433+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:26.753099+0000 mon.a (mon.0) 2718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:28.658 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-02-18T00:40:28.658 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight-by-utilization 110 0.05 2024-02-18T00:40:28.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:27.748302+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:27.748412+0000 mon.a (mon.0) 2720 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:27.751014+0000 mon.c (mon.2) 464 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:27.760599+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:27.761394+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:27.938395+0000 mgr.y (mgr.24454) 473 : cluster [DBG] pgmap v756: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:28.714300+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:28.714385+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:28.718776+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:28.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:28.719102+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:27.748302+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:27.748412+0000 mon.a (mon.0) 2720 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:27.751014+0000 mon.c (mon.2) 464 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:27.760599+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:27.761394+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:27.938395+0000 mgr.y (mgr.24454) 473 : cluster [DBG] pgmap v756: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:28.714300+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:28.714385+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-18T00:40:28.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:28.718776+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:28.994 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:28.719102+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:27.748302+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:27.748412+0000 mon.a (mon.0) 2720 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:27.751014+0000 mon.c (mon.2) 464 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:27.760599+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:27.761394+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:27.938395+0000 mgr.y (mgr.24454) 473 : cluster [DBG] pgmap v756: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:28.714300+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:28.714385+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-18T00:40:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:28.718776+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:29.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:28.719102+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:29.528 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-18T00:40:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:29.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:29.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:29.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:40:29.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:29.702 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:29.725 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:29.728 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:29.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:29.829 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:29.860 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:29.880 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:29.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:29.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:29.984 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:30.012 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:30.034 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:30.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:30.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:30.165 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:30.407 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:30.414 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:30.418 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:30.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:30.448 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:30.449 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:30.519 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:30.548 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:30.568 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:30.571 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:30.572 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:30.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:30.705 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:30.835 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:30.840 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:30.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:30.942 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:29.717091+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:29.717201+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:29.720830+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:29.721309+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:29.938946+0000 mgr.y (mgr.24454) 474 : cluster [DBG] pgmap v759: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:29.717091+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:29.717201+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-18T00:40:30.968 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:29.720830+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:30.969 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:29.721309+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:30.969 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:29.938946+0000 mgr.y (mgr.24454) 474 : cluster [DBG] pgmap v759: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:30.970 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:29.717091+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:29.717201+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-18T00:40:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:29.720830+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:29.721309+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:29.938946+0000 mgr.y (mgr.24454) 474 : cluster [DBG] pgmap v759: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:31.355 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:40:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:40:31.731 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:40:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:40:31.919 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:31.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:30.722260+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:31.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:30.722342+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-18T00:40:31.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:30.726416+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:31.921 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:30.727438+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:31.921 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:30.722260+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:31.921 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:30.722342+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-18T00:40:31.922 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:30.726416+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:31.922 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:30.727438+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:31.922 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:31.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/878151245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fdbb0 msgr2=0x7f6aa80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/878151245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fdbb0 0x7f6aa80fdf90 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 rx=0x7f6aa4004870 tx=0x7f6aa4009cb0).stop 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/878151245 shutdown_connections 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/878151245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa80f8420 0x7f6aa8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/878151245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fdbb0 0x7f6aa80fdf90 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/878151245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aa80fe510 0x7f6aa80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/878151245 >> 172.21.15.59:0/878151245 conn(0x7f6aa80f3800 msgr2=0x7f6aa80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/878151245 shutdown_connections 2024-02-18T00:40:31.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.925+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/878151245 wait complete. 2024-02-18T00:40:31.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 librados: starting msgr at 2024-02-18T00:40:31.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 librados: starting objecter 2024-02-18T00:40:31.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 Processor -- start 2024-02-18T00:40:31.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 -- start start 2024-02-18T00:40:31.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 librados: setting wanted keys 2024-02-18T00:40:31.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 librados: calling monclient init 2024-02-18T00:40:31.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aa80f8420 0x7f6aa810e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa80fdbb0 0x7f6aa810e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fe510 0x7f6aa81083f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa81088e0 con 0x7f6aa80f8420 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8108a50 con 0x7f6aa80fdbb0 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aaf9f4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa8108c10 con 0x7f6aa80fe510 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aadf91700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fe510 0x7f6aa81083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aad790700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aa80f8420 0x7f6aa810e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aadf91700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fe510 0x7f6aa81083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55522/0 (socket says 172.21.15.59:55522) 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aadf91700 1 -- 172.21.15.59:0/3513643919 learned_addr learned my addr 172.21.15.59:0/3513643919 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.926+0000 7f6aacf8f700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa80fdbb0 0x7f6aa810e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aadf91700 1 -- 172.21.15.59:0/3513643919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa80fdbb0 msgr2=0x7f6aa810e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aadf91700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa80fdbb0 0x7f6aa810e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:31.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aadf91700 1 -- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aa80f8420 msgr2=0x7f6aa810e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aadf91700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aa80f8420 0x7f6aa810e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aadf91700 1 -- 172.21.15.59:0/3513643919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a98005040 con 0x7f6aa80fe510 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aad790700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aa80f8420 0x7f6aa810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aadf91700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fe510 0x7f6aa81083f0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 rx=0x7f6aa000bc30 tx=0x7f6aa0009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6a9e7fc700 10 client.?.objecter ms_handle_connect 0x7f6aa80fe510 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6a9e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6a9e7fc700 1 -- 172.21.15.59:0/3513643919 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000c630 con 0x7f6aa80fe510 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6a9e7fc700 1 -- 172.21.15.59:0/3513643919 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa0010070 con 0x7f6aa80fe510 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa4005040 con 0x7f6aa80fe510 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6a9e7fc700 1 -- 172.21.15.59:0/3513643919 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0007810 con 0x7f6aa80fe510 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aaf9f4700 10 client.41315.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:31.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa8109250 con 0x7f6aa80fe510 2024-02-18T00:40:31.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.927+0000 7f6aaf9f4700 1 librados: init done 2024-02-18T00:40:31.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.928+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa805ee80 con 0x7f6aa80fe510 2024-02-18T00:40:31.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.928+0000 7f6a9e7fc700 1 -- 172.21.15.59:0/3513643919 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6aa0007bc0 con 0x7f6aa80fe510 2024-02-18T00:40:31.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6a9e7fc700 1 --2- 172.21.15.59:0/3513643919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6a94062c90 0x7f6a94065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6a9e7fc700 1 -- 172.21.15.59:0/3513643919 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(498..498 src has 180..498) v4 ==== 8218+0+0 (secure 0 0 0) 0x7f6aa007cb60 con 0x7f6aa80fe510 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6a9e7fc700 10 client.41315.objecter ms_dispatch 0x7f6aa8001550 osd_map(498..498 src has 180..498) v4 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6a9e7fc700 3 client.41315.objecter handle_osd_map got epochs [498,498] > 0 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6a9e7fc700 3 client.41315.objecter handle_osd_map decoding full epoch 498 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6aad790700 1 --2- 172.21.15.59:0/3513643919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6a94062c90 0x7f6a94065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6a9e7fc700 20 client.41315.objecter dump_active .. 0 homeless 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6a9e7fc700 10 client.41315.objecter ms_handle_connect 0x7f6a94062c90 2024-02-18T00:40:31.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.929+0000 7f6aad790700 1 --2- 172.21.15.59:0/3513643919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6a94062c90 0x7f6a94065150 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 rx=0x7f6aa40043d0 tx=0x7f6aa4009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:31.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:31.933+0000 7f6a9e7fc700 1 -- 172.21.15.59:0/3513643919 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6aa0050010 con 0x7f6aa80fe510 2024-02-18T00:40:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:30.722260+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:30.722342+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-18T00:40:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:30.726416+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:30.727438+0000 mon.a (mon.0) 2730 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:32.023 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:32.076 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:32.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:32.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:32.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.120+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f6aa81099a0 con 0x7f6aa80fe510 2024-02-18T00:40:32.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:32.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:32] "GET /metrics HTTP/1.1" 200 190119 "" "Prometheus/2.33.4" 2024-02-18T00:40:32.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.743+0000 7f6a9e7fc700 1 -- 172.21.15.59:0/3513643919 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v499) v1 ==== 91+0+0 (secure 0 0 0) 0x7f6aa81099a0 con 0x7f6aa80fe510 2024-02-18T00:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 10 librados: watch_flush enter 2024-02-18T00:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 10 librados: watch_flush exit 2024-02-18T00:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 20 client.41315.objecter shutdown clearing up homeless session... 2024-02-18T00:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 10 client.41315.objecter successfully canceled tick 2024-02-18T00:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6a94062c90 msgr2=0x7f6a94065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/3513643919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6a94062c90 0x7f6a94065150 secure :-1 s=READY pgs=3139 cs=0 l=1 rev1=1 rx=0x7f6aa40043d0 tx=0x7f6aa4009cb0).stop 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fe510 msgr2=0x7f6aa81083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fe510 0x7f6aa81083f0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 rx=0x7f6aa000bc30 tx=0x7f6aa0009840).stop 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 shutdown_connections 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6aa80fe510 0x7f6aa81083f0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aa80f8420 0x7f6aa810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/3513643919 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6a94062c90 0x7f6a94065150 unknown :-1 s=CLOSED pgs=3139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 --2- 172.21.15.59:0/3513643919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6aa80fdbb0 0x7f6aa810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 >> 172.21.15.59:0/3513643919 conn(0x7f6aa80f3800 msgr2=0x7f6aa80fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 shutdown_connections 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 -- 172.21.15.59:0/3513643919 wait complete. 2024-02-18T00:40:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:32.746+0000 7f6aaf9f4700 1 librados: shutdown 2024-02-18T00:40:32.750 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:40:32.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:31.354694+0000 mgr.y (mgr.24454) 475 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:31.725462+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:31.725543+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:31.750531+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:31.750919+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:31.939319+0000 mgr.y (mgr.24454) 476 : cluster [DBG] pgmap v762: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:32.121808+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.59:0/3513643919' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:32.122107+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:40:32.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:31.354694+0000 mgr.y (mgr.24454) 475 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:32.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:31.725462+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:32.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:31.725543+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-18T00:40:32.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:31.750531+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T00:40:32.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:31.750919+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T00:40:32.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:31.939319+0000 mgr.y (mgr.24454) 476 : cluster [DBG] pgmap v762: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:32.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:32.121808+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.59:0/3513643919' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:40:32.996 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:32.122107+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:31.354694+0000 mgr.y (mgr.24454) 475 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:31.725462+0000 mon.a (mon.0) 2731 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:31.725543+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:31.750531+0000 mon.c (mon.2) 469 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:31.750919+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:31.939319+0000 mgr.y (mgr.24454) 476 : cluster [DBG] pgmap v762: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:32.121808+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.59:0/3513643919' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:40:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:32.122107+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:40:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:33.867 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:33.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:33.969 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:33.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:32.731122+0000 mon.a (mon.0) 2735 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:33.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:32.741352+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-18T00:40:33.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:32.741450+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:40:33.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:32.741508+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-18T00:40:33.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:32.763725+0000 mon.c (mon.2) 471 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T00:40:33.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:32.764425+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T00:40:33.997 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:33.082001+0000 osd.3 (osd.3) 269 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:40:33.997 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:33.083140+0000 osd.3 (osd.3) 270 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:40:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:34.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:32.731122+0000 mon.a (mon.0) 2735 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:34.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:32.741352+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:32.741450+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:32.741508+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:32.763725+0000 mon.c (mon.2) 471 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:32.764425+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:33.082001+0000 osd.3 (osd.3) 269 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:33.083140+0000 osd.3 (osd.3) 270 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:32.731122+0000 mon.a (mon.0) 2735 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:32.741352+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:32.741450+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:40:34.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:32.741508+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-18T00:40:34.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:32.763725+0000 mon.c (mon.2) 471 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T00:40:34.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:32.764425+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T00:40:34.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:33.082001+0000 osd.3 (osd.3) 269 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:40:34.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:33.083140+0000 osd.3 (osd.3) 270 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:40:34.018 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:34.020 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:34.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:34.122 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:34.150 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:34.170 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:34.172 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:34.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:34.273 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:34.300 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:34.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:34.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:34.422 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:34.449 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:34.468 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:34.470 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:34.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:34.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:34.605 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:34.808 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:34.811 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:34.812 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:34.913 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:33.212285+0000 osd.4 (osd.4) 291 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:33.214353+0000 osd.4 (osd.4) 292 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:33.795396+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:33.795502+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:33.809324+0000 mon.c (mon.2) 472 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:33.810055+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:33.939680+0000 mgr.y (mgr.24454) 477 : cluster [DBG] pgmap v765: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:34.939 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:33.212285+0000 osd.4 (osd.4) 291 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:40:34.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:33.214353+0000 osd.4 (osd.4) 292 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:40:34.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:33.795396+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-18T00:40:34.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:33.795502+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-18T00:40:34.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:33.809324+0000 mon.c (mon.2) 472 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:34.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:33.810055+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:34.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:33.939680+0000 mgr.y (mgr.24454) 477 : cluster [DBG] pgmap v765: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:35.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:33.212285+0000 osd.4 (osd.4) 291 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:40:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:33.214353+0000 osd.4 (osd.4) 292 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:40:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:33.795396+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-18T00:40:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:33.795502+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-18T00:40:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:33.809324+0000 mon.c (mon.2) 472 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:33.810055+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:33.939680+0000 mgr.y (mgr.24454) 477 : cluster [DBG] pgmap v765: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:35.493 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:35.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:35.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:35.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:35.628 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:35.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.237153+0000 osd.4 (osd.4) 293 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.240061+0000 osd.4 (osd.4) 294 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.246095+0000 osd.0 (osd.0) 230 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.246937+0000 osd.0 (osd.0) 231 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.339797+0000 osd.2 (osd.2) 205 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.340620+0000 osd.2 (osd.2) 206 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:34.803139+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.803239+0000 mon.a (mon.0) 2744 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.827183+0000 osd.6 (osd.6) 241 : cluster [DBG] 25.d scrub starts 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:34.828145+0000 osd.6 (osd.6) 242 : cluster [DBG] 25.d scrub ok 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:34.843857+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:34.844280+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:35.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:34.844733+0000 mon.c (mon.2) 474 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-77"}]: dispatch 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:34.844990+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-77"}]: dispatch 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.237153+0000 osd.4 (osd.4) 293 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.240061+0000 osd.4 (osd.4) 294 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.246095+0000 osd.0 (osd.0) 230 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.246937+0000 osd.0 (osd.0) 231 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.339797+0000 osd.2 (osd.2) 205 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.340620+0000 osd.2 (osd.2) 206 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:34.803139+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.803239+0000 mon.a (mon.0) 2744 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.827183+0000 osd.6 (osd.6) 241 : cluster [DBG] 25.d scrub starts 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:34.828145+0000 osd.6 (osd.6) 242 : cluster [DBG] 25.d scrub ok 2024-02-18T00:40:35.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:34.843857+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:34.844280+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:34.844733+0000 mon.c (mon.2) 474 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-77"}]: dispatch 2024-02-18T00:40:35.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:34.844990+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-77"}]: dispatch 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.237153+0000 osd.4 (osd.4) 293 : cluster [DBG] 6.1e scrub starts 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.240061+0000 osd.4 (osd.4) 294 : cluster [DBG] 6.1e scrub ok 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.246095+0000 osd.0 (osd.0) 230 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.246937+0000 osd.0 (osd.0) 231 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.339797+0000 osd.2 (osd.2) 205 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.340620+0000 osd.2 (osd.2) 206 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:34.803139+0000 mon.a (mon.0) 2743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.803239+0000 mon.a (mon.0) 2744 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.827183+0000 osd.6 (osd.6) 241 : cluster [DBG] 25.d scrub starts 2024-02-18T00:40:36.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:34.828145+0000 osd.6 (osd.6) 242 : cluster [DBG] 25.d scrub ok 2024-02-18T00:40:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:34.843857+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:34.844280+0000 mon.a (mon.0) 2745 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:34.844733+0000 mon.c (mon.2) 474 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-77"}]: dispatch 2024-02-18T00:40:36.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:34.844990+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-77"}]: dispatch 2024-02-18T00:40:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:35.804001+0000 mon.a (mon.0) 2747 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-18T00:40:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:35.940207+0000 mgr.y (mgr.24454) 478 : cluster [DBG] pgmap v768: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T00:40:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.426224+0000 osd.5 (osd.5) 219 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:40:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.427201+0000 osd.5 (osd.5) 220 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:40:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.682618+0000 osd.7 (osd.7) 125 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:40:37.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.683598+0000 osd.7 (osd.7) 126 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:40:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:35.804001+0000 mon.a (mon.0) 2747 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-18T00:40:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:35.940207+0000 mgr.y (mgr.24454) 478 : cluster [DBG] pgmap v768: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T00:40:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.426224+0000 osd.5 (osd.5) 219 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:40:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.427201+0000 osd.5 (osd.5) 220 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:40:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.682618+0000 osd.7 (osd.7) 125 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:40:37.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.683598+0000 osd.7 (osd.7) 126 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:40:37.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:35.804001+0000 mon.a (mon.0) 2747 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-18T00:40:37.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:35.940207+0000 mgr.y (mgr.24454) 478 : cluster [DBG] pgmap v768: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T00:40:37.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.426224+0000 osd.5 (osd.5) 219 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:40:37.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.427201+0000 osd.5 (osd.5) 220 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:40:37.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.682618+0000 osd.7 (osd.7) 125 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:40:37.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.683598+0000 osd.7 (osd.7) 126 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:40:37.795 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:40:37.795 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:40:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.181296+0000 osd.0 (osd.0) 232 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:40:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.184212+0000 osd.0 (osd.0) 233 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:40:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.516465+0000 osd.1 (osd.1) 261 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:40:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.519465+0000 osd.1 (osd.1) 262 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:40:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:36.809831+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-18T00:40:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:36.812983+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:36.814860+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.181296+0000 osd.0 (osd.0) 232 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:40:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.184212+0000 osd.0 (osd.0) 233 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:40:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.516465+0000 osd.1 (osd.1) 261 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:40:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.519465+0000 osd.1 (osd.1) 262 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:40:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:36.809831+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-18T00:40:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:36.812983+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:36.814860+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 -- 172.21.15.59:0/3975935633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 msgr2=0x7f2f40103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/3975935633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f40103a20 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 rx=0x7f2f38009be0 tx=0x7f2f38004d10).stop 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 -- 172.21.15.59:0/3975935633 shutdown_connections 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/3975935633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f40103a20 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/3975935633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f400fcbb0 0x7f2f400fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/3975935633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f400fd510 0x7f2f400f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 -- 172.21.15.59:0/3975935633 >> 172.21.15.59:0/3975935633 conn(0x7f2f400f2820 msgr2=0x7f2f400f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 -- 172.21.15.59:0/3975935633 shutdown_connections 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.174+0000 7f2f44d6f700 1 -- 172.21.15.59:0/3975935633 wait complete. 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 librados: starting msgr at 2024-02-18T00:40:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 librados: starting objecter 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 Processor -- start 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 -- start start 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 librados: setting wanted keys 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 librados: calling monclient init 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f400f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f400fcbb0 0x7f2f400f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f400fd510 0x7f2f400f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40116440 con 0x7f2f400fd510 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f401165b0 con 0x7f2f400fcbb0 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f44d6f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40116770 con 0x7f2f400f7400 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f3e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f400f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f3ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f400fd510 0x7f2f400f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f3e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f400f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55552/0 (socket says 172.21.15.59:55552) 2024-02-18T00:40:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.175+0000 7f2f3e59c700 1 -- 172.21.15.59:0/991224131 learned_addr learned my addr 172.21.15.59:0/991224131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3dd9b700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f400fcbb0 0x7f2f400f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3e59c700 1 -- 172.21.15.59:0/991224131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f400fcbb0 msgr2=0x7f2f400f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3e59c700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f400fcbb0 0x7f2f400f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3e59c700 1 -- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f400fd510 msgr2=0x7f2f400f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3e59c700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f400fd510 0x7f2f400f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3e59c700 1 -- 172.21.15.59:0/991224131 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f380097c0 con 0x7f2f400f7400 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3ed9d700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f400fd510 0x7f2f400f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f3e59c700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f400f89b0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 rx=0x7f2f3000e970 tx=0x7f2f3000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f377fe700 10 client.?.objecter ms_handle_connect 0x7f2f400f7400 2024-02-18T00:40:38.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f377fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:38.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f377fe700 1 -- 172.21.15.59:0/991224131 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004660 con 0x7f2f400f7400 2024-02-18T00:40:38.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f377fe700 1 -- 172.21.15.59:0/991224131 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f3000f040 con 0x7f2f400f7400 2024-02-18T00:40:38.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f377fe700 1 -- 172.21.15.59:0/991224131 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30014790 con 0x7f2f400f7400 2024-02-18T00:40:38.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f40116a50 con 0x7f2f400f7400 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f44d6f700 10 client.41321.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f40116f90 con 0x7f2f400f7400 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.176+0000 7f2f44d6f700 1 librados: init done 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.177+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4005fa60 con 0x7f2f400f7400 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.177+0000 7f2f377fe700 1 -- 172.21.15.59:0/991224131 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2f300133c0 con 0x7f2f400f7400 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f377fe700 1 --2- 172.21.15.59:0/991224131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2f2c062c90 0x7f2f2c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f3dd9b700 1 --2- 172.21.15.59:0/991224131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2f2c062c90 0x7f2f2c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f377fe700 1 -- 172.21.15.59:0/991224131 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(503..503 src has 180..503) v4 ==== 8138+0+0 (secure 0 0 0) 0x7f2f3007cce0 con 0x7f2f400f7400 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f377fe700 10 client.41321.objecter ms_dispatch 0x7f2f40001550 osd_map(503..503 src has 180..503) v4 2024-02-18T00:40:38.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f377fe700 3 client.41321.objecter handle_osd_map got epochs [503,503] > 0 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f377fe700 3 client.41321.objecter handle_osd_map decoding full epoch 503 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f377fe700 20 client.41321.objecter dump_active .. 0 homeless 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f377fe700 10 client.41321.objecter ms_handle_connect 0x7f2f2c062c90 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.178+0000 7f2f3dd9b700 1 --2- 172.21.15.59:0/991224131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2f2c062c90 0x7f2f2c065150 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 rx=0x7f2f400fa150 tx=0x7f2f2800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.182+0000 7f2f377fe700 1 -- 172.21.15.59:0/991224131 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2f3004af60 con 0x7f2f400f7400 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.354+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}) v1 -- 0x7f2f400f9f00 con 0x7f2f2c062c90 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.356+0000 7f2f377fe700 1 -- 172.21.15.59:0/991224131 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+297 (secure 0 0 0) 0x7f2f400f9f00 con 0x7f2f2c062c90 2024-02-18T00:40:38.161 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 714 (0%) 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:avg 89.25 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 8.46685 -> 8.46685 (expected baseline 8.83707) 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.1 with 78 -> 78 pgs (0.87395 -> 0.87395 * mean) 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.3 with 103 -> 103 pgs (1.15406 -> 1.15406 * mean) 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:oload 110 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 0.05 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0033 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0037 2024-02-18T00:40:38.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.357+0000 7f2f44d6f700 10 librados: watch_flush enter 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.357+0000 7f2f44d6f700 10 librados: watch_flush exit 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.357+0000 7f2f44d6f700 20 client.41321.objecter shutdown clearing up homeless session... 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.357+0000 7f2f44d6f700 10 client.41321.objecter successfully canceled tick 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2f2c062c90 msgr2=0x7f2f2c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/991224131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2f2c062c90 0x7f2f2c065150 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 rx=0x7f2f400fa150 tx=0x7f2f2800b040).stop 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 msgr2=0x7f2f400f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f400f89b0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 rx=0x7f2f3000e970 tx=0x7f2f3000b040).stop 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 shutdown_connections 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f400fd510 0x7f2f400f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f400f7400 0x7f2f400f89b0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/991224131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2f2c062c90 0x7f2f2c065150 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 --2- 172.21.15.59:0/991224131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f400fcbb0 0x7f2f400f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 >> 172.21.15.59:0/991224131 conn(0x7f2f400f2820 msgr2=0x7f2f40060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 shutdown_connections 2024-02-18T00:40:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 -- 172.21.15.59:0/991224131 wait complete. 2024-02-18T00:40:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:37.358+0000 7f2f44d6f700 1 librados: shutdown 2024-02-18T00:40:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-02-18T00:40:38.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.181296+0000 osd.0 (osd.0) 232 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:40:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.184212+0000 osd.0 (osd.0) 233 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:40:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.516465+0000 osd.1 (osd.1) 261 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:40:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.519465+0000 osd.1 (osd.1) 262 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:40:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:36.809831+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-18T00:40:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:36.812983+0000 mon.c (mon.2) 475 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:36.814860+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:38.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:40:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 -- 172.21.15.59:0/2366811921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800f8420 msgr2=0x7f4c80104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 --2- 172.21.15.59:0/2366811921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800f8420 0x7f4c80104a20 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 rx=0x7f4c78009be0 tx=0x7f4c78004d10).stop 2024-02-18T00:40:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 -- 172.21.15.59:0/2366811921 shutdown_connections 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 --2- 172.21.15.59:0/2366811921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800f8420 0x7f4c80104a20 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 --2- 172.21.15.59:0/2366811921 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c800fdbb0 0x7f4c800fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 --2- 172.21.15.59:0/2366811921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c800fe510 0x7f4c800f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 -- 172.21.15.59:0/2366811921 >> 172.21.15.59:0/2366811921 conn(0x7f4c800f3800 msgr2=0x7f4c800f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 -- 172.21.15.59:0/2366811921 shutdown_connections 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.956+0000 7f4c87888700 1 -- 172.21.15.59:0/2366811921 wait complete. 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 librados: starting msgr at 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 librados: starting objecter 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 Processor -- start 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 -- start start 2024-02-18T00:40:38.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 librados: setting wanted keys 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 librados: calling monclient init 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c800f8420 0x7f4c8010dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800fdbb0 0x7f4c8010e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c800fe510 0x7f4c80107f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c80108420 con 0x7f4c800fdbb0 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c80108590 con 0x7f4c800fe510 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c87888700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c80108750 con 0x7f4c800f8420 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c84e23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800fdbb0 0x7f4c8010e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c84e23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800fdbb0 0x7f4c8010e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44080/0 (socket says 172.21.15.59:44080) 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c84e23700 1 -- 172.21.15.59:0/2786761466 learned_addr learned my addr 172.21.15.59:0/2786761466 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c85e25700 1 --2- 172.21.15.59:0/2786761466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c800fe510 0x7f4c80107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.957+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c800f8420 0x7f4c8010dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c84e23700 1 -- 172.21.15.59:0/2786761466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c800f8420 msgr2=0x7f4c8010dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c84e23700 1 --2- 172.21.15.59:0/2786761466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c800f8420 0x7f4c8010dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c84e23700 1 -- 172.21.15.59:0/2786761466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c800fe510 msgr2=0x7f4c80107f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c84e23700 1 --2- 172.21.15.59:0/2786761466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c800fe510 0x7f4c80107f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c84e23700 1 -- 172.21.15.59:0/2786761466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c780097c0 con 0x7f4c800fdbb0 2024-02-18T00:40:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c84e23700 1 --2- 172.21.15.59:0/2786761466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800fdbb0 0x7f4c8010e280 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 rx=0x7f4c700027f0 tx=0x7f4c70002b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c767fc700 10 client.?.objecter ms_handle_connect 0x7f4c800fdbb0 2024-02-18T00:40:38.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c767fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70007b40 con 0x7f4c800fdbb0 2024-02-18T00:40:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c80108a30 con 0x7f4c800fdbb0 2024-02-18T00:40:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c87888700 10 client.31521.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c80108db0 con 0x7f4c800fdbb0 2024-02-18T00:40:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c87888700 1 librados: init done 2024-02-18T00:40:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c7000f040 con 0x7f4c800fdbb0 2024-02-18T00:40:38.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.958+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c700146f0 con 0x7f4c800fdbb0 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.959+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f4c64004f30 con 0x7f4c800fdbb0 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4c70007ca0 con 0x7f4c800fdbb0 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4c6c062c90 0x7f4c6c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(505..505 src has 180..505) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f4c7007ce90 con 0x7f4c800fdbb0 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 osd_map(505..505 src has 180..505) v4 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 3 client.31521.objecter handle_osd_map got epochs [505,505] > 0 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 3 client.31521.objecter handle_osd_map decoding full epoch 505 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4c6c062c90 0x7f4c6c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 20 client.31521.objecter dump_active .. 0 homeless 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.960+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v505) v1 ==== 54+0+16 (secure 0 0 0) 0x7f4c7007f560 con 0x7f4c800fdbb0 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c6c062c90 2024-02-18T00:40:38.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4c6c062c90 0x7f4c6c065150 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 rx=0x7f4c7c004de0 tx=0x7f4c7c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 10 client.31521.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 1 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4c64004560 0x7f4c640049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 1 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 0 1 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640051d0 osd=0 4 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 10 client.31521.objecter _send_command 1 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c64008960 con 0x7f4c64004560 2024-02-18T00:40:38.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c85e25700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4c64004560 0x7f4c640049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c64004560 2024-02-18T00:40:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.961+0000 7f4c85e25700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4c64004560 0x7f4c640049a0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.962+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c64008960 con 0x7f4c64004560 2024-02-18T00:40:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.962+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:40:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.962+0000 7f4c767fc700 10 client.31521.objecter _finish_command 1 = system:0 2024-02-18T00:40:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.962+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640051d0 osd=0 4 2024-02-18T00:40:38.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.962+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 0 1 2024-02-18T00:40:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 10 client.31521.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 2 2024-02-18T00:40:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 2 2024-02-18T00:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640051d0 osd=0 3 2024-02-18T00:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 0 2 2024-02-18T00:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640051d0 osd=0 4 2024-02-18T00:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 10 client.31521.objecter _send_command 2 2024-02-18T00:40:38.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.977+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c64003fa0 con 0x7f4c64004560 2024-02-18T00:40:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.978+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c64003fa0 con 0x7f4c64004560 2024-02-18T00:40:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.978+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.978+0000 7f4c767fc700 10 client.31521.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.978+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640051d0 osd=0 4 2024-02-18T00:40:38.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.978+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 0 2 2024-02-18T00:40:38.980 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:40:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 10 client.31521.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:38.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6805/26279110 conn(0x7f4c640084c0 0x7f4c64002740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 1 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640043a0 osd=1 4 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 10 client.31521.objecter _send_command 3 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c64002de0 con 0x7f4c640084c0 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6805/26279110 conn(0x7f4c640084c0 0x7f4c64002740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c640084c0 2024-02-18T00:40:38.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6805/26279110 conn(0x7f4c640084c0 0x7f4c64002740 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.979+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c64002de0 con 0x7f4c640084c0 2024-02-18T00:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.980+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.980+0000 7f4c767fc700 10 client.31521.objecter _finish_command 3 = system:0 2024-02-18T00:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.980+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640043a0 osd=1 4 2024-02-18T00:40:38.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.980+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 1 3 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 10 client.31521.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 4 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 4 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640043a0 osd=1 3 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 1 4 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640043a0 osd=1 4 2024-02-18T00:40:38.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 10 client.31521.objecter _send_command 4 2024-02-18T00:40:38.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.994+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c64007ae0 con 0x7f4c640084c0 2024-02-18T00:40:38.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c64007ae0 con 0x7f4c640084c0 2024-02-18T00:40:38.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:38.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c767fc700 10 client.31521.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:38.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640043a0 osd=1 4 2024-02-18T00:40:38.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 1 4 2024-02-18T00:40:38.998 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 10 client.31521.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 5 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4c64003970 0x7f4c6400d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 5 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 2 5 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640037d0 osd=2 4 2024-02-18T00:40:38.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 10 client.31521.objecter _send_command 5 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c6400ff30 con 0x7f4c64003970 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c84e23700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4c64003970 0x7f4c6400d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c84e23700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4c64003970 0x7f4c6400d0b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.996+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c64003970 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.997+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c6400ff30 con 0x7f4c64003970 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.997+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.997+0000 7f4c767fc700 10 client.31521.objecter _finish_command 5 = system:0 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.997+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640037d0 osd=2 4 2024-02-18T00:40:39.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:38.997+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 2 5 2024-02-18T00:40:39.001 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:37.155355+0000 osd.0 (osd.0) 234 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:37.156209+0000 osd.0 (osd.0) 235 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:37.356223+0000 mgr.y (mgr.24454) 479 : audit [DBG] from='client.41321 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:37.551203+0000 osd.1 (osd.1) 263 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:37.554658+0000 osd.1 (osd.1) 264 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:37.810697+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:37.810847+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:37.830919+0000 mon.c (mon.2) 476 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:37.833832+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:37.834129+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:37.940549+0000 mgr.y (mgr.24454) 480 : cluster [DBG] pgmap v771: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T00:40:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.230344+0000 osd.4 (osd.4) 295 : cluster [DBG] 5.e scrub starts 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.233834+0000 osd.4 (osd.4) 296 : cluster [DBG] 5.e scrub ok 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:38.717723+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.717806+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:38.719036+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:38.726665+0000 mon.a (mon.0) 2755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:37.155355+0000 osd.0 (osd.0) 234 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:37.156209+0000 osd.0 (osd.0) 235 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:37.356223+0000 mgr.y (mgr.24454) 479 : audit [DBG] from='client.41321 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:37.551203+0000 osd.1 (osd.1) 263 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:37.554658+0000 osd.1 (osd.1) 264 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:40:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:37.810697+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:37.810847+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:37.830919+0000 mon.c (mon.2) 476 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:37.833832+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:37.834129+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:37.940549+0000 mgr.y (mgr.24454) 480 : cluster [DBG] pgmap v771: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.230344+0000 osd.4 (osd.4) 295 : cluster [DBG] 5.e scrub starts 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.233834+0000 osd.4 (osd.4) 296 : cluster [DBG] 5.e scrub ok 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:38.717723+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.717806+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:38.719036+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:39.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:38.726665+0000 mon.a (mon.0) 2755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 10 client.31521.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 6 2024-02-18T00:40:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:39.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 6 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640037d0 osd=2 3 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 2 6 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640037d0 osd=2 4 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 10 client.31521.objecter _send_command 6 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.012+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c640032a0 con 0x7f4c64003970 2024-02-18T00:40:39.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c640032a0 con 0x7f4c64003970 2024-02-18T00:40:39.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:39.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c767fc700 10 client.31521.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640037d0 osd=2 4 2024-02-18T00:40:39.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 2 6 2024-02-18T00:40:39.015 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 10 client.31521.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 7 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4c6400e840 0x7f4c6400ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 7 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 3 7 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64001bb0 osd=3 4 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 10 client.31521.objecter _send_command 7 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c64002260 con 0x7f4c6400e840 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.013+0000 7f4c85e25700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4c6400e840 0x7f4c6400ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.014+0000 7f4c85e25700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4c6400e840 0x7f4c6400ecd0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.014+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c6400e840 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.014+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c64002260 con 0x7f4c6400e840 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.014+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.014+0000 7f4c767fc700 10 client.31521.objecter _finish_command 7 = system:0 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.014+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c64001bb0 osd=3 4 2024-02-18T00:40:39.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.014+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 3 7 2024-02-18T00:40:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 10 client.31521.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 8 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 8 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c64001bb0 osd=3 3 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 3 8 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64001bb0 osd=3 4 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 10 client.31521.objecter _send_command 8 2024-02-18T00:40:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.029+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c6400f990 con 0x7f4c6400e840 2024-02-18T00:40:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c6400f990 con 0x7f4c6400e840 2024-02-18T00:40:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c767fc700 10 client.31521.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c64001bb0 osd=3 4 2024-02-18T00:40:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 3 8 2024-02-18T00:40:39.032 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:40:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 10 client.31521.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 9 2024-02-18T00:40:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4c64012160 0x7f4c64014550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 9 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 4 9 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c6400f990 osd=4 4 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 10 client.31521.objecter _send_command 9 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.030+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c640171d0 con 0x7f4c64012160 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.031+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4c64012160 0x7f4c64014550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.031+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c64012160 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.031+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4c64012160 0x7f4c64014550 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.032+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c640171d0 con 0x7f4c64012160 2024-02-18T00:40:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.032+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:40:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.032+0000 7f4c767fc700 10 client.31521.objecter _finish_command 9 = system:0 2024-02-18T00:40:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.032+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c6400f990 osd=4 4 2024-02-18T00:40:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.032+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 4 9 2024-02-18T00:40:39.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 10 client.31521.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 10 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 10 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c6400f990 osd=4 3 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 4 10 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c6400f990 osd=4 4 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 10 client.31521.objecter _send_command 10 2024-02-18T00:40:39.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.052+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c640171d0 con 0x7f4c64012160 2024-02-18T00:40:39.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c640171d0 con 0x7f4c64012160 2024-02-18T00:40:39.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c767fc700 10 client.31521.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c6400f990 osd=4 4 2024-02-18T00:40:39.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 4 10 2024-02-18T00:40:39.056 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 10 client.31521.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 11 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4c64016600 0x7f4c64016a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 11 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 5 11 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640171d0 osd=5 4 2024-02-18T00:40:39.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 10 client.31521.objecter _send_command 11 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.054+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c6401a580 con 0x7f4c64016600 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.055+0000 7f4c84e23700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4c64016600 0x7f4c64016a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.055+0000 7f4c84e23700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4c64016600 0x7f4c64016a20 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.055+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c64016600 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.056+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c6401a580 con 0x7f4c64016600 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.056+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.056+0000 7f4c767fc700 10 client.31521.objecter _finish_command 11 = system:0 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.056+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640171d0 osd=5 4 2024-02-18T00:40:39.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.056+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 5 11 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 10 client.31521.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 12 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 12 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c640171d0 osd=5 3 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 5 12 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640171d0 osd=5 4 2024-02-18T00:40:39.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 10 client.31521.objecter _send_command 12 2024-02-18T00:40:39.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.070+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c64014a40 con 0x7f4c64016600 2024-02-18T00:40:39.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.072+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c64014a40 con 0x7f4c64016600 2024-02-18T00:40:39.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.072+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:39.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.072+0000 7f4c767fc700 10 client.31521.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.072+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c640171d0 osd=5 4 2024-02-18T00:40:39.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.072+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 5 12 2024-02-18T00:40:39.075 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:40:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 10 client.31521.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 13 2024-02-18T00:40:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4c640150f0 0x7f4c64015550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 13 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 6 13 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64014a40 osd=6 4 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 10 client.31521.objecter _send_command 13 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c64015bc0 con 0x7f4c640150f0 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c85e25700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4c640150f0 0x7f4c64015550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c640150f0 2024-02-18T00:40:39.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.073+0000 7f4c85e25700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4c640150f0 0x7f4c64015550 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.074+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c64015bc0 con 0x7f4c640150f0 2024-02-18T00:40:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.074+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:40:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.074+0000 7f4c767fc700 10 client.31521.objecter _finish_command 13 = system:0 2024-02-18T00:40:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.074+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c64014a40 osd=6 4 2024-02-18T00:40:39.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.074+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 6 13 2024-02-18T00:40:39.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 10 client.31521.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:39.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 14 2024-02-18T00:40:39.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 14 2024-02-18T00:40:39.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c64014a40 osd=6 3 2024-02-18T00:40:39.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 6 14 2024-02-18T00:40:39.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64014a40 osd=6 4 2024-02-18T00:40:39.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 10 client.31521.objecter _send_command 14 2024-02-18T00:40:39.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.089+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c64015bc0 con 0x7f4c640150f0 2024-02-18T00:40:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c64015bc0 con 0x7f4c640150f0 2024-02-18T00:40:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c767fc700 10 client.31521.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c64014a40 osd=6 4 2024-02-18T00:40:39.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 6 14 2024-02-18T00:40:39.093 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:40:39.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 10 client.31521.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:39.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 15 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6812/53025171 conn(0x7f4c6401b920 0x7f4c6401dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 15 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 7 15 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64015bc0 osd=7 4 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 10 client.31521.objecter _send_command 15 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c64020960 con 0x7f4c6401b920 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.091+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6812/53025171 conn(0x7f4c6401b920 0x7f4c6401dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.092+0000 7f4c767fc700 10 client.31521.objecter ms_handle_connect 0x7f4c6401b920 2024-02-18T00:40:39.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.092+0000 7f4c85624700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6812/53025171 conn(0x7f4c6401b920 0x7f4c6401dd10 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:39.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.092+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f4c64020960 con 0x7f4c6401b920 2024-02-18T00:40:39.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.092+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:40:39.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.092+0000 7f4c767fc700 10 client.31521.objecter _finish_command 15 = system:0 2024-02-18T00:40:39.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.092+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c64015bc0 osd=7 4 2024-02-18T00:40:39.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.092+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 7 15 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 10 client.31521.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign -1 16 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 20 client.31521.objecter _get_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 15 client.31521.objecter _session_command_op_remove -1 16 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 20 client.31521.objecter get_session s=0x7f4c64015bc0 osd=7 3 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 15 client.31521.objecter _session_command_op_assign 7 16 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64015bc0 osd=7 4 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 10 client.31521.objecter _send_command 16 2024-02-18T00:40:39.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.107+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f4c6401e200 con 0x7f4c6401b920 2024-02-18T00:40:39.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c767fc700 1 -- 172.21.15.59:0/2786761466 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f4c6401e200 con 0x7f4c6401b920 2024-02-18T00:40:39.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c767fc700 10 client.31521.objecter ms_dispatch 0x7f4c80001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:39.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c767fc700 10 client.31521.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c767fc700 20 client.31521.objecter put_session s=0x7f4c64015bc0 osd=7 4 2024-02-18T00:40:39.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c767fc700 15 client.31521.objecter _session_command_op_remove 7 16 2024-02-18T00:40:39.111 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:40:39.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 librados: watch_flush enter 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 librados: watch_flush exit 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.0 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4c64004560 msgr2=0x7f4c640049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6801/3700238977 conn(0x7f4c64004560 0x7f4c640049a0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640051d0 osd=0 1 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.1 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6805/26279110 conn(0x7f4c640084c0 msgr2=0x7f4c64002740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6805/26279110 conn(0x7f4c640084c0 0x7f4c64002740 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640043a0 osd=1 1 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.2 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4c64003970 msgr2=0x7f4c6400d0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6809/1153193637 conn(0x7f4c64003970 0x7f4c6400d0b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640037d0 osd=2 1 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.3 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4c6400e840 msgr2=0x7f4c6400ecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6813/1805700964 conn(0x7f4c6400e840 0x7f4c6400ecd0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64001bb0 osd=3 1 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.4 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4c64012160 msgr2=0x7f4c64014550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6800/2044387056 conn(0x7f4c64012160 0x7f4c64014550 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c6400f990 osd=4 1 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.5 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4c64016600 msgr2=0x7f4c64016a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6804/2170432263 conn(0x7f4c64016600 0x7f4c64016a20 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c640171d0 osd=5 1 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.6 2024-02-18T00:40:39.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4c640150f0 msgr2=0x7f4c64015550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6808/3356009390 conn(0x7f4c640150f0 0x7f4c64015550 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64014a40 osd=6 1 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter close_session for osd.7 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6812/53025171 conn(0x7f4c6401b920 msgr2=0x7f4c6401dd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.137:6812/53025171 conn(0x7f4c6401b920 0x7f4c6401dd10 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter put_session s=0x7f4c64015bc0 osd=7 1 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 20 client.31521.objecter shutdown clearing up homeless session... 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 10 client.31521.objecter successfully canceled tick 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c85e25700 1 -- 172.21.15.59:0/2786761466 reap_dead start 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c85e25700 1 -- 172.21.15.59:0/2786761466 reap_dead start 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c85e25700 1 -- 172.21.15.59:0/2786761466 reap_dead start 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c85e25700 1 -- 172.21.15.59:0/2786761466 reap_dead start 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c85e25700 1 -- 172.21.15.59:0/2786761466 reap_dead start 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4c6c062c90 msgr2=0x7f4c6c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.109+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4c6c062c90 0x7f4c6c065150 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 rx=0x7f4c7c004de0 tx=0x7f4c7c00b040).stop 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800fdbb0 msgr2=0x7f4c8010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800fdbb0 0x7f4c8010e280 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 rx=0x7f4c700027f0 tx=0x7f4c70002b00).stop 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 shutdown_connections 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c800fdbb0 0x7f4c8010e280 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 --2- 172.21.15.59:0/2786761466 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4c6c062c90 0x7f4c6c065150 unknown :-1 s=CLOSED pgs=3141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 >> 172.21.15.59:0/2786761466 conn(0x7f4c800f3800 msgr2=0x7f4c800f5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 shutdown_connections 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 -- 172.21.15.59:0/2786761466 wait complete. 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:39.110+0000 7f4c87888700 1 librados: shutdown 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.116 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.117 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.117 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.117 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.117 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:37.155355+0000 osd.0 (osd.0) 234 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:37.156209+0000 osd.0 (osd.0) 235 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:37.356223+0000 mgr.y (mgr.24454) 479 : audit [DBG] from='client.41321 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:37.551203+0000 osd.1 (osd.1) 263 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:37.554658+0000 osd.1 (osd.1) 264 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:37.810697+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:37.810847+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:37.830919+0000 mon.c (mon.2) 476 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:37.833832+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:37.834129+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:37.940549+0000 mgr.y (mgr.24454) 480 : cluster [DBG] pgmap v771: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T00:40:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.230344+0000 osd.4 (osd.4) 295 : cluster [DBG] 5.e scrub starts 2024-02-18T00:40:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.233834+0000 osd.4 (osd.4) 296 : cluster [DBG] 5.e scrub ok 2024-02-18T00:40:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:38.717723+0000 mon.a (mon.0) 2753 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.717806+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-18T00:40:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:38.719036+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:38.726665+0000 mon.a (mon.0) 2755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:39.411 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:39.413 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:39.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:39.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:39.550 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:39.575 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:39.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:39.676 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:39.707 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:39.728 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:39.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:39.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:39.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:39.860 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:39.880 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:39.882 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:39.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:39.894 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-18T00:40:39.983 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:40.015 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:40.191 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.508576+0000 osd.5 (osd.5) 221 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.511619+0000 osd.5 (osd.5) 222 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.547482+0000 osd.1 (osd.1) 265 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.550319+0000 osd.1 (osd.1) 266 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.673422+0000 osd.6 (osd.6) 243 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.674372+0000 osd.6 (osd.6) 244 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.745415+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:40:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:38.746417+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:38.961492+0000 mon.a (mon.0) 2756 : audit [DBG] from='client.? 172.21.15.59:0/2786761466' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.135499+0000 osd.3 (osd.3) 271 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.136877+0000 osd.3 (osd.3) 272 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.189880+0000 osd.4 (osd.4) 297 : cluster [DBG] 6.e scrub starts 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.193060+0000 osd.4 (osd.4) 298 : cluster [DBG] 6.e scrub ok 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:39.725862+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.725959+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:39.729707+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:39.730239+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.508576+0000 osd.5 (osd.5) 221 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.511619+0000 osd.5 (osd.5) 222 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:40:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.547482+0000 osd.1 (osd.1) 265 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.550319+0000 osd.1 (osd.1) 266 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.673422+0000 osd.6 (osd.6) 243 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.674372+0000 osd.6 (osd.6) 244 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.745415+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:38.746417+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:38.961492+0000 mon.a (mon.0) 2756 : audit [DBG] from='client.? 172.21.15.59:0/2786761466' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.135499+0000 osd.3 (osd.3) 271 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.136877+0000 osd.3 (osd.3) 272 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.189880+0000 osd.4 (osd.4) 297 : cluster [DBG] 6.e scrub starts 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.193060+0000 osd.4 (osd.4) 298 : cluster [DBG] 6.e scrub ok 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:39.725862+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.725959+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:39.729707+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:40.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:39.730239+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.508576+0000 osd.5 (osd.5) 221 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.511619+0000 osd.5 (osd.5) 222 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.547482+0000 osd.1 (osd.1) 265 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.550319+0000 osd.1 (osd.1) 266 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.673422+0000 osd.6 (osd.6) 243 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.674372+0000 osd.6 (osd.6) 244 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.745415+0000 osd.7 (osd.7) 127 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:38.746417+0000 osd.7 (osd.7) 128 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:38.961492+0000 mon.a (mon.0) 2756 : audit [DBG] from='client.? 172.21.15.59:0/2786761466' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:40.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.135499+0000 osd.3 (osd.3) 271 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:40:40.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.136877+0000 osd.3 (osd.3) 272 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:40:40.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.189880+0000 osd.4 (osd.4) 297 : cluster [DBG] 6.e scrub starts 2024-02-18T00:40:40.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.193060+0000 osd.4 (osd.4) 298 : cluster [DBG] 6.e scrub ok 2024-02-18T00:40:40.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:39.725862+0000 mon.a (mon.0) 2757 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:40.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.725959+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-18T00:40:40.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:39.729707+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:40.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:39.730239+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:40.464 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:40.466 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:40.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:40.568 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:40.602 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:40.890 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-02-18T00:40:40.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight-by-utilization 1000 1.0 2024-02-18T00:40:40.901 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:41.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.222907+0000 osd.0 (osd.0) 236 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:40:41.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.224201+0000 osd.0 (osd.0) 237 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:40:41.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.522829+0000 osd.5 (osd.5) 223 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.524382+0000 osd.5 (osd.5) 224 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.737635+0000 osd.7 (osd.7) 129 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.738826+0000 osd.7 (osd.7) 130 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:39.941082+0000 mgr.y (mgr.24454) 481 : cluster [DBG] pgmap v774: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.154875+0000 osd.4 (osd.4) 299 : cluster [DBG] 2.f scrub starts 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.158584+0000 osd.4 (osd.4) 300 : cluster [DBG] 2.f scrub ok 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:40.726111+0000 mon.a (mon.0) 2760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.726176+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:40.731036+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:40.731468+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:41.226 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.222907+0000 osd.0 (osd.0) 236 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.224201+0000 osd.0 (osd.0) 237 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.522829+0000 osd.5 (osd.5) 223 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.524382+0000 osd.5 (osd.5) 224 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.737635+0000 osd.7 (osd.7) 129 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.738826+0000 osd.7 (osd.7) 130 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:39.941082+0000 mgr.y (mgr.24454) 481 : cluster [DBG] pgmap v774: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.154875+0000 osd.4 (osd.4) 299 : cluster [DBG] 2.f scrub starts 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.158584+0000 osd.4 (osd.4) 300 : cluster [DBG] 2.f scrub ok 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:40.726111+0000 mon.a (mon.0) 2760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.726176+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:40.731036+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:41.227 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:40.731468+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:41.249 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:41.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.222907+0000 osd.0 (osd.0) 236 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.224201+0000 osd.0 (osd.0) 237 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.522829+0000 osd.5 (osd.5) 223 : cluster [DBG] 62.8 scrub starts 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.524382+0000 osd.5 (osd.5) 224 : cluster [DBG] 62.8 scrub ok 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.737635+0000 osd.7 (osd.7) 129 : cluster [DBG] 62.1e scrub starts 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.738826+0000 osd.7 (osd.7) 130 : cluster [DBG] 62.1e scrub ok 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:39.941082+0000 mgr.y (mgr.24454) 481 : cluster [DBG] pgmap v774: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.154875+0000 osd.4 (osd.4) 299 : cluster [DBG] 2.f scrub starts 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.158584+0000 osd.4 (osd.4) 300 : cluster [DBG] 2.f scrub ok 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:40.726111+0000 mon.a (mon.0) 2760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.726176+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-18T00:40:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:40.731036+0000 mon.c (mon.2) 480 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:41.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:40.731468+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:41.255 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:41.256 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:41.357 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:41.386 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:41.629 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:40:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.192420+0000 osd.0 (osd.0) 238 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.195877+0000 osd.0 (osd.0) 239 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.556029+0000 osd.5 (osd.5) 225 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.557540+0000 osd.5 (osd.5) 226 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.611115+0000 osd.1 (osd.1) 267 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.614064+0000 osd.1 (osd.1) 268 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.754552+0000 osd.7 (osd.7) 131 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:40:41.840 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:40.755970+0000 osd.7 (osd.7) 132 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:41.126428+0000 osd.3 (osd.3) 273 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:41.127429+0000 osd.3 (osd.3) 274 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:41.729915+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:41.729959+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.192420+0000 osd.0 (osd.0) 238 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.195877+0000 osd.0 (osd.0) 239 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.556029+0000 osd.5 (osd.5) 225 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.557540+0000 osd.5 (osd.5) 226 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:40:41.841 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.611115+0000 osd.1 (osd.1) 267 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:40:41.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.614064+0000 osd.1 (osd.1) 268 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:40:41.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.754552+0000 osd.7 (osd.7) 131 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:40:41.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:40.755970+0000 osd.7 (osd.7) 132 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:40:41.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:41.126428+0000 osd.3 (osd.3) 273 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:40:41.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:41.127429+0000 osd.3 (osd.3) 274 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:40:41.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:41.729915+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:41.842 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:41.729959+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-18T00:40:41.890 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:40:41.890 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:40:41.890 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:40:41.890 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:40:41.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.192420+0000 osd.0 (osd.0) 238 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:40:41.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.195877+0000 osd.0 (osd.0) 239 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.556029+0000 osd.5 (osd.5) 225 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.557540+0000 osd.5 (osd.5) 226 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.611115+0000 osd.1 (osd.1) 267 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.614064+0000 osd.1 (osd.1) 268 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.754552+0000 osd.7 (osd.7) 131 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:40.755970+0000 osd.7 (osd.7) 132 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:41.126428+0000 osd.3 (osd.3) 273 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:41.127429+0000 osd.3 (osd.3) 274 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:41.729915+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:41.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:41.729959+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-18T00:40:42.436 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:42.438 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:40:42.440 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:42.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:42.464 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:42] "GET /metrics HTTP/1.1" 200 190230 "" "Prometheus/2.33.4" 2024-02-18T00:40:42.542 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:42.574 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:42.612 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:42.614 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:42.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:42.625 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-18T00:40:42.716 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:42.770 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:42.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:42.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:42.875 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:42.904 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:42.946 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:41.364895+0000 mgr.y (mgr.24454) 482 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:41.581683+0000 osd.1 (osd.1) 269 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:41.581935+0000 osd.1 (osd.1) 270 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:41.941554+0000 mgr.y (mgr.24454) 483 : cluster [DBG] pgmap v777: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.121302+0000 osd.3 (osd.3) 275 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.124615+0000 osd.3 (osd.3) 276 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.143181+0000 osd.4 (osd.4) 301 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.144398+0000 osd.4 (osd.4) 302 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:40:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.723123+0000 mon.a (mon.0) 2765 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1", "id": [2, 1]}]: dispatch 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.723400+0000 mon.a (mon.0) 2766 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.4", "id": [0, 1]}]: dispatch 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.723559+0000 mon.a (mon.0) 2767 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.19", "id": [3, 5]}]: dispatch 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.723709+0000 mon.a (mon.0) 2768 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1c", "id": [3, 6]}]: dispatch 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.729407+0000 mon.a (mon.0) 2769 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1", "id": [2, 1]}]': finished 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.729497+0000 mon.a (mon.0) 2770 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.4", "id": [0, 1]}]': finished 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.729560+0000 mon.a (mon.0) 2771 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.19", "id": [3, 5]}]': finished 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:42.729623+0000 mon.a (mon.0) 2772 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1c", "id": [3, 6]}]': finished 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.729677+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-18T00:40:43.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:41.364895+0000 mgr.y (mgr.24454) 482 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:41.581683+0000 osd.1 (osd.1) 269 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:41.581935+0000 osd.1 (osd.1) 270 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:41.941554+0000 mgr.y (mgr.24454) 483 : cluster [DBG] pgmap v777: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.121302+0000 osd.3 (osd.3) 275 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.124615+0000 osd.3 (osd.3) 276 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.143181+0000 osd.4 (osd.4) 301 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.144398+0000 osd.4 (osd.4) 302 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.723123+0000 mon.a (mon.0) 2765 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1", "id": [2, 1]}]: dispatch 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.723400+0000 mon.a (mon.0) 2766 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.4", "id": [0, 1]}]: dispatch 2024-02-18T00:40:43.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.723559+0000 mon.a (mon.0) 2767 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.19", "id": [3, 5]}]: dispatch 2024-02-18T00:40:43.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.723709+0000 mon.a (mon.0) 2768 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1c", "id": [3, 6]}]: dispatch 2024-02-18T00:40:43.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.729407+0000 mon.a (mon.0) 2769 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1", "id": [2, 1]}]': finished 2024-02-18T00:40:43.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.729497+0000 mon.a (mon.0) 2770 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.4", "id": [0, 1]}]': finished 2024-02-18T00:40:43.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.729560+0000 mon.a (mon.0) 2771 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.19", "id": [3, 5]}]': finished 2024-02-18T00:40:43.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:42.729623+0000 mon.a (mon.0) 2772 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1c", "id": [3, 6]}]': finished 2024-02-18T00:40:43.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.729677+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.036+0000 7efed3e6c700 1 -- 172.21.15.59:0/4255821031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 msgr2=0x7efecc0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.036+0000 7efed3e6c700 1 --2- 172.21.15.59:0/4255821031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc0fdf90 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 rx=0x7efec8004870 tx=0x7efec8009cb0).stop 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.036+0000 7efed3e6c700 1 -- 172.21.15.59:0/4255821031 shutdown_connections 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.036+0000 7efed3e6c700 1 --2- 172.21.15.59:0/4255821031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efecc0f8420 0x7efecc104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.036+0000 7efed3e6c700 1 --2- 172.21.15.59:0/4255821031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc0fdf90 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.036+0000 7efed3e6c700 1 --2- 172.21.15.59:0/4255821031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc0fe510 0x7efecc0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.036+0000 7efed3e6c700 1 -- 172.21.15.59:0/4255821031 >> 172.21.15.59:0/4255821031 conn(0x7efecc0f3800 msgr2=0x7efecc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.037+0000 7efed3e6c700 1 -- 172.21.15.59:0/4255821031 shutdown_connections 2024-02-18T00:40:43.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.037+0000 7efed3e6c700 1 -- 172.21.15.59:0/4255821031 wait complete. 2024-02-18T00:40:43.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 librados: starting msgr at 2024-02-18T00:40:43.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 librados: starting objecter 2024-02-18T00:40:43.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 Processor -- start 2024-02-18T00:40:43.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 -- start start 2024-02-18T00:40:43.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 librados: setting wanted keys 2024-02-18T00:40:43.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 librados: calling monclient init 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc0f8420 0x7efecc10e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc10e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efecc0fe510 0x7efecc1081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1086b0 con 0x7efecc0fdbb0 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc108820 con 0x7efecc0fe510 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed3e6c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1089e0 con 0x7efecc0f8420 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed1407700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc10e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed1407700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc10e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54588/0 (socket says 172.21.15.59:54588) 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed1407700 1 -- 172.21.15.59:0/2670327934 learned_addr learned my addr 172.21.15.59:0/2670327934 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.038+0000 7efed2409700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efecc0fe510 0x7efecc1081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed1407700 1 -- 172.21.15.59:0/2670327934 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc0f8420 msgr2=0x7efecc10e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed1407700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc0f8420 0x7efecc10e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:40:43.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed1407700 1 -- 172.21.15.59:0/2670327934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efecc0fe510 msgr2=0x7efecc1081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed1407700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efecc0fe510 0x7efecc1081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed1407700 1 -- 172.21.15.59:0/2670327934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc005040 con 0x7efecc0fdbb0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed1407700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc10e510 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 rx=0x7efebc0048f0 tx=0x7efebc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efec2ffd700 10 client.?.objecter ms_handle_connect 0x7efecc0fdbb0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efec2ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efec2ffd700 1 -- 172.21.15.59:0/2670327934 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc007b00 con 0x7efecc0fdbb0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec8005040 con 0x7efecc0fdbb0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efec2ffd700 1 -- 172.21.15.59:0/2670327934 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efebc00f040 con 0x7efecc0fdbb0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efec2ffd700 1 -- 172.21.15.59:0/2670327934 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc014aa0 con 0x7efecc0fdbb0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed3e6c700 10 client.31527.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc114e50 con 0x7efecc0fdbb0 2024-02-18T00:40:43.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.039+0000 7efed3e6c700 1 librados: init done 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 1 -- 172.21.15.59:0/2670327934 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efebc00e660 con 0x7efecc0fdbb0 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 1 --2- 172.21.15.59:0/2670327934 >> v2:172.21.15.59:6800/3721891797 conn(0x7efeb8062c90 0x7efeb8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 1 -- 172.21.15.59:0/2670327934 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(509..509 src has 180..509) v4 ==== 8334+0+0 (secure 0 0 0) 0x7efebc07cfd0 con 0x7efecc0fdbb0 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 10 client.31527.objecter ms_dispatch 0x7efecc001550 osd_map(509..509 src has 180..509) v4 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 3 client.31527.objecter handle_osd_map got epochs [509,509] > 0 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 3 client.31527.objecter handle_osd_map decoding full epoch 509 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efed1c08700 1 --2- 172.21.15.59:0/2670327934 >> v2:172.21.15.59:6800/3721891797 conn(0x7efeb8062c90 0x7efeb8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 20 client.31527.objecter dump_active .. 0 homeless 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efec2ffd700 10 client.31527.objecter ms_handle_connect 0x7efeb8062c90 2024-02-18T00:40:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.041+0000 7efed1c08700 1 --2- 172.21.15.59:0/2670327934 >> v2:172.21.15.59:6800/3721891797 conn(0x7efeb8062c90 0x7efeb8065150 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 rx=0x7efec8009da0 tx=0x7efec800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:43.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.042+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc05ee80 con 0x7efecc0fdbb0 2024-02-18T00:40:43.046 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:43.049 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:43.050 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:43.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.049+0000 7efec2ffd700 1 -- 172.21.15.59:0/2670327934 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efebc049f60 con 0x7efecc0fdbb0 2024-02-18T00:40:43.076 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:43.150 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:43.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:41.364895+0000 mgr.y (mgr.24454) 482 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:43.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:41.581683+0000 osd.1 (osd.1) 269 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:43.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:41.581935+0000 osd.1 (osd.1) 270 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:43.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:41.941554+0000 mgr.y (mgr.24454) 483 : cluster [DBG] pgmap v777: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:43.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.121302+0000 osd.3 (osd.3) 275 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:40:43.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.124615+0000 osd.3 (osd.3) 276 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:40:43.179 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.143181+0000 osd.4 (osd.4) 301 : cluster [DBG] 6.1a scrub starts 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.144398+0000 osd.4 (osd.4) 302 : cluster [DBG] 6.1a scrub ok 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.723123+0000 mon.a (mon.0) 2765 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1", "id": [2, 1]}]: dispatch 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.723400+0000 mon.a (mon.0) 2766 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.4", "id": [0, 1]}]: dispatch 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.723559+0000 mon.a (mon.0) 2767 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.19", "id": [3, 5]}]: dispatch 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.723709+0000 mon.a (mon.0) 2768 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1c", "id": [3, 6]}]: dispatch 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.729407+0000 mon.a (mon.0) 2769 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1", "id": [2, 1]}]': finished 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.729497+0000 mon.a (mon.0) 2770 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.4", "id": [0, 1]}]': finished 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.729560+0000 mon.a (mon.0) 2771 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.19", "id": [3, 5]}]': finished 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:42.729623+0000 mon.a (mon.0) 2772 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "221.1c", "id": [3, 6]}]': finished 2024-02-18T00:40:43.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.729677+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-18T00:40:43.181 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:43.200 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:43.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:43.204 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:43.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.235+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7efecc1096a0 con 0x7efecc0fdbb0 2024-02-18T00:40:43.305 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:43.332 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:43.404 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:43.408 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:43.408 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:43.509 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:43.536 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:43.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.731+0000 7efec2ffd700 1 -- 172.21.15.59:0/2670327934 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v510) v1 ==== 101+0+0 (secure 0 0 0) 0x7efebc050390 con 0x7efecc0fdbb0 2024-02-18T00:40:43.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.733+0000 7efed3e6c700 10 librados: watch_flush enter 2024-02-18T00:40:43.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 10 librados: watch_flush exit 2024-02-18T00:40:43.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 20 client.31527.objecter shutdown clearing up homeless session... 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 10 client.31527.objecter successfully canceled tick 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 >> v2:172.21.15.59:6800/3721891797 conn(0x7efeb8062c90 msgr2=0x7efeb8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 --2- 172.21.15.59:0/2670327934 >> v2:172.21.15.59:6800/3721891797 conn(0x7efeb8062c90 0x7efeb8065150 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 rx=0x7efec8009da0 tx=0x7efec800b040).stop 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 msgr2=0x7efecc10e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc10e510 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 rx=0x7efebc0048f0 tx=0x7efebc009cb0).stop 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 shutdown_connections 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 --2- 172.21.15.59:0/2670327934 >> v2:172.21.15.59:6800/3721891797 conn(0x7efeb8062c90 0x7efeb8065150 unknown :-1 s=CLOSED pgs=3142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efecc0fe510 0x7efecc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc0f8420 0x7efecc10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 --2- 172.21.15.59:0/2670327934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0fdbb0 0x7efecc10e510 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 >> 172.21.15.59:0/2670327934 conn(0x7efecc0f3800 msgr2=0x7efecc0f52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 shutdown_connections 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 -- 172.21.15.59:0/2670327934 wait complete. 2024-02-18T00:40:43.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:43.734+0000 7efed3e6c700 1 librados: shutdown 2024-02-18T00:40:43.738 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:40:43.863 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:43.867 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:43.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:43.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.619642+0000 osd.1 (osd.1) 271 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:40:43.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.622529+0000 osd.1 (osd.1) 272 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:40:43.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.630340+0000 osd.6 (osd.6) 245 : cluster [DBG] 4.a scrub starts 2024-02-18T00:40:43.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.632348+0000 osd.6 (osd.6) 246 : cluster [DBG] 4.a scrub ok 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.798253+0000 osd.7 (osd.7) 133 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:42.799087+0000 osd.7 (osd.7) 134 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:43.112037+0000 osd.3 (osd.3) 277 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:43.113105+0000 osd.3 (osd.3) 278 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:43.236968+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? 172.21.15.59:0/2670327934' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:43.726290+0000 mon.a (mon.0) 2775 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:43.731857+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? 172.21.15.59:0/2670327934' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:43.731911+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:43.767341+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:43.767696+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:43.768122+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-79"}]: dispatch 2024-02-18T00:40:43.946 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:43.768386+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-79"}]: dispatch 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.619642+0000 osd.1 (osd.1) 271 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.622529+0000 osd.1 (osd.1) 272 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.630340+0000 osd.6 (osd.6) 245 : cluster [DBG] 4.a scrub starts 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.632348+0000 osd.6 (osd.6) 246 : cluster [DBG] 4.a scrub ok 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.798253+0000 osd.7 (osd.7) 133 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:42.799087+0000 osd.7 (osd.7) 134 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:43.112037+0000 osd.3 (osd.3) 277 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:43.113105+0000 osd.3 (osd.3) 278 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:43.236968+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? 172.21.15.59:0/2670327934' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:43.726290+0000 mon.a (mon.0) 2775 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:40:43.947 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:43.731857+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? 172.21.15.59:0/2670327934' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:40:43.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:43.731911+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-18T00:40:43.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:43.767341+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:43.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:43.767696+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:43.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:43.768122+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-79"}]: dispatch 2024-02-18T00:40:43.948 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:43.768386+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-79"}]: dispatch 2024-02-18T00:40:43.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:44.003 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.619642+0000 osd.1 (osd.1) 271 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.622529+0000 osd.1 (osd.1) 272 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.630340+0000 osd.6 (osd.6) 245 : cluster [DBG] 4.a scrub starts 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.632348+0000 osd.6 (osd.6) 246 : cluster [DBG] 4.a scrub ok 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.798253+0000 osd.7 (osd.7) 133 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:42.799087+0000 osd.7 (osd.7) 134 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:43.112037+0000 osd.3 (osd.3) 277 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:43.113105+0000 osd.3 (osd.3) 278 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:43.236968+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? 172.21.15.59:0/2670327934' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:43.726290+0000 mon.a (mon.0) 2775 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:40:44.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:43.731857+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? 172.21.15.59:0/2670327934' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:40:44.130 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:43.731911+0000 mon.a (mon.0) 2777 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-18T00:40:44.130 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:43.767341+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:44.130 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:43.767696+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:44.130 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:43.768122+0000 mon.c (mon.2) 482 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-79"}]: dispatch 2024-02-18T00:40:44.130 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:43.768386+0000 mon.a (mon.0) 2779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-79"}]: dispatch 2024-02-18T00:40:44.591 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:40:44.591 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:40:44.591 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:40:44.591 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:40:44.591 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:40:44.730 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:44.738 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:44.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:44.840 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:44.870 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:45.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-18T00:40:45.297 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:45.300 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:45.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:45.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:45.428 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:45.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:43.617757+0000 osd.5 (osd.5) 227 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:40:45.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:43.621220+0000 osd.5 (osd.5) 228 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:40:45.484 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:43.942041+0000 mgr.y (mgr.24454) 484 : cluster [DBG] pgmap v780: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:45.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:44.069608+0000 osd.3 (osd.3) 279 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:40:45.485 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:44.070969+0000 osd.3 (osd.3) 280 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:40:45.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:43.617757+0000 osd.5 (osd.5) 227 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:40:45.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:43.621220+0000 osd.5 (osd.5) 228 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:40:45.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:43.942041+0000 mgr.y (mgr.24454) 484 : cluster [DBG] pgmap v780: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:45.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:44.069608+0000 osd.3 (osd.3) 279 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:40:45.485 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:44.070969+0000 osd.3 (osd.3) 280 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:40:45.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:43.617757+0000 osd.5 (osd.5) 227 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:40:45.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:43.621220+0000 osd.5 (osd.5) 228 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:40:45.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:43.942041+0000 mgr.y (mgr.24454) 484 : cluster [DBG] pgmap v780: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:40:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:44.069608+0000 osd.3 (osd.3) 279 : cluster [DBG] 6.1b scrub starts 2024-02-18T00:40:45.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:44.070969+0000 osd.3 (osd.3) 280 : cluster [DBG] 6.1b scrub ok 2024-02-18T00:40:45.529 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:45.811 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:45.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:45.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:45.915 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:45.943 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:46.041 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:46.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:46.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:46.145 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:46.177 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:46.303 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:46.306 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:46.306 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:46.407 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:46.429 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:44.108473+0000 osd.4 (osd.4) 303 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:40:46.429 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:44.111380+0000 osd.4 (osd.4) 304 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:40:46.429 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:44.732767+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-18T00:40:46.429 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:45.733370+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-18T00:40:46.429 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:45.735803+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:46.429 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:45.740669+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:46.434 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:44.108473+0000 osd.4 (osd.4) 303 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:40:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:44.111380+0000 osd.4 (osd.4) 304 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:40:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:44.732767+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-18T00:40:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:45.733370+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-18T00:40:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:45.735803+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:45.740669+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:44.108473+0000 osd.4 (osd.4) 303 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:40:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:44.111380+0000 osd.4 (osd.4) 304 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:40:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:44.732767+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-18T00:40:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:45.733370+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-18T00:40:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:45.735803+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:46.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:45.740669+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:46.747 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:40:46.783 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:46.785 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:46.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:46.886 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:46.913 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 -- 172.21.15.59:0/2871981399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f80f6980 msgr2=0x7fc5f80f6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:46.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2871981399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f80f6980 0x7fc5f80f6d60 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7fc5e80048a0 tx=0x7fc5e8009cb0).stop 2024-02-18T00:40:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 -- 172.21.15.59:0/2871981399 shutdown_connections 2024-02-18T00:40:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2871981399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f81044e0 0x7fc5f81068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2871981399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f80f6980 0x7fc5f80f6d60 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2871981399 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5f80f7250 0x7fc5f8103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:46.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 -- 172.21.15.59:0/2871981399 >> 172.21.15.59:0/2871981399 conn(0x7fc5f80f2660 msgr2=0x7fc5f80f4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 -- 172.21.15.59:0/2871981399 shutdown_connections 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 -- 172.21.15.59:0/2871981399 wait complete. 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 librados: starting msgr at 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 librados: starting objecter 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.879+0000 7fc5fd791700 1 Processor -- start 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 -- start start 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 librados: setting wanted keys 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 librados: calling monclient init 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5f80f6980 0x7fc5f8110230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f80f7250 0x7fc5f8110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f81044e0 0x7fc5f810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f810a8c0 con 0x7fc5f81044e0 2024-02-18T00:40:46.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f810aa30 con 0x7fc5f80f7250 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5fd791700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f810abf0 con 0x7fc5f80f6980 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5f80f6980 0x7fc5f8110230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5f80f6980 0x7fc5f8110230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57324/0 (socket says 172.21.15.59:57324) 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 -- 172.21.15.59:0/2137728597 learned_addr learned my addr 172.21.15.59:0/2137728597 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f77fe700 1 --2- 172.21.15.59:0/2137728597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f81044e0 0x7fc5f810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f80f7250 0x7fc5f8110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 -- 172.21.15.59:0/2137728597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f80f7250 msgr2=0x7fc5f8110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f80f7250 0x7fc5f8110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 -- 172.21.15.59:0/2137728597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f81044e0 msgr2=0x7fc5f810a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5f81044e0 0x7fc5f810a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 -- 172.21.15.59:0/2137728597 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec005040 con 0x7fc5f80f6980 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f80f7250 0x7fc5f8110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5f80f6980 0x7fc5f8110230 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 rx=0x7fc5e8004710 tx=0x7fc5e8007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5dffff700 10 client.?.objecter ms_handle_connect 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5dffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.880+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800e070 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.881+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e8005040 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.881+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5e80093b0 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.881+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8018070 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.881+0000 7fc5fd791700 10 client.41336.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.881+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f810b110 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.881+0000 7fc5fd791700 1 librados: init done 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.881+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fc5f8047540 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.882+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc5e8013ad0 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.882+0000 7fc5dffff700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5e0062c90 0x7fc5e0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.882+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(513..513 src has 180..513) v4 ==== 8151+0+0 (secure 0 0 0) 0x7fc5e807dd20 con 0x7fc5f80f6980 2024-02-18T00:40:46.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.882+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 osd_map(513..513 src has 180..513) v4 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.882+0000 7fc5dffff700 3 client.41336.objecter handle_osd_map got epochs [513,513] > 0 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.882+0000 7fc5dffff700 3 client.41336.objecter handle_osd_map decoding full epoch 513 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.882+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5e0062c90 0x7fc5e0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5dffff700 20 client.41336.objecter dump_active .. 0 homeless 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v513) v1 ==== 54+0+16 (secure 0 0 0) 0x7fc5e80803f0 con 0x7fc5f80f6980 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5e0062c90 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 10 client.41336.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5e0062c90 0x7fc5e0065150 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 rx=0x7fc5f810bb70 tx=0x7fc5ec009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 1 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5f80fe4c0 0x7fc5f80fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 1 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 0 1 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8109240 osd=0 4 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 10 client.41336.objecter _send_command 1 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f810b8b0 con 0x7fc5f80fe4c0 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5f77fe700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5f80fe4c0 0x7fc5f80fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f80fe4c0 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.883+0000 7fc5f77fe700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5f80fe4c0 0x7fc5f80fe8a0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.884+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5f810b8b0 con 0x7fc5f80fe4c0 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.884+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.884+0000 7fc5dffff700 10 client.41336.objecter _finish_command 1 = system:0 2024-02-18T00:40:46.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.884+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8109240 osd=0 4 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.884+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 0 1 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 10 client.41336.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 2 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 2 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8109240 osd=0 3 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 0 2 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8109240 osd=0 4 2024-02-18T00:40:46.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 10 client.41336.objecter _send_command 2 2024-02-18T00:40:46.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.899+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f80273c0 con 0x7fc5f80fe4c0 2024-02-18T00:40:46.922 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f80273c0 con 0x7fc5f80fe4c0 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5dffff700 10 client.41336.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8109240 osd=0 4 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 0 2 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 10 client.41336.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 3 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5f80598f0 0x7fc5f8059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 3 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 1 3 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8028710 osd=1 4 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 10 client.41336.objecter _send_command 3 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f811d7e0 con 0x7fc5f80598f0 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5f80598f0 0x7fc5f8059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f80598f0 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.902+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5f80598f0 0x7fc5f8059cd0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.903+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5f811d7e0 con 0x7fc5f80598f0 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.903+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.903+0000 7fc5dffff700 10 client.41336.objecter _finish_command 3 = system:0 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.903+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8028710 osd=1 4 2024-02-18T00:40:46.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.903+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 1 3 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 10 client.41336.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 4 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 4 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8028710 osd=1 3 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 1 4 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8028710 osd=1 4 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 10 client.41336.objecter _send_command 4 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.918+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f8059360 con 0x7fc5f80598f0 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f8059360 con 0x7fc5f80598f0 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:46.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5dffff700 10 client.41336.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8028710 osd=1 4 2024-02-18T00:40:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 1 4 2024-02-18T00:40:46.926 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:40:46.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 10 client.41336.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 5 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5f811c200 0x7fc5f811c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 5 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 2 5 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8027d60 osd=2 4 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 10 client.41336.objecter _send_command 5 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f80283e0 con 0x7fc5f811c200 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.920+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5f811c200 0x7fc5f811c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.921+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f811c200 2024-02-18T00:40:46.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.921+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5f811c200 0x7fc5f811c640 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.921+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5f80283e0 con 0x7fc5f811c200 2024-02-18T00:40:46.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.921+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:40:46.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.921+0000 7fc5dffff700 10 client.41336.objecter _finish_command 5 = system:0 2024-02-18T00:40:46.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.921+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8027d60 osd=2 4 2024-02-18T00:40:46.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.921+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 2 5 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 10 client.41336.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 6 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 6 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8027d60 osd=2 3 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 2 6 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8027d60 osd=2 4 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 10 client.41336.objecter _send_command 6 2024-02-18T00:40:46.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.936+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f811d300 con 0x7fc5f811c200 2024-02-18T00:40:46.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f811d300 con 0x7fc5f811c200 2024-02-18T00:40:46.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:46.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5dffff700 10 client.41336.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:46.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8027d60 osd=2 4 2024-02-18T00:40:46.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 2 6 2024-02-18T00:40:46.940 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:40:46.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 10 client.41336.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:46.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 7 2024-02-18T00:40:46.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5f8027940 0x7fc5f8121e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 7 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 3 7 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f80277d0 osd=3 4 2024-02-18T00:40:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 10 client.41336.objecter _send_command 7 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f8124ad0 con 0x7fc5f8027940 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5f77fe700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5f8027940 0x7fc5f8121e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f8027940 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.938+0000 7fc5f77fe700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5f8027940 0x7fc5f8121e80 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.939+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5f8124ad0 con 0x7fc5f8027940 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.939+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.939+0000 7fc5dffff700 10 client.41336.objecter _finish_command 7 = system:0 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.939+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f80277d0 osd=3 4 2024-02-18T00:40:46.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.939+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 3 7 2024-02-18T00:40:46.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 10 client.41336.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 8 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 8 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f80277d0 osd=3 3 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 3 8 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f80277d0 osd=3 4 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 10 client.41336.objecter _send_command 8 2024-02-18T00:40:46.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.954+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f811cd70 con 0x7fc5f8027940 2024-02-18T00:40:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f811cd70 con 0x7fc5f8027940 2024-02-18T00:40:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5dffff700 10 client.41336.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f80277d0 osd=3 4 2024-02-18T00:40:46.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 3 8 2024-02-18T00:40:46.958 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:40:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 10 client.41336.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 9 2024-02-18T00:40:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5f811cee0 0x7fc5f81234f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 9 2024-02-18T00:40:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 4 9 2024-02-18T00:40:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811cd70 osd=4 4 2024-02-18T00:40:46.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 10 client.41336.objecter _send_command 9 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.956+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f811be90 con 0x7fc5f811cee0 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.957+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5f811cee0 0x7fc5f81234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.957+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f811cee0 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.957+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5f811cee0 0x7fc5f81234f0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.958+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5f811be90 con 0x7fc5f811cee0 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.958+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.958+0000 7fc5dffff700 10 client.41336.objecter _finish_command 9 = system:0 2024-02-18T00:40:46.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.958+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f811cd70 osd=4 4 2024-02-18T00:40:46.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.958+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 4 9 2024-02-18T00:40:46.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 10 client.41336.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:46.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 10 2024-02-18T00:40:46.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 10 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f811cd70 osd=4 3 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 4 10 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811cd70 osd=4 4 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 10 client.41336.objecter _send_command 10 2024-02-18T00:40:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.978+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f811c060 con 0x7fc5f811cee0 2024-02-18T00:40:46.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f811c060 con 0x7fc5f811cee0 2024-02-18T00:40:46.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:46.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5dffff700 10 client.41336.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:46.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f811cd70 osd=4 4 2024-02-18T00:40:46.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 4 10 2024-02-18T00:40:46.983 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:40:46.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 10 client.41336.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:46.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 11 2024-02-18T00:40:46.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5f8123ef0 0x7fc5f8124350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 11 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 5 11 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811c060 osd=5 4 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 10 client.41336.objecter _send_command 11 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.980+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f81248b0 con 0x7fc5f8123ef0 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.981+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5f8123ef0 0x7fc5f8124350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.981+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f8123ef0 2024-02-18T00:40:46.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.981+0000 7fc5f67fc700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5f8123ef0 0x7fc5f8124350 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.982+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5f81248b0 con 0x7fc5f8123ef0 2024-02-18T00:40:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.982+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:40:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.982+0000 7fc5dffff700 10 client.41336.objecter _finish_command 11 = system:0 2024-02-18T00:40:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.982+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f811c060 osd=5 4 2024-02-18T00:40:46.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.982+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 5 11 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 10 client.41336.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 12 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 12 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f811c060 osd=5 3 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 5 12 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811c060 osd=5 4 2024-02-18T00:40:46.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 10 client.41336.objecter _send_command 12 2024-02-18T00:40:46.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.996+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f8124870 con 0x7fc5f8123ef0 2024-02-18T00:40:47.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f8124870 con 0x7fc5f8123ef0 2024-02-18T00:40:47.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:47.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5dffff700 10 client.41336.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f811c060 osd=5 4 2024-02-18T00:40:47.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 5 12 2024-02-18T00:40:47.000 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:40:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 10 client.41336.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 13 2024-02-18T00:40:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5f81291d0 0x7fc5f812b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 13 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 6 13 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8124870 osd=6 4 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 10 client.41336.objecter _send_command 13 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.998+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f812e440 con 0x7fc5f81291d0 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.999+0000 7fc5f77fe700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5f81291d0 0x7fc5f812b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.999+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f81291d0 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:46.999+0000 7fc5f77fe700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5f81291d0 0x7fc5f812b600 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.000+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5f812e440 con 0x7fc5f81291d0 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.000+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.000+0000 7fc5dffff700 10 client.41336.objecter _finish_command 13 = system:0 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.000+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8124870 osd=6 4 2024-02-18T00:40:47.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.000+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 6 13 2024-02-18T00:40:47.010 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:47.014 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:47.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:47.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 10 client.41336.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 14 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 14 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f8124870 osd=6 3 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 6 14 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8124870 osd=6 4 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 10 client.41336.objecter _send_command 14 2024-02-18T00:40:47.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.014+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f812e400 con 0x7fc5f81291d0 2024-02-18T00:40:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.016+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f812e400 con 0x7fc5f81291d0 2024-02-18T00:40:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.016+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.016+0000 7fc5dffff700 10 client.41336.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.016+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f8124870 osd=6 4 2024-02-18T00:40:47.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.016+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 6 14 2024-02-18T00:40:47.018 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:40:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 10 client.41336.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 15 2024-02-18T00:40:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5f812d700 0x7fc5f812db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:47.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 15 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 7 15 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f812e400 osd=7 4 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 10 client.41336.objecter _send_command 15 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5f812e210 con 0x7fc5f812d700 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5f812d700 0x7fc5f812db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5dffff700 10 client.41336.objecter ms_handle_connect 0x7fc5f812d700 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.017+0000 7fc5f6ffd700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5f812d700 0x7fc5f812db80 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:47.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.018+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc5e8081020 con 0x7fc5f812d700 2024-02-18T00:40:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.018+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:40:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.018+0000 7fc5dffff700 10 client.41336.objecter _finish_command 15 = system:0 2024-02-18T00:40:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.018+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f812e400 osd=7 4 2024-02-18T00:40:47.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.018+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 7 15 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 10 client.41336.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign -1 16 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 20 client.41336.objecter _get_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_remove -1 16 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 20 client.41336.objecter get_session s=0x7fc5f812e400 osd=7 3 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 15 client.41336.objecter _session_command_op_assign 7 16 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f812e400 osd=7 4 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 10 client.41336.objecter _send_command 16 2024-02-18T00:40:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.032+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fc5f8130650 con 0x7fc5f812d700 2024-02-18T00:40:47.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5dffff700 1 -- 172.21.15.59:0/2137728597 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fc5f8130650 con 0x7fc5f812d700 2024-02-18T00:40:47.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5dffff700 10 client.41336.objecter ms_dispatch 0x7fc5f8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:40:47.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5dffff700 10 client.41336.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5dffff700 20 client.41336.objecter put_session s=0x7fc5f812e400 osd=7 4 2024-02-18T00:40:47.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5dffff700 15 client.41336.objecter _session_command_op_remove 7 16 2024-02-18T00:40:47.036 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:40:47.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 librados: watch_flush enter 2024-02-18T00:40:47.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 librados: watch_flush exit 2024-02-18T00:40:47.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.0 2024-02-18T00:40:47.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5f80fe4c0 msgr2=0x7fc5f80fe8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6801/3700238977 conn(0x7fc5f80fe4c0 0x7fc5f80fe8a0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8109240 osd=0 1 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.1 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5f80598f0 msgr2=0x7fc5f8059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6805/26279110 conn(0x7fc5f80598f0 0x7fc5f8059cd0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8028710 osd=1 1 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.2 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5f811c200 msgr2=0x7fc5f811c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6809/1153193637 conn(0x7fc5f811c200 0x7fc5f811c640 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8027d60 osd=2 1 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.3 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5f8027940 msgr2=0x7fc5f8121e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6813/1805700964 conn(0x7fc5f8027940 0x7fc5f8121e80 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5f77fe700 1 -- 172.21.15.59:0/2137728597 reap_dead start 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f80277d0 osd=3 1 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.4 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5f811cee0 msgr2=0x7fc5f81234f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6800/2044387056 conn(0x7fc5f811cee0 0x7fc5f81234f0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811cd70 osd=4 1 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.5 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5f77fe700 1 -- 172.21.15.59:0/2137728597 reap_dead start 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5f8123ef0 msgr2=0x7fc5f8124350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6804/2170432263 conn(0x7fc5f8123ef0 0x7fc5f8124350 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f811c060 osd=5 1 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.6 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5f81291d0 msgr2=0x7fc5f812b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6808/3356009390 conn(0x7fc5f81291d0 0x7fc5f812b600 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f8124870 osd=6 1 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter close_session for osd.7 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5f812d700 msgr2=0x7fc5f812db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.137:6812/53025171 conn(0x7fc5f812d700 0x7fc5f812db80 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter put_session s=0x7fc5f812e400 osd=7 1 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 20 client.41336.objecter shutdown clearing up homeless session... 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 10 client.41336.objecter successfully canceled tick 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5e0062c90 msgr2=0x7fc5e0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc5e0062c90 0x7fc5e0065150 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 rx=0x7fc5f810bb70 tx=0x7fc5ec009cb0).stop 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5f80f6980 msgr2=0x7fc5f8110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.034+0000 7fc5fd791700 1 --2- 172.21.15.59:0/2137728597 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc5f80f6980 0x7fc5f8110230 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 rx=0x7fc5e8004710 tx=0x7fc5e8007f90).stop 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.035+0000 7fc5f77fe700 1 -- 172.21.15.59:0/2137728597 reap_dead start 2024-02-18T00:40:47.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.035+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 shutdown_connections 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.035+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 >> 172.21.15.59:0/2137728597 conn(0x7fc5f80f2660 msgr2=0x7fc5f80f4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.035+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 shutdown_connections 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.035+0000 7fc5fd791700 1 -- 172.21.15.59:0/2137728597 wait complete. 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:47.035+0000 7fc5fd791700 1 librados: shutdown 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.041 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:40:47.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:47.148 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:47.271 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:47.274 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:47.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:47.275 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:40:47.275 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:45.655265+0000 osd.5 (osd.5) 229 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:45.658284+0000 osd.5 (osd.5) 230 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:45.682086+0000 osd.6 (osd.6) 247 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:45.683284+0000 osd.6 (osd.6) 248 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:45.942867+0000 mgr.y (mgr.24454) 485 : cluster [DBG] pgmap v783: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.008942+0000 osd.3 (osd.3) 281 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.010806+0000 osd.3 (osd.3) 282 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.130067+0000 osd.4 (osd.4) 305 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.132002+0000 osd.4 (osd.4) 306 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.381393+0000 osd.2 (osd.2) 207 : cluster [DBG] 5.a scrub starts 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.382241+0000 osd.2 (osd.2) 208 : cluster [DBG] 5.a scrub ok 2024-02-18T00:40:47.298 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:46.739057+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:47.299 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.739173+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-18T00:40:47.299 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:46.745833+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:47.299 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:46.749241+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:47.299 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:46.750169+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:47.299 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:46.883767+0000 mon.c (mon.2) 486 : audit [DBG] from='client.? 172.21.15.59:0/2137728597' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:47.376 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:47.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:45.655265+0000 osd.5 (osd.5) 229 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:40:47.402 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:45.658284+0000 osd.5 (osd.5) 230 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:45.682086+0000 osd.6 (osd.6) 247 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:45.683284+0000 osd.6 (osd.6) 248 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:45.942867+0000 mgr.y (mgr.24454) 485 : cluster [DBG] pgmap v783: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.008942+0000 osd.3 (osd.3) 281 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.010806+0000 osd.3 (osd.3) 282 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.130067+0000 osd.4 (osd.4) 305 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.132002+0000 osd.4 (osd.4) 306 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.381393+0000 osd.2 (osd.2) 207 : cluster [DBG] 5.a scrub starts 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.382241+0000 osd.2 (osd.2) 208 : cluster [DBG] 5.a scrub ok 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:46.739057+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.739173+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-18T00:40:47.403 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:46.745833+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:46.749241+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:46.750169+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:46.883767+0000 mon.c (mon.2) 486 : audit [DBG] from='client.? 172.21.15.59:0/2137728597' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:45.655265+0000 osd.5 (osd.5) 229 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:45.658284+0000 osd.5 (osd.5) 230 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:45.682086+0000 osd.6 (osd.6) 247 : cluster [DBG] 62.19 scrub starts 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:45.683284+0000 osd.6 (osd.6) 248 : cluster [DBG] 62.19 scrub ok 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:45.942867+0000 mgr.y (mgr.24454) 485 : cluster [DBG] pgmap v783: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.008942+0000 osd.3 (osd.3) 281 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:40:47.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.010806+0000 osd.3 (osd.3) 282 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.130067+0000 osd.4 (osd.4) 305 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.132002+0000 osd.4 (osd.4) 306 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.381393+0000 osd.2 (osd.2) 207 : cluster [DBG] 5.a scrub starts 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.382241+0000 osd.2 (osd.2) 208 : cluster [DBG] 5.a scrub ok 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:46.739057+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.739173+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:46.745833+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:46.749241+0000 mon.c (mon.2) 485 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:46.750169+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:47.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:46.883767+0000 mon.c (mon.2) 486 : audit [DBG] from='client.? 172.21.15.59:0/2137728597' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:47.554 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:47.556 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:47.556 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:47.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:47.686 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:47.901 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-18T00:40:48.168 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:48.235 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:48.238 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:48.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:48.339 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:48.366 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:48.387 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:48.389 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:48.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:48.491 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.259836+0000 osd.6 (osd.6) 249 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.260012+0000 osd.6 (osd.6) 250 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.697863+0000 osd.6 (osd.6) 251 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:46.699694+0000 osd.6 (osd.6) 252 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:47.407213+0000 osd.2 (osd.2) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:47.407403+0000 osd.2 (osd.2) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:47.739026+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:47.739106+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:47.740124+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:47.742331+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:48.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.259836+0000 osd.6 (osd.6) 249 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.260012+0000 osd.6 (osd.6) 250 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.697863+0000 osd.6 (osd.6) 251 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:46.699694+0000 osd.6 (osd.6) 252 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:47.407213+0000 osd.2 (osd.2) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:47.407403+0000 osd.2 (osd.2) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:47.739026+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:47.739106+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:47.740124+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:47.742331+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.259836+0000 osd.6 (osd.6) 249 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.260012+0000 osd.6 (osd.6) 250 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.697863+0000 osd.6 (osd.6) 251 : cluster [DBG] 62.1a scrub starts 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:46.699694+0000 osd.6 (osd.6) 252 : cluster [DBG] 62.1a scrub ok 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:47.407213+0000 osd.2 (osd.2) 209 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:47.407403+0000 osd.2 (osd.2) 210 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:47.739026+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:47.739106+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:47.740124+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:47.742331+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:48.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 -- 172.21.15.59:0/2853935196 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80f8420 msgr2=0x7f59f8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 --2- 172.21.15.59:0/2853935196 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80f8420 0x7f59f8104a20 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 rx=0x7f59f0009be0 tx=0x7f59f0004d10).stop 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 -- 172.21.15.59:0/2853935196 shutdown_connections 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 --2- 172.21.15.59:0/2853935196 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80f8420 0x7f59f8104a20 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 --2- 172.21.15.59:0/2853935196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80fdbb0 0x7f59f80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 --2- 172.21.15.59:0/2853935196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59f80fe510 0x7f59f80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 -- 172.21.15.59:0/2853935196 >> 172.21.15.59:0/2853935196 conn(0x7f59f80f3800 msgr2=0x7f59f80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 -- 172.21.15.59:0/2853935196 shutdown_connections 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.599+0000 7f59fdae3700 1 -- 172.21.15.59:0/2853935196 wait complete. 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 librados: starting msgr at 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 librados: starting objecter 2024-02-18T00:40:48.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 Processor -- start 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 -- start start 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 librados: setting wanted keys 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 librados: calling monclient init 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80f8420 0x7f59f810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80fdbb0 0x7f59f810e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59f80fe510 0x7f59f81080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f81085a0 con 0x7f59f80f8420 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8108710 con 0x7f59f80fe510 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.600+0000 7f59fdae3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f81088d0 con 0x7f59f80fdbb0 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80f8420 0x7f59f810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f6ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80fdbb0 0x7f59f810e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:48.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80f8420 0x7f59f810df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54620/0 (socket says 172.21.15.59:54620) 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f77fe700 1 -- 172.21.15.59:0/3691203985 learned_addr learned my addr 172.21.15.59:0/3691203985 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f7fff700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59f80fe510 0x7f59f81080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f6ffd700 1 -- 172.21.15.59:0/3691203985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59f80fe510 msgr2=0x7f59f81080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f6ffd700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59f80fe510 0x7f59f81080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f6ffd700 1 -- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80f8420 msgr2=0x7f59f810df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f6ffd700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80f8420 0x7f59f810df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f6ffd700 1 -- 172.21.15.59:0/3691203985 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e8005040 con 0x7f59f80fdbb0 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f77fe700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80f8420 0x7f59f810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f6ffd700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80fdbb0 0x7f59f810e400 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 rx=0x7f59e000e8f0 tx=0x7f59e000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f4f74700 10 client.?.objecter ms_handle_connect 0x7f59f80fdbb0 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f4f74700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f4f74700 1 -- 172.21.15.59:0/3691203985 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004620 con 0x7f59f80fdbb0 2024-02-18T00:40:48.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f4f74700 1 -- 172.21.15.59:0/3691203985 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59e000f040 con 0x7f59f80fdbb0 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f00097c0 con 0x7f59f80fdbb0 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59f4f74700 1 -- 172.21.15.59:0/3691203985 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0014830 con 0x7f59f80fdbb0 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59fdae3700 10 client.41342.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f8114d40 con 0x7f59f80fdbb0 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.601+0000 7f59fdae3700 1 librados: init done 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.602+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f805ee80 con 0x7f59f80fdbb0 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 1 -- 172.21.15.59:0/3691203985 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f59e00090d0 con 0x7f59f80fdbb0 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 1 --2- 172.21.15.59:0/3691203985 >> v2:172.21.15.59:6800/3721891797 conn(0x7f59e4062c90 0x7f59e4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 1 -- 172.21.15.59:0/3691203985 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(514..514 src has 180..514) v4 ==== 8167+0+0 (secure 0 0 0) 0x7f59e007cb00 con 0x7f59f80fdbb0 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 10 client.41342.objecter ms_dispatch 0x7f59f8001550 osd_map(514..514 src has 180..514) v4 2024-02-18T00:40:48.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 3 client.41342.objecter handle_osd_map got epochs [514,514] > 0 2024-02-18T00:40:48.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 3 client.41342.objecter handle_osd_map decoding full epoch 514 2024-02-18T00:40:48.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f77fe700 1 --2- 172.21.15.59:0/3691203985 >> v2:172.21.15.59:6800/3721891797 conn(0x7f59e4062c90 0x7f59e4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:48.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 20 client.41342.objecter dump_active .. 0 homeless 2024-02-18T00:40:48.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f4f74700 10 client.41342.objecter ms_handle_connect 0x7f59e4062c90 2024-02-18T00:40:48.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.603+0000 7f59f77fe700 1 --2- 172.21.15.59:0/3691203985 >> v2:172.21.15.59:6800/3721891797 conn(0x7f59e4062c90 0x7f59e4065150 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 rx=0x7f59e8009810 tx=0x7f59e800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:48.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.607+0000 7f59f4f74700 1 -- 172.21.15.59:0/3691203985 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f59e00500c0 con 0x7f59f80fdbb0 2024-02-18T00:40:48.655 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:48.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:48.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:48.676 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:48.759 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:48.789 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:48.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.795+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}) v1 -- 0x7f59f81096a0 con 0x7f59e4062c90 2024-02-18T00:40:48.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.796+0000 7f59f4f74700 1 -- 172.21.15.59:0/3691203985 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 no change) v1 ==== 17+0+297 (secure 0 0 0) 0x7f59f81096a0 con 0x7f59e4062c90 2024-02-18T00:40:48.798 INFO:teuthology.orchestra.run.smithi059.stdout:moved 0 / 714 (0%) 2024-02-18T00:40:48.798 INFO:teuthology.orchestra.run.smithi059.stdout:avg 89.25 2024-02-18T00:40:48.798 INFO:teuthology.orchestra.run.smithi059.stdout:stddev 6.57172 -> 6.57172 (expected baseline 8.83707) 2024-02-18T00:40:48.798 INFO:teuthology.orchestra.run.smithi059.stdout:min osd.5 with 81 -> 81 pgs (0.907563 -> 0.907563 * mean) 2024-02-18T00:40:48.798 INFO:teuthology.orchestra.run.smithi059.stdout:max osd.3 with 102 -> 102 pgs (1.14286 -> 1.14286 * mean) 2024-02-18T00:40:48.798 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:40:48.799 INFO:teuthology.orchestra.run.smithi059.stdout:oload 1000 2024-02-18T00:40:48.799 INFO:teuthology.orchestra.run.smithi059.stdout:max_change 1 2024-02-18T00:40:48.799 INFO:teuthology.orchestra.run.smithi059.stdout:max_change_osds 4 2024-02-18T00:40:48.799 INFO:teuthology.orchestra.run.smithi059.stdout:average_utilization 0.0033 2024-02-18T00:40:48.799 INFO:teuthology.orchestra.run.smithi059.stdout:overload_utilization 0.0334 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 10 librados: watch_flush enter 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 10 librados: watch_flush exit 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 20 client.41342.objecter shutdown clearing up homeless session... 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 10 client.41342.objecter successfully canceled tick 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 >> v2:172.21.15.59:6800/3721891797 conn(0x7f59e4062c90 msgr2=0x7f59e4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 1 --2- 172.21.15.59:0/3691203985 >> v2:172.21.15.59:6800/3721891797 conn(0x7f59e4062c90 0x7f59e4065150 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 rx=0x7f59e8009810 tx=0x7f59e800b040).stop 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80fdbb0 msgr2=0x7f59f810e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.798+0000 7f59fdae3700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80fdbb0 0x7f59f810e400 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 rx=0x7f59e000e8f0 tx=0x7f59e000b040).stop 2024-02-18T00:40:48.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 shutdown_connections 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 --2- 172.21.15.59:0/3691203985 >> v2:172.21.15.59:6800/3721891797 conn(0x7f59e4062c90 0x7f59e4065150 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59f80fe510 0x7f59f81080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59f80f8420 0x7f59f810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 --2- 172.21.15.59:0/3691203985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59f80fdbb0 0x7f59f810e400 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 >> 172.21.15.59:0/3691203985 conn(0x7f59f80f3800 msgr2=0x7f59f80f5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 shutdown_connections 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 -- 172.21.15.59:0/3691203985 wait complete. 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:48.799+0000 7f59fdae3700 1 librados: shutdown 2024-02-18T00:40:48.802 INFO:teuthology.orchestra.run.smithi059.stderr:no change 2024-02-18T00:40:48.886 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:48.889 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:48.889 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:48.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:49.023 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:49.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:47.678212+0000 osd.1 (osd.1) 273 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:40:49.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:47.681681+0000 osd.1 (osd.1) 274 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:40:49.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:47.943233+0000 mgr.y (mgr.24454) 486 : cluster [DBG] pgmap v786: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T00:40:49.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:48.409768+0000 osd.2 (osd.2) 211 : cluster [DBG] 2.e scrub starts 2024-02-18T00:40:49.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:48.410672+0000 osd.2 (osd.2) 212 : cluster [DBG] 2.e scrub ok 2024-02-18T00:40:49.447 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:48.722224+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:48.722306+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:48.723534+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:48.728769+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:47.678212+0000 osd.1 (osd.1) 273 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:47.681681+0000 osd.1 (osd.1) 274 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:47.943233+0000 mgr.y (mgr.24454) 486 : cluster [DBG] pgmap v786: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:48.409768+0000 osd.2 (osd.2) 211 : cluster [DBG] 2.e scrub starts 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:48.410672+0000 osd.2 (osd.2) 212 : cluster [DBG] 2.e scrub ok 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:48.722224+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:48.722306+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-18T00:40:49.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:48.723534+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:49.449 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:48.728769+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:49.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:47.678212+0000 osd.1 (osd.1) 273 : cluster [DBG] 62.1 scrub starts 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:47.681681+0000 osd.1 (osd.1) 274 : cluster [DBG] 62.1 scrub ok 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:47.943233+0000 mgr.y (mgr.24454) 486 : cluster [DBG] pgmap v786: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:48.409768+0000 osd.2 (osd.2) 211 : cluster [DBG] 2.e scrub starts 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:48.410672+0000 osd.2 (osd.2) 212 : cluster [DBG] 2.e scrub ok 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:48.722224+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_tier","val": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:48.722306+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:48.723534+0000 mon.c (mon.2) 488 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:48.728769+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:49.824 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:40:49.824 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:40:49.972 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:49.975 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:49.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:50.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:50.105 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:50.196 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:48.797147+0000 mgr.y (mgr.24454) 487 : audit [DBG] from='client.41342 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:50.214 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:40:50.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:48.797147+0000 mgr.y (mgr.24454) 487 : audit [DBG] from='client.41342 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:50.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:49.591934+0000 mon.a (mon.0) 2792 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:40:50.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:49.727853+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:50.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:49.727914+0000 mon.a (mon.0) 2794 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-18T00:40:50.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:49.731598+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:50.427 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:49.732017+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:50.427 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:50.073620+0000 mon.a (mon.0) 2796 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:40:50.427 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:50.074352+0000 mon.a (mon.0) 2797 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:40:50.427 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:50.079298+0000 mon.a (mon.0) 2798 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:50.445 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-18T00:40:50.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:49.591934+0000 mon.a (mon.0) 2792 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:49.727853+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:49.727914+0000 mon.a (mon.0) 2794 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:49.731598+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:49.732017+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:50.073620+0000 mon.a (mon.0) 2796 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:50.074352+0000 mon.a (mon.0) 2797 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:50.079298+0000 mon.a (mon.0) 2798 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:48.797147+0000 mgr.y (mgr.24454) 487 : audit [DBG] from='client.41342 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T00:40:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:49.591934+0000 mon.a (mon.0) 2792 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:40:50.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:49.727853+0000 mon.a (mon.0) 2793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:40:50.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:49.727914+0000 mon.a (mon.0) 2794 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-18T00:40:50.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:49.731598+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:50.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:49.732017+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:40:50.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:50.073620+0000 mon.a (mon.0) 2796 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:40:50.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:50.074352+0000 mon.a (mon.0) 2797 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:40:50.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:50.079298+0000 mon.a (mon.0) 2798 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:40:50.711 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:50.954 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:50.958 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:50.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:51.060 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:51.088 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:51.161 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:51.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:51.164 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:51.182 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:40:51.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:51.296 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:51.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:49.853135+0000 osd.7 (osd.7) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:49.853219+0000 osd.7 (osd.7) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:49.943589+0000 mgr.y (mgr.24454) 488 : cluster [DBG] pgmap v789: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:50.015749+0000 osd.3 (osd.3) 283 : cluster [DBG] 6.c scrub starts 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:50.016770+0000 osd.3 (osd.3) 284 : cluster [DBG] 6.c scrub ok 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:50.734519+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:50.734603+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:49.853135+0000 osd.7 (osd.7) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:51.467 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:49.853219+0000 osd.7 (osd.7) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:51.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:49.943589+0000 mgr.y (mgr.24454) 488 : cluster [DBG] pgmap v789: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:51.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:50.015749+0000 osd.3 (osd.3) 283 : cluster [DBG] 6.c scrub starts 2024-02-18T00:40:51.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:50.016770+0000 osd.3 (osd.3) 284 : cluster [DBG] 6.c scrub ok 2024-02-18T00:40:51.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:50.734519+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:51.468 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:50.734603+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-18T00:40:51.497 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:51.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:51.500 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:51.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:40:51.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:49.853135+0000 osd.7 (osd.7) 135 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:51.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:49.853219+0000 osd.7 (osd.7) 136 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:49.943589+0000 mgr.y (mgr.24454) 488 : cluster [DBG] pgmap v789: 238 pgs: 32 creating+peering, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:40:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:50.015749+0000 osd.3 (osd.3) 283 : cluster [DBG] 6.c scrub starts 2024-02-18T00:40:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:50.016770+0000 osd.3 (osd.3) 284 : cluster [DBG] 6.c scrub ok 2024-02-18T00:40:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:50.734519+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:40:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:50.734603+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-18T00:40:51.600 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:51.631 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:51.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 -- 172.21.15.59:0/1422715842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f8cb0 msgr2=0x7f0e880f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:51.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 --2- 172.21.15.59:0/1422715842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f8cb0 0x7f0e880f9090 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 rx=0x7f0e70004870 tx=0x7f0e70009cb0).stop 2024-02-18T00:40:51.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 -- 172.21.15.59:0/1422715842 shutdown_connections 2024-02-18T00:40:51.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 --2- 172.21.15.59:0/1422715842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e88102210 0x7f0e88104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:51.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 --2- 172.21.15.59:0/1422715842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f8cb0 0x7f0e880f9090 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:51.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 --2- 172.21.15.59:0/1422715842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e880f9610 0x7f0e88101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:51.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 -- 172.21.15.59:0/1422715842 >> 172.21.15.59:0/1422715842 conn(0x7f0e880f2820 msgr2=0x7f0e880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 -- 172.21.15.59:0/1422715842 shutdown_connections 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.790+0000 7f0e8d591700 1 -- 172.21.15.59:0/1422715842 wait complete. 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.791+0000 7f0e8d591700 1 librados: starting msgr at 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.791+0000 7f0e8d591700 1 librados: starting objecter 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.791+0000 7f0e8d591700 1 Processor -- start 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.791+0000 7f0e8d591700 1 -- start start 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.791+0000 7f0e8d591700 1 librados: setting wanted keys 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.791+0000 7f0e8d591700 1 librados: calling monclient init 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.791+0000 7f0e8d591700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e880f8cb0 0x7f0e880f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f9610 0x7f0e880f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e88102210 0x7f0e880f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e881166b0 con 0x7f0e88102210 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e88116820 con 0x7f0e880f8cb0 2024-02-18T00:40:51.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e881169e0 con 0x7f0e880f9610 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e867fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f9610 0x7f0e880f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e88102210 0x7f0e880f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e867fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f9610 0x7f0e880f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:50700/0 (socket says 172.21.15.59:50700) 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e867fc700 1 -- 172.21.15.59:0/76230469 learned_addr learned my addr 172.21.15.59:0/76230469 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e86ffd700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e880f8cb0 0x7f0e880f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e877fe700 1 -- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f9610 msgr2=0x7f0e880f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e877fe700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f9610 0x7f0e880f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e877fe700 1 -- 172.21.15.59:0/76230469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e880f8cb0 msgr2=0x7f0e880f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e877fe700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e880f8cb0 0x7f0e880f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e877fe700 1 -- 172.21.15.59:0/76230469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e78005040 con 0x7f0e88102210 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e867fc700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f9610 0x7f0e880f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e877fe700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e88102210 0x7f0e880f7640 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 rx=0x7f0e8000d8a0 tx=0x7f0e80004db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:51.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e7ffff700 10 client.?.objecter ms_handle_connect 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e7ffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e7ffff700 1 -- 172.21.15.59:0/76230469 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80004020 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e70005040 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e7ffff700 1 -- 172.21.15.59:0/76230469 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e8000f040 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e7ffff700 1 -- 172.21.15.59:0/76230469 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800036c0 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 10 client.31545.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e88117070 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.792+0000 7f0e8d591700 1 librados: init done 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.793+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8805fa60 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e7ffff700 1 -- 172.21.15.59:0/76230469 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0e80003af0 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e7ffff700 1 --2- 172.21.15.59:0/76230469 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e74062c90 0x7f0e74065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e7ffff700 1 -- 172.21.15.59:0/76230469 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(518..518 src has 180..518) v4 ==== 8218+0+0 (secure 0 0 0) 0x7f0e8007cc50 con 0x7f0e88102210 2024-02-18T00:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e7ffff700 10 client.31545.objecter ms_dispatch 0x7f0e88001550 osd_map(518..518 src has 180..518) v4 2024-02-18T00:40:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e7ffff700 3 client.31545.objecter handle_osd_map got epochs [518,518] > 0 2024-02-18T00:40:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e7ffff700 3 client.31545.objecter handle_osd_map decoding full epoch 518 2024-02-18T00:40:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e86ffd700 1 --2- 172.21.15.59:0/76230469 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e74062c90 0x7f0e74065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.794+0000 7f0e7ffff700 20 client.31545.objecter dump_active .. 0 homeless 2024-02-18T00:40:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.795+0000 7f0e7ffff700 10 client.31545.objecter ms_handle_connect 0x7f0e74062c90 2024-02-18T00:40:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.795+0000 7f0e86ffd700 1 --2- 172.21.15.59:0/76230469 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e74062c90 0x7f0e74065150 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 rx=0x7f0e700043d0 tx=0x7f0e7000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:51.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.799+0000 7f0e7ffff700 1 -- 172.21.15.59:0/76230469 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0e80048f60 con 0x7f0e88102210 2024-02-18T00:40:51.879 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:51.881 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:51.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:51.982 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:51.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:51.987+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f0e88107290 con 0x7f0e88102210 2024-02-18T00:40:52.010 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:52.158 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:52.161 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:52.161 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:52.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:40:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:52.408 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:40:52.408 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:40:52.431 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:50.606096+0000 osd.5 (osd.5) 231 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:40:52.431 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:50.608979+0000 osd.5 (osd.5) 232 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:40:52.431 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:50.651543+0000 osd.1 (osd.1) 275 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:40:52.431 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:50.652409+0000 osd.1 (osd.1) 276 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:40:52.431 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.006949+0000 osd.3 (osd.3) 285 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:40:52.431 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.008032+0000 osd.3 (osd.3) 286 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:40:52.431 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.154562+0000 osd.4 (osd.4) 307 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:40:52.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.157607+0000 osd.4 (osd.4) 308 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:40:52.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.298119+0000 osd.0 (osd.0) 240 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:40:52.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.299786+0000 osd.0 (osd.0) 241 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:40:52.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.348292+0000 osd.2 (osd.2) 213 : cluster [DBG] 5.b scrub starts 2024-02-18T00:40:52.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.349131+0000 osd.2 (osd.2) 214 : cluster [DBG] 5.b scrub ok 2024-02-18T00:40:52.432 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.581663+0000 osd.7 (osd.7) 137 : cluster [DBG] 2.20 deep-scrub starts 2024-02-18T00:40:52.433 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.582537+0000 osd.7 (osd.7) 138 : cluster [DBG] 2.20 deep-scrub ok 2024-02-18T00:40:52.433 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.736324+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-18T00:40:52.433 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:51.988532+0000 mon.a (mon.0) 2802 : audit [INF] from='client.? 172.21.15.59:0/76230469' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:40:52.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:40:52] "GET /metrics HTTP/1.1" 200 190230 "" "Prometheus/2.33.4" 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:50.606096+0000 osd.5 (osd.5) 231 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:50.608979+0000 osd.5 (osd.5) 232 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:50.651543+0000 osd.1 (osd.1) 275 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:50.652409+0000 osd.1 (osd.1) 276 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.006949+0000 osd.3 (osd.3) 285 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.008032+0000 osd.3 (osd.3) 286 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.154562+0000 osd.4 (osd.4) 307 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.157607+0000 osd.4 (osd.4) 308 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.298119+0000 osd.0 (osd.0) 240 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.299786+0000 osd.0 (osd.0) 241 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:40:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.348292+0000 osd.2 (osd.2) 213 : cluster [DBG] 5.b scrub starts 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.349131+0000 osd.2 (osd.2) 214 : cluster [DBG] 5.b scrub ok 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.581663+0000 osd.7 (osd.7) 137 : cluster [DBG] 2.20 deep-scrub starts 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.582537+0000 osd.7 (osd.7) 138 : cluster [DBG] 2.20 deep-scrub ok 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.736324+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:51.988532+0000 mon.a (mon.0) 2802 : audit [INF] from='client.? 172.21.15.59:0/76230469' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:50.606096+0000 osd.5 (osd.5) 231 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:50.608979+0000 osd.5 (osd.5) 232 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:50.651543+0000 osd.1 (osd.1) 275 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:50.652409+0000 osd.1 (osd.1) 276 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.006949+0000 osd.3 (osd.3) 285 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:40:52.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.008032+0000 osd.3 (osd.3) 286 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.154562+0000 osd.4 (osd.4) 307 : cluster [DBG] 6.17 scrub starts 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.157607+0000 osd.4 (osd.4) 308 : cluster [DBG] 6.17 scrub ok 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.298119+0000 osd.0 (osd.0) 240 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.299786+0000 osd.0 (osd.0) 241 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.348292+0000 osd.2 (osd.2) 213 : cluster [DBG] 5.b scrub starts 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.349131+0000 osd.2 (osd.2) 214 : cluster [DBG] 5.b scrub ok 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.581663+0000 osd.7 (osd.7) 137 : cluster [DBG] 2.20 deep-scrub starts 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.582537+0000 osd.7 (osd.7) 138 : cluster [DBG] 2.20 deep-scrub ok 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.736324+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-18T00:40:52.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:51.988532+0000 mon.a (mon.0) 2802 : audit [INF] from='client.? 172.21.15.59:0/76230469' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:40:52.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.739+0000 7f0e7ffff700 1 -- 172.21.15.59:0/76230469 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v519) v1 ==== 87+0+0 (secure 0 0 0) 0x7f0e80050080 con 0x7f0e88102210 2024-02-18T00:40:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.743+0000 7f0e8d591700 10 librados: watch_flush enter 2024-02-18T00:40:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.743+0000 7f0e8d591700 10 librados: watch_flush exit 2024-02-18T00:40:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.743+0000 7f0e8d591700 20 client.31545.objecter shutdown clearing up homeless session... 2024-02-18T00:40:52.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.743+0000 7f0e8d591700 10 client.31545.objecter successfully canceled tick 2024-02-18T00:40:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.744+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e74062c90 msgr2=0x7f0e74065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.744+0000 7f0e8d591700 1 --2- 172.21.15.59:0/76230469 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e74062c90 0x7f0e74065150 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 rx=0x7f0e700043d0 tx=0x7f0e7000b040).stop 2024-02-18T00:40:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.744+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e88102210 msgr2=0x7f0e880f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:52.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.744+0000 7f0e8d591700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e88102210 0x7f0e880f7640 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 rx=0x7f0e8000d8a0 tx=0x7f0e80004db0).stop 2024-02-18T00:40:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.745+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 shutdown_connections 2024-02-18T00:40:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.745+0000 7f0e8d591700 1 --2- 172.21.15.59:0/76230469 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0e74062c90 0x7f0e74065150 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.745+0000 7f0e8d591700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e880f8cb0 0x7f0e880f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.745+0000 7f0e8d591700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e88102210 0x7f0e880f7640 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.745+0000 7f0e8d591700 1 --2- 172.21.15.59:0/76230469 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0e880f9610 0x7f0e880f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:52.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.745+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 >> 172.21.15.59:0/76230469 conn(0x7f0e880f2820 msgr2=0x7f0e880f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.747+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 shutdown_connections 2024-02-18T00:40:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.747+0000 7f0e8d591700 1 -- 172.21.15.59:0/76230469 wait complete. 2024-02-18T00:40:52.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:52.747+0000 7f0e8d591700 1 librados: shutdown 2024-02-18T00:40:52.750 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:40:52.794 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:40:52.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:52.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:40:52.900 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:53.078 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:53.082 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:53.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:53.092 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-18T00:40:53.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:53.216 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:53.366 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:53.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:53.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:53.396 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:53.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:53.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:51.371071+0000 mgr.y (mgr.24454) 489 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:53.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.609831+0000 osd.6 (osd.6) 253 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:40:53.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.610627+0000 osd.6 (osd.6) 254 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:40:53.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.721652+0000 osd.3 (osd.3) 287 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.721840+0000 osd.3 (osd.3) 288 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:51.943965+0000 mgr.y (mgr.24454) 490 : cluster [DBG] pgmap v792: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.017721+0000 osd.3 (osd.3) 289 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.018623+0000 osd.3 (osd.3) 290 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.318826+0000 osd.2 (osd.2) 215 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.319654+0000 osd.2 (osd.2) 216 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.592596+0000 osd.7 (osd.7) 139 : cluster [DBG] 2.27 deep-scrub starts 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.593398+0000 osd.7 (osd.7) 140 : cluster [DBG] 2.27 deep-scrub ok 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.733285+0000 mon.a (mon.0) 2803 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:52.740689+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? 172.21.15.59:0/76230469' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.740749+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:52.773641+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:52.773971+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:53.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:52.774442+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-81"}]: dispatch 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:52.774686+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-81"}]: dispatch 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:51.371071+0000 mgr.y (mgr.24454) 489 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.609831+0000 osd.6 (osd.6) 253 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.610627+0000 osd.6 (osd.6) 254 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.721652+0000 osd.3 (osd.3) 287 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.721840+0000 osd.3 (osd.3) 288 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:51.943965+0000 mgr.y (mgr.24454) 490 : cluster [DBG] pgmap v792: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.017721+0000 osd.3 (osd.3) 289 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.018623+0000 osd.3 (osd.3) 290 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.318826+0000 osd.2 (osd.2) 215 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.319654+0000 osd.2 (osd.2) 216 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:40:53.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.592596+0000 osd.7 (osd.7) 139 : cluster [DBG] 2.27 deep-scrub starts 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.593398+0000 osd.7 (osd.7) 140 : cluster [DBG] 2.27 deep-scrub ok 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.733285+0000 mon.a (mon.0) 2803 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:52.740689+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? 172.21.15.59:0/76230469' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.740749+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:52.773641+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:52.773971+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:52.774442+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-81"}]: dispatch 2024-02-18T00:40:53.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:52.774686+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-81"}]: dispatch 2024-02-18T00:40:53.499 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:51.371071+0000 mgr.y (mgr.24454) 489 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.609831+0000 osd.6 (osd.6) 253 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.610627+0000 osd.6 (osd.6) 254 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.721652+0000 osd.3 (osd.3) 287 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.721840+0000 osd.3 (osd.3) 288 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:51.943965+0000 mgr.y (mgr.24454) 490 : cluster [DBG] pgmap v792: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.017721+0000 osd.3 (osd.3) 289 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.018623+0000 osd.3 (osd.3) 290 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.318826+0000 osd.2 (osd.2) 215 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:40:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.319654+0000 osd.2 (osd.2) 216 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.592596+0000 osd.7 (osd.7) 139 : cluster [DBG] 2.27 deep-scrub starts 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.593398+0000 osd.7 (osd.7) 140 : cluster [DBG] 2.27 deep-scrub ok 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.733285+0000 mon.a (mon.0) 2803 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:52.740689+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? 172.21.15.59:0/76230469' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.740749+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:52.773641+0000 mon.c (mon.2) 490 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:52.773971+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:52.774442+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-81"}]: dispatch 2024-02-18T00:40:53.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:52.774686+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-81"}]: dispatch 2024-02-18T00:40:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:53.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:53.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:53.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:53.759 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:54.107 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:54.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:54.111 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:54.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:54.241 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:54.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 -- 172.21.15.59:0/1026110080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 msgr2=0x7f1ca4103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 --2- 172.21.15.59:0/1026110080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca4103a20 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 rx=0x7f1c9c009be0 tx=0x7f1c9c004d10).stop 2024-02-18T00:40:54.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 -- 172.21.15.59:0/1026110080 shutdown_connections 2024-02-18T00:40:54.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 --2- 172.21.15.59:0/1026110080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca4103a20 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 --2- 172.21.15.59:0/1026110080 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40fcbb0 0x7f1ca40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 --2- 172.21.15.59:0/1026110080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fd510 0x7f1ca40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 -- 172.21.15.59:0/1026110080 >> 172.21.15.59:0/1026110080 conn(0x7f1ca40f2820 msgr2=0x7f1ca40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 -- 172.21.15.59:0/1026110080 shutdown_connections 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.735+0000 7f1caa13a700 1 -- 172.21.15.59:0/1026110080 wait complete. 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 librados: starting msgr at 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 librados: starting objecter 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 Processor -- start 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 -- start start 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 librados: setting wanted keys 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 librados: calling monclient init 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca410ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca410d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40fd510 0x7f1ca4106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41074d0 con 0x7f1ca40f7400 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4107640 con 0x7f1ca40fcbb0 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.736+0000 7f1caa13a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4107800 con 0x7f1ca40fd510 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca410ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca410ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56726/0 (socket says 172.21.15.59:56726) 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 -- 172.21.15.59:0/3018882108 learned_addr learned my addr 172.21.15.59:0/3018882108 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca2ffd700 1 --2- 172.21.15.59:0/3018882108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca410d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca3fff700 1 --2- 172.21.15.59:0/3018882108 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40fd510 0x7f1ca4106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 -- 172.21.15.59:0/3018882108 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40fd510 msgr2=0x7f1ca4106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40fd510 0x7f1ca4106fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 -- 172.21.15.59:0/3018882108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fcbb0 msgr2=0x7f1ca410d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca410d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 -- 172.21.15.59:0/3018882108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94005040 con 0x7f1ca40f7400 2024-02-18T00:40:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca410ce40 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7f1c940048f0 tx=0x7f1c94009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca0ff9700 10 client.?.objecter ms_handle_connect 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.737+0000 7f1ca0ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.744+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94007b00 con 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.745+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c9400f040 con 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.745+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94014990 con 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.745+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c0097c0 con 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.745+0000 7f1caa13a700 10 client.31551.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.745+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca4107d90 con 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.745+0000 7f1caa13a700 1 librados: init done 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.750+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1c94010500 con 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.751+0000 7f1ca0ff9700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c8c062ce0 0x7f1c8c0651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.751+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(520..520 src has 180..520) v4 ==== 7766+0+0 (secure 0 0 0) 0x7f1c9407cb60 con 0x7f1ca40f7400 2024-02-18T00:40:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.751+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 osd_map(520..520 src has 180..520) v4 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.751+0000 7f1ca0ff9700 3 client.31551.objecter handle_osd_map got epochs [520,520] > 0 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.751+0000 7f1ca0ff9700 3 client.31551.objecter handle_osd_map decoding full epoch 520 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.751+0000 7f1ca0ff9700 20 client.31551.objecter dump_active .. 0 homeless 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.751+0000 7f1ca2ffd700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c8c062ce0 0x7f1c8c0651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.752+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f1ca405fa60 con 0x7f1ca40f7400 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.752+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v520) v1 ==== 54+0+16 (secure 0 0 0) 0x7f1c94050190 con 0x7f1ca40f7400 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.752+0000 7f1caa13a700 10 client.31551.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.752+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 1 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.752+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1ca4061320 0x7f1ca4061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 1 2024-02-18T00:40:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 0 1 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca405ee80 osd=0 4 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 10 client.31551.objecter _send_command 1 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca41060e0 con 0x7f1ca4061320 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1c8c062ce0 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1ca2ffd700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c8c062ce0 0x7f1c8c0651a0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 rx=0x7f1ca4108780 tx=0x7f1c9800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1ca3fff700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1ca4061320 0x7f1ca4061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca4061320 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.753+0000 7f1ca3fff700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1ca4061320 0x7f1ca4061700 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.754+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca41060e0 con 0x7f1ca4061320 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.754+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.754+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 1 = system:0 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.754+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca405ee80 osd=0 4 2024-02-18T00:40:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.754+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 0 1 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 10 client.31551.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 2 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 2 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca405ee80 osd=0 3 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 0 2 2024-02-18T00:40:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca405ee80 osd=0 4 2024-02-18T00:40:54.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 10 client.31551.objecter _send_command 2 2024-02-18T00:40:54.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.769+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca40fd990 con 0x7f1ca4061320 2024-02-18T00:40:54.256 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca40fd990 con 0x7f1ca4061320 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca405ee80 osd=0 4 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 0 2 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 10 client.31551.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 3 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6805/26279110 conn(0x7f1ca40f4ba0 0x7f1ca40273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 3 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 1 3 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40f4a30 osd=1 4 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 10 client.31551.objecter _send_command 3 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca40fe7e0 con 0x7f1ca40f4ba0 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6805/26279110 conn(0x7f1ca40f4ba0 0x7f1ca40273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca40f4ba0 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.771+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6805/26279110 conn(0x7f1ca40f4ba0 0x7f1ca40273c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.772+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca40fe7e0 con 0x7f1ca40f4ba0 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.772+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.772+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 3 = system:0 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.772+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca40f4a30 osd=1 4 2024-02-18T00:40:54.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.772+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 1 3 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 10 client.31551.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 4 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 4 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca40f4a30 osd=1 3 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 1 4 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40f4a30 osd=1 4 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 10 client.31551.objecter _send_command 4 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.788+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca40fdd40 con 0x7f1ca40f4ba0 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca40fdd40 con 0x7f1ca40f4ba0 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca40f4a30 osd=1 4 2024-02-18T00:40:54.260 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:40:54.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 1 4 2024-02-18T00:40:54.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 10 client.31551.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 5 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1ca4028590 0x7f1ca41224b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 5 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 2 5 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40fe150 osd=2 4 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 10 client.31551.objecter _send_command 5 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca4125190 con 0x7f1ca4028590 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.789+0000 7f1ca2ffd700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1ca4028590 0x7f1ca41224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.790+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca4028590 2024-02-18T00:40:54.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.790+0000 7f1ca2ffd700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1ca4028590 0x7f1ca41224b0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.790+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca4125190 con 0x7f1ca4028590 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.790+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.790+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 5 = system:0 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.790+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca40fe150 osd=2 4 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.790+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 2 5 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 10 client.31551.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 6 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 6 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca40fe150 osd=2 3 2024-02-18T00:40:54.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 2 6 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40fe150 osd=2 4 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 10 client.31551.objecter _send_command 6 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.808+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca4027b50 con 0x7f1ca4028590 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.809+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca4027b50 con 0x7f1ca4028590 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca40fe150 osd=2 4 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 2 6 2024-02-18T00:40:54.263 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 10 client.31551.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 7 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1ca4123b20 0x7f1ca4123f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 7 2024-02-18T00:40:54.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 3 7 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40598f0 osd=3 4 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 10 client.31551.objecter _send_command 7 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca405a100 con 0x7f1ca4123b20 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1ca3fff700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1ca4123b20 0x7f1ca4123f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca4123b20 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.810+0000 7f1ca3fff700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1ca4123b20 0x7f1ca4123f60 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.811+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca405a100 con 0x7f1ca4123b20 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.811+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.811+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 7 = system:0 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.811+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca40598f0 osd=3 4 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.811+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 3 7 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 10 client.31551.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 8 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 8 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca40598f0 osd=3 3 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 3 8 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40598f0 osd=3 4 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 10 client.31551.objecter _send_command 8 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.825+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca4124ca0 con 0x7f1ca4123b20 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca4124ca0 con 0x7f1ca4123b20 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.267 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:40:54.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca40598f0 osd=3 4 2024-02-18T00:40:54.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 3 8 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 10 client.31551.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 9 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1ca4127360 0x7f1ca4129750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 9 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 4 9 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca41244d0 osd=4 4 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 10 client.31551.objecter _send_command 9 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca4124c80 con 0x7f1ca4127360 2024-02-18T00:40:54.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.827+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1ca4127360 0x7f1ca4129750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.828+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca4127360 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.828+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1ca4127360 0x7f1ca4129750 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.829+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca4124c80 con 0x7f1ca4127360 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.829+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.829+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 9 = system:0 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.829+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca41244d0 osd=4 4 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.829+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 4 9 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.843+0000 7f1caa13a700 10 client.31551.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.843+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 10 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.843+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.843+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.843+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.843+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 10 2024-02-18T00:40:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.843+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca41244d0 osd=4 3 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.844+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 4 10 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.844+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca41244d0 osd=4 4 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.844+0000 7f1caa13a700 10 client.31551.objecter _send_command 10 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.844+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca4124f10 con 0x7f1ca4127360 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.845+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca4124f10 con 0x7f1ca4127360 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.845+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.845+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.270 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:40:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.845+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca41244d0 osd=4 4 2024-02-18T00:40:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.845+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 4 10 2024-02-18T00:40:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.845+0000 7f1caa13a700 10 client.31551.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 11 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1ca412a150 0x7f1ca412a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 11 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 5 11 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca4124f10 osd=5 4 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 10 client.31551.objecter _send_command 11 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca412c1b0 con 0x7f1ca412a150 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1ca2ffd700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1ca412a150 0x7f1ca412a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca412a150 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.846+0000 7f1ca2ffd700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1ca412a150 0x7f1ca412a5b0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.847+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca412c1b0 con 0x7f1ca412a150 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.847+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.847+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 11 = system:0 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.847+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca4124f10 osd=5 4 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.847+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 5 11 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 10 client.31551.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 12 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 12 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca4124f10 osd=5 3 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 5 12 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca4124f10 osd=5 4 2024-02-18T00:40:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 10 client.31551.objecter _send_command 12 2024-02-18T00:40:54.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.862+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca412c1b0 con 0x7f1ca412a150 2024-02-18T00:40:54.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca412c1b0 con 0x7f1ca412a150 2024-02-18T00:40:54.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca4124f10 osd=5 4 2024-02-18T00:40:54.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 5 12 2024-02-18T00:40:54.274 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:40:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 10 client.31551.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 13 2024-02-18T00:40:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1ca412e9a0 0x7f1ca412ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 13 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 6 13 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412c1b0 osd=6 4 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 10 client.31551.objecter _send_command 13 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.864+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca412f450 con 0x7f1ca412e9a0 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.865+0000 7f1ca3fff700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1ca412e9a0 0x7f1ca412ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.865+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca412e9a0 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.865+0000 7f1ca3fff700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1ca412e9a0 0x7f1ca412ede0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.866+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca412f450 con 0x7f1ca412e9a0 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.866+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.866+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 13 = system:0 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.866+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca412c1b0 osd=6 4 2024-02-18T00:40:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.866+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 6 13 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 10 client.31551.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 14 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 14 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca412c1b0 osd=6 3 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 6 14 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412c1b0 osd=6 4 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 10 client.31551.objecter _send_command 14 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.880+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca412f450 con 0x7f1ca412e9a0 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca412f450 con 0x7f1ca412e9a0 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca412c1b0 osd=6 4 2024-02-18T00:40:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 6 14 2024-02-18T00:40:54.278 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:40:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 10 client.31551.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 15 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6812/53025171 conn(0x7f1ca4131bf0 0x7f1ca4132030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 15 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 7 15 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412f450 osd=7 4 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 10 client.31551.objecter _send_command 15 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ca41359e0 con 0x7f1ca4131bf0 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.882+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6812/53025171 conn(0x7f1ca4131bf0 0x7f1ca4132030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.883+0000 7f1ca0ff9700 10 client.31551.objecter ms_handle_connect 0x7f1ca4131bf0 2024-02-18T00:40:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.883+0000 7f1ca37fe700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6812/53025171 conn(0x7f1ca4131bf0 0x7f1ca4132030 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.883+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f1ca41359e0 con 0x7f1ca4131bf0 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.883+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.883+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 15 = system:0 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.883+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca412f450 osd=7 4 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.883+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 7 15 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 10 client.31551.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign -1 16 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 20 client.31551.objecter _get_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_remove -1 16 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 20 client.31551.objecter get_session s=0x7f1ca412f450 osd=7 3 2024-02-18T00:40:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 15 client.31551.objecter _session_command_op_assign 7 16 2024-02-18T00:40:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412f450 osd=7 4 2024-02-18T00:40:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 10 client.31551.objecter _send_command 16 2024-02-18T00:40:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.898+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f1ca412ab60 con 0x7f1ca4131bf0 2024-02-18T00:40:54.281 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.899+0000 7f1ca0ff9700 1 -- 172.21.15.59:0/3018882108 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f1ca412ab60 con 0x7f1ca4131bf0 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.899+0000 7f1ca0ff9700 10 client.31551.objecter ms_dispatch 0x7f1ca4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.899+0000 7f1ca0ff9700 10 client.31551.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.899+0000 7f1ca0ff9700 20 client.31551.objecter put_session s=0x7f1ca412f450 osd=7 4 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.899+0000 7f1ca0ff9700 15 client.31551.objecter _session_command_op_remove 7 16 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 librados: watch_flush enter 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 librados: watch_flush exit 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.0 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1ca4061320 msgr2=0x7f1ca4061700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6801/3700238977 conn(0x7f1ca4061320 0x7f1ca4061700 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca405ee80 osd=0 1 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.1 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6805/26279110 conn(0x7f1ca40f4ba0 msgr2=0x7f1ca40273c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6805/26279110 conn(0x7f1ca40f4ba0 0x7f1ca40273c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40f4a30 osd=1 1 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.2 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1ca4028590 msgr2=0x7f1ca41224b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6809/1153193637 conn(0x7f1ca4028590 0x7f1ca41224b0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40fe150 osd=2 1 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.3 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1ca4123b20 msgr2=0x7f1ca4123f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1ca3fff700 1 -- 172.21.15.59:0/3018882108 reap_dead start 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6813/1805700964 conn(0x7f1ca4123b20 0x7f1ca4123f60 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca40598f0 osd=3 1 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.4 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1ca4127360 msgr2=0x7f1ca4129750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6800/2044387056 conn(0x7f1ca4127360 0x7f1ca4129750 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca41244d0 osd=4 1 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.5 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1ca412a150 msgr2=0x7f1ca412a5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6804/2170432263 conn(0x7f1ca412a150 0x7f1ca412a5b0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca4124f10 osd=5 1 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.6 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1ca412e9a0 msgr2=0x7f1ca412ede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6808/3356009390 conn(0x7f1ca412e9a0 0x7f1ca412ede0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412c1b0 osd=6 1 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter close_session for osd.7 2024-02-18T00:40:54.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6812/53025171 conn(0x7f1ca4131bf0 msgr2=0x7f1ca4132030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.137:6812/53025171 conn(0x7f1ca4131bf0 0x7f1ca4132030 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter put_session s=0x7f1ca412f450 osd=7 1 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 20 client.31551.objecter shutdown clearing up homeless session... 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 10 client.31551.objecter successfully canceled tick 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1ca3fff700 1 -- 172.21.15.59:0/3018882108 reap_dead start 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c8c062ce0 msgr2=0x7f1c8c0651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c8c062ce0 0x7f1c8c0651a0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 rx=0x7f1ca4108780 tx=0x7f1c9800b040).stop 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 msgr2=0x7f1ca410ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca410ce40 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7f1c940048f0 tx=0x7f1c94009cb0).stop 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 shutdown_connections 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c8c062ce0 0x7f1c8c0651a0 unknown :-1 s=CLOSED pgs=3146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 --2- 172.21.15.59:0/3018882108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40f7400 0x7f1ca410ce40 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:54.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 >> 172.21.15.59:0/3018882108 conn(0x7f1ca40f2820 msgr2=0x7f1ca40f40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 shutdown_connections 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 -- 172.21.15.59:0/3018882108 wait complete. 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:53.900+0000 7f1caa13a700 1 librados: shutdown 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.286 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:40:54.492 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.641348+0000 osd.6 (osd.6) 255 : cluster [DBG] 2.7 deep-scrub starts 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:52.642070+0000 osd.6 (osd.6) 256 : cluster [DBG] 2.7 deep-scrub ok 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:53.745057+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:53.753508+0000 mon.a (mon.0) 2809 : audit [DBG] from='client.? 172.21.15.59:0/3018882108' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.641348+0000 osd.6 (osd.6) 255 : cluster [DBG] 2.7 deep-scrub starts 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:52.642070+0000 osd.6 (osd.6) 256 : cluster [DBG] 2.7 deep-scrub ok 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:53.745057+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-18T00:40:54.494 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:53.753508+0000 mon.a (mon.0) 2809 : audit [DBG] from='client.? 172.21.15.59:0/3018882108' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:54.495 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:54.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:54.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.641348+0000 osd.6 (osd.6) 255 : cluster [DBG] 2.7 deep-scrub starts 2024-02-18T00:40:54.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:52.642070+0000 osd.6 (osd.6) 256 : cluster [DBG] 2.7 deep-scrub ok 2024-02-18T00:40:54.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:53.745057+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-18T00:40:54.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:53.753508+0000 mon.a (mon.0) 2809 : audit [DBG] from='client.? 172.21.15.59:0/3018882108' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:40:54.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:54.626 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:54.967 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:40:54.967 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:40:54.967 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-02-18T00:40:54.968 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-02-18T00:40:54.968 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 1 1 2024-02-18T00:40:55.175 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:55.178 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:55.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2064 17.19 17:9c638acd:test-rados-api-smithi059-82949-16::big.copy2:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.855375+0000", 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 471.32946303599999, 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.076127541000000007, 2024-02-18T00:40:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2064 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855375+0000", 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.189 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855375+0000", 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855372+0000", 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4627000000000001e-05 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855387+0000", 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6183e-05 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.190 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855413+0000", 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2763e-05 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855425+0000", 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7112000000000001e-05 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855523+0000", 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000116888 2024-02-18T00:40:55.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.855639+0000", 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.062832378999999994 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.918472+0000", 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012550298 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.192 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931022+0000", 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000323741 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931346+0000", 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010463600000000001 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931451+0000", 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8654e-05 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.193 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931479+0000", 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2969999999999999e-05 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931502+0000", 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.076127541000000007 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2065 17.19 17:9c638acd:test-rados-api-smithi059-82949-16::big.copy2:head [read 0~25165836] snapc 0=[] ondisk+read+known_if_redirected e108)", 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:03.931708+0000", 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 471.25313007, 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018638658999999998, 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2065 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931708+0000", 2024-02-18T00:40:55.195 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931708+0000", 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931706+0000", 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3489999999999999e-06 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931712+0000", 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7899999999999994e-06 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931722+0000", 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4341e-05 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931736+0000", 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4643000000000001e-05 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931761+0000", 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8535999999999997e-05 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.931809+0000", 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018537102 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:03.950346+0000", 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018638658999999998 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.198 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2070 17.15 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.111507+0000", 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 471.07333108900002, 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096054278000000007, 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2070 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.199 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111507+0000", 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111507+0000", 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.111506+0000", 2024-02-18T00:40:55.200 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.024949676 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136455+0000", 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3879999999999998e-05 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136479+0000", 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0896e-05 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.201 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136490+0000", 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9072000000000005e-05 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136569+0000", 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4692999999999996e-05 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136664+0000", 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011364199999999999 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,7", 2024-02-18T00:40:55.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.136778+0000", 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.063359347999999996 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.200137+0000", 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0030212659999999999 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.203158+0000", 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0043070729999999998 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.207465+0000", 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1044000000000001e-05 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.207516+0000", 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4481000000000003e-05 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.207561+0000", 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096054278000000007 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.204 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2073 17.15 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head [omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.210743+0000", 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.97409445300002, 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.016597794999999999, 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2073 2024-02-18T00:40:55.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.210743+0000", 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.210743+0000", 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.206 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.210742+0000", 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0028509130000000001 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.213593+0000", 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6073e-05 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.213609+0000", 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8129999999999996e-06 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.207 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.213617+0000", 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9215999999999997e-05 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.213676+0000", 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000107519 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.213783+0000", 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0029959380000000001 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,7", 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.216779+0000", 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0078444889999999996 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.224624+0000", 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001385531 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.226009+0000", 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001257307 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.227267+0000", 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6641000000000003e-05 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.227313+0000", 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7718000000000001e-05 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.210 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.227341+0000", 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.016597794999999999 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25025.0:41 22.e 22:7639cdab:::46.obj2:head [write 0~56] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.302312+0000", 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.88252566800003, 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019327460000000001, 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.211 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25025", 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 41 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302312+0000", 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.212 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302312+0000", 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302310+0000", 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2055999999999999e-05 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302322+0000", 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5353e-05 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.213 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302337+0000", 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1449999999999996e-06 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302343+0000", 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1335000000000003e-05 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302385+0000", 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103322 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.214 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302488+0000", 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000113997 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 3,7", 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.302602+0000", 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.007515023 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.310117+0000", 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0092762130000000002 2024-02-18T00:40:55.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.319393+0000", 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021771540000000002 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321570+0000", 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9118000000000001e-05 2024-02-18T00:40:55.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321600+0000", 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9898000000000003e-05 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.321639+0000", 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019327460000000001 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24806.0:10 24.2 24:4061fbc0:test-rados-api-smithi059-82933-5::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.558717+0000", 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.62612081600003, 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.081172484000000003, 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24806", 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1001788920", 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 10 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.558717+0000", 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.558717+0000", 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.558715+0000", 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5581999999999999e-05 2024-02-18T00:40:55.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.558751+0000", 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1439e-05 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.558762+0000", 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3420000000000002e-06 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.558770+0000", 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016918400000000001 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.558939+0000", 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7200000000000005e-05 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.559026+0000", 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.080863203999999994 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.221 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.639889+0000", 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.081172484000000003 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24847.0:15 23.a 23:5074b981:test-rados-api-smithi059-83005-4::foo:head [watch unwatch cookie 94878363377232] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.576343+0000", 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.60849469499999, 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.079271681999999996, 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:40:55.222 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24847", 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1733075084", 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 15 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.576343+0000", 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.576343+0000", 2024-02-18T00:40:55.223 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.576341+0000", 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0450000000000003e-06 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.576348+0000", 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6160000000000002e-06 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.224 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.576357+0000", 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6230000000000002e-06 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.576362+0000", 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00091820599999999997 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.577281+0000", 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7866000000000002e-05 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.577369+0000", 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.078246097000000001 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655615+0000", 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.079271681999999996 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.226 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24847.0:16 23.a 23:5267809f:test-rados-api-smithi059-83005-5::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.655899+0000", 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.52893905100001, 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0213035, 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24847", 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1733075084", 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 16 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.227 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655899+0000", 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655899+0000", 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655897+0000", 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6080000000000001e-06 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.228 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655904+0000", 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0069e-05 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655914+0000", 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8960000000000002e-06 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655919+0000", 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7186999999999999e-05 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.229 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.655946+0000", 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2686e-05 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.656039+0000", 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000110258 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,3", 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.656149+0000", 2024-02-18T00:40:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029373000000000002 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.656443+0000", 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018594848000000001 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.675037+0000", 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002098585 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.677136+0000", 2024-02-18T00:40:55.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3003000000000002e-05 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.677169+0000", 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3077999999999997e-05 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.677202+0000", 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0213035 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.232 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24847.0:21 23.a 23:5267809f:test-rados-api-smithi059-83005-5::foo:head [watch unwatch cookie 94878363377232] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.685272+0000", 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.49956588600003, 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.028108968000000002, 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24847", 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1733075084", 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 21 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.233 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685272+0000", 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685272+0000", 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685271+0000", 2024-02-18T00:40:55.234 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.571e-06 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685274+0000", 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3979999999999998e-06 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685279+0000", 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8890000000000002e-06 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.235 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685282+0000", 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9593999999999999e-05 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685302+0000", 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1344999999999998e-05 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685343+0000", 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3657000000000001e-05 2024-02-18T00:40:55.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,3", 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685437+0000", 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019875599999999999 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.685635+0000", 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0064343229999999996 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.692070+0000", 2024-02-18T00:40:55.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021191726000000001 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.713261+0000", 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1273999999999997e-05 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.713313+0000", 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8021999999999993e-05 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.238 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.713381+0000", 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.028108968000000002 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24806.0:12 24.3 24:c252271c:test-rados-api-smithi059-82933-6::foo:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected e109)", 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:04.752476+0000", 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.43236158600001, 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015525035, 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24806", 2024-02-18T00:40:55.239 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1001788920", 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 12 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752476+0000", 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752476+0000", 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:40:55.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752475+0000", 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.006e-06 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752480+0000", 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7220000000000005e-06 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752489+0000", 2024-02-18T00:40:55.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2440000000000004e-06 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752493+0000", 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9545000000000002e-05 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752533+0000", 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8053999999999995e-05 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752611+0000", 2024-02-18T00:40:55.242 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012816100000000001 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752739+0000", 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00024318899999999999 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.752982+0000", 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0030490909999999999 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.243 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.756031+0000", 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.011882734000000001 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.767914+0000", 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8000000000000001e-05 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.767962+0000", 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9026999999999998e-05 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:04.768001+0000", 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015525035 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25000.0:2 34.6 34:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e110)", 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:05.564143+0000", 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 469.620694152, 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.038385066000000002, 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.245 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25000", 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2984699099", 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.564143+0000", 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.246 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.564143+0000", 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.564142+0000", 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8789999999999997e-06 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.564149+0000", 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0076e-05 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.247 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.564159+0000", 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4460000000000004e-06 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.564166+0000", 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016384959999999999 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.565805+0000", 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4360000000000001e-06 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.248 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.565810+0000", 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.03036057 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.596171+0000", 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000123819 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.596295+0000", 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016105000000000001 2024-02-18T00:40:55.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.596456+0000", 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020010370000000002 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.598457+0000", 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001647918 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.600105+0000", 2024-02-18T00:40:55.250 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023314939999999999 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.602436+0000", 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1400999999999998e-05 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.602488+0000", 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1022999999999997e-05 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:05.602529+0000", 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.038385066000000002 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.17835.0:2 70.16 70:68f45781:::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e129)", 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:27.025555+0000", 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 448.159283104, 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.044774520999999998, 2024-02-18T00:40:55.252 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.17835", 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3676653592", 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025555+0000", 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.253 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025555+0000", 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025553+0000", 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2979999999999997e-06 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025559+0000", 2024-02-18T00:40:55.254 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2739999999999996e-06 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025569+0000", 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0579999999999997e-06 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025574+0000", 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3038000000000002e-05 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025637+0000", 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5499999999999997e-06 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.025644+0000", 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020588898000000001 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.046233+0000", 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012292800000000001 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.257 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.046356+0000", 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016614200000000001 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,4", 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.046522+0000", 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00059029999999999998 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.260 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.047113+0000", 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00066992500000000003 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.047783+0000", 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.022447313 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.070230+0000", 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3319000000000002e-05 2024-02-18T00:40:55.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.070283+0000", 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5868999999999997e-05 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:27.070329+0000", 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.044774520999999998 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.24823.0:2078 17.19 17:9dd58b98:test-rados-api-smithi059-82949-17::big.copy:head [copy-from ver 0] snapc 0=[] ondisk+write+known_if_redirected e136)", 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.386788+0000", 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 440.79804968500002, 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.120928518, 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.24823", 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2078 2024-02-18T00:40:55.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386788+0000", 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386788+0000", 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386787+0000", 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5050000000000001e-06 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386794+0000", 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2551999999999999e-05 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386807+0000", 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2330000000000003e-06 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386813+0000", 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0116e-05 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386853+0000", 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6520000000000005e-05 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.386939+0000", 2024-02-18T00:40:55.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.066957558 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.453897+0000", 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0078882450000000003 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.461785+0000", 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00035055200000000001 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.462136+0000", 2024-02-18T00:40:55.267 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.045483510999999997 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.507619+0000", 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4848000000000001e-05 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.507674+0000", 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2314999999999997e-05 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.507716+0000", 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.120928518 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.7.0:212 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.522360+0000", 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 440.66247812500001, 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02223292, 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.269 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.522360+0000", 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.522360+0000", 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.522358+0000", 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002601389 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.270 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.524959+0000", 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0676000000000001e-05 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.524970+0000", 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0030000000000001e-05 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.524980+0000", 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3642e-05 2024-02-18T00:40:55.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.525024+0000", 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3981e-05 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.525048+0000", 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019503639 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.544551+0000", 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0985000000000001e-05 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.544592+0000", 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02223292 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.7.0:213 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.547285+0000", 2024-02-18T00:40:55.273 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 440.63755239099999, 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.022264842, 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.547285+0000", 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.547285+0000", 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.547284+0000", 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022978479999999999 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.549582+0000", 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7156000000000001e-05 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.275 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.549619+0000", 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3731999999999999e-05 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.549642+0000", 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3849000000000005e-05 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.549706+0000", 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6724000000000003e-05 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.276 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.549743+0000", 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019743277 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.569486+0000", 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3758999999999997e-05 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.569550+0000", 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.022264842 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.277 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.7.0:214 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.568038+0000", 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 440.61679990099998, 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017004907999999999, 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.568038+0000", 2024-02-18T00:40:55.278 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.568038+0000", 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.568036+0000", 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0043341220000000001 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.572370+0000", 2024-02-18T00:40:55.279 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6135999999999999e-05 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.572386+0000", 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0013000000000001e-05 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.572396+0000", 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0827000000000001e-05 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.572437+0000", 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8788999999999999e-05 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.572456+0000", 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012539191999999999 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.584995+0000", 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7478999999999997e-05 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.585043+0000", 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017004907999999999 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40478.0:1 135.6s0 135:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e201)", 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:34:39.637698+0000", 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 375.54713997499999, 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0054391140000001, 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.282 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40478", 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2213758534", 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637698+0000", 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.283 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637698+0000", 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637696+0000", 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1990000000000006e-06 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637704+0000", 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6707e-05 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637730+0000", 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3950000000000007e-06 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637740+0000", 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4720000000000004e-05 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637805+0000", 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8290000000000001e-06 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:39.637808+0000", 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.99703395399999994 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:40.634842+0000", 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016213100000000001 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:40.635004+0000", 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00032247999999999999 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:40.635327+0000", 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00071126900000000005 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:40.636038+0000", 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0070263260000000003 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:40.643065+0000", 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2247000000000001e-05 2024-02-18T00:40:55.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:40.643137+0000", 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0054391140000001 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:499 62.b e205/149 62:d7ac73f7:test-rados-api-smithi059-82957-14::foo9:4 v 205'39, mlcod=202'35)", 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:34:43.645440+0000", 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 371.53939726800002, 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025487418000000001, 2024-02-18T00:40:55.288 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645440+0000", 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645440+0000", 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999962 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645436+0000", 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8258999999999999e-05 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645455+0000", 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8575000000000001e-05 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645473+0000", 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.755e-06 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645482+0000", 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6451000000000005e-05 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645548+0000", 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9217000000000002e-05 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.645578+0000", 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025297969 2024-02-18T00:40:55.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670876+0000", 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2119e-05 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670928+0000", 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025487418000000001 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25220.0:2195 166.d 166:b6d48a5a:test-rados-api-smithi059-82957-28::foo:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e258)", 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:36.628246+0000", 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 318.55659159099997, 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013407024, 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25220", 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2195 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628246+0000", 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628246+0000", 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628244+0000", 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0868e-05 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628255+0000", 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2198e-05 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628277+0000", 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7149999999999993e-06 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628287+0000", 2024-02-18T00:40:55.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9270000000000006e-05 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628356+0000", 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026479299999999998 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for blocked object", 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.628621+0000", 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010769641 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.639390+0000", 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9772000000000006e-05 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for rw locks", 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.639490+0000", 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001139426 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.640629+0000", 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9162e-05 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.640689+0000", 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00096442999999999995 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.641653+0000", 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013407024 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.14403.0:1099 4.4 4:22d26bf9:::data_loggenerations_metadata:head [watch reconnect cookie 94750322754560 gen 1] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:43.350647+0000", 2024-02-18T00:40:55.298 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 311.83419093200001, 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.098772475999999998, 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.14403", 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2016795673", 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1099 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350647+0000", 2024-02-18T00:40:55.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350647+0000", 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350646+0000", 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7480000000000001e-06 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:40:55.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350650+0000", 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.012e-05 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350661+0000", 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8450000000000002e-06 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350665+0000", 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4172999999999999e-05 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.301 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350710+0000", 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5444000000000004e-05 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for missing object", 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.350795+0000", 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010434629999999999 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.351838+0000", 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0715000000000003e-05 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for rw locks", 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.351909+0000", 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096469013000000006 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.448378+0000", 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4978000000000001e-05 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.448433+0000", 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00098599100000000008 2024-02-18T00:40:55.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.449419+0000", 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.098772475999999998 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:40:55.304 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:40:55.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:40:55.305 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:53.944341+0000 mgr.y (mgr.24454) 491 : cluster [DBG] pgmap v795: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:54.747125+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:54.755485+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:54.756099+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:53.944341+0000 mgr.y (mgr.24454) 491 : cluster [DBG] pgmap v795: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:54.747125+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:54.755485+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:55.306 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:54.756099+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:55.333 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:53.944341+0000 mgr.y (mgr.24454) 491 : cluster [DBG] pgmap v795: 206 pgs: 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:40:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:54.747125+0000 mon.a (mon.0) 2810 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-18T00:40:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:54.755485+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:54.756099+0000 mon.a (mon.0) 2811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:40:55.532 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:40:55.535 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:55.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:40:55.636 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:40:55.667 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:55.846 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-18T00:40:55.866 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:40:55.869 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:55.869 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:40:55.970 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:40:55.997 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:56.111 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:40:56.115 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:56.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:40:56.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:40:56.128 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:56.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:40:56.242 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:56.391 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:40:56.394 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:56.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:40:56.495 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:56.522 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:56.575 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:56.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:56.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:56.679 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:56.904 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:56.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:55.748887+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:56.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:55.749016+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-18T00:40:56.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:55.764240+0000 mon.c (mon.2) 493 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:56.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:55.945172+0000 mgr.y (mgr.24454) 492 : cluster [DBG] pgmap v798: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-18T00:40:56.907 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:56.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:57.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:55.748887+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:55.749016+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-18T00:40:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:55.764240+0000 mon.c (mon.2) 493 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:55.945172+0000 mgr.y (mgr.24454) 492 : cluster [DBG] pgmap v798: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-18T00:40:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:55.748887+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:40:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:55.749016+0000 mon.a (mon.0) 2813 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-18T00:40:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:55.764240+0000 mon.c (mon.2) 493 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:40:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:55.945172+0000 mgr.y (mgr.24454) 492 : cluster [DBG] pgmap v798: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-02-18T00:40:57.008 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:57.034 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:57.106 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:57.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:57.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:57.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:40:57.212 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:40:57.238 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:57.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 -- 172.21.15.59:0/3395088443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 msgr2=0x7f62680f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:57.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 --2- 172.21.15.59:0/3395088443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 0x7f62680f9090 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 rx=0x7f62640047b0 tx=0x7f6264009cb0).stop 2024-02-18T00:40:57.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 -- 172.21.15.59:0/3395088443 shutdown_connections 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 --2- 172.21.15.59:0/3395088443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6268102210 0x7f6268104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 --2- 172.21.15.59:0/3395088443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 0x7f62680f9090 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 --2- 172.21.15.59:0/3395088443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62680f9610 0x7f6268101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 -- 172.21.15.59:0/3395088443 >> 172.21.15.59:0/3395088443 conn(0x7f62680f2820 msgr2=0x7f62680f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 -- 172.21.15.59:0/3395088443 shutdown_connections 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 -- 172.21.15.59:0/3395088443 wait complete. 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 librados: starting msgr at 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 librados: starting objecter 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.365+0000 7f626f5bc700 1 Processor -- start 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 -- start start 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 librados: setting wanted keys 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 librados: calling monclient init 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 0x7f62680f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:57.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62680f9610 0x7f62680f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6268102210 0x7f62680f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62681166e0 con 0x7f62680f8cb0 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6268116850 con 0x7f62680f9610 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626f5bc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6268116a10 con 0x7f6268102210 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6268102210 0x7f62680f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6268102210 0x7f62680f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:50734/0 (socket says 172.21.15.59:50734) 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 -- 172.21.15.59:0/1447486458 learned_addr learned my addr 172.21.15.59:0/1447486458 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626d358700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 0x7f62680f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 -- 172.21.15.59:0/1447486458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62680f9610 msgr2=0x7f62680f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62680f9610 0x7f62680f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 -- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 msgr2=0x7f62680f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 0x7f62680f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:57.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 -- 172.21.15.59:0/1447486458 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6258005040 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f626db59700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6268102210 0x7f62680f7640 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 rx=0x7f626000d940 tx=0x7f6260004e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f625e7fc700 10 client.?.objecter ms_handle_connect 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f625e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.366+0000 7f625e7fc700 1 -- 172.21.15.59:0/1447486458 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6260004020 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.367+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6264005040 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.367+0000 7f625e7fc700 1 -- 172.21.15.59:0/1447486458 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626000f040 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.367+0000 7f625e7fc700 1 -- 172.21.15.59:0/1447486458 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600036c0 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.367+0000 7f626f5bc700 10 client.41360.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.367+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6268117000 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.367+0000 7f626f5bc700 1 librados: init done 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.368+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626805fa60 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.368+0000 7f625e7fc700 1 -- 172.21.15.59:0/1447486458 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6260003af0 con 0x7f6268102210 2024-02-18T00:40:57.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f625e7fc700 1 --2- 172.21.15.59:0/1447486458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6254062c90 0x7f6254065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f625e7fc700 1 -- 172.21.15.59:0/1447486458 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(523..523 src has 180..523) v4 ==== 8534+0+0 (secure 0 0 0) 0x7f626007ceb0 con 0x7f6268102210 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f625e7fc700 10 client.41360.objecter ms_dispatch 0x7f6268001550 osd_map(523..523 src has 180..523) v4 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f625e7fc700 3 client.41360.objecter handle_osd_map got epochs [523,523] > 0 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f625e7fc700 3 client.41360.objecter handle_osd_map decoding full epoch 523 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f626d358700 1 --2- 172.21.15.59:0/1447486458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6254062c90 0x7f6254065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f625e7fc700 20 client.41360.objecter dump_active .. 0 homeless 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f625e7fc700 10 client.41360.objecter ms_handle_connect 0x7f6254062c90 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.369+0000 7f626d358700 1 --2- 172.21.15.59:0/1447486458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6254062c90 0x7f6254065150 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 rx=0x7f6264005010 tx=0x7f6264009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:40:57.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.373+0000 7f625e7fc700 1 -- 172.21.15.59:0/1447486458 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f62600501a0 con 0x7f6268102210 2024-02-18T00:40:57.451 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:57.483 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:40:57.486 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:40:57.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:57.489 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:40:57.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.585+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f62680f8140 con 0x7f6268102210 2024-02-18T00:40:57.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:40:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:57.715 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:40:57.717 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:57.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:40:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.756+0000 7f625e7fc700 1 -- 172.21.15.59:0/1447486458 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v524) v1 ==== 133+0+0 (secure 0 0 0) 0x7f626004fd90 con 0x7f6268102210 2024-02-18T00:40:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.766+0000 7f626f5bc700 10 librados: watch_flush enter 2024-02-18T00:40:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.767+0000 7f626f5bc700 10 librados: watch_flush exit 2024-02-18T00:40:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.767+0000 7f626f5bc700 20 client.41360.objecter shutdown clearing up homeless session... 2024-02-18T00:40:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.767+0000 7f626f5bc700 10 client.41360.objecter successfully canceled tick 2024-02-18T00:40:57.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.771+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6254062c90 msgr2=0x7f6254065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:57.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.771+0000 7f626f5bc700 1 --2- 172.21.15.59:0/1447486458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6254062c90 0x7f6254065150 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 rx=0x7f6264005010 tx=0x7f6264009cb0).stop 2024-02-18T00:40:57.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.771+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6268102210 msgr2=0x7f62680f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:40:57.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.771+0000 7f626f5bc700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6268102210 0x7f62680f7640 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 rx=0x7f626000d940 tx=0x7f6260004e40).stop 2024-02-18T00:40:57.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 shutdown_connections 2024-02-18T00:40:57.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6268102210 0x7f62680f7640 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62680f8cb0 0x7f62680f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:40:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 --2- 172.21.15.59:0/1447486458 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6254062c90 0x7f6254065150 secure :-1 s=CLOSED pgs=3147 cs=0 l=1 rev1=1 rx=0x7f6264005010 tx=0x7f6264009cb0).stop 2024-02-18T00:40:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 --2- 172.21.15.59:0/1447486458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62680f9610 0x7f62680f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:40:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 >> 172.21.15.59:0/1447486458 conn(0x7f62680f2820 msgr2=0x7f62680f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:40:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 shutdown_connections 2024-02-18T00:40:57.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.773+0000 7f626f5bc700 1 -- 172.21.15.59:0/1447486458 wait complete. 2024-02-18T00:40:57.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:40:57.776+0000 7f626f5bc700 1 librados: shutdown 2024-02-18T00:40:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-02-18T00:40:57.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:57.851 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:57.983 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:40:57.983 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:40:57.983 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:40:57.983 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:56.757055+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:56.760819+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:56.761255+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:57.586594+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.59:0/1447486458' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:57.586940+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:56.757055+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:56.760819+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:58.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:56.761255+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:58.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:57.586594+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.59:0/1447486458' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:40:58.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:57.586940+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:40:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:56.757055+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-18T00:40:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:56.760819+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:56.761255+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:40:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:57.586594+0000 mon.c (mon.2) 495 : audit [INF] from='client.? 172.21.15.59:0/1447486458' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:40:58.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:57.586940+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:40:58.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-18T00:40:58.900 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:40:59.000 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:57.755714+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:57.755882+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:57.755949+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:57.764752+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]: dispatch 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:57.777634+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]: dispatch 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:57.945652+0000 mgr.y (mgr.24454) 493 : cluster [DBG] pgmap v801: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:58.723219+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]': finished 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:58.723307+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:58.724862+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:58.725289+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:57.755714+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:57.755882+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:57.755949+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:57.764752+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]: dispatch 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:57.777634+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]: dispatch 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:57.945652+0000 mgr.y (mgr.24454) 493 : cluster [DBG] pgmap v801: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:58.723219+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]': finished 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:58.723307+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:58.724862+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:40:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:58.725289+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:59.003 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:59.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:57.755714+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:40:59.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:57.755882+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:57.755949+0000 mon.a (mon.0) 2819 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:57.764752+0000 mon.c (mon.2) 496 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]: dispatch 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:57.777634+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]: dispatch 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:57.945652+0000 mgr.y (mgr.24454) 493 : cluster [DBG] pgmap v801: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:58.723219+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_tier","val": "test-rados-api-smithi059-82957-85-test-flush"}]': finished 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:58.723307+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:58.724862+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:59.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:40:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:58.725289+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:40:59.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:59.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:59.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:59.138 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:40:59.767 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:40:59.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:40:59.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:40:59.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:40:59.904 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:00.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 -- 172.21.15.59:0/3955721445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b40f9610 msgr2=0x7f80b4101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 --2- 172.21.15.59:0/3955721445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b40f9610 0x7f80b4101d20 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 rx=0x7f80a4004870 tx=0x7f80a4009cb0).stop 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 -- 172.21.15.59:0/3955721445 shutdown_connections 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 --2- 172.21.15.59:0/3955721445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80b4102210 0x7f80b4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 --2- 172.21.15.59:0/3955721445 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f8cb0 0x7f80b40f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 --2- 172.21.15.59:0/3955721445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b40f9610 0x7f80b4101d20 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 -- 172.21.15.59:0/3955721445 >> 172.21.15.59:0/3955721445 conn(0x7f80b40f2820 msgr2=0x7f80b40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 -- 172.21.15.59:0/3955721445 shutdown_connections 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.499+0000 7f80bc215700 1 -- 172.21.15.59:0/3955721445 wait complete. 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 librados: starting msgr at 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 librados: starting objecter 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 Processor -- start 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 -- start start 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 librados: setting wanted keys 2024-02-18T00:41:00.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 librados: calling monclient init 2024-02-18T00:41:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80b40f8cb0 0x7f80b410e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f9610 0x7f80b410e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4102210 0x7f80b4108430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4108920 con 0x7f80b4102210 2024-02-18T00:41:00.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4108a90 con 0x7f80b40f8cb0 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80bc215700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4108c50 con 0x7f80b40f9610 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80b97b0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f9610 0x7f80b410e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.500+0000 7f80b97b0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f9610 0x7f80b410e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38808/0 (socket says 172.21.15.59:38808) 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b97b0700 1 -- 172.21.15.59:0/500826547 learned_addr learned my addr 172.21.15.59:0/500826547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80b40f8cb0 0x7f80b410e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80ba7b2700 1 --2- 172.21.15.59:0/500826547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4102210 0x7f80b4108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b97b0700 1 -- 172.21.15.59:0/500826547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80b40f8cb0 msgr2=0x7f80b410e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b97b0700 1 --2- 172.21.15.59:0/500826547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80b40f8cb0 0x7f80b410e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b97b0700 1 -- 172.21.15.59:0/500826547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4102210 msgr2=0x7f80b4108430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b97b0700 1 --2- 172.21.15.59:0/500826547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4102210 0x7f80b4108430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b97b0700 1 -- 172.21.15.59:0/500826547 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4005040 con 0x7f80b40f9610 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80b97b0700 1 --2- 172.21.15.59:0/500826547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f9610 0x7f80b410e780 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 rx=0x7f80a40043a0 tx=0x7f80a400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80aaffd700 10 client.?.objecter ms_handle_connect 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80aaffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400e070 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80a4009710 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b4108ed0 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4019070 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80bc215700 10 client.41366.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b41150c0 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.501+0000 7f80bc215700 1 librados: init done 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.502+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f80b405fa60 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f80a40090d0 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80a0062c90 0x7f80a0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(526..526 src has 180..526) v4 ==== 8617+0+0 (secure 0 0 0) 0x7f80a407e290 con 0x7f80b40f9610 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 osd_map(526..526 src has 180..526) v4 2024-02-18T00:41:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 3 client.41366.objecter handle_osd_map got epochs [526,526] > 0 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 3 client.41366.objecter handle_osd_map decoding full epoch 526 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80a0062c90 0x7f80a0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 20 client.41366.objecter dump_active .. 0 homeless 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v526) v1 ==== 54+0+16 (secure 0 0 0) 0x7f80a407e770 con 0x7f80b40f9610 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80a0062c90 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80a0062c90 0x7f80a0065150 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 rx=0x7f80b0004790 tx=0x7f80b000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 10 client.41366.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 1 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6801/3700238977 conn(0x7f80b40fc150 0x7f80b40fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 1 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 0 1 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b405ee80 osd=0 4 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 10 client.41366.objecter _send_command 1 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.503+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b4107290 con 0x7f80b40fc150 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80ba7b2700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6801/3700238977 conn(0x7f80b40fc150 0x7f80b40fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80ba7b2700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6801/3700238977 conn(0x7f80b40fc150 0x7f80b40fc530 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b40fc150 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b4107290 con 0x7f80b40fc150 2024-02-18T00:41:00.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:41:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80aaffd700 10 client.41366.objecter _finish_command 1 = system:0 2024-02-18T00:41:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b405ee80 osd=0 4 2024-02-18T00:41:00.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.504+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 0 1 2024-02-18T00:41:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 10 client.41366.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 2 2024-02-18T00:41:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 2 2024-02-18T00:41:00.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b405ee80 osd=0 3 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 0 2 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b405ee80 osd=0 4 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 10 client.41366.objecter _send_command 2 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.519+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b40273c0 con 0x7f80b40fc150 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b40273c0 con 0x7f80b40fc150 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80aaffd700 10 client.41366.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b405ee80 osd=0 4 2024-02-18T00:41:00.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 0 2 2024-02-18T00:41:00.523 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:41:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 10 client.41366.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 3 2024-02-18T00:41:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6805/26279110 conn(0x7f80b40598f0 0x7f80b4059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 3 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 1 3 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4028710 osd=1 4 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 10 client.41366.objecter _send_command 3 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b40f8ab0 con 0x7f80b40598f0 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6805/26279110 conn(0x7f80b40598f0 0x7f80b4059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b40598f0 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.521+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6805/26279110 conn(0x7f80b40598f0 0x7f80b4059d10 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.522+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b40f8ab0 con 0x7f80b40598f0 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.522+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.522+0000 7f80aaffd700 10 client.41366.objecter _finish_command 3 = system:0 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.522+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b4028710 osd=1 4 2024-02-18T00:41:00.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.522+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 1 3 2024-02-18T00:41:00.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 10 client.41366.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 4 2024-02-18T00:41:00.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 4 2024-02-18T00:41:00.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b4028710 osd=1 3 2024-02-18T00:41:00.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 1 4 2024-02-18T00:41:00.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4028710 osd=1 4 2024-02-18T00:41:00.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 10 client.41366.objecter _send_command 4 2024-02-18T00:41:00.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.537+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b4059360 con 0x7f80b40598f0 2024-02-18T00:41:00.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b4059360 con 0x7f80b40598f0 2024-02-18T00:41:00.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80aaffd700 10 client.41366.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b4028710 osd=1 4 2024-02-18T00:41:00.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 1 4 2024-02-18T00:41:00.540 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:41:00.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 10 client.41366.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 5 2024-02-18T00:41:00.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6809/1153193637 conn(0x7f80b411c7f0 0x7f80b411ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 5 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 2 5 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027d60 osd=2 4 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 10 client.41366.objecter _send_command 5 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b40283e0 con 0x7f80b411c7f0 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.538+0000 7f80b97b0700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6809/1153193637 conn(0x7f80b411c7f0 0x7f80b411ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.539+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b411c7f0 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.539+0000 7f80b97b0700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6809/1153193637 conn(0x7f80b411c7f0 0x7f80b411ebe0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.539+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b40283e0 con 0x7f80b411c7f0 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.539+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:41:00.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.539+0000 7f80aaffd700 10 client.41366.objecter _finish_command 5 = system:0 2024-02-18T00:41:00.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.539+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b4027d60 osd=2 4 2024-02-18T00:41:00.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.539+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 2 5 2024-02-18T00:41:00.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 10 client.41366.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 6 2024-02-18T00:41:00.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 6 2024-02-18T00:41:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b4027d60 osd=2 3 2024-02-18T00:41:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 2 6 2024-02-18T00:41:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027d60 osd=2 4 2024-02-18T00:41:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 10 client.41366.objecter _send_command 6 2024-02-18T00:41:00.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.554+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b40277d0 con 0x7f80b411c7f0 2024-02-18T00:41:00.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b40277d0 con 0x7f80b411c7f0 2024-02-18T00:41:00.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80aaffd700 10 client.41366.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b4027d60 osd=2 4 2024-02-18T00:41:00.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 2 6 2024-02-18T00:41:00.557 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:41:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 10 client.41366.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 7 2024-02-18T00:41:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6813/1805700964 conn(0x7f80b411c350 0x7f80b4122260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 7 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 3 7 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027be0 osd=3 4 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 10 client.41366.objecter _send_command 7 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b4124eb0 con 0x7f80b411c350 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.555+0000 7f80ba7b2700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6813/1805700964 conn(0x7f80b411c350 0x7f80b4122260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.556+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b411c350 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.556+0000 7f80ba7b2700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6813/1805700964 conn(0x7f80b411c350 0x7f80b4122260 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.556+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b4124eb0 con 0x7f80b411c350 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.556+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.556+0000 7f80aaffd700 10 client.41366.objecter _finish_command 7 = system:0 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.556+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b4027be0 osd=3 4 2024-02-18T00:41:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.556+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 3 7 2024-02-18T00:41:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 10 client.41366.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 8 2024-02-18T00:41:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 8 2024-02-18T00:41:00.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b4027be0 osd=3 3 2024-02-18T00:41:00.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 3 8 2024-02-18T00:41:00.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027be0 osd=3 4 2024-02-18T00:41:00.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 10 client.41366.objecter _send_command 8 2024-02-18T00:41:00.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.571+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b411b8b0 con 0x7f80b411c350 2024-02-18T00:41:00.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b411b8b0 con 0x7f80b411c350 2024-02-18T00:41:00.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80aaffd700 10 client.41366.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b4027be0 osd=3 4 2024-02-18T00:41:00.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 3 8 2024-02-18T00:41:00.574 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 10 client.41366.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 9 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6800/2044387056 conn(0x7f80b411fdb0 0x7f80b41238d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 9 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 4 9 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411b8b0 osd=4 4 2024-02-18T00:41:00.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 10 client.41366.objecter _send_command 9 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.572+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b411ba80 con 0x7f80b411fdb0 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.573+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6800/2044387056 conn(0x7f80b411fdb0 0x7f80b41238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.573+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b411fdb0 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.573+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6800/2044387056 conn(0x7f80b411fdb0 0x7f80b41238d0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.574+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b411ba80 con 0x7f80b411fdb0 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.574+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.574+0000 7f80aaffd700 10 client.41366.objecter _finish_command 9 = system:0 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.574+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b411b8b0 osd=4 4 2024-02-18T00:41:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.574+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 4 9 2024-02-18T00:41:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 10 client.41366.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 10 2024-02-18T00:41:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 10 2024-02-18T00:41:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b411b8b0 osd=4 3 2024-02-18T00:41:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 4 10 2024-02-18T00:41:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411b8b0 osd=4 4 2024-02-18T00:41:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 10 client.41366.objecter _send_command 10 2024-02-18T00:41:00.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.589+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b411ba80 con 0x7f80b411fdb0 2024-02-18T00:41:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b411ba80 con 0x7f80b411fdb0 2024-02-18T00:41:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80aaffd700 10 client.41366.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b411b8b0 osd=4 4 2024-02-18T00:41:00.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 4 10 2024-02-18T00:41:00.593 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 10 client.41366.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 11 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6804/2170432263 conn(0x7f80b41242d0 0x7f80b4124750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 11 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 5 11 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411ba80 osd=5 4 2024-02-18T00:41:00.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 10 client.41366.objecter _send_command 11 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b411bca0 con 0x7f80b41242d0 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.591+0000 7f80b97b0700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6804/2170432263 conn(0x7f80b41242d0 0x7f80b4124750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.592+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b41242d0 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.592+0000 7f80b97b0700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6804/2170432263 conn(0x7f80b41242d0 0x7f80b4124750 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.592+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b411bca0 con 0x7f80b41242d0 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.592+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.592+0000 7f80aaffd700 10 client.41366.objecter _finish_command 11 = system:0 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.592+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b411ba80 osd=5 4 2024-02-18T00:41:00.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.592+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 5 11 2024-02-18T00:41:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 10 client.41366.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 12 2024-02-18T00:41:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 12 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b411ba80 osd=5 3 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 5 12 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411ba80 osd=5 4 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 10 client.41366.objecter _send_command 12 2024-02-18T00:41:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.607+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b411bca0 con 0x7f80b41242d0 2024-02-18T00:41:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b411bca0 con 0x7f80b41242d0 2024-02-18T00:41:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80aaffd700 10 client.41366.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b411ba80 osd=5 4 2024-02-18T00:41:00.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 5 12 2024-02-18T00:41:00.611 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 10 client.41366.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 13 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6808/3356009390 conn(0x7f80b4129550 0x7f80b412b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 13 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 6 13 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411bca0 osd=6 4 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 10 client.41366.objecter _send_command 13 2024-02-18T00:41:00.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b412e5d0 con 0x7f80b4129550 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80ba7b2700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6808/3356009390 conn(0x7f80b4129550 0x7f80b412b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80ba7b2700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6808/3356009390 conn(0x7f80b4129550 0x7f80b412b940 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.609+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b4129550 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.610+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b412e5d0 con 0x7f80b4129550 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.610+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.610+0000 7f80aaffd700 10 client.41366.objecter _finish_command 13 = system:0 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.610+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b411bca0 osd=6 4 2024-02-18T00:41:00.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.610+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 6 13 2024-02-18T00:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.624+0000 7f80bc215700 10 client.41366.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 14 2024-02-18T00:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 14 2024-02-18T00:41:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b411bca0 osd=6 3 2024-02-18T00:41:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 6 14 2024-02-18T00:41:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411bca0 osd=6 4 2024-02-18T00:41:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 10 client.41366.objecter _send_command 14 2024-02-18T00:41:00.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.625+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b412e5d0 con 0x7f80b4129550 2024-02-18T00:41:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.626+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b412e5d0 con 0x7f80b4129550 2024-02-18T00:41:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.626+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.626+0000 7f80aaffd700 10 client.41366.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.626+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b411bca0 osd=6 4 2024-02-18T00:41:00.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.626+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 6 14 2024-02-18T00:41:00.628 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.626+0000 7f80bc215700 10 client.41366.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 15 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6812/53025171 conn(0x7f80b412c340 0x7f80b412c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 15 2024-02-18T00:41:00.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 7 15 2024-02-18T00:41:00.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b412e5d0 osd=7 4 2024-02-18T00:41:00.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 10 client.41366.objecter _send_command 15 2024-02-18T00:41:00.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80b412e3f0 con 0x7f80b412c340 2024-02-18T00:41:00.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6812/53025171 conn(0x7f80b412c340 0x7f80b412c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:00.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80aaffd700 10 client.41366.objecter ms_handle_connect 0x7f80b412c340 2024-02-18T00:41:00.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.627+0000 7f80b9fb1700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6812/53025171 conn(0x7f80b412c340 0x7f80b412c7a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.628+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f80b412e3f0 con 0x7f80b412c340 2024-02-18T00:41:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.628+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:41:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.628+0000 7f80aaffd700 10 client.41366.objecter _finish_command 15 = system:0 2024-02-18T00:41:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.628+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b412e5d0 osd=7 4 2024-02-18T00:41:00.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.628+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 7 15 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 10 client.41366.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign -1 16 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 20 client.41366.objecter _get_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 15 client.41366.objecter _session_command_op_remove -1 16 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 20 client.41366.objecter get_session s=0x7f80b412e5d0 osd=7 3 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 15 client.41366.objecter _session_command_op_assign 7 16 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b412e5d0 osd=7 4 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 10 client.41366.objecter _send_command 16 2024-02-18T00:41:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.642+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f80b412e3f0 con 0x7f80b412c340 2024-02-18T00:41:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.644+0000 7f80aaffd700 1 -- 172.21.15.59:0/500826547 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f80b412e3f0 con 0x7f80b412c340 2024-02-18T00:41:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.644+0000 7f80aaffd700 10 client.41366.objecter ms_dispatch 0x7f80b4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.644+0000 7f80aaffd700 10 client.41366.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.644+0000 7f80aaffd700 20 client.41366.objecter put_session s=0x7f80b412e5d0 osd=7 4 2024-02-18T00:41:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.644+0000 7f80aaffd700 15 client.41366.objecter _session_command_op_remove 7 16 2024-02-18T00:41:00.646 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.644+0000 7f80bc215700 10 librados: watch_flush enter 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 librados: watch_flush exit 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.0 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6801/3700238977 conn(0x7f80b40fc150 msgr2=0x7f80b40fc530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6801/3700238977 conn(0x7f80b40fc150 0x7f80b40fc530 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b405ee80 osd=0 1 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.1 2024-02-18T00:41:00.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6805/26279110 conn(0x7f80b40598f0 msgr2=0x7f80b4059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6805/26279110 conn(0x7f80b40598f0 0x7f80b4059d10 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4028710 osd=1 1 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.2 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6809/1153193637 conn(0x7f80b411c7f0 msgr2=0x7f80b411ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6809/1153193637 conn(0x7f80b411c7f0 0x7f80b411ebe0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027d60 osd=2 1 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.3 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6813/1805700964 conn(0x7f80b411c350 msgr2=0x7f80b4122260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80ba7b2700 1 -- 172.21.15.59:0/500826547 reap_dead start 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6813/1805700964 conn(0x7f80b411c350 0x7f80b4122260 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b4027be0 osd=3 1 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.4 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6800/2044387056 conn(0x7f80b411fdb0 msgr2=0x7f80b41238d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6800/2044387056 conn(0x7f80b411fdb0 0x7f80b41238d0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411b8b0 osd=4 1 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.5 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6804/2170432263 conn(0x7f80b41242d0 msgr2=0x7f80b4124750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6804/2170432263 conn(0x7f80b41242d0 0x7f80b4124750 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411ba80 osd=5 1 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.6 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6808/3356009390 conn(0x7f80b4129550 msgr2=0x7f80b412b940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6808/3356009390 conn(0x7f80b4129550 0x7f80b412b940 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b411bca0 osd=6 1 2024-02-18T00:41:00.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter close_session for osd.7 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6812/53025171 conn(0x7f80b412c340 msgr2=0x7f80b412c7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.137:6812/53025171 conn(0x7f80b412c340 0x7f80b412c7a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter put_session s=0x7f80b412e5d0 osd=7 1 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 20 client.41366.objecter shutdown clearing up homeless session... 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 10 client.41366.objecter successfully canceled tick 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80ba7b2700 1 -- 172.21.15.59:0/500826547 reap_dead start 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80a0062c90 msgr2=0x7f80a0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80a0062c90 0x7f80a0065150 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 rx=0x7f80b0004790 tx=0x7f80b000b040).stop 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f9610 msgr2=0x7f80b410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f9610 0x7f80b410e780 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 rx=0x7f80a40043a0 tx=0x7f80a400b040).stop 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 shutdown_connections 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b40f9610 0x7f80b410e780 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 --2- 172.21.15.59:0/500826547 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80a0062c90 0x7f80a0065150 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 >> 172.21.15.59:0/500826547 conn(0x7f80b40f2820 msgr2=0x7f80b40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 shutdown_connections 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 -- 172.21.15.59:0/500826547 wait complete. 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:00.645+0000 7f80bc215700 1 librados: shutdown 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.652 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:41:00.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:59.725028+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:41:00.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:59.725107+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-18T00:41:00.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:59.726574+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:00.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:40:59.727721+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:00.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:40:59.946258+0000 mgr.y (mgr.24454) 494 : cluster [DBG] pgmap v804: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:00.945 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:00.503786+0000 mon.c (mon.2) 499 : audit [DBG] from='client.? 172.21.15.59:0/500826547' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:00.946 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:59.725028+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:41:00.946 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:59.725107+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-18T00:41:00.946 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:59.726574+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:00.946 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:40:59.727721+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:00.946 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:40:59.946258+0000 mgr.y (mgr.24454) 494 : cluster [DBG] pgmap v804: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:00.946 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:00.503786+0000 mon.c (mon.2) 499 : audit [DBG] from='client.? 172.21.15.59:0/500826547' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:59.725028+0000 mon.a (mon.0) 2824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:41:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:59.725107+0000 mon.a (mon.0) 2825 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-18T00:41:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:59.726574+0000 mon.c (mon.2) 498 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:01.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:40:59.727721+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:01.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:40:59.946258+0000 mgr.y (mgr.24454) 494 : cluster [DBG] pgmap v804: 270 pgs: 32 unknown, 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:01.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:00.503786+0000 mon.c (mon.2) 499 : audit [DBG] from='client.? 172.21.15.59:0/500826547' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:01.145 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:01.145 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:01.383 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:41:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:41:01.456 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:01.461 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:01.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:01.562 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:01.594 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:01.729 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:41:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:41:01.924 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-18T00:41:01.943 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:01.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:01.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:00.443709+0000 osd.0 (osd.0) 242 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:00.443928+0000 osd.0 (osd.0) 243 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:00.729728+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:00.729798+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:00.443709+0000 osd.0 (osd.0) 242 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:00.443928+0000 osd.0 (osd.0) 243 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:00.729728+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:41:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:00.729798+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-18T00:41:02.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:00.443709+0000 osd.0 (osd.0) 242 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:02.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:00.443928+0000 osd.0 (osd.0) 243 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:02.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:00.729728+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:41:02.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:00.729798+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-18T00:41:02.048 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:02.075 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:02.094 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:02.097 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:02.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:02.198 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:02.215 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:41:02.225 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.240+0000 7fd2667f5700 1 -- 172.21.15.59:0/247498314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 msgr2=0x7fd260104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.240+0000 7fd2667f5700 1 --2- 172.21.15.59:0/247498314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 0x7fd260104a20 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 rx=0x7fd258009be0 tx=0x7fd258004d10).stop 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.240+0000 7fd2667f5700 1 -- 172.21.15.59:0/247498314 shutdown_connections 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.240+0000 7fd2667f5700 1 --2- 172.21.15.59:0/247498314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2600fdbb0 0x7fd2600fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.240+0000 7fd2667f5700 1 --2- 172.21.15.59:0/247498314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 0x7fd260104a20 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.240+0000 7fd2667f5700 1 --2- 172.21.15.59:0/247498314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 0x7fd2600f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.240+0000 7fd2667f5700 1 -- 172.21.15.59:0/247498314 >> 172.21.15.59:0/247498314 conn(0x7fd2600f3800 msgr2=0x7fd2600f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 -- 172.21.15.59:0/247498314 shutdown_connections 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 -- 172.21.15.59:0/247498314 wait complete. 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 librados: starting msgr at 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 librados: starting objecter 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 Processor -- start 2024-02-18T00:41:02.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 -- start start 2024-02-18T00:41:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 librados: setting wanted keys 2024-02-18T00:41:02.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 librados: calling monclient init 2024-02-18T00:41:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 0x7fd26006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.241+0000 7fd2667f5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2600fdbb0 0x7fd26006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:02.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 0x7fd26006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260116280 con 0x7fd2600f8420 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2601163f0 con 0x7fd2600fdbb0 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2601165b0 con 0x7fd2600fe510 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd264d92700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 0x7fd26006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 0x7fd26006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd264d92700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 0x7fd26006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38822/0 (socket says 172.21.15.59:38822) 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd264d92700 1 -- 172.21.15.59:0/7430830 learned_addr learned my addr 172.21.15.59:0/7430830 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25f7fe700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2600fdbb0 0x7fd26006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25ffff700 1 -- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 msgr2=0x7fd26006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25ffff700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 0x7fd26006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25ffff700 1 -- 172.21.15.59:0/7430830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2600fdbb0 msgr2=0x7fd26006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25ffff700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2600fdbb0 0x7fd26006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25ffff700 1 -- 172.21.15.59:0/7430830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254005040 con 0x7fd2600f8420 2024-02-18T00:41:02.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd264d92700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 0x7fd26006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25ffff700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 0x7fd26006b0d0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7fd250002800 tx=0x7fd250009ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25d7fa700 10 client.?.objecter ms_handle_connect 0x7fd2600f8420 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25d7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25d7fa700 1 -- 172.21.15.59:0/7430830 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250007b40 con 0x7fd2600f8420 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2580097c0 con 0x7fd2600f8420 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25d7fa700 1 -- 172.21.15.59:0/7430830 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd25000f040 con 0x7fd2600f8420 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd25d7fa700 1 -- 172.21.15.59:0/7430830 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250014790 con 0x7fd2600f8420 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 10 client.31569.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd260116c70 con 0x7fd2600f8420 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.242+0000 7fd2667f5700 1 librados: init done 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.243+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26005ee80 con 0x7fd2600f8420 2024-02-18T00:41:02.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.244+0000 7fd25d7fa700 1 -- 172.21.15.59:0/7430830 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd250007ca0 con 0x7fd2600f8420 2024-02-18T00:41:02.248 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.244+0000 7fd25d7fa700 1 --2- 172.21.15.59:0/7430830 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd248062c90 0x7fd248065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.244+0000 7fd25d7fa700 1 -- 172.21.15.59:0/7430830 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(528..528 src has 180..528) v4 ==== 8250+0+0 (secure 0 0 0) 0x7fd25007d020 con 0x7fd2600f8420 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.244+0000 7fd25d7fa700 10 client.31569.objecter ms_dispatch 0x7fd260001550 osd_map(528..528 src has 180..528) v4 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.244+0000 7fd25d7fa700 3 client.31569.objecter handle_osd_map got epochs [528,528] > 0 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.244+0000 7fd25d7fa700 3 client.31569.objecter handle_osd_map decoding full epoch 528 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.244+0000 7fd25f7fe700 1 --2- 172.21.15.59:0/7430830 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd248062c90 0x7fd248065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.245+0000 7fd25d7fa700 20 client.31569.objecter dump_active .. 0 homeless 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.245+0000 7fd25d7fa700 10 client.31569.objecter ms_handle_connect 0x7fd248062c90 2024-02-18T00:41:02.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.245+0000 7fd25f7fe700 1 --2- 172.21.15.59:0/7430830 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd248062c90 0x7fd248065150 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 rx=0x7fd26006c870 tx=0x7fd25400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:02.250 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:02.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:02.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.249+0000 7fd25d7fa700 1 -- 172.21.15.59:0/7430830 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd250050430 con 0x7fd2600f8420 2024-02-18T00:41:02.351 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:02.377 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:02.430 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:02.436 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:02.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:02.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.432+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fd260107110 con 0x7fd2600f8420 2024-02-18T00:41:02.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:02] "GET /metrics HTTP/1.1" 200 190344 "" "Prometheus/2.33.4" 2024-02-18T00:41:02.537 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:02.567 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:02.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.741+0000 7fd25d7fa700 1 -- 172.21.15.59:0/7430830 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v529) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd250050020 con 0x7fd2600f8420 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.746+0000 7fd2667f5700 10 librados: watch_flush enter 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.746+0000 7fd2667f5700 10 librados: watch_flush exit 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.746+0000 7fd2667f5700 20 client.31569.objecter shutdown clearing up homeless session... 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.746+0000 7fd2667f5700 10 client.31569.objecter successfully canceled tick 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.746+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd248062c90 msgr2=0x7fd248065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.746+0000 7fd2667f5700 1 --2- 172.21.15.59:0/7430830 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd248062c90 0x7fd248065150 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 rx=0x7fd26006c870 tx=0x7fd25400b040).stop 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 msgr2=0x7fd26006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:02.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 0x7fd26006b0d0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7fd250002800 tx=0x7fd250009ec0).stop 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 shutdown_connections 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2600fe510 0x7fd26006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2600f8420 0x7fd26006b0d0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 --2- 172.21.15.59:0/7430830 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd248062c90 0x7fd248065150 unknown :-1 s=CLOSED pgs=3149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 --2- 172.21.15.59:0/7430830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2600fdbb0 0x7fd26006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 >> 172.21.15.59:0/7430830 conn(0x7fd2600f3800 msgr2=0x7fd2600fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 shutdown_connections 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 -- 172.21.15.59:0/7430830 wait complete. 2024-02-18T00:41:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:02.747+0000 7fd2667f5700 1 librados: shutdown 2024-02-18T00:41:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:41:02.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:01.382380+0000 mgr.y (mgr.24454) 495 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:02.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:01.735612+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-18T00:41:02.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:01.776901+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:02.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:01.777251+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:02.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:01.777767+0000 mon.c (mon.2) 501 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-83"}]: dispatch 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:01.778008+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-83"}]: dispatch 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:01.946673+0000 mgr.y (mgr.24454) 496 : cluster [DBG] pgmap v807: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:02.434308+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? 172.21.15.59:0/7430830' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:01.382380+0000 mgr.y (mgr.24454) 495 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:01.735612+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:01.776901+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:01.777251+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:01.777767+0000 mon.c (mon.2) 501 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-83"}]: dispatch 2024-02-18T00:41:02.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:01.778008+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-83"}]: dispatch 2024-02-18T00:41:02.854 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:01.946673+0000 mgr.y (mgr.24454) 496 : cluster [DBG] pgmap v807: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-18T00:41:02.854 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:02.434308+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? 172.21.15.59:0/7430830' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:03.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:01.382380+0000 mgr.y (mgr.24454) 495 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:01.735612+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-18T00:41:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:01.776901+0000 mon.c (mon.2) 500 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:01.777251+0000 mon.a (mon.0) 2830 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:01.777767+0000 mon.c (mon.2) 501 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-83"}]: dispatch 2024-02-18T00:41:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:01.778008+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-83"}]: dispatch 2024-02-18T00:41:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:01.946673+0000 mgr.y (mgr.24454) 496 : cluster [DBG] pgmap v807: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-18T00:41:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:02.434308+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? 172.21.15.59:0/7430830' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:03.617 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:03.620 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:03.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:03.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:03.776 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:03.798 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:03.800 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:03.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:03.902 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:03.931 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:03.954 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:03.957 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:03.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:03.962 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:03.962 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:04.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:02.732868+0000 mon.a (mon.0) 2833 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:04.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:02.742094+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.59:0/7430830' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:04.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:02.742164+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-18T00:41:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:02.732868+0000 mon.a (mon.0) 2833 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:02.742094+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.59:0/7430830' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:04.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:02.742164+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-18T00:41:04.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:02.732868+0000 mon.a (mon.0) 2833 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:04.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:02.742094+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? 172.21.15.59:0/7430830' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:04.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:02.742164+0000 mon.a (mon.0) 2835 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-18T00:41:04.059 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:04.087 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:04.185 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:04.187 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:04.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:04.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:04.320 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:04.344 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:04.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:04.446 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:04.496 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:04.499 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:04.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:04.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:04.649 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:04.653 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:04.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:04.750 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-18T00:41:04.755 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:04.773 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:41:04.773 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:41:04.773 INFO:tasks.thrashosds.thrasher:Shrinking pool default.rgw.meta 2024-02-18T00:41:04.774 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool default.rgw.meta by 10 min 8 2024-02-18T00:41:04.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:41:04.798 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:04.817 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:04.820 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:04.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:04.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:41:04.895 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:03.750239+0000 mon.a (mon.0) 2836 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-18T00:41:04.895 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:03.759840+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:04.895 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:03.760835+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:04.895 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:03.947118+0000 mgr.y (mgr.24454) 497 : cluster [DBG] pgmap v810: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:04.896 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:03.750239+0000 mon.a (mon.0) 2836 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-18T00:41:04.896 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:03.759840+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:04.896 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:03.760835+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:04.896 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:03.947118+0000 mgr.y (mgr.24454) 497 : cluster [DBG] pgmap v810: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:04.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:04.950 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:04.976 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:04.979 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:04.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:03.750239+0000 mon.a (mon.0) 2836 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-18T00:41:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:03.759840+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:03.760835+0000 mon.a (mon.0) 2837 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:05.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:03.947118+0000 mgr.y (mgr.24454) 497 : cluster [DBG] pgmap v810: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:05.031 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:41:05.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:05.096 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:05.111 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:05.137 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:05.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:05.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:05.241 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:05.268 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:05.289 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:05.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:05.292 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:05.393 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:05.419 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:05.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:41:05.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:04.756020+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:05.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:04.756195+0000 mon.a (mon.0) 2839 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-18T00:41:05.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:04.759950+0000 mon.c (mon.2) 503 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:41:05.915 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:04.764169+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:05.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:04.764681+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:05.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:05.608994+0000 osd.4 (osd.4) 309 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:05.916 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:05.609272+0000 osd.4 (osd.4) 310 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:05.945 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:04.756020+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:04.756195+0000 mon.a (mon.0) 2839 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:04.759950+0000 mon.c (mon.2) 503 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:04.764169+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:04.764681+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:05.608994+0000 osd.4 (osd.4) 309 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:05.609272+0000 osd.4 (osd.4) 310 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:04.756020+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:06.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:04.756195+0000 mon.a (mon.0) 2839 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-18T00:41:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:04.759950+0000 mon.c (mon.2) 503 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:41:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:04.764169+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:04.764681+0000 mon.a (mon.0) 2840 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:05.608994+0000 osd.4 (osd.4) 309 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:06.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:05.609272+0000 osd.4 (osd.4) 310 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:06.771 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:06.774 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:06.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:06.875 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:06.907 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:06.968 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:06.970 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:06.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:07.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:05.760604+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:41:07.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:05.760684+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-18T00:41:07.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:05.948320+0000 mgr.y (mgr.24454) 498 : cluster [DBG] pgmap v813: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:41:07.011 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:07.011 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:07.072 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:07.100 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:05.760604+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:41:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:05.760684+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-18T00:41:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:05.948320+0000 mgr.y (mgr.24454) 498 : cluster [DBG] pgmap v813: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:41:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:05.760604+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:41:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:05.760684+0000 mon.a (mon.0) 2842 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-18T00:41:07.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:05.948320+0000 mgr.y (mgr.24454) 498 : cluster [DBG] pgmap v813: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:41:07.300 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:07.303 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:07.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:07.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:07.440 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:07.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-18T00:41:08.075 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:41:08.098 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:06.777069+0000 mon.a (mon.0) 2843 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-18T00:41:08.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:06.777069+0000 mon.a (mon.0) 2843 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-18T00:41:08.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:06.777069+0000 mon.a (mon.0) 2843 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-18T00:41:08.521 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22248 ms) 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (2759 ms) 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3012 ms) 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (4031 ms) 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (2958 ms) 2024-02-18T00:41:08.741 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4020 ms) 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39271 ms) 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16082 ms) 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4021 ms) 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3022 ms) 2024-02-18T00:41:08.742 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24616 ms) 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14459 ms) 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5031 ms) 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: waiting for scrubs... 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: done waiting 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24535 ms) 2024-02-18T00:41:08.743 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10069 ms) 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (8931 ms) 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9016 ms) 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.744 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (8993 ms) 2024-02-18T00:41:08.745 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-18T00:41:08.745 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:41:08.745 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (5993 ms) 2024-02-18T00:41:08.866 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:08.869 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:08.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:08.971 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:09.000 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:09.157 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:09.164 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:09.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:09.227 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:07.778188+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-18T00:41:09.227 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:07.824600+0000 mon.c (mon.2) 505 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:09.227 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:07.824996+0000 mon.a (mon.0) 2845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:09.227 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:07.825581+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-86"}]: dispatch 2024-02-18T00:41:09.227 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:07.825882+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-86"}]: dispatch 2024-02-18T00:41:09.227 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:07.948811+0000 mgr.y (mgr.24454) 499 : cluster [DBG] pgmap v816: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T00:41:09.227 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:08.730786+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-18T00:41:09.266 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:09.294 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:09.441 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:09.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:09.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:07.778188+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-18T00:41:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:07.824600+0000 mon.c (mon.2) 505 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:09.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:07.824996+0000 mon.a (mon.0) 2845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:09.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:07.825581+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-86"}]: dispatch 2024-02-18T00:41:09.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:07.825882+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-86"}]: dispatch 2024-02-18T00:41:09.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:07.948811+0000 mgr.y (mgr.24454) 499 : cluster [DBG] pgmap v816: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T00:41:09.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:08.730786+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-18T00:41:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:07.778188+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-18T00:41:09.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:07.824600+0000 mon.c (mon.2) 505 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:09.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:07.824996+0000 mon.a (mon.0) 2845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:09.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:07.825581+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-86"}]: dispatch 2024-02-18T00:41:09.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:07.825882+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-86"}]: dispatch 2024-02-18T00:41:09.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:07.948811+0000 mgr.y (mgr.24454) 499 : cluster [DBG] pgmap v816: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T00:41:09.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:08.730786+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 -- 172.21.15.59:0/2859491877 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 msgr2=0x7f712c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 --2- 172.21.15.59:0/2859491877 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c103a20 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 rx=0x7f7124009be0 tx=0x7f7124004d10).stop 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 -- 172.21.15.59:0/2859491877 shutdown_connections 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 --2- 172.21.15.59:0/2859491877 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c103a20 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 --2- 172.21.15.59:0/2859491877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f712c0fcbb0 0x7f712c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 --2- 172.21.15.59:0/2859491877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c0fd510 0x7f712c0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 -- 172.21.15.59:0/2859491877 >> 172.21.15.59:0/2859491877 conn(0x7f712c0f2820 msgr2=0x7f712c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 -- 172.21.15.59:0/2859491877 shutdown_connections 2024-02-18T00:41:09.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 -- 172.21.15.59:0/2859491877 wait complete. 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 librados: starting msgr at 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.521+0000 7f7131e39700 1 librados: starting objecter 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 Processor -- start 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 -- start start 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 librados: setting wanted keys 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 librados: calling monclient init 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c10e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c0fcbb0 0x7f712c10e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f712c0fd510 0x7f712c108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c108730 con 0x7f712c0fd510 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1088a0 con 0x7f712c0fcbb0 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f7131e39700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c108a10 con 0x7f712c0f7400 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c10e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c10e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57004/0 (socket says 172.21.15.59:57004) 2024-02-18T00:41:09.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 -- 172.21.15.59:0/3694358696 learned_addr learned my addr 172.21.15.59:0/3694358696 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712bfff700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f712c0fd510 0x7f712c108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712affd700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c0fcbb0 0x7f712c10e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 -- 172.21.15.59:0/3694358696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c0fcbb0 msgr2=0x7f712c10e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c0fcbb0 0x7f712c10e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 -- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f712c0fd510 msgr2=0x7f712c108240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f712c0fd510 0x7f712c108240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712b7fe700 1 -- 172.21.15.59:0/3694358696 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71240097c0 con 0x7f712c0f7400 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.522+0000 7f712bfff700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f712c0fd510 0x7f712c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f712b7fe700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c10e010 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 rx=0x7f712000e970 tx=0x7f712000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7128ff9700 10 client.?.objecter ms_handle_connect 0x7f712c0f7400 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7128ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7128ff9700 1 -- 172.21.15.59:0/3694358696 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000ee60 con 0x7f712c0f7400 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c108cf0 con 0x7f712c0f7400 2024-02-18T00:41:09.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7128ff9700 1 -- 172.21.15.59:0/3694358696 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712000f040 con 0x7f712c0f7400 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7128ff9700 1 -- 172.21.15.59:0/3694358696 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120007ea0 con 0x7f712c0f7400 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7131e39700 10 client.41378.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c114e80 con 0x7f712c0f7400 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.523+0000 7f7131e39700 1 librados: init done 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.524+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c05fa60 con 0x7f712c0f7400 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.524+0000 7f7128ff9700 1 -- 172.21.15.59:0/3694358696 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f712000cb10 con 0x7f712c0f7400 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f7128ff9700 1 --2- 172.21.15.59:0/3694358696 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7114062c90 0x7f7114065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f7128ff9700 1 -- 172.21.15.59:0/3694358696 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(535..535 src has 418..535) v4 ==== 7798+0+0 (secure 0 0 0) 0x7f712007cd20 con 0x7f712c0f7400 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f7128ff9700 10 client.41378.objecter ms_dispatch 0x7f712c001550 osd_map(535..535 src has 418..535) v4 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f7128ff9700 3 client.41378.objecter handle_osd_map got epochs [535,535] > 0 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f7128ff9700 3 client.41378.objecter handle_osd_map decoding full epoch 535 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f712affd700 1 --2- 172.21.15.59:0/3694358696 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7114062c90 0x7f7114065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:09.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f7128ff9700 20 client.41378.objecter dump_active .. 0 homeless 2024-02-18T00:41:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f7128ff9700 10 client.41378.objecter ms_handle_connect 0x7f7114062c90 2024-02-18T00:41:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.525+0000 7f712affd700 1 --2- 172.21.15.59:0/3694358696 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7114062c90 0x7f7114065150 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 rx=0x7f712c109990 tx=0x7f711c008040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:09.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.529+0000 7f7128ff9700 1 -- 172.21.15.59:0/3694358696 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f712004af60 con 0x7f712c0f7400 2024-02-18T00:41:09.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:09.576 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:09.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.698+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f712c1096d0 con 0x7f7114062c90 2024-02-18T00:41:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.713+0000 7f7128ff9700 1 -- 172.21.15.59:0/3694358696 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529475 (secure 0 0 0) 0x7f712c1096d0 con 0x7f7114062c90 2024-02-18T00:41:09.722 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:41:09.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 10 librados: watch_flush enter 2024-02-18T00:41:09.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 10 librados: watch_flush exit 2024-02-18T00:41:09.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 20 client.41378.objecter shutdown clearing up homeless session... 2024-02-18T00:41:09.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 10 client.41378.objecter successfully canceled tick 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7114062c90 msgr2=0x7f7114065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 --2- 172.21.15.59:0/3694358696 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7114062c90 0x7f7114065150 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 rx=0x7f712c109990 tx=0x7f711c008040).stop 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 msgr2=0x7f712c10e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c10e010 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 rx=0x7f712000e970 tx=0x7f712000b040).stop 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 shutdown_connections 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f712c0fd510 0x7f712c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f712c0f7400 0x7f712c10e010 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 --2- 172.21.15.59:0/3694358696 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7114062c90 0x7f7114065150 unknown :-1 s=CLOSED pgs=3150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 --2- 172.21.15.59:0/3694358696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c0fcbb0 0x7f712c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 >> 172.21.15.59:0/3694358696 conn(0x7f712c0f2820 msgr2=0x7f712c060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 shutdown_connections 2024-02-18T00:41:09.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 -- 172.21.15.59:0/3694358696 wait complete. 2024-02-18T00:41:09.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:09.717+0000 7f7131e39700 1 librados: shutdown 2024-02-18T00:41:09.729 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:41:09.729 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:09.729 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:10.326 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:10.329 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:10.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:10.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:10.444 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-18T00:41:10.461 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:10.714 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:41:10.821 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":817,"stamp":"2024-02-18T00:41:08.727850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4026233,"num_objects":414,"num_object_clones":9,"num_object_copies":1242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414,"num_whiteouts":0,"num_read":2444,"num_read_kb":2011,"num_write":1683,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":571,"num_bytes_recovered":4005421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":18,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3063,"ondisk_log_size":3063,"up":618,"acting":618,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":707,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2523492,"kb_used_data":53572,"kb_used_omap":0,"kb_used_meta":2463744,"kb_avail":747437724,"statfs":{"total":767960285184,"available":765376229376,"internally_reserved":0,"allocated":54857728,"data_stored":34369966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2522873856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.975207"},"pg_stats":[{"pgid":"62.0","version":"510'72","reported_seq":120,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769999+0000","last_change":"2024-02-18T00:39:38.459941+0000","last_active":"2024-02-18T00:41:04.769999+0000","last_peered":"2024-02-18T00:41:04.769999+0000","last_clean":"2024-02-18T00:41:04.769999+0000","last_became_active":"2024-02-18T00:39:36.768967+0000","last_became_peered":"2024-02-18T00:39:36.768967+0000","last_unstale":"2024-02-18T00:41:04.769999+0000","last_undegraded":"2024-02-18T00:41:04.769999+0000","last_fullsized":"2024-02-18T00:41:04.769999+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'68","last_scrub_stamp":"2024-02-18T00:38:57.506738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.506738+0000","log_size":72,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":26,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1","version":"519'8","reported_seq":509,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769434+0000","last_change":"2024-02-18T00:40:47.681768+0000","last_active":"2024-02-18T00:41:04.769434+0000","last_peered":"2024-02-18T00:41:04.769434+0000","last_clean":"2024-02-18T00:41:04.769434+0000","last_became_active":"2024-02-18T00:33:21.214410+0000","last_became_peered":"2024-02-18T00:33:21.214410+0000","last_unstale":"2024-02-18T00:41:04.769434+0000","last_undegraded":"2024-02-18T00:41:04.769434+0000","last_fullsized":"2024-02-18T00:41:04.769434+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"401'6","last_scrub_stamp":"2024-02-18T00:40:47.681685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:47.681685+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1941,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.2","version":"519'92","reported_seq":633,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762900+0000","last_change":"2024-02-18T00:39:38.447789+0000","last_active":"2024-02-18T00:41:04.762900+0000","last_peered":"2024-02-18T00:41:04.762900+0000","last_clean":"2024-02-18T00:41:04.762900+0000","last_became_active":"2024-02-18T00:39:36.751479+0000","last_became_peered":"2024-02-18T00:39:36.751479+0000","last_unstale":"2024-02-18T00:41:04.762900+0000","last_undegraded":"2024-02-18T00:41:04.762900+0000","last_fullsized":"2024-02-18T00:41:04.762900+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'85","last_scrub_stamp":"2024-02-18T00:38:58.749675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:58.749675+0000","log_size":92,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":772,"num_objects":8,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":35,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":72,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.3","version":"510'78","reported_seq":560,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755694+0000","last_change":"2024-02-18T00:39:14.407030+0000","last_active":"2024-02-18T00:41:04.755694+0000","last_peered":"2024-02-18T00:41:04.755694+0000","last_clean":"2024-02-18T00:41:04.755694+0000","last_became_active":"2024-02-18T00:33:21.169284+0000","last_became_peered":"2024-02-18T00:33:21.169284+0000","last_unstale":"2024-02-18T00:41:04.755694+0000","last_undegraded":"2024-02-18T00:41:04.755694+0000","last_fullsized":"2024-02-18T00:41:04.755694+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'75","last_scrub_stamp":"2024-02-18T00:39:14.406966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.406966+0000","log_size":78,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":586,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":29,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.4","version":"491'126","reported_seq":695,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756195+0000","last_change":"2024-02-18T00:39:43.724364+0000","last_active":"2024-02-18T00:41:04.756195+0000","last_peered":"2024-02-18T00:41:04.756195+0000","last_clean":"2024-02-18T00:41:04.756195+0000","last_became_active":"2024-02-18T00:39:43.723634+0000","last_became_peered":"2024-02-18T00:39:43.723634+0000","last_unstale":"2024-02-18T00:41:04.756195+0000","last_undegraded":"2024-02-18T00:41:04.756195+0000","last_fullsized":"2024-02-18T00:41:04.756195+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"404'123","last_scrub_stamp":"2024-02-18T00:39:13.645424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:13.645424+0000","log_size":126,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":85,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":42,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":85,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.5","version":"510'96","reported_seq":570,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764068+0000","last_change":"2024-02-18T00:38:59.691992+0000","last_active":"2024-02-18T00:41:04.764068+0000","last_peered":"2024-02-18T00:41:04.764068+0000","last_clean":"2024-02-18T00:41:04.764068+0000","last_became_active":"2024-02-18T00:33:21.205582+0000","last_became_peered":"2024-02-18T00:33:21.205582+0000","last_unstale":"2024-02-18T00:41:04.764068+0000","last_undegraded":"2024-02-18T00:41:04.764068+0000","last_fullsized":"2024-02-18T00:41:04.764068+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"296'95","last_scrub_stamp":"2024-02-18T00:38:59.313626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.313626+0000","log_size":96,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2378,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":6,"num_read_kb":0,"num_write":34,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.6","version":"404'127","reported_seq":499,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.770918+0000","last_change":"2024-02-18T00:39:55.722681+0000","last_active":"2024-02-18T00:41:04.770918+0000","last_peered":"2024-02-18T00:41:04.770918+0000","last_clean":"2024-02-18T00:41:04.770918+0000","last_became_active":"2024-02-18T00:39:39.708457+0000","last_became_peered":"2024-02-18T00:39:39.708457+0000","last_unstale":"2024-02-18T00:41:04.770918+0000","last_undegraded":"2024-02-18T00:41:04.770918+0000","last_fullsized":"2024-02-18T00:41:04.770918+0000","mapping_epoch":467,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":468,"parent":"0.0","parent_split_bits":0,"last_scrub":"404'127","last_scrub_stamp":"2024-02-18T00:39:55.722602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:55.722602+0000","log_size":127,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":95,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":39,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":95,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.7","version":"457'82","reported_seq":560,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757651+0000","last_change":"2024-02-18T00:39:21.441449+0000","last_active":"2024-02-18T00:41:04.757651+0000","last_peered":"2024-02-18T00:41:04.757651+0000","last_clean":"2024-02-18T00:41:04.757651+0000","last_became_active":"2024-02-18T00:33:21.200119+0000","last_became_peered":"2024-02-18T00:33:21.200119+0000","last_unstale":"2024-02-18T00:41:04.757651+0000","last_undegraded":"2024-02-18T00:41:04.757651+0000","last_fullsized":"2024-02-18T00:41:04.757651+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'82","last_scrub_stamp":"2024-02-18T00:39:21.441381+0000","last_deep_scrub":"457'82","last_deep_scrub_stamp":"2024-02-18T00:39:21.441381+0000","last_clean_scrub_stamp":"2024-02-18T00:39:21.441381+0000","log_size":82,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":24,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.8","version":"347'70","reported_seq":122,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758481+0000","last_change":"2024-02-18T00:40:39.524488+0000","last_active":"2024-02-18T00:41:04.758481+0000","last_peered":"2024-02-18T00:41:04.758481+0000","last_clean":"2024-02-18T00:41:04.758481+0000","last_became_active":"2024-02-18T00:39:43.715780+0000","last_became_peered":"2024-02-18T00:39:43.715780+0000","last_unstale":"2024-02-18T00:41:04.758481+0000","last_undegraded":"2024-02-18T00:41:04.758481+0000","last_fullsized":"2024-02-18T00:41:04.758481+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'70","last_scrub_stamp":"2024-02-18T00:40:39.524387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.524387+0000","log_size":70,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":22,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":60,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"62.9","version":"519'53","reported_seq":533,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768579+0000","last_change":"2024-02-18T00:38:59.690323+0000","last_active":"2024-02-18T00:41:04.768579+0000","last_peered":"2024-02-18T00:41:04.768579+0000","last_clean":"2024-02-18T00:41:04.768579+0000","last_became_active":"2024-02-18T00:33:21.193043+0000","last_became_peered":"2024-02-18T00:33:21.193043+0000","last_unstale":"2024-02-18T00:41:04.768579+0000","last_undegraded":"2024-02-18T00:41:04.768579+0000","last_fullsized":"2024-02-18T00:41:04.768579+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'52","last_scrub_stamp":"2024-02-18T00:38:27.136753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:27.136753+0000","log_size":53,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2351,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":18,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.a","version":"501'47","reported_seq":112,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771190+0000","last_change":"2024-02-18T00:39:38.424668+0000","last_active":"2024-02-18T00:41:04.771190+0000","last_peered":"2024-02-18T00:41:04.771190+0000","last_clean":"2024-02-18T00:41:04.771190+0000","last_became_active":"2024-02-18T00:39:36.767609+0000","last_became_peered":"2024-02-18T00:39:36.767609+0000","last_unstale":"2024-02-18T00:41:04.771190+0000","last_undegraded":"2024-02-18T00:41:04.771190+0000","last_fullsized":"2024-02-18T00:41:04.771190+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'41","last_scrub_stamp":"2024-02-18T00:39:02.513916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.513916+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":53,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":22,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":43,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.b","version":"510'67","reported_seq":115,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.772427+0000","last_change":"2024-02-18T00:39:38.425532+0000","last_active":"2024-02-18T00:41:04.772427+0000","last_peered":"2024-02-18T00:41:04.772427+0000","last_clean":"2024-02-18T00:41:04.772427+0000","last_became_active":"2024-02-18T00:39:36.765301+0000","last_became_peered":"2024-02-18T00:39:36.765301+0000","last_unstale":"2024-02-18T00:41:04.772427+0000","last_undegraded":"2024-02-18T00:41:04.772427+0000","last_fullsized":"2024-02-18T00:41:04.772427+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"428'66","last_scrub_stamp":"2024-02-18T00:38:34.725717+0000","last_deep_scrub":"428'66","last_deep_scrub_stamp":"2024-02-18T00:38:34.725717+0000","last_clean_scrub_stamp":"2024-02-18T00:38:34.725717+0000","log_size":67,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":626,"num_objects":8,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":22,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":59,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.c","version":"500'20","reported_seq":504,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765047+0000","last_change":"2024-02-18T00:38:59.689614+0000","last_active":"2024-02-18T00:41:04.765047+0000","last_peered":"2024-02-18T00:41:04.765047+0000","last_clean":"2024-02-18T00:41:04.765047+0000","last_became_active":"2024-02-18T00:33:21.190840+0000","last_became_peered":"2024-02-18T00:33:21.190840+0000","last_unstale":"2024-02-18T00:41:04.765047+0000","last_undegraded":"2024-02-18T00:41:04.765047+0000","last_fullsized":"2024-02-18T00:41:04.765047+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"379'18","last_scrub_stamp":"2024-02-18T00:38:30.791397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:30.791397+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.d","version":"500'33","reported_seq":573,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763961+0000","last_change":"2024-02-18T00:39:37.717407+0000","last_active":"2024-02-18T00:41:04.763961+0000","last_peered":"2024-02-18T00:41:04.763961+0000","last_clean":"2024-02-18T00:41:04.763961+0000","last_became_active":"2024-02-18T00:39:36.743603+0000","last_became_peered":"2024-02-18T00:39:36.743603+0000","last_unstale":"2024-02-18T00:41:04.763961+0000","last_undegraded":"2024-02-18T00:41:04.763961+0000","last_fullsized":"2024-02-18T00:41:04.763961+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'29","last_scrub_stamp":"2024-02-18T00:38:28.139054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.139054+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":17,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":45,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.e","version":"501'87","reported_seq":559,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757195+0000","last_change":"2024-02-18T00:38:59.691183+0000","last_active":"2024-02-18T00:41:04.757195+0000","last_peered":"2024-02-18T00:41:04.757195+0000","last_clean":"2024-02-18T00:41:04.757195+0000","last_became_active":"2024-02-18T00:33:21.154999+0000","last_became_peered":"2024-02-18T00:33:21.154999+0000","last_unstale":"2024-02-18T00:41:04.757195+0000","last_undegraded":"2024-02-18T00:41:04.757195+0000","last_fullsized":"2024-02-18T00:41:04.757195+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'83","last_scrub_stamp":"2024-02-18T00:38:30.700348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:30.700348+0000","log_size":87,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":67,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":10,"num_read_kb":5,"num_write":33,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,3],"acting":[0,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.f","version":"510'76","reported_seq":545,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764249+0000","last_change":"2024-02-18T00:39:08.309384+0000","last_active":"2024-02-18T00:41:04.764249+0000","last_peered":"2024-02-18T00:41:04.764249+0000","last_clean":"2024-02-18T00:41:04.764249+0000","last_became_active":"2024-02-18T00:33:21.176077+0000","last_became_peered":"2024-02-18T00:33:21.176077+0000","last_unstale":"2024-02-18T00:41:04.764249+0000","last_undegraded":"2024-02-18T00:41:04.764249+0000","last_fullsized":"2024-02-18T00:41:04.764249+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"229'68","last_scrub_stamp":"2024-02-18T00:39:08.309293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:08.309293+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1596,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":28,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.10","version":"510'58","reported_seq":605,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762528+0000","last_change":"2024-02-18T00:40:03.466882+0000","last_active":"2024-02-18T00:41:04.762528+0000","last_peered":"2024-02-18T00:41:04.762528+0000","last_clean":"2024-02-18T00:41:04.762528+0000","last_became_active":"2024-02-18T00:39:36.755406+0000","last_became_peered":"2024-02-18T00:39:36.755406+0000","last_unstale":"2024-02-18T00:41:04.762528+0000","last_undegraded":"2024-02-18T00:41:04.762528+0000","last_fullsized":"2024-02-18T00:41:04.762528+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'57","last_scrub_stamp":"2024-02-18T00:40:03.466769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.466769+0000","log_size":58,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1256,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":49,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.11","version":"500'27","reported_seq":522,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762959+0000","last_change":"2024-02-18T00:39:55.464807+0000","last_active":"2024-02-18T00:41:04.762959+0000","last_peered":"2024-02-18T00:41:04.762959+0000","last_clean":"2024-02-18T00:41:04.762959+0000","last_became_active":"2024-02-18T00:33:21.214785+0000","last_became_peered":"2024-02-18T00:33:21.214785+0000","last_unstale":"2024-02-18T00:41:04.762959+0000","last_undegraded":"2024-02-18T00:41:04.762959+0000","last_fullsized":"2024-02-18T00:41:04.762959+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"398'21","last_scrub_stamp":"2024-02-18T00:39:55.464737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:55.464737+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"62.12","version":"510'80","reported_seq":558,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755294+0000","last_change":"2024-02-18T00:39:01.511082+0000","last_active":"2024-02-18T00:41:04.755294+0000","last_peered":"2024-02-18T00:41:04.755294+0000","last_clean":"2024-02-18T00:41:04.755294+0000","last_became_active":"2024-02-18T00:33:21.200535+0000","last_became_peered":"2024-02-18T00:33:21.200535+0000","last_unstale":"2024-02-18T00:41:04.755294+0000","last_undegraded":"2024-02-18T00:41:04.755294+0000","last_fullsized":"2024-02-18T00:41:04.755294+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"408'77","last_scrub_stamp":"2024-02-18T00:39:01.510999+0000","last_deep_scrub":"288'72","last_deep_scrub_stamp":"2024-02-18T00:36:06.758769+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.510999+0000","log_size":80,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":60,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":6,"num_read_kb":1,"num_write":28,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.13","version":"501'26","reported_seq":513,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757878+0000","last_change":"2024-02-18T00:39:18.521234+0000","last_active":"2024-02-18T00:41:04.757878+0000","last_peered":"2024-02-18T00:41:04.757878+0000","last_clean":"2024-02-18T00:41:04.757878+0000","last_became_active":"2024-02-18T00:33:21.224374+0000","last_became_peered":"2024-02-18T00:33:21.224374+0000","last_unstale":"2024-02-18T00:41:04.757878+0000","last_undegraded":"2024-02-18T00:41:04.757878+0000","last_fullsized":"2024-02-18T00:41:04.757878+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"361'18","last_scrub_stamp":"2024-02-18T00:39:18.521162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.521162+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":17,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"62.16","version":"519'27","reported_seq":518,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764480+0000","last_change":"2024-02-18T00:38:59.692296+0000","last_active":"2024-02-18T00:41:04.764480+0000","last_peered":"2024-02-18T00:41:04.764480+0000","last_clean":"2024-02-18T00:41:04.764480+0000","last_became_active":"2024-02-18T00:33:21.225281+0000","last_became_peered":"2024-02-18T00:33:21.225281+0000","last_unstale":"2024-02-18T00:41:04.764480+0000","last_undegraded":"2024-02-18T00:41:04.764480+0000","last_fullsized":"2024-02-18T00:41:04.764480+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'24","last_scrub_stamp":"2024-02-18T00:38:57.350170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.350170+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1353,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":1,"num_write":14,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.17","version":"528'46","reported_seq":575,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764793+0000","last_change":"2024-02-18T00:40:02.838331+0000","last_active":"2024-02-18T00:41:04.764793+0000","last_peered":"2024-02-18T00:41:04.764793+0000","last_clean":"2024-02-18T00:41:04.764793+0000","last_became_active":"2024-02-18T00:39:36.761151+0000","last_became_peered":"2024-02-18T00:39:36.761151+0000","last_unstale":"2024-02-18T00:41:04.764793+0000","last_undegraded":"2024-02-18T00:41:04.764793+0000","last_fullsized":"2024-02-18T00:41:04.764793+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"461'38","last_scrub_stamp":"2024-02-18T00:40:02.838262+0000","last_deep_scrub":"244'18","last_deep_scrub_stamp":"2024-02-18T00:35:23.483932+0000","last_clean_scrub_stamp":"2024-02-18T00:40:02.838262+0000","log_size":46,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4001270,"num_objects":7,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":27,"num_write_kb":3927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":4000055,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.15","version":"519'123","reported_seq":676,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765163+0000","last_change":"2024-02-18T00:39:39.707766+0000","last_active":"2024-02-18T00:41:04.765163+0000","last_peered":"2024-02-18T00:41:04.765163+0000","last_clean":"2024-02-18T00:41:04.765163+0000","last_became_active":"2024-02-18T00:39:39.707071+0000","last_became_peered":"2024-02-18T00:39:39.707071+0000","last_unstale":"2024-02-18T00:41:04.765163+0000","last_undegraded":"2024-02-18T00:41:04.765163+0000","last_fullsized":"2024-02-18T00:41:04.765163+0000","mapping_epoch":467,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":468,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'102","last_scrub_stamp":"2024-02-18T00:39:03.663869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.663869+0000","log_size":123,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":96,"num_objects":9,"num_object_clones":2,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":42,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":66,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":458,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762667+0000","last_change":"2024-02-18T00:39:36.752108+0000","last_active":"2024-02-18T00:41:04.762667+0000","last_peered":"2024-02-18T00:41:04.762667+0000","last_clean":"2024-02-18T00:41:04.762667+0000","last_became_active":"2024-02-18T00:39:36.751760+0000","last_became_peered":"2024-02-18T00:39:36.751760+0000","last_unstale":"2024-02-18T00:41:04.762667+0000","last_undegraded":"2024-02-18T00:41:04.762667+0000","last_fullsized":"2024-02-18T00:41:04.762667+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:18.857990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:39:18.857990+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.14","version":"532'117","reported_seq":608,"reported_epoch":532,"state":"active+clean","last_fresh":"2024-02-18T00:41:05.756425+0000","last_change":"2024-02-18T00:39:51.746807+0000","last_active":"2024-02-18T00:41:05.756425+0000","last_peered":"2024-02-18T00:41:05.756425+0000","last_clean":"2024-02-18T00:41:05.756425+0000","last_became_active":"2024-02-18T00:39:39.706895+0000","last_became_peered":"2024-02-18T00:39:39.706895+0000","last_unstale":"2024-02-18T00:41:05.756425+0000","last_undegraded":"2024-02-18T00:41:05.756425+0000","last_fullsized":"2024-02-18T00:41:05.756425+0000","mapping_epoch":467,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":468,"parent":"0.0","parent_split_bits":0,"last_scrub":"412'114","last_scrub_stamp":"2024-02-18T00:39:51.746713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:39:51.746713+0000","log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":94,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":39,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":97,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":457,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.761888+0000","last_change":"2024-02-18T00:40:41.127499+0000","last_active":"2024-02-18T00:41:04.761888+0000","last_peered":"2024-02-18T00:41:04.761888+0000","last_clean":"2024-02-18T00:41:04.761888+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:41:04.761888+0000","last_undegraded":"2024-02-18T00:41:04.761888+0000","last_fullsized":"2024-02-18T00:41:04.761888+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:41.127432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:41.127432+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1b","version":"519'35","reported_seq":559,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765734+0000","last_change":"2024-02-18T00:39:37.623683+0000","last_active":"2024-02-18T00:41:04.765734+0000","last_peered":"2024-02-18T00:41:04.765734+0000","last_clean":"2024-02-18T00:41:04.765734+0000","last_became_active":"2024-02-18T00:39:36.742497+0000","last_became_peered":"2024-02-18T00:39:36.742497+0000","last_unstale":"2024-02-18T00:41:04.765734+0000","last_undegraded":"2024-02-18T00:41:04.765734+0000","last_fullsized":"2024-02-18T00:41:04.765734+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"223'34","last_scrub_stamp":"2024-02-18T00:38:25.166249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.166249+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1564,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":12,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769143+0000","last_change":"2024-02-18T00:40:52.593461+0000","last_active":"2024-02-18T00:41:04.769143+0000","last_peered":"2024-02-18T00:41:04.769143+0000","last_clean":"2024-02-18T00:41:04.769143+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:41:04.769143+0000","last_undegraded":"2024-02-18T00:41:04.769143+0000","last_fullsized":"2024-02-18T00:41:04.769143+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.593401+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.1a","version":"501'107","reported_seq":668,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762804+0000","last_change":"2024-02-18T00:40:46.699779+0000","last_active":"2024-02-18T00:41:04.762804+0000","last_peered":"2024-02-18T00:41:04.762804+0000","last_clean":"2024-02-18T00:41:04.762804+0000","last_became_active":"2024-02-18T00:39:39.703671+0000","last_became_peered":"2024-02-18T00:39:39.703671+0000","last_unstale":"2024-02-18T00:41:04.762804+0000","last_undegraded":"2024-02-18T00:41:04.762804+0000","last_fullsized":"2024-02-18T00:41:04.762804+0000","mapping_epoch":467,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":468,"parent":"0.0","parent_split_bits":0,"last_scrub":"501'107","last_scrub_stamp":"2024-02-18T00:40:46.699698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.699698+0000","log_size":107,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":37,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":74,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":74,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768309+0000","last_change":"2024-02-18T00:39:36.761244+0000","last_active":"2024-02-18T00:41:04.768309+0000","last_peered":"2024-02-18T00:41:04.768309+0000","last_clean":"2024-02-18T00:41:04.768309+0000","last_became_active":"2024-02-18T00:39:36.760004+0000","last_became_peered":"2024-02-18T00:39:36.760004+0000","last_unstale":"2024-02-18T00:41:04.768309+0000","last_undegraded":"2024-02-18T00:41:04.768309+0000","last_fullsized":"2024-02-18T00:41:04.768309+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:22.316428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:39:22.316428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"62.19","version":"519'76","reported_seq":561,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762642+0000","last_change":"2024-02-18T00:40:45.683353+0000","last_active":"2024-02-18T00:41:04.762642+0000","last_peered":"2024-02-18T00:41:04.762642+0000","last_clean":"2024-02-18T00:41:04.762642+0000","last_became_active":"2024-02-18T00:33:21.199757+0000","last_became_peered":"2024-02-18T00:33:21.199757+0000","last_unstale":"2024-02-18T00:41:04.762642+0000","last_undegraded":"2024-02-18T00:41:04.762642+0000","last_fullsized":"2024-02-18T00:41:04.762642+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'75","last_scrub_stamp":"2024-02-18T00:40:45.683290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:45.683290+0000","log_size":76,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":624,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":7,"num_read_kb":0,"num_write":26,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":459,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769130+0000","last_change":"2024-02-18T00:39:02.380493+0000","last_active":"2024-02-18T00:41:04.769130+0000","last_peered":"2024-02-18T00:41:04.769130+0000","last_clean":"2024-02-18T00:41:04.769130+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:41:04.769130+0000","last_undegraded":"2024-02-18T00:41:04.769130+0000","last_fullsized":"2024-02-18T00:41:04.769130+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:39:02.380409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.380409+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"62.18","version":"500'35","reported_seq":546,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762050+0000","last_change":"2024-02-18T00:39:38.477451+0000","last_active":"2024-02-18T00:41:04.762050+0000","last_peered":"2024-02-18T00:41:04.762050+0000","last_clean":"2024-02-18T00:41:04.762050+0000","last_became_active":"2024-02-18T00:39:36.761254+0000","last_became_peered":"2024-02-18T00:39:36.761254+0000","last_unstale":"2024-02-18T00:41:04.762050+0000","last_undegraded":"2024-02-18T00:41:04.762050+0000","last_fullsized":"2024-02-18T00:41:04.762050+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"347'18","last_scrub_stamp":"2024-02-18T00:38:59.775767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.775767+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":49,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":458,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763591+0000","last_change":"2024-02-18T00:39:36.755213+0000","last_active":"2024-02-18T00:41:04.763591+0000","last_peered":"2024-02-18T00:41:04.763591+0000","last_clean":"2024-02-18T00:41:04.763591+0000","last_became_active":"2024-02-18T00:39:36.751486+0000","last_became_peered":"2024-02-18T00:39:36.751486+0000","last_unstale":"2024-02-18T00:41:04.763591+0000","last_undegraded":"2024-02-18T00:41:04.763591+0000","last_fullsized":"2024-02-18T00:41:04.763591+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:11.795514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.795514+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1f","version":"519'59","reported_seq":589,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757544+0000","last_change":"2024-02-18T00:40:01.997296+0000","last_active":"2024-02-18T00:41:04.757544+0000","last_peered":"2024-02-18T00:41:04.757544+0000","last_clean":"2024-02-18T00:41:04.757544+0000","last_became_active":"2024-02-18T00:39:36.761857+0000","last_became_peered":"2024-02-18T00:39:36.761857+0000","last_unstale":"2024-02-18T00:41:04.757544+0000","last_undegraded":"2024-02-18T00:41:04.757544+0000","last_fullsized":"2024-02-18T00:41:04.757544+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":468,"parent":"0.0","parent_split_bits":0,"last_scrub":"253'52","last_scrub_stamp":"2024-02-18T00:40:01.997230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:01.997230+0000","log_size":59,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":557,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":27,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":36,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":439,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762355+0000","last_change":"2024-02-18T00:39:20.629493+0000","last_active":"2024-02-18T00:41:04.762355+0000","last_peered":"2024-02-18T00:41:04.762355+0000","last_clean":"2024-02-18T00:41:04.762355+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:41:04.762355+0000","last_undegraded":"2024-02-18T00:41:04.762355+0000","last_fullsized":"2024-02-18T00:41:04.762355+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:20.629419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:20.629419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"62.1e","version":"231'34","reported_seq":101,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768211+0000","last_change":"2024-02-18T00:40:39.738913+0000","last_active":"2024-02-18T00:41:04.768211+0000","last_peered":"2024-02-18T00:41:04.768211+0000","last_clean":"2024-02-18T00:41:04.768211+0000","last_became_active":"2024-02-18T00:39:36.763316+0000","last_became_peered":"2024-02-18T00:39:36.763316+0000","last_unstale":"2024-02-18T00:41:04.768211+0000","last_undegraded":"2024-02-18T00:41:04.768211+0000","last_fullsized":"2024-02-18T00:41:04.768211+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"231'34","last_scrub_stamp":"2024-02-18T00:40:39.738830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.738830+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":447,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763366+0000","last_change":"2024-02-18T00:40:51.610695+0000","last_active":"2024-02-18T00:41:04.763366+0000","last_peered":"2024-02-18T00:41:04.763366+0000","last_clean":"2024-02-18T00:41:04.763366+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:41:04.763366+0000","last_undegraded":"2024-02-18T00:41:04.763366+0000","last_fullsized":"2024-02-18T00:41:04.763366+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.610630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.610630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"62.1d","version":"449'45","reported_seq":532,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760575+0000","last_change":"2024-02-18T00:38:59.766313+0000","last_active":"2024-02-18T00:41:04.760575+0000","last_peered":"2024-02-18T00:41:04.760575+0000","last_clean":"2024-02-18T00:41:04.760575+0000","last_became_active":"2024-02-18T00:33:21.131879+0000","last_became_peered":"2024-02-18T00:33:21.131879+0000","last_unstale":"2024-02-18T00:41:04.760575+0000","last_undegraded":"2024-02-18T00:41:04.760575+0000","last_fullsized":"2024-02-18T00:41:04.760575+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"431'39","last_scrub_stamp":"2024-02-18T00:38:58.470138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:38:58.470138+0000","log_size":45,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":38,"num_objects":5,"num_object_clones":1,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":15,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":534,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756424+0000","last_change":"2024-02-18T00:39:36.763536+0000","last_active":"2024-02-18T00:41:04.756424+0000","last_peered":"2024-02-18T00:41:04.756424+0000","last_clean":"2024-02-18T00:41:04.756424+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:41:04.756424+0000","last_undegraded":"2024-02-18T00:41:04.756424+0000","last_fullsized":"2024-02-18T00:41:04.756424+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:25.388161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:25.388161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"62.1c","version":"519'22","reported_seq":510,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763455+0000","last_change":"2024-02-18T00:40:09.724061+0000","last_active":"2024-02-18T00:41:04.763455+0000","last_peered":"2024-02-18T00:41:04.763455+0000","last_clean":"2024-02-18T00:41:04.763455+0000","last_became_active":"2024-02-18T00:33:21.154730+0000","last_became_peered":"2024-02-18T00:33:21.154730+0000","last_unstale":"2024-02-18T00:41:04.763455+0000","last_undegraded":"2024-02-18T00:41:04.763455+0000","last_fullsized":"2024-02-18T00:41:04.763455+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"267'18","last_scrub_stamp":"2024-02-18T00:40:09.723976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:19.952766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:09.723976+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":2571,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769082+0000","last_change":"2024-02-18T00:40:51.582600+0000","last_active":"2024-02-18T00:41:04.769082+0000","last_peered":"2024-02-18T00:41:04.769082+0000","last_clean":"2024-02-18T00:41:04.769082+0000","last_became_active":"2024-02-18T00:39:36.769282+0000","last_became_peered":"2024-02-18T00:39:36.769282+0000","last_unstale":"2024-02-18T00:41:04.769082+0000","last_undegraded":"2024-02-18T00:41:04.769082+0000","last_fullsized":"2024-02-18T00:41:04.769082+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.582542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":457,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756681+0000","last_change":"2024-02-18T00:39:36.771119+0000","last_active":"2024-02-18T00:41:04.756681+0000","last_peered":"2024-02-18T00:41:04.756681+0000","last_clean":"2024-02-18T00:41:04.756681+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:41:04.756681+0000","last_undegraded":"2024-02-18T00:41:04.756681+0000","last_fullsized":"2024-02-18T00:41:04.756681+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.691955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"466'10","reported_seq":101,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768156+0000","last_change":"2024-02-18T00:40:02.730763+0000","last_active":"2024-02-18T00:41:04.768156+0000","last_peered":"2024-02-18T00:41:04.768156+0000","last_clean":"2024-02-18T00:41:04.768156+0000","last_became_active":"2024-02-18T00:39:36.757493+0000","last_became_peered":"2024-02-18T00:39:36.757493+0000","last_unstale":"2024-02-18T00:41:04.768156+0000","last_undegraded":"2024-02-18T00:41:04.768156+0000","last_fullsized":"2024-02-18T00:41:04.768156+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:40:02.730686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:02.730686+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":631,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758576+0000","last_change":"2024-02-18T00:40:40.557614+0000","last_active":"2024-02-18T00:41:04.758576+0000","last_peered":"2024-02-18T00:41:04.758576+0000","last_clean":"2024-02-18T00:41:04.758576+0000","last_became_active":"2024-02-18T00:39:36.755699+0000","last_became_peered":"2024-02-18T00:39:36.755699+0000","last_unstale":"2024-02-18T00:41:04.758576+0000","last_undegraded":"2024-02-18T00:41:04.758576+0000","last_fullsized":"2024-02-18T00:41:04.758576+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:40:40.557544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.557544+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":527,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756810+0000","last_change":"2024-02-18T00:40:48.410752+0000","last_active":"2024-02-18T00:41:04.756810+0000","last_peered":"2024-02-18T00:41:04.756810+0000","last_clean":"2024-02-18T00:41:04.756810+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:41:04.756810+0000","last_undegraded":"2024-02-18T00:41:04.756810+0000","last_fullsized":"2024-02-18T00:41:04.756810+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:48.410676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:48.410676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758817+0000","last_change":"2024-02-18T00:40:10.634698+0000","last_active":"2024-02-18T00:41:04.758817+0000","last_peered":"2024-02-18T00:41:04.758817+0000","last_clean":"2024-02-18T00:41:04.758817+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:41:04.758817+0000","last_undegraded":"2024-02-18T00:41:04.758817+0000","last_fullsized":"2024-02-18T00:41:04.758817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:10.634595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:10.634595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":455,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.767349+0000","last_change":"2024-02-18T00:40:50.652492+0000","last_active":"2024-02-18T00:41:04.767349+0000","last_peered":"2024-02-18T00:41:04.767349+0000","last_clean":"2024-02-18T00:41:04.767349+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:41:04.767349+0000","last_undegraded":"2024-02-18T00:41:04.767349+0000","last_fullsized":"2024-02-18T00:41:04.767349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.652414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.652414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":557,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758407+0000","last_change":"2024-02-18T00:37:49.523198+0000","last_active":"2024-02-18T00:41:04.758407+0000","last_peered":"2024-02-18T00:41:04.758407+0000","last_clean":"2024-02-18T00:41:04.758407+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:41:04.758407+0000","last_undegraded":"2024-02-18T00:41:04.758407+0000","last_fullsized":"2024-02-18T00:41:04.758407+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:49.523121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:49.523121+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":513,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771288+0000","last_change":"2024-02-18T00:40:07.475798+0000","last_active":"2024-02-18T00:41:04.771288+0000","last_peered":"2024-02-18T00:41:04.771288+0000","last_clean":"2024-02-18T00:41:04.771288+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:41:04.771288+0000","last_undegraded":"2024-02-18T00:41:04.771288+0000","last_fullsized":"2024-02-18T00:41:04.771288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:07.475732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:40:07.475732+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":503,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755446+0000","last_change":"2024-02-18T00:40:46.382314+0000","last_active":"2024-02-18T00:41:04.755446+0000","last_peered":"2024-02-18T00:41:04.755446+0000","last_clean":"2024-02-18T00:41:04.755446+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:41:04.755446+0000","last_undegraded":"2024-02-18T00:41:04.755446+0000","last_fullsized":"2024-02-18T00:41:04.755446+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:46.382246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.382246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":519,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756958+0000","last_change":"2024-02-18T00:39:36.750599+0000","last_active":"2024-02-18T00:41:04.756958+0000","last_peered":"2024-02-18T00:41:04.756958+0000","last_clean":"2024-02-18T00:41:04.756958+0000","last_became_active":"2024-02-18T00:39:36.750333+0000","last_became_peered":"2024-02-18T00:39:36.750333+0000","last_unstale":"2024-02-18T00:41:04.756958+0000","last_undegraded":"2024-02-18T00:41:04.756958+0000","last_fullsized":"2024-02-18T00:41:04.756958+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:47.488821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:37:47.488821+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":454,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763938+0000","last_change":"2024-02-18T00:39:36.763384+0000","last_active":"2024-02-18T00:41:04.763938+0000","last_peered":"2024-02-18T00:41:04.763938+0000","last_clean":"2024-02-18T00:41:04.763938+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:41:04.763938+0000","last_undegraded":"2024-02-18T00:41:04.763938+0000","last_fullsized":"2024-02-18T00:41:04.763938+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.732112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.732112+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"457'42","reported_seq":638,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763511+0000","last_change":"2024-02-18T00:40:42.632457+0000","last_active":"2024-02-18T00:41:04.763511+0000","last_peered":"2024-02-18T00:41:04.763511+0000","last_clean":"2024-02-18T00:41:04.763511+0000","last_became_active":"2024-02-18T00:39:36.751206+0000","last_became_peered":"2024-02-18T00:39:36.751206+0000","last_unstale":"2024-02-18T00:41:04.763511+0000","last_undegraded":"2024-02-18T00:41:04.763511+0000","last_fullsized":"2024-02-18T00:41:04.763511+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-18T00:40:42.632357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.632357+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756592+0000","last_change":"2024-02-18T00:39:50.125802+0000","last_active":"2024-02-18T00:41:04.756592+0000","last_peered":"2024-02-18T00:41:04.756592+0000","last_clean":"2024-02-18T00:41:04.756592+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:41:04.756592+0000","last_undegraded":"2024-02-18T00:41:04.756592+0000","last_fullsized":"2024-02-18T00:41:04.756592+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:50.125735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:39:50.125735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":503,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755730+0000","last_change":"2024-02-18T00:40:51.349216+0000","last_active":"2024-02-18T00:41:04.755730+0000","last_peered":"2024-02-18T00:41:04.755730+0000","last_clean":"2024-02-18T00:41:04.755730+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:41:04.755730+0000","last_undegraded":"2024-02-18T00:41:04.755730+0000","last_fullsized":"2024-02-18T00:41:04.755730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.349134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.349134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769905+0000","last_change":"2024-02-18T00:40:08.782777+0000","last_active":"2024-02-18T00:41:04.769905+0000","last_peered":"2024-02-18T00:41:04.769905+0000","last_clean":"2024-02-18T00:41:04.769905+0000","last_became_active":"2024-02-18T00:39:36.768371+0000","last_became_peered":"2024-02-18T00:39:36.768371+0000","last_unstale":"2024-02-18T00:41:04.769905+0000","last_undegraded":"2024-02-18T00:41:04.769905+0000","last_fullsized":"2024-02-18T00:41:04.769905+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.782667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.782667+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":437,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758882+0000","last_change":"2024-02-18T00:38:59.474203+0000","last_active":"2024-02-18T00:41:04.758882+0000","last_peered":"2024-02-18T00:41:04.758882+0000","last_clean":"2024-02-18T00:41:04.758882+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:41:04.758882+0000","last_undegraded":"2024-02-18T00:41:04.758882+0000","last_fullsized":"2024-02-18T00:41:04.758882+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:59.474124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:59.474124+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"457'31","reported_seq":553,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768964+0000","last_change":"2024-02-18T00:38:14.260160+0000","last_active":"2024-02-18T00:41:04.768964+0000","last_peered":"2024-02-18T00:41:04.768964+0000","last_clean":"2024-02-18T00:41:04.768964+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:41:04.768964+0000","last_undegraded":"2024-02-18T00:41:04.768964+0000","last_fullsized":"2024-02-18T00:41:04.768964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:38:14.260073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:38:14.260073+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":74,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769613+0000","last_change":"2024-02-18T00:39:36.758410+0000","last_active":"2024-02-18T00:41:04.769613+0000","last_peered":"2024-02-18T00:41:04.769613+0000","last_clean":"2024-02-18T00:41:04.769613+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:41:04.769613+0000","last_undegraded":"2024-02-18T00:41:04.769613+0000","last_fullsized":"2024-02-18T00:41:04.769613+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:39:19.319010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":504,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769319+0000","last_change":"2024-02-18T00:39:20.450122+0000","last_active":"2024-02-18T00:41:04.769319+0000","last_peered":"2024-02-18T00:41:04.769319+0000","last_clean":"2024-02-18T00:41:04.769319+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:41:04.769319+0000","last_undegraded":"2024-02-18T00:41:04.769319+0000","last_fullsized":"2024-02-18T00:41:04.769319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:20.450046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:20.450046+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757892+0000","last_change":"2024-02-18T00:36:58.602573+0000","last_active":"2024-02-18T00:41:04.757892+0000","last_peered":"2024-02-18T00:41:04.757892+0000","last_clean":"2024-02-18T00:41:04.757892+0000","last_became_active":"2024-02-18T00:29:21.101204+0000","last_became_peered":"2024-02-18T00:29:21.101204+0000","last_unstale":"2024-02-18T00:41:04.757892+0000","last_undegraded":"2024-02-18T00:41:04.757892+0000","last_fullsized":"2024-02-18T00:41:04.757892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:36:58.602466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:36:58.602466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":295,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755432+0000","last_change":"2024-02-18T00:39:36.758179+0000","last_active":"2024-02-18T00:41:04.755432+0000","last_peered":"2024-02-18T00:41:04.755432+0000","last_clean":"2024-02-18T00:41:04.755432+0000","last_became_active":"2024-02-18T00:39:36.757501+0000","last_became_peered":"2024-02-18T00:39:36.757501+0000","last_unstale":"2024-02-18T00:41:04.755432+0000","last_undegraded":"2024-02-18T00:41:04.755432+0000","last_fullsized":"2024-02-18T00:41:04.755432+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:33.312079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.312079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":534,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768420+0000","last_change":"2024-02-18T00:39:55.199244+0000","last_active":"2024-02-18T00:41:04.768420+0000","last_peered":"2024-02-18T00:41:04.768420+0000","last_clean":"2024-02-18T00:41:04.768420+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:41:04.768420+0000","last_undegraded":"2024-02-18T00:41:04.768420+0000","last_fullsized":"2024-02-18T00:41:04.768420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:39:55.199132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:55.199132+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":531,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763138+0000","last_change":"2024-02-18T00:40:05.408718+0000","last_active":"2024-02-18T00:41:04.763138+0000","last_peered":"2024-02-18T00:41:04.763138+0000","last_clean":"2024-02-18T00:41:04.763138+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:41:04.763138+0000","last_undegraded":"2024-02-18T00:41:04.763138+0000","last_fullsized":"2024-02-18T00:41:04.763138+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:05.408648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:40:05.408648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":677,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755770+0000","last_change":"2024-02-18T00:39:38.397578+0000","last_active":"2024-02-18T00:41:04.755770+0000","last_peered":"2024-02-18T00:41:04.755770+0000","last_clean":"2024-02-18T00:41:04.755770+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:41:04.755770+0000","last_undegraded":"2024-02-18T00:41:04.755770+0000","last_fullsized":"2024-02-18T00:41:04.755770+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:37:38.433877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:38.433877+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765127+0000","last_change":"2024-02-18T00:40:39.193183+0000","last_active":"2024-02-18T00:41:04.765127+0000","last_peered":"2024-02-18T00:41:04.765127+0000","last_clean":"2024-02-18T00:41:04.765127+0000","last_became_active":"2024-02-18T00:29:21.104441+0000","last_became_peered":"2024-02-18T00:29:21.104441+0000","last_unstale":"2024-02-18T00:41:04.765127+0000","last_undegraded":"2024-02-18T00:41:04.765127+0000","last_fullsized":"2024-02-18T00:41:04.765127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:39.193082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.193082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":453,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.766622+0000","last_change":"2024-02-18T00:40:36.519540+0000","last_active":"2024-02-18T00:41:04.766622+0000","last_peered":"2024-02-18T00:41:04.766622+0000","last_clean":"2024-02-18T00:41:04.766622+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:41:04.766622+0000","last_undegraded":"2024-02-18T00:41:04.766622+0000","last_fullsized":"2024-02-18T00:41:04.766622+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.519470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.519470+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":521,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759525+0000","last_change":"2024-02-18T00:39:09.538209+0000","last_active":"2024-02-18T00:41:04.759525+0000","last_peered":"2024-02-18T00:41:04.759525+0000","last_clean":"2024-02-18T00:41:04.759525+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:41:04.759525+0000","last_undegraded":"2024-02-18T00:41:04.759525+0000","last_fullsized":"2024-02-18T00:41:04.759525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"82'22","last_scrub_stamp":"2024-02-18T00:39:09.538136+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.538136+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":540,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769212+0000","last_change":"2024-02-18T00:40:01.450474+0000","last_active":"2024-02-18T00:41:04.769212+0000","last_peered":"2024-02-18T00:41:04.769212+0000","last_clean":"2024-02-18T00:41:04.769212+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:41:04.769212+0000","last_undegraded":"2024-02-18T00:41:04.769212+0000","last_fullsized":"2024-02-18T00:41:04.769212+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:01.450328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:40:01.450328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":652,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765049+0000","last_change":"2024-02-18T00:40:38.233919+0000","last_active":"2024-02-18T00:41:04.765049+0000","last_peered":"2024-02-18T00:41:04.765049+0000","last_clean":"2024-02-18T00:41:04.765049+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:41:04.765049+0000","last_undegraded":"2024-02-18T00:41:04.765049+0000","last_fullsized":"2024-02-18T00:41:04.765049+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:40:38.233838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.233838+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":491,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.761006+0000","last_change":"2024-02-18T00:38:57.423470+0000","last_active":"2024-02-18T00:41:04.761006+0000","last_peered":"2024-02-18T00:41:04.761006+0000","last_clean":"2024-02-18T00:41:04.761006+0000","last_became_active":"2024-02-18T00:29:21.105743+0000","last_became_peered":"2024-02-18T00:29:21.105743+0000","last_unstale":"2024-02-18T00:41:04.761006+0000","last_undegraded":"2024-02-18T00:41:04.761006+0000","last_fullsized":"2024-02-18T00:41:04.761006+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:57.423405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.423405+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":441,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765867+0000","last_change":"2024-02-18T00:38:26.149235+0000","last_active":"2024-02-18T00:41:04.765867+0000","last_peered":"2024-02-18T00:41:04.765867+0000","last_clean":"2024-02-18T00:41:04.765867+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:41:04.765867+0000","last_undegraded":"2024-02-18T00:41:04.765867+0000","last_fullsized":"2024-02-18T00:41:04.765867+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:38:26.149169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'25","reported_seq":573,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764813+0000","last_change":"2024-02-18T00:39:03.341590+0000","last_active":"2024-02-18T00:41:04.764813+0000","last_peered":"2024-02-18T00:41:04.764813+0000","last_clean":"2024-02-18T00:41:04.764813+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:41:04.764813+0000","last_undegraded":"2024-02-18T00:41:04.764813+0000","last_fullsized":"2024-02-18T00:41:04.764813+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:39:03.341502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.341502+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":75,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755375+0000","last_change":"2024-02-18T00:40:52.319724+0000","last_active":"2024-02-18T00:41:04.755375+0000","last_peered":"2024-02-18T00:41:04.755375+0000","last_clean":"2024-02-18T00:41:04.755375+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:41:04.755375+0000","last_undegraded":"2024-02-18T00:41:04.755375+0000","last_fullsized":"2024-02-18T00:41:04.755375+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.319660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.319660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":498,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759859+0000","last_change":"2024-02-18T00:40:08.617435+0000","last_active":"2024-02-18T00:41:04.759859+0000","last_peered":"2024-02-18T00:41:04.759859+0000","last_clean":"2024-02-18T00:41:04.759859+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:41:04.759859+0000","last_undegraded":"2024-02-18T00:41:04.759859+0000","last_fullsized":"2024-02-18T00:41:04.759859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.617349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.617349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":495,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765206+0000","last_change":"2024-02-18T00:40:50.016843+0000","last_active":"2024-02-18T00:41:04.765206+0000","last_peered":"2024-02-18T00:41:04.765206+0000","last_clean":"2024-02-18T00:41:04.765206+0000","last_became_active":"2024-02-18T00:29:21.242636+0000","last_became_peered":"2024-02-18T00:29:21.242636+0000","last_unstale":"2024-02-18T00:41:04.765206+0000","last_undegraded":"2024-02-18T00:41:04.765206+0000","last_fullsized":"2024-02-18T00:41:04.765206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.016774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.016774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":446,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768720+0000","last_change":"2024-02-18T00:40:44.111488+0000","last_active":"2024-02-18T00:41:04.768720+0000","last_peered":"2024-02-18T00:41:04.768720+0000","last_clean":"2024-02-18T00:41:04.768720+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:41:04.768720+0000","last_undegraded":"2024-02-18T00:41:04.768720+0000","last_fullsized":"2024-02-18T00:41:04.768720+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:44.111388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:40:44.111388+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":648,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762319+0000","last_change":"2024-02-18T00:40:39.136955+0000","last_active":"2024-02-18T00:41:04.762319+0000","last_peered":"2024-02-18T00:41:04.762319+0000","last_clean":"2024-02-18T00:41:04.762319+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:41:04.762319+0000","last_undegraded":"2024-02-18T00:41:04.762319+0000","last_fullsized":"2024-02-18T00:41:04.762319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:40:39.136881+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.136881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":561,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768843+0000","last_change":"2024-02-18T00:39:11.324891+0000","last_active":"2024-02-18T00:41:04.768843+0000","last_peered":"2024-02-18T00:41:04.768843+0000","last_clean":"2024-02-18T00:41:04.768843+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:41:04.768843+0000","last_undegraded":"2024-02-18T00:41:04.768843+0000","last_fullsized":"2024-02-18T00:41:04.768843+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:39:11.324814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.324814+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":534,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762218+0000","last_change":"2024-02-18T00:40:52.642135+0000","last_active":"2024-02-18T00:41:04.762218+0000","last_peered":"2024-02-18T00:41:04.762218+0000","last_clean":"2024-02-18T00:41:04.762218+0000","last_became_active":"2024-02-18T00:39:36.752210+0000","last_became_peered":"2024-02-18T00:39:36.752210+0000","last_unstale":"2024-02-18T00:41:04.762218+0000","last_undegraded":"2024-02-18T00:41:04.762218+0000","last_fullsized":"2024-02-18T00:41:04.762218+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.642073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771492+0000","last_change":"2024-02-18T00:39:56.697241+0000","last_active":"2024-02-18T00:41:04.771492+0000","last_peered":"2024-02-18T00:41:04.771492+0000","last_clean":"2024-02-18T00:41:04.771492+0000","last_became_active":"2024-02-18T00:39:36.767278+0000","last_became_peered":"2024-02-18T00:39:36.767278+0000","last_unstale":"2024-02-18T00:41:04.771492+0000","last_undegraded":"2024-02-18T00:41:04.771492+0000","last_fullsized":"2024-02-18T00:41:04.771492+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:56.697153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:56.697153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":442,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760674+0000","last_change":"2024-02-18T00:40:45.658350+0000","last_active":"2024-02-18T00:41:04.760674+0000","last_peered":"2024-02-18T00:41:04.760674+0000","last_clean":"2024-02-18T00:41:04.760674+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:41:04.760674+0000","last_undegraded":"2024-02-18T00:41:04.760674+0000","last_fullsized":"2024-02-18T00:41:04.760674+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:45.658288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:45.658288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":520,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769019+0000","last_change":"2024-02-18T00:39:36.743164+0000","last_active":"2024-02-18T00:41:04.769019+0000","last_peered":"2024-02-18T00:41:04.769019+0000","last_clean":"2024-02-18T00:41:04.769019+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:41:04.769019+0000","last_undegraded":"2024-02-18T00:41:04.769019+0000","last_fullsized":"2024-02-18T00:41:04.769019+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:21.298549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:21.298549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":651,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762227+0000","last_change":"2024-02-18T00:40:46.010884+0000","last_active":"2024-02-18T00:41:04.762227+0000","last_peered":"2024-02-18T00:41:04.762227+0000","last_clean":"2024-02-18T00:41:04.762227+0000","last_became_active":"2024-02-18T00:39:36.766141+0000","last_became_peered":"2024-02-18T00:39:36.766141+0000","last_unstale":"2024-02-18T00:41:04.762227+0000","last_undegraded":"2024-02-18T00:41:04.762227+0000","last_fullsized":"2024-02-18T00:41:04.762227+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:40:46.010811+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.010811+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":522,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.767428+0000","last_change":"2024-02-18T00:39:22.368248+0000","last_active":"2024-02-18T00:41:04.767428+0000","last_peered":"2024-02-18T00:41:04.767428+0000","last_clean":"2024-02-18T00:41:04.767428+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:41:04.767428+0000","last_undegraded":"2024-02-18T00:41:04.767428+0000","last_fullsized":"2024-02-18T00:41:04.767428+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:22.368185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:39:22.368185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765496+0000","last_change":"2024-02-18T00:40:04.189826+0000","last_active":"2024-02-18T00:41:04.765496+0000","last_peered":"2024-02-18T00:41:04.765496+0000","last_clean":"2024-02-18T00:41:04.765496+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:41:04.765496+0000","last_undegraded":"2024-02-18T00:41:04.765496+0000","last_fullsized":"2024-02-18T00:41:04.765496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:04.189756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:04.189756+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":436,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764233+0000","last_change":"2024-02-18T00:38:28.807296+0000","last_active":"2024-02-18T00:41:04.764233+0000","last_peered":"2024-02-18T00:41:04.764233+0000","last_clean":"2024-02-18T00:41:04.764233+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:41:04.764233+0000","last_undegraded":"2024-02-18T00:41:04.764233+0000","last_fullsized":"2024-02-18T00:41:04.764233+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:28.807229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:28.807229+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":498,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763176+0000","last_change":"2024-02-18T00:39:54.964783+0000","last_active":"2024-02-18T00:41:04.763176+0000","last_peered":"2024-02-18T00:41:04.763176+0000","last_clean":"2024-02-18T00:41:04.763176+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:41:04.763176+0000","last_undegraded":"2024-02-18T00:41:04.763176+0000","last_fullsized":"2024-02-18T00:41:04.763176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.964677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.964677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":625,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758246+0000","last_change":"2024-02-18T00:40:40.195949+0000","last_active":"2024-02-18T00:41:04.758246+0000","last_peered":"2024-02-18T00:41:04.758246+0000","last_clean":"2024-02-18T00:41:04.758246+0000","last_became_active":"2024-02-18T00:39:36.750472+0000","last_became_peered":"2024-02-18T00:39:36.750472+0000","last_unstale":"2024-02-18T00:41:04.758246+0000","last_undegraded":"2024-02-18T00:41:04.758246+0000","last_fullsized":"2024-02-18T00:41:04.758246+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:40:40.195882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.195882+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":70,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755506+0000","last_change":"2024-02-18T00:39:43.723945+0000","last_active":"2024-02-18T00:41:04.755506+0000","last_peered":"2024-02-18T00:41:04.755506+0000","last_clean":"2024-02-18T00:41:04.755506+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:41:04.755506+0000","last_undegraded":"2024-02-18T00:41:04.755506+0000","last_fullsized":"2024-02-18T00:41:04.755506+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:37:01.250771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:01.250771+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769294+0000","last_change":"2024-02-18T00:40:40.614193+0000","last_active":"2024-02-18T00:41:04.769294+0000","last_peered":"2024-02-18T00:41:04.769294+0000","last_clean":"2024-02-18T00:41:04.769294+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:41:04.769294+0000","last_undegraded":"2024-02-18T00:41:04.769294+0000","last_fullsized":"2024-02-18T00:41:04.769294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:40.614069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.614069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":436,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764930+0000","last_change":"2024-02-18T00:38:57.682162+0000","last_active":"2024-02-18T00:41:04.764930+0000","last_peered":"2024-02-18T00:41:04.764930+0000","last_clean":"2024-02-18T00:41:04.764930+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:41:04.764930+0000","last_undegraded":"2024-02-18T00:41:04.764930+0000","last_fullsized":"2024-02-18T00:41:04.764930+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:57.682091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.682091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":498,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759993+0000","last_change":"2024-02-18T00:40:38.511704+0000","last_active":"2024-02-18T00:41:04.759993+0000","last_peered":"2024-02-18T00:41:04.759993+0000","last_clean":"2024-02-18T00:41:04.759993+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:41:04.759993+0000","last_undegraded":"2024-02-18T00:41:04.759993+0000","last_fullsized":"2024-02-18T00:41:04.759993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.511625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.511625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'24","reported_seq":537,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757371+0000","last_change":"2024-02-18T00:40:39.224307+0000","last_active":"2024-02-18T00:41:04.757371+0000","last_peered":"2024-02-18T00:41:04.757371+0000","last_clean":"2024-02-18T00:41:04.757371+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:41:04.757371+0000","last_undegraded":"2024-02-18T00:41:04.757371+0000","last_fullsized":"2024-02-18T00:41:04.757371+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:40:39.224211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.224211+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755801+0000","last_change":"2024-02-18T00:39:43.733207+0000","last_active":"2024-02-18T00:41:04.755801+0000","last_peered":"2024-02-18T00:41:04.755801+0000","last_clean":"2024-02-18T00:41:04.755801+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:41:04.755801+0000","last_undegraded":"2024-02-18T00:41:04.755801+0000","last_fullsized":"2024-02-18T00:41:04.755801+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:16.494804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:16.494804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756663+0000","last_change":"2024-02-18T00:40:37.156277+0000","last_active":"2024-02-18T00:41:04.756663+0000","last_peered":"2024-02-18T00:41:04.756663+0000","last_clean":"2024-02-18T00:41:04.756663+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:41:04.756663+0000","last_undegraded":"2024-02-18T00:41:04.756663+0000","last_fullsized":"2024-02-18T00:41:04.756663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:37.156213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:40:37.156213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769249+0000","last_change":"2024-02-18T00:39:54.475780+0000","last_active":"2024-02-18T00:41:04.769249+0000","last_peered":"2024-02-18T00:41:04.769249+0000","last_clean":"2024-02-18T00:41:04.769249+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:41:04.769249+0000","last_undegraded":"2024-02-18T00:41:04.769249+0000","last_fullsized":"2024-02-18T00:41:04.769249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.475703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.475703+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":457,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759837+0000","last_change":"2024-02-18T00:39:36.752309+0000","last_active":"2024-02-18T00:41:04.759837+0000","last_peered":"2024-02-18T00:41:04.759837+0000","last_clean":"2024-02-18T00:41:04.759837+0000","last_became_active":"2024-02-18T00:39:36.751747+0000","last_became_peered":"2024-02-18T00:39:36.751747+0000","last_unstale":"2024-02-18T00:41:04.759837+0000","last_undegraded":"2024-02-18T00:41:04.759837+0000","last_fullsized":"2024-02-18T00:41:04.759837+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:19.635822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:19.635822+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771438+0000","last_change":"2024-02-18T00:40:38.746495+0000","last_active":"2024-02-18T00:41:04.771438+0000","last_peered":"2024-02-18T00:41:04.771438+0000","last_clean":"2024-02-18T00:41:04.771438+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:41:04.771438+0000","last_undegraded":"2024-02-18T00:41:04.771438+0000","last_fullsized":"2024-02-18T00:41:04.771438+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.746421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.746421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":595,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764398+0000","last_change":"2024-02-18T00:39:50.936881+0000","last_active":"2024-02-18T00:41:04.764398+0000","last_peered":"2024-02-18T00:41:04.764398+0000","last_clean":"2024-02-18T00:41:04.764398+0000","last_became_active":"2024-02-18T00:39:36.751539+0000","last_became_peered":"2024-02-18T00:39:36.751539+0000","last_unstale":"2024-02-18T00:41:04.764398+0000","last_undegraded":"2024-02-18T00:41:04.764398+0000","last_fullsized":"2024-02-18T00:41:04.764398+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:39:50.936751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:50.936751+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.767943+0000","last_change":"2024-02-18T00:39:58.671410+0000","last_active":"2024-02-18T00:41:04.767943+0000","last_peered":"2024-02-18T00:41:04.767943+0000","last_clean":"2024-02-18T00:41:04.767943+0000","last_became_active":"2024-02-18T00:39:36.759224+0000","last_became_peered":"2024-02-18T00:39:36.759224+0000","last_unstale":"2024-02-18T00:41:04.767943+0000","last_undegraded":"2024-02-18T00:41:04.767943+0000","last_fullsized":"2024-02-18T00:41:04.767943+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:58.671327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:39:58.671327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":538,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759046+0000","last_change":"2024-02-18T00:40:00.515636+0000","last_active":"2024-02-18T00:41:04.759046+0000","last_peered":"2024-02-18T00:41:04.759046+0000","last_clean":"2024-02-18T00:41:04.759046+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:41:04.759046+0000","last_undegraded":"2024-02-18T00:41:04.759046+0000","last_fullsized":"2024-02-18T00:41:04.759046+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:00.515564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:00.515564+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":495,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762861+0000","last_change":"2024-02-18T00:39:02.673159+0000","last_active":"2024-02-18T00:41:04.762861+0000","last_peered":"2024-02-18T00:41:04.762861+0000","last_clean":"2024-02-18T00:41:04.762861+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:41:04.762861+0000","last_undegraded":"2024-02-18T00:41:04.762861+0000","last_fullsized":"2024-02-18T00:41:04.762861+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.673088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.673088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760069+0000","last_change":"2024-02-18T00:39:56.621857+0000","last_active":"2024-02-18T00:41:04.760069+0000","last_peered":"2024-02-18T00:41:04.760069+0000","last_clean":"2024-02-18T00:41:04.760069+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:41:04.760069+0000","last_undegraded":"2024-02-18T00:41:04.760069+0000","last_fullsized":"2024-02-18T00:41:04.760069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:56.621792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:39:56.621792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":439,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757805+0000","last_change":"2024-02-18T00:38:25.649197+0000","last_active":"2024-02-18T00:41:04.757805+0000","last_peered":"2024-02-18T00:41:04.757805+0000","last_clean":"2024-02-18T00:41:04.757805+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:41:04.757805+0000","last_undegraded":"2024-02-18T00:41:04.757805+0000","last_fullsized":"2024-02-18T00:41:04.757805+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:25.649119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:25.649119+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":521,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756921+0000","last_change":"2024-02-18T00:40:34.340695+0000","last_active":"2024-02-18T00:41:04.756921+0000","last_peered":"2024-02-18T00:41:04.756921+0000","last_clean":"2024-02-18T00:41:04.756921+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:41:04.756921+0000","last_undegraded":"2024-02-18T00:41:04.756921+0000","last_fullsized":"2024-02-18T00:41:04.756921+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.340624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.340624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":559,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.770593+0000","last_change":"2024-02-18T00:40:11.447344+0000","last_active":"2024-02-18T00:41:04.770593+0000","last_peered":"2024-02-18T00:41:04.770593+0000","last_clean":"2024-02-18T00:41:04.770593+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:41:04.770593+0000","last_undegraded":"2024-02-18T00:41:04.770593+0000","last_fullsized":"2024-02-18T00:41:04.770593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:40:11.447198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:11.447198+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":516,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756107+0000","last_change":"2024-02-18T00:40:06.037002+0000","last_active":"2024-02-18T00:41:04.756107+0000","last_peered":"2024-02-18T00:41:04.756107+0000","last_clean":"2024-02-18T00:41:04.756107+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:41:04.756107+0000","last_undegraded":"2024-02-18T00:41:04.756107+0000","last_fullsized":"2024-02-18T00:41:04.756107+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:06.036932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.036932+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1217,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:05.157383+0000","last_change":"2024-02-18T00:40:02.439178+0000","last_active":"2024-02-18T00:41:05.157383+0000","last_peered":"2024-02-18T00:41:05.157383+0000","last_clean":"2024-02-18T00:41:05.157383+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:41:05.157383+0000","last_undegraded":"2024-02-18T00:41:05.157383+0000","last_fullsized":"2024-02-18T00:41:05.157383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:40:02.439080+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:40:02.439080+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":701,"num_read_kb":696,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":78,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.770054+0000","last_change":"2024-02-18T00:39:36.775123+0000","last_active":"2024-02-18T00:41:04.770054+0000","last_peered":"2024-02-18T00:41:04.770054+0000","last_clean":"2024-02-18T00:41:04.770054+0000","last_became_active":"2024-02-18T00:39:36.764502+0000","last_became_peered":"2024-02-18T00:39:36.764502+0000","last_unstale":"2024-02-18T00:41:04.770054+0000","last_undegraded":"2024-02-18T00:41:04.770054+0000","last_fullsized":"2024-02-18T00:41:04.770054+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:39:24.877392+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:39:24.877392+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":459,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769062+0000","last_change":"2024-02-18T00:39:36.742786+0000","last_active":"2024-02-18T00:41:04.769062+0000","last_peered":"2024-02-18T00:41:04.769062+0000","last_clean":"2024-02-18T00:41:04.769062+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:41:04.769062+0000","last_undegraded":"2024-02-18T00:41:04.769062+0000","last_fullsized":"2024-02-18T00:41:04.769062+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_clean_scrub_stamp":"2024-02-18T00:38:29.090653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":499,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756994+0000","last_change":"2024-02-18T00:40:34.247010+0000","last_active":"2024-02-18T00:41:04.756994+0000","last_peered":"2024-02-18T00:41:04.756994+0000","last_clean":"2024-02-18T00:41:04.756994+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:41:04.756994+0000","last_undegraded":"2024-02-18T00:41:04.756994+0000","last_fullsized":"2024-02-18T00:41:04.756994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.246941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.246941+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'47","reported_seq":388,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758976+0000","last_change":"2024-02-18T00:39:57.582291+0000","last_active":"2024-02-18T00:41:04.758976+0000","last_peered":"2024-02-18T00:41:04.758976+0000","last_clean":"2024-02-18T00:41:04.758976+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:41:04.758976+0000","last_undegraded":"2024-02-18T00:41:04.758976+0000","last_fullsized":"2024-02-18T00:41:04.758976+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:39:57.582210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:57.582210+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":511,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758963+0000","last_change":"2024-02-18T00:40:03.521856+0000","last_active":"2024-02-18T00:41:04.758963+0000","last_peered":"2024-02-18T00:41:04.758963+0000","last_clean":"2024-02-18T00:41:04.758963+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:41:04.758963+0000","last_undegraded":"2024-02-18T00:41:04.758963+0000","last_fullsized":"2024-02-18T00:41:04.758963+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:03.521798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.521798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":508,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763760+0000","last_change":"2024-02-18T00:39:36.761763+0000","last_active":"2024-02-18T00:41:04.763760+0000","last_peered":"2024-02-18T00:41:04.763760+0000","last_clean":"2024-02-18T00:41:04.763760+0000","last_became_active":"2024-02-18T00:39:36.761490+0000","last_became_peered":"2024-02-18T00:39:36.761490+0000","last_unstale":"2024-02-18T00:41:04.763760+0000","last_undegraded":"2024-02-18T00:41:04.763760+0000","last_fullsized":"2024-02-18T00:41:04.763760+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.570919+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":439,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756017+0000","last_change":"2024-02-18T00:39:02.635218+0000","last_active":"2024-02-18T00:41:04.756017+0000","last_peered":"2024-02-18T00:41:04.756017+0000","last_clean":"2024-02-18T00:41:04.756017+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:41:04.756017+0000","last_undegraded":"2024-02-18T00:41:04.756017+0000","last_fullsized":"2024-02-18T00:41:04.756017+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.635104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.635104+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":644,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758507+0000","last_change":"2024-02-18T00:37:29.141966+0000","last_active":"2024-02-18T00:41:04.758507+0000","last_peered":"2024-02-18T00:41:04.758507+0000","last_clean":"2024-02-18T00:41:04.758507+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:41:04.758507+0000","last_undegraded":"2024-02-18T00:41:04.758507+0000","last_fullsized":"2024-02-18T00:41:04.758507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:37:29.141901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:37:29.141901+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":515,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755676+0000","last_change":"2024-02-18T00:37:43.309558+0000","last_active":"2024-02-18T00:41:04.755676+0000","last_peered":"2024-02-18T00:41:04.755676+0000","last_clean":"2024-02-18T00:41:04.755676+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:41:04.755676+0000","last_undegraded":"2024-02-18T00:41:04.755676+0000","last_fullsized":"2024-02-18T00:41:04.755676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'22","last_scrub_stamp":"2024-02-18T00:37:43.309488+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.309488+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":545,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769142+0000","last_change":"2024-02-18T00:40:38.550394+0000","last_active":"2024-02-18T00:41:04.769142+0000","last_peered":"2024-02-18T00:41:04.769142+0000","last_clean":"2024-02-18T00:41:04.769142+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:41:04.769142+0000","last_undegraded":"2024-02-18T00:41:04.769142+0000","last_fullsized":"2024-02-18T00:41:04.769142+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.550323+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":492,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759676+0000","last_change":"2024-02-18T00:39:49.207377+0000","last_active":"2024-02-18T00:41:04.759676+0000","last_peered":"2024-02-18T00:41:04.759676+0000","last_clean":"2024-02-18T00:41:04.759676+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:41:04.759676+0000","last_undegraded":"2024-02-18T00:41:04.759676+0000","last_fullsized":"2024-02-18T00:41:04.759676+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:49.207314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:49.207314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":441,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768636+0000","last_change":"2024-02-18T00:38:24.128263+0000","last_active":"2024-02-18T00:41:04.768636+0000","last_peered":"2024-02-18T00:41:04.768636+0000","last_clean":"2024-02-18T00:41:04.768636+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:41:04.768636+0000","last_undegraded":"2024-02-18T00:41:04.768636+0000","last_fullsized":"2024-02-18T00:41:04.768636+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.128170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.128170+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":498,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756236+0000","last_change":"2024-02-18T00:37:44.335458+0000","last_active":"2024-02-18T00:41:04.756236+0000","last_peered":"2024-02-18T00:41:04.756236+0000","last_clean":"2024-02-18T00:41:04.756236+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:41:04.756236+0000","last_undegraded":"2024-02-18T00:41:04.756236+0000","last_fullsized":"2024-02-18T00:41:04.756236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:37:44.335390+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":566,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768906+0000","last_change":"2024-02-18T00:40:33.214440+0000","last_active":"2024-02-18T00:41:04.768906+0000","last_peered":"2024-02-18T00:41:04.768906+0000","last_clean":"2024-02-18T00:41:04.768906+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:41:04.768906+0000","last_undegraded":"2024-02-18T00:41:04.768906+0000","last_fullsized":"2024-02-18T00:41:04.768906+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:40:33.214358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:33.214358+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":554,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764530+0000","last_change":"2024-02-18T00:40:40.158702+0000","last_active":"2024-02-18T00:41:04.764530+0000","last_peered":"2024-02-18T00:41:04.764530+0000","last_clean":"2024-02-18T00:41:04.764530+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:41:04.764530+0000","last_undegraded":"2024-02-18T00:41:04.764530+0000","last_fullsized":"2024-02-18T00:41:04.764530+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:40:40.158593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.158593+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":513,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764711+0000","last_change":"2024-02-18T00:40:03.980698+0000","last_active":"2024-02-18T00:41:04.764711+0000","last_peered":"2024-02-18T00:41:04.764711+0000","last_clean":"2024-02-18T00:41:04.764711+0000","last_became_active":"2024-02-18T00:39:36.762201+0000","last_became_peered":"2024-02-18T00:39:36.762201+0000","last_unstale":"2024-02-18T00:41:04.764711+0000","last_undegraded":"2024-02-18T00:41:04.764711+0000","last_fullsized":"2024-02-18T00:41:04.764711+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:03.980632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.980632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":441,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765940+0000","last_change":"2024-02-18T00:38:18.292304+0000","last_active":"2024-02-18T00:41:04.765940+0000","last_peered":"2024-02-18T00:41:04.765940+0000","last_clean":"2024-02-18T00:41:04.765940+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:41:04.765940+0000","last_undegraded":"2024-02-18T00:41:04.765940+0000","last_fullsized":"2024-02-18T00:41:04.765940+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:18.292215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:18.292215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":602,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757418+0000","last_change":"2024-02-18T00:40:51.299875+0000","last_active":"2024-02-18T00:41:04.757418+0000","last_peered":"2024-02-18T00:41:04.757418+0000","last_clean":"2024-02-18T00:41:04.757418+0000","last_became_active":"2024-02-18T00:39:36.771394+0000","last_became_peered":"2024-02-18T00:39:36.771394+0000","last_unstale":"2024-02-18T00:41:04.757418+0000","last_undegraded":"2024-02-18T00:41:04.757418+0000","last_fullsized":"2024-02-18T00:41:04.757418+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:40:51.299791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.299791+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756139+0000","last_change":"2024-02-18T00:37:27.357689+0000","last_active":"2024-02-18T00:41:04.756139+0000","last_peered":"2024-02-18T00:41:04.756139+0000","last_clean":"2024-02-18T00:41:04.756139+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:41:04.756139+0000","last_undegraded":"2024-02-18T00:41:04.756139+0000","last_fullsized":"2024-02-18T00:41:04.756139+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:27.357610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:37:27.357610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":515,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763792+0000","last_change":"2024-02-18T00:40:33.083220+0000","last_active":"2024-02-18T00:41:04.763792+0000","last_peered":"2024-02-18T00:41:04.763792+0000","last_clean":"2024-02-18T00:41:04.763792+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:41:04.763792+0000","last_undegraded":"2024-02-18T00:41:04.763792+0000","last_fullsized":"2024-02-18T00:41:04.763792+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:33.083145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:33.083145+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":516,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756353+0000","last_change":"2024-02-18T00:39:36.758086+0000","last_active":"2024-02-18T00:41:04.756353+0000","last_peered":"2024-02-18T00:41:04.756353+0000","last_clean":"2024-02-18T00:41:04.756353+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:41:04.756353+0000","last_undegraded":"2024-02-18T00:41:04.756353+0000","last_fullsized":"2024-02-18T00:41:04.756353+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:18.211326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:38:18.211326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":439,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757754+0000","last_change":"2024-02-18T00:39:09.589674+0000","last_active":"2024-02-18T00:41:04.757754+0000","last_peered":"2024-02-18T00:41:04.757754+0000","last_clean":"2024-02-18T00:41:04.757754+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:41:04.757754+0000","last_undegraded":"2024-02-18T00:41:04.757754+0000","last_fullsized":"2024-02-18T00:41:04.757754+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:09.589601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.589601+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":589,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.761270+0000","last_change":"2024-02-18T00:39:38.409643+0000","last_active":"2024-02-18T00:41:04.761270+0000","last_peered":"2024-02-18T00:41:04.761270+0000","last_clean":"2024-02-18T00:41:04.761270+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:41:04.761270+0000","last_undegraded":"2024-02-18T00:41:04.761270+0000","last_fullsized":"2024-02-18T00:41:04.761270+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:39:07.653633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.653633+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762823+0000","last_change":"2024-02-18T00:40:38.674457+0000","last_active":"2024-02-18T00:41:04.762823+0000","last_peered":"2024-02-18T00:41:04.762823+0000","last_clean":"2024-02-18T00:41:04.762823+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:41:04.762823+0000","last_undegraded":"2024-02-18T00:41:04.762823+0000","last_fullsized":"2024-02-18T00:41:04.762823+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.674378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.674378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":498,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760445+0000","last_change":"2024-02-18T00:40:04.499875+0000","last_active":"2024-02-18T00:41:04.760445+0000","last_peered":"2024-02-18T00:41:04.760445+0000","last_clean":"2024-02-18T00:41:04.760445+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:41:04.760445+0000","last_undegraded":"2024-02-18T00:41:04.760445+0000","last_fullsized":"2024-02-18T00:41:04.760445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:04.499796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:04.499796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":74,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769792+0000","last_change":"2024-02-18T00:39:36.762287+0000","last_active":"2024-02-18T00:41:04.769792+0000","last_peered":"2024-02-18T00:41:04.769792+0000","last_clean":"2024-02-18T00:41:04.769792+0000","last_became_active":"2024-02-18T00:39:36.759264+0000","last_became_peered":"2024-02-18T00:39:36.759264+0000","last_unstale":"2024-02-18T00:41:04.769792+0000","last_undegraded":"2024-02-18T00:41:04.769792+0000","last_fullsized":"2024-02-18T00:41:04.769792+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:00.786470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:00.786470+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":451,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756369+0000","last_change":"2024-02-18T00:38:20.206553+0000","last_active":"2024-02-18T00:41:04.756369+0000","last_peered":"2024-02-18T00:41:04.756369+0000","last_clean":"2024-02-18T00:41:04.756369+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:41:04.756369+0000","last_undegraded":"2024-02-18T00:41:04.756369+0000","last_fullsized":"2024-02-18T00:41:04.756369+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.206476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.206476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":609,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765464+0000","last_change":"2024-02-18T00:40:42.124689+0000","last_active":"2024-02-18T00:41:04.765464+0000","last_peered":"2024-02-18T00:41:04.765464+0000","last_clean":"2024-02-18T00:41:04.765464+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:41:04.765464+0000","last_undegraded":"2024-02-18T00:41:04.765464+0000","last_fullsized":"2024-02-18T00:41:04.765464+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:40:42.124619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.124619+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":529,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758929+0000","last_change":"2024-02-18T00:40:36.427315+0000","last_active":"2024-02-18T00:41:04.758929+0000","last_peered":"2024-02-18T00:41:04.758929+0000","last_clean":"2024-02-18T00:41:04.758929+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:41:04.758929+0000","last_undegraded":"2024-02-18T00:41:04.758929+0000","last_fullsized":"2024-02-18T00:41:04.758929+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.427206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.427206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":649,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760878+0000","last_change":"2024-02-18T00:40:09.668977+0000","last_active":"2024-02-18T00:41:04.760878+0000","last_peered":"2024-02-18T00:41:04.760878+0000","last_clean":"2024-02-18T00:41:04.760878+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:41:04.760878+0000","last_undegraded":"2024-02-18T00:41:04.760878+0000","last_fullsized":"2024-02-18T00:41:04.760878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:40:09.668906+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:40:09.668906+0000","last_clean_scrub_stamp":"2024-02-18T00:40:09.668906+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":515,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757327+0000","last_change":"2024-02-18T00:39:36.772082+0000","last_active":"2024-02-18T00:41:04.757327+0000","last_peered":"2024-02-18T00:41:04.757327+0000","last_clean":"2024-02-18T00:41:04.757327+0000","last_became_active":"2024-02-18T00:39:36.771683+0000","last_became_peered":"2024-02-18T00:39:36.771683+0000","last_unstale":"2024-02-18T00:41:04.757327+0000","last_undegraded":"2024-02-18T00:41:04.757327+0000","last_fullsized":"2024-02-18T00:41:04.757327+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:14.633705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.633705+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.633705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":460,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762561+0000","last_change":"2024-02-18T00:40:06.822506+0000","last_active":"2024-02-18T00:41:04.762561+0000","last_peered":"2024-02-18T00:41:04.762561+0000","last_clean":"2024-02-18T00:41:04.762561+0000","last_became_active":"2024-02-18T00:39:36.751162+0000","last_became_peered":"2024-02-18T00:39:36.751162+0000","last_unstale":"2024-02-18T00:41:04.762561+0000","last_undegraded":"2024-02-18T00:41:04.762561+0000","last_fullsized":"2024-02-18T00:41:04.762561+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:06.822430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.822430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":582,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760756+0000","last_change":"2024-02-18T00:40:43.621296+0000","last_active":"2024-02-18T00:41:04.760756+0000","last_peered":"2024-02-18T00:41:04.760756+0000","last_clean":"2024-02-18T00:41:04.760756+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:41:04.760756+0000","last_undegraded":"2024-02-18T00:41:04.760756+0000","last_fullsized":"2024-02-18T00:41:04.760756+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:40:43.621226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:43.621226+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":503,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758325+0000","last_change":"2024-02-18T00:40:36.184282+0000","last_active":"2024-02-18T00:41:04.758325+0000","last_peered":"2024-02-18T00:41:04.758325+0000","last_clean":"2024-02-18T00:41:04.758325+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:41:04.758325+0000","last_undegraded":"2024-02-18T00:41:04.758325+0000","last_fullsized":"2024-02-18T00:41:04.758325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.184216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.184216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":674,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765694+0000","last_change":"2024-02-18T00:40:06.960621+0000","last_active":"2024-02-18T00:41:04.765694+0000","last_peered":"2024-02-18T00:41:04.765694+0000","last_clean":"2024-02-18T00:41:04.765694+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:41:04.765694+0000","last_undegraded":"2024-02-18T00:41:04.765694+0000","last_fullsized":"2024-02-18T00:41:04.765694+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.960509+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.767664+0000","last_change":"2024-02-18T00:40:51.157676+0000","last_active":"2024-02-18T00:41:04.767664+0000","last_peered":"2024-02-18T00:41:04.767664+0000","last_clean":"2024-02-18T00:41:04.767664+0000","last_became_active":"2024-02-18T00:29:21.095593+0000","last_became_peered":"2024-02-18T00:29:21.095593+0000","last_unstale":"2024-02-18T00:41:04.767664+0000","last_undegraded":"2024-02-18T00:41:04.767664+0000","last_fullsized":"2024-02-18T00:41:04.767664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.157612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.157612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":442,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760144+0000","last_change":"2024-02-18T00:40:06.573220+0000","last_active":"2024-02-18T00:41:04.760144+0000","last_peered":"2024-02-18T00:41:04.760144+0000","last_clean":"2024-02-18T00:41:04.760144+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:41:04.760144+0000","last_undegraded":"2024-02-18T00:41:04.760144+0000","last_fullsized":"2024-02-18T00:41:04.760144+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.573143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":536,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768071+0000","last_change":"2024-02-18T00:40:37.554739+0000","last_active":"2024-02-18T00:41:04.768071+0000","last_peered":"2024-02-18T00:41:04.768071+0000","last_clean":"2024-02-18T00:41:04.768071+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:41:04.768071+0000","last_undegraded":"2024-02-18T00:41:04.768071+0000","last_fullsized":"2024-02-18T00:41:04.768071+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:40:37.554662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:37.554662+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763755+0000","last_change":"2024-02-18T00:39:53.940625+0000","last_active":"2024-02-18T00:41:04.763755+0000","last_peered":"2024-02-18T00:41:04.763755+0000","last_clean":"2024-02-18T00:41:04.763755+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:41:04.763755+0000","last_undegraded":"2024-02-18T00:41:04.763755+0000","last_fullsized":"2024-02-18T00:41:04.763755+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:53.940533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:53.940533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":497,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764626+0000","last_change":"2024-02-18T00:40:08.961647+0000","last_active":"2024-02-18T00:41:04.764626+0000","last_peered":"2024-02-18T00:41:04.764626+0000","last_clean":"2024-02-18T00:41:04.764626+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:41:04.764626+0000","last_undegraded":"2024-02-18T00:41:04.764626+0000","last_fullsized":"2024-02-18T00:41:04.764626+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.961546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.961546+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":497,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759643+0000","last_change":"2024-02-18T00:39:01.603631+0000","last_active":"2024-02-18T00:41:04.759643+0000","last_peered":"2024-02-18T00:41:04.759643+0000","last_clean":"2024-02-18T00:41:04.759643+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:41:04.759643+0000","last_undegraded":"2024-02-18T00:41:04.759643+0000","last_fullsized":"2024-02-18T00:41:04.759643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:01.603551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.603551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":439,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757284+0000","last_change":"2024-02-18T00:40:07.241181+0000","last_active":"2024-02-18T00:41:04.757284+0000","last_peered":"2024-02-18T00:41:04.757284+0000","last_clean":"2024-02-18T00:41:04.757284+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:41:04.757284+0000","last_undegraded":"2024-02-18T00:41:04.757284+0000","last_fullsized":"2024-02-18T00:41:04.757284+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_clean_scrub_stamp":"2024-02-18T00:40:07.241102+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":535,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.766192+0000","last_change":"2024-02-18T00:39:50.310946+0000","last_active":"2024-02-18T00:41:04.766192+0000","last_peered":"2024-02-18T00:41:04.766192+0000","last_clean":"2024-02-18T00:41:04.766192+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:41:04.766192+0000","last_undegraded":"2024-02-18T00:41:04.766192+0000","last_fullsized":"2024-02-18T00:41:04.766192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:39:50.310868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:50.310868+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":508,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771243+0000","last_change":"2024-02-18T00:39:21.416335+0000","last_active":"2024-02-18T00:41:04.771243+0000","last_peered":"2024-02-18T00:41:04.771243+0000","last_clean":"2024-02-18T00:41:04.771243+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:41:04.771243+0000","last_undegraded":"2024-02-18T00:41:04.771243+0000","last_fullsized":"2024-02-18T00:41:04.771243+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:21.416250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:21.416250+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":509,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.766468+0000","last_change":"2024-02-18T00:40:09.454111+0000","last_active":"2024-02-18T00:41:04.766468+0000","last_peered":"2024-02-18T00:41:04.766468+0000","last_clean":"2024-02-18T00:41:04.766468+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:41:04.766468+0000","last_undegraded":"2024-02-18T00:41:04.766468+0000","last_fullsized":"2024-02-18T00:41:04.766468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:09.454047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:09.454047+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":495,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762463+0000","last_change":"2024-02-18T00:40:52.018692+0000","last_active":"2024-02-18T00:41:04.762463+0000","last_peered":"2024-02-18T00:41:04.762463+0000","last_clean":"2024-02-18T00:41:04.762463+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:41:04.762463+0000","last_undegraded":"2024-02-18T00:41:04.762463+0000","last_fullsized":"2024-02-18T00:41:04.762463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.018627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.018627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":442,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764478+0000","last_change":"2024-02-18T00:40:34.828210+0000","last_active":"2024-02-18T00:41:04.764478+0000","last_peered":"2024-02-18T00:41:04.764478+0000","last_clean":"2024-02-18T00:41:04.764478+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:41:04.764478+0000","last_undegraded":"2024-02-18T00:41:04.764478+0000","last_fullsized":"2024-02-18T00:41:04.764478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.828148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.828148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":511,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764048+0000","last_change":"2024-02-18T00:40:43.113198+0000","last_active":"2024-02-18T00:41:04.764048+0000","last_peered":"2024-02-18T00:41:04.764048+0000","last_clean":"2024-02-18T00:41:04.764048+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:41:04.764048+0000","last_undegraded":"2024-02-18T00:41:04.764048+0000","last_fullsized":"2024-02-18T00:41:04.764048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-18T00:40:43.113109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:43.113109+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.760515+0000","last_change":"2024-02-18T00:40:50.609050+0000","last_active":"2024-02-18T00:41:04.760515+0000","last_peered":"2024-02-18T00:41:04.760515+0000","last_clean":"2024-02-18T00:41:04.760515+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:41:04.760515+0000","last_undegraded":"2024-02-18T00:41:04.760515+0000","last_fullsized":"2024-02-18T00:41:04.760515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.608983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.608983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":516,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762792+0000","last_change":"2024-02-18T00:39:36.752952+0000","last_active":"2024-02-18T00:41:04.762792+0000","last_peered":"2024-02-18T00:41:04.762792+0000","last_clean":"2024-02-18T00:41:04.762792+0000","last_became_active":"2024-02-18T00:39:36.750866+0000","last_became_peered":"2024-02-18T00:39:36.750866+0000","last_unstale":"2024-02-18T00:41:04.762792+0000","last_undegraded":"2024-02-18T00:41:04.762792+0000","last_fullsized":"2024-02-18T00:41:04.762792+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:09.867259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.867259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771250+0000","last_change":"2024-02-18T00:40:36.683678+0000","last_active":"2024-02-18T00:41:04.771250+0000","last_peered":"2024-02-18T00:41:04.771250+0000","last_clean":"2024-02-18T00:41:04.771250+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:41:04.771250+0000","last_undegraded":"2024-02-18T00:41:04.771250+0000","last_fullsized":"2024-02-18T00:41:04.771250+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.683604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.683604+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":455,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758573+0000","last_change":"2024-02-18T00:39:36.755426+0000","last_active":"2024-02-18T00:41:04.758573+0000","last_peered":"2024-02-18T00:41:04.758573+0000","last_clean":"2024-02-18T00:41:04.758573+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:41:04.758573+0000","last_undegraded":"2024-02-18T00:41:04.758573+0000","last_fullsized":"2024-02-18T00:41:04.758573+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:01.514191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.514191+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":586,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763007+0000","last_change":"2024-02-18T00:39:38.432345+0000","last_active":"2024-02-18T00:41:04.763007+0000","last_peered":"2024-02-18T00:41:04.763007+0000","last_clean":"2024-02-18T00:41:04.763007+0000","last_became_active":"2024-02-18T00:39:36.761140+0000","last_became_peered":"2024-02-18T00:39:36.761140+0000","last_unstale":"2024-02-18T00:41:04.763007+0000","last_undegraded":"2024-02-18T00:41:04.763007+0000","last_fullsized":"2024-02-18T00:41:04.763007+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:37:45.838523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.838523+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764927+0000","last_change":"2024-02-18T00:39:06.776476+0000","last_active":"2024-02-18T00:41:04.764927+0000","last_peered":"2024-02-18T00:41:04.764927+0000","last_clean":"2024-02-18T00:41:04.764927+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:41:04.764927+0000","last_undegraded":"2024-02-18T00:41:04.764927+0000","last_fullsized":"2024-02-18T00:41:04.764927+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:06.776398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.776398+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":74,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.767906+0000","last_change":"2024-02-18T00:39:36.758700+0000","last_active":"2024-02-18T00:41:04.767906+0000","last_peered":"2024-02-18T00:41:04.767906+0000","last_clean":"2024-02-18T00:41:04.767906+0000","last_became_active":"2024-02-18T00:39:36.758138+0000","last_became_peered":"2024-02-18T00:39:36.758138+0000","last_unstale":"2024-02-18T00:41:04.767906+0000","last_undegraded":"2024-02-18T00:41:04.767906+0000","last_fullsized":"2024-02-18T00:41:04.767906+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:16.384118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:16.384118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":495,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764794+0000","last_change":"2024-02-18T00:40:51.008109+0000","last_active":"2024-02-18T00:41:04.764794+0000","last_peered":"2024-02-18T00:41:04.764794+0000","last_clean":"2024-02-18T00:41:04.764794+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:41:04.764794+0000","last_undegraded":"2024-02-18T00:41:04.764794+0000","last_fullsized":"2024-02-18T00:41:04.764794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.008036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.008036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.772806+0000","last_change":"2024-02-18T00:40:40.756033+0000","last_active":"2024-02-18T00:41:04.772806+0000","last_peered":"2024-02-18T00:41:04.772806+0000","last_clean":"2024-02-18T00:41:04.772806+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:41:04.772806+0000","last_undegraded":"2024-02-18T00:41:04.772806+0000","last_fullsized":"2024-02-18T00:41:04.772806+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:40.755973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.755973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":598,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757530+0000","last_change":"2024-02-18T00:39:38.426514+0000","last_active":"2024-02-18T00:41:04.757530+0000","last_peered":"2024-02-18T00:41:04.757530+0000","last_clean":"2024-02-18T00:41:04.757530+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:41:04.757530+0000","last_undegraded":"2024-02-18T00:41:04.757530+0000","last_fullsized":"2024-02-18T00:41:04.757530+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:39:11.643593+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.643593+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":524,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758633+0000","last_change":"2024-02-18T00:39:43.717101+0000","last_active":"2024-02-18T00:41:04.758633+0000","last_peered":"2024-02-18T00:41:04.758633+0000","last_clean":"2024-02-18T00:41:04.758633+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:41:04.758633+0000","last_undegraded":"2024-02-18T00:41:04.758633+0000","last_fullsized":"2024-02-18T00:41:04.758633+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:06.562567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.562567+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"466'10","reported_seq":682,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765024+0000","last_change":"2024-02-18T00:39:38.399472+0000","last_active":"2024-02-18T00:41:04.765024+0000","last_peered":"2024-02-18T00:41:04.765024+0000","last_clean":"2024-02-18T00:41:04.765024+0000","last_became_active":"2024-02-18T00:39:36.751716+0000","last_became_peered":"2024-02-18T00:39:36.751716+0000","last_unstale":"2024-02-18T00:41:04.765024+0000","last_undegraded":"2024-02-18T00:41:04.765024+0000","last_fullsized":"2024-02-18T00:41:04.765024+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"150'9","last_scrub_stamp":"2024-02-18T00:39:17.834141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:17.834141+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"64'1","reported_seq":83,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771573+0000","last_change":"2024-02-18T00:39:53.688040+0000","last_active":"2024-02-18T00:41:04.771573+0000","last_peered":"2024-02-18T00:41:04.771573+0000","last_clean":"2024-02-18T00:41:04.771573+0000","last_became_active":"2024-02-18T00:39:36.763793+0000","last_became_peered":"2024-02-18T00:39:36.763793+0000","last_unstale":"2024-02-18T00:41:04.771573+0000","last_undegraded":"2024-02-18T00:41:04.771573+0000","last_fullsized":"2024-02-18T00:41:04.771573+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:39:53.687964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:53.687964+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.769078+0000","last_change":"2024-02-18T00:40:42.799161+0000","last_active":"2024-02-18T00:41:04.769078+0000","last_peered":"2024-02-18T00:41:04.769078+0000","last_clean":"2024-02-18T00:41:04.769078+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:41:04.769078+0000","last_undegraded":"2024-02-18T00:41:04.769078+0000","last_fullsized":"2024-02-18T00:41:04.769078+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:42.799091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.799091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":546,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765119+0000","last_change":"2024-02-18T00:39:10.827687+0000","last_active":"2024-02-18T00:41:04.765119+0000","last_peered":"2024-02-18T00:41:04.765119+0000","last_clean":"2024-02-18T00:41:04.765119+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:41:04.765119+0000","last_undegraded":"2024-02-18T00:41:04.765119+0000","last_fullsized":"2024-02-18T00:41:04.765119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'17","last_scrub_stamp":"2024-02-18T00:39:10.827603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:10.827603+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":501,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762692+0000","last_change":"2024-02-18T00:39:54.855516+0000","last_active":"2024-02-18T00:41:04.762692+0000","last_peered":"2024-02-18T00:41:04.762692+0000","last_clean":"2024-02-18T00:41:04.762692+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:41:04.762692+0000","last_undegraded":"2024-02-18T00:41:04.762692+0000","last_fullsized":"2024-02-18T00:41:04.762692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.855452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.855452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":512,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.757617+0000","last_change":"2024-02-18T00:39:48.227178+0000","last_active":"2024-02-18T00:41:04.757617+0000","last_peered":"2024-02-18T00:41:04.757617+0000","last_clean":"2024-02-18T00:41:04.757617+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:41:04.757617+0000","last_undegraded":"2024-02-18T00:41:04.757617+0000","last_fullsized":"2024-02-18T00:41:04.757617+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:48.227102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:48.227102+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":507,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771169+0000","last_change":"2024-02-18T00:39:57.390338+0000","last_active":"2024-02-18T00:41:04.771169+0000","last_peered":"2024-02-18T00:41:04.771169+0000","last_clean":"2024-02-18T00:41:04.771169+0000","last_became_active":"2024-02-18T00:29:21.105127+0000","last_became_peered":"2024-02-18T00:29:21.105127+0000","last_unstale":"2024-02-18T00:41:04.771169+0000","last_undegraded":"2024-02-18T00:41:04.771169+0000","last_fullsized":"2024-02-18T00:41:04.771169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:57.390269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:59.800355+0000","last_clean_scrub_stamp":"2024-02-18T00:39:57.390269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":455,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756283+0000","last_change":"2024-02-18T00:37:48.395278+0000","last_active":"2024-02-18T00:41:04.756283+0000","last_peered":"2024-02-18T00:41:04.756283+0000","last_clean":"2024-02-18T00:41:04.756283+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:41:04.756283+0000","last_undegraded":"2024-02-18T00:41:04.756283+0000","last_fullsized":"2024-02-18T00:41:04.756283+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:37:48.395205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:37:48.395205+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":548,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756692+0000","last_change":"2024-02-18T00:37:39.385746+0000","last_active":"2024-02-18T00:41:04.756692+0000","last_peered":"2024-02-18T00:41:04.756692+0000","last_clean":"2024-02-18T00:41:04.756692+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:41:04.756692+0000","last_undegraded":"2024-02-18T00:41:04.756692+0000","last_fullsized":"2024-02-18T00:41:04.756692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'15","last_scrub_stamp":"2024-02-18T00:37:39.385675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:39.385675+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":527,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763101+0000","last_change":"2024-02-18T00:39:58.895781+0000","last_active":"2024-02-18T00:41:04.763101+0000","last_peered":"2024-02-18T00:41:04.763101+0000","last_clean":"2024-02-18T00:41:04.763101+0000","last_became_active":"2024-02-18T00:39:36.741489+0000","last_became_peered":"2024-02-18T00:39:36.741489+0000","last_unstale":"2024-02-18T00:41:04.763101+0000","last_undegraded":"2024-02-18T00:41:04.763101+0000","last_fullsized":"2024-02-18T00:41:04.763101+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:58.895686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:58.895686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":509,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771339+0000","last_change":"2024-02-18T00:40:08.429634+0000","last_active":"2024-02-18T00:41:04.771339+0000","last_peered":"2024-02-18T00:41:04.771339+0000","last_clean":"2024-02-18T00:41:04.771339+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:41:04.771339+0000","last_undegraded":"2024-02-18T00:41:04.771339+0000","last_fullsized":"2024-02-18T00:41:04.771339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.429566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.429566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764901+0000","last_change":"2024-02-18T00:40:34.240139+0000","last_active":"2024-02-18T00:41:04.764901+0000","last_peered":"2024-02-18T00:41:04.764901+0000","last_clean":"2024-02-18T00:41:04.764901+0000","last_became_active":"2024-02-18T00:29:21.242688+0000","last_became_peered":"2024-02-18T00:29:21.242688+0000","last_unstale":"2024-02-18T00:41:04.764901+0000","last_undegraded":"2024-02-18T00:41:04.764901+0000","last_fullsized":"2024-02-18T00:41:04.764901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.240064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:30.225625+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.240064+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":442,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763840+0000","last_change":"2024-02-18T00:40:03.814573+0000","last_active":"2024-02-18T00:41:04.763840+0000","last_peered":"2024-02-18T00:41:04.763840+0000","last_clean":"2024-02-18T00:41:04.763840+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:41:04.763840+0000","last_undegraded":"2024-02-18T00:41:04.763840+0000","last_fullsized":"2024-02-18T00:41:04.763840+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:03.814478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.814478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":541,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765319+0000","last_change":"2024-02-18T00:37:43.883242+0000","last_active":"2024-02-18T00:41:04.765319+0000","last_peered":"2024-02-18T00:41:04.765319+0000","last_clean":"2024-02-18T00:41:04.765319+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:41:04.765319+0000","last_undegraded":"2024-02-18T00:41:04.765319+0000","last_fullsized":"2024-02-18T00:41:04.765319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'18","last_scrub_stamp":"2024-02-18T00:37:43.883172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:37:43.883172+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":534,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764543+0000","last_change":"2024-02-18T00:39:38.541343+0000","last_active":"2024-02-18T00:41:04.764543+0000","last_peered":"2024-02-18T00:41:04.764543+0000","last_clean":"2024-02-18T00:41:04.764543+0000","last_became_active":"2024-02-18T00:39:36.761839+0000","last_became_peered":"2024-02-18T00:39:36.761839+0000","last_unstale":"2024-02-18T00:41:04.764543+0000","last_undegraded":"2024-02-18T00:41:04.764543+0000","last_fullsized":"2024-02-18T00:41:04.764543+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:39:15.558360+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.558360+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.768502+0000","last_change":"2024-02-18T00:39:53.231555+0000","last_active":"2024-02-18T00:41:04.768502+0000","last_peered":"2024-02-18T00:41:04.768502+0000","last_clean":"2024-02-18T00:41:04.768502+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:41:04.768502+0000","last_undegraded":"2024-02-18T00:41:04.768502+0000","last_fullsized":"2024-02-18T00:41:04.768502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:53.231447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:53.231447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":490,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763622+0000","last_change":"2024-02-18T00:39:25.693245+0000","last_active":"2024-02-18T00:41:04.763622+0000","last_peered":"2024-02-18T00:41:04.763622+0000","last_clean":"2024-02-18T00:41:04.763622+0000","last_became_active":"2024-02-18T00:29:21.242644+0000","last_became_peered":"2024-02-18T00:29:21.242644+0000","last_unstale":"2024-02-18T00:41:04.763622+0000","last_undegraded":"2024-02-18T00:41:04.763622+0000","last_fullsized":"2024-02-18T00:41:04.763622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:25.693173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:25.693173+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":457,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756090+0000","last_change":"2024-02-18T00:39:36.772194+0000","last_active":"2024-02-18T00:41:04.756090+0000","last_peered":"2024-02-18T00:41:04.756090+0000","last_clean":"2024-02-18T00:41:04.756090+0000","last_became_active":"2024-02-18T00:39:36.771648+0000","last_became_peered":"2024-02-18T00:39:36.771648+0000","last_unstale":"2024-02-18T00:41:04.756090+0000","last_undegraded":"2024-02-18T00:41:04.756090+0000","last_fullsized":"2024-02-18T00:41:04.756090+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:27.705442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:27.705442+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":573,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765189+0000","last_change":"2024-02-18T00:40:46.132082+0000","last_active":"2024-02-18T00:41:04.765189+0000","last_peered":"2024-02-18T00:41:04.765189+0000","last_clean":"2024-02-18T00:41:04.765189+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:41:04.765189+0000","last_undegraded":"2024-02-18T00:41:04.765189+0000","last_fullsized":"2024-02-18T00:41:04.765189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:40:46.132007+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.132007+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":74,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.774868+0000","last_change":"2024-02-18T00:39:36.769794+0000","last_active":"2024-02-18T00:41:04.774868+0000","last_peered":"2024-02-18T00:41:04.774868+0000","last_clean":"2024-02-18T00:41:04.774868+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:41:04.774868+0000","last_undegraded":"2024-02-18T00:41:04.774868+0000","last_fullsized":"2024-02-18T00:41:04.774868+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:22.640720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.640720+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":525,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758978+0000","last_change":"2024-02-18T00:39:36.750168+0000","last_active":"2024-02-18T00:41:04.758978+0000","last_peered":"2024-02-18T00:41:04.758978+0000","last_clean":"2024-02-18T00:41:04.758978+0000","last_became_active":"2024-02-18T00:39:36.749467+0000","last_became_peered":"2024-02-18T00:39:36.749467+0000","last_unstale":"2024-02-18T00:41:04.758978+0000","last_undegraded":"2024-02-18T00:41:04.758978+0000","last_fullsized":"2024-02-18T00:41:04.758978+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:10.582430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:10.582430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":515,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.759925+0000","last_change":"2024-02-18T00:40:04.255429+0000","last_active":"2024-02-18T00:41:04.759925+0000","last_peered":"2024-02-18T00:41:04.759925+0000","last_clean":"2024-02-18T00:41:04.759925+0000","last_became_active":"2024-02-18T00:39:36.751306+0000","last_became_peered":"2024-02-18T00:39:36.751306+0000","last_unstale":"2024-02-18T00:41:04.759925+0000","last_undegraded":"2024-02-18T00:41:04.759925+0000","last_fullsized":"2024-02-18T00:41:04.759925+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:04.255350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:04.255350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":442,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.756507+0000","last_change":"2024-02-18T00:39:03.503323+0000","last_active":"2024-02-18T00:41:04.756507+0000","last_peered":"2024-02-18T00:41:04.756507+0000","last_clean":"2024-02-18T00:41:04.756507+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:41:04.756507+0000","last_undegraded":"2024-02-18T00:41:04.756507+0000","last_fullsized":"2024-02-18T00:41:04.756507+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:03.503256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.503256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":535,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.766067+0000","last_change":"2024-02-18T00:39:23.328438+0000","last_active":"2024-02-18T00:41:04.766067+0000","last_peered":"2024-02-18T00:41:04.766067+0000","last_clean":"2024-02-18T00:41:04.766067+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:41:04.766067+0000","last_undegraded":"2024-02-18T00:41:04.766067+0000","last_fullsized":"2024-02-18T00:41:04.766067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:39:23.328352+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:39:23.328352+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":79,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.771386+0000","last_change":"2024-02-18T00:39:54.729770+0000","last_active":"2024-02-18T00:41:04.771386+0000","last_peered":"2024-02-18T00:41:04.771386+0000","last_clean":"2024-02-18T00:41:04.771386+0000","last_became_active":"2024-02-18T00:39:36.761469+0000","last_became_peered":"2024-02-18T00:39:36.761469+0000","last_unstale":"2024-02-18T00:41:04.771386+0000","last_undegraded":"2024-02-18T00:41:04.771386+0000","last_fullsized":"2024-02-18T00:41:04.771386+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.729675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.729675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":74,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.770315+0000","last_change":"2024-02-18T00:39:36.759178+0000","last_active":"2024-02-18T00:41:04.770315+0000","last_peered":"2024-02-18T00:41:04.770315+0000","last_clean":"2024-02-18T00:41:04.770315+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:41:04.770315+0000","last_undegraded":"2024-02-18T00:41:04.770315+0000","last_fullsized":"2024-02-18T00:41:04.770315+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:07.344483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.344483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.758851+0000","last_change":"2024-02-18T00:39:53.576343+0000","last_active":"2024-02-18T00:41:04.758851+0000","last_peered":"2024-02-18T00:41:04.758851+0000","last_clean":"2024-02-18T00:41:04.758851+0000","last_became_active":"2024-02-18T00:29:21.104565+0000","last_became_peered":"2024-02-18T00:29:21.104565+0000","last_unstale":"2024-02-18T00:41:04.758851+0000","last_undegraded":"2024-02-18T00:41:04.758851+0000","last_fullsized":"2024-02-18T00:41:04.758851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:53.576276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:39:53.576276+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":454,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765591+0000","last_change":"2024-02-18T00:39:36.766737+0000","last_active":"2024-02-18T00:41:04.765591+0000","last_peered":"2024-02-18T00:41:04.765591+0000","last_clean":"2024-02-18T00:41:04.765591+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:41:04.765591+0000","last_undegraded":"2024-02-18T00:41:04.765591+0000","last_fullsized":"2024-02-18T00:41:04.765591+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:24.768673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:24.768673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":544,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764031+0000","last_change":"2024-02-18T00:38:17.269122+0000","last_active":"2024-02-18T00:41:04.764031+0000","last_peered":"2024-02-18T00:41:04.764031+0000","last_clean":"2024-02-18T00:41:04.764031+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:41:04.764031+0000","last_undegraded":"2024-02-18T00:41:04.764031+0000","last_fullsized":"2024-02-18T00:41:04.764031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"251'20","last_scrub_stamp":"2024-02-18T00:38:17.269050+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:38:17.269050+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":496,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764021+0000","last_change":"2024-02-18T00:38:22.776453+0000","last_active":"2024-02-18T00:41:04.764021+0000","last_peered":"2024-02-18T00:41:04.764021+0000","last_clean":"2024-02-18T00:41:04.764021+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:41:04.764021+0000","last_undegraded":"2024-02-18T00:41:04.764021+0000","last_fullsized":"2024-02-18T00:41:04.764021+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:22.776331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:38:22.776331+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":527,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.763294+0000","last_change":"2024-02-18T00:40:42.622606+0000","last_active":"2024-02-18T00:41:04.763294+0000","last_peered":"2024-02-18T00:41:04.763294+0000","last_clean":"2024-02-18T00:41:04.763294+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:41:04.763294+0000","last_undegraded":"2024-02-18T00:41:04.763294+0000","last_fullsized":"2024-02-18T00:41:04.763294+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:42.622532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.622532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":500,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765092+0000","last_change":"2024-02-18T00:40:42.144511+0000","last_active":"2024-02-18T00:41:04.765092+0000","last_peered":"2024-02-18T00:41:04.765092+0000","last_clean":"2024-02-18T00:41:04.765092+0000","last_became_active":"2024-02-18T00:29:21.104533+0000","last_became_peered":"2024-02-18T00:29:21.104533+0000","last_unstale":"2024-02-18T00:41:04.765092+0000","last_undegraded":"2024-02-18T00:41:04.765092+0000","last_fullsized":"2024-02-18T00:41:04.765092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:42.144406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.144406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":436,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.765902+0000","last_change":"2024-02-18T00:38:31.741746+0000","last_active":"2024-02-18T00:41:04.765902+0000","last_peered":"2024-02-18T00:41:04.765902+0000","last_clean":"2024-02-18T00:41:04.765902+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:41:04.765902+0000","last_undegraded":"2024-02-18T00:41:04.765902+0000","last_fullsized":"2024-02-18T00:41:04.765902+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:31.741672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:31.741672+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":599,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.762611+0000","last_change":"2024-02-18T00:39:06.697405+0000","last_active":"2024-02-18T00:41:04.762611+0000","last_peered":"2024-02-18T00:41:04.762611+0000","last_clean":"2024-02-18T00:41:04.762611+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:41:04.762611+0000","last_undegraded":"2024-02-18T00:41:04.762611+0000","last_fullsized":"2024-02-18T00:41:04.762611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:39:06.697317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.697317+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":518,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.755831+0000","last_change":"2024-02-18T00:39:36.772437+0000","last_active":"2024-02-18T00:41:04.755831+0000","last_peered":"2024-02-18T00:41:04.755831+0000","last_clean":"2024-02-18T00:41:04.755831+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:41:04.755831+0000","last_undegraded":"2024-02-18T00:41:04.755831+0000","last_fullsized":"2024-02-18T00:41:04.755831+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:37:45.454069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":502,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764305+0000","last_change":"2024-02-18T00:39:57.211234+0000","last_active":"2024-02-18T00:41:04.764305+0000","last_peered":"2024-02-18T00:41:04.764305+0000","last_clean":"2024-02-18T00:41:04.764305+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:41:04.764305+0000","last_undegraded":"2024-02-18T00:41:04.764305+0000","last_fullsized":"2024-02-18T00:41:04.764305+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:57.211166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:57.211166+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":513,"reported_epoch":531,"state":"active+clean","last_fresh":"2024-02-18T00:41:04.764327+0000","last_change":"2024-02-18T00:40:44.071040+0000","last_active":"2024-02-18T00:41:04.764327+0000","last_peered":"2024-02-18T00:41:04.764327+0000","last_clean":"2024-02-18T00:41:04.764327+0000","last_became_active":"2024-02-18T00:39:36.762288+0000","last_became_peered":"2024-02-18T00:39:36.762288+0000","last_unstale":"2024-02-18T00:41:04.764327+0000","last_undegraded":"2024-02-18T00:41:04.764327+0000","last_fullsized":"2024-02-18T00:41:04.764327+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:44.070974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:44.070974+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":62,"num_pg":32,"stat_sum":{"num_bytes":4020432,"num_objects":189,"num_object_clones":9,"num_object_copies":567,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":210,"num_read_kb":57,"num_write":771,"num_write_kb":4613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":359,"num_bytes_recovered":4000943,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":18,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14278656,"data_stored":12061329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2051,"ondisk_log_size":2051,"up":96,"acting":96,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1511,"num_read_kb":1236,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3196,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":915,"ondisk_log_size":915,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":701,"num_read_kb":696,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266834,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315140,"kb_used_data":9260,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93430012,"statfs":{"total":95995035648,"available":95672332288,"internally_reserved":0,"allocated":9482240,"data_stored":6795176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.56399999999999995,"15min":0.55700000000000005},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":1.0589999999999999,"5min":6.681,"15min":6.8700000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":1.0780000000000001,"5min":9.0850000000000009,"15min":11.288},"last":0.36099999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.58799999999999997,"15min":0.623},"min":{"1min":0.32400000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67100000000000004,"5min":6.0670000000000002,"15min":7.4729999999999999},"last":0.439},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.61299999999999999,"15min":0.64500000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1200000000000001,"5min":6.0800000000000001,"15min":7.7869999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.57599999999999996},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.997,"5min":2.145,"15min":6.7380000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.54400000000000004,"15min":0.59799999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.67400000000000004,"5min":2.125,"15min":12.861000000000001},"last":0.33100000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55800000000000005,"15min":0.57499999999999996},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1319999999999999,"5min":1.6719999999999999,"15min":7.1360000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54900000000000004,"15min":0.57999999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":1.4370000000000001,"15min":3.7989999999999999},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.44400000000000001,"15min":0.42799999999999999},"min":{"1min":0.27200000000000002,"5min":0.26900000000000002,"15min":0.215},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47799999999999998,"15min":0.44700000000000001},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.246},"max":{"1min":1.032,"5min":1.032,"15min":1.4770000000000001},"last":0.38300000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45800000000000002,"15min":0.45400000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.24099999999999999},"max":{"1min":0.64600000000000002,"5min":1.3220000000000001,"15min":1.603},"last":0.372},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.5,"15min":0.498},"min":{"1min":0.32900000000000001,"5min":0.27800000000000002,"15min":0.25600000000000001},"max":{"1min":1.151,"5min":1.151,"15min":4.9130000000000003},"last":0.34300000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:40:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.51300000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.432}]}]},{"osd":6,"up_from":47,"seq":201863463064,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319608,"kb_used_data":9152,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93425544,"statfs":{"total":95995035648,"available":95667757056,"internally_reserved":0,"allocated":9371648,"data_stored":6795287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55100000000000005,"15min":1.2569999999999999},"min":{"1min":0.29599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69699999999999995,"5min":4.0129999999999999,"15min":69.194999999999993},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54000000000000004,"15min":0.73499999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71199999999999997,"5min":1.7170000000000001,"15min":10.433},"last":0.37}]},{"osd":1,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54200000000000004,"15min":1.333},"min":{"1min":0.36099999999999999,"5min":0.29399999999999998,"15min":0.26000000000000001},"max":{"1min":0.77300000000000002,"5min":1.788,"15min":76.108999999999995},"last":0.437},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52400000000000002,"15min":1.3200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":1.216,"15min":71.872},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52200000000000002,"15min":0.80600000000000005},"min":{"1min":0.318,"5min":0.31,"15min":0.31},"max":{"1min":0.63100000000000001,"5min":1.196,"15min":39.939999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59199999999999997,"15min":0.98499999999999999},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.34200000000000003},"max":{"1min":0.67000000000000004,"5min":3.1600000000000001,"15min":39.673000000000002},"last":0.32800000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.62,"15min":0.83799999999999997},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":0.63500000000000001,"5min":8.3420000000000005,"15min":21.219000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54300000000000004,"15min":0.78100000000000003},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68500000000000005,"5min":1.506,"15min":14.007999999999999},"last":0.443}]},{"osd":4,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.44600000000000001,"15min":0.50900000000000001},"min":{"1min":0.32000000000000001,"5min":0.28399999999999997,"15min":0.22800000000000001},"max":{"1min":0.67400000000000004,"5min":0.71099999999999997,"15min":8.1479999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47899999999999998,"15min":0.69099999999999995},"min":{"1min":0.33800000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69999999999999996,"5min":0.85099999999999998,"15min":35.18},"last":0.505}]},{"osd":5,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48799999999999999,"15min":0.53800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27500000000000002},"max":{"1min":0.755,"5min":2.673,"15min":10.606},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.71599999999999997},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":35.119},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:40:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66400000000000003,"5min":5.4770000000000003,"15min":5.4770000000000003},"last":0.495},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56000000000000005,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.32600000000000001,"15min":0.32000000000000001},"max":{"1min":0.77800000000000002,"5min":2.8730000000000002,"15min":32.985999999999997},"last":0.52300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575023,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315580,"kb_used_data":5124,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429572,"statfs":{"total":95995035648,"available":95671881728,"internally_reserved":0,"allocated":5246976,"data_stored":2795320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.33900000000000002,"15min":0.64000000000000001},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.193},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":32.106999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.39200000000000002,"15min":0.68700000000000006},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.21099999999999999},"max":{"1min":0.65400000000000003,"5min":2.7040000000000002,"15min":43.137},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36699999999999999,"15min":0.76000000000000001},"min":{"1min":0.23499999999999999,"5min":0.20699999999999999,"15min":0.19900000000000001},"max":{"1min":0.49399999999999999,"5min":0.78300000000000003,"15min":49.201000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40300000000000002,"15min":0.59199999999999997},"min":{"1min":0.27500000000000002,"5min":0.27000000000000002,"15min":0.217},"max":{"1min":0.55300000000000005,"5min":0.83199999999999996,"15min":17.596},"last":0.39300000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:40:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.73699999999999999},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26200000000000001},"max":{"1min":0.76000000000000001,"5min":0.92800000000000005,"15min":37.524000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.755},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.71699999999999997,"15min":32.036000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:40:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55500000000000005,"15min":0.76700000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.77500000000000002,"5min":0.874,"15min":17.486999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59899999999999998,"15min":0.77700000000000002},"min":{"1min":0.41899999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88300000000000001,"5min":1.9790000000000001,"15min":12.75},"last":0.436}]},{"osd":5,"last update":"Sun Feb 18 00:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.77200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34399999999999997},"max":{"1min":0.85199999999999998,"5min":0.85299999999999998,"15min":12.611000000000001},"last":0.502},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.78300000000000003},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.90100000000000002,"15min":17.417000000000002},"last":0.51900000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:40:08 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.80900000000000005},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86099999999999999,"5min":2.0430000000000001,"15min":12.834},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.79200000000000004},"min":{"1min":0.44700000000000001,"5min":0.42199999999999999,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":12.651999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:40:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61699999999999999,"15min":0.80000000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76900000000000002,"5min":1.109,"15min":17.452000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.64300000000000002,"15min":0.81200000000000006},"min":{"1min":0.44900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":2.0859999999999999,"15min":12.805},"last":0.51100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738549,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311572,"kb_used_data":5212,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433580,"statfs":{"total":95995035648,"available":95675985920,"internally_reserved":0,"allocated":5337088,"data_stored":2798821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35299999999999998,"15min":1.036},"min":{"1min":0.23799999999999999,"5min":0.19500000000000001,"15min":0.18099999999999999},"max":{"1min":0.42999999999999999,"5min":1.2330000000000001,"15min":133.38999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.78700000000000003},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":1.8069999999999999,"5min":4.577,"15min":90.623000000000005},"last":0.312}]},{"osd":2,"last update":"Sun Feb 18 00:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.45600000000000002,"15min":0.89400000000000002},"min":{"1min":0.27000000000000002,"5min":0.255,"15min":0.20200000000000001},"max":{"1min":0.46800000000000003,"5min":4.54,"15min":90.850999999999999},"last":0.375},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.36199999999999999,"15min":0.86699999999999999},"min":{"1min":0.20300000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.61799999999999999,"5min":1.482,"15min":80.537000000000006},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38300000000000001,"15min":0.65900000000000003},"min":{"1min":0.21299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.503,"5min":1.1359999999999999,"15min":19.006},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48599999999999999,"15min":1.0629999999999999},"min":{"1min":0.308,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.6910000000000001,"5min":4.4489999999999998,"15min":146.71899999999999},"last":0.499}]},{"osd":4,"last update":"Sun Feb 18 00:40:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54100000000000004,"15min":0.91700000000000004},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.26700000000000002},"max":{"1min":0.72499999999999998,"5min":1.5269999999999999,"15min":57.914000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.64600000000000002,"15min":0.97899999999999998},"min":{"1min":0.40799999999999997,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":1.2589999999999999,"5min":4.2919999999999998,"15min":90.070999999999998},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65700000000000003,"15min":0.95299999999999996},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.32000000000000001},"max":{"1min":1.714,"5min":4.3760000000000003,"15min":90.049000000000007},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57099999999999995,"15min":0.97599999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32200000000000001},"max":{"1min":0.70599999999999996,"5min":1.5640000000000001,"15min":57.991},"last":0.41699999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.79700000000000004},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":19.039000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65400000000000003,"15min":0.98099999999999998},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":1.76,"5min":4.343,"15min":89.989999999999995},"last":0.46100000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:40:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.66700000000000004,"15min":1.048},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36499999999999999},"max":{"1min":0.77100000000000002,"5min":4.4100000000000001,"15min":90.025000000000006},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.59099999999999997,"15min":0.99399999999999999},"min":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.37},"max":{"1min":0.79900000000000004,"5min":1.603,"15min":57.962000000000003},"last":0.38700000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411500,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311704,"kb_used_data":5344,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433448,"statfs":{"total":95995035648,"available":95675850752,"internally_reserved":0,"allocated":5472256,"data_stored":2795806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33400000000000002,"15min":0.48599999999999999},"min":{"1min":0.23300000000000001,"5min":0.192,"15min":0.192},"max":{"1min":0.56200000000000006,"5min":0.78500000000000003,"15min":12.144},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35099999999999998,"15min":0.60299999999999998},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46600000000000003,"5min":0.63600000000000001,"15min":18.891999999999999},"last":0.40600000000000003}]},{"osd":1,"last update":"Sun Feb 18 00:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.49299999999999999},"min":{"1min":0.27500000000000002,"5min":0.23300000000000001,"15min":0.19700000000000001},"max":{"1min":0.49099999999999999,"5min":0.76800000000000002,"15min":10.138999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.36599999999999999,"15min":0.53900000000000003},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":0.496,"5min":0.64300000000000002,"15min":18.919},"last":0.442}]},{"osd":3,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.59599999999999997},"min":{"1min":0.33200000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.53100000000000003,"5min":0.67400000000000004,"15min":22.712},"last":0.376},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39700000000000002,"15min":0.60099999999999998},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.57099999999999995,"15min":22.359000000000002},"last":0.31900000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.68500000000000005},"min":{"1min":0.42099999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":18.812000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.67100000000000004},"min":{"1min":0.39200000000000002,"5min":0.36099999999999999,"15min":0.28699999999999998},"max":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":10.016},"last":0.54200000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.65000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":6.766},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.68799999999999994},"min":{"1min":0.42899999999999999,"5min":0.36699999999999999,"15min":0.317},"max":{"1min":0.84199999999999997,"5min":0.95999999999999996,"15min":10.114000000000001},"last":0.45300000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.66600000000000004},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.35799999999999998},"max":{"1min":0.83299999999999996,"5min":0.88,"15min":6.806},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.71599999999999997},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.34100000000000003},"max":{"1min":0.90500000000000003,"5min":0.92100000000000004,"15min":18.861000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59399999999999997,"15min":0.70399999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":6.8300000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64400000000000002,"15min":0.73999999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.372},"max":{"1min":0.95699999999999996,"5min":1.149,"15min":10.058999999999999},"last":0.501}]}]},{"osd":3,"up_from":26,"seq":111669149862,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315500,"kb_used_data":9140,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93429652,"statfs":{"total":95995035648,"available":95671963648,"internally_reserved":0,"allocated":9359360,"data_stored":6794888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.374,"15min":0.46999999999999997},"min":{"1min":0.219,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":0.496,"5min":0.92200000000000004,"15min":25.082999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.51900000000000002},"min":{"1min":0.25800000000000001,"5min":0.23699999999999999,"15min":0.20599999999999999},"max":{"1min":0.66600000000000004,"5min":1.2110000000000001,"15min":18.911000000000001},"last":0.33900000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38600000000000001,"15min":0.46800000000000003},"min":{"1min":0.28999999999999998,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":6.8479999999999999},"last":0.442},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.47599999999999998,"15min":0.69099999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.20599999999999999},"max":{"1min":0.65300000000000002,"5min":8.0779999999999994,"15min":23.606999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45400000000000001,"15min":0.65300000000000002},"min":{"1min":0.25700000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":4.0190000000000001,"15min":23.405000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42299999999999999,"15min":0.46500000000000002},"min":{"1min":0.28899999999999998,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.53800000000000003,"5min":1.4430000000000001,"15min":5.3479999999999999},"last":0.41799999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:40:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.60399999999999998,"15min":0.68700000000000006},"min":{"1min":0.36299999999999999,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.85899999999999999,"5min":3.8410000000000002,"15min":10.131},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59199999999999997,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":5.4630000000000001},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:40:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.45900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.7469999999999999,"5min":4.7469999999999999,"15min":7.4249999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.32400000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":6.7750000000000004},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63,"15min":0.747},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":3.8940000000000001,"15min":10.189},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.65000000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.84699999999999998,"5min":2.484,"15min":6.4279999999999999},"last":0.58599999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:40:30 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.35199999999999998},"max":{"1min":5.0830000000000002,"5min":5.0830000000000002,"15min":6.8220000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66300000000000003,"15min":0.72699999999999998},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.39800000000000002},"max":{"1min":1.0760000000000001,"5min":3.9289999999999998,"15min":10.164999999999999},"last":0.51700000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986337,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319308,"kb_used_data":5236,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93425844,"statfs":{"total":95995035648,"available":95668064256,"internally_reserved":0,"allocated":5361664,"data_stored":2802126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.497,"15min":0.88700000000000001},"min":{"1min":0.316,"5min":0.247,"15min":0.247},"max":{"1min":0.76500000000000001,"5min":0.92600000000000005,"15min":77.581999999999994},"last":0.249},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.49399999999999999,"15min":0.85399999999999998},"min":{"1min":0.29399999999999998,"5min":0.28199999999999997,"15min":0.27800000000000002},"max":{"1min":0.72499999999999998,"5min":0.79300000000000004,"15min":78.063999999999993},"last":0.26100000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50600000000000001,"15min":0.63},"min":{"1min":0.313,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.72599999999999998,"5min":0.89800000000000002,"15min":9.3160000000000007},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.88800000000000001},"min":{"1min":0.35099999999999998,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.73499999999999999,"5min":0.88,"15min":81.242000000000004},"last":0.32500000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.53200000000000003,"15min":0.64200000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":1.119,"15min":11.691000000000001},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52600000000000002,"15min":0.66700000000000004},"min":{"1min":0.35799999999999998,"5min":0.33500000000000002,"15min":0.27200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":17.77},"last":0.30199999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:40:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.60499999999999998},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.28299999999999997},"max":{"1min":0.749,"5min":1.8939999999999999,"15min":7.218},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.72899999999999998},"min":{"1min":0.34000000000000002,"5min":0.33300000000000002,"15min":0.309},"max":{"1min":0.76000000000000001,"5min":1.349,"15min":33.530999999999999},"last":0.313}]},{"osd":5,"last update":"Sun Feb 18 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45200000000000001,"15min":0.41699999999999998},"min":{"1min":0.32400000000000001,"5min":0.253,"15min":0.20300000000000001},"max":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":0.72299999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.45200000000000001,"15min":0.42599999999999999},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.24299999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.77900000000000003},"last":0.35299999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:40:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45500000000000002,"15min":0.43099999999999999},"min":{"1min":0.35399999999999998,"5min":0.27300000000000002,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.70299999999999996,"15min":0.76100000000000001},"last":0.221},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.47199999999999998},"min":{"1min":0.35199999999999998,"5min":0.28299999999999997,"15min":0.26000000000000001},"max":{"1min":0.63500000000000001,"5min":0.69299999999999995,"15min":0.751},"last":0.34499999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:40:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.502},"min":{"1min":0.42499999999999999,"5min":0.28499999999999998,"15min":0.27900000000000003},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.76900000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51100000000000001,"15min":0.495},"min":{"1min":0.41399999999999998,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.67500000000000004,"5min":0.73599999999999999,"15min":0.83299999999999996},"last":0.36099999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724700,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315080,"kb_used_data":5104,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93430072,"statfs":{"total":95995035648,"available":95672393728,"internally_reserved":0,"allocated":5226496,"data_stored":2792542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.36699999999999999,"5min":0.24399999999999999,"15min":0.217},"max":{"1min":0.70499999999999996,"5min":2.2549999999999999,"15min":6.3150000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.28199999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.61199999999999999,"5min":6.8680000000000003,"15min":12.744},"last":0.40999999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.27400000000000002},"max":{"1min":0.83399999999999996,"5min":13.25,"15min":13.25},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.69199999999999995,"15min":0.66400000000000003},"min":{"1min":0.41099999999999998,"5min":0.32400000000000001,"15min":0.29699999999999999},"max":{"1min":0.85699999999999998,"5min":13.262,"15min":13.262},"last":0.48599999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.59699999999999998},"min":{"1min":0.32800000000000001,"5min":0.27700000000000002,"15min":0.251},"max":{"1min":0.84299999999999997,"5min":3.2959999999999998,"15min":6.1680000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.66700000000000004,"15min":0.64500000000000002},"min":{"1min":0.35099999999999998,"5min":0.312,"15min":0.308},"max":{"1min":0.86799999999999999,"5min":5.0659999999999998,"15min":6.1879999999999997},"last":0.39300000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.624,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.32000000000000001,"15min":0.28399999999999997},"max":{"1min":0.69699999999999995,"5min":7.641,"15min":7.641},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.64500000000000002,"15min":0.61099999999999999},"min":{"1min":0.377,"5min":0.30399999999999999,"15min":0.27600000000000002},"max":{"1min":0.66200000000000003,"5min":7.4580000000000002,"15min":8.8650000000000002},"last":0.42699999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:40:37 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48999999999999999,"15min":0.46800000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.23599999999999999},"max":{"1min":0.68300000000000005,"5min":4.6859999999999999,"15min":11.818},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.48599999999999999,"15min":0.45200000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64500000000000002,"5min":1.5680000000000001,"15min":4.274},"last":0.35199999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:41:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.57799999999999996,"15min":0.54000000000000004},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.624,"5min":12.362,"15min":18.605},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":1.46,"15min":4.2089999999999996},"last":0.317}]},{"osd":7,"last update":"Sun Feb 18 00:40:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52100000000000002,"15min":0.48499999999999999},"min":{"1min":0.27400000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64300000000000002,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.56000000000000005,"15min":0.51700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68200000000000005,"5min":4.649,"15min":4.649},"last":0.38100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":10891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":6868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":7737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4268032,"data_stored":4006211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":13289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":3434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4304896,"data_stored":4006207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":62,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4411392,"data_stored":4006692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:41:10.833 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-02-18T00:41:10.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool set default.rgw.meta pg_num 22 2024-02-18T00:41:11.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:09.699741+0000 mgr.y (mgr.24454) 500 : audit [DBG] from='client.41378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:41:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:09.730740+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-18T00:41:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:09.739984+0000 mon.c (mon.2) 507 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:09.740837+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:09.949141+0000 mgr.y (mgr.24454) 501 : cluster [DBG] pgmap v819: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:09.699741+0000 mgr.y (mgr.24454) 500 : audit [DBG] from='client.41378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:41:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:09.730740+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-18T00:41:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:09.739984+0000 mon.c (mon.2) 507 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:09.740837+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:09.949141+0000 mgr.y (mgr.24454) 501 : cluster [DBG] pgmap v819: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:11.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:09.699741+0000 mgr.y (mgr.24454) 500 : audit [DBG] from='client.41378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:41:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:09.730740+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-18T00:41:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:09.739984+0000 mon.c (mon.2) 507 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:09.740837+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:09.949141+0000 mgr.y (mgr.24454) 501 : cluster [DBG] pgmap v819: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:11.394 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:41:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:41:11.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:41:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:41:11.790 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:11.793 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:11.794 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:11.894 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:11.923 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:12.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:10.737059+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:12.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:10.737185+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-18T00:41:12.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:10.759653+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:12.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:10.760127+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:10.737059+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:10.737185+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-18T00:41:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:10.759653+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:12.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:10.760127+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:12.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:10.737059+0000 mon.a (mon.0) 2850 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:12.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:10.737185+0000 mon.a (mon.0) 2851 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-18T00:41:12.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:10.759653+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:12.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:10.760127+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:12.123 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:12.125 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:12.126 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:12.227 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:12.255 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:12.329 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:12.331 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:12.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:12.413 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:12] "GET /metrics HTTP/1.1" 200 190487 "" "Prometheus/2.33.4" 2024-02-18T00:41:12.433 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:12.460 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:12.513 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:12.515 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:12.516 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:12.617 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:12.626 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:12.626 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:12.643 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:12.662 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:12.664 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:12.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:12.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 -- 172.21.15.59:0/1077150519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80fdbb0 msgr2=0x7ff1e80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/1077150519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80fdbb0 0x7ff1e80fdf90 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 rx=0x7ff1e4004870 tx=0x7ff1e4009cb0).stop 2024-02-18T00:41:12.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 -- 172.21.15.59:0/1077150519 shutdown_connections 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/1077150519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e80f8420 0x7ff1e8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/1077150519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80fdbb0 0x7ff1e80fdf90 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/1077150519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e80fe510 0x7ff1e80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 -- 172.21.15.59:0/1077150519 >> 172.21.15.59:0/1077150519 conn(0x7ff1e80f3800 msgr2=0x7ff1e80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 -- 172.21.15.59:0/1077150519 shutdown_connections 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 -- 172.21.15.59:0/1077150519 wait complete. 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 librados: starting msgr at 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 librados: starting objecter 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.647+0000 7ff1f06d7700 1 Processor -- start 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 -- start start 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 librados: setting wanted keys 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 librados: calling monclient init 2024-02-18T00:41:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80f8420 0x7ff1e810e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e80fdbb0 0x7ff1e810e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e80fe510 0x7ff1e8108270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8108760 con 0x7ff1e80fe510 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e81088d0 con 0x7ff1e80fdbb0 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1f06d7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8108a90 con 0x7ff1e80f8420 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1ee473700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80f8420 0x7ff1e810e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1ee473700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80f8420 0x7ff1e810e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57018/0 (socket says 172.21.15.59:57018) 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1eec74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e80fe510 0x7ff1e8108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1ee473700 1 -- 172.21.15.59:0/2917872390 learned_addr learned my addr 172.21.15.59:0/2917872390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1edc72700 1 --2- 172.21.15.59:0/2917872390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e80fdbb0 0x7ff1e810e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1eec74700 1 -- 172.21.15.59:0/2917872390 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80f8420 msgr2=0x7ff1e810e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1eec74700 1 --2- 172.21.15.59:0/2917872390 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80f8420 0x7ff1e810e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1eec74700 1 -- 172.21.15.59:0/2917872390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e80fdbb0 msgr2=0x7ff1e810e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1eec74700 1 --2- 172.21.15.59:0/2917872390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1e80fdbb0 0x7ff1e810e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1eec74700 1 -- 172.21.15.59:0/2917872390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00097c0 con 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1e80f8420 0x7ff1e810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1eec74700 1 --2- 172.21.15.59:0/2917872390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e80fe510 0x7ff1e8108270 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 rx=0x7ff1e0001cb0 tx=0x7ff1e0004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1df7fe700 10 client.?.objecter ms_handle_connect 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1df7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.648+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00079b0 con 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.649+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e4005040 con 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.649+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1e00090d0 con 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.649+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000fc20 con 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.649+0000 7ff1f06d7700 10 client.31581.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.649+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e81090d0 con 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.649+0000 7ff1f06d7700 1 librados: init done 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.649+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff1e81070a0 con 0x7ff1e80fe510 2024-02-18T00:41:12.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff1e000fd80 con 0x7ff1e80fe510 2024-02-18T00:41:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1df7fe700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff1d4062c90 0x7ff1d4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.700 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:41:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(538..538 src has 418..538) v4 ==== 8191+0+0 (secure 0 0 0) 0x7ff1e007db00 con 0x7ff1e80fe510 2024-02-18T00:41:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 osd_map(538..538 src has 418..538) v4 2024-02-18T00:41:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1df7fe700 3 client.31581.objecter handle_osd_map got epochs [538,538] > 0 2024-02-18T00:41:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1df7fe700 3 client.31581.objecter handle_osd_map decoding full epoch 538 2024-02-18T00:41:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff1d4062c90 0x7ff1d4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.650+0000 7ff1df7fe700 20 client.31581.objecter dump_active .. 0 homeless 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v538) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff1e007de60 con 0x7ff1e80fe510 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 1 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff1e8028540 0x7ff1e80fe970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 1 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 0 1 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8109750 osd=0 4 2024-02-18T00:41:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 10 client.31581.objecter _send_command 1 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff1d4062c90 0x7ff1d4065150 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 rx=0x7ff1e40043d0 tx=0x7ff1e400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e80fffe0 con 0x7ff1e8028540 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1d4062c90 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1edc72700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff1e8028540 0x7ff1e80fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1edc72700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff1e8028540 0x7ff1e80fe970 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.651+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e8028540 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.652+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e80fffe0 con 0x7ff1e8028540 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.652+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.652+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 1 = system:0 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.652+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8109750 osd=0 4 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.652+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 0 1 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 2 2024-02-18T00:41:12.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 2 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8109750 osd=0 3 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 0 2 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8109750 osd=0 4 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 10 client.31581.objecter _send_command 2 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.667+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e805a040 con 0x7ff1e8028540 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.668+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e805a040 con 0x7ff1e8028540 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.668+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.668+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.668+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8109750 osd=0 4 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.668+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 0 2 2024-02-18T00:41:12.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 3 2024-02-18T00:41:12.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:41:12.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6805/26279110 conn(0x7ff1e80594d0 0x7ff1e8116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 3 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 1 3 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8059360 osd=1 4 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 10 client.31581.objecter _send_command 3 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e806cf20 con 0x7ff1e80594d0 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6805/26279110 conn(0x7ff1e80594d0 0x7ff1e8116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e80594d0 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.669+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6805/26279110 conn(0x7ff1e80594d0 0x7ff1e8116fd0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.670+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e806cf20 con 0x7ff1e80594d0 2024-02-18T00:41:12.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.670+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.670+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 3 = system:0 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.670+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8059360 osd=1 4 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.670+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 1 3 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 4 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 4 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8059360 osd=1 3 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 1 4 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8059360 osd=1 4 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 10 client.31581.objecter _send_command 4 2024-02-18T00:41:12.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.685+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e80282e0 con 0x7ff1e80594d0 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e80282e0 con 0x7ff1e80594d0 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8059360 osd=1 4 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 1 4 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 5 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.686+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff1e81181a0 0x7ff1e811e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 5 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.707 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 2 5 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8027600 osd=2 4 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 10 client.31581.objecter _send_command 5 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e8027770 con 0x7ff1e81181a0 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1edc72700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff1e81181a0 0x7ff1e811e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e81181a0 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1edc72700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff1e81181a0 0x7ff1e811e9d0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e8027770 con 0x7ff1e81181a0 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 5 = system:0 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8027600 osd=2 4 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.687+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 2 5 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 6 2024-02-18T00:41:12.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 6 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8027600 osd=2 3 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 2 6 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8027600 osd=2 4 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 10 client.31581.objecter _send_command 6 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.703+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e8027960 con 0x7ff1e81181a0 2024-02-18T00:41:12.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e8027960 con 0x7ff1e81181a0 2024-02-18T00:41:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8027600 osd=2 4 2024-02-18T00:41:12.710 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 2 6 2024-02-18T00:41:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 7 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff1e80ffb40 0x7ff1e8122050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 7 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 3 7 2024-02-18T00:41:12.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8117940 osd=3 4 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 10 client.31581.objecter _send_command 7 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.704+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e8117ab0 con 0x7ff1e80ffb40 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1eec74700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff1e80ffb40 0x7ff1e8122050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e80ffb40 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1eec74700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff1e80ffb40 0x7ff1e8122050 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e8117ab0 con 0x7ff1e80ffb40 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 7 = system:0 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8117940 osd=3 4 2024-02-18T00:41:12.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.705+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 3 7 2024-02-18T00:41:12.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 8 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 8 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e8117940 osd=3 3 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 3 8 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8117940 osd=3 4 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 10 client.31581.objecter _send_command 8 2024-02-18T00:41:12.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.720+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e80ff0a0 con 0x7ff1e80ffb40 2024-02-18T00:41:12.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.721+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e80ff0a0 con 0x7ff1e80ffb40 2024-02-18T00:41:12.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.721+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.721+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.723 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:41:12.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.721+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e8117940 osd=3 4 2024-02-18T00:41:12.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.721+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 3 8 2024-02-18T00:41:12.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 9 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff1e811fba0 0x7ff1e81256d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 9 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 4 9 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff0a0 osd=4 4 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 10 client.31581.objecter _send_command 9 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e80ff360 con 0x7ff1e811fba0 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff1e811fba0 0x7ff1e81256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff1e811fba0 0x7ff1e81256d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.722+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e811fba0 2024-02-18T00:41:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.723+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e80ff360 con 0x7ff1e811fba0 2024-02-18T00:41:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.723+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:41:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.723+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 9 = system:0 2024-02-18T00:41:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.723+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e80ff0a0 osd=4 4 2024-02-18T00:41:12.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.723+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 4 9 2024-02-18T00:41:12.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 10 2024-02-18T00:41:12.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 10 2024-02-18T00:41:12.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e80ff0a0 osd=4 3 2024-02-18T00:41:12.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 4 10 2024-02-18T00:41:12.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff0a0 osd=4 4 2024-02-18T00:41:12.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 10 client.31581.objecter _send_command 10 2024-02-18T00:41:12.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.738+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e80ff360 con 0x7ff1e811fba0 2024-02-18T00:41:12.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e80ff360 con 0x7ff1e811fba0 2024-02-18T00:41:12.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e80ff0a0 osd=4 4 2024-02-18T00:41:12.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 4 10 2024-02-18T00:41:12.742 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 11 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff1e81260d0 0x7ff1e8128500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 11 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 5 11 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff360 osd=5 4 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 10 client.31581.objecter _send_command 11 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e812b2d0 con 0x7ff1e81260d0 2024-02-18T00:41:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1edc72700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff1e81260d0 0x7ff1e8128500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1edc72700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff1e81260d0 0x7ff1e8128500 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.740+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e81260d0 2024-02-18T00:41:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.741+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e812b2d0 con 0x7ff1e81260d0 2024-02-18T00:41:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.741+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:41:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.741+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 11 = system:0 2024-02-18T00:41:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.741+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e80ff360 osd=5 4 2024-02-18T00:41:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.741+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 5 11 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 12 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 12 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e80ff360 osd=5 3 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 5 12 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff360 osd=5 4 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 10 client.31581.objecter _send_command 12 2024-02-18T00:41:12.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.762+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e812b2d0 con 0x7ff1e81260d0 2024-02-18T00:41:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e812b2d0 con 0x7ff1e81260d0 2024-02-18T00:41:12.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e80ff360 osd=5 4 2024-02-18T00:41:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 5 12 2024-02-18T00:41:12.766 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:41:12.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 13 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff1e812a5d0 0x7ff1e812aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 13 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 6 13 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b2d0 osd=6 4 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 10 client.31581.objecter _send_command 13 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e812b070 con 0x7ff1e812a5d0 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1eec74700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff1e812a5d0 0x7ff1e812aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e812a5d0 2024-02-18T00:41:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.764+0000 7ff1eec74700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff1e812a5d0 0x7ff1e812aa50 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.768 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:12.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.765+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e812b070 con 0x7ff1e812a5d0 2024-02-18T00:41:12.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.765+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:41:12.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.765+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 13 = system:0 2024-02-18T00:41:12.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.765+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e812b2d0 osd=6 4 2024-02-18T00:41:12.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.765+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 6 13 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 14 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 14 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e812b2d0 osd=6 3 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 6 14 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b2d0 osd=6 4 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 10 client.31581.objecter _send_command 14 2024-02-18T00:41:12.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.781+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e812b070 con 0x7ff1e812a5d0 2024-02-18T00:41:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e812b070 con 0x7ff1e812a5d0 2024-02-18T00:41:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e812b2d0 osd=6 4 2024-02-18T00:41:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 6 14 2024-02-18T00:41:12.785 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:41:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:12.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 15 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6812/53025171 conn(0x7ff1e812dae0 0x7ff1e812df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 15 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 7 15 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b070 osd=7 4 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 10 client.31581.objecter _send_command 15 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1e8131870 con 0x7ff1e812dae0 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.783+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6812/53025171 conn(0x7ff1e812dae0 0x7ff1e812df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.784+0000 7ff1df7fe700 10 client.31581.objecter ms_handle_connect 0x7ff1e812dae0 2024-02-18T00:41:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.784+0000 7ff1ee473700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6812/53025171 conn(0x7ff1e812dae0 0x7ff1e812df40 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.784+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff1e8131870 con 0x7ff1e812dae0 2024-02-18T00:41:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.784+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:41:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.784+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 15 = system:0 2024-02-18T00:41:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.784+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e812b070 osd=7 4 2024-02-18T00:41:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.784+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 7 15 2024-02-18T00:41:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 10 client.31581.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign -1 16 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 20 client.31581.objecter _get_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_remove -1 16 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 20 client.31581.objecter get_session s=0x7ff1e812b070 osd=7 3 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 15 client.31581.objecter _session_command_op_assign 7 16 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b070 osd=7 4 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 10 client.31581.objecter _send_command 16 2024-02-18T00:41:12.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.799+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7ff1e8128ab0 con 0x7ff1e812dae0 2024-02-18T00:41:12.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1df7fe700 1 -- 172.21.15.59:0/2917872390 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7ff1e8128ab0 con 0x7ff1e812dae0 2024-02-18T00:41:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1df7fe700 10 client.31581.objecter ms_dispatch 0x7ff1e8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1df7fe700 10 client.31581.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1df7fe700 20 client.31581.objecter put_session s=0x7ff1e812b070 osd=7 4 2024-02-18T00:41:12.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1df7fe700 15 client.31581.objecter _session_command_op_remove 7 16 2024-02-18T00:41:12.803 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:41:12.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 librados: watch_flush enter 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 librados: watch_flush exit 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.0 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff1e8028540 msgr2=0x7ff1e80fe970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff1e8028540 0x7ff1e80fe970 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8109750 osd=0 1 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.1 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6805/26279110 conn(0x7ff1e80594d0 msgr2=0x7ff1e8116fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6805/26279110 conn(0x7ff1e80594d0 0x7ff1e8116fd0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8059360 osd=1 1 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.2 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff1e81181a0 msgr2=0x7ff1e811e9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff1e81181a0 0x7ff1e811e9d0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8027600 osd=2 1 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.3 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff1e80ffb40 msgr2=0x7ff1e8122050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff1e80ffb40 0x7ff1e8122050 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e8117940 osd=3 1 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.4 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff1e811fba0 msgr2=0x7ff1e81256d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff1e811fba0 0x7ff1e81256d0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1eec74700 1 -- 172.21.15.59:0/2917872390 reap_dead start 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff0a0 osd=4 1 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.5 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff1e81260d0 msgr2=0x7ff1e8128500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff1e81260d0 0x7ff1e8128500 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1eec74700 1 -- 172.21.15.59:0/2917872390 reap_dead start 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e80ff360 osd=5 1 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.6 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff1e812a5d0 msgr2=0x7ff1e812aa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff1e812a5d0 0x7ff1e812aa50 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b2d0 osd=6 1 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter close_session for osd.7 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6812/53025171 conn(0x7ff1e812dae0 msgr2=0x7ff1e812df40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.137:6812/53025171 conn(0x7ff1e812dae0 0x7ff1e812df40 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter put_session s=0x7ff1e812b070 osd=7 1 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 20 client.31581.objecter shutdown clearing up homeless session... 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 10 client.31581.objecter successfully canceled tick 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1eec74700 1 -- 172.21.15.59:0/2917872390 reap_dead start 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff1d4062c90 msgr2=0x7ff1d4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff1d4062c90 0x7ff1d4065150 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 rx=0x7ff1e40043d0 tx=0x7ff1e400b040).stop 2024-02-18T00:41:12.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e80fe510 msgr2=0x7ff1e8108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e80fe510 0x7ff1e8108270 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 rx=0x7ff1e0001cb0 tx=0x7ff1e0004c50).stop 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 shutdown_connections 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff1d4062c90 0x7ff1d4065150 unknown :-1 s=CLOSED pgs=3151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 --2- 172.21.15.59:0/2917872390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1e80fe510 0x7ff1e8108270 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 >> 172.21.15.59:0/2917872390 conn(0x7ff1e80f3800 msgr2=0x7ff1e80f53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 shutdown_connections 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 -- 172.21.15.59:0/2917872390 wait complete. 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:12.802+0000 7ff1f06d7700 1 librados: shutdown 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.808 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.809 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.809 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.809 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:41:12.868 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:12.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:12.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:12.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:12.978 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:11.393032+0000 mgr.y (mgr.24454) 502 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:12.978 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:11.740209+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:41:12.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:11.740298+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-18T00:41:12.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:11.743287+0000 mon.c (mon.2) 509 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]: dispatch 2024-02-18T00:41:12.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:11.748195+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]: dispatch 2024-02-18T00:41:12.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:11.949508+0000 mgr.y (mgr.24454) 503 : cluster [DBG] pgmap v822: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:12.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:12.379514+0000 osd.5 (osd.5) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:12.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:12.379642+0000 osd.5 (osd.5) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:12.979 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:12.651694+0000 mon.a (mon.0) 2856 : audit [DBG] from='client.? 172.21.15.59:0/2917872390' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:11.393032+0000 mgr.y (mgr.24454) 502 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:11.740209+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:11.740298+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:11.743287+0000 mon.c (mon.2) 509 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]: dispatch 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:11.748195+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]: dispatch 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:11.949508+0000 mgr.y (mgr.24454) 503 : cluster [DBG] pgmap v822: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:12.379514+0000 osd.5 (osd.5) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:13.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:12.379642+0000 osd.5 (osd.5) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:12.651694+0000 mon.a (mon.0) 2856 : audit [DBG] from='client.? 172.21.15.59:0/2917872390' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:11.393032+0000 mgr.y (mgr.24454) 502 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:11.740209+0000 mon.a (mon.0) 2853 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:11.740298+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:11.743287+0000 mon.c (mon.2) 509 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]: dispatch 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:11.748195+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]: dispatch 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:11.949508+0000 mgr.y (mgr.24454) 503 : cluster [DBG] pgmap v822: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:12.379514+0000 osd.5 (osd.5) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:12.379642+0000 osd.5 (osd.5) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:13.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:12.651694+0000 mon.a (mon.0) 2856 : audit [DBG] from='client.? 172.21.15.59:0/2917872390' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:13.004 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:13.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 -- 172.21.15.59:0/503806089 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000f8420 msgr2=0x7fb500104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 --2- 172.21.15.59:0/503806089 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000f8420 0x7fb500104a20 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 rx=0x7fb4f8009be0 tx=0x7fb4f8004d10).stop 2024-02-18T00:41:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 -- 172.21.15.59:0/503806089 shutdown_connections 2024-02-18T00:41:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 --2- 172.21.15.59:0/503806089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5000fdbb0 0x7fb5000fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 --2- 172.21.15.59:0/503806089 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000f8420 0x7fb500104a20 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 --2- 172.21.15.59:0/503806089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5000fe510 0x7fb5000f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 -- 172.21.15.59:0/503806089 >> 172.21.15.59:0/503806089 conn(0x7fb5000f3800 msgr2=0x7fb5000f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 -- 172.21.15.59:0/503806089 shutdown_connections 2024-02-18T00:41:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 -- 172.21.15.59:0/503806089 wait complete. 2024-02-18T00:41:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 librados: starting msgr at 2024-02-18T00:41:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.055+0000 7fb504f1a700 1 librados: starting objecter 2024-02-18T00:41:13.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 Processor -- start 2024-02-18T00:41:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 -- start start 2024-02-18T00:41:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 librados: setting wanted keys 2024-02-18T00:41:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 librados: calling monclient init 2024-02-18T00:41:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5000f8420 0x7fb50010e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000fdbb0 0x7fb50010e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:13.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5000fe510 0x7fb5001081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001086d0 con 0x7fb5000fe510 2024-02-18T00:41:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb500108840 con 0x7fb5000f8420 2024-02-18T00:41:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb504f1a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb500108a00 con 0x7fb5000fdbb0 2024-02-18T00:41:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb4fdd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000fdbb0 0x7fb50010e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb4fed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5000fe510 0x7fb5001081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:13.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb4fdd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000fdbb0 0x7fb50010e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57032/0 (socket says 172.21.15.59:57032) 2024-02-18T00:41:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb4fdd9b700 1 -- 172.21.15.59:0/3536070984 learned_addr learned my addr 172.21.15.59:0/3536070984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb4fe59c700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5000f8420 0x7fb50010e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb4fed9d700 1 -- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000fdbb0 msgr2=0x7fb50010e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.056+0000 7fb4fed9d700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000fdbb0 0x7fb50010e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4fed9d700 1 -- 172.21.15.59:0/3536070984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5000f8420 msgr2=0x7fb50010e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:13.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4fed9d700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5000f8420 0x7fb50010e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4fed9d700 1 -- 172.21.15.59:0/3536070984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4005040 con 0x7fb5000fe510 2024-02-18T00:41:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4fdd9b700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000fdbb0 0x7fb50010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4fed9d700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5000fe510 0x7fb5001081e0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7fb4f8009be0 tx=0x7fb4f8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:13.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4ef7fe700 10 client.?.objecter ms_handle_connect 0x7fb5000fe510 2024-02-18T00:41:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4ef7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4ef7fe700 1 -- 172.21.15.59:0/3536070984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f800e030 con 0x7fb5000fe510 2024-02-18T00:41:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f80097c0 con 0x7fb5000fe510 2024-02-18T00:41:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4ef7fe700 1 -- 172.21.15.59:0/3536070984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4f800b980 con 0x7fb5000fe510 2024-02-18T00:41:13.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb4ef7fe700 1 -- 172.21.15.59:0/3536070984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8018070 con 0x7fb5000fe510 2024-02-18T00:41:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb504f1a700 10 client.31587.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb500108ea0 con 0x7fb5000fe510 2024-02-18T00:41:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.057+0000 7fb504f1a700 1 librados: init done 2024-02-18T00:41:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.058+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb500109090 con 0x7fb5000fe510 2024-02-18T00:41:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.058+0000 7fb4ef7fe700 1 -- 172.21.15.59:0/3536070984 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb4f801d4b0 con 0x7fb5000fe510 2024-02-18T00:41:13.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4ef7fe700 1 --2- 172.21.15.59:0/3536070984 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb4e8062c90 0x7fb4e8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:13.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4ef7fe700 1 -- 172.21.15.59:0/3536070984 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(539..539 src has 418..539) v4 ==== 8191+0+0 (secure 0 0 0) 0x7fb4f807e880 con 0x7fb5000fe510 2024-02-18T00:41:13.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4ef7fe700 10 client.31587.objecter ms_dispatch 0x7fb500001550 osd_map(539..539 src has 418..539) v4 2024-02-18T00:41:13.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4ef7fe700 3 client.31587.objecter handle_osd_map got epochs [539,539] > 0 2024-02-18T00:41:13.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4ef7fe700 3 client.31587.objecter handle_osd_map decoding full epoch 539 2024-02-18T00:41:13.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4fe59c700 1 --2- 172.21.15.59:0/3536070984 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb4e8062c90 0x7fb4e8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:13.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4ef7fe700 20 client.31587.objecter dump_active .. 0 homeless 2024-02-18T00:41:13.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4ef7fe700 10 client.31587.objecter ms_handle_connect 0x7fb4e8062c90 2024-02-18T00:41:13.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.059+0000 7fb4fe59c700 1 --2- 172.21.15.59:0/3536070984 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb4e8062c90 0x7fb4e8065150 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 rx=0x7fb4f00048b0 tx=0x7fb4f000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:13.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.063+0000 7fb4ef7fe700 1 -- 172.21.15.59:0/3536070984 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb4f8051cd0 con 0x7fb5000fe510 2024-02-18T00:41:13.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.249+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fb5001071b0 con 0x7fb5000fe510 2024-02-18T00:41:13.285 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:13.368 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-18T00:41:13.556 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:13.559 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:13.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:13.660 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:13.688 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:13.707 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:13.709 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:13.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:13.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.753+0000 7fb4ef7fe700 1 -- 172.21.15.59:0/3536070984 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v540) v1 ==== 91+0+0 (secure 0 0 0) 0x7fb4f80518c0 con 0x7fb5000fe510 2024-02-18T00:41:13.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 10 librados: watch_flush enter 2024-02-18T00:41:13.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 10 librados: watch_flush exit 2024-02-18T00:41:13.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 20 client.31587.objecter shutdown clearing up homeless session... 2024-02-18T00:41:13.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 10 client.31587.objecter successfully canceled tick 2024-02-18T00:41:13.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb4e8062c90 msgr2=0x7fb4e8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:13.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 --2- 172.21.15.59:0/3536070984 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb4e8062c90 0x7fb4e8065150 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 rx=0x7fb4f00048b0 tx=0x7fb4f000b040).stop 2024-02-18T00:41:13.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5000fe510 msgr2=0x7fb5001081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:13.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5000fe510 0x7fb5001081e0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7fb4f8009be0 tx=0x7fb4f8004d10).stop 2024-02-18T00:41:13.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 shutdown_connections 2024-02-18T00:41:13.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 --2- 172.21.15.59:0/3536070984 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb4e8062c90 0x7fb4e8065150 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5000f8420 0x7fb50010e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5000fe510 0x7fb5001081e0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 --2- 172.21.15.59:0/3536070984 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb5000fdbb0 0x7fb50010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:13.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 >> 172.21.15.59:0/3536070984 conn(0x7fb5000f3800 msgr2=0x7fb5000f5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:13.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 shutdown_connections 2024-02-18T00:41:13.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 -- 172.21.15.59:0/3536070984 wait complete. 2024-02-18T00:41:13.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:13.758+0000 7fb504f1a700 1 librados: shutdown 2024-02-18T00:41:13.763 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:41:13.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:13.838 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:13.856 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:13.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:13.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:13.960 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:13.986 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:12.744351+0000 mon.a (mon.0) 2857 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:13.986 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:12.753829+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]': finished 2024-02-18T00:41:13.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:12.753914+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-18T00:41:13.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:12.755505+0000 mon.c (mon.2) 510 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:13.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:12.755895+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:13.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:13.251047+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? 172.21.15.59:0/3536070984' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:41:13.987 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:12.744351+0000 mon.a (mon.0) 2857 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:13.988 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:12.753829+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]': finished 2024-02-18T00:41:13.988 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:12.753914+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-18T00:41:13.988 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:12.755505+0000 mon.c (mon.2) 510 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:13.988 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:12.755895+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:13.988 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:13.251047+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? 172.21.15.59:0/3536070984' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:41:13.989 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:14.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:12.744351+0000 mon.a (mon.0) 2857 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:14.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:12.753829+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-88", "mode": "writeback"}]': finished 2024-02-18T00:41:14.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:12.753914+0000 mon.a (mon.0) 2859 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-18T00:41:14.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:12.755505+0000 mon.c (mon.2) 510 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:14.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:12.755895+0000 mon.a (mon.0) 2860 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:14.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:13.251047+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? 172.21.15.59:0/3536070984' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:41:14.187 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:14.190 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:14.191 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:14.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:14.320 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:14.870 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:14.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:14.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:14.975 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:15.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:13.750633+0000 mon.a (mon.0) 2862 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:13.754820+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]': finished 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:13.754896+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.59:0/3536070984' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:13.754942+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:13.756202+0000 mon.c (mon.2) 511 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:13.757619+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:13.949963+0000 mgr.y (mgr.24454) 504 : cluster [DBG] pgmap v825: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:13.750633+0000 mon.a (mon.0) 2862 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:13.754820+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]': finished 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:13.754896+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.59:0/3536070984' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:13.754942+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:13.756202+0000 mon.c (mon.2) 511 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:41:15.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:13.757619+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:41:15.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:13.949963+0000 mgr.y (mgr.24454) 504 : cluster [DBG] pgmap v825: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T00:41:15.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:13.750633+0000 mon.a (mon.0) 2862 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:15.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:13.754820+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-88"}]': finished 2024-02-18T00:41:15.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:13.754896+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? 172.21.15.59:0/3536070984' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:41:15.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:13.754942+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-18T00:41:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:13.756202+0000 mon.c (mon.2) 511 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:41:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:13.757619+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:41:15.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:13.949963+0000 mgr.y (mgr.24454) 504 : cluster [DBG] pgmap v825: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T00:41:15.008 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:15.029 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:15.032 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:15.032 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:15.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:15.163 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:15.183 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:15.186 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:15.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:15.287 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:15.320 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:15.343 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:15.347 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:15.347 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:15.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:15.839 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:16.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:14.751995+0000 mon.a (mon.0) 2867 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:14.765172+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:14.765254+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:14.767636+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:14.768810+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:14.820664+0000 osd.1 (osd.1) 277 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:14.821503+0000 osd.1 (osd.1) 278 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.121859+0000 osd.4 (osd.4) 311 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.124765+0000 osd.4 (osd.4) 312 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:14.751995+0000 mon.a (mon.0) 2867 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:14.765172+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:14.765254+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-18T00:41:16.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:14.767636+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:14.768810+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:14.820664+0000 osd.1 (osd.1) 277 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:14.821503+0000 osd.1 (osd.1) 278 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.121859+0000 osd.4 (osd.4) 311 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.124765+0000 osd.4 (osd.4) 312 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:14.751995+0000 mon.a (mon.0) 2867 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:14.765172+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:14.765254+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-18T00:41:16.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:14.767636+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T00:41:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:14.768810+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T00:41:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:14.820664+0000 osd.1 (osd.1) 277 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:41:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:14.821503+0000 osd.1 (osd.1) 278 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:41:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.121859+0000 osd.4 (osd.4) 311 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:41:16.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.124765+0000 osd.4 (osd.4) 312 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:41:16.126 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:16.129 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:16.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:16.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:41:16.230 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:16.262 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:16.283 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:16.286 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:16.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:16.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:16.418 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:16.444 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:16.447 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:16.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:16.549 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:16.578 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:16.727 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:16.731 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:16.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:16.832 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:16.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.830+0000 7f8bd4b60700 1 -- 172.21.15.59:0/1259422005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b110 msgr2=0x7f8bd006b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:16.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.830+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/1259422005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b110 0x7f8bd006b4f0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 rx=0x7f8bc4004870 tx=0x7f8bc4009cb0).stop 2024-02-18T00:41:16.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 -- 172.21.15.59:0/1259422005 shutdown_connections 2024-02-18T00:41:16.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/1259422005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8bd01044a0 0x7f8bd0106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:16.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/1259422005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b110 0x7f8bd006b4f0 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:16.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/1259422005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bd006b9e0 0x7f8bd0103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:16.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 -- 172.21.15.59:0/1259422005 >> 172.21.15.59:0/1259422005 conn(0x7f8bd00f3800 msgr2=0x7f8bd00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 -- 172.21.15.59:0/1259422005 shutdown_connections 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 -- 172.21.15.59:0/1259422005 wait complete. 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 librados: starting msgr at 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 librados: starting objecter 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 Processor -- start 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 -- start start 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 librados: setting wanted keys 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.831+0000 7f8bd4b60700 1 librados: calling monclient init 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8bd006b110 0x7f8bd0110290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b9e0 0x7f8bd0110780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bd01044a0 0x7f8bd010a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd010a970 con 0x7f8bd006b9e0 2024-02-18T00:41:16.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd010aae0 con 0x7f8bd01044a0 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd010aca0 con 0x7f8bd006b110 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bce59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8bd006b110 0x7f8bd0110290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b9e0 0x7f8bd0110780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b9e0 0x7f8bd0110780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37254/0 (socket says 172.21.15.59:37254) 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bce59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8bd006b110 0x7f8bd0110290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57044/0 (socket says 172.21.15.59:57044) 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 -- 172.21.15.59:0/2173839280 learned_addr learned my addr 172.21.15.59:0/2173839280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bced9d700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bd01044a0 0x7f8bd010a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 -- 172.21.15.59:0/2173839280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8bd006b110 msgr2=0x7f8bd0110290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8bd006b110 0x7f8bd0110290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 -- 172.21.15.59:0/2173839280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bd01044a0 msgr2=0x7f8bd010a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bd01044a0 0x7f8bd010a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 -- 172.21.15.59:0/2173839280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4005040 con 0x7f8bd006b9e0 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bced9d700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bd01044a0 0x7f8bd010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:41:16.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bcdd9b700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b9e0 0x7f8bd0110780 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7f8bc000c7c0 tx=0x7f8bc0007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bbf7fe700 10 client.?.objecter ms_handle_connect 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bbf7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bbf7fe700 1 -- 172.21.15.59:0/2173839280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0004430 con 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd010af80 con 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bbf7fe700 1 -- 172.21.15.59:0/2173839280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc000e970 con 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bbf7fe700 1 -- 172.21.15.59:0/2173839280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0013a50 con 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 10 client.31593.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd010b300 con 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.832+0000 7f8bd4b60700 1 librados: init done 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.833+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd005ee80 con 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 1 -- 172.21.15.59:0/2173839280 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8bc000e4a0 con 0x7f8bd006b9e0 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 1 --2- 172.21.15.59:0/2173839280 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8bb8062c90 0x7f8bb8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:16.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 1 -- 172.21.15.59:0/2173839280 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(543..543 src has 418..543) v4 ==== 8217+0+0 (secure 0 0 0) 0x7f8bc007d500 con 0x7f8bd006b9e0 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 10 client.31593.objecter ms_dispatch 0x7f8bd0001550 osd_map(543..543 src has 418..543) v4 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 3 client.31593.objecter handle_osd_map got epochs [543,543] > 0 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 3 client.31593.objecter handle_osd_map decoding full epoch 543 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bce59c700 1 --2- 172.21.15.59:0/2173839280 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8bb8062c90 0x7f8bb8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 20 client.31593.objecter dump_active .. 0 homeless 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bbf7fe700 10 client.31593.objecter ms_handle_connect 0x7f8bb8062c90 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.834+0000 7f8bce59c700 1 --2- 172.21.15.59:0/2173839280 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8bb8062c90 0x7f8bb8065150 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 rx=0x7f8bc4004560 tx=0x7f8bc4007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:16.839+0000 7f8bbf7fe700 1 -- 172.21.15.59:0/2173839280 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8bc0050930 con 0x7f8bd006b9e0 2024-02-18T00:41:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:16.884 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:16.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:41:16.887 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:16.888 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:16.989 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.141580+0000 osd.0 (osd.0) 244 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.145049+0000 osd.0 (osd.0) 245 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.565792+0000 osd.5 (osd.5) 235 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.566814+0000 osd.5 (osd.5) 236 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.605958+0000 osd.6 (osd.6) 257 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.608148+0000 osd.6 (osd.6) 258 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:15.765917+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]': finished 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.765997+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:15.770037+0000 mon.c (mon.2) 513 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:15.771424+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.818136+0000 osd.1 (osd.1) 279 : cluster [DBG] 5.c scrub starts 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.821333+0000 osd.1 (osd.1) 280 : cluster [DBG] 5.c scrub ok 2024-02-18T00:41:17.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:15.951060+0000 mgr.y (mgr.24454) 505 : cluster [DBG] pgmap v828: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:16.101562+0000 osd.4 (osd.4) 313 : cluster [DBG] 62.d scrub starts 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:16.105008+0000 osd.4 (osd.4) 314 : cluster [DBG] 62.d scrub ok 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.141580+0000 osd.0 (osd.0) 244 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.145049+0000 osd.0 (osd.0) 245 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.565792+0000 osd.5 (osd.5) 235 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.566814+0000 osd.5 (osd.5) 236 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.605958+0000 osd.6 (osd.6) 257 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.608148+0000 osd.6 (osd.6) 258 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:15.765917+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]': finished 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.765997+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-18T00:41:17.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:15.770037+0000 mon.c (mon.2) 513 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:17.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:15.771424+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:17.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.818136+0000 osd.1 (osd.1) 279 : cluster [DBG] 5.c scrub starts 2024-02-18T00:41:17.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.821333+0000 osd.1 (osd.1) 280 : cluster [DBG] 5.c scrub ok 2024-02-18T00:41:17.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:15.951060+0000 mgr.y (mgr.24454) 505 : cluster [DBG] pgmap v828: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:17.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:16.101562+0000 osd.4 (osd.4) 313 : cluster [DBG] 62.d scrub starts 2024-02-18T00:41:17.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:16.105008+0000 osd.4 (osd.4) 314 : cluster [DBG] 62.d scrub ok 2024-02-18T00:41:17.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.013+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"} v 0) v1 -- 0x7f8bd005fa60 con 0x7f8bd006b9e0 2024-02-18T00:41:17.019 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.141580+0000 osd.0 (osd.0) 244 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.145049+0000 osd.0 (osd.0) 245 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.565792+0000 osd.5 (osd.5) 235 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.566814+0000 osd.5 (osd.5) 236 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.605958+0000 osd.6 (osd.6) 257 : cluster [DBG] 62.2 scrub starts 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.608148+0000 osd.6 (osd.6) 258 : cluster [DBG] 62.2 scrub ok 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:15.765917+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_count","val": "1"}]': finished 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.765997+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:15.770037+0000 mon.c (mon.2) 513 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:15.771424+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.818136+0000 osd.1 (osd.1) 279 : cluster [DBG] 5.c scrub starts 2024-02-18T00:41:17.042 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.821333+0000 osd.1 (osd.1) 280 : cluster [DBG] 5.c scrub ok 2024-02-18T00:41:17.043 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:15.951060+0000 mgr.y (mgr.24454) 505 : cluster [DBG] pgmap v828: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:17.043 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:16.101562+0000 osd.4 (osd.4) 313 : cluster [DBG] 62.d scrub starts 2024-02-18T00:41:17.043 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:16.105008+0000 osd.4 (osd.4) 314 : cluster [DBG] 62.d scrub ok 2024-02-18T00:41:17.043 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:17.044 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:17.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:17.172 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:17.246 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:17.252 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:17.253 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:17.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:17.383 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:17.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.774+0000 7f8bbf7fe700 1 -- 172.21.15.59:0/2173839280 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]=0 set pool 6 pg_num to 22 v544) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bc0050520 con 0x7f8bd006b9e0 2024-02-18T00:41:17.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 10 librados: watch_flush enter 2024-02-18T00:41:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 10 librados: watch_flush exit 2024-02-18T00:41:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 20 client.31593.objecter shutdown clearing up homeless session... 2024-02-18T00:41:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 10 client.31593.objecter successfully canceled tick 2024-02-18T00:41:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8bb8062c90 msgr2=0x7f8bb8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/2173839280 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8bb8062c90 0x7f8bb8065150 secure :-1 s=READY pgs=3153 cs=0 l=1 rev1=1 rx=0x7f8bc4004560 tx=0x7f8bc4007f90).stop 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b9e0 msgr2=0x7f8bd0110780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.777+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b9e0 0x7f8bd0110780 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7f8bc000c7c0 tx=0x7f8bc0007ef0).stop 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 shutdown_connections 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/2173839280 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8bb8062c90 0x7f8bb8065150 unknown :-1 s=CLOSED pgs=3153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8bd01044a0 0x7f8bd010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8bd006b110 0x7f8bd0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 --2- 172.21.15.59:0/2173839280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bd006b9e0 0x7f8bd0110780 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 >> 172.21.15.59:0/2173839280 conn(0x7f8bd00f3800 msgr2=0x7f8bd00f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:17.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 shutdown_connections 2024-02-18T00:41:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 -- 172.21.15.59:0/2173839280 wait complete. 2024-02-18T00:41:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:17.778+0000 7f8bd4b60700 1 librados: shutdown 2024-02-18T00:41:17.781 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 6 pg_num to 22 2024-02-18T00:41:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:17.833 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:17.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:17.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:17.937 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:17.991 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:17.993 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:17.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:18.095 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:18.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:16.153928+0000 osd.0 (osd.0) 246 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:41:18.123 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:16.155183+0000 osd.0 (osd.0) 247 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:16.772765+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:16.772850+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:16.773609+0000 mon.c (mon.2) 514 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:16.774823+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:17.014704+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.59:0/2173839280' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.089807+0000 osd.4 (osd.4) 315 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.092710+0000 osd.4 (osd.4) 316 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.539596+0000 osd.7 (osd.7) 141 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:41:18.124 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.540786+0000 osd.7 (osd.7) 142 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:41:18.125 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:18.146 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:18.149 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:18.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:16.153928+0000 osd.0 (osd.0) 246 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:16.155183+0000 osd.0 (osd.0) 247 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:16.772765+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:16.772850+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:16.773609+0000 mon.c (mon.2) 514 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:16.774823+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:17.014704+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.59:0/2173839280' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.089807+0000 osd.4 (osd.4) 315 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.092710+0000 osd.4 (osd.4) 316 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.539596+0000 osd.7 (osd.7) 141 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:41:18.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.540786+0000 osd.7 (osd.7) 142 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:41:18.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:16.153928+0000 osd.0 (osd.0) 246 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:41:18.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:16.155183+0000 osd.0 (osd.0) 247 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:41:18.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:16.772765+0000 mon.a (mon.0) 2874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:41:18.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:16.772850+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-18T00:41:18.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:16.773609+0000 mon.c (mon.2) 514 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T00:41:18.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:16.774823+0000 mon.a (mon.0) 2876 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T00:41:18.190 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:17.014704+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? 172.21.15.59:0/2173839280' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]: dispatch 2024-02-18T00:41:18.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.089807+0000 osd.4 (osd.4) 315 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:41:18.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.092710+0000 osd.4 (osd.4) 316 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:41:18.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.539596+0000 osd.7 (osd.7) 141 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:41:18.191 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.540786+0000 osd.7 (osd.7) 142 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:41:18.252 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:18.280 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:18.301 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:18.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:18.304 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:18.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:18.438 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:18.689 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:18.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:18.791 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:18.820 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:18.839 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:18.843 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:18.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:18.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.134645+0000 osd.0 (osd.0) 248 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.135761+0000 osd.0 (osd.0) 249 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.544287+0000 osd.6 (osd.6) 259 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.546224+0000 osd.6 (osd.6) 260 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:17.773456+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]': finished 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:17.773550+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? 172.21.15.59:0/2173839280' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.773610+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:17.809040+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:18.971 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:17.809368+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.827623+0000 osd.1 (osd.1) 281 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.828792+0000 osd.1 (osd.1) 282 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:17.951452+0000 mgr.y (mgr.24454) 506 : cluster [DBG] pgmap v831: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:17.952266+0000 mon.a (mon.0) 2882 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:18.734093+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:18.734188+0000 mon.a (mon.0) 2884 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.734240+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:18.737392+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:18.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:18.748253+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.134645+0000 osd.0 (osd.0) 248 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.135761+0000 osd.0 (osd.0) 249 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.544287+0000 osd.6 (osd.6) 259 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.546224+0000 osd.6 (osd.6) 260 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:17.773456+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]': finished 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:17.773550+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? 172.21.15.59:0/2173839280' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.773610+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:17.809040+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:17.809368+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.827623+0000 osd.1 (osd.1) 281 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.828792+0000 osd.1 (osd.1) 282 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:41:18.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:17.951452+0000 mgr.y (mgr.24454) 506 : cluster [DBG] pgmap v831: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:18.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:17.952266+0000 mon.a (mon.0) 2882 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-18T00:41:18.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:18.734093+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:41:18.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:18.734188+0000 mon.a (mon.0) 2884 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-18T00:41:18.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.734240+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-18T00:41:18.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:18.737392+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:18.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:18.748253+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:18.974 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:18.996 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:18.999 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:18.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:19.100 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:19.127 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.134645+0000 osd.0 (osd.0) 248 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:41:19.127 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.135761+0000 osd.0 (osd.0) 249 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:41:19.127 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.544287+0000 osd.6 (osd.6) 259 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.546224+0000 osd.6 (osd.6) 260 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:17.773456+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-88","var": "target_max_objects","val": "250"}]': finished 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:17.773550+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? 172.21.15.59:0/2173839280' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num", "val": "22"}]': finished 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.773610+0000 mon.a (mon.0) 2880 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:17.809040+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:17.809368+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.827623+0000 osd.1 (osd.1) 281 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.828792+0000 osd.1 (osd.1) 282 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:17.951452+0000 mgr.y (mgr.24454) 506 : cluster [DBG] pgmap v831: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:17.952266+0000 mon.a (mon.0) 2882 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]: dispatch 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:18.734093+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:41:19.128 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:18.734188+0000 mon.a (mon.0) 2884 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num_actual", "val": "22"}]': finished 2024-02-18T00:41:19.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.734240+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-18T00:41:19.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:18.737392+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:19.129 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:18.748253+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]: dispatch 2024-02-18T00:41:19.129 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:19.152 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:19.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:19.253 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:19.280 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:19.299 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:19.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:19.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:19.314 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:41:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:41:19.314 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:41:19.314 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:41:19.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:19.883 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:19.886 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:19.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:19.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:20.017 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.148666+0000 osd.2 (osd.2) 217 : cluster [DBG] 6.f scrub starts 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.151667+0000 osd.2 (osd.2) 218 : cluster [DBG] 6.f scrub ok 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.578299+0000 osd.5 (osd.5) 237 : cluster [DBG] 25.c scrub starts 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.581307+0000 osd.5 (osd.5) 238 : cluster [DBG] 25.c scrub ok 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.782319+0000 osd.1 (osd.1) 283 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.785785+0000 osd.1 (osd.1) 284 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.895056+0000 osd.3 (osd.3) 291 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:41:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:18.896583+0000 osd.3 (osd.3) 292 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:19.501936+0000 osd.7 (osd.7) 143 : cluster [DBG] 62.b scrub starts 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:19.504293+0000 osd.7 (osd.7) 144 : cluster [DBG] 62.b scrub ok 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:19.734816+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]': finished 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:19.734934+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.148666+0000 osd.2 (osd.2) 217 : cluster [DBG] 6.f scrub starts 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.151667+0000 osd.2 (osd.2) 218 : cluster [DBG] 6.f scrub ok 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.578299+0000 osd.5 (osd.5) 237 : cluster [DBG] 25.c scrub starts 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.581307+0000 osd.5 (osd.5) 238 : cluster [DBG] 25.c scrub ok 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.782319+0000 osd.1 (osd.1) 283 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.785785+0000 osd.1 (osd.1) 284 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:41:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.895056+0000 osd.3 (osd.3) 291 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:18.896583+0000 osd.3 (osd.3) 292 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:19.501936+0000 osd.7 (osd.7) 143 : cluster [DBG] 62.b scrub starts 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:19.504293+0000 osd.7 (osd.7) 144 : cluster [DBG] 62.b scrub ok 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:19.734816+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]': finished 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:19.734934+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.148666+0000 osd.2 (osd.2) 217 : cluster [DBG] 6.f scrub starts 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.151667+0000 osd.2 (osd.2) 218 : cluster [DBG] 6.f scrub ok 2024-02-18T00:41:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.578299+0000 osd.5 (osd.5) 237 : cluster [DBG] 25.c scrub starts 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.581307+0000 osd.5 (osd.5) 238 : cluster [DBG] 25.c scrub ok 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.782319+0000 osd.1 (osd.1) 283 : cluster [DBG] 62.11 scrub starts 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.785785+0000 osd.1 (osd.1) 284 : cluster [DBG] 62.11 scrub ok 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.895056+0000 osd.3 (osd.3) 291 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:18.896583+0000 osd.3 (osd.3) 292 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:19.501936+0000 osd.7 (osd.7) 143 : cluster [DBG] 62.b scrub starts 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:19.504293+0000 osd.7 (osd.7) 144 : cluster [DBG] 62.b scrub ok 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:19.734816+0000 mon.a (mon.0) 2887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-88"}]': finished 2024-02-18T00:41:20.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:19.734934+0000 mon.a (mon.0) 2888 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-18T00:41:20.559 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-18T00:41:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:20.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:20.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:20.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:20.702 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:20.723 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:20.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:20.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:20.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:20.855 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:20.976 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:20.977 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:20.980 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:20.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:21.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:21.114 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:21.396 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:41:21.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:19.073041+0000 osd.0 (osd.0) 250 : cluster [DBG] 4.b scrub starts 2024-02-18T00:41:21.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:19.076762+0000 osd.0 (osd.0) 251 : cluster [DBG] 4.b scrub ok 2024-02-18T00:41:21.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:19.951865+0000 mgr.y (mgr.24454) 507 : cluster [DBG] pgmap v834: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:21.396 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.478529+0000 osd.7 (osd.7) 145 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:41:21.397 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.480127+0000 osd.7 (osd.7) 146 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:41:21.397 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.735797+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-18T00:41:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:19.073041+0000 osd.0 (osd.0) 250 : cluster [DBG] 4.b scrub starts 2024-02-18T00:41:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:19.076762+0000 osd.0 (osd.0) 251 : cluster [DBG] 4.b scrub ok 2024-02-18T00:41:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:19.951865+0000 mgr.y (mgr.24454) 507 : cluster [DBG] pgmap v834: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.478529+0000 osd.7 (osd.7) 145 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:41:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.480127+0000 osd.7 (osd.7) 146 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:41:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.735797+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-18T00:41:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:19.073041+0000 osd.0 (osd.0) 250 : cluster [DBG] 4.b scrub starts 2024-02-18T00:41:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:19.076762+0000 osd.0 (osd.0) 251 : cluster [DBG] 4.b scrub ok 2024-02-18T00:41:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:19.951865+0000 mgr.y (mgr.24454) 507 : cluster [DBG] pgmap v834: 238 pgs: 238 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.478529+0000 osd.7 (osd.7) 145 : cluster [DBG] 62.0 scrub starts 2024-02-18T00:41:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.480127+0000 osd.7 (osd.7) 146 : cluster [DBG] 62.0 scrub ok 2024-02-18T00:41:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.735797+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-18T00:41:21.564 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:21.566 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:21.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:21.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:21.698 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:21.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:41:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:41:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:21.850 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:21.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:21.951 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:21.983 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:22.005 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:22.008 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:22.009 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:22.109 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:22.238 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:22.242 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:22.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:22.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.137258+0000 osd.2 (osd.2) 219 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.138175+0000 osd.2 (osd.2) 220 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.521801+0000 osd.5 (osd.5) 239 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.525266+0000 osd.5 (osd.5) 240 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.990972+0000 osd.0 (osd.0) 252 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:20.995503+0000 osd.0 (osd.0) 253 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.033603+0000 osd.4 (osd.4) 317 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.034932+0000 osd.4 (osd.4) 318 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.140646+0000 osd.2 (osd.2) 221 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.141461+0000 osd.2 (osd.2) 222 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.477284+0000 osd.7 (osd.7) 147 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.478454+0000 osd.7 (osd.7) 148 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.737981+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-18T00:41:22.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:21.744939+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:22.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:21.745297+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:22.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:21.953244+0000 mon.a (mon.0) 2892 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T00:41:22.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:22.376 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:22.398 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:22.400 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:22.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:22.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:22.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:22] "GET /metrics HTTP/1.1" 200 190487 "" "Prometheus/2.33.4" 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.137258+0000 osd.2 (osd.2) 219 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.138175+0000 osd.2 (osd.2) 220 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.521801+0000 osd.5 (osd.5) 239 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.525266+0000 osd.5 (osd.5) 240 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.990972+0000 osd.0 (osd.0) 252 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:20.995503+0000 osd.0 (osd.0) 253 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.033603+0000 osd.4 (osd.4) 317 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.034932+0000 osd.4 (osd.4) 318 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.140646+0000 osd.2 (osd.2) 221 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.141461+0000 osd.2 (osd.2) 222 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:41:22.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.477284+0000 osd.7 (osd.7) 147 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.478454+0000 osd.7 (osd.7) 148 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.737981+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:21.744939+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:21.745297+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:21.953244+0000 mon.a (mon.0) 2892 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.137258+0000 osd.2 (osd.2) 219 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.138175+0000 osd.2 (osd.2) 220 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.521801+0000 osd.5 (osd.5) 239 : cluster [DBG] 62.1d scrub starts 2024-02-18T00:41:22.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.525266+0000 osd.5 (osd.5) 240 : cluster [DBG] 62.1d scrub ok 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.990972+0000 osd.0 (osd.0) 252 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:20.995503+0000 osd.0 (osd.0) 253 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.033603+0000 osd.4 (osd.4) 317 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.034932+0000 osd.4 (osd.4) 318 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.140646+0000 osd.2 (osd.2) 221 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.141461+0000 osd.2 (osd.2) 222 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.477284+0000 osd.7 (osd.7) 147 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.478454+0000 osd.7 (osd.7) 148 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.737981+0000 mon.a (mon.0) 2890 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:21.744939+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:21.745297+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:22.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:21.953244+0000 mon.a (mon.0) 2892 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T00:41:22.536 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 -- 172.21.15.59:0/399803564 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840fbef0 msgr2=0x7fdb840fc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/399803564 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840fbef0 0x7fdb840fc370 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 rx=0x7fdb7c004870 tx=0x7fdb7c009cb0).stop 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 -- 172.21.15.59:0/399803564 shutdown_connections 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/399803564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb840f69b0 0x7fdb840f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/399803564 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840fbef0 0x7fdb840fc370 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/399803564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb840f7310 0x7fdb840fba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 -- 172.21.15.59:0/399803564 >> 172.21.15.59:0/399803564 conn(0x7fdb840f26f0 msgr2=0x7fdb840f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 -- 172.21.15.59:0/399803564 shutdown_connections 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 -- 172.21.15.59:0/399803564 wait complete. 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 librados: starting msgr at 2024-02-18T00:41:22.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.502+0000 7fdb8bc99700 1 librados: starting objecter 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 Processor -- start 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 -- start start 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 librados: setting wanted keys 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 librados: calling monclient init 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb840f69b0 0x7fdb841144b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840f7310 0x7fdb841149a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb840fbef0 0x7fdb8410e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8410eb40 con 0x7fdb840f69b0 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8410ec80 con 0x7fdb840fbef0 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8bc99700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8410ee40 con 0x7fdb840f7310 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840f7310 0x7fdb841149a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840f7310 0x7fdb841149a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:34042/0 (socket says 172.21.15.59:34042) 2024-02-18T00:41:22.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 -- 172.21.15.59:0/27736849 learned_addr learned my addr 172.21.15.59:0/27736849 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb840fbef0 0x7fdb8410e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb840f69b0 0x7fdb841144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 -- 172.21.15.59:0/27736849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb840fbef0 msgr2=0x7fdb8410e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb840fbef0 0x7fdb8410e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 -- 172.21.15.59:0/27736849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb840f69b0 msgr2=0x7fdb841144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb840f69b0 0x7fdb841144b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 -- 172.21.15.59:0/27736849 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb78005040 con 0x7fdb840f7310 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb840f69b0 0x7fdb841144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb840fbef0 0x7fdb8410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb89234700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840f7310 0x7fdb841149a0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 rx=0x7fdb780048f0 tx=0x7fdb78009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb82ffd700 10 client.?.objecter ms_handle_connect 0x7fdb840f7310 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.503+0000 7fdb82ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78007b00 con 0x7fdb840f7310 2024-02-18T00:41:22.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb7800f040 con 0x7fdb840f7310 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c005040 con 0x7fdb840f7310 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78014aa0 con 0x7fdb840f7310 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb8bc99700 10 client.41402.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8410f4d0 con 0x7fdb840f7310 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb8bc99700 1 librados: init done 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.504+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fdb8405f990 con 0x7fdb840f7310 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.505+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fdb7800e660 con 0x7fdb840f7310 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.505+0000 7fdb82ffd700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdb6c062c90 0x7fdb6c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(548..548 src has 418..548) v4 ==== 8170+0+0 (secure 0 0 0) 0x7fdb7807ce30 con 0x7fdb840f7310 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 osd_map(548..548 src has 418..548) v4 2024-02-18T00:41:22.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 3 client.41402.objecter handle_osd_map got epochs [548,548] > 0 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 3 client.41402.objecter handle_osd_map decoding full epoch 548 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdb6c062c90 0x7fdb6c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 20 client.41402.objecter dump_active .. 0 homeless 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v548) v1 ==== 54+0+16 (secure 0 0 0) 0x7fdb7807f550 con 0x7fdb840f7310 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 1 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb6c062c90 2024-02-18T00:41:22.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdb6c062c90 0x7fdb6c065150 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 rx=0x7fdb74004550 tx=0x7fdb74009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdb84106d20 0x7fdb84107100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 1 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 0 1 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8410d6c0 osd=0 4 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 10 client.41402.objecter _send_command 1 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb8405ee80 con 0x7fdb84106d20 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdb84106d20 0x7fdb84107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdb84106d20 0x7fdb84107100 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.506+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb84106d20 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.507+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb8405ee80 con 0x7fdb84106d20 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.507+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.507+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 1 = system:0 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.507+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb8410d6c0 osd=0 4 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.507+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 0 1 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 2 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 2 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb8410d6c0 osd=0 3 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 0 2 2024-02-18T00:41:22.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8410d6c0 osd=0 4 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 10 client.41402.objecter _send_command 2 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.525+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb840273c0 con 0x7fdb84106d20 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb840273c0 con 0x7fdb84106d20 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb8410d6c0 osd=0 4 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 0 2 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 3 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6805/26279110 conn(0x7fdb840598f0 0x7fdb84059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 3 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 1 3 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84028710 osd=1 4 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 10 client.41402.objecter _send_command 3 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.527+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb84121ac0 con 0x7fdb840598f0 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6805/26279110 conn(0x7fdb840598f0 0x7fdb84059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb840598f0 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6805/26279110 conn(0x7fdb840598f0 0x7fdb84059cd0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb84121ac0 con 0x7fdb840598f0 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:41:22.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 3 = system:0 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84028710 osd=1 4 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.528+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 1 3 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 4 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 4 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84028710 osd=1 3 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 1 4 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84028710 osd=1 4 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 10 client.41402.objecter _send_command 4 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.544+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb84059360 con 0x7fdb840598f0 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb84059360 con 0x7fdb840598f0 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84028710 osd=1 4 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 1 4 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 5 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdb84028270 0x7fdb841204e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 5 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 2 5 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84059710 osd=2 4 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 10 client.41402.objecter _send_command 5 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb841209d0 con 0x7fdb84028270 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.546+0000 7fdb89234700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdb84028270 0x7fdb841204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.547+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb84028270 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.547+0000 7fdb89234700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdb84028270 0x7fdb841204e0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.547+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb841209d0 con 0x7fdb84028270 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.547+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.547+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 5 = system:0 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.547+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84059710 osd=2 4 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.547+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 2 5 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 6 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 6 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84059710 osd=2 3 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 2 6 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84059710 osd=2 4 2024-02-18T00:41:22.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 10 client.41402.objecter _send_command 6 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.563+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb840277d0 con 0x7fdb84028270 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb840277d0 con 0x7fdb84028270 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84059710 osd=2 4 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 2 6 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 7 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdb84123c40 0x7fdb84126030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 7 2024-02-18T00:41:22.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 3 7 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841213c0 osd=3 4 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 10 client.41402.objecter _send_command 7 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb84128c80 con 0x7fdb84123c40 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.565+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdb84123c40 0x7fdb84126030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.566+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb84123c40 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.566+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdb84123c40 0x7fdb84126030 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.566+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb84128c80 con 0x7fdb84123c40 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.566+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.566+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 7 = system:0 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.566+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb841213c0 osd=3 4 2024-02-18T00:41:22.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.566+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 3 7 2024-02-18T00:41:22.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:22.586 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:22.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:22.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 8 2024-02-18T00:41:22.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 8 2024-02-18T00:41:22.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb841213c0 osd=3 3 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 3 8 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841213c0 osd=3 4 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 10 client.41402.objecter _send_command 8 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.582+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb84120e30 con 0x7fdb84123c40 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb84120e30 con 0x7fdb84123c40 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb841213c0 osd=3 4 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 3 8 2024-02-18T00:41:22.588 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:41:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 9 2024-02-18T00:41:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdb84120040 0x7fdb841276a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 9 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 4 9 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120bf0 osd=4 4 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 10 client.41402.objecter _send_command 9 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb84120d60 con 0x7fdb84120040 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdb84120040 0x7fdb841276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb84120040 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.584+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdb84120040 0x7fdb841276a0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.585+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb84120d60 con 0x7fdb84120040 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.585+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.585+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 9 = system:0 2024-02-18T00:41:22.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.585+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84120bf0 osd=4 4 2024-02-18T00:41:22.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.585+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 4 9 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 10 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 10 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84120bf0 osd=4 3 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 4 10 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120bf0 osd=4 4 2024-02-18T00:41:22.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 10 client.41402.objecter _send_command 10 2024-02-18T00:41:22.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.601+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb84120d60 con 0x7fdb84120040 2024-02-18T00:41:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb84120d60 con 0x7fdb84120040 2024-02-18T00:41:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84120bf0 osd=4 4 2024-02-18T00:41:22.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 4 10 2024-02-18T00:41:22.604 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:41:22.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 11 2024-02-18T00:41:22.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdb841280d0 0x7fdb841284f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 11 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 5 11 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120d60 osd=5 4 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 10 client.41402.objecter _send_command 11 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.602+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb8412f5a0 con 0x7fdb841280d0 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.603+0000 7fdb89234700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdb841280d0 0x7fdb841284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.603+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb841280d0 2024-02-18T00:41:22.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.603+0000 7fdb89234700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdb841280d0 0x7fdb841284f0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.604+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb8412f5a0 con 0x7fdb841280d0 2024-02-18T00:41:22.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.604+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:41:22.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.604+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 11 = system:0 2024-02-18T00:41:22.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.604+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84120d60 osd=5 4 2024-02-18T00:41:22.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.604+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 5 11 2024-02-18T00:41:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 12 2024-02-18T00:41:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 12 2024-02-18T00:41:22.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb84120d60 osd=5 3 2024-02-18T00:41:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 5 12 2024-02-18T00:41:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120d60 osd=5 4 2024-02-18T00:41:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 10 client.41402.objecter _send_command 12 2024-02-18T00:41:22.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.618+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb8412ce40 con 0x7fdb841280d0 2024-02-18T00:41:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb8412ce40 con 0x7fdb841280d0 2024-02-18T00:41:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb84120d60 osd=5 4 2024-02-18T00:41:22.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 5 12 2024-02-18T00:41:22.622 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 13 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdb8412d520 0x7fdb8412f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 13 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 6 13 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8412ce40 osd=6 4 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 10 client.41402.objecter _send_command 13 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.620+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb841326e0 con 0x7fdb8412d520 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.621+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdb8412d520 0x7fdb8412f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.621+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb8412d520 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.621+0000 7fdb8a236700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdb8412d520 0x7fdb8412f910 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.622+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb841326e0 con 0x7fdb8412d520 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.622+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:41:22.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.622+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 13 = system:0 2024-02-18T00:41:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.622+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb8412ce40 osd=6 4 2024-02-18T00:41:22.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.622+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 6 13 2024-02-18T00:41:22.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 14 2024-02-18T00:41:22.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 14 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb8412ce40 osd=6 3 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 6 14 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8412ce40 osd=6 4 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 10 client.41402.objecter _send_command 14 2024-02-18T00:41:22.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.637+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb841326e0 con 0x7fdb8412d520 2024-02-18T00:41:22.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb841326e0 con 0x7fdb8412d520 2024-02-18T00:41:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb8412ce40 osd=6 4 2024-02-18T00:41:22.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 6 14 2024-02-18T00:41:22.641 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:41:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 15 2024-02-18T00:41:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6812/53025171 conn(0x7fdb841319e0 0x7fdb84131e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 15 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 7 15 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841326e0 osd=7 4 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 10 client.41402.objecter _send_command 15 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb841324d0 con 0x7fdb841319e0 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.639+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6812/53025171 conn(0x7fdb841319e0 0x7fdb84131e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.640+0000 7fdb82ffd700 10 client.41402.objecter ms_handle_connect 0x7fdb841319e0 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.640+0000 7fdb89a35700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6812/53025171 conn(0x7fdb841319e0 0x7fdb84131e60 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.640+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fdb841324d0 con 0x7fdb841319e0 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.640+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:41:22.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.640+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 15 = system:0 2024-02-18T00:41:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.640+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb841326e0 osd=7 4 2024-02-18T00:41:22.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.640+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 7 15 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 10 client.41402.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign -1 16 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 20 client.41402.objecter _get_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_remove -1 16 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 20 client.41402.objecter get_session s=0x7fdb841326e0 osd=7 3 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 15 client.41402.objecter _session_command_op_assign 7 16 2024-02-18T00:41:22.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841326e0 osd=7 4 2024-02-18T00:41:22.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 10 client.41402.objecter _send_command 16 2024-02-18T00:41:22.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.655+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fdb841324d0 con 0x7fdb841319e0 2024-02-18T00:41:22.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb82ffd700 1 -- 172.21.15.59:0/27736849 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fdb841324d0 con 0x7fdb841319e0 2024-02-18T00:41:22.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb82ffd700 10 client.41402.objecter ms_dispatch 0x7fdb84001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:22.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb82ffd700 10 client.41402.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb82ffd700 20 client.41402.objecter put_session s=0x7fdb841326e0 osd=7 4 2024-02-18T00:41:22.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb82ffd700 15 client.41402.objecter _session_command_op_remove 7 16 2024-02-18T00:41:22.659 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 librados: watch_flush enter 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 librados: watch_flush exit 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.0 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdb84106d20 msgr2=0x7fdb84107100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6801/3700238977 conn(0x7fdb84106d20 0x7fdb84107100 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8410d6c0 osd=0 1 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.1 2024-02-18T00:41:22.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6805/26279110 conn(0x7fdb840598f0 msgr2=0x7fdb84059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6805/26279110 conn(0x7fdb840598f0 0x7fdb84059cd0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84028710 osd=1 1 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.2 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdb84028270 msgr2=0x7fdb841204e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6809/1153193637 conn(0x7fdb84028270 0x7fdb841204e0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84059710 osd=2 1 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.3 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdb84123c40 msgr2=0x7fdb84126030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6813/1805700964 conn(0x7fdb84123c40 0x7fdb84126030 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8a236700 1 -- 172.21.15.59:0/27736849 reap_dead start 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841213c0 osd=3 1 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.4 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdb84120040 msgr2=0x7fdb841276a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6800/2044387056 conn(0x7fdb84120040 0x7fdb841276a0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8a236700 1 -- 172.21.15.59:0/27736849 reap_dead start 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120bf0 osd=4 1 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.5 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdb841280d0 msgr2=0x7fdb841284f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6804/2170432263 conn(0x7fdb841280d0 0x7fdb841284f0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb84120d60 osd=5 1 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.6 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdb8412d520 msgr2=0x7fdb8412f910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6808/3356009390 conn(0x7fdb8412d520 0x7fdb8412f910 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb8412ce40 osd=6 1 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter close_session for osd.7 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6812/53025171 conn(0x7fdb841319e0 msgr2=0x7fdb84131e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.137:6812/53025171 conn(0x7fdb841319e0 0x7fdb84131e60 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter put_session s=0x7fdb841326e0 osd=7 1 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 20 client.41402.objecter shutdown clearing up homeless session... 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 10 client.41402.objecter successfully canceled tick 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdb6c062c90 msgr2=0x7fdb6c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdb6c062c90 0x7fdb6c065150 secure :-1 s=READY pgs=3154 cs=0 l=1 rev1=1 rx=0x7fdb74004550 tx=0x7fdb74009040).stop 2024-02-18T00:41:22.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840f7310 msgr2=0x7fdb841149a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:22.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 --2- 172.21.15.59:0/27736849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb840f7310 0x7fdb841149a0 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 rx=0x7fdb780048f0 tx=0x7fdb78009cb0).stop 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8a236700 1 -- 172.21.15.59:0/27736849 reap_dead start 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8a236700 1 -- 172.21.15.59:0/27736849 reap_dead start 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 shutdown_connections 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 >> 172.21.15.59:0/27736849 conn(0x7fdb840f26f0 msgr2=0x7fdb841057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 shutdown_connections 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 -- 172.21.15.59:0/27736849 wait complete. 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:22.658+0000 7fdb8bc99700 1 librados: shutdown 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.666 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.667 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:41:22.688 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:22.719 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:22.741 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:22.749 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:22.749 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:22.850 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:22.879 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:22.898 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:22.904 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:22.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:23.006 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:23.033 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:23.052 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:23.054 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:23.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:23.156 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:23.186 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:21.394992+0000 mgr.y (mgr.24454) 508 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.503985+0000 osd.6 (osd.6) 261 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.505489+0000 osd.6 (osd.6) 262 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.952425+0000 mgr.y (mgr.24454) 509 : cluster [DBG] pgmap v837: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.990021+0000 osd.3 (osd.3) 293 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:21.991783+0000 osd.3 (osd.3) 294 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:22.120093+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.d scrub starts 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:22.121875+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.d scrub ok 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:22.506759+0000 mon.c (mon.2) 518 : audit [DBG] from='client.? 172.21.15.59:0/27736849' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:22.742527+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:22.742606+0000 mon.a (mon.0) 2894 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T00:41:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:22.742655+0000 mon.a (mon.0) 2895 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:22.749645+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:22.751433+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.005834+0000 osd.0 (osd.0) 254 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.006684+0000 osd.0 (osd.0) 255 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:21.394992+0000 mgr.y (mgr.24454) 508 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.503985+0000 osd.6 (osd.6) 261 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.505489+0000 osd.6 (osd.6) 262 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.952425+0000 mgr.y (mgr.24454) 509 : cluster [DBG] pgmap v837: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.990021+0000 osd.3 (osd.3) 293 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:21.991783+0000 osd.3 (osd.3) 294 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:22.120093+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.d scrub starts 2024-02-18T00:41:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:22.121875+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.d scrub ok 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:22.506759+0000 mon.c (mon.2) 518 : audit [DBG] from='client.? 172.21.15.59:0/27736849' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:22.742527+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:22.742606+0000 mon.a (mon.0) 2894 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:22.742655+0000 mon.a (mon.0) 2895 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:22.749645+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:22.751433+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.005834+0000 osd.0 (osd.0) 254 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.006684+0000 osd.0 (osd.0) 255 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:41:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:21.394992+0000 mgr.y (mgr.24454) 508 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.503985+0000 osd.6 (osd.6) 261 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.505489+0000 osd.6 (osd.6) 262 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.952425+0000 mgr.y (mgr.24454) 509 : cluster [DBG] pgmap v837: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.990021+0000 osd.3 (osd.3) 293 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:21.991783+0000 osd.3 (osd.3) 294 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:22.120093+0000 osd.2 (osd.2) 223 : cluster [DBG] 5.d scrub starts 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:22.121875+0000 osd.2 (osd.2) 224 : cluster [DBG] 5.d scrub ok 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:22.506759+0000 mon.c (mon.2) 518 : audit [DBG] from='client.? 172.21.15.59:0/27736849' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:22.742527+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:22.742606+0000 mon.a (mon.0) 2894 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:22.742655+0000 mon.a (mon.0) 2895 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:22.749645+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:23.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:22.751433+0000 mon.a (mon.0) 2896 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:41:23.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.005834+0000 osd.0 (osd.0) 254 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:41:23.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.006684+0000 osd.0 (osd.0) 255 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:41:23.635 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:23.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:23.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:23.702 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:41:23.702 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:41:23.702 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-02-18T00:41:23.702 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-18T00:41:23.740 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:23.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:23.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:23.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:23.926 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:23.963 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:23.966 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:23.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:24.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:24.095 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:24.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:22.987137+0000 osd.3 (osd.3) 295 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:41:24.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:22.988035+0000 osd.3 (osd.3) 296 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:41:24.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:23.743026+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:41:24.495 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.743107+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:23.747144+0000 mon.c (mon.2) 520 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:23.748006+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:22.987137+0000 osd.3 (osd.3) 295 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:22.988035+0000 osd.3 (osd.3) 296 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:23.743026+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.743107+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:23.747144+0000 mon.c (mon.2) 520 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:24.496 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:23.748006+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:24.497 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:24.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:22.987137+0000 osd.3 (osd.3) 295 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:41:24.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:22.988035+0000 osd.3 (osd.3) 296 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:41:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:23.743026+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:41:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.743107+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-18T00:41:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:23.747144+0000 mon.c (mon.2) 520 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:23.748006+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:24.545 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:24.546 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:24.548 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:24.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:24.649 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:24.681 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:24.703 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:24.705 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:24.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:24.807 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:24.838 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:24.862 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:24.865 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:24.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:24.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:24.995 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.547794+0000 osd.6 (osd.6) 263 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:41:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.548867+0000 osd.6 (osd.6) 264 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:41:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.574081+0000 osd.5 (osd.5) 241 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:41:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.574961+0000 osd.5 (osd.5) 242 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.793719+0000 osd.1 (osd.1) 285 : cluster [DBG] 2.d scrub starts 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.794675+0000 osd.1 (osd.1) 286 : cluster [DBG] 2.d scrub ok 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:23.952830+0000 mgr.y (mgr.24454) 510 : cluster [DBG] pgmap v840: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:24.121762+0000 osd.2 (osd.2) 225 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:24.123651+0000 osd.2 (osd.2) 226 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:24.748721+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]': finished 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:24.748831+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:24.751363+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]: dispatch 2024-02-18T00:41:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:24.751803+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]: dispatch 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.547794+0000 osd.6 (osd.6) 263 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.548867+0000 osd.6 (osd.6) 264 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.574081+0000 osd.5 (osd.5) 241 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.574961+0000 osd.5 (osd.5) 242 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.793719+0000 osd.1 (osd.1) 285 : cluster [DBG] 2.d scrub starts 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.794675+0000 osd.1 (osd.1) 286 : cluster [DBG] 2.d scrub ok 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:23.952830+0000 mgr.y (mgr.24454) 510 : cluster [DBG] pgmap v840: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:24.121762+0000 osd.2 (osd.2) 225 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:24.123651+0000 osd.2 (osd.2) 226 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:24.748721+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]': finished 2024-02-18T00:41:25.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:24.748831+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-18T00:41:25.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:24.751363+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]: dispatch 2024-02-18T00:41:25.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:24.751803+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]: dispatch 2024-02-18T00:41:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.547794+0000 osd.6 (osd.6) 263 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:41:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.548867+0000 osd.6 (osd.6) 264 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:41:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.574081+0000 osd.5 (osd.5) 241 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:41:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.574961+0000 osd.5 (osd.5) 242 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:41:25.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.793719+0000 osd.1 (osd.1) 285 : cluster [DBG] 2.d scrub starts 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.794675+0000 osd.1 (osd.1) 286 : cluster [DBG] 2.d scrub ok 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:23.952830+0000 mgr.y (mgr.24454) 510 : cluster [DBG] pgmap v840: 238 pgs: 32 unknown, 206 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:24.121762+0000 osd.2 (osd.2) 225 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:24.123651+0000 osd.2 (osd.2) 226 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:24.748721+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-6", "overlaypool": "test-rados-api-smithi059-82957-90"}]': finished 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:24.748831+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:24.751363+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]: dispatch 2024-02-18T00:41:25.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:24.751803+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]: dispatch 2024-02-18T00:41:25.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:41:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 -- 172.21.15.59:0/902681913 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500f7400 msgr2=0x7f9d50103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 --2- 172.21.15.59:0/902681913 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500f7400 0x7f9d50103a20 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 rx=0x7f9d48009c30 tx=0x7f9d48004c50).stop 2024-02-18T00:41:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 -- 172.21.15.59:0/902681913 shutdown_connections 2024-02-18T00:41:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 --2- 172.21.15.59:0/902681913 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500f7400 0x7f9d50103a20 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 --2- 172.21.15.59:0/902681913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500fcbb0 0x7f9d500fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 --2- 172.21.15.59:0/902681913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d500fd510 0x7f9d500f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 -- 172.21.15.59:0/902681913 >> 172.21.15.59:0/902681913 conn(0x7f9d500f2820 msgr2=0x7f9d500f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 -- 172.21.15.59:0/902681913 shutdown_connections 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 -- 172.21.15.59:0/902681913 wait complete. 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 librados: starting msgr at 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 librados: starting objecter 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 Processor -- start 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.594+0000 7f9d54f73700 1 -- start start 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 librados: setting wanted keys 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 librados: calling monclient init 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500f7400 0x7f9d50108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500fcbb0 0x7f9d501094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d500fd510 0x7f9d50109af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d50114200 con 0x7f9d500f7400 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d50114370 con 0x7f9d500fd510 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d54f73700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d50114530 con 0x7f9d500fcbb0 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500f7400 0x7f9d50108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500fcbb0 0x7f9d501094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500f7400 0x7f9d50108fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52510/0 (socket says 172.21.15.59:52510) 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500fcbb0 0x7f9d501094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:34058/0 (socket says 172.21.15.59:34058) 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 -- 172.21.15.59:0/2578662525 learned_addr learned my addr 172.21.15.59:0/2578662525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4ed9d700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d500fd510 0x7f9d50109af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 -- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500fcbb0 msgr2=0x7f9d501094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500fcbb0 0x7f9d501094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 -- 172.21.15.59:0/2578662525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d500fd510 msgr2=0x7f9d50109af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d500fd510 0x7f9d50109af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 -- 172.21.15.59:0/2578662525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40005040 con 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4dd9b700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500fcbb0 0x7f9d501094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d4e59c700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500f7400 0x7f9d50108fd0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7f9d40009400 tx=0x7f9d40009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d3f7fe700 10 client.?.objecter ms_handle_connect 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d3f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.595+0000 7f9d3f7fe700 1 -- 172.21.15.59:0/2578662525 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40007a40 con 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.596+0000 7f9d3f7fe700 1 -- 172.21.15.59:0/2578662525 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d4000f040 con 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.596+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480097c0 con 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.596+0000 7f9d3f7fe700 1 -- 172.21.15.59:0/2578662525 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40014a70 con 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.596+0000 7f9d54f73700 10 client.31602.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.596+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d50114b70 con 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.596+0000 7f9d54f73700 1 librados: init done 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5005fa60 con 0x7f9d500f7400 2024-02-18T00:41:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d3f7fe700 1 -- 172.21.15.59:0/2578662525 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9d40010530 con 0x7f9d500f7400 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d3f7fe700 1 --2- 172.21.15.59:0/2578662525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d38062c90 0x7f9d38065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d3f7fe700 1 -- 172.21.15.59:0/2578662525 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(551..551 src has 418..551) v4 ==== 8191+0+0 (secure 0 0 0) 0x7f9d4007ce70 con 0x7f9d500f7400 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d3f7fe700 10 client.31602.objecter ms_dispatch 0x7f9d50001550 osd_map(551..551 src has 418..551) v4 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d3f7fe700 3 client.31602.objecter handle_osd_map got epochs [551,551] > 0 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d4dd9b700 1 --2- 172.21.15.59:0/2578662525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d38062c90 0x7f9d38065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.597+0000 7f9d3f7fe700 3 client.31602.objecter handle_osd_map decoding full epoch 551 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.598+0000 7f9d3f7fe700 20 client.31602.objecter dump_active .. 0 homeless 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.598+0000 7f9d3f7fe700 10 client.31602.objecter ms_handle_connect 0x7f9d38062c90 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.598+0000 7f9d4dd9b700 1 --2- 172.21.15.59:0/2578662525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d38062c90 0x7f9d38065150 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 rx=0x7f9d50061760 tx=0x7f9d4400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.602+0000 7f9d3f7fe700 1 -- 172.21.15.59:0/2578662525 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9d40049f60 con 0x7f9d500f7400 2024-02-18T00:41:26.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:25.793+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f9d50106110 con 0x7f9d500f7400 2024-02-18T00:41:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:24.776104+0000 osd.1 (osd.1) 287 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:41:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:24.779476+0000 osd.1 (osd.1) 288 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:41:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:25.050094+0000 osd.0 (osd.0) 256 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:41:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:25.051143+0000 osd.0 (osd.0) 257 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:41:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:25.744908+0000 mon.a (mon.0) 2903 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:25.749423+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]': finished 2024-02-18T00:41:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:25.749496+0000 mon.a (mon.0) 2905 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-18T00:41:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:25.752529+0000 mon.c (mon.2) 522 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:41:26.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:25.754644+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:41:26.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:25.793555+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? 172.21.15.59:0/2578662525' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:26.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:25.955056+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-18T00:41:26.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.018216+0000 osd.0 (osd.0) 258 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:41:26.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.021711+0000 osd.0 (osd.0) 259 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:41:26.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:24.776104+0000 osd.1 (osd.1) 287 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:41:26.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:24.779476+0000 osd.1 (osd.1) 288 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:41:26.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:25.050094+0000 osd.0 (osd.0) 256 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:41:26.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:25.051143+0000 osd.0 (osd.0) 257 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:41:26.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:25.744908+0000 mon.a (mon.0) 2903 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:26.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:25.749423+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]': finished 2024-02-18T00:41:26.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:25.749496+0000 mon.a (mon.0) 2905 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-18T00:41:26.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:25.752529+0000 mon.c (mon.2) 522 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:41:26.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:25.754644+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:41:26.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:25.793555+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? 172.21.15.59:0/2578662525' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:26.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:25.955056+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-18T00:41:26.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.018216+0000 osd.0 (osd.0) 258 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:41:26.506 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.021711+0000 osd.0 (osd.0) 259 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:41:26.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:24.776104+0000 osd.1 (osd.1) 287 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:41:26.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:24.779476+0000 osd.1 (osd.1) 288 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:41:26.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:25.050094+0000 osd.0 (osd.0) 256 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:41:26.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:25.051143+0000 osd.0 (osd.0) 257 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:41:26.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:25.744908+0000 mon.a (mon.0) 2903 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:26.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:25.749423+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-90", "mode": "writeback"}]': finished 2024-02-18T00:41:26.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:25.749496+0000 mon.a (mon.0) 2905 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-18T00:41:26.508 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:25.752529+0000 mon.c (mon.2) 522 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:41:26.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:25.754644+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:41:26.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:25.793555+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? 172.21.15.59:0/2578662525' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:26.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:25.955056+0000 mon.a (mon.0) 2908 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-18T00:41:26.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.018216+0000 osd.0 (osd.0) 258 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:41:26.509 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.021711+0000 osd.0 (osd.0) 259 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:41:26.544 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:26.547 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:26.548 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:26.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:26.678 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:26.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.759+0000 7f9d3f7fe700 1 -- 172.21.15.59:0/2578662525 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v553) v1 ==== 101+0+0 (secure 0 0 0) 0x7f9d4007ae60 con 0x7f9d500f7400 2024-02-18T00:41:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 10 librados: watch_flush enter 2024-02-18T00:41:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 10 librados: watch_flush exit 2024-02-18T00:41:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 20 client.31602.objecter shutdown clearing up homeless session... 2024-02-18T00:41:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 10 client.31602.objecter successfully canceled tick 2024-02-18T00:41:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d38062c90 msgr2=0x7f9d38065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 1 --2- 172.21.15.59:0/2578662525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d38062c90 0x7f9d38065150 secure :-1 s=READY pgs=3155 cs=0 l=1 rev1=1 rx=0x7f9d50061760 tx=0x7f9d4400b040).stop 2024-02-18T00:41:26.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500f7400 msgr2=0x7f9d50108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.762+0000 7f9d54f73700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500f7400 0x7f9d50108fd0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7f9d40009400 tx=0x7f9d40009cb0).stop 2024-02-18T00:41:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 shutdown_connections 2024-02-18T00:41:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 --2- 172.21.15.59:0/2578662525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d38062c90 0x7f9d38065150 unknown :-1 s=CLOSED pgs=3155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d500fd510 0x7f9d50109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d500f7400 0x7f9d50108fd0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 --2- 172.21.15.59:0/2578662525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d500fcbb0 0x7f9d501094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:26.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 >> 172.21.15.59:0/2578662525 conn(0x7f9d500f2820 msgr2=0x7f9d500f40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:26.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 shutdown_connections 2024-02-18T00:41:26.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 -- 172.21.15.59:0/2578662525 wait complete. 2024-02-18T00:41:26.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:26.763+0000 7f9d54f73700 1 librados: shutdown 2024-02-18T00:41:26.765 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:41:27.108 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:27.110 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:27.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:27.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:27.241 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:27.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:25.817716+0000 osd.1 (osd.1) 289 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:41:27.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:25.818807+0000 osd.1 (osd.1) 290 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:41:27.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:25.953690+0000 mgr.y (mgr.24454) 511 : cluster [DBG] pgmap v843: 237 pgs: 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:27.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.022344+0000 osd.4 (osd.4) 319 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:41:27.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.024163+0000 osd.4 (osd.4) 320 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:41:27.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.746888+0000 mon.a (mon.0) 2909 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:41:27.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:26.758055+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:26.758138+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? 172.21.15.59:0/2578662525' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:26.758208+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.758256+0000 mon.a (mon.0) 2913 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:26.763561+0000 mon.c (mon.2) 523 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:26.763983+0000 mon.a (mon.0) 2914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.996568+0000 osd.0 (osd.0) 260 : cluster [DBG] 62.e scrub starts 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.998043+0000 osd.0 (osd.0) 261 : cluster [DBG] 62.e scrub ok 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:25.817716+0000 osd.1 (osd.1) 289 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:25.818807+0000 osd.1 (osd.1) 290 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:41:27.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:25.953690+0000 mgr.y (mgr.24454) 511 : cluster [DBG] pgmap v843: 237 pgs: 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.022344+0000 osd.4 (osd.4) 319 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.024163+0000 osd.4 (osd.4) 320 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.746888+0000 mon.a (mon.0) 2909 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:26.758055+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:26.758138+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? 172.21.15.59:0/2578662525' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:26.758208+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.758256+0000 mon.a (mon.0) 2913 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:26.763561+0000 mon.c (mon.2) 523 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:26.763983+0000 mon.a (mon.0) 2914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.996568+0000 osd.0 (osd.0) 260 : cluster [DBG] 62.e scrub starts 2024-02-18T00:41:27.374 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.998043+0000 osd.0 (osd.0) 261 : cluster [DBG] 62.e scrub ok 2024-02-18T00:41:27.389 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:27.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:27.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:27.496 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:27.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:25.817716+0000 osd.1 (osd.1) 289 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:41:27.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:25.818807+0000 osd.1 (osd.1) 290 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:25.953690+0000 mgr.y (mgr.24454) 511 : cluster [DBG] pgmap v843: 237 pgs: 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.022344+0000 osd.4 (osd.4) 319 : cluster [DBG] 62.9 scrub starts 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.024163+0000 osd.4 (osd.4) 320 : cluster [DBG] 62.9 scrub ok 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.746888+0000 mon.a (mon.0) 2909 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:26.758055+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:26.758138+0000 mon.a (mon.0) 2911 : audit [INF] from='client.? 172.21.15.59:0/2578662525' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:26.758208+0000 mon.a (mon.0) 2912 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.758256+0000 mon.a (mon.0) 2913 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:26.763561+0000 mon.c (mon.2) 523 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:26.763983+0000 mon.a (mon.0) 2914 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.996568+0000 osd.0 (osd.0) 260 : cluster [DBG] 62.e scrub starts 2024-02-18T00:41:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.998043+0000 osd.0 (osd.0) 261 : cluster [DBG] 62.e scrub ok 2024-02-18T00:41:27.528 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:27.550 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:27.552 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:27.552 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:27.653 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:27.683 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:27.703 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:27.706 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:27.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:27.766 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:27.766 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:27.807 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:27.835 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:27.854 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:27.857 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:27.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:27.958 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:27.989 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:28.012 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:28.013 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:28.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:28.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:28.142 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.593719+0000 osd.5 (osd.5) 243 : cluster [DBG] 6.d scrub starts 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.596662+0000 osd.5 (osd.5) 244 : cluster [DBG] 6.d scrub ok 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.951469+0000 osd.3 (osd.3) 297 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:26.952666+0000 osd.3 (osd.3) 298 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.139154+0000 osd.2 (osd.2) 227 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.140532+0000 osd.2 (osd.2) 228 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.569512+0000 osd.7 (osd.7) 149 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.570434+0000 osd.7 (osd.7) 150 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:27.756996+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:41:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.757082+0000 mon.a (mon.0) 2916 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:27.758816+0000 mon.c (mon.2) 524 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:27.759116+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.593719+0000 osd.5 (osd.5) 243 : cluster [DBG] 6.d scrub starts 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.596662+0000 osd.5 (osd.5) 244 : cluster [DBG] 6.d scrub ok 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.951469+0000 osd.3 (osd.3) 297 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:26.952666+0000 osd.3 (osd.3) 298 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.139154+0000 osd.2 (osd.2) 227 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.140532+0000 osd.2 (osd.2) 228 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.569512+0000 osd.7 (osd.7) 149 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.570434+0000 osd.7 (osd.7) 150 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:41:28.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:27.756996+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:41:28.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.757082+0000 mon.a (mon.0) 2916 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-18T00:41:28.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:27.758816+0000 mon.c (mon.2) 524 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:41:28.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:27.759116+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:41:28.340 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:28.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:28.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:28.367 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:28.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:28.473 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:28.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.593719+0000 osd.5 (osd.5) 243 : cluster [DBG] 6.d scrub starts 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.596662+0000 osd.5 (osd.5) 244 : cluster [DBG] 6.d scrub ok 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.951469+0000 osd.3 (osd.3) 297 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:26.952666+0000 osd.3 (osd.3) 298 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.139154+0000 osd.2 (osd.2) 227 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.140532+0000 osd.2 (osd.2) 228 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.569512+0000 osd.7 (osd.7) 149 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.570434+0000 osd.7 (osd.7) 150 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:27.756996+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.757082+0000 mon.a (mon.0) 2916 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:27.758816+0000 mon.c (mon.2) 524 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:41:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:27.759116+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.920551+0000 osd.3 (osd.3) 299 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.921414+0000 osd.3 (osd.3) 300 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:27.954278+0000 mgr.y (mgr.24454) 512 : cluster [DBG] pgmap v846: 237 pgs: 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:28.731379+0000 mon.a (mon.0) 2918 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:28.736930+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:28.737026+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:28.738143+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T00:41:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:28.739340+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.920551+0000 osd.3 (osd.3) 299 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.921414+0000 osd.3 (osd.3) 300 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:27.954278+0000 mgr.y (mgr.24454) 512 : cluster [DBG] pgmap v846: 237 pgs: 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:28.731379+0000 mon.a (mon.0) 2918 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:28.736930+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:28.737026+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:28.738143+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T00:41:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:28.739340+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.920551+0000 osd.3 (osd.3) 299 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.921414+0000 osd.3 (osd.3) 300 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:27.954278+0000 mgr.y (mgr.24454) 512 : cluster [DBG] pgmap v846: 237 pgs: 237 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:28.731379+0000 mon.a (mon.0) 2918 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:28.736930+0000 mon.a (mon.0) 2919 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:28.737026+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:28.738143+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T00:41:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:28.739340+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T00:41:30.223 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:30.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:30.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:30.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-18T00:41:30.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:30.362 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:30.812 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:30.814 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:30.815 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:30.916 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:30.948 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:30.969 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:30.972 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:30.972 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:30.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:41:30.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:29.045171+0000 osd.4 (osd.4) 321 : cluster [DBG] 4.d deep-scrub starts 2024-02-18T00:41:30.993 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:29.051216+0000 osd.4 (osd.4) 322 : cluster [DBG] 4.d deep-scrub ok 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:29.737690+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:29.737791+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:29.772057+0000 mon.c (mon.2) 526 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:29.772429+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:29.954631+0000 mgr.y (mgr.24454) 513 : cluster [DBG] pgmap v849: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:29.955755+0000 mon.a (mon.0) 2925 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:29.045171+0000 osd.4 (osd.4) 321 : cluster [DBG] 4.d deep-scrub starts 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:29.051216+0000 osd.4 (osd.4) 322 : cluster [DBG] 4.d deep-scrub ok 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:29.737690+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-18T00:41:30.994 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:29.737791+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-18T00:41:30.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:29.772057+0000 mon.c (mon.2) 526 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:30.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:29.772429+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:30.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:29.954631+0000 mgr.y (mgr.24454) 513 : cluster [DBG] pgmap v849: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:30.995 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:29.955755+0000 mon.a (mon.0) 2925 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-18T00:41:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:29.045171+0000 osd.4 (osd.4) 321 : cluster [DBG] 4.d deep-scrub starts 2024-02-18T00:41:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:29.051216+0000 osd.4 (osd.4) 322 : cluster [DBG] 4.d deep-scrub ok 2024-02-18T00:41:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:29.737690+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-18T00:41:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:29.737791+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-18T00:41:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:29.772057+0000 mon.c (mon.2) 526 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:29.772429+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:29.954631+0000 mgr.y (mgr.24454) 513 : cluster [DBG] pgmap v849: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:29.955755+0000 mon.a (mon.0) 2925 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]: dispatch 2024-02-18T00:41:31.074 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:31.101 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:31.398 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:41:31.524 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:31.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:41:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:30.743827+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:41:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:30.743938+0000 mon.a (mon.0) 2927 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-18T00:41:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:30.744012+0000 mon.a (mon.0) 2928 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-18T00:41:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:30.745807+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:30.746120+0000 mon.a (mon.0) 2929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.529825+0000 osd.7 (osd.7) 151 : cluster [DBG] 62.a scrub starts 2024-02-18T00:41:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.531335+0000 osd.7 (osd.7) 152 : cluster [DBG] 62.a scrub ok 2024-02-18T00:41:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:30.743827+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:41:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:30.743938+0000 mon.a (mon.0) 2927 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-18T00:41:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:30.744012+0000 mon.a (mon.0) 2928 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-18T00:41:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:30.745807+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:30.746120+0000 mon.a (mon.0) 2929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.529825+0000 osd.7 (osd.7) 151 : cluster [DBG] 62.a scrub starts 2024-02-18T00:41:32.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.531335+0000 osd.7 (osd.7) 152 : cluster [DBG] 62.a scrub ok 2024-02-18T00:41:32.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:30.743827+0000 mon.a (mon.0) 2926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]': finished 2024-02-18T00:41:32.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:30.743938+0000 mon.a (mon.0) 2927 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "29"}]': finished 2024-02-18T00:41:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:30.744012+0000 mon.a (mon.0) 2928 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-18T00:41:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:30.745807+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:30.746120+0000 mon.a (mon.0) 2929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]: dispatch 2024-02-18T00:41:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.529825+0000 osd.7 (osd.7) 151 : cluster [DBG] 62.a scrub starts 2024-02-18T00:41:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.531335+0000 osd.7 (osd.7) 152 : cluster [DBG] 62.a scrub ok 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.080+0000 7fe885b8f700 1 -- 172.21.15.59:0/1882484321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 msgr2=0x7fe8800f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.080+0000 7fe885b8f700 1 --2- 172.21.15.59:0/1882484321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 0x7fe8800f8d80 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 rx=0x7fe870004870 tx=0x7fe870009cb0).stop 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 -- 172.21.15.59:0/1882484321 shutdown_connections 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 --2- 172.21.15.59:0/1882484321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8800f7330 0x7fe880103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 --2- 172.21.15.59:0/1882484321 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 0x7fe8800f8d80 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 --2- 172.21.15.59:0/1882484321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8800f9270 0x7fe8800f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 -- 172.21.15.59:0/1882484321 >> 172.21.15.59:0/1882484321 conn(0x7fe8800f26f0 msgr2=0x7fe8800f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 -- 172.21.15.59:0/1882484321 shutdown_connections 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 -- 172.21.15.59:0/1882484321 wait complete. 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 librados: starting msgr at 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 librados: starting objecter 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 Processor -- start 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 -- start start 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 librados: setting wanted keys 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.081+0000 7fe885b8f700 1 librados: calling monclient init 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe885b8f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8800f7330 0x7fe88010e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe885b8f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 0x7fe88010e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe885b8f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8800f9270 0x7fe880108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe885b8f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880108800 con 0x7fe8800f7330 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe885b8f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe880108970 con 0x7fe8800f9270 2024-02-18T00:41:32.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe885b8f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe880108b30 con 0x7fe8800f89a0 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8800f7330 0x7fe88010e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8800f7330 0x7fe88010e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43226/0 (socket says 172.21.15.59:43226) 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 -- 172.21.15.59:0/3676818098 learned_addr learned my addr 172.21.15.59:0/3676818098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 0x7fe88010e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8800f9270 0x7fe880108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 -- 172.21.15.59:0/3676818098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 msgr2=0x7fe88010e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 0x7fe88010e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 -- 172.21.15.59:0/3676818098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8800f9270 msgr2=0x7fe880108310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8800f9270 0x7fe880108310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 -- 172.21.15.59:0/3676818098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe874005040 con 0x7fe8800f7330 2024-02-18T00:41:32.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8800f9270 0x7fe880108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8800f89a0 0x7fe88010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8800f7330 0x7fe88010e170 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 rx=0x7fe870009eb0 tx=0x7fe87000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87cff9700 10 client.?.objecter ms_handle_connect 0x7fe8800f7330 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87cff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000e070 con 0x7fe8800f7330 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe870013da0 con 0x7fe8800f7330 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe870005040 con 0x7fe8800f7330 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.082+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe870019070 con 0x7fe8800f7330 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.083+0000 7fe885b8f700 10 client.31608.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.083+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8801090a0 con 0x7fe8800f7330 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.083+0000 7fe885b8f700 1 librados: init done 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.083+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe88005ee80 con 0x7fe8800f7330 2024-02-18T00:41:32.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.084+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe87001e420 con 0x7fe8800f7330 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.084+0000 7fe87cff9700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe868062c90 0x7fe868065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.084+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(558..558 src has 418..558) v4 ==== 8183+0+0 (secure 0 0 0) 0x7fe87007e270 con 0x7fe8800f7330 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.084+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 osd_map(558..558 src has 418..558) v4 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.084+0000 7fe87cff9700 3 client.31608.objecter handle_osd_map got epochs [558,558] > 0 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.084+0000 7fe87cff9700 3 client.31608.objecter handle_osd_map decoding full epoch 558 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.084+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe868062c90 0x7fe868065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe87cff9700 20 client.31608.objecter dump_active .. 0 homeless 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v558) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe87007e710 con 0x7fe8800f7330 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 10 client.31608.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 1 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe868062c90 2024-02-18T00:41:32.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe880028540 0x7fe8800fe890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe868062c90 0x7fe868065150 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 rx=0x7fe880109ab0 tx=0x7fe87400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 1 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 0 1 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880106080 osd=0 4 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 10 client.31608.objecter _send_command 1 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8800fff00 con 0x7fe880028540 2024-02-18T00:41:32.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe880028540 0x7fe8800fe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe880028540 2024-02-18T00:41:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.085+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe880028540 0x7fe8800fe890 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.086+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8800fff00 con 0x7fe880028540 2024-02-18T00:41:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.086+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:41:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.086+0000 7fe87cff9700 10 client.31608.objecter _finish_command 1 = system:0 2024-02-18T00:41:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.086+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880106080 osd=0 4 2024-02-18T00:41:32.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.086+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 0 1 2024-02-18T00:41:32.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 10 client.31608.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 2 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 2 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880106080 osd=0 3 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 0 2 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880106080 osd=0 4 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 10 client.31608.objecter _send_command 2 2024-02-18T00:41:32.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.102+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe88005a040 con 0x7fe880028540 2024-02-18T00:41:32.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe88005a040 con 0x7fe880028540 2024-02-18T00:41:32.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87cff9700 10 client.31608.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880106080 osd=0 4 2024-02-18T00:41:32.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 0 2 2024-02-18T00:41:32.105 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 10 client.31608.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 3 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6805/26279110 conn(0x7fe8800594d0 0x7fe880118fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 3 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 1 3 2024-02-18T00:41:32.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880059360 osd=1 4 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 10 client.31608.objecter _send_command 3 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8800280a0 con 0x7fe8800594d0 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6805/26279110 conn(0x7fe8800594d0 0x7fe880118fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe8800594d0 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.104+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6805/26279110 conn(0x7fe8800594d0 0x7fe880118fc0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.105+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8800280a0 con 0x7fe8800594d0 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.105+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.105+0000 7fe87cff9700 10 client.31608.objecter _finish_command 3 = system:0 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.105+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880059360 osd=1 4 2024-02-18T00:41:32.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.105+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 1 3 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 10 client.31608.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 4 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 4 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880059360 osd=1 3 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 1 4 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880059360 osd=1 4 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 10 client.31608.objecter _send_command 4 2024-02-18T00:41:32.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.122+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe8801194b0 con 0x7fe8800594d0 2024-02-18T00:41:32.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe8801194b0 con 0x7fe8800594d0 2024-02-18T00:41:32.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87cff9700 10 client.31608.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880059360 osd=1 4 2024-02-18T00:41:32.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 1 4 2024-02-18T00:41:32.125 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:41:32.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 10 client.31608.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 5 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe880027770 0x7fe88011c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 5 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 2 5 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880027600 osd=2 4 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 10 client.31608.objecter _send_command 5 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8800ffa60 con 0x7fe880027770 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe880027770 0x7fe88011c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe880027770 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.124+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe880027770 0x7fe88011c7b0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.125+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8800ffa60 con 0x7fe880027770 2024-02-18T00:41:32.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.125+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:41:32.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.125+0000 7fe87cff9700 10 client.31608.objecter _finish_command 5 = system:0 2024-02-18T00:41:32.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.125+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880027600 osd=2 4 2024-02-18T00:41:32.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.125+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 2 5 2024-02-18T00:41:32.129 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:32.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 10 client.31608.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 6 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 6 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880027600 osd=2 3 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 2 6 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880027600 osd=2 4 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 10 client.31608.objecter _send_command 6 2024-02-18T00:41:32.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.151+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe88011cca0 con 0x7fe880027770 2024-02-18T00:41:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.153+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe88011cca0 con 0x7fe880027770 2024-02-18T00:41:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.153+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.153+0000 7fe87cff9700 10 client.31608.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.153+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880027600 osd=2 4 2024-02-18T00:41:32.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.153+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 2 6 2024-02-18T00:41:32.155 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:41:32.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 10 client.31608.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 7 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe8800ff130 0x7fe880120030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 7 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 3 7 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fefc0 osd=3 4 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 10 client.31608.objecter _send_command 7 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe88011a300 con 0x7fe8800ff130 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe8800ff130 0x7fe880120030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe8800ff130 2024-02-18T00:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.154+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe8800ff130 0x7fe880120030 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.155+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe88011a300 con 0x7fe8800ff130 2024-02-18T00:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.155+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.155+0000 7fe87cff9700 10 client.31608.objecter _finish_command 7 = system:0 2024-02-18T00:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.155+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe8800fefc0 osd=3 4 2024-02-18T00:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.155+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 3 7 2024-02-18T00:41:32.157 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:32.160 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:32.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 10 client.31608.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 8 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 8 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe8800fefc0 osd=3 3 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 3 8 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fefc0 osd=3 4 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 10 client.31608.objecter _send_command 8 2024-02-18T00:41:32.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.174+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe880100070 con 0x7fe8800ff130 2024-02-18T00:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.175+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe880100070 con 0x7fe8800ff130 2024-02-18T00:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.175+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.175+0000 7fe87cff9700 10 client.31608.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.175+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe8800fefc0 osd=3 4 2024-02-18T00:41:32.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.175+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 3 8 2024-02-18T00:41:32.177 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:41:32.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 10 client.31608.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 9 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe8800f9cf0 0x7fe880122530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 9 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 4 9 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880100070 osd=4 4 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 10 client.31608.objecter _send_command 9 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8800fb730 con 0x7fe8800f9cf0 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe8800f9cf0 0x7fe880122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe8800f9cf0 2024-02-18T00:41:32.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.176+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe8800f9cf0 0x7fe880122530 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.177+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8800fb730 con 0x7fe8800f9cf0 2024-02-18T00:41:32.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.177+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:41:32.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.177+0000 7fe87cff9700 10 client.31608.objecter _finish_command 9 = system:0 2024-02-18T00:41:32.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.177+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880100070 osd=4 4 2024-02-18T00:41:32.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.177+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 4 9 2024-02-18T00:41:32.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 10 client.31608.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 10 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 10 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe880100070 osd=4 3 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 4 10 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880100070 osd=4 4 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 10 client.31608.objecter _send_command 10 2024-02-18T00:41:32.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.192+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe8800fa150 con 0x7fe8800f9cf0 2024-02-18T00:41:32.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.194+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe8800fa150 con 0x7fe8800f9cf0 2024-02-18T00:41:32.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.194+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.194+0000 7fe87cff9700 10 client.31608.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.194+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe880100070 osd=4 4 2024-02-18T00:41:32.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.194+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 4 10 2024-02-18T00:41:32.195 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:41:32.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 10 client.31608.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 11 2024-02-18T00:41:32.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe8800fa860 0x7fe8800facc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 11 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 5 11 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fa150 osd=5 4 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 10 client.31608.objecter _send_command 11 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8800fb1b0 con 0x7fe8800fa860 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe8800fa860 0x7fe8800facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe8800fa860 2024-02-18T00:41:32.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.195+0000 7fe87effd700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe8800fa860 0x7fe8800facc0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.196+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8800fb1b0 con 0x7fe8800fa860 2024-02-18T00:41:32.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.196+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:41:32.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.196+0000 7fe87cff9700 10 client.31608.objecter _finish_command 11 = system:0 2024-02-18T00:41:32.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.196+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe8800fa150 osd=5 4 2024-02-18T00:41:32.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.196+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 5 11 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 10 client.31608.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 12 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 12 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe8800fa150 osd=5 3 2024-02-18T00:41:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 5 12 2024-02-18T00:41:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fa150 osd=5 4 2024-02-18T00:41:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 10 client.31608.objecter _send_command 12 2024-02-18T00:41:32.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.210+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe8800fb1b0 con 0x7fe8800fa860 2024-02-18T00:41:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe8800fb1b0 con 0x7fe8800fa860 2024-02-18T00:41:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe87cff9700 10 client.31608.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe8800fa150 osd=5 4 2024-02-18T00:41:32.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 5 12 2024-02-18T00:41:32.213 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:41:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe885b8f700 10 client.31608.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 13 2024-02-18T00:41:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.212+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe8800fb400 0x7fe88012b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 13 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 6 13 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fb1b0 osd=6 4 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 10 client.31608.objecter _send_command 13 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe88012e320 con 0x7fe8800fb400 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe8800fb400 0x7fe88012b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe8800fb400 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.213+0000 7fe87ffff700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe8800fb400 0x7fe88012b5c0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.214+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe88012e320 con 0x7fe8800fb400 2024-02-18T00:41:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.214+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:41:32.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.214+0000 7fe87cff9700 10 client.31608.objecter _finish_command 13 = system:0 2024-02-18T00:41:32.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.214+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe8800fb1b0 osd=6 4 2024-02-18T00:41:32.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.214+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 6 13 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 10 client.31608.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 14 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 14 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe8800fb1b0 osd=6 3 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 6 14 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fb1b0 osd=6 4 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 10 client.31608.objecter _send_command 14 2024-02-18T00:41:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.228+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe88012e320 con 0x7fe8800fb400 2024-02-18T00:41:32.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe88012e320 con 0x7fe8800fb400 2024-02-18T00:41:32.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe87cff9700 10 client.31608.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe8800fb1b0 osd=6 4 2024-02-18T00:41:32.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 6 14 2024-02-18T00:41:32.231 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:41:32.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe885b8f700 10 client.31608.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:32.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 15 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6812/53025171 conn(0x7fe88012c100 0x7fe88012c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.230+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 15 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 7 15 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe88012e320 osd=7 4 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe885b8f700 10 client.31608.objecter _send_command 15 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8801316d0 con 0x7fe88012c100 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6812/53025171 conn(0x7fe88012c100 0x7fe88012c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe87cff9700 10 client.31608.objecter ms_handle_connect 0x7fe88012c100 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.231+0000 7fe87f7fe700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6812/53025171 conn(0x7fe88012c100 0x7fe88012c540 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:32.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.232+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8801316d0 con 0x7fe88012c100 2024-02-18T00:41:32.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.232+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:41:32.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.232+0000 7fe87cff9700 10 client.31608.objecter _finish_command 15 = system:0 2024-02-18T00:41:32.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.232+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe88012e320 osd=7 4 2024-02-18T00:41:32.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.232+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 7 15 2024-02-18T00:41:32.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 10 client.31608.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:32.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign -1 16 2024-02-18T00:41:32.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 20 client.31608.objecter _get_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_remove -1 16 2024-02-18T00:41:32.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 20 client.31608.objecter get_session s=0x7fe88012e320 osd=7 3 2024-02-18T00:41:32.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 15 client.31608.objecter _session_command_op_assign 7 16 2024-02-18T00:41:32.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe88012e320 osd=7 4 2024-02-18T00:41:32.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 10 client.31608.objecter _send_command 16 2024-02-18T00:41:32.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.246+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe88012cb20 con 0x7fe88012c100 2024-02-18T00:41:32.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.248+0000 7fe87cff9700 1 -- 172.21.15.59:0/3676818098 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe88012cb20 con 0x7fe88012c100 2024-02-18T00:41:32.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.248+0000 7fe87cff9700 10 client.31608.objecter ms_dispatch 0x7fe880001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:32.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.248+0000 7fe87cff9700 10 client.31608.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.248+0000 7fe87cff9700 20 client.31608.objecter put_session s=0x7fe88012e320 osd=7 4 2024-02-18T00:41:32.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.248+0000 7fe87cff9700 15 client.31608.objecter _session_command_op_remove 7 16 2024-02-18T00:41:32.250 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:41:32.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 librados: watch_flush enter 2024-02-18T00:41:32.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 librados: watch_flush exit 2024-02-18T00:41:32.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.0 2024-02-18T00:41:32.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe880028540 msgr2=0x7fe8800fe890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe880028540 0x7fe8800fe890 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880106080 osd=0 1 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.1 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6805/26279110 conn(0x7fe8800594d0 msgr2=0x7fe880118fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6805/26279110 conn(0x7fe8800594d0 0x7fe880118fc0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880059360 osd=1 1 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.2 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe880027770 msgr2=0x7fe88011c7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe880027770 0x7fe88011c7b0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880027600 osd=2 1 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.3 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe87ffff700 1 -- 172.21.15.59:0/3676818098 reap_dead start 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe8800ff130 msgr2=0x7fe880120030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe8800ff130 0x7fe880120030 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fefc0 osd=3 1 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.4 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe8800f9cf0 msgr2=0x7fe880122530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe8800f9cf0 0x7fe880122530 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe880100070 osd=4 1 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.5 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe8800fa860 msgr2=0x7fe8800facc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe8800fa860 0x7fe8800facc0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fa150 osd=5 1 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.6 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe8800fb400 msgr2=0x7fe88012b5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe8800fb400 0x7fe88012b5c0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe8800fb1b0 osd=6 1 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter close_session for osd.7 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6812/53025171 conn(0x7fe88012c100 msgr2=0x7fe88012c540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.137:6812/53025171 conn(0x7fe88012c100 0x7fe88012c540 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter put_session s=0x7fe88012e320 osd=7 1 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 20 client.31608.objecter shutdown clearing up homeless session... 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 10 client.31608.objecter successfully canceled tick 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe868062c90 msgr2=0x7fe868065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe868062c90 0x7fe868065150 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 rx=0x7fe880109ab0 tx=0x7fe87400b040).stop 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8800f7330 msgr2=0x7fe88010e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 --2- 172.21.15.59:0/3676818098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8800f7330 0x7fe88010e170 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 rx=0x7fe870009eb0 tx=0x7fe87000b040).stop 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe87ffff700 1 -- 172.21.15.59:0/3676818098 reap_dead start 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe87ffff700 1 -- 172.21.15.59:0/3676818098 reap_dead start 2024-02-18T00:41:32.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 shutdown_connections 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 >> 172.21.15.59:0/3676818098 conn(0x7fe8800f26f0 msgr2=0x7fe880061680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 shutdown_connections 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 -- 172.21.15.59:0/3676818098 wait complete. 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:32.249+0000 7fe885b8f700 1 librados: shutdown 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.255 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:41:32.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:32.291 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:32.487 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:32] "GET /metrics HTTP/1.1" 200 190602 "" "Prometheus/2.33.4" 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.056293+0000 osd.4 (osd.4) 323 : cluster [DBG] 25.b scrub starts 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.059281+0000 osd.4 (osd.4) 324 : cluster [DBG] 25.b scrub ok 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:31.397557+0000 mgr.y (mgr.24454) 514 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.661825+0000 osd.5 (osd.5) 245 : cluster [DBG] 4.f scrub starts 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.665424+0000 osd.5 (osd.5) 246 : cluster [DBG] 4.f scrub ok 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:31.743422+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]': finished 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.743506+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-18T00:41:33.123 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.788181+0000 osd.1 (osd.1) 291 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.791113+0000 osd.1 (osd.1) 292 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:31.955261+0000 mgr.y (mgr.24454) 515 : cluster [DBG] pgmap v852: 236 pgs: 2 peering, 234 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:32.084185+0000 mon.a (mon.0) 2932 : audit [DBG] from='client.? 172.21.15.59:0/3676818098' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.056293+0000 osd.4 (osd.4) 323 : cluster [DBG] 25.b scrub starts 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.059281+0000 osd.4 (osd.4) 324 : cluster [DBG] 25.b scrub ok 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:31.397557+0000 mgr.y (mgr.24454) 514 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.661825+0000 osd.5 (osd.5) 245 : cluster [DBG] 4.f scrub starts 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.665424+0000 osd.5 (osd.5) 246 : cluster [DBG] 4.f scrub ok 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:31.743422+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]': finished 2024-02-18T00:41:33.124 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.743506+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-18T00:41:33.125 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.788181+0000 osd.1 (osd.1) 291 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:41:33.125 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.791113+0000 osd.1 (osd.1) 292 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:41:33.125 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:31.955261+0000 mgr.y (mgr.24454) 515 : cluster [DBG] pgmap v852: 236 pgs: 2 peering, 234 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:41:33.125 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:32.084185+0000 mon.a (mon.0) 2932 : audit [DBG] from='client.? 172.21.15.59:0/3676818098' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:33.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.056293+0000 osd.4 (osd.4) 323 : cluster [DBG] 25.b scrub starts 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.059281+0000 osd.4 (osd.4) 324 : cluster [DBG] 25.b scrub ok 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:31.397557+0000 mgr.y (mgr.24454) 514 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.661825+0000 osd.5 (osd.5) 245 : cluster [DBG] 4.f scrub starts 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.665424+0000 osd.5 (osd.5) 246 : cluster [DBG] 4.f scrub ok 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:31.743422+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-90"}]': finished 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.743506+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.788181+0000 osd.1 (osd.1) 291 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.791113+0000 osd.1 (osd.1) 292 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:31.955261+0000 mgr.y (mgr.24454) 515 : cluster [DBG] pgmap v852: 236 pgs: 2 peering, 234 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:41:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:32.084185+0000 mon.a (mon.0) 2932 : audit [DBG] from='client.? 172.21.15.59:0/3676818098' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:33.941 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:33.944 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:33.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:34.045 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:34.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:32.115001+0000 osd.0 (osd.0) 262 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:41:34.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:32.116357+0000 osd.0 (osd.0) 263 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:41:34.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:32.149116+0000 osd.2 (osd.2) 229 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:41:34.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:32.149981+0000 osd.2 (osd.2) 230 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:41:34.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:32.761008+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-18T00:41:34.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:32.875574+0000 osd.3 (osd.3) 301 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:41:34.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:32.876853+0000 osd.3 (osd.3) 302 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:41:34.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:32.115001+0000 osd.0 (osd.0) 262 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:41:34.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:32.116357+0000 osd.0 (osd.0) 263 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:41:34.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:32.149116+0000 osd.2 (osd.2) 229 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:41:34.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:32.149981+0000 osd.2 (osd.2) 230 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:41:34.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:32.761008+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-18T00:41:34.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:32.875574+0000 osd.3 (osd.3) 301 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:41:34.075 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:32.876853+0000 osd.3 (osd.3) 302 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:41:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:32.115001+0000 osd.0 (osd.0) 262 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:41:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:32.116357+0000 osd.0 (osd.0) 263 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:41:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:32.149116+0000 osd.2 (osd.2) 229 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:41:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:32.149981+0000 osd.2 (osd.2) 230 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:41:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:32.761008+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-18T00:41:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:32.875574+0000 osd.3 (osd.3) 301 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:41:34.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:32.876853+0000 osd.3 (osd.3) 302 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:41:34.426 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:34.429 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:34.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:34.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:34.560 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:35.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:33.806439+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:33.830051+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:33.830814+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:33.955622+0000 mgr.y (mgr.24454) 516 : cluster [DBG] pgmap v855: 235 pgs: 32 unknown, 1 peering, 202 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:33.956497+0000 mon.a (mon.0) 2936 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.090886+0000 osd.4 (osd.4) 325 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.091908+0000 osd.4 (osd.4) 326 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.508210+0000 osd.7 (osd.7) 153 : cluster [DBG] 2.b scrub starts 2024-02-18T00:41:35.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.509179+0000 osd.7 (osd.7) 154 : cluster [DBG] 2.b scrub ok 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:33.806439+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:33.830051+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:33.830814+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:33.955622+0000 mgr.y (mgr.24454) 516 : cluster [DBG] pgmap v855: 235 pgs: 32 unknown, 1 peering, 202 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:33.956497+0000 mon.a (mon.0) 2936 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.090886+0000 osd.4 (osd.4) 325 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.091908+0000 osd.4 (osd.4) 326 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.508210+0000 osd.7 (osd.7) 153 : cluster [DBG] 2.b scrub starts 2024-02-18T00:41:35.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.509179+0000 osd.7 (osd.7) 154 : cluster [DBG] 2.b scrub ok 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:33.806439+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:33.830051+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:33.830814+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:33.955622+0000 mgr.y (mgr.24454) 516 : cluster [DBG] pgmap v855: 235 pgs: 32 unknown, 1 peering, 202 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:33.956497+0000 mon.a (mon.0) 2936 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]: dispatch 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.090886+0000 osd.4 (osd.4) 325 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.091908+0000 osd.4 (osd.4) 326 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.508210+0000 osd.7 (osd.7) 153 : cluster [DBG] 2.b scrub starts 2024-02-18T00:41:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.509179+0000 osd.7 (osd.7) 154 : cluster [DBG] 2.b scrub ok 2024-02-18T00:41:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.448+0000 7f2d09132700 1 -- 172.21.15.59:0/3468930822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 msgr2=0x7f2d04104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.448+0000 7f2d09132700 1 --2- 172.21.15.59:0/3468930822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 0x7f2d04104170 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 rx=0x7f2cf80047b0 tx=0x7f2cf8009cb0).stop 2024-02-18T00:41:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.448+0000 7f2d09132700 1 -- 172.21.15.59:0/3468930822 shutdown_connections 2024-02-18T00:41:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.448+0000 7f2d09132700 1 --2- 172.21.15.59:0/3468930822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d04104660 0x7f2d04106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.448+0000 7f2d09132700 1 --2- 172.21.15.59:0/3468930822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d040f6b00 0x7f2d040f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.448+0000 7f2d09132700 1 --2- 172.21.15.59:0/3468930822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 0x7f2d04104170 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.448+0000 7f2d09132700 1 -- 172.21.15.59:0/3468930822 >> 172.21.15.59:0/3468930822 conn(0x7f2d040f2820 msgr2=0x7f2d040f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.449+0000 7f2d09132700 1 -- 172.21.15.59:0/3468930822 shutdown_connections 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.449+0000 7f2d09132700 1 -- 172.21.15.59:0/3468930822 wait complete. 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.449+0000 7f2d09132700 1 librados: starting msgr at 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.449+0000 7f2d09132700 1 librados: starting objecter 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.449+0000 7f2d09132700 1 Processor -- start 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 -- start start 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 librados: setting wanted keys 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 librados: calling monclient init 2024-02-18T00:41:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d040f6b00 0x7f2d041103f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 0x7f2d041108e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d04104660 0x7f2d0410a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0410aad0 con 0x7f2d04104660 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0410ac40 con 0x7f2d040f6b00 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d09132700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0410ae00 con 0x7f2d040f73d0 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d04104660 0x7f2d0410a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0259c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 0x7f2d041108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d04104660 0x7f2d0410a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43242/0 (socket says 172.21.15.59:43242) 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 -- 172.21.15.59:0/1319117393 learned_addr learned my addr 172.21.15.59:0/1319117393 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d02d9d700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d040f6b00 0x7f2d041103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 -- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 msgr2=0x7f2d041108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 0x7f2d041108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 -- 172.21.15.59:0/1319117393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d040f6b00 msgr2=0x7f2d041103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d040f6b00 0x7f2d041103f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 -- 172.21.15.59:0/1319117393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4005040 con 0x7f2d04104660 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0259c700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 0x7f2d041108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2d0359e700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d04104660 0x7f2d0410a5e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 rx=0x7f2cfc00d940 tx=0x7f2cfc004e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2cebfff700 10 client.?.objecter ms_handle_connect 0x7f2d04104660 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.450+0000 7f2cebfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.451+0000 7f2cebfff700 1 -- 172.21.15.59:0/1319117393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc004020 con 0x7f2d04104660 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.451+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf8005040 con 0x7f2d04104660 2024-02-18T00:41:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.451+0000 7f2cebfff700 1 -- 172.21.15.59:0/1319117393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cfc00f040 con 0x7f2d04104660 2024-02-18T00:41:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.451+0000 7f2cebfff700 1 -- 172.21.15.59:0/1319117393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0036c0 con 0x7f2d04104660 2024-02-18T00:41:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.451+0000 7f2d09132700 10 client.31614.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.451+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0410b440 con 0x7f2d04104660 2024-02-18T00:41:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.451+0000 7f2d09132700 1 librados: init done 2024-02-18T00:41:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.452+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf0004f50 con 0x7f2d04104660 2024-02-18T00:41:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.452+0000 7f2cebfff700 1 -- 172.21.15.59:0/1319117393 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2cfc004180 con 0x7f2d04104660 2024-02-18T00:41:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2cebfff700 1 --2- 172.21.15.59:0/1319117393 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2cec062c90 0x7f2cec065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2cebfff700 1 -- 172.21.15.59:0/1319117393 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(561..561 src has 418..561) v4 ==== 8183+0+0 (secure 0 0 0) 0x7f2cfc07ca90 con 0x7f2d04104660 2024-02-18T00:41:35.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2cebfff700 10 client.31614.objecter ms_dispatch 0x7f2d04001550 osd_map(561..561 src has 418..561) v4 2024-02-18T00:41:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2cebfff700 3 client.31614.objecter handle_osd_map got epochs [561,561] > 0 2024-02-18T00:41:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2cebfff700 3 client.31614.objecter handle_osd_map decoding full epoch 561 2024-02-18T00:41:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2d02d9d700 1 --2- 172.21.15.59:0/1319117393 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2cec062c90 0x7f2cec065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2cebfff700 20 client.31614.objecter dump_active .. 0 homeless 2024-02-18T00:41:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2cebfff700 10 client.31614.objecter ms_handle_connect 0x7f2cec062c90 2024-02-18T00:41:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.453+0000 7f2d02d9d700 1 --2- 172.21.15.59:0/1319117393 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2cec062c90 0x7f2cec065150 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 rx=0x7f2cf4004870 tx=0x7f2cf400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.457+0000 7f2cebfff700 1 -- 172.21.15.59:0/1319117393 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2cfc048f60 con 0x7f2d04104660 2024-02-18T00:41:35.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.629+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2cf0005200 con 0x7f2d04104660 2024-02-18T00:41:35.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.631+0000 7f2cebfff700 1 -- 172.21.15.59:0/1319117393 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v561) v1 ==== 81+0+48949 (secure 0 0 0) 0x7f2cfc04fe80 con 0x7f2d04104660 2024-02-18T00:41:35.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:41:35.633 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 561, 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-02-18T00:23:11.390869+0000", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-02-18T00:41:34.766909+0000", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-02-18T00:29:06.503960+0000", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-02-18T00:39:35.702780+0000", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5799936, 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-02-18T00:41:35.634 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 99, 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 228, 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 8, 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "pacific", 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-02-18T00:41:35.635 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "device_health_metrics", 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:27:02.909667+0000", 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.636 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "67", 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.638 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:41:35.640 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_devicehealth": {} 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".rgw.root", 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:13.697458+0000", 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.641 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 42, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 42, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 42, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 42, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 42, 2024-02-18T00:41:35.642 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "170", 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "107", 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.643 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.644 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.645 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "iscsi", 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:14.132992+0000", 2024-02-18T00:41:35.646 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 3, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 3, 2024-02-18T00:41:35.647 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 3, 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 3, 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 3, 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "60", 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.648 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.649 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.650 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 4, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.log", 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:15.191206+0000", 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.652 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.653 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "60", 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.654 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:41:35.656 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 5, 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.control", 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:17.526023+0000", 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.657 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.658 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "62", 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.659 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.660 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.661 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 6, 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.meta", 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:19.600478+0000", 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.662 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 29, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 22, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 22, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 22, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 28, 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.663 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "6.1d", 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 559, 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 558, 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 558, 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "561", 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "561", 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "561", 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.664 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.665 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.666 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_bias": 4 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 25, 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "ceph_test_rados_api_asio", 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:32:59.546190+0000", 2024-02-18T00:41:35.667 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:41:35.668 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "111", 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:41:35.669 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.670 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.671 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 62, 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-6", 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:33:19.569877+0000", 2024-02-18T00:41:35.672 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:41:35.673 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "558", 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "557", 2024-02-18T00:41:35.674 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "557", 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "557", 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 38, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 550, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.675 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.676 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "fingerprint_algorithm": "sha1" 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: "rados": {} 2024-02-18T00:41:35.677 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 228, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-92", 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:41:33.216727+0000", 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 32769, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,creating", 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:41:35.678 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:41:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "561", 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:41:35.680 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:41:35.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:41:35.682 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "rados": {} 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85", 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.683 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 8, 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.684 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:41:35.685 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/3700238977", 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6802/3700238977", 2024-02-18T00:41:35.686 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6804/3700238977", 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6803/3700238977", 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0", 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.687 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 13, 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.688 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.689 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.690 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6805/26279110", 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6806/26279110", 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6808/26279110", 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6807/26279110", 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7", 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.691 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 18, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:41:35.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.693 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:41:35.694 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1153193637", 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6810/1153193637", 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6812/1153193637", 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6811/1153193637", 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b", 2024-02-18T00:41:35.695 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 26, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.696 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-02-18T00:41:35.698 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6813/1805700964", 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6814/1805700964", 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6816/1805700964", 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6815/1805700964", 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.699 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231", 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 31, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.700 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6800", 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6801", 2024-02-18T00:41:35.701 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6803", 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6802", 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6800/2044387056", 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6801/2044387056", 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6803/2044387056", 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6802/2044387056", 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.703 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01", 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 39, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.704 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6804", 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.705 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6805", 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6807", 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.706 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6806", 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6804/2170432263", 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6805/2170432263", 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6807/2170432263", 2024-02-18T00:41:35.707 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6806/2170432263", 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f", 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.708 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 47, 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6808", 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6809", 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6811", 2024-02-18T00:41:35.710 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6810", 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6808/3356009390", 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6809/3356009390", 2024-02-18T00:41:35.711 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6811/3356009390", 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6810/3356009390", 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb", 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:41:35.712 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 53, 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 560, 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6812", 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.713 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6813", 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6815", 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6814", 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.715 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6812/53025171", 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6813/53025171", 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6815/53025171", 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6814/53025171", 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:41:35.716 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:41:00.443932+0000", 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.717 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:40:41.581937+0000", 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:40:47.407406+0000", 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:41:35.718 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:40:51.721842+0000", 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.719 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:41:05.609275+0000", 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:41:12.379645+0000", 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.720 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:40:46.260015+0000", 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:41:35.721 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:40:49.853221+0000", 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [], 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [ 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.0", 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:41:35.722 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.3", 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.5", 2024-02-18T00:41:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.8", 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.18", 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.2", 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.4", 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.b", 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.e", 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.727 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 0 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.f", 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.17", 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "5.1b", 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:41:35.729 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 6 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.9", 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 4 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.11", 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "62.8", 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:41:35.731 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": { 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/1243102711": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/817089002": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3174805070": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/3506000666": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/4167740619": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/999852549": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:41:35.732 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3575161033": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/2403979355": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/642821245": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/1656814063": "2024-02-19T00:23:45.307159+0000" 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "ruleset-failure-domain": "osd", 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [ 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 11, 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 3, 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.734 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 14, 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 9, 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.735 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 60, 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 78, 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 86, 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.737 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 94, 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 97, 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 4, 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 4 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 108, 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 114, 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 128, 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 10, 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.741 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 134, 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 12, 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 154, 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 14, 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 166, 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 17, 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 177, 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 20, 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-02-18T00:41:35.744 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:35.745 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:41:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 10 librados: watch_flush enter 2024-02-18T00:41:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 10 librados: watch_flush exit 2024-02-18T00:41:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 20 client.31614.objecter shutdown clearing up homeless session... 2024-02-18T00:41:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 10 client.31614.objecter successfully canceled tick 2024-02-18T00:41:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2cec062c90 msgr2=0x7f2cec065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 --2- 172.21.15.59:0/1319117393 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2cec062c90 0x7f2cec065150 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 rx=0x7f2cf4004870 tx=0x7f2cf400b040).stop 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d04104660 msgr2=0x7f2d0410a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d04104660 0x7f2d0410a5e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 rx=0x7f2cfc00d940 tx=0x7f2cfc004e40).stop 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 shutdown_connections 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 --2- 172.21.15.59:0/1319117393 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2cec062c90 0x7f2cec065150 unknown :-1 s=CLOSED pgs=3157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d040f6b00 0x7f2d041103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d04104660 0x7f2d0410a5e0 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 --2- 172.21.15.59:0/1319117393 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2d040f73d0 0x7f2d041108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 >> 172.21.15.59:0/1319117393 conn(0x7f2d040f2820 msgr2=0x7f2d040f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 shutdown_connections 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 -- 172.21.15.59:0/1319117393 wait complete. 2024-02-18T00:41:35.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:35.633+0000 7f2d09132700 1 librados: shutdown 2024-02-18T00:41:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.740415+0000 osd.6 (osd.6) 265 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.741613+0000 osd.6 (osd.6) 266 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:34.776936+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:34.776989+0000 mon.a (mon.0) 2938 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.777016+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:34.789831+0000 mon.c (mon.2) 529 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:34.796548+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:34.796941+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.819801+0000 osd.3 (osd.3) 303 : cluster [DBG] 62.c scrub starts 2024-02-18T00:41:36.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:34.821436+0000 osd.3 (osd.3) 304 : cluster [DBG] 62.c scrub ok 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:35.075833+0000 osd.4 (osd.4) 327 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:35.079373+0000 osd.4 (osd.4) 328 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:35.631398+0000 mon.a (mon.0) 2941 : audit [DBG] from='client.? 172.21.15.59:0/1319117393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.740415+0000 osd.6 (osd.6) 265 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.741613+0000 osd.6 (osd.6) 266 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:34.776936+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:34.776989+0000 mon.a (mon.0) 2938 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.777016+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:34.789831+0000 mon.c (mon.2) 529 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:41:36.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:34.796548+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:36.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:34.796941+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:36.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.819801+0000 osd.3 (osd.3) 303 : cluster [DBG] 62.c scrub starts 2024-02-18T00:41:36.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:34.821436+0000 osd.3 (osd.3) 304 : cluster [DBG] 62.c scrub ok 2024-02-18T00:41:36.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:35.075833+0000 osd.4 (osd.4) 327 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:41:36.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:35.079373+0000 osd.4 (osd.4) 328 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:41:36.006 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:35.631398+0000 mon.a (mon.0) 2941 : audit [DBG] from='client.? 172.21.15.59:0/1319117393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:41:36.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.740415+0000 osd.6 (osd.6) 265 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.741613+0000 osd.6 (osd.6) 266 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:34.776936+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:34.776989+0000 mon.a (mon.0) 2938 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "28"}]': finished 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.777016+0000 mon.a (mon.0) 2939 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:34.789831+0000 mon.c (mon.2) 529 : audit [DBG] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:34.796548+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:34.796941+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.819801+0000 osd.3 (osd.3) 303 : cluster [DBG] 62.c scrub starts 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:34.821436+0000 osd.3 (osd.3) 304 : cluster [DBG] 62.c scrub ok 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:35.075833+0000 osd.4 (osd.4) 327 : cluster [DBG] 62.1b scrub starts 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:35.079373+0000 osd.4 (osd.4) 328 : cluster [DBG] 62.1b scrub ok 2024-02-18T00:41:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:35.631398+0000 mon.a (mon.0) 2941 : audit [DBG] from='client.? 172.21.15.59:0/1319117393' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:41:36.602 INFO:tasks.thrashosds.thrasher:j is {'epoch': 561, 'fsid': 'd129b4f8-cdf3-11ee-95ba-87774f69a715', 'created': '2024-02-18T00:23:11.390869+0000', 'modified': '2024-02-18T00:41:34.766909+0000', 'last_up_change': '2024-02-18T00:29:06.503960+0000', 'last_in_change': '2024-02-18T00:39:35.702780+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 99, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 228, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-18T00:27:02.909667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-02-18T00:29:13.697458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 42, 'pg_placement_num': 42, 'pg_placement_num_target': 42, 'pg_num_target': 42, 'pg_num_pending': 42, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '107', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'iscsi', 'create_time': '2024-02-18T00:29:14.132992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-18T00:29:15.191206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-18T00:29:17.526023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-18T00:29:19.600478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 29, 'pg_placement_num': 22, 'pg_placement_num_target': 22, 'pg_num_target': 22, 'pg_num_pending': 28, 'last_pg_merge_meta': {'source_pgid': '6.1d', 'ready_epoch': 559, 'last_epoch_started': 558, 'last_epoch_clean': 558, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '561', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '561', 'last_force_op_resend_preluminous': '561', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 25, 'pool_name': 'ceph_test_rados_api_asio', 'create_time': '2024-02-18T00:32:59.546190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 62, 'pool_name': 'test-rados-api-smithi059-82957-6', 'create_time': '2024-02-18T00:33:19.569877+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '558', 'last_force_op_resend': '557', 'last_force_op_resend_prenautilus': '557', 'last_force_op_resend_preluminous': '557', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 38, 'snap_epoch': 550, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'fingerprint_algorithm': 'sha1'}, 'application_metadata': {'rados': {}}}, {'pool': 228, 'pool_name': 'test-rados-api-smithi059-82957-92', 'create_time': '2024-02-18T00:41:33.216727+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '561', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '6e729102-80e2-4a4e-92ca-600ee8b1ed85', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6801', 'nonce': 3700238977}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 3700238977}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 3700238977}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6803', 'nonce': 3700238977}]}, 'public_addr': '172.21.15.59:6801/3700238977', 'cluster_addr': '172.21.15.59:6802/3700238977', 'heartbeat_back_addr': '172.21.15.59:6804/3700238977', 'heartbeat_front_addr': '172.21.15.59:6803/3700238977', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f67a7206-6d31-497f-97e0-672d134b13d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6805', 'nonce': 26279110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 26279110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 26279110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6807', 'nonce': 26279110}]}, 'public_addr': '172.21.15.59:6805/26279110', 'cluster_addr': '172.21.15.59:6806/26279110', 'heartbeat_back_addr': '172.21.15.59:6808/26279110', 'heartbeat_front_addr': '172.21.15.59:6807/26279110', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6223dbd-e539-4770-9e5a-7b756304f8f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6809', 'nonce': 1153193637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1153193637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1153193637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6811', 'nonce': 1153193637}]}, 'public_addr': '172.21.15.59:6809/1153193637', 'cluster_addr': '172.21.15.59:6810/1153193637', 'heartbeat_back_addr': '172.21.15.59:6812/1153193637', 'heartbeat_front_addr': '172.21.15.59:6811/1153193637', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fff275df-95e0-41f6-96c3-e531f05de32b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6813', 'nonce': 1805700964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1805700964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 1805700964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6815', 'nonce': 1805700964}]}, 'public_addr': '172.21.15.59:6813/1805700964', 'cluster_addr': '172.21.15.59:6814/1805700964', 'heartbeat_back_addr': '172.21.15.59:6816/1805700964', 'heartbeat_front_addr': '172.21.15.59:6815/1805700964', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a0bae50b-4f29-4310-ac5f-bc7285464231', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6800', 'nonce': 2044387056}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6801', 'nonce': 2044387056}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6803', 'nonce': 2044387056}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6802', 'nonce': 2044387056}]}, 'public_addr': '172.21.15.137:6800/2044387056', 'cluster_addr': '172.21.15.137:6801/2044387056', 'heartbeat_back_addr': '172.21.15.137:6803/2044387056', 'heartbeat_front_addr': '172.21.15.137:6802/2044387056', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '513a4192-45fb-4444-a1f9-4bd2f2a5fa01', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6804', 'nonce': 2170432263}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6805', 'nonce': 2170432263}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6807', 'nonce': 2170432263}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6806', 'nonce': 2170432263}]}, 'public_addr': '172.21.15.137:6804/2170432263', 'cluster_addr': '172.21.15.137:6805/2170432263', 'heartbeat_back_addr': '172.21.15.137:6807/2170432263', 'heartbeat_front_addr': '172.21.15.137:6806/2170432263', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5bbb7683-ea5a-490c-8c8b-67b53c45406f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6808', 'nonce': 3356009390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6809', 'nonce': 3356009390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6811', 'nonce': 3356009390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6810', 'nonce': 3356009390}]}, 'public_addr': '172.21.15.137:6808/3356009390', 'cluster_addr': '172.21.15.137:6809/3356009390', 'heartbeat_back_addr': '172.21.15.137:6811/3356009390', 'heartbeat_front_addr': '172.21.15.137:6810/3356009390', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '951d317b-7a34-41a0-b1a1-a2851b2070bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 560, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6812', 'nonce': 53025171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6813', 'nonce': 53025171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6815', 'nonce': 53025171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6814', 'nonce': 53025171}]}, 'public_addr': '172.21.15.137:6812/53025171', 'cluster_addr': '172.21.15.137:6813/53025171', 'heartbeat_back_addr': '172.21.15.137:6815/53025171', 'heartbeat_front_addr': '172.21.15.137:6814/53025171', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:41:00.443932+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:40:41.581937+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:40:47.407406+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:40:51.721842+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:41:05.609275+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:41:12.379645+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:40:46.260015+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:40:49.853221+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.3', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '2.5', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.8', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.18', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.2', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.4', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 0}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.1b', 'mappings': [{'from': 0, 'to': 6}]}, {'pgid': '25.9', 'mappings': [{'from': 0, 'to': 4}]}, {'pgid': '25.11', 'mappings': [{'from': 0, 'to': 2}]}, {'pgid': '62.8', 'mappings': [{'from': 7, 'to': 5}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.59:0/1243102711': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/817089002': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/3174805070': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/3506000666': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/4167740619': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/999852549': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/3575161033': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:0/2403979355': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/642821245': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:6800/1656814063': '2024-02-19T00:23:45.307159+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 11, 'snaps': [{'begin': 3, 'length': 1}, {'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 14, 'snaps': [{'begin': 9, 'length': 2}]}, {'pool': 60, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 78, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 86, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 94, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 97, 'snaps': [{'begin': 4, 'length': 4}]}, {'pool': 108, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 114, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 128, 'snaps': [{'begin': 2, 'length': 2}, {'begin': 6, 'length': 2}, {'begin': 10, 'length': 2}]}, {'pool': 134, 'snaps': [{'begin': 12, 'length': 1}]}, {'pool': 154, 'snaps': [{'begin': 14, 'length': 1}]}, {'pool': 166, 'snaps': [{'begin': 17, 'length': 1}]}, {'pool': 177, 'snaps': [{'begin': 20, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-18T00:41:36.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:41:36.965 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:35.738049+0000 osd.6 (osd.6) 267 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:35.739041+0000 osd.6 (osd.6) 268 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:35.776924+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:35.777000+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:35.778114+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:35.786637+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:35.956799+0000 mgr.y (mgr.24454) 517 : cluster [DBG] pgmap v858: 235 pgs: 2 clean+premerge+peered, 233 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.040116+0000 osd.4 (osd.4) 329 : cluster [DBG] 4.e scrub starts 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.043571+0000 osd.4 (osd.4) 330 : cluster [DBG] 4.e scrub ok 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.513701+0000 osd.7 (osd.7) 155 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:41:36.966 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.515089+0000 osd.7 (osd.7) 156 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:35.738049+0000 osd.6 (osd.6) 267 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:35.739041+0000 osd.6 (osd.6) 268 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:35.776924+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:35.777000+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:35.778114+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:35.786637+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:35.956799+0000 mgr.y (mgr.24454) 517 : cluster [DBG] pgmap v858: 235 pgs: 2 clean+premerge+peered, 233 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.040116+0000 osd.4 (osd.4) 329 : cluster [DBG] 4.e scrub starts 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.043571+0000 osd.4 (osd.4) 330 : cluster [DBG] 4.e scrub ok 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.513701+0000 osd.7 (osd.7) 155 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:41:36.967 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.515089+0000 osd.7 (osd.7) 156 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:35.738049+0000 osd.6 (osd.6) 267 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:35.739041+0000 osd.6 (osd.6) 268 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:35.776924+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:35.777000+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:35.778114+0000 mon.c (mon.2) 531 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:35.786637+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:35.956799+0000 mgr.y (mgr.24454) 517 : cluster [DBG] pgmap v858: 235 pgs: 2 clean+premerge+peered, 233 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.040116+0000 osd.4 (osd.4) 329 : cluster [DBG] 4.e scrub starts 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.043571+0000 osd.4 (osd.4) 330 : cluster [DBG] 4.e scrub ok 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.513701+0000 osd.7 (osd.7) 155 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:41:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.515089+0000 osd.7 (osd.7) 156 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24818.0:6 27.11 27:8dd16f86:::obj3:head [stat] snapc 0=[] ondisk+read+known_if_redirected e107)", 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:41:37.701 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 514.77642155700005, 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7480210029999999, 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24818", 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1844909298", 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:41:37.702 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921758+0000", 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921757+0000", 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2109999999999997e-06 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.703 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921762+0000", 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9610000000000005e-06 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921770+0000", 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.0260000000000002e-06 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921777+0000", 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4104000000000003e-05 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.704 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.705 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.705 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921821+0000", 2024-02-18T00:41:37.705 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3690000000000003e-06 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.921827+0000", 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7478305119999999 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.669658+0000", 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012113200000000001 2024-02-18T00:41:37.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.669779+0000", 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7480210029999999 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:2061 17.4 17:238a5d4d:test-rados-api-smithi059-82949-16::big.copy:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 513.95621583100001, 2024-02-18T00:41:37.707 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.082181671999999997, 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2061 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:41:37.708 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741964+0000", 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741963+0000", 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2389999999999999e-06 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.709 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741969+0000", 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0874e-05 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741980+0000", 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1250000000000001e-06 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.741985+0000", 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0633999999999998e-05 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.710 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.742025+0000", 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0579000000000001e-05 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.742116+0000", 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.069955422000000003 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,4", 2024-02-18T00:41:37.711 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.812071+0000", 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011236616 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.823308+0000", 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00026764999999999997 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.823576+0000", 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000464523 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824040+0000", 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.995e-05 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824100+0000", 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.5300000000000003e-05 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.824146+0000", 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.082181671999999997 2024-02-18T00:41:37.713 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2070 17.15 e108/104 17:ae45de0e:test-rados-api-smithi059-82949-17::big2:head v 108'1, mlcod=0'0)", 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 513.56135702400002, 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070392237999999996, 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.714 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136823+0000", 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.136822+0000", 2024-02-18T00:41:37.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044622728 2024-02-18T00:41:37.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181444+0000", 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5222e-05 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181469+0000", 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0170000000000002e-06 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181478+0000", 2024-02-18T00:41:37.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.237e-05 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181551+0000", 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9718999999999999e-05 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.181571+0000", 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025600378999999999 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.207171+0000", 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3922999999999999e-05 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.207215+0000", 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.070392237999999996 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.25025.0:40 22.c e108/105 22:33eb54d8:::39.obj2:head v 108'5, mlcod=108'3)", 2024-02-18T00:41:37.719 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 513.42912455600003, 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032619941, 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269055+0000", 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269053+0000", 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7693000000000002e-05 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269071+0000", 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9028000000000001e-05 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269090+0000", 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5519999999999999e-06 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269097+0000", 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0609999999999998e-05 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269148+0000", 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1269000000000001e-05 2024-02-18T00:41:37.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.269189+0000", 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032451386999999998 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.301640+0000", 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4557000000000001e-05 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.301675+0000", 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.032619941 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(24.2 epoch 109/105 rep_tid 111 entries 109'2 (0'0) error 24:4061fbc0:test-rados-api-smithi059-82933-5::foo:head by client.24806.0:10 0.000000 -17 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 0'0)", 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 513.13029350700003, 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051717428000000003, 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:41:37.724 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567886+0000", 2024-02-18T00:41:37.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0070000000000003e-06 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567895+0000", 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1107999999999999e-05 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567906+0000", 2024-02-18T00:41:37.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0720000000000006e-06 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.567915+0000", 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023667508 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.591582+0000", 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028021134999999999 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.619604+0000", 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051717428000000003 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.26167.0:11 66.17 66:e89f79a2:test-rados-api-smithi059-82981-9::foo:head [write 0~128] snapc 6=[6] ondisk+write+known_if_redirected e131)", 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 488.39199226400001, 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062117686999999998, 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.26167", 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 11 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306187+0000", 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306186+0000", 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6010000000000001e-06 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306195+0000", 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2559e-05 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306208+0000", 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6709999999999999e-06 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306215+0000", 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00019698599999999999 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306412+0000", 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8329999999999996e-06 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.306419+0000", 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012823485000000001 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.319243+0000", 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012152899999999999 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.319364+0000", 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010919998 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.330284+0000", 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011843823 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.342128+0000", 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2046000000000003e-05 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.733 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.342180+0000", 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.026025063000000001 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368205+0000", 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5210000000000002e-05 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368260+0000", 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4642999999999999e-05 2024-02-18T00:41:37.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.368305+0000", 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062117686999999998 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.29425.0:2 73.6 73:602f83fe:::foo:head [append 0~128] snapc 0=[] ondisk+write+known_if_redirected e131)", 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:41:37.735 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 488.39056659099998, 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.075318273000000005, 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.29425", 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/4028720008", 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.736 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307613+0000", 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307611+0000", 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0359999999999995e-06 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307620+0000", 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1882000000000001e-05 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307632+0000", 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7689999999999996e-06 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.307640+0000", 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.030238405999999999 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.337878+0000", 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6550000000000004e-06 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.337886+0000", 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028640344000000002 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366526+0000", 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000118879 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366645+0000", 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014485400000000001 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,6", 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.366790+0000", 2024-02-18T00:41:37.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003334025 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.370124+0000", 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6338999999999997e-05 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.370181+0000", 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012666067 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382847+0000", 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6785e-05 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382893+0000", 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7939999999999999e-05 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:29.382931+0000", 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.075318273000000005 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.742 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.26167.0:16 66.17 66:e89f79a2:test-rados-api-smithi059-82981-9::foo:head [write 128~128] snapc 7=[7,6] ondisk+write+known_if_redirected e132)", 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 487.37850917700001, 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059589835000000001, 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.26167", 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 16 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.743 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319670+0000", 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.744 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319668+0000", 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1216000000000001e-05 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.319679+0000", 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0010448810000000001 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320724+0000", 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0662999999999999e-05 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320735+0000", 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4954999999999998e-05 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320790+0000", 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8499999999999994e-06 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.320798+0000", 2024-02-18T00:41:37.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021397759999999998 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342195+0000", 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4319000000000003e-05 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342290+0000", 2024-02-18T00:41:37.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000201516 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.342491+0000", 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0097928909999999997 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.352284+0000", 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013119486 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.365404+0000", 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013735841 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379139+0000", 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3324999999999993e-05 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379213+0000", 2024-02-18T00:41:37.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7633999999999998e-05 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:30.379260+0000", 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.059589835000000001 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.7.0:212 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:41:37.750 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 483.174828977, 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083227508000000006, 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523351+0000", 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.523349+0000", 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064971570000000006 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588321+0000", 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2592999999999999e-05 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588343+0000", 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4349999999999997e-06 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.588352+0000", 2024-02-18T00:41:37.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000863093 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.589215+0000", 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6400000000000001e-05 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.589241+0000", 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017286283 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.606528+0000", 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0519000000000002e-05 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.606578+0000", 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083227508000000006 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.7.0:214 17.c e136/104 -19:36e2cd47:::temp_17.c_0_24310_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:41:37.755 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 483.09792066400001, 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047186580999999998, 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600259+0000", 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.600257+0000", 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.024532449000000001 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624790+0000", 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9515000000000001e-05 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624819+0000", 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0849000000000001e-05 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.624830+0000", 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00025507999999999998 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.625085+0000", 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4088e-05 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.625109+0000", 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022278747000000002 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.647388+0000", 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7408999999999999e-05 2024-02-18T00:41:37.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.647446+0000", 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047186580999999998 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42808.0:2 99.6 99:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e156)", 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 463.11680525000003, 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041101081999999997, 2024-02-18T00:41:37.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42808", 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3195038165", 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581374+0000", 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581373+0000", 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4339999999999992e-06 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581381+0000", 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8351000000000001e-05 2024-02-18T00:41:37.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581399+0000", 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3760000000000008e-06 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581408+0000", 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2623e-05 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581470+0000", 2024-02-18T00:41:37.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.43e-06 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.581478+0000", 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013746548000000001 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.595224+0000", 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2150000000000004e-06 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.595232+0000", 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021220474 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616452+0000", 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000106791 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616559+0000", 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000140035 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,4", 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.616699+0000", 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001518248 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.618217+0000", 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0015478130000000001 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.619765+0000", 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0026234829999999998 2024-02-18T00:41:37.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622388+0000", 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9582000000000001e-05 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622438+0000", 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7565999999999999e-05 2024-02-18T00:41:37.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:54.622475+0000", 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.041101081999999997 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42916.0:1 114.5s0 114:a1464ac0:test-rados-api-smithi059-82981-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e176)", 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:41:37.768 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 442.30598182099999, 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996, 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42916", 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2278224535", 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392195+0000", 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1378e-05 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392206+0000", 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3589e-05 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392220+0000", 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1283e-05 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392231+0000", 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0343999999999997e-05 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392292+0000", 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3749999999999999e-06 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392295+0000", 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.58736239099999998 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979657+0000", 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103571 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979761+0000", 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000375254 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.980136+0000", 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061387799999999999 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.986275+0000", 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032303779999999999 2024-02-18T00:41:37.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989505+0000", 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7123000000000003e-05 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989552+0000", 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42910.0:2 115.13 115:ce638d0e:test-rados-api-smithi059-82949-26::foo:head [writefull 0~8388608] snapc 0=[] ondisk+write+known_if_redirected e178)", 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 440.71822974899999, 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003, 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42910", 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/955395839", 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979948+0000", 2024-02-18T00:41:37.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0051422639999999997 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985090+0000", 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1109e-05 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985101+0000", 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2439999999999993e-06 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985110+0000", 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032139199999999999 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985431+0000", 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115649 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985547+0000", 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014544800000000001 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,3", 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985692+0000", 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016096787000000001 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.001789+0000", 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016631943 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018421+0000", 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016582200000000001 2024-02-18T00:41:37.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018587+0000", 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4758999999999994e-05 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018651+0000", 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8247000000000002e-05 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018690+0000", 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:588 62.9 e205/123 62:9441d9b2:test-rados-api-smithi059-82957-14::foo82:4 v 205'35, mlcod=193'31)", 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 414.02721208399998, 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999, 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.781 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670966+0000", 2024-02-18T00:41:37.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2614e-05 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670979+0000", 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1616000000000001e-05 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671000+0000", 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9030000000000004e-06 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671008+0000", 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3940000000000001e-05 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671072+0000", 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0826999999999999e-05 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671093+0000", 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034117472000000003 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705210+0000", 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5481000000000005e-05 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705276+0000", 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.785 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:589 62.9 e205/123 62:93987215:test-rados-api-smithi059-82957-14::foo76:4 v 205'37, mlcod=193'31)", 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 414.026726285, 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001, 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.109e-06 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671460+0000", 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.027e-05 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671470+0000", 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.993e-06 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671476+0000", 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2646999999999999e-05 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671529+0000", 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0522000000000001e-05 2024-02-18T00:41:37.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671549+0000", 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036671306000000001 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708221+0000", 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8853000000000002e-05 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708259+0000", 2024-02-18T00:41:37.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30969.0:1 179.5s0 179:b3610dda:::oid_RoundTripPP2_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 334.06053030700002, 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001, 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.790 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30969", 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3697991996", 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:41:37.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637648+0000", 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2819999999999999e-06 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637655+0000", 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9309999999999997e-06 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637664+0000", 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.153e-06 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637671+0000", 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2398000000000003e-05 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637713+0000", 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2790000000000001e-06 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637716+0000", 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0156700160000001 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653386+0000", 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131438 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653517+0000", 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043157100000000001 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653949+0000", 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031255889999999998 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657074+0000", 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2801999999999999e-05 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657147+0000", 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000247073 2024-02-18T00:41:37.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657394+0000", 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30966.0:1 178.6s0 178:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 334.059811878, 2024-02-18T00:41:37.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001, 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30966", 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3784803143", 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638367+0000", 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.622e-06 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638372+0000", 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7880000000000001e-06 2024-02-18T00:41:37.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638381+0000", 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7909999999999999e-06 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638388+0000", 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.171e-05 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638420+0000", 2024-02-18T00:41:37.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6700000000000001e-06 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638422+0000", 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0205109640000001 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.658932+0000", 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014527400000000001 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659078+0000", 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043918800000000002 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659517+0000", 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00089794300000000001 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660415+0000", 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9174e-05 2024-02-18T00:41:37.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660474+0000", 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024629900000000001 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660720+0000", 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.40973.0:1 199.2s0 199:482bd08b:::oid_SimpleStatPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e354)", 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 264.923011207, 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0539473699999999, 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.40973", 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1800225797", 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775166+0000", 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0146000000000001e-05 2024-02-18T00:41:37.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775176+0000", 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2602000000000001e-05 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775189+0000", 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4409999999999999e-06 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775198+0000", 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9956e-05 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775258+0000", 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7239999999999998e-06 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775262+0000", 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.050233969 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.825496+0000", 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000114114 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.825610+0000", 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045617599999999998 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.826066+0000", 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049755600000000002 2024-02-18T00:41:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.826564+0000", 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024990279999999999 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.829063+0000", 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3276000000000003e-05 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.829116+0000", 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0539473699999999 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.41009.0:1 203.4s0 203:311cbe94:::oid_ExecuteClassPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 244.02692810600001, 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.014258211, 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:41:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.41009", 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1940889306", 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671249+0000", 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2910000000000007e-06 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671257+0000", 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0814e-05 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671268+0000", 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5390000000000002e-06 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671276+0000", 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9815999999999998e-05 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671336+0000", 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-02-18T00:41:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671339+0000", 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.011917613 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683257+0000", 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016619700000000001 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683423+0000", 2024-02-18T00:41:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00048756299999999999 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683910+0000", 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00099101599999999995 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.684901+0000", 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00052922899999999998 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:41:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.685431+0000", 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9234999999999996e-05 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.685510+0000", 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.014258211 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25220.0:5782 209.10 209:08000000:%01:::head [pgnls start_epoch 404] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e404)", 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 211.963181134, 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.965927443, 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25220", 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 5782 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:41:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:41:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734998+0000", 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7509999999999997e-06 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735003+0000", 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0363000000000001e-05 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735014+0000", 2024-02-18T00:41:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8989999999999998e-06 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735020+0000", 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2460000000000003e-05 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735072+0000", 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8979999999999996e-06 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:41:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735080+0000", 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96567722099999997 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700757+0000", 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4641000000000002e-05 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700802+0000", 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012411200000000001 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:41:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700926+0000", 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.965927443 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:41:37.821 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:41:37.822 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:37.822 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:37.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 -- 172.21.15.59:0/1151046383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b80f6b20 msgr2=0x7f33b80f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 --2- 172.21.15.59:0/1151046383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b80f6b20 0x7f33b80f6f00 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7f33a8004870 tx=0x7f33a8009cb0).stop 2024-02-18T00:41:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 -- 172.21.15.59:0/1151046383 shutdown_connections 2024-02-18T00:41:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 --2- 172.21.15.59:0/1151046383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b8104680 0x7f33b8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 --2- 172.21.15.59:0/1151046383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b80f6b20 0x7f33b80f6f00 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 --2- 172.21.15.59:0/1151046383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 0x7f33b8104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 -- 172.21.15.59:0/1151046383 >> 172.21.15.59:0/1151046383 conn(0x7f33b80f2820 msgr2=0x7f33b80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 -- 172.21.15.59:0/1151046383 shutdown_connections 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.819+0000 7f33be473700 1 -- 172.21.15.59:0/1151046383 wait complete. 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 librados: starting msgr at 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 librados: starting objecter 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 Processor -- start 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 -- start start 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 librados: setting wanted keys 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 librados: calling monclient init 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b80f6b20 0x7f33b8110410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 0x7f33b8110900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b8104680 0x7f33b810a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b810aaa0 con 0x7f33b8104680 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b810abe0 con 0x7f33b80f6b20 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33be473700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b810ada0 con 0x7f33b80f73f0 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33b77fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 0x7f33b8110900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33bca10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b8104680 0x7f33b810a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33b77fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 0x7f33b8110900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38748/0 (socket says 172.21.15.59:38748) 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33bca10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b8104680 0x7f33b810a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43258/0 (socket says 172.21.15.59:43258) 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.820+0000 7f33b77fe700 1 -- 172.21.15.59:0/3386435631 learned_addr learned my addr 172.21.15.59:0/3386435631 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b7fff700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b80f6b20 0x7f33b8110410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b77fe700 1 -- 172.21.15.59:0/3386435631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b80f6b20 msgr2=0x7f33b8110410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b77fe700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b80f6b20 0x7f33b8110410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b77fe700 1 -- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b8104680 msgr2=0x7f33b810a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b77fe700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b8104680 0x7f33b810a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b77fe700 1 -- 172.21.15.59:0/3386435631 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac005040 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33bca10700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b8104680 0x7f33b810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b77fe700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 0x7f33b8110900 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 rx=0x7f33ac0048f0 tx=0x7f33ac009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b57fa700 10 client.?.objecter ms_handle_connect 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b57fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b57fa700 1 -- 172.21.15.59:0/3386435631 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac007b00 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a8005040 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b57fa700 1 -- 172.21.15.59:0/3386435631 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33ac00f040 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33b57fa700 1 -- 172.21.15.59:0/3386435631 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac014990 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33be473700 10 client.41426.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b810b3e0 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.821+0000 7f33be473700 1 librados: init done 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.822+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b805fa60 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.822+0000 7f33b57fa700 1 -- 172.21.15.59:0/3386435631 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f33ac0133c0 con 0x7f33b80f73f0 2024-02-18T00:41:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b57fa700 1 --2- 172.21.15.59:0/3386435631 >> v2:172.21.15.59:6800/3721891797 conn(0x7f33a0062c90 0x7f33a0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b57fa700 1 -- 172.21.15.59:0/3386435631 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(563..563 src has 418..563) v4 ==== 8218+0+0 (secure 0 0 0) 0x7f33ac07ce00 con 0x7f33b80f73f0 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b57fa700 10 client.41426.objecter ms_dispatch 0x7f33b8001550 osd_map(563..563 src has 418..563) v4 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b57fa700 3 client.41426.objecter handle_osd_map got epochs [563,563] > 0 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b57fa700 3 client.41426.objecter handle_osd_map decoding full epoch 563 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b7fff700 1 --2- 172.21.15.59:0/3386435631 >> v2:172.21.15.59:6800/3721891797 conn(0x7f33a0062c90 0x7f33a0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b57fa700 20 client.41426.objecter dump_active .. 0 homeless 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b57fa700 10 client.41426.objecter ms_handle_connect 0x7f33a0062c90 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.823+0000 7f33b7fff700 1 --2- 172.21.15.59:0/3386435631 >> v2:172.21.15.59:6800/3721891797 conn(0x7f33a0062c90 0x7f33a0065150 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 rx=0x7f33a80043d0 tx=0x7f33a800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:36.828+0000 7f33b57fa700 1 -- 172.21.15.59:0/3386435631 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f33ac0501d0 con 0x7f33b80f73f0 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.023+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f33b81093f0 con 0x7f33b80f73f0 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.792+0000 7f33b57fa700 1 -- 172.21.15.59:0/3386435631 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v564) v1 ==== 87+0+0 (secure 0 0 0) 0x7f33ac04c070 con 0x7f33b80f73f0 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.795+0000 7f33be473700 10 librados: watch_flush enter 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 10 librados: watch_flush exit 2024-02-18T00:41:37.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 20 client.41426.objecter shutdown clearing up homeless session... 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 10 client.41426.objecter successfully canceled tick 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 >> v2:172.21.15.59:6800/3721891797 conn(0x7f33a0062c90 msgr2=0x7f33a0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 --2- 172.21.15.59:0/3386435631 >> v2:172.21.15.59:6800/3721891797 conn(0x7f33a0062c90 0x7f33a0065150 secure :-1 s=READY pgs=3158 cs=0 l=1 rev1=1 rx=0x7f33a80043d0 tx=0x7f33a800b040).stop 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 msgr2=0x7f33b8110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 0x7f33b8110900 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 rx=0x7f33ac0048f0 tx=0x7f33ac009cb0).stop 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 shutdown_connections 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 --2- 172.21.15.59:0/3386435631 >> v2:172.21.15.59:6800/3721891797 conn(0x7f33a0062c90 0x7f33a0065150 unknown :-1 s=CLOSED pgs=3158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b80f6b20 0x7f33b8110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33b8104680 0x7f33b810a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 --2- 172.21.15.59:0/3386435631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f33b80f73f0 0x7f33b8110900 secure :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 rx=0x7f33ac0048f0 tx=0x7f33ac009cb0).stop 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.796+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 >> 172.21.15.59:0/3386435631 conn(0x7f33b80f2820 msgr2=0x7f33b80f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.797+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 shutdown_connections 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.797+0000 7f33be473700 1 -- 172.21.15.59:0/3386435631 wait complete. 2024-02-18T00:41:37.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:37.797+0000 7f33be473700 1 librados: shutdown 2024-02-18T00:41:37.864 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:41:37.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.067515+0000 osd.2 (osd.2) 231 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.069247+0000 osd.2 (osd.2) 232 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.775439+0000 osd.6 (osd.6) 269 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.776423+0000 osd.6 (osd.6) 270 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:36.789217+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.789286+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:36.793569+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:36.794009+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.840311+0000 osd.3 (osd.3) 305 : cluster [DBG] 25.17 deep-scrub starts 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:36.841303+0000 osd.3 (osd.3) 306 : cluster [DBG] 25.17 deep-scrub ok 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.024599+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.59:0/3386435631' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.024952+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:37.064231+0000 osd.4 (osd.4) 331 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:37.067685+0000 osd.4 (osd.4) 332 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.067515+0000 osd.2 (osd.2) 231 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.069247+0000 osd.2 (osd.2) 232 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.775439+0000 osd.6 (osd.6) 269 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.776423+0000 osd.6 (osd.6) 270 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:36.789217+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:41:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.789286+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:36.793569+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:36.794009+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.840311+0000 osd.3 (osd.3) 305 : cluster [DBG] 25.17 deep-scrub starts 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:36.841303+0000 osd.3 (osd.3) 306 : cluster [DBG] 25.17 deep-scrub ok 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.024599+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.59:0/3386435631' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.024952+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:37.064231+0000 osd.4 (osd.4) 331 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:41:38.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:37.067685+0000 osd.4 (osd.4) 332 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.067515+0000 osd.2 (osd.2) 231 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.069247+0000 osd.2 (osd.2) 232 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.775439+0000 osd.6 (osd.6) 269 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.776423+0000 osd.6 (osd.6) 270 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:36.789217+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.789286+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:36.793569+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:36.794009+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.840311+0000 osd.3 (osd.3) 305 : cluster [DBG] 25.17 deep-scrub starts 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:36.841303+0000 osd.3 (osd.3) 306 : cluster [DBG] 25.17 deep-scrub ok 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.024599+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.59:0/3386435631' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:41:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.024952+0000 mon.a (mon.0) 2948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:41:38.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:37.064231+0000 osd.4 (osd.4) 331 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:41:38.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:37.067685+0000 osd.4 (osd.4) 332 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:41:38.375 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:38.402 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:38.405 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:38.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:38.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:38.534 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:37.672561+0000 osd.5 (osd.5) 247 : cluster [DBG] 5.1b deep-scrub starts 2024-02-18T00:41:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:37.673669+0000 osd.5 (osd.5) 248 : cluster [DBG] 5.1b deep-scrub ok 2024-02-18T00:41:39.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:37.785017+0000 mon.a (mon.0) 2949 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.791860+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.791938+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:37.791973+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.821598+0000 mon.c (mon.2) 534 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.821987+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.822441+0000 mon.c (mon.2) 535 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-92"}]: dispatch 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.822649+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-92"}]: dispatch 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:37.957225+0000 mgr.y (mgr.24454) 518 : cluster [DBG] pgmap v861: 234 pgs: 1 clean+premerge+peered, 233 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:37.958091+0000 mon.a (mon.0) 2955 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:38.738843+0000 mon.a (mon.0) 2956 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-18T00:41:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:38.738916+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-18T00:41:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:37.672561+0000 osd.5 (osd.5) 247 : cluster [DBG] 5.1b deep-scrub starts 2024-02-18T00:41:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:37.673669+0000 osd.5 (osd.5) 248 : cluster [DBG] 5.1b deep-scrub ok 2024-02-18T00:41:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:37.785017+0000 mon.a (mon.0) 2949 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.791860+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:41:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.791938+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:41:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:37.791973+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-18T00:41:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.821598+0000 mon.c (mon.2) 534 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.821987+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.822441+0000 mon.c (mon.2) 535 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-92"}]: dispatch 2024-02-18T00:41:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.822649+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-92"}]: dispatch 2024-02-18T00:41:39.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:37.957225+0000 mgr.y (mgr.24454) 518 : cluster [DBG] pgmap v861: 234 pgs: 1 clean+premerge+peered, 233 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:39.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:37.958091+0000 mon.a (mon.0) 2955 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-18T00:41:39.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:38.738843+0000 mon.a (mon.0) 2956 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-18T00:41:39.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:38.738916+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-18T00:41:39.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:37.672561+0000 osd.5 (osd.5) 247 : cluster [DBG] 5.1b deep-scrub starts 2024-02-18T00:41:39.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:37.673669+0000 osd.5 (osd.5) 248 : cluster [DBG] 5.1b deep-scrub ok 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:37.785017+0000 mon.a (mon.0) 2949 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.791860+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.791938+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:37.791973+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.821598+0000 mon.c (mon.2) 534 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.821987+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-6"}]: dispatch 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.822441+0000 mon.c (mon.2) 535 : audit [INF] from='client.? 172.21.15.59:0/699604112' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-92"}]: dispatch 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.822649+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-6", "tierpool": "test-rados-api-smithi059-82957-92"}]: dispatch 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:37.957225+0000 mgr.y (mgr.24454) 518 : cluster [DBG] pgmap v861: 234 pgs: 1 clean+premerge+peered, 233 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:37.958091+0000 mon.a (mon.0) 2955 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]: dispatch 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:38.738843+0000 mon.a (mon.0) 2956 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "27"}]': finished 2024-02-18T00:41:39.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:38.738916+0000 mon.a (mon.0) 2957 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-18T00:41:39.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:41:39.687 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:39.690 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:39.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:39.792 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:39.823 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:39.741111+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-18T00:41:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:39.766037+0000 mon.c (mon.2) 536 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:39.766375+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:39.767260+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:39.767467+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:39.767925+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:41.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:39.768133+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:41.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:39.957570+0000 mgr.y (mgr.24454) 519 : cluster [DBG] pgmap v864: 170 pgs: 1 clean+premerge+peered, 169 active+clean; 5.3 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:41.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:39.741111+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-18T00:41:41.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:39.766037+0000 mon.c (mon.2) 536 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:39.766375+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:39.767260+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:39.767467+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:39.767925+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:41.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:39.768133+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:41.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:39.957570+0000 mgr.y (mgr.24454) 519 : cluster [DBG] pgmap v864: 170 pgs: 1 clean+premerge+peered, 169 active+clean; 5.3 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:39.741111+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-18T00:41:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:39.766037+0000 mon.c (mon.2) 536 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:39.766375+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:39.767260+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:39.767467+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:39.767925+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:41.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:39.768133+0000 mon.a (mon.0) 2961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:41.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:39.957570+0000 mgr.y (mgr.24454) 519 : cluster [DBG] pgmap v864: 170 pgs: 1 clean+premerge+peered, 169 active+clean; 5.3 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:41.373 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:41.375 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:41.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:41.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-18T00:41:41.405 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:41:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:41:41.476 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:41.510 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:41.750 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:41:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:41:41.763 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:41.770 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:41.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:41.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:41.871 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:41.906 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:41.928 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:41.931 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:41.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:42.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:40.747931+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:41:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:40.748026+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-18T00:41:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:40.750875+0000 mon.c (mon.2) 539 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:40.751506+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:40.747931+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:41:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:40.748026+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-18T00:41:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:40.750875+0000 mon.c (mon.2) 539 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:40.751506+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:42.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:40.747931+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:41:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:40.748026+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-18T00:41:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:40.750875+0000 mon.c (mon.2) 539 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:42.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:40.751506+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:41:42.032 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:42.063 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:42.086 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:42.086 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:42.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:42.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:42.217 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:42.237 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:42.239 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:42.239 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 -- 172.21.15.59:0/3740677160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0fd510 msgr2=0x7fedac0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/3740677160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0fd510 0x7fedac0f6d40 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 rx=0x7feda0004870 tx=0x7feda0009cb0).stop 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 -- 172.21.15.59:0/3740677160 shutdown_connections 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/3740677160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedac0fcbb0 0x7fedac0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/3740677160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedac0f7400 0x7fedac103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/3740677160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0fd510 0x7fedac0f6d40 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 -- 172.21.15.59:0/3740677160 >> 172.21.15.59:0/3740677160 conn(0x7fedac0f2820 msgr2=0x7fedac0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 -- 172.21.15.59:0/3740677160 shutdown_connections 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 -- 172.21.15.59:0/3740677160 wait complete. 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 librados: starting msgr at 2024-02-18T00:41:42.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 librados: starting objecter 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.303+0000 7fedb1b1c700 1 Processor -- start 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 -- start start 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 librados: setting wanted keys 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 librados: calling monclient init 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0f7400 0x7fedac109260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedac0fcbb0 0x7fedac109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedac0fd510 0x7fedac109d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac114440 con 0x7fedac0fd510 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1145b0 con 0x7fedac0fcbb0 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedb1b1c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac114770 con 0x7fedac0f7400 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0f7400 0x7fedac109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0f7400 0x7fedac109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:33152/0 (socket says 172.21.15.59:33152) 2024-02-18T00:41:42.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedabfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedac0fd510 0x7fedac109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 -- 172.21.15.59:0/2953481570 learned_addr learned my addr 172.21.15.59:0/2953481570 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedaaffd700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedac0fcbb0 0x7fedac109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 -- 172.21.15.59:0/2953481570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedac0fcbb0 msgr2=0x7fedac109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedac0fcbb0 0x7fedac109750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 -- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedac0fd510 msgr2=0x7fedac109d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedac0fd510 0x7fedac109d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 -- 172.21.15.59:0/2953481570 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40097c0 con 0x7fedac0f7400 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedabfff700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedac0fd510 0x7fedac109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7fedab7fe700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0f7400 0x7fedac109260 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 rx=0x7fed9c0041d0 tx=0x7fed9c009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7feda8ff9700 10 client.?.objecter ms_handle_connect 0x7fedac0f7400 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7feda8ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.304+0000 7feda8ff9700 1 -- 172.21.15.59:0/2953481570 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007b40 con 0x7fedac0f7400 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.305+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda0005040 con 0x7fedac0f7400 2024-02-18T00:41:42.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.305+0000 7feda8ff9700 1 -- 172.21.15.59:0/2953481570 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c00f040 con 0x7fedac0f7400 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.305+0000 7feda8ff9700 1 -- 172.21.15.59:0/2953481570 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c014730 con 0x7fedac0f7400 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.305+0000 7fedb1b1c700 10 client.41438.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.305+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac114e00 con 0x7fedac0f7400 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.305+0000 7fedb1b1c700 1 librados: init done 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac05fa60 con 0x7fedac0f7400 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7feda8ff9700 1 -- 172.21.15.59:0/2953481570 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fed9c009110 con 0x7fedac0f7400 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7feda8ff9700 1 --2- 172.21.15.59:0/2953481570 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed94062c90 0x7fed94065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7feda8ff9700 1 -- 172.21.15.59:0/2953481570 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(568..568 src has 418..568) v4 ==== 7772+0+0 (secure 0 0 0) 0x7fed9c07d0d0 con 0x7fedac0f7400 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7feda8ff9700 10 client.41438.objecter ms_dispatch 0x7fedac001550 osd_map(568..568 src has 418..568) v4 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7feda8ff9700 3 client.41438.objecter handle_osd_map got epochs [568,568] > 0 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7feda8ff9700 3 client.41438.objecter handle_osd_map decoding full epoch 568 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7fedaaffd700 1 --2- 172.21.15.59:0/2953481570 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed94062c90 0x7fed94065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.306+0000 7feda8ff9700 20 client.41438.objecter dump_active .. 0 homeless 2024-02-18T00:41:42.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.307+0000 7feda8ff9700 10 client.41438.objecter ms_handle_connect 0x7fed94062c90 2024-02-18T00:41:42.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.307+0000 7fedaaffd700 1 --2- 172.21.15.59:0/2953481570 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed94062c90 0x7fed94065150 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 rx=0x7feda0009a00 tx=0x7feda000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:42.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.311+0000 7feda8ff9700 1 -- 172.21.15.59:0/2953481570 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fed9c050700 con 0x7fedac0f7400 2024-02-18T00:41:42.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:42.357 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:42.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:41:42.372 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:42] "GET /metrics HTTP/1.1" 200 185756 "" "Prometheus/2.33.4" 2024-02-18T00:41:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:42.401 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:42.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:42.407 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.489+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedac0614b0 con 0x7fed94062c90 2024-02-18T00:41:42.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.502+0000 7feda8ff9700 1 -- 172.21.15.59:0/2953481570 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+443233 (secure 0 0 0) 0x7fedac0614b0 con 0x7fed94062c90 2024-02-18T00:41:42.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:42.511 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.506+0000 7fedb1b1c700 10 librados: watch_flush enter 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.506+0000 7fedb1b1c700 10 librados: watch_flush exit 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.506+0000 7fedb1b1c700 20 client.41438.objecter shutdown clearing up homeless session... 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.506+0000 7fedb1b1c700 10 client.41438.objecter successfully canceled tick 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed94062c90 msgr2=0x7fed94065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/2953481570 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed94062c90 0x7fed94065150 secure :-1 s=READY pgs=3160 cs=0 l=1 rev1=1 rx=0x7feda0009a00 tx=0x7feda000b040).stop 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0f7400 msgr2=0x7fedac109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0f7400 0x7fedac109260 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 rx=0x7fed9c0041d0 tx=0x7fed9c009f60).stop 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 shutdown_connections 2024-02-18T00:41:42.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedac0fd510 0x7fedac109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fedac0f7400 0x7fedac109260 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/2953481570 >> v2:172.21.15.59:6800/3721891797 conn(0x7fed94062c90 0x7fed94065150 unknown :-1 s=CLOSED pgs=3160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 --2- 172.21.15.59:0/2953481570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fedac0fcbb0 0x7fedac109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 >> 172.21.15.59:0/2953481570 conn(0x7fedac0f2820 msgr2=0x7fedac0fa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 shutdown_connections 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 -- 172.21.15.59:0/2953481570 wait complete. 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:42.507+0000 7fedb1b1c700 1 librados: shutdown 2024-02-18T00:41:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:41:42.580 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:41.404399+0000 mgr.y (mgr.24454) 520 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:41.957890+0000 mgr.y (mgr.24454) 521 : cluster [DBG] pgmap v867: 169 pgs: 1 peering, 168 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:42.193555+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:42.194668+0000 mon.a (mon.0) 2966 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:41.404399+0000 mgr.y (mgr.24454) 520 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:41.957890+0000 mgr.y (mgr.24454) 521 : cluster [DBG] pgmap v867: 169 pgs: 1 peering, 168 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:42.193555+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-18T00:41:43.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:42.194668+0000 mon.a (mon.0) 2966 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-18T00:41:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:41.404399+0000 mgr.y (mgr.24454) 520 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:41.957890+0000 mgr.y (mgr.24454) 521 : cluster [DBG] pgmap v867: 169 pgs: 1 peering, 168 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:42.193555+0000 mon.a (mon.0) 2965 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-18T00:41:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:42.194668+0000 mon.a (mon.0) 2966 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-02-18T00:41:43.955 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":867,"stamp":"2024-02-18T00:41:41.957708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2269,"num_read_kb":1989,"num_write":912,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1019,"ondisk_log_size":1019,"up":507,"acting":507,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":671,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2469656,"kb_used_data":22232,"kb_used_omap":0,"kb_used_meta":2447360,"kb_avail":747491560,"statfs":{"total":767960285184,"available":765431357440,"internally_reserved":0,"allocated":22765568,"data_stored":11128795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506096640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4020066,"num_objects":-192,"num_object_clones":-11,"num_object_copies":-576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-192,"num_whiteouts":0,"num_read":-207,"num_read_kb":-52,"num_write":-777,"num_write_kb":-4618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-358,"num_bytes_recovered":-4000574,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":-1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-21,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000720"},"pg_stats":[{"pgid":"2.29","version":"0'0","reported_seq":497,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142613+0000","last_change":"2024-02-18T00:41:36.776506+0000","last_active":"2024-02-18T00:41:39.142613+0000","last_peered":"2024-02-18T00:41:39.142613+0000","last_clean":"2024-02-18T00:41:39.142613+0000","last_became_active":"2024-02-18T00:39:36.751760+0000","last_became_peered":"2024-02-18T00:39:36.751760+0000","last_unstale":"2024-02-18T00:41:39.142613+0000","last_undegraded":"2024-02-18T00:41:39.142613+0000","last_fullsized":"2024-02-18T00:41:39.142613+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:36.776429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.776429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":490,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.751714+0000","last_change":"2024-02-18T00:40:41.127499+0000","last_active":"2024-02-18T00:41:39.751714+0000","last_peered":"2024-02-18T00:41:39.751714+0000","last_clean":"2024-02-18T00:41:39.751714+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:41:39.751714+0000","last_undegraded":"2024-02-18T00:41:39.751714+0000","last_fullsized":"2024-02-18T00:41:39.751714+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:41.127432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:41.127432+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744558+0000","last_change":"2024-02-18T00:40:52.593461+0000","last_active":"2024-02-18T00:41:39.744558+0000","last_peered":"2024-02-18T00:41:39.744558+0000","last_clean":"2024-02-18T00:41:39.744558+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:41:39.744558+0000","last_undegraded":"2024-02-18T00:41:39.744558+0000","last_fullsized":"2024-02-18T00:41:39.744558+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.593401+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743741+0000","last_change":"2024-02-18T00:41:27.570515+0000","last_active":"2024-02-18T00:41:39.743741+0000","last_peered":"2024-02-18T00:41:39.743741+0000","last_clean":"2024-02-18T00:41:39.743741+0000","last_became_active":"2024-02-18T00:39:36.760004+0000","last_became_peered":"2024-02-18T00:39:36.760004+0000","last_unstale":"2024-02-18T00:41:39.743741+0000","last_undegraded":"2024-02-18T00:41:39.743741+0000","last_fullsized":"2024-02-18T00:41:39.743741+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:27.570440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.570440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":498,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.824536+0000","last_change":"2024-02-18T00:41:37.067790+0000","last_active":"2024-02-18T00:41:39.824536+0000","last_peered":"2024-02-18T00:41:39.824536+0000","last_clean":"2024-02-18T00:41:39.824536+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:41:39.824536+0000","last_undegraded":"2024-02-18T00:41:39.824536+0000","last_fullsized":"2024-02-18T00:41:39.824536+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:41:37.067689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:37.067689+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":497,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145061+0000","last_change":"2024-02-18T00:41:34.741697+0000","last_active":"2024-02-18T00:41:39.145061+0000","last_peered":"2024-02-18T00:41:39.145061+0000","last_clean":"2024-02-18T00:41:39.145061+0000","last_became_active":"2024-02-18T00:39:36.751486+0000","last_became_peered":"2024-02-18T00:39:36.751486+0000","last_unstale":"2024-02-18T00:41:39.145061+0000","last_undegraded":"2024-02-18T00:41:39.145061+0000","last_fullsized":"2024-02-18T00:41:39.145061+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.741618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.741618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":472,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.746498+0000","last_change":"2024-02-18T00:39:20.629493+0000","last_active":"2024-02-18T00:41:39.746498+0000","last_peered":"2024-02-18T00:41:39.746498+0000","last_clean":"2024-02-18T00:41:39.746498+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:41:39.746498+0000","last_undegraded":"2024-02-18T00:41:39.746498+0000","last_fullsized":"2024-02-18T00:41:39.746498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:20.629419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:20.629419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":481,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.143825+0000","last_change":"2024-02-18T00:40:51.610695+0000","last_active":"2024-02-18T00:41:39.143825+0000","last_peered":"2024-02-18T00:41:39.143825+0000","last_clean":"2024-02-18T00:41:39.143825+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:41:39.143825+0000","last_undegraded":"2024-02-18T00:41:39.143825+0000","last_fullsized":"2024-02-18T00:41:39.143825+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.610630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.610630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":569,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742548+0000","last_change":"2024-02-18T00:39:36.763536+0000","last_active":"2024-02-18T00:41:39.742548+0000","last_peered":"2024-02-18T00:41:39.742548+0000","last_clean":"2024-02-18T00:41:39.742548+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:41:39.742548+0000","last_undegraded":"2024-02-18T00:41:39.742548+0000","last_fullsized":"2024-02-18T00:41:39.742548+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:25.388161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:25.388161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741323+0000","last_change":"2024-02-18T00:40:51.582600+0000","last_active":"2024-02-18T00:41:39.741323+0000","last_peered":"2024-02-18T00:41:39.741323+0000","last_clean":"2024-02-18T00:41:39.741323+0000","last_became_active":"2024-02-18T00:39:36.769282+0000","last_became_peered":"2024-02-18T00:39:36.769282+0000","last_unstale":"2024-02-18T00:41:39.741323+0000","last_undegraded":"2024-02-18T00:41:39.741323+0000","last_fullsized":"2024-02-18T00:41:39.741323+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.582542+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":497,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.740858+0000","last_change":"2024-02-18T00:41:23.006759+0000","last_active":"2024-02-18T00:41:39.740858+0000","last_peered":"2024-02-18T00:41:39.740858+0000","last_clean":"2024-02-18T00:41:39.740858+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:41:39.740858+0000","last_undegraded":"2024-02-18T00:41:39.740858+0000","last_fullsized":"2024-02-18T00:41:39.740858+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.006689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.006689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"466'10","reported_seq":141,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.740788+0000","last_change":"2024-02-18T00:40:02.730763+0000","last_active":"2024-02-18T00:41:39.740788+0000","last_peered":"2024-02-18T00:41:39.740788+0000","last_clean":"2024-02-18T00:41:39.740788+0000","last_became_active":"2024-02-18T00:39:36.757493+0000","last_became_peered":"2024-02-18T00:39:36.757493+0000","last_unstale":"2024-02-18T00:41:39.740788+0000","last_undegraded":"2024-02-18T00:41:39.740788+0000","last_fullsized":"2024-02-18T00:41:39.740788+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:40:02.730686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:02.730686+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":665,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.141780+0000","last_change":"2024-02-18T00:40:40.557614+0000","last_active":"2024-02-18T00:41:39.141780+0000","last_peered":"2024-02-18T00:41:39.141780+0000","last_clean":"2024-02-18T00:41:39.141780+0000","last_became_active":"2024-02-18T00:39:36.755699+0000","last_became_peered":"2024-02-18T00:39:36.755699+0000","last_unstale":"2024-02-18T00:41:39.141780+0000","last_undegraded":"2024-02-18T00:41:39.141780+0000","last_fullsized":"2024-02-18T00:41:39.141780+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:40:40.557544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.557544+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":562,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742987+0000","last_change":"2024-02-18T00:40:48.410752+0000","last_active":"2024-02-18T00:41:39.742987+0000","last_peered":"2024-02-18T00:41:39.742987+0000","last_clean":"2024-02-18T00:41:39.742987+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:41:39.742987+0000","last_undegraded":"2024-02-18T00:41:39.742987+0000","last_fullsized":"2024-02-18T00:41:39.742987+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:48.410676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:48.410676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":530,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.143071+0000","last_change":"2024-02-18T00:40:10.634698+0000","last_active":"2024-02-18T00:41:39.143071+0000","last_peered":"2024-02-18T00:41:39.143071+0000","last_clean":"2024-02-18T00:41:39.143071+0000","last_became_active":"2024-02-18T00:29:21.242439+0000","last_became_peered":"2024-02-18T00:29:21.242439+0000","last_unstale":"2024-02-18T00:41:39.143071+0000","last_undegraded":"2024-02-18T00:41:39.143071+0000","last_fullsized":"2024-02-18T00:41:39.143071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:10.634595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:10.634595+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":488,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819568+0000","last_change":"2024-02-18T00:40:50.652492+0000","last_active":"2024-02-18T00:41:39.819568+0000","last_peered":"2024-02-18T00:41:39.819568+0000","last_clean":"2024-02-18T00:41:39.819568+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:41:39.819568+0000","last_undegraded":"2024-02-18T00:41:39.819568+0000","last_fullsized":"2024-02-18T00:41:39.819568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.652414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.652414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":597,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.745388+0000","last_change":"2024-02-18T00:41:19.076836+0000","last_active":"2024-02-18T00:41:39.745388+0000","last_peered":"2024-02-18T00:41:39.745388+0000","last_clean":"2024-02-18T00:41:39.745388+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:41:39.745388+0000","last_undegraded":"2024-02-18T00:41:39.745388+0000","last_fullsized":"2024-02-18T00:41:39.745388+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:41:19.076766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:19.076766+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":551,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819316+0000","last_change":"2024-02-18T00:41:23.794790+0000","last_active":"2024-02-18T00:41:39.819316+0000","last_peered":"2024-02-18T00:41:39.819316+0000","last_clean":"2024-02-18T00:41:39.819316+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:41:39.819316+0000","last_undegraded":"2024-02-18T00:41:39.819316+0000","last_fullsized":"2024-02-18T00:41:39.819316+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.794682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.794682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741391+0000","last_change":"2024-02-18T00:40:46.382314+0000","last_active":"2024-02-18T00:41:39.741391+0000","last_peered":"2024-02-18T00:41:39.741391+0000","last_clean":"2024-02-18T00:41:39.741391+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:41:39.741391+0000","last_undegraded":"2024-02-18T00:41:39.741391+0000","last_fullsized":"2024-02-18T00:41:39.741391+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:46.382246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.382246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":559,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742845+0000","last_change":"2024-02-18T00:41:16.155260+0000","last_active":"2024-02-18T00:41:39.742845+0000","last_peered":"2024-02-18T00:41:39.742845+0000","last_clean":"2024-02-18T00:41:39.742845+0000","last_became_active":"2024-02-18T00:39:36.750333+0000","last_became_peered":"2024-02-18T00:39:36.750333+0000","last_unstale":"2024-02-18T00:41:39.742845+0000","last_undegraded":"2024-02-18T00:41:39.742845+0000","last_fullsized":"2024-02-18T00:41:39.742845+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:16.155189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:16.155189+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":492,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.750029+0000","last_change":"2024-02-18T00:41:36.841374+0000","last_active":"2024-02-18T00:41:39.750029+0000","last_peered":"2024-02-18T00:41:39.750029+0000","last_clean":"2024-02-18T00:41:39.750029+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:41:39.750029+0000","last_undegraded":"2024-02-18T00:41:39.750029+0000","last_fullsized":"2024-02-18T00:41:39.750029+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.841306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"457'42","reported_seq":672,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145362+0000","last_change":"2024-02-18T00:40:42.632457+0000","last_active":"2024-02-18T00:41:39.145362+0000","last_peered":"2024-02-18T00:41:39.145362+0000","last_clean":"2024-02-18T00:41:39.145362+0000","last_became_active":"2024-02-18T00:39:36.751206+0000","last_became_peered":"2024-02-18T00:39:36.751206+0000","last_unstale":"2024-02-18T00:41:39.145362+0000","last_undegraded":"2024-02-18T00:41:39.145362+0000","last_fullsized":"2024-02-18T00:41:39.145362+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-18T00:40:42.632357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.632357+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744524+0000","last_change":"2024-02-18T00:39:50.125802+0000","last_active":"2024-02-18T00:41:39.744524+0000","last_peered":"2024-02-18T00:41:39.744524+0000","last_clean":"2024-02-18T00:41:39.744524+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:41:39.744524+0000","last_undegraded":"2024-02-18T00:41:39.744524+0000","last_fullsized":"2024-02-18T00:41:39.744524+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:50.125735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:39:50.125735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743343+0000","last_change":"2024-02-18T00:40:51.349216+0000","last_active":"2024-02-18T00:41:39.743343+0000","last_peered":"2024-02-18T00:41:39.743343+0000","last_clean":"2024-02-18T00:41:39.743343+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:41:39.743343+0000","last_undegraded":"2024-02-18T00:41:39.743343+0000","last_fullsized":"2024-02-18T00:41:39.743343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.349134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.349134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741458+0000","last_change":"2024-02-18T00:40:08.782777+0000","last_active":"2024-02-18T00:41:39.741458+0000","last_peered":"2024-02-18T00:41:39.741458+0000","last_clean":"2024-02-18T00:41:39.741458+0000","last_became_active":"2024-02-18T00:39:36.768371+0000","last_became_peered":"2024-02-18T00:39:36.768371+0000","last_unstale":"2024-02-18T00:41:39.741458+0000","last_undegraded":"2024-02-18T00:41:39.741458+0000","last_fullsized":"2024-02-18T00:41:39.741458+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.782667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.782667+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":476,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142101+0000","last_change":"2024-02-18T00:41:23.575091+0000","last_active":"2024-02-18T00:41:39.142101+0000","last_peered":"2024-02-18T00:41:39.142101+0000","last_clean":"2024-02-18T00:41:39.142101+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:41:39.142101+0000","last_undegraded":"2024-02-18T00:41:39.142101+0000","last_fullsized":"2024-02-18T00:41:39.142101+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.574969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.574969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":600,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.824406+0000","last_change":"2024-02-18T00:41:29.051297+0000","last_active":"2024-02-18T00:41:39.824406+0000","last_peered":"2024-02-18T00:41:39.824406+0000","last_clean":"2024-02-18T00:41:39.824406+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:41:39.824406+0000","last_undegraded":"2024-02-18T00:41:39.824406+0000","last_fullsized":"2024-02-18T00:41:39.824406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:41:29.051225+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742069+0000","last_change":"2024-02-18T00:41:34.509265+0000","last_active":"2024-02-18T00:41:39.742069+0000","last_peered":"2024-02-18T00:41:39.742069+0000","last_clean":"2024-02-18T00:41:39.742069+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:41:39.742069+0000","last_undegraded":"2024-02-18T00:41:39.742069+0000","last_fullsized":"2024-02-18T00:41:39.742069+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.509185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.509185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818530+0000","last_change":"2024-02-18T00:41:15.821414+0000","last_active":"2024-02-18T00:41:39.818530+0000","last_peered":"2024-02-18T00:41:39.818530+0000","last_clean":"2024-02-18T00:41:39.818530+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:41:39.818530+0000","last_undegraded":"2024-02-18T00:41:39.818530+0000","last_fullsized":"2024-02-18T00:41:39.818530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:15.821337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.821337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":554,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744231+0000","last_change":"2024-02-18T00:41:25.756779+0000","last_active":"2024-02-18T00:41:39.744231+0000","last_peered":"2024-02-18T00:41:39.744231+0000","last_clean":"2024-02-18T00:41:39.744231+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:41:39.744231+0000","last_undegraded":"2024-02-18T00:41:39.744231+0000","last_fullsized":"2024-02-18T00:41:39.744231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:18.151671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.151671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":330,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.740058+0000","last_change":"2024-02-18T00:39:36.758179+0000","last_active":"2024-02-18T00:41:39.740058+0000","last_peered":"2024-02-18T00:41:39.740058+0000","last_clean":"2024-02-18T00:41:39.740058+0000","last_became_active":"2024-02-18T00:39:36.757501+0000","last_became_peered":"2024-02-18T00:39:36.757501+0000","last_unstale":"2024-02-18T00:41:39.740058+0000","last_undegraded":"2024-02-18T00:41:39.740058+0000","last_fullsized":"2024-02-18T00:41:39.740058+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:33.312079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:33.312079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":568,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.827000+0000","last_change":"2024-02-18T00:39:55.199244+0000","last_active":"2024-02-18T00:41:39.827000+0000","last_peered":"2024-02-18T00:41:39.827000+0000","last_clean":"2024-02-18T00:41:39.827000+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:41:39.827000+0000","last_undegraded":"2024-02-18T00:41:39.827000+0000","last_fullsized":"2024-02-18T00:41:39.827000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:39:55.199132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:55.199132+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":566,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.817808+0000","last_change":"2024-02-18T00:41:32.807129+0000","last_active":"2024-02-18T00:41:39.817808+0000","last_peered":"2024-02-18T00:41:39.817808+0000","last_clean":"2024-02-18T00:41:39.817808+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:41:39.817808+0000","last_undegraded":"2024-02-18T00:41:39.817808+0000","last_fullsized":"2024-02-18T00:41:39.817808+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:05.408648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:40:05.408648+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":724,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742719+0000","last_change":"2024-02-18T00:41:22.121963+0000","last_active":"2024-02-18T00:41:39.742719+0000","last_peered":"2024-02-18T00:41:39.742719+0000","last_clean":"2024-02-18T00:41:39.742719+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:41:39.742719+0000","last_undegraded":"2024-02-18T00:41:39.742719+0000","last_fullsized":"2024-02-18T00:41:39.742719+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:41:22.121879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:22.121879+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":552,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.826272+0000","last_change":"2024-02-18T00:41:29.740810+0000","last_active":"2024-02-18T00:41:39.826272+0000","last_peered":"2024-02-18T00:41:39.826272+0000","last_clean":"2024-02-18T00:41:39.826272+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:41:39.826272+0000","last_undegraded":"2024-02-18T00:41:39.826272+0000","last_fullsized":"2024-02-18T00:41:39.826272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:39.193082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.193082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":486,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818141+0000","last_change":"2024-02-18T00:40:36.519540+0000","last_active":"2024-02-18T00:41:39.818141+0000","last_peered":"2024-02-18T00:41:39.818141+0000","last_clean":"2024-02-18T00:41:39.818141+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:41:39.818141+0000","last_undegraded":"2024-02-18T00:41:39.818141+0000","last_fullsized":"2024-02-18T00:41:39.818141+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.519470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.519470+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":560,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144035+0000","last_change":"2024-02-18T00:41:31.665553+0000","last_active":"2024-02-18T00:41:39.144035+0000","last_peered":"2024-02-18T00:41:39.144035+0000","last_clean":"2024-02-18T00:41:39.144035+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:41:39.144035+0000","last_undegraded":"2024-02-18T00:41:39.144035+0000","last_fullsized":"2024-02-18T00:41:39.144035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:41:31.665435+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.665435+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":573,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818556+0000","last_change":"2024-02-18T00:40:01.450474+0000","last_active":"2024-02-18T00:41:39.818556+0000","last_peered":"2024-02-18T00:41:39.818556+0000","last_clean":"2024-02-18T00:41:39.818556+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:41:39.818556+0000","last_undegraded":"2024-02-18T00:41:39.818556+0000","last_fullsized":"2024-02-18T00:41:39.818556+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:01.450328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:40:01.450328+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":693,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.826200+0000","last_change":"2024-02-18T00:40:38.233919+0000","last_active":"2024-02-18T00:41:39.826200+0000","last_peered":"2024-02-18T00:41:39.826200+0000","last_clean":"2024-02-18T00:41:39.826200+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:41:39.826200+0000","last_undegraded":"2024-02-18T00:41:39.826200+0000","last_fullsized":"2024-02-18T00:41:39.826200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:40:38.233838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.233838+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":548,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145586+0000","last_change":"2024-02-18T00:41:33.828221+0000","last_active":"2024-02-18T00:41:39.145586+0000","last_peered":"2024-02-18T00:41:39.145586+0000","last_clean":"2024-02-18T00:41:39.145586+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:41:39.145586+0000","last_undegraded":"2024-02-18T00:41:39.145586+0000","last_fullsized":"2024-02-18T00:41:39.145586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.596666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.596666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":480,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819189+0000","last_change":"2024-02-18T00:41:15.124842+0000","last_active":"2024-02-18T00:41:39.819189+0000","last_peered":"2024-02-18T00:41:39.819189+0000","last_clean":"2024-02-18T00:41:39.819189+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:41:39.819189+0000","last_undegraded":"2024-02-18T00:41:39.819189+0000","last_fullsized":"2024-02-18T00:41:39.819189+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:15.124769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.124769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'25","reported_seq":612,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.826041+0000","last_change":"2024-02-18T00:41:36.043652+0000","last_active":"2024-02-18T00:41:39.826041+0000","last_peered":"2024-02-18T00:41:39.826041+0000","last_clean":"2024-02-18T00:41:39.826041+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:41:39.826041+0000","last_undegraded":"2024-02-18T00:41:39.826041+0000","last_fullsized":"2024-02-18T00:41:39.826041+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:36.043577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.043577+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":110,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.740613+0000","last_change":"2024-02-18T00:40:52.319724+0000","last_active":"2024-02-18T00:41:39.740613+0000","last_peered":"2024-02-18T00:41:39.740613+0000","last_clean":"2024-02-18T00:41:39.740613+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:41:39.740613+0000","last_undegraded":"2024-02-18T00:41:39.740613+0000","last_fullsized":"2024-02-18T00:41:39.740613+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.319660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.319660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":532,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144850+0000","last_change":"2024-02-18T00:40:08.617435+0000","last_active":"2024-02-18T00:41:39.144850+0000","last_peered":"2024-02-18T00:41:39.144850+0000","last_clean":"2024-02-18T00:41:39.144850+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:41:39.144850+0000","last_undegraded":"2024-02-18T00:41:39.144850+0000","last_fullsized":"2024-02-18T00:41:39.144850+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.617349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.617349+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":546,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.752176+0000","last_change":"2024-02-18T00:41:37.801538+0000","last_active":"2024-02-18T00:41:39.752176+0000","last_peered":"2024-02-18T00:41:39.752176+0000","last_clean":"2024-02-18T00:41:39.752176+0000","last_became_active":"2024-02-18T00:41:37.801183+0000","last_became_peered":"2024-02-18T00:41:37.801183+0000","last_unstale":"2024-02-18T00:41:39.752176+0000","last_undegraded":"2024-02-18T00:41:39.752176+0000","last_fullsized":"2024-02-18T00:41:39.752176+0000","mapping_epoch":62,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":564,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.016774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.016774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":480,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.827167+0000","last_change":"2024-02-18T00:40:44.111488+0000","last_active":"2024-02-18T00:41:39.827167+0000","last_peered":"2024-02-18T00:41:39.827167+0000","last_clean":"2024-02-18T00:41:39.827167+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:41:39.827167+0000","last_undegraded":"2024-02-18T00:41:39.827167+0000","last_fullsized":"2024-02-18T00:41:39.827167+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:44.111388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:40:44.111388+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":688,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.751934+0000","last_change":"2024-02-18T00:40:39.136955+0000","last_active":"2024-02-18T00:41:39.751934+0000","last_peered":"2024-02-18T00:41:39.751934+0000","last_clean":"2024-02-18T00:41:39.751934+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:41:39.751934+0000","last_undegraded":"2024-02-18T00:41:39.751934+0000","last_fullsized":"2024-02-18T00:41:39.751934+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:40:39.136881+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.136881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":595,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.828341+0000","last_change":"2024-02-18T00:39:11.324891+0000","last_active":"2024-02-18T00:41:39.828341+0000","last_peered":"2024-02-18T00:41:39.828341+0000","last_clean":"2024-02-18T00:41:39.828341+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:41:39.828341+0000","last_undegraded":"2024-02-18T00:41:39.828341+0000","last_fullsized":"2024-02-18T00:41:39.828341+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'20","last_scrub_stamp":"2024-02-18T00:39:11.324814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.324814+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":568,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.146352+0000","last_change":"2024-02-18T00:40:52.642135+0000","last_active":"2024-02-18T00:41:39.146352+0000","last_peered":"2024-02-18T00:41:39.146352+0000","last_clean":"2024-02-18T00:41:39.146352+0000","last_became_active":"2024-02-18T00:39:36.752210+0000","last_became_peered":"2024-02-18T00:39:36.752210+0000","last_unstale":"2024-02-18T00:41:39.146352+0000","last_undegraded":"2024-02-18T00:41:39.146352+0000","last_fullsized":"2024-02-18T00:41:39.146352+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.642073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742206+0000","last_change":"2024-02-18T00:39:56.697241+0000","last_active":"2024-02-18T00:41:39.742206+0000","last_peered":"2024-02-18T00:41:39.742206+0000","last_clean":"2024-02-18T00:41:39.742206+0000","last_became_active":"2024-02-18T00:39:36.767278+0000","last_became_peered":"2024-02-18T00:39:36.767278+0000","last_unstale":"2024-02-18T00:41:39.742206+0000","last_undegraded":"2024-02-18T00:41:39.742206+0000","last_fullsized":"2024-02-18T00:41:39.742206+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:56.697153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:56.697153+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":476,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144146+0000","last_change":"2024-02-18T00:40:45.658350+0000","last_active":"2024-02-18T00:41:39.144146+0000","last_peered":"2024-02-18T00:41:39.144146+0000","last_clean":"2024-02-18T00:41:39.144146+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:41:39.144146+0000","last_undegraded":"2024-02-18T00:41:39.144146+0000","last_fullsized":"2024-02-18T00:41:39.144146+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:45.658288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:45.658288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":554,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.828625+0000","last_change":"2024-02-18T00:39:36.743164+0000","last_active":"2024-02-18T00:41:39.828625+0000","last_peered":"2024-02-18T00:41:39.828625+0000","last_clean":"2024-02-18T00:41:39.828625+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:41:39.828625+0000","last_undegraded":"2024-02-18T00:41:39.828625+0000","last_fullsized":"2024-02-18T00:41:39.828625+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:21.298549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:21.298549+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":684,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.751886+0000","last_change":"2024-02-18T00:40:46.010884+0000","last_active":"2024-02-18T00:41:39.751886+0000","last_peered":"2024-02-18T00:41:39.751886+0000","last_clean":"2024-02-18T00:41:39.751886+0000","last_became_active":"2024-02-18T00:39:36.766141+0000","last_became_peered":"2024-02-18T00:39:36.766141+0000","last_unstale":"2024-02-18T00:41:39.751886+0000","last_undegraded":"2024-02-18T00:41:39.751886+0000","last_fullsized":"2024-02-18T00:41:39.751886+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:40:46.010811+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.010811+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":560,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819800+0000","last_change":"2024-02-18T00:41:17.828916+0000","last_active":"2024-02-18T00:41:39.819800+0000","last_peered":"2024-02-18T00:41:39.819800+0000","last_clean":"2024-02-18T00:41:39.819800+0000","last_became_active":"2024-02-18T00:33:04.715444+0000","last_became_peered":"2024-02-18T00:33:04.715444+0000","last_unstale":"2024-02-18T00:41:39.819800+0000","last_undegraded":"2024-02-18T00:41:39.819800+0000","last_fullsized":"2024-02-18T00:41:39.819800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.828800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.828800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.817912+0000","last_change":"2024-02-18T00:40:04.189826+0000","last_active":"2024-02-18T00:41:39.817912+0000","last_peered":"2024-02-18T00:41:39.817912+0000","last_clean":"2024-02-18T00:41:39.817912+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:41:39.817912+0000","last_undegraded":"2024-02-18T00:41:39.817912+0000","last_fullsized":"2024-02-18T00:41:39.817912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:04.189756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:04.189756+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":474,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.749662+0000","last_change":"2024-02-18T00:41:26.953890+0000","last_active":"2024-02-18T00:41:39.749662+0000","last_peered":"2024-02-18T00:41:39.749662+0000","last_clean":"2024-02-18T00:41:39.749662+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:41:39.749662+0000","last_undegraded":"2024-02-18T00:41:39.749662+0000","last_fullsized":"2024-02-18T00:41:39.749662+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.952674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.952674+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":532,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.143057+0000","last_change":"2024-02-18T00:39:54.964783+0000","last_active":"2024-02-18T00:41:39.143057+0000","last_peered":"2024-02-18T00:41:39.143057+0000","last_clean":"2024-02-18T00:41:39.143057+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:41:39.143057+0000","last_undegraded":"2024-02-18T00:41:39.143057+0000","last_fullsized":"2024-02-18T00:41:39.143057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.964677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.964677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":660,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.745303+0000","last_change":"2024-02-18T00:40:40.195949+0000","last_active":"2024-02-18T00:41:39.745303+0000","last_peered":"2024-02-18T00:41:39.745303+0000","last_clean":"2024-02-18T00:41:39.745303+0000","last_became_active":"2024-02-18T00:39:36.750472+0000","last_became_peered":"2024-02-18T00:39:36.750472+0000","last_unstale":"2024-02-18T00:41:39.745303+0000","last_undegraded":"2024-02-18T00:41:39.745303+0000","last_fullsized":"2024-02-18T00:41:39.745303+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:40:40.195882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.195882+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":110,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742003+0000","last_change":"2024-02-18T00:41:20.138251+0000","last_active":"2024-02-18T00:41:39.742003+0000","last_peered":"2024-02-18T00:41:39.742003+0000","last_clean":"2024-02-18T00:41:39.742003+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:41:39.742003+0000","last_undegraded":"2024-02-18T00:41:39.742003+0000","last_fullsized":"2024-02-18T00:41:39.742003+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:41:20.138183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:20.138183+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":540,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818672+0000","last_change":"2024-02-18T00:40:40.614193+0000","last_active":"2024-02-18T00:41:39.818672+0000","last_peered":"2024-02-18T00:41:39.818672+0000","last_clean":"2024-02-18T00:41:39.818672+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:41:39.818672+0000","last_undegraded":"2024-02-18T00:41:39.818672+0000","last_fullsized":"2024-02-18T00:41:39.818672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:40.614069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.614069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":469,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.746931+0000","last_change":"2024-02-18T00:38:57.682162+0000","last_active":"2024-02-18T00:41:39.746931+0000","last_peered":"2024-02-18T00:41:39.746931+0000","last_clean":"2024-02-18T00:41:39.746931+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:41:39.746931+0000","last_undegraded":"2024-02-18T00:41:39.746931+0000","last_fullsized":"2024-02-18T00:41:39.746931+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:57.682091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:57.682091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":532,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145002+0000","last_change":"2024-02-18T00:40:38.511704+0000","last_active":"2024-02-18T00:41:39.145002+0000","last_peered":"2024-02-18T00:41:39.145002+0000","last_clean":"2024-02-18T00:41:39.145002+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:41:39.145002+0000","last_undegraded":"2024-02-18T00:41:39.145002+0000","last_fullsized":"2024-02-18T00:41:39.145002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.511625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.511625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'24","reported_seq":572,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743922+0000","last_change":"2024-02-18T00:40:39.224307+0000","last_active":"2024-02-18T00:41:39.743922+0000","last_peered":"2024-02-18T00:41:39.743922+0000","last_clean":"2024-02-18T00:41:39.743922+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:41:39.743922+0000","last_undegraded":"2024-02-18T00:41:39.743922+0000","last_fullsized":"2024-02-18T00:41:39.743922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:40:39.224211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.224211+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":105,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.739990+0000","last_change":"2024-02-18T00:39:43.733207+0000","last_active":"2024-02-18T00:41:39.739990+0000","last_peered":"2024-02-18T00:41:39.739990+0000","last_clean":"2024-02-18T00:41:39.739990+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:41:39.739990+0000","last_undegraded":"2024-02-18T00:41:39.739990+0000","last_fullsized":"2024-02-18T00:41:39.739990+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:16.494804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:38:16.494804+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":537,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742596+0000","last_change":"2024-02-18T00:40:37.156277+0000","last_active":"2024-02-18T00:41:39.742596+0000","last_peered":"2024-02-18T00:41:39.742596+0000","last_clean":"2024-02-18T00:41:39.742596+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:41:39.742596+0000","last_undegraded":"2024-02-18T00:41:39.742596+0000","last_fullsized":"2024-02-18T00:41:39.742596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:37.156213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:40:37.156213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":540,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818619+0000","last_change":"2024-02-18T00:41:31.791248+0000","last_active":"2024-02-18T00:41:39.818619+0000","last_peered":"2024-02-18T00:41:39.818619+0000","last_clean":"2024-02-18T00:41:39.818619+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:41:39.818619+0000","last_undegraded":"2024-02-18T00:41:39.818619+0000","last_fullsized":"2024-02-18T00:41:39.818619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:31.791120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.791120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":497,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741998+0000","last_change":"2024-02-18T00:41:20.995750+0000","last_active":"2024-02-18T00:41:39.741998+0000","last_peered":"2024-02-18T00:41:39.741998+0000","last_clean":"2024-02-18T00:41:39.741998+0000","last_became_active":"2024-02-18T00:39:36.751747+0000","last_became_peered":"2024-02-18T00:39:36.751747+0000","last_unstale":"2024-02-18T00:41:39.741998+0000","last_undegraded":"2024-02-18T00:41:39.741998+0000","last_fullsized":"2024-02-18T00:41:39.741998+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:20.995657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:20.995657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742003+0000","last_change":"2024-02-18T00:40:38.746495+0000","last_active":"2024-02-18T00:41:39.742003+0000","last_peered":"2024-02-18T00:41:39.742003+0000","last_clean":"2024-02-18T00:41:39.742003+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:41:39.742003+0000","last_undegraded":"2024-02-18T00:41:39.742003+0000","last_fullsized":"2024-02-18T00:41:39.742003+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.746421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.746421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":629,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.147371+0000","last_change":"2024-02-18T00:39:50.936881+0000","last_active":"2024-02-18T00:41:39.147371+0000","last_peered":"2024-02-18T00:41:39.147371+0000","last_clean":"2024-02-18T00:41:39.147371+0000","last_became_active":"2024-02-18T00:39:36.751539+0000","last_became_peered":"2024-02-18T00:39:36.751539+0000","last_unstale":"2024-02-18T00:41:39.147371+0000","last_undegraded":"2024-02-18T00:41:39.147371+0000","last_fullsized":"2024-02-18T00:41:39.147371+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:39:50.936751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:50.936751+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741224+0000","last_change":"2024-02-18T00:39:58.671410+0000","last_active":"2024-02-18T00:41:39.741224+0000","last_peered":"2024-02-18T00:41:39.741224+0000","last_clean":"2024-02-18T00:41:39.741224+0000","last_became_active":"2024-02-18T00:39:36.759224+0000","last_became_peered":"2024-02-18T00:39:36.759224+0000","last_unstale":"2024-02-18T00:41:39.741224+0000","last_undegraded":"2024-02-18T00:41:39.741224+0000","last_fullsized":"2024-02-18T00:41:39.741224+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:58.671327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:39:58.671327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":572,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142245+0000","last_change":"2024-02-18T00:40:00.515636+0000","last_active":"2024-02-18T00:41:39.142245+0000","last_peered":"2024-02-18T00:41:39.142245+0000","last_clean":"2024-02-18T00:41:39.142245+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:41:39.142245+0000","last_undegraded":"2024-02-18T00:41:39.142245+0000","last_fullsized":"2024-02-18T00:41:39.142245+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:00.515564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:00.515564+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":528,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.748218+0000","last_change":"2024-02-18T00:39:02.673159+0000","last_active":"2024-02-18T00:41:39.748218+0000","last_peered":"2024-02-18T00:41:39.748218+0000","last_clean":"2024-02-18T00:41:39.748218+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:41:39.748218+0000","last_undegraded":"2024-02-18T00:41:39.748218+0000","last_fullsized":"2024-02-18T00:41:39.748218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:02.673088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:39:02.673088+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":530,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145105+0000","last_change":"2024-02-18T00:39:56.621857+0000","last_active":"2024-02-18T00:41:39.145105+0000","last_peered":"2024-02-18T00:41:39.145105+0000","last_clean":"2024-02-18T00:41:39.145105+0000","last_became_active":"2024-02-18T00:29:21.100282+0000","last_became_peered":"2024-02-18T00:29:21.100282+0000","last_unstale":"2024-02-18T00:41:39.145105+0000","last_undegraded":"2024-02-18T00:41:39.145105+0000","last_fullsized":"2024-02-18T00:41:39.145105+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:56.621792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:39:56.621792+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742459+0000","last_change":"2024-02-18T00:41:26.021783+0000","last_active":"2024-02-18T00:41:39.742459+0000","last_peered":"2024-02-18T00:41:39.742459+0000","last_clean":"2024-02-18T00:41:39.742459+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:41:39.742459+0000","last_undegraded":"2024-02-18T00:41:39.742459+0000","last_fullsized":"2024-02-18T00:41:39.742459+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.021716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.021716+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":556,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743183+0000","last_change":"2024-02-18T00:40:34.340695+0000","last_active":"2024-02-18T00:41:39.743183+0000","last_peered":"2024-02-18T00:41:39.743183+0000","last_clean":"2024-02-18T00:41:39.743183+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:41:39.743183+0000","last_undegraded":"2024-02-18T00:41:39.743183+0000","last_fullsized":"2024-02-18T00:41:39.743183+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.340624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.340624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":592,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819927+0000","last_change":"2024-02-18T00:40:11.447344+0000","last_active":"2024-02-18T00:41:39.819927+0000","last_peered":"2024-02-18T00:41:39.819927+0000","last_clean":"2024-02-18T00:41:39.819927+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:41:39.819927+0000","last_undegraded":"2024-02-18T00:41:39.819927+0000","last_fullsized":"2024-02-18T00:41:39.819927+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:40:11.447198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:11.447198+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":551,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741329+0000","last_change":"2024-02-18T00:40:06.037002+0000","last_active":"2024-02-18T00:41:39.741329+0000","last_peered":"2024-02-18T00:41:39.741329+0000","last_clean":"2024-02-18T00:41:39.741329+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:41:39.741329+0000","last_undegraded":"2024-02-18T00:41:39.741329+0000","last_fullsized":"2024-02-18T00:41:39.741329+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:06.036932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.036932+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1290,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:40.216559+0000","last_change":"2024-02-18T00:41:25.818891+0000","last_active":"2024-02-18T00:41:40.216559+0000","last_peered":"2024-02-18T00:41:40.216559+0000","last_clean":"2024-02-18T00:41:40.216559+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:41:40.216559+0000","last_undegraded":"2024-02-18T00:41:40.216559+0000","last_fullsized":"2024-02-18T00:41:40.216559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:41:25.818811+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:41:25.818811+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":736,"num_read_kb":731,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":116,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743307+0000","last_change":"2024-02-18T00:41:36.515174+0000","last_active":"2024-02-18T00:41:39.743307+0000","last_peered":"2024-02-18T00:41:39.743307+0000","last_clean":"2024-02-18T00:41:39.743307+0000","last_became_active":"2024-02-18T00:39:36.764502+0000","last_became_peered":"2024-02-18T00:39:36.764502+0000","last_unstale":"2024-02-18T00:41:39.743307+0000","last_undegraded":"2024-02-18T00:41:39.743307+0000","last_fullsized":"2024-02-18T00:41:39.743307+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:41:36.515095+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.515095+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":498,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.828705+0000","last_change":"2024-02-18T00:41:34.091992+0000","last_active":"2024-02-18T00:41:39.828705+0000","last_peered":"2024-02-18T00:41:39.828705+0000","last_clean":"2024-02-18T00:41:39.828705+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:41:39.828705+0000","last_undegraded":"2024-02-18T00:41:39.828705+0000","last_fullsized":"2024-02-18T00:41:39.828705+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.091913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.091913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741172+0000","last_change":"2024-02-18T00:40:34.247010+0000","last_active":"2024-02-18T00:41:39.741172+0000","last_peered":"2024-02-18T00:41:39.741172+0000","last_clean":"2024-02-18T00:41:39.741172+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:41:39.741172+0000","last_undegraded":"2024-02-18T00:41:39.741172+0000","last_fullsized":"2024-02-18T00:41:39.741172+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.246941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.246941+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'47","reported_seq":429,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144118+0000","last_change":"2024-02-18T00:39:57.582291+0000","last_active":"2024-02-18T00:41:39.144118+0000","last_peered":"2024-02-18T00:41:39.144118+0000","last_clean":"2024-02-18T00:41:39.144118+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:41:39.144118+0000","last_undegraded":"2024-02-18T00:41:39.144118+0000","last_fullsized":"2024-02-18T00:41:39.144118+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:39:57.582210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:57.582210+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":545,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.143238+0000","last_change":"2024-02-18T00:40:03.521856+0000","last_active":"2024-02-18T00:41:39.143238+0000","last_peered":"2024-02-18T00:41:39.143238+0000","last_clean":"2024-02-18T00:41:39.143238+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:41:39.143238+0000","last_undegraded":"2024-02-18T00:41:39.143238+0000","last_fullsized":"2024-02-18T00:41:39.143238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:03.521798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.521798+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":541,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.749183+0000","last_change":"2024-02-18T00:39:36.761763+0000","last_active":"2024-02-18T00:41:39.749183+0000","last_peered":"2024-02-18T00:41:39.749183+0000","last_clean":"2024-02-18T00:41:39.749183+0000","last_became_active":"2024-02-18T00:39:36.761490+0000","last_became_peered":"2024-02-18T00:39:36.761490+0000","last_unstale":"2024-02-18T00:41:39.749183+0000","last_undegraded":"2024-02-18T00:41:39.749183+0000","last_fullsized":"2024-02-18T00:41:39.749183+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.570919+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":479,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744532+0000","last_change":"2024-02-18T00:41:32.116478+0000","last_active":"2024-02-18T00:41:39.744532+0000","last_peered":"2024-02-18T00:41:39.744532+0000","last_clean":"2024-02-18T00:41:39.744532+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:41:39.744532+0000","last_undegraded":"2024-02-18T00:41:39.744532+0000","last_fullsized":"2024-02-18T00:41:39.744532+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.116365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.116365+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":691,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.745425+0000","last_change":"2024-02-18T00:41:15.145124+0000","last_active":"2024-02-18T00:41:39.745425+0000","last_peered":"2024-02-18T00:41:39.745425+0000","last_clean":"2024-02-18T00:41:39.745425+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:41:39.745425+0000","last_undegraded":"2024-02-18T00:41:39.745425+0000","last_fullsized":"2024-02-18T00:41:39.745425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:41:15.145053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.145053+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":555,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741597+0000","last_change":"2024-02-18T00:41:36.069318+0000","last_active":"2024-02-18T00:41:39.741597+0000","last_peered":"2024-02-18T00:41:39.741597+0000","last_clean":"2024-02-18T00:41:39.741597+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:41:39.741597+0000","last_undegraded":"2024-02-18T00:41:39.741597+0000","last_fullsized":"2024-02-18T00:41:39.741597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:41:36.069252+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.069252+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":578,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818376+0000","last_change":"2024-02-18T00:40:38.550394+0000","last_active":"2024-02-18T00:41:39.818376+0000","last_peered":"2024-02-18T00:41:39.818376+0000","last_clean":"2024-02-18T00:41:39.818376+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:41:39.818376+0000","last_undegraded":"2024-02-18T00:41:39.818376+0000","last_fullsized":"2024-02-18T00:41:39.818376+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.550323+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":527,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741753+0000","last_change":"2024-02-18T00:39:49.207377+0000","last_active":"2024-02-18T00:41:39.741753+0000","last_peered":"2024-02-18T00:41:39.741753+0000","last_clean":"2024-02-18T00:41:39.741753+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:41:39.741753+0000","last_undegraded":"2024-02-18T00:41:39.741753+0000","last_fullsized":"2024-02-18T00:41:39.741753+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:49.207314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:49.207314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":480,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.827169+0000","last_change":"2024-02-18T00:41:17.092835+0000","last_active":"2024-02-18T00:41:39.827169+0000","last_peered":"2024-02-18T00:41:39.827169+0000","last_clean":"2024-02-18T00:41:39.827169+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:41:39.827169+0000","last_undegraded":"2024-02-18T00:41:39.827169+0000","last_fullsized":"2024-02-18T00:41:39.827169+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.092717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.092717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742027+0000","last_change":"2024-02-18T00:41:32.150056+0000","last_active":"2024-02-18T00:41:39.742027+0000","last_peered":"2024-02-18T00:41:39.742027+0000","last_clean":"2024-02-18T00:41:39.742027+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:41:39.742027+0000","last_undegraded":"2024-02-18T00:41:39.742027+0000","last_fullsized":"2024-02-18T00:41:39.742027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.149984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.149984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":600,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.828436+0000","last_change":"2024-02-18T00:40:33.214440+0000","last_active":"2024-02-18T00:41:39.828436+0000","last_peered":"2024-02-18T00:41:39.828436+0000","last_clean":"2024-02-18T00:41:39.828436+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:41:39.828436+0000","last_undegraded":"2024-02-18T00:41:39.828436+0000","last_fullsized":"2024-02-18T00:41:39.828436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:40:33.214358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:33.214358+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":588,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818541+0000","last_change":"2024-02-18T00:40:40.158702+0000","last_active":"2024-02-18T00:41:39.818541+0000","last_peered":"2024-02-18T00:41:39.818541+0000","last_clean":"2024-02-18T00:41:39.818541+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:41:39.818541+0000","last_undegraded":"2024-02-18T00:41:39.818541+0000","last_fullsized":"2024-02-18T00:41:39.818541+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:40:40.158593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.158593+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":556,"reported_epoch":567,"state":"peering","last_fresh":"2024-02-18T00:41:40.752092+0000","last_change":"2024-02-18T00:41:40.752288+0000","last_active":"2024-02-18T00:41:38.743738+0000","last_peered":"2024-02-18T00:41:39.819251+0000","last_clean":"2024-02-18T00:41:39.819251+0000","last_became_active":"2024-02-18T00:39:36.762201+0000","last_became_peered":"2024-02-18T00:41:39.817636+0000","last_unstale":"2024-02-18T00:41:40.752092+0000","last_undegraded":"2024-02-18T00:41:40.752092+0000","last_fullsized":"2024-02-18T00:41:40.752092+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:03.980632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.980632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":480,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819193+0000","last_change":"2024-02-18T00:41:31.059363+0000","last_active":"2024-02-18T00:41:39.819193+0000","last_peered":"2024-02-18T00:41:39.819193+0000","last_clean":"2024-02-18T00:41:39.819193+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:41:39.819193+0000","last_undegraded":"2024-02-18T00:41:39.819193+0000","last_fullsized":"2024-02-18T00:41:39.819193+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:31.059285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.059285+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":637,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744012+0000","last_change":"2024-02-18T00:40:51.299875+0000","last_active":"2024-02-18T00:41:39.744012+0000","last_peered":"2024-02-18T00:41:39.744012+0000","last_clean":"2024-02-18T00:41:39.744012+0000","last_became_active":"2024-02-18T00:39:36.771394+0000","last_became_peered":"2024-02-18T00:39:36.771394+0000","last_unstale":"2024-02-18T00:41:39.744012+0000","last_undegraded":"2024-02-18T00:41:39.744012+0000","last_fullsized":"2024-02-18T00:41:39.744012+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:40:51.299791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.299791+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741881+0000","last_change":"2024-02-18T00:41:21.141528+0000","last_active":"2024-02-18T00:41:39.741881+0000","last_peered":"2024-02-18T00:41:39.741881+0000","last_clean":"2024-02-18T00:41:39.741881+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:41:39.741881+0000","last_undegraded":"2024-02-18T00:41:39.741881+0000","last_fullsized":"2024-02-18T00:41:39.741881+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:21.141464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.141464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":548,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.749958+0000","last_change":"2024-02-18T00:40:33.083220+0000","last_active":"2024-02-18T00:41:39.749958+0000","last_peered":"2024-02-18T00:41:39.749958+0000","last_clean":"2024-02-18T00:41:39.749958+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:41:39.749958+0000","last_undegraded":"2024-02-18T00:41:39.749958+0000","last_fullsized":"2024-02-18T00:41:39.749958+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:33.083145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:33.083145+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":551,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744297+0000","last_change":"2024-02-18T00:39:36.758086+0000","last_active":"2024-02-18T00:41:39.744297+0000","last_peered":"2024-02-18T00:41:39.744297+0000","last_clean":"2024-02-18T00:41:39.744297+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:41:39.744297+0000","last_undegraded":"2024-02-18T00:41:39.744297+0000","last_fullsized":"2024-02-18T00:41:39.744297+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:18.211326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:38:18.211326+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":474,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742381+0000","last_change":"2024-02-18T00:39:09.589674+0000","last_active":"2024-02-18T00:41:39.742381+0000","last_peered":"2024-02-18T00:41:39.742381+0000","last_clean":"2024-02-18T00:41:39.742381+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:41:39.742381+0000","last_undegraded":"2024-02-18T00:41:39.742381+0000","last_fullsized":"2024-02-18T00:41:39.742381+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:09.589601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:09.589601+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":622,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.749881+0000","last_change":"2024-02-18T00:39:38.409643+0000","last_active":"2024-02-18T00:41:39.749881+0000","last_peered":"2024-02-18T00:41:39.749881+0000","last_clean":"2024-02-18T00:41:39.749881+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:41:39.749881+0000","last_undegraded":"2024-02-18T00:41:39.749881+0000","last_fullsized":"2024-02-18T00:41:39.749881+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'12","last_scrub_stamp":"2024-02-18T00:39:07.653633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.653633+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":541,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142899+0000","last_change":"2024-02-18T00:40:38.674457+0000","last_active":"2024-02-18T00:41:39.142899+0000","last_peered":"2024-02-18T00:41:39.142899+0000","last_clean":"2024-02-18T00:41:39.142899+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:41:39.142899+0000","last_undegraded":"2024-02-18T00:41:39.142899+0000","last_fullsized":"2024-02-18T00:41:39.142899+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.674378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.674378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":532,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144693+0000","last_change":"2024-02-18T00:40:04.499875+0000","last_active":"2024-02-18T00:41:39.144693+0000","last_peered":"2024-02-18T00:41:39.144693+0000","last_clean":"2024-02-18T00:41:39.144693+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:41:39.144693+0000","last_undegraded":"2024-02-18T00:41:39.144693+0000","last_fullsized":"2024-02-18T00:41:39.144693+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:04.499796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:04.499796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743363+0000","last_change":"2024-02-18T00:41:21.478541+0000","last_active":"2024-02-18T00:41:39.743363+0000","last_peered":"2024-02-18T00:41:39.743363+0000","last_clean":"2024-02-18T00:41:39.743363+0000","last_became_active":"2024-02-18T00:39:36.759264+0000","last_became_peered":"2024-02-18T00:39:36.759264+0000","last_unstale":"2024-02-18T00:41:39.743363+0000","last_undegraded":"2024-02-18T00:41:39.743363+0000","last_fullsized":"2024-02-18T00:41:39.743363+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:21.478458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.478458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":486,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742466+0000","last_change":"2024-02-18T00:38:20.206553+0000","last_active":"2024-02-18T00:41:39.742466+0000","last_peered":"2024-02-18T00:41:39.742466+0000","last_clean":"2024-02-18T00:41:39.742466+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:41:39.742466+0000","last_undegraded":"2024-02-18T00:41:39.742466+0000","last_fullsized":"2024-02-18T00:41:39.742466+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:38:20.206476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:38:20.206476+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":642,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.752389+0000","last_change":"2024-02-18T00:40:42.124689+0000","last_active":"2024-02-18T00:41:39.752389+0000","last_peered":"2024-02-18T00:41:39.752389+0000","last_clean":"2024-02-18T00:41:39.752389+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:41:39.752389+0000","last_undegraded":"2024-02-18T00:41:39.752389+0000","last_fullsized":"2024-02-18T00:41:39.752389+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:40:42.124619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.124619+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":563,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.143154+0000","last_change":"2024-02-18T00:40:36.427315+0000","last_active":"2024-02-18T00:41:39.143154+0000","last_peered":"2024-02-18T00:41:39.143154+0000","last_clean":"2024-02-18T00:41:39.143154+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:41:39.143154+0000","last_undegraded":"2024-02-18T00:41:39.143154+0000","last_fullsized":"2024-02-18T00:41:39.143154+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.427206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.427206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":690,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145485+0000","last_change":"2024-02-18T00:40:09.668977+0000","last_active":"2024-02-18T00:41:39.145485+0000","last_peered":"2024-02-18T00:41:39.145485+0000","last_clean":"2024-02-18T00:41:39.145485+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:41:39.145485+0000","last_undegraded":"2024-02-18T00:41:39.145485+0000","last_fullsized":"2024-02-18T00:41:39.145485+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:40:09.668906+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:40:09.668906+0000","last_clean_scrub_stamp":"2024-02-18T00:40:09.668906+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":474,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.749587+0000","last_change":"2024-02-18T00:41:19.745830+0000","last_active":"2024-02-18T00:41:39.749587+0000","last_peered":"2024-02-18T00:41:39.749587+0000","last_clean":"2024-02-18T00:41:39.749587+0000","last_became_active":"2024-02-18T00:41:19.745421+0000","last_became_peered":"2024-02-18T00:41:19.745421+0000","last_unstale":"2024-02-18T00:41:39.749587+0000","last_undegraded":"2024-02-18T00:41:39.749587+0000","last_fullsized":"2024-02-18T00:41:39.749587+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:14.633705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.633705+0000","last_clean_scrub_stamp":"2024-02-18T00:39:14.633705+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":494,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.141207+0000","last_change":"2024-02-18T00:40:06.822506+0000","last_active":"2024-02-18T00:41:39.141207+0000","last_peered":"2024-02-18T00:41:39.141207+0000","last_clean":"2024-02-18T00:41:39.141207+0000","last_became_active":"2024-02-18T00:39:36.751162+0000","last_became_peered":"2024-02-18T00:39:36.751162+0000","last_unstale":"2024-02-18T00:41:39.141207+0000","last_undegraded":"2024-02-18T00:41:39.141207+0000","last_fullsized":"2024-02-18T00:41:39.141207+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:06.822430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.822430+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":616,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145373+0000","last_change":"2024-02-18T00:40:43.621296+0000","last_active":"2024-02-18T00:41:39.145373+0000","last_peered":"2024-02-18T00:41:39.145373+0000","last_clean":"2024-02-18T00:41:39.145373+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:41:39.145373+0000","last_undegraded":"2024-02-18T00:41:39.145373+0000","last_fullsized":"2024-02-18T00:41:39.145373+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:40:43.621226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:43.621226+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":538,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.745339+0000","last_change":"2024-02-18T00:40:36.184282+0000","last_active":"2024-02-18T00:41:39.745339+0000","last_peered":"2024-02-18T00:41:39.745339+0000","last_clean":"2024-02-18T00:41:39.745339+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:41:39.745339+0000","last_undegraded":"2024-02-18T00:41:39.745339+0000","last_fullsized":"2024-02-18T00:41:39.745339+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.184216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.184216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":714,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.752561+0000","last_change":"2024-02-18T00:40:06.960621+0000","last_active":"2024-02-18T00:41:39.752561+0000","last_peered":"2024-02-18T00:41:39.752561+0000","last_clean":"2024-02-18T00:41:39.752561+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:41:39.752561+0000","last_undegraded":"2024-02-18T00:41:39.752561+0000","last_fullsized":"2024-02-18T00:41:39.752561+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.960509+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":32,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144162+0000","last_change":"2024-02-18T00:41:19.747873+0000","last_active":"2024-02-18T00:41:39.144162+0000","last_peered":"2024-02-18T00:41:39.144162+0000","last_clean":"2024-02-18T00:41:39.144162+0000","last_became_active":"2024-02-18T00:41:19.747396+0000","last_became_peered":"2024-02-18T00:41:19.747396+0000","last_unstale":"2024-02-18T00:41:39.144162+0000","last_undegraded":"2024-02-18T00:41:39.144162+0000","last_fullsized":"2024-02-18T00:41:39.144162+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.157612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:12.043994+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.157612+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":476,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145194+0000","last_change":"2024-02-18T00:40:06.573220+0000","last_active":"2024-02-18T00:41:39.145194+0000","last_peered":"2024-02-18T00:41:39.145194+0000","last_clean":"2024-02-18T00:41:39.145194+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:41:39.145194+0000","last_undegraded":"2024-02-18T00:41:39.145194+0000","last_fullsized":"2024-02-18T00:41:39.145194+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.573143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":569,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.817897+0000","last_change":"2024-02-18T00:40:37.554739+0000","last_active":"2024-02-18T00:41:39.817897+0000","last_peered":"2024-02-18T00:41:39.817897+0000","last_clean":"2024-02-18T00:41:39.817897+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:41:39.817897+0000","last_undegraded":"2024-02-18T00:41:39.817897+0000","last_fullsized":"2024-02-18T00:41:39.817897+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:40:37.554662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:37.554662+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":541,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.145227+0000","last_change":"2024-02-18T00:39:53.940625+0000","last_active":"2024-02-18T00:41:39.145227+0000","last_peered":"2024-02-18T00:41:39.145227+0000","last_clean":"2024-02-18T00:41:39.145227+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:41:39.145227+0000","last_undegraded":"2024-02-18T00:41:39.145227+0000","last_fullsized":"2024-02-18T00:41:39.145227+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:53.940533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:53.940533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":530,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.746820+0000","last_change":"2024-02-18T00:40:08.961647+0000","last_active":"2024-02-18T00:41:39.746820+0000","last_peered":"2024-02-18T00:41:39.746820+0000","last_clean":"2024-02-18T00:41:39.746820+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:41:39.746820+0000","last_undegraded":"2024-02-18T00:41:39.746820+0000","last_fullsized":"2024-02-18T00:41:39.746820+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.961546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.961546+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":532,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741672+0000","last_change":"2024-02-18T00:39:01.603631+0000","last_active":"2024-02-18T00:41:39.741672+0000","last_peered":"2024-02-18T00:41:39.741672+0000","last_clean":"2024-02-18T00:41:39.741672+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:41:39.741672+0000","last_undegraded":"2024-02-18T00:41:39.741672+0000","last_fullsized":"2024-02-18T00:41:39.741672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:01.603551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:39:01.603551+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":474,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743836+0000","last_change":"2024-02-18T00:40:07.241181+0000","last_active":"2024-02-18T00:41:39.743836+0000","last_peered":"2024-02-18T00:41:39.743836+0000","last_clean":"2024-02-18T00:41:39.743836+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:41:39.743836+0000","last_undegraded":"2024-02-18T00:41:39.743836+0000","last_fullsized":"2024-02-18T00:41:39.743836+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_clean_scrub_stamp":"2024-02-18T00:40:07.241102+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":569,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819439+0000","last_change":"2024-02-18T00:39:50.310946+0000","last_active":"2024-02-18T00:41:39.819439+0000","last_peered":"2024-02-18T00:41:39.819439+0000","last_clean":"2024-02-18T00:41:39.819439+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:41:39.819439+0000","last_undegraded":"2024-02-18T00:41:39.819439+0000","last_fullsized":"2024-02-18T00:41:39.819439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:39:50.310868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:50.310868+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":546,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819176+0000","last_change":"2024-02-18T00:41:14.821572+0000","last_active":"2024-02-18T00:41:39.819176+0000","last_peered":"2024-02-18T00:41:39.819176+0000","last_clean":"2024-02-18T00:41:39.819176+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:41:39.819176+0000","last_undegraded":"2024-02-18T00:41:39.819176+0000","last_fullsized":"2024-02-18T00:41:39.819176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:14.821507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:14.821507+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":547,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.817974+0000","last_change":"2024-02-18T00:41:24.779552+0000","last_active":"2024-02-18T00:41:39.817974+0000","last_peered":"2024-02-18T00:41:39.817974+0000","last_clean":"2024-02-18T00:41:39.817974+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:41:39.817974+0000","last_undegraded":"2024-02-18T00:41:39.817974+0000","last_fullsized":"2024-02-18T00:41:39.817974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:24.779481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:24.779481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":528,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.747066+0000","last_change":"2024-02-18T00:40:52.018692+0000","last_active":"2024-02-18T00:41:39.747066+0000","last_peered":"2024-02-18T00:41:39.747066+0000","last_clean":"2024-02-18T00:41:39.747066+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:41:39.747066+0000","last_undegraded":"2024-02-18T00:41:39.747066+0000","last_fullsized":"2024-02-18T00:41:39.747066+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.018627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.018627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":476,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.147367+0000","last_change":"2024-02-18T00:40:34.828210+0000","last_active":"2024-02-18T00:41:39.147367+0000","last_peered":"2024-02-18T00:41:39.147367+0000","last_clean":"2024-02-18T00:41:39.147367+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:41:39.147367+0000","last_undegraded":"2024-02-18T00:41:39.147367+0000","last_fullsized":"2024-02-18T00:41:39.147367+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.828148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.828148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":544,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.751799+0000","last_change":"2024-02-18T00:40:43.113198+0000","last_active":"2024-02-18T00:41:39.751799+0000","last_peered":"2024-02-18T00:41:39.751799+0000","last_clean":"2024-02-18T00:41:39.751799+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:41:39.751799+0000","last_undegraded":"2024-02-18T00:41:39.751799+0000","last_fullsized":"2024-02-18T00:41:39.751799+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-18T00:40:43.113109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:43.113109+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":541,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144807+0000","last_change":"2024-02-18T00:40:50.609050+0000","last_active":"2024-02-18T00:41:39.144807+0000","last_peered":"2024-02-18T00:41:39.144807+0000","last_clean":"2024-02-18T00:41:39.144807+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:41:39.144807+0000","last_undegraded":"2024-02-18T00:41:39.144807+0000","last_fullsized":"2024-02-18T00:41:39.144807+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.608983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.608983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":555,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142807+0000","last_change":"2024-02-18T00:41:35.739124+0000","last_active":"2024-02-18T00:41:39.142807+0000","last_peered":"2024-02-18T00:41:39.142807+0000","last_clean":"2024-02-18T00:41:39.142807+0000","last_became_active":"2024-02-18T00:39:36.750866+0000","last_became_peered":"2024-02-18T00:39:36.750866+0000","last_unstale":"2024-02-18T00:41:39.142807+0000","last_undegraded":"2024-02-18T00:41:39.142807+0000","last_fullsized":"2024-02-18T00:41:39.142807+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:35.739045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:35.739045+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741683+0000","last_change":"2024-02-18T00:40:36.683678+0000","last_active":"2024-02-18T00:41:39.741683+0000","last_peered":"2024-02-18T00:41:39.741683+0000","last_clean":"2024-02-18T00:41:39.741683+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:41:39.741683+0000","last_undegraded":"2024-02-18T00:41:39.741683+0000","last_fullsized":"2024-02-18T00:41:39.741683+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.683604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.683604+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":494,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.144566+0000","last_change":"2024-02-18T00:41:18.581382+0000","last_active":"2024-02-18T00:41:39.144566+0000","last_peered":"2024-02-18T00:41:39.144566+0000","last_clean":"2024-02-18T00:41:39.144566+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:41:39.144566+0000","last_undegraded":"2024-02-18T00:41:39.144566+0000","last_fullsized":"2024-02-18T00:41:39.144566+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:18.581310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.581310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":624,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.747591+0000","last_change":"2024-02-18T00:41:18.896665+0000","last_active":"2024-02-18T00:41:39.747591+0000","last_peered":"2024-02-18T00:41:39.747591+0000","last_clean":"2024-02-18T00:41:39.747591+0000","last_became_active":"2024-02-18T00:39:36.761140+0000","last_became_peered":"2024-02-18T00:39:36.761140+0000","last_unstale":"2024-02-18T00:41:39.747591+0000","last_undegraded":"2024-02-18T00:41:39.747591+0000","last_fullsized":"2024-02-18T00:41:39.747591+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:18.896586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.896586+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":541,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.146255+0000","last_change":"2024-02-18T00:41:23.548946+0000","last_active":"2024-02-18T00:41:39.146255+0000","last_peered":"2024-02-18T00:41:39.146255+0000","last_clean":"2024-02-18T00:41:39.146255+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:41:39.146255+0000","last_undegraded":"2024-02-18T00:41:39.146255+0000","last_fullsized":"2024-02-18T00:41:39.146255+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.548872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.548872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":107,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741604+0000","last_change":"2024-02-18T00:39:36.758700+0000","last_active":"2024-02-18T00:41:39.741604+0000","last_peered":"2024-02-18T00:41:39.741604+0000","last_clean":"2024-02-18T00:41:39.741604+0000","last_became_active":"2024-02-18T00:39:36.758138+0000","last_became_peered":"2024-02-18T00:39:36.758138+0000","last_unstale":"2024-02-18T00:41:39.741604+0000","last_undegraded":"2024-02-18T00:41:39.741604+0000","last_fullsized":"2024-02-18T00:41:39.741604+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:16.384118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:16.384118+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":528,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.746851+0000","last_change":"2024-02-18T00:40:51.008109+0000","last_active":"2024-02-18T00:41:39.746851+0000","last_peered":"2024-02-18T00:41:39.746851+0000","last_clean":"2024-02-18T00:41:39.746851+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:41:39.746851+0000","last_undegraded":"2024-02-18T00:41:39.746851+0000","last_fullsized":"2024-02-18T00:41:39.746851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.008036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.008036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742266+0000","last_change":"2024-02-18T00:40:40.756033+0000","last_active":"2024-02-18T00:41:39.742266+0000","last_peered":"2024-02-18T00:41:39.742266+0000","last_clean":"2024-02-18T00:41:39.742266+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:41:39.742266+0000","last_undegraded":"2024-02-18T00:41:39.742266+0000","last_fullsized":"2024-02-18T00:41:39.742266+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:40.755973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.755973+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":633,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744240+0000","last_change":"2024-02-18T00:39:38.426514+0000","last_active":"2024-02-18T00:41:39.744240+0000","last_peered":"2024-02-18T00:41:39.744240+0000","last_clean":"2024-02-18T00:41:39.744240+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:41:39.744240+0000","last_undegraded":"2024-02-18T00:41:39.744240+0000","last_fullsized":"2024-02-18T00:41:39.744240+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'22","last_scrub_stamp":"2024-02-18T00:39:11.643593+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:39:11.643593+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":563,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.141864+0000","last_change":"2024-02-18T00:41:15.566910+0000","last_active":"2024-02-18T00:41:39.141864+0000","last_peered":"2024-02-18T00:41:39.141864+0000","last_clean":"2024-02-18T00:41:39.141864+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:41:39.141864+0000","last_undegraded":"2024-02-18T00:41:39.141864+0000","last_fullsized":"2024-02-18T00:41:39.141864+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:15.566820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.566820+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"466'10","reported_seq":728,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.146453+0000","last_change":"2024-02-18T00:41:21.505578+0000","last_active":"2024-02-18T00:41:39.146453+0000","last_peered":"2024-02-18T00:41:39.146453+0000","last_clean":"2024-02-18T00:41:39.146453+0000","last_became_active":"2024-02-18T00:39:36.751716+0000","last_became_peered":"2024-02-18T00:39:36.751716+0000","last_unstale":"2024-02-18T00:41:39.146453+0000","last_undegraded":"2024-02-18T00:41:39.146453+0000","last_fullsized":"2024-02-18T00:41:39.146453+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:41:21.505495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.505495+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741581+0000","last_change":"2024-02-18T00:40:42.799161+0000","last_active":"2024-02-18T00:41:39.741581+0000","last_peered":"2024-02-18T00:41:39.741581+0000","last_clean":"2024-02-18T00:41:39.741581+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:41:39.741581+0000","last_undegraded":"2024-02-18T00:41:39.741581+0000","last_fullsized":"2024-02-18T00:41:39.741581+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:42.799091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.799091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":585,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.146561+0000","last_change":"2024-02-18T00:41:17.546351+0000","last_active":"2024-02-18T00:41:39.146561+0000","last_peered":"2024-02-18T00:41:39.146561+0000","last_clean":"2024-02-18T00:41:39.146561+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:41:39.146561+0000","last_undegraded":"2024-02-18T00:41:39.146561+0000","last_fullsized":"2024-02-18T00:41:39.146561+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:17.546234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.546234+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.747301+0000","last_change":"2024-02-18T00:39:54.855516+0000","last_active":"2024-02-18T00:41:39.747301+0000","last_peered":"2024-02-18T00:41:39.747301+0000","last_clean":"2024-02-18T00:41:39.747301+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:41:39.747301+0000","last_undegraded":"2024-02-18T00:41:39.747301+0000","last_fullsized":"2024-02-18T00:41:39.747301+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.855452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.855452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":547,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744322+0000","last_change":"2024-02-18T00:39:48.227178+0000","last_active":"2024-02-18T00:41:39.744322+0000","last_peered":"2024-02-18T00:41:39.744322+0000","last_clean":"2024-02-18T00:41:39.744322+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:41:39.744322+0000","last_undegraded":"2024-02-18T00:41:39.744322+0000","last_fullsized":"2024-02-18T00:41:39.744322+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:48.227102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:48.227102+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":495,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742377+0000","last_change":"2024-02-18T00:41:27.140619+0000","last_active":"2024-02-18T00:41:39.742377+0000","last_peered":"2024-02-18T00:41:39.742377+0000","last_clean":"2024-02-18T00:41:39.742377+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:41:39.742377+0000","last_undegraded":"2024-02-18T00:41:39.742377+0000","last_fullsized":"2024-02-18T00:41:39.742377+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:41:27.140537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.140537+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":588,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744606+0000","last_change":"2024-02-18T00:41:24.123727+0000","last_active":"2024-02-18T00:41:39.744606+0000","last_peered":"2024-02-18T00:41:39.744606+0000","last_clean":"2024-02-18T00:41:39.744606+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:41:39.744606+0000","last_undegraded":"2024-02-18T00:41:39.744606+0000","last_fullsized":"2024-02-18T00:41:39.744606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:41:24.123655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:24.123655+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":561,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142965+0000","last_change":"2024-02-18T00:39:58.895781+0000","last_active":"2024-02-18T00:41:39.142965+0000","last_peered":"2024-02-18T00:41:39.142965+0000","last_clean":"2024-02-18T00:41:39.142965+0000","last_became_active":"2024-02-18T00:39:36.741489+0000","last_became_peered":"2024-02-18T00:39:36.741489+0000","last_unstale":"2024-02-18T00:41:39.142965+0000","last_undegraded":"2024-02-18T00:41:39.142965+0000","last_fullsized":"2024-02-18T00:41:39.142965+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:58.895686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:39:58.895686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":542,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819453+0000","last_change":"2024-02-18T00:40:08.429634+0000","last_active":"2024-02-18T00:41:39.819453+0000","last_peered":"2024-02-18T00:41:39.819453+0000","last_clean":"2024-02-18T00:41:39.819453+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:41:39.819453+0000","last_undegraded":"2024-02-18T00:41:39.819453+0000","last_fullsized":"2024-02-18T00:41:39.819453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.429566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.429566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":476,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.141926+0000","last_change":"2024-02-18T00:40:03.814573+0000","last_active":"2024-02-18T00:41:39.141926+0000","last_peered":"2024-02-18T00:41:39.141926+0000","last_clean":"2024-02-18T00:41:39.141926+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:41:39.141926+0000","last_undegraded":"2024-02-18T00:41:39.141926+0000","last_fullsized":"2024-02-18T00:41:39.141926+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:03.814478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.814478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":579,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.750235+0000","last_change":"2024-02-18T00:41:21.991859+0000","last_active":"2024-02-18T00:41:39.750235+0000","last_peered":"2024-02-18T00:41:39.750235+0000","last_clean":"2024-02-18T00:41:39.750235+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:41:39.750235+0000","last_undegraded":"2024-02-18T00:41:39.750235+0000","last_fullsized":"2024-02-18T00:41:39.750235+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:41:21.991789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.991789+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":567,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.746735+0000","last_change":"2024-02-18T00:39:38.541343+0000","last_active":"2024-02-18T00:41:39.746735+0000","last_peered":"2024-02-18T00:41:39.746735+0000","last_clean":"2024-02-18T00:41:39.746735+0000","last_became_active":"2024-02-18T00:39:36.761839+0000","last_became_peered":"2024-02-18T00:39:36.761839+0000","last_unstale":"2024-02-18T00:41:39.746735+0000","last_undegraded":"2024-02-18T00:41:39.746735+0000","last_fullsized":"2024-02-18T00:41:39.746735+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:39:15.558360+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:39:15.558360+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":536,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.827047+0000","last_change":"2024-02-18T00:39:53.231555+0000","last_active":"2024-02-18T00:41:39.827047+0000","last_peered":"2024-02-18T00:41:39.827047+0000","last_clean":"2024-02-18T00:41:39.827047+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:41:39.827047+0000","last_undegraded":"2024-02-18T00:41:39.827047+0000","last_fullsized":"2024-02-18T00:41:39.827047+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:53.231447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:53.231447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":497,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744578+0000","last_change":"2024-02-18T00:41:25.051216+0000","last_active":"2024-02-18T00:41:39.744578+0000","last_peered":"2024-02-18T00:41:39.744578+0000","last_clean":"2024-02-18T00:41:39.744578+0000","last_became_active":"2024-02-18T00:39:36.771648+0000","last_became_peered":"2024-02-18T00:39:36.771648+0000","last_unstale":"2024-02-18T00:41:39.744578+0000","last_undegraded":"2024-02-18T00:41:39.744578+0000","last_fullsized":"2024-02-18T00:41:39.744578+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:25.051146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:25.051146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":607,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.817563+0000","last_change":"2024-02-18T00:40:46.132082+0000","last_active":"2024-02-18T00:41:39.817563+0000","last_peered":"2024-02-18T00:41:39.817563+0000","last_clean":"2024-02-18T00:41:39.817563+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:41:39.817563+0000","last_undegraded":"2024-02-18T00:41:39.817563+0000","last_fullsized":"2024-02-18T00:41:39.817563+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:40:46.132007+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.132007+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741800+0000","last_change":"2024-02-18T00:41:17.540872+0000","last_active":"2024-02-18T00:41:39.741800+0000","last_peered":"2024-02-18T00:41:39.741800+0000","last_clean":"2024-02-18T00:41:39.741800+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:41:39.741800+0000","last_undegraded":"2024-02-18T00:41:39.741800+0000","last_fullsized":"2024-02-18T00:41:39.741800+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.540791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.540791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":564,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142142+0000","last_change":"2024-02-18T00:41:37.673766+0000","last_active":"2024-02-18T00:41:39.142142+0000","last_peered":"2024-02-18T00:41:39.142142+0000","last_clean":"2024-02-18T00:41:39.142142+0000","last_became_active":"2024-02-18T00:39:36.749467+0000","last_became_peered":"2024-02-18T00:39:36.749467+0000","last_unstale":"2024-02-18T00:41:39.142142+0000","last_undegraded":"2024-02-18T00:41:39.142142+0000","last_fullsized":"2024-02-18T00:41:39.142142+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_clean_scrub_stamp":"2024-02-18T00:41:37.673673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":469,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.740335+0000","last_change":"2024-02-18T00:41:19.746873+0000","last_active":"2024-02-18T00:41:39.740335+0000","last_peered":"2024-02-18T00:41:39.740335+0000","last_clean":"2024-02-18T00:41:39.740335+0000","last_became_active":"2024-02-18T00:41:19.746436+0000","last_became_peered":"2024-02-18T00:41:19.746436+0000","last_unstale":"2024-02-18T00:41:39.740335+0000","last_undegraded":"2024-02-18T00:41:39.740335+0000","last_fullsized":"2024-02-18T00:41:39.740335+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:04.255350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:04.255350+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":477,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.742851+0000","last_change":"2024-02-18T00:39:03.503323+0000","last_active":"2024-02-18T00:41:39.742851+0000","last_peered":"2024-02-18T00:41:39.742851+0000","last_clean":"2024-02-18T00:41:39.742851+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:41:39.742851+0000","last_undegraded":"2024-02-18T00:41:39.742851+0000","last_fullsized":"2024-02-18T00:41:39.742851+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:03.503256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:39:03.503256+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":569,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.819355+0000","last_change":"2024-02-18T00:39:23.328438+0000","last_active":"2024-02-18T00:41:39.819355+0000","last_peered":"2024-02-18T00:41:39.819355+0000","last_clean":"2024-02-18T00:41:39.819355+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:41:39.819355+0000","last_undegraded":"2024-02-18T00:41:39.819355+0000","last_fullsized":"2024-02-18T00:41:39.819355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:39:23.328352+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:39:23.328352+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":112,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.743053+0000","last_change":"2024-02-18T00:39:54.729770+0000","last_active":"2024-02-18T00:41:39.743053+0000","last_peered":"2024-02-18T00:41:39.743053+0000","last_clean":"2024-02-18T00:41:39.743053+0000","last_became_active":"2024-02-18T00:39:36.761469+0000","last_became_peered":"2024-02-18T00:39:36.761469+0000","last_unstale":"2024-02-18T00:41:39.743053+0000","last_undegraded":"2024-02-18T00:41:39.743053+0000","last_fullsized":"2024-02-18T00:41:39.743053+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.729675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.729675+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":107,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741791+0000","last_change":"2024-02-18T00:39:36.759178+0000","last_active":"2024-02-18T00:41:39.741791+0000","last_peered":"2024-02-18T00:41:39.741791+0000","last_clean":"2024-02-18T00:41:39.741791+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:41:39.741791+0000","last_undegraded":"2024-02-18T00:41:39.741791+0000","last_fullsized":"2024-02-18T00:41:39.741791+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:07.344483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:39:07.344483+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":31,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.741452+0000","last_change":"2024-02-18T00:41:19.748034+0000","last_active":"2024-02-18T00:41:39.741452+0000","last_peered":"2024-02-18T00:41:39.741452+0000","last_clean":"2024-02-18T00:41:39.741452+0000","last_became_active":"2024-02-18T00:41:19.747629+0000","last_became_peered":"2024-02-18T00:41:19.747629+0000","last_unstale":"2024-02-18T00:41:39.741452+0000","last_undegraded":"2024-02-18T00:41:39.741452+0000","last_fullsized":"2024-02-18T00:41:39.741452+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:53.576276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:44.843494+0000","last_clean_scrub_stamp":"2024-02-18T00:39:53.576276+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":492,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.750467+0000","last_change":"2024-02-18T00:41:27.921493+0000","last_active":"2024-02-18T00:41:39.750467+0000","last_peered":"2024-02-18T00:41:39.750467+0000","last_clean":"2024-02-18T00:41:39.750467+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:41:39.750467+0000","last_undegraded":"2024-02-18T00:41:39.750467+0000","last_fullsized":"2024-02-18T00:41:39.750467+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:27.921417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.921417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":583,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.820079+0000","last_change":"2024-02-18T00:41:21.035047+0000","last_active":"2024-02-18T00:41:39.820079+0000","last_peered":"2024-02-18T00:41:39.820079+0000","last_clean":"2024-02-18T00:41:39.820079+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:41:39.820079+0000","last_undegraded":"2024-02-18T00:41:39.820079+0000","last_fullsized":"2024-02-18T00:41:39.820079+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:41:21.034941+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.034941+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":534,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.749441+0000","last_change":"2024-02-18T00:41:22.988106+0000","last_active":"2024-02-18T00:41:39.749441+0000","last_peered":"2024-02-18T00:41:39.749441+0000","last_clean":"2024-02-18T00:41:39.749441+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:41:39.749441+0000","last_undegraded":"2024-02-18T00:41:39.749441+0000","last_fullsized":"2024-02-18T00:41:39.749441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:22.988039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:41:22.988039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":560,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.818475+0000","last_change":"2024-02-18T00:40:42.622606+0000","last_active":"2024-02-18T00:41:39.818475+0000","last_peered":"2024-02-18T00:41:39.818475+0000","last_clean":"2024-02-18T00:41:39.818475+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:41:39.818475+0000","last_undegraded":"2024-02-18T00:41:39.818475+0000","last_fullsized":"2024-02-18T00:41:39.818475+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:42.622532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.622532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":32,"reported_epoch":565,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.142964+0000","last_change":"2024-02-18T00:41:19.748009+0000","last_active":"2024-02-18T00:41:39.142964+0000","last_peered":"2024-02-18T00:41:39.142964+0000","last_clean":"2024-02-18T00:41:39.142964+0000","last_became_active":"2024-02-18T00:41:19.747594+0000","last_became_peered":"2024-02-18T00:41:19.747594+0000","last_unstale":"2024-02-18T00:41:39.142964+0000","last_undegraded":"2024-02-18T00:41:39.142964+0000","last_fullsized":"2024-02-18T00:41:39.142964+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:42.144406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.144406+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":474,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.752684+0000","last_change":"2024-02-18T00:41:32.876920+0000","last_active":"2024-02-18T00:41:39.752684+0000","last_peered":"2024-02-18T00:41:39.752684+0000","last_clean":"2024-02-18T00:41:39.752684+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:41:39.752684+0000","last_undegraded":"2024-02-18T00:41:39.752684+0000","last_fullsized":"2024-02-18T00:41:39.752684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.876858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.876858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":632,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.747221+0000","last_change":"2024-02-18T00:39:06.697405+0000","last_active":"2024-02-18T00:41:39.747221+0000","last_peered":"2024-02-18T00:41:39.747221+0000","last_clean":"2024-02-18T00:41:39.747221+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:41:39.747221+0000","last_undegraded":"2024-02-18T00:41:39.747221+0000","last_fullsized":"2024-02-18T00:41:39.747221+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'39","last_scrub_stamp":"2024-02-18T00:39:06.697317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:39:06.697317+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":558,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.744421+0000","last_change":"2024-02-18T00:41:17.135859+0000","last_active":"2024-02-18T00:41:39.744421+0000","last_peered":"2024-02-18T00:41:39.744421+0000","last_clean":"2024-02-18T00:41:39.744421+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:41:39.744421+0000","last_undegraded":"2024-02-18T00:41:39.744421+0000","last_fullsized":"2024-02-18T00:41:39.744421+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.135767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.135767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":536,"reported_epoch":566,"state":"active+clean","last_fresh":"2024-02-18T00:41:39.824175+0000","last_change":"2024-02-18T00:39:57.211234+0000","last_active":"2024-02-18T00:41:39.824175+0000","last_peered":"2024-02-18T00:41:39.824175+0000","last_clean":"2024-02-18T00:41:39.824175+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:41:39.824175+0000","last_undegraded":"2024-02-18T00:41:39.824175+0000","last_fullsized":"2024-02-18T00:41:39.824175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:57.211166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:39:57.211166+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":27,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":81,"acting":81,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1511,"num_read_kb":1236,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3196,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":923,"ondisk_log_size":923,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":736,"num_read_kb":731,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266841,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308192,"kb_used_data":2776,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436960,"statfs":{"total":95995035648,"available":95679447040,"internally_reserved":0,"allocated":2842624,"data_stored":1390641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.503,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.65800000000000003,"5min":1.1020000000000001,"15min":6.8700000000000001},"last":4.7389999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.503,"15min":0.59499999999999997},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.68100000000000005,"5min":1.0780000000000001,"15min":11.288},"last":0.42099999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52800000000000002,"15min":0.621},"min":{"1min":0.29799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67400000000000004,"5min":1.163,"15min":7.4729999999999999},"last":1.8240000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55700000000000005,"15min":0.64300000000000002},"min":{"1min":0.311,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68500000000000005,"5min":1.1930000000000001,"15min":7.7869999999999999},"last":0.377}]},{"osd":2,"last update":"Sun Feb 18 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.57399999999999995},"min":{"1min":0.26600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.753,"5min":1.0920000000000001,"15min":6.7380000000000004},"last":4.8220000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.51000000000000001,"15min":0.59599999999999997},"min":{"1min":0.33100000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.69299999999999995,"5min":1.0649999999999999,"15min":12.861000000000001},"last":1.913}]},{"osd":3,"last update":"Sun Feb 18 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54200000000000004,"15min":0.57299999999999995},"min":{"1min":0.33200000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69799999999999995,"5min":1.6719999999999999,"15min":7.1360000000000001},"last":4.8520000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53300000000000003,"15min":0.57899999999999996},"min":{"1min":0.374,"5min":0.31,"15min":0.31},"max":{"1min":0.71499999999999997,"5min":1.2929999999999999,"15min":3.7989999999999999},"last":1.948}]},{"osd":4,"last update":"Sun Feb 18 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.44,"15min":0.42699999999999999},"min":{"1min":0.28399999999999997,"5min":0.26900000000000002,"15min":0.215},"max":{"1min":0.58999999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.47099999999999997,"15min":0.44900000000000001},"min":{"1min":0.312,"5min":0.252,"15min":0.246},"max":{"1min":0.58699999999999997,"5min":1.032,"15min":1.4770000000000001},"last":4.7859999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.29999999999999999,"5min":0.27300000000000002,"15min":0.24099999999999999},"max":{"1min":0.622,"5min":0.70099999999999996,"15min":1.603},"last":1.877},{"interface":"front","average":{"1min":0.441,"5min":0.48999999999999999,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.27800000000000002,"15min":0.25600000000000001},"max":{"1min":0.68300000000000005,"5min":1.151,"15min":4.9130000000000003},"last":1.853}]},{"osd":6,"last update":"Sun Feb 18 00:41:28 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52100000000000002,"15min":0.50900000000000001},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":0.65600000000000003,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":1.9019999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.52400000000000002,"15min":0.51300000000000001},"min":{"1min":0.32200000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69399999999999995,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":5.3860000000000001}]}]},{"osd":6,"up_from":47,"seq":201863463071,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312288,"kb_used_data":2776,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432864,"statfs":{"total":95995035648,"available":95675252736,"internally_reserved":0,"allocated":2842624,"data_stored":1391965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55100000000000005,"15min":1.2569999999999999},"min":{"1min":0.29599999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.69699999999999995,"5min":4.0129999999999999,"15min":69.194999999999993},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54000000000000004,"15min":0.73499999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.71199999999999997,"5min":1.7170000000000001,"15min":10.433},"last":0.39300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54200000000000004,"15min":1.333},"min":{"1min":0.36099999999999999,"5min":0.29399999999999998,"15min":0.26000000000000001},"max":{"1min":0.77300000000000002,"5min":1.788,"15min":76.108999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52400000000000002,"15min":1.3200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":1.216,"15min":71.872},"last":0.436}]},{"osd":2,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52200000000000002,"15min":0.80600000000000005},"min":{"1min":0.318,"5min":0.31,"15min":0.31},"max":{"1min":0.63100000000000001,"5min":1.196,"15min":39.939999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.59199999999999997,"15min":0.98499999999999999},"min":{"1min":0.41199999999999998,"5min":0.38200000000000001,"15min":0.34200000000000003},"max":{"1min":0.67000000000000004,"5min":3.1600000000000001,"15min":39.673000000000002},"last":0.45200000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.62,"15min":0.83799999999999997},"min":{"1min":0.40400000000000003,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":0.63500000000000001,"5min":8.3420000000000005,"15min":21.219000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.54300000000000004,"15min":0.78100000000000003},"min":{"1min":0.38800000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68500000000000005,"5min":1.506,"15min":14.007999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.44600000000000001,"15min":0.50900000000000001},"min":{"1min":0.32000000000000001,"5min":0.28399999999999997,"15min":0.22800000000000001},"max":{"1min":0.67400000000000004,"5min":0.71099999999999997,"15min":8.1479999999999997},"last":0.499},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47899999999999998,"15min":0.69099999999999995},"min":{"1min":0.33800000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69999999999999996,"5min":0.85099999999999998,"15min":35.18},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48799999999999999,"15min":0.53800000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.27500000000000002},"max":{"1min":0.755,"5min":2.673,"15min":10.606},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49299999999999999,"15min":0.71599999999999997},"min":{"1min":0.37,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":35.119},"last":0.57999999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:41:29 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.55000000000000004},"min":{"1min":0.36599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":5.4770000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.72499999999999998},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.32000000000000001},"max":{"1min":0.69099999999999995,"5min":0.77800000000000002,"15min":32.985999999999997},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312296,"kb_used_data":2784,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432856,"statfs":{"total":95995035648,"available":95675244544,"internally_reserved":0,"allocated":2850816,"data_stored":1390968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.64400000000000002},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.193},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":32.106999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.45700000000000002,"15min":0.71099999999999997},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.21099999999999999},"max":{"1min":6.625,"5min":6.625,"15min":43.137},"last":0.30299999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36799999999999999,"15min":0.75800000000000001},"min":{"1min":0.246,"5min":0.22900000000000001,"15min":0.19900000000000001},"max":{"1min":0.45000000000000001,"5min":0.78300000000000003,"15min":49.201000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.40300000000000002,"15min":0.59199999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.217},"max":{"1min":0.503,"5min":0.83199999999999996,"15min":17.596},"last":0.35099999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:40:51 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.73699999999999999},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26200000000000001},"max":{"1min":0.76000000000000001,"5min":0.92800000000000005,"15min":37.524000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.42699999999999999,"15min":0.755},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.626,"5min":0.71699999999999997,"15min":32.036000000000001},"last":0.379}]},{"osd":4,"last update":"Sun Feb 18 00:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.76200000000000001},"min":{"1min":0.42299999999999999,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.75900000000000001,"5min":0.874,"15min":17.486999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.58399999999999996,"15min":0.77500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.9790000000000001,"15min":12.75},"last":0.54700000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.77200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34399999999999997},"max":{"1min":0.85199999999999998,"5min":0.85299999999999998,"15min":12.611000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.78300000000000003},"min":{"1min":0.437,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79000000000000004,"5min":0.90100000000000002,"15min":17.417000000000002},"last":0.46899999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:41:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60099999999999998,"15min":0.80500000000000005},"min":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.35899999999999999},"max":{"1min":0.82699999999999996,"5min":2.0430000000000001,"15min":12.834},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58699999999999997,"15min":0.78700000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.35499999999999998},"max":{"1min":0.72699999999999998,"5min":0.89000000000000001,"15min":12.651999999999999},"last":0.47499999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.79700000000000004},"min":{"1min":0.47299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":0.86899999999999999,"15min":17.452000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63400000000000001,"15min":0.80900000000000005},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":0.77800000000000002,"5min":2.0859999999999999,"15min":12.805},"last":0.504}]}]},{"osd":0,"up_from":8,"seq":34359738556,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308192,"kb_used_data":2776,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436960,"statfs":{"total":95995035648,"available":95679447040,"internally_reserved":0,"allocated":2842624,"data_stored":1390446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35299999999999998,"15min":1.036},"min":{"1min":0.23799999999999999,"5min":0.19500000000000001,"15min":0.18099999999999999},"max":{"1min":0.42999999999999999,"5min":1.2330000000000001,"15min":133.38999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47199999999999998,"15min":0.78700000000000003},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":1.8069999999999999,"5min":4.577,"15min":90.623000000000005},"last":0.57599999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:41:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.89900000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.20200000000000001},"max":{"1min":1.782,"5min":4.54,"15min":90.850999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.374,"15min":0.86599999999999999},"min":{"1min":0.20599999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.77600000000000002,"5min":1.482,"15min":80.537000000000006},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38300000000000001,"15min":0.65900000000000003},"min":{"1min":0.21299999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.503,"5min":1.1359999999999999,"15min":19.006},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.48599999999999999,"15min":1.0629999999999999},"min":{"1min":0.308,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.6910000000000001,"5min":4.4489999999999998,"15min":146.71899999999999},"last":0.51400000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56899999999999995,"15min":0.92300000000000004},"min":{"1min":0.36599999999999999,"5min":0.33600000000000002,"15min":0.26700000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":57.914000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64400000000000002,"15min":0.97999999999999998},"min":{"1min":0.40899999999999997,"5min":0.32800000000000001,"15min":0.30399999999999999},"max":{"1min":1.6619999999999999,"5min":4.2919999999999998,"15min":90.070999999999998},"last":0.88800000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:40:38 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65700000000000003,"15min":0.95299999999999996},"min":{"1min":0.39600000000000002,"5min":0.36499999999999999,"15min":0.32000000000000001},"max":{"1min":1.714,"5min":4.3760000000000003,"15min":90.049000000000007},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57099999999999995,"15min":0.97599999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.32200000000000001},"max":{"1min":0.70599999999999996,"5min":1.5640000000000001,"15min":57.991},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.79700000000000004},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.35999999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":19.039000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.65400000000000003,"15min":0.98099999999999998},"min":{"1min":0.41899999999999998,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":1.76,"5min":4.343,"15min":89.989999999999995},"last":0.77700000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:41:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.66900000000000004,"15min":1.0509999999999999},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.36499999999999999},"max":{"1min":1.732,"5min":4.4100000000000001,"15min":90.025000000000006},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.603,"15min":57.962000000000003},"last":0.89700000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411507,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308176,"kb_used_data":2760,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436976,"statfs":{"total":95995035648,"available":95679463424,"internally_reserved":0,"allocated":2826240,"data_stored":1390226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:31 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.34100000000000003,"15min":0.48399999999999999},"min":{"1min":0.24299999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.45600000000000002,"5min":0.78500000000000003,"15min":12.144},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.35299999999999998,"15min":0.60399999999999998},"min":{"1min":0.26800000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":0.63600000000000001,"15min":18.891999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Feb 18 00:41:31 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36099999999999999,"15min":0.49199999999999999},"min":{"1min":0.27700000000000002,"5min":0.23300000000000001,"15min":0.19700000000000001},"max":{"1min":0.5,"5min":0.76800000000000002,"15min":10.138999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.36899999999999999,"15min":0.53800000000000003},"min":{"1min":0.23999999999999999,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":0.53000000000000003,"5min":0.64300000000000002,"15min":18.919},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39400000000000002,"15min":0.59599999999999997},"min":{"1min":0.33200000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.53100000000000003,"5min":0.67400000000000004,"15min":22.712},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39700000000000002,"15min":0.60099999999999998},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.51600000000000001,"5min":0.57099999999999995,"15min":22.359000000000002},"last":0.61699999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:41:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.68300000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.32500000000000001},"max":{"1min":0.872,"5min":0.94899999999999995,"15min":18.812000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56100000000000005,"15min":0.66900000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.28699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":10.016},"last":0.69399999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:40:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.65000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":6.766},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.59099999999999997,"15min":0.68799999999999994},"min":{"1min":0.42899999999999999,"5min":0.36699999999999999,"15min":0.317},"max":{"1min":0.84199999999999997,"5min":0.95999999999999996,"15min":10.114000000000001},"last":0.628}]},{"osd":6,"last update":"Sun Feb 18 00:41:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.66600000000000004},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.35799999999999998},"max":{"1min":0.83299999999999996,"5min":0.88,"15min":6.806},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.71599999999999997},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.34100000000000003},"max":{"1min":0.90500000000000003,"5min":0.92100000000000004,"15min":18.861000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:41:31 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.69699999999999995},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":6.8300000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.65100000000000002,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.45200000000000001,"15min":0.372},"max":{"1min":0.83099999999999996,"5min":1.149,"15min":10.058999999999999},"last":0.79600000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149869,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308216,"kb_used_data":2800,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93436936,"statfs":{"total":95995035648,"available":95679422464,"internally_reserved":0,"allocated":2867200,"data_stored":1391562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.374,"15min":0.46999999999999997},"min":{"1min":0.219,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":0.496,"5min":0.92200000000000004,"15min":25.082999999999998},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.51900000000000002},"min":{"1min":0.25800000000000001,"5min":0.23699999999999999,"15min":0.20599999999999999},"max":{"1min":0.66600000000000004,"5min":1.2110000000000001,"15min":18.911000000000001},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38600000000000001,"15min":0.46800000000000003},"min":{"1min":0.28999999999999998,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":6.8479999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.47599999999999998,"15min":0.69099999999999995},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.20599999999999999},"max":{"1min":0.65300000000000002,"5min":8.0779999999999994,"15min":23.606999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:40:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45400000000000001,"15min":0.65300000000000002},"min":{"1min":0.25700000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.66600000000000004,"5min":4.0190000000000001,"15min":23.405000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.42299999999999999,"15min":0.46500000000000002},"min":{"1min":0.28899999999999998,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.53800000000000003,"5min":1.4430000000000001,"15min":5.3479999999999999},"last":0.30299999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:41:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.61199999999999999,"15min":0.68899999999999995},"min":{"1min":0.46800000000000003,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.79600000000000004,"5min":3.8410000000000002,"15min":10.131},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.63100000000000001},"min":{"1min":0.47299999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.90900000000000003,"5min":1.1699999999999999,"15min":5.4630000000000001},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:40:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.61499999999999999,"15min":0.63700000000000001},"min":{"1min":0.45900000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":4.7469999999999999,"5min":4.7469999999999999,"15min":7.4249999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.67700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.32400000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":6.7750000000000004},"last":0.80500000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:41:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63,"15min":0.747},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":3.8940000000000001,"15min":10.189},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.65000000000000002,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.84699999999999998,"5min":2.484,"15min":6.4279999999999999},"last":0.753}]},{"osd":7,"last update":"Sun Feb 18 00:41:32 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.47799999999999998,"5min":0.41899999999999998,"15min":0.35199999999999998},"max":{"1min":0.83299999999999996,"5min":5.0830000000000002,"15min":6.8220000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66100000000000003,"15min":0.72999999999999998},"min":{"1min":0.436,"5min":0.40799999999999997,"15min":0.39800000000000002},"max":{"1min":1.0369999999999999,"5min":3.9289999999999998,"15min":10.164999999999999},"last":0.67000000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986344,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":300004,"kb_used_data":2780,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93445148,"statfs":{"total":95995035648,"available":95687831552,"internally_reserved":0,"allocated":2846720,"data_stored":1391363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.504,"15min":0.88100000000000001},"min":{"1min":0.249,"5min":0.247,"15min":0.247},"max":{"1min":0.71599999999999997,"5min":0.92600000000000005,"15min":77.581999999999994},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48799999999999999,"15min":0.84799999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59399999999999997,"5min":0.79300000000000004,"15min":78.063999999999993},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Feb 18 00:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.499,"15min":0.624},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69799999999999995,"5min":0.89800000000000002,"15min":9.3160000000000007},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.503,"15min":0.88400000000000001},"min":{"1min":0.32500000000000001,"5min":0.29699999999999999,"15min":0.27100000000000002},"max":{"1min":0.68300000000000005,"5min":0.88,"15min":81.242000000000004},"last":0.51000000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52400000000000002,"15min":0.63500000000000001},"min":{"1min":0.29299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72299999999999998,"5min":1.022,"15min":11.691000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.51300000000000001,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.27200000000000002},"max":{"1min":0.66900000000000004,"5min":0.80200000000000005,"15min":17.77},"last":0.60999999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50900000000000001,"15min":0.59899999999999998},"min":{"1min":0.33700000000000002,"5min":0.318,"15min":0.28299999999999997},"max":{"1min":0.63700000000000001,"5min":0.82199999999999995,"15min":7.218},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56000000000000005,"15min":0.72799999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.309},"max":{"1min":0.72999999999999998,"5min":1.2350000000000001,"15min":33.530999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:41:39 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45400000000000001,"15min":0.42299999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.20300000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.72299999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.433},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.77900000000000003},"last":0.55200000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.443,"15min":0.435},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.623,"5min":0.67700000000000005,"15min":0.76100000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.47699999999999998},"min":{"1min":0.34499999999999997,"5min":0.28299999999999997,"15min":0.26000000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:41:27 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.35399999999999998,"5min":0.34399999999999997,"15min":0.27900000000000003},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.76900000000000002},"last":0.624},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.50900000000000001,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.70099999999999996,"5min":0.73499999999999999,"15min":0.83299999999999996},"last":0.57899999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724707,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312292,"kb_used_data":2780,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93432860,"statfs":{"total":95995035648,"available":95675248640,"internally_reserved":0,"allocated":2846720,"data_stored":1391624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.28699999999999998,"5min":0.24399999999999999,"15min":0.217},"max":{"1min":0.73599999999999999,"5min":2.157,"15min":6.3150000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.54600000000000004,"15min":0.61299999999999999},"min":{"1min":0.32800000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.623,"5min":4.4779999999999998,"15min":12.744},"last":0.55800000000000005}]},{"osd":1,"last update":"Sun Feb 18 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":0.77000000000000002,"5min":13.25,"15min":13.25},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.29699999999999999},"max":{"1min":0.76400000000000001,"5min":13.262,"15min":13.262},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Feb 18 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51600000000000001,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.27700000000000002,"15min":0.251},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":6.1680000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60599999999999998,"15min":0.64000000000000001},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.308},"max":{"1min":0.748,"5min":5.0659999999999998,"15min":6.1879999999999997},"last":0.68500000000000005}]},{"osd":3,"last update":"Sun Feb 18 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.56699999999999995,"15min":0.623},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.28399999999999997},"max":{"1min":0.78000000000000003,"5min":7.641,"15min":7.641},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":0.60599999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65900000000000003,"5min":7.4580000000000002,"15min":8.8650000000000002},"last":0.61699999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:41:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.439,"15min":0.47099999999999997},"min":{"1min":0.255,"5min":0.25,"15min":0.23599999999999999},"max":{"1min":0.66800000000000004,"5min":0.92600000000000005,"15min":11.818},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.44,"5min":0.46999999999999997,"15min":0.45700000000000002},"min":{"1min":0.27100000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.60299999999999998,"5min":0.84899999999999998,"15min":4.274},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:41:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.57799999999999996,"15min":0.54000000000000004},"min":{"1min":0.27700000000000002,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.624,"5min":12.362,"15min":18.605},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66600000000000004,"5min":1.46,"15min":4.2089999999999996},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.495,"15min":0.48499999999999999},"min":{"1min":0.29799999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69299999999999995,"5min":1.5269999999999999,"15min":2.0739999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50800000000000001,"15min":0.51600000000000001},"min":{"1min":0.28899999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65400000000000003,"5min":0.76000000000000001,"15min":4.649},"last":0.745}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:41:43.963 INFO:tasks.thrashosds.thrasher:Setting 2.6 to [6, 7, 4, 1, 3, 5] 2024-02-18T00:41:43.963 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.6', '6', '7', '4', '1', '3', '5'] 2024-02-18T00:41:43.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pg-upmap-items 2.6 6 7 4 1 3 5 2024-02-18T00:41:44.033 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:44.036 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:44.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:44.138 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:44.168 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:42.490771+0000 mgr.y (mgr.24454) 522 : audit [DBG] from='client.41438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:41:44.168 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:42.766832+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:41:44.168 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:42.766928+0000 mon.a (mon.0) 2968 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-18T00:41:44.168 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:42.766986+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-18T00:41:44.170 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:44.193 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:44.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:44.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:44.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:42.490771+0000 mgr.y (mgr.24454) 522 : audit [DBG] from='client.41438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:41:44.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:42.766832+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:41:44.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:42.766928+0000 mon.a (mon.0) 2968 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-18T00:41:44.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:42.766986+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-18T00:41:44.225 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:42.490771+0000 mgr.y (mgr.24454) 522 : audit [DBG] from='client.41438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:41:44.225 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:42.766832+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi059-82957-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:41:44.225 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:42.766928+0000 mon.a (mon.0) 2968 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "26"}]': finished 2024-02-18T00:41:44.226 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:42.766986+0000 mon.a (mon.0) 2969 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-18T00:41:44.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:44.324 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:44.380 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:44.388 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:44.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:44.490 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:44.521 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:44.543 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:44.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:44.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:44.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:44.675 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:43.770909+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-18T00:41:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:43.958548+0000 mgr.y (mgr.24454) 523 : cluster [DBG] pgmap v870: 177 pgs: 8 unknown, 1 peering, 168 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-18T00:41:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:44.768802+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-18T00:41:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:44.777375+0000 mon.c (mon.2) 540 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:44.778225+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:45.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:43.770909+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:43.958548+0000 mgr.y (mgr.24454) 523 : cluster [DBG] pgmap v870: 177 pgs: 8 unknown, 1 peering, 168 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:44.768802+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:44.777375+0000 mon.c (mon.2) 540 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:44.778225+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:43.770909+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:43.958548+0000 mgr.y (mgr.24454) 523 : cluster [DBG] pgmap v870: 177 pgs: 8 unknown, 1 peering, 168 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s; 92 B/s, 0 objects/s recovering 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:44.768802+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-18T00:41:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:44.777375+0000 mon.c (mon.2) 540 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:44.778225+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:45.455 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:45.457 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:45.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:45.558 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:45.590 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:45.614 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:45.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:45.716 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:45.745 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:45.764 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:45.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:45.767 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:45.868 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:45.898 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:46.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 -- 172.21.15.59:0/3166152575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 msgr2=0x7ff3300fc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 --2- 172.21.15.59:0/3166152575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 0x7ff3300fc620 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 rx=0x7ff3200047b0 tx=0x7ff320009cb0).stop 2024-02-18T00:41:46.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 -- 172.21.15.59:0/3166152575 shutdown_connections 2024-02-18T00:41:46.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 --2- 172.21.15.59:0/3166152575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3300f6b10 0x7ff3301030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 --2- 172.21.15.59:0/3166152575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 0x7ff3300fc620 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 --2- 172.21.15.59:0/3166152575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3300fcba0 0x7ff3300f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 -- 172.21.15.59:0/3166152575 >> 172.21.15.59:0/3166152575 conn(0x7ff33006c370 msgr2=0x7ff33006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:46.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 -- 172.21.15.59:0/3166152575 shutdown_connections 2024-02-18T00:41:46.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.425+0000 7ff336389700 1 -- 172.21.15.59:0/3166152575 wait complete. 2024-02-18T00:41:46.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 librados: starting msgr at 2024-02-18T00:41:46.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 librados: starting objecter 2024-02-18T00:41:46.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 Processor -- start 2024-02-18T00:41:46.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 -- start start 2024-02-18T00:41:46.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 librados: setting wanted keys 2024-02-18T00:41:46.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 librados: calling monclient init 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3300f6b10 0x7ff330110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 0x7ff330110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3300fcba0 0x7ff33010ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010b340 con 0x7ff3300fcba0 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010b4b0 con 0x7ff3300f6b10 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff336389700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33010b670 con 0x7ff3300fc240 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff32f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 0x7ff330110930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff334926700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3300fcba0 0x7ff33010ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff32f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 0x7ff330110930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:33186/0 (socket says 172.21.15.59:33186) 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff32f7fe700 1 -- 172.21.15.59:0/3235541742 learned_addr learned my addr 172.21.15.59:0/3235541742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff334926700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3300fcba0 0x7ff33010ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46006/0 (socket says 172.21.15.59:46006) 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.426+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3300f6b10 0x7ff330110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32f7fe700 1 -- 172.21.15.59:0/3235541742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3300f6b10 msgr2=0x7ff330110440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32f7fe700 1 --2- 172.21.15.59:0/3235541742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3300f6b10 0x7ff330110440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32f7fe700 1 -- 172.21.15.59:0/3235541742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3300fcba0 msgr2=0x7ff33010ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32f7fe700 1 --2- 172.21.15.59:0/3235541742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3300fcba0 0x7ff33010ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32f7fe700 1 -- 172.21.15.59:0/3235541742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff324005040 con 0x7ff3300fc240 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff334926700 1 --2- 172.21.15.59:0/3235541742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3300fcba0 0x7ff33010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32f7fe700 1 --2- 172.21.15.59:0/3235541742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 0x7ff330110930 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 rx=0x7ff324009400 tx=0x7ff324009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32d7fa700 10 client.?.objecter ms_handle_connect 0x7ff3300fc240 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32d7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324007a40 con 0x7ff3300fc240 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff32400f040 con 0x7ff3300fc240 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320005040 con 0x7ff3300fc240 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324014a70 con 0x7ff3300fc240 2024-02-18T00:41:46.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff336389700 10 client.41450.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:46.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3301171e0 con 0x7ff3300fc240 2024-02-18T00:41:46.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.427+0000 7ff336389700 1 librados: init done 2024-02-18T00:41:46.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.428+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7ff33005fa60 con 0x7ff3300fc240 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff324010530 con 0x7ff3300fc240 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff318062c90 0x7ff318065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(572..572 src has 418..572) v4 ==== 8578+0+0 (secure 0 0 0) 0x7ff32407d510 con 0x7ff3300fc240 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 osd_map(572..572 src has 418..572) v4 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 3 client.41450.objecter handle_osd_map got epochs [572,572] > 0 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 3 client.41450.objecter handle_osd_map decoding full epoch 572 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff318062c90 0x7ff318065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 20 client.41450.objecter dump_active .. 0 homeless 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.429+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v572) v1 ==== 54+0+16 (secure 0 0 0) 0x7ff32407d930 con 0x7ff3300fc240 2024-02-18T00:41:46.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 10 client.41450.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 1 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff318062c90 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff3300fd530 0x7ff3300fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff318062c90 0x7ff318065150 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 rx=0x7ff320005010 tx=0x7ff320009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 1 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 0 1 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33005ee80 osd=0 4 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 10 client.41450.objecter _send_command 1 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff330104e60 con 0x7ff3300fd530 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff334926700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff3300fd530 0x7ff3300fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff3300fd530 2024-02-18T00:41:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.430+0000 7ff334926700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff3300fd530 0x7ff3300fd910 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.431+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff330104e60 con 0x7ff3300fd530 2024-02-18T00:41:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.431+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:41:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.431+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 1 = system:0 2024-02-18T00:41:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.431+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff33005ee80 osd=0 4 2024-02-18T00:41:46.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.431+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 0 1 2024-02-18T00:41:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:46.449 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:46.449 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:46.451 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:44.779936+0000 osd.1 (osd.1) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:46.451 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:44.780246+0000 osd.1 (osd.1) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:46.451 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:45.770308+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:46.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:45.770439+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-18T00:41:46.452 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:46.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:46.453 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:44.779936+0000 osd.1 (osd.1) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:46.453 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:44.780246+0000 osd.1 (osd.1) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:46.453 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:45.770308+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:46.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:45.770439+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-18T00:41:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 10 client.41450.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 2 2024-02-18T00:41:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 2 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff33005ee80 osd=0 3 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 0 2 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33005ee80 osd=0 4 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 10 client.41450.objecter _send_command 2 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.449+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff3300fe040 con 0x7ff3300fd530 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff32800f040 con 0x7ff3300fd530 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff33005ee80 osd=0 4 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 0 2 2024-02-18T00:41:46.455 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:41:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 10 client.41450.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 3 2024-02-18T00:41:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6805/26279110 conn(0x7ff3300598f0 0x7ff330059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 3 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 1 3 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330028540 osd=1 4 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 10 client.41450.objecter _send_command 3 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.451+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3300286b0 con 0x7ff3300598f0 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.452+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6805/26279110 conn(0x7ff3300598f0 0x7ff330059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.452+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff3300598f0 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.452+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6805/26279110 conn(0x7ff3300598f0 0x7ff330059cd0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.453+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff3300286b0 con 0x7ff3300598f0 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.453+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.453+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 3 = system:0 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.453+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330028540 osd=1 4 2024-02-18T00:41:46.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.453+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 1 3 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 10 client.41450.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 4 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 4 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330028540 osd=1 3 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 1 4 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330028540 osd=1 4 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 10 client.41450.objecter _send_command 4 2024-02-18T00:41:46.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.481+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff330059360 con 0x7ff3300598f0 2024-02-18T00:41:46.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff330059360 con 0x7ff3300598f0 2024-02-18T00:41:46.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330028540 osd=1 4 2024-02-18T00:41:46.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 1 4 2024-02-18T00:41:46.485 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 10 client.41450.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 5 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff3300280a0 0x7ff33011e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 5 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 2 5 2024-02-18T00:41:46.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330059770 osd=2 4 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 10 client.41450.objecter _send_command 5 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff330121170 con 0x7ff3300280a0 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32f7fe700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff3300280a0 0x7ff33011e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff3300280a0 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.483+0000 7ff32f7fe700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff3300280a0 0x7ff33011e520 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.484+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff330121170 con 0x7ff3300280a0 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.484+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.484+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 5 = system:0 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.484+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330059770 osd=2 4 2024-02-18T00:41:46.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.484+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 2 5 2024-02-18T00:41:46.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 10 client.41450.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 6 2024-02-18T00:41:46.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 6 2024-02-18T00:41:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330059770 osd=2 3 2024-02-18T00:41:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 2 6 2024-02-18T00:41:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330059770 osd=2 4 2024-02-18T00:41:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 10 client.41450.objecter _send_command 6 2024-02-18T00:41:46.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.498+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff330027600 con 0x7ff3300280a0 2024-02-18T00:41:46.502 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff330027600 con 0x7ff3300280a0 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330059770 osd=2 4 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 2 6 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 10 client.41450.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 7 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff33011c070 0x7ff33011fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 7 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 3 7 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330027a10 osd=3 4 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 10 client.41450.objecter _send_command 7 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3301200f0 con 0x7ff33011c070 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff334926700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff33011c070 0x7ff33011fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff33011c070 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.500+0000 7ff334926700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff33011c070 0x7ff33011fb90 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.501+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff3301200f0 con 0x7ff33011c070 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.501+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.501+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 7 = system:0 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.501+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330027a10 osd=3 4 2024-02-18T00:41:46.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.501+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 3 7 2024-02-18T00:41:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:44.779936+0000 osd.1 (osd.1) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:41:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:44.780246+0000 osd.1 (osd.1) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:41:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:45.770308+0000 mon.a (mon.0) 2973 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:46.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:45.770439+0000 mon.a (mon.0) 2974 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-18T00:41:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 10 client.41450.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 8 2024-02-18T00:41:46.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 8 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330027a10 osd=3 3 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 3 8 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330027a10 osd=3 4 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 10 client.41450.objecter _send_command 8 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.516+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff33011b5d0 con 0x7ff33011c070 2024-02-18T00:41:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff33011b5d0 con 0x7ff33011c070 2024-02-18T00:41:46.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330027a10 osd=3 4 2024-02-18T00:41:46.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 3 8 2024-02-18T00:41:46.519 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 10 client.41450.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 9 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff3301232f0 0x7ff3301256e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 9 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 4 9 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b390 osd=4 4 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 10 client.41450.objecter _send_command 9 2024-02-18T00:41:46.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.517+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff33011b500 con 0x7ff3301232f0 2024-02-18T00:41:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.518+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff3301232f0 0x7ff3301256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.518+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff3301232f0 2024-02-18T00:41:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.518+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff3301232f0 0x7ff3301256e0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.519+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff33011b500 con 0x7ff3301232f0 2024-02-18T00:41:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.519+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:41:46.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.519+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 9 = system:0 2024-02-18T00:41:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.519+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff33011b390 osd=4 4 2024-02-18T00:41:46.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.519+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 4 9 2024-02-18T00:41:46.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 10 client.41450.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 10 2024-02-18T00:41:46.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 10 2024-02-18T00:41:46.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff33011b390 osd=4 3 2024-02-18T00:41:46.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 4 10 2024-02-18T00:41:46.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b390 osd=4 4 2024-02-18T00:41:46.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 10 client.41450.objecter _send_command 10 2024-02-18T00:41:46.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.533+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff33011b500 con 0x7ff3301232f0 2024-02-18T00:41:46.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff33011b500 con 0x7ff3301232f0 2024-02-18T00:41:46.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff33011b390 osd=4 4 2024-02-18T00:41:46.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 4 10 2024-02-18T00:41:46.539 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:41:46.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 10 client.41450.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 11 2024-02-18T00:41:46.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff3301260e0 0x7ff330126520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 11 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 5 11 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b500 osd=5 4 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 10 client.41450.objecter _send_command 11 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3301280e0 con 0x7ff3301260e0 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.536+0000 7ff32f7fe700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff3301260e0 0x7ff330126520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.537+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff3301260e0 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.537+0000 7ff32f7fe700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff3301260e0 0x7ff330126520 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.537+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff3301280e0 con 0x7ff3301260e0 2024-02-18T00:41:46.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.537+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:41:46.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.537+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 11 = system:0 2024-02-18T00:41:46.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.537+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff33011b500 osd=5 4 2024-02-18T00:41:46.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.537+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 5 11 2024-02-18T00:41:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 10 client.41450.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 12 2024-02-18T00:41:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 12 2024-02-18T00:41:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff33011b500 osd=5 3 2024-02-18T00:41:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 5 12 2024-02-18T00:41:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b500 osd=5 4 2024-02-18T00:41:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 10 client.41450.objecter _send_command 12 2024-02-18T00:41:46.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.552+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff3301280e0 con 0x7ff3301260e0 2024-02-18T00:41:46.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff3301280e0 con 0x7ff3301260e0 2024-02-18T00:41:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff33011b500 osd=5 4 2024-02-18T00:41:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 5 12 2024-02-18T00:41:46.556 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:41:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 10 client.41450.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 13 2024-02-18T00:41:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff330126fe0 0x7ff330127440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 13 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 6 13 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff3301280e0 osd=6 4 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 10 client.41450.objecter _send_command 13 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff330127ab0 con 0x7ff330126fe0 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff334926700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff330126fe0 0x7ff330127440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff330126fe0 2024-02-18T00:41:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.554+0000 7ff334926700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff330126fe0 0x7ff330127440 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.555+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff330127ab0 con 0x7ff330126fe0 2024-02-18T00:41:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.555+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:41:46.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.555+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 13 = system:0 2024-02-18T00:41:46.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.555+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff3301280e0 osd=6 4 2024-02-18T00:41:46.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.555+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 6 13 2024-02-18T00:41:46.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 10 client.41450.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 14 2024-02-18T00:41:46.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 14 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff3301280e0 osd=6 3 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 6 14 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff3301280e0 osd=6 4 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 10 client.41450.objecter _send_command 14 2024-02-18T00:41:46.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.570+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff330127ab0 con 0x7ff330126fe0 2024-02-18T00:41:46.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff330127ab0 con 0x7ff330126fe0 2024-02-18T00:41:46.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff3301280e0 osd=6 4 2024-02-18T00:41:46.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 6 14 2024-02-18T00:41:46.574 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 10 client.41450.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 15 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6812/53025171 conn(0x7ff330127cd0 0x7ff33012e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 15 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 7 15 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330127ab0 osd=7 4 2024-02-18T00:41:46.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 10 client.41450.objecter _send_command 15 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.572+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff330131700 con 0x7ff330127cd0 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.573+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6812/53025171 conn(0x7ff330127cd0 0x7ff33012e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.573+0000 7ff32d7fa700 10 client.41450.objecter ms_handle_connect 0x7ff330127cd0 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.573+0000 7ff32ffff700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6812/53025171 conn(0x7ff330127cd0 0x7ff33012e900 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.574+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff330131700 con 0x7ff330127cd0 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.574+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.574+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 15 = system:0 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.574+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330127ab0 osd=7 4 2024-02-18T00:41:46.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.574+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 7 15 2024-02-18T00:41:46.582 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:46.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 10 client.41450.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign -1 16 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 20 client.41450.objecter _get_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 15 client.41450.objecter _session_command_op_remove -1 16 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 20 client.41450.objecter get_session s=0x7ff330127ab0 osd=7 3 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 15 client.41450.objecter _session_command_op_assign 7 16 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330127ab0 osd=7 4 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 10 client.41450.objecter _send_command 16 2024-02-18T00:41:46.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.589+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7ff330131700 con 0x7ff330127cd0 2024-02-18T00:41:46.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff32d7fa700 1 -- 172.21.15.59:0/3235541742 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7ff330131700 con 0x7ff330127cd0 2024-02-18T00:41:46.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff32d7fa700 10 client.41450.objecter ms_dispatch 0x7ff330001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:41:46.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff32d7fa700 10 client.41450.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff32d7fa700 20 client.41450.objecter put_session s=0x7ff330127ab0 osd=7 4 2024-02-18T00:41:46.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff32d7fa700 15 client.41450.objecter _session_command_op_remove 7 16 2024-02-18T00:41:46.594 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:41:46.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 10 librados: watch_flush enter 2024-02-18T00:41:46.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 10 librados: watch_flush exit 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 10 client.41450.objecter close_session for osd.0 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff3300fd530 msgr2=0x7ff3300fd910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6801/3700238977 conn(0x7ff3300fd530 0x7ff3300fd910 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33005ee80 osd=0 1 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 10 client.41450.objecter close_session for osd.1 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6805/26279110 conn(0x7ff3300598f0 msgr2=0x7ff330059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6805/26279110 conn(0x7ff3300598f0 0x7ff330059cd0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330028540 osd=1 1 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 10 client.41450.objecter close_session for osd.2 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff3300280a0 msgr2=0x7ff33011e520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6809/1153193637 conn(0x7ff3300280a0 0x7ff33011e520 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.591+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330059770 osd=2 1 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 10 client.41450.objecter close_session for osd.3 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff33011c070 msgr2=0x7ff33011fb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6813/1805700964 conn(0x7ff33011c070 0x7ff33011fb90 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330027a10 osd=3 1 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 10 client.41450.objecter close_session for osd.4 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff3301232f0 msgr2=0x7ff3301256e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6800/2044387056 conn(0x7ff3301232f0 0x7ff3301256e0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff334926700 1 -- 172.21.15.59:0/3235541742 reap_dead start 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff334926700 1 -- 172.21.15.59:0/3235541742 reap_dead start 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b390 osd=4 1 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 10 client.41450.objecter close_session for osd.5 2024-02-18T00:41:46.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff3301260e0 msgr2=0x7ff330126520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6804/2170432263 conn(0x7ff3301260e0 0x7ff330126520 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff33011b500 osd=5 1 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 10 client.41450.objecter close_session for osd.6 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff330126fe0 msgr2=0x7ff330127440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6808/3356009390 conn(0x7ff330126fe0 0x7ff330127440 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff3301280e0 osd=6 1 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 10 client.41450.objecter close_session for osd.7 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6812/53025171 conn(0x7ff330127cd0 msgr2=0x7ff33012e900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.137:6812/53025171 conn(0x7ff330127cd0 0x7ff33012e900 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 20 client.41450.objecter put_session s=0x7ff330127ab0 osd=7 1 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 20 client.41450.objecter shutdown clearing up homeless session... 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 10 client.41450.objecter successfully canceled tick 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff318062c90 msgr2=0x7ff318065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff318062c90 0x7ff318065150 secure :-1 s=READY pgs=3162 cs=0 l=1 rev1=1 rx=0x7ff320005010 tx=0x7ff320009cb0).stop 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 msgr2=0x7ff330110930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 --2- 172.21.15.59:0/3235541742 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff3300fc240 0x7ff330110930 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 rx=0x7ff324009400 tx=0x7ff324009cb0).stop 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff334926700 1 -- 172.21.15.59:0/3235541742 reap_dead start 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 shutdown_connections 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 >> 172.21.15.59:0/3235541742 conn(0x7ff33006c370 msgr2=0x7ff3300f43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.592+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 shutdown_connections 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.593+0000 7ff336389700 1 -- 172.21.15.59:0/3235541742 wait complete. 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:46.593+0000 7ff336389700 1 librados: shutdown 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.598 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.599 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.599 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.599 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.599 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.599 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.599 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:41:46.913 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:46.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:46.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:47.018 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:47.050 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:47.071 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:47.074 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:47.075 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:47.164 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:47.164 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:47.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:47.206 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:45.959397+0000 mgr.y (mgr.24454) 524 : cluster [DBG] pgmap v873: 208 pgs: 4 creating+activating, 1 activating, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:46.240183+0000 mon.a (mon.0) 2975 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:46.429713+0000 mon.c (mon.2) 541 : audit [DBG] from='client.? 172.21.15.59:0/3235541742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:46.766620+0000 mon.a (mon.0) 2976 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:46.775198+0000 mon.a (mon.0) 2977 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:46.775268+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:46.786854+0000 mon.c (mon.2) 542 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:46.787167+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:45.959397+0000 mgr.y (mgr.24454) 524 : cluster [DBG] pgmap v873: 208 pgs: 4 creating+activating, 1 activating, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:46.240183+0000 mon.a (mon.0) 2975 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-18T00:41:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:46.429713+0000 mon.c (mon.2) 541 : audit [DBG] from='client.? 172.21.15.59:0/3235541742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:46.766620+0000 mon.a (mon.0) 2976 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:41:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:46.775198+0000 mon.a (mon.0) 2977 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-18T00:41:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:46.775268+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-18T00:41:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:46.786854+0000 mon.c (mon.2) 542 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:46.787167+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:45.959397+0000 mgr.y (mgr.24454) 524 : cluster [DBG] pgmap v873: 208 pgs: 4 creating+activating, 1 activating, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:46.240183+0000 mon.a (mon.0) 2975 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]: dispatch 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:46.429713+0000 mon.c (mon.2) 541 : audit [DBG] from='client.? 172.21.15.59:0/3235541742' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:46.766620+0000 mon.a (mon.0) 2976 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:46.775198+0000 mon.a (mon.0) 2977 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "25"}]': finished 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:46.775268+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:46.786854+0000 mon.c (mon.2) 542 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:46.787167+0000 mon.a (mon.0) 2979 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:47.960 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:47.964 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:47.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:48.065 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:48.097 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:48.118 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:48.121 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:48.121 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:48.222 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:48.254 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:48.275 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:48.277 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:48.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:48.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:48.415 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:48.567 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:48.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:48.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:48.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-18T00:41:48.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:48.703 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:48.724 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:48.726 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:48.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:48.828 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:48.857 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:48.877 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:48.879 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:48.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:48.981 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:49.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:47.778101+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:47.778210+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:47.783871+0000 mon.c (mon.2) 543 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:47.784832+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:47.959898+0000 mgr.y (mgr.24454) 525 : cluster [DBG] pgmap v876: 240 pgs: 32 unknown, 4 creating+activating, 1 activating, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:48.747290+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:48.747378+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:48.750439+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:48.750778+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.009 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:49.029 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:49.031 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:49.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:49.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:47.778101+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:49.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:47.778210+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-18T00:41:49.051 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:47.783871+0000 mon.c (mon.2) 543 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:47.784832+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:47.959898+0000 mgr.y (mgr.24454) 525 : cluster [DBG] pgmap v876: 240 pgs: 32 unknown, 4 creating+activating, 1 activating, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:49.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:48.747290+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:49.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:48.747378+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-18T00:41:49.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:48.750439+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:48.750778+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:47.778101+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:47.778210+0000 mon.a (mon.0) 2981 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:47.783871+0000 mon.c (mon.2) 543 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:47.784832+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:47.959898+0000 mgr.y (mgr.24454) 525 : cluster [DBG] pgmap v876: 240 pgs: 32 unknown, 4 creating+activating, 1 activating, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:48.747290+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:48.747378+0000 mon.a (mon.0) 2984 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:48.750439+0000 mon.c (mon.2) 544 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.053 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:48.750778+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:49.132 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:49.158 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:49.177 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:49.180 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:49.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:49.281 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:49.308 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:49.326 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:49.329 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:49.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:49.430 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:49.457 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:49.475 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:49.478 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:49.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:49.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:49.610 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:49.648 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:49.651 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:49.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:49.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:49.758 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:41:49.780 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:50.630 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:50.636 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:50.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:50.737 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:50.771 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:50.790 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:50.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:50.793 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:50.894 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:50.920 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:49.746246+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:49.746324+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-18T00:41:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:49.748199+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:41:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:49.748947+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:41:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:49.960304+0000 mgr.y (mgr.24454) 526 : cluster [DBG] pgmap v879: 239 pgs: 32 unknown, 4 creating+activating, 1 activating, 202 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:49.961889+0000 mon.a (mon.0) 2989 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-18T00:41:51.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:50.083820+0000 mon.a (mon.0) 2990 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:41:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:49.746246+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:49.746324+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-18T00:41:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:49.748199+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:41:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:49.748947+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:41:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:49.960304+0000 mgr.y (mgr.24454) 526 : cluster [DBG] pgmap v879: 239 pgs: 32 unknown, 4 creating+activating, 1 activating, 202 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:49.961889+0000 mon.a (mon.0) 2989 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-18T00:41:51.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:50.083820+0000 mon.a (mon.0) 2990 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:41:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:49.746246+0000 mon.a (mon.0) 2986 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-97", "overlaypool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:49.746324+0000 mon.a (mon.0) 2987 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-18T00:41:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:49.748199+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:41:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:49.748947+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:41:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:49.960304+0000 mgr.y (mgr.24454) 526 : cluster [DBG] pgmap v879: 239 pgs: 32 unknown, 4 creating+activating, 1 activating, 202 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:41:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:49.961889+0000 mon.a (mon.0) 2989 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]: dispatch 2024-02-18T00:41:51.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:50.083820+0000 mon.a (mon.0) 2990 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:41:51.117 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:51.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:51.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:51.222 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:51.251 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:51.273 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:51.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:51.374 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:51.405 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:51.406 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:41:51.665 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:41:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:51.760 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:51.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:51.859 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:51.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 -- 172.21.15.59:0/2291117623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300f7400 msgr2=0x7f2830103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:51.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 --2- 172.21.15.59:0/2291117623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300f7400 0x7f2830103a20 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 rx=0x7f2828009c30 tx=0x7f2828004c50).stop 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 -- 172.21.15.59:0/2291117623 shutdown_connections 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 --2- 172.21.15.59:0/2291117623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300f7400 0x7f2830103a20 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 --2- 172.21.15.59:0/2291117623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28300fcbb0 0x7f28300fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 --2- 172.21.15.59:0/2291117623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28300fd510 0x7f28300f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 -- 172.21.15.59:0/2291117623 >> 172.21.15.59:0/2291117623 conn(0x7f28300f2820 msgr2=0x7f28300f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 -- 172.21.15.59:0/2291117623 shutdown_connections 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.859+0000 7f28368e8700 1 -- 172.21.15.59:0/2291117623 wait complete. 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 librados: starting msgr at 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 librados: starting objecter 2024-02-18T00:41:51.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 Processor -- start 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 -- start start 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 librados: setting wanted keys 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 librados: calling monclient init 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28300f7400 0x7f28300632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28300fcbb0 0x7f28300637b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300fd510 0x7f283011c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2830063eb0 con 0x7f28300fd510 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283011cdc0 con 0x7f28300fcbb0 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.860+0000 7f28368e8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2830104050 con 0x7f28300f7400 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300fd510 0x7f283011c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f282ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28300f7400 0x7f28300632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300fd510 0x7f283011c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39268/0 (socket says 172.21.15.59:39268) 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 -- 172.21.15.59:0/2996689726 learned_addr learned my addr 172.21.15.59:0/2996689726 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:51.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f282f7fe700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28300fcbb0 0x7f28300637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 -- 172.21.15.59:0/2996689726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28300f7400 msgr2=0x7f28300632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28300f7400 0x7f28300632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 -- 172.21.15.59:0/2996689726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28300fcbb0 msgr2=0x7f28300637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28300fcbb0 0x7f28300637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 -- 172.21.15.59:0/2996689726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820005040 con 0x7f28300fd510 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f282f7fe700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28300fcbb0 0x7f28300637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f2834e85700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300fd510 0x7f283011c8d0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 rx=0x7f2828004890 tx=0x7f2828004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f282d7fa700 10 client.?.objecter ms_handle_connect 0x7f28300fd510 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f282d7fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.861+0000 7f282d7fa700 1 -- 172.21.15.59:0/2996689726 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800e040 con 0x7f28300fd510 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.862+0000 7f28368e8700 1 -- 172.21.15.59:0/2996689726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28280097c0 con 0x7f28300fd510 2024-02-18T00:41:51.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.862+0000 7f282d7fa700 1 -- 172.21.15.59:0/2996689726 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2828018040 con 0x7f28300fd510 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.862+0000 7f282d7fa700 1 -- 172.21.15.59:0/2996689726 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828003b90 con 0x7f28300fd510 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.862+0000 7f28368e8700 10 client.31650.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.862+0000 7f28368e8700 1 -- 172.21.15.59:0/2996689726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28301044e0 con 0x7f28300fd510 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.862+0000 7f28368e8700 1 librados: init done 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.865+0000 7f282d7fa700 1 -- 172.21.15.59:0/2996689726 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2828003430 con 0x7f28300fd510 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.865+0000 7f282d7fa700 1 --2- 172.21.15.59:0/2996689726 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2818062c90 0x7f2818065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.865+0000 7f282d7fa700 1 -- 172.21.15.59:0/2996689726 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(578..578 src has 418..578) v4 ==== 8977+0+0 (secure 0 0 0) 0x7f282807e5b0 con 0x7f28300fd510 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.865+0000 7f282d7fa700 10 client.31650.objecter ms_dispatch 0x7f2830001550 osd_map(578..578 src has 418..578) v4 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.865+0000 7f282d7fa700 3 client.31650.objecter handle_osd_map got epochs [578,578] > 0 2024-02-18T00:41:51.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.865+0000 7f282d7fa700 3 client.31650.objecter handle_osd_map decoding full epoch 578 2024-02-18T00:41:51.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.866+0000 7f282ffff700 1 --2- 172.21.15.59:0/2996689726 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2818062c90 0x7f2818065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:51.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.866+0000 7f282d7fa700 20 client.31650.objecter dump_active .. 0 homeless 2024-02-18T00:41:51.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.866+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283005fa60 con 0x7f28300fd510 2024-02-18T00:41:51.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.866+0000 7f282ffff700 1 --2- 172.21.15.59:0/2996689726 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2818062c90 0x7f2818065150 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 rx=0x7f2820004870 tx=0x7f282000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:51.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.866+0000 7f282d7fa700 10 client.31650.objecter ms_handle_connect 0x7f2818062c90 2024-02-18T00:41:51.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:51.871+0000 7f282d7fa700 1 -- 172.21.15.59:0/2996689726 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f28280516e0 con 0x7f28300fd510 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:50.744037+0000 mon.a (mon.0) 2991 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:50.751317+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]': finished 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:50.751417+0000 mon.a (mon.0) 2993 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:50.751470+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:50.770576+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]: dispatch 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:50.770880+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]: dispatch 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:50.744037+0000 mon.a (mon.0) 2991 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:51.894 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:50.751317+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]': finished 2024-02-18T00:41:51.895 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:50.751417+0000 mon.a (mon.0) 2993 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-18T00:41:51.895 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:50.751470+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-18T00:41:51.895 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:50.770576+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]: dispatch 2024-02-18T00:41:51.895 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:50.770880+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]: dispatch 2024-02-18T00:41:51.897 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:52.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:50.744037+0000 mon.a (mon.0) 2991 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:41:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:50.751317+0000 mon.a (mon.0) 2992 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-97-cache", "mode": "writeback"}]': finished 2024-02-18T00:41:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:50.751417+0000 mon.a (mon.0) 2993 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "24"}]': finished 2024-02-18T00:41:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:50.751470+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-18T00:41:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:50.770576+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]: dispatch 2024-02-18T00:41:52.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:50.770880+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]: dispatch 2024-02-18T00:41:52.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.064+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]} v 0) v1 -- 0x7f2830107220 con 0x7f28300fd510 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 -- 172.21.15.59:0/3405587374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 msgr2=0x7f161c0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 --2- 172.21.15.59:0/3405587374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 0x7f161c0f7d40 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 rx=0x7f1610004870 tx=0x7f1610009cb0).stop 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 -- 172.21.15.59:0/3405587374 shutdown_connections 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 --2- 172.21.15.59:0/3405587374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f161c0f8420 0x7f161c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 --2- 172.21.15.59:0/3405587374 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 0x7f161c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 --2- 172.21.15.59:0/3405587374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 0x7f161c0f7d40 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 -- 172.21.15.59:0/3405587374 >> 172.21.15.59:0/3405587374 conn(0x7f161c0f3800 msgr2=0x7f161c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:52.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 -- 172.21.15.59:0/3405587374 shutdown_connections 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.169+0000 7f1624060700 1 -- 172.21.15.59:0/3405587374 wait complete. 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 librados: starting msgr at 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 librados: starting objecter 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 Processor -- start 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 -- start start 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 librados: setting wanted keys 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 librados: calling monclient init 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f161c0f8420 0x7f161c1102e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:52.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 0x7f161c1107d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 0x7f161c10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c10a9c0 con 0x7f161c0fe510 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c10ab30 con 0x7f161c0f8420 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1624060700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c10acf0 con 0x7f161c0fdbb0 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f16215fb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 0x7f161c1107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f16225fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 0x7f161c10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f16215fb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 0x7f161c1107d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49848/0 (socket says 172.21.15.59:49848) 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f16215fb700 1 -- 172.21.15.59:0/4138938131 learned_addr learned my addr 172.21.15.59:0/4138938131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.170+0000 7f1621dfc700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f161c0f8420 0x7f161c1102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f16225fd700 1 -- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 msgr2=0x7f161c1107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f16225fd700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 0x7f161c1107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f16225fd700 1 -- 172.21.15.59:0/4138938131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f161c0f8420 msgr2=0x7f161c1102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f16225fd700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f161c0f8420 0x7f161c1102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f16225fd700 1 -- 172.21.15.59:0/4138938131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610005040 con 0x7f161c0fe510 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f16215fb700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 0x7f161c1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f16225fd700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 0x7f161c10a4d0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 rx=0x7f161400e870 tx=0x7f161400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f161affd700 10 client.?.objecter ms_handle_connect 0x7f161c0fe510 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f161affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f161affd700 1 -- 172.21.15.59:0/4138938131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614007b40 con 0x7f161c0fe510 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f161affd700 1 -- 172.21.15.59:0/4138938131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f161400f040 con 0x7f161c0fe510 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c10afd0 con 0x7f161c0fe510 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f161affd700 1 -- 172.21.15.59:0/4138938131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614014710 con 0x7f161c0fe510 2024-02-18T00:41:52.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f1624060700 10 client.31656.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:52.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c117110 con 0x7f161c0fe510 2024-02-18T00:41:52.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.171+0000 7f1624060700 1 librados: init done 2024-02-18T00:41:52.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.172+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c060280 con 0x7f161c0fe510 2024-02-18T00:41:52.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.172+0000 7f161affd700 1 -- 172.21.15.59:0/4138938131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1614014b40 con 0x7f161c0fe510 2024-02-18T00:41:52.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f161affd700 1 --2- 172.21.15.59:0/4138938131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1604062c90 0x7f1604065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:52.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f161affd700 1 -- 172.21.15.59:0/4138938131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(578..578 src has 418..578) v4 ==== 8977+0+0 (secure 0 0 0) 0x7f161407d040 con 0x7f161c0fe510 2024-02-18T00:41:52.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f161affd700 10 client.31656.objecter ms_dispatch 0x7f161c001550 osd_map(578..578 src has 418..578) v4 2024-02-18T00:41:52.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f161affd700 3 client.31656.objecter handle_osd_map got epochs [578,578] > 0 2024-02-18T00:41:52.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f161affd700 3 client.31656.objecter handle_osd_map decoding full epoch 578 2024-02-18T00:41:52.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f1621dfc700 1 --2- 172.21.15.59:0/4138938131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1604062c90 0x7f1604065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:52.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f161affd700 20 client.31656.objecter dump_active .. 0 homeless 2024-02-18T00:41:52.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f161affd700 10 client.31656.objecter ms_handle_connect 0x7f1604062c90 2024-02-18T00:41:52.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.173+0000 7f1621dfc700 1 --2- 172.21.15.59:0/4138938131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1604062c90 0x7f1604065150 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 rx=0x7f160c004fd0 tx=0x7f160c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:52.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.177+0000 7f161affd700 1 -- 172.21.15.59:0/4138938131 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1614050170 con 0x7f161c0fe510 2024-02-18T00:41:52.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.367+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f161c107110 con 0x7f161c0fe510 2024-02-18T00:41:52.469 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:41:52] "GET /metrics HTTP/1.1" 200 185756 "" "Prometheus/2.33.4" 2024-02-18T00:41:52.545 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:52.548 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:52.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:52.563 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:52.649 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:52.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.755+0000 7f161affd700 1 -- 172.21.15.59:0/4138938131 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v579) v1 ==== 97+0+0 (secure 0 0 0) 0x7f161404fd60 con 0x7f161c0fe510 2024-02-18T00:41:52.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.757+0000 7f282d7fa700 1 -- 172.21.15.59:0/2996689726 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]}]=0 set 2.6 pg_upmap_items mapping to [6->7,4->1,3->5] v579) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2828019250 con 0x7f28300fd510 2024-02-18T00:41:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 10 librados: watch_flush enter 2024-02-18T00:41:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 10 librados: watch_flush exit 2024-02-18T00:41:52.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 20 client.31656.objecter shutdown clearing up homeless session... 2024-02-18T00:41:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 10 client.31656.objecter successfully canceled tick 2024-02-18T00:41:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1604062c90 msgr2=0x7f1604065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 --2- 172.21.15.59:0/4138938131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1604062c90 0x7f1604065150 secure :-1 s=READY pgs=3164 cs=0 l=1 rev1=1 rx=0x7f160c004fd0 tx=0x7f160c00b040).stop 2024-02-18T00:41:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 msgr2=0x7f161c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 0x7f161c10a4d0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 rx=0x7f161400e870 tx=0x7f161400b040).stop 2024-02-18T00:41:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 shutdown_connections 2024-02-18T00:41:52.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 --2- 172.21.15.59:0/4138938131 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1604062c90 0x7f1604065150 unknown :-1 s=CLOSED pgs=3164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f161c0f8420 0x7f161c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f161c0fe510 0x7f161c10a4d0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 --2- 172.21.15.59:0/4138938131 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f161c0fdbb0 0x7f161c1107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 >> 172.21.15.59:0/4138938131 conn(0x7f161c0f3800 msgr2=0x7f161c1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 shutdown_connections 2024-02-18T00:41:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 -- 172.21.15.59:0/4138938131 wait complete. 2024-02-18T00:41:52.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f1624060700 1 librados: shutdown 2024-02-18T00:41:52.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f2816ffd700 10 librados: watch_flush enter 2024-02-18T00:41:52.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f2816ffd700 10 librados: watch_flush exit 2024-02-18T00:41:52.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f2816ffd700 20 client.31650.objecter shutdown clearing up homeless session... 2024-02-18T00:41:52.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f2816ffd700 10 client.31650.objecter successfully canceled tick 2024-02-18T00:41:52.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2818062c90 msgr2=0x7f2818065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.759+0000 7f2816ffd700 1 --2- 172.21.15.59:0/2996689726 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2818062c90 0x7f2818065150 secure :-1 s=READY pgs=3163 cs=0 l=1 rev1=1 rx=0x7f2820004870 tx=0x7f282000b040).stop 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300fd510 msgr2=0x7f283011c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300fd510 0x7f283011c8d0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 rx=0x7f2828004890 tx=0x7f2828004c50).stop 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 shutdown_connections 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28300fd510 0x7f283011c8d0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f28300f7400 0x7f28300632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 --2- 172.21.15.59:0/2996689726 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2818062c90 0x7f2818065150 unknown :-1 s=CLOSED pgs=3163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 --2- 172.21.15.59:0/2996689726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28300fcbb0 0x7f28300637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 >> 172.21.15.59:0/2996689726 conn(0x7f28300f2820 msgr2=0x7f28300f4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 shutdown_connections 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 -- 172.21.15.59:0/2996689726 wait complete. 2024-02-18T00:41:52.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:52.760+0000 7f2816ffd700 1 librados: shutdown 2024-02-18T00:41:52.767 INFO:teuthology.orchestra.run.smithi059.stderr:set 2.6 pg_upmap_items mapping to [6->7,4->1,3->5] 2024-02-18T00:41:52.767 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:41:52.875 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:52.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:52.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:52.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:51.405391+0000 mgr.y (mgr.24454) 527 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:51.753918+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]': finished 2024-02-18T00:41:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:51.753999+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-18T00:41:53.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:51.758046+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:51.758501+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:51.960718+0000 mgr.y (mgr.24454) 528 : cluster [DBG] pgmap v882: 239 pgs: 2 peering, 237 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:41:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:52.065921+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? 172.21.15.59:0/2996689726' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]}]: dispatch 2024-02-18T00:41:53.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:52.369313+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? 172.21.15.59:0/4138938131' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:53.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:51.405391+0000 mgr.y (mgr.24454) 527 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:53.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:51.753918+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]': finished 2024-02-18T00:41:53.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:51.753999+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-18T00:41:53.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:51.758046+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:53.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:51.758501+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:53.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:51.960718+0000 mgr.y (mgr.24454) 528 : cluster [DBG] pgmap v882: 239 pgs: 2 peering, 237 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:41:53.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:52.065921+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? 172.21.15.59:0/2996689726' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]}]: dispatch 2024-02-18T00:41:53.005 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:52.369313+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? 172.21.15.59:0/4138938131' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:53.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:51.405391+0000 mgr.y (mgr.24454) 527 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:41:53.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:51.753918+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-97"}]': finished 2024-02-18T00:41:53.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:51.753999+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-18T00:41:53.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:51.758046+0000 mon.c (mon.2) 547 : audit [INF] from='client.? 172.21.15.59:0/2335330047' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:53.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:51.758501+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]: dispatch 2024-02-18T00:41:53.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:51.960718+0000 mgr.y (mgr.24454) 528 : cluster [DBG] pgmap v882: 239 pgs: 2 peering, 237 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:41:53.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:52.065921+0000 mon.a (mon.0) 2999 : audit [INF] from='client.? 172.21.15.59:0/2996689726' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]}]: dispatch 2024-02-18T00:41:53.007 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:52.369313+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? 172.21.15.59:0/4138938131' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:41:53.009 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:53.082 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:53.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:53.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:53.185 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:53.211 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:53.233 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:53.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:53.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:53.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:53.365 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:53.388 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:53.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:53.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:53.493 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:41:53.520 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:53.573 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:41:53.576 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:53.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:41:53.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:53.705 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:53.725 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:53.727 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:53.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:53.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:53.861 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:52.751420+0000 mon.a (mon.0) 3001 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:52.751445+0000 mon.a (mon.0) 3002 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:52.755714+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:52.755814+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? 172.21.15.59:0/2996689726' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]}]': finished 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:52.755892+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? 172.21.15.59:0/4138938131' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:52.755929+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:53.757497+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:52.751420+0000 mon.a (mon.0) 3001 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:52.751445+0000 mon.a (mon.0) 3002 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:54.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:52.755714+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:54.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:52.755814+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? 172.21.15.59:0/2996689726' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]}]': finished 2024-02-18T00:41:54.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:52.755892+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? 172.21.15.59:0/4138938131' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:54.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:52.755929+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-18T00:41:54.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:53.757497+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-18T00:41:54.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:52.751420+0000 mon.a (mon.0) 3001 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:41:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:52.751445+0000 mon.a (mon.0) 3002 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:41:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:52.755714+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-97", "tierpool": "test-rados-api-smithi059-82957-97-cache"}]': finished 2024-02-18T00:41:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:52.755814+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? 172.21.15.59:0/2996689726' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.6", "id": [6, 7, 4, 1, 3, 5]}]': finished 2024-02-18T00:41:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:52.755892+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? 172.21.15.59:0/4138938131' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:41:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:52.755929+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-18T00:41:54.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:53.757497+0000 mon.a (mon.0) 3007 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-18T00:41:55.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:53.961283+0000 mgr.y (mgr.24454) 529 : cluster [DBG] pgmap v885: 206 pgs: 1 peering, 205 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:55.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:53.962180+0000 mon.a (mon.0) 3008 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-18T00:41:55.137 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.391819+0000 mon.a (mon.0) 3009 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:41:55.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:53.961283+0000 mgr.y (mgr.24454) 529 : cluster [DBG] pgmap v885: 206 pgs: 1 peering, 205 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:55.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:53.962180+0000 mon.a (mon.0) 3008 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-18T00:41:55.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.391819+0000 mon.a (mon.0) 3009 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:41:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:53.961283+0000 mgr.y (mgr.24454) 529 : cluster [DBG] pgmap v885: 206 pgs: 1 peering, 205 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:41:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:53.962180+0000 mon.a (mon.0) 3008 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]: dispatch 2024-02-18T00:41:55.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.391819+0000 mon.a (mon.0) 3009 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:41:55.711 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:55.714 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:55.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:55.815 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:41:55.848 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:41:55.873 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:55.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:41:55.974 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:56.004 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:56.023 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:41:56.024 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:41:56.024 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:56.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:56.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:56.128 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:56.135 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.782871+0000 mon.a (mon.0) 3010 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:54.782931+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.813646+0000 mon.b (mon.1) 369 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.813865+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.814854+0000 mon.b (mon.1) 370 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.814957+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.815540+0000 mon.b (mon.1) 371 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:54.815657+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:56.136 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.782871+0000 mon.a (mon.0) 3010 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-18T00:41:56.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:54.782931+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-18T00:41:56.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.813646+0000 mon.b (mon.1) 369 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.813865+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.814854+0000 mon.b (mon.1) 370 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.814957+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.815540+0000 mon.b (mon.1) 371 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:56.137 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:54.815657+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.782871+0000 mon.a (mon.0) 3010 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "23"}]': finished 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:54.782931+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.813646+0000 mon.b (mon.1) 369 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.813865+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.814854+0000 mon.b (mon.1) 370 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.814957+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.815540+0000 mon.b (mon.1) 371 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:56.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:54.815657+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:41:56.156 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:56.175 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:56.177 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:56.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:56.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:56.310 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:56.363 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:56.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:56.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:56.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:41:56.494 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:57.150 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:41:57.154 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:57.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:41:57.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:41:57.281 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:55.785275+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:41:57.281 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:55.785368+0000 mon.a (mon.0) 3016 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-18T00:41:57.281 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:55.786965+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:57.281 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:55.787235+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:57.282 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:55.962069+0000 mgr.y (mgr.24454) 530 : cluster [DBG] pgmap v888: 174 pgs: 2 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:57.282 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:55.785275+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:41:57.282 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:55.785368+0000 mon.a (mon.0) 3016 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-18T00:41:57.282 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:55.786965+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:57.282 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:55.787235+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:57.282 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:55.962069+0000 mgr.y (mgr.24454) 530 : cluster [DBG] pgmap v888: 174 pgs: 2 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:57.283 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:57.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:55.785275+0000 mon.a (mon.0) 3015 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:41:57.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:55.785368+0000 mon.a (mon.0) 3016 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-18T00:41:57.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:55.786965+0000 mon.b (mon.1) 372 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:57.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:55.787235+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:41:57.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:55.962069+0000 mgr.y (mgr.24454) 530 : cluster [DBG] pgmap v888: 174 pgs: 2 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:57.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:41:58.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-18T00:41:58.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:57.223296+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-18T00:41:58.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:57.787040+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:41:58.316 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:57.787116+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-18T00:41:58.317 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:57.223296+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-18T00:41:58.317 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:57.787040+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:41:58.317 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:57.787116+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-18T00:41:58.332 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:41:58.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:58.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:41:58.436 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:58.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:57.223296+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-18T00:41:58.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:57.787040+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:41:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:57.787116+0000 mon.a (mon.0) 3020 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-18T00:41:58.565 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.565+0000 7f56b227d700 1 -- 172.21.15.59:0/4143219073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 msgr2=0x7f56ac0f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.565+0000 7f56b227d700 1 --2- 172.21.15.59:0/4143219073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 0x7f56ac0f7d40 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 rx=0x7f56a0004870 tx=0x7f56a0009cb0).stop 2024-02-18T00:41:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.565+0000 7f56b227d700 1 -- 172.21.15.59:0/4143219073 shutdown_connections 2024-02-18T00:41:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.565+0000 7f56b227d700 1 --2- 172.21.15.59:0/4143219073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ac0f8420 0x7f56ac104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.565+0000 7f56b227d700 1 --2- 172.21.15.59:0/4143219073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ac0fdbb0 0x7f56ac0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.565+0000 7f56b227d700 1 --2- 172.21.15.59:0/4143219073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 0x7f56ac0f7d40 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 -- 172.21.15.59:0/4143219073 >> 172.21.15.59:0/4143219073 conn(0x7f56ac0f3800 msgr2=0x7f56ac0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 -- 172.21.15.59:0/4143219073 shutdown_connections 2024-02-18T00:41:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 -- 172.21.15.59:0/4143219073 wait complete. 2024-02-18T00:41:58.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:58.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 librados: starting msgr at 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 librados: starting objecter 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 Processor -- start 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 -- start start 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 librados: setting wanted keys 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.566+0000 7f56b227d700 1 librados: calling monclient init 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b227d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ac0f8420 0x7f56ac10e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b227d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ac0fdbb0 0x7f56ac10e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b227d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 0x7f56ac108260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b227d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac108750 con 0x7f56ac0fdbb0 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b227d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac1088c0 con 0x7f56ac0f8420 2024-02-18T00:41:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b227d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac108a80 con 0x7f56ac0fe510 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ac0fdbb0 0x7f56ac10e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b081a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 0x7f56ac108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ac0fdbb0 0x7f56ac10e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39324/0 (socket says 172.21.15.59:39324) 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56ab7fe700 1 -- 172.21.15.59:0/2769254549 learned_addr learned my addr 172.21.15.59:0/2769254549 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ac0f8420 0x7f56ac10e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ac0f8420 msgr2=0x7f56ac10e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b081a700 1 --2- 172.21.15.59:0/2769254549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ac0f8420 0x7f56ac10e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ac0fdbb0 msgr2=0x7f56ac10e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b081a700 1 --2- 172.21.15.59:0/2769254549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ac0fdbb0 0x7f56ac10e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c005040 con 0x7f56ac0fe510 2024-02-18T00:41:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56ab7fe700 1 --2- 172.21.15.59:0/2769254549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ac0fdbb0 0x7f56ac10e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56b081a700 1 --2- 172.21.15.59:0/2769254549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 0x7f56ac108260 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 rx=0x7f56a400d8d0 tx=0x7f56a4004db0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56a9775700 10 client.?.objecter ms_handle_connect 0x7f56ac0fe510 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.567+0000 7f56a9775700 10 client.?.objecter resend_mon_ops 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.568+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a4004020 con 0x7f56ac0fe510 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.568+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56a400f040 con 0x7f56ac0fe510 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.568+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a0005040 con 0x7f56ac0fe510 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.568+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003430 con 0x7f56ac0fe510 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.568+0000 7f56b227d700 10 client.41474.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.568+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac114ef0 con 0x7f56ac0fe510 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.568+0000 7f56b227d700 1 librados: init done 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f56ac05ee80 con 0x7f56ac0fe510 2024-02-18T00:41:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f56a4004180 con 0x7f56ac0fe510 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56a9775700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5694062c90 0x7f5694065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(584..584 src has 418..584) v4 ==== 9046+0+0 (secure 0 0 0) 0x7f56a407cf60 con 0x7f56ac0fe510 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 osd_map(584..584 src has 418..584) v4 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56a9775700 3 client.41474.objecter handle_osd_map got epochs [584,584] > 0 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56a9775700 3 client.41474.objecter handle_osd_map decoding full epoch 584 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.569+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5694062c90 0x7f5694065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56a9775700 20 client.41474.objecter dump_active .. 0 homeless 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v584) v1 ==== 54+0+16 (secure 0 0 0) 0x7f56a407f5f0 con 0x7f56ac0fe510 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 10 client.41474.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 1 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ac028540 0x7f56ac0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5694062c90 0x7f5694065150 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 rx=0x7f569c009330 tx=0x7f569c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f5694062c90 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 1 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 0 1 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1070a0 osd=0 4 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 10 client.41474.objecter _send_command 1 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac109740 con 0x7f56ac028540 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56ab7fe700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ac028540 0x7f56ac0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac028540 2024-02-18T00:41:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.570+0000 7f56ab7fe700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ac028540 0x7f56ac0fe8f0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.571+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ac109740 con 0x7f56ac028540 2024-02-18T00:41:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.571+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:41:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.571+0000 7f56a9775700 10 client.41474.objecter _finish_command 1 = system:0 2024-02-18T00:41:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.571+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac1070a0 osd=0 4 2024-02-18T00:41:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.571+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 0 1 2024-02-18T00:41:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 10 client.41474.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 2 2024-02-18T00:41:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 2 2024-02-18T00:41:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac1070a0 osd=0 3 2024-02-18T00:41:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 0 2 2024-02-18T00:41:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1070a0 osd=0 4 2024-02-18T00:41:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 10 client.41474.objecter _send_command 2 2024-02-18T00:41:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.586+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac05a040 con 0x7f56ac028540 2024-02-18T00:41:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac05a040 con 0x7f56ac028540 2024-02-18T00:41:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56a9775700 10 client.41474.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac1070a0 osd=0 4 2024-02-18T00:41:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 0 2 2024-02-18T00:41:58.590 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:41:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 10 client.41474.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ac027b90 0x7f56ac027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 1 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0fff60 osd=1 4 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 10 client.41474.objecter _send_command 3 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.587+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac1000d0 con 0x7f56ac027b90 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ac027b90 0x7f56ac027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac027b90 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ac027b90 0x7f56ac027f70 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ac1000d0 con 0x7f56ac027b90 2024-02-18T00:41:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:41:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56a9775700 10 client.41474.objecter _finish_command 3 = system:0 2024-02-18T00:41:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac0fff60 osd=1 4 2024-02-18T00:41:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.588+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 1 3 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 10 client.41474.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 4 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 4 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac0fff60 osd=1 3 2024-02-18T00:41:58.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 1 4 2024-02-18T00:41:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0fff60 osd=1 4 2024-02-18T00:41:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 10 client.41474.objecter _send_command 4 2024-02-18T00:41:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.603+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac027600 con 0x7f56ac027b90 2024-02-18T00:41:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac027600 con 0x7f56ac027b90 2024-02-18T00:41:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56a9775700 10 client.41474.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac0fff60 osd=1 4 2024-02-18T00:41:58.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 1 4 2024-02-18T00:41:58.607 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 10 client.41474.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 5 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ac0ffac0 0x7f56ac11c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 5 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 2 5 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac027a10 osd=2 4 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 10 client.41474.objecter _send_command 5 2024-02-18T00:41:58.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac11ef10 con 0x7f56ac0ffac0 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56ab7fe700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ac0ffac0 0x7f56ac11c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac0ffac0 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.605+0000 7f56ab7fe700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ac0ffac0 0x7f56ac11c230 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.606+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ac11ef10 con 0x7f56ac0ffac0 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.606+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.606+0000 7f56a9775700 10 client.41474.objecter _finish_command 5 = system:0 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.606+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac027a10 osd=2 4 2024-02-18T00:41:58.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.606+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 2 5 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 10 client.41474.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 6 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 6 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac027a10 osd=2 3 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 2 6 2024-02-18T00:41:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac027a10 osd=2 4 2024-02-18T00:41:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 10 client.41474.objecter _send_command 6 2024-02-18T00:41:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.620+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac0ff020 con 0x7f56ac0ffac0 2024-02-18T00:41:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac0ff020 con 0x7f56ac0ffac0 2024-02-18T00:41:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56a9775700 10 client.41474.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac027a10 osd=2 4 2024-02-18T00:41:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 2 6 2024-02-18T00:41:58.624 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:41:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 10 client.41474.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 7 2024-02-18T00:41:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ac119d80 0x7f56ac11e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 7 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 3 7 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0ff430 osd=3 4 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 10 client.41474.objecter _send_command 7 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac11ed10 con 0x7f56ac119d80 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b081a700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ac119d80 0x7f56ac11e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac119d80 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.622+0000 7f56b081a700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ac119d80 0x7f56ac11e730 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.623+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ac11ed10 con 0x7f56ac119d80 2024-02-18T00:41:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.623+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:41:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.623+0000 7f56a9775700 10 client.41474.objecter _finish_command 7 = system:0 2024-02-18T00:41:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.623+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac0ff430 osd=3 4 2024-02-18T00:41:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.623+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 3 7 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 10 client.41474.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 8 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 8 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac0ff430 osd=3 3 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 3 8 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0ff430 osd=3 4 2024-02-18T00:41:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 10 client.41474.objecter _send_command 8 2024-02-18T00:41:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.637+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac1192e0 con 0x7f56ac119d80 2024-02-18T00:41:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac1192e0 con 0x7f56ac119d80 2024-02-18T00:41:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56a9775700 10 client.41474.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac0ff430 osd=3 4 2024-02-18T00:41:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 3 8 2024-02-18T00:41:58.641 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:41:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 10 client.41474.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 9 2024-02-18T00:41:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ac123510 0x7f56ac125900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 9 2024-02-18T00:41:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 4 9 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1190a0 osd=4 4 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 10 client.41474.objecter _send_command 9 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac1192d0 con 0x7f56ac123510 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.639+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ac123510 0x7f56ac125900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.640+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac123510 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.640+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ac123510 0x7f56ac125900 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.640+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f569c01f040 con 0x7f56ac123510 2024-02-18T00:41:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.640+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:41:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.640+0000 7f56a9775700 10 client.41474.objecter _finish_command 9 = system:0 2024-02-18T00:41:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.640+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac1190a0 osd=4 4 2024-02-18T00:41:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.640+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 4 9 2024-02-18T00:41:58.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 10 client.41474.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 10 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 10 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac1190a0 osd=4 3 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 4 10 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1190a0 osd=4 4 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 10 client.41474.objecter _send_command 10 2024-02-18T00:41:58.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.655+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac06ced0 con 0x7f56ac123510 2024-02-18T00:41:58.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac06ced0 con 0x7f56ac123510 2024-02-18T00:41:58.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56a9775700 10 client.41474.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac1190a0 osd=4 4 2024-02-18T00:41:58.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 4 10 2024-02-18T00:41:58.659 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 10 client.41474.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 11 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ac126300 0x7f56ac1286f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 11 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 5 11 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac06ced0 osd=5 4 2024-02-18T00:41:58.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 10 client.41474.objecter _send_command 11 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.657+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac12b480 con 0x7f56ac126300 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.658+0000 7f56ab7fe700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ac126300 0x7f56ac1286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.658+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac126300 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.658+0000 7f56ab7fe700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ac126300 0x7f56ac1286f0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.659+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ac12b480 con 0x7f56ac126300 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.659+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.659+0000 7f56a9775700 10 client.41474.objecter _finish_command 11 = system:0 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.659+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac06ced0 osd=5 4 2024-02-18T00:41:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.659+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 5 11 2024-02-18T00:41:58.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 10 client.41474.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 12 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 12 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac06ced0 osd=5 3 2024-02-18T00:41:58.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 5 12 2024-02-18T00:41:58.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac06ced0 osd=5 4 2024-02-18T00:41:58.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 10 client.41474.objecter _send_command 12 2024-02-18T00:41:58.676 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.673+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac12b480 con 0x7f56ac126300 2024-02-18T00:41:58.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac12b480 con 0x7f56ac126300 2024-02-18T00:41:58.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56a9775700 10 client.41474.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac06ced0 osd=5 4 2024-02-18T00:41:58.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 5 12 2024-02-18T00:41:58.677 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:41:58.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 10 client.41474.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 13 2024-02-18T00:41:58.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ac12a7a0 0x7f56ac12abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 13 2024-02-18T00:41:58.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 6 13 2024-02-18T00:41:58.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b480 osd=6 4 2024-02-18T00:41:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 10 client.41474.objecter _send_command 13 2024-02-18T00:41:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.675+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac12b280 con 0x7f56ac12a7a0 2024-02-18T00:41:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.676+0000 7f56b081a700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ac12a7a0 0x7f56ac12abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.676+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac12a7a0 2024-02-18T00:41:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.676+0000 7f56b081a700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ac12a7a0 0x7f56ac12abe0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.677+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ac12b280 con 0x7f56ac12a7a0 2024-02-18T00:41:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.677+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:41:58.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.677+0000 7f56a9775700 10 client.41474.objecter _finish_command 13 = system:0 2024-02-18T00:41:58.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.677+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac12b480 osd=6 4 2024-02-18T00:41:58.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.677+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 6 13 2024-02-18T00:41:58.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 10 client.41474.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 14 2024-02-18T00:41:58.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 14 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac12b480 osd=6 3 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 6 14 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b480 osd=6 4 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 10 client.41474.objecter _send_command 14 2024-02-18T00:41:58.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.692+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac12b280 con 0x7f56ac12a7a0 2024-02-18T00:41:58.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac12b280 con 0x7f56ac12a7a0 2024-02-18T00:41:58.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56a9775700 10 client.41474.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac12b480 osd=6 4 2024-02-18T00:41:58.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 6 14 2024-02-18T00:41:58.696 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 10 client.41474.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 15 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ac12dc80 0x7f56ac12e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 15 2024-02-18T00:41:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 7 15 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b280 osd=7 4 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 10 client.41474.objecter _send_command 15 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.694+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ac12e6e0 con 0x7f56ac12dc80 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.695+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ac12dc80 0x7f56ac12e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.695+0000 7f56a9775700 10 client.41474.objecter ms_handle_connect 0x7f56ac12dc80 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.695+0000 7f56abfff700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ac12dc80 0x7f56ac12e0c0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.696+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f569c00f040 con 0x7f56ac12dc80 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.696+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.696+0000 7f56a9775700 10 client.41474.objecter _finish_command 15 = system:0 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.696+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac12b280 osd=7 4 2024-02-18T00:41:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.696+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 7 15 2024-02-18T00:41:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:58.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 10 client.41474.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:41:58.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign -1 16 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 20 client.41474.objecter _get_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 15 client.41474.objecter _session_command_op_remove -1 16 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 20 client.41474.objecter get_session s=0x7f56ac12b280 osd=7 3 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 15 client.41474.objecter _session_command_op_assign 7 16 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b280 osd=7 4 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 10 client.41474.objecter _send_command 16 2024-02-18T00:41:58.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.711+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ac130920 con 0x7f56ac12dc80 2024-02-18T00:41:58.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56a9775700 1 -- 172.21.15.59:0/2769254549 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ac130920 con 0x7f56ac12dc80 2024-02-18T00:41:58.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56a9775700 10 client.41474.objecter ms_dispatch 0x7f56ac001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:41:58.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56a9775700 10 client.41474.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56a9775700 20 client.41474.objecter put_session s=0x7f56ac12b280 osd=7 4 2024-02-18T00:41:58.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56a9775700 15 client.41474.objecter _session_command_op_remove 7 16 2024-02-18T00:41:58.715 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:41:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 librados: watch_flush enter 2024-02-18T00:41:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 librados: watch_flush exit 2024-02-18T00:41:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.0 2024-02-18T00:41:58.717 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ac028540 msgr2=0x7f56ac0fe8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ac028540 0x7f56ac0fe8f0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1070a0 osd=0 1 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.1 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ac027b90 msgr2=0x7f56ac027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ac027b90 0x7f56ac027f70 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0fff60 osd=1 1 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.2 2024-02-18T00:41:58.718 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ac0ffac0 msgr2=0x7f56ac11c230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ac0ffac0 0x7f56ac11c230 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac027a10 osd=2 1 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.3 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ac119d80 msgr2=0x7f56ac11e730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ac119d80 0x7f56ac11e730 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac0ff430 osd=3 1 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.4 2024-02-18T00:41:58.719 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ac123510 msgr2=0x7f56ac125900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ac123510 0x7f56ac125900 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac1190a0 osd=4 1 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.5 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ac126300 msgr2=0x7f56ac1286f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ac126300 0x7f56ac1286f0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac06ced0 osd=5 1 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.6 2024-02-18T00:41:58.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ac12a7a0 msgr2=0x7f56ac12abe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ac12a7a0 0x7f56ac12abe0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b480 osd=6 1 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter close_session for osd.7 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ac12dc80 msgr2=0x7f56ac12e0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ac12dc80 0x7f56ac12e0c0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter put_session s=0x7f56ac12b280 osd=7 1 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 20 client.41474.objecter shutdown clearing up homeless session... 2024-02-18T00:41:58.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 10 client.41474.objecter successfully canceled tick 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 reap_dead start 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 reap_dead start 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 reap_dead start 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 reap_dead start 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5694062c90 msgr2=0x7f5694065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b081a700 1 -- 172.21.15.59:0/2769254549 reap_dead start 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5694062c90 0x7f5694065150 secure :-1 s=READY pgs=3166 cs=0 l=1 rev1=1 rx=0x7f569c009330 tx=0x7f569c00b040).stop 2024-02-18T00:41:58.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 msgr2=0x7f56ac108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.713+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 0x7f56ac108260 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 rx=0x7f56a400d8d0 tx=0x7f56a4004db0).stop 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.714+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 shutdown_connections 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.714+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ac0fe510 0x7f56ac108260 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.714+0000 7f56b227d700 1 --2- 172.21.15.59:0/2769254549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5694062c90 0x7f5694065150 unknown :-1 s=CLOSED pgs=3166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.714+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 >> 172.21.15.59:0/2769254549 conn(0x7f56ac0f3800 msgr2=0x7f56ac0f5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.714+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 shutdown_connections 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.714+0000 7f56b227d700 1 -- 172.21.15.59:0/2769254549 wait complete. 2024-02-18T00:41:58.723 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:41:58.714+0000 7f56b227d700 1 librados: shutdown 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.724 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:41:58.729 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:41:58.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:58.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:41:58.836 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:41:58.864 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:41:58.885 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:58.886 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:41:58.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:41:59.015 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:59.376 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:59.381 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:41:59.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:57.962488+0000 mgr.y (mgr.24454) 531 : cluster [DBG] pgmap v891: 181 pgs: 8 unknown, 1 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:58.232321+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:58.570785+0000 mon.c (mon.2) 548 : audit [DBG] from='client.? 172.21.15.59:0/2769254549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:58.748017+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:58.748096+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:58.763584+0000 mon.b (mon.1) 373 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:58.764218+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:57.962488+0000 mgr.y (mgr.24454) 531 : cluster [DBG] pgmap v891: 181 pgs: 8 unknown, 1 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:58.232321+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:58.570785+0000 mon.c (mon.2) 548 : audit [DBG] from='client.? 172.21.15.59:0/2769254549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:58.748017+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-18T00:41:59.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:58.748096+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:58.763584+0000 mon.b (mon.1) 373 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:41:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:58.764218+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:57.962488+0000 mgr.y (mgr.24454) 531 : cluster [DBG] pgmap v891: 181 pgs: 8 unknown, 1 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:58.232321+0000 mon.a (mon.0) 3021 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]: dispatch 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:58.570785+0000 mon.c (mon.2) 548 : audit [DBG] from='client.? 172.21.15.59:0/2769254549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:58.748017+0000 mon.a (mon.0) 3022 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_num_actual", "val": "22"}]': finished 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:58.748096+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:58.763584+0000 mon.b (mon.1) 373 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:41:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:58.764218+0000 mon.a (mon.0) 3024 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:41:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:41:59.668 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:59.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:41:59.769 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:59.799 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:59.818 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:59.820 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:59.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:41:59.922 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:41:59.949 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:41:59.971 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:41:59.974 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:41:59.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:00.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:00.453 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:00.456 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:00.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:00.557 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:00.737 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:00.743 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:00.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:00.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:00.873 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:01.001 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:59.749806+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:59.749918+0000 mon.a (mon.0) 3026 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-18T00:42:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:59.756220+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:41:59.762843+0000 mon.a (mon.0) 3027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:41:59.962939+0000 mgr.y (mgr.24454) 532 : cluster [DBG] pgmap v894: 213 pgs: 40 unknown, 1 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:59.749806+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:59.749918+0000 mon.a (mon.0) 3026 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-18T00:42:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:59.756220+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:41:59.762843+0000 mon.a (mon.0) 3027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:01.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:41:59.962939+0000 mgr.y (mgr.24454) 532 : cluster [DBG] pgmap v894: 213 pgs: 40 unknown, 1 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:01.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:59.749806+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:01.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:59.749918+0000 mon.a (mon.0) 3026 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-18T00:42:01.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:59.756220+0000 mon.b (mon.1) 374 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:01.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:41:59.762843+0000 mon.a (mon.0) 3027 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:01.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:41:59.962939+0000 mgr.y (mgr.24454) 532 : cluster [DBG] pgmap v894: 213 pgs: 40 unknown, 1 activating, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:01.322 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:01.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:01.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:01.408 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:42:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:42:01.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:01.454 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:01.702 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:01.704 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:01.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:01.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:42:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:42:01.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:01.835 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:02.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:00.756009+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:00.756112+0000 mon.a (mon.0) 3029 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-18T00:42:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:00.757482+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:02.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:00.758516+0000 mon.a (mon.0) 3030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:00.756009+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:00.756112+0000 mon.a (mon.0) 3029 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-18T00:42:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:00.757482+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:02.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:00.758516+0000 mon.a (mon.0) 3030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:02.037 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:42:02.037 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:42:02.037 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-02-18T00:42:02.037 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-02-18T00:42:02.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 0 1 2024-02-18T00:42:02.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:00.756009+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:02.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:00.756112+0000 mon.a (mon.0) 3029 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-18T00:42:02.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:00.757482+0000 mon.b (mon.1) 375 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:02.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:00.758516+0000 mon.a (mon.0) 3030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:02.482 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:02] "GET /metrics HTTP/1.1" 200 188303 "" "Prometheus/2.33.4" 2024-02-18T00:42:02.484 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:02.487 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:02.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:02.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:02.617 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:02.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:02.779 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:02.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:02.908 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 -- 172.21.15.59:0/1557764919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fdbb0 msgr2=0x7f32bc0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 --2- 172.21.15.59:0/1557764919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fdbb0 0x7f32bc0fdf90 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 rx=0x7f32ac009be0 tx=0x7f32ac004d10).stop 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 -- 172.21.15.59:0/1557764919 shutdown_connections 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 --2- 172.21.15.59:0/1557764919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32bc0f8420 0x7f32bc104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 --2- 172.21.15.59:0/1557764919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fdbb0 0x7f32bc0fdf90 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 --2- 172.21.15.59:0/1557764919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fe510 0x7f32bc0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 -- 172.21.15.59:0/1557764919 >> 172.21.15.59:0/1557764919 conn(0x7f32bc0f3800 msgr2=0x7f32bc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 -- 172.21.15.59:0/1557764919 shutdown_connections 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 -- 172.21.15.59:0/1557764919 wait complete. 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 librados: starting msgr at 2024-02-18T00:42:03.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.075+0000 7f32c1393700 1 librados: starting objecter 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 Processor -- start 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 -- start start 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 librados: setting wanted keys 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 librados: calling monclient init 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32bc0f8420 0x7f32bc10e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fdbb0 0x7f32bc10e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fe510 0x7f32bc1082d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc1087c0 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc108930 con 0x7f32bc0f8420 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32c1393700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc108af0 con 0x7f32bc0fe510 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fdbb0 0x7f32bc10e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fdbb0 0x7f32bc10e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52532/0 (socket says 172.21.15.59:52532) 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32bb7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fe510 0x7f32bc1082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:03.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 -- 172.21.15.59:0/2081043978 learned_addr learned my addr 172.21.15.59:0/2081043978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32baffd700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32bc0f8420 0x7f32bc10e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 -- 172.21.15.59:0/2081043978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fe510 msgr2=0x7f32bc1082d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fe510 0x7f32bc1082d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 -- 172.21.15.59:0/2081043978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32bc0f8420 msgr2=0x7f32bc10e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32bc0f8420 0x7f32bc10e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.076+0000 7f32ba7fc700 1 -- 172.21.15.59:0/2081043978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4005040 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32ba7fc700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fdbb0 0x7f32bc10e620 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 rx=0x7f32a40048f0 tx=0x7f32a4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32b3fff700 10 client.?.objecter ms_handle_connect 0x7f32bc0fdbb0 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32b3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32b3fff700 1 -- 172.21.15.59:0/2081043978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4007b00 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac0097c0 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32c1393700 10 client.31674.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32b3fff700 1 -- 172.21.15.59:0/2081043978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a400f040 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc114f60 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32c1393700 1 librados: init done 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.077+0000 7f32b3fff700 1 -- 172.21.15.59:0/2081043978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4014970 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.078+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc05ee80 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.078+0000 7f32b3fff700 1 -- 172.21.15.59:0/2081043978 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f32a400e790 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.078+0000 7f32b3fff700 1 --2- 172.21.15.59:0/2081043978 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32a8062c90 0x7f32a8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32b3fff700 1 -- 172.21.15.59:0/2081043978 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(589..589 src has 418..589) v4 ==== 9432+0+0 (secure 0 0 0) 0x7f32a407d310 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32b3fff700 10 client.31674.objecter ms_dispatch 0x7f32bc001550 osd_map(589..589 src has 418..589) v4 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32b3fff700 3 client.31674.objecter handle_osd_map got epochs [589,589] > 0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32b3fff700 3 client.31674.objecter handle_osd_map decoding full epoch 589 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32baffd700 1 --2- 172.21.15.59:0/2081043978 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32a8062c90 0x7f32a8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32b3fff700 20 client.31674.objecter dump_active .. 0 homeless 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32baffd700 1 --2- 172.21.15.59:0/2081043978 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32a8062c90 0x7f32a8065150 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 rx=0x7f32ac004e00 tx=0x7f32ac004d10).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.079+0000 7f32b3fff700 10 client.31674.objecter ms_handle_connect 0x7f32a8062c90 2024-02-18T00:42:03.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.083+0000 7f32b3fff700 1 -- 172.21.15.59:0/2081043978 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f32a4049f60 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.107 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:03.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:00.888158+0000 osd.2 (osd.2) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:03.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:00.888486+0000 osd.2 (osd.2) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:03.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:01.406875+0000 mgr.y (mgr.24454) 533 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:03.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:01.755910+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:03.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:01.756018+0000 mon.a (mon.0) 3032 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-18T00:42:03.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:01.758029+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:42:03.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:01.764430+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:42:03.109 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:01.963378+0000 mgr.y (mgr.24454) 534 : cluster [DBG] pgmap v897: 212 pgs: 1 peering, 211 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:03.109 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:03.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:03.110 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:00.888158+0000 osd.2 (osd.2) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:03.110 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:00.888486+0000 osd.2 (osd.2) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:03.110 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:01.406875+0000 mgr.y (mgr.24454) 533 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:03.110 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:01.755910+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:03.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:01.756018+0000 mon.a (mon.0) 3032 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-18T00:42:03.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:01.758029+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:42:03.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:01.764430+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:42:03.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:01.963378+0000 mgr.y (mgr.24454) 534 : cluster [DBG] pgmap v897: 212 pgs: 1 peering, 211 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:03.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:03.240 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:00.888158+0000 osd.2 (osd.2) 233 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:00.888486+0000 osd.2 (osd.2) 234 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:01.406875+0000 mgr.y (mgr.24454) 533 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:01.755910+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-99", "overlaypool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:01.756018+0000 mon.a (mon.0) 3032 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-18T00:42:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:01.758029+0000 mon.b (mon.1) 376 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:42:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:01.764430+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T00:42:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:01.963378+0000 mgr.y (mgr.24454) 534 : cluster [DBG] pgmap v897: 212 pgs: 1 peering, 211 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:03.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:42:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.279+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f32bc1097b0 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.362 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:03.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:03.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:03.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:03.497 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:03.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.775+0000 7f32b3fff700 1 -- 172.21.15.59:0/2081043978 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v590) v1 ==== 91+0+0 (secure 0 0 0) 0x7f32a4050280 con 0x7f32bc0fdbb0 2024-02-18T00:42:03.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 10 librados: watch_flush enter 2024-02-18T00:42:03.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 10 librados: watch_flush exit 2024-02-18T00:42:03.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 20 client.31674.objecter shutdown clearing up homeless session... 2024-02-18T00:42:03.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 10 client.31674.objecter successfully canceled tick 2024-02-18T00:42:03.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32a8062c90 msgr2=0x7f32a8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:03.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 --2- 172.21.15.59:0/2081043978 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32a8062c90 0x7f32a8065150 secure :-1 s=READY pgs=3167 cs=0 l=1 rev1=1 rx=0x7f32ac004e00 tx=0x7f32ac004d10).stop 2024-02-18T00:42:03.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fdbb0 msgr2=0x7f32bc10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fdbb0 0x7f32bc10e620 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 rx=0x7f32a40048f0 tx=0x7f32a4009cb0).stop 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 shutdown_connections 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 --2- 172.21.15.59:0/2081043978 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32a8062c90 0x7f32a8065150 unknown :-1 s=CLOSED pgs=3167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32bc0f8420 0x7f32bc10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32bc0fe510 0x7f32bc1082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 --2- 172.21.15.59:0/2081043978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32bc0fdbb0 0x7f32bc10e620 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.778+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 >> 172.21.15.59:0/2081043978 conn(0x7f32bc0f3800 msgr2=0x7f32bc0fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.779+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 shutdown_connections 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.779+0000 7f32c1393700 1 -- 172.21.15.59:0/2081043978 wait complete. 2024-02-18T00:42:03.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:03.779+0000 7f32c1393700 1 librados: shutdown 2024-02-18T00:42:03.782 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:02.761030+0000 mon.a (mon.0) 3034 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:02.774965+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]': finished 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:02.775028+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:02.779782+0000 mon.b (mon.1) 377 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:02.779986+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:03.280855+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? 172.21.15.59:0/2081043978' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:02.761030+0000 mon.a (mon.0) 3034 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:02.774965+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]': finished 2024-02-18T00:42:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:02.775028+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-18T00:42:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:02.779782+0000 mon.b (mon.1) 377 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:42:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:02.779986+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:42:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:03.280855+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? 172.21.15.59:0/2081043978' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:04.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:02.761030+0000 mon.a (mon.0) 3034 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:04.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:02.774965+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-99-cache", "mode": "writeback"}]': finished 2024-02-18T00:42:04.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:02.775028+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-18T00:42:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:02.779782+0000 mon.b (mon.1) 377 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:42:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:02.779986+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:42:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:03.280855+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? 172.21.15.59:0/2081043978' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:04.746 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:04.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:04.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:04.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:04.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:03.768965+0000 mon.a (mon.0) 3039 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:04.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:03.776027+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:42:04.881 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:03.776113+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? 172.21.15.59:0/2081043978' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:42:04.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:03.776173+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-18T00:42:04.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:03.778571+0000 mon.b (mon.1) 378 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:42:04.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:03.782680+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:42:04.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:03.963872+0000 mgr.y (mgr.24454) 535 : cluster [DBG] pgmap v900: 212 pgs: 1 peering, 211 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:04.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:04.342247+0000 osd.4 (osd.4) 333 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:42:04.882 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:04.345825+0000 osd.4 (osd.4) 334 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:42:04.882 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:03.768965+0000 mon.a (mon.0) 3039 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:03.776027+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:03.776113+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? 172.21.15.59:0/2081043978' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:03.776173+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:03.778571+0000 mon.b (mon.1) 378 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:03.782680+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:03.963872+0000 mgr.y (mgr.24454) 535 : cluster [DBG] pgmap v900: 212 pgs: 1 peering, 211 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:04.342247+0000 osd.4 (osd.4) 333 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:42:04.883 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:04.345825+0000 osd.4 (osd.4) 334 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:42:04.884 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:05.084 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:05.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:05.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:05.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:05.219 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:05.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:03.768965+0000 mon.a (mon.0) 3039 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:03.776027+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:03.776113+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? 172.21.15.59:0/2081043978' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:03.776173+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:03.778571+0000 mon.b (mon.1) 378 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:03.782680+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:03.963872+0000 mgr.y (mgr.24454) 535 : cluster [DBG] pgmap v900: 212 pgs: 1 peering, 211 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:04.342247+0000 osd.4 (osd.4) 333 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:42:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:04.345825+0000 osd.4 (osd.4) 334 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:42:05.438 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:05.445 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:05.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:05.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:05.576 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:05.712 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:42:05.712 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:42:05.712 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:42:05.712 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:42:05.712 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.712 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:975 17.f 17:f7837858:test-rados-api-smithi059-82949-15::foo:head [setxattr a463 in=4b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.393216+0000", 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 543.31015651500002, 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.085327403999999996, 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 975 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.713 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393216+0000", 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393216+0000", 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393215+0000", 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5339999999999999e-06 2024-02-18T00:42:05.714 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393221+0000", 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7600000000000008e-06 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393229+0000", 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0729999999999998e-06 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.715 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393234+0000", 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9961999999999999e-05 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393264+0000", 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.9594e-05 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393313+0000", 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1745999999999999e-05 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.716 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.393405+0000", 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001648465 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.395053+0000", 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013003799999999999 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.395183+0000", 2024-02-18T00:42:05.717 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.083269832000000002 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478453+0000", 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0034999999999998e-05 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478503+0000", 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0154999999999998e-05 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.718 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478543+0000", 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.085327403999999996 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24847.0:2 23.c 23:3096c917:test-rados-api-smithi059-83005-2::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.472340+0000", 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 543.23103244699996, 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.074309046000000004, 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24847", 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1733075084", 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472340+0000", 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.720 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472340+0000", 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472339+0000", 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.401e-06 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472346+0000", 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2798e-05 2024-02-18T00:42:05.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472359+0000", 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1589999999999993e-06 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.472367+0000", 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0065191090000000004 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478886+0000", 2024-02-18T00:42:05.722 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5880000000000003e-06 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.478893+0000", 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055446065000000003 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534339+0000", 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000120016 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.723 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534459+0000", 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014479200000000001 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,6", 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534603+0000", 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0053728430000000004 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.539976+0000", 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045922500000000001 2024-02-18T00:42:05.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.540436+0000", 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061339020000000001 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.546569+0000", 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3572000000000002e-05 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.546613+0000", 2024-02-18T00:42:05.725 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6047000000000002e-05 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.546649+0000", 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.074309046000000004 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:977 17.f 17:f7837858:test-rados-api-smithi059-82949-15::foo:head [setxattr a464 in=4b] snapc 0=[] ondisk+write+known_if_redirected e106)", 2024-02-18T00:42:05.726 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:02.482200+0000", 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 543.22117253500005, 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051936980000000001, 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 977 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482200+0000", 2024-02-18T00:42:05.727 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482200+0000", 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482198+0000", 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5739999999999998e-06 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.728 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482206+0000", 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0079e-05 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482216+0000", 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2129999999999996e-06 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482222+0000", 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2865999999999999e-05 2024-02-18T00:42:05.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482265+0000", 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7281999999999995e-05 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482362+0000", 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00015163199999999999 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.482514+0000", 2024-02-18T00:42:05.730 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00288076 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.485395+0000", 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025177637999999999 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.510572+0000", 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023472665 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.731 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534045+0000", 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.8717000000000001e-05 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534094+0000", 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3095e-05 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:02.534137+0000", 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.051936980000000001 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.732 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:1921 17.f 17:f7837858:test-rados-api-smithi059-82949-15::foo:head [setxattr a936 in=4b] snapc 0=[] ondisk+write+known_if_redirected e107)", 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.441237+0000", 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 542.26213521900002, 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044349531999999997, 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1921 2024-02-18T00:42:05.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441237+0000", 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441237+0000", 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.734 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441236+0000", 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1530000000000003e-06 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441245+0000", 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.575e-05 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441261+0000", 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4599999999999997e-06 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.735 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441268+0000", 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2765999999999997e-05 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441311+0000", 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9658000000000001e-05 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.736 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441401+0000", 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000165329 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.441566+0000", 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0079216749999999996 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.449488+0000", 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.010947795999999999 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.737 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.460436+0000", 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.025060289999999999 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.485496+0000", 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1431000000000003e-05 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.485547+0000", 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9501e-05 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.738 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.485587+0000", 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.044349531999999997 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(client.24823.0:2059 17.11 e108/104 17:8d70d4b8:test-rados-api-smithi059-82949-16::big:head v 108'2, mlcod=0'0)", 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:03.674229+0000", 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 542.02914353799997, 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062844539000000005, 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.674229+0000", 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.674229+0000", 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.740 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.674227+0000", 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016560637 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690788+0000", 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.747e-05 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690815+0000", 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.164e-05 2024-02-18T00:42:05.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690827+0000", 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.4209999999999998e-05 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690881+0000", 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3047999999999999e-05 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.742 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.690914+0000", 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.046112126000000003 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.737026+0000", 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7259999999999998e-05 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:03.737073+0000", 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.062844539000000005 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25025.0:6 22.2 22:47a9abb2:::36.obj2:head [write 0~171] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.068256+0000", 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 541.63511651900001, 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.035616041000000001, 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25025", 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/836362896", 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 6 2024-02-18T00:42:05.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068256+0000", 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068256+0000", 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.745 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068255+0000", 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9129999999999997e-06 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068262+0000", 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0930999999999999e-05 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068273+0000", 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6259999999999997e-06 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.746 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068279+0000", 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0180999999999999e-05 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068330+0000", 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0109000000000002e-05 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068410+0000", 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000111469 2024-02-18T00:42:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 4,7", 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068521+0000", 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00047116299999999997 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.068992+0000", 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034599035 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.748 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103591+0000", 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000171175 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103762+0000", 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7640999999999999e-05 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103820+0000", 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1907999999999998e-05 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.749 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.103872+0000", 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.035616041000000001 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24823.0:2074 17.1c 17:38a558b5:test-rados-api-smithi059-82949-17::big3:head [writefull 0~25165828] snapc 0=[] ondisk+write+known_if_redirected e108)", 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.227584+0000", 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 541.47578809799995, 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.094293116999999996, 2024-02-18T00:42:05.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24823", 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/147524826", 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2074 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.227584+0000", 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.751 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.227584+0000", 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.227583+0000", 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.021037292999999999 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248620+0000", 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4719999999999999e-05 2024-02-18T00:42:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248665+0000", 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7345000000000001e-05 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248682+0000", 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.3851999999999997e-05 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248756+0000", 2024-02-18T00:42:05.753 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000160879 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.248917+0000", 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017516499999999999 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 3,6", 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.249092+0000", 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.057748466999999998 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.306841+0000", 2024-02-18T00:42:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.003428871 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.310269+0000", 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.011530294999999999 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.321800+0000", 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3850000000000003e-05 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.755 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.321834+0000", 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.3900000000000003e-05 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.321878+0000", 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.094293116999999996 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "pg_update_log_missing(23.a epoch 109/105 rep_tid 112 entries 109'4 (0'0) error 23:5074b981:test-rados-api-smithi059-83005-4::foo:head by client.24847.0:15 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 108'2)", 2024-02-18T00:42:05.756 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 541.12003739500005, 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071963654000000002, 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.757 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583335+0000", 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6400000000000003e-06 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583343+0000", 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1528999999999999e-05 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.758 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583355+0000", 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.1840000000000005e-06 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.583364+0000", 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016310387999999999 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.599675+0000", 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.055624172 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:04.655299+0000", 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.071963654000000002 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25073.0:2 37.1a 37:5c6776db:::0.obj:head [write 0~103] snapc 0=[] ondisk+write+known_if_redirected e110)", 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:05.519257+0000", 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 540.18411548899996, 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9908002250000001, 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.760 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25073", 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1549676353", 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519257+0000", 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.761 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519257+0000", 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519255+0000", 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4740000000000005e-06 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519264+0000", 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1605e-05 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.762 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519275+0000", 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1300000000000001e-06 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.519284+0000", 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033962747000000001 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.553246+0000", 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.8060000000000001e-06 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:05.553255+0000", 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.947313686 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.500569+0000", 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00011259100000000001 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.500681+0000", 2024-02-18T00:42:05.764 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014137700000000001 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,4", 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.500823+0000", 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0039772840000000002 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.504800+0000", 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000815236 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.505615+0000", 2024-02-18T00:42:05.765 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0043505820000000004 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.509966+0000", 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0556000000000002e-05 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.510016+0000", 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0825000000000003e-05 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.766 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:07.510057+0000", 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9908002250000001 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.3.0:334 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.683941+0000", 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 511.01943111499997, 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048476497, 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.767 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.683941+0000", 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.683941+0000", 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.683939+0000", 2024-02-18T00:42:05.768 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.013830864999999999 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.697770+0000", 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6985e-05 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.697797+0000", 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0740000000000001e-05 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.769 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.697808+0000", 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00057474900000000005 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.698383+0000", 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3714e-05 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.698406+0000", 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.033983909 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.770 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.732390+0000", 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7523000000000001e-05 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.732418+0000", 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.048476497 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.771 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.3.0:335 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:34.721154+0000", 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 510.98221814200002, 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064423909000000001, 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.721154+0000", 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.721154+0000", 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999914 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.721146+0000", 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.028492059 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749638+0000", 2024-02-18T00:42:05.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.7851000000000002e-05 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749666+0000", 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.223e-05 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749678+0000", 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.0841000000000003e-05 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749729+0000", 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2019999999999999e-05 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.749751+0000", 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.035739964999999999 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.785491+0000", 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7519e-05 2024-02-18T00:42:05.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:34.785578+0000", 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.064423909000000001 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25220.0:573 62.8 62:16784af7:test-rados-api-smithi059-82957-14::foo0:head [writefull 0~11] snapc 5=[5,4] ondisk+write+known_if_redirected e157)", 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:55.580624+0000", 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 490.12274818899999, 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039479232000000003, 2024-02-18T00:42:05.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25220", 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 573 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580624+0000", 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580624+0000", 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580621+0000", 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.7116999999999999e-05 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580638+0000", 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6254999999999998e-05 2024-02-18T00:42:05.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580665+0000", 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0737000000000001e-05 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580675+0000", 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7051999999999994e-05 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580742+0000", 2024-02-18T00:42:05.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6139999999999992e-06 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.580751+0000", 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.022686853 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.603438+0000", 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6570000000000005e-05 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.603535+0000", 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000201819 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 2,3", 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.603736+0000", 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022144249999999999 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.605951+0000", 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.012419968999999999 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.618371+0000", 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0016625450000000001 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.620033+0000", 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0306999999999999e-05 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.620074+0000", 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9947e-05 2024-02-18T00:42:05.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:55.620104+0000", 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.039479232000000003 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "MOSDPGPushReply(26.1b 161/160 [PushReplyOp(26:d83876eb:::4:head)])", 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:33:59.978530+0000", 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 485.72484229600002, 2024-02-18T00:42:05.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089631667999999998, 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978530+0000", 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978530+0000", 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:42:05.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978529+0000", 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.6989999999999997e-06 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978536+0000", 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0166000000000001e-05 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978546+0000", 2024-02-18T00:42:05.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.074e-06 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:33:59.978551+0000", 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089439904000000001 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:00.067991+0000", 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000170688 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:00.068162+0000", 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.089631667999999998 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24862.0:1065 26.d 26:b0000000::::head [pgnls start_epoch 173] snapc 0=[] RETRY=1 ondisk+retry+read+ignore_overlay+known_if_redirected e173)", 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:12.650773+0000", 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 473.05259956499998, 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96835666399999998, 2024-02-18T00:42:05.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24862", 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1312135677", 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650773+0000", 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650773+0000", 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650771+0000", 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1140000000000006e-06 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.650779+0000", 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0012827730000000001 2024-02-18T00:42:05.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.652062+0000", 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1474000000000001e-05 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.652073+0000", 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031773589999999998 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.655251+0000", 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2709e-05 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.655264+0000", 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96304593400000005 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.618309+0000", 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6347999999999997e-05 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.618356+0000", 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00077375499999999999 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619130+0000", 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96835666399999998 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.24862.0:1065 26.d 26:b0000000::::head [pgnls start_epoch 173] snapc 0=[] RETRY=2 ondisk+retry+read+ignore_overlay+known_if_redirected e174)", 2024-02-18T00:42:05.792 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:12.659255+0000", 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 473.044117481, 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96029274099999995, 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.24862", 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1312135677", 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1065 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659255+0000", 2024-02-18T00:42:05.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659255+0000", 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659252+0000", 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2879e-05 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659265+0000", 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6169000000000001e-05 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659281+0000", 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.6491000000000001e-05 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659308+0000", 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9378e-05 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659367+0000", 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1171e-05 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:12.659378+0000", 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.95979067600000001 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619169+0000", 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7784000000000003e-05 2024-02-18T00:42:05.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619207+0000", 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00034093200000000001 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:13.619548+0000", 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96029274099999995 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30543.0:2 117.13 117:cbb9903e:::test_obj:head [write-same] snapc 0=[] ondisk+write+known_if_redirected e178)", 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:16.963477+0000", 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 468.73989515199997, 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047047512999999999, 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30543", 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/481047181", 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963477+0000", 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963477+0000", 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963476+0000", 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4200000000000007e-06 2024-02-18T00:42:05.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963484+0000", 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1267000000000001e-05 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963495+0000", 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8210000000000002e-06 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.963503+0000", 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00052219300000000001 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.964025+0000", 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1019999999999997e-06 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.964033+0000", 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.020100052 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984133+0000", 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000125695 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984259+0000", 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017865900000000001 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 0,3", 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984438+0000", 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00053644399999999996 2024-02-18T00:42:05.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.984974+0000", 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.017858202 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.002832+0000", 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074311840000000004 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.010264+0000", 2024-02-18T00:42:05.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5982e-05 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.010320+0000", 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020525200000000001 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.010525+0000", 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.047047512999999999 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.0.0:492 62.13 e210/123 62:cdd2dcb4:test-rados-api-smithi059-82957-14::foo49:7 v 210'17, mlcod=201'13)", 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:48.642857+0000", 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 437.06051526599998, 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038104723, 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642857+0000", 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642857+0000", 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642856+0000", 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9489999999999996e-06 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642864+0000", 2024-02-18T00:42:05.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9909999999999997e-06 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642874+0000", 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.199e-06 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642881+0000", 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1041000000000001e-05 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642942+0000", 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3368000000000001e-05 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.642966+0000", 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.037956973999999997 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.680923+0000", 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.9112000000000001e-05 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:48.680962+0000", 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038104723 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30798.0:1 156.5s0 156:ab97ba35:::nonexistent:head [read 0~128] snapc 0=[] ondisk+read+known_if_redirected e236)", 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:35:14.627697+0000", 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 411.075675725, 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.011055861, 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:42:05.809 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30798", 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/49510408", 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627697+0000", 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627697+0000", 2024-02-18T00:42:05.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627695+0000", 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5110000000000003e-06 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627701+0000", 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.6500000000000008e-06 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627711+0000", 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2419999999999997e-06 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627717+0000", 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.0540999999999997e-05 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627757+0000", 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8550000000000002e-06 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:14.627760+0000", 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.010886739 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:15.638647+0000", 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000105617 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:15.638753+0000", 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.011055861 2024-02-18T00:42:05.813 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.43222.0:1 168.6s0 168:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e257)", 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:35:35.628336+0000", 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 390.07503603100002, 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.016274704, 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.43222", 2024-02-18T00:42:05.814 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/360131685", 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628336+0000", 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628336+0000", 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628335+0000", 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.3909999999999998e-06 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628343+0000", 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.8959999999999998e-06 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628353+0000", 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.0339999999999997e-06 2024-02-18T00:42:05.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.628361+0000", 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00068555099999999996 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.629047+0000", 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.1219999999999999e-06 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:35.629050+0000", 2024-02-18T00:42:05.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0131460990000001 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.642196+0000", 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012572500000000001 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.642322+0000", 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00041691800000000001 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.642738+0000", 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00063511000000000004 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.643374+0000", 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001199117 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.644573+0000", 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8461000000000003e-05 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:35:36.644611+0000", 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.016274704 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31038.0:1 187.2s0 187:5c55be4c:::oid_RoundTripAppendPP_:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e302)", 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:20.660730+0000", 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 345.04264229799998, 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2238795689999999, 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:42:05.820 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31038", 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2265555398", 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660730+0000", 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:42:05.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660730+0000", 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660728+0000", 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.4997999999999999e-05 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660743+0000", 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0403e-05 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660753+0000", 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.268e-06 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.660760+0000", 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0030131009999999998 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.663773+0000", 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.6430000000000001e-06 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:20.663776+0000", 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2185161870000001 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.882293+0000", 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000134227 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.882427+0000", 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00042525700000000001 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.882852+0000", 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00078715700000000003 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.883639+0000", 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00090266100000000002 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.884542+0000", 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7854999999999993e-05 2024-02-18T00:42:05.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:21.884610+0000", 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2238795689999999 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:05.826 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:42:05.827 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:04.262626+0000 osd.0 (osd.0) 264 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:04.263579+0000 osd.0 (osd.0) 265 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:04.727002+0000 osd.2 (osd.2) 235 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:04.727897+0000 osd.2 (osd.2) 236 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:04.787406+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:04.787484+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:04.790121+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:04.790645+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.610679+0000 osd.7 (osd.7) 157 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:42:05.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.611883+0000 osd.7 (osd.7) 158 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:04.262626+0000 osd.0 (osd.0) 264 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:04.263579+0000 osd.0 (osd.0) 265 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:04.727002+0000 osd.2 (osd.2) 235 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:04.727897+0000 osd.2 (osd.2) 236 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:04.787406+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:04.787484+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:04.790121+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:04.790645+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.610679+0000 osd.7 (osd.7) 157 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:42:05.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.611883+0000 osd.7 (osd.7) 158 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:42:05.934 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:05.938 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:05.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:06.039 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:06.069 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:06.167 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:06.170 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:06.170 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:06.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:04.262626+0000 osd.0 (osd.0) 264 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:04.263579+0000 osd.0 (osd.0) 265 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:04.727002+0000 osd.2 (osd.2) 235 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:04.727897+0000 osd.2 (osd.2) 236 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:04.787406+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:04.787484+0000 mon.a (mon.0) 3045 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:04.790121+0000 mon.b (mon.1) 379 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:04.790645+0000 mon.a (mon.0) 3046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.610679+0000 osd.7 (osd.7) 157 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:42:06.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.611883+0000 osd.7 (osd.7) 158 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:42:06.271 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:06.399 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:06.400 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:06.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:06.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:06.590 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:42:06.653 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:06.656 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:06.656 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:06.757 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:06.789 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.682858+0000 osd.5 (osd.5) 249 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:42:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.684134+0000 osd.5 (osd.5) 250 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:42:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.783784+0000 mon.a (mon.0) 3047 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:05.789089+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:42:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.789169+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-18T00:42:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:05.790844+0000 mon.b (mon.1) 380 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T00:42:06.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:05.795504+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T00:42:06.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.812666+0000 osd.1 (osd.1) 295 : cluster [DBG] 2.a scrub starts 2024-02-18T00:42:06.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.815645+0000 osd.1 (osd.1) 296 : cluster [DBG] 2.a scrub ok 2024-02-18T00:42:06.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:05.964880+0000 mgr.y (mgr.24454) 536 : cluster [DBG] pgmap v903: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:06.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.372978+0000 osd.4 (osd.4) 335 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:42:06.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.374770+0000 osd.4 (osd.4) 336 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:42:06.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.604439+0000 osd.7 (osd.7) 159 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:42:06.800 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.606761+0000 osd.7 (osd.7) 160 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:42:06.815 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:06.818 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:06.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:06.920 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:06.949 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:06.968 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:06.970 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:06.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:07.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:07.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.682858+0000 osd.5 (osd.5) 249 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:42:07.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.684134+0000 osd.5 (osd.5) 250 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:42:07.098 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.783784+0000 mon.a (mon.0) 3047 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:05.789089+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.789169+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:05.790844+0000 mon.b (mon.1) 380 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:05.795504+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.812666+0000 osd.1 (osd.1) 295 : cluster [DBG] 2.a scrub starts 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.815645+0000 osd.1 (osd.1) 296 : cluster [DBG] 2.a scrub ok 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:05.964880+0000 mgr.y (mgr.24454) 536 : cluster [DBG] pgmap v903: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.372978+0000 osd.4 (osd.4) 335 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.374770+0000 osd.4 (osd.4) 336 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.604439+0000 osd.7 (osd.7) 159 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:42:07.099 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.606761+0000 osd.7 (osd.7) 160 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:42:07.100 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:07.224 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:07.229 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:07.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.682858+0000 osd.5 (osd.5) 249 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.684134+0000 osd.5 (osd.5) 250 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.783784+0000 mon.a (mon.0) 3047 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:05.789089+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.789169+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:05.790844+0000 mon.b (mon.1) 380 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:05.795504+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.812666+0000 osd.1 (osd.1) 295 : cluster [DBG] 2.a scrub starts 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.815645+0000 osd.1 (osd.1) 296 : cluster [DBG] 2.a scrub ok 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:05.964880+0000 mgr.y (mgr.24454) 536 : cluster [DBG] pgmap v903: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.372978+0000 osd.4 (osd.4) 335 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:42:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.374770+0000 osd.4 (osd.4) 336 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:42:07.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.604439+0000 osd.7 (osd.7) 159 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:42:07.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.606761+0000 osd.7 (osd.7) 160 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:42:07.330 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:07.361 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:07.382 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:07.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:07.385 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:07.486 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:07.516 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:07.536 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:07.538 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:07.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:07.640 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:07.671 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:07.691 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:07.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:07.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:07.796 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:07.827 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:07.848 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:07.851 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:07.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:07.952 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:07.983 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.731925+0000 osd.5 (osd.5) 251 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.734951+0000 osd.5 (osd.5) 252 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:06.799296+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.799378+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:06.833856+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:06.834102+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.866540+0000 osd.3 (osd.3) 307 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:06.868820+0000 osd.3 (osd.3) 308 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:42:08.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.731925+0000 osd.5 (osd.5) 251 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:42:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.734951+0000 osd.5 (osd.5) 252 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:42:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:06.799296+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-18T00:42:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.799378+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-18T00:42:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:06.833856+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:06.834102+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.866540+0000 osd.3 (osd.3) 307 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:42:08.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:06.868820+0000 osd.3 (osd.3) 308 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:42:08.003 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:08.005 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:08.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:08.106 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.731925+0000 osd.5 (osd.5) 251 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.734951+0000 osd.5 (osd.5) 252 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:06.799296+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.799378+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:06.833856+0000 mon.b (mon.1) 381 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:06.834102+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.866540+0000 osd.3 (osd.3) 307 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:42:08.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:06.868820+0000 osd.3 (osd.3) 308 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:42:08.136 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:08.156 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:08.159 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:08.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:08.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:08.446 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:08.466 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-18T00:42:08.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:07.799865+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:08.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:07.799959+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:07.802088+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:07.805259+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:07.965317+0000 mgr.y (mgr.24454) 537 : cluster [DBG] pgmap v906: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:08.072817+0000 mon.a (mon.0) 3057 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.193561+0000 osd.7 (osd.7) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.193643+0000 osd.7 (osd.7) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:08.554897+0000 mon.a (mon.0) 3058 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:08.555602+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:08.560124+0000 mon.a (mon.0) 3060 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:08.751008+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:08.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.751060+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:07.799865+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:07.799959+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:07.802088+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:07.805259+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:07.965317+0000 mgr.y (mgr.24454) 537 : cluster [DBG] pgmap v906: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:08.072817+0000 mon.a (mon.0) 3057 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.193561+0000 osd.7 (osd.7) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.193643+0000 osd.7 (osd.7) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:08.554897+0000 mon.a (mon.0) 3058 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:08.555602+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:08.560124+0000 mon.a (mon.0) 3060 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:08.751008+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:08.992 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.751060+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-18T00:42:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:07.799865+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:07.799959+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-18T00:42:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:07.802088+0000 mon.b (mon.1) 382 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:07.805259+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]: dispatch 2024-02-18T00:42:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:07.965317+0000 mgr.y (mgr.24454) 537 : cluster [DBG] pgmap v906: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:08.072817+0000 mon.a (mon.0) 3057 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.193561+0000 osd.7 (osd.7) 161 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:09.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.193643+0000 osd.7 (osd.7) 162 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:09.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:08.554897+0000 mon.a (mon.0) 3058 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:42:09.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:08.555602+0000 mon.a (mon.0) 3059 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:42:09.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:08.560124+0000 mon.a (mon.0) 3060 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:09.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:08.751008+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-99", "tierpool": "test-rados-api-smithi059-82957-99-cache"}]': finished 2024-02-18T00:42:09.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.751060+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 -- 172.21.15.59:0/4272013117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 msgr2=0x7f914c104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 --2- 172.21.15.59:0/4272013117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 0x7f914c104a20 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 rx=0x7f9144009be0 tx=0x7f9144004d10).stop 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 -- 172.21.15.59:0/4272013117 shutdown_connections 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 --2- 172.21.15.59:0/4272013117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f914c0fdbb0 0x7f914c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 --2- 172.21.15.59:0/4272013117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 0x7f914c104a20 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 --2- 172.21.15.59:0/4272013117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 0x7f914c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 -- 172.21.15.59:0/4272013117 >> 172.21.15.59:0/4272013117 conn(0x7f914c0f3800 msgr2=0x7f914c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:09.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 -- 172.21.15.59:0/4272013117 shutdown_connections 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.296+0000 7f9153b85700 1 -- 172.21.15.59:0/4272013117 wait complete. 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 librados: starting msgr at 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 librados: starting objecter 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 Processor -- start 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 -- start start 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 librados: setting wanted keys 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 librados: calling monclient init 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 0x7f914c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f914c0fdbb0 0x7f914c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 0x7f914c06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c116510 con 0x7f914c0fe510 2024-02-18T00:42:09.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c116680 con 0x7f914c0fdbb0 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9153b85700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914c116840 con 0x7f914c0f8420 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 0x7f914c06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9151921700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 0x7f914c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 0x7f914c06bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52552/0 (socket says 172.21.15.59:52552) 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 -- 172.21.15.59:0/1390174845 learned_addr learned my addr 172.21.15.59:0/1390174845 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9151120700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f914c0fdbb0 0x7f914c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 -- 172.21.15.59:0/1390174845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 msgr2=0x7f914c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 0x7f914c06b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 -- 172.21.15.59:0/1390174845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f914c0fdbb0 msgr2=0x7f914c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f914c0fdbb0 0x7f914c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 -- 172.21.15.59:0/1390174845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c005040 con 0x7f914c0fe510 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9151120700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f914c0fdbb0 0x7f914c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9152122700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 0x7f914c06bad0 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 rx=0x7f9144005010 tx=0x7f9144004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9142ffd700 10 client.?.objecter ms_handle_connect 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.297+0000 7f9142ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.298+0000 7f9142ffd700 1 -- 172.21.15.59:0/1390174845 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914400e030 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.298+0000 7f9142ffd700 1 -- 172.21.15.59:0/1390174845 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9144007bf0 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.298+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440097c0 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.298+0000 7f9142ffd700 1 -- 172.21.15.59:0/1390174845 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144018070 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.298+0000 7f9153b85700 10 client.31680.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.298+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914c116ce0 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.298+0000 7f9153b85700 1 librados: init done 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.299+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914c05ee80 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.299+0000 7f9142ffd700 1 -- 172.21.15.59:0/1390174845 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f914401d420 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.299+0000 7f9142ffd700 1 --2- 172.21.15.59:0/1390174845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9138062c90 0x7f9138065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9142ffd700 1 -- 172.21.15.59:0/1390174845 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(595..595 src has 418..595) v4 ==== 9424+0+0 (secure 0 0 0) 0x7f914407f0c0 con 0x7f914c0fe510 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9142ffd700 10 client.31680.objecter ms_dispatch 0x7f914c001550 osd_map(595..595 src has 418..595) v4 2024-02-18T00:42:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9142ffd700 3 client.31680.objecter handle_osd_map got epochs [595,595] > 0 2024-02-18T00:42:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9142ffd700 3 client.31680.objecter handle_osd_map decoding full epoch 595 2024-02-18T00:42:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9151921700 1 --2- 172.21.15.59:0/1390174845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9138062c90 0x7f9138065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9142ffd700 20 client.31680.objecter dump_active .. 0 homeless 2024-02-18T00:42:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9142ffd700 10 client.31680.objecter ms_handle_connect 0x7f9138062c90 2024-02-18T00:42:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.300+0000 7f9151921700 1 --2- 172.21.15.59:0/1390174845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9138062c90 0x7f9138065150 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 rx=0x7f9148006fd0 tx=0x7f9148009060).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.304+0000 7f9142ffd700 1 -- 172.21.15.59:0/1390174845 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9144052040 con 0x7f914c0fe510 2024-02-18T00:42:09.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.501+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) v1 -- 0x7f914c05fa60 con 0x7f914c0fe510 2024-02-18T00:42:09.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.755+0000 7f9142ffd700 1 -- 172.21.15.59:0/1390174845 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v596) v1 ==== 133+0+0 (secure 0 0 0) 0x7f914400b560 con 0x7f914c0fe510 2024-02-18T00:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.757+0000 7f9153b85700 10 librados: watch_flush enter 2024-02-18T00:42:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.758+0000 7f9153b85700 10 librados: watch_flush exit 2024-02-18T00:42:09.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.758+0000 7f9153b85700 20 client.31680.objecter shutdown clearing up homeless session... 2024-02-18T00:42:09.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.758+0000 7f9153b85700 10 client.31680.objecter successfully canceled tick 2024-02-18T00:42:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.760+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9138062c90 msgr2=0x7f9138065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.760+0000 7f9153b85700 1 --2- 172.21.15.59:0/1390174845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9138062c90 0x7f9138065150 secure :-1 s=READY pgs=3168 cs=0 l=1 rev1=1 rx=0x7f9148006fd0 tx=0x7f9148009060).stop 2024-02-18T00:42:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.760+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 msgr2=0x7f914c06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.760+0000 7f9153b85700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 0x7f914c06bad0 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 rx=0x7f9144005010 tx=0x7f9144004d10).stop 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 shutdown_connections 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f914c0fe510 0x7f914c06bad0 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f914c0f8420 0x7f914c06b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 --2- 172.21.15.59:0/1390174845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9138062c90 0x7f9138065150 unknown :-1 s=CLOSED pgs=3168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 --2- 172.21.15.59:0/1390174845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f914c0fdbb0 0x7f914c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 >> 172.21.15.59:0/1390174845 conn(0x7f914c0f3800 msgr2=0x7f914c0f50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 shutdown_connections 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 -- 172.21.15.59:0/1390174845 wait complete. 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:09.761+0000 7f9153b85700 1 librados: shutdown 2024-02-18T00:42:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-02-18T00:42:10.139 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.209346+0000 osd.0 (osd.0) 266 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.213403+0000 osd.0 (osd.0) 267 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.811256+0000 osd.1 (osd.1) 297 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.816524+0000 osd.1 (osd.1) 298 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.843221+0000 osd.3 (osd.3) 309 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:08.844924+0000 osd.3 (osd.3) 310 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:09.503039+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? 172.21.15.59:0/1390174845' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:09.755114+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.59:0/1390174845' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:42:10.141 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.755200+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.209346+0000 osd.0 (osd.0) 266 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.213403+0000 osd.0 (osd.0) 267 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.811256+0000 osd.1 (osd.1) 297 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.816524+0000 osd.1 (osd.1) 298 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.843221+0000 osd.3 (osd.3) 309 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:08.844924+0000 osd.3 (osd.3) 310 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:09.503039+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? 172.21.15.59:0/1390174845' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:09.755114+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.59:0/1390174845' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:42:10.142 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.755200+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-18T00:42:10.143 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:10.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:10.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:10.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.209346+0000 osd.0 (osd.0) 266 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:42:10.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.213403+0000 osd.0 (osd.0) 267 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:42:10.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.811256+0000 osd.1 (osd.1) 297 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:42:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.816524+0000 osd.1 (osd.1) 298 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:42:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.843221+0000 osd.3 (osd.3) 309 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:42:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:08.844924+0000 osd.3 (osd.3) 310 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:42:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:09.503039+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? 172.21.15.59:0/1390174845' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:42:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:09.755114+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? 172.21.15.59:0/1390174845' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:42:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.755200+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-18T00:42:10.276 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:11.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.212114+0000 osd.0 (osd.0) 268 : cluster [DBG] 25.a scrub starts 2024-02-18T00:42:11.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.215157+0000 osd.0 (osd.0) 269 : cluster [DBG] 25.a scrub ok 2024-02-18T00:42:11.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.720908+0000 osd.5 (osd.5) 253 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:42:11.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.721894+0000 osd.5 (osd.5) 254 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:42:11.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.794650+0000 osd.3 (osd.3) 311 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:42:11.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.796268+0000 osd.3 (osd.3) 312 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:09.965697+0000 mgr.y (mgr.24454) 538 : cluster [DBG] pgmap v909: 180 pgs: 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:10.345969+0000 osd.4 (osd.4) 337 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:10.346943+0000 osd.4 (osd.4) 338 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:10.703520+0000 osd.2 (osd.2) 237 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:10.704540+0000 osd.2 (osd.2) 238 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:10.754642+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:10.757859+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:10.758820+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.212114+0000 osd.0 (osd.0) 268 : cluster [DBG] 25.a scrub starts 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.215157+0000 osd.0 (osd.0) 269 : cluster [DBG] 25.a scrub ok 2024-02-18T00:42:11.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.720908+0000 osd.5 (osd.5) 253 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.721894+0000 osd.5 (osd.5) 254 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.794650+0000 osd.3 (osd.3) 311 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.796268+0000 osd.3 (osd.3) 312 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:09.965697+0000 mgr.y (mgr.24454) 538 : cluster [DBG] pgmap v909: 180 pgs: 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:10.345969+0000 osd.4 (osd.4) 337 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:10.346943+0000 osd.4 (osd.4) 338 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:10.703520+0000 osd.2 (osd.2) 237 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:10.704540+0000 osd.2 (osd.2) 238 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:10.754642+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:10.757859+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:11.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:10.758820+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.212114+0000 osd.0 (osd.0) 268 : cluster [DBG] 25.a scrub starts 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.215157+0000 osd.0 (osd.0) 269 : cluster [DBG] 25.a scrub ok 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.720908+0000 osd.5 (osd.5) 253 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.721894+0000 osd.5 (osd.5) 254 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.794650+0000 osd.3 (osd.3) 311 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.796268+0000 osd.3 (osd.3) 312 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:42:11.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:09.965697+0000 mgr.y (mgr.24454) 538 : cluster [DBG] pgmap v909: 180 pgs: 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:10.345969+0000 osd.4 (osd.4) 337 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:42:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:10.346943+0000 osd.4 (osd.4) 338 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:42:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:10.703520+0000 osd.2 (osd.2) 237 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:42:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:10.704540+0000 osd.2 (osd.2) 238 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:42:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:10.754642+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-18T00:42:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:10.757859+0000 mon.b (mon.1) 383 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:11.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:10.758820+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:11.528 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:11.532 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:11.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:11.634 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:11.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:42:11.821 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:10.768134+0000 osd.1 (osd.1) 299 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:42:11.864 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:11.866 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:11.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:11.967 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:11.999 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:12.019 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:12.022 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:12.022 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.074+0000 7fccdb718700 1 -- 172.21.15.59:0/2110231972 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fcbb0 msgr2=0x7fccd40fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.074+0000 7fccdb718700 1 --2- 172.21.15.59:0/2110231972 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fcbb0 0x7fccd40fcf90 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 rx=0x7fccc4004870 tx=0x7fccc4009cb0).stop 2024-02-18T00:42:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 -- 172.21.15.59:0/2110231972 shutdown_connections 2024-02-18T00:42:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 --2- 172.21.15.59:0/2110231972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40f7400 0x7fccd4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 --2- 172.21.15.59:0/2110231972 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fcbb0 0x7fccd40fcf90 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 --2- 172.21.15.59:0/2110231972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd40fd510 0x7fccd40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 -- 172.21.15.59:0/2110231972 >> 172.21.15.59:0/2110231972 conn(0x7fccd40f2820 msgr2=0x7fccd40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 -- 172.21.15.59:0/2110231972 shutdown_connections 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 -- 172.21.15.59:0/2110231972 wait complete. 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 librados: starting msgr at 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 librados: starting objecter 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 Processor -- start 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 -- start start 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 librados: setting wanted keys 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.075+0000 7fccdb718700 1 librados: calling monclient init 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccdb718700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd40f7400 0x7fccd4109260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccdb718700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40fcbb0 0x7fccd4109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccdb718700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fd510 0x7fccd4109d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccdb718700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4114440 con 0x7fccd40fcbb0 2024-02-18T00:42:12.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccdb718700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd41145b0 con 0x7fccd40f7400 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccdb718700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4114770 con 0x7fccd40fd510 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40fcbb0 0x7fccd4109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40fcbb0 0x7fccd4109750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59668/0 (socket says 172.21.15.59:59668) 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 -- 172.21.15.59:0/3956748013 learned_addr learned my addr 172.21.15.59:0/3956748013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fd510 0x7fccd4109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd40f7400 0x7fccd4109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 -- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fd510 msgr2=0x7fccd4109d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fd510 0x7fccd4109d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 -- 172.21.15.59:0/3956748013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd40f7400 msgr2=0x7fccd4109260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccd40f7400 0x7fccd4109260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 -- 172.21.15.59:0/3956748013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc0097c0 con 0x7fccd40fcbb0 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fccd40fd510 0x7fccd4109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccd8cb3700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40fcbb0 0x7fccd4109750 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 rx=0x7fccd000c810 tx=0x7fccd0004330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccca7fc700 10 client.?.objecter ms_handle_connect 0x7fccd40fcbb0 2024-02-18T00:42:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccca7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd00043a0 con 0x7fccd40fcbb0 2024-02-18T00:42:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc4005040 con 0x7fccd40fcbb0 2024-02-18T00:42:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd000e970 con 0x7fccd40fcbb0 2024-02-18T00:42:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.076+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0012940 con 0x7fccd40fcbb0 2024-02-18T00:42:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.077+0000 7fccdb718700 10 client.31686.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.077+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd4114e00 con 0x7fccd40fcbb0 2024-02-18T00:42:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.077+0000 7fccdb718700 1 librados: init done 2024-02-18T00:42:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.077+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fccd405fa60 con 0x7fccd40fcbb0 2024-02-18T00:42:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fccd000eae0 con 0x7fccd40fcbb0 2024-02-18T00:42:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6800/3721891797 conn(0x7fccc0062c90 0x7fccc0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(598..598 src has 418..598) v4 ==== 8339+0+0 (secure 0 0 0) 0x7fccd007caf0 con 0x7fccd40fcbb0 2024-02-18T00:42:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 osd_map(598..598 src has 418..598) v4 2024-02-18T00:42:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 3 client.31686.objecter handle_osd_map got epochs [598,598] > 0 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 3 client.31686.objecter handle_osd_map decoding full epoch 598 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6800/3721891797 conn(0x7fccc0062c90 0x7fccc0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 20 client.31686.objecter dump_active .. 0 homeless 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.078+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v598) v1 ==== 54+0+16 (secure 0 0 0) 0x7fccd0081220 con 0x7fccd40fcbb0 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccc0062c90 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6800/3721891797 conn(0x7fccc0062c90 0x7fccc0065150 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 rx=0x7fccc40043d0 tx=0x7fccc400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 10 client.31686.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 1 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6801/3700238977 conn(0x7fccd4028540 0x7fccd40fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 1 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 0 1 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd405ee80 osd=0 4 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 10 client.31686.objecter _send_command 1 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd41060e0 con 0x7fccd4028540 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6801/3700238977 conn(0x7fccd4028540 0x7fccd40fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd4028540 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.079+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6801/3700238977 conn(0x7fccd4028540 0x7fccd40fd990 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.080+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccd41060e0 con 0x7fccd4028540 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.080+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.080+0000 7fccca7fc700 10 client.31686.objecter _finish_command 1 = system:0 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.080+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd405ee80 osd=0 4 2024-02-18T00:42:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.080+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 0 1 2024-02-18T00:42:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 10 client.31686.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 2 2024-02-18T00:42:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 2 2024-02-18T00:42:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd405ee80 osd=0 3 2024-02-18T00:42:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 0 2 2024-02-18T00:42:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd405ee80 osd=0 4 2024-02-18T00:42:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 10 client.31686.objecter _send_command 2 2024-02-18T00:42:12.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.095+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd405a040 con 0x7fccd4028540 2024-02-18T00:42:12.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccd405a040 con 0x7fccd4028540 2024-02-18T00:42:12.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccca7fc700 10 client.31686.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd405ee80 osd=0 4 2024-02-18T00:42:12.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 0 2 2024-02-18T00:42:12.098 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:42:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 10 client.31686.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 3 2024-02-18T00:42:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6805/26279110 conn(0x7fccd4027b90 0x7fccd4027fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 3 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 1 3 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd40ff000 osd=1 4 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 10 client.31686.objecter _send_command 3 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.096+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd40ff170 con 0x7fccd4027b90 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6805/26279110 conn(0x7fccd4027b90 0x7fccd4027fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6805/26279110 conn(0x7fccd4027b90 0x7fccd4027fb0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd4027b90 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccd40ff170 con 0x7fccd4027b90 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccca7fc700 10 client.31686.objecter _finish_command 3 = system:0 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd40ff000 osd=1 4 2024-02-18T00:42:12.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.097+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 1 3 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 10 client.31686.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 4 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 4 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd40ff000 osd=1 3 2024-02-18T00:42:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 1 4 2024-02-18T00:42:12.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd40ff000 osd=1 4 2024-02-18T00:42:12.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 10 client.31686.objecter _send_command 4 2024-02-18T00:42:12.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.116+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd4027600 con 0x7fccd4027b90 2024-02-18T00:42:12.121 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:42:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccd4027600 con 0x7fccd4027b90 2024-02-18T00:42:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccca7fc700 10 client.31686.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd40ff000 osd=1 4 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 1 4 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 10 client.31686.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 5 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6809/1153193637 conn(0x7fccd40feb60 0x7fccd41081d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 5 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 2 5 2024-02-18T00:42:12.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4027a10 osd=2 4 2024-02-18T00:42:12.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 10 client.31686.objecter _send_command 5 2024-02-18T00:42:12.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd411c6c0 con 0x7fccd40feb60 2024-02-18T00:42:12.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.119+0000 7fccd8cb3700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6809/1153193637 conn(0x7fccd40feb60 0x7fccd41081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.120+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd40feb60 2024-02-18T00:42:12.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.120+0000 7fccd8cb3700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6809/1153193637 conn(0x7fccd40feb60 0x7fccd41081d0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.123 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:12.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.121+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccd411c6c0 con 0x7fccd40feb60 2024-02-18T00:42:12.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.121+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:42:12.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.121+0000 7fccca7fc700 10 client.31686.objecter _finish_command 5 = system:0 2024-02-18T00:42:12.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.121+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4027a10 osd=2 4 2024-02-18T00:42:12.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.121+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 2 5 2024-02-18T00:42:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.136+0000 7fccdb718700 10 client.31686.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 6 2024-02-18T00:42:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 6 2024-02-18T00:42:12.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4027a10 osd=2 3 2024-02-18T00:42:12.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 2 6 2024-02-18T00:42:12.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4027a10 osd=2 4 2024-02-18T00:42:12.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 10 client.31686.objecter _send_command 6 2024-02-18T00:42:12.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.137+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd4108e90 con 0x7fccd40feb60 2024-02-18T00:42:12.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccd4108e90 con 0x7fccd40feb60 2024-02-18T00:42:12.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccca7fc700 10 client.31686.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4027a10 osd=2 4 2024-02-18T00:42:12.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 2 6 2024-02-18T00:42:12.141 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:42:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 10 client.31686.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 7 2024-02-18T00:42:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6813/1805700964 conn(0x7fccd411b560 0x7fccd411b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 7 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 3 7 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4109060 osd=3 4 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 10 client.31686.objecter _send_command 7 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.139+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd40fe3f0 con 0x7fccd411b560 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.140+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6813/1805700964 conn(0x7fccd411b560 0x7fccd411b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.140+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd411b560 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.140+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6813/1805700964 conn(0x7fccd411b560 0x7fccd411b9a0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.141+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccd40fe3f0 con 0x7fccd411b560 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.141+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.141+0000 7fccca7fc700 10 client.31686.objecter _finish_command 7 = system:0 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.141+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4109060 osd=3 4 2024-02-18T00:42:12.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.141+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 3 7 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:10.768134+0000 osd.1 (osd.1) 299 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:10.770915+0000 osd.1 (osd.1) 300 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:11.495780+0000 osd.7 (osd.7) 163 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:11.496842+0000 osd.7 (osd.7) 164 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:11.758063+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:11.758146+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:11.758923+0000 mon.b (mon.1) 384 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:12.150 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:11.759075+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 10 client.31686.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 8 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 8 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4109060 osd=3 3 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 3 8 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4109060 osd=3 4 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 10 client.31686.objecter _send_command 8 2024-02-18T00:42:12.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.155+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd411c0d0 con 0x7fccd411b560 2024-02-18T00:42:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccd411c0d0 con 0x7fccd411b560 2024-02-18T00:42:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccca7fc700 10 client.31686.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4109060 osd=3 4 2024-02-18T00:42:12.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 3 8 2024-02-18T00:42:12.159 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 10 client.31686.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 9 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6800/2044387056 conn(0x7fccd4123250 0x7fccd4125640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 9 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 4 9 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd411be90 osd=4 4 2024-02-18T00:42:12.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 10 client.31686.objecter _send_command 9 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd411c0c0 con 0x7fccd4123250 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.157+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6800/2044387056 conn(0x7fccd4123250 0x7fccd4125640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.158+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd4123250 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.158+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6800/2044387056 conn(0x7fccd4123250 0x7fccd4125640 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.158+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccc401a040 con 0x7fccd4123250 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.158+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.158+0000 7fccca7fc700 10 client.31686.objecter _finish_command 9 = system:0 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.158+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd411be90 osd=4 4 2024-02-18T00:42:12.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.158+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 4 9 2024-02-18T00:42:12.171 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:12.174 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:12.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 10 client.31686.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 10 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 10 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd411be90 osd=4 3 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 4 10 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd411be90 osd=4 4 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 10 client.31686.objecter _send_command 10 2024-02-18T00:42:12.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.173+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd4125b30 con 0x7fccd4123250 2024-02-18T00:42:12.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccc401a020 con 0x7fccd4123250 2024-02-18T00:42:12.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccca7fc700 10 client.31686.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.176 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd411be90 osd=4 4 2024-02-18T00:42:12.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 4 10 2024-02-18T00:42:12.177 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 10 client.31686.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 11 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6804/2170432263 conn(0x7fccd4126360 0x7fccd41267a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 11 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 5 11 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4125ca0 osd=5 4 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 10 client.31686.objecter _send_command 11 2024-02-18T00:42:12.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd4127fc0 con 0x7fccd4126360 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.175+0000 7fccd8cb3700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6804/2170432263 conn(0x7fccd4126360 0x7fccd41267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.176+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd4126360 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.176+0000 7fccd8cb3700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6804/2170432263 conn(0x7fccd4126360 0x7fccd41267a0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.176+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccd4127fc0 con 0x7fccd4126360 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.176+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.176+0000 7fccca7fc700 10 client.31686.objecter _finish_command 11 = system:0 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.176+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4125ca0 osd=5 4 2024-02-18T00:42:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.176+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 5 11 2024-02-18T00:42:12.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 10 client.31686.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 12 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 12 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4125ca0 osd=5 3 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 5 12 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4125ca0 osd=5 4 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 10 client.31686.objecter _send_command 12 2024-02-18T00:42:12.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.191+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd4127fc0 con 0x7fccd4126360 2024-02-18T00:42:12.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccd4127fc0 con 0x7fccd4126360 2024-02-18T00:42:12.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccca7fc700 10 client.31686.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4125ca0 osd=5 4 2024-02-18T00:42:12.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 5 12 2024-02-18T00:42:12.195 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 10 client.31686.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 13 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6808/3356009390 conn(0x7fccd41272c0 0x7fccd4127740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 13 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 6 13 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127fc0 osd=6 4 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 10 client.31686.objecter _send_command 13 2024-02-18T00:42:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd4127cc0 con 0x7fccd41272c0 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.193+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6808/3356009390 conn(0x7fccd41272c0 0x7fccd4127740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.194+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd41272c0 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.194+0000 7fccd9cb5700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6808/3356009390 conn(0x7fccd41272c0 0x7fccd4127740 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.194+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccd4127cc0 con 0x7fccd41272c0 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.194+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.194+0000 7fccca7fc700 10 client.31686.objecter _finish_command 13 = system:0 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.194+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4127fc0 osd=6 4 2024-02-18T00:42:12.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.194+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 6 13 2024-02-18T00:42:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:10.768134+0000 osd.1 (osd.1) 299 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:42:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:10.770915+0000 osd.1 (osd.1) 300 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:42:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:11.495780+0000 osd.7 (osd.7) 163 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:42:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:11.496842+0000 osd.7 (osd.7) 164 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:42:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:11.758063+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:11.758146+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-18T00:42:12.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:11.758923+0000 mon.b (mon.1) 384 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:11.759075+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:10.770915+0000 osd.1 (osd.1) 300 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:11.495780+0000 osd.7 (osd.7) 163 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:11.496842+0000 osd.7 (osd.7) 164 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:11.758063+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:11.758146+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:11.758923+0000 mon.b (mon.1) 384 : audit [INF] from='client.? 172.21.15.59:0/2925057028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:11.759075+0000 mon.a (mon.0) 3070 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]: dispatch 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 10 client.31686.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 14 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 14 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4127fc0 osd=6 3 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 6 14 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127fc0 osd=6 4 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 10 client.31686.objecter _send_command 14 2024-02-18T00:42:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.209+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd4127cc0 con 0x7fccd41272c0 2024-02-18T00:42:12.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccd4127cc0 con 0x7fccd41272c0 2024-02-18T00:42:12.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccca7fc700 10 client.31686.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4127fc0 osd=6 4 2024-02-18T00:42:12.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 6 14 2024-02-18T00:42:12.213 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:42:12.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 10 client.31686.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:12.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 15 2024-02-18T00:42:12.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6812/53025171 conn(0x7fccd412c870 0x7fccd412ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 15 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 7 15 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127cc0 osd=7 4 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 10 client.31686.objecter _send_command 15 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd412e920 con 0x7fccd412c870 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6812/53025171 conn(0x7fccd412c870 0x7fccd412ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccca7fc700 10 client.31686.objecter ms_handle_connect 0x7fccd412c870 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.211+0000 7fccd94b4700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6812/53025171 conn(0x7fccd412c870 0x7fccd412ccd0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:12.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.212+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fccd412e920 con 0x7fccd412c870 2024-02-18T00:42:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.212+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:42:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.212+0000 7fccca7fc700 10 client.31686.objecter _finish_command 15 = system:0 2024-02-18T00:42:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.212+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4127cc0 osd=7 4 2024-02-18T00:42:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.212+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 7 15 2024-02-18T00:42:12.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 10 client.31686.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:12.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign -1 16 2024-02-18T00:42:12.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 20 client.31686.objecter _get_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 15 client.31686.objecter _session_command_op_remove -1 16 2024-02-18T00:42:12.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 20 client.31686.objecter get_session s=0x7fccd4127cc0 osd=7 3 2024-02-18T00:42:12.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 15 client.31686.objecter _session_command_op_assign 7 16 2024-02-18T00:42:12.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127cc0 osd=7 4 2024-02-18T00:42:12.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 10 client.31686.objecter _send_command 16 2024-02-18T00:42:12.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.227+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fccd412e920 con 0x7fccd412c870 2024-02-18T00:42:12.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.228+0000 7fccca7fc700 1 -- 172.21.15.59:0/3956748013 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fccd412e920 con 0x7fccd412c870 2024-02-18T00:42:12.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.228+0000 7fccca7fc700 10 client.31686.objecter ms_dispatch 0x7fccd4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:12.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.228+0000 7fccca7fc700 10 client.31686.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.228+0000 7fccca7fc700 20 client.31686.objecter put_session s=0x7fccd4127cc0 osd=7 4 2024-02-18T00:42:12.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.228+0000 7fccca7fc700 15 client.31686.objecter _session_command_op_remove 7 16 2024-02-18T00:42:12.230 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:42:12.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 librados: watch_flush enter 2024-02-18T00:42:12.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 librados: watch_flush exit 2024-02-18T00:42:12.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.0 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6801/3700238977 conn(0x7fccd4028540 msgr2=0x7fccd40fd990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6801/3700238977 conn(0x7fccd4028540 0x7fccd40fd990 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd405ee80 osd=0 1 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.1 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6805/26279110 conn(0x7fccd4027b90 msgr2=0x7fccd4027fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6805/26279110 conn(0x7fccd4027b90 0x7fccd4027fb0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd40ff000 osd=1 1 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.2 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6809/1153193637 conn(0x7fccd40feb60 msgr2=0x7fccd41081d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6809/1153193637 conn(0x7fccd40feb60 0x7fccd41081d0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4027a10 osd=2 1 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.3 2024-02-18T00:42:12.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6813/1805700964 conn(0x7fccd411b560 msgr2=0x7fccd411b9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6813/1805700964 conn(0x7fccd411b560 0x7fccd411b9a0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4109060 osd=3 1 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.4 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6800/2044387056 conn(0x7fccd4123250 msgr2=0x7fccd4125640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6800/2044387056 conn(0x7fccd4123250 0x7fccd4125640 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd411be90 osd=4 1 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.5 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccd9cb5700 1 -- 172.21.15.59:0/3956748013 reap_dead start 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6804/2170432263 conn(0x7fccd4126360 msgr2=0x7fccd41267a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6804/2170432263 conn(0x7fccd4126360 0x7fccd41267a0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccd9cb5700 1 -- 172.21.15.59:0/3956748013 reap_dead start 2024-02-18T00:42:12.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4125ca0 osd=5 1 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.6 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6808/3356009390 conn(0x7fccd41272c0 msgr2=0x7fccd4127740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6808/3356009390 conn(0x7fccd41272c0 0x7fccd4127740 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127fc0 osd=6 1 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter close_session for osd.7 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6812/53025171 conn(0x7fccd412c870 msgr2=0x7fccd412ccd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6812/53025171 conn(0x7fccd412c870 0x7fccd412ccd0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccd9cb5700 1 -- 172.21.15.59:0/3956748013 reap_dead start 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter put_session s=0x7fccd4127cc0 osd=7 1 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 20 client.31686.objecter shutdown clearing up homeless session... 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 10 client.31686.objecter successfully canceled tick 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6800/3721891797 conn(0x7fccc0062c90 msgr2=0x7fccc0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6800/3721891797 conn(0x7fccc0062c90 0x7fccc0065150 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 rx=0x7fccc40043d0 tx=0x7fccc400b040).stop 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40fcbb0 msgr2=0x7fccd4109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40fcbb0 0x7fccd4109750 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 rx=0x7fccd000c810 tx=0x7fccd0004330).stop 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 shutdown_connections 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.137:6812/53025171 conn(0x7fccd412c870 0x7fccd412ccd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fccd40fcbb0 0x7fccd4109750 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 --2- 172.21.15.59:0/3956748013 >> v2:172.21.15.59:6800/3721891797 conn(0x7fccc0062c90 0x7fccc0065150 unknown :-1 s=CLOSED pgs=3169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 >> 172.21.15.59:0/3956748013 conn(0x7fccd40f2820 msgr2=0x7fccd40fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 shutdown_connections 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 -- 172.21.15.59:0/3956748013 wait complete. 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:12.229+0000 7fccdb718700 1 librados: shutdown 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.235 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.236 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.236 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.236 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.236 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:42:12.275 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:12.303 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:12.322 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:12.324 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:12.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:12.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:12.456 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:12.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:12] "GET /metrics HTTP/1.1" 200 190826 "" "Prometheus/2.33.4" 2024-02-18T00:42:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:11.415221+0000 mgr.y (mgr.24454) 539 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:11.848187+0000 osd.3 (osd.3) 313 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:42:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:11.849064+0000 osd.3 (osd.3) 314 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:42:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:11.966271+0000 mgr.y (mgr.24454) 540 : cluster [DBG] pgmap v912: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:12.079652+0000 mon.a (mon.0) 3071 : audit [DBG] from='client.? 172.21.15.59:0/3956748013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:12.756070+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:13.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:12.756117+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-18T00:42:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:11.415221+0000 mgr.y (mgr.24454) 539 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:11.848187+0000 osd.3 (osd.3) 313 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:42:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:11.849064+0000 osd.3 (osd.3) 314 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:42:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:11.966271+0000 mgr.y (mgr.24454) 540 : cluster [DBG] pgmap v912: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:12.079652+0000 mon.a (mon.0) 3071 : audit [DBG] from='client.? 172.21.15.59:0/3956748013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:12.756070+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:13.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:12.756117+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-18T00:42:13.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:11.415221+0000 mgr.y (mgr.24454) 539 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:11.848187+0000 osd.3 (osd.3) 313 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:42:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:11.849064+0000 osd.3 (osd.3) 314 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:42:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:11.966271+0000 mgr.y (mgr.24454) 540 : cluster [DBG] pgmap v912: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:12.079652+0000 mon.a (mon.0) 3071 : audit [DBG] from='client.? 172.21.15.59:0/3956748013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:12.756070+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-99"}]': finished 2024-02-18T00:42:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:12.756117+0000 mon.a (mon.0) 3073 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-18T00:42:13.639 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:13.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:13.644 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:13.648 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:13.745 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:13.779 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:13.799 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:13.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:13.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:13.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:13.934 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:13.953 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:13.955 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:13.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:14.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:12.787254+0000 osd.1 (osd.1) 301 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:12.788458+0000 osd.1 (osd.1) 302 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:12.897791+0000 osd.6 (osd.6) 271 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:12.900642+0000 osd.6 (osd.6) 272 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.766356+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:13.773806+0000 mon.c (mon.2) 549 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:13.774675+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:14.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:12.787254+0000 osd.1 (osd.1) 301 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:12.788458+0000 osd.1 (osd.1) 302 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:12.897791+0000 osd.6 (osd.6) 271 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:12.900642+0000 osd.6 (osd.6) 272 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.766356+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:13.773806+0000 mon.c (mon.2) 549 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:13.774675+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:12.787254+0000 osd.1 (osd.1) 301 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:42:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:12.788458+0000 osd.1 (osd.1) 302 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:42:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:12.897791+0000 osd.6 (osd.6) 271 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:42:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:12.900642+0000 osd.6 (osd.6) 272 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:42:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.766356+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-18T00:42:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:13.773806+0000 mon.c (mon.2) 549 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:13.774675+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:14.889 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:14.891 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:14.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:14.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:15.021 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:15.042 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:15.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:15.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:15.152 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:15.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.676841+0000 osd.5 (osd.5) 255 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:42:15.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.680318+0000 osd.5 (osd.5) 256 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:42:15.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.910534+0000 osd.3 (osd.3) 315 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:42:15.180 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.911909+0000 osd.3 (osd.3) 316 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.924386+0000 osd.6 (osd.6) 273 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.925191+0000 osd.6 (osd.6) 274 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:13.966617+0000 mgr.y (mgr.24454) 541 : cluster [DBG] pgmap v915: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.523482+0000 osd.7 (osd.7) 165 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.524447+0000 osd.7 (osd.7) 166 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.737895+0000 osd.2 (osd.2) 239 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.739590+0000 osd.2 (osd.2) 240 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:14.767137+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.767202+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:14.771093+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:15.181 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:14.771404+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:15.182 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:15.202 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:15.204 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:15.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.676841+0000 osd.5 (osd.5) 255 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:42:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.680318+0000 osd.5 (osd.5) 256 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:42:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.910534+0000 osd.3 (osd.3) 315 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:42:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.911909+0000 osd.3 (osd.3) 316 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:42:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.924386+0000 osd.6 (osd.6) 273 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:42:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.925191+0000 osd.6 (osd.6) 274 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:42:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:13.966617+0000 mgr.y (mgr.24454) 541 : cluster [DBG] pgmap v915: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.523482+0000 osd.7 (osd.7) 165 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.524447+0000 osd.7 (osd.7) 166 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.737895+0000 osd.2 (osd.2) 239 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.739590+0000 osd.2 (osd.2) 240 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:14.767137+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.767202+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:14.771093+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:14.771404+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.676841+0000 osd.5 (osd.5) 255 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:42:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.680318+0000 osd.5 (osd.5) 256 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.910534+0000 osd.3 (osd.3) 315 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.911909+0000 osd.3 (osd.3) 316 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.924386+0000 osd.6 (osd.6) 273 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.925191+0000 osd.6 (osd.6) 274 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:13.966617+0000 mgr.y (mgr.24454) 541 : cluster [DBG] pgmap v915: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.523482+0000 osd.7 (osd.7) 165 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.524447+0000 osd.7 (osd.7) 166 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.737895+0000 osd.2 (osd.2) 239 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.739590+0000 osd.2 (osd.2) 240 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:14.767137+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:42:15.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.767202+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-18T00:42:15.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:14.771093+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.59:0/1987956390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:15.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:14.771404+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]: dispatch 2024-02-18T00:42:15.305 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:15.334 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:15.354 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:15.356 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:15.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:15.458 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:15.485 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:15.503 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:15.506 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:15.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:15.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:42:15.607 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:15.633 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:15.653 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:15.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:15.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:15.756 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:15.783 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:15.802 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:15.804 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:15.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:15.905 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:15.936 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.836879+0000 osd.1 (osd.1) 303 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.840313+0000 osd.1 (osd.1) 304 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:14.881245+0000 mon.a (mon.0) 3079 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.534495+0000 osd.7 (osd.7) 167 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.535838+0000 osd.7 (osd.7) 168 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:15.769860+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.769952+0000 mon.a (mon.0) 3081 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:15.787527+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:15.788487+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:16.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:15.788774+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.836879+0000 osd.1 (osd.1) 303 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.840313+0000 osd.1 (osd.1) 304 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:14.881245+0000 mon.a (mon.0) 3079 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.534495+0000 osd.7 (osd.7) 167 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.535838+0000 osd.7 (osd.7) 168 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:15.769860+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.769952+0000 mon.a (mon.0) 3081 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:15.787527+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:15.788487+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:16.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:15.788774+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:42:16.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.836879+0000 osd.1 (osd.1) 303 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.840313+0000 osd.1 (osd.1) 304 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:14.881245+0000 mon.a (mon.0) 3079 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.534495+0000 osd.7 (osd.7) 167 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.535838+0000 osd.7 (osd.7) 168 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:15.769860+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi059-82957-94"}]': finished 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.769952+0000 mon.a (mon.0) 3081 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:15.787527+0000 mon.a (mon.0) 3082 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:15.788487+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:15.788774+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T00:42:16.385 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:16.387 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:16.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:16.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:16.521 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:16.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:16.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:16.647 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:16.677 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:16.696 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:16.698 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:16.699 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:16.800 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:16.827 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:16.845 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:16.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:16.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:16.949 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:16.976 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:16.995 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:16.997 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:16.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:17.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 -- 172.21.15.59:0/4064534822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40f8420 msgr2=0x7f5da4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 --2- 172.21.15.59:0/4064534822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40f8420 0x7f5da4104a20 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7f5d9c009be0 tx=0x7f5d9c004d10).stop 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 -- 172.21.15.59:0/4064534822 shutdown_connections 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 --2- 172.21.15.59:0/4064534822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5da40fdbb0 0x7f5da40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 --2- 172.21.15.59:0/4064534822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40f8420 0x7f5da4104a20 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 --2- 172.21.15.59:0/4064534822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40fe510 0x7f5da40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 -- 172.21.15.59:0/4064534822 >> 172.21.15.59:0/4064534822 conn(0x7f5da40f3800 msgr2=0x7f5da40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 -- 172.21.15.59:0/4064534822 shutdown_connections 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 -- 172.21.15.59:0/4064534822 wait complete. 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 librados: starting msgr at 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.974+0000 7f5daac59700 1 librados: starting objecter 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 Processor -- start 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 -- start start 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 librados: setting wanted keys 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 librados: calling monclient init 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40f8420 0x7f5da410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40fdbb0 0x7f5da410e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5da40fe510 0x7f5da41080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41085d0 con 0x7f5da40fdbb0 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4108740 con 0x7f5da40fe510 2024-02-18T00:42:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5daac59700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da4108900 con 0x7f5da40f8420 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40f8420 0x7f5da410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40fdbb0 0x7f5da410e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40f8420 0x7f5da410df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:54664/0 (socket says 172.21.15.59:54664) 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 -- 172.21.15.59:0/1617339312 learned_addr learned my addr 172.21.15.59:0/1617339312 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da91f6700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5da40fe510 0x7f5da41080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 -- 172.21.15.59:0/1617339312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5da40fe510 msgr2=0x7f5da41080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5da40fe510 0x7f5da41080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 -- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40fdbb0 msgr2=0x7f5da410e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40fdbb0 0x7f5da410e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 -- 172.21.15.59:0/1617339312 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90005040 con 0x7f5da40f8420 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da3fff700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40fdbb0 0x7f5da410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da89f5700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40f8420 0x7f5da410df40 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 rx=0x7f5d9800e970 tx=0x7f5d9800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da1ffb700 10 client.?.objecter ms_handle_connect 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.975+0000 7f5da1ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.976+0000 7f5da1ffb700 1 -- 172.21.15.59:0/1617339312 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98004660 con 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.976+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0097c0 con 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.976+0000 7f5da1ffb700 1 -- 172.21.15.59:0/1617339312 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d9800f040 con 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.976+0000 7f5da1ffb700 1 -- 172.21.15.59:0/1617339312 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98014790 con 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.976+0000 7f5daac59700 10 client.41504.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.976+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da4114d70 con 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.976+0000 7f5daac59700 1 librados: init done 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da4107110 con 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5da1ffb700 1 -- 172.21.15.59:0/1617339312 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5d980090d0 con 0x7f5da40f8420 2024-02-18T00:42:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5da1ffb700 1 --2- 172.21.15.59:0/1617339312 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d94062c90 0x7f5d94065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5da1ffb700 1 -- 172.21.15.59:0/1617339312 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(602..602 src has 418..602) v4 ==== 7406+0+0 (secure 0 0 0) 0x7f5d9807caa0 con 0x7f5da40f8420 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5da1ffb700 10 client.41504.objecter ms_dispatch 0x7f5da4001550 osd_map(602..602 src has 418..602) v4 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5da1ffb700 3 client.41504.objecter handle_osd_map got epochs [602,602] > 0 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5da1ffb700 3 client.41504.objecter handle_osd_map decoding full epoch 602 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.977+0000 7f5da3fff700 1 --2- 172.21.15.59:0/1617339312 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d94062c90 0x7f5d94065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.978+0000 7f5da1ffb700 20 client.41504.objecter dump_active .. 0 homeless 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.978+0000 7f5da1ffb700 10 client.41504.objecter ms_handle_connect 0x7f5d94062c90 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.978+0000 7f5da3fff700 1 --2- 172.21.15.59:0/1617339312 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d94062c90 0x7f5d94065150 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 rx=0x7f5d90009fd0 tx=0x7f5d9000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:15.982+0000 7f5da1ffb700 1 -- 172.21.15.59:0/1617339312 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5d9804af60 con 0x7f5da40f8420 2024-02-18T00:42:17.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.162+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f5da405fa60 con 0x7f5da40f8420 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.771+0000 7f5da1ffb700 1 -- 172.21.15.59:0/1617339312 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v603) v1 ==== 101+0+0 (secure 0 0 0) 0x7f5d98019090 con 0x7f5da40f8420 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 10 librados: watch_flush enter 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 10 librados: watch_flush exit 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 20 client.41504.objecter shutdown clearing up homeless session... 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 10 client.41504.objecter successfully canceled tick 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d94062c90 msgr2=0x7f5d94065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 --2- 172.21.15.59:0/1617339312 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d94062c90 0x7f5d94065150 secure :-1 s=READY pgs=3171 cs=0 l=1 rev1=1 rx=0x7f5d90009fd0 tx=0x7f5d9000b040).stop 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40f8420 msgr2=0x7f5da410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40f8420 0x7f5da410df40 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 rx=0x7f5d9800e970 tx=0x7f5d9800b040).stop 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 shutdown_connections 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 --2- 172.21.15.59:0/1617339312 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5d94062c90 0x7f5d94065150 unknown :-1 s=CLOSED pgs=3171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5da40fe510 0x7f5da41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5da40f8420 0x7f5da410df40 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 --2- 172.21.15.59:0/1617339312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5da40fdbb0 0x7f5da410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.773+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 >> 172.21.15.59:0/1617339312 conn(0x7f5da40f3800 msgr2=0x7f5da40f5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.774+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 shutdown_connections 2024-02-18T00:42:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.774+0000 7f5daac59700 1 -- 172.21.15.59:0/1617339312 wait complete. 2024-02-18T00:42:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:16.774+0000 7f5daac59700 1 librados: shutdown 2024-02-18T00:42:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:42:17.128 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.350640+0000 osd.6 (osd.6) 275 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.350876+0000 osd.6 (osd.6) 276 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.646776+0000 osd.5 (osd.5) 257 : cluster [DBG] 6.a scrub starts 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.647803+0000 osd.5 (osd.5) 258 : cluster [DBG] 6.a scrub ok 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.863556+0000 osd.1 (osd.1) 305 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.866460+0000 osd.1 (osd.1) 306 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:15.967314+0000 mgr.y (mgr.24454) 542 : cluster [DBG] pgmap v918: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:16.164196+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.59:0/1617339312' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:16.164478+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:16.576162+0000 osd.7 (osd.7) 169 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:16.577259+0000 osd.7 (osd.7) 170 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:16.766598+0000 mon.a (mon.0) 3086 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:16.771353+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:42:17.129 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:16.771437+0000 mon.a (mon.0) 3088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:16.771488+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:16.772602+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.350640+0000 osd.6 (osd.6) 275 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.350876+0000 osd.6 (osd.6) 276 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.646776+0000 osd.5 (osd.5) 257 : cluster [DBG] 6.a scrub starts 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.647803+0000 osd.5 (osd.5) 258 : cluster [DBG] 6.a scrub ok 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.863556+0000 osd.1 (osd.1) 305 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.866460+0000 osd.1 (osd.1) 306 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:15.967314+0000 mgr.y (mgr.24454) 542 : cluster [DBG] pgmap v918: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:42:17.130 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:16.164196+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.59:0/1617339312' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:16.164478+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:16.576162+0000 osd.7 (osd.7) 169 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:16.577259+0000 osd.7 (osd.7) 170 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:16.766598+0000 mon.a (mon.0) 3086 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:16.771353+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:16.771437+0000 mon.a (mon.0) 3088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:16.771488+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-18T00:42:17.131 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:16.772602+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.350640+0000 osd.6 (osd.6) 275 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.350876+0000 osd.6 (osd.6) 276 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.646776+0000 osd.5 (osd.5) 257 : cluster [DBG] 6.a scrub starts 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.647803+0000 osd.5 (osd.5) 258 : cluster [DBG] 6.a scrub ok 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.863556+0000 osd.1 (osd.1) 305 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.866460+0000 osd.1 (osd.1) 306 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:15.967314+0000 mgr.y (mgr.24454) 542 : cluster [DBG] pgmap v918: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:16.164196+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.59:0/1617339312' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:16.164478+0000 mon.a (mon.0) 3085 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:16.576162+0000 osd.7 (osd.7) 169 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:16.577259+0000 osd.7 (osd.7) 170 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:42:17.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:16.766598+0000 mon.a (mon.0) 3086 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:42:17.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:16.771353+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi059-82957-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T00:42:17.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:16.771437+0000 mon.a (mon.0) 3088 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:42:17.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:16.771488+0000 mon.a (mon.0) 3089 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-18T00:42:17.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:16.772602+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:17.437 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:17.746 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:42:17.747 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:42:17.747 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-02-18T00:42:17.747 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-18T00:42:17.779 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:17.784 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:17.785 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:17.886 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:17.918 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:17.939 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:17.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:17.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:18.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:18.081 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:18.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:16.893373+0000 osd.6 (osd.6) 277 : cluster [DBG] 25.0 deep-scrub starts 2024-02-18T00:42:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:16.894511+0000 osd.6 (osd.6) 278 : cluster [DBG] 25.0 deep-scrub ok 2024-02-18T00:42:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:17.367822+0000 osd.4 (osd.4) 339 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:17.368056+0000 osd.4 (osd.4) 340 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:16.893373+0000 osd.6 (osd.6) 277 : cluster [DBG] 25.0 deep-scrub starts 2024-02-18T00:42:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:16.894511+0000 osd.6 (osd.6) 278 : cluster [DBG] 25.0 deep-scrub ok 2024-02-18T00:42:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:17.367822+0000 osd.4 (osd.4) 339 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:17.368056+0000 osd.4 (osd.4) 340 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:16.893373+0000 osd.6 (osd.6) 277 : cluster [DBG] 25.0 deep-scrub starts 2024-02-18T00:42:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:16.894511+0000 osd.6 (osd.6) 278 : cluster [DBG] 25.0 deep-scrub ok 2024-02-18T00:42:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:17.367822+0000 osd.4 (osd.4) 339 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:17.368056+0000 osd.4 (osd.4) 340 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:18.543 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:18.546 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:18.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:18.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:18.677 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:19.229 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:19.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:19.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:19.355 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:17.756554+0000 osd.3 (osd.3) 317 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:17.756802+0000 osd.3 (osd.3) 318 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:17.907270+0000 osd.6 (osd.6) 279 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:17.908306+0000 osd.6 (osd.6) 280 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:17.967697+0000 mgr.y (mgr.24454) 543 : cluster [DBG] pgmap v921: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:18.191634+0000 osd.0 (osd.0) 270 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:18.192796+0000 osd.0 (osd.0) 271 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:18.224721+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:18.754260+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:19.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:18.754328+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:17.756554+0000 osd.3 (osd.3) 317 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:17.756802+0000 osd.3 (osd.3) 318 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:17.907270+0000 osd.6 (osd.6) 279 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:17.908306+0000 osd.6 (osd.6) 280 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:17.967697+0000 mgr.y (mgr.24454) 543 : cluster [DBG] pgmap v921: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:18.191634+0000 osd.0 (osd.0) 270 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:18.192796+0000 osd.0 (osd.0) 271 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:18.224721+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:18.754260+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:19.357 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:18.754328+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-18T00:42:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:19.422 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:19.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:19.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:17.756554+0000 osd.3 (osd.3) 317 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:17.756802+0000 osd.3 (osd.3) 318 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:17.907270+0000 osd.6 (osd.6) 279 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:42:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:17.908306+0000 osd.6 (osd.6) 280 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:42:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:17.967697+0000 mgr.y (mgr.24454) 543 : cluster [DBG] pgmap v921: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:18.191634+0000 osd.0 (osd.0) 270 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:42:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:18.192796+0000 osd.0 (osd.0) 271 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:42:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:18.224721+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-18T00:42:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:18.754260+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi059-82957-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:19.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:18.754328+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-18T00:42:19.523 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:19.556 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:19.577 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:19.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:19.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:19.682 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:19.711 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:19.730 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:19.733 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:19.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:19.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:19.864 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:19.886 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:19.888 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:19.889 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:19.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:20.018 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:20.217 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:20.220 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:20.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:20.321 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:20.352 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:20.375 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:20.375 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:20.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 -- 172.21.15.59:0/3311773335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 msgr2=0x7fd18c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 --2- 172.21.15.59:0/3311773335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 0x7fd18c101d80 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 rx=0x7fd174004870 tx=0x7fd174009cb0).stop 2024-02-18T00:42:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 -- 172.21.15.59:0/3311773335 shutdown_connections 2024-02-18T00:42:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 --2- 172.21.15.59:0/3311773335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c102270 0x7fd18c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 --2- 172.21.15.59:0/3311773335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd18c0f8d10 0x7fd18c0f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 --2- 172.21.15.59:0/3311773335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 0x7fd18c101d80 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 -- 172.21.15.59:0/3311773335 >> 172.21.15.59:0/3311773335 conn(0x7fd18c0f2820 msgr2=0x7fd18c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 -- 172.21.15.59:0/3311773335 shutdown_connections 2024-02-18T00:42:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 -- 172.21.15.59:0/3311773335 wait complete. 2024-02-18T00:42:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 librados: starting msgr at 2024-02-18T00:42:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.452+0000 7fd1914ae700 1 librados: starting objecter 2024-02-18T00:42:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 Processor -- start 2024-02-18T00:42:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 -- start start 2024-02-18T00:42:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 librados: setting wanted keys 2024-02-18T00:42:20.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 librados: calling monclient init 2024-02-18T00:42:20.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c0f8d10 0x7fd18c0f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 0x7fd18c0f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd18c102270 0x7fd18c0f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0f8a20 con 0x7fd18c102270 2024-02-18T00:42:20.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0f8b90 con 0x7fd18c0f8d10 2024-02-18T00:42:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd1914ae700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c11af10 con 0x7fd18c0f9670 2024-02-18T00:42:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd18c102270 0x7fd18c0f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 0x7fd18c0f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd18c102270 0x7fd18c0f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60864/0 (socket says 172.21.15.59:60864) 2024-02-18T00:42:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 0x7fd18c0f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:40714/0 (socket says 172.21.15.59:40714) 2024-02-18T00:42:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18b7fe700 1 -- 172.21.15.59:0/1208558152 learned_addr learned my addr 172.21.15.59:0/1208558152 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c0f8d10 0x7fd18c0f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18b7fe700 1 -- 172.21.15.59:0/1208558152 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 msgr2=0x7fd18c0f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18b7fe700 1 --2- 172.21.15.59:0/1208558152 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 0x7fd18c0f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18b7fe700 1 -- 172.21.15.59:0/1208558152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c0f8d10 msgr2=0x7fd18c0f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.453+0000 7fd18b7fe700 1 --2- 172.21.15.59:0/1208558152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd18c0f8d10 0x7fd18c0f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd18b7fe700 1 -- 172.21.15.59:0/1208558152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c005040 con 0x7fd18c102270 2024-02-18T00:42:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd18a7fc700 1 --2- 172.21.15.59:0/1208558152 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd18c0f9670 0x7fd18c0f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd18b7fe700 1 --2- 172.21.15.59:0/1208558152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd18c102270 0x7fd18c0f76a0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7fd18400d970 tx=0x7fd184004e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd183fff700 10 client.?.objecter ms_handle_connect 0x7fd18c102270 2024-02-18T00:42:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd183fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184004020 con 0x7fd18c102270 2024-02-18T00:42:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd174005040 con 0x7fd18c102270 2024-02-18T00:42:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd18400f040 con 0x7fd18c102270 2024-02-18T00:42:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd1914ae700 10 client.31704.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c11b5a0 con 0x7fd18c102270 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd1914ae700 1 librados: init done 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.454+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184003640 con 0x7fd18c102270 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.455+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fd18c05fa60 con 0x7fd18c102270 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.455+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd18400b510 con 0x7fd18c102270 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd178062c90 0x7fd178065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(606..606 src has 418..606) v4 ==== 8599+0+0 (secure 0 0 0) 0x7fd18407d880 con 0x7fd18c102270 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 osd_map(606..606 src has 418..606) v4 2024-02-18T00:42:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 3 client.31704.objecter handle_osd_map got epochs [606,606] > 0 2024-02-18T00:42:20.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 3 client.31704.objecter handle_osd_map decoding full epoch 606 2024-02-18T00:42:20.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd178062c90 0x7fd178065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 20 client.31704.objecter dump_active .. 0 homeless 2024-02-18T00:42:20.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v606) v1 ==== 54+0+16 (secure 0 0 0) 0x7fd18407e0a0 con 0x7fd18c102270 2024-02-18T00:42:20.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 10 client.31704.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd178062c90 0x7fd178065150 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 rx=0x7fd17c009350 tx=0x7fd17c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd178062c90 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 1 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd18c0fc200 0x7fd18c0fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 1 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 0 1 2024-02-18T00:42:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c05ee80 osd=0 4 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 10 client.31704.objecter _send_command 1 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c1072a0 con 0x7fd18c0fc200 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd18a7fc700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd18c0fc200 0x7fd18c0fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c0fc200 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.456+0000 7fd18a7fc700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd18c0fc200 0x7fd18c0fc5e0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.457+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c1072a0 con 0x7fd18c0fc200 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.457+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:42:20.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.457+0000 7fd183fff700 10 client.31704.objecter _finish_command 1 = system:0 2024-02-18T00:42:20.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.457+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c05ee80 osd=0 4 2024-02-18T00:42:20.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.457+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 0 1 2024-02-18T00:42:20.476 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 10 client.31704.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 2 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 2 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c05ee80 osd=0 3 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 0 2 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c05ee80 osd=0 4 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 10 client.31704.objecter _send_command 2 2024-02-18T00:42:20.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.472+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c0273c0 con 0x7fd18c0fc200 2024-02-18T00:42:20.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c0273c0 con 0x7fd18c0fc200 2024-02-18T00:42:20.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd183fff700 10 client.31704.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c05ee80 osd=0 4 2024-02-18T00:42:20.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 0 2 2024-02-18T00:42:20.479 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 10 client.31704.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 3 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6805/26279110 conn(0x7fd18c0598f0 0x7fd18c059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 3 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 1 3 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c028710 osd=1 4 2024-02-18T00:42:20.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 10 client.31704.objecter _send_command 3 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c121fa0 con 0x7fd18c0598f0 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6805/26279110 conn(0x7fd18c0598f0 0x7fd18c059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c0598f0 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.474+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6805/26279110 conn(0x7fd18c0598f0 0x7fd18c059cd0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.475+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c121fa0 con 0x7fd18c0598f0 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.475+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.475+0000 7fd183fff700 10 client.31704.objecter _finish_command 3 = system:0 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.475+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c028710 osd=1 4 2024-02-18T00:42:20.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.475+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 1 3 2024-02-18T00:42:20.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 10 client.31704.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 4 2024-02-18T00:42:20.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 4 2024-02-18T00:42:20.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c028710 osd=1 3 2024-02-18T00:42:20.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 1 4 2024-02-18T00:42:20.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c028710 osd=1 4 2024-02-18T00:42:20.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 10 client.31704.objecter _send_command 4 2024-02-18T00:42:20.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.490+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c059360 con 0x7fd18c0598f0 2024-02-18T00:42:20.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c059360 con 0x7fd18c0598f0 2024-02-18T00:42:20.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd183fff700 10 client.31704.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c028710 osd=1 4 2024-02-18T00:42:20.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 1 4 2024-02-18T00:42:20.494 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 10 client.31704.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 5 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd18c028270 0x7fd18c1209c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 5 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 2 5 2024-02-18T00:42:20.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c059770 osd=2 4 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 10 client.31704.objecter _send_command 5 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c120eb0 con 0x7fd18c028270 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.491+0000 7fd18a7fc700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd18c028270 0x7fd18c1209c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.492+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c028270 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.492+0000 7fd18a7fc700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd18c028270 0x7fd18c1209c0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.492+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c120eb0 con 0x7fd18c028270 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.492+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.492+0000 7fd183fff700 10 client.31704.objecter _finish_command 5 = system:0 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.492+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c059770 osd=2 4 2024-02-18T00:42:20.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.492+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 2 5 2024-02-18T00:42:20.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:18.691419+0000 osd.5 (osd.5) 259 : cluster [DBG] 5.f scrub starts 2024-02-18T00:42:20.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:18.694353+0000 osd.5 (osd.5) 260 : cluster [DBG] 5.f scrub ok 2024-02-18T00:42:20.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.188294+0000 osd.0 (osd.0) 272 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.189227+0000 osd.0 (osd.0) 273 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.756803+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:19.760405+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.793028+0000 osd.2 (osd.2) 241 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.794842+0000 osd.2 (osd.2) 242 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:18.691419+0000 osd.5 (osd.5) 259 : cluster [DBG] 5.f scrub starts 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:18.694353+0000 osd.5 (osd.5) 260 : cluster [DBG] 5.f scrub ok 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.188294+0000 osd.0 (osd.0) 272 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:42:20.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.189227+0000 osd.0 (osd.0) 273 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:42:20.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.756803+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-18T00:42:20.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:19.760405+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:20.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.793028+0000 osd.2 (osd.2) 241 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:42:20.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.794842+0000 osd.2 (osd.2) 242 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:42:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:18.691419+0000 osd.5 (osd.5) 259 : cluster [DBG] 5.f scrub starts 2024-02-18T00:42:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:18.694353+0000 osd.5 (osd.5) 260 : cluster [DBG] 5.f scrub ok 2024-02-18T00:42:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.188294+0000 osd.0 (osd.0) 272 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:42:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.189227+0000 osd.0 (osd.0) 273 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:42:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.756803+0000 mon.a (mon.0) 3094 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-18T00:42:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:19.760405+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.793028+0000 osd.2 (osd.2) 241 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:42:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.794842+0000 osd.2 (osd.2) 242 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:42:20.506 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 10 client.31704.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 6 2024-02-18T00:42:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 6 2024-02-18T00:42:20.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c059770 osd=2 3 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 2 6 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c059770 osd=2 4 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 10 client.31704.objecter _send_command 6 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.507+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c0277d0 con 0x7fd18c028270 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c0277d0 con 0x7fd18c028270 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd183fff700 10 client.31704.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c059770 osd=2 4 2024-02-18T00:42:20.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 2 6 2024-02-18T00:42:20.511 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 10 client.31704.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 7 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd18c124120 0x7fd18c126510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 7 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 3 7 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c027be0 osd=3 4 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 10 client.31704.objecter _send_command 7 2024-02-18T00:42:20.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c121db0 con 0x7fd18c124120 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd18b7fe700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd18c124120 0x7fd18c126510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c124120 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.509+0000 7fd18b7fe700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd18c124120 0x7fd18c126510 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.510+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c121db0 con 0x7fd18c124120 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.510+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.510+0000 7fd183fff700 10 client.31704.objecter _finish_command 7 = system:0 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.510+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c027be0 osd=3 4 2024-02-18T00:42:20.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.510+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 3 7 2024-02-18T00:42:20.526 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:20.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 10 client.31704.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 8 2024-02-18T00:42:20.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 8 2024-02-18T00:42:20.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c027be0 osd=3 3 2024-02-18T00:42:20.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 3 8 2024-02-18T00:42:20.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c027be0 osd=3 4 2024-02-18T00:42:20.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 10 client.31704.objecter _send_command 8 2024-02-18T00:42:20.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.525+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c121310 con 0x7fd18c124120 2024-02-18T00:42:20.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c121310 con 0x7fd18c124120 2024-02-18T00:42:20.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd183fff700 10 client.31704.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c027be0 osd=3 4 2024-02-18T00:42:20.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 3 8 2024-02-18T00:42:20.528 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 10 client.31704.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 9 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd18c120520 0x7fd18c129b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 9 2024-02-18T00:42:20.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 4 9 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c1210d0 osd=4 4 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 10 client.31704.objecter _send_command 9 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.526+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c121240 con 0x7fd18c120520 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.527+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd18c120520 0x7fd18c129b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.527+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c120520 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.527+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd18c120520 0x7fd18c129b90 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.528+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c121240 con 0x7fd18c120520 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.528+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.528+0000 7fd183fff700 10 client.31704.objecter _finish_command 9 = system:0 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.528+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c1210d0 osd=4 4 2024-02-18T00:42:20.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.528+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 4 9 2024-02-18T00:42:20.531 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:20.531 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:20.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 10 client.31704.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 10 2024-02-18T00:42:20.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 10 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c1210d0 osd=4 3 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 4 10 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c1210d0 osd=4 4 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 10 client.31704.objecter _send_command 10 2024-02-18T00:42:20.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.543+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c121240 con 0x7fd18c120520 2024-02-18T00:42:20.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.544+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c121240 con 0x7fd18c120520 2024-02-18T00:42:20.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.544+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.544+0000 7fd183fff700 10 client.31704.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.544+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c1210d0 osd=4 4 2024-02-18T00:42:20.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.544+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 4 10 2024-02-18T00:42:20.546 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:42:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 10 client.31704.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 11 2024-02-18T00:42:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd18c12a590 0x7fd18c12c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 11 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 5 11 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c121240 osd=5 4 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 10 client.31704.objecter _send_command 11 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c12f7b0 con 0x7fd18c12a590 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd18a7fc700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd18c12a590 0x7fd18c12c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c12a590 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.545+0000 7fd18a7fc700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd18c12a590 0x7fd18c12c9c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.546+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c12f7b0 con 0x7fd18c12a590 2024-02-18T00:42:20.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.546+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:42:20.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.546+0000 7fd183fff700 10 client.31704.objecter _finish_command 11 = system:0 2024-02-18T00:42:20.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.546+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c121240 osd=5 4 2024-02-18T00:42:20.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.546+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 5 11 2024-02-18T00:42:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 10 client.31704.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 12 2024-02-18T00:42:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 12 2024-02-18T00:42:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c121240 osd=5 3 2024-02-18T00:42:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 5 12 2024-02-18T00:42:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c121240 osd=5 4 2024-02-18T00:42:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 10 client.31704.objecter _send_command 12 2024-02-18T00:42:20.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.561+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c12f7b0 con 0x7fd18c12a590 2024-02-18T00:42:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.562+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c12f7b0 con 0x7fd18c12a590 2024-02-18T00:42:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.562+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.562+0000 7fd183fff700 10 client.31704.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.562+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c121240 osd=5 4 2024-02-18T00:42:20.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.562+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 5 12 2024-02-18T00:42:20.564 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 10 client.31704.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 13 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd18c12eaf0 0x7fd18c12ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 13 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 6 13 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12f7b0 osd=6 4 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 10 client.31704.objecter _send_command 13 2024-02-18T00:42:20.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c132b60 con 0x7fd18c12eaf0 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd18b7fe700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd18c12eaf0 0x7fd18c12ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c12eaf0 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.563+0000 7fd18b7fe700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd18c12eaf0 0x7fd18c12ef70 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.564+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c132b60 con 0x7fd18c12eaf0 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.564+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.564+0000 7fd183fff700 10 client.31704.objecter _finish_command 13 = system:0 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.564+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c12f7b0 osd=6 4 2024-02-18T00:42:20.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.564+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 6 13 2024-02-18T00:42:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 10 client.31704.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 14 2024-02-18T00:42:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 14 2024-02-18T00:42:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c12f7b0 osd=6 3 2024-02-18T00:42:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 6 14 2024-02-18T00:42:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12f7b0 osd=6 4 2024-02-18T00:42:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 10 client.31704.objecter _send_command 14 2024-02-18T00:42:20.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.578+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c12cfd0 con 0x7fd18c12eaf0 2024-02-18T00:42:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.580+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c12cfd0 con 0x7fd18c12eaf0 2024-02-18T00:42:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.580+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.580+0000 7fd183fff700 10 client.31704.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.580+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c12f7b0 osd=6 4 2024-02-18T00:42:20.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.580+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 6 14 2024-02-18T00:42:20.582 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:42:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 10 client.31704.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 15 2024-02-18T00:42:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6812/53025171 conn(0x7fd18c12d650 0x7fd18c12dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 15 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 7 15 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12cfd0 osd=7 4 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 10 client.31704.objecter _send_command 15 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd18c12e010 con 0x7fd18c12d650 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6812/53025171 conn(0x7fd18c12d650 0x7fd18c12dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd18affd700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6812/53025171 conn(0x7fd18c12d650 0x7fd18c12dab0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.581+0000 7fd183fff700 10 client.31704.objecter ms_handle_connect 0x7fd18c12d650 2024-02-18T00:42:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.582+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd18c12e010 con 0x7fd18c12d650 2024-02-18T00:42:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.582+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:42:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.582+0000 7fd183fff700 10 client.31704.objecter _finish_command 15 = system:0 2024-02-18T00:42:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.582+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c12cfd0 osd=7 4 2024-02-18T00:42:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.582+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 7 15 2024-02-18T00:42:20.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 10 client.31704.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:20.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign -1 16 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 20 client.31704.objecter _get_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_remove -1 16 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 20 client.31704.objecter get_session s=0x7fd18c12cfd0 osd=7 3 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 15 client.31704.objecter _session_command_op_assign 7 16 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12cfd0 osd=7 4 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 10 client.31704.objecter _send_command 16 2024-02-18T00:42:20.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.597+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fd18c12e010 con 0x7fd18c12d650 2024-02-18T00:42:20.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd183fff700 1 -- 172.21.15.59:0/1208558152 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fd18c12e010 con 0x7fd18c12d650 2024-02-18T00:42:20.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd183fff700 10 client.31704.objecter ms_dispatch 0x7fd18c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:20.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd183fff700 10 client.31704.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd183fff700 20 client.31704.objecter put_session s=0x7fd18c12cfd0 osd=7 4 2024-02-18T00:42:20.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd183fff700 15 client.31704.objecter _session_command_op_remove 7 16 2024-02-18T00:42:20.601 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:42:20.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 librados: watch_flush enter 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 librados: watch_flush exit 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.0 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd18c0fc200 msgr2=0x7fd18c0fc5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6801/3700238977 conn(0x7fd18c0fc200 0x7fd18c0fc5e0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c05ee80 osd=0 1 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.1 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6805/26279110 conn(0x7fd18c0598f0 msgr2=0x7fd18c059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6805/26279110 conn(0x7fd18c0598f0 0x7fd18c059cd0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c028710 osd=1 1 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.2 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd18c028270 msgr2=0x7fd18c1209c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6809/1153193637 conn(0x7fd18c028270 0x7fd18c1209c0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c059770 osd=2 1 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.3 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd18c124120 msgr2=0x7fd18c126510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd18c124120 0x7fd18c126510 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd18b7fe700 1 -- 172.21.15.59:0/1208558152 reap_dead start 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c027be0 osd=3 1 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.4 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd18c120520 msgr2=0x7fd18c129b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6800/2044387056 conn(0x7fd18c120520 0x7fd18c129b90 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c1210d0 osd=4 1 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.5 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd18b7fe700 1 -- 172.21.15.59:0/1208558152 reap_dead start 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd18c12a590 msgr2=0x7fd18c12c9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6804/2170432263 conn(0x7fd18c12a590 0x7fd18c12c9c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c121240 osd=5 1 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.6 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd18c12eaf0 msgr2=0x7fd18c12ef70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6808/3356009390 conn(0x7fd18c12eaf0 0x7fd18c12ef70 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12f7b0 osd=6 1 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter close_session for osd.7 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6812/53025171 conn(0x7fd18c12d650 msgr2=0x7fd18c12dab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.137:6812/53025171 conn(0x7fd18c12d650 0x7fd18c12dab0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter put_session s=0x7fd18c12cfd0 osd=7 1 2024-02-18T00:42:20.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 20 client.31704.objecter shutdown clearing up homeless session... 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 10 client.31704.objecter successfully canceled tick 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd178062c90 msgr2=0x7fd178065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd178062c90 0x7fd178065150 secure :-1 s=READY pgs=3172 cs=0 l=1 rev1=1 rx=0x7fd17c009350 tx=0x7fd17c00b040).stop 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd18c102270 msgr2=0x7fd18c0f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd1914ae700 1 --2- 172.21.15.59:0/1208558152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd18c102270 0x7fd18c0f76a0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7fd18400d970 tx=0x7fd184004e40).stop 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.599+0000 7fd18b7fe700 1 -- 172.21.15.59:0/1208558152 reap_dead start 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.600+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 shutdown_connections 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.600+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 >> 172.21.15.59:0/1208558152 conn(0x7fd18c0f2820 msgr2=0x7fd18c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:20.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.600+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 shutdown_connections 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.600+0000 7fd1914ae700 1 -- 172.21.15.59:0/1208558152 wait complete. 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:20.600+0000 7fd1914ae700 1 librados: shutdown 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.607 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:42:20.632 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:20.660 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:20.679 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:20.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:20.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:20.782 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:20.809 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:20.827 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:42:20.828 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:42:20.828 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:42:20.828 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:42:20.828 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:20.830 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:20.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:20.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:20.960 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:21.084 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:21.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:21.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:21.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:42:21.188 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:21.217 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:21.427 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.937265+0000 osd.3 (osd.3) 319 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.938178+0000 osd.3 (osd.3) 320 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:19.968288+0000 mgr.y (mgr.24454) 544 : cluster [DBG] pgmap v924: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:20.352512+0000 osd.4 (osd.4) 341 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:20.356198+0000 osd.4 (osd.4) 342 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:20.456468+0000 mon.a (mon.0) 3096 : audit [DBG] from='client.? 172.21.15.59:0/1208558152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:20.637635+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:20.638563+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:20.762214+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:21.428 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:20.762374+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-18T00:42:21.429 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:20.785439+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:21.467 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:21.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.937265+0000 osd.3 (osd.3) 319 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:42:21.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.938178+0000 osd.3 (osd.3) 320 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:19.968288+0000 mgr.y (mgr.24454) 544 : cluster [DBG] pgmap v924: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:20.352512+0000 osd.4 (osd.4) 341 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:20.356198+0000 osd.4 (osd.4) 342 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:20.456468+0000 mon.a (mon.0) 3096 : audit [DBG] from='client.? 172.21.15.59:0/1208558152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:20.637635+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:20.638563+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:20.762214+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:20.762374+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:20.785439+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.937265+0000 osd.3 (osd.3) 319 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:42:21.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.938178+0000 osd.3 (osd.3) 320 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:19.968288+0000 mgr.y (mgr.24454) 544 : cluster [DBG] pgmap v924: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:20.352512+0000 osd.4 (osd.4) 341 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:20.356198+0000 osd.4 (osd.4) 342 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:20.456468+0000 mon.a (mon.0) 3096 : audit [DBG] from='client.? 172.21.15.59:0/1208558152' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:20.637635+0000 osd.7 (osd.7) 171 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:20.638563+0000 osd.7 (osd.7) 172 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:20.762214+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:20.762374+0000 mon.a (mon.0) 3098 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-18T00:42:21.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:20.785439+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:21.471 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:21.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:21.572 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:21.626 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:21.628 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:21.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:21.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:21.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:42:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:42:21.758 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:22.409 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:22.412 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:22.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:22.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-18T00:42:22.494 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:22] "GET /metrics HTTP/1.1" 200 190826 "" "Prometheus/2.33.4" 2024-02-18T00:42:22.514 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:22.546 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:22.568 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:22.570 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:22.571 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:22.670 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:22.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:22.701 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:22.721 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:22.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:22.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:22.825 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:22.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:21.426544+0000 mgr.y (mgr.24454) 545 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:22.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:21.694285+0000 osd.2 (osd.2) 243 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:42:22.852 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:21.695685+0000 osd.2 (osd.2) 244 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:42:22.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:21.765201+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:22.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:21.765275+0000 mon.a (mon.0) 3101 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-18T00:42:22.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:21.768043+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-102"}]: dispatch 2024-02-18T00:42:22.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:21.968730+0000 mgr.y (mgr.24454) 546 : cluster [DBG] pgmap v927: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:42:22.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.361753+0000 osd.4 (osd.4) 343 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:42:22.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.362847+0000 osd.4 (osd.4) 344 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:42:22.854 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:22.873 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:22.876 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:22.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:22.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:23.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:21.426544+0000 mgr.y (mgr.24454) 545 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:23.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:21.694285+0000 osd.2 (osd.2) 243 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:42:23.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:21.695685+0000 osd.2 (osd.2) 244 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:42:23.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:21.765201+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:23.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:21.765275+0000 mon.a (mon.0) 3101 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-18T00:42:23.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:21.768043+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-102"}]: dispatch 2024-02-18T00:42:23.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:21.968730+0000 mgr.y (mgr.24454) 546 : cluster [DBG] pgmap v927: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.361753+0000 osd.4 (osd.4) 343 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.362847+0000 osd.4 (osd.4) 344 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:21.426544+0000 mgr.y (mgr.24454) 545 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:21.694285+0000 osd.2 (osd.2) 243 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:21.695685+0000 osd.2 (osd.2) 244 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:21.765201+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:21.765275+0000 mon.a (mon.0) 3101 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-18T00:42:23.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:21.768043+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-102"}]: dispatch 2024-02-18T00:42:23.232 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:21.968730+0000 mgr.y (mgr.24454) 546 : cluster [DBG] pgmap v927: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:42:23.232 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.361753+0000 osd.4 (osd.4) 343 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:42:23.232 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.362847+0000 osd.4 (osd.4) 344 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:42:23.256 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:23.259 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:23.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:23.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:23.392 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:23.416 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:23.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:23.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:23.548 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:23.797 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:23.800 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:23.801 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:23.902 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:23.929 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.177501+0000 osd.0 (osd.0) 274 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.177821+0000 osd.0 (osd.0) 275 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.761650+0000 mon.a (mon.0) 3103 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:22.768998+0000 mon.a (mon.0) 3104 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-102"}]': finished 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.769059+0000 mon.a (mon.0) 3105 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:22.796794+0000 mon.a (mon.0) 3106 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.875188+0000 osd.3 (osd.3) 321 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.876052+0000 osd.3 (osd.3) 322 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.963800+0000 osd.6 (osd.6) 281 : cluster [DBG] 5.2 deep-scrub starts 2024-02-18T00:42:24.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:22.964680+0000 osd.6 (osd.6) 282 : cluster [DBG] 5.2 deep-scrub ok 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:23.604108+0000 osd.7 (osd.7) 173 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:23.605262+0000 osd.7 (osd.7) 174 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.177501+0000 osd.0 (osd.0) 274 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.177821+0000 osd.0 (osd.0) 275 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.761650+0000 mon.a (mon.0) 3103 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:22.768998+0000 mon.a (mon.0) 3104 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-102"}]': finished 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.769059+0000 mon.a (mon.0) 3105 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:22.796794+0000 mon.a (mon.0) 3106 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:24.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.875188+0000 osd.3 (osd.3) 321 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.876052+0000 osd.3 (osd.3) 322 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.963800+0000 osd.6 (osd.6) 281 : cluster [DBG] 5.2 deep-scrub starts 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:22.964680+0000 osd.6 (osd.6) 282 : cluster [DBG] 5.2 deep-scrub ok 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:23.604108+0000 osd.7 (osd.7) 173 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:23.605262+0000 osd.7 (osd.7) 174 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.177501+0000 osd.0 (osd.0) 274 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.177821+0000 osd.0 (osd.0) 275 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.761650+0000 mon.a (mon.0) 3103 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:22.768998+0000 mon.a (mon.0) 3104 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-102"}]': finished 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.769059+0000 mon.a (mon.0) 3105 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:22.796794+0000 mon.a (mon.0) 3106 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.875188+0000 osd.3 (osd.3) 321 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.876052+0000 osd.3 (osd.3) 322 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.963800+0000 osd.6 (osd.6) 281 : cluster [DBG] 5.2 deep-scrub starts 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:22.964680+0000 osd.6 (osd.6) 282 : cluster [DBG] 5.2 deep-scrub ok 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:23.604108+0000 osd.7 (osd.7) 173 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:42:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:23.605262+0000 osd.7 (osd.7) 174 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:42:24.661 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:42:24.678 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:24.681 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:24.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:24.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:25.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:23.781053+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:23.781126+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:23.782803+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102"}]: dispatch 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:23.969223+0000 mgr.y (mgr.24454) 547 : cluster [DBG] pgmap v930: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:24.596578+0000 osd.7 (osd.7) 175 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:24.598253+0000 osd.7 (osd.7) 176 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:23.781053+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:23.781126+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:23.782803+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102"}]: dispatch 2024-02-18T00:42:25.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:23.969223+0000 mgr.y (mgr.24454) 547 : cluster [DBG] pgmap v930: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:42:25.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:24.596578+0000 osd.7 (osd.7) 175 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:42:25.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:24.598253+0000 osd.7 (osd.7) 176 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:42:25.171 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:25.175 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:25.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:25.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:25.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:23.781053+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:23.781126+0000 mon.a (mon.0) 3108 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-18T00:42:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:23.782803+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102"}]: dispatch 2024-02-18T00:42:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:23.969223+0000 mgr.y (mgr.24454) 547 : cluster [DBG] pgmap v930: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:42:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:24.596578+0000 osd.7 (osd.7) 175 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:42:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:24.598253+0000 osd.7 (osd.7) 176 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:42:25.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:25.308 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:25.332 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:25.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:25.433 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:25.463 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:25.484 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:25.487 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:25.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:25.588 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:25.637 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:25.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:25.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:25.741 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:25.772 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:25.848 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:25.851 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:25.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:25.953 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:25.980 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:24.785399+0000 mon.a (mon.0) 3110 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102"}]': finished 2024-02-18T00:42:25.980 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:24.785476+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-18T00:42:25.980 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:24.906120+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:25.980 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.356854+0000 osd.4 (osd.4) 345 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:42:25.980 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.359759+0000 osd.4 (osd.4) 346 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:42:25.980 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.582822+0000 osd.7 (osd.7) 177 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:42:25.980 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.583817+0000 osd.7 (osd.7) 178 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:42:25.981 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:26.001 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:26.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:26.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:26.105 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:26.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 -- 172.21.15.59:0/4199220275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe754102210 msgr2=0x7fe754104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 --2- 172.21.15.59:0/4199220275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe754102210 0x7fe754104600 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 rx=0x7fe74c009be0 tx=0x7fe74c004d10).stop 2024-02-18T00:42:26.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 -- 172.21.15.59:0/4199220275 shutdown_connections 2024-02-18T00:42:26.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 --2- 172.21.15.59:0/4199220275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7540f8cb0 0x7fe7540f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 --2- 172.21.15.59:0/4199220275 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe754102210 0x7fe754104600 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 --2- 172.21.15.59:0/4199220275 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 0x7fe754101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 -- 172.21.15.59:0/4199220275 >> 172.21.15.59:0/4199220275 conn(0x7fe7540f2820 msgr2=0x7fe7540f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 -- 172.21.15.59:0/4199220275 shutdown_connections 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.071+0000 7fe758d71700 1 -- 172.21.15.59:0/4199220275 wait complete. 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 librados: starting msgr at 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 librados: starting objecter 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 Processor -- start 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 -- start start 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 librados: setting wanted keys 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 librados: calling monclient init 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7540f8cb0 0x7fe75410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 0x7fe75410e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe754102210 0x7fe754108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7541086f0 con 0x7fe7540f9610 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754108860 con 0x7fe754102210 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.072+0000 7fe758d71700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7541089d0 con 0x7fe7540f8cb0 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 0x7fe75410e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 0x7fe75410e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60876/0 (socket says 172.21.15.59:60876) 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 -- 172.21.15.59:0/1700865388 learned_addr learned my addr 172.21.15.59:0/1700865388 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe752d9d700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe754102210 0x7fe754108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe75259c700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7540f8cb0 0x7fe75410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 -- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7540f8cb0 msgr2=0x7fe75410e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7540f8cb0 0x7fe75410e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 -- 172.21.15.59:0/1700865388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe754102210 msgr2=0x7fe754108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe754102210 0x7fe754108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 -- 172.21.15.59:0/1700865388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c0097c0 con 0x7fe7540f9610 2024-02-18T00:42:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe751d9b700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 0x7fe75410e510 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7fe74800e970 tx=0x7fe74800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe7437fe700 10 client.?.objecter ms_handle_connect 0x7fe7540f9610 2024-02-18T00:42:26.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.073+0000 7fe7437fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:26.134 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:26.153 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:26.155 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:26.155 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:26.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.246+0000 7fe7437fe700 1 -- 172.21.15.59:0/1700865388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748004660 con 0x7fe7540f9610 2024-02-18T00:42:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.246+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe754108cb0 con 0x7fe7540f9610 2024-02-18T00:42:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.246+0000 7fe7437fe700 1 -- 172.21.15.59:0/1700865388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe74800f040 con 0x7fe7540f9610 2024-02-18T00:42:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.246+0000 7fe7437fe700 1 -- 172.21.15.59:0/1700865388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748014790 con 0x7fe7540f9610 2024-02-18T00:42:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.246+0000 7fe758d71700 10 client.31710.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.246+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe754114e90 con 0x7fe7540f9610 2024-02-18T00:42:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.246+0000 7fe758d71700 1 librados: init done 2024-02-18T00:42:26.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.247+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75405fa60 con 0x7fe7540f9610 2024-02-18T00:42:26.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.247+0000 7fe7437fe700 1 -- 172.21.15.59:0/1700865388 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe7480090d0 con 0x7fe7540f9610 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe7437fe700 1 --2- 172.21.15.59:0/1700865388 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe73c062c90 0x7fe73c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe7437fe700 1 -- 172.21.15.59:0/1700865388 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(612..612 src has 418..612) v4 ==== 8226+0+0 (secure 0 0 0) 0x7fe74807d140 con 0x7fe7540f9610 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe7437fe700 10 client.31710.objecter ms_dispatch 0x7fe754001550 osd_map(612..612 src has 418..612) v4 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe7437fe700 3 client.31710.objecter handle_osd_map got epochs [612,612] > 0 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe7437fe700 3 client.31710.objecter handle_osd_map decoding full epoch 612 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe75259c700 1 --2- 172.21.15.59:0/1700865388 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe73c062c90 0x7fe73c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe7437fe700 20 client.31710.objecter dump_active .. 0 homeless 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe7437fe700 10 client.31710.objecter ms_handle_connect 0x7fe73c062c90 2024-02-18T00:42:26.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.248+0000 7fe75259c700 1 --2- 172.21.15.59:0/1700865388 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe73c062c90 0x7fe73c065150 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 rx=0x7fe744004dc0 tx=0x7fe74400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:26.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:24.785399+0000 mon.a (mon.0) 3110 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102"}]': finished 2024-02-18T00:42:26.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:24.785476+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-18T00:42:26.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:24.906120+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:26.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.356854+0000 osd.4 (osd.4) 345 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.359759+0000 osd.4 (osd.4) 346 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.582822+0000 osd.7 (osd.7) 177 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.583817+0000 osd.7 (osd.7) 178 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:24.785399+0000 mon.a (mon.0) 3110 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-102"}]': finished 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:24.785476+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:24.906120+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.356854+0000 osd.4 (osd.4) 345 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.359759+0000 osd.4 (osd.4) 346 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.582822+0000 osd.7 (osd.7) 177 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:42:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.583817+0000 osd.7 (osd.7) 178 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:42:26.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.252+0000 7fe7437fe700 1 -- 172.21.15.59:0/1700865388 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe748050690 con 0x7fe7540f9610 2024-02-18T00:42:26.256 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:26.287 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:26.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.441+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fe75405ee80 con 0x7fe7540f9610 2024-02-18T00:42:26.486 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:26.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:26.489 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:26.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:26.626 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:26.645 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:26.648 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:26.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:26.750 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:26.780 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:26.802 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:26.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.805+0000 7fe7437fe700 1 -- 172.21.15.59:0/1700865388 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v613) v1 ==== 87+0+0 (secure 0 0 0) 0x7fe748019070 con 0x7fe7540f9610 2024-02-18T00:42:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.807+0000 7fe758d71700 10 librados: watch_flush enter 2024-02-18T00:42:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.807+0000 7fe758d71700 10 librados: watch_flush exit 2024-02-18T00:42:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.807+0000 7fe758d71700 20 client.31710.objecter shutdown clearing up homeless session... 2024-02-18T00:42:26.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.807+0000 7fe758d71700 10 client.31710.objecter successfully canceled tick 2024-02-18T00:42:26.809 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:26.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.810+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe73c062c90 msgr2=0x7fe73c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.810+0000 7fe758d71700 1 --2- 172.21.15.59:0/1700865388 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe73c062c90 0x7fe73c065150 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 rx=0x7fe744004dc0 tx=0x7fe74400b040).stop 2024-02-18T00:42:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.810+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 msgr2=0x7fe75410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.810+0000 7fe758d71700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 0x7fe75410e510 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7fe74800e970 tx=0x7fe74800b040).stop 2024-02-18T00:42:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 shutdown_connections 2024-02-18T00:42:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 --2- 172.21.15.59:0/1700865388 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe73c062c90 0x7fe73c065150 unknown :-1 s=CLOSED pgs=3173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe754102210 0x7fe754108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7540f8cb0 0x7fe75410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 --2- 172.21.15.59:0/1700865388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7540f9610 0x7fe75410e510 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 >> 172.21.15.59:0/1700865388 conn(0x7fe7540f2820 msgr2=0x7fe7540f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:26.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 shutdown_connections 2024-02-18T00:42:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 -- 172.21.15.59:0/1700865388 wait complete. 2024-02-18T00:42:26.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.811+0000 7fe758d71700 1 librados: shutdown 2024-02-18T00:42:26.815 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:42:26.911 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:26.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 -- 172.21.15.59:0/515779751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 msgr2=0x7f94080fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 --2- 172.21.15.59:0/515779751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 0x7f94080fdf90 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f93f80047b0 tx=0x7f93f8009cb0).stop 2024-02-18T00:42:26.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 -- 172.21.15.59:0/515779751 shutdown_connections 2024-02-18T00:42:26.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 --2- 172.21.15.59:0/515779751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94080f8420 0x7f9408104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 --2- 172.21.15.59:0/515779751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 0x7f94080fdf90 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 --2- 172.21.15.59:0/515779751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 0x7f94080f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 -- 172.21.15.59:0/515779751 >> 172.21.15.59:0/515779751 conn(0x7f94080f3800 msgr2=0x7f94080f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:26.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 -- 172.21.15.59:0/515779751 shutdown_connections 2024-02-18T00:42:26.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.923+0000 7f940dc7e700 1 -- 172.21.15.59:0/515779751 wait complete. 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.924+0000 7f940dc7e700 1 librados: starting msgr at 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.924+0000 7f940dc7e700 1 librados: starting objecter 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.924+0000 7f940dc7e700 1 Processor -- start 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.924+0000 7f940dc7e700 1 -- start start 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.924+0000 7f940dc7e700 1 librados: setting wanted keys 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.924+0000 7f940dc7e700 1 librados: calling monclient init 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f940dc7e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94080f8420 0x7f940810e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f940dc7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 0x7f940810e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f940dc7e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 0x7f9408108270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f940dc7e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9408108760 con 0x7f94080fdbb0 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f940dc7e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94081088d0 con 0x7f94080f8420 2024-02-18T00:42:26.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f940dc7e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9408108a90 con 0x7f94080fe510 2024-02-18T00:42:26.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f94077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94080f8420 0x7f940810e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 0x7f9408108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 0x7f9408108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:40738/0 (socket says 172.21.15.59:40738) 2024-02-18T00:42:26.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f94077fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94080f8420 0x7f940810e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.59:54430/0 (socket says 172.21.15.59:54430) 2024-02-18T00:42:26.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 -- 172.21.15.59:0/2534940579 learned_addr learned my addr 172.21.15.59:0/2534940579 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:26.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9406ffd700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 0x7f940810e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 -- 172.21.15.59:0/2534940579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94080f8420 msgr2=0x7f940810e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94080f8420 0x7f940810e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 -- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 msgr2=0x7f940810e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 0x7f940810e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 -- 172.21.15.59:0/2534940579 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc005040 con 0x7f94080fe510 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9406ffd700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 0x7f940810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9407fff700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 0x7f9408108270 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f940000ec20 tx=0x7f940000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9404ff9700 10 client.?.objecter ms_handle_connect 0x7f94080fe510 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.925+0000 7f9404ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.926+0000 7f9404ff9700 1 -- 172.21.15.59:0/2534940579 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400009950 con 0x7f94080fe510 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.926+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f8005040 con 0x7f94080fe510 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.926+0000 7f9404ff9700 1 -- 172.21.15.59:0/2534940579 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f940000f040 con 0x7f94080fe510 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.926+0000 7f9404ff9700 1 -- 172.21.15.59:0/2534940579 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400007df0 con 0x7f94080fe510 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.926+0000 7f940dc7e700 10 client.41522.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:26.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.926+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9408109040 con 0x7f94080fe510 2024-02-18T00:42:26.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.926+0000 7f940dc7e700 1 librados: init done 2024-02-18T00:42:26.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940805ee80 con 0x7f94080fe510 2024-02-18T00:42:26.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f9404ff9700 1 -- 172.21.15.59:0/2534940579 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f94000092e0 con 0x7f94080fe510 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f9404ff9700 1 --2- 172.21.15.59:0/2534940579 >> v2:172.21.15.59:6800/3721891797 conn(0x7f93f0062c90 0x7f93f0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f9404ff9700 1 -- 172.21.15.59:0/2534940579 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(613..613 src has 418..613) v4 ==== 8599+0+0 (secure 0 0 0) 0x7f940007d100 con 0x7f94080fe510 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f9404ff9700 10 client.41522.objecter ms_dispatch 0x7f9408001550 osd_map(613..613 src has 418..613) v4 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f9404ff9700 3 client.41522.objecter handle_osd_map got epochs [613,613] > 0 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f9404ff9700 3 client.41522.objecter handle_osd_map decoding full epoch 613 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f94077fe700 1 --2- 172.21.15.59:0/2534940579 >> v2:172.21.15.59:6800/3721891797 conn(0x7f93f0062c90 0x7f93f0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.927+0000 7f9404ff9700 20 client.41522.objecter dump_active .. 0 homeless 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.928+0000 7f9404ff9700 10 client.41522.objecter ms_handle_connect 0x7f93f0062c90 2024-02-18T00:42:26.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.928+0000 7f94077fe700 1 --2- 172.21.15.59:0/2534940579 >> v2:172.21.15.59:6800/3721891797 conn(0x7f93f0062c90 0x7f93f0065150 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 rx=0x7f93f8004e20 tx=0x7f93f8007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:26.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:26.932+0000 7f9404ff9700 1 -- 172.21.15.59:0/2534940579 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9400049f60 con 0x7f94080fe510 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.664509+0000 osd.5 (osd.5) 261 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.667413+0000 osd.5 (osd.5) 262 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.769187+0000 osd.2 (osd.2) 245 : cluster [DBG] 2.21 deep-scrub starts 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.770124+0000 osd.2 (osd.2) 246 : cluster [DBG] 2.21 deep-scrub ok 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.796273+0000 mon.a (mon.0) 3113 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:25.969810+0000 mgr.y (mgr.24454) 548 : cluster [DBG] pgmap v933: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:26.442569+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? 172.21.15.59:0/1700865388' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:26.621937+0000 osd.7 (osd.7) 179 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:42:26.937 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:26.623069+0000 osd.7 (osd.7) 180 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:42:26.939 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:26.958 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:26.960 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:26.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.664509+0000 osd.5 (osd.5) 261 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:42:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.667413+0000 osd.5 (osd.5) 262 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:42:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.769187+0000 osd.2 (osd.2) 245 : cluster [DBG] 2.21 deep-scrub starts 2024-02-18T00:42:27.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.770124+0000 osd.2 (osd.2) 246 : cluster [DBG] 2.21 deep-scrub ok 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.796273+0000 mon.a (mon.0) 3113 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:25.969810+0000 mgr.y (mgr.24454) 548 : cluster [DBG] pgmap v933: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:26.442569+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? 172.21.15.59:0/1700865388' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:26.621937+0000 osd.7 (osd.7) 179 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:26.623069+0000 osd.7 (osd.7) 180 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.664509+0000 osd.5 (osd.5) 261 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.667413+0000 osd.5 (osd.5) 262 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.769187+0000 osd.2 (osd.2) 245 : cluster [DBG] 2.21 deep-scrub starts 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.770124+0000 osd.2 (osd.2) 246 : cluster [DBG] 2.21 deep-scrub ok 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.796273+0000 mon.a (mon.0) 3113 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-18T00:42:27.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:25.969810+0000 mgr.y (mgr.24454) 548 : cluster [DBG] pgmap v933: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:42:27.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:26.442569+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? 172.21.15.59:0/1700865388' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:42:27.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:26.621937+0000 osd.7 (osd.7) 179 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:42:27.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:26.623069+0000 osd.7 (osd.7) 180 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:42:27.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:27.091 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:27.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.113+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9408109860 con 0x7f94080fe510 2024-02-18T00:42:27.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.116+0000 7f9404ff9700 1 -- 172.21.15.59:0/2534940579 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v613) v1 ==== 81+0+49382 (secure 0 0 0) 0x7f94000503b0 con 0x7f94080fe510 2024-02-18T00:42:27.118 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:42:27.118 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:42:27.118 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 613, 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-02-18T00:23:11.390869+0000", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-02-18T00:42:26.791563+0000", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-02-18T00:29:06.503960+0000", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-02-18T00:39:35.702780+0000", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5801984, 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "noscrub", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-02-18T00:42:27.119 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 104, 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 236, 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 8, 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "pacific", 2024-02-18T00:42:27.120 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "device_health_metrics", 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:27:02.909667+0000", 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.121 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.122 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "67", 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.124 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:42:27.125 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_devicehealth": {} 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".rgw.root", 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:13.697458+0000", 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.126 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 42, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 42, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 42, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 42, 2024-02-18T00:42:27.127 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 42, 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "170", 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "107", 2024-02-18T00:42:27.128 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.129 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.130 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "iscsi", 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:14.132992+0000", 2024-02-18T00:42:27.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 3, 2024-02-18T00:42:27.132 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 3, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 3, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 3, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 3, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "60", 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.133 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.134 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.135 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 4, 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.log", 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:15.191206+0000", 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.137 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.138 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "60", 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.140 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.141 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 5, 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.control", 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:17.526023+0000", 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.142 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.143 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "62", 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.144 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.145 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.146 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 6, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.meta", 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:19.600478+0000", 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.147 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 22, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 22, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 22, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 22, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 22, 2024-02-18T00:42:27.148 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "6.16", 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 587, 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 586, 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 586, 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "587", 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "587", 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "585", 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.149 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.150 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.151 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_bias": 4 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 25, 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "ceph_test_rados_api_asio", 2024-02-18T00:42:27.152 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:32:59.546190+0000", 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:42:27.153 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "111", 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:42:27.154 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.155 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.156 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 234, 2024-02-18T00:42:27.157 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-101", 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:42:18.224666+0000", 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 3, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 1, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.158 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "611", 2024-02-18T00:42:27.159 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "610", 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "610", 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "610", 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 608, 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.160 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "testprofile-test-rados-api-smithi059-82957-101", 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.161 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 8192, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.162 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 236, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-104", 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:42:26.246265+0000", 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 32769, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,creating", 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:42:27.163 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.164 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "613", 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:42:27.165 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:42:27.166 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-02-18T00:42:27.167 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85", 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 8, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.168 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:42:27.169 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.170 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/3700238977", 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6802/3700238977", 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6804/3700238977", 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6803/3700238977", 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.171 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0", 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 13, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.172 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6805/26279110", 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6806/26279110", 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6808/26279110", 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6807/26279110", 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.175 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7", 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.176 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 18, 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.177 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:42:27.178 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1153193637", 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6810/1153193637", 2024-02-18T00:42:27.179 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6812/1153193637", 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6811/1153193637", 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b", 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.180 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 26, 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-02-18T00:42:27.182 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6813/1805700964", 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6814/1805700964", 2024-02-18T00:42:27.183 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6816/1805700964", 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6815/1805700964", 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231", 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.184 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 31, 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6800", 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6801", 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.186 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6803", 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6802", 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.187 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6800/2044387056", 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6801/2044387056", 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6803/2044387056", 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6802/2044387056", 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01", 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.188 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 39, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6804", 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:42:27.189 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6805", 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.190 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6807", 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6806", 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6804/2170432263", 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6805/2170432263", 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6807/2170432263", 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6806/2170432263", 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f", 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.192 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 47, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6808", 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:42:27.193 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6809", 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.194 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6811", 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6810", 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6808/3356009390", 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6809/3356009390", 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6811/3356009390", 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6810/3356009390", 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb", 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:42:27.196 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 53, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 606, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6812", 2024-02-18T00:42:27.197 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6813", 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.198 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6815", 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6814", 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.199 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6812/53025171", 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6813/53025171", 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6815/53025171", 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6814/53025171", 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.200 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:22.177823+0000", 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.201 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:41:44.780249+0000", 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:00.888490+0000", 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:17.756804+0000", 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.203 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:17.368058+0000", 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:41:12.379645+0000", 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:15.350878+0000", 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:42:27.205 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:08.193646+0000", 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [], 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [ 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.0", 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.3", 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.207 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.5", 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.6", 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 6, 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 7 2024-02-18T00:42:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 4, 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.8", 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.209 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.18", 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.2", 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.4", 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.211 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.b", 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.e", 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:42:27.212 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 0 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.f", 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.213 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.17", 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "5.1b", 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 6 2024-02-18T00:42:27.214 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.9", 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 4 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.11", 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:42:27.215 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": { 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/1243102711": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/817089002": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3174805070": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/3506000666": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:42:27.216 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/4167740619": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/999852549": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3575161033": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/2403979355": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/642821245": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/1656814063": "2024-02-19T00:23:45.307159+0000" 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-02-18T00:42:27.217 INFO:teuthology.orchestra.run.smithi059.stdout: "ruleset-failure-domain": "osd", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "testprofile-test-rados-api-smithi059-82957-101": { 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-device-class": "", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-root": "default", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "jerasure-per-chunk-alignment": "false", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "ruleset-failure-domain": "osd", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van", 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: "w": "8" 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [ 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 11, 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 3, 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:42:27.219 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 14, 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 9, 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 60, 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 78, 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.221 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 86, 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.222 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 94, 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 97, 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.223 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 4, 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 4 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 108, 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 114, 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 128, 2024-02-18T00:42:27.225 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 10, 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 134, 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 12, 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 154, 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.227 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 14, 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 166, 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 17, 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:42:27.228 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 177, 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 20, 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-02-18T00:42:27.229 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-02-18T00:42:27.230 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-02-18T00:42:27.230 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-02-18T00:42:27.230 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-02-18T00:42:27.230 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-02-18T00:42:27.230 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-02-18T00:42:27.230 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:42:27.230 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.117+0000 7f940dc7e700 10 librados: watch_flush enter 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 10 librados: watch_flush exit 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 20 client.41522.objecter shutdown clearing up homeless session... 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 10 client.41522.objecter successfully canceled tick 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 >> v2:172.21.15.59:6800/3721891797 conn(0x7f93f0062c90 msgr2=0x7f93f0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 --2- 172.21.15.59:0/2534940579 >> v2:172.21.15.59:6800/3721891797 conn(0x7f93f0062c90 0x7f93f0065150 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 rx=0x7f93f8004e20 tx=0x7f93f8007f90).stop 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 msgr2=0x7f9408108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:27.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 0x7f9408108270 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 rx=0x7f940000ec20 tx=0x7f940000b040).stop 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 shutdown_connections 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 --2- 172.21.15.59:0/2534940579 >> v2:172.21.15.59:6800/3721891797 conn(0x7f93f0062c90 0x7f93f0065150 unknown :-1 s=CLOSED pgs=3174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94080f8420 0x7f940810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94080fe510 0x7f9408108270 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 --2- 172.21.15.59:0/2534940579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94080fdbb0 0x7f940810e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 >> 172.21.15.59:0/2534940579 conn(0x7f94080f3800 msgr2=0x7f94080f5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 shutdown_connections 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 -- 172.21.15.59:0/2534940579 wait complete. 2024-02-18T00:42:27.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:27.118+0000 7f940dc7e700 1 librados: shutdown 2024-02-18T00:42:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:27.650 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:27.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:27.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:27.782 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:28.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:26.792004+0000 mon.a (mon.0) 3115 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:26.803016+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? 172.21.15.59:0/1700865388' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:26.803068+0000 mon.a (mon.0) 3117 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:26.806693+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:27.115769+0000 mon.c (mon.2) 552 : audit [DBG] from='client.? 172.21.15.59:0/2534940579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:26.792004+0000 mon.a (mon.0) 3115 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:26.803016+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? 172.21.15.59:0/1700865388' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:26.803068+0000 mon.a (mon.0) 3117 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:26.806693+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:28.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:27.115769+0000 mon.c (mon.2) 552 : audit [DBG] from='client.? 172.21.15.59:0/2534940579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:42:28.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:26.792004+0000 mon.a (mon.0) 3115 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:28.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:26.803016+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? 172.21.15.59:0/1700865388' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:42:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:26.803068+0000 mon.a (mon.0) 3117 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-18T00:42:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:26.806693+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:27.115769+0000 mon.c (mon.2) 552 : audit [DBG] from='client.? 172.21.15.59:0/2534940579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:27.808564+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:27.808731+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:27.830023+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:27.970361+0000 mgr.y (mgr.24454) 549 : cluster [DBG] pgmap v936: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:28.765435+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:28.765518+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:28.768754+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-104"}]: dispatch 2024-02-18T00:42:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:27.808564+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:27.808731+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-18T00:42:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:27.830023+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:27.970361+0000 mgr.y (mgr.24454) 549 : cluster [DBG] pgmap v936: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:42:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:28.765435+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:28.765518+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-18T00:42:29.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:28.768754+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-104"}]: dispatch 2024-02-18T00:42:29.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:27.808564+0000 mon.a (mon.0) 3119 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:27.808731+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-18T00:42:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:27.830023+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:27.970361+0000 mgr.y (mgr.24454) 549 : cluster [DBG] pgmap v936: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T00:42:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:28.765435+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:28.765518+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-18T00:42:29.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:28.768754+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-104"}]: dispatch 2024-02-18T00:42:29.272 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:42:29.272 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:42:29.272 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:42:29.273 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:42:29.273 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:42:29.667 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:30.147 INFO:tasks.thrashosds.thrasher:j is {'epoch': 613, 'fsid': 'd129b4f8-cdf3-11ee-95ba-87774f69a715', 'created': '2024-02-18T00:23:11.390869+0000', 'modified': '2024-02-18T00:42:26.791563+0000', 'last_up_change': '2024-02-18T00:29:06.503960+0000', 'last_in_change': '2024-02-18T00:39:35.702780+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 104, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 236, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-18T00:27:02.909667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-02-18T00:29:13.697458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 42, 'pg_placement_num': 42, 'pg_placement_num_target': 42, 'pg_num_target': 42, 'pg_num_pending': 42, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '107', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'iscsi', 'create_time': '2024-02-18T00:29:14.132992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-18T00:29:15.191206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-18T00:29:17.526023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-18T00:29:19.600478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 22, 'pg_placement_num': 22, 'pg_placement_num_target': 22, 'pg_num_target': 22, 'pg_num_pending': 22, 'last_pg_merge_meta': {'source_pgid': '6.16', 'ready_epoch': 587, 'last_epoch_started': 586, 'last_epoch_clean': 586, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '587', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '587', 'last_force_op_resend_preluminous': '585', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 25, 'pool_name': 'ceph_test_rados_api_asio', 'create_time': '2024-02-18T00:32:59.546190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 234, 'pool_name': 'test-rados-api-smithi059-82957-101', 'create_time': '2024-02-18T00:42:18.224666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '611', 'last_force_op_resend': '610', 'last_force_op_resend_prenautilus': '610', 'last_force_op_resend_preluminous': '610', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 608, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'testprofile-test-rados-api-smithi059-82957-101', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 236, 'pool_name': 'test-rados-api-smithi059-82957-104', 'create_time': '2024-02-18T00:42:26.246265+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '613', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}], 'osds': [{'osd': 0, 'uuid': '6e729102-80e2-4a4e-92ca-600ee8b1ed85', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6801', 'nonce': 3700238977}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 3700238977}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 3700238977}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6803', 'nonce': 3700238977}]}, 'public_addr': '172.21.15.59:6801/3700238977', 'cluster_addr': '172.21.15.59:6802/3700238977', 'heartbeat_back_addr': '172.21.15.59:6804/3700238977', 'heartbeat_front_addr': '172.21.15.59:6803/3700238977', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f67a7206-6d31-497f-97e0-672d134b13d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6805', 'nonce': 26279110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 26279110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 26279110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6807', 'nonce': 26279110}]}, 'public_addr': '172.21.15.59:6805/26279110', 'cluster_addr': '172.21.15.59:6806/26279110', 'heartbeat_back_addr': '172.21.15.59:6808/26279110', 'heartbeat_front_addr': '172.21.15.59:6807/26279110', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6223dbd-e539-4770-9e5a-7b756304f8f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6809', 'nonce': 1153193637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1153193637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1153193637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6811', 'nonce': 1153193637}]}, 'public_addr': '172.21.15.59:6809/1153193637', 'cluster_addr': '172.21.15.59:6810/1153193637', 'heartbeat_back_addr': '172.21.15.59:6812/1153193637', 'heartbeat_front_addr': '172.21.15.59:6811/1153193637', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fff275df-95e0-41f6-96c3-e531f05de32b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6813', 'nonce': 1805700964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1805700964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 1805700964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6815', 'nonce': 1805700964}]}, 'public_addr': '172.21.15.59:6813/1805700964', 'cluster_addr': '172.21.15.59:6814/1805700964', 'heartbeat_back_addr': '172.21.15.59:6816/1805700964', 'heartbeat_front_addr': '172.21.15.59:6815/1805700964', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a0bae50b-4f29-4310-ac5f-bc7285464231', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6800', 'nonce': 2044387056}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6801', 'nonce': 2044387056}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6803', 'nonce': 2044387056}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6802', 'nonce': 2044387056}]}, 'public_addr': '172.21.15.137:6800/2044387056', 'cluster_addr': '172.21.15.137:6801/2044387056', 'heartbeat_back_addr': '172.21.15.137:6803/2044387056', 'heartbeat_front_addr': '172.21.15.137:6802/2044387056', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '513a4192-45fb-4444-a1f9-4bd2f2a5fa01', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6804', 'nonce': 2170432263}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6805', 'nonce': 2170432263}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6807', 'nonce': 2170432263}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6806', 'nonce': 2170432263}]}, 'public_addr': '172.21.15.137:6804/2170432263', 'cluster_addr': '172.21.15.137:6805/2170432263', 'heartbeat_back_addr': '172.21.15.137:6807/2170432263', 'heartbeat_front_addr': '172.21.15.137:6806/2170432263', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5bbb7683-ea5a-490c-8c8b-67b53c45406f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6808', 'nonce': 3356009390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6809', 'nonce': 3356009390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6811', 'nonce': 3356009390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6810', 'nonce': 3356009390}]}, 'public_addr': '172.21.15.137:6808/3356009390', 'cluster_addr': '172.21.15.137:6809/3356009390', 'heartbeat_back_addr': '172.21.15.137:6811/3356009390', 'heartbeat_front_addr': '172.21.15.137:6810/3356009390', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '951d317b-7a34-41a0-b1a1-a2851b2070bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 606, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6812', 'nonce': 53025171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6813', 'nonce': 53025171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6815', 'nonce': 53025171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6814', 'nonce': 53025171}]}, 'public_addr': '172.21.15.137:6812/53025171', 'cluster_addr': '172.21.15.137:6813/53025171', 'heartbeat_back_addr': '172.21.15.137:6815/53025171', 'heartbeat_front_addr': '172.21.15.137:6814/53025171', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:22.177823+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:41:44.780249+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:00.888490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:17.756804+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:17.368058+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:41:12.379645+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:15.350878+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:08.193646+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.3', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '2.5', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.6', 'mappings': [{'from': 6, 'to': 7}, {'from': 4, 'to': 1}]}, {'pgid': '2.8', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.18', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.2', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.4', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 0}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.1b', 'mappings': [{'from': 0, 'to': 6}]}, {'pgid': '25.9', 'mappings': [{'from': 0, 'to': 4}]}, {'pgid': '25.11', 'mappings': [{'from': 0, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.59:0/1243102711': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/817089002': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/3174805070': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/3506000666': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/4167740619': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/999852549': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/3575161033': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:0/2403979355': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/642821245': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:6800/1656814063': '2024-02-19T00:23:45.307159+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}, 'testprofile-test-rados-api-smithi059-82957-101': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van', 'w': '8'}}, 'removed_snaps_queue': [{'pool': 11, 'snaps': [{'begin': 3, 'length': 1}, {'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 14, 'snaps': [{'begin': 9, 'length': 2}]}, {'pool': 60, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 78, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 86, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 94, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 97, 'snaps': [{'begin': 4, 'length': 4}]}, {'pool': 108, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 114, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 128, 'snaps': [{'begin': 2, 'length': 2}, {'begin': 6, 'length': 2}, {'begin': 10, 'length': 2}]}, {'pool': 134, 'snaps': [{'begin': 12, 'length': 1}]}, {'pool': 154, 'snaps': [{'begin': 14, 'length': 1}]}, {'pool': 166, 'snaps': [{'begin': 17, 'length': 1}]}, {'pool': 177, 'snaps': [{'begin': 20, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-18T00:42:30.148 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 25.11 2024-02-18T00:42:30.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd rm-pg-upmap-items 25.11 2024-02-18T00:42:30.337 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:30.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:30.341 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:30.442 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:30.474 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:30.495 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:30.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:30.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:30.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:30.629 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:30.785 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:42:31.011 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-18T00:42:31.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:29.761319+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-104"}]': finished 2024-02-18T00:42:31.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:29.761387+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-18T00:42:31.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:29.763871+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-104", "mode": "writeback"}]: dispatch 2024-02-18T00:42:31.038 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:29.970941+0000 mgr.y (mgr.24454) 550 : cluster [DBG] pgmap v939: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:31.038 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:29.761319+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-104"}]': finished 2024-02-18T00:42:31.038 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:29.761387+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-18T00:42:31.038 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:29.763871+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-104", "mode": "writeback"}]: dispatch 2024-02-18T00:42:31.038 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:29.970941+0000 mgr.y (mgr.24454) 550 : cluster [DBG] pgmap v939: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:31.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:29.761319+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-104"}]': finished 2024-02-18T00:42:31.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:29.761387+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-18T00:42:31.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:29.763871+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-104", "mode": "writeback"}]: dispatch 2024-02-18T00:42:31.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:29.970941+0000 mgr.y (mgr.24454) 550 : cluster [DBG] pgmap v939: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:31.378 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:31.378 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:31.382 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:31.384 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:31.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:31.438 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:42:31.485 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:31.517 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:31.538 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:31.541 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:31.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:31.642 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:31.672 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:31.692 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:31.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:42:31.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:31.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:31.796 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:31.825 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:32.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:30.520548+0000 osd.5 (osd.5) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:32.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:30.520696+0000 osd.5 (osd.5) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:30.758729+0000 mon.a (mon.0) 3128 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:30.766891+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-104", "mode": "writeback"}]': finished 2024-02-18T00:42:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:30.766956+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-18T00:42:32.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:30.808206+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:32.023 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:32.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:30.520548+0000 osd.5 (osd.5) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:32.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:30.520696+0000 osd.5 (osd.5) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:32.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:30.758729+0000 mon.a (mon.0) 3128 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:32.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:30.766891+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-104", "mode": "writeback"}]': finished 2024-02-18T00:42:32.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:30.766956+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-18T00:42:32.025 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:30.808206+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:32.026 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:30.520548+0000 osd.5 (osd.5) 263 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:42:32.026 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:30.520696+0000 osd.5 (osd.5) 264 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:42:32.026 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:30.758729+0000 mon.a (mon.0) 3128 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:32.026 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:30.766891+0000 mon.a (mon.0) 3129 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-104", "mode": "writeback"}]': finished 2024-02-18T00:42:32.026 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:30.766956+0000 mon.a (mon.0) 3130 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-18T00:42:32.026 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:30.808206+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:32.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:32.027 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:32.046 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:32.128 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:32.182 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:32.184 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:32.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:32.288 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:32.298 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:32] "GET /metrics HTTP/1.1" 200 191139 "" "Prometheus/2.33.4" 2024-02-18T00:42:32.319 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:32.341 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:32.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:32.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:32.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:32.476 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:32.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:32.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:32.680 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:32.711 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:32.732 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:32.734 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:32.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:32.835 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:32.862 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:32.881 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:32.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:32.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:32.984 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:33.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:31.437480+0000 mgr.y (mgr.24454) 551 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:31.767686+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:31.767748+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-18T00:42:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:31.768404+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104"}]: dispatch 2024-02-18T00:42:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:31.971407+0000 mgr.y (mgr.24454) 552 : cluster [DBG] pgmap v942: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:42:33.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:32.764636+0000 mon.a (mon.0) 3135 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:33.013 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:33.067 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:31.437480+0000 mgr.y (mgr.24454) 551 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:31.767686+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:31.767748+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:31.768404+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104"}]: dispatch 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:31.971407+0000 mgr.y (mgr.24454) 552 : cluster [DBG] pgmap v942: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:32.764636+0000 mon.a (mon.0) 3135 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:31.437480+0000 mgr.y (mgr.24454) 551 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:33.070 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:31.767686+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:33.071 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:31.767748+0000 mon.a (mon.0) 3133 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-18T00:42:33.071 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:31.768404+0000 mon.a (mon.0) 3134 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104"}]: dispatch 2024-02-18T00:42:33.071 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:31.971407+0000 mgr.y (mgr.24454) 552 : cluster [DBG] pgmap v942: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:42:33.071 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:32.764636+0000 mon.a (mon.0) 3135 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:33.071 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:33.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:33.172 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:33.201 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:33.220 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:33.223 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:33.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:33.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:33.353 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:34.003 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:34.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:34.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:34.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:34.172 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:34.175 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:34.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:34.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:34.308 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:34.330 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:34.332 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:34.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:34.433 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:34.464 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:34.484 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:34.486 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:34.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:34.587 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:34.613 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:32.768846+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104"}]': finished 2024-02-18T00:42:34.614 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:32.768917+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-18T00:42:34.615 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:34.635 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:34.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:34.638 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:34.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:32.768846+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104"}]': finished 2024-02-18T00:42:34.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:32.768917+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-18T00:42:34.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:32.768846+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-104"}]': finished 2024-02-18T00:42:34.664 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:32.768917+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-18T00:42:34.739 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:34.786 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:34.788 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:34.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:34.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:34.919 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 -- 172.21.15.59:0/1193020082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86cc0fdbb0 msgr2=0x7f86cc0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 --2- 172.21.15.59:0/1193020082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86cc0fdbb0 0x7f86cc0fdf90 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7f86c8004870 tx=0x7f86c8009cb0).stop 2024-02-18T00:42:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 -- 172.21.15.59:0/1193020082 shutdown_connections 2024-02-18T00:42:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 --2- 172.21.15.59:0/1193020082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86cc0f8420 0x7f86cc104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 --2- 172.21.15.59:0/1193020082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86cc0fdbb0 0x7f86cc0fdf90 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 --2- 172.21.15.59:0/1193020082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fe510 0x7f86cc0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 -- 172.21.15.59:0/1193020082 >> 172.21.15.59:0/1193020082 conn(0x7f86cc0f3800 msgr2=0x7f86cc0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.307+0000 7f86d431b700 1 -- 172.21.15.59:0/1193020082 shutdown_connections 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 -- 172.21.15.59:0/1193020082 wait complete. 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 librados: starting msgr at 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 librados: starting objecter 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 Processor -- start 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 -- start start 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 librados: setting wanted keys 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 librados: calling monclient init 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86cc0f8420 0x7f86cc10e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fdbb0 0x7f86cc10e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86cc0fe510 0x7f86cc108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc108770 con 0x7f86cc0f8420 2024-02-18T00:42:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1088e0 con 0x7f86cc0fe510 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.308+0000 7f86d431b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc108aa0 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fdbb0 0x7f86cc10e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fdbb0 0x7f86cc10e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57636/0 (socket says 172.21.15.59:57636) 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 -- 172.21.15.59:0/3296406581 learned_addr learned my addr 172.21.15.59:0/3296406581 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d28b8700 1 --2- 172.21.15.59:0/3296406581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86cc0fe510 0x7f86cc108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86cc0f8420 0x7f86cc10e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 -- 172.21.15.59:0/3296406581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86cc0fe510 msgr2=0x7f86cc108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 --2- 172.21.15.59:0/3296406581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86cc0fe510 0x7f86cc108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 -- 172.21.15.59:0/3296406581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86cc0f8420 msgr2=0x7f86cc10e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 --2- 172.21.15.59:0/3296406581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86cc0f8420 0x7f86cc10e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 -- 172.21.15.59:0/3296406581 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc005040 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d18b6700 1 --2- 172.21.15.59:0/3296406581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fdbb0 0x7f86cc10e530 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 rx=0x7f86bc0048f0 tx=0x7f86bc009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86c37fe700 10 client.?.objecter ms_handle_connect 0x7f86cc0fdbb0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86c37fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc007b00 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c8005040 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86bc00f040 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc014990 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d431b700 10 client.41528.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc1090e0 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.309+0000 7f86d431b700 1 librados: init done 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.310+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f86cc107060 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.311+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f86bc010500 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.311+0000 7f86c37fe700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6800/3721891797 conn(0x7f86b8062c90 0x7f86b8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(621..621 src has 418..621) v4 ==== 8599+0+0 (secure 0 0 0) 0x7f86bc07d950 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 osd_map(621..621 src has 418..621) v4 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 3 client.41528.objecter handle_osd_map got epochs [621,621] > 0 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 3 client.41528.objecter handle_osd_map decoding full epoch 621 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6800/3721891797 conn(0x7f86b8062c90 0x7f86b8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 20 client.41528.objecter dump_active .. 0 homeless 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v621) v1 ==== 54+0+16 (secure 0 0 0) 0x7f86bc07e0a0 con 0x7f86cc0fdbb0 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86b8062c90 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6800/3721891797 conn(0x7f86b8062c90 0x7f86b8065150 secure :-1 s=READY pgs=3175 cs=0 l=1 rev1=1 rx=0x7f86c80043d0 tx=0x7f86c800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 10 client.41528.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 1 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6801/3700238977 conn(0x7f86cc0fee60 0x7f86cc0ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 1 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 0 1 2024-02-18T00:42:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc108d20 osd=0 4 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 10 client.41528.objecter _send_command 1 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc1096c0 con 0x7f86cc0fee60 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d28b8700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6801/3700238977 conn(0x7f86cc0fee60 0x7f86cc0ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc0fee60 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.312+0000 7f86d28b8700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6801/3700238977 conn(0x7f86cc0fee60 0x7f86cc0ff240 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.313+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc1096c0 con 0x7f86cc0fee60 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.313+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.313+0000 7f86c37fe700 10 client.41528.objecter _finish_command 1 = system:0 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.313+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc108d20 osd=0 4 2024-02-18T00:42:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.313+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 0 1 2024-02-18T00:42:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 10 client.41528.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 2 2024-02-18T00:42:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 2 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc108d20 osd=0 3 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 0 2 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc108d20 osd=0 4 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 10 client.41528.objecter _send_command 2 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.328+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc0273c0 con 0x7f86cc0fee60 2024-02-18T00:42:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.329+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc0273c0 con 0x7f86cc0fee60 2024-02-18T00:42:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.329+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.329+0000 7f86c37fe700 10 client.41528.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.329+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc108d20 osd=0 4 2024-02-18T00:42:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.329+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 0 2 2024-02-18T00:42:35.334 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:42:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 10 client.41528.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6805/26279110 conn(0x7f86cc0598f0 0x7f86cc059d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 1 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc028710 osd=1 4 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 10 client.41528.objecter _send_command 3 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc11b740 con 0x7f86cc0598f0 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6805/26279110 conn(0x7f86cc0598f0 0x7f86cc059d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6805/26279110 conn(0x7f86cc0598f0 0x7f86cc059d10 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.330+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc0598f0 2024-02-18T00:42:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.331+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc11b740 con 0x7f86cc0598f0 2024-02-18T00:42:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.331+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:42:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.331+0000 7f86c37fe700 10 client.41528.objecter _finish_command 3 = system:0 2024-02-18T00:42:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.331+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc028710 osd=1 4 2024-02-18T00:42:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.331+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 1 3 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 10 client.41528.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 4 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 4 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc028710 osd=1 3 2024-02-18T00:42:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 1 4 2024-02-18T00:42:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc028710 osd=1 4 2024-02-18T00:42:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 10 client.41528.objecter _send_command 4 2024-02-18T00:42:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.345+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc059360 con 0x7f86cc0598f0 2024-02-18T00:42:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc059360 con 0x7f86cc0598f0 2024-02-18T00:42:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86c37fe700 10 client.41528.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc028710 osd=1 4 2024-02-18T00:42:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 1 4 2024-02-18T00:42:35.349 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 10 client.41528.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 5 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6809/1153193637 conn(0x7f86cc11a160 0x7f86cc11a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 5 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 2 5 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc027d60 osd=2 4 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 10 client.41528.objecter _send_command 5 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc0283e0 con 0x7f86cc11a160 2024-02-18T00:42:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.347+0000 7f86d18b6700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6809/1153193637 conn(0x7f86cc11a160 0x7f86cc11a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.348+0000 7f86d18b6700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6809/1153193637 conn(0x7f86cc11a160 0x7f86cc11a580 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.348+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc11a160 2024-02-18T00:42:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.348+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc0283e0 con 0x7f86cc11a160 2024-02-18T00:42:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.348+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:42:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.348+0000 7f86c37fe700 10 client.41528.objecter _finish_command 5 = system:0 2024-02-18T00:42:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.348+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc027d60 osd=2 4 2024-02-18T00:42:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.348+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 2 5 2024-02-18T00:42:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 10 client.41528.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 6 2024-02-18T00:42:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 6 2024-02-18T00:42:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc027d60 osd=2 3 2024-02-18T00:42:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 2 6 2024-02-18T00:42:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc027d60 osd=2 4 2024-02-18T00:42:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 10 client.41528.objecter _send_command 6 2024-02-18T00:42:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.363+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc11b240 con 0x7f86cc11a160 2024-02-18T00:42:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc11b240 con 0x7f86cc11a160 2024-02-18T00:42:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86c37fe700 10 client.41528.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc027d60 osd=2 4 2024-02-18T00:42:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 2 6 2024-02-18T00:42:35.366 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 10 client.41528.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 7 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6813/1805700964 conn(0x7f86cc027940 0x7f86cc11fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 7 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 3 7 2024-02-18T00:42:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc0277d0 osd=3 4 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 10 client.41528.objecter _send_command 7 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc122a30 con 0x7f86cc027940 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.364+0000 7f86d28b8700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6813/1805700964 conn(0x7f86cc027940 0x7f86cc11fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.365+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc027940 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.365+0000 7f86d28b8700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6813/1805700964 conn(0x7f86cc027940 0x7f86cc11fde0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.365+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc122a30 con 0x7f86cc027940 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.365+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.365+0000 7f86c37fe700 10 client.41528.objecter _finish_command 7 = system:0 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.365+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc0277d0 osd=3 4 2024-02-18T00:42:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.365+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 3 7 2024-02-18T00:42:35.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 10 client.41528.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 8 2024-02-18T00:42:35.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 8 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc0277d0 osd=3 3 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 3 8 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc0277d0 osd=3 4 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 10 client.41528.objecter _send_command 8 2024-02-18T00:42:35.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.380+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc11acb0 con 0x7f86cc027940 2024-02-18T00:42:35.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc11acb0 con 0x7f86cc027940 2024-02-18T00:42:35.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86c37fe700 10 client.41528.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc0277d0 osd=3 4 2024-02-18T00:42:35.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 3 8 2024-02-18T00:42:35.383 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 10 client.41528.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 9 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6800/2044387056 conn(0x7f86cc11ae20 0x7f86cc121450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 9 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 4 9 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc11acb0 osd=4 4 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 10 client.41528.objecter _send_command 9 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.381+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc119cc0 con 0x7f86cc11ae20 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.382+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6800/2044387056 conn(0x7f86cc11ae20 0x7f86cc121450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.382+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc11ae20 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.382+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6800/2044387056 conn(0x7f86cc11ae20 0x7f86cc121450 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.383+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc119cc0 con 0x7f86cc11ae20 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.383+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:42:35.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.383+0000 7f86c37fe700 10 client.41528.objecter _finish_command 9 = system:0 2024-02-18T00:42:35.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.383+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc11acb0 osd=4 4 2024-02-18T00:42:35.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.383+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 4 9 2024-02-18T00:42:35.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 10 client.41528.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 10 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 10 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc11acb0 osd=4 3 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 4 10 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc11acb0 osd=4 4 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 10 client.41528.objecter _send_command 10 2024-02-18T00:42:35.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.398+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc119cc0 con 0x7f86cc11ae20 2024-02-18T00:42:35.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc119cc0 con 0x7f86cc11ae20 2024-02-18T00:42:35.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86c37fe700 10 client.41528.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc11acb0 osd=4 4 2024-02-18T00:42:35.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 4 10 2024-02-18T00:42:35.402 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:42:35.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 10 client.41528.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 11 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6804/2170432263 conn(0x7f86cc121e50 0x7f86cc122290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 11 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 5 11 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119cc0 osd=5 4 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 10 client.41528.objecter _send_command 11 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.400+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc119f70 con 0x7f86cc121e50 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.401+0000 7f86d18b6700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6804/2170432263 conn(0x7f86cc121e50 0x7f86cc122290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.401+0000 7f86d18b6700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6804/2170432263 conn(0x7f86cc121e50 0x7f86cc122290 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.401+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc121e50 2024-02-18T00:42:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.402+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc119f70 con 0x7f86cc121e50 2024-02-18T00:42:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.402+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:42:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.402+0000 7f86c37fe700 10 client.41528.objecter _finish_command 11 = system:0 2024-02-18T00:42:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.402+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc119cc0 osd=5 4 2024-02-18T00:42:35.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.402+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 5 11 2024-02-18T00:42:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 10 client.41528.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 12 2024-02-18T00:42:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 12 2024-02-18T00:42:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc119cc0 osd=5 3 2024-02-18T00:42:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 5 12 2024-02-18T00:42:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119cc0 osd=5 4 2024-02-18T00:42:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 10 client.41528.objecter _send_command 12 2024-02-18T00:42:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.416+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc119f70 con 0x7f86cc121e50 2024-02-18T00:42:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc119f70 con 0x7f86cc121e50 2024-02-18T00:42:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86c37fe700 10 client.41528.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc119cc0 osd=5 4 2024-02-18T00:42:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 5 12 2024-02-18T00:42:35.420 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:42:35.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 10 client.41528.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 13 2024-02-18T00:42:35.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6808/3356009390 conn(0x7f86cc06b420 0x7f86cc06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 13 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 6 13 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119f70 osd=6 4 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 10 client.41528.objecter _send_command 13 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.418+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc06cef0 con 0x7f86cc06b420 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.419+0000 7f86d28b8700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6808/3356009390 conn(0x7f86cc06b420 0x7f86cc06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.419+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc06b420 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.419+0000 7f86d28b8700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6808/3356009390 conn(0x7f86cc06b420 0x7f86cc06b880 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.420+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc06cef0 con 0x7f86cc06b420 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.420+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.420+0000 7f86c37fe700 10 client.41528.objecter _finish_command 13 = system:0 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.420+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc119f70 osd=6 4 2024-02-18T00:42:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.420+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 6 13 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 10 client.41528.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 14 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 14 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc119f70 osd=6 3 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 6 14 2024-02-18T00:42:35.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119f70 osd=6 4 2024-02-18T00:42:35.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 10 client.41528.objecter _send_command 14 2024-02-18T00:42:35.437 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.434+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc06bd70 con 0x7f86cc06b420 2024-02-18T00:42:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc06bd70 con 0x7f86cc06b420 2024-02-18T00:42:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86c37fe700 10 client.41528.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc119f70 osd=6 4 2024-02-18T00:42:35.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 6 14 2024-02-18T00:42:35.438 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 10 client.41528.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 15 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6812/53025171 conn(0x7f86cc06c450 0x7f86cc06c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 15 2024-02-18T00:42:35.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 7 15 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc06bd70 osd=7 4 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 10 client.41528.objecter _send_command 15 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.436+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86cc06ce20 con 0x7f86cc06c450 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.437+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6812/53025171 conn(0x7f86cc06c450 0x7f86cc06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.437+0000 7f86d20b7700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6812/53025171 conn(0x7f86cc06c450 0x7f86cc06c890 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.437+0000 7f86c37fe700 10 client.41528.objecter ms_handle_connect 0x7f86cc06c450 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.438+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f86cc06ce20 con 0x7f86cc06c450 2024-02-18T00:42:35.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.438+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:42:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.438+0000 7f86c37fe700 10 client.41528.objecter _finish_command 15 = system:0 2024-02-18T00:42:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.438+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc06bd70 osd=7 4 2024-02-18T00:42:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.438+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 7 15 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 10 client.41528.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign -1 16 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 20 client.41528.objecter _get_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 15 client.41528.objecter _session_command_op_remove -1 16 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 20 client.41528.objecter get_session s=0x7f86cc06bd70 osd=7 3 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 15 client.41528.objecter _session_command_op_assign 7 16 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc06bd70 osd=7 4 2024-02-18T00:42:35.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 10 client.41528.objecter _send_command 16 2024-02-18T00:42:35.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.452+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f86cc06ce20 con 0x7f86cc06c450 2024-02-18T00:42:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86c37fe700 1 -- 172.21.15.59:0/3296406581 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f86cc06ce20 con 0x7f86cc06c450 2024-02-18T00:42:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86c37fe700 10 client.41528.objecter ms_dispatch 0x7f86cc001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86c37fe700 10 client.41528.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86c37fe700 20 client.41528.objecter put_session s=0x7f86cc06bd70 osd=7 4 2024-02-18T00:42:35.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86c37fe700 15 client.41528.objecter _session_command_op_remove 7 16 2024-02-18T00:42:35.457 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 librados: watch_flush enter 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 librados: watch_flush exit 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.0 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6801/3700238977 conn(0x7f86cc0fee60 msgr2=0x7f86cc0ff240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6801/3700238977 conn(0x7f86cc0fee60 0x7f86cc0ff240 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc108d20 osd=0 1 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.1 2024-02-18T00:42:35.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6805/26279110 conn(0x7f86cc0598f0 msgr2=0x7f86cc059d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6805/26279110 conn(0x7f86cc0598f0 0x7f86cc059d10 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc028710 osd=1 1 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.2 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6809/1153193637 conn(0x7f86cc11a160 msgr2=0x7f86cc11a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6809/1153193637 conn(0x7f86cc11a160 0x7f86cc11a580 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc027d60 osd=2 1 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.3 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6813/1805700964 conn(0x7f86cc027940 msgr2=0x7f86cc11fde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6813/1805700964 conn(0x7f86cc027940 0x7f86cc11fde0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc0277d0 osd=3 1 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d28b8700 1 -- 172.21.15.59:0/3296406581 reap_dead start 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.4 2024-02-18T00:42:35.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6800/2044387056 conn(0x7f86cc11ae20 msgr2=0x7f86cc121450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6800/2044387056 conn(0x7f86cc11ae20 0x7f86cc121450 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d28b8700 1 -- 172.21.15.59:0/3296406581 reap_dead start 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc11acb0 osd=4 1 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.5 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6804/2170432263 conn(0x7f86cc121e50 msgr2=0x7f86cc122290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6804/2170432263 conn(0x7f86cc121e50 0x7f86cc122290 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119cc0 osd=5 1 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.6 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6808/3356009390 conn(0x7f86cc06b420 msgr2=0x7f86cc06b880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6808/3356009390 conn(0x7f86cc06b420 0x7f86cc06b880 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc119f70 osd=6 1 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter close_session for osd.7 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6812/53025171 conn(0x7f86cc06c450 msgr2=0x7f86cc06c890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.137:6812/53025171 conn(0x7f86cc06c450 0x7f86cc06c890 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter put_session s=0x7f86cc06bd70 osd=7 1 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 20 client.41528.objecter shutdown clearing up homeless session... 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 10 client.41528.objecter successfully canceled tick 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6800/3721891797 conn(0x7f86b8062c90 msgr2=0x7f86b8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> v2:172.21.15.59:6800/3721891797 conn(0x7f86b8062c90 0x7f86b8065150 secure :-1 s=READY pgs=3175 cs=0 l=1 rev1=1 rx=0x7f86c80043d0 tx=0x7f86c800b040).stop 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fdbb0 msgr2=0x7f86cc10e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d28b8700 1 -- 172.21.15.59:0/3296406581 reap_dead start 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fdbb0 0x7f86cc10e530 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 rx=0x7f86bc0048f0 tx=0x7f86bc009cb0).stop 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 shutdown_connections 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 --2- 172.21.15.59:0/3296406581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86cc0fdbb0 0x7f86cc10e530 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 >> 172.21.15.59:0/3296406581 conn(0x7f86cc0f3800 msgr2=0x7f86cc0fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 shutdown_connections 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 -- 172.21.15.59:0/3296406581 wait complete. 2024-02-18T00:42:35.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:35.455+0000 7f86d431b700 1 librados: shutdown 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.462 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:42:35.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:33.840475+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-18T00:42:35.595 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:33.971760+0000 mgr.y (mgr.24454) 553 : cluster [DBG] pgmap v945: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:42:35.595 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:34.840476+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-18T00:42:35.595 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:34.843668+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:35.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:33.840475+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-18T00:42:35.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:33.971760+0000 mgr.y (mgr.24454) 553 : cluster [DBG] pgmap v945: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:42:35.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:34.840476+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-18T00:42:35.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:34.843668+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:35.598 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:35.602 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:35.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:35.703 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:35.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:33.840475+0000 mon.a (mon.0) 3138 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-18T00:42:35.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:33.971760+0000 mgr.y (mgr.24454) 553 : cluster [DBG] pgmap v945: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:42:35.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:34.840476+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-18T00:42:35.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:34.843668+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:42:35.735 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:35.760 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:35.760 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:35.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:35.861 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:35.891 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:35.911 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:35.913 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:35.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:36.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:36.119 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:36.128 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:36.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:36.229 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:36.259 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:36.279 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:36.281 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:36.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:36.383 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:36.430 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:36.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:36.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:35.311849+0000 mon.c (mon.2) 553 : audit [DBG] from='client.? 172.21.15.59:0/3296406581' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:35.845081+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:35.845286+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-18T00:42:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:36.270654+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.59:0/1914161525' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]: dispatch 2024-02-18T00:42:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:36.271011+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]: dispatch 2024-02-18T00:42:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:35.311849+0000 mon.c (mon.2) 553 : audit [DBG] from='client.? 172.21.15.59:0/3296406581' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:35.845081+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:36.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:35.845286+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-18T00:42:36.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:36.270654+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.59:0/1914161525' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]: dispatch 2024-02-18T00:42:36.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:36.271011+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]: dispatch 2024-02-18T00:42:36.534 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:36.559 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:35.311849+0000 mon.c (mon.2) 553 : audit [DBG] from='client.? 172.21.15.59:0/3296406581' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:36.559 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:35.845081+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:42:36.559 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:35.845286+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-18T00:42:36.559 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:36.270654+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.59:0/1914161525' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]: dispatch 2024-02-18T00:42:36.560 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:36.271011+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]: dispatch 2024-02-18T00:42:36.561 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:36.580 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:36.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:36.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:36.685 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 -- 172.21.15.59:0/3907547175 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00fd510 msgr2=0x7ff0c00f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 --2- 172.21.15.59:0/3907547175 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00fd510 0x7ff0c00f6d40 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 rx=0x7ff0b00047b0 tx=0x7ff0b0009cb0).stop 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 -- 172.21.15.59:0/3907547175 shutdown_connections 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 --2- 172.21.15.59:0/3907547175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0c00f7400 0x7ff0c0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 --2- 172.21.15.59:0/3907547175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0c00fcbb0 0x7ff0c00fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 --2- 172.21.15.59:0/3907547175 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00fd510 0x7ff0c00f6d40 unknown :-1 s=CLOSED pgs=2587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 -- 172.21.15.59:0/3907547175 >> 172.21.15.59:0/3907547175 conn(0x7ff0c00f2820 msgr2=0x7ff0c00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 -- 172.21.15.59:0/3907547175 shutdown_connections 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 -- 172.21.15.59:0/3907547175 wait complete. 2024-02-18T00:42:36.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 librados: starting msgr at 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.072+0000 7ff0c5831700 1 librados: starting objecter 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 Processor -- start 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 -- start start 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 librados: setting wanted keys 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 librados: calling monclient init 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00f7400 0x7ff0c010e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0c00fcbb0 0x7ff0c010e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0c00fd510 0x7ff0c01084b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c01089a0 con 0x7ff0c00fd510 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0108b10 con 0x7ff0c00fcbb0 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0c5831700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0108cd0 con 0x7ff0c00f7400 2024-02-18T00:42:36.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0beffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00f7400 0x7ff0c010e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0beffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00f7400 0x7ff0c010e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57658/0 (socket says 172.21.15.59:57658) 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0beffd700 1 -- 172.21.15.59:0/1914161525 learned_addr learned my addr 172.21.15.59:0/1914161525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0bf7fe700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0c00fd510 0x7ff0c01084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.073+0000 7ff0be7fc700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0c00fcbb0 0x7ff0c010e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0beffd700 1 -- 172.21.15.59:0/1914161525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0c00fcbb0 msgr2=0x7ff0c010e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0beffd700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0c00fcbb0 0x7ff0c010e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0beffd700 1 -- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0c00fd510 msgr2=0x7ff0c01084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0beffd700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0c00fd510 0x7ff0c01084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0beffd700 1 -- 172.21.15.59:0/1914161525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4005040 con 0x7ff0c00f7400 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0bf7fe700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0c00fd510 0x7ff0c01084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0beffd700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00f7400 0x7ff0c010e310 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7ff0b4009400 tx=0x7ff0b4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:36.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0a7fff700 10 client.?.objecter ms_handle_connect 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0a7fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0a7fff700 1 -- 172.21.15.59:0/1914161525 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4007a40 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0a7fff700 1 -- 172.21.15.59:0/1914161525 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0b400f040 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b0005040 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0a7fff700 1 -- 172.21.15.59:0/1914161525 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4014a70 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0c5831700 10 client.41534.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c01092b0 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.074+0000 7ff0c5831700 1 librados: init done 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.075+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c005fa60 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.075+0000 7ff0a7fff700 1 -- 172.21.15.59:0/1914161525 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff0b4010530 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0a7fff700 1 --2- 172.21.15.59:0/1914161525 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff0a8062c90 0x7ff0a8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0a7fff700 1 -- 172.21.15.59:0/1914161525 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(622..622 src has 418..622) v4 ==== 8612+0+0 (secure 0 0 0) 0x7ff0b407d020 con 0x7ff0c00f7400 2024-02-18T00:42:36.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0a7fff700 10 client.41534.objecter ms_dispatch 0x7ff0c0001550 osd_map(622..622 src has 418..622) v4 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0be7fc700 1 --2- 172.21.15.59:0/1914161525 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff0a8062c90 0x7ff0a8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0a7fff700 3 client.41534.objecter handle_osd_map got epochs [622,622] > 0 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0a7fff700 3 client.41534.objecter handle_osd_map decoding full epoch 622 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0a7fff700 20 client.41534.objecter dump_active .. 0 homeless 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0a7fff700 10 client.41534.objecter ms_handle_connect 0x7ff0a8062c90 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.076+0000 7ff0be7fc700 1 --2- 172.21.15.59:0/1914161525 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff0a8062c90 0x7ff0a8065150 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 rx=0x7ff0b0005010 tx=0x7ff0b0009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.080+0000 7ff0a7fff700 1 -- 172.21.15.59:0/1914161525 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff0b4049f60 con 0x7ff0c00f7400 2024-02-18T00:42:36.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.269+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"} v 0) v1 -- 0x7ff0c0106150 con 0x7ff0c00f7400 2024-02-18T00:42:36.711 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:36.729 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:36.732 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:36.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:36.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:36.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.847+0000 7ff0a7fff700 1 -- 172.21.15.59:0/1914161525 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]=0 clear 25.11 pg_upmap_items mapping v623) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff0b40502c0 con 0x7ff0c00f7400 2024-02-18T00:42:36.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 10 librados: watch_flush enter 2024-02-18T00:42:36.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 10 librados: watch_flush exit 2024-02-18T00:42:36.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 20 client.41534.objecter shutdown clearing up homeless session... 2024-02-18T00:42:36.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 10 client.41534.objecter successfully canceled tick 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff0a8062c90 msgr2=0x7ff0a8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 1 --2- 172.21.15.59:0/1914161525 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff0a8062c90 0x7ff0a8065150 secure :-1 s=READY pgs=3176 cs=0 l=1 rev1=1 rx=0x7ff0b0005010 tx=0x7ff0b0009cb0).stop 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00f7400 msgr2=0x7ff0c010e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.849+0000 7ff0c5831700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00f7400 0x7ff0c010e310 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 rx=0x7ff0b4009400 tx=0x7ff0b4009cb0).stop 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 shutdown_connections 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0c00fd510 0x7ff0c01084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0c00f7400 0x7ff0c010e310 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 --2- 172.21.15.59:0/1914161525 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff0a8062c90 0x7ff0a8065150 unknown :-1 s=CLOSED pgs=3176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 --2- 172.21.15.59:0/1914161525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0c00fcbb0 0x7ff0c010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 >> 172.21.15.59:0/1914161525 conn(0x7ff0c00f2820 msgr2=0x7ff0c0060eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:36.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 shutdown_connections 2024-02-18T00:42:36.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 -- 172.21.15.59:0/1914161525 wait complete. 2024-02-18T00:42:36.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:36.850+0000 7ff0c5831700 1 librados: shutdown 2024-02-18T00:42:36.855 INFO:teuthology.orchestra.run.smithi059.stderr:clear 25.11 pg_upmap_items mapping 2024-02-18T00:42:36.862 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:36.881 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:36.883 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:36.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:36.984 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:37.033 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:37.036 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:37.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:37.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:37.167 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:37.292 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:37.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:37.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:37.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:37.424 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:35.972120+0000 mgr.y (mgr.24454) 554 : cluster [DBG] pgmap v948: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:42:37.424 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:36.846634+0000 mon.a (mon.0) 3144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]': finished 2024-02-18T00:42:37.424 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:36.846707+0000 mon.a (mon.0) 3145 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-18T00:42:37.424 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:36.858770+0000 mon.a (mon.0) 3146 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:37.425 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:35.972120+0000 mgr.y (mgr.24454) 554 : cluster [DBG] pgmap v948: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:42:37.425 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:36.846634+0000 mon.a (mon.0) 3144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]': finished 2024-02-18T00:42:37.425 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:36.846707+0000 mon.a (mon.0) 3145 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-18T00:42:37.425 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:36.858770+0000 mon.a (mon.0) 3146 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:37.625 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:37.627 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:37.627 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:37.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:37.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:35.972120+0000 mgr.y (mgr.24454) 554 : cluster [DBG] pgmap v948: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:42:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:36.846634+0000 mon.a (mon.0) 3144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "25.11"}]': finished 2024-02-18T00:42:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:36.846707+0000 mon.a (mon.0) 3145 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-18T00:42:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:36.858770+0000 mon.a (mon.0) 3146 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:42:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 -- 172.21.15.59:0/999701538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 msgr2=0x7fc0ec0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 --2- 172.21.15.59:0/999701538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 0x7fc0ec0fdf90 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 rx=0x7fc0dc004870 tx=0x7fc0dc009cb0).stop 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 -- 172.21.15.59:0/999701538 shutdown_connections 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 --2- 172.21.15.59:0/999701538 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0f8420 0x7fc0ec104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 --2- 172.21.15.59:0/999701538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 0x7fc0ec0fdf90 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 --2- 172.21.15.59:0/999701538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0ec0fe510 0x7fc0ec0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 -- 172.21.15.59:0/999701538 >> 172.21.15.59:0/999701538 conn(0x7fc0ec0f3800 msgr2=0x7fc0ec0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 -- 172.21.15.59:0/999701538 shutdown_connections 2024-02-18T00:42:38.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.255+0000 7fc0f356d700 1 -- 172.21.15.59:0/999701538 wait complete. 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 librados: starting msgr at 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 librados: starting objecter 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 Processor -- start 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 -- start start 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 librados: setting wanted keys 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 librados: calling monclient init 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0ec0f8420 0x7fc0ec10e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 0x7fc0ec10e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0fe510 0x7fc0ec108290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec108780 con 0x7fc0ec0fdbb0 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec1088f0 con 0x7fc0ec0f8420 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f356d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec108ab0 con 0x7fc0ec0fe510 2024-02-18T00:42:38.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f1b0a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0fe510 0x7fc0ec108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f0b08700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 0x7fc0ec10e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f1b0a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0fe510 0x7fc0ec108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:57678/0 (socket says 172.21.15.59:57678) 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f1b0a700 1 -- 172.21.15.59:0/2974867064 learned_addr learned my addr 172.21.15.59:0/2974867064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.256+0000 7fc0f1309700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0ec0f8420 0x7fc0ec10e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f1b0a700 1 -- 172.21.15.59:0/2974867064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0ec0f8420 msgr2=0x7fc0ec10e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f1b0a700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0ec0f8420 0x7fc0ec10e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f1b0a700 1 -- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 msgr2=0x7fc0ec10e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f1b0a700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 0x7fc0ec10e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f1b0a700 1 -- 172.21.15.59:0/2974867064 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40097c0 con 0x7fc0ec0fe510 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f0b08700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 0x7fc0ec10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f1b0a700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0fe510 0x7fc0ec108290 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7fc0e4001df0 tx=0x7fc0e4004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0e27fc700 10 client.?.objecter ms_handle_connect 0x7fc0ec0fe510 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0e27fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:38.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0e27fc700 1 -- 172.21.15.59:0/2974867064 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4009a10 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0e27fc700 1 -- 172.21.15.59:0/2974867064 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0e40090d0 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc005040 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0e27fc700 1 -- 172.21.15.59:0/2974867064 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007d30 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f356d700 10 client.41540.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec1090a0 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.257+0000 7fc0f356d700 1 librados: init done 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.258+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec05ee80 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.258+0000 7fc0e27fc700 1 -- 172.21.15.59:0/2974867064 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fc0e4009240 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0e27fc700 1 --2- 172.21.15.59:0/2974867064 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc0d8062c90 0x7fc0d8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0e27fc700 1 -- 172.21.15.59:0/2974867064 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(624..624 src has 418..624) v4 ==== 8578+0+0 (secure 0 0 0) 0x7fc0e407dcb0 con 0x7fc0ec0fe510 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0e27fc700 10 client.41540.objecter ms_dispatch 0x7fc0ec001550 osd_map(624..624 src has 418..624) v4 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0e27fc700 3 client.41540.objecter handle_osd_map got epochs [624,624] > 0 2024-02-18T00:42:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0e27fc700 3 client.41540.objecter handle_osd_map decoding full epoch 624 2024-02-18T00:42:38.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0f1309700 1 --2- 172.21.15.59:0/2974867064 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc0d8062c90 0x7fc0d8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:38.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0e27fc700 20 client.41540.objecter dump_active .. 0 homeless 2024-02-18T00:42:38.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0e27fc700 10 client.41540.objecter ms_handle_connect 0x7fc0d8062c90 2024-02-18T00:42:38.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.259+0000 7fc0f1309700 1 --2- 172.21.15.59:0/2974867064 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc0d8062c90 0x7fc0d8065150 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 rx=0x7fc0dc0043d0 tx=0x7fc0dc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:38.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.263+0000 7fc0e27fc700 1 -- 172.21.15.59:0/2974867064 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc0e4050f10 con 0x7fc0ec0fe510 2024-02-18T00:42:38.306 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:38.308 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:38.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:38.410 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:38.438 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:38.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.449+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fc0ec109840 con 0x7fc0ec0fe510 2024-02-18T00:42:38.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:42:38.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.763+0000 7fc0e27fc700 1 -- 172.21.15.59:0/2974867064 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v625) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc0e401c300 con 0x7fc0ec0fe510 2024-02-18T00:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 10 librados: watch_flush enter 2024-02-18T00:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 10 librados: watch_flush exit 2024-02-18T00:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 20 client.41540.objecter shutdown clearing up homeless session... 2024-02-18T00:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 10 client.41540.objecter successfully canceled tick 2024-02-18T00:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc0d8062c90 msgr2=0x7fc0d8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 --2- 172.21.15.59:0/2974867064 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc0d8062c90 0x7fc0d8065150 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 rx=0x7fc0dc0043d0 tx=0x7fc0dc00b040).stop 2024-02-18T00:42:38.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0fe510 msgr2=0x7fc0ec108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0fe510 0x7fc0ec108290 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 rx=0x7fc0e4001df0 tx=0x7fc0e4004c50).stop 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 shutdown_connections 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 --2- 172.21.15.59:0/2974867064 >> v2:172.21.15.59:6800/3721891797 conn(0x7fc0d8062c90 0x7fc0d8065150 unknown :-1 s=CLOSED pgs=3177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0ec0f8420 0x7fc0ec10e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc0ec0fe510 0x7fc0ec108290 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 --2- 172.21.15.59:0/2974867064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ec0fdbb0 0x7fc0ec10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 >> 172.21.15.59:0/2974867064 conn(0x7fc0ec0f3800 msgr2=0x7fc0ec0fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.766+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 shutdown_connections 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.767+0000 7fc0f356d700 1 -- 172.21.15.59:0/2974867064 wait complete. 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:38.767+0000 7fc0f356d700 1 librados: shutdown 2024-02-18T00:42:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:42:38.915 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:37.848018+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:37.848094+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:37.848850+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-106"}]: dispatch 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:37.972541+0000 mgr.y (mgr.24454) 555 : cluster [DBG] pgmap v951: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:38.451269+0000 mon.c (mon.2) 555 : audit [INF] from='client.? 172.21.15.59:0/2974867064' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:38.451582+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:38.757192+0000 mon.a (mon.0) 3151 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:38.761776+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-106"}]': finished 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:38.761853+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:38.761913+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-18T00:42:38.916 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:38.766070+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-106", "mode": "writeback"}]: dispatch 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:37.848018+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:37.848094+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:37.848850+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-106"}]: dispatch 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:37.972541+0000 mgr.y (mgr.24454) 555 : cluster [DBG] pgmap v951: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:38.451269+0000 mon.c (mon.2) 555 : audit [INF] from='client.? 172.21.15.59:0/2974867064' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:38.451582+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:38.757192+0000 mon.a (mon.0) 3151 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:38.761776+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-106"}]': finished 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:38.761853+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:38.761913+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-18T00:42:38.917 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:38.766070+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-106", "mode": "writeback"}]: dispatch 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:37.848018+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:37.848094+0000 mon.a (mon.0) 3148 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:37.848850+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-106"}]: dispatch 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:37.972541+0000 mgr.y (mgr.24454) 555 : cluster [DBG] pgmap v951: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:38.451269+0000 mon.c (mon.2) 555 : audit [INF] from='client.? 172.21.15.59:0/2974867064' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:38.451582+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:38.757192+0000 mon.a (mon.0) 3151 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:38.761776+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-106"}]': finished 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:38.761853+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:42:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:38.761913+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-18T00:42:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:38.766070+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-106", "mode": "writeback"}]: dispatch 2024-02-18T00:42:39.852 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:39.758745+0000 mon.a (mon.0) 3156 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:39.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:39.758745+0000 mon.a (mon.0) 3156 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:39.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:39.768660+0000 mon.a (mon.0) 3157 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-106", "mode": "writeback"}]': finished 2024-02-18T00:42:39.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:39.768772+0000 mon.a (mon.0) 3158 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-18T00:42:39.853 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:39.784284+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.0"}]: dispatch 2024-02-18T00:42:40.105 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:39.768660+0000 mon.a (mon.0) 3157 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-106", "mode": "writeback"}]': finished 2024-02-18T00:42:40.105 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:39.768772+0000 mon.a (mon.0) 3158 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-18T00:42:40.105 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:39.784284+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.0"}]: dispatch 2024-02-18T00:42:40.189 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:40.195 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:40.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:40.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:39.758745+0000 mon.a (mon.0) 3156 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:42:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:39.768660+0000 mon.a (mon.0) 3157 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-106", "mode": "writeback"}]': finished 2024-02-18T00:42:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:39.768772+0000 mon.a (mon.0) 3158 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-18T00:42:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:39.784284+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.0"}]: dispatch 2024-02-18T00:42:40.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:40.328 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:40.430 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:40.434 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:40.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:40.535 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:40.566 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:40.717 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:40.720 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:40.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:40.743 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:42:40.743 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:42:40.746 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:40.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:40.850 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:41.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:39.784487+0000 mgr.y (mgr.24454) 556 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.0"}]: dispatch 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:39.906738+0000 mon.a (mon.0) 3160 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:39.973050+0000 mgr.y (mgr.24454) 557 : cluster [DBG] pgmap v954: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:40.738872+0000 osd.2 (osd.2) 247 : cluster [DBG] 237.0 scrub starts 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:40.739997+0000 osd.2 (osd.2) 248 : cluster [DBG] 237.0 scrub ok 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:39.784487+0000 mgr.y (mgr.24454) 556 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.0"}]: dispatch 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:39.906738+0000 mon.a (mon.0) 3160 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:39.973050+0000 mgr.y (mgr.24454) 557 : cluster [DBG] pgmap v954: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:40.738872+0000 osd.2 (osd.2) 247 : cluster [DBG] 237.0 scrub starts 2024-02-18T00:42:41.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:40.739997+0000 osd.2 (osd.2) 248 : cluster [DBG] 237.0 scrub ok 2024-02-18T00:42:41.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:42:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:42:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:39.784487+0000 mgr.y (mgr.24454) 556 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.0"}]: dispatch 2024-02-18T00:42:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:39.906738+0000 mon.a (mon.0) 3160 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:42:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:39.973050+0000 mgr.y (mgr.24454) 557 : cluster [DBG] pgmap v954: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:42:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:40.738872+0000 osd.2 (osd.2) 247 : cluster [DBG] 237.0 scrub starts 2024-02-18T00:42:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:40.739997+0000 osd.2 (osd.2) 248 : cluster [DBG] 237.0 scrub ok 2024-02-18T00:42:41.754 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:42:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:42:41.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-18T00:42:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:42.031 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:42.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:42.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:42:42.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:42.202 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:42.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:42:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:42] "GET /metrics HTTP/1.1" 200 191233 "" "Prometheus/2.33.4" 2024-02-18T00:42:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:41.448478+0000 mgr.y (mgr.24454) 558 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:41.974017+0000 mgr.y (mgr.24454) 559 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-18T00:42:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:42.740953+0000 mon.a (mon.0) 3161 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [4, 1]}]: dispatch 2024-02-18T00:42:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:42.741200+0000 mon.a (mon.0) 3162 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.18", "id": [0, 2]}]: dispatch 2024-02-18T00:42:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:42.741370+0000 mon.a (mon.0) 3163 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1d", "id": [7, 2]}]: dispatch 2024-02-18T00:42:43.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:42.741541+0000 mon.a (mon.0) 3164 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1f", "id": [7, 3]}]: dispatch 2024-02-18T00:42:43.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:41.448478+0000 mgr.y (mgr.24454) 558 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:41.974017+0000 mgr.y (mgr.24454) 559 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:42.740953+0000 mon.a (mon.0) 3161 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [4, 1]}]: dispatch 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:42.741200+0000 mon.a (mon.0) 3162 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.18", "id": [0, 2]}]: dispatch 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:42.741370+0000 mon.a (mon.0) 3163 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1d", "id": [7, 2]}]: dispatch 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:42.741541+0000 mon.a (mon.0) 3164 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1f", "id": [7, 3]}]: dispatch 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:41.448478+0000 mgr.y (mgr.24454) 558 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:41.974017+0000 mgr.y (mgr.24454) 559 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:42.740953+0000 mon.a (mon.0) 3161 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [4, 1]}]: dispatch 2024-02-18T00:42:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:42.741200+0000 mon.a (mon.0) 3162 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.18", "id": [0, 2]}]: dispatch 2024-02-18T00:42:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:42.741370+0000 mon.a (mon.0) 3163 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1d", "id": [7, 2]}]: dispatch 2024-02-18T00:42:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:42.741541+0000 mon.a (mon.0) 3164 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1f", "id": [7, 3]}]: dispatch 2024-02-18T00:42:43.584 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:43.587 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:43.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:43.688 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:43.771 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:43.794 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:43.796 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:43.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:43.897 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:43.926 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:43.935 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:42:43.936 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:42:43.936 INFO:tasks.thrashosds.thrasher:Growing pool default.rgw.log 2024-02-18T00:42:43.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:42:43.946 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:43.948 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:43.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:44.050 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:44.077 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:44.100 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:44.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 -- 172.21.15.59:0/3637652415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59100f9610 msgr2=0x7f5910101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 --2- 172.21.15.59:0/3637652415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59100f9610 0x7f5910101d20 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7f59000047b0 tx=0x7f5900009cb0).stop 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 -- 172.21.15.59:0/3637652415 shutdown_connections 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 --2- 172.21.15.59:0/3637652415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59100f8cb0 0x7f59100f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 --2- 172.21.15.59:0/3637652415 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5910102210 0x7f5910104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 --2- 172.21.15.59:0/3637652415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59100f9610 0x7f5910101d20 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 -- 172.21.15.59:0/3637652415 >> 172.21.15.59:0/3637652415 conn(0x7f59100f2820 msgr2=0x7f59100f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 -- 172.21.15.59:0/3637652415 shutdown_connections 2024-02-18T00:42:44.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.158+0000 7f5917966700 1 -- 172.21.15.59:0/3637652415 wait complete. 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 librados: starting msgr at 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 librados: starting objecter 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 Processor -- start 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 -- start start 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 librados: setting wanted keys 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 librados: calling monclient init 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59100f8cb0 0x7f59100f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59100f9610 0x7f59100f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5910102210 0x7f59100f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5910116700 con 0x7f59100f8cb0 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5910116870 con 0x7f5910102210 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5917966700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5910116a30 con 0x7f59100f9610 2024-02-18T00:42:44.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5915702700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59100f8cb0 0x7f59100f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59100f9610 0x7f59100f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59100f9610 0x7f59100f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39054/0 (socket says 172.21.15.59:39054) 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 -- 172.21.15.59:0/101349270 learned_addr learned my addr 172.21.15.59:0/101349270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5915f03700 1 --2- 172.21.15.59:0/101349270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5910102210 0x7f59100f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 -- 172.21.15.59:0/101349270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5910102210 msgr2=0x7f59100f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 --2- 172.21.15.59:0/101349270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5910102210 0x7f59100f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 -- 172.21.15.59:0/101349270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59100f8cb0 msgr2=0x7f59100f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 --2- 172.21.15.59:0/101349270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59100f8cb0 0x7f59100f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.159+0000 7f5914f01700 1 -- 172.21.15.59:0/101349270 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080097c0 con 0x7f59100f9610 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f5914f01700 1 --2- 172.21.15.59:0/101349270 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59100f9610 0x7f59100f7150 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 rx=0x7f5900009b10 tx=0x7f5900009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f59067fc700 10 client.?.objecter ms_handle_connect 0x7f59100f9610 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f59067fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900018070 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5900005040 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f5917966700 10 client.41546.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5900013da0 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910116f20 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f5917966700 1 librados: init done 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.160+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590001dcc0 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.161+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f591005fa60 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.161+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f590001d420 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6800/3721891797 conn(0x7f58fc062c90 0x7f58fc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(628..628 src has 418..628) v4 ==== 8657+0+0 (secure 0 0 0) 0x7f590007e940 con 0x7f59100f9610 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 osd_map(628..628 src has 418..628) v4 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 3 client.41546.objecter handle_osd_map got epochs [628,628] > 0 2024-02-18T00:42:44.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 3 client.41546.objecter handle_osd_map decoding full epoch 628 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6800/3721891797 conn(0x7f58fc062c90 0x7f58fc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 20 client.41546.objecter dump_active .. 0 homeless 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6800/3721891797 conn(0x7f58fc062c90 0x7f58fc065150 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 rx=0x7f590c004790 tx=0x7f590c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f58fc062c90 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v628) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5900051bb0 con 0x7f59100f9610 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 10 client.41546.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 1 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6801/3700238977 conn(0x7f59100fc1e0 0x7f59100fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 1 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 0 1 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591005ee80 osd=0 4 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 10 client.41546.objecter _send_command 1 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.162+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5910107290 con 0x7f59100fc1e0 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f5915f03700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6801/3700238977 conn(0x7f59100fc1e0 0x7f59100fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f59100fc1e0 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f5915f03700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6801/3700238977 conn(0x7f59100fc1e0 0x7f59100fc5e0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5910107290 con 0x7f59100fc1e0 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f59067fc700 10 client.41546.objecter _finish_command 1 = system:0 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591005ee80 osd=0 4 2024-02-18T00:42:44.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.163+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 0 1 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 10 client.41546.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 2 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 2 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591005ee80 osd=0 3 2024-02-18T00:42:44.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 0 2 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591005ee80 osd=0 4 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 10 client.41546.objecter _send_command 2 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.179+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f59100273c0 con 0x7f59100fc1e0 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f59100273c0 con 0x7f59100fc1e0 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f59067fc700 10 client.41546.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591005ee80 osd=0 4 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 0 2 2024-02-18T00:42:44.186 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 10 client.41546.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 3 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6805/26279110 conn(0x7f59100598f0 0x7f5910059d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 3 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 1 3 2024-02-18T00:42:44.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910028710 osd=1 4 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 10 client.41546.objecter _send_command 3 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.182+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f591011d8f0 con 0x7f59100598f0 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.183+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6805/26279110 conn(0x7f59100598f0 0x7f5910059d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.183+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f59100598f0 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.183+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6805/26279110 conn(0x7f59100598f0 0x7f5910059d30 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.184+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f591011d8f0 con 0x7f59100598f0 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.184+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.184+0000 7f59067fc700 10 client.41546.objecter _finish_command 3 = system:0 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.184+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f5910028710 osd=1 4 2024-02-18T00:42:44.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.184+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 1 3 2024-02-18T00:42:44.201 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 10 client.41546.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 4 2024-02-18T00:42:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 4 2024-02-18T00:42:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f5910028710 osd=1 3 2024-02-18T00:42:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 1 4 2024-02-18T00:42:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910028710 osd=1 4 2024-02-18T00:42:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 10 client.41546.objecter _send_command 4 2024-02-18T00:42:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.208+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5910059360 con 0x7f59100598f0 2024-02-18T00:42:44.219 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:42:44.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.215+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5910059360 con 0x7f59100598f0 2024-02-18T00:42:44.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.215+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.215+0000 7f59067fc700 10 client.41546.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.215+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f5910028710 osd=1 4 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.215+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 1 4 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 10 client.41546.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 5 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6809/1153193637 conn(0x7f591011c310 0x7f591011c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 5 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 2 5 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910027d60 osd=2 4 2024-02-18T00:42:44.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 10 client.41546.objecter _send_command 5 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59100283e0 con 0x7f591011c310 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5914f01700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6809/1153193637 conn(0x7f591011c310 0x7f591011c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.216+0000 7f5914f01700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6809/1153193637 conn(0x7f591011c310 0x7f591011c750 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.218+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f591011c310 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.218+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f59100283e0 con 0x7f591011c310 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.218+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.218+0000 7f59067fc700 10 client.41546.objecter _finish_command 5 = system:0 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.218+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f5910027d60 osd=2 4 2024-02-18T00:42:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.218+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 2 5 2024-02-18T00:42:44.228 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 10 client.41546.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 6 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 6 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f5910027d60 osd=2 3 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 2 6 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910027d60 osd=2 4 2024-02-18T00:42:44.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 10 client.41546.objecter _send_command 6 2024-02-18T00:42:44.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.233+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f591011d410 con 0x7f591011c310 2024-02-18T00:42:44.236 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:44.238 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:44.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.236+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f591011d410 con 0x7f591011c310 2024-02-18T00:42:44.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.236+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.236+0000 7f59067fc700 10 client.41546.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.236+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f5910027d60 osd=2 4 2024-02-18T00:42:44.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.236+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 2 6 2024-02-18T00:42:44.238 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:42:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 10 client.41546.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 7 2024-02-18T00:42:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5910027940 0x7f5910121f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 7 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 3 7 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f59100277d0 osd=3 4 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 10 client.41546.objecter _send_command 7 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5910124be0 con 0x7f5910027940 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5915f03700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5910027940 0x7f5910121f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f5910027940 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.237+0000 7f5915f03700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5910027940 0x7f5910121f90 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.238+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5910124be0 con 0x7f5910027940 2024-02-18T00:42:44.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.238+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:42:44.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.238+0000 7f59067fc700 10 client.41546.objecter _finish_command 7 = system:0 2024-02-18T00:42:44.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.238+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f59100277d0 osd=3 4 2024-02-18T00:42:44.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.238+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 3 7 2024-02-18T00:42:44.246 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:44.249 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:44.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:44.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:43.032834+0000 mon.a (mon.0) 3165 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [4, 1]}]': finished 2024-02-18T00:42:44.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:43.032970+0000 mon.a (mon.0) 3166 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.18", "id": [0, 2]}]': finished 2024-02-18T00:42:44.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:43.033087+0000 mon.a (mon.0) 3167 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1d", "id": [7, 2]}]': finished 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:43.033191+0000 mon.a (mon.0) 3168 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1f", "id": [7, 3]}]': finished 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:43.033293+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:44.033971+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:43.032834+0000 mon.a (mon.0) 3165 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [4, 1]}]': finished 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:43.032970+0000 mon.a (mon.0) 3166 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.18", "id": [0, 2]}]': finished 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:43.033087+0000 mon.a (mon.0) 3167 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1d", "id": [7, 2]}]': finished 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:43.033191+0000 mon.a (mon.0) 3168 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1f", "id": [7, 3]}]': finished 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:43.033293+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-18T00:42:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:44.033971+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 10 client.41546.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 8 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 8 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f59100277d0 osd=3 3 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 3 8 2024-02-18T00:42:44.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f59100277d0 osd=3 4 2024-02-18T00:42:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 10 client.41546.objecter _send_command 8 2024-02-18T00:42:44.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.262+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f591011ce80 con 0x7f5910027940 2024-02-18T00:42:44.268 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:42:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f591011ce80 con 0x7f5910027940 2024-02-18T00:42:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f59067fc700 10 client.41546.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f59100277d0 osd=3 4 2024-02-18T00:42:44.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 3 8 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 10 client.41546.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 9 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6800/2044387056 conn(0x7f591011d020 0x7f5910123600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 9 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 4 9 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011ce80 osd=4 4 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 10 client.41546.objecter _send_command 9 2024-02-18T00:42:44.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.266+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f591011be70 con 0x7f591011d020 2024-02-18T00:42:44.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.269+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6800/2044387056 conn(0x7f591011d020 0x7f5910123600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.269+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f591011d020 2024-02-18T00:42:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.269+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6800/2044387056 conn(0x7f591011d020 0x7f5910123600 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.270+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f591011be70 con 0x7f591011d020 2024-02-18T00:42:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.270+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:42:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.270+0000 7f59067fc700 10 client.41546.objecter _finish_command 9 = system:0 2024-02-18T00:42:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.270+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591011ce80 osd=4 4 2024-02-18T00:42:44.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.270+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 4 9 2024-02-18T00:42:44.286 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 10 client.41546.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 10 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 10 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591011ce80 osd=4 3 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 4 10 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011ce80 osd=4 4 2024-02-18T00:42:44.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 10 client.41546.objecter _send_command 10 2024-02-18T00:42:44.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.286+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f591011be70 con 0x7f591011d020 2024-02-18T00:42:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f591011be70 con 0x7f591011d020 2024-02-18T00:42:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f59067fc700 10 client.41546.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591011ce80 osd=4 4 2024-02-18T00:42:44.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 4 10 2024-02-18T00:42:44.290 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 10 client.41546.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 11 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5910124000 0x7f5910124440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 11 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 5 11 2024-02-18T00:42:44.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011be70 osd=5 4 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 10 client.41546.objecter _send_command 11 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5910124930 con 0x7f5910124000 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5914f01700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5910124000 0x7f5910124440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f5910124000 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.289+0000 7f5914f01700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5910124000 0x7f5910124440 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.290+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5910124930 con 0x7f5910124000 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.290+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.290+0000 7f59067fc700 10 client.41546.objecter _finish_command 11 = system:0 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.290+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591011be70 osd=5 4 2024-02-18T00:42:44.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.290+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 5 11 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 10 client.41546.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 12 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 12 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591011be70 osd=5 3 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 5 12 2024-02-18T00:42:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011be70 osd=5 4 2024-02-18T00:42:44.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 10 client.41546.objecter _send_command 12 2024-02-18T00:42:44.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.305+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f5910124930 con 0x7f5910124000 2024-02-18T00:42:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.307+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f5910124930 con 0x7f5910124000 2024-02-18T00:42:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.307+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.307+0000 7f59067fc700 10 client.41546.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.307+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591011be70 osd=5 4 2024-02-18T00:42:44.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.307+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 5 12 2024-02-18T00:42:44.309 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 10 client.41546.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 13 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5910129280 0x7f591012b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 13 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 6 13 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910124930 osd=6 4 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 10 client.41546.objecter _send_command 13 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f591012e390 con 0x7f5910129280 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5915f03700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5910129280 0x7f591012b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f5910129280 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.308+0000 7f5915f03700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5910129280 0x7f591012b670 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.310+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f591012e390 con 0x7f5910129280 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.310+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.310+0000 7f59067fc700 10 client.41546.objecter _finish_command 13 = system:0 2024-02-18T00:42:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.310+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f5910124930 osd=6 4 2024-02-18T00:42:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.310+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 6 13 2024-02-18T00:42:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 10 client.41546.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 14 2024-02-18T00:42:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 14 2024-02-18T00:42:44.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f5910124930 osd=6 3 2024-02-18T00:42:44.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 6 14 2024-02-18T00:42:44.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910124930 osd=6 4 2024-02-18T00:42:44.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 10 client.41546.objecter _send_command 14 2024-02-18T00:42:44.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.324+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f591012e350 con 0x7f5910129280 2024-02-18T00:42:44.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f591012e350 con 0x7f5910129280 2024-02-18T00:42:44.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f59067fc700 10 client.41546.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f5910124930 osd=6 4 2024-02-18T00:42:44.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 6 14 2024-02-18T00:42:44.327 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 10 client.41546.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 15 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6812/53025171 conn(0x7f591012d6c0 0x7f591012db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 15 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 7 15 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591012e350 osd=7 4 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 10 client.41546.objecter _send_command 15 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5910131720 con 0x7f591012d6c0 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6812/53025171 conn(0x7f591012d6c0 0x7f591012db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f59067fc700 10 client.41546.objecter ms_handle_connect 0x7f591012d6c0 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.326+0000 7f5915702700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6812/53025171 conn(0x7f591012d6c0 0x7f591012db40 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.327+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5910131720 con 0x7f591012d6c0 2024-02-18T00:42:44.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.327+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:42:44.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.327+0000 7f59067fc700 10 client.41546.objecter _finish_command 15 = system:0 2024-02-18T00:42:44.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.327+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591012e350 osd=7 4 2024-02-18T00:42:44.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.327+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 7 15 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 10 client.41546.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign -1 16 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 20 client.41546.objecter _get_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 15 client.41546.objecter _session_command_op_remove -1 16 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 20 client.41546.objecter get_session s=0x7f591012e350 osd=7 3 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 15 client.41546.objecter _session_command_op_assign 7 16 2024-02-18T00:42:44.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591012e350 osd=7 4 2024-02-18T00:42:44.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 10 client.41546.objecter _send_command 16 2024-02-18T00:42:44.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.341+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f591012bb60 con 0x7f591012d6c0 2024-02-18T00:42:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f59067fc700 1 -- 172.21.15.59:0/101349270 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f591012bb60 con 0x7f591012d6c0 2024-02-18T00:42:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f59067fc700 10 client.41546.objecter ms_dispatch 0x7f5910001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:42:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f59067fc700 10 client.41546.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f59067fc700 20 client.41546.objecter put_session s=0x7f591012e350 osd=7 4 2024-02-18T00:42:44.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f59067fc700 15 client.41546.objecter _session_command_op_remove 7 16 2024-02-18T00:42:44.345 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:42:44.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 librados: watch_flush enter 2024-02-18T00:42:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 librados: watch_flush exit 2024-02-18T00:42:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.0 2024-02-18T00:42:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6801/3700238977 conn(0x7f59100fc1e0 msgr2=0x7f59100fc5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6801/3700238977 conn(0x7f59100fc1e0 0x7f59100fc5e0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591005ee80 osd=0 1 2024-02-18T00:42:44.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.1 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6805/26279110 conn(0x7f59100598f0 msgr2=0x7f5910059d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6805/26279110 conn(0x7f59100598f0 0x7f5910059d30 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910028710 osd=1 1 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.2 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6809/1153193637 conn(0x7f591011c310 msgr2=0x7f591011c750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6809/1153193637 conn(0x7f591011c310 0x7f591011c750 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910027d60 osd=2 1 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.3 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5910027940 msgr2=0x7f5910121f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5915f03700 1 -- 172.21.15.59:0/101349270 reap_dead start 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5910027940 0x7f5910121f90 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f59100277d0 osd=3 1 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.4 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6800/2044387056 conn(0x7f591011d020 msgr2=0x7f5910123600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6800/2044387056 conn(0x7f591011d020 0x7f5910123600 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011ce80 osd=4 1 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.5 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5910124000 msgr2=0x7f5910124440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5910124000 0x7f5910124440 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591011be70 osd=5 1 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.6 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5910129280 msgr2=0x7f591012b670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5910129280 0x7f591012b670 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f5910124930 osd=6 1 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter close_session for osd.7 2024-02-18T00:42:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6812/53025171 conn(0x7f591012d6c0 msgr2=0x7f591012db40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.137:6812/53025171 conn(0x7f591012d6c0 0x7f591012db40 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter put_session s=0x7f591012e350 osd=7 1 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 20 client.41546.objecter shutdown clearing up homeless session... 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 10 client.41546.objecter successfully canceled tick 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5915f03700 1 -- 172.21.15.59:0/101349270 reap_dead start 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6800/3721891797 conn(0x7f58fc062c90 msgr2=0x7f58fc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6800/3721891797 conn(0x7f58fc062c90 0x7f58fc065150 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 rx=0x7f590c004790 tx=0x7f590c00b040).stop 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59100f9610 msgr2=0x7f59100f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59100f9610 0x7f59100f7150 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 rx=0x7f5900009b10 tx=0x7f5900009cb0).stop 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 shutdown_connections 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59100f9610 0x7f59100f7150 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 --2- 172.21.15.59:0/101349270 >> v2:172.21.15.59:6800/3721891797 conn(0x7f58fc062c90 0x7f58fc065150 unknown :-1 s=CLOSED pgs=3178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 >> 172.21.15.59:0/101349270 conn(0x7f59100f2820 msgr2=0x7f59100f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:44.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 shutdown_connections 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 -- 172.21.15.59:0/101349270 wait complete. 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:44.344+0000 7f5917966700 1 librados: shutdown 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:42:44.351 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:44.376 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:43.032834+0000 mon.a (mon.0) 3165 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [4, 1]}]': finished 2024-02-18T00:42:44.376 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:43.032970+0000 mon.a (mon.0) 3166 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "25.18", "id": [0, 2]}]': finished 2024-02-18T00:42:44.376 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:43.033087+0000 mon.a (mon.0) 3167 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1d", "id": [7, 2]}]': finished 2024-02-18T00:42:44.376 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:43.033191+0000 mon.a (mon.0) 3168 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "237.1f", "id": [7, 3]}]': finished 2024-02-18T00:42:44.376 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:43.033293+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-18T00:42:44.376 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:44.033971+0000 mon.a (mon.0) 3170 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-18T00:42:44.378 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:44.396 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:44.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:44.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:44.500 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:44.530 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:44.558 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:44.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:44.561 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:44.662 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:44.708 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:44.710 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:44.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:44.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:44.856 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:44.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:44.859 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:44.960 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:44.986 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:45.005 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:45.007 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:45.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:45.109 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:45.135 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:45.154 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:45.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:45.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:43.974395+0000 mgr.y (mgr.24454) 560 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:42:45.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:44.163637+0000 mon.c (mon.2) 556 : audit [DBG] from='client.? 172.21.15.59:0/101349270' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:45.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:43.974395+0000 mgr.y (mgr.24454) 560 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:42:45.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:44.163637+0000 mon.c (mon.2) 556 : audit [DBG] from='client.? 172.21.15.59:0/101349270' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:43.974395+0000 mgr.y (mgr.24454) 560 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T00:42:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:44.163637+0000 mon.c (mon.2) 556 : audit [DBG] from='client.? 172.21.15.59:0/101349270' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:45.257 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:45.289 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:46.942 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:46.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:46.945 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:47.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:47.082 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:45.975391+0000 mgr.y (mgr.24454) 561 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-18T00:42:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:45.975391+0000 mgr.y (mgr.24454) 561 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-18T00:42:47.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:45.975391+0000 mgr.y (mgr.24454) 561 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-18T00:42:47.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:42:48.434 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:48.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:48.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:48.542 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 -- 172.21.15.59:0/3597901138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b9e0 msgr2=0x7ff378103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 --2- 172.21.15.59:0/3597901138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b9e0 0x7ff378103fb0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 rx=0x7ff360004870 tx=0x7ff360009cb0).stop 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 -- 172.21.15.59:0/3597901138 shutdown_connections 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 --2- 172.21.15.59:0/3597901138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3781044a0 0x7ff378106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 --2- 172.21.15.59:0/3597901138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff37806b110 0x7ff37806b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 --2- 172.21.15.59:0/3597901138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b9e0 0x7ff378103fb0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 -- 172.21.15.59:0/3597901138 >> 172.21.15.59:0/3597901138 conn(0x7ff3780f3800 msgr2=0x7ff3780f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 -- 172.21.15.59:0/3597901138 shutdown_connections 2024-02-18T00:42:48.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 -- 172.21.15.59:0/3597901138 wait complete. 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 librados: starting msgr at 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.542+0000 7ff37db2d700 1 librados: starting objecter 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 Processor -- start 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 -- start start 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 librados: setting wanted keys 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 librados: calling monclient init 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b110 0x7ff3781103b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff37806b9e0 0x7ff3781108a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3781044a0 0x7ff37810a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37810aa90 con 0x7ff37806b9e0 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37810ac00 con 0x7ff3781044a0 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff37db2d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37810adc0 con 0x7ff37806b110 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b110 0x7ff3781103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:48.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b110 0x7ff3781103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39072/0 (socket says 172.21.15.59:39072) 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 -- 172.21.15.59:0/1105270227 learned_addr learned my addr 172.21.15.59:0/1105270227 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff376ffd700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff37806b9e0 0x7ff3781108a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff377fff700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3781044a0 0x7ff37810a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 -- 172.21.15.59:0/1105270227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3781044a0 msgr2=0x7ff37810a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3781044a0 0x7ff37810a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 -- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff37806b9e0 msgr2=0x7ff3781108a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff37806b9e0 0x7ff3781108a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff3777fe700 1 -- 172.21.15.59:0/1105270227 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360005040 con 0x7ff37806b110 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.543+0000 7ff376ffd700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff37806b9e0 0x7ff3781108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff3777fe700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b110 0x7ff3781103b0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 rx=0x7ff37000e970 tx=0x7ff37000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff374ff9700 10 client.?.objecter ms_handle_connect 0x7ff37806b110 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff374ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:48.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff374ff9700 1 -- 172.21.15.59:0/1105270227 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000ee60 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff374ff9700 1 -- 172.21.15.59:0/1105270227 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff37000f040 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37810b0a0 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff374ff9700 1 -- 172.21.15.59:0/1105270227 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370007ea0 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff37db2d700 10 client.41552.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37810b420 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.544+0000 7ff37db2d700 1 librados: init done 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.545+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff358004f50 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.545+0000 7ff374ff9700 1 -- 172.21.15.59:0/1105270227 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff37000cb10 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff374ff9700 1 --2- 172.21.15.59:0/1105270227 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff364062c90 0x7ff364065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff374ff9700 1 -- 172.21.15.59:0/1105270227 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(628..628 src has 418..628) v4 ==== 8657+0+0 (secure 0 0 0) 0x7ff37007d0f0 con 0x7ff37806b110 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff374ff9700 10 client.41552.objecter ms_dispatch 0x7ff378001550 osd_map(628..628 src has 418..628) v4 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff374ff9700 3 client.41552.objecter handle_osd_map got epochs [628,628] > 0 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff374ff9700 3 client.41552.objecter handle_osd_map decoding full epoch 628 2024-02-18T00:42:48.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff376ffd700 1 --2- 172.21.15.59:0/1105270227 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff364062c90 0x7ff364065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff374ff9700 20 client.41552.objecter dump_active .. 0 homeless 2024-02-18T00:42:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff374ff9700 10 client.41552.objecter ms_handle_connect 0x7ff364062c90 2024-02-18T00:42:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.546+0000 7ff376ffd700 1 --2- 172.21.15.59:0/1105270227 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff364062c90 0x7ff364065150 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 rx=0x7ff360004ee0 tx=0x7ff36000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:48.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.550+0000 7ff374ff9700 1 -- 172.21.15.59:0/1105270227 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff37004af60 con 0x7ff37806b110 2024-02-18T00:42:48.575 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:48.674 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:48.676 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:48.677 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:48.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.741+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7ff358005200 con 0x7ff37806b110 2024-02-18T00:42:48.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.765+0000 7ff374ff9700 1 -- 172.21.15.59:0/1105270227 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v629) v1 ==== 91+0+0 (secure 0 0 0) 0x7ff370050410 con 0x7ff37806b110 2024-02-18T00:42:48.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 10 librados: watch_flush enter 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 10 librados: watch_flush exit 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 20 client.41552.objecter shutdown clearing up homeless session... 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 10 client.41552.objecter successfully canceled tick 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff364062c90 msgr2=0x7ff364065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 --2- 172.21.15.59:0/1105270227 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff364062c90 0x7ff364065150 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 rx=0x7ff360004ee0 tx=0x7ff36000b040).stop 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b110 msgr2=0x7ff3781103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b110 0x7ff3781103b0 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 rx=0x7ff37000e970 tx=0x7ff37000b040).stop 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 shutdown_connections 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 --2- 172.21.15.59:0/1105270227 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff364062c90 0x7ff364065150 unknown :-1 s=CLOSED pgs=3179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3781044a0 0x7ff37810a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff37806b110 0x7ff3781103b0 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 --2- 172.21.15.59:0/1105270227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff37806b9e0 0x7ff3781108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 >> 172.21.15.59:0/1105270227 conn(0x7ff3780f3800 msgr2=0x7ff3780f4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:48.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 shutdown_connections 2024-02-18T00:42:48.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 -- 172.21.15.59:0/1105270227 wait complete. 2024-02-18T00:42:48.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:48.768+0000 7ff37db2d700 1 librados: shutdown 2024-02-18T00:42:48.770 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:42:48.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:48.808 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:49.008 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:49.010 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:49.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:49.111 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:49.143 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:49.164 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:49.166 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:49.167 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:49.267 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:49.295 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:47.975768+0000 mgr.y (mgr.24454) 562 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:48.741396+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.59:0/1105270227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:48.741721+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:48.760146+0000 mon.a (mon.0) 3172 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:48.764369+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:48.764427+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:47.975768+0000 mgr.y (mgr.24454) 562 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:48.741396+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.59:0/1105270227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:49.296 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:48.741721+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:49.297 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:48.760146+0000 mon.a (mon.0) 3172 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:49.297 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:48.764369+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:42:49.297 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:48.764427+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-18T00:42:49.298 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:49.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:47.975768+0000 mgr.y (mgr.24454) 562 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-18T00:42:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:48.741396+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.59:0/1105270227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:48.741721+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:42:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:48.760146+0000 mon.a (mon.0) 3172 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:42:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:48.764369+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:42:49.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:48.764427+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-18T00:42:50.263 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:42:50.263 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:42:50.263 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:42:50.263 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.347689+0000 osd.4 (osd.4) 347 : cluster [DBG] 4.c scrub starts 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.351174+0000 osd.4 (osd.4) 348 : cluster [DBG] 4.c scrub ok 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.448572+0000 osd.0 (osd.0) 276 : cluster [DBG] 25.e scrub starts 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.449840+0000 osd.0 (osd.0) 277 : cluster [DBG] 25.e scrub ok 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.663417+0000 osd.7 (osd.7) 181 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.664374+0000 osd.7 (osd.7) 182 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.673022+0000 osd.2 (osd.2) 249 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:42:50.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.673911+0000 osd.2 (osd.2) 250 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.347689+0000 osd.4 (osd.4) 347 : cluster [DBG] 4.c scrub starts 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.351174+0000 osd.4 (osd.4) 348 : cluster [DBG] 4.c scrub ok 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.448572+0000 osd.0 (osd.0) 276 : cluster [DBG] 25.e scrub starts 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.449840+0000 osd.0 (osd.0) 277 : cluster [DBG] 25.e scrub ok 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.663417+0000 osd.7 (osd.7) 181 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.664374+0000 osd.7 (osd.7) 182 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.673022+0000 osd.2 (osd.2) 249 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.673911+0000 osd.2 (osd.2) 250 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:42:50.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.347689+0000 osd.4 (osd.4) 347 : cluster [DBG] 4.c scrub starts 2024-02-18T00:42:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.351174+0000 osd.4 (osd.4) 348 : cluster [DBG] 4.c scrub ok 2024-02-18T00:42:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.448572+0000 osd.0 (osd.0) 276 : cluster [DBG] 25.e scrub starts 2024-02-18T00:42:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.449840+0000 osd.0 (osd.0) 277 : cluster [DBG] 25.e scrub ok 2024-02-18T00:42:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.663417+0000 osd.7 (osd.7) 181 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:42:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.664374+0000 osd.7 (osd.7) 182 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:42:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.673022+0000 osd.2 (osd.2) 249 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:42:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.673911+0000 osd.2 (osd.2) 250 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:42:50.952 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:50.955 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:50.955 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:50.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-18T00:42:51.056 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.777439+0000 osd.6 (osd.6) 283 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.778673+0000 osd.6 (osd.6) 284 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:49.976135+0000 mgr.y (mgr.24454) 563 : cluster [DBG] pgmap v962: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 0 op/s 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:50.105416+0000 osd.1 (osd.1) 307 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:50.108887+0000 osd.1 (osd.1) 308 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:50.370491+0000 osd.4 (osd.4) 349 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:50.371586+0000 osd.4 (osd.4) 350 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:42:51.061 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.777439+0000 osd.6 (osd.6) 283 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:42:51.062 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.778673+0000 osd.6 (osd.6) 284 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:42:51.062 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:49.976135+0000 mgr.y (mgr.24454) 563 : cluster [DBG] pgmap v962: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 0 op/s 2024-02-18T00:42:51.062 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:50.105416+0000 osd.1 (osd.1) 307 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:42:51.062 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:50.108887+0000 osd.1 (osd.1) 308 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:42:51.062 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:50.370491+0000 osd.4 (osd.4) 349 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:42:51.086 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:51.107 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:51.109 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:51.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:51.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:51.237 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:50.371586+0000 osd.4 (osd.4) 350 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:42:51.239 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:51.437 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:51.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:51.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:51.460 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:42:51.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.777439+0000 osd.6 (osd.6) 283 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:42:51.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.778673+0000 osd.6 (osd.6) 284 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:42:51.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:49.976135+0000 mgr.y (mgr.24454) 563 : cluster [DBG] pgmap v962: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 0 op/s 2024-02-18T00:42:51.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:50.105416+0000 osd.1 (osd.1) 307 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:42:51.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:50.108887+0000 osd.1 (osd.1) 308 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:42:51.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:50.370491+0000 osd.4 (osd.4) 349 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:42:51.461 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:50.371586+0000 osd.4 (osd.4) 350 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:42:51.543 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:51.575 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:51.596 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:51.598 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:51.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:51.700 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:51.728 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:42:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:42:51.730 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:51.750 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:51.753 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:51.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:51.822 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:51.850 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:51.854 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:51.885 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:52.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:42:52.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:51.093117+0000 osd.1 (osd.1) 309 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:42:52.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:51.096006+0000 osd.1 (osd.1) 310 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:42:52.385 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:51.378314+0000 osd.4 (osd.4) 351 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:42:52.386 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:51.380340+0000 osd.4 (osd.4) 352 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:42:52.386 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:51.093117+0000 osd.1 (osd.1) 309 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:42:52.386 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:51.096006+0000 osd.1 (osd.1) 310 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:42:52.386 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:51.378314+0000 osd.4 (osd.4) 351 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:42:52.386 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:51.380340+0000 osd.4 (osd.4) 352 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:42:52.386 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:42:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:42:52] "GET /metrics HTTP/1.1" 200 191233 "" "Prometheus/2.33.4" 2024-02-18T00:42:52.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:51.093117+0000 osd.1 (osd.1) 309 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:42:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:51.096006+0000 osd.1 (osd.1) 310 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:42:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:51.378314+0000 osd.4 (osd.4) 351 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:42:52.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:51.380340+0000 osd.4 (osd.4) 352 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:42:53.035 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:53.038 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:53.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:53.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:53.170 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:53.193 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:53.196 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:53.199 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:53.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 -- 172.21.15.59:0/3082208169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 msgr2=0x7f26180f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 --2- 172.21.15.59:0/3082208169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f26180f8d80 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 rx=0x7f2608004870 tx=0x7f2608009cb0).stop 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 -- 172.21.15.59:0/3082208169 shutdown_connections 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 --2- 172.21.15.59:0/3082208169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f7330 0x7f2618103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 --2- 172.21.15.59:0/3082208169 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f26180f8d80 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 --2- 172.21.15.59:0/3082208169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26180f9270 0x7f26180f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 -- 172.21.15.59:0/3082208169 >> 172.21.15.59:0/3082208169 conn(0x7f26180f26f0 msgr2=0x7f26180f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 -- 172.21.15.59:0/3082208169 shutdown_connections 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.659+0000 7f261d58d700 1 -- 172.21.15.59:0/3082208169 wait complete. 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 librados: starting msgr at 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 librados: starting objecter 2024-02-18T00:42:53.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 Processor -- start 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 -- start start 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 librados: setting wanted keys 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 librados: calling monclient init 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26180f7330 0x7f261810d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f261810d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.660+0000 7f261d58d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f9270 0x7f26181072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f261d58d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618107790 con 0x7f26180f9270 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f261d58d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618107900 con 0x7f26180f7330 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f261d58d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2618107ac0 con 0x7f26180f89a0 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f261810d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26177fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f9270 0x7f26181072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f261810d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:45630/0 (socket says 172.21.15.59:45630) 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 -- 172.21.15.59:0/2310212432 learned_addr learned my addr 172.21.15.59:0/2310212432 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26177fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f9270 0x7f26181072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43602/0 (socket says 172.21.15.59:43602) 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f2616ffd700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26180f7330 0x7f261810d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 -- 172.21.15.59:0/2310212432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26180f7330 msgr2=0x7f261810d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26180f7330 0x7f261810d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 -- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f9270 msgr2=0x7f26181072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f9270 0x7f26181072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 -- 172.21.15.59:0/2310212432 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26100097c0 con 0x7f26180f89a0 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26177fe700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f9270 0x7f26181072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f26167fc700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f261810d5f0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7f260c00b530 tx=0x7f260c0090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f25fffff700 10 client.?.objecter ms_handle_connect 0x7f26180f89a0 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f25fffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:53.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f25fffff700 1 -- 172.21.15.59:0/2310212432 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c002320 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2608005040 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f25fffff700 1 -- 172.21.15.59:0/2310212432 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f260c002950 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f25fffff700 1 -- 172.21.15.59:0/2310212432 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c0129a0 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f261d58d700 10 client.41558.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26181080b0 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.661+0000 7f261d58d700 1 librados: init done 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.662+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261805ee80 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 1 -- 172.21.15.59:0/2310212432 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f260c002480 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 1 --2- 172.21.15.59:0/2310212432 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2600062c90 0x7f2600065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 1 -- 172.21.15.59:0/2310212432 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(629..629 src has 418..629) v4 ==== 8657+0+0 (secure 0 0 0) 0x7f260c07d500 con 0x7f26180f89a0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 10 client.41558.objecter ms_dispatch 0x7f2618001550 osd_map(629..629 src has 418..629) v4 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 3 client.41558.objecter handle_osd_map got epochs [629,629] > 0 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 3 client.41558.objecter handle_osd_map decoding full epoch 629 2024-02-18T00:42:53.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f2616ffd700 1 --2- 172.21.15.59:0/2310212432 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2600062c90 0x7f2600065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:53.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 20 client.41558.objecter dump_active .. 0 homeless 2024-02-18T00:42:53.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f25fffff700 10 client.41558.objecter ms_handle_connect 0x7f2600062c90 2024-02-18T00:42:53.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.663+0000 7f2616ffd700 1 --2- 172.21.15.59:0/2310212432 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2600062c90 0x7f2600065150 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 rx=0x7f26080043d0 tx=0x7f260800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:53.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.667+0000 7f25fffff700 1 -- 172.21.15.59:0/2310212432 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f260c050710 con 0x7f26180f89a0 2024-02-18T00:42:53.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.840+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2618108780 con 0x7f2600062c90 2024-02-18T00:42:53.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:52.855+0000 7f25fffff700 1 -- 172.21.15.59:0/2310212432 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+527197 (secure 0 0 0) 0x7f2618108780 con 0x7f2600062c90 2024-02-18T00:42:53.319 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 10 librados: watch_flush enter 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 10 librados: watch_flush exit 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 20 client.41558.objecter shutdown clearing up homeless session... 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 10 client.41558.objecter successfully canceled tick 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2600062c90 msgr2=0x7f2600065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 --2- 172.21.15.59:0/2310212432 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2600062c90 0x7f2600065150 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 rx=0x7f26080043d0 tx=0x7f260800b040).stop 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 msgr2=0x7f261810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f261810d5f0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 rx=0x7f260c00b530 tx=0x7f260c0090f0).stop 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 shutdown_connections 2024-02-18T00:42:53.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 --2- 172.21.15.59:0/2310212432 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2600062c90 0x7f2600065150 unknown :-1 s=CLOSED pgs=3180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26180f7330 0x7f261810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26180f9270 0x7f26181072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 --2- 172.21.15.59:0/2310212432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f26180f89a0 0x7f261810d5f0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 >> 172.21.15.59:0/2310212432 conn(0x7f26180f26f0 msgr2=0x7f26180fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 shutdown_connections 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 -- 172.21.15.59:0/2310212432 wait complete. 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:53.310+0000 7f261d58d700 1 librados: shutdown 2024-02-18T00:42:53.325 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:42:53.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:51.459439+0000 mgr.y (mgr.24454) 564 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:51.816738+0000 osd.6 (osd.6) 285 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:51.817610+0000 osd.6 (osd.6) 286 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:51.977137+0000 mgr.y (mgr.24454) 565 : cluster [DBG] pgmap v963: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:52.124502+0000 osd.1 (osd.1) 311 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:52.127825+0000 osd.1 (osd.1) 312 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:52.419005+0000 osd.4 (osd.4) 353 : cluster [DBG] 6.e scrub starts 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:52.422082+0000 osd.4 (osd.4) 354 : cluster [DBG] 6.e scrub ok 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:51.459439+0000 mgr.y (mgr.24454) 564 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:51.816738+0000 osd.6 (osd.6) 285 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:51.817610+0000 osd.6 (osd.6) 286 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:42:53.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:51.977137+0000 mgr.y (mgr.24454) 565 : cluster [DBG] pgmap v963: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:53.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:52.124502+0000 osd.1 (osd.1) 311 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:42:53.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:52.127825+0000 osd.1 (osd.1) 312 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:42:53.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:52.419005+0000 osd.4 (osd.4) 353 : cluster [DBG] 6.e scrub starts 2024-02-18T00:42:53.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:52.422082+0000 osd.4 (osd.4) 354 : cluster [DBG] 6.e scrub ok 2024-02-18T00:42:53.330 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:53.431 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:42:53.433 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:53.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:42:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:51.459439+0000 mgr.y (mgr.24454) 564 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:42:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:51.816738+0000 osd.6 (osd.6) 285 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:42:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:51.817610+0000 osd.6 (osd.6) 286 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:42:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:51.977137+0000 mgr.y (mgr.24454) 565 : cluster [DBG] pgmap v963: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:42:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:52.124502+0000 osd.1 (osd.1) 311 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:42:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:52.127825+0000 osd.1 (osd.1) 312 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:42:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:52.419005+0000 osd.4 (osd.4) 353 : cluster [DBG] 6.e scrub starts 2024-02-18T00:42:53.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:52.422082+0000 osd.4 (osd.4) 354 : cluster [DBG] 6.e scrub ok 2024-02-18T00:42:53.534 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:53.567 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:53.588 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:53.591 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:53.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:53.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:53.720 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:52.841029+0000 mgr.y (mgr.24454) 566 : audit [DBG] from='client.41558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:52.998376+0000 osd.3 (osd.3) 323 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:52.999204+0000 osd.3 (osd.3) 324 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:53.459605+0000 osd.0 (osd.0) 278 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:53.461021+0000 osd.0 (osd.0) 279 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:53.600624+0000 osd.2 (osd.2) 251 : cluster [DBG] 2.c scrub starts 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:53.601529+0000 osd.2 (osd.2) 252 : cluster [DBG] 2.c scrub ok 2024-02-18T00:42:54.342 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:52.841029+0000 mgr.y (mgr.24454) 566 : audit [DBG] from='client.41558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:42:54.343 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:52.998376+0000 osd.3 (osd.3) 323 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:42:54.343 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:52.999204+0000 osd.3 (osd.3) 324 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:42:54.343 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:53.459605+0000 osd.0 (osd.0) 278 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:42:54.343 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:53.461021+0000 osd.0 (osd.0) 279 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:42:54.343 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:53.600624+0000 osd.2 (osd.2) 251 : cluster [DBG] 2.c scrub starts 2024-02-18T00:42:54.343 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:53.601529+0000 osd.2 (osd.2) 252 : cluster [DBG] 2.c scrub ok 2024-02-18T00:42:54.370 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:54.372 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:54.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:54.474 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:52.841029+0000 mgr.y (mgr.24454) 566 : audit [DBG] from='client.41558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:42:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:52.998376+0000 osd.3 (osd.3) 323 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:42:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:52.999204+0000 osd.3 (osd.3) 324 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:42:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:53.459605+0000 osd.0 (osd.0) 278 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:42:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:53.461021+0000 osd.0 (osd.0) 279 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:42:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:53.600624+0000 osd.2 (osd.2) 251 : cluster [DBG] 2.c scrub starts 2024-02-18T00:42:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:53.601529+0000 osd.2 (osd.2) 252 : cluster [DBG] 2.c scrub ok 2024-02-18T00:42:54.505 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:54.652 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:54.655 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:54.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:54.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:54.787 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:53.977668+0000 mgr.y (mgr.24454) 567 : cluster [DBG] pgmap v964: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:42:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:54.179629+0000 osd.1 (osd.1) 313 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:42:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:54.180843+0000 osd.1 (osd.1) 314 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:42:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:53.977668+0000 mgr.y (mgr.24454) 567 : cluster [DBG] pgmap v964: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:42:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:54.179629+0000 osd.1 (osd.1) 313 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:42:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:54.180843+0000 osd.1 (osd.1) 314 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:42:55.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:53.977668+0000 mgr.y (mgr.24454) 567 : cluster [DBG] pgmap v964: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:42:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:54.179629+0000 osd.1 (osd.1) 313 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:42:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:54.180843+0000 osd.1 (osd.1) 314 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:42:55.938 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:55.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:55.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:55.955 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":963,"stamp":"2024-02-18T00:42:51.976363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5870,"num_objects":237,"num_object_clones":4,"num_object_copies":711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":1,"num_read":2345,"num_read_kb":2062,"num_write":922,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":4478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1035,"ondisk_log_size":1035,"up":612,"acting":612,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":612,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2477628,"kb_used_data":30204,"kb_used_omap":0,"kb_used_meta":2447360,"kb_avail":747483588,"statfs":{"total":767960285184,"available":765423194112,"internally_reserved":0,"allocated":30928896,"data_stored":15997817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506096640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002178"},"pg_stats":[{"pgid":"237.1f","version":"0'0","reported_seq":25,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.991176+0000","last_change":"2024-02-18T00:42:44.990911+0000","last_active":"2024-02-18T00:42:44.991176+0000","last_peered":"2024-02-18T00:42:44.991176+0000","last_clean":"2024-02-18T00:42:44.991176+0000","last_became_active":"2024-02-18T00:42:44.990546+0000","last_became_peered":"2024-02-18T00:42:44.990546+0000","last_unstale":"2024-02-18T00:42:44.991176+0000","last_undegraded":"2024-02-18T00:42:44.991176+0000","last_fullsized":"2024-02-18T00:42:44.991176+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"237.1e","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507230+0000","last_change":"2024-02-18T00:42:35.859307+0000","last_active":"2024-02-18T00:42:44.507230+0000","last_peered":"2024-02-18T00:42:44.507230+0000","last_clean":"2024-02-18T00:42:44.507230+0000","last_became_active":"2024-02-18T00:42:35.858845+0000","last_became_peered":"2024-02-18T00:42:35.858845+0000","last_unstale":"2024-02-18T00:42:44.507230+0000","last_undegraded":"2024-02-18T00:42:44.507230+0000","last_fullsized":"2024-02-18T00:42:44.507230+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"237.1d","version":"0'0","reported_seq":25,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990655+0000","last_change":"2024-02-18T00:42:44.990367+0000","last_active":"2024-02-18T00:42:44.990655+0000","last_peered":"2024-02-18T00:42:44.990655+0000","last_clean":"2024-02-18T00:42:44.990655+0000","last_became_active":"2024-02-18T00:42:44.990063+0000","last_became_peered":"2024-02-18T00:42:44.990063+0000","last_unstale":"2024-02-18T00:42:44.990655+0000","last_undegraded":"2024-02-18T00:42:44.990655+0000","last_fullsized":"2024-02-18T00:42:44.990655+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"237.1c","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035910+0000","last_change":"2024-02-18T00:42:35.855686+0000","last_active":"2024-02-18T00:42:44.035910+0000","last_peered":"2024-02-18T00:42:44.035910+0000","last_clean":"2024-02-18T00:42:44.035910+0000","last_became_active":"2024-02-18T00:42:35.855324+0000","last_became_peered":"2024-02-18T00:42:35.855324+0000","last_unstale":"2024-02-18T00:42:44.035910+0000","last_undegraded":"2024-02-18T00:42:44.035910+0000","last_fullsized":"2024-02-18T00:42:44.035910+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"237.1b","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036253+0000","last_change":"2024-02-18T00:42:35.857411+0000","last_active":"2024-02-18T00:42:44.036253+0000","last_peered":"2024-02-18T00:42:44.036253+0000","last_clean":"2024-02-18T00:42:44.036253+0000","last_became_active":"2024-02-18T00:42:35.857059+0000","last_became_peered":"2024-02-18T00:42:35.857059+0000","last_unstale":"2024-02-18T00:42:44.036253+0000","last_undegraded":"2024-02-18T00:42:44.036253+0000","last_fullsized":"2024-02-18T00:42:44.036253+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"237.1a","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690290+0000","last_change":"2024-02-18T00:42:35.861667+0000","last_active":"2024-02-18T00:42:44.690290+0000","last_peered":"2024-02-18T00:42:44.690290+0000","last_clean":"2024-02-18T00:42:44.690290+0000","last_became_active":"2024-02-18T00:42:35.861354+0000","last_became_peered":"2024-02-18T00:42:35.861354+0000","last_unstale":"2024-02-18T00:42:44.690290+0000","last_undegraded":"2024-02-18T00:42:44.690290+0000","last_fullsized":"2024-02-18T00:42:44.690290+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"237.19","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739114+0000","last_change":"2024-02-18T00:42:35.856205+0000","last_active":"2024-02-18T00:42:44.739114+0000","last_peered":"2024-02-18T00:42:44.739114+0000","last_clean":"2024-02-18T00:42:44.739114+0000","last_became_active":"2024-02-18T00:42:35.855912+0000","last_became_peered":"2024-02-18T00:42:35.855912+0000","last_unstale":"2024-02-18T00:42:44.739114+0000","last_undegraded":"2024-02-18T00:42:44.739114+0000","last_fullsized":"2024-02-18T00:42:44.739114+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"237.18","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689221+0000","last_change":"2024-02-18T00:42:35.860160+0000","last_active":"2024-02-18T00:42:44.689221+0000","last_peered":"2024-02-18T00:42:44.689221+0000","last_clean":"2024-02-18T00:42:44.689221+0000","last_became_active":"2024-02-18T00:42:35.859817+0000","last_became_peered":"2024-02-18T00:42:35.859817+0000","last_unstale":"2024-02-18T00:42:44.689221+0000","last_undegraded":"2024-02-18T00:42:44.689221+0000","last_fullsized":"2024-02-18T00:42:44.689221+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"237.17","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690603+0000","last_change":"2024-02-18T00:42:35.859943+0000","last_active":"2024-02-18T00:42:44.690603+0000","last_peered":"2024-02-18T00:42:44.690603+0000","last_clean":"2024-02-18T00:42:44.690603+0000","last_became_active":"2024-02-18T00:42:35.859646+0000","last_became_peered":"2024-02-18T00:42:35.859646+0000","last_unstale":"2024-02-18T00:42:44.690603+0000","last_undegraded":"2024-02-18T00:42:44.690603+0000","last_fullsized":"2024-02-18T00:42:44.690603+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"237.16","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036824+0000","last_change":"2024-02-18T00:42:35.858862+0000","last_active":"2024-02-18T00:42:44.036824+0000","last_peered":"2024-02-18T00:42:44.036824+0000","last_clean":"2024-02-18T00:42:44.036824+0000","last_became_active":"2024-02-18T00:42:35.858600+0000","last_became_peered":"2024-02-18T00:42:35.858600+0000","last_unstale":"2024-02-18T00:42:44.036824+0000","last_undegraded":"2024-02-18T00:42:44.036824+0000","last_fullsized":"2024-02-18T00:42:44.036824+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"237.15","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507160+0000","last_change":"2024-02-18T00:42:35.859884+0000","last_active":"2024-02-18T00:42:44.507160+0000","last_peered":"2024-02-18T00:42:44.507160+0000","last_clean":"2024-02-18T00:42:44.507160+0000","last_became_active":"2024-02-18T00:42:35.859683+0000","last_became_peered":"2024-02-18T00:42:35.859683+0000","last_unstale":"2024-02-18T00:42:44.507160+0000","last_undegraded":"2024-02-18T00:42:44.507160+0000","last_fullsized":"2024-02-18T00:42:44.507160+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"237.14","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033090+0000","last_change":"2024-02-18T00:42:35.857750+0000","last_active":"2024-02-18T00:42:44.033090+0000","last_peered":"2024-02-18T00:42:44.033090+0000","last_clean":"2024-02-18T00:42:44.033090+0000","last_became_active":"2024-02-18T00:42:35.857447+0000","last_became_peered":"2024-02-18T00:42:35.857447+0000","last_unstale":"2024-02-18T00:42:44.033090+0000","last_undegraded":"2024-02-18T00:42:44.033090+0000","last_fullsized":"2024-02-18T00:42:44.033090+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,3],"acting":[2,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"237.13","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739434+0000","last_change":"2024-02-18T00:42:35.862345+0000","last_active":"2024-02-18T00:42:44.739434+0000","last_peered":"2024-02-18T00:42:44.739434+0000","last_clean":"2024-02-18T00:42:44.739434+0000","last_became_active":"2024-02-18T00:42:35.862060+0000","last_became_peered":"2024-02-18T00:42:35.862060+0000","last_unstale":"2024-02-18T00:42:44.739434+0000","last_undegraded":"2024-02-18T00:42:44.739434+0000","last_fullsized":"2024-02-18T00:42:44.739434+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"237.12","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506411+0000","last_change":"2024-02-18T00:42:35.856348+0000","last_active":"2024-02-18T00:42:44.506411+0000","last_peered":"2024-02-18T00:42:44.506411+0000","last_clean":"2024-02-18T00:42:44.506411+0000","last_became_active":"2024-02-18T00:42:35.856058+0000","last_became_peered":"2024-02-18T00:42:35.856058+0000","last_unstale":"2024-02-18T00:42:44.506411+0000","last_undegraded":"2024-02-18T00:42:44.506411+0000","last_fullsized":"2024-02-18T00:42:44.506411+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"237.11","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739120+0000","last_change":"2024-02-18T00:42:35.859503+0000","last_active":"2024-02-18T00:42:44.739120+0000","last_peered":"2024-02-18T00:42:44.739120+0000","last_clean":"2024-02-18T00:42:44.739120+0000","last_became_active":"2024-02-18T00:42:35.859212+0000","last_became_peered":"2024-02-18T00:42:35.859212+0000","last_unstale":"2024-02-18T00:42:44.739120+0000","last_undegraded":"2024-02-18T00:42:44.739120+0000","last_fullsized":"2024-02-18T00:42:44.739120+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"237.10","version":"626'1","reported_seq":18,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035287+0000","last_change":"2024-02-18T00:42:35.857998+0000","last_active":"2024-02-18T00:42:44.035287+0000","last_peered":"2024-02-18T00:42:44.035287+0000","last_clean":"2024-02-18T00:42:44.035287+0000","last_became_active":"2024-02-18T00:42:35.857663+0000","last_became_peered":"2024-02-18T00:42:35.857663+0000","last_unstale":"2024-02-18T00:42:44.035287+0000","last_undegraded":"2024-02-18T00:42:44.035287+0000","last_fullsized":"2024-02-18T00:42:44.035287+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"237.f","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507281+0000","last_change":"2024-02-18T00:42:35.858074+0000","last_active":"2024-02-18T00:42:44.507281+0000","last_peered":"2024-02-18T00:42:44.507281+0000","last_clean":"2024-02-18T00:42:44.507281+0000","last_became_active":"2024-02-18T00:42:35.857793+0000","last_became_peered":"2024-02-18T00:42:35.857793+0000","last_unstale":"2024-02-18T00:42:44.507281+0000","last_undegraded":"2024-02-18T00:42:44.507281+0000","last_fullsized":"2024-02-18T00:42:44.507281+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"237.e","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.740085+0000","last_change":"2024-02-18T00:42:35.860312+0000","last_active":"2024-02-18T00:42:44.740085+0000","last_peered":"2024-02-18T00:42:44.740085+0000","last_clean":"2024-02-18T00:42:44.740085+0000","last_became_active":"2024-02-18T00:42:35.859984+0000","last_became_peered":"2024-02-18T00:42:35.859984+0000","last_unstale":"2024-02-18T00:42:44.740085+0000","last_undegraded":"2024-02-18T00:42:44.740085+0000","last_fullsized":"2024-02-18T00:42:44.740085+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"237.d","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036968+0000","last_change":"2024-02-18T00:42:35.860280+0000","last_active":"2024-02-18T00:42:44.036968+0000","last_peered":"2024-02-18T00:42:44.036968+0000","last_clean":"2024-02-18T00:42:44.036968+0000","last_became_active":"2024-02-18T00:42:35.859968+0000","last_became_peered":"2024-02-18T00:42:35.859968+0000","last_unstale":"2024-02-18T00:42:44.036968+0000","last_undegraded":"2024-02-18T00:42:44.036968+0000","last_fullsized":"2024-02-18T00:42:44.036968+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"237.c","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035909+0000","last_change":"2024-02-18T00:42:35.858100+0000","last_active":"2024-02-18T00:42:44.035909+0000","last_peered":"2024-02-18T00:42:44.035909+0000","last_clean":"2024-02-18T00:42:44.035909+0000","last_became_active":"2024-02-18T00:42:35.857801+0000","last_became_peered":"2024-02-18T00:42:35.857801+0000","last_unstale":"2024-02-18T00:42:44.035909+0000","last_undegraded":"2024-02-18T00:42:44.035909+0000","last_fullsized":"2024-02-18T00:42:44.035909+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"237.b","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990586+0000","last_change":"2024-02-18T00:42:35.856564+0000","last_active":"2024-02-18T00:42:44.990586+0000","last_peered":"2024-02-18T00:42:44.990586+0000","last_clean":"2024-02-18T00:42:44.990586+0000","last_became_active":"2024-02-18T00:42:35.856223+0000","last_became_peered":"2024-02-18T00:42:35.856223+0000","last_unstale":"2024-02-18T00:42:44.990586+0000","last_undegraded":"2024-02-18T00:42:44.990586+0000","last_fullsized":"2024-02-18T00:42:44.990586+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"237.a","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.031851+0000","last_change":"2024-02-18T00:42:35.853836+0000","last_active":"2024-02-18T00:42:44.031851+0000","last_peered":"2024-02-18T00:42:44.031851+0000","last_clean":"2024-02-18T00:42:44.031851+0000","last_became_active":"2024-02-18T00:42:35.853478+0000","last_became_peered":"2024-02-18T00:42:35.853478+0000","last_unstale":"2024-02-18T00:42:44.031851+0000","last_undegraded":"2024-02-18T00:42:44.031851+0000","last_fullsized":"2024-02-18T00:42:44.031851+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,4],"acting":[6,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"237.9","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689198+0000","last_change":"2024-02-18T00:42:35.857202+0000","last_active":"2024-02-18T00:42:44.689198+0000","last_peered":"2024-02-18T00:42:44.689198+0000","last_clean":"2024-02-18T00:42:44.689198+0000","last_became_active":"2024-02-18T00:42:35.856419+0000","last_became_peered":"2024-02-18T00:42:35.856419+0000","last_unstale":"2024-02-18T00:42:44.689198+0000","last_undegraded":"2024-02-18T00:42:44.689198+0000","last_fullsized":"2024-02-18T00:42:44.689198+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"237.8","version":"626'1","reported_seq":18,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989582+0000","last_change":"2024-02-18T00:42:35.855433+0000","last_active":"2024-02-18T00:42:44.989582+0000","last_peered":"2024-02-18T00:42:44.989582+0000","last_clean":"2024-02-18T00:42:44.989582+0000","last_became_active":"2024-02-18T00:42:35.855125+0000","last_became_peered":"2024-02-18T00:42:35.855125+0000","last_unstale":"2024-02-18T00:42:44.989582+0000","last_undegraded":"2024-02-18T00:42:44.989582+0000","last_fullsized":"2024-02-18T00:42:44.989582+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"237.0","version":"0'0","reported_seq":21,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032875+0000","last_change":"2024-02-18T00:42:40.740068+0000","last_active":"2024-02-18T00:42:44.032875+0000","last_peered":"2024-02-18T00:42:44.032875+0000","last_clean":"2024-02-18T00:42:44.032875+0000","last_became_active":"2024-02-18T00:42:35.857346+0000","last_became_peered":"2024-02-18T00:42:35.857346+0000","last_unstale":"2024-02-18T00:42:44.032875+0000","last_undegraded":"2024-02-18T00:42:44.032875+0000","last_fullsized":"2024-02-18T00:42:44.032875+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:40.740003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:40.740003+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"234.7","version":"0'0","reported_seq":31,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690125+0000","last_change":"2024-02-18T00:42:19.768305+0000","last_active":"2024-02-18T00:42:44.690125+0000","last_peered":"2024-02-18T00:42:44.690125+0000","last_clean":"2024-02-18T00:42:44.690125+0000","last_became_active":"2024-02-18T00:42:19.767995+0000","last_became_peered":"2024-02-18T00:42:19.767995+0000","last_unstale":"2024-02-18T00:42:44.690125+0000","last_undegraded":"2024-02-18T00:42:44.690125+0000","last_fullsized":"2024-02-18T00:42:44.690125+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"237.1","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739219+0000","last_change":"2024-02-18T00:42:35.858875+0000","last_active":"2024-02-18T00:42:44.739219+0000","last_peered":"2024-02-18T00:42:44.739219+0000","last_clean":"2024-02-18T00:42:44.739219+0000","last_became_active":"2024-02-18T00:42:35.858468+0000","last_became_peered":"2024-02-18T00:42:35.858468+0000","last_unstale":"2024-02-18T00:42:44.739219+0000","last_undegraded":"2024-02-18T00:42:44.739219+0000","last_fullsized":"2024-02-18T00:42:44.739219+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"234.6","version":"0'0","reported_seq":31,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032384+0000","last_change":"2024-02-18T00:42:19.766657+0000","last_active":"2024-02-18T00:42:44.032384+0000","last_peered":"2024-02-18T00:42:44.032384+0000","last_clean":"2024-02-18T00:42:44.032384+0000","last_became_active":"2024-02-18T00:42:19.766349+0000","last_became_peered":"2024-02-18T00:42:19.766349+0000","last_unstale":"2024-02-18T00:42:44.032384+0000","last_undegraded":"2024-02-18T00:42:44.032384+0000","last_fullsized":"2024-02-18T00:42:44.032384+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"237.2","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989791+0000","last_change":"2024-02-18T00:42:35.864071+0000","last_active":"2024-02-18T00:42:44.989791+0000","last_peered":"2024-02-18T00:42:44.989791+0000","last_clean":"2024-02-18T00:42:44.989791+0000","last_became_active":"2024-02-18T00:42:35.863822+0000","last_became_peered":"2024-02-18T00:42:35.863822+0000","last_unstale":"2024-02-18T00:42:44.989791+0000","last_undegraded":"2024-02-18T00:42:44.989791+0000","last_fullsized":"2024-02-18T00:42:44.989791+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"234.5","version":"0'0","reported_seq":32,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032881+0000","last_change":"2024-02-18T00:42:19.764321+0000","last_active":"2024-02-18T00:42:44.032881+0000","last_peered":"2024-02-18T00:42:44.032881+0000","last_clean":"2024-02-18T00:42:44.032881+0000","last_became_active":"2024-02-18T00:42:19.763983+0000","last_became_peered":"2024-02-18T00:42:19.763983+0000","last_unstale":"2024-02-18T00:42:44.032881+0000","last_undegraded":"2024-02-18T00:42:44.032881+0000","last_fullsized":"2024-02-18T00:42:44.032881+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"237.3","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036446+0000","last_change":"2024-02-18T00:42:35.861357+0000","last_active":"2024-02-18T00:42:44.036446+0000","last_peered":"2024-02-18T00:42:44.036446+0000","last_clean":"2024-02-18T00:42:44.036446+0000","last_became_active":"2024-02-18T00:42:35.859511+0000","last_became_peered":"2024-02-18T00:42:35.859511+0000","last_unstale":"2024-02-18T00:42:44.036446+0000","last_undegraded":"2024-02-18T00:42:44.036446+0000","last_fullsized":"2024-02-18T00:42:44.036446+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.4","version":"0'0","reported_seq":31,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990199+0000","last_change":"2024-02-18T00:42:19.768281+0000","last_active":"2024-02-18T00:42:44.990199+0000","last_peered":"2024-02-18T00:42:44.990199+0000","last_clean":"2024-02-18T00:42:44.990199+0000","last_became_active":"2024-02-18T00:42:19.767925+0000","last_became_peered":"2024-02-18T00:42:19.767925+0000","last_unstale":"2024-02-18T00:42:44.990199+0000","last_undegraded":"2024-02-18T00:42:44.990199+0000","last_fullsized":"2024-02-18T00:42:44.990199+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"237.4","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033187+0000","last_change":"2024-02-18T00:42:35.862807+0000","last_active":"2024-02-18T00:42:44.033187+0000","last_peered":"2024-02-18T00:42:44.033187+0000","last_clean":"2024-02-18T00:42:44.033187+0000","last_became_active":"2024-02-18T00:42:35.862481+0000","last_became_peered":"2024-02-18T00:42:35.862481+0000","last_unstale":"2024-02-18T00:42:44.033187+0000","last_undegraded":"2024-02-18T00:42:44.033187+0000","last_fullsized":"2024-02-18T00:42:44.033187+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"234.3","version":"0'0","reported_seq":32,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739340+0000","last_change":"2024-02-18T00:42:19.768085+0000","last_active":"2024-02-18T00:42:44.739340+0000","last_peered":"2024-02-18T00:42:44.739340+0000","last_clean":"2024-02-18T00:42:44.739340+0000","last_became_active":"2024-02-18T00:42:19.767739+0000","last_became_peered":"2024-02-18T00:42:19.767739+0000","last_unstale":"2024-02-18T00:42:44.739340+0000","last_undegraded":"2024-02-18T00:42:44.739340+0000","last_fullsized":"2024-02-18T00:42:44.739340+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"237.5","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689497+0000","last_change":"2024-02-18T00:42:35.861586+0000","last_active":"2024-02-18T00:42:44.689497+0000","last_peered":"2024-02-18T00:42:44.689497+0000","last_clean":"2024-02-18T00:42:44.689497+0000","last_became_active":"2024-02-18T00:42:35.861277+0000","last_became_peered":"2024-02-18T00:42:35.861277+0000","last_unstale":"2024-02-18T00:42:44.689497+0000","last_undegraded":"2024-02-18T00:42:44.689497+0000","last_fullsized":"2024-02-18T00:42:44.689497+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.2","version":"623'7","reported_seq":38,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.031885+0000","last_change":"2024-02-18T00:42:19.764286+0000","last_active":"2024-02-18T00:42:44.031885+0000","last_peered":"2024-02-18T00:42:44.031885+0000","last_clean":"2024-02-18T00:42:44.031885+0000","last_became_active":"2024-02-18T00:42:19.763974+0000","last_became_peered":"2024-02-18T00:42:19.763974+0000","last_unstale":"2024-02-18T00:42:44.031885+0000","last_undegraded":"2024-02-18T00:42:44.031885+0000","last_fullsized":"2024-02-18T00:42:44.031885+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":5,"num_object_clones":2,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"237.6","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032354+0000","last_change":"2024-02-18T00:42:35.861076+0000","last_active":"2024-02-18T00:42:44.032354+0000","last_peered":"2024-02-18T00:42:44.032354+0000","last_clean":"2024-02-18T00:42:44.032354+0000","last_became_active":"2024-02-18T00:42:35.860784+0000","last_became_peered":"2024-02-18T00:42:35.860784+0000","last_unstale":"2024-02-18T00:42:44.032354+0000","last_undegraded":"2024-02-18T00:42:44.032354+0000","last_fullsized":"2024-02-18T00:42:44.032354+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.1","version":"607'1","reported_seq":33,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989679+0000","last_change":"2024-02-18T00:42:19.764293+0000","last_active":"2024-02-18T00:42:44.989679+0000","last_peered":"2024-02-18T00:42:44.989679+0000","last_clean":"2024-02-18T00:42:44.989679+0000","last_became_active":"2024-02-18T00:42:19.763959+0000","last_became_peered":"2024-02-18T00:42:19.763959+0000","last_unstale":"2024-02-18T00:42:44.989679+0000","last_undegraded":"2024-02-18T00:42:44.989679+0000","last_fullsized":"2024-02-18T00:42:44.989679+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"237.7","version":"0'0","reported_seq":16,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036747+0000","last_change":"2024-02-18T00:42:35.860015+0000","last_active":"2024-02-18T00:42:44.036747+0000","last_peered":"2024-02-18T00:42:44.036747+0000","last_clean":"2024-02-18T00:42:44.036747+0000","last_became_active":"2024-02-18T00:42:35.859733+0000","last_became_peered":"2024-02-18T00:42:35.859733+0000","last_unstale":"2024-02-18T00:42:44.036747+0000","last_undegraded":"2024-02-18T00:42:44.036747+0000","last_fullsized":"2024-02-18T00:42:44.036747+0000","mapping_epoch":621,"log_start":"0'0","ondisk_log_start":"0'0","created":621,"last_epoch_clean":622,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:34.833622+0000","last_clean_scrub_stamp":"2024-02-18T00:42:34.833622+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.0","version":"623'6","reported_seq":40,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035305+0000","last_change":"2024-02-18T00:42:19.768201+0000","last_active":"2024-02-18T00:42:44.035305+0000","last_peered":"2024-02-18T00:42:44.035305+0000","last_clean":"2024-02-18T00:42:44.035305+0000","last_became_active":"2024-02-18T00:42:19.767896+0000","last_became_peered":"2024-02-18T00:42:19.767896+0000","last_unstale":"2024-02-18T00:42:44.035305+0000","last_undegraded":"2024-02-18T00:42:44.035305+0000","last_fullsized":"2024-02-18T00:42:44.035305+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":559,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.031760+0000","last_change":"2024-02-18T00:41:36.776506+0000","last_active":"2024-02-18T00:42:44.031760+0000","last_peered":"2024-02-18T00:42:44.031760+0000","last_clean":"2024-02-18T00:42:44.031760+0000","last_became_active":"2024-02-18T00:39:36.751760+0000","last_became_peered":"2024-02-18T00:39:36.751760+0000","last_unstale":"2024-02-18T00:42:44.031760+0000","last_undegraded":"2024-02-18T00:42:44.031760+0000","last_fullsized":"2024-02-18T00:42:44.031760+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:36.776429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.776429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":551,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989675+0000","last_change":"2024-02-18T00:40:41.127499+0000","last_active":"2024-02-18T00:42:44.989675+0000","last_peered":"2024-02-18T00:42:44.989675+0000","last_clean":"2024-02-18T00:42:44.989675+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:42:44.989675+0000","last_undegraded":"2024-02-18T00:42:44.989675+0000","last_fullsized":"2024-02-18T00:42:44.989675+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:41.127432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:41.127432+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690817+0000","last_change":"2024-02-18T00:40:52.593461+0000","last_active":"2024-02-18T00:42:44.690817+0000","last_peered":"2024-02-18T00:42:44.690817+0000","last_clean":"2024-02-18T00:42:44.690817+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:42:44.690817+0000","last_undegraded":"2024-02-18T00:42:44.690817+0000","last_fullsized":"2024-02-18T00:42:44.690817+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.593401+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690001+0000","last_change":"2024-02-18T00:41:27.570515+0000","last_active":"2024-02-18T00:42:44.690001+0000","last_peered":"2024-02-18T00:42:44.690001+0000","last_clean":"2024-02-18T00:42:44.690001+0000","last_became_active":"2024-02-18T00:39:36.760004+0000","last_became_peered":"2024-02-18T00:39:36.760004+0000","last_unstale":"2024-02-18T00:42:44.690001+0000","last_undegraded":"2024-02-18T00:42:44.690001+0000","last_fullsized":"2024-02-18T00:42:44.690001+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:27.570440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.570440+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":559,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036458+0000","last_change":"2024-02-18T00:41:37.067790+0000","last_active":"2024-02-18T00:42:44.036458+0000","last_peered":"2024-02-18T00:42:44.036458+0000","last_clean":"2024-02-18T00:42:44.036458+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:42:44.036458+0000","last_undegraded":"2024-02-18T00:42:44.036458+0000","last_fullsized":"2024-02-18T00:42:44.036458+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:41:37.067689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:37.067689+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":559,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032265+0000","last_change":"2024-02-18T00:41:34.741697+0000","last_active":"2024-02-18T00:42:44.032265+0000","last_peered":"2024-02-18T00:42:44.032265+0000","last_clean":"2024-02-18T00:42:44.032265+0000","last_became_active":"2024-02-18T00:39:36.751486+0000","last_became_peered":"2024-02-18T00:39:36.751486+0000","last_unstale":"2024-02-18T00:42:44.032265+0000","last_undegraded":"2024-02-18T00:42:44.032265+0000","last_fullsized":"2024-02-18T00:42:44.032265+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.741618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.741618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":538,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990675+0000","last_change":"2024-02-18T00:42:22.876124+0000","last_active":"2024-02-18T00:42:44.990675+0000","last_peered":"2024-02-18T00:42:44.990675+0000","last_clean":"2024-02-18T00:42:44.990675+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:42:44.990675+0000","last_undegraded":"2024-02-18T00:42:44.990675+0000","last_fullsized":"2024-02-18T00:42:44.990675+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.876055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.876055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":543,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032440+0000","last_change":"2024-02-18T00:40:51.610695+0000","last_active":"2024-02-18T00:42:44.032440+0000","last_peered":"2024-02-18T00:42:44.032440+0000","last_clean":"2024-02-18T00:42:44.032440+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:42:44.032440+0000","last_undegraded":"2024-02-18T00:42:44.032440+0000","last_fullsized":"2024-02-18T00:42:44.032440+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.610630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.610630+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":636,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032670+0000","last_change":"2024-02-18T00:42:25.770240+0000","last_active":"2024-02-18T00:42:44.032670+0000","last_peered":"2024-02-18T00:42:44.032670+0000","last_clean":"2024-02-18T00:42:44.032670+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:42:44.032670+0000","last_undegraded":"2024-02-18T00:42:44.032670+0000","last_fullsized":"2024-02-18T00:42:44.032670+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.770128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":177,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:49.664459+0000","last_change":"2024-02-18T00:42:49.664459+0000","last_active":"2024-02-18T00:42:49.664459+0000","last_peered":"2024-02-18T00:42:49.664459+0000","last_clean":"2024-02-18T00:42:49.664459+0000","last_became_active":"2024-02-18T00:39:36.769282+0000","last_became_peered":"2024-02-18T00:39:36.769282+0000","last_unstale":"2024-02-18T00:42:49.664459+0000","last_undegraded":"2024-02-18T00:42:49.664459+0000","last_fullsized":"2024-02-18T00:42:49.664459+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.664380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.664380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":559,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507140+0000","last_change":"2024-02-18T00:41:23.006759+0000","last_active":"2024-02-18T00:42:44.507140+0000","last_peered":"2024-02-18T00:42:44.507140+0000","last_clean":"2024-02-18T00:42:44.507140+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:42:44.507140+0000","last_undegraded":"2024-02-18T00:42:44.507140+0000","last_fullsized":"2024-02-18T00:42:44.507140+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.006689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:28.691955+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.006689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"466'10","reported_seq":220,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430555+0000","last_change":"2024-02-18T00:42:16.577349+0000","last_active":"2024-02-18T00:42:48.430555+0000","last_peered":"2024-02-18T00:42:48.430555+0000","last_clean":"2024-02-18T00:42:48.430555+0000","last_became_active":"2024-02-18T00:39:36.757493+0000","last_became_peered":"2024-02-18T00:39:36.757493+0000","last_unstale":"2024-02-18T00:42:48.430555+0000","last_undegraded":"2024-02-18T00:42:48.430555+0000","last_fullsized":"2024-02-18T00:42:48.430555+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:42:16.577265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:16.577265+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":727,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739078+0000","last_change":"2024-02-18T00:40:40.557614+0000","last_active":"2024-02-18T00:42:44.739078+0000","last_peered":"2024-02-18T00:42:44.739078+0000","last_clean":"2024-02-18T00:42:44.739078+0000","last_became_active":"2024-02-18T00:39:36.755699+0000","last_became_peered":"2024-02-18T00:39:36.755699+0000","last_unstale":"2024-02-18T00:42:44.739078+0000","last_undegraded":"2024-02-18T00:42:44.739078+0000","last_fullsized":"2024-02-18T00:42:44.739078+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:40:40.557544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.557544+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":57,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":624,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033307+0000","last_change":"2024-02-18T00:40:48.410752+0000","last_active":"2024-02-18T00:42:44.033307+0000","last_peered":"2024-02-18T00:42:44.033307+0000","last_clean":"2024-02-18T00:42:44.033307+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:42:44.033307+0000","last_undegraded":"2024-02-18T00:42:44.033307+0000","last_fullsized":"2024-02-18T00:42:44.033307+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:48.410676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:48.410676+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":615,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739319+0000","last_change":"2024-02-18T00:42:15.647876+0000","last_active":"2024-02-18T00:42:44.739319+0000","last_peered":"2024-02-18T00:42:44.739319+0000","last_clean":"2024-02-18T00:42:44.739319+0000","last_became_active":"2024-02-18T00:41:45.788049+0000","last_became_peered":"2024-02-18T00:41:45.788049+0000","last_unstale":"2024-02-18T00:42:44.739319+0000","last_undegraded":"2024-02-18T00:42:44.739319+0000","last_fullsized":"2024-02-18T00:42:44.739319+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":572,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:15.647808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:15.647808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":548,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036960+0000","last_change":"2024-02-18T00:40:50.652492+0000","last_active":"2024-02-18T00:42:44.036960+0000","last_peered":"2024-02-18T00:42:44.036960+0000","last_clean":"2024-02-18T00:42:44.036960+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:42:44.036960+0000","last_undegraded":"2024-02-18T00:42:44.036960+0000","last_fullsized":"2024-02-18T00:42:44.036960+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.652414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.652414+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":659,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507090+0000","last_change":"2024-02-18T00:41:19.076836+0000","last_active":"2024-02-18T00:42:44.507090+0000","last_peered":"2024-02-18T00:42:44.507090+0000","last_clean":"2024-02-18T00:42:44.507090+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:42:44.507090+0000","last_undegraded":"2024-02-18T00:42:44.507090+0000","last_fullsized":"2024-02-18T00:42:44.507090+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:41:19.076766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:19.076766+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":611,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.037322+0000","last_change":"2024-02-18T00:41:23.794790+0000","last_active":"2024-02-18T00:42:44.037322+0000","last_peered":"2024-02-18T00:42:44.037322+0000","last_clean":"2024-02-18T00:42:44.037322+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:42:44.037322+0000","last_undegraded":"2024-02-18T00:42:44.037322+0000","last_fullsized":"2024-02-18T00:42:44.037322+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.794682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.794682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":600,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032659+0000","last_change":"2024-02-18T00:40:46.382314+0000","last_active":"2024-02-18T00:42:44.032659+0000","last_peered":"2024-02-18T00:42:44.032659+0000","last_clean":"2024-02-18T00:42:44.032659+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:42:44.032659+0000","last_undegraded":"2024-02-18T00:42:44.032659+0000","last_fullsized":"2024-02-18T00:42:44.032659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:46.382246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.382246+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":639,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506616+0000","last_change":"2024-02-18T00:41:49.824585+0000","last_active":"2024-02-18T00:42:44.506616+0000","last_peered":"2024-02-18T00:42:44.506616+0000","last_clean":"2024-02-18T00:42:44.506616+0000","last_became_active":"2024-02-18T00:41:49.824190+0000","last_became_peered":"2024-02-18T00:41:49.824190+0000","last_unstale":"2024-02-18T00:42:44.506616+0000","last_undegraded":"2024-02-18T00:42:44.506616+0000","last_fullsized":"2024-02-18T00:42:44.506616+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:16.155189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:16.155189+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":553,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990904+0000","last_change":"2024-02-18T00:41:36.841374+0000","last_active":"2024-02-18T00:42:44.990904+0000","last_peered":"2024-02-18T00:42:44.990904+0000","last_clean":"2024-02-18T00:42:44.990904+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:42:44.990904+0000","last_undegraded":"2024-02-18T00:42:44.990904+0000","last_fullsized":"2024-02-18T00:42:44.990904+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.841306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"457'42","reported_seq":734,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032576+0000","last_change":"2024-02-18T00:40:42.632457+0000","last_active":"2024-02-18T00:42:44.032576+0000","last_peered":"2024-02-18T00:42:44.032576+0000","last_clean":"2024-02-18T00:42:44.032576+0000","last_became_active":"2024-02-18T00:39:36.751206+0000","last_became_peered":"2024-02-18T00:39:36.751206+0000","last_unstale":"2024-02-18T00:42:44.032576+0000","last_undegraded":"2024-02-18T00:42:44.032576+0000","last_fullsized":"2024-02-18T00:42:44.032576+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-18T00:40:42.632357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.632357+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":604,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033438+0000","last_change":"2024-02-18T00:39:50.125802+0000","last_active":"2024-02-18T00:42:44.033438+0000","last_peered":"2024-02-18T00:42:44.033438+0000","last_clean":"2024-02-18T00:42:44.033438+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:42:44.033438+0000","last_undegraded":"2024-02-18T00:42:44.033438+0000","last_fullsized":"2024-02-18T00:42:44.033438+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:50.125735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:39:50.125735+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":600,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032868+0000","last_change":"2024-02-18T00:40:51.349216+0000","last_active":"2024-02-18T00:42:44.032868+0000","last_peered":"2024-02-18T00:42:44.032868+0000","last_clean":"2024-02-18T00:42:44.032868+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:42:44.032868+0000","last_undegraded":"2024-02-18T00:42:44.032868+0000","last_fullsized":"2024-02-18T00:42:44.032868+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.349134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.349134+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":195,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689626+0000","last_change":"2024-02-18T00:42:20.638644+0000","last_active":"2024-02-18T00:42:44.689626+0000","last_peered":"2024-02-18T00:42:44.689626+0000","last_clean":"2024-02-18T00:42:44.689626+0000","last_became_active":"2024-02-18T00:41:53.837106+0000","last_became_peered":"2024-02-18T00:41:53.837106+0000","last_unstale":"2024-02-18T00:42:44.689626+0000","last_undegraded":"2024-02-18T00:42:44.689626+0000","last_fullsized":"2024-02-18T00:42:44.689626+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:20.638566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:20.638566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":538,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739241+0000","last_change":"2024-02-18T00:41:23.575091+0000","last_active":"2024-02-18T00:42:44.739241+0000","last_peered":"2024-02-18T00:42:44.739241+0000","last_clean":"2024-02-18T00:42:44.739241+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:42:44.739241+0000","last_undegraded":"2024-02-18T00:42:44.739241+0000","last_fullsized":"2024-02-18T00:42:44.739241+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.574969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.574969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":661,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036427+0000","last_change":"2024-02-18T00:41:29.051297+0000","last_active":"2024-02-18T00:42:44.036427+0000","last_peered":"2024-02-18T00:42:44.036427+0000","last_clean":"2024-02-18T00:42:44.036427+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:42:44.036427+0000","last_undegraded":"2024-02-18T00:42:44.036427+0000","last_fullsized":"2024-02-18T00:42:44.036427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:41:29.051225+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690106+0000","last_change":"2024-02-18T00:41:34.509265+0000","last_active":"2024-02-18T00:42:44.690106+0000","last_peered":"2024-02-18T00:42:44.690106+0000","last_clean":"2024-02-18T00:42:44.690106+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:42:44.690106+0000","last_undegraded":"2024-02-18T00:42:44.690106+0000","last_fullsized":"2024-02-18T00:42:44.690106+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.509185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.509185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":602,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.037447+0000","last_change":"2024-02-18T00:41:15.821414+0000","last_active":"2024-02-18T00:42:44.037447+0000","last_peered":"2024-02-18T00:42:44.037447+0000","last_clean":"2024-02-18T00:42:44.037447+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:42:44.037447+0000","last_undegraded":"2024-02-18T00:42:44.037447+0000","last_fullsized":"2024-02-18T00:42:44.037447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:15.821337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.821337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":616,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033065+0000","last_change":"2024-02-18T00:41:25.756779+0000","last_active":"2024-02-18T00:42:44.033065+0000","last_peered":"2024-02-18T00:42:44.033065+0000","last_clean":"2024-02-18T00:42:44.033065+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:42:44.033065+0000","last_undegraded":"2024-02-18T00:42:44.033065+0000","last_fullsized":"2024-02-18T00:42:44.033065+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:18.151671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.151671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":15,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506279+0000","last_change":"2024-02-18T00:42:38.553481+0000","last_active":"2024-02-18T00:42:44.506279+0000","last_peered":"2024-02-18T00:42:44.506279+0000","last_clean":"2024-02-18T00:42:44.506279+0000","last_became_active":"2024-02-18T00:42:38.553174+0000","last_became_peered":"2024-02-18T00:42:38.553174+0000","last_unstale":"2024-02-18T00:42:44.506279+0000","last_undegraded":"2024-02-18T00:42:44.506279+0000","last_fullsized":"2024-02-18T00:42:44.506279+0000","mapping_epoch":623,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:14.739594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:14.739594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":634,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:49.351292+0000","last_change":"2024-02-18T00:42:49.351292+0000","last_active":"2024-02-18T00:42:49.351292+0000","last_peered":"2024-02-18T00:42:49.351292+0000","last_clean":"2024-02-18T00:42:49.351292+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:42:49.351292+0000","last_undegraded":"2024-02-18T00:42:49.351292+0000","last_fullsized":"2024-02-18T00:42:49.351292+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:42:49.351183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.351183+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":631,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036068+0000","last_change":"2024-02-18T00:42:05.815709+0000","last_active":"2024-02-18T00:42:44.036068+0000","last_peered":"2024-02-18T00:42:44.036068+0000","last_clean":"2024-02-18T00:42:44.036068+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:42:44.036068+0000","last_undegraded":"2024-02-18T00:42:44.036068+0000","last_fullsized":"2024-02-18T00:42:44.036068+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:05.815649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:42:05.815649+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":800,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430209+0000","last_change":"2024-02-18T00:41:22.121963+0000","last_active":"2024-02-18T00:42:48.430209+0000","last_peered":"2024-02-18T00:42:48.430209+0000","last_clean":"2024-02-18T00:42:48.430209+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:42:48.430209+0000","last_undegraded":"2024-02-18T00:42:48.430209+0000","last_fullsized":"2024-02-18T00:42:48.430209+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:41:22.121879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:22.121879+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":613,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036333+0000","last_change":"2024-02-18T00:41:29.740810+0000","last_active":"2024-02-18T00:42:44.036333+0000","last_peered":"2024-02-18T00:42:44.036333+0000","last_clean":"2024-02-18T00:42:44.036333+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:42:44.036333+0000","last_undegraded":"2024-02-18T00:42:44.036333+0000","last_fullsized":"2024-02-18T00:42:44.036333+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:39.193082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.193082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":546,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036326+0000","last_change":"2024-02-18T00:40:36.519540+0000","last_active":"2024-02-18T00:42:44.036326+0000","last_peered":"2024-02-18T00:42:44.036326+0000","last_clean":"2024-02-18T00:42:44.036326+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:42:44.036326+0000","last_undegraded":"2024-02-18T00:42:44.036326+0000","last_fullsized":"2024-02-18T00:42:44.036326+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.519470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.519470+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":622,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739916+0000","last_change":"2024-02-18T00:41:31.665553+0000","last_active":"2024-02-18T00:42:44.739916+0000","last_peered":"2024-02-18T00:42:44.739916+0000","last_clean":"2024-02-18T00:42:44.739916+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:42:44.739916+0000","last_undegraded":"2024-02-18T00:42:44.739916+0000","last_fullsized":"2024-02-18T00:42:44.739916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:41:31.665435+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.665435+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":638,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035951+0000","last_change":"2024-02-18T00:42:08.816658+0000","last_active":"2024-02-18T00:42:44.035951+0000","last_peered":"2024-02-18T00:42:44.035951+0000","last_clean":"2024-02-18T00:42:44.035951+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:42:44.035951+0000","last_undegraded":"2024-02-18T00:42:44.035951+0000","last_fullsized":"2024-02-18T00:42:44.035951+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:08.816534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:42:08.816534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":768,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430538+0000","last_change":"2024-02-18T00:40:38.233919+0000","last_active":"2024-02-18T00:42:48.430538+0000","last_peered":"2024-02-18T00:42:48.430538+0000","last_clean":"2024-02-18T00:42:48.430538+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:42:48.430538+0000","last_undegraded":"2024-02-18T00:42:48.430538+0000","last_fullsized":"2024-02-18T00:42:48.430538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:40:38.233838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.233838+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":610,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739868+0000","last_change":"2024-02-18T00:41:33.828221+0000","last_active":"2024-02-18T00:42:44.739868+0000","last_peered":"2024-02-18T00:42:44.739868+0000","last_clean":"2024-02-18T00:42:44.739868+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:42:44.739868+0000","last_undegraded":"2024-02-18T00:42:44.739868+0000","last_fullsized":"2024-02-18T00:42:44.739868+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.596666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.596666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":541,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035582+0000","last_change":"2024-02-18T00:41:15.124842+0000","last_active":"2024-02-18T00:42:44.035582+0000","last_peered":"2024-02-18T00:42:44.035582+0000","last_clean":"2024-02-18T00:42:44.035582+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:42:44.035582+0000","last_undegraded":"2024-02-18T00:42:44.035582+0000","last_fullsized":"2024-02-18T00:42:44.035582+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:15.124769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.124769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'25","reported_seq":673,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036222+0000","last_change":"2024-02-18T00:41:36.043652+0000","last_active":"2024-02-18T00:42:44.036222+0000","last_peered":"2024-02-18T00:42:44.036222+0000","last_clean":"2024-02-18T00:42:44.036222+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:42:44.036222+0000","last_undegraded":"2024-02-18T00:42:44.036222+0000","last_fullsized":"2024-02-18T00:42:44.036222+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:36.043577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.043577+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033035+0000","last_change":"2024-02-18T00:40:52.319724+0000","last_active":"2024-02-18T00:42:44.033035+0000","last_peered":"2024-02-18T00:42:44.033035+0000","last_clean":"2024-02-18T00:42:44.033035+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:42:44.033035+0000","last_undegraded":"2024-02-18T00:42:44.033035+0000","last_fullsized":"2024-02-18T00:42:44.033035+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.319660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.319660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":599,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739991+0000","last_change":"2024-02-18T00:42:18.694442+0000","last_active":"2024-02-18T00:42:44.739991+0000","last_peered":"2024-02-18T00:42:44.739991+0000","last_clean":"2024-02-18T00:42:44.739991+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:42:44.739991+0000","last_undegraded":"2024-02-18T00:42:44.739991+0000","last_fullsized":"2024-02-18T00:42:44.739991+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.694358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.694358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":607,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990569+0000","last_change":"2024-02-18T00:41:37.801538+0000","last_active":"2024-02-18T00:42:44.990569+0000","last_peered":"2024-02-18T00:42:44.990569+0000","last_clean":"2024-02-18T00:42:44.990569+0000","last_became_active":"2024-02-18T00:41:37.801183+0000","last_became_peered":"2024-02-18T00:41:37.801183+0000","last_unstale":"2024-02-18T00:42:44.990569+0000","last_undegraded":"2024-02-18T00:42:44.990569+0000","last_fullsized":"2024-02-18T00:42:44.990569+0000","mapping_epoch":62,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":564,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.016774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.016774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":541,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036169+0000","last_change":"2024-02-18T00:40:44.111488+0000","last_active":"2024-02-18T00:42:44.036169+0000","last_peered":"2024-02-18T00:42:44.036169+0000","last_clean":"2024-02-18T00:42:44.036169+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:42:44.036169+0000","last_undegraded":"2024-02-18T00:42:44.036169+0000","last_fullsized":"2024-02-18T00:42:44.036169+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:44.111388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:40:44.111388+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":763,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430230+0000","last_change":"2024-02-18T00:40:39.136955+0000","last_active":"2024-02-18T00:42:48.430230+0000","last_peered":"2024-02-18T00:42:48.430230+0000","last_clean":"2024-02-18T00:42:48.430230+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:42:48.430230+0000","last_undegraded":"2024-02-18T00:42:48.430230+0000","last_fullsized":"2024-02-18T00:42:48.430230+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:40:39.136881+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:36:53.910333+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.136881+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":661,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035644+0000","last_change":"2024-02-18T00:42:06.374897+0000","last_active":"2024-02-18T00:42:44.035644+0000","last_peered":"2024-02-18T00:42:44.035644+0000","last_clean":"2024-02-18T00:42:44.035644+0000","last_became_active":"2024-02-18T00:29:17.083962+0000","last_became_peered":"2024-02-18T00:29:17.083962+0000","last_unstale":"2024-02-18T00:42:44.035644+0000","last_undegraded":"2024-02-18T00:42:44.035644+0000","last_fullsized":"2024-02-18T00:42:44.035644+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:42:06.374780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.374780+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":46,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":630,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032851+0000","last_change":"2024-02-18T00:40:52.642135+0000","last_active":"2024-02-18T00:42:44.032851+0000","last_peered":"2024-02-18T00:42:44.032851+0000","last_clean":"2024-02-18T00:42:44.032851+0000","last_became_active":"2024-02-18T00:39:36.752210+0000","last_became_peered":"2024-02-18T00:39:36.752210+0000","last_unstale":"2024-02-18T00:42:44.032851+0000","last_undegraded":"2024-02-18T00:42:44.032851+0000","last_fullsized":"2024-02-18T00:42:44.032851+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.642073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":177,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690001+0000","last_change":"2024-02-18T00:42:15.535915+0000","last_active":"2024-02-18T00:42:44.690001+0000","last_peered":"2024-02-18T00:42:44.690001+0000","last_clean":"2024-02-18T00:42:44.690001+0000","last_became_active":"2024-02-18T00:39:36.767278+0000","last_became_peered":"2024-02-18T00:39:36.767278+0000","last_unstale":"2024-02-18T00:42:44.690001+0000","last_undegraded":"2024-02-18T00:42:44.690001+0000","last_fullsized":"2024-02-18T00:42:44.690001+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:15.535843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:15.535843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":538,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739389+0000","last_change":"2024-02-18T00:40:45.658350+0000","last_active":"2024-02-18T00:42:44.739389+0000","last_peered":"2024-02-18T00:42:44.739389+0000","last_clean":"2024-02-18T00:42:44.739389+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:42:44.739389+0000","last_undegraded":"2024-02-18T00:42:44.739389+0000","last_fullsized":"2024-02-18T00:42:44.739389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:45.658288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:45.658288+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":620,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035813+0000","last_change":"2024-02-18T00:42:10.347030+0000","last_active":"2024-02-18T00:42:44.035813+0000","last_peered":"2024-02-18T00:42:44.035813+0000","last_clean":"2024-02-18T00:42:44.035813+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:42:44.035813+0000","last_undegraded":"2024-02-18T00:42:44.035813+0000","last_fullsized":"2024-02-18T00:42:44.035813+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:10.346948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:10.346948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":745,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989882+0000","last_change":"2024-02-18T00:40:46.010884+0000","last_active":"2024-02-18T00:42:44.989882+0000","last_peered":"2024-02-18T00:42:44.989882+0000","last_clean":"2024-02-18T00:42:44.989882+0000","last_became_active":"2024-02-18T00:39:36.766141+0000","last_became_peered":"2024-02-18T00:39:36.766141+0000","last_unstale":"2024-02-18T00:42:44.989882+0000","last_undegraded":"2024-02-18T00:42:44.989882+0000","last_fullsized":"2024-02-18T00:42:44.989882+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:40:46.010811+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.010811+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":76,"num_read_kb":62,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":638,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.044439+0000","last_change":"2024-02-18T00:42:44.039707+0000","last_active":"2024-02-18T00:42:44.044439+0000","last_peered":"2024-02-18T00:42:44.044439+0000","last_clean":"2024-02-18T00:42:44.044439+0000","last_became_active":"2024-02-18T00:42:44.039326+0000","last_became_peered":"2024-02-18T00:42:44.039326+0000","last_unstale":"2024-02-18T00:42:44.044439+0000","last_undegraded":"2024-02-18T00:42:44.044439+0000","last_fullsized":"2024-02-18T00:42:44.044439+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.828800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.828800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":600,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:50.371671+0000","last_change":"2024-02-18T00:42:50.371671+0000","last_active":"2024-02-18T00:42:50.371671+0000","last_peered":"2024-02-18T00:42:50.371671+0000","last_clean":"2024-02-18T00:42:50.371671+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:42:50.371671+0000","last_undegraded":"2024-02-18T00:42:50.371671+0000","last_fullsized":"2024-02-18T00:42:50.371671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:50.371592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.371592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":535,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.991224+0000","last_change":"2024-02-18T00:41:26.953890+0000","last_active":"2024-02-18T00:42:44.991224+0000","last_peered":"2024-02-18T00:42:44.991224+0000","last_clean":"2024-02-18T00:42:44.991224+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:42:44.991224+0000","last_undegraded":"2024-02-18T00:42:44.991224+0000","last_fullsized":"2024-02-18T00:42:44.991224+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.952674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.952674+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":599,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032086+0000","last_change":"2024-02-18T00:42:22.964768+0000","last_active":"2024-02-18T00:42:44.032086+0000","last_peered":"2024-02-18T00:42:44.032086+0000","last_clean":"2024-02-18T00:42:44.032086+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:42:44.032086+0000","last_undegraded":"2024-02-18T00:42:44.032086+0000","last_fullsized":"2024-02-18T00:42:44.032086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.964687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:22.964687+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.964687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":722,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506974+0000","last_change":"2024-02-18T00:40:40.195949+0000","last_active":"2024-02-18T00:42:44.506974+0000","last_peered":"2024-02-18T00:42:44.506974+0000","last_clean":"2024-02-18T00:42:44.506974+0000","last_became_active":"2024-02-18T00:39:36.750472+0000","last_became_peered":"2024-02-18T00:39:36.750472+0000","last_unstale":"2024-02-18T00:42:44.506974+0000","last_undegraded":"2024-02-18T00:42:44.506974+0000","last_fullsized":"2024-02-18T00:42:44.506974+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:40:40.195882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.195882+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032968+0000","last_change":"2024-02-18T00:41:20.138251+0000","last_active":"2024-02-18T00:42:44.032968+0000","last_peered":"2024-02-18T00:42:44.032968+0000","last_clean":"2024-02-18T00:42:44.032968+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:42:44.032968+0000","last_undegraded":"2024-02-18T00:42:44.032968+0000","last_fullsized":"2024-02-18T00:42:44.032968+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:41:20.138183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:20.138183+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":605,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036045+0000","last_change":"2024-02-18T00:42:14.840374+0000","last_active":"2024-02-18T00:42:44.036045+0000","last_peered":"2024-02-18T00:42:44.036045+0000","last_clean":"2024-02-18T00:42:44.036045+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:42:44.036045+0000","last_undegraded":"2024-02-18T00:42:44.036045+0000","last_fullsized":"2024-02-18T00:42:44.036045+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:14.840316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:14.840316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":535,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.991202+0000","last_change":"2024-02-18T00:42:06.868889+0000","last_active":"2024-02-18T00:42:44.991202+0000","last_peered":"2024-02-18T00:42:44.991202+0000","last_clean":"2024-02-18T00:42:44.991202+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:42:44.991202+0000","last_undegraded":"2024-02-18T00:42:44.991202+0000","last_fullsized":"2024-02-18T00:42:44.991202+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:06.868823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.868823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":594,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.740064+0000","last_change":"2024-02-18T00:40:38.511704+0000","last_active":"2024-02-18T00:42:44.740064+0000","last_peered":"2024-02-18T00:42:44.740064+0000","last_clean":"2024-02-18T00:42:44.740064+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:42:44.740064+0000","last_undegraded":"2024-02-18T00:42:44.740064+0000","last_fullsized":"2024-02-18T00:42:44.740064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.511625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.511625+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'24","reported_seq":634,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507420+0000","last_change":"2024-02-18T00:40:39.224307+0000","last_active":"2024-02-18T00:42:44.507420+0000","last_peered":"2024-02-18T00:42:44.507420+0000","last_clean":"2024-02-18T00:42:44.507420+0000","last_became_active":"2024-02-18T00:29:17.089593+0000","last_became_peered":"2024-02-18T00:29:17.089593+0000","last_unstale":"2024-02-18T00:42:44.507420+0000","last_undegraded":"2024-02-18T00:42:44.507420+0000","last_fullsized":"2024-02-18T00:42:44.507420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:40:39.224211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:39.224211+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033116+0000","last_change":"2024-02-18T00:42:04.727971+0000","last_active":"2024-02-18T00:42:44.033116+0000","last_peered":"2024-02-18T00:42:44.033116+0000","last_clean":"2024-02-18T00:42:44.033116+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:42:44.033116+0000","last_undegraded":"2024-02-18T00:42:44.033116+0000","last_fullsized":"2024-02-18T00:42:44.033116+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:04.727901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:04.727901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":599,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506158+0000","last_change":"2024-02-18T00:40:37.156277+0000","last_active":"2024-02-18T00:42:44.506158+0000","last_peered":"2024-02-18T00:42:44.506158+0000","last_clean":"2024-02-18T00:42:44.506158+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:42:44.506158+0000","last_undegraded":"2024-02-18T00:42:44.506158+0000","last_fullsized":"2024-02-18T00:42:44.506158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:37.156213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:40:37.156213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":600,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.037481+0000","last_change":"2024-02-18T00:41:31.791248+0000","last_active":"2024-02-18T00:42:44.037481+0000","last_peered":"2024-02-18T00:42:44.037481+0000","last_clean":"2024-02-18T00:42:44.037481+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:42:44.037481+0000","last_undegraded":"2024-02-18T00:42:44.037481+0000","last_fullsized":"2024-02-18T00:42:44.037481+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:31.791120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.791120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":11,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.691386+0000","last_change":"2024-02-18T00:42:44.691165+0000","last_active":"2024-02-18T00:42:44.691386+0000","last_peered":"2024-02-18T00:42:44.691386+0000","last_clean":"2024-02-18T00:42:44.691386+0000","last_became_active":"2024-02-18T00:42:44.690928+0000","last_became_peered":"2024-02-18T00:42:44.690928+0000","last_unstale":"2024-02-18T00:42:44.691386+0000","last_undegraded":"2024-02-18T00:42:44.691386+0000","last_fullsized":"2024-02-18T00:42:44.691386+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:20.995657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:20.995657+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":177,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689901+0000","last_change":"2024-02-18T00:42:23.605382+0000","last_active":"2024-02-18T00:42:44.689901+0000","last_peered":"2024-02-18T00:42:44.689901+0000","last_clean":"2024-02-18T00:42:44.689901+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:42:44.689901+0000","last_undegraded":"2024-02-18T00:42:44.689901+0000","last_fullsized":"2024-02-18T00:42:44.689901+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:23.605268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:42:23.605268+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":698,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032412+0000","last_change":"2024-02-18T00:42:12.900735+0000","last_active":"2024-02-18T00:42:44.032412+0000","last_peered":"2024-02-18T00:42:44.032412+0000","last_clean":"2024-02-18T00:42:44.032412+0000","last_became_active":"2024-02-18T00:39:36.751539+0000","last_became_peered":"2024-02-18T00:39:36.751539+0000","last_unstale":"2024-02-18T00:42:44.032412+0000","last_undegraded":"2024-02-18T00:42:44.032412+0000","last_fullsized":"2024-02-18T00:42:44.032412+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:42:12.900647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:12.900647+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":177,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689406+0000","last_change":"2024-02-18T00:42:11.496928+0000","last_active":"2024-02-18T00:42:44.689406+0000","last_peered":"2024-02-18T00:42:44.689406+0000","last_clean":"2024-02-18T00:42:44.689406+0000","last_became_active":"2024-02-18T00:39:36.759224+0000","last_became_peered":"2024-02-18T00:39:36.759224+0000","last_unstale":"2024-02-18T00:42:44.689406+0000","last_undegraded":"2024-02-18T00:42:44.689406+0000","last_fullsized":"2024-02-18T00:42:44.689406+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:11.496846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:42:11.496846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":639,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739650+0000","last_change":"2024-02-18T00:42:05.684260+0000","last_active":"2024-02-18T00:42:44.739650+0000","last_peered":"2024-02-18T00:42:44.739650+0000","last_clean":"2024-02-18T00:42:44.739650+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:42:44.739650+0000","last_undegraded":"2024-02-18T00:42:44.739650+0000","last_fullsized":"2024-02-18T00:42:44.739650+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:05.684148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:05.684148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":594,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989249+0000","last_change":"2024-02-18T00:42:11.849138+0000","last_active":"2024-02-18T00:42:44.989249+0000","last_peered":"2024-02-18T00:42:44.989249+0000","last_clean":"2024-02-18T00:42:44.989249+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:42:44.989249+0000","last_undegraded":"2024-02-18T00:42:44.989249+0000","last_fullsized":"2024-02-18T00:42:44.989249+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:11.849068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:42:11.849068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":615,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.740058+0000","last_change":"2024-02-18T00:42:06.735073+0000","last_active":"2024-02-18T00:42:44.740058+0000","last_peered":"2024-02-18T00:42:44.740058+0000","last_clean":"2024-02-18T00:42:44.740058+0000","last_became_active":"2024-02-18T00:41:57.791956+0000","last_became_peered":"2024-02-18T00:41:57.791956+0000","last_unstale":"2024-02-18T00:42:44.740058+0000","last_undegraded":"2024-02-18T00:42:44.740058+0000","last_fullsized":"2024-02-18T00:42:44.740058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":584,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:06.734956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.734956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":541,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506819+0000","last_change":"2024-02-18T00:41:26.021783+0000","last_active":"2024-02-18T00:42:44.506819+0000","last_peered":"2024-02-18T00:42:44.506819+0000","last_clean":"2024-02-18T00:42:44.506819+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:42:44.506819+0000","last_undegraded":"2024-02-18T00:42:44.506819+0000","last_fullsized":"2024-02-18T00:42:44.506819+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.021716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.021716+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":618,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033538+0000","last_change":"2024-02-18T00:40:34.340695+0000","last_active":"2024-02-18T00:42:44.033538+0000","last_peered":"2024-02-18T00:42:44.033538+0000","last_clean":"2024-02-18T00:42:44.033538+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:42:44.033538+0000","last_undegraded":"2024-02-18T00:42:44.033538+0000","last_fullsized":"2024-02-18T00:42:44.033538+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.340624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.340624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":657,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036961+0000","last_change":"2024-02-18T00:42:10.771025+0000","last_active":"2024-02-18T00:42:44.036961+0000","last_peered":"2024-02-18T00:42:44.036961+0000","last_clean":"2024-02-18T00:42:44.036961+0000","last_became_active":"2024-02-18T00:29:17.084037+0000","last_became_peered":"2024-02-18T00:29:17.084037+0000","last_unstale":"2024-02-18T00:42:44.036961+0000","last_undegraded":"2024-02-18T00:42:44.036961+0000","last_fullsized":"2024-02-18T00:42:44.036961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:42:10.770922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:10.770922+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":36,"num_write":26,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":618,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:49.673980+0000","last_change":"2024-02-18T00:42:49.673980+0000","last_active":"2024-02-18T00:42:49.673980+0000","last_peered":"2024-02-18T00:42:49.673980+0000","last_clean":"2024-02-18T00:42:49.673980+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:42:49.673980+0000","last_undegraded":"2024-02-18T00:42:49.673980+0000","last_fullsized":"2024-02-18T00:42:49.673980+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.673915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.673915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1420,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:49.330105+0000","last_change":"2024-02-18T00:41:25.818891+0000","last_active":"2024-02-18T00:42:49.330105+0000","last_peered":"2024-02-18T00:42:49.330105+0000","last_clean":"2024-02-18T00:42:49.330105+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:42:49.330105+0000","last_undegraded":"2024-02-18T00:42:49.330105+0000","last_fullsized":"2024-02-18T00:42:49.330105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:41:25.818811+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:41:25.818811+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":806,"num_read_kb":801,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":176,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690273+0000","last_change":"2024-02-18T00:41:36.515174+0000","last_active":"2024-02-18T00:42:44.690273+0000","last_peered":"2024-02-18T00:42:44.690273+0000","last_clean":"2024-02-18T00:42:44.690273+0000","last_became_active":"2024-02-18T00:39:36.764502+0000","last_became_peered":"2024-02-18T00:39:36.764502+0000","last_unstale":"2024-02-18T00:42:44.690273+0000","last_undegraded":"2024-02-18T00:42:44.690273+0000","last_fullsized":"2024-02-18T00:42:44.690273+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:41:36.515095+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:32:55.588750+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.515095+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":559,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035856+0000","last_change":"2024-02-18T00:41:34.091992+0000","last_active":"2024-02-18T00:42:44.035856+0000","last_peered":"2024-02-18T00:42:44.035856+0000","last_clean":"2024-02-18T00:42:44.035856+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:42:44.035856+0000","last_undegraded":"2024-02-18T00:42:44.035856+0000","last_fullsized":"2024-02-18T00:42:44.035856+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.091913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.091913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":596,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507376+0000","last_change":"2024-02-18T00:40:34.247010+0000","last_active":"2024-02-18T00:42:44.507376+0000","last_peered":"2024-02-18T00:42:44.507376+0000","last_clean":"2024-02-18T00:42:44.507376+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:42:44.507376+0000","last_undegraded":"2024-02-18T00:42:44.507376+0000","last_fullsized":"2024-02-18T00:42:44.507376+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.246941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.246941+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"457'47","reported_seq":510,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430676+0000","last_change":"2024-02-18T00:42:13.680415+0000","last_active":"2024-02-18T00:42:48.430676+0000","last_peered":"2024-02-18T00:42:48.430676+0000","last_clean":"2024-02-18T00:42:48.430676+0000","last_became_active":"2024-02-18T00:35:42.670330+0000","last_became_peered":"2024-02-18T00:35:42.670330+0000","last_unstale":"2024-02-18T00:42:48.430676+0000","last_undegraded":"2024-02-18T00:42:48.430676+0000","last_fullsized":"2024-02-18T00:42:48.430676+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:42:13.680322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.680322+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":78,"num_read_kb":62,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":612,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739398+0000","last_change":"2024-02-18T00:42:09.721978+0000","last_active":"2024-02-18T00:42:44.739398+0000","last_peered":"2024-02-18T00:42:44.739398+0000","last_clean":"2024-02-18T00:42:44.739398+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:42:44.739398+0000","last_undegraded":"2024-02-18T00:42:44.739398+0000","last_fullsized":"2024-02-18T00:42:44.739398+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:09.721900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:42:09.721900+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":625,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990827+0000","last_change":"2024-02-18T00:42:19.938250+0000","last_active":"2024-02-18T00:42:44.990827+0000","last_peered":"2024-02-18T00:42:44.990827+0000","last_clean":"2024-02-18T00:42:44.990827+0000","last_became_active":"2024-02-18T00:42:02.125704+0000","last_became_peered":"2024-02-18T00:42:02.125704+0000","last_unstale":"2024-02-18T00:42:44.990827+0000","last_undegraded":"2024-02-18T00:42:44.990827+0000","last_fullsized":"2024-02-18T00:42:44.990827+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.938182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.938182+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":541,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507613+0000","last_change":"2024-02-18T00:41:32.116478+0000","last_active":"2024-02-18T00:42:44.507613+0000","last_peered":"2024-02-18T00:42:44.507613+0000","last_clean":"2024-02-18T00:42:44.507613+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:42:44.507613+0000","last_undegraded":"2024-02-18T00:42:44.507613+0000","last_fullsized":"2024-02-18T00:42:44.507613+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.116365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.116365+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":767,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430157+0000","last_change":"2024-02-18T00:41:15.145124+0000","last_active":"2024-02-18T00:42:48.430157+0000","last_peered":"2024-02-18T00:42:48.430157+0000","last_clean":"2024-02-18T00:42:48.430157+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:42:48.430157+0000","last_undegraded":"2024-02-18T00:42:48.430157+0000","last_fullsized":"2024-02-18T00:42:48.430157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:41:15.145053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.145053+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":617,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032885+0000","last_change":"2024-02-18T00:41:36.069318+0000","last_active":"2024-02-18T00:42:44.032885+0000","last_peered":"2024-02-18T00:42:44.032885+0000","last_clean":"2024-02-18T00:42:44.032885+0000","last_became_active":"2024-02-18T00:30:41.640000+0000","last_became_peered":"2024-02-18T00:30:41.640000+0000","last_unstale":"2024-02-18T00:42:44.032885+0000","last_undegraded":"2024-02-18T00:42:44.032885+0000","last_fullsized":"2024-02-18T00:42:44.032885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:41:36.069252+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.069252+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":643,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.037325+0000","last_change":"2024-02-18T00:42:15.866570+0000","last_active":"2024-02-18T00:42:44.037325+0000","last_peered":"2024-02-18T00:42:44.037325+0000","last_clean":"2024-02-18T00:42:44.037325+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:42:44.037325+0000","last_undegraded":"2024-02-18T00:42:44.037325+0000","last_fullsized":"2024-02-18T00:42:44.037325+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:15.866466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:15.866466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":594,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506267+0000","last_change":"2024-02-18T00:42:18.192907+0000","last_active":"2024-02-18T00:42:44.506267+0000","last_peered":"2024-02-18T00:42:44.506267+0000","last_clean":"2024-02-18T00:42:44.506267+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:42:44.506267+0000","last_undegraded":"2024-02-18T00:42:44.506267+0000","last_fullsized":"2024-02-18T00:42:44.506267+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.192802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.192802+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":541,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036159+0000","last_change":"2024-02-18T00:41:17.092835+0000","last_active":"2024-02-18T00:42:44.036159+0000","last_peered":"2024-02-18T00:42:44.036159+0000","last_clean":"2024-02-18T00:42:44.036159+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:42:44.036159+0000","last_undegraded":"2024-02-18T00:42:44.036159+0000","last_fullsized":"2024-02-18T00:42:44.036159+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.092717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.092717+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":600,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033362+0000","last_change":"2024-02-18T00:41:32.150056+0000","last_active":"2024-02-18T00:42:44.033362+0000","last_peered":"2024-02-18T00:42:44.033362+0000","last_clean":"2024-02-18T00:42:44.033362+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:42:44.033362+0000","last_undegraded":"2024-02-18T00:42:44.033362+0000","last_fullsized":"2024-02-18T00:42:44.033362+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.149984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.149984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":661,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035695+0000","last_change":"2024-02-18T00:40:33.214440+0000","last_active":"2024-02-18T00:42:44.035695+0000","last_peered":"2024-02-18T00:42:44.035695+0000","last_clean":"2024-02-18T00:42:44.035695+0000","last_became_active":"2024-02-18T00:29:17.086015+0000","last_became_peered":"2024-02-18T00:29:17.086015+0000","last_unstale":"2024-02-18T00:42:44.035695+0000","last_undegraded":"2024-02-18T00:42:44.035695+0000","last_fullsized":"2024-02-18T00:42:44.035695+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:40:33.214358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:33.214358+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":47,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":649,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036480+0000","last_change":"2024-02-18T00:40:40.158702+0000","last_active":"2024-02-18T00:42:44.036480+0000","last_peered":"2024-02-18T00:42:44.036480+0000","last_clean":"2024-02-18T00:42:44.036480+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:42:44.036480+0000","last_undegraded":"2024-02-18T00:42:44.036480+0000","last_fullsized":"2024-02-18T00:42:44.036480+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:40:40.158593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.158593+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":625,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.991032+0000","last_change":"2024-02-18T00:41:42.776580+0000","last_active":"2024-02-18T00:42:44.991032+0000","last_peered":"2024-02-18T00:42:44.991032+0000","last_clean":"2024-02-18T00:42:44.991032+0000","last_became_active":"2024-02-18T00:41:42.776144+0000","last_became_peered":"2024-02-18T00:41:42.776144+0000","last_unstale":"2024-02-18T00:42:44.991032+0000","last_undegraded":"2024-02-18T00:42:44.991032+0000","last_fullsized":"2024-02-18T00:42:44.991032+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:03.980632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:40:03.980632+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":541,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035681+0000","last_change":"2024-02-18T00:41:31.059363+0000","last_active":"2024-02-18T00:42:44.035681+0000","last_peered":"2024-02-18T00:42:44.035681+0000","last_clean":"2024-02-18T00:42:44.035681+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:42:44.035681+0000","last_undegraded":"2024-02-18T00:42:44.035681+0000","last_fullsized":"2024-02-18T00:42:44.035681+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:31.059285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.059285+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":699,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507449+0000","last_change":"2024-02-18T00:40:51.299875+0000","last_active":"2024-02-18T00:42:44.507449+0000","last_peered":"2024-02-18T00:42:44.507449+0000","last_clean":"2024-02-18T00:42:44.507449+0000","last_became_active":"2024-02-18T00:39:36.771394+0000","last_became_peered":"2024-02-18T00:39:36.771394+0000","last_unstale":"2024-02-18T00:42:44.507449+0000","last_undegraded":"2024-02-18T00:42:44.507449+0000","last_fullsized":"2024-02-18T00:42:44.507449+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:40:51.299791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.299791+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":604,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033267+0000","last_change":"2024-02-18T00:41:21.141528+0000","last_active":"2024-02-18T00:42:44.033267+0000","last_peered":"2024-02-18T00:42:44.033267+0000","last_clean":"2024-02-18T00:42:44.033267+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:42:44.033267+0000","last_undegraded":"2024-02-18T00:42:44.033267+0000","last_fullsized":"2024-02-18T00:42:44.033267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:21.141464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.141464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":609,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990726+0000","last_change":"2024-02-18T00:40:33.083220+0000","last_active":"2024-02-18T00:42:44.990726+0000","last_peered":"2024-02-18T00:42:44.990726+0000","last_clean":"2024-02-18T00:42:44.990726+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:42:44.990726+0000","last_undegraded":"2024-02-18T00:42:44.990726+0000","last_fullsized":"2024-02-18T00:42:44.990726+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:33.083145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:33.083145+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":618,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033276+0000","last_change":"2024-02-18T00:42:10.704623+0000","last_active":"2024-02-18T00:42:44.033276+0000","last_peered":"2024-02-18T00:42:44.033276+0000","last_clean":"2024-02-18T00:42:44.033276+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:42:44.033276+0000","last_undegraded":"2024-02-18T00:42:44.033276+0000","last_fullsized":"2024-02-18T00:42:44.033276+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:10.704545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:42:10.704545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":541,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506771+0000","last_change":"2024-02-18T00:42:09.215232+0000","last_active":"2024-02-18T00:42:44.506771+0000","last_peered":"2024-02-18T00:42:44.506771+0000","last_clean":"2024-02-18T00:42:44.506771+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:42:44.506771+0000","last_undegraded":"2024-02-18T00:42:44.506771+0000","last_fullsized":"2024-02-18T00:42:44.506771+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:09.215164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:09.215164+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":688,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990623+0000","last_change":"2024-02-18T00:42:08.844996+0000","last_active":"2024-02-18T00:42:44.990623+0000","last_peered":"2024-02-18T00:42:44.990623+0000","last_clean":"2024-02-18T00:42:44.990623+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:42:44.990623+0000","last_undegraded":"2024-02-18T00:42:44.990623+0000","last_fullsized":"2024-02-18T00:42:44.990623+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'18","last_scrub_stamp":"2024-02-18T00:42:08.844930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:08.844930+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":603,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.031944+0000","last_change":"2024-02-18T00:40:38.674457+0000","last_active":"2024-02-18T00:42:44.031944+0000","last_peered":"2024-02-18T00:42:44.031944+0000","last_clean":"2024-02-18T00:42:44.031944+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:42:44.031944+0000","last_undegraded":"2024-02-18T00:42:44.031944+0000","last_fullsized":"2024-02-18T00:42:44.031944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:38.674378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:38.674378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":599,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739732+0000","last_change":"2024-02-18T00:42:25.667489+0000","last_active":"2024-02-18T00:42:44.739732+0000","last_peered":"2024-02-18T00:42:44.739732+0000","last_clean":"2024-02-18T00:42:44.739732+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:42:44.739732+0000","last_undegraded":"2024-02-18T00:42:44.739732+0000","last_fullsized":"2024-02-18T00:42:44.739732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.667419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.667419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690360+0000","last_change":"2024-02-18T00:41:21.478541+0000","last_active":"2024-02-18T00:42:44.690360+0000","last_peered":"2024-02-18T00:42:44.690360+0000","last_clean":"2024-02-18T00:42:44.690360+0000","last_became_active":"2024-02-18T00:39:36.759264+0000","last_became_peered":"2024-02-18T00:39:36.759264+0000","last_unstale":"2024-02-18T00:42:44.690360+0000","last_undegraded":"2024-02-18T00:42:44.690360+0000","last_fullsized":"2024-02-18T00:42:44.690360+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:21.478458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.478458+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":553,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032596+0000","last_change":"2024-02-18T00:42:19.794911+0000","last_active":"2024-02-18T00:42:44.032596+0000","last_peered":"2024-02-18T00:42:44.032596+0000","last_clean":"2024-02-18T00:42:44.032596+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:42:44.032596+0000","last_undegraded":"2024-02-18T00:42:44.032596+0000","last_fullsized":"2024-02-18T00:42:44.032596+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.794845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.794845+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":703,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990621+0000","last_change":"2024-02-18T00:40:42.124689+0000","last_active":"2024-02-18T00:42:44.990621+0000","last_peered":"2024-02-18T00:42:44.990621+0000","last_clean":"2024-02-18T00:42:44.990621+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:42:44.990621+0000","last_undegraded":"2024-02-18T00:42:44.990621+0000","last_fullsized":"2024-02-18T00:42:44.990621+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:40:42.124619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.124619+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":625,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739344+0000","last_change":"2024-02-18T00:40:36.427315+0000","last_active":"2024-02-18T00:42:44.739344+0000","last_peered":"2024-02-18T00:42:44.739344+0000","last_clean":"2024-02-18T00:42:44.739344+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:42:44.739344+0000","last_undegraded":"2024-02-18T00:42:44.739344+0000","last_fullsized":"2024-02-18T00:42:44.739344+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.427206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.427206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":766,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430599+0000","last_change":"2024-02-18T00:40:09.668977+0000","last_active":"2024-02-18T00:42:48.430599+0000","last_peered":"2024-02-18T00:42:48.430599+0000","last_clean":"2024-02-18T00:42:48.430599+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:42:48.430599+0000","last_undegraded":"2024-02-18T00:42:48.430599+0000","last_fullsized":"2024-02-18T00:42:48.430599+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:40:09.668906+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:40:09.668906+0000","last_clean_scrub_stamp":"2024-02-18T00:40:09.668906+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":561,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:49.778761+0000","last_change":"2024-02-18T00:42:49.778761+0000","last_active":"2024-02-18T00:42:49.778761+0000","last_peered":"2024-02-18T00:42:49.778761+0000","last_clean":"2024-02-18T00:42:49.778761+0000","last_became_active":"2024-02-18T00:39:36.751162+0000","last_became_peered":"2024-02-18T00:39:36.751162+0000","last_unstale":"2024-02-18T00:42:49.778761+0000","last_undegraded":"2024-02-18T00:42:49.778761+0000","last_fullsized":"2024-02-18T00:42:49.778761+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.778678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.778678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":678,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739493+0000","last_change":"2024-02-18T00:40:43.621296+0000","last_active":"2024-02-18T00:42:44.739493+0000","last_peered":"2024-02-18T00:42:44.739493+0000","last_clean":"2024-02-18T00:42:44.739493+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:42:44.739493+0000","last_undegraded":"2024-02-18T00:42:44.739493+0000","last_fullsized":"2024-02-18T00:42:44.739493+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:40:43.621226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:43.621226+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":600,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507049+0000","last_change":"2024-02-18T00:40:36.184282+0000","last_active":"2024-02-18T00:42:44.507049+0000","last_peered":"2024-02-18T00:42:44.507049+0000","last_clean":"2024-02-18T00:42:44.507049+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:42:44.507049+0000","last_undegraded":"2024-02-18T00:42:44.507049+0000","last_fullsized":"2024-02-18T00:42:44.507049+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:36.184216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:40:36.184216+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":789,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430147+0000","last_change":"2024-02-18T00:40:06.960621+0000","last_active":"2024-02-18T00:42:48.430147+0000","last_peered":"2024-02-18T00:42:48.430147+0000","last_clean":"2024-02-18T00:42:48.430147+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:42:48.430147+0000","last_undegraded":"2024-02-18T00:42:48.430147+0000","last_fullsized":"2024-02-18T00:42:48.430147+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.960509+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":538,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.740102+0000","last_change":"2024-02-18T00:40:06.573220+0000","last_active":"2024-02-18T00:42:44.740102+0000","last_peered":"2024-02-18T00:42:44.740102+0000","last_clean":"2024-02-18T00:42:44.740102+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:42:44.740102+0000","last_undegraded":"2024-02-18T00:42:44.740102+0000","last_fullsized":"2024-02-18T00:42:44.740102+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:40:06.573143+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":636,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:50.108975+0000","last_change":"2024-02-18T00:42:50.108975+0000","last_active":"2024-02-18T00:42:50.108975+0000","last_peered":"2024-02-18T00:42:50.108975+0000","last_clean":"2024-02-18T00:42:50.108975+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:42:50.108975+0000","last_undegraded":"2024-02-18T00:42:50.108975+0000","last_fullsized":"2024-02-18T00:42:50.108975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:42:50.108893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.108893+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":608,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032315+0000","last_change":"2024-02-18T00:42:17.908390+0000","last_active":"2024-02-18T00:42:44.032315+0000","last_peered":"2024-02-18T00:42:44.032315+0000","last_clean":"2024-02-18T00:42:44.032315+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:42:44.032315+0000","last_undegraded":"2024-02-18T00:42:44.032315+0000","last_fullsized":"2024-02-18T00:42:44.032315+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:17.908310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:17.908310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":591,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990981+0000","last_change":"2024-02-18T00:40:08.961647+0000","last_active":"2024-02-18T00:42:44.990981+0000","last_peered":"2024-02-18T00:42:44.990981+0000","last_clean":"2024-02-18T00:42:44.990981+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:42:44.990981+0000","last_undegraded":"2024-02-18T00:42:44.990981+0000","last_fullsized":"2024-02-18T00:42:44.990981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:08.961546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:08.961546+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":599,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.506198+0000","last_change":"2024-02-18T00:42:04.263648+0000","last_active":"2024-02-18T00:42:44.506198+0000","last_peered":"2024-02-18T00:42:44.506198+0000","last_clean":"2024-02-18T00:42:44.506198+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:42:44.506198+0000","last_undegraded":"2024-02-18T00:42:44.506198+0000","last_fullsized":"2024-02-18T00:42:44.506198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:04.263582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:04.263582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":543,"reported_epoch":629,"state":"active+clean","last_fresh":"2024-02-18T00:42:49.449913+0000","last_change":"2024-02-18T00:42:49.449913+0000","last_active":"2024-02-18T00:42:49.449913+0000","last_peered":"2024-02-18T00:42:49.449913+0000","last_clean":"2024-02-18T00:42:49.449913+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:42:49.449913+0000","last_undegraded":"2024-02-18T00:42:49.449913+0000","last_fullsized":"2024-02-18T00:42:49.449913+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.449844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.449844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":635,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035840+0000","last_change":"2024-02-18T00:42:20.356302+0000","last_active":"2024-02-18T00:42:44.035840+0000","last_peered":"2024-02-18T00:42:44.035840+0000","last_clean":"2024-02-18T00:42:44.035840+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:42:44.035840+0000","last_undegraded":"2024-02-18T00:42:44.035840+0000","last_fullsized":"2024-02-18T00:42:44.035840+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:42:20.356203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:20.356203+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":606,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.037105+0000","last_change":"2024-02-18T00:41:14.821572+0000","last_active":"2024-02-18T00:42:44.037105+0000","last_peered":"2024-02-18T00:42:44.037105+0000","last_clean":"2024-02-18T00:42:44.037105+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:42:44.037105+0000","last_undegraded":"2024-02-18T00:42:44.037105+0000","last_fullsized":"2024-02-18T00:42:44.037105+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:14.821507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:14.821507+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":607,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036250+0000","last_change":"2024-02-18T00:41:24.779552+0000","last_active":"2024-02-18T00:42:44.036250+0000","last_peered":"2024-02-18T00:42:44.036250+0000","last_clean":"2024-02-18T00:42:44.036250+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:42:44.036250+0000","last_undegraded":"2024-02-18T00:42:44.036250+0000","last_fullsized":"2024-02-18T00:42:44.036250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:24.779481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:24.779481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":589,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989805+0000","last_change":"2024-02-18T00:40:52.018692+0000","last_active":"2024-02-18T00:42:44.989805+0000","last_peered":"2024-02-18T00:42:44.989805+0000","last_clean":"2024-02-18T00:42:44.989805+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:42:44.989805+0000","last_undegraded":"2024-02-18T00:42:44.989805+0000","last_fullsized":"2024-02-18T00:42:44.989805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.018627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.018627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":538,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032601+0000","last_change":"2024-02-18T00:40:34.828210+0000","last_active":"2024-02-18T00:42:44.032601+0000","last_peered":"2024-02-18T00:42:44.032601+0000","last_clean":"2024-02-18T00:42:44.032601+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:42:44.032601+0000","last_undegraded":"2024-02-18T00:42:44.032601+0000","last_fullsized":"2024-02-18T00:42:44.032601+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:34.828148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:40:34.828148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":605,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989784+0000","last_change":"2024-02-18T00:40:43.113198+0000","last_active":"2024-02-18T00:42:44.989784+0000","last_peered":"2024-02-18T00:42:44.989784+0000","last_clean":"2024-02-18T00:42:44.989784+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:42:44.989784+0000","last_undegraded":"2024-02-18T00:42:44.989784+0000","last_fullsized":"2024-02-18T00:42:44.989784+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-18T00:40:43.113109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:43.113109+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":603,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739810+0000","last_change":"2024-02-18T00:40:50.609050+0000","last_active":"2024-02-18T00:42:44.739810+0000","last_peered":"2024-02-18T00:42:44.739810+0000","last_clean":"2024-02-18T00:42:44.739810+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:42:44.739810+0000","last_undegraded":"2024-02-18T00:42:44.739810+0000","last_fullsized":"2024-02-18T00:42:44.739810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.608983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:30.789634+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.608983+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":617,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032053+0000","last_change":"2024-02-18T00:41:35.739124+0000","last_active":"2024-02-18T00:42:44.032053+0000","last_peered":"2024-02-18T00:42:44.032053+0000","last_clean":"2024-02-18T00:42:44.032053+0000","last_became_active":"2024-02-18T00:39:36.750866+0000","last_became_peered":"2024-02-18T00:39:36.750866+0000","last_unstale":"2024-02-18T00:42:44.032053+0000","last_undegraded":"2024-02-18T00:42:44.032053+0000","last_fullsized":"2024-02-18T00:42:44.032053+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:35.739045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:35.739045+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":177,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690635+0000","last_change":"2024-02-18T00:42:25.583900+0000","last_active":"2024-02-18T00:42:44.690635+0000","last_peered":"2024-02-18T00:42:44.690635+0000","last_clean":"2024-02-18T00:42:44.690635+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:42:44.690635+0000","last_undegraded":"2024-02-18T00:42:44.690635+0000","last_fullsized":"2024-02-18T00:42:44.690635+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.583823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.583823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":556,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739751+0000","last_change":"2024-02-18T00:41:18.581382+0000","last_active":"2024-02-18T00:42:44.739751+0000","last_peered":"2024-02-18T00:42:44.739751+0000","last_clean":"2024-02-18T00:42:44.739751+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:42:44.739751+0000","last_undegraded":"2024-02-18T00:42:44.739751+0000","last_fullsized":"2024-02-18T00:42:44.739751+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:18.581310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.581310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":685,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990172+0000","last_change":"2024-02-18T00:41:18.896665+0000","last_active":"2024-02-18T00:42:44.990172+0000","last_peered":"2024-02-18T00:42:44.990172+0000","last_clean":"2024-02-18T00:42:44.990172+0000","last_became_active":"2024-02-18T00:39:36.761140+0000","last_became_peered":"2024-02-18T00:39:36.761140+0000","last_unstale":"2024-02-18T00:42:44.990172+0000","last_undegraded":"2024-02-18T00:42:44.990172+0000","last_fullsized":"2024-02-18T00:42:44.990172+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:18.896586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.896586+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":603,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032769+0000","last_change":"2024-02-18T00:41:23.548946+0000","last_active":"2024-02-18T00:42:44.032769+0000","last_peered":"2024-02-18T00:42:44.032769+0000","last_clean":"2024-02-18T00:42:44.032769+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:42:44.032769+0000","last_undegraded":"2024-02-18T00:42:44.032769+0000","last_fullsized":"2024-02-18T00:42:44.032769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.548872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:11.551602+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.548872+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690821+0000","last_change":"2024-02-18T00:42:06.606911+0000","last_active":"2024-02-18T00:42:44.690821+0000","last_peered":"2024-02-18T00:42:44.690821+0000","last_clean":"2024-02-18T00:42:44.690821+0000","last_became_active":"2024-02-18T00:39:36.758138+0000","last_became_peered":"2024-02-18T00:39:36.758138+0000","last_unstale":"2024-02-18T00:42:44.690821+0000","last_undegraded":"2024-02-18T00:42:44.690821+0000","last_fullsized":"2024-02-18T00:42:44.690821+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:06.606773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.606773+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":589,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.991077+0000","last_change":"2024-02-18T00:40:51.008109+0000","last_active":"2024-02-18T00:42:44.991077+0000","last_peered":"2024-02-18T00:42:44.991077+0000","last_clean":"2024-02-18T00:42:44.991077+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:42:44.991077+0000","last_undegraded":"2024-02-18T00:42:44.991077+0000","last_fullsized":"2024-02-18T00:42:44.991077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.008036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.008036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":177,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690075+0000","last_change":"2024-02-18T00:42:24.598343+0000","last_active":"2024-02-18T00:42:44.690075+0000","last_peered":"2024-02-18T00:42:44.690075+0000","last_clean":"2024-02-18T00:42:44.690075+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:42:44.690075+0000","last_undegraded":"2024-02-18T00:42:44.690075+0000","last_fullsized":"2024-02-18T00:42:44.690075+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:24.598257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:24.598257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":700,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507623+0000","last_change":"2024-02-18T00:42:08.213474+0000","last_active":"2024-02-18T00:42:44.507623+0000","last_peered":"2024-02-18T00:42:44.507623+0000","last_clean":"2024-02-18T00:42:44.507623+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:42:44.507623+0000","last_undegraded":"2024-02-18T00:42:44.507623+0000","last_fullsized":"2024-02-18T00:42:44.507623+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:42:08.213407+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:42:08.213407+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":625,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739111+0000","last_change":"2024-02-18T00:41:15.566910+0000","last_active":"2024-02-18T00:42:44.739111+0000","last_peered":"2024-02-18T00:42:44.739111+0000","last_clean":"2024-02-18T00:42:44.739111+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:42:44.739111+0000","last_undegraded":"2024-02-18T00:42:44.739111+0000","last_fullsized":"2024-02-18T00:42:44.739111+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:15.566820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.566820+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"466'10","reported_seq":804,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:48.430608+0000","last_change":"2024-02-18T00:41:21.505578+0000","last_active":"2024-02-18T00:42:48.430608+0000","last_peered":"2024-02-18T00:42:48.430608+0000","last_clean":"2024-02-18T00:42:48.430608+0000","last_became_active":"2024-02-18T00:39:36.751716+0000","last_became_peered":"2024-02-18T00:39:36.751716+0000","last_unstale":"2024-02-18T00:42:48.430608+0000","last_undegraded":"2024-02-18T00:42:48.430608+0000","last_fullsized":"2024-02-18T00:42:48.430608+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:41:21.505495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.505495+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":177,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690560+0000","last_change":"2024-02-18T00:42:26.623217+0000","last_active":"2024-02-18T00:42:44.690560+0000","last_peered":"2024-02-18T00:42:44.690560+0000","last_clean":"2024-02-18T00:42:44.690560+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:42:44.690560+0000","last_undegraded":"2024-02-18T00:42:44.690560+0000","last_fullsized":"2024-02-18T00:42:44.690560+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:26.623073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:26.623073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":647,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033031+0000","last_change":"2024-02-18T00:41:17.546351+0000","last_active":"2024-02-18T00:42:44.033031+0000","last_peered":"2024-02-18T00:42:44.033031+0000","last_clean":"2024-02-18T00:42:44.033031+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:42:44.033031+0000","last_undegraded":"2024-02-18T00:42:44.033031+0000","last_fullsized":"2024-02-18T00:42:44.033031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:17.546234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.546234+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":595,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989913+0000","last_change":"2024-02-18T00:39:54.855516+0000","last_active":"2024-02-18T00:42:44.989913+0000","last_peered":"2024-02-18T00:42:44.989913+0000","last_clean":"2024-02-18T00:42:44.989913+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:42:44.989913+0000","last_undegraded":"2024-02-18T00:42:44.989913+0000","last_fullsized":"2024-02-18T00:42:44.989913+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:39:54.855452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:39:54.855452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":614,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507642+0000","last_change":"2024-02-18T00:42:19.189296+0000","last_active":"2024-02-18T00:42:44.507642+0000","last_peered":"2024-02-18T00:42:44.507642+0000","last_clean":"2024-02-18T00:42:44.507642+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:42:44.507642+0000","last_undegraded":"2024-02-18T00:42:44.507642+0000","last_fullsized":"2024-02-18T00:42:44.507642+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.189231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.189231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":557,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032598+0000","last_change":"2024-02-18T00:41:27.140619+0000","last_active":"2024-02-18T00:42:44.032598+0000","last_peered":"2024-02-18T00:42:44.032598+0000","last_clean":"2024-02-18T00:42:44.032598+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:42:44.032598+0000","last_undegraded":"2024-02-18T00:42:44.032598+0000","last_fullsized":"2024-02-18T00:42:44.032598+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:41:27.140537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.140537+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":650,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033533+0000","last_change":"2024-02-18T00:41:24.123727+0000","last_active":"2024-02-18T00:42:44.033533+0000","last_peered":"2024-02-18T00:42:44.033533+0000","last_clean":"2024-02-18T00:42:44.033533+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:42:44.033533+0000","last_undegraded":"2024-02-18T00:42:44.033533+0000","last_fullsized":"2024-02-18T00:42:44.033533+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:41:24.123655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:24.123655+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":628,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032150+0000","last_change":"2024-02-18T00:42:13.925258+0000","last_active":"2024-02-18T00:42:44.032150+0000","last_peered":"2024-02-18T00:42:44.032150+0000","last_clean":"2024-02-18T00:42:44.032150+0000","last_became_active":"2024-02-18T00:39:36.741489+0000","last_became_peered":"2024-02-18T00:39:36.741489+0000","last_unstale":"2024-02-18T00:42:44.032150+0000","last_undegraded":"2024-02-18T00:42:44.032150+0000","last_fullsized":"2024-02-18T00:42:44.032150+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:13.925195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.925195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":607,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.037462+0000","last_change":"2024-02-18T00:42:12.788524+0000","last_active":"2024-02-18T00:42:44.037462+0000","last_peered":"2024-02-18T00:42:44.037462+0000","last_clean":"2024-02-18T00:42:44.037462+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:42:44.037462+0000","last_undegraded":"2024-02-18T00:42:44.037462+0000","last_fullsized":"2024-02-18T00:42:44.037462+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:12.788462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:42:12.788462+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":543,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.032644+0000","last_change":"2024-02-18T00:42:16.894617+0000","last_active":"2024-02-18T00:42:44.032644+0000","last_peered":"2024-02-18T00:42:44.032644+0000","last_clean":"2024-02-18T00:42:44.032644+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:42:44.032644+0000","last_undegraded":"2024-02-18T00:42:44.032644+0000","last_fullsized":"2024-02-18T00:42:44.032644+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_clean_scrub_stamp":"2024-02-18T00:42:16.894524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":640,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990345+0000","last_change":"2024-02-18T00:41:21.991859+0000","last_active":"2024-02-18T00:42:44.990345+0000","last_peered":"2024-02-18T00:42:44.990345+0000","last_clean":"2024-02-18T00:42:44.990345+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:42:44.990345+0000","last_undegraded":"2024-02-18T00:42:44.990345+0000","last_fullsized":"2024-02-18T00:42:44.990345+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:41:21.991789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.991789+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":633,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990913+0000","last_change":"2024-02-18T00:42:13.912023+0000","last_active":"2024-02-18T00:42:44.990913+0000","last_peered":"2024-02-18T00:42:44.990913+0000","last_clean":"2024-02-18T00:42:44.990913+0000","last_became_active":"2024-02-18T00:39:36.761839+0000","last_became_peered":"2024-02-18T00:39:36.761839+0000","last_unstale":"2024-02-18T00:42:44.990913+0000","last_undegraded":"2024-02-18T00:42:44.990913+0000","last_fullsized":"2024-02-18T00:42:44.990913+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:42:13.911916+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.911916+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":602,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.036066+0000","last_change":"2024-02-18T00:42:25.359836+0000","last_active":"2024-02-18T00:42:44.036066+0000","last_peered":"2024-02-18T00:42:44.036066+0000","last_clean":"2024-02-18T00:42:44.036066+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:42:44.036066+0000","last_undegraded":"2024-02-18T00:42:44.036066+0000","last_fullsized":"2024-02-18T00:42:44.036066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.359763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.359763+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":559,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507633+0000","last_change":"2024-02-18T00:41:25.051216+0000","last_active":"2024-02-18T00:42:44.507633+0000","last_peered":"2024-02-18T00:42:44.507633+0000","last_clean":"2024-02-18T00:42:44.507633+0000","last_became_active":"2024-02-18T00:39:36.771648+0000","last_became_peered":"2024-02-18T00:39:36.771648+0000","last_unstale":"2024-02-18T00:42:44.507633+0000","last_undegraded":"2024-02-18T00:42:44.507633+0000","last_fullsized":"2024-02-18T00:42:44.507633+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:25.051146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:25.051146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":668,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035574+0000","last_change":"2024-02-18T00:40:46.132082+0000","last_active":"2024-02-18T00:42:44.035574+0000","last_peered":"2024-02-18T00:42:44.035574+0000","last_clean":"2024-02-18T00:42:44.035574+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:42:44.035574+0000","last_undegraded":"2024-02-18T00:42:44.035574+0000","last_fullsized":"2024-02-18T00:42:44.035574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:40:46.132007+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:40:46.132007+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689798+0000","last_change":"2024-02-18T00:41:17.540872+0000","last_active":"2024-02-18T00:42:44.689798+0000","last_peered":"2024-02-18T00:42:44.689798+0000","last_clean":"2024-02-18T00:42:44.689798+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:42:44.689798+0000","last_undegraded":"2024-02-18T00:42:44.689798+0000","last_fullsized":"2024-02-18T00:42:44.689798+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.540791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.540791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":626,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.739572+0000","last_change":"2024-02-18T00:41:37.673766+0000","last_active":"2024-02-18T00:42:44.739572+0000","last_peered":"2024-02-18T00:42:44.739572+0000","last_clean":"2024-02-18T00:42:44.739572+0000","last_became_active":"2024-02-18T00:39:36.749467+0000","last_became_peered":"2024-02-18T00:39:36.749467+0000","last_unstale":"2024-02-18T00:42:44.739572+0000","last_undegraded":"2024-02-18T00:42:44.739572+0000","last_fullsized":"2024-02-18T00:42:44.739572+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_clean_scrub_stamp":"2024-02-18T00:41:37.673673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":544,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.033229+0000","last_change":"2024-02-18T00:42:21.695753+0000","last_active":"2024-02-18T00:42:44.033229+0000","last_peered":"2024-02-18T00:42:44.033229+0000","last_clean":"2024-02-18T00:42:44.033229+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:42:44.033229+0000","last_undegraded":"2024-02-18T00:42:44.033229+0000","last_fullsized":"2024-02-18T00:42:44.033229+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:21.695689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:21.695689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":635,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035787+0000","last_change":"2024-02-18T00:42:04.345907+0000","last_active":"2024-02-18T00:42:44.035787+0000","last_peered":"2024-02-18T00:42:44.035787+0000","last_clean":"2024-02-18T00:42:44.035787+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:42:44.035787+0000","last_undegraded":"2024-02-18T00:42:44.035787+0000","last_fullsized":"2024-02-18T00:42:44.035787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:42:04.345831+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:42:04.345831+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":177,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.689999+0000","last_change":"2024-02-18T00:42:14.524534+0000","last_active":"2024-02-18T00:42:44.689999+0000","last_peered":"2024-02-18T00:42:44.689999+0000","last_clean":"2024-02-18T00:42:44.689999+0000","last_became_active":"2024-02-18T00:39:36.761469+0000","last_became_peered":"2024-02-18T00:39:36.761469+0000","last_unstale":"2024-02-18T00:42:44.689999+0000","last_undegraded":"2024-02-18T00:42:44.689999+0000","last_fullsized":"2024-02-18T00:42:44.689999+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:14.524453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:42:14.524453+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":172,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.690769+0000","last_change":"2024-02-18T00:42:05.612003+0000","last_active":"2024-02-18T00:42:44.690769+0000","last_peered":"2024-02-18T00:42:44.690769+0000","last_clean":"2024-02-18T00:42:44.690769+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:42:44.690769+0000","last_undegraded":"2024-02-18T00:42:44.690769+0000","last_fullsized":"2024-02-18T00:42:44.690769+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:05.611891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:42:05.611891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":553,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990645+0000","last_change":"2024-02-18T00:41:27.921493+0000","last_active":"2024-02-18T00:42:44.990645+0000","last_peered":"2024-02-18T00:42:44.990645+0000","last_clean":"2024-02-18T00:42:44.990645+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:42:44.990645+0000","last_undegraded":"2024-02-18T00:42:44.990645+0000","last_fullsized":"2024-02-18T00:42:44.990645+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:27.921417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.921417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":644,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035397+0000","last_change":"2024-02-18T00:41:21.035047+0000","last_active":"2024-02-18T00:42:44.035397+0000","last_peered":"2024-02-18T00:42:44.035397+0000","last_clean":"2024-02-18T00:42:44.035397+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:42:44.035397+0000","last_undegraded":"2024-02-18T00:42:44.035397+0000","last_fullsized":"2024-02-18T00:42:44.035397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:41:21.034941+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.034941+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":595,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.991100+0000","last_change":"2024-02-18T00:41:22.988106+0000","last_active":"2024-02-18T00:42:44.991100+0000","last_peered":"2024-02-18T00:42:44.991100+0000","last_clean":"2024-02-18T00:42:44.991100+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:42:44.991100+0000","last_undegraded":"2024-02-18T00:42:44.991100+0000","last_fullsized":"2024-02-18T00:42:44.991100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:22.988039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:41:22.988039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":620,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035921+0000","last_change":"2024-02-18T00:40:42.622606+0000","last_active":"2024-02-18T00:42:44.035921+0000","last_peered":"2024-02-18T00:42:44.035921+0000","last_clean":"2024-02-18T00:42:44.035921+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:42:44.035921+0000","last_undegraded":"2024-02-18T00:42:44.035921+0000","last_fullsized":"2024-02-18T00:42:44.035921+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:42.622532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.622532+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":535,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.990880+0000","last_change":"2024-02-18T00:41:32.876920+0000","last_active":"2024-02-18T00:42:44.990880+0000","last_peered":"2024-02-18T00:42:44.990880+0000","last_clean":"2024-02-18T00:42:44.990880+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:42:44.990880+0000","last_undegraded":"2024-02-18T00:42:44.990880+0000","last_fullsized":"2024-02-18T00:42:44.990880+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.876858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.876858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":698,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.989890+0000","last_change":"2024-02-18T00:42:09.796340+0000","last_active":"2024-02-18T00:42:44.989890+0000","last_peered":"2024-02-18T00:42:44.989890+0000","last_clean":"2024-02-18T00:42:44.989890+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:42:44.989890+0000","last_undegraded":"2024-02-18T00:42:44.989890+0000","last_fullsized":"2024-02-18T00:42:44.989890+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-18T00:42:09.796275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:09.796275+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":620,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.507491+0000","last_change":"2024-02-18T00:41:17.135859+0000","last_active":"2024-02-18T00:42:44.507491+0000","last_peered":"2024-02-18T00:42:44.507491+0000","last_clean":"2024-02-18T00:42:44.507491+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:42:44.507491+0000","last_undegraded":"2024-02-18T00:42:44.507491+0000","last_fullsized":"2024-02-18T00:42:44.507491+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.135767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.135767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":602,"reported_epoch":628,"state":"active+clean","last_fresh":"2024-02-18T00:42:44.035537+0000","last_change":"2024-02-18T00:42:22.362932+0000","last_active":"2024-02-18T00:42:44.035537+0000","last_peered":"2024-02-18T00:42:44.035537+0000","last_clean":"2024-02-18T00:42:44.035537+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:42:44.035537+0000","last_undegraded":"2024-02-18T00:42:44.035537+0000","last_fullsized":"2024-02-18T00:42:44.035537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.362852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.362852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":237,"num_pg":32,"stat_sum":{"num_bytes":10,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":59,"num_objects":10,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":4,"num_read_kb":1,"num_write":10,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":110592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":24,"acting":24,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1511,"num_read_kb":1236,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":3196,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":923,"ondisk_log_size":923,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":806,"num_read_kb":801,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266855,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309192,"kb_used_data":3776,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435960,"statfs":{"total":95995035648,"available":95678423040,"internally_reserved":0,"allocated":3866624,"data_stored":2001830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.55300000000000005,"15min":0.57299999999999995},"min":{"1min":0.30499999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.7389999999999999,"5min":4.7389999999999999,"15min":6.8700000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.499,"15min":0.59299999999999997},"min":{"1min":0.22500000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.873,"5min":1.0780000000000001,"15min":11.288},"last":0.56999999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53500000000000003,"15min":0.623},"min":{"1min":0.309,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":7.4729999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.64000000000000001},"min":{"1min":0.34899999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82899999999999996,"5min":1.1930000000000001,"15min":7.7869999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":4.8220000000000001,"5min":4.8220000000000001,"15min":6.7380000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.52700000000000002,"15min":0.60099999999999998},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":1.913,"5min":1.913,"15min":12.861000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.8520000000000003,"5min":4.8520000000000003,"15min":7.1360000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53800000000000003,"15min":0.58399999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.948,"5min":1.948,"15min":3.7989999999999999},"last":0.54900000000000004}]},{"osd":4,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45500000000000002,"15min":0.433},"min":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.215},"max":{"1min":1.115,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.55400000000000005,"15min":0.48099999999999998},"min":{"1min":0.317,"5min":0.252,"15min":0.246},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.46800000000000003,"15min":0.46200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24099999999999999},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.25600000000000001},"max":{"1min":1.853,"5min":1.853,"15min":4.9130000000000003},"last":0.49399999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.55000000000000004,"15min":0.52100000000000002},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.60199999999999998,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.3860000000000001,"5min":5.3860000000000001,"15min":5.3860000000000001},"last":0.627}]}]},{"osd":6,"up_from":47,"seq":201863463085,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301012,"kb_used_data":3788,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93444140,"statfs":{"total":95995035648,"available":95686799360,"internally_reserved":0,"allocated":3878912,"data_stored":2019533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":1.262},"min":{"1min":0.39000000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.75800000000000001,"5min":4.0129999999999999,"15min":69.194999999999993},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.73799999999999999},"min":{"1min":0.35099999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72899999999999998,"5min":1.7170000000000001,"15min":10.433},"last":0.58899999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":1.337},"min":{"1min":0.313,"5min":0.29399999999999998,"15min":0.26000000000000001},"max":{"1min":0.754,"5min":1.788,"15min":76.108999999999995},"last":0.625},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51300000000000001,"15min":1.321},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.28100000000000003},"max":{"1min":0.72599999999999998,"5min":1.1519999999999999,"15min":71.872},"last":0.46000000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52000000000000002,"15min":0.80800000000000005},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.31},"max":{"1min":0.755,"5min":1.196,"15min":39.939999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56899999999999995,"15min":0.98599999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77800000000000002,"5min":3.1600000000000001,"15min":39.673000000000002},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Feb 18 00:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61199999999999999,"15min":0.83999999999999997},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.29599999999999999},"max":{"1min":0.65400000000000003,"5min":8.3420000000000005,"15min":21.219000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53600000000000003,"15min":0.78300000000000003},"min":{"1min":0.40899999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73199999999999998,"5min":1.506,"15min":14.007999999999999},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.437,"15min":0.51300000000000001},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.22800000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":8.1479999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.69399999999999995},"min":{"1min":0.315,"5min":0.28000000000000003,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":0.79500000000000004,"15min":35.18},"last":0.36099999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.54200000000000004},"min":{"1min":0.30399999999999999,"5min":0.29299999999999998,"15min":0.27500000000000002},"max":{"1min":0.65900000000000003,"5min":0.755,"15min":10.606},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.49399999999999999,"15min":0.72199999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.71299999999999997,"5min":0.83399999999999996,"15min":35.119},"last":0.47699999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.55100000000000005},"min":{"1min":0.35099999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66400000000000003,"5min":0.76700000000000002,"15min":5.4770000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52700000000000002,"15min":0.72699999999999998},"min":{"1min":0.378,"5min":0.32600000000000001,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":32.985999999999997},"last":0.40600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313280,"kb_used_data":3768,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431872,"statfs":{"total":95995035648,"available":95674236928,"internally_reserved":0,"allocated":3858432,"data_stored":1985773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:10 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.64600000000000002},"min":{"1min":0.216,"5min":0.20599999999999999,"15min":0.193},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":32.106999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.45100000000000001,"15min":0.71299999999999997},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.71899999999999997,"5min":6.625,"15min":43.137},"last":0.40400000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36699999999999999,"15min":0.75800000000000001},"min":{"1min":0.23799999999999999,"5min":0.23499999999999999,"15min":0.19900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":49.201000000000001},"last":0.437},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.40100000000000002,"15min":0.59199999999999997},"min":{"1min":0.247,"5min":0.23999999999999999,"15min":0.217},"max":{"1min":0.748,"5min":0.83199999999999996,"15min":17.596},"last":0.45200000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:41:52 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.40000000000000002,"15min":0.72999999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.51300000000000001,"5min":0.92800000000000005,"15min":37.524000000000001},"last":0.502},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.42199999999999999,"15min":0.748},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":0.52100000000000002,"5min":0.626,"15min":32.036000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":17.486999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.77700000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.32500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":12.75},"last":0.52100000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.76800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":12.611000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.78100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33100000000000002},"max":{"1min":1.02,"5min":1.02,"15min":17.417000000000002},"last":0.59899999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:42:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60799999999999998,"15min":0.80300000000000005},"min":{"1min":0.443,"5min":0.40400000000000003,"15min":0.35899999999999999},"max":{"1min":0.92300000000000004,"5min":2.0430000000000001,"15min":12.834},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59299999999999997,"15min":0.78600000000000003},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.35499999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":12.651999999999999},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.79700000000000004},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":17.452000000000002},"last":0.63},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.80700000000000005},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":1.012,"5min":1.012,"15min":12.805},"last":0.66100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738570,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313292,"kb_used_data":3780,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431860,"statfs":{"total":95995035648,"available":95674224640,"internally_reserved":0,"allocated":3870720,"data_stored":2005731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:42:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34899999999999998,"15min":1.016},"min":{"1min":0.217,"5min":0.19500000000000001,"15min":0.18099999999999999},"max":{"1min":0.56499999999999995,"5min":0.873,"15min":133.38999999999999},"last":0.38},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.42499999999999999,"15min":0.79100000000000004},"min":{"1min":0.29199999999999998,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":0.57599999999999996,"5min":2.7999999999999998,"15min":90.623000000000005},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42299999999999999,"15min":0.90100000000000002},"min":{"1min":0.27500000000000002,"5min":0.25700000000000001,"15min":0.20200000000000001},"max":{"1min":0.61799999999999999,"5min":1.782,"15min":90.850999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.36699999999999999,"15min":0.86599999999999999},"min":{"1min":0.27400000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54300000000000004,"5min":1.482,"15min":80.537000000000006},"last":0.36099999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:42:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.38200000000000001,"15min":0.65400000000000003},"min":{"1min":0.24399999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53000000000000003,"5min":1.1359999999999999,"15min":19.006},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.44600000000000001,"15min":1.056},"min":{"1min":0.29199999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63200000000000001,"5min":1.6910000000000001,"15min":146.71899999999999},"last":0.55800000000000005}]},{"osd":4,"last update":"Sun Feb 18 00:42:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.92400000000000004},"min":{"1min":0.41899999999999998,"5min":0.33600000000000002,"15min":0.26700000000000002},"max":{"1min":0.748,"5min":1.6830000000000001,"15min":57.914000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61099999999999999,"15min":0.97999999999999998},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.30399999999999999},"max":{"1min":0.88800000000000001,"5min":1.6619999999999999,"15min":90.070999999999998},"last":0.76300000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:42:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59399999999999997,"15min":0.95299999999999996},"min":{"1min":0.435,"5min":0.36499999999999999,"15min":0.32000000000000001},"max":{"1min":0.86499999999999999,"5min":1.714,"15min":90.049000000000007},"last":0.627},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56799999999999995,"15min":0.98399999999999999},"min":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.86399999999999999,"15min":57.991},"last":0.74299999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:42:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59199999999999997,"15min":0.80000000000000004},"min":{"1min":0.47599999999999998,"5min":0.38100000000000001,"15min":0.35999999999999999},"max":{"1min":0.90700000000000003,"5min":1.0129999999999999,"15min":19.039000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60999999999999999,"15min":0.97999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.33500000000000002},"max":{"1min":0.77700000000000002,"5min":1.76,"15min":89.989999999999995},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":1.0529999999999999},"min":{"1min":0.47999999999999998,"5min":0.36799999999999999,"15min":0.36499999999999999},"max":{"1min":1.01,"5min":1.732,"15min":90.025000000000006},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.603,"15min":57.962000000000003},"last":0.68799999999999994}]}]},{"osd":2,"up_from":18,"seq":77309411521,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313256,"kb_used_data":3744,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431896,"statfs":{"total":95995035648,"available":95674261504,"internally_reserved":0,"allocated":3833856,"data_stored":1985031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.48499999999999999},"min":{"1min":0.251,"5min":0.192,"15min":0.192},"max":{"1min":0.54200000000000004,"5min":0.78500000000000003,"15min":12.144},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.60299999999999998},"min":{"1min":0.23000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58199999999999996,"5min":0.63600000000000001,"15min":18.891999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.375,"15min":0.495},"min":{"1min":0.24399999999999999,"5min":0.23300000000000001,"15min":0.19700000000000001},"max":{"1min":0.63500000000000001,"5min":0.76800000000000002,"15min":10.138999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.372,"15min":0.53800000000000003},"min":{"1min":0.245,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":18.919},"last":0.48299999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:41:53 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.59299999999999997},"min":{"1min":0.29899999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.55300000000000005,"5min":0.67400000000000004,"15min":22.712},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39900000000000002,"15min":0.59499999999999997},"min":{"1min":0.29999999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":22.359000000000002},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:42:27 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58499999999999996,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.32500000000000001},"max":{"1min":0.88900000000000001,"5min":0.94899999999999995,"15min":18.812000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.67100000000000004},"min":{"1min":0.42699999999999999,"5min":0.36099999999999999,"15min":0.28699999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":10.016},"last":0.69599999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.65100000000000002},"min":{"1min":0.437,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82499999999999996,"5min":0.96699999999999997,"15min":6.766},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.68600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.317},"max":{"1min":0.871,"5min":0.92400000000000004,"15min":10.114000000000001},"last":0.77500000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:42:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57599999999999996,"15min":0.66200000000000003},"min":{"1min":0.41999999999999998,"5min":0.38,"15min":0.35799999999999998},"max":{"1min":0.78400000000000003,"5min":0.874,"15min":6.806},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.71099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.82199999999999995,"5min":0.92100000000000004,"15min":18.861000000000001},"last":0.751}]},{"osd":7,"last update":"Sun Feb 18 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.68999999999999995},"min":{"1min":0.439,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.95499999999999996,"15min":6.8300000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.45200000000000001,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":1.149,"15min":10.058999999999999},"last":0.80700000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149883,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309220,"kb_used_data":3804,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93435932,"statfs":{"total":95995035648,"available":95678394368,"internally_reserved":0,"allocated":3895296,"data_stored":2006847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.371,"15min":0.46800000000000003},"min":{"1min":0.25800000000000001,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":0.56699999999999995,"5min":0.92200000000000004,"15min":25.082999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39800000000000002,"15min":0.51900000000000002},"min":{"1min":0.28299999999999997,"5min":0.23699999999999999,"15min":0.20599999999999999},"max":{"1min":0.54100000000000004,"5min":1.2110000000000001,"15min":18.911000000000001},"last":0.433}]},{"osd":1,"last update":"Sun Feb 18 00:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38700000000000001,"15min":0.46600000000000003},"min":{"1min":0.307,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.56299999999999994,"5min":0.61299999999999999,"15min":6.8479999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.47699999999999998,"15min":0.69099999999999995},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.20599999999999999},"max":{"1min":0.68999999999999995,"5min":8.0779999999999994,"15min":23.606999999999999},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:41:56 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45600000000000002,"15min":0.65400000000000003},"min":{"1min":0.32600000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.83699999999999997,"5min":4.0190000000000001,"15min":23.405000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41999999999999998,"15min":0.46300000000000002},"min":{"1min":0.30299999999999999,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.61699999999999999,"5min":1.4430000000000001,"15min":5.3479999999999999},"last":0.35399999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.93000000000000005,"5min":0.95499999999999996,"15min":10.131},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":1.1699999999999999,"15min":5.4630000000000001},"last":0.44800000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:41:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.623,"15min":0.63600000000000001},"min":{"1min":0.39800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":4.7469999999999999,"15min":7.4249999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.67800000000000005},"min":{"1min":0.47299999999999998,"5min":0.38500000000000001,"15min":0.32400000000000001},"max":{"1min":0.91300000000000003,"5min":1.0860000000000001,"15min":6.7750000000000004},"last":0.57499999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:42:04 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.748},"min":{"1min":0.47299999999999998,"5min":0.377,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.98199999999999998,"15min":10.189},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63600000000000001,"15min":0.66700000000000004},"min":{"1min":0.434,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.84899999999999998,"5min":2.484,"15min":6.4279999999999999},"last":0.55700000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:42:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.67100000000000004,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":5.0830000000000002,"15min":6.8220000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.626,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.39800000000000002},"max":{"1min":0.874,"5min":1.0760000000000001,"15min":10.164999999999999},"last":0.58799999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986358,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301008,"kb_used_data":3784,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93444144,"statfs":{"total":95995035648,"available":95686803456,"internally_reserved":0,"allocated":3874816,"data_stored":2006648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.874},"min":{"1min":0.373,"5min":0.247,"15min":0.247},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":77.581999999999994},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.48799999999999999,"15min":0.84299999999999997},"min":{"1min":0.33600000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63100000000000001,"5min":0.72499999999999998,"15min":78.063999999999993},"last":0.63500000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50700000000000001,"15min":0.62},"min":{"1min":0.379,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":9.3160000000000007},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.88200000000000001},"min":{"1min":0.35699999999999998,"5min":0.30099999999999999,"15min":0.27100000000000002},"max":{"1min":0.66000000000000003,"5min":0.73499999999999999,"15min":81.242000000000004},"last":0.56899999999999995}]},{"osd":2,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.72299999999999998,"15min":11.691000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52100000000000002,"15min":0.66000000000000003},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.27200000000000002},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":17.77},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52000000000000002,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.28299999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":7.218},"last":0.747},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55300000000000005,"15min":0.73099999999999998},"min":{"1min":0.40300000000000002,"5min":0.313,"15min":0.309},"max":{"1min":1.21,"5min":1.21,"15min":33.530999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.32900000000000001,"5min":0.23799999999999999,"15min":0.20300000000000001},"max":{"1min":0.65500000000000003,"5min":0.71899999999999997,"15min":0.72299999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.44},"min":{"1min":0.308,"5min":0.307,"15min":0.24299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47899999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.309,"5min":0.221,"15min":0.221},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.76100000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.36799999999999999,"5min":0.28299999999999997,"15min":0.26000000000000001},"max":{"1min":0.63500000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.502}]},{"osd":7,"last update":"Sun Feb 18 00:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.27900000000000003},"max":{"1min":0.70099999999999996,"5min":0.72899999999999998,"15min":0.76900000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.58799999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724721,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":317368,"kb_used_data":3760,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93427784,"statfs":{"total":95995035648,"available":95670050816,"internally_reserved":0,"allocated":3850240,"data_stored":1986424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51900000000000002,"15min":0.54300000000000004},"min":{"1min":0.35799999999999998,"5min":0.24399999999999999,"15min":0.217},"max":{"1min":2.21,"5min":2.21,"15min":6.3150000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.61199999999999999},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.782,"5min":2.1859999999999999,"15min":12.744},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.36499999999999999,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":2.1509999999999998,"5min":13.25,"15min":13.25},"last":0.626},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.34000000000000002,"5min":0.314,"15min":0.29699999999999999},"max":{"1min":1.9430000000000001,"5min":13.262,"15min":13.262},"last":0.36499999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.53800000000000003,"15min":0.59299999999999997},"min":{"1min":0.36399999999999999,"5min":0.27700000000000002,"15min":0.251},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":6.1680000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.60799999999999998,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.312,"15min":0.308},"max":{"1min":1.8799999999999999,"5min":5.0659999999999998,"15min":6.1879999999999997},"last":0.57299999999999995}]},{"osd":3,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.59799999999999998,"15min":0.63100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.28399999999999997},"max":{"1min":3.637,"5min":7.641,"15min":7.641},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.27600000000000002},"max":{"1min":1.853,"5min":7.4580000000000002,"15min":8.8650000000000002},"last":0.54200000000000004}]},{"osd":4,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.48399999999999999},"min":{"1min":0.29899999999999999,"5min":0.25,"15min":0.23599999999999999},"max":{"1min":1.897,"5min":1.897,"15min":11.818},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.496,"15min":0.47599999999999998},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":4.274},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:42:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.54600000000000004},"min":{"1min":0.33500000000000002,"5min":0.27500000000000002,"15min":0.26300000000000001},"max":{"1min":0.61499999999999999,"5min":2.5939999999999999,"15min":18.605},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69399999999999995,"5min":0.88600000000000001,"15min":4.2089999999999996},"last":0.50600000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:42:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.31900000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.819,"5min":1.819,"15min":2.0739999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.745,"5min":0.76000000000000001,"15min":4.649},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":237,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:42:55.965 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-02-18T00:42:55.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool set default.rgw.log pg_num 42 2024-02-18T00:42:56.042 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:56.072 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:56.174 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:42:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:42:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:42:56.174 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:42:56.174 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:42:56.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:54.823219+0000 osd.6 (osd.6) 287 : cluster [DBG] 4.a scrub starts 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:54.825403+0000 osd.6 (osd.6) 288 : cluster [DBG] 4.a scrub ok 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:55.612961+0000 osd.2 (osd.2) 253 : cluster [DBG] 2.e scrub starts 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:55.614284+0000 osd.2 (osd.2) 254 : cluster [DBG] 2.e scrub ok 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:55.770555+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:55.794286+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:54.823219+0000 osd.6 (osd.6) 287 : cluster [DBG] 4.a scrub starts 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:54.825403+0000 osd.6 (osd.6) 288 : cluster [DBG] 4.a scrub ok 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:55.612961+0000 osd.2 (osd.2) 253 : cluster [DBG] 2.e scrub starts 2024-02-18T00:42:56.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:55.614284+0000 osd.2 (osd.2) 254 : cluster [DBG] 2.e scrub ok 2024-02-18T00:42:56.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:55.770555+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-18T00:42:56.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:55.794286+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:56.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:54.823219+0000 osd.6 (osd.6) 287 : cluster [DBG] 4.a scrub starts 2024-02-18T00:42:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:54.825403+0000 osd.6 (osd.6) 288 : cluster [DBG] 4.a scrub ok 2024-02-18T00:42:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:55.612961+0000 osd.2 (osd.2) 253 : cluster [DBG] 2.e scrub starts 2024-02-18T00:42:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:55.614284+0000 osd.2 (osd.2) 254 : cluster [DBG] 2.e scrub ok 2024-02-18T00:42:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:55.770555+0000 mon.a (mon.0) 3175 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-18T00:42:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:55.794286+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:42:57.024 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:57.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:57.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:57.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-18T00:42:57.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:57.165 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:57.285 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:55.765000+0000 osd.7 (osd.7) 183 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:55.766057+0000 osd.7 (osd.7) 184 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:55.978569+0000 mgr.y (mgr.24454) 568 : cluster [DBG] pgmap v966: 204 pgs: 3 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 196 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.032567+0000 osd.3 (osd.3) 325 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.033415+0000 osd.3 (osd.3) 326 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.468830+0000 osd.4 (osd.4) 355 : cluster [DBG] 5.e scrub starts 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.472286+0000 osd.4 (osd.4) 356 : cluster [DBG] 5.e scrub ok 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:56.779447+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.779529+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:56.782146+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106"}]: dispatch 2024-02-18T00:42:57.286 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:55.765000+0000 osd.7 (osd.7) 183 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:55.766057+0000 osd.7 (osd.7) 184 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:55.978569+0000 mgr.y (mgr.24454) 568 : cluster [DBG] pgmap v966: 204 pgs: 3 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 196 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.032567+0000 osd.3 (osd.3) 325 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.033415+0000 osd.3 (osd.3) 326 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.468830+0000 osd.4 (osd.4) 355 : cluster [DBG] 5.e scrub starts 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.472286+0000 osd.4 (osd.4) 356 : cluster [DBG] 5.e scrub ok 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:56.779447+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.779529+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-18T00:42:57.287 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:56.782146+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106"}]: dispatch 2024-02-18T00:42:57.290 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:57.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:57.294 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:57.299 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:42:57.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:55.765000+0000 osd.7 (osd.7) 183 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:55.766057+0000 osd.7 (osd.7) 184 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:55.978569+0000 mgr.y (mgr.24454) 568 : cluster [DBG] pgmap v966: 204 pgs: 3 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 196 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.032567+0000 osd.3 (osd.3) 325 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.033415+0000 osd.3 (osd.3) 326 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.468830+0000 osd.4 (osd.4) 355 : cluster [DBG] 5.e scrub starts 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.472286+0000 osd.4 (osd.4) 356 : cluster [DBG] 5.e scrub ok 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:56.779447+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.779529+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-18T00:42:57.426 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:56.782146+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106"}]: dispatch 2024-02-18T00:42:57.428 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:57.450 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:42:57.452 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:57.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:42:57.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:42:57.584 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:57.605 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:42:57.610 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:57.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:42:57.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:57.935 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:42:57.935 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.892+0000 7f836500e700 1 -- 172.21.15.59:0/2693779888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600f8420 msgr2=0x7f8360104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.892+0000 7f836500e700 1 --2- 172.21.15.59:0/2693779888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600f8420 0x7f8360104a20 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 rx=0x7f8358009c30 tx=0x7f8358004c50).stop 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.892+0000 7f836500e700 1 -- 172.21.15.59:0/2693779888 shutdown_connections 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.892+0000 7f836500e700 1 --2- 172.21.15.59:0/2693779888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600f8420 0x7f8360104a20 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.892+0000 7f836500e700 1 --2- 172.21.15.59:0/2693779888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83600fdbb0 0x7f83600fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.892+0000 7f836500e700 1 --2- 172.21.15.59:0/2693779888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83600fe510 0x7f83600f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.892+0000 7f836500e700 1 -- 172.21.15.59:0/2693779888 >> 172.21.15.59:0/2693779888 conn(0x7f83600f3800 msgr2=0x7f83600f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 -- 172.21.15.59:0/2693779888 shutdown_connections 2024-02-18T00:42:57.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 -- 172.21.15.59:0/2693779888 wait complete. 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 librados: starting msgr at 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 librados: starting objecter 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 Processor -- start 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 -- start start 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 librados: setting wanted keys 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.893+0000 7f836500e700 1 librados: calling monclient init 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f836500e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83600f8420 0x7f836006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f836500e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600fdbb0 0x7f836006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f836500e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83600fe510 0x7f836006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f836500e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83601163f0 con 0x7f83600f8420 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f836500e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8360116560 con 0x7f83600fe510 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f836500e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8360116720 con 0x7f83600fdbb0 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600fdbb0 0x7f836006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83600f8420 0x7f836006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600fdbb0 0x7f836006b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:45644/0 (socket says 172.21.15.59:45644) 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835e59c700 1 -- 172.21.15.59:0/1323421181 learned_addr learned my addr 172.21.15.59:0/1323421181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835f59e700 1 --2- 172.21.15.59:0/1323421181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83600fe510 0x7f836006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835ed9d700 1 -- 172.21.15.59:0/1323421181 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600fdbb0 msgr2=0x7f836006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600fdbb0 0x7f836006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835ed9d700 1 -- 172.21.15.59:0/1323421181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83600fe510 msgr2=0x7f836006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83600fe510 0x7f836006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835ed9d700 1 -- 172.21.15.59:0/1323421181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350005040 con 0x7f83600f8420 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835e59c700 1 --2- 172.21.15.59:0/1323421181 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83600fdbb0 0x7f836006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83600f8420 0x7f836006b0d0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7f83500091f0 tx=0x7f8350009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f8347fff700 10 client.?.objecter ms_handle_connect 0x7f83600f8420 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f8347fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:57.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500079e0 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.894+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83580097c0 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.895+0000 7f836500e700 10 client.31758.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.895+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8360116e30 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.895+0000 7f836500e700 1 librados: init done 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.895+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f835000f040 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.895+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350014840 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f834c004f30 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8350010700 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8348062c90 0x7f8348065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(632..632 src has 418..632) v4 ==== 8705+0+0 (secure 0 0 0) 0x7f835007cf60 con 0x7f83600f8420 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 osd_map(632..632 src has 418..632) v4 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 3 client.31758.objecter handle_osd_map got epochs [632,632] > 0 2024-02-18T00:42:57.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 3 client.31758.objecter handle_osd_map decoding full epoch 632 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f835e59c700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8348062c90 0x7f8348065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 20 client.31758.objecter dump_active .. 0 homeless 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.896+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v632) v1 ==== 54+0+16 (secure 0 0 0) 0x7f835007f5f0 con 0x7f83600f8420 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 10 client.31758.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f835e59c700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8348062c90 0x7f8348065150 secure :-1 s=READY pgs=3181 cs=0 l=1 rev1=1 rx=0x7f8354004fd0 tx=0x7f835400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 1 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f8348062c90 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6801/3700238977 conn(0x7f834c004560 0x7f834c0049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 1 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 0 1 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0051d0 osd=0 4 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 10 client.31758.objecter _send_command 1 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c008960 con 0x7f834c004560 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f835f59e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6801/3700238977 conn(0x7f834c004560 0x7f834c0049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c004560 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.897+0000 7f835f59e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6801/3700238977 conn(0x7f834c004560 0x7f834c0049a0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.898+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c008960 con 0x7f834c004560 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.898+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.898+0000 7f8347fff700 10 client.31758.objecter _finish_command 1 = system:0 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.898+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c0051d0 osd=0 4 2024-02-18T00:42:57.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.898+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 0 1 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 10 client.31758.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 2 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 2 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c0051d0 osd=0 3 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 0 2 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0051d0 osd=0 4 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 10 client.31758.objecter _send_command 2 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.913+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c0081f0 con 0x7f834c004560 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c0081f0 con 0x7f834c004560 2024-02-18T00:42:57.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f8347fff700 10 client.31758.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c0051d0 osd=0 4 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 0 2 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 10 client.31758.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6805/26279110 conn(0x7f834c002c90 0x7f834c0030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 1 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c008610 osd=1 4 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 10 client.31758.objecter _send_command 3 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.914+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c003600 con 0x7f834c002c90 2024-02-18T00:42:57.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6805/26279110 conn(0x7f834c002c90 0x7f834c0030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c002c90 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6805/26279110 conn(0x7f834c002c90 0x7f834c0030d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c003600 con 0x7f834c002c90 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f8347fff700 10 client.31758.objecter _finish_command 3 = system:0 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c008610 osd=1 4 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.915+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 1 3 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 10 client.31758.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 4 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 4 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c008610 osd=1 3 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 1 4 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c008610 osd=1 4 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 10 client.31758.objecter _send_command 4 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.930+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c007d20 con 0x7f834c002c90 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c007d20 con 0x7f834c002c90 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f8347fff700 10 client.31758.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c008610 osd=1 4 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 1 4 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 10 client.31758.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 5 2024-02-18T00:42:57.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6809/1153193637 conn(0x7f834c004160 0x7f834c00d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 5 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 2 5 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c003ff0 osd=2 4 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 10 client.31758.objecter _send_command 5 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c00fec0 con 0x7f834c004160 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f835e59c700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6809/1153193637 conn(0x7f834c004160 0x7f834c00d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c004160 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.932+0000 7f835e59c700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6809/1153193637 conn(0x7f834c004160 0x7f834c00d000 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.933+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c00fec0 con 0x7f834c004160 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.933+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:42:57.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.933+0000 7f8347fff700 10 client.31758.objecter _finish_command 5 = system:0 2024-02-18T00:42:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.933+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c003ff0 osd=2 4 2024-02-18T00:42:57.947 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.933+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 2 5 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 10 client.31758.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 6 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 6 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c003ff0 osd=2 3 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 2 6 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c003ff0 osd=2 4 2024-02-18T00:42:57.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 10 client.31758.objecter _send_command 6 2024-02-18T00:42:57.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.948+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c003b20 con 0x7f834c004160 2024-02-18T00:42:57.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c003b20 con 0x7f834c004160 2024-02-18T00:42:57.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:57.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f8347fff700 10 client.31758.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c003ff0 osd=2 4 2024-02-18T00:42:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 2 6 2024-02-18T00:42:57.952 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 10 client.31758.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 7 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6813/1805700964 conn(0x7f834c00d610 0x7f834c00da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 7 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 3 7 2024-02-18T00:42:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c001bb0 osd=3 4 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 10 client.31758.objecter _send_command 7 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c002390 con 0x7f834c00d610 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.950+0000 7f835f59e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6813/1805700964 conn(0x7f834c00d610 0x7f834c00da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.951+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c00d610 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.951+0000 7f835f59e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6813/1805700964 conn(0x7f834c00d610 0x7f834c00da70 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.951+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c002390 con 0x7f834c00d610 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.951+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.951+0000 7f8347fff700 10 client.31758.objecter _finish_command 7 = system:0 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.951+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c001bb0 osd=3 4 2024-02-18T00:42:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.951+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 3 7 2024-02-18T00:42:57.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 10 client.31758.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:57.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 8 2024-02-18T00:42:57.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 8 2024-02-18T00:42:57.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c001bb0 osd=3 3 2024-02-18T00:42:57.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 3 8 2024-02-18T00:42:57.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c001bb0 osd=3 4 2024-02-18T00:42:57.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 10 client.31758.objecter _send_command 8 2024-02-18T00:42:57.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.966+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c00e730 con 0x7f834c00d610 2024-02-18T00:42:57.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c00e730 con 0x7f834c00d610 2024-02-18T00:42:57.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:57.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f8347fff700 10 client.31758.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:57.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c001bb0 osd=3 4 2024-02-18T00:42:57.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 3 8 2024-02-18T00:42:57.969 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 10 client.31758.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 9 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6800/2044387056 conn(0x7f834c0120f0 0x7f834c0144e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 9 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 4 9 2024-02-18T00:42:57.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c00df60 osd=4 4 2024-02-18T00:42:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 10 client.31758.objecter _send_command 9 2024-02-18T00:42:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.967+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c017130 con 0x7f834c0120f0 2024-02-18T00:42:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.968+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6800/2044387056 conn(0x7f834c0120f0 0x7f834c0144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.968+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c0120f0 2024-02-18T00:42:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.968+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6800/2044387056 conn(0x7f834c0120f0 0x7f834c0144e0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.969+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c017130 con 0x7f834c0120f0 2024-02-18T00:42:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.969+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:42:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.969+0000 7f8347fff700 10 client.31758.objecter _finish_command 9 = system:0 2024-02-18T00:42:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.969+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c00df60 osd=4 4 2024-02-18T00:42:57.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.969+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 4 9 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 10 client.31758.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 10 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 10 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c00df60 osd=4 3 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 4 10 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c00df60 osd=4 4 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 10 client.31758.objecter _send_command 10 2024-02-18T00:42:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.984+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c0149d0 con 0x7f834c0120f0 2024-02-18T00:42:57.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c0149d0 con 0x7f834c0120f0 2024-02-18T00:42:57.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:57.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f8347fff700 10 client.31758.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:57.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c00df60 osd=4 4 2024-02-18T00:42:57.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 4 10 2024-02-18T00:42:57.988 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:42:57.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 10 client.31758.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:57.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 11 2024-02-18T00:42:57.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6804/2170432263 conn(0x7f834c0150b0 0x7f834c0174c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:57.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:57.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:57.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 11 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 5 11 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0149d0 osd=5 4 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 10 client.31758.objecter _send_command 11 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.986+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c01a2d0 con 0x7f834c0150b0 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.987+0000 7f835e59c700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6804/2170432263 conn(0x7f834c0150b0 0x7f834c0174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.987+0000 7f835e59c700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6804/2170432263 conn(0x7f834c0150b0 0x7f834c0174c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.987+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c0150b0 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.988+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c01a2d0 con 0x7f834c0150b0 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.988+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.988+0000 7f8347fff700 10 client.31758.objecter _finish_command 11 = system:0 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.988+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c0149d0 osd=5 4 2024-02-18T00:42:57.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:57.988+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 5 11 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 10 client.31758.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 12 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 12 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c0149d0 osd=5 3 2024-02-18T00:42:58.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 5 12 2024-02-18T00:42:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0149d0 osd=5 4 2024-02-18T00:42:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 10 client.31758.objecter _send_command 12 2024-02-18T00:42:58.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.002+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c01a290 con 0x7f834c0150b0 2024-02-18T00:42:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.004+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c01a290 con 0x7f834c0150b0 2024-02-18T00:42:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.004+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.004+0000 7f8347fff700 10 client.31758.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.004+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c0149d0 osd=5 4 2024-02-18T00:42:58.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.004+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 5 12 2024-02-18T00:42:58.006 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:42:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 10 client.31758.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 13 2024-02-18T00:42:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6808/3356009390 conn(0x7f834c0195d0 0x7f834c019a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 13 2024-02-18T00:42:58.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 6 13 2024-02-18T00:42:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c01a2d0 osd=6 4 2024-02-18T00:42:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 10 client.31758.objecter _send_command 13 2024-02-18T00:42:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c01d660 con 0x7f834c0195d0 2024-02-18T00:42:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f835f59e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6808/3356009390 conn(0x7f834c0195d0 0x7f834c019a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f835f59e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6808/3356009390 conn(0x7f834c0195d0 0x7f834c019a50 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:58.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.005+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c0195d0 2024-02-18T00:42:58.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.006+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c01d660 con 0x7f834c0195d0 2024-02-18T00:42:58.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.006+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:42:58.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.006+0000 7f8347fff700 10 client.31758.objecter _finish_command 13 = system:0 2024-02-18T00:42:58.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.006+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c01a2d0 osd=6 4 2024-02-18T00:42:58.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.006+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 6 13 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 10 client.31758.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 14 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 14 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c01a2d0 osd=6 3 2024-02-18T00:42:58.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 6 14 2024-02-18T00:42:58.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c01a2d0 osd=6 4 2024-02-18T00:42:58.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 10 client.31758.objecter _send_command 14 2024-02-18T00:42:58.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.020+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c017a70 con 0x7f834c0195d0 2024-02-18T00:42:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c017a70 con 0x7f834c0195d0 2024-02-18T00:42:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f8347fff700 10 client.31758.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c01a2d0 osd=6 4 2024-02-18T00:42:58.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 6 14 2024-02-18T00:42:58.024 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:42:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 10 client.31758.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:42:58.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 15 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6812/53025171 conn(0x7f834c0180f0 0x7f834c018530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 15 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 7 15 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c017a70 osd=7 4 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 10 client.31758.objecter _send_command 15 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f834c018b70 con 0x7f834c0180f0 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6812/53025171 conn(0x7f834c0180f0 0x7f834c018530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f8347fff700 10 client.31758.objecter ms_handle_connect 0x7f834c0180f0 2024-02-18T00:42:58.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.023+0000 7f835ed9d700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6812/53025171 conn(0x7f834c0180f0 0x7f834c018530 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:58.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.024+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f834c018b70 con 0x7f834c0180f0 2024-02-18T00:42:58.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.024+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:42:58.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.024+0000 7f8347fff700 10 client.31758.objecter _finish_command 15 = system:0 2024-02-18T00:42:58.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.024+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c017a70 osd=7 4 2024-02-18T00:42:58.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.024+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 7 15 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 10 client.31758.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign -1 16 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 20 client.31758.objecter _get_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 15 client.31758.objecter _session_command_op_remove -1 16 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 20 client.31758.objecter get_session s=0x7f834c017a70 osd=7 3 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 15 client.31758.objecter _session_command_op_assign 7 16 2024-02-18T00:42:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c017a70 osd=7 4 2024-02-18T00:42:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 10 client.31758.objecter _send_command 16 2024-02-18T00:42:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.039+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f834c018b70 con 0x7f834c0180f0 2024-02-18T00:42:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f8347fff700 1 -- 172.21.15.59:0/1323421181 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f834c018b70 con 0x7f834c0180f0 2024-02-18T00:42:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f8347fff700 10 client.31758.objecter ms_dispatch 0x7f8360001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:42:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f8347fff700 10 client.31758.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f8347fff700 20 client.31758.objecter put_session s=0x7f834c017a70 osd=7 4 2024-02-18T00:42:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f8347fff700 15 client.31758.objecter _session_command_op_remove 7 16 2024-02-18T00:42:58.044 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:42:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 librados: watch_flush enter 2024-02-18T00:42:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 librados: watch_flush exit 2024-02-18T00:42:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.0 2024-02-18T00:42:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6801/3700238977 conn(0x7f834c004560 msgr2=0x7f834c0049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6801/3700238977 conn(0x7f834c004560 0x7f834c0049a0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0051d0 osd=0 1 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.1 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6805/26279110 conn(0x7f834c002c90 msgr2=0x7f834c0030d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6805/26279110 conn(0x7f834c002c90 0x7f834c0030d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c008610 osd=1 1 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.2 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6809/1153193637 conn(0x7f834c004160 msgr2=0x7f834c00d000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6809/1153193637 conn(0x7f834c004160 0x7f834c00d000 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c003ff0 osd=2 1 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.3 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6813/1805700964 conn(0x7f834c00d610 msgr2=0x7f834c00da70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6813/1805700964 conn(0x7f834c00d610 0x7f834c00da70 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f835f59e700 1 -- 172.21.15.59:0/1323421181 reap_dead start 2024-02-18T00:42:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c001bb0 osd=3 1 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.4 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6800/2044387056 conn(0x7f834c0120f0 msgr2=0x7f834c0144e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6800/2044387056 conn(0x7f834c0120f0 0x7f834c0144e0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f835f59e700 1 -- 172.21.15.59:0/1323421181 reap_dead start 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c00df60 osd=4 1 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.5 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6804/2170432263 conn(0x7f834c0150b0 msgr2=0x7f834c0174c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6804/2170432263 conn(0x7f834c0150b0 0x7f834c0174c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c0149d0 osd=5 1 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.6 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6808/3356009390 conn(0x7f834c0195d0 msgr2=0x7f834c019a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6808/3356009390 conn(0x7f834c0195d0 0x7f834c019a50 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c01a2d0 osd=6 1 2024-02-18T00:42:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter close_session for osd.7 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6812/53025171 conn(0x7f834c0180f0 msgr2=0x7f834c018530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.137:6812/53025171 conn(0x7f834c0180f0 0x7f834c018530 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter put_session s=0x7f834c017a70 osd=7 1 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 20 client.31758.objecter shutdown clearing up homeless session... 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 10 client.31758.objecter successfully canceled tick 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8348062c90 msgr2=0x7f8348065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8348062c90 0x7f8348065150 secure :-1 s=READY pgs=3181 cs=0 l=1 rev1=1 rx=0x7f8354004fd0 tx=0x7f835400b040).stop 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83600f8420 msgr2=0x7f836006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.042+0000 7f836500e700 1 --2- 172.21.15.59:0/1323421181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83600f8420 0x7f836006b0d0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7f83500091f0 tx=0x7f8350009cb0).stop 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.043+0000 7f835f59e700 1 -- 172.21.15.59:0/1323421181 reap_dead start 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.043+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 shutdown_connections 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.043+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 >> 172.21.15.59:0/1323421181 conn(0x7f83600f3800 msgr2=0x7f83600f5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.043+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 shutdown_connections 2024-02-18T00:42:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.043+0000 7f836500e700 1 -- 172.21.15.59:0/1323421181 wait complete. 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:58.043+0000 7f836500e700 1 librados: shutdown 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:42:58.050 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:58.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:42:58.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:42:58.183 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:58.205 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:42:58.207 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:58.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:42:58.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:42:58.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.651422+0000 osd.5 (osd.5) 265 : cluster [DBG] 25.f scrub starts 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.655128+0000 osd.5 (osd.5) 266 : cluster [DBG] 25.f scrub ok 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.818469+0000 osd.6 (osd.6) 289 : cluster [DBG] 25.d scrub starts 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:56.820442+0000 osd.6 (osd.6) 290 : cluster [DBG] 25.d scrub ok 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:57.462742+0000 osd.0 (osd.0) 280 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:57.465833+0000 osd.0 (osd.0) 281 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:57.774570+0000 mon.a (mon.0) 3180 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:57.778920+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106"}]': finished 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:57.778989+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:57.897629+0000 mon.a (mon.0) 3183 : audit [DBG] from='client.? 172.21.15.59:0/1323421181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:58.337 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.651422+0000 osd.5 (osd.5) 265 : cluster [DBG] 25.f scrub starts 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.655128+0000 osd.5 (osd.5) 266 : cluster [DBG] 25.f scrub ok 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.818469+0000 osd.6 (osd.6) 289 : cluster [DBG] 25.d scrub starts 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:56.820442+0000 osd.6 (osd.6) 290 : cluster [DBG] 25.d scrub ok 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:57.462742+0000 osd.0 (osd.0) 280 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:57.465833+0000 osd.0 (osd.0) 281 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:57.774570+0000 mon.a (mon.0) 3180 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:57.778920+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106"}]': finished 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:57.778989+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-18T00:42:58.338 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:57.897629+0000 mon.a (mon.0) 3183 : audit [DBG] from='client.? 172.21.15.59:0/1323421181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:58.338 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:42:58.414 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:58.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:42:58.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.651422+0000 osd.5 (osd.5) 265 : cluster [DBG] 25.f scrub starts 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.655128+0000 osd.5 (osd.5) 266 : cluster [DBG] 25.f scrub ok 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.818469+0000 osd.6 (osd.6) 289 : cluster [DBG] 25.d scrub starts 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:56.820442+0000 osd.6 (osd.6) 290 : cluster [DBG] 25.d scrub ok 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:57.462742+0000 osd.0 (osd.0) 280 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:57.465833+0000 osd.0 (osd.0) 281 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:57.774570+0000 mon.a (mon.0) 3180 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:57.778920+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-106"}]': finished 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:57.778989+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-18T00:42:58.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:57.897629+0000 mon.a (mon.0) 3183 : audit [DBG] from='client.? 172.21.15.59:0/1323421181' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:42:58.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:42:58.546 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:58.566 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:42:58.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:58.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:42:58.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:42:58.700 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:58.719 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:42:58.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:58.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:42:58.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:42:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:42:59.225 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:57.979033+0000 mgr.y (mgr.24454) 569 : cluster [DBG] pgmap v969: 204 pgs: 3 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 196 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:42:59.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:42:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:58.767639+0000 mon.a (mon.0) 3184 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-18T00:42:59.226 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:57.979033+0000 mgr.y (mgr.24454) 569 : cluster [DBG] pgmap v969: 204 pgs: 3 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 196 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:42:59.226 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:42:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:58.767639+0000 mon.a (mon.0) 3184 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-18T00:42:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:57.979033+0000 mgr.y (mgr.24454) 569 : cluster [DBG] pgmap v969: 204 pgs: 3 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 196 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T00:42:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:42:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:58.767639+0000 mon.a (mon.0) 3184 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-18T00:42:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 -- 172.21.15.59:0/3199815296 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640f7280 msgr2=0x7fe364103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3199815296 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640f7280 0x7fe364103880 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 rx=0x7fe35c009c30 tx=0x7fe35c004c50).stop 2024-02-18T00:42:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 -- 172.21.15.59:0/3199815296 shutdown_connections 2024-02-18T00:42:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3199815296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3640fca10 0x7fe3640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:59.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3199815296 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640f7280 0x7fe364103880 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3199815296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3640fd370 0x7fe3640f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 -- 172.21.15.59:0/3199815296 >> 172.21.15.59:0/3199815296 conn(0x7fe3640f2660 msgr2=0x7fe3640f4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 -- 172.21.15.59:0/3199815296 shutdown_connections 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.907+0000 7fe36afd6700 1 -- 172.21.15.59:0/3199815296 wait complete. 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 librados: starting msgr at 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 librados: starting objecter 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 Processor -- start 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 -- start start 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 librados: setting wanted keys 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 librados: calling monclient init 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3640f7280 0x7fe3640f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640fca10 0x7fe3640f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:59.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3640fd370 0x7fe3640f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3641163d0 con 0x7fe3640fd370 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364116540 con 0x7fe3640f7280 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe36afd6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe364116700 con 0x7fe3640fca10 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.908+0000 7fe363fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640fca10 0x7fe3640f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe369573700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3640fd370 0x7fe3640f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640fca10 0x7fe3640f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38992/0 (socket says 172.21.15.59:38992) 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 -- 172.21.15.59:0/3976910112 learned_addr learned my addr 172.21.15.59:0/3976910112 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe368d72700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3640f7280 0x7fe3640f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 -- 172.21.15.59:0/3976910112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3640f7280 msgr2=0x7fe3640f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3640f7280 0x7fe3640f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:59.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 -- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3640fd370 msgr2=0x7fe3640f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3640fd370 0x7fe3640f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 -- 172.21.15.59:0/3976910112 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350005040 con 0x7fe3640fca10 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe369573700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3640fd370 0x7fe3640f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe363fff700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640fca10 0x7fe3640f8d00 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 rx=0x7fe350009400 tx=0x7fe350009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe361ffb700 10 client.?.objecter ms_handle_connect 0x7fe3640fca10 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe361ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe361ffb700 1 -- 172.21.15.59:0/3976910112 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350007a40 con 0x7fe3640fca10 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe361ffb700 1 -- 172.21.15.59:0/3976910112 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe35000f040 con 0x7fe3640fca10 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0097c0 con 0x7fe3640fca10 2024-02-18T00:42:59.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe361ffb700 1 -- 172.21.15.59:0/3976910112 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350014a70 con 0x7fe3640fca10 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe36afd6700 10 client.41570.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe364116dc0 con 0x7fe3640fca10 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.909+0000 7fe36afd6700 1 librados: init done 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.910+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe364047540 con 0x7fe3640fca10 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 1 -- 172.21.15.59:0/3976910112 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe350010530 con 0x7fe3640fca10 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 1 --2- 172.21.15.59:0/3976910112 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe354062c90 0x7fe354065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 1 -- 172.21.15.59:0/3976910112 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(634..634 src has 418..634) v4 ==== 8647+0+0 (secure 0 0 0) 0x7fe35007cf40 con 0x7fe3640fca10 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 10 client.41570.objecter ms_dispatch 0x7fe364001550 osd_map(634..634 src has 418..634) v4 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 3 client.41570.objecter handle_osd_map got epochs [634,634] > 0 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 3 client.41570.objecter handle_osd_map decoding full epoch 634 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe368d72700 1 --2- 172.21.15.59:0/3976910112 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe354062c90 0x7fe354065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:42:59.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 20 client.41570.objecter dump_active .. 0 homeless 2024-02-18T00:42:59.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe361ffb700 10 client.41570.objecter ms_handle_connect 0x7fe354062c90 2024-02-18T00:42:59.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.911+0000 7fe368d72700 1 --2- 172.21.15.59:0/3976910112 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe354062c90 0x7fe354065150 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 rx=0x7fe358004ee0 tx=0x7fe35800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:42:59.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:42:59.915+0000 7fe361ffb700 1 -- 172.21.15.59:0/3976910112 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe350049f60 con 0x7fe3640fca10 2024-02-18T00:42:59.941 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:42:59.944 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:42:59.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:00.045 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:00.078 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:00.099 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:00.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.100+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fe36405fa60 con 0x7fe3640fca10 2024-02-18T00:43:00.102 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:00.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:00.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:00.234 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:00.432 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:00.433 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:00.437 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:00.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:00.472 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:00.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:58.420030+0000 osd.0 (osd.0) 282 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:58.421048+0000 osd.0 (osd.0) 283 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:58.720705+0000 osd.2 (osd.2) 255 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:58.721584+0000 osd.2 (osd.2) 256 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:58.838526+0000 osd.6 (osd.6) 291 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:58.841296+0000 osd.6 (osd.6) 292 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.352312+0000 osd.4 (osd.4) 357 : cluster [DBG] 2.f scrub starts 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.355781+0000 osd.4 (osd.4) 358 : cluster [DBG] 2.f scrub ok 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.466342+0000 osd.0 (osd.0) 284 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.467706+0000 osd.0 (osd.0) 285 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.672919+0000 osd.2 (osd.2) 257 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.673837+0000 osd.2 (osd.2) 258 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.770039+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-18T00:43:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:42:59.771697+0000 mon.a (mon.0) 3186 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.866498+0000 osd.6 (osd.6) 293 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.867448+0000 osd.6 (osd.6) 294 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:00.102030+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.59:0/3976910112' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:00.102363+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:58.420030+0000 osd.0 (osd.0) 282 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:58.421048+0000 osd.0 (osd.0) 283 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:58.720705+0000 osd.2 (osd.2) 255 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:58.721584+0000 osd.2 (osd.2) 256 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:58.838526+0000 osd.6 (osd.6) 291 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:58.841296+0000 osd.6 (osd.6) 292 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.352312+0000 osd.4 (osd.4) 357 : cluster [DBG] 2.f scrub starts 2024-02-18T00:43:00.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.355781+0000 osd.4 (osd.4) 358 : cluster [DBG] 2.f scrub ok 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.466342+0000 osd.0 (osd.0) 284 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.467706+0000 osd.0 (osd.0) 285 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.672919+0000 osd.2 (osd.2) 257 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.673837+0000 osd.2 (osd.2) 258 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.770039+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:42:59.771697+0000 mon.a (mon.0) 3186 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.866498+0000 osd.6 (osd.6) 293 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.867448+0000 osd.6 (osd.6) 294 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:00.102030+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.59:0/3976910112' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:00.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:00.102363+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:58.420030+0000 osd.0 (osd.0) 282 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:58.421048+0000 osd.0 (osd.0) 283 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:58.720705+0000 osd.2 (osd.2) 255 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:58.721584+0000 osd.2 (osd.2) 256 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:58.838526+0000 osd.6 (osd.6) 291 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:58.841296+0000 osd.6 (osd.6) 292 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.352312+0000 osd.4 (osd.4) 357 : cluster [DBG] 2.f scrub starts 2024-02-18T00:43:00.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.355781+0000 osd.4 (osd.4) 358 : cluster [DBG] 2.f scrub ok 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.466342+0000 osd.0 (osd.0) 284 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.467706+0000 osd.0 (osd.0) 285 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.672919+0000 osd.2 (osd.2) 257 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.673837+0000 osd.2 (osd.2) 258 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.770039+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:42:59.771697+0000 mon.a (mon.0) 3186 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.866498+0000 osd.6 (osd.6) 293 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.867448+0000 osd.6 (osd.6) 294 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:00.102030+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.59:0/3976910112' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:00.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:00.102363+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:00.538 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:00.764 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:00.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:00.768 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.784+0000 7fe361ffb700 1 -- 172.21.15.59:0/3976910112 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v635) v1 ==== 101+0+0 (secure 0 0 0) 0x7fe3500501c0 con 0x7fe3640fca10 2024-02-18T00:43:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.787+0000 7fe36afd6700 10 librados: watch_flush enter 2024-02-18T00:43:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.788+0000 7fe36afd6700 10 librados: watch_flush exit 2024-02-18T00:43:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.788+0000 7fe36afd6700 20 client.41570.objecter shutdown clearing up homeless session... 2024-02-18T00:43:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.788+0000 7fe36afd6700 10 client.41570.objecter successfully canceled tick 2024-02-18T00:43:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.788+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe354062c90 msgr2=0x7fe354065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:00.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.788+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3976910112 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe354062c90 0x7fe354065150 secure :-1 s=READY pgs=3182 cs=0 l=1 rev1=1 rx=0x7fe358004ee0 tx=0x7fe35800b040).stop 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.788+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640fca10 msgr2=0x7fe3640f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.788+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640fca10 0x7fe3640f8d00 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 rx=0x7fe350009400 tx=0x7fe350009cb0).stop 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.791+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 shutdown_connections 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.791+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3976910112 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe354062c90 0x7fe354065150 secure :-1 s=CLOSED pgs=3182 cs=0 l=1 rev1=1 rx=0x7fe358004ee0 tx=0x7fe35800b040).stop 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.791+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3640f7280 0x7fe3640f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.791+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3640fd370 0x7fe3640f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.791+0000 7fe36afd6700 1 --2- 172.21.15.59:0/3976910112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3640fca10 0x7fe3640f8d00 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:00.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.791+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 >> 172.21.15.59:0/3976910112 conn(0x7fe3640f2660 msgr2=0x7fe364060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:00.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.794+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 shutdown_connections 2024-02-18T00:43:00.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.795+0000 7fe36afd6700 1 -- 172.21.15.59:0/3976910112 wait complete. 2024-02-18T00:43:00.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:00.795+0000 7fe36afd6700 1 librados: shutdown 2024-02-18T00:43:00.798 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:43:00.869 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:00.901 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:00.922 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:00.924 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:00.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:01.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:01.055 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:01.074 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:01.078 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:01.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:01.179 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:01.209 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:43:01.211 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:01.233 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:01.236 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:01.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:01.337 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:42:59.979416+0000 mgr.y (mgr.24454) 570 : cluster [DBG] pgmap v972: 204 pgs: 32 unknown, 2 active+clean+snaptrim, 170 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:00.079840+0000 osd.3 (osd.3) 327 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:00.080978+0000 osd.3 (osd.3) 328 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:00.311065+0000 osd.4 (osd.4) 359 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:00.314085+0000 osd.4 (osd.4) 360 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:00.767174+0000 mon.a (mon.0) 3188 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:00.775928+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:00.776130+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:00.776194+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:42:59.979416+0000 mgr.y (mgr.24454) 570 : cluster [DBG] pgmap v972: 204 pgs: 32 unknown, 2 active+clean+snaptrim, 170 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:00.079840+0000 osd.3 (osd.3) 327 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:00.080978+0000 osd.3 (osd.3) 328 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:00.311065+0000 osd.4 (osd.4) 359 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:00.314085+0000 osd.4 (osd.4) 360 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:00.767174+0000 mon.a (mon.0) 3188 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:00.775928+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:00.776130+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:01.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:00.776194+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-18T00:43:01.368 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:01.389 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:01.392 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:01.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:42:59.979416+0000 mgr.y (mgr.24454) 570 : cluster [DBG] pgmap v972: 204 pgs: 32 unknown, 2 active+clean+snaptrim, 170 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:00.079840+0000 osd.3 (osd.3) 327 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:00.080978+0000 osd.3 (osd.3) 328 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:00.311065+0000 osd.4 (osd.4) 359 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:00.314085+0000 osd.4 (osd.4) 360 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:00.767174+0000 mon.a (mon.0) 3188 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:00.775928+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:01.471 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:00.776130+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:01.472 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:00.776194+0000 mon.a (mon.0) 3191 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-18T00:43:01.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:01.523 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:01.721 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:01.724 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:01.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:01.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:43:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:43:01.826 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:01.858 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:01.879 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:01.882 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:01.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:01.983 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:02.015 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:02.037 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:02.037 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:02.037 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:02.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:02.167 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:02.464 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:43:02.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.097257+0000 osd.3 (osd.3) 329 : cluster [DBG] 6.b scrub starts 2024-02-18T00:43:02.490 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.098561+0000 osd.3 (osd.3) 330 : cluster [DBG] 6.b scrub ok 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.281984+0000 osd.4 (osd.4) 361 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.284136+0000 osd.4 (osd.4) 362 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.778419+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:01.782051+0000 mon.a (mon.0) 3193 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:02] "GET /metrics HTTP/1.1" 200 191270 "" "Prometheus/2.33.4" 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.097257+0000 osd.3 (osd.3) 329 : cluster [DBG] 6.b scrub starts 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.098561+0000 osd.3 (osd.3) 330 : cluster [DBG] 6.b scrub ok 2024-02-18T00:43:02.491 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.281984+0000 osd.4 (osd.4) 361 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:43:02.492 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.284136+0000 osd.4 (osd.4) 362 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:43:02.492 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.778419+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-18T00:43:02.492 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:01.782051+0000 mon.a (mon.0) 3193 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:02.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.097257+0000 osd.3 (osd.3) 329 : cluster [DBG] 6.b scrub starts 2024-02-18T00:43:02.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.098561+0000 osd.3 (osd.3) 330 : cluster [DBG] 6.b scrub ok 2024-02-18T00:43:02.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.281984+0000 osd.4 (osd.4) 361 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:43:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.284136+0000 osd.4 (osd.4) 362 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:43:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.778419+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-18T00:43:02.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:01.782051+0000 mon.a (mon.0) 3193 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:02.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:02.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:03.022 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:03.054 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:03.075 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:03.077 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:03.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:03.178 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:03.209 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:03.229 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:03.232 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:03.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:03.333 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:01.470055+0000 mgr.y (mgr.24454) 571 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.679399+0000 osd.5 (osd.5) 267 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.682876+0000 osd.5 (osd.5) 268 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:01.979792+0000 mgr.y (mgr.24454) 572 : cluster [DBG] pgmap v975: 204 pgs: 4 creating+activating, 200 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.132044+0000 osd.3 (osd.3) 331 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.133847+0000 osd.3 (osd.3) 332 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.515931+0000 osd.0 (osd.0) 286 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.517830+0000 osd.0 (osd.0) 287 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.695025+0000 osd.2 (osd.2) 259 : cluster [DBG] 5.a scrub starts 2024-02-18T00:43:03.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.696396+0000 osd.2 (osd.2) 260 : cluster [DBG] 5.a scrub ok 2024-02-18T00:43:03.362 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:02.786798+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:03.362 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.786880+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-18T00:43:03.362 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:02.789267+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-108"}]: dispatch 2024-02-18T00:43:03.363 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:03.382 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:03.385 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:03.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:03.486 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:03.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:01.470055+0000 mgr.y (mgr.24454) 571 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:03.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.679399+0000 osd.5 (osd.5) 267 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.682876+0000 osd.5 (osd.5) 268 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:01.979792+0000 mgr.y (mgr.24454) 572 : cluster [DBG] pgmap v975: 204 pgs: 4 creating+activating, 200 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.132044+0000 osd.3 (osd.3) 331 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.133847+0000 osd.3 (osd.3) 332 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.515931+0000 osd.0 (osd.0) 286 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.517830+0000 osd.0 (osd.0) 287 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.695025+0000 osd.2 (osd.2) 259 : cluster [DBG] 5.a scrub starts 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.696396+0000 osd.2 (osd.2) 260 : cluster [DBG] 5.a scrub ok 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:02.786798+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.786880+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-18T00:43:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:02.789267+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-108"}]: dispatch 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:01.470055+0000 mgr.y (mgr.24454) 571 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.679399+0000 osd.5 (osd.5) 267 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.682876+0000 osd.5 (osd.5) 268 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:01.979792+0000 mgr.y (mgr.24454) 572 : cluster [DBG] pgmap v975: 204 pgs: 4 creating+activating, 200 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.132044+0000 osd.3 (osd.3) 331 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.133847+0000 osd.3 (osd.3) 332 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.515931+0000 osd.0 (osd.0) 286 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.517830+0000 osd.0 (osd.0) 287 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.695025+0000 osd.2 (osd.2) 259 : cluster [DBG] 5.a scrub starts 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.696396+0000 osd.2 (osd.2) 260 : cluster [DBG] 5.a scrub ok 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:02.786798+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.786880+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-18T00:43:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:02.789267+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-108"}]: dispatch 2024-02-18T00:43:03.519 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:03.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:43:03.768 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:03.773 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:03.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:03.874 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:03.902 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:04.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.013+0000 7efdd44fa700 1 -- 172.21.15.59:0/4173282194 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0f67a0 msgr2=0x7efdcc104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.013+0000 7efdd44fa700 1 --2- 172.21.15.59:0/4173282194 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0f67a0 0x7efdcc104a50 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 rx=0x7efdc4009be0 tx=0x7efdc4004d10).stop 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- 172.21.15.59:0/4173282194 shutdown_connections 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 --2- 172.21.15.59:0/4173282194 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0f67a0 0x7efdcc104a50 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 --2- 172.21.15.59:0/4173282194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fdbe0 0x7efdcc0fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 --2- 172.21.15.59:0/4173282194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdcc0fe540 0x7efdcc0f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- 172.21.15.59:0/4173282194 >> 172.21.15.59:0/4173282194 conn(0x7efdcc06ffc0 msgr2=0x7efdcc0f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- 172.21.15.59:0/4173282194 shutdown_connections 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- 172.21.15.59:0/4173282194 wait complete. 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 librados: starting msgr at 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 librados: starting objecter 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 Processor -- start 2024-02-18T00:43:04.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- start start 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 librados: setting wanted keys 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 librados: calling monclient init 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdcc0f67a0 0x7efdcc0f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0fdbe0 0x7efdcc0f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fe540 0x7efdcc0f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc116460 con 0x7efdcc0fe540 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc1165d0 con 0x7efdcc0f67a0 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.014+0000 7efdd44fa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc116790 con 0x7efdcc0fdbe0 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd1a95700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0fdbe0 0x7efdcc0f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fe540 0x7efdcc0f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd1a95700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0fdbe0 0x7efdcc0f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39012/0 (socket says 172.21.15.59:39012) 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd1a95700 1 -- 172.21.15.59:0/3165574154 learned_addr learned my addr 172.21.15.59:0/3165574154 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:04.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fe540 0x7efdcc0f7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37408/0 (socket says 172.21.15.59:37408) 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2296700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdcc0f67a0 0x7efdcc0f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 -- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0fdbe0 msgr2=0x7efdcc0f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0fdbe0 0x7efdcc0f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 -- 172.21.15.59:0/3165574154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdcc0f67a0 msgr2=0x7efdcc0f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdcc0f67a0 0x7efdcc0f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 -- 172.21.15.59:0/3165574154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc8005040 con 0x7efdcc0fe540 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd1a95700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0fdbe0 0x7efdcc0f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd2a97700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fe540 0x7efdcc0f7d70 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7efdc400b6d0 tx=0x7efdc4004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdc37fe700 10 client.?.objecter ms_handle_connect 0x7efdcc0fe540 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdc37fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdc37fe700 1 -- 172.21.15.59:0/3165574154 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400e040 con 0x7efdcc0fe540 2024-02-18T00:43:04.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc40097c0 con 0x7efdcc0fe540 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdc37fe700 1 -- 172.21.15.59:0/3165574154 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdc4018040 con 0x7efdcc0fe540 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdc37fe700 1 -- 172.21.15.59:0/3165574154 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc401c3f0 con 0x7efdcc0fe540 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd44fa700 10 client.31770.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdcc116c80 con 0x7efdcc0fe540 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.015+0000 7efdd44fa700 1 librados: init done 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.016+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdcc05f9b0 con 0x7efdcc0fe540 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 1 -- 172.21.15.59:0/3165574154 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efdc4013520 con 0x7efdcc0fe540 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 1 --2- 172.21.15.59:0/3165574154 >> v2:172.21.15.59:6800/3721891797 conn(0x7efdb8062c90 0x7efdb8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 1 -- 172.21.15.59:0/3165574154 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(638..638 src has 418..638) v4 ==== 8655+0+0 (secure 0 0 0) 0x7efdc407dfa0 con 0x7efdcc0fe540 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 10 client.31770.objecter ms_dispatch 0x7efdcc001550 osd_map(638..638 src has 418..638) v4 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 3 client.31770.objecter handle_osd_map got epochs [638,638] > 0 2024-02-18T00:43:04.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 3 client.31770.objecter handle_osd_map decoding full epoch 638 2024-02-18T00:43:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdd2296700 1 --2- 172.21.15.59:0/3165574154 >> v2:172.21.15.59:6800/3721891797 conn(0x7efdb8062c90 0x7efdb8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 20 client.31770.objecter dump_active .. 0 homeless 2024-02-18T00:43:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdc37fe700 10 client.31770.objecter ms_handle_connect 0x7efdb8062c90 2024-02-18T00:43:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.017+0000 7efdd2296700 1 --2- 172.21.15.59:0/3165574154 >> v2:172.21.15.59:6800/3721891797 conn(0x7efdb8062c90 0x7efdb8065150 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 rx=0x7efdc8009350 tx=0x7efdc800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:04.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.021+0000 7efdc37fe700 1 -- 172.21.15.59:0/3165574154 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efdc40512d0 con 0x7efdcc0fe540 2024-02-18T00:43:04.151 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:04.154 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:04.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:04.155 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:04.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.210+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"} v 0) v1 -- 0x7efdcc107100 con 0x7efdcc0fe540 2024-02-18T00:43:04.255 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:04.287 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:04.308 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:04.310 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:04.311 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:04.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:04.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.728470+0000 osd.5 (osd.5) 269 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:43:04.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.729340+0000 osd.5 (osd.5) 270 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:43:04.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.845241+0000 osd.2 (osd.2) 261 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:02.845572+0000 osd.2 (osd.2) 262 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.269096+0000 osd.1 (osd.1) 315 : cluster [DBG] 5.c scrub starts 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.272274+0000 osd.1 (osd.1) 316 : cluster [DBG] 5.c scrub ok 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.508034+0000 osd.0 (osd.0) 288 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.511879+0000 osd.0 (osd.0) 289 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.730928+0000 osd.2 (osd.2) 263 : cluster [DBG] 5.b scrub starts 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.731914+0000 osd.2 (osd.2) 264 : cluster [DBG] 5.b scrub ok 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:03.771448+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-108"}]': finished 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.771512+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:03.772965+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-108", "mode": "writeback"}]: dispatch 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.851164+0000 osd.6 (osd.6) 295 : cluster [DBG] 2.17 deep-scrub starts 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.852184+0000 osd.6 (osd.6) 296 : cluster [DBG] 2.17 deep-scrub ok 2024-02-18T00:43:04.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:04.211750+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? 172.21.15.59:0/3165574154' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.728470+0000 osd.5 (osd.5) 269 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.729340+0000 osd.5 (osd.5) 270 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.845241+0000 osd.2 (osd.2) 261 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:02.845572+0000 osd.2 (osd.2) 262 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.269096+0000 osd.1 (osd.1) 315 : cluster [DBG] 5.c scrub starts 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.272274+0000 osd.1 (osd.1) 316 : cluster [DBG] 5.c scrub ok 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.508034+0000 osd.0 (osd.0) 288 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.511879+0000 osd.0 (osd.0) 289 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.730928+0000 osd.2 (osd.2) 263 : cluster [DBG] 5.b scrub starts 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.731914+0000 osd.2 (osd.2) 264 : cluster [DBG] 5.b scrub ok 2024-02-18T00:43:04.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:03.771448+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-108"}]': finished 2024-02-18T00:43:04.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.771512+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-18T00:43:04.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:03.772965+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-108", "mode": "writeback"}]: dispatch 2024-02-18T00:43:04.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.851164+0000 osd.6 (osd.6) 295 : cluster [DBG] 2.17 deep-scrub starts 2024-02-18T00:43:04.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.852184+0000 osd.6 (osd.6) 296 : cluster [DBG] 2.17 deep-scrub ok 2024-02-18T00:43:04.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:04.211750+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? 172.21.15.59:0/3165574154' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:43:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.728470+0000 osd.5 (osd.5) 269 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:43:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.729340+0000 osd.5 (osd.5) 270 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:43:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.845241+0000 osd.2 (osd.2) 261 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:02.845572+0000 osd.2 (osd.2) 262 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.269096+0000 osd.1 (osd.1) 315 : cluster [DBG] 5.c scrub starts 2024-02-18T00:43:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.272274+0000 osd.1 (osd.1) 316 : cluster [DBG] 5.c scrub ok 2024-02-18T00:43:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.508034+0000 osd.0 (osd.0) 288 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.511879+0000 osd.0 (osd.0) 289 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.730928+0000 osd.2 (osd.2) 263 : cluster [DBG] 5.b scrub starts 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.731914+0000 osd.2 (osd.2) 264 : cluster [DBG] 5.b scrub ok 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:03.771448+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-108"}]': finished 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.771512+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:03.772965+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-108", "mode": "writeback"}]: dispatch 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.851164+0000 osd.6 (osd.6) 295 : cluster [DBG] 2.17 deep-scrub starts 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.852184+0000 osd.6 (osd.6) 296 : cluster [DBG] 2.17 deep-scrub ok 2024-02-18T00:43:04.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:04.211750+0000 mon.a (mon.0) 3200 : audit [INF] from='client.? 172.21.15.59:0/3165574154' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"}]: dispatch 2024-02-18T00:43:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.774+0000 7efdc37fe700 1 -- 172.21.15.59:0/3165574154 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"}]=0 set pool 4 pg_num to 42 v639) v1 ==== 140+0+0 (secure 0 0 0) 0x7efdc4021030 con 0x7efdcc0fe540 2024-02-18T00:43:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.777+0000 7efdd44fa700 10 librados: watch_flush enter 2024-02-18T00:43:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 10 librados: watch_flush exit 2024-02-18T00:43:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 20 client.31770.objecter shutdown clearing up homeless session... 2024-02-18T00:43:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 10 client.31770.objecter successfully canceled tick 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 >> v2:172.21.15.59:6800/3721891797 conn(0x7efdb8062c90 msgr2=0x7efdb8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 --2- 172.21.15.59:0/3165574154 >> v2:172.21.15.59:6800/3721891797 conn(0x7efdb8062c90 0x7efdb8065150 secure :-1 s=READY pgs=3183 cs=0 l=1 rev1=1 rx=0x7efdc8009350 tx=0x7efdc800b040).stop 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fe540 msgr2=0x7efdcc0f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fe540 0x7efdcc0f7d70 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7efdc400b6d0 tx=0x7efdc4004d10).stop 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 shutdown_connections 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 --2- 172.21.15.59:0/3165574154 >> v2:172.21.15.59:6800/3721891797 conn(0x7efdb8062c90 0x7efdb8065150 unknown :-1 s=CLOSED pgs=3183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdcc0f67a0 0x7efdcc0f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efdcc0fe540 0x7efdcc0f7d70 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 --2- 172.21.15.59:0/3165574154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efdcc0fdbe0 0x7efdcc0f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.778+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 >> 172.21.15.59:0/3165574154 conn(0x7efdcc06ffc0 msgr2=0x7efdcc06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.779+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 shutdown_connections 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.779+0000 7efdd44fa700 1 -- 172.21.15.59:0/3165574154 wait complete. 2024-02-18T00:43:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:04.779+0000 7efdd44fa700 1 librados: shutdown 2024-02-18T00:43:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 4 pg_num to 42 2024-02-18T00:43:05.168 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:05.171 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:05.172 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:05.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:05.302 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:05.450 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:05.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.702544+0000 osd.5 (osd.5) 271 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.706006+0000 osd.5 (osd.5) 272 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.808993+0000 osd.7 (osd.7) 185 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.809902+0000 osd.7 (osd.7) 186 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:03.980277+0000 mgr.y (mgr.24454) 573 : cluster [DBG] pgmap v978: 204 pgs: 4 creating+activating, 200 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.182993+0000 osd.3 (osd.3) 333 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.185539+0000 osd.3 (osd.3) 334 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.251166+0000 osd.1 (osd.1) 317 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.252530+0000 osd.1 (osd.1) 318 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.467839+0000 osd.0 (osd.0) 290 : cluster [DBG] 4.b scrub starts 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.471407+0000 osd.0 (osd.0) 291 : cluster [DBG] 4.b scrub ok 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.766756+0000 mon.a (mon.0) 3201 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:05.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.769606+0000 osd.2 (osd.2) 265 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:43:05.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.770871+0000 osd.2 (osd.2) 266 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:43:05.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:04.775869+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-108", "mode": "writeback"}]': finished 2024-02-18T00:43:05.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:04.775946+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? 172.21.15.59:0/3165574154' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"}]': finished 2024-02-18T00:43:05.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.775993+0000 mon.a (mon.0) 3204 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-18T00:43:05.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.885803+0000 osd.6 (osd.6) 297 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:43:05.454 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.887766+0000 osd.6 (osd.6) 298 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:43:05.454 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:05.455 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:05.455 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.702544+0000 osd.5 (osd.5) 271 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:43:05.455 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.706006+0000 osd.5 (osd.5) 272 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:43:05.455 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.808993+0000 osd.7 (osd.7) 185 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:43:05.455 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.809902+0000 osd.7 (osd.7) 186 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:03.980277+0000 mgr.y (mgr.24454) 573 : cluster [DBG] pgmap v978: 204 pgs: 4 creating+activating, 200 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.182993+0000 osd.3 (osd.3) 333 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.185539+0000 osd.3 (osd.3) 334 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.251166+0000 osd.1 (osd.1) 317 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.252530+0000 osd.1 (osd.1) 318 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.467839+0000 osd.0 (osd.0) 290 : cluster [DBG] 4.b scrub starts 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.471407+0000 osd.0 (osd.0) 291 : cluster [DBG] 4.b scrub ok 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.766756+0000 mon.a (mon.0) 3201 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.769606+0000 osd.2 (osd.2) 265 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.770871+0000 osd.2 (osd.2) 266 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:04.775869+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-108", "mode": "writeback"}]': finished 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:04.775946+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? 172.21.15.59:0/3165574154' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"}]': finished 2024-02-18T00:43:05.456 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.775993+0000 mon.a (mon.0) 3204 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-18T00:43:05.457 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.885803+0000 osd.6 (osd.6) 297 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:43:05.457 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.887766+0000 osd.6 (osd.6) 298 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.702544+0000 osd.5 (osd.5) 271 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.706006+0000 osd.5 (osd.5) 272 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.808993+0000 osd.7 (osd.7) 185 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.809902+0000 osd.7 (osd.7) 186 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:03.980277+0000 mgr.y (mgr.24454) 573 : cluster [DBG] pgmap v978: 204 pgs: 4 creating+activating, 200 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.182993+0000 osd.3 (osd.3) 333 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.185539+0000 osd.3 (osd.3) 334 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.251166+0000 osd.1 (osd.1) 317 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.252530+0000 osd.1 (osd.1) 318 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.467839+0000 osd.0 (osd.0) 290 : cluster [DBG] 4.b scrub starts 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.471407+0000 osd.0 (osd.0) 291 : cluster [DBG] 4.b scrub ok 2024-02-18T00:43:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.766756+0000 mon.a (mon.0) 3201 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.769606+0000 osd.2 (osd.2) 265 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:43:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.770871+0000 osd.2 (osd.2) 266 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:43:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:04.775869+0000 mon.a (mon.0) 3202 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-108", "mode": "writeback"}]': finished 2024-02-18T00:43:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:04.775946+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? 172.21.15.59:0/3165574154' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num", "val": "42"}]': finished 2024-02-18T00:43:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.775993+0000 mon.a (mon.0) 3204 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-18T00:43:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.885803+0000 osd.6 (osd.6) 297 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:43:05.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.887766+0000 osd.6 (osd.6) 298 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:43:05.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:05.585 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:05.934 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:05.937 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:05.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:06.038 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:06.070 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:06.193 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:06.195 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:06.196 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:06.196 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:43:06.196 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:43:06.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:06.327 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:06.400 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:06.404 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:06.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.697228+0000 osd.5 (osd.5) 273 : cluster [DBG] 5.7 deep-scrub starts 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:04.700143+0000 osd.5 (osd.5) 274 : cluster [DBG] 5.7 deep-scrub ok 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.210077+0000 osd.3 (osd.3) 335 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.211599+0000 osd.3 (osd.3) 336 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.466362+0000 osd.0 (osd.0) 292 : cluster [DBG] 25.1a deep-scrub starts 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.469630+0000 osd.0 (osd.0) 293 : cluster [DBG] 25.1a deep-scrub ok 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.776193+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:05.805735+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:06.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:05.981997+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.697228+0000 osd.5 (osd.5) 273 : cluster [DBG] 5.7 deep-scrub starts 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:04.700143+0000 osd.5 (osd.5) 274 : cluster [DBG] 5.7 deep-scrub ok 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.210077+0000 osd.3 (osd.3) 335 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.211599+0000 osd.3 (osd.3) 336 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.466362+0000 osd.0 (osd.0) 292 : cluster [DBG] 25.1a deep-scrub starts 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.469630+0000 osd.0 (osd.0) 293 : cluster [DBG] 25.1a deep-scrub ok 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.776193+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:05.805735+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:06.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:05.981997+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.697228+0000 osd.5 (osd.5) 273 : cluster [DBG] 5.7 deep-scrub starts 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:04.700143+0000 osd.5 (osd.5) 274 : cluster [DBG] 5.7 deep-scrub ok 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.210077+0000 osd.3 (osd.3) 335 : cluster [DBG] 5.0 deep-scrub starts 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.211599+0000 osd.3 (osd.3) 336 : cluster [DBG] 5.0 deep-scrub ok 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.466362+0000 osd.0 (osd.0) 292 : cluster [DBG] 25.1a deep-scrub starts 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.469630+0000 osd.0 (osd.0) 293 : cluster [DBG] 25.1a deep-scrub ok 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.776193+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:05.805735+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:05.981997+0000 mon.a (mon.0) 3207 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num_actual", "val": "42"}]: dispatch 2024-02-18T00:43:06.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:06.532 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.326452+0000 osd.1 (osd.1) 319 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.326792+0000 osd.1 (osd.1) 320 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:05.981132+0000 mgr.y (mgr.24454) 574 : cluster [DBG] pgmap v981: 204 pgs: 1 active+clean+snaptrim, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:06.779722+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:06.779806+0000 mon.a (mon.0) 3209 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:06.779873+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:06.783274+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108"}]: dispatch 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.326452+0000 osd.1 (osd.1) 319 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:07.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.326792+0000 osd.1 (osd.1) 320 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:05.981132+0000 mgr.y (mgr.24454) 574 : cluster [DBG] pgmap v981: 204 pgs: 1 active+clean+snaptrim, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:06.779722+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:06.779806+0000 mon.a (mon.0) 3209 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:06.779873+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:06.783274+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108"}]: dispatch 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.326452+0000 osd.1 (osd.1) 319 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.326792+0000 osd.1 (osd.1) 320 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:05.981132+0000 mgr.y (mgr.24454) 574 : cluster [DBG] pgmap v981: 204 pgs: 1 active+clean+snaptrim, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:06.779722+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:06.779806+0000 mon.a (mon.0) 3209 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pg_num_actual", "val": "42"}]': finished 2024-02-18T00:43:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:06.779873+0000 mon.a (mon.0) 3210 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-18T00:43:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:06.783274+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108"}]: dispatch 2024-02-18T00:43:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:07.793 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:07.793 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:07.894 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:07.946 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:07.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:08.047 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:08.078 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:08.098 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:08.101 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:08.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:08.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:08.250 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:08.395 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:43:08.395 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:43:08.395 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.271+0000 7f47bb9e9700 1 -- 172.21.15.59:0/1993448988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fd410 msgr2=0x7f47b40f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.271+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/1993448988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fd410 0x7f47b40f6c40 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 rx=0x7f47a4004870 tx=0x7f47a4009cb0).stop 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.271+0000 7f47bb9e9700 1 -- 172.21.15.59:0/1993448988 shutdown_connections 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.271+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/1993448988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b40fcab0 0x7f47b40fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.271+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/1993448988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b40f7320 0x7f47b4103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.271+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/1993448988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fd410 0x7f47b40f6c40 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.271+0000 7f47bb9e9700 1 -- 172.21.15.59:0/1993448988 >> 172.21.15.59:0/1993448988 conn(0x7f47b40f26e0 msgr2=0x7f47b40f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 -- 172.21.15.59:0/1993448988 shutdown_connections 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 -- 172.21.15.59:0/1993448988 wait complete. 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 librados: starting msgr at 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 librados: starting objecter 2024-02-18T00:43:08.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 Processor -- start 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 -- start start 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 librados: setting wanted keys 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.272+0000 7f47bb9e9700 1 librados: calling monclient init 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47bb9e9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b40f7320 0x7f47b41064c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47bb9e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fcab0 0x7f47b4104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47bb9e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b40fd410 0x7f47b4105090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47bb9e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4105750 con 0x7f47b40fcab0 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47bb9e9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b41058c0 con 0x7f47b40f7320 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47bb9e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b411ad50 con 0x7f47b40fd410 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47b8f84700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fcab0 0x7f47b4104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47b8f84700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fcab0 0x7f47b4104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37428/0 (socket says 172.21.15.59:37428) 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.273+0000 7f47b8f84700 1 -- 172.21.15.59:0/2842132328 learned_addr learned my addr 172.21.15.59:0/2842132328 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47b9f86700 1 --2- 172.21.15.59:0/2842132328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b40fd410 0x7f47b4105090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47b8f84700 1 -- 172.21.15.59:0/2842132328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b40fd410 msgr2=0x7f47b4105090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47b8f84700 1 --2- 172.21.15.59:0/2842132328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47b40fd410 0x7f47b4105090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47b8f84700 1 -- 172.21.15.59:0/2842132328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b40f7320 msgr2=0x7f47b41064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47b8f84700 1 --2- 172.21.15.59:0/2842132328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b40f7320 0x7f47b41064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47b8f84700 1 -- 172.21.15.59:0/2842132328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a4005040 con 0x7f47b40fcab0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47b8f84700 1 --2- 172.21.15.59:0/2842132328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fcab0 0x7f47b4104a60 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7f47a40043a0 tx=0x7f47a4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47aa7fc700 10 client.?.objecter ms_handle_connect 0x7f47b40fcab0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47aa7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a400e030 con 0x7f47b40fcab0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a4013e70 con 0x7f47b40fcab0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.274+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4018070 con 0x7f47b40fcab0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.275+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b411afd0 con 0x7f47b40fcab0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.275+0000 7f47bb9e9700 10 client.31776.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.275+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b411b380 con 0x7f47b40fcab0 2024-02-18T00:43:08.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.275+0000 7f47bb9e9700 1 librados: init done 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.276+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f47b405f970 con 0x7f47b40fcab0 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.277+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f47a40092e0 con 0x7f47b40fcab0 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47aa7fc700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6800/3721891797 conn(0x7f47a0062c90 0x7f47a0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(642..642 src has 418..642) v4 ==== 8675+0+0 (secure 0 0 0) 0x7f47a407ec90 con 0x7f47b40fcab0 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 osd_map(642..642 src has 418..642) v4 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47aa7fc700 3 client.31776.objecter handle_osd_map got epochs [642,642] > 0 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47aa7fc700 3 client.31776.objecter handle_osd_map decoding full epoch 642 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47aa7fc700 20 client.31776.objecter dump_active .. 0 homeless 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v642) v1 ==== 54+0+16 (secure 0 0 0) 0x7f47a407f170 con 0x7f47b40fcab0 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 1 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6801/3700238977 conn(0x7f47b40598f0 0x7f47b4059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 1 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 0 1 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b405ee80 osd=0 4 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 10 client.31776.objecter _send_command 1 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.278+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b4107010 con 0x7f47b40598f0 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.279+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6800/3721891797 conn(0x7f47a0062c90 0x7f47a0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.279+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6800/3721891797 conn(0x7f47a0062c90 0x7f47a0065150 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 rx=0x7f47b0004c60 tx=0x7f47b000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.279+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47a0062c90 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.279+0000 7f47b9f86700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6801/3700238977 conn(0x7f47b40598f0 0x7f47b4059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.280+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b40598f0 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.280+0000 7f47b9f86700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6801/3700238977 conn(0x7f47b40598f0 0x7f47b4059cd0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.280+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b4107010 con 0x7f47b40598f0 2024-02-18T00:43:08.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.280+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.280+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 1 = system:0 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.280+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b405ee80 osd=0 4 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.280+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 0 1 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 2 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 2 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b405ee80 osd=0 3 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 0 2 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b405ee80 osd=0 4 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 10 client.31776.objecter _send_command 2 2024-02-18T00:43:08.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.300+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b40273c0 con 0x7f47b40598f0 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.302+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b40273c0 con 0x7f47b40598f0 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b405ee80 osd=0 4 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 0 2 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 3 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6805/26279110 conn(0x7f47b40fdfc0 0x7f47b40fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 3 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 1 3 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4028710 osd=1 4 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 10 client.31776.objecter _send_command 3 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.303+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b4121db0 con 0x7f47b40fdfc0 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.304+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6805/26279110 conn(0x7f47b40fdfc0 0x7f47b40fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.307+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6805/26279110 conn(0x7f47b40fdfc0 0x7f47b40fe3c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.307+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b40fdfc0 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.308+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b4121db0 con 0x7f47b40fdfc0 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.308+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.308+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 3 = system:0 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.308+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4028710 osd=1 4 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.308+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 1 3 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 4 2024-02-18T00:43:08.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 4 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4028710 osd=1 3 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 1 4 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4028710 osd=1 4 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 10 client.31776.objecter _send_command 4 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.329+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b40fda30 con 0x7f47b40fdfc0 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.334+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b40fda30 con 0x7f47b40fdfc0 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.334+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.334+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.334+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4028710 osd=1 4 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.334+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 1 4 2024-02-18T00:43:08.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 5 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6809/1153193637 conn(0x7f47b41207d0 0x7f47b4120c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 5 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 2 5 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4027d60 osd=2 4 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 10 client.31776.objecter _send_command 5 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b40283e0 con 0x7f47b41207d0 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47b8f84700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6809/1153193637 conn(0x7f47b41207d0 0x7f47b4120c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47b8f84700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6809/1153193637 conn(0x7f47b41207d0 0x7f47b4120c30 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.335+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b41207d0 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.336+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b40283e0 con 0x7f47b41207d0 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.336+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.336+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 5 = system:0 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.336+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4027d60 osd=2 4 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.336+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 2 5 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 6 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 6 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4027d60 osd=2 3 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 2 6 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4027d60 osd=2 4 2024-02-18T00:43:08.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 10 client.31776.objecter _send_command 6 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.366+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b41218f0 con 0x7f47b41207d0 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.372+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b41218f0 con 0x7f47b41207d0 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.372+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.372+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.372+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4027d60 osd=2 4 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.372+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 2 6 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 7 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6813/1805700964 conn(0x7f47b4027940 0x7f47b4126450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 7 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 3 7 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b40277d0 osd=3 4 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 10 client.31776.objecter _send_command 7 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.373+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b41290a0 con 0x7f47b4027940 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.375+0000 7f47b9f86700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6813/1805700964 conn(0x7f47b4027940 0x7f47b4126450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.375+0000 7f47b9f86700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6813/1805700964 conn(0x7f47b4027940 0x7f47b4126450 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.375+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b4027940 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.376+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b41290a0 con 0x7f47b4027940 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.376+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.376+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 7 = system:0 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.376+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b40277d0 osd=3 4 2024-02-18T00:43:08.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.376+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 3 7 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 8 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 8 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b40277d0 osd=3 3 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 3 8 2024-02-18T00:43:08.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b40277d0 osd=3 4 2024-02-18T00:43:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 10 client.31776.objecter _send_command 8 2024-02-18T00:43:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.403+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b4121360 con 0x7f47b4027940 2024-02-18T00:43:08.410 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:08.412 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:43:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.409+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b4121360 con 0x7f47b4027940 2024-02-18T00:43:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.409+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.409+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.409+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b40277d0 osd=3 4 2024-02-18T00:43:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.409+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 3 8 2024-02-18T00:43:08.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 9 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6800/2044387056 conn(0x7f47b41214d0 0x7f47b4127ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 9 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 4 9 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4121360 osd=4 4 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 10 client.31776.objecter _send_command 9 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b4120330 con 0x7f47b41214d0 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6800/2044387056 conn(0x7f47b41214d0 0x7f47b4127ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6800/2044387056 conn(0x7f47b41214d0 0x7f47b4127ac0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.410+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b41214d0 2024-02-18T00:43:08.414 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:08.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:08.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.411+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b4120330 con 0x7f47b41214d0 2024-02-18T00:43:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.411+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:43:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.411+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 9 = system:0 2024-02-18T00:43:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.411+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4121360 osd=4 4 2024-02-18T00:43:08.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.411+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 4 9 2024-02-18T00:43:08.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 10 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 10 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4121360 osd=4 3 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 4 10 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4121360 osd=4 4 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 10 client.31776.objecter _send_command 10 2024-02-18T00:43:08.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.437+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b4120330 con 0x7f47b41214d0 2024-02-18T00:43:08.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b4120330 con 0x7f47b41214d0 2024-02-18T00:43:08.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4121360 osd=4 4 2024-02-18T00:43:08.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 4 10 2024-02-18T00:43:08.441 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:43:08.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 11 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6804/2170432263 conn(0x7f47b41284c0 0x7f47b4128920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 11 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 5 11 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4120330 osd=5 4 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 10 client.31776.objecter _send_command 11 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.439+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b4128e10 con 0x7f47b41284c0 2024-02-18T00:43:08.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.440+0000 7f47b8f84700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6804/2170432263 conn(0x7f47b41284c0 0x7f47b4128920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.440+0000 7f47b8f84700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6804/2170432263 conn(0x7f47b41284c0 0x7f47b4128920 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.440+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b41284c0 2024-02-18T00:43:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.441+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b4128e10 con 0x7f47b41284c0 2024-02-18T00:43:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.441+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:43:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.441+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 11 = system:0 2024-02-18T00:43:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.441+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4120330 osd=5 4 2024-02-18T00:43:08.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.441+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 5 11 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 12 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 12 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4120330 osd=5 3 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 5 12 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4120330 osd=5 4 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 10 client.31776.objecter _send_command 12 2024-02-18T00:43:08.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.467+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b4128e10 con 0x7f47b41284c0 2024-02-18T00:43:08.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b4128e10 con 0x7f47b41284c0 2024-02-18T00:43:08.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4120330 osd=5 4 2024-02-18T00:43:08.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 5 12 2024-02-18T00:43:08.470 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:43:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 13 2024-02-18T00:43:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6808/3356009390 conn(0x7f47b412d740 0x7f47b412fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 13 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 6 13 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4128e10 osd=6 4 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 10 client.31776.objecter _send_command 13 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b4132900 con 0x7f47b412d740 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47b9f86700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6808/3356009390 conn(0x7f47b412d740 0x7f47b412fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b412d740 2024-02-18T00:43:08.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.469+0000 7f47b9f86700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6808/3356009390 conn(0x7f47b412d740 0x7f47b412fb30 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.470+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b4132900 con 0x7f47b412d740 2024-02-18T00:43:08.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.470+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:43:08.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.470+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 13 = system:0 2024-02-18T00:43:08.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.470+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4128e10 osd=6 4 2024-02-18T00:43:08.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.470+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 6 13 2024-02-18T00:43:08.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 14 2024-02-18T00:43:08.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 14 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4128e10 osd=6 3 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 6 14 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4128e10 osd=6 4 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 10 client.31776.objecter _send_command 14 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.496+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b4132900 con 0x7f47b412d740 2024-02-18T00:43:08.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.498+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b4132900 con 0x7f47b412d740 2024-02-18T00:43:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.498+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.498+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.498+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4128e10 osd=6 4 2024-02-18T00:43:08.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.498+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 6 14 2024-02-18T00:43:08.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:07.777487+0000 mon.a (mon.0) 3212 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:08.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:07.784684+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108"}]': finished 2024-02-18T00:43:08.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:07.784881+0000 mon.a (mon.0) 3214 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-18T00:43:08.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:07.983703+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-18T00:43:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:07.777487+0000 mon.a (mon.0) 3212 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:07.784684+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108"}]': finished 2024-02-18T00:43:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:07.784881+0000 mon.a (mon.0) 3214 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-18T00:43:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:07.983703+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-18T00:43:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:07.777487+0000 mon.a (mon.0) 3212 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:07.784684+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-108"}]': finished 2024-02-18T00:43:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:07.784881+0000 mon.a (mon.0) 3214 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-18T00:43:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:07.983703+0000 mon.a (mon.0) 3215 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num_actual", "val": "42"}]: dispatch 2024-02-18T00:43:08.506 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:43:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 15 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6812/53025171 conn(0x7f47b4131d40 0x7f47b41321a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 15 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 7 15 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4132900 osd=7 4 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 10 client.31776.objecter _send_command 15 2024-02-18T00:43:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47b4135cb0 con 0x7f47b4131d40 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.505+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6812/53025171 conn(0x7f47b4131d40 0x7f47b41321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.506+0000 7f47b9785700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6812/53025171 conn(0x7f47b4131d40 0x7f47b41321a0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.506+0000 7f47aa7fc700 10 client.31776.objecter ms_handle_connect 0x7f47b4131d40 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.507+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f47b4135cb0 con 0x7f47b4131d40 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.507+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.507+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 15 = system:0 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.507+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4132900 osd=7 4 2024-02-18T00:43:08.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.507+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 7 15 2024-02-18T00:43:08.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:08.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 10 client.31776.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign -1 16 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 20 client.31776.objecter _get_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_remove -1 16 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 20 client.31776.objecter get_session s=0x7f47b4132900 osd=7 3 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 15 client.31776.objecter _session_command_op_assign 7 16 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4132900 osd=7 4 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 10 client.31776.objecter _send_command 16 2024-02-18T00:43:08.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.532+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f47b4134b30 con 0x7f47b4131d40 2024-02-18T00:43:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.534+0000 7f47aa7fc700 1 -- 172.21.15.59:0/2842132328 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f47b4134b30 con 0x7f47b4131d40 2024-02-18T00:43:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.534+0000 7f47aa7fc700 10 client.31776.objecter ms_dispatch 0x7f47b4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.534+0000 7f47aa7fc700 10 client.31776.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.534+0000 7f47aa7fc700 20 client.31776.objecter put_session s=0x7f47b4132900 osd=7 4 2024-02-18T00:43:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.534+0000 7f47aa7fc700 15 client.31776.objecter _session_command_op_remove 7 16 2024-02-18T00:43:08.536 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:43:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.534+0000 7f47bb9e9700 10 librados: watch_flush enter 2024-02-18T00:43:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 librados: watch_flush exit 2024-02-18T00:43:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.0 2024-02-18T00:43:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6801/3700238977 conn(0x7f47b40598f0 msgr2=0x7f47b4059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6801/3700238977 conn(0x7f47b40598f0 0x7f47b4059cd0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b405ee80 osd=0 1 2024-02-18T00:43:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.1 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6805/26279110 conn(0x7f47b40fdfc0 msgr2=0x7f47b40fe3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6805/26279110 conn(0x7f47b40fdfc0 0x7f47b40fe3c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4028710 osd=1 1 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.2 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6809/1153193637 conn(0x7f47b41207d0 msgr2=0x7f47b4120c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6809/1153193637 conn(0x7f47b41207d0 0x7f47b4120c30 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4027d60 osd=2 1 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.3 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6813/1805700964 conn(0x7f47b4027940 msgr2=0x7f47b4126450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6813/1805700964 conn(0x7f47b4027940 0x7f47b4126450 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b40277d0 osd=3 1 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.4 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6800/2044387056 conn(0x7f47b41214d0 msgr2=0x7f47b4127ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6800/2044387056 conn(0x7f47b41214d0 0x7f47b4127ac0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47b9f86700 1 -- 172.21.15.59:0/2842132328 reap_dead start 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4121360 osd=4 1 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.5 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6804/2170432263 conn(0x7f47b41284c0 msgr2=0x7f47b4128920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6804/2170432263 conn(0x7f47b41284c0 0x7f47b4128920 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4120330 osd=5 1 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.6 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6808/3356009390 conn(0x7f47b412d740 msgr2=0x7f47b412fb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6808/3356009390 conn(0x7f47b412d740 0x7f47b412fb30 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47b9f86700 1 -- 172.21.15.59:0/2842132328 reap_dead start 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47b9f86700 1 -- 172.21.15.59:0/2842132328 reap_dead start 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4128e10 osd=6 1 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter close_session for osd.7 2024-02-18T00:43:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6812/53025171 conn(0x7f47b4131d40 msgr2=0x7f47b41321a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6812/53025171 conn(0x7f47b4131d40 0x7f47b41321a0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter put_session s=0x7f47b4132900 osd=7 1 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 20 client.31776.objecter shutdown clearing up homeless session... 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 10 client.31776.objecter successfully canceled tick 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6800/3721891797 conn(0x7f47a0062c90 msgr2=0x7f47a0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6800/3721891797 conn(0x7f47a0062c90 0x7f47a0065150 secure :-1 s=READY pgs=3184 cs=0 l=1 rev1=1 rx=0x7f47b0004c60 tx=0x7f47b000b040).stop 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fcab0 msgr2=0x7f47b4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fcab0 0x7f47b4104a60 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7f47a40043a0 tx=0x7f47a4009cb0).stop 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 shutdown_connections 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6812/53025171 conn(0x7f47b4131d40 0x7f47b41321a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.137:6808/3356009390 conn(0x7f47b412d740 0x7f47b412fb30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47b40fcab0 0x7f47b4104a60 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 --2- 172.21.15.59:0/2842132328 >> v2:172.21.15.59:6800/3721891797 conn(0x7f47a0062c90 0x7f47a0065150 unknown :-1 s=CLOSED pgs=3184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.535+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 >> 172.21.15.59:0/2842132328 conn(0x7f47b40f26e0 msgr2=0x7f47b40f42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:08.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.536+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 shutdown_connections 2024-02-18T00:43:08.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.536+0000 7f47bb9e9700 1 -- 172.21.15.59:0/2842132328 wait complete. 2024-02-18T00:43:08.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:08.536+0000 7f47bb9e9700 1 librados: shutdown 2024-02-18T00:43:08.542 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.550 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:43:08.563 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:08.566 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:08.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:08.667 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:08.697 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:08.719 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:08.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:08.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:08.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:08.854 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-18T00:43:08.861 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:09.088 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:09.112 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:09.117 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:09.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:09.117 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:09.218 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:09.248 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:07.779230+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.16 deep-scrub starts 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:07.782274+0000 osd.5 (osd.5) 276 : cluster [DBG] 2.16 deep-scrub ok 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:07.981788+0000 mgr.y (mgr.24454) 575 : cluster [DBG] pgmap v984: 214 pgs: 10 unknown, 1 active+clean+snaptrim, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:08.278203+0000 mon.a (mon.0) 3216 : audit [DBG] from='client.? 172.21.15.59:0/2842132328' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:08.565810+0000 mon.a (mon.0) 3217 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:08.780251+0000 mon.a (mon.0) 3218 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.780332+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-18T00:43:09.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:07.779230+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.16 deep-scrub starts 2024-02-18T00:43:09.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:07.782274+0000 osd.5 (osd.5) 276 : cluster [DBG] 2.16 deep-scrub ok 2024-02-18T00:43:09.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:07.981788+0000 mgr.y (mgr.24454) 575 : cluster [DBG] pgmap v984: 214 pgs: 10 unknown, 1 active+clean+snaptrim, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:09.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:08.278203+0000 mon.a (mon.0) 3216 : audit [DBG] from='client.? 172.21.15.59:0/2842132328' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:09.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:08.565810+0000 mon.a (mon.0) 3217 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:43:09.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:08.780251+0000 mon.a (mon.0) 3218 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-18T00:43:09.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.780332+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-18T00:43:09.450 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:09.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:09.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:09.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:09.580 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:07.779230+0000 osd.5 (osd.5) 275 : cluster [DBG] 2.16 deep-scrub starts 2024-02-18T00:43:09.580 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:07.782274+0000 osd.5 (osd.5) 276 : cluster [DBG] 2.16 deep-scrub ok 2024-02-18T00:43:09.580 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:07.981788+0000 mgr.y (mgr.24454) 575 : cluster [DBG] pgmap v984: 214 pgs: 10 unknown, 1 active+clean+snaptrim, 203 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:09.581 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:08.278203+0000 mon.a (mon.0) 3216 : audit [DBG] from='client.? 172.21.15.59:0/2842132328' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:09.581 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:08.565810+0000 mon.a (mon.0) 3217 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:43:09.581 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:08.780251+0000 mon.a (mon.0) 3218 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num_actual", "val": "42"}]': finished 2024-02-18T00:43:09.581 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.780332+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-18T00:43:09.584 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:09.683 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:09.685 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:09.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:09.786 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:09.830 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:10.180 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:10.183 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:10.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:10.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:10.311 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.782274+0000 mon.a (mon.0) 3220 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.827610+0000 osd.5 (osd.5) 277 : cluster [DBG] 25.1d deep-scrub starts 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.831929+0000 osd.5 (osd.5) 278 : cluster [DBG] 25.1d deep-scrub ok 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.892071+0000 osd.2 (osd.2) 267 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.893256+0000 osd.2 (osd.2) 268 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.920218+0000 osd.7 (osd.7) 187 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:08.921348+0000 osd.7 (osd.7) 188 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.186371+0000 osd.1 (osd.1) 321 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.187267+0000 osd.1 (osd.1) 322 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.211047+0000 osd.3 (osd.3) 337 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.211952+0000 osd.3 (osd.3) 338 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.315643+0000 osd.4 (osd.4) 363 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:43:10.312 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.318795+0000 osd.4 (osd.4) 364 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:43:10.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.777660+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-18T00:43:10.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:09.782466+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:10.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:09.907196+0000 mon.a (mon.0) 3223 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:10.314 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:10.388 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:10.391 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:10.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:10.492 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:10.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.782274+0000 mon.a (mon.0) 3220 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:10.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.827610+0000 osd.5 (osd.5) 277 : cluster [DBG] 25.1d deep-scrub starts 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.831929+0000 osd.5 (osd.5) 278 : cluster [DBG] 25.1d deep-scrub ok 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.892071+0000 osd.2 (osd.2) 267 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.893256+0000 osd.2 (osd.2) 268 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.920218+0000 osd.7 (osd.7) 187 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:08.921348+0000 osd.7 (osd.7) 188 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.186371+0000 osd.1 (osd.1) 321 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.187267+0000 osd.1 (osd.1) 322 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.211047+0000 osd.3 (osd.3) 337 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.211952+0000 osd.3 (osd.3) 338 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.315643+0000 osd.4 (osd.4) 363 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.318795+0000 osd.4 (osd.4) 364 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:43:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.777660+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:09.782466+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:09.907196+0000 mon.a (mon.0) 3223 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.782274+0000 mon.a (mon.0) 3220 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.827610+0000 osd.5 (osd.5) 277 : cluster [DBG] 25.1d deep-scrub starts 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.831929+0000 osd.5 (osd.5) 278 : cluster [DBG] 25.1d deep-scrub ok 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.892071+0000 osd.2 (osd.2) 267 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.893256+0000 osd.2 (osd.2) 268 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.920218+0000 osd.7 (osd.7) 187 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:43:10.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:08.921348+0000 osd.7 (osd.7) 188 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.186371+0000 osd.1 (osd.1) 321 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.187267+0000 osd.1 (osd.1) 322 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.211047+0000 osd.3 (osd.3) 337 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.211952+0000 osd.3 (osd.3) 338 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.315643+0000 osd.4 (osd.4) 363 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.318795+0000 osd.4 (osd.4) 364 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.777660+0000 mon.a (mon.0) 3221 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:09.782466+0000 mon.a (mon.0) 3222 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:10.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:09.907196+0000 mon.a (mon.0) 3223 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:10.518 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:10.667 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:10.669 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:10.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:10.771 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:10.812 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:10.916 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:10.919 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:10.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:10.924 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:43:10.924 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:43:10.924 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:43:10.925 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:43:11.020 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:11.049 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:11.198 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:11.201 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:11.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:11.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:43:11.303 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:11.331 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:11.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:09.982269+0000 mgr.y (mgr.24454) 576 : cluster [DBG] pgmap v987: 214 pgs: 42 unknown, 1 active+clean+snaptrim, 171 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:10.274250+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:10.275062+0000 mon.a (mon.0) 3225 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:10.279649+0000 mon.a (mon.0) 3226 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:10.776704+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:10.776816+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:10.797010+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:09.982269+0000 mgr.y (mgr.24454) 576 : cluster [DBG] pgmap v987: 214 pgs: 42 unknown, 1 active+clean+snaptrim, 171 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:10.274250+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:43:11.500 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:10.275062+0000 mon.a (mon.0) 3225 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:43:11.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:10.279649+0000 mon.a (mon.0) 3226 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:11.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:10.776704+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:11.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:10.776816+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-18T00:43:11.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:10.797010+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:11.545 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:43:11.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:09.982269+0000 mgr.y (mgr.24454) 576 : cluster [DBG] pgmap v987: 214 pgs: 42 unknown, 1 active+clean+snaptrim, 171 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:11.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:10.274250+0000 mon.a (mon.0) 3224 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:43:11.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:10.275062+0000 mon.a (mon.0) 3225 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:43:11.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:10.279649+0000 mon.a (mon.0) 3226 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:11.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:10.776704+0000 mon.a (mon.0) 3227 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:11.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:10.776816+0000 mon.a (mon.0) 3228 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-18T00:43:11.546 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:10.797010+0000 mon.a (mon.0) 3229 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:11.563 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-18T00:43:11.579 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:11.581 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:11.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:11.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:11.712 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:11.741 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:11.744 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:11.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:11.818 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:43:11.818 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:43:11.819 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-02-18T00:43:11.819 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-18T00:43:11.844 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:11.845 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 -- 172.21.15.59:0/2326741867 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 msgr2=0x7f9d700fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/2326741867 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d700fcf90 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 rx=0x7f9d60004870 tx=0x7f9d60009cb0).stop 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 -- 172.21.15.59:0/2326741867 shutdown_connections 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/2326741867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d700f7400 0x7f9d70103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/2326741867 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d700fcf90 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/2326741867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d700fd510 0x7f9d700f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 -- 172.21.15.59:0/2326741867 >> 172.21.15.59:0/2326741867 conn(0x7f9d700f2820 msgr2=0x7f9d700f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 -- 172.21.15.59:0/2326741867 shutdown_connections 2024-02-18T00:43:11.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.845+0000 7f9d74ac4700 1 -- 172.21.15.59:0/2326741867 wait complete. 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 librados: starting msgr at 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 librados: starting objecter 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 Processor -- start 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 -- start start 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 librados: setting wanted keys 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 librados: calling monclient init 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d700f7400 0x7f9d7010d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d7010d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d700fd510 0x7f9d701072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d701077a0 con 0x7f9d700fd510 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70107910 con 0x7f9d700f7400 2024-02-18T00:43:11.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.846+0000 7f9d74ac4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70107ad0 con 0x7f9d700fcbb0 2024-02-18T00:43:11.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d7010d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:11.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d7010d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43950/0 (socket says 172.21.15.59:43950) 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 -- 172.21.15.59:0/3795783875 learned_addr learned my addr 172.21.15.59:0/3795783875 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 -- 172.21.15.59:0/3795783875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d700f7400 msgr2=0x7f9d7010d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 --2- 172.21.15.59:0/3795783875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d700f7400 0x7f9d7010d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 -- 172.21.15.59:0/3795783875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d700fd510 msgr2=0x7f9d701072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 --2- 172.21.15.59:0/3795783875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d700fd510 0x7f9d701072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 -- 172.21.15.59:0/3795783875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d64005040 con 0x7f9d700fcbb0 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d6dd9b700 1 --2- 172.21.15.59:0/3795783875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d7010d600 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 rx=0x7f9d640048f0 tx=0x7f9d64009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d5f7fe700 10 client.?.objecter ms_handle_connect 0x7f9d700fcbb0 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d5f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d5f7fe700 1 -- 172.21.15.59:0/3795783875 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64007b00 con 0x7f9d700fcbb0 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d60005040 con 0x7f9d700fcbb0 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d5f7fe700 1 -- 172.21.15.59:0/3795783875 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6400f040 con 0x7f9d700fcbb0 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d5f7fe700 1 -- 172.21.15.59:0/3795783875 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64014aa0 con 0x7f9d700fcbb0 2024-02-18T00:43:11.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d74ac4700 10 client.41588.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:11.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d70113f40 con 0x7f9d700fcbb0 2024-02-18T00:43:11.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.847+0000 7f9d74ac4700 1 librados: init done 2024-02-18T00:43:11.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.848+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7005fa60 con 0x7f9d700fcbb0 2024-02-18T00:43:11.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.849+0000 7f9d5f7fe700 1 -- 172.21.15.59:0/3795783875 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9d6400e660 con 0x7f9d700fcbb0 2024-02-18T00:43:11.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d5f7fe700 1 --2- 172.21.15.59:0/3795783875 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d58062c90 0x7f9d58065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:11.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d5f7fe700 1 -- 172.21.15.59:0/3795783875 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(646..646 src has 418..646) v4 ==== 8683+0+0 (secure 0 0 0) 0x7f9d6407d130 con 0x7f9d700fcbb0 2024-02-18T00:43:11.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d5f7fe700 10 client.41588.objecter ms_dispatch 0x7f9d70001550 osd_map(646..646 src has 418..646) v4 2024-02-18T00:43:11.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d5f7fe700 3 client.41588.objecter handle_osd_map got epochs [646,646] > 0 2024-02-18T00:43:11.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d5f7fe700 3 client.41588.objecter handle_osd_map decoding full epoch 646 2024-02-18T00:43:11.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d6e59c700 1 --2- 172.21.15.59:0/3795783875 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d58062c90 0x7f9d58065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:11.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d5f7fe700 20 client.41588.objecter dump_active .. 0 homeless 2024-02-18T00:43:11.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d6e59c700 1 --2- 172.21.15.59:0/3795783875 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d58062c90 0x7f9d58065150 secure :-1 s=READY pgs=3185 cs=0 l=1 rev1=1 rx=0x7f9d60009da0 tx=0x7f9d6000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:11.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.850+0000 7f9d5f7fe700 10 client.41588.objecter ms_handle_connect 0x7f9d58062c90 2024-02-18T00:43:11.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:11.858+0000 7f9d5f7fe700 1 -- 172.21.15.59:0/3795783875 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9d64049f60 con 0x7f9d700fcbb0 2024-02-18T00:43:11.875 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:11.896 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:11.899 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:11.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:12.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:12.031 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:12.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.053+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f9d701060e0 con 0x7f9d700fcbb0 2024-02-18T00:43:12.055 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:12.060 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:12.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:12.107 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:12.161 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:12.162 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:43:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:12.264 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:12.264 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:12.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:12.299 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:10.795500+0000 osd.5 (osd.5) 279 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:10.797782+0000 osd.5 (osd.5) 280 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:10.874692+0000 osd.7 (osd.7) 189 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:10.875474+0000 osd.7 (osd.7) 190 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:11.624996+0000 osd.0 (osd.0) 294 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:11.626022+0000 osd.0 (osd.0) 295 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:11.779346+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:11.779414+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:11.780739+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-110"}]: dispatch 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:12.054675+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.59:0/3795783875' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:12.300 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:12.055085+0000 mon.a (mon.0) 3233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:12.366 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:12.393 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:10.795500+0000 osd.5 (osd.5) 279 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:10.797782+0000 osd.5 (osd.5) 280 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:10.874692+0000 osd.7 (osd.7) 189 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:10.875474+0000 osd.7 (osd.7) 190 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:11.624996+0000 osd.0 (osd.0) 294 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:11.626022+0000 osd.0 (osd.0) 295 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:11.779346+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:11.779414+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-18T00:43:12.439 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:11.780739+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-110"}]: dispatch 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:12.054675+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.59:0/3795783875' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:12.055085+0000 mon.a (mon.0) 3233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:10.795500+0000 osd.5 (osd.5) 279 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:10.797782+0000 osd.5 (osd.5) 280 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:10.874692+0000 osd.7 (osd.7) 189 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:10.875474+0000 osd.7 (osd.7) 190 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:11.624996+0000 osd.0 (osd.0) 294 : cluster [DBG] 25.15 deep-scrub starts 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:11.626022+0000 osd.0 (osd.0) 295 : cluster [DBG] 25.15 deep-scrub ok 2024-02-18T00:43:12.440 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:11.779346+0000 mon.a (mon.0) 3230 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:12.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:11.779414+0000 mon.a (mon.0) 3231 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-18T00:43:12.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:11.780739+0000 mon.a (mon.0) 3232 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-110"}]: dispatch 2024-02-18T00:43:12.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:12.054675+0000 mon.c (mon.2) 559 : audit [INF] from='client.? 172.21.15.59:0/3795783875' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:12.441 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:12.055085+0000 mon.a (mon.0) 3233 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:12.441 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:43:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:12] "GET /metrics HTTP/1.1" 200 191361 "" "Prometheus/2.33.4" 2024-02-18T00:43:12.592 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:12.594 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:12.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:12.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:12.723 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.782+0000 7f9d5f7fe700 1 -- 172.21.15.59:0/3795783875 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v647) v1 ==== 87+0+0 (secure 0 0 0) 0x7f9d64050390 con 0x7f9d700fcbb0 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 10 librados: watch_flush enter 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 10 librados: watch_flush exit 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 20 client.41588.objecter shutdown clearing up homeless session... 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 10 client.41588.objecter successfully canceled tick 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d58062c90 msgr2=0x7f9d58065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/3795783875 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d58062c90 0x7f9d58065150 secure :-1 s=READY pgs=3185 cs=0 l=1 rev1=1 rx=0x7f9d60009da0 tx=0x7f9d6000b040).stop 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 msgr2=0x7f9d7010d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/3795783875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d7010d600 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 rx=0x7f9d640048f0 tx=0x7f9d64009cb0).stop 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 shutdown_connections 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/3795783875 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9d58062c90 0x7f9d58065150 unknown :-1 s=CLOSED pgs=3185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/3795783875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9d700f7400 0x7f9d7010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:43:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/3795783875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d700fd510 0x7f9d701072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:43:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 --2- 172.21.15.59:0/3795783875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9d700fcbb0 0x7f9d7010d600 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.784+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 >> 172.21.15.59:0/3795783875 conn(0x7f9d700f2820 msgr2=0x7f9d700fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.785+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 shutdown_connections 2024-02-18T00:43:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.785+0000 7f9d74ac4700 1 -- 172.21.15.59:0/3795783875 wait complete. 2024-02-18T00:43:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:12.785+0000 7f9d74ac4700 1 librados: shutdown 2024-02-18T00:43:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:43:13.458 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:43:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:43:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:43:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:43:13.458 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:43:13.480 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:11.480843+0000 mgr.y (mgr.24454) 577 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:11.982631+0000 mgr.y (mgr.24454) 578 : cluster [DBG] pgmap v990: 214 pgs: 2 creating+activating, 1 creating+peering, 1 active+clean+scrubbing, 210 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s; 159 B/s, 7 objects/s recovering 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.333101+0000 osd.4 (osd.4) 365 : cluster [DBG] 4.d scrub starts 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.336559+0000 osd.4 (osd.4) 366 : cluster [DBG] 4.d scrub ok 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.774616+0000 mon.a (mon.0) 3234 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:12.781218+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-110"}]': finished 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:12.781295+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.781355+0000 mon.a (mon.0) 3237 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-18T00:43:13.481 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:12.783393+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-110", "mode": "writeback"}]: dispatch 2024-02-18T00:43:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:11.480843+0000 mgr.y (mgr.24454) 577 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:11.982631+0000 mgr.y (mgr.24454) 578 : cluster [DBG] pgmap v990: 214 pgs: 2 creating+activating, 1 creating+peering, 1 active+clean+scrubbing, 210 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s; 159 B/s, 7 objects/s recovering 2024-02-18T00:43:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.333101+0000 osd.4 (osd.4) 365 : cluster [DBG] 4.d scrub starts 2024-02-18T00:43:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.336559+0000 osd.4 (osd.4) 366 : cluster [DBG] 4.d scrub ok 2024-02-18T00:43:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.774616+0000 mon.a (mon.0) 3234 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:12.781218+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-110"}]': finished 2024-02-18T00:43:13.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:12.781295+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.781355+0000 mon.a (mon.0) 3237 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:12.783393+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-110", "mode": "writeback"}]: dispatch 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:11.480843+0000 mgr.y (mgr.24454) 577 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:11.982631+0000 mgr.y (mgr.24454) 578 : cluster [DBG] pgmap v990: 214 pgs: 2 creating+activating, 1 creating+peering, 1 active+clean+scrubbing, 210 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s; 159 B/s, 7 objects/s recovering 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.333101+0000 osd.4 (osd.4) 365 : cluster [DBG] 4.d scrub starts 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.336559+0000 osd.4 (osd.4) 366 : cluster [DBG] 4.d scrub ok 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.774616+0000 mon.a (mon.0) 3234 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:12.781218+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-110"}]': finished 2024-02-18T00:43:13.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:12.781295+0000 mon.a (mon.0) 3236 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:43:13.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.781355+0000 mon.a (mon.0) 3237 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-18T00:43:13.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:12.783393+0000 mon.a (mon.0) 3238 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-110", "mode": "writeback"}]: dispatch 2024-02-18T00:43:14.128 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-18T00:43:14.394 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.760604+0000 osd.5 (osd.5) 281 : cluster [DBG] 4.f scrub starts 2024-02-18T00:43:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.764166+0000 osd.5 (osd.5) 282 : cluster [DBG] 4.f scrub ok 2024-02-18T00:43:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.836114+0000 osd.7 (osd.7) 191 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:43:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:12.838931+0000 osd.7 (osd.7) 192 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:43:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:13.769130+0000 mon.a (mon.0) 3239 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:13.775599+0000 mon.a (mon.0) 3240 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-110", "mode": "writeback"}]': finished 2024-02-18T00:43:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:13.775688+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-18T00:43:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:13.897069+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:14.540 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.760604+0000 osd.5 (osd.5) 281 : cluster [DBG] 4.f scrub starts 2024-02-18T00:43:14.540 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.764166+0000 osd.5 (osd.5) 282 : cluster [DBG] 4.f scrub ok 2024-02-18T00:43:14.540 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.836114+0000 osd.7 (osd.7) 191 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:43:14.540 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:12.838931+0000 osd.7 (osd.7) 192 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:43:14.540 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:13.769130+0000 mon.a (mon.0) 3239 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:14.540 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:13.775599+0000 mon.a (mon.0) 3240 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-110", "mode": "writeback"}]': finished 2024-02-18T00:43:14.540 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:13.775688+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:13.897069+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.760604+0000 osd.5 (osd.5) 281 : cluster [DBG] 4.f scrub starts 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.764166+0000 osd.5 (osd.5) 282 : cluster [DBG] 4.f scrub ok 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.836114+0000 osd.7 (osd.7) 191 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:12.838931+0000 osd.7 (osd.7) 192 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:13.769130+0000 mon.a (mon.0) 3239 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:13.775599+0000 mon.a (mon.0) 3240 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-110", "mode": "writeback"}]': finished 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:13.775688+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-18T00:43:14.541 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:13.897069+0000 mon.a (mon.0) 3242 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:14.575 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:14.578 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:14.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:14.626 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:14.679 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:14.708 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:14.727 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:14.730 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:14.730 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:14.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:14.861 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:15.562 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:13.983062+0000 mgr.y (mgr.24454) 579 : cluster [DBG] pgmap v993: 214 pgs: 2 creating+activating, 1 creating+peering, 1 active+clean+scrubbing, 210 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s; 159 B/s, 7 objects/s recovering 2024-02-18T00:43:15.563 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:14.788085+0000 mon.a (mon.0) 3243 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:15.563 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:14.788161+0000 mon.a (mon.0) 3244 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-18T00:43:15.563 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:14.789643+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110"}]: dispatch 2024-02-18T00:43:15.563 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:13.983062+0000 mgr.y (mgr.24454) 579 : cluster [DBG] pgmap v993: 214 pgs: 2 creating+activating, 1 creating+peering, 1 active+clean+scrubbing, 210 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s; 159 B/s, 7 objects/s recovering 2024-02-18T00:43:15.563 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:14.788085+0000 mon.a (mon.0) 3243 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:15.563 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:14.788161+0000 mon.a (mon.0) 3244 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-18T00:43:15.563 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:14.789643+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110"}]: dispatch 2024-02-18T00:43:15.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:13.983062+0000 mgr.y (mgr.24454) 579 : cluster [DBG] pgmap v993: 214 pgs: 2 creating+activating, 1 creating+peering, 1 active+clean+scrubbing, 210 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 255 B/s wr, 1 op/s; 159 B/s, 7 objects/s recovering 2024-02-18T00:43:15.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:14.788085+0000 mon.a (mon.0) 3243 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:15.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:14.788161+0000 mon.a (mon.0) 3244 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-18T00:43:15.667 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:14.789643+0000 mon.a (mon.0) 3245 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110"}]: dispatch 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:2 71.15 71:ad551702:::test:head [create,write 0~4] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 588.08351778199994, 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067209718000000002, 2024-02-18T00:43:16.175 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.176 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088773+0000", 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088771+0000", 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7769999999999997e-06 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088778+0000", 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9749999999999996e-06 2024-02-18T00:43:16.177 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088787+0000", 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6979999999999998e-06 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088794+0000", 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3955999999999997e-05 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088838+0000", 2024-02-18T00:43:16.178 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0620000000000003e-06 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.088845+0000", 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.042483522000000003 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131329+0000", 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014866200000000001 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.179 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131477+0000", 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019506600000000001 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.131673+0000", 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074819399999999999 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.132421+0000", 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.010435201999999999 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.180 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.142856+0000", 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013070307 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155926+0000", 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2119000000000002e-05 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155958+0000", 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3989999999999999e-05 2024-02-18T00:43:16.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.155982+0000", 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067209718000000002 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:6 71.15 71:ad551702:::test:head [cmpext 0~4,write 0~4] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 587.98835240300002, 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026666680000000002, 2024-02-18T00:43:16.182 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 6 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.183 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183938+0000", 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183937+0000", 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4240000000000003e-06 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183942+0000", 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2269999999999996e-06 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.184 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183949+0000", 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9459999999999997e-06 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183954+0000", 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4397e-05 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.183978+0000", 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0843000000000002e-05 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.185 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.184019+0000", 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026585647 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.210605+0000", 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.026666680000000002 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.186 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.28735.0:7 71.15 71:ad551702:::test:head [delete] snapc 0=[] ondisk+write+known_if_redirected e130)", 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 587.95819781399996, 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019741999999999999, 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.28735", 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/158288180", 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 7 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.187 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214093+0000", 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214092+0000", 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1780000000000002e-06 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.188 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214097+0000", 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0831e-05 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214108+0000", 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6280000000000004e-06 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214112+0000", 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3321000000000001e-05 2024-02-18T00:43:16.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214156+0000", 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5122000000000004e-05 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214221+0000", 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7605e-05 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214298+0000", 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029685200000000001 2024-02-18T00:43:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.214595+0000", 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019093308999999999 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233688+0000", 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5790000000000003e-05 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233784+0000", 2024-02-18T00:43:16.191 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9073000000000001e-05 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233813+0000", 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1229999999999998e-05 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:28.233835+0000", 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019741999999999999 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.192 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30097.0:2 74.1f 74:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e132)", 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 585.80849189100002, 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014294953000000001, 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30097", 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3002694168", 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.193 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363798+0000", 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363797+0000", 2024-02-18T00:43:16.194 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9099999999999999e-06 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363804+0000", 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1026000000000001e-05 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363815+0000", 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9299999999999997e-06 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.195 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363822+0000", 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5386000000000002e-05 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363867+0000", 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0513999999999995e-05 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.363948+0000", 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5987000000000001e-05 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.196 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,6", 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364044+0000", 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00049711199999999997 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364541+0000", 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000204945 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.364746+0000", 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013291124 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.197 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378037+0000", 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2733e-05 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378069+0000", 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4020999999999999e-05 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:30.378093+0000", 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014294953000000001 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.198 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.25220.0:109 62.d 62:b295a385:test-rados-api-smithi059-82957-10::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e133)", 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 584.80699216899995, 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031474052000000002, 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.25220", 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 109 2024-02-18T00:43:16.199 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365298+0000", 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.200 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365297+0000", 2024-02-18T00:43:16.201 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8029999999999997e-06 2024-02-18T00:43:16.201 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365305+0000", 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.093e-05 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365315+0000", 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1999999999999997e-06 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365323+0000", 2024-02-18T00:43:16.202 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3896999999999997e-05 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365367+0000", 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2156000000000004e-05 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365449+0000", 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3950999999999995e-05 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 2,7", 2024-02-18T00:43:16.203 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365543+0000", 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00031575199999999998 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.365858+0000", 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000242795 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.366101+0000", 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.030572499 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.204 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396674+0000", 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8595999999999999e-05 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396732+0000", 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9944000000000002e-05 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:31.396772+0000", 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031474052000000002 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.205 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:334 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 581.48928975000001, 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02568811, 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.206 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.683001+0000", 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.682999+0000", 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015711060999999998 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.207 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698710+0000", 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7644999999999994e-05 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698778+0000", 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2190000000000004e-06 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698786+0000", 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3647999999999999e-05 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.208 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698840+0000", 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8643000000000001e-05 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.698859+0000", 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0097918710000000006 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.708650+0000", 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8368999999999998e-05 2024-02-18T00:43:16.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.708689+0000", 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.02568811 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:335 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 581.45382541000004, 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019566244, 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.210 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718465+0000", 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.211 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.718462+0000", 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0098851159999999993 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728347+0000", 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7931000000000002e-05 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728365+0000", 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7789999999999996e-06 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.212 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728373+0000", 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7117999999999998e-05 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728430+0000", 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9799000000000001e-05 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.728450+0000", 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0095431630000000003 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.213 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.737993+0000", 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8220999999999999e-05 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.738031+0000", 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.019566244 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:336 17.14 e136/104 -19:2db9b0ac:::temp_17.14_0_14313_1:head v 0'0, mlcod=0'0)", 2024-02-18T00:43:16.214 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 581.41519390500002, 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017739901999999998, 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757096+0000", 2024-02-18T00:43:16.215 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999967 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.757093+0000", 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0078518660000000007 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764945+0000", 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1234999999999999e-05 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.216 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764966+0000", 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.858e-06 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.764974+0000", 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2244000000000003e-05 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.765027+0000", 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1681000000000001e-05 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.217 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.765048+0000", 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0097585759999999997 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.774807+0000", 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9601999999999999e-05 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.218 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.774836+0000", 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.017739901999999998 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.3.0:500 62.c e205/123 62:348d0773:test-rados-api-smithi059-82957-14::foo71:5 v 205'15, mlcod=193'11)", 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 512.521557476, 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020248832000000001, 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.219 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650733+0000", 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999866 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.220 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650720+0000", 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.51e-05 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650745+0000", 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7669e-05 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650762+0000", 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.289e-06 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.221 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650770+0000", 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5961000000000001e-05 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650836+0000", 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1322000000000003e-05 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.650867+0000", 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020080569 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.222 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670948+0000", 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4106000000000002e-05 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:34:43.670982+0000", 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.020248832000000001 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40613.0:4 155.7 155:e19d0d35:::oid_RmXattrPP_:head [rmxattr attr1 in=5b] snapc 0=[] ondisk+write+known_if_redirected e235)", 2024-02-18T00:43:16.223 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 480.54842669599998, 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013802841, 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40613", 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/558528338", 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 4 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:43:16.224 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623864+0000", 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623861+0000", 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1669e-05 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.225 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623873+0000", 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2911999999999999e-05 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623886+0000", 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5609999999999999e-06 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623894+0000", 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3367000000000003e-05 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.226 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.623978+0000", 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6638999999999994e-05 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.624074+0000", 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015765399999999999 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,2", 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.624232+0000", 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001632643 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.227 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.625865+0000", 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051082250000000001 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.630973+0000", 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0065934679999999999 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637566+0000", 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8032000000000003e-05 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.228 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637624+0000", 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2246000000000002e-05 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:15.637667+0000", 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.013802841 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40868.0:1 189.6s0 189:72c88bc2:::oid_IsCompletePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e309)", 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 408.51381844500003, 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.008077509, 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40868", 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/458568879", 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.230 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658470+0000", 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9140000000000008e-06 2024-02-18T00:43:16.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.232 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.232 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.232 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658478+0000", 2024-02-18T00:43:16.232 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0781999999999999e-05 2024-02-18T00:43:16.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658489+0000", 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3030000000000002e-06 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658496+0000", 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051365309999999997 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.663633+0000", 2024-02-18T00:43:16.233 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4510000000000001e-06 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.663636+0000", 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0011838420000001 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.664820+0000", 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014268199999999999 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.664963+0000", 2024-02-18T00:43:16.235 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047133100000000002 2024-02-18T00:43:16.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.235 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.235 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:16.235 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.665434+0000", 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00069780299999999999 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666132+0000", 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00035506600000000002 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666487+0000", 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2547999999999998e-05 2024-02-18T00:43:16.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666549+0000", 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.008077509 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.41033.0:1 205.3s0 205:cbb9903e:::test_obj:head [omap-set-vals,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e382)", 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 335.48331617399998, 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3087259229999999, 2024-02-18T00:43:16.237 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.41033", 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3621370462", 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.238 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688972+0000", 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.114e-06 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688979+0000", 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8130000000000007e-06 2024-02-18T00:43:16.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688989+0000", 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7789999999999996e-06 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688997+0000", 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0035324459999999998 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.240 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.692529+0000", 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4070000000000003e-06 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.692534+0000", 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.302913354 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.995447+0000", 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015382699999999999 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.241 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.995601+0000", 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020991709999999999 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.997700+0000", 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3087259229999999 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDPGPull(4.22 e642/642 cost 1049576)", 2024-02-18T00:43:16.242 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:08.666483+0000", 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 7.5058075659999997, 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000330086, 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666483+0000", 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666483+0000", 2024-02-18T00:43:16.243 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967296 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666483+0000", 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2388000000000001e-05 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666495+0000", 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2709999999999992e-06 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.244 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666503+0000", 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8189999999999992e-06 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666511+0000", 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000185682 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666697+0000", 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000116135 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:08.666813+0000", 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000330086 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDPGPushReply(4.22 644/643 [PushReplyOp(4:4578209b:::obj_delete_at_hint.0000000014:head)])", 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:09.790987+0000", 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.3813029160000001, 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030654100000000002, 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.790987+0000", 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.790987+0000", 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.247 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.790987+0000", 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7600000000000002e-06 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.790992+0000", 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0591000000000001e-05 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.791002+0000", 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5869999999999998e-06 2024-02-18T00:43:16.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.791009+0000", 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1514999999999998e-05 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.791051+0000", 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000243477 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.791294+0000", 2024-02-18T00:43:16.249 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030654100000000002 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDPGPushReply(4.22 644/643 [PushReplyOp(4:479ba34f:::obj_delete_at_hint.0000000021:head)])", 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:09.792107+0000", 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.3801837729999997, 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029271499999999998, 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792107+0000", 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792107+0000", 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792106+0000", 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3839999999999996e-06 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792111+0000", 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0232e-05 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792122+0000", 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6640000000000001e-06 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792128+0000", 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5590999999999998e-05 2024-02-18T00:43:16.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792174+0000", 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000225446 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:09.792399+0000", 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029271499999999998 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(234.5s1 645/605 ECSubWrite(tid=750, reqid=client.31692.0:474, at_version=645'6, trim_to=0'0, roll_forward_to=640'5))", 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:10.789975+0000", 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.3823155160000002, 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002094865, 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.789975+0000", 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.254 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.789975+0000", 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.789974+0000", 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5749e-05 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790000+0000", 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2935999999999999e-05 2024-02-18T00:43:16.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790013+0000", 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8609999999999998e-06 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790021+0000", 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7022999999999997e-05 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.256 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790108+0000", 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1747999999999999e-05 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790129+0000", 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019237900000000001 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.792053+0000", 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6538999999999999e-05 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.257 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.792070+0000", 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002094865 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(234.5s1 645/605 ECSubWrite(tid=751, reqid=client.31692.0:474, at_version=0'0, trim_to=0'0, roll_forward_to=645'6))", 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:10.795677+0000", 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.3766138410000002, 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00054104100000000003, 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:16.258 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795677+0000", 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795677+0000", 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795676+0000", 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4072e-05 2024-02-18T00:43:16.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795690+0000", 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2184999999999999e-05 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795702+0000", 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.43e-06 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.260 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795709+0000", 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5678000000000003e-05 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795785+0000", 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1085e-05 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.795796+0000", 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00040183599999999998 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.261 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.796198+0000", 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9556e-05 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.796218+0000", 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00054104100000000003 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.262 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.14403.0:2037 5.e 5:7759931f:::notify.3:head [watch ping cookie 94750322748800] snapc 0=[] ondisk+write+known_if_redirected e642)", 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:13.434580+0000", 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 2.7377105259999999, 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00036040399999999999, 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.14403", 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2016795673", 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2037 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434580+0000", 2024-02-18T00:43:16.263 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434580+0000", 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434577+0000", 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4251e-05 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.264 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434592+0000", 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9610999999999999e-05 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434611+0000", 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8479999999999996e-06 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434621+0000", 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014344600000000001 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.265 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434765+0000", 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0185999999999995e-05 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434835+0000", 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000105445 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.434940+0000", 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00036040399999999999 2024-02-18T00:43:16.266 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpRead(234.5s1 648/605 ECSubRead(tid=752, to_read={234:bd34d09e:test-rados-api-smithi059-82957-111::foo:head=0,4096,40}, subchunks={234:bd34d09e:test-rados-api-smithi059-82957-111::foo:head=[0,1]}, attrs_to_read=))", 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:13.779112+0000", 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 2.3931779249999998, 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000294875, 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.267 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779112+0000", 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779112+0000", 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967296 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779112+0000", 2024-02-18T00:43:16.268 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.156e-06 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779118+0000", 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.079e-05 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779129+0000", 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6369999999999994e-06 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.269 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779139+0000", 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6904999999999998e-05 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779196+0000", 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00021151500000000001 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.779407+0000", 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000294875 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.270 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(234.5s1 648/605 ECSubWrite(tid=757, reqid=osd.0.8:114, at_version=648'7, trim_to=0'0, roll_forward_to=645'6))", 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:13.894807+0000", 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 2.2774832950000001, 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00042400100000000002, 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894807+0000", 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:16.271 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894807+0000", 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894806+0000", 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0869e-05 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894827+0000", 2024-02-18T00:43:16.272 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0688e-05 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894838+0000", 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.775e-06 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894842+0000", 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0120999999999997e-05 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894893+0000", 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2119e-05 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.894905+0000", 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030947800000000002 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.895214+0000", 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6888000000000002e-05 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:16.274 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.895231+0000", 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00042400100000000002 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:16.275 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:43:16.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:15.785580+0000 mon.a (mon.0) 3246 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:16.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:15.789677+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110"}]': finished 2024-02-18T00:43:16.308 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:15.789750+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-18T00:43:16.309 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:15.785580+0000 mon.a (mon.0) 3246 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:16.309 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:15.789677+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110"}]': finished 2024-02-18T00:43:16.309 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:15.789750+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-18T00:43:16.313 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:16.318 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:16.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:16.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:16.451 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:16.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:15.785580+0000 mon.a (mon.0) 3246 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:16.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:15.789677+0000 mon.a (mon.0) 3247 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-110"}]': finished 2024-02-18T00:43:16.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:15.789750+0000 mon.a (mon.0) 3248 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-18T00:43:16.562 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:43:16.602 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:16.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:16.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:16.706 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:16.737 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:16.834 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:16.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:16.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:16.851 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-18T00:43:16.938 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:16.964 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:16.984 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:16.987 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:16.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:17.090 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:17.144 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:17.148 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:17.149 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:17.163 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:17.250 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:17.278 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:17.394 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:17.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:17.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:17.500 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:17.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:15.984056+0000 mgr.y (mgr.24454) 580 : cluster [DBG] pgmap v996: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T00:43:17.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:16.798476+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-18T00:43:17.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:15.984056+0000 mgr.y (mgr.24454) 580 : cluster [DBG] pgmap v996: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T00:43:17.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:16.798476+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-18T00:43:17.527 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:15.984056+0000 mgr.y (mgr.24454) 580 : cluster [DBG] pgmap v996: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T00:43:17.527 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:16.798476+0000 mon.a (mon.0) 3249 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-18T00:43:17.528 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:17.626 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:17.629 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:17.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:17.730 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:17.757 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:17.913 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:17.916 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:17.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:18.017 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:18.046 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:18.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 -- 172.21.15.59:0/1020892234 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 msgr2=0x7f32c00f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:18.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 --2- 172.21.15.59:0/1020892234 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 0x7f32c00f6d40 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 rx=0x7f32ac004870 tx=0x7f32ac009cb0).stop 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 -- 172.21.15.59:0/1020892234 shutdown_connections 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 --2- 172.21.15.59:0/1020892234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32c00fcbb0 0x7f32c00fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 --2- 172.21.15.59:0/1020892234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 0x7f32c0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 --2- 172.21.15.59:0/1020892234 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 0x7f32c00f6d40 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 -- 172.21.15.59:0/1020892234 >> 172.21.15.59:0/1020892234 conn(0x7f32c00f2820 msgr2=0x7f32c00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 -- 172.21.15.59:0/1020892234 shutdown_connections 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.998+0000 7f32c7065700 1 -- 172.21.15.59:0/1020892234 wait complete. 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 librados: starting msgr at 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 librados: starting objecter 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 Processor -- start 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 -- start start 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 librados: setting wanted keys 2024-02-18T00:43:18.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 librados: calling monclient init 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 0x7f32c010e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32c00fcbb0 0x7f32c010e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 0x7f32c0108250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c0108740 con 0x7f32c00f7400 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c01088b0 con 0x7f32c00fcbb0 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c7065700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c0108a70 con 0x7f32c00fd510 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c4e01700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 0x7f32c010e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c5602700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 0x7f32c0108250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c4e01700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 0x7f32c010e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40382/0 (socket says 172.21.15.59:40382) 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32c4e01700 1 -- 172.21.15.59:0/3955501743 learned_addr learned my addr 172.21.15.59:0/3955501743 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:17.999+0000 7f32bffff700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32c00fcbb0 0x7f32c010e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c4e01700 1 -- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 msgr2=0x7f32c0108250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:18.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c4e01700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 0x7f32c0108250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c4e01700 1 -- 172.21.15.59:0/3955501743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32c00fcbb0 msgr2=0x7f32c010e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c4e01700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32c00fcbb0 0x7f32c010e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c4e01700 1 -- 172.21.15.59:0/3955501743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac005040 con 0x7f32c00f7400 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c5602700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 0x7f32c0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c4e01700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 0x7f32c010e0b0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 rx=0x7f32b400c7c0 tx=0x7f32b4007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32bdffb700 10 client.?.objecter ms_handle_connect 0x7f32c00f7400 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32bdffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32bdffb700 1 -- 172.21.15.59:0/3955501743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4004430 con 0x7f32c00f7400 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c0108d50 con 0x7f32c00f7400 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32bdffb700 1 -- 172.21.15.59:0/3955501743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32b400e970 con 0x7f32c00f7400 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32bdffb700 1 -- 172.21.15.59:0/3955501743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4013a70 con 0x7f32c00f7400 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c7065700 10 client.31785.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c01090d0 con 0x7f32c00f7400 2024-02-18T00:43:18.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.000+0000 7f32c7065700 1 librados: init done 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.001+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c005fa60 con 0x7f32c00f7400 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.001+0000 7f32bdffb700 1 -- 172.21.15.59:0/3955501743 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f32b400eae0 con 0x7f32c00f7400 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bdffb700 1 --2- 172.21.15.59:0/3955501743 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32b0062c90 0x7f32b0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bdffb700 1 -- 172.21.15.59:0/3955501743 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(652..652 src has 418..652) v4 ==== 8647+0+0 (secure 0 0 0) 0x7f32b407d0a0 con 0x7f32c00f7400 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bdffb700 10 client.31785.objecter ms_dispatch 0x7f32c0001550 osd_map(652..652 src has 418..652) v4 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bdffb700 3 client.31785.objecter handle_osd_map got epochs [652,652] > 0 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bdffb700 3 client.31785.objecter handle_osd_map decoding full epoch 652 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bffff700 1 --2- 172.21.15.59:0/3955501743 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32b0062c90 0x7f32b0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bdffb700 20 client.31785.objecter dump_active .. 0 homeless 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bdffb700 10 client.31785.objecter ms_handle_connect 0x7f32b0062c90 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.002+0000 7f32bffff700 1 --2- 172.21.15.59:0/3955501743 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32b0062c90 0x7f32b0065150 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 rx=0x7f32ac009a00 tx=0x7f32ac00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:18.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.006+0000 7f32bdffb700 1 -- 172.21.15.59:0/3955501743 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f32b4050320 con 0x7f32c00f7400 2024-02-18T00:43:18.083 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:18.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:18.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:18.186 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:18.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.185+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f32c0109730 con 0x7f32c00f7400 2024-02-18T00:43:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:43:18.189 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:43:18.189 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 652, 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-02-18T00:23:11.390869+0000", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-02-18T00:43:17.793300+0000", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-02-18T00:29:06.503960+0000", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-02-18T00:39:35.702780+0000", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5801984, 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "noscrub", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-02-18T00:43:18.190 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 104, 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 240, 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 8, 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "pacific", 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-02-18T00:43:18.191 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "device_health_metrics", 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:27:02.909667+0000", 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.192 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "67", 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:43:18.193 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.194 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.195 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_devicehealth": {} 2024-02-18T00:43:18.196 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".rgw.root", 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:13.697458+0000", 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.197 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 42, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 42, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 42, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 42, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 42, 2024-02-18T00:43:18.198 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "170", 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "107", 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.199 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.200 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.201 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "iscsi", 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:14.132992+0000", 2024-02-18T00:43:18.202 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 3, 2024-02-18T00:43:18.203 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 3, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 3, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 3, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 3, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "60", 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.204 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.205 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.206 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 4, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.log", 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:15.191206+0000", 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.208 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 42, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 42, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 42, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 42, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 42, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.209 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "643", 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "641", 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.211 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:43:18.212 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 5, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.control", 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:17.526023+0000", 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.213 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.214 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "62", 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:43:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 6, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.meta", 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:19.600478+0000", 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.218 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 22, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 22, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 22, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 22, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 22, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "6.16", 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 587, 2024-02-18T00:43:18.219 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 586, 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 586, 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "587", 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "587", 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "585", 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.220 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.221 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.222 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:43:18.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.187+0000 7f32bdffb700 1 -- 172.21.15.59:0/3955501743 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v652) v1 ==== 81+0+49676 (secure 0 0 0) 0x7f32b404ff10 con 0x7f32c00f7400 2024-02-18T00:43:18.223 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_bias": 4 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 25, 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "ceph_test_rados_api_asio", 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:32:59.546190+0000", 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.224 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.225 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "111", 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.226 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.227 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.228 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 234, 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-101", 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:42:18.224666+0000", 2024-02-18T00:43:18.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 3, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 1, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-02-18T00:43:18.230 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "650", 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "649", 2024-02-18T00:43:18.231 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "649", 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "649", 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 5, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 646, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.232 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "testprofile-test-rados-api-smithi059-82957-101", 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.233 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 8192, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.234 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 240, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-112", 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:43:17.244351+0000", 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 32769, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,creating", 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:43:18.235 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:43:18.236 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "652", 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:43:18.237 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:43:18.238 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:43:18.239 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85", 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.240 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 8, 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.242 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/3700238977", 2024-02-18T00:43:18.243 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6802/3700238977", 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6804/3700238977", 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6803/3700238977", 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0", 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.244 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 13, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.245 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6805/26279110", 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6806/26279110", 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6808/26279110", 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6807/26279110", 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7", 2024-02-18T00:43:18.248 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 18, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-02-18T00:43:18.249 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.250 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.251 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1153193637", 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6810/1153193637", 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6812/1153193637", 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6811/1153193637", 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:43:18.252 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b", 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 26, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-02-18T00:43:18.253 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.254 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.255 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6813/1805700964", 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6814/1805700964", 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6816/1805700964", 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6815/1805700964", 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231", 2024-02-18T00:43:18.256 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 31, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.257 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6800", 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6801", 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.258 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6803", 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6802", 2024-02-18T00:43:18.259 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6800/2044387056", 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6801/2044387056", 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6803/2044387056", 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6802/2044387056", 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.260 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01", 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 39, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.261 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6804", 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6805", 2024-02-18T00:43:18.262 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6807", 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.263 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6806", 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6804/2170432263", 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6805/2170432263", 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6807/2170432263", 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6806/2170432263", 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.264 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f", 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 47, 2024-02-18T00:43:18.265 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6808", 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6809", 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6811", 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.267 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6810", 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6808/3356009390", 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6809/3356009390", 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6811/3356009390", 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6810/3356009390", 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb", 2024-02-18T00:43:18.269 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 53, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 644, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.270 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6812", 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6813", 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.271 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6815", 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:43:18.272 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6814", 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6812/53025171", 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6813/53025171", 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6815/53025171", 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6814/53025171", 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.273 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:22.177823+0000", 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.274 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:43:05.326794+0000", 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.275 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:43:02.845578+0000", 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:17.756804+0000", 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.276 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:17.368058+0000", 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.277 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:30.520698+0000", 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:43:17.410231+0000", 2024-02-18T00:43:18.278 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:42:08.193646+0000", 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [], 2024-02-18T00:43:18.279 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [ 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.0", 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.3", 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.280 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.5", 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.281 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.6", 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 4, 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.8", 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.18", 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.2", 2024-02-18T00:43:18.283 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.4", 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:43:18.284 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.b", 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.e", 2024-02-18T00:43:18.285 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 0 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.f", 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.286 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.17", 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "5.1b", 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 6 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.9", 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 4 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.18", 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [], 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": { 2024-02-18T00:43:18.289 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/1243102711": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/817089002": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3174805070": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/3506000666": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/4167740619": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/999852549": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3575161033": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/2403979355": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/642821245": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/1656814063": "2024-02-19T00:23:45.307159+0000" 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-02-18T00:43:18.290 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "ruleset-failure-domain": "osd", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "testprofile-test-rados-api-smithi059-82957-101": { 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-device-class": "", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-root": "default", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "jerasure-per-chunk-alignment": "false", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-02-18T00:43:18.291 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "ruleset-failure-domain": "osd", 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van", 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "w": "8" 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [ 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 11, 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 3, 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 14, 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 9, 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 60, 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.294 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 78, 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.295 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 86, 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 94, 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:43:18.296 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 97, 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 4, 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 4 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 108, 2024-02-18T00:43:18.297 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 114, 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.298 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 128, 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:43:18.299 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 10, 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 134, 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 12, 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.300 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 154, 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 14, 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 166, 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.301 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 17, 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 177, 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 20, 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 237, 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 238, 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:43:18.303 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 4, 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-02-18T00:43:18.304 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-02-18T00:43:18.305 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:43:18.305 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:43:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 10 librados: watch_flush enter 2024-02-18T00:43:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 10 librados: watch_flush exit 2024-02-18T00:43:18.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 20 client.31785.objecter shutdown clearing up homeless session... 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 10 client.31785.objecter successfully canceled tick 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32b0062c90 msgr2=0x7f32b0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 1 --2- 172.21.15.59:0/3955501743 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32b0062c90 0x7f32b0065150 secure :-1 s=READY pgs=3186 cs=0 l=1 rev1=1 rx=0x7f32ac009a00 tx=0x7f32ac00b040).stop 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 msgr2=0x7f32c010e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.188+0000 7f32c7065700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 0x7f32c010e0b0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 rx=0x7f32b400c7c0 tx=0x7f32b4007ef0).stop 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 shutdown_connections 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32c00fd510 0x7f32c0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32c00f7400 0x7f32c010e0b0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 --2- 172.21.15.59:0/3955501743 >> v2:172.21.15.59:6800/3721891797 conn(0x7f32b0062c90 0x7f32b0065150 unknown :-1 s=CLOSED pgs=3186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 --2- 172.21.15.59:0/3955501743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32c00fcbb0 0x7f32c010e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 >> 172.21.15.59:0/3955501743 conn(0x7f32c00f2820 msgr2=0x7f32c0060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 shutdown_connections 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 -- 172.21.15.59:0/3955501743 wait complete. 2024-02-18T00:43:18.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:18.189+0000 7f32c7065700 1 librados: shutdown 2024-02-18T00:43:18.308 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:18.947 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:43:18.947 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:43:18.947 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:43:18.947 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:17.409984+0000 osd.6 (osd.6) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:17.410229+0000 osd.6 (osd.6) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:17.799188+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:17.812060+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:17.984437+0000 mgr.y (mgr.24454) 581 : cluster [DBG] pgmap v999: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:18.186994+0000 mon.a (mon.0) 3252 : audit [DBG] from='client.? 172.21.15.59:0/3955501743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:18.770464+0000 mon.a (mon.0) 3253 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:18.776432+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:18.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:18.776507+0000 mon.a (mon.0) 3255 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:17.409984+0000 osd.6 (osd.6) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:17.410229+0000 osd.6 (osd.6) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:17.799188+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:17.812060+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:17.984437+0000 mgr.y (mgr.24454) 581 : cluster [DBG] pgmap v999: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:18.186994+0000 mon.a (mon.0) 3252 : audit [DBG] from='client.? 172.21.15.59:0/3955501743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:18.770464+0000 mon.a (mon.0) 3253 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:18.776432+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:18.776507+0000 mon.a (mon.0) 3255 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-18T00:43:18.978 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:17.409984+0000 osd.6 (osd.6) 299 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:17.410229+0000 osd.6 (osd.6) 300 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:17.799188+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:17.812060+0000 mon.a (mon.0) 3251 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:17.984437+0000 mgr.y (mgr.24454) 581 : cluster [DBG] pgmap v999: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:18.186994+0000 mon.a (mon.0) 3252 : audit [DBG] from='client.? 172.21.15.59:0/3955501743' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:18.770464+0000 mon.a (mon.0) 3253 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:18.776432+0000 mon.a (mon.0) 3254 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:18.979 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:18.776507+0000 mon.a (mon.0) 3255 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-18T00:43:18.998 INFO:tasks.thrashosds.thrasher:j is {'epoch': 652, 'fsid': 'd129b4f8-cdf3-11ee-95ba-87774f69a715', 'created': '2024-02-18T00:23:11.390869+0000', 'modified': '2024-02-18T00:43:17.793300+0000', 'last_up_change': '2024-02-18T00:29:06.503960+0000', 'last_in_change': '2024-02-18T00:39:35.702780+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 104, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 240, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-18T00:27:02.909667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-02-18T00:29:13.697458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 42, 'pg_placement_num': 42, 'pg_placement_num_target': 42, 'pg_num_target': 42, 'pg_num_pending': 42, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '107', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'iscsi', 'create_time': '2024-02-18T00:29:14.132992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-18T00:29:15.191206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 42, 'pg_placement_num': 42, 'pg_placement_num_target': 42, 'pg_num_target': 42, 'pg_num_pending': 42, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '643', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '641', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-18T00:29:17.526023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-18T00:29:19.600478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 22, 'pg_placement_num': 22, 'pg_placement_num_target': 22, 'pg_num_target': 22, 'pg_num_pending': 22, 'last_pg_merge_meta': {'source_pgid': '6.16', 'ready_epoch': 587, 'last_epoch_started': 586, 'last_epoch_clean': 586, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '587', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '587', 'last_force_op_resend_preluminous': '585', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 25, 'pool_name': 'ceph_test_rados_api_asio', 'create_time': '2024-02-18T00:32:59.546190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 234, 'pool_name': 'test-rados-api-smithi059-82957-101', 'create_time': '2024-02-18T00:42:18.224666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '650', 'last_force_op_resend': '649', 'last_force_op_resend_prenautilus': '649', 'last_force_op_resend_preluminous': '649', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 646, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'testprofile-test-rados-api-smithi059-82957-101', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 240, 'pool_name': 'test-rados-api-smithi059-82957-112', 'create_time': '2024-02-18T00:43:17.244351+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '652', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}], 'osds': [{'osd': 0, 'uuid': '6e729102-80e2-4a4e-92ca-600ee8b1ed85', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6801', 'nonce': 3700238977}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 3700238977}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 3700238977}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6803', 'nonce': 3700238977}]}, 'public_addr': '172.21.15.59:6801/3700238977', 'cluster_addr': '172.21.15.59:6802/3700238977', 'heartbeat_back_addr': '172.21.15.59:6804/3700238977', 'heartbeat_front_addr': '172.21.15.59:6803/3700238977', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f67a7206-6d31-497f-97e0-672d134b13d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6805', 'nonce': 26279110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 26279110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 26279110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6807', 'nonce': 26279110}]}, 'public_addr': '172.21.15.59:6805/26279110', 'cluster_addr': '172.21.15.59:6806/26279110', 'heartbeat_back_addr': '172.21.15.59:6808/26279110', 'heartbeat_front_addr': '172.21.15.59:6807/26279110', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6223dbd-e539-4770-9e5a-7b756304f8f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6809', 'nonce': 1153193637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1153193637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1153193637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6811', 'nonce': 1153193637}]}, 'public_addr': '172.21.15.59:6809/1153193637', 'cluster_addr': '172.21.15.59:6810/1153193637', 'heartbeat_back_addr': '172.21.15.59:6812/1153193637', 'heartbeat_front_addr': '172.21.15.59:6811/1153193637', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fff275df-95e0-41f6-96c3-e531f05de32b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6813', 'nonce': 1805700964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1805700964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 1805700964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6815', 'nonce': 1805700964}]}, 'public_addr': '172.21.15.59:6813/1805700964', 'cluster_addr': '172.21.15.59:6814/1805700964', 'heartbeat_back_addr': '172.21.15.59:6816/1805700964', 'heartbeat_front_addr': '172.21.15.59:6815/1805700964', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a0bae50b-4f29-4310-ac5f-bc7285464231', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6800', 'nonce': 2044387056}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6801', 'nonce': 2044387056}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6803', 'nonce': 2044387056}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6802', 'nonce': 2044387056}]}, 'public_addr': '172.21.15.137:6800/2044387056', 'cluster_addr': '172.21.15.137:6801/2044387056', 'heartbeat_back_addr': '172.21.15.137:6803/2044387056', 'heartbeat_front_addr': '172.21.15.137:6802/2044387056', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '513a4192-45fb-4444-a1f9-4bd2f2a5fa01', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6804', 'nonce': 2170432263}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6805', 'nonce': 2170432263}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6807', 'nonce': 2170432263}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6806', 'nonce': 2170432263}]}, 'public_addr': '172.21.15.137:6804/2170432263', 'cluster_addr': '172.21.15.137:6805/2170432263', 'heartbeat_back_addr': '172.21.15.137:6807/2170432263', 'heartbeat_front_addr': '172.21.15.137:6806/2170432263', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5bbb7683-ea5a-490c-8c8b-67b53c45406f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6808', 'nonce': 3356009390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6809', 'nonce': 3356009390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6811', 'nonce': 3356009390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6810', 'nonce': 3356009390}]}, 'public_addr': '172.21.15.137:6808/3356009390', 'cluster_addr': '172.21.15.137:6809/3356009390', 'heartbeat_back_addr': '172.21.15.137:6811/3356009390', 'heartbeat_front_addr': '172.21.15.137:6810/3356009390', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '951d317b-7a34-41a0-b1a1-a2851b2070bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 644, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6812', 'nonce': 53025171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6813', 'nonce': 53025171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6815', 'nonce': 53025171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6814', 'nonce': 53025171}]}, 'public_addr': '172.21.15.137:6812/53025171', 'cluster_addr': '172.21.15.137:6813/53025171', 'heartbeat_back_addr': '172.21.15.137:6815/53025171', 'heartbeat_front_addr': '172.21.15.137:6814/53025171', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:22.177823+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:43:05.326794+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:43:02.845578+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:17.756804+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:17.368058+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:30.520698+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:43:17.410231+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:42:08.193646+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.3', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '2.5', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.6', 'mappings': [{'from': 4, 'to': 1}]}, {'pgid': '2.8', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.18', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.2', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.4', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 0}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.1b', 'mappings': [{'from': 0, 'to': 6}]}, {'pgid': '25.9', 'mappings': [{'from': 0, 'to': 4}]}, {'pgid': '25.18', 'mappings': [{'from': 0, 'to': 2}]}], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.59:0/1243102711': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/817089002': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/3174805070': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/3506000666': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/4167740619': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/999852549': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/3575161033': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:0/2403979355': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/642821245': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:6800/1656814063': '2024-02-19T00:23:45.307159+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}, 'testprofile-test-rados-api-smithi059-82957-101': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van', 'w': '8'}}, 'removed_snaps_queue': [{'pool': 11, 'snaps': [{'begin': 3, 'length': 1}, {'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 14, 'snaps': [{'begin': 9, 'length': 2}]}, {'pool': 60, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 78, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 86, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 94, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 97, 'snaps': [{'begin': 4, 'length': 4}]}, {'pool': 108, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 114, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 128, 'snaps': [{'begin': 2, 'length': 2}, {'begin': 6, 'length': 2}, {'begin': 10, 'length': 2}]}, {'pool': 134, 'snaps': [{'begin': 12, 'length': 1}]}, {'pool': 154, 'snaps': [{'begin': 14, 'length': 1}]}, {'pool': 166, 'snaps': [{'begin': 17, 'length': 1}]}, {'pool': 177, 'snaps': [{'begin': 20, 'length': 1}]}, {'pool': 237, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 238, 'snaps': [{'begin': 4, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-18T00:43:18.998 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:43:19.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.279+0000 7f7fddb92700 1 -- 172.21.15.59:0/2497388002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 msgr2=0x7f7fd80fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.279+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2497388002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd80fcf90 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 rx=0x7f7fc80047b0 tx=0x7f7fc8009cb0).stop 2024-02-18T00:43:19.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.279+0000 7f7fddb92700 1 -- 172.21.15.59:0/2497388002 shutdown_connections 2024-02-18T00:43:19.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.279+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2497388002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80f7400 0x7f7fd8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.279+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2497388002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd80fcf90 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.279+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2497388002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fd80fd510 0x7f7fd80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.279+0000 7f7fddb92700 1 -- 172.21.15.59:0/2497388002 >> 172.21.15.59:0/2497388002 conn(0x7f7fd80f2820 msgr2=0x7f7fd80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:19.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 -- 172.21.15.59:0/2497388002 shutdown_connections 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 -- 172.21.15.59:0/2497388002 wait complete. 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 librados: starting msgr at 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 librados: starting objecter 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 Processor -- start 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 -- start start 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 librados: setting wanted keys 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.280+0000 7f7fddb92700 1 librados: calling monclient init 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fddb92700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fd80f7400 0x7f7fd810c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fddb92700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd810c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fddb92700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80fd510 0x7f7fd81061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fddb92700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81066a0 con 0x7f7fd80f7400 2024-02-18T00:43:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fddb92700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8106810 con 0x7f7fd80fd510 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fddb92700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd81069d0 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd810c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd810c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43988/0 (socket says 172.21.15.59:43988) 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 -- 172.21.15.59:0/2831984988 learned_addr learned my addr 172.21.15.59:0/2831984988 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fd80f7400 0x7f7fd810c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 -- 172.21.15.59:0/2831984988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80fd510 msgr2=0x7f7fd81061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80fd510 0x7f7fd81061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80fd510 0x7f7fd81061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 -- 172.21.15.59:0/2831984988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fd80f7400 msgr2=0x7f7fd810c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7fd80f7400 0x7f7fd810c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd6ffd700 1 -- 172.21.15.59:0/2831984988 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc005040 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.281+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fd80fd510 0x7f7fd81061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fd6ffd700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd810c500 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 rx=0x7f7fcc009400 tx=0x7f7fcc011040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fd4ff9700 10 client.?.objecter ms_handle_connect 0x7f7fd80fcbb0 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fd4ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc014040 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc8005040 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fcc009890 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc019e20 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fddb92700 10 client.41600.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd8107010 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.282+0000 7f7fddb92700 1 librados: init done 2024-02-18T00:43:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.283+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f7fd805fa60 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.284+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f7fcc00fbf0 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.284+0000 7f7fd4ff9700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7fc0062c90 0x7f7fc0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.284+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(653..653 src has 418..653) v4 ==== 8660+0+0 (secure 0 0 0) 0x7f7fcc081d40 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.284+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 osd_map(653..653 src has 418..653) v4 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.284+0000 7f7fd4ff9700 3 client.41600.objecter handle_osd_map got epochs [653,653] > 0 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.284+0000 7f7fd4ff9700 3 client.41600.objecter handle_osd_map decoding full epoch 653 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.284+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7fc0062c90 0x7f7fc0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fd4ff9700 20 client.41600.objecter dump_active .. 0 homeless 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v653) v1 ==== 54+0+16 (secure 0 0 0) 0x7f7fcc082220 con 0x7f7fd80fcbb0 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fc0062c90 2024-02-18T00:43:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7fc0062c90 0x7f7fc0065150 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 rx=0x7f7fc8005010 tx=0x7f7fc8009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 10 client.41600.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 1 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7fd80fdea0 0x7f7fd80fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 1 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 0 1 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd805ee80 osd=0 4 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 10 client.41600.objecter _send_command 1 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd8105010 con 0x7f7fd80fdea0 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7fd80fdea0 0x7f7fd80fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd80fdea0 2024-02-18T00:43:19.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.285+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7fd80fdea0 0x7f7fd80fe280 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.286+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd8105010 con 0x7f7fd80fdea0 2024-02-18T00:43:19.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.286+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:43:19.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.286+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 1 = system:0 2024-02-18T00:43:19.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.286+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd805ee80 osd=0 4 2024-02-18T00:43:19.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.286+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 0 1 2024-02-18T00:43:19.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 10 client.41600.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 2 2024-02-18T00:43:19.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.305 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 2 2024-02-18T00:43:19.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd805ee80 osd=0 3 2024-02-18T00:43:19.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 0 2 2024-02-18T00:43:19.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd805ee80 osd=0 4 2024-02-18T00:43:19.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 10 client.41600.objecter _send_command 2 2024-02-18T00:43:19.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.303+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd80fe9b0 con 0x7f7fd80fdea0 2024-02-18T00:43:19.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd00048f0 con 0x7f7fd80fdea0 2024-02-18T00:43:19.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd805ee80 osd=0 4 2024-02-18T00:43:19.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 0 2 2024-02-18T00:43:19.308 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:43:19.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 10 client.41600.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6805/26279110 conn(0x7f7fd80598f0 0x7f7fd8059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 1 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8028540 osd=1 4 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 10 client.41600.objecter _send_command 3 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.305+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd80286b0 con 0x7f7fd80598f0 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.306+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6805/26279110 conn(0x7f7fd80598f0 0x7f7fd8059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.306+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6805/26279110 conn(0x7f7fd80598f0 0x7f7fd8059cf0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.306+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd80598f0 2024-02-18T00:43:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.307+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd80286b0 con 0x7f7fd80598f0 2024-02-18T00:43:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.307+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:43:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.307+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 3 = system:0 2024-02-18T00:43:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.307+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8028540 osd=1 4 2024-02-18T00:43:19.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.307+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 1 3 2024-02-18T00:43:19.312 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:19.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 10 client.41600.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 4 2024-02-18T00:43:19.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 4 2024-02-18T00:43:19.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8028540 osd=1 3 2024-02-18T00:43:19.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 1 4 2024-02-18T00:43:19.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8028540 osd=1 4 2024-02-18T00:43:19.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 10 client.41600.objecter _send_command 4 2024-02-18T00:43:19.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.331+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd8059360 con 0x7f7fd80598f0 2024-02-18T00:43:19.336 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:43:19.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd8059360 con 0x7f7fd80598f0 2024-02-18T00:43:19.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8028540 osd=1 4 2024-02-18T00:43:19.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 1 4 2024-02-18T00:43:19.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 10 client.41600.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 5 2024-02-18T00:43:19.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7fd80280a0 0x7f7fd811a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 5 2024-02-18T00:43:19.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 2 5 2024-02-18T00:43:19.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8059770 osd=2 4 2024-02-18T00:43:19.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 10 client.41600.objecter _send_command 5 2024-02-18T00:43:19.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.334+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd811cd00 con 0x7f7fd80280a0 2024-02-18T00:43:19.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.335+0000 7f7fd6ffd700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7fd80280a0 0x7f7fd811a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.335+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd80280a0 2024-02-18T00:43:19.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.335+0000 7f7fd6ffd700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7fd80280a0 0x7f7fd811a0b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.336+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd811cd00 con 0x7f7fd80280a0 2024-02-18T00:43:19.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.336+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:43:19.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.336+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 5 = system:0 2024-02-18T00:43:19.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.336+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8059770 osd=2 4 2024-02-18T00:43:19.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.336+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 2 5 2024-02-18T00:43:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 10 client.41600.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 6 2024-02-18T00:43:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 6 2024-02-18T00:43:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8059770 osd=2 3 2024-02-18T00:43:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 2 6 2024-02-18T00:43:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8059770 osd=2 4 2024-02-18T00:43:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 10 client.41600.objecter _send_command 6 2024-02-18T00:43:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.352+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd8027600 con 0x7f7fd80280a0 2024-02-18T00:43:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd8027600 con 0x7f7fd80280a0 2024-02-18T00:43:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8059770 osd=2 4 2024-02-18T00:43:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 2 6 2024-02-18T00:43:19.358 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 10 client.41600.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 7 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7fd8117c00 0x7f7fd811b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 7 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 3 7 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8027a10 osd=3 4 2024-02-18T00:43:19.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 10 client.41600.objecter _send_command 7 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd811bcd0 con 0x7f7fd8117c00 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.354+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7fd8117c00 0x7f7fd811b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.355+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd8117c00 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.355+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7fd8117c00 0x7f7fd811b720 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.355+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd811bcd0 con 0x7f7fd8117c00 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.355+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.355+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 7 = system:0 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.355+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8027a10 osd=3 4 2024-02-18T00:43:19.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.355+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 3 7 2024-02-18T00:43:19.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 10 client.41600.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 8 2024-02-18T00:43:19.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 8 2024-02-18T00:43:19.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8027a10 osd=3 3 2024-02-18T00:43:19.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 3 8 2024-02-18T00:43:19.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8027a10 osd=3 4 2024-02-18T00:43:19.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 10 client.41600.objecter _send_command 8 2024-02-18T00:43:19.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.371+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd8117160 con 0x7f7fd8117c00 2024-02-18T00:43:19.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.372+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd8117160 con 0x7f7fd8117c00 2024-02-18T00:43:19.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.372+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.372+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.372+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8027a10 osd=3 4 2024-02-18T00:43:19.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.372+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 3 8 2024-02-18T00:43:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 10 client.41600.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 9 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7fd811ee80 0x7f7fd8121270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 9 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 4 9 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8116f20 osd=4 4 2024-02-18T00:43:19.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 10 client.41600.objecter _send_command 9 2024-02-18T00:43:19.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd8117090 con 0x7f7fd811ee80 2024-02-18T00:43:19.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7fd811ee80 0x7f7fd8121270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd811ee80 2024-02-18T00:43:19.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.373+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7fd811ee80 0x7f7fd8121270 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.374+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd8117090 con 0x7f7fd811ee80 2024-02-18T00:43:19.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.374+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:43:19.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.374+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 9 = system:0 2024-02-18T00:43:19.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.374+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8116f20 osd=4 4 2024-02-18T00:43:19.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.374+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 4 9 2024-02-18T00:43:19.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 10 client.41600.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 10 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 10 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8116f20 osd=4 3 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 4 10 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8116f20 osd=4 4 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 10 client.41600.objecter _send_command 10 2024-02-18T00:43:19.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.389+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd8117090 con 0x7f7fd811ee80 2024-02-18T00:43:19.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd8117090 con 0x7f7fd811ee80 2024-02-18T00:43:19.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8116f20 osd=4 4 2024-02-18T00:43:19.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 4 10 2024-02-18T00:43:19.395 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:43:19.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 10 client.41600.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 11 2024-02-18T00:43:19.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7fd8121c70 0x7f7fd81240a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 11 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 5 11 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8117090 osd=5 4 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 10 client.41600.objecter _send_command 11 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.391+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd8126e70 con 0x7f7fd8121c70 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.392+0000 7f7fd6ffd700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7fd8121c70 0x7f7fd81240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.392+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd8121c70 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.392+0000 7f7fd6ffd700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7fd8121c70 0x7f7fd81240a0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.393+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd8126e70 con 0x7f7fd8121c70 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.393+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:43:19.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.393+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 11 = system:0 2024-02-18T00:43:19.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.393+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8117090 osd=5 4 2024-02-18T00:43:19.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.393+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 5 11 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 10 client.41600.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 12 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 12 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8117090 osd=5 3 2024-02-18T00:43:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 5 12 2024-02-18T00:43:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8117090 osd=5 4 2024-02-18T00:43:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 10 client.41600.objecter _send_command 12 2024-02-18T00:43:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.407+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd8126e70 con 0x7f7fd8121c70 2024-02-18T00:43:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd8126e70 con 0x7f7fd8121c70 2024-02-18T00:43:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8117090 osd=5 4 2024-02-18T00:43:19.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 5 12 2024-02-18T00:43:19.411 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:43:19.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 10 client.41600.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 13 2024-02-18T00:43:19.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7fd81261b0 0x7f7fd8126610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 13 2024-02-18T00:43:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 6 13 2024-02-18T00:43:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8126e70 osd=6 4 2024-02-18T00:43:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 10 client.41600.objecter _send_command 13 2024-02-18T00:43:19.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd8126c80 con 0x7f7fd81261b0 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.409+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7fd81261b0 0x7f7fd8126610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.410+0000 7f7fd7fff700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7fd81261b0 0x7f7fd8126610 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.410+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd81261b0 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.410+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd0020ba0 con 0x7f7fd81261b0 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.410+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.410+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 13 = system:0 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.410+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8126e70 osd=6 4 2024-02-18T00:43:19.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.410+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 6 13 2024-02-18T00:43:19.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 10 client.41600.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 14 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 14 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd8126e70 osd=6 3 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 6 14 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8126e70 osd=6 4 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 10 client.41600.objecter _send_command 14 2024-02-18T00:43:19.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.425+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd81290f0 con 0x7f7fd81261b0 2024-02-18T00:43:19.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd81290f0 con 0x7f7fd81261b0 2024-02-18T00:43:19.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd8126e70 osd=6 4 2024-02-18T00:43:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 6 14 2024-02-18T00:43:19.429 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 10 client.41600.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 15 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6812/53025171 conn(0x7f7fd81297a0 0x7f7fd8129c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 15 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 7 15 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd81290f0 osd=7 4 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 10 client.41600.objecter _send_command 15 2024-02-18T00:43:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fd812d4b0 con 0x7f7fd81297a0 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.427+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6812/53025171 conn(0x7f7fd81297a0 0x7f7fd8129c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.428+0000 7f7fd4ff9700 10 client.41600.objecter ms_handle_connect 0x7f7fd81297a0 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.428+0000 7f7fd77fe700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6812/53025171 conn(0x7f7fd81297a0 0x7f7fd8129c00 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.428+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7fd812d4b0 con 0x7f7fd81297a0 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.428+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.428+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 15 = system:0 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.428+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd81290f0 osd=7 4 2024-02-18T00:43:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.428+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 7 15 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 10 client.41600.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign -1 16 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 20 client.41600.objecter _get_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_remove -1 16 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 20 client.41600.objecter get_session s=0x7f7fd81290f0 osd=7 3 2024-02-18T00:43:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 15 client.41600.objecter _session_command_op_assign 7 16 2024-02-18T00:43:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd81290f0 osd=7 4 2024-02-18T00:43:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 10 client.41600.objecter _send_command 16 2024-02-18T00:43:19.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.443+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f7fd812c330 con 0x7f7fd81297a0 2024-02-18T00:43:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd4ff9700 1 -- 172.21.15.59:0/2831984988 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f7fd812c330 con 0x7f7fd81297a0 2024-02-18T00:43:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd4ff9700 10 client.41600.objecter ms_dispatch 0x7f7fd8001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd4ff9700 10 client.41600.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd4ff9700 20 client.41600.objecter put_session s=0x7f7fd81290f0 osd=7 4 2024-02-18T00:43:19.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd4ff9700 15 client.41600.objecter _session_command_op_remove 7 16 2024-02-18T00:43:19.447 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:43:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 librados: watch_flush enter 2024-02-18T00:43:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 librados: watch_flush exit 2024-02-18T00:43:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.0 2024-02-18T00:43:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7fd80fdea0 msgr2=0x7f7fd80fe280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6801/3700238977 conn(0x7f7fd80fdea0 0x7f7fd80fe280 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd805ee80 osd=0 1 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.1 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6805/26279110 conn(0x7f7fd80598f0 msgr2=0x7f7fd8059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6805/26279110 conn(0x7f7fd80598f0 0x7f7fd8059cf0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8028540 osd=1 1 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.2 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7fd80280a0 msgr2=0x7f7fd811a0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6809/1153193637 conn(0x7f7fd80280a0 0x7f7fd811a0b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8059770 osd=2 1 2024-02-18T00:43:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.3 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7fd8117c00 msgr2=0x7f7fd811b720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6813/1805700964 conn(0x7f7fd8117c00 0x7f7fd811b720 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8027a10 osd=3 1 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.4 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7fd811ee80 msgr2=0x7f7fd8121270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6800/2044387056 conn(0x7f7fd811ee80 0x7f7fd8121270 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd7fff700 1 -- 172.21.15.59:0/2831984988 reap_dead start 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd7fff700 1 -- 172.21.15.59:0/2831984988 reap_dead start 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8116f20 osd=4 1 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.5 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7fd8121c70 msgr2=0x7f7fd81240a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6804/2170432263 conn(0x7f7fd8121c70 0x7f7fd81240a0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8117090 osd=5 1 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.6 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7fd81261b0 msgr2=0x7f7fd8126610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6808/3356009390 conn(0x7f7fd81261b0 0x7f7fd8126610 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd8126e70 osd=6 1 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter close_session for osd.7 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6812/53025171 conn(0x7f7fd81297a0 msgr2=0x7f7fd8129c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.137:6812/53025171 conn(0x7f7fd81297a0 0x7f7fd8129c00 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter put_session s=0x7f7fd81290f0 osd=7 1 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 20 client.41600.objecter shutdown clearing up homeless session... 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 10 client.41600.objecter successfully canceled tick 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7fc0062c90 msgr2=0x7f7fc0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7fc0062c90 0x7f7fc0065150 secure :-1 s=READY pgs=3187 cs=0 l=1 rev1=1 rx=0x7f7fc8005010 tx=0x7f7fc8009cb0).stop 2024-02-18T00:43:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 msgr2=0x7f7fd810c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fd7fff700 1 -- 172.21.15.59:0/2831984988 reap_dead start 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd810c500 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 rx=0x7f7fcc009400 tx=0x7f7fcc011040).stop 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 shutdown_connections 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 --2- 172.21.15.59:0/2831984988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7fd80fcbb0 0x7f7fd810c500 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 >> 172.21.15.59:0/2831984988 conn(0x7f7fd80f2820 msgr2=0x7f7fd80fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.445+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 shutdown_connections 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.446+0000 7f7fddb92700 1 -- 172.21.15.59:0/2831984988 wait complete. 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:19.446+0000 7f7fddb92700 1 librados: shutdown 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.454 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.455 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:43:19.643 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-18T00:43:19.912 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:20.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:18.796826+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:19.284736+0000 mon.c (mon.2) 560 : audit [DBG] from='client.? 172.21.15.59:0/2831984988' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:19.761972+0000 osd.4 (osd.4) 367 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:19.762178+0000 osd.4 (osd.4) 368 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:19.787633+0000 mon.a (mon.0) 3257 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:19.787673+0000 mon.a (mon.0) 3258 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-18T00:43:20.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:19.788041+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-112"}]: dispatch 2024-02-18T00:43:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:18.796826+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:19.284736+0000 mon.c (mon.2) 560 : audit [DBG] from='client.? 172.21.15.59:0/2831984988' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:19.761972+0000 osd.4 (osd.4) 367 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:19.762178+0000 osd.4 (osd.4) 368 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:19.787633+0000 mon.a (mon.0) 3257 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:19.787673+0000 mon.a (mon.0) 3258 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-18T00:43:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:19.788041+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-112"}]: dispatch 2024-02-18T00:43:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:18.796826+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:19.284736+0000 mon.c (mon.2) 560 : audit [DBG] from='client.? 172.21.15.59:0/2831984988' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:19.761972+0000 osd.4 (osd.4) 367 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:19.762178+0000 osd.4 (osd.4) 368 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:19.787633+0000 mon.a (mon.0) 3257 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:19.787673+0000 mon.a (mon.0) 3258 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-18T00:43:20.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:19.788041+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-112"}]: dispatch 2024-02-18T00:43:20.265 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:20.269 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:20.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:20.370 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:20.400 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:19.985016+0000 mgr.y (mgr.24454) 582 : cluster [DBG] pgmap v1002: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:20.787866+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-112"}]': finished 2024-02-18T00:43:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:20.787942+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-18T00:43:21.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:20.789675+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-112", "mode": "writeback"}]: dispatch 2024-02-18T00:43:21.049 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:21.055 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:21.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:21.157 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:21.185 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:21.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:19.985016+0000 mgr.y (mgr.24454) 582 : cluster [DBG] pgmap v1002: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:20.787866+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-112"}]': finished 2024-02-18T00:43:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:20.787942+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-18T00:43:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:20.789675+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-112", "mode": "writeback"}]: dispatch 2024-02-18T00:43:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:19.985016+0000 mgr.y (mgr.24454) 582 : cluster [DBG] pgmap v1002: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:20.787866+0000 mon.a (mon.0) 3260 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-112"}]': finished 2024-02-18T00:43:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:20.787942+0000 mon.a (mon.0) 3261 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-18T00:43:21.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:20.789675+0000 mon.a (mon.0) 3262 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-112", "mode": "writeback"}]: dispatch 2024-02-18T00:43:21.259 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:21.260 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:21.261 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:21.286 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:43:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:43:21.361 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:21.389 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:21.487 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:21.489 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:21.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:21.515 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:43:21.516 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:43:21.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:21.617 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:43:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:43:21.619 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:21.767 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:21.770 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:21.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:21.871 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:21.903 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:21.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:21.784917+0000 mon.a (mon.0) 3263 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:21.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:21.789805+0000 mon.a (mon.0) 3264 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-112", "mode": "writeback"}]': finished 2024-02-18T00:43:21.940 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:21.789883+0000 mon.a (mon.0) 3265 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-18T00:43:21.940 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:21.784917+0000 mon.a (mon.0) 3263 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:21.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:21.789805+0000 mon.a (mon.0) 3264 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-112", "mode": "writeback"}]': finished 2024-02-18T00:43:21.941 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:21.789883+0000 mon.a (mon.0) 3265 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-18T00:43:21.941 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:21.947 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:21.947 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:22.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:21.784917+0000 mon.a (mon.0) 3263 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:21.789805+0000 mon.a (mon.0) 3264 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-112", "mode": "writeback"}]': finished 2024-02-18T00:43:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:21.789883+0000 mon.a (mon.0) 3265 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-18T00:43:22.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:22.250 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-18T00:43:22.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:22] "GET /metrics HTTP/1.1" 200 191361 "" "Prometheus/2.33.4" 2024-02-18T00:43:22.516 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:22.627 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:22.630 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:22.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:22.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:22.765 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.210+0000 7fae541aa700 1 -- 172.21.15.59:0/1202084539 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0fdbb0 msgr2=0x7fae4c0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.210+0000 7fae541aa700 1 --2- 172.21.15.59:0/1202084539 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0fdbb0 0x7fae4c0fdf90 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 rx=0x7fae3c009be0 tx=0x7fae3c004d10).stop 2024-02-18T00:43:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.210+0000 7fae541aa700 1 -- 172.21.15.59:0/1202084539 shutdown_connections 2024-02-18T00:43:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.210+0000 7fae541aa700 1 --2- 172.21.15.59:0/1202084539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae4c0f8420 0x7fae4c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.210+0000 7fae541aa700 1 --2- 172.21.15.59:0/1202084539 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0fdbb0 0x7fae4c0fdf90 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.210+0000 7fae541aa700 1 --2- 172.21.15.59:0/1202084539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c0fe510 0x7fae4c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.210+0000 7fae541aa700 1 -- 172.21.15.59:0/1202084539 >> 172.21.15.59:0/1202084539 conn(0x7fae4c0f3800 msgr2=0x7fae4c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 -- 172.21.15.59:0/1202084539 shutdown_connections 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 -- 172.21.15.59:0/1202084539 wait complete. 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 librados: starting msgr at 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 librados: starting objecter 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 Processor -- start 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 -- start start 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 librados: setting wanted keys 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 librados: calling monclient init 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0f8420 0x7fae4c10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:22.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae4c0fdbb0 0x7fae4c10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c0fe510 0x7fae4c1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1088c0 con 0x7fae4c0fdbb0 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c108a30 con 0x7fae4c0fe510 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.211+0000 7fae541aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c108bf0 con 0x7fae4c0f8420 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0f8420 0x7fae4c10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51745700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae4c0fdbb0 0x7fae4c10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0f8420 0x7fae4c10e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56110/0 (socket says 172.21.15.59:56110) 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 -- 172.21.15.59:0/2330244044 learned_addr learned my addr 172.21.15.59:0/2330244044 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae52747700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c0fe510 0x7fae4c1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 -- 172.21.15.59:0/2330244044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c0fe510 msgr2=0x7fae4c1083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c0fe510 0x7fae4c1083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 -- 172.21.15.59:0/2330244044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae4c0fdbb0 msgr2=0x7fae4c10e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:22.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae4c0fdbb0 0x7fae4c10e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 -- 172.21.15.59:0/2330244044 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae44005040 con 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae51f46700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0f8420 0x7fae4c10e1e0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7fae3c004b70 tx=0x7fae3c004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae4affd700 10 client.?.objecter ms_handle_connect 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae4affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae4affd700 1 -- 172.21.15.59:0/2330244044 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00e040 con 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae4affd700 1 -- 172.21.15.59:0/2330244044 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae3c018040 con 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0097c0 con 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae4affd700 1 -- 172.21.15.59:0/2330244044 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c013cf0 con 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae541aa700 10 client.41606.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c109110 con 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.212+0000 7fae541aa700 1 librados: init done 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.213+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c05ee80 con 0x7fae4c0f8420 2024-02-18T00:43:22.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.213+0000 7fae4affd700 1 -- 172.21.15.59:0/2330244044 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fae3c013420 con 0x7fae4c0f8420 2024-02-18T00:43:22.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae4affd700 1 --2- 172.21.15.59:0/2330244044 >> v2:172.21.15.59:6800/3721891797 conn(0x7fae34062c90 0x7fae34065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:22.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae4affd700 1 -- 172.21.15.59:0/2330244044 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(656..656 src has 418..656) v4 ==== 8655+0+0 (secure 0 0 0) 0x7fae3c07edc0 con 0x7fae4c0f8420 2024-02-18T00:43:22.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae4affd700 10 client.41606.objecter ms_dispatch 0x7fae4c001550 osd_map(656..656 src has 418..656) v4 2024-02-18T00:43:22.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae4affd700 3 client.41606.objecter handle_osd_map got epochs [656,656] > 0 2024-02-18T00:43:22.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae4affd700 3 client.41606.objecter handle_osd_map decoding full epoch 656 2024-02-18T00:43:22.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae51745700 1 --2- 172.21.15.59:0/2330244044 >> v2:172.21.15.59:6800/3721891797 conn(0x7fae34062c90 0x7fae34065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:22.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae4affd700 20 client.41606.objecter dump_active .. 0 homeless 2024-02-18T00:43:22.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae4affd700 10 client.41606.objecter ms_handle_connect 0x7fae34062c90 2024-02-18T00:43:22.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.214+0000 7fae51745700 1 --2- 172.21.15.59:0/2330244044 >> v2:172.21.15.59:6800/3721891797 conn(0x7fae34062c90 0x7fae34065150 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 rx=0x7fae4c109b20 tx=0x7fae4000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:22.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.218+0000 7fae4affd700 1 -- 172.21.15.59:0/2330244044 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fae3c052040 con 0x7fae4c0f8420 2024-02-18T00:43:22.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.407+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fae4c05fa60 con 0x7fae4c0f8420 2024-02-18T00:43:22.805 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:22.807 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:22.807 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.817+0000 7fae4affd700 1 -- 172.21.15.59:0/2330244044 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v657) v1 ==== 97+0+0 (secure 0 0 0) 0x7fae3c004810 con 0x7fae4c0f8420 2024-02-18T00:43:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 10 librados: watch_flush enter 2024-02-18T00:43:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 10 librados: watch_flush exit 2024-02-18T00:43:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 20 client.41606.objecter shutdown clearing up homeless session... 2024-02-18T00:43:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 10 client.41606.objecter successfully canceled tick 2024-02-18T00:43:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 >> v2:172.21.15.59:6800/3721891797 conn(0x7fae34062c90 msgr2=0x7fae34065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 --2- 172.21.15.59:0/2330244044 >> v2:172.21.15.59:6800/3721891797 conn(0x7fae34062c90 0x7fae34065150 secure :-1 s=READY pgs=3188 cs=0 l=1 rev1=1 rx=0x7fae4c109b20 tx=0x7fae4000b040).stop 2024-02-18T00:43:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0f8420 msgr2=0x7fae4c10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0f8420 0x7fae4c10e1e0 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 rx=0x7fae3c004b70 tx=0x7fae3c004d10).stop 2024-02-18T00:43:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 shutdown_connections 2024-02-18T00:43:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 --2- 172.21.15.59:0/2330244044 >> v2:172.21.15.59:6800/3721891797 conn(0x7fae34062c90 0x7fae34065150 unknown :-1 s=CLOSED pgs=3188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c0fe510 0x7fae4c1083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae4c0f8420 0x7fae4c10e1e0 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 --2- 172.21.15.59:0/2330244044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae4c0fdbb0 0x7fae4c10e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.820+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 >> 172.21.15.59:0/2330244044 conn(0x7fae4c0f3800 msgr2=0x7fae4c0f5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.821+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 shutdown_connections 2024-02-18T00:43:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.821+0000 7fae541aa700 1 -- 172.21.15.59:0/2330244044 wait complete. 2024-02-18T00:43:22.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:22.821+0000 7fae541aa700 1 librados: shutdown 2024-02-18T00:43:22.825 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:43:22.909 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:22.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:21.490948+0000 mgr.y (mgr.24454) 583 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:22.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:21.833947+0000 mon.a (mon.0) 3266 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:22.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:21.985443+0000 mgr.y (mgr.24454) 584 : cluster [DBG] pgmap v1005: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:22.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:22.408920+0000 mon.c (mon.2) 561 : audit [INF] from='client.? 172.21.15.59:0/2330244044' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:22.937 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:22.409330+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:22.937 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:21.490948+0000 mgr.y (mgr.24454) 583 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:22.938 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:21.833947+0000 mon.a (mon.0) 3266 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:22.938 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:21.985443+0000 mgr.y (mgr.24454) 584 : cluster [DBG] pgmap v1005: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:22.938 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:22.408920+0000 mon.c (mon.2) 561 : audit [INF] from='client.? 172.21.15.59:0/2330244044' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:22.938 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:22.409330+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:22.939 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:22.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:21.490948+0000 mgr.y (mgr.24454) 583 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:22.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:21.833947+0000 mon.a (mon.0) 3266 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:22.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:21.985443+0000 mgr.y (mgr.24454) 584 : cluster [DBG] pgmap v1005: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:22.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:22.408920+0000 mon.c (mon.2) 561 : audit [INF] from='client.? 172.21.15.59:0/2330244044' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:22.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:22.409330+0000 mon.a (mon.0) 3267 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:23.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:43:23.691 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:23.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:23.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:23.796 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:23.827 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:23.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:22.809944+0000 mon.a (mon.0) 3268 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:23.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:22.818361+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:23.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:22.818442+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:23.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:22.818506+0000 mon.a (mon.0) 3271 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-18T00:43:23.986 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:22.819200+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112"}]: dispatch 2024-02-18T00:43:23.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:23.035554+0000 osd.7 (osd.7) 193 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:23.987 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:23.035667+0000 osd.7 (osd.7) 194 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:24.026 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:24.029 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:24.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:24.130 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:24.160 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:24.242 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 -- 172.21.15.59:0/899578356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 msgr2=0x7f09b40f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/899578356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b40f9090 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7f09b0004870 tx=0x7f09b0009cb0).stop 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 -- 172.21.15.59:0/899578356 shutdown_connections 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/899578356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b4102210 0x7f09b4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/899578356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b40f9090 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/899578356 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09b40f9610 0x7f09b4101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 -- 172.21.15.59:0/899578356 >> 172.21.15.59:0/899578356 conn(0x7f09b40f2820 msgr2=0x7f09b40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 -- 172.21.15.59:0/899578356 shutdown_connections 2024-02-18T00:43:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.244+0000 7f09bbf7f700 1 -- 172.21.15.59:0/899578356 wait complete. 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 librados: starting msgr at 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 librados: starting objecter 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 Processor -- start 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 -- start start 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 librados: setting wanted keys 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 librados: calling monclient init 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b410e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b40f9610 0x7f09b410e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09b4102210 0x7f09b41083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b41088c0 con 0x7f09b40f8cb0 2024-02-18T00:43:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4108a30 con 0x7f09b40f9610 2024-02-18T00:43:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09bbf7f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4108bf0 con 0x7f09b4102210 2024-02-18T00:43:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09b9d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b410e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09b9d1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b410e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49712/0 (socket says 172.21.15.59:49712) 2024-02-18T00:43:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09b9d1b700 1 -- 172.21.15.59:0/3799871346 learned_addr learned my addr 172.21.15.59:0/3799871346 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.245+0000 7f09b951a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b40f9610 0x7f09b410e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09ba51c700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09b4102210 0x7f09b41083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09b9d1b700 1 -- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09b4102210 msgr2=0x7f09b41083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09b9d1b700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09b4102210 0x7f09b41083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09b9d1b700 1 -- 172.21.15.59:0/3799871346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b40f9610 msgr2=0x7f09b410e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09b9d1b700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b40f9610 0x7f09b410e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09b9d1b700 1 -- 172.21.15.59:0/3799871346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0005040 con 0x7f09b40f8cb0 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09b951a700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b40f9610 0x7f09b410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09ba51c700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09b4102210 0x7f09b41083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09b9d1b700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b410e230 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 rx=0x7f09b00043a0 tx=0x7f09b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09aaffd700 10 client.?.objecter ms_handle_connect 0x7f09b40f8cb0 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09aaffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09aaffd700 1 -- 172.21.15.59:0/3799871346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000e070 con 0x7f09b40f8cb0 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09aaffd700 1 -- 172.21.15.59:0/3799871346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b0009710 con 0x7f09b40f8cb0 2024-02-18T00:43:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b4108e70 con 0x7f09b40f8cb0 2024-02-18T00:43:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09aaffd700 1 -- 172.21.15.59:0/3799871346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0019070 con 0x7f09b40f8cb0 2024-02-18T00:43:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09bbf7f700 10 client.31803.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b4115060 con 0x7f09b40f8cb0 2024-02-18T00:43:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.246+0000 7f09bbf7f700 1 librados: init done 2024-02-18T00:43:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.247+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b405fa60 con 0x7f09b40f8cb0 2024-02-18T00:43:24.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09aaffd700 1 -- 172.21.15.59:0/3799871346 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f09b00090d0 con 0x7f09b40f8cb0 2024-02-18T00:43:24.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09aaffd700 1 --2- 172.21.15.59:0/3799871346 >> v2:172.21.15.59:6800/3721891797 conn(0x7f09a0062c90 0x7f09a0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:24.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09aaffd700 1 -- 172.21.15.59:0/3799871346 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(658..658 src has 418..658) v4 ==== 8647+0+0 (secure 0 0 0) 0x7f09b007ddb0 con 0x7f09b40f8cb0 2024-02-18T00:43:24.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09aaffd700 10 client.31803.objecter ms_dispatch 0x7f09b4001550 osd_map(658..658 src has 418..658) v4 2024-02-18T00:43:24.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09aaffd700 3 client.31803.objecter handle_osd_map got epochs [658,658] > 0 2024-02-18T00:43:24.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09aaffd700 3 client.31803.objecter handle_osd_map decoding full epoch 658 2024-02-18T00:43:24.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09b951a700 1 --2- 172.21.15.59:0/3799871346 >> v2:172.21.15.59:6800/3721891797 conn(0x7f09a0062c90 0x7f09a0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:24.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.248+0000 7f09aaffd700 20 client.31803.objecter dump_active .. 0 homeless 2024-02-18T00:43:24.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.249+0000 7f09aaffd700 10 client.31803.objecter ms_handle_connect 0x7f09a0062c90 2024-02-18T00:43:24.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.249+0000 7f09b951a700 1 --2- 172.21.15.59:0/3799871346 >> v2:172.21.15.59:6800/3721891797 conn(0x7f09a0062c90 0x7f09a0065150 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 rx=0x7f09b4109b70 tx=0x7f09a400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:24.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:22.809944+0000 mon.a (mon.0) 3268 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:24.254 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:22.818361+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:22.818442+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:22.818506+0000 mon.a (mon.0) 3271 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:22.819200+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112"}]: dispatch 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:23.035554+0000 osd.7 (osd.7) 193 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:23.035667+0000 osd.7 (osd.7) 194 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:22.809944+0000 mon.a (mon.0) 3268 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:22.818361+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:24.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:22.818442+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:24.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:22.818506+0000 mon.a (mon.0) 3271 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-18T00:43:24.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:22.819200+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112"}]: dispatch 2024-02-18T00:43:24.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:23.035554+0000 osd.7 (osd.7) 193 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:24.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:23.035667+0000 osd.7 (osd.7) 194 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.253+0000 7f09aaffd700 1 -- 172.21.15.59:0/3799871346 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f09b00510a0 con 0x7f09b40f8cb0 2024-02-18T00:43:24.258 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:43:24.258 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:43:24.259 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:24.261 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:24.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:24.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:24.392 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:24.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.425+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09b4107300 con 0x7f09a0062c90 2024-02-18T00:43:24.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.442+0000 7f09aaffd700 1 -- 172.21.15.59:0/3799871346 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+549611 (secure 0 0 0) 0x7f09b4107300 con 0x7f09a0062c90 2024-02-18T00:43:24.451 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.445+0000 7f09bbf7f700 10 librados: watch_flush enter 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.445+0000 7f09bbf7f700 10 librados: watch_flush exit 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.445+0000 7f09bbf7f700 20 client.31803.objecter shutdown clearing up homeless session... 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.445+0000 7f09bbf7f700 10 client.31803.objecter successfully canceled tick 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 >> v2:172.21.15.59:6800/3721891797 conn(0x7f09a0062c90 msgr2=0x7f09a0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/3799871346 >> v2:172.21.15.59:6800/3721891797 conn(0x7f09a0062c90 0x7f09a0065150 secure :-1 s=READY pgs=3189 cs=0 l=1 rev1=1 rx=0x7f09b4109b70 tx=0x7f09a400b040).stop 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 msgr2=0x7f09b410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b410e230 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 rx=0x7f09b00043a0 tx=0x7f09b000b040).stop 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 shutdown_connections 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09b4102210 0x7f09b41083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09b40f8cb0 0x7f09b410e230 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/3799871346 >> v2:172.21.15.59:6800/3721891797 conn(0x7f09a0062c90 0x7f09a0065150 unknown :-1 s=CLOSED pgs=3189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 --2- 172.21.15.59:0/3799871346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b40f9610 0x7f09b410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 >> 172.21.15.59:0/3799871346 conn(0x7f09b40f2820 msgr2=0x7f09b40f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 shutdown_connections 2024-02-18T00:43:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 -- 172.21.15.59:0/3799871346 wait complete. 2024-02-18T00:43:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:24.446+0000 7f09bbf7f700 1 librados: shutdown 2024-02-18T00:43:24.457 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:43:24.931 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-18T00:43:25.196 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:25.243 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:23.814702+0000 mon.a (mon.0) 3273 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:25.243 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:23.860803+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112"}]': finished 2024-02-18T00:43:25.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:23.860871+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-18T00:43:25.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:23.985838+0000 mgr.y (mgr.24454) 585 : cluster [DBG] pgmap v1008: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:25.244 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:23.814702+0000 mon.a (mon.0) 3273 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:25.244 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:23.860803+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112"}]': finished 2024-02-18T00:43:25.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:23.860871+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-18T00:43:25.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:23.985838+0000 mgr.y (mgr.24454) 585 : cluster [DBG] pgmap v1008: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:25.245 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:25.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:25.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:23.814702+0000 mon.a (mon.0) 3273 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:23.860803+0000 mon.a (mon.0) 3274 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-112"}]': finished 2024-02-18T00:43:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:23.860871+0000 mon.a (mon.0) 3275 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-18T00:43:25.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:23.985838+0000 mgr.y (mgr.24454) 585 : cluster [DBG] pgmap v1008: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:25.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:25.377 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:25.827 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:25.832 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:25.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:25.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:26.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:24.426985+0000 mgr.y (mgr.24454) 586 : audit [DBG] from='client.31803 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:43:26.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:24.872493+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-18T00:43:26.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:25.320516+0000 mon.a (mon.0) 3277 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:24.426985+0000 mgr.y (mgr.24454) 586 : audit [DBG] from='client.31803 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:43:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:24.872493+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-18T00:43:26.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:25.320516+0000 mon.a (mon.0) 3277 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:26.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:24.426985+0000 mgr.y (mgr.24454) 586 : audit [DBG] from='client.31803 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:43:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:24.872493+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-18T00:43:26.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:25.320516+0000 mon.a (mon.0) 3277 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:26.565 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:26.569 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:26.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:26.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:26.700 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:26.800 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:26.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:26.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:26.903 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:26.930 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:27.003 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:27.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:27.006 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:27.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:27.121 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:43:27.121 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:43:27.133 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:25.876931+0000 mon.a (mon.0) 3278 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-18T00:43:27.133 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:25.881094+0000 mon.a (mon.0) 3279 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:27.133 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:25.986606+0000 mgr.y (mgr.24454) 587 : cluster [DBG] pgmap v1011: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:27.135 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:27.155 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:27.158 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:27.158 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:27.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:25.876931+0000 mon.a (mon.0) 3278 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-18T00:43:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:25.881094+0000 mon.a (mon.0) 3279 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:25.986606+0000 mgr.y (mgr.24454) 587 : cluster [DBG] pgmap v1011: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:25.876931+0000 mon.a (mon.0) 3278 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-18T00:43:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:25.881094+0000 mon.a (mon.0) 3279 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:25.986606+0000 mgr.y (mgr.24454) 587 : cluster [DBG] pgmap v1011: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:27.260 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:27.289 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:27.487 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:27.490 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:27.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:27.504 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":1008,"stamp":"2024-02-18T00:43:23.985624+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5849,"num_objects":233,"num_object_clones":0,"num_object_copies":699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":0,"num_read":2376,"num_read_kb":2088,"num_write":927,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":5116,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1392,"ondisk_log_size":1392,"up":642,"acting":642,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":638,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2480856,"kb_used_data":33432,"kb_used_omap":0,"kb_used_meta":2447360,"kb_avail":747480360,"statfs":{"total":767960285184,"available":765419888640,"internally_reserved":0,"allocated":34234368,"data_stored":17854441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506096640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000851"},"pg_stats":[{"pgid":"240.17","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788562+0000","last_change":"2024-02-18T00:43:18.793789+0000","last_active":"2024-02-18T00:43:19.788562+0000","last_peered":"2024-02-18T00:43:19.788562+0000","last_clean":"2024-02-18T00:43:19.788562+0000","last_became_active":"2024-02-18T00:43:18.793464+0000","last_became_peered":"2024-02-18T00:43:18.793464+0000","last_unstale":"2024-02-18T00:43:19.788562+0000","last_undegraded":"2024-02-18T00:43:19.788562+0000","last_fullsized":"2024-02-18T00:43:19.788562+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"240.16","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791235+0000","last_change":"2024-02-18T00:43:18.790827+0000","last_active":"2024-02-18T00:43:19.791235+0000","last_peered":"2024-02-18T00:43:19.791235+0000","last_clean":"2024-02-18T00:43:19.791235+0000","last_became_active":"2024-02-18T00:43:18.790477+0000","last_became_peered":"2024-02-18T00:43:18.790477+0000","last_unstale":"2024-02-18T00:43:19.791235+0000","last_undegraded":"2024-02-18T00:43:19.791235+0000","last_fullsized":"2024-02-18T00:43:19.791235+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"240.15","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790777+0000","last_change":"2024-02-18T00:43:18.789995+0000","last_active":"2024-02-18T00:43:18.790777+0000","last_peered":"2024-02-18T00:43:18.790777+0000","last_clean":"2024-02-18T00:43:18.790777+0000","last_became_active":"2024-02-18T00:43:18.788787+0000","last_became_peered":"2024-02-18T00:43:18.788787+0000","last_unstale":"2024-02-18T00:43:18.790777+0000","last_undegraded":"2024-02-18T00:43:18.790777+0000","last_fullsized":"2024-02-18T00:43:18.790777+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"240.14","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.792707+0000","last_change":"2024-02-18T00:43:18.792264+0000","last_active":"2024-02-18T00:43:18.792707+0000","last_peered":"2024-02-18T00:43:18.792707+0000","last_clean":"2024-02-18T00:43:18.792707+0000","last_became_active":"2024-02-18T00:43:18.791951+0000","last_became_peered":"2024-02-18T00:43:18.791951+0000","last_unstale":"2024-02-18T00:43:18.792707+0000","last_undegraded":"2024-02-18T00:43:18.792707+0000","last_fullsized":"2024-02-18T00:43:18.792707+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"240.13","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790294+0000","last_change":"2024-02-18T00:43:18.793825+0000","last_active":"2024-02-18T00:43:19.790294+0000","last_peered":"2024-02-18T00:43:19.790294+0000","last_clean":"2024-02-18T00:43:19.790294+0000","last_became_active":"2024-02-18T00:43:18.793494+0000","last_became_peered":"2024-02-18T00:43:18.793494+0000","last_unstale":"2024-02-18T00:43:19.790294+0000","last_undegraded":"2024-02-18T00:43:19.790294+0000","last_fullsized":"2024-02-18T00:43:19.790294+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"240.12","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.792612+0000","last_change":"2024-02-18T00:43:18.792151+0000","last_active":"2024-02-18T00:43:18.792612+0000","last_peered":"2024-02-18T00:43:18.792612+0000","last_clean":"2024-02-18T00:43:18.792612+0000","last_became_active":"2024-02-18T00:43:18.791846+0000","last_became_peered":"2024-02-18T00:43:18.791846+0000","last_unstale":"2024-02-18T00:43:18.792612+0000","last_undegraded":"2024-02-18T00:43:18.792612+0000","last_fullsized":"2024-02-18T00:43:18.792612+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"240.11","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790295+0000","last_change":"2024-02-18T00:43:18.789619+0000","last_active":"2024-02-18T00:43:18.790295+0000","last_peered":"2024-02-18T00:43:18.790295+0000","last_clean":"2024-02-18T00:43:18.790295+0000","last_became_active":"2024-02-18T00:43:18.789318+0000","last_became_peered":"2024-02-18T00:43:18.789318+0000","last_unstale":"2024-02-18T00:43:18.790295+0000","last_undegraded":"2024-02-18T00:43:18.790295+0000","last_fullsized":"2024-02-18T00:43:18.790295+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"240.10","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788753+0000","last_change":"2024-02-18T00:43:18.786922+0000","last_active":"2024-02-18T00:43:19.788753+0000","last_peered":"2024-02-18T00:43:19.788753+0000","last_clean":"2024-02-18T00:43:19.788753+0000","last_became_active":"2024-02-18T00:43:18.786630+0000","last_became_peered":"2024-02-18T00:43:18.786630+0000","last_unstale":"2024-02-18T00:43:19.788753+0000","last_undegraded":"2024-02-18T00:43:19.788753+0000","last_fullsized":"2024-02-18T00:43:19.788753+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"240.f","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790004+0000","last_change":"2024-02-18T00:43:18.794120+0000","last_active":"2024-02-18T00:43:19.790004+0000","last_peered":"2024-02-18T00:43:19.790004+0000","last_clean":"2024-02-18T00:43:19.790004+0000","last_became_active":"2024-02-18T00:43:18.793798+0000","last_became_peered":"2024-02-18T00:43:18.793798+0000","last_unstale":"2024-02-18T00:43:19.790004+0000","last_undegraded":"2024-02-18T00:43:19.790004+0000","last_fullsized":"2024-02-18T00:43:19.790004+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"240.e","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.792852+0000","last_change":"2024-02-18T00:43:18.790233+0000","last_active":"2024-02-18T00:43:18.792852+0000","last_peered":"2024-02-18T00:43:18.792852+0000","last_clean":"2024-02-18T00:43:18.792852+0000","last_became_active":"2024-02-18T00:43:18.789556+0000","last_became_peered":"2024-02-18T00:43:18.789556+0000","last_unstale":"2024-02-18T00:43:18.792852+0000","last_undegraded":"2024-02-18T00:43:18.792852+0000","last_fullsized":"2024-02-18T00:43:18.792852+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"240.d","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791115+0000","last_change":"2024-02-18T00:43:18.794369+0000","last_active":"2024-02-18T00:43:19.791115+0000","last_peered":"2024-02-18T00:43:19.791115+0000","last_clean":"2024-02-18T00:43:19.791115+0000","last_became_active":"2024-02-18T00:43:18.794077+0000","last_became_peered":"2024-02-18T00:43:18.794077+0000","last_unstale":"2024-02-18T00:43:19.791115+0000","last_undegraded":"2024-02-18T00:43:19.791115+0000","last_fullsized":"2024-02-18T00:43:19.791115+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"240.c","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.796410+0000","last_change":"2024-02-18T00:43:18.796223+0000","last_active":"2024-02-18T00:43:18.796410+0000","last_peered":"2024-02-18T00:43:18.796410+0000","last_clean":"2024-02-18T00:43:18.796410+0000","last_became_active":"2024-02-18T00:43:18.795940+0000","last_became_peered":"2024-02-18T00:43:18.795940+0000","last_unstale":"2024-02-18T00:43:18.796410+0000","last_undegraded":"2024-02-18T00:43:18.796410+0000","last_fullsized":"2024-02-18T00:43:18.796410+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"240.b","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.791515+0000","last_change":"2024-02-18T00:43:18.791125+0000","last_active":"2024-02-18T00:43:18.791515+0000","last_peered":"2024-02-18T00:43:18.791515+0000","last_clean":"2024-02-18T00:43:18.791515+0000","last_became_active":"2024-02-18T00:43:18.790773+0000","last_became_peered":"2024-02-18T00:43:18.790773+0000","last_unstale":"2024-02-18T00:43:18.791515+0000","last_undegraded":"2024-02-18T00:43:18.791515+0000","last_fullsized":"2024-02-18T00:43:18.791515+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"240.a","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790334+0000","last_change":"2024-02-18T00:43:18.789149+0000","last_active":"2024-02-18T00:43:18.790334+0000","last_peered":"2024-02-18T00:43:18.790334+0000","last_clean":"2024-02-18T00:43:18.790334+0000","last_became_active":"2024-02-18T00:43:18.788836+0000","last_became_peered":"2024-02-18T00:43:18.788836+0000","last_unstale":"2024-02-18T00:43:18.790334+0000","last_undegraded":"2024-02-18T00:43:18.790334+0000","last_fullsized":"2024-02-18T00:43:18.790334+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"240.9","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787661+0000","last_change":"2024-02-18T00:43:18.787362+0000","last_active":"2024-02-18T00:43:19.787661+0000","last_peered":"2024-02-18T00:43:19.787661+0000","last_clean":"2024-02-18T00:43:19.787661+0000","last_became_active":"2024-02-18T00:43:18.786390+0000","last_became_peered":"2024-02-18T00:43:18.786390+0000","last_unstale":"2024-02-18T00:43:19.787661+0000","last_undegraded":"2024-02-18T00:43:19.787661+0000","last_fullsized":"2024-02-18T00:43:19.787661+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"240.8","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788753+0000","last_change":"2024-02-18T00:43:18.792766+0000","last_active":"2024-02-18T00:43:19.788753+0000","last_peered":"2024-02-18T00:43:19.788753+0000","last_clean":"2024-02-18T00:43:19.788753+0000","last_became_active":"2024-02-18T00:43:18.792464+0000","last_became_peered":"2024-02-18T00:43:18.792464+0000","last_unstale":"2024-02-18T00:43:19.788753+0000","last_undegraded":"2024-02-18T00:43:19.788753+0000","last_fullsized":"2024-02-18T00:43:19.788753+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"240.7","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.785447+0000","last_change":"2024-02-18T00:43:18.782713+0000","last_active":"2024-02-18T00:43:18.785447+0000","last_peered":"2024-02-18T00:43:18.785447+0000","last_clean":"2024-02-18T00:43:18.785447+0000","last_became_active":"2024-02-18T00:43:18.782436+0000","last_became_peered":"2024-02-18T00:43:18.782436+0000","last_unstale":"2024-02-18T00:43:18.785447+0000","last_undegraded":"2024-02-18T00:43:18.785447+0000","last_fullsized":"2024-02-18T00:43:18.785447+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"240.6","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.792871+0000","last_change":"2024-02-18T00:43:18.790399+0000","last_active":"2024-02-18T00:43:18.792871+0000","last_peered":"2024-02-18T00:43:18.792871+0000","last_clean":"2024-02-18T00:43:18.792871+0000","last_became_active":"2024-02-18T00:43:18.789947+0000","last_became_peered":"2024-02-18T00:43:18.789947+0000","last_unstale":"2024-02-18T00:43:18.792871+0000","last_undegraded":"2024-02-18T00:43:18.792871+0000","last_fullsized":"2024-02-18T00:43:18.792871+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"240.5","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789319+0000","last_change":"2024-02-18T00:43:18.790247+0000","last_active":"2024-02-18T00:43:19.789319+0000","last_peered":"2024-02-18T00:43:19.789319+0000","last_clean":"2024-02-18T00:43:19.789319+0000","last_became_active":"2024-02-18T00:43:18.789968+0000","last_became_peered":"2024-02-18T00:43:18.789968+0000","last_unstale":"2024-02-18T00:43:19.789319+0000","last_undegraded":"2024-02-18T00:43:19.789319+0000","last_fullsized":"2024-02-18T00:43:19.789319+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"240.4","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.793282+0000","last_change":"2024-02-18T00:43:18.789518+0000","last_active":"2024-02-18T00:43:18.793282+0000","last_peered":"2024-02-18T00:43:18.793282+0000","last_clean":"2024-02-18T00:43:18.793282+0000","last_became_active":"2024-02-18T00:43:18.789231+0000","last_became_peered":"2024-02-18T00:43:18.789231+0000","last_unstale":"2024-02-18T00:43:18.793282+0000","last_undegraded":"2024-02-18T00:43:18.793282+0000","last_fullsized":"2024-02-18T00:43:18.793282+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"240.3","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.789779+0000","last_change":"2024-02-18T00:43:18.788294+0000","last_active":"2024-02-18T00:43:18.789779+0000","last_peered":"2024-02-18T00:43:18.789779+0000","last_clean":"2024-02-18T00:43:18.789779+0000","last_became_active":"2024-02-18T00:43:18.788004+0000","last_became_peered":"2024-02-18T00:43:18.788004+0000","last_unstale":"2024-02-18T00:43:18.789779+0000","last_undegraded":"2024-02-18T00:43:18.789779+0000","last_fullsized":"2024-02-18T00:43:18.789779+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"240.2","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789497+0000","last_change":"2024-02-18T00:43:18.791774+0000","last_active":"2024-02-18T00:43:19.789497+0000","last_peered":"2024-02-18T00:43:19.789497+0000","last_clean":"2024-02-18T00:43:19.789497+0000","last_became_active":"2024-02-18T00:43:18.791496+0000","last_became_peered":"2024-02-18T00:43:18.791496+0000","last_unstale":"2024-02-18T00:43:19.789497+0000","last_undegraded":"2024-02-18T00:43:19.789497+0000","last_fullsized":"2024-02-18T00:43:19.789497+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,6],"acting":[2,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"240.1","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.792351+0000","last_change":"2024-02-18T00:43:18.791476+0000","last_active":"2024-02-18T00:43:18.792351+0000","last_peered":"2024-02-18T00:43:18.792351+0000","last_clean":"2024-02-18T00:43:18.792351+0000","last_became_active":"2024-02-18T00:43:18.791169+0000","last_became_peered":"2024-02-18T00:43:18.791169+0000","last_unstale":"2024-02-18T00:43:18.792351+0000","last_undegraded":"2024-02-18T00:43:18.792351+0000","last_fullsized":"2024-02-18T00:43:18.792351+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"240.0","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790317+0000","last_change":"2024-02-18T00:43:18.792093+0000","last_active":"2024-02-18T00:43:19.790317+0000","last_peered":"2024-02-18T00:43:19.790317+0000","last_clean":"2024-02-18T00:43:19.790317+0000","last_became_active":"2024-02-18T00:43:18.791766+0000","last_became_peered":"2024-02-18T00:43:18.791766+0000","last_unstale":"2024-02-18T00:43:19.790317+0000","last_undegraded":"2024-02-18T00:43:19.790317+0000","last_fullsized":"2024-02-18T00:43:19.790317+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,5],"acting":[1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"457'30","reported_seq":522,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777048+0000","last_change":"2024-02-18T00:43:09.813056+0000","last_active":"2024-02-18T00:43:18.777048+0000","last_peered":"2024-02-18T00:43:18.777048+0000","last_clean":"2024-02-18T00:43:18.777048+0000","last_became_active":"2024-02-18T00:43:09.798744+0000","last_became_peered":"2024-02-18T00:43:09.798744+0000","last_unstale":"2024-02-18T00:43:18.777048+0000","last_undegraded":"2024-02-18T00:43:18.777048+0000","last_fullsized":"2024-02-18T00:43:18.777048+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:42:51.380344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:51.380344+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"457'39","reported_seq":565,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.780009+0000","last_change":"2024-02-18T00:43:09.805778+0000","last_active":"2024-02-18T00:43:18.780009+0000","last_peered":"2024-02-18T00:43:18.780009+0000","last_clean":"2024-02-18T00:43:18.780009+0000","last_became_active":"2024-02-18T00:43:09.791383+0000","last_became_peered":"2024-02-18T00:43:09.791383+0000","last_unstale":"2024-02-18T00:43:18.780009+0000","last_undegraded":"2024-02-18T00:43:18.780009+0000","last_fullsized":"2024-02-18T00:43:18.780009+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:40:40.557544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.557544+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":29,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":165,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"240.1d","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788350+0000","last_change":"2024-02-18T00:43:18.793952+0000","last_active":"2024-02-18T00:43:19.788350+0000","last_peered":"2024-02-18T00:43:19.788350+0000","last_clean":"2024-02-18T00:43:19.788350+0000","last_became_active":"2024-02-18T00:43:18.793643+0000","last_became_peered":"2024-02-18T00:43:18.793643+0000","last_unstale":"2024-02-18T00:43:19.788350+0000","last_undegraded":"2024-02-18T00:43:19.788350+0000","last_fullsized":"2024-02-18T00:43:19.788350+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.7","version":"0'0","reported_seq":65,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.791738+0000","last_change":"2024-02-18T00:43:05.781481+0000","last_active":"2024-02-18T00:43:18.791738+0000","last_peered":"2024-02-18T00:43:18.791738+0000","last_clean":"2024-02-18T00:43:18.791738+0000","last_became_active":"2024-02-18T00:42:19.767995+0000","last_became_peered":"2024-02-18T00:42:19.767995+0000","last_unstale":"2024-02-18T00:43:18.791738+0000","last_undegraded":"2024-02-18T00:43:18.791738+0000","last_fullsized":"2024-02-18T00:43:18.791738+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"240.1c","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790715+0000","last_change":"2024-02-18T00:43:18.788412+0000","last_active":"2024-02-18T00:43:18.790715+0000","last_peered":"2024-02-18T00:43:18.790715+0000","last_clean":"2024-02-18T00:43:18.790715+0000","last_became_active":"2024-02-18T00:43:18.788128+0000","last_became_peered":"2024-02-18T00:43:18.788128+0000","last_unstale":"2024-02-18T00:43:18.790715+0000","last_undegraded":"2024-02-18T00:43:18.790715+0000","last_fullsized":"2024-02-18T00:43:18.790715+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.6","version":"0'0","reported_seq":65,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.782493+0000","last_change":"2024-02-18T00:43:05.796553+0000","last_active":"2024-02-18T00:43:18.782493+0000","last_peered":"2024-02-18T00:43:18.782493+0000","last_clean":"2024-02-18T00:43:18.782493+0000","last_became_active":"2024-02-18T00:42:19.766349+0000","last_became_peered":"2024-02-18T00:42:19.766349+0000","last_unstale":"2024-02-18T00:43:18.782493+0000","last_undegraded":"2024-02-18T00:43:18.782493+0000","last_fullsized":"2024-02-18T00:43:18.782493+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"240.1f","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791318+0000","last_change":"2024-02-18T00:43:18.792226+0000","last_active":"2024-02-18T00:43:19.791318+0000","last_peered":"2024-02-18T00:43:19.791318+0000","last_clean":"2024-02-18T00:43:19.791318+0000","last_became_active":"2024-02-18T00:43:18.791912+0000","last_became_peered":"2024-02-18T00:43:18.791912+0000","last_unstale":"2024-02-18T00:43:19.791318+0000","last_undegraded":"2024-02-18T00:43:19.791318+0000","last_fullsized":"2024-02-18T00:43:19.791318+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.5","version":"653'8","reported_seq":79,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789080+0000","last_change":"2024-02-18T00:43:07.787470+0000","last_active":"2024-02-18T00:43:19.789080+0000","last_peered":"2024-02-18T00:43:19.789080+0000","last_clean":"2024-02-18T00:43:19.789080+0000","last_became_active":"2024-02-18T00:42:19.763983+0000","last_became_peered":"2024-02-18T00:42:19.763983+0000","last_unstale":"2024-02-18T00:43:19.789080+0000","last_undegraded":"2024-02-18T00:43:19.789080+0000","last_fullsized":"2024-02-18T00:43:19.789080+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"240.1e","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.787896+0000","last_change":"2024-02-18T00:43:18.787373+0000","last_active":"2024-02-18T00:43:18.787896+0000","last_peered":"2024-02-18T00:43:18.787896+0000","last_clean":"2024-02-18T00:43:18.787896+0000","last_became_active":"2024-02-18T00:43:18.787059+0000","last_became_peered":"2024-02-18T00:43:18.787059+0000","last_unstale":"2024-02-18T00:43:18.787896+0000","last_undegraded":"2024-02-18T00:43:18.787896+0000","last_fullsized":"2024-02-18T00:43:18.787896+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.4","version":"0'0","reported_seq":66,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775131+0000","last_change":"2024-02-18T00:43:05.776564+0000","last_active":"2024-02-18T00:43:18.775131+0000","last_peered":"2024-02-18T00:43:18.775131+0000","last_clean":"2024-02-18T00:43:18.775131+0000","last_became_active":"2024-02-18T00:42:19.767925+0000","last_became_peered":"2024-02-18T00:42:19.767925+0000","last_unstale":"2024-02-18T00:43:18.775131+0000","last_undegraded":"2024-02-18T00:43:18.775131+0000","last_fullsized":"2024-02-18T00:43:18.775131+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"240.19","version":"0'0","reported_seq":11,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790103+0000","last_change":"2024-02-18T00:43:18.792534+0000","last_active":"2024-02-18T00:43:19.790103+0000","last_peered":"2024-02-18T00:43:19.790103+0000","last_clean":"2024-02-18T00:43:19.790103+0000","last_became_active":"2024-02-18T00:43:18.791700+0000","last_became_peered":"2024-02-18T00:43:18.791700+0000","last_unstale":"2024-02-18T00:43:19.790103+0000","last_undegraded":"2024-02-18T00:43:19.790103+0000","last_fullsized":"2024-02-18T00:43:19.790103+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.3","version":"0'0","reported_seq":67,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776024+0000","last_change":"2024-02-18T00:43:05.788964+0000","last_active":"2024-02-18T00:43:18.776024+0000","last_peered":"2024-02-18T00:43:18.776024+0000","last_clean":"2024-02-18T00:43:18.776024+0000","last_became_active":"2024-02-18T00:42:19.767739+0000","last_became_peered":"2024-02-18T00:42:19.767739+0000","last_unstale":"2024-02-18T00:43:18.776024+0000","last_undegraded":"2024-02-18T00:43:18.776024+0000","last_fullsized":"2024-02-18T00:43:18.776024+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"240.18","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.789193+0000","last_change":"2024-02-18T00:43:18.785603+0000","last_active":"2024-02-18T00:43:18.789193+0000","last_peered":"2024-02-18T00:43:18.789193+0000","last_clean":"2024-02-18T00:43:18.789193+0000","last_became_active":"2024-02-18T00:43:18.785185+0000","last_became_peered":"2024-02-18T00:43:18.785185+0000","last_unstale":"2024-02-18T00:43:18.789193+0000","last_undegraded":"2024-02-18T00:43:18.789193+0000","last_fullsized":"2024-02-18T00:43:18.789193+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"234.2","version":"630'11","reported_seq":80,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.784181+0000","last_change":"2024-02-18T00:43:05.796153+0000","last_active":"2024-02-18T00:43:18.784181+0000","last_peered":"2024-02-18T00:43:18.784181+0000","last_clean":"2024-02-18T00:43:18.784181+0000","last_became_active":"2024-02-18T00:42:19.763974+0000","last_became_peered":"2024-02-18T00:42:19.763974+0000","last_unstale":"2024-02-18T00:43:18.784181+0000","last_undegraded":"2024-02-18T00:43:18.784181+0000","last_fullsized":"2024-02-18T00:43:18.784181+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"240.1b","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.792812+0000","last_change":"2024-02-18T00:43:18.790505+0000","last_active":"2024-02-18T00:43:18.792812+0000","last_peered":"2024-02-18T00:43:18.792812+0000","last_clean":"2024-02-18T00:43:18.792812+0000","last_became_active":"2024-02-18T00:43:18.790172+0000","last_became_peered":"2024-02-18T00:43:18.790172+0000","last_unstale":"2024-02-18T00:43:18.792812+0000","last_undegraded":"2024-02-18T00:43:18.792812+0000","last_fullsized":"2024-02-18T00:43:18.792812+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"234.1","version":"607'1","reported_seq":68,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777143+0000","last_change":"2024-02-18T00:43:05.775949+0000","last_active":"2024-02-18T00:43:18.777143+0000","last_peered":"2024-02-18T00:43:18.777143+0000","last_clean":"2024-02-18T00:43:18.777143+0000","last_became_active":"2024-02-18T00:42:19.763959+0000","last_became_peered":"2024-02-18T00:42:19.763959+0000","last_unstale":"2024-02-18T00:43:18.777143+0000","last_undegraded":"2024-02-18T00:43:18.777143+0000","last_fullsized":"2024-02-18T00:43:18.777143+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"240.1a","version":"0'0","reported_seq":10,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.795326+0000","last_change":"2024-02-18T00:43:18.794612+0000","last_active":"2024-02-18T00:43:18.795326+0000","last_peered":"2024-02-18T00:43:18.795326+0000","last_clean":"2024-02-18T00:43:18.795326+0000","last_became_active":"2024-02-18T00:43:18.794407+0000","last_became_peered":"2024-02-18T00:43:18.794407+0000","last_unstale":"2024-02-18T00:43:18.795326+0000","last_undegraded":"2024-02-18T00:43:18.795326+0000","last_fullsized":"2024-02-18T00:43:18.795326+0000","mapping_epoch":652,"log_start":"0'0","ondisk_log_start":"0'0","created":652,"last_epoch_clean":653,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:17.793300+0000","last_clean_scrub_stamp":"2024-02-18T00:43:17.793300+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.0","version":"630'10","reported_seq":80,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787507+0000","last_change":"2024-02-18T00:43:05.775601+0000","last_active":"2024-02-18T00:43:19.787507+0000","last_peered":"2024-02-18T00:43:19.787507+0000","last_clean":"2024-02-18T00:43:19.787507+0000","last_became_active":"2024-02-18T00:42:19.767896+0000","last_became_peered":"2024-02-18T00:42:19.767896+0000","last_unstale":"2024-02-18T00:43:19.787507+0000","last_undegraded":"2024-02-18T00:43:19.787507+0000","last_fullsized":"2024-02-18T00:43:19.787507+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":10,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":583,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.783138+0000","last_change":"2024-02-18T00:41:36.776506+0000","last_active":"2024-02-18T00:43:18.783138+0000","last_peered":"2024-02-18T00:43:18.783138+0000","last_clean":"2024-02-18T00:43:18.783138+0000","last_became_active":"2024-02-18T00:39:36.751760+0000","last_became_peered":"2024-02-18T00:39:36.751760+0000","last_unstale":"2024-02-18T00:43:18.783138+0000","last_undegraded":"2024-02-18T00:43:18.783138+0000","last_fullsized":"2024-02-18T00:43:18.783138+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:36.776429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.776429+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":576,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.774928+0000","last_change":"2024-02-18T00:40:41.127499+0000","last_active":"2024-02-18T00:43:18.774928+0000","last_peered":"2024-02-18T00:43:18.774928+0000","last_clean":"2024-02-18T00:43:18.774928+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:43:18.774928+0000","last_undegraded":"2024-02-18T00:43:18.774928+0000","last_fullsized":"2024-02-18T00:43:18.774928+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:41.127432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:40:41.127432+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.21","version":"457'34","reported_seq":39,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776958+0000","last_change":"2024-02-18T00:43:09.814460+0000","last_active":"2024-02-18T00:43:18.776958+0000","last_peered":"2024-02-18T00:43:18.776958+0000","last_clean":"2024-02-18T00:43:18.776958+0000","last_became_active":"2024-02-18T00:43:09.798147+0000","last_became_peered":"2024-02-18T00:43:09.798147+0000","last_unstale":"2024-02-18T00:43:18.776958+0000","last_undegraded":"2024-02-18T00:43:18.776958+0000","last_fullsized":"2024-02-18T00:43:18.776958+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:42:06.374780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.374780+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":23,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.792279+0000","last_change":"2024-02-18T00:42:55.766143+0000","last_active":"2024-02-18T00:43:18.792279+0000","last_peered":"2024-02-18T00:43:18.792279+0000","last_clean":"2024-02-18T00:43:18.792279+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:43:18.792279+0000","last_undegraded":"2024-02-18T00:43:18.792279+0000","last_fullsized":"2024-02-18T00:43:18.792279+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:55.766062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:42:55.766062+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.20","version":"457'52","reported_seq":33,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790465+0000","last_change":"2024-02-18T00:43:09.811514+0000","last_active":"2024-02-18T00:43:18.790465+0000","last_peered":"2024-02-18T00:43:18.790465+0000","last_clean":"2024-02-18T00:43:18.790465+0000","last_became_active":"2024-02-18T00:43:09.796538+0000","last_became_peered":"2024-02-18T00:43:09.796538+0000","last_unstale":"2024-02-18T00:43:18.790465+0000","last_undegraded":"2024-02-18T00:43:18.790465+0000","last_fullsized":"2024-02-18T00:43:18.790465+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:43:04.185549+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.185549+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790486+0000","last_change":"2024-02-18T00:43:10.875551+0000","last_active":"2024-02-18T00:43:18.790486+0000","last_peered":"2024-02-18T00:43:18.790486+0000","last_clean":"2024-02-18T00:43:18.790486+0000","last_became_active":"2024-02-18T00:39:36.760004+0000","last_became_peered":"2024-02-18T00:39:36.760004+0000","last_unstale":"2024-02-18T00:43:18.790486+0000","last_undegraded":"2024-02-18T00:43:18.790486+0000","last_fullsized":"2024-02-18T00:43:18.790486+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:10.875479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:43:10.875479+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.23","version":"457'35","reported_seq":31,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776262+0000","last_change":"2024-02-18T00:43:09.815162+0000","last_active":"2024-02-18T00:43:18.776262+0000","last_peered":"2024-02-18T00:43:18.776262+0000","last_clean":"2024-02-18T00:43:18.776262+0000","last_became_active":"2024-02-18T00:43:09.796063+0000","last_became_peered":"2024-02-18T00:43:09.796063+0000","last_unstale":"2024-02-18T00:43:18.776262+0000","last_undegraded":"2024-02-18T00:43:18.776262+0000","last_fullsized":"2024-02-18T00:43:18.776262+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:43:03.511887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.511887+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":585,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788582+0000","last_change":"2024-02-18T00:41:37.067790+0000","last_active":"2024-02-18T00:43:19.788582+0000","last_peered":"2024-02-18T00:43:19.788582+0000","last_clean":"2024-02-18T00:43:19.788582+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:43:19.788582+0000","last_undegraded":"2024-02-18T00:43:19.788582+0000","last_fullsized":"2024-02-18T00:43:19.788582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:41:37.067689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:37.067689+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.22","version":"457'32","reported_seq":476,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787682+0000","last_change":"2024-02-18T00:43:09.793648+0000","last_active":"2024-02-18T00:43:19.787682+0000","last_peered":"2024-02-18T00:43:19.787682+0000","last_clean":"2024-02-18T00:43:19.787682+0000","last_became_active":"2024-02-18T00:43:09.785609+0000","last_became_peered":"2024-02-18T00:43:09.785609+0000","last_unstale":"2024-02-18T00:43:19.787682+0000","last_undegraded":"2024-02-18T00:43:19.787682+0000","last_fullsized":"2024-02-18T00:43:19.787682+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:41:36.069252+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.069252+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":18,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":585,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.781923+0000","last_change":"2024-02-18T00:43:08.748536+0000","last_active":"2024-02-18T00:43:18.781923+0000","last_peered":"2024-02-18T00:43:18.781923+0000","last_clean":"2024-02-18T00:43:18.781923+0000","last_became_active":"2024-02-18T00:39:36.751486+0000","last_became_peered":"2024-02-18T00:39:36.751486+0000","last_unstale":"2024-02-18T00:43:18.781923+0000","last_undegraded":"2024-02-18T00:43:18.781923+0000","last_fullsized":"2024-02-18T00:43:18.781923+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.741618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.741618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.25","version":"457'26","reported_seq":34,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776570+0000","last_change":"2024-02-18T00:43:10.481144+0000","last_active":"2024-02-18T00:43:18.776570+0000","last_peered":"2024-02-18T00:43:18.776570+0000","last_clean":"2024-02-18T00:43:18.776570+0000","last_became_active":"2024-02-18T00:43:09.791838+0000","last_became_peered":"2024-02-18T00:43:09.791838+0000","last_unstale":"2024-02-18T00:43:18.776570+0000","last_undegraded":"2024-02-18T00:43:18.776570+0000","last_fullsized":"2024-02-18T00:43:18.776570+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:42:12.900647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:12.900647+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":16,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":77,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":563,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776339+0000","last_change":"2024-02-18T00:42:22.876124+0000","last_active":"2024-02-18T00:43:18.776339+0000","last_peered":"2024-02-18T00:43:18.776339+0000","last_clean":"2024-02-18T00:43:18.776339+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:43:18.776339+0000","last_undegraded":"2024-02-18T00:43:18.776339+0000","last_fullsized":"2024-02-18T00:43:18.776339+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.876055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.876055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"457'47","reported_seq":73,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790820+0000","last_change":"2024-02-18T00:43:10.692823+0000","last_active":"2024-02-18T00:43:19.790820+0000","last_peered":"2024-02-18T00:43:19.790820+0000","last_clean":"2024-02-18T00:43:19.790820+0000","last_became_active":"2024-02-18T00:43:09.789985+0000","last_became_peered":"2024-02-18T00:43:09.789985+0000","last_unstale":"2024-02-18T00:43:19.790820+0000","last_undegraded":"2024-02-18T00:43:19.790820+0000","last_fullsized":"2024-02-18T00:43:19.790820+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:42:13.680322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.680322+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":179,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":289,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":572,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.784486+0000","last_change":"2024-02-18T00:42:58.841398+0000","last_active":"2024-02-18T00:43:18.784486+0000","last_peered":"2024-02-18T00:43:18.784486+0000","last_clean":"2024-02-18T00:43:18.784486+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:43:18.784486+0000","last_undegraded":"2024-02-18T00:43:18.784486+0000","last_fullsized":"2024-02-18T00:43:18.784486+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:58.841303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:42:58.841303+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"457'23","reported_seq":709,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791303+0000","last_change":"2024-02-18T00:43:09.801999+0000","last_active":"2024-02-18T00:43:19.791303+0000","last_peered":"2024-02-18T00:43:19.791303+0000","last_clean":"2024-02-18T00:43:19.791303+0000","last_became_active":"2024-02-18T00:43:09.788606+0000","last_became_peered":"2024-02-18T00:43:09.788606+0000","last_unstale":"2024-02-18T00:43:19.791303+0000","last_undegraded":"2024-02-18T00:43:19.791303+0000","last_fullsized":"2024-02-18T00:43:19.791303+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:42:10.770922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:10.770922+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":13,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":662,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789201+0000","last_change":"2024-02-18T00:42:25.770240+0000","last_active":"2024-02-18T00:43:19.789201+0000","last_peered":"2024-02-18T00:43:19.789201+0000","last_clean":"2024-02-18T00:43:19.789201+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:43:19.789201+0000","last_undegraded":"2024-02-18T00:43:19.789201+0000","last_fullsized":"2024-02-18T00:43:19.789201+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.770128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"457'24","reported_seq":489,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791241+0000","last_change":"2024-02-18T00:43:09.804299+0000","last_active":"2024-02-18T00:43:19.791241+0000","last_peered":"2024-02-18T00:43:19.791241+0000","last_clean":"2024-02-18T00:43:19.791241+0000","last_became_active":"2024-02-18T00:43:09.796496+0000","last_became_peered":"2024-02-18T00:43:09.796496+0000","last_unstale":"2024-02-18T00:43:19.791241+0000","last_undegraded":"2024-02-18T00:43:19.791241+0000","last_fullsized":"2024-02-18T00:43:19.791241+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:42:59.467711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.467711+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.780411+0000","last_change":"2024-02-18T00:42:49.664459+0000","last_active":"2024-02-18T00:43:18.780411+0000","last_peered":"2024-02-18T00:43:18.780411+0000","last_clean":"2024-02-18T00:43:18.780411+0000","last_became_active":"2024-02-18T00:39:36.769282+0000","last_became_peered":"2024-02-18T00:39:36.769282+0000","last_unstale":"2024-02-18T00:43:18.780411+0000","last_undegraded":"2024-02-18T00:43:18.780411+0000","last_fullsized":"2024-02-18T00:43:18.780411+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.664380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.664380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":591,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776828+0000","last_change":"2024-02-18T00:43:11.626087+0000","last_active":"2024-02-18T00:43:18.776828+0000","last_peered":"2024-02-18T00:43:18.776828+0000","last_clean":"2024-02-18T00:43:18.776828+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:43:18.776828+0000","last_undegraded":"2024-02-18T00:43:18.776828+0000","last_fullsized":"2024-02-18T00:43:18.776828+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_clean_scrub_stamp":"2024-02-18T00:43:11.626025+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"466'10","reported_seq":250,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.788856+0000","last_change":"2024-02-18T00:42:16.577349+0000","last_active":"2024-02-18T00:43:18.788856+0000","last_peered":"2024-02-18T00:43:18.788856+0000","last_clean":"2024-02-18T00:43:18.788856+0000","last_became_active":"2024-02-18T00:39:36.757493+0000","last_became_peered":"2024-02-18T00:39:36.757493+0000","last_unstale":"2024-02-18T00:43:18.788856+0000","last_undegraded":"2024-02-18T00:43:18.788856+0000","last_fullsized":"2024-02-18T00:43:18.788856+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:42:16.577265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:16.577265+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":762,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776473+0000","last_change":"2024-02-18T00:43:07.792420+0000","last_active":"2024-02-18T00:43:18.776473+0000","last_peered":"2024-02-18T00:43:18.776473+0000","last_clean":"2024-02-18T00:43:18.776473+0000","last_became_active":"2024-02-18T00:43:07.792054+0000","last_became_peered":"2024-02-18T00:43:07.792054+0000","last_unstale":"2024-02-18T00:43:18.776473+0000","last_undegraded":"2024-02-18T00:43:18.776473+0000","last_fullsized":"2024-02-18T00:43:18.776473+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:40:40.557544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:40.557544+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":655,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789772+0000","last_change":"2024-02-18T00:42:55.614402+0000","last_active":"2024-02-18T00:43:19.789772+0000","last_peered":"2024-02-18T00:43:19.789772+0000","last_clean":"2024-02-18T00:43:19.789772+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:43:19.789772+0000","last_undegraded":"2024-02-18T00:43:19.789772+0000","last_fullsized":"2024-02-18T00:43:19.789772+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:55.614294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:55.614294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":640,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776303+0000","last_change":"2024-02-18T00:42:15.647876+0000","last_active":"2024-02-18T00:43:18.776303+0000","last_peered":"2024-02-18T00:43:18.776303+0000","last_clean":"2024-02-18T00:43:18.776303+0000","last_became_active":"2024-02-18T00:41:45.788049+0000","last_became_peered":"2024-02-18T00:41:45.788049+0000","last_unstale":"2024-02-18T00:43:18.776303+0000","last_undegraded":"2024-02-18T00:43:18.776303+0000","last_fullsized":"2024-02-18T00:43:18.776303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":572,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:15.647808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:15.647808+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":579,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791347+0000","last_change":"2024-02-18T00:42:54.180923+0000","last_active":"2024-02-18T00:43:19.791347+0000","last_peered":"2024-02-18T00:43:19.791347+0000","last_clean":"2024-02-18T00:43:19.791347+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:43:19.791347+0000","last_undegraded":"2024-02-18T00:43:19.791347+0000","last_fullsized":"2024-02-18T00:43:19.791347+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:54.180851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:54.180851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":689,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775680+0000","last_change":"2024-02-18T00:43:04.471487+0000","last_active":"2024-02-18T00:43:18.775680+0000","last_peered":"2024-02-18T00:43:18.775680+0000","last_clean":"2024-02-18T00:43:18.775680+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:43:18.775680+0000","last_undegraded":"2024-02-18T00:43:18.775680+0000","last_fullsized":"2024-02-18T00:43:18.775680+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:43:04.471412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.471412+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":637,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791231+0000","last_change":"2024-02-18T00:41:23.794790+0000","last_active":"2024-02-18T00:43:19.791231+0000","last_peered":"2024-02-18T00:43:19.791231+0000","last_clean":"2024-02-18T00:43:19.791231+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:43:19.791231+0000","last_undegraded":"2024-02-18T00:43:19.791231+0000","last_fullsized":"2024-02-18T00:43:19.791231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.794682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.794682+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":631,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789642+0000","last_change":"2024-02-18T00:43:02.696466+0000","last_active":"2024-02-18T00:43:19.789642+0000","last_peered":"2024-02-18T00:43:19.789642+0000","last_clean":"2024-02-18T00:43:19.789642+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:43:19.789642+0000","last_undegraded":"2024-02-18T00:43:19.789642+0000","last_fullsized":"2024-02-18T00:43:19.789642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:02.696399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.696399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":664,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776735+0000","last_change":"2024-02-18T00:41:49.824585+0000","last_active":"2024-02-18T00:43:18.776735+0000","last_peered":"2024-02-18T00:43:18.776735+0000","last_clean":"2024-02-18T00:43:18.776735+0000","last_became_active":"2024-02-18T00:41:49.824190+0000","last_became_peered":"2024-02-18T00:41:49.824190+0000","last_unstale":"2024-02-18T00:43:18.776735+0000","last_undegraded":"2024-02-18T00:43:18.776735+0000","last_fullsized":"2024-02-18T00:43:18.776735+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:16.155189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:16.155189+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":578,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.779046+0000","last_change":"2024-02-18T00:41:36.841374+0000","last_active":"2024-02-18T00:43:18.779046+0000","last_peered":"2024-02-18T00:43:18.779046+0000","last_clean":"2024-02-18T00:43:18.779046+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:43:18.779046+0000","last_undegraded":"2024-02-18T00:43:18.779046+0000","last_fullsized":"2024-02-18T00:43:18.779046+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.841306+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"457'42","reported_seq":763,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.784539+0000","last_change":"2024-02-18T00:42:54.825530+0000","last_active":"2024-02-18T00:43:18.784539+0000","last_peered":"2024-02-18T00:43:18.784539+0000","last_clean":"2024-02-18T00:43:18.784539+0000","last_became_active":"2024-02-18T00:39:36.751206+0000","last_became_peered":"2024-02-18T00:39:36.751206+0000","last_unstale":"2024-02-18T00:43:18.784539+0000","last_undegraded":"2024-02-18T00:43:18.784539+0000","last_fullsized":"2024-02-18T00:43:18.784539+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-18T00:42:54.825413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:54.825413+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":635,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789656+0000","last_change":"2024-02-18T00:42:53.601598+0000","last_active":"2024-02-18T00:43:19.789656+0000","last_peered":"2024-02-18T00:43:19.789656+0000","last_clean":"2024-02-18T00:43:19.789656+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:43:19.789656+0000","last_undegraded":"2024-02-18T00:43:19.789656+0000","last_fullsized":"2024-02-18T00:43:19.789656+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:53.601533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:42:53.601533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":631,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790514+0000","last_change":"2024-02-18T00:43:03.732009+0000","last_active":"2024-02-18T00:43:19.790514+0000","last_peered":"2024-02-18T00:43:19.790514+0000","last_clean":"2024-02-18T00:43:19.790514+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:43:19.790514+0000","last_undegraded":"2024-02-18T00:43:19.790514+0000","last_fullsized":"2024-02-18T00:43:19.790514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.731917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.731917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":219,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.780545+0000","last_change":"2024-02-18T00:42:20.638644+0000","last_active":"2024-02-18T00:43:18.780545+0000","last_peered":"2024-02-18T00:43:18.780545+0000","last_clean":"2024-02-18T00:43:18.780545+0000","last_became_active":"2024-02-18T00:41:53.837106+0000","last_became_peered":"2024-02-18T00:41:53.837106+0000","last_unstale":"2024-02-18T00:43:18.780545+0000","last_undegraded":"2024-02-18T00:43:18.780545+0000","last_fullsized":"2024-02-18T00:43:18.780545+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:20.638566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:20.638566+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":563,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776700+0000","last_change":"2024-02-18T00:41:23.575091+0000","last_active":"2024-02-18T00:43:18.776700+0000","last_peered":"2024-02-18T00:43:18.776700+0000","last_clean":"2024-02-18T00:43:18.776700+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:43:18.776700+0000","last_undegraded":"2024-02-18T00:43:18.776700+0000","last_fullsized":"2024-02-18T00:43:18.776700+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:23.574969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:23.574969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":694,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788504+0000","last_change":"2024-02-18T00:43:12.336661+0000","last_active":"2024-02-18T00:43:19.788504+0000","last_peered":"2024-02-18T00:43:19.788504+0000","last_clean":"2024-02-18T00:43:19.788504+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:43:19.788504+0000","last_undegraded":"2024-02-18T00:43:19.788504+0000","last_fullsized":"2024-02-18T00:43:19.788504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:43:12.336565+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.336565+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":196,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.789531+0000","last_change":"2024-02-18T00:41:34.509265+0000","last_active":"2024-02-18T00:43:18.789531+0000","last_peered":"2024-02-18T00:43:18.789531+0000","last_clean":"2024-02-18T00:43:18.789531+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:43:18.789531+0000","last_undegraded":"2024-02-18T00:43:18.789531+0000","last_fullsized":"2024-02-18T00:43:18.789531+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.509185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.509185+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":633,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790992+0000","last_change":"2024-02-18T00:43:03.272338+0000","last_active":"2024-02-18T00:43:19.790992+0000","last_peered":"2024-02-18T00:43:19.790992+0000","last_clean":"2024-02-18T00:43:19.790992+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:43:19.790992+0000","last_undegraded":"2024-02-18T00:43:19.790992+0000","last_fullsized":"2024-02-18T00:43:19.790992+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.272278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.272278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":642,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790171+0000","last_change":"2024-02-18T00:41:25.756779+0000","last_active":"2024-02-18T00:43:19.790171+0000","last_peered":"2024-02-18T00:43:19.790171+0000","last_clean":"2024-02-18T00:43:19.790171+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:43:19.790171+0000","last_undegraded":"2024-02-18T00:43:19.790171+0000","last_fullsized":"2024-02-18T00:43:19.790171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:18.151671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.151671+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":40,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776102+0000","last_change":"2024-02-18T00:42:38.553481+0000","last_active":"2024-02-18T00:43:18.776102+0000","last_peered":"2024-02-18T00:43:18.776102+0000","last_clean":"2024-02-18T00:43:18.776102+0000","last_became_active":"2024-02-18T00:42:38.553174+0000","last_became_peered":"2024-02-18T00:42:38.553174+0000","last_unstale":"2024-02-18T00:43:18.776102+0000","last_undegraded":"2024-02-18T00:43:18.776102+0000","last_fullsized":"2024-02-18T00:43:18.776102+0000","mapping_epoch":623,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:14.739594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:14.739594+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":660,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788096+0000","last_change":"2024-02-18T00:42:49.351292+0000","last_active":"2024-02-18T00:43:19.788096+0000","last_peered":"2024-02-18T00:43:19.788096+0000","last_clean":"2024-02-18T00:43:19.788096+0000","last_became_active":"2024-02-18T00:29:17.085798+0000","last_became_peered":"2024-02-18T00:29:17.085798+0000","last_unstale":"2024-02-18T00:43:19.788096+0000","last_undegraded":"2024-02-18T00:43:19.788096+0000","last_fullsized":"2024-02-18T00:43:19.788096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:42:49.351183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.351183+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":657,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791815+0000","last_change":"2024-02-18T00:42:05.815709+0000","last_active":"2024-02-18T00:43:19.791815+0000","last_peered":"2024-02-18T00:43:19.791815+0000","last_clean":"2024-02-18T00:43:19.791815+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:43:19.791815+0000","last_undegraded":"2024-02-18T00:43:19.791815+0000","last_fullsized":"2024-02-18T00:43:19.791815+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:05.815649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:42:05.815649+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":834,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789380+0000","last_change":"2024-02-18T00:43:09.903967+0000","last_active":"2024-02-18T00:43:19.789380+0000","last_peered":"2024-02-18T00:43:19.789380+0000","last_clean":"2024-02-18T00:43:19.789380+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:43:19.789380+0000","last_undegraded":"2024-02-18T00:43:19.789380+0000","last_fullsized":"2024-02-18T00:43:19.789380+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:41:22.121879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:22.121879+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":644,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788584+0000","last_change":"2024-02-18T00:42:52.422173+0000","last_active":"2024-02-18T00:43:19.788584+0000","last_peered":"2024-02-18T00:43:19.788584+0000","last_clean":"2024-02-18T00:43:19.788584+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:43:19.788584+0000","last_undegraded":"2024-02-18T00:43:19.788584+0000","last_fullsized":"2024-02-18T00:43:19.788584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:52.422086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:52.422086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":577,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791966+0000","last_change":"2024-02-18T00:42:52.127903+0000","last_active":"2024-02-18T00:43:19.791966+0000","last_peered":"2024-02-18T00:43:19.791966+0000","last_clean":"2024-02-18T00:43:19.791966+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:43:19.791966+0000","last_undegraded":"2024-02-18T00:43:19.791966+0000","last_fullsized":"2024-02-18T00:43:19.791966+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:52.127829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:52.127829+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":652,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777197+0000","last_change":"2024-02-18T00:43:12.764242+0000","last_active":"2024-02-18T00:43:18.777197+0000","last_peered":"2024-02-18T00:43:18.777197+0000","last_clean":"2024-02-18T00:43:18.777197+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:43:18.777197+0000","last_undegraded":"2024-02-18T00:43:18.777197+0000","last_fullsized":"2024-02-18T00:43:18.777197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:43:12.764172+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.764172+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":664,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790518+0000","last_change":"2024-02-18T00:42:08.816658+0000","last_active":"2024-02-18T00:43:19.790518+0000","last_peered":"2024-02-18T00:43:19.790518+0000","last_clean":"2024-02-18T00:43:19.790518+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:43:19.790518+0000","last_undegraded":"2024-02-18T00:43:19.790518+0000","last_fullsized":"2024-02-18T00:43:19.790518+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:08.816534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:42:08.816534+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":805,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788556+0000","last_change":"2024-02-18T00:42:56.472380+0000","last_active":"2024-02-18T00:43:19.788556+0000","last_peered":"2024-02-18T00:43:19.788556+0000","last_clean":"2024-02-18T00:43:19.788556+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:43:19.788556+0000","last_undegraded":"2024-02-18T00:43:19.788556+0000","last_fullsized":"2024-02-18T00:43:19.788556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:42:56.472290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.472290+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":635,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777081+0000","last_change":"2024-02-18T00:41:33.828221+0000","last_active":"2024-02-18T00:43:18.777081+0000","last_peered":"2024-02-18T00:43:18.777081+0000","last_clean":"2024-02-18T00:43:18.777081+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:43:18.777081+0000","last_undegraded":"2024-02-18T00:43:18.777081+0000","last_fullsized":"2024-02-18T00:43:18.777081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.596666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.596666+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":567,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787923+0000","last_change":"2024-02-18T00:41:15.124842+0000","last_active":"2024-02-18T00:43:19.787923+0000","last_peered":"2024-02-18T00:43:19.787923+0000","last_clean":"2024-02-18T00:43:19.787923+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:43:19.787923+0000","last_undegraded":"2024-02-18T00:43:19.787923+0000","last_fullsized":"2024-02-18T00:43:19.787923+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:15.124769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.124769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'25","reported_seq":699,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788501+0000","last_change":"2024-02-18T00:41:36.043652+0000","last_active":"2024-02-18T00:43:19.788501+0000","last_peered":"2024-02-18T00:43:19.788501+0000","last_clean":"2024-02-18T00:43:19.788501+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:43:19.788501+0000","last_undegraded":"2024-02-18T00:43:19.788501+0000","last_fullsized":"2024-02-18T00:43:19.788501+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:36.043577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.043577+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":203,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789015+0000","last_change":"2024-02-18T00:42:59.673905+0000","last_active":"2024-02-18T00:43:19.789015+0000","last_peered":"2024-02-18T00:43:19.789015+0000","last_clean":"2024-02-18T00:43:19.789015+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:43:19.789015+0000","last_undegraded":"2024-02-18T00:43:19.789015+0000","last_fullsized":"2024-02-18T00:43:19.789015+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:59.673841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.673841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":624,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777331+0000","last_change":"2024-02-18T00:42:18.694442+0000","last_active":"2024-02-18T00:43:18.777331+0000","last_peered":"2024-02-18T00:43:18.777331+0000","last_clean":"2024-02-18T00:43:18.777331+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:43:18.777331+0000","last_undegraded":"2024-02-18T00:43:18.777331+0000","last_fullsized":"2024-02-18T00:43:18.777331+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.694358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.694358+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":632,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775297+0000","last_change":"2024-02-18T00:41:37.801538+0000","last_active":"2024-02-18T00:43:18.775297+0000","last_peered":"2024-02-18T00:43:18.775297+0000","last_clean":"2024-02-18T00:43:18.775297+0000","last_became_active":"2024-02-18T00:41:37.801183+0000","last_became_peered":"2024-02-18T00:41:37.801183+0000","last_unstale":"2024-02-18T00:43:18.775297+0000","last_undegraded":"2024-02-18T00:43:18.775297+0000","last_fullsized":"2024-02-18T00:43:18.775297+0000","mapping_epoch":62,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":564,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:50.016774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:50.016774+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":572,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788395+0000","last_change":"2024-02-18T00:43:00.314164+0000","last_active":"2024-02-18T00:43:19.788395+0000","last_peered":"2024-02-18T00:43:19.788395+0000","last_clean":"2024-02-18T00:43:19.788395+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:43:19.788395+0000","last_undegraded":"2024-02-18T00:43:19.788395+0000","last_fullsized":"2024-02-18T00:43:19.788395+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:00.314091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.314091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":799,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775231+0000","last_change":"2024-02-18T00:43:05.211680+0000","last_active":"2024-02-18T00:43:18.775231+0000","last_peered":"2024-02-18T00:43:18.775231+0000","last_clean":"2024-02-18T00:43:18.775231+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:43:18.775231+0000","last_undegraded":"2024-02-18T00:43:18.775231+0000","last_fullsized":"2024-02-18T00:43:18.775231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_clean_scrub_stamp":"2024-02-18T00:43:05.211604+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":697,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787716+0000","last_change":"2024-02-18T00:43:07.790591+0000","last_active":"2024-02-18T00:43:19.787716+0000","last_peered":"2024-02-18T00:43:19.787716+0000","last_clean":"2024-02-18T00:43:19.787716+0000","last_became_active":"2024-02-18T00:43:07.790268+0000","last_became_peered":"2024-02-18T00:43:07.790268+0000","last_unstale":"2024-02-18T00:43:19.787716+0000","last_undegraded":"2024-02-18T00:43:19.787716+0000","last_fullsized":"2024-02-18T00:43:19.787716+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:42:06.374780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.374780+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":23,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":659,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.783995+0000","last_change":"2024-02-18T00:42:59.867528+0000","last_active":"2024-02-18T00:43:18.783995+0000","last_peered":"2024-02-18T00:43:18.783995+0000","last_clean":"2024-02-18T00:43:18.783995+0000","last_became_active":"2024-02-18T00:39:36.752210+0000","last_became_peered":"2024-02-18T00:39:36.752210+0000","last_unstale":"2024-02-18T00:43:18.783995+0000","last_undegraded":"2024-02-18T00:43:18.783995+0000","last_fullsized":"2024-02-18T00:43:18.783995+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:59.867452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.867452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.789591+0000","last_change":"2024-02-18T00:42:15.535915+0000","last_active":"2024-02-18T00:43:18.789591+0000","last_peered":"2024-02-18T00:43:18.789591+0000","last_clean":"2024-02-18T00:43:18.789591+0000","last_became_active":"2024-02-18T00:39:36.767278+0000","last_became_peered":"2024-02-18T00:39:36.767278+0000","last_unstale":"2024-02-18T00:43:18.789591+0000","last_undegraded":"2024-02-18T00:43:18.789591+0000","last_fullsized":"2024-02-18T00:43:18.789591+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:15.535843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:15.535843+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":568,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776645+0000","last_change":"2024-02-18T00:43:08.832006+0000","last_active":"2024-02-18T00:43:18.776645+0000","last_peered":"2024-02-18T00:43:18.776645+0000","last_clean":"2024-02-18T00:43:18.776645+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:43:18.776645+0000","last_undegraded":"2024-02-18T00:43:18.776645+0000","last_fullsized":"2024-02-18T00:43:18.776645+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.831934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:08.831934+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.831934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":646,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787865+0000","last_change":"2024-02-18T00:42:10.347030+0000","last_active":"2024-02-18T00:43:19.787865+0000","last_peered":"2024-02-18T00:43:19.787865+0000","last_clean":"2024-02-18T00:43:19.787865+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:43:19.787865+0000","last_undegraded":"2024-02-18T00:43:19.787865+0000","last_fullsized":"2024-02-18T00:43:19.787865+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:10.346948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:10.346948+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":785,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775122+0000","last_change":"2024-02-18T00:43:07.799547+0000","last_active":"2024-02-18T00:43:18.775122+0000","last_peered":"2024-02-18T00:43:18.775122+0000","last_clean":"2024-02-18T00:43:18.775122+0000","last_became_active":"2024-02-18T00:43:07.799144+0000","last_became_peered":"2024-02-18T00:43:07.799144+0000","last_unstale":"2024-02-18T00:43:18.775122+0000","last_undegraded":"2024-02-18T00:43:18.775122+0000","last_fullsized":"2024-02-18T00:43:18.775122+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:43:04.185549+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.185549+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":664,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791413+0000","last_change":"2024-02-18T00:42:44.039707+0000","last_active":"2024-02-18T00:43:19.791413+0000","last_peered":"2024-02-18T00:43:19.791413+0000","last_clean":"2024-02-18T00:43:19.791413+0000","last_became_active":"2024-02-18T00:42:44.039326+0000","last_became_peered":"2024-02-18T00:42:44.039326+0000","last_unstale":"2024-02-18T00:43:19.791413+0000","last_undegraded":"2024-02-18T00:43:19.791413+0000","last_fullsized":"2024-02-18T00:43:19.791413+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.828800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.828800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":626,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788051+0000","last_change":"2024-02-18T00:42:50.371671+0000","last_active":"2024-02-18T00:43:19.788051+0000","last_peered":"2024-02-18T00:43:19.788051+0000","last_clean":"2024-02-18T00:43:19.788051+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:43:19.788051+0000","last_undegraded":"2024-02-18T00:43:19.788051+0000","last_fullsized":"2024-02-18T00:43:19.788051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:50.371592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.371592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":560,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.779394+0000","last_change":"2024-02-18T00:41:26.953890+0000","last_active":"2024-02-18T00:43:18.779394+0000","last_peered":"2024-02-18T00:43:18.779394+0000","last_clean":"2024-02-18T00:43:18.779394+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:43:18.779394+0000","last_undegraded":"2024-02-18T00:43:18.779394+0000","last_fullsized":"2024-02-18T00:43:18.779394+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:26.952674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:26.952674+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":623,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.784399+0000","last_change":"2024-02-18T00:42:22.964768+0000","last_active":"2024-02-18T00:43:18.784399+0000","last_peered":"2024-02-18T00:43:18.784399+0000","last_clean":"2024-02-18T00:43:18.784399+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:43:18.784399+0000","last_undegraded":"2024-02-18T00:43:18.784399+0000","last_fullsized":"2024-02-18T00:43:18.784399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.964687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:22.964687+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.964687+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":762,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775532+0000","last_change":"2024-02-18T00:43:07.797908+0000","last_active":"2024-02-18T00:43:18.775532+0000","last_peered":"2024-02-18T00:43:18.775532+0000","last_clean":"2024-02-18T00:43:18.775532+0000","last_became_active":"2024-02-18T00:43:07.797505+0000","last_became_peered":"2024-02-18T00:43:07.797505+0000","last_unstale":"2024-02-18T00:43:18.775532+0000","last_undegraded":"2024-02-18T00:43:18.775532+0000","last_fullsized":"2024-02-18T00:43:18.775532+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:43:03.511887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.511887+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":203,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789172+0000","last_change":"2024-02-18T00:43:04.771039+0000","last_active":"2024-02-18T00:43:19.789172+0000","last_peered":"2024-02-18T00:43:19.789172+0000","last_clean":"2024-02-18T00:43:19.789172+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:43:19.789172+0000","last_undegraded":"2024-02-18T00:43:19.789172+0000","last_fullsized":"2024-02-18T00:43:19.789172+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:43:04.770880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.770880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":631,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790604+0000","last_change":"2024-02-18T00:42:14.840374+0000","last_active":"2024-02-18T00:43:19.790604+0000","last_peered":"2024-02-18T00:43:19.790604+0000","last_clean":"2024-02-18T00:43:19.790604+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:43:19.790604+0000","last_undegraded":"2024-02-18T00:43:19.790604+0000","last_fullsized":"2024-02-18T00:43:19.790604+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:14.840316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:14.840316+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":560,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776900+0000","last_change":"2024-02-18T00:42:06.868889+0000","last_active":"2024-02-18T00:43:18.776900+0000","last_peered":"2024-02-18T00:43:18.776900+0000","last_clean":"2024-02-18T00:43:18.776900+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:43:18.776900+0000","last_undegraded":"2024-02-18T00:43:18.776900+0000","last_fullsized":"2024-02-18T00:43:18.776900+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:06.868823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.868823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":624,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777390+0000","last_change":"2024-02-18T00:43:04.700238+0000","last_active":"2024-02-18T00:43:18.777390+0000","last_peered":"2024-02-18T00:43:18.777390+0000","last_clean":"2024-02-18T00:43:18.777390+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:43:18.777390+0000","last_undegraded":"2024-02-18T00:43:18.777390+0000","last_fullsized":"2024-02-18T00:43:18.777390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:04.700147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:04.700147+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.700147+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'24","reported_seq":674,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776525+0000","last_change":"2024-02-18T00:43:07.800668+0000","last_active":"2024-02-18T00:43:18.776525+0000","last_peered":"2024-02-18T00:43:18.776525+0000","last_clean":"2024-02-18T00:43:18.776525+0000","last_became_active":"2024-02-18T00:43:07.799985+0000","last_became_peered":"2024-02-18T00:43:07.799985+0000","last_unstale":"2024-02-18T00:43:18.776525+0000","last_undegraded":"2024-02-18T00:43:18.776525+0000","last_fullsized":"2024-02-18T00:43:18.776525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:42:59.467711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.467711+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789047+0000","last_change":"2024-02-18T00:42:04.727971+0000","last_active":"2024-02-18T00:43:19.789047+0000","last_peered":"2024-02-18T00:43:19.789047+0000","last_clean":"2024-02-18T00:43:19.789047+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:43:19.789047+0000","last_undegraded":"2024-02-18T00:43:19.789047+0000","last_fullsized":"2024-02-18T00:43:19.789047+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:04.727901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:04.727901+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":629,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776465+0000","last_change":"2024-02-18T00:42:53.461135+0000","last_active":"2024-02-18T00:43:18.776465+0000","last_peered":"2024-02-18T00:43:18.776465+0000","last_clean":"2024-02-18T00:43:18.776465+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:43:18.776465+0000","last_undegraded":"2024-02-18T00:43:18.776465+0000","last_fullsized":"2024-02-18T00:43:18.776465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:53.461030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:42:53.461030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":626,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791063+0000","last_change":"2024-02-18T00:41:31.791248+0000","last_active":"2024-02-18T00:43:19.791063+0000","last_peered":"2024-02-18T00:43:19.791063+0000","last_clean":"2024-02-18T00:43:19.791063+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:43:19.791063+0000","last_undegraded":"2024-02-18T00:43:19.791063+0000","last_fullsized":"2024-02-18T00:43:19.791063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:31.791120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.791120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":42,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788940+0000","last_change":"2024-02-18T00:43:08.893333+0000","last_active":"2024-02-18T00:43:19.788940+0000","last_peered":"2024-02-18T00:43:19.788940+0000","last_clean":"2024-02-18T00:43:19.788940+0000","last_became_active":"2024-02-18T00:42:44.690928+0000","last_became_peered":"2024-02-18T00:42:44.690928+0000","last_unstale":"2024-02-18T00:43:19.788940+0000","last_undegraded":"2024-02-18T00:43:19.788940+0000","last_fullsized":"2024-02-18T00:43:19.788940+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.893262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.893262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790024+0000","last_change":"2024-02-18T00:42:23.605382+0000","last_active":"2024-02-18T00:43:18.790024+0000","last_peered":"2024-02-18T00:43:18.790024+0000","last_clean":"2024-02-18T00:43:18.790024+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:43:18.790024+0000","last_undegraded":"2024-02-18T00:43:18.790024+0000","last_fullsized":"2024-02-18T00:43:18.790024+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:23.605268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:42:23.605268+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":732,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.780880+0000","last_change":"2024-02-18T00:43:07.792612+0000","last_active":"2024-02-18T00:43:18.780880+0000","last_peered":"2024-02-18T00:43:18.780880+0000","last_clean":"2024-02-18T00:43:18.780880+0000","last_became_active":"2024-02-18T00:43:07.792246+0000","last_became_peered":"2024-02-18T00:43:07.792246+0000","last_unstale":"2024-02-18T00:43:18.780880+0000","last_undegraded":"2024-02-18T00:43:18.780880+0000","last_fullsized":"2024-02-18T00:43:18.780880+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:42:12.900647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:12.900647+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.780331+0000","last_change":"2024-02-18T00:42:11.496928+0000","last_active":"2024-02-18T00:43:18.780331+0000","last_peered":"2024-02-18T00:43:18.780331+0000","last_clean":"2024-02-18T00:43:18.780331+0000","last_became_active":"2024-02-18T00:39:36.759224+0000","last_became_peered":"2024-02-18T00:39:36.759224+0000","last_unstale":"2024-02-18T00:43:18.780331+0000","last_undegraded":"2024-02-18T00:43:18.780331+0000","last_fullsized":"2024-02-18T00:43:18.780331+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:11.496846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:27:03.466250+0000","last_clean_scrub_stamp":"2024-02-18T00:42:11.496846+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":664,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776466+0000","last_change":"2024-02-18T00:42:05.684260+0000","last_active":"2024-02-18T00:43:18.776466+0000","last_peered":"2024-02-18T00:43:18.776466+0000","last_clean":"2024-02-18T00:43:18.776466+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:43:18.776466+0000","last_undegraded":"2024-02-18T00:43:18.776466+0000","last_fullsized":"2024-02-18T00:43:18.776466+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:05.684148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:05.684148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":619,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775407+0000","last_change":"2024-02-18T00:42:11.849138+0000","last_active":"2024-02-18T00:43:18.775407+0000","last_peered":"2024-02-18T00:43:18.775407+0000","last_clean":"2024-02-18T00:43:18.775407+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:43:18.775407+0000","last_undegraded":"2024-02-18T00:43:18.775407+0000","last_fullsized":"2024-02-18T00:43:18.775407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:11.849068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:42:11.849068+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":640,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777410+0000","last_change":"2024-02-18T00:42:06.735073+0000","last_active":"2024-02-18T00:43:18.777410+0000","last_peered":"2024-02-18T00:43:18.777410+0000","last_clean":"2024-02-18T00:43:18.777410+0000","last_became_active":"2024-02-18T00:41:57.791956+0000","last_became_peered":"2024-02-18T00:41:57.791956+0000","last_unstale":"2024-02-18T00:43:18.777410+0000","last_undegraded":"2024-02-18T00:43:18.777410+0000","last_fullsized":"2024-02-18T00:43:18.777410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":584,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:06.734956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.734956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":571,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776533+0000","last_change":"2024-02-18T00:43:05.469702+0000","last_active":"2024-02-18T00:43:18.776533+0000","last_peered":"2024-02-18T00:43:18.776533+0000","last_clean":"2024-02-18T00:43:18.776533+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:43:18.776533+0000","last_undegraded":"2024-02-18T00:43:18.776533+0000","last_fullsized":"2024-02-18T00:43:18.776533+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_clean_scrub_stamp":"2024-02-18T00:43:05.469634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":649,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789905+0000","last_change":"2024-02-18T00:42:58.721654+0000","last_active":"2024-02-18T00:43:19.789905+0000","last_peered":"2024-02-18T00:43:19.789905+0000","last_clean":"2024-02-18T00:43:19.789905+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:43:19.789905+0000","last_undegraded":"2024-02-18T00:43:19.789905+0000","last_fullsized":"2024-02-18T00:43:19.789905+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:58.721587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:58.721587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"457'23","reported_seq":693,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791425+0000","last_change":"2024-02-18T00:43:07.801861+0000","last_active":"2024-02-18T00:43:19.791425+0000","last_peered":"2024-02-18T00:43:19.791425+0000","last_clean":"2024-02-18T00:43:19.791425+0000","last_became_active":"2024-02-18T00:43:07.801362+0000","last_became_peered":"2024-02-18T00:43:07.801362+0000","last_unstale":"2024-02-18T00:43:19.791425+0000","last_undegraded":"2024-02-18T00:43:19.791425+0000","last_fullsized":"2024-02-18T00:43:19.791425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:42:10.770922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:10.770922+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":165,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":644,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788971+0000","last_change":"2024-02-18T00:42:49.673980+0000","last_active":"2024-02-18T00:43:19.788971+0000","last_peered":"2024-02-18T00:43:19.788971+0000","last_clean":"2024-02-18T00:43:19.788971+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:43:19.788971+0000","last_undegraded":"2024-02-18T00:43:19.788971+0000","last_fullsized":"2024-02-18T00:43:19.788971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.673915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.673915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1481,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:20.377219+0000","last_change":"2024-02-18T00:43:04.252649+0000","last_active":"2024-02-18T00:43:20.377219+0000","last_peered":"2024-02-18T00:43:20.377219+0000","last_clean":"2024-02-18T00:43:20.377219+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:43:20.377219+0000","last_undegraded":"2024-02-18T00:43:20.377219+0000","last_fullsized":"2024-02-18T00:43:20.377219+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:43:04.252540+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.252540+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":836,"num_read_kb":831,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":205,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790372+0000","last_change":"2024-02-18T00:43:12.839041+0000","last_active":"2024-02-18T00:43:18.790372+0000","last_peered":"2024-02-18T00:43:18.790372+0000","last_clean":"2024-02-18T00:43:18.790372+0000","last_became_active":"2024-02-18T00:39:36.764502+0000","last_became_peered":"2024-02-18T00:39:36.764502+0000","last_unstale":"2024-02-18T00:43:18.790372+0000","last_undegraded":"2024-02-18T00:43:18.790372+0000","last_fullsized":"2024-02-18T00:43:18.790372+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:43:12.838938+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:43:12.838938+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.838938+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":585,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788020+0000","last_change":"2024-02-18T00:41:34.091992+0000","last_active":"2024-02-18T00:43:19.788020+0000","last_peered":"2024-02-18T00:43:19.788020+0000","last_clean":"2024-02-18T00:43:19.788020+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:43:19.788020+0000","last_undegraded":"2024-02-18T00:43:19.788020+0000","last_fullsized":"2024-02-18T00:43:19.788020+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:34.091913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:29.090653+0000","last_clean_scrub_stamp":"2024-02-18T00:41:34.091913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":626,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777059+0000","last_change":"2024-02-18T00:42:58.421119+0000","last_active":"2024-02-18T00:43:18.777059+0000","last_peered":"2024-02-18T00:43:18.777059+0000","last_clean":"2024-02-18T00:43:18.777059+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:43:18.777059+0000","last_undegraded":"2024-02-18T00:43:18.777059+0000","last_fullsized":"2024-02-18T00:43:18.777059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:58.421052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:58.421052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"642'48","reported_seq":551,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776378+0000","last_change":"2024-02-18T00:43:07.797582+0000","last_active":"2024-02-18T00:43:18.776378+0000","last_peered":"2024-02-18T00:43:18.776378+0000","last_clean":"2024-02-18T00:43:18.776378+0000","last_became_active":"2024-02-18T00:43:07.797252+0000","last_became_peered":"2024-02-18T00:43:07.797252+0000","last_unstale":"2024-02-18T00:43:18.776378+0000","last_undegraded":"2024-02-18T00:43:18.776378+0000","last_fullsized":"2024-02-18T00:43:18.776378+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:42:13.680322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.680322+0000","log_size":48,"ondisk_log_size":48,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":179,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":637,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776833+0000","last_change":"2024-02-18T00:42:09.721978+0000","last_active":"2024-02-18T00:43:18.776833+0000","last_peered":"2024-02-18T00:43:18.776833+0000","last_clean":"2024-02-18T00:43:18.776833+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:43:18.776833+0000","last_undegraded":"2024-02-18T00:43:18.776833+0000","last_fullsized":"2024-02-18T00:43:18.776833+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:09.721900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:42:09.721900+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":650,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.778491+0000","last_change":"2024-02-18T00:42:19.938250+0000","last_active":"2024-02-18T00:43:18.778491+0000","last_peered":"2024-02-18T00:43:18.778491+0000","last_clean":"2024-02-18T00:43:18.778491+0000","last_became_active":"2024-02-18T00:42:02.125704+0000","last_became_peered":"2024-02-18T00:42:02.125704+0000","last_unstale":"2024-02-18T00:43:18.778491+0000","last_undegraded":"2024-02-18T00:43:18.778491+0000","last_fullsized":"2024-02-18T00:43:18.778491+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.938182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.938182+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":566,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776658+0000","last_change":"2024-02-18T00:41:32.116478+0000","last_active":"2024-02-18T00:43:18.776658+0000","last_peered":"2024-02-18T00:43:18.776658+0000","last_clean":"2024-02-18T00:43:18.776658+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:43:18.776658+0000","last_undegraded":"2024-02-18T00:43:18.776658+0000","last_fullsized":"2024-02-18T00:43:18.776658+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.116365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.116365+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":798,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775762+0000","last_change":"2024-02-18T00:41:15.145124+0000","last_active":"2024-02-18T00:43:18.775762+0000","last_peered":"2024-02-18T00:43:18.775762+0000","last_clean":"2024-02-18T00:43:18.775762+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:43:18.775762+0000","last_undegraded":"2024-02-18T00:43:18.775762+0000","last_fullsized":"2024-02-18T00:43:18.775762+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:41:15.145053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:15.145053+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":652,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789838+0000","last_change":"2024-02-18T00:43:07.795825+0000","last_active":"2024-02-18T00:43:19.789838+0000","last_peered":"2024-02-18T00:43:19.789838+0000","last_clean":"2024-02-18T00:43:19.789838+0000","last_became_active":"2024-02-18T00:43:07.795433+0000","last_became_peered":"2024-02-18T00:43:07.795433+0000","last_unstale":"2024-02-18T00:43:19.789838+0000","last_undegraded":"2024-02-18T00:43:19.789838+0000","last_fullsized":"2024-02-18T00:43:19.789838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:41:36.069252+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:41:36.069252+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":18,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":669,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790970+0000","last_change":"2024-02-18T00:42:15.866570+0000","last_active":"2024-02-18T00:43:19.790970+0000","last_peered":"2024-02-18T00:43:19.790970+0000","last_clean":"2024-02-18T00:43:19.790970+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:43:19.790970+0000","last_undegraded":"2024-02-18T00:43:19.790970+0000","last_fullsized":"2024-02-18T00:43:19.790970+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:15.866466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:15.866466+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":619,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777029+0000","last_change":"2024-02-18T00:42:18.192907+0000","last_active":"2024-02-18T00:43:18.777029+0000","last_peered":"2024-02-18T00:43:18.777029+0000","last_clean":"2024-02-18T00:43:18.777029+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:43:18.777029+0000","last_undegraded":"2024-02-18T00:43:18.777029+0000","last_fullsized":"2024-02-18T00:43:18.777029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.192802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.192802+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":572,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788289+0000","last_change":"2024-02-18T00:43:09.318894+0000","last_active":"2024-02-18T00:43:19.788289+0000","last_peered":"2024-02-18T00:43:19.788289+0000","last_clean":"2024-02-18T00:43:19.788289+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:43:19.788289+0000","last_undegraded":"2024-02-18T00:43:19.788289+0000","last_fullsized":"2024-02-18T00:43:19.788289+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.318803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.318803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":626,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789189+0000","last_change":"2024-02-18T00:41:32.150056+0000","last_active":"2024-02-18T00:43:19.789189+0000","last_peered":"2024-02-18T00:43:19.789189+0000","last_clean":"2024-02-18T00:43:19.789189+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:43:19.789189+0000","last_undegraded":"2024-02-18T00:43:19.789189+0000","last_fullsized":"2024-02-18T00:43:19.789189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.149984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.149984+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":702,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787759+0000","last_change":"2024-02-18T00:43:07.795659+0000","last_active":"2024-02-18T00:43:19.787759+0000","last_peered":"2024-02-18T00:43:19.787759+0000","last_clean":"2024-02-18T00:43:19.787759+0000","last_became_active":"2024-02-18T00:43:07.795322+0000","last_became_peered":"2024-02-18T00:43:07.795322+0000","last_unstale":"2024-02-18T00:43:19.787759+0000","last_undegraded":"2024-02-18T00:43:19.787759+0000","last_fullsized":"2024-02-18T00:43:19.787759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:42:51.380344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:51.380344+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":680,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788694+0000","last_change":"2024-02-18T00:42:59.355866+0000","last_active":"2024-02-18T00:43:19.788694+0000","last_peered":"2024-02-18T00:43:19.788694+0000","last_clean":"2024-02-18T00:43:19.788694+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:43:19.788694+0000","last_undegraded":"2024-02-18T00:43:19.788694+0000","last_fullsized":"2024-02-18T00:43:19.788694+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:42:59.355788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.355788+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":655,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776690+0000","last_change":"2024-02-18T00:43:01.098679+0000","last_active":"2024-02-18T00:43:18.776690+0000","last_peered":"2024-02-18T00:43:18.776690+0000","last_clean":"2024-02-18T00:43:18.776690+0000","last_became_active":"2024-02-18T00:41:42.776144+0000","last_became_peered":"2024-02-18T00:41:42.776144+0000","last_unstale":"2024-02-18T00:43:18.776690+0000","last_undegraded":"2024-02-18T00:43:18.776690+0000","last_fullsized":"2024-02-18T00:43:18.776690+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:01.098569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.098569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":567,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787996+0000","last_change":"2024-02-18T00:41:31.059363+0000","last_active":"2024-02-18T00:43:19.787996+0000","last_peered":"2024-02-18T00:43:19.787996+0000","last_clean":"2024-02-18T00:43:19.787996+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:43:19.787996+0000","last_undegraded":"2024-02-18T00:43:19.787996+0000","last_fullsized":"2024-02-18T00:43:19.787996+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:31.059285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:31.059285+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":729,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776623+0000","last_change":"2024-02-18T00:43:02.517912+0000","last_active":"2024-02-18T00:43:18.776623+0000","last_peered":"2024-02-18T00:43:18.776623+0000","last_clean":"2024-02-18T00:43:18.776623+0000","last_became_active":"2024-02-18T00:39:36.771394+0000","last_became_peered":"2024-02-18T00:39:36.771394+0000","last_unstale":"2024-02-18T00:43:18.776623+0000","last_undegraded":"2024-02-18T00:43:18.776623+0000","last_fullsized":"2024-02-18T00:43:18.776623+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:43:02.517835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.517835+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":630,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789137+0000","last_change":"2024-02-18T00:41:21.141528+0000","last_active":"2024-02-18T00:43:19.789137+0000","last_peered":"2024-02-18T00:43:19.789137+0000","last_clean":"2024-02-18T00:43:19.789137+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:43:19.789137+0000","last_undegraded":"2024-02-18T00:43:19.789137+0000","last_fullsized":"2024-02-18T00:43:19.789137+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:21.141464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.141464+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":639,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.778911+0000","last_change":"2024-02-18T00:43:09.212023+0000","last_active":"2024-02-18T00:43:18.778911+0000","last_peered":"2024-02-18T00:43:18.778911+0000","last_clean":"2024-02-18T00:43:18.778911+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:43:18.778911+0000","last_undegraded":"2024-02-18T00:43:18.778911+0000","last_fullsized":"2024-02-18T00:43:18.778911+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.211956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.211956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":644,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789397+0000","last_change":"2024-02-18T00:42:10.704623+0000","last_active":"2024-02-18T00:43:19.789397+0000","last_peered":"2024-02-18T00:43:19.789397+0000","last_clean":"2024-02-18T00:43:19.789397+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:43:19.789397+0000","last_undegraded":"2024-02-18T00:43:19.789397+0000","last_fullsized":"2024-02-18T00:43:19.789397+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:10.704545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:42:10.704545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":566,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776421+0000","last_change":"2024-02-18T00:42:09.215232+0000","last_active":"2024-02-18T00:43:18.776421+0000","last_peered":"2024-02-18T00:43:18.776421+0000","last_clean":"2024-02-18T00:43:18.776421+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:43:18.776421+0000","last_undegraded":"2024-02-18T00:43:18.776421+0000","last_fullsized":"2024-02-18T00:43:18.776421+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:09.215164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:09.215164+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":713,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.778773+0000","last_change":"2024-02-18T00:42:08.844996+0000","last_active":"2024-02-18T00:43:18.778773+0000","last_peered":"2024-02-18T00:43:18.778773+0000","last_clean":"2024-02-18T00:43:18.778773+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:43:18.778773+0000","last_undegraded":"2024-02-18T00:43:18.778773+0000","last_fullsized":"2024-02-18T00:43:18.778773+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'18","last_scrub_stamp":"2024-02-18T00:42:08.844930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:08.844930+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":632,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.783283+0000","last_change":"2024-02-18T00:42:51.817684+0000","last_active":"2024-02-18T00:43:18.783283+0000","last_peered":"2024-02-18T00:43:18.783283+0000","last_clean":"2024-02-18T00:43:18.783283+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:43:18.783283+0000","last_undegraded":"2024-02-18T00:43:18.783283+0000","last_fullsized":"2024-02-18T00:43:18.783283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:51.817614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:51.817614+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":624,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776282+0000","last_change":"2024-02-18T00:42:25.667489+0000","last_active":"2024-02-18T00:43:18.776282+0000","last_peered":"2024-02-18T00:43:18.776282+0000","last_clean":"2024-02-18T00:43:18.776282+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:43:18.776282+0000","last_undegraded":"2024-02-18T00:43:18.776282+0000","last_fullsized":"2024-02-18T00:43:18.776282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.667419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.667419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790463+0000","last_change":"2024-02-18T00:43:08.921418+0000","last_active":"2024-02-18T00:43:18.790463+0000","last_peered":"2024-02-18T00:43:18.790463+0000","last_clean":"2024-02-18T00:43:18.790463+0000","last_became_active":"2024-02-18T00:39:36.759264+0000","last_became_peered":"2024-02-18T00:39:36.759264+0000","last_unstale":"2024-02-18T00:43:18.790463+0000","last_undegraded":"2024-02-18T00:43:18.790463+0000","last_fullsized":"2024-02-18T00:43:18.790463+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.921352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.921352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":579,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789090+0000","last_change":"2024-02-18T00:42:19.794911+0000","last_active":"2024-02-18T00:43:19.789090+0000","last_peered":"2024-02-18T00:43:19.789090+0000","last_clean":"2024-02-18T00:43:19.789090+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:43:19.789090+0000","last_undegraded":"2024-02-18T00:43:19.789090+0000","last_fullsized":"2024-02-18T00:43:19.789090+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.794845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.794845+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":728,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775547+0000","last_change":"2024-02-18T00:40:42.124689+0000","last_active":"2024-02-18T00:43:18.775547+0000","last_peered":"2024-02-18T00:43:18.775547+0000","last_clean":"2024-02-18T00:43:18.775547+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:43:18.775547+0000","last_undegraded":"2024-02-18T00:43:18.775547+0000","last_fullsized":"2024-02-18T00:43:18.775547+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:40:42.124619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:40:42.124619+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":655,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776691+0000","last_change":"2024-02-18T00:43:02.729427+0000","last_active":"2024-02-18T00:43:18.776691+0000","last_peered":"2024-02-18T00:43:18.776691+0000","last_clean":"2024-02-18T00:43:18.776691+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:43:18.776691+0000","last_undegraded":"2024-02-18T00:43:18.776691+0000","last_fullsized":"2024-02-18T00:43:18.776691+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:02.729344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.729344+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":802,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776912+0000","last_change":"2024-02-18T00:43:01.683005+0000","last_active":"2024-02-18T00:43:18.776912+0000","last_peered":"2024-02-18T00:43:18.776912+0000","last_clean":"2024-02-18T00:43:18.776912+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:43:18.776912+0000","last_undegraded":"2024-02-18T00:43:18.776912+0000","last_fullsized":"2024-02-18T00:43:18.776912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:43:01.682885+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:40:09.668906+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.682885+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":585,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.786691+0000","last_change":"2024-02-18T00:42:49.778761+0000","last_active":"2024-02-18T00:43:18.786691+0000","last_peered":"2024-02-18T00:43:18.786691+0000","last_clean":"2024-02-18T00:43:18.786691+0000","last_became_active":"2024-02-18T00:39:36.751162+0000","last_became_peered":"2024-02-18T00:39:36.751162+0000","last_unstale":"2024-02-18T00:43:18.786691+0000","last_undegraded":"2024-02-18T00:43:18.786691+0000","last_fullsized":"2024-02-18T00:43:18.786691+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.778678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.778678+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":708,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776767+0000","last_change":"2024-02-18T00:43:03.706116+0000","last_active":"2024-02-18T00:43:18.776767+0000","last_peered":"2024-02-18T00:43:18.776767+0000","last_clean":"2024-02-18T00:43:18.776767+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:43:18.776767+0000","last_undegraded":"2024-02-18T00:43:18.776767+0000","last_fullsized":"2024-02-18T00:43:18.776767+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:43:03.706028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.706028+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":630,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775601+0000","last_change":"2024-02-18T00:42:57.465908+0000","last_active":"2024-02-18T00:43:18.775601+0000","last_peered":"2024-02-18T00:43:18.775601+0000","last_clean":"2024-02-18T00:43:18.775601+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:43:18.775601+0000","last_undegraded":"2024-02-18T00:43:18.775601+0000","last_fullsized":"2024-02-18T00:43:18.775601+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:57.465836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:42:57.465836+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":825,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775747+0000","last_change":"2024-02-18T00:43:00.081109+0000","last_active":"2024-02-18T00:43:18.775747+0000","last_peered":"2024-02-18T00:43:18.775747+0000","last_clean":"2024-02-18T00:43:18.775747+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:43:18.775747+0000","last_undegraded":"2024-02-18T00:43:18.775747+0000","last_fullsized":"2024-02-18T00:43:18.775747+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:43:00.080987+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.080987+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":568,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777435+0000","last_change":"2024-02-18T00:42:56.655224+0000","last_active":"2024-02-18T00:43:18.777435+0000","last_peered":"2024-02-18T00:43:18.777435+0000","last_clean":"2024-02-18T00:43:18.777435+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:43:18.777435+0000","last_undegraded":"2024-02-18T00:43:18.777435+0000","last_fullsized":"2024-02-18T00:43:18.777435+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:56.655132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.655132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":662,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791858+0000","last_change":"2024-02-18T00:42:50.108975+0000","last_active":"2024-02-18T00:43:19.791858+0000","last_peered":"2024-02-18T00:43:19.791858+0000","last_clean":"2024-02-18T00:43:19.791858+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:43:19.791858+0000","last_undegraded":"2024-02-18T00:43:19.791858+0000","last_fullsized":"2024-02-18T00:43:19.791858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:42:50.108893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.108893+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":632,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.782060+0000","last_change":"2024-02-18T00:42:17.908390+0000","last_active":"2024-02-18T00:43:18.782060+0000","last_peered":"2024-02-18T00:43:18.782060+0000","last_clean":"2024-02-18T00:43:18.782060+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:43:18.782060+0000","last_undegraded":"2024-02-18T00:43:18.782060+0000","last_fullsized":"2024-02-18T00:43:18.782060+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:17.908310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:17.908310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":621,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776610+0000","last_change":"2024-02-18T00:42:56.033487+0000","last_active":"2024-02-18T00:43:18.776610+0000","last_peered":"2024-02-18T00:43:18.776610+0000","last_clean":"2024-02-18T00:43:18.776610+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:43:18.776610+0000","last_undegraded":"2024-02-18T00:43:18.776610+0000","last_fullsized":"2024-02-18T00:43:18.776610+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:56.033419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.033419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":624,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776937+0000","last_change":"2024-02-18T00:42:04.263648+0000","last_active":"2024-02-18T00:43:18.776937+0000","last_peered":"2024-02-18T00:43:18.776937+0000","last_clean":"2024-02-18T00:43:18.776937+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:43:18.776937+0000","last_undegraded":"2024-02-18T00:43:18.776937+0000","last_fullsized":"2024-02-18T00:43:18.776937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:04.263582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:04.263582+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":568,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776692+0000","last_change":"2024-02-18T00:42:49.449913+0000","last_active":"2024-02-18T00:43:18.776692+0000","last_peered":"2024-02-18T00:43:18.776692+0000","last_clean":"2024-02-18T00:43:18.776692+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:43:18.776692+0000","last_undegraded":"2024-02-18T00:43:18.776692+0000","last_fullsized":"2024-02-18T00:43:18.776692+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.449844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.449844+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":661,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788391+0000","last_change":"2024-02-18T00:42:20.356302+0000","last_active":"2024-02-18T00:43:19.788391+0000","last_peered":"2024-02-18T00:43:19.788391+0000","last_clean":"2024-02-18T00:43:19.788391+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:43:19.788391+0000","last_undegraded":"2024-02-18T00:43:19.788391+0000","last_fullsized":"2024-02-18T00:43:19.788391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:42:20.356203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:20.356203+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":637,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791175+0000","last_change":"2024-02-18T00:43:09.187333+0000","last_active":"2024-02-18T00:43:19.791175+0000","last_peered":"2024-02-18T00:43:19.791175+0000","last_clean":"2024-02-18T00:43:19.791175+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:43:19.791175+0000","last_undegraded":"2024-02-18T00:43:19.791175+0000","last_fullsized":"2024-02-18T00:43:19.791175+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.187270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.187270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":633,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791896+0000","last_change":"2024-02-18T00:41:24.779552+0000","last_active":"2024-02-18T00:43:19.791896+0000","last_peered":"2024-02-18T00:43:19.791896+0000","last_clean":"2024-02-18T00:43:19.791896+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:43:19.791896+0000","last_undegraded":"2024-02-18T00:43:19.791896+0000","last_fullsized":"2024-02-18T00:43:19.791896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:24.779481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:24.779481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":614,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777280+0000","last_change":"2024-02-18T00:40:52.018692+0000","last_active":"2024-02-18T00:43:18.777280+0000","last_peered":"2024-02-18T00:43:18.777280+0000","last_clean":"2024-02-18T00:43:18.777280+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:43:18.777280+0000","last_undegraded":"2024-02-18T00:43:18.777280+0000","last_fullsized":"2024-02-18T00:43:18.777280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:52.018627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:52.018627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":567,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.781001+0000","last_change":"2024-02-18T00:42:56.820557+0000","last_active":"2024-02-18T00:43:18.781001+0000","last_peered":"2024-02-18T00:43:18.781001+0000","last_clean":"2024-02-18T00:43:18.781001+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:43:18.781001+0000","last_undegraded":"2024-02-18T00:43:18.781001+0000","last_fullsized":"2024-02-18T00:43:18.781001+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:56.820451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.820451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":635,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.775025+0000","last_change":"2024-02-18T00:43:02.133961+0000","last_active":"2024-02-18T00:43:18.775025+0000","last_peered":"2024-02-18T00:43:18.775025+0000","last_clean":"2024-02-18T00:43:18.775025+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:43:18.775025+0000","last_undegraded":"2024-02-18T00:43:18.775025+0000","last_fullsized":"2024-02-18T00:43:18.775025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-18T00:43:02.133854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.133854+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":633,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776342+0000","last_change":"2024-02-18T00:43:07.782401+0000","last_active":"2024-02-18T00:43:18.776342+0000","last_peered":"2024-02-18T00:43:18.776342+0000","last_clean":"2024-02-18T00:43:18.776342+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:43:18.776342+0000","last_undegraded":"2024-02-18T00:43:18.776342+0000","last_fullsized":"2024-02-18T00:43:18.776342+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_clean_scrub_stamp":"2024-02-18T00:43:07.782283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":641,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.783208+0000","last_change":"2024-02-18T00:41:35.739124+0000","last_active":"2024-02-18T00:43:18.783208+0000","last_peered":"2024-02-18T00:43:18.783208+0000","last_clean":"2024-02-18T00:43:18.783208+0000","last_became_active":"2024-02-18T00:39:36.750866+0000","last_became_peered":"2024-02-18T00:39:36.750866+0000","last_unstale":"2024-02-18T00:43:18.783208+0000","last_undegraded":"2024-02-18T00:43:18.783208+0000","last_fullsized":"2024-02-18T00:43:18.783208+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:35.739045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:35.739045+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.790092+0000","last_change":"2024-02-18T00:42:25.583900+0000","last_active":"2024-02-18T00:43:18.790092+0000","last_peered":"2024-02-18T00:43:18.790092+0000","last_clean":"2024-02-18T00:43:18.790092+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:43:18.790092+0000","last_undegraded":"2024-02-18T00:43:18.790092+0000","last_fullsized":"2024-02-18T00:43:18.790092+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.583823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.583823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":581,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777245+0000","last_change":"2024-02-18T00:41:18.581382+0000","last_active":"2024-02-18T00:43:18.777245+0000","last_peered":"2024-02-18T00:43:18.777245+0000","last_clean":"2024-02-18T00:43:18.777245+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:43:18.777245+0000","last_undegraded":"2024-02-18T00:43:18.777245+0000","last_fullsized":"2024-02-18T00:43:18.777245+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:18.581310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.581310+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":710,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777837+0000","last_change":"2024-02-18T00:41:18.896665+0000","last_active":"2024-02-18T00:43:18.777837+0000","last_peered":"2024-02-18T00:43:18.777837+0000","last_clean":"2024-02-18T00:43:18.777837+0000","last_became_active":"2024-02-18T00:39:36.761140+0000","last_became_peered":"2024-02-18T00:39:36.761140+0000","last_unstale":"2024-02-18T00:43:18.777837+0000","last_undegraded":"2024-02-18T00:43:18.777837+0000","last_fullsized":"2024-02-18T00:43:18.777837+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:41:18.896586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:18.896586+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":632,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.783927+0000","last_change":"2024-02-18T00:43:03.852309+0000","last_active":"2024-02-18T00:43:18.783927+0000","last_peered":"2024-02-18T00:43:18.783927+0000","last_clean":"2024-02-18T00:43:18.783927+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:43:18.783927+0000","last_undegraded":"2024-02-18T00:43:18.783927+0000","last_fullsized":"2024-02-18T00:43:18.783927+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.852193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":196,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.780624+0000","last_change":"2024-02-18T00:42:06.606911+0000","last_active":"2024-02-18T00:43:18.780624+0000","last_peered":"2024-02-18T00:43:18.780624+0000","last_clean":"2024-02-18T00:43:18.780624+0000","last_became_active":"2024-02-18T00:39:36.758138+0000","last_became_peered":"2024-02-18T00:39:36.758138+0000","last_unstale":"2024-02-18T00:43:18.780624+0000","last_undegraded":"2024-02-18T00:43:18.780624+0000","last_fullsized":"2024-02-18T00:43:18.780624+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:06.606773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:06.606773+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":614,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776769+0000","last_change":"2024-02-18T00:40:51.008109+0000","last_active":"2024-02-18T00:43:18.776769+0000","last_peered":"2024-02-18T00:43:18.776769+0000","last_clean":"2024-02-18T00:43:18.776769+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:43:18.776769+0000","last_undegraded":"2024-02-18T00:43:18.776769+0000","last_fullsized":"2024-02-18T00:43:18.776769+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:40:51.008036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:40:51.008036+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.789637+0000","last_change":"2024-02-18T00:42:24.598343+0000","last_active":"2024-02-18T00:43:18.789637+0000","last_peered":"2024-02-18T00:43:18.789637+0000","last_clean":"2024-02-18T00:43:18.789637+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:43:18.789637+0000","last_undegraded":"2024-02-18T00:43:18.789637+0000","last_fullsized":"2024-02-18T00:43:18.789637+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:24.598257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:24.598257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":725,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776902+0000","last_change":"2024-02-18T00:42:08.213474+0000","last_active":"2024-02-18T00:43:18.776902+0000","last_peered":"2024-02-18T00:43:18.776902+0000","last_clean":"2024-02-18T00:43:18.776902+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:43:18.776902+0000","last_undegraded":"2024-02-18T00:43:18.776902+0000","last_fullsized":"2024-02-18T00:43:18.776902+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:42:08.213407+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:42:08.213407+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":655,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776548+0000","last_change":"2024-02-18T00:43:10.797905+0000","last_active":"2024-02-18T00:43:18.776548+0000","last_peered":"2024-02-18T00:43:18.776548+0000","last_clean":"2024-02-18T00:43:18.776548+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:43:18.776548+0000","last_undegraded":"2024-02-18T00:43:18.776548+0000","last_fullsized":"2024-02-18T00:43:18.776548+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:10.797791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:10.797791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"466'10","reported_seq":834,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.784049+0000","last_change":"2024-02-18T00:41:21.505578+0000","last_active":"2024-02-18T00:43:18.784049+0000","last_peered":"2024-02-18T00:43:18.784049+0000","last_clean":"2024-02-18T00:43:18.784049+0000","last_became_active":"2024-02-18T00:39:36.751716+0000","last_became_peered":"2024-02-18T00:39:36.751716+0000","last_unstale":"2024-02-18T00:43:18.784049+0000","last_undegraded":"2024-02-18T00:43:18.784049+0000","last_fullsized":"2024-02-18T00:43:18.784049+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:41:21.505495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.505495+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.789939+0000","last_change":"2024-02-18T00:42:26.623217+0000","last_active":"2024-02-18T00:43:18.789939+0000","last_peered":"2024-02-18T00:43:18.789939+0000","last_clean":"2024-02-18T00:43:18.789939+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:43:18.789939+0000","last_undegraded":"2024-02-18T00:43:18.789939+0000","last_fullsized":"2024-02-18T00:43:18.789939+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:26.623073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:26.623073+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":676,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.784106+0000","last_change":"2024-02-18T00:43:04.887857+0000","last_active":"2024-02-18T00:43:18.784106+0000","last_peered":"2024-02-18T00:43:18.784106+0000","last_clean":"2024-02-18T00:43:18.784106+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:43:18.784106+0000","last_undegraded":"2024-02-18T00:43:18.784106+0000","last_fullsized":"2024-02-18T00:43:18.784106+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:43:04.887771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.887771+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":625,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777520+0000","last_change":"2024-02-18T00:42:52.999275+0000","last_active":"2024-02-18T00:43:18.777520+0000","last_peered":"2024-02-18T00:43:18.777520+0000","last_clean":"2024-02-18T00:43:18.777520+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:43:18.777520+0000","last_undegraded":"2024-02-18T00:43:18.777520+0000","last_fullsized":"2024-02-18T00:43:18.777520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:52.999208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:42:52.999208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":639,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776838+0000","last_change":"2024-02-18T00:42:19.189296+0000","last_active":"2024-02-18T00:43:18.776838+0000","last_peered":"2024-02-18T00:43:18.776838+0000","last_clean":"2024-02-18T00:43:18.776838+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:43:18.776838+0000","last_undegraded":"2024-02-18T00:43:18.776838+0000","last_fullsized":"2024-02-18T00:43:18.776838+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.189231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.189231+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":583,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789156+0000","last_change":"2024-02-18T00:41:27.140619+0000","last_active":"2024-02-18T00:43:19.789156+0000","last_peered":"2024-02-18T00:43:19.789156+0000","last_clean":"2024-02-18T00:43:19.789156+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:43:19.789156+0000","last_undegraded":"2024-02-18T00:43:19.789156+0000","last_fullsized":"2024-02-18T00:43:19.789156+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:41:27.140537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.140537+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":676,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789774+0000","last_change":"2024-02-18T00:41:24.123727+0000","last_active":"2024-02-18T00:43:19.789774+0000","last_peered":"2024-02-18T00:43:19.789774+0000","last_clean":"2024-02-18T00:43:19.789774+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:43:19.789774+0000","last_undegraded":"2024-02-18T00:43:19.789774+0000","last_fullsized":"2024-02-18T00:43:19.789774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:41:24.123655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:24.123655+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":652,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.784374+0000","last_change":"2024-02-18T00:42:13.925258+0000","last_active":"2024-02-18T00:43:18.784374+0000","last_peered":"2024-02-18T00:43:18.784374+0000","last_clean":"2024-02-18T00:43:18.784374+0000","last_became_active":"2024-02-18T00:39:36.741489+0000","last_became_peered":"2024-02-18T00:39:36.741489+0000","last_unstale":"2024-02-18T00:43:18.784374+0000","last_undegraded":"2024-02-18T00:43:18.784374+0000","last_fullsized":"2024-02-18T00:43:18.784374+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:13.925195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.925195+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":633,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.791319+0000","last_change":"2024-02-18T00:42:12.788524+0000","last_active":"2024-02-18T00:43:19.791319+0000","last_peered":"2024-02-18T00:43:19.791319+0000","last_clean":"2024-02-18T00:43:19.791319+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:43:19.791319+0000","last_undegraded":"2024-02-18T00:43:19.791319+0000","last_fullsized":"2024-02-18T00:43:19.791319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:12.788462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:42:12.788462+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":567,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.787230+0000","last_change":"2024-02-18T00:42:16.894617+0000","last_active":"2024-02-18T00:43:18.787230+0000","last_peered":"2024-02-18T00:43:18.787230+0000","last_clean":"2024-02-18T00:43:18.787230+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:43:18.787230+0000","last_undegraded":"2024-02-18T00:43:18.787230+0000","last_fullsized":"2024-02-18T00:43:18.787230+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_clean_scrub_stamp":"2024-02-18T00:42:16.894524+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":665,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777447+0000","last_change":"2024-02-18T00:41:21.991859+0000","last_active":"2024-02-18T00:43:18.777447+0000","last_peered":"2024-02-18T00:43:18.777447+0000","last_clean":"2024-02-18T00:43:18.777447+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:43:18.777447+0000","last_undegraded":"2024-02-18T00:43:18.777447+0000","last_fullsized":"2024-02-18T00:43:18.777447+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:41:21.991789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.991789+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":658,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776534+0000","last_change":"2024-02-18T00:42:13.912023+0000","last_active":"2024-02-18T00:43:18.776534+0000","last_peered":"2024-02-18T00:43:18.776534+0000","last_clean":"2024-02-18T00:43:18.776534+0000","last_became_active":"2024-02-18T00:39:36.761839+0000","last_became_peered":"2024-02-18T00:39:36.761839+0000","last_unstale":"2024-02-18T00:43:18.776534+0000","last_undegraded":"2024-02-18T00:43:18.776534+0000","last_fullsized":"2024-02-18T00:43:18.776534+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:42:13.911916+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.911916+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":628,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788170+0000","last_change":"2024-02-18T00:42:25.359836+0000","last_active":"2024-02-18T00:43:19.788170+0000","last_peered":"2024-02-18T00:43:19.788170+0000","last_clean":"2024-02-18T00:43:19.788170+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:43:19.788170+0000","last_undegraded":"2024-02-18T00:43:19.788170+0000","last_fullsized":"2024-02-18T00:43:19.788170+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.359763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.359763+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":584,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776750+0000","last_change":"2024-02-18T00:41:25.051216+0000","last_active":"2024-02-18T00:43:18.776750+0000","last_peered":"2024-02-18T00:43:18.776750+0000","last_clean":"2024-02-18T00:43:18.776750+0000","last_became_active":"2024-02-18T00:39:36.771648+0000","last_became_peered":"2024-02-18T00:39:36.771648+0000","last_unstale":"2024-02-18T00:43:18.776750+0000","last_undegraded":"2024-02-18T00:43:18.776750+0000","last_fullsized":"2024-02-18T00:43:18.776750+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:25.051146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:25.051146+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":699,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787851+0000","last_change":"2024-02-18T00:43:01.284219+0000","last_active":"2024-02-18T00:43:19.787851+0000","last_peered":"2024-02-18T00:43:19.787851+0000","last_clean":"2024-02-18T00:43:19.787851+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:43:19.787851+0000","last_undegraded":"2024-02-18T00:43:19.787851+0000","last_fullsized":"2024-02-18T00:43:19.787851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:43:01.284140+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.284140+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.789703+0000","last_change":"2024-02-18T00:43:03.809975+0000","last_active":"2024-02-18T00:43:18.789703+0000","last_peered":"2024-02-18T00:43:18.789703+0000","last_clean":"2024-02-18T00:43:18.789703+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:43:18.789703+0000","last_undegraded":"2024-02-18T00:43:18.789703+0000","last_fullsized":"2024-02-18T00:43:18.789703+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.809907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.809907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":651,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776390+0000","last_change":"2024-02-18T00:41:37.673766+0000","last_active":"2024-02-18T00:43:18.776390+0000","last_peered":"2024-02-18T00:43:18.776390+0000","last_clean":"2024-02-18T00:43:18.776390+0000","last_became_active":"2024-02-18T00:39:36.749467+0000","last_became_peered":"2024-02-18T00:39:36.749467+0000","last_unstale":"2024-02-18T00:43:18.776390+0000","last_undegraded":"2024-02-18T00:43:18.776390+0000","last_fullsized":"2024-02-18T00:43:18.776390+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_clean_scrub_stamp":"2024-02-18T00:41:37.673673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":570,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.789679+0000","last_change":"2024-02-18T00:42:21.695753+0000","last_active":"2024-02-18T00:43:19.789679+0000","last_peered":"2024-02-18T00:43:19.789679+0000","last_clean":"2024-02-18T00:43:19.789679+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:43:19.789679+0000","last_undegraded":"2024-02-18T00:43:19.789679+0000","last_fullsized":"2024-02-18T00:43:19.789679+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:21.695689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:21.695689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":661,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.788136+0000","last_change":"2024-02-18T00:42:04.345907+0000","last_active":"2024-02-18T00:43:19.788136+0000","last_peered":"2024-02-18T00:43:19.788136+0000","last_clean":"2024-02-18T00:43:19.788136+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:43:19.788136+0000","last_undegraded":"2024-02-18T00:43:19.788136+0000","last_fullsized":"2024-02-18T00:43:19.788136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:42:04.345831+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:42:04.345831+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":201,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.791418+0000","last_change":"2024-02-18T00:42:14.524534+0000","last_active":"2024-02-18T00:43:18.791418+0000","last_peered":"2024-02-18T00:43:18.791418+0000","last_clean":"2024-02-18T00:43:18.791418+0000","last_became_active":"2024-02-18T00:39:36.761469+0000","last_became_peered":"2024-02-18T00:39:36.761469+0000","last_unstale":"2024-02-18T00:43:18.791418+0000","last_undegraded":"2024-02-18T00:43:18.791418+0000","last_fullsized":"2024-02-18T00:43:18.791418+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:14.524453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:42:14.524453+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":196,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.791379+0000","last_change":"2024-02-18T00:42:05.612003+0000","last_active":"2024-02-18T00:43:18.791379+0000","last_peered":"2024-02-18T00:43:18.791379+0000","last_clean":"2024-02-18T00:43:18.791379+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:43:18.791379+0000","last_undegraded":"2024-02-18T00:43:18.791379+0000","last_fullsized":"2024-02-18T00:43:18.791379+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:05.611891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:42:05.611891+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":578,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777673+0000","last_change":"2024-02-18T00:41:27.921493+0000","last_active":"2024-02-18T00:43:18.777673+0000","last_peered":"2024-02-18T00:43:18.777673+0000","last_clean":"2024-02-18T00:43:18.777673+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:43:18.777673+0000","last_undegraded":"2024-02-18T00:43:18.777673+0000","last_fullsized":"2024-02-18T00:43:18.777673+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:27.921417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:27.921417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":670,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787527+0000","last_change":"2024-02-18T00:41:21.035047+0000","last_active":"2024-02-18T00:43:19.787527+0000","last_peered":"2024-02-18T00:43:19.787527+0000","last_clean":"2024-02-18T00:43:19.787527+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:43:19.787527+0000","last_undegraded":"2024-02-18T00:43:19.787527+0000","last_fullsized":"2024-02-18T00:43:19.787527+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:41:21.034941+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:41:21.034941+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":620,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.779266+0000","last_change":"2024-02-18T00:41:22.988106+0000","last_active":"2024-02-18T00:43:18.779266+0000","last_peered":"2024-02-18T00:43:18.779266+0000","last_clean":"2024-02-18T00:43:18.779266+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:43:18.779266+0000","last_undegraded":"2024-02-18T00:43:18.779266+0000","last_fullsized":"2024-02-18T00:43:18.779266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:22.988039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:41:22.988039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":651,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.790431+0000","last_change":"2024-02-18T00:42:51.096086+0000","last_active":"2024-02-18T00:43:19.790431+0000","last_peered":"2024-02-18T00:43:19.790431+0000","last_clean":"2024-02-18T00:43:19.790431+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:43:19.790431+0000","last_undegraded":"2024-02-18T00:43:19.790431+0000","last_fullsized":"2024-02-18T00:43:19.790431+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:51.096010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:51.096010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":560,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.778665+0000","last_change":"2024-02-18T00:41:32.876920+0000","last_active":"2024-02-18T00:43:18.778665+0000","last_peered":"2024-02-18T00:43:18.778665+0000","last_clean":"2024-02-18T00:43:18.778665+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:43:18.778665+0000","last_undegraded":"2024-02-18T00:43:18.778665+0000","last_fullsized":"2024-02-18T00:43:18.778665+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:32.876858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:41:32.876858+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":723,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.777436+0000","last_change":"2024-02-18T00:42:09.796340+0000","last_active":"2024-02-18T00:43:18.777436+0000","last_peered":"2024-02-18T00:43:18.777436+0000","last_clean":"2024-02-18T00:43:18.777436+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:43:18.777436+0000","last_undegraded":"2024-02-18T00:43:18.777436+0000","last_fullsized":"2024-02-18T00:43:18.777436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-18T00:42:09.796275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:09.796275+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":645,"reported_epoch":653,"state":"active+clean","last_fresh":"2024-02-18T00:43:18.776460+0000","last_change":"2024-02-18T00:41:17.135859+0000","last_active":"2024-02-18T00:43:18.776460+0000","last_peered":"2024-02-18T00:43:18.776460+0000","last_clean":"2024-02-18T00:43:18.776460+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:43:18.776460+0000","last_undegraded":"2024-02-18T00:43:18.776460+0000","last_fullsized":"2024-02-18T00:43:18.776460+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:41:17.135767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:41:17.135767+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":628,"reported_epoch":654,"state":"active+clean","last_fresh":"2024-02-18T00:43:19.787752+0000","last_change":"2024-02-18T00:42:22.362932+0000","last_active":"2024-02-18T00:43:19.787752+0000","last_peered":"2024-02-18T00:43:19.787752+0000","last_clean":"2024-02-18T00:43:19.787752+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:43:19.787752+0000","last_undegraded":"2024-02-18T00:43:19.787752+0000","last_fullsized":"2024-02-18T00:43:19.787752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.362852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.362852+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":240,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":48,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":8,"num_read_kb":1,"num_write":15,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":98304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":24,"acting":24,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":42,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1510,"num_read_kb":1234,"num_write":895,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3834,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1266,"ondisk_log_size":1266,"up":126,"acting":126,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":836,"num_read_kb":831,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266861,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313696,"kb_used_data":4184,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431456,"statfs":{"total":95995035648,"available":95673810944,"internally_reserved":0,"allocated":4284416,"data_stored":2229872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.55300000000000005,"15min":0.57299999999999995},"min":{"1min":0.30499999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.7389999999999999,"5min":4.7389999999999999,"15min":6.8700000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.499,"15min":0.59299999999999997},"min":{"1min":0.22500000000000001,"5min":0.218,"15min":0.218},"max":{"1min":0.873,"5min":1.0780000000000001,"15min":11.288},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53500000000000003,"15min":0.623},"min":{"1min":0.309,"5min":0.29799999999999999,"15min":0.29599999999999999},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":7.4729999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.64000000000000001},"min":{"1min":0.34899999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82899999999999996,"5min":1.1930000000000001,"15min":7.7869999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.32500000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":4.8220000000000001,"5min":4.8220000000000001,"15min":6.7380000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.52700000000000002,"15min":0.60099999999999998},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":1.913,"5min":1.913,"15min":12.861000000000001},"last":0.435}]},{"osd":3,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.36799999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.8520000000000003,"5min":4.8520000000000003,"15min":7.1360000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53800000000000003,"15min":0.58399999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.948,"5min":1.948,"15min":3.7989999999999999},"last":0.378}]},{"osd":4,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45500000000000002,"15min":0.433},"min":{"1min":0.28000000000000003,"5min":0.27200000000000002,"15min":0.215},"max":{"1min":1.115,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.55400000000000005,"15min":0.48099999999999998},"min":{"1min":0.317,"5min":0.252,"15min":0.246},"max":{"1min":4.7859999999999996,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.46800000000000003,"15min":0.46200000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.24099999999999999},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.51500000000000001,"15min":0.50800000000000001},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.25600000000000001},"max":{"1min":1.853,"5min":1.853,"15min":4.9130000000000003},"last":0.42499999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.55000000000000004,"15min":0.52100000000000002},"min":{"1min":0.36599999999999999,"5min":0.28899999999999998,"15min":0.28799999999999998},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.60199999999999998,"15min":0.54200000000000004},"min":{"1min":0.38900000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.3860000000000001,"5min":5.3860000000000001,"15min":5.3860000000000001},"last":0.45100000000000001}]}]},{"osd":6,"up_from":47,"seq":201863463091,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301416,"kb_used_data":4192,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443736,"statfs":{"total":95995035648,"available":95686385664,"internally_reserved":0,"allocated":4292608,"data_stored":2239388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":1.2649999999999999},"min":{"1min":0.27000000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.70399999999999996,"5min":0.82599999999999996,"15min":69.194999999999993},"last":0.372},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50900000000000001,"15min":0.73699999999999999},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67800000000000005,"5min":0.72899999999999998,"15min":10.433},"last":0.46899999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53100000000000003,"15min":1.3380000000000001},"min":{"1min":0.316,"5min":0.29399999999999998,"15min":0.26000000000000001},"max":{"1min":0.73499999999999999,"5min":0.77300000000000002,"15min":76.108999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.504,"15min":1.3220000000000001},"min":{"1min":0.38900000000000001,"5min":0.32700000000000001,"15min":0.28100000000000003},"max":{"1min":0.67500000000000004,"5min":0.81299999999999994,"15min":71.872},"last":3.665}]},{"osd":2,"last update":"Sun Feb 18 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52100000000000002,"15min":0.81100000000000005},"min":{"1min":0.443,"5min":0.318,"15min":0.31},"max":{"1min":0.751,"5min":0.76200000000000001,"15min":39.939999999999998},"last":3.702},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.98699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":39.673000000000002},"last":0.55200000000000005}]},{"osd":3,"last update":"Sun Feb 18 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.84199999999999997},"min":{"1min":0.375,"5min":0.35599999999999998,"15min":0.29599999999999999},"max":{"1min":0.749,"5min":0.84499999999999997,"15min":21.219000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51900000000000002,"15min":0.78300000000000003},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73999999999999999,"5min":0.83499999999999996,"15min":14.007999999999999},"last":3.7280000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.51800000000000002},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.22800000000000001},"max":{"1min":0.748,"5min":0.748,"15min":8.1479999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.46899999999999997,"15min":0.70099999999999996},"min":{"1min":0.32100000000000001,"5min":0.28000000000000003,"15min":0.26100000000000001},"max":{"1min":0.69699999999999995,"5min":0.79500000000000004,"15min":35.18},"last":4.1310000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:43:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46100000000000002,"15min":0.54800000000000004},"min":{"1min":0.313,"5min":0.29299999999999998,"15min":0.27500000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":10.606},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.497,"15min":0.72999999999999998},"min":{"1min":0.31,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75800000000000001,"5min":0.83399999999999996,"15min":35.119},"last":3.8439999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.55100000000000005},"min":{"1min":0.35099999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66400000000000003,"5min":0.76700000000000002,"15min":5.4770000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.52700000000000002,"15min":0.72699999999999998},"min":{"1min":0.378,"5min":0.32600000000000001,"15min":0.32000000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":32.985999999999997},"last":0.48499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575050,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":317784,"kb_used_data":4176,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93427368,"statfs":{"total":95995035648,"available":95669624832,"internally_reserved":0,"allocated":4276224,"data_stored":2221787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.65100000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":32.106999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.44400000000000001,"15min":0.72499999999999998},"min":{"1min":0.27300000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.1949999999999998,"5min":6.625,"15min":43.137},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36699999999999999,"15min":0.75800000000000001},"min":{"1min":0.23799999999999999,"5min":0.23499999999999999,"15min":0.19900000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":49.201000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.40100000000000002,"15min":0.59199999999999997},"min":{"1min":0.247,"5min":0.23999999999999999,"15min":0.217},"max":{"1min":0.748,"5min":0.83199999999999996,"15min":17.596},"last":0.32000000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.39600000000000002,"15min":0.72499999999999998},"min":{"1min":0.27600000000000002,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":37.524000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.42799999999999999,"15min":0.745},"min":{"1min":0.29999999999999999,"5min":0.28699999999999998,"15min":0.26400000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":32.036000000000001},"last":0.42599999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:42:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55400000000000005,"15min":0.76000000000000001},"min":{"1min":0.40999999999999998,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":17.486999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.77700000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.32500000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":12.75},"last":0.44700000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:42:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.76800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.34399999999999997},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":12.611000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59499999999999997,"15min":0.78100000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33100000000000002},"max":{"1min":1.02,"5min":1.02,"15min":17.417000000000002},"last":0.56399999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:43:11 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.59199999999999997,"15min":0.79900000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.92300000000000004,"15min":12.834},"last":0.496},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59599999999999997,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.35499999999999998},"max":{"1min":0.93300000000000005,"5min":0.97199999999999998,"15min":12.651999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:42:40 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61199999999999999,"15min":0.79700000000000004},"min":{"1min":0.42399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":17.452000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.80700000000000005},"min":{"1min":0.44700000000000001,"5min":0.40300000000000002,"15min":0.38700000000000001},"max":{"1min":1.012,"5min":1.012,"15min":12.805},"last":0.55000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738576,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313680,"kb_used_data":4168,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431472,"statfs":{"total":95995035648,"available":95673827328,"internally_reserved":0,"allocated":4268032,"data_stored":2236016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:43:04 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.34100000000000003,"15min":0.99399999999999999},"min":{"1min":0.222,"5min":0.19500000000000001,"15min":0.18099999999999999},"max":{"1min":0.47799999999999998,"5min":0.873,"15min":133.38999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.38900000000000001,"15min":0.79200000000000004},"min":{"1min":0.23499999999999999,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":0.48799999999999999,"5min":1.8069999999999999,"15min":90.623000000000005},"last":0.30299999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.42299999999999999,"15min":0.90100000000000002},"min":{"1min":0.27500000000000002,"5min":0.25700000000000001,"15min":0.20200000000000001},"max":{"1min":0.61799999999999999,"5min":1.782,"15min":90.850999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.373,"5min":0.36699999999999999,"15min":0.86599999999999999},"min":{"1min":0.27400000000000002,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54300000000000004,"5min":1.482,"15min":80.537000000000006},"last":0.41999999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:43:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36899999999999999,"15min":0.64800000000000002},"min":{"1min":0.28999999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47999999999999998,"5min":0.76600000000000001,"15min":19.006},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.433,"15min":1.048},"min":{"1min":0.29599999999999999,"5min":0.27300000000000002,"15min":0.23999999999999999},"max":{"1min":0.55800000000000005,"5min":1.6910000000000001,"15min":146.71899999999999},"last":0.375}]},{"osd":4,"last update":"Sun Feb 18 00:43:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55700000000000005,"15min":0.92500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.26700000000000002},"max":{"1min":0.83699999999999997,"5min":1.6830000000000001,"15min":57.914000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60099999999999998,"15min":0.97699999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.30399999999999999},"max":{"1min":0.82199999999999995,"5min":1.6619999999999999,"15min":90.070999999999998},"last":0.53200000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:42:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59399999999999997,"15min":0.95299999999999996},"min":{"1min":0.435,"5min":0.36499999999999999,"15min":0.32000000000000001},"max":{"1min":0.86499999999999999,"5min":1.714,"15min":90.049000000000007},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56799999999999995,"15min":0.98399999999999999},"min":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.32200000000000001},"max":{"1min":0.76000000000000001,"5min":0.86399999999999999,"15min":57.991},"last":0.48099999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:43:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58399999999999996,"15min":0.80000000000000004},"min":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.35999999999999999},"max":{"1min":0.80500000000000005,"5min":1.0129999999999999,"15min":19.039000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59999999999999998,"15min":0.97999999999999998},"min":{"1min":0.44900000000000001,"5min":0.39700000000000002,"15min":0.33500000000000002},"max":{"1min":0.70699999999999996,"5min":1.76,"15min":89.989999999999995},"last":0.46500000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:42:30 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63100000000000001,"15min":1.0529999999999999},"min":{"1min":0.47999999999999998,"5min":0.36799999999999999,"15min":0.36499999999999999},"max":{"1min":1.01,"5min":1.732,"15min":90.025000000000006},"last":0.501},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59899999999999998,"15min":0.997},"min":{"1min":0.46100000000000002,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.603,"15min":57.962000000000003},"last":0.54400000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411527,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313688,"kb_used_data":4176,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431464,"statfs":{"total":95995035648,"available":95673819136,"internally_reserved":0,"allocated":4276224,"data_stored":2233663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.48499999999999999},"min":{"1min":0.251,"5min":0.192,"15min":0.192},"max":{"1min":0.54200000000000004,"5min":0.78500000000000003,"15min":12.144},"last":0.374},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.60299999999999998},"min":{"1min":0.23000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58199999999999996,"5min":0.63600000000000001,"15min":18.891999999999999},"last":0.31}]},{"osd":1,"last update":"Sun Feb 18 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.375,"15min":0.495},"min":{"1min":0.24399999999999999,"5min":0.23300000000000001,"15min":0.19700000000000001},"max":{"1min":0.63500000000000001,"5min":0.76800000000000002,"15min":10.138999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.372,"15min":0.53800000000000003},"min":{"1min":0.245,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":18.919},"last":0.34399999999999997}]},{"osd":3,"last update":"Sun Feb 18 00:42:55 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.40799999999999997,"15min":0.59099999999999997},"min":{"1min":0.28499999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.59499999999999997,"5min":0.67400000000000004,"15min":22.712},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.40400000000000003,"15min":0.59199999999999997},"min":{"1min":0.28000000000000003,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56799999999999995,"5min":0.61699999999999999,"15min":22.359000000000002},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:42:27 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58499999999999996,"15min":0.68500000000000005},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.32500000000000001},"max":{"1min":0.88900000000000001,"5min":0.94899999999999995,"15min":18.812000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.67100000000000004},"min":{"1min":0.42699999999999999,"5min":0.36099999999999999,"15min":0.28699999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":10.016},"last":0.53500000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:42:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58599999999999997,"15min":0.65100000000000002},"min":{"1min":0.437,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.82499999999999996,"5min":0.96699999999999997,"15min":6.766},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.60399999999999998,"15min":0.68600000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.317},"max":{"1min":0.871,"5min":0.92400000000000004,"15min":10.114000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:43:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.57999999999999996,"15min":0.65900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.35799999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":6.806},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60499999999999998,"15min":0.70899999999999996},"min":{"1min":0.49099999999999999,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.85699999999999998,"5min":0.91100000000000003,"15min":18.861000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:42:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60399999999999998,"15min":0.68999999999999995},"min":{"1min":0.439,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.86599999999999999,"5min":0.95499999999999996,"15min":6.8300000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.73699999999999999},"min":{"1min":0.52200000000000002,"5min":0.45200000000000001,"15min":0.372},"max":{"1min":0.92600000000000005,"5min":1.149,"15min":10.058999999999999},"last":0.70899999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149889,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":313724,"kb_used_data":4212,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93431428,"statfs":{"total":95995035648,"available":95673782272,"internally_reserved":0,"allocated":4313088,"data_stored":2234889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.35699999999999998,"15min":0.46300000000000002},"min":{"1min":0.223,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":0.41699999999999998,"5min":0.92200000000000004,"15min":25.082999999999998},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.379,"15min":0.51500000000000001},"min":{"1min":0.28000000000000003,"5min":0.23699999999999999,"15min":0.20599999999999999},"max":{"1min":0.50800000000000001,"5min":0.66600000000000004,"15min":18.911000000000001},"last":0.379}]},{"osd":1,"last update":"Sun Feb 18 00:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.46100000000000002},"min":{"1min":0.27700000000000002,"5min":0.23300000000000001,"15min":0.223},"max":{"1min":0.49099999999999999,"5min":0.61299999999999999,"15min":6.8479999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.379,"15min":0.68700000000000006},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.20599999999999999},"max":{"1min":0.41599999999999998,"5min":0.68999999999999995,"15min":23.606999999999999},"last":0.36499999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:42:57 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40699999999999997,"15min":0.65000000000000002},"min":{"1min":0.30399999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.48799999999999999,"5min":0.83699999999999997,"15min":23.405000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.40100000000000002,"15min":0.45800000000000002},"min":{"1min":0.26900000000000002,"5min":0.254,"15min":0.23999999999999999},"max":{"1min":0.46400000000000002,"5min":0.66300000000000003,"15min":5.3479999999999999},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57299999999999995,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.93000000000000005,"5min":0.95499999999999996,"15min":10.131},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.63},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.78300000000000003,"5min":1.1699999999999999,"15min":5.4630000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:42:56 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88600000000000001,"5min":4.7469999999999999,"15min":7.4249999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61199999999999999,"15min":0.67700000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.32400000000000001},"max":{"1min":0.81399999999999995,"5min":1.0860000000000001,"15min":6.7750000000000004},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:43:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60599999999999998,"15min":0.747},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.32300000000000001},"max":{"1min":0.78400000000000003,"5min":0.98199999999999998,"15min":10.189},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.84699999999999998,"5min":2.484,"15min":6.4279999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:42:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.67100000000000004,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.35199999999999998},"max":{"1min":0.86199999999999999,"5min":5.0830000000000002,"15min":6.8220000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.626,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.40799999999999997,"15min":0.39800000000000002},"max":{"1min":0.874,"5min":1.0760000000000001,"15min":10.164999999999999},"last":0.58299999999999996}]}]},{"osd":4,"up_from":31,"seq":133143986364,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305512,"kb_used_data":4192,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93439640,"statfs":{"total":95995035648,"available":95682191360,"internally_reserved":0,"allocated":4292608,"data_stored":2246648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50600000000000001,"15min":0.874},"min":{"1min":0.373,"5min":0.247,"15min":0.247},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":77.581999999999994},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.48799999999999999,"15min":0.84299999999999997},"min":{"1min":0.33600000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63100000000000001,"5min":0.72499999999999998,"15min":78.063999999999993},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.50700000000000001,"15min":0.62},"min":{"1min":0.379,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":9.3160000000000007},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.88200000000000001},"min":{"1min":0.35699999999999998,"5min":0.30099999999999999,"15min":0.27100000000000002},"max":{"1min":0.66000000000000003,"5min":0.73499999999999999,"15min":81.242000000000004},"last":0.36699999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72099999999999997,"5min":0.72299999999999998,"15min":11.691000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.52100000000000002,"15min":0.66000000000000003},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.27200000000000002},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":17.77},"last":0.41899999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52000000000000002,"15min":0.59499999999999997},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.28299999999999997},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":7.218},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55300000000000005,"15min":0.73099999999999998},"min":{"1min":0.40300000000000002,"5min":0.313,"15min":0.309},"max":{"1min":1.21,"5min":1.21,"15min":33.530999999999999},"last":0.499}]},{"osd":5,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44900000000000001,"15min":0.42999999999999999},"min":{"1min":0.32900000000000001,"5min":0.23799999999999999,"15min":0.20300000000000001},"max":{"1min":0.65500000000000003,"5min":0.71899999999999997,"15min":0.72299999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.45200000000000001,"15min":0.44},"min":{"1min":0.308,"5min":0.307,"15min":0.24299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.33400000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:42:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.309,"5min":0.221,"15min":0.221},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.76100000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49399999999999999,"15min":0.48099999999999998},"min":{"1min":0.36799999999999999,"5min":0.28299999999999997,"15min":0.26000000000000001},"max":{"1min":0.63500000000000001,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40799999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:42:28 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.50600000000000001},"min":{"1min":0.374,"5min":0.34599999999999997,"15min":0.27900000000000003},"max":{"1min":0.70099999999999996,"5min":0.72899999999999998,"15min":0.76900000000000002},"last":0.441},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34200000000000003,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.45900000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724727,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":301356,"kb_used_data":4132,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93443796,"statfs":{"total":95995035648,"available":95686447104,"internally_reserved":0,"allocated":4231168,"data_stored":2212178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51900000000000002,"15min":0.54300000000000004},"min":{"1min":0.35799999999999998,"5min":0.24399999999999999,"15min":0.217},"max":{"1min":2.21,"5min":2.21,"15min":6.3150000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.61199999999999999},"min":{"1min":0.311,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.782,"5min":2.1859999999999999,"15min":12.744},"last":0.38800000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.36499999999999999,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":2.1509999999999998,"5min":13.25,"15min":13.25},"last":0.437},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65700000000000003,"15min":0.66300000000000003},"min":{"1min":0.34000000000000002,"5min":0.314,"15min":0.29699999999999999},"max":{"1min":1.9430000000000001,"5min":13.262,"15min":13.262},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.53800000000000003,"15min":0.59299999999999997},"min":{"1min":0.36399999999999999,"5min":0.27700000000000002,"15min":0.251},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":6.1680000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.60799999999999998,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.312,"15min":0.308},"max":{"1min":1.8799999999999999,"5min":5.0659999999999998,"15min":6.1879999999999997},"last":0.46800000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.59799999999999998,"15min":0.63100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.28399999999999997},"max":{"1min":3.637,"5min":7.641,"15min":7.641},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.60899999999999999},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.27600000000000002},"max":{"1min":1.853,"5min":7.4580000000000002,"15min":8.8650000000000002},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:42:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.48399999999999999},"min":{"1min":0.29899999999999999,"5min":0.25,"15min":0.23599999999999999},"max":{"1min":1.897,"5min":1.897,"15min":11.818},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.496,"15min":0.47599999999999998},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":4.274},"last":0.40899999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:43:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.495,"15min":0.56000000000000005},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":2.7120000000000002,"5min":2.7120000000000002,"15min":18.605},"last":0.443},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.28299999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":4.2089999999999996},"last":0.56999999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:42:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.31900000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.819,"5min":1.819,"15min":2.0739999999999998},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.35399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.745,"5min":0.76000000000000001,"15min":4.649},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":24576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":240,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:43:27.514 INFO:tasks.thrashosds.thrasher:Setting 6.9 to [0, 7, 2, 1, 4, 3] 2024-02-18T00:43:27.514 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '6.9', '0', '7', '2', '1', '4', '3'] 2024-02-18T00:43:27.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pg-upmap-items 6.9 0 7 2 1 4 3 2024-02-18T00:43:27.592 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:27.619 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 -- 172.21.15.59:0/3016047734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef940f9610 msgr2=0x7fef94101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3016047734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef940f9610 0x7fef94101d20 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 rx=0x7fef84004870 tx=0x7fef84009cb0).stop 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 -- 172.21.15.59:0/3016047734 shutdown_connections 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3016047734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef940f8cb0 0x7fef940f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3016047734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef94102210 0x7fef94104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3016047734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef940f9610 0x7fef94101d20 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 -- 172.21.15.59:0/3016047734 >> 172.21.15.59:0/3016047734 conn(0x7fef940f2820 msgr2=0x7fef940f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:27.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 -- 172.21.15.59:0/3016047734 shutdown_connections 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 -- 172.21.15.59:0/3016047734 wait complete. 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 librados: starting msgr at 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 librados: starting objecter 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 Processor -- start 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 -- start start 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 librados: setting wanted keys 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.198+0000 7fef9bee0700 1 librados: calling monclient init 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9bee0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef940f8cb0 0x7fef9410e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9bee0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef940f9610 0x7fef9410e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9bee0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef94102210 0x7fef941083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9bee0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef941088c0 con 0x7fef940f8cb0 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9bee0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef94108a30 con 0x7fef940f9610 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9bee0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef94108ba0 con 0x7fef94102210 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef940f8cb0 0x7fef9410e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef940f8cb0 0x7fef9410e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49726/0 (socket says 172.21.15.59:49726) 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 -- 172.21.15.59:0/3254366775 learned_addr learned my addr 172.21.15.59:0/3254366775 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef94102210 0x7fef941083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9947b700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef940f9610 0x7fef9410e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 -- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef94102210 msgr2=0x7fef941083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef94102210 0x7fef941083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 -- 172.21.15.59:0/3254366775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef940f9610 msgr2=0x7fef9410e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef940f9610 0x7fef9410e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 -- 172.21.15.59:0/3254366775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef84005040 con 0x7fef940f8cb0 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef94102210 0x7fef941083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef940f8cb0 0x7fef9410e1f0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7fef900041d0 tx=0x7fef90009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef8affd700 10 client.?.objecter ms_handle_connect 0x7fef940f8cb0 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef8affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:27.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90007b40 con 0x7fef940f8cb0 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.199+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef94108e80 con 0x7fef940f8cb0 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.200+0000 7fef9bee0700 10 client.31809.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.200+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef94115060 con 0x7fef940f8cb0 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.200+0000 7fef9bee0700 1 librados: init done 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.200+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef9000f040 con 0x7fef940f8cb0 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.200+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90014750 con 0x7fef940f8cb0 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.200+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fef78004f30 con 0x7fef940f8cb0 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.201+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fef90014b80 con 0x7fef940f8cb0 2024-02-18T00:43:27.641 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:43:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.201+0000 7fef8affd700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fef80062c90 0x7fef80065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.201+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(661..661 src has 418..661) v4 ==== 8660+0+0 (secure 0 0 0) 0x7fef9007d630 con 0x7fef940f8cb0 2024-02-18T00:43:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.201+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 osd_map(661..661 src has 418..661) v4 2024-02-18T00:43:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.201+0000 7fef8affd700 3 client.31809.objecter handle_osd_map got epochs [661,661] > 0 2024-02-18T00:43:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.201+0000 7fef8affd700 3 client.31809.objecter handle_osd_map decoding full epoch 661 2024-02-18T00:43:27.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.201+0000 7fef9947b700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fef80062c90 0x7fef80065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef8affd700 20 client.31809.objecter dump_active .. 0 homeless 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v661) v1 ==== 54+0+16 (secure 0 0 0) 0x7fef9007da50 con 0x7fef940f8cb0 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9947b700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fef80062c90 0x7fef80065150 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 rx=0x7fef84009a00 tx=0x7fef8400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef80062c90 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 10 client.31809.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 1 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6801/3700238977 conn(0x7fef78004560 0x7fef78004980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 1 2024-02-18T00:43:27.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 0 1 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef780051d0 osd=0 4 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 10 client.31809.objecter _send_command 1 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef78008960 con 0x7fef78004560 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6801/3700238977 conn(0x7fef78004560 0x7fef78004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef78004560 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.202+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6801/3700238977 conn(0x7fef78004560 0x7fef78004980 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.203+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef78008960 con 0x7fef78004560 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.203+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.203+0000 7fef8affd700 10 client.31809.objecter _finish_command 1 = system:0 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.203+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef780051d0 osd=0 4 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.203+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 0 1 2024-02-18T00:43:27.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 10 client.31809.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 2 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 2 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef780051d0 osd=0 3 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 0 2 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef780051d0 osd=0 4 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 10 client.31809.objecter _send_command 2 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.218+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef78003fd0 con 0x7fef78004560 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.219+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef78003fd0 con 0x7fef78004560 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.219+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.219+0000 7fef8affd700 10 client.31809.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.219+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef780051d0 osd=0 4 2024-02-18T00:43:27.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.219+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 0 2 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 10 client.31809.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6805/26279110 conn(0x7fef78002780 0x7fef78002bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 1 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007fb0 osd=1 4 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 10 client.31809.objecter _send_command 3 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef780086c0 con 0x7fef78002780 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6805/26279110 conn(0x7fef78002780 0x7fef78002bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef78002780 2024-02-18T00:43:27.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.220+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6805/26279110 conn(0x7fef78002780 0x7fef78002bd0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.221+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef780086c0 con 0x7fef78002780 2024-02-18T00:43:27.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.221+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:43:27.647 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:43:27.647 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:43:27.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.221+0000 7fef8affd700 10 client.31809.objecter _finish_command 3 = system:0 2024-02-18T00:43:27.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.221+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78007fb0 osd=1 4 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.221+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 1 3 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 10 client.31809.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 4 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 4 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78007fb0 osd=1 3 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 1 4 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007fb0 osd=1 4 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 10 client.31809.objecter _send_command 4 2024-02-18T00:43:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.235+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef78003890 con 0x7fef78002780 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef78003890 con 0x7fef78002780 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef8affd700 10 client.31809.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78007fb0 osd=1 4 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 1 4 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 10 client.31809.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 5 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6809/1153193637 conn(0x7fef7800aff0 0x7fef7800d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 5 2024-02-18T00:43:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 2 5 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007a80 osd=2 4 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 10 client.31809.objecter _send_command 5 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef78007dc0 con 0x7fef7800aff0 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9947b700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6809/1153193637 conn(0x7fef7800aff0 0x7fef7800d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef7800aff0 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.237+0000 7fef9947b700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6809/1153193637 conn(0x7fef7800aff0 0x7fef7800d3e0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.238+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef78007dc0 con 0x7fef7800aff0 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.238+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:43:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.238+0000 7fef8affd700 10 client.31809.objecter _finish_command 5 = system:0 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.238+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78007a80 osd=2 4 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.238+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 2 5 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 10 client.31809.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 6 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 6 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78007a80 osd=2 3 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 2 6 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007a80 osd=2 4 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 10 client.31809.objecter _send_command 6 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.252+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef78003360 con 0x7fef7800aff0 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef78003360 con 0x7fef7800aff0 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef8affd700 10 client.31809.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78007a80 osd=2 4 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 2 6 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 10 client.31809.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 7 2024-02-18T00:43:27.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6813/1805700964 conn(0x7fef7800ea50 0x7fef7800ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 7 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 3 7 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78001b90 osd=3 4 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 10 client.31809.objecter _send_command 7 2024-02-18T00:43:27.653 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef780022d0 con 0x7fef7800ea50 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6813/1805700964 conn(0x7fef7800ea50 0x7fef7800ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef7800ea50 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.254+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6813/1805700964 conn(0x7fef7800ea50 0x7fef7800ef00 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.255+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef780022d0 con 0x7fef7800ea50 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.255+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.255+0000 7fef8affd700 10 client.31809.objecter _finish_command 7 = system:0 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.255+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78001b90 osd=3 4 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.255+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 3 7 2024-02-18T00:43:27.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.269+0000 7fef9bee0700 10 client.31809.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.269+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 8 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 8 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78001b90 osd=3 3 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 3 8 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78001b90 osd=3 4 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 10 client.31809.objecter _send_command 8 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.270+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef7800fbc0 con 0x7fef7800ea50 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef7800fbc0 con 0x7fef7800ea50 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef8affd700 10 client.31809.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78001b90 osd=3 4 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 3 8 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 10 client.31809.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 9 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6800/2044387056 conn(0x7fef78012070 0x7fef78014460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 9 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 4 9 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7800f3f0 osd=4 4 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 10 client.31809.objecter _send_command 9 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.271+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef780170b0 con 0x7fef78012070 2024-02-18T00:43:27.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.272+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6800/2044387056 conn(0x7fef78012070 0x7fef78014460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.272+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef78012070 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.272+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6800/2044387056 conn(0x7fef78012070 0x7fef78014460 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.273+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef780170b0 con 0x7fef78012070 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.273+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.273+0000 7fef8affd700 10 client.31809.objecter _finish_command 9 = system:0 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.273+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef7800f3f0 osd=4 4 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.273+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 4 9 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 10 client.31809.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 10 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 10 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef7800f3f0 osd=4 3 2024-02-18T00:43:27.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 4 10 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7800f3f0 osd=4 4 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 10 client.31809.objecter _send_command 10 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.289+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef78014950 con 0x7fef78012070 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef78014950 con 0x7fef78012070 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef8affd700 10 client.31809.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef7800f3f0 osd=4 4 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 4 10 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 10 client.31809.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 11 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6804/2170432263 conn(0x7fef78015060 0x7fef780174b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 11 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 5 11 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78014950 osd=5 4 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 10 client.31809.objecter _send_command 11 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef7801a280 con 0x7fef78015060 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.291+0000 7fef9947b700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6804/2170432263 conn(0x7fef78015060 0x7fef780174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.292+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef78015060 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.292+0000 7fef9947b700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6804/2170432263 conn(0x7fef78015060 0x7fef780174b0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.292+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef7801a280 con 0x7fef78015060 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.292+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.292+0000 7fef8affd700 10 client.31809.objecter _finish_command 11 = system:0 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.292+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78014950 osd=5 4 2024-02-18T00:43:27.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.292+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 5 11 2024-02-18T00:43:27.660 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:43:27.660 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:43:27.660 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:43:27.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 10 client.31809.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 12 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 12 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef78014950 osd=5 3 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 5 12 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78014950 osd=5 4 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 10 client.31809.objecter _send_command 12 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.308+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef7801a280 con 0x7fef78015060 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef7801a280 con 0x7fef78015060 2024-02-18T00:43:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef8affd700 10 client.31809.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef78014950 osd=5 4 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 5 12 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 10 client.31809.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 13 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6808/3356009390 conn(0x7fef78019580 0x7fef78019a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 13 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 6 13 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a280 osd=6 4 2024-02-18T00:43:27.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 10 client.31809.objecter _send_command 13 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.310+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef7801a020 con 0x7fef78019580 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.311+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6808/3356009390 conn(0x7fef78019580 0x7fef78019a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.311+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef78019580 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.311+0000 7fef9a47d700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6808/3356009390 conn(0x7fef78019580 0x7fef78019a00 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.312+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef7801a020 con 0x7fef78019580 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.312+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.312+0000 7fef8affd700 10 client.31809.objecter _finish_command 13 = system:0 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.312+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef7801a280 osd=6 4 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.312+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 6 13 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 10 client.31809.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 14 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 14 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef7801a280 osd=6 3 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 6 14 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a280 osd=6 4 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 10 client.31809.objecter _send_command 14 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.328+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef7801a020 con 0x7fef78019580 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.329+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef7801a020 con 0x7fef78019580 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.329+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.329+0000 7fef8affd700 10 client.31809.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.329+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef7801a280 osd=6 4 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.329+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 6 14 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 10 client.31809.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 15 2024-02-18T00:43:27.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6812/53025171 conn(0x7fef7801ca30 0x7fef7801ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 15 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 7 15 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a020 osd=7 4 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 10 client.31809.objecter _send_command 15 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef78020850 con 0x7fef7801ca30 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6812/53025171 conn(0x7fef7801ca30 0x7fef7801ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef99c7c700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6812/53025171 conn(0x7fef7801ca30 0x7fef7801ce90 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.330+0000 7fef8affd700 10 client.31809.objecter ms_handle_connect 0x7fef7801ca30 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.331+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fef78020850 con 0x7fef7801ca30 2024-02-18T00:43:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.331+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.331+0000 7fef8affd700 10 client.31809.objecter _finish_command 15 = system:0 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.331+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef7801a020 osd=7 4 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.331+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 7 15 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.345+0000 7fef9bee0700 10 client.31809.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.345+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign -1 16 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.345+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.345+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.345+0000 7fef9bee0700 20 client.31809.objecter _get_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.345+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_remove -1 16 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.345+0000 7fef9bee0700 20 client.31809.objecter get_session s=0x7fef7801a020 osd=7 3 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.346+0000 7fef9bee0700 15 client.31809.objecter _session_command_op_assign 7 16 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.346+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a020 osd=7 4 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.346+0000 7fef9bee0700 10 client.31809.objecter _send_command 16 2024-02-18T00:43:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.346+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fef78017a60 con 0x7fef7801ca30 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef8affd700 1 -- 172.21.15.59:0/3254366775 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fef78017a60 con 0x7fef7801ca30 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef8affd700 10 client.31809.objecter ms_dispatch 0x7fef94001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef8affd700 10 client.31809.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef8affd700 20 client.31809.objecter put_session s=0x7fef7801a020 osd=7 4 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef8affd700 15 client.31809.objecter _session_command_op_remove 7 16 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 10 librados: watch_flush enter 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 10 librados: watch_flush exit 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.0 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6801/3700238977 conn(0x7fef78004560 msgr2=0x7fef78004980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6801/3700238977 conn(0x7fef78004560 0x7fef78004980 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef780051d0 osd=0 1 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.1 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6805/26279110 conn(0x7fef78002780 msgr2=0x7fef78002bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6805/26279110 conn(0x7fef78002780 0x7fef78002bd0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007fb0 osd=1 1 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.2 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6809/1153193637 conn(0x7fef7800aff0 msgr2=0x7fef7800d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6809/1153193637 conn(0x7fef7800aff0 0x7fef7800d3e0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78007a80 osd=2 1 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.3 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6813/1805700964 conn(0x7fef7800ea50 msgr2=0x7fef7800ef00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6813/1805700964 conn(0x7fef7800ea50 0x7fef7800ef00 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78001b90 osd=3 1 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.4 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6800/2044387056 conn(0x7fef78012070 msgr2=0x7fef78014460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6800/2044387056 conn(0x7fef78012070 0x7fef78014460 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9a47d700 1 -- 172.21.15.59:0/3254366775 reap_dead start 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.347+0000 7fef9a47d700 1 -- 172.21.15.59:0/3254366775 reap_dead start 2024-02-18T00:43:27.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7800f3f0 osd=4 1 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.5 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6804/2170432263 conn(0x7fef78015060 msgr2=0x7fef780174b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6804/2170432263 conn(0x7fef78015060 0x7fef780174b0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef78014950 osd=5 1 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.6 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6808/3356009390 conn(0x7fef78019580 msgr2=0x7fef78019a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6808/3356009390 conn(0x7fef78019580 0x7fef78019a00 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a280 osd=6 1 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 10 client.31809.objecter close_session for osd.7 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6812/53025171 conn(0x7fef7801ca30 msgr2=0x7fef7801ce90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.137:6812/53025171 conn(0x7fef7801ca30 0x7fef7801ce90 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 20 client.31809.objecter put_session s=0x7fef7801a020 osd=7 1 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 20 client.31809.objecter shutdown clearing up homeless session... 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 10 client.31809.objecter successfully canceled tick 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fef80062c90 msgr2=0x7fef80065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> v2:172.21.15.59:6800/3721891797 conn(0x7fef80062c90 0x7fef80065150 secure :-1 s=READY pgs=3190 cs=0 l=1 rev1=1 rx=0x7fef84009a00 tx=0x7fef8400b040).stop 2024-02-18T00:43:27.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef940f8cb0 msgr2=0x7fef9410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9a47d700 1 -- 172.21.15.59:0/3254366775 reap_dead start 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef940f8cb0 0x7fef9410e1f0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7fef900041d0 tx=0x7fef90009f60).stop 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 shutdown_connections 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 --2- 172.21.15.59:0/3254366775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef940f8cb0 0x7fef9410e1f0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 >> 172.21.15.59:0/3254366775 conn(0x7fef940f2820 msgr2=0x7fef940f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 shutdown_connections 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 -- 172.21.15.59:0/3254366775 wait complete. 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:27.348+0000 7fef9bee0700 1 librados: shutdown 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:43:27.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:43:27.909 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-18T00:43:27.967 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:27.970 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:27.970 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:28.071 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:28.097 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:28.116 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:28.119 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:28.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:28.218 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:28.224 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:28.250 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:26.883470+0000 mon.a (mon.0) 3280 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:28.250 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:26.883630+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-18T00:43:28.250 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:27.202197+0000 mon.a (mon.0) 3282 : audit [DBG] from='client.? 172.21.15.59:0/3254366775' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:28.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:26.883470+0000 mon.a (mon.0) 3280 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:26.883630+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-18T00:43:28.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:27.202197+0000 mon.a (mon.0) 3282 : audit [DBG] from='client.? 172.21.15.59:0/3254366775' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:28.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:26.883470+0000 mon.a (mon.0) 3280 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:28.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:26.883630+0000 mon.a (mon.0) 3281 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-18T00:43:28.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:27.202197+0000 mon.a (mon.0) 3282 : audit [DBG] from='client.? 172.21.15.59:0/3254366775' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:28.252 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:28.450 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:28.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:28.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:28.472 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:28.477 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:28.555 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:28.583 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:28.656 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:28.659 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:28.659 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:28.760 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:28.793 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:29.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:27.888135+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:27.907481+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:27.987073+0000 mgr.y (mgr.24454) 588 : cluster [DBG] pgmap v1014: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:28.781038+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:28.781114+0000 mon.a (mon.0) 3286 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:28.782676+0000 mon.a (mon.0) 3287 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-114"}]: dispatch 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:27.888135+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:27.907481+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:27.987073+0000 mgr.y (mgr.24454) 588 : cluster [DBG] pgmap v1014: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:29.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:28.781038+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:28.781114+0000 mon.a (mon.0) 3286 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-18T00:43:29.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:28.782676+0000 mon.a (mon.0) 3287 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-114"}]: dispatch 2024-02-18T00:43:29.142 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:29.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:29.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:29.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:29.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:27.888135+0000 mon.a (mon.0) 3283 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-18T00:43:29.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:27.907481+0000 mon.a (mon.0) 3284 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:29.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:27.987073+0000 mgr.y (mgr.24454) 588 : cluster [DBG] pgmap v1014: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:29.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:28.781038+0000 mon.a (mon.0) 3285 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:29.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:28.781114+0000 mon.a (mon.0) 3286 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-18T00:43:29.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:28.782676+0000 mon.a (mon.0) 3287 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-114"}]: dispatch 2024-02-18T00:43:29.274 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:29.912 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:43:29.912 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:43:29.912 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:43:29.912 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:43:29.913 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:43:30.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:43:30.619 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-18T00:43:30.894 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:30.969 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:29.781682+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-114"}]': finished 2024-02-18T00:43:30.969 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:29.781758+0000 mon.a (mon.0) 3289 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-18T00:43:30.969 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:29.783171+0000 mon.a (mon.0) 3290 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-114", "mode": "writeback"}]: dispatch 2024-02-18T00:43:30.969 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:29.987442+0000 mgr.y (mgr.24454) 589 : cluster [DBG] pgmap v1017: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:30.969 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:29.781682+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-114"}]': finished 2024-02-18T00:43:30.969 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:29.781758+0000 mon.a (mon.0) 3289 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-18T00:43:30.969 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:29.783171+0000 mon.a (mon.0) 3290 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-114", "mode": "writeback"}]: dispatch 2024-02-18T00:43:30.970 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:29.987442+0000 mgr.y (mgr.24454) 589 : cluster [DBG] pgmap v1017: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:29.781682+0000 mon.a (mon.0) 3288 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-114"}]': finished 2024-02-18T00:43:31.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:29.781758+0000 mon.a (mon.0) 3289 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-18T00:43:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:29.783171+0000 mon.a (mon.0) 3290 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-114", "mode": "writeback"}]: dispatch 2024-02-18T00:43:31.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:29.987442+0000 mgr.y (mgr.24454) 589 : cluster [DBG] pgmap v1017: 214 pgs: 32 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 -- 172.21.15.59:0/1742854333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 msgr2=0x7fd4300fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 --2- 172.21.15.59:0/1742854333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300fcf90 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 rx=0x7fd4240047b0 tx=0x7fd424009cb0).stop 2024-02-18T00:43:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 -- 172.21.15.59:0/1742854333 shutdown_connections 2024-02-18T00:43:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 --2- 172.21.15.59:0/1742854333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4300f7400 0x7fd430103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 --2- 172.21.15.59:0/1742854333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300fcf90 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 --2- 172.21.15.59:0/1742854333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4300fd510 0x7fd4300f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 -- 172.21.15.59:0/1742854333 >> 172.21.15.59:0/1742854333 conn(0x7fd4300f2820 msgr2=0x7fd4300f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 -- 172.21.15.59:0/1742854333 shutdown_connections 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.263+0000 7fd437418700 1 -- 172.21.15.59:0/1742854333 wait complete. 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 librados: starting msgr at 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 librados: starting objecter 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 Processor -- start 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 -- start start 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 librados: setting wanted keys 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 librados: calling monclient init 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4300f7400 0x7fd4300f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4300fd510 0x7fd4300f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4301166c0 con 0x7fd4300fcbb0 2024-02-18T00:43:31.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd430116830 con 0x7fd4300fd510 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd437418700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4301169f0 con 0x7fd4300f7400 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4351b4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4300f7400 0x7fd4300f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40252/0 (socket says 172.21.15.59:40252) 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 -- 172.21.15.59:0/578578291 learned_addr learned my addr 172.21.15.59:0/578578291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4359b5700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4300fd510 0x7fd4300f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 -- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4300f7400 msgr2=0x7fd4300f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4300f7400 0x7fd4300f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 -- 172.21.15.59:0/578578291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4300fd510 msgr2=0x7fd4300f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4300fd510 0x7fd4300f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4349b3700 1 -- 172.21.15.59:0/578578291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c005040 con 0x7fd4300fcbb0 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.264+0000 7fd4351b4700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4300f7400 0x7fd4300f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:31.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd4349b3700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300f8ea0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7fd42c009400 tx=0x7fd42c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd4227fc700 10 client.?.objecter ms_handle_connect 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd4227fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd4227fc700 1 -- 172.21.15.59:0/578578291 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c007a40 con 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd424005040 con 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd4227fc700 1 -- 172.21.15.59:0/578578291 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd42c00f040 con 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd4227fc700 1 -- 172.21.15.59:0/578578291 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c014a70 con 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd437418700 10 client.31815.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd430117080 con 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.265+0000 7fd437418700 1 librados: init done 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43005fa60 con 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd4227fc700 1 -- 172.21.15.59:0/578578291 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd42c010530 con 0x7fd4300fcbb0 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd4227fc700 1 --2- 172.21.15.59:0/578578291 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd41c062c90 0x7fd41c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:31.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd4227fc700 1 -- 172.21.15.59:0/578578291 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(665..665 src has 418..665) v4 ==== 8655+0+0 (secure 0 0 0) 0x7fd42c07d010 con 0x7fd4300fcbb0 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd4227fc700 10 client.31815.objecter ms_dispatch 0x7fd430001550 osd_map(665..665 src has 418..665) v4 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd4227fc700 3 client.31815.objecter handle_osd_map got epochs [665,665] > 0 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd4227fc700 3 client.31815.objecter handle_osd_map decoding full epoch 665 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.266+0000 7fd4351b4700 1 --2- 172.21.15.59:0/578578291 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd41c062c90 0x7fd41c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.267+0000 7fd4227fc700 20 client.31815.objecter dump_active .. 0 homeless 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.267+0000 7fd4351b4700 1 --2- 172.21.15.59:0/578578291 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd41c062c90 0x7fd41c065150 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 rx=0x7fd424009bc0 tx=0x7fd424009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.267+0000 7fd4227fc700 10 client.31815.objecter ms_handle_connect 0x7fd41c062c90 2024-02-18T00:43:31.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.271+0000 7fd4227fc700 1 -- 172.21.15.59:0/578578291 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd42c049f60 con 0x7fd4300fcbb0 2024-02-18T00:43:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:31.328 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:31.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:31.354 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:43:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:43:31.429 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:31.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.451+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fd4300f9e90 con 0x7fd4300fcbb0 2024-02-18T00:43:31.459 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:31.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:43:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:43:31.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.794+0000 7fd4227fc700 1 -- 172.21.15.59:0/578578291 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v666) v1 ==== 91+0+0 (secure 0 0 0) 0x7fd42c050290 con 0x7fd4300fcbb0 2024-02-18T00:43:31.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.801+0000 7fd437418700 10 librados: watch_flush enter 2024-02-18T00:43:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.802+0000 7fd437418700 10 librados: watch_flush exit 2024-02-18T00:43:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.802+0000 7fd437418700 20 client.31815.objecter shutdown clearing up homeless session... 2024-02-18T00:43:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.802+0000 7fd437418700 10 client.31815.objecter successfully canceled tick 2024-02-18T00:43:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd41c062c90 msgr2=0x7fd41c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 --2- 172.21.15.59:0/578578291 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd41c062c90 0x7fd41c065150 secure :-1 s=READY pgs=3191 cs=0 l=1 rev1=1 rx=0x7fd424009bc0 tx=0x7fd424009cb0).stop 2024-02-18T00:43:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 msgr2=0x7fd4300f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:31.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300f8ea0 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7fd42c009400 tx=0x7fd42c009cb0).stop 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 shutdown_connections 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 --2- 172.21.15.59:0/578578291 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd41c062c90 0x7fd41c065150 unknown :-1 s=CLOSED pgs=3191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4300fd510 0x7fd4300f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd4300f7400 0x7fd4300f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 --2- 172.21.15.59:0/578578291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd4300fcbb0 0x7fd4300f8ea0 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 >> 172.21.15.59:0/578578291 conn(0x7fd4300f2820 msgr2=0x7fd4300f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 shutdown_connections 2024-02-18T00:43:31.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 -- 172.21.15.59:0/578578291 wait complete. 2024-02-18T00:43:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:31.803+0000 7fd437418700 1 librados: shutdown 2024-02-18T00:43:31.806 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:43:31.807 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:31.808 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:31.812 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:31.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:31.913 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:31.939 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:32.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:30.779160+0000 mon.a (mon.0) 3291 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:32.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:30.787558+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-114", "mode": "writeback"}]': finished 2024-02-18T00:43:32.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:30.787627+0000 mon.a (mon.0) 3293 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-18T00:43:32.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:31.453040+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? 172.21.15.59:0/578578291' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:43:32.036 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:30.779160+0000 mon.a (mon.0) 3291 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:32.037 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:30.787558+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-114", "mode": "writeback"}]': finished 2024-02-18T00:43:32.037 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:30.787627+0000 mon.a (mon.0) 3293 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-18T00:43:32.037 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:31.453040+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? 172.21.15.59:0/578578291' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:43:32.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:30.779160+0000 mon.a (mon.0) 3291 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:32.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:30.787558+0000 mon.a (mon.0) 3292 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-114", "mode": "writeback"}]': finished 2024-02-18T00:43:32.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:30.787627+0000 mon.a (mon.0) 3293 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-18T00:43:32.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:31.453040+0000 mon.a (mon.0) 3294 : audit [INF] from='client.? 172.21.15.59:0/578578291' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:43:32.288 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:32.291 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:32.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:32.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:32.419 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:32.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:32] "GET /metrics HTTP/1.1" 200 188933 "" "Prometheus/2.33.4" 2024-02-18T00:43:32.541 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:32.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:32.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:32.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:32.677 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:32.828 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:31.502053+0000 mgr.y (mgr.24454) 590 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:32.828 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:31.783409+0000 mon.a (mon.0) 3295 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:32.828 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:31.787597+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? 172.21.15.59:0/578578291' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:31.787649+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:31.829949+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:31.987845+0000 mgr.y (mgr.24454) 591 : cluster [DBG] pgmap v1020: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.134193+0000 osd.3 (osd.3) 339 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.135211+0000 osd.3 (osd.3) 340 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:31.502053+0000 mgr.y (mgr.24454) 590 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:31.783409+0000 mon.a (mon.0) 3295 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:31.787597+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? 172.21.15.59:0/578578291' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:31.787649+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:31.829949+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:32.829 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:31.987845+0000 mgr.y (mgr.24454) 591 : cluster [DBG] pgmap v1020: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:43:32.830 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.134193+0000 osd.3 (osd.3) 339 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:43:32.830 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.135211+0000 osd.3 (osd.3) 340 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:28 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 0~128] snapc a=[a] ondisk+write+known_if_redirected e136)", 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.222686+0000", 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 598.61353171799999, 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065099489999999996, 2024-02-18T00:43:32.839 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 28 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222686+0000", 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222686+0000", 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222685+0000", 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2450000000000001e-06 2024-02-18T00:43:32.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222692+0000", 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2241e-05 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222704+0000", 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2179999999999996e-06 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222710+0000", 2024-02-18T00:43:32.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5794999999999998e-05 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222756+0000", 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8658000000000003e-05 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.222855+0000", 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016973699999999999 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:43:32.843 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.223025+0000", 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00068663000000000003 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.223711+0000", 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051602410000000003 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.228871+0000", 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.058841457 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.844 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.287713+0000", 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7420999999999999e-05 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.287760+0000", 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5474999999999999e-05 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.287786+0000", 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065099489999999996 2024-02-18T00:43:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:29 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 256~128] snapc a=[a] ondisk+write+known_if_redirected e136)", 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.288196+0000", 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 598.54802193900002, 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.070967524000000004, 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 29 2024-02-18T00:43:32.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288196+0000", 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288196+0000", 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288195+0000", 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8059999999999995e-06 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288202+0000", 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1141000000000001e-05 2024-02-18T00:43:32.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288214+0000", 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.922e-06 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288220+0000", 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6390000000000002e-05 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288257+0000", 2024-02-18T00:43:32.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0473000000000003e-05 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288307+0000", 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6013000000000004e-05 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288403+0000", 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00042509999999999998 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.288828+0000", 2024-02-18T00:43:32.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012590334999999999 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.301419+0000", 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.057660309 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.359079+0000", 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5225000000000002e-05 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.359124+0000", 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9319000000000001e-05 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.359164+0000", 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.070967524000000004 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.852 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(osd.5.0:246 17.19 e136/104 -19:9dd58b98:::temp_17.19_0_24256_2:head v 0'0, mlcod=0'0)", 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:34.397742+0000", 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 598.43847567299997, 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.023808631, 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.397742+0000", 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.397742+0000", 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.397740+0000", 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0053101390000000002 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403050+0000", 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4936e-05 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.854 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403075+0000", 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1406999999999999e-05 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403086+0000", 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4187999999999997e-05 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403141+0000", 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6684999999999999e-05 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.403167+0000", 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.018339692000000001 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.421507+0000", 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4008999999999998e-05 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:34.421551+0000", 2024-02-18T00:43:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.023808631 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:35 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 128~128] snapc b=[b,a] ondisk+write+known_if_redirected e138)", 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:36.604817+0000", 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 596.23140065200005, 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.083436367999999997, 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:43:32.857 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 35 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604817+0000", 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604817+0000", 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604816+0000", 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5280000000000003e-06 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604824+0000", 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2872e-05 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604837+0000", 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0390000000000005e-06 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604845+0000", 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1752000000000003e-05 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604887+0000", 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6851000000000001e-05 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.604954+0000", 2024-02-18T00:43:32.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013683599999999999 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.605091+0000", 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00042250399999999998 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.605513+0000", 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.034101531999999997 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.639615+0000", 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.048551496 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.688166+0000", 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5745999999999999e-05 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.688212+0000", 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1726e-05 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:36.688254+0000", 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.083436367999999997 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.26167.0:42 66.12 66:4ab997f0:test-rados-api-smithi059-82981-10::foo:head [write 128~128] snapc c=[c,b,a] ondisk+write+known_if_redirected e139)", 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:37.597494+0000", 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 595.23872363199996, 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065026311000000003, 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.863 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.26167", 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2043219380", 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 42 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597494+0000", 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597494+0000", 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597493+0000", 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8929999999999994e-06 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597502+0000", 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1167e-05 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597513+0000", 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7940000000000003e-06 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.597521+0000", 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014479580000000001 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.612000+0000", 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0443000000000003e-05 2024-02-18T00:43:32.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.612071+0000", 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020784500000000001 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 1,2", 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.612279+0000", 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024797320000000001 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.614758+0000", 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.015656066999999999 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.630414+0000", 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.032035302000000002 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.662450+0000", 2024-02-18T00:43:32.868 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4518999999999999e-05 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.662484+0000", 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6553000000000001e-05 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:37.662521+0000", 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.065026311000000003 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30748.0:9 78.2s0 78:5be0bd51:test-rados-api-smithi059-82973-12::foo:head [writefull 0~128] snapc 0=[] ondisk+write+known_if_redirected e141)", 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:33:39.311751+0000", 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 593.52446671500002, 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.038382069999999997, 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30748", 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/510272679", 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 9 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311751+0000", 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311751+0000", 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311750+0000", 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5599999999999996e-06 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311757+0000", 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0591000000000001e-05 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311768+0000", 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0659999999999998e-06 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311775+0000", 2024-02-18T00:43:32.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000115534 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311890+0000", 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000107452 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.311998+0000", 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00065203199999999998 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:32.873 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.312650+0000", 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058148760000000001 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.318465+0000", 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.031589093999999998 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.350054+0000", 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9618999999999998e-05 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:33:39.350133+0000", 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.038382069999999997 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30876.0:1 167.6s0 167:68f45781:::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e257)", 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:35.626513+0000", 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 477.20970456999999, 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0131212380000001, 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.875 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30876", 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/4143483286", 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626513+0000", 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.876 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626513+0000", 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626512+0000", 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0230000000000004e-06 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626519+0000", 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0539999999999995e-06 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626527+0000", 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7159999999999997e-06 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626533+0000", 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8000000000000002e-05 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626571+0000", 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.484e-06 2024-02-18T00:43:32.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:35.626575+0000", 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0085786489999999 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.635154+0000", 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015310399999999999 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.635307+0000", 2024-02-18T00:43:32.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000564885 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.635872+0000", 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00052139999999999999 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.636393+0000", 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0031555540000000001 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.639549+0000", 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6174999999999994e-05 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:36.639635+0000", 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0131212380000001 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.30897.0:1 171.7s0 171:fd8eaea7:nspace::oid_SimpleWritePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e264)", 2024-02-18T00:43:32.881 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:35:42.665607+0000", 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 470.17061089399999, 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.96649063599999996, 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.30897", 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/450379973", 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665607+0000", 2024-02-18T00:43:32.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665607+0000", 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665605+0000", 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3909999999999998e-06 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665613+0000", 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.218e-05 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665625+0000", 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5990000000000002e-06 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665634+0000", 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5861e-05 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665680+0000", 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9500000000000001e-06 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:42.665683+0000", 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.96409705300000004 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.629780+0000", 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000143356 2024-02-18T00:43:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.629923+0000", 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00055660099999999999 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.630480+0000", 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00054972099999999998 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.631029+0000", 2024-02-18T00:43:32.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00098787500000000004 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.632017+0000", 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0391999999999999e-05 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:35:43.632098+0000", 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.96649063599999996 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.2.18:31 62.17 62:ee291d87:test-rados-api-smithi059-82957-38::foo:head [copy-from ver 2] snapc 0=[] ondisk+write+ignore_overlay+enforce_snapc+known_if_redirected e304)", 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 430.126549764, 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027920693, 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709665+0000", 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1831000000000001e-05 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709677+0000", 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7572e-05 2024-02-18T00:43:32.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709695+0000", 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1678000000000001e-05 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709717+0000", 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0013999999999998e-05 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709777+0000", 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015211700000000001 2024-02-18T00:43:32.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709929+0000", 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014904904 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 3,5", 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.724834+0000", 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0095493409999999994 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.734383+0000", 2024-02-18T00:43:32.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4509999999999997e-06 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.734389+0000", 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003096742 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737486+0000", 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6187000000000003e-05 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737542+0000", 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6628000000000001e-05 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737589+0000", 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027920693 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40913.0:1 194.5s0 194:b5a28c71:::oid_FlushPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e330)", 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:48.680019+0000", 2024-02-18T00:43:32.893 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 404.15619928299998, 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.000655482, 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40913", 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1248208728", 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680019+0000", 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680019+0000", 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680017+0000", 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8699999999999997e-06 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680023+0000", 2024-02-18T00:43:32.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8769999999999999e-06 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680032+0000", 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5729999999999996e-06 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680039+0000", 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2058999999999998e-05 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680081+0000", 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2289999999999999e-06 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680084+0000", 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.99713135600000002 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.677215+0000", 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000111112 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.677326+0000", 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029093199999999999 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.677617+0000", 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00039891900000000002 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.678016+0000", 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0026187279999999999 2024-02-18T00:43:32.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.680635+0000", 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9344999999999997e-05 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.680674+0000", 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.000655482 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDScrubReserve(6.5 RELEASE e647)", 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:12.839123+0000", 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 19.997095358999999, 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014531959999999999, 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.839123+0000", 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.839123+0000", 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.839122+0000", 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6490000000000001e-06 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.839126+0000", 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9299999999999997e-06 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.839133+0000", 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0100000000000003e-06 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.839138+0000", 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001397835 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.840535+0000", 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4517e-05 2024-02-18T00:43:32.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.840550+0000", 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5871e-05 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:12.840576+0000", 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014531959999999999 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "pg_update_log_missing(239.1d epoch 648/644 rep_tid 669 entries 648'3 (0'0) error 239:bd34d09e:test-rados-api-smithi059-82957-111::foo:head by client.31692.0:510 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 648'1)", 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:13.811213+0000", 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 19.025004829, 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021552380999999999, 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811213+0000", 2024-02-18T00:43:32.904 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811213+0000", 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811212+0000", 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4560000000000006e-06 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811222+0000", 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1851999999999999e-05 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811234+0000", 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3380000000000004e-06 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811239+0000", 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2014e-05 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811291+0000", 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021474416999999999 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.832766+0000", 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021552380999999999 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:582 239.8 239:10000000::::head [pgnls start_epoch 650] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e650)", 2024-02-18T00:43:32.907 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:15.798066+0000", 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 17.038152342, 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019188230000000001, 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 582 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798066+0000", 2024-02-18T00:43:32.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798066+0000", 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798064+0000", 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6529999999999994e-06 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798072+0000", 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3191e-05 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798085+0000", 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9119999999999998e-06 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798093+0000", 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9267000000000001e-05 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798142+0000", 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9910000000000001e-05 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.798162+0000", 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016795930000000001 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.799841+0000", 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6498999999999997e-05 2024-02-18T00:43:32.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.799878+0000", 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010650300000000001 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:15.799984+0000", 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019188230000000001 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:797 234.6s0 234:76cde755:test-rados-api-smithi059-82957-115::bar:head [writefull 0~5] snapc 6=[6] ondisk+write+known_if_redirected e662)", 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:27.899368+0000", 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9368503459999999, 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047340079999999996, 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 797 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899368+0000", 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899368+0000", 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899366+0000", 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6719999999999993e-06 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899375+0000", 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6360000000000006e-06 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899384+0000", 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6490000000000002e-06 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899391+0000", 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3920999999999999e-05 2024-02-18T00:43:32.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899465+0000", 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3130000000000001e-06 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899472+0000", 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025174059999999998 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.901990+0000", 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010505099999999999 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.902095+0000", 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00068075299999999998 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.902775+0000", 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00051217000000000005 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.903287+0000", 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00075513000000000002 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904043+0000", 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9049999999999999e-05 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904102+0000", 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047340079999999996 2024-02-18T00:43:32.918 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:798 234.6s0 234:60ad7d05:test-rados-api-smithi059-82957-115::baz:head [delete] snapc 6=[6] ondisk+write+known_if_redirected e662)", 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:27.904303+0000", 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.931915128, 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0013479760000000001, 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:43:32.919 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 798 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904303+0000", 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904303+0000", 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904302+0000", 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1830000000000002e-06 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904307+0000", 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0430000000000008e-06 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904315+0000", 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5390000000000001e-06 2024-02-18T00:43:32.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904320+0000", 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5407000000000002e-05 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904355+0000", 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0305999999999999e-05 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904445+0000", 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00050143500000000005 2024-02-18T00:43:32.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904947+0000", 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00034529100000000001 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.905292+0000", 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030803699999999999 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.905600+0000", 2024-02-18T00:43:32.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0711000000000003e-05 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.905651+0000", 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0013479760000000001 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.0.8:115 234.6s0 234:76cde755:test-rados-api-smithi059-82957-115::bar:snapdir [list-snaps] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e665)", 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:30.810973+0000", 2024-02-18T00:43:32.924 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 2.025245027, 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0028050499999999999, 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810973+0000", 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810973+0000", 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:32.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810971+0000", 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2240000000000001e-06 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810981+0000", 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6467999999999999e-05 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810997+0000", 2024-02-18T00:43:32.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4749999999999998e-06 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811004+0000", 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4396e-05 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811048+0000", 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0390000000000004e-06 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:43:32.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811053+0000", 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025337060000000002 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813587+0000", 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4364999999999995e-05 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813681+0000", 2024-02-18T00:43:32.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6979999999999996e-05 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813778+0000", 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0028050499999999999 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.0.8:116 234.6s0 234:76cde755:test-rados-api-smithi059-82957-115::bar:head [copy-get max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e665)", 2024-02-18T00:43:32.931 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 2.0252066690000001, 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003792218, 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:43:32.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2819999999999998e-06 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811015+0000", 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4520000000000002e-06 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811019+0000", 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8539999999999999e-06 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811022+0000", 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5864000000000004e-05 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811098+0000", 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9850000000000006e-06 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting_for_map not empty", 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811106+0000", 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00269615 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813802+0000", 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8986000000000006e-05 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813881+0000", 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00092213699999999996 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.814804+0000", 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003792218 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.0.8:117 234.6s0 234:76cde755:test-rados-api-smithi059-82957-115::bar:6 [copy-get max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e665)", 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:30.819640+0000", 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 2.0165779850000001, 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093346399999999995, 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.936 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819640+0000", 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819640+0000", 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.937 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819638+0000", 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1551e-05 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819649+0000", 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8079000000000001e-05 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819667+0000", 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8539999999999992e-06 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819677+0000", 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4523e-05 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819732+0000", 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5667999999999994e-05 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.819817+0000", 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00075619499999999998 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.820573+0000", 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093346399999999995 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:816 241.10 241:08000000:%01:::head [pgnls start_epoch 666] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e666)", 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:31.791623+0000", 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 1.0445951819999999, 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002031148, 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:43:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 816 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791623+0000", 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791623+0000", 2024-02-18T00:43:32.941 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791621+0000", 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4400000000000005e-06 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791630+0000", 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2847000000000001e-05 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.942 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791643+0000", 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7970000000000003e-06 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791649+0000", 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1350999999999998e-05 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791691+0000", 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0839999999999997e-06 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.791697+0000", 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0018440100000000001 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.793541+0000", 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6910000000000002e-05 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.793568+0000", 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6263999999999999e-05 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.793654+0000", 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002031148 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(234.6s0 666/605 ECSubWriteReply(tid=770, last_complete=666'10, committed=1, applied=1))", 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:31.804743+0000", 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 1.031475057, 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016053459999999999, 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:43:32.945 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.804743+0000", 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.804743+0000", 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:43:32.946 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.804742+0000", 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0780000000000002e-06 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.804751+0000", 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0496000000000001e-05 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.804762+0000", 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7739999999999999e-06 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.947 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.804768+0000", 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001529243 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.806298+0000", 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0612000000000003e-05 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:31.806348+0000", 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016053459999999999 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.948 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.949 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.949 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:43:32.949 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:43:32.949 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:43:33.195 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:31.502053+0000 mgr.y (mgr.24454) 590 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:33.195 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:31.783409+0000 mon.a (mon.0) 3295 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:33.195 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:31.787597+0000 mon.a (mon.0) 3296 : audit [INF] from='client.? 172.21.15.59:0/578578291' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:43:33.195 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:31.787649+0000 mon.a (mon.0) 3297 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-18T00:43:33.196 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:31.829949+0000 mon.a (mon.0) 3298 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:33.196 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:31.987845+0000 mgr.y (mgr.24454) 591 : cluster [DBG] pgmap v1020: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:43:33.196 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.134193+0000 osd.3 (osd.3) 339 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:43:33.196 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.135211+0000 osd.3 (osd.3) 340 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:43:33.553 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-18T00:43:33.826 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.801+0000 7f21429a4700 1 -- 172.21.15.59:0/2682090970 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 msgr2=0x7f213c104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.801+0000 7f21429a4700 1 --2- 172.21.15.59:0/2682090970 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c104170 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 rx=0x7f21300047b0 tx=0x7f2130009cb0).stop 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.802+0000 7f21429a4700 1 -- 172.21.15.59:0/2682090970 shutdown_connections 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.802+0000 7f21429a4700 1 --2- 172.21.15.59:0/2682090970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f213c104660 0x7f213c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.802+0000 7f21429a4700 1 --2- 172.21.15.59:0/2682090970 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 0x7f213c0f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.802+0000 7f21429a4700 1 --2- 172.21.15.59:0/2682090970 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c104170 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.802+0000 7f21429a4700 1 -- 172.21.15.59:0/2682090970 >> 172.21.15.59:0/2682090970 conn(0x7f213c0f2820 msgr2=0x7f213c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.802+0000 7f21429a4700 1 -- 172.21.15.59:0/2682090970 shutdown_connections 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.802+0000 7f21429a4700 1 -- 172.21.15.59:0/2682090970 wait complete. 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 librados: starting msgr at 2024-02-18T00:43:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 librados: starting objecter 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 Processor -- start 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 -- start start 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 librados: setting wanted keys 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 librados: calling monclient init 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 0x7f213c110400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c1108f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f213c104660 0x7f213c10a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c10aa90 con 0x7f213c0f6b00 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c10abd0 con 0x7f213c104660 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f21429a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c10ad90 con 0x7f213c0f73d0 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f213bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 0x7f213c110400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c1108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:33.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f213bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 0x7f213c110400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40268/0 (socket says 172.21.15.59:40268) 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f213bfff700 1 -- 172.21.15.59:0/1022149249 learned_addr learned my addr 172.21.15.59:0/1022149249 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.803+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c1108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:41236/0 (socket says 172.21.15.59:41236) 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f2140f41700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f213c104660 0x7f213c10a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f213bfff700 1 -- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 msgr2=0x7f213c1108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f213bfff700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c1108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f213bfff700 1 -- 172.21.15.59:0/1022149249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f213c104660 msgr2=0x7f213c10a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f213bfff700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f213c104660 0x7f213c10a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f213bfff700 1 -- 172.21.15.59:0/1022149249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c005040 con 0x7f213c0f6b00 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f213b7fe700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c1108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f213bfff700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 0x7f213c110400 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7f212c009400 tx=0x7f212c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21397fa700 10 client.?.objecter ms_handle_connect 0x7f213c0f6b00 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21397fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:33.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21397fa700 1 -- 172.21.15.59:0/1022149249 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c007a40 con 0x7f213c0f6b00 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2130005040 con 0x7f213c0f6b00 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21397fa700 1 -- 172.21.15.59:0/1022149249 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212c00f040 con 0x7f213c0f6b00 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21397fa700 1 -- 172.21.15.59:0/1022149249 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c014a70 con 0x7f213c0f6b00 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21429a4700 10 client.31821.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c10b3d0 con 0x7f213c0f6b00 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.804+0000 7f21429a4700 1 librados: init done 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21397fa700 1 -- 172.21.15.59:0/1022149249 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f212c010530 con 0x7f213c0f6b00 2024-02-18T00:43:33.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21397fa700 1 --2- 172.21.15.59:0/1022149249 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2124062cd0 0x7f2124065190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:33.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21397fa700 1 -- 172.21.15.59:0/1022149249 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(667..667 src has 418..667) v4 ==== 8711+0+0 (secure 0 0 0) 0x7f212c07d080 con 0x7f213c0f6b00 2024-02-18T00:43:33.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21397fa700 10 client.31821.objecter ms_dispatch 0x7f213c001550 osd_map(667..667 src has 418..667) v4 2024-02-18T00:43:33.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21397fa700 3 client.31821.objecter handle_osd_map got epochs [667,667] > 0 2024-02-18T00:43:33.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21397fa700 3 client.31821.objecter handle_osd_map decoding full epoch 667 2024-02-18T00:43:33.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f213b7fe700 1 --2- 172.21.15.59:0/1022149249 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2124062cd0 0x7f2124065190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21397fa700 20 client.31821.objecter dump_active .. 0 homeless 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.806+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c05fa60 con 0x7f213c0f6b00 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.807+0000 7f21397fa700 10 client.31821.objecter ms_handle_connect 0x7f2124062cd0 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.807+0000 7f213b7fe700 1 --2- 172.21.15.59:0/1022149249 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2124062cd0 0x7f2124065190 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 rx=0x7f213c10bd40 tx=0x7f2130009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:32.811+0000 7f21397fa700 1 -- 172.21.15.59:0/1022149249 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f212c049f60 con 0x7f213c0f6b00 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.000+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]} v 0) v1 -- 0x7f213c109520 con 0x7f213c0f6b00 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.802+0000 7f21397fa700 1 -- 172.21.15.59:0/1022149249 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]}]=0 set 6.9 pg_upmap_items mapping to [0->7,2->1,4->3] v668) v1 ==== 157+0+0 (secure 0 0 0) 0x7f212c07ae60 con 0x7f213c0f6b00 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.806+0000 7f21429a4700 10 librados: watch_flush enter 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.809+0000 7f21429a4700 10 librados: watch_flush exit 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.809+0000 7f21429a4700 20 client.31821.objecter shutdown clearing up homeless session... 2024-02-18T00:43:33.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.809+0000 7f21429a4700 10 client.31821.objecter successfully canceled tick 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.809+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2124062cd0 msgr2=0x7f2124065190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.809+0000 7f21429a4700 1 --2- 172.21.15.59:0/1022149249 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2124062cd0 0x7f2124065190 secure :-1 s=READY pgs=3192 cs=0 l=1 rev1=1 rx=0x7f213c10bd40 tx=0x7f2130009cb0).stop 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.809+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 msgr2=0x7f213c110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.809+0000 7f21429a4700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 0x7f213c110400 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7f212c009400 tx=0x7f212c009cb0).stop 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.810+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 shutdown_connections 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.810+0000 7f21429a4700 1 --2- 172.21.15.59:0/1022149249 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2124062cd0 0x7f2124065190 unknown :-1 s=CLOSED pgs=3192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.810+0000 7f21429a4700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f213c104660 0x7f213c10a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.810+0000 7f21429a4700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f213c0f6b00 0x7f213c110400 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.810+0000 7f21429a4700 1 --2- 172.21.15.59:0/1022149249 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f213c0f73d0 0x7f213c1108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.810+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 >> 172.21.15.59:0/1022149249 conn(0x7f213c0f2820 msgr2=0x7f213c0f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.815+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 shutdown_connections 2024-02-18T00:43:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.815+0000 7f21429a4700 1 -- 172.21.15.59:0/1022149249 wait complete. 2024-02-18T00:43:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:33.815+0000 7f21429a4700 1 librados: shutdown 2024-02-18T00:43:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:set 6.9 pg_upmap_items mapping to [0->7,2->1,4->3] 2024-02-18T00:43:33.859 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:33.861 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:33.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:33.963 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:33.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.075921+0000 osd.2 (osd.2) 269 : cluster [DBG] 5.d scrub starts 2024-02-18T00:43:33.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.077463+0000 osd.2 (osd.2) 270 : cluster [DBG] 5.d scrub ok 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.498248+0000 osd.0 (osd.0) 296 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.499128+0000 osd.0 (osd.0) 297 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:32.801673+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.801743+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:32.803607+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114"}]: dispatch 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.815816+0000 osd.5 (osd.5) 283 : cluster [DBG] 25.c scrub starts 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:32.818705+0000 osd.5 (osd.5) 284 : cluster [DBG] 25.c scrub ok 2024-02-18T00:43:33.992 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:33.001776+0000 mon.a (mon.0) 3302 : audit [INF] from='client.? 172.21.15.59:0/1022149249' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]}]: dispatch 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.075921+0000 osd.2 (osd.2) 269 : cluster [DBG] 5.d scrub starts 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.077463+0000 osd.2 (osd.2) 270 : cluster [DBG] 5.d scrub ok 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.498248+0000 osd.0 (osd.0) 296 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.499128+0000 osd.0 (osd.0) 297 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:32.801673+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.801743+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:32.803607+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114"}]: dispatch 2024-02-18T00:43:33.993 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.815816+0000 osd.5 (osd.5) 283 : cluster [DBG] 25.c scrub starts 2024-02-18T00:43:33.994 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:32.818705+0000 osd.5 (osd.5) 284 : cluster [DBG] 25.c scrub ok 2024-02-18T00:43:33.994 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:33.001776+0000 mon.a (mon.0) 3302 : audit [INF] from='client.? 172.21.15.59:0/1022149249' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]}]: dispatch 2024-02-18T00:43:33.994 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.075921+0000 osd.2 (osd.2) 269 : cluster [DBG] 5.d scrub starts 2024-02-18T00:43:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.077463+0000 osd.2 (osd.2) 270 : cluster [DBG] 5.d scrub ok 2024-02-18T00:43:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.498248+0000 osd.0 (osd.0) 296 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:43:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.499128+0000 osd.0 (osd.0) 297 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:43:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:32.801673+0000 mon.a (mon.0) 3299 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.801743+0000 mon.a (mon.0) 3300 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-18T00:43:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:32.803607+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114"}]: dispatch 2024-02-18T00:43:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.815816+0000 osd.5 (osd.5) 283 : cluster [DBG] 25.c scrub starts 2024-02-18T00:43:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:32.818705+0000 osd.5 (osd.5) 284 : cluster [DBG] 25.c scrub ok 2024-02-18T00:43:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:33.001776+0000 mon.a (mon.0) 3302 : audit [INF] from='client.? 172.21.15.59:0/1022149249' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]}]: dispatch 2024-02-18T00:43:34.118 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:34.120 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:34.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:34.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:34.250 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:34.277 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:34.279 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:34.280 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:34.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:34.410 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:34.759 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:34.761 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:34.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:34.862 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:34.889 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:35.001 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:33.516727+0000 osd.0 (osd.0) 298 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:43:35.001 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:33.517946+0000 osd.0 (osd.0) 299 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:33.797541+0000 mon.a (mon.0) 3303 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:33.802988+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114"}]': finished 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:33.803095+0000 mon.a (mon.0) 3305 : audit [INF] from='client.? 172.21.15.59:0/1022149249' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]}]': finished 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:33.803177+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:33.988244+0000 mgr.y (mgr.24454) 592 : cluster [DBG] pgmap v1023: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.003054+0000 osd.7 (osd.7) 195 : cluster [DBG] 2.b scrub starts 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.003917+0000 osd.7 (osd.7) 196 : cluster [DBG] 2.b scrub ok 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.072997+0000 osd.1 (osd.1) 323 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.073796+0000 osd.1 (osd.1) 324 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.566157+0000 osd.4 (osd.4) 369 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:43:35.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.567361+0000 osd.4 (osd.4) 370 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:33.516727+0000 osd.0 (osd.0) 298 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:33.517946+0000 osd.0 (osd.0) 299 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:33.797541+0000 mon.a (mon.0) 3303 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:33.802988+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114"}]': finished 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:33.803095+0000 mon.a (mon.0) 3305 : audit [INF] from='client.? 172.21.15.59:0/1022149249' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]}]': finished 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:33.803177+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:33.988244+0000 mgr.y (mgr.24454) 592 : cluster [DBG] pgmap v1023: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.003054+0000 osd.7 (osd.7) 195 : cluster [DBG] 2.b scrub starts 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.003917+0000 osd.7 (osd.7) 196 : cluster [DBG] 2.b scrub ok 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.072997+0000 osd.1 (osd.1) 323 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.073796+0000 osd.1 (osd.1) 324 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:43:35.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.566157+0000 osd.4 (osd.4) 369 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:43:35.004 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.567361+0000 osd.4 (osd.4) 370 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:43:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:33.516727+0000 osd.0 (osd.0) 298 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:43:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:33.517946+0000 osd.0 (osd.0) 299 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:43:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:33.797541+0000 mon.a (mon.0) 3303 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:35.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:33.802988+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-114"}]': finished 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:33.803095+0000 mon.a (mon.0) 3305 : audit [INF] from='client.? 172.21.15.59:0/1022149249' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "6.9", "id": [0, 7, 2, 1, 4, 3]}]': finished 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:33.803177+0000 mon.a (mon.0) 3306 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:33.988244+0000 mgr.y (mgr.24454) 592 : cluster [DBG] pgmap v1023: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.003054+0000 osd.7 (osd.7) 195 : cluster [DBG] 2.b scrub starts 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.003917+0000 osd.7 (osd.7) 196 : cluster [DBG] 2.b scrub ok 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.072997+0000 osd.1 (osd.1) 323 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.073796+0000 osd.1 (osd.1) 324 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.566157+0000 osd.4 (osd.4) 369 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:43:35.005 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.567361+0000 osd.4 (osd.4) 370 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:43:35.089 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:35.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:35.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:35.193 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:35.222 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:35.320 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:35.322 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:35.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:35.424 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:35.451 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:35.470 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:35.473 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:35.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:35.574 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:35.597 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:43:35.597 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:43:35.597 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:43:35.597 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:43:35.601 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:35.621 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:35.623 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:35.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:35.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:43:35.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:35.754 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:35.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.110336+0000 osd.2 (osd.2) 271 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:43:35.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.111125+0000 osd.2 (osd.2) 272 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:43:35.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.783617+0000 osd.5 (osd.5) 285 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:43:35.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.784441+0000 osd.5 (osd.5) 286 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:43:35.927 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.810984+0000 mon.a (mon.0) 3307 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:35.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:34.821189+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-18T00:43:35.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.020853+0000 osd.7 (osd.7) 197 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:43:35.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.023925+0000 osd.7 (osd.7) 198 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:43:35.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 -- 172.21.15.59:0/1702054193 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380f7400 msgr2=0x7efd38103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:35.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 --2- 172.21.15.59:0/1702054193 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380f7400 0x7efd38103a20 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 rx=0x7efd30009be0 tx=0x7efd30004d10).stop 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 -- 172.21.15.59:0/1702054193 shutdown_connections 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 --2- 172.21.15.59:0/1702054193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd380fcbb0 0x7efd380fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 --2- 172.21.15.59:0/1702054193 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380f7400 0x7efd38103a20 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 --2- 172.21.15.59:0/1702054193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd380fd510 0x7efd380f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 -- 172.21.15.59:0/1702054193 >> 172.21.15.59:0/1702054193 conn(0x7efd380f2820 msgr2=0x7efd380f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 -- 172.21.15.59:0/1702054193 shutdown_connections 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.967+0000 7efd3fa42700 1 -- 172.21.15.59:0/1702054193 wait complete. 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 librados: starting msgr at 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 librados: starting objecter 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 Processor -- start 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 -- start start 2024-02-18T00:43:35.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 librados: setting wanted keys 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 librados: calling monclient init 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd380f7400 0x7efd38110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd380fcbb0 0x7efd381106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380fd510 0x7efd3810a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3810a890 con 0x7efd380fcbb0 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3810aa00 con 0x7efd380f7400 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3fa42700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3810abc0 con 0x7efd380fd510 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3cfdd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd380fcbb0 0x7efd381106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3dfdf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380fd510 0x7efd3810a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3dfdf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380fd510 0x7efd3810a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:41250/0 (socket says 172.21.15.59:41250) 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3cfdd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd380fcbb0 0x7efd381106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40284/0 (socket says 172.21.15.59:40284) 2024-02-18T00:43:35.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.968+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 learned_addr learned my addr 172.21.15.59:0/2827586909 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd380f7400 0x7efd38110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd380f7400 msgr2=0x7efd38110200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3dfdf700 1 --2- 172.21.15.59:0/2827586909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd380f7400 0x7efd38110200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd380fcbb0 msgr2=0x7efd381106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3dfdf700 1 --2- 172.21.15.59:0/2827586909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd380fcbb0 0x7efd381106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c005040 con 0x7efd380fd510 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3cfdd700 1 --2- 172.21.15.59:0/2827586909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd380fcbb0 0x7efd381106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3dfdf700 1 --2- 172.21.15.59:0/2827586909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380fd510 0x7efd3810a3a0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 rx=0x7efd30004ed0 tx=0x7efd30004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd2a7fc700 10 client.?.objecter ms_handle_connect 0x7efd380fd510 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd2a7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3000e040 con 0x7efd380fd510 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd30018040 con 0x7efd380fd510 2024-02-18T00:43:35.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd300097c0 con 0x7efd380fd510 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3001c3f0 con 0x7efd380fd510 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3fa42700 10 client.41636.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3810b060 con 0x7efd380fd510 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.969+0000 7efd3fa42700 1 librados: init done 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.970+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7efd3810b250 con 0x7efd380fd510 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.970+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efd30013420 con 0x7efd380fd510 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd24062c90 0x7efd24065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(670..670 src has 418..670) v4 ==== 8740+0+0 (secure 0 0 0) 0x7efd3007e4b0 con 0x7efd380fd510 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 osd_map(670..670 src has 418..670) v4 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 3 client.41636.objecter handle_osd_map got epochs [670,670] > 0 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 3 client.41636.objecter handle_osd_map decoding full epoch 670 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd24062c90 0x7efd24065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:35.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 20 client.41636.objecter dump_active .. 0 homeless 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v670) v1 ==== 54+0+16 (secure 0 0 0) 0x7efd30085070 con 0x7efd380fd510 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd24062c90 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd24062c90 0x7efd24065150 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 rx=0x7efd340041d0 tx=0x7efd34009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 10 client.41636.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 1 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd380f9d10 0x7efd380fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 1 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 0 1 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3805fa60 osd=0 4 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 10 client.41636.objecter _send_command 1 2024-02-18T00:43:35.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd3805ee80 con 0x7efd380f9d10 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.971+0000 7efd3cfdd700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd380f9d10 0x7efd380fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.972+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd380f9d10 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.972+0000 7efd3cfdd700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd380f9d10 0x7efd380fa0f0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.972+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd3805ee80 con 0x7efd380f9d10 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.972+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.972+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 1 = system:0 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.972+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3805fa60 osd=0 4 2024-02-18T00:43:35.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.972+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 0 1 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 10 client.41636.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 2 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 2 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3805fa60 osd=0 3 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 0 2 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3805fa60 osd=0 4 2024-02-18T00:43:35.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 10 client.41636.objecter _send_command 2 2024-02-18T00:43:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.988+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd38106120 con 0x7efd380f9d10 2024-02-18T00:43:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd38106120 con 0x7efd380f9d10 2024-02-18T00:43:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3805fa60 osd=0 4 2024-02-18T00:43:35.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 0 2 2024-02-18T00:43:35.998 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 10 client.41636.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 3 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6805/26279110 conn(0x7efd380598f0 0x7efd38059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 3 2024-02-18T00:43:35.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 1 3 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38028540 osd=1 4 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 10 client.41636.objecter _send_command 3 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.989+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd380286b0 con 0x7efd380598f0 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6805/26279110 conn(0x7efd380598f0 0x7efd38059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd380598f0 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6805/26279110 conn(0x7efd380598f0 0x7efd38059cd0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd380286b0 con 0x7efd380598f0 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 3 = system:0 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd38028540 osd=1 4 2024-02-18T00:43:36.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:35.990+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 1 3 2024-02-18T00:43:36.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.110336+0000 osd.2 (osd.2) 271 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:43:36.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.111125+0000 osd.2 (osd.2) 272 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:43:36.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.783617+0000 osd.5 (osd.5) 285 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:43:36.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.784441+0000 osd.5 (osd.5) 286 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.810984+0000 mon.a (mon.0) 3307 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:34.821189+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.020853+0000 osd.7 (osd.7) 197 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.023925+0000 osd.7 (osd.7) 198 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.110336+0000 osd.2 (osd.2) 271 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.111125+0000 osd.2 (osd.2) 272 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.783617+0000 osd.5 (osd.5) 285 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.784441+0000 osd.5 (osd.5) 286 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.810984+0000 mon.a (mon.0) 3307 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:34.821189+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.020853+0000 osd.7 (osd.7) 197 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:43:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.023925+0000 osd.7 (osd.7) 198 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:43:36.007 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 10 client.41636.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 4 2024-02-18T00:43:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 4 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd38028540 osd=1 3 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 1 4 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38028540 osd=1 4 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 10 client.41636.objecter _send_command 4 2024-02-18T00:43:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.007+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd38059360 con 0x7efd380598f0 2024-02-18T00:43:36.010 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:36.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:36.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd38059360 con 0x7efd380598f0 2024-02-18T00:43:36.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:36.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd38028540 osd=1 4 2024-02-18T00:43:36.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 1 4 2024-02-18T00:43:36.011 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:43:36.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 10 client.41636.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:36.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 5 2024-02-18T00:43:36.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd3811c1d0 0x7efd3811e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:36.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 5 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 2 5 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38027b90 osd=2 4 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 10 client.41636.objecter _send_command 5 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd38028210 con 0x7efd3811c1d0 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.009+0000 7efd3cfdd700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd3811c1d0 0x7efd3811e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.010+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd3811c1d0 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.010+0000 7efd3cfdd700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd3811c1d0 0x7efd3811e5c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.010+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd38028210 con 0x7efd3811c1d0 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.010+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:43:36.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.010+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 5 = system:0 2024-02-18T00:43:36.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.010+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd38027b90 osd=2 4 2024-02-18T00:43:36.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.010+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 2 5 2024-02-18T00:43:36.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 10 client.41636.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 6 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 6 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd38027b90 osd=2 3 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 2 6 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38027b90 osd=2 4 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 10 client.41636.objecter _send_command 6 2024-02-18T00:43:36.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.026+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd38027600 con 0x7efd3811c1d0 2024-02-18T00:43:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.027+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd38027600 con 0x7efd3811c1d0 2024-02-18T00:43:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.027+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.027+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.027+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd38027b90 osd=2 4 2024-02-18T00:43:36.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.027+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 2 6 2024-02-18T00:43:36.029 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 10 client.41636.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 7 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd3811fc30 0x7efd38122020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 7 2024-02-18T00:43:36.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 3 7 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b820 osd=3 4 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 10 client.41636.objecter _send_command 7 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd3811bea0 con 0x7efd3811fc30 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3dfdf700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd3811fc30 0x7efd38122020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd3811fc30 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.028+0000 7efd3dfdf700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd3811fc30 0x7efd38122020 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.029+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd3811bea0 con 0x7efd3811fc30 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.029+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:43:36.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.029+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 7 = system:0 2024-02-18T00:43:36.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.029+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3811b820 osd=3 4 2024-02-18T00:43:36.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.029+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 3 7 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 10 client.41636.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 8 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 8 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3811b820 osd=3 3 2024-02-18T00:43:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 3 8 2024-02-18T00:43:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b820 osd=3 4 2024-02-18T00:43:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 10 client.41636.objecter _send_command 8 2024-02-18T00:43:36.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.045+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd3811b290 con 0x7efd3811fc30 2024-02-18T00:43:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.047+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd3811b290 con 0x7efd3811fc30 2024-02-18T00:43:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.047+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.047+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.047+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3811b820 osd=3 4 2024-02-18T00:43:36.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.047+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 3 8 2024-02-18T00:43:36.049 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:43:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 10 client.41636.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 9 2024-02-18T00:43:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd3811b400 0x7efd381256a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 9 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 4 9 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b290 osd=4 4 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 10 client.41636.objecter _send_command 9 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd3811f790 con 0x7efd3811b400 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd3811b400 0x7efd381256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd3811b400 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.048+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd3811b400 0x7efd381256a0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.049+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd3811f790 con 0x7efd3811b400 2024-02-18T00:43:36.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.049+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:43:36.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.049+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 9 = system:0 2024-02-18T00:43:36.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.049+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3811b290 osd=4 4 2024-02-18T00:43:36.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.049+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 4 9 2024-02-18T00:43:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 10 client.41636.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 10 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 10 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3811b290 osd=4 3 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 4 10 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b290 osd=4 4 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 10 client.41636.objecter _send_command 10 2024-02-18T00:43:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.076+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd3811f790 con 0x7efd3811b400 2024-02-18T00:43:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd3811f790 con 0x7efd3811b400 2024-02-18T00:43:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3811b290 osd=4 4 2024-02-18T00:43:36.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 4 10 2024-02-18T00:43:36.080 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 10 client.41636.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 11 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd381260a0 0x7efd38128490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 11 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 5 11 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811f790 osd=5 4 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 10 client.41636.objecter _send_command 11 2024-02-18T00:43:36.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd3812b0e0 con 0x7efd381260a0 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.078+0000 7efd3cfdd700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd381260a0 0x7efd38128490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.079+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd381260a0 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.079+0000 7efd3cfdd700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd381260a0 0x7efd38128490 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.079+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd3812b0e0 con 0x7efd381260a0 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.079+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.079+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 11 = system:0 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.079+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3811f790 osd=5 4 2024-02-18T00:43:36.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.079+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 5 11 2024-02-18T00:43:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 10 client.41636.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 12 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 12 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3811f790 osd=5 3 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 5 12 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811f790 osd=5 4 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 10 client.41636.objecter _send_command 12 2024-02-18T00:43:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.101+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd38128980 con 0x7efd381260a0 2024-02-18T00:43:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd38128980 con 0x7efd381260a0 2024-02-18T00:43:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3811f790 osd=5 4 2024-02-18T00:43:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 5 12 2024-02-18T00:43:36.105 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 10 client.41636.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 13 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd38129000 0x7efd3812b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 13 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 6 13 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38128980 osd=6 4 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 10 client.41636.objecter _send_command 13 2024-02-18T00:43:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd3812e280 con 0x7efd38129000 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3dfdf700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd38129000 0x7efd3812b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd38129000 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.103+0000 7efd3dfdf700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd38129000 0x7efd3812b470 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.104+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd3812e280 con 0x7efd38129000 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.104+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.104+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 13 = system:0 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.104+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd38128980 osd=6 4 2024-02-18T00:43:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.104+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 6 13 2024-02-18T00:43:36.112 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 10 client.41636.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 14 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 14 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd38128980 osd=6 3 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 6 14 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38128980 osd=6 4 2024-02-18T00:43:36.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 10 client.41636.objecter _send_command 14 2024-02-18T00:43:36.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.123+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd3812e240 con 0x7efd38129000 2024-02-18T00:43:36.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd3812e240 con 0x7efd38129000 2024-02-18T00:43:36.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:36.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd38128980 osd=6 4 2024-02-18T00:43:36.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 6 14 2024-02-18T00:43:36.127 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 10 client.41636.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 15 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6812/53025171 conn(0x7efd3812d5a0 0x7efd3812da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 15 2024-02-18T00:43:36.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 7 15 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3812e240 osd=7 4 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 10 client.41636.objecter _send_command 15 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.125+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd381315e0 con 0x7efd3812d5a0 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.126+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6812/53025171 conn(0x7efd3812d5a0 0x7efd3812da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.126+0000 7efd2a7fc700 10 client.41636.objecter ms_handle_connect 0x7efd3812d5a0 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.126+0000 7efd3d7de700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6812/53025171 conn(0x7efd3812d5a0 0x7efd3812da20 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.127+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7efd381315e0 con 0x7efd3812d5a0 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.127+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.127+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 15 = system:0 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.127+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3812e240 osd=7 4 2024-02-18T00:43:36.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.127+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 7 15 2024-02-18T00:43:36.141 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 10 client.41636.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign -1 16 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 20 client.41636.objecter _get_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_remove -1 16 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 20 client.41636.objecter get_session s=0x7efd3812e240 osd=7 3 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 15 client.41636.objecter _session_command_op_assign 7 16 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3812e240 osd=7 4 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 10 client.41636.objecter _send_command 16 2024-02-18T00:43:36.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.144+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7efd3812ba20 con 0x7efd3812d5a0 2024-02-18T00:43:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd2a7fc700 1 -- 172.21.15.59:0/2827586909 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7efd3812ba20 con 0x7efd3812d5a0 2024-02-18T00:43:36.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd2a7fc700 10 client.41636.objecter ms_dispatch 0x7efd38001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd2a7fc700 10 client.41636.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd2a7fc700 20 client.41636.objecter put_session s=0x7efd3812e240 osd=7 4 2024-02-18T00:43:36.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd2a7fc700 15 client.41636.objecter _session_command_op_remove 7 16 2024-02-18T00:43:36.148 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:43:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 librados: watch_flush enter 2024-02-18T00:43:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 librados: watch_flush exit 2024-02-18T00:43:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.0 2024-02-18T00:43:36.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd380f9d10 msgr2=0x7efd380fa0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6801/3700238977 conn(0x7efd380f9d10 0x7efd380fa0f0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3805fa60 osd=0 1 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.1 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6805/26279110 conn(0x7efd380598f0 msgr2=0x7efd38059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6805/26279110 conn(0x7efd380598f0 0x7efd38059cd0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38028540 osd=1 1 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.2 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd3811c1d0 msgr2=0x7efd3811e5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6809/1153193637 conn(0x7efd3811c1d0 0x7efd3811e5c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38027b90 osd=2 1 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.3 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd3811fc30 msgr2=0x7efd38122020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6813/1805700964 conn(0x7efd3811fc30 0x7efd38122020 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b820 osd=3 1 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.4 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd3811b400 msgr2=0x7efd381256a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6800/2044387056 conn(0x7efd3811b400 0x7efd381256a0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811b290 osd=4 1 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.5 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd381260a0 msgr2=0x7efd38128490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6804/2170432263 conn(0x7efd381260a0 0x7efd38128490 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3811f790 osd=5 1 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.6 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd38129000 msgr2=0x7efd3812b470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.146+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6808/3356009390 conn(0x7efd38129000 0x7efd3812b470 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd38128980 osd=6 1 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 10 client.41636.objecter close_session for osd.7 2024-02-18T00:43:36.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6812/53025171 conn(0x7efd3812d5a0 msgr2=0x7efd3812da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.137:6812/53025171 conn(0x7efd3812d5a0 0x7efd3812da20 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 20 client.41636.objecter put_session s=0x7efd3812e240 osd=7 1 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 20 client.41636.objecter shutdown clearing up homeless session... 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 10 client.41636.objecter successfully canceled tick 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 reap_dead start 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd24062c90 msgr2=0x7efd24065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> v2:172.21.15.59:6800/3721891797 conn(0x7efd24062c90 0x7efd24065150 secure :-1 s=READY pgs=3193 cs=0 l=1 rev1=1 rx=0x7efd340041d0 tx=0x7efd34009040).stop 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 reap_dead start 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380fd510 msgr2=0x7efd3810a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 --2- 172.21.15.59:0/2827586909 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd380fd510 0x7efd3810a3a0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 rx=0x7efd30004ed0 tx=0x7efd30004d10).stop 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 reap_dead start 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 reap_dead start 2024-02-18T00:43:36.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3dfdf700 1 -- 172.21.15.59:0/2827586909 reap_dead start 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 shutdown_connections 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 >> 172.21.15.59:0/2827586909 conn(0x7efd380f2820 msgr2=0x7efd38060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 shutdown_connections 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 -- 172.21.15.59:0/2827586909 wait complete. 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:36.147+0000 7efd3fa42700 1 librados: shutdown 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.153 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:43:36.327 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-18T00:43:36.593 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:36.692 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:36.695 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:36.696 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:36.797 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:36.825 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:36.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.105281+0000 osd.2 (osd.2) 273 : cluster [DBG] 6.f scrub starts 2024-02-18T00:43:36.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.108754+0000 osd.2 (osd.2) 274 : cluster [DBG] 6.f scrub ok 2024-02-18T00:43:36.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.525096+0000 osd.0 (osd.0) 300 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:43:36.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.528563+0000 osd.0 (osd.0) 301 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:43:36.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.826269+0000 mon.a (mon.0) 3309 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-18T00:43:36.986 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:35.835184+0000 mon.a (mon.0) 3310 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:35.972142+0000 mon.c (mon.2) 562 : audit [DBG] from='client.? 172.21.15.59:0/2827586909' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:35.988667+0000 mgr.y (mgr.24454) 593 : cluster [DBG] pgmap v1026: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:36.569736+0000 osd.4 (osd.4) 371 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:36.572960+0000 osd.4 (osd.4) 372 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.105281+0000 osd.2 (osd.2) 273 : cluster [DBG] 6.f scrub starts 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.108754+0000 osd.2 (osd.2) 274 : cluster [DBG] 6.f scrub ok 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.525096+0000 osd.0 (osd.0) 300 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.528563+0000 osd.0 (osd.0) 301 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.826269+0000 mon.a (mon.0) 3309 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:35.835184+0000 mon.a (mon.0) 3310 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:36.987 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:35.972142+0000 mon.c (mon.2) 562 : audit [DBG] from='client.? 172.21.15.59:0/2827586909' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:36.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:35.988667+0000 mgr.y (mgr.24454) 593 : cluster [DBG] pgmap v1026: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:36.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:36.569736+0000 osd.4 (osd.4) 371 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:43:36.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:36.572960+0000 osd.4 (osd.4) 372 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:43:37.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.105281+0000 osd.2 (osd.2) 273 : cluster [DBG] 6.f scrub starts 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.108754+0000 osd.2 (osd.2) 274 : cluster [DBG] 6.f scrub ok 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.525096+0000 osd.0 (osd.0) 300 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.528563+0000 osd.0 (osd.0) 301 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.826269+0000 mon.a (mon.0) 3309 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:35.835184+0000 mon.a (mon.0) 3310 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:35.972142+0000 mon.c (mon.2) 562 : audit [DBG] from='client.? 172.21.15.59:0/2827586909' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:35.988667+0000 mgr.y (mgr.24454) 593 : cluster [DBG] pgmap v1026: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:36.569736+0000 osd.4 (osd.4) 371 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:43:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:36.572960+0000 osd.4 (osd.4) 372 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:43:37.375 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:37.378 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:37.379 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:37.479 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:37.510 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:37.609 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:37.612 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:37.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:37.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:37.749 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:37.777 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:37.777 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:37.878 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:37.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:36.515816+0000 osd.6 (osd.6) 301 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:43:37.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:36.516696+0000 osd.6 (osd.6) 302 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:43:37.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:36.836365+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:37.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:36.836432+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-18T00:43:37.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:36.842866+0000 mon.a (mon.0) 3313 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:37.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.191750+0000 osd.2 (osd.2) 275 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:43:37.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.193684+0000 osd.2 (osd.2) 276 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:43:37.906 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.204261+0000 osd.3 (osd.3) 341 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:43:37.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.205084+0000 osd.3 (osd.3) 342 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:43:37.907 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:37.961 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:37.963 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:37.963 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:36.515816+0000 osd.6 (osd.6) 301 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:36.516696+0000 osd.6 (osd.6) 302 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:36.836365+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:36.836432+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:36.842866+0000 mon.a (mon.0) 3313 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.191750+0000 osd.2 (osd.2) 275 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.193684+0000 osd.2 (osd.2) 276 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.204261+0000 osd.3 (osd.3) 341 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:43:38.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.205084+0000 osd.3 (osd.3) 342 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:36.515816+0000 osd.6 (osd.6) 301 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:36.516696+0000 osd.6 (osd.6) 302 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:36.836365+0000 mon.a (mon.0) 3311 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:36.836432+0000 mon.a (mon.0) 3312 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:36.842866+0000 mon.a (mon.0) 3313 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.191750+0000 osd.2 (osd.2) 275 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.193684+0000 osd.2 (osd.2) 276 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.204261+0000 osd.3 (osd.3) 341 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:43:38.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.205084+0000 osd.3 (osd.3) 342 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:43:38.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:38.098 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:38.299 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:38.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:38.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:38.404 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:38.432 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:38.505 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:38.508 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:38.508 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:38.609 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:38.637 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:38.759 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:38.762 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:38.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:38.863 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:38.892 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:38.919 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:38.922 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:38.922 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 -- 172.21.15.59:0/1461481129 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fd4a0 msgr2=0x7f10bc0f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 --2- 172.21.15.59:0/1461481129 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fd4a0 0x7f10bc0f74c0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 rx=0x7f10a40047b0 tx=0x7f10a4009cb0).stop 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 -- 172.21.15.59:0/1461481129 shutdown_connections 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 --2- 172.21.15.59:0/1461481129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc0f7ba0 0x7f10bc103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 --2- 172.21.15.59:0/1461481129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0fcb40 0x7f10bc0fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 --2- 172.21.15.59:0/1461481129 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fd4a0 0x7f10bc0f74c0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 -- 172.21.15.59:0/1461481129 >> 172.21.15.59:0/1461481129 conn(0x7f10bc06c370 msgr2=0x7f10bc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 -- 172.21.15.59:0/1461481129 shutdown_connections 2024-02-18T00:43:38.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.923+0000 7f10c3074700 1 -- 172.21.15.59:0/1461481129 wait complete. 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 librados: starting msgr at 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 librados: starting objecter 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 Processor -- start 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 -- start start 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 librados: setting wanted keys 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 librados: calling monclient init 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0f7ba0 0x7f10bc107060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fcb40 0x7f10bc107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc0fd4a0 0x7f10bc107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc1166e0 con 0x7f10bc0f7ba0 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc116820 con 0x7f10bc0fd4a0 2024-02-18T00:43:38.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c3074700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc116990 con 0x7f10bc0fcb40 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c0e10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0f7ba0 0x7f10bc107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10bbfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fcb40 0x7f10bc107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c0e10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0f7ba0 0x7f10bc107060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40312/0 (socket says 172.21.15.59:40312) 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c0e10700 1 -- 172.21.15.59:0/3825809574 learned_addr learned my addr 172.21.15.59:0/3825809574 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c1611700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc0fd4a0 0x7f10bc107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10bbfff700 1 -- 172.21.15.59:0/3825809574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc0fd4a0 msgr2=0x7f10bc107a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10bbfff700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc0fd4a0 0x7f10bc107a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10bbfff700 1 -- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0f7ba0 msgr2=0x7f10bc107060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10bbfff700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0f7ba0 0x7f10bc107060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10bbfff700 1 -- 172.21.15.59:0/3825809574 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac005040 con 0x7f10bc0fcb40 2024-02-18T00:43:38.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.924+0000 7f10c0e10700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0f7ba0 0x7f10bc107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10bbfff700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fcb40 0x7f10bc107550 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 rx=0x7f10a40047b0 tx=0x7f10a4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10b9ffb700 10 client.?.objecter ms_handle_connect 0x7f10bc0fcb40 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10b9ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10b9ffb700 1 -- 172.21.15.59:0/3825809574 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4018070 con 0x7f10bc0fcb40 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10b9ffb700 1 -- 172.21.15.59:0/3825809574 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10a4009710 con 0x7f10bc0fcb40 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10c3074700 1 -- 172.21.15.59:0/3825809574 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a4005040 con 0x7f10bc0fcb40 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10b9ffb700 1 -- 172.21.15.59:0/3825809574 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a401db30 con 0x7f10bc0fcb40 2024-02-18T00:43:38.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10c3074700 10 client.41642.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10c3074700 1 -- 172.21.15.59:0/3825809574 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc116e30 con 0x7f10bc0fcb40 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.925+0000 7f10c3074700 1 librados: init done 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.926+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0004f20 con 0x7f10bc0fcb40 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.926+0000 7f10b9ffb700 1 -- 172.21.15.59:0/3825809574 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f10a4009070 con 0x7f10bc0fcb40 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10b9ffb700 1 --2- 172.21.15.59:0/3825809574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f10a8062c90 0x7f10a8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10c0e10700 1 --2- 172.21.15.59:0/3825809574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f10a8062c90 0x7f10a8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10b9ffb700 1 -- 172.21.15.59:0/3825809574 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(673..673 src has 418..673) v4 ==== 8748+0+0 (secure 0 0 0) 0x7f10a407e770 con 0x7f10bc0fcb40 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10b9ffb700 10 client.41642.objecter ms_dispatch 0x7f10bc001550 osd_map(673..673 src has 418..673) v4 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10b9ffb700 3 client.41642.objecter handle_osd_map got epochs [673,673] > 0 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10b9ffb700 3 client.41642.objecter handle_osd_map decoding full epoch 673 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10b9ffb700 20 client.41642.objecter dump_active .. 0 homeless 2024-02-18T00:43:38.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10b9ffb700 10 client.41642.objecter ms_handle_connect 0x7f10a8062c90 2024-02-18T00:43:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.927+0000 7f10c0e10700 1 --2- 172.21.15.59:0/3825809574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f10a8062c90 0x7f10a8065150 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 rx=0x7f10ac004d40 tx=0x7f10ac00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:38.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:38.931+0000 7f10b9ffb700 1 -- 172.21.15.59:0/3825809574 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f10a4051b60 con 0x7f10bc0fcb40 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.544479+0000 osd.6 (osd.6) 303 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.545934+0000 osd.6 (osd.6) 304 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.720946+0000 osd.5 (osd.5) 287 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.722002+0000 osd.5 (osd.5) 288 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:37.838829+0000 mon.a (mon.0) 3314 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.838925+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:37.841111+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-116"}]: dispatch 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:37.989168+0000 mgr.y (mgr.24454) 594 : cluster [DBG] pgmap v1029: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:38.783409+0000 mon.a (mon.0) 3317 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-116"}]': finished 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:38.783513+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-18T00:43:39.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:38.788087+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-116", "mode": "writeback"}]: dispatch 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.544479+0000 osd.6 (osd.6) 303 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.545934+0000 osd.6 (osd.6) 304 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.720946+0000 osd.5 (osd.5) 287 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.722002+0000 osd.5 (osd.5) 288 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:37.838829+0000 mon.a (mon.0) 3314 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.838925+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:37.841111+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-116"}]: dispatch 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:37.989168+0000 mgr.y (mgr.24454) 594 : cluster [DBG] pgmap v1029: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:38.783409+0000 mon.a (mon.0) 3317 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-116"}]': finished 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:38.783513+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-18T00:43:39.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:38.788087+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-116", "mode": "writeback"}]: dispatch 2024-02-18T00:43:39.023 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:39.036 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:43:39.036 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.544479+0000 osd.6 (osd.6) 303 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.545934+0000 osd.6 (osd.6) 304 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.720946+0000 osd.5 (osd.5) 287 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.722002+0000 osd.5 (osd.5) 288 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:37.838829+0000 mon.a (mon.0) 3314 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.838925+0000 mon.a (mon.0) 3315 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:37.841111+0000 mon.a (mon.0) 3316 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-116"}]: dispatch 2024-02-18T00:43:39.049 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:37.989168+0000 mgr.y (mgr.24454) 594 : cluster [DBG] pgmap v1029: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:39.050 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:38.783409+0000 mon.a (mon.0) 3317 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-116"}]': finished 2024-02-18T00:43:39.050 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:38.783513+0000 mon.a (mon.0) 3318 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-18T00:43:39.050 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:38.788087+0000 mon.a (mon.0) 3319 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-116", "mode": "writeback"}]: dispatch 2024-02-18T00:43:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:39.073 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:39.075 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:39.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:39.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.113+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f10a00051d0 con 0x7f10bc0fcb40 2024-02-18T00:43:39.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:39.207 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:39.227 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:39.229 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:39.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:39.331 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:39.360 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:39.378 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:39.380 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:39.381 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:39.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:43:39.482 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:39.511 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:39.713 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:39.749 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:43:39.749 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 0 1 2024-02-18T00:43:39.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.789+0000 7f10b9ffb700 1 -- 172.21.15.59:0/3825809574 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v674) v1 ==== 101+0+0 (secure 0 0 0) 0x7f10a4051b60 con 0x7f10bc0fcb40 2024-02-18T00:43:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 10 librados: watch_flush enter 2024-02-18T00:43:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 10 librados: watch_flush exit 2024-02-18T00:43:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 20 client.41642.objecter shutdown clearing up homeless session... 2024-02-18T00:43:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 10 client.41642.objecter successfully canceled tick 2024-02-18T00:43:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f10a8062c90 msgr2=0x7f10a8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 --2- 172.21.15.59:0/3825809574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f10a8062c90 0x7f10a8065150 secure :-1 s=READY pgs=3194 cs=0 l=1 rev1=1 rx=0x7f10ac004d40 tx=0x7f10ac00b040).stop 2024-02-18T00:43:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fcb40 msgr2=0x7f10bc107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fcb40 0x7f10bc107550 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 rx=0x7f10a40047b0 tx=0x7f10a4009cb0).stop 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 shutdown_connections 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 --2- 172.21.15.59:0/3825809574 >> v2:172.21.15.59:6800/3721891797 conn(0x7f10a8062c90 0x7f10a8065150 unknown :-1 s=CLOSED pgs=3194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10bc0fd4a0 0x7f10bc107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10bc0f7ba0 0x7f10bc107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 --2- 172.21.15.59:0/3825809574 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10bc0fcb40 0x7f10bc107550 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.792+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 >> 172.21.15.59:0/3825809574 conn(0x7f10bc06c370 msgr2=0x7f10bc0f5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.793+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 shutdown_connections 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.793+0000 7f10b37fe700 1 -- 172.21.15.59:0/3825809574 wait complete. 2024-02-18T00:43:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:39.793+0000 7f10b37fe700 1 librados: shutdown 2024-02-18T00:43:39.796 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:43:39.859 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:39.862 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:39.863 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:39.877 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-18T00:43:39.964 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:39.993 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:40.011 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:40.014 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:40.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:40.115 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:40.141 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:38.754956+0000 osd.5 (osd.5) 289 : cluster [DBG] 6.d scrub starts 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:38.758064+0000 osd.5 (osd.5) 290 : cluster [DBG] 6.d scrub ok 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.061610+0000 osd.7 (osd.7) 199 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.065944+0000 osd.7 (osd.7) 200 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:39.115109+0000 mon.c (mon.2) 563 : audit [INF] from='client.? 172.21.15.59:0/3825809574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:39.115415+0000 mon.a (mon.0) 3320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.152842+0000 osd.2 (osd.2) 277 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.154557+0000 osd.2 (osd.2) 278 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.785000+0000 mon.a (mon.0) 3321 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.785036+0000 mon.a (mon.0) 3322 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:39.789465+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-116", "mode": "writeback"}]': finished 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:39.789543+0000 mon.a (mon.0) 3324 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.789596+0000 mon.a (mon.0) 3325 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-18T00:43:40.142 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:39.850104+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:40.144 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:40.163 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:40.166 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:40.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:40.195 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:43:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:38.754956+0000 osd.5 (osd.5) 289 : cluster [DBG] 6.d scrub starts 2024-02-18T00:43:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:38.758064+0000 osd.5 (osd.5) 290 : cluster [DBG] 6.d scrub ok 2024-02-18T00:43:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.061610+0000 osd.7 (osd.7) 199 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:43:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.065944+0000 osd.7 (osd.7) 200 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:43:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:39.115109+0000 mon.c (mon.2) 563 : audit [INF] from='client.? 172.21.15.59:0/3825809574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:39.115415+0000 mon.a (mon.0) 3320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.152842+0000 osd.2 (osd.2) 277 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.154557+0000 osd.2 (osd.2) 278 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.785000+0000 mon.a (mon.0) 3321 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.785036+0000 mon.a (mon.0) 3322 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:39.789465+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-116", "mode": "writeback"}]': finished 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:39.789543+0000 mon.a (mon.0) 3324 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.789596+0000 mon.a (mon.0) 3325 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:39.850104+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:38.754956+0000 osd.5 (osd.5) 289 : cluster [DBG] 6.d scrub starts 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:38.758064+0000 osd.5 (osd.5) 290 : cluster [DBG] 6.d scrub ok 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.061610+0000 osd.7 (osd.7) 199 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:43:40.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.065944+0000 osd.7 (osd.7) 200 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:39.115109+0000 mon.c (mon.2) 563 : audit [INF] from='client.? 172.21.15.59:0/3825809574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:39.115415+0000 mon.a (mon.0) 3320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.152842+0000 osd.2 (osd.2) 277 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.154557+0000 osd.2 (osd.2) 278 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.785000+0000 mon.a (mon.0) 3321 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.785036+0000 mon.a (mon.0) 3322 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:39.789465+0000 mon.a (mon.0) 3323 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-116", "mode": "writeback"}]': finished 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:39.789543+0000 mon.a (mon.0) 3324 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.789596+0000 mon.a (mon.0) 3325 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-18T00:43:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:39.850104+0000 mon.a (mon.0) 3326 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:40.268 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:40.296 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:40.745 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:40.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:40.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:40.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:40.878 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:40.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.790767+0000 osd.5 (osd.5) 291 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:43:40.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.793341+0000 osd.5 (osd.5) 292 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:43:40.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:39.910020+0000 mon.a (mon.0) 3327 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:40.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.955306+0000 osd.1 (osd.1) 325 : cluster [DBG] 5.12 deep-scrub starts 2024-02-18T00:43:40.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.958196+0000 osd.1 (osd.1) 326 : cluster [DBG] 5.12 deep-scrub ok 2024-02-18T00:43:40.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:39.989569+0000 mgr.y (mgr.24454) 595 : cluster [DBG] pgmap v1032: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:40.988 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:40.021625+0000 osd.7 (osd.7) 201 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:40.022779+0000 osd.7 (osd.7) 202 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:40.106800+0000 osd.3 (osd.3) 343 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:40.107753+0000 osd.3 (osd.3) 344 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:40.791281+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:40.791360+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:40.793604+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116"}]: dispatch 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.790767+0000 osd.5 (osd.5) 291 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.793341+0000 osd.5 (osd.5) 292 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:39.910020+0000 mon.a (mon.0) 3327 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.955306+0000 osd.1 (osd.1) 325 : cluster [DBG] 5.12 deep-scrub starts 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.958196+0000 osd.1 (osd.1) 326 : cluster [DBG] 5.12 deep-scrub ok 2024-02-18T00:43:40.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:39.989569+0000 mgr.y (mgr.24454) 595 : cluster [DBG] pgmap v1032: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:40.990 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:40.021625+0000 osd.7 (osd.7) 201 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:43:40.990 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:40.022779+0000 osd.7 (osd.7) 202 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:43:40.990 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:40.106800+0000 osd.3 (osd.3) 343 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:43:40.990 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:40.107753+0000 osd.3 (osd.3) 344 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:43:40.990 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:40.791281+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:40.990 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:40.791360+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-18T00:43:40.990 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:40.793604+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116"}]: dispatch 2024-02-18T00:43:41.001 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:41.004 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:41.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:41.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:41.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.790767+0000 osd.5 (osd.5) 291 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.793341+0000 osd.5 (osd.5) 292 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:39.910020+0000 mon.a (mon.0) 3327 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.955306+0000 osd.1 (osd.1) 325 : cluster [DBG] 5.12 deep-scrub starts 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.958196+0000 osd.1 (osd.1) 326 : cluster [DBG] 5.12 deep-scrub ok 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:39.989569+0000 mgr.y (mgr.24454) 595 : cluster [DBG] pgmap v1032: 214 pgs: 5 creating+peering, 27 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:40.021625+0000 osd.7 (osd.7) 201 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:40.022779+0000 osd.7 (osd.7) 202 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:40.106800+0000 osd.3 (osd.3) 343 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:40.107753+0000 osd.3 (osd.3) 344 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:40.791281+0000 mon.a (mon.0) 3328 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:40.791360+0000 mon.a (mon.0) 3329 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-18T00:43:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:40.793604+0000 mon.a (mon.0) 3330 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116"}]: dispatch 2024-02-18T00:43:41.551 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:41.713 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:43:41.947 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:43:41.948 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:43:41.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.137866+0000 osd.3 (osd.3) 345 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:43:41.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.139825+0000 osd.3 (osd.3) 346 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:43:41.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.160602+0000 osd.3 (osd.3) 347 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:41.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.160923+0000 osd.3 (osd.3) 348 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:41.972 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.441156+0000 osd.5 (osd.5) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:41.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.441294+0000 osd.5 (osd.5) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:41.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.788180+0000 mon.a (mon.0) 3331 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:41.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:41.791852+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116"}]': finished 2024-02-18T00:43:41.973 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.791923+0000 mon.a (mon.0) 3333 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-18T00:43:42.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.137866+0000 osd.3 (osd.3) 345 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.139825+0000 osd.3 (osd.3) 346 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.160602+0000 osd.3 (osd.3) 347 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.160923+0000 osd.3 (osd.3) 348 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.441156+0000 osd.5 (osd.5) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.441294+0000 osd.5 (osd.5) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.788180+0000 mon.a (mon.0) 3331 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:41.791852+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116"}]': finished 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.791923+0000 mon.a (mon.0) 3333 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.137866+0000 osd.3 (osd.3) 345 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:43:42.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.139825+0000 osd.3 (osd.3) 346 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:43:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.160602+0000 osd.3 (osd.3) 347 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.160923+0000 osd.3 (osd.3) 348 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.441156+0000 osd.5 (osd.5) 293 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.441294+0000 osd.5 (osd.5) 294 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.788180+0000 mon.a (mon.0) 3331 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:41.791852+0000 mon.a (mon.0) 3332 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-116"}]': finished 2024-02-18T00:43:42.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.791923+0000 mon.a (mon.0) 3333 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-18T00:43:42.464 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:43:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:42] "GET /metrics HTTP/1.1" 200 191489 "" "Prometheus/2.33.4" 2024-02-18T00:43:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:42.489 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:42.489 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:42.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:42.622 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:42.647 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:42.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:42.666 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_ops_in_flight 2024-02-18T00:43:42.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:42.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:43:42.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/3556879625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280fdbe0 msgr2=0x7fcc280fdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/3556879625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280fdbe0 0x7fcc280fdfc0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 rx=0x7fcc18004870 tx=0x7fcc18009cb0).stop 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/3556879625 shutdown_connections 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/3556879625 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280f67a0 0x7fcc28104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/3556879625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280fdbe0 0x7fcc280fdfc0 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/3556879625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280fe540 0x7fcc280f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/3556879625 >> 172.21.15.59:0/3556879625 conn(0x7fcc2806ffc0 msgr2=0x7fcc280f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/3556879625 shutdown_connections 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.761+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/3556879625 wait complete. 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.762+0000 7fcc2d9c3700 1 librados: starting msgr at 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.762+0000 7fcc2d9c3700 1 librados: starting objecter 2024-02-18T00:43:42.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.762+0000 7fcc2d9c3700 1 Processor -- start 2024-02-18T00:43:42.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.762+0000 7fcc2d9c3700 1 -- start start 2024-02-18T00:43:42.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.762+0000 7fcc2d9c3700 1 librados: setting wanted keys 2024-02-18T00:43:42.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.762+0000 7fcc2d9c3700 1 librados: calling monclient init 2024-02-18T00:43:42.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc2d9c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f67a0 0x7fcc2810e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc2d9c3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280fdbe0 0x7fcc2810e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc2d9c3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280fe540 0x7fcc281082f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc2d9c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc281087e0 con 0x7fcc280f67a0 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc2d9c3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28108950 con 0x7fcc280fe540 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc2d9c3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28108b10 con 0x7fcc280fdbe0 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f67a0 0x7fcc2810e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f67a0 0x7fcc2810e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60804/0 (socket says 172.21.15.59:60804) 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 -- 172.21.15.59:0/4224935306 learned_addr learned my addr 172.21.15.59:0/4224935306 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc277fe700 1 --2- 172.21.15.59:0/4224935306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280fe540 0x7fcc281082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 -- 172.21.15.59:0/4224935306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280fdbe0 msgr2=0x7fcc2810e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc280fdbe0 0x7fcc2810e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 -- 172.21.15.59:0/4224935306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280fe540 msgr2=0x7fcc281082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc280fe540 0x7fcc281082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 -- 172.21.15.59:0/4224935306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc200097c0 con 0x7fcc280f67a0 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.763+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f67a0 0x7fcc2810e150 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7fcc180043a0 tx=0x7fcc1800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc0ffff700 10 client.?.objecter ms_handle_connect 0x7fcc280f67a0 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc0ffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1800e070 con 0x7fcc280f67a0 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc18005040 con 0x7fcc280f67a0 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc2d9c3700 10 client.31839.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc28109030 con 0x7fcc280f67a0 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc2d9c3700 1 librados: init done 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1801e420 con 0x7fcc280f67a0 2024-02-18T00:43:42.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.764+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18019070 con 0x7fcc280f67a0 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.765+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fcc14004f30 con 0x7fcc280f67a0 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.765+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fcc180125d0 con 0x7fcc280f67a0 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc0ffff700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc100670a0 0x7fcc10069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(676..676 src has 418..676) v4 ==== 8712+0+0 (secure 0 0 0) 0x7fcc1807dcf0 con 0x7fcc280f67a0 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 osd_map(676..676 src has 418..676) v4 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc0ffff700 3 client.31839.objecter handle_osd_map got epochs [676,676] > 0 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc0ffff700 3 client.31839.objecter handle_osd_map decoding full epoch 676 2024-02-18T00:43:42.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc0ffff700 20 client.31839.objecter dump_active .. 0 homeless 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v676) v1 ==== 54+0+16 (secure 0 0 0) 0x7fcc180803c0 con 0x7fcc280f67a0 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.766+0000 7fcc267fc700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc100670a0 0x7fcc10069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 1 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc14004560 0x7fcc140049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 1 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 0 1 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140051d0 osd=0 4 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 1 2024-02-18T00:43:42.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc14008960 con 0x7fcc14004560 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc277fe700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc14004560 0x7fcc140049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc267fc700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc100670a0 0x7fcc10069560 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 rx=0x7fcc28109a90 tx=0x7fcc1c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc100670a0 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc277fe700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc14004560 0x7fcc140049a0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.767+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc14004560 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.768+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc14008960 con 0x7fcc14004560 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.768+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.768+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 1 = system:0 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.768+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc140051d0 osd=0 4 2024-02-18T00:43:42.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.768+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 0 1 2024-02-18T00:43:42.782 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:42.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 2 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 2 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc140051d0 osd=0 3 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 0 2 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140051d0 osd=0 4 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 2 2024-02-18T00:43:42.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.801+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc140081f0 con 0x7fcc14004560 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.803+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc140081f0 con 0x7fcc14004560 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.804+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.804+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.804+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc140051d0 osd=0 4 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.804+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 0 2 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 3 2024-02-18T00:43:42.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6805/26279110 conn(0x7fcc14002c90 0x7fcc14003120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 3 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 1 3 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140086c0 osd=1 4 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 3 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc14003700 con 0x7fcc14002c90 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6805/26279110 conn(0x7fcc14002c90 0x7fcc14003120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6805/26279110 conn(0x7fcc14002c90 0x7fcc14003120 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.805+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc14002c90 2024-02-18T00:43:42.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.806+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc14003700 con 0x7fcc14002c90 2024-02-18T00:43:42.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.806+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:43:42.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.806+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 3 = system:0 2024-02-18T00:43:42.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.806+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc140086c0 osd=1 4 2024-02-18T00:43:42.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.806+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 1 3 2024-02-18T00:43:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 4 2024-02-18T00:43:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 4 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc140086c0 osd=1 3 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 1 4 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140086c0 osd=1 4 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 4 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.829+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc140040f0 con 0x7fcc14002c90 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc140040f0 con 0x7fcc14002c90 2024-02-18T00:43:42.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc140086c0 osd=1 4 2024-02-18T00:43:42.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 1 4 2024-02-18T00:43:42.834 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 5 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcc1400aff0 0x7fcc1400d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 5 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 2 5 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14007d20 osd=2 4 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 5 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.831+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc14010030 con 0x7fcc1400aff0 2024-02-18T00:43:42.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.832+0000 7fcc267fc700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcc1400aff0 0x7fcc1400d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.832+0000 7fcc267fc700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcc1400aff0 0x7fcc1400d3e0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.832+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc1400aff0 2024-02-18T00:43:42.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.835+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc14010030 con 0x7fcc1400aff0 2024-02-18T00:43:42.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.835+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:43:42.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.835+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 5 = system:0 2024-02-18T00:43:42.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.835+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14007d20 osd=2 4 2024-02-18T00:43:42.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.835+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 2 5 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 6 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 6 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14007d20 osd=2 3 2024-02-18T00:43:42.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 2 6 2024-02-18T00:43:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14007d20 osd=2 4 2024-02-18T00:43:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 6 2024-02-18T00:43:42.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.864+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc14003b60 con 0x7fcc1400aff0 2024-02-18T00:43:42.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc14003b60 con 0x7fcc1400aff0 2024-02-18T00:43:42.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14007d20 osd=2 4 2024-02-18T00:43:42.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 2 6 2024-02-18T00:43:42.870 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 7 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcc1400ea50 0x7fcc1400ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 7 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 3 7 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14001bb0 osd=3 4 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 7 2024-02-18T00:43:42.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc140022f0 con 0x7fcc1400ea50 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.868+0000 7fcc277fe700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcc1400ea50 0x7fcc1400ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.869+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc1400ea50 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.869+0000 7fcc277fe700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcc1400ea50 0x7fcc1400ef00 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.869+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc140022f0 con 0x7fcc1400ea50 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.869+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.869+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 7 = system:0 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.869+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14001bb0 osd=3 4 2024-02-18T00:43:42.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.869+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 3 7 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 8 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 8 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14001bb0 osd=3 3 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 3 8 2024-02-18T00:43:42.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14001bb0 osd=3 4 2024-02-18T00:43:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 8 2024-02-18T00:43:42.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.909+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc1400fbc0 con 0x7fcc1400ea50 2024-02-18T00:43:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc1400fbc0 con 0x7fcc1400ea50 2024-02-18T00:43:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14001bb0 osd=3 4 2024-02-18T00:43:42.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 3 8 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 9 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcc140121b0 0x7fcc140145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 9 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 4 9 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc1400fbc0 osd=4 4 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 9 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc140172d0 con 0x7fcc140121b0 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.912+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcc140121b0 0x7fcc140145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.913+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc140121b0 2024-02-18T00:43:42.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.913+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcc140121b0 0x7fcc140145a0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.914+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc140172d0 con 0x7fcc140121b0 2024-02-18T00:43:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.914+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:43:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.914+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 9 = system:0 2024-02-18T00:43:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.914+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc1400fbc0 osd=4 4 2024-02-18T00:43:42.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.914+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 4 9 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.929+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 10 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 10 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc1400fbc0 osd=4 3 2024-02-18T00:43:42.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 4 10 2024-02-18T00:43:42.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc1400fbc0 osd=4 4 2024-02-18T00:43:42.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 10 2024-02-18T00:43:42.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.930+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc140172d0 con 0x7fcc140121b0 2024-02-18T00:43:42.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.931+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc140172d0 con 0x7fcc140121b0 2024-02-18T00:43:42.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.931+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.931+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.931+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc1400fbc0 osd=4 4 2024-02-18T00:43:42.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.931+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 4 10 2024-02-18T00:43:42.934 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:43:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 11 2024-02-18T00:43:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcc14015210 0x7fcc14015670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 11 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 5 11 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14014b90 osd=5 4 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 11 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc1401a680 con 0x7fcc14015210 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc267fc700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcc14015210 0x7fcc14015670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc14015210 2024-02-18T00:43:42.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.932+0000 7fcc267fc700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcc14015210 0x7fcc14015670 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.939+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc1401a680 con 0x7fcc14015210 2024-02-18T00:43:42.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.939+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:43:42.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.939+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 11 = system:0 2024-02-18T00:43:42.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.939+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14014b90 osd=5 4 2024-02-18T00:43:42.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.939+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 5 11 2024-02-18T00:43:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 12 2024-02-18T00:43:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 12 2024-02-18T00:43:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14014b90 osd=5 3 2024-02-18T00:43:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 5 12 2024-02-18T00:43:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14014b90 osd=5 4 2024-02-18T00:43:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 12 2024-02-18T00:43:42.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.955+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc14015b60 con 0x7fcc14015210 2024-02-18T00:43:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc14015b60 con 0x7fcc14015210 2024-02-18T00:43:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14014b90 osd=5 4 2024-02-18T00:43:42.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 5 12 2024-02-18T00:43:42.959 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 13 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.957+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcc14016070 0x7fcc140164d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 13 2024-02-18T00:43:42.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 6 13 2024-02-18T00:43:42.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14016e80 osd=6 4 2024-02-18T00:43:42.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 13 2024-02-18T00:43:42.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc140169f0 con 0x7fcc14016070 2024-02-18T00:43:42.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc277fe700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcc14016070 0x7fcc140164d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc14016070 2024-02-18T00:43:42.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.958+0000 7fcc277fe700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcc14016070 0x7fcc140164d0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.959+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc140169f0 con 0x7fcc14016070 2024-02-18T00:43:42.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.959+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:43:42.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.959+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 13 = system:0 2024-02-18T00:43:42.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.959+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14016e80 osd=6 4 2024-02-18T00:43:42.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.959+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 6 13 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 14 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 14 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc14016e80 osd=6 3 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 6 14 2024-02-18T00:43:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14016e80 osd=6 4 2024-02-18T00:43:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 14 2024-02-18T00:43:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.974+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc140169f0 con 0x7fcc14016070 2024-02-18T00:43:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc140169f0 con 0x7fcc14016070 2024-02-18T00:43:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc14016e80 osd=6 4 2024-02-18T00:43:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 6 14 2024-02-18T00:43:42.977 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:43:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 15 2024-02-18T00:43:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6812/53025171 conn(0x7fcc1401ba60 0x7fcc1401bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 15 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 7 15 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140169f0 osd=7 4 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 15 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.975+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc14020e30 con 0x7fcc1401ba60 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.976+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6812/53025171 conn(0x7fcc1401ba60 0x7fcc1401bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.976+0000 7fcc0ffff700 10 client.31839.objecter ms_handle_connect 0x7fcc1401ba60 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.976+0000 7fcc26ffd700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6812/53025171 conn(0x7fcc1401ba60 0x7fcc1401bec0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.977+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc14020e30 con 0x7fcc1401ba60 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.977+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.977+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 15 = system:0 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.977+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc140169f0 osd=7 4 2024-02-18T00:43:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.977+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 7 15 2024-02-18T00:43:42.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 10 client.31839.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:43:42.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign -1 16 2024-02-18T00:43:42.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 20 client.31839.objecter _get_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_remove -1 16 2024-02-18T00:43:42.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 20 client.31839.objecter get_session s=0x7fcc140169f0 osd=7 3 2024-02-18T00:43:42.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 15 client.31839.objecter _session_command_op_assign 7 16 2024-02-18T00:43:42.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140169f0 osd=7 4 2024-02-18T00:43:42.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 10 client.31839.objecter _send_command 16 2024-02-18T00:43:42.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.995+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fcc1401c4d0 con 0x7fcc1401ba60 2024-02-18T00:43:42.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc0ffff700 1 -- 172.21.15.59:0/4224935306 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fcc1401c4d0 con 0x7fcc1401ba60 2024-02-18T00:43:42.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc0ffff700 10 client.31839.objecter ms_dispatch 0x7fcc28001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:43:42.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc0ffff700 10 client.31839.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:43:42.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc0ffff700 20 client.31839.objecter put_session s=0x7fcc140169f0 osd=7 4 2024-02-18T00:43:42.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc0ffff700 15 client.31839.objecter _session_command_op_remove 7 16 2024-02-18T00:43:42.999 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:43:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 librados: watch_flush enter 2024-02-18T00:43:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 librados: watch_flush exit 2024-02-18T00:43:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.0 2024-02-18T00:43:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc14004560 msgr2=0x7fcc140049a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc14004560 0x7fcc140049a0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140051d0 osd=0 1 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.1 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6805/26279110 conn(0x7fcc14002c90 msgr2=0x7fcc14003120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6805/26279110 conn(0x7fcc14002c90 0x7fcc14003120 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140086c0 osd=1 1 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.2 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcc1400aff0 msgr2=0x7fcc1400d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcc1400aff0 0x7fcc1400d3e0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14007d20 osd=2 1 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.3 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcc1400ea50 msgr2=0x7fcc1400ef00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcc1400ea50 0x7fcc1400ef00 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14001bb0 osd=3 1 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.4 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcc140121b0 msgr2=0x7fcc140145a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcc140121b0 0x7fcc140145a0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc277fe700 1 -- 172.21.15.59:0/4224935306 reap_dead start 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc1400fbc0 osd=4 1 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.5 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcc14015210 msgr2=0x7fcc14015670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcc14015210 0x7fcc14015670 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14014b90 osd=5 1 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.6 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcc14016070 msgr2=0x7fcc140164d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcc14016070 0x7fcc140164d0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc14016e80 osd=6 1 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter close_session for osd.7 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6812/53025171 conn(0x7fcc1401ba60 msgr2=0x7fcc1401bec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.137:6812/53025171 conn(0x7fcc1401ba60 0x7fcc1401bec0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter put_session s=0x7fcc140169f0 osd=7 1 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 20 client.31839.objecter shutdown clearing up homeless session... 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.997+0000 7fcc2d9c3700 10 client.31839.objecter successfully canceled tick 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc277fe700 1 -- 172.21.15.59:0/4224935306 reap_dead start 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc277fe700 1 -- 172.21.15.59:0/4224935306 reap_dead start 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc277fe700 1 -- 172.21.15.59:0/4224935306 reap_dead start 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc277fe700 1 -- 172.21.15.59:0/4224935306 reap_dead start 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc100670a0 msgr2=0x7fcc10069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc100670a0 0x7fcc10069560 secure :-1 s=READY pgs=3195 cs=0 l=1 rev1=1 rx=0x7fcc28109a90 tx=0x7fcc1c00b040).stop 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f67a0 msgr2=0x7fcc2810e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f67a0 0x7fcc2810e150 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7fcc180043a0 tx=0x7fcc1800b040).stop 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 shutdown_connections 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc280f67a0 0x7fcc2810e150 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 --2- 172.21.15.59:0/4224935306 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc100670a0 0x7fcc10069560 unknown :-1 s=CLOSED pgs=3195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 >> 172.21.15.59:0/4224935306 conn(0x7fcc2806ffc0 msgr2=0x7fcc280fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 shutdown_connections 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 -- 172.21.15.59:0/4224935306 wait complete. 2024-02-18T00:43:43.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:42.998+0000 7fcc2d9c3700 1 librados: shutdown 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.005 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:43:43.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:41.512160+0000 mgr.y (mgr.24454) 596 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.931537+0000 osd.1 (osd.1) 327 : cluster [DBG] 6.4 deep-scrub starts 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.934478+0000 osd.1 (osd.1) 328 : cluster [DBG] 6.4 deep-scrub ok 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:41.989931+0000 mgr.y (mgr.24454) 597 : cluster [DBG] pgmap v1035: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:42.066986+0000 osd.7 (osd.7) 203 : cluster [DBG] 1.0 deep-scrub starts 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:42.067806+0000 osd.7 (osd.7) 204 : cluster [DBG] 1.0 deep-scrub ok 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:42.102955+0000 osd.3 (osd.3) 349 : cluster [DBG] 6.c scrub starts 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:42.104359+0000 osd.3 (osd.3) 350 : cluster [DBG] 6.c scrub ok 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:42.767392+0000 mon.a (mon.0) 3334 : audit [DBG] from='client.? 172.21.15.59:0/4224935306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:43.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:42.793656+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:41.512160+0000 mgr.y (mgr.24454) 596 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.931537+0000 osd.1 (osd.1) 327 : cluster [DBG] 6.4 deep-scrub starts 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.934478+0000 osd.1 (osd.1) 328 : cluster [DBG] 6.4 deep-scrub ok 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:41.989931+0000 mgr.y (mgr.24454) 597 : cluster [DBG] pgmap v1035: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:42.066986+0000 osd.7 (osd.7) 203 : cluster [DBG] 1.0 deep-scrub starts 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:42.067806+0000 osd.7 (osd.7) 204 : cluster [DBG] 1.0 deep-scrub ok 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:42.102955+0000 osd.3 (osd.3) 349 : cluster [DBG] 6.c scrub starts 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:42.104359+0000 osd.3 (osd.3) 350 : cluster [DBG] 6.c scrub ok 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:42.767392+0000 mon.a (mon.0) 3334 : audit [DBG] from='client.? 172.21.15.59:0/4224935306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:43.479 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:42.793656+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-18T00:43:43.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:41.512160+0000 mgr.y (mgr.24454) 596 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:43.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.931537+0000 osd.1 (osd.1) 327 : cluster [DBG] 6.4 deep-scrub starts 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.934478+0000 osd.1 (osd.1) 328 : cluster [DBG] 6.4 deep-scrub ok 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:41.989931+0000 mgr.y (mgr.24454) 597 : cluster [DBG] pgmap v1035: 214 pgs: 214 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:42.066986+0000 osd.7 (osd.7) 203 : cluster [DBG] 1.0 deep-scrub starts 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:42.067806+0000 osd.7 (osd.7) 204 : cluster [DBG] 1.0 deep-scrub ok 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:42.102955+0000 osd.3 (osd.3) 349 : cluster [DBG] 6.c scrub starts 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:42.104359+0000 osd.3 (osd.3) 350 : cluster [DBG] 6.c scrub ok 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:42.767392+0000 mon.a (mon.0) 3334 : audit [DBG] from='client.? 172.21.15.59:0/4224935306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:43.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:42.793656+0000 mon.a (mon.0) 3335 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-18T00:43:43.853 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:43.856 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:43.856 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:43.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:43.989 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:44.339 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:44.341 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:44.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:44.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:44.475 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:44.497 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:44.500 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:44.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:44.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:44.630 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.137630+0000 osd.3 (osd.3) 351 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:43:44.630 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.140126+0000 osd.3 (osd.3) 352 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:43:44.630 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:43.260798+0000 mon.a (mon.0) 3336 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5, 3, 0]}]: dispatch 2024-02-18T00:43:44.630 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:43.261010+0000 mon.a (mon.0) 3337 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [3, 5]}]: dispatch 2024-02-18T00:43:44.630 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:43.265172+0000 mon.a (mon.0) 3338 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.21", "id": [3, 5]}]: dispatch 2024-02-18T00:43:44.630 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.779787+0000 mon.a (mon.0) 3339 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:43.798567+0000 mon.a (mon.0) 3340 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5, 3, 0]}]': finished 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:43.798658+0000 mon.a (mon.0) 3341 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [3, 5]}]': finished 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:43.798721+0000 mon.a (mon.0) 3342 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.21", "id": [3, 5]}]': finished 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.798778+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:43.808975+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.137630+0000 osd.3 (osd.3) 351 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.140126+0000 osd.3 (osd.3) 352 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:43.260798+0000 mon.a (mon.0) 3336 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5, 3, 0]}]: dispatch 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:43.261010+0000 mon.a (mon.0) 3337 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [3, 5]}]: dispatch 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:43.265172+0000 mon.a (mon.0) 3338 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.21", "id": [3, 5]}]: dispatch 2024-02-18T00:43:44.631 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.779787+0000 mon.a (mon.0) 3339 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:44.632 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:43.798567+0000 mon.a (mon.0) 3340 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5, 3, 0]}]': finished 2024-02-18T00:43:44.632 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:43.798658+0000 mon.a (mon.0) 3341 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [3, 5]}]': finished 2024-02-18T00:43:44.632 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:43.798721+0000 mon.a (mon.0) 3342 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.21", "id": [3, 5]}]': finished 2024-02-18T00:43:44.632 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.798778+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-18T00:43:44.632 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:43.808975+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:44.632 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:44.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.137630+0000 osd.3 (osd.3) 351 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:43:44.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.140126+0000 osd.3 (osd.3) 352 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:43.260798+0000 mon.a (mon.0) 3336 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5, 3, 0]}]: dispatch 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:43.261010+0000 mon.a (mon.0) 3337 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [3, 5]}]: dispatch 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:43.265172+0000 mon.a (mon.0) 3338 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.21", "id": [3, 5]}]: dispatch 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.779787+0000 mon.a (mon.0) 3339 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:43.798567+0000 mon.a (mon.0) 3340 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5, 3, 0]}]': finished 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:43.798658+0000 mon.a (mon.0) 3341 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [3, 5]}]': finished 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:43.798721+0000 mon.a (mon.0) 3342 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.21", "id": [3, 5]}]': finished 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.798778+0000 mon.a (mon.0) 3343 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-18T00:43:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:43.808975+0000 mon.a (mon.0) 3344 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:45.483 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:45.487 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:45.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:45.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 -- 172.21.15.59:0/1144031245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c98102290 msgr2=0x7f7c98104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/1144031245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c98102290 0x7f7c98104680 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 rx=0x7f7c90009c30 tx=0x7f7c90004c50).stop 2024-02-18T00:43:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 -- 172.21.15.59:0/1144031245 shutdown_connections 2024-02-18T00:43:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/1144031245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c980f8d30 0x7f7c980f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/1144031245 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c98102290 0x7f7c98104680 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/1144031245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c980f9600 0x7f7c98101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:45.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 -- 172.21.15.59:0/1144031245 >> 172.21.15.59:0/1144031245 conn(0x7f7c980f2820 msgr2=0x7f7c980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 -- 172.21.15.59:0/1144031245 shutdown_connections 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.594+0000 7f7c9d46d700 1 -- 172.21.15.59:0/1144031245 wait complete. 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 librados: starting msgr at 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 librados: starting objecter 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 Processor -- start 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 -- start start 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 librados: setting wanted keys 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 librados: calling monclient init 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c980f8d30 0x7f7c980f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c980f9600 0x7f7c980f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.595+0000 7f7c9d46d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c98102290 0x7f7c980f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c9d46d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98116480 con 0x7f7c98102290 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c9d46d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c981165f0 con 0x7f7c980f9600 2024-02-18T00:43:45.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c9d46d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c981167b0 con 0x7f7c980f8d30 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c980f8d30 0x7f7c980f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c980f8d30 0x7f7c980f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:47632/0 (socket says 172.21.15.59:47632) 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 -- 172.21.15.59:0/2031042926 learned_addr learned my addr 172.21.15.59:0/2031042926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c977fe700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c98102290 0x7f7c980f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c967fc700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c980f9600 0x7f7c980f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 -- 172.21.15.59:0/2031042926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c980f9600 msgr2=0x7f7c980f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c980f9600 0x7f7c980f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 -- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c98102290 msgr2=0x7f7c980f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c98102290 0x7f7c980f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 -- 172.21.15.59:0/2031042926 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c005040 con 0x7f7c980f8d30 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c977fe700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c98102290 0x7f7c980f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:45.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c96ffd700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c980f8d30 0x7f7c980f6ce0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 rx=0x7f7c880041d0 tx=0x7f7c88009f60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c7ffff700 10 client.?.objecter ms_handle_connect 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c7ffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c7ffff700 1 -- 172.21.15.59:0/2031042926 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88007b40 con 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c900097c0 con 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c7ffff700 1 -- 172.21.15.59:0/2031042926 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c8800f040 con 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c7ffff700 1 -- 172.21.15.59:0/2031042926 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88014770 con 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c9d46d700 10 client.41651.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c98108000 con 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.596+0000 7f7c9d46d700 1 librados: init done 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9805fa60 con 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c7ffff700 1 -- 172.21.15.59:0/2031042926 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f7c88009110 con 0x7f7c980f8d30 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c7ffff700 1 --2- 172.21.15.59:0/2031042926 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7c80062c90 0x7f7c80065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:45.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c7ffff700 1 -- 172.21.15.59:0/2031042926 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(679..679 src has 418..679) v4 ==== 8791+0+0 (secure 0 0 0) 0x7f7c8807d5b0 con 0x7f7c980f8d30 2024-02-18T00:43:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c7ffff700 10 client.41651.objecter ms_dispatch 0x7f7c98001550 osd_map(679..679 src has 418..679) v4 2024-02-18T00:43:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c7ffff700 3 client.41651.objecter handle_osd_map got epochs [679,679] > 0 2024-02-18T00:43:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c7ffff700 3 client.41651.objecter handle_osd_map decoding full epoch 679 2024-02-18T00:43:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.598+0000 7f7c967fc700 1 --2- 172.21.15.59:0/2031042926 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7c80062c90 0x7f7c80065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.599+0000 7f7c7ffff700 20 client.41651.objecter dump_active .. 0 homeless 2024-02-18T00:43:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.599+0000 7f7c7ffff700 10 client.41651.objecter ms_handle_connect 0x7f7c80062c90 2024-02-18T00:43:45.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.599+0000 7f7c967fc700 1 --2- 172.21.15.59:0/2031042926 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7c80062c90 0x7f7c80065150 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 rx=0x7f7c8c009f20 tx=0x7f7c8c009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:45.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.606+0000 7f7c7ffff700 1 -- 172.21.15.59:0/2031042926 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7c880507a0 con 0x7f7c980f8d30 2024-02-18T00:43:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:45.611 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:45.623 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.644264+0000 osd.5 (osd.5) 295 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:43:45.623 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.645794+0000 osd.5 (osd.5) 296 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:43:45.624 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.926053+0000 osd.1 (osd.1) 329 : cluster [DBG] 2.d scrub starts 2024-02-18T00:43:45.624 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.927166+0000 osd.1 (osd.1) 330 : cluster [DBG] 2.d scrub ok 2024-02-18T00:43:45.624 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:43.990331+0000 mgr.y (mgr.24454) 598 : cluster [DBG] pgmap v1038: 214 pgs: 32 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:45.624 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.120351+0000 osd.7 (osd.7) 205 : cluster [DBG] 25.3 deep-scrub starts 2024-02-18T00:43:45.624 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.122050+0000 osd.7 (osd.7) 206 : cluster [DBG] 25.3 deep-scrub ok 2024-02-18T00:43:45.624 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.141645+0000 osd.3 (osd.3) 353 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:43:45.625 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.142676+0000 osd.3 (osd.3) 354 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:43:45.625 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.167312+0000 osd.2 (osd.2) 279 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.168089+0000 osd.2 (osd.2) 280 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.562778+0000 osd.4 (osd.4) 373 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.566257+0000 osd.4 (osd.4) 374 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.574564+0000 osd.6 (osd.6) 305 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:43:45.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.575414+0000 osd.6 (osd.6) 306 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:43:45.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:44.799922+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:45.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:44.800002+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-18T00:43:45.627 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:44.803577+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:45.628 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.644264+0000 osd.5 (osd.5) 295 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:43:45.628 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.645794+0000 osd.5 (osd.5) 296 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:43:45.628 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.926053+0000 osd.1 (osd.1) 329 : cluster [DBG] 2.d scrub starts 2024-02-18T00:43:45.628 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.927166+0000 osd.1 (osd.1) 330 : cluster [DBG] 2.d scrub ok 2024-02-18T00:43:45.628 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:43.990331+0000 mgr.y (mgr.24454) 598 : cluster [DBG] pgmap v1038: 214 pgs: 32 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:45.628 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.120351+0000 osd.7 (osd.7) 205 : cluster [DBG] 25.3 deep-scrub starts 2024-02-18T00:43:45.628 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.122050+0000 osd.7 (osd.7) 206 : cluster [DBG] 25.3 deep-scrub ok 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.141645+0000 osd.3 (osd.3) 353 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.142676+0000 osd.3 (osd.3) 354 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.167312+0000 osd.2 (osd.2) 279 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.168089+0000 osd.2 (osd.2) 280 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.562778+0000 osd.4 (osd.4) 373 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.566257+0000 osd.4 (osd.4) 374 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.574564+0000 osd.6 (osd.6) 305 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.575414+0000 osd.6 (osd.6) 306 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:44.799922+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:44.800002+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-18T00:43:45.629 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:44.803577+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:45.629 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:45.650 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:45.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:45.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:45.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.644264+0000 osd.5 (osd.5) 295 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.645794+0000 osd.5 (osd.5) 296 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.926053+0000 osd.1 (osd.1) 329 : cluster [DBG] 2.d scrub starts 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.927166+0000 osd.1 (osd.1) 330 : cluster [DBG] 2.d scrub ok 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:43.990331+0000 mgr.y (mgr.24454) 598 : cluster [DBG] pgmap v1038: 214 pgs: 32 unknown, 182 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.120351+0000 osd.7 (osd.7) 205 : cluster [DBG] 25.3 deep-scrub starts 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.122050+0000 osd.7 (osd.7) 206 : cluster [DBG] 25.3 deep-scrub ok 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.141645+0000 osd.3 (osd.3) 353 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:43:45.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.142676+0000 osd.3 (osd.3) 354 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.167312+0000 osd.2 (osd.2) 279 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.168089+0000 osd.2 (osd.2) 280 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.562778+0000 osd.4 (osd.4) 373 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.566257+0000 osd.4 (osd.4) 374 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.574564+0000 osd.6 (osd.6) 305 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.575414+0000 osd.6 (osd.6) 306 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:44.799922+0000 mon.a (mon.0) 3345 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:44.800002+0000 mon.a (mon.0) 3346 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-18T00:43:45.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:44.803577+0000 mon.a (mon.0) 3347 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:45.784 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:45.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:45.815+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) v1 -- 0x7f7c980f8170 con 0x7f7c980f8d30 2024-02-18T00:43:45.864 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:43:46.065 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:46.070 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:46.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:46.171 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:46.201 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:46.221 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:46.224 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:46.224 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:46.325 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:46.353 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:46.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.806+0000 7f7c7ffff700 1 -- 172.21.15.59:0/2031042926 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v681) v1 ==== 113+0+0 (secure 0 0 0) 0x7f7c88050390 con 0x7f7c980f8d30 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 10 librados: watch_flush enter 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 10 librados: watch_flush exit 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 20 client.41651.objecter shutdown clearing up homeless session... 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 10 client.41651.objecter successfully canceled tick 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7c80062c90 msgr2=0x7f7c80065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/2031042926 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7c80062c90 0x7f7c80065150 secure :-1 s=READY pgs=3196 cs=0 l=1 rev1=1 rx=0x7f7c8c009f20 tx=0x7f7c8c009cb0).stop 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c980f8d30 msgr2=0x7f7c980f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c980f8d30 0x7f7c980f6ce0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 rx=0x7f7c880041d0 tx=0x7f7c88009f60).stop 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 shutdown_connections 2024-02-18T00:43:46.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c98102290 0x7f7c980f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7c980f8d30 0x7f7c980f6ce0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/2031042926 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7c80062c90 0x7f7c80065150 unknown :-1 s=CLOSED pgs=3196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 --2- 172.21.15.59:0/2031042926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c980f9600 0x7f7c980f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.808+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 >> 172.21.15.59:0/2031042926 conn(0x7f7c980f2820 msgr2=0x7f7c980f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.809+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 shutdown_connections 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.809+0000 7f7c9d46d700 1 -- 172.21.15.59:0/2031042926 wait complete. 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:46.809+0000 7f7c9d46d700 1 librados: shutdown 2024-02-18T00:43:46.811 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.0 to 1 (10000) 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:45.798576+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:45.798646+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:45.800603+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-118"}]: dispatch 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:45.816622+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.59:0/2031042926' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:45.816921+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:45.985855+0000 osd.1 (osd.1) 331 : cluster [DBG] 4.7 deep-scrub starts 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:45.989421+0000 osd.1 (osd.1) 332 : cluster [DBG] 4.7 deep-scrub ok 2024-02-18T00:43:47.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:45.991075+0000 mgr.y (mgr.24454) 599 : cluster [DBG] pgmap v1041: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 34 B/s, 0 objects/s recovering 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:45.798576+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:45.798646+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:45.800603+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-118"}]: dispatch 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:45.816622+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.59:0/2031042926' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:45.816921+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:45.985855+0000 osd.1 (osd.1) 331 : cluster [DBG] 4.7 deep-scrub starts 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:45.989421+0000 osd.1 (osd.1) 332 : cluster [DBG] 4.7 deep-scrub ok 2024-02-18T00:43:47.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:45.991075+0000 mgr.y (mgr.24454) 599 : cluster [DBG] pgmap v1041: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 34 B/s, 0 objects/s recovering 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:45.798576+0000 mon.a (mon.0) 3348 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:45.798646+0000 mon.a (mon.0) 3349 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:45.800603+0000 mon.a (mon.0) 3350 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-118"}]: dispatch 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:45.816622+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.59:0/2031042926' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:45.816921+0000 mon.a (mon.0) 3351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:45.985855+0000 osd.1 (osd.1) 331 : cluster [DBG] 4.7 deep-scrub starts 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:45.989421+0000 osd.1 (osd.1) 332 : cluster [DBG] 4.7 deep-scrub ok 2024-02-18T00:43:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:45.991075+0000 mgr.y (mgr.24454) 599 : cluster [DBG] pgmap v1041: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 34 B/s, 0 objects/s recovering 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:46.805734+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-118"}]': finished 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:46.805814+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:46.805879+0000 mon.a (mon.0) 3354 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:46.807162+0000 mon.a (mon.0) 3355 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-118", "mode": "writeback"}]: dispatch 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:46.805734+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-118"}]': finished 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:46.805814+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:46.805879+0000 mon.a (mon.0) 3354 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-18T00:43:48.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:46.807162+0000 mon.a (mon.0) 3355 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-118", "mode": "writeback"}]: dispatch 2024-02-18T00:43:48.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:46.805734+0000 mon.a (mon.0) 3352 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-118"}]': finished 2024-02-18T00:43:48.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:46.805814+0000 mon.a (mon.0) 3353 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:43:48.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:46.805879+0000 mon.a (mon.0) 3354 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-18T00:43:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:46.807162+0000 mon.a (mon.0) 3355 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-118", "mode": "writeback"}]: dispatch 2024-02-18T00:43:48.304 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:48.307 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:48.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:48.409 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:48.441 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:48.463 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:48.466 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:48.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:48.567 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:48.599 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:48.620 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:48.623 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:48.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:48.724 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:48.753 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:48.773 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:48.776 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:48.776 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:48.877 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:48.906 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:48.925 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:48.927 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:48.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:49.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:47.094772+0000 osd.0 (osd.0) 302 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:47.094966+0000 osd.0 (osd.0) 303 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:47.801413+0000 mon.a (mon.0) 3356 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:47.806904+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-118", "mode": "writeback"}]': finished 2024-02-18T00:43:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:47.806990+0000 mon.a (mon.0) 3358 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-18T00:43:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:47.991580+0000 mgr.y (mgr.24454) 600 : cluster [DBG] pgmap v1044: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 34 B/s, 0 objects/s recovering 2024-02-18T00:43:49.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:48.789606+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-18T00:43:49.029 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:49.059 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:49.077 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:49.081 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:49.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:49.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:49.217 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:47.094772+0000 osd.0 (osd.0) 302 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:47.094966+0000 osd.0 (osd.0) 303 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:47.801413+0000 mon.a (mon.0) 3356 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:47.806904+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-118", "mode": "writeback"}]': finished 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:47.806990+0000 mon.a (mon.0) 3358 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:47.991580+0000 mgr.y (mgr.24454) 600 : cluster [DBG] pgmap v1044: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 34 B/s, 0 objects/s recovering 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:48.789606+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:47.094772+0000 osd.0 (osd.0) 302 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:43:49.218 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:47.094966+0000 osd.0 (osd.0) 303 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:43:49.219 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:47.801413+0000 mon.a (mon.0) 3356 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:43:49.219 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:47.806904+0000 mon.a (mon.0) 3357 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-118", "mode": "writeback"}]': finished 2024-02-18T00:43:49.219 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:47.806990+0000 mon.a (mon.0) 3358 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-18T00:43:49.219 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:47.991580+0000 mgr.y (mgr.24454) 600 : cluster [DBG] pgmap v1044: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 34 B/s, 0 objects/s recovering 2024-02-18T00:43:49.219 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:48.789606+0000 mon.a (mon.0) 3359 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-18T00:43:49.221 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:49.373 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:49.378 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:49.378 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:49.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 1 1 2024-02-18T00:43:49.480 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:49.507 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:49.526 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:49.529 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:49.529 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:49.630 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:49.676 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:49.678 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:49.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:49.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:49.810 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:49.831 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:49.843 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:49.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:49.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:49.973 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:48.484185+0000 osd.0 (osd.0) 304 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:43:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:48.486031+0000 osd.0 (osd.0) 305 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:43:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:48.724127+0000 osd.5 (osd.5) 297 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:43:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:48.727631+0000 osd.5 (osd.5) 298 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:43:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:48.484185+0000 osd.0 (osd.0) 304 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:43:50.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:48.486031+0000 osd.0 (osd.0) 305 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:43:50.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:48.724127+0000 osd.5 (osd.5) 297 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:43:50.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:48.727631+0000 osd.5 (osd.5) 298 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:43:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:48.484185+0000 osd.0 (osd.0) 304 : cluster [DBG] 25.7 scrub starts 2024-02-18T00:43:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:48.486031+0000 osd.0 (osd.0) 305 : cluster [DBG] 25.7 scrub ok 2024-02-18T00:43:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:48.724127+0000 osd.5 (osd.5) 297 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:43:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:48.727631+0000 osd.5 (osd.5) 298 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:43:50.321 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:43:50.321 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:43:50.321 INFO:teuthology.orchestra.run.smithi059.stdout: "num_ops": 0 2024-02-18T00:43:50.321 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:43:50.621 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:50.624 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:50.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:50.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:50.756 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:50.776 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:50.779 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:50.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:50.880 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:50.908 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:49.992161+0000 mgr.y (mgr.24454) 601 : cluster [DBG] pgmap v1046: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 32 B/s, 0 objects/s recovering 2024-02-18T00:43:50.908 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:50.154086+0000 osd.3 (osd.3) 355 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:50.155201+0000 osd.3 (osd.3) 356 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:50.441005+0000 osd.4 (osd.4) 375 : cluster [DBG] 25.b scrub starts 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:50.444276+0000 osd.4 (osd.4) 376 : cluster [DBG] 25.b scrub ok 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:49.992161+0000 mgr.y (mgr.24454) 601 : cluster [DBG] pgmap v1046: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 32 B/s, 0 objects/s recovering 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:50.154086+0000 osd.3 (osd.3) 355 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:50.155201+0000 osd.3 (osd.3) 356 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:50.441005+0000 osd.4 (osd.4) 375 : cluster [DBG] 25.b scrub starts 2024-02-18T00:43:50.909 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:50.444276+0000 osd.4 (osd.4) 376 : cluster [DBG] 25.b scrub ok 2024-02-18T00:43:50.911 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:51.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:49.992161+0000 mgr.y (mgr.24454) 601 : cluster [DBG] pgmap v1046: 214 pgs: 214 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 32 B/s, 0 objects/s recovering 2024-02-18T00:43:51.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:50.154086+0000 osd.3 (osd.3) 355 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:43:51.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:50.155201+0000 osd.3 (osd.3) 356 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:43:51.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:50.441005+0000 osd.4 (osd.4) 375 : cluster [DBG] 25.b scrub starts 2024-02-18T00:43:51.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:50.444276+0000 osd.4 (osd.4) 376 : cluster [DBG] 25.b scrub ok 2024-02-18T00:43:51.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_blocked_ops 2024-02-18T00:43:51.260 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:51.263 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:51.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:51.364 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:51.394 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:43:51.396 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:51.402 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:51.418 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:51.420 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:51.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:51.447 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:51.521 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:51.551 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:51.571 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:51.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:51.574 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:51.675 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:51.710 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:51.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:43:51.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:50.555405+0000 osd.6 (osd.6) 307 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:43:51.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:50.556173+0000 osd.6 (osd.6) 308 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:43:51.942 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.008129+0000 osd.1 (osd.1) 333 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.009569+0000 osd.1 (osd.1) 334 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.042041+0000 osd.2 (osd.2) 281 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.042985+0000 osd.2 (osd.2) 282 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.100775+0000 osd.7 (osd.7) 207 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.101791+0000 osd.7 (osd.7) 208 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:50.555405+0000 osd.6 (osd.6) 307 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:50.556173+0000 osd.6 (osd.6) 308 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.008129+0000 osd.1 (osd.1) 333 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.009569+0000 osd.1 (osd.1) 334 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.042041+0000 osd.2 (osd.2) 281 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:43:51.943 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.042985+0000 osd.2 (osd.2) 282 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:43:51.944 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.100775+0000 osd.7 (osd.7) 207 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:43:51.944 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.101791+0000 osd.7 (osd.7) 208 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:50.555405+0000 osd.6 (osd.6) 307 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:50.556173+0000 osd.6 (osd.6) 308 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.008129+0000 osd.1 (osd.1) 333 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.009569+0000 osd.1 (osd.1) 334 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.042041+0000 osd.2 (osd.2) 281 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:43:52.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.042985+0000 osd.2 (osd.2) 282 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:43:52.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.100775+0000 osd.7 (osd.7) 207 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:43:52.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.101791+0000 osd.7 (osd.7) 208 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:43:52.261 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:52.263 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:52.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1984060464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c102270 msgr2=0x7fcf8c104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:52.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1984060464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c102270 0x7fcf8c104660 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 rx=0x7fcf84009c30 tx=0x7fcf84004c50).stop 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1984060464 shutdown_connections 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1984060464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c0f8d10 0x7fcf8c0f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1984060464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c102270 0x7fcf8c104660 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1984060464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf8c0f9670 0x7fcf8c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1984060464 >> 172.21.15.59:0/1984060464 conn(0x7fcf8c0f2820 msgr2=0x7fcf8c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1984060464 shutdown_connections 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1984060464 wait complete. 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 librados: starting msgr at 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 librados: starting objecter 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.284+0000 7fcf93c4d700 1 Processor -- start 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 -- start start 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 librados: setting wanted keys 2024-02-18T00:43:52.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 librados: calling monclient init 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c0f8d10 0x7fcf8c0f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c0f9670 0x7fcf8c0f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf8c102270 0x7fcf8c0f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c116450 con 0x7fcf8c0f9670 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c1165c0 con 0x7fcf8c0f8d10 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf93c4d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c116780 con 0x7fcf8c102270 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c0f9670 0x7fcf8c0f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf921ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf8c102270 0x7fcf8c0f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c0f9670 0x7fcf8c0f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52568/0 (socket says 172.21.15.59:52568) 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 -- 172.21.15.59:0/1286544366 learned_addr learned my addr 172.21.15.59:0/1286544366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf919e9700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c0f8d10 0x7fcf8c0f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 -- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf8c102270 msgr2=0x7fcf8c0f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:52.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf8c102270 0x7fcf8c0f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 -- 172.21.15.59:0/1286544366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c0f8d10 msgr2=0x7fcf8c0f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c0f8d10 0x7fcf8c0f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 -- 172.21.15.59:0/1286544366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c005040 con 0x7fcf8c0f9670 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf921ea700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf8c102270 0x7fcf8c0f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf919e9700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c0f8d10 0x7fcf8c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf911e8700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c0f9670 0x7fcf8c0f71b0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 rx=0x7fcf7c009400 tx=0x7fcf7c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf82ffd700 10 client.?.objecter ms_handle_connect 0x7fcf8c0f9670 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf82ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.285+0000 7fcf82ffd700 1 -- 172.21.15.59:0/1286544366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c007a40 con 0x7fcf8c0f9670 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.286+0000 7fcf82ffd700 1 -- 172.21.15.59:0/1286544366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf7c00f040 con 0x7fcf8c0f9670 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.286+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf840097c0 con 0x7fcf8c0f9670 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.286+0000 7fcf82ffd700 1 -- 172.21.15.59:0/1286544366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c014aa0 con 0x7fcf8c0f9670 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.286+0000 7fcf93c4d700 10 client.31851.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:52.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.286+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c116dc0 con 0x7fcf8c0f9670 2024-02-18T00:43:52.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.286+0000 7fcf93c4d700 1 librados: init done 2024-02-18T00:43:52.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.287+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c05fa60 con 0x7fcf8c0f9670 2024-02-18T00:43:52.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.287+0000 7fcf82ffd700 1 -- 172.21.15.59:0/1286544366 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fcf7c00e630 con 0x7fcf8c0f9670 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.287+0000 7fcf82ffd700 1 --2- 172.21.15.59:0/1286544366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf78062c90 0x7fcf78065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf82ffd700 1 -- 172.21.15.59:0/1286544366 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(683..683 src has 418..683) v4 ==== 8786+0+0 (secure 0 0 0) 0x7fcf7c07d150 con 0x7fcf8c0f9670 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf82ffd700 10 client.31851.objecter ms_dispatch 0x7fcf8c001550 osd_map(683..683 src has 418..683) v4 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf82ffd700 3 client.31851.objecter handle_osd_map got epochs [683,683] > 0 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf82ffd700 3 client.31851.objecter handle_osd_map decoding full epoch 683 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf919e9700 1 --2- 172.21.15.59:0/1286544366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf78062c90 0x7fcf78065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf82ffd700 20 client.31851.objecter dump_active .. 0 homeless 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf82ffd700 10 client.31851.objecter ms_handle_connect 0x7fcf78062c90 2024-02-18T00:43:52.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.288+0000 7fcf919e9700 1 --2- 172.21.15.59:0/1286544366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf78062c90 0x7fcf78065150 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 rx=0x7fcf88009250 tx=0x7fcf88009de0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:52.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.292+0000 7fcf82ffd700 1 -- 172.21.15.59:0/1286544366 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcf7c049f60 con 0x7fcf8c0f9670 2024-02-18T00:43:52.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:52.398 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:52.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.477+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fcf8c1072d0 con 0x7fcf8c0f9670 2024-02-18T00:43:52.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:43:52] "GET /metrics HTTP/1.1" 200 191489 "" "Prometheus/2.33.4" 2024-02-18T00:43:52.596 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:52.600 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:52.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:52.701 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:52.732 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:52.753 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:52.755 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:52.756 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:52.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.837+0000 7fcf82ffd700 1 -- 172.21.15.59:0/1286544366 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v684) v1 ==== 87+0+0 (secure 0 0 0) 0x7fcf7c050340 con 0x7fcf8c0f9670 2024-02-18T00:43:52.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 10 librados: watch_flush enter 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 10 librados: watch_flush exit 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 20 client.31851.objecter shutdown clearing up homeless session... 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 10 client.31851.objecter successfully canceled tick 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf78062c90 msgr2=0x7fcf78065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1286544366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf78062c90 0x7fcf78065150 secure :-1 s=READY pgs=3197 cs=0 l=1 rev1=1 rx=0x7fcf88009250 tx=0x7fcf88009de0).stop 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c0f9670 msgr2=0x7fcf8c0f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c0f9670 0x7fcf8c0f71b0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 rx=0x7fcf7c009400 tx=0x7fcf7c009cb0).stop 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 shutdown_connections 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1286544366 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf78062c90 0x7fcf78065150 unknown :-1 s=CLOSED pgs=3197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf8c0f8d10 0x7fcf8c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf8c102270 0x7fcf8c0f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 --2- 172.21.15.59:0/1286544366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf8c0f9670 0x7fcf8c0f71b0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:52.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 >> 172.21.15.59:0/1286544366 conn(0x7fcf8c0f2820 msgr2=0x7fcf8c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:52.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 shutdown_connections 2024-02-18T00:43:52.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 -- 172.21.15.59:0/1286544366 wait complete. 2024-02-18T00:43:52.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:52.840+0000 7fcf93c4d700 1 librados: shutdown 2024-02-18T00:43:52.843 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:43:52.857 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.457025+0000 osd.0 (osd.0) 306 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.459741+0000 osd.0 (osd.0) 307 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:51.523178+0000 mgr.y (mgr.24454) 602 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.586855+0000 osd.6 (osd.6) 309 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.588404+0000 osd.6 (osd.6) 310 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.721704+0000 osd.5 (osd.5) 299 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.722656+0000 osd.5 (osd.5) 300 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:51.992948+0000 mgr.y (mgr.24454) 603 : cluster [DBG] pgmap v1047: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.428682+0000 osd.4 (osd.4) 377 : cluster [DBG] 25.19 deep-scrub starts 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.430004+0000 osd.4 (osd.4) 378 : cluster [DBG] 25.19 deep-scrub ok 2024-02-18T00:43:52.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:52.478812+0000 mon.a (mon.0) 3360 : audit [INF] from='client.? 172.21.15.59:0/1286544366' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.457025+0000 osd.0 (osd.0) 306 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.459741+0000 osd.0 (osd.0) 307 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:51.523178+0000 mgr.y (mgr.24454) 602 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.586855+0000 osd.6 (osd.6) 309 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.588404+0000 osd.6 (osd.6) 310 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.721704+0000 osd.5 (osd.5) 299 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.722656+0000 osd.5 (osd.5) 300 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:43:52.868 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:51.992948+0000 mgr.y (mgr.24454) 603 : cluster [DBG] pgmap v1047: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:52.869 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.428682+0000 osd.4 (osd.4) 377 : cluster [DBG] 25.19 deep-scrub starts 2024-02-18T00:43:52.869 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.430004+0000 osd.4 (osd.4) 378 : cluster [DBG] 25.19 deep-scrub ok 2024-02-18T00:43:52.869 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:52.478812+0000 mon.a (mon.0) 3360 : audit [INF] from='client.? 172.21.15.59:0/1286544366' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:52.888 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:53.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.457025+0000 osd.0 (osd.0) 306 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.459741+0000 osd.0 (osd.0) 307 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:51.523178+0000 mgr.y (mgr.24454) 602 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.586855+0000 osd.6 (osd.6) 309 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.588404+0000 osd.6 (osd.6) 310 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.721704+0000 osd.5 (osd.5) 299 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.722656+0000 osd.5 (osd.5) 300 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:51.992948+0000 mgr.y (mgr.24454) 603 : cluster [DBG] pgmap v1047: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.428682+0000 osd.4 (osd.4) 377 : cluster [DBG] 25.19 deep-scrub starts 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.430004+0000 osd.4 (osd.4) 378 : cluster [DBG] 25.19 deep-scrub ok 2024-02-18T00:43:53.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:52.478812+0000 mon.a (mon.0) 3360 : audit [INF] from='client.? 172.21.15.59:0/1286544366' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:43:53.745 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:43:53.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:53.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:43:53.850 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:53.880 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 -- 172.21.15.59:0/3329248775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c2c0f7330 msgr2=0x7f5c2c103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 --2- 172.21.15.59:0/3329248775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c2c0f7330 0x7f5c2c103930 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 rx=0x7f5c24004870 tx=0x7f5c24009cb0).stop 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 -- 172.21.15.59:0/3329248775 shutdown_connections 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 --2- 172.21.15.59:0/3329248775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c2c0fcac0 0x7f5c2c0fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 --2- 172.21.15.59:0/3329248775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c2c0f7330 0x7f5c2c103930 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 --2- 172.21.15.59:0/3329248775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c2c0fd420 0x7f5c2c0f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 -- 172.21.15.59:0/3329248775 >> 172.21.15.59:0/3329248775 conn(0x7f5c2c0f26f0 msgr2=0x7f5c2c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 -- 172.21.15.59:0/3329248775 shutdown_connections 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 -- 172.21.15.59:0/3329248775 wait complete. 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 librados: starting msgr at 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 librados: starting objecter 2024-02-18T00:43:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.119+0000 7f5c31f31700 1 Processor -- start 2024-02-18T00:43:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 -- start start 2024-02-18T00:43:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 librados: setting wanted keys 2024-02-18T00:43:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 librados: calling monclient init 2024-02-18T00:43:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c2c0f7330 0x7f5c2c0f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c2c0fcac0 0x7f5c2c0f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c2c0fd420 0x7f5c2c0f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c116310 con 0x7f5c2c0f7330 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c116480 con 0x7f5c2c0fcac0 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c31f31700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c116640 con 0x7f5c2c0fd420 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c2c0f7330 0x7f5c2c0f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c2c0f7330 0x7f5c2c0f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52572/0 (socket says 172.21.15.59:52572) 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 -- 172.21.15.59:0/2125227438 learned_addr learned my addr 172.21.15.59:0/2125227438 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2bfff700 1 --2- 172.21.15.59:0/2125227438 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c2c0fd420 0x7f5c2c0f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2affd700 1 --2- 172.21.15.59:0/2125227438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c2c0fcac0 0x7f5c2c0f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 -- 172.21.15.59:0/2125227438 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c2c0fd420 msgr2=0x7f5c2c0f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c2c0fd420 0x7f5c2c0f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 -- 172.21.15.59:0/2125227438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c2c0fcac0 msgr2=0x7f5c2c0f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c2c0fcac0 0x7f5c2c0f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 -- 172.21.15.59:0/2125227438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14005040 con 0x7f5c2c0f7330 2024-02-18T00:43:54.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c2c0f7330 0x7f5c2c0f88c0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 rx=0x7f5c1c00db80 tx=0x7f5c1c004f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c28ff9700 10 client.?.objecter ms_handle_connect 0x7f5c2c0f7330 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c28ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.120+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c00b750 con 0x7f5c2c0f7330 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.121+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c24005040 con 0x7f5c2c0f7330 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.121+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c1c00f040 con 0x7f5c2c0f7330 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.121+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c013c50 con 0x7f5c2c0f7330 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.121+0000 7f5c31f31700 10 client.31857.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.121+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c116cd0 con 0x7f5c2c0f7330 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.121+0000 7f5c31f31700 1 librados: init done 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.121+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f5c2c05f990 con 0x7f5c2c0f7330 2024-02-18T00:43:54.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.122+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5c1c00b8b0 con 0x7f5c2c0f7330 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.122+0000 7f5c28ff9700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5c18062c90 0x7f5c18065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.122+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(685..685 src has 418..685) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f5c1c07d1a0 con 0x7f5c2c0f7330 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.122+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 osd_map(685..685 src has 418..685) v4 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.122+0000 7f5c28ff9700 3 client.31857.objecter handle_osd_map got epochs [685,685] > 0 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.122+0000 7f5c28ff9700 3 client.31857.objecter handle_osd_map decoding full epoch 685 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.122+0000 7f5c2affd700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5c18062c90 0x7f5c18065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c28ff9700 20 client.31857.objecter dump_active .. 0 homeless 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v685) v1 ==== 54+0+16 (secure 0 0 0) 0x7f5c1c084070 con 0x7f5c2c0f7330 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c18062c90 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c2affd700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5c18062c90 0x7f5c18065150 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 rx=0x7f5c14009f20 tx=0x7f5c14009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 10 client.31857.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 1 2024-02-18T00:43:54.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5c2c0ffd80 0x7f5c2c100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 1 2024-02-18T00:43:54.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 0 1 2024-02-18T00:43:54.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c05ee80 osd=0 4 2024-02-18T00:43:54.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 10 client.31857.objecter _send_command 1 2024-02-18T00:43:54.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c105fb0 con 0x7f5c2c0ffd80 2024-02-18T00:43:54.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c2bfff700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5c2c0ffd80 0x7f5c2c100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c2bfff700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5c2c0ffd80 0x7f5c2c100160 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.123+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c0ffd80 2024-02-18T00:43:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.124+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c105fb0 con 0x7f5c2c0ffd80 2024-02-18T00:43:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.124+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:43:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.124+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 1 = system:0 2024-02-18T00:43:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.124+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c05ee80 osd=0 4 2024-02-18T00:43:54.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.124+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 0 1 2024-02-18T00:43:54.131 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.458323+0000 osd.0 (osd.0) 308 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.461815+0000 osd.0 (osd.0) 309 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.538337+0000 osd.6 (osd.6) 311 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.539759+0000 osd.6 (osd.6) 312 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.830925+0000 mon.a (mon.0) 3361 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:52.837823+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? 172.21.15.59:0/1286544366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:52.837873+0000 mon.a (mon.0) 3363 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:52.890865+0000 mon.a (mon.0) 3364 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:53.788471+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:53.788512+0000 mon.a (mon.0) 3366 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-18T00:43:54.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:53.789067+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118"}]: dispatch 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.458323+0000 osd.0 (osd.0) 308 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.461815+0000 osd.0 (osd.0) 309 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.538337+0000 osd.6 (osd.6) 311 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.539759+0000 osd.6 (osd.6) 312 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.830925+0000 mon.a (mon.0) 3361 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:52.837823+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? 172.21.15.59:0/1286544366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:52.837873+0000 mon.a (mon.0) 3363 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:52.890865+0000 mon.a (mon.0) 3364 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:53.788471+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:53.788512+0000 mon.a (mon.0) 3366 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-18T00:43:54.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:53.789067+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118"}]: dispatch 2024-02-18T00:43:54.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:54.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 10 client.31857.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 2 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 2 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c05ee80 osd=0 3 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 0 2 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c05ee80 osd=0 4 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 10 client.31857.objecter _send_command 2 2024-02-18T00:43:54.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.139+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c0273c0 con 0x7f5c2c0ffd80 2024-02-18T00:43:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c0273c0 con 0x7f5c2c0ffd80 2024-02-18T00:43:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c05ee80 osd=0 4 2024-02-18T00:43:54.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 0 2 2024-02-18T00:43:54.143 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 10 client.31857.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 3 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6805/26279110 conn(0x7f5c2c0598f0 0x7f5c2c059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 3 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 1 3 2024-02-18T00:43:54.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c028710 osd=1 4 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 10 client.31857.objecter _send_command 3 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c11d6d0 con 0x7f5c2c0598f0 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.141+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6805/26279110 conn(0x7f5c2c0598f0 0x7f5c2c059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.142+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c0598f0 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.142+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6805/26279110 conn(0x7f5c2c0598f0 0x7f5c2c059cf0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.142+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c11d6d0 con 0x7f5c2c0598f0 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.142+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.142+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 3 = system:0 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.142+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c028710 osd=1 4 2024-02-18T00:43:54.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.142+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 1 3 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 10 client.31857.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 4 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 4 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c028710 osd=1 3 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 1 4 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c028710 osd=1 4 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 10 client.31857.objecter _send_command 4 2024-02-18T00:43:54.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.159+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c059360 con 0x7f5c2c0598f0 2024-02-18T00:43:54.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c059360 con 0x7f5c2c0598f0 2024-02-18T00:43:54.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c028710 osd=1 4 2024-02-18T00:43:54.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 1 4 2024-02-18T00:43:54.162 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:43:54.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 10 client.31857.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 5 2024-02-18T00:43:54.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5c2c11c0f0 0x7f5c2c11c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 5 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 2 5 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c027d60 osd=2 4 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 10 client.31857.objecter _send_command 5 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c0283e0 con 0x7f5c2c11c0f0 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c2affd700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5c2c11c0f0 0x7f5c2c11c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c11c0f0 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.161+0000 7f5c2affd700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5c2c11c0f0 0x7f5c2c11c560 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.162+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c0283e0 con 0x7f5c2c11c0f0 2024-02-18T00:43:54.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.162+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:43:54.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.162+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 5 = system:0 2024-02-18T00:43:54.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.162+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c027d60 osd=2 4 2024-02-18T00:43:54.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.162+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 2 5 2024-02-18T00:43:54.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 10 client.31857.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 6 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 6 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c027d60 osd=2 3 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 2 6 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c027d60 osd=2 4 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 10 client.31857.objecter _send_command 6 2024-02-18T00:43:54.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.176+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c11d220 con 0x7f5c2c11c0f0 2024-02-18T00:43:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c11d220 con 0x7f5c2c11c0f0 2024-02-18T00:43:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c027d60 osd=2 4 2024-02-18T00:43:54.179 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 2 6 2024-02-18T00:43:54.180 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:43:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 10 client.31857.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 7 2024-02-18T00:43:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5c2c027940 0x7f5c2c121d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 7 2024-02-18T00:43:54.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 3 7 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c0277d0 osd=3 4 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 10 client.31857.objecter _send_command 7 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c1249c0 con 0x7f5c2c027940 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c2bfff700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5c2c027940 0x7f5c2c121d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c2bfff700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5c2c027940 0x7f5c2c121d70 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.178+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c027940 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.179+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c1249c0 con 0x7f5c2c027940 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.179+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.179+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 7 = system:0 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.179+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c0277d0 osd=3 4 2024-02-18T00:43:54.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.179+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 3 7 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 10 client.31857.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 8 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 8 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c0277d0 osd=3 3 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 3 8 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c0277d0 osd=3 4 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 10 client.31857.objecter _send_command 8 2024-02-18T00:43:54.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.193+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c11cc90 con 0x7f5c2c027940 2024-02-18T00:43:54.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c11cc90 con 0x7f5c2c027940 2024-02-18T00:43:54.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c0277d0 osd=3 4 2024-02-18T00:43:54.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 3 8 2024-02-18T00:43:54.197 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 10 client.31857.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 9 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5c2c11bc50 0x7f5c2c1233e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 9 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 4 9 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11ca50 osd=4 4 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 10 client.31857.objecter _send_command 9 2024-02-18T00:43:54.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c11cbc0 con 0x7f5c2c11bc50 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.195+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5c2c11bc50 0x7f5c2c1233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.196+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c11bc50 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.196+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5c2c11bc50 0x7f5c2c1233e0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.196+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c11cbc0 con 0x7f5c2c11bc50 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.196+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.196+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 9 = system:0 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.196+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c11ca50 osd=4 4 2024-02-18T00:43:54.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.196+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 4 9 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 10 client.31857.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 10 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 10 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c11ca50 osd=4 3 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 4 10 2024-02-18T00:43:54.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11ca50 osd=4 4 2024-02-18T00:43:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 10 client.31857.objecter _send_command 10 2024-02-18T00:43:54.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.211+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c11cbc0 con 0x7f5c2c11bc50 2024-02-18T00:43:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c11cbc0 con 0x7f5c2c11bc50 2024-02-18T00:43:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c11ca50 osd=4 4 2024-02-18T00:43:54.215 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 4 10 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 10 client.31857.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 11 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5c2c123de0 0x7f5c2c124220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 11 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 5 11 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11cbc0 osd=5 4 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 10 client.31857.objecter _send_command 11 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.213+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c124740 con 0x7f5c2c123de0 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.214+0000 7f5c2affd700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5c2c123de0 0x7f5c2c124220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.214+0000 7f5c2affd700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5c2c123de0 0x7f5c2c124220 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.214+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c123de0 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.215+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c124740 con 0x7f5c2c123de0 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.215+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.215+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 11 = system:0 2024-02-18T00:43:54.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.215+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c11cbc0 osd=5 4 2024-02-18T00:43:54.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.215+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 5 11 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 10 client.31857.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 12 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 12 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c11cbc0 osd=5 3 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 5 12 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11cbc0 osd=5 4 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 10 client.31857.objecter _send_command 12 2024-02-18T00:43:54.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.229+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c124740 con 0x7f5c2c123de0 2024-02-18T00:43:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.231+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c124740 con 0x7f5c2c123de0 2024-02-18T00:43:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.231+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.231+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.231+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c11cbc0 osd=5 4 2024-02-18T00:43:54.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.231+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 5 12 2024-02-18T00:43:54.233 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:43:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 10 client.31857.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 13 2024-02-18T00:43:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5c2c1290a0 0x7f5c2c1294c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 13 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 6 13 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c124740 osd=6 4 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 10 client.31857.objecter _send_command 13 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c12b110 con 0x7f5c2c1290a0 2024-02-18T00:43:54.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c2bfff700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5c2c1290a0 0x7f5c2c1294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c1290a0 2024-02-18T00:43:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.232+0000 7f5c2bfff700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5c2c1290a0 0x7f5c2c1294c0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.233+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c12b110 con 0x7f5c2c1290a0 2024-02-18T00:43:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.233+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:43:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.233+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 13 = system:0 2024-02-18T00:43:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.233+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c124740 osd=6 4 2024-02-18T00:43:54.236 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.233+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 6 13 2024-02-18T00:43:54.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:43:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 10 client.31857.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 14 2024-02-18T00:43:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 14 2024-02-18T00:43:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c124740 osd=6 3 2024-02-18T00:43:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 6 14 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c124740 osd=6 4 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 10 client.31857.objecter _send_command 14 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.250+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c12b110 con 0x7f5c2c1290a0 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c12b110 con 0x7f5c2c1290a0 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c124740 osd=6 4 2024-02-18T00:43:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 6 14 2024-02-18T00:43:54.254 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:43:54.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 10 client.31857.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 15 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6812/53025171 conn(0x7f5c2c12d800 0x7f5c2c12dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 15 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 7 15 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c12b110 osd=7 4 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 10 client.31857.objecter _send_command 15 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c2c12e2a0 con 0x7f5c2c12d800 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.252+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6812/53025171 conn(0x7f5c2c12d800 0x7f5c2c12dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:54.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.253+0000 7f5c28ff9700 10 client.31857.objecter ms_handle_connect 0x7f5c2c12d800 2024-02-18T00:43:54.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.253+0000 7f5c2b7fe700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6812/53025171 conn(0x7f5c2c12d800 0x7f5c2c12dc60 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.458323+0000 osd.0 (osd.0) 308 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:43:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.461815+0000 osd.0 (osd.0) 309 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:43:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.538337+0000 osd.6 (osd.6) 311 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:43:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.539759+0000 osd.6 (osd.6) 312 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:43:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.830925+0000 mon.a (mon.0) 3361 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:43:54.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:52.837823+0000 mon.a (mon.0) 3362 : audit [INF] from='client.? 172.21.15.59:0/1286544366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:43:54.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:52.837873+0000 mon.a (mon.0) 3363 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-18T00:43:54.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:52.890865+0000 mon.a (mon.0) 3364 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:43:54.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:53.788471+0000 mon.a (mon.0) 3365 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:43:54.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:53.788512+0000 mon.a (mon.0) 3366 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-18T00:43:54.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:53.789067+0000 mon.a (mon.0) 3367 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118"}]: dispatch 2024-02-18T00:43:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.253+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5c2c12e2a0 con 0x7f5c2c12d800 2024-02-18T00:43:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.253+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:43:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.253+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 15 = system:0 2024-02-18T00:43:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.253+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c12b110 osd=7 4 2024-02-18T00:43:54.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.253+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 7 15 2024-02-18T00:43:54.265 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 10 client.31857.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:43:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign -1 16 2024-02-18T00:43:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 20 client.31857.objecter _get_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_remove -1 16 2024-02-18T00:43:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 20 client.31857.objecter get_session s=0x7f5c2c12b110 osd=7 3 2024-02-18T00:43:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 15 client.31857.objecter _session_command_op_assign 7 16 2024-02-18T00:43:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c12b110 osd=7 4 2024-02-18T00:43:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 10 client.31857.objecter _send_command 16 2024-02-18T00:43:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.269+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f5c2c12e2a0 con 0x7f5c2c12d800 2024-02-18T00:43:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.271+0000 7f5c28ff9700 1 -- 172.21.15.59:0/2125227438 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f5c2c12e2a0 con 0x7f5c2c12d800 2024-02-18T00:43:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.271+0000 7f5c28ff9700 10 client.31857.objecter ms_dispatch 0x7f5c2c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:43:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.271+0000 7f5c28ff9700 10 client.31857.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.271+0000 7f5c28ff9700 20 client.31857.objecter put_session s=0x7f5c2c12b110 osd=7 4 2024-02-18T00:43:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.271+0000 7f5c28ff9700 15 client.31857.objecter _session_command_op_remove 7 16 2024-02-18T00:43:54.274 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:43:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 librados: watch_flush enter 2024-02-18T00:43:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 librados: watch_flush exit 2024-02-18T00:43:54.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.0 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5c2c0ffd80 msgr2=0x7f5c2c100160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6801/3700238977 conn(0x7f5c2c0ffd80 0x7f5c2c100160 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c05ee80 osd=0 1 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.1 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6805/26279110 conn(0x7f5c2c0598f0 msgr2=0x7f5c2c059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6805/26279110 conn(0x7f5c2c0598f0 0x7f5c2c059cf0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c028710 osd=1 1 2024-02-18T00:43:54.276 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.2 2024-02-18T00:43:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5c2c11c0f0 msgr2=0x7f5c2c11c560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6809/1153193637 conn(0x7f5c2c11c0f0 0x7f5c2c11c560 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c027d60 osd=2 1 2024-02-18T00:43:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.3 2024-02-18T00:43:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5c2c027940 msgr2=0x7f5c2c121d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6813/1805700964 conn(0x7f5c2c027940 0x7f5c2c121d70 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c2bfff700 1 -- 172.21.15.59:0/2125227438 reap_dead start 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c0277d0 osd=3 1 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.4 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5c2c11bc50 msgr2=0x7f5c2c1233e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5c2c11bc50 0x7f5c2c1233e0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c2bfff700 1 -- 172.21.15.59:0/2125227438 reap_dead start 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11ca50 osd=4 1 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.5 2024-02-18T00:43:54.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5c2c123de0 msgr2=0x7f5c2c124220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6804/2170432263 conn(0x7f5c2c123de0 0x7f5c2c124220 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c11cbc0 osd=5 1 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.6 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5c2c1290a0 msgr2=0x7f5c2c1294c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6808/3356009390 conn(0x7f5c2c1290a0 0x7f5c2c1294c0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c124740 osd=6 1 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter close_session for osd.7 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6812/53025171 conn(0x7f5c2c12d800 msgr2=0x7f5c2c12dc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.137:6812/53025171 conn(0x7f5c2c12d800 0x7f5c2c12dc60 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:54.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter put_session s=0x7f5c2c12b110 osd=7 1 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 20 client.31857.objecter shutdown clearing up homeless session... 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 10 client.31857.objecter successfully canceled tick 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5c18062c90 msgr2=0x7f5c18065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5c18062c90 0x7f5c18065150 secure :-1 s=READY pgs=3198 cs=0 l=1 rev1=1 rx=0x7f5c14009f20 tx=0x7f5c14009cb0).stop 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c2c0f7330 msgr2=0x7f5c2c0f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 --2- 172.21.15.59:0/2125227438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c2c0f7330 0x7f5c2c0f88c0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 rx=0x7f5c1c00db80 tx=0x7f5c1c004f00).stop 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c2bfff700 1 -- 172.21.15.59:0/2125227438 reap_dead start 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 shutdown_connections 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 >> 172.21.15.59:0/2125227438 conn(0x7f5c2c0f26f0 msgr2=0x7f5c2c060e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 shutdown_connections 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.272+0000 7f5c31f31700 1 -- 172.21.15.59:0/2125227438 wait complete. 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:54.273+0000 7f5c31f31700 1 librados: shutdown 2024-02-18T00:43:54.280 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.281 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:43:54.515 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:43:54.518 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:54.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:43:54.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:54.652 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:54.676 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:54.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:54.777 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:54.831 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:54.834 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:54.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:54.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:53.993516+0000 mgr.y (mgr.24454) 604 : cluster [DBG] pgmap v1050: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:54.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:54.123744+0000 mon.a (mon.0) 3368 : audit [DBG] from='client.? 172.21.15.59:0/2125227438' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:54.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:54.786334+0000 mon.a (mon.0) 3369 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:54.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:54.790706+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118"}]': finished 2024-02-18T00:43:54.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:54.790765+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-18T00:43:54.935 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:54.963 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:53.993516+0000 mgr.y (mgr.24454) 604 : cluster [DBG] pgmap v1050: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:54.963 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:54.123744+0000 mon.a (mon.0) 3368 : audit [DBG] from='client.? 172.21.15.59:0/2125227438' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:54.963 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:54.786334+0000 mon.a (mon.0) 3369 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:54.963 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:54.790706+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118"}]': finished 2024-02-18T00:43:54.963 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:54.790765+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-18T00:43:54.964 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:54.984 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:54.986 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:54.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:55.087 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:43:55.115 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:55.133 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:43:55.135 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:55.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:43:55.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:55.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:53.993516+0000 mgr.y (mgr.24454) 604 : cluster [DBG] pgmap v1050: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:54.123744+0000 mon.a (mon.0) 3368 : audit [DBG] from='client.? 172.21.15.59:0/2125227438' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:43:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:54.786334+0000 mon.a (mon.0) 3369 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:43:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:54.790706+0000 mon.a (mon.0) 3370 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-118"}]': finished 2024-02-18T00:43:55.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:54.790765+0000 mon.a (mon.0) 3371 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-18T00:43:55.263 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:55.281 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:55.284 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:55.284 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:55.385 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:43:55.412 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:55.432 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:55.432 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:43:55.433 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:43:55.534 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:55.619 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:55.621 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:55.621 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:55.722 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 -- 172.21.15.59:0/2477398387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fd510 msgr2=0x7f93100f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 --2- 172.21.15.59:0/2477398387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fd510 0x7f93100f6d40 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 rx=0x7f9304004870 tx=0x7f9304009cb0).stop 2024-02-18T00:43:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 -- 172.21.15.59:0/2477398387 shutdown_connections 2024-02-18T00:43:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 --2- 172.21.15.59:0/2477398387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93100fcbb0 0x7f93100fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 --2- 172.21.15.59:0/2477398387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93100f7400 0x7f9310103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 --2- 172.21.15.59:0/2477398387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fd510 0x7f93100f6d40 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 -- 172.21.15.59:0/2477398387 >> 172.21.15.59:0/2477398387 conn(0x7f93100f2820 msgr2=0x7f93100f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 -- 172.21.15.59:0/2477398387 shutdown_connections 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.727+0000 7f9315138700 1 -- 172.21.15.59:0/2477398387 wait complete. 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 librados: starting msgr at 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 librados: starting objecter 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 Processor -- start 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 -- start start 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 librados: setting wanted keys 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 librados: calling monclient init 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93100f7400 0x7f931010e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fcbb0 0x7f931010e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93100fd510 0x7f93101082e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93101087d0 con 0x7f93100fd510 2024-02-18T00:43:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310108940 con 0x7f93100f7400 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.728+0000 7f9315138700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9310108b00 con 0x7f93100fcbb0 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fcbb0 0x7f931010e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93100fd510 0x7f93101082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fcbb0 0x7f931010e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:45190/0 (socket says 172.21.15.59:45190) 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 -- 172.21.15.59:0/1894044976 learned_addr learned my addr 172.21.15.59:0/1894044976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930ed9d700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93100f7400 0x7f931010e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 -- 172.21.15.59:0/1894044976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93100f7400 msgr2=0x7f931010e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93100f7400 0x7f931010e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 -- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93100fd510 msgr2=0x7f93101082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93100fd510 0x7f93101082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 -- 172.21.15.59:0/1894044976 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304005040 con 0x7f93100fcbb0 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930f59e700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93100fd510 0x7f93101082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f930e59c700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fcbb0 0x7f931010e630 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7f9304009eb0 tx=0x7f930400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:55.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f92f7fff700 10 client.?.objecter ms_handle_connect 0x7f93100fcbb0 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f92f7fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f92f7fff700 1 -- 172.21.15.59:0/1894044976 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930400e070 con 0x7f93100fcbb0 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f92f7fff700 1 -- 172.21.15.59:0/1894044976 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9304013da0 con 0x7f93100fcbb0 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f92f7fff700 1 -- 172.21.15.59:0/1894044976 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9304019070 con 0x7f93100fcbb0 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9310108d20 con 0x7f93100fcbb0 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f9315138700 10 client.41669.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93100f87f0 con 0x7f93100fcbb0 2024-02-18T00:43:55.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.729+0000 7f9315138700 1 librados: init done 2024-02-18T00:43:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.730+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93101091b0 con 0x7f93100fcbb0 2024-02-18T00:43:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.731+0000 7f92f7fff700 1 -- 172.21.15.59:0/1894044976 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f930401e420 con 0x7f93100fcbb0 2024-02-18T00:43:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f92f7fff700 1 --2- 172.21.15.59:0/1894044976 >> v2:172.21.15.59:6800/3721891797 conn(0x7f92f8062c90 0x7f92f8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:43:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f92f7fff700 1 -- 172.21.15.59:0/1894044976 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(686..686 src has 418..686) v4 ==== 8778+0+0 (secure 0 0 0) 0x7f930407e1c0 con 0x7f93100fcbb0 2024-02-18T00:43:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f92f7fff700 10 client.41669.objecter ms_dispatch 0x7f9310001550 osd_map(686..686 src has 418..686) v4 2024-02-18T00:43:55.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f92f7fff700 3 client.41669.objecter handle_osd_map got epochs [686,686] > 0 2024-02-18T00:43:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f92f7fff700 3 client.41669.objecter handle_osd_map decoding full epoch 686 2024-02-18T00:43:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f930ed9d700 1 --2- 172.21.15.59:0/1894044976 >> v2:172.21.15.59:6800/3721891797 conn(0x7f92f8062c90 0x7f92f8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:43:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f92f7fff700 20 client.41669.objecter dump_active .. 0 homeless 2024-02-18T00:43:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f92f7fff700 10 client.41669.objecter ms_handle_connect 0x7f92f8062c90 2024-02-18T00:43:55.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.732+0000 7f930ed9d700 1 --2- 172.21.15.59:0/1894044976 >> v2:172.21.15.59:6800/3721891797 conn(0x7f92f8062c90 0x7f92f8065150 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 rx=0x7f9300004dc0 tx=0x7f930000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:43:55.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.736+0000 7f92f7fff700 1 -- 172.21.15.59:0/1894044976 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f93040513c0 con 0x7f93100fcbb0 2024-02-18T00:43:55.753 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:55.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:55.931+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f931005fa60 con 0x7f93100fcbb0 2024-02-18T00:43:56.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:54.908078+0000 mon.a (mon.0) 3372 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:56.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:55.792733+0000 mon.a (mon.0) 3373 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-18T00:43:56.465 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:54.908078+0000 mon.a (mon.0) 3372 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:56.465 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:55.792733+0000 mon.a (mon.0) 3373 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-18T00:43:56.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:54.908078+0000 mon.a (mon.0) 3372 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:43:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:55.792733+0000 mon.a (mon.0) 3373 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-18T00:43:56.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.804+0000 7f92f7fff700 1 -- 172.21.15.59:0/1894044976 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v688) v1 ==== 113+0+0 (secure 0 0 0) 0x7f9304023020 con 0x7f93100fcbb0 2024-02-18T00:43:56.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.809+0000 7f9315138700 10 librados: watch_flush enter 2024-02-18T00:43:56.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.809+0000 7f9315138700 10 librados: watch_flush exit 2024-02-18T00:43:56.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.809+0000 7f9315138700 20 client.41669.objecter shutdown clearing up homeless session... 2024-02-18T00:43:56.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.809+0000 7f9315138700 10 client.41669.objecter successfully canceled tick 2024-02-18T00:43:56.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.810+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 >> v2:172.21.15.59:6800/3721891797 conn(0x7f92f8062c90 msgr2=0x7f92f8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:56.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.810+0000 7f9315138700 1 --2- 172.21.15.59:0/1894044976 >> v2:172.21.15.59:6800/3721891797 conn(0x7f92f8062c90 0x7f92f8065150 secure :-1 s=READY pgs=3199 cs=0 l=1 rev1=1 rx=0x7f9300004dc0 tx=0x7f930000b040).stop 2024-02-18T00:43:56.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.810+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fcbb0 msgr2=0x7f931010e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:43:56.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.810+0000 7f9315138700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fcbb0 0x7f931010e630 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 rx=0x7f9304009eb0 tx=0x7f930400b040).stop 2024-02-18T00:43:56.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 shutdown_connections 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 --2- 172.21.15.59:0/1894044976 >> v2:172.21.15.59:6800/3721891797 conn(0x7f92f8062c90 0x7f92f8065150 unknown :-1 s=CLOSED pgs=3199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93100f7400 0x7f931010e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93100fd510 0x7f93101082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 --2- 172.21.15.59:0/1894044976 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93100fcbb0 0x7f931010e630 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 >> 172.21.15.59:0/1894044976 conn(0x7f93100f2820 msgr2=0x7f9310060d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 shutdown_connections 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.811+0000 7f9315138700 1 -- 172.21.15.59:0/1894044976 wait complete. 2024-02-18T00:43:56.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:43:56.812+0000 7f9315138700 1 librados: shutdown 2024-02-18T00:43:56.815 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.1 to 1 (10000) 2024-02-18T00:43:57.404 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:57.406 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:57.406 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:57.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:55.933213+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.59:0/1894044976' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:55.994244+0000 mgr.y (mgr.24454) 605 : cluster [DBG] pgmap v1053: 182 pgs: 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:56.240076+0000 mon.a (mon.0) 3374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:56.805135+0000 mon.a (mon.0) 3375 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:56.805220+0000 mon.a (mon.0) 3376 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:56.816185+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:55.933213+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.59:0/1894044976' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:55.994244+0000 mgr.y (mgr.24454) 605 : cluster [DBG] pgmap v1053: 182 pgs: 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:56.240076+0000 mon.a (mon.0) 3374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:56.805135+0000 mon.a (mon.0) 3375 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:43:57.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:56.805220+0000 mon.a (mon.0) 3376 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-18T00:43:57.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:56.816185+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:57.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:55.933213+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.59:0/1894044976' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:43:57.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:55.994244+0000 mgr.y (mgr.24454) 605 : cluster [DBG] pgmap v1053: 182 pgs: 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:57.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:56.240076+0000 mon.a (mon.0) 3374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:43:57.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:56.805135+0000 mon.a (mon.0) 3375 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:43:57.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:56.805220+0000 mon.a (mon.0) 3376 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-18T00:43:57.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:56.816185+0000 mon.a (mon.0) 3377 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:43:57.507 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:57.537 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:58.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 2 1 2024-02-18T00:43:58.087 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:58.090 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:58.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:58.191 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:58.199 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:58.223 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:58.245 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:58.247 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:58.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:58.348 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:43:58.377 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:58.397 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:43:58.399 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:58.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:43:58.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:43:58.501 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:58.528 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:58.547 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:58.550 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:58.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:58.651 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:43:58.680 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:43:58.708 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:58.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:43:58.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:43:58.839 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:57.802326+0000 mon.a (mon.0) 3378 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:59.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:57.802454+0000 mon.a (mon.0) 3379 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-18T00:43:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:57.818630+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:57.994695+0000 mgr.y (mgr.24454) 606 : cluster [DBG] pgmap v1056: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:57.802326+0000 mon.a (mon.0) 3378 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:57.802454+0000 mon.a (mon.0) 3379 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-18T00:43:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:57.818630+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:59.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:57.994695+0000 mgr.y (mgr.24454) 606 : cluster [DBG] pgmap v1056: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:59.079 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:59.089 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:43:59.091 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:59.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:43:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:43:59.192 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:43:59.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:57.802326+0000 mon.a (mon.0) 3378 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:43:59.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:57.802454+0000 mon.a (mon.0) 3379 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-18T00:43:59.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:57.818630+0000 mon.a (mon.0) 3380 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:43:59.221 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:57.994695+0000 mgr.y (mgr.24454) 606 : cluster [DBG] pgmap v1056: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:43:59.223 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:43:59.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:43:59.243 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:43:59.246 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:43:59.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:43:59.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:43:59.378 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:00.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:58.810857+0000 mon.a (mon.0) 3381 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:58.810922+0000 mon.a (mon.0) 3382 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-18T00:44:00.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:43:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:58.812076+0000 mon.a (mon.0) 3383 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-120"}]: dispatch 2024-02-18T00:44:00.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:58.810857+0000 mon.a (mon.0) 3381 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:00.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:58.810922+0000 mon.a (mon.0) 3382 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-18T00:44:00.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:43:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:58.812076+0000 mon.a (mon.0) 3383 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-120"}]: dispatch 2024-02-18T00:44:00.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:58.810857+0000 mon.a (mon.0) 3381 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:58.810922+0000 mon.a (mon.0) 3382 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-18T00:44:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:43:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:58.812076+0000 mon.a (mon.0) 3383 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-120"}]: dispatch 2024-02-18T00:44:01.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:59.812676+0000 mon.a (mon.0) 3384 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-120"}]': finished 2024-02-18T00:44:01.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:59.812752+0000 mon.a (mon.0) 3385 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-18T00:44:01.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:43:59.815136+0000 mon.a (mon.0) 3386 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-120", "mode": "writeback"}]: dispatch 2024-02-18T00:44:01.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:43:59.995141+0000 mgr.y (mgr.24454) 607 : cluster [DBG] pgmap v1059: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:01.114 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:00.807610+0000 mon.a (mon.0) 3387 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:00.813157+0000 mon.a (mon.0) 3388 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-120", "mode": "writeback"}]': finished 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:00.813254+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:59.812676+0000 mon.a (mon.0) 3384 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-120"}]': finished 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:59.812752+0000 mon.a (mon.0) 3385 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:43:59.815136+0000 mon.a (mon.0) 3386 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-120", "mode": "writeback"}]: dispatch 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:43:59.995141+0000 mgr.y (mgr.24454) 607 : cluster [DBG] pgmap v1059: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:00.807610+0000 mon.a (mon.0) 3387 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:00.813157+0000 mon.a (mon.0) 3388 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-120", "mode": "writeback"}]': finished 2024-02-18T00:44:01.115 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:00.813254+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-18T00:44:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:01.136 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:01.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:01.238 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:01.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:44:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:44:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:59.812676+0000 mon.a (mon.0) 3384 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-120"}]': finished 2024-02-18T00:44:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:59.812752+0000 mon.a (mon.0) 3385 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-18T00:44:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:43:59.815136+0000 mon.a (mon.0) 3386 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-120", "mode": "writeback"}]: dispatch 2024-02-18T00:44:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:43:59.995141+0000 mgr.y (mgr.24454) 607 : cluster [DBG] pgmap v1059: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:00.807610+0000 mon.a (mon.0) 3387 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:00.813157+0000 mon.a (mon.0) 3388 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-120", "mode": "writeback"}]': finished 2024-02-18T00:44:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:00.813254+0000 mon.a (mon.0) 3389 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-18T00:44:01.270 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:01.292 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:01.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:01.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:01.396 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:01.446 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:01.449 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:01.449 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:01.550 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:01.579 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:01.599 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:01.600 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:01.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:01.702 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:01.729 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:01.748 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:01.750 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:01.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:01.825 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:44:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:44:01.852 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:01.897 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:01.903 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:01.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:02.004 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:02.036 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:02.088 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:00.901469+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:02.089 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:00.901469+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:02.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:00.901469+0000 mon.a (mon.0) 3390 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:44:02.451 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:44:02.478 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:02] "GET /metrics HTTP/1.1" 200 191636 "" "Prometheus/2.33.4" 2024-02-18T00:44:02.787 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:02.790 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:02.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:02.891 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:02.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:02.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:03.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:03.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:01.532550+0000 mgr.y (mgr.24454) 608 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:03.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:01.831814+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:03.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:01.831902+0000 mon.a (mon.0) 3392 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-18T00:44:03.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:01.833836+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120"}]: dispatch 2024-02-18T00:44:03.075 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:01.995514+0000 mgr.y (mgr.24454) 609 : cluster [DBG] pgmap v1062: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:03.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:01.532550+0000 mgr.y (mgr.24454) 608 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:03.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:01.831814+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:03.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:01.831902+0000 mon.a (mon.0) 3392 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-18T00:44:03.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:01.833836+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120"}]: dispatch 2024-02-18T00:44:03.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:01.995514+0000 mgr.y (mgr.24454) 609 : cluster [DBG] pgmap v1062: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:03.077 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:01.532550+0000 mgr.y (mgr.24454) 608 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:01.831814+0000 mon.a (mon.0) 3391 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:01.831902+0000 mon.a (mon.0) 3392 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-18T00:44:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:01.833836+0000 mon.a (mon.0) 3393 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120"}]: dispatch 2024-02-18T00:44:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:01.995514+0000 mgr.y (mgr.24454) 609 : cluster [DBG] pgmap v1062: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:03.391 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.0 dump_historic_ops 2024-02-18T00:44:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 -- 172.21.15.59:0/600611538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640fd510 msgr2=0x7ff6640f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 --2- 172.21.15.59:0/600611538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640fd510 0x7ff6640f6d40 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 rx=0x7ff6500047b0 tx=0x7ff650009cb0).stop 2024-02-18T00:44:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 -- 172.21.15.59:0/600611538 shutdown_connections 2024-02-18T00:44:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 --2- 172.21.15.59:0/600611538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6640f7400 0x7ff664103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 --2- 172.21.15.59:0/600611538 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 0x7ff6640fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 --2- 172.21.15.59:0/600611538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640fd510 0x7ff6640f6d40 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 -- 172.21.15.59:0/600611538 >> 172.21.15.59:0/600611538 conn(0x7ff6640f2820 msgr2=0x7ff6640f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 -- 172.21.15.59:0/600611538 shutdown_connections 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.672+0000 7ff66aaba700 1 -- 172.21.15.59:0/600611538 wait complete. 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 librados: starting msgr at 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 librados: starting objecter 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 Processor -- start 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 -- start start 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 librados: setting wanted keys 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 librados: calling monclient init 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640f7400 0x7ff664109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 0x7ff664109880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6640fd510 0x7ff664109e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff664114580 con 0x7ff6640f7400 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641146f0 con 0x7ff6640fd510 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff66aaba700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6641148b0 con 0x7ff6640fcbb0 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff668856700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640f7400 0x7ff664109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff663fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 0x7ff664109880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff663fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 0x7ff664109880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:42018/0 (socket says 172.21.15.59:42018) 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff668856700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640f7400 0x7ff664109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42408/0 (socket says 172.21.15.59:42408) 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff663fff700 1 -- 172.21.15.59:0/1898582875 learned_addr learned my addr 172.21.15.59:0/1898582875 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.673+0000 7ff663fff700 1 -- 172.21.15.59:0/1898582875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6640fd510 msgr2=0x7ff664109e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff669057700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6640fd510 0x7ff664109e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff663fff700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6640fd510 0x7ff664109e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff663fff700 1 -- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640f7400 msgr2=0x7ff664109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff663fff700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640f7400 0x7ff664109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff663fff700 1 -- 172.21.15.59:0/1898582875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658005040 con 0x7ff6640fcbb0 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff668856700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640f7400 0x7ff664109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff663fff700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 0x7ff664109880 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 rx=0x7ff650009b10 tx=0x7ff650009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff661f76700 10 client.?.objecter ms_handle_connect 0x7ff6640fcbb0 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff661f76700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff661f76700 1 -- 172.21.15.59:0/1898582875 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650018070 con 0x7ff6640fcbb0 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff661f76700 1 -- 172.21.15.59:0/1898582875 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff650013e70 con 0x7ff6640fcbb0 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff650005040 con 0x7ff6640fcbb0 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff661f76700 1 -- 172.21.15.59:0/1898582875 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65001ddb0 con 0x7ff6640fcbb0 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff66aaba700 10 client.41675.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff664114d50 con 0x7ff6640fcbb0 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.674+0000 7ff66aaba700 1 librados: init done 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.675+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66405fa60 con 0x7ff6640fcbb0 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.675+0000 7ff661f76700 1 -- 172.21.15.59:0/1898582875 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff65001d420 con 0x7ff6640fcbb0 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff661f76700 1 --2- 172.21.15.59:0/1898582875 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff654062c90 0x7ff654065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff661f76700 1 -- 172.21.15.59:0/1898582875 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(693..693 src has 418..693) v4 ==== 8786+0+0 (secure 0 0 0) 0x7ff65007e960 con 0x7ff6640fcbb0 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff661f76700 10 client.41675.objecter ms_dispatch 0x7ff664001550 osd_map(693..693 src has 418..693) v4 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff661f76700 3 client.41675.objecter handle_osd_map got epochs [693,693] > 0 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff661f76700 3 client.41675.objecter handle_osd_map decoding full epoch 693 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff668856700 1 --2- 172.21.15.59:0/1898582875 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff654062c90 0x7ff654065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff661f76700 20 client.41675.objecter dump_active .. 0 homeless 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff661f76700 10 client.41675.objecter ms_handle_connect 0x7ff654062c90 2024-02-18T00:44:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.676+0000 7ff668856700 1 --2- 172.21.15.59:0/1898582875 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff654062c90 0x7ff654065150 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 rx=0x7ff658004c80 tx=0x7ff65800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:03.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.680+0000 7ff661f76700 1 -- 172.21.15.59:0/1898582875 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff650051b50 con 0x7ff6640fcbb0 2024-02-18T00:44:03.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:02.875+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) v1 -- 0x7ff6641060e0 con 0x7ff6640fcbb0 2024-02-18T00:44:03.526 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:03.528 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:03.529 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:03.630 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:03.662 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:03.683 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:03.686 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:03.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:03.787 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:03.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.792+0000 7ff661f76700 1 -- 172.21.15.59:0/1898582875 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v695) v1 ==== 113+0+0 (secure 0 0 0) 0x7ff650051740 con 0x7ff6640fcbb0 2024-02-18T00:44:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.795+0000 7ff66aaba700 10 librados: watch_flush enter 2024-02-18T00:44:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.799+0000 7ff66aaba700 10 librados: watch_flush exit 2024-02-18T00:44:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.799+0000 7ff66aaba700 20 client.41675.objecter shutdown clearing up homeless session... 2024-02-18T00:44:03.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.799+0000 7ff66aaba700 10 client.41675.objecter successfully canceled tick 2024-02-18T00:44:03.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.799+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff654062c90 msgr2=0x7ff654065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:03.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.800+0000 7ff66aaba700 1 --2- 172.21.15.59:0/1898582875 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff654062c90 0x7ff654065150 secure :-1 s=READY pgs=3200 cs=0 l=1 rev1=1 rx=0x7ff658004c80 tx=0x7ff65800b040).stop 2024-02-18T00:44:03.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.800+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 msgr2=0x7ff664109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:03.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.800+0000 7ff66aaba700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 0x7ff664109880 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 rx=0x7ff650009b10 tx=0x7ff650009cb0).stop 2024-02-18T00:44:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 shutdown_connections 2024-02-18T00:44:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 --2- 172.21.15.59:0/1898582875 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff654062c90 0x7ff654065150 unknown :-1 s=CLOSED pgs=3200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6640fd510 0x7ff664109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6640f7400 0x7ff664109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 --2- 172.21.15.59:0/1898582875 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6640fcbb0 0x7ff664109880 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 >> 172.21.15.59:0/1898582875 conn(0x7ff6640f2820 msgr2=0x7ff6640f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 shutdown_connections 2024-02-18T00:44:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 -- 172.21.15.59:0/1898582875 wait complete. 2024-02-18T00:44:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:03.802+0000 7ff66aaba700 1 librados: shutdown 2024-02-18T00:44:03.804 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.2 to 1 (10000) 2024-02-18T00:44:03.822 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:02.829227+0000 mon.a (mon.0) 3394 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:02.837712+0000 mon.a (mon.0) 3395 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120"}]': finished 2024-02-18T00:44:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:02.837805+0000 mon.a (mon.0) 3396 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-18T00:44:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:02.876614+0000 mon.c (mon.2) 566 : audit [INF] from='client.? 172.21.15.59:0/1898582875' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:44:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:02.876879+0000 mon.a (mon.0) 3397 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:44:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:03.792510+0000 mon.a (mon.0) 3398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:44:04.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:03.792592+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-18T00:44:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:02.829227+0000 mon.a (mon.0) 3394 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:02.837712+0000 mon.a (mon.0) 3395 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120"}]': finished 2024-02-18T00:44:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:02.837805+0000 mon.a (mon.0) 3396 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-18T00:44:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:02.876614+0000 mon.c (mon.2) 566 : audit [INF] from='client.? 172.21.15.59:0/1898582875' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:44:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:02.876879+0000 mon.a (mon.0) 3397 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:44:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:03.792510+0000 mon.a (mon.0) 3398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:44:04.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:03.792592+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-18T00:44:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:02.829227+0000 mon.a (mon.0) 3394 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:02.837712+0000 mon.a (mon.0) 3395 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-120"}]': finished 2024-02-18T00:44:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:02.837805+0000 mon.a (mon.0) 3396 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-18T00:44:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:02.876614+0000 mon.c (mon.2) 566 : audit [INF] from='client.? 172.21.15.59:0/1898582875' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:44:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:02.876879+0000 mon.a (mon.0) 3397 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:44:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:03.792510+0000 mon.a (mon.0) 3398 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:44:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:03.792592+0000 mon.a (mon.0) 3399 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-18T00:44:04.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 3 1 2024-02-18T00:44:05.423 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:05.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:03.995849+0000 mgr.y (mgr.24454) 610 : cluster [DBG] pgmap v1065: 182 pgs: 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:05.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:04.798496+0000 mon.a (mon.0) 3400 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-18T00:44:05.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:04.801845+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:05.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:03.995849+0000 mgr.y (mgr.24454) 610 : cluster [DBG] pgmap v1065: 182 pgs: 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:05.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:04.798496+0000 mon.a (mon.0) 3400 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-18T00:44:05.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:04.801845+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:05.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:03.995849+0000 mgr.y (mgr.24454) 610 : cluster [DBG] pgmap v1065: 182 pgs: 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:05.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:04.798496+0000 mon.a (mon.0) 3400 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-18T00:44:05.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:04.801845+0000 mon.a (mon.0) 3401 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:05.784 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:05.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:05.885 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:05.920 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:05.940 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:05.942 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:05.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:06.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:06.077 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:06.115 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:06.117 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:06.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:06.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:06.248 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:06.275 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:06.278 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:06.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:06.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:06.408 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:06.429 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:06.431 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:06.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:06.533 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:06.566 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:06.587 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:06.589 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:06.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:06.691 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:06.720 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:06.843 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:06.856 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:06.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:06.958 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:06.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:05.803101+0000 mon.a (mon.0) 3402 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:06.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:05.803301+0000 mon.a (mon.0) 3403 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-18T00:44:06.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:05.821077+0000 mon.a (mon.0) 3404 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:06.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:05.996917+0000 mgr.y (mgr.24454) 611 : cluster [DBG] pgmap v1068: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:44:06.988 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:06.577677+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? 172.21.15.59:0/812532317' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:06.990 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:07.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:05.803101+0000 mon.a (mon.0) 3402 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:05.803301+0000 mon.a (mon.0) 3403 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:05.821077+0000 mon.a (mon.0) 3404 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:05.996917+0000 mgr.y (mgr.24454) 611 : cluster [DBG] pgmap v1068: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:06.577677+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? 172.21.15.59:0/812532317' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:05.803101+0000 mon.a (mon.0) 3402 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:05.803301+0000 mon.a (mon.0) 3403 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:05.821077+0000 mon.a (mon.0) 3404 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:07.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:05.996917+0000 mgr.y (mgr.24454) 611 : cluster [DBG] pgmap v1068: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:44:07.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:06.577677+0000 mon.a (mon.0) 3405 : audit [INF] from='client.? 172.21.15.59:0/812532317' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:07.011 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:07.014 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:07.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:07.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:07.345 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:07.345 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:07.446 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:07.479 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:07.500 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:07.502 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:07.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:07.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:07.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 -- 172.21.15.59:0/3906403979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 msgr2=0x7f650c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:07.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 --2- 172.21.15.59:0/3906403979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c103a20 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 rx=0x7f6504009be0 tx=0x7f6504004d10).stop 2024-02-18T00:44:07.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 -- 172.21.15.59:0/3906403979 shutdown_connections 2024-02-18T00:44:07.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 --2- 172.21.15.59:0/3906403979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c103a20 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 --2- 172.21.15.59:0/3906403979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650c0fcbb0 0x7f650c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 --2- 172.21.15.59:0/3906403979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c0fd510 0x7f650c0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 -- 172.21.15.59:0/3906403979 >> 172.21.15.59:0/3906403979 conn(0x7f650c0f2820 msgr2=0x7f650c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 -- 172.21.15.59:0/3906403979 shutdown_connections 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 -- 172.21.15.59:0/3906403979 wait complete. 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 librados: starting msgr at 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 librados: starting objecter 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 Processor -- start 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 -- start start 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 librados: setting wanted keys 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 librados: calling monclient init 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.375+0000 7f6513722700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c10e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:07.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650c0fcbb0 0x7f650c10e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c0fd510 0x7f650c108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c108730 con 0x7f650c0fcbb0 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c1088a0 con 0x7f650c0fd510 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c108a10 con 0x7f650c0f7400 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f65114be700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c10e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f65114be700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c10e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:42040/0 (socket says 172.21.15.59:42040) 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6510cbd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650c0fcbb0 0x7f650c10e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f65114be700 1 -- 172.21.15.59:0/812532317 learned_addr learned my addr 172.21.15.59:0/812532317 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6511cbf700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c0fd510 0x7f650c108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6510cbd700 1 -- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 msgr2=0x7f650c10e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6510cbd700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c10e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6510cbd700 1 -- 172.21.15.59:0/812532317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c0fd510 msgr2=0x7f650c108240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6510cbd700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c0fd510 0x7f650c108240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6510cbd700 1 -- 172.21.15.59:0/812532317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500005040 con 0x7f650c0fcbb0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f65114be700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6511cbf700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c0fd510 0x7f650c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6510cbd700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650c0fcbb0 0x7f650c10e500 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f650800b4e0 tx=0x7f6508009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f64fe7fc700 10 client.?.objecter ms_handle_connect 0x7f650c0fcbb0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f64fe7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f64fe7fc700 1 -- 172.21.15.59:0/812532317 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508002320 con 0x7f650c0fcbb0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65040097c0 con 0x7f650c0fcbb0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f64fe7fc700 1 -- 172.21.15.59:0/812532317 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6508002950 con 0x7f650c0fcbb0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f64fe7fc700 1 -- 172.21.15.59:0/812532317 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080129c0 con 0x7f650c0fcbb0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 10 client.31875.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c1090a0 con 0x7f650c0fcbb0 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.376+0000 7f6513722700 1 librados: init done 2024-02-18T00:44:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.377+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c05fa60 con 0x7f650c0fcbb0 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 1 -- 172.21.15.59:0/812532317 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6508002480 con 0x7f650c0fcbb0 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 1 --2- 172.21.15.59:0/812532317 >> v2:172.21.15.59:6800/3721891797 conn(0x7f64f8062c90 0x7f64f8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 1 -- 172.21.15.59:0/812532317 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(697..697 src has 418..697) v4 ==== 8791+0+0 (secure 0 0 0) 0x7f650807d450 con 0x7f650c0fcbb0 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 10 client.31875.objecter ms_dispatch 0x7f650c001550 osd_map(697..697 src has 418..697) v4 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 3 client.31875.objecter handle_osd_map got epochs [697,697] > 0 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 3 client.31875.objecter handle_osd_map decoding full epoch 697 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f65114be700 1 --2- 172.21.15.59:0/812532317 >> v2:172.21.15.59:6800/3721891797 conn(0x7f64f8062c90 0x7f64f8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 20 client.31875.objecter dump_active .. 0 homeless 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f65114be700 1 --2- 172.21.15.59:0/812532317 >> v2:172.21.15.59:6800/3721891797 conn(0x7f64f8062c90 0x7f64f8065150 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 rx=0x7f6500009810 tx=0x7f650000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.378+0000 7f64fe7fc700 10 client.31875.objecter ms_handle_connect 0x7f64f8062c90 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.383+0000 7f64fe7fc700 1 -- 172.21.15.59:0/812532317 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6508050640 con 0x7f650c0fcbb0 2024-02-18T00:44:07.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.577+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f650c1061b0 con 0x7f650c0fcbb0 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.809+0000 7f64fe7fc700 1 -- 172.21.15.59:0/812532317 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v698) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6508050230 con 0x7f650c0fcbb0 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 10 librados: watch_flush enter 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 10 librados: watch_flush exit 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 20 client.31875.objecter shutdown clearing up homeless session... 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 10 client.31875.objecter successfully canceled tick 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 >> v2:172.21.15.59:6800/3721891797 conn(0x7f64f8062c90 msgr2=0x7f64f8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 --2- 172.21.15.59:0/812532317 >> v2:172.21.15.59:6800/3721891797 conn(0x7f64f8062c90 0x7f64f8065150 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 rx=0x7f6500009810 tx=0x7f650000b040).stop 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650c0fcbb0 msgr2=0x7f650c10e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650c0fcbb0 0x7f650c10e500 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f650800b4e0 tx=0x7f6508009060).stop 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 shutdown_connections 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 --2- 172.21.15.59:0/812532317 >> v2:172.21.15.59:6800/3721891797 conn(0x7f64f8062c90 0x7f64f8065150 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f650c0fd510 0x7f650c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650c0f7400 0x7f650c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 --2- 172.21.15.59:0/812532317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f650c0fcbb0 0x7f650c10e500 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 >> 172.21.15.59:0/812532317 conn(0x7f650c0f2820 msgr2=0x7f650c060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.811+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 shutdown_connections 2024-02-18T00:44:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.812+0000 7f6513722700 1 -- 172.21.15.59:0/812532317 wait complete. 2024-02-18T00:44:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:06.812+0000 7f6513722700 1 librados: shutdown 2024-02-18T00:44:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:44:07.938 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:07.987 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:07.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:06.800496+0000 mon.a (mon.0) 3406 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:07.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:06.809367+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:07.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:06.809456+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? 172.21.15.59:0/812532317' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:44:07.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:06.809506+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-18T00:44:07.989 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:06.810655+0000 mon.a (mon.0) 3410 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:07.989 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:06.800496+0000 mon.a (mon.0) 3406 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:07.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:06.809367+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:07.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:06.809456+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? 172.21.15.59:0/812532317' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:44:07.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:06.809506+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-18T00:44:07.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:06.810655+0000 mon.a (mon.0) 3410 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:07.990 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:07.991 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:07.992 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:08.092 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:08.121 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:08.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:06.800496+0000 mon.a (mon.0) 3406 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:06.809367+0000 mon.a (mon.0) 3407 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:06.809456+0000 mon.a (mon.0) 3408 : audit [INF] from='client.? 172.21.15.59:0/812532317' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:44:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:06.809506+0000 mon.a (mon.0) 3409 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-18T00:44:08.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:06.810655+0000 mon.a (mon.0) 3410 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:08.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.515+0000 7fe85768c700 1 -- 172.21.15.59:0/560723592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fdb60 msgr2=0x7fe8500fdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.515+0000 7fe85768c700 1 --2- 172.21.15.59:0/560723592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fdb60 0x7fe8500fdf40 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 rx=0x7fe84c004870 tx=0x7fe84c009cb0).stop 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.515+0000 7fe85768c700 1 -- 172.21.15.59:0/560723592 shutdown_connections 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.515+0000 7fe85768c700 1 --2- 172.21.15.59:0/560723592 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8500f83d0 0x7fe8501049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.515+0000 7fe85768c700 1 --2- 172.21.15.59:0/560723592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fdb60 0x7fe8500fdf40 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.515+0000 7fe85768c700 1 --2- 172.21.15.59:0/560723592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8500fe4c0 0x7fe8500f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.515+0000 7fe85768c700 1 -- 172.21.15.59:0/560723592 >> 172.21.15.59:0/560723592 conn(0x7fe8500f37f0 msgr2=0x7fe8500f5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 -- 172.21.15.59:0/560723592 shutdown_connections 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 -- 172.21.15.59:0/560723592 wait complete. 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 librados: starting msgr at 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 librados: starting objecter 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 Processor -- start 2024-02-18T00:44:08.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 -- start start 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 librados: setting wanted keys 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 librados: calling monclient init 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8500f83d0 0x7fe85010dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8500fdb60 0x7fe85010e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fe4c0 0x7fe850108160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe850108650 con 0x7fe8500fe4c0 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8501087c0 con 0x7fe8500f83d0 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe85768c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe850108980 con 0x7fe8500fdb60 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.516+0000 7fe854c27700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8500fdb60 0x7fe85010e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855c29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fe4c0 0x7fe850108160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe854c27700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8500fdb60 0x7fe85010e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:42066/0 (socket says 172.21.15.59:42066) 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe854c27700 1 -- 172.21.15.59:0/1229750432 learned_addr learned my addr 172.21.15.59:0/1229750432 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:08.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8500f83d0 0x7fe85010dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8500fdb60 msgr2=0x7fe85010e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855c29700 1 --2- 172.21.15.59:0/1229750432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8500fdb60 0x7fe85010e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8500f83d0 msgr2=0x7fe85010dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855c29700 1 --2- 172.21.15.59:0/1229750432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8500f83d0 0x7fe85010dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8480097c0 con 0x7fe8500fe4c0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe854c27700 1 --2- 172.21.15.59:0/1229750432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe8500fdb60 0x7fe85010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe855c29700 1 --2- 172.21.15.59:0/1229750432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fe4c0 0x7fe850108160 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 rx=0x7fe848001cb0 tx=0x7fe848004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe8427fc700 10 client.?.objecter ms_handle_connect 0x7fe8500fe4c0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe8427fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8480079b0 con 0x7fe8500fe4c0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8480090d0 con 0x7fe8500fe4c0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84800fc20 con 0x7fe8500fe4c0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c005040 con 0x7fe8500fe4c0 2024-02-18T00:44:08.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe85768c700 10 client.31881.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe850108f70 con 0x7fe8500fe4c0 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.517+0000 7fe85768c700 1 librados: init done 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.518+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fe850107050 con 0x7fe8500fe4c0 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe84800fd80 con 0x7fe8500fe4c0 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe83c062c90 0x7fe83c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(699..699 src has 418..699) v4 ==== 8786+0+0 (secure 0 0 0) 0x7fe84807dcb0 con 0x7fe8500fe4c0 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 osd_map(699..699 src has 418..699) v4 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 3 client.31881.objecter handle_osd_map got epochs [699,699] > 0 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 3 client.31881.objecter handle_osd_map decoding full epoch 699 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe83c062c90 0x7fe83c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 20 client.31881.objecter dump_active .. 0 homeless 2024-02-18T00:44:08.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.519+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v699) v1 ==== 54+0+16 (secure 0 0 0) 0x7fe848080380 con 0x7fe8500fe4c0 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 10 client.31881.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 1 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe8500fee30 0x7fe8500ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 1 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 0 1 2024-02-18T00:44:08.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850109640 osd=0 4 2024-02-18T00:44:08.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 10 client.31881.objecter _send_command 1 2024-02-18T00:44:08.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.520+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8500ff700 con 0x7fe8500fee30 2024-02-18T00:44:08.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.521+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe83c062c90 0x7fe83c065150 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 rx=0x7fe84c0043d0 tx=0x7fe84c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.521+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe83c062c90 2024-02-18T00:44:08.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.521+0000 7fe854c27700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe8500fee30 0x7fe8500ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.521+0000 7fe854c27700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe8500fee30 0x7fe8500ff210 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.522+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe8500fee30 2024-02-18T00:44:08.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.524+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8500ff700 con 0x7fe8500fee30 2024-02-18T00:44:08.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.524+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:44:08.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.524+0000 7fe8427fc700 10 client.31881.objecter _finish_command 1 = system:0 2024-02-18T00:44:08.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.524+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850109640 osd=0 4 2024-02-18T00:44:08.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.524+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 0 1 2024-02-18T00:44:08.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 10 client.31881.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 2 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 2 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850109640 osd=0 3 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 0 2 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850109640 osd=0 4 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 10 client.31881.objecter _send_command 2 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.539+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe8500ff8a0 con 0x7fe8500fee30 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe8500ff8a0 con 0x7fe8500fee30 2024-02-18T00:44:08.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe8427fc700 10 client.31881.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850109640 osd=0 4 2024-02-18T00:44:08.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 0 2 2024-02-18T00:44:08.547 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:44:08.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 10 client.31881.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 3 2024-02-18T00:44:08.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6805/26279110 conn(0x7fe850028540 0x7fe850118ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 3 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 1 3 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe8500f5ed0 osd=1 4 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 10 client.31881.objecter _send_command 3 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8500f6040 con 0x7fe850028540 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6805/26279110 conn(0x7fe850028540 0x7fe850118ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe850028540 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.541+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6805/26279110 conn(0x7fe850028540 0x7fe850118ed0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.542+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8500f6040 con 0x7fe850028540 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.542+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.542+0000 7fe8427fc700 10 client.31881.objecter _finish_command 3 = system:0 2024-02-18T00:44:08.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.542+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe8500f5ed0 osd=1 4 2024-02-18T00:44:08.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.542+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 1 3 2024-02-18T00:44:08.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 10 client.31881.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 4 2024-02-18T00:44:08.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 4 2024-02-18T00:44:08.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe8500f5ed0 osd=1 3 2024-02-18T00:44:08.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 1 4 2024-02-18T00:44:08.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe8500f5ed0 osd=1 4 2024-02-18T00:44:08.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 10 client.31881.objecter _send_command 4 2024-02-18T00:44:08.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.558+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe85005a040 con 0x7fe850028540 2024-02-18T00:44:08.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe85005a040 con 0x7fe850028540 2024-02-18T00:44:08.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe8427fc700 10 client.31881.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe8500f5ed0 osd=1 4 2024-02-18T00:44:08.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 1 4 2024-02-18T00:44:08.560 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:44:08.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 10 client.31881.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 5 2024-02-18T00:44:08.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe8500594d0 0x7fe85011c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 5 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 2 5 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850059360 osd=2 4 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 10 client.31881.objecter _send_command 5 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.559+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8500280a0 con 0x7fe8500594d0 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe854c27700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe8500594d0 0x7fe85011c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe8500594d0 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe854c27700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe8500594d0 0x7fe85011c550 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8500280a0 con 0x7fe8500594d0 2024-02-18T00:44:08.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:44:08.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe8427fc700 10 client.31881.objecter _finish_command 5 = system:0 2024-02-18T00:44:08.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850059360 osd=2 4 2024-02-18T00:44:08.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.560+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 2 5 2024-02-18T00:44:08.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 10 client.31881.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 6 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 6 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850059360 osd=2 3 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 2 6 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850059360 osd=2 4 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 10 client.31881.objecter _send_command 6 2024-02-18T00:44:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.575+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe85006af10 con 0x7fe8500594d0 2024-02-18T00:44:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe85006af10 con 0x7fe8500594d0 2024-02-18T00:44:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe8427fc700 10 client.31881.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850059360 osd=2 4 2024-02-18T00:44:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 2 6 2024-02-18T00:44:08.578 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:44:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 10 client.31881.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 7 2024-02-18T00:44:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe850027770 0x7fe85006c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 7 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 3 7 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850027600 osd=3 4 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 10 client.31881.objecter _send_command 7 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.576+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe85011a1c0 con 0x7fe850027770 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe855c29700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe850027770 0x7fe85006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe850027770 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe855c29700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe850027770 0x7fe85006c260 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe85011a1c0 con 0x7fe850027770 2024-02-18T00:44:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:44:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe8427fc700 10 client.31881.objecter _finish_command 7 = system:0 2024-02-18T00:44:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850027600 osd=3 4 2024-02-18T00:44:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.577+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 3 7 2024-02-18T00:44:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 10 client.31881.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 8 2024-02-18T00:44:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 8 2024-02-18T00:44:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850027600 osd=3 3 2024-02-18T00:44:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 3 8 2024-02-18T00:44:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850027600 osd=3 4 2024-02-18T00:44:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 10 client.31881.objecter _send_command 8 2024-02-18T00:44:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.592+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe85006cf20 con 0x7fe850027770 2024-02-18T00:44:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe85006cf20 con 0x7fe850027770 2024-02-18T00:44:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe8427fc700 10 client.31881.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850027600 osd=3 4 2024-02-18T00:44:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 3 8 2024-02-18T00:44:08.594 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:44:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 10 client.31881.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 9 2024-02-18T00:44:08.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe850122e60 0x7fe850125250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 9 2024-02-18T00:44:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 4 9 2024-02-18T00:44:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe85006cf20 osd=4 4 2024-02-18T00:44:08.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 10 client.31881.objecter _send_command 9 2024-02-18T00:44:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.593+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe850127f10 con 0x7fe850122e60 2024-02-18T00:44:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.594+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe850122e60 0x7fe850125250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.594+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe850122e60 2024-02-18T00:44:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.594+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe850122e60 0x7fe850125250 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.595+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe850127f10 con 0x7fe850122e60 2024-02-18T00:44:08.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.595+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:44:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.595+0000 7fe8427fc700 10 client.31881.objecter _finish_command 9 = system:0 2024-02-18T00:44:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.595+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe85006cf20 osd=4 4 2024-02-18T00:44:08.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.595+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 4 9 2024-02-18T00:44:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 10 client.31881.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 10 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 10 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe85006cf20 osd=4 3 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 4 10 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe85006cf20 osd=4 4 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 10 client.31881.objecter _send_command 10 2024-02-18T00:44:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.610+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe850127f10 con 0x7fe850122e60 2024-02-18T00:44:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe850127f10 con 0x7fe850122e60 2024-02-18T00:44:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe8427fc700 10 client.31881.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe85006cf20 osd=4 4 2024-02-18T00:44:08.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 4 10 2024-02-18T00:44:08.613 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 10 client.31881.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 11 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe850125c50 0x7fe8501260d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 11 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 5 11 2024-02-18T00:44:08.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127f10 osd=5 4 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 10 client.31881.objecter _send_command 11 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe850127c90 con 0x7fe850125c50 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.612+0000 7fe854c27700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe850125c50 0x7fe8501260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.613+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe850125c50 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.613+0000 7fe854c27700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe850125c50 0x7fe8501260d0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.614+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe850127c90 con 0x7fe850125c50 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.614+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.614+0000 7fe8427fc700 10 client.31881.objecter _finish_command 11 = system:0 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.614+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850127f10 osd=5 4 2024-02-18T00:44:08.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.614+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 5 11 2024-02-18T00:44:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 10 client.31881.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 12 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 12 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850127f10 osd=5 3 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 5 12 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127f10 osd=5 4 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 10 client.31881.objecter _send_command 12 2024-02-18T00:44:08.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.628+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe850127c90 con 0x7fe850125c50 2024-02-18T00:44:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe850127c90 con 0x7fe850125c50 2024-02-18T00:44:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe8427fc700 10 client.31881.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850127f10 osd=5 4 2024-02-18T00:44:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 5 12 2024-02-18T00:44:08.631 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:44:08.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 10 client.31881.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 13 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe85012a740 0x7fe85012aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 13 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 6 13 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127c90 osd=6 4 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 10 client.31881.objecter _send_command 13 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe85012e4b0 con 0x7fe85012a740 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.630+0000 7fe855c29700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe85012a740 0x7fe85012aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.631+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe85012a740 2024-02-18T00:44:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.631+0000 7fe855c29700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe85012a740 0x7fe85012aba0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.631+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe85012e4b0 con 0x7fe85012a740 2024-02-18T00:44:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.631+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:44:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.631+0000 7fe8427fc700 10 client.31881.objecter _finish_command 13 = system:0 2024-02-18T00:44:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.631+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850127c90 osd=6 4 2024-02-18T00:44:08.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.631+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 6 13 2024-02-18T00:44:08.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 10 client.31881.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 14 2024-02-18T00:44:08.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 14 2024-02-18T00:44:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850127c90 osd=6 3 2024-02-18T00:44:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 6 14 2024-02-18T00:44:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127c90 osd=6 4 2024-02-18T00:44:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 10 client.31881.objecter _send_command 14 2024-02-18T00:44:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.646+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe850126680 con 0x7fe85012a740 2024-02-18T00:44:08.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe850126680 con 0x7fe85012a740 2024-02-18T00:44:08.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe8427fc700 10 client.31881.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850127c90 osd=6 4 2024-02-18T00:44:08.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 6 14 2024-02-18T00:44:08.649 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:44:08.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 10 client.31881.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:08.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 15 2024-02-18T00:44:08.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6812/53025171 conn(0x7fe850126d40 0x7fe850127180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:08.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 15 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 7 15 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850126680 osd=7 4 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 10 client.31881.objecter _send_command 15 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8501277c0 con 0x7fe850126d40 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.648+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6812/53025171 conn(0x7fe850126d40 0x7fe850127180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.649+0000 7fe855428700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6812/53025171 conn(0x7fe850126d40 0x7fe850127180 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:08.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.649+0000 7fe8427fc700 10 client.31881.objecter ms_handle_connect 0x7fe850126d40 2024-02-18T00:44:08.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.649+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe8501277c0 con 0x7fe850126d40 2024-02-18T00:44:08.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.649+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:44:08.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.649+0000 7fe8427fc700 10 client.31881.objecter _finish_command 15 = system:0 2024-02-18T00:44:08.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.649+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850126680 osd=7 4 2024-02-18T00:44:08.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.649+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 7 15 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 10 client.31881.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign -1 16 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 20 client.31881.objecter _get_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 15 client.31881.objecter _session_command_op_remove -1 16 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 20 client.31881.objecter get_session s=0x7fe850126680 osd=7 3 2024-02-18T00:44:08.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 15 client.31881.objecter _session_command_op_assign 7 16 2024-02-18T00:44:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850126680 osd=7 4 2024-02-18T00:44:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 10 client.31881.objecter _send_command 16 2024-02-18T00:44:08.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.663+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fe8501277c0 con 0x7fe850126d40 2024-02-18T00:44:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe8427fc700 1 -- 172.21.15.59:0/1229750432 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fe8501277c0 con 0x7fe850126d40 2024-02-18T00:44:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe8427fc700 10 client.31881.objecter ms_dispatch 0x7fe850001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe8427fc700 10 client.31881.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe8427fc700 20 client.31881.objecter put_session s=0x7fe850126680 osd=7 4 2024-02-18T00:44:08.666 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe8427fc700 15 client.31881.objecter _session_command_op_remove 7 16 2024-02-18T00:44:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:44:08.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 librados: watch_flush enter 2024-02-18T00:44:08.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 librados: watch_flush exit 2024-02-18T00:44:08.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.0 2024-02-18T00:44:08.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe8500fee30 msgr2=0x7fe8500ff210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6801/3700238977 conn(0x7fe8500fee30 0x7fe8500ff210 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850109640 osd=0 1 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.1 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6805/26279110 conn(0x7fe850028540 msgr2=0x7fe850118ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6805/26279110 conn(0x7fe850028540 0x7fe850118ed0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe8500f5ed0 osd=1 1 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.2 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe8500594d0 msgr2=0x7fe85011c550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6809/1153193637 conn(0x7fe8500594d0 0x7fe85011c550 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850059360 osd=2 1 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.3 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe850027770 msgr2=0x7fe85006c260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6813/1805700964 conn(0x7fe850027770 0x7fe85006c260 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850027600 osd=3 1 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.4 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe850122e60 msgr2=0x7fe850125250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6800/2044387056 conn(0x7fe850122e60 0x7fe850125250 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe85006cf20 osd=4 1 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.5 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe850125c50 msgr2=0x7fe8501260d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6804/2170432263 conn(0x7fe850125c50 0x7fe8501260d0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127f10 osd=5 1 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.6 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe85012a740 msgr2=0x7fe85012aba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6808/3356009390 conn(0x7fe85012a740 0x7fe85012aba0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850127c90 osd=6 1 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter close_session for osd.7 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6812/53025171 conn(0x7fe850126d40 msgr2=0x7fe850127180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.137:6812/53025171 conn(0x7fe850126d40 0x7fe850127180 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter put_session s=0x7fe850126680 osd=7 1 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 20 client.31881.objecter shutdown clearing up homeless session... 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 10 client.31881.objecter successfully canceled tick 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 reap_dead start 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 reap_dead start 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 reap_dead start 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 reap_dead start 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe855c29700 1 -- 172.21.15.59:0/1229750432 reap_dead start 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe83c062c90 msgr2=0x7fe83c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe83c062c90 0x7fe83c065150 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 rx=0x7fe84c0043d0 tx=0x7fe84c00b040).stop 2024-02-18T00:44:08.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fe4c0 msgr2=0x7fe850108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fe4c0 0x7fe850108160 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 rx=0x7fe848001cb0 tx=0x7fe848004c50).stop 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 shutdown_connections 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8500fe4c0 0x7fe850108160 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 --2- 172.21.15.59:0/1229750432 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe83c062c90 0x7fe83c065150 unknown :-1 s=CLOSED pgs=3202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 >> 172.21.15.59:0/1229750432 conn(0x7fe8500f37f0 msgr2=0x7fe8500f5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 shutdown_connections 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 -- 172.21.15.59:0/1229750432 wait complete. 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:08.666+0000 7fe85768c700 1 librados: shutdown 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.672 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:44:08.673 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:44:09.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:07.808836+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:07.808893+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-18T00:44:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:07.809719+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-122", "mode": "writeback"}]: dispatch 2024-02-18T00:44:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:07.997561+0000 mgr.y (mgr.24454) 612 : cluster [DBG] pgmap v1071: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:44:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:08.519481+0000 mon.a (mon.0) 3414 : audit [DBG] from='client.? 172.21.15.59:0/1229750432' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:09.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:08.807030+0000 mon.a (mon.0) 3415 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:07.808836+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:07.808893+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-18T00:44:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:07.809719+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-122", "mode": "writeback"}]: dispatch 2024-02-18T00:44:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:07.997561+0000 mgr.y (mgr.24454) 612 : cluster [DBG] pgmap v1071: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:44:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:08.519481+0000 mon.a (mon.0) 3414 : audit [DBG] from='client.? 172.21.15.59:0/1229750432' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:09.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:08.807030+0000 mon.a (mon.0) 3415 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:07.808836+0000 mon.a (mon.0) 3411 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:07.808893+0000 mon.a (mon.0) 3412 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-18T00:44:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:07.809719+0000 mon.a (mon.0) 3413 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-122", "mode": "writeback"}]: dispatch 2024-02-18T00:44:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:07.997561+0000 mgr.y (mgr.24454) 612 : cluster [DBG] pgmap v1071: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T00:44:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:08.519481+0000 mon.a (mon.0) 3414 : audit [DBG] from='client.? 172.21.15.59:0/1229750432' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:08.807030+0000 mon.a (mon.0) 3415 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:08.815020+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-122", "mode": "writeback"}]': finished 2024-02-18T00:44:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:08.815098+0000 mon.a (mon.0) 3417 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-18T00:44:10.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:09.812976+0000 mon.a (mon.0) 3418 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-18T00:44:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:08.815020+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-122", "mode": "writeback"}]': finished 2024-02-18T00:44:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:08.815098+0000 mon.a (mon.0) 3417 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-18T00:44:10.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:09.812976+0000 mon.a (mon.0) 3418 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-18T00:44:10.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:08.815020+0000 mon.a (mon.0) 3416 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-122", "mode": "writeback"}]': finished 2024-02-18T00:44:10.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:08.815098+0000 mon.a (mon.0) 3417 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-18T00:44:10.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:09.812976+0000 mon.a (mon.0) 3418 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-18T00:44:10.571 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:10.574 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:10.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:10.675 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:10.702 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:10.721 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:10.723 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:10.724 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:10.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:10.857 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:11.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:09.787747+0000 osd.1 (osd.1) 335 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:09.788007+0000 osd.1 (osd.1) 336 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:09.908359+0000 mon.a (mon.0) 3419 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:09.997926+0000 mgr.y (mgr.24454) 613 : cluster [DBG] pgmap v1074: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:10.284170+0000 mon.a (mon.0) 3420 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:10.646935+0000 mon.a (mon.0) 3421 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:10.647629+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:10.651932+0000 mon.a (mon.0) 3423 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:10.814257+0000 mon.a (mon.0) 3424 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-18T00:44:11.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:10.850672+0000 mon.a (mon.0) 3425 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:11.107 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:11.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:09.787747+0000 osd.1 (osd.1) 335 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:11.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:09.788007+0000 osd.1 (osd.1) 336 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:11.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:09.908359+0000 mon.a (mon.0) 3419 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:11.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:09.997926+0000 mgr.y (mgr.24454) 613 : cluster [DBG] pgmap v1074: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:11.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:10.284170+0000 mon.a (mon.0) 3420 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:44:11.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:10.646935+0000 mon.a (mon.0) 3421 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:44:11.110 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:10.647629+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:10.651932+0000 mon.a (mon.0) 3423 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:10.814257+0000 mon.a (mon.0) 3424 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:10.850672+0000 mon.a (mon.0) 3425 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:09.787747+0000 osd.1 (osd.1) 335 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:09.788007+0000 osd.1 (osd.1) 336 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:09.908359+0000 mon.a (mon.0) 3419 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:09.997926+0000 mgr.y (mgr.24454) 613 : cluster [DBG] pgmap v1074: 214 pgs: 10 creating+activating, 16 creating+peering, 188 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:10.284170+0000 mon.a (mon.0) 3420 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:10.646935+0000 mon.a (mon.0) 3421 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:44:11.111 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:10.647629+0000 mon.a (mon.0) 3422 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:44:11.112 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:10.651932+0000 mon.a (mon.0) 3423 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:11.112 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:10.814257+0000 mon.a (mon.0) 3424 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-18T00:44:11.112 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:10.850672+0000 mon.a (mon.0) 3425 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:11.112 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:11.112 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:11.213 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:11.243 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:11.262 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:11.265 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:11.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:11.366 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:11.396 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:11.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:44:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 20, 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 600, 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [ 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42916.0:1 114.5s0 114:a1464ac0:test-rados-api-smithi059-82981-16::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e176)", 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 596.37662709999995, 2024-02-18T00:44:11.772 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996, 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42916", 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2278224535", 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392198+0000", 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999971 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392195+0000", 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1378e-05 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.774 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392206+0000", 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3589e-05 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392220+0000", 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1283e-05 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392231+0000", 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.0343999999999997e-05 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.775 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392292+0000", 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.3749999999999999e-06 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.392295+0000", 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.58736239099999998 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979657+0000", 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000103571 2024-02-18T00:44:11.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.979761+0000", 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000375254 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.980136+0000", 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0061387799999999999 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.986275+0000", 2024-02-18T00:44:11.777 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032303779999999999 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989505+0000", 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7123000000000003e-05 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:15.989552+0000", 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.59735459999999996 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.42910.0:2 115.13 115:ce638d0e:test-rados-api-smithi059-82949-26::foo:head [writefull 0~8388608] snapc 0=[] ondisk+write+known_if_redirected e178)", 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 594.78887502800001, 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003, 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.42910", 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/955395839", 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.779 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979950+0000", 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.979948+0000", 2024-02-18T00:44:11.780 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0051422639999999997 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985090+0000", 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1109e-05 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985101+0000", 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2439999999999993e-06 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.781 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985110+0000", 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00032139199999999999 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985431+0000", 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000115649 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985547+0000", 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014544800000000001 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.782 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 1,3", 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:16.985692+0000", 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016096787000000001 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.001789+0000", 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.016631943 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018421+0000", 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016582200000000001 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018587+0000", 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4758999999999994e-05 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018651+0000", 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8247000000000002e-05 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:17.018690+0000", 2024-02-18T00:44:11.784 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.038739764000000003 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:588 62.9 e205/123 62:9441d9b2:test-rados-api-smithi059-82957-14::foo82:4 v 205'35, mlcod=193'31)", 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 568.097857363, 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999, 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.785 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670968+0000", 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670966+0000", 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2614e-05 2024-02-18T00:44:11.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.670979+0000", 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.1616000000000001e-05 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671000+0000", 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9030000000000004e-06 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671008+0000", 2024-02-18T00:44:11.787 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.3940000000000001e-05 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671072+0000", 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0826999999999999e-05 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671093+0000", 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034117472000000003 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.788 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705210+0000", 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5481000000000005e-05 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.705276+0000", 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.034308393999999999 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_repop(osd.4.0:589 62.9 e205/123 62:93987215:test-rados-api-smithi059-82957-14::foo76:4 v 205'37, mlcod=193'31)", 2024-02-18T00:44:11.789 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 568.09737156400001, 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001, 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:44:11.790 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671453+0000", 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.109e-06 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671460+0000", 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.027e-05 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.791 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671470+0000", 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.993e-06 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671476+0000", 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2646999999999999e-05 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671529+0000", 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.0522000000000001e-05 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.671549+0000", 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036671306000000001 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708221+0000", 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.8853000000000002e-05 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:34:43.708259+0000", 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.036806009000000001 2024-02-18T00:44:11.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30969.0:1 179.5s0 179:b3610dda:::oid_RoundTripPP2_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 488.13117558599998, 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001, 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30969", 2024-02-18T00:44:11.794 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3697991996", 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637649+0000", 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.795 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637648+0000", 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2819999999999999e-06 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637655+0000", 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.9309999999999997e-06 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637664+0000", 2024-02-18T00:44:11.796 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.153e-06 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637671+0000", 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2398000000000003e-05 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637713+0000", 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2790000000000001e-06 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.637716+0000", 2024-02-18T00:44:11.797 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0156700160000001 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653386+0000", 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000131438 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653517+0000", 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043157100000000001 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:44:11.798 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.653949+0000", 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0031255889999999998 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657074+0000", 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.2801999999999999e-05 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657147+0000", 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000247073 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.799 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.657394+0000", 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0197448570000001 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.30966.0:1 178.6s0 178:602f83fe:::foo:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e285)", 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 488.13045715700002, 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001, 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.30966", 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/3784803143", 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.801 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638368+0000", 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638367+0000", 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.622e-06 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638372+0000", 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.7880000000000001e-06 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.802 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638381+0000", 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.7909999999999999e-06 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638388+0000", 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.171e-05 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638420+0000", 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6700000000000001e-06 2024-02-18T00:44:11.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:03.638422+0000", 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0205109640000001 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.658932+0000", 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00014527400000000001 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659078+0000", 2024-02-18T00:44:11.804 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00043918800000000002 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.659517+0000", 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00089794300000000001 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660415+0000", 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9174e-05 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.805 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660474+0000", 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00024629900000000001 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:36:04.660720+0000", 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0223525920000001 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.40973.0:1 199.2s0 199:482bd08b:::oid_SimpleStatPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e354)", 2024-02-18T00:44:11.806 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 418.99365648600002, 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0539473699999999, 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.40973", 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1800225797", 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.807 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775168+0000", 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775166+0000", 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0146000000000001e-05 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.808 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775176+0000", 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2602000000000001e-05 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775189+0000", 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4409999999999999e-06 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775198+0000", 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9956e-05 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775258+0000", 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.7239999999999998e-06 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:12.775262+0000", 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.050233969 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.825496+0000", 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000114114 2024-02-18T00:44:11.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.825610+0000", 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00045617599999999998 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.826066+0000", 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00049755600000000002 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.826564+0000", 2024-02-18T00:44:11.811 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024990279999999999 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.829063+0000", 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3276000000000003e-05 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:13.829116+0000", 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0539473699999999 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.41009.0:1 203.4s0 203:311cbe94:::oid_ExecuteClassPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e375)", 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 398.09757338499998, 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.014258211, 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.41009", 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/1940889306", 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.813 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671252+0000", 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.814 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671249+0000", 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.2910000000000007e-06 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671257+0000", 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0814e-05 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671268+0000", 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.5390000000000002e-06 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.815 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671276+0000", 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.9815999999999998e-05 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671336+0000", 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 3.4039999999999999e-06 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:33.671339+0000", 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.011917613 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683257+0000", 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00016619700000000001 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683423+0000", 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00048756299999999999 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.683910+0000", 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00099101599999999995 2024-02-18T00:44:11.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.684901+0000", 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00052922899999999998 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.685431+0000", 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.9234999999999996e-05 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:37:34.685510+0000", 2024-02-18T00:44:11.818 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.014258211 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.25220.0:5782 209.10 209:08000000:%01:::head [pgnls start_epoch 404] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e404)", 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 366.03382641299999, 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.965927443, 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.25220", 2024-02-18T00:44:11.819 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/699604112", 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 5782 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734999+0000", 2024-02-18T00:44:11.820 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.734998+0000", 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7509999999999997e-06 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735003+0000", 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0363000000000001e-05 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.821 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735014+0000", 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.8989999999999998e-06 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735020+0000", 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.2460000000000003e-05 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735072+0000", 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.8979999999999996e-06 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for peered", 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:05.735080+0000", 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.96567722099999997 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700757+0000", 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4641000000000002e-05 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700802+0000", 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012411200000000001 2024-02-18T00:44:11.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:38:06.700926+0000", 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.965927443 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "MOSDECSubOpWrite(234.5s2 645/605 ECSubWrite(tid=750, reqid=client.31692.0:474, at_version=645'6, trim_to=0'0, roll_forward_to=640'5))", 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:10.787402+0000", 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 60.981423405999998, 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074311389999999998, 2024-02-18T00:44:11.824 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787402+0000", 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787402+0000", 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.825 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787401+0000", 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.3751999999999999e-05 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787425+0000", 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.7970000000000001e-06 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787433+0000", 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.1310000000000006e-06 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787441+0000", 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00010906 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787550+0000", 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0730000000000001e-05 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.787561+0000", 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0072582460000000003 2024-02-18T00:44:11.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.794819+0000", 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3699999999999999e-05 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:10.794833+0000", 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074311389999999998 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31692.0:475 239.1d 239:bd34d09e:test-rados-api-smithi059-82957-111::foo:head [stat,delete] snapc 0=[] ondisk+write+known_if_redirected e648)", 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:13.777998+0000", 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 57.990826454, 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0055900910000000002, 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31692", 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 475 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.777998+0000", 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.777998+0000", 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.777997+0000", 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.5710000000000004e-06 2024-02-18T00:44:11.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.778006+0000", 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1289e-05 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.778017+0000", 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.8260000000000004e-06 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.831 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.778024+0000", 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4125000000000002e-05 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.778068+0000", 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00022173200000000001 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for blocked object", 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.778290+0000", 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001821317 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.780111+0000", 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.9710000000000006e-05 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.780211+0000", 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00017087600000000001 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for subops from 5,6", 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.780382+0000", 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00099310200000000005 2024-02-18T00:44:11.833 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op_commit", 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.781375+0000", 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001107685 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.782483+0000", 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00095371799999999999 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.783437+0000", 2024-02-18T00:44:11.834 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.0308000000000003e-05 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "commit_sent", 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.783527+0000", 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.1743999999999997e-05 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.783589+0000", 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0055900910000000002 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31692.0:510 239.1d 239:bd34d09e:test-rados-api-smithi059-82957-111::foo:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected e648)", 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:13.810542+0000", 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 57.958282818999997, 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023795012000000001, 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31692", 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 510 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.836 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810542+0000", 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810542+0000", 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810540+0000", 2024-02-18T00:44:11.837 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.0907000000000001e-05 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810551+0000", 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.6441999999999999e-05 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810567+0000", 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.2979999999999997e-06 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.838 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810577+0000", 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.2586e-05 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810639+0000", 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013122399999999999 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.810770+0000", 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023566823000000001 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.839 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:13.834337+0000", 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.023795012000000001 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31692.0:581 239.10 239:08000000::::head [pgnls start_epoch 650] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e650)", 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:15.795313+0000", 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 55.97351175, 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023706280000000001, 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.840 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31692", 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 581 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795313+0000", 2024-02-18T00:44:11.841 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795313+0000", 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.842 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795311+0000", 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3451999999999999e-05 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795324+0000", 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2379000000000002e-05 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795366+0000", 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.7069999999999998e-06 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.843 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795375+0000", 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9813000000000004e-05 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795445+0000", 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.2485e-05 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.795457+0000", 2024-02-18T00:44:11.844 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0020610469999999999 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.797519+0000", 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.2493000000000001e-05 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.797561+0000", 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00012279599999999999 2024-02-18T00:44:11.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:15.797684+0000", 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023706280000000001 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "MOSDECSubOpWrite(234.5s2 653/605 ECSubWrite(tid=759, reqid=client.31692.0:646, at_version=653'8, trim_to=0'0, roll_forward_to=648'7))", 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:18.793542+0000", 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 52.975283161, 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024831689999999999, 2024-02-18T00:44:11.846 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "reached pg", 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793542+0000", 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793542+0000", 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999995 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.847 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793541+0000", 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5262999999999999e-05 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793567+0000", 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.4610000000000008e-06 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793575+0000", 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.6790000000000005e-06 2024-02-18T00:44:11.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793583+0000", 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3261999999999999e-05 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793636+0000", 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.8019000000000001e-05 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_started", 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.793664+0000", 2024-02-18T00:44:11.849 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0023353829999999999 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "sub_op_committed", 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.795999+0000", 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5562e-05 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:18.796025+0000", 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0024831689999999999 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31692.0:808 241.e 241:76cde755:test-rados-api-smithi059-82957-115::bar:head [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e665)", 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:30.808299+0000", 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.960525848000003, 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01070305, 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31692", 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 808 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.851 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808299+0000", 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808299+0000", 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.852 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808298+0000", 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9659999999999999e-06 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808305+0000", 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1511999999999999e-05 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808316+0000", 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7209999999999999e-06 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808322+0000", 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1829999999999998e-05 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808364+0000", 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.6940000000000003e-06 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.808368+0000", 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0022986259999999998 2024-02-18T00:44:11.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.810667+0000", 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00013062800000000001 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for blocked object", 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.810798+0000", 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0045770689999999996 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.815375+0000", 2024-02-18T00:44:11.855 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5575999999999997e-05 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.815440+0000", 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0025742320000000001 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.818014+0000", 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.1746000000000002e-05 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.856 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.818066+0000", 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00093592600000000001 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819002+0000", 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.01070305 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31692.0:809 241.e 241:76cde755:test-rados-api-smithi059-82957-115::bar:6 [read 0~1] snapc 0=[] ondisk+read+known_if_redirected e665)", 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:30.819183+0000", 2024-02-18T00:44:11.857 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 40.949641620000001, 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032754860000000002, 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31692", 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 809 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819183+0000", 2024-02-18T00:44:11.858 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819183+0000", 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.999999 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819182+0000", 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.519e-06 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.859 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819188+0000", 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3419999999999997e-06 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819197+0000", 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.7199999999999997e-06 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819202+0000", 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.1783999999999998e-05 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.860 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819244+0000", 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00020095000000000001 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for blocked object", 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.819445+0000", 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0014722100000000001 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.820917+0000", 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6045999999999996e-05 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "waiting for rw locks", 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.821003+0000", 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00048183200000000002 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.821485+0000", 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.5720999999999997e-05 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.821540+0000", 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00091836000000000001 2024-02-18T00:44:11.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:30.822459+0000", 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0032754860000000002 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31692.0:829 241.e 241:70000000:%01:::head [pgnls start_epoch 666] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e666)", 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:31.810428+0000", 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 39.958396788000002, 2024-02-18T00:44:11.863 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074600550000000002, 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31692", 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 829 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810428+0000", 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.864 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810428+0000", 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999986 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810427+0000", 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.152e-05 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810438+0000", 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.9386e-05 2024-02-18T00:44:11.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810457+0000", 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.6489999999999997e-06 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810466+0000", 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.7992000000000003e-05 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810524+0000", 2024-02-18T00:44:11.866 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4719999999999998e-06 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.810532+0000", 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0071922749999999997 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.817724+0000", 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4.4490000000000003e-05 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.867 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.817768+0000", 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000119881 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:31.817888+0000", 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.0074600550000000002 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "MOSDScrubReserve(4.7 REQUEST e680)", 2024-02-18T00:44:11.868 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:45.983360+0000", 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 25.785464695000002, 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002320613, 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983360+0000", 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983360+0000", 2024-02-18T00:44:11.869 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967296 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983360+0000", 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.9289999999999998e-06 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983363+0000", 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5251e-05 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.870 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983378+0000", 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 7.4889999999999998e-06 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983386+0000", 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.3075999999999998e-05 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983439+0000", 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.3519999999999996e-06 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.871 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.983448+0000", 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002190254 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.985638+0000", 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.8114e-05 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.985657+0000", 2024-02-18T00:44:11.872 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.4287999999999999e-05 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:45.985681+0000", 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.002320613 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.14403.0:2096 5.3 5:c609908c:::notify.5:head [watch ping cookie 94750322751104] snapc 0=[] ondisk+write+known_if_redirected e677)", 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:43:48.441297+0000", 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 23.3275279, 2024-02-18T00:44:11.873 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061402199999999998, 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.14403", 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/2016795673", 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 2096 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441297+0000", 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441297+0000", 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999976 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441295+0000", 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.3311e-05 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441308+0000", 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.5743999999999999e-05 2024-02-18T00:44:11.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441334+0000", 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.1694e-05 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441345+0000", 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00036377700000000002 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.876 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441709+0000", 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.000116937 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441826+0000", 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 8.5028e-05 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:43:48.441911+0000", 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.00061402199999999998 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.877 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "description": "osd_op(client.31692.0:1532 245.18 245:18000000:test-rados-api-smithi059-82957-123:::head [pgnls start_epoch 702] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e702)", 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "initiated_at": "2024-02-18T00:44:10.823588+0000", 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.94523722600000004, 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001786342, 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "type_data": { 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "flag_point": "started", 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "client_info": { 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "client": "client.31692", 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: "tid": 1532 2024-02-18T00:44:11.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "events": [ 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "initiated", 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823588+0000", 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "throttled", 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823588+0000", 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 4294967295.9999981 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "header_read", 2024-02-18T00:44:11.879 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823586+0000", 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 9.4769999999999992e-06 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "all_read", 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823595+0000", 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 1.5605000000000001e-05 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "dispatched", 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823611+0000", 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.4180000000000002e-06 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.880 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "queued_for_pg", 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823617+0000", 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 5.6261999999999998e-05 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823674+0000", 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.9650000000000002e-06 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "op must wait for map", 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.823681+0000", 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001604734 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "reached_pg", 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.825285+0000", 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 2.2617e-05 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "started", 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.825308+0000", 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 6.5994000000000005e-05 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "event": "done", 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "time": "2024-02-18T00:44:10.825374+0000", 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: "duration": 0.001786342 2024-02-18T00:44:11.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.883 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:44:11.883 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:44:11.883 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:44:12.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:44:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:44:12.145 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:12.147 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:12.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:12.249 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:12.281 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:12.302 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:12.305 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:12.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:12.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:12.436 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:12] "GET /metrics HTTP/1.1" 200 191729 "" "Prometheus/2.33.4" 2024-02-18T00:44:12.439 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:12.561 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:44:12.989 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:12.992 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:12.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:13.093 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:13.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:11.543499+0000 mgr.y (mgr.24454) 614 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:13.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:11.818049+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:13.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:11.818114+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-18T00:44:13.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:11.824499+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:13.120 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:11.998434+0000 mgr.y (mgr.24454) 615 : cluster [DBG] pgmap v1077: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:44:13.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:11.543499+0000 mgr.y (mgr.24454) 614 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:13.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:11.818049+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:13.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:11.818114+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-18T00:44:13.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:11.824499+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:13.121 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:11.998434+0000 mgr.y (mgr.24454) 615 : cluster [DBG] pgmap v1077: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:44:13.122 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:13.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:11.543499+0000 mgr.y (mgr.24454) 614 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:13.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:11.818049+0000 mon.a (mon.0) 3426 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:13.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:11.818114+0000 mon.a (mon.0) 3427 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-18T00:44:13.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:11.824499+0000 mon.a (mon.0) 3428 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:13.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:11.998434+0000 mgr.y (mgr.24454) 615 : cluster [DBG] pgmap v1077: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:44:13.270 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:13.273 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:13.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:13.375 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:13.407 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:13.428 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:13.431 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:13.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:13.532 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:13.562 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:13.582 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:13.584 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:13.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 -- 172.21.15.59:0/2244651832 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fd510 msgr2=0x7f36b80f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 --2- 172.21.15.59:0/2244651832 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fd510 0x7f36b80f6d40 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 rx=0x7f36ac004870 tx=0x7f36ac009cb0).stop 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 -- 172.21.15.59:0/2244651832 shutdown_connections 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 --2- 172.21.15.59:0/2244651832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b80fcbb0 0x7f36b80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 --2- 172.21.15.59:0/2244651832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 0x7f36b8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 --2- 172.21.15.59:0/2244651832 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fd510 0x7f36b80f6d40 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 -- 172.21.15.59:0/2244651832 >> 172.21.15.59:0/2244651832 conn(0x7f36b80f2820 msgr2=0x7f36b80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 -- 172.21.15.59:0/2244651832 shutdown_connections 2024-02-18T00:44:13.685 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.682+0000 7f36c0667700 1 -- 172.21.15.59:0/2244651832 wait complete. 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 librados: starting msgr at 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 librados: starting objecter 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 Processor -- start 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 -- start start 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 librados: setting wanted keys 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 librados: calling monclient init 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 0x7f36b80f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fcbb0 0x7f36b80f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b80fd510 0x7f36b80f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81166b0 con 0x7f36b80f7400 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81167f0 con 0x7f36b80fd510 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36c0667700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81169b0 con 0x7f36b80fcbb0 2024-02-18T00:44:13.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36be403700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 0x7f36b80f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36bdc02700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fcbb0 0x7f36b80f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36be403700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 0x7f36b80f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52208/0 (socket says 172.21.15.59:52208) 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36bdc02700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fcbb0 0x7f36b80f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43336/0 (socket says 172.21.15.59:43336) 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.683+0000 7f36be403700 1 -- 172.21.15.59:0/955188577 learned_addr learned my addr 172.21.15.59:0/955188577 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36bec04700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b80fd510 0x7f36b80f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36bdc02700 1 -- 172.21.15.59:0/955188577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b80fd510 msgr2=0x7f36b80f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36bdc02700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b80fd510 0x7f36b80f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36bdc02700 1 -- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 msgr2=0x7f36b80f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36bdc02700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 0x7f36b80f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36bdc02700 1 -- 172.21.15.59:0/955188577 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac005040 con 0x7f36b80fcbb0 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36be403700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 0x7f36b80f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36ab7fe700 10 client.?.objecter ms_handle_connect 0x7f36b80fcbb0 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36bdc02700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fcbb0 0x7f36b80f8db0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 rx=0x7f36ac004870 tx=0x7f36ac009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:13.687 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36ab7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:13.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36ab7fe700 1 -- 172.21.15.59:0/955188577 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00e030 con 0x7f36b80fcbb0 2024-02-18T00:44:13.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36ab7fe700 1 -- 172.21.15.59:0/955188577 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ac009710 con 0x7f36b80fcbb0 2024-02-18T00:44:13.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b8116c30 con 0x7f36b80fcbb0 2024-02-18T00:44:13.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36ab7fe700 1 -- 172.21.15.59:0/955188577 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac018070 con 0x7f36b80fcbb0 2024-02-18T00:44:13.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36c0667700 10 client.41693.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:13.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b8117030 con 0x7f36b80fcbb0 2024-02-18T00:44:13.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.684+0000 7f36c0667700 1 librados: init done 2024-02-18T00:44:13.689 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:13.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.685+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b805fa60 con 0x7f36b80fcbb0 2024-02-18T00:44:13.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.685+0000 7f36ab7fe700 1 -- 172.21.15.59:0/955188577 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f36ac0090d0 con 0x7f36b80fcbb0 2024-02-18T00:44:13.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36ab7fe700 1 --2- 172.21.15.59:0/955188577 >> v2:172.21.15.59:6800/3721891797 conn(0x7f36a4062c90 0x7f36a4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:13.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36ab7fe700 1 -- 172.21.15.59:0/955188577 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(704..704 src has 418..704) v4 ==== 8786+0+0 (secure 0 0 0) 0x7f36ac07e500 con 0x7f36b80fcbb0 2024-02-18T00:44:13.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36ab7fe700 10 client.41693.objecter ms_dispatch 0x7f36b8001550 osd_map(704..704 src has 418..704) v4 2024-02-18T00:44:13.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36ab7fe700 3 client.41693.objecter handle_osd_map got epochs [704,704] > 0 2024-02-18T00:44:13.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36ab7fe700 3 client.41693.objecter handle_osd_map decoding full epoch 704 2024-02-18T00:44:13.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36be403700 1 --2- 172.21.15.59:0/955188577 >> v2:172.21.15.59:6800/3721891797 conn(0x7f36a4062c90 0x7f36a4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:13.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36ab7fe700 20 client.41693.objecter dump_active .. 0 homeless 2024-02-18T00:44:13.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36ab7fe700 10 client.41693.objecter ms_handle_connect 0x7f36a4062c90 2024-02-18T00:44:13.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.686+0000 7f36be403700 1 --2- 172.21.15.59:0/955188577 >> v2:172.21.15.59:6800/3721891797 conn(0x7f36a4062c90 0x7f36a4065150 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 rx=0x7f36b4004fd0 tx=0x7f36b400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:13.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.690+0000 7f36ab7fe700 1 -- 172.21.15.59:0/955188577 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f36ac051770 con 0x7f36b80fcbb0 2024-02-18T00:44:13.720 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:13.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:13.908+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f36b8106170 con 0x7f36b80fcbb0 2024-02-18T00:44:14.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:12.821071+0000 mon.a (mon.0) 3429 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:12.821132+0000 mon.a (mon.0) 3430 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-18T00:44:14.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:13.795479+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-18T00:44:14.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:12.821071+0000 mon.a (mon.0) 3429 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:14.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:12.821132+0000 mon.a (mon.0) 3430 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-18T00:44:14.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:13.795479+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-18T00:44:14.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:44:14.177 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:14.181 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:14.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:14.184 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:14.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_ops_in_flight 2024-02-18T00:44:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:12.821071+0000 mon.a (mon.0) 3429 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:12.821132+0000 mon.a (mon.0) 3430 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-18T00:44:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:13.795479+0000 mon.a (mon.0) 3431 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-18T00:44:14.283 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:14.313 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:14.333 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:14.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:14.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:14.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:14.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:14.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:14.598 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:14.629 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:14.649 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:14.652 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:14.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:14.754 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:14.784 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:14.803 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:14.806 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:14.806 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:14.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.831+0000 7f36ab7fe700 1 -- 172.21.15.59:0/955188577 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v706) v1 ==== 113+0+0 (secure 0 0 0) 0x7f36ac051360 con 0x7f36b80fcbb0 2024-02-18T00:44:14.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 10 librados: watch_flush enter 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 10 librados: watch_flush exit 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 20 client.41693.objecter shutdown clearing up homeless session... 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 10 client.41693.objecter successfully canceled tick 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 >> v2:172.21.15.59:6800/3721891797 conn(0x7f36a4062c90 msgr2=0x7f36a4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 --2- 172.21.15.59:0/955188577 >> v2:172.21.15.59:6800/3721891797 conn(0x7f36a4062c90 0x7f36a4065150 secure :-1 s=READY pgs=3203 cs=0 l=1 rev1=1 rx=0x7f36b4004fd0 tx=0x7f36b400b040).stop 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fcbb0 msgr2=0x7f36b80f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fcbb0 0x7f36b80f8db0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 rx=0x7f36ac004870 tx=0x7f36ac009cb0).stop 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 shutdown_connections 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 --2- 172.21.15.59:0/955188577 >> v2:172.21.15.59:6800/3721891797 conn(0x7f36a4062c90 0x7f36a4065150 unknown :-1 s=CLOSED pgs=3203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b80fd510 0x7f36b80f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36b80f7400 0x7f36b80f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:14.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 --2- 172.21.15.59:0/955188577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36b80fcbb0 0x7f36b80f8db0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:14.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 >> 172.21.15.59:0/955188577 conn(0x7f36b80f2820 msgr2=0x7f36b8060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:14.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 shutdown_connections 2024-02-18T00:44:14.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 -- 172.21.15.59:0/955188577 wait complete. 2024-02-18T00:44:14.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:14.833+0000 7f36c0667700 1 librados: shutdown 2024-02-18T00:44:14.835 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.3 to 1 (10000) 2024-02-18T00:44:14.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:14.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:13.838148+0000 mon.a (mon.0) 3432 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:14.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:13.908593+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.59:0/955188577' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:44:14.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:13.908924+0000 mon.a (mon.0) 3433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:44:14.933 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:13.998917+0000 mgr.y (mgr.24454) 616 : cluster [DBG] pgmap v1080: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:44:14.935 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:14.954 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:14.956 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:14.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:15.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:13.838148+0000 mon.a (mon.0) 3432 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:13.908593+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.59:0/955188577' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:44:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:13.908924+0000 mon.a (mon.0) 3433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:44:15.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:13.998917+0000 mgr.y (mgr.24454) 616 : cluster [DBG] pgmap v1080: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:44:15.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:13.838148+0000 mon.a (mon.0) 3432 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:15.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:13.908593+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.59:0/955188577' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:44:15.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:13.908924+0000 mon.a (mon.0) 3433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:44:15.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:13.998917+0000 mgr.y (mgr.24454) 616 : cluster [DBG] pgmap v1080: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T00:44:15.057 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:15.089 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:15.339 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:15.343 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:15.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:15.344 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:15.347 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:15.445 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:15.473 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:15.492 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:15.494 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:15.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:15.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:15.624 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:15.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:14.827390+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:15.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:14.827448+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:44:15.887 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:14.827498+0000 mon.a (mon.0) 3436 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-18T00:44:15.887 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:14.828127+0000 mon.a (mon.0) 3437 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:15.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:14.827390+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:15.887 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:14.827448+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:44:15.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:14.827498+0000 mon.a (mon.0) 3436 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-18T00:44:15.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:14.828127+0000 mon.a (mon.0) 3437 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:15.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 4 1 2024-02-18T00:44:16.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:14.827390+0000 mon.a (mon.0) 3434 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:14.827448+0000 mon.a (mon.0) 3435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:44:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:14.827498+0000 mon.a (mon.0) 3436 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-18T00:44:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:14.828127+0000 mon.a (mon.0) 3437 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122"}]: dispatch 2024-02-18T00:44:16.677 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:16.680 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:16.681 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:16.782 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:16.812 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:16.833 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:16.835 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:16.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:16.937 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:15.825254+0000 mon.a (mon.0) 3438 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:15.831915+0000 mon.a (mon.0) 3439 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:15.831983+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:15.999470+0000 mgr.y (mgr.24454) 617 : cluster [DBG] pgmap v1083: 214 pgs: 1 active+clean+snaptrim, 213 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:15.825254+0000 mon.a (mon.0) 3438 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:15.831915+0000 mon.a (mon.0) 3439 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:15.831983+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-18T00:44:16.950 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:15.999470+0000 mgr.y (mgr.24454) 617 : cluster [DBG] pgmap v1083: 214 pgs: 1 active+clean+snaptrim, 213 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T00:44:16.966 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:16.987 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:16.989 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:16.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:17.090 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:17.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:15.825254+0000 mon.a (mon.0) 3438 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:17.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:15.831915+0000 mon.a (mon.0) 3439 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-122"}]': finished 2024-02-18T00:44:17.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:15.831983+0000 mon.a (mon.0) 3440 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-18T00:44:17.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:15.999470+0000 mgr.y (mgr.24454) 617 : cluster [DBG] pgmap v1083: 214 pgs: 1 active+clean+snaptrim, 213 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T00:44:17.118 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:17.137 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:17.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:17.140 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:17.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:17.271 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:17.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 -- 172.21.15.59:0/1385108663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 msgr2=0x7f0c84104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 --2- 172.21.15.59:0/1385108663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 0x7f0c84104680 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 rx=0x7f0c7c009be0 tx=0x7f0c7c004d10).stop 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 -- 172.21.15.59:0/1385108663 shutdown_connections 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 --2- 172.21.15.59:0/1385108663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 0x7f0c84104680 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 --2- 172.21.15.59:0/1385108663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c840f8d30 0x7f0c840f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 --2- 172.21.15.59:0/1385108663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c840f9600 0x7f0c84101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 -- 172.21.15.59:0/1385108663 >> 172.21.15.59:0/1385108663 conn(0x7f0c840f2820 msgr2=0x7f0c840f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 -- 172.21.15.59:0/1385108663 shutdown_connections 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.525+0000 7f0c8a514700 1 -- 172.21.15.59:0/1385108663 wait complete. 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 librados: starting msgr at 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 librados: starting objecter 2024-02-18T00:44:17.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 Processor -- start 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 -- start start 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 librados: setting wanted keys 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 librados: calling monclient init 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c840f8d30 0x7f0c84110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c840f9600 0x7f0c84110770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 0x7f0c8410a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8410a910 con 0x7f0c840f9600 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8410aa80 con 0x7f0c840f8d30 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c8a514700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8410ac40 con 0x7f0c84102290 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c837fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c840f9600 0x7f0c84110770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.526+0000 7f0c88ab1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 0x7f0c8410a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c837fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c840f9600 0x7f0c84110770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52226/0 (socket says 172.21.15.59:52226) 2024-02-18T00:44:17.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c837fe700 1 -- 172.21.15.59:0/2678270461 learned_addr learned my addr 172.21.15.59:0/2678270461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c840f8d30 0x7f0c84110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c88ab1700 1 -- 172.21.15.59:0/2678270461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c840f8d30 msgr2=0x7f0c84110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c88ab1700 1 --2- 172.21.15.59:0/2678270461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0c840f8d30 0x7f0c84110280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c88ab1700 1 -- 172.21.15.59:0/2678270461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c840f9600 msgr2=0x7f0c84110770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c88ab1700 1 --2- 172.21.15.59:0/2678270461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c840f9600 0x7f0c84110770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c88ab1700 1 -- 172.21.15.59:0/2678270461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0097c0 con 0x7f0c84102290 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c837fe700 1 --2- 172.21.15.59:0/2678270461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c840f9600 0x7f0c84110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c88ab1700 1 --2- 172.21.15.59:0/2678270461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 0x7f0c8410a420 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7f0c7c004e00 tx=0x7f0c7c004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c817fa700 10 client.?.objecter ms_handle_connect 0x7f0c84102290 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c817fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00e040 con 0x7f0c84102290 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c7c018040 con 0x7f0c84102290 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c013cf0 con 0x7f0c84102290 2024-02-18T00:44:17.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8410aec0 con 0x7f0c84102290 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c8a514700 10 client.41699.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c841170b0 con 0x7f0c84102290 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.527+0000 7f0c8a514700 1 librados: init done 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.528+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f0c8410b300 con 0x7f0c84102290 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0c7c013420 con 0x7f0c84102290 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0c6c062c90 0x7f0c6c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(708..708 src has 418..708) v4 ==== 8461+0+0 (secure 0 0 0) 0x7f0c7c07ea60 con 0x7f0c84102290 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 osd_map(708..708 src has 418..708) v4 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 3 client.41699.objecter handle_osd_map got epochs [708,708] > 0 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 3 client.41699.objecter handle_osd_map decoding full epoch 708 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0c6c062c90 0x7f0c6c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 20 client.41699.objecter dump_active .. 0 homeless 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v708) v1 ==== 54+0+16 (secure 0 0 0) 0x7f0c7c07f0a0 con 0x7f0c84102290 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c6c062c90 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.529+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0c6c062c90 0x7f0c6c065150 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 rx=0x7f0c740048b0 tx=0x7f0c7400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 10 client.41699.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 1 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0c840fe5f0 0x7f0c840fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 1 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 0 1 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8405fa60 osd=0 4 2024-02-18T00:44:17.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 10 client.41699.objecter _send_command 1 2024-02-18T00:44:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c8405ee80 con 0x7f0c840fe5f0 2024-02-18T00:44:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c837fe700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0c840fe5f0 0x7f0c840fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c840fe5f0 2024-02-18T00:44:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.530+0000 7f0c837fe700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0c840fe5f0 0x7f0c840fe9d0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.531+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c8405ee80 con 0x7f0c840fe5f0 2024-02-18T00:44:17.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.531+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:44:17.582 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.531+0000 7f0c817fa700 10 client.41699.objecter _finish_command 1 = system:0 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.531+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8405fa60 osd=0 4 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.531+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 0 1 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 10 client.41699.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 2 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 2 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8405fa60 osd=0 3 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 0 2 2024-02-18T00:44:17.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8405fa60 osd=0 4 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 10 client.41699.objecter _send_command 2 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.546+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c84107240 con 0x7f0c840fe5f0 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c84107240 con 0x7f0c840fe5f0 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c817fa700 10 client.41699.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8405fa60 osd=0 4 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 0 2 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 10 client.41699.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 3 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6805/26279110 conn(0x7f0c840598f0 0x7f0c84059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 3 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 1 3 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84028540 osd=1 4 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 10 client.41699.objecter _send_command 3 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c840286b0 con 0x7f0c840598f0 2024-02-18T00:44:17.585 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:44:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.547+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6805/26279110 conn(0x7f0c840598f0 0x7f0c84059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.548+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c840598f0 2024-02-18T00:44:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.548+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6805/26279110 conn(0x7f0c840598f0 0x7f0c84059cd0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.548+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c840286b0 con 0x7f0c840598f0 2024-02-18T00:44:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.548+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:44:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.548+0000 7f0c817fa700 10 client.41699.objecter _finish_command 3 = system:0 2024-02-18T00:44:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.548+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84028540 osd=1 4 2024-02-18T00:44:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.548+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 1 3 2024-02-18T00:44:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 10 client.41699.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 4 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 4 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84028540 osd=1 3 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 1 4 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84028540 osd=1 4 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 10 client.41699.objecter _send_command 4 2024-02-18T00:44:17.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.563+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c84059360 con 0x7f0c840598f0 2024-02-18T00:44:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c84059360 con 0x7f0c840598f0 2024-02-18T00:44:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c817fa700 10 client.41699.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84028540 osd=1 4 2024-02-18T00:44:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 1 4 2024-02-18T00:44:17.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 10 client.41699.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 5 2024-02-18T00:44:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0c840280a0 0x7f0c8411e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 5 2024-02-18T00:44:17.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 2 5 2024-02-18T00:44:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84059770 osd=2 4 2024-02-18T00:44:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 10 client.41699.objecter _send_command 5 2024-02-18T00:44:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c84121040 con 0x7f0c840280a0 2024-02-18T00:44:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c837fe700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0c840280a0 0x7f0c8411e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c840280a0 2024-02-18T00:44:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.565+0000 7f0c837fe700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0c840280a0 0x7f0c8411e3f0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.566+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c84121040 con 0x7f0c840280a0 2024-02-18T00:44:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.566+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:44:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.566+0000 7f0c817fa700 10 client.41699.objecter _finish_command 5 = system:0 2024-02-18T00:44:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.566+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84059770 osd=2 4 2024-02-18T00:44:17.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.566+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 2 5 2024-02-18T00:44:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 10 client.41699.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 6 2024-02-18T00:44:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 6 2024-02-18T00:44:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84059770 osd=2 3 2024-02-18T00:44:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 2 6 2024-02-18T00:44:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84059770 osd=2 4 2024-02-18T00:44:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 10 client.41699.objecter _send_command 6 2024-02-18T00:44:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.581+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c84027660 con 0x7f0c840280a0 2024-02-18T00:44:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.582+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c84027660 con 0x7f0c840280a0 2024-02-18T00:44:17.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.582+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.582+0000 7f0c817fa700 10 client.41699.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.582+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84059770 osd=2 4 2024-02-18T00:44:17.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.582+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 2 6 2024-02-18T00:44:17.596 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:44:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.582+0000 7f0c8a514700 10 client.41699.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.582+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 7 2024-02-18T00:44:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0c8411fa60 0x7f0c8411fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 7 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 3 7 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8411ba30 osd=3 4 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 10 client.41699.objecter _send_command 7 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c8411c140 con 0x7f0c8411fa60 2024-02-18T00:44:17.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c88ab1700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0c8411fa60 0x7f0c8411fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c8411fa60 2024-02-18T00:44:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.583+0000 7f0c88ab1700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0c8411fa60 0x7f0c8411fe80 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.584+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c8411c140 con 0x7f0c8411fa60 2024-02-18T00:44:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.584+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:44:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.584+0000 7f0c817fa700 10 client.41699.objecter _finish_command 7 = system:0 2024-02-18T00:44:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.584+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8411ba30 osd=3 4 2024-02-18T00:44:17.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.584+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 3 7 2024-02-18T00:44:17.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 10 client.41699.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 8 2024-02-18T00:44:17.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 8 2024-02-18T00:44:17.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8411ba30 osd=3 3 2024-02-18T00:44:17.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 3 8 2024-02-18T00:44:17.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8411ba30 osd=3 4 2024-02-18T00:44:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 10 client.41699.objecter _send_command 8 2024-02-18T00:44:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.598+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c84120b40 con 0x7f0c8411fa60 2024-02-18T00:44:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c84120b40 con 0x7f0c8411fa60 2024-02-18T00:44:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c817fa700 10 client.41699.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8411ba30 osd=3 4 2024-02-18T00:44:17.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 3 8 2024-02-18T00:44:17.603 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 10 client.41699.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 9 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0c841231f0 0x7f0c841255e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 9 2024-02-18T00:44:17.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 4 9 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120370 osd=4 4 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 10 client.41699.objecter _send_command 9 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c84120b40 con 0x7f0c841231f0 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0c841231f0 0x7f0c841255e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0c841231f0 0x7f0c841255e0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.600+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c841231f0 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.601+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c84120b40 con 0x7f0c841231f0 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.601+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.601+0000 7f0c817fa700 10 client.41699.objecter _finish_command 9 = system:0 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.601+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84120370 osd=4 4 2024-02-18T00:44:17.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.601+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 4 9 2024-02-18T00:44:17.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 10 client.41699.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 10 2024-02-18T00:44:17.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 10 2024-02-18T00:44:17.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84120370 osd=4 3 2024-02-18T00:44:17.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 4 10 2024-02-18T00:44:17.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120370 osd=4 4 2024-02-18T00:44:17.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 10 client.41699.objecter _send_command 10 2024-02-18T00:44:17.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.616+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c84120d80 con 0x7f0c841231f0 2024-02-18T00:44:17.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c84120d80 con 0x7f0c841231f0 2024-02-18T00:44:17.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c817fa700 10 client.41699.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84120370 osd=4 4 2024-02-18T00:44:17.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 4 10 2024-02-18T00:44:17.621 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:44:17.622 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:17.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 10 client.41699.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 11 2024-02-18T00:44:17.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0c84126020 0x7f0c84126460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 11 2024-02-18T00:44:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 5 11 2024-02-18T00:44:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120d80 osd=5 4 2024-02-18T00:44:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 10 client.41699.objecter _send_command 11 2024-02-18T00:44:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.618+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c841280b0 con 0x7f0c84126020 2024-02-18T00:44:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.619+0000 7f0c837fe700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0c84126020 0x7f0c84126460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.620+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c84126020 2024-02-18T00:44:17.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.620+0000 7f0c837fe700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0c84126020 0x7f0c84126460 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.621+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c7800f040 con 0x7f0c84126020 2024-02-18T00:44:17.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.621+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:44:17.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.621+0000 7f0c817fa700 10 client.41699.objecter _finish_command 11 = system:0 2024-02-18T00:44:17.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.621+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84120d80 osd=5 4 2024-02-18T00:44:17.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.621+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 5 11 2024-02-18T00:44:17.627 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:17.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 10 client.41699.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 12 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 12 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c84120d80 osd=5 3 2024-02-18T00:44:17.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 5 12 2024-02-18T00:44:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120d80 osd=5 4 2024-02-18T00:44:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 10 client.41699.objecter _send_command 12 2024-02-18T00:44:17.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.635+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c8412a2f0 con 0x7f0c84126020 2024-02-18T00:44:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c8412a2f0 con 0x7f0c84126020 2024-02-18T00:44:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c817fa700 10 client.41699.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c84120d80 osd=5 4 2024-02-18T00:44:17.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 5 12 2024-02-18T00:44:17.640 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 10 client.41699.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 13 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0c8412a9e0 0x7f0c8412ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 13 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 6 13 2024-02-18T00:44:17.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412a2f0 osd=6 4 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 10 client.41699.objecter _send_command 13 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.638+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c8412e6f0 con 0x7f0c8412a9e0 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.639+0000 7f0c88ab1700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0c8412a9e0 0x7f0c8412ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.639+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c8412a9e0 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.639+0000 7f0c88ab1700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0c8412a9e0 0x7f0c8412ae40 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.640+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c8412e6f0 con 0x7f0c8412a9e0 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.640+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.640+0000 7f0c817fa700 10 client.41699.objecter _finish_command 13 = system:0 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.640+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8412a2f0 osd=6 4 2024-02-18T00:44:17.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.640+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 6 13 2024-02-18T00:44:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 10 client.41699.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 14 2024-02-18T00:44:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 14 2024-02-18T00:44:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8412a2f0 osd=6 3 2024-02-18T00:44:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 6 14 2024-02-18T00:44:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412a2f0 osd=6 4 2024-02-18T00:44:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 10 client.41699.objecter _send_command 14 2024-02-18T00:44:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.655+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c8412e6f0 con 0x7f0c8412a9e0 2024-02-18T00:44:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c8412e6f0 con 0x7f0c8412a9e0 2024-02-18T00:44:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c817fa700 10 client.41699.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8412a2f0 osd=6 4 2024-02-18T00:44:17.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 6 14 2024-02-18T00:44:17.659 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:44:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 10 client.41699.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 15 2024-02-18T00:44:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6812/53025171 conn(0x7f0c8412da40 0x7f0c8412dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 15 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 7 15 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412e6f0 osd=7 4 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 10 client.41699.objecter _send_command 15 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c8412e510 con 0x7f0c8412da40 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.657+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6812/53025171 conn(0x7f0c8412da40 0x7f0c8412dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.658+0000 7f0c817fa700 10 client.41699.objecter ms_handle_connect 0x7f0c8412da40 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.658+0000 7f0c83fff700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6812/53025171 conn(0x7f0c8412da40 0x7f0c8412dea0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.658+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0c8412e510 con 0x7f0c8412da40 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.658+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.658+0000 7f0c817fa700 10 client.41699.objecter _finish_command 15 = system:0 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.658+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8412e6f0 osd=7 4 2024-02-18T00:44:17.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.658+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 7 15 2024-02-18T00:44:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 10 client.41699.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign -1 16 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 20 client.41699.objecter _get_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_remove -1 16 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 20 client.41699.objecter get_session s=0x7f0c8412e6f0 osd=7 3 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 15 client.41699.objecter _session_command_op_assign 7 16 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412e6f0 osd=7 4 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 10 client.41699.objecter _send_command 16 2024-02-18T00:44:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.673+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0c8412e510 con 0x7f0c8412da40 2024-02-18T00:44:17.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c817fa700 1 -- 172.21.15.59:0/2678270461 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0c8412e510 con 0x7f0c8412da40 2024-02-18T00:44:17.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c817fa700 10 client.41699.objecter ms_dispatch 0x7f0c84001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:17.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c817fa700 10 client.41699.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c817fa700 20 client.41699.objecter put_session s=0x7f0c8412e6f0 osd=7 4 2024-02-18T00:44:17.677 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c817fa700 15 client.41699.objecter _session_command_op_remove 7 16 2024-02-18T00:44:17.677 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:44:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c8a514700 10 librados: watch_flush enter 2024-02-18T00:44:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c8a514700 10 librados: watch_flush exit 2024-02-18T00:44:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.0 2024-02-18T00:44:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0c840fe5f0 msgr2=0x7f0c840fe9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.675+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6801/3700238977 conn(0x7f0c840fe5f0 0x7f0c840fe9d0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8405fa60 osd=0 1 2024-02-18T00:44:17.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.1 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6805/26279110 conn(0x7f0c840598f0 msgr2=0x7f0c84059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6805/26279110 conn(0x7f0c840598f0 0x7f0c84059cd0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84028540 osd=1 1 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.2 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0c840280a0 msgr2=0x7f0c8411e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6809/1153193637 conn(0x7f0c840280a0 0x7f0c8411e3f0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84059770 osd=2 1 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.3 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0c8411fa60 msgr2=0x7f0c8411fe80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6813/1805700964 conn(0x7f0c8411fa60 0x7f0c8411fe80 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8411ba30 osd=3 1 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.4 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0c841231f0 msgr2=0x7f0c841255e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c88ab1700 1 -- 172.21.15.59:0/2678270461 reap_dead start 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0c841231f0 0x7f0c841255e0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c88ab1700 1 -- 172.21.15.59:0/2678270461 reap_dead start 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120370 osd=4 1 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.5 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0c84126020 msgr2=0x7f0c84126460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0c84126020 0x7f0c84126460 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c84120d80 osd=5 1 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.6 2024-02-18T00:44:17.681 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0c8412a9e0 msgr2=0x7f0c8412ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0c8412a9e0 0x7f0c8412ae40 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412a2f0 osd=6 1 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter close_session for osd.7 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6812/53025171 conn(0x7f0c8412da40 msgr2=0x7f0c8412dea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.137:6812/53025171 conn(0x7f0c8412da40 0x7f0c8412dea0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter put_session s=0x7f0c8412e6f0 osd=7 1 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 20 client.41699.objecter shutdown clearing up homeless session... 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 10 client.41699.objecter successfully canceled tick 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0c6c062c90 msgr2=0x7f0c6c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0c6c062c90 0x7f0c6c065150 secure :-1 s=READY pgs=3204 cs=0 l=1 rev1=1 rx=0x7f0c740048b0 tx=0x7f0c7400b040).stop 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 msgr2=0x7f0c8410a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 --2- 172.21.15.59:0/2678270461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0c84102290 0x7f0c8410a420 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 rx=0x7f0c7c004e00 tx=0x7f0c7c004d10).stop 2024-02-18T00:44:17.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c88ab1700 1 -- 172.21.15.59:0/2678270461 reap_dead start 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c88ab1700 1 -- 172.21.15.59:0/2678270461 reap_dead start 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 shutdown_connections 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 >> 172.21.15.59:0/2678270461 conn(0x7f0c840f2820 msgr2=0x7f0c840f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 shutdown_connections 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 -- 172.21.15.59:0/2678270461 wait complete. 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:17.676+0000 7f0c8a514700 1 librados: shutdown 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.683 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.684 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:44:17.729 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:17.761 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:17.782 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:17.785 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:17.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:17.886 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:17.914 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:17.933 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:17.936 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:17.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:16.835042+0000 mon.a (mon.0) 3441 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:44:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:16.844201+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-18T00:44:18.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:17.529804+0000 mon.c (mon.2) 568 : audit [DBG] from='client.? 172.21.15.59:0/2678270461' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:18.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:16.835042+0000 mon.a (mon.0) 3441 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:44:18.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:16.844201+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-18T00:44:18.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:17.529804+0000 mon.c (mon.2) 568 : audit [DBG] from='client.? 172.21.15.59:0/2678270461' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:18.037 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:18.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:16.835042+0000 mon.a (mon.0) 3441 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:44:18.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:16.844201+0000 mon.a (mon.0) 3442 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-18T00:44:18.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:17.529804+0000 mon.c (mon.2) 568 : audit [DBG] from='client.? 172.21.15.59:0/2678270461' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:18.063 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:18.082 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:18.084 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:18.085 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:18.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:18.216 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:17.850865+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:17.856385+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:17.999845+0000 mgr.y (mgr.24454) 618 : cluster [DBG] pgmap v1086: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:18.797782+0000 mon.a (mon.0) 3445 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:18.797993+0000 mon.a (mon.0) 3446 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:18.801402+0000 mon.a (mon.0) 3447 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:17.850865+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-18T00:44:19.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:17.856385+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:17.999845+0000 mgr.y (mgr.24454) 618 : cluster [DBG] pgmap v1086: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T00:44:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:18.797782+0000 mon.a (mon.0) 3445 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:18.797993+0000 mon.a (mon.0) 3446 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-18T00:44:19.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:18.801402+0000 mon.a (mon.0) 3447 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:19.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:17.850865+0000 mon.a (mon.0) 3443 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-18T00:44:19.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:17.856385+0000 mon.a (mon.0) 3444 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:19.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:17.999845+0000 mgr.y (mgr.24454) 618 : cluster [DBG] pgmap v1086: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T00:44:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:18.797782+0000 mon.a (mon.0) 3445 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:18.797993+0000 mon.a (mon.0) 3446 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-18T00:44:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:18.801402+0000 mon.a (mon.0) 3447 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:19.466 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:19.470 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:19.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:19.571 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:19.603 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:19.624 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:19.626 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:19.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:19.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:19.762 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:21.061 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:19.801128+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:21.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:19.801206+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-18T00:44:21.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:19.802459+0000 mon.a (mon.0) 3450 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-124"}]: dispatch 2024-02-18T00:44:21.062 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:20.000355+0000 mgr.y (mgr.24454) 619 : cluster [DBG] pgmap v1089: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:21.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:19.801128+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:19.801206+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-18T00:44:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:19.802459+0000 mon.a (mon.0) 3450 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-124"}]: dispatch 2024-02-18T00:44:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:20.000355+0000 mgr.y (mgr.24454) 619 : cluster [DBG] pgmap v1089: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:19.801128+0000 mon.a (mon.0) 3448 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:19.801206+0000 mon.a (mon.0) 3449 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-18T00:44:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:19.802459+0000 mon.a (mon.0) 3450 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-124"}]: dispatch 2024-02-18T00:44:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:20.000355+0000 mgr.y (mgr.24454) 619 : cluster [DBG] pgmap v1089: 214 pgs: 32 unknown, 182 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:21.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:44:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:44:21.506 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:22.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:44:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:44:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:20.806122+0000 mon.a (mon.0) 3451 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-124"}]': finished 2024-02-18T00:44:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:20.806229+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-18T00:44:22.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:20.806886+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-124", "mode": "writeback"}]: dispatch 2024-02-18T00:44:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 -- 172.21.15.59:0/313784957 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040fd510 msgr2=0x7faa040f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/313784957 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040fd510 0x7faa040f6d40 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 rx=0x7fa9f4004870 tx=0x7fa9f4009cb0).stop 2024-02-18T00:44:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 -- 172.21.15.59:0/313784957 shutdown_connections 2024-02-18T00:44:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/313784957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa040fcbb0 0x7faa040fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/313784957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040f7400 0x7faa04103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/313784957 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040fd510 0x7faa040f6d40 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 -- 172.21.15.59:0/313784957 >> 172.21.15.59:0/313784957 conn(0x7faa040f2820 msgr2=0x7faa040f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 -- 172.21.15.59:0/313784957 shutdown_connections 2024-02-18T00:44:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.140+0000 7faa0c5ea700 1 -- 172.21.15.59:0/313784957 wait complete. 2024-02-18T00:44:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 librados: starting msgr at 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 librados: starting objecter 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 Processor -- start 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 -- start start 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 librados: setting wanted keys 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 librados: calling monclient init 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040f7400 0x7faa04109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa040fcbb0 0x7faa04109730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040fd510 0x7faa04109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:22.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa04114430 con 0x7faa040fd510 2024-02-18T00:44:22.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041145a0 con 0x7faa040fcbb0 2024-02-18T00:44:22.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0c5ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa04114760 con 0x7faa040f7400 2024-02-18T00:44:22.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0ab87700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040fd510 0x7faa04109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:22.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0a386700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040f7400 0x7faa04109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:22.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0ab87700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040fd510 0x7faa04109cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55560/0 (socket says 172.21.15.59:55560) 2024-02-18T00:44:22.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0ab87700 1 -- 172.21.15.59:0/660824 learned_addr learned my addr 172.21.15.59:0/660824 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:22.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.141+0000 7faa0a386700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040f7400 0x7faa04109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55432/0 (socket says 172.21.15.59:55432) 2024-02-18T00:44:22.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa09b85700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa040fcbb0 0x7faa04109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:22.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0ab87700 1 -- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040f7400 msgr2=0x7faa04109240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:22.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0ab87700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040f7400 0x7faa04109240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0ab87700 1 -- 172.21.15.59:0/660824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa040fcbb0 msgr2=0x7faa04109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0ab87700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa040fcbb0 0x7faa04109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0ab87700 1 -- 172.21.15.59:0/660824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0097c0 con 0x7faa040fd510 2024-02-18T00:44:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0a386700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040f7400 0x7faa04109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0ab87700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040fd510 0x7faa04109cf0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7fa9fc001df0 tx=0x7fa9fc004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7fa9fb7fe700 10 client.?.objecter ms_handle_connect 0x7faa040fd510 2024-02-18T00:44:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7fa9fb7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:22.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7fa9fb7fe700 1 -- 172.21.15.59:0/660824 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc009a10 con 0x7faa040fd510 2024-02-18T00:44:22.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f4005040 con 0x7faa040fd510 2024-02-18T00:44:22.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7fa9fb7fe700 1 -- 172.21.15.59:0/660824 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9fc0090d0 con 0x7faa040fd510 2024-02-18T00:44:22.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7fa9fb7fe700 1 -- 172.21.15.59:0/660824 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc007d30 con 0x7faa040fd510 2024-02-18T00:44:22.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0c5ea700 10 client.31899.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:22.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa04114d50 con 0x7faa040fd510 2024-02-18T00:44:22.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.142+0000 7faa0c5ea700 1 librados: init done 2024-02-18T00:44:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.143+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0405fa60 con 0x7faa040fd510 2024-02-18T00:44:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7fa9fb7fe700 1 -- 172.21.15.59:0/660824 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa9fc009240 con 0x7faa040fd510 2024-02-18T00:44:22.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7fa9fb7fe700 1 --2- 172.21.15.59:0/660824 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa9f0062c90 0x7fa9f0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7fa9fb7fe700 1 -- 172.21.15.59:0/660824 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(713..713 src has 418..713) v4 ==== 8814+0+0 (secure 0 0 0) 0x7fa9fc07dd90 con 0x7faa040fd510 2024-02-18T00:44:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7fa9fb7fe700 10 client.31899.objecter ms_dispatch 0x7faa04001550 osd_map(713..713 src has 418..713) v4 2024-02-18T00:44:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7fa9fb7fe700 3 client.31899.objecter handle_osd_map got epochs [713,713] > 0 2024-02-18T00:44:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7fa9fb7fe700 3 client.31899.objecter handle_osd_map decoding full epoch 713 2024-02-18T00:44:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7faa0a386700 1 --2- 172.21.15.59:0/660824 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa9f0062c90 0x7fa9f0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:22.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.144+0000 7fa9fb7fe700 20 client.31899.objecter dump_active .. 0 homeless 2024-02-18T00:44:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.145+0000 7fa9fb7fe700 10 client.31899.objecter ms_handle_connect 0x7fa9f0062c90 2024-02-18T00:44:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.145+0000 7faa0a386700 1 --2- 172.21.15.59:0/660824 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa9f0062c90 0x7fa9f0065150 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 rx=0x7faa000041d0 tx=0x7faa00009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:22.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.149+0000 7fa9fb7fe700 1 -- 172.21.15.59:0/660824 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa9fc081030 con 0x7faa040fd510 2024-02-18T00:44:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:20.806122+0000 mon.a (mon.0) 3451 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-124"}]': finished 2024-02-18T00:44:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:20.806229+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-18T00:44:22.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:20.806886+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-124", "mode": "writeback"}]: dispatch 2024-02-18T00:44:22.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:20.806122+0000 mon.a (mon.0) 3451 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-124"}]': finished 2024-02-18T00:44:22.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:20.806229+0000 mon.a (mon.0) 3452 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-18T00:44:22.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:20.806886+0000 mon.a (mon.0) 3453 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-124", "mode": "writeback"}]: dispatch 2024-02-18T00:44:22.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.335+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7faa04061460 con 0x7faa040fd510 2024-02-18T00:44:22.416 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:22.419 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:22.419 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:22.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:22] "GET /metrics HTTP/1.1" 200 191729 "" "Prometheus/2.33.4" 2024-02-18T00:44:22.520 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:22.554 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:22.803 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:22.805 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:22.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:22.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.818+0000 7fa9fb7fe700 1 -- 172.21.15.59:0/660824 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v714) v1 ==== 91+0+0 (secure 0 0 0) 0x7fa9fc050f10 con 0x7faa040fd510 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 10 librados: watch_flush enter 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 10 librados: watch_flush exit 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 20 client.31899.objecter shutdown clearing up homeless session... 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 10 client.31899.objecter successfully canceled tick 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa9f0062c90 msgr2=0x7fa9f0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/660824 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa9f0062c90 0x7fa9f0065150 secure :-1 s=READY pgs=3205 cs=0 l=1 rev1=1 rx=0x7faa000041d0 tx=0x7faa00009040).stop 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040fd510 msgr2=0x7faa04109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040fd510 0x7faa04109cf0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7fa9fc001df0 tx=0x7fa9fc004c50).stop 2024-02-18T00:44:22.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 shutdown_connections 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa040fd510 0x7faa04109cf0 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa040f7400 0x7faa04109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/660824 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa9f0062c90 0x7fa9f0065150 unknown :-1 s=CLOSED pgs=3205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 --2- 172.21.15.59:0/660824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa040fcbb0 0x7faa04109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 >> 172.21.15.59:0/660824 conn(0x7faa040f2820 msgr2=0x7faa040f4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 shutdown_connections 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 -- 172.21.15.59:0/660824 wait complete. 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:22.820+0000 7faa0c5ea700 1 librados: shutdown 2024-02-18T00:44:22.823 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:44:22.907 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:22.935 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:23.184 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:23.186 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:21.554387+0000 mgr.y (mgr.24454) 620 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:23.186 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:21.803120+0000 mon.a (mon.0) 3454 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:23.186 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:21.811095+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-124", "mode": "writeback"}]': finished 2024-02-18T00:44:23.186 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:21.811187+0000 mon.a (mon.0) 3456 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:21.852053+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:22.000937+0000 mgr.y (mgr.24454) 621 : cluster [DBG] pgmap v1092: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:22.336778+0000 mon.a (mon.0) 3458 : audit [INF] from='client.? 172.21.15.59:0/660824' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:21.554387+0000 mgr.y (mgr.24454) 620 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:21.803120+0000 mon.a (mon.0) 3454 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:21.811095+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-124", "mode": "writeback"}]': finished 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:21.811187+0000 mon.a (mon.0) 3456 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:21.852053+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:22.000937+0000 mgr.y (mgr.24454) 621 : cluster [DBG] pgmap v1092: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:23.187 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:22.336778+0000 mon.a (mon.0) 3458 : audit [INF] from='client.? 172.21.15.59:0/660824' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:44:23.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:23.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:23.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:21.554387+0000 mgr.y (mgr.24454) 620 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:23.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:21.803120+0000 mon.a (mon.0) 3454 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:23.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:21.811095+0000 mon.a (mon.0) 3455 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-124", "mode": "writeback"}]': finished 2024-02-18T00:44:23.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:21.811187+0000 mon.a (mon.0) 3456 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-18T00:44:23.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:21.852053+0000 mon.a (mon.0) 3457 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:23.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:22.000937+0000 mgr.y (mgr.24454) 621 : cluster [DBG] pgmap v1092: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:23.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:22.336778+0000 mon.a (mon.0) 3458 : audit [INF] from='client.? 172.21.15.59:0/660824' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:44:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:44:23.288 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:44:23.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:23.316 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:23.492 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:44:23.817 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:22.254695+0000 osd.6 (osd.6) 313 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:23.817 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:22.254960+0000 osd.6 (osd.6) 314 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:23.817 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:22.808475+0000 mon.a (mon.0) 3459 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:22.816903+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:22.816979+0000 mon.a (mon.0) 3461 : audit [INF] from='client.? 172.21.15.59:0/660824' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:22.817022+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:22.818103+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124"}]: dispatch 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.212541+0000 osd.7 (osd.7) 209 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.214004+0000 osd.7 (osd.7) 210 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.565031+0000 osd.4 (osd.4) 379 : cluster [DBG] 4.e scrub starts 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.568508+0000 osd.4 (osd.4) 380 : cluster [DBG] 4.e scrub ok 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.793524+0000 mon.a (mon.0) 3464 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.794298+0000 mon.a (mon.0) 3465 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:23.800349+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124"}]': finished 2024-02-18T00:44:23.818 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.800415+0000 mon.a (mon.0) 3467 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-18T00:44:24.115 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:22.254695+0000 osd.6 (osd.6) 313 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:22.254960+0000 osd.6 (osd.6) 314 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:22.808475+0000 mon.a (mon.0) 3459 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:22.816903+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:22.816979+0000 mon.a (mon.0) 3461 : audit [INF] from='client.? 172.21.15.59:0/660824' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:22.817022+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:22.818103+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124"}]: dispatch 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.212541+0000 osd.7 (osd.7) 209 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.214004+0000 osd.7 (osd.7) 210 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.565031+0000 osd.4 (osd.4) 379 : cluster [DBG] 4.e scrub starts 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.568508+0000 osd.4 (osd.4) 380 : cluster [DBG] 4.e scrub ok 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.793524+0000 mon.a (mon.0) 3464 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:44:24.116 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.794298+0000 mon.a (mon.0) 3465 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:24.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:23.800349+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124"}]': finished 2024-02-18T00:44:24.117 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.800415+0000 mon.a (mon.0) 3467 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-18T00:44:24.136 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_blocked_ops 2024-02-18T00:44:24.166 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:24.168 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:24.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:24.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:22.254695+0000 osd.6 (osd.6) 313 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:22.254960+0000 osd.6 (osd.6) 314 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:22.808475+0000 mon.a (mon.0) 3459 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:22.816903+0000 mon.a (mon.0) 3460 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:22.816979+0000 mon.a (mon.0) 3461 : audit [INF] from='client.? 172.21.15.59:0/660824' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:22.817022+0000 mon.a (mon.0) 3462 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:22.818103+0000 mon.a (mon.0) 3463 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124"}]: dispatch 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.212541+0000 osd.7 (osd.7) 209 : cluster [DBG] 5.10 scrub starts 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.214004+0000 osd.7 (osd.7) 210 : cluster [DBG] 5.10 scrub ok 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.565031+0000 osd.4 (osd.4) 379 : cluster [DBG] 4.e scrub starts 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.568508+0000 osd.4 (osd.4) 380 : cluster [DBG] 4.e scrub ok 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.793524+0000 mon.a (mon.0) 3464 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:44:24.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.794298+0000 mon.a (mon.0) 3465 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:23.800349+0000 mon.a (mon.0) 3466 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-124"}]': finished 2024-02-18T00:44:24.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.800415+0000 mon.a (mon.0) 3467 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-18T00:44:24.269 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:24.301 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:24.323 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:24.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:24.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:24.426 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:24.458 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:24.480 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:24.482 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:24.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:24.584 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:24.616 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:24.636 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:24.639 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:24.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:24.741 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:24.772 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:24.793 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:24.795 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:24.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:24.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:24.924 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:24.999 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:25.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.125906+0000 osd.2 (osd.2) 283 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:25.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:23.126091+0000 osd.2 (osd.2) 284 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:25.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.001368+0000 mgr.y (mgr.24454) 622 : cluster [DBG] pgmap v1095: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.178850+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:44:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.180449+0000 osd.1 (osd.1) 338 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:44:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.228806+0000 osd.3 (osd.3) 357 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:44:25.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.230806+0000 osd.3 (osd.3) 358 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:44:25.005 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:25.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:25.106 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:25.135 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:25.154 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:25.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:25.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:25.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.125906+0000 osd.2 (osd.2) 283 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:25.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:23.126091+0000 osd.2 (osd.2) 284 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:25.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.001368+0000 mgr.y (mgr.24454) 622 : cluster [DBG] pgmap v1095: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:25.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.178850+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:44:25.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.180449+0000 osd.1 (osd.1) 338 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:44:25.159 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.228806+0000 osd.3 (osd.3) 357 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:44:25.160 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.230806+0000 osd.3 (osd.3) 358 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:44:25.160 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.125906+0000 osd.2 (osd.2) 283 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:25.160 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:23.126091+0000 osd.2 (osd.2) 284 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:25.160 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.001368+0000 mgr.y (mgr.24454) 622 : cluster [DBG] pgmap v1095: 214 pgs: 214 active+clean; 5.6 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:44:25.160 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.178850+0000 osd.1 (osd.1) 337 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:44:25.160 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.180449+0000 osd.1 (osd.1) 338 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:44:25.161 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.228806+0000 osd.3 (osd.3) 357 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:44:25.161 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.230806+0000 osd.3 (osd.3) 358 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:44:25.258 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:25.287 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:25.541 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:25.545 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:25.546 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:25.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:25.675 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:25.799 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:25.802 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:25.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:25.903 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:25.934 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 -- 172.21.15.59:0/2669325934 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fd510 msgr2=0x7f35440f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 --2- 172.21.15.59:0/2669325934 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fd510 0x7f35440f6d40 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 rx=0x7f3538004870 tx=0x7f3538009cb0).stop 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 -- 172.21.15.59:0/2669325934 shutdown_connections 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 --2- 172.21.15.59:0/2669325934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35440f7400 0x7f3544103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 --2- 172.21.15.59:0/2669325934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35440fcbb0 0x7f35440fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 --2- 172.21.15.59:0/2669325934 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fd510 0x7f35440f6d40 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 -- 172.21.15.59:0/2669325934 >> 172.21.15.59:0/2669325934 conn(0x7f35440f2820 msgr2=0x7f35440f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 -- 172.21.15.59:0/2669325934 shutdown_connections 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.066+0000 7f354a20c700 1 -- 172.21.15.59:0/2669325934 wait complete. 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 librados: starting msgr at 2024-02-18T00:44:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 librados: starting objecter 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 Processor -- start 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 -- start start 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 librados: setting wanted keys 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 librados: calling monclient init 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35440f7400 0x7f35441093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fcbb0 0x7f35441098e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35440fd510 0x7f3544109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35441145e0 con 0x7f35440f7400 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3544114750 con 0x7f35440fd510 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f354a20c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3544114910 con 0x7f35440fcbb0 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f35437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35440f7400 0x7f35441093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:26.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f3542ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fcbb0 0x7f35441098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f35437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35440f7400 0x7f35441093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55588/0 (socket says 172.21.15.59:55588) 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f3542ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fcbb0 0x7f35441098e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55450/0 (socket says 172.21.15.59:55450) 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.067+0000 7f35437fe700 1 -- 172.21.15.59:0/149353556 learned_addr learned my addr 172.21.15.59:0/149353556 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3543fff700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35440fd510 0x7f3544109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3542ffd700 1 -- 172.21.15.59:0/149353556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35440fd510 msgr2=0x7f3544109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3542ffd700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35440fd510 0x7f3544109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3542ffd700 1 -- 172.21.15.59:0/149353556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35440f7400 msgr2=0x7f35441093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3542ffd700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35440f7400 0x7f35441093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3542ffd700 1 -- 172.21.15.59:0/149353556 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534005040 con 0x7f35440fcbb0 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3542ffd700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fcbb0 0x7f35441098e0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7f3538004870 tx=0x7f353800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3540ff9700 10 client.?.objecter ms_handle_connect 0x7f35440fcbb0 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3540ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3540ff9700 1 -- 172.21.15.59:0/149353556 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800e070 con 0x7f35440fcbb0 2024-02-18T00:44:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3540ff9700 1 -- 172.21.15.59:0/149353556 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3538013da0 con 0x7f35440fcbb0 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3538005040 con 0x7f35440fcbb0 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f3540ff9700 1 -- 172.21.15.59:0/149353556 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538019070 con 0x7f35440fcbb0 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f354a20c700 10 client.41711.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3544114e00 con 0x7f35440fcbb0 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.068+0000 7f354a20c700 1 librados: init done 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.069+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354405fa60 con 0x7f35440fcbb0 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f3540ff9700 1 -- 172.21.15.59:0/149353556 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f353801e420 con 0x7f35440fcbb0 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f3540ff9700 1 --2- 172.21.15.59:0/149353556 >> v2:172.21.15.59:6800/3721891797 conn(0x7f352c062c90 0x7f352c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f3540ff9700 1 -- 172.21.15.59:0/149353556 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(717..717 src has 418..717) v4 ==== 8806+0+0 (secure 0 0 0) 0x7f353807e090 con 0x7f35440fcbb0 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f3540ff9700 10 client.41711.objecter ms_dispatch 0x7f3544001550 osd_map(717..717 src has 418..717) v4 2024-02-18T00:44:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f3540ff9700 3 client.41711.objecter handle_osd_map got epochs [717,717] > 0 2024-02-18T00:44:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f3540ff9700 3 client.41711.objecter handle_osd_map decoding full epoch 717 2024-02-18T00:44:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f35437fe700 1 --2- 172.21.15.59:0/149353556 >> v2:172.21.15.59:6800/3721891797 conn(0x7f352c062c90 0x7f352c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.070+0000 7f3540ff9700 20 client.41711.objecter dump_active .. 0 homeless 2024-02-18T00:44:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.071+0000 7f3540ff9700 10 client.41711.objecter ms_handle_connect 0x7f352c062c90 2024-02-18T00:44:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.071+0000 7f35437fe700 1 --2- 172.21.15.59:0/149353556 >> v2:172.21.15.59:6800/3721891797 conn(0x7f352c062c90 0x7f352c065150 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 rx=0x7f3534009410 tx=0x7f353400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.074+0000 7f3540ff9700 1 -- 172.21.15.59:0/149353556 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f35380512f0 con 0x7f35440fcbb0 2024-02-18T00:44:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.820233+0000 mon.a (mon.0) 3468 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.977335+0000 osd.4 (osd.4) 381 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:24.977476+0000 osd.4 (osd.4) 382 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:25.279557+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.820233+0000 mon.a (mon.0) 3468 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.977335+0000 osd.4 (osd.4) 381 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:24.977476+0000 osd.4 (osd.4) 382 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:26.104 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:25.279557+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:26.105 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:26.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:26.106 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:26.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:26.236 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:26.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.820233+0000 mon.a (mon.0) 3468 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-18T00:44:26.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.977335+0000 osd.4 (osd.4) 381 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:26.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:24.977476+0000 osd.4 (osd.4) 382 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:26.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:25.279557+0000 mon.a (mon.0) 3469 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:26.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.273+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7f354405ee80 con 0x7f35440fcbb0 2024-02-18T00:44:26.359 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:26.361 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:26.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:26.463 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:26.490 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:26.590 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:26.593 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:26.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:26.694 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:26.722 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:26.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.831+0000 7f3540ff9700 1 -- 172.21.15.59:0/149353556 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v718) v1 ==== 113+0+0 (secure 0 0 0) 0x7f3538023070 con 0x7f35440fcbb0 2024-02-18T00:44:26.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.835+0000 7f354a20c700 10 librados: watch_flush enter 2024-02-18T00:44:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.836+0000 7f354a20c700 10 librados: watch_flush exit 2024-02-18T00:44:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.836+0000 7f354a20c700 20 client.41711.objecter shutdown clearing up homeless session... 2024-02-18T00:44:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.836+0000 7f354a20c700 10 client.41711.objecter successfully canceled tick 2024-02-18T00:44:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.837+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 >> v2:172.21.15.59:6800/3721891797 conn(0x7f352c062c90 msgr2=0x7f352c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.837+0000 7f354a20c700 1 --2- 172.21.15.59:0/149353556 >> v2:172.21.15.59:6800/3721891797 conn(0x7f352c062c90 0x7f352c065150 secure :-1 s=READY pgs=3206 cs=0 l=1 rev1=1 rx=0x7f3534009410 tx=0x7f353400b040).stop 2024-02-18T00:44:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.837+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fcbb0 msgr2=0x7f35441098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:26.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.837+0000 7f354a20c700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fcbb0 0x7f35441098e0 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 rx=0x7f3538004870 tx=0x7f353800b040).stop 2024-02-18T00:44:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.845+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 shutdown_connections 2024-02-18T00:44:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.845+0000 7f354a20c700 1 --2- 172.21.15.59:0/149353556 >> v2:172.21.15.59:6800/3721891797 conn(0x7f352c062c90 0x7f352c065150 unknown :-1 s=CLOSED pgs=3206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.845+0000 7f354a20c700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35440fd510 0x7f3544109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.845+0000 7f354a20c700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35440f7400 0x7f35441093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.845+0000 7f354a20c700 1 --2- 172.21.15.59:0/149353556 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35440fcbb0 0x7f35441098e0 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.845+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 >> 172.21.15.59:0/149353556 conn(0x7f35440f2820 msgr2=0x7f35440fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.846+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 shutdown_connections 2024-02-18T00:44:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.846+0000 7f354a20c700 1 -- 172.21.15.59:0/149353556 wait complete. 2024-02-18T00:44:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:26.846+0000 7f354a20c700 1 librados: shutdown 2024-02-18T00:44:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.4 to 1 (10000) 2024-02-18T00:44:26.972 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:25.508645+0000 osd.6 (osd.6) 315 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:25.509519+0000 osd.6 (osd.6) 316 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:25.531291+0000 osd.0 (osd.0) 310 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:25.533442+0000 osd.0 (osd.0) 311 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:25.761949+0000 osd.5 (osd.5) 301 : cluster [DBG] 5.f scrub starts 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:25.764902+0000 osd.5 (osd.5) 302 : cluster [DBG] 5.f scrub ok 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:25.826990+0000 mon.a (mon.0) 3470 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-18T00:44:26.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:25.834646+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:26.001904+0000 mgr.y (mgr.24454) 623 : cluster [DBG] pgmap v1098: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:26.093738+0000 osd.1 (osd.1) 339 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:26.097363+0000 osd.1 (osd.1) 340 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:26.274701+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.59:0/149353556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:26.275008+0000 mon.a (mon.0) 3472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:25.508645+0000 osd.6 (osd.6) 315 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:25.509519+0000 osd.6 (osd.6) 316 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:25.531291+0000 osd.0 (osd.0) 310 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:25.533442+0000 osd.0 (osd.0) 311 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:44:26.975 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:25.761949+0000 osd.5 (osd.5) 301 : cluster [DBG] 5.f scrub starts 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:25.764902+0000 osd.5 (osd.5) 302 : cluster [DBG] 5.f scrub ok 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:25.826990+0000 mon.a (mon.0) 3470 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:25.834646+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:26.001904+0000 mgr.y (mgr.24454) 623 : cluster [DBG] pgmap v1098: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:26.093738+0000 osd.1 (osd.1) 339 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:26.097363+0000 osd.1 (osd.1) 340 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:26.274701+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.59:0/149353556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:44:26.976 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:26.275008+0000 mon.a (mon.0) 3472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:44:26.977 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:26.977 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:27.078 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:25.508645+0000 osd.6 (osd.6) 315 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:25.509519+0000 osd.6 (osd.6) 316 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:25.531291+0000 osd.0 (osd.0) 310 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:25.533442+0000 osd.0 (osd.0) 311 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:25.761949+0000 osd.5 (osd.5) 301 : cluster [DBG] 5.f scrub starts 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:25.764902+0000 osd.5 (osd.5) 302 : cluster [DBG] 5.f scrub ok 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:25.826990+0000 mon.a (mon.0) 3470 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-18T00:44:27.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:25.834646+0000 mon.a (mon.0) 3471 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:27.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:26.001904+0000 mgr.y (mgr.24454) 623 : cluster [DBG] pgmap v1098: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:27.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:26.093738+0000 osd.1 (osd.1) 339 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:44:27.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:26.097363+0000 osd.1 (osd.1) 340 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:44:27.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:26.274701+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.59:0/149353556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:44:27.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:26.275008+0000 mon.a (mon.0) 3472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:44:27.110 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:27.131 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:27.134 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:27.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:27.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:27.267 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:27.559 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:44:27.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:26.831987+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:27.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:26.832170+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:44:27.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:26.832235+0000 mon.a (mon.0) 3475 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-18T00:44:27.862 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:26.838702+0000 mon.a (mon.0) 3476 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:27.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:27.617373+0000 osd.4 (osd.4) 383 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:44:27.863 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:27.619292+0000 osd.4 (osd.4) 384 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:44:27.863 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:26.831987+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:27.863 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:26.832170+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:44:27.863 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:26.832235+0000 mon.a (mon.0) 3475 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-18T00:44:27.863 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:26.838702+0000 mon.a (mon.0) 3476 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:27.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:27.617373+0000 osd.4 (osd.4) 383 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:44:27.864 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:27.619292+0000 osd.4 (osd.4) 384 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:44:27.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 5 1 2024-02-18T00:44:27.916 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:27.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:27.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:28.021 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:28.055 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:28.079 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:28.080 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:28.180 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:28.209 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:26.831987+0000 mon.a (mon.0) 3473 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:28.209 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:26.832170+0000 mon.a (mon.0) 3474 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:44:28.209 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:26.832235+0000 mon.a (mon.0) 3475 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-18T00:44:28.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:26.838702+0000 mon.a (mon.0) 3476 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:28.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:27.617373+0000 osd.4 (osd.4) 383 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:44:28.210 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:27.619292+0000 osd.4 (osd.4) 384 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:44:28.211 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:28.232 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:28.235 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:28.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:28.336 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:28.364 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:28.462 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:28.466 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:28.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:28.567 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:28.595 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:28.845 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:28.849 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:28.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:28.950 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:28.982 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:29.003 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:29.006 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:29.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:29.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:29.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:27.710462+0000 osd.5 (osd.5) 303 : cluster [DBG] 234.3 scrub starts 2024-02-18T00:44:29.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:27.711702+0000 osd.5 (osd.5) 304 : cluster [DBG] 234.3 scrub ok 2024-02-18T00:44:29.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:27.832274+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:29.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:27.832349+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-18T00:44:29.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:27.833892+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-126"}]: dispatch 2024-02-18T00:44:29.124 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.002321+0000 mgr.y (mgr.24454) 624 : cluster [DBG] pgmap v1101: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.069253+0000 osd.1 (osd.1) 341 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.072264+0000 osd.1 (osd.1) 342 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.185359+0000 osd.7 (osd.7) 211 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.186412+0000 osd.7 (osd.7) 212 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.249698+0000 osd.3 (osd.3) 359 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.251165+0000 osd.3 (osd.3) 360 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.631613+0000 osd.4 (osd.4) 385 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.633073+0000 osd.4 (osd.4) 386 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:27.710462+0000 osd.5 (osd.5) 303 : cluster [DBG] 234.3 scrub starts 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:27.711702+0000 osd.5 (osd.5) 304 : cluster [DBG] 234.3 scrub ok 2024-02-18T00:44:29.125 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:27.832274+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:27.832349+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:27.833892+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-126"}]: dispatch 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.002321+0000 mgr.y (mgr.24454) 624 : cluster [DBG] pgmap v1101: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.069253+0000 osd.1 (osd.1) 341 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.072264+0000 osd.1 (osd.1) 342 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.185359+0000 osd.7 (osd.7) 211 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.186412+0000 osd.7 (osd.7) 212 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.249698+0000 osd.3 (osd.3) 359 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.251165+0000 osd.3 (osd.3) 360 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:44:29.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.631613+0000 osd.4 (osd.4) 385 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:44:29.127 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.633073+0000 osd.4 (osd.4) 386 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:27.710462+0000 osd.5 (osd.5) 303 : cluster [DBG] 234.3 scrub starts 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:27.711702+0000 osd.5 (osd.5) 304 : cluster [DBG] 234.3 scrub ok 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:27.832274+0000 mon.a (mon.0) 3477 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:27.832349+0000 mon.a (mon.0) 3478 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:27.833892+0000 mon.a (mon.0) 3479 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-126"}]: dispatch 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.002321+0000 mgr.y (mgr.24454) 624 : cluster [DBG] pgmap v1101: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.069253+0000 osd.1 (osd.1) 341 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.072264+0000 osd.1 (osd.1) 342 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:44:29.135 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.185359+0000 osd.7 (osd.7) 211 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:44:29.136 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.186412+0000 osd.7 (osd.7) 212 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:44:29.136 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.249698+0000 osd.3 (osd.3) 359 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:44:29.136 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.251165+0000 osd.3 (osd.3) 360 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:44:29.136 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.631613+0000 osd.4 (osd.4) 385 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:44:29.136 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.633073+0000 osd.4 (osd.4) 386 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:44:29.137 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:29.158 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:29.160 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:29.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:29.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:29.291 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:29.312 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:29.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:29.413 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:29.442 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:29.463 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:29.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:29.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:29.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 -- 172.21.15.59:0/3374511330 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fe510 msgr2=0x7fca600f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3374511330 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fe510 0x7fca600f7d40 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 rx=0x7fca4c004870 tx=0x7fca4c009cb0).stop 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 -- 172.21.15.59:0/3374511330 shutdown_connections 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3374511330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca600f8420 0x7fca60104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3374511330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca600fdbb0 0x7fca600fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3374511330 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fe510 0x7fca600f7d40 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 -- 172.21.15.59:0/3374511330 >> 172.21.15.59:0/3374511330 conn(0x7fca600f3800 msgr2=0x7fca600f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 -- 172.21.15.59:0/3374511330 shutdown_connections 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 -- 172.21.15.59:0/3374511330 wait complete. 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 librados: starting msgr at 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 librados: starting objecter 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 Processor -- start 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 -- start start 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 librados: setting wanted keys 2024-02-18T00:44:29.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.397+0000 7fca66c2e700 1 librados: calling monclient init 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca66c2e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca600f8420 0x7fca6010e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca66c2e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fdbb0 0x7fca6010e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca66c2e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca600fe510 0x7fca60108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca66c2e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60108720 con 0x7fca600fe510 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca66c2e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca60108890 con 0x7fca600f8420 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca66c2e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca60108a50 con 0x7fca600fdbb0 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fdbb0 0x7fca6010e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca651cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca600fe510 0x7fca60108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fdbb0 0x7fca6010e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:55478/0 (socket says 172.21.15.59:55478) 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 -- 172.21.15.59:0/3577181501 learned_addr learned my addr 172.21.15.59:0/3577181501 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca600f8420 0x7fca6010e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 -- 172.21.15.59:0/3577181501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca600f8420 msgr2=0x7fca6010e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 --2- 172.21.15.59:0/3577181501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca600f8420 0x7fca6010e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 -- 172.21.15.59:0/3577181501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca600fe510 msgr2=0x7fca60108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 --2- 172.21.15.59:0/3577181501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca600fe510 0x7fca60108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 -- 172.21.15.59:0/3577181501 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca54005040 con 0x7fca600fdbb0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca651cb700 1 --2- 172.21.15.59:0/3577181501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca600fe510 0x7fca60108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5ffff700 1 --2- 172.21.15.59:0/3577181501 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fdbb0 0x7fca6010e580 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 rx=0x7fca4c009eb0 tx=0x7fca4c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5dffb700 10 client.?.objecter ms_handle_connect 0x7fca600fdbb0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5dffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.398+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00e070 con 0x7fca600fdbb0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.399+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4c013da0 con 0x7fca600fdbb0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.399+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c005040 con 0x7fca600fdbb0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.399+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c019070 con 0x7fca600fdbb0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.399+0000 7fca66c2e700 10 client.41717.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.399+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca60108fc0 con 0x7fca600fdbb0 2024-02-18T00:44:29.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.399+0000 7fca66c2e700 1 librados: init done 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.399+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fca6005ee80 con 0x7fca600fdbb0 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.400+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fca4c01e420 con 0x7fca600fdbb0 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6800/3721891797 conn(0x7fca50062c90 0x7fca50065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6800/3721891797 conn(0x7fca50062c90 0x7fca50065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(720..720 src has 418..720) v4 ==== 8814+0+0 (secure 0 0 0) 0x7fca4c07df70 con 0x7fca600fdbb0 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 osd_map(720..720 src has 418..720) v4 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 3 client.41717.objecter handle_osd_map got epochs [720,720] > 0 2024-02-18T00:44:29.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 3 client.41717.objecter handle_osd_map decoding full epoch 720 2024-02-18T00:44:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:44:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:44:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:44:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:44:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:44:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:44:29.538 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 20 client.41717.objecter dump_active .. 0 homeless 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v720) v1 ==== 54+0+16 (secure 0 0 0) 0x7fca4c080610 con 0x7fca600fdbb0 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca50062c90 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6800/3721891797 conn(0x7fca50062c90 0x7fca50065150 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 rx=0x7fca54009350 tx=0x7fca5400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 10 client.41717.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 1 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6801/3700238977 conn(0x7fca600fee20 0x7fca600ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 1 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 0 1 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60107060 osd=0 4 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 10 client.41717.objecter _send_command 1 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca60109710 con 0x7fca600fee20 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.401+0000 7fca651cb700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6801/3700238977 conn(0x7fca600fee20 0x7fca600ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.402+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca600fee20 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.402+0000 7fca651cb700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6801/3700238977 conn(0x7fca600fee20 0x7fca600ff200 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.402+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca60109710 con 0x7fca600fee20 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.402+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.402+0000 7fca5dffb700 10 client.41717.objecter _finish_command 1 = system:0 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.402+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60107060 osd=0 4 2024-02-18T00:44:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.402+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 0 1 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 10 client.41717.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 2 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 2 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60107060 osd=0 3 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 0 2 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60107060 osd=0 4 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 10 client.41717.objecter _send_command 2 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.417+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca600273c0 con 0x7fca600fee20 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca600273c0 con 0x7fca600fee20 2024-02-18T00:44:29.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca5dffb700 10 client.41717.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60107060 osd=0 4 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 0 2 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 10 client.41717.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 3 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6805/26279110 conn(0x7fca600598f0 0x7fca60059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 3 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 1 3 2024-02-18T00:44:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60028710 osd=1 4 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 10 client.41717.objecter _send_command 3 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca6011b640 con 0x7fca600598f0 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.419+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6805/26279110 conn(0x7fca600598f0 0x7fca60059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.420+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca600598f0 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.420+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6805/26279110 conn(0x7fca600598f0 0x7fca60059cf0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.420+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca6011b640 con 0x7fca600598f0 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.420+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.420+0000 7fca5dffb700 10 client.41717.objecter _finish_command 3 = system:0 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.420+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60028710 osd=1 4 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.420+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 1 3 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 10 client.41717.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 4 2024-02-18T00:44:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 4 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60028710 osd=1 3 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 1 4 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60028710 osd=1 4 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 10 client.41717.objecter _send_command 4 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.435+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca60059360 con 0x7fca600598f0 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.436+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca60059360 con 0x7fca600598f0 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.436+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.436+0000 7fca5dffb700 10 client.41717.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.436+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60028710 osd=1 4 2024-02-18T00:44:29.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.436+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 1 4 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 10 client.41717.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 5 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6809/1153193637 conn(0x7fca6006c090 0x7fca6011aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 5 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 2 5 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60027d60 osd=2 4 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 10 client.41717.objecter _send_command 5 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca6011b3e0 con 0x7fca6006c090 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5ffff700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6809/1153193637 conn(0x7fca6006c090 0x7fca6011aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5ffff700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6809/1153193637 conn(0x7fca6006c090 0x7fca6011aef0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca6006c090 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca6011b3e0 con 0x7fca6006c090 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5dffb700 10 client.41717.objecter _finish_command 5 = system:0 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60027d60 osd=2 4 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.437+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 2 5 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 10 client.41717.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 6 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 6 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60027d60 osd=2 3 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 2 6 2024-02-18T00:44:29.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60027d60 osd=2 4 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 10 client.41717.objecter _send_command 6 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.453+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca6006c4f0 con 0x7fca6006c090 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.454+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca6006c4f0 con 0x7fca6006c090 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.454+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.454+0000 7fca5dffb700 10 client.41717.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.454+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60027d60 osd=2 4 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.454+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 2 6 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 10 client.41717.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 7 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6813/1805700964 conn(0x7fca6011fbc0 0x7fca60121fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 7 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 3 7 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006ce90 osd=3 4 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 10 client.41717.objecter _send_command 7 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca60124c00 con 0x7fca6011fbc0 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca651cb700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6813/1805700964 conn(0x7fca6011fbc0 0x7fca60121fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca6011fbc0 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.455+0000 7fca651cb700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6813/1805700964 conn(0x7fca6011fbc0 0x7fca60121fb0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.456+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca60124c00 con 0x7fca6011fbc0 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.456+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.456+0000 7fca5dffb700 10 client.41717.objecter _finish_command 7 = system:0 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.456+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6006ce90 osd=3 4 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.456+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 3 7 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 10 client.41717.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 8 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 8 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6006ce90 osd=3 3 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 3 8 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006ce90 osd=3 4 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 10 client.41717.objecter _send_command 8 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.470+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca6006c8a0 con 0x7fca6011fbc0 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca6006c8a0 con 0x7fca6011fbc0 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca5dffb700 10 client.41717.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6006ce90 osd=3 4 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 3 8 2024-02-18T00:44:29.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 10 client.41717.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 9 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6800/2044387056 conn(0x7fca6006ca70 0x7fca60123620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 9 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 4 9 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006c8a0 osd=4 4 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 10 client.41717.objecter _send_command 9 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca6006bbf0 con 0x7fca6006ca70 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.472+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6800/2044387056 conn(0x7fca6006ca70 0x7fca60123620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.473+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca6006ca70 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.473+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6800/2044387056 conn(0x7fca6006ca70 0x7fca60123620 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.474+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca6006bbf0 con 0x7fca6006ca70 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.474+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.474+0000 7fca5dffb700 10 client.41717.objecter _finish_command 9 = system:0 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.474+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6006c8a0 osd=4 4 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.474+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 4 9 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 10 client.41717.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 10 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 10 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6006c8a0 osd=4 3 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 4 10 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006c8a0 osd=4 4 2024-02-18T00:44:29.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 10 client.41717.objecter _send_command 10 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.489+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca6006bbf0 con 0x7fca6006ca70 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca6006bbf0 con 0x7fca6006ca70 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca5dffb700 10 client.41717.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6006c8a0 osd=4 4 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 4 10 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 10 client.41717.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 11 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6804/2170432263 conn(0x7fca60124020 0x7fca60124460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 11 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 5 11 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006bbf0 osd=5 4 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 10 client.41717.objecter _send_command 11 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.491+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca60124980 con 0x7fca60124020 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.492+0000 7fca5ffff700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6804/2170432263 conn(0x7fca60124020 0x7fca60124460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.492+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca60124020 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.492+0000 7fca5ffff700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6804/2170432263 conn(0x7fca60124020 0x7fca60124460 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.493+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca60124980 con 0x7fca60124020 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.493+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.493+0000 7fca5dffb700 10 client.41717.objecter _finish_command 11 = system:0 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.493+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6006bbf0 osd=5 4 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.493+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 5 11 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 10 client.41717.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 12 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 12 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6006bbf0 osd=5 3 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 5 12 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006bbf0 osd=5 4 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 10 client.41717.objecter _send_command 12 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.508+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca60124980 con 0x7fca60124020 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca60124980 con 0x7fca60124020 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca5dffb700 10 client.41717.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6006bbf0 osd=5 4 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 5 12 2024-02-18T00:44:29.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 10 client.41717.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 13 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6808/3356009390 conn(0x7fca601292a0 0x7fca60129700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 13 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 6 13 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60124980 osd=6 4 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 10 client.41717.objecter _send_command 13 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca6012b350 con 0x7fca601292a0 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca651cb700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6808/3356009390 conn(0x7fca601292a0 0x7fca60129700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca651cb700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6808/3356009390 conn(0x7fca601292a0 0x7fca60129700 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.510+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca601292a0 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.511+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca6012b350 con 0x7fca601292a0 2024-02-18T00:44:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.511+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.511+0000 7fca5dffb700 10 client.41717.objecter _finish_command 13 = system:0 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.511+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60124980 osd=6 4 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.511+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 6 13 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 10 client.41717.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 14 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 14 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca60124980 osd=6 3 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 6 14 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60124980 osd=6 4 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 10 client.41717.objecter _send_command 14 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.525+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca6012b350 con 0x7fca601292a0 2024-02-18T00:44:29.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca6012b350 con 0x7fca601292a0 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca5dffb700 10 client.41717.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca60124980 osd=6 4 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 6 14 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 10 client.41717.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 15 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6812/53025171 conn(0x7fca6012da40 0x7fca6012de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 15 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 7 15 2024-02-18T00:44:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6012b350 osd=7 4 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 10 client.41717.objecter _send_command 15 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.527+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca6012e520 con 0x7fca6012da40 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.528+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6812/53025171 conn(0x7fca6012da40 0x7fca6012de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.528+0000 7fca649ca700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6812/53025171 conn(0x7fca6012da40 0x7fca6012de80 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.528+0000 7fca5dffb700 10 client.41717.objecter ms_handle_connect 0x7fca6012da40 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.529+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fca6012e520 con 0x7fca6012da40 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.529+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.529+0000 7fca5dffb700 10 client.41717.objecter _finish_command 15 = system:0 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.529+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6012b350 osd=7 4 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.529+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 7 15 2024-02-18T00:44:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 10 client.41717.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign -1 16 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 20 client.41717.objecter _get_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_remove -1 16 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 20 client.41717.objecter get_session s=0x7fca6012b350 osd=7 3 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 15 client.41717.objecter _session_command_op_assign 7 16 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6012b350 osd=7 4 2024-02-18T00:44:29.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 10 client.41717.objecter _send_command 16 2024-02-18T00:44:29.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.545+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fca6012e520 con 0x7fca6012da40 2024-02-18T00:44:29.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca5dffb700 1 -- 172.21.15.59:0/3577181501 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fca6012e520 con 0x7fca6012da40 2024-02-18T00:44:29.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca5dffb700 10 client.41717.objecter ms_dispatch 0x7fca60001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:29.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca5dffb700 10 client.41717.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca5dffb700 20 client.41717.objecter put_session s=0x7fca6012b350 osd=7 4 2024-02-18T00:44:29.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca5dffb700 15 client.41717.objecter _session_command_op_remove 7 16 2024-02-18T00:44:29.560 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:44:29.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 librados: watch_flush enter 2024-02-18T00:44:29.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 librados: watch_flush exit 2024-02-18T00:44:29.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.0 2024-02-18T00:44:29.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6801/3700238977 conn(0x7fca600fee20 msgr2=0x7fca600ff200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6801/3700238977 conn(0x7fca600fee20 0x7fca600ff200 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60107060 osd=0 1 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.1 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6805/26279110 conn(0x7fca600598f0 msgr2=0x7fca60059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6805/26279110 conn(0x7fca600598f0 0x7fca60059cf0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60028710 osd=1 1 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.2 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6809/1153193637 conn(0x7fca6006c090 msgr2=0x7fca6011aef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6809/1153193637 conn(0x7fca6006c090 0x7fca6011aef0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60027d60 osd=2 1 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.3 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6813/1805700964 conn(0x7fca6011fbc0 msgr2=0x7fca60121fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca651cb700 1 -- 172.21.15.59:0/3577181501 reap_dead start 2024-02-18T00:44:29.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6813/1805700964 conn(0x7fca6011fbc0 0x7fca60121fb0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006ce90 osd=3 1 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.4 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6800/2044387056 conn(0x7fca6006ca70 msgr2=0x7fca60123620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6800/2044387056 conn(0x7fca6006ca70 0x7fca60123620 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006c8a0 osd=4 1 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.5 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6804/2170432263 conn(0x7fca60124020 msgr2=0x7fca60124460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6804/2170432263 conn(0x7fca60124020 0x7fca60124460 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6006bbf0 osd=5 1 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.6 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6808/3356009390 conn(0x7fca601292a0 msgr2=0x7fca60129700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6808/3356009390 conn(0x7fca601292a0 0x7fca60129700 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca60124980 osd=6 1 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter close_session for osd.7 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6812/53025171 conn(0x7fca6012da40 msgr2=0x7fca6012de80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.137:6812/53025171 conn(0x7fca6012da40 0x7fca6012de80 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter put_session s=0x7fca6012b350 osd=7 1 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 20 client.41717.objecter shutdown clearing up homeless session... 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.547+0000 7fca66c2e700 10 client.41717.objecter successfully canceled tick 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca651cb700 1 -- 172.21.15.59:0/3577181501 reap_dead start 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6800/3721891797 conn(0x7fca50062c90 msgr2=0x7fca50065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6800/3721891797 conn(0x7fca50062c90 0x7fca50065150 secure :-1 s=READY pgs=3207 cs=0 l=1 rev1=1 rx=0x7fca54009350 tx=0x7fca5400b040).stop 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fdbb0 msgr2=0x7fca6010e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fdbb0 0x7fca6010e580 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 rx=0x7fca4c009eb0 tx=0x7fca4c00b040).stop 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 shutdown_connections 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fca600fdbb0 0x7fca6010e580 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:29.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 --2- 172.21.15.59:0/3577181501 >> v2:172.21.15.59:6800/3721891797 conn(0x7fca50062c90 0x7fca50065150 secure :-1 s=CLOSED pgs=3207 cs=0 l=1 rev1=1 rx=0x7fca54009350 tx=0x7fca5400b040).stop 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 >> 172.21.15.59:0/3577181501 conn(0x7fca600f3800 msgr2=0x7fca600fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 shutdown_connections 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 -- 172.21.15.59:0/3577181501 wait complete. 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:29.548+0000 7fca66c2e700 1 librados: shutdown 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:44:29.567 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:29.594 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:29.612 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:29.615 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:29.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:29.716 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:29.761 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:29.763 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:29.764 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:29.864 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:29.897 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:29.918 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:29.920 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:29.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:30.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:28.838083+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-126"}]': finished 2024-02-18T00:44:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:28.838161+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-18T00:44:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:28.840633+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-126", "mode": "writeback"}]: dispatch 2024-02-18T00:44:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:29.401869+0000 mon.c (mon.2) 570 : audit [DBG] from='client.? 172.21.15.59:0/3577181501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:29.832955+0000 mon.a (mon.0) 3483 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:29.839502+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-126", "mode": "writeback"}]': finished 2024-02-18T00:44:30.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:29.839564+0000 mon.a (mon.0) 3485 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-18T00:44:30.021 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:30.048 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:30.066 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:30.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:30.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:30.170 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:30.197 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:30.215 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:30.217 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:30.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:30.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:28.838083+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-126"}]': finished 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:28.838161+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:28.840633+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-126", "mode": "writeback"}]: dispatch 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:29.401869+0000 mon.c (mon.2) 570 : audit [DBG] from='client.? 172.21.15.59:0/3577181501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:29.832955+0000 mon.a (mon.0) 3483 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:29.839502+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-126", "mode": "writeback"}]': finished 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:29.839564+0000 mon.a (mon.0) 3485 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:28.838083+0000 mon.a (mon.0) 3480 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-126"}]': finished 2024-02-18T00:44:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:28.838161+0000 mon.a (mon.0) 3481 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-18T00:44:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:28.840633+0000 mon.a (mon.0) 3482 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-126", "mode": "writeback"}]: dispatch 2024-02-18T00:44:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:29.401869+0000 mon.c (mon.2) 570 : audit [DBG] from='client.? 172.21.15.59:0/3577181501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:29.832955+0000 mon.a (mon.0) 3483 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:44:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:29.839502+0000 mon.a (mon.0) 3484 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-126", "mode": "writeback"}]': finished 2024-02-18T00:44:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:29.839564+0000 mon.a (mon.0) 3485 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-18T00:44:30.318 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:30.344 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:30.362 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:30.365 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:30.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:30.466 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:30.492 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:30.510 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:30.512 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:30.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:30.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:30.644 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:30.796 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:30.799 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:30.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:30.900 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:30.930 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:30.948 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:30.950 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:30.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:29.904561+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:30.002743+0000 mgr.y (mgr.24454) 625 : cluster [DBG] pgmap v1104: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:30.253677+0000 osd.3 (osd.3) 361 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:30.255194+0000 osd.3 (osd.3) 362 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:29.904561+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:30.002743+0000 mgr.y (mgr.24454) 625 : cluster [DBG] pgmap v1104: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:30.253677+0000 osd.3 (osd.3) 361 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:44:31.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:30.255194+0000 osd.3 (osd.3) 362 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:44:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:31.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:31.078 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:31.097 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:31.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:31.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:31.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:31.231 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:31.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:44:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:29.904561+0000 mon.a (mon.0) 3486 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:30.002743+0000 mgr.y (mgr.24454) 625 : cluster [DBG] pgmap v1104: 214 pgs: 6 creating+peering, 26 unknown, 182 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:30.253677+0000 osd.3 (osd.3) 361 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:44:31.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:30.255194+0000 osd.3 (osd.3) 362 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:44:31.318 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:44:31.318 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:44:31.318 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:44:31.319 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:44:31.319 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:44:31.857 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:44:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:30.738203+0000 osd.5 (osd.5) 305 : cluster [DBG] 6.a scrub starts 2024-02-18T00:44:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:30.739083+0000 osd.5 (osd.5) 306 : cluster [DBG] 6.a scrub ok 2024-02-18T00:44:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:30.859925+0000 mon.a (mon.0) 3487 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:32.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:30.859980+0000 mon.a (mon.0) 3488 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:30.861320+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126"}]: dispatch 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.151707+0000 osd.7 (osd.7) 213 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.152511+0000 osd.7 (osd.7) 214 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.283450+0000 osd.3 (osd.3) 363 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.284907+0000 osd.3 (osd.3) 364 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:30.738203+0000 osd.5 (osd.5) 305 : cluster [DBG] 6.a scrub starts 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:30.739083+0000 osd.5 (osd.5) 306 : cluster [DBG] 6.a scrub ok 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:30.859925+0000 mon.a (mon.0) 3487 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:30.859980+0000 mon.a (mon.0) 3488 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-18T00:44:32.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:30.861320+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126"}]: dispatch 2024-02-18T00:44:32.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.151707+0000 osd.7 (osd.7) 213 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:44:32.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.152511+0000 osd.7 (osd.7) 214 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:44:32.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.283450+0000 osd.3 (osd.3) 363 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:44:32.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.284907+0000 osd.3 (osd.3) 364 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:44:32.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:30.738203+0000 osd.5 (osd.5) 305 : cluster [DBG] 6.a scrub starts 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:30.739083+0000 osd.5 (osd.5) 306 : cluster [DBG] 6.a scrub ok 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:30.859925+0000 mon.a (mon.0) 3487 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:30.859980+0000 mon.a (mon.0) 3488 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:30.861320+0000 mon.a (mon.0) 3489 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126"}]: dispatch 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.151707+0000 osd.7 (osd.7) 213 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.152511+0000 osd.7 (osd.7) 214 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.283450+0000 osd.3 (osd.3) 363 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:44:32.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.284907+0000 osd.3 (osd.3) 364 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:44:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:32.283 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:32.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:32.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:32.417 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:32.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:32] "GET /metrics HTTP/1.1" 200 189316 "" "Prometheus/2.33.4" 2024-02-18T00:44:32.867 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:32.871 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:32.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:32.972 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:33.027 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:33.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.442839+0000 osd.6 (osd.6) 317 : cluster [DBG] 234.2 scrub starts 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.444622+0000 osd.6 (osd.6) 318 : cluster [DBG] 234.2 scrub ok 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:31.565353+0000 mgr.y (mgr.24454) 626 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.856054+0000 mon.a (mon.0) 3490 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:31.868315+0000 mon.a (mon.0) 3491 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126"}]': finished 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:31.868384+0000 mon.a (mon.0) 3492 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.003368+0000 mgr.y (mgr.24454) 627 : cluster [DBG] pgmap v1107: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.281965+0000 osd.3 (osd.3) 365 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:44:33.106 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.283631+0000 osd.3 (osd.3) 366 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.657336+0000 osd.4 (osd.4) 387 : cluster [DBG] 4.22 scrub starts 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.659293+0000 osd.4 (osd.4) 388 : cluster [DBG] 4.22 scrub ok 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.442839+0000 osd.6 (osd.6) 317 : cluster [DBG] 234.2 scrub starts 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.444622+0000 osd.6 (osd.6) 318 : cluster [DBG] 234.2 scrub ok 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:31.565353+0000 mgr.y (mgr.24454) 626 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.856054+0000 mon.a (mon.0) 3490 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:31.868315+0000 mon.a (mon.0) 3491 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126"}]': finished 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:31.868384+0000 mon.a (mon.0) 3492 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.003368+0000 mgr.y (mgr.24454) 627 : cluster [DBG] pgmap v1107: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.281965+0000 osd.3 (osd.3) 365 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:44:33.107 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.283631+0000 osd.3 (osd.3) 366 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:44:33.108 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.657336+0000 osd.4 (osd.4) 387 : cluster [DBG] 4.22 scrub starts 2024-02-18T00:44:33.108 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.659293+0000 osd.4 (osd.4) 388 : cluster [DBG] 4.22 scrub ok 2024-02-18T00:44:33.128 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:33.156 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:33.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.442839+0000 osd.6 (osd.6) 317 : cluster [DBG] 234.2 scrub starts 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.444622+0000 osd.6 (osd.6) 318 : cluster [DBG] 234.2 scrub ok 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:31.565353+0000 mgr.y (mgr.24454) 626 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.856054+0000 mon.a (mon.0) 3490 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:31.868315+0000 mon.a (mon.0) 3491 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-126"}]': finished 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:31.868384+0000 mon.a (mon.0) 3492 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.003368+0000 mgr.y (mgr.24454) 627 : cluster [DBG] pgmap v1107: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.281965+0000 osd.3 (osd.3) 365 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.283631+0000 osd.3 (osd.3) 366 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.657336+0000 osd.4 (osd.4) 387 : cluster [DBG] 4.22 scrub starts 2024-02-18T00:44:33.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.659293+0000 osd.4 (osd.4) 388 : cluster [DBG] 4.22 scrub ok 2024-02-18T00:44:33.507 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:33.510 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:33.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:33.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.1 dump_historic_ops 2024-02-18T00:44:33.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:33.639 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:33.678 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:33.682 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:33.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:33.783 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:33.817 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:34.026 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:34.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:34.127 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:34.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.812180+0000 osd.5 (osd.5) 307 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:44:34.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.815421+0000 osd.5 (osd.5) 308 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:44:34.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:32.875650+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-18T00:44:34.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:33.156349+0000 osd.7 (osd.7) 215 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:44:34.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:33.157361+0000 osd.7 (osd.7) 216 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:44:34.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:33.804800+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-18T00:44:34.157 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:33.810523+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:34.159 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:34.180 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:34.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:34.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:34.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.812180+0000 osd.5 (osd.5) 307 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:44:34.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.815421+0000 osd.5 (osd.5) 308 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:44:34.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:32.875650+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-18T00:44:34.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:33.156349+0000 osd.7 (osd.7) 215 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:44:34.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:33.157361+0000 osd.7 (osd.7) 216 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:33.804800+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:33.810523+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.812180+0000 osd.5 (osd.5) 307 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.815421+0000 osd.5 (osd.5) 308 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:32.875650+0000 mon.a (mon.0) 3493 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:33.156349+0000 osd.7 (osd.7) 215 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:33.157361+0000 osd.7 (osd.7) 216 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:33.804800+0000 mon.a (mon.0) 3494 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-18T00:44:34.196 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:33.810523+0000 mon.a (mon.0) 3495 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:34.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:34.314 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:34.335 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:34.337 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:34.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:34.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:34.666 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:34.668 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:34.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:34.769 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:34.797 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:34.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:44:34.979 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:34.982 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:34.982 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:35.007 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:35.083 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:35.115 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:35.136 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:35.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:35.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:35.201 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:34.003843+0000 mgr.y (mgr.24454) 628 : cluster [DBG] pgmap v1110: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:35.201 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:34.745500+0000 osd.4 (osd.4) 389 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:34.748961+0000 osd.4 (osd.4) 390 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:34.805270+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:34.805404+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:34.809147+0000 mon.a (mon.0) 3498 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:34.003843+0000 mgr.y (mgr.24454) 628 : cluster [DBG] pgmap v1110: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:34.745500+0000 osd.4 (osd.4) 389 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:34.748961+0000 osd.4 (osd.4) 390 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:34.805270+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:34.805404+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-18T00:44:35.202 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:34.809147+0000 mon.a (mon.0) 3498 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:35.241 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:34.003843+0000 mgr.y (mgr.24454) 628 : cluster [DBG] pgmap v1110: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:34.745500+0000 osd.4 (osd.4) 389 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:44:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:34.748961+0000 osd.4 (osd.4) 390 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:44:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:34.805270+0000 mon.a (mon.0) 3496 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:34.805404+0000 mon.a (mon.0) 3497 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-18T00:44:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:34.809147+0000 mon.a (mon.0) 3498 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:35.271 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:35.291 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:35.294 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:35.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:35.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 -- 172.21.15.59:0/4180392389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40fcbb0 msgr2=0x7f8ef40fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 --2- 172.21.15.59:0/4180392389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40fcbb0 0x7f8ef40fcf90 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 rx=0x7f8ef00047b0 tx=0x7f8ef0009cb0).stop 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 -- 172.21.15.59:0/4180392389 shutdown_connections 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 --2- 172.21.15.59:0/4180392389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef40f7400 0x7f8ef4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 --2- 172.21.15.59:0/4180392389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40fcbb0 0x7f8ef40fcf90 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 --2- 172.21.15.59:0/4180392389 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ef40fd510 0x7f8ef40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 -- 172.21.15.59:0/4180392389 >> 172.21.15.59:0/4180392389 conn(0x7f8ef40f2820 msgr2=0x7f8ef40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 -- 172.21.15.59:0/4180392389 shutdown_connections 2024-02-18T00:44:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.409+0000 7f8efb683700 1 -- 172.21.15.59:0/4180392389 wait complete. 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 librados: starting msgr at 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 librados: starting objecter 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 Processor -- start 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 -- start start 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 librados: setting wanted keys 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 librados: calling monclient init 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40f7400 0x7f8ef410e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ef40fcbb0 0x7f8ef410e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef40fd510 0x7f8ef4108410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4108900 con 0x7f8ef40f7400 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4108a70 con 0x7f8ef40fd510 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8efb683700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4108c30 con 0x7f8ef40fcbb0 2024-02-18T00:44:35.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8ef941f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40f7400 0x7f8ef410e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8ef941f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40f7400 0x7f8ef410e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52806/0 (socket says 172.21.15.59:52806) 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8ef941f700 1 -- 172.21.15.59:0/3839886649 learned_addr learned my addr 172.21.15.59:0/3839886649 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.410+0000 7f8ef9c20700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef40fd510 0x7f8ef4108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef941f700 1 -- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ef40fcbb0 msgr2=0x7f8ef410e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef8c1e700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ef40fcbb0 0x7f8ef410e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef941f700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ef40fcbb0 0x7f8ef410e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef941f700 1 -- 172.21.15.59:0/3839886649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef40fd510 msgr2=0x7f8ef4108410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef941f700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef40fd510 0x7f8ef4108410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef941f700 1 -- 172.21.15.59:0/3839886649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee4005040 con 0x7f8ef40f7400 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef941f700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40f7400 0x7f8ef410e220 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 rx=0x7f8ef0009b10 tx=0x7f8ef0009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8eea7fc700 10 client.?.objecter ms_handle_connect 0x7f8ef40f7400 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8eea7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8ef8c1e700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ef40fcbb0 0x7f8ef410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:44:35.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8eea7fc700 1 -- 172.21.15.59:0/3839886649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0018070 con 0x7f8ef40f7400 2024-02-18T00:44:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef0005040 con 0x7f8ef40f7400 2024-02-18T00:44:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8efb683700 10 client.31917.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef41090d0 con 0x7f8ef40f7400 2024-02-18T00:44:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8efb683700 1 librados: init done 2024-02-18T00:44:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8eea7fc700 1 -- 172.21.15.59:0/3839886649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ef0013e70 con 0x7f8ef40f7400 2024-02-18T00:44:35.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.411+0000 7f8eea7fc700 1 -- 172.21.15.59:0/3839886649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef001de40 con 0x7f8ef40f7400 2024-02-18T00:44:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.412+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed8004f50 con 0x7f8ef40f7400 2024-02-18T00:44:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.413+0000 7f8eea7fc700 1 -- 172.21.15.59:0/3839886649 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8ef001d420 con 0x7f8ef40f7400 2024-02-18T00:44:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8eea7fc700 1 --2- 172.21.15.59:0/3839886649 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8ee0062c90 0x7f8ee0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8eea7fc700 1 -- 172.21.15.59:0/3839886649 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(726..726 src has 418..726) v4 ==== 8819+0+0 (secure 0 0 0) 0x7f8ef007e8c0 con 0x7f8ef40f7400 2024-02-18T00:44:35.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8eea7fc700 10 client.31917.objecter ms_dispatch 0x7f8ef4001550 osd_map(726..726 src has 418..726) v4 2024-02-18T00:44:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8eea7fc700 3 client.31917.objecter handle_osd_map got epochs [726,726] > 0 2024-02-18T00:44:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8eea7fc700 3 client.31917.objecter handle_osd_map decoding full epoch 726 2024-02-18T00:44:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8ef8c1e700 1 --2- 172.21.15.59:0/3839886649 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8ee0062c90 0x7f8ee0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8eea7fc700 20 client.31917.objecter dump_active .. 0 homeless 2024-02-18T00:44:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8eea7fc700 10 client.31917.objecter ms_handle_connect 0x7f8ee0062c90 2024-02-18T00:44:35.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.414+0000 7f8ef8c1e700 1 --2- 172.21.15.59:0/3839886649 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8ee0062c90 0x7f8ee0065150 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 rx=0x7f8ee4009f20 tx=0x7f8ee4009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.419+0000 7f8eea7fc700 1 -- 172.21.15.59:0/3839886649 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8ef0051a90 con 0x7f8ef40f7400 2024-02-18T00:44:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:35.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.605+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f8ed8005200 con 0x7f8ef40f7400 2024-02-18T00:44:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.807+0000 7f8eea7fc700 1 -- 172.21.15.59:0/3839886649 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v727) v1 ==== 113+0+0 (secure 0 0 0) 0x7f8ef0051680 con 0x7f8ef40f7400 2024-02-18T00:44:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.809+0000 7f8efb683700 10 librados: watch_flush enter 2024-02-18T00:44:35.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 10 librados: watch_flush exit 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 20 client.31917.objecter shutdown clearing up homeless session... 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 10 client.31917.objecter successfully canceled tick 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8ee0062c90 msgr2=0x7f8ee0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 --2- 172.21.15.59:0/3839886649 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8ee0062c90 0x7f8ee0065150 secure :-1 s=READY pgs=3208 cs=0 l=1 rev1=1 rx=0x7f8ee4009f20 tx=0x7f8ee4009cb0).stop 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40f7400 msgr2=0x7f8ef410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40f7400 0x7f8ef410e220 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 rx=0x7f8ef0009b10 tx=0x7f8ef0009cb0).stop 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 shutdown_connections 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 --2- 172.21.15.59:0/3839886649 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8ee0062c90 0x7f8ee0065150 unknown :-1 s=CLOSED pgs=3208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ef40fd510 0x7f8ef4108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ef40f7400 0x7f8ef410e220 unknown :-1 s=CLOSED pgs=2686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 --2- 172.21.15.59:0/3839886649 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ef40fcbb0 0x7f8ef410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:35.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 >> 172.21.15.59:0/3839886649 conn(0x7f8ef40f2820 msgr2=0x7f8ef40f42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:35.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 shutdown_connections 2024-02-18T00:44:35.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 -- 172.21.15.59:0/3839886649 wait complete. 2024-02-18T00:44:35.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:35.810+0000 7f8efb683700 1 librados: shutdown 2024-02-18T00:44:35.814 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.5 to 1 (10000) 2024-02-18T00:44:36.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:34.575093+0000 osd.0 (osd.0) 312 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:34.576670+0000 osd.0 (osd.0) 313 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.011340+0000 osd.1 (osd.1) 343 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.012790+0000 osd.1 (osd.1) 344 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:35.607379+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? 172.21.15.59:0/3839886649' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:35.808349+0000 mon.a (mon.0) 3500 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:35.808429+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? 172.21.15.59:0/3839886649' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.808489+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-18T00:44:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:35.808926+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:34.575093+0000 osd.0 (osd.0) 312 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:34.576670+0000 osd.0 (osd.0) 313 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.011340+0000 osd.1 (osd.1) 343 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.012790+0000 osd.1 (osd.1) 344 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:35.607379+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? 172.21.15.59:0/3839886649' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:35.808349+0000 mon.a (mon.0) 3500 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:35.808429+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? 172.21.15.59:0/3839886649' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.808489+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-18T00:44:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:35.808926+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:34.575093+0000 osd.0 (osd.0) 312 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:34.576670+0000 osd.0 (osd.0) 313 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.011340+0000 osd.1 (osd.1) 343 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.012790+0000 osd.1 (osd.1) 344 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:35.607379+0000 mon.a (mon.0) 3499 : audit [INF] from='client.? 172.21.15.59:0/3839886649' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:35.808349+0000 mon.a (mon.0) 3500 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:35.808429+0000 mon.a (mon.0) 3501 : audit [INF] from='client.? 172.21.15.59:0/3839886649' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:44:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.808489+0000 mon.a (mon.0) 3502 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-18T00:44:36.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:35.808926+0000 mon.a (mon.0) 3503 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:44:36.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 6 1 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.345291+0000 osd.6 (osd.6) 319 : cluster [DBG] 234.6 scrub starts 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.346667+0000 osd.6 (osd.6) 320 : cluster [DBG] 234.6 scrub ok 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.782750+0000 osd.4 (osd.4) 391 : cluster [DBG] 234.0 scrub starts 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.784133+0000 osd.4 (osd.4) 392 : cluster [DBG] 234.0 scrub ok 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.865970+0000 osd.5 (osd.5) 309 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:35.869828+0000 osd.5 (osd.5) 310 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:36.004660+0000 mgr.y (mgr.24454) 629 : cluster [DBG] pgmap v1113: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:37.064 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:36.808969+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:36.809046+0000 mon.a (mon.0) 3505 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:36.813692+0000 mon.a (mon.0) 3506 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.345291+0000 osd.6 (osd.6) 319 : cluster [DBG] 234.6 scrub starts 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.346667+0000 osd.6 (osd.6) 320 : cluster [DBG] 234.6 scrub ok 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.782750+0000 osd.4 (osd.4) 391 : cluster [DBG] 234.0 scrub starts 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.784133+0000 osd.4 (osd.4) 392 : cluster [DBG] 234.0 scrub ok 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.865970+0000 osd.5 (osd.5) 309 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:35.869828+0000 osd.5 (osd.5) 310 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:36.004660+0000 mgr.y (mgr.24454) 629 : cluster [DBG] pgmap v1113: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:36.808969+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:44:37.065 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:36.809046+0000 mon.a (mon.0) 3505 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-18T00:44:37.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:36.813692+0000 mon.a (mon.0) 3506 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:44:37.081 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:37.085 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:37.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:37.186 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:37.216 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.345291+0000 osd.6 (osd.6) 319 : cluster [DBG] 234.6 scrub starts 2024-02-18T00:44:37.216 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.346667+0000 osd.6 (osd.6) 320 : cluster [DBG] 234.6 scrub ok 2024-02-18T00:44:37.216 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.782750+0000 osd.4 (osd.4) 391 : cluster [DBG] 234.0 scrub starts 2024-02-18T00:44:37.216 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.784133+0000 osd.4 (osd.4) 392 : cluster [DBG] 234.0 scrub ok 2024-02-18T00:44:37.216 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.865970+0000 osd.5 (osd.5) 309 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:44:37.216 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:35.869828+0000 osd.5 (osd.5) 310 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:44:37.217 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:36.004660+0000 mgr.y (mgr.24454) 629 : cluster [DBG] pgmap v1113: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:37.217 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:36.808969+0000 mon.a (mon.0) 3504 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:44:37.217 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:36.809046+0000 mon.a (mon.0) 3505 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-18T00:44:37.217 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:36.813692+0000 mon.a (mon.0) 3506 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:44:37.218 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:37.240 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:37.242 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:37.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:37.344 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:37.374 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:37.395 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:37.397 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:37.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:37.499 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:37.528 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:38.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 -- 172.21.15.59:0/234130609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 msgr2=0x7fb218104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:38.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 --2- 172.21.15.59:0/234130609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb218104a20 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 rx=0x7fb210009c30 tx=0x7fb210004c50).stop 2024-02-18T00:44:38.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 -- 172.21.15.59:0/234130609 shutdown_connections 2024-02-18T00:44:38.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 --2- 172.21.15.59:0/234130609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb218104a20 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 --2- 172.21.15.59:0/234130609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2180fdbb0 0x7fb2180fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 --2- 172.21.15.59:0/234130609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2180fe510 0x7fb2180f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 -- 172.21.15.59:0/234130609 >> 172.21.15.59:0/234130609 conn(0x7fb2180f3800 msgr2=0x7fb2180f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:38.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 -- 172.21.15.59:0/234130609 shutdown_connections 2024-02-18T00:44:38.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.084+0000 7fb220269700 1 -- 172.21.15.59:0/234130609 wait complete. 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 librados: starting msgr at 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 librados: starting objecter 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 Processor -- start 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 -- start start 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 librados: setting wanted keys 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 librados: calling monclient init 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb21810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2180fdbb0 0x7fb21810e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:38.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2180fe510 0x7fb2181080f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181085e0 con 0x7fb2180fe510 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb218108750 con 0x7fb2180fdbb0 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb220269700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb218108910 con 0x7fb2180f8420 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb21e005700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb21810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb21e005700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb21810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49756/0 (socket says 172.21.15.59:49756) 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.085+0000 7fb21e005700 1 -- 172.21.15.59:0/2774156835 learned_addr learned my addr 172.21.15.59:0/2774156835 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21d804700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2180fdbb0 0x7fb21810e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e806700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2180fe510 0x7fb2181080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e005700 1 -- 172.21.15.59:0/2774156835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2180fdbb0 msgr2=0x7fb21810e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:38.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e005700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2180fdbb0 0x7fb21810e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e005700 1 -- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2180fe510 msgr2=0x7fb2181080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e005700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2180fe510 0x7fb2181080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e005700 1 -- 172.21.15.59:0/2774156835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2100097c0 con 0x7fb2180f8420 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e806700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2180fe510 0x7fb2181080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb21e005700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb21810df50 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 rx=0x7fb21400e970 tx=0x7fb21400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb20f779700 10 client.?.objecter ms_handle_connect 0x7fb2180f8420 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb20f779700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb20f779700 1 -- 172.21.15.59:0/2774156835 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140046e0 con 0x7fb2180f8420 2024-02-18T00:44:38.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb20f779700 1 -- 172.21.15.59:0/2774156835 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21400f040 con 0x7fb2180f8420 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb20f779700 1 -- 172.21.15.59:0/2774156835 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140146f0 con 0x7fb2180f8420 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb218108bf0 con 0x7fb2180f8420 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb220269700 10 client.41726.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218108f70 con 0x7fb2180f8420 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.086+0000 7fb220269700 1 librados: init done 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.087+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb218060280 con 0x7fb2180f8420 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.087+0000 7fb20f779700 1 -- 172.21.15.59:0/2774156835 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb2140090d0 con 0x7fb2180f8420 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb20f779700 1 --2- 172.21.15.59:0/2774156835 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb204062c90 0x7fb204065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb20f779700 1 -- 172.21.15.59:0/2774156835 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(729..729 src has 418..729) v4 ==== 8814+0+0 (secure 0 0 0) 0x7fb21407cf20 con 0x7fb2180f8420 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb20f779700 10 client.41726.objecter ms_dispatch 0x7fb218001550 osd_map(729..729 src has 418..729) v4 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb20f779700 3 client.41726.objecter handle_osd_map got epochs [729,729] > 0 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb20f779700 3 client.41726.objecter handle_osd_map decoding full epoch 729 2024-02-18T00:44:38.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb21d804700 1 --2- 172.21.15.59:0/2774156835 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb204062c90 0x7fb204065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:38.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb20f779700 20 client.41726.objecter dump_active .. 0 homeless 2024-02-18T00:44:38.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb20f779700 10 client.41726.objecter ms_handle_connect 0x7fb204062c90 2024-02-18T00:44:38.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.088+0000 7fb21d804700 1 --2- 172.21.15.59:0/2774156835 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb204062c90 0x7fb204065150 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 rx=0x7fb218109890 tx=0x7fb20800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:38.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.092+0000 7fb20f779700 1 -- 172.21.15.59:0/2774156835 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb214050230 con 0x7fb2180f8420 2024-02-18T00:44:38.107 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:36.371238+0000 osd.6 (osd.6) 321 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:36.372307+0000 osd.6 (osd.6) 322 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.106836+0000 osd.1 (osd.1) 345 : cluster [DBG] 2.a scrub starts 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.109790+0000 osd.1 (osd.1) 346 : cluster [DBG] 2.a scrub ok 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.152893+0000 osd.2 (osd.2) 285 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.153779+0000 osd.2 (osd.2) 286 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.327920+0000 osd.3 (osd.3) 367 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.330356+0000 osd.3 (osd.3) 368 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:37.809983+0000 mon.a (mon.0) 3507 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:44:38.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.810049+0000 mon.a (mon.0) 3508 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:37.811782+0000 mon.a (mon.0) 3509 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:36.371238+0000 osd.6 (osd.6) 321 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:36.372307+0000 osd.6 (osd.6) 322 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.106836+0000 osd.1 (osd.1) 345 : cluster [DBG] 2.a scrub starts 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.109790+0000 osd.1 (osd.1) 346 : cluster [DBG] 2.a scrub ok 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.152893+0000 osd.2 (osd.2) 285 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.153779+0000 osd.2 (osd.2) 286 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.327920+0000 osd.3 (osd.3) 367 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.330356+0000 osd.3 (osd.3) 368 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:37.809983+0000 mon.a (mon.0) 3507 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.810049+0000 mon.a (mon.0) 3508 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-18T00:44:38.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:37.811782+0000 mon.a (mon.0) 3509 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:36.371238+0000 osd.6 (osd.6) 321 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:36.372307+0000 osd.6 (osd.6) 322 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.106836+0000 osd.1 (osd.1) 345 : cluster [DBG] 2.a scrub starts 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.109790+0000 osd.1 (osd.1) 346 : cluster [DBG] 2.a scrub ok 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.152893+0000 osd.2 (osd.2) 285 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.153779+0000 osd.2 (osd.2) 286 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.327920+0000 osd.3 (osd.3) 367 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.330356+0000 osd.3 (osd.3) 368 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:37.809983+0000 mon.a (mon.0) 3507 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:44:38.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.810049+0000 mon.a (mon.0) 3508 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-18T00:44:38.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:37.811782+0000 mon.a (mon.0) 3509 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T00:44:38.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.278+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fb2181095d0 con 0x7fb2180f8420 2024-02-18T00:44:38.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.811+0000 7fb20f779700 1 -- 172.21.15.59:0/2774156835 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v730) v1 ==== 101+0+0 (secure 0 0 0) 0x7fb214019070 con 0x7fb2180f8420 2024-02-18T00:44:38.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.813+0000 7fb220269700 10 librados: watch_flush enter 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 10 librados: watch_flush exit 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 20 client.41726.objecter shutdown clearing up homeless session... 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 10 client.41726.objecter successfully canceled tick 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb204062c90 msgr2=0x7fb204065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 --2- 172.21.15.59:0/2774156835 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb204062c90 0x7fb204065150 secure :-1 s=READY pgs=3209 cs=0 l=1 rev1=1 rx=0x7fb218109890 tx=0x7fb20800b040).stop 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 msgr2=0x7fb21810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb21810df50 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 rx=0x7fb21400e970 tx=0x7fb21400b040).stop 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 shutdown_connections 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2180fe510 0x7fb2181080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2180f8420 0x7fb21810df50 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 --2- 172.21.15.59:0/2774156835 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb204062c90 0x7fb204065150 unknown :-1 s=CLOSED pgs=3209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 --2- 172.21.15.59:0/2774156835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2180fdbb0 0x7fb21810e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:38.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 >> 172.21.15.59:0/2774156835 conn(0x7fb2180f3800 msgr2=0x7fb2180f50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:38.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 shutdown_connections 2024-02-18T00:44:38.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 -- 172.21.15.59:0/2774156835 wait complete. 2024-02-18T00:44:38.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:38.814+0000 7fb220269700 1 librados: shutdown 2024-02-18T00:44:38.817 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:44:39.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.522731+0000 osd.0 (osd.0) 314 : cluster [DBG] 25.a scrub starts 2024-02-18T00:44:39.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:37.525704+0000 osd.0 (osd.0) 315 : cluster [DBG] 25.a scrub ok 2024-02-18T00:44:39.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.005055+0000 mgr.y (mgr.24454) 630 : cluster [DBG] pgmap v1116: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:39.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.079690+0000 osd.1 (osd.1) 347 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:44:39.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.082844+0000 osd.1 (osd.1) 348 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:44:39.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:38.279380+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.59:0/2774156835' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:39.171 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:38.279656+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.709236+0000 osd.4 (osd.4) 393 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.710192+0000 osd.4 (osd.4) 394 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.807350+0000 mon.a (mon.0) 3511 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:38.811466+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:38.811538+0000 mon.a (mon.0) 3513 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.811598+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.522731+0000 osd.0 (osd.0) 314 : cluster [DBG] 25.a scrub starts 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:37.525704+0000 osd.0 (osd.0) 315 : cluster [DBG] 25.a scrub ok 2024-02-18T00:44:39.172 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.005055+0000 mgr.y (mgr.24454) 630 : cluster [DBG] pgmap v1116: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.079690+0000 osd.1 (osd.1) 347 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.082844+0000 osd.1 (osd.1) 348 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:38.279380+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.59:0/2774156835' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:38.279656+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.709236+0000 osd.4 (osd.4) 393 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.710192+0000 osd.4 (osd.4) 394 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.807350+0000 mon.a (mon.0) 3511 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:38.811466+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:38.811538+0000 mon.a (mon.0) 3513 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:44:39.173 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.811598+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-18T00:44:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:39.182 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:39.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.522731+0000 osd.0 (osd.0) 314 : cluster [DBG] 25.a scrub starts 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:37.525704+0000 osd.0 (osd.0) 315 : cluster [DBG] 25.a scrub ok 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.005055+0000 mgr.y (mgr.24454) 630 : cluster [DBG] pgmap v1116: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.079690+0000 osd.1 (osd.1) 347 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.082844+0000 osd.1 (osd.1) 348 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:38.279380+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.59:0/2774156835' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:38.279656+0000 mon.a (mon.0) 3510 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.709236+0000 osd.4 (osd.4) 393 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.710192+0000 osd.4 (osd.4) 394 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:44:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.807350+0000 mon.a (mon.0) 3511 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:44:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:38.811466+0000 mon.a (mon.0) 3512 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T00:44:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:38.811538+0000 mon.a (mon.0) 3513 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:44:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.811598+0000 mon.a (mon.0) 3514 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-18T00:44:39.283 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:39.339 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:39.342 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:39.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:39.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:39.475 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:39.498 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:39.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:39.599 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:39.628 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:39.647 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:39.649 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:39.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:39.751 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:39.778 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:39.796 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:39.798 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:39.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:39.899 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:39.935 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:39.989 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:39.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:40.090 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:40.118 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:40.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.347158+0000 osd.6 (osd.6) 323 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.348256+0000 osd.6 (osd.6) 324 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.483186+0000 osd.0 (osd.0) 316 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:38.484283+0000 osd.0 (osd.0) 317 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:39.131293+0000 osd.2 (osd.2) 287 : cluster [DBG] 234.5 scrub starts 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:39.133073+0000 osd.2 (osd.2) 288 : cluster [DBG] 234.5 scrub ok 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:39.695414+0000 osd.4 (osd.4) 395 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:39.696249+0000 osd.4 (osd.4) 396 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:39.829071+0000 mon.a (mon.0) 3515 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:39.829535+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128"}]: dispatch 2024-02-18T00:44:40.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:39.909481+0000 mon.a (mon.0) 3517 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.347158+0000 osd.6 (osd.6) 323 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.348256+0000 osd.6 (osd.6) 324 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.483186+0000 osd.0 (osd.0) 316 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:38.484283+0000 osd.0 (osd.0) 317 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:39.131293+0000 osd.2 (osd.2) 287 : cluster [DBG] 234.5 scrub starts 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:39.133073+0000 osd.2 (osd.2) 288 : cluster [DBG] 234.5 scrub ok 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:39.695414+0000 osd.4 (osd.4) 395 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:39.696249+0000 osd.4 (osd.4) 396 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:39.829071+0000 mon.a (mon.0) 3515 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:39.829535+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128"}]: dispatch 2024-02-18T00:44:40.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:39.909481+0000 mon.a (mon.0) 3517 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.347158+0000 osd.6 (osd.6) 323 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.348256+0000 osd.6 (osd.6) 324 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.483186+0000 osd.0 (osd.0) 316 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:38.484283+0000 osd.0 (osd.0) 317 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:39.131293+0000 osd.2 (osd.2) 287 : cluster [DBG] 234.5 scrub starts 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:39.133073+0000 osd.2 (osd.2) 288 : cluster [DBG] 234.5 scrub ok 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:39.695414+0000 osd.4 (osd.4) 395 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:39.696249+0000 osd.4 (osd.4) 396 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:39.829071+0000 mon.a (mon.0) 3515 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:39.829535+0000 mon.a (mon.0) 3516 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128"}]: dispatch 2024-02-18T00:44:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:39.909481+0000 mon.a (mon.0) 3517 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:40.934 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (11997 ms) 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: 00000012 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12034 ms) 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (5973 ms) 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP (497601 ms total) 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: 2024-02-18T00:44:40.935 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1067 ms) 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10957 ms) 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (17965 ms) 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (29991 ms total) 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-18T00:44:40.936 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7043 ms) 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8042 ms) 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: waiting for scrub... 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: done waiting 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24931 ms) 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10010 ms) 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8021 ms) 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8071 ms) 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-18T00:44:40.937 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (9949 ms) 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7971 ms) 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13005 ms) 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (7994 ms) 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13054 ms) 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7975 ms) 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8060 ms) 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-18T00:44:40.938 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-18T00:44:40.939 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: ok, hit_set contains 248:602f83fe:::foo:head 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:39.929454+0000 mon.a (mon.0) 3518 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128"}]': finished 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:39.929535+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:40.005489+0000 mgr.y (mgr.24454) 631 : cluster [DBG] pgmap v1119: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:40.212658+0000 osd.3 (osd.3) 369 : cluster [DBG] 4.29 deep-scrub starts 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:40.215757+0000 osd.3 (osd.3) 370 : cluster [DBG] 4.29 deep-scrub ok 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:39.929454+0000 mon.a (mon.0) 3518 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128"}]': finished 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:39.929535+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:40.005489+0000 mgr.y (mgr.24454) 631 : cluster [DBG] pgmap v1119: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:41.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:40.212658+0000 osd.3 (osd.3) 369 : cluster [DBG] 4.29 deep-scrub starts 2024-02-18T00:44:41.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:40.215757+0000 osd.3 (osd.3) 370 : cluster [DBG] 4.29 deep-scrub ok 2024-02-18T00:44:41.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:44:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:39.929454+0000 mon.a (mon.0) 3518 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-128"}]': finished 2024-02-18T00:44:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:39.929535+0000 mon.a (mon.0) 3519 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-18T00:44:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:40.005489+0000 mgr.y (mgr.24454) 631 : cluster [DBG] pgmap v1119: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:40.212658+0000 osd.3 (osd.3) 369 : cluster [DBG] 4.29 deep-scrub starts 2024-02-18T00:44:41.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:40.215757+0000 osd.3 (osd.3) 370 : cluster [DBG] 4.29 deep-scrub ok 2024-02-18T00:44:41.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 -- 172.21.15.59:0/2954328312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fe510 msgr2=0x7fb2d00f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/2954328312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fe510 0x7fb2d00f7d40 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 rx=0x7fb2c0004870 tx=0x7fb2c0009cb0).stop 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 -- 172.21.15.59:0/2954328312 shutdown_connections 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/2954328312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d00fdbb0 0x7fb2d00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/2954328312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d00f8420 0x7fb2d0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/2954328312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fe510 0x7fb2d00f7d40 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 -- 172.21.15.59:0/2954328312 >> 172.21.15.59:0/2954328312 conn(0x7fb2d00f3800 msgr2=0x7fb2d00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 -- 172.21.15.59:0/2954328312 shutdown_connections 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 -- 172.21.15.59:0/2954328312 wait complete. 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 librados: starting msgr at 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 librados: starting objecter 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.815+0000 7fb2d75c2700 1 Processor -- start 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 -- start start 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 librados: setting wanted keys 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 librados: calling monclient init 2024-02-18T00:44:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d00f8420 0x7fb2d010e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fdbb0 0x7fb2d010e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d00fe510 0x7fb2d0108210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0108700 con 0x7fb2d00fe510 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0108870 con 0x7fb2d00f8420 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d75c2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0108a30 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fdbb0 0x7fb2d010e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d5b5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d00fe510 0x7fb2d0108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d5b5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d00fe510 0x7fb2d0108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43034/0 (socket says 172.21.15.59:43034) 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fdbb0 0x7fb2d010e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60090/0 (socket says 172.21.15.59:60090) 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d5b5f700 1 -- 172.21.15.59:0/4276509213 learned_addr learned my addr 172.21.15.59:0/4276509213 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:41.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d00f8420 0x7fb2d010e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 -- 172.21.15.59:0/4276509213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d00f8420 msgr2=0x7fb2d010e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 --2- 172.21.15.59:0/4276509213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d00f8420 0x7fb2d010e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 -- 172.21.15.59:0/4276509213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d00fe510 msgr2=0x7fb2d0108210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 --2- 172.21.15.59:0/4276509213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d00fe510 0x7fb2d0108210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 -- 172.21.15.59:0/4276509213 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0005040 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d5b5f700 1 --2- 172.21.15.59:0/4276509213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d00fe510 0x7fb2d0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2d4b5d700 1 --2- 172.21.15.59:0/4276509213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fdbb0 0x7fb2d010e560 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7fb2c0004870 tx=0x7fb2c000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2c67fc700 10 client.?.objecter ms_handle_connect 0x7fb2d00fdbb0 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.816+0000 7fb2c67fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.817+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000e030 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.817+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d0108cb0 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.817+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c00092e0 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.817+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c001d070 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.817+0000 7fb2d75c2700 10 client.41732.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.817+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d006af10 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.817+0000 7fb2d75c2700 1 librados: init done 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.818+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fb2d005ee80 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.818+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb2c0022420 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2bc062c90 0x7fb2bc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(732..732 src has 418..732) v4 ==== 8433+0+0 (secure 0 0 0) 0x7fb2c00828f0 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 osd_map(732..732 src has 418..732) v4 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 3 client.41732.objecter handle_osd_map got epochs [732,732] > 0 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 3 client.41732.objecter handle_osd_map decoding full epoch 732 2024-02-18T00:44:41.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2bc062c90 0x7fb2bc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 20 client.41732.objecter dump_active .. 0 homeless 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v732) v1 ==== 54+0+16 (secure 0 0 0) 0x7fb2c0087220 con 0x7fb2d00fdbb0 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2bc062c90 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2bc062c90 0x7fb2bc065150 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 rx=0x7fb2cc0041d0 tx=0x7fb2cc009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 1 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb2d0028540 0x7fb2d00fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 1 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 0 1 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d01070a0 osd=0 4 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 10 client.41732.objecter _send_command 1 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.819+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d005fa60 con 0x7fb2d0028540 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.820+0000 7fb2d5b5f700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb2d0028540 0x7fb2d00fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.820+0000 7fb2d5b5f700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb2d0028540 0x7fb2d00fe910 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.820+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d0028540 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.821+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d005fa60 con 0x7fb2d0028540 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.821+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.821+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 1 = system:0 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.821+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d01070a0 osd=0 4 2024-02-18T00:44:41.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.821+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 0 1 2024-02-18T00:44:41.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 2 2024-02-18T00:44:41.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 2 2024-02-18T00:44:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d01070a0 osd=0 3 2024-02-18T00:44:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 0 2 2024-02-18T00:44:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d01070a0 osd=0 4 2024-02-18T00:44:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 10 client.41732.objecter _send_command 2 2024-02-18T00:44:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.836+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d005a040 con 0x7fb2d0028540 2024-02-18T00:44:41.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d005a040 con 0x7fb2d0028540 2024-02-18T00:44:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d01070a0 osd=0 4 2024-02-18T00:44:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 0 2 2024-02-18T00:44:41.840 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:44:41.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6805/26279110 conn(0x7fb2d0027b90 0x7fb2d0027f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 1 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00fff80 osd=1 4 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 10 client.41732.objecter _send_command 3 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.838+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d01000f0 con 0x7fb2d0027b90 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6805/26279110 conn(0x7fb2d0027b90 0x7fb2d0027f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6805/26279110 conn(0x7fb2d0027b90 0x7fb2d0027f70 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d0027b90 2024-02-18T00:44:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d01000f0 con 0x7fb2d0027b90 2024-02-18T00:44:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:44:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 3 = system:0 2024-02-18T00:44:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d00fff80 osd=1 4 2024-02-18T00:44:41.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.839+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 1 3 2024-02-18T00:44:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 4 2024-02-18T00:44:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 4 2024-02-18T00:44:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d00fff80 osd=1 3 2024-02-18T00:44:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 1 4 2024-02-18T00:44:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00fff80 osd=1 4 2024-02-18T00:44:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 10 client.41732.objecter _send_command 4 2024-02-18T00:44:41.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.855+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d0027600 con 0x7fb2d0027b90 2024-02-18T00:44:41.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.856+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d0027600 con 0x7fb2d0027b90 2024-02-18T00:44:41.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.856+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.856+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.856+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d00fff80 osd=1 4 2024-02-18T00:44:41.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.856+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 1 4 2024-02-18T00:44:41.858 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:44:41.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 5 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb2d00ffae0 0x7fb2d006c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 5 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 2 5 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0027a10 osd=2 4 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 10 client.41732.objecter _send_command 5 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d006c830 con 0x7fb2d00ffae0 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d4b5d700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb2d00ffae0 0x7fb2d006c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d00ffae0 2024-02-18T00:44:41.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.857+0000 7fb2d4b5d700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb2d00ffae0 0x7fb2d006c2b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.858+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d006c830 con 0x7fb2d00ffae0 2024-02-18T00:44:41.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.858+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:44:41.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.858+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 5 = system:0 2024-02-18T00:44:41.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.858+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d0027a10 osd=2 4 2024-02-18T00:44:41.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.858+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 2 5 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 6 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 6 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d0027a10 osd=2 3 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 2 6 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0027a10 osd=2 4 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 10 client.41732.objecter _send_command 6 2024-02-18T00:44:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.872+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d006cca0 con 0x7fb2d00ffae0 2024-02-18T00:44:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d006cca0 con 0x7fb2d00ffae0 2024-02-18T00:44:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d0027a10 osd=2 4 2024-02-18T00:44:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 2 6 2024-02-18T00:44:41.876 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:44:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 7 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb2d011b550 0x7fb2d011d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 7 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 3 7 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d011aed0 osd=3 4 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 10 client.41732.objecter _send_command 7 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d0120730 con 0x7fb2d011b550 2024-02-18T00:44:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d5b5f700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb2d011b550 0x7fb2d011d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d011b550 2024-02-18T00:44:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.874+0000 7fb2d5b5f700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb2d011b550 0x7fb2d011d9a0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.875+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d0120730 con 0x7fb2d011b550 2024-02-18T00:44:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.875+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:44:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.875+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 7 = system:0 2024-02-18T00:44:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.875+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d011aed0 osd=3 4 2024-02-18T00:44:41.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.875+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 3 7 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 8 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 8 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d011aed0 osd=3 3 2024-02-18T00:44:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 3 8 2024-02-18T00:44:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d011aed0 osd=3 4 2024-02-18T00:44:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 10 client.41732.objecter _send_command 8 2024-02-18T00:44:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.890+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d00ff040 con 0x7fb2d011b550 2024-02-18T00:44:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d00ff040 con 0x7fb2d011b550 2024-02-18T00:44:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d011aed0 osd=3 4 2024-02-18T00:44:41.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 3 8 2024-02-18T00:44:41.893 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 9 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb2d006be10 0x7fb2d011df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 9 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 4 9 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff210 osd=4 4 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 10 client.41732.objecter _send_command 9 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.891+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d00ff380 con 0x7fb2d006be10 2024-02-18T00:44:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.892+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb2d006be10 0x7fb2d011df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.892+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d006be10 2024-02-18T00:44:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.892+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb2d006be10 0x7fb2d011df50 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.893+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d00ff380 con 0x7fb2d006be10 2024-02-18T00:44:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.893+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:44:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.893+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 9 = system:0 2024-02-18T00:44:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.893+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d00ff210 osd=4 4 2024-02-18T00:44:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.893+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 4 9 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 10 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 10 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d00ff210 osd=4 3 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 4 10 2024-02-18T00:44:41.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff210 osd=4 4 2024-02-18T00:44:41.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 10 client.41732.objecter _send_command 10 2024-02-18T00:44:41.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.908+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d00ff380 con 0x7fb2d006be10 2024-02-18T00:44:41.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d00ff380 con 0x7fb2d006be10 2024-02-18T00:44:41.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d00ff210 osd=4 4 2024-02-18T00:44:41.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 4 10 2024-02-18T00:44:41.912 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:44:41.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 11 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb2d011e990 0x7fb2d011edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 11 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 5 11 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff380 osd=5 4 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 10 client.41732.objecter _send_command 11 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d01270d0 con 0x7fb2d011e990 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.910+0000 7fb2d4b5d700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb2d011e990 0x7fb2d011edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.911+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d011e990 2024-02-18T00:44:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.911+0000 7fb2d4b5d700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb2d011e990 0x7fb2d011edb0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.911+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d01270d0 con 0x7fb2d011e990 2024-02-18T00:44:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.911+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:44:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.911+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 11 = system:0 2024-02-18T00:44:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.911+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d00ff380 osd=5 4 2024-02-18T00:44:41.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.911+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 5 11 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 12 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 12 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d00ff380 osd=5 3 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 5 12 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff380 osd=5 4 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 10 client.41732.objecter _send_command 12 2024-02-18T00:44:41.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.926+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d0124970 con 0x7fb2d011e990 2024-02-18T00:44:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.928+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d0124970 con 0x7fb2d011e990 2024-02-18T00:44:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.928+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.928+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.928+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d00ff380 osd=5 4 2024-02-18T00:44:41.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.928+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 5 12 2024-02-18T00:44:41.931 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:44:41.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 13 2024-02-18T00:44:41.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2d0124ff0 0x7fb2d0127460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 13 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 6 13 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0124970 osd=6 4 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 10 client.41732.objecter _send_command 13 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d012a230 con 0x7fb2d0124ff0 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.929+0000 7fb2d5b5f700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2d0124ff0 0x7fb2d0127460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.930+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d0124ff0 2024-02-18T00:44:41.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.930+0000 7fb2d5b5f700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2d0124ff0 0x7fb2d0127460 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.930+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d012a230 con 0x7fb2d0124ff0 2024-02-18T00:44:41.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.930+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:44:41.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.930+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 13 = system:0 2024-02-18T00:44:41.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.930+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d0124970 osd=6 4 2024-02-18T00:44:41.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.930+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 6 13 2024-02-18T00:44:41.933 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:40.277738+0000 osd.6 (osd.6) 325 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:40.278557+0000 osd.6 (osd.6) 326 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:40.933963+0000 mon.a (mon.0) 3520 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:41.341380+0000 osd.7 (osd.7) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:41.341502+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:41.699885+0000 osd.4 (osd.4) 397 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:41.703450+0000 osd.4 (osd.4) 398 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:44:41.935 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:41.820311+0000 mon.c (mon.2) 572 : audit [DBG] from='client.? 172.21.15.59:0/4276509213' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:40.277738+0000 osd.6 (osd.6) 325 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:40.278557+0000 osd.6 (osd.6) 326 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:40.933963+0000 mon.a (mon.0) 3520 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:41.341380+0000 osd.7 (osd.7) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:41.341502+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:41.699885+0000 osd.4 (osd.4) 397 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:41.703450+0000 osd.4 (osd.4) 398 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:44:41.936 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:41.820311+0000 mon.c (mon.2) 572 : audit [DBG] from='client.? 172.21.15.59:0/4276509213' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 14 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 14 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d0124970 osd=6 3 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 6 14 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0124970 osd=6 4 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 10 client.41732.objecter _send_command 14 2024-02-18T00:44:41.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.953+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d012a230 con 0x7fb2d0124ff0 2024-02-18T00:44:41.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d012a230 con 0x7fb2d0124ff0 2024-02-18T00:44:41.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d0124970 osd=6 4 2024-02-18T00:44:41.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 6 14 2024-02-18T00:44:41.957 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:44:41.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:41.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 15 2024-02-18T00:44:41.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6812/53025171 conn(0x7fb2d0129530 0x7fb2d0129990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:41.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 15 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 7 15 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d012a230 osd=7 4 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 10 client.41732.objecter _send_command 15 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.955+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d012a000 con 0x7fb2d0129530 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.956+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6812/53025171 conn(0x7fb2d0129530 0x7fb2d0129990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.956+0000 7fb2c67fc700 10 client.41732.objecter ms_handle_connect 0x7fb2d0129530 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.956+0000 7fb2d535e700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6812/53025171 conn(0x7fb2d0129530 0x7fb2d0129990 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.957+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2d012a000 con 0x7fb2d0129530 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.957+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.957+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 15 = system:0 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.957+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d012a230 osd=7 4 2024-02-18T00:44:41.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.957+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 7 15 2024-02-18T00:44:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 10 client.41732.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:44:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign -1 16 2024-02-18T00:44:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 20 client.41732.objecter _get_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_remove -1 16 2024-02-18T00:44:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 20 client.41732.objecter get_session s=0x7fb2d012a230 osd=7 3 2024-02-18T00:44:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 15 client.41732.objecter _session_command_op_assign 7 16 2024-02-18T00:44:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d012a230 osd=7 4 2024-02-18T00:44:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 10 client.41732.objecter _send_command 16 2024-02-18T00:44:41.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.972+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fb2d012a000 con 0x7fb2d0129530 2024-02-18T00:44:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.973+0000 7fb2c67fc700 1 -- 172.21.15.59:0/4276509213 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fb2d012a000 con 0x7fb2d0129530 2024-02-18T00:44:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.973+0000 7fb2c67fc700 10 client.41732.objecter ms_dispatch 0x7fb2d0001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:44:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.973+0000 7fb2c67fc700 10 client.41732.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.973+0000 7fb2c67fc700 20 client.41732.objecter put_session s=0x7fb2d012a230 osd=7 4 2024-02-18T00:44:41.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.973+0000 7fb2c67fc700 15 client.41732.objecter _session_command_op_remove 7 16 2024-02-18T00:44:41.975 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:44:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 librados: watch_flush enter 2024-02-18T00:44:41.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 librados: watch_flush exit 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.0 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb2d0028540 msgr2=0x7fb2d00fe910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6801/3700238977 conn(0x7fb2d0028540 0x7fb2d00fe910 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d01070a0 osd=0 1 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.1 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6805/26279110 conn(0x7fb2d0027b90 msgr2=0x7fb2d0027f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6805/26279110 conn(0x7fb2d0027b90 0x7fb2d0027f70 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00fff80 osd=1 1 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.2 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb2d00ffae0 msgr2=0x7fb2d006c2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb2d00ffae0 0x7fb2d006c2b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0027a10 osd=2 1 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.3 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d5b5f700 1 -- 172.21.15.59:0/4276509213 reap_dead start 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb2d011b550 msgr2=0x7fb2d011d9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6813/1805700964 conn(0x7fb2d011b550 0x7fb2d011d9a0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d011aed0 osd=3 1 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.4 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb2d006be10 msgr2=0x7fb2d011df50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6800/2044387056 conn(0x7fb2d006be10 0x7fb2d011df50 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff210 osd=4 1 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.5 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb2d011e990 msgr2=0x7fb2d011edb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6804/2170432263 conn(0x7fb2d011e990 0x7fb2d011edb0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d00ff380 osd=5 1 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.6 2024-02-18T00:44:41.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2d0124ff0 msgr2=0x7fb2d0127460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2d0124ff0 0x7fb2d0127460 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d0124970 osd=6 1 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter close_session for osd.7 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6812/53025171 conn(0x7fb2d0129530 msgr2=0x7fb2d0129990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.137:6812/53025171 conn(0x7fb2d0129530 0x7fb2d0129990 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter put_session s=0x7fb2d012a230 osd=7 1 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 20 client.41732.objecter shutdown clearing up homeless session... 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 10 client.41732.objecter successfully canceled tick 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2bc062c90 msgr2=0x7fb2bc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2bc062c90 0x7fb2bc065150 secure :-1 s=READY pgs=3218 cs=0 l=1 rev1=1 rx=0x7fb2cc0041d0 tx=0x7fb2cc009040).stop 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d5b5f700 1 -- 172.21.15.59:0/4276509213 reap_dead start 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fdbb0 msgr2=0x7fb2d010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:41.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fdbb0 0x7fb2d010e560 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 rx=0x7fb2c0004870 tx=0x7fb2c000b040).stop 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d5b5f700 1 -- 172.21.15.59:0/4276509213 reap_dead start 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 shutdown_connections 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 --2- 172.21.15.59:0/4276509213 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2d00fdbb0 0x7fb2d010e560 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 >> 172.21.15.59:0/4276509213 conn(0x7fb2d00f3800 msgr2=0x7fb2d00fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 shutdown_connections 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 -- 172.21.15.59:0/4276509213 wait complete. 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:41.974+0000 7fb2d75c2700 1 librados: shutdown 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.980 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.981 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.981 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:44:41.981 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:44:42.123 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:44:42.123 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:44:42.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:40.277738+0000 osd.6 (osd.6) 325 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:44:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:40.278557+0000 osd.6 (osd.6) 326 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:44:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:40.933963+0000 mon.a (mon.0) 3520 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-18T00:44:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:41.341380+0000 osd.7 (osd.7) 217 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:41.341502+0000 osd.7 (osd.7) 218 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:41.699885+0000 osd.4 (osd.4) 397 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:44:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:41.703450+0000 osd.4 (osd.4) 398 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:44:42.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:41.820311+0000 mon.c (mon.2) 572 : audit [DBG] from='client.? 172.21.15.59:0/4276509213' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:42.478 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:42] "GET /metrics HTTP/1.1" 200 191859 "" "Prometheus/2.33.4" 2024-02-18T00:44:42.973 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:42.976 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:42.976 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:43.077 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:43.110 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:43.183 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:43.185 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:43.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:43.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:43.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:41.576207+0000 mgr.y (mgr.24454) 632 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:41.708967+0000 osd.5 (osd.5) 311 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:41.710905+0000 osd.5 (osd.5) 312 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:41.940630+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:41.942658+0000 mon.a (mon.0) 3522 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:42.005930+0000 mgr.y (mgr.24454) 633 : cluster [DBG] pgmap v1122: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:42.125926+0000 osd.2 (osd.2) 289 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:42.127144+0000 osd.2 (osd.2) 290 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:41.576207+0000 mgr.y (mgr.24454) 632 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:43.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:41.708967+0000 osd.5 (osd.5) 311 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:41.710905+0000 osd.5 (osd.5) 312 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:41.940630+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:41.942658+0000 mon.a (mon.0) 3522 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:42.005930+0000 mgr.y (mgr.24454) 633 : cluster [DBG] pgmap v1122: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:42.125926+0000 osd.2 (osd.2) 289 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:42.127144+0000 osd.2 (osd.2) 290 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:41.576207+0000 mgr.y (mgr.24454) 632 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:41.708967+0000 osd.5 (osd.5) 311 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:41.710905+0000 osd.5 (osd.5) 312 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:44:43.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:41.940630+0000 mon.a (mon.0) 3521 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-18T00:44:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:41.942658+0000 mon.a (mon.0) 3522 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:44:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:42.005930+0000 mgr.y (mgr.24454) 633 : cluster [DBG] pgmap v1122: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:42.125926+0000 osd.2 (osd.2) 289 : cluster [DBG] 6.14 scrub starts 2024-02-18T00:44:43.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:42.127144+0000 osd.2 (osd.2) 290 : cluster [DBG] 6.14 scrub ok 2024-02-18T00:44:43.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:43.320 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:43.359 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:43.368 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:43.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:43.469 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:43.501 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:43.523 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:43.525 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:43.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:43.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:44:43.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:43.682 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:43.703 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:43.709 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:43.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:43.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_ops_in_flight 2024-02-18T00:44:43.810 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:43.843 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:43.863 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:43.865 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:43.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:43.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:43.995 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:44.022 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:44.025 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:44.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:44.126 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:44:44.152 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:42.309071+0000 osd.6 (osd.6) 327 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:44:44.152 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:42.310372+0000 osd.6 (osd.6) 328 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:42.949085+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:42.949418+0000 mon.a (mon.0) 3524 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:42.957921+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.110166+0000 osd.1 (osd.1) 349 : cluster [DBG] 2.4 deep-scrub starts 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.113283+0000 osd.1 (osd.1) 350 : cluster [DBG] 2.4 deep-scrub ok 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.233222+0000 osd.3 (osd.3) 371 : cluster [DBG] 4.23 deep-scrub starts 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.235967+0000 osd.3 (osd.3) 372 : cluster [DBG] 4.23 deep-scrub ok 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.295563+0000 osd.7 (osd.7) 219 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.296442+0000 osd.7 (osd.7) 220 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:43.807366+0000 mon.a (mon.0) 3526 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:44.153 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.807445+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-18T00:44:44.154 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:43.810236+0000 mon.a (mon.0) 3528 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:44:44.154 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:44.175 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:44:44.177 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:44.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:44:44.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:42.309071+0000 osd.6 (osd.6) 327 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:42.310372+0000 osd.6 (osd.6) 328 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:42.949085+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:42.949418+0000 mon.a (mon.0) 3524 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:42.957921+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.110166+0000 osd.1 (osd.1) 349 : cluster [DBG] 2.4 deep-scrub starts 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.113283+0000 osd.1 (osd.1) 350 : cluster [DBG] 2.4 deep-scrub ok 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.233222+0000 osd.3 (osd.3) 371 : cluster [DBG] 4.23 deep-scrub starts 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.235967+0000 osd.3 (osd.3) 372 : cluster [DBG] 4.23 deep-scrub ok 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.295563+0000 osd.7 (osd.7) 219 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:44:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.296442+0000 osd.7 (osd.7) 220 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:43.807366+0000 mon.a (mon.0) 3526 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.807445+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:43.810236+0000 mon.a (mon.0) 3528 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:42.309071+0000 osd.6 (osd.6) 327 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:42.310372+0000 osd.6 (osd.6) 328 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:42.949085+0000 mon.a (mon.0) 3523 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:42.949418+0000 mon.a (mon.0) 3524 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:42.957921+0000 mon.a (mon.0) 3525 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.110166+0000 osd.1 (osd.1) 349 : cluster [DBG] 2.4 deep-scrub starts 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.113283+0000 osd.1 (osd.1) 350 : cluster [DBG] 2.4 deep-scrub ok 2024-02-18T00:44:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.233222+0000 osd.3 (osd.3) 371 : cluster [DBG] 4.23 deep-scrub starts 2024-02-18T00:44:44.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.235967+0000 osd.3 (osd.3) 372 : cluster [DBG] 4.23 deep-scrub ok 2024-02-18T00:44:44.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.295563+0000 osd.7 (osd.7) 219 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:44:44.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.296442+0000 osd.7 (osd.7) 220 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:44:44.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:43.807366+0000 mon.a (mon.0) 3526 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:44:44.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.807445+0000 mon.a (mon.0) 3527 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-18T00:44:44.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:43.810236+0000 mon.a (mon.0) 3528 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T00:44:44.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:44.307 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:44.562 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:44.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:44.663 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:44.694 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:44.718 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:44.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:44.819 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:44.867 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:44.870 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:44.870 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:44.971 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:44.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:44:44.999 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:45.017 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:45.020 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:45.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:45.122 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:45.148 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:45.166 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:45.169 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:45.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:45.211 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:45.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.358617+0000 osd.6 (osd.6) 329 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:43.359506+0000 osd.6 (osd.6) 330 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.006326+0000 mgr.y (mgr.24454) 634 : cluster [DBG] pgmap v1125: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.265987+0000 osd.3 (osd.3) 373 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.267173+0000 osd.3 (osd.3) 374 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.752503+0000 osd.4 (osd.4) 399 : cluster [DBG] 5.1c deep-scrub starts 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.755383+0000 osd.4 (osd.4) 400 : cluster [DBG] 5.1c deep-scrub ok 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:44.808373+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_count","val": "3"}]': finished 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.808444+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-18T00:44:45.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:44.810868+0000 mon.a (mon.0) 3531 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:44:45.213 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.358617+0000 osd.6 (osd.6) 329 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:44:45.213 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:43.359506+0000 osd.6 (osd.6) 330 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:44:45.213 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.006326+0000 mgr.y (mgr.24454) 634 : cluster [DBG] pgmap v1125: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:45.213 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.265987+0000 osd.3 (osd.3) 373 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:44:45.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.267173+0000 osd.3 (osd.3) 374 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:44:45.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.752503+0000 osd.4 (osd.4) 399 : cluster [DBG] 5.1c deep-scrub starts 2024-02-18T00:44:45.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.755383+0000 osd.4 (osd.4) 400 : cluster [DBG] 5.1c deep-scrub ok 2024-02-18T00:44:45.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:44.808373+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_count","val": "3"}]': finished 2024-02-18T00:44:45.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.808444+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-18T00:44:45.214 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:44.810868+0000 mon.a (mon.0) 3531 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:44:45.214 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:45.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.358617+0000 osd.6 (osd.6) 329 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:43.359506+0000 osd.6 (osd.6) 330 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.006326+0000 mgr.y (mgr.24454) 634 : cluster [DBG] pgmap v1125: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.265987+0000 osd.3 (osd.3) 373 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.267173+0000 osd.3 (osd.3) 374 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.752503+0000 osd.4 (osd.4) 399 : cluster [DBG] 5.1c deep-scrub starts 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.755383+0000 osd.4 (osd.4) 400 : cluster [DBG] 5.1c deep-scrub ok 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:44.808373+0000 mon.a (mon.0) 3529 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_count","val": "3"}]': finished 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.808444+0000 mon.a (mon.0) 3530 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-18T00:44:45.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:44.810868+0000 mon.a (mon.0) 3531 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T00:44:45.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:45.297 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.468586+0000 osd.0 (osd.0) 318 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:44.469408+0000 osd.0 (osd.0) 319 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:45.068934+0000 osd.1 (osd.1) 351 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:45.071831+0000 osd.1 (osd.1) 352 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:45.264230+0000 osd.7 (osd.7) 221 : cluster [DBG] 5.9 deep-scrub starts 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:45.265966+0000 osd.7 (osd.7) 222 : cluster [DBG] 5.9 deep-scrub ok 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:45.810513+0000 mon.a (mon.0) 3532 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_period","val": "3"}]': finished 2024-02-18T00:44:46.068 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:45.810582+0000 mon.a (mon.0) 3533 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:45.811401+0000 mon.a (mon.0) 3534 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.468586+0000 osd.0 (osd.0) 318 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:44.469408+0000 osd.0 (osd.0) 319 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:45.068934+0000 osd.1 (osd.1) 351 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:45.071831+0000 osd.1 (osd.1) 352 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:45.264230+0000 osd.7 (osd.7) 221 : cluster [DBG] 5.9 deep-scrub starts 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:45.265966+0000 osd.7 (osd.7) 222 : cluster [DBG] 5.9 deep-scrub ok 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:45.810513+0000 mon.a (mon.0) 3532 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_period","val": "3"}]': finished 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:45.810582+0000 mon.a (mon.0) 3533 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-18T00:44:46.069 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:45.811401+0000 mon.a (mon.0) 3534 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:44:46.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.468586+0000 osd.0 (osd.0) 318 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:44:46.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:44.469408+0000 osd.0 (osd.0) 319 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:44:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:45.068934+0000 osd.1 (osd.1) 351 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:44:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:45.071831+0000 osd.1 (osd.1) 352 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:44:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:45.264230+0000 osd.7 (osd.7) 221 : cluster [DBG] 5.9 deep-scrub starts 2024-02-18T00:44:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:45.265966+0000 osd.7 (osd.7) 222 : cluster [DBG] 5.9 deep-scrub ok 2024-02-18T00:44:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:45.810513+0000 mon.a (mon.0) 3532 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_period","val": "3"}]': finished 2024-02-18T00:44:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:45.810582+0000 mon.a (mon.0) 3533 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-18T00:44:46.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:45.811401+0000 mon.a (mon.0) 3534 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:44:46.873 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:47.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:45.663086+0000 osd.5 (osd.5) 313 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:45.665977+0000 osd.5 (osd.5) 314 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.006954+0000 mgr.y (mgr.24454) 635 : cluster [DBG] pgmap v1128: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.281955+0000 osd.3 (osd.3) 375 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.283664+0000 osd.3 (osd.3) 376 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:46.811622+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.811702+0000 mon.a (mon.0) 3536 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:46.814203+0000 mon.a (mon.0) 3537 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:45.663086+0000 osd.5 (osd.5) 313 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:45.665977+0000 osd.5 (osd.5) 314 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:44:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.006954+0000 mgr.y (mgr.24454) 635 : cluster [DBG] pgmap v1128: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.281955+0000 osd.3 (osd.3) 375 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.283664+0000 osd.3 (osd.3) 376 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:46.811622+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.811702+0000 mon.a (mon.0) 3536 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:46.814203+0000 mon.a (mon.0) 3537 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:45.663086+0000 osd.5 (osd.5) 313 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:45.665977+0000 osd.5 (osd.5) 314 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:44:47.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.006954+0000 mgr.y (mgr.24454) 635 : cluster [DBG] pgmap v1128: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.281955+0000 osd.3 (osd.3) 375 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:44:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.283664+0000 osd.3 (osd.3) 376 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:44:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:46.811622+0000 mon.a (mon.0) 3535 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:44:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.811702+0000 mon.a (mon.0) 3536 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-18T00:44:47.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:46.814203+0000 mon.a (mon.0) 3537 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T00:44:47.548 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:47.551 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:47.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:47.653 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:47.684 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:47.705 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:47.707 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:47.708 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:47.809 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:47.851 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:48.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.439324+0000 osd.6 (osd.6) 331 : cluster [DBG] 4.a scrub starts 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.441501+0000 osd.6 (osd.6) 332 : cluster [DBG] 4.a scrub ok 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.487018+0000 osd.0 (osd.0) 320 : cluster [DBG] 5.1e deep-scrub starts 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.487951+0000 osd.0 (osd.0) 321 : cluster [DBG] 5.1e deep-scrub ok 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.633526+0000 osd.5 (osd.5) 315 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:46.634719+0000 osd.5 (osd.5) 316 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.199272+0000 osd.2 (osd.2) 291 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.201289+0000 osd.2 (osd.2) 292 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:44:48.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.738311+0000 osd.4 (osd.4) 401 : cluster [DBG] 2.f scrub starts 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.741782+0000 osd.4 (osd.4) 402 : cluster [DBG] 2.f scrub ok 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:47.814428+0000 mon.a (mon.0) 3538 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.814518+0000 mon.a (mon.0) 3539 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.439324+0000 osd.6 (osd.6) 331 : cluster [DBG] 4.a scrub starts 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.441501+0000 osd.6 (osd.6) 332 : cluster [DBG] 4.a scrub ok 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.487018+0000 osd.0 (osd.0) 320 : cluster [DBG] 5.1e deep-scrub starts 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.487951+0000 osd.0 (osd.0) 321 : cluster [DBG] 5.1e deep-scrub ok 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.633526+0000 osd.5 (osd.5) 315 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:46.634719+0000 osd.5 (osd.5) 316 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.199272+0000 osd.2 (osd.2) 291 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.201289+0000 osd.2 (osd.2) 292 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:44:48.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.738311+0000 osd.4 (osd.4) 401 : cluster [DBG] 2.f scrub starts 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.741782+0000 osd.4 (osd.4) 402 : cluster [DBG] 2.f scrub ok 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:47.814428+0000 mon.a (mon.0) 3538 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.814518+0000 mon.a (mon.0) 3539 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.439324+0000 osd.6 (osd.6) 331 : cluster [DBG] 4.a scrub starts 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.441501+0000 osd.6 (osd.6) 332 : cluster [DBG] 4.a scrub ok 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.487018+0000 osd.0 (osd.0) 320 : cluster [DBG] 5.1e deep-scrub starts 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.487951+0000 osd.0 (osd.0) 321 : cluster [DBG] 5.1e deep-scrub ok 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.633526+0000 osd.5 (osd.5) 315 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:44:48.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:46.634719+0000 osd.5 (osd.5) 316 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:44:48.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.199272+0000 osd.2 (osd.2) 291 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:44:48.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.201289+0000 osd.2 (osd.2) 292 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:44:48.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.738311+0000 osd.4 (osd.4) 401 : cluster [DBG] 2.f scrub starts 2024-02-18T00:44:48.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.741782+0000 osd.4 (osd.4) 402 : cluster [DBG] 2.f scrub ok 2024-02-18T00:44:48.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:47.814428+0000 mon.a (mon.0) 3538 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T00:44:48.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.814518+0000 mon.a (mon.0) 3539 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-18T00:44:48.301 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:48.304 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:48.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:48.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:48.438 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:48.638 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:48.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:48.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:48.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:48.775 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:48.972 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.425580+0000 osd.6 (osd.6) 333 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.426809+0000 osd.6 (osd.6) 334 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.647967+0000 osd.5 (osd.5) 317 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.651419+0000 osd.5 (osd.5) 318 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.965884+0000 osd.1 (osd.1) 353 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:47.967454+0000 osd.1 (osd.1) 354 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:48.007574+0000 mgr.y (mgr.24454) 636 : cluster [DBG] pgmap v1131: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.425580+0000 osd.6 (osd.6) 333 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.426809+0000 osd.6 (osd.6) 334 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.647967+0000 osd.5 (osd.5) 317 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:44:48.973 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.651419+0000 osd.5 (osd.5) 318 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:44:48.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.965884+0000 osd.1 (osd.1) 353 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:44:48.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:47.967454+0000 osd.1 (osd.1) 354 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:44:48.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:48.007574+0000 mgr.y (mgr.24454) 636 : cluster [DBG] pgmap v1131: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 -- 172.21.15.59:0/3396203945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d78102210 msgr2=0x7f6d78104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 --2- 172.21.15.59:0/3396203945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d78102210 0x7f6d78104600 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 rx=0x7f6d70009be0 tx=0x7f6d70004d10).stop 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 -- 172.21.15.59:0/3396203945 shutdown_connections 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 --2- 172.21.15.59:0/3396203945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d780f8cb0 0x7f6d780f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 --2- 172.21.15.59:0/3396203945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d78102210 0x7f6d78104600 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 --2- 172.21.15.59:0/3396203945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d780f9610 0x7f6d78101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 -- 172.21.15.59:0/3396203945 >> 172.21.15.59:0/3396203945 conn(0x7f6d780f2820 msgr2=0x7f6d780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:49.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 -- 172.21.15.59:0/3396203945 shutdown_connections 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.018+0000 7f6d7f654700 1 -- 172.21.15.59:0/3396203945 wait complete. 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 librados: starting msgr at 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 librados: starting objecter 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 Processor -- start 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 -- start start 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 librados: setting wanted keys 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 librados: calling monclient init 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d780f8cb0 0x7f6d780f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d780f9610 0x7f6d780f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78102210 0x7f6d780f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d781164a0 con 0x7f6d780f9610 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d78116610 con 0x7f6d78102210 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7f654700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d781167d0 con 0x7f6d780f8cb0 2024-02-18T00:44:49.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7d3f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d780f8cb0 0x7f6d780f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7d3f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d780f8cb0 0x7f6d780f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:60136/0 (socket says 172.21.15.59:60136) 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7d3f0700 1 -- 172.21.15.59:0/2173431162 learned_addr learned my addr 172.21.15.59:0/2173431162 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7cbef700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d780f9610 0x7f6d780f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.019+0000 7f6d7dbf1700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78102210 0x7f6d780f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7cbef700 1 -- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d780f8cb0 msgr2=0x7f6d780f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7cbef700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d780f8cb0 0x7f6d780f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7cbef700 1 -- 172.21.15.59:0/2173431162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78102210 msgr2=0x7f6d780f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7cbef700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78102210 0x7f6d780f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7cbef700 1 -- 172.21.15.59:0/2173431162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c005040 con 0x7f6d780f9610 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7cbef700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d780f9610 0x7f6d780f7150 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7f6d6c004920 tx=0x7f6d6c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d6a7fc700 10 client.?.objecter ms_handle_connect 0x7f6d780f9610 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d6a7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:49.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d6a7fc700 1 -- 172.21.15.59:0/2173431162 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00e030 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d700097c0 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d6a7fc700 1 -- 172.21.15.59:0/2173431162 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6c00f040 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7f654700 10 client.31941.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d6a7fc700 1 -- 172.21.15.59:0/2173431162 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c014850 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d78116e60 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.020+0000 7f6d7f654700 1 librados: init done 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.021+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7805fa60 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.021+0000 7f6d6a7fc700 1 -- 172.21.15.59:0/2173431162 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6d6c010500 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d6a7fc700 1 --2- 172.21.15.59:0/2173431162 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6d64062c90 0x7f6d64065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d6a7fc700 1 -- 172.21.15.59:0/2173431162 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(739..739 src has 418..739) v4 ==== 8840+0+0 (secure 0 0 0) 0x7f6d6c07d600 con 0x7f6d780f9610 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d6a7fc700 10 client.31941.objecter ms_dispatch 0x7f6d78001550 osd_map(739..739 src has 418..739) v4 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d6a7fc700 3 client.31941.objecter handle_osd_map got epochs [739,739] > 0 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d6a7fc700 3 client.31941.objecter handle_osd_map decoding full epoch 739 2024-02-18T00:44:49.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d7d3f0700 1 --2- 172.21.15.59:0/2173431162 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6d64062c90 0x7f6d64065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d6a7fc700 20 client.31941.objecter dump_active .. 0 homeless 2024-02-18T00:44:49.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d6a7fc700 10 client.31941.objecter ms_handle_connect 0x7f6d64062c90 2024-02-18T00:44:49.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.022+0000 7f6d7d3f0700 1 --2- 172.21.15.59:0/2173431162 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6d64062c90 0x7f6d64065150 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 rx=0x7f6d74009fd0 tx=0x7f6d7400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:49.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.026+0000 7f6d6a7fc700 1 -- 172.21.15.59:0/2173431162 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6d6c0507c0 con 0x7f6d780f9610 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/933413305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 msgr2=0x7fa0e4103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/933413305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 0x7fa0e4103a20 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 rx=0x7fa0dc009be0 tx=0x7fa0dc004d10).stop 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/933413305 shutdown_connections 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/933413305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0e40fcbb0 0x7fa0e40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/933413305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 0x7fa0e4103a20 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/933413305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fd510 0x7fa0e40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/933413305 >> 172.21.15.59:0/933413305 conn(0x7fa0e40f2820 msgr2=0x7fa0e40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:49.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/933413305 shutdown_connections 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.918+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/933413305 wait complete. 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 librados: starting msgr at 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 librados: starting objecter 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 Processor -- start 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 -- start start 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 librados: setting wanted keys 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 librados: calling monclient init 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 0x7fa0e410cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fcbb0 0x7fa0e410d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0e40fd510 0x7fa0e4106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e4107470 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e41075e0 con 0x7fa0e40fd510 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0ec4c5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e41077a0 con 0x7fa0e40f7400 2024-02-18T00:44:49.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0e9a60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fcbb0 0x7fa0e410d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.919+0000 7fa0e9a60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fcbb0 0x7fa0e410d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43066/0 (socket says 172.21.15.59:43066) 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0e9a60700 1 -- 172.21.15.59:0/569529675 learned_addr learned my addr 172.21.15.59:0/569529675 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0eaa62700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0e40fd510 0x7fa0e4106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0e9a60700 1 -- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 msgr2=0x7fa0e410cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0ea261700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 0x7fa0e410cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0e9a60700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 0x7fa0e410cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0e9a60700 1 -- 172.21.15.59:0/569529675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0e40fd510 msgr2=0x7fa0e4106f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0e9a60700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0e40fd510 0x7fa0e4106f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0e9a60700 1 -- 172.21.15.59:0/569529675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d4005040 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0eaa62700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0e40fd510 0x7fa0e4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0ea261700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 0x7fa0e410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0e9a60700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fcbb0 0x7fa0e410d2d0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7fa0d40048f0 tx=0x7fa0d4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:49.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0db7fe700 10 client.?.objecter ms_handle_connect 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0db7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0db7fe700 1 -- 172.21.15.59:0/569529675 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4007b00 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc0097c0 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0db7fe700 1 -- 172.21.15.59:0/569529675 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0d4010d40 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0db7fe700 1 -- 172.21.15.59:0/569529675 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4012df0 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0ec4c5700 10 client.31935.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e4107dd0 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.920+0000 7fa0ec4c5700 1 librados: init done 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e405fa60 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0db7fe700 1 -- 172.21.15.59:0/569529675 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa0d4010870 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0db7fe700 1 --2- 172.21.15.59:0/569529675 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0d0062c90 0x7fa0d0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0db7fe700 1 -- 172.21.15.59:0/569529675 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(739..739 src has 418..739) v4 ==== 8840+0+0 (secure 0 0 0) 0x7fa0d407fd20 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0db7fe700 10 client.31935.objecter ms_dispatch 0x7fa0e4001550 osd_map(739..739 src has 418..739) v4 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0db7fe700 3 client.31935.objecter handle_osd_map got epochs [739,739] > 0 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0db7fe700 3 client.31935.objecter handle_osd_map decoding full epoch 739 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0ea261700 1 --2- 172.21.15.59:0/569529675 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0d0062c90 0x7fa0d0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.922+0000 7fa0db7fe700 20 client.31935.objecter dump_active .. 0 homeless 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.923+0000 7fa0db7fe700 10 client.31935.objecter ms_handle_connect 0x7fa0d0062c90 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.923+0000 7fa0ea261700 1 --2- 172.21.15.59:0/569529675 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0d0062c90 0x7fa0d0065150 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 rx=0x7fa0e0004ba0 tx=0x7fa0e0009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:48.927+0000 7fa0db7fe700 1 -- 172.21.15.59:0/569529675 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa0d4052ee0 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.120+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7fa0e4106120 con 0x7fa0e40fcbb0 2024-02-18T00:44:49.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.211+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7f6d78116aa0 con 0x7f6d780f9610 2024-02-18T00:44:49.224 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:49.227 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:49.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:49.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.425580+0000 osd.6 (osd.6) 333 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:44:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.426809+0000 osd.6 (osd.6) 334 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:44:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.647967+0000 osd.5 (osd.5) 317 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:44:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.651419+0000 osd.5 (osd.5) 318 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:44:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.965884+0000 osd.1 (osd.1) 353 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:44:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:47.967454+0000 osd.1 (osd.1) 354 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:44:49.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:48.007574+0000 mgr.y (mgr.24454) 636 : cluster [DBG] pgmap v1131: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:49.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:49.358 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:49.560 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:49.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:49.661 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:49.690 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:49.891 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:49.893 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:49.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:49.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.973+0000 7f6d6a7fc700 1 -- 172.21.15.59:0/2173431162 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v740) v1 ==== 87+0+0 (secure 0 0 0) 0x7f6d6c0503b0 con 0x7f6d780f9610 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 10 librados: watch_flush enter 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 10 librados: watch_flush exit 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 20 client.31941.objecter shutdown clearing up homeless session... 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 10 client.31941.objecter successfully canceled tick 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6d64062c90 msgr2=0x7f6d64065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 --2- 172.21.15.59:0/2173431162 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6d64062c90 0x7f6d64065150 secure :-1 s=READY pgs=3220 cs=0 l=1 rev1=1 rx=0x7f6d74009fd0 tx=0x7f6d7400b040).stop 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d780f9610 msgr2=0x7f6d780f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:49.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d780f9610 0x7f6d780f7150 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7f6d6c004920 tx=0x7f6d6c009cb0).stop 2024-02-18T00:44:49.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 shutdown_connections 2024-02-18T00:44:49.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 --2- 172.21.15.59:0/2173431162 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6d64062c90 0x7f6d64065150 unknown :-1 s=CLOSED pgs=3220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d78102210 0x7f6d780f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6d780f8cb0 0x7f6d780f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 --2- 172.21.15.59:0/2173431162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d780f9610 0x7f6d780f7150 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:49.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 >> 172.21.15.59:0/2173431162 conn(0x7f6d780f2820 msgr2=0x7f6d780f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:49.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.978+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 shutdown_connections 2024-02-18T00:44:49.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.979+0000 7f6d7f654700 1 -- 172.21.15.59:0/2173431162 wait complete. 2024-02-18T00:44:49.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.979+0000 7f6d7f654700 1 librados: shutdown 2024-02-18T00:44:49.982 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:44:49.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:50.024 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.973+0000 7fa0db7fe700 1 -- 172.21.15.59:0/569529675 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v740) v1 ==== 113+0+0 (secure 0 0 0) 0x7fa0d4052ad0 con 0x7fa0e40fcbb0 2024-02-18T00:44:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 10 librados: watch_flush enter 2024-02-18T00:44:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 10 librados: watch_flush exit 2024-02-18T00:44:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 20 client.31935.objecter shutdown clearing up homeless session... 2024-02-18T00:44:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 10 client.31935.objecter successfully canceled tick 2024-02-18T00:44:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0d0062c90 msgr2=0x7fa0d0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/569529675 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0d0062c90 0x7fa0d0065150 secure :-1 s=READY pgs=3219 cs=0 l=1 rev1=1 rx=0x7fa0e0004ba0 tx=0x7fa0e0009040).stop 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fcbb0 msgr2=0x7fa0e410d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fcbb0 0x7fa0e410d2d0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7fa0d40048f0 tx=0x7fa0d4009cb0).stop 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 shutdown_connections 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/569529675 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0d0062c90 0x7fa0d0065150 unknown :-1 s=CLOSED pgs=3219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0e40fd510 0x7fa0e4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0e40f7400 0x7fa0e410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 --2- 172.21.15.59:0/569529675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0e40fcbb0 0x7fa0e410d2d0 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 >> 172.21.15.59:0/569529675 conn(0x7fa0e40f2820 msgr2=0x7fa0e40fab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.975+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 shutdown_connections 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.976+0000 7fa0ec4c5700 1 -- 172.21.15.59:0/569529675 wait complete. 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:49.977+0000 7fa0ec4c5700 1 librados: shutdown 2024-02-18T00:44:50.040 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.6 to 1 (10000) 2024-02-18T00:44:50.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:48.548706+0000 osd.0 (osd.0) 322 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:44:50.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:48.550078+0000 osd.0 (osd.0) 323 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:48.693556+0000 osd.5 (osd.5) 319 : cluster [DBG] 25.f scrub starts 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:48.697953+0000 osd.5 (osd.5) 320 : cluster [DBG] 25.f scrub ok 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:49.121472+0000 mon.a (mon.0) 3540 : audit [INF] from='client.? 172.21.15.59:0/569529675' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:49.213300+0000 mon.a (mon.0) 3541 : audit [INF] from='client.? 172.21.15.59:0/2173431162' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:48.548706+0000 osd.0 (osd.0) 322 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:48.550078+0000 osd.0 (osd.0) 323 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:48.693556+0000 osd.5 (osd.5) 319 : cluster [DBG] 25.f scrub starts 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:48.697953+0000 osd.5 (osd.5) 320 : cluster [DBG] 25.f scrub ok 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:49.121472+0000 mon.a (mon.0) 3540 : audit [INF] from='client.? 172.21.15.59:0/569529675' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:44:50.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:49.213300+0000 mon.a (mon.0) 3541 : audit [INF] from='client.? 172.21.15.59:0/2173431162' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:44:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:48.548706+0000 osd.0 (osd.0) 322 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:44:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:48.550078+0000 osd.0 (osd.0) 323 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:44:50.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:48.693556+0000 osd.5 (osd.5) 319 : cluster [DBG] 25.f scrub starts 2024-02-18T00:44:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:48.697953+0000 osd.5 (osd.5) 320 : cluster [DBG] 25.f scrub ok 2024-02-18T00:44:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:49.121472+0000 mon.a (mon.0) 3540 : audit [INF] from='client.? 172.21.15.59:0/569529675' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:44:50.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:49.213300+0000 mon.a (mon.0) 3541 : audit [INF] from='client.? 172.21.15.59:0/2173431162' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:44:51.182 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:44:51.182 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:44:51.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:49.581772+0000 osd.0 (osd.0) 324 : cluster [DBG] 25.e scrub starts 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:49.583027+0000 osd.0 (osd.0) 325 : cluster [DBG] 25.e scrub ok 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:49.709199+0000 osd.5 (osd.5) 321 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:49.712082+0000 osd.5 (osd.5) 322 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:49.970232+0000 mon.a (mon.0) 3542 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:49.973856+0000 mon.a (mon.0) 3543 : audit [INF] from='client.? 172.21.15.59:0/569529675' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:49.973918+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? 172.21.15.59:0/2173431162' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:49.973949+0000 mon.a (mon.0) 3545 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:50.007938+0000 mgr.y (mgr.24454) 637 : cluster [DBG] pgmap v1133: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:51.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:50.316251+0000 osd.3 (osd.3) 377 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:49.581772+0000 osd.0 (osd.0) 324 : cluster [DBG] 25.e scrub starts 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:49.583027+0000 osd.0 (osd.0) 325 : cluster [DBG] 25.e scrub ok 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:49.709199+0000 osd.5 (osd.5) 321 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:49.712082+0000 osd.5 (osd.5) 322 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:49.970232+0000 mon.a (mon.0) 3542 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:49.973856+0000 mon.a (mon.0) 3543 : audit [INF] from='client.? 172.21.15.59:0/569529675' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:49.973918+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? 172.21.15.59:0/2173431162' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:49.973949+0000 mon.a (mon.0) 3545 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:50.007938+0000 mgr.y (mgr.24454) 637 : cluster [DBG] pgmap v1133: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:51.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:50.316251+0000 osd.3 (osd.3) 377 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:44:51.254 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:44:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:44:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:49.581772+0000 osd.0 (osd.0) 324 : cluster [DBG] 25.e scrub starts 2024-02-18T00:44:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:49.583027+0000 osd.0 (osd.0) 325 : cluster [DBG] 25.e scrub ok 2024-02-18T00:44:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:49.709199+0000 osd.5 (osd.5) 321 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:44:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:49.712082+0000 osd.5 (osd.5) 322 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:44:51.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:49.970232+0000 mon.a (mon.0) 3542 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:44:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:49.973856+0000 mon.a (mon.0) 3543 : audit [INF] from='client.? 172.21.15.59:0/569529675' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:44:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:49.973918+0000 mon.a (mon.0) 3544 : audit [INF] from='client.? 172.21.15.59:0/2173431162' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:44:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:49.973949+0000 mon.a (mon.0) 3545 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-18T00:44:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:50.007938+0000 mgr.y (mgr.24454) 637 : cluster [DBG] pgmap v1133: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:44:51.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:50.316251+0000 osd.3 (osd.3) 377 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:44:51.875 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:51.878 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:51.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:51.979 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:52.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:44:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:44:52.011 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:52.033 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:52.035 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:52.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:52.136 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:52.171 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:52.193 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:52.195 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:52.196 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:52.297 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:52.327 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:52.348 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:52.350 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:52.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:52.452 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:44:52.481 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:52.501 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:44:52.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:44:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:44:52] "GET /metrics HTTP/1.1" 200 191859 "" "Prometheus/2.33.4" 2024-02-18T00:44:52.503 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:52.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:44:52.604 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:52.639 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:53.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:51.587122+0000 mgr.y (mgr.24454) 638 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:53.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:52.008933+0000 mgr.y (mgr.24454) 639 : cluster [DBG] pgmap v1134: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-18T00:44:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:51.587122+0000 mgr.y (mgr.24454) 638 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:53.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:52.008933+0000 mgr.y (mgr.24454) 639 : cluster [DBG] pgmap v1134: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-18T00:44:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:51.587122+0000 mgr.y (mgr.24454) 638 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:44:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:52.008933+0000 mgr.y (mgr.24454) 639 : cluster [DBG] pgmap v1134: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-18T00:44:53.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 7 1 2024-02-18T00:44:54.280 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_blocked_ops 2024-02-18T00:44:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:44:54.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:54.325 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:44:54.426 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:54.454 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:54.704 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:54.707 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:54.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:54.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:44:54.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:44:54.836 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:55.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:53.367205+0000 osd.0 (osd.0) 326 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:55.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:53.367388+0000 osd.0 (osd.0) 327 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:55.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:53.810450+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-18T00:44:55.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:54.009549+0000 mgr.y (mgr.24454) 640 : cluster [DBG] pgmap v1136: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-02-18T00:44:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:53.367205+0000 osd.0 (osd.0) 326 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:53.367388+0000 osd.0 (osd.0) 327 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:53.810450+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-18T00:44:55.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:54.009549+0000 mgr.y (mgr.24454) 640 : cluster [DBG] pgmap v1136: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-02-18T00:44:55.037 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:44:55.039 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:55.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:44:55.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:55.169 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:55.243 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:55.245 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:55.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:53.367205+0000 osd.0 (osd.0) 326 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:44:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:53.367388+0000 osd.0 (osd.0) 327 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:44:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:53.810450+0000 mon.a (mon.0) 3546 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-18T00:44:55.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:54.009549+0000 mgr.y (mgr.24454) 640 : cluster [DBG] pgmap v1136: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-02-18T00:44:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3531182254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fe510 msgr2=0x7f6b480f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3531182254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fe510 0x7f6b480f7d40 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 rx=0x7f6b3c004870 tx=0x7f6b3c009cb0).stop 2024-02-18T00:44:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3531182254 shutdown_connections 2024-02-18T00:44:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3531182254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b480f8420 0x7f6b48104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3531182254 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fdbb0 0x7f6b480fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3531182254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fe510 0x7f6b480f7d40 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3531182254 >> 172.21.15.59:0/3531182254 conn(0x7f6b480f3800 msgr2=0x7f6b480f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3531182254 shutdown_connections 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3531182254 wait complete. 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 librados: starting msgr at 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.338+0000 7f6b4cbb9700 1 librados: starting objecter 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 Processor -- start 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 -- start start 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 librados: setting wanted keys 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 librados: calling monclient init 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b480f8420 0x7f6b4810e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fdbb0 0x7f6b4810e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fe510 0x7f6b48108220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48108710 con 0x7f6b480fdbb0 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48108880 con 0x7f6b480f8420 2024-02-18T00:44:55.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b4cbb9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48108a40 con 0x7f6b480fe510 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b45d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fdbb0 0x7f6b4810e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b46d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fe510 0x7f6b48108220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b46d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fe510 0x7f6b48108220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39154/0 (socket says 172.21.15.59:39154) 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b45d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fdbb0 0x7f6b4810e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49670/0 (socket says 172.21.15.59:49670) 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.339+0000 7f6b46d9d700 1 -- 172.21.15.59:0/3641680864 learned_addr learned my addr 172.21.15.59:0/3641680864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b480f8420 0x7f6b4810e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b46d9d700 1 -- 172.21.15.59:0/3641680864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b480f8420 msgr2=0x7f6b4810e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b46d9d700 1 --2- 172.21.15.59:0/3641680864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b480f8420 0x7f6b4810e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b46d9d700 1 -- 172.21.15.59:0/3641680864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fdbb0 msgr2=0x7f6b4810e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b46d9d700 1 --2- 172.21.15.59:0/3641680864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fdbb0 0x7f6b4810e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b46d9d700 1 -- 172.21.15.59:0/3641680864 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38005040 con 0x7f6b480fe510 2024-02-18T00:44:55.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b45d9b700 1 --2- 172.21.15.59:0/3641680864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b480fdbb0 0x7f6b4810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b46d9d700 1 --2- 172.21.15.59:0/3641680864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fe510 0x7f6b48108220 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7f6b4000d970 tx=0x7f6b40004e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b377fe700 10 client.?.objecter ms_handle_connect 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b377fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000de60 con 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b4000f040 con 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c005040 con 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400036c0 con 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b4cbb9700 10 client.41747.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b48109050 con 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.340+0000 7f6b4cbb9700 1 librados: init done 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.341+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f6b48107060 con 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.341+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6b40003ab0 con 0x7f6b480fe510 2024-02-18T00:44:55.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6b30062c90 0x7f6b30065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(741..741 src has 418..741) v4 ==== 8840+0+0 (secure 0 0 0) 0x7f6b4007d0c0 con 0x7f6b480fe510 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 osd_map(741..741 src has 418..741) v4 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 3 client.41747.objecter handle_osd_map got epochs [741,741] > 0 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 3 client.41747.objecter handle_osd_map decoding full epoch 741 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6b30062c90 0x7f6b30065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 20 client.41747.objecter dump_active .. 0 homeless 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v741) v1 ==== 54+0+16 (secure 0 0 0) 0x7f6b40084070 con 0x7f6b480fe510 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b30062c90 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6b30062c90 0x7f6b30065150 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 rx=0x7f6b380092f0 tx=0x7f6b38009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 1 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6801/3700238977 conn(0x7f6b480598f0 0x7f6b48059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 1 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 0 1 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48109700 osd=0 4 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 1 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b4806cf80 con 0x7f6b480598f0 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.342+0000 7f6b45d9b700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6801/3700238977 conn(0x7f6b480598f0 0x7f6b48059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.343+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b480598f0 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.343+0000 7f6b45d9b700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6801/3700238977 conn(0x7f6b480598f0 0x7f6b48059cd0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.343+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b4806cf80 con 0x7f6b480598f0 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.343+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.343+0000 7f6b377fe700 10 client.41747.objecter _finish_command 1 = system:0 2024-02-18T00:44:55.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.343+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48109700 osd=0 4 2024-02-18T00:44:55.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.343+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 0 1 2024-02-18T00:44:55.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:55.359 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:55.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 2 2024-02-18T00:44:55.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 2 2024-02-18T00:44:55.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48109700 osd=0 3 2024-02-18T00:44:55.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 0 2 2024-02-18T00:44:55.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48109700 osd=0 4 2024-02-18T00:44:55.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 2 2024-02-18T00:44:55.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.367+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b480273c0 con 0x7f6b480598f0 2024-02-18T00:44:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.370+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b480273c0 con 0x7f6b480598f0 2024-02-18T00:44:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.370+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.370+0000 7f6b377fe700 10 client.41747.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.370+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48109700 osd=0 4 2024-02-18T00:44:55.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.370+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 0 2 2024-02-18T00:44:55.372 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:44:55.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.370+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 3 2024-02-18T00:44:55.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6805/26279110 conn(0x7f6b480ff0c0 0x7f6b480ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 3 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 1 3 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48028710 osd=1 4 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 3 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b48119620 con 0x7f6b480ff0c0 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6805/26279110 conn(0x7f6b480ff0c0 0x7f6b480ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b480ff0c0 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.371+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6805/26279110 conn(0x7f6b480ff0c0 0x7f6b480ff4a0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.372+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b48119620 con 0x7f6b480ff0c0 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.372+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.372+0000 7f6b377fe700 10 client.41747.objecter _finish_command 3 = system:0 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.372+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48028710 osd=1 4 2024-02-18T00:44:55.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.372+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 1 3 2024-02-18T00:44:55.390 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 4 2024-02-18T00:44:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 4 2024-02-18T00:44:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48028710 osd=1 3 2024-02-18T00:44:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 1 4 2024-02-18T00:44:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48028710 osd=1 4 2024-02-18T00:44:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 4 2024-02-18T00:44:55.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.393+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b480feb30 con 0x7f6b480ff0c0 2024-02-18T00:44:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.396+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b480feb30 con 0x7f6b480ff0c0 2024-02-18T00:44:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.396+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.396+0000 7f6b377fe700 10 client.41747.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.396+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48028710 osd=1 4 2024-02-18T00:44:55.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.396+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 1 4 2024-02-18T00:44:55.398 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:44:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 5 2024-02-18T00:44:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6809/1153193637 conn(0x7f6b48028270 0x7f6b48118040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 5 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 2 5 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b480fef40 osd=2 4 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 5 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.397+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b48118530 con 0x7f6b48028270 2024-02-18T00:44:55.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.398+0000 7f6b45d9b700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6809/1153193637 conn(0x7f6b48028270 0x7f6b48118040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.399+0000 7f6b45d9b700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6809/1153193637 conn(0x7f6b48028270 0x7f6b48118040 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.399+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b48028270 2024-02-18T00:44:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.399+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b48118530 con 0x7f6b48028270 2024-02-18T00:44:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.399+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:44:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.399+0000 7f6b377fe700 10 client.41747.objecter _finish_command 5 = system:0 2024-02-18T00:44:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.399+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b480fef40 osd=2 4 2024-02-18T00:44:55.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.399+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 2 5 2024-02-18T00:44:55.405 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:55.408 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:44:55.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 6 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 6 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b480fef40 osd=2 3 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 2 6 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b480fef40 osd=2 4 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 6 2024-02-18T00:44:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.418+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b480277d0 con 0x7f6b48028270 2024-02-18T00:44:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.420+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b480277d0 con 0x7f6b48028270 2024-02-18T00:44:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.420+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.420+0000 7f6b377fe700 10 client.41747.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.420+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b480fef40 osd=2 4 2024-02-18T00:44:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.420+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 2 6 2024-02-18T00:44:55.422 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 7 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6813/1805700964 conn(0x7f6b4811b7a0 0x7f6b4811db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 7 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 3 7 2024-02-18T00:44:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48027be0 osd=3 4 2024-02-18T00:44:55.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 7 2024-02-18T00:44:55.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b48119430 con 0x7f6b4811b7a0 2024-02-18T00:44:55.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.421+0000 7f6b46d9d700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6813/1805700964 conn(0x7f6b4811b7a0 0x7f6b4811db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.423+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b4811b7a0 2024-02-18T00:44:55.424 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.423+0000 7f6b46d9d700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6813/1805700964 conn(0x7f6b4811b7a0 0x7f6b4811db90 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.424+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b48119430 con 0x7f6b4811b7a0 2024-02-18T00:44:55.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.424+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:44:55.425 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.424+0000 7f6b377fe700 10 client.41747.objecter _finish_command 7 = system:0 2024-02-18T00:44:55.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.424+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48027be0 osd=3 4 2024-02-18T00:44:55.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.424+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 3 7 2024-02-18T00:44:55.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 8 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 8 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48027be0 osd=3 3 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 3 8 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48027be0 osd=3 4 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 8 2024-02-18T00:44:55.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.439+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b48118990 con 0x7f6b4811b7a0 2024-02-18T00:44:55.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b48118990 con 0x7f6b4811b7a0 2024-02-18T00:44:55.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b377fe700 10 client.41747.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48027be0 osd=3 4 2024-02-18T00:44:55.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 3 8 2024-02-18T00:44:55.443 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:44:55.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 9 2024-02-18T00:44:55.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6800/2044387056 conn(0x7f6b48117ba0 0x7f6b48121210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 9 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 4 9 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48118750 osd=4 4 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 9 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.441+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b481188c0 con 0x7f6b48117ba0 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.442+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6800/2044387056 conn(0x7f6b48117ba0 0x7f6b48121210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.442+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6800/2044387056 conn(0x7f6b48117ba0 0x7f6b48121210 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.442+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b48117ba0 2024-02-18T00:44:55.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.443+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b481188c0 con 0x7f6b48117ba0 2024-02-18T00:44:55.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.443+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:44:55.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.443+0000 7f6b377fe700 10 client.41747.objecter _finish_command 9 = system:0 2024-02-18T00:44:55.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.443+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48118750 osd=4 4 2024-02-18T00:44:55.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.443+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 4 9 2024-02-18T00:44:55.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 10 2024-02-18T00:44:55.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 10 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48118750 osd=4 3 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 4 10 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48118750 osd=4 4 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 10 2024-02-18T00:44:55.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.458+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b481188c0 con 0x7f6b48117ba0 2024-02-18T00:44:55.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b481188c0 con 0x7f6b48117ba0 2024-02-18T00:44:55.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b377fe700 10 client.41747.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48118750 osd=4 4 2024-02-18T00:44:55.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 4 10 2024-02-18T00:44:55.462 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:44:55.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 11 2024-02-18T00:44:55.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6804/2170432263 conn(0x7f6b48121c10 0x7f6b48124060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 11 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 5 11 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b481188c0 osd=5 4 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 11 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b48126e30 con 0x7f6b48121c10 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.460+0000 7f6b45d9b700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6804/2170432263 conn(0x7f6b48121c10 0x7f6b48124060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.461+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b48121c10 2024-02-18T00:44:55.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.461+0000 7f6b45d9b700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6804/2170432263 conn(0x7f6b48121c10 0x7f6b48124060 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.461+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b48126e30 con 0x7f6b48121c10 2024-02-18T00:44:55.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.461+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:44:55.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.461+0000 7f6b377fe700 10 client.41747.objecter _finish_command 11 = system:0 2024-02-18T00:44:55.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.461+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b481188c0 osd=5 4 2024-02-18T00:44:55.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.461+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 5 11 2024-02-18T00:44:55.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 12 2024-02-18T00:44:55.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 12 2024-02-18T00:44:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b481188c0 osd=5 3 2024-02-18T00:44:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 5 12 2024-02-18T00:44:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b481188c0 osd=5 4 2024-02-18T00:44:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 12 2024-02-18T00:44:55.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.476+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b48126e30 con 0x7f6b48121c10 2024-02-18T00:44:55.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b48126e30 con 0x7f6b48121c10 2024-02-18T00:44:55.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b377fe700 10 client.41747.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b481188c0 osd=5 4 2024-02-18T00:44:55.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 5 12 2024-02-18T00:44:55.480 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:44:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 13 2024-02-18T00:44:55.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6808/3356009390 conn(0x7f6b48126130 0x7f6b481265b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 13 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 6 13 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126e30 osd=6 4 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 13 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b48126c20 con 0x7f6b48126130 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.478+0000 7f6b46d9d700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6808/3356009390 conn(0x7f6b48126130 0x7f6b481265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.479+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b48126130 2024-02-18T00:44:55.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.479+0000 7f6b46d9d700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6808/3356009390 conn(0x7f6b48126130 0x7f6b481265b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.479+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b48126c20 con 0x7f6b48126130 2024-02-18T00:44:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.479+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:44:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.479+0000 7f6b377fe700 10 client.41747.objecter _finish_command 13 = system:0 2024-02-18T00:44:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.479+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48126e30 osd=6 4 2024-02-18T00:44:55.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.479+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 6 13 2024-02-18T00:44:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 14 2024-02-18T00:44:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 14 2024-02-18T00:44:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48126e30 osd=6 3 2024-02-18T00:44:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 6 14 2024-02-18T00:44:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126e30 osd=6 4 2024-02-18T00:44:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 14 2024-02-18T00:44:55.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.494+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b48126c20 con 0x7f6b48126130 2024-02-18T00:44:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b48126c20 con 0x7f6b48126130 2024-02-18T00:44:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b377fe700 10 client.41747.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48126e30 osd=6 4 2024-02-18T00:44:55.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 6 14 2024-02-18T00:44:55.498 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:44:55.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:44:55.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 15 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6812/53025171 conn(0x7f6b481295a0 0x7f6b48129a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 15 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 7 15 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126c20 osd=7 4 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 15 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b4812a070 con 0x7f6b481295a0 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.496+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6812/53025171 conn(0x7f6b481295a0 0x7f6b48129a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:44:55.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.497+0000 7f6b377fe700 10 client.41747.objecter ms_handle_connect 0x7f6b481295a0 2024-02-18T00:44:55.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.497+0000 7f6b4659c700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6812/53025171 conn(0x7f6b481295a0 0x7f6b48129a00 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:44:55.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.498+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f6b4812a070 con 0x7f6b481295a0 2024-02-18T00:44:55.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.498+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:44:55.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.498+0000 7f6b377fe700 10 client.41747.objecter _finish_command 15 = system:0 2024-02-18T00:44:55.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.498+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48126c20 osd=7 4 2024-02-18T00:44:55.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.498+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 7 15 2024-02-18T00:44:55.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 10 client.41747.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign -1 16 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 20 client.41747.objecter _get_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_remove -1 16 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 20 client.41747.objecter get_session s=0x7f6b48126c20 osd=7 3 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 15 client.41747.objecter _session_command_op_assign 7 16 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126c20 osd=7 4 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 10 client.41747.objecter _send_command 16 2024-02-18T00:44:55.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.512+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f6b4812a070 con 0x7f6b481295a0 2024-02-18T00:44:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b377fe700 1 -- 172.21.15.59:0/3641680864 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f6b4812a070 con 0x7f6b481295a0 2024-02-18T00:44:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b377fe700 10 client.41747.objecter ms_dispatch 0x7f6b48001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:44:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b377fe700 10 client.41747.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b377fe700 20 client.41747.objecter put_session s=0x7f6b48126c20 osd=7 4 2024-02-18T00:44:55.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b377fe700 15 client.41747.objecter _session_command_op_remove 7 16 2024-02-18T00:44:55.516 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 10 librados: watch_flush enter 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 10 librados: watch_flush exit 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.0 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6801/3700238977 conn(0x7f6b480598f0 msgr2=0x7f6b48059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6801/3700238977 conn(0x7f6b480598f0 0x7f6b48059cd0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48109700 osd=0 1 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.1 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6805/26279110 conn(0x7f6b480ff0c0 msgr2=0x7f6b480ff4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.514+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6805/26279110 conn(0x7f6b480ff0c0 0x7f6b480ff4a0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48028710 osd=1 1 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.2 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6809/1153193637 conn(0x7f6b48028270 msgr2=0x7f6b48118040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6809/1153193637 conn(0x7f6b48028270 0x7f6b48118040 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b480fef40 osd=2 1 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.3 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6813/1805700964 conn(0x7f6b4811b7a0 msgr2=0x7f6b4811db90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b46d9d700 1 -- 172.21.15.59:0/3641680864 reap_dead start 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6813/1805700964 conn(0x7f6b4811b7a0 0x7f6b4811db90 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48027be0 osd=3 1 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.4 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6800/2044387056 conn(0x7f6b48117ba0 msgr2=0x7f6b48121210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6800/2044387056 conn(0x7f6b48117ba0 0x7f6b48121210 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48118750 osd=4 1 2024-02-18T00:44:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.5 2024-02-18T00:44:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6804/2170432263 conn(0x7f6b48121c10 msgr2=0x7f6b48124060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6804/2170432263 conn(0x7f6b48121c10 0x7f6b48124060 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b481188c0 osd=5 1 2024-02-18T00:44:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.6 2024-02-18T00:44:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6808/3356009390 conn(0x7f6b48126130 msgr2=0x7f6b481265b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6808/3356009390 conn(0x7f6b48126130 0x7f6b481265b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126e30 osd=6 1 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 10 client.41747.objecter close_session for osd.7 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6812/53025171 conn(0x7f6b481295a0 msgr2=0x7f6b48129a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.137:6812/53025171 conn(0x7f6b481295a0 0x7f6b48129a00 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter put_session s=0x7f6b48126c20 osd=7 1 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 20 client.41747.objecter shutdown clearing up homeless session... 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 10 client.41747.objecter successfully canceled tick 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b46d9d700 1 -- 172.21.15.59:0/3641680864 reap_dead start 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6b30062c90 msgr2=0x7f6b30065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6b30062c90 0x7f6b30065150 secure :-1 s=READY pgs=3221 cs=0 l=1 rev1=1 rx=0x7f6b380092f0 tx=0x7f6b38009cb0).stop 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fe510 msgr2=0x7f6b48108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fe510 0x7f6b48108220 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 rx=0x7f6b4000d970 tx=0x7f6b40004e60).stop 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 shutdown_connections 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b480fe510 0x7f6b48108220 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 --2- 172.21.15.59:0/3641680864 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6b30062c90 0x7f6b30065150 unknown :-1 s=CLOSED pgs=3221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 >> 172.21.15.59:0/3641680864 conn(0x7f6b480f3800 msgr2=0x7f6b480fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 shutdown_connections 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 -- 172.21.15.59:0/3641680864 wait complete. 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:44:55.515+0000 7f6b4cbb9700 1 librados: shutdown 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.522 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:44:55.523 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:44:56.050 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:56.053 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:56.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:56.155 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:44:56.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:54.910688+0000 mon.a (mon.0) 3547 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:56.183 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:55.343230+0000 mon.c (mon.2) 573 : audit [DBG] from='client.? 172.21.15.59:0/3641680864' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:56.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:54.910688+0000 mon.a (mon.0) 3547 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:56.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:55.343230+0000 mon.c (mon.2) 573 : audit [DBG] from='client.? 172.21.15.59:0/3641680864' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:56.185 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:56.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:54.910688+0000 mon.a (mon.0) 3547 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:44:56.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:55.343230+0000 mon.c (mon.2) 573 : audit [DBG] from='client.? 172.21.15.59:0/3641680864' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:44:56.955 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:56.010588+0000 mgr.y (mgr.24454) 641 : cluster [DBG] pgmap v1137: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-18T00:44:56.955 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:56.010588+0000 mgr.y (mgr.24454) 641 : cluster [DBG] pgmap v1137: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-18T00:44:57.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:56.010588+0000 mgr.y (mgr.24454) 641 : cluster [DBG] pgmap v1137: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-18T00:44:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:44:58.139 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:58.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:44:58.241 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:58.273 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:58.295 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:58.298 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:58.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:58.399 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:44:58.430 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:58.451 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:44:58.453 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:58.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:44:58.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:44:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:58.661 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:44:58.664 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:58.664 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:44:58.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:44:58.765 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:44:58.795 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:58.817 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:44:58.820 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:44:58.821 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:44:58.921 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:44:58.953 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:44:59.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:44:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:44:58.011189+0000 mgr.y (mgr.24454) 642 : cluster [DBG] pgmap v1138: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-18T00:44:59.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:44:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:44:58.011189+0000 mgr.y (mgr.24454) 642 : cluster [DBG] pgmap v1138: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-18T00:44:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:44:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:44:58.011189+0000 mgr.y (mgr.24454) 642 : cluster [DBG] pgmap v1138: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-18T00:44:59.924 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:44:59.924 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:44:59.924 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:44:59.924 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:44:59.924 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:45:00.076 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:59.921189+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:00.076 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:44:59.921527+0000 mon.a (mon.0) 3549 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130"}]: dispatch 2024-02-18T00:45:00.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:59.921189+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:00.076 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:44:59.921527+0000 mon.a (mon.0) 3549 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130"}]: dispatch 2024-02-18T00:45:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:59.921189+0000 mon.a (mon.0) 3548 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:44:59.921527+0000 mon.a (mon.0) 3549 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130"}]: dispatch 2024-02-18T00:45:00.706 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:00.710 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:00.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:00.712 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.2 dump_historic_ops 2024-02-18T00:45:00.812 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:00.843 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:01.044 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:01.046 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:01.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:01.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:01.181 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:01.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:45:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:45:01.981 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:00.011562+0000 mgr.y (mgr.24454) 643 : cluster [DBG] pgmap v1139: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-18T00:45:01.982 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:00.071498+0000 mon.a (mon.0) 3550 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130"}]': finished 2024-02-18T00:45:01.982 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:00.071568+0000 mon.a (mon.0) 3551 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-18T00:45:01.982 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:00.011562+0000 mgr.y (mgr.24454) 643 : cluster [DBG] pgmap v1139: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-18T00:45:01.982 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:00.071498+0000 mon.a (mon.0) 3550 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130"}]': finished 2024-02-18T00:45:01.982 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:00.071568+0000 mon.a (mon.0) 3551 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-18T00:45:02.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:45:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:45:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:00.011562+0000 mgr.y (mgr.24454) 643 : cluster [DBG] pgmap v1139: 214 pgs: 214 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-02-18T00:45:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:00.071498+0000 mon.a (mon.0) 3550 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-130"}]': finished 2024-02-18T00:45:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:00.071568+0000 mon.a (mon.0) 3551 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-18T00:45:02.018 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:02.251 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:02] "GET /metrics HTTP/1.1" 200 192014 "" "Prometheus/2.33.4" 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 -- 172.21.15.59:0/1666066960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8588102210 msgr2=0x7f8588104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 --2- 172.21.15.59:0/1666066960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8588102210 0x7f8588104600 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 rx=0x7f8580009be0 tx=0x7f8580004d10).stop 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 -- 172.21.15.59:0/1666066960 shutdown_connections 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 --2- 172.21.15.59:0/1666066960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8588102210 0x7f8588104600 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 --2- 172.21.15.59:0/1666066960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85880f8cb0 0x7f85880f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 --2- 172.21.15.59:0/1666066960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85880f9610 0x7f8588101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 -- 172.21.15.59:0/1666066960 >> 172.21.15.59:0/1666066960 conn(0x7f85880f2820 msgr2=0x7f85880f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 -- 172.21.15.59:0/1666066960 shutdown_connections 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.231+0000 7f858eb65700 1 -- 172.21.15.59:0/1666066960 wait complete. 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 librados: starting msgr at 2024-02-18T00:45:02.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 librados: starting objecter 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 Processor -- start 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 -- start start 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 librados: setting wanted keys 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 librados: calling monclient init 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85880f8cb0 0x7f85880f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85880f9610 0x7f85880f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588102210 0x7f85880f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588116440 con 0x7f85880f9610 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85881165b0 con 0x7f8588102210 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858eb65700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8588116770 con 0x7f85880f8cb0 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858c901700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85880f8cb0 0x7f85880f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858c901700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85880f8cb0 0x7f85880f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39194/0 (socket says 172.21.15.59:39194) 2024-02-18T00:45:02.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f858c901700 1 -- 172.21.15.59:0/3335571062 learned_addr learned my addr 172.21.15.59:0/3335571062 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.232+0000 7f8587fff700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85880f9610 0x7f85880f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858d102700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588102210 0x7f85880f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858c901700 1 -- 172.21.15.59:0/3335571062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588102210 msgr2=0x7f85880f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858c901700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588102210 0x7f85880f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858c901700 1 -- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85880f9610 msgr2=0x7f85880f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858c901700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85880f9610 0x7f85880f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858c901700 1 -- 172.21.15.59:0/3335571062 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c005040 con 0x7f85880f8cb0 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f8587fff700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85880f9610 0x7f85880f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858c901700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85880f8cb0 0x7f85880f6c60 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 rx=0x7f857c0048f0 tx=0x7f857c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f8585ffb700 10 client.?.objecter ms_handle_connect 0x7f85880f8cb0 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f8585ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f8585ffb700 1 -- 172.21.15.59:0/3335571062 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c007b00 con 0x7f85880f8cb0 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85800097c0 con 0x7f85880f8cb0 2024-02-18T00:45:02.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f8585ffb700 1 -- 172.21.15.59:0/3335571062 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f857c00f040 con 0x7f85880f8cb0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f8585ffb700 1 -- 172.21.15.59:0/3335571062 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c014a00 con 0x7f85880f8cb0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858eb65700 10 client.41753.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8588116e00 con 0x7f85880f8cb0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.233+0000 7f858eb65700 1 librados: init done 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.234+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858805fa60 con 0x7f85880f8cb0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.234+0000 7f8585ffb700 1 -- 172.21.15.59:0/3335571062 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f857c00e580 con 0x7f85880f8cb0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8585ffb700 1 --2- 172.21.15.59:0/3335571062 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8578062c90 0x7f8578065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8585ffb700 1 -- 172.21.15.59:0/3335571062 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(744..744 src has 418..744) v4 ==== 8806+0+0 (secure 0 0 0) 0x7f857c07d090 con 0x7f85880f8cb0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8585ffb700 10 client.41753.objecter ms_dispatch 0x7f8588001550 osd_map(744..744 src has 418..744) v4 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8585ffb700 3 client.41753.objecter handle_osd_map got epochs [744,744] > 0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8585ffb700 3 client.41753.objecter handle_osd_map decoding full epoch 744 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8587fff700 1 --2- 172.21.15.59:0/3335571062 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8578062c90 0x7f8578065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:02.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8585ffb700 20 client.41753.objecter dump_active .. 0 homeless 2024-02-18T00:45:02.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8585ffb700 10 client.41753.objecter ms_handle_connect 0x7f8578062c90 2024-02-18T00:45:02.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.235+0000 7f8587fff700 1 --2- 172.21.15.59:0/3335571062 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8578062c90 0x7f8578065150 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 rx=0x7f85880f8400 tx=0x7f8574009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:02.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.239+0000 7f8585ffb700 1 -- 172.21.15.59:0/3335571062 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f857c049f60 con 0x7f85880f8cb0 2024-02-18T00:45:02.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:02.429+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) v1 -- 0x7f858805ee80 con 0x7f85880f8cb0 2024-02-18T00:45:02.519 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:02.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:01.078464+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-18T00:45:02.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:01.588704+0000 mgr.y (mgr.24454) 644 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:02.011927+0000 mgr.y (mgr.24454) 645 : cluster [DBG] pgmap v1142: 182 pgs: 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:02.080525+0000 mon.a (mon.0) 3553 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:02.082971+0000 mon.a (mon.0) 3554 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:02.430792+0000 mon.c (mon.2) 574 : audit [INF] from='client.? 172.21.15.59:0/3335571062' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:02.431068+0000 mon.a (mon.0) 3555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:01.078464+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:01.588704+0000 mgr.y (mgr.24454) 644 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:02.011927+0000 mgr.y (mgr.24454) 645 : cluster [DBG] pgmap v1142: 182 pgs: 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:02.080525+0000 mon.a (mon.0) 3553 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-18T00:45:02.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:02.082971+0000 mon.a (mon.0) 3554 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:02.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:02.430792+0000 mon.c (mon.2) 574 : audit [INF] from='client.? 172.21.15.59:0/3335571062' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:45:02.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:02.431068+0000 mon.a (mon.0) 3555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:45:02.839 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:02.842 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:02.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:02.944 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:02.976 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:01.078464+0000 mon.a (mon.0) 3552 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-18T00:45:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:01.588704+0000 mgr.y (mgr.24454) 644 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:02.011927+0000 mgr.y (mgr.24454) 645 : cluster [DBG] pgmap v1142: 182 pgs: 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:02.080525+0000 mon.a (mon.0) 3553 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-18T00:45:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:02.082971+0000 mon.a (mon.0) 3554 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:02.430792+0000 mon.c (mon.2) 574 : audit [INF] from='client.? 172.21.15.59:0/3335571062' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:45:03.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:02.431068+0000 mon.a (mon.0) 3555 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:45:03.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.092+0000 7f8585ffb700 1 -- 172.21.15.59:0/3335571062 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v745) v1 ==== 113+0+0 (secure 0 0 0) 0x7f857c050270 con 0x7f85880f8cb0 2024-02-18T00:45:03.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.095+0000 7f858eb65700 10 librados: watch_flush enter 2024-02-18T00:45:03.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.099+0000 7f858eb65700 10 librados: watch_flush exit 2024-02-18T00:45:03.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.099+0000 7f858eb65700 20 client.41753.objecter shutdown clearing up homeless session... 2024-02-18T00:45:03.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.099+0000 7f858eb65700 10 client.41753.objecter successfully canceled tick 2024-02-18T00:45:03.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.100+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8578062c90 msgr2=0x7f8578065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:03.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.100+0000 7f858eb65700 1 --2- 172.21.15.59:0/3335571062 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8578062c90 0x7f8578065150 secure :-1 s=READY pgs=3222 cs=0 l=1 rev1=1 rx=0x7f85880f8400 tx=0x7f8574009040).stop 2024-02-18T00:45:03.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.100+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85880f8cb0 msgr2=0x7f85880f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:03.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.100+0000 7f858eb65700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85880f8cb0 0x7f85880f6c60 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 rx=0x7f857c0048f0 tx=0x7f857c009cb0).stop 2024-02-18T00:45:03.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 shutdown_connections 2024-02-18T00:45:03.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 --2- 172.21.15.59:0/3335571062 >> v2:172.21.15.59:6800/3721891797 conn(0x7f8578062c90 0x7f8578065150 unknown :-1 s=CLOSED pgs=3222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588102210 0x7f85880f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85880f8cb0 0x7f85880f6c60 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 --2- 172.21.15.59:0/3335571062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85880f9610 0x7f85880f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 >> 172.21.15.59:0/3335571062 conn(0x7f85880f2820 msgr2=0x7f85880f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 shutdown_connections 2024-02-18T00:45:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.110+0000 7f858eb65700 1 -- 172.21.15.59:0/3335571062 wait complete. 2024-02-18T00:45:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.112+0000 7f858eb65700 1 librados: shutdown 2024-02-18T00:45:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.7 to 1 (10000) 2024-02-18T00:45:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/1487755206 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fcbb0 msgr2=0x7f7b080fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/1487755206 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fcbb0 0x7f7b080fcf90 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 rx=0x7f7af8004870 tx=0x7f7af8009cb0).stop 2024-02-18T00:45:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/1487755206 shutdown_connections 2024-02-18T00:45:03.277 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/1487755206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 0x7f7b08103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/1487755206 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fcbb0 0x7f7b080fcf90 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/1487755206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b080fd510 0x7f7b080f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/1487755206 >> 172.21.15.59:0/1487755206 conn(0x7f7b080f2820 msgr2=0x7f7b080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/1487755206 shutdown_connections 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.061+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/1487755206 wait complete. 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 librados: starting msgr at 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 librados: starting objecter 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 Processor -- start 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 -- start start 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 librados: setting wanted keys 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 librados: calling monclient init 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 0x7f7b08105c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b080fcbb0 0x7f7b08104210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:03.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fd510 0x7f7b08104840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b08104f00 con 0x7f7b080f7400 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b08105070 con 0x7f7b080fcbb0 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b0d8a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0811ade0 con 0x7f7b080fd510 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b06ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 0x7f7b08105c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b06ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 0x7f7b08105c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39394/0 (socket says 172.21.15.59:39394) 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b06ffd700 1 -- 172.21.15.59:0/45876074 learned_addr learned my addr 172.21.15.59:0/45876074 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b077fe700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fd510 0x7f7b08104840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.062+0000 7f7b067fc700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b080fcbb0 0x7f7b08104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b06ffd700 1 -- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fd510 msgr2=0x7f7b08104840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b06ffd700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fd510 0x7f7b08104840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b06ffd700 1 -- 172.21.15.59:0/45876074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b080fcbb0 msgr2=0x7f7b08104210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b06ffd700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b080fcbb0 0x7f7b08104210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:03.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b06ffd700 1 -- 172.21.15.59:0/45876074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b000097c0 con 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b077fe700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fd510 0x7f7b08104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b06ffd700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 0x7f7b08105c70 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 rx=0x7f7af8009eb0 tx=0x7f7af800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7aeffff700 10 client.?.objecter ms_handle_connect 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7aeffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7aeffff700 1 -- 172.21.15.59:0/45876074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800e070 con 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af8005040 con 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7aeffff700 1 -- 172.21.15.59:0/45876074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7af8013da0 con 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7aeffff700 1 -- 172.21.15.59:0/45876074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af8019070 con 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b0d8a4700 10 client.31959.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0811b2d0 con 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.063+0000 7f7b0d8a4700 1 librados: init done 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.064+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0805fa60 con 0x7f7b080f7400 2024-02-18T00:45:03.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 1 -- 172.21.15.59:0/45876074 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f7af801e420 con 0x7f7b080f7400 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 1 --2- 172.21.15.59:0/45876074 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7af0062c90 0x7f7af0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 1 -- 172.21.15.59:0/45876074 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(744..744 src has 418..744) v4 ==== 8806+0+0 (secure 0 0 0) 0x7f7af807dfc0 con 0x7f7b080f7400 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 10 client.31959.objecter ms_dispatch 0x7f7b08001550 osd_map(744..744 src has 418..744) v4 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 3 client.31959.objecter handle_osd_map got epochs [744,744] > 0 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 3 client.31959.objecter handle_osd_map decoding full epoch 744 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7b067fc700 1 --2- 172.21.15.59:0/45876074 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7af0062c90 0x7f7af0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 20 client.31959.objecter dump_active .. 0 homeless 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7aeffff700 10 client.31959.objecter ms_handle_connect 0x7f7af0062c90 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.065+0000 7f7b067fc700 1 --2- 172.21.15.59:0/45876074 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7af0062c90 0x7f7af0065150 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 rx=0x7f7b081059b0 tx=0x7f7afc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:03.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.070+0000 7f7aeffff700 1 -- 172.21.15.59:0/45876074 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7af80acf60 con 0x7f7b080f7400 2024-02-18T00:45:03.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:03.282+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7f7b08107220 con 0x7f7b080f7400 2024-02-18T00:45:04.008 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-18T00:45:04.008 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:45:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.088+0000 7f7aeffff700 1 -- 172.21.15.59:0/45876074 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v746) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7af8023030 con 0x7f7b080f7400 2024-02-18T00:45:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 10 librados: watch_flush enter 2024-02-18T00:45:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 10 librados: watch_flush exit 2024-02-18T00:45:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 20 client.31959.objecter shutdown clearing up homeless session... 2024-02-18T00:45:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 10 client.31959.objecter successfully canceled tick 2024-02-18T00:45:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7af0062c90 msgr2=0x7f7af0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/45876074 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7af0062c90 0x7f7af0065150 secure :-1 s=READY pgs=3223 cs=0 l=1 rev1=1 rx=0x7f7b081059b0 tx=0x7f7afc00b040).stop 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 msgr2=0x7f7b08105c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.091+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 0x7f7b08105c70 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 rx=0x7f7af8009eb0 tx=0x7f7af800b040).stop 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 shutdown_connections 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7b080fd510 0x7f7b08104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b080f7400 0x7f7b08105c70 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/45876074 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7af0062c90 0x7f7af0065150 unknown :-1 s=CLOSED pgs=3223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 --2- 172.21.15.59:0/45876074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b080fcbb0 0x7f7b08104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 >> 172.21.15.59:0/45876074 conn(0x7f7b080f2820 msgr2=0x7f7b080f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 shutdown_connections 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 -- 172.21.15.59:0/45876074 wait complete. 2024-02-18T00:45:04.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:04.092+0000 7f7b0d8a4700 1 librados: shutdown 2024-02-18T00:45:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:45:04.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:03.085258+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:04.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:03.085440+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:03.085501+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:03.146103+0000 mon.a (mon.0) 3559 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:03.284280+0000 mon.a (mon.0) 3560 : audit [INF] from='client.? 172.21.15.59:0/45876074' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:03.085258+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:03.085440+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:03.085501+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:03.146103+0000 mon.a (mon.0) 3559 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:03.284280+0000 mon.a (mon.0) 3560 : audit [INF] from='client.? 172.21.15.59:0/45876074' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:04.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:03.085258+0000 mon.a (mon.0) 3556 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:03.085440+0000 mon.a (mon.0) 3557 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:45:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:03.085501+0000 mon.a (mon.0) 3558 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-18T00:45:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:03.146103+0000 mon.a (mon.0) 3559 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:03.284280+0000 mon.a (mon.0) 3560 : audit [INF] from='client.? 172.21.15.59:0/45876074' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:05.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:04.012366+0000 mgr.y (mgr.24454) 646 : cluster [DBG] pgmap v1145: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:05.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:04.080048+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:05.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:04.088992+0000 mon.a (mon.0) 3562 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:05.094 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:04.089072+0000 mon.a (mon.0) 3563 : audit [INF] from='client.? 172.21.15.59:0/45876074' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:05.095 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:04.089141+0000 mon.a (mon.0) 3564 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-02-18T00:45:05.095 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:04.090876+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:05.348 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:04.012366+0000 mgr.y (mgr.24454) 646 : cluster [DBG] pgmap v1145: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:05.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:04.080048+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:05.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:04.088992+0000 mon.a (mon.0) 3562 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:05.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:04.089072+0000 mon.a (mon.0) 3563 : audit [INF] from='client.? 172.21.15.59:0/45876074' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:05.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:04.089141+0000 mon.a (mon.0) 3564 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-02-18T00:45:05.349 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:04.090876+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:04.012366+0000 mgr.y (mgr.24454) 646 : cluster [DBG] pgmap v1145: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:04.080048+0000 mon.a (mon.0) 3561 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:04.088992+0000 mon.a (mon.0) 3562 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:04.089072+0000 mon.a (mon.0) 3563 : audit [INF] from='client.? 172.21.15.59:0/45876074' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:04.089141+0000 mon.a (mon.0) 3564 : cluster [DBG] osdmap e746: 8 total, 8 up, 8 in 2024-02-18T00:45:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:04.090876+0000 mon.a (mon.0) 3565 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:05.731 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:05.735 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:05.736 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:05.836 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:05.871 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:06.420 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:06.423 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:05.089813+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-132"}]': finished 2024-02-18T00:45:06.423 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:05.089873+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-02-18T00:45:06.423 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:05.096109+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-132", "mode": "writeback"}]: dispatch 2024-02-18T00:45:06.423 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:06.032432+0000 mon.a (mon.0) 3569 : audit [DBG] from='client.? 172.21.15.59:0/1200393767' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:06.424 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:05.089813+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-132"}]': finished 2024-02-18T00:45:06.424 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:05.089873+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-02-18T00:45:06.424 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:05.096109+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-132", "mode": "writeback"}]: dispatch 2024-02-18T00:45:06.424 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:06.032432+0000 mon.a (mon.0) 3569 : audit [DBG] from='client.? 172.21.15.59:0/1200393767' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:06.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:06.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:05.089813+0000 mon.a (mon.0) 3566 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-132"}]': finished 2024-02-18T00:45:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:05.089873+0000 mon.a (mon.0) 3567 : cluster [DBG] osdmap e747: 8 total, 8 up, 8 in 2024-02-18T00:45:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:05.096109+0000 mon.a (mon.0) 3568 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-132", "mode": "writeback"}]: dispatch 2024-02-18T00:45:06.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:06.032432+0000 mon.a (mon.0) 3569 : audit [DBG] from='client.? 172.21.15.59:0/1200393767' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:06.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:06.555 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:06.574 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:06.583 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:06.583 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:06.684 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:06.717 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:06.747 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:06.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:06.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:06.877 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:07.000 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:07.002 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:07.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:07.104 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:07.131 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:06.013158+0000 mgr.y (mgr.24454) 647 : cluster [DBG] pgmap v1148: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:06.092806+0000 mon.a (mon.0) 3570 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:06.101734+0000 mon.a (mon.0) 3571 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-132", "mode": "writeback"}]': finished 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:06.101863+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:06.102991+0000 mon.a (mon.0) 3573 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:06.013158+0000 mgr.y (mgr.24454) 647 : cluster [DBG] pgmap v1148: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:06.092806+0000 mon.a (mon.0) 3570 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:06.101734+0000 mon.a (mon.0) 3571 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-132", "mode": "writeback"}]': finished 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:06.101863+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-02-18T00:45:07.132 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:06.102991+0000 mon.a (mon.0) 3573 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:45:07.133 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:07.332 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:07.335 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:07.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:07.436 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.027+0000 7f74fa95d700 1 -- 172.21.15.59:0/2462364898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40fcbb0 msgr2=0x7f74f40fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.027+0000 7f74fa95d700 1 --2- 172.21.15.59:0/2462364898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40fcbb0 0x7f74f40fcf90 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 rx=0x7f74e4004870 tx=0x7f74e4009cb0).stop 2024-02-18T00:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.027+0000 7f74fa95d700 1 -- 172.21.15.59:0/2462364898 shutdown_connections 2024-02-18T00:45:07.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.027+0000 7f74fa95d700 1 --2- 172.21.15.59:0/2462364898 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74f40f7400 0x7f74f4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.027+0000 7f74fa95d700 1 --2- 172.21.15.59:0/2462364898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40fcbb0 0x7f74f40fcf90 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.027+0000 7f74fa95d700 1 --2- 172.21.15.59:0/2462364898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f40fd510 0x7f74f40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.027+0000 7f74fa95d700 1 -- 172.21.15.59:0/2462364898 >> 172.21.15.59:0/2462364898 conn(0x7f74f40f2820 msgr2=0x7f74f40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 -- 172.21.15.59:0/2462364898 shutdown_connections 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 -- 172.21.15.59:0/2462364898 wait complete. 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 librados: starting msgr at 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 librados: starting objecter 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 Processor -- start 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 -- start start 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 librados: setting wanted keys 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.028+0000 7f74fa95d700 1 librados: calling monclient init 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74fa95d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40f7400 0x7f74f40f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74fa95d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74f40fcbb0 0x7f74f40f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74fa95d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f40fd510 0x7f74f40f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74fa95d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f4116670 con 0x7f74f40f7400 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74fa95d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f41167e0 con 0x7f74f40fd510 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74fa95d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f41169a0 con 0x7f74f40fcbb0 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40f7400 0x7f74f40f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74f40fcbb0 0x7f74f40f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40f7400 0x7f74f40f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39410/0 (socket says 172.21.15.59:39410) 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 -- 172.21.15.59:0/1200393767 learned_addr learned my addr 172.21.15.59:0/1200393767 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f40fd510 0x7f74f40f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 -- 172.21.15.59:0/1200393767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74f40fcbb0 msgr2=0x7f74f40f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74f40fcbb0 0x7f74f40f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 -- 172.21.15.59:0/1200393767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f40fd510 msgr2=0x7f74f40f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f40fd510 0x7f74f40f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 -- 172.21.15.59:0/1200393767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec0097c0 con 0x7f74f40f7400 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f37fe700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f74f40fcbb0 0x7f74f40f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74f40fd510 0x7f74f40f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40f7400 0x7f74f40f89b0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7f74e4009eb0 tx=0x7f74e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f17fa700 10 client.?.objecter ms_handle_connect 0x7f74f40f7400 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.029+0000 7f74f17fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e400e070 con 0x7f74f40f7400 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e4005040 con 0x7f74f40f7400 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74e4013da0 con 0x7f74f40f7400 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4019070 con 0x7f74f40f7400 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74fa95d700 10 client.31965.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f4116e90 con 0x7f74f40f7400 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74fa95d700 1 librados: init done 2024-02-18T00:45:07.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.030+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f74f405fa60 con 0x7f74f40f7400 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.031+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f74e401e420 con 0x7f74f40f7400 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6800/3721891797 conn(0x7f74dc062c90 0x7f74dc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(747..747 src has 418..747) v4 ==== 8814+0+0 (secure 0 0 0) 0x7f74e407e4e0 con 0x7f74f40f7400 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 osd_map(747..747 src has 418..747) v4 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 3 client.31965.objecter handle_osd_map got epochs [747,747] > 0 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 3 client.31965.objecter handle_osd_map decoding full epoch 747 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f37fe700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6800/3721891797 conn(0x7f74dc062c90 0x7f74dc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 20 client.31965.objecter dump_active .. 0 homeless 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v747) v1 ==== 54+0+16 (secure 0 0 0) 0x7f74e407e980 con 0x7f74f40f7400 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 10 client.31965.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 1 2024-02-18T00:45:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74f40ffed0 0x7f74f41002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 1 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 0 1 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f405ee80 osd=0 4 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 10 client.31965.objecter _send_command 1 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f41060e0 con 0x7f74f40ffed0 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74dc062c90 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f37fe700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6800/3721891797 conn(0x7f74dc062c90 0x7f74dc065150 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 rx=0x7f74f40fa150 tx=0x7f74e800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.032+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74f40ffed0 0x7f74f41002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.033+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f40ffed0 2024-02-18T00:45:07.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.033+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74f40ffed0 0x7f74f41002b0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.033+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f41060e0 con 0x7f74f40ffed0 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.033+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.033+0000 7f74f17fa700 10 client.31965.objecter _finish_command 1 = system:0 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.033+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f405ee80 osd=0 4 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.033+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 0 1 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 10 client.31965.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 2 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 2 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f405ee80 osd=0 3 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 0 2 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f405ee80 osd=0 4 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 10 client.31965.objecter _send_command 2 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.048+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f40273c0 con 0x7f74f40ffed0 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f40273c0 con 0x7f74f40ffed0 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74f17fa700 10 client.31965.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f405ee80 osd=0 4 2024-02-18T00:45:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 0 2 2024-02-18T00:45:07.468 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 10 client.31965.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 3 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6805/26279110 conn(0x7f74f40598f0 0x7f74f4059cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 3 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 1 3 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4028710 osd=1 4 2024-02-18T00:45:07.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 10 client.31965.objecter _send_command 3 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f411d7a0 con 0x7f74f40598f0 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.050+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6805/26279110 conn(0x7f74f40598f0 0x7f74f4059cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.051+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f40598f0 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.051+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6805/26279110 conn(0x7f74f40598f0 0x7f74f4059cf0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.051+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f411d7a0 con 0x7f74f40598f0 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.051+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.051+0000 7f74f17fa700 10 client.31965.objecter _finish_command 3 = system:0 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.051+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f4028710 osd=1 4 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.051+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 1 3 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 10 client.31965.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 4 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.471 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:45:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 4 2024-02-18T00:45:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f4028710 osd=1 3 2024-02-18T00:45:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 1 4 2024-02-18T00:45:07.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4028710 osd=1 4 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 10 client.31965.objecter _send_command 4 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.066+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f4059360 con 0x7f74f40598f0 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.067+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f4059360 con 0x7f74f40598f0 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.067+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.067+0000 7f74f17fa700 10 client.31965.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.067+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f4028710 osd=1 4 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.067+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 1 4 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 10 client.31965.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 5 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6809/1153193637 conn(0x7f74f411c1c0 0x7f74f411c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 5 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 2 5 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4027d60 osd=2 4 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 10 client.31965.objecter _send_command 5 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f40283e0 con 0x7f74f411c1c0 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f37fe700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6809/1153193637 conn(0x7f74f411c1c0 0x7f74f411c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f411c1c0 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f37fe700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6809/1153193637 conn(0x7f74f411c1c0 0x7f74f411c5e0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f40283e0 con 0x7f74f411c1c0 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:45:07.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f17fa700 10 client.31965.objecter _finish_command 5 = system:0 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f4027d60 osd=2 4 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.068+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 2 5 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 10 client.31965.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 6 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 6 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f4027d60 osd=2 3 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 2 6 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4027d60 osd=2 4 2024-02-18T00:45:07.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 10 client.31965.objecter _send_command 6 2024-02-18T00:45:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.083+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f411d2a0 con 0x7f74f411c1c0 2024-02-18T00:45:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.084+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f411d2a0 con 0x7f74f411c1c0 2024-02-18T00:45:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.084+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.475 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:45:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.084+0000 7f74f17fa700 10 client.31965.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.084+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f4027d60 osd=2 4 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.084+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 2 6 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 10 client.31965.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 7 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6813/1805700964 conn(0x7f74f4027940 0x7f74f4121e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 7 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 3 7 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f40277d0 osd=3 4 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 10 client.31965.objecter _send_command 7 2024-02-18T00:45:07.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f4124a90 con 0x7f74f4027940 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6813/1805700964 conn(0x7f74f4027940 0x7f74f4121e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f4027940 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.085+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6813/1805700964 conn(0x7f74f4027940 0x7f74f4121e40 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.086+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f4124a90 con 0x7f74f4027940 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.086+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.086+0000 7f74f17fa700 10 client.31965.objecter _finish_command 7 = system:0 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.086+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f40277d0 osd=3 4 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.086+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 3 7 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 10 client.31965.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 8 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 8 2024-02-18T00:45:07.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f40277d0 osd=3 3 2024-02-18T00:45:07.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 3 8 2024-02-18T00:45:07.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f40277d0 osd=3 4 2024-02-18T00:45:07.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 10 client.31965.objecter _send_command 8 2024-02-18T00:45:07.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.106+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f411cd10 con 0x7f74f4027940 2024-02-18T00:45:07.478 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f411cd10 con 0x7f74f4027940 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74f17fa700 10 client.31965.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f40277d0 osd=3 4 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 3 8 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 10 client.31965.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 9 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6800/2044387056 conn(0x7f74f411bd20 0x7f74f41234b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 9 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 4 9 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f411cf50 osd=4 4 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 10 client.31965.objecter _send_command 9 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.108+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f41239a0 con 0x7f74f411bd20 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.109+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6800/2044387056 conn(0x7f74f411bd20 0x7f74f41234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.109+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f411bd20 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.109+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6800/2044387056 conn(0x7f74f411bd20 0x7f74f41234b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.110+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f41239a0 con 0x7f74f411bd20 2024-02-18T00:45:07.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.110+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.110+0000 7f74f17fa700 10 client.31965.objecter _finish_command 9 = system:0 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.110+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f411cf50 osd=4 4 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.110+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 4 9 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 10 client.31965.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 10 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 10 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f411cf50 osd=4 3 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 4 10 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f411cf50 osd=4 4 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 10 client.31965.objecter _send_command 10 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.125+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f41239a0 con 0x7f74f411bd20 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f41239a0 con 0x7f74f411bd20 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74f17fa700 10 client.31965.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f411cf50 osd=4 4 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 4 10 2024-02-18T00:45:07.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 10 client.31965.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 11 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6804/2170432263 conn(0x7f74f41240d0 0x7f74f4124510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 11 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 5 11 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f41239a0 osd=5 4 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 10 client.31965.objecter _send_command 11 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f412b3c0 con 0x7f74f41240d0 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.127+0000 7f74f37fe700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6804/2170432263 conn(0x7f74f41240d0 0x7f74f4124510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.128+0000 7f74f37fe700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6804/2170432263 conn(0x7f74f41240d0 0x7f74f4124510 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.128+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f41240d0 2024-02-18T00:45:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.129+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f412b3c0 con 0x7f74f41240d0 2024-02-18T00:45:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.129+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:45:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.129+0000 7f74f17fa700 10 client.31965.objecter _finish_command 11 = system:0 2024-02-18T00:45:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.129+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f41239a0 osd=5 4 2024-02-18T00:45:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.129+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 5 11 2024-02-18T00:45:07.484 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:45:07.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 10 client.31965.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 12 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 12 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f41239a0 osd=5 3 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 5 12 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f41239a0 osd=5 4 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 10 client.31965.objecter _send_command 12 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.143+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f412b3c0 con 0x7f74f41240d0 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f412b3c0 con 0x7f74f41240d0 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74f17fa700 10 client.31965.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f41239a0 osd=5 4 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 5 12 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 10 client.31965.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 13 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6808/3356009390 conn(0x7f74f4129170 0x7f74f41295d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 13 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 6 13 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f412b3c0 osd=6 4 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 10 client.31965.objecter _send_command 13 2024-02-18T00:45:07.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.145+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f412e7a0 con 0x7f74f4129170 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.146+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6808/3356009390 conn(0x7f74f4129170 0x7f74f41295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.146+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f4129170 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.146+0000 7f74f8efa700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6808/3356009390 conn(0x7f74f4129170 0x7f74f41295d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.147+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f412e7a0 con 0x7f74f4129170 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.147+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.147+0000 7f74f17fa700 10 client.31965.objecter _finish_command 13 = system:0 2024-02-18T00:45:07.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.147+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f412b3c0 osd=6 4 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.147+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 6 13 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 10 client.31965.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 14 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 14 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f412b3c0 osd=6 3 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 6 14 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f412b3c0 osd=6 4 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 10 client.31965.objecter _send_command 14 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.161+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f4129b80 con 0x7f74f4129170 2024-02-18T00:45:07.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f4129b80 con 0x7f74f4129170 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74f17fa700 10 client.31965.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f412b3c0 osd=6 4 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 6 14 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 10 client.31965.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 15 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6812/53025171 conn(0x7f74f412a200 0x7f74f412a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 15 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 7 15 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4129b80 osd=7 4 2024-02-18T00:45:07.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 10 client.31965.objecter _send_command 15 2024-02-18T00:45:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.163+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74f412aca0 con 0x7f74f412a200 2024-02-18T00:45:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.164+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6812/53025171 conn(0x7f74f412a200 0x7f74f412a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.164+0000 7f74f17fa700 10 client.31965.objecter ms_handle_connect 0x7f74f412a200 2024-02-18T00:45:07.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.164+0000 7f74f3fff700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6812/53025171 conn(0x7f74f412a200 0x7f74f412a660 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:07.490 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:45:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.165+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74f412aca0 con 0x7f74f412a200 2024-02-18T00:45:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.165+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:45:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.165+0000 7f74f17fa700 10 client.31965.objecter _finish_command 15 = system:0 2024-02-18T00:45:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.165+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f4129b80 osd=7 4 2024-02-18T00:45:07.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.165+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 7 15 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 10 client.31965.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign -1 16 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 20 client.31965.objecter _get_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_remove -1 16 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 20 client.31965.objecter get_session s=0x7f74f4129b80 osd=7 3 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 15 client.31965.objecter _session_command_op_assign 7 16 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4129b80 osd=7 4 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 10 client.31965.objecter _send_command 16 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.179+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f74f412aca0 con 0x7f74f412a200 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74f17fa700 1 -- 172.21.15.59:0/1200393767 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f74f412aca0 con 0x7f74f412a200 2024-02-18T00:45:07.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74f17fa700 10 client.31965.objecter ms_dispatch 0x7f74f4001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74f17fa700 10 client.31965.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74f17fa700 20 client.31965.objecter put_session s=0x7f74f4129b80 osd=7 4 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74f17fa700 15 client.31965.objecter _session_command_op_remove 7 16 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 10 librados: watch_flush enter 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 10 librados: watch_flush exit 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.0 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74f40ffed0 msgr2=0x7f74f41002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6801/3700238977 conn(0x7f74f40ffed0 0x7f74f41002b0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f405ee80 osd=0 1 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.1 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6805/26279110 conn(0x7f74f40598f0 msgr2=0x7f74f4059cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6805/26279110 conn(0x7f74f40598f0 0x7f74f4059cf0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4028710 osd=1 1 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.2 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6809/1153193637 conn(0x7f74f411c1c0 msgr2=0x7f74f411c5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6809/1153193637 conn(0x7f74f411c1c0 0x7f74f411c5e0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4027d60 osd=2 1 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.3 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6813/1805700964 conn(0x7f74f4027940 msgr2=0x7f74f4121e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6813/1805700964 conn(0x7f74f4027940 0x7f74f4121e40 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f40277d0 osd=3 1 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.4 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74f8efa700 1 -- 172.21.15.59:0/1200393767 reap_dead start 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6800/2044387056 conn(0x7f74f411bd20 msgr2=0x7f74f41234b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.181+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6800/2044387056 conn(0x7f74f411bd20 0x7f74f41234b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74f8efa700 1 -- 172.21.15.59:0/1200393767 reap_dead start 2024-02-18T00:45:07.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f411cf50 osd=4 1 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.5 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6804/2170432263 conn(0x7f74f41240d0 msgr2=0x7f74f4124510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6804/2170432263 conn(0x7f74f41240d0 0x7f74f4124510 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f41239a0 osd=5 1 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.6 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6808/3356009390 conn(0x7f74f4129170 msgr2=0x7f74f41295d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6808/3356009390 conn(0x7f74f4129170 0x7f74f41295d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f412b3c0 osd=6 1 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 10 client.31965.objecter close_session for osd.7 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6812/53025171 conn(0x7f74f412a200 msgr2=0x7f74f412a660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.137:6812/53025171 conn(0x7f74f412a200 0x7f74f412a660 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 20 client.31965.objecter put_session s=0x7f74f4129b80 osd=7 1 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 20 client.31965.objecter shutdown clearing up homeless session... 2024-02-18T00:45:07.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 10 client.31965.objecter successfully canceled tick 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6800/3721891797 conn(0x7f74dc062c90 msgr2=0x7f74dc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> v2:172.21.15.59:6800/3721891797 conn(0x7f74dc062c90 0x7f74dc065150 secure :-1 s=READY pgs=3224 cs=0 l=1 rev1=1 rx=0x7f74f40fa150 tx=0x7f74e800b040).stop 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40f7400 msgr2=0x7f74f40f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74f8efa700 1 -- 172.21.15.59:0/1200393767 reap_dead start 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40f7400 0x7f74f40f89b0 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7f74e4009eb0 tx=0x7f74e400b040).stop 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 shutdown_connections 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 --2- 172.21.15.59:0/1200393767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f74f40f7400 0x7f74f40f89b0 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 >> 172.21.15.59:0/1200393767 conn(0x7f74f40f2820 msgr2=0x7f74f40f4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 shutdown_connections 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 -- 172.21.15.59:0/1200393767 wait complete. 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:06.182+0000 7f74fa95d700 1 librados: shutdown 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.496 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.497 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.497 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.497 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.497 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.497 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:45:07.497 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:06.013158+0000 mgr.y (mgr.24454) 647 : cluster [DBG] pgmap v1148: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:45:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:06.092806+0000 mon.a (mon.0) 3570 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:06.101734+0000 mon.a (mon.0) 3571 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-132", "mode": "writeback"}]': finished 2024-02-18T00:45:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:06.101863+0000 mon.a (mon.0) 3572 : cluster [DBG] osdmap e748: 8 total, 8 up, 8 in 2024-02-18T00:45:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:06.102991+0000 mon.a (mon.0) 3573 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:45:08.374 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:45:08.375 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:45:08.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:06.652855+0000 osd.3 (osd.3) 378 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:08.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:06.653245+0000 osd.3 (osd.3) 379 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:07.106268+0000 mon.a (mon.0) 3574 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:07.106343+0000 mon.a (mon.0) 3575 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:07.107940+0000 mon.a (mon.0) 3576 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:06.652855+0000 osd.3 (osd.3) 378 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:06.653245+0000 osd.3 (osd.3) 379 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:07.106268+0000 mon.a (mon.0) 3574 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:07.106343+0000 mon.a (mon.0) 3575 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-02-18T00:45:08.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:07.107940+0000 mon.a (mon.0) 3576 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:45:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:06.652855+0000 osd.3 (osd.3) 378 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:06.653245+0000 osd.3 (osd.3) 379 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:07.106268+0000 mon.a (mon.0) 3574 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:45:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:07.106343+0000 mon.a (mon.0) 3575 : cluster [DBG] osdmap e749: 8 total, 8 up, 8 in 2024-02-18T00:45:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:07.107940+0000 mon.a (mon.0) 3576 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:45:09.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:08.013529+0000 mgr.y (mgr.24454) 648 : cluster [DBG] pgmap v1151: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:45:09.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:08.113106+0000 mon.a (mon.0) 3577 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:45:09.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:08.113181+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-02-18T00:45:09.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:08.113984+0000 mon.a (mon.0) 3579 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:45:09.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:08.809774+0000 mon.a (mon.0) 3580 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:09.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:08.822061+0000 mon.a (mon.0) 3581 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:08.822160+0000 mon.a (mon.0) 3582 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:08.823342+0000 mon.a (mon.0) 3583 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:08.013529+0000 mgr.y (mgr.24454) 648 : cluster [DBG] pgmap v1151: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:08.113106+0000 mon.a (mon.0) 3577 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:08.113181+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:08.113984+0000 mon.a (mon.0) 3579 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:08.809774+0000 mon.a (mon.0) 3580 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:08.822061+0000 mon.a (mon.0) 3581 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:45:09.356 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:08.822160+0000 mon.a (mon.0) 3582 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-02-18T00:45:09.357 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:08.823342+0000 mon.a (mon.0) 3583 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:45:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:08.013529+0000 mgr.y (mgr.24454) 648 : cluster [DBG] pgmap v1151: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T00:45:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:08.113106+0000 mon.a (mon.0) 3577 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:45:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:08.113181+0000 mon.a (mon.0) 3578 : cluster [DBG] osdmap e750: 8 total, 8 up, 8 in 2024-02-18T00:45:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:08.113984+0000 mon.a (mon.0) 3579 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:45:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:08.809774+0000 mon.a (mon.0) 3580 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:08.822061+0000 mon.a (mon.0) 3581 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:45:09.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:08.822160+0000 mon.a (mon.0) 3582 : cluster [DBG] osdmap e751: 8 total, 8 up, 8 in 2024-02-18T00:45:09.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:08.823342+0000 mon.a (mon.0) 3583 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:45:09.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_ops_in_flight 2024-02-18T00:45:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:09.721 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:09.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:09.822 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:09.854 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:09.877 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:09.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:09.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:10.008 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:10.048 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:10.051 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:10.051 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:10.152 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:10.182 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:10.203 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:10.205 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:10.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:10.306 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:10.355 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:10.356 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:10.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:10.457 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:10.485 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:10.723 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:10.733 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:10.736 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:10.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:10.838 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:10.867 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:11.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:09.820112+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:09.820195+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:09.822586+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:09.989828+0000 osd.5 (osd.5) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:09.990073+0000 osd.5 (osd.5) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:10.013989+0000 mgr.y (mgr.24454) 649 : cluster [DBG] pgmap v1155: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:10.656633+0000 mon.a (mon.0) 3587 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:09.820112+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:09.820195+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:09.822586+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:09.989828+0000 osd.5 (osd.5) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:09.990073+0000 osd.5 (osd.5) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:10.013989+0000 mgr.y (mgr.24454) 649 : cluster [DBG] pgmap v1155: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:45:11.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:10.656633+0000 mon.a (mon.0) 3587 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:45:11.016 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:11.019 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:11.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:11.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:45:11.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:11.162 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:11.192 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:11.203 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:11.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:11.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:45:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:09.820112+0000 mon.a (mon.0) 3584 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:45:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:09.820195+0000 mon.a (mon.0) 3585 : cluster [DBG] osdmap e752: 8 total, 8 up, 8 in 2024-02-18T00:45:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:09.822586+0000 mon.a (mon.0) 3586 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T00:45:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:09.989828+0000 osd.5 (osd.5) 323 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:09.990073+0000 osd.5 (osd.5) 324 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:10.013989+0000 mgr.y (mgr.24454) 649 : cluster [DBG] pgmap v1155: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:45:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:10.656633+0000 mon.a (mon.0) 3587 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:45:11.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:45:11.305 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:11.332 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:11.351 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:11.353 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:11.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:11.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:11.492 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:12.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:10.821320+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:10.821389+0000 mon.a (mon.0) 3589 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:10.824635+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:11.094137+0000 mon.a (mon.0) 3591 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:11.476666+0000 mon.a (mon.0) 3592 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:11.477609+0000 mon.a (mon.0) 3593 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:11.488291+0000 mon.a (mon.0) 3594 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:11.821929+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T00:45:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:11.822008+0000 mon.a (mon.0) 3596 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-02-18T00:45:12.209 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:10.821320+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:10.821389+0000 mon.a (mon.0) 3589 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:10.824635+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:11.094137+0000 mon.a (mon.0) 3591 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:11.476666+0000 mon.a (mon.0) 3592 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:11.477609+0000 mon.a (mon.0) 3593 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:11.488291+0000 mon.a (mon.0) 3594 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:11.821929+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:11.822008+0000 mon.a (mon.0) 3596 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:10.821320+0000 mon.a (mon.0) 3588 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T00:45:12.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:10.821389+0000 mon.a (mon.0) 3589 : cluster [DBG] osdmap e753: 8 total, 8 up, 8 in 2024-02-18T00:45:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:10.824635+0000 mon.a (mon.0) 3590 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T00:45:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:11.094137+0000 mon.a (mon.0) 3591 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:11.476666+0000 mon.a (mon.0) 3592 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:45:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:11.477609+0000 mon.a (mon.0) 3593 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:45:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:11.488291+0000 mon.a (mon.0) 3594 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:11.821929+0000 mon.a (mon.0) 3595 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T00:45:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:11.822008+0000 mon.a (mon.0) 3596 : cluster [DBG] osdmap e754: 8 total, 8 up, 8 in 2024-02-18T00:45:12.342 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:12.348 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:12.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:12.450 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:12.463 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:45:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:12] "GET /metrics HTTP/1.1" 200 192091 "" "Prometheus/2.33.4" 2024-02-18T00:45:12.483 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:12.505 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:12.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:12.606 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:12.636 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:12.656 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:12.658 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:12.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:12.759 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:12.808 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:12.811 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:12.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:12.912 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:12.939 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:12.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:12.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:13.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:13.091 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:11.599516+0000 mgr.y (mgr.24454) 650 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:13.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:11.860619+0000 mon.a (mon.0) 3597 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:13.092 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:12.014667+0000 mgr.y (mgr.24454) 651 : cluster [DBG] pgmap v1158: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:45:13.092 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:11.599516+0000 mgr.y (mgr.24454) 650 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:13.092 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:11.860619+0000 mon.a (mon.0) 3597 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:13.092 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:12.014667+0000 mgr.y (mgr.24454) 651 : cluster [DBG] pgmap v1158: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:45:13.094 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:13.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:11.599516+0000 mgr.y (mgr.24454) 650 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:11.860619+0000 mon.a (mon.0) 3597 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:12.014667+0000 mgr.y (mgr.24454) 651 : cluster [DBG] pgmap v1158: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:45:13.444 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:13.449 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:13.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:13.450 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:13.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:13.583 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:14.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:12.837572+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:14.134 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:12.837664+0000 mon.a (mon.0) 3599 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-02-18T00:45:14.135 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:12.840688+0000 mon.a (mon.0) 3600 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:14.135 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:14.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:12.837572+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:14.135 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:12.837664+0000 mon.a (mon.0) 3599 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-02-18T00:45:14.136 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:12.840688+0000 mon.a (mon.0) 3600 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:14.137 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:14.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:14.238 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:14.247 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:45:14.247 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:45:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:12.837572+0000 mon.a (mon.0) 3598 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:12.837664+0000 mon.a (mon.0) 3599 : cluster [DBG] osdmap e755: 8 total, 8 up, 8 in 2024-02-18T00:45:14.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:12.840688+0000 mon.a (mon.0) 3600 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:14.266 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:14.285 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:14.287 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:14.288 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:14.388 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:14.420 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:14.496 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:14.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:14.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:14.625 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:13.841211+0000 mon.a (mon.0) 3601 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]': finished 2024-02-18T00:45:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:13.841294+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-02-18T00:45:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:13.955234+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:13.955692+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:14.015017+0000 mgr.y (mgr.24454) 652 : cluster [DBG] pgmap v1161: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:14.131370+0000 mgr.y (mgr.24454) 653 : audit [DBG] from='client.41771 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:13.841211+0000 mon.a (mon.0) 3601 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]': finished 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:13.841294+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:13.955234+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:13.955692+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:14.015017+0000 mgr.y (mgr.24454) 652 : cluster [DBG] pgmap v1161: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:14.131370+0000 mgr.y (mgr.24454) 653 : audit [DBG] from='client.41771 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:13.841211+0000 mon.a (mon.0) 3601 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]': finished 2024-02-18T00:45:15.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:13.841294+0000 mon.a (mon.0) 3602 : cluster [DBG] osdmap e756: 8 total, 8 up, 8 in 2024-02-18T00:45:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:13.955234+0000 mon.a (mon.0) 3603 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:13.955692+0000 mon.a (mon.0) 3604 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-132"}]: dispatch 2024-02-18T00:45:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:14.015017+0000 mgr.y (mgr.24454) 652 : cluster [DBG] pgmap v1161: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:14.131370+0000 mgr.y (mgr.24454) 653 : audit [DBG] from='client.41771 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:45:15.444 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_blocked_ops 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 -- 172.21.15.59:0/2582345285 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000f6a90 msgr2=0x7fdf00104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 --2- 172.21.15.59:0/2582345285 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000f6a90 0x7fdf00104a20 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 rx=0x7fdef8009c30 tx=0x7fdef8004c50).stop 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 -- 172.21.15.59:0/2582345285 shutdown_connections 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 --2- 172.21.15.59:0/2582345285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf000fdbd0 0x7fdf000fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 --2- 172.21.15.59:0/2582345285 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000f6a90 0x7fdf00104a20 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 --2- 172.21.15.59:0/2582345285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf000fe530 0x7fdf000f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 -- 172.21.15.59:0/2582345285 >> 172.21.15.59:0/2582345285 conn(0x7fdf0006c370 msgr2=0x7fdf0006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:15.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 -- 172.21.15.59:0/2582345285 shutdown_connections 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 -- 172.21.15.59:0/2582345285 wait complete. 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 librados: starting msgr at 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.930+0000 7fdf05010700 1 librados: starting objecter 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 Processor -- start 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 -- start start 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 librados: setting wanted keys 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 librados: calling monclient init 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf000f6a90 0x7fdf000f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf000fdbd0 0x7fdf000f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000fe530 0x7fdf000f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001165b0 con 0x7fdf000fdbd0 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001166f0 con 0x7fdf000f6a90 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdf05010700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001168b0 con 0x7fdf000fe530 2024-02-18T00:45:15.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf000fdbd0 0x7fdf000f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdeff59e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000fe530 0x7fdf000f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf000fdbd0 0x7fdf000f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38290/0 (socket says 172.21.15.59:38290) 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdefe59c700 1 -- 172.21.15.59:0/2871588008 learned_addr learned my addr 172.21.15.59:0/2871588008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.931+0000 7fdefed9d700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf000f6a90 0x7fdf000f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdeff59e700 1 -- 172.21.15.59:0/2871588008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf000f6a90 msgr2=0x7fdf000f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdeff59e700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf000f6a90 0x7fdf000f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdeff59e700 1 -- 172.21.15.59:0/2871588008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf000fdbd0 msgr2=0x7fdf000f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdeff59e700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf000fdbd0 0x7fdf000f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdeff59e700 1 -- 172.21.15.59:0/2871588008 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef4005040 con 0x7fdf000fe530 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdeff59e700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000fe530 0x7fdf000f8060 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 rx=0x7fdef8009ad0 tx=0x7fdef8004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdee7fff700 10 client.?.objecter ms_handle_connect 0x7fdf000fe530 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdee7fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:15.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdee7fff700 1 -- 172.21.15.59:0/2871588008 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef800e040 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef80097c0 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdee7fff700 1 -- 172.21.15.59:0/2871588008 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef8018040 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdee7fff700 1 -- 172.21.15.59:0/2871588008 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80039f0 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdf05010700 10 client.41771.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf00116da0 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.932+0000 7fdf05010700 1 librados: init done 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.934+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0005fa40 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.935+0000 7fdee7fff700 1 -- 172.21.15.59:0/2871588008 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fdef8003e20 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdee7fff700 1 --2- 172.21.15.59:0/2871588008 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdee8062c90 0x7fdee8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdee7fff700 1 -- 172.21.15.59:0/2871588008 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(756..756 src has 418..756) v4 ==== 8806+0+0 (secure 0 0 0) 0x7fdef807e490 con 0x7fdf000fe530 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdee7fff700 10 client.41771.objecter ms_dispatch 0x7fdf00001550 osd_map(756..756 src has 418..756) v4 2024-02-18T00:45:15.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdee7fff700 3 client.41771.objecter handle_osd_map got epochs [756,756] > 0 2024-02-18T00:45:15.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdee7fff700 3 client.41771.objecter handle_osd_map decoding full epoch 756 2024-02-18T00:45:15.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdefed9d700 1 --2- 172.21.15.59:0/2871588008 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdee8062c90 0x7fdee8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:15.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdee7fff700 20 client.41771.objecter dump_active .. 0 homeless 2024-02-18T00:45:15.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdefed9d700 1 --2- 172.21.15.59:0/2871588008 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdee8062c90 0x7fdee8065150 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 rx=0x7fdef00092f0 tx=0x7fdef000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:15.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.936+0000 7fdee7fff700 10 client.41771.objecter ms_handle_connect 0x7fdee8062c90 2024-02-18T00:45:15.552 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:45:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:13.944+0000 7fdee7fff700 1 -- 172.21.15.59:0/2871588008 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdef8051670 con 0x7fdf000fe530 2024-02-18T00:45:15.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:14.129+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf000f8b30 con 0x7fdee8062c90 2024-02-18T00:45:15.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:14.146+0000 7fdee7fff700 1 -- 172.21.15.59:0/2871588008 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+549623 (secure 0 0 0) 0x7fdf000f8b30 con 0x7fdee8062c90 2024-02-18T00:45:15.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 10 librados: watch_flush enter 2024-02-18T00:45:15.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 10 librados: watch_flush exit 2024-02-18T00:45:15.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 20 client.41771.objecter shutdown clearing up homeless session... 2024-02-18T00:45:15.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 10 client.41771.objecter successfully canceled tick 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdee8062c90 msgr2=0x7fdee8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 --2- 172.21.15.59:0/2871588008 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdee8062c90 0x7fdee8065150 secure :-1 s=READY pgs=3225 cs=0 l=1 rev1=1 rx=0x7fdef00092f0 tx=0x7fdef000b040).stop 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000fe530 msgr2=0x7fdf000f8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000fe530 0x7fdf000f8060 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 rx=0x7fdef8009ad0 tx=0x7fdef8004c50).stop 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 shutdown_connections 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 --2- 172.21.15.59:0/2871588008 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdee8062c90 0x7fdee8065150 unknown :-1 s=CLOSED pgs=3225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf000f6a90 0x7fdf000f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf000fe530 0x7fdf000f8060 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 --2- 172.21.15.59:0/2871588008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf000fdbd0 0x7fdf000f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 >> 172.21.15.59:0/2871588008 conn(0x7fdf0006c370 msgr2=0x7fdf000f5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 shutdown_connections 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 -- 172.21.15.59:0/2871588008 wait complete. 2024-02-18T00:45:15.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:15.548+0000 7fdf05010700 1 librados: shutdown 2024-02-18T00:45:15.572 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:45:15.572 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:15.575 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:15.580 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:15.581 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:15.681 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:15.710 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:15.748 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:15.750 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:15.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:15.852 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:15.880 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:16.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:14.922946+0000 mon.a (mon.0) 3605 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-02-18T00:45:16.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:14.922946+0000 mon.a (mon.0) 3605 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-02-18T00:45:16.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:14.922946+0000 mon.a (mon.0) 3605 : cluster [DBG] osdmap e757: 8 total, 8 up, 8 in 2024-02-18T00:45:17.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:15.927681+0000 mon.a (mon.0) 3606 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-02-18T00:45:17.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:15.939284+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:17.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:16.015557+0000 mgr.y (mgr.24454) 654 : cluster [DBG] pgmap v1164: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:45:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:15.927681+0000 mon.a (mon.0) 3606 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-02-18T00:45:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:15.939284+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:16.015557+0000 mgr.y (mgr.24454) 654 : cluster [DBG] pgmap v1164: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:45:17.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:15.927681+0000 mon.a (mon.0) 3606 : cluster [DBG] osdmap e758: 8 total, 8 up, 8 in 2024-02-18T00:45:17.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:15.939284+0000 mon.a (mon.0) 3607 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:17.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:16.015557+0000 mgr.y (mgr.24454) 654 : cluster [DBG] pgmap v1164: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:45:17.427 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":1161,"stamp":"2024-02-18T00:45:14.014811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4006050,"num_objects":262,"num_object_clones":1,"num_object_copies":786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":0,"num_read":1926,"num_read_kb":1743,"num_write":636,"num_write_kb":3990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":2492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1465,"ondisk_log_size":1465,"up":642,"acting":642,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":642,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2467160,"kb_used_data":52504,"kb_used_omap":0,"kb_used_meta":2414592,"kb_avail":747494056,"statfs":{"total":767960285184,"available":765433913344,"internally_reserved":0,"allocated":53764096,"data_stored":31610578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2472542208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001113"},"pg_stats":[{"pgid":"250.1f","version":"0'0","reported_seq":16,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011918+0000","last_change":"2024-02-18T00:45:03.104286+0000","last_active":"2024-02-18T00:45:10.011918+0000","last_peered":"2024-02-18T00:45:10.011918+0000","last_clean":"2024-02-18T00:45:10.011918+0000","last_became_active":"2024-02-18T00:45:03.103996+0000","last_became_peered":"2024-02-18T00:45:03.103996+0000","last_unstale":"2024-02-18T00:45:10.011918+0000","last_undegraded":"2024-02-18T00:45:10.011918+0000","last_fullsized":"2024-02-18T00:45:10.011918+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"250.1e","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816898+0000","last_change":"2024-02-18T00:45:03.095711+0000","last_active":"2024-02-18T00:45:08.816898+0000","last_peered":"2024-02-18T00:45:08.816898+0000","last_clean":"2024-02-18T00:45:08.816898+0000","last_became_active":"2024-02-18T00:45:03.095397+0000","last_became_peered":"2024-02-18T00:45:03.095397+0000","last_unstale":"2024-02-18T00:45:08.816898+0000","last_undegraded":"2024-02-18T00:45:08.816898+0000","last_fullsized":"2024-02-18T00:45:08.816898+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.1d","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730829+0000","last_change":"2024-02-18T00:45:03.104324+0000","last_active":"2024-02-18T00:45:09.730829+0000","last_peered":"2024-02-18T00:45:09.730829+0000","last_clean":"2024-02-18T00:45:09.730829+0000","last_became_active":"2024-02-18T00:45:03.104008+0000","last_became_peered":"2024-02-18T00:45:03.104008+0000","last_unstale":"2024-02-18T00:45:09.730829+0000","last_undegraded":"2024-02-18T00:45:09.730829+0000","last_fullsized":"2024-02-18T00:45:09.730829+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.1c","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816345+0000","last_change":"2024-02-18T00:45:03.112511+0000","last_active":"2024-02-18T00:45:08.816345+0000","last_peered":"2024-02-18T00:45:08.816345+0000","last_clean":"2024-02-18T00:45:08.816345+0000","last_became_active":"2024-02-18T00:45:03.112179+0000","last_became_peered":"2024-02-18T00:45:03.112179+0000","last_unstale":"2024-02-18T00:45:08.816345+0000","last_undegraded":"2024-02-18T00:45:08.816345+0000","last_fullsized":"2024-02-18T00:45:08.816345+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.1b","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730914+0000","last_change":"2024-02-18T00:45:03.096129+0000","last_active":"2024-02-18T00:45:09.730914+0000","last_peered":"2024-02-18T00:45:09.730914+0000","last_clean":"2024-02-18T00:45:09.730914+0000","last_became_active":"2024-02-18T00:45:03.095818+0000","last_became_peered":"2024-02-18T00:45:03.095818+0000","last_unstale":"2024-02-18T00:45:09.730914+0000","last_undegraded":"2024-02-18T00:45:09.730914+0000","last_fullsized":"2024-02-18T00:45:09.730914+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.1a","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816223+0000","last_change":"2024-02-18T00:45:03.103191+0000","last_active":"2024-02-18T00:45:08.816223+0000","last_peered":"2024-02-18T00:45:08.816223+0000","last_clean":"2024-02-18T00:45:08.816223+0000","last_became_active":"2024-02-18T00:45:03.102909+0000","last_became_peered":"2024-02-18T00:45:03.102909+0000","last_unstale":"2024-02-18T00:45:08.816223+0000","last_undegraded":"2024-02-18T00:45:08.816223+0000","last_fullsized":"2024-02-18T00:45:08.816223+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.19","version":"0'0","reported_seq":17,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011412+0000","last_change":"2024-02-18T00:45:03.100941+0000","last_active":"2024-02-18T00:45:10.011412+0000","last_peered":"2024-02-18T00:45:10.011412+0000","last_clean":"2024-02-18T00:45:10.011412+0000","last_became_active":"2024-02-18T00:45:03.100533+0000","last_became_peered":"2024-02-18T00:45:03.100533+0000","last_unstale":"2024-02-18T00:45:10.011412+0000","last_undegraded":"2024-02-18T00:45:10.011412+0000","last_fullsized":"2024-02-18T00:45:10.011412+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.18","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.304668+0000","last_change":"2024-02-18T00:45:03.112292+0000","last_active":"2024-02-18T00:45:09.304668+0000","last_peered":"2024-02-18T00:45:09.304668+0000","last_clean":"2024-02-18T00:45:09.304668+0000","last_became_active":"2024-02-18T00:45:03.111993+0000","last_became_peered":"2024-02-18T00:45:03.111993+0000","last_unstale":"2024-02-18T00:45:09.304668+0000","last_undegraded":"2024-02-18T00:45:09.304668+0000","last_fullsized":"2024-02-18T00:45:09.304668+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.f","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822432+0000","last_change":"2024-02-18T00:45:03.109285+0000","last_active":"2024-02-18T00:45:08.822432+0000","last_peered":"2024-02-18T00:45:08.822432+0000","last_clean":"2024-02-18T00:45:08.822432+0000","last_became_active":"2024-02-18T00:45:03.108816+0000","last_became_peered":"2024-02-18T00:45:03.108816+0000","last_unstale":"2024-02-18T00:45:08.822432+0000","last_undegraded":"2024-02-18T00:45:08.822432+0000","last_fullsized":"2024-02-18T00:45:08.822432+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"250.e","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305541+0000","last_change":"2024-02-18T00:45:03.109393+0000","last_active":"2024-02-18T00:45:09.305541+0000","last_peered":"2024-02-18T00:45:09.305541+0000","last_clean":"2024-02-18T00:45:09.305541+0000","last_became_active":"2024-02-18T00:45:03.109078+0000","last_became_peered":"2024-02-18T00:45:03.109078+0000","last_unstale":"2024-02-18T00:45:09.305541+0000","last_undegraded":"2024-02-18T00:45:09.305541+0000","last_fullsized":"2024-02-18T00:45:09.305541+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.d","version":"0'0","reported_seq":16,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011585+0000","last_change":"2024-02-18T00:45:03.106902+0000","last_active":"2024-02-18T00:45:10.011585+0000","last_peered":"2024-02-18T00:45:10.011585+0000","last_clean":"2024-02-18T00:45:10.011585+0000","last_became_active":"2024-02-18T00:45:03.106509+0000","last_became_peered":"2024-02-18T00:45:03.106509+0000","last_unstale":"2024-02-18T00:45:10.011585+0000","last_undegraded":"2024-02-18T00:45:10.011585+0000","last_fullsized":"2024-02-18T00:45:10.011585+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"250.c","version":"0'0","reported_seq":15,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664756+0000","last_change":"2024-02-18T00:45:03.096906+0000","last_active":"2024-02-18T00:45:09.664756+0000","last_peered":"2024-02-18T00:45:09.664756+0000","last_clean":"2024-02-18T00:45:09.664756+0000","last_became_active":"2024-02-18T00:45:03.096580+0000","last_became_peered":"2024-02-18T00:45:03.096580+0000","last_unstale":"2024-02-18T00:45:09.664756+0000","last_undegraded":"2024-02-18T00:45:09.664756+0000","last_fullsized":"2024-02-18T00:45:09.664756+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"250.b","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816042+0000","last_change":"2024-02-18T00:45:03.095923+0000","last_active":"2024-02-18T00:45:08.816042+0000","last_peered":"2024-02-18T00:45:08.816042+0000","last_clean":"2024-02-18T00:45:08.816042+0000","last_became_active":"2024-02-18T00:45:03.095537+0000","last_became_peered":"2024-02-18T00:45:03.095537+0000","last_unstale":"2024-02-18T00:45:08.816042+0000","last_undegraded":"2024-02-18T00:45:08.816042+0000","last_fullsized":"2024-02-18T00:45:08.816042+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.a","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305424+0000","last_change":"2024-02-18T00:45:03.104972+0000","last_active":"2024-02-18T00:45:09.305424+0000","last_peered":"2024-02-18T00:45:09.305424+0000","last_clean":"2024-02-18T00:45:09.305424+0000","last_became_active":"2024-02-18T00:45:03.104678+0000","last_became_peered":"2024-02-18T00:45:03.104678+0000","last_unstale":"2024-02-18T00:45:09.305424+0000","last_undegraded":"2024-02-18T00:45:09.305424+0000","last_fullsized":"2024-02-18T00:45:09.305424+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"250.9","version":"0'0","reported_seq":17,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011736+0000","last_change":"2024-02-18T00:45:03.100056+0000","last_active":"2024-02-18T00:45:10.011736+0000","last_peered":"2024-02-18T00:45:10.011736+0000","last_clean":"2024-02-18T00:45:10.011736+0000","last_became_active":"2024-02-18T00:45:03.099749+0000","last_became_peered":"2024-02-18T00:45:03.099749+0000","last_unstale":"2024-02-18T00:45:10.011736+0000","last_undegraded":"2024-02-18T00:45:10.011736+0000","last_fullsized":"2024-02-18T00:45:10.011736+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.8","version":"0'0","reported_seq":16,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010282+0000","last_change":"2024-02-18T00:45:03.104176+0000","last_active":"2024-02-18T00:45:10.010282+0000","last_peered":"2024-02-18T00:45:10.010282+0000","last_clean":"2024-02-18T00:45:10.010282+0000","last_became_active":"2024-02-18T00:45:03.103594+0000","last_became_peered":"2024-02-18T00:45:03.103594+0000","last_unstale":"2024-02-18T00:45:10.010282+0000","last_undegraded":"2024-02-18T00:45:10.010282+0000","last_fullsized":"2024-02-18T00:45:10.010282+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"250.7","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.817043+0000","last_change":"2024-02-18T00:45:03.106198+0000","last_active":"2024-02-18T00:45:08.817043+0000","last_peered":"2024-02-18T00:45:08.817043+0000","last_clean":"2024-02-18T00:45:08.817043+0000","last_became_active":"2024-02-18T00:45:03.105883+0000","last_became_peered":"2024-02-18T00:45:03.105883+0000","last_unstale":"2024-02-18T00:45:08.817043+0000","last_undegraded":"2024-02-18T00:45:08.817043+0000","last_fullsized":"2024-02-18T00:45:08.817043+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.6","version":"0'0","reported_seq":17,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013686+0000","last_change":"2024-02-18T00:45:03.103584+0000","last_active":"2024-02-18T00:45:10.013686+0000","last_peered":"2024-02-18T00:45:10.013686+0000","last_clean":"2024-02-18T00:45:10.013686+0000","last_became_active":"2024-02-18T00:45:03.103273+0000","last_became_peered":"2024-02-18T00:45:03.103273+0000","last_unstale":"2024-02-18T00:45:10.013686+0000","last_undegraded":"2024-02-18T00:45:10.013686+0000","last_fullsized":"2024-02-18T00:45:10.013686+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.5","version":"0'0","reported_seq":17,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010173+0000","last_change":"2024-02-18T00:45:03.103758+0000","last_active":"2024-02-18T00:45:10.010173+0000","last_peered":"2024-02-18T00:45:10.010173+0000","last_clean":"2024-02-18T00:45:10.010173+0000","last_became_active":"2024-02-18T00:45:03.103144+0000","last_became_peered":"2024-02-18T00:45:03.103144+0000","last_unstale":"2024-02-18T00:45:10.010173+0000","last_undegraded":"2024-02-18T00:45:10.010173+0000","last_fullsized":"2024-02-18T00:45:10.010173+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"250.4","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730973+0000","last_change":"2024-02-18T00:45:03.112508+0000","last_active":"2024-02-18T00:45:09.730973+0000","last_peered":"2024-02-18T00:45:09.730973+0000","last_clean":"2024-02-18T00:45:09.730973+0000","last_became_active":"2024-02-18T00:45:03.112122+0000","last_became_peered":"2024-02-18T00:45:03.112122+0000","last_unstale":"2024-02-18T00:45:09.730973+0000","last_undegraded":"2024-02-18T00:45:09.730973+0000","last_fullsized":"2024-02-18T00:45:09.730973+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.3","version":"0'0","reported_seq":16,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010864+0000","last_change":"2024-02-18T00:45:03.111977+0000","last_active":"2024-02-18T00:45:10.010864+0000","last_peered":"2024-02-18T00:45:10.010864+0000","last_clean":"2024-02-18T00:45:10.010864+0000","last_became_active":"2024-02-18T00:45:03.111669+0000","last_became_peered":"2024-02-18T00:45:03.111669+0000","last_unstale":"2024-02-18T00:45:10.010864+0000","last_undegraded":"2024-02-18T00:45:10.010864+0000","last_fullsized":"2024-02-18T00:45:10.010864+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"250.2","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820579+0000","last_change":"2024-02-18T00:45:03.108559+0000","last_active":"2024-02-18T00:45:08.820579+0000","last_peered":"2024-02-18T00:45:08.820579+0000","last_clean":"2024-02-18T00:45:08.820579+0000","last_became_active":"2024-02-18T00:45:03.108257+0000","last_became_peered":"2024-02-18T00:45:03.108257+0000","last_unstale":"2024-02-18T00:45:08.820579+0000","last_undegraded":"2024-02-18T00:45:08.820579+0000","last_fullsized":"2024-02-18T00:45:08.820579+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"250.1","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.815785+0000","last_change":"2024-02-18T00:45:03.100179+0000","last_active":"2024-02-18T00:45:08.815785+0000","last_peered":"2024-02-18T00:45:08.815785+0000","last_clean":"2024-02-18T00:45:08.815785+0000","last_became_active":"2024-02-18T00:45:03.099873+0000","last_became_peered":"2024-02-18T00:45:03.099873+0000","last_unstale":"2024-02-18T00:45:08.815785+0000","last_undegraded":"2024-02-18T00:45:08.815785+0000","last_fullsized":"2024-02-18T00:45:08.815785+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"250.0","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.729988+0000","last_change":"2024-02-18T00:45:03.099068+0000","last_active":"2024-02-18T00:45:09.729988+0000","last_peered":"2024-02-18T00:45:09.729988+0000","last_clean":"2024-02-18T00:45:09.729988+0000","last_became_active":"2024-02-18T00:45:03.098788+0000","last_became_peered":"2024-02-18T00:45:03.098788+0000","last_unstale":"2024-02-18T00:45:09.729988+0000","last_undegraded":"2024-02-18T00:45:09.729988+0000","last_fullsized":"2024-02-18T00:45:09.729988+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.29","version":"731'34","reported_seq":629,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012311+0000","last_change":"2024-02-18T00:44:40.215823+0000","last_active":"2024-02-18T00:45:10.012311+0000","last_peered":"2024-02-18T00:45:10.012311+0000","last_clean":"2024-02-18T00:45:10.012311+0000","last_became_active":"2024-02-18T00:43:09.798744+0000","last_became_peered":"2024-02-18T00:43:09.798744+0000","last_unstale":"2024-02-18T00:45:10.012311+0000","last_undegraded":"2024-02-18T00:45:10.012311+0000","last_fullsized":"2024-02-18T00:45:10.012311+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"731'34","last_scrub_stamp":"2024-02-18T00:44:40.215762+0000","last_deep_scrub":"731'34","last_deep_scrub_stamp":"2024-02-18T00:44:40.215762+0000","last_clean_scrub_stamp":"2024-02-18T00:44:40.215762+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"457'39","reported_seq":664,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.151690+0000","last_change":"2024-02-18T00:43:35.024037+0000","last_active":"2024-02-18T00:45:09.151690+0000","last_peered":"2024-02-18T00:45:09.151690+0000","last_clean":"2024-02-18T00:45:09.151690+0000","last_became_active":"2024-02-18T00:43:09.791383+0000","last_became_peered":"2024-02-18T00:43:09.791383+0000","last_unstale":"2024-02-18T00:45:09.151690+0000","last_undegraded":"2024-02-18T00:45:09.151690+0000","last_fullsized":"2024-02-18T00:45:09.151690+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:43:35.023935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.023935+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"250.17","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.306000+0000","last_change":"2024-02-18T00:45:03.106485+0000","last_active":"2024-02-18T00:45:09.306000+0000","last_peered":"2024-02-18T00:45:09.306000+0000","last_clean":"2024-02-18T00:45:09.306000+0000","last_became_active":"2024-02-18T00:45:03.106177+0000","last_became_peered":"2024-02-18T00:45:03.106177+0000","last_unstale":"2024-02-18T00:45:09.306000+0000","last_undegraded":"2024-02-18T00:45:09.306000+0000","last_fullsized":"2024-02-18T00:45:09.306000+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.7","version":"745'8","reported_seq":179,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153527+0000","last_change":"2024-02-18T00:44:13.812846+0000","last_active":"2024-02-18T00:45:09.153527+0000","last_peered":"2024-02-18T00:45:09.153527+0000","last_clean":"2024-02-18T00:45:09.153527+0000","last_became_active":"2024-02-18T00:42:19.767995+0000","last_became_peered":"2024-02-18T00:42:19.767995+0000","last_unstale":"2024-02-18T00:45:09.153527+0000","last_undegraded":"2024-02-18T00:45:09.153527+0000","last_fullsized":"2024-02-18T00:45:09.153527+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"250.16","version":"0'0","reported_seq":15,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665147+0000","last_change":"2024-02-18T00:45:03.099977+0000","last_active":"2024-02-18T00:45:09.665147+0000","last_peered":"2024-02-18T00:45:09.665147+0000","last_clean":"2024-02-18T00:45:09.665147+0000","last_became_active":"2024-02-18T00:45:03.099593+0000","last_became_peered":"2024-02-18T00:45:03.099593+0000","last_unstale":"2024-02-18T00:45:09.665147+0000","last_undegraded":"2024-02-18T00:45:09.665147+0000","last_fullsized":"2024-02-18T00:45:09.665147+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"234.6","version":"745'14","reported_seq":194,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731089+0000","last_change":"2024-02-18T00:44:35.346778+0000","last_active":"2024-02-18T00:45:09.731089+0000","last_peered":"2024-02-18T00:45:09.731089+0000","last_clean":"2024-02-18T00:45:09.731089+0000","last_became_active":"2024-02-18T00:42:19.766349+0000","last_became_peered":"2024-02-18T00:42:19.766349+0000","last_unstale":"2024-02-18T00:45:09.731089+0000","last_undegraded":"2024-02-18T00:45:09.731089+0000","last_fullsized":"2024-02-18T00:45:09.731089+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"721'13","last_scrub_stamp":"2024-02-18T00:44:35.346673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.346673+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000021,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.15","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731005+0000","last_change":"2024-02-18T00:45:03.104218+0000","last_active":"2024-02-18T00:45:09.731005+0000","last_peered":"2024-02-18T00:45:09.731005+0000","last_clean":"2024-02-18T00:45:09.731005+0000","last_became_active":"2024-02-18T00:45:03.103914+0000","last_became_peered":"2024-02-18T00:45:03.103914+0000","last_unstale":"2024-02-18T00:45:09.731005+0000","last_undegraded":"2024-02-18T00:45:09.731005+0000","last_fullsized":"2024-02-18T00:45:09.731005+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.5","version":"745'21","reported_seq":206,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822876+0000","last_change":"2024-02-18T00:44:39.133161+0000","last_active":"2024-02-18T00:45:08.822876+0000","last_peered":"2024-02-18T00:45:08.822876+0000","last_clean":"2024-02-18T00:45:08.822876+0000","last_became_active":"2024-02-18T00:42:19.763983+0000","last_became_peered":"2024-02-18T00:42:19.763983+0000","last_unstale":"2024-02-18T00:45:08.822876+0000","last_undegraded":"2024-02-18T00:45:08.822876+0000","last_fullsized":"2024-02-18T00:45:08.822876+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"705'17","last_scrub_stamp":"2024-02-18T00:44:39.133079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:39.133079+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"250.14","version":"0'0","reported_seq":16,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013239+0000","last_change":"2024-02-18T00:45:03.110690+0000","last_active":"2024-02-18T00:45:10.013239+0000","last_peered":"2024-02-18T00:45:10.013239+0000","last_clean":"2024-02-18T00:45:10.013239+0000","last_became_active":"2024-02-18T00:45:03.110452+0000","last_became_peered":"2024-02-18T00:45:03.110452+0000","last_unstale":"2024-02-18T00:45:10.013239+0000","last_undegraded":"2024-02-18T00:45:10.013239+0000","last_fullsized":"2024-02-18T00:45:10.013239+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"234.4","version":"745'3","reported_seq":177,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013341+0000","last_change":"2024-02-18T00:44:13.817133+0000","last_active":"2024-02-18T00:45:10.013341+0000","last_peered":"2024-02-18T00:45:10.013341+0000","last_clean":"2024-02-18T00:45:10.013341+0000","last_became_active":"2024-02-18T00:42:19.767925+0000","last_became_peered":"2024-02-18T00:42:19.767925+0000","last_unstale":"2024-02-18T00:45:10.013341+0000","last_undegraded":"2024-02-18T00:45:10.013341+0000","last_fullsized":"2024-02-18T00:45:10.013341+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"250.13","version":"0'0","reported_seq":15,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152201+0000","last_change":"2024-02-18T00:45:03.098089+0000","last_active":"2024-02-18T00:45:09.152201+0000","last_peered":"2024-02-18T00:45:09.152201+0000","last_clean":"2024-02-18T00:45:09.152201+0000","last_became_active":"2024-02-18T00:45:03.097775+0000","last_became_peered":"2024-02-18T00:45:03.097775+0000","last_unstale":"2024-02-18T00:45:09.152201+0000","last_undegraded":"2024-02-18T00:45:09.152201+0000","last_fullsized":"2024-02-18T00:45:09.152201+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"234.3","version":"745'8","reported_seq":186,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664411+0000","last_change":"2024-02-18T00:44:27.711800+0000","last_active":"2024-02-18T00:45:09.664411+0000","last_peered":"2024-02-18T00:45:09.664411+0000","last_clean":"2024-02-18T00:45:09.664411+0000","last_became_active":"2024-02-18T00:42:19.767739+0000","last_became_peered":"2024-02-18T00:42:19.767739+0000","last_unstale":"2024-02-18T00:45:09.664411+0000","last_undegraded":"2024-02-18T00:45:09.664411+0000","last_fullsized":"2024-02-18T00:45:09.664411+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'5","last_scrub_stamp":"2024-02-18T00:44:27.711710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:27.711710+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"250.12","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.304936+0000","last_change":"2024-02-18T00:45:03.106471+0000","last_active":"2024-02-18T00:45:09.304936+0000","last_peered":"2024-02-18T00:45:09.304936+0000","last_clean":"2024-02-18T00:45:09.304936+0000","last_became_active":"2024-02-18T00:45:03.106134+0000","last_became_peered":"2024-02-18T00:45:03.106134+0000","last_unstale":"2024-02-18T00:45:09.304936+0000","last_undegraded":"2024-02-18T00:45:09.304936+0000","last_fullsized":"2024-02-18T00:45:09.304936+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.2","version":"745'13","reported_seq":195,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730335+0000","last_change":"2024-02-18T00:44:31.444702+0000","last_active":"2024-02-18T00:45:09.730335+0000","last_peered":"2024-02-18T00:45:09.730335+0000","last_clean":"2024-02-18T00:45:09.730335+0000","last_became_active":"2024-02-18T00:42:19.763974+0000","last_became_peered":"2024-02-18T00:42:19.763974+0000","last_unstale":"2024-02-18T00:45:09.730335+0000","last_undegraded":"2024-02-18T00:45:09.730335+0000","last_fullsized":"2024-02-18T00:45:09.730335+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'12","last_scrub_stamp":"2024-02-18T00:44:31.444627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.444627+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"250.11","version":"0'0","reported_seq":17,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011552+0000","last_change":"2024-02-18T00:45:03.103589+0000","last_active":"2024-02-18T00:45:10.011552+0000","last_peered":"2024-02-18T00:45:10.011552+0000","last_clean":"2024-02-18T00:45:10.011552+0000","last_became_active":"2024-02-18T00:45:03.103298+0000","last_became_peered":"2024-02-18T00:45:03.103298+0000","last_unstale":"2024-02-18T00:45:10.011552+0000","last_undegraded":"2024-02-18T00:45:10.011552+0000","last_fullsized":"2024-02-18T00:45:10.011552+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.1","version":"745'11","reported_seq":186,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012362+0000","last_change":"2024-02-18T00:44:13.817532+0000","last_active":"2024-02-18T00:45:10.012362+0000","last_peered":"2024-02-18T00:45:10.012362+0000","last_clean":"2024-02-18T00:45:10.012362+0000","last_became_active":"2024-02-18T00:42:19.763959+0000","last_became_peered":"2024-02-18T00:42:19.763959+0000","last_unstale":"2024-02-18T00:45:10.012362+0000","last_undegraded":"2024-02-18T00:45:10.012362+0000","last_fullsized":"2024-02-18T00:45:10.012362+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":44,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"250.10","version":"0'0","reported_seq":16,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.304717+0000","last_change":"2024-02-18T00:45:03.103463+0000","last_active":"2024-02-18T00:45:09.304717+0000","last_peered":"2024-02-18T00:45:09.304717+0000","last_clean":"2024-02-18T00:45:09.304717+0000","last_became_active":"2024-02-18T00:45:03.102921+0000","last_became_peered":"2024-02-18T00:45:03.102921+0000","last_unstale":"2024-02-18T00:45:09.304717+0000","last_undegraded":"2024-02-18T00:45:09.304717+0000","last_fullsized":"2024-02-18T00:45:09.304717+0000","mapping_epoch":744,"log_start":"0'0","ondisk_log_start":"0'0","created":744,"last_epoch_clean":745,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:02.075600+0000","last_clean_scrub_stamp":"2024-02-18T00:45:02.075600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"234.0","version":"745'12","reported_seq":194,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.815539+0000","last_change":"2024-02-18T00:44:35.784224+0000","last_active":"2024-02-18T00:45:08.815539+0000","last_peered":"2024-02-18T00:45:08.815539+0000","last_clean":"2024-02-18T00:45:08.815539+0000","last_became_active":"2024-02-18T00:42:19.767896+0000","last_became_peered":"2024-02-18T00:42:19.767896+0000","last_unstale":"2024-02-18T00:45:08.815539+0000","last_undegraded":"2024-02-18T00:45:08.815539+0000","last_fullsized":"2024-02-18T00:45:08.815539+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"630'10","last_scrub_stamp":"2024-02-18T00:44:35.784138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.784138+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":686,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730015+0000","last_change":"2024-02-18T00:43:50.556280+0000","last_active":"2024-02-18T00:45:09.730015+0000","last_peered":"2024-02-18T00:45:09.730015+0000","last_clean":"2024-02-18T00:45:09.730015+0000","last_became_active":"2024-02-18T00:39:36.751760+0000","last_became_peered":"2024-02-18T00:39:36.751760+0000","last_unstale":"2024-02-18T00:45:09.730015+0000","last_undegraded":"2024-02-18T00:45:09.730015+0000","last_fullsized":"2024-02-18T00:45:09.730015+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.556181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.556181+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":679,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.009751+0000","last_change":"2024-02-18T00:43:37.205168+0000","last_active":"2024-02-18T00:45:10.009751+0000","last_peered":"2024-02-18T00:45:10.009751+0000","last_clean":"2024-02-18T00:45:10.009751+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:45:10.009751+0000","last_undegraded":"2024-02-18T00:45:10.009751+0000","last_fullsized":"2024-02-18T00:45:10.009751+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:37.205089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.205089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.21","version":"457'34","reported_seq":97,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664168+0000","last_change":"2024-02-18T00:44:41.710983+0000","last_active":"2024-02-18T00:45:09.664168+0000","last_peered":"2024-02-18T00:45:09.664168+0000","last_clean":"2024-02-18T00:45:09.664168+0000","last_became_active":"2024-02-18T00:43:44.817896+0000","last_became_peered":"2024-02-18T00:43:44.817896+0000","last_unstale":"2024-02-18T00:45:09.664168+0000","last_undegraded":"2024-02-18T00:45:09.664168+0000","last_fullsized":"2024-02-18T00:45:09.664168+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:44:41.710910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:41.710910+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153864+0000","last_change":"2024-02-18T00:42:55.766143+0000","last_active":"2024-02-18T00:45:09.153864+0000","last_peered":"2024-02-18T00:45:09.153864+0000","last_clean":"2024-02-18T00:45:09.153864+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:45:09.153864+0000","last_undegraded":"2024-02-18T00:45:09.153864+0000","last_fullsized":"2024-02-18T00:45:09.153864+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:55.766062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:42:55.766062+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.20","version":"457'52","reported_seq":136,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.729864+0000","last_change":"2024-02-18T00:43:52.539836+0000","last_active":"2024-02-18T00:45:09.729864+0000","last_peered":"2024-02-18T00:45:09.729864+0000","last_clean":"2024-02-18T00:45:09.729864+0000","last_became_active":"2024-02-18T00:43:09.796538+0000","last_became_peered":"2024-02-18T00:43:09.796538+0000","last_unstale":"2024-02-18T00:45:09.729864+0000","last_undegraded":"2024-02-18T00:45:09.729864+0000","last_fullsized":"2024-02-18T00:45:09.729864+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:43:52.539763+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.539763+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153501+0000","last_change":"2024-02-18T00:43:10.875551+0000","last_active":"2024-02-18T00:45:09.153501+0000","last_peered":"2024-02-18T00:45:09.153501+0000","last_clean":"2024-02-18T00:45:09.153501+0000","last_became_active":"2024-02-18T00:39:36.760004+0000","last_became_peered":"2024-02-18T00:39:36.760004+0000","last_unstale":"2024-02-18T00:45:09.153501+0000","last_undegraded":"2024-02-18T00:45:09.153501+0000","last_fullsized":"2024-02-18T00:45:09.153501+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:10.875479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:43:10.875479+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.23","version":"734'38","reported_seq":137,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010915+0000","last_change":"2024-02-18T00:44:43.236041+0000","last_active":"2024-02-18T00:45:10.010915+0000","last_peered":"2024-02-18T00:45:10.010915+0000","last_clean":"2024-02-18T00:45:10.010915+0000","last_became_active":"2024-02-18T00:43:09.796063+0000","last_became_peered":"2024-02-18T00:43:09.796063+0000","last_unstale":"2024-02-18T00:45:10.010915+0000","last_undegraded":"2024-02-18T00:45:10.010915+0000","last_fullsized":"2024-02-18T00:45:10.010915+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"734'38","last_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_deep_scrub":"734'38","last_deep_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.235973+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":687,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.817157+0000","last_change":"2024-02-18T00:43:44.566336+0000","last_active":"2024-02-18T00:45:08.817157+0000","last_peered":"2024-02-18T00:45:08.817157+0000","last_clean":"2024-02-18T00:45:08.817157+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:45:08.817157+0000","last_undegraded":"2024-02-18T00:45:08.817157+0000","last_fullsized":"2024-02-18T00:45:08.817157+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:43:44.566261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.566261+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.22","version":"457'32","reported_seq":578,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.815880+0000","last_change":"2024-02-18T00:44:32.659374+0000","last_active":"2024-02-18T00:45:08.815880+0000","last_peered":"2024-02-18T00:45:08.815880+0000","last_clean":"2024-02-18T00:45:08.815880+0000","last_became_active":"2024-02-18T00:43:09.785609+0000","last_became_peered":"2024-02-18T00:43:09.785609+0000","last_unstale":"2024-02-18T00:45:08.815880+0000","last_undegraded":"2024-02-18T00:45:08.815880+0000","last_fullsized":"2024-02-18T00:45:08.815880+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:44:32.659299+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.659299+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":688,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730871+0000","last_change":"2024-02-18T00:43:36.516834+0000","last_active":"2024-02-18T00:45:09.730871+0000","last_peered":"2024-02-18T00:45:09.730871+0000","last_clean":"2024-02-18T00:45:09.730871+0000","last_became_active":"2024-02-18T00:39:36.751486+0000","last_became_peered":"2024-02-18T00:39:36.751486+0000","last_unstale":"2024-02-18T00:45:09.730871+0000","last_undegraded":"2024-02-18T00:45:09.730871+0000","last_fullsized":"2024-02-18T00:45:09.730871+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:36.516704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:36.516704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.25","version":"457'26","reported_seq":134,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664743+0000","last_change":"2024-02-18T00:43:48.727713+0000","last_active":"2024-02-18T00:45:09.664743+0000","last_peered":"2024-02-18T00:45:09.664743+0000","last_clean":"2024-02-18T00:45:09.664743+0000","last_became_active":"2024-02-18T00:43:09.791838+0000","last_became_peered":"2024-02-18T00:43:09.791838+0000","last_unstale":"2024-02-18T00:45:09.664743+0000","last_undegraded":"2024-02-18T00:45:09.664743+0000","last_fullsized":"2024-02-18T00:45:09.664743+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:48.727636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:48.727636+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":661,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010956+0000","last_change":"2024-02-18T00:42:22.876124+0000","last_active":"2024-02-18T00:45:10.010956+0000","last_peered":"2024-02-18T00:45:10.010956+0000","last_clean":"2024-02-18T00:45:10.010956+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:45:10.010956+0000","last_undegraded":"2024-02-18T00:45:10.010956+0000","last_fullsized":"2024-02-18T00:45:10.010956+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.876055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.876055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"457'47","reported_seq":175,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010387+0000","last_change":"2024-02-18T00:44:26.097493+0000","last_active":"2024-02-18T00:45:10.010387+0000","last_peered":"2024-02-18T00:45:10.010387+0000","last_clean":"2024-02-18T00:45:10.010387+0000","last_became_active":"2024-02-18T00:43:09.789985+0000","last_became_peered":"2024-02-18T00:43:09.789985+0000","last_unstale":"2024-02-18T00:45:10.010387+0000","last_undegraded":"2024-02-18T00:45:10.010387+0000","last_fullsized":"2024-02-18T00:45:10.010387+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:44:26.097373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:26.097373+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":675,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730669+0000","last_change":"2024-02-18T00:44:47.426895+0000","last_active":"2024-02-18T00:45:09.730669+0000","last_peered":"2024-02-18T00:45:09.730669+0000","last_clean":"2024-02-18T00:45:09.730669+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:45:09.730669+0000","last_undegraded":"2024-02-18T00:45:09.730669+0000","last_fullsized":"2024-02-18T00:45:09.730669+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.426814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.426814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"457'23","reported_seq":811,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013918+0000","last_change":"2024-02-18T00:44:24.180563+0000","last_active":"2024-02-18T00:45:10.013918+0000","last_peered":"2024-02-18T00:45:10.013918+0000","last_clean":"2024-02-18T00:45:10.013918+0000","last_became_active":"2024-02-18T00:43:09.788606+0000","last_became_peered":"2024-02-18T00:43:09.788606+0000","last_unstale":"2024-02-18T00:45:10.013918+0000","last_undegraded":"2024-02-18T00:45:10.013918+0000","last_fullsized":"2024-02-18T00:45:10.013918+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:44:24.180458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:24.180458+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":758,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822717+0000","last_change":"2024-02-18T00:42:25.770240+0000","last_active":"2024-02-18T00:45:08.822717+0000","last_peered":"2024-02-18T00:45:08.822717+0000","last_clean":"2024-02-18T00:45:08.822717+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:45:08.822717+0000","last_undegraded":"2024-02-18T00:45:08.822717+0000","last_fullsized":"2024-02-18T00:45:08.822717+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_clean_scrub_stamp":"2024-02-18T00:42:25.770128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"457'24","reported_seq":591,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013420+0000","last_change":"2024-02-18T00:43:51.009652+0000","last_active":"2024-02-18T00:45:10.013420+0000","last_peered":"2024-02-18T00:45:10.013420+0000","last_clean":"2024-02-18T00:45:10.013420+0000","last_became_active":"2024-02-18T00:43:09.796496+0000","last_became_peered":"2024-02-18T00:43:09.796496+0000","last_unstale":"2024-02-18T00:45:10.013420+0000","last_undegraded":"2024-02-18T00:45:10.013420+0000","last_fullsized":"2024-02-18T00:45:10.013420+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:43:51.009575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.009575+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152124+0000","last_change":"2024-02-18T00:42:49.664459+0000","last_active":"2024-02-18T00:45:09.152124+0000","last_peered":"2024-02-18T00:45:09.152124+0000","last_clean":"2024-02-18T00:45:09.152124+0000","last_became_active":"2024-02-18T00:39:36.769282+0000","last_became_peered":"2024-02-18T00:39:36.769282+0000","last_unstale":"2024-02-18T00:45:09.152124+0000","last_undegraded":"2024-02-18T00:45:09.152124+0000","last_fullsized":"2024-02-18T00:45:09.152124+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.664380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.664380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":688,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305577+0000","last_change":"2024-02-18T00:43:11.626087+0000","last_active":"2024-02-18T00:45:09.305577+0000","last_peered":"2024-02-18T00:45:09.305577+0000","last_clean":"2024-02-18T00:45:09.305577+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:45:09.305577+0000","last_undegraded":"2024-02-18T00:45:09.305577+0000","last_fullsized":"2024-02-18T00:45:09.305577+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_clean_scrub_stamp":"2024-02-18T00:43:11.626025+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"736'11","reported_seq":372,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152361+0000","last_change":"2024-02-18T00:44:45.266035+0000","last_active":"2024-02-18T00:45:09.152361+0000","last_peered":"2024-02-18T00:45:09.152361+0000","last_clean":"2024-02-18T00:45:09.152361+0000","last_became_active":"2024-02-18T00:39:36.757493+0000","last_became_peered":"2024-02-18T00:39:36.757493+0000","last_unstale":"2024-02-18T00:45:09.152361+0000","last_undegraded":"2024-02-18T00:45:09.152361+0000","last_fullsized":"2024-02-18T00:45:09.152361+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"736'11","last_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_deep_scrub":"736'11","last_deep_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.265972+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":862,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.663891+0000","last_change":"2024-02-18T00:43:43.645878+0000","last_active":"2024-02-18T00:45:09.663891+0000","last_peered":"2024-02-18T00:45:09.663891+0000","last_clean":"2024-02-18T00:45:09.663891+0000","last_became_active":"2024-02-18T00:43:07.792054+0000","last_became_peered":"2024-02-18T00:43:07.792054+0000","last_unstale":"2024-02-18T00:45:09.663891+0000","last_undegraded":"2024-02-18T00:45:09.663891+0000","last_fullsized":"2024-02-18T00:45:09.663891+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:43:43.645800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.645800+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":751,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.823349+0000","last_change":"2024-02-18T00:42:55.614402+0000","last_active":"2024-02-18T00:45:08.823349+0000","last_peered":"2024-02-18T00:45:08.823349+0000","last_clean":"2024-02-18T00:45:08.823349+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:45:08.823349+0000","last_undegraded":"2024-02-18T00:45:08.823349+0000","last_fullsized":"2024-02-18T00:45:08.823349+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:55.614294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:55.614294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":740,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664471+0000","last_change":"2024-02-18T00:44:30.739149+0000","last_active":"2024-02-18T00:45:09.664471+0000","last_peered":"2024-02-18T00:45:09.664471+0000","last_clean":"2024-02-18T00:45:09.664471+0000","last_became_active":"2024-02-18T00:41:45.788049+0000","last_became_peered":"2024-02-18T00:41:45.788049+0000","last_unstale":"2024-02-18T00:45:09.664471+0000","last_undegraded":"2024-02-18T00:45:09.664471+0000","last_fullsized":"2024-02-18T00:45:09.664471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":572,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:30.739087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:30.739087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":681,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013550+0000","last_change":"2024-02-18T00:44:47.967541+0000","last_active":"2024-02-18T00:45:10.013550+0000","last_peered":"2024-02-18T00:45:10.013550+0000","last_clean":"2024-02-18T00:45:10.013550+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:45:10.013550+0000","last_undegraded":"2024-02-18T00:45:10.013550+0000","last_fullsized":"2024-02-18T00:45:10.013550+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.967459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.967459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":786,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305730+0000","last_change":"2024-02-18T00:43:04.471487+0000","last_active":"2024-02-18T00:45:09.305730+0000","last_peered":"2024-02-18T00:45:09.305730+0000","last_clean":"2024-02-18T00:45:09.305730+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:45:09.305730+0000","last_undegraded":"2024-02-18T00:45:09.305730+0000","last_fullsized":"2024-02-18T00:45:09.305730+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:43:04.471412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.471412+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":739,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013483+0000","last_change":"2024-02-18T00:43:43.927240+0000","last_active":"2024-02-18T00:45:10.013483+0000","last_peered":"2024-02-18T00:45:10.013483+0000","last_clean":"2024-02-18T00:45:10.013483+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:45:10.013483+0000","last_undegraded":"2024-02-18T00:45:10.013483+0000","last_fullsized":"2024-02-18T00:45:10.013483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:43.927169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.927169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":727,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820715+0000","last_change":"2024-02-18T00:43:02.696466+0000","last_active":"2024-02-18T00:45:08.820715+0000","last_peered":"2024-02-18T00:45:08.820715+0000","last_clean":"2024-02-18T00:45:08.820715+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:45:08.820715+0000","last_undegraded":"2024-02-18T00:45:08.820715+0000","last_fullsized":"2024-02-18T00:45:08.820715+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:02.696399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.696399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":775,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305515+0000","last_change":"2024-02-18T00:43:35.061129+0000","last_active":"2024-02-18T00:45:09.305515+0000","last_peered":"2024-02-18T00:45:09.305515+0000","last_clean":"2024-02-18T00:45:09.305515+0000","last_became_active":"2024-02-18T00:43:35.060523+0000","last_became_peered":"2024-02-18T00:43:35.060523+0000","last_unstale":"2024-02-18T00:45:09.305515+0000","last_undegraded":"2024-02-18T00:45:09.305515+0000","last_fullsized":"2024-02-18T00:45:09.305515+0000","mapping_epoch":668,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":669,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:33.517952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:33.517952+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":681,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012229+0000","last_change":"2024-02-18T00:44:24.230876+0000","last_active":"2024-02-18T00:45:10.012229+0000","last_peered":"2024-02-18T00:45:10.012229+0000","last_clean":"2024-02-18T00:45:10.012229+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:45:10.012229+0000","last_undegraded":"2024-02-18T00:45:10.012229+0000","last_fullsized":"2024-02-18T00:45:10.012229+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:24.230809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:44:24.230809+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"457'42","reported_seq":866,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730793+0000","last_change":"2024-02-18T00:44:46.441595+0000","last_active":"2024-02-18T00:45:09.730793+0000","last_peered":"2024-02-18T00:45:09.730793+0000","last_clean":"2024-02-18T00:45:09.730793+0000","last_became_active":"2024-02-18T00:39:36.751206+0000","last_became_peered":"2024-02-18T00:39:36.751206+0000","last_unstale":"2024-02-18T00:45:09.730793+0000","last_undegraded":"2024-02-18T00:45:09.730793+0000","last_fullsized":"2024-02-18T00:45:09.730793+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-18T00:44:46.441505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.441505+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":731,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.821878+0000","last_change":"2024-02-18T00:42:53.601598+0000","last_active":"2024-02-18T00:45:08.821878+0000","last_peered":"2024-02-18T00:45:08.821878+0000","last_clean":"2024-02-18T00:45:08.821878+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:45:08.821878+0000","last_undegraded":"2024-02-18T00:45:08.821878+0000","last_fullsized":"2024-02-18T00:45:08.821878+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:53.601533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:42:53.601533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":727,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.821335+0000","last_change":"2024-02-18T00:43:03.732009+0000","last_active":"2024-02-18T00:45:08.821335+0000","last_peered":"2024-02-18T00:45:08.821335+0000","last_clean":"2024-02-18T00:45:08.821335+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:45:08.821335+0000","last_undegraded":"2024-02-18T00:45:08.821335+0000","last_fullsized":"2024-02-18T00:45:08.821335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.731917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.731917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":318,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152279+0000","last_change":"2024-02-18T00:44:43.296521+0000","last_active":"2024-02-18T00:45:09.152279+0000","last_peered":"2024-02-18T00:45:09.152279+0000","last_clean":"2024-02-18T00:45:09.152279+0000","last_became_active":"2024-02-18T00:41:53.837106+0000","last_became_peered":"2024-02-18T00:41:53.837106+0000","last_unstale":"2024-02-18T00:45:09.152279+0000","last_undegraded":"2024-02-18T00:45:09.152279+0000","last_fullsized":"2024-02-18T00:45:09.152279+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.296447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.296447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":663,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664345+0000","last_change":"2024-02-18T00:43:37.722070+0000","last_active":"2024-02-18T00:45:09.664345+0000","last_peered":"2024-02-18T00:45:09.664345+0000","last_clean":"2024-02-18T00:45:09.664345+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:45:09.664345+0000","last_undegraded":"2024-02-18T00:45:09.664345+0000","last_fullsized":"2024-02-18T00:45:09.664345+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:37.722006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.722006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":791,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.817048+0000","last_change":"2024-02-18T00:43:12.336661+0000","last_active":"2024-02-18T00:45:08.817048+0000","last_peered":"2024-02-18T00:45:08.817048+0000","last_clean":"2024-02-18T00:45:08.817048+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:45:08.817048+0000","last_undegraded":"2024-02-18T00:45:08.817048+0000","last_fullsized":"2024-02-18T00:45:08.817048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:43:12.336565+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.336565+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152764+0000","last_change":"2024-02-18T00:43:34.003990+0000","last_active":"2024-02-18T00:45:09.152764+0000","last_peered":"2024-02-18T00:45:09.152764+0000","last_clean":"2024-02-18T00:45:09.152764+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:45:09.152764+0000","last_undegraded":"2024-02-18T00:45:09.152764+0000","last_fullsized":"2024-02-18T00:45:09.152764+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.003922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.003922+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":730,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010698+0000","last_change":"2024-02-18T00:43:03.272338+0000","last_active":"2024-02-18T00:45:10.010698+0000","last_peered":"2024-02-18T00:45:10.010698+0000","last_clean":"2024-02-18T00:45:10.010698+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:45:10.010698+0000","last_undegraded":"2024-02-18T00:45:10.010698+0000","last_fullsized":"2024-02-18T00:45:10.010698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.272278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.272278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":743,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822725+0000","last_change":"2024-02-18T00:43:35.108868+0000","last_active":"2024-02-18T00:45:08.822725+0000","last_peered":"2024-02-18T00:45:08.822725+0000","last_clean":"2024-02-18T00:45:08.822725+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:45:08.822725+0000","last_undegraded":"2024-02-18T00:45:08.822725+0000","last_fullsized":"2024-02-18T00:45:08.822725+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:35.108761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.108761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":142,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.304802+0000","last_change":"2024-02-18T00:44:48.550165+0000","last_active":"2024-02-18T00:45:09.304802+0000","last_peered":"2024-02-18T00:45:09.304802+0000","last_clean":"2024-02-18T00:45:09.304802+0000","last_became_active":"2024-02-18T00:42:38.553174+0000","last_became_peered":"2024-02-18T00:42:38.553174+0000","last_unstale":"2024-02-18T00:45:09.304802+0000","last_undegraded":"2024-02-18T00:45:09.304802+0000","last_fullsized":"2024-02-18T00:45:09.304802+0000","mapping_epoch":623,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:48.550082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:48.550082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":778,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816461+0000","last_change":"2024-02-18T00:43:45.250052+0000","last_active":"2024-02-18T00:45:08.816461+0000","last_peered":"2024-02-18T00:45:08.816461+0000","last_clean":"2024-02-18T00:45:08.816461+0000","last_became_active":"2024-02-18T00:43:44.815508+0000","last_became_peered":"2024-02-18T00:43:44.815508+0000","last_unstale":"2024-02-18T00:45:08.816461+0000","last_undegraded":"2024-02-18T00:45:08.816461+0000","last_fullsized":"2024-02-18T00:45:08.816461+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:42:49.351183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.351183+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":759,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010984+0000","last_change":"2024-02-18T00:44:37.109916+0000","last_active":"2024-02-18T00:45:10.010984+0000","last_peered":"2024-02-18T00:45:10.010984+0000","last_clean":"2024-02-18T00:45:10.010984+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:45:10.010984+0000","last_undegraded":"2024-02-18T00:45:10.010984+0000","last_fullsized":"2024-02-18T00:45:10.010984+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.109800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.109800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":957,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.823056+0000","last_change":"2024-02-18T00:43:32.077585+0000","last_active":"2024-02-18T00:45:08.823056+0000","last_peered":"2024-02-18T00:45:08.823056+0000","last_clean":"2024-02-18T00:45:08.823056+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:45:08.823056+0000","last_undegraded":"2024-02-18T00:45:08.823056+0000","last_fullsized":"2024-02-18T00:45:08.823056+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:43:32.077472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.077472+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":741,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.817081+0000","last_change":"2024-02-18T00:42:52.422173+0000","last_active":"2024-02-18T00:45:08.817081+0000","last_peered":"2024-02-18T00:45:08.817081+0000","last_clean":"2024-02-18T00:45:08.817081+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:45:08.817081+0000","last_undegraded":"2024-02-18T00:45:08.817081+0000","last_fullsized":"2024-02-18T00:45:08.817081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:52.422086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:52.422086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":679,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011262+0000","last_change":"2024-02-18T00:44:45.071900+0000","last_active":"2024-02-18T00:45:10.011262+0000","last_peered":"2024-02-18T00:45:10.011262+0000","last_clean":"2024-02-18T00:45:10.011262+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:45:10.011262+0000","last_undegraded":"2024-02-18T00:45:10.011262+0000","last_fullsized":"2024-02-18T00:45:10.011262+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:45.071835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.071835+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":747,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665436+0000","last_change":"2024-02-18T00:43:12.764242+0000","last_active":"2024-02-18T00:45:09.665436+0000","last_peered":"2024-02-18T00:45:09.665436+0000","last_clean":"2024-02-18T00:45:09.665436+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:45:09.665436+0000","last_undegraded":"2024-02-18T00:45:09.665436+0000","last_fullsized":"2024-02-18T00:45:09.665436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:43:12.764172+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.764172+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":766,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012028+0000","last_change":"2024-02-18T00:44:28.072377+0000","last_active":"2024-02-18T00:45:10.012028+0000","last_peered":"2024-02-18T00:45:10.012028+0000","last_clean":"2024-02-18T00:45:10.012028+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:45:10.012028+0000","last_undegraded":"2024-02-18T00:45:10.012028+0000","last_fullsized":"2024-02-18T00:45:10.012028+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:28.072271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.072271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":924,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.817040+0000","last_change":"2024-02-18T00:42:56.472380+0000","last_active":"2024-02-18T00:45:08.817040+0000","last_peered":"2024-02-18T00:45:08.817040+0000","last_clean":"2024-02-18T00:45:08.817040+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:45:08.817040+0000","last_undegraded":"2024-02-18T00:45:08.817040+0000","last_fullsized":"2024-02-18T00:45:08.817040+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:42:56.472290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.472290+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":735,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665674+0000","last_change":"2024-02-18T00:43:38.758186+0000","last_active":"2024-02-18T00:45:09.665674+0000","last_peered":"2024-02-18T00:45:09.665674+0000","last_clean":"2024-02-18T00:45:09.665674+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:45:09.665674+0000","last_undegraded":"2024-02-18T00:45:09.665674+0000","last_fullsized":"2024-02-18T00:45:09.665674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:38.758071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:43:38.758071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":669,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816124+0000","last_change":"2024-02-18T00:43:36.573041+0000","last_active":"2024-02-18T00:45:08.816124+0000","last_peered":"2024-02-18T00:45:08.816124+0000","last_clean":"2024-02-18T00:45:08.816124+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:45:08.816124+0000","last_undegraded":"2024-02-18T00:45:08.816124+0000","last_fullsized":"2024-02-18T00:45:08.816124+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:36.572964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:43:36.572964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'25","reported_seq":801,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816926+0000","last_change":"2024-02-18T00:44:23.568592+0000","last_active":"2024-02-18T00:45:08.816926+0000","last_peered":"2024-02-18T00:45:08.816926+0000","last_clean":"2024-02-18T00:45:08.816926+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:45:08.816926+0000","last_undegraded":"2024-02-18T00:45:08.816926+0000","last_fullsized":"2024-02-18T00:45:08.816926+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:44:23.568512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:23.568512+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":299,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820143+0000","last_change":"2024-02-18T00:42:59.673905+0000","last_active":"2024-02-18T00:45:08.820143+0000","last_peered":"2024-02-18T00:45:08.820143+0000","last_clean":"2024-02-18T00:45:08.820143+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:45:08.820143+0000","last_undegraded":"2024-02-18T00:45:08.820143+0000","last_fullsized":"2024-02-18T00:45:08.820143+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:59.673841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.673841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":724,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665600+0000","last_change":"2024-02-18T00:44:25.765000+0000","last_active":"2024-02-18T00:45:09.665600+0000","last_peered":"2024-02-18T00:45:09.665600+0000","last_clean":"2024-02-18T00:45:09.665600+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:45:09.665600+0000","last_undegraded":"2024-02-18T00:45:09.665600+0000","last_fullsized":"2024-02-18T00:45:09.665600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:25.764907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.764907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":735,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013506+0000","last_change":"2024-02-18T00:43:42.104431+0000","last_active":"2024-02-18T00:45:10.013506+0000","last_peered":"2024-02-18T00:45:10.013506+0000","last_clean":"2024-02-18T00:45:10.013506+0000","last_became_active":"2024-02-18T00:41:37.801183+0000","last_became_peered":"2024-02-18T00:41:37.801183+0000","last_unstale":"2024-02-18T00:45:10.013506+0000","last_undegraded":"2024-02-18T00:45:10.013506+0000","last_fullsized":"2024-02-18T00:45:10.013506+0000","mapping_epoch":62,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":564,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:43:42.104363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:42.104363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":669,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816675+0000","last_change":"2024-02-18T00:43:00.314164+0000","last_active":"2024-02-18T00:45:08.816675+0000","last_peered":"2024-02-18T00:45:08.816675+0000","last_clean":"2024-02-18T00:45:08.816675+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:45:08.816675+0000","last_undegraded":"2024-02-18T00:45:08.816675+0000","last_fullsized":"2024-02-18T00:45:08.816675+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:00.314091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.314091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":919,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010366+0000","last_change":"2024-02-18T00:43:05.211680+0000","last_active":"2024-02-18T00:45:10.010366+0000","last_peered":"2024-02-18T00:45:10.010366+0000","last_clean":"2024-02-18T00:45:10.010366+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:45:10.010366+0000","last_undegraded":"2024-02-18T00:45:10.010366+0000","last_fullsized":"2024-02-18T00:45:10.010366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_clean_scrub_stamp":"2024-02-18T00:43:05.211604+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":799,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.815934+0000","last_change":"2024-02-18T00:44:28.633151+0000","last_active":"2024-02-18T00:45:08.815934+0000","last_peered":"2024-02-18T00:45:08.815934+0000","last_clean":"2024-02-18T00:45:08.815934+0000","last_became_active":"2024-02-18T00:43:07.790268+0000","last_became_peered":"2024-02-18T00:43:07.790268+0000","last_unstale":"2024-02-18T00:45:08.815934+0000","last_undegraded":"2024-02-18T00:45:08.815934+0000","last_fullsized":"2024-02-18T00:45:08.815934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:44:28.633078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.633078+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":757,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731352+0000","last_change":"2024-02-18T00:42:59.867528+0000","last_active":"2024-02-18T00:45:09.731352+0000","last_peered":"2024-02-18T00:45:09.731352+0000","last_clean":"2024-02-18T00:45:09.731352+0000","last_became_active":"2024-02-18T00:39:36.752210+0000","last_became_peered":"2024-02-18T00:39:36.752210+0000","last_unstale":"2024-02-18T00:45:09.731352+0000","last_undegraded":"2024-02-18T00:45:09.731352+0000","last_fullsized":"2024-02-18T00:45:09.731352+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:59.867452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.867452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":300,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152867+0000","last_change":"2024-02-18T00:43:40.022882+0000","last_active":"2024-02-18T00:45:09.152867+0000","last_peered":"2024-02-18T00:45:09.152867+0000","last_clean":"2024-02-18T00:45:09.152867+0000","last_became_active":"2024-02-18T00:39:36.767278+0000","last_became_peered":"2024-02-18T00:39:36.767278+0000","last_unstale":"2024-02-18T00:45:09.152867+0000","last_undegraded":"2024-02-18T00:45:09.152867+0000","last_fullsized":"2024-02-18T00:45:09.152867+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:40.022788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:40.022788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":663,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665337+0000","last_change":"2024-02-18T00:43:08.832006+0000","last_active":"2024-02-18T00:45:09.665337+0000","last_peered":"2024-02-18T00:45:09.665337+0000","last_clean":"2024-02-18T00:45:09.665337+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:45:09.665337+0000","last_undegraded":"2024-02-18T00:45:09.665337+0000","last_fullsized":"2024-02-18T00:45:09.665337+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.831934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:08.831934+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.831934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":748,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816162+0000","last_change":"2024-02-18T00:44:38.710273+0000","last_active":"2024-02-18T00:45:08.816162+0000","last_peered":"2024-02-18T00:45:08.816162+0000","last_clean":"2024-02-18T00:45:08.816162+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:45:08.816162+0000","last_undegraded":"2024-02-18T00:45:08.816162+0000","last_fullsized":"2024-02-18T00:45:08.816162+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.710196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.710196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":888,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010036+0000","last_change":"2024-02-18T00:44:32.283711+0000","last_active":"2024-02-18T00:45:10.010036+0000","last_peered":"2024-02-18T00:45:10.010036+0000","last_clean":"2024-02-18T00:45:10.010036+0000","last_became_active":"2024-02-18T00:43:07.799144+0000","last_became_peered":"2024-02-18T00:43:07.799144+0000","last_unstale":"2024-02-18T00:45:10.010036+0000","last_undegraded":"2024-02-18T00:45:10.010036+0000","last_fullsized":"2024-02-18T00:45:10.010036+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:44:32.283637+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.283637+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":766,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013630+0000","last_change":"2024-02-18T00:43:34.073871+0000","last_active":"2024-02-18T00:45:10.013630+0000","last_peered":"2024-02-18T00:45:10.013630+0000","last_clean":"2024-02-18T00:45:10.013630+0000","last_became_active":"2024-02-18T00:42:44.039326+0000","last_became_peered":"2024-02-18T00:42:44.039326+0000","last_unstale":"2024-02-18T00:45:10.013630+0000","last_undegraded":"2024-02-18T00:45:10.013630+0000","last_fullsized":"2024-02-18T00:45:10.013630+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.073800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.073800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":723,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816522+0000","last_change":"2024-02-18T00:42:50.371671+0000","last_active":"2024-02-18T00:45:08.816522+0000","last_peered":"2024-02-18T00:45:08.816522+0000","last_clean":"2024-02-18T00:45:08.816522+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:45:08.816522+0000","last_undegraded":"2024-02-18T00:45:08.816522+0000","last_fullsized":"2024-02-18T00:45:08.816522+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:50.371592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.371592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":663,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011765+0000","last_change":"2024-02-18T00:44:31.284979+0000","last_active":"2024-02-18T00:45:10.011765+0000","last_peered":"2024-02-18T00:45:10.011765+0000","last_clean":"2024-02-18T00:45:10.011765+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:45:10.011765+0000","last_undegraded":"2024-02-18T00:45:10.011765+0000","last_fullsized":"2024-02-18T00:45:10.011765+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:31.284910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.284910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":726,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730543+0000","last_change":"2024-02-18T00:44:40.278626+0000","last_active":"2024-02-18T00:45:09.730543+0000","last_peered":"2024-02-18T00:45:09.730543+0000","last_clean":"2024-02-18T00:45:09.730543+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:45:09.730543+0000","last_undegraded":"2024-02-18T00:45:09.730543+0000","last_fullsized":"2024-02-18T00:45:09.730543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:40.278560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:22.964687+0000","last_clean_scrub_stamp":"2024-02-18T00:44:40.278560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":864,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305453+0000","last_change":"2024-02-18T00:43:52.461877+0000","last_active":"2024-02-18T00:45:09.305453+0000","last_peered":"2024-02-18T00:45:09.305453+0000","last_clean":"2024-02-18T00:45:09.305453+0000","last_became_active":"2024-02-18T00:43:07.797505+0000","last_became_peered":"2024-02-18T00:43:07.797505+0000","last_unstale":"2024-02-18T00:45:09.305453+0000","last_undegraded":"2024-02-18T00:45:09.305453+0000","last_fullsized":"2024-02-18T00:45:09.305453+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:43:52.461819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.461819+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":299,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822934+0000","last_change":"2024-02-18T00:43:04.771039+0000","last_active":"2024-02-18T00:45:08.822934+0000","last_peered":"2024-02-18T00:45:08.822934+0000","last_clean":"2024-02-18T00:45:08.822934+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:45:08.822934+0000","last_undegraded":"2024-02-18T00:45:08.822934+0000","last_fullsized":"2024-02-18T00:45:08.822934+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:43:04.770880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.770880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":733,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012076+0000","last_change":"2024-02-18T00:44:38.082951+0000","last_active":"2024-02-18T00:45:10.012076+0000","last_peered":"2024-02-18T00:45:10.012076+0000","last_clean":"2024-02-18T00:45:10.012076+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:45:10.012076+0000","last_undegraded":"2024-02-18T00:45:10.012076+0000","last_fullsized":"2024-02-18T00:45:10.012076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.082853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.082853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":663,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011322+0000","last_change":"2024-02-18T00:44:37.330430+0000","last_active":"2024-02-18T00:45:10.011322+0000","last_peered":"2024-02-18T00:45:10.011322+0000","last_clean":"2024-02-18T00:45:10.011322+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:45:10.011322+0000","last_undegraded":"2024-02-18T00:45:10.011322+0000","last_fullsized":"2024-02-18T00:45:10.011322+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.330361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.330361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":724,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665756+0000","last_change":"2024-02-18T00:44:49.712183+0000","last_active":"2024-02-18T00:45:09.665756+0000","last_peered":"2024-02-18T00:45:09.665756+0000","last_clean":"2024-02-18T00:45:09.665756+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:45:09.665756+0000","last_undegraded":"2024-02-18T00:45:09.665756+0000","last_fullsized":"2024-02-18T00:45:09.665756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:49.712086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:04.700147+0000","last_clean_scrub_stamp":"2024-02-18T00:44:49.712086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'24","reported_seq":776,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305956+0000","last_change":"2024-02-18T00:44:25.533581+0000","last_active":"2024-02-18T00:45:09.305956+0000","last_peered":"2024-02-18T00:45:09.305956+0000","last_clean":"2024-02-18T00:45:09.305956+0000","last_became_active":"2024-02-18T00:43:07.799985+0000","last_became_peered":"2024-02-18T00:43:07.799985+0000","last_unstale":"2024-02-18T00:45:09.305956+0000","last_undegraded":"2024-02-18T00:45:09.305956+0000","last_fullsized":"2024-02-18T00:45:09.305956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:44:25.533453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.533453+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":299,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820262+0000","last_change":"2024-02-18T00:44:37.153891+0000","last_active":"2024-02-18T00:45:08.820262+0000","last_peered":"2024-02-18T00:45:08.820262+0000","last_clean":"2024-02-18T00:45:08.820262+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:45:08.820262+0000","last_undegraded":"2024-02-18T00:45:08.820262+0000","last_fullsized":"2024-02-18T00:45:08.820262+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.153787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.153787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":726,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305478+0000","last_change":"2024-02-18T00:42:53.461135+0000","last_active":"2024-02-18T00:45:09.305478+0000","last_peered":"2024-02-18T00:45:09.305478+0000","last_clean":"2024-02-18T00:45:09.305478+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:45:09.305478+0000","last_undegraded":"2024-02-18T00:45:09.305478+0000","last_fullsized":"2024-02-18T00:45:09.305478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:53.461030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:42:53.461030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":728,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010752+0000","last_change":"2024-02-18T00:43:41.934547+0000","last_active":"2024-02-18T00:45:10.010752+0000","last_peered":"2024-02-18T00:45:10.010752+0000","last_clean":"2024-02-18T00:45:10.010752+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:45:10.010752+0000","last_undegraded":"2024-02-18T00:45:10.010752+0000","last_fullsized":"2024-02-18T00:45:10.010752+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_clean_scrub_stamp":"2024-02-18T00:43:41.934482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":138,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820031+0000","last_change":"2024-02-18T00:43:08.893333+0000","last_active":"2024-02-18T00:45:08.820031+0000","last_peered":"2024-02-18T00:45:08.820031+0000","last_clean":"2024-02-18T00:45:08.820031+0000","last_became_active":"2024-02-18T00:42:44.690928+0000","last_became_peered":"2024-02-18T00:42:44.690928+0000","last_unstale":"2024-02-18T00:45:08.820031+0000","last_undegraded":"2024-02-18T00:45:08.820031+0000","last_fullsized":"2024-02-18T00:45:08.820031+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.893262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.893262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":300,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153332+0000","last_change":"2024-02-18T00:44:33.157482+0000","last_active":"2024-02-18T00:45:09.153332+0000","last_peered":"2024-02-18T00:45:09.153332+0000","last_clean":"2024-02-18T00:45:09.153332+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:45:09.153332+0000","last_undegraded":"2024-02-18T00:45:09.153332+0000","last_fullsized":"2024-02-18T00:45:09.153332+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:33.157367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:44:33.157367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":835,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731048+0000","last_change":"2024-02-18T00:43:51.588473+0000","last_active":"2024-02-18T00:45:09.731048+0000","last_peered":"2024-02-18T00:45:09.731048+0000","last_clean":"2024-02-18T00:45:09.731048+0000","last_became_active":"2024-02-18T00:43:07.792246+0000","last_became_peered":"2024-02-18T00:43:07.792246+0000","last_unstale":"2024-02-18T00:45:09.731048+0000","last_undegraded":"2024-02-18T00:45:09.731048+0000","last_fullsized":"2024-02-18T00:45:09.731048+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:51.588408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.588408+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":300,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152068+0000","last_change":"2024-02-18T00:43:42.067873+0000","last_active":"2024-02-18T00:45:09.152068+0000","last_peered":"2024-02-18T00:45:09.152068+0000","last_clean":"2024-02-18T00:45:09.152068+0000","last_became_active":"2024-02-18T00:39:36.759224+0000","last_became_peered":"2024-02-18T00:39:36.759224+0000","last_unstale":"2024-02-18T00:45:09.152068+0000","last_undegraded":"2024-02-18T00:45:09.152068+0000","last_fullsized":"2024-02-18T00:45:09.152068+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_clean_scrub_stamp":"2024-02-18T00:43:42.067810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":764,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664618+0000","last_change":"2024-02-18T00:43:39.793439+0000","last_active":"2024-02-18T00:45:09.664618+0000","last_peered":"2024-02-18T00:45:09.664618+0000","last_clean":"2024-02-18T00:45:09.664618+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:45:09.664618+0000","last_undegraded":"2024-02-18T00:45:09.664618+0000","last_fullsized":"2024-02-18T00:45:09.664618+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.793345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.793345+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":722,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010497+0000","last_change":"2024-02-18T00:44:44.267245+0000","last_active":"2024-02-18T00:45:10.010497+0000","last_peered":"2024-02-18T00:45:10.010497+0000","last_clean":"2024-02-18T00:45:10.010497+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:45:10.010497+0000","last_undegraded":"2024-02-18T00:45:10.010497+0000","last_fullsized":"2024-02-18T00:45:10.010497+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.267177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.267177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":740,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665761+0000","last_change":"2024-02-18T00:44:32.815550+0000","last_active":"2024-02-18T00:45:09.665761+0000","last_peered":"2024-02-18T00:45:09.665761+0000","last_clean":"2024-02-18T00:45:09.665761+0000","last_became_active":"2024-02-18T00:41:57.791956+0000","last_became_peered":"2024-02-18T00:41:57.791956+0000","last_unstale":"2024-02-18T00:45:09.665761+0000","last_undegraded":"2024-02-18T00:45:09.665761+0000","last_fullsized":"2024-02-18T00:45:09.665761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":584,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:32.815428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.815428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":668,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305428+0000","last_change":"2024-02-18T00:43:05.469702+0000","last_active":"2024-02-18T00:45:09.305428+0000","last_peered":"2024-02-18T00:45:09.305428+0000","last_clean":"2024-02-18T00:45:09.305428+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:45:09.305428+0000","last_undegraded":"2024-02-18T00:45:09.305428+0000","last_fullsized":"2024-02-18T00:45:09.305428+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_clean_scrub_stamp":"2024-02-18T00:43:05.469634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":745,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.823581+0000","last_change":"2024-02-18T00:42:58.721654+0000","last_active":"2024-02-18T00:45:08.823581+0000","last_peered":"2024-02-18T00:45:08.823581+0000","last_clean":"2024-02-18T00:45:08.823581+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:45:08.823581+0000","last_undegraded":"2024-02-18T00:45:08.823581+0000","last_fullsized":"2024-02-18T00:45:08.823581+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:58.721587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:58.721587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"680'27","reported_seq":801,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.014083+0000","last_change":"2024-02-18T00:43:45.989486+0000","last_active":"2024-02-18T00:45:10.014083+0000","last_peered":"2024-02-18T00:45:10.014083+0000","last_clean":"2024-02-18T00:45:10.014083+0000","last_became_active":"2024-02-18T00:43:07.801362+0000","last_became_peered":"2024-02-18T00:43:07.801362+0000","last_unstale":"2024-02-18T00:45:10.014083+0000","last_undegraded":"2024-02-18T00:45:10.014083+0000","last_fullsized":"2024-02-18T00:45:10.014083+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"680'27","last_scrub_stamp":"2024-02-18T00:43:45.989427+0000","last_deep_scrub":"680'27","last_deep_scrub_stamp":"2024-02-18T00:43:45.989427+0000","last_clean_scrub_stamp":"2024-02-18T00:43:45.989427+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":740,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822397+0000","last_change":"2024-02-18T00:42:49.673980+0000","last_active":"2024-02-18T00:45:08.822397+0000","last_peered":"2024-02-18T00:45:08.822397+0000","last_clean":"2024-02-18T00:45:08.822397+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:45:08.822397+0000","last_undegraded":"2024-02-18T00:45:08.822397+0000","last_fullsized":"2024-02-18T00:45:08.822397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.673915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.673915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1688,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.569489+0000","last_change":"2024-02-18T00:43:04.252649+0000","last_active":"2024-02-18T00:45:10.569489+0000","last_peered":"2024-02-18T00:45:10.569489+0000","last_clean":"2024-02-18T00:45:10.569489+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:45:10.569489+0000","last_undegraded":"2024-02-18T00:45:10.569489+0000","last_fullsized":"2024-02-18T00:45:10.569489+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:43:04.252540+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.252540+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":946,"num_read_kb":941,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":299,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153372+0000","last_change":"2024-02-18T00:43:12.839041+0000","last_active":"2024-02-18T00:45:09.153372+0000","last_peered":"2024-02-18T00:45:09.153372+0000","last_clean":"2024-02-18T00:45:09.153372+0000","last_became_active":"2024-02-18T00:39:36.764502+0000","last_became_peered":"2024-02-18T00:39:36.764502+0000","last_unstale":"2024-02-18T00:45:09.153372+0000","last_undegraded":"2024-02-18T00:45:09.153372+0000","last_fullsized":"2024-02-18T00:45:09.153372+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:43:12.838938+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:43:12.838938+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.838938+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":687,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816205+0000","last_change":"2024-02-18T00:43:52.430089+0000","last_active":"2024-02-18T00:45:08.816205+0000","last_peered":"2024-02-18T00:45:08.816205+0000","last_clean":"2024-02-18T00:45:08.816205+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:45:08.816205+0000","last_undegraded":"2024-02-18T00:45:08.816205+0000","last_fullsized":"2024-02-18T00:45:08.816205+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.430010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":723,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305813+0000","last_change":"2024-02-18T00:42:58.421119+0000","last_active":"2024-02-18T00:45:09.305813+0000","last_peered":"2024-02-18T00:45:09.305813+0000","last_clean":"2024-02-18T00:45:09.305813+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:45:09.305813+0000","last_undegraded":"2024-02-18T00:45:09.305813+0000","last_fullsized":"2024-02-18T00:45:09.305813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:58.421052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:58.421052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"682'49","reported_seq":690,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665600+0000","last_change":"2024-02-18T00:44:35.869910+0000","last_active":"2024-02-18T00:45:09.665600+0000","last_peered":"2024-02-18T00:45:09.665600+0000","last_clean":"2024-02-18T00:45:09.665600+0000","last_became_active":"2024-02-18T00:43:44.816299+0000","last_became_peered":"2024-02-18T00:43:44.816299+0000","last_unstale":"2024-02-18T00:45:09.665600+0000","last_undegraded":"2024-02-18T00:45:09.665600+0000","last_fullsized":"2024-02-18T00:45:09.665600+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":0,"last_scrub":"682'49","last_scrub_stamp":"2024-02-18T00:44:35.869834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.869834+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":248,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":737,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664906+0000","last_change":"2024-02-18T00:43:51.722733+0000","last_active":"2024-02-18T00:45:09.664906+0000","last_peered":"2024-02-18T00:45:09.664906+0000","last_clean":"2024-02-18T00:45:09.664906+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:45:09.664906+0000","last_undegraded":"2024-02-18T00:45:09.664906+0000","last_fullsized":"2024-02-18T00:45:09.664906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.722660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.722660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":748,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011507+0000","last_change":"2024-02-18T00:42:19.938250+0000","last_active":"2024-02-18T00:45:10.011507+0000","last_peered":"2024-02-18T00:45:10.011507+0000","last_clean":"2024-02-18T00:45:10.011507+0000","last_became_active":"2024-02-18T00:42:02.125704+0000","last_became_peered":"2024-02-18T00:42:02.125704+0000","last_unstale":"2024-02-18T00:45:10.011507+0000","last_undegraded":"2024-02-18T00:45:10.011507+0000","last_fullsized":"2024-02-18T00:45:10.011507+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.938182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.938182+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":668,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.306205+0000","last_change":"2024-02-18T00:43:51.459881+0000","last_active":"2024-02-18T00:45:09.306205+0000","last_peered":"2024-02-18T00:45:09.306205+0000","last_clean":"2024-02-18T00:45:09.306205+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:45:09.306205+0000","last_undegraded":"2024-02-18T00:45:09.306205+0000","last_fullsized":"2024-02-18T00:45:09.306205+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.459750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.459750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":922,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305902+0000","last_change":"2024-02-18T00:43:35.528634+0000","last_active":"2024-02-18T00:45:09.305902+0000","last_peered":"2024-02-18T00:45:09.305902+0000","last_clean":"2024-02-18T00:45:09.305902+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:45:09.305902+0000","last_undegraded":"2024-02-18T00:45:09.305902+0000","last_fullsized":"2024-02-18T00:45:09.305902+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:43:35.528567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.528567+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":755,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820899+0000","last_change":"2024-02-18T00:43:51.043051+0000","last_active":"2024-02-18T00:45:08.820899+0000","last_peered":"2024-02-18T00:45:08.820899+0000","last_clean":"2024-02-18T00:45:08.820899+0000","last_became_active":"2024-02-18T00:43:07.795433+0000","last_became_peered":"2024-02-18T00:43:07.795433+0000","last_unstale":"2024-02-18T00:45:08.820899+0000","last_undegraded":"2024-02-18T00:45:08.820899+0000","last_fullsized":"2024-02-18T00:45:08.820899+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:43:51.042989+0000","last_deep_scrub":"457'32","last_deep_scrub_stamp":"2024-02-18T00:43:51.042989+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.042989+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":771,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.010626+0000","last_change":"2024-02-18T00:44:43.113419+0000","last_active":"2024-02-18T00:45:10.010626+0000","last_peered":"2024-02-18T00:45:10.010626+0000","last_clean":"2024-02-18T00:45:10.010626+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:45:10.010626+0000","last_undegraded":"2024-02-18T00:45:10.010626+0000","last_fullsized":"2024-02-18T00:45:10.010626+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.113292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":721,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305195+0000","last_change":"2024-02-18T00:44:44.469481+0000","last_active":"2024-02-18T00:45:09.305195+0000","last_peered":"2024-02-18T00:45:09.305195+0000","last_clean":"2024-02-18T00:45:09.305195+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:45:09.305195+0000","last_undegraded":"2024-02-18T00:45:09.305195+0000","last_fullsized":"2024-02-18T00:45:09.305195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.469413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.469413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":669,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816639+0000","last_change":"2024-02-18T00:43:09.318894+0000","last_active":"2024-02-18T00:45:08.816639+0000","last_peered":"2024-02-18T00:45:08.816639+0000","last_clean":"2024-02-18T00:45:08.816639+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:45:08.816639+0000","last_undegraded":"2024-02-18T00:45:08.816639+0000","last_fullsized":"2024-02-18T00:45:08.816639+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.318803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.318803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":727,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820458+0000","last_change":"2024-02-18T00:43:44.168202+0000","last_active":"2024-02-18T00:45:08.820458+0000","last_peered":"2024-02-18T00:45:08.820458+0000","last_clean":"2024-02-18T00:45:08.820458+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:45:08.820458+0000","last_undegraded":"2024-02-18T00:45:08.820458+0000","last_fullsized":"2024-02-18T00:45:08.820458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.168096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.168096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":804,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.815992+0000","last_change":"2024-02-18T00:44:27.619378+0000","last_active":"2024-02-18T00:45:08.815992+0000","last_peered":"2024-02-18T00:45:08.815992+0000","last_clean":"2024-02-18T00:45:08.815992+0000","last_became_active":"2024-02-18T00:43:07.795322+0000","last_became_peered":"2024-02-18T00:43:07.795322+0000","last_unstale":"2024-02-18T00:45:08.815992+0000","last_undegraded":"2024-02-18T00:45:08.815992+0000","last_fullsized":"2024-02-18T00:45:08.815992+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:44:27.619298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:27.619298+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":782,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.817246+0000","last_change":"2024-02-18T00:44:47.741854+0000","last_active":"2024-02-18T00:45:08.817246+0000","last_peered":"2024-02-18T00:45:08.817246+0000","last_clean":"2024-02-18T00:45:08.817246+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:45:08.817246+0000","last_undegraded":"2024-02-18T00:45:08.817246+0000","last_fullsized":"2024-02-18T00:45:08.817246+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:44:47.741787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.741787+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":753,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011164+0000","last_change":"2024-02-18T00:43:01.098679+0000","last_active":"2024-02-18T00:45:10.011164+0000","last_peered":"2024-02-18T00:45:10.011164+0000","last_clean":"2024-02-18T00:45:10.011164+0000","last_became_active":"2024-02-18T00:41:42.776144+0000","last_became_peered":"2024-02-18T00:41:42.776144+0000","last_unstale":"2024-02-18T00:45:10.011164+0000","last_undegraded":"2024-02-18T00:45:10.011164+0000","last_fullsized":"2024-02-18T00:45:10.011164+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:01.098569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.098569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":669,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816178+0000","last_change":"2024-02-18T00:43:50.444348+0000","last_active":"2024-02-18T00:45:08.816178+0000","last_peered":"2024-02-18T00:45:08.816178+0000","last_clean":"2024-02-18T00:45:08.816178+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:45:08.816178+0000","last_undegraded":"2024-02-18T00:45:08.816178+0000","last_fullsized":"2024-02-18T00:45:08.816178+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.444279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":826,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.306034+0000","last_change":"2024-02-18T00:43:02.517912+0000","last_active":"2024-02-18T00:45:09.306034+0000","last_peered":"2024-02-18T00:45:09.306034+0000","last_clean":"2024-02-18T00:45:09.306034+0000","last_became_active":"2024-02-18T00:39:36.771394+0000","last_became_peered":"2024-02-18T00:39:36.771394+0000","last_unstale":"2024-02-18T00:45:09.306034+0000","last_undegraded":"2024-02-18T00:45:09.306034+0000","last_fullsized":"2024-02-18T00:45:09.306034+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:43:02.517835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.517835+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":731,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.820372+0000","last_change":"2024-02-18T00:43:34.111205+0000","last_active":"2024-02-18T00:45:08.820372+0000","last_peered":"2024-02-18T00:45:08.820372+0000","last_clean":"2024-02-18T00:45:08.820372+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:45:08.820372+0000","last_undegraded":"2024-02-18T00:45:08.820372+0000","last_fullsized":"2024-02-18T00:45:08.820372+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.111128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.111128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":737,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012126+0000","last_change":"2024-02-18T00:43:09.212023+0000","last_active":"2024-02-18T00:45:10.012126+0000","last_peered":"2024-02-18T00:45:10.012126+0000","last_clean":"2024-02-18T00:45:10.012126+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:45:10.012126+0000","last_undegraded":"2024-02-18T00:45:10.012126+0000","last_fullsized":"2024-02-18T00:45:10.012126+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.211956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.211956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":745,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.821634+0000","last_change":"2024-02-18T00:44:42.127223+0000","last_active":"2024-02-18T00:45:08.821634+0000","last_peered":"2024-02-18T00:45:08.821634+0000","last_clean":"2024-02-18T00:45:08.821634+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:45:08.821634+0000","last_undegraded":"2024-02-18T00:45:08.821634+0000","last_fullsized":"2024-02-18T00:45:08.821634+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:42.127148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:44:42.127148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":668,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305188+0000","last_change":"2024-02-18T00:44:37.525779+0000","last_active":"2024-02-18T00:45:09.305188+0000","last_peered":"2024-02-18T00:45:09.305188+0000","last_clean":"2024-02-18T00:45:09.305188+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:45:09.305188+0000","last_undegraded":"2024-02-18T00:45:09.305188+0000","last_fullsized":"2024-02-18T00:45:09.305188+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.525709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.525709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":814,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012013+0000","last_change":"2024-02-18T00:44:50.318205+0000","last_active":"2024-02-18T00:45:10.012013+0000","last_peered":"2024-02-18T00:45:10.012013+0000","last_clean":"2024-02-18T00:45:10.012013+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:45:10.012013+0000","last_undegraded":"2024-02-18T00:45:10.012013+0000","last_fullsized":"2024-02-18T00:45:10.012013+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'18","last_scrub_stamp":"2024-02-18T00:42:08.844930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:08.844930+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":735,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730189+0000","last_change":"2024-02-18T00:44:43.359587+0000","last_active":"2024-02-18T00:45:09.730189+0000","last_peered":"2024-02-18T00:45:09.730189+0000","last_clean":"2024-02-18T00:45:09.730189+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:45:09.730189+0000","last_undegraded":"2024-02-18T00:45:09.730189+0000","last_fullsized":"2024-02-18T00:45:09.730189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.359510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.359510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":724,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665179+0000","last_change":"2024-02-18T00:44:45.666045+0000","last_active":"2024-02-18T00:45:09.665179+0000","last_peered":"2024-02-18T00:45:09.665179+0000","last_clean":"2024-02-18T00:45:09.665179+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:45:09.665179+0000","last_undegraded":"2024-02-18T00:45:09.665179+0000","last_fullsized":"2024-02-18T00:45:09.665179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:45.665981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.665981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153386+0000","last_change":"2024-02-18T00:43:08.921418+0000","last_active":"2024-02-18T00:45:09.153386+0000","last_peered":"2024-02-18T00:45:09.153386+0000","last_clean":"2024-02-18T00:45:09.153386+0000","last_became_active":"2024-02-18T00:39:36.759264+0000","last_became_peered":"2024-02-18T00:39:36.759264+0000","last_unstale":"2024-02-18T00:45:09.153386+0000","last_undegraded":"2024-02-18T00:45:09.153386+0000","last_fullsized":"2024-02-18T00:45:09.153386+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.921352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.921352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":680,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822651+0000","last_change":"2024-02-18T00:44:47.201367+0000","last_active":"2024-02-18T00:45:08.822651+0000","last_peered":"2024-02-18T00:45:08.822651+0000","last_clean":"2024-02-18T00:45:08.822651+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:45:08.822651+0000","last_undegraded":"2024-02-18T00:45:08.822651+0000","last_fullsized":"2024-02-18T00:45:08.822651+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.201293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.201293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":831,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013675+0000","last_change":"2024-02-18T00:43:41.139904+0000","last_active":"2024-02-18T00:45:10.013675+0000","last_peered":"2024-02-18T00:45:10.013675+0000","last_clean":"2024-02-18T00:45:10.013675+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:45:10.013675+0000","last_undegraded":"2024-02-18T00:45:10.013675+0000","last_fullsized":"2024-02-18T00:45:10.013675+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:43:41.139829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:41.139829+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":755,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664749+0000","last_change":"2024-02-18T00:44:46.634819+0000","last_active":"2024-02-18T00:45:09.664749+0000","last_peered":"2024-02-18T00:45:09.664749+0000","last_clean":"2024-02-18T00:45:09.664749+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:45:09.664749+0000","last_undegraded":"2024-02-18T00:45:09.664749+0000","last_fullsized":"2024-02-18T00:45:09.664749+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:46.634734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.634734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":924,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665516+0000","last_change":"2024-02-18T00:44:47.651515+0000","last_active":"2024-02-18T00:45:09.665516+0000","last_peered":"2024-02-18T00:45:09.665516+0000","last_clean":"2024-02-18T00:45:09.665516+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:45:09.665516+0000","last_undegraded":"2024-02-18T00:45:09.665516+0000","last_fullsized":"2024-02-18T00:45:09.665516+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:44:47.651424+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:44:47.651424+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.651424+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":688,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730077+0000","last_change":"2024-02-18T00:44:42.310455+0000","last_active":"2024-02-18T00:45:09.730077+0000","last_peered":"2024-02-18T00:45:09.730077+0000","last_clean":"2024-02-18T00:45:09.730077+0000","last_became_active":"2024-02-18T00:39:36.751162+0000","last_became_peered":"2024-02-18T00:39:36.751162+0000","last_unstale":"2024-02-18T00:45:09.730077+0000","last_undegraded":"2024-02-18T00:45:09.730077+0000","last_fullsized":"2024-02-18T00:45:09.730077+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:42.310377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:42.310377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":803,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665345+0000","last_change":"2024-02-18T00:43:03.706116+0000","last_active":"2024-02-18T00:45:09.665345+0000","last_peered":"2024-02-18T00:45:09.665345+0000","last_clean":"2024-02-18T00:45:09.665345+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:45:09.665345+0000","last_undegraded":"2024-02-18T00:45:09.665345+0000","last_fullsized":"2024-02-18T00:45:09.665345+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:43:03.706028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.706028+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":727,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305555+0000","last_change":"2024-02-18T00:42:57.465908+0000","last_active":"2024-02-18T00:45:09.305555+0000","last_peered":"2024-02-18T00:45:09.305555+0000","last_clean":"2024-02-18T00:45:09.305555+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:45:09.305555+0000","last_undegraded":"2024-02-18T00:45:09.305555+0000","last_fullsized":"2024-02-18T00:45:09.305555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:57.465836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:42:57.465836+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":945,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013874+0000","last_change":"2024-02-18T00:43:00.081109+0000","last_active":"2024-02-18T00:45:10.013874+0000","last_peered":"2024-02-18T00:45:10.013874+0000","last_clean":"2024-02-18T00:45:10.013874+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:45:10.013874+0000","last_undegraded":"2024-02-18T00:45:10.013874+0000","last_fullsized":"2024-02-18T00:45:10.013874+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:43:00.080987+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.080987+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":668,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665844+0000","last_change":"2024-02-18T00:44:48.698039+0000","last_active":"2024-02-18T00:45:09.665844+0000","last_peered":"2024-02-18T00:45:09.665844+0000","last_clean":"2024-02-18T00:45:09.665844+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:45:09.665844+0000","last_undegraded":"2024-02-18T00:45:09.665844+0000","last_fullsized":"2024-02-18T00:45:09.665844+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:48.697958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:44:48.697958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":759,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011030+0000","last_change":"2024-02-18T00:42:50.108975+0000","last_active":"2024-02-18T00:45:10.011030+0000","last_peered":"2024-02-18T00:45:10.011030+0000","last_clean":"2024-02-18T00:45:10.011030+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:45:10.011030+0000","last_undegraded":"2024-02-18T00:45:10.011030+0000","last_fullsized":"2024-02-18T00:45:10.011030+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:42:50.108893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.108893+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":735,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731088+0000","last_change":"2024-02-18T00:44:36.372427+0000","last_active":"2024-02-18T00:45:09.731088+0000","last_peered":"2024-02-18T00:45:09.731088+0000","last_clean":"2024-02-18T00:45:09.731088+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:45:09.731088+0000","last_undegraded":"2024-02-18T00:45:09.731088+0000","last_fullsized":"2024-02-18T00:45:09.731088+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:36.372315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:36.372315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":719,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011113+0000","last_change":"2024-02-18T00:42:56.033487+0000","last_active":"2024-02-18T00:45:10.011113+0000","last_peered":"2024-02-18T00:45:10.011113+0000","last_clean":"2024-02-18T00:45:10.011113+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:45:10.011113+0000","last_undegraded":"2024-02-18T00:45:10.011113+0000","last_fullsized":"2024-02-18T00:45:10.011113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:56.033419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.033419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":726,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.304822+0000","last_change":"2024-02-18T00:44:38.484355+0000","last_active":"2024-02-18T00:45:09.304822+0000","last_peered":"2024-02-18T00:45:09.304822+0000","last_clean":"2024-02-18T00:45:09.304822+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:45:09.304822+0000","last_undegraded":"2024-02-18T00:45:09.304822+0000","last_fullsized":"2024-02-18T00:45:09.304822+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.484287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.484287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":670,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.305821+0000","last_change":"2024-02-18T00:44:49.583138+0000","last_active":"2024-02-18T00:45:09.305821+0000","last_peered":"2024-02-18T00:45:09.305821+0000","last_clean":"2024-02-18T00:45:09.305821+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:45:09.305821+0000","last_undegraded":"2024-02-18T00:45:09.305821+0000","last_fullsized":"2024-02-18T00:45:09.305821+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:49.583035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_clean_scrub_stamp":"2024-02-18T00:44:49.583035+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":763,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816445+0000","last_change":"2024-02-18T00:44:41.703555+0000","last_active":"2024-02-18T00:45:08.816445+0000","last_peered":"2024-02-18T00:45:08.816445+0000","last_clean":"2024-02-18T00:45:08.816445+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:45:08.816445+0000","last_undegraded":"2024-02-18T00:45:08.816445+0000","last_fullsized":"2024-02-18T00:45:08.816445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:44:41.703456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:41.703456+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":734,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013402+0000","last_change":"2024-02-18T00:43:09.187333+0000","last_active":"2024-02-18T00:45:10.013402+0000","last_peered":"2024-02-18T00:45:10.013402+0000","last_clean":"2024-02-18T00:45:10.013402+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:45:10.013402+0000","last_undegraded":"2024-02-18T00:45:10.013402+0000","last_fullsized":"2024-02-18T00:45:10.013402+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.187270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.187270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":735,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011124+0000","last_change":"2024-02-18T00:43:39.958283+0000","last_active":"2024-02-18T00:45:10.011124+0000","last_peered":"2024-02-18T00:45:10.011124+0000","last_clean":"2024-02-18T00:45:10.011124+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:45:10.011124+0000","last_undegraded":"2024-02-18T00:45:10.011124+0000","last_fullsized":"2024-02-18T00:45:10.011124+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.958201+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":717,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012446+0000","last_change":"2024-02-18T00:43:40.107832+0000","last_active":"2024-02-18T00:45:10.012446+0000","last_peered":"2024-02-18T00:45:10.012446+0000","last_clean":"2024-02-18T00:45:10.012446+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:45:10.012446+0000","last_undegraded":"2024-02-18T00:45:10.012446+0000","last_fullsized":"2024-02-18T00:45:10.012446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:40.107757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:40.107757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":665,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731268+0000","last_change":"2024-02-18T00:42:56.820557+0000","last_active":"2024-02-18T00:45:09.731268+0000","last_peered":"2024-02-18T00:45:09.731268+0000","last_clean":"2024-02-18T00:45:09.731268+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:45:09.731268+0000","last_undegraded":"2024-02-18T00:45:09.731268+0000","last_fullsized":"2024-02-18T00:45:09.731268+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:56.820451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.820451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":733,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.009892+0000","last_change":"2024-02-18T00:43:02.133961+0000","last_active":"2024-02-18T00:45:10.009892+0000","last_peered":"2024-02-18T00:45:10.009892+0000","last_clean":"2024-02-18T00:45:10.009892+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:45:10.009892+0000","last_undegraded":"2024-02-18T00:45:10.009892+0000","last_fullsized":"2024-02-18T00:45:10.009892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-18T00:43:02.133854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.133854+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":728,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665324+0000","last_change":"2024-02-18T00:43:07.782401+0000","last_active":"2024-02-18T00:45:09.665324+0000","last_peered":"2024-02-18T00:45:09.665324+0000","last_clean":"2024-02-18T00:45:09.665324+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:45:09.665324+0000","last_undegraded":"2024-02-18T00:45:09.665324+0000","last_fullsized":"2024-02-18T00:45:09.665324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_clean_scrub_stamp":"2024-02-18T00:43:07.782283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":744,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730185+0000","last_change":"2024-02-18T00:43:44.575490+0000","last_active":"2024-02-18T00:45:09.730185+0000","last_peered":"2024-02-18T00:45:09.730185+0000","last_clean":"2024-02-18T00:45:09.730185+0000","last_became_active":"2024-02-18T00:39:36.750866+0000","last_became_peered":"2024-02-18T00:39:36.750866+0000","last_unstale":"2024-02-18T00:45:09.730185+0000","last_undegraded":"2024-02-18T00:45:09.730185+0000","last_fullsized":"2024-02-18T00:45:09.730185+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.575417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.575417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":300,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152467+0000","last_change":"2024-02-18T00:44:28.186498+0000","last_active":"2024-02-18T00:45:09.152467+0000","last_peered":"2024-02-18T00:45:09.152467+0000","last_clean":"2024-02-18T00:45:09.152467+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:45:09.152467+0000","last_undegraded":"2024-02-18T00:45:09.152467+0000","last_fullsized":"2024-02-18T00:45:09.152467+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:28.186417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.186417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":681,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.665664+0000","last_change":"2024-02-18T00:43:32.818797+0000","last_active":"2024-02-18T00:45:09.665664+0000","last_peered":"2024-02-18T00:45:09.665664+0000","last_clean":"2024-02-18T00:45:09.665664+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:45:09.665664+0000","last_undegraded":"2024-02-18T00:45:09.665664+0000","last_fullsized":"2024-02-18T00:45:09.665664+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.818709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.818709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":813,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012916+0000","last_change":"2024-02-18T00:44:28.251258+0000","last_active":"2024-02-18T00:45:10.012916+0000","last_peered":"2024-02-18T00:45:10.012916+0000","last_clean":"2024-02-18T00:45:10.012916+0000","last_became_active":"2024-02-18T00:39:36.761140+0000","last_became_peered":"2024-02-18T00:39:36.761140+0000","last_unstale":"2024-02-18T00:45:10.012916+0000","last_undegraded":"2024-02-18T00:45:10.012916+0000","last_fullsized":"2024-02-18T00:45:10.012916+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:44:28.251169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.251169+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":730,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731247+0000","last_change":"2024-02-18T00:43:03.852309+0000","last_active":"2024-02-18T00:45:09.731247+0000","last_peered":"2024-02-18T00:45:09.731247+0000","last_clean":"2024-02-18T00:45:09.731247+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:45:09.731247+0000","last_undegraded":"2024-02-18T00:45:09.731247+0000","last_fullsized":"2024-02-18T00:45:09.731247+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.852193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152324+0000","last_change":"2024-02-18T00:44:23.214099+0000","last_active":"2024-02-18T00:45:09.152324+0000","last_peered":"2024-02-18T00:45:09.152324+0000","last_clean":"2024-02-18T00:45:09.152324+0000","last_became_active":"2024-02-18T00:39:36.758138+0000","last_became_peered":"2024-02-18T00:39:36.758138+0000","last_unstale":"2024-02-18T00:45:09.152324+0000","last_undegraded":"2024-02-18T00:45:09.152324+0000","last_fullsized":"2024-02-18T00:45:09.152324+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:23.214010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:23.214010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":717,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011238+0000","last_change":"2024-02-18T00:43:32.135279+0000","last_active":"2024-02-18T00:45:10.011238+0000","last_peered":"2024-02-18T00:45:10.011238+0000","last_clean":"2024-02-18T00:45:10.011238+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:45:10.011238+0000","last_undegraded":"2024-02-18T00:45:10.011238+0000","last_fullsized":"2024-02-18T00:45:10.011238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.135214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.135214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":300,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153002+0000","last_change":"2024-02-18T00:43:44.122130+0000","last_active":"2024-02-18T00:45:09.153002+0000","last_peered":"2024-02-18T00:45:09.153002+0000","last_clean":"2024-02-18T00:45:09.153002+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:45:09.153002+0000","last_undegraded":"2024-02-18T00:45:09.153002+0000","last_fullsized":"2024-02-18T00:45:09.153002+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.122056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:44.122056+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.122056+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":827,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.306133+0000","last_change":"2024-02-18T00:44:34.576745+0000","last_active":"2024-02-18T00:45:09.306133+0000","last_peered":"2024-02-18T00:45:09.306133+0000","last_clean":"2024-02-18T00:45:09.306133+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:45:09.306133+0000","last_undegraded":"2024-02-18T00:45:09.306133+0000","last_fullsized":"2024-02-18T00:45:09.306133+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:44:34.576675+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:44:34.576675+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":750,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664136+0000","last_change":"2024-02-18T00:43:10.797905+0000","last_active":"2024-02-18T00:45:09.664136+0000","last_peered":"2024-02-18T00:45:09.664136+0000","last_clean":"2024-02-18T00:45:09.664136+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:45:09.664136+0000","last_undegraded":"2024-02-18T00:45:09.664136+0000","last_fullsized":"2024-02-18T00:45:09.664136+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:10.797791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:10.797791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"466'10","reported_seq":959,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.731434+0000","last_change":"2024-02-18T00:43:37.546013+0000","last_active":"2024-02-18T00:45:09.731434+0000","last_peered":"2024-02-18T00:45:09.731434+0000","last_clean":"2024-02-18T00:45:09.731434+0000","last_became_active":"2024-02-18T00:39:36.751716+0000","last_became_peered":"2024-02-18T00:39:36.751716+0000","last_unstale":"2024-02-18T00:45:09.731434+0000","last_undegraded":"2024-02-18T00:45:09.731434+0000","last_fullsized":"2024-02-18T00:45:09.731434+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:43:37.545939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.545939+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":300,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152361+0000","last_change":"2024-02-18T00:43:51.101862+0000","last_active":"2024-02-18T00:45:09.152361+0000","last_peered":"2024-02-18T00:45:09.152361+0000","last_clean":"2024-02-18T00:45:09.152361+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:45:09.152361+0000","last_undegraded":"2024-02-18T00:45:09.152361+0000","last_fullsized":"2024-02-18T00:45:09.152361+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.101796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.101796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":774,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.729782+0000","last_change":"2024-02-18T00:43:04.887857+0000","last_active":"2024-02-18T00:45:09.729782+0000","last_peered":"2024-02-18T00:45:09.729782+0000","last_clean":"2024-02-18T00:45:09.729782+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:45:09.729782+0000","last_undegraded":"2024-02-18T00:45:09.729782+0000","last_fullsized":"2024-02-18T00:45:09.729782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:43:04.887771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.887771+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":723,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012559+0000","last_change":"2024-02-18T00:42:52.999275+0000","last_active":"2024-02-18T00:45:10.012559+0000","last_peered":"2024-02-18T00:45:10.012559+0000","last_clean":"2024-02-18T00:45:10.012559+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:45:10.012559+0000","last_undegraded":"2024-02-18T00:45:10.012559+0000","last_fullsized":"2024-02-18T00:45:10.012559+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:52.999208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:42:52.999208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":741,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.306225+0000","last_change":"2024-02-18T00:44:46.488019+0000","last_active":"2024-02-18T00:45:09.306225+0000","last_peered":"2024-02-18T00:45:09.306225+0000","last_clean":"2024-02-18T00:45:09.306225+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:45:09.306225+0000","last_undegraded":"2024-02-18T00:45:09.306225+0000","last_fullsized":"2024-02-18T00:45:09.306225+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.487955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":684,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.822557+0000","last_change":"2024-02-18T00:43:37.193806+0000","last_active":"2024-02-18T00:45:08.822557+0000","last_peered":"2024-02-18T00:45:08.822557+0000","last_clean":"2024-02-18T00:45:08.822557+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:45:08.822557+0000","last_undegraded":"2024-02-18T00:45:08.822557+0000","last_fullsized":"2024-02-18T00:45:08.822557+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:43:37.193693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.193693+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":777,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.821963+0000","last_change":"2024-02-18T00:43:39.154626+0000","last_active":"2024-02-18T00:45:08.821963+0000","last_peered":"2024-02-18T00:45:08.821963+0000","last_clean":"2024-02-18T00:45:08.821963+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:45:08.821963+0000","last_undegraded":"2024-02-18T00:45:08.821963+0000","last_fullsized":"2024-02-18T00:45:08.821963+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:43:39.154561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.154561+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":755,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730517+0000","last_change":"2024-02-18T00:44:25.509600+0000","last_active":"2024-02-18T00:45:09.730517+0000","last_peered":"2024-02-18T00:45:09.730517+0000","last_clean":"2024-02-18T00:45:09.730517+0000","last_became_active":"2024-02-18T00:39:36.741489+0000","last_became_peered":"2024-02-18T00:39:36.741489+0000","last_unstale":"2024-02-18T00:45:09.730517+0000","last_undegraded":"2024-02-18T00:45:09.730517+0000","last_fullsized":"2024-02-18T00:45:09.730517+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:25.509523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.509523+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":735,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.013559+0000","last_change":"2024-02-18T00:44:35.012928+0000","last_active":"2024-02-18T00:45:10.013559+0000","last_peered":"2024-02-18T00:45:10.013559+0000","last_clean":"2024-02-18T00:45:10.013559+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:45:10.013559+0000","last_undegraded":"2024-02-18T00:45:10.013559+0000","last_fullsized":"2024-02-18T00:45:10.013559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:35.012802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.012802+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":670,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.730410+0000","last_change":"2024-02-18T00:44:38.348343+0000","last_active":"2024-02-18T00:45:09.730410+0000","last_peered":"2024-02-18T00:45:09.730410+0000","last_clean":"2024-02-18T00:45:09.730410+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:45:09.730410+0000","last_undegraded":"2024-02-18T00:45:09.730410+0000","last_fullsized":"2024-02-18T00:45:09.730410+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.348263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.348263+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":768,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011811+0000","last_change":"2024-02-18T00:43:43.140277+0000","last_active":"2024-02-18T00:45:10.011811+0000","last_peered":"2024-02-18T00:45:10.011811+0000","last_clean":"2024-02-18T00:45:10.011811+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:45:10.011811+0000","last_undegraded":"2024-02-18T00:45:10.011811+0000","last_fullsized":"2024-02-18T00:45:10.011811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:43:43.140135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.140135+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":756,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011069+0000","last_change":"2024-02-18T00:42:13.912023+0000","last_active":"2024-02-18T00:45:10.011069+0000","last_peered":"2024-02-18T00:45:10.011069+0000","last_clean":"2024-02-18T00:45:10.011069+0000","last_became_active":"2024-02-18T00:39:36.761839+0000","last_became_peered":"2024-02-18T00:39:36.761839+0000","last_unstale":"2024-02-18T00:45:10.011069+0000","last_undegraded":"2024-02-18T00:45:10.011069+0000","last_fullsized":"2024-02-18T00:45:10.011069+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:42:13.911916+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.911916+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":730,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816586+0000","last_change":"2024-02-18T00:44:44.755473+0000","last_active":"2024-02-18T00:45:08.816586+0000","last_peered":"2024-02-18T00:45:08.816586+0000","last_clean":"2024-02-18T00:45:08.816586+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:45:08.816586+0000","last_undegraded":"2024-02-18T00:45:08.816586+0000","last_fullsized":"2024-02-18T00:45:08.816586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.755386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:44.755386+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.755386+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":686,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.306225+0000","last_change":"2024-02-18T00:43:48.486113+0000","last_active":"2024-02-18T00:45:09.306225+0000","last_peered":"2024-02-18T00:45:09.306225+0000","last_clean":"2024-02-18T00:45:09.306225+0000","last_became_active":"2024-02-18T00:39:36.771648+0000","last_became_peered":"2024-02-18T00:39:36.771648+0000","last_unstale":"2024-02-18T00:45:09.306225+0000","last_undegraded":"2024-02-18T00:45:09.306225+0000","last_fullsized":"2024-02-18T00:45:09.306225+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:48.486037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:48.486037+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":796,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816317+0000","last_change":"2024-02-18T00:43:01.284219+0000","last_active":"2024-02-18T00:45:08.816317+0000","last_peered":"2024-02-18T00:45:08.816317+0000","last_clean":"2024-02-18T00:45:08.816317+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:45:08.816317+0000","last_undegraded":"2024-02-18T00:45:08.816317+0000","last_fullsized":"2024-02-18T00:45:08.816317+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:43:01.284140+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.284140+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153179+0000","last_change":"2024-02-18T00:43:03.809975+0000","last_active":"2024-02-18T00:45:09.153179+0000","last_peered":"2024-02-18T00:45:09.153179+0000","last_clean":"2024-02-18T00:45:09.153179+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:45:09.153179+0000","last_undegraded":"2024-02-18T00:45:09.153179+0000","last_fullsized":"2024-02-18T00:45:09.153179+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.809907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.809907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":751,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.664579+0000","last_change":"2024-02-18T00:43:34.784518+0000","last_active":"2024-02-18T00:45:09.664579+0000","last_peered":"2024-02-18T00:45:09.664579+0000","last_clean":"2024-02-18T00:45:09.664579+0000","last_became_active":"2024-02-18T00:39:36.749467+0000","last_became_peered":"2024-02-18T00:39:36.749467+0000","last_unstale":"2024-02-18T00:45:09.664579+0000","last_undegraded":"2024-02-18T00:45:09.664579+0000","last_fullsized":"2024-02-18T00:45:09.664579+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.784446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.784446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":666,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.823208+0000","last_change":"2024-02-18T00:42:21.695753+0000","last_active":"2024-02-18T00:45:08.823208+0000","last_peered":"2024-02-18T00:45:08.823208+0000","last_clean":"2024-02-18T00:45:08.823208+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:45:08.823208+0000","last_undegraded":"2024-02-18T00:45:08.823208+0000","last_fullsized":"2024-02-18T00:45:08.823208+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:21.695689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:21.695689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":763,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816329+0000","last_change":"2024-02-18T00:44:34.749044+0000","last_active":"2024-02-18T00:45:08.816329+0000","last_peered":"2024-02-18T00:45:08.816329+0000","last_clean":"2024-02-18T00:45:08.816329+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:45:08.816329+0000","last_undegraded":"2024-02-18T00:45:08.816329+0000","last_fullsized":"2024-02-18T00:45:08.816329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:44:34.748966+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:44:34.748966+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":300,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.153179+0000","last_change":"2024-02-18T00:44:31.152577+0000","last_active":"2024-02-18T00:45:09.153179+0000","last_peered":"2024-02-18T00:45:09.153179+0000","last_clean":"2024-02-18T00:45:09.153179+0000","last_became_active":"2024-02-18T00:39:36.761469+0000","last_became_peered":"2024-02-18T00:39:36.761469+0000","last_unstale":"2024-02-18T00:45:09.153179+0000","last_undegraded":"2024-02-18T00:45:09.153179+0000","last_fullsized":"2024-02-18T00:45:09.153179+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:31.152515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.152515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":295,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.152535+0000","last_change":"2024-02-18T00:43:39.066050+0000","last_active":"2024-02-18T00:45:09.152535+0000","last_peered":"2024-02-18T00:45:09.152535+0000","last_clean":"2024-02-18T00:45:09.152535+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:45:09.152535+0000","last_undegraded":"2024-02-18T00:45:09.152535+0000","last_fullsized":"2024-02-18T00:45:09.152535+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.065952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.065952+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":681,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012094+0000","last_change":"2024-02-18T00:44:30.255270+0000","last_active":"2024-02-18T00:45:10.012094+0000","last_peered":"2024-02-18T00:45:10.012094+0000","last_clean":"2024-02-18T00:45:10.012094+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:45:10.012094+0000","last_undegraded":"2024-02-18T00:45:10.012094+0000","last_fullsized":"2024-02-18T00:45:10.012094+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:30.255199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:30.255199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":772,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.815613+0000","last_change":"2024-02-18T00:43:34.567432+0000","last_active":"2024-02-18T00:45:08.815613+0000","last_peered":"2024-02-18T00:45:08.815613+0000","last_clean":"2024-02-18T00:45:08.815613+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:45:08.815613+0000","last_undegraded":"2024-02-18T00:45:08.815613+0000","last_fullsized":"2024-02-18T00:45:08.815613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:34.567365+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.567365+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":723,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011677+0000","last_change":"2024-02-18T00:43:44.142781+0000","last_active":"2024-02-18T00:45:10.011677+0000","last_peered":"2024-02-18T00:45:10.011677+0000","last_clean":"2024-02-18T00:45:10.011677+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:45:10.011677+0000","last_undegraded":"2024-02-18T00:45:10.011677+0000","last_fullsized":"2024-02-18T00:45:10.011677+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.142683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.142683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":748,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.011958+0000","last_change":"2024-02-18T00:42:51.096086+0000","last_active":"2024-02-18T00:45:10.011958+0000","last_peered":"2024-02-18T00:45:10.011958+0000","last_clean":"2024-02-18T00:45:10.011958+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:45:10.011958+0000","last_undegraded":"2024-02-18T00:45:10.011958+0000","last_fullsized":"2024-02-18T00:45:10.011958+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:51.096010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:51.096010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":665,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.014049+0000","last_change":"2024-02-18T00:43:50.155273+0000","last_active":"2024-02-18T00:45:10.014049+0000","last_peered":"2024-02-18T00:45:10.014049+0000","last_clean":"2024-02-18T00:45:10.014049+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:45:10.014049+0000","last_undegraded":"2024-02-18T00:45:10.014049+0000","last_fullsized":"2024-02-18T00:45:10.014049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.155205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.155205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":826,"reported_epoch":752,"state":"active+clean","last_fresh":"2024-02-18T00:45:10.012520+0000","last_change":"2024-02-18T00:44:46.283742+0000","last_active":"2024-02-18T00:45:10.012520+0000","last_peered":"2024-02-18T00:45:10.012520+0000","last_clean":"2024-02-18T00:45:10.012520+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:45:10.012520+0000","last_undegraded":"2024-02-18T00:45:10.012520+0000","last_fullsized":"2024-02-18T00:45:10.012520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-18T00:44:46.283668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.283668+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":747,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:09.306002+0000","last_change":"2024-02-18T00:43:32.499214+0000","last_active":"2024-02-18T00:45:09.306002+0000","last_peered":"2024-02-18T00:45:09.306002+0000","last_clean":"2024-02-18T00:45:09.306002+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:45:09.306002+0000","last_undegraded":"2024-02-18T00:45:09.306002+0000","last_fullsized":"2024-02-18T00:45:09.306002+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.499132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.499132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":730,"reported_epoch":751,"state":"active+clean","last_fresh":"2024-02-18T00:45:08.816090+0000","last_change":"2024-02-18T00:44:39.696355+0000","last_active":"2024-02-18T00:45:08.816090+0000","last_peered":"2024-02-18T00:45:08.816090+0000","last_clean":"2024-02-18T00:45:08.816090+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:45:08.816090+0000","last_undegraded":"2024-02-18T00:45:08.816090+0000","last_fullsized":"2024-02-18T00:45:08.816090+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:39.696257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:39.696257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":250,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":4000249,"num_objects":37,"num_object_clones":1,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":57,"num_write_kb":3959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6438912,"data_stored":6438912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90,"ondisk_log_size":90,"up":24,"acting":24,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":42,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":942,"num_read_kb":776,"num_write":562,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":1210,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1278,"ondisk_log_size":1278,"up":126,"acting":126,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":946,"num_read_kb":941,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266883,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315324,"kb_used_data":5812,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429828,"statfs":{"total":95995035648,"available":95672143872,"internally_reserved":0,"allocated":5951488,"data_stored":3176065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.53600000000000003,"15min":0.56899999999999995},"min":{"1min":0.26100000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.60299999999999998,"5min":4.7389999999999999,"15min":6.8700000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.497,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.57599999999999996,"5min":1.0780000000000001,"15min":11.288},"last":0.626}]},{"osd":1,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.63200000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.7330000000000001,"5min":2.968,"15min":7.4729999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.64800000000000002},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.7150000000000001,"5min":3.0019999999999998,"15min":7.7869999999999999},"last":0.78300000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.56999999999999995,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56299999999999994,"5min":4.8220000000000001,"15min":6.7380000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.51300000000000001,"15min":0.59699999999999998},"min":{"1min":0.33800000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.64100000000000001,"5min":1.913,"15min":12.861000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.34799999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56999999999999995,"5min":4.8520000000000003,"15min":7.1360000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.53000000000000003,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61199999999999999,"5min":1.948,"15min":3.7989999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.436,"15min":0.433},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.215},"max":{"1min":0.64600000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.53300000000000003,"15min":0.48399999999999999},"min":{"1min":0.28799999999999998,"5min":0.26300000000000001,"15min":0.246},"max":{"1min":0.62,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.67600000000000005,"5min":1.877,"15min":1.877},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.498,"15min":0.50800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":1.853,"15min":4.9130000000000003},"last":0.54800000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68899999999999995,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59799999999999998,"15min":0.54700000000000004},"min":{"1min":0.309,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68799999999999994,"5min":5.3860000000000001,"15min":5.3860000000000001},"last":0.67400000000000004}]}]},{"osd":6,"up_from":47,"seq":201863463113,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309128,"kb_used_data":7808,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436024,"statfs":{"total":95995035648,"available":95678488576,"internally_reserved":0,"allocated":7995392,"data_stored":5221293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":1.2689999999999999},"min":{"1min":0.371,"5min":0.27000000000000002,"15min":0.22500000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":69.194999999999993},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.51800000000000002,"15min":0.74299999999999999},"min":{"1min":0.38300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":10.433},"last":0.42799999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55600000000000005,"15min":1.3500000000000001},"min":{"1min":0.37,"5min":0.313,"15min":0.26000000000000001},"max":{"1min":0.85399999999999998,"5min":1.9610000000000001,"15min":76.108999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54300000000000004,"15min":1.339},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.28100000000000003},"max":{"1min":0.77200000000000002,"5min":3.665,"15min":71.872},"last":0.38400000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.82599999999999996},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.31},"max":{"1min":0.85099999999999998,"5min":3.702,"15min":39.939999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.53700000000000003,"15min":0.99099999999999999},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83099999999999996,"5min":1.5489999999999999,"15min":39.673000000000002},"last":0.40600000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.84799999999999998},"min":{"1min":0.44400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":21.219000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.80000000000000004},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.29299999999999998},"max":{"1min":0.81699999999999995,"5min":3.7280000000000002,"15min":14.007999999999999},"last":0.503}]},{"osd":4,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.25800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65400000000000003,"5min":0.748,"15min":8.1479999999999997},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.50800000000000001,"15min":0.72399999999999998},"min":{"1min":0.33900000000000002,"5min":0.27100000000000002,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":4.1310000000000002,"15min":35.18},"last":0.438}]},{"osd":5,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":10.606},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.755},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.28199999999999997},"max":{"1min":0.75700000000000001,"5min":3.8439999999999999,"15min":35.119},"last":0.37}]},{"osd":7,"last update":"Sun Feb 18 00:44:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.55100000000000005},"min":{"1min":0.39000000000000001,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.72299999999999998,"5min":0.79200000000000004,"15min":5.4770000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.72499999999999998},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":0.77400000000000002,"5min":0.79100000000000004,"15min":32.985999999999997},"last":0.41499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575072,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304968,"kb_used_data":7744,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440184,"statfs":{"total":95995035648,"available":95682748416,"internally_reserved":0,"allocated":7929856,"data_stored":5161539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34300000000000003,"15min":0.64700000000000002},"min":{"1min":0.22700000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":0.56899999999999995,"5min":0.77400000000000002,"15min":32.106999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.435,"15min":0.72099999999999997},"min":{"1min":0.26900000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41799999999999998,"5min":6.625,"15min":43.137},"last":0.34499999999999997}]},{"osd":2,"last update":"Sun Feb 18 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.35099999999999998,"15min":0.752},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.19900000000000001},"max":{"1min":0.47799999999999998,"5min":0.70199999999999996,"15min":49.201000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38400000000000001,"15min":0.58699999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.217},"max":{"1min":0.57599999999999996,"5min":0.85299999999999998,"15min":17.596},"last":0.46200000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.71699999999999997},"min":{"1min":0.28199999999999997,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":37.524000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42999999999999999,"15min":0.73899999999999999},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.77000000000000002,"5min":0.89200000000000002,"15min":32.036000000000001},"last":0.375}]},{"osd":4,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54600000000000004,"15min":0.753},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":0.99299999999999999,"15min":17.486999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56100000000000005,"15min":0.77700000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.32500000000000001},"max":{"1min":0.749,"5min":0.98299999999999998,"15min":12.75},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.76000000000000001},"min":{"1min":0.38900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84599999999999997,"5min":0.93300000000000005,"15min":12.611000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.77700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":1.141,"15min":17.417000000000002},"last":0.58299999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:44:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57799999999999996,"15min":0.79300000000000004},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71399999999999997,"5min":0.92300000000000004,"15min":12.834},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.58599999999999997,"15min":0.77800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.35499999999999998},"max":{"1min":0.72599999999999998,"5min":0.97199999999999998,"15min":12.651999999999999},"last":1.046}]},{"osd":7,"last update":"Sun Feb 18 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.79400000000000004},"min":{"1min":0.40500000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76300000000000001,"5min":1.175,"15min":17.452000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":1.012,"15min":12.805},"last":0.68100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738598,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303076,"kb_used_data":5852,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93442076,"statfs":{"total":95995035648,"available":95684685824,"internally_reserved":0,"allocated":5992448,"data_stored":3220734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.35399999999999998,"15min":0.99299999999999999},"min":{"1min":0.23300000000000001,"5min":0.217,"15min":0.18099999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":133.38999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.40699999999999997,"15min":0.79300000000000004},"min":{"1min":0.26700000000000002,"5min":0.23499999999999999,"15min":0.20999999999999999},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":90.623000000000005},"last":0.36799999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40300000000000002,"15min":0.89700000000000002},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.20200000000000001},"max":{"1min":0.61699999999999999,"5min":1.782,"15min":90.850999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36099999999999999,"15min":0.85999999999999999},"min":{"1min":0.26700000000000002,"5min":0.20300000000000001,"15min":0.20000000000000001},"max":{"1min":0.497,"5min":0.77600000000000002,"15min":80.537000000000006},"last":0.248}]},{"osd":3,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.39600000000000002,"15min":0.64900000000000002},"min":{"1min":0.29899999999999999,"5min":0.21299999999999999,"15min":0.20899999999999999},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":19.006},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.46500000000000002,"15min":1.054},"min":{"1min":0.33900000000000002,"5min":0.29199999999999998,"15min":0.23999999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":146.71899999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:44:15 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56599999999999995,"15min":0.92600000000000005},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.26700000000000002},"max":{"1min":0.69899999999999995,"5min":1.6830000000000001,"15min":57.914000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60199999999999998,"15min":0.97599999999999998},"min":{"1min":0.45200000000000001,"5min":0.33400000000000002,"15min":0.30399999999999999},"max":{"1min":0.755,"5min":1.6619999999999999,"15min":90.070999999999998},"last":0.497}]},{"osd":5,"last update":"Sun Feb 18 00:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.63100000000000001,"15min":0.96599999999999997},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.32000000000000001},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":90.049000000000007},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.99199999999999999},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.32200000000000001},"max":{"1min":1.853,"5min":1.853,"15min":57.991},"last":0.34699999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60899999999999999,"15min":0.80800000000000005},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.35999999999999999},"max":{"1min":1.194,"5min":1.194,"15min":19.039000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.63300000000000001,"15min":0.996},"min":{"1min":0.53000000000000003,"5min":0.39700000000000002,"15min":0.33500000000000002},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":89.989999999999995},"last":0.59499999999999997}]},{"osd":7,"last update":"Sun Feb 18 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":1.0569999999999999},"min":{"1min":0.51500000000000001,"5min":0.36799999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":1.732,"15min":90.025000000000006},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59499999999999997,"15min":1},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.83499999999999996,"5min":0.89700000000000002,"15min":57.962000000000003},"last":0.44700000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411549,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305012,"kb_used_data":7788,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440140,"statfs":{"total":95995035648,"available":95682703360,"internally_reserved":0,"allocated":7974912,"data_stored":5211472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.45100000000000001,"15min":0.51100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.192},"max":{"1min":8.6489999999999991,"5min":8.6489999999999991,"15min":12.144},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36399999999999999,"15min":0.59799999999999998},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":18.891999999999999},"last":0.41199999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.49399999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.19700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":10.138999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.53400000000000003},"min":{"1min":0.248,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":18.919},"last":0.46899999999999997}]},{"osd":3,"last update":"Sun Feb 18 00:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.505,"15min":0.61199999999999999},"min":{"1min":0.30299999999999999,"5min":0.28499999999999998,"15min":0.24099999999999999},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":22.712},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.58699999999999997},"min":{"1min":0.29099999999999998,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":22.359000000000002},"last":0.503}]},{"osd":4,"last update":"Sun Feb 18 00:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.68700000000000006},"min":{"1min":0.379,"5min":0.37,"15min":0.32500000000000001},"max":{"1min":0.83999999999999997,"5min":0.94899999999999995,"15min":18.812000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59499999999999997,"15min":0.67400000000000004},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.28699999999999998},"max":{"1min":1.002,"5min":1.002,"15min":10.016},"last":0.73799999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":0.71599999999999997,"15min":0.69399999999999995},"min":{"1min":0.39600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":11.805,"5min":11.805,"15min":11.805},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.05,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.317},"max":{"1min":8.6059999999999999,"5min":8.6059999999999999,"15min":10.114000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:45:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.72999999999999998,"15min":0.69699999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35799999999999998},"max":{"1min":1.663,"5min":11.680999999999999,"15min":11.680999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.70099999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.34100000000000003},"max":{"1min":0.871,"5min":0.90500000000000003,"15min":18.861000000000001},"last":0.69999999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":11.746,"5min":11.746,"15min":11.746},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67300000000000004,"15min":0.73699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.372},"max":{"1min":1.115,"5min":1.115,"15min":10.058999999999999},"last":0.84599999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149911,"num_pgs":91,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315360,"kb_used_data":5848,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429792,"statfs":{"total":95995035648,"available":95672107008,"internally_reserved":0,"allocated":5988352,"data_stored":3201314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.45700000000000002},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":25.082999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38,"15min":0.50800000000000001},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":18.911000000000001},"last":0.30399999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.45300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.223},"max":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":6.8479999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.68400000000000005},"min":{"1min":0.26700000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.59999999999999998,"5min":0.73899999999999999,"15min":23.606999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41299999999999998,"15min":0.64300000000000002},"min":{"1min":0.28000000000000003,"5min":0.25700000000000001,"15min":0.23799999999999999},"max":{"1min":0.877,"5min":0.877,"15min":23.405000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.40000000000000002,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.23999999999999999},"max":{"1min":0.57699999999999996,"5min":0.85599999999999998,"15min":5.3479999999999999},"last":0.38500000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58599999999999997,"15min":0.68899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36299999999999999,"15min":0.33600000000000002},"max":{"1min":0.82299999999999995,"5min":0.93000000000000005,"15min":10.131},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.32700000000000001},"max":{"1min":0.85999999999999999,"5min":1.1699999999999999,"15min":5.4630000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.90000000000000002,"5min":4.7469999999999999,"15min":7.4249999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.67500000000000004},"min":{"1min":0.375,"5min":0.34200000000000003,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":1.0860000000000001,"15min":6.7750000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.747},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.93700000000000006,"15min":10.189},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":0.96199999999999997,"15min":6.4279999999999999},"last":0.78300000000000003}]},{"osd":7,"last update":"Sun Feb 18 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.39500000000000002,"15min":0.35199999999999998},"max":{"1min":0.95399999999999996,"5min":5.0830000000000002,"15min":6.8220000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.432,"15min":0.39800000000000002},"max":{"1min":0.91600000000000004,"5min":1.0760000000000001,"15min":10.164999999999999},"last":0.63100000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986386,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307172,"kb_used_data":5852,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93437980,"statfs":{"total":95995035648,"available":95680491520,"internally_reserved":0,"allocated":5992448,"data_stored":3225609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.85999999999999999},"min":{"1min":0.38700000000000001,"5min":0.249,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.86899999999999999,"15min":77.581999999999994},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.83799999999999997},"min":{"1min":0.36099999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69999999999999996,"5min":0.753,"15min":78.063999999999993},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52100000000000002,"15min":0.61499999999999999},"min":{"1min":0.33100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.74199999999999999,"15min":9.3160000000000007},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.876},"min":{"1min":0.40100000000000002,"5min":0.32500000000000001,"15min":0.29699999999999999},"max":{"1min":0.66300000000000003,"5min":0.754,"15min":81.242000000000004},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.35299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69099999999999995,"5min":0.72299999999999998,"15min":11.691000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.66800000000000004},"min":{"1min":0.35999999999999999,"5min":0.30199999999999999,"15min":0.27200000000000002},"max":{"1min":0.752,"5min":4.7149999999999999,"15min":17.77},"last":0.37}]},{"osd":3,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.33700000000000002,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":7.218},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.313,"15min":0.309},"max":{"1min":0.70199999999999996,"5min":1.21,"15min":33.530999999999999},"last":0.441}]},{"osd":5,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.28899999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65100000000000002,"5min":0.71899999999999997,"15min":0.72299999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.46400000000000002,"15min":0.45500000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.48399999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46000000000000002,"15min":0.45100000000000001},"min":{"1min":0.27500000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.66700000000000004,"5min":0.67100000000000004,"15min":0.76100000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.76700000000000002,"15min":0.76900000000000002},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.64800000000000002,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.52900000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724749,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307120,"kb_used_data":5800,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93438032,"statfs":{"total":95995035648,"available":95680544768,"internally_reserved":0,"allocated":5939200,"data_stored":3192552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.502,"15min":0.53500000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.217},"max":{"1min":0.71499999999999997,"5min":2.21,"15min":6.3150000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.498,"15min":0.60299999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.26000000000000001},"max":{"1min":1.123,"5min":1.782,"15min":12.744},"last":0.68799999999999994}]},{"osd":1,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.64300000000000002},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":1.2010000000000001,"5min":2.1509999999999998,"15min":13.25},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.65900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.29699999999999999},"max":{"1min":0.72899999999999998,"5min":1.9430000000000001,"15min":13.262},"last":0.61899999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.58899999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.251},"max":{"1min":1.4510000000000001,"5min":2.2559999999999998,"15min":6.1680000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.308},"max":{"1min":1.4370000000000001,"5min":1.8799999999999999,"15min":6.1879999999999997},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.622},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":3.637,"15min":7.641},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.60599999999999998},"min":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.27600000000000002},"max":{"1min":1.22,"5min":1.853,"15min":8.8650000000000002},"last":0.628}]},{"osd":4,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.49299999999999999},"min":{"1min":0.32300000000000001,"5min":0.25,"15min":0.23599999999999999},"max":{"1min":0.64400000000000002,"5min":1.897,"15min":11.818},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.36899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74199999999999999,"5min":3.5920000000000001,"15min":4.274},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:44:08 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47799999999999998,"15min":0.56299999999999994},"min":{"1min":0.30499999999999999,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":0.66400000000000003,"5min":2.7120000000000002,"15min":18.605},"last":0.436},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51600000000000001,"15min":0.50900000000000001},"min":{"1min":0.27200000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0629999999999999,"5min":2.6699999999999999,"15min":4.2089999999999996},"last":0.45400000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.50600000000000001},"min":{"1min":0.314,"5min":0.27400000000000002,"15min":0.26600000000000001},"max":{"1min":1.2370000000000001,"5min":1.819,"15min":2.0739999999999998},"last":0.499},{"interface":"front","average":{"1min":0.502,"5min":0.503,"15min":0.52700000000000002},"min":{"1min":0.34799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":4.649},"last":0.53400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":73728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2023424,"data_stored":2023424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":2064384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":53248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":77824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":45056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":2072576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":250,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:45:17.440 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:45:17.441 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:17.441 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:17.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:17.543 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:17.594 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:17.615 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:17.617 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:17.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:17.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:17.748 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:17.824 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:17.824 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:17.925 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:17.958 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:17.980 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:17.983 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:17.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:18.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:16.937001+0000 mon.a (mon.0) 3608 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:18.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:16.937065+0000 mon.a (mon.0) 3609 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-02-18T00:45:18.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:16.962733+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:18.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:16.937001+0000 mon.a (mon.0) 3608 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:18.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:16.937065+0000 mon.a (mon.0) 3609 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-02-18T00:45:18.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:16.962733+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:18.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:18.113 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:18.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:16.937001+0000 mon.a (mon.0) 3608 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:16.937065+0000 mon.a (mon.0) 3609 : cluster [DBG] osdmap e759: 8 total, 8 up, 8 in 2024-02-18T00:45:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:16.962733+0000 mon.a (mon.0) 3610 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:18.465 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:18.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:18.566 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:18.595 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:18.617 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:18.620 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:18.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:18.721 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:18.753 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:18.776 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:18.778 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:18.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:18.880 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:18.911 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:18.931 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:18.933 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:18.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:19.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:17.935765+0000 mon.a (mon.0) 3611 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:19.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:17.935838+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-02-18T00:45:19.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:17.938486+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:19.003 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:18.015953+0000 mgr.y (mgr.24454) 655 : cluster [DBG] pgmap v1167: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:45:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:18.818689+0000 mon.a (mon.0) 3614 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-134"}]': finished 2024-02-18T00:45:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:18.818748+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-02-18T00:45:19.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:18.819202+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-134", "mode": "readproxy"}]: dispatch 2024-02-18T00:45:19.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:19.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 -- 172.21.15.59:0/3865043379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40f67e0 msgr2=0x7fa4c4104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/3865043379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40f67e0 0x7fa4c4104a60 secure :-1 s=READY pgs=4065 cs=0 l=1 rev1=1 rx=0x7fa4bc009be0 tx=0x7fa4bc004d10).stop 2024-02-18T00:45:19.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 -- 172.21.15.59:0/3865043379 shutdown_connections 2024-02-18T00:45:19.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/3865043379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40f67e0 0x7fa4c4104a60 unknown :-1 s=CLOSED pgs=4065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/3865043379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4c40fdbf0 0x7fa4c40fdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/3865043379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40fe550 0x7fa4c40f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 -- 172.21.15.59:0/3865043379 >> 172.21.15.59:0/3865043379 conn(0x7fa4c4070030 msgr2=0x7fa4c40f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 -- 172.21.15.59:0/3865043379 shutdown_connections 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 -- 172.21.15.59:0/3865043379 wait complete. 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 librados: starting msgr at 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.014+0000 7fa4cbeba700 1 librados: starting objecter 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 Processor -- start 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 -- start start 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 librados: setting wanted keys 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 librados: calling monclient init 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40f67e0 0x7fa4c410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40fdbf0 0x7fa4c410e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4c40fe550 0x7fa4c41081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41086b0 con 0x7fa4c40f67e0 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4108820 con 0x7fa4c40fdbf0 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4cbeba700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41089e0 con 0x7fa4c40fe550 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40f67e0 0x7fa4c410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4ca457700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4c40fe550 0x7fa4c41081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40f67e0 0x7fa4c410e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38304/0 (socket says 172.21.15.59:38304) 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 -- 172.21.15.59:0/2476999962 learned_addr learned my addr 172.21.15.59:0/2476999962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9455700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40fdbf0 0x7fa4c410e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 -- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4c40fe550 msgr2=0x7fa4c41081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4c40fe550 0x7fa4c41081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 -- 172.21.15.59:0/2476999962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40fdbf0 msgr2=0x7fa4c410e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40fdbf0 0x7fa4c410e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 -- 172.21.15.59:0/2476999962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc0097c0 con 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4ca457700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4c40fe550 0x7fa4c41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4c9c56700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40f67e0 0x7fa4c410e020 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 rx=0x7fa4c000e940 tx=0x7fa4c000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4b6ffd700 10 client.?.objecter ms_handle_connect 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.015+0000 7fa4b6ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4b6ffd700 1 -- 172.21.15.59:0/2476999962 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0004660 con 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c4108cc0 con 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4cbeba700 10 client.31977.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c4109040 con 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4cbeba700 1 librados: init done 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4b6ffd700 1 -- 172.21.15.59:0/2476999962 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4c000f040 con 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4b6ffd700 1 -- 172.21.15.59:0/2476999962 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0014640 con 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.016+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a8004f50 con 0x7fa4c40f67e0 2024-02-18T00:45:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.017+0000 7fa4b6ffd700 1 -- 172.21.15.59:0/2476999962 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa4c001d3b0 con 0x7fa4c40f67e0 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.017+0000 7fa4b6ffd700 1 --2- 172.21.15.59:0/2476999962 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa4b0062c90 0x7fa4b0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.017+0000 7fa4b6ffd700 1 -- 172.21.15.59:0/2476999962 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(761..761 src has 418..761) v4 ==== 8814+0+0 (secure 0 0 0) 0x7fa4c007cfa0 con 0x7fa4c40f67e0 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.017+0000 7fa4b6ffd700 10 client.31977.objecter ms_dispatch 0x7fa4c4001550 osd_map(761..761 src has 418..761) v4 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.017+0000 7fa4b6ffd700 3 client.31977.objecter handle_osd_map got epochs [761,761] > 0 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.017+0000 7fa4b6ffd700 3 client.31977.objecter handle_osd_map decoding full epoch 761 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.017+0000 7fa4c9455700 1 --2- 172.21.15.59:0/2476999962 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa4b0062c90 0x7fa4b0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.018+0000 7fa4b6ffd700 20 client.31977.objecter dump_active .. 0 homeless 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.018+0000 7fa4b6ffd700 10 client.31977.objecter ms_handle_connect 0x7fa4b0062c90 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.018+0000 7fa4c9455700 1 --2- 172.21.15.59:0/2476999962 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa4b0062c90 0x7fa4b0065150 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 rx=0x7fa4c4109960 tx=0x7fa4b800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.022+0000 7fa4b6ffd700 1 -- 172.21.15.59:0/2476999962 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa4c00501d0 con 0x7fa4c40f67e0 2024-02-18T00:45:19.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:17.935765+0000 mon.a (mon.0) 3611 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:19.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:17.935838+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-02-18T00:45:19.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:17.938486+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:19.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:18.015953+0000 mgr.y (mgr.24454) 655 : cluster [DBG] pgmap v1167: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:45:19.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:18.818689+0000 mon.a (mon.0) 3614 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-134"}]': finished 2024-02-18T00:45:19.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:18.818748+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-02-18T00:45:19.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:18.819202+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-134", "mode": "readproxy"}]: dispatch 2024-02-18T00:45:19.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:17.935765+0000 mon.a (mon.0) 3611 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:19.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:17.935838+0000 mon.a (mon.0) 3612 : cluster [DBG] osdmap e760: 8 total, 8 up, 8 in 2024-02-18T00:45:19.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:17.938486+0000 mon.a (mon.0) 3613 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:19.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:18.015953+0000 mgr.y (mgr.24454) 655 : cluster [DBG] pgmap v1167: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:45:19.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:18.818689+0000 mon.a (mon.0) 3614 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-134"}]': finished 2024-02-18T00:45:19.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:18.818748+0000 mon.a (mon.0) 3615 : cluster [DBG] osdmap e761: 8 total, 8 up, 8 in 2024-02-18T00:45:19.066 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:18.819202+0000 mon.a (mon.0) 3616 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-134", "mode": "readproxy"}]: dispatch 2024-02-18T00:45:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:19.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.214+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fa4a8005200 con 0x7fa4c40f67e0 2024-02-18T00:45:19.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 -- 172.21.15.59:0/3104502330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 msgr2=0x7f2b0c0fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 --2- 172.21.15.59:0/3104502330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 0x7f2b0c0fdf90 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 rx=0x7f2b080047b0 tx=0x7f2b08009cb0).stop 2024-02-18T00:45:19.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 -- 172.21.15.59:0/3104502330 shutdown_connections 2024-02-18T00:45:19.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 --2- 172.21.15.59:0/3104502330 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b0c0f8420 0x7f2b0c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 --2- 172.21.15.59:0/3104502330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 0x7f2b0c0fdf90 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 --2- 172.21.15.59:0/3104502330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0c0fe510 0x7f2b0c0f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 -- 172.21.15.59:0/3104502330 >> 172.21.15.59:0/3104502330 conn(0x7f2b0c0f3800 msgr2=0x7f2b0c0f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:19.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 -- 172.21.15.59:0/3104502330 shutdown_connections 2024-02-18T00:45:19.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 -- 172.21.15.59:0/3104502330 wait complete. 2024-02-18T00:45:19.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 librados: starting msgr at 2024-02-18T00:45:19.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 librados: starting objecter 2024-02-18T00:45:19.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 Processor -- start 2024-02-18T00:45:19.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 -- start start 2024-02-18T00:45:19.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 librados: setting wanted keys 2024-02-18T00:45:19.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.428+0000 7f2b13883700 1 librados: calling monclient init 2024-02-18T00:45:19.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b0c0f8420 0x7f2b0c06b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 0x7f2b0c06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0c0fe510 0x7f2b0c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c116570 con 0x7f2b0c0fdbb0 2024-02-18T00:45:19.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c1166e0 con 0x7f2b0c0fe510 2024-02-18T00:45:19.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c1168a0 con 0x7f2b0c0f8420 2024-02-18T00:45:19.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b0c0f8420 0x7f2b0c06b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b10e1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 0x7f2b0c06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b0c0f8420 0x7f2b0c06b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37200/0 (socket says 172.21.15.59:37200) 2024-02-18T00:45:19.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b10e1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 0x7f2b0c06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38318/0 (socket says 172.21.15.59:38318) 2024-02-18T00:45:19.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 -- 172.21.15.59:0/1725869750 learned_addr learned my addr 172.21.15.59:0/1725869750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:19.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0c0fe510 0x7f2b0c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 -- 172.21.15.59:0/1725869750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0c0fe510 msgr2=0x7f2b0c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0c0fe510 0x7f2b0c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 -- 172.21.15.59:0/1725869750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 msgr2=0x7f2b0c06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 0x7f2b0c06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 -- 172.21.15.59:0/1725869750 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b040097c0 con 0x7f2b0c0f8420 2024-02-18T00:45:19.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b0c0fe510 0x7f2b0c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:45:19.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b10e1e700 1 --2- 172.21.15.59:0/1725869750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b0c0fdbb0 0x7f2b0c06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:19.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b0c0f8420 0x7f2b0c06b0d0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7f2b080044e0 tx=0x7f2b08007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b027fc700 10 client.?.objecter ms_handle_connect 0x7f2b0c0f8420 2024-02-18T00:45:19.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b027fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:19.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08018070 con 0x7f2b0c0f8420 2024-02-18T00:45:19.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b080093b0 con 0x7f2b0c0f8420 2024-02-18T00:45:19.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b08005040 con 0x7f2b0c0f8420 2024-02-18T00:45:19.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0801dd30 con 0x7f2b0c0f8420 2024-02-18T00:45:19.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 10 client.41783.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:19.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0c116dc0 con 0x7f2b0c0f8420 2024-02-18T00:45:19.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.429+0000 7f2b13883700 1 librados: init done 2024-02-18T00:45:19.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.430+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f2b0c107110 con 0x7f2b0c0f8420 2024-02-18T00:45:19.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2b0801d420 con 0x7f2b0c0f8420 2024-02-18T00:45:19.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2af8062c90 0x7f2af8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(761..761 src has 418..761) v4 ==== 8814+0+0 (secure 0 0 0) 0x7f2b0807e920 con 0x7f2b0c0f8420 2024-02-18T00:45:19.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 osd_map(761..761 src has 418..761) v4 2024-02-18T00:45:19.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 3 client.41783.objecter handle_osd_map got epochs [761,761] > 0 2024-02-18T00:45:19.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 3 client.41783.objecter handle_osd_map decoding full epoch 761 2024-02-18T00:45:19.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b10e1e700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2af8062c90 0x7f2af8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 20 client.41783.objecter dump_active .. 0 homeless 2024-02-18T00:45:19.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.431+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v761) v1 ==== 54+0+16 (secure 0 0 0) 0x7f2b08083220 con 0x7f2b0c0f8420 2024-02-18T00:45:19.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 10 client.41783.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 1 2024-02-18T00:45:19.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2b0c0598f0 0x7f2b0c059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.468 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2af8062c90 2024-02-18T00:45:19.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b10e1e700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2af8062c90 0x7f2af8065150 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 rx=0x7f2b0c06c870 tx=0x7f2afc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 1 2024-02-18T00:45:19.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 0 1 2024-02-18T00:45:19.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c06c5b0 osd=0 4 2024-02-18T00:45:19.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 10 client.41783.objecter _send_command 1 2024-02-18T00:45:19.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c05fa60 con 0x7f2b0c0598f0 2024-02-18T00:45:19.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2b0c0598f0 0x7f2b0c059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c0598f0 2024-02-18T00:45:19.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.432+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2b0c0598f0 0x7f2b0c059cd0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.433+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c05fa60 con 0x7f2b0c0598f0 2024-02-18T00:45:19.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.433+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:45:19.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.433+0000 7f2b027fc700 10 client.41783.objecter _finish_command 1 = system:0 2024-02-18T00:45:19.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.433+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c06c5b0 osd=0 4 2024-02-18T00:45:19.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.433+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 0 1 2024-02-18T00:45:19.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 10 client.41783.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 2 2024-02-18T00:45:19.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.472 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:45:19.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 2 2024-02-18T00:45:19.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c06c5b0 osd=0 3 2024-02-18T00:45:19.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 0 2 2024-02-18T00:45:19.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c06c5b0 osd=0 4 2024-02-18T00:45:19.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 10 client.41783.objecter _send_command 2 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.448+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c028540 con 0x7f2b0c0598f0 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c028540 con 0x7f2b0c0598f0 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b027fc700 10 client.41783.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c06c5b0 osd=0 4 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 0 2 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 10 client.41783.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 3 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6805/26279110 conn(0x7f2b0c0594d0 0x7f2b0c11b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 3 2024-02-18T00:45:19.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 1 3 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c059360 osd=1 4 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 10 client.41783.objecter _send_command 3 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c0ff650 con 0x7f2b0c0594d0 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.449+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6805/26279110 conn(0x7f2b0c0594d0 0x7f2b0c11b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.450+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c0594d0 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.450+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6805/26279110 conn(0x7f2b0c0594d0 0x7f2b0c11b000 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.450+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c0ff650 con 0x7f2b0c0594d0 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.450+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.450+0000 7f2b027fc700 10 client.41783.objecter _finish_command 3 = system:0 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.450+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c059360 osd=1 4 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.450+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 1 3 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 10 client.41783.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 4 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 4 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c059360 osd=1 3 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 1 4 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c059360 osd=1 4 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 10 client.41783.objecter _send_command 4 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.465+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c11b4f0 con 0x7f2b0c0594d0 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c11b4f0 con 0x7f2b0c0594d0 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b027fc700 10 client.41783.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c059360 osd=1 4 2024-02-18T00:45:19.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 1 4 2024-02-18T00:45:19.477 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:45:19.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 10 client.41783.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 5 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2b0c0fed20 0x7f2b0c11e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 5 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 2 5 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c0febb0 osd=2 4 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 10 client.41783.objecter _send_command 5 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c0280a0 con 0x7f2b0c0fed20 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b10e1e700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2b0c0fed20 0x7f2b0c11e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c0fed20 2024-02-18T00:45:19.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.467+0000 7f2b10e1e700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2b0c0fed20 0x7f2b0c11e880 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.468+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c0280a0 con 0x7f2b0c0fed20 2024-02-18T00:45:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.468+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:45:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.468+0000 7f2b027fc700 10 client.41783.objecter _finish_command 5 = system:0 2024-02-18T00:45:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.468+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c0febb0 osd=2 4 2024-02-18T00:45:19.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.468+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 2 5 2024-02-18T00:45:19.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 10 client.41783.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 6 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 6 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c0febb0 osd=2 3 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 2 6 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c0febb0 osd=2 4 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 10 client.41783.objecter _send_command 6 2024-02-18T00:45:19.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.482+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c11ed70 con 0x7f2b0c0fed20 2024-02-18T00:45:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c11ed70 con 0x7f2b0c0fed20 2024-02-18T00:45:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b027fc700 10 client.41783.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c0febb0 osd=2 4 2024-02-18T00:45:19.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 2 6 2024-02-18T00:45:19.486 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:45:19.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 10 client.41783.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 7 2024-02-18T00:45:19.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2b0c027770 0x7f2b0c1220a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 7 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 3 7 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c027600 osd=3 4 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 10 client.41783.objecter _send_command 7 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c11c4b0 con 0x7f2b0c027770 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2b0c027770 0x7f2b0c1220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c027770 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.484+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2b0c027770 0x7f2b0c1220a0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.485+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c11c4b0 con 0x7f2b0c027770 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.485+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.485+0000 7f2b027fc700 10 client.41783.objecter _finish_command 7 = system:0 2024-02-18T00:45:19.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.485+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c027600 osd=3 4 2024-02-18T00:45:19.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.485+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 3 7 2024-02-18T00:45:19.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 10 client.41783.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 8 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 8 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c027600 osd=3 3 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 3 8 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c027600 osd=3 4 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 10 client.41783.objecter _send_command 8 2024-02-18T00:45:19.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.499+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c11c6d0 con 0x7f2b0c027770 2024-02-18T00:45:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c11c6d0 con 0x7f2b0c027770 2024-02-18T00:45:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b027fc700 10 client.41783.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c027600 osd=3 4 2024-02-18T00:45:19.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 3 8 2024-02-18T00:45:19.503 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:45:19.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 10 client.41783.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 9 2024-02-18T00:45:19.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2b0c11f6b0 0x7f2b0c11faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 9 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 4 9 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c11c6d0 osd=4 4 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 10 client.41783.objecter _send_command 9 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c127f40 con 0x7f2b0c11f6b0 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.501+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2b0c11f6b0 0x7f2b0c11faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.502+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c11f6b0 2024-02-18T00:45:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.502+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2b0c11f6b0 0x7f2b0c11faf0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.502+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c127f40 con 0x7f2b0c11f6b0 2024-02-18T00:45:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.502+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:45:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.502+0000 7f2b027fc700 10 client.41783.objecter _finish_command 9 = system:0 2024-02-18T00:45:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.502+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c11c6d0 osd=4 4 2024-02-18T00:45:19.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.502+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 4 9 2024-02-18T00:45:19.519 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 10 client.41783.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 10 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 10 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c11c6d0 osd=4 3 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 4 10 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c11c6d0 osd=4 4 2024-02-18T00:45:19.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 10 client.41783.objecter _send_command 10 2024-02-18T00:45:19.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.519+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c127f40 con 0x7f2b0c11f6b0 2024-02-18T00:45:19.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.521+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c127f40 con 0x7f2b0c11f6b0 2024-02-18T00:45:19.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.521+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.521+0000 7f2b027fc700 10 client.41783.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.521+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c11c6d0 osd=4 4 2024-02-18T00:45:19.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.521+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 4 10 2024-02-18T00:45:19.523 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:45:19.524 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:19.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:19.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 10 client.41783.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 11 2024-02-18T00:45:19.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2b0c1272d0 0x7f2b0c127750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 11 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 5 11 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c127f40 osd=5 4 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 10 client.41783.objecter _send_command 11 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c12b2f0 con 0x7f2b0c1272d0 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.522+0000 7f2b10e1e700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2b0c1272d0 0x7f2b0c127750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.523+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c1272d0 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.523+0000 7f2b10e1e700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2b0c1272d0 0x7f2b0c127750 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.523+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c12b2f0 con 0x7f2b0c1272d0 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.523+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.523+0000 7f2b027fc700 10 client.41783.objecter _finish_command 11 = system:0 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.523+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c127f40 osd=5 4 2024-02-18T00:45:19.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.523+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 5 11 2024-02-18T00:45:19.527 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 10 client.41783.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 12 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 12 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c127f40 osd=5 3 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 5 12 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c127f40 osd=5 4 2024-02-18T00:45:19.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 10 client.41783.objecter _send_command 12 2024-02-18T00:45:19.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.539+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c125720 con 0x7f2b0c1272d0 2024-02-18T00:45:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.541+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c125720 con 0x7f2b0c1272d0 2024-02-18T00:45:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.541+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.541+0000 7f2b027fc700 10 client.41783.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.541+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c127f40 osd=5 4 2024-02-18T00:45:19.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.541+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 5 12 2024-02-18T00:45:19.544 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:45:19.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 10 client.41783.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 13 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2b0c125dd0 0x7f2b0c126230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 13 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 6 13 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c125720 osd=6 4 2024-02-18T00:45:19.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 10 client.41783.objecter _send_command 13 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c126720 con 0x7f2b0c125dd0 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.542+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2b0c125dd0 0x7f2b0c126230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.543+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c125dd0 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.543+0000 7f2b11e20700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2b0c125dd0 0x7f2b0c126230 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.543+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c126720 con 0x7f2b0c125dd0 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.543+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.543+0000 7f2b027fc700 10 client.41783.objecter _finish_command 13 = system:0 2024-02-18T00:45:19.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.543+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c125720 osd=6 4 2024-02-18T00:45:19.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.543+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 6 13 2024-02-18T00:45:19.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 10 client.41783.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 14 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 14 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c125720 osd=6 3 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 6 14 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c125720 osd=6 4 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 10 client.41783.objecter _send_command 14 2024-02-18T00:45:19.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.564+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c126720 con 0x7f2b0c125dd0 2024-02-18T00:45:19.567 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:19.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c126720 con 0x7f2b0c125dd0 2024-02-18T00:45:19.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b027fc700 10 client.41783.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c125720 osd=6 4 2024-02-18T00:45:19.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 6 14 2024-02-18T00:45:19.568 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:45:19.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 10 client.41783.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:19.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 15 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6812/53025171 conn(0x7f2b0c12c690 0x7f2b0c12ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 15 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 7 15 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c126720 osd=7 4 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 10 client.41783.objecter _send_command 15 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b0c131700 con 0x7f2b0c12c690 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.566+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6812/53025171 conn(0x7f2b0c12c690 0x7f2b0c12ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.567+0000 7f2b027fc700 10 client.41783.objecter ms_handle_connect 0x7f2b0c12c690 2024-02-18T00:45:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.567+0000 7f2b1161f700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6812/53025171 conn(0x7f2b0c12c690 0x7f2b0c12ea80 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.567+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f2b0c131700 con 0x7f2b0c12c690 2024-02-18T00:45:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.567+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:45:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.567+0000 7f2b027fc700 10 client.41783.objecter _finish_command 15 = system:0 2024-02-18T00:45:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.567+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c126720 osd=7 4 2024-02-18T00:45:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.567+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 7 15 2024-02-18T00:45:19.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 10 client.41783.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign -1 16 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 20 client.41783.objecter _get_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 15 client.41783.objecter _session_command_op_remove -1 16 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 20 client.41783.objecter get_session s=0x7f2b0c126720 osd=7 3 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 15 client.41783.objecter _session_command_op_assign 7 16 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c126720 osd=7 4 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 10 client.41783.objecter _send_command 16 2024-02-18T00:45:19.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.584+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f2b0c131700 con 0x7f2b0c12c690 2024-02-18T00:45:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b027fc700 1 -- 172.21.15.59:0/1725869750 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f2b0c131700 con 0x7f2b0c12c690 2024-02-18T00:45:19.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b027fc700 10 client.41783.objecter ms_dispatch 0x7f2b0c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b027fc700 10 client.41783.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b027fc700 20 client.41783.objecter put_session s=0x7f2b0c126720 osd=7 4 2024-02-18T00:45:19.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b027fc700 15 client.41783.objecter _session_command_op_remove 7 16 2024-02-18T00:45:19.588 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:45:19.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 10 librados: watch_flush enter 2024-02-18T00:45:19.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 10 librados: watch_flush exit 2024-02-18T00:45:19.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 10 client.41783.objecter close_session for osd.0 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2b0c0598f0 msgr2=0x7f2b0c059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6801/3700238977 conn(0x7f2b0c0598f0 0x7f2b0c059cd0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c06c5b0 osd=0 1 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 10 client.41783.objecter close_session for osd.1 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6805/26279110 conn(0x7f2b0c0594d0 msgr2=0x7f2b0c11b000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6805/26279110 conn(0x7f2b0c0594d0 0x7f2b0c11b000 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c059360 osd=1 1 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 10 client.41783.objecter close_session for osd.2 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2b0c0fed20 msgr2=0x7f2b0c11e880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6809/1153193637 conn(0x7f2b0c0fed20 0x7f2b0c11e880 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c0febb0 osd=2 1 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 10 client.41783.objecter close_session for osd.3 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2b0c027770 msgr2=0x7f2b0c1220a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6813/1805700964 conn(0x7f2b0c027770 0x7f2b0c1220a0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c027600 osd=3 1 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 10 client.41783.objecter close_session for osd.4 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2b0c11f6b0 msgr2=0x7f2b0c11faf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6800/2044387056 conn(0x7f2b0c11f6b0 0x7f2b0c11faf0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.586+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c11c6d0 osd=4 1 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 10 client.41783.objecter close_session for osd.5 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2b0c1272d0 msgr2=0x7f2b0c127750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6804/2170432263 conn(0x7f2b0c1272d0 0x7f2b0c127750 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c127f40 osd=5 1 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 10 client.41783.objecter close_session for osd.6 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2b0c125dd0 msgr2=0x7f2b0c126230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6808/3356009390 conn(0x7f2b0c125dd0 0x7f2b0c126230 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c125720 osd=6 1 2024-02-18T00:45:19.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 10 client.41783.objecter close_session for osd.7 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6812/53025171 conn(0x7f2b0c12c690 msgr2=0x7f2b0c12ea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.137:6812/53025171 conn(0x7f2b0c12c690 0x7f2b0c12ea80 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 20 client.41783.objecter put_session s=0x7f2b0c126720 osd=7 1 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 20 client.41783.objecter shutdown clearing up homeless session... 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 10 client.41783.objecter successfully canceled tick 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2af8062c90 msgr2=0x7f2af8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2af8062c90 0x7f2af8065150 secure :-1 s=READY pgs=3227 cs=0 l=1 rev1=1 rx=0x7f2b0c06c870 tx=0x7f2afc00b040).stop 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b0c0f8420 msgr2=0x7f2b0c06b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 --2- 172.21.15.59:0/1725869750 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b0c0f8420 0x7f2b0c06b0d0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 rx=0x7f2b080044e0 tx=0x7f2b08007f90).stop 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b11e20700 1 -- 172.21.15.59:0/1725869750 reap_dead start 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b11e20700 1 -- 172.21.15.59:0/1725869750 reap_dead start 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b11e20700 1 -- 172.21.15.59:0/1725869750 reap_dead start 2024-02-18T00:45:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b11e20700 1 -- 172.21.15.59:0/1725869750 reap_dead start 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b11e20700 1 -- 172.21.15.59:0/1725869750 reap_dead start 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 shutdown_connections 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 >> 172.21.15.59:0/1725869750 conn(0x7f2b0c0f3800 msgr2=0x7f2b0c0f5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 shutdown_connections 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 -- 172.21.15.59:0/1725869750 wait complete. 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.587+0000 7f2b13883700 1 librados: shutdown 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.593 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.594 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:45:19.625 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:19.653 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:19.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.821+0000 7fa4b6ffd700 1 -- 172.21.15.59:0/2476999962 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v762) v1 ==== 91+0+0 (secure 0 0 0) 0x7fa4c0019070 con 0x7fa4c40f67e0 2024-02-18T00:45:19.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 10 librados: watch_flush enter 2024-02-18T00:45:19.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 10 librados: watch_flush exit 2024-02-18T00:45:19.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 20 client.31977.objecter shutdown clearing up homeless session... 2024-02-18T00:45:19.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 10 client.31977.objecter successfully canceled tick 2024-02-18T00:45:19.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa4b0062c90 msgr2=0x7fa4b0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/2476999962 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa4b0062c90 0x7fa4b0065150 secure :-1 s=READY pgs=3226 cs=0 l=1 rev1=1 rx=0x7fa4c4109960 tx=0x7fa4b800b040).stop 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40f67e0 msgr2=0x7fa4c410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40f67e0 0x7fa4c410e020 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 rx=0x7fa4c000e940 tx=0x7fa4c000b040).stop 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 shutdown_connections 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4c40fe550 0x7fa4c41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4c40f67e0 0x7fa4c410e020 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/2476999962 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa4b0062c90 0x7fa4b0065150 unknown :-1 s=CLOSED pgs=3226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 --2- 172.21.15.59:0/2476999962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4c40fdbf0 0x7fa4c410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.823+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 >> 172.21.15.59:0/2476999962 conn(0x7fa4c4070030 msgr2=0x7fa4c40fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.824+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 shutdown_connections 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.824+0000 7fa4cbeba700 1 -- 172.21.15.59:0/2476999962 wait complete. 2024-02-18T00:45:19.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:19.824+0000 7fa4cbeba700 1 librados: shutdown 2024-02-18T00:45:19.827 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:45:19.853 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:19.856 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:19.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:19.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:19.987 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:20.110 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:20.112 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:19.216116+0000 mon.a (mon.0) 3617 : audit [INF] from='client.? 172.21.15.59:0/2476999962' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:19.432605+0000 mon.c (mon.2) 575 : audit [DBG] from='client.? 172.21.15.59:0/1725869750' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:19.817053+0000 mon.a (mon.0) 3618 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:19.817087+0000 mon.a (mon.0) 3619 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:19.822232+0000 mon.a (mon.0) 3620 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-134", "mode": "readproxy"}]': finished 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:19.822327+0000 mon.a (mon.0) 3621 : audit [INF] from='client.? 172.21.15.59:0/2476999962' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:19.822374+0000 mon.a (mon.0) 3622 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:19.216116+0000 mon.a (mon.0) 3617 : audit [INF] from='client.? 172.21.15.59:0/2476999962' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:19.432605+0000 mon.c (mon.2) 575 : audit [DBG] from='client.? 172.21.15.59:0/1725869750' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:19.817053+0000 mon.a (mon.0) 3618 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:20.113 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:19.817087+0000 mon.a (mon.0) 3619 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:20.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:19.822232+0000 mon.a (mon.0) 3620 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-134", "mode": "readproxy"}]': finished 2024-02-18T00:45:20.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:19.822327+0000 mon.a (mon.0) 3621 : audit [INF] from='client.? 172.21.15.59:0/2476999962' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:45:20.114 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:19.822374+0000 mon.a (mon.0) 3622 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-02-18T00:45:20.114 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:20.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:20.215 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:20.247 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:20.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:19.216116+0000 mon.a (mon.0) 3617 : audit [INF] from='client.? 172.21.15.59:0/2476999962' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:45:20.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:19.432605+0000 mon.c (mon.2) 575 : audit [DBG] from='client.? 172.21.15.59:0/1725869750' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:19.817053+0000 mon.a (mon.0) 3618 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:19.817087+0000 mon.a (mon.0) 3619 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:19.822232+0000 mon.a (mon.0) 3620 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-134", "mode": "readproxy"}]': finished 2024-02-18T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:19.822327+0000 mon.a (mon.0) 3621 : audit [INF] from='client.? 172.21.15.59:0/2476999962' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:45:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:19.822374+0000 mon.a (mon.0) 3622 : cluster [DBG] osdmap e762: 8 total, 8 up, 8 in 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:20.016559+0000 mgr.y (mgr.24454) 656 : cluster [DBG] pgmap v1170: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:20.334204+0000 osd.3 (osd.3) 380 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:20.335848+0000 osd.3 (osd.3) 381 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:20.754290+0000 osd.2 (osd.2) 293 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:20.755186+0000 osd.2 (osd.2) 294 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:20.016559+0000 mgr.y (mgr.24454) 656 : cluster [DBG] pgmap v1170: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:20.334204+0000 osd.3 (osd.3) 380 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:45:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:20.335848+0000 osd.3 (osd.3) 381 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:45:21.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:20.754290+0000 osd.2 (osd.2) 293 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:45:21.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:20.755186+0000 osd.2 (osd.2) 294 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:45:21.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:45:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:45:21.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:20.016559+0000 mgr.y (mgr.24454) 656 : cluster [DBG] pgmap v1170: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:45:21.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:20.334204+0000 osd.3 (osd.3) 380 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:45:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:20.335848+0000 osd.3 (osd.3) 381 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:45:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:20.754290+0000 osd.2 (osd.2) 293 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:45:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:20.755186+0000 osd.2 (osd.2) 294 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:45:22.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:45:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:45:22.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:45:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:22] "GET /metrics HTTP/1.1" 200 192091 "" "Prometheus/2.33.4" 2024-02-18T00:45:22.601 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:22.604 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:22.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:22.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:45:22.705 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:22.738 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:22.760 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:22.762 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:22.763 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:22.864 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:23.175 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:21.373866+0000 osd.3 (osd.3) 382 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:21.375098+0000 osd.3 (osd.3) 383 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:21.610103+0000 mgr.y (mgr.24454) 657 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:21.688004+0000 osd.5 (osd.5) 325 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:21.691765+0000 osd.5 (osd.5) 326 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:22.017652+0000 mgr.y (mgr.24454) 658 : cluster [DBG] pgmap v1171: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 201 B/s wr, 1 op/s 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:21.373866+0000 osd.3 (osd.3) 382 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:21.375098+0000 osd.3 (osd.3) 383 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:21.610103+0000 mgr.y (mgr.24454) 657 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:21.688004+0000 osd.5 (osd.5) 325 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:45:23.176 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:21.691765+0000 osd.5 (osd.5) 326 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:45:23.177 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:22.017652+0000 mgr.y (mgr.24454) 658 : cluster [DBG] pgmap v1171: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 201 B/s wr, 1 op/s 2024-02-18T00:45:23.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:21.373866+0000 osd.3 (osd.3) 382 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:45:23.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:21.375098+0000 osd.3 (osd.3) 383 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:45:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:21.610103+0000 mgr.y (mgr.24454) 657 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:21.688004+0000 osd.5 (osd.5) 325 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:45:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:21.691765+0000 osd.5 (osd.5) 326 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:45:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:22.017652+0000 mgr.y (mgr.24454) 658 : cluster [DBG] pgmap v1171: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 201 B/s wr, 1 op/s 2024-02-18T00:45:24.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:22.808151+0000 osd.6 (osd.6) 335 : cluster [DBG] 25.d scrub starts 2024-02-18T00:45:24.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:22.809500+0000 osd.6 (osd.6) 336 : cluster [DBG] 25.d scrub ok 2024-02-18T00:45:24.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:22.908932+0000 osd.1 (osd.1) 355 : cluster [DBG] 5.c scrub starts 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:22.911908+0000 osd.1 (osd.1) 356 : cluster [DBG] 5.c scrub ok 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:23.207432+0000 osd.7 (osd.7) 223 : cluster [DBG] 234.7 scrub starts 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:23.209379+0000 osd.7 (osd.7) 224 : cluster [DBG] 234.7 scrub ok 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:23.656861+0000 osd.2 (osd.2) 295 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:23.658049+0000 osd.2 (osd.2) 296 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:23.822233+0000 mon.a (mon.0) 3623 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:22.808151+0000 osd.6 (osd.6) 335 : cluster [DBG] 25.d scrub starts 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:22.809500+0000 osd.6 (osd.6) 336 : cluster [DBG] 25.d scrub ok 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:22.908932+0000 osd.1 (osd.1) 355 : cluster [DBG] 5.c scrub starts 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:22.911908+0000 osd.1 (osd.1) 356 : cluster [DBG] 5.c scrub ok 2024-02-18T00:45:24.328 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:23.207432+0000 osd.7 (osd.7) 223 : cluster [DBG] 234.7 scrub starts 2024-02-18T00:45:24.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:23.209379+0000 osd.7 (osd.7) 224 : cluster [DBG] 234.7 scrub ok 2024-02-18T00:45:24.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:23.656861+0000 osd.2 (osd.2) 295 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:45:24.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:23.658049+0000 osd.2 (osd.2) 296 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:45:24.329 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:23.822233+0000 mon.a (mon.0) 3623 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-02-18T00:45:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:22.808151+0000 osd.6 (osd.6) 335 : cluster [DBG] 25.d scrub starts 2024-02-18T00:45:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:22.809500+0000 osd.6 (osd.6) 336 : cluster [DBG] 25.d scrub ok 2024-02-18T00:45:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:22.908932+0000 osd.1 (osd.1) 355 : cluster [DBG] 5.c scrub starts 2024-02-18T00:45:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:22.911908+0000 osd.1 (osd.1) 356 : cluster [DBG] 5.c scrub ok 2024-02-18T00:45:24.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:23.207432+0000 osd.7 (osd.7) 223 : cluster [DBG] 234.7 scrub starts 2024-02-18T00:45:24.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:23.209379+0000 osd.7 (osd.7) 224 : cluster [DBG] 234.7 scrub ok 2024-02-18T00:45:24.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:23.656861+0000 osd.2 (osd.2) 295 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:45:24.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:23.658049+0000 osd.2 (osd.2) 296 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:45:24.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:23.822233+0000 mon.a (mon.0) 3623 : cluster [DBG] osdmap e763: 8 total, 8 up, 8 in 2024-02-18T00:45:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:24.748 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:24.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:24.760 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:24.850 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:24.881 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:24.903 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:24.906 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:24.906 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:25.007 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:25.036 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:25.055 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:25.057 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:25.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:25.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:25.189 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:25.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:45:25.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:24.018212+0000 mgr.y (mgr.24454) 659 : cluster [DBG] pgmap v1173: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-18T00:45:25.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:24.910642+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:25.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:24.018212+0000 mgr.y (mgr.24454) 659 : cluster [DBG] pgmap v1173: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-18T00:45:25.369 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:24.910642+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:25.369 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:24.018212+0000 mgr.y (mgr.24454) 659 : cluster [DBG] pgmap v1173: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-18T00:45:25.369 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:24.910642+0000 mon.a (mon.0) 3624 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:25.693 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:45:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "ops": [], 2024-02-18T00:45:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "complaint_time": 30, 2024-02-18T00:45:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: "num_blocked_ops": 0 2024-02-18T00:45:25.693 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:45:25.938 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:25.941 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:25.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:26.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 -- 172.21.15.59:0/2032327270 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fe530 msgr2=0x7f95280f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 --2- 172.21.15.59:0/2032327270 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fe530 0x7f95280f63b0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 rx=0x7f951c004870 tx=0x7f951c009cb0).stop 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 -- 172.21.15.59:0/2032327270 shutdown_connections 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 --2- 172.21.15.59:0/2032327270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280f6a90 0x7f9528104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 --2- 172.21.15.59:0/2032327270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280fdbd0 0x7f95280fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 --2- 172.21.15.59:0/2032327270 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fe530 0x7f95280f63b0 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 -- 172.21.15.59:0/2032327270 >> 172.21.15.59:0/2032327270 conn(0x7f952806c370 msgr2=0x7f952806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 -- 172.21.15.59:0/2032327270 shutdown_connections 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.722+0000 7f952d863700 1 -- 172.21.15.59:0/2032327270 wait complete. 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 librados: starting msgr at 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 librados: starting objecter 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 Processor -- start 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 -- start start 2024-02-18T00:45:26.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 librados: setting wanted keys 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 librados: calling monclient init 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280f6a90 0x7f952810e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fdbd0 0x7f952810e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280fe530 0x7f9528108330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528108820 con 0x7f95280f6a90 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528108990 con 0x7f95280fe530 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.723+0000 7f952d863700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9528108b50 con 0x7f95280fdbd0 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280f6a90 0x7f952810e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280f6a90 0x7f952810e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33224/0 (socket says 172.21.15.59:33224) 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 -- 172.21.15.59:0/162409417 learned_addr learned my addr 172.21.15.59:0/162409417 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f95267fc700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fdbd0 0x7f952810e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:26.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f95277fe700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280fe530 0x7f9528108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 -- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fdbd0 msgr2=0x7f952810e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fdbd0 0x7f952810e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 -- 172.21.15.59:0/162409417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280fe530 msgr2=0x7f9528108330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280fe530 0x7f9528108330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 -- 172.21.15.59:0/162409417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518005040 con 0x7f95280f6a90 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f95277fe700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280fe530 0x7f9528108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f9526ffd700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280f6a90 0x7f952810e190 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 rx=0x7f95180048f0 tx=0x7f9518009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f950ffff700 10 client.?.objecter ms_handle_connect 0x7f95280f6a90 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f950ffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f950ffff700 1 -- 172.21.15.59:0/162409417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518007b00 con 0x7f95280f6a90 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c005040 con 0x7f95280f6a90 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f950ffff700 1 -- 172.21.15.59:0/162409417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951800f040 con 0x7f95280f6a90 2024-02-18T00:45:26.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f950ffff700 1 -- 172.21.15.59:0/162409417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9518014a00 con 0x7f95280f6a90 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f952d863700 10 client.31989.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9528109190 con 0x7f95280f6a90 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.724+0000 7f952d863700 1 librados: init done 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.725+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952805f910 con 0x7f95280f6a90 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f950ffff700 1 -- 172.21.15.59:0/162409417 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f95180133c0 con 0x7f95280f6a90 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f950ffff700 1 --2- 172.21.15.59:0/162409417 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9510062c90 0x7f9510065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f950ffff700 1 -- 172.21.15.59:0/162409417 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(763..763 src has 418..763) v4 ==== 8814+0+0 (secure 0 0 0) 0x7f951807d0b0 con 0x7f95280f6a90 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f950ffff700 10 client.31989.objecter ms_dispatch 0x7f9528001550 osd_map(763..763 src has 418..763) v4 2024-02-18T00:45:26.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f950ffff700 3 client.31989.objecter handle_osd_map got epochs [763,763] > 0 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f950ffff700 3 client.31989.objecter handle_osd_map decoding full epoch 763 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f95267fc700 1 --2- 172.21.15.59:0/162409417 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9510062c90 0x7f9510065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.726+0000 7f950ffff700 20 client.31989.objecter dump_active .. 0 homeless 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.727+0000 7f950ffff700 10 client.31989.objecter ms_handle_connect 0x7f9510062c90 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.727+0000 7f95267fc700 1 --2- 172.21.15.59:0/162409417 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9510062c90 0x7f9510065150 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 rx=0x7f951c004ee0 tx=0x7f951c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.731+0000 7f950ffff700 1 -- 172.21.15.59:0/162409417 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9518050230 con 0x7f95280f6a90 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.900+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9528109810 con 0x7f9510062c90 2024-02-18T00:45:26.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:25.915+0000 7f950ffff700 1 -- 172.21.15.59:0/162409417 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+549583 (secure 0 0 0) 0x7f9528109810 con 0x7f9510062c90 2024-02-18T00:45:26.024 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:45:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.014+0000 7f952d863700 10 librados: watch_flush enter 2024-02-18T00:45:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.014+0000 7f952d863700 10 librados: watch_flush exit 2024-02-18T00:45:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.014+0000 7f952d863700 20 client.31989.objecter shutdown clearing up homeless session... 2024-02-18T00:45:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.014+0000 7f952d863700 10 client.31989.objecter successfully canceled tick 2024-02-18T00:45:26.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9510062c90 msgr2=0x7f9510065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 --2- 172.21.15.59:0/162409417 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9510062c90 0x7f9510065150 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 rx=0x7f951c004ee0 tx=0x7f951c00b040).stop 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280f6a90 msgr2=0x7f952810e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280f6a90 0x7f952810e190 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 rx=0x7f95180048f0 tx=0x7f9518009cb0).stop 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 shutdown_connections 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 --2- 172.21.15.59:0/162409417 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9510062c90 0x7f9510065150 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280fe530 0x7f9528108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95280f6a90 0x7f952810e190 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 --2- 172.21.15.59:0/162409417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95280fdbd0 0x7f952810e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 >> 172.21.15.59:0/162409417 conn(0x7f952806c370 msgr2=0x7f95280fb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 shutdown_connections 2024-02-18T00:45:26.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 -- 172.21.15.59:0/162409417 wait complete. 2024-02-18T00:45:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:26.015+0000 7f952d863700 1 librados: shutdown 2024-02-18T00:45:26.036 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:45:26.042 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:26.074 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:26.097 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:26.099 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:26.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:26.201 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:26.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:24.790214+0000 osd.6 (osd.6) 337 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:45:26.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:24.791375+0000 osd.6 (osd.6) 338 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:45:26.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:24.892515+0000 osd.4 (osd.4) 403 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:45:26.227 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:24.894516+0000 osd.4 (osd.4) 404 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:45:26.228 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:24.790214+0000 osd.6 (osd.6) 337 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:45:26.228 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:24.791375+0000 osd.6 (osd.6) 338 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:45:26.228 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:24.892515+0000 osd.4 (osd.4) 403 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:45:26.228 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:24.894516+0000 osd.4 (osd.4) 404 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:45:26.229 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:26.328 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:26.330 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:26.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:26.432 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:26.462 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:24.790214+0000 osd.6 (osd.6) 337 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:45:26.462 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:24.791375+0000 osd.6 (osd.6) 338 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:45:26.462 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:24.892515+0000 osd.4 (osd.4) 403 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:45:26.462 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:24.894516+0000 osd.4 (osd.4) 404 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:45:26.464 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:26.486 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:26.488 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:26.489 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:26.590 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:26.619 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:26.639 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:26.641 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:26.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:26.743 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:26.771 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:27.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:25.900345+0000 mgr.y (mgr.24454) 660 : audit [DBG] from='client.31989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:45:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:26.019068+0000 mgr.y (mgr.24454) 661 : cluster [DBG] pgmap v1174: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 142 B/s wr, 1 op/s 2024-02-18T00:45:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:26.281264+0000 osd.3 (osd.3) 384 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:45:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:26.282470+0000 osd.3 (osd.3) 385 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:45:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:25.900345+0000 mgr.y (mgr.24454) 660 : audit [DBG] from='client.31989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:45:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:26.019068+0000 mgr.y (mgr.24454) 661 : cluster [DBG] pgmap v1174: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 142 B/s wr, 1 op/s 2024-02-18T00:45:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:26.281264+0000 osd.3 (osd.3) 384 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:45:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:26.282470+0000 osd.3 (osd.3) 385 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:45:27.438 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":1173,"stamp":"2024-02-18T00:45:24.017893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4006058,"num_objects":263,"num_object_clones":1,"num_object_copies":789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":0,"num_read":1940,"num_read_kb":1756,"num_write":637,"num_write_kb":3991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":2492,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1466,"ondisk_log_size":1466,"up":642,"acting":642,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":638,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2472596,"kb_used_data":53844,"kb_used_omap":0,"kb_used_meta":2418688,"kb_avail":747488620,"statfs":{"total":767960285184,"available":765428346880,"internally_reserved":0,"allocated":55136256,"data_stored":32390851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2476736512},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002141"},"pg_stats":[{"pgid":"251.1f","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056039+0000","last_change":"2024-02-18T00:45:16.949627+0000","last_active":"2024-02-18T00:45:19.056039+0000","last_peered":"2024-02-18T00:45:19.056039+0000","last_clean":"2024-02-18T00:45:19.056039+0000","last_became_active":"2024-02-18T00:45:16.949340+0000","last_became_peered":"2024-02-18T00:45:16.949340+0000","last_unstale":"2024-02-18T00:45:19.056039+0000","last_undegraded":"2024-02-18T00:45:19.056039+0000","last_fullsized":"2024-02-18T00:45:19.056039+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"251.1e","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054947+0000","last_change":"2024-02-18T00:45:16.948769+0000","last_active":"2024-02-18T00:45:19.054947+0000","last_peered":"2024-02-18T00:45:19.054947+0000","last_clean":"2024-02-18T00:45:19.054947+0000","last_became_active":"2024-02-18T00:45:16.948469+0000","last_became_peered":"2024-02-18T00:45:16.948469+0000","last_unstale":"2024-02-18T00:45:19.054947+0000","last_undegraded":"2024-02-18T00:45:19.054947+0000","last_fullsized":"2024-02-18T00:45:19.054947+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"251.1d","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056277+0000","last_change":"2024-02-18T00:45:16.952247+0000","last_active":"2024-02-18T00:45:19.056277+0000","last_peered":"2024-02-18T00:45:19.056277+0000","last_clean":"2024-02-18T00:45:19.056277+0000","last_became_active":"2024-02-18T00:45:16.951721+0000","last_became_peered":"2024-02-18T00:45:16.951721+0000","last_unstale":"2024-02-18T00:45:19.056277+0000","last_undegraded":"2024-02-18T00:45:19.056277+0000","last_fullsized":"2024-02-18T00:45:19.056277+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"251.1c","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820853+0000","last_change":"2024-02-18T00:45:16.954973+0000","last_active":"2024-02-18T00:45:18.820853+0000","last_peered":"2024-02-18T00:45:18.820853+0000","last_clean":"2024-02-18T00:45:18.820853+0000","last_became_active":"2024-02-18T00:45:16.954659+0000","last_became_peered":"2024-02-18T00:45:16.954659+0000","last_unstale":"2024-02-18T00:45:18.820853+0000","last_undegraded":"2024-02-18T00:45:18.820853+0000","last_fullsized":"2024-02-18T00:45:18.820853+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,6],"acting":[2,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"251.1b","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053344+0000","last_change":"2024-02-18T00:45:16.958110+0000","last_active":"2024-02-18T00:45:19.053344+0000","last_peered":"2024-02-18T00:45:19.053344+0000","last_clean":"2024-02-18T00:45:19.053344+0000","last_became_active":"2024-02-18T00:45:16.957651+0000","last_became_peered":"2024-02-18T00:45:16.957651+0000","last_unstale":"2024-02-18T00:45:19.053344+0000","last_undegraded":"2024-02-18T00:45:19.053344+0000","last_fullsized":"2024-02-18T00:45:19.053344+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"251.1a","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053112+0000","last_change":"2024-02-18T00:45:16.956567+0000","last_active":"2024-02-18T00:45:19.053112+0000","last_peered":"2024-02-18T00:45:19.053112+0000","last_clean":"2024-02-18T00:45:19.053112+0000","last_became_active":"2024-02-18T00:45:16.956335+0000","last_became_peered":"2024-02-18T00:45:16.956335+0000","last_unstale":"2024-02-18T00:45:19.053112+0000","last_undegraded":"2024-02-18T00:45:19.053112+0000","last_fullsized":"2024-02-18T00:45:19.053112+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"251.19","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054160+0000","last_change":"2024-02-18T00:45:16.958747+0000","last_active":"2024-02-18T00:45:19.054160+0000","last_peered":"2024-02-18T00:45:19.054160+0000","last_clean":"2024-02-18T00:45:19.054160+0000","last_became_active":"2024-02-18T00:45:16.957569+0000","last_became_peered":"2024-02-18T00:45:16.957569+0000","last_unstale":"2024-02-18T00:45:19.054160+0000","last_undegraded":"2024-02-18T00:45:19.054160+0000","last_fullsized":"2024-02-18T00:45:19.054160+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"251.18","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055525+0000","last_change":"2024-02-18T00:45:16.951659+0000","last_active":"2024-02-18T00:45:19.055525+0000","last_peered":"2024-02-18T00:45:19.055525+0000","last_clean":"2024-02-18T00:45:19.055525+0000","last_became_active":"2024-02-18T00:45:16.951316+0000","last_became_peered":"2024-02-18T00:45:16.951316+0000","last_unstale":"2024-02-18T00:45:19.055525+0000","last_undegraded":"2024-02-18T00:45:19.055525+0000","last_fullsized":"2024-02-18T00:45:19.055525+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"251.f","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055307+0000","last_change":"2024-02-18T00:45:16.950468+0000","last_active":"2024-02-18T00:45:19.055307+0000","last_peered":"2024-02-18T00:45:19.055307+0000","last_clean":"2024-02-18T00:45:19.055307+0000","last_became_active":"2024-02-18T00:45:16.950155+0000","last_became_peered":"2024-02-18T00:45:16.950155+0000","last_unstale":"2024-02-18T00:45:19.055307+0000","last_undegraded":"2024-02-18T00:45:19.055307+0000","last_fullsized":"2024-02-18T00:45:19.055307+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"251.e","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055040+0000","last_change":"2024-02-18T00:45:16.956534+0000","last_active":"2024-02-18T00:45:19.055040+0000","last_peered":"2024-02-18T00:45:19.055040+0000","last_clean":"2024-02-18T00:45:19.055040+0000","last_became_active":"2024-02-18T00:45:16.956066+0000","last_became_peered":"2024-02-18T00:45:16.956066+0000","last_unstale":"2024-02-18T00:45:19.055040+0000","last_undegraded":"2024-02-18T00:45:19.055040+0000","last_fullsized":"2024-02-18T00:45:19.055040+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,5],"acting":[3,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"251.d","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054964+0000","last_change":"2024-02-18T00:45:16.958214+0000","last_active":"2024-02-18T00:45:19.054964+0000","last_peered":"2024-02-18T00:45:19.054964+0000","last_clean":"2024-02-18T00:45:19.054964+0000","last_became_active":"2024-02-18T00:45:16.957920+0000","last_became_peered":"2024-02-18T00:45:16.957920+0000","last_unstale":"2024-02-18T00:45:19.054964+0000","last_undegraded":"2024-02-18T00:45:19.054964+0000","last_fullsized":"2024-02-18T00:45:19.054964+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"251.c","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054331+0000","last_change":"2024-02-18T00:45:16.953577+0000","last_active":"2024-02-18T00:45:19.054331+0000","last_peered":"2024-02-18T00:45:19.054331+0000","last_clean":"2024-02-18T00:45:19.054331+0000","last_became_active":"2024-02-18T00:45:16.953341+0000","last_became_peered":"2024-02-18T00:45:16.953341+0000","last_unstale":"2024-02-18T00:45:19.054331+0000","last_undegraded":"2024-02-18T00:45:19.054331+0000","last_fullsized":"2024-02-18T00:45:19.054331+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"251.b","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821412+0000","last_change":"2024-02-18T00:45:16.947831+0000","last_active":"2024-02-18T00:45:18.821412+0000","last_peered":"2024-02-18T00:45:18.821412+0000","last_clean":"2024-02-18T00:45:18.821412+0000","last_became_active":"2024-02-18T00:45:16.947568+0000","last_became_peered":"2024-02-18T00:45:16.947568+0000","last_unstale":"2024-02-18T00:45:18.821412+0000","last_undegraded":"2024-02-18T00:45:18.821412+0000","last_fullsized":"2024-02-18T00:45:18.821412+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"251.a","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054196+0000","last_change":"2024-02-18T00:45:16.952937+0000","last_active":"2024-02-18T00:45:19.054196+0000","last_peered":"2024-02-18T00:45:19.054196+0000","last_clean":"2024-02-18T00:45:19.054196+0000","last_became_active":"2024-02-18T00:45:16.952137+0000","last_became_peered":"2024-02-18T00:45:16.952137+0000","last_unstale":"2024-02-18T00:45:19.054196+0000","last_undegraded":"2024-02-18T00:45:19.054196+0000","last_fullsized":"2024-02-18T00:45:19.054196+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"251.9","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054259+0000","last_change":"2024-02-18T00:45:16.958542+0000","last_active":"2024-02-18T00:45:19.054259+0000","last_peered":"2024-02-18T00:45:19.054259+0000","last_clean":"2024-02-18T00:45:19.054259+0000","last_became_active":"2024-02-18T00:45:16.958265+0000","last_became_peered":"2024-02-18T00:45:16.958265+0000","last_unstale":"2024-02-18T00:45:19.054259+0000","last_undegraded":"2024-02-18T00:45:19.054259+0000","last_fullsized":"2024-02-18T00:45:19.054259+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,4],"acting":[1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"251.8","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821091+0000","last_change":"2024-02-18T00:45:16.953577+0000","last_active":"2024-02-18T00:45:18.821091+0000","last_peered":"2024-02-18T00:45:18.821091+0000","last_clean":"2024-02-18T00:45:18.821091+0000","last_became_active":"2024-02-18T00:45:16.953297+0000","last_became_peered":"2024-02-18T00:45:16.953297+0000","last_unstale":"2024-02-18T00:45:18.821091+0000","last_undegraded":"2024-02-18T00:45:18.821091+0000","last_fullsized":"2024-02-18T00:45:18.821091+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"251.7","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055686+0000","last_change":"2024-02-18T00:45:16.961124+0000","last_active":"2024-02-18T00:45:19.055686+0000","last_peered":"2024-02-18T00:45:19.055686+0000","last_clean":"2024-02-18T00:45:19.055686+0000","last_became_active":"2024-02-18T00:45:16.960853+0000","last_became_peered":"2024-02-18T00:45:16.960853+0000","last_unstale":"2024-02-18T00:45:19.055686+0000","last_undegraded":"2024-02-18T00:45:19.055686+0000","last_fullsized":"2024-02-18T00:45:19.055686+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"251.6","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055968+0000","last_change":"2024-02-18T00:45:16.958022+0000","last_active":"2024-02-18T00:45:19.055968+0000","last_peered":"2024-02-18T00:45:19.055968+0000","last_clean":"2024-02-18T00:45:19.055968+0000","last_became_active":"2024-02-18T00:45:16.955338+0000","last_became_peered":"2024-02-18T00:45:16.955338+0000","last_unstale":"2024-02-18T00:45:19.055968+0000","last_undegraded":"2024-02-18T00:45:19.055968+0000","last_fullsized":"2024-02-18T00:45:19.055968+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"251.5","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819938+0000","last_change":"2024-02-18T00:45:16.951890+0000","last_active":"2024-02-18T00:45:18.819938+0000","last_peered":"2024-02-18T00:45:18.819938+0000","last_clean":"2024-02-18T00:45:18.819938+0000","last_became_active":"2024-02-18T00:45:16.951551+0000","last_became_peered":"2024-02-18T00:45:16.951551+0000","last_unstale":"2024-02-18T00:45:18.819938+0000","last_undegraded":"2024-02-18T00:45:18.819938+0000","last_fullsized":"2024-02-18T00:45:18.819938+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"251.4","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053965+0000","last_change":"2024-02-18T00:45:16.958254+0000","last_active":"2024-02-18T00:45:19.053965+0000","last_peered":"2024-02-18T00:45:19.053965+0000","last_clean":"2024-02-18T00:45:19.053965+0000","last_became_active":"2024-02-18T00:45:16.957942+0000","last_became_peered":"2024-02-18T00:45:16.957942+0000","last_unstale":"2024-02-18T00:45:19.053965+0000","last_undegraded":"2024-02-18T00:45:19.053965+0000","last_fullsized":"2024-02-18T00:45:19.053965+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"251.3","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056155+0000","last_change":"2024-02-18T00:45:16.954702+0000","last_active":"2024-02-18T00:45:19.056155+0000","last_peered":"2024-02-18T00:45:19.056155+0000","last_clean":"2024-02-18T00:45:19.056155+0000","last_became_active":"2024-02-18T00:45:16.954333+0000","last_became_peered":"2024-02-18T00:45:16.954333+0000","last_unstale":"2024-02-18T00:45:19.056155+0000","last_undegraded":"2024-02-18T00:45:19.056155+0000","last_fullsized":"2024-02-18T00:45:19.056155+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"251.2","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054768+0000","last_change":"2024-02-18T00:45:16.957591+0000","last_active":"2024-02-18T00:45:19.054768+0000","last_peered":"2024-02-18T00:45:19.054768+0000","last_clean":"2024-02-18T00:45:19.054768+0000","last_became_active":"2024-02-18T00:45:16.957258+0000","last_became_peered":"2024-02-18T00:45:16.957258+0000","last_unstale":"2024-02-18T00:45:19.054768+0000","last_undegraded":"2024-02-18T00:45:19.054768+0000","last_fullsized":"2024-02-18T00:45:19.054768+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"251.1","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056296+0000","last_change":"2024-02-18T00:45:16.949601+0000","last_active":"2024-02-18T00:45:19.056296+0000","last_peered":"2024-02-18T00:45:19.056296+0000","last_clean":"2024-02-18T00:45:19.056296+0000","last_became_active":"2024-02-18T00:45:16.949279+0000","last_became_peered":"2024-02-18T00:45:16.949279+0000","last_unstale":"2024-02-18T00:45:19.056296+0000","last_undegraded":"2024-02-18T00:45:19.056296+0000","last_fullsized":"2024-02-18T00:45:19.056296+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"251.0","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820427+0000","last_change":"2024-02-18T00:45:16.951493+0000","last_active":"2024-02-18T00:45:18.820427+0000","last_peered":"2024-02-18T00:45:18.820427+0000","last_clean":"2024-02-18T00:45:18.820427+0000","last_became_active":"2024-02-18T00:45:16.951174+0000","last_became_peered":"2024-02-18T00:45:16.951174+0000","last_unstale":"2024-02-18T00:45:18.820427+0000","last_undegraded":"2024-02-18T00:45:18.820427+0000","last_fullsized":"2024-02-18T00:45:18.820427+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.29","version":"731'34","reported_seq":638,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054254+0000","last_change":"2024-02-18T00:44:40.215823+0000","last_active":"2024-02-18T00:45:19.054254+0000","last_peered":"2024-02-18T00:45:19.054254+0000","last_clean":"2024-02-18T00:45:19.054254+0000","last_became_active":"2024-02-18T00:43:09.798744+0000","last_became_peered":"2024-02-18T00:43:09.798744+0000","last_unstale":"2024-02-18T00:45:19.054254+0000","last_undegraded":"2024-02-18T00:45:19.054254+0000","last_fullsized":"2024-02-18T00:45:19.054254+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"731'34","last_scrub_stamp":"2024-02-18T00:44:40.215762+0000","last_deep_scrub":"731'34","last_deep_scrub_stamp":"2024-02-18T00:44:40.215762+0000","last_clean_scrub_stamp":"2024-02-18T00:44:40.215762+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"457'39","reported_seq":672,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053667+0000","last_change":"2024-02-18T00:43:35.024037+0000","last_active":"2024-02-18T00:45:19.053667+0000","last_peered":"2024-02-18T00:45:19.053667+0000","last_clean":"2024-02-18T00:45:19.053667+0000","last_became_active":"2024-02-18T00:43:09.791383+0000","last_became_peered":"2024-02-18T00:43:09.791383+0000","last_unstale":"2024-02-18T00:45:19.053667+0000","last_undegraded":"2024-02-18T00:45:19.053667+0000","last_fullsized":"2024-02-18T00:45:19.053667+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:43:35.023935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.023935+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"251.16","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820204+0000","last_change":"2024-02-18T00:45:16.955237+0000","last_active":"2024-02-18T00:45:18.820204+0000","last_peered":"2024-02-18T00:45:18.820204+0000","last_clean":"2024-02-18T00:45:18.820204+0000","last_became_active":"2024-02-18T00:45:16.954990+0000","last_became_peered":"2024-02-18T00:45:16.954990+0000","last_unstale":"2024-02-18T00:45:18.820204+0000","last_undegraded":"2024-02-18T00:45:18.820204+0000","last_fullsized":"2024-02-18T00:45:18.820204+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"234.7","version":"745'8","reported_seq":187,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055406+0000","last_change":"2024-02-18T00:44:13.812846+0000","last_active":"2024-02-18T00:45:19.055406+0000","last_peered":"2024-02-18T00:45:19.055406+0000","last_clean":"2024-02-18T00:45:19.055406+0000","last_became_active":"2024-02-18T00:42:19.767995+0000","last_became_peered":"2024-02-18T00:42:19.767995+0000","last_unstale":"2024-02-18T00:45:19.055406+0000","last_undegraded":"2024-02-18T00:45:19.055406+0000","last_fullsized":"2024-02-18T00:45:19.055406+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"251.17","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056168+0000","last_change":"2024-02-18T00:45:16.957625+0000","last_active":"2024-02-18T00:45:19.056168+0000","last_peered":"2024-02-18T00:45:19.056168+0000","last_clean":"2024-02-18T00:45:19.056168+0000","last_became_active":"2024-02-18T00:45:16.957325+0000","last_became_peered":"2024-02-18T00:45:16.957325+0000","last_unstale":"2024-02-18T00:45:19.056168+0000","last_undegraded":"2024-02-18T00:45:19.056168+0000","last_fullsized":"2024-02-18T00:45:19.056168+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.6","version":"745'14","reported_seq":203,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054357+0000","last_change":"2024-02-18T00:44:35.346778+0000","last_active":"2024-02-18T00:45:19.054357+0000","last_peered":"2024-02-18T00:45:19.054357+0000","last_clean":"2024-02-18T00:45:19.054357+0000","last_became_active":"2024-02-18T00:42:19.766349+0000","last_became_peered":"2024-02-18T00:42:19.766349+0000","last_unstale":"2024-02-18T00:45:19.054357+0000","last_undegraded":"2024-02-18T00:45:19.054357+0000","last_fullsized":"2024-02-18T00:45:19.054357+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"721'13","last_scrub_stamp":"2024-02-18T00:44:35.346673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.346673+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000021,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"251.14","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056632+0000","last_change":"2024-02-18T00:45:16.956634+0000","last_active":"2024-02-18T00:45:19.056632+0000","last_peered":"2024-02-18T00:45:19.056632+0000","last_clean":"2024-02-18T00:45:19.056632+0000","last_became_active":"2024-02-18T00:45:16.955692+0000","last_became_peered":"2024-02-18T00:45:16.955692+0000","last_unstale":"2024-02-18T00:45:19.056632+0000","last_undegraded":"2024-02-18T00:45:19.056632+0000","last_fullsized":"2024-02-18T00:45:19.056632+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"234.5","version":"745'21","reported_seq":216,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820928+0000","last_change":"2024-02-18T00:44:39.133161+0000","last_active":"2024-02-18T00:45:18.820928+0000","last_peered":"2024-02-18T00:45:18.820928+0000","last_clean":"2024-02-18T00:45:18.820928+0000","last_became_active":"2024-02-18T00:42:19.763983+0000","last_became_peered":"2024-02-18T00:42:19.763983+0000","last_unstale":"2024-02-18T00:45:18.820928+0000","last_undegraded":"2024-02-18T00:45:18.820928+0000","last_fullsized":"2024-02-18T00:45:18.820928+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"705'17","last_scrub_stamp":"2024-02-18T00:44:39.133079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:39.133079+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":55,"num_objects":9,"num_object_clones":1,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"251.15","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820814+0000","last_change":"2024-02-18T00:45:16.953534+0000","last_active":"2024-02-18T00:45:18.820814+0000","last_peered":"2024-02-18T00:45:18.820814+0000","last_clean":"2024-02-18T00:45:18.820814+0000","last_became_active":"2024-02-18T00:45:16.953223+0000","last_became_peered":"2024-02-18T00:45:16.953223+0000","last_unstale":"2024-02-18T00:45:18.820814+0000","last_undegraded":"2024-02-18T00:45:18.820814+0000","last_fullsized":"2024-02-18T00:45:18.820814+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,4],"acting":[2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"234.4","version":"745'3","reported_seq":186,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056529+0000","last_change":"2024-02-18T00:44:13.817133+0000","last_active":"2024-02-18T00:45:19.056529+0000","last_peered":"2024-02-18T00:45:19.056529+0000","last_clean":"2024-02-18T00:45:19.056529+0000","last_became_active":"2024-02-18T00:42:19.767925+0000","last_became_peered":"2024-02-18T00:42:19.767925+0000","last_unstale":"2024-02-18T00:45:19.056529+0000","last_undegraded":"2024-02-18T00:45:19.056529+0000","last_fullsized":"2024-02-18T00:45:19.056529+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":24,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"251.12","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053251+0000","last_change":"2024-02-18T00:45:16.956464+0000","last_active":"2024-02-18T00:45:19.053251+0000","last_peered":"2024-02-18T00:45:19.053251+0000","last_clean":"2024-02-18T00:45:19.053251+0000","last_became_active":"2024-02-18T00:45:16.956170+0000","last_became_peered":"2024-02-18T00:45:16.956170+0000","last_unstale":"2024-02-18T00:45:19.053251+0000","last_undegraded":"2024-02-18T00:45:19.053251+0000","last_fullsized":"2024-02-18T00:45:19.053251+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"234.3","version":"745'8","reported_seq":200,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.057141+0000","last_change":"2024-02-18T00:44:27.711800+0000","last_active":"2024-02-18T00:45:19.057141+0000","last_peered":"2024-02-18T00:45:19.057141+0000","last_clean":"2024-02-18T00:45:19.057141+0000","last_became_active":"2024-02-18T00:42:19.767739+0000","last_became_peered":"2024-02-18T00:42:19.767739+0000","last_unstale":"2024-02-18T00:45:19.057141+0000","last_undegraded":"2024-02-18T00:45:19.057141+0000","last_fullsized":"2024-02-18T00:45:19.057141+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'5","last_scrub_stamp":"2024-02-18T00:44:27.711710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:27.711710+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"251.13","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053423+0000","last_change":"2024-02-18T00:45:16.957933+0000","last_active":"2024-02-18T00:45:19.053423+0000","last_peered":"2024-02-18T00:45:19.053423+0000","last_clean":"2024-02-18T00:45:19.053423+0000","last_became_active":"2024-02-18T00:45:16.957347+0000","last_became_peered":"2024-02-18T00:45:16.957347+0000","last_unstale":"2024-02-18T00:45:19.053423+0000","last_undegraded":"2024-02-18T00:45:19.053423+0000","last_fullsized":"2024-02-18T00:45:19.053423+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"234.2","version":"745'13","reported_seq":204,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053646+0000","last_change":"2024-02-18T00:44:31.444702+0000","last_active":"2024-02-18T00:45:19.053646+0000","last_peered":"2024-02-18T00:45:19.053646+0000","last_clean":"2024-02-18T00:45:19.053646+0000","last_became_active":"2024-02-18T00:42:19.763974+0000","last_became_peered":"2024-02-18T00:42:19.763974+0000","last_unstale":"2024-02-18T00:45:19.053646+0000","last_undegraded":"2024-02-18T00:45:19.053646+0000","last_fullsized":"2024-02-18T00:45:19.053646+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'12","last_scrub_stamp":"2024-02-18T00:44:31.444627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.444627+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"251.10","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053820+0000","last_change":"2024-02-18T00:45:16.956250+0000","last_active":"2024-02-18T00:45:19.053820+0000","last_peered":"2024-02-18T00:45:19.053820+0000","last_clean":"2024-02-18T00:45:19.053820+0000","last_became_active":"2024-02-18T00:45:16.955875+0000","last_became_peered":"2024-02-18T00:45:16.955875+0000","last_unstale":"2024-02-18T00:45:19.053820+0000","last_undegraded":"2024-02-18T00:45:19.053820+0000","last_fullsized":"2024-02-18T00:45:19.053820+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"234.1","version":"759'12","reported_seq":197,"reported_epoch":762,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.826068+0000","last_change":"2024-02-18T00:44:13.817532+0000","last_active":"2024-02-18T00:45:19.826068+0000","last_peered":"2024-02-18T00:45:19.826068+0000","last_clean":"2024-02-18T00:45:19.826068+0000","last_became_active":"2024-02-18T00:42:19.763959+0000","last_became_peered":"2024-02-18T00:42:19.763959+0000","last_unstale":"2024-02-18T00:45:19.826068+0000","last_undegraded":"2024-02-18T00:45:19.826068+0000","last_fullsized":"2024-02-18T00:45:19.826068+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:42:18.749711+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"251.11","version":"0'0","reported_seq":12,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053374+0000","last_change":"2024-02-18T00:45:16.948124+0000","last_active":"2024-02-18T00:45:19.053374+0000","last_peered":"2024-02-18T00:45:19.053374+0000","last_clean":"2024-02-18T00:45:19.053374+0000","last_became_active":"2024-02-18T00:45:16.947710+0000","last_became_peered":"2024-02-18T00:45:16.947710+0000","last_unstale":"2024-02-18T00:45:19.053374+0000","last_undegraded":"2024-02-18T00:45:19.053374+0000","last_fullsized":"2024-02-18T00:45:19.053374+0000","mapping_epoch":758,"log_start":"0'0","ondisk_log_start":"0'0","created":758,"last_epoch_clean":759,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:15.919602+0000","last_clean_scrub_stamp":"2024-02-18T00:45:15.919602+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"234.0","version":"745'12","reported_seq":203,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.818899+0000","last_change":"2024-02-18T00:44:35.784224+0000","last_active":"2024-02-18T00:45:18.818899+0000","last_peered":"2024-02-18T00:45:18.818899+0000","last_clean":"2024-02-18T00:45:18.818899+0000","last_became_active":"2024-02-18T00:42:19.767896+0000","last_became_peered":"2024-02-18T00:42:19.767896+0000","last_unstale":"2024-02-18T00:45:18.818899+0000","last_undegraded":"2024-02-18T00:45:18.818899+0000","last_fullsized":"2024-02-18T00:45:18.818899+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"630'10","last_scrub_stamp":"2024-02-18T00:44:35.784138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.784138+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":26,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":695,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054029+0000","last_change":"2024-02-18T00:43:50.556280+0000","last_active":"2024-02-18T00:45:19.054029+0000","last_peered":"2024-02-18T00:45:19.054029+0000","last_clean":"2024-02-18T00:45:19.054029+0000","last_became_active":"2024-02-18T00:39:36.751760+0000","last_became_peered":"2024-02-18T00:39:36.751760+0000","last_unstale":"2024-02-18T00:45:19.054029+0000","last_undegraded":"2024-02-18T00:45:19.054029+0000","last_fullsized":"2024-02-18T00:45:19.054029+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.556181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.556181+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":688,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054145+0000","last_change":"2024-02-18T00:43:37.205168+0000","last_active":"2024-02-18T00:45:19.054145+0000","last_peered":"2024-02-18T00:45:19.054145+0000","last_clean":"2024-02-18T00:45:19.054145+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:45:19.054145+0000","last_undegraded":"2024-02-18T00:45:19.054145+0000","last_fullsized":"2024-02-18T00:45:19.054145+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:37.205089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.205089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.21","version":"457'34","reported_seq":107,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054097+0000","last_change":"2024-02-18T00:44:41.710983+0000","last_active":"2024-02-18T00:45:19.054097+0000","last_peered":"2024-02-18T00:45:19.054097+0000","last_clean":"2024-02-18T00:45:19.054097+0000","last_became_active":"2024-02-18T00:43:44.817896+0000","last_became_peered":"2024-02-18T00:43:44.817896+0000","last_unstale":"2024-02-18T00:45:19.054097+0000","last_undegraded":"2024-02-18T00:45:19.054097+0000","last_fullsized":"2024-02-18T00:45:19.054097+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:44:41.710910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:41.710910+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055721+0000","last_change":"2024-02-18T00:42:55.766143+0000","last_active":"2024-02-18T00:45:19.055721+0000","last_peered":"2024-02-18T00:45:19.055721+0000","last_clean":"2024-02-18T00:45:19.055721+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:45:19.055721+0000","last_undegraded":"2024-02-18T00:45:19.055721+0000","last_fullsized":"2024-02-18T00:45:19.055721+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:55.766062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:42:55.766062+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.20","version":"457'52","reported_seq":145,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053258+0000","last_change":"2024-02-18T00:43:52.539836+0000","last_active":"2024-02-18T00:45:19.053258+0000","last_peered":"2024-02-18T00:45:19.053258+0000","last_clean":"2024-02-18T00:45:19.053258+0000","last_became_active":"2024-02-18T00:43:09.796538+0000","last_became_peered":"2024-02-18T00:43:09.796538+0000","last_unstale":"2024-02-18T00:45:19.053258+0000","last_undegraded":"2024-02-18T00:45:19.053258+0000","last_fullsized":"2024-02-18T00:45:19.053258+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:43:52.539763+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.539763+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055242+0000","last_change":"2024-02-18T00:43:10.875551+0000","last_active":"2024-02-18T00:45:19.055242+0000","last_peered":"2024-02-18T00:45:19.055242+0000","last_clean":"2024-02-18T00:45:19.055242+0000","last_became_active":"2024-02-18T00:39:36.760004+0000","last_became_peered":"2024-02-18T00:39:36.760004+0000","last_unstale":"2024-02-18T00:45:19.055242+0000","last_undegraded":"2024-02-18T00:45:19.055242+0000","last_fullsized":"2024-02-18T00:45:19.055242+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:10.875479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:43:10.875479+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.23","version":"734'38","reported_seq":146,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053698+0000","last_change":"2024-02-18T00:44:43.236041+0000","last_active":"2024-02-18T00:45:19.053698+0000","last_peered":"2024-02-18T00:45:19.053698+0000","last_clean":"2024-02-18T00:45:19.053698+0000","last_became_active":"2024-02-18T00:43:09.796063+0000","last_became_peered":"2024-02-18T00:43:09.796063+0000","last_unstale":"2024-02-18T00:45:19.053698+0000","last_undegraded":"2024-02-18T00:45:19.053698+0000","last_fullsized":"2024-02-18T00:45:19.053698+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"734'38","last_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_deep_scrub":"734'38","last_deep_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.235973+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":696,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820299+0000","last_change":"2024-02-18T00:43:44.566336+0000","last_active":"2024-02-18T00:45:18.820299+0000","last_peered":"2024-02-18T00:45:18.820299+0000","last_clean":"2024-02-18T00:45:18.820299+0000","last_became_active":"2024-02-18T00:33:05.580380+0000","last_became_peered":"2024-02-18T00:33:05.580380+0000","last_unstale":"2024-02-18T00:45:18.820299+0000","last_undegraded":"2024-02-18T00:45:18.820299+0000","last_fullsized":"2024-02-18T00:45:18.820299+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:43:44.566261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.566261+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.22","version":"457'32","reported_seq":587,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819211+0000","last_change":"2024-02-18T00:44:32.659374+0000","last_active":"2024-02-18T00:45:18.819211+0000","last_peered":"2024-02-18T00:45:18.819211+0000","last_clean":"2024-02-18T00:45:18.819211+0000","last_became_active":"2024-02-18T00:43:09.785609+0000","last_became_peered":"2024-02-18T00:43:09.785609+0000","last_unstale":"2024-02-18T00:45:18.819211+0000","last_undegraded":"2024-02-18T00:45:18.819211+0000","last_fullsized":"2024-02-18T00:45:18.819211+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:44:32.659299+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.659299+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":697,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055541+0000","last_change":"2024-02-18T00:43:36.516834+0000","last_active":"2024-02-18T00:45:19.055541+0000","last_peered":"2024-02-18T00:45:19.055541+0000","last_clean":"2024-02-18T00:45:19.055541+0000","last_became_active":"2024-02-18T00:39:36.751486+0000","last_became_peered":"2024-02-18T00:39:36.751486+0000","last_unstale":"2024-02-18T00:45:19.055541+0000","last_undegraded":"2024-02-18T00:45:19.055541+0000","last_fullsized":"2024-02-18T00:45:19.055541+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:36.516704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:36.516704+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.25","version":"457'26","reported_seq":144,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056361+0000","last_change":"2024-02-18T00:43:48.727713+0000","last_active":"2024-02-18T00:45:19.056361+0000","last_peered":"2024-02-18T00:45:19.056361+0000","last_clean":"2024-02-18T00:45:19.056361+0000","last_became_active":"2024-02-18T00:43:09.791838+0000","last_became_peered":"2024-02-18T00:43:09.791838+0000","last_unstale":"2024-02-18T00:45:19.056361+0000","last_undegraded":"2024-02-18T00:45:19.056361+0000","last_fullsized":"2024-02-18T00:45:19.056361+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:48.727636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:48.727636+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":670,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053781+0000","last_change":"2024-02-18T00:42:22.876124+0000","last_active":"2024-02-18T00:45:19.053781+0000","last_peered":"2024-02-18T00:45:19.053781+0000","last_clean":"2024-02-18T00:45:19.053781+0000","last_became_active":"2024-02-18T00:33:05.575396+0000","last_became_peered":"2024-02-18T00:33:05.575396+0000","last_unstale":"2024-02-18T00:45:19.053781+0000","last_undegraded":"2024-02-18T00:45:19.053781+0000","last_fullsized":"2024-02-18T00:45:19.053781+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:22.876055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:22.876055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"457'47","reported_seq":183,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054048+0000","last_change":"2024-02-18T00:44:26.097493+0000","last_active":"2024-02-18T00:45:19.054048+0000","last_peered":"2024-02-18T00:45:19.054048+0000","last_clean":"2024-02-18T00:45:19.054048+0000","last_became_active":"2024-02-18T00:43:09.789985+0000","last_became_peered":"2024-02-18T00:43:09.789985+0000","last_unstale":"2024-02-18T00:45:19.054048+0000","last_undegraded":"2024-02-18T00:45:19.054048+0000","last_fullsized":"2024-02-18T00:45:19.054048+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:44:26.097373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:26.097373+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":684,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054362+0000","last_change":"2024-02-18T00:44:47.426895+0000","last_active":"2024-02-18T00:45:19.054362+0000","last_peered":"2024-02-18T00:45:19.054362+0000","last_clean":"2024-02-18T00:45:19.054362+0000","last_became_active":"2024-02-18T00:33:05.591768+0000","last_became_peered":"2024-02-18T00:33:05.591768+0000","last_unstale":"2024-02-18T00:45:19.054362+0000","last_undegraded":"2024-02-18T00:45:19.054362+0000","last_fullsized":"2024-02-18T00:45:19.054362+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.426814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.426814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"457'23","reported_seq":819,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056257+0000","last_change":"2024-02-18T00:44:24.180563+0000","last_active":"2024-02-18T00:45:19.056257+0000","last_peered":"2024-02-18T00:45:19.056257+0000","last_clean":"2024-02-18T00:45:19.056257+0000","last_became_active":"2024-02-18T00:43:09.788606+0000","last_became_peered":"2024-02-18T00:43:09.788606+0000","last_unstale":"2024-02-18T00:45:19.056257+0000","last_undegraded":"2024-02-18T00:45:19.056257+0000","last_fullsized":"2024-02-18T00:45:19.056257+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:44:24.180458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:24.180458+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":773,"reported_epoch":762,"state":"active+clean","last_fresh":"2024-02-18T00:45:20.755268+0000","last_change":"2024-02-18T00:45:20.755268+0000","last_active":"2024-02-18T00:45:20.755268+0000","last_peered":"2024-02-18T00:45:20.755268+0000","last_clean":"2024-02-18T00:45:20.755268+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:45:20.755268+0000","last_undegraded":"2024-02-18T00:45:20.755268+0000","last_fullsized":"2024-02-18T00:45:20.755268+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:20.755190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_clean_scrub_stamp":"2024-02-18T00:45:20.755190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"457'24","reported_seq":599,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055989+0000","last_change":"2024-02-18T00:43:51.009652+0000","last_active":"2024-02-18T00:45:19.055989+0000","last_peered":"2024-02-18T00:45:19.055989+0000","last_clean":"2024-02-18T00:45:19.055989+0000","last_became_active":"2024-02-18T00:43:09.796496+0000","last_became_peered":"2024-02-18T00:43:09.796496+0000","last_unstale":"2024-02-18T00:45:19.055989+0000","last_undegraded":"2024-02-18T00:45:19.055989+0000","last_fullsized":"2024-02-18T00:45:19.055989+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:43:51.009575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.009575+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053999+0000","last_change":"2024-02-18T00:42:49.664459+0000","last_active":"2024-02-18T00:45:19.053999+0000","last_peered":"2024-02-18T00:45:19.053999+0000","last_clean":"2024-02-18T00:45:19.053999+0000","last_became_active":"2024-02-18T00:39:36.769282+0000","last_became_peered":"2024-02-18T00:39:36.769282+0000","last_unstale":"2024-02-18T00:45:19.053999+0000","last_undegraded":"2024-02-18T00:45:19.053999+0000","last_fullsized":"2024-02-18T00:45:19.053999+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.664380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.664380+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":697,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821771+0000","last_change":"2024-02-18T00:43:11.626087+0000","last_active":"2024-02-18T00:45:18.821771+0000","last_peered":"2024-02-18T00:45:18.821771+0000","last_clean":"2024-02-18T00:45:18.821771+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:45:18.821771+0000","last_undegraded":"2024-02-18T00:45:18.821771+0000","last_fullsized":"2024-02-18T00:45:18.821771+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_clean_scrub_stamp":"2024-02-18T00:43:11.626025+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"736'11","reported_seq":382,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056567+0000","last_change":"2024-02-18T00:44:45.266035+0000","last_active":"2024-02-18T00:45:19.056567+0000","last_peered":"2024-02-18T00:45:19.056567+0000","last_clean":"2024-02-18T00:45:19.056567+0000","last_became_active":"2024-02-18T00:39:36.757493+0000","last_became_peered":"2024-02-18T00:39:36.757493+0000","last_unstale":"2024-02-18T00:45:19.056567+0000","last_undegraded":"2024-02-18T00:45:19.056567+0000","last_fullsized":"2024-02-18T00:45:19.056567+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"736'11","last_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_deep_scrub":"736'11","last_deep_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.265972+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":872,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055659+0000","last_change":"2024-02-18T00:43:43.645878+0000","last_active":"2024-02-18T00:45:19.055659+0000","last_peered":"2024-02-18T00:45:19.055659+0000","last_clean":"2024-02-18T00:45:19.055659+0000","last_became_active":"2024-02-18T00:43:07.792054+0000","last_became_peered":"2024-02-18T00:43:07.792054+0000","last_unstale":"2024-02-18T00:45:19.055659+0000","last_undegraded":"2024-02-18T00:45:19.055659+0000","last_fullsized":"2024-02-18T00:45:19.055659+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:43:43.645800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.645800+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":761,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.822721+0000","last_change":"2024-02-18T00:42:55.614402+0000","last_active":"2024-02-18T00:45:18.822721+0000","last_peered":"2024-02-18T00:45:18.822721+0000","last_clean":"2024-02-18T00:45:18.822721+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:45:18.822721+0000","last_undegraded":"2024-02-18T00:45:18.822721+0000","last_fullsized":"2024-02-18T00:45:18.822721+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:55.614294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:55.614294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":750,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054415+0000","last_change":"2024-02-18T00:44:30.739149+0000","last_active":"2024-02-18T00:45:19.054415+0000","last_peered":"2024-02-18T00:45:19.054415+0000","last_clean":"2024-02-18T00:45:19.054415+0000","last_became_active":"2024-02-18T00:41:45.788049+0000","last_became_peered":"2024-02-18T00:41:45.788049+0000","last_unstale":"2024-02-18T00:45:19.054415+0000","last_undegraded":"2024-02-18T00:45:19.054415+0000","last_fullsized":"2024-02-18T00:45:19.054415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":572,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:30.739087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:30.739087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":689,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056066+0000","last_change":"2024-02-18T00:44:47.967541+0000","last_active":"2024-02-18T00:45:19.056066+0000","last_peered":"2024-02-18T00:45:19.056066+0000","last_clean":"2024-02-18T00:45:19.056066+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:45:19.056066+0000","last_undegraded":"2024-02-18T00:45:19.056066+0000","last_fullsized":"2024-02-18T00:45:19.056066+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.967459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.967459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":795,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823615+0000","last_change":"2024-02-18T00:43:04.471487+0000","last_active":"2024-02-18T00:45:18.823615+0000","last_peered":"2024-02-18T00:45:18.823615+0000","last_clean":"2024-02-18T00:45:18.823615+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:45:18.823615+0000","last_undegraded":"2024-02-18T00:45:18.823615+0000","last_fullsized":"2024-02-18T00:45:18.823615+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:43:04.471412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.471412+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":747,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055692+0000","last_change":"2024-02-18T00:43:43.927240+0000","last_active":"2024-02-18T00:45:19.055692+0000","last_peered":"2024-02-18T00:45:19.055692+0000","last_clean":"2024-02-18T00:45:19.055692+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:45:19.055692+0000","last_undegraded":"2024-02-18T00:45:19.055692+0000","last_fullsized":"2024-02-18T00:45:19.055692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:43.927169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.927169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":737,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823566+0000","last_change":"2024-02-18T00:43:02.696466+0000","last_active":"2024-02-18T00:45:18.823566+0000","last_peered":"2024-02-18T00:45:18.823566+0000","last_clean":"2024-02-18T00:45:18.823566+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:45:18.823566+0000","last_undegraded":"2024-02-18T00:45:18.823566+0000","last_fullsized":"2024-02-18T00:45:18.823566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:02.696399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.696399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":784,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821997+0000","last_change":"2024-02-18T00:43:35.061129+0000","last_active":"2024-02-18T00:45:18.821997+0000","last_peered":"2024-02-18T00:45:18.821997+0000","last_clean":"2024-02-18T00:45:18.821997+0000","last_became_active":"2024-02-18T00:43:35.060523+0000","last_became_peered":"2024-02-18T00:43:35.060523+0000","last_unstale":"2024-02-18T00:45:18.821997+0000","last_undegraded":"2024-02-18T00:45:18.821997+0000","last_fullsized":"2024-02-18T00:45:18.821997+0000","mapping_epoch":668,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":669,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:33.517952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:33.517952+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":690,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056122+0000","last_change":"2024-02-18T00:44:24.230876+0000","last_active":"2024-02-18T00:45:19.056122+0000","last_peered":"2024-02-18T00:45:19.056122+0000","last_clean":"2024-02-18T00:45:19.056122+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:45:19.056122+0000","last_undegraded":"2024-02-18T00:45:19.056122+0000","last_fullsized":"2024-02-18T00:45:19.056122+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:24.230809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:44:24.230809+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"457'42","reported_seq":875,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054447+0000","last_change":"2024-02-18T00:44:46.441595+0000","last_active":"2024-02-18T00:45:19.054447+0000","last_peered":"2024-02-18T00:45:19.054447+0000","last_clean":"2024-02-18T00:45:19.054447+0000","last_became_active":"2024-02-18T00:39:36.751206+0000","last_became_peered":"2024-02-18T00:39:36.751206+0000","last_unstale":"2024-02-18T00:45:19.054447+0000","last_undegraded":"2024-02-18T00:45:19.054447+0000","last_fullsized":"2024-02-18T00:45:19.054447+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-18T00:44:46.441505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.441505+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":741,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821901+0000","last_change":"2024-02-18T00:42:53.601598+0000","last_active":"2024-02-18T00:45:18.821901+0000","last_peered":"2024-02-18T00:45:18.821901+0000","last_clean":"2024-02-18T00:45:18.821901+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:45:18.821901+0000","last_undegraded":"2024-02-18T00:45:18.821901+0000","last_fullsized":"2024-02-18T00:45:18.821901+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:53.601533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:42:53.601533+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":737,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821716+0000","last_change":"2024-02-18T00:43:03.732009+0000","last_active":"2024-02-18T00:45:18.821716+0000","last_peered":"2024-02-18T00:45:18.821716+0000","last_clean":"2024-02-18T00:45:18.821716+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:45:18.821716+0000","last_undegraded":"2024-02-18T00:45:18.821716+0000","last_fullsized":"2024-02-18T00:45:18.821716+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.731917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.731917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":326,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054142+0000","last_change":"2024-02-18T00:44:43.296521+0000","last_active":"2024-02-18T00:45:19.054142+0000","last_peered":"2024-02-18T00:45:19.054142+0000","last_clean":"2024-02-18T00:45:19.054142+0000","last_became_active":"2024-02-18T00:41:53.837106+0000","last_became_peered":"2024-02-18T00:41:53.837106+0000","last_unstale":"2024-02-18T00:45:19.054142+0000","last_undegraded":"2024-02-18T00:45:19.054142+0000","last_fullsized":"2024-02-18T00:45:19.054142+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.296447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.296447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":673,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055871+0000","last_change":"2024-02-18T00:43:37.722070+0000","last_active":"2024-02-18T00:45:19.055871+0000","last_peered":"2024-02-18T00:45:19.055871+0000","last_clean":"2024-02-18T00:45:19.055871+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:45:19.055871+0000","last_undegraded":"2024-02-18T00:45:19.055871+0000","last_fullsized":"2024-02-18T00:45:19.055871+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:37.722006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.722006+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":800,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820184+0000","last_change":"2024-02-18T00:43:12.336661+0000","last_active":"2024-02-18T00:45:18.820184+0000","last_peered":"2024-02-18T00:45:18.820184+0000","last_clean":"2024-02-18T00:45:18.820184+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:45:18.820184+0000","last_undegraded":"2024-02-18T00:45:18.820184+0000","last_fullsized":"2024-02-18T00:45:18.820184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:43:12.336565+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.336565+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055503+0000","last_change":"2024-02-18T00:43:34.003990+0000","last_active":"2024-02-18T00:45:19.055503+0000","last_peered":"2024-02-18T00:45:19.055503+0000","last_clean":"2024-02-18T00:45:19.055503+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:45:19.055503+0000","last_undegraded":"2024-02-18T00:45:19.055503+0000","last_fullsized":"2024-02-18T00:45:19.055503+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.003922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.003922+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":738,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054456+0000","last_change":"2024-02-18T00:43:03.272338+0000","last_active":"2024-02-18T00:45:19.054456+0000","last_peered":"2024-02-18T00:45:19.054456+0000","last_clean":"2024-02-18T00:45:19.054456+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:45:19.054456+0000","last_undegraded":"2024-02-18T00:45:19.054456+0000","last_fullsized":"2024-02-18T00:45:19.054456+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.272278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.272278+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":753,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823088+0000","last_change":"2024-02-18T00:43:35.108868+0000","last_active":"2024-02-18T00:45:18.823088+0000","last_peered":"2024-02-18T00:45:18.823088+0000","last_clean":"2024-02-18T00:45:18.823088+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:45:18.823088+0000","last_undegraded":"2024-02-18T00:45:18.823088+0000","last_fullsized":"2024-02-18T00:45:18.823088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:35.108761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.108761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":151,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821639+0000","last_change":"2024-02-18T00:44:48.550165+0000","last_active":"2024-02-18T00:45:18.821639+0000","last_peered":"2024-02-18T00:45:18.821639+0000","last_clean":"2024-02-18T00:45:18.821639+0000","last_became_active":"2024-02-18T00:42:38.553174+0000","last_became_peered":"2024-02-18T00:42:38.553174+0000","last_unstale":"2024-02-18T00:45:18.821639+0000","last_undegraded":"2024-02-18T00:45:18.821639+0000","last_fullsized":"2024-02-18T00:45:18.821639+0000","mapping_epoch":623,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:48.550082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:48.550082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":787,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819655+0000","last_change":"2024-02-18T00:43:45.250052+0000","last_active":"2024-02-18T00:45:18.819655+0000","last_peered":"2024-02-18T00:45:18.819655+0000","last_clean":"2024-02-18T00:45:18.819655+0000","last_became_active":"2024-02-18T00:43:44.815508+0000","last_became_peered":"2024-02-18T00:43:44.815508+0000","last_unstale":"2024-02-18T00:45:18.819655+0000","last_undegraded":"2024-02-18T00:45:18.819655+0000","last_fullsized":"2024-02-18T00:45:18.819655+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:42:49.351183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.351183+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":767,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055061+0000","last_change":"2024-02-18T00:44:37.109916+0000","last_active":"2024-02-18T00:45:19.055061+0000","last_peered":"2024-02-18T00:45:19.055061+0000","last_clean":"2024-02-18T00:45:19.055061+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:45:19.055061+0000","last_undegraded":"2024-02-18T00:45:19.055061+0000","last_fullsized":"2024-02-18T00:45:19.055061+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.109800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.109800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":969,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821239+0000","last_change":"2024-02-18T00:43:32.077585+0000","last_active":"2024-02-18T00:45:18.821239+0000","last_peered":"2024-02-18T00:45:18.821239+0000","last_clean":"2024-02-18T00:45:18.821239+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:45:18.821239+0000","last_undegraded":"2024-02-18T00:45:18.821239+0000","last_fullsized":"2024-02-18T00:45:18.821239+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:43:32.077472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.077472+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":750,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820135+0000","last_change":"2024-02-18T00:42:52.422173+0000","last_active":"2024-02-18T00:45:18.820135+0000","last_peered":"2024-02-18T00:45:18.820135+0000","last_clean":"2024-02-18T00:45:18.820135+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:45:18.820135+0000","last_undegraded":"2024-02-18T00:45:18.820135+0000","last_fullsized":"2024-02-18T00:45:18.820135+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:52.422086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:52.422086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":687,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055351+0000","last_change":"2024-02-18T00:44:45.071900+0000","last_active":"2024-02-18T00:45:19.055351+0000","last_peered":"2024-02-18T00:45:19.055351+0000","last_clean":"2024-02-18T00:45:19.055351+0000","last_became_active":"2024-02-18T00:33:02.535370+0000","last_became_peered":"2024-02-18T00:33:02.535370+0000","last_unstale":"2024-02-18T00:45:19.055351+0000","last_undegraded":"2024-02-18T00:45:19.055351+0000","last_fullsized":"2024-02-18T00:45:19.055351+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:45.071835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.071835+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":757,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056239+0000","last_change":"2024-02-18T00:43:12.764242+0000","last_active":"2024-02-18T00:45:19.056239+0000","last_peered":"2024-02-18T00:45:19.056239+0000","last_clean":"2024-02-18T00:45:19.056239+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:45:19.056239+0000","last_undegraded":"2024-02-18T00:45:19.056239+0000","last_fullsized":"2024-02-18T00:45:19.056239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:43:12.764172+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.764172+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":774,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054706+0000","last_change":"2024-02-18T00:44:28.072377+0000","last_active":"2024-02-18T00:45:19.054706+0000","last_peered":"2024-02-18T00:45:19.054706+0000","last_clean":"2024-02-18T00:45:19.054706+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:45:19.054706+0000","last_undegraded":"2024-02-18T00:45:19.054706+0000","last_fullsized":"2024-02-18T00:45:19.054706+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:28.072271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.072271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":935,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820042+0000","last_change":"2024-02-18T00:42:56.472380+0000","last_active":"2024-02-18T00:45:18.820042+0000","last_peered":"2024-02-18T00:45:18.820042+0000","last_clean":"2024-02-18T00:45:18.820042+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:45:18.820042+0000","last_undegraded":"2024-02-18T00:45:18.820042+0000","last_fullsized":"2024-02-18T00:45:18.820042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:42:56.472290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.472290+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":745,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056703+0000","last_change":"2024-02-18T00:43:38.758186+0000","last_active":"2024-02-18T00:45:19.056703+0000","last_peered":"2024-02-18T00:45:19.056703+0000","last_clean":"2024-02-18T00:45:19.056703+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:45:19.056703+0000","last_undegraded":"2024-02-18T00:45:19.056703+0000","last_fullsized":"2024-02-18T00:45:19.056703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:38.758071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:43:38.758071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":678,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819366+0000","last_change":"2024-02-18T00:43:36.573041+0000","last_active":"2024-02-18T00:45:18.819366+0000","last_peered":"2024-02-18T00:45:18.819366+0000","last_clean":"2024-02-18T00:45:18.819366+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:45:18.819366+0000","last_undegraded":"2024-02-18T00:45:18.819366+0000","last_fullsized":"2024-02-18T00:45:18.819366+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:36.572964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:43:36.572964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'25","reported_seq":810,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819897+0000","last_change":"2024-02-18T00:44:23.568592+0000","last_active":"2024-02-18T00:45:18.819897+0000","last_peered":"2024-02-18T00:45:18.819897+0000","last_clean":"2024-02-18T00:45:18.819897+0000","last_became_active":"2024-02-18T00:35:42.665164+0000","last_became_peered":"2024-02-18T00:35:42.665164+0000","last_unstale":"2024-02-18T00:45:18.819897+0000","last_undegraded":"2024-02-18T00:45:18.819897+0000","last_fullsized":"2024-02-18T00:45:18.819897+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:44:23.568512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:23.568512+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":309,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820206+0000","last_change":"2024-02-18T00:42:59.673905+0000","last_active":"2024-02-18T00:45:18.820206+0000","last_peered":"2024-02-18T00:45:18.820206+0000","last_clean":"2024-02-18T00:45:18.820206+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:45:18.820206+0000","last_undegraded":"2024-02-18T00:45:18.820206+0000","last_fullsized":"2024-02-18T00:45:18.820206+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:59.673841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.673841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":734,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056356+0000","last_change":"2024-02-18T00:44:25.765000+0000","last_active":"2024-02-18T00:45:19.056356+0000","last_peered":"2024-02-18T00:45:19.056356+0000","last_clean":"2024-02-18T00:45:19.056356+0000","last_became_active":"2024-02-18T00:29:19.097160+0000","last_became_peered":"2024-02-18T00:29:19.097160+0000","last_unstale":"2024-02-18T00:45:19.056356+0000","last_undegraded":"2024-02-18T00:45:19.056356+0000","last_fullsized":"2024-02-18T00:45:19.056356+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:25.764907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.764907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":744,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056784+0000","last_change":"2024-02-18T00:43:42.104431+0000","last_active":"2024-02-18T00:45:19.056784+0000","last_peered":"2024-02-18T00:45:19.056784+0000","last_clean":"2024-02-18T00:45:19.056784+0000","last_became_active":"2024-02-18T00:41:37.801183+0000","last_became_peered":"2024-02-18T00:41:37.801183+0000","last_unstale":"2024-02-18T00:45:19.056784+0000","last_undegraded":"2024-02-18T00:45:19.056784+0000","last_fullsized":"2024-02-18T00:45:19.056784+0000","mapping_epoch":62,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":564,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:43:42.104363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:42.104363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":678,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819883+0000","last_change":"2024-02-18T00:43:00.314164+0000","last_active":"2024-02-18T00:45:18.819883+0000","last_peered":"2024-02-18T00:45:18.819883+0000","last_clean":"2024-02-18T00:45:18.819883+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:45:18.819883+0000","last_undegraded":"2024-02-18T00:45:18.819883+0000","last_fullsized":"2024-02-18T00:45:18.819883+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:00.314091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.314091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":930,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056332+0000","last_change":"2024-02-18T00:43:05.211680+0000","last_active":"2024-02-18T00:45:19.056332+0000","last_peered":"2024-02-18T00:45:19.056332+0000","last_clean":"2024-02-18T00:45:19.056332+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:45:19.056332+0000","last_undegraded":"2024-02-18T00:45:19.056332+0000","last_fullsized":"2024-02-18T00:45:19.056332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_clean_scrub_stamp":"2024-02-18T00:43:05.211604+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":808,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819252+0000","last_change":"2024-02-18T00:44:28.633151+0000","last_active":"2024-02-18T00:45:18.819252+0000","last_peered":"2024-02-18T00:45:18.819252+0000","last_clean":"2024-02-18T00:45:18.819252+0000","last_became_active":"2024-02-18T00:43:07.790268+0000","last_became_peered":"2024-02-18T00:43:07.790268+0000","last_unstale":"2024-02-18T00:45:18.819252+0000","last_undegraded":"2024-02-18T00:45:18.819252+0000","last_fullsized":"2024-02-18T00:45:18.819252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:44:28.633078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.633078+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":766,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055007+0000","last_change":"2024-02-18T00:42:59.867528+0000","last_active":"2024-02-18T00:45:19.055007+0000","last_peered":"2024-02-18T00:45:19.055007+0000","last_clean":"2024-02-18T00:45:19.055007+0000","last_became_active":"2024-02-18T00:39:36.752210+0000","last_became_peered":"2024-02-18T00:39:36.752210+0000","last_unstale":"2024-02-18T00:45:19.055007+0000","last_undegraded":"2024-02-18T00:45:19.055007+0000","last_fullsized":"2024-02-18T00:45:19.055007+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:59.867452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:42:59.867452+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":308,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055590+0000","last_change":"2024-02-18T00:43:40.022882+0000","last_active":"2024-02-18T00:45:19.055590+0000","last_peered":"2024-02-18T00:45:19.055590+0000","last_clean":"2024-02-18T00:45:19.055590+0000","last_became_active":"2024-02-18T00:39:36.767278+0000","last_became_peered":"2024-02-18T00:39:36.767278+0000","last_unstale":"2024-02-18T00:45:19.055590+0000","last_undegraded":"2024-02-18T00:45:19.055590+0000","last_fullsized":"2024-02-18T00:45:19.055590+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:40.022788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:40.022788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":673,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056436+0000","last_change":"2024-02-18T00:43:08.832006+0000","last_active":"2024-02-18T00:45:19.056436+0000","last_peered":"2024-02-18T00:45:19.056436+0000","last_clean":"2024-02-18T00:45:19.056436+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:45:19.056436+0000","last_undegraded":"2024-02-18T00:45:19.056436+0000","last_fullsized":"2024-02-18T00:45:19.056436+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.831934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:08.831934+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.831934+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":757,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820005+0000","last_change":"2024-02-18T00:44:38.710273+0000","last_active":"2024-02-18T00:45:18.820005+0000","last_peered":"2024-02-18T00:45:18.820005+0000","last_clean":"2024-02-18T00:45:18.820005+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:45:18.820005+0000","last_undegraded":"2024-02-18T00:45:18.820005+0000","last_fullsized":"2024-02-18T00:45:18.820005+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.710196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.710196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":897,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056240+0000","last_change":"2024-02-18T00:44:32.283711+0000","last_active":"2024-02-18T00:45:19.056240+0000","last_peered":"2024-02-18T00:45:19.056240+0000","last_clean":"2024-02-18T00:45:19.056240+0000","last_became_active":"2024-02-18T00:43:07.799144+0000","last_became_peered":"2024-02-18T00:43:07.799144+0000","last_unstale":"2024-02-18T00:45:19.056240+0000","last_undegraded":"2024-02-18T00:45:19.056240+0000","last_fullsized":"2024-02-18T00:45:19.056240+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:44:32.283637+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.283637+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":774,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056168+0000","last_change":"2024-02-18T00:43:34.073871+0000","last_active":"2024-02-18T00:45:19.056168+0000","last_peered":"2024-02-18T00:45:19.056168+0000","last_clean":"2024-02-18T00:45:19.056168+0000","last_became_active":"2024-02-18T00:42:44.039326+0000","last_became_peered":"2024-02-18T00:42:44.039326+0000","last_unstale":"2024-02-18T00:45:19.056168+0000","last_undegraded":"2024-02-18T00:45:19.056168+0000","last_fullsized":"2024-02-18T00:45:19.056168+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.073800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.073800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":732,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819644+0000","last_change":"2024-02-18T00:42:50.371671+0000","last_active":"2024-02-18T00:45:18.819644+0000","last_peered":"2024-02-18T00:45:18.819644+0000","last_clean":"2024-02-18T00:45:18.819644+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:45:18.819644+0000","last_undegraded":"2024-02-18T00:45:18.819644+0000","last_fullsized":"2024-02-18T00:45:18.819644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:50.371592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.371592+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":672,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055506+0000","last_change":"2024-02-18T00:44:31.284979+0000","last_active":"2024-02-18T00:45:19.055506+0000","last_peered":"2024-02-18T00:45:19.055506+0000","last_clean":"2024-02-18T00:45:19.055506+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:45:19.055506+0000","last_undegraded":"2024-02-18T00:45:19.055506+0000","last_fullsized":"2024-02-18T00:45:19.055506+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:31.284910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.284910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":735,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054252+0000","last_change":"2024-02-18T00:44:40.278626+0000","last_active":"2024-02-18T00:45:19.054252+0000","last_peered":"2024-02-18T00:45:19.054252+0000","last_clean":"2024-02-18T00:45:19.054252+0000","last_became_active":"2024-02-18T00:29:19.101542+0000","last_became_peered":"2024-02-18T00:29:19.101542+0000","last_unstale":"2024-02-18T00:45:19.054252+0000","last_undegraded":"2024-02-18T00:45:19.054252+0000","last_fullsized":"2024-02-18T00:45:19.054252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:40.278560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:22.964687+0000","last_clean_scrub_stamp":"2024-02-18T00:44:40.278560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":873,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823439+0000","last_change":"2024-02-18T00:43:52.461877+0000","last_active":"2024-02-18T00:45:18.823439+0000","last_peered":"2024-02-18T00:45:18.823439+0000","last_clean":"2024-02-18T00:45:18.823439+0000","last_became_active":"2024-02-18T00:43:07.797505+0000","last_became_peered":"2024-02-18T00:43:07.797505+0000","last_unstale":"2024-02-18T00:45:18.823439+0000","last_undegraded":"2024-02-18T00:45:18.823439+0000","last_fullsized":"2024-02-18T00:45:18.823439+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:43:52.461819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.461819+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":309,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821018+0000","last_change":"2024-02-18T00:43:04.771039+0000","last_active":"2024-02-18T00:45:18.821018+0000","last_peered":"2024-02-18T00:45:18.821018+0000","last_clean":"2024-02-18T00:45:18.821018+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:45:18.821018+0000","last_undegraded":"2024-02-18T00:45:18.821018+0000","last_fullsized":"2024-02-18T00:45:18.821018+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:43:04.770880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.770880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":741,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054787+0000","last_change":"2024-02-18T00:44:38.082951+0000","last_active":"2024-02-18T00:45:19.054787+0000","last_peered":"2024-02-18T00:45:19.054787+0000","last_clean":"2024-02-18T00:45:19.054787+0000","last_became_active":"2024-02-18T00:29:21.242654+0000","last_became_peered":"2024-02-18T00:29:21.242654+0000","last_unstale":"2024-02-18T00:45:19.054787+0000","last_undegraded":"2024-02-18T00:45:19.054787+0000","last_fullsized":"2024-02-18T00:45:19.054787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.082853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.082853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":672,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054412+0000","last_change":"2024-02-18T00:44:37.330430+0000","last_active":"2024-02-18T00:45:19.054412+0000","last_peered":"2024-02-18T00:45:19.054412+0000","last_clean":"2024-02-18T00:45:19.054412+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:45:19.054412+0000","last_undegraded":"2024-02-18T00:45:19.054412+0000","last_fullsized":"2024-02-18T00:45:19.054412+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.330361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.330361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":734,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056441+0000","last_change":"2024-02-18T00:44:49.712183+0000","last_active":"2024-02-18T00:45:19.056441+0000","last_peered":"2024-02-18T00:45:19.056441+0000","last_clean":"2024-02-18T00:45:19.056441+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:45:19.056441+0000","last_undegraded":"2024-02-18T00:45:19.056441+0000","last_fullsized":"2024-02-18T00:45:19.056441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:49.712086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:04.700147+0000","last_clean_scrub_stamp":"2024-02-18T00:44:49.712086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'24","reported_seq":785,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.824102+0000","last_change":"2024-02-18T00:44:25.533581+0000","last_active":"2024-02-18T00:45:18.824102+0000","last_peered":"2024-02-18T00:45:18.824102+0000","last_clean":"2024-02-18T00:45:18.824102+0000","last_became_active":"2024-02-18T00:43:07.799985+0000","last_became_peered":"2024-02-18T00:43:07.799985+0000","last_unstale":"2024-02-18T00:45:18.824102+0000","last_undegraded":"2024-02-18T00:45:18.824102+0000","last_fullsized":"2024-02-18T00:45:18.824102+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:44:25.533453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.533453+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":309,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820234+0000","last_change":"2024-02-18T00:44:37.153891+0000","last_active":"2024-02-18T00:45:18.820234+0000","last_peered":"2024-02-18T00:45:18.820234+0000","last_clean":"2024-02-18T00:45:18.820234+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:45:18.820234+0000","last_undegraded":"2024-02-18T00:45:18.820234+0000","last_fullsized":"2024-02-18T00:45:18.820234+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.153787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.153787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":735,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821858+0000","last_change":"2024-02-18T00:42:53.461135+0000","last_active":"2024-02-18T00:45:18.821858+0000","last_peered":"2024-02-18T00:45:18.821858+0000","last_clean":"2024-02-18T00:45:18.821858+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:45:18.821858+0000","last_undegraded":"2024-02-18T00:45:18.821858+0000","last_fullsized":"2024-02-18T00:45:18.821858+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:53.461030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:42:53.461030+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":736,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054602+0000","last_change":"2024-02-18T00:43:41.934547+0000","last_active":"2024-02-18T00:45:19.054602+0000","last_peered":"2024-02-18T00:45:19.054602+0000","last_clean":"2024-02-18T00:45:19.054602+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:45:19.054602+0000","last_undegraded":"2024-02-18T00:45:19.054602+0000","last_fullsized":"2024-02-18T00:45:19.054602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_clean_scrub_stamp":"2024-02-18T00:43:41.934482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":148,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820119+0000","last_change":"2024-02-18T00:43:08.893333+0000","last_active":"2024-02-18T00:45:18.820119+0000","last_peered":"2024-02-18T00:45:18.820119+0000","last_clean":"2024-02-18T00:45:18.820119+0000","last_became_active":"2024-02-18T00:42:44.690928+0000","last_became_peered":"2024-02-18T00:42:44.690928+0000","last_unstale":"2024-02-18T00:45:18.820119+0000","last_undegraded":"2024-02-18T00:45:18.820119+0000","last_fullsized":"2024-02-18T00:45:18.820119+0000","mapping_epoch":627,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":628,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.893262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.893262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":308,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054946+0000","last_change":"2024-02-18T00:44:33.157482+0000","last_active":"2024-02-18T00:45:19.054946+0000","last_peered":"2024-02-18T00:45:19.054946+0000","last_clean":"2024-02-18T00:45:19.054946+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:45:19.054946+0000","last_undegraded":"2024-02-18T00:45:19.054946+0000","last_fullsized":"2024-02-18T00:45:19.054946+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:33.157367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:44:33.157367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":844,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055222+0000","last_change":"2024-02-18T00:43:51.588473+0000","last_active":"2024-02-18T00:45:19.055222+0000","last_peered":"2024-02-18T00:45:19.055222+0000","last_clean":"2024-02-18T00:45:19.055222+0000","last_became_active":"2024-02-18T00:43:07.792246+0000","last_became_peered":"2024-02-18T00:43:07.792246+0000","last_unstale":"2024-02-18T00:45:19.055222+0000","last_undegraded":"2024-02-18T00:45:19.055222+0000","last_fullsized":"2024-02-18T00:45:19.055222+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:51.588408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.588408+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":308,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053915+0000","last_change":"2024-02-18T00:43:42.067873+0000","last_active":"2024-02-18T00:45:19.053915+0000","last_peered":"2024-02-18T00:45:19.053915+0000","last_clean":"2024-02-18T00:45:19.053915+0000","last_became_active":"2024-02-18T00:39:36.759224+0000","last_became_peered":"2024-02-18T00:39:36.759224+0000","last_unstale":"2024-02-18T00:45:19.053915+0000","last_undegraded":"2024-02-18T00:45:19.053915+0000","last_fullsized":"2024-02-18T00:45:19.053915+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_clean_scrub_stamp":"2024-02-18T00:43:42.067810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":774,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.057198+0000","last_change":"2024-02-18T00:43:39.793439+0000","last_active":"2024-02-18T00:45:19.057198+0000","last_peered":"2024-02-18T00:45:19.057198+0000","last_clean":"2024-02-18T00:45:19.057198+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:45:19.057198+0000","last_undegraded":"2024-02-18T00:45:19.057198+0000","last_fullsized":"2024-02-18T00:45:19.057198+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.793345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.793345+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":731,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053653+0000","last_change":"2024-02-18T00:44:44.267245+0000","last_active":"2024-02-18T00:45:19.053653+0000","last_peered":"2024-02-18T00:45:19.053653+0000","last_clean":"2024-02-18T00:45:19.053653+0000","last_became_active":"2024-02-18T00:29:16.075589+0000","last_became_peered":"2024-02-18T00:29:16.075589+0000","last_unstale":"2024-02-18T00:45:19.053653+0000","last_undegraded":"2024-02-18T00:45:19.053653+0000","last_fullsized":"2024-02-18T00:45:19.053653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.267177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.267177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":750,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056484+0000","last_change":"2024-02-18T00:44:32.815550+0000","last_active":"2024-02-18T00:45:19.056484+0000","last_peered":"2024-02-18T00:45:19.056484+0000","last_clean":"2024-02-18T00:45:19.056484+0000","last_became_active":"2024-02-18T00:41:57.791956+0000","last_became_peered":"2024-02-18T00:41:57.791956+0000","last_unstale":"2024-02-18T00:45:19.056484+0000","last_undegraded":"2024-02-18T00:45:19.056484+0000","last_fullsized":"2024-02-18T00:45:19.056484+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":584,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:32.815428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.815428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":677,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821529+0000","last_change":"2024-02-18T00:43:05.469702+0000","last_active":"2024-02-18T00:45:18.821529+0000","last_peered":"2024-02-18T00:45:18.821529+0000","last_clean":"2024-02-18T00:45:18.821529+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:45:18.821529+0000","last_undegraded":"2024-02-18T00:45:18.821529+0000","last_fullsized":"2024-02-18T00:45:18.821529+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_clean_scrub_stamp":"2024-02-18T00:43:05.469634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":755,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.822834+0000","last_change":"2024-02-18T00:42:58.721654+0000","last_active":"2024-02-18T00:45:18.822834+0000","last_peered":"2024-02-18T00:45:18.822834+0000","last_clean":"2024-02-18T00:45:18.822834+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:45:18.822834+0000","last_undegraded":"2024-02-18T00:45:18.822834+0000","last_fullsized":"2024-02-18T00:45:18.822834+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:58.721587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:58.721587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"680'27","reported_seq":809,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056383+0000","last_change":"2024-02-18T00:43:45.989486+0000","last_active":"2024-02-18T00:45:19.056383+0000","last_peered":"2024-02-18T00:45:19.056383+0000","last_clean":"2024-02-18T00:45:19.056383+0000","last_became_active":"2024-02-18T00:43:07.801362+0000","last_became_peered":"2024-02-18T00:43:07.801362+0000","last_unstale":"2024-02-18T00:45:19.056383+0000","last_undegraded":"2024-02-18T00:45:19.056383+0000","last_fullsized":"2024-02-18T00:45:19.056383+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"680'27","last_scrub_stamp":"2024-02-18T00:43:45.989427+0000","last_deep_scrub":"680'27","last_deep_scrub_stamp":"2024-02-18T00:43:45.989427+0000","last_clean_scrub_stamp":"2024-02-18T00:43:45.989427+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":750,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820573+0000","last_change":"2024-02-18T00:42:49.673980+0000","last_active":"2024-02-18T00:45:18.820573+0000","last_peered":"2024-02-18T00:45:18.820573+0000","last_clean":"2024-02-18T00:45:18.820573+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:45:18.820573+0000","last_undegraded":"2024-02-18T00:45:18.820573+0000","last_fullsized":"2024-02-18T00:45:18.820573+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:49.673915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:42:49.673915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1706,"reported_epoch":762,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.827979+0000","last_change":"2024-02-18T00:43:04.252649+0000","last_active":"2024-02-18T00:45:19.827979+0000","last_peered":"2024-02-18T00:45:19.827979+0000","last_clean":"2024-02-18T00:45:19.827979+0000","last_became_active":"2024-02-18T00:29:16.075833+0000","last_became_peered":"2024-02-18T00:29:16.075833+0000","last_unstale":"2024-02-18T00:45:19.827979+0000","last_undegraded":"2024-02-18T00:45:19.827979+0000","last_fullsized":"2024-02-18T00:45:19.827979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:43:04.252540+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.252540+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":956,"num_read_kb":951,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":307,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056030+0000","last_change":"2024-02-18T00:43:12.839041+0000","last_active":"2024-02-18T00:45:19.056030+0000","last_peered":"2024-02-18T00:45:19.056030+0000","last_clean":"2024-02-18T00:45:19.056030+0000","last_became_active":"2024-02-18T00:39:36.764502+0000","last_became_peered":"2024-02-18T00:39:36.764502+0000","last_unstale":"2024-02-18T00:45:19.056030+0000","last_undegraded":"2024-02-18T00:45:19.056030+0000","last_fullsized":"2024-02-18T00:45:19.056030+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:43:12.838938+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:43:12.838938+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.838938+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":696,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819460+0000","last_change":"2024-02-18T00:43:52.430089+0000","last_active":"2024-02-18T00:45:18.819460+0000","last_peered":"2024-02-18T00:45:18.819460+0000","last_clean":"2024-02-18T00:45:18.819460+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:45:18.819460+0000","last_undegraded":"2024-02-18T00:45:18.819460+0000","last_fullsized":"2024-02-18T00:45:18.819460+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.430010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":732,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.822127+0000","last_change":"2024-02-18T00:42:58.421119+0000","last_active":"2024-02-18T00:45:18.822127+0000","last_peered":"2024-02-18T00:45:18.822127+0000","last_clean":"2024-02-18T00:45:18.822127+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:45:18.822127+0000","last_undegraded":"2024-02-18T00:45:18.822127+0000","last_fullsized":"2024-02-18T00:45:18.822127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:58.421052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:58.421052+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"682'49","reported_seq":702,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056789+0000","last_change":"2024-02-18T00:44:35.869910+0000","last_active":"2024-02-18T00:45:19.056789+0000","last_peered":"2024-02-18T00:45:19.056789+0000","last_clean":"2024-02-18T00:45:19.056789+0000","last_became_active":"2024-02-18T00:43:44.816299+0000","last_became_peered":"2024-02-18T00:43:44.816299+0000","last_unstale":"2024-02-18T00:45:19.056789+0000","last_undegraded":"2024-02-18T00:45:19.056789+0000","last_fullsized":"2024-02-18T00:45:19.056789+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":0,"last_scrub":"682'49","last_scrub_stamp":"2024-02-18T00:44:35.869834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.869834+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":248,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":747,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054563+0000","last_change":"2024-02-18T00:43:51.722733+0000","last_active":"2024-02-18T00:45:19.054563+0000","last_peered":"2024-02-18T00:45:19.054563+0000","last_clean":"2024-02-18T00:45:19.054563+0000","last_became_active":"2024-02-18T00:33:04.569172+0000","last_became_peered":"2024-02-18T00:33:04.569172+0000","last_unstale":"2024-02-18T00:45:19.054563+0000","last_undegraded":"2024-02-18T00:45:19.054563+0000","last_fullsized":"2024-02-18T00:45:19.054563+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.722660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.722660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":757,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055298+0000","last_change":"2024-02-18T00:42:19.938250+0000","last_active":"2024-02-18T00:45:19.055298+0000","last_peered":"2024-02-18T00:45:19.055298+0000","last_clean":"2024-02-18T00:45:19.055298+0000","last_became_active":"2024-02-18T00:42:02.125704+0000","last_became_peered":"2024-02-18T00:42:02.125704+0000","last_unstale":"2024-02-18T00:45:19.055298+0000","last_undegraded":"2024-02-18T00:45:19.055298+0000","last_fullsized":"2024-02-18T00:45:19.055298+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:19.938182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:42:19.938182+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":677,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.822931+0000","last_change":"2024-02-18T00:43:51.459881+0000","last_active":"2024-02-18T00:45:18.822931+0000","last_peered":"2024-02-18T00:45:18.822931+0000","last_clean":"2024-02-18T00:45:18.822931+0000","last_became_active":"2024-02-18T00:33:04.718627+0000","last_became_peered":"2024-02-18T00:33:04.718627+0000","last_unstale":"2024-02-18T00:45:18.822931+0000","last_undegraded":"2024-02-18T00:45:18.822931+0000","last_fullsized":"2024-02-18T00:45:18.822931+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.459750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.459750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"64'8","reported_seq":933,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823753+0000","last_change":"2024-02-18T00:43:35.528634+0000","last_active":"2024-02-18T00:45:18.823753+0000","last_peered":"2024-02-18T00:45:18.823753+0000","last_clean":"2024-02-18T00:45:18.823753+0000","last_became_active":"2024-02-18T00:29:19.099177+0000","last_became_peered":"2024-02-18T00:29:19.099177+0000","last_unstale":"2024-02-18T00:45:18.823753+0000","last_undegraded":"2024-02-18T00:45:18.823753+0000","last_fullsized":"2024-02-18T00:45:18.823753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:43:35.528567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.528567+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":765,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823797+0000","last_change":"2024-02-18T00:43:51.043051+0000","last_active":"2024-02-18T00:45:18.823797+0000","last_peered":"2024-02-18T00:45:18.823797+0000","last_clean":"2024-02-18T00:45:18.823797+0000","last_became_active":"2024-02-18T00:43:07.795433+0000","last_became_peered":"2024-02-18T00:43:07.795433+0000","last_unstale":"2024-02-18T00:45:18.823797+0000","last_undegraded":"2024-02-18T00:45:18.823797+0000","last_fullsized":"2024-02-18T00:45:18.823797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:43:51.042989+0000","last_deep_scrub":"457'32","last_deep_scrub_stamp":"2024-02-18T00:43:51.042989+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.042989+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":779,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054321+0000","last_change":"2024-02-18T00:44:43.113419+0000","last_active":"2024-02-18T00:45:19.054321+0000","last_peered":"2024-02-18T00:45:19.054321+0000","last_clean":"2024-02-18T00:45:19.054321+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:45:19.054321+0000","last_undegraded":"2024-02-18T00:45:19.054321+0000","last_fullsized":"2024-02-18T00:45:19.054321+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.113292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":730,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821177+0000","last_change":"2024-02-18T00:44:44.469481+0000","last_active":"2024-02-18T00:45:18.821177+0000","last_peered":"2024-02-18T00:45:18.821177+0000","last_clean":"2024-02-18T00:45:18.821177+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:45:18.821177+0000","last_undegraded":"2024-02-18T00:45:18.821177+0000","last_fullsized":"2024-02-18T00:45:18.821177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.469413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.469413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":678,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819860+0000","last_change":"2024-02-18T00:43:09.318894+0000","last_active":"2024-02-18T00:45:18.819860+0000","last_peered":"2024-02-18T00:45:18.819860+0000","last_clean":"2024-02-18T00:45:18.819860+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:45:18.819860+0000","last_undegraded":"2024-02-18T00:45:18.819860+0000","last_fullsized":"2024-02-18T00:45:18.819860+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.318803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.318803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":737,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820372+0000","last_change":"2024-02-18T00:43:44.168202+0000","last_active":"2024-02-18T00:45:18.820372+0000","last_peered":"2024-02-18T00:45:18.820372+0000","last_clean":"2024-02-18T00:45:18.820372+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:45:18.820372+0000","last_undegraded":"2024-02-18T00:45:18.820372+0000","last_fullsized":"2024-02-18T00:45:18.820372+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.168096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.168096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":813,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819300+0000","last_change":"2024-02-18T00:44:27.619378+0000","last_active":"2024-02-18T00:45:18.819300+0000","last_peered":"2024-02-18T00:45:18.819300+0000","last_clean":"2024-02-18T00:45:18.819300+0000","last_became_active":"2024-02-18T00:43:07.795322+0000","last_became_peered":"2024-02-18T00:43:07.795322+0000","last_unstale":"2024-02-18T00:45:18.819300+0000","last_undegraded":"2024-02-18T00:45:18.819300+0000","last_fullsized":"2024-02-18T00:45:18.819300+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:44:27.619298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:27.619298+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":791,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820084+0000","last_change":"2024-02-18T00:44:47.741854+0000","last_active":"2024-02-18T00:45:18.820084+0000","last_peered":"2024-02-18T00:45:18.820084+0000","last_clean":"2024-02-18T00:45:18.820084+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:45:18.820084+0000","last_undegraded":"2024-02-18T00:45:18.820084+0000","last_fullsized":"2024-02-18T00:45:18.820084+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:44:47.741787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.741787+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":762,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054139+0000","last_change":"2024-02-18T00:43:01.098679+0000","last_active":"2024-02-18T00:45:19.054139+0000","last_peered":"2024-02-18T00:45:19.054139+0000","last_clean":"2024-02-18T00:45:19.054139+0000","last_became_active":"2024-02-18T00:41:42.776144+0000","last_became_peered":"2024-02-18T00:41:42.776144+0000","last_unstale":"2024-02-18T00:45:19.054139+0000","last_undegraded":"2024-02-18T00:45:19.054139+0000","last_fullsized":"2024-02-18T00:45:19.054139+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:01.098569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.098569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":678,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819416+0000","last_change":"2024-02-18T00:43:50.444348+0000","last_active":"2024-02-18T00:45:18.819416+0000","last_peered":"2024-02-18T00:45:18.819416+0000","last_clean":"2024-02-18T00:45:18.819416+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:45:18.819416+0000","last_undegraded":"2024-02-18T00:45:18.819416+0000","last_fullsized":"2024-02-18T00:45:18.819416+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.444279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"457'27","reported_seq":835,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.824160+0000","last_change":"2024-02-18T00:43:02.517912+0000","last_active":"2024-02-18T00:45:18.824160+0000","last_peered":"2024-02-18T00:45:18.824160+0000","last_clean":"2024-02-18T00:45:18.824160+0000","last_became_active":"2024-02-18T00:39:36.771394+0000","last_became_peered":"2024-02-18T00:39:36.771394+0000","last_unstale":"2024-02-18T00:45:18.824160+0000","last_undegraded":"2024-02-18T00:45:18.824160+0000","last_fullsized":"2024-02-18T00:45:18.824160+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:43:02.517835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.517835+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":741,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820302+0000","last_change":"2024-02-18T00:43:34.111205+0000","last_active":"2024-02-18T00:45:18.820302+0000","last_peered":"2024-02-18T00:45:18.820302+0000","last_clean":"2024-02-18T00:45:18.820302+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:45:18.820302+0000","last_undegraded":"2024-02-18T00:45:18.820302+0000","last_fullsized":"2024-02-18T00:45:18.820302+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.111128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.111128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":746,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055986+0000","last_change":"2024-02-18T00:43:09.212023+0000","last_active":"2024-02-18T00:45:19.055986+0000","last_peered":"2024-02-18T00:45:19.055986+0000","last_clean":"2024-02-18T00:45:19.055986+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:45:19.055986+0000","last_undegraded":"2024-02-18T00:45:19.055986+0000","last_fullsized":"2024-02-18T00:45:19.055986+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.211956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.211956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":755,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821057+0000","last_change":"2024-02-18T00:44:42.127223+0000","last_active":"2024-02-18T00:45:18.821057+0000","last_peered":"2024-02-18T00:45:18.821057+0000","last_clean":"2024-02-18T00:45:18.821057+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:45:18.821057+0000","last_undegraded":"2024-02-18T00:45:18.821057+0000","last_fullsized":"2024-02-18T00:45:18.821057+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:42.127148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:44:42.127148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":677,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821445+0000","last_change":"2024-02-18T00:44:37.525779+0000","last_active":"2024-02-18T00:45:18.821445+0000","last_peered":"2024-02-18T00:45:18.821445+0000","last_clean":"2024-02-18T00:45:18.821445+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:45:18.821445+0000","last_undegraded":"2024-02-18T00:45:18.821445+0000","last_fullsized":"2024-02-18T00:45:18.821445+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.525709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.525709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":828,"reported_epoch":762,"state":"active+clean","last_fresh":"2024-02-18T00:45:20.335925+0000","last_change":"2024-02-18T00:45:20.335925+0000","last_active":"2024-02-18T00:45:20.335925+0000","last_peered":"2024-02-18T00:45:20.335925+0000","last_clean":"2024-02-18T00:45:20.335925+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:45:20.335925+0000","last_undegraded":"2024-02-18T00:45:20.335925+0000","last_fullsized":"2024-02-18T00:45:20.335925+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'18","last_scrub_stamp":"2024-02-18T00:45:20.335853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:20.335853+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":744,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053657+0000","last_change":"2024-02-18T00:44:43.359587+0000","last_active":"2024-02-18T00:45:19.053657+0000","last_peered":"2024-02-18T00:45:19.053657+0000","last_clean":"2024-02-18T00:45:19.053657+0000","last_became_active":"2024-02-18T00:29:14.718291+0000","last_became_peered":"2024-02-18T00:29:14.718291+0000","last_unstale":"2024-02-18T00:45:19.053657+0000","last_undegraded":"2024-02-18T00:45:19.053657+0000","last_fullsized":"2024-02-18T00:45:19.053657+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.359510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.359510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":734,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055072+0000","last_change":"2024-02-18T00:44:45.666045+0000","last_active":"2024-02-18T00:45:19.055072+0000","last_peered":"2024-02-18T00:45:19.055072+0000","last_clean":"2024-02-18T00:45:19.055072+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:45:19.055072+0000","last_undegraded":"2024-02-18T00:45:19.055072+0000","last_fullsized":"2024-02-18T00:45:19.055072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:45.665981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.665981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056114+0000","last_change":"2024-02-18T00:43:08.921418+0000","last_active":"2024-02-18T00:45:19.056114+0000","last_peered":"2024-02-18T00:45:19.056114+0000","last_clean":"2024-02-18T00:45:19.056114+0000","last_became_active":"2024-02-18T00:39:36.759264+0000","last_became_peered":"2024-02-18T00:39:36.759264+0000","last_unstale":"2024-02-18T00:45:19.056114+0000","last_undegraded":"2024-02-18T00:45:19.056114+0000","last_fullsized":"2024-02-18T00:45:19.056114+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.921352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.921352+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":690,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820670+0000","last_change":"2024-02-18T00:44:47.201367+0000","last_active":"2024-02-18T00:45:18.820670+0000","last_peered":"2024-02-18T00:45:18.820670+0000","last_clean":"2024-02-18T00:45:18.820670+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:45:18.820670+0000","last_undegraded":"2024-02-18T00:45:18.820670+0000","last_fullsized":"2024-02-18T00:45:18.820670+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.201293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.201293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":840,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056985+0000","last_change":"2024-02-18T00:43:41.139904+0000","last_active":"2024-02-18T00:45:19.056985+0000","last_peered":"2024-02-18T00:45:19.056985+0000","last_clean":"2024-02-18T00:45:19.056985+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:45:19.056985+0000","last_undegraded":"2024-02-18T00:45:19.056985+0000","last_fullsized":"2024-02-18T00:45:19.056985+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:43:41.139829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:41.139829+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":765,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054492+0000","last_change":"2024-02-18T00:44:46.634819+0000","last_active":"2024-02-18T00:45:19.054492+0000","last_peered":"2024-02-18T00:45:19.054492+0000","last_clean":"2024-02-18T00:45:19.054492+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:45:19.054492+0000","last_undegraded":"2024-02-18T00:45:19.054492+0000","last_fullsized":"2024-02-18T00:45:19.054492+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:46.634734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.634734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":936,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056618+0000","last_change":"2024-02-18T00:44:47.651515+0000","last_active":"2024-02-18T00:45:19.056618+0000","last_peered":"2024-02-18T00:45:19.056618+0000","last_clean":"2024-02-18T00:45:19.056618+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:45:19.056618+0000","last_undegraded":"2024-02-18T00:45:19.056618+0000","last_fullsized":"2024-02-18T00:45:19.056618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:44:47.651424+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:44:47.651424+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.651424+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":697,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053468+0000","last_change":"2024-02-18T00:44:42.310455+0000","last_active":"2024-02-18T00:45:19.053468+0000","last_peered":"2024-02-18T00:45:19.053468+0000","last_clean":"2024-02-18T00:45:19.053468+0000","last_became_active":"2024-02-18T00:39:36.751162+0000","last_became_peered":"2024-02-18T00:39:36.751162+0000","last_unstale":"2024-02-18T00:45:19.053468+0000","last_undegraded":"2024-02-18T00:45:19.053468+0000","last_fullsized":"2024-02-18T00:45:19.053468+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:42.310377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:42.310377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":813,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056527+0000","last_change":"2024-02-18T00:43:03.706116+0000","last_active":"2024-02-18T00:45:19.056527+0000","last_peered":"2024-02-18T00:45:19.056527+0000","last_clean":"2024-02-18T00:45:19.056527+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:45:19.056527+0000","last_undegraded":"2024-02-18T00:45:19.056527+0000","last_fullsized":"2024-02-18T00:45:19.056527+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:43:03.706028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.706028+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":736,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823520+0000","last_change":"2024-02-18T00:42:57.465908+0000","last_active":"2024-02-18T00:45:18.823520+0000","last_peered":"2024-02-18T00:45:18.823520+0000","last_clean":"2024-02-18T00:45:18.823520+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:45:18.823520+0000","last_undegraded":"2024-02-18T00:45:18.823520+0000","last_fullsized":"2024-02-18T00:45:18.823520+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:57.465836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:42:57.465836+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":956,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.057161+0000","last_change":"2024-02-18T00:43:00.081109+0000","last_active":"2024-02-18T00:45:19.057161+0000","last_peered":"2024-02-18T00:45:19.057161+0000","last_clean":"2024-02-18T00:45:19.057161+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:45:19.057161+0000","last_undegraded":"2024-02-18T00:45:19.057161+0000","last_fullsized":"2024-02-18T00:45:19.057161+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:43:00.080987+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.080987+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":678,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.056522+0000","last_change":"2024-02-18T00:44:48.698039+0000","last_active":"2024-02-18T00:45:19.056522+0000","last_peered":"2024-02-18T00:45:19.056522+0000","last_clean":"2024-02-18T00:45:19.056522+0000","last_became_active":"2024-02-18T00:33:02.532365+0000","last_became_peered":"2024-02-18T00:33:02.532365+0000","last_unstale":"2024-02-18T00:45:19.056522+0000","last_undegraded":"2024-02-18T00:45:19.056522+0000","last_fullsized":"2024-02-18T00:45:19.056522+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:48.697958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:44:48.697958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":767,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055139+0000","last_change":"2024-02-18T00:42:50.108975+0000","last_active":"2024-02-18T00:45:19.055139+0000","last_peered":"2024-02-18T00:45:19.055139+0000","last_clean":"2024-02-18T00:45:19.055139+0000","last_became_active":"2024-02-18T00:29:17.091935+0000","last_became_peered":"2024-02-18T00:29:17.091935+0000","last_unstale":"2024-02-18T00:45:19.055139+0000","last_undegraded":"2024-02-18T00:45:19.055139+0000","last_fullsized":"2024-02-18T00:45:19.055139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:42:50.108893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:42:50.108893+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":744,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055941+0000","last_change":"2024-02-18T00:44:36.372427+0000","last_active":"2024-02-18T00:45:19.055941+0000","last_peered":"2024-02-18T00:45:19.055941+0000","last_clean":"2024-02-18T00:45:19.055941+0000","last_became_active":"2024-02-18T00:29:14.713970+0000","last_became_peered":"2024-02-18T00:29:14.713970+0000","last_unstale":"2024-02-18T00:45:19.055941+0000","last_undegraded":"2024-02-18T00:45:19.055941+0000","last_fullsized":"2024-02-18T00:45:19.055941+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:36.372315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:36.372315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":728,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054059+0000","last_change":"2024-02-18T00:42:56.033487+0000","last_active":"2024-02-18T00:45:19.054059+0000","last_peered":"2024-02-18T00:45:19.054059+0000","last_clean":"2024-02-18T00:45:19.054059+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:45:19.054059+0000","last_undegraded":"2024-02-18T00:45:19.054059+0000","last_fullsized":"2024-02-18T00:45:19.054059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:56.033419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.033419+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":735,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821013+0000","last_change":"2024-02-18T00:44:38.484355+0000","last_active":"2024-02-18T00:45:18.821013+0000","last_peered":"2024-02-18T00:45:18.821013+0000","last_clean":"2024-02-18T00:45:18.821013+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:45:18.821013+0000","last_undegraded":"2024-02-18T00:45:18.821013+0000","last_fullsized":"2024-02-18T00:45:18.821013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.484287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.484287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":679,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.824058+0000","last_change":"2024-02-18T00:44:49.583138+0000","last_active":"2024-02-18T00:45:18.824058+0000","last_peered":"2024-02-18T00:45:18.824058+0000","last_clean":"2024-02-18T00:45:18.824058+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:45:18.824058+0000","last_undegraded":"2024-02-18T00:45:18.824058+0000","last_fullsized":"2024-02-18T00:45:18.824058+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:49.583035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_clean_scrub_stamp":"2024-02-18T00:44:49.583035+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":772,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819613+0000","last_change":"2024-02-18T00:44:41.703555+0000","last_active":"2024-02-18T00:45:18.819613+0000","last_peered":"2024-02-18T00:45:18.819613+0000","last_clean":"2024-02-18T00:45:18.819613+0000","last_became_active":"2024-02-18T00:29:17.085418+0000","last_became_peered":"2024-02-18T00:29:17.085418+0000","last_unstale":"2024-02-18T00:45:18.819613+0000","last_undegraded":"2024-02-18T00:45:18.819613+0000","last_fullsized":"2024-02-18T00:45:18.819613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:44:41.703456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:41.703456+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":742,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055618+0000","last_change":"2024-02-18T00:43:09.187333+0000","last_active":"2024-02-18T00:45:19.055618+0000","last_peered":"2024-02-18T00:45:19.055618+0000","last_clean":"2024-02-18T00:45:19.055618+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:45:19.055618+0000","last_undegraded":"2024-02-18T00:45:19.055618+0000","last_fullsized":"2024-02-18T00:45:19.055618+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.187270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.187270+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":743,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055222+0000","last_change":"2024-02-18T00:43:39.958283+0000","last_active":"2024-02-18T00:45:19.055222+0000","last_peered":"2024-02-18T00:45:19.055222+0000","last_clean":"2024-02-18T00:45:19.055222+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:45:19.055222+0000","last_undegraded":"2024-02-18T00:45:19.055222+0000","last_fullsized":"2024-02-18T00:45:19.055222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.958201+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":726,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054526+0000","last_change":"2024-02-18T00:43:40.107832+0000","last_active":"2024-02-18T00:45:19.054526+0000","last_peered":"2024-02-18T00:45:19.054526+0000","last_clean":"2024-02-18T00:45:19.054526+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:45:19.054526+0000","last_undegraded":"2024-02-18T00:45:19.054526+0000","last_fullsized":"2024-02-18T00:45:19.054526+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:40.107757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:40.107757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":674,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055361+0000","last_change":"2024-02-18T00:42:56.820557+0000","last_active":"2024-02-18T00:45:19.055361+0000","last_peered":"2024-02-18T00:45:19.055361+0000","last_clean":"2024-02-18T00:45:19.055361+0000","last_became_active":"2024-02-18T00:33:02.499487+0000","last_became_peered":"2024-02-18T00:33:02.499487+0000","last_unstale":"2024-02-18T00:45:19.055361+0000","last_undegraded":"2024-02-18T00:45:19.055361+0000","last_fullsized":"2024-02-18T00:45:19.055361+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:56.820451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:56.820451+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":742,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054307+0000","last_change":"2024-02-18T00:43:02.133961+0000","last_active":"2024-02-18T00:45:19.054307+0000","last_peered":"2024-02-18T00:45:19.054307+0000","last_clean":"2024-02-18T00:45:19.054307+0000","last_became_active":"2024-02-18T00:29:17.088983+0000","last_became_peered":"2024-02-18T00:29:17.088983+0000","last_unstale":"2024-02-18T00:45:19.054307+0000","last_undegraded":"2024-02-18T00:45:19.054307+0000","last_fullsized":"2024-02-18T00:45:19.054307+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-18T00:43:02.133854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:02.133854+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":738,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055145+0000","last_change":"2024-02-18T00:43:07.782401+0000","last_active":"2024-02-18T00:45:19.055145+0000","last_peered":"2024-02-18T00:45:19.055145+0000","last_clean":"2024-02-18T00:45:19.055145+0000","last_became_active":"2024-02-18T00:29:14.710856+0000","last_became_peered":"2024-02-18T00:29:14.710856+0000","last_unstale":"2024-02-18T00:45:19.055145+0000","last_undegraded":"2024-02-18T00:45:19.055145+0000","last_fullsized":"2024-02-18T00:45:19.055145+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_clean_scrub_stamp":"2024-02-18T00:43:07.782283+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":753,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053583+0000","last_change":"2024-02-18T00:43:44.575490+0000","last_active":"2024-02-18T00:45:19.053583+0000","last_peered":"2024-02-18T00:45:19.053583+0000","last_clean":"2024-02-18T00:45:19.053583+0000","last_became_active":"2024-02-18T00:39:36.750866+0000","last_became_peered":"2024-02-18T00:39:36.750866+0000","last_unstale":"2024-02-18T00:45:19.053583+0000","last_undegraded":"2024-02-18T00:45:19.053583+0000","last_fullsized":"2024-02-18T00:45:19.053583+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.575417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.575417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":308,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054653+0000","last_change":"2024-02-18T00:44:28.186498+0000","last_active":"2024-02-18T00:45:19.054653+0000","last_peered":"2024-02-18T00:45:19.054653+0000","last_clean":"2024-02-18T00:45:19.054653+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:45:19.054653+0000","last_undegraded":"2024-02-18T00:45:19.054653+0000","last_fullsized":"2024-02-18T00:45:19.054653+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:28.186417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.186417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":691,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.057108+0000","last_change":"2024-02-18T00:43:32.818797+0000","last_active":"2024-02-18T00:45:19.057108+0000","last_peered":"2024-02-18T00:45:19.057108+0000","last_clean":"2024-02-18T00:45:19.057108+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:45:19.057108+0000","last_undegraded":"2024-02-18T00:45:19.057108+0000","last_fullsized":"2024-02-18T00:45:19.057108+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.818709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.818709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":822,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055138+0000","last_change":"2024-02-18T00:44:28.251258+0000","last_active":"2024-02-18T00:45:19.055138+0000","last_peered":"2024-02-18T00:45:19.055138+0000","last_clean":"2024-02-18T00:45:19.055138+0000","last_became_active":"2024-02-18T00:39:36.761140+0000","last_became_peered":"2024-02-18T00:39:36.761140+0000","last_unstale":"2024-02-18T00:45:19.055138+0000","last_undegraded":"2024-02-18T00:45:19.055138+0000","last_fullsized":"2024-02-18T00:45:19.055138+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:44:28.251169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.251169+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":739,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054928+0000","last_change":"2024-02-18T00:43:03.852309+0000","last_active":"2024-02-18T00:45:19.054928+0000","last_peered":"2024-02-18T00:45:19.054928+0000","last_clean":"2024-02-18T00:45:19.054928+0000","last_became_active":"2024-02-18T00:29:14.719790+0000","last_became_peered":"2024-02-18T00:29:14.719790+0000","last_unstale":"2024-02-18T00:45:19.054928+0000","last_undegraded":"2024-02-18T00:45:19.054928+0000","last_fullsized":"2024-02-18T00:45:19.054928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.852193+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054225+0000","last_change":"2024-02-18T00:44:23.214099+0000","last_active":"2024-02-18T00:45:19.054225+0000","last_peered":"2024-02-18T00:45:19.054225+0000","last_clean":"2024-02-18T00:45:19.054225+0000","last_became_active":"2024-02-18T00:39:36.758138+0000","last_became_peered":"2024-02-18T00:39:36.758138+0000","last_unstale":"2024-02-18T00:45:19.054225+0000","last_undegraded":"2024-02-18T00:45:19.054225+0000","last_fullsized":"2024-02-18T00:45:19.054225+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:23.214010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:23.214010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":726,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054281+0000","last_change":"2024-02-18T00:43:32.135279+0000","last_active":"2024-02-18T00:45:19.054281+0000","last_peered":"2024-02-18T00:45:19.054281+0000","last_clean":"2024-02-18T00:45:19.054281+0000","last_became_active":"2024-02-18T00:29:21.244097+0000","last_became_peered":"2024-02-18T00:29:21.244097+0000","last_unstale":"2024-02-18T00:45:19.054281+0000","last_undegraded":"2024-02-18T00:45:19.054281+0000","last_fullsized":"2024-02-18T00:45:19.054281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.135214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.135214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":308,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055673+0000","last_change":"2024-02-18T00:43:44.122130+0000","last_active":"2024-02-18T00:45:19.055673+0000","last_peered":"2024-02-18T00:45:19.055673+0000","last_clean":"2024-02-18T00:45:19.055673+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:45:19.055673+0000","last_undegraded":"2024-02-18T00:45:19.055673+0000","last_fullsized":"2024-02-18T00:45:19.055673+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.122056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:44.122056+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.122056+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":836,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.824310+0000","last_change":"2024-02-18T00:44:34.576745+0000","last_active":"2024-02-18T00:45:18.824310+0000","last_peered":"2024-02-18T00:45:18.824310+0000","last_clean":"2024-02-18T00:45:18.824310+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:45:18.824310+0000","last_undegraded":"2024-02-18T00:45:18.824310+0000","last_fullsized":"2024-02-18T00:45:18.824310+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:44:34.576675+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:44:34.576675+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":760,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055741+0000","last_change":"2024-02-18T00:43:10.797905+0000","last_active":"2024-02-18T00:45:19.055741+0000","last_peered":"2024-02-18T00:45:19.055741+0000","last_clean":"2024-02-18T00:45:19.055741+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:45:19.055741+0000","last_undegraded":"2024-02-18T00:45:19.055741+0000","last_fullsized":"2024-02-18T00:45:19.055741+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:10.797791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:10.797791+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"466'10","reported_seq":970,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055092+0000","last_change":"2024-02-18T00:43:37.546013+0000","last_active":"2024-02-18T00:45:19.055092+0000","last_peered":"2024-02-18T00:45:19.055092+0000","last_clean":"2024-02-18T00:45:19.055092+0000","last_became_active":"2024-02-18T00:39:36.751716+0000","last_became_peered":"2024-02-18T00:39:36.751716+0000","last_unstale":"2024-02-18T00:45:19.055092+0000","last_undegraded":"2024-02-18T00:45:19.055092+0000","last_fullsized":"2024-02-18T00:45:19.055092+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:43:37.545939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.545939+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":308,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054569+0000","last_change":"2024-02-18T00:43:51.101862+0000","last_active":"2024-02-18T00:45:19.054569+0000","last_peered":"2024-02-18T00:45:19.054569+0000","last_clean":"2024-02-18T00:45:19.054569+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:45:19.054569+0000","last_undegraded":"2024-02-18T00:45:19.054569+0000","last_fullsized":"2024-02-18T00:45:19.054569+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.101796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.101796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":783,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055178+0000","last_change":"2024-02-18T00:43:04.887857+0000","last_active":"2024-02-18T00:45:19.055178+0000","last_peered":"2024-02-18T00:45:19.055178+0000","last_clean":"2024-02-18T00:45:19.055178+0000","last_became_active":"2024-02-18T00:29:17.093940+0000","last_became_peered":"2024-02-18T00:29:17.093940+0000","last_unstale":"2024-02-18T00:45:19.055178+0000","last_undegraded":"2024-02-18T00:45:19.055178+0000","last_fullsized":"2024-02-18T00:45:19.055178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:43:04.887771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.887771+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":732,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054835+0000","last_change":"2024-02-18T00:42:52.999275+0000","last_active":"2024-02-18T00:45:19.054835+0000","last_peered":"2024-02-18T00:45:19.054835+0000","last_clean":"2024-02-18T00:45:19.054835+0000","last_became_active":"2024-02-18T00:29:14.713448+0000","last_became_peered":"2024-02-18T00:29:14.713448+0000","last_unstale":"2024-02-18T00:45:19.054835+0000","last_undegraded":"2024-02-18T00:45:19.054835+0000","last_fullsized":"2024-02-18T00:45:19.054835+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:52.999208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:49.512572+0000","last_clean_scrub_stamp":"2024-02-18T00:42:52.999208+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":750,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.824385+0000","last_change":"2024-02-18T00:44:46.488019+0000","last_active":"2024-02-18T00:45:18.824385+0000","last_peered":"2024-02-18T00:45:18.824385+0000","last_clean":"2024-02-18T00:45:18.824385+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:45:18.824385+0000","last_undegraded":"2024-02-18T00:45:18.824385+0000","last_fullsized":"2024-02-18T00:45:18.824385+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.487955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":694,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.820666+0000","last_change":"2024-02-18T00:43:37.193806+0000","last_active":"2024-02-18T00:45:18.820666+0000","last_peered":"2024-02-18T00:45:18.820666+0000","last_clean":"2024-02-18T00:45:18.820666+0000","last_became_active":"2024-02-18T00:33:04.698004+0000","last_became_peered":"2024-02-18T00:33:04.698004+0000","last_unstale":"2024-02-18T00:45:18.820666+0000","last_undegraded":"2024-02-18T00:45:18.820666+0000","last_fullsized":"2024-02-18T00:45:18.820666+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:43:37.193693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.193693+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":787,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.821982+0000","last_change":"2024-02-18T00:43:39.154626+0000","last_active":"2024-02-18T00:45:18.821982+0000","last_peered":"2024-02-18T00:45:18.821982+0000","last_clean":"2024-02-18T00:45:18.821982+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:45:18.821982+0000","last_undegraded":"2024-02-18T00:45:18.821982+0000","last_fullsized":"2024-02-18T00:45:18.821982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:43:39.154561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.154561+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":764,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054194+0000","last_change":"2024-02-18T00:44:25.509600+0000","last_active":"2024-02-18T00:45:19.054194+0000","last_peered":"2024-02-18T00:45:19.054194+0000","last_clean":"2024-02-18T00:45:19.054194+0000","last_became_active":"2024-02-18T00:39:36.741489+0000","last_became_peered":"2024-02-18T00:39:36.741489+0000","last_unstale":"2024-02-18T00:45:19.054194+0000","last_undegraded":"2024-02-18T00:45:19.054194+0000","last_fullsized":"2024-02-18T00:45:19.054194+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:25.509523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.509523+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":743,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055766+0000","last_change":"2024-02-18T00:44:35.012928+0000","last_active":"2024-02-18T00:45:19.055766+0000","last_peered":"2024-02-18T00:45:19.055766+0000","last_clean":"2024-02-18T00:45:19.055766+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:45:19.055766+0000","last_undegraded":"2024-02-18T00:45:19.055766+0000","last_fullsized":"2024-02-18T00:45:19.055766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:35.012802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.012802+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":679,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053963+0000","last_change":"2024-02-18T00:44:38.348343+0000","last_active":"2024-02-18T00:45:19.053963+0000","last_peered":"2024-02-18T00:45:19.053963+0000","last_clean":"2024-02-18T00:45:19.053963+0000","last_became_active":"2024-02-18T00:33:02.473162+0000","last_became_peered":"2024-02-18T00:33:02.473162+0000","last_unstale":"2024-02-18T00:45:19.053963+0000","last_undegraded":"2024-02-18T00:45:19.053963+0000","last_fullsized":"2024-02-18T00:45:19.053963+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.348263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.348263+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":777,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054631+0000","last_change":"2024-02-18T00:43:43.140277+0000","last_active":"2024-02-18T00:45:19.054631+0000","last_peered":"2024-02-18T00:45:19.054631+0000","last_clean":"2024-02-18T00:45:19.054631+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:45:19.054631+0000","last_undegraded":"2024-02-18T00:45:19.054631+0000","last_fullsized":"2024-02-18T00:45:19.054631+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:43:43.140135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.140135+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":765,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.053982+0000","last_change":"2024-02-18T00:42:13.912023+0000","last_active":"2024-02-18T00:45:19.053982+0000","last_peered":"2024-02-18T00:45:19.053982+0000","last_clean":"2024-02-18T00:45:19.053982+0000","last_became_active":"2024-02-18T00:39:36.761839+0000","last_became_peered":"2024-02-18T00:39:36.761839+0000","last_unstale":"2024-02-18T00:45:19.053982+0000","last_undegraded":"2024-02-18T00:45:19.053982+0000","last_fullsized":"2024-02-18T00:45:19.053982+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:42:13.911916+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:42:13.911916+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":739,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819703+0000","last_change":"2024-02-18T00:44:44.755473+0000","last_active":"2024-02-18T00:45:18.819703+0000","last_peered":"2024-02-18T00:45:18.819703+0000","last_clean":"2024-02-18T00:45:18.819703+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:45:18.819703+0000","last_undegraded":"2024-02-18T00:45:18.819703+0000","last_fullsized":"2024-02-18T00:45:18.819703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.755386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:44.755386+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.755386+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":695,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.823007+0000","last_change":"2024-02-18T00:43:48.486113+0000","last_active":"2024-02-18T00:45:18.823007+0000","last_peered":"2024-02-18T00:45:18.823007+0000","last_clean":"2024-02-18T00:45:18.823007+0000","last_became_active":"2024-02-18T00:39:36.771648+0000","last_became_peered":"2024-02-18T00:39:36.771648+0000","last_unstale":"2024-02-18T00:45:18.823007+0000","last_undegraded":"2024-02-18T00:45:18.823007+0000","last_fullsized":"2024-02-18T00:45:18.823007+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:48.486037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:48.486037+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":805,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819392+0000","last_change":"2024-02-18T00:43:01.284219+0000","last_active":"2024-02-18T00:45:18.819392+0000","last_peered":"2024-02-18T00:45:18.819392+0000","last_clean":"2024-02-18T00:45:18.819392+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:45:18.819392+0000","last_undegraded":"2024-02-18T00:45:18.819392+0000","last_fullsized":"2024-02-18T00:45:18.819392+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:43:01.284140+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.284140+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054758+0000","last_change":"2024-02-18T00:43:03.809975+0000","last_active":"2024-02-18T00:45:19.054758+0000","last_peered":"2024-02-18T00:45:19.054758+0000","last_clean":"2024-02-18T00:45:19.054758+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:45:19.054758+0000","last_undegraded":"2024-02-18T00:45:19.054758+0000","last_fullsized":"2024-02-18T00:45:19.054758+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.809907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.809907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":761,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.057170+0000","last_change":"2024-02-18T00:43:34.784518+0000","last_active":"2024-02-18T00:45:19.057170+0000","last_peered":"2024-02-18T00:45:19.057170+0000","last_clean":"2024-02-18T00:45:19.057170+0000","last_became_active":"2024-02-18T00:39:36.749467+0000","last_became_peered":"2024-02-18T00:39:36.749467+0000","last_unstale":"2024-02-18T00:45:19.057170+0000","last_undegraded":"2024-02-18T00:45:19.057170+0000","last_fullsized":"2024-02-18T00:45:19.057170+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.784446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.784446+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":676,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.822619+0000","last_change":"2024-02-18T00:42:21.695753+0000","last_active":"2024-02-18T00:45:18.822619+0000","last_peered":"2024-02-18T00:45:18.822619+0000","last_clean":"2024-02-18T00:45:18.822619+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:45:18.822619+0000","last_undegraded":"2024-02-18T00:45:18.822619+0000","last_fullsized":"2024-02-18T00:45:18.822619+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:21.695689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:42:21.695689+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":772,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819758+0000","last_change":"2024-02-18T00:44:34.749044+0000","last_active":"2024-02-18T00:45:18.819758+0000","last_peered":"2024-02-18T00:45:18.819758+0000","last_clean":"2024-02-18T00:45:18.819758+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:45:18.819758+0000","last_undegraded":"2024-02-18T00:45:18.819758+0000","last_fullsized":"2024-02-18T00:45:18.819758+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:44:34.748966+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:44:34.748966+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":308,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055886+0000","last_change":"2024-02-18T00:44:31.152577+0000","last_active":"2024-02-18T00:45:19.055886+0000","last_peered":"2024-02-18T00:45:19.055886+0000","last_clean":"2024-02-18T00:45:19.055886+0000","last_became_active":"2024-02-18T00:39:36.761469+0000","last_became_peered":"2024-02-18T00:39:36.761469+0000","last_unstale":"2024-02-18T00:45:19.055886+0000","last_undegraded":"2024-02-18T00:45:19.055886+0000","last_fullsized":"2024-02-18T00:45:19.055886+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:31.152515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.152515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":303,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054823+0000","last_change":"2024-02-18T00:43:39.066050+0000","last_active":"2024-02-18T00:45:19.054823+0000","last_peered":"2024-02-18T00:45:19.054823+0000","last_clean":"2024-02-18T00:45:19.054823+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:45:19.054823+0000","last_undegraded":"2024-02-18T00:45:19.054823+0000","last_fullsized":"2024-02-18T00:45:19.054823+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.065952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.065952+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":690,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054880+0000","last_change":"2024-02-18T00:44:30.255270+0000","last_active":"2024-02-18T00:45:19.054880+0000","last_peered":"2024-02-18T00:45:19.054880+0000","last_clean":"2024-02-18T00:45:19.054880+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:45:19.054880+0000","last_undegraded":"2024-02-18T00:45:19.054880+0000","last_fullsized":"2024-02-18T00:45:19.054880+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:30.255199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:30.255199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":781,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.818996+0000","last_change":"2024-02-18T00:43:34.567432+0000","last_active":"2024-02-18T00:45:18.818996+0000","last_peered":"2024-02-18T00:45:18.818996+0000","last_clean":"2024-02-18T00:45:18.818996+0000","last_became_active":"2024-02-18T00:29:17.085242+0000","last_became_peered":"2024-02-18T00:29:17.085242+0000","last_unstale":"2024-02-18T00:45:18.818996+0000","last_undegraded":"2024-02-18T00:45:18.818996+0000","last_fullsized":"2024-02-18T00:45:18.818996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:34.567365+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.567365+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":732,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.055430+0000","last_change":"2024-02-18T00:43:44.142781+0000","last_active":"2024-02-18T00:45:19.055430+0000","last_peered":"2024-02-18T00:45:19.055430+0000","last_clean":"2024-02-18T00:45:19.055430+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:45:19.055430+0000","last_undegraded":"2024-02-18T00:45:19.055430+0000","last_fullsized":"2024-02-18T00:45:19.055430+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.142683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.142683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":756,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054629+0000","last_change":"2024-02-18T00:42:51.096086+0000","last_active":"2024-02-18T00:45:19.054629+0000","last_peered":"2024-02-18T00:45:19.054629+0000","last_clean":"2024-02-18T00:45:19.054629+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:45:19.054629+0000","last_undegraded":"2024-02-18T00:45:19.054629+0000","last_fullsized":"2024-02-18T00:45:19.054629+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:42:51.096010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:42:51.096010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":674,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.057295+0000","last_change":"2024-02-18T00:43:50.155273+0000","last_active":"2024-02-18T00:45:19.057295+0000","last_peered":"2024-02-18T00:45:19.057295+0000","last_clean":"2024-02-18T00:45:19.057295+0000","last_became_active":"2024-02-18T00:33:04.720747+0000","last_became_peered":"2024-02-18T00:33:04.720747+0000","last_unstale":"2024-02-18T00:45:19.057295+0000","last_undegraded":"2024-02-18T00:45:19.057295+0000","last_fullsized":"2024-02-18T00:45:19.057295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.155205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.155205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":835,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:19.054756+0000","last_change":"2024-02-18T00:44:46.283742+0000","last_active":"2024-02-18T00:45:19.054756+0000","last_peered":"2024-02-18T00:45:19.054756+0000","last_clean":"2024-02-18T00:45:19.054756+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:45:19.054756+0000","last_undegraded":"2024-02-18T00:45:19.054756+0000","last_fullsized":"2024-02-18T00:45:19.054756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-18T00:44:46.283668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.283668+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":756,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.822756+0000","last_change":"2024-02-18T00:43:32.499214+0000","last_active":"2024-02-18T00:45:18.822756+0000","last_peered":"2024-02-18T00:45:18.822756+0000","last_clean":"2024-02-18T00:45:18.822756+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:45:18.822756+0000","last_undegraded":"2024-02-18T00:45:18.822756+0000","last_fullsized":"2024-02-18T00:45:18.822756+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.499132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.499132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":739,"reported_epoch":761,"state":"active+clean","last_fresh":"2024-02-18T00:45:18.819113+0000","last_change":"2024-02-18T00:44:39.696355+0000","last_active":"2024-02-18T00:45:18.819113+0000","last_peered":"2024-02-18T00:45:18.819113+0000","last_clean":"2024-02-18T00:45:18.819113+0000","last_became_active":"2024-02-18T00:29:19.097216+0000","last_became_peered":"2024-02-18T00:29:19.097216+0000","last_unstale":"2024-02-18T00:45:18.819113+0000","last_undegraded":"2024-02-18T00:45:18.819113+0000","last_fullsized":"2024-02-18T00:45:18.819113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:39.696257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:39.696257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":251,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":4000257,"num_objects":38,"num_object_clones":1,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":20,"num_read_kb":7,"num_write":58,"num_write_kb":3960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6451200,"data_stored":6451200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":24,"acting":24,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":382,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":42,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":942,"num_read_kb":776,"num_write":562,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":1210,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1278,"ondisk_log_size":1278,"up":126,"acting":126,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":956,"num_read_kb":951,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":900,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266885,"num_pgs":86,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315488,"kb_used_data":5976,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429664,"statfs":{"total":95995035648,"available":95671975936,"internally_reserved":0,"allocated":6119424,"data_stored":3270890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.53600000000000003,"15min":0.56899999999999995},"min":{"1min":0.26100000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.60299999999999998,"5min":4.7389999999999999,"15min":6.8700000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.497,"15min":0.59299999999999997},"min":{"1min":0.36199999999999999,"5min":0.218,"15min":0.218},"max":{"1min":0.57599999999999996,"5min":1.0780000000000001,"15min":11.288},"last":1.24}]},{"osd":1,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.63200000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.7330000000000001,"5min":2.968,"15min":7.4729999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.64800000000000002},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.7150000000000001,"5min":3.0019999999999998,"15min":7.7869999999999999},"last":1.145}]},{"osd":2,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.56999999999999995,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56299999999999994,"5min":4.8220000000000001,"15min":6.7380000000000004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.51300000000000001,"15min":0.59699999999999998},"min":{"1min":0.33800000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.64100000000000001,"5min":1.913,"15min":12.861000000000001},"last":1.3029999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.56999999999999995,"15min":0.58699999999999997},"min":{"1min":0.34799999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56999999999999995,"5min":4.8520000000000003,"15min":7.1360000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.53000000000000003,"15min":0.58199999999999996},"min":{"1min":0.376,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61199999999999999,"5min":1.948,"15min":3.7989999999999999},"last":1.2549999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.436,"15min":0.433},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.215},"max":{"1min":0.64600000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.53300000000000003,"15min":0.48399999999999999},"min":{"1min":0.28799999999999998,"5min":0.26300000000000001,"15min":0.246},"max":{"1min":0.62,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.67600000000000005,"5min":1.877,"15min":1.877},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.498,"15min":0.50800000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.25600000000000001},"max":{"1min":0.624,"5min":1.853,"15min":4.9130000000000003},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68899999999999995,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.622},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.59799999999999998,"15min":0.54700000000000004},"min":{"1min":0.309,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68799999999999994,"5min":5.3860000000000001,"15min":5.3860000000000001},"last":0.59099999999999997}]}]},{"osd":6,"up_from":47,"seq":201863463115,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309292,"kb_used_data":7972,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435860,"statfs":{"total":95995035648,"available":95678320640,"internally_reserved":0,"allocated":8163328,"data_stored":5316118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52500000000000002,"15min":1.2689999999999999},"min":{"1min":0.371,"5min":0.27000000000000002,"15min":0.22500000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":69.194999999999993},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.51800000000000002,"15min":0.74299999999999999},"min":{"1min":0.38300000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":10.433},"last":0.46300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55600000000000005,"15min":1.3500000000000001},"min":{"1min":0.37,"5min":0.313,"15min":0.26000000000000001},"max":{"1min":0.85399999999999998,"5min":1.9610000000000001,"15min":76.108999999999995},"last":0.502},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54300000000000004,"15min":1.339},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.28100000000000003},"max":{"1min":0.77200000000000002,"5min":3.665,"15min":71.872},"last":0.39900000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.82599999999999996},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.31},"max":{"1min":0.85099999999999998,"5min":3.702,"15min":39.939999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.53700000000000003,"15min":0.99099999999999999},"min":{"1min":0.44800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83099999999999996,"5min":1.5489999999999999,"15min":39.673000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.53500000000000003,"15min":0.84799999999999998},"min":{"1min":0.44400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":21.219000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57099999999999995,"15min":0.80000000000000004},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.29299999999999998},"max":{"1min":0.81699999999999995,"5min":3.7280000000000002,"15min":14.007999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.45600000000000002,"15min":0.52600000000000002},"min":{"1min":0.39000000000000001,"5min":0.25800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65400000000000003,"5min":0.748,"15min":8.1479999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.50800000000000001,"15min":0.72399999999999998},"min":{"1min":0.33900000000000002,"5min":0.27100000000000002,"15min":0.26100000000000001},"max":{"1min":0.73699999999999999,"5min":4.1310000000000002,"15min":35.18},"last":0.36599999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:45:04 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.47699999999999998,"15min":0.55900000000000005},"min":{"1min":0.39800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.75700000000000001,"5min":0.77200000000000002,"15min":10.606},"last":0.434},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.755},"min":{"1min":0.34399999999999997,"5min":0.30199999999999999,"15min":0.28199999999999997},"max":{"1min":0.75700000000000001,"5min":3.8439999999999999,"15min":35.119},"last":0.443}]},{"osd":7,"last update":"Sun Feb 18 00:44:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.55100000000000005},"min":{"1min":0.39000000000000001,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.72299999999999998,"5min":0.79200000000000004,"15min":5.4770000000000003},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.72499999999999998},"min":{"1min":0.377,"5min":0.318,"15min":0.318},"max":{"1min":0.77400000000000002,"5min":0.79100000000000004,"15min":32.985999999999997},"last":0.41599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":84,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305148,"kb_used_data":7924,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440004,"statfs":{"total":95995035648,"available":95682564096,"internally_reserved":0,"allocated":8114176,"data_stored":5265749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33800000000000002,"15min":0.64300000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":0.51200000000000001,"5min":0.77400000000000002,"15min":32.106999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.436,"15min":0.71899999999999997},"min":{"1min":0.247,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.83499999999999996,"5min":6.625,"15min":43.137},"last":0.35599999999999998}]},{"osd":2,"last update":"Sun Feb 18 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.35099999999999998,"15min":0.752},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.19900000000000001},"max":{"1min":0.47799999999999998,"5min":0.70199999999999996,"15min":49.201000000000001},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38400000000000001,"15min":0.58699999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.217},"max":{"1min":0.57599999999999996,"5min":0.85299999999999998,"15min":17.596},"last":0.34100000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.71699999999999997},"min":{"1min":0.28199999999999997,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":37.524000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42999999999999999,"15min":0.73899999999999999},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.77000000000000002,"5min":0.89200000000000002,"15min":32.036000000000001},"last":0.375}]},{"osd":4,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54600000000000004,"15min":0.753},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29499999999999998},"max":{"1min":0.70299999999999996,"5min":0.99299999999999999,"15min":17.486999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56100000000000005,"15min":0.77700000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.32500000000000001},"max":{"1min":0.749,"5min":0.98299999999999998,"15min":12.75},"last":0.63600000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.76000000000000001},"min":{"1min":0.38900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84599999999999997,"5min":0.93300000000000005,"15min":12.611000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.77700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.33100000000000002},"max":{"1min":0.88700000000000001,"5min":1.141,"15min":17.417000000000002},"last":0.57099999999999995}]},{"osd":6,"last update":"Sun Feb 18 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.79200000000000004},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":12.834},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58999999999999997,"15min":0.77700000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.35499999999999998},"max":{"1min":1.046,"5min":1.046,"15min":12.651999999999999},"last":0.621}]},{"osd":7,"last update":"Sun Feb 18 00:44:42 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.79400000000000004},"min":{"1min":0.40500000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76300000000000001,"5min":1.175,"15min":17.452000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.79900000000000004},"min":{"1min":0.44500000000000001,"5min":0.41799999999999998,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":1.012,"15min":12.805},"last":0.67300000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738600,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":303244,"kb_used_data":6020,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93441908,"statfs":{"total":95995035648,"available":95684513792,"internally_reserved":0,"allocated":6164480,"data_stored":3319655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.35399999999999998,"15min":0.99299999999999999},"min":{"1min":0.23300000000000001,"5min":0.217,"15min":0.18099999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":133.38999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.40699999999999997,"15min":0.79300000000000004},"min":{"1min":0.26700000000000002,"5min":0.23499999999999999,"15min":0.20999999999999999},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":90.623000000000005},"last":0.40600000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40300000000000002,"15min":0.89700000000000002},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.20200000000000001},"max":{"1min":0.61699999999999999,"5min":1.782,"15min":90.850999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.36099999999999999,"15min":0.85999999999999999},"min":{"1min":0.26700000000000002,"5min":0.20300000000000001,"15min":0.20000000000000001},"max":{"1min":0.497,"5min":0.77600000000000002,"15min":80.537000000000006},"last":0.53100000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.39600000000000002,"15min":0.64900000000000002},"min":{"1min":0.29899999999999999,"5min":0.21299999999999999,"15min":0.20899999999999999},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":19.006},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.46500000000000002,"15min":1.054},"min":{"1min":0.33900000000000002,"5min":0.29199999999999998,"15min":0.23999999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":146.71899999999999},"last":0.45000000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59399999999999997,"15min":0.93200000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26700000000000002},"max":{"1min":1.885,"5min":1.885,"15min":57.914000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.627,"15min":0.98599999999999999},"min":{"1min":0.45400000000000001,"5min":0.33400000000000002,"15min":0.30399999999999999},"max":{"1min":2.8690000000000002,"5min":2.8690000000000002,"15min":90.070999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Sun Feb 18 00:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.63100000000000001,"15min":0.96599999999999997},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.32000000000000001},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":90.049000000000007},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.99199999999999999},"min":{"1min":0.47499999999999998,"5min":0.34399999999999997,"15min":0.32200000000000001},"max":{"1min":1.853,"5min":1.853,"15min":57.991},"last":0.65800000000000003}]},{"osd":6,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60899999999999999,"15min":0.80800000000000005},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.35999999999999999},"max":{"1min":1.194,"5min":1.194,"15min":19.039000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.63300000000000001,"15min":0.996},"min":{"1min":0.53000000000000003,"5min":0.39700000000000002,"15min":0.33500000000000002},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":89.989999999999995},"last":0.56100000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":1.0569999999999999},"min":{"1min":0.51500000000000001,"5min":0.36799999999999999,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":1.732,"15min":90.025000000000006},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59499999999999997,"15min":1},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.83499999999999996,"5min":0.89700000000000002,"15min":57.962000000000003},"last":0.59999999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411551,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305176,"kb_used_data":7952,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93439976,"statfs":{"total":95995035648,"available":95682535424,"internally_reserved":0,"allocated":8142848,"data_stored":5306297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.45100000000000001,"15min":0.51100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.192},"max":{"1min":8.6489999999999991,"5min":8.6489999999999991,"15min":12.144},"last":0.315},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.36399999999999999,"15min":0.59799999999999998},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":18.891999999999999},"last":0.24299999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38100000000000001,"15min":0.49399999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.19700000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":10.138999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.38500000000000001,"15min":0.53400000000000003},"min":{"1min":0.248,"5min":0.22600000000000001,"15min":0.20399999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":18.919},"last":0.28499999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.505,"15min":0.61199999999999999},"min":{"1min":0.30299999999999999,"5min":0.28499999999999998,"15min":0.24099999999999999},"max":{"1min":8.5640000000000001,"5min":8.5640000000000001,"15min":22.712},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.58699999999999997},"min":{"1min":0.29099999999999998,"5min":0.26400000000000001,"15min":0.23899999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":22.359000000000002},"last":0.32400000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.68700000000000006},"min":{"1min":0.379,"5min":0.37,"15min":0.32500000000000001},"max":{"1min":0.83999999999999997,"5min":0.94899999999999995,"15min":18.812000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59499999999999997,"15min":0.67400000000000004},"min":{"1min":0.39000000000000001,"5min":0.38300000000000001,"15min":0.28699999999999998},"max":{"1min":1.002,"5min":1.002,"15min":10.016},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:44:49 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":0.71599999999999997,"15min":0.69399999999999995},"min":{"1min":0.39600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":11.805,"5min":11.805,"15min":11.805},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.05,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.317},"max":{"1min":8.6059999999999999,"5min":8.6059999999999999,"15min":10.114000000000001},"last":0.68400000000000005}]},{"osd":6,"last update":"Sun Feb 18 00:45:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.72999999999999998,"15min":0.69699999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35799999999999998},"max":{"1min":1.663,"5min":11.680999999999999,"15min":11.680999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.70099999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.34100000000000003},"max":{"1min":0.871,"5min":0.90500000000000003,"15min":18.861000000000001},"last":0.60899999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:44:36 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":11.746,"5min":11.746,"15min":11.746},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.67300000000000004,"15min":0.73699999999999999},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.372},"max":{"1min":1.115,"5min":1.115,"15min":10.058999999999999},"last":0.65300000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149913,"num_pgs":93,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":315528,"kb_used_data":6016,"kb_used_omap":0,"kb_used_meta":309504,"kb_avail":93429624,"statfs":{"total":95995035648,"available":95671934976,"internally_reserved":0,"allocated":6160384,"data_stored":3300235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316932096},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.45700000000000002},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":25.082999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38,"15min":0.50800000000000001},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":18.911000000000001},"last":0.312}]},{"osd":1,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.45300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.223},"max":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":6.8479999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.68400000000000005},"min":{"1min":0.26700000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.59999999999999998,"5min":0.73899999999999999,"15min":23.606999999999999},"last":0.29699999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41299999999999998,"15min":0.64300000000000002},"min":{"1min":0.28000000000000003,"5min":0.25700000000000001,"15min":0.23799999999999999},"max":{"1min":0.877,"5min":0.877,"15min":23.405000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.40000000000000002,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.23999999999999999},"max":{"1min":0.57699999999999996,"5min":0.85599999999999998,"15min":5.3479999999999999},"last":0.36399999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58599999999999997,"15min":0.68899999999999995},"min":{"1min":0.36799999999999999,"5min":0.36299999999999999,"15min":0.33600000000000002},"max":{"1min":0.82299999999999995,"5min":0.93000000000000005,"15min":10.131},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.36599999999999999,"15min":0.32700000000000001},"max":{"1min":0.85999999999999999,"5min":1.1699999999999999,"15min":5.4630000000000001},"last":0.71299999999999997}]},{"osd":5,"last update":"Sun Feb 18 00:44:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.38700000000000001,"15min":0.32800000000000001},"max":{"1min":0.90000000000000002,"5min":4.7469999999999999,"15min":7.4249999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61199999999999999,"15min":0.67500000000000004},"min":{"1min":0.375,"5min":0.34200000000000003,"15min":0.32400000000000001},"max":{"1min":0.98999999999999999,"5min":1.0860000000000001,"15min":6.7750000000000004},"last":0.749}]},{"osd":6,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.747},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.93700000000000006,"15min":10.189},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":0.96199999999999997,"15min":6.4279999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:44:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.39500000000000002,"15min":0.35199999999999998},"max":{"1min":0.95399999999999996,"5min":5.0830000000000002,"15min":6.8220000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.63200000000000001,"15min":0.73599999999999999},"min":{"1min":0.441,"5min":0.432,"15min":0.39800000000000002},"max":{"1min":0.91600000000000004,"5min":1.0760000000000001,"15min":10.164999999999999},"last":0.83599999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986388,"num_pgs":83,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311436,"kb_used_data":6020,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433716,"statfs":{"total":95995035648,"available":95676125184,"internally_reserved":0,"allocated":6164480,"data_stored":3324530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.85999999999999999},"min":{"1min":0.38700000000000001,"5min":0.249,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.86899999999999999,"15min":77.581999999999994},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.83799999999999997},"min":{"1min":0.36099999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.69999999999999996,"5min":0.753,"15min":78.063999999999993},"last":0.41299999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52100000000000002,"15min":0.61499999999999999},"min":{"1min":0.33100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68700000000000006,"5min":0.74199999999999999,"15min":9.3160000000000007},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52000000000000002,"15min":0.876},"min":{"1min":0.40100000000000002,"5min":0.32500000000000001,"15min":0.29699999999999999},"max":{"1min":0.66300000000000003,"5min":0.754,"15min":81.242000000000004},"last":0.51900000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.61599999999999999},"min":{"1min":0.35299999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69099999999999995,"5min":0.72299999999999998,"15min":11.691000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.57499999999999996,"15min":0.66800000000000004},"min":{"1min":0.35999999999999999,"5min":0.30199999999999999,"15min":0.27200000000000002},"max":{"1min":0.752,"5min":4.7149999999999999,"15min":17.77},"last":0.66000000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54000000000000004,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.33700000000000002,"15min":0.29599999999999999},"max":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":7.218},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.73099999999999998},"min":{"1min":0.41799999999999998,"5min":0.313,"15min":0.309},"max":{"1min":0.70199999999999996,"5min":1.21,"15min":33.530999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.443,"15min":0.442},"min":{"1min":0.28899999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65100000000000002,"5min":0.71899999999999997,"15min":0.72299999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.46400000000000002,"15min":0.45500000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.39000000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:44:29 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46000000000000002,"15min":0.45100000000000001},"min":{"1min":0.27500000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.66700000000000004,"5min":0.67100000000000004,"15min":0.76100000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.498,"15min":0.49199999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.26000000000000001},"max":{"1min":0.67800000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.44900000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:44:32 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.76700000000000002,"15min":0.76900000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.495,"5min":0.51000000000000001,"15min":0.50600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.64800000000000002,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.48799999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724751,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307284,"kb_used_data":5964,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93437868,"statfs":{"total":95995035648,"available":95680376832,"internally_reserved":0,"allocated":6107136,"data_stored":3287377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.502,"15min":0.53500000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.217},"max":{"1min":0.71499999999999997,"5min":2.21,"15min":6.3150000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.498,"15min":0.60299999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.26000000000000001},"max":{"1min":1.123,"5min":1.782,"15min":12.744},"last":0.52600000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.64300000000000002},"min":{"1min":0.313,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":1.2010000000000001,"5min":2.1509999999999998,"15min":13.25},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.65900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.29699999999999999},"max":{"1min":0.72899999999999998,"5min":1.9430000000000001,"15min":13.262},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.58899999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.251},"max":{"1min":1.4510000000000001,"5min":2.2559999999999998,"15min":6.1680000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.58899999999999997,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.35099999999999998,"15min":0.308},"max":{"1min":1.4370000000000001,"5min":1.8799999999999999,"15min":6.1879999999999997},"last":0.55400000000000005}]},{"osd":3,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.622},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":0.70599999999999996,"5min":3.637,"15min":7.641},"last":0.496},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53700000000000003,"15min":0.60599999999999998},"min":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.27600000000000002},"max":{"1min":1.22,"5min":1.853,"15min":8.8650000000000002},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Feb 18 00:44:45 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.44800000000000001,"15min":0.49299999999999999},"min":{"1min":0.32300000000000001,"5min":0.25,"15min":0.23599999999999999},"max":{"1min":0.64400000000000002,"5min":1.897,"15min":11.818},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.36899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.74199999999999999,"5min":3.5920000000000001,"15min":4.274},"last":0.40799999999999997}]},{"osd":6,"last update":"Sun Feb 18 00:45:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47999999999999998,"15min":0.56799999999999995},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":0.77900000000000003,"5min":2.7120000000000002,"15min":18.605},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.26900000000000002,"15min":0.26500000000000001},"max":{"1min":0.79900000000000004,"5min":2.6699999999999999,"15min":4.2089999999999996},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Feb 18 00:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.50600000000000001},"min":{"1min":0.314,"5min":0.27400000000000002,"15min":0.26600000000000001},"max":{"1min":1.2370000000000001,"5min":1.819,"15min":2.0739999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.503,"15min":0.52700000000000002},"min":{"1min":0.34799999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":4.649},"last":0.72399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":77824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2023424,"data_stored":2023424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2064384,"data_stored":2064384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":57344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":81920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":45056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2072576,"data_stored":2072576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":251,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:45:27.447 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-18T00:45:27.447 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:45:27.448 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:45:27.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd out 6 2024-02-18T00:45:27.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:25.900345+0000 mgr.y (mgr.24454) 660 : audit [DBG] from='client.31989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:45:27.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:26.019068+0000 mgr.y (mgr.24454) 661 : cluster [DBG] pgmap v1174: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 142 B/s wr, 1 op/s 2024-02-18T00:45:27.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:26.281264+0000 osd.3 (osd.3) 384 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:45:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:26.282470+0000 osd.3 (osd.3) 385 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:45:27.849 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:27.852 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:27.853 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:27.861 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.3 dump_historic_ops 2024-02-18T00:45:27.954 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:27.986 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:28.007 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:28.009 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:28.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:28.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:26.679466+0000 osd.5 (osd.5) 327 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:45:28.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:26.682932+0000 osd.5 (osd.5) 328 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:45:28.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:26.808824+0000 osd.1 (osd.1) 357 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:45:28.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:26.810073+0000 osd.1 (osd.1) 358 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:45:28.094 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.154895+0000 osd.2 (osd.2) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.155172+0000 osd.2 (osd.2) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.301409+0000 osd.3 (osd.3) 386 : cluster [DBG] 234.4 scrub starts 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.303042+0000 osd.3 (osd.3) 387 : cluster [DBG] 234.4 scrub ok 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.336600+0000 osd.0 (osd.0) 328 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.339503+0000 osd.0 (osd.0) 329 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:26.679466+0000 osd.5 (osd.5) 327 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:26.682932+0000 osd.5 (osd.5) 328 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:26.808824+0000 osd.1 (osd.1) 357 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:26.810073+0000 osd.1 (osd.1) 358 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:45:28.095 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.154895+0000 osd.2 (osd.2) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:28.096 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.155172+0000 osd.2 (osd.2) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:28.096 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.301409+0000 osd.3 (osd.3) 386 : cluster [DBG] 234.4 scrub starts 2024-02-18T00:45:28.096 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.303042+0000 osd.3 (osd.3) 387 : cluster [DBG] 234.4 scrub ok 2024-02-18T00:45:28.096 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.336600+0000 osd.0 (osd.0) 328 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:45:28.096 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.339503+0000 osd.0 (osd.0) 329 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:45:28.110 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:28.141 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:28.161 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:28.163 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:28.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:28.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:26.679466+0000 osd.5 (osd.5) 327 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:26.682932+0000 osd.5 (osd.5) 328 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:26.808824+0000 osd.1 (osd.1) 357 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:26.810073+0000 osd.1 (osd.1) 358 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.154895+0000 osd.2 (osd.2) 297 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.155172+0000 osd.2 (osd.2) 298 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.301409+0000 osd.3 (osd.3) 386 : cluster [DBG] 234.4 scrub starts 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.303042+0000 osd.3 (osd.3) 387 : cluster [DBG] 234.4 scrub ok 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.336600+0000 osd.0 (osd.0) 328 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:45:28.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.339503+0000 osd.0 (osd.0) 329 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:45:28.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:28.312 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:28.315 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:28.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:28.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:28.448 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:28.485 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:28.492 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:28.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:28.593 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:28.622 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:28.642 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:28.644 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:28.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:28.745 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:28.776 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:29.003 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:45:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:45:28+0000 lvl=info msg="Database locked, sleeping then retrying" logger=sqlstore error="database is locked" retry=0 2024-02-18T00:45:29.045 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:29.045 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:29.382 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.833653+0000 osd.1 (osd.1) 359 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:27.836704+0000 osd.1 (osd.1) 360 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.019502+0000 mgr.y (mgr.24454) 662 : cluster [DBG] pgmap v1175: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 124 B/s wr, 1 op/s 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.090725+0000 mon.a (mon.0) 3625 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.217568+0000 osd.7 (osd.7) 225 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.218404+0000 osd.7 (osd.7) 226 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.563043+0000 osd.2 (osd.2) 299 : cluster [DBG] 2.e scrub starts 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.563928+0000 osd.2 (osd.2) 300 : cluster [DBG] 2.e scrub ok 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.824350+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.833653+0000 osd.1 (osd.1) 359 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:27.836704+0000 osd.1 (osd.1) 360 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.019502+0000 mgr.y (mgr.24454) 662 : cluster [DBG] pgmap v1175: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 124 B/s wr, 1 op/s 2024-02-18T00:45:29.383 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.090725+0000 mon.a (mon.0) 3625 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-02-18T00:45:29.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.217568+0000 osd.7 (osd.7) 225 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:45:29.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.218404+0000 osd.7 (osd.7) 226 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:45:29.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.563043+0000 osd.2 (osd.2) 299 : cluster [DBG] 2.e scrub starts 2024-02-18T00:45:29.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.563928+0000 osd.2 (osd.2) 300 : cluster [DBG] 2.e scrub ok 2024-02-18T00:45:29.384 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.824350+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-02-18T00:45:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.833653+0000 osd.1 (osd.1) 359 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:45:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:27.836704+0000 osd.1 (osd.1) 360 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:45:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.019502+0000 mgr.y (mgr.24454) 662 : cluster [DBG] pgmap v1175: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 124 B/s wr, 1 op/s 2024-02-18T00:45:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.090725+0000 mon.a (mon.0) 3625 : cluster [DBG] osdmap e764: 8 total, 8 up, 8 in 2024-02-18T00:45:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.217568+0000 osd.7 (osd.7) 225 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:45:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.218404+0000 osd.7 (osd.7) 226 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:45:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.563043+0000 osd.2 (osd.2) 299 : cluster [DBG] 2.e scrub starts 2024-02-18T00:45:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.563928+0000 osd.2 (osd.2) 300 : cluster [DBG] 2.e scrub ok 2024-02-18T00:45:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.824350+0000 mon.a (mon.0) 3626 : cluster [DBG] osdmap e765: 8 total, 8 up, 8 in 2024-02-18T00:45:30.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.351589+0000 osd.3 (osd.3) 388 : cluster [DBG] 234.1 scrub starts 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:28.354831+0000 osd.3 (osd.3) 389 : cluster [DBG] 234.1 scrub ok 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:29.169928+0000 osd.7 (osd.7) 227 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:29.170781+0000 osd.7 (osd.7) 228 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:29.274813+0000 osd.0 (osd.0) 330 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:29.275696+0000 osd.0 (osd.0) 331 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:29.975915+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.351589+0000 osd.3 (osd.3) 388 : cluster [DBG] 234.1 scrub starts 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:28.354831+0000 osd.3 (osd.3) 389 : cluster [DBG] 234.1 scrub ok 2024-02-18T00:45:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:29.169928+0000 osd.7 (osd.7) 227 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:45:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:29.170781+0000 osd.7 (osd.7) 228 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:45:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:29.274813+0000 osd.0 (osd.0) 330 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:45:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:29.275696+0000 osd.0 (osd.0) 331 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:45:30.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:29.975915+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:30.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.351589+0000 osd.3 (osd.3) 388 : cluster [DBG] 234.1 scrub starts 2024-02-18T00:45:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:28.354831+0000 osd.3 (osd.3) 389 : cluster [DBG] 234.1 scrub ok 2024-02-18T00:45:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:29.169928+0000 osd.7 (osd.7) 227 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:45:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:29.170781+0000 osd.7 (osd.7) 228 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:45:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:29.274813+0000 osd.0 (osd.0) 330 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:45:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:29.275696+0000 osd.0 (osd.0) 331 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:45:30.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:29.975915+0000 mon.a (mon.0) 3627 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:30.611 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:30.627 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:30.632 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:30.633 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:30.735 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:30.768 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:30.790 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:30.792 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:30.793 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:30.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:30.923 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:30.944 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:30.945 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:30.946 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:31.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:31.079 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:31.371 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:29.641094+0000 osd.5 (osd.5) 329 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:45:31.371 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:29.643972+0000 osd.5 (osd.5) 330 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:45:31.371 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.019903+0000 mgr.y (mgr.24454) 663 : cluster [DBG] pgmap v1178: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:30.100052+0000 mon.a (mon.0) 3628 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.100112+0000 mon.a (mon.0) 3629 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:30.100822+0000 mon.a (mon.0) 3630 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.245368+0000 osd.0 (osd.0) 332 : cluster [DBG] 4.b scrub starts 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.248830+0000 osd.0 (osd.0) 333 : cluster [DBG] 4.b scrub ok 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.658667+0000 osd.2 (osd.2) 301 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.659577+0000 osd.2 (osd.2) 302 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:29.641094+0000 osd.5 (osd.5) 329 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:29.643972+0000 osd.5 (osd.5) 330 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:45:31.372 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.019903+0000 mgr.y (mgr.24454) 663 : cluster [DBG] pgmap v1178: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:45:31.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:30.100052+0000 mon.a (mon.0) 3628 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:31.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.100112+0000 mon.a (mon.0) 3629 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-02-18T00:45:31.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:30.100822+0000 mon.a (mon.0) 3630 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:31.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.245368+0000 osd.0 (osd.0) 332 : cluster [DBG] 4.b scrub starts 2024-02-18T00:45:31.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.248830+0000 osd.0 (osd.0) 333 : cluster [DBG] 4.b scrub ok 2024-02-18T00:45:31.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.658667+0000 osd.2 (osd.2) 301 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:45:31.373 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.659577+0000 osd.2 (osd.2) 302 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:45:31.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:29.641094+0000 osd.5 (osd.5) 329 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:29.643972+0000 osd.5 (osd.5) 330 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.019903+0000 mgr.y (mgr.24454) 663 : cluster [DBG] pgmap v1178: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:30.100052+0000 mon.a (mon.0) 3628 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.100112+0000 mon.a (mon.0) 3629 : cluster [DBG] osdmap e766: 8 total, 8 up, 8 in 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:30.100822+0000 mon.a (mon.0) 3630 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.245368+0000 osd.0 (osd.0) 332 : cluster [DBG] 4.b scrub starts 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.248830+0000 osd.0 (osd.0) 333 : cluster [DBG] 4.b scrub ok 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.658667+0000 osd.2 (osd.2) 301 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:45:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.659577+0000 osd.2 (osd.2) 302 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:45:32.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:45:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:45:32.133 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:32.137 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.837603+0000 osd.6 (osd.6) 339 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:45:32.137 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.839679+0000 osd.6 (osd.6) 340 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:45:32.137 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.876365+0000 osd.1 (osd.1) 361 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:45:32.137 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.879852+0000 osd.1 (osd.1) 362 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.984773+0000 osd.4 (osd.4) 405 : cluster [DBG] 4.c scrub starts 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:30.988255+0000 osd.4 (osd.4) 406 : cluster [DBG] 4.c scrub ok 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:31.096251+0000 mon.a (mon.0) 3631 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:31.100883+0000 mon.a (mon.0) 3632 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]': finished 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:31.100943+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:31.128325+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:31.128653+0000 mon.a (mon.0) 3635 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:31.662090+0000 osd.2 (osd.2) 303 : cluster [DBG] 5.a scrub starts 2024-02-18T00:45:32.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:31.663176+0000 osd.2 (osd.2) 304 : cluster [DBG] 5.a scrub ok 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.837603+0000 osd.6 (osd.6) 339 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.839679+0000 osd.6 (osd.6) 340 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.876365+0000 osd.1 (osd.1) 361 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.879852+0000 osd.1 (osd.1) 362 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.984773+0000 osd.4 (osd.4) 405 : cluster [DBG] 4.c scrub starts 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:30.988255+0000 osd.4 (osd.4) 406 : cluster [DBG] 4.c scrub ok 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:31.096251+0000 mon.a (mon.0) 3631 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:31.100883+0000 mon.a (mon.0) 3632 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]': finished 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:31.100943+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:31.128325+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:31.128653+0000 mon.a (mon.0) 3635 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:31.662090+0000 osd.2 (osd.2) 303 : cluster [DBG] 5.a scrub starts 2024-02-18T00:45:32.139 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:31.663176+0000 osd.2 (osd.2) 304 : cluster [DBG] 5.a scrub ok 2024-02-18T00:45:32.140 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:32.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:32.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:32.271 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:32.291 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:32.293 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:32.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:32.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:32.425 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:45:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:32] "GET /metrics HTTP/1.1" 200 192216 "" "Prometheus/2.33.4" 2024-02-18T00:45:32.427 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.837603+0000 osd.6 (osd.6) 339 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.839679+0000 osd.6 (osd.6) 340 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.876365+0000 osd.1 (osd.1) 361 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.879852+0000 osd.1 (osd.1) 362 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.984773+0000 osd.4 (osd.4) 405 : cluster [DBG] 4.c scrub starts 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:30.988255+0000 osd.4 (osd.4) 406 : cluster [DBG] 4.c scrub ok 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:31.096251+0000 mon.a (mon.0) 3631 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:31.100883+0000 mon.a (mon.0) 3632 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]': finished 2024-02-18T00:45:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:31.100943+0000 mon.a (mon.0) 3633 : cluster [DBG] osdmap e767: 8 total, 8 up, 8 in 2024-02-18T00:45:32.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:31.128325+0000 mon.a (mon.0) 3634 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:32.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:31.128653+0000 mon.a (mon.0) 3635 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-134"}]: dispatch 2024-02-18T00:45:32.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:31.662090+0000 osd.2 (osd.2) 303 : cluster [DBG] 5.a scrub starts 2024-02-18T00:45:32.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:31.663176+0000 osd.2 (osd.2) 304 : cluster [DBG] 5.a scrub ok 2024-02-18T00:45:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.568+0000 7f0190caa700 1 -- 172.21.15.59:0/2713249475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 msgr2=0x7f018c0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.568+0000 7f0190caa700 1 --2- 172.21.15.59:0/2713249475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f9090 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 rx=0x7f017c0047b0 tx=0x7f017c009cb0).stop 2024-02-18T00:45:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 -- 172.21.15.59:0/2713249475 shutdown_connections 2024-02-18T00:45:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 --2- 172.21.15.59:0/2713249475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f018c102210 0x7f018c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 --2- 172.21.15.59:0/2713249475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f9090 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 --2- 172.21.15.59:0/2713249475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f018c0f9610 0x7f018c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 -- 172.21.15.59:0/2713249475 >> 172.21.15.59:0/2713249475 conn(0x7f018c0f2820 msgr2=0x7f018c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:32.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 -- 172.21.15.59:0/2713249475 shutdown_connections 2024-02-18T00:45:32.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 -- 172.21.15.59:0/2713249475 wait complete. 2024-02-18T00:45:32.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 librados: starting msgr at 2024-02-18T00:45:32.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.569+0000 7f0190caa700 1 librados: starting objecter 2024-02-18T00:45:32.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 Processor -- start 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 -- start start 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 librados: setting wanted keys 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 librados: calling monclient init 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f018c0f9610 0x7f018c0f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f018c102210 0x7f018c0f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c116650 con 0x7f018c0f8cb0 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c1167c0 con 0x7f018c0f9610 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0190caa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c116980 con 0x7f018c102210 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34114/0 (socket says 172.21.15.59:34114) 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018ad9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f018c102210 0x7f018c0f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 -- 172.21.15.59:0/2472702306 learned_addr learned my addr 172.21.15.59:0/2472702306 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f018c0f9610 0x7f018c0f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 -- 172.21.15.59:0/2472702306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f018c102210 msgr2=0x7f018c0f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f018c102210 0x7f018c0f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 -- 172.21.15.59:0/2472702306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f018c0f9610 msgr2=0x7f018c0f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f018c0f9610 0x7f018c0f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018a59c700 1 -- 172.21.15.59:0/2472702306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01840097c0 con 0x7f018c0f8cb0 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f018ad9d700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f018c102210 0x7f018c0f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.570+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f018c0f9610 0x7f018c0f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f6c60 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 rx=0x7f017c009b10 tx=0x7f017c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f017b7fe700 10 client.?.objecter ms_handle_connect 0x7f018c0f8cb0 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f017b7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:32.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c018070 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c005040 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017c013e70 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c01ddb0 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f0190caa700 10 client.31995.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c116e70 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.571+0000 7f0190caa700 1 librados: init done 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.572+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f018c05fa60 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.572+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f017c01d420 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0174062c90 0x7f0174065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(768..768 src has 418..768) v4 ==== 8433+0+0 (secure 0 0 0) 0x7f017c07f020 con 0x7f018c0f8cb0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 osd_map(768..768 src has 418..768) v4 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 3 client.31995.objecter handle_osd_map got epochs [768,768] > 0 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 3 client.31995.objecter handle_osd_map decoding full epoch 768 2024-02-18T00:45:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0174062c90 0x7f0174065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 20 client.31995.objecter dump_active .. 0 homeless 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v768) v1 ==== 54+0+16 (secure 0 0 0) 0x7f017c07f4c0 con 0x7f018c0f8cb0 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 10 client.31995.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 1 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6801/3700238977 conn(0x7f018c0fc1e0 0x7f018c0fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 1 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 0 1 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c05ee80 osd=0 4 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 10 client.31995.objecter _send_command 1 2024-02-18T00:45:32.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c107290 con 0x7f018c0fc1e0 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f0174062c90 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0174062c90 0x7f0174065150 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 rx=0x7f018c0f8400 tx=0x7f018000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.573+0000 7f018ad9d700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6801/3700238977 conn(0x7f018c0fc1e0 0x7f018c0fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.574+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c0fc1e0 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.574+0000 7f018ad9d700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6801/3700238977 conn(0x7f018c0fc1e0 0x7f018c0fc5c0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.574+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c107290 con 0x7f018c0fc1e0 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.574+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.574+0000 7f017b7fe700 10 client.31995.objecter _finish_command 1 = system:0 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.574+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c05ee80 osd=0 4 2024-02-18T00:45:32.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.574+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 0 1 2024-02-18T00:45:32.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 10 client.31995.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 2 2024-02-18T00:45:32.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 2 2024-02-18T00:45:32.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c05ee80 osd=0 3 2024-02-18T00:45:32.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 0 2 2024-02-18T00:45:32.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c05ee80 osd=0 4 2024-02-18T00:45:32.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 10 client.31995.objecter _send_command 2 2024-02-18T00:45:32.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.590+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c0273c0 con 0x7f018c0fc1e0 2024-02-18T00:45:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c0273c0 con 0x7f018c0fc1e0 2024-02-18T00:45:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f017b7fe700 10 client.31995.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c05ee80 osd=0 4 2024-02-18T00:45:32.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 0 2 2024-02-18T00:45:32.592 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:45:32.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f0190caa700 10 client.31995.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 3 2024-02-18T00:45:32.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6805/26279110 conn(0x7f018c0598f0 0x7f018c059d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.591+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 3 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 1 3 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c028710 osd=1 4 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 10 client.31995.objecter _send_command 3 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c11d780 con 0x7f018c0598f0 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6805/26279110 conn(0x7f018c0598f0 0x7f018c059d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c0598f0 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6805/26279110 conn(0x7f018c0598f0 0x7f018c059d30 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c11d780 con 0x7f018c0598f0 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f017b7fe700 10 client.31995.objecter _finish_command 3 = system:0 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c028710 osd=1 4 2024-02-18T00:45:32.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.592+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 1 3 2024-02-18T00:45:32.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 10 client.31995.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 4 2024-02-18T00:45:32.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 4 2024-02-18T00:45:32.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c028710 osd=1 3 2024-02-18T00:45:32.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 1 4 2024-02-18T00:45:32.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c028710 osd=1 4 2024-02-18T00:45:32.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 10 client.31995.objecter _send_command 4 2024-02-18T00:45:32.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.607+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c059360 con 0x7f018c0598f0 2024-02-18T00:45:32.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c059360 con 0x7f018c0598f0 2024-02-18T00:45:32.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f017b7fe700 10 client.31995.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c028710 osd=1 4 2024-02-18T00:45:32.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 1 4 2024-02-18T00:45:32.610 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:45:32.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 10 client.31995.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 5 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6809/1153193637 conn(0x7f018c028270 0x7f018c11c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 5 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 2 5 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c059770 osd=2 4 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 10 client.31995.objecter _send_command 5 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c11c690 con 0x7f018c028270 2024-02-18T00:45:32.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.609+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6809/1153193637 conn(0x7f018c028270 0x7f018c11c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.610+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6809/1153193637 conn(0x7f018c028270 0x7f018c11c1a0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.610+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c028270 2024-02-18T00:45:32.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.610+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c11c690 con 0x7f018c028270 2024-02-18T00:45:32.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.610+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:45:32.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.610+0000 7f017b7fe700 10 client.31995.objecter _finish_command 5 = system:0 2024-02-18T00:45:32.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.610+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c059770 osd=2 4 2024-02-18T00:45:32.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.610+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 2 5 2024-02-18T00:45:32.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 10 client.31995.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 6 2024-02-18T00:45:32.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 6 2024-02-18T00:45:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c059770 osd=2 3 2024-02-18T00:45:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 2 6 2024-02-18T00:45:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c059770 osd=2 4 2024-02-18T00:45:32.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 10 client.31995.objecter _send_command 6 2024-02-18T00:45:32.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.624+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c0277d0 con 0x7f018c028270 2024-02-18T00:45:32.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c0277d0 con 0x7f018c028270 2024-02-18T00:45:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f017b7fe700 10 client.31995.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c059770 osd=2 4 2024-02-18T00:45:32.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 2 6 2024-02-18T00:45:32.630 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:45:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 10 client.31995.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 7 2024-02-18T00:45:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6813/1805700964 conn(0x7f018c11f900 0x7f018c121cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 7 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 3 7 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c027be0 osd=3 4 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 10 client.31995.objecter _send_command 7 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c124940 con 0x7f018c11f900 2024-02-18T00:45:32.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.626+0000 7f018ad9d700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6813/1805700964 conn(0x7f018c11f900 0x7f018c121cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.627+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c11f900 2024-02-18T00:45:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.627+0000 7f018ad9d700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6813/1805700964 conn(0x7f018c11f900 0x7f018c121cf0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.627+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c124940 con 0x7f018c11f900 2024-02-18T00:45:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.627+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:45:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.627+0000 7f017b7fe700 10 client.31995.objecter _finish_command 7 = system:0 2024-02-18T00:45:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.627+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c027be0 osd=3 4 2024-02-18T00:45:32.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.627+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 3 7 2024-02-18T00:45:32.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 10 client.31995.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 8 2024-02-18T00:45:32.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 8 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c027be0 osd=3 3 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 3 8 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c027be0 osd=3 4 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 10 client.31995.objecter _send_command 8 2024-02-18T00:45:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.642+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c11caf0 con 0x7f018c11f900 2024-02-18T00:45:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c11caf0 con 0x7f018c11f900 2024-02-18T00:45:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f017b7fe700 10 client.31995.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c027be0 osd=3 4 2024-02-18T00:45:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 3 8 2024-02-18T00:45:32.644 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:45:32.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 10 client.31995.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 9 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6800/2044387056 conn(0x7f018c11cc90 0x7f018c123360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 9 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 4 9 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.643+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11caf0 osd=4 4 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.644+0000 7f0190caa700 10 client.31995.objecter _send_command 9 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.644+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c11bee0 con 0x7f018c11cc90 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.644+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6800/2044387056 conn(0x7f018c11cc90 0x7f018c123360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.644+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6800/2044387056 conn(0x7f018c11cc90 0x7f018c123360 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.644+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c11cc90 2024-02-18T00:45:32.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.645+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c11bee0 con 0x7f018c11cc90 2024-02-18T00:45:32.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.645+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:45:32.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.645+0000 7f017b7fe700 10 client.31995.objecter _finish_command 9 = system:0 2024-02-18T00:45:32.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.645+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c11caf0 osd=4 4 2024-02-18T00:45:32.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.645+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 4 9 2024-02-18T00:45:32.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 10 client.31995.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 10 2024-02-18T00:45:32.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 10 2024-02-18T00:45:32.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c11caf0 osd=4 3 2024-02-18T00:45:32.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 4 10 2024-02-18T00:45:32.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11caf0 osd=4 4 2024-02-18T00:45:32.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 10 client.31995.objecter _send_command 10 2024-02-18T00:45:32.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.660+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c11bee0 con 0x7f018c11cc90 2024-02-18T00:45:32.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c11bee0 con 0x7f018c11cc90 2024-02-18T00:45:32.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f017b7fe700 10 client.31995.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c11caf0 osd=4 4 2024-02-18T00:45:32.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 4 10 2024-02-18T00:45:32.663 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 10 client.31995.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 11 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6804/2170432263 conn(0x7f018c123d60 0x7f018c1241c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 11 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 5 11 2024-02-18T00:45:32.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11bee0 osd=5 4 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 10 client.31995.objecter _send_command 11 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c12b230 con 0x7f018c123d60 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6804/2170432263 conn(0x7f018c123d60 0x7f018c1241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f0189d9b700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6804/2170432263 conn(0x7f018c123d60 0x7f018c1241c0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.662+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c123d60 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.663+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c12b230 con 0x7f018c123d60 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.663+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.663+0000 7f017b7fe700 10 client.31995.objecter _finish_command 11 = system:0 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.663+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c11bee0 osd=5 4 2024-02-18T00:45:32.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.663+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 5 11 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 10 client.31995.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 12 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 12 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c11bee0 osd=5 3 2024-02-18T00:45:32.678 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 5 12 2024-02-18T00:45:32.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11bee0 osd=5 4 2024-02-18T00:45:32.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 10 client.31995.objecter _send_command 12 2024-02-18T00:45:32.679 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.677+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c128ad0 con 0x7f018c123d60 2024-02-18T00:45:32.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.679+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c128ad0 con 0x7f018c123d60 2024-02-18T00:45:32.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.679+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.679+0000 7f017b7fe700 10 client.31995.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.679+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c11bee0 osd=5 4 2024-02-18T00:45:32.680 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.679+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 5 12 2024-02-18T00:45:32.681 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 10 client.31995.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 13 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6808/3356009390 conn(0x7f018c1291b0 0x7f018c12b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 13 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 6 13 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c128ad0 osd=6 4 2024-02-18T00:45:32.682 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 10 client.31995.objecter _send_command 13 2024-02-18T00:45:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c12e370 con 0x7f018c1291b0 2024-02-18T00:45:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f018ad9d700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6808/3356009390 conn(0x7f018c1291b0 0x7f018c12b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f018ad9d700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6808/3356009390 conn(0x7f018c1291b0 0x7f018c12b5a0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.680+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c1291b0 2024-02-18T00:45:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.681+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c12e370 con 0x7f018c1291b0 2024-02-18T00:45:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.681+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:45:32.683 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.681+0000 7f017b7fe700 10 client.31995.objecter _finish_command 13 = system:0 2024-02-18T00:45:32.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.681+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c128ad0 osd=6 4 2024-02-18T00:45:32.684 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.681+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 6 13 2024-02-18T00:45:32.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 10 client.31995.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 14 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 14 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c128ad0 osd=6 3 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 6 14 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c128ad0 osd=6 4 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 10 client.31995.objecter _send_command 14 2024-02-18T00:45:32.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.700+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c12e370 con 0x7f018c1291b0 2024-02-18T00:45:32.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c12e370 con 0x7f018c1291b0 2024-02-18T00:45:32.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f017b7fe700 10 client.31995.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c128ad0 osd=6 4 2024-02-18T00:45:32.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 6 14 2024-02-18T00:45:32.703 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:45:32.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 10 client.31995.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 15 2024-02-18T00:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6812/53025171 conn(0x7f018c12d6e0 0x7f018c12db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 15 2024-02-18T00:45:32.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 7 15 2024-02-18T00:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c12e370 osd=7 4 2024-02-18T00:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 10 client.31995.objecter _send_command 15 2024-02-18T00:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c131720 con 0x7f018c12d6e0 2024-02-18T00:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.702+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6812/53025171 conn(0x7f018c12d6e0 0x7f018c12db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.703+0000 7f017b7fe700 10 client.31995.objecter ms_handle_connect 0x7f018c12d6e0 2024-02-18T00:45:32.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.703+0000 7f018a59c700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6812/53025171 conn(0x7f018c12d6e0 0x7f018c12db60 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.704+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f018c131720 con 0x7f018c12d6e0 2024-02-18T00:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.704+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.704+0000 7f017b7fe700 10 client.31995.objecter _finish_command 15 = system:0 2024-02-18T00:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.704+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c12e370 osd=7 4 2024-02-18T00:45:32.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.704+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 7 15 2024-02-18T00:45:32.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 10 client.31995.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:32.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign -1 16 2024-02-18T00:45:32.720 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 20 client.31995.objecter _get_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 15 client.31995.objecter _session_command_op_remove -1 16 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 20 client.31995.objecter get_session s=0x7f018c12e370 osd=7 3 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 15 client.31995.objecter _session_command_op_assign 7 16 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c12e370 osd=7 4 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 10 client.31995.objecter _send_command 16 2024-02-18T00:45:32.721 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.720+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f018c12bb50 con 0x7f018c12d6e0 2024-02-18T00:45:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f017b7fe700 1 -- 172.21.15.59:0/2472702306 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f018c12bb50 con 0x7f018c12d6e0 2024-02-18T00:45:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f017b7fe700 10 client.31995.objecter ms_dispatch 0x7f018c001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f017b7fe700 10 client.31995.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f017b7fe700 20 client.31995.objecter put_session s=0x7f018c12e370 osd=7 4 2024-02-18T00:45:32.722 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f017b7fe700 15 client.31995.objecter _session_command_op_remove 7 16 2024-02-18T00:45:32.723 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:45:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 librados: watch_flush enter 2024-02-18T00:45:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 librados: watch_flush exit 2024-02-18T00:45:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.0 2024-02-18T00:45:32.724 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6801/3700238977 conn(0x7f018c0fc1e0 msgr2=0x7f018c0fc5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6801/3700238977 conn(0x7f018c0fc1e0 0x7f018c0fc5c0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c05ee80 osd=0 1 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.1 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6805/26279110 conn(0x7f018c0598f0 msgr2=0x7f018c059d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6805/26279110 conn(0x7f018c0598f0 0x7f018c059d30 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c028710 osd=1 1 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.2 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6809/1153193637 conn(0x7f018c028270 msgr2=0x7f018c11c1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6809/1153193637 conn(0x7f018c028270 0x7f018c11c1a0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c059770 osd=2 1 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.3 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6813/1805700964 conn(0x7f018c11f900 msgr2=0x7f018c121cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f018ad9d700 1 -- 172.21.15.59:0/2472702306 reap_dead start 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6813/1805700964 conn(0x7f018c11f900 0x7f018c121cf0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c027be0 osd=3 1 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.4 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6800/2044387056 conn(0x7f018c11cc90 msgr2=0x7f018c123360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6800/2044387056 conn(0x7f018c11cc90 0x7f018c123360 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11caf0 osd=4 1 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.5 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6804/2170432263 conn(0x7f018c123d60 msgr2=0x7f018c1241c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6804/2170432263 conn(0x7f018c123d60 0x7f018c1241c0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c11bee0 osd=5 1 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.6 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6808/3356009390 conn(0x7f018c1291b0 msgr2=0x7f018c12b5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6808/3356009390 conn(0x7f018c1291b0 0x7f018c12b5a0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c128ad0 osd=6 1 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter close_session for osd.7 2024-02-18T00:45:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6812/53025171 conn(0x7f018c12d6e0 msgr2=0x7f018c12db60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.137:6812/53025171 conn(0x7f018c12d6e0 0x7f018c12db60 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter put_session s=0x7f018c12e370 osd=7 1 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 20 client.31995.objecter shutdown clearing up homeless session... 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 10 client.31995.objecter successfully canceled tick 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f018ad9d700 1 -- 172.21.15.59:0/2472702306 reap_dead start 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0174062c90 msgr2=0x7f0174065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0174062c90 0x7f0174065150 secure :-1 s=READY pgs=3229 cs=0 l=1 rev1=1 rx=0x7f018c0f8400 tx=0x7f018000b040).stop 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 msgr2=0x7f018c0f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f6c60 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 rx=0x7f017c009b10 tx=0x7f017c009cb0).stop 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 shutdown_connections 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0174062c90 0x7f0174065150 unknown :-1 s=CLOSED pgs=3229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 --2- 172.21.15.59:0/2472702306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c0f8cb0 0x7f018c0f6c60 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.722+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 >> 172.21.15.59:0/2472702306 conn(0x7f018c0f2820 msgr2=0x7f018c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.723+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 shutdown_connections 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.723+0000 7f0190caa700 1 -- 172.21.15.59:0/2472702306 wait complete. 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.723+0000 7f0190caa700 1 librados: shutdown 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:45:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:45:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 -- 172.21.15.59:0/3894376902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20380f8d10 msgr2=0x7f20380f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 --2- 172.21.15.59:0/3894376902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20380f8d10 0x7f20380f90f0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 rx=0x7f2020004870 tx=0x7f2020009cb0).stop 2024-02-18T00:45:33.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 -- 172.21.15.59:0/3894376902 shutdown_connections 2024-02-18T00:45:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 --2- 172.21.15.59:0/3894376902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2038102270 0x7f2038104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 --2- 172.21.15.59:0/3894376902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20380f8d10 0x7f20380f90f0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 --2- 172.21.15.59:0/3894376902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380f9670 0x7f2038101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 -- 172.21.15.59:0/3894376902 >> 172.21.15.59:0/3894376902 conn(0x7f20380f2820 msgr2=0x7f20380f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:33.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 -- 172.21.15.59:0/3894376902 shutdown_connections 2024-02-18T00:45:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 -- 172.21.15.59:0/3894376902 wait complete. 2024-02-18T00:45:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 librados: starting msgr at 2024-02-18T00:45:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.996+0000 7f203cf64700 1 librados: starting objecter 2024-02-18T00:45:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 Processor -- start 2024-02-18T00:45:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 -- start start 2024-02-18T00:45:33.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 librados: setting wanted keys 2024-02-18T00:45:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 librados: calling monclient init 2024-02-18T00:45:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380f8d10 0x7f20380f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20380f9670 0x7f20380f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2038102270 0x7f20380f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2038116640 con 0x7f2038102270 2024-02-18T00:45:33.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20381167b0 con 0x7f20380f8d10 2024-02-18T00:45:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203cf64700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2038116970 con 0x7f20380f9670 2024-02-18T00:45:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2035d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20380f9670 0x7f20380f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2036d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2038102270 0x7f20380f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2035d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20380f9670 0x7f20380f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:47440/0 (socket says 172.21.15.59:47440) 2024-02-18T00:45:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2035d9b700 1 -- 172.21.15.59:0/2138950133 learned_addr learned my addr 172.21.15.59:0/2138950133 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:33.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f203659c700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380f8d10 0x7f20380f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2036d9d700 1 -- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20380f9670 msgr2=0x7f20380f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2036d9d700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20380f9670 0x7f20380f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2036d9d700 1 -- 172.21.15.59:0/2138950133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380f8d10 msgr2=0x7f20380f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2036d9d700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380f8d10 0x7f20380f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2036d9d700 1 -- 172.21.15.59:0/2138950133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20300097c0 con 0x7f2038102270 2024-02-18T00:45:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2035d9b700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20380f9670 0x7f20380f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:33.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f2036d9d700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2038102270 0x7f20380f76a0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 rx=0x7f2038061e90 tx=0x7f2030004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f202f7fe700 10 client.?.objecter ms_handle_connect 0x7f2038102270 2024-02-18T00:45:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.997+0000 7f202f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.998+0000 7f202f7fe700 1 -- 172.21.15.59:0/2138950133 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20300079b0 con 0x7f2038102270 2024-02-18T00:45:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.998+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2020005040 con 0x7f2038102270 2024-02-18T00:45:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.998+0000 7f202f7fe700 1 -- 172.21.15.59:0/2138950133 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20300090d0 con 0x7f2038102270 2024-02-18T00:45:33.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.998+0000 7f202f7fe700 1 -- 172.21.15.59:0/2138950133 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000fd40 con 0x7f2038102270 2024-02-18T00:45:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.998+0000 7f203cf64700 10 client.32001.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.998+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2038116fb0 con 0x7f2038102270 2024-02-18T00:45:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.998+0000 7f203cf64700 1 librados: init done 2024-02-18T00:45:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.999+0000 7f202f7fe700 1 -- 172.21.15.59:0/2138950133 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f2030009240 con 0x7f2038102270 2024-02-18T00:45:33.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:32.999+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203805fa60 con 0x7f2038102270 2024-02-18T00:45:33.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f202f7fe700 1 --2- 172.21.15.59:0/2138950133 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2024062c90 0x7f2024065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:33.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f203659c700 1 --2- 172.21.15.59:0/2138950133 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2024062c90 0x7f2024065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:33.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f202f7fe700 1 -- 172.21.15.59:0/2138950133 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(768..768 src has 418..768) v4 ==== 8433+0+0 (secure 0 0 0) 0x7f203007dbe0 con 0x7f2038102270 2024-02-18T00:45:33.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f202f7fe700 10 client.32001.objecter ms_dispatch 0x7f2038001550 osd_map(768..768 src has 418..768) v4 2024-02-18T00:45:33.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f202f7fe700 3 client.32001.objecter handle_osd_map got epochs [768,768] > 0 2024-02-18T00:45:33.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f202f7fe700 3 client.32001.objecter handle_osd_map decoding full epoch 768 2024-02-18T00:45:33.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f202f7fe700 20 client.32001.objecter dump_active .. 0 homeless 2024-02-18T00:45:33.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f202f7fe700 10 client.32001.objecter ms_handle_connect 0x7f2024062c90 2024-02-18T00:45:33.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.000+0000 7f203659c700 1 --2- 172.21.15.59:0/2138950133 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2024062c90 0x7f2024065150 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 rx=0x7f20200043d0 tx=0x7f202000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:33.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.004+0000 7f202f7fe700 1 -- 172.21.15.59:0/2138950133 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2030050f30 con 0x7f2038102270 2024-02-18T00:45:33.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:33.197+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd out", "ids": ["6"]} v 0) v1 -- 0x7f20381072a0 con 0x7f2038102270 2024-02-18T00:45:33.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:31.621046+0000 mgr.y (mgr.24454) 664 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:33.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:31.817970+0000 osd.6 (osd.6) 341 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:45:33.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:31.818946+0000 osd.6 (osd.6) 342 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:45:33.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:32.020581+0000 mgr.y (mgr.24454) 665 : cluster [DBG] pgmap v1181: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:33.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:32.113734+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-02-18T00:45:33.389 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:32.236895+0000 osd.0 (osd.0) 334 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:45:33.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:32.237905+0000 osd.0 (osd.0) 335 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:45:33.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:32.572625+0000 mon.a (mon.0) 3637 : audit [DBG] from='client.? 172.21.15.59:0/2472702306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:33.390 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:45:33.198+0000 7f6151a5e700 -1 mon.a@0(leader).osd e769 definitely_dead 0 2024-02-18T00:45:33.390 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:31.621046+0000 mgr.y (mgr.24454) 664 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:33.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:31.817970+0000 osd.6 (osd.6) 341 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:45:33.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:31.818946+0000 osd.6 (osd.6) 342 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:45:33.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:32.020581+0000 mgr.y (mgr.24454) 665 : cluster [DBG] pgmap v1181: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:33.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:32.113734+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-02-18T00:45:33.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:32.236895+0000 osd.0 (osd.0) 334 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:45:33.391 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:32.237905+0000 osd.0 (osd.0) 335 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:45:33.392 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:32.572625+0000 mon.a (mon.0) 3637 : audit [DBG] from='client.? 172.21.15.59:0/2472702306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:33.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:31.621046+0000 mgr.y (mgr.24454) 664 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:31.817970+0000 osd.6 (osd.6) 341 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:45:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:31.818946+0000 osd.6 (osd.6) 342 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:45:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:32.020581+0000 mgr.y (mgr.24454) 665 : cluster [DBG] pgmap v1181: 214 pgs: 214 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:32.113734+0000 mon.a (mon.0) 3636 : cluster [DBG] osdmap e768: 8 total, 8 up, 8 in 2024-02-18T00:45:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:32.236895+0000 osd.0 (osd.0) 334 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:45:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:32.237905+0000 osd.0 (osd.0) 335 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:45:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:32.572625+0000 mon.a (mon.0) 3637 : audit [DBG] from='client.? 172.21.15.59:0/2472702306' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:34.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.123+0000 7f202f7fe700 1 -- 172.21.15.59:0/2138950133 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["6"]}]=0 marked out osd.6. v770) v1 ==== 87+0+0 (secure 0 0 0) 0x7f203000fea0 con 0x7f2038102270 2024-02-18T00:45:34.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 10 librados: watch_flush enter 2024-02-18T00:45:34.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 10 librados: watch_flush exit 2024-02-18T00:45:34.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 20 client.32001.objecter shutdown clearing up homeless session... 2024-02-18T00:45:34.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 10 client.32001.objecter successfully canceled tick 2024-02-18T00:45:34.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2024062c90 msgr2=0x7f2024065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:34.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 --2- 172.21.15.59:0/2138950133 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2024062c90 0x7f2024065150 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 rx=0x7f20200043d0 tx=0x7f202000b040).stop 2024-02-18T00:45:34.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2038102270 msgr2=0x7f20380f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2038102270 0x7f20380f76a0 secure :-1 s=READY pgs=2699 cs=0 l=1 rev1=1 rx=0x7f2038061e90 tx=0x7f2030004c50).stop 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 shutdown_connections 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 --2- 172.21.15.59:0/2138950133 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2024062c90 0x7f2024065150 unknown :-1 s=CLOSED pgs=3230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20380f8d10 0x7f20380f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2038102270 0x7f20380f76a0 unknown :-1 s=CLOSED pgs=2699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 --2- 172.21.15.59:0/2138950133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20380f9670 0x7f20380f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 >> 172.21.15.59:0/2138950133 conn(0x7f20380f2820 msgr2=0x7f20380f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 shutdown_connections 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.131+0000 7f203cf64700 1 -- 172.21.15.59:0/2138950133 wait complete. 2024-02-18T00:45:34.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.132+0000 7f203cf64700 1 librados: shutdown 2024-02-18T00:45:34.134 INFO:teuthology.orchestra.run.smithi059.stderr:marked out osd.6. 2024-02-18T00:45:34.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:32.628499+0000 osd.5 (osd.5) 331 : cluster [DBG] 25.c scrub starts 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:32.631394+0000 osd.5 (osd.5) 332 : cluster [DBG] 25.c scrub ok 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.078657+0000 osd.7 (osd.7) 229 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.079911+0000 osd.7 (osd.7) 230 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.118513+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:33.120077+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:33.198236+0000 mon.a (mon.0) 3640 : audit [INF] from='client.? 172.21.15.59:0/2138950133' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.198318+0000 mon.a (mon.0) 3641 : cluster [INF] Client client.admin marked osd.6 out, while it was still marked up 2024-02-18T00:45:34.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.296968+0000 osd.3 (osd.3) 390 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.297910+0000 osd.3 (osd.3) 391 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.653269+0000 osd.2 (osd.2) 305 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.654023+0000 osd.2 (osd.2) 306 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:32.628499+0000 osd.5 (osd.5) 331 : cluster [DBG] 25.c scrub starts 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:32.631394+0000 osd.5 (osd.5) 332 : cluster [DBG] 25.c scrub ok 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.078657+0000 osd.7 (osd.7) 229 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.079911+0000 osd.7 (osd.7) 230 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.118513+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:33.120077+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:33.198236+0000 mon.a (mon.0) 3640 : audit [INF] from='client.? 172.21.15.59:0/2138950133' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.198318+0000 mon.a (mon.0) 3641 : cluster [INF] Client client.admin marked osd.6 out, while it was still marked up 2024-02-18T00:45:34.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.296968+0000 osd.3 (osd.3) 390 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:45:34.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.297910+0000 osd.3 (osd.3) 391 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:45:34.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.653269+0000 osd.2 (osd.2) 305 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:45:34.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.654023+0000 osd.2 (osd.2) 306 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:32.628499+0000 osd.5 (osd.5) 331 : cluster [DBG] 25.c scrub starts 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:32.631394+0000 osd.5 (osd.5) 332 : cluster [DBG] 25.c scrub ok 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.078657+0000 osd.7 (osd.7) 229 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.079911+0000 osd.7 (osd.7) 230 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.118513+0000 mon.a (mon.0) 3638 : cluster [DBG] osdmap e769: 8 total, 8 up, 8 in 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:33.120077+0000 mon.a (mon.0) 3639 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:33.198236+0000 mon.a (mon.0) 3640 : audit [INF] from='client.? 172.21.15.59:0/2138950133' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.198318+0000 mon.a (mon.0) 3641 : cluster [INF] Client client.admin marked osd.6 out, while it was still marked up 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.296968+0000 osd.3 (osd.3) 390 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.297910+0000 osd.3 (osd.3) 391 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:45:34.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.653269+0000 osd.2 (osd.2) 305 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:45:34.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.654023+0000 osd.2 (osd.2) 306 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.723+0000 7f1d77bf6700 1 -- 172.21.15.59:0/3146911199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 msgr2=0x7f1d70103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.723+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/3146911199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 0x7f1d70103a20 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 rx=0x7f1d68009be0 tx=0x7f1d68004d10).stop 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- 172.21.15.59:0/3146911199 shutdown_connections 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/3146911199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d700fcbb0 0x7f1d700fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/3146911199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 0x7f1d70103a20 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/3146911199 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 0x7f1d700f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- 172.21.15.59:0/3146911199 >> 172.21.15.59:0/3146911199 conn(0x7f1d700f2820 msgr2=0x7f1d700f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- 172.21.15.59:0/3146911199 shutdown_connections 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- 172.21.15.59:0/3146911199 wait complete. 2024-02-18T00:45:34.725 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 librados: starting msgr at 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 librados: starting objecter 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 Processor -- start 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- start start 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 librados: setting wanted keys 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 librados: calling monclient init 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 0x7f1d7010e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d700fcbb0 0x7f1d7010e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:34.726 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 0x7f1d701081e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d701086d0 con 0x7f1d700f7400 2024-02-18T00:45:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d70108840 con 0x7f1d700fcbb0 2024-02-18T00:45:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.724+0000 7f1d77bf6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d70108a00 con 0x7f1d700fd510 2024-02-18T00:45:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d75992700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 0x7f1d7010e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 0x7f1d701081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d75992700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 0x7f1d7010e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34136/0 (socket says 172.21.15.59:34136) 2024-02-18T00:45:34.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d75992700 1 -- 172.21.15.59:0/2373986257 learned_addr learned my addr 172.21.15.59:0/2373986257 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 0x7f1d701081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:47464/0 (socket says 172.21.15.59:47464) 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d75191700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d700fcbb0 0x7f1d7010e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 -- 172.21.15.59:0/2373986257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d700fcbb0 msgr2=0x7f1d7010e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d700fcbb0 0x7f1d7010e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 -- 172.21.15.59:0/2373986257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 msgr2=0x7f1d7010e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 0x7f1d7010e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 -- 172.21.15.59:0/2373986257 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60005040 con 0x7f1d700fd510 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d76193700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 0x7f1d701081e0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7f1d6800b6d0 tx=0x7f1d68004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d66ffd700 10 client.?.objecter ms_handle_connect 0x7f1d700fd510 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d66ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d66ffd700 1 -- 172.21.15.59:0/2373986257 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800e030 con 0x7f1d700fd510 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d680097c0 con 0x7f1d700fd510 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d66ffd700 1 -- 172.21.15.59:0/2373986257 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d68007bf0 con 0x7f1d700fd510 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d66ffd700 1 -- 172.21.15.59:0/2373986257 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68018070 con 0x7f1d700fd510 2024-02-18T00:45:34.729 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d77bf6700 10 client.41807.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d70108e90 con 0x7f1d700fd510 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.725+0000 7f1d77bf6700 1 librados: init done 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.726+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7005fa60 con 0x7f1d700fd510 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d66ffd700 1 -- 172.21.15.59:0/2373986257 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1d6801d420 con 0x7f1d700fd510 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d66ffd700 1 --2- 172.21.15.59:0/2373986257 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1d5c062c90 0x7f1d5c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d66ffd700 1 -- 172.21.15.59:0/2373986257 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(770..770 src has 418..770) v4 ==== 8790+0+0 (secure 0 0 0) 0x7f1d6807ec10 con 0x7f1d700fd510 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d66ffd700 10 client.41807.objecter ms_dispatch 0x7f1d70001550 osd_map(770..770 src has 418..770) v4 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d66ffd700 3 client.41807.objecter handle_osd_map got epochs [770,770] > 0 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d66ffd700 3 client.41807.objecter handle_osd_map decoding full epoch 770 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d75992700 1 --2- 172.21.15.59:0/2373986257 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1d5c062c90 0x7f1d5c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.727+0000 7f1d66ffd700 20 client.41807.objecter dump_active .. 0 homeless 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.728+0000 7f1d66ffd700 10 client.41807.objecter ms_handle_connect 0x7f1d5c062c90 2024-02-18T00:45:34.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.728+0000 7f1d75992700 1 --2- 172.21.15.59:0/2373986257 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1d5c062c90 0x7f1d5c065150 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 rx=0x7f1d6c009fd0 tx=0x7f1d6c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:34.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.731+0000 7f1d66ffd700 1 -- 172.21.15.59:0/2373986257 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1d68051e00 con 0x7f1d700fd510 2024-02-18T00:45:34.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:34.916+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7f1d701061c0 con 0x7f1d700fd510 2024-02-18T00:45:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.130+0000 7f1d66ffd700 1 -- 172.21.15.59:0/2373986257 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v771) v1 ==== 101+0+0 (secure 0 0 0) 0x7f1d680519f0 con 0x7f1d700fd510 2024-02-18T00:45:35.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.139+0000 7f1d77bf6700 10 librados: watch_flush enter 2024-02-18T00:45:35.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.141+0000 7f1d77bf6700 10 librados: watch_flush exit 2024-02-18T00:45:35.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.141+0000 7f1d77bf6700 20 client.41807.objecter shutdown clearing up homeless session... 2024-02-18T00:45:35.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.141+0000 7f1d77bf6700 10 client.41807.objecter successfully canceled tick 2024-02-18T00:45:35.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.142+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1d5c062c90 msgr2=0x7f1d5c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:35.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.142+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/2373986257 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1d5c062c90 0x7f1d5c065150 secure :-1 s=READY pgs=3231 cs=0 l=1 rev1=1 rx=0x7f1d6c009fd0 tx=0x7f1d6c00b040).stop 2024-02-18T00:45:35.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.142+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 msgr2=0x7f1d701081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:35.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.142+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 0x7f1d701081e0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 rx=0x7f1d6800b6d0 tx=0x7f1d68004d10).stop 2024-02-18T00:45:35.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.143+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 shutdown_connections 2024-02-18T00:45:35.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.143+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1d700fd510 0x7f1d701081e0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:35.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.143+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1d700f7400 0x7f1d7010e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:35.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.144+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/2373986257 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1d5c062c90 0x7f1d5c065150 unknown :-1 s=CLOSED pgs=3231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:35.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.144+0000 7f1d77bf6700 1 --2- 172.21.15.59:0/2373986257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d700fcbb0 0x7f1d7010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:35.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.144+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 >> 172.21.15.59:0/2373986257 conn(0x7f1d700f2820 msgr2=0x7f1d70060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:35.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.148+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 shutdown_connections 2024-02-18T00:45:35.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.150+0000 7f1d77bf6700 1 -- 172.21.15.59:0/2373986257 wait complete. 2024-02-18T00:45:35.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:35.151+0000 7f1d77bf6700 1 librados: shutdown 2024-02-18T00:45:35.156 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:45:35.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.605338+0000 osd.5 (osd.5) 333 : cluster [DBG] 4.f scrub starts 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.608937+0000 osd.5 (osd.5) 334 : cluster [DBG] 4.f scrub ok 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.950590+0000 osd.1 (osd.1) 363 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:33.952071+0000 osd.1 (osd.1) 364 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:34.021183+0000 mgr.y (mgr.24454) 666 : cluster [DBG] pgmap v1184: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:34.072396+0000 osd.7 (osd.7) 231 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:34.073689+0000 osd.7 (osd.7) 232 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:34.123146+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:34.123361+0000 mon.a (mon.0) 3643 : audit [INF] from='client.? 172.21.15.59:0/2138950133' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:34.123441+0000 mon.a (mon.0) 3644 : cluster [DBG] osdmap e770: 8 total, 8 up, 7 in 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:34.916637+0000 mon.c (mon.2) 576 : audit [INF] from='client.? 172.21.15.59:0/2373986257' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:34.916963+0000 mon.a (mon.0) 3645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:35.116410+0000 mon.a (mon.0) 3646 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:35.125354+0000 mon.a (mon.0) 3647 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:35.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:35.125425+0000 mon.a (mon.0) 3648 : cluster [DBG] osdmap e771: 8 total, 8 up, 7 in 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.605338+0000 osd.5 (osd.5) 333 : cluster [DBG] 4.f scrub starts 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.608937+0000 osd.5 (osd.5) 334 : cluster [DBG] 4.f scrub ok 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.950590+0000 osd.1 (osd.1) 363 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:33.952071+0000 osd.1 (osd.1) 364 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:34.021183+0000 mgr.y (mgr.24454) 666 : cluster [DBG] pgmap v1184: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:34.072396+0000 osd.7 (osd.7) 231 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:34.073689+0000 osd.7 (osd.7) 232 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:34.123146+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:34.123361+0000 mon.a (mon.0) 3643 : audit [INF] from='client.? 172.21.15.59:0/2138950133' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-02-18T00:45:35.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:34.123441+0000 mon.a (mon.0) 3644 : cluster [DBG] osdmap e770: 8 total, 8 up, 7 in 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:34.916637+0000 mon.c (mon.2) 576 : audit [INF] from='client.? 172.21.15.59:0/2373986257' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:34.916963+0000 mon.a (mon.0) 3645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:35.116410+0000 mon.a (mon.0) 3646 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:35.125354+0000 mon.a (mon.0) 3647 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:35.125425+0000 mon.a (mon.0) 3648 : cluster [DBG] osdmap e771: 8 total, 8 up, 7 in 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.605338+0000 osd.5 (osd.5) 333 : cluster [DBG] 4.f scrub starts 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.608937+0000 osd.5 (osd.5) 334 : cluster [DBG] 4.f scrub ok 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.950590+0000 osd.1 (osd.1) 363 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:33.952071+0000 osd.1 (osd.1) 364 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:45:35.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:34.021183+0000 mgr.y (mgr.24454) 666 : cluster [DBG] pgmap v1184: 214 pgs: 32 unknown, 182 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:34.072396+0000 osd.7 (osd.7) 231 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:34.073689+0000 osd.7 (osd.7) 232 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:34.123146+0000 mon.a (mon.0) 3642 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:34.123361+0000 mon.a (mon.0) 3643 : audit [INF] from='client.? 172.21.15.59:0/2138950133' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:34.123441+0000 mon.a (mon.0) 3644 : cluster [DBG] osdmap e770: 8 total, 8 up, 7 in 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:34.916637+0000 mon.c (mon.2) 576 : audit [INF] from='client.? 172.21.15.59:0/2373986257' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:34.916963+0000 mon.a (mon.0) 3645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:35.116410+0000 mon.a (mon.0) 3646 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:35.125354+0000 mon.a (mon.0) 3647 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:35.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:35.125425+0000 mon.a (mon.0) 3648 : cluster [DBG] osdmap e771: 8 total, 8 up, 7 in 2024-02-18T00:45:35.608 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:35.612 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:35.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:35.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:35.746 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:35.940 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:45:36.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:34.979436+0000 osd.4 (osd.4) 407 : cluster [DBG] 5.e scrub starts 2024-02-18T00:45:36.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:34.982920+0000 osd.4 (osd.4) 408 : cluster [DBG] 5.e scrub ok 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:35.181104+0000 mon.a (mon.0) 3649 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:36.126716+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:36.126779+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e772: 8 total, 8 up, 7 in 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:36.129784+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:34.979436+0000 osd.4 (osd.4) 407 : cluster [DBG] 5.e scrub starts 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:34.982920+0000 osd.4 (osd.4) 408 : cluster [DBG] 5.e scrub ok 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:35.181104+0000 mon.a (mon.0) 3649 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:36.126716+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:36.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:36.126779+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e772: 8 total, 8 up, 7 in 2024-02-18T00:45:36.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:36.129784+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:36.295 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:36.298 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:36.298 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:36.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:36.426 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:36.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:34.979436+0000 osd.4 (osd.4) 407 : cluster [DBG] 5.e scrub starts 2024-02-18T00:45:36.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:34.982920+0000 osd.4 (osd.4) 408 : cluster [DBG] 5.e scrub ok 2024-02-18T00:45:36.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:35.181104+0000 mon.a (mon.0) 3649 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:36.126716+0000 mon.a (mon.0) 3650 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:36.126779+0000 mon.a (mon.0) 3651 : cluster [DBG] osdmap e772: 8 total, 8 up, 7 in 2024-02-18T00:45:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:36.129784+0000 mon.a (mon.0) 3652 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:36.530 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:36.533 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:36.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:36.634 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:36.663 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:36.683 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:36.686 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:36.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:36.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:36.815 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:36.835 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:36.837 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:36.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:36.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:36.967 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:37.024 INFO:teuthology.orchestra.run.smithi059.stdout:ERROR: (22) Invalid argument 2024-02-18T00:45:37.024 INFO:teuthology.orchestra.run.smithi059.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:36.021999+0000 mgr.y (mgr.24454) 667 : cluster [DBG] pgmap v1187: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 12 KiB/s rd, 1.2 KiB/s wr, 24 op/s; 8/804 objects misplaced (0.995%); 977 KiB/s, 25 objects/s recovering 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:36.128493+0000 osd.3 (osd.3) 392 : cluster [DBG] 6.c starting backfill to osd.7 from (0'0,0'0] MAX to 64'1 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:36.715516+0000 osd.2 (osd.2) 307 : cluster [DBG] 2.c scrub starts 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:36.716669+0000 osd.2 (osd.2) 308 : cluster [DBG] 2.c scrub ok 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:37.126470+0000 mon.a (mon.0) 3653 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-136"}]': finished 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:37.126541+0000 mon.a (mon.0) 3654 : cluster [DBG] osdmap e773: 8 total, 8 up, 7 in 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:37.129080+0000 mon.a (mon.0) 3655 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-136", "mode": "writeback"}]: dispatch 2024-02-18T00:45:37.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:36.021999+0000 mgr.y (mgr.24454) 667 : cluster [DBG] pgmap v1187: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 12 KiB/s rd, 1.2 KiB/s wr, 24 op/s; 8/804 objects misplaced (0.995%); 977 KiB/s, 25 objects/s recovering 2024-02-18T00:45:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:36.128493+0000 osd.3 (osd.3) 392 : cluster [DBG] 6.c starting backfill to osd.7 from (0'0,0'0] MAX to 64'1 2024-02-18T00:45:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:36.715516+0000 osd.2 (osd.2) 307 : cluster [DBG] 2.c scrub starts 2024-02-18T00:45:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:36.716669+0000 osd.2 (osd.2) 308 : cluster [DBG] 2.c scrub ok 2024-02-18T00:45:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:37.126470+0000 mon.a (mon.0) 3653 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-136"}]': finished 2024-02-18T00:45:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:37.126541+0000 mon.a (mon.0) 3654 : cluster [DBG] osdmap e773: 8 total, 8 up, 7 in 2024-02-18T00:45:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:37.129080+0000 mon.a (mon.0) 3655 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-136", "mode": "writeback"}]: dispatch 2024-02-18T00:45:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:36.021999+0000 mgr.y (mgr.24454) 667 : cluster [DBG] pgmap v1187: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 12 KiB/s rd, 1.2 KiB/s wr, 24 op/s; 8/804 objects misplaced (0.995%); 977 KiB/s, 25 objects/s recovering 2024-02-18T00:45:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:36.128493+0000 osd.3 (osd.3) 392 : cluster [DBG] 6.c starting backfill to osd.7 from (0'0,0'0] MAX to 64'1 2024-02-18T00:45:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:36.715516+0000 osd.2 (osd.2) 307 : cluster [DBG] 2.c scrub starts 2024-02-18T00:45:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:36.716669+0000 osd.2 (osd.2) 308 : cluster [DBG] 2.c scrub ok 2024-02-18T00:45:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:37.126470+0000 mon.a (mon.0) 3653 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi059-82957-101", "overlaypool": "test-rados-api-smithi059-82957-136"}]': finished 2024-02-18T00:45:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:37.126541+0000 mon.a (mon.0) 3654 : cluster [DBG] osdmap e773: 8 total, 8 up, 7 in 2024-02-18T00:45:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:37.129080+0000 mon.a (mon.0) 3655 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-136", "mode": "writeback"}]: dispatch 2024-02-18T00:45:38.318 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:38.324 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:36.545030+0000 osd.5 (osd.5) 335 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:45:38.324 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:36.546436+0000 osd.5 (osd.5) 336 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:45:38.324 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:37.273244+0000 osd.3 (osd.3) 393 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:45:38.324 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:37.274293+0000 osd.3 (osd.3) 394 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:45:38.324 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.123481+0000 mon.a (mon.0) 3656 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:38.324 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:38.131442+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-136", "mode": "writeback"}]': finished 2024-02-18T00:45:38.324 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.131517+0000 mon.a (mon.0) 3658 : cluster [DBG] osdmap e774: 8 total, 8 up, 7 in 2024-02-18T00:45:38.325 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:36.545030+0000 osd.5 (osd.5) 335 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:45:38.325 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:36.546436+0000 osd.5 (osd.5) 336 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:45:38.325 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:37.273244+0000 osd.3 (osd.3) 393 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:45:38.325 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:37.274293+0000 osd.3 (osd.3) 394 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:45:38.325 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.123481+0000 mon.a (mon.0) 3656 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:38.325 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:38.131442+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-136", "mode": "writeback"}]': finished 2024-02-18T00:45:38.325 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.131517+0000 mon.a (mon.0) 3658 : cluster [DBG] osdmap e774: 8 total, 8 up, 7 in 2024-02-18T00:45:38.325 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:38.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:38.427 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:38.458 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:36.545030+0000 osd.5 (osd.5) 335 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:45:38.458 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:36.546436+0000 osd.5 (osd.5) 336 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:45:38.458 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:37.273244+0000 osd.3 (osd.3) 393 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:45:38.458 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:37.274293+0000 osd.3 (osd.3) 394 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:45:38.458 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.123481+0000 mon.a (mon.0) 3656 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T00:45:38.458 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:38.131442+0000 mon.a (mon.0) 3657 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi059-82957-136", "mode": "writeback"}]': finished 2024-02-18T00:45:38.458 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.131517+0000 mon.a (mon.0) 3658 : cluster [DBG] osdmap e774: 8 total, 8 up, 7 in 2024-02-18T00:45:38.460 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:38.482 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:38.485 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:38.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:38.586 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:38.616 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:38.636 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:38.638 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:38.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:38.740 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:38.777 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:38.877 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:38.877 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:38.881 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:38.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:38.901 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_ops_in_flight 2024-02-18T00:45:38.983 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:39.027 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:39.029 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:39.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:39.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:39.163 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:39.173 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:39.186 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:39.189 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:39.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.022503+0000 mgr.y (mgr.24454) 668 : cluster [DBG] pgmap v1190: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 12 KiB/s rd, 1.2 KiB/s wr, 24 op/s; 8/804 objects misplaced (0.995%); 977 KiB/s, 25 objects/s recovering 2024-02-18T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.051306+0000 osd.7 (osd.7) 233 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.052297+0000 osd.7 (osd.7) 234 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.157763+0000 osd.0 (osd.0) 336 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.159073+0000 osd.0 (osd.0) 337 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:38.173064+0000 mon.a (mon.0) 3659 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:45:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.321451+0000 osd.3 (osd.3) 395 : cluster [DBG] 2.19 deep-scrub starts 2024-02-18T00:45:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.322784+0000 osd.3 (osd.3) 396 : cluster [DBG] 2.19 deep-scrub ok 2024-02-18T00:45:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.501410+0000 osd.5 (osd.5) 337 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:45:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.502374+0000 osd.5 (osd.5) 338 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:45:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:38.828298+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:45:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:38.828393+0000 mon.a (mon.0) 3661 : cluster [DBG] osdmap e775: 8 total, 8 up, 7 in 2024-02-18T00:45:39.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:38.830771+0000 mon.a (mon.0) 3662 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:45:39.290 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:39.415 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:39.417 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:39.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:39.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.022503+0000 mgr.y (mgr.24454) 668 : cluster [DBG] pgmap v1190: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 12 KiB/s rd, 1.2 KiB/s wr, 24 op/s; 8/804 objects misplaced (0.995%); 977 KiB/s, 25 objects/s recovering 2024-02-18T00:45:39.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.051306+0000 osd.7 (osd.7) 233 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:45:39.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.052297+0000 osd.7 (osd.7) 234 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:45:39.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.157763+0000 osd.0 (osd.0) 336 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.159073+0000 osd.0 (osd.0) 337 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:38.173064+0000 mon.a (mon.0) 3659 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.321451+0000 osd.3 (osd.3) 395 : cluster [DBG] 2.19 deep-scrub starts 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.322784+0000 osd.3 (osd.3) 396 : cluster [DBG] 2.19 deep-scrub ok 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.501410+0000 osd.5 (osd.5) 337 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.502374+0000 osd.5 (osd.5) 338 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:38.828298+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:38.828393+0000 mon.a (mon.0) 3661 : cluster [DBG] osdmap e775: 8 total, 8 up, 7 in 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:38.830771+0000 mon.a (mon.0) 3662 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.022503+0000 mgr.y (mgr.24454) 668 : cluster [DBG] pgmap v1190: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 12 KiB/s rd, 1.2 KiB/s wr, 24 op/s; 8/804 objects misplaced (0.995%); 977 KiB/s, 25 objects/s recovering 2024-02-18T00:45:39.481 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.051306+0000 osd.7 (osd.7) 233 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.052297+0000 osd.7 (osd.7) 234 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.157763+0000 osd.0 (osd.0) 336 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.159073+0000 osd.0 (osd.0) 337 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:38.173064+0000 mon.a (mon.0) 3659 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.321451+0000 osd.3 (osd.3) 395 : cluster [DBG] 2.19 deep-scrub starts 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.322784+0000 osd.3 (osd.3) 396 : cluster [DBG] 2.19 deep-scrub ok 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.501410+0000 osd.5 (osd.5) 337 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.502374+0000 osd.5 (osd.5) 338 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:38.828298+0000 mon.a (mon.0) 3660 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_count","val": "2"}]': finished 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:38.828393+0000 mon.a (mon.0) 3661 : cluster [DBG] osdmap e775: 8 total, 8 up, 7 in 2024-02-18T00:45:39.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:38.830771+0000 mon.a (mon.0) 3662 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T00:45:39.519 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:39.671 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:39.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:39.674 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:39.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:39.807 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:39.827 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:39.833 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:39.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:39.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:40.346 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set noscrub 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.116162+0000 osd.0 (osd.0) 338 : cluster [DBG] 4.16 deep-scrub starts 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.120110+0000 osd.0 (osd.0) 339 : cluster [DBG] 4.16 deep-scrub ok 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.336235+0000 osd.3 (osd.3) 397 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.337342+0000 osd.3 (osd.3) 398 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.681880+0000 osd.2 (osd.2) 309 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.682680+0000 osd.2 (osd.2) 310 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:39.829304+0000 mon.a (mon.0) 3663 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.829380+0000 mon.a (mon.0) 3664 : cluster [DBG] osdmap e776: 8 total, 8 up, 7 in 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:39.830440+0000 mon.a (mon.0) 3665 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:45:40.468 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:39.909816+0000 mon.a (mon.0) 3666 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.116162+0000 osd.0 (osd.0) 338 : cluster [DBG] 4.16 deep-scrub starts 2024-02-18T00:45:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.120110+0000 osd.0 (osd.0) 339 : cluster [DBG] 4.16 deep-scrub ok 2024-02-18T00:45:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.336235+0000 osd.3 (osd.3) 397 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.337342+0000 osd.3 (osd.3) 398 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.681880+0000 osd.2 (osd.2) 309 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.682680+0000 osd.2 (osd.2) 310 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:39.829304+0000 mon.a (mon.0) 3663 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.829380+0000 mon.a (mon.0) 3664 : cluster [DBG] osdmap e776: 8 total, 8 up, 7 in 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:39.830440+0000 mon.a (mon.0) 3665 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:39.909816+0000 mon.a (mon.0) 3666 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.116162+0000 osd.0 (osd.0) 338 : cluster [DBG] 4.16 deep-scrub starts 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.120110+0000 osd.0 (osd.0) 339 : cluster [DBG] 4.16 deep-scrub ok 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.336235+0000 osd.3 (osd.3) 397 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:45:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.337342+0000 osd.3 (osd.3) 398 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:45:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.681880+0000 osd.2 (osd.2) 309 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:45:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.682680+0000 osd.2 (osd.2) 310 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:45:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:39.829304+0000 mon.a (mon.0) 3663 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_period","val": "600"}]': finished 2024-02-18T00:45:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.829380+0000 mon.a (mon.0) 3664 : cluster [DBG] osdmap e776: 8 total, 8 up, 7 in 2024-02-18T00:45:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:39.830440+0000 mon.a (mon.0) 3665 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T00:45:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:39.909816+0000 mon.a (mon.0) 3666 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:40.965 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:45:40.965 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.111+0000 7fa474cad700 1 -- 172.21.15.59:0/3286135540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700f9670 msgr2=0x7fa470101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.111+0000 7fa474cad700 1 --2- 172.21.15.59:0/3286135540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700f9670 0x7fa470101d80 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 rx=0x7fa4580047b0 tx=0x7fa458009cb0).stop 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 -- 172.21.15.59:0/3286135540 shutdown_connections 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 --2- 172.21.15.59:0/3286135540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa470102270 0x7fa470104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 --2- 172.21.15.59:0/3286135540 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4700f8d10 0x7fa4700f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 --2- 172.21.15.59:0/3286135540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4700f9670 0x7fa470101d80 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 -- 172.21.15.59:0/3286135540 >> 172.21.15.59:0/3286135540 conn(0x7fa4700f2820 msgr2=0x7fa4700f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 -- 172.21.15.59:0/3286135540 shutdown_connections 2024-02-18T00:45:41.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 -- 172.21.15.59:0/3286135540 wait complete. 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 librados: starting msgr at 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 librados: starting objecter 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 Processor -- start 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 -- start start 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 librados: setting wanted keys 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.112+0000 7fa474cad700 1 librados: calling monclient init 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4700f8d10 0x7fa47010e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4700f9670 0x7fa47010e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa470102270 0x7fa4701083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa470108890 con 0x7fa470102270 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa470108a00 con 0x7fa4700f9670 2024-02-18T00:45:41.149 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa470108bc0 con 0x7fa4700f8d10 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4700f8d10 0x7fa47010e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4700f8d10 0x7fa47010e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38536/0 (socket says 172.21.15.59:38536) 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 -- 172.21.15.59:0/2076700142 learned_addr learned my addr 172.21.15.59:0/2076700142 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa470102270 0x7fa4701083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46dd9b700 1 --2- 172.21.15.59:0/2076700142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4700f9670 0x7fa47010e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 -- 172.21.15.59:0/2076700142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4700f9670 msgr2=0x7fa47010e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4700f9670 0x7fa47010e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 -- 172.21.15.59:0/2076700142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa470102270 msgr2=0x7fa4701083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa470102270 0x7fa4701083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 -- 172.21.15.59:0/2076700142 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460005040 con 0x7fa4700f8d10 2024-02-18T00:45:41.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa470102270 0x7fa4701083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4700f8d10 0x7fa47010e1b0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7fa460009400 tx=0x7fa460009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa4677fe700 10 client.?.objecter ms_handle_connect 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa4677fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460007a40 con 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46000f040 con 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460014aa0 con 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa458005040 con 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 10 client.41813.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4701091b0 con 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.113+0000 7fa474cad700 1 librados: init done 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.114+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fa47005fa60 con 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa46000e630 con 0x7fa4700f8d10 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa45c062c90 0x7fa45c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(777..777 src has 418..777) v4 ==== 8844+0+0 (secure 0 0 0) 0x7fa46007d180 con 0x7fa4700f8d10 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 osd_map(777..777 src has 418..777) v4 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 3 client.41813.objecter handle_osd_map got epochs [777,777] > 0 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 3 client.41813.objecter handle_osd_map decoding full epoch 777 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa46dd9b700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa45c062c90 0x7fa45c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 20 client.41813.objecter dump_active .. 0 homeless 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v777) v1 ==== 54+0+16 (secure 0 0 0) 0x7fa460084070 con 0x7fa4700f8d10 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa474cad700 10 client.41813.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 1 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa45c062c90 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa46dd9b700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa45c062c90 0x7fa45c065150 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 rx=0x7fa458005010 tx=0x7fa458009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa4700fc1b0 0x7fa4700fc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.115+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 1 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 0 1 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47005ee80 osd=0 4 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa474cad700 10 client.41813.objecter _send_command 1 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4701072a0 con 0x7fa4700fc1b0 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa4700fc1b0 0x7fa4700fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa4700fc1b0 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa4700fc1b0 0x7fa4700fc590 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa4701072a0 con 0x7fa4700fc1b0 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa4677fe700 10 client.41813.objecter _finish_command 1 = system:0 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa47005ee80 osd=0 4 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.116+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 0 1 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 10 client.41813.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 2 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 2 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa47005ee80 osd=0 3 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 0 2 2024-02-18T00:45:41.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47005ee80 osd=0 4 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 10 client.41813.objecter _send_command 2 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.132+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa4700273c0 con 0x7fa4700fc1b0 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa4700273c0 con 0x7fa4700fc1b0 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa4677fe700 10 client.41813.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa47005ee80 osd=0 4 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 0 2 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 10 client.41813.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 3 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6805/26279110 conn(0x7fa4700598f0 0x7fa470059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 3 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 1 3 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470028710 osd=1 4 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 10 client.41813.objecter _send_command 3 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4700f8b10 con 0x7fa4700598f0 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6805/26279110 conn(0x7fa4700598f0 0x7fa470059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa4700598f0 2024-02-18T00:45:41.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.134+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6805/26279110 conn(0x7fa4700598f0 0x7fa470059cd0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.157 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:45:41.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.135+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa4700f8b10 con 0x7fa4700598f0 2024-02-18T00:45:41.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.135+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:45:41.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.135+0000 7fa4677fe700 10 client.41813.objecter _finish_command 3 = system:0 2024-02-18T00:45:41.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.135+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470028710 osd=1 4 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.135+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 1 3 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 10 client.41813.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 4 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 4 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470028710 osd=1 3 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 1 4 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470028710 osd=1 4 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 10 client.41813.objecter _send_command 4 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.150+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa470059360 con 0x7fa4700598f0 2024-02-18T00:45:41.158 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.151+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa470059360 con 0x7fa4700598f0 2024-02-18T00:45:41.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.151+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.151+0000 7fa4677fe700 10 client.41813.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.151+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470028710 osd=1 4 2024-02-18T00:45:41.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.151+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 1 4 2024-02-18T00:45:41.159 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 10 client.41813.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 5 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa470028270 0x7fa47011e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 5 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 2 5 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470059770 osd=2 4 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 10 client.41813.objecter _send_command 5 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4701211e0 con 0x7fa470028270 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa46dd9b700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa470028270 0x7fa47011e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa470028270 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.152+0000 7fa46dd9b700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa470028270 0x7fa47011e590 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.153+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa4701211e0 con 0x7fa470028270 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.153+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.153+0000 7fa4677fe700 10 client.41813.objecter _finish_command 5 = system:0 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.153+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470059770 osd=2 4 2024-02-18T00:45:41.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.153+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 2 5 2024-02-18T00:45:41.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 10 client.41813.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 6 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 6 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470059770 osd=2 3 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 2 6 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470059770 osd=2 4 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 10 client.41813.objecter _send_command 6 2024-02-18T00:45:41.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.167+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa4700277d0 con 0x7fa470028270 2024-02-18T00:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa4700277d0 con 0x7fa470028270 2024-02-18T00:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa4677fe700 10 client.41813.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470059770 osd=2 4 2024-02-18T00:45:41.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 2 6 2024-02-18T00:45:41.171 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:45:41.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 10 client.41813.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 7 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa47011c0e0 0x7fa47011fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 7 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 3 7 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470027be0 osd=3 4 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 10 client.41813.objecter _send_command 7 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4701201f0 con 0x7fa47011c0e0 2024-02-18T00:45:41.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa47011c0e0 0x7fa47011fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa47011c0e0 2024-02-18T00:45:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.169+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa47011c0e0 0x7fa47011fc00 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.170+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa4701201f0 con 0x7fa47011c0e0 2024-02-18T00:45:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.170+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:45:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.170+0000 7fa4677fe700 10 client.41813.objecter _finish_command 7 = system:0 2024-02-18T00:45:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.170+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470027be0 osd=3 4 2024-02-18T00:45:41.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.170+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 3 7 2024-02-18T00:45:41.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 10 client.41813.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 8 2024-02-18T00:45:41.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 8 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470027be0 osd=3 3 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 3 8 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470027be0 osd=3 4 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 10 client.41813.objecter _send_command 8 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.186+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa47011b640 con 0x7fa47011c0e0 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.187+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa47011b640 con 0x7fa47011c0e0 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.187+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.187+0000 7fa4677fe700 10 client.41813.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.187+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470027be0 osd=3 4 2024-02-18T00:45:41.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.187+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 3 8 2024-02-18T00:45:41.191 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 10 client.41813.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 9 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa470123360 0x7fa470125750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 9 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 4 9 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b400 osd=4 4 2024-02-18T00:45:41.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 10 client.41813.objecter _send_command 9 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa47011b570 con 0x7fa470123360 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa470123360 0x7fa470125750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa470123360 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.188+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa470123360 0x7fa470125750 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.189+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa47011b570 con 0x7fa470123360 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.189+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.189+0000 7fa4677fe700 10 client.41813.objecter _finish_command 9 = system:0 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.189+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa47011b400 osd=4 4 2024-02-18T00:45:41.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.189+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 4 9 2024-02-18T00:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 10 client.41813.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 10 2024-02-18T00:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 10 2024-02-18T00:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa47011b400 osd=4 3 2024-02-18T00:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 4 10 2024-02-18T00:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b400 osd=4 4 2024-02-18T00:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 10 client.41813.objecter _send_command 10 2024-02-18T00:45:41.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.204+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa47011b570 con 0x7fa470123360 2024-02-18T00:45:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa47011b570 con 0x7fa470123360 2024-02-18T00:45:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa4677fe700 10 client.41813.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa47011b400 osd=4 4 2024-02-18T00:45:41.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 4 10 2024-02-18T00:45:41.207 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:45:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 10 client.41813.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 11 2024-02-18T00:45:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa470126150 0x7fa470126590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 11 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 5 11 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b570 osd=5 4 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 10 client.41813.objecter _send_command 11 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa470128150 con 0x7fa470126150 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa46dd9b700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa470126150 0x7fa470126590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa470126150 2024-02-18T00:45:41.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.206+0000 7fa46dd9b700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa470126150 0x7fa470126590 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.207+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa470128150 con 0x7fa470126150 2024-02-18T00:45:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.207+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:45:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.207+0000 7fa4677fe700 10 client.41813.objecter _finish_command 11 = system:0 2024-02-18T00:45:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.207+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa47011b570 osd=5 4 2024-02-18T00:45:41.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.207+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 5 11 2024-02-18T00:45:41.211 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:41.217 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:41.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:41.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 10 client.41813.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 12 2024-02-18T00:45:41.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 12 2024-02-18T00:45:41.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa47011b570 osd=5 3 2024-02-18T00:45:41.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 5 12 2024-02-18T00:45:41.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b570 osd=5 4 2024-02-18T00:45:41.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 10 client.41813.objecter _send_command 12 2024-02-18T00:45:41.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.224+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa470128150 con 0x7fa470126150 2024-02-18T00:45:41.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.225+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa470128150 con 0x7fa470126150 2024-02-18T00:45:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.225+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.225+0000 7fa4677fe700 10 client.41813.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.225+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa47011b570 osd=5 4 2024-02-18T00:45:41.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.225+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 5 12 2024-02-18T00:45:41.229 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:45:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 10 client.41813.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 13 2024-02-18T00:45:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa470127050 0x7fa4701274b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 13 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 6 13 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470128150 osd=6 4 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 10 client.41813.objecter _send_command 13 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa470127b60 con 0x7fa470127050 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa470127050 0x7fa4701274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa470127050 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.226+0000 7fa46ed9d700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa470127050 0x7fa4701274b0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.227+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa470127b60 con 0x7fa470127050 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.227+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:45:41.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.227+0000 7fa4677fe700 10 client.41813.objecter _finish_command 13 = system:0 2024-02-18T00:45:41.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.227+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470128150 osd=6 4 2024-02-18T00:45:41.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.227+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 6 13 2024-02-18T00:45:41.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 10 client.41813.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 14 2024-02-18T00:45:41.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 14 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470128150 osd=6 3 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 6 14 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470128150 osd=6 4 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 10 client.41813.objecter _send_command 14 2024-02-18T00:45:41.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.249+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa470127b20 con 0x7fa470127050 2024-02-18T00:45:41.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa470127b20 con 0x7fa470127050 2024-02-18T00:45:41.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa4677fe700 10 client.41813.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470128150 osd=6 4 2024-02-18T00:45:41.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 6 14 2024-02-18T00:45:41.253 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 10 client.41813.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 15 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6812/53025171 conn(0x7fa470127d30 0x7fa47012e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 15 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 7 15 2024-02-18T00:45:41.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470127b20 osd=7 4 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 10 client.41813.objecter _send_command 15 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa470131790 con 0x7fa470127d30 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6812/53025171 conn(0x7fa470127d30 0x7fa47012e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa46e59c700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6812/53025171 conn(0x7fa470127d30 0x7fa47012e950 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.251+0000 7fa4677fe700 10 client.41813.objecter ms_handle_connect 0x7fa470127d30 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.252+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa470131790 con 0x7fa470127d30 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.252+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.252+0000 7fa4677fe700 10 client.41813.objecter _finish_command 15 = system:0 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.252+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470127b20 osd=7 4 2024-02-18T00:45:41.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.252+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 7 15 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.921346+0000 osd.4 (osd.4) 409 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:39.922246+0000 osd.4 (osd.4) 410 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:40.006759+0000 osd.7 (osd.7) 235 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:40.007599+0000 osd.7 (osd.7) 236 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:40.022974+0000 mgr.y (mgr.24454) 669 : cluster [DBG] pgmap v1194: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8/804 objects misplaced (0.995%) 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:40.130010+0000 osd.1 (osd.1) 365 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:40.130222+0000 osd.1 (osd.1) 366 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:41.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:40.826702+0000 mon.a (mon.0) 3667 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:41.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:40.836055+0000 mon.a (mon.0) 3668 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:45:41.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:40.836131+0000 mon.a (mon.0) 3669 : cluster [DBG] osdmap e777: 8 total, 8 up, 7 in 2024-02-18T00:45:41.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:40.837896+0000 mon.a (mon.0) 3670 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:45:41.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:41.116465+0000 mon.c (mon.2) 577 : audit [DBG] from='client.? 172.21.15.59:0/2076700142' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:41.277 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:41.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 10 client.41813.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:41.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign -1 16 2024-02-18T00:45:41.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.278 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 20 client.41813.objecter _get_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 15 client.41813.objecter _session_command_op_remove -1 16 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 20 client.41813.objecter get_session s=0x7fa470127b20 osd=7 3 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 15 client.41813.objecter _session_command_op_assign 7 16 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470127b20 osd=7 4 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 10 client.41813.objecter _send_command 16 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.276+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7fa470131750 con 0x7fa470127d30 2024-02-18T00:45:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa4677fe700 1 -- 172.21.15.59:0/2076700142 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7fa470131750 con 0x7fa470127d30 2024-02-18T00:45:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa4677fe700 10 client.41813.objecter ms_dispatch 0x7fa470001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa4677fe700 10 client.41813.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa4677fe700 20 client.41813.objecter put_session s=0x7fa470127b20 osd=7 4 2024-02-18T00:45:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa4677fe700 15 client.41813.objecter _session_command_op_remove 7 16 2024-02-18T00:45:41.280 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 librados: watch_flush enter 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 librados: watch_flush exit 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.0 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa4700fc1b0 msgr2=0x7fa4700fc590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6801/3700238977 conn(0x7fa4700fc1b0 0x7fa4700fc590 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47005ee80 osd=0 1 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.1 2024-02-18T00:45:41.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6805/26279110 conn(0x7fa4700598f0 msgr2=0x7fa470059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6805/26279110 conn(0x7fa4700598f0 0x7fa470059cd0 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470028710 osd=1 1 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.2 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa470028270 msgr2=0x7fa47011e590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6809/1153193637 conn(0x7fa470028270 0x7fa47011e590 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470059770 osd=2 1 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.3 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa47011c0e0 msgr2=0x7fa47011fc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6813/1805700964 conn(0x7fa47011c0e0 0x7fa47011fc00 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa46ed9d700 1 -- 172.21.15.59:0/2076700142 reap_dead start 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470027be0 osd=3 1 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.4 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa470123360 msgr2=0x7fa470125750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6800/2044387056 conn(0x7fa470123360 0x7fa470125750 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b400 osd=4 1 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa46ed9d700 1 -- 172.21.15.59:0/2076700142 reap_dead start 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.5 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa470126150 msgr2=0x7fa470126590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6804/2170432263 conn(0x7fa470126150 0x7fa470126590 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa47011b570 osd=5 1 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.6 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa470127050 msgr2=0x7fa4701274b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6808/3356009390 conn(0x7fa470127050 0x7fa4701274b0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470128150 osd=6 1 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter close_session for osd.7 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6812/53025171 conn(0x7fa470127d30 msgr2=0x7fa47012e950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.137:6812/53025171 conn(0x7fa470127d30 0x7fa47012e950 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:41.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter put_session s=0x7fa470127b20 osd=7 1 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 20 client.41813.objecter shutdown clearing up homeless session... 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 10 client.41813.objecter successfully canceled tick 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa45c062c90 msgr2=0x7fa45c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa45c062c90 0x7fa45c065150 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 rx=0x7fa458005010 tx=0x7fa458009cb0).stop 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4700f8d10 msgr2=0x7fa47010e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa474cad700 1 --2- 172.21.15.59:0/2076700142 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4700f8d10 0x7fa47010e1b0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 rx=0x7fa460009400 tx=0x7fa460009cb0).stop 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.278+0000 7fa46ed9d700 1 -- 172.21.15.59:0/2076700142 reap_dead start 2024-02-18T00:45:41.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.279+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 shutdown_connections 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.279+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 >> 172.21.15.59:0/2076700142 conn(0x7fa4700f2820 msgr2=0x7fa4700f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.279+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 shutdown_connections 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.279+0000 7fa474cad700 1 -- 172.21.15.59:0/2076700142 wait complete. 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:41.279+0000 7fa474cad700 1 librados: shutdown 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.289 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.290 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:45:41.318 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:41.351 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:41.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.921346+0000 osd.4 (osd.4) 409 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:39.922246+0000 osd.4 (osd.4) 410 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:40.006759+0000 osd.7 (osd.7) 235 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:40.007599+0000 osd.7 (osd.7) 236 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:40.022974+0000 mgr.y (mgr.24454) 669 : cluster [DBG] pgmap v1194: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8/804 objects misplaced (0.995%) 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:40.130010+0000 osd.1 (osd.1) 365 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:40.130222+0000 osd.1 (osd.1) 366 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:40.826702+0000 mon.a (mon.0) 3667 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:40.836055+0000 mon.a (mon.0) 3668 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:40.836131+0000 mon.a (mon.0) 3669 : cluster [DBG] osdmap e777: 8 total, 8 up, 7 in 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:40.837896+0000 mon.a (mon.0) 3670 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:45:41.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:41.116465+0000 mon.c (mon.2) 577 : audit [DBG] from='client.? 172.21.15.59:0/2076700142' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.921346+0000 osd.4 (osd.4) 409 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:39.922246+0000 osd.4 (osd.4) 410 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:40.006759+0000 osd.7 (osd.7) 235 : cluster [DBG] 2.1c scrub starts 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:40.007599+0000 osd.7 (osd.7) 236 : cluster [DBG] 2.1c scrub ok 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:40.022974+0000 mgr.y (mgr.24454) 669 : cluster [DBG] pgmap v1194: 214 pgs: 1 remapped+peering, 4 active+recovering, 209 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 8/804 objects misplaced (0.995%) 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:40.130010+0000 osd.1 (osd.1) 365 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:40.130222+0000 osd.1 (osd.1) 366 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:40.826702+0000 mon.a (mon.0) 3667 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:40.836055+0000 mon.a (mon.0) 3668 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:40.836131+0000 mon.a (mon.0) 3669 : cluster [DBG] osdmap e777: 8 total, 8 up, 7 in 2024-02-18T00:45:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:40.837896+0000 mon.a (mon.0) 3670 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T00:45:41.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:41.116465+0000 mon.c (mon.2) 577 : audit [DBG] from='client.? 172.21.15.59:0/2076700142' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:41.603 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:41.607 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:41.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:41.708 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:41.742 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:42.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:45:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:45:42.390 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_blocked_ops 2024-02-18T00:45:42.393 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:42.396 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:42.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:42.497 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:42.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:42] "GET /metrics HTTP/1.1" 200 192338 "" "Prometheus/2.33.4" 2024-02-18T00:45:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:41.474847+0000 osd.5 (osd.5) 339 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:45:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:41.475820+0000 osd.5 (osd.5) 340 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:45:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:41.831870+0000 mon.a (mon.0) 3671 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:45:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:41.831948+0000 mon.a (mon.0) 3672 : cluster [DBG] osdmap e778: 8 total, 8 up, 7 in 2024-02-18T00:45:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:41.833878+0000 mon.a (mon.0) 3673 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:45:42.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:41.474847+0000 osd.5 (osd.5) 339 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:45:42.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:41.475820+0000 osd.5 (osd.5) 340 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:41.831870+0000 mon.a (mon.0) 3671 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:41.831948+0000 mon.a (mon.0) 3672 : cluster [DBG] osdmap e778: 8 total, 8 up, 7 in 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:41.833878+0000 mon.a (mon.0) 3673 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:41.474847+0000 osd.5 (osd.5) 339 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:41.475820+0000 osd.5 (osd.5) 340 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:41.831870+0000 mon.a (mon.0) 3671 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:41.831948+0000 mon.a (mon.0) 3672 : cluster [DBG] osdmap e778: 8 total, 8 up, 7 in 2024-02-18T00:45:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:41.833878+0000 mon.a (mon.0) 3673 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T00:45:42.523 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:42.542 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:42.544 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:42.544 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:42.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:42.664 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:42.678 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:43.025 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:45:43.025 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:45:43.026 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-02-18T00:45:43.026 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-02-18T00:45:43.026 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-02-18T00:45:43.026 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-02-18T00:45:43.029 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:43.032 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:43.033 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:43.133 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:41.631844+0000 mgr.y (mgr.24454) 670 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:41.911643+0000 osd.4 (osd.4) 411 : cluster [DBG] 6.e scrub starts 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:41.914777+0000 osd.4 (osd.4) 412 : cluster [DBG] 6.e scrub ok 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.023088+0000 osd.7 (osd.7) 237 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.023424+0000 mgr.y (mgr.24454) 671 : cluster [DBG] pgmap v1197: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s; 1/816 objects misplaced (0.123%); 0 B/s, 2 objects/s recovering 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.024425+0000 osd.7 (osd.7) 238 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.056075+0000 osd.0 (osd.0) 340 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.059228+0000 osd.0 (osd.0) 341 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:45:43.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.829160+0000 mon.a (mon.0) 3674 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:42.836428+0000 mon.a (mon.0) 3675 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "target_max_objects","val": "1"}]': finished 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.836505+0000 mon.a (mon.0) 3676 : cluster [DBG] osdmap e779: 8 total, 8 up, 7 in 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:41.631844+0000 mgr.y (mgr.24454) 670 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:41.911643+0000 osd.4 (osd.4) 411 : cluster [DBG] 6.e scrub starts 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:41.914777+0000 osd.4 (osd.4) 412 : cluster [DBG] 6.e scrub ok 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.023088+0000 osd.7 (osd.7) 237 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.023424+0000 mgr.y (mgr.24454) 671 : cluster [DBG] pgmap v1197: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s; 1/816 objects misplaced (0.123%); 0 B/s, 2 objects/s recovering 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.024425+0000 osd.7 (osd.7) 238 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.056075+0000 osd.0 (osd.0) 340 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.059228+0000 osd.0 (osd.0) 341 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.829160+0000 mon.a (mon.0) 3674 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-18T00:45:43.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:42.836428+0000 mon.a (mon.0) 3675 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "target_max_objects","val": "1"}]': finished 2024-02-18T00:45:43.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.836505+0000 mon.a (mon.0) 3676 : cluster [DBG] osdmap e779: 8 total, 8 up, 7 in 2024-02-18T00:45:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:41.631844+0000 mgr.y (mgr.24454) 670 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:41.911643+0000 osd.4 (osd.4) 411 : cluster [DBG] 6.e scrub starts 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:41.914777+0000 osd.4 (osd.4) 412 : cluster [DBG] 6.e scrub ok 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.023088+0000 osd.7 (osd.7) 237 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.023424+0000 mgr.y (mgr.24454) 671 : cluster [DBG] pgmap v1197: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s; 1/816 objects misplaced (0.123%); 0 B/s, 2 objects/s recovering 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.024425+0000 osd.7 (osd.7) 238 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.056075+0000 osd.0 (osd.0) 340 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.059228+0000 osd.0 (osd.0) 341 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.829160+0000 mon.a (mon.0) 3674 : cluster [WRN] Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:42.836428+0000 mon.a (mon.0) 3675 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-136","var": "target_max_objects","val": "1"}]': finished 2024-02-18T00:45:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.836505+0000 mon.a (mon.0) 3676 : cluster [DBG] osdmap e779: 8 total, 8 up, 7 in 2024-02-18T00:45:43.514 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:43.516 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:43.517 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:43.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 -- 172.21.15.59:0/3776186910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 msgr2=0x7fb234104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 --2- 172.21.15.59:0/3776186910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 0x7fb234104a20 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 rx=0x7fb22c009be0 tx=0x7fb22c004d10).stop 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 -- 172.21.15.59:0/3776186910 shutdown_connections 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 --2- 172.21.15.59:0/3776186910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 0x7fb234104a20 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 --2- 172.21.15.59:0/3776186910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2340fdbb0 0x7fb2340fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 --2- 172.21.15.59:0/3776186910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2340fe510 0x7fb2340f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 -- 172.21.15.59:0/3776186910 >> 172.21.15.59:0/3776186910 conn(0x7fb2340f3800 msgr2=0x7fb2340f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 -- 172.21.15.59:0/3776186910 shutdown_connections 2024-02-18T00:45:43.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.573+0000 7fb2395a4700 1 -- 172.21.15.59:0/3776186910 wait complete. 2024-02-18T00:45:43.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 librados: starting msgr at 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 librados: starting objecter 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 Processor -- start 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 -- start start 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 librados: setting wanted keys 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 librados: calling monclient init 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 0x7fb23406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2340fdbb0 0x7fb23406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2340fe510 0x7fb23406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb234116430 con 0x7fb2340fe510 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341165a0 con 0x7fb2340fdbb0 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2395a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb234116760 con 0x7fb2340f8420 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb232ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 0x7fb23406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:43.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2340fe510 0x7fb23406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb232ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 0x7fb23406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38548/0 (socket says 172.21.15.59:38548) 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb232ffd700 1 -- 172.21.15.59:0/1143322398 learned_addr learned my addr 172.21.15.59:0/1143322398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.574+0000 7fb2327fc700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2340fdbb0 0x7fb23406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2337fe700 1 -- 172.21.15.59:0/1143322398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 msgr2=0x7fb23406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2337fe700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 0x7fb23406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2337fe700 1 -- 172.21.15.59:0/1143322398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2340fdbb0 msgr2=0x7fb23406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2337fe700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2340fdbb0 0x7fb23406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2337fe700 1 -- 172.21.15.59:0/1143322398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c0097c0 con 0x7fb2340fe510 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2327fc700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2340fdbb0 0x7fb23406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2337fe700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2340fe510 0x7fb23406bab0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 rx=0x7fb22c00b6d0 tx=0x7fb22c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb22bfff700 10 client.?.objecter ms_handle_connect 0x7fb2340fe510 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb22bfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb22bfff700 1 -- 172.21.15.59:0/1143322398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00e040 con 0x7fb2340fe510 2024-02-18T00:45:43.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb22bfff700 1 -- 172.21.15.59:0/1143322398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22c018040 con 0x7fb2340fe510 2024-02-18T00:45:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2395a4700 1 -- 172.21.15.59:0/1143322398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2341169e0 con 0x7fb2340fe510 2024-02-18T00:45:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb22bfff700 1 -- 172.21.15.59:0/1143322398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c01c3f0 con 0x7fb2340fe510 2024-02-18T00:45:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2395a4700 10 client.32019.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2395a4700 1 -- 172.21.15.59:0/1143322398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb234116de0 con 0x7fb2340fe510 2024-02-18T00:45:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.575+0000 7fb2395a4700 1 librados: init done 2024-02-18T00:45:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.576+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb218004f20 con 0x7fb2340fe510 2024-02-18T00:45:43.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.576+0000 7fb22bfff700 1 -- 172.21.15.59:0/1143322398 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb22c013420 con 0x7fb2340fe510 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb22bfff700 1 --2- 172.21.15.59:0/1143322398 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb220062c90 0x7fb220065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb22bfff700 1 -- 172.21.15.59:0/1143322398 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(779..779 src has 418..779) v4 ==== 8844+0+0 (secure 0 0 0) 0x7fb22c07e150 con 0x7fb2340fe510 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb22bfff700 10 client.32019.objecter ms_dispatch 0x7fb234001550 osd_map(779..779 src has 418..779) v4 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb22bfff700 3 client.32019.objecter handle_osd_map got epochs [779,779] > 0 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb22bfff700 3 client.32019.objecter handle_osd_map decoding full epoch 779 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb232ffd700 1 --2- 172.21.15.59:0/1143322398 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb220062c90 0x7fb220065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb22bfff700 20 client.32019.objecter dump_active .. 0 homeless 2024-02-18T00:45:43.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb22bfff700 10 client.32019.objecter ms_handle_connect 0x7fb220062c90 2024-02-18T00:45:43.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.577+0000 7fb232ffd700 1 --2- 172.21.15.59:0/1143322398 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb220062c90 0x7fb220065150 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 rx=0x7fb224004e80 tx=0x7fb22400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:43.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.581+0000 7fb22bfff700 1 -- 172.21.15.59:0/1143322398 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb22c0513c0 con 0x7fb2340fe510 2024-02-18T00:45:43.618 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:43.634 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:43.647 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:43.668 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:43.684 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:43.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:43.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:43.770+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) v1 -- 0x7fb218005750 con 0x7fb2340fe510 2024-02-18T00:45:43.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:43.811 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:43.959 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:43.961 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:43.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:44.063 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:44.089 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:44.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.190+0000 7fb22bfff700 1 -- 172.21.15.59:0/1143322398 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v780) v1 ==== 87+0+0 (secure 0 0 0) 0x7fb22c021070 con 0x7fb2340fe510 2024-02-18T00:45:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.194+0000 7fb229ffb700 10 librados: watch_flush enter 2024-02-18T00:45:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.194+0000 7fb229ffb700 10 librados: watch_flush exit 2024-02-18T00:45:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.194+0000 7fb229ffb700 20 client.32019.objecter shutdown clearing up homeless session... 2024-02-18T00:45:44.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.194+0000 7fb229ffb700 10 client.32019.objecter successfully canceled tick 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb220062c90 msgr2=0x7fb220065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 --2- 172.21.15.59:0/1143322398 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb220062c90 0x7fb220065150 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 rx=0x7fb224004e80 tx=0x7fb22400b040).stop 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2340fe510 msgr2=0x7fb23406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2340fe510 0x7fb23406bab0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 rx=0x7fb22c00b6d0 tx=0x7fb22c004d10).stop 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 shutdown_connections 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2340fe510 0x7fb23406bab0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2340f8420 0x7fb23406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 --2- 172.21.15.59:0/1143322398 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb220062c90 0x7fb220065150 unknown :-1 s=CLOSED pgs=3233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 --2- 172.21.15.59:0/1143322398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2340fdbb0 0x7fb23406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 >> 172.21.15.59:0/1143322398 conn(0x7fb2340f3800 msgr2=0x7fb2340fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 shutdown_connections 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 -- 172.21.15.59:0/1143322398 wait complete. 2024-02-18T00:45:44.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:44.195+0000 7fb229ffb700 1 librados: shutdown 2024-02-18T00:45:44.198 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is set 2024-02-18T00:45:44.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.825948+0000 osd.1 (osd.1) 367 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:45:44.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:42.829073+0000 osd.1 (osd.1) 368 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:45:44.423 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:43.771525+0000 mon.a (mon.0) 3677 : audit [INF] from='client.? 172.21.15.59:0/1143322398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:45:44.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.825948+0000 osd.1 (osd.1) 367 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:45:44.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:42.829073+0000 osd.1 (osd.1) 368 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:45:44.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:43.771525+0000 mon.a (mon.0) 3677 : audit [INF] from='client.? 172.21.15.59:0/1143322398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:45:44.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.825948+0000 osd.1 (osd.1) 367 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:45:44.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:42.829073+0000 osd.1 (osd.1) 368 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:45:44.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:43.771525+0000 mon.a (mon.0) 3677 : audit [INF] from='client.? 172.21.15.59:0/1143322398' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-02-18T00:45:45.138 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:45.142 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:45.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:45.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:45.271 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.023829+0000 mgr.y (mgr.24454) 672 : cluster [DBG] pgmap v1199: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s; 1/816 objects misplaced (0.123%); 0 B/s, 1 objects/s recovering 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.092300+0000 osd.7 (osd.7) 239 : cluster [DBG] 2.b scrub starts 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.093138+0000 osd.7 (osd.7) 240 : cluster [DBG] 2.b scrub ok 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.181279+0000 mon.a (mon.0) 3678 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.181866+0000 mon.a (mon.0) 3679 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:44.191127+0000 mon.a (mon.0) 3680 : audit [INF] from='client.? 172.21.15.59:0/1143322398' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.191204+0000 mon.a (mon.0) 3681 : cluster [DBG] osdmap e780: 8 total, 8 up, 7 in 2024-02-18T00:45:45.384 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.413006+0000 osd.5 (osd.5) 341 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:45:45.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.023829+0000 mgr.y (mgr.24454) 672 : cluster [DBG] pgmap v1199: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s; 1/816 objects misplaced (0.123%); 0 B/s, 1 objects/s recovering 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.092300+0000 osd.7 (osd.7) 239 : cluster [DBG] 2.b scrub starts 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.093138+0000 osd.7 (osd.7) 240 : cluster [DBG] 2.b scrub ok 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.181279+0000 mon.a (mon.0) 3678 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.181866+0000 mon.a (mon.0) 3679 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:44.191127+0000 mon.a (mon.0) 3680 : audit [INF] from='client.? 172.21.15.59:0/1143322398' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.191204+0000 mon.a (mon.0) 3681 : cluster [DBG] osdmap e780: 8 total, 8 up, 7 in 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.413006+0000 osd.5 (osd.5) 341 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.023829+0000 mgr.y (mgr.24454) 672 : cluster [DBG] pgmap v1199: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s; 1/816 objects misplaced (0.123%); 0 B/s, 1 objects/s recovering 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.092300+0000 osd.7 (osd.7) 239 : cluster [DBG] 2.b scrub starts 2024-02-18T00:45:45.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.093138+0000 osd.7 (osd.7) 240 : cluster [DBG] 2.b scrub ok 2024-02-18T00:45:45.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.181279+0000 mon.a (mon.0) 3678 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T00:45:45.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.181866+0000 mon.a (mon.0) 3679 : cluster [WRN] Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:45.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:44.191127+0000 mon.a (mon.0) 3680 : audit [INF] from='client.? 172.21.15.59:0/1143322398' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-02-18T00:45:45.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.191204+0000 mon.a (mon.0) 3681 : cluster [DBG] osdmap e780: 8 total, 8 up, 7 in 2024-02-18T00:45:45.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.413006+0000 osd.5 (osd.5) 341 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:45:45.917 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:45:45.917 INFO:teuthology.orchestra.run.smithi137.stdout: "success": "" 2024-02-18T00:45:45.917 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:45:46.224 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:45:46.302 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:45:46.302 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:45:46.302 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:45:46.302 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:45:46.302 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:45:46.518 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:46.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:44.368245+0000 osd.3 (osd.3) 399 : cluster [DBG] 6.b scrub starts 2024-02-18T00:45:46.589 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:45.299169+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:46.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:44.368245+0000 osd.3 (osd.3) 399 : cluster [DBG] 6.b scrub starts 2024-02-18T00:45:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:45.299169+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:46.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:44.368245+0000 osd.3 (osd.3) 399 : cluster [DBG] 6.b scrub starts 2024-02-18T00:45:46.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:45.299169+0000 mon.a (mon.0) 3682 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:46.959 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.4 dump_historic_ops 2024-02-18T00:45:47.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd set nodeep-scrub 2024-02-18T00:45:47.421 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:47.424 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:47.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:47.452 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:46.025026+0000 mgr.y (mgr.24454) 673 : cluster [DBG] pgmap v1201: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:46.025026+0000 mgr.y (mgr.24454) 673 : cluster [DBG] pgmap v1201: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:47.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:47.553 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:47.606 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:47.610 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:47.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:47.712 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:47.713 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:46.025026+0000 mgr.y (mgr.24454) 673 : cluster [DBG] pgmap v1201: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:47.762 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:47.764 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:47.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:47.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:47.893 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:48.058 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:48.092 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:48.095 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:48.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:48.196 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:48.225 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:48.252 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:48.295 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:48.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:48.397 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:48.431 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:47.386631+0000 osd.6 (osd.6) 343 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:47.386854+0000 osd.6 (osd.6) 344 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:48.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:47.386631+0000 osd.6 (osd.6) 343 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:48.617 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:47.386854+0000 osd.6 (osd.6) 344 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:48.617 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:47.386631+0000 osd.6 (osd.6) 343 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:48.617 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:47.386854+0000 osd.6 (osd.6) 344 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:49.181 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:49.188 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:49.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:49.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:49.369 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:45:49.369 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:45:49.369 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:45:49.369 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:45:49.369 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40868.0:1 189.6s0 189:72c88bc2:::oid_IsCompletePP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e309)", 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 561.70785375499997, 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.008077509, 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40868", 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/458568879", 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.370 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658472+0000", 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658470+0000", 2024-02-18T00:45:49.371 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9140000000000008e-06 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658478+0000", 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0781999999999999e-05 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658489+0000", 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3030000000000002e-06 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.658496+0000", 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0051365309999999997 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.663633+0000", 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4510000000000001e-06 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:27.663636+0000", 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0011838420000001 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.373 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.664820+0000", 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014268199999999999 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.664963+0000", 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047133100000000002 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.665434+0000", 2024-02-18T00:45:49.374 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00069780299999999999 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666132+0000", 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00035506600000000002 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666487+0000", 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2547999999999998e-05 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.375 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:28.666549+0000", 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.008077509 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.41033.0:1 205.3s0 205:cbb9903e:::test_obj:head [omap-set-vals,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected e382)", 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:45:49.376 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 488.67735148399998, 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3087259229999999, 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.41033", 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/3621370462", 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:45:49.377 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688974+0000", 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688972+0000", 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.114e-06 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.378 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688979+0000", 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8130000000000007e-06 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688989+0000", 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7789999999999996e-06 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.688997+0000", 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0035324459999999998 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.379 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.692529+0000", 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4070000000000003e-06 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:40.692534+0000", 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.302913354 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.995447+0000", 2024-02-18T00:45:49.380 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015382699999999999 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.995601+0000", 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020991709999999999 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:37:41.997700+0000", 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3087259229999999 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.381 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(234.5s1 645/605 ECSubWrite(tid=750, reqid=client.31692.0:474, at_version=645'6, trim_to=0'0, roll_forward_to=640'5))", 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:10.789975+0000", 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 158.57635082600001, 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002094865, 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.789975+0000", 2024-02-18T00:45:49.382 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.789975+0000", 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.789974+0000", 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5749e-05 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.383 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790000+0000", 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2935999999999999e-05 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790013+0000", 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8609999999999998e-06 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790021+0000", 2024-02-18T00:45:49.384 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7022999999999997e-05 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790108+0000", 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1747999999999999e-05 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.790129+0000", 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019237900000000001 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.385 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.792053+0000", 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6538999999999999e-05 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:10.792070+0000", 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002094865 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.386 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(234.5s1 653/605 ECSubWrite(tid=759, reqid=client.31692.0:646, at_version=653'8, trim_to=0'0, roll_forward_to=648'7))", 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:18.790887+0000", 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 150.575438295, 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025838480000000001, 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.790887+0000", 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.387 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.790887+0000", 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.999999 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.790887+0000", 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5321000000000003e-05 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.790922+0000", 2024-02-18T00:45:49.388 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8609000000000001e-05 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.790940+0000", 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5270000000000004e-06 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.790950+0000", 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6882000000000002e-05 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.389 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.791007+0000", 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2217000000000001e-05 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.791029+0000", 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024297490000000001 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.793459+0000", 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2412000000000001e-05 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.390 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:18.793471+0000", 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025838480000000001 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.14403.0:2091 5.e 5:7759931f:::notify.3:head [watch ping cookie 94750322748800] snapc 0=[] ondisk+write+known_if_redirected e672)", 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:43.440510+0000", 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 125.92581602, 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010051260000000001, 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.391 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.14403", 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/2016795673", 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2091 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.440510+0000", 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.392 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.440510+0000", 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.440507+0000", 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5792999999999999e-05 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.440523+0000", 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4513e-05 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.393 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.440557+0000", 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4497e-05 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.440572+0000", 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074262899999999997 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.441314+0000", 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.242e-05 2024-02-18T00:45:49.394 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.441397+0000", 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011819099999999999 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:43.441515+0000", 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010051260000000001 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.395 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1107 243.14 243:28000000:test-rados-api-smithi059-82957-119:::head [pgnls start_epoch 682] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e682)", 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:47.821098+0000", 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 121.545227865, 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024823390000000001, 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1107 2024-02-18T00:45:49.396 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821098+0000", 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821098+0000", 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.397 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821096+0000", 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2343e-05 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821108+0000", 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9351000000000002e-05 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821127+0000", 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5879999999999998e-06 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.398 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821136+0000", 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010935 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821245+0000", 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3079999999999998e-06 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.821250+0000", 2024-02-18T00:45:49.399 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021979009999999999 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.823448+0000", 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7628e-05 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.823476+0000", 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010416000000000001 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.400 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.823580+0000", 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024823390000000001 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1628 245.16 245:68000000::::head [pgnls start_epoch 707] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e707)", 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:15.845214+0000", 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 93.521111993000005, 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024007730000000001, 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.401 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1628 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845214+0000", 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.402 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845214+0000", 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845212+0000", 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8480000000000001e-06 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845220+0000", 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2481e-05 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.403 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845233+0000", 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0759999999999999e-06 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845239+0000", 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0584000000000002e-05 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845289+0000", 2024-02-18T00:45:49.404 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6559999999999997e-06 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.845295+0000", 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021985720000000002 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.847493+0000", 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2660999999999998e-05 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.405 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.847526+0000", 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8350999999999995e-05 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:15.847614+0000", 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024007730000000001 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.406 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.31692.0:1794 247.6 e721/717 247:642486d8:test-rados-api-smithi059-82957-127::foo:head v 721'2, mlcod=0'0)", 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:29.851002+0000", 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 79.515324129999996, 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0098142060000000007, 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.851002+0000", 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.407 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.851002+0000", 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.851001+0000", 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0041079369999999999 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.408 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.855109+0000", 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5888000000000001e-05 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.855125+0000", 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8079999999999994e-06 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.855134+0000", 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7089999999999999e-05 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.409 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.855171+0000", 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8351000000000001e-05 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.855189+0000", 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0055952789999999999 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.860784+0000", 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1430000000000002e-05 2024-02-18T00:45:49.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.860816+0000", 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0098142060000000007 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDScrubReserve(234.5s1 REQUEST e730)", 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:39.128305+0000", 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 70.238020875000004, 2024-02-18T00:45:49.411 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0029367070000000002, 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128305+0000", 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128305+0000", 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967296 2024-02-18T00:45:49.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128305+0000", 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8250000000000001e-06 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128308+0000", 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8879999999999992e-06 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128317+0000", 2024-02-18T00:45:49.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6375000000000001e-05 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128344+0000", 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3155999999999997e-05 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128397+0000", 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0659999999999993e-06 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.128406+0000", 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025286750000000002 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.130935+0000", 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026675499999999998 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.131201+0000", 2024-02-18T00:45:49.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0098999999999997e-05 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.131242+0000", 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0029367070000000002 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2014 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e741)", 2024-02-18T00:45:49.416 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:55.843260+0000", 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.523066174, 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014997019999999999, 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2014 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843260+0000", 2024-02-18T00:45:49.417 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843260+0000", 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843258+0000", 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.031e-05 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.418 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843278+0000", 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2382999999999998e-05 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843310+0000", 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5712e-05 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.419 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843336+0000", 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9132999999999998e-05 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843395+0000", 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5995000000000006e-05 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843481+0000", 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000150395 2024-02-18T00:45:49.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,1", 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.843632+0000", 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00053921899999999996 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.844171+0000", 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000468166 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.421 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.844639+0000", 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3055e-05 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.844682+0000", 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8452999999999999e-05 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.844720+0000", 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8766999999999997e-05 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:55.844759+0000", 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014997019999999999 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2016 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e741)", 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:56.845579+0000", 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 52.520746402999997, 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0017172579999999999, 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.423 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2016 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.845579+0000", 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.424 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.845579+0000", 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999962 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.845576+0000", 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9770000000000001e-05 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.845605+0000", 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0474999999999999e-05 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.845636+0000", 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7246999999999999e-05 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.845653+0000", 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012559800000000001 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.845779+0000", 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00068589699999999998 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.846465+0000", 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011514 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,1", 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.846580+0000", 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00049705099999999998 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.847077+0000", 2024-02-18T00:45:49.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012865500000000001 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.847206+0000", 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9179e-05 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.847235+0000", 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3652000000000002e-05 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.428 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.847268+0000", 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8189999999999999e-05 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:56.847297+0000", 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0017172579999999999 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.429 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2018 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e741)", 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:57.847968+0000", 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 51.518357375999997, 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001541724, 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2018 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.430 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.847968+0000", 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.847968+0000", 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999967 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.847965+0000", 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0389e-05 2024-02-18T00:45:49.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.847995+0000", 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0626000000000001e-05 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.848026+0000", 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6877000000000001e-05 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.848043+0000", 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109131 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.848152+0000", 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011507500000000001 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.848267+0000", 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012625099999999999 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,1", 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.848393+0000", 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00055785999999999997 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.848951+0000", 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00044756 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.849399+0000", 2024-02-18T00:45:49.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0531999999999998e-05 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.849439+0000", 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8787e-05 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.849478+0000", 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2042000000000003e-05 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.435 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:57.849510+0000", 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001541724 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2020 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e741)", 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:58.850191+0000", 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 50.516134718000004, 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.054841146, 2024-02-18T00:45:49.436 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2020 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850191+0000", 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850191+0000", 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999967 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850188+0000", 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4204999999999999e-05 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850212+0000", 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2011e-05 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850234+0000", 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6261000000000001e-05 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850260+0000", 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8733999999999998e-05 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850309+0000", 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3838999999999996e-05 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850383+0000", 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000120135 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,1", 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.850503+0000", 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.012706093 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:45:49.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.863209+0000", 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.041736835999999999 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.904946+0000", 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8345000000000001e-05 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.904974+0000", 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2993000000000001e-05 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.905007+0000", 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5097e-05 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:58.905032+0000", 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.054841146 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.442 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2022 249.6 249:602f83fe:::foo:head [append 0~8192] snapc 0=[] ondisk+write+known_if_redirected e741)", 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:59.905846+0000", 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 49.460479806000002, 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020605240000000002, 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2022 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.443 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.905846+0000", 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.905846+0000", 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.905844+0000", 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0378e-05 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.905864+0000", 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5770999999999999e-05 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.905880+0000", 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1489999999999997e-06 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.905887+0000", 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2385000000000005e-05 2024-02-18T00:45:49.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.905969+0000", 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000946158 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.906916+0000", 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000165663 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,1", 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.907081+0000", 2024-02-18T00:45:49.446 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000424089 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.907505+0000", 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00021458499999999999 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.907720+0000", 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.48e-05 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.907815+0000", 2024-02-18T00:45:49.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9638999999999997e-05 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.907864+0000", 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2095000000000003e-05 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:59.907906+0000", 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020605240000000002 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.448 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2070 249.6 249:60000000::::head [pgnls start_epoch 742] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e742)", 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:00.085772+0000", 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 49.280554025000001, 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0034790749999999999, 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2070 2024-02-18T00:45:49.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085772+0000", 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085772+0000", 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085769+0000", 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2796000000000001e-05 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085782+0000", 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0285999999999999e-05 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085802+0000", 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4199999999999996e-06 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085812+0000", 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8670999999999994e-05 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085880+0000", 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6619999999999999e-06 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085887+0000", 2024-02-18T00:45:49.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0032584649999999999 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.089145+0000", 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2419e-05 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.089168+0000", 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3021000000000004e-05 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.089251+0000", 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0034790749999999999 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2127 234.0s0 234:1148e476:test-rados-api-smithi059-82957-133::foo2:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e745)", 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:03.112575+0000", 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 46.253750459999999, 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014643200000000001, 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.454 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2127 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112575+0000", 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112575+0000", 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:45:49.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112573+0000", 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8239999999999995e-06 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112583+0000", 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2607e-05 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112596+0000", 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4070000000000004e-06 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112603+0000", 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5322999999999999e-05 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112658+0000", 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017337900000000001 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.457 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.112832+0000", 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000429873 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.113262+0000", 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00035109799999999999 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.113613+0000", 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00038451099999999998 2024-02-18T00:45:49.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.113997+0000", 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2268000000000003e-05 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.114040+0000", 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014643200000000001 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2135 234.0s0 234:11cebcab:test-rados-api-smithi059-82957-133::foo10:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e745)", 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:03.124589+0000", 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 46.241736531000001, 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019328430000000001, 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2135 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.460 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124589+0000", 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124589+0000", 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124587+0000", 2024-02-18T00:45:49.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6099999999999995e-06 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124597+0000", 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3356000000000001e-05 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124610+0000", 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6620000000000005e-06 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124618+0000", 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2942000000000002e-05 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124671+0000", 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013702899999999999 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.124808+0000", 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00043054299999999997 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.125238+0000", 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000310809 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.125549+0000", 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00090425800000000001 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.126453+0000", 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8559999999999994e-05 2024-02-18T00:45:49.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:03.126522+0000", 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019328430000000001 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2327 251.16 251:68000000:test-rados-api-smithi059-82957-135:::head [pgnls start_epoch 762] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e762)", 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:19.840697+0000", 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 29.525628838999999, 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0027738860000000001, 2024-02-18T00:45:49.465 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2327 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840697+0000", 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.466 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840697+0000", 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840695+0000", 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2163e-05 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840707+0000", 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.512e-05 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840732+0000", 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0710000000000001e-06 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840741+0000", 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6168000000000006e-05 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840827+0000", 2024-02-18T00:45:49.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3209999999999999e-06 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.840834+0000", 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025464630000000001 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.843381+0000", 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1639999999999999e-05 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.843403+0000", 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8231000000000005e-05 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.843471+0000", 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0027738860000000001 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDScrubReserve(4.b REQUEST e766)", 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:30.242858+0000", 2024-02-18T00:45:49.470 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 19.123467599000001, 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024065100000000002, 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.242858+0000", 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.242858+0000", 2024-02-18T00:45:49.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.242858+0000", 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8069999999999996e-06 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.242863+0000", 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2391999999999998e-05 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.242886+0000", 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3584000000000001e-05 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.242899+0000", 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3465e-05 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.242993+0000", 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1526e-05 2024-02-18T00:45:49.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.243014+0000", 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021919320000000002 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.245206+0000", 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2684000000000002e-05 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.245229+0000", 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5546999999999997e-05 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:30.245265+0000", 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024065100000000002 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2647 251.16 251:68000000:%01:::head [pgnls start_epoch 767] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e767)", 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:31.119794+0000", 2024-02-18T00:45:49.475 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 18.246531955999998, 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023526369999999999, 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2647 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119794+0000", 2024-02-18T00:45:49.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119794+0000", 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119791+0000", 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1032e-05 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:45:49.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119802+0000", 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.747e-05 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119820+0000", 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0829999999999998e-06 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119827+0000", 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5237000000000001e-05 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119872+0000", 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.895e-06 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.119876+0000", 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021835069999999999 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.122059+0000", 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0883e-05 2024-02-18T00:45:49.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.122080+0000", 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6245e-05 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:31.122146+0000", 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023526369999999999 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.480 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:45:49.481 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:45:49.481 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:45:49.481 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:49.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:48.025459+0000 mgr.y (mgr.24454) 674 : cluster [DBG] pgmap v1202: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 826 B/s rd, 0 B/s wr, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:49.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:48.313706+0000 mon.a (mon.0) 3683 : cluster [DBG] osdmap e781: 8 total, 8 up, 7 in 2024-02-18T00:45:49.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:48.025459+0000 mgr.y (mgr.24454) 674 : cluster [DBG] pgmap v1202: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 826 B/s rd, 0 B/s wr, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:49.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:48.313706+0000 mon.a (mon.0) 3683 : cluster [DBG] osdmap e781: 8 total, 8 up, 7 in 2024-02-18T00:45:49.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:48.025459+0000 mgr.y (mgr.24454) 674 : cluster [DBG] pgmap v1202: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 826 B/s rd, 0 B/s wr, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:49.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:48.313706+0000 mon.a (mon.0) 3683 : cluster [DBG] osdmap e781: 8 total, 8 up, 7 in 2024-02-18T00:45:49.483 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:49.483 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:49.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.417+0000 7fcf97fa8700 1 -- 172.21.15.59:0/1279025294 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 msgr2=0x7fcf900f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.417+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/1279025294 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf900f90f0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 rx=0x7fcf8c004870 tx=0x7fcf8c009cb0).stop 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.417+0000 7fcf97fa8700 1 -- 172.21.15.59:0/1279025294 shutdown_connections 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/1279025294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf90102270 0x7fcf90104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/1279025294 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf900f90f0 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/1279025294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf900f9670 0x7fcf90101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 -- 172.21.15.59:0/1279025294 >> 172.21.15.59:0/1279025294 conn(0x7fcf900f2820 msgr2=0x7fcf900f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 -- 172.21.15.59:0/1279025294 shutdown_connections 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 -- 172.21.15.59:0/1279025294 wait complete. 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 librados: starting msgr at 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 librados: starting objecter 2024-02-18T00:45:49.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 Processor -- start 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 -- start start 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 librados: setting wanted keys 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 librados: calling monclient init 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf9010e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf900f9670 0x7fcf9010e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf90102270 0x7fcf901083f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf901088e0 con 0x7fcf90102270 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90108a50 con 0x7fcf900f9670 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.418+0000 7fcf97fa8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90108c10 con 0x7fcf900f8d10 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf9010e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf96545700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf90102270 0x7fcf901083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf9010e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38568/0 (socket says 172.21.15.59:38568) 2024-02-18T00:45:49.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 -- 172.21.15.59:0/4275126807 learned_addr learned my addr 172.21.15.59:0/4275126807 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95543700 1 --2- 172.21.15.59:0/4275126807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf900f9670 0x7fcf9010e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 -- 172.21.15.59:0/4275126807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf900f9670 msgr2=0x7fcf9010e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf900f9670 0x7fcf9010e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 -- 172.21.15.59:0/4275126807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf90102270 msgr2=0x7fcf901083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf90102270 0x7fcf901083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 -- 172.21.15.59:0/4275126807 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf880097c0 con 0x7fcf900f8d10 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf96545700 1 --2- 172.21.15.59:0/4275126807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf90102270 0x7fcf901083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf9010e250 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7fcf8c0043a0 tx=0x7fcf8c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf86ffd700 10 client.?.objecter ms_handle_connect 0x7fcf900f8d10 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf86ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00e070 con 0x7fcf900f8d10 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf8c009710 con 0x7fcf900f8d10 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c005040 con 0x7fcf900f8d10 2024-02-18T00:45:49.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c019070 con 0x7fcf900f8d10 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf97fa8700 10 client.41825.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf901090b0 con 0x7fcf900f8d10 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.419+0000 7fcf97fa8700 1 librados: init done 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.420+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7fcf901092a0 con 0x7fcf900f8d10 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.420+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fcf8c0090d0 con 0x7fcf900f8d10 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf7c062c90 0x7fcf7c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(781..781 src has 418..781) v4 ==== 8844+0+0 (secure 0 0 0) 0x7fcf8c07de70 con 0x7fcf900f8d10 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 osd_map(781..781 src has 418..781) v4 2024-02-18T00:45:49.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 3 client.41825.objecter handle_osd_map got epochs [781,781] > 0 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 3 client.41825.objecter handle_osd_map decoding full epoch 781 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf95543700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf7c062c90 0x7fcf7c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 20 client.41825.objecter dump_active .. 0 homeless 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v781) v1 ==== 54+0+16 (secure 0 0 0) 0x7fcf8c080500 con 0x7fcf900f8d10 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 1 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf7c062c90 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf95543700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf7c062c90 0x7fcf7c065150 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 rx=0x7fcf90109b90 tx=0x7fcf8000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcf900fc240 0x7fcf900fc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 1 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 0 1 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9005fa60 osd=0 4 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 10 client.41825.objecter _send_command 1 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf9005ee80 con 0x7fcf900fc240 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.421+0000 7fcf96545700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcf900fc240 0x7fcf900fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.422+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf900fc240 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.422+0000 7fcf96545700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcf900fc240 0x7fcf900fc620 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.422+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf9005ee80 con 0x7fcf900fc240 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.422+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.422+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 1 = system:0 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.422+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf9005fa60 osd=0 4 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.422+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 0 1 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 2 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 2 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf9005fa60 osd=0 3 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 0 2 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9005fa60 osd=0 4 2024-02-18T00:45:49.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 10 client.41825.objecter _send_command 2 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.438+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf901072a0 con 0x7fcf900fc240 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf901072a0 con 0x7fcf900fc240 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf9005fa60 osd=0 4 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 0 2 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 3 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6805/26279110 conn(0x7fcf900598f0 0x7fcf90059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 3 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 1 3 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90028540 osd=1 4 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 10 client.41825.objecter _send_command 3 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf900286b0 con 0x7fcf900598f0 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6805/26279110 conn(0x7fcf900598f0 0x7fcf90059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf900598f0 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.440+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6805/26279110 conn(0x7fcf900598f0 0x7fcf90059cd0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.441+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf900286b0 con 0x7fcf900598f0 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.441+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:45:49.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.441+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 3 = system:0 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.441+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90028540 osd=1 4 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.441+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 1 3 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 4 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 4 2024-02-18T00:45:49.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90028540 osd=1 3 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 1 4 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90028540 osd=1 4 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 10 client.41825.objecter _send_command 4 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.456+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf90059360 con 0x7fcf900598f0 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf90059360 con 0x7fcf900598f0 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90028540 osd=1 4 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 1 4 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 5 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcf9011a220 0x7fcf9011c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 5 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 2 5 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90027b90 osd=2 4 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 10 client.41825.objecter _send_command 5 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf90028210 con 0x7fcf9011a220 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf95543700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcf9011a220 0x7fcf9011c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf9011a220 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.458+0000 7fcf95543700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcf9011a220 0x7fcf9011c610 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.459+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf90028210 con 0x7fcf9011a220 2024-02-18T00:45:49.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.459+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.459+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 5 = system:0 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.459+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90027b90 osd=2 4 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.459+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 2 5 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 6 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 6 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90027b90 osd=2 3 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 2 6 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90027b90 osd=2 4 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 10 client.41825.objecter _send_command 6 2024-02-18T00:45:49.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.473+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf90027600 con 0x7fcf9011a220 2024-02-18T00:45:49.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf90027600 con 0x7fcf9011a220 2024-02-18T00:45:49.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90027b90 osd=2 4 2024-02-18T00:45:49.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 2 6 2024-02-18T00:45:49.566 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:45:49.566 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 7 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcf9011dc80 0x7fcf90120070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 7 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 3 7 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90119870 osd=3 4 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 10 client.41825.objecter _send_command 7 2024-02-18T00:45:49.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf90119ef0 con 0x7fcf9011dc80 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf96545700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcf9011dc80 0x7fcf90120070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf9011dc80 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.475+0000 7fcf96545700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcf9011dc80 0x7fcf90120070 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.476+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf90119ef0 con 0x7fcf9011dc80 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.476+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.476+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 7 = system:0 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.476+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90119870 osd=3 4 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.476+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 3 7 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 8 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 8 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90119870 osd=3 3 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 3 8 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90119870 osd=3 4 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 10 client.41825.objecter _send_command 8 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.490+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf901192e0 con 0x7fcf9011dc80 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf901192e0 con 0x7fcf9011dc80 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90119870 osd=3 4 2024-02-18T00:45:49.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 3 8 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 9 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcf90119450 0x7fcf901236f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 9 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 4 9 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf901192e0 osd=4 4 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 10 client.41825.objecter _send_command 9 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.492+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf9011d850 con 0x7fcf90119450 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.493+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcf90119450 0x7fcf901236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.493+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf90119450 2024-02-18T00:45:49.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.493+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcf90119450 0x7fcf901236f0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.494+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf9011d850 con 0x7fcf90119450 2024-02-18T00:45:49.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.494+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:45:49.571 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:45:49.571 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:45:49.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.494+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 9 = system:0 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.494+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf901192e0 osd=4 4 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.494+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 4 9 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 10 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 10 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf901192e0 osd=4 3 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 4 10 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf901192e0 osd=4 4 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 10 client.41825.objecter _send_command 10 2024-02-18T00:45:49.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.508+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf9011d850 con 0x7fcf90119450 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.510+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf9011d850 con 0x7fcf90119450 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.510+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.510+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.510+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf901192e0 osd=4 4 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.510+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 4 10 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 11 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcf901240f0 0x7fcf901264e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 11 2024-02-18T00:45:49.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 5 11 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9011d850 osd=5 4 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 10 client.41825.objecter _send_command 11 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf90129160 con 0x7fcf901240f0 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf95543700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcf901240f0 0x7fcf901264e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf95543700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcf901240f0 0x7fcf901264e0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.511+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf901240f0 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.512+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf90129160 con 0x7fcf901240f0 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.512+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.512+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 11 = system:0 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.512+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf9011d850 osd=5 4 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.512+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 5 11 2024-02-18T00:45:49.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 12 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 12 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf9011d850 osd=5 3 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 5 12 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9011d850 osd=5 4 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 10 client.41825.objecter _send_command 12 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.526+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf90129160 con 0x7fcf901240f0 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf90129160 con 0x7fcf901240f0 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf9011d850 osd=5 4 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 5 12 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 13 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcf90126ee0 0x7fcf90127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 13 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 6 13 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90129160 osd=6 4 2024-02-18T00:45:49.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 10 client.41825.objecter _send_command 13 2024-02-18T00:45:49.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.528+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf90128f90 con 0x7fcf90126ee0 2024-02-18T00:45:49.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.529+0000 7fcf96545700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcf90126ee0 0x7fcf90127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.529+0000 7fcf96545700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcf90126ee0 0x7fcf90127340 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.529+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf90126ee0 2024-02-18T00:45:49.577 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.530+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf90128f90 con 0x7fcf90126ee0 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.530+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.530+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 13 = system:0 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.530+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90129160 osd=6 4 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.530+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 6 13 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 14 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 14 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90129160 osd=6 3 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 6 14 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90129160 osd=6 4 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 10 client.41825.objecter _send_command 14 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.544+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf90128f90 con 0x7fcf90126ee0 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf90128f90 con 0x7fcf90126ee0 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90129160 osd=6 4 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 6 14 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:49.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 15 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6812/53025171 conn(0x7fcf9012b8d0 0x7fcf9012bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 15 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 7 15 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90128f90 osd=7 4 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 10 client.41825.objecter _send_command 15 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf9012c3d0 con 0x7fcf9012b8d0 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6812/53025171 conn(0x7fcf9012b8d0 0x7fcf9012bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf86ffd700 10 client.41825.objecter ms_handle_connect 0x7fcf9012b8d0 2024-02-18T00:45:49.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.546+0000 7fcf95d44700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6812/53025171 conn(0x7fcf9012b8d0 0x7fcf9012bd30 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.547+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcf9012c3d0 con 0x7fcf9012b8d0 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.547+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.547+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 15 = system:0 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.547+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90128f90 osd=7 4 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.547+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 7 15 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 10 client.41825.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign -1 16 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 20 client.41825.objecter _get_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_remove -1 16 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 20 client.41825.objecter get_session s=0x7fcf90128f90 osd=7 3 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 15 client.41825.objecter _session_command_op_assign 7 16 2024-02-18T00:45:49.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90128f90 osd=7 4 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 10 client.41825.objecter _send_command 16 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.561+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7fcf9012c3d0 con 0x7fcf9012b8d0 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.563+0000 7fcf86ffd700 1 -- 172.21.15.59:0/4275126807 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7fcf9012c3d0 con 0x7fcf9012b8d0 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.563+0000 7fcf86ffd700 10 client.41825.objecter ms_dispatch 0x7fcf90001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.563+0000 7fcf86ffd700 10 client.41825.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.563+0000 7fcf86ffd700 20 client.41825.objecter put_session s=0x7fcf90128f90 osd=7 4 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.563+0000 7fcf86ffd700 15 client.41825.objecter _session_command_op_remove 7 16 2024-02-18T00:45:49.582 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 librados: watch_flush enter 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 librados: watch_flush exit 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.0 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcf900fc240 msgr2=0x7fcf900fc620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcf900fc240 0x7fcf900fc620 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9005fa60 osd=0 1 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.1 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6805/26279110 conn(0x7fcf900598f0 msgr2=0x7fcf90059cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6805/26279110 conn(0x7fcf900598f0 0x7fcf90059cd0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90028540 osd=1 1 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.2 2024-02-18T00:45:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcf9011a220 msgr2=0x7fcf9011c610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6809/1153193637 conn(0x7fcf9011a220 0x7fcf9011c610 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90027b90 osd=2 1 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.3 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcf9011dc80 msgr2=0x7fcf90120070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf96545700 1 -- 172.21.15.59:0/4275126807 reap_dead start 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6813/1805700964 conn(0x7fcf9011dc80 0x7fcf90120070 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90119870 osd=3 1 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.4 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcf90119450 msgr2=0x7fcf901236f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6800/2044387056 conn(0x7fcf90119450 0x7fcf901236f0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf901192e0 osd=4 1 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.5 2024-02-18T00:45:49.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcf901240f0 msgr2=0x7fcf901264e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6804/2170432263 conn(0x7fcf901240f0 0x7fcf901264e0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf9011d850 osd=5 1 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.6 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcf90126ee0 msgr2=0x7fcf90127340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6808/3356009390 conn(0x7fcf90126ee0 0x7fcf90127340 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90129160 osd=6 1 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter close_session for osd.7 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6812/53025171 conn(0x7fcf9012b8d0 msgr2=0x7fcf9012bd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.137:6812/53025171 conn(0x7fcf9012b8d0 0x7fcf9012bd30 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter put_session s=0x7fcf90128f90 osd=7 1 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 20 client.41825.objecter shutdown clearing up homeless session... 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 10 client.41825.objecter successfully canceled tick 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf96545700 1 -- 172.21.15.59:0/4275126807 reap_dead start 2024-02-18T00:45:49.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf7c062c90 msgr2=0x7fcf7c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf7c062c90 0x7fcf7c065150 secure :-1 s=READY pgs=3234 cs=0 l=1 rev1=1 rx=0x7fcf90109b90 tx=0x7fcf8000b040).stop 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 msgr2=0x7fcf9010e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf9010e250 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 rx=0x7fcf8c0043a0 tx=0x7fcf8c00b040).stop 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 shutdown_connections 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf900f8d10 0x7fcf9010e250 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 --2- 172.21.15.59:0/4275126807 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcf7c062c90 0x7fcf7c065150 unknown :-1 s=CLOSED pgs=3234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 >> 172.21.15.59:0/4275126807 conn(0x7fcf900f2820 msgr2=0x7fcf900f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 shutdown_connections 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 -- 172.21.15.59:0/4275126807 wait complete. 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:49.564+0000 7fcf97fa8700 1 librados: shutdown 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.587 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.588 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.588 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.588 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.588 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.588 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:45:49.588 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:49.615 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:50.067 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:50.069 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:50.070 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:50.147 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_ops_in_flight 2024-02-18T00:45:50.171 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:50.219 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:50.240 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:50.242 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:50.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:50.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:49.422286+0000 mon.c (mon.2) 578 : audit [DBG] from='client.? 172.21.15.59:0/4275126807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:50.327 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:50.310773+0000 mon.a (mon.0) 3684 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:50.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:49.422286+0000 mon.c (mon.2) 578 : audit [DBG] from='client.? 172.21.15.59:0/4275126807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:50.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:50.310773+0000 mon.a (mon.0) 3684 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:50.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:50.371 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:50.401 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:50.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:49.422286+0000 mon.c (mon.2) 578 : audit [DBG] from='client.? 172.21.15.59:0/4275126807' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:50.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:50.310773+0000 mon.a (mon.0) 3684 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:50.403 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:50.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:50.440 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:50.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:50.537 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:50.611 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:50.613 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:50.614 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:50.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:50.746 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:50.766 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:50.769 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:50.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:50.870 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:50.900 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:51.148 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:51.151 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:51.152 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:51.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:51.284 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:51.307 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:51.309 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:51.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:51.324 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:45:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:45:51.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:51.443 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3841265247 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 msgr2=0x7fb1a00f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3841265247 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f9160 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 rx=0x7fb19c004870 tx=0x7fb19c009cb0).stop 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3841265247 shutdown_connections 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3841265247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1a0102250 0x7fb1a0104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3841265247 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f9160 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3841265247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a00f96e0 0x7fb1a0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3841265247 >> 172.21.15.59:0/3841265247 conn(0x7fb1a00f26f0 msgr2=0x7fb1a00f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3841265247 shutdown_connections 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.421+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3841265247 wait complete. 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 librados: starting msgr at 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 librados: starting objecter 2024-02-18T00:45:51.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 Processor -- start 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 -- start start 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 librados: setting wanted keys 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 librados: calling monclient init 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1a00f96e0 0x7fb1a00f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a0102250 0x7fb1a00f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a01166f0 con 0x7fb1a0102250 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0116860 con 0x7fb1a00f96e0 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.422+0000 7fb1a72d1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0116a20 con 0x7fb1a00f8d80 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:52866/0 (socket says 172.21.15.59:52866) 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 -- 172.21.15.59:0/3282976429 learned_addr learned my addr 172.21.15.59:0/3282976429 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:51.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a486c700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1a00f96e0 0x7fb1a00f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 -- 172.21.15.59:0/3282976429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1a00f96e0 msgr2=0x7fb1a00f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1a00f96e0 0x7fb1a00f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 -- 172.21.15.59:0/3282976429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a0102250 msgr2=0x7fb1a00f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a0102250 0x7fb1a00f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.423+0000 7fb1a506d700 1 -- 172.21.15.59:0/3282976429 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c005040 con 0x7fb1a00f8d80 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb1a506d700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f6b70 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 rx=0x7fb19c009eb0 tx=0x7fb19c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb196777700 10 client.?.objecter ms_handle_connect 0x7fb1a00f8d80 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb196777700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb196777700 1 -- 172.21.15.59:0/3282976429 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00e070 con 0x7fb1a00f8d80 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb196777700 1 -- 172.21.15.59:0/3282976429 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb19c013da0 con 0x7fb1a00f8d80 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a0116ca0 con 0x7fb1a00f8d80 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb196777700 1 -- 172.21.15.59:0/3282976429 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c019070 con 0x7fb1a00f8d80 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb1a72d1700 10 client.41831.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:51.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a01170a0 con 0x7fb1a00f8d80 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.424+0000 7fb1a72d1700 1 librados: init done 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.425+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a005ee80 con 0x7fb1a00f8d80 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.425+0000 7fb196777700 1 -- 172.21.15.59:0/3282976429 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb19c01e420 con 0x7fb1a00f8d80 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb196777700 1 --2- 172.21.15.59:0/3282976429 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb18c062c90 0x7fb18c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb196777700 1 -- 172.21.15.59:0/3282976429 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(782..782 src has 418..782) v4 ==== 8844+0+0 (secure 0 0 0) 0x7fb19c07dfa0 con 0x7fb1a00f8d80 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb196777700 10 client.41831.objecter ms_dispatch 0x7fb1a0001550 osd_map(782..782 src has 418..782) v4 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb196777700 3 client.41831.objecter handle_osd_map got epochs [782,782] > 0 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb196777700 3 client.41831.objecter handle_osd_map decoding full epoch 782 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb1a486c700 1 --2- 172.21.15.59:0/3282976429 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb18c062c90 0x7fb18c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb196777700 20 client.41831.objecter dump_active .. 0 homeless 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb196777700 10 client.41831.objecter ms_handle_connect 0x7fb18c062c90 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.426+0000 7fb1a486c700 1 --2- 172.21.15.59:0/3282976429 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb18c062c90 0x7fb18c065150 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 rx=0x7fb1a00f8310 tx=0x7fb19000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:51.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.430+0000 7fb196777700 1 -- 172.21.15.59:0/3282976429 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb19c0acf60 con 0x7fb1a00f8d80 2024-02-18T00:45:51.467 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:51.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:49.802685+0000 osd.4 (osd.4) 413 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:51.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:49.802854+0000 osd.4 (osd.4) 414 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:51.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:50.026052+0000 mgr.y (mgr.24454) 675 : cluster [DBG] pgmap v1204: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 711 B/s rd, 0 B/s wr, 0 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:51.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:50.326486+0000 mon.a (mon.0) 3685 : cluster [DBG] osdmap e782: 8 total, 8 up, 7 in 2024-02-18T00:45:51.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:49.802685+0000 osd.4 (osd.4) 413 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:51.471 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:49.802854+0000 osd.4 (osd.4) 414 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:51.471 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:50.026052+0000 mgr.y (mgr.24454) 675 : cluster [DBG] pgmap v1204: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 711 B/s rd, 0 B/s wr, 0 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:51.471 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:50.326486+0000 mon.a (mon.0) 3685 : cluster [DBG] osdmap e782: 8 total, 8 up, 7 in 2024-02-18T00:45:51.472 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:51.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:51.573 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:51.599 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:49.802685+0000 osd.4 (osd.4) 413 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:51.599 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:49.802854+0000 osd.4 (osd.4) 414 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:51.599 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:50.026052+0000 mgr.y (mgr.24454) 675 : cluster [DBG] pgmap v1204: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 711 B/s rd, 0 B/s wr, 0 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:51.599 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:50.326486+0000 mon.a (mon.0) 3685 : cluster [DBG] osdmap e782: 8 total, 8 up, 7 in 2024-02-18T00:45:51.601 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:51.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:51.626+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) v1 -- 0x7fb1a00f8050 con 0x7fb1a00f8d80 2024-02-18T00:45:51.725 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:51.727 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:51.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:51.828 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:51.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:45:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:45:51.855 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:51.978 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:51.981 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:51.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:52.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:52.110 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:52.130 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:52.132 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:52.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:52.234 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:52.249 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:45:52.249 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:45:52.262 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:52.282 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:52.285 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:52.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:52.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.331+0000 7fb196777700 1 -- 172.21.15.59:0/3282976429 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v783) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb19c04d020 con 0x7fb1a00f8d80 2024-02-18T00:45:52.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 10 librados: watch_flush enter 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 10 librados: watch_flush exit 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 20 client.41831.objecter shutdown clearing up homeless session... 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 10 client.41831.objecter successfully canceled tick 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb18c062c90 msgr2=0x7fb18c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3282976429 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb18c062c90 0x7fb18c065150 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 rx=0x7fb1a00f8310 tx=0x7fb19000b040).stop 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 msgr2=0x7fb1a00f6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f6b70 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 rx=0x7fb19c009eb0 tx=0x7fb19c00b040).stop 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 shutdown_connections 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a0102250 0x7fb1a00f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1a00f8d80 0x7fb1a00f6b70 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3282976429 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb18c062c90 0x7fb18c065150 unknown :-1 s=CLOSED pgs=3235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:52.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 --2- 172.21.15.59:0/3282976429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1a00f96e0 0x7fb1a00f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 >> 172.21.15.59:0/3282976429 conn(0x7fb1a00f26f0 msgr2=0x7fb1a00f32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 shutdown_connections 2024-02-18T00:45:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 -- 172.21.15.59:0/3282976429 wait complete. 2024-02-18T00:45:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:52.334+0000 7fb1a72d1700 1 librados: shutdown 2024-02-18T00:45:52.337 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is set 2024-02-18T00:45:52.358 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:51.627593+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.59:0/3282976429' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:52.359 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:51.627929+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:52.359 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:45:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:45:52] "GET /metrics HTTP/1.1" 200 192338 "" "Prometheus/2.33.4" 2024-02-18T00:45:52.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:51.627593+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.59:0/3282976429' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:52.359 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:51.627929+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:52.383 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-02-18T00:45:52.383 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-02-18T00:45:52.383 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-02-18T00:45:52.383 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump -f json-pretty 2024-02-18T00:45:52.386 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:52.413 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:52.510 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:52.512 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:51.627593+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.59:0/3282976429' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:52.512 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:51.627929+0000 mon.a (mon.0) 3686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:45:52.513 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:52.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:52.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:52.643 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:53.053 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_blocked_ops 2024-02-18T00:45:53.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-02-18T00:45:53.300 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:53.302 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:53.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:53.309 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:53.316 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:53.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:53.431 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:53.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:51.635122+0000 mgr.y (mgr.24454) 676 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:53.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:52.027147+0000 mgr.y (mgr.24454) 677 : cluster [DBG] pgmap v1206: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:53.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:52.324747+0000 mon.a (mon.0) 3687 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:53.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:52.330830+0000 mon.a (mon.0) 3688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:53.504 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:52.330908+0000 mon.a (mon.0) 3689 : cluster [DBG] osdmap e783: 8 total, 8 up, 7 in 2024-02-18T00:45:53.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:52.861419+0000 mon.a (mon.0) 3690 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:53.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:51.635122+0000 mgr.y (mgr.24454) 676 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:53.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:52.027147+0000 mgr.y (mgr.24454) 677 : cluster [DBG] pgmap v1206: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:53.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:52.324747+0000 mon.a (mon.0) 3687 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:53.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:52.330830+0000 mon.a (mon.0) 3688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:53.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:52.330908+0000 mon.a (mon.0) 3689 : cluster [DBG] osdmap e783: 8 total, 8 up, 7 in 2024-02-18T00:45:53.505 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:52.861419+0000 mon.a (mon.0) 3690 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:53.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:51.635122+0000 mgr.y (mgr.24454) 676 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:45:53.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:52.027147+0000 mgr.y (mgr.24454) 677 : cluster [DBG] pgmap v1206: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:53.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:52.324747+0000 mon.a (mon.0) 3687 : cluster [WRN] Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:45:53.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:52.330830+0000 mon.a (mon.0) 3688 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-02-18T00:45:53.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:52.330908+0000 mon.a (mon.0) 3689 : cluster [DBG] osdmap e783: 8 total, 8 up, 7 in 2024-02-18T00:45:53.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:52.861419+0000 mon.a (mon.0) 3690 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:53.333172+0000 mon.a (mon.0) 3691 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:53.333241+0000 mon.a (mon.0) 3692 : cluster [DBG] osdmap e784: 8 total, 8 up, 7 in 2024-02-18T00:45:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:53.339292+0000 mon.a (mon.0) 3693 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:53.431984+0000 osd.7 (osd.7) 241 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:53.432063+0000 osd.7 (osd.7) 242 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:53.831762+0000 mon.a (mon.0) 3694 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]': finished 2024-02-18T00:45:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:53.831841+0000 mon.a (mon.0) 3695 : cluster [DBG] osdmap e785: 8 total, 8 up, 7 in 2024-02-18T00:45:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:53.867236+0000 mon.a (mon.0) 3696 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:54.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:53.867603+0000 mon.a (mon.0) 3697 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:54.583 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:54.586 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:54.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:54.687 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:54.718 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:54.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:53.333172+0000 mon.a (mon.0) 3691 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:53.333241+0000 mon.a (mon.0) 3692 : cluster [DBG] osdmap e784: 8 total, 8 up, 7 in 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:53.339292+0000 mon.a (mon.0) 3693 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:53.431984+0000 osd.7 (osd.7) 241 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:53.432063+0000 osd.7 (osd.7) 242 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:53.831762+0000 mon.a (mon.0) 3694 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]': finished 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:53.831841+0000 mon.a (mon.0) 3695 : cluster [DBG] osdmap e785: 8 total, 8 up, 7 in 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:53.867236+0000 mon.a (mon.0) 3696 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:53.867603+0000 mon.a (mon.0) 3697 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:53.333172+0000 mon.a (mon.0) 3691 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:53.333241+0000 mon.a (mon.0) 3692 : cluster [DBG] osdmap e784: 8 total, 8 up, 7 in 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:53.339292+0000 mon.a (mon.0) 3693 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:53.431984+0000 osd.7 (osd.7) 241 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:53.432063+0000 osd.7 (osd.7) 242 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:53.831762+0000 mon.a (mon.0) 3694 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]': finished 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:53.831841+0000 mon.a (mon.0) 3695 : cluster [DBG] osdmap e785: 8 total, 8 up, 7 in 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:53.867236+0000 mon.a (mon.0) 3696 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:54.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:53.867603+0000 mon.a (mon.0) 3697 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-136"}]: dispatch 2024-02-18T00:45:54.967 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:54.970 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:54.971 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:55.072 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:55.100 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:55.126 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:55.174 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:55.177 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:55.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:55.219 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:45:55.223 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:45:55.223 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:45:55.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:55.307 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:55.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:54.027705+0000 mgr.y (mgr.24454) 678 : cluster [DBG] pgmap v1210: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:54.083457+0000 osd.0 (osd.0) 342 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:54.083625+0000 osd.0 (osd.0) 343 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:54.836233+0000 mon.a (mon.0) 3698 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:45:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:54.836270+0000 mon.a (mon.0) 3699 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T00:45:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:54.843037+0000 mon.a (mon.0) 3700 : cluster [DBG] osdmap e786: 8 total, 8 up, 7 in 2024-02-18T00:45:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:55.303525+0000 mon.a (mon.0) 3701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:55.513 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:54.027705+0000 mgr.y (mgr.24454) 678 : cluster [DBG] pgmap v1210: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:55.513 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:54.083457+0000 osd.0 (osd.0) 342 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:54.083625+0000 osd.0 (osd.0) 343 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:54.836233+0000 mon.a (mon.0) 3698 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:54.836270+0000 mon.a (mon.0) 3699 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:54.843037+0000 mon.a (mon.0) 3700 : cluster [DBG] osdmap e786: 8 total, 8 up, 7 in 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:55.303525+0000 mon.a (mon.0) 3701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:54.027705+0000 mgr.y (mgr.24454) 678 : cluster [DBG] pgmap v1210: 214 pgs: 1 active+remapped+backfill_toofull, 213 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:54.083457+0000 osd.0 (osd.0) 342 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:54.083625+0000 osd.0 (osd.0) 343 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:54.836233+0000 mon.a (mon.0) 3698 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:45:55.514 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:54.836270+0000 mon.a (mon.0) 3699 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T00:45:55.515 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:54.843037+0000 mon.a (mon.0) 3700 : cluster [DBG] osdmap e786: 8 total, 8 up, 7 in 2024-02-18T00:45:55.515 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:55.303525+0000 mon.a (mon.0) 3701 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:45:55.978 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.5 dump_historic_ops 2024-02-18T00:45:56.056 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:56.058 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:56.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:56.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:56.186 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:56.208 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:56.209 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:56.287 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:56.310 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:56.336 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:56.534 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:45:56.537 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:56.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:45:56.638 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:56.671 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:56.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 -- 172.21.15.59:0/1326280940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 msgr2=0x7f9c200f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 --2- 172.21.15.59:0/1326280940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 0x7f9c200f7d60 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 rx=0x7f9c08004870 tx=0x7f9c08009cb0).stop 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 -- 172.21.15.59:0/1326280940 shutdown_connections 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 --2- 172.21.15.59:0/1326280940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c200f8440 0x7f9c20104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 --2- 172.21.15.59:0/1326280940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f9ab0 0x7f9c200f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 --2- 172.21.15.59:0/1326280940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 0x7f9c200f7d60 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 -- 172.21.15.59:0/1326280940 >> 172.21.15.59:0/1326280940 conn(0x7f9c200f3800 msgr2=0x7f9c200f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 -- 172.21.15.59:0/1326280940 shutdown_connections 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.629+0000 7f9c259e2700 1 -- 172.21.15.59:0/1326280940 wait complete. 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 librados: starting msgr at 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 librados: starting objecter 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 Processor -- start 2024-02-18T00:45:56.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 -- start start 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 librados: setting wanted keys 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 librados: calling monclient init 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f8440 0x7f9c2006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c200f9ab0 0x7f9c2006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 0x7f9c2006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20116620 con 0x7f9c200fa380 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20116790 con 0x7f9c200f9ab0 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c259e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c20116950 con 0x7f9c200f8440 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c1effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f8440 0x7f9c2006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c1effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f8440 0x7f9c2006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:52882/0 (socket says 172.21.15.59:52882) 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c1f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 0x7f9c2006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c1effd700 1 -- 172.21.15.59:0/979681183 learned_addr learned my addr 172.21.15.59:0/979681183 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:56.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.630+0000 7f9c1e7fc700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c200f9ab0 0x7f9c2006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c1effd700 1 -- 172.21.15.59:0/979681183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c200f9ab0 msgr2=0x7f9c2006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c1effd700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c200f9ab0 0x7f9c2006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c1effd700 1 -- 172.21.15.59:0/979681183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 msgr2=0x7f9c2006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c1effd700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 0x7f9c2006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c1effd700 1 -- 172.21.15.59:0/979681183 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10005040 con 0x7f9c200f8440 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c1effd700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f8440 0x7f9c2006b0d0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 rx=0x7f9c100048f0 tx=0x7f9c10009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c17fff700 10 client.?.objecter ms_handle_connect 0x7f9c200f8440 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c17fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c17fff700 1 -- 172.21.15.59:0/979681183 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10007b00 con 0x7f9c200f8440 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c17fff700 1 -- 172.21.15.59:0/979681183 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c1000f040 con 0x7f9c200f8440 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c08005040 con 0x7f9c200f8440 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c17fff700 1 -- 172.21.15.59:0/979681183 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10014a00 con 0x7f9c200f8440 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c259e2700 10 client.41837.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c20116f90 con 0x7f9c200f8440 2024-02-18T00:45:56.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.631+0000 7f9c259e2700 1 librados: init done 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.632+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2005ee80 con 0x7f9c200f8440 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.632+0000 7f9c17fff700 1 -- 172.21.15.59:0/979681183 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9c1000e580 con 0x7f9c200f8440 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c17fff700 1 --2- 172.21.15.59:0/979681183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9c0c062c90 0x7f9c0c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c17fff700 1 -- 172.21.15.59:0/979681183 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(787..787 src has 418..787) v4 ==== 8810+0+0 (secure 0 0 0) 0x7f9c1007d0e0 con 0x7f9c200f8440 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c17fff700 10 client.41837.objecter ms_dispatch 0x7f9c20001550 osd_map(787..787 src has 418..787) v4 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c17fff700 3 client.41837.objecter handle_osd_map got epochs [787,787] > 0 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c17fff700 3 client.41837.objecter handle_osd_map decoding full epoch 787 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c1e7fc700 1 --2- 172.21.15.59:0/979681183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9c0c062c90 0x7f9c0c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c17fff700 20 client.41837.objecter dump_active .. 0 homeless 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c17fff700 10 client.41837.objecter ms_handle_connect 0x7f9c0c062c90 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.633+0000 7f9c1e7fc700 1 --2- 172.21.15.59:0/979681183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9c0c062c90 0x7f9c0c065150 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 rx=0x7f9c08004ee0 tx=0x7f9c0800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:56.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.637+0000 7f9c17fff700 1 -- 172.21.15.59:0/979681183 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9c10049f60 con 0x7f9c200f8440 2024-02-18T00:45:56.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.820+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9c2006c5b0 con 0x7f9c200f8440 2024-02-18T00:45:56.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.821+0000 7f9c17fff700 1 -- 172.21.15.59:0/979681183 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v787) v1 ==== 81+0+50786 (secure 0 0 0) 0x7f9c100502c0 con 0x7f9c200f8440 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 787, 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "fsid": "d129b4f8-cdf3-11ee-95ba-87774f69a715", 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "created": "2024-02-18T00:23:11.390869+0000", 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "modified": "2024-02-18T00:45:55.840505+0000", 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "last_up_change": "2024-02-18T00:29:06.503960+0000", 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "last_in_change": "2024-02-18T00:45:34.113789+0000", 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_num": 5806080, 2024-02-18T00:45:56.824 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_set": [ 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "nodeep-scrub", 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "noscrub", 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "pglog_hardlimit", 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "purged_snapdirs", 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_deletes", 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "sortbitwise" 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_version": 104, 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "full_ratio": 0.94999998807907104, 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "nearfull_ratio": 0.85000002384185791, 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_snapshot": "", 2024-02-18T00:45:56.825 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_max": 253, 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd": 8, 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "require_min_compat_client": "luminous", 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "min_compat_client": "luminous", 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "require_osd_release": "pacific", 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "pools": [ 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 1, 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "device_health_metrics", 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:27:02.909667+0000", 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.826 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 1, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 1, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 1, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 1, 2024-02-18T00:45:56.827 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 1, 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "67", 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:45:56.828 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.829 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.830 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_max": 32, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_min": 1 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_devicehealth": {} 2024-02-18T00:45:56.831 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 2, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": ".rgw.root", 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:13.697458+0000", 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.832 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 42, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 42, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 42, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 42, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 42, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.833 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "170", 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "107", 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.834 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.835 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.836 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 3, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "iscsi", 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:14.132992+0000", 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.837 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 3, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 3, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 3, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 3, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 3, 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.838 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "60", 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.839 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.840 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.841 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd": {} 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 4, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.log", 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:15.191206+0000", 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.842 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 42, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 42, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 42, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 42, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 42, 2024-02-18T00:45:56.843 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "643", 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "641", 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.844 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.845 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.846 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 5, 2024-02-18T00:45:56.847 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.control", 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:17.526023+0000", 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.848 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "62", 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:45:56.852 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 6, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "default.rgw.meta", 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:29:19.600478+0000", 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.853 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 22, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 22, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 22, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 22, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 22, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "6.16", 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 587, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 586, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 586, 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.854 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "587", 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "587", 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "585", 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.855 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "options": { 2024-02-18T00:45:56.857 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_bias": 4 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": { 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "rgw": {} 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 25, 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "ceph_test_rados_api_asio", 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:32:59.546190+0000", 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool", 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.858 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.859 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "111", 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.860 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.861 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.862 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 234, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-101", 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:42:18.224666+0000", 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 8193, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 3, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.863 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 1, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 8, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 8, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 8, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 8, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 8, 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.864 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "785", 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "784", 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "784", 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "784", 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 10, 2024-02-18T00:45:56.865 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 772, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.866 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "testprofile-test-rados-api-smithi059-82957-101", 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.867 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 8192, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 253, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": "test-rados-api-smithi059-82957-138", 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "create_time": "2024-02-18T00:45:55.299278+0000", 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 32769, 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "flags_names": "hashpspool,creating", 2024-02-18T00:45:56.868 INFO:teuthology.orchestra.run.smithi059.stdout: "type": 1, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "size": 3, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": 2, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_rule": 0, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_count": 0, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_target": 0, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_barrier": 0, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "object_hash": 2, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscale_mode": "off", 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": 32, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num": 32, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_placement_num_target": 32, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_target": 32, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num_pending": 32, 2024-02-18T00:45:56.869 INFO:teuthology.orchestra.run.smithi059.stdout: "last_pg_merge_meta": { 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "source_pgid": "0.0", 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "ready_epoch": 0, 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_started": 0, 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "last_epoch_clean": 0, 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "source_version": "0'0", 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "target_version": "0'0" 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "last_change": "787", 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend": "0", 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_prenautilus": "0", 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "last_force_op_resend_preluminous": "0", 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "auid": 0, 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_mode": "selfmanaged", 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_seq": 0, 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_epoch": 0, 2024-02-18T00:45:56.870 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_snaps": [], 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps": "[]", 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_bytes": 0, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "quota_max_objects": 0, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tiers": [], 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tier_of": -1, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "read_tier": -1, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "write_tier": -1, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_mode": "none", 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_bytes": 0, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "target_max_objects": 0, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_target_full_ratio_micro": 800000, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_flush_age": 0, 2024-02-18T00:45:56.871 INFO:teuthology.orchestra.run.smithi059.stdout: "cache_min_evict_age": 0, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profile": "", 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_params": { 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "none" 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_period": 0, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_count": 0, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "use_gmt_hitset": true, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "min_read_recency_for_promote": 0, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "min_write_recency_for_promote": 0, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_grade_decay_rate": 0, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "hit_set_search_last_n": 0, 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "grade_table": [], 2024-02-18T00:45:56.872 INFO:teuthology.orchestra.run.smithi059.stdout: "stripe_width": 0, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "expected_num_objects": 0, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "fast_read": false, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "options": {}, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "application_metadata": {} 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "6e729102-80e2-4a4e-92ca-600ee8b1ed85", 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.873 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 8, 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 770, 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6802", 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6804", 2024-02-18T00:45:56.875 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6803", 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3700238977 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6801/3700238977", 2024-02-18T00:45:56.876 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6802/3700238977", 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6804/3700238977", 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6803/3700238977", 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "f67a7206-6d31-497f-97e0-672d134b13d0", 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:45:56.877 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 13, 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 770, 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6805", 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.878 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6806", 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.879 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6808", 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6807", 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 26279110 2024-02-18T00:45:56.880 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6805/26279110", 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6806/26279110", 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6808/26279110", 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6807/26279110", 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:45:56.881 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "d6223dbd-e539-4770-9e5a-7b756304f8f7", 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 18, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 770, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6809", 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6810", 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.883 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6812", 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.884 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6811", 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1153193637 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6809/1153193637", 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6810/1153193637", 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6812/1153193637", 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6811/1153193637", 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "fff275df-95e0-41f6-96c3-e531f05de32b", 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 26, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 771, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.886 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6813", 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6814", 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.887 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6816", 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.888 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6815", 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1805700964 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.59:6813/1805700964", 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.59:6814/1805700964", 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.59:6816/1805700964", 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.59:6815/1805700964", 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "a0bae50b-4f29-4310-ac5f-bc7285464231", 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 31, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 770, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.890 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6800", 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6801", 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6803", 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.892 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6802", 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2044387056 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6800/2044387056", 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6801/2044387056", 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6803/2044387056", 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6802/2044387056", 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.893 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "513a4192-45fb-4444-a1f9-4bd2f2a5fa01", 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 39, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 770, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.894 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6804", 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6805", 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.895 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6807", 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6806", 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2170432263 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6804/2170432263", 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6805/2170432263", 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6807/2170432263", 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6806/2170432263", 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.897 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "5bbb7683-ea5a-490c-8c8b-67b53c45406f", 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 0, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 0, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 47, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 769, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.898 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6808", 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6809", 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.899 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6811", 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.900 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6810", 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3356009390 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6808/3356009390", 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6809/3356009390", 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6811/3356009390", 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6810/3356009390", 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.901 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "uuid": "951d317b-7a34-41a0-b1a1-a2851b2070bb", 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "up": 1, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "in": 1, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "weight": 1, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_affinity": 1, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_begin": 0, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "last_clean_end": 0, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "up_from": 53, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "up_thru": 770, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "down_at": 0, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "lost_at": 0, 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addrs": { 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.902 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6812", 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addrs": { 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6813", 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addrs": { 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6815", 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addrs": { 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.137:6814", 2024-02-18T00:45:56.904 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 53025171 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "public_addr": "172.21.15.137:6812/53025171", 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_addr": "172.21.15.137:6813/53025171", 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_back_addr": "172.21.15.137:6815/53025171", 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "heartbeat_front_addr": "172.21.15.137:6814/53025171", 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "state": [ 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "exists", 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "up" 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_xinfo": [ 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.905 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 0, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:54.083627+0000", 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 1, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.906 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:40.130224+0000", 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 2, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.907 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:27.155174+0000", 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 3, 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:06.653248+0000", 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 4, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:49.802856+0000", 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 5, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.909 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:09.990075+0000", 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 6, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 65536, 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:47.386856+0000", 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.910 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": 7, 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "down_stamp": "0.000000", 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_probability": 0, 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "laggy_interval": 0, 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "features": 4540138314316775423, 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "old_weight": 0, 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "last_purged_snaps_scrub": "2024-02-18T00:45:53.432065+0000", 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "dead_epoch": 0 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap": [], 2024-02-18T00:45:56.911 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_upmap_items": [ 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.0", 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.3", 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.912 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.5", 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.913 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.6", 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 4, 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.8", 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:45:56.914 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "2.18", 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 7, 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.915 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.2", 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.4", 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:45:56.916 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 3, 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 0 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.b", 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.917 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.c", 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 3, 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.918 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.919 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.919 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.e", 2024-02-18T00:45:56.919 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 0 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.f", 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.920 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.17", 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 1, 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "4.21", 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 3, 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 5 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "6.9", 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 7 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 2, 2024-02-18T00:45:56.922 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 1 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.9", 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 4 2024-02-18T00:45:56.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.924 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.924 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "25.18", 2024-02-18T00:45:56.924 INFO:teuthology.orchestra.run.smithi059.stdout: "mappings": [ 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: "from": 0, 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: "to": 2 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_temp": [ 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: "pgid": "6.c", 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: "osds": [ 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: 3, 2024-02-18T00:45:56.925 INFO:teuthology.orchestra.run.smithi059.stdout: 5, 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: 6 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "primary_temp": [], 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklist": { 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/1243102711": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/817089002": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3174805070": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/3506000666": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/4167740619": "2024-02-19T00:29:39.703735+0000", 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/999852549": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/3575161033": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:45:56.926 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/2403979355": "2024-02-19T00:24:10.195218+0000", 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:0/642821245": "2024-02-19T00:23:45.307159+0000", 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6800/1656814063": "2024-02-19T00:23:45.307159+0000" 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "range_blocklist": {}, 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "erasure_code_profiles": { 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "default": { 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "ruleset-failure-domain": "osd", 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van" 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.927 INFO:teuthology.orchestra.run.smithi059.stdout: "testprofile-test-rados-api-smithi059-82957-101": { 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-device-class": "", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-failure-domain": "osd", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-root": "default", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "jerasure-per-chunk-alignment": "false", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "k": "2", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "m": "1", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "plugin": "jerasure", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "ruleset-failure-domain": "osd", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "technique": "reed_sol_van", 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "w": "8" 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: "removed_snaps_queue": [ 2024-02-18T00:45:56.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 11, 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 3, 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.929 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 14, 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 9, 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 60, 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.930 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 78, 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:45:56.931 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 86, 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.932 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 94, 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 97, 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 4, 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 4 2024-02-18T00:45:56.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 108, 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 114, 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.934 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 5, 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 7, 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 128, 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:45:56.935 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 10, 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 2 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.936 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 134, 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 12, 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 154, 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 14, 2024-02-18T00:45:56.937 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 166, 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 17, 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.938 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 177, 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 20, 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 237, 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 2, 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.939 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 238, 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 4, 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.940 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 241, 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 6, 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "pool": 245, 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "snaps": [ 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "begin": 9, 2024-02-18T00:45:56.941 INFO:teuthology.orchestra.run.smithi059.stdout: "length": 1 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "new_removed_snaps": [], 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "new_purged_snaps": [], 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_node_flags": {}, 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "device_class_flags": {}, 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode": { 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_enabled": false, 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_bucket_count": 0, 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "degraded_stretch_mode": 0, 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "recovering_stretch_mode": 0, 2024-02-18T00:45:56.942 INFO:teuthology.orchestra.run.smithi059.stdout: "stretch_mode_bucket": 0 2024-02-18T00:45:56.943 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-02-18T00:45:56.943 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 10 librados: watch_flush enter 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 10 librados: watch_flush exit 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 20 client.41837.objecter shutdown clearing up homeless session... 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 10 client.41837.objecter successfully canceled tick 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9c0c062c90 msgr2=0x7f9c0c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 --2- 172.21.15.59:0/979681183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9c0c062c90 0x7f9c0c065150 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 rx=0x7f9c08004ee0 tx=0x7f9c0800b040).stop 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f8440 msgr2=0x7f9c2006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f8440 0x7f9c2006b0d0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 rx=0x7f9c100048f0 tx=0x7f9c10009cb0).stop 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 shutdown_connections 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c200fa380 0x7f9c2006bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c200f8440 0x7f9c2006b0d0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 --2- 172.21.15.59:0/979681183 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9c0c062c90 0x7f9c0c065150 unknown :-1 s=CLOSED pgs=3236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 --2- 172.21.15.59:0/979681183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c200f9ab0 0x7f9c2006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 >> 172.21.15.59:0/979681183 conn(0x7f9c200f3800 msgr2=0x7f9c200f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 shutdown_connections 2024-02-18T00:45:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 -- 172.21.15.59:0/979681183 wait complete. 2024-02-18T00:45:56.945 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:56.823+0000 7f9c259e2700 1 librados: shutdown 2024-02-18T00:45:57.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:55.850187+0000 mon.a (mon.0) 3702 : cluster [DBG] osdmap e787: 8 total, 8 up, 7 in 2024-02-18T00:45:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:55.856169+0000 mon.a (mon.0) 3703 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:56.028370+0000 mgr.y (mgr.24454) 679 : cluster [DBG] pgmap v1213: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 276 B/s rd, 0 op/s; 1/804 objects misplaced (0.124%) 2024-02-18T00:45:57.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:56.821522+0000 mon.c (mon.2) 580 : audit [DBG] from='client.? 172.21.15.59:0/979681183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:45:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:55.850187+0000 mon.a (mon.0) 3702 : cluster [DBG] osdmap e787: 8 total, 8 up, 7 in 2024-02-18T00:45:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:55.856169+0000 mon.a (mon.0) 3703 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:56.028370+0000 mgr.y (mgr.24454) 679 : cluster [DBG] pgmap v1213: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 276 B/s rd, 0 op/s; 1/804 objects misplaced (0.124%) 2024-02-18T00:45:57.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:56.821522+0000 mon.c (mon.2) 580 : audit [DBG] from='client.? 172.21.15.59:0/979681183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:45:57.122 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:57.126 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:57.126 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:57.161 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:55.850187+0000 mon.a (mon.0) 3702 : cluster [DBG] osdmap e787: 8 total, 8 up, 7 in 2024-02-18T00:45:57.161 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:55.856169+0000 mon.a (mon.0) 3703 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:45:57.162 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:56.028370+0000 mgr.y (mgr.24454) 679 : cluster [DBG] pgmap v1213: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 276 B/s rd, 0 op/s; 1/804 objects misplaced (0.124%) 2024-02-18T00:45:57.162 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:56.821522+0000 mon.c (mon.2) 580 : audit [DBG] from='client.? 172.21.15.59:0/979681183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-02-18T00:45:57.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:57.256 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:57.405 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:57.407 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:57.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:57.411 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:45:57.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2 2024-02-18T00:45:57.537 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:57.981 INFO:tasks.thrashosds.thrasher:j is {'epoch': 787, 'fsid': 'd129b4f8-cdf3-11ee-95ba-87774f69a715', 'created': '2024-02-18T00:23:11.390869+0000', 'modified': '2024-02-18T00:45:55.840505+0000', 'last_up_change': '2024-02-18T00:29:06.503960+0000', 'last_in_change': '2024-02-18T00:45:34.113789+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 104, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 253, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'pacific', 'pools': [{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-18T00:27:02.909667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': '.rgw.root', 'create_time': '2024-02-18T00:29:13.697458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 42, 'pg_placement_num': 42, 'pg_placement_num_target': 42, 'pg_num_target': 42, 'pg_num_pending': 42, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '107', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 3, 'pool_name': 'iscsi', 'create_time': '2024-02-18T00:29:14.132992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-18T00:29:15.191206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 42, 'pg_placement_num': 42, 'pg_placement_num_target': 42, 'pg_num_target': 42, 'pg_num_pending': 42, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '643', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '641', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-18T00:29:17.526023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-18T00:29:19.600478+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 22, 'pg_placement_num': 22, 'pg_placement_num_target': 22, 'pg_num_target': 22, 'pg_num_pending': 22, 'last_pg_merge_meta': {'source_pgid': '6.16', 'ready_epoch': 587, 'last_epoch_started': 586, 'last_epoch_clean': 586, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '587', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '587', 'last_force_op_resend_preluminous': '585', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}, {'pool': 25, 'pool_name': 'ceph_test_rados_api_asio', 'create_time': '2024-02-18T00:32:59.546190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 234, 'pool_name': 'test-rados-api-smithi059-82957-101', 'create_time': '2024-02-18T00:42:18.224666+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 3, 'min_size': 2, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '785', 'last_force_op_resend': '784', 'last_force_op_resend_prenautilus': '784', 'last_force_op_resend_preluminous': '784', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 10, 'snap_epoch': 772, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'testprofile-test-rados-api-smithi059-82957-101', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}, {'pool': 253, 'pool_name': 'test-rados-api-smithi059-82957-138', 'create_time': '2024-02-18T00:45:55.299278+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '787', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}}], 'osds': [{'osd': 0, 'uuid': '6e729102-80e2-4a4e-92ca-600ee8b1ed85', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 770, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6801', 'nonce': 3700238977}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6802', 'nonce': 3700238977}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6804', 'nonce': 3700238977}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6803', 'nonce': 3700238977}]}, 'public_addr': '172.21.15.59:6801/3700238977', 'cluster_addr': '172.21.15.59:6802/3700238977', 'heartbeat_back_addr': '172.21.15.59:6804/3700238977', 'heartbeat_front_addr': '172.21.15.59:6803/3700238977', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f67a7206-6d31-497f-97e0-672d134b13d0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 770, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6805', 'nonce': 26279110}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6806', 'nonce': 26279110}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6808', 'nonce': 26279110}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6807', 'nonce': 26279110}]}, 'public_addr': '172.21.15.59:6805/26279110', 'cluster_addr': '172.21.15.59:6806/26279110', 'heartbeat_back_addr': '172.21.15.59:6808/26279110', 'heartbeat_front_addr': '172.21.15.59:6807/26279110', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd6223dbd-e539-4770-9e5a-7b756304f8f7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 770, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6809', 'nonce': 1153193637}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6810', 'nonce': 1153193637}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6812', 'nonce': 1153193637}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6811', 'nonce': 1153193637}]}, 'public_addr': '172.21.15.59:6809/1153193637', 'cluster_addr': '172.21.15.59:6810/1153193637', 'heartbeat_back_addr': '172.21.15.59:6812/1153193637', 'heartbeat_front_addr': '172.21.15.59:6811/1153193637', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fff275df-95e0-41f6-96c3-e531f05de32b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 771, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6813', 'nonce': 1805700964}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6814', 'nonce': 1805700964}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6816', 'nonce': 1805700964}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6815', 'nonce': 1805700964}]}, 'public_addr': '172.21.15.59:6813/1805700964', 'cluster_addr': '172.21.15.59:6814/1805700964', 'heartbeat_back_addr': '172.21.15.59:6816/1805700964', 'heartbeat_front_addr': '172.21.15.59:6815/1805700964', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a0bae50b-4f29-4310-ac5f-bc7285464231', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 770, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6800', 'nonce': 2044387056}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6801', 'nonce': 2044387056}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6803', 'nonce': 2044387056}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6802', 'nonce': 2044387056}]}, 'public_addr': '172.21.15.137:6800/2044387056', 'cluster_addr': '172.21.15.137:6801/2044387056', 'heartbeat_back_addr': '172.21.15.137:6803/2044387056', 'heartbeat_front_addr': '172.21.15.137:6802/2044387056', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '513a4192-45fb-4444-a1f9-4bd2f2a5fa01', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 770, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6804', 'nonce': 2170432263}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6805', 'nonce': 2170432263}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6807', 'nonce': 2170432263}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6806', 'nonce': 2170432263}]}, 'public_addr': '172.21.15.137:6804/2170432263', 'cluster_addr': '172.21.15.137:6805/2170432263', 'heartbeat_back_addr': '172.21.15.137:6807/2170432263', 'heartbeat_front_addr': '172.21.15.137:6806/2170432263', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5bbb7683-ea5a-490c-8c8b-67b53c45406f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 769, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6808', 'nonce': 3356009390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6809', 'nonce': 3356009390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6811', 'nonce': 3356009390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6810', 'nonce': 3356009390}]}, 'public_addr': '172.21.15.137:6808/3356009390', 'cluster_addr': '172.21.15.137:6809/3356009390', 'heartbeat_back_addr': '172.21.15.137:6811/3356009390', 'heartbeat_front_addr': '172.21.15.137:6810/3356009390', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '951d317b-7a34-41a0-b1a1-a2851b2070bb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 770, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6812', 'nonce': 53025171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6813', 'nonce': 53025171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6815', 'nonce': 53025171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6814', 'nonce': 53025171}]}, 'public_addr': '172.21.15.137:6812/53025171', 'cluster_addr': '172.21.15.137:6813/53025171', 'heartbeat_back_addr': '172.21.15.137:6815/53025171', 'heartbeat_front_addr': '172.21.15.137:6814/53025171', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:45:54.083627+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:45:40.130224+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:45:27.155174+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:45:06.653248+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:45:49.802856+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:45:09.990075+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-02-18T00:45:47.386856+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138314316775423, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-02-18T00:45:53.432065+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '2.0', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.3', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '2.5', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.6', 'mappings': [{'from': 4, 'to': 1}]}, {'pgid': '2.8', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '2.18', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '4.2', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.4', 'mappings': [{'from': 1, 'to': 5}, {'from': 3, 'to': 0}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.c', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.e', 'mappings': [{'from': 1, 'to': 0}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.21', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '6.9', 'mappings': [{'from': 0, 'to': 7}, {'from': 2, 'to': 1}]}, {'pgid': '25.9', 'mappings': [{'from': 0, 'to': 4}]}, {'pgid': '25.18', 'mappings': [{'from': 0, 'to': 2}]}], 'pg_temp': [{'pgid': '6.c', 'osds': [3, 5, 6]}], 'primary_temp': [], 'blocklist': {'172.21.15.59:0/1243102711': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/817089002': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/3174805070': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:6800/3506000666': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/4167740619': '2024-02-19T00:29:39.703735+0000', '172.21.15.59:0/999852549': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/3575161033': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:0/2403979355': '2024-02-19T00:24:10.195218+0000', '172.21.15.59:0/642821245': '2024-02-19T00:23:45.307159+0000', '172.21.15.59:6800/1656814063': '2024-02-19T00:23:45.307159+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van'}, 'testprofile-test-rados-api-smithi059-82957-101': {'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-root': 'default', 'jerasure-per-chunk-alignment': 'false', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'ruleset-failure-domain': 'osd', 'technique': 'reed_sol_van', 'w': '8'}}, 'removed_snaps_queue': [{'pool': 11, 'snaps': [{'begin': 3, 'length': 1}, {'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 14, 'snaps': [{'begin': 9, 'length': 2}]}, {'pool': 60, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 78, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 86, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 94, 'snaps': [{'begin': 6, 'length': 2}]}, {'pool': 97, 'snaps': [{'begin': 4, 'length': 4}]}, {'pool': 108, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 114, 'snaps': [{'begin': 5, 'length': 1}, {'begin': 7, 'length': 1}]}, {'pool': 128, 'snaps': [{'begin': 2, 'length': 2}, {'begin': 6, 'length': 2}, {'begin': 10, 'length': 2}]}, {'pool': 134, 'snaps': [{'begin': 12, 'length': 1}]}, {'pool': 154, 'snaps': [{'begin': 14, 'length': 1}]}, {'pool': 166, 'snaps': [{'begin': 17, 'length': 1}]}, {'pool': 177, 'snaps': [{'begin': 20, 'length': 1}]}, {'pool': 237, 'snaps': [{'begin': 2, 'length': 1}]}, {'pool': 238, 'snaps': [{'begin': 4, 'length': 1}]}, {'pool': 241, 'snaps': [{'begin': 6, 'length': 1}]}, {'pool': 245, 'snaps': [{'begin': 9, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-02-18T00:45:57.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:45:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:45:57.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:57.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2' 2024-02-18T00:45:58.038 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:45:58.038 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:45:58.068 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:56.853078+0000 mon.a (mon.0) 3704 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:58.068 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:56.853212+0000 mon.a (mon.0) 3705 : cluster [DBG] osdmap e788: 8 total, 8 up, 7 in 2024-02-18T00:45:58.068 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:56.874016+0000 mon.a (mon.0) 3706 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:58.089 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:45:58.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:56.853078+0000 mon.a (mon.0) 3704 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:58.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:56.853212+0000 mon.a (mon.0) 3705 : cluster [DBG] osdmap e788: 8 total, 8 up, 7 in 2024-02-18T00:45:58.108 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:56.874016+0000 mon.a (mon.0) 3706 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:58.108 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:56.853078+0000 mon.a (mon.0) 3704 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:45:58.108 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:56.853212+0000 mon.a (mon.0) 3705 : cluster [DBG] osdmap e788: 8 total, 8 up, 7 in 2024-02-18T00:45:58.108 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:56.874016+0000 mon.a (mon.0) 3706 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T00:45:58.120 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:58.318 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:45:58.321 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:58.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:45:58.422 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:45:58.450 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:58.477 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:45:58.495 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:58.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:45:58.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:45:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:58.910 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_ops_in_flight 2024-02-18T00:45:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:57.859572+0000 mon.a (mon.0) 3707 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:57.859654+0000 mon.a (mon.0) 3708 : cluster [DBG] osdmap e789: 8 total, 8 up, 7 in 2024-02-18T00:45:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:57.869785+0000 mon.a (mon.0) 3709 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]: dispatch 2024-02-18T00:45:59.049 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:58.028800+0000 mgr.y (mgr.24454) 680 : cluster [DBG] pgmap v1216: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 255 B/s rd, 0 op/s; 1/804 objects misplaced (0.124%) 2024-02-18T00:45:59.050 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:57.859572+0000 mon.a (mon.0) 3707 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:59.050 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:57.859654+0000 mon.a (mon.0) 3708 : cluster [DBG] osdmap e789: 8 total, 8 up, 7 in 2024-02-18T00:45:59.050 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:57.869785+0000 mon.a (mon.0) 3709 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]: dispatch 2024-02-18T00:45:59.050 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:58.028800+0000 mgr.y (mgr.24454) 680 : cluster [DBG] pgmap v1216: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 255 B/s rd, 0 op/s; 1/804 objects misplaced (0.124%) 2024-02-18T00:45:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:45:59.056 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:59.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:45:59.157 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:45:59.178 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:45:59.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:57.859572+0000 mon.a (mon.0) 3707 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T00:45:59.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:57.859654+0000 mon.a (mon.0) 3708 : cluster [DBG] osdmap e789: 8 total, 8 up, 7 in 2024-02-18T00:45:59.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:57.869785+0000 mon.a (mon.0) 3709 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]: dispatch 2024-02-18T00:45:59.184 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:58.028800+0000 mgr.y (mgr.24454) 680 : cluster [DBG] pgmap v1216: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 255 B/s rd, 0 op/s; 1/804 objects misplaced (0.124%) 2024-02-18T00:45:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:45:59.213 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:59.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:45:59.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:45:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:59.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.429+0000 7f56f0983700 1 -- 172.21.15.59:0/1504049155 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ec0fcbb0 msgr2=0x7f56ec0fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.429+0000 7f56f0983700 1 --2- 172.21.15.59:0/1504049155 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ec0fcbb0 0x7f56ec0fcf90 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 rx=0x7f56dc0047b0 tx=0x7f56dc009cb0).stop 2024-02-18T00:45:59.440 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 -- 172.21.15.59:0/1504049155 shutdown_connections 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 --2- 172.21.15.59:0/1504049155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ec0f7400 0x7f56ec103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 --2- 172.21.15.59:0/1504049155 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ec0fcbb0 0x7f56ec0fcf90 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 --2- 172.21.15.59:0/1504049155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ec0fd510 0x7f56ec0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 -- 172.21.15.59:0/1504049155 >> 172.21.15.59:0/1504049155 conn(0x7f56ec0f2820 msgr2=0x7f56ec0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 -- 172.21.15.59:0/1504049155 shutdown_connections 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 -- 172.21.15.59:0/1504049155 wait complete. 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 librados: starting msgr at 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 librados: starting objecter 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.430+0000 7f56f0983700 1 Processor -- start 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 -- start start 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 librados: setting wanted keys 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 librados: calling monclient init 2024-02-18T00:45:59.441 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ec0f7400 0x7f56ec10e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ec0fcbb0 0x7f56ec10e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ec0fd510 0x7f56ec108390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec108880 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec1089f0 con 0x7f56ec0fd510 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec108bb0 con 0x7f56ec0f7400 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ec0fcbb0 0x7f56ec10e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56ea59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ec0f7400 0x7f56ec10e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ec0fcbb0 0x7f56ec10e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60912/0 (socket says 172.21.15.59:60912) 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 -- 172.21.15.59:0/2205170732 learned_addr learned my addr 172.21.15.59:0/2205170732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56ead9d700 1 --2- 172.21.15.59:0/2205170732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ec0fd510 0x7f56ec108390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 -- 172.21.15.59:0/2205170732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ec0f7400 msgr2=0x7f56ec10e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 --2- 172.21.15.59:0/2205170732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56ec0f7400 0x7f56ec10e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 -- 172.21.15.59:0/2205170732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ec0fd510 msgr2=0x7f56ec108390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 --2- 172.21.15.59:0/2205170732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56ec0fd510 0x7f56ec108390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 -- 172.21.15.59:0/2205170732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d4005040 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e9d9b700 1 --2- 172.21.15.59:0/2205170732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ec0fcbb0 0x7f56ec10e6e0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 rx=0x7f56d4009400 tx=0x7f56d4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e37fe700 10 client.?.objecter ms_handle_connect 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e37fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4007a40 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.431+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc005040 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.432+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56d400f040 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.432+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d4014a70 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.432+0000 7f56f0983700 10 client.32040.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.432+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec115020 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.432+0000 7f56f0983700 1 librados: init done 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.432+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f56ec05fa60 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f56d4010530 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6800/3721891797 conn(0x7f56d8062c90 0x7f56d8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(790..790 src has 418..790) v4 ==== 8810+0+0 (secure 0 0 0) 0x7f56d407d0e0 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 osd_map(790..790 src has 418..790) v4 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 3 client.32040.objecter handle_osd_map got epochs [790,790] > 0 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 3 client.32040.objecter handle_osd_map decoding full epoch 790 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6800/3721891797 conn(0x7f56d8062c90 0x7f56d8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 20 client.32040.objecter dump_active .. 0 homeless 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.433+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v790) v1 ==== 54+0+16 (secure 0 0 0) 0x7f56d4084070 con 0x7f56ec0fcbb0 2024-02-18T00:45:59.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 10 client.32040.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 1 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56d8062c90 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6800/3721891797 conn(0x7f56d8062c90 0x7f56d8065150 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 rx=0x7f56dc005010 tx=0x7f56dc009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ec0ffed0 0x7f56ec1002b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 1 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 0 1 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec106120 osd=0 4 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 10 client.32040.objecter _send_command 1 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec1007a0 con 0x7f56ec0ffed0 2024-02-18T00:45:59.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56ead9d700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ec0ffed0 0x7f56ec1002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec0ffed0 2024-02-18T00:45:59.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.434+0000 7f56ead9d700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ec0ffed0 0x7f56ec1002b0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.435+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec1007a0 con 0x7f56ec0ffed0 2024-02-18T00:45:59.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.435+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:45:59.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.435+0000 7f56e37fe700 10 client.32040.objecter _finish_command 1 = system:0 2024-02-18T00:45:59.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.435+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec106120 osd=0 4 2024-02-18T00:45:59.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.435+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 0 1 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 10 client.32040.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 2 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 2 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec106120 osd=0 3 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 0 2 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec106120 osd=0 4 2024-02-18T00:45:59.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 10 client.32040.objecter _send_command 2 2024-02-18T00:45:59.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.450+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec100970 con 0x7f56ec0ffed0 2024-02-18T00:45:59.453 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:45:59.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec100970 con 0x7f56ec0ffed0 2024-02-18T00:45:59.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56e37fe700 10 client.32040.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec106120 osd=0 4 2024-02-18T00:45:59.454 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 0 2 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 10 client.32040.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 3 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ec028540 0x7f56ec1171c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 3 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 1 3 2024-02-18T00:45:59.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec0f4de0 osd=1 4 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 10 client.32040.objecter _send_command 3 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec0f4f50 con 0x7f56ec028540 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.452+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ec028540 0x7f56ec1171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.453+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec028540 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.453+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ec028540 0x7f56ec1171c0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.453+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec0f4f50 con 0x7f56ec028540 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.453+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.453+0000 7f56e37fe700 10 client.32040.objecter _finish_command 3 = system:0 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.453+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec0f4de0 osd=1 4 2024-02-18T00:45:59.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.453+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 1 3 2024-02-18T00:45:59.457 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:59.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:45:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 10 client.32040.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 4 2024-02-18T00:45:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 4 2024-02-18T00:45:59.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec0f4de0 osd=1 3 2024-02-18T00:45:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 1 4 2024-02-18T00:45:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec0f4de0 osd=1 4 2024-02-18T00:45:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 10 client.32040.objecter _send_command 4 2024-02-18T00:45:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.468+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec0fa800 con 0x7f56ec028540 2024-02-18T00:45:59.471 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec0fa800 con 0x7f56ec028540 2024-02-18T00:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e37fe700 10 client.32040.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec0f4de0 osd=1 4 2024-02-18T00:45:59.472 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 1 4 2024-02-18T00:45:59.472 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 10 client.32040.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 5 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ec117d30 0x7f56ec118150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 5 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 2 5 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec1176b0 osd=2 4 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 10 client.32040.objecter _send_command 5 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec121320 con 0x7f56ec117d30 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e9d9b700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ec117d30 0x7f56ec118150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec117d30 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.470+0000 7f56e9d9b700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ec117d30 0x7f56ec118150 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.471+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec121320 con 0x7f56ec117d30 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.471+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.471+0000 7f56e37fe700 10 client.32040.objecter _finish_command 5 = system:0 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.471+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec1176b0 osd=2 4 2024-02-18T00:45:59.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.471+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 2 5 2024-02-18T00:45:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 10 client.32040.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 6 2024-02-18T00:45:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 6 2024-02-18T00:45:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec1176b0 osd=2 3 2024-02-18T00:45:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 2 6 2024-02-18T00:45:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec1176b0 osd=2 4 2024-02-18T00:45:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 10 client.32040.objecter _send_command 6 2024-02-18T00:45:59.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.486+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec0593c0 con 0x7f56ec117d30 2024-02-18T00:45:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec0593c0 con 0x7f56ec117d30 2024-02-18T00:45:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56e37fe700 10 client.32040.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec1176b0 osd=2 4 2024-02-18T00:45:59.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 2 6 2024-02-18T00:45:59.489 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:45:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 10 client.32040.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 7 2024-02-18T00:45:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ec11fd40 0x7f56ec120140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.490 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 7 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 3 7 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec027b90 osd=3 4 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 10 client.32040.objecter _send_command 7 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec028210 con 0x7f56ec11fd40 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.487+0000 7f56ead9d700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ec11fd40 0x7f56ec120140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.488+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec11fd40 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.488+0000 7f56ead9d700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ec11fd40 0x7f56ec120140 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.491 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.488+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec028210 con 0x7f56ec11fd40 2024-02-18T00:45:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.488+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:45:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.488+0000 7f56e37fe700 10 client.32040.objecter _finish_command 7 = system:0 2024-02-18T00:45:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.488+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec027b90 osd=3 4 2024-02-18T00:45:59.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.488+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 3 7 2024-02-18T00:45:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 10 client.32040.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 8 2024-02-18T00:45:59.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 8 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec027b90 osd=3 3 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 3 8 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec027b90 osd=3 4 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 10 client.32040.objecter _send_command 8 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.503+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec120e00 con 0x7f56ec11fd40 2024-02-18T00:45:59.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec120e00 con 0x7f56ec11fd40 2024-02-18T00:45:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56e37fe700 10 client.32040.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec027b90 osd=3 4 2024-02-18T00:45:59.506 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 3 8 2024-02-18T00:45:59.506 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 10 client.32040.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 9 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ec1234a0 0x7f56ec125890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 9 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 4 9 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec120e00 osd=4 4 2024-02-18T00:45:59.507 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 10 client.32040.objecter _send_command 9 2024-02-18T00:45:59.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.504+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec120f70 con 0x7f56ec1234a0 2024-02-18T00:45:59.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.505+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ec1234a0 0x7f56ec125890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.505+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec1234a0 2024-02-18T00:45:59.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.505+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ec1234a0 0x7f56ec125890 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.506+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec120f70 con 0x7f56ec1234a0 2024-02-18T00:45:59.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.506+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:45:59.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.506+0000 7f56e37fe700 10 client.32040.objecter _finish_command 9 = system:0 2024-02-18T00:45:59.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.506+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec120e00 osd=4 4 2024-02-18T00:45:59.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.506+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 4 9 2024-02-18T00:45:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 10 client.32040.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 10 2024-02-18T00:45:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 10 2024-02-18T00:45:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec120e00 osd=4 3 2024-02-18T00:45:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 4 10 2024-02-18T00:45:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec120e00 osd=4 4 2024-02-18T00:45:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 10 client.32040.objecter _send_command 10 2024-02-18T00:45:59.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.521+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec121190 con 0x7f56ec1234a0 2024-02-18T00:45:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec121190 con 0x7f56ec1234a0 2024-02-18T00:45:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e37fe700 10 client.32040.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec120e00 osd=4 4 2024-02-18T00:45:59.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 4 10 2024-02-18T00:45:59.525 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 10 client.32040.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 11 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ec126290 0x7f56ec128680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 11 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 5 11 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec121190 osd=5 4 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 10 client.32040.objecter _send_command 11 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec12b300 con 0x7f56ec126290 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e9d9b700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ec126290 0x7f56ec128680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec126290 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.523+0000 7f56e9d9b700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ec126290 0x7f56ec128680 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.524+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec12b300 con 0x7f56ec126290 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.524+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.524+0000 7f56e37fe700 10 client.32040.objecter _finish_command 11 = system:0 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.524+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec121190 osd=5 4 2024-02-18T00:45:59.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.524+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 5 11 2024-02-18T00:45:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 10 client.32040.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 12 2024-02-18T00:45:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 12 2024-02-18T00:45:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec121190 osd=5 3 2024-02-18T00:45:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 5 12 2024-02-18T00:45:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec121190 osd=5 4 2024-02-18T00:45:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 10 client.32040.objecter _send_command 12 2024-02-18T00:45:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.539+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec12b300 con 0x7f56ec126290 2024-02-18T00:45:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.540+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec12b300 con 0x7f56ec126290 2024-02-18T00:45:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.540+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.540+0000 7f56e37fe700 10 client.32040.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.540+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec121190 osd=5 4 2024-02-18T00:45:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.540+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 5 12 2024-02-18T00:45:59.543 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:45:59.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 10 client.32040.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 13 2024-02-18T00:45:59.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ec12a6f0 0x7f56ec12ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 13 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 6 13 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec12b300 osd=6 4 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 10 client.32040.objecter _send_command 13 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec12e6b0 con 0x7f56ec12a6f0 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56ead9d700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ec12a6f0 0x7f56ec12ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec12a6f0 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.541+0000 7f56ead9d700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ec12a6f0 0x7f56ec12ab30 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.542+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec12e6b0 con 0x7f56ec12a6f0 2024-02-18T00:45:59.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.542+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:45:59.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.542+0000 7f56e37fe700 10 client.32040.objecter _finish_command 13 = system:0 2024-02-18T00:45:59.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.542+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec12b300 osd=6 4 2024-02-18T00:45:59.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.542+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 6 13 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 10 client.32040.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 14 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 14 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec12b300 osd=6 3 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 6 14 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec12b300 osd=6 4 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 10 client.32040.objecter _send_command 14 2024-02-18T00:45:59.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.556+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec128b70 con 0x7f56ec12a6f0 2024-02-18T00:45:59.559 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:45:59.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec128b70 con 0x7f56ec12a6f0 2024-02-18T00:45:59.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56e37fe700 10 client.32040.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec12b300 osd=6 4 2024-02-18T00:45:59.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 6 14 2024-02-18T00:45:59.560 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:45:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 10 client.32040.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:45:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 15 2024-02-18T00:45:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ec129250 0x7f56ec1296b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:45:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 15 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 7 15 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec128b70 osd=7 4 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 10 client.32040.objecter _send_command 15 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56ec129c90 con 0x7f56ec129250 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ec129250 0x7f56ec1296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56ea59c700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ec129250 0x7f56ec1296b0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.558+0000 7f56e37fe700 10 client.32040.objecter ms_handle_connect 0x7f56ec129250 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.559+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f56ec129c90 con 0x7f56ec129250 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.559+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.559+0000 7f56e37fe700 10 client.32040.objecter _finish_command 15 = system:0 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.559+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec128b70 osd=7 4 2024-02-18T00:45:59.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.559+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 7 15 2024-02-18T00:45:59.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 10 client.32040.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}] 2024-02-18T00:45:59.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign -1 16 2024-02-18T00:45:59.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 20 client.32040.objecter _get_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 15 client.32040.objecter _session_command_op_remove -1 16 2024-02-18T00:45:59.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 20 client.32040.objecter get_session s=0x7f56ec128b70 osd=7 3 2024-02-18T00:45:59.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 15 client.32040.objecter _session_command_op_assign 7 16 2024-02-18T00:45:59.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec128b70 osd=7 4 2024-02-18T00:45:59.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 10 client.32040.objecter _send_command 16 2024-02-18T00:45:59.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.574+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) v1 -- 0x7f56ec129c90 con 0x7f56ec129250 2024-02-18T00:45:59.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56e37fe700 1 -- 172.21.15.59:0/2205170732 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 ==== 39+0+3 (crc 0 0 0) 0x7f56ec129c90 con 0x7f56ec129250 2024-02-18T00:45:59.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56e37fe700 10 client.32040.objecter ms_dispatch 0x7f56ec001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'true' ) v1 2024-02-18T00:45:59.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56e37fe700 10 client.32040.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56e37fe700 20 client.32040.objecter put_session s=0x7f56ec128b70 osd=7 4 2024-02-18T00:45:59.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56e37fe700 15 client.32040.objecter _session_command_op_remove 7 16 2024-02-18T00:45:59.577 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 librados: watch_flush enter 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 librados: watch_flush exit 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.0 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ec0ffed0 msgr2=0x7f56ec1002b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6801/3700238977 conn(0x7f56ec0ffed0 0x7f56ec1002b0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec106120 osd=0 1 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.1 2024-02-18T00:45:59.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ec028540 msgr2=0x7f56ec1171c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6805/26279110 conn(0x7f56ec028540 0x7f56ec1171c0 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec0f4de0 osd=1 1 2024-02-18T00:45:59.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.2 2024-02-18T00:45:59.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ec117d30 msgr2=0x7f56ec118150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6809/1153193637 conn(0x7f56ec117d30 0x7f56ec118150 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec1176b0 osd=2 1 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.3 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ec11fd40 msgr2=0x7f56ec120140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6813/1805700964 conn(0x7f56ec11fd40 0x7f56ec120140 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec027b90 osd=3 1 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.4 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56ead9d700 1 -- 172.21.15.59:0/2205170732 reap_dead start 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ec1234a0 msgr2=0x7f56ec125890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6800/2044387056 conn(0x7f56ec1234a0 0x7f56ec125890 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56ead9d700 1 -- 172.21.15.59:0/2205170732 reap_dead start 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec120e00 osd=4 1 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.5 2024-02-18T00:45:59.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ec126290 msgr2=0x7f56ec128680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6804/2170432263 conn(0x7f56ec126290 0x7f56ec128680 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec121190 osd=5 1 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.6 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ec12a6f0 msgr2=0x7f56ec12ab30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6808/3356009390 conn(0x7f56ec12a6f0 0x7f56ec12ab30 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec12b300 osd=6 1 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter close_session for osd.7 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ec129250 msgr2=0x7f56ec1296b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.137:6812/53025171 conn(0x7f56ec129250 0x7f56ec1296b0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter put_session s=0x7f56ec128b70 osd=7 1 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 20 client.32040.objecter shutdown clearing up homeless session... 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 10 client.32040.objecter successfully canceled tick 2024-02-18T00:45:59.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6800/3721891797 conn(0x7f56d8062c90 msgr2=0x7f56d8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> v2:172.21.15.59:6800/3721891797 conn(0x7f56d8062c90 0x7f56d8065150 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 rx=0x7f56dc005010 tx=0x7f56dc009cb0).stop 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ec0fcbb0 msgr2=0x7f56ec10e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 --2- 172.21.15.59:0/2205170732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56ec0fcbb0 0x7f56ec10e6e0 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 rx=0x7f56d4009400 tx=0x7f56d4009cb0).stop 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56ead9d700 1 -- 172.21.15.59:0/2205170732 reap_dead start 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 shutdown_connections 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 >> 172.21.15.59:0/2205170732 conn(0x7f56ec0f2820 msgr2=0x7f56ec0f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 shutdown_connections 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 -- 172.21.15.59:0/2205170732 wait complete. 2024-02-18T00:45:59.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:45:59.576+0000 7f56f0983700 1 librados: shutdown 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.583 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'true' 2024-02-18T00:45:59.588 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:45:59.840 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:45:59.844 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:45:59.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:45:59.945 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5 2024-02-18T00:45:59.974 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:58.863293+0000 mon.a (mon.0) 3710 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]': finished 2024-02-18T00:45:59.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:58.863361+0000 mon.a (mon.0) 3711 : cluster [DBG] osdmap e790: 8 total, 8 up, 7 in 2024-02-18T00:45:59.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:58.890609+0000 mon.a (mon.0) 3712 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:45:59.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:58.890961+0000 mon.a (mon.0) 3713 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]: dispatch 2024-02-18T00:45:59.975 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:45:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:45:59.434688+0000 mon.a (mon.0) 3714 : audit [DBG] from='client.? 172.21.15.59:0/2205170732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:45:59.977 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:00.076 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:46:00.087 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:00.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5' 2024-02-18T00:46:00.189 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:46:00.216 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:00.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:58.863293+0000 mon.a (mon.0) 3710 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]': finished 2024-02-18T00:46:00.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:58.863361+0000 mon.a (mon.0) 3711 : cluster [DBG] osdmap e790: 8 total, 8 up, 7 in 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:58.890609+0000 mon.a (mon.0) 3712 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:58.890961+0000 mon.a (mon.0) 3713 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]: dispatch 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:45:59.434688+0000 mon.a (mon.0) 3714 : audit [DBG] from='client.? 172.21.15.59:0/2205170732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:58.863293+0000 mon.a (mon.0) 3710 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]': finished 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:58.863361+0000 mon.a (mon.0) 3711 : cluster [DBG] osdmap e790: 8 total, 8 up, 7 in 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:58.890609+0000 mon.a (mon.0) 3712 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:58.890961+0000 mon.a (mon.0) 3713 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-138"}]: dispatch 2024-02-18T00:46:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:45:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:45:59.434688+0000 mon.a (mon.0) 3714 : audit [DBG] from='client.? 172.21.15.59:0/2205170732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:46:00.764 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:46:00.767 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:00.768 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:46:00.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:46:00.905 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:01.006 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:46:01.007 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:46:01.007 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-02-18T00:46:01.007 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:46:01.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:45:59.875509+0000 mon.a (mon.0) 3715 : cluster [DBG] osdmap e791: 8 total, 8 up, 7 in 2024-02-18T00:46:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:00.029179+0000 mgr.y (mgr.24454) 681 : cluster [DBG] pgmap v1219: 182 pgs: 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/804 objects misplaced (0.124%) 2024-02-18T00:46:01.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:45:59.875509+0000 mon.a (mon.0) 3715 : cluster [DBG] osdmap e791: 8 total, 8 up, 7 in 2024-02-18T00:46:01.138 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:00.029179+0000 mgr.y (mgr.24454) 681 : cluster [DBG] pgmap v1219: 182 pgs: 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/804 objects misplaced (0.124%) 2024-02-18T00:46:01.138 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:45:59.875509+0000 mon.a (mon.0) 3715 : cluster [DBG] osdmap e791: 8 total, 8 up, 7 in 2024-02-18T00:46:01.138 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:00.029179+0000 mgr.y (mgr.24454) 681 : cluster [DBG] pgmap v1219: 182 pgs: 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1/804 objects misplaced (0.124%) 2024-02-18T00:46:01.154 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:46:01.157 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:01.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:46:01.206 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:01.259 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:46:01.284 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:46:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:46:01.287 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:01.307 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:46:01.309 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:01.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:46:01.410 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:46:01.439 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:01.511 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:46:01.517 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:01.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:46:01.618 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:46:01.648 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:01.671 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:46:01.674 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:01.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:46:01.775 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 -- 172.21.15.59:0/1906853887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 msgr2=0x7f1ca4103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1906853887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca4103a20 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 rx=0x7f1c9c009be0 tx=0x7f1c9c004d10).stop 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 -- 172.21.15.59:0/1906853887 shutdown_connections 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1906853887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca4103a20 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1906853887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1906853887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fd510 0x7f1ca40f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 -- 172.21.15.59:0/1906853887 >> 172.21.15.59:0/1906853887 conn(0x7f1ca40f2820 msgr2=0x7f1ca40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:01.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 -- 172.21.15.59:0/1906853887 shutdown_connections 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 -- 172.21.15.59:0/1906853887 wait complete. 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 librados: starting msgr at 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.772+0000 7f1cabd37700 1 librados: starting objecter 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 Processor -- start 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 -- start start 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 librados: setting wanted keys 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 librados: calling monclient init 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca40f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca40f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fd510 0x7f1ca40f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4116430 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41165a0 con 0x7f1ca40fd510 2024-02-18T00:46:01.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1cabd37700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4116760 con 0x7f1ca40f7400 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca9ad3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca40f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca92d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca40f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca9ad3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca40f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38988/0 (socket says 172.21.15.59:38988) 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca9ad3700 1 -- 172.21.15.59:0/1953954925 learned_addr learned my addr 172.21.15.59:0/1953954925 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1caa2d4700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fd510 0x7f1ca40f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca92d2700 1 -- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 msgr2=0x7f1ca40f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca92d2700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca40f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca92d2700 1 -- 172.21.15.59:0/1953954925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fd510 msgr2=0x7f1ca40f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca92d2700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fd510 0x7f1ca40f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca92d2700 1 -- 172.21.15.59:0/1953954925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca0005040 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca9ad3700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca40f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1ca92d2700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca40f8ea0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 rx=0x7f1c94002780 tx=0x7f1c94009e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1c9affd700 10 client.?.objecter ms_handle_connect 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.773+0000 7f1c9affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1c9affd700 1 -- 172.21.15.59:0/1953954925 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94007b40 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c0097c0 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1c9affd700 1 -- 172.21.15.59:0/1953954925 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c9400f040 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1cabd37700 10 client.32046.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1c9affd700 1 -- 172.21.15.59:0/1953954925 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94014850 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca4116df0 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1cabd37700 1 librados: init done 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.774+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca405fa60 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.775+0000 7f1c9affd700 1 -- 172.21.15.59:0/1953954925 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f1c94007ca0 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.775+0000 7f1c9affd700 1 --2- 172.21.15.59:0/1953954925 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c90062c90 0x7f1c90065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1c9affd700 1 -- 172.21.15.59:0/1953954925 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(792..792 src has 418..792) v4 ==== 8810+0+0 (secure 0 0 0) 0x7f1c9407d450 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1c9affd700 10 client.32046.objecter ms_dispatch 0x7f1ca4001550 osd_map(792..792 src has 418..792) v4 2024-02-18T00:46:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1c9affd700 3 client.32046.objecter handle_osd_map got epochs [792,792] > 0 2024-02-18T00:46:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1c9affd700 3 client.32046.objecter handle_osd_map decoding full epoch 792 2024-02-18T00:46:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1ca9ad3700 1 --2- 172.21.15.59:0/1953954925 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c90062c90 0x7f1c90065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1c9affd700 20 client.32046.objecter dump_active .. 0 homeless 2024-02-18T00:46:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1c9affd700 10 client.32046.objecter ms_handle_connect 0x7f1c90062c90 2024-02-18T00:46:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.776+0000 7f1ca9ad3700 1 --2- 172.21.15.59:0/1953954925 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c90062c90 0x7f1c90065150 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 rx=0x7f1ca0009810 tx=0x7f1ca000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:01.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.780+0000 7f1c9affd700 1 -- 172.21.15.59:0/1953954925 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1c94050630 con 0x7f1ca40fcbb0 2024-02-18T00:46:01.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_blocked_ops 2024-02-18T00:46:01.803 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:46:01.824 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:01.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:46:01.904 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:46:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:46:01.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:00.883388+0000 mon.a (mon.0) 3716 : cluster [DBG] osdmap e792: 8 total, 8 up, 7 in 2024-02-18T00:46:01.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:00.885729+0000 mon.a (mon.0) 3717 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:01.925 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:46:01.952 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:01.971 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:46:01.974 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:01.974 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:46:01.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:01.987+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7f1ca41169e0 con 0x7f1ca40fcbb0 2024-02-18T00:46:02.075 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:46:02.087 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:46:02.102 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:02.123 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:46:02.124 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:02.125 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:46:02.161 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:00.883388+0000 mon.a (mon.0) 3716 : cluster [DBG] osdmap e792: 8 total, 8 up, 7 in 2024-02-18T00:46:02.162 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:00.885729+0000 mon.a (mon.0) 3717 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:02.162 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:00.883388+0000 mon.a (mon.0) 3716 : cluster [DBG] osdmap e792: 8 total, 8 up, 7 in 2024-02-18T00:46:02.162 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:00.885729+0000 mon.a (mon.0) 3717 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:02.225 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:46:02.255 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:02.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:46:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:02] "GET /metrics HTTP/1.1" 200 189979 "" "Prometheus/2.33.4" 2024-02-18T00:46:02.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.887+0000 7f1c9affd700 1 -- 172.21.15.59:0/1953954925 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v794) v1 ==== 91+0+0 (secure 0 0 0) 0x7f1c94050220 con 0x7f1ca40fcbb0 2024-02-18T00:46:02.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.890+0000 7f1cabd37700 10 librados: watch_flush enter 2024-02-18T00:46:02.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 10 librados: watch_flush exit 2024-02-18T00:46:02.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 20 client.32046.objecter shutdown clearing up homeless session... 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 10 client.32046.objecter successfully canceled tick 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c90062c90 msgr2=0x7f1c90065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1953954925 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c90062c90 0x7f1c90065150 secure :-1 s=READY pgs=3238 cs=0 l=1 rev1=1 rx=0x7f1ca0009810 tx=0x7f1ca000b040).stop 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40fcbb0 msgr2=0x7f1ca40f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca40f8ea0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 rx=0x7f1c94002780 tx=0x7f1c94009e40).stop 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 shutdown_connections 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1953954925 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1c90062c90 0x7f1c90065150 unknown :-1 s=CLOSED pgs=3238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:02.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ca40fd510 0x7f1ca40f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:02.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ca40f7400 0x7f1ca40f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:02.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 --2- 172.21.15.59:0/1953954925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ca40fcbb0 0x7f1ca40f8ea0 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:02.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.891+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 >> 172.21.15.59:0/1953954925 conn(0x7f1ca40f2820 msgr2=0x7f1ca4060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:02.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.893+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 shutdown_connections 2024-02-18T00:46:02.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.894+0000 7f1cabd37700 1 -- 172.21.15.59:0/1953954925 wait complete. 2024-02-18T00:46:02.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:02.894+0000 7f1cabd37700 1 librados: shutdown 2024-02-18T00:46:02.897 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:46:03.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:01.640825+0000 mgr.y (mgr.24454) 682 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:01.883688+0000 mon.a (mon.0) 3718 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:01.883844+0000 mon.a (mon.0) 3719 : cluster [DBG] osdmap e793: 8 total, 8 up, 7 in 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:01.926501+0000 mon.a (mon.0) 3720 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:01.926871+0000 mon.a (mon.0) 3721 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-140"}]: dispatch 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:01.988398+0000 mon.a (mon.0) 3722 : audit [INF] from='client.? 172.21.15.59:0/1953954925' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:02.029620+0000 mgr.y (mgr.24454) 683 : cluster [DBG] pgmap v1222: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 1/807 objects misplaced (0.124%) 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:02.880678+0000 mon.a (mon.0) 3723 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:02.886140+0000 mon.a (mon.0) 3724 : audit [INF] from='client.? 172.21.15.59:0/1953954925' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:46:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:02.886220+0000 mon.a (mon.0) 3725 : cluster [DBG] osdmap e794: 8 total, 8 up, 7 in 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:01.640825+0000 mgr.y (mgr.24454) 682 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:01.883688+0000 mon.a (mon.0) 3718 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:01.883844+0000 mon.a (mon.0) 3719 : cluster [DBG] osdmap e793: 8 total, 8 up, 7 in 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:01.926501+0000 mon.a (mon.0) 3720 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:01.926871+0000 mon.a (mon.0) 3721 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-140"}]: dispatch 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:01.988398+0000 mon.a (mon.0) 3722 : audit [INF] from='client.? 172.21.15.59:0/1953954925' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:02.029620+0000 mgr.y (mgr.24454) 683 : cluster [DBG] pgmap v1222: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 1/807 objects misplaced (0.124%) 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:02.880678+0000 mon.a (mon.0) 3723 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:02.886140+0000 mon.a (mon.0) 3724 : audit [INF] from='client.? 172.21.15.59:0/1953954925' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:46:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:02.886220+0000 mon.a (mon.0) 3725 : cluster [DBG] osdmap e794: 8 total, 8 up, 7 in 2024-02-18T00:46:03.552 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:01.640825+0000 mgr.y (mgr.24454) 682 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:03.552 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:01.883688+0000 mon.a (mon.0) 3718 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:03.552 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:01.883844+0000 mon.a (mon.0) 3719 : cluster [DBG] osdmap e793: 8 total, 8 up, 7 in 2024-02-18T00:46:03.553 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:01.926501+0000 mon.a (mon.0) 3720 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:03.553 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:01.926871+0000 mon.a (mon.0) 3721 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-140"}]: dispatch 2024-02-18T00:46:03.553 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:01.988398+0000 mon.a (mon.0) 3722 : audit [INF] from='client.? 172.21.15.59:0/1953954925' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:46:03.553 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:02.029620+0000 mgr.y (mgr.24454) 683 : cluster [DBG] pgmap v1222: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 1/807 objects misplaced (0.124%) 2024-02-18T00:46:03.553 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:02.880678+0000 mon.a (mon.0) 3723 : cluster [WRN] Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-02-18T00:46:03.553 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:02.886140+0000 mon.a (mon.0) 3724 : audit [INF] from='client.? 172.21.15.59:0/1953954925' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:46:03.553 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:02.886220+0000 mon.a (mon.0) 3725 : cluster [DBG] osdmap e794: 8 total, 8 up, 7 in 2024-02-18T00:46:03.784 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:46:03.784 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-02-18T00:46:03.784 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-02-18T00:46:03.784 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-02-18T00:46:03.784 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:46:04.142 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:46:04.145 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:04.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:46:04.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:46:04.274 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:04.473 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:46:04.476 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:04.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:46:04.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:03.828405+0000 mon.a (mon.0) 3726 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:46:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:03.834305+0000 mon.a (mon.0) 3727 : cluster [DBG] osdmap e795: 8 total, 8 up, 7 in 2024-02-18T00:46:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:03.836379+0000 mon.a (mon.0) 3728 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:04.549 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.6 dump_historic_ops 2024-02-18T00:46:04.578 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:46:04.620 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:04.640 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:46:04.643 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:04.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:46:04.744 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:46:04.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:03.828405+0000 mon.a (mon.0) 3726 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:46:04.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:03.834305+0000 mon.a (mon.0) 3727 : cluster [DBG] osdmap e795: 8 total, 8 up, 7 in 2024-02-18T00:46:04.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:03.836379+0000 mon.a (mon.0) 3728 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:03.828405+0000 mon.a (mon.0) 3726 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:46:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:03.834305+0000 mon.a (mon.0) 3727 : cluster [DBG] osdmap e795: 8 total, 8 up, 7 in 2024-02-18T00:46:04.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:03.836379+0000 mon.a (mon.0) 3728 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:04.772 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:04.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.763+0000 7fad9956b700 1 -- 172.21.15.59:0/2955166651 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940fe540 msgr2=0x7fad940f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.763+0000 7fad9956b700 1 --2- 172.21.15.59:0/2955166651 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940fe540 0x7fad940f60c0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 rx=0x7fad880047b0 tx=0x7fad88009cb0).stop 2024-02-18T00:46:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 -- 172.21.15.59:0/2955166651 shutdown_connections 2024-02-18T00:46:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 --2- 172.21.15.59:0/2955166651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad940fdbe0 0x7fad940fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 --2- 172.21.15.59:0/2955166651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad940f67a0 0x7fad94104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 --2- 172.21.15.59:0/2955166651 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940fe540 0x7fad940f60c0 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 -- 172.21.15.59:0/2955166651 >> 172.21.15.59:0/2955166651 conn(0x7fad9406ffc0 msgr2=0x7fad940f43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:04.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 -- 172.21.15.59:0/2955166651 shutdown_connections 2024-02-18T00:46:04.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 -- 172.21.15.59:0/2955166651 wait complete. 2024-02-18T00:46:04.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 librados: starting msgr at 2024-02-18T00:46:04.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 librados: starting objecter 2024-02-18T00:46:04.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 Processor -- start 2024-02-18T00:46:04.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 -- start start 2024-02-18T00:46:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 librados: setting wanted keys 2024-02-18T00:46:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.764+0000 7fad9956b700 1 librados: calling monclient init 2024-02-18T00:46:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad9956b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940f67a0 0x7fad940f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad9956b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad940fdbe0 0x7fad940f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad9956b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad940fe540 0x7fad940f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:04.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad9956b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94116630 con 0x7fad940fdbe0 2024-02-18T00:46:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad9956b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad941167a0 con 0x7fad940fe540 2024-02-18T00:46:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad9956b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad94116960 con 0x7fad940f67a0 2024-02-18T00:46:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940f67a0 0x7fad940f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940f67a0 0x7fad940f7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:39008/0 (socket says 172.21.15.59:39008) 2024-02-18T00:46:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 -- 172.21.15.59:0/2533246993 learned_addr learned my addr 172.21.15.59:0/2533246993 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad927fc700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad940fdbe0 0x7fad940f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad937fe700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad940fe540 0x7fad940f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 -- 172.21.15.59:0/2533246993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad940fe540 msgr2=0x7fad940f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad940fe540 0x7fad940f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 -- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad940fdbe0 msgr2=0x7fad940f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad940fdbe0 0x7fad940f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 -- 172.21.15.59:0/2533246993 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0097c0 con 0x7fad940f67a0 2024-02-18T00:46:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad92ffd700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940f67a0 0x7fad940f7390 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 rx=0x7fad8400e970 tx=0x7fad8400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad7bfff700 10 client.?.objecter ms_handle_connect 0x7fad940f67a0 2024-02-18T00:46:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad7bfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad7bfff700 1 -- 172.21.15.59:0/2533246993 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8400ee60 con 0x7fad940f67a0 2024-02-18T00:46:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad7bfff700 1 -- 172.21.15.59:0/2533246993 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad8400f040 con 0x7fad940f67a0 2024-02-18T00:46:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad88005040 con 0x7fad940f67a0 2024-02-18T00:46:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.765+0000 7fad7bfff700 1 -- 172.21.15.59:0/2533246993 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84007ea0 con 0x7fad940f67a0 2024-02-18T00:46:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.766+0000 7fad9956b700 10 client.41855.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.766+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad94116ff0 con 0x7fad940f67a0 2024-02-18T00:46:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.766+0000 7fad9956b700 1 librados: init done 2024-02-18T00:46:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.766+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9405f9b0 con 0x7fad940f67a0 2024-02-18T00:46:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad7bfff700 1 -- 172.21.15.59:0/2533246993 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fad8400cb10 con 0x7fad940f67a0 2024-02-18T00:46:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad7bfff700 1 --2- 172.21.15.59:0/2533246993 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad7c062c90 0x7fad7c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad7bfff700 1 -- 172.21.15.59:0/2533246993 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(795..795 src has 418..795) v4 ==== 8810+0+0 (secure 0 0 0) 0x7fad8407d0f0 con 0x7fad940f67a0 2024-02-18T00:46:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad7bfff700 10 client.41855.objecter ms_dispatch 0x7fad94001550 osd_map(795..795 src has 418..795) v4 2024-02-18T00:46:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad7bfff700 3 client.41855.objecter handle_osd_map got epochs [795,795] > 0 2024-02-18T00:46:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad7bfff700 3 client.41855.objecter handle_osd_map decoding full epoch 795 2024-02-18T00:46:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad927fc700 1 --2- 172.21.15.59:0/2533246993 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad7c062c90 0x7fad7c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.767+0000 7fad7bfff700 20 client.41855.objecter dump_active .. 0 homeless 2024-02-18T00:46:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.768+0000 7fad927fc700 1 --2- 172.21.15.59:0/2533246993 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad7c062c90 0x7fad7c065150 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 rx=0x7fad88005010 tx=0x7fad88009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.768+0000 7fad7bfff700 10 client.41855.objecter ms_handle_connect 0x7fad7c062c90 2024-02-18T00:46:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.771+0000 7fad7bfff700 1 -- 172.21.15.59:0/2533246993 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fad84050380 con 0x7fad940f67a0 2024-02-18T00:46:04.786 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:04.792 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:46:04.797 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:04.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:46:04.826 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:46:04.899 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:46:04.926 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:04.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.969+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad940f88d0 con 0x7fad7c062c90 2024-02-18T00:46:04.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.984+0000 7fad7bfff700 1 -- 172.21.15.59:0/2533246993 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+541018 (secure 0 0 0) 0x7fad940f88d0 con 0x7fad7c062c90 2024-02-18T00:46:04.993 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 10 librados: watch_flush enter 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 10 librados: watch_flush exit 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 20 client.41855.objecter shutdown clearing up homeless session... 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 10 client.41855.objecter successfully canceled tick 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad7c062c90 msgr2=0x7fad7c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 --2- 172.21.15.59:0/2533246993 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad7c062c90 0x7fad7c065150 secure :-1 s=READY pgs=3239 cs=0 l=1 rev1=1 rx=0x7fad88005010 tx=0x7fad88009cb0).stop 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940f67a0 msgr2=0x7fad940f7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:04.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940f67a0 0x7fad940f7390 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 rx=0x7fad8400e970 tx=0x7fad8400b040).stop 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 shutdown_connections 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 --2- 172.21.15.59:0/2533246993 >> v2:172.21.15.59:6800/3721891797 conn(0x7fad7c062c90 0x7fad7c065150 unknown :-1 s=CLOSED pgs=3239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad940fe540 0x7fad940f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad940f67a0 0x7fad940f7390 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 --2- 172.21.15.59:0/2533246993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad940fdbe0 0x7fad940f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.988+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 >> 172.21.15.59:0/2533246993 conn(0x7fad9406ffc0 msgr2=0x7fad940fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.989+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 shutdown_connections 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.989+0000 7fad9956b700 1 -- 172.21.15.59:0/2533246993 wait complete. 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:04.989+0000 7fad9956b700 1 librados: shutdown 2024-02-18T00:46:04.999 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:46:05.123 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:46:05.126 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:05.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:46:05.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:46:05.257 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:05.380 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:46:05.383 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:05.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:46:05.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:46:05.512 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:05.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.030108+0000 mgr.y (mgr.24454) 684 : cluster [DBG] pgmap v1225: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 1/807 objects misplaced (0.124%) 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.104538+0000 osd.0 (osd.0) 344 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.105419+0000 osd.0 (osd.0) 345 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.333730+0000 osd.3 (osd.3) 400 : cluster [DBG] 6.b scrub starts 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.334836+0000 osd.3 (osd.3) 401 : cluster [DBG] 6.b scrub ok 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.456049+0000 osd.5 (osd.5) 342 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.459506+0000 osd.5 (osd.5) 343 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.756845+0000 osd.1 (osd.1) 369 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.757782+0000 osd.1 (osd.1) 370 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:04.835735+0000 mon.a (mon.0) 3729 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:04.835892+0000 mon.a (mon.0) 3730 : cluster [DBG] osdmap e796: 8 total, 8 up, 7 in 2024-02-18T00:46:05.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:04.887436+0000 mon.a (mon.0) 3731 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:04.887783+0000 mon.a (mon.0) 3732 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-142"}]: dispatch 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.030108+0000 mgr.y (mgr.24454) 684 : cluster [DBG] pgmap v1225: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 1/807 objects misplaced (0.124%) 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.104538+0000 osd.0 (osd.0) 344 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.105419+0000 osd.0 (osd.0) 345 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.333730+0000 osd.3 (osd.3) 400 : cluster [DBG] 6.b scrub starts 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.334836+0000 osd.3 (osd.3) 401 : cluster [DBG] 6.b scrub ok 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.456049+0000 osd.5 (osd.5) 342 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.459506+0000 osd.5 (osd.5) 343 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:46:05.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.756845+0000 osd.1 (osd.1) 369 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.757782+0000 osd.1 (osd.1) 370 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:04.835735+0000 mon.a (mon.0) 3729 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:04.835892+0000 mon.a (mon.0) 3730 : cluster [DBG] osdmap e796: 8 total, 8 up, 7 in 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:04.887436+0000 mon.a (mon.0) 3731 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:04.887783+0000 mon.a (mon.0) 3732 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-142"}]: dispatch 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.030108+0000 mgr.y (mgr.24454) 684 : cluster [DBG] pgmap v1225: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.1 GiB used, 624 GiB / 626 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s; 1/807 objects misplaced (0.124%) 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.104538+0000 osd.0 (osd.0) 344 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.105419+0000 osd.0 (osd.0) 345 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:46:05.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.333730+0000 osd.3 (osd.3) 400 : cluster [DBG] 6.b scrub starts 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.334836+0000 osd.3 (osd.3) 401 : cluster [DBG] 6.b scrub ok 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.456049+0000 osd.5 (osd.5) 342 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.459506+0000 osd.5 (osd.5) 343 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.756845+0000 osd.1 (osd.1) 369 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.757782+0000 osd.1 (osd.1) 370 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:04.835735+0000 mon.a (mon.0) 3729 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:04.835892+0000 mon.a (mon.0) 3730 : cluster [DBG] osdmap e796: 8 total, 8 up, 7 in 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:04.887436+0000 mon.a (mon.0) 3731 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:05.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:04.887783+0000 mon.a (mon.0) 3732 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-142"}]: dispatch 2024-02-18T00:46:06.041 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":1226,"stamp":"2024-02-18T00:46:04.838866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4006013,"num_objects":269,"num_object_clones":1,"num_object_copies":807,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":269,"num_whiteouts":0,"num_read":2040,"num_read_kb":1840,"num_write":677,"num_write_kb":3998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":4005587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1481,"ondisk_log_size":1481,"up":546,"acting":546,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":619,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":2181020,"kb_used_data":51300,"kb_used_omap":0,"kb_used_meta":2129664,"kb_avail":654035044,"statfs":{"total":671965249536,"available":669731885056,"internally_reserved":0,"allocated":52531200,"data_stored":31089053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2180775936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.959184"},"pg_stats":[{"pgid":"255.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"255.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"731'34","reported_seq":666,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883415+0000","last_change":"2024-02-18T00:44:40.215823+0000","last_active":"2024-02-18T00:45:58.883415+0000","last_peered":"2024-02-18T00:45:58.883415+0000","last_clean":"2024-02-18T00:45:58.883415+0000","last_became_active":"2024-02-18T00:43:09.798744+0000","last_became_peered":"2024-02-18T00:43:09.798744+0000","last_unstale":"2024-02-18T00:45:58.883415+0000","last_undegraded":"2024-02-18T00:45:58.883415+0000","last_fullsized":"2024-02-18T00:45:58.883415+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"731'34","last_scrub_stamp":"2024-02-18T00:44:40.215762+0000","last_deep_scrub":"731'34","last_deep_scrub_stamp":"2024-02-18T00:44:40.215762+0000","last_clean_scrub_stamp":"2024-02-18T00:44:40.215762+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"457'39","reported_seq":704,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874258+0000","last_change":"2024-02-18T00:45:42.024498+0000","last_active":"2024-02-18T00:45:58.874258+0000","last_peered":"2024-02-18T00:45:58.874258+0000","last_clean":"2024-02-18T00:45:58.874258+0000","last_became_active":"2024-02-18T00:43:09.791383+0000","last_became_peered":"2024-02-18T00:43:09.791383+0000","last_unstale":"2024-02-18T00:45:58.874258+0000","last_undegraded":"2024-02-18T00:45:58.874258+0000","last_fullsized":"2024-02-18T00:45:58.874258+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:45:42.024429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:42.024429+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"255.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.7","version":"771'9","reported_seq":243,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875417+0000","last_change":"2024-02-18T00:45:35.200141+0000","last_active":"2024-02-18T00:45:58.875417+0000","last_peered":"2024-02-18T00:45:58.875417+0000","last_clean":"2024-02-18T00:45:58.875417+0000","last_became_active":"2024-02-18T00:45:35.153559+0000","last_became_peered":"2024-02-18T00:45:35.153559+0000","last_unstale":"2024-02-18T00:45:58.875417+0000","last_undegraded":"2024-02-18T00:45:58.875417+0000","last_fullsized":"2024-02-18T00:45:58.875417+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"745'8","last_scrub_stamp":"2024-02-18T00:45:23.209384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:45:23.209384+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":37,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":29,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"255.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.6","version":"745'14","reported_seq":46,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889373+0000","last_change":"2024-02-18T00:45:35.188329+0000","last_active":"2024-02-18T00:45:58.889373+0000","last_peered":"2024-02-18T00:45:58.889373+0000","last_clean":"2024-02-18T00:45:58.889373+0000","last_became_active":"2024-02-18T00:45:35.152227+0000","last_became_peered":"2024-02-18T00:45:35.152227+0000","last_unstale":"2024-02-18T00:45:58.889373+0000","last_undegraded":"2024-02-18T00:45:58.889373+0000","last_fullsized":"2024-02-18T00:45:58.889373+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"721'13","last_scrub_stamp":"2024-02-18T00:44:35.346673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.346673+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":4000021,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":4000021,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"255.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.5","version":"771'22","reported_seq":247,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863279+0000","last_change":"2024-02-18T00:44:39.133161+0000","last_active":"2024-02-18T00:45:58.863279+0000","last_peered":"2024-02-18T00:45:58.863279+0000","last_clean":"2024-02-18T00:45:58.863279+0000","last_became_active":"2024-02-18T00:42:19.763983+0000","last_became_peered":"2024-02-18T00:42:19.763983+0000","last_unstale":"2024-02-18T00:45:58.863279+0000","last_undegraded":"2024-02-18T00:45:58.863279+0000","last_fullsized":"2024-02-18T00:45:58.863279+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"705'17","last_scrub_stamp":"2024-02-18T00:44:39.133079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:39.133079+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":63,"num_objects":10,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":14,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"255.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.4","version":"771'4","reported_seq":222,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884597+0000","last_change":"2024-02-18T00:45:27.303126+0000","last_active":"2024-02-18T00:45:58.884597+0000","last_peered":"2024-02-18T00:45:58.884597+0000","last_clean":"2024-02-18T00:45:58.884597+0000","last_became_active":"2024-02-18T00:42:19.767925+0000","last_became_peered":"2024-02-18T00:42:19.767925+0000","last_unstale":"2024-02-18T00:45:58.884597+0000","last_undegraded":"2024-02-18T00:45:58.884597+0000","last_fullsized":"2024-02-18T00:45:58.884597+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"745'3","last_scrub_stamp":"2024-02-18T00:45:27.303046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:45:27.303046+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":32,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,5],"acting":[3,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"255.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.3","version":"745'8","reported_seq":249,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871478+0000","last_change":"2024-02-18T00:45:35.188936+0000","last_active":"2024-02-18T00:45:58.871478+0000","last_peered":"2024-02-18T00:45:58.871478+0000","last_clean":"2024-02-18T00:45:58.871478+0000","last_became_active":"2024-02-18T00:45:35.164888+0000","last_became_peered":"2024-02-18T00:45:35.164888+0000","last_unstale":"2024-02-18T00:45:58.871478+0000","last_undegraded":"2024-02-18T00:45:58.871478+0000","last_fullsized":"2024-02-18T00:45:58.871478+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"666'5","last_scrub_stamp":"2024-02-18T00:44:27.711710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:27.711710+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":29,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":29,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"255.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.2","version":"789'15","reported_seq":47,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871493+0000","last_change":"2024-02-18T00:45:35.182045+0000","last_active":"2024-02-18T00:45:58.871493+0000","last_peered":"2024-02-18T00:45:58.871493+0000","last_clean":"2024-02-18T00:45:58.871493+0000","last_became_active":"2024-02-18T00:45:35.149345+0000","last_became_peered":"2024-02-18T00:45:35.149345+0000","last_unstale":"2024-02-18T00:45:58.871493+0000","last_undegraded":"2024-02-18T00:45:58.871493+0000","last_fullsized":"2024-02-18T00:45:58.871493+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"656'12","last_scrub_stamp":"2024-02-18T00:44:31.444627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.444627+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":21,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":21,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"255.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.1","version":"759'12","reported_seq":230,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883509+0000","last_change":"2024-02-18T00:45:28.354937+0000","last_active":"2024-02-18T00:45:58.883509+0000","last_peered":"2024-02-18T00:45:58.883509+0000","last_clean":"2024-02-18T00:45:58.883509+0000","last_became_active":"2024-02-18T00:42:19.763959+0000","last_became_peered":"2024-02-18T00:42:19.763959+0000","last_unstale":"2024-02-18T00:45:58.883509+0000","last_undegraded":"2024-02-18T00:45:58.883509+0000","last_fullsized":"2024-02-18T00:45:58.883509+0000","mapping_epoch":605,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":606,"parent":"0.0","parent_split_bits":0,"last_scrub":"759'12","last_scrub_stamp":"2024-02-18T00:45:28.354842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:45:28.354842+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":52,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"255.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-02-18T00:46:03.828733+0000","last_change":"2024-02-18T00:46:03.828733+0000","last_active":"2024-02-18T00:46:03.828733+0000","last_peered":"2024-02-18T00:46:03.828733+0000","last_clean":"2024-02-18T00:46:03.828733+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T00:46:03.828733+0000","last_undegraded":"2024-02-18T00:46:03.828733+0000","last_fullsized":"2024-02-18T00:46:03.828733+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:03.828733+0000","last_clean_scrub_stamp":"2024-02-18T00:46:03.828733+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"234.0","version":"771'13","reported_seq":258,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.867740+0000","last_change":"2024-02-18T00:45:35.346245+0000","last_active":"2024-02-18T00:45:58.867740+0000","last_peered":"2024-02-18T00:45:58.867740+0000","last_clean":"2024-02-18T00:45:58.867740+0000","last_became_active":"2024-02-18T00:45:35.146974+0000","last_became_peered":"2024-02-18T00:45:35.146974+0000","last_unstale":"2024-02-18T00:45:58.867740+0000","last_undegraded":"2024-02-18T00:45:58.867740+0000","last_fullsized":"2024-02-18T00:45:58.867740+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":605,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"630'10","last_scrub_stamp":"2024-02-18T00:44:35.784138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:18.749711+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.784138+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":416,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883516+0000","last_change":"2024-02-18T00:45:35.149220+0000","last_active":"2024-02-18T00:45:58.883516+0000","last_peered":"2024-02-18T00:45:58.883516+0000","last_clean":"2024-02-18T00:45:58.883516+0000","last_became_active":"2024-02-18T00:45:35.148086+0000","last_became_peered":"2024-02-18T00:45:35.148086+0000","last_unstale":"2024-02-18T00:45:58.883516+0000","last_undegraded":"2024-02-18T00:45:58.883516+0000","last_fullsized":"2024-02-18T00:45:58.883516+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.556181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.556181+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":716,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883568+0000","last_change":"2024-02-18T00:43:37.205168+0000","last_active":"2024-02-18T00:45:58.883568+0000","last_peered":"2024-02-18T00:45:58.883568+0000","last_clean":"2024-02-18T00:45:58.883568+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:45:58.883568+0000","last_undegraded":"2024-02-18T00:45:58.883568+0000","last_fullsized":"2024-02-18T00:45:58.883568+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:37.205089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.205089+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.21","version":"457'34","reported_seq":135,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871372+0000","last_change":"2024-02-18T00:44:41.710983+0000","last_active":"2024-02-18T00:45:58.871372+0000","last_peered":"2024-02-18T00:45:58.871372+0000","last_clean":"2024-02-18T00:45:58.871372+0000","last_became_active":"2024-02-18T00:43:44.817896+0000","last_became_peered":"2024-02-18T00:43:44.817896+0000","last_unstale":"2024-02-18T00:45:58.871372+0000","last_undegraded":"2024-02-18T00:45:58.871372+0000","last_fullsized":"2024-02-18T00:45:58.871372+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:44:41.710910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:41.710910+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":335,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875584+0000","last_change":"2024-02-18T00:45:29.170854+0000","last_active":"2024-02-18T00:45:58.875584+0000","last_peered":"2024-02-18T00:45:58.875584+0000","last_clean":"2024-02-18T00:45:58.875584+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:45:58.875584+0000","last_undegraded":"2024-02-18T00:45:58.875584+0000","last_fullsized":"2024-02-18T00:45:58.875584+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:29.170786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:45:29.170786+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.20","version":"457'52","reported_seq":42,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.861895+0000","last_change":"2024-02-18T00:45:35.175468+0000","last_active":"2024-02-18T00:45:58.861895+0000","last_peered":"2024-02-18T00:45:58.861895+0000","last_clean":"2024-02-18T00:45:58.861895+0000","last_became_active":"2024-02-18T00:45:35.140467+0000","last_became_peered":"2024-02-18T00:45:35.140467+0000","last_unstale":"2024-02-18T00:45:58.861895+0000","last_undegraded":"2024-02-18T00:45:58.861895+0000","last_fullsized":"2024-02-18T00:45:58.861895+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:43:52.539763+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.539763+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":344,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875373+0000","last_change":"2024-02-18T00:45:38.052386+0000","last_active":"2024-02-18T00:45:58.875373+0000","last_peered":"2024-02-18T00:45:58.875373+0000","last_clean":"2024-02-18T00:45:58.875373+0000","last_became_active":"2024-02-18T00:45:35.161804+0000","last_became_peered":"2024-02-18T00:45:35.161804+0000","last_unstale":"2024-02-18T00:45:58.875373+0000","last_undegraded":"2024-02-18T00:45:58.875373+0000","last_fullsized":"2024-02-18T00:45:58.875373+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:38.052301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:53.345872+0000","last_clean_scrub_stamp":"2024-02-18T00:45:38.052301+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.23","version":"734'38","reported_seq":174,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884340+0000","last_change":"2024-02-18T00:44:43.236041+0000","last_active":"2024-02-18T00:45:58.884340+0000","last_peered":"2024-02-18T00:45:58.884340+0000","last_clean":"2024-02-18T00:45:58.884340+0000","last_became_active":"2024-02-18T00:43:09.796063+0000","last_became_peered":"2024-02-18T00:43:09.796063+0000","last_unstale":"2024-02-18T00:45:58.884340+0000","last_undegraded":"2024-02-18T00:45:58.884340+0000","last_fullsized":"2024-02-18T00:45:58.884340+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"734'38","last_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_deep_scrub":"734'38","last_deep_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.235973+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":738,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868983+0000","last_change":"2024-02-18T00:45:35.332463+0000","last_active":"2024-02-18T00:45:58.868983+0000","last_peered":"2024-02-18T00:45:58.868983+0000","last_clean":"2024-02-18T00:45:58.868983+0000","last_became_active":"2024-02-18T00:45:35.146341+0000","last_became_peered":"2024-02-18T00:45:35.146341+0000","last_unstale":"2024-02-18T00:45:58.868983+0000","last_undegraded":"2024-02-18T00:45:58.868983+0000","last_fullsized":"2024-02-18T00:45:58.868983+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:43:44.566261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.566261+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":873,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.22","version":"457'32","reported_seq":639,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868269+0000","last_change":"2024-02-18T00:45:35.205792+0000","last_active":"2024-02-18T00:45:58.868269+0000","last_peered":"2024-02-18T00:45:58.868269+0000","last_clean":"2024-02-18T00:45:58.868269+0000","last_became_active":"2024-02-18T00:45:35.142563+0000","last_became_peered":"2024-02-18T00:45:35.142563+0000","last_unstale":"2024-02-18T00:45:58.868269+0000","last_undegraded":"2024-02-18T00:45:58.868269+0000","last_fullsized":"2024-02-18T00:45:58.868269+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:44:32.659299+0000","last_deep_scrub":"102'22","last_deep_scrub_stamp":"2024-02-18T00:32:49.398867+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.659299+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":420,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863818+0000","last_change":"2024-02-18T00:45:42.829148+0000","last_active":"2024-02-18T00:45:58.863818+0000","last_peered":"2024-02-18T00:45:58.863818+0000","last_clean":"2024-02-18T00:45:58.863818+0000","last_became_active":"2024-02-18T00:45:35.160558+0000","last_became_peered":"2024-02-18T00:45:35.160558+0000","last_unstale":"2024-02-18T00:45:58.863818+0000","last_undegraded":"2024-02-18T00:45:58.863818+0000","last_fullsized":"2024-02-18T00:45:58.863818+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:42.829077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:42.829077+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"457'26","reported_seq":175,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872086+0000","last_change":"2024-02-18T00:45:44.414886+0000","last_active":"2024-02-18T00:45:58.872086+0000","last_peered":"2024-02-18T00:45:58.872086+0000","last_clean":"2024-02-18T00:45:58.872086+0000","last_became_active":"2024-02-18T00:43:09.791838+0000","last_became_peered":"2024-02-18T00:43:09.791838+0000","last_unstale":"2024-02-18T00:45:58.872086+0000","last_undegraded":"2024-02-18T00:45:58.872086+0000","last_fullsized":"2024-02-18T00:45:58.872086+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:48.727636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:48.727636+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":712,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884420+0000","last_change":"2024-02-18T00:45:35.149969+0000","last_active":"2024-02-18T00:45:58.884420+0000","last_peered":"2024-02-18T00:45:58.884420+0000","last_clean":"2024-02-18T00:45:58.884420+0000","last_became_active":"2024-02-18T00:45:35.149579+0000","last_became_peered":"2024-02-18T00:45:35.149579+0000","last_unstale":"2024-02-18T00:45:58.884420+0000","last_undegraded":"2024-02-18T00:45:58.884420+0000","last_fullsized":"2024-02-18T00:45:58.884420+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:33.297914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:33.297914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"457'47","reported_seq":210,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863661+0000","last_change":"2024-02-18T00:44:26.097493+0000","last_active":"2024-02-18T00:45:58.863661+0000","last_peered":"2024-02-18T00:45:58.863661+0000","last_clean":"2024-02-18T00:45:58.863661+0000","last_became_active":"2024-02-18T00:43:09.789985+0000","last_became_peered":"2024-02-18T00:43:09.789985+0000","last_unstale":"2024-02-18T00:45:58.863661+0000","last_undegraded":"2024-02-18T00:45:58.863661+0000","last_fullsized":"2024-02-18T00:45:58.863661+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:44:26.097373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:26.097373+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":35,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868316+0000","last_change":"2024-02-18T00:45:35.140860+0000","last_active":"2024-02-18T00:45:58.868316+0000","last_peered":"2024-02-18T00:45:58.868316+0000","last_clean":"2024-02-18T00:45:58.868316+0000","last_became_active":"2024-02-18T00:45:35.140264+0000","last_became_peered":"2024-02-18T00:45:35.140264+0000","last_unstale":"2024-02-18T00:45:58.868316+0000","last_undegraded":"2024-02-18T00:45:58.868316+0000","last_fullsized":"2024-02-18T00:45:58.868316+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.426814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.426814+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"457'23","reported_seq":861,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864610+0000","last_change":"2024-02-18T00:45:35.701243+0000","last_active":"2024-02-18T00:45:58.864610+0000","last_peered":"2024-02-18T00:45:58.864610+0000","last_clean":"2024-02-18T00:45:58.864610+0000","last_became_active":"2024-02-18T00:45:35.162377+0000","last_became_peered":"2024-02-18T00:45:35.162377+0000","last_unstale":"2024-02-18T00:45:58.864610+0000","last_undegraded":"2024-02-18T00:45:58.864610+0000","last_fullsized":"2024-02-18T00:45:58.864610+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:44:24.180458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:24.180458+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":801,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862869+0000","last_change":"2024-02-18T00:45:20.755268+0000","last_active":"2024-02-18T00:45:58.862869+0000","last_peered":"2024-02-18T00:45:58.862869+0000","last_clean":"2024-02-18T00:45:58.862869+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:45:58.862869+0000","last_undegraded":"2024-02-18T00:45:58.862869+0000","last_fullsized":"2024-02-18T00:45:58.862869+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:20.755190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_clean_scrub_stamp":"2024-02-18T00:45:20.755190+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"457'24","reported_seq":643,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863147+0000","last_change":"2024-02-18T00:45:35.175395+0000","last_active":"2024-02-18T00:45:58.863147+0000","last_peered":"2024-02-18T00:45:58.863147+0000","last_clean":"2024-02-18T00:45:58.863147+0000","last_became_active":"2024-02-18T00:45:35.158749+0000","last_became_peered":"2024-02-18T00:45:35.158749+0000","last_unstale":"2024-02-18T00:45:58.863147+0000","last_undegraded":"2024-02-18T00:45:58.863147+0000","last_fullsized":"2024-02-18T00:45:58.863147+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:43:51.009575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.009575+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":344,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874462+0000","last_change":"2024-02-18T00:45:35.151270+0000","last_active":"2024-02-18T00:45:58.874462+0000","last_peered":"2024-02-18T00:45:58.874462+0000","last_clean":"2024-02-18T00:45:58.874462+0000","last_became_active":"2024-02-18T00:45:35.150849+0000","last_became_peered":"2024-02-18T00:45:35.150849+0000","last_unstale":"2024-02-18T00:45:58.874462+0000","last_undegraded":"2024-02-18T00:45:58.874462+0000","last_fullsized":"2024-02-18T00:45:58.874462+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:28.218409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:45:28.218409+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":731,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889218+0000","last_change":"2024-02-18T00:45:38.159144+0000","last_active":"2024-02-18T00:45:58.889218+0000","last_peered":"2024-02-18T00:45:58.889218+0000","last_clean":"2024-02-18T00:45:58.889218+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:45:58.889218+0000","last_undegraded":"2024-02-18T00:45:58.889218+0000","last_fullsized":"2024-02-18T00:45:58.889218+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:38.159076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_clean_scrub_stamp":"2024-02-18T00:45:38.159076+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"774'12","reported_seq":430,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874794+0000","last_change":"2024-02-18T00:45:35.174483+0000","last_active":"2024-02-18T00:45:58.874794+0000","last_peered":"2024-02-18T00:45:58.874794+0000","last_clean":"2024-02-18T00:45:58.874794+0000","last_became_active":"2024-02-18T00:45:35.149887+0000","last_became_peered":"2024-02-18T00:45:35.149887+0000","last_unstale":"2024-02-18T00:45:58.874794+0000","last_undegraded":"2024-02-18T00:45:58.874794+0000","last_fullsized":"2024-02-18T00:45:58.874794+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"736'11","last_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_deep_scrub":"736'11","last_deep_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.265972+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":921,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871355+0000","last_change":"2024-02-18T00:45:35.176275+0000","last_active":"2024-02-18T00:45:58.871355+0000","last_peered":"2024-02-18T00:45:58.871355+0000","last_clean":"2024-02-18T00:45:58.871355+0000","last_became_active":"2024-02-18T00:45:35.163895+0000","last_became_peered":"2024-02-18T00:45:35.163895+0000","last_unstale":"2024-02-18T00:45:58.871355+0000","last_undegraded":"2024-02-18T00:45:58.871355+0000","last_fullsized":"2024-02-18T00:45:58.871355+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:43:43.645800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.645800+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":794,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862675+0000","last_change":"2024-02-18T00:45:28.563996+0000","last_active":"2024-02-18T00:45:58.862675+0000","last_peered":"2024-02-18T00:45:58.862675+0000","last_clean":"2024-02-18T00:45:58.862675+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:45:58.862675+0000","last_undegraded":"2024-02-18T00:45:58.862675+0000","last_fullsized":"2024-02-18T00:45:58.862675+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:28.563932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:28.563932+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":787,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871614+0000","last_change":"2024-02-18T00:45:35.141237+0000","last_active":"2024-02-18T00:45:58.871614+0000","last_peered":"2024-02-18T00:45:58.871614+0000","last_clean":"2024-02-18T00:45:58.871614+0000","last_became_active":"2024-02-18T00:45:35.140901+0000","last_became_peered":"2024-02-18T00:45:35.140901+0000","last_unstale":"2024-02-18T00:45:58.871614+0000","last_undegraded":"2024-02-18T00:45:58.871614+0000","last_fullsized":"2024-02-18T00:45:58.871614+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:30.739087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:30.739087+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":716,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864305+0000","last_change":"2024-02-18T00:44:47.967541+0000","last_active":"2024-02-18T00:45:58.864305+0000","last_peered":"2024-02-18T00:45:58.864305+0000","last_clean":"2024-02-18T00:45:58.864305+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:45:58.864305+0000","last_undegraded":"2024-02-18T00:45:58.864305+0000","last_fullsized":"2024-02-18T00:45:58.864305+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.967459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.967459+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":829,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.888784+0000","last_change":"2024-02-18T00:45:30.248902+0000","last_active":"2024-02-18T00:45:58.888784+0000","last_peered":"2024-02-18T00:45:58.888784+0000","last_clean":"2024-02-18T00:45:58.888784+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:45:58.888784+0000","last_undegraded":"2024-02-18T00:45:58.888784+0000","last_fullsized":"2024-02-18T00:45:58.888784+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:45:30.248834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:30.248834+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":774,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865426+0000","last_change":"2024-02-18T00:43:43.927240+0000","last_active":"2024-02-18T00:45:58.865426+0000","last_peered":"2024-02-18T00:45:58.865426+0000","last_clean":"2024-02-18T00:45:58.865426+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:45:58.865426+0000","last_undegraded":"2024-02-18T00:45:58.865426+0000","last_fullsized":"2024-02-18T00:45:58.865426+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:43.927169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.927169+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":770,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864004+0000","last_change":"2024-02-18T00:45:31.663259+0000","last_active":"2024-02-18T00:45:58.864004+0000","last_peered":"2024-02-18T00:45:58.864004+0000","last_clean":"2024-02-18T00:45:58.864004+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:45:58.864004+0000","last_undegraded":"2024-02-18T00:45:58.864004+0000","last_fullsized":"2024-02-18T00:45:58.864004+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:31.663180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:45:31.663180+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":813,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889398+0000","last_change":"2024-02-18T00:43:35.061129+0000","last_active":"2024-02-18T00:45:58.889398+0000","last_peered":"2024-02-18T00:45:58.889398+0000","last_clean":"2024-02-18T00:45:58.889398+0000","last_became_active":"2024-02-18T00:43:35.060523+0000","last_became_peered":"2024-02-18T00:43:35.060523+0000","last_unstale":"2024-02-18T00:45:58.889398+0000","last_undegraded":"2024-02-18T00:45:58.889398+0000","last_fullsized":"2024-02-18T00:45:58.889398+0000","mapping_epoch":668,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":669,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:33.517952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:33.517952+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":718,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884425+0000","last_change":"2024-02-18T00:44:24.230876+0000","last_active":"2024-02-18T00:45:58.884425+0000","last_peered":"2024-02-18T00:45:58.884425+0000","last_clean":"2024-02-18T00:45:58.884425+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:45:58.884425+0000","last_undegraded":"2024-02-18T00:45:58.884425+0000","last_fullsized":"2024-02-18T00:45:58.884425+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:24.230809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:44:24.230809+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"457'42","reported_seq":580,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863245+0000","last_change":"2024-02-18T00:45:36.473929+0000","last_active":"2024-02-18T00:45:58.863245+0000","last_peered":"2024-02-18T00:45:58.863245+0000","last_clean":"2024-02-18T00:45:58.863245+0000","last_became_active":"2024-02-18T00:45:35.160147+0000","last_became_peered":"2024-02-18T00:45:35.160147+0000","last_unstale":"2024-02-18T00:45:58.863245+0000","last_undegraded":"2024-02-18T00:45:58.863245+0000","last_fullsized":"2024-02-18T00:45:58.863245+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":772,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'42","last_scrub_stamp":"2024-02-18T00:44:46.441505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.441505+0000","log_size":42,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,5],"acting":[1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":774,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864710+0000","last_change":"2024-02-18T00:45:36.716737+0000","last_active":"2024-02-18T00:45:58.864710+0000","last_peered":"2024-02-18T00:45:58.864710+0000","last_clean":"2024-02-18T00:45:58.864710+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:45:58.864710+0000","last_undegraded":"2024-02-18T00:45:58.864710+0000","last_fullsized":"2024-02-18T00:45:58.864710+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:36.716673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:45:36.716673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":765,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865119+0000","last_change":"2024-02-18T00:43:03.732009+0000","last_active":"2024-02-18T00:45:58.865119+0000","last_peered":"2024-02-18T00:45:58.865119+0000","last_clean":"2024-02-18T00:45:58.865119+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:45:58.865119+0000","last_undegraded":"2024-02-18T00:45:58.865119+0000","last_fullsized":"2024-02-18T00:45:58.865119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:03.731917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:03.731917+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":353,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874568+0000","last_change":"2024-02-18T00:44:43.296521+0000","last_active":"2024-02-18T00:45:58.874568+0000","last_peered":"2024-02-18T00:45:58.874568+0000","last_clean":"2024-02-18T00:45:58.874568+0000","last_became_active":"2024-02-18T00:41:53.837106+0000","last_became_peered":"2024-02-18T00:41:53.837106+0000","last_unstale":"2024-02-18T00:45:58.874568+0000","last_undegraded":"2024-02-18T00:45:58.874568+0000","last_fullsized":"2024-02-18T00:45:58.874568+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.296447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.296447+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":706,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871482+0000","last_change":"2024-02-18T00:45:41.475891+0000","last_active":"2024-02-18T00:45:58.871482+0000","last_peered":"2024-02-18T00:45:58.871482+0000","last_clean":"2024-02-18T00:45:58.871482+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:45:58.871482+0000","last_undegraded":"2024-02-18T00:45:58.871482+0000","last_fullsized":"2024-02-18T00:45:58.871482+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:41.475825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:45:41.475825+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":829,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868893+0000","last_change":"2024-02-18T00:43:12.336661+0000","last_active":"2024-02-18T00:45:58.868893+0000","last_peered":"2024-02-18T00:45:58.868893+0000","last_clean":"2024-02-18T00:45:58.868893+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:45:58.868893+0000","last_undegraded":"2024-02-18T00:45:58.868893+0000","last_fullsized":"2024-02-18T00:45:58.868893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:43:12.336565+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:43:12.336565+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":335,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875046+0000","last_change":"2024-02-18T00:45:44.093213+0000","last_active":"2024-02-18T00:45:58.875046+0000","last_peered":"2024-02-18T00:45:58.875046+0000","last_clean":"2024-02-18T00:45:58.875046+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:45:58.875046+0000","last_undegraded":"2024-02-18T00:45:58.875046+0000","last_fullsized":"2024-02-18T00:45:58.875046+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:44.093141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:45:44.093141+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":770,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864077+0000","last_change":"2024-02-18T00:45:22.911985+0000","last_active":"2024-02-18T00:45:58.864077+0000","last_peered":"2024-02-18T00:45:58.864077+0000","last_clean":"2024-02-18T00:45:58.864077+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:45:58.864077+0000","last_undegraded":"2024-02-18T00:45:58.864077+0000","last_fullsized":"2024-02-18T00:45:58.864077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:22.911912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:45:22.911912+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":781,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864075+0000","last_change":"2024-02-18T00:43:35.108868+0000","last_active":"2024-02-18T00:45:58.864075+0000","last_peered":"2024-02-18T00:45:58.864075+0000","last_clean":"2024-02-18T00:45:58.864075+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:45:58.864075+0000","last_undegraded":"2024-02-18T00:45:58.864075+0000","last_fullsized":"2024-02-18T00:45:58.864075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:35.108761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.108761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":180,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.888517+0000","last_change":"2024-02-18T00:44:48.550165+0000","last_active":"2024-02-18T00:45:58.888517+0000","last_peered":"2024-02-18T00:45:58.888517+0000","last_clean":"2024-02-18T00:45:58.888517+0000","last_became_active":"2024-02-18T00:42:38.553174+0000","last_became_peered":"2024-02-18T00:42:38.553174+0000","last_unstale":"2024-02-18T00:45:58.888517+0000","last_undegraded":"2024-02-18T00:45:58.888517+0000","last_fullsized":"2024-02-18T00:45:58.888517+0000","mapping_epoch":623,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:48.550082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:48.550082+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":842,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868316+0000","last_change":"2024-02-18T00:45:35.635841+0000","last_active":"2024-02-18T00:45:58.868316+0000","last_peered":"2024-02-18T00:45:58.868316+0000","last_clean":"2024-02-18T00:45:58.868316+0000","last_became_active":"2024-02-18T00:45:35.145092+0000","last_became_peered":"2024-02-18T00:45:35.145092+0000","last_unstale":"2024-02-18T00:45:58.868316+0000","last_undegraded":"2024-02-18T00:45:58.868316+0000","last_fullsized":"2024-02-18T00:45:58.868316+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:45:30.988259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:30.988259+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":794,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863326+0000","last_change":"2024-02-18T00:44:37.109916+0000","last_active":"2024-02-18T00:45:58.863326+0000","last_peered":"2024-02-18T00:45:58.863326+0000","last_clean":"2024-02-18T00:45:58.863326+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:45:58.863326+0000","last_undegraded":"2024-02-18T00:45:58.863326+0000","last_fullsized":"2024-02-18T00:45:58.863326+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.109800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.109800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":1005,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863459+0000","last_change":"2024-02-18T00:43:32.077585+0000","last_active":"2024-02-18T00:45:58.863459+0000","last_peered":"2024-02-18T00:45:58.863459+0000","last_clean":"2024-02-18T00:45:58.863459+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:45:58.863459+0000","last_undegraded":"2024-02-18T00:45:58.863459+0000","last_fullsized":"2024-02-18T00:45:58.863459+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:43:32.077472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.077472+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":784,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868472+0000","last_change":"2024-02-18T00:45:41.914842+0000","last_active":"2024-02-18T00:45:58.868472+0000","last_peered":"2024-02-18T00:45:58.868472+0000","last_clean":"2024-02-18T00:45:58.868472+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:45:58.868472+0000","last_undegraded":"2024-02-18T00:45:58.868472+0000","last_fullsized":"2024-02-18T00:45:58.868472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:41.914780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:45:41.914780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":723,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863635+0000","last_change":"2024-02-18T00:45:35.159654+0000","last_active":"2024-02-18T00:45:58.863635+0000","last_peered":"2024-02-18T00:45:58.863635+0000","last_clean":"2024-02-18T00:45:58.863635+0000","last_became_active":"2024-02-18T00:45:35.159282+0000","last_became_peered":"2024-02-18T00:45:35.159282+0000","last_unstale":"2024-02-18T00:45:58.863635+0000","last_undegraded":"2024-02-18T00:45:58.863635+0000","last_fullsized":"2024-02-18T00:45:58.863635+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:45.071835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.071835+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":790,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872199+0000","last_change":"2024-02-18T00:45:33.609051+0000","last_active":"2024-02-18T00:45:58.872199+0000","last_peered":"2024-02-18T00:45:58.872199+0000","last_clean":"2024-02-18T00:45:58.872199+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:45:58.872199+0000","last_undegraded":"2024-02-18T00:45:58.872199+0000","last_fullsized":"2024-02-18T00:45:58.872199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:45:33.608947+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:45:33.608947+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":801,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865533+0000","last_change":"2024-02-18T00:44:28.072377+0000","last_active":"2024-02-18T00:45:58.865533+0000","last_peered":"2024-02-18T00:45:58.865533+0000","last_clean":"2024-02-18T00:45:58.865533+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:45:58.865533+0000","last_undegraded":"2024-02-18T00:45:58.865533+0000","last_fullsized":"2024-02-18T00:45:58.865533+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:28.072271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.072271+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":977,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868388+0000","last_change":"2024-02-18T00:45:34.983056+0000","last_active":"2024-02-18T00:45:58.868388+0000","last_peered":"2024-02-18T00:45:58.868388+0000","last_clean":"2024-02-18T00:45:58.868388+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:45:58.868388+0000","last_undegraded":"2024-02-18T00:45:58.868388+0000","last_fullsized":"2024-02-18T00:45:58.868388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:45:34.982928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:45:34.982928+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":773,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872336+0000","last_change":"2024-02-18T00:43:38.758186+0000","last_active":"2024-02-18T00:45:58.872336+0000","last_peered":"2024-02-18T00:45:58.872336+0000","last_clean":"2024-02-18T00:45:58.872336+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:45:58.872336+0000","last_undegraded":"2024-02-18T00:45:58.872336+0000","last_fullsized":"2024-02-18T00:45:58.872336+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:38.758071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:43:38.758071+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":707,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868490+0000","last_change":"2024-02-18T00:43:36.573041+0000","last_active":"2024-02-18T00:45:58.868490+0000","last_peered":"2024-02-18T00:45:58.868490+0000","last_clean":"2024-02-18T00:45:58.868490+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:45:58.868490+0000","last_undegraded":"2024-02-18T00:45:58.868490+0000","last_fullsized":"2024-02-18T00:45:58.868490+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:36.572964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:43:36.572964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"457'25","reported_seq":862,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868257+0000","last_change":"2024-02-18T00:45:35.698455+0000","last_active":"2024-02-18T00:45:58.868257+0000","last_peered":"2024-02-18T00:45:58.868257+0000","last_clean":"2024-02-18T00:45:58.868257+0000","last_became_active":"2024-02-18T00:45:35.144073+0000","last_became_peered":"2024-02-18T00:45:35.144073+0000","last_unstale":"2024-02-18T00:45:58.868257+0000","last_undegraded":"2024-02-18T00:45:58.868257+0000","last_fullsized":"2024-02-18T00:45:58.868257+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:44:23.568512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:23.568512+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3],"acting":[4,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":342,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862003+0000","last_change":"2024-02-18T00:45:39.682750+0000","last_active":"2024-02-18T00:45:58.862003+0000","last_peered":"2024-02-18T00:45:58.862003+0000","last_clean":"2024-02-18T00:45:58.862003+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:45:58.862003+0000","last_undegraded":"2024-02-18T00:45:58.862003+0000","last_fullsized":"2024-02-18T00:45:58.862003+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:39.682684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:39.682684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":771,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872317+0000","last_change":"2024-02-18T00:45:35.159267+0000","last_active":"2024-02-18T00:45:58.872317+0000","last_peered":"2024-02-18T00:45:58.872317+0000","last_clean":"2024-02-18T00:45:58.872317+0000","last_became_active":"2024-02-18T00:45:35.158941+0000","last_became_peered":"2024-02-18T00:45:35.158941+0000","last_unstale":"2024-02-18T00:45:58.872317+0000","last_undegraded":"2024-02-18T00:45:58.872317+0000","last_fullsized":"2024-02-18T00:45:58.872317+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:25.764907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.764907+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":784,"reported_epoch":790,"state":"active+remapped+backfill_toofull","last_fresh":"2024-02-18T00:45:58.884808+0000","last_change":"2024-02-18T00:45:36.132469+0000","last_active":"2024-02-18T00:45:58.884808+0000","last_peered":"2024-02-18T00:45:58.884808+0000","last_clean":"2024-02-18T00:45:33.117831+0000","last_became_active":"2024-02-18T00:45:36.131553+0000","last_became_peered":"2024-02-18T00:45:36.131553+0000","last_unstale":"2024-02-18T00:45:58.884808+0000","last_undegraded":"2024-02-18T00:45:58.884808+0000","last_fullsized":"2024-02-18T00:45:58.884808+0000","mapping_epoch":771,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":564,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:43:42.104363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:42.104363+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":369,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,6],"avail_no_missing":["3","5","6"],"object_location_counts":[{"shards":"3,5,6","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":707,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868660+0000","last_change":"2024-02-18T00:43:00.314164+0000","last_active":"2024-02-18T00:45:58.868660+0000","last_peered":"2024-02-18T00:45:58.868660+0000","last_clean":"2024-02-18T00:45:58.868660+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:45:58.868660+0000","last_undegraded":"2024-02-18T00:45:58.868660+0000","last_fullsized":"2024-02-18T00:45:58.868660+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:00.314091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.314091+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":966,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883779+0000","last_change":"2024-02-18T00:43:05.211680+0000","last_active":"2024-02-18T00:45:58.883779+0000","last_peered":"2024-02-18T00:45:58.883779+0000","last_clean":"2024-02-18T00:45:58.883779+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:45:58.883779+0000","last_undegraded":"2024-02-18T00:45:58.883779+0000","last_fullsized":"2024-02-18T00:45:58.883779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_clean_scrub_stamp":"2024-02-18T00:43:05.211604+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":856,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.867909+0000","last_change":"2024-02-18T00:45:35.175706+0000","last_active":"2024-02-18T00:45:58.867909+0000","last_peered":"2024-02-18T00:45:58.867909+0000","last_clean":"2024-02-18T00:45:58.867909+0000","last_became_active":"2024-02-18T00:45:35.141322+0000","last_became_peered":"2024-02-18T00:45:35.141322+0000","last_unstale":"2024-02-18T00:45:58.867909+0000","last_undegraded":"2024-02-18T00:45:58.867909+0000","last_fullsized":"2024-02-18T00:45:58.867909+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:44:28.633078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.633078+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":486,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875511+0000","last_change":"2024-02-18T00:45:35.156540+0000","last_active":"2024-02-18T00:45:58.875511+0000","last_peered":"2024-02-18T00:45:58.875511+0000","last_clean":"2024-02-18T00:45:58.875511+0000","last_became_active":"2024-02-18T00:45:35.156213+0000","last_became_peered":"2024-02-18T00:45:35.156213+0000","last_unstale":"2024-02-18T00:45:58.875511+0000","last_undegraded":"2024-02-18T00:45:58.875511+0000","last_fullsized":"2024-02-18T00:45:58.875511+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:24.791379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:45:24.791379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":344,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875046+0000","last_change":"2024-02-18T00:45:35.152101+0000","last_active":"2024-02-18T00:45:58.875046+0000","last_peered":"2024-02-18T00:45:58.875046+0000","last_clean":"2024-02-18T00:45:58.875046+0000","last_became_active":"2024-02-18T00:45:35.151648+0000","last_became_peered":"2024-02-18T00:45:35.151648+0000","last_unstale":"2024-02-18T00:45:58.875046+0000","last_undegraded":"2024-02-18T00:45:58.875046+0000","last_fullsized":"2024-02-18T00:45:58.875046+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:40.022788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:40.022788+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":706,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872113+0000","last_change":"2024-02-18T00:45:21.691843+0000","last_active":"2024-02-18T00:45:58.872113+0000","last_peered":"2024-02-18T00:45:58.872113+0000","last_clean":"2024-02-18T00:45:58.872113+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:45:58.872113+0000","last_undegraded":"2024-02-18T00:45:58.872113+0000","last_fullsized":"2024-02-18T00:45:58.872113+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:21.691770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:08.831934+0000","last_clean_scrub_stamp":"2024-02-18T00:45:21.691770+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":786,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868213+0000","last_change":"2024-02-18T00:44:38.710273+0000","last_active":"2024-02-18T00:45:58.868213+0000","last_peered":"2024-02-18T00:45:58.868213+0000","last_clean":"2024-02-18T00:45:58.868213+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:45:58.868213+0000","last_undegraded":"2024-02-18T00:45:58.868213+0000","last_fullsized":"2024-02-18T00:45:58.868213+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.710196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.710196+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":925,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883757+0000","last_change":"2024-02-18T00:44:32.283711+0000","last_active":"2024-02-18T00:45:58.883757+0000","last_peered":"2024-02-18T00:45:58.883757+0000","last_clean":"2024-02-18T00:45:58.883757+0000","last_became_active":"2024-02-18T00:43:07.799144+0000","last_became_peered":"2024-02-18T00:43:07.799144+0000","last_unstale":"2024-02-18T00:45:58.883757+0000","last_undegraded":"2024-02-18T00:45:58.883757+0000","last_fullsized":"2024-02-18T00:45:58.883757+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:44:32.283637+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.283637+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":810,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863263+0000","last_change":"2024-02-18T00:45:35.150020+0000","last_active":"2024-02-18T00:45:58.863263+0000","last_peered":"2024-02-18T00:45:58.863263+0000","last_clean":"2024-02-18T00:45:58.863263+0000","last_became_active":"2024-02-18T00:45:35.149345+0000","last_became_peered":"2024-02-18T00:45:35.149345+0000","last_unstale":"2024-02-18T00:45:58.863263+0000","last_undegraded":"2024-02-18T00:45:58.863263+0000","last_fullsized":"2024-02-18T00:45:58.863263+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.073800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.073800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":766,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868706+0000","last_change":"2024-02-18T00:45:39.922312+0000","last_active":"2024-02-18T00:45:58.868706+0000","last_peered":"2024-02-18T00:45:58.868706+0000","last_clean":"2024-02-18T00:45:58.868706+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:45:58.868706+0000","last_undegraded":"2024-02-18T00:45:58.868706+0000","last_fullsized":"2024-02-18T00:45:58.868706+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:39.922251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:45:39.922251+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":700,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884732+0000","last_change":"2024-02-18T00:44:31.284979+0000","last_active":"2024-02-18T00:45:58.884732+0000","last_peered":"2024-02-18T00:45:58.884732+0000","last_clean":"2024-02-18T00:45:58.884732+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:45:58.884732+0000","last_undegraded":"2024-02-18T00:45:58.884732+0000","last_fullsized":"2024-02-18T00:45:58.884732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:31.284910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.284910+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":32,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889023+0000","last_change":"2024-02-18T00:45:35.149957+0000","last_active":"2024-02-18T00:45:58.889023+0000","last_peered":"2024-02-18T00:45:58.889023+0000","last_clean":"2024-02-18T00:45:58.889023+0000","last_became_active":"2024-02-18T00:45:35.148940+0000","last_became_peered":"2024-02-18T00:45:35.148940+0000","last_unstale":"2024-02-18T00:45:58.889023+0000","last_undegraded":"2024-02-18T00:45:58.889023+0000","last_fullsized":"2024-02-18T00:45:58.889023+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:40.278560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:22.964687+0000","last_clean_scrub_stamp":"2024-02-18T00:44:40.278560+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":902,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.888675+0000","last_change":"2024-02-18T00:43:52.461877+0000","last_active":"2024-02-18T00:45:58.888675+0000","last_peered":"2024-02-18T00:45:58.888675+0000","last_clean":"2024-02-18T00:45:58.888675+0000","last_became_active":"2024-02-18T00:43:07.797505+0000","last_became_peered":"2024-02-18T00:43:07.797505+0000","last_unstale":"2024-02-18T00:45:58.888675+0000","last_undegraded":"2024-02-18T00:45:58.888675+0000","last_fullsized":"2024-02-18T00:45:58.888675+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:43:52.461819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.461819+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":337,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863368+0000","last_change":"2024-02-18T00:43:04.771039+0000","last_active":"2024-02-18T00:45:58.863368+0000","last_peered":"2024-02-18T00:45:58.863368+0000","last_clean":"2024-02-18T00:45:58.863368+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:45:58.863368+0000","last_undegraded":"2024-02-18T00:45:58.863368+0000","last_fullsized":"2024-02-18T00:45:58.863368+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:43:04.770880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:04.770880+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":777,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865636+0000","last_change":"2024-02-18T00:45:35.162584+0000","last_active":"2024-02-18T00:45:58.865636+0000","last_peered":"2024-02-18T00:45:58.865636+0000","last_clean":"2024-02-18T00:45:58.865636+0000","last_became_active":"2024-02-18T00:45:35.162232+0000","last_became_peered":"2024-02-18T00:45:35.162232+0000","last_unstale":"2024-02-18T00:45:58.865636+0000","last_undegraded":"2024-02-18T00:45:58.865636+0000","last_fullsized":"2024-02-18T00:45:58.865636+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.082853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.082853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":700,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884847+0000","last_change":"2024-02-18T00:44:37.330430+0000","last_active":"2024-02-18T00:45:58.884847+0000","last_peered":"2024-02-18T00:45:58.884847+0000","last_clean":"2024-02-18T00:45:58.884847+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:45:58.884847+0000","last_undegraded":"2024-02-18T00:45:58.884847+0000","last_fullsized":"2024-02-18T00:45:58.884847+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.330361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.330361+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":762,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872374+0000","last_change":"2024-02-18T00:44:49.712183+0000","last_active":"2024-02-18T00:45:58.872374+0000","last_peered":"2024-02-18T00:45:58.872374+0000","last_clean":"2024-02-18T00:45:58.872374+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:45:58.872374+0000","last_undegraded":"2024-02-18T00:45:58.872374+0000","last_fullsized":"2024-02-18T00:45:58.872374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:49.712086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:04.700147+0000","last_clean_scrub_stamp":"2024-02-18T00:44:49.712086+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"457'24","reported_seq":814,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889599+0000","last_change":"2024-02-18T00:44:25.533581+0000","last_active":"2024-02-18T00:45:58.889599+0000","last_peered":"2024-02-18T00:45:58.889599+0000","last_clean":"2024-02-18T00:45:58.889599+0000","last_became_active":"2024-02-18T00:43:07.799985+0000","last_became_peered":"2024-02-18T00:43:07.799985+0000","last_unstale":"2024-02-18T00:45:58.889599+0000","last_undegraded":"2024-02-18T00:45:58.889599+0000","last_fullsized":"2024-02-18T00:45:58.889599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:44:25.533453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.533453+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":337,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862040+0000","last_change":"2024-02-18T00:44:37.153891+0000","last_active":"2024-02-18T00:45:58.862040+0000","last_peered":"2024-02-18T00:45:58.862040+0000","last_clean":"2024-02-18T00:45:58.862040+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:45:58.862040+0000","last_undegraded":"2024-02-18T00:45:58.862040+0000","last_fullsized":"2024-02-18T00:45:58.862040+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.153787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.153787+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":769,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889240+0000","last_change":"2024-02-18T00:45:29.275779+0000","last_active":"2024-02-18T00:45:58.889240+0000","last_peered":"2024-02-18T00:45:58.889240+0000","last_clean":"2024-02-18T00:45:58.889240+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:45:58.889240+0000","last_undegraded":"2024-02-18T00:45:58.889240+0000","last_fullsized":"2024-02-18T00:45:58.889240+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:29.275710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:45:29.275710+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":763,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864166+0000","last_change":"2024-02-18T00:43:41.934547+0000","last_active":"2024-02-18T00:45:58.864166+0000","last_peered":"2024-02-18T00:45:58.864166+0000","last_clean":"2024-02-18T00:45:58.864166+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:45:58.864166+0000","last_undegraded":"2024-02-18T00:45:58.864166+0000","last_fullsized":"2024-02-18T00:45:58.864166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_clean_scrub_stamp":"2024-02-18T00:43:41.934482+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":185,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.861918+0000","last_change":"2024-02-18T00:45:35.152064+0000","last_active":"2024-02-18T00:45:58.861918+0000","last_peered":"2024-02-18T00:45:58.861918+0000","last_clean":"2024-02-18T00:45:58.861918+0000","last_became_active":"2024-02-18T00:45:35.151710+0000","last_became_peered":"2024-02-18T00:45:35.151710+0000","last_unstale":"2024-02-18T00:45:58.861918+0000","last_undegraded":"2024-02-18T00:45:58.861918+0000","last_fullsized":"2024-02-18T00:45:58.861918+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:08.893262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:08.893262+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":335,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875193+0000","last_change":"2024-02-18T00:44:33.157482+0000","last_active":"2024-02-18T00:45:58.875193+0000","last_peered":"2024-02-18T00:45:58.875193+0000","last_clean":"2024-02-18T00:45:58.875193+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:45:58.875193+0000","last_undegraded":"2024-02-18T00:45:58.875193+0000","last_fullsized":"2024-02-18T00:45:58.875193+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:33.157367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:44:33.157367+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"457'26","reported_seq":531,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868797+0000","last_change":"2024-02-18T00:45:35.973320+0000","last_active":"2024-02-18T00:45:58.868797+0000","last_peered":"2024-02-18T00:45:58.868797+0000","last_clean":"2024-02-18T00:45:58.868797+0000","last_became_active":"2024-02-18T00:45:35.158198+0000","last_became_peered":"2024-02-18T00:45:35.158198+0000","last_unstale":"2024-02-18T00:45:58.868797+0000","last_undegraded":"2024-02-18T00:45:58.868797+0000","last_fullsized":"2024-02-18T00:45:58.868797+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:51.588408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.588408+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":77,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":231,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":344,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874422+0000","last_change":"2024-02-18T00:45:35.160430+0000","last_active":"2024-02-18T00:45:58.874422+0000","last_peered":"2024-02-18T00:45:58.874422+0000","last_clean":"2024-02-18T00:45:58.874422+0000","last_became_active":"2024-02-18T00:45:35.160070+0000","last_became_peered":"2024-02-18T00:45:35.160070+0000","last_unstale":"2024-02-18T00:45:58.874422+0000","last_undegraded":"2024-02-18T00:45:58.874422+0000","last_fullsized":"2024-02-18T00:45:58.874422+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_clean_scrub_stamp":"2024-02-18T00:43:42.067810+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":802,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871699+0000","last_change":"2024-02-18T00:43:39.793439+0000","last_active":"2024-02-18T00:45:58.871699+0000","last_peered":"2024-02-18T00:45:58.871699+0000","last_clean":"2024-02-18T00:45:58.871699+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:45:58.871699+0000","last_undegraded":"2024-02-18T00:45:58.871699+0000","last_fullsized":"2024-02-18T00:45:58.871699+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.793345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.793345+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":768,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883789+0000","last_change":"2024-02-18T00:45:35.148748+0000","last_active":"2024-02-18T00:45:58.883789+0000","last_peered":"2024-02-18T00:45:58.883789+0000","last_clean":"2024-02-18T00:45:58.883789+0000","last_became_active":"2024-02-18T00:45:35.148407+0000","last_became_peered":"2024-02-18T00:45:35.148407+0000","last_unstale":"2024-02-18T00:45:58.883789+0000","last_undegraded":"2024-02-18T00:45:58.883789+0000","last_fullsized":"2024-02-18T00:45:58.883789+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.267177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.267177+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":778,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872550+0000","last_change":"2024-02-18T00:44:32.815550+0000","last_active":"2024-02-18T00:45:58.872550+0000","last_peered":"2024-02-18T00:45:58.872550+0000","last_clean":"2024-02-18T00:45:58.872550+0000","last_became_active":"2024-02-18T00:41:57.791956+0000","last_became_peered":"2024-02-18T00:41:57.791956+0000","last_unstale":"2024-02-18T00:45:58.872550+0000","last_undegraded":"2024-02-18T00:45:58.872550+0000","last_fullsized":"2024-02-18T00:45:58.872550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":584,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:32.815428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:44:32.815428+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":711,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889201+0000","last_change":"2024-02-18T00:45:42.059295+0000","last_active":"2024-02-18T00:45:58.889201+0000","last_peered":"2024-02-18T00:45:58.889201+0000","last_clean":"2024-02-18T00:45:58.889201+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:45:58.889201+0000","last_undegraded":"2024-02-18T00:45:58.889201+0000","last_fullsized":"2024-02-18T00:45:58.889201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:42.059232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_clean_scrub_stamp":"2024-02-18T00:45:42.059232+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":788,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862869+0000","last_change":"2024-02-18T00:45:30.659653+0000","last_active":"2024-02-18T00:45:58.862869+0000","last_peered":"2024-02-18T00:45:58.862869+0000","last_clean":"2024-02-18T00:45:58.862869+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:45:58.862869+0000","last_undegraded":"2024-02-18T00:45:58.862869+0000","last_fullsized":"2024-02-18T00:45:58.862869+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:30.659580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:45:30.659580+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"680'27","reported_seq":836,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864810+0000","last_change":"2024-02-18T00:43:45.989486+0000","last_active":"2024-02-18T00:45:58.864810+0000","last_peered":"2024-02-18T00:45:58.864810+0000","last_clean":"2024-02-18T00:45:58.864810+0000","last_became_active":"2024-02-18T00:43:07.801362+0000","last_became_peered":"2024-02-18T00:43:07.801362+0000","last_unstale":"2024-02-18T00:45:58.864810+0000","last_undegraded":"2024-02-18T00:45:58.864810+0000","last_fullsized":"2024-02-18T00:45:58.864810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"680'27","last_scrub_stamp":"2024-02-18T00:43:45.989427+0000","last_deep_scrub":"680'27","last_deep_scrub_stamp":"2024-02-18T00:43:45.989427+0000","last_clean_scrub_stamp":"2024-02-18T00:43:45.989427+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":783,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862573+0000","last_change":"2024-02-18T00:45:33.654096+0000","last_active":"2024-02-18T00:45:58.862573+0000","last_peered":"2024-02-18T00:45:58.862573+0000","last_clean":"2024-02-18T00:45:58.862573+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:45:58.862573+0000","last_undegraded":"2024-02-18T00:45:58.862573+0000","last_fullsized":"2024-02-18T00:45:58.862573+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:33.654028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:33.654028+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":1790,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:46:00.202075+0000","last_change":"2024-02-18T00:45:35.745108+0000","last_active":"2024-02-18T00:46:00.202075+0000","last_peered":"2024-02-18T00:46:00.202075+0000","last_clean":"2024-02-18T00:46:00.202075+0000","last_became_active":"2024-02-18T00:45:35.163038+0000","last_became_peered":"2024-02-18T00:45:35.163038+0000","last_unstale":"2024-02-18T00:46:00.202075+0000","last_undegraded":"2024-02-18T00:46:00.202075+0000","last_fullsized":"2024-02-18T00:46:00.202075+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:45:33.952076+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:45:33.952076+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":995,"num_read_kb":990,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":352,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875142+0000","last_change":"2024-02-18T00:45:35.179995+0000","last_active":"2024-02-18T00:45:58.875142+0000","last_peered":"2024-02-18T00:45:58.875142+0000","last_clean":"2024-02-18T00:45:58.875142+0000","last_became_active":"2024-02-18T00:45:35.153426+0000","last_became_peered":"2024-02-18T00:45:35.153426+0000","last_unstale":"2024-02-18T00:45:58.875142+0000","last_undegraded":"2024-02-18T00:45:58.875142+0000","last_fullsized":"2024-02-18T00:45:58.875142+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:45:34.073694+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:43:12.838938+0000","last_clean_scrub_stamp":"2024-02-18T00:45:34.073694+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":725,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868270+0000","last_change":"2024-02-18T00:43:52.430089+0000","last_active":"2024-02-18T00:45:58.868270+0000","last_peered":"2024-02-18T00:45:58.868270+0000","last_clean":"2024-02-18T00:45:58.868270+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:45:58.868270+0000","last_undegraded":"2024-02-18T00:45:58.868270+0000","last_fullsized":"2024-02-18T00:45:58.868270+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_clean_scrub_stamp":"2024-02-18T00:43:52.430010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":766,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889656+0000","last_change":"2024-02-18T00:45:32.238012+0000","last_active":"2024-02-18T00:45:58.889656+0000","last_peered":"2024-02-18T00:45:58.889656+0000","last_clean":"2024-02-18T00:45:58.889656+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:45:58.889656+0000","last_undegraded":"2024-02-18T00:45:58.889656+0000","last_fullsized":"2024-02-18T00:45:58.889656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:32.237913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:45:32.237913+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"682'49","reported_seq":738,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871821+0000","last_change":"2024-02-18T00:44:35.869910+0000","last_active":"2024-02-18T00:45:58.871821+0000","last_peered":"2024-02-18T00:45:58.871821+0000","last_clean":"2024-02-18T00:45:58.871821+0000","last_became_active":"2024-02-18T00:43:44.816299+0000","last_became_peered":"2024-02-18T00:43:44.816299+0000","last_unstale":"2024-02-18T00:45:58.871821+0000","last_undegraded":"2024-02-18T00:45:58.871821+0000","last_fullsized":"2024-02-18T00:45:58.871821+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":0,"last_scrub":"682'49","last_scrub_stamp":"2024-02-18T00:44:35.869834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.869834+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":248,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":784,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871720+0000","last_change":"2024-02-18T00:45:35.144280+0000","last_active":"2024-02-18T00:45:58.871720+0000","last_peered":"2024-02-18T00:45:58.871720+0000","last_clean":"2024-02-18T00:45:58.871720+0000","last_became_active":"2024-02-18T00:45:35.143854+0000","last_became_peered":"2024-02-18T00:45:35.143854+0000","last_unstale":"2024-02-18T00:45:58.871720+0000","last_undegraded":"2024-02-18T00:45:58.871720+0000","last_fullsized":"2024-02-18T00:45:58.871720+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.722660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:08.045946+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.722660+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":790,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884575+0000","last_change":"2024-02-18T00:45:21.375205+0000","last_active":"2024-02-18T00:45:58.884575+0000","last_peered":"2024-02-18T00:45:58.884575+0000","last_clean":"2024-02-18T00:45:58.884575+0000","last_became_active":"2024-02-18T00:42:02.125704+0000","last_became_peered":"2024-02-18T00:42:02.125704+0000","last_unstale":"2024-02-18T00:45:58.884575+0000","last_undegraded":"2024-02-18T00:45:58.884575+0000","last_fullsized":"2024-02-18T00:45:58.884575+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:21.375102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:45:21.375102+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":715,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889292+0000","last_change":"2024-02-18T00:45:35.143768+0000","last_active":"2024-02-18T00:45:58.889292+0000","last_peered":"2024-02-18T00:45:58.889292+0000","last_clean":"2024-02-18T00:45:58.889292+0000","last_became_active":"2024-02-18T00:45:35.143438+0000","last_became_peered":"2024-02-18T00:45:35.143438+0000","last_unstale":"2024-02-18T00:45:58.889292+0000","last_undegraded":"2024-02-18T00:45:58.889292+0000","last_fullsized":"2024-02-18T00:45:58.889292+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.459750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.459750+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"774'9","reported_seq":983,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.888817+0000","last_change":"2024-02-18T00:45:35.245914+0000","last_active":"2024-02-18T00:45:58.888817+0000","last_peered":"2024-02-18T00:45:58.888817+0000","last_clean":"2024-02-18T00:45:58.888817+0000","last_became_active":"2024-02-18T00:45:35.149314+0000","last_became_peered":"2024-02-18T00:45:35.149314+0000","last_unstale":"2024-02-18T00:45:58.888817+0000","last_undegraded":"2024-02-18T00:45:58.888817+0000","last_fullsized":"2024-02-18T00:45:58.888817+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:43:35.528567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:35.528567+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":793,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864233+0000","last_change":"2024-02-18T00:43:51.043051+0000","last_active":"2024-02-18T00:45:58.864233+0000","last_peered":"2024-02-18T00:45:58.864233+0000","last_clean":"2024-02-18T00:45:58.864233+0000","last_became_active":"2024-02-18T00:43:07.795433+0000","last_became_peered":"2024-02-18T00:43:07.795433+0000","last_unstale":"2024-02-18T00:45:58.864233+0000","last_undegraded":"2024-02-18T00:45:58.864233+0000","last_fullsized":"2024-02-18T00:45:58.864233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:43:51.042989+0000","last_deep_scrub":"457'32","last_deep_scrub_stamp":"2024-02-18T00:43:51.042989+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.042989+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":806,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863944+0000","last_change":"2024-02-18T00:44:43.113419+0000","last_active":"2024-02-18T00:45:58.863944+0000","last_peered":"2024-02-18T00:45:58.863944+0000","last_clean":"2024-02-18T00:45:58.863944+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:45:58.863944+0000","last_undegraded":"2024-02-18T00:45:58.863944+0000","last_fullsized":"2024-02-18T00:45:58.863944+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.113292+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":759,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.888402+0000","last_change":"2024-02-18T00:44:44.469481+0000","last_active":"2024-02-18T00:45:58.888402+0000","last_peered":"2024-02-18T00:45:58.888402+0000","last_clean":"2024-02-18T00:45:58.888402+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:45:58.888402+0000","last_undegraded":"2024-02-18T00:45:58.888402+0000","last_fullsized":"2024-02-18T00:45:58.888402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.469413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.469413+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":707,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868562+0000","last_change":"2024-02-18T00:43:09.318894+0000","last_active":"2024-02-18T00:45:58.868562+0000","last_peered":"2024-02-18T00:45:58.868562+0000","last_clean":"2024-02-18T00:45:58.868562+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:45:58.868562+0000","last_undegraded":"2024-02-18T00:45:58.868562+0000","last_fullsized":"2024-02-18T00:45:58.868562+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.318803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.318803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":765,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862157+0000","last_change":"2024-02-18T00:43:44.168202+0000","last_active":"2024-02-18T00:45:58.862157+0000","last_peered":"2024-02-18T00:45:58.862157+0000","last_clean":"2024-02-18T00:45:58.862157+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:45:58.862157+0000","last_undegraded":"2024-02-18T00:45:58.862157+0000","last_fullsized":"2024-02-18T00:45:58.862157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.168096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.168096+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":842,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.867992+0000","last_change":"2024-02-18T00:44:27.619378+0000","last_active":"2024-02-18T00:45:58.867992+0000","last_peered":"2024-02-18T00:45:58.867992+0000","last_clean":"2024-02-18T00:45:58.867992+0000","last_became_active":"2024-02-18T00:43:07.795322+0000","last_became_peered":"2024-02-18T00:43:07.795322+0000","last_unstale":"2024-02-18T00:45:58.867992+0000","last_undegraded":"2024-02-18T00:45:58.867992+0000","last_fullsized":"2024-02-18T00:45:58.867992+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:44:27.619298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:27.619298+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":820,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.869102+0000","last_change":"2024-02-18T00:44:47.741854+0000","last_active":"2024-02-18T00:45:58.869102+0000","last_peered":"2024-02-18T00:45:58.869102+0000","last_clean":"2024-02-18T00:45:58.869102+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:45:58.869102+0000","last_undegraded":"2024-02-18T00:45:58.869102+0000","last_fullsized":"2024-02-18T00:45:58.869102+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:44:47.741787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.741787+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":793,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884695+0000","last_change":"2024-02-18T00:45:44.370261+0000","last_active":"2024-02-18T00:45:58.884695+0000","last_peered":"2024-02-18T00:45:58.884695+0000","last_clean":"2024-02-18T00:45:58.884695+0000","last_became_active":"2024-02-18T00:41:42.776144+0000","last_became_peered":"2024-02-18T00:41:42.776144+0000","last_unstale":"2024-02-18T00:45:58.884695+0000","last_undegraded":"2024-02-18T00:45:58.884695+0000","last_fullsized":"2024-02-18T00:45:58.884695+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:01.098569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:00.804656+0000","last_clean_scrub_stamp":"2024-02-18T00:43:01.098569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":707,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868579+0000","last_change":"2024-02-18T00:43:50.444348+0000","last_active":"2024-02-18T00:45:58.868579+0000","last_peered":"2024-02-18T00:45:58.868579+0000","last_clean":"2024-02-18T00:45:58.868579+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:45:58.868579+0000","last_undegraded":"2024-02-18T00:45:58.868579+0000","last_fullsized":"2024-02-18T00:45:58.868579+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.444279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.444279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"774'33","reported_seq":875,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889681+0000","last_change":"2024-02-18T00:45:39.120191+0000","last_active":"2024-02-18T00:45:58.889681+0000","last_peered":"2024-02-18T00:45:58.889681+0000","last_clean":"2024-02-18T00:45:58.889681+0000","last_became_active":"2024-02-18T00:39:36.771394+0000","last_became_peered":"2024-02-18T00:39:36.771394+0000","last_unstale":"2024-02-18T00:45:58.889681+0000","last_undegraded":"2024-02-18T00:45:58.889681+0000","last_fullsized":"2024-02-18T00:45:58.889681+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"774'33","last_scrub_stamp":"2024-02-18T00:45:39.120115+0000","last_deep_scrub":"774'33","last_deep_scrub_stamp":"2024-02-18T00:45:39.120115+0000","last_clean_scrub_stamp":"2024-02-18T00:45:39.120115+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":769,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862118+0000","last_change":"2024-02-18T00:43:34.111205+0000","last_active":"2024-02-18T00:45:58.862118+0000","last_peered":"2024-02-18T00:45:58.862118+0000","last_clean":"2024-02-18T00:45:58.862118+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:45:58.862118+0000","last_undegraded":"2024-02-18T00:45:58.862118+0000","last_fullsized":"2024-02-18T00:45:58.862118+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:34.111128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.111128+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":774,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884355+0000","last_change":"2024-02-18T00:43:09.212023+0000","last_active":"2024-02-18T00:45:58.884355+0000","last_peered":"2024-02-18T00:45:58.884355+0000","last_clean":"2024-02-18T00:45:58.884355+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:45:58.884355+0000","last_undegraded":"2024-02-18T00:45:58.884355+0000","last_fullsized":"2024-02-18T00:45:58.884355+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:09.211956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:09.211956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":783,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864522+0000","last_change":"2024-02-18T00:44:42.127223+0000","last_active":"2024-02-18T00:45:58.864522+0000","last_peered":"2024-02-18T00:45:58.864522+0000","last_clean":"2024-02-18T00:45:58.864522+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:45:58.864522+0000","last_undegraded":"2024-02-18T00:45:58.864522+0000","last_fullsized":"2024-02-18T00:45:58.864522+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:42.127148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:46.308454+0000","last_clean_scrub_stamp":"2024-02-18T00:44:42.127148+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":706,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889129+0000","last_change":"2024-02-18T00:44:37.525779+0000","last_active":"2024-02-18T00:45:58.889129+0000","last_peered":"2024-02-18T00:45:58.889129+0000","last_clean":"2024-02-18T00:45:58.889129+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:45:58.889129+0000","last_undegraded":"2024-02-18T00:45:58.889129+0000","last_fullsized":"2024-02-18T00:45:58.889129+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:37.525709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:37.525709+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":856,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884266+0000","last_change":"2024-02-18T00:45:20.335925+0000","last_active":"2024-02-18T00:45:58.884266+0000","last_peered":"2024-02-18T00:45:58.884266+0000","last_clean":"2024-02-18T00:45:58.884266+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:45:58.884266+0000","last_undegraded":"2024-02-18T00:45:58.884266+0000","last_fullsized":"2024-02-18T00:45:58.884266+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'18","last_scrub_stamp":"2024-02-18T00:45:20.335853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:20.335853+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":32,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868086+0000","last_change":"2024-02-18T00:45:35.142446+0000","last_active":"2024-02-18T00:45:58.868086+0000","last_peered":"2024-02-18T00:45:58.868086+0000","last_clean":"2024-02-18T00:45:58.868086+0000","last_became_active":"2024-02-18T00:45:35.141951+0000","last_became_peered":"2024-02-18T00:45:35.141951+0000","last_unstale":"2024-02-18T00:45:58.868086+0000","last_undegraded":"2024-02-18T00:45:58.868086+0000","last_fullsized":"2024-02-18T00:45:58.868086+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:43.359510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:43.359510+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":762,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872127+0000","last_change":"2024-02-18T00:44:45.666045+0000","last_active":"2024-02-18T00:45:58.872127+0000","last_peered":"2024-02-18T00:45:58.872127+0000","last_clean":"2024-02-18T00:45:58.872127+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:45:58.872127+0000","last_undegraded":"2024-02-18T00:45:58.872127+0000","last_fullsized":"2024-02-18T00:45:58.872127+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:45.665981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:45.665981+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":344,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875193+0000","last_change":"2024-02-18T00:45:35.149800+0000","last_active":"2024-02-18T00:45:58.875193+0000","last_peered":"2024-02-18T00:45:58.875193+0000","last_clean":"2024-02-18T00:45:58.875193+0000","last_became_active":"2024-02-18T00:45:35.149331+0000","last_became_peered":"2024-02-18T00:45:35.149331+0000","last_unstale":"2024-02-18T00:45:58.875193+0000","last_undegraded":"2024-02-18T00:45:58.875193+0000","last_fullsized":"2024-02-18T00:45:58.875193+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:33.079915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:45:33.079915+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":718,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862792+0000","last_change":"2024-02-18T00:44:47.201367+0000","last_active":"2024-02-18T00:45:58.862792+0000","last_peered":"2024-02-18T00:45:58.862792+0000","last_clean":"2024-02-18T00:45:58.862792+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:45:58.862792+0000","last_undegraded":"2024-02-18T00:45:58.862792+0000","last_fullsized":"2024-02-18T00:45:58.862792+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:47.201293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.201293+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":868,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884952+0000","last_change":"2024-02-18T00:43:41.139904+0000","last_active":"2024-02-18T00:45:58.884952+0000","last_peered":"2024-02-18T00:45:58.884952+0000","last_clean":"2024-02-18T00:45:58.884952+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:45:58.884952+0000","last_undegraded":"2024-02-18T00:45:58.884952+0000","last_fullsized":"2024-02-18T00:45:58.884952+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:43:41.139829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:41.139829+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":793,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871630+0000","last_change":"2024-02-18T00:44:46.634819+0000","last_active":"2024-02-18T00:45:58.871630+0000","last_peered":"2024-02-18T00:45:58.871630+0000","last_clean":"2024-02-18T00:45:58.871630+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:45:58.871630+0000","last_undegraded":"2024-02-18T00:45:58.871630+0000","last_fullsized":"2024-02-18T00:45:58.871630+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:46.634734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.634734+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":972,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872272+0000","last_change":"2024-02-18T00:44:47.651515+0000","last_active":"2024-02-18T00:45:58.872272+0000","last_peered":"2024-02-18T00:45:58.872272+0000","last_clean":"2024-02-18T00:45:58.872272+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:45:58.872272+0000","last_undegraded":"2024-02-18T00:45:58.872272+0000","last_fullsized":"2024-02-18T00:45:58.872272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:44:47.651424+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:44:47.651424+0000","last_clean_scrub_stamp":"2024-02-18T00:44:47.651424+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":412,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874519+0000","last_change":"2024-02-18T00:45:35.147104+0000","last_active":"2024-02-18T00:45:58.874519+0000","last_peered":"2024-02-18T00:45:58.874519+0000","last_clean":"2024-02-18T00:45:58.874519+0000","last_became_active":"2024-02-18T00:45:35.146551+0000","last_became_peered":"2024-02-18T00:45:35.146551+0000","last_unstale":"2024-02-18T00:45:58.874519+0000","last_undegraded":"2024-02-18T00:45:58.874519+0000","last_fullsized":"2024-02-18T00:45:58.874519+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:42.310377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:42.310377+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"457'21","reported_seq":846,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872188+0000","last_change":"2024-02-18T00:45:26.683021+0000","last_active":"2024-02-18T00:45:58.872188+0000","last_peered":"2024-02-18T00:45:58.872188+0000","last_clean":"2024-02-18T00:45:58.872188+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:45:58.872188+0000","last_undegraded":"2024-02-18T00:45:58.872188+0000","last_fullsized":"2024-02-18T00:45:58.872188+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:45:26.682936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:26.682936+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":770,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.888719+0000","last_change":"2024-02-18T00:45:27.339567+0000","last_active":"2024-02-18T00:45:58.888719+0000","last_peered":"2024-02-18T00:45:58.888719+0000","last_clean":"2024-02-18T00:45:58.888719+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:45:58.888719+0000","last_undegraded":"2024-02-18T00:45:58.888719+0000","last_fullsized":"2024-02-18T00:45:58.888719+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:27.339506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:45:27.339506+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":992,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.885040+0000","last_change":"2024-02-18T00:43:00.081109+0000","last_active":"2024-02-18T00:45:58.885040+0000","last_peered":"2024-02-18T00:45:58.885040+0000","last_clean":"2024-02-18T00:45:58.885040+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:45:58.885040+0000","last_undegraded":"2024-02-18T00:45:58.885040+0000","last_fullsized":"2024-02-18T00:45:58.885040+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:43:00.080987+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:43:00.080987+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":715,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872539+0000","last_change":"2024-02-18T00:45:35.164444+0000","last_active":"2024-02-18T00:45:58.872539+0000","last_peered":"2024-02-18T00:45:58.872539+0000","last_clean":"2024-02-18T00:45:58.872539+0000","last_became_active":"2024-02-18T00:45:35.163794+0000","last_became_peered":"2024-02-18T00:45:35.163794+0000","last_unstale":"2024-02-18T00:45:58.872539+0000","last_undegraded":"2024-02-18T00:45:58.872539+0000","last_fullsized":"2024-02-18T00:45:58.872539+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:48.697958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:44:48.697958+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":818,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863421+0000","last_change":"2024-02-18T00:45:36.044332+0000","last_active":"2024-02-18T00:45:58.863421+0000","last_peered":"2024-02-18T00:45:58.863421+0000","last_clean":"2024-02-18T00:45:58.863421+0000","last_became_active":"2024-02-18T00:45:35.151145+0000","last_became_peered":"2024-02-18T00:45:35.151145+0000","last_unstale":"2024-02-18T00:45:58.863421+0000","last_undegraded":"2024-02-18T00:45:58.863421+0000","last_fullsized":"2024-02-18T00:45:58.863421+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:45:30.879861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:30.879861+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":31,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884812+0000","last_change":"2024-02-18T00:45:35.165549+0000","last_active":"2024-02-18T00:45:58.884812+0000","last_peered":"2024-02-18T00:45:58.884812+0000","last_clean":"2024-02-18T00:45:58.884812+0000","last_became_active":"2024-02-18T00:45:35.164995+0000","last_became_peered":"2024-02-18T00:45:35.164995+0000","last_unstale":"2024-02-18T00:45:58.884812+0000","last_undegraded":"2024-02-18T00:45:58.884812+0000","last_fullsized":"2024-02-18T00:45:58.884812+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:36.372315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:36.372315+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":761,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884635+0000","last_change":"2024-02-18T00:45:37.274407+0000","last_active":"2024-02-18T00:45:58.884635+0000","last_peered":"2024-02-18T00:45:58.884635+0000","last_clean":"2024-02-18T00:45:58.884635+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:45:58.884635+0000","last_undegraded":"2024-02-18T00:45:58.884635+0000","last_fullsized":"2024-02-18T00:45:58.884635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:37.274299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:45:37.274299+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":764,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.888399+0000","last_change":"2024-02-18T00:44:38.484355+0000","last_active":"2024-02-18T00:45:58.888399+0000","last_peered":"2024-02-18T00:45:58.888399+0000","last_clean":"2024-02-18T00:45:58.888399+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:45:58.888399+0000","last_undegraded":"2024-02-18T00:45:58.888399+0000","last_fullsized":"2024-02-18T00:45:58.888399+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.484287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.484287+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":708,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889545+0000","last_change":"2024-02-18T00:44:49.583138+0000","last_active":"2024-02-18T00:45:58.889545+0000","last_peered":"2024-02-18T00:45:58.889545+0000","last_clean":"2024-02-18T00:45:58.889545+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:45:58.889545+0000","last_undegraded":"2024-02-18T00:45:58.889545+0000","last_fullsized":"2024-02-18T00:45:58.889545+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:49.583035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:07.241102+0000","last_clean_scrub_stamp":"2024-02-18T00:44:49.583035+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":822,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868945+0000","last_change":"2024-02-18T00:45:35.776786+0000","last_active":"2024-02-18T00:45:58.868945+0000","last_peered":"2024-02-18T00:45:58.868945+0000","last_clean":"2024-02-18T00:45:58.868945+0000","last_became_active":"2024-02-18T00:45:35.147993+0000","last_became_peered":"2024-02-18T00:45:35.147993+0000","last_unstale":"2024-02-18T00:45:58.868945+0000","last_undegraded":"2024-02-18T00:45:58.868945+0000","last_fullsized":"2024-02-18T00:45:58.868945+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:44:41.703456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:41.703456+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5],"acting":[4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":774,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865379+0000","last_change":"2024-02-18T00:45:26.810144+0000","last_active":"2024-02-18T00:45:58.865379+0000","last_peered":"2024-02-18T00:45:58.865379+0000","last_clean":"2024-02-18T00:45:58.865379+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:45:58.865379+0000","last_undegraded":"2024-02-18T00:45:58.865379+0000","last_fullsized":"2024-02-18T00:45:58.865379+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:26.810078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:26.810078+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":770,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.863502+0000","last_change":"2024-02-18T00:43:39.958283+0000","last_active":"2024-02-18T00:45:58.863502+0000","last_peered":"2024-02-18T00:45:58.863502+0000","last_clean":"2024-02-18T00:45:58.863502+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:45:58.863502+0000","last_undegraded":"2024-02-18T00:45:58.863502+0000","last_fullsized":"2024-02-18T00:45:58.863502+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.958201+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":754,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883620+0000","last_change":"2024-02-18T00:43:40.107832+0000","last_active":"2024-02-18T00:45:58.883620+0000","last_peered":"2024-02-18T00:45:58.883620+0000","last_clean":"2024-02-18T00:45:58.883620+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:45:58.883620+0000","last_undegraded":"2024-02-18T00:45:58.883620+0000","last_fullsized":"2024-02-18T00:45:58.883620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:40.107757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:40.107757+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":31,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884156+0000","last_change":"2024-02-18T00:45:35.145460+0000","last_active":"2024-02-18T00:45:58.884156+0000","last_peered":"2024-02-18T00:45:58.884156+0000","last_clean":"2024-02-18T00:45:58.884156+0000","last_became_active":"2024-02-18T00:45:35.144593+0000","last_became_peered":"2024-02-18T00:45:35.144593+0000","last_unstale":"2024-02-18T00:45:58.884156+0000","last_undegraded":"2024-02-18T00:45:58.884156+0000","last_fullsized":"2024-02-18T00:45:58.884156+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:22.809509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:45:22.809509+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"457'8","reported_seq":790,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883682+0000","last_change":"2024-02-18T00:45:39.337419+0000","last_active":"2024-02-18T00:45:58.883682+0000","last_peered":"2024-02-18T00:45:58.883682+0000","last_clean":"2024-02-18T00:45:58.883682+0000","last_became_active":"2024-02-18T00:45:35.158536+0000","last_became_peered":"2024-02-18T00:45:35.158536+0000","last_unstale":"2024-02-18T00:45:58.883682+0000","last_undegraded":"2024-02-18T00:45:58.883682+0000","last_fullsized":"2024-02-18T00:45:58.883682+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'8","last_scrub_stamp":"2024-02-18T00:45:39.337347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:39.337347+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":780,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872145+0000","last_change":"2024-02-18T00:45:35.144661+0000","last_active":"2024-02-18T00:45:58.872145+0000","last_peered":"2024-02-18T00:45:58.872145+0000","last_clean":"2024-02-18T00:45:58.872145+0000","last_became_active":"2024-02-18T00:45:35.143838+0000","last_became_peered":"2024-02-18T00:45:35.143838+0000","last_unstale":"2024-02-18T00:45:58.872145+0000","last_undegraded":"2024-02-18T00:45:58.872145+0000","last_fullsized":"2024-02-18T00:45:58.872145+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:29.643976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_clean_scrub_stamp":"2024-02-18T00:45:29.643976+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":475,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.867829+0000","last_change":"2024-02-18T00:45:35.138713+0000","last_active":"2024-02-18T00:45:58.867829+0000","last_peered":"2024-02-18T00:45:58.867829+0000","last_clean":"2024-02-18T00:45:58.867829+0000","last_became_active":"2024-02-18T00:45:35.138378+0000","last_became_peered":"2024-02-18T00:45:35.138378+0000","last_unstale":"2024-02-18T00:45:58.867829+0000","last_undegraded":"2024-02-18T00:45:58.867829+0000","last_fullsized":"2024-02-18T00:45:58.867829+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.575417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.575417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":335,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874747+0000","last_change":"2024-02-18T00:44:28.186498+0000","last_active":"2024-02-18T00:45:58.874747+0000","last_peered":"2024-02-18T00:45:58.874747+0000","last_clean":"2024-02-18T00:45:58.874747+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:45:58.874747+0000","last_undegraded":"2024-02-18T00:45:58.874747+0000","last_fullsized":"2024-02-18T00:45:58.874747+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:28.186417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.186417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":724,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872234+0000","last_change":"2024-02-18T00:45:32.631480+0000","last_active":"2024-02-18T00:45:58.872234+0000","last_peered":"2024-02-18T00:45:58.872234+0000","last_clean":"2024-02-18T00:45:58.872234+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:45:58.872234+0000","last_undegraded":"2024-02-18T00:45:58.872234+0000","last_fullsized":"2024-02-18T00:45:58.872234+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:32.631399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:45:32.631399+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":873,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884113+0000","last_change":"2024-02-18T00:45:35.185293+0000","last_active":"2024-02-18T00:45:58.884113+0000","last_peered":"2024-02-18T00:45:58.884113+0000","last_clean":"2024-02-18T00:45:58.884113+0000","last_became_active":"2024-02-18T00:45:35.165378+0000","last_became_peered":"2024-02-18T00:45:35.165378+0000","last_unstale":"2024-02-18T00:45:58.884113+0000","last_undegraded":"2024-02-18T00:45:58.884113+0000","last_fullsized":"2024-02-18T00:45:58.884113+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:44:28.251169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:28.251169+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":31,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872226+0000","last_change":"2024-02-18T00:45:35.143293+0000","last_active":"2024-02-18T00:45:58.872226+0000","last_peered":"2024-02-18T00:45:58.872226+0000","last_clean":"2024-02-18T00:45:58.872226+0000","last_became_active":"2024-02-18T00:45:35.142978+0000","last_became_peered":"2024-02-18T00:45:35.142978+0000","last_unstale":"2024-02-18T00:45:58.872226+0000","last_undegraded":"2024-02-18T00:45:58.872226+0000","last_fullsized":"2024-02-18T00:45:58.872226+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:31.818950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_clean_scrub_stamp":"2024-02-18T00:45:31.818950+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":339,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874610+0000","last_change":"2024-02-18T00:45:35.146889+0000","last_active":"2024-02-18T00:45:58.874610+0000","last_peered":"2024-02-18T00:45:58.874610+0000","last_clean":"2024-02-18T00:45:58.874610+0000","last_became_active":"2024-02-18T00:45:35.146045+0000","last_became_peered":"2024-02-18T00:45:35.146045+0000","last_unstale":"2024-02-18T00:45:58.874610+0000","last_undegraded":"2024-02-18T00:45:58.874610+0000","last_fullsized":"2024-02-18T00:45:58.874610+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:23.214010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:23.214010+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":763,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884744+0000","last_change":"2024-02-18T00:45:35.138674+0000","last_active":"2024-02-18T00:45:58.884744+0000","last_peered":"2024-02-18T00:45:58.884744+0000","last_clean":"2024-02-18T00:45:58.884744+0000","last_became_active":"2024-02-18T00:45:35.138308+0000","last_became_peered":"2024-02-18T00:45:35.138308+0000","last_unstale":"2024-02-18T00:45:58.884744+0000","last_undegraded":"2024-02-18T00:45:58.884744+0000","last_fullsized":"2024-02-18T00:45:58.884744+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.135214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.135214+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":335,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875153+0000","last_change":"2024-02-18T00:43:44.122130+0000","last_active":"2024-02-18T00:45:58.875153+0000","last_peered":"2024-02-18T00:45:58.875153+0000","last_clean":"2024-02-18T00:45:58.875153+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:45:58.875153+0000","last_undegraded":"2024-02-18T00:45:58.875153+0000","last_fullsized":"2024-02-18T00:45:58.875153+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.122056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:44.122056+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.122056+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":865,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889779+0000","last_change":"2024-02-18T00:44:34.576745+0000","last_active":"2024-02-18T00:45:58.889779+0000","last_peered":"2024-02-18T00:45:58.889779+0000","last_clean":"2024-02-18T00:45:58.889779+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:45:58.889779+0000","last_undegraded":"2024-02-18T00:45:58.889779+0000","last_fullsized":"2024-02-18T00:45:58.889779+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:44:34.576675+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:44:34.576675+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":793,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871395+0000","last_change":"2024-02-18T00:45:36.546508+0000","last_active":"2024-02-18T00:45:58.871395+0000","last_peered":"2024-02-18T00:45:58.871395+0000","last_clean":"2024-02-18T00:45:58.871395+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:45:58.871395+0000","last_undegraded":"2024-02-18T00:45:58.871395+0000","last_fullsized":"2024-02-18T00:45:58.871395+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:36.546441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:36.546441+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"774'11","reported_seq":63,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.869231+0000","last_change":"2024-02-18T00:45:35.785281+0000","last_active":"2024-02-18T00:45:58.869231+0000","last_peered":"2024-02-18T00:45:58.869231+0000","last_clean":"2024-02-18T00:45:58.869231+0000","last_became_active":"2024-02-18T00:45:35.160045+0000","last_became_peered":"2024-02-18T00:45:35.160045+0000","last_unstale":"2024-02-18T00:45:58.869231+0000","last_undegraded":"2024-02-18T00:45:58.869231+0000","last_fullsized":"2024-02-18T00:45:58.869231+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:43:37.545939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.545939+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":335,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874645+0000","last_change":"2024-02-18T00:43:51.101862+0000","last_active":"2024-02-18T00:45:58.874645+0000","last_peered":"2024-02-18T00:45:58.874645+0000","last_clean":"2024-02-18T00:45:58.874645+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:45:58.874645+0000","last_undegraded":"2024-02-18T00:45:58.874645+0000","last_fullsized":"2024-02-18T00:45:58.874645+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:51.101796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:51.101796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":517,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.872374+0000","last_change":"2024-02-18T00:45:35.742949+0000","last_active":"2024-02-18T00:45:58.872374+0000","last_peered":"2024-02-18T00:45:58.872374+0000","last_clean":"2024-02-18T00:45:58.872374+0000","last_became_active":"2024-02-18T00:45:35.155641+0000","last_became_peered":"2024-02-18T00:45:35.155641+0000","last_unstale":"2024-02-18T00:45:58.872374+0000","last_undegraded":"2024-02-18T00:45:58.872374+0000","last_fullsized":"2024-02-18T00:45:58.872374+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:45:30.839684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:45:30.839684+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":35,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":774,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883786+0000","last_change":"2024-02-18T00:45:38.322896+0000","last_active":"2024-02-18T00:45:58.883786+0000","last_peered":"2024-02-18T00:45:58.883786+0000","last_clean":"2024-02-18T00:45:58.883786+0000","last_became_active":"2024-02-18T00:45:35.148886+0000","last_became_peered":"2024-02-18T00:45:35.148886+0000","last_unstale":"2024-02-18T00:45:58.883786+0000","last_undegraded":"2024-02-18T00:45:58.883786+0000","last_fullsized":"2024-02-18T00:45:58.883786+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:38.322790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:38.322790+0000","last_clean_scrub_stamp":"2024-02-18T00:45:38.322790+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":779,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889822+0000","last_change":"2024-02-18T00:44:46.488019+0000","last_active":"2024-02-18T00:45:58.889822+0000","last_peered":"2024-02-18T00:45:58.889822+0000","last_clean":"2024-02-18T00:45:58.889822+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:45:58.889822+0000","last_undegraded":"2024-02-18T00:45:58.889822+0000","last_fullsized":"2024-02-18T00:45:58.889822+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.487955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":735,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862713+0000","last_change":"2024-02-18T00:45:35.169902+0000","last_active":"2024-02-18T00:45:58.862713+0000","last_peered":"2024-02-18T00:45:58.862713+0000","last_clean":"2024-02-18T00:45:58.862713+0000","last_became_active":"2024-02-18T00:45:35.143537+0000","last_became_peered":"2024-02-18T00:45:35.143537+0000","last_unstale":"2024-02-18T00:45:58.862713+0000","last_undegraded":"2024-02-18T00:45:58.862713+0000","last_fullsized":"2024-02-18T00:45:58.862713+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:43:37.193693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:37.193693+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,7],"acting":[2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":815,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.864776+0000","last_change":"2024-02-18T00:43:39.154626+0000","last_active":"2024-02-18T00:45:58.864776+0000","last_peered":"2024-02-18T00:45:58.864776+0000","last_clean":"2024-02-18T00:45:58.864776+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:45:58.864776+0000","last_undegraded":"2024-02-18T00:45:58.864776+0000","last_fullsized":"2024-02-18T00:45:58.864776+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:43:39.154561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.154561+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":481,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868491+0000","last_change":"2024-02-18T00:45:35.137790+0000","last_active":"2024-02-18T00:45:58.868491+0000","last_peered":"2024-02-18T00:45:58.868491+0000","last_clean":"2024-02-18T00:45:58.868491+0000","last_became_active":"2024-02-18T00:45:35.137365+0000","last_became_peered":"2024-02-18T00:45:35.137365+0000","last_unstale":"2024-02-18T00:45:58.868491+0000","last_undegraded":"2024-02-18T00:45:58.868491+0000","last_fullsized":"2024-02-18T00:45:58.868491+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:25.509523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:44:25.509523+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2],"acting":[4,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":770,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865539+0000","last_change":"2024-02-18T00:44:35.012928+0000","last_active":"2024-02-18T00:45:58.865539+0000","last_peered":"2024-02-18T00:45:58.865539+0000","last_clean":"2024-02-18T00:45:58.865539+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:45:58.865539+0000","last_undegraded":"2024-02-18T00:45:58.865539+0000","last_fullsized":"2024-02-18T00:45:58.865539+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:35.012802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:44:35.012802+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":30,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865688+0000","last_change":"2024-02-18T00:45:35.151939+0000","last_active":"2024-02-18T00:45:58.865688+0000","last_peered":"2024-02-18T00:45:58.865688+0000","last_clean":"2024-02-18T00:45:58.865688+0000","last_became_active":"2024-02-18T00:45:35.151608+0000","last_became_peered":"2024-02-18T00:45:35.151608+0000","last_unstale":"2024-02-18T00:45:58.865688+0000","last_undegraded":"2024-02-18T00:45:58.865688+0000","last_fullsized":"2024-02-18T00:45:58.865688+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:38.348263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_clean_scrub_stamp":"2024-02-18T00:44:38.348263+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"457'24","reported_seq":805,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883986+0000","last_change":"2024-02-18T00:43:43.140277+0000","last_active":"2024-02-18T00:45:58.883986+0000","last_peered":"2024-02-18T00:45:58.883986+0000","last_clean":"2024-02-18T00:45:58.883986+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:45:58.883986+0000","last_undegraded":"2024-02-18T00:45:58.883986+0000","last_fullsized":"2024-02-18T00:45:58.883986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'24","last_scrub_stamp":"2024-02-18T00:43:43.140135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:43:43.140135+0000","log_size":24,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":811,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884591+0000","last_change":"2024-02-18T00:45:35.167664+0000","last_active":"2024-02-18T00:45:58.884591+0000","last_peered":"2024-02-18T00:45:58.884591+0000","last_clean":"2024-02-18T00:45:58.884591+0000","last_became_active":"2024-02-18T00:45:35.150738+0000","last_became_peered":"2024-02-18T00:45:35.150738+0000","last_unstale":"2024-02-18T00:45:58.884591+0000","last_undegraded":"2024-02-18T00:45:58.884591+0000","last_fullsized":"2024-02-18T00:45:58.884591+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:45:26.282475+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:45:26.282475+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":768,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868390+0000","last_change":"2024-02-18T00:44:44.755473+0000","last_active":"2024-02-18T00:45:58.868390+0000","last_peered":"2024-02-18T00:45:58.868390+0000","last_clean":"2024-02-18T00:45:58.868390+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:45:58.868390+0000","last_undegraded":"2024-02-18T00:45:58.868390+0000","last_fullsized":"2024-02-18T00:45:58.868390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:44.755386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:44.755386+0000","last_clean_scrub_stamp":"2024-02-18T00:44:44.755386+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":733,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889388+0000","last_change":"2024-02-18T00:45:35.161848+0000","last_active":"2024-02-18T00:45:58.889388+0000","last_peered":"2024-02-18T00:45:58.889388+0000","last_clean":"2024-02-18T00:45:58.889388+0000","last_became_active":"2024-02-18T00:45:35.161508+0000","last_became_peered":"2024-02-18T00:45:35.161508+0000","last_unstale":"2024-02-18T00:45:58.889388+0000","last_undegraded":"2024-02-18T00:45:58.889388+0000","last_fullsized":"2024-02-18T00:45:58.889388+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:48.486037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:48.486037+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":839,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868479+0000","last_change":"2024-02-18T00:45:24.894618+0000","last_active":"2024-02-18T00:45:58.868479+0000","last_peered":"2024-02-18T00:45:58.868479+0000","last_clean":"2024-02-18T00:45:58.868479+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:45:58.868479+0000","last_undegraded":"2024-02-18T00:45:58.868479+0000","last_fullsized":"2024-02-18T00:45:58.868479+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:45:24.894520+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:45:24.894520+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":335,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875104+0000","last_change":"2024-02-18T00:45:40.007668+0000","last_active":"2024-02-18T00:45:58.875104+0000","last_peered":"2024-02-18T00:45:58.875104+0000","last_clean":"2024-02-18T00:45:58.875104+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:45:58.875104+0000","last_undegraded":"2024-02-18T00:45:58.875104+0000","last_fullsized":"2024-02-18T00:45:58.875104+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:40.007604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:45:40.007604+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":803,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.871619+0000","last_change":"2024-02-18T00:45:38.502458+0000","last_active":"2024-02-18T00:45:58.871619+0000","last_peered":"2024-02-18T00:45:58.871619+0000","last_clean":"2024-02-18T00:45:58.871619+0000","last_became_active":"2024-02-18T00:45:35.150420+0000","last_became_peered":"2024-02-18T00:45:35.150420+0000","last_unstale":"2024-02-18T00:45:58.871619+0000","last_undegraded":"2024-02-18T00:45:58.871619+0000","last_fullsized":"2024-02-18T00:45:58.871619+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:38.502378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:37.673673+0000","last_clean_scrub_stamp":"2024-02-18T00:45:38.502378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":709,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.862543+0000","last_change":"2024-02-18T00:45:23.658121+0000","last_active":"2024-02-18T00:45:58.862543+0000","last_peered":"2024-02-18T00:45:58.862543+0000","last_clean":"2024-02-18T00:45:58.862543+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:45:58.862543+0000","last_undegraded":"2024-02-18T00:45:58.862543+0000","last_fullsized":"2024-02-18T00:45:58.862543+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:23.658053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:45:23.658053+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":801,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.868727+0000","last_change":"2024-02-18T00:44:34.749044+0000","last_active":"2024-02-18T00:45:58.868727+0000","last_peered":"2024-02-18T00:45:58.868727+0000","last_clean":"2024-02-18T00:45:58.868727+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:45:58.868727+0000","last_undegraded":"2024-02-18T00:45:58.868727+0000","last_fullsized":"2024-02-18T00:45:58.868727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:44:34.748966+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:44:34.748966+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":344,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.875015+0000","last_change":"2024-02-18T00:45:35.157988+0000","last_active":"2024-02-18T00:45:58.875015+0000","last_peered":"2024-02-18T00:45:58.875015+0000","last_clean":"2024-02-18T00:45:58.875015+0000","last_became_active":"2024-02-18T00:45:35.157638+0000","last_became_peered":"2024-02-18T00:45:35.157638+0000","last_unstale":"2024-02-18T00:45:58.875015+0000","last_undegraded":"2024-02-18T00:45:58.875015+0000","last_fullsized":"2024-02-18T00:45:58.875015+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:31.152515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:44:31.152515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":330,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.874943+0000","last_change":"2024-02-18T00:43:39.066050+0000","last_active":"2024-02-18T00:45:58.874943+0000","last_peered":"2024-02-18T00:45:58.874943+0000","last_clean":"2024-02-18T00:45:58.874943+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:45:58.874943+0000","last_undegraded":"2024-02-18T00:45:58.874943+0000","last_fullsized":"2024-02-18T00:45:58.874943+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:39.065952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:41.607567+0000","last_clean_scrub_stamp":"2024-02-18T00:43:39.065952+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":718,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884246+0000","last_change":"2024-02-18T00:44:30.255270+0000","last_active":"2024-02-18T00:45:58.884246+0000","last_peered":"2024-02-18T00:45:58.884246+0000","last_clean":"2024-02-18T00:45:58.884246+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:45:58.884246+0000","last_undegraded":"2024-02-18T00:45:58.884246+0000","last_fullsized":"2024-02-18T00:45:58.884246+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:30.255199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:44:30.255199+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":831,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.867756+0000","last_change":"2024-02-18T00:45:35.338045+0000","last_active":"2024-02-18T00:45:58.867756+0000","last_peered":"2024-02-18T00:45:58.867756+0000","last_clean":"2024-02-18T00:45:58.867756+0000","last_became_active":"2024-02-18T00:45:35.145933+0000","last_became_peered":"2024-02-18T00:45:35.145933+0000","last_unstale":"2024-02-18T00:45:58.867756+0000","last_undegraded":"2024-02-18T00:45:58.867756+0000","last_fullsized":"2024-02-18T00:45:58.867756+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:43:34.567365+0000","last_deep_scrub":"251'20","last_deep_scrub_stamp":"2024-02-18T00:35:30.243229+0000","last_clean_scrub_stamp":"2024-02-18T00:43:34.567365+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":760,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.884670+0000","last_change":"2024-02-18T00:43:44.142781+0000","last_active":"2024-02-18T00:45:58.884670+0000","last_peered":"2024-02-18T00:45:58.884670+0000","last_clean":"2024-02-18T00:45:58.884670+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:45:58.884670+0000","last_undegraded":"2024-02-18T00:45:58.884670+0000","last_fullsized":"2024-02-18T00:45:58.884670+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:44.142683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:43:44.142683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":788,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.865504+0000","last_change":"2024-02-18T00:45:27.836792+0000","last_active":"2024-02-18T00:45:58.865504+0000","last_peered":"2024-02-18T00:45:58.865504+0000","last_clean":"2024-02-18T00:45:58.865504+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:45:58.865504+0000","last_undegraded":"2024-02-18T00:45:58.865504+0000","last_fullsized":"2024-02-18T00:45:58.865504+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:45:27.836708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:45:27.836708+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":711,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.885136+0000","last_change":"2024-02-18T00:45:35.159517+0000","last_active":"2024-02-18T00:45:58.885136+0000","last_peered":"2024-02-18T00:45:58.885136+0000","last_clean":"2024-02-18T00:45:58.885136+0000","last_became_active":"2024-02-18T00:45:35.159194+0000","last_became_peered":"2024-02-18T00:45:35.159194+0000","last_unstale":"2024-02-18T00:45:58.885136+0000","last_undegraded":"2024-02-18T00:45:58.885136+0000","last_fullsized":"2024-02-18T00:45:58.885136+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:50.155205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:43:50.155205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":863,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.883751+0000","last_change":"2024-02-18T00:44:46.283742+0000","last_active":"2024-02-18T00:45:58.883751+0000","last_peered":"2024-02-18T00:45:58.883751+0000","last_clean":"2024-02-18T00:45:58.883751+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:45:58.883751+0000","last_undegraded":"2024-02-18T00:45:58.883751+0000","last_fullsized":"2024-02-18T00:45:58.883751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-18T00:44:46.283668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:44:46.283668+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":785,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.889137+0000","last_change":"2024-02-18T00:43:32.499214+0000","last_active":"2024-02-18T00:45:58.889137+0000","last_peered":"2024-02-18T00:45:58.889137+0000","last_clean":"2024-02-18T00:45:58.889137+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:45:58.889137+0000","last_undegraded":"2024-02-18T00:45:58.889137+0000","last_fullsized":"2024-02-18T00:45:58.889137+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:43:32.499132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:45.454069+0000","last_clean_scrub_stamp":"2024-02-18T00:43:32.499132+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":777,"reported_epoch":790,"state":"active+clean","last_fresh":"2024-02-18T00:45:58.867948+0000","last_change":"2024-02-18T00:45:35.147574+0000","last_active":"2024-02-18T00:45:58.867948+0000","last_peered":"2024-02-18T00:45:58.867948+0000","last_clean":"2024-02-18T00:45:58.867948+0000","last_became_active":"2024-02-18T00:45:35.146427+0000","last_became_peered":"2024-02-18T00:45:35.146427+0000","last_unstale":"2024-02-18T00:45:58.867948+0000","last_undegraded":"2024-02-18T00:45:58.867948+0000","last_fullsized":"2024-02-18T00:45:58.867948+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:44:39.696257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:44:39.696257+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":255,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":234,"num_pg":8,"stat_sum":{"num_bytes":4000289,"num_objects":43,"num_object_clones":1,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":24,"num_read_kb":7,"num_write":64,"num_write_kb":3966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":4000126,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6500352,"data_stored":6500352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":24,"acting":24,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":42,"stat_sum":{"num_bytes":3625,"num_objects":210,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":0,"num_read":999,"num_read_kb":821,"num_write":596,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":2541,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":11480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1284,"ondisk_log_size":1284,"up":126,"acting":126,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":995,"num_read_kb":990,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266893,"num_pgs":92,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":320048,"kb_used_data":6440,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93425104,"statfs":{"total":95995035648,"available":95667306496,"internally_reserved":0,"allocated":6594560,"data_stored":3538521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.26100000000000001,"15min":0.16900000000000001},"max":{"1min":0.96399999999999997,"5min":4.7389999999999999,"15min":6.8700000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.51800000000000002,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":1.24,"5min":1.24,"15min":11.288},"last":0.36899999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.63600000000000001},"min":{"1min":0.34999999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.1220000000000001,"5min":2.968,"15min":7.4729999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58999999999999997,"15min":0.65300000000000002},"min":{"1min":0.42599999999999999,"5min":0.311,"15min":0.29899999999999999},"max":{"1min":1.145,"5min":3.0019999999999998,"15min":7.7869999999999999},"last":0.39500000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56999999999999995,"15min":0.59599999999999997},"min":{"1min":0.33900000000000002,"5min":0.26600000000000001,"15min":0.23300000000000001},"max":{"1min":0.66000000000000003,"5min":4.8220000000000001,"15min":6.7380000000000004},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.53900000000000003,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.247},"max":{"1min":1.3029999999999999,"5min":1.913,"15min":12.861000000000001},"last":0.38500000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.26300000000000001},"max":{"1min":0.69199999999999995,"5min":4.8520000000000003,"15min":7.1360000000000001},"last":0.375},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54700000000000004,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.2549999999999999,"5min":1.948,"15min":3.7989999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.43099999999999999,"15min":0.435},"min":{"1min":0.30399999999999999,"5min":0.22700000000000001,"15min":0.215},"max":{"1min":0.66300000000000003,"5min":1.115,"15min":1.1779999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52500000000000002,"15min":0.48699999999999999},"min":{"1min":0.307,"5min":0.26300000000000001,"15min":0.246},"max":{"1min":0.61499999999999999,"5min":4.7859999999999996,"15min":4.7859999999999996},"last":0.29099999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45500000000000002,"15min":0.45900000000000002},"min":{"1min":0.33500000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.58999999999999997,"5min":1.877,"15min":1.877},"last":0.31},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48499999999999999,"15min":0.50900000000000001},"min":{"1min":0.32000000000000001,"5min":0.27500000000000002,"15min":0.25600000000000001},"max":{"1min":0.58499999999999996,"5min":1.853,"15min":4.9130000000000003},"last":0.41399999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.39400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.75600000000000001,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.38,"5min":0.309,"15min":0.28100000000000003},"max":{"1min":0.67400000000000004,"5min":5.3860000000000001,"15min":5.3860000000000001},"last":0.436}]}]},{"osd":6,"up_from":0,"seq":201863463123,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":13,"seq":55834575082,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":309768,"kb_used_data":8448,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93435384,"statfs":{"total":95995035648,"available":95677833216,"internally_reserved":0,"allocated":8650752,"data_stored":5584206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33800000000000002,"15min":0.64300000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":0.51200000000000001,"5min":0.77400000000000002,"15min":32.106999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.436,"15min":0.71899999999999997},"min":{"1min":0.247,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.83499999999999996,"5min":6.625,"15min":43.137},"last":0.30099999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:45:43 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.36199999999999999,"15min":0.755},"min":{"1min":0.23499999999999999,"5min":0.21099999999999999,"15min":0.19900000000000001},"max":{"1min":1.139,"5min":1.139,"15min":49.201000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.38900000000000001,"15min":0.58899999999999997},"min":{"1min":0.26800000000000002,"5min":0.23899999999999999,"15min":0.217},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":17.596},"last":0.28899999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:45:03 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.71699999999999997},"min":{"1min":0.28199999999999997,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":37.524000000000001},"last":0.31},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42999999999999999,"15min":0.73899999999999999},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.77000000000000002,"5min":0.89200000000000002,"15min":32.036000000000001},"last":0.372}]},{"osd":4,"last update":"Sun Feb 18 00:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.752},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.29499999999999998},"max":{"1min":0.84999999999999998,"5min":0.99299999999999999,"15min":17.486999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57299999999999995,"15min":0.77900000000000003},"min":{"1min":0.377,"5min":0.33500000000000002,"15min":0.32500000000000001},"max":{"1min":1.222,"5min":1.222,"15min":12.75},"last":0.48699999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.75900000000000001},"min":{"1min":0.39900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":12.611000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.77800000000000002},"min":{"1min":0.49299999999999999,"5min":0.35599999999999998,"15min":0.33100000000000002},"max":{"1min":0.89100000000000001,"5min":1.141,"15min":17.417000000000002},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.79200000000000004},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":12.834},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58999999999999997,"15min":0.77700000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.35499999999999998},"max":{"1min":1.046,"5min":1.046,"15min":12.651999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Sun Feb 18 00:45:43 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.79700000000000004},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.38700000000000001},"max":{"1min":0.94699999999999995,"5min":1.175,"15min":17.452000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.61799999999999999,"15min":0.80000000000000004},"min":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.38700000000000001},"max":{"1min":1.149,"5min":1.149,"15min":12.805},"last":0.51200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738608,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305712,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93439440,"statfs":{"total":95995035648,"available":95681986560,"internally_reserved":0,"allocated":8691712,"data_stored":5608398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.35399999999999998,"15min":0.99299999999999999},"min":{"1min":0.23300000000000001,"5min":0.217,"15min":0.18099999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":133.38999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.40699999999999997,"15min":0.79300000000000004},"min":{"1min":0.26700000000000002,"5min":0.23499999999999999,"15min":0.20999999999999999},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":90.623000000000005},"last":0.20599999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.44600000000000001,"15min":0.91000000000000003},"min":{"1min":0.26200000000000001,"5min":0.248,"15min":0.20200000000000001},"max":{"1min":2.915,"5min":2.915,"15min":90.850999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.40000000000000002,"15min":0.873},"min":{"1min":0.248,"5min":0.20599999999999999,"15min":0.20000000000000001},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":80.537000000000006},"last":0.253}]},{"osd":3,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.39600000000000002,"15min":0.64900000000000002},"min":{"1min":0.29899999999999999,"5min":0.21299999999999999,"15min":0.20899999999999999},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":19.006},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.46500000000000002,"15min":1.054},"min":{"1min":0.33900000000000002,"5min":0.29199999999999998,"15min":0.23999999999999999},"max":{"1min":4.2030000000000003,"5min":4.2030000000000003,"15min":146.71899999999999},"last":0.32300000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59399999999999997,"15min":0.93200000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26700000000000002},"max":{"1min":1.885,"5min":1.885,"15min":57.914000000000001},"last":0.443},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.627,"15min":0.98599999999999999},"min":{"1min":0.45400000000000001,"5min":0.33400000000000002,"15min":0.30399999999999999},"max":{"1min":2.8690000000000002,"5min":2.8690000000000002,"15min":90.070999999999998},"last":0.433}]},{"osd":5,"last update":"Sun Feb 18 00:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63900000000000001,"15min":0.97099999999999997},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.32000000000000001},"max":{"1min":1.5600000000000001,"5min":4.5499999999999998,"15min":90.049000000000007},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.99299999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.84599999999999997,"5min":1.853,"15min":57.991},"last":0.46500000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60899999999999999,"15min":0.80800000000000005},"min":{"1min":0.45600000000000002,"5min":0.377,"15min":0.35999999999999999},"max":{"1min":1.194,"5min":1.194,"15min":19.039000000000001},"last":0.499},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.63300000000000001,"15min":0.996},"min":{"1min":0.53000000000000003,"5min":0.39700000000000002,"15min":0.33500000000000002},"max":{"1min":3.3540000000000001,"5min":3.3540000000000001,"15min":89.989999999999995},"last":0.47399999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.67700000000000005,"15min":1.071},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.36499999999999999},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":90.025000000000006},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":1.006},"min":{"1min":0.40699999999999997,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":1.911,"5min":1.911,"15min":57.962000000000003},"last":0.40999999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411559,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305660,"kb_used_data":8436,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93439492,"statfs":{"total":95995035648,"available":95682039808,"internally_reserved":0,"allocated":8638464,"data_stored":5582934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.45700000000000002,"15min":0.50700000000000001},"min":{"1min":0.255,"5min":0.222,"15min":0.192},"max":{"1min":0.55200000000000005,"5min":8.6489999999999991,"15min":12.144},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36699999999999999,"15min":0.59199999999999997},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.19800000000000001},"max":{"1min":0.47799999999999998,"5min":0.59099999999999997,"15min":18.891999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.48799999999999999},"min":{"1min":0.26600000000000001,"5min":0.217,"15min":0.19700000000000001},"max":{"1min":0.67500000000000004,"5min":0.85199999999999998,"15min":10.138999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.40999999999999998,"15min":0.53500000000000003},"min":{"1min":0.25,"5min":0.23999999999999999,"15min":0.20399999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":18.919},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Feb 18 00:46:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.504,"15min":0.60699999999999998},"min":{"1min":0.33900000000000002,"5min":0.28499999999999998,"15min":0.24099999999999999},"max":{"1min":0.60799999999999998,"5min":8.5640000000000001,"15min":22.712},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.42399999999999999,"15min":0.58099999999999996},"min":{"1min":0.32400000000000001,"5min":0.28000000000000003,"15min":0.23899999999999999},"max":{"1min":0.56100000000000005,"5min":1.0229999999999999,"15min":22.359000000000002},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Feb 18 00:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.68200000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":18.812000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.67400000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.28699999999999998},"max":{"1min":0.76600000000000001,"5min":1.002,"15min":10.016},"last":0.751}]},{"osd":5,"last update":"Sun Feb 18 00:45:53 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.72299999999999998,"15min":0.69399999999999995},"min":{"1min":0.442,"5min":0.39600000000000002,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":11.805,"15min":11.805},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.70299999999999996,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.36899999999999999,"15min":0.317},"max":{"1min":0.81499999999999995,"5min":8.6059999999999999,"15min":10.114000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:45:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.72999999999999998,"15min":0.69699999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.35799999999999998},"max":{"1min":1.663,"5min":11.680999999999999,"15min":11.680999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60299999999999998,"15min":0.70099999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.34100000000000003},"max":{"1min":0.871,"5min":0.90500000000000003,"15min":18.861000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.752,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.40799999999999997,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":11.746,"15min":11.746},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67300000000000004,"15min":0.73799999999999999},"min":{"1min":0.54000000000000004,"5min":0.45300000000000001,"15min":0.372},"max":{"1min":0.90200000000000002,"5min":1.115,"15min":10.058999999999999},"last":0.72299999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149921,"num_pgs":85,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":320096,"kb_used_data":6488,"kb_used_omap":0,"kb_used_meta":313600,"kb_avail":93425056,"statfs":{"total":95995035648,"available":95667257344,"internally_reserved":0,"allocated":6643712,"data_stored":3572177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321126400},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34999999999999998,"15min":0.45700000000000002},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":25.082999999999998},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38,"15min":0.50800000000000001},"min":{"1min":0.222,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":18.911000000000001},"last":0.316}]},{"osd":1,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.37,"15min":0.45300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.223},"max":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":6.8479999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.38400000000000001,"15min":0.68400000000000005},"min":{"1min":0.26700000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.59999999999999998,"5min":0.73899999999999999,"15min":23.606999999999999},"last":0.376}]},{"osd":2,"last update":"Sun Feb 18 00:45:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41299999999999998,"15min":0.64300000000000002},"min":{"1min":0.28000000000000003,"5min":0.25700000000000001,"15min":0.23799999999999999},"max":{"1min":0.877,"5min":0.877,"15min":23.405000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.378,"5min":0.40000000000000002,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.253,"15min":0.23999999999999999},"max":{"1min":0.57699999999999996,"5min":0.85599999999999998,"15min":5.3479999999999999},"last":0.30099999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:45:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.61199999999999999,"15min":0.69899999999999995},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.33600000000000002},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":10.131},"last":0.441},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.63100000000000001},"min":{"1min":0.41399999999999998,"5min":0.36599999999999999,"15min":0.32700000000000001},"max":{"1min":0.75900000000000001,"5min":0.90900000000000003,"15min":5.4630000000000001},"last":0.35299999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58299999999999996,"15min":0.63400000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.32800000000000001},"max":{"1min":0.70899999999999996,"5min":0.92700000000000005,"15min":7.4249999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.67900000000000005},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.32400000000000001},"max":{"1min":0.81599999999999995,"5min":0.98999999999999999,"15min":6.7750000000000004},"last":0.36199999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:45:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60599999999999998,"15min":0.747},"min":{"1min":0.45600000000000002,"5min":0.375,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.93700000000000006,"15min":10.189},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.66900000000000004},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.39000000000000001},"max":{"1min":0.93999999999999995,"5min":0.96199999999999997,"15min":6.4279999999999999},"last":0.44800000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.621,"15min":0.69699999999999995},"min":{"1min":0.47599999999999998,"5min":0.39500000000000002,"15min":0.35199999999999998},"max":{"1min":0.88100000000000001,"5min":0.95399999999999996,"15min":6.8220000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.65000000000000002,"15min":0.745},"min":{"1min":0.435,"5min":0.435,"15min":0.39800000000000002},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":10.164999999999999},"last":0.41299999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986396,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311940,"kb_used_data":6524,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93433212,"statfs":{"total":95995035648,"available":95675609088,"internally_reserved":0,"allocated":6680576,"data_stored":3621992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53000000000000003,"15min":0.86299999999999999},"min":{"1min":0.33300000000000002,"5min":0.249,"15min":0.247},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":77.581999999999994},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.51100000000000001,"15min":0.83799999999999997},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":78.063999999999993},"last":0.60799999999999998}]},{"osd":1,"last update":"Sun Feb 18 00:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.61599999999999999},"min":{"1min":0.34599999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":9.3160000000000007},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.876},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.29699999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":81.242000000000004},"last":0.63100000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53200000000000003,"15min":0.61599999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.27900000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":11.691000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.66800000000000004},"min":{"1min":0.34200000000000003,"5min":0.30199999999999999,"15min":0.27200000000000002},"max":{"1min":0.90700000000000003,"5min":4.7149999999999999,"15min":17.77},"last":0.63900000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54600000000000004,"15min":0.58799999999999997},"min":{"1min":0.39000000000000001,"5min":0.33700000000000002,"15min":0.29599999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":7.218},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55900000000000005,"15min":0.73199999999999998},"min":{"1min":0.36099999999999999,"5min":0.313,"15min":0.309},"max":{"1min":0.95599999999999996,"5min":1.21,"15min":33.530999999999999},"last":0.623}]},{"osd":5,"last update":"Sun Feb 18 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45400000000000001,"15min":0.44800000000000001},"min":{"1min":0.308,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.498},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46300000000000002,"15min":0.45800000000000002},"min":{"1min":0.32900000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.73699999999999999,"5min":1.45,"15min":1.45},"last":0.41699999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46899999999999997,"15min":0.45900000000000002},"min":{"1min":0.308,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.52700000000000002,"15min":0.50600000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.2970000000000002,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.27900000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51100000000000001},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.28000000000000003},"max":{"1min":0.66500000000000004,"5min":0.81100000000000005,"15min":0.83299999999999996},"last":0.47899999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724759,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":307796,"kb_used_data":6476,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93437356,"statfs":{"total":95995035648,"available":95679852544,"internally_reserved":0,"allocated":6631424,"data_stored":3580825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.497,"15min":0.53500000000000003},"min":{"1min":0.33000000000000002,"5min":0.27300000000000002,"15min":0.217},"max":{"1min":0.72399999999999998,"5min":2.21,"15min":6.3150000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.501,"15min":0.60399999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26000000000000001},"max":{"1min":0.69799999999999995,"5min":1.782,"15min":12.744},"last":0.36499999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55600000000000005,"15min":0.64300000000000002},"min":{"1min":0.36599999999999999,"5min":0.28699999999999998,"15min":0.27400000000000002},"max":{"1min":0.66300000000000003,"5min":2.1509999999999998,"15min":13.25},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54400000000000004,"15min":0.65700000000000003},"min":{"1min":0.33000000000000002,"5min":0.311,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":1.9430000000000001,"15min":13.262},"last":0.46600000000000003}]},{"osd":2,"last update":"Sun Feb 18 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.58899999999999997},"min":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.251},"max":{"1min":0.76800000000000002,"5min":2.2559999999999998,"15min":6.1680000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58699999999999997,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.36699999999999999,"15min":0.308},"max":{"1min":0.70899999999999996,"5min":1.8799999999999999,"15min":6.1879999999999997},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.61899999999999999},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":0.81599999999999995,"5min":3.637,"15min":7.641},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53700000000000003,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.33600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65500000000000003,"5min":1.853,"15min":8.8650000000000002},"last":0.42399999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.44900000000000001,"15min":0.495},"min":{"1min":0.25800000000000001,"5min":0.255,"15min":0.23599999999999999},"max":{"1min":0.79400000000000004,"5min":1.897,"15min":11.818},"last":0.436},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.50700000000000001,"15min":0.49399999999999999},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.23300000000000001},"max":{"1min":0.80700000000000005,"5min":3.5920000000000001,"15min":4.274},"last":0.32000000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:45:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47999999999999998,"15min":0.56799999999999995},"min":{"1min":0.33000000000000002,"5min":0.27700000000000002,"15min":0.26300000000000001},"max":{"1min":0.77900000000000003,"5min":2.7120000000000002,"15min":18.605},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.32200000000000001,"5min":0.26900000000000002,"15min":0.26500000000000001},"max":{"1min":0.79900000000000004,"5min":2.6699999999999999,"15min":4.2089999999999996},"last":0.39000000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:45:35 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51900000000000002,"15min":0.51000000000000001},"min":{"1min":0.313,"5min":0.29799999999999999,"15min":0.26600000000000001},"max":{"1min":0.70899999999999996,"5min":1.819,"15min":2.0739999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.28899999999999998,"15min":0.28299999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":4.649},"last":0.372}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2093056,"data_stored":2093056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2068480,"data_stored":2068480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2068480,"data_stored":2068480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":65536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":106496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":65536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":234,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:46:06.050 INFO:tasks.thrashosds.thrasher:Setting 4.16 to [7, 2, 6, 3, 1, 0] 2024-02-18T00:46:06.051 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.16', '7', '2', '6', '3', '1', '0'] 2024-02-18T00:46:06.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pg-upmap-items 4.16 7 2 6 3 1 0 2024-02-18T00:46:06.096 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:04.970992+0000 mgr.y (mgr.24454) 685 : audit [DBG] from='client.41855 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:05.312222+0000 osd.3 (osd.3) 402 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:05.313213+0000 osd.3 (osd.3) 403 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:05.479174+0000 osd.5 (osd.5) 344 : cluster [DBG] 6.d scrub starts 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:05.482106+0000 osd.5 (osd.5) 345 : cluster [DBG] 6.d scrub ok 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:05.713888+0000 osd.1 (osd.1) 371 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:05.716812+0000 osd.1 (osd.1) 372 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:46:06.410 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:05.837080+0000 mon.a (mon.0) 3733 : cluster [DBG] osdmap e797: 8 total, 8 up, 7 in 2024-02-18T00:46:06.566 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:46:06.571 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:06.571 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:46:06.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.2.18:31 62.17 62:ee291d87:test-rados-api-smithi059-82957-38::foo:head [copy-from ver 2] snapc 0=[] ondisk+write+ignore_overlay+enforce_snapc+known_if_redirected e304)", 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 583.98481100599997, 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027920693, 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:46:06.697 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709668+0000", 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709665+0000", 2024-02-18T00:46:06.698 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1831000000000001e-05 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709677+0000", 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7572e-05 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709695+0000", 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1678000000000001e-05 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709717+0000", 2024-02-18T00:46:06.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0013999999999998e-05 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709777+0000", 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015211700000000001 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.709929+0000", 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014904904 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 3,5", 2024-02-18T00:46:06.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.724834+0000", 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0095493409999999994 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.734383+0000", 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4509999999999997e-06 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.734389+0000", 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003096742 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737486+0000", 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6187000000000003e-05 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737542+0000", 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6628000000000001e-05 2024-02-18T00:46:06.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:22.737589+0000", 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.027920693 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.40913.0:1 194.5s0 194:b5a28c71:::oid_FlushPP_:head [write 0~128] snapc 0=[] ondisk+write+known_if_redirected e330)", 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:36:48.680019+0000", 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 558.014460525, 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.000655482, 2024-02-18T00:46:06.703 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.40913", 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/1248208728", 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680019+0000", 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.704 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680019+0000", 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680017+0000", 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8699999999999997e-06 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680023+0000", 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8769999999999999e-06 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680032+0000", 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5729999999999996e-06 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680039+0000", 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2058999999999998e-05 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680081+0000", 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2289999999999999e-06 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.706 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for peered", 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:48.680084+0000", 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.99713135600000002 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.677215+0000", 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000111112 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.677326+0000", 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00029093199999999999 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.707 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.677617+0000", 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00039891900000000002 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.678016+0000", 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0026187279999999999 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.680635+0000", 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9344999999999997e-05 2024-02-18T00:46:06.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:36:49.680674+0000", 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.000655482 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "pg_update_log_missing(239.1d epoch 648/644 rep_tid 669 entries 648'3 (0'0) error 239:bd34d09e:test-rados-api-smithi059-82957-111::foo:head by client.31692.0:510 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 648'1)", 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:13.811213+0000", 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 172.88326607100001, 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021552380999999999, 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.709 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811213+0000", 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811213+0000", 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811212+0000", 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4560000000000006e-06 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811222+0000", 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1851999999999999e-05 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811234+0000", 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3380000000000004e-06 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811239+0000", 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2014e-05 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.811291+0000", 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021474416999999999 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:13.832766+0000", 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021552380999999999 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:797 234.6s0 234:76cde755:test-rados-api-smithi059-82957-115::bar:head [writefull 0~5] snapc 6=[6] ondisk+write+known_if_redirected e662)", 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:27.899368+0000", 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 158.795111588, 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047340079999999996, 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 797 2024-02-18T00:46:06.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899368+0000", 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899368+0000", 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899366+0000", 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.6719999999999993e-06 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899375+0000", 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6360000000000006e-06 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899384+0000", 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6490000000000002e-06 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899391+0000", 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3920999999999999e-05 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899465+0000", 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3130000000000001e-06 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.899472+0000", 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025174059999999998 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.901990+0000", 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010505099999999999 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.902095+0000", 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00068075299999999998 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.902775+0000", 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00051217000000000005 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.903287+0000", 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00075513000000000002 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904043+0000", 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9049999999999999e-05 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:27.904102+0000", 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047340079999999996 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.718 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.0.8:115 234.6s0 234:76cde755:test-rados-api-smithi059-82957-115::bar:snapdir [list-snaps] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e665)", 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:30.810973+0000", 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 155.88350626900001, 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0028050499999999999, 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810973+0000", 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810973+0000", 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999986 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810971+0000", 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2240000000000001e-06 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810981+0000", 2024-02-18T00:46:06.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6467999999999999e-05 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.810997+0000", 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4749999999999998e-06 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811004+0000", 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4396e-05 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811048+0000", 2024-02-18T00:46:06.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0390000000000004e-06 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811053+0000", 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025337060000000002 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813587+0000", 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4364999999999995e-05 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813681+0000", 2024-02-18T00:46:06.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6979999999999996e-05 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813778+0000", 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0028050499999999999 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.0.8:116 234.6s0 234:76cde755:test-rados-api-smithi059-82957-115::bar:head [copy-get max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e665)", 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 155.883467911, 2024-02-18T00:46:06.723 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003792218, 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811011+0000", 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2819999999999998e-06 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811015+0000", 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4520000000000002e-06 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811019+0000", 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8539999999999999e-06 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.725 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811022+0000", 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5864000000000004e-05 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811098+0000", 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9850000000000006e-06 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting_for_map not empty", 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.811106+0000", 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00269615 2024-02-18T00:46:06.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813802+0000", 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8986000000000006e-05 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.813881+0000", 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00092213699999999996 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:30.814804+0000", 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003792218 2024-02-18T00:46:06.727 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:850 241.10 241:08000000::::head [pgnls start_epoch 668] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e668)", 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:33.811761+0000", 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 152.882718657, 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.029407101000000001, 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 850 2024-02-18T00:46:06.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811761+0000", 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811761+0000", 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.729 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811758+0000", 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3589e-05 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811771+0000", 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8590999999999999e-05 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811790+0000", 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.73e-06 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.730 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811800+0000", 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6365000000000001e-05 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811846+0000", 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9429999999999997e-06 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.811852+0000", 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.029175264999999999 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.731 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.841027+0000", 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7128000000000001e-05 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.841064+0000", 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103311 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:33.841168+0000", 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.029407101000000001 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1101 243.11 243:8a5ddf46:test-rados-api-smithi059-82957-119::foo:head [writefull 0~8] snapc 0=[] ondisk+write+known_if_redirected e682)", 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:47.809417+0000", 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 138.88506183300001, 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0060360800000000001, 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1101 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.733 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809417+0000", 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809417+0000", 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999967 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809414+0000", 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8352999999999999e-05 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.734 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809432+0000", 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3720000000000002e-05 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809466+0000", 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2218999999999999e-05 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809488+0000", 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016282800000000001 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.735 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809651+0000", 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2593e-05 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.809664+0000", 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0032550019999999999 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.812919+0000", 2024-02-18T00:46:06.736 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000245875 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for blocked object", 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.813165+0000", 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001204608 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.814369+0000", 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8076000000000003e-05 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.814437+0000", 2024-02-18T00:46:06.737 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015850799999999999 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for subops from 0,5", 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.814596+0000", 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047855400000000001 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op_commit", 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.815074+0000", 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00021771599999999999 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:46:06.738 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.815292+0000", 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4890999999999999e-05 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_commit_rec", 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.815357+0000", 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7215e-05 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.815414+0000", 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9218999999999998e-05 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.739 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:47.815454+0000", 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0060360800000000001 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1147 234.2s0 234:40000000:test-rados-api-smithi059-82957-119:::head [pgnls start_epoch 684] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e684)", 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:43:52.847595+0000", 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 133.84688467300001, 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024422189999999998, 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.740 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1147 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847595+0000", 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.741 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847595+0000", 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847592+0000", 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7149000000000001e-05 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847609+0000", 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9403e-05 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.742 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847638+0000", 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9372999999999999e-05 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847658+0000", 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000112433 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847770+0000", 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1550000000000001e-06 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.743 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.847777+0000", 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020388390000000002 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.849816+0000", 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8438e-05 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.744 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.849854+0000", 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00018240500000000001 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:43:52.850037+0000", 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024422189999999998 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1529 245.0 245:00000000:test-rados-api-smithi059-82957-123:::head [pgnls start_epoch 702] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e702)", 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:10.820786+0000", 2024-02-18T00:46:06.745 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 115.87369361499999, 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021249279999999999, 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1529 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820786+0000", 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.746 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820786+0000", 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820783+0000", 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7389999999999994e-06 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820793+0000", 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5e-05 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.747 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820808+0000", 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2380000000000002e-06 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820814+0000", 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3826999999999999e-05 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820858+0000", 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.339e-06 2024-02-18T00:46:06.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.820863+0000", 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0017620730000000001 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.822626+0000", 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2886999999999999e-05 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.822658+0000", 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00025217099999999998 2024-02-18T00:46:06.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:10.822911+0000", 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021249279999999999 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1581 245.0 245:00000000:%01:::head [pgnls start_epoch 705] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e705)", 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:13.802550+0000", 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 112.891929457, 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024888089999999998, 2024-02-18T00:46:06.750 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1581 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802550+0000", 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.751 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802550+0000", 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802548+0000", 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2290000000000003e-06 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802556+0000", 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3628000000000001e-05 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.752 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802570+0000", 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6420000000000001e-06 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802577+0000", 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8942000000000001e-05 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802616+0000", 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4139999999999996e-06 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.753 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.802620+0000", 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019871089999999999 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.804607+0000", 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1579999999999998e-05 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.804649+0000", 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00038994600000000001 2024-02-18T00:46:06.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:13.805039+0000", 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024888089999999998 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.0.8:119 234.6s0 234:642486d8:test-rados-api-smithi059-82957-127::foo:head [copy-get max 8388608] snapc 0=[] ondisk+read+rwordered+ignore_cache+ignore_overlay+map_snap_clone+known_if_redirected e721)", 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:29.847140+0000", 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 96.847339446999996, 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020759730000000001, 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.755 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847140+0000", 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847140+0000", 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999995 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.756 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847139+0000", 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.162e-06 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847143+0000", 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6330000000000002e-06 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847146+0000", 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7470000000000001e-06 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.757 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847149+0000", 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5585e-05 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847245+0000", 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1570000000000002e-06 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting_for_map not empty", 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.847249+0000", 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001874459 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.758 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.849123+0000", 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2660999999999999e-05 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.849216+0000", 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020759730000000001 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(osd.0.8:120 234.6s0 234:642486d8:test-rados-api-smithi059-82957-127::foo:head [copy-from ver 2] snapc 0=[] ondisk+write+ignore_overlay+enforce_snapc+known_if_redirected e721)", 2024-02-18T00:46:06.759 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:29.861955+0000", 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 96.832523901000002, 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021618116, 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.861955+0000", 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.861955+0000", 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999967 2024-02-18T00:46:06.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.861952+0000", 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8233e-05 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.861970+0000", 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2002999999999998e-05 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.862002+0000", 2024-02-18T00:46:06.761 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5243e-05 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.862018+0000", 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117435 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.862135+0000", 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00021718599999999999 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.862352+0000", 2024-02-18T00:46:06.762 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.014832789000000001 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.877185+0000", 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002678897 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.879864+0000", 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.003657651 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-02-18T00:46:06.763 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.883521+0000", 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1986e-05 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:29.883573+0000", 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.021618116 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1832 247.8 247:10000000::::head [pgnls start_epoch 723] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e723)", 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:31.875551+0000", 2024-02-18T00:46:06.764 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 94.818928163999999, 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020881599999999999, 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1832 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875551+0000", 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.765 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875551+0000", 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999971 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875548+0000", 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4603000000000001e-05 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875563+0000", 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1263999999999998e-05 2024-02-18T00:46:06.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875584+0000", 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6459999999999993e-06 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875592+0000", 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.914e-05 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875641+0000", 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7079999999999996e-06 2024-02-18T00:46:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.875646+0000", 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0018277980000000001 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.877473+0000", 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5420000000000003e-05 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.877509+0000", 2024-02-18T00:46:06.768 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013045 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:31.877639+0000", 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020881599999999999 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:1941 248.12 248:48000000::::head [pgnls start_epoch 731] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e731)", 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:44:39.943681+0000", 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 86.750798680000003, 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020802479999999998, 2024-02-18T00:46:06.769 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 1941 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943681+0000", 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.770 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943681+0000", 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943678+0000", 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1816e-05 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943690+0000", 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8504000000000001e-05 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.771 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943709+0000", 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8009999999999993e-06 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943718+0000", 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1575e-05 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943769+0000", 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.429e-06 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.772 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.943775+0000", 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0018853299999999999 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.945660+0000", 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0380000000000001e-05 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.945690+0000", 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0581999999999996e-05 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:44:39.945761+0000", 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020802479999999998 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2068 249.a 249:50000000::::head [pgnls start_epoch 742] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e742)", 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:00.082517+0000", 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 66.611962120000001, 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0026657959999999998, 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.774 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2068 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082517+0000", 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082517+0000", 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:46:06.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082515+0000", 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2374999999999999e-05 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082527+0000", 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9616999999999999e-05 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082547+0000", 2024-02-18T00:46:06.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2339999999999997e-06 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082554+0000", 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4197999999999996e-05 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082638+0000", 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3719999999999999e-06 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.082645+0000", 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023924340000000001 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085038+0000", 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9458000000000001e-05 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085077+0000", 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000105596 2024-02-18T00:46:06.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:00.085183+0000", 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0026657959999999998 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2331 251.11 251:88000000:test-rados-api-smithi059-82957-135:::head [pgnls start_epoch 762] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e762)", 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:19.844858+0000", 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 46.849621646999999, 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002481876, 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.779 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2331 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844858+0000", 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844858+0000", 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999981 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844856+0000", 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0881e-05 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844866+0000", 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8110000000000001e-05 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844885+0000", 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6389999999999992e-06 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844892+0000", 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1388000000000003e-05 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844944+0000", 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1789999999999996e-06 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.844950+0000", 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022441010000000001 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.847194+0000", 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3050999999999999e-05 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.847227+0000", 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000112617 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.783 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:19.847339+0000", 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002481876 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.31692.0:2459 251.11 251:88000000:test-rados-api-smithi059-82957-135:::head [pgnls start_epoch 763] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected e763)", 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-02-18T00:45:23.909260+0000", 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 42.785219695000002, 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025523159999999998, 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-02-18T00:46:06.784 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.31692", 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.59:0/337723220", 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 2459 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909260+0000", 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909260+0000", 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4294967295.9999976 2024-02-18T00:46:06.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909257+0000", 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.312e-05 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909270+0000", 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9596e-05 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909290+0000", 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.153e-06 2024-02-18T00:46:06.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909297+0000", 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6468999999999998e-05 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909353+0000", 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1000000000000003e-06 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "op must wait for map", 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.909358+0000", 2024-02-18T00:46:06.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023051479999999999 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.911664+0000", 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4335999999999999e-05 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.911688+0000", 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000123934 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-02-18T00:45:23.911812+0000", 2024-02-18T00:46:06.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025523159999999998 2024-02-18T00:46:06.789 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.789 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.789 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.789 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-02-18T00:46:06.789 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-02-18T00:46:06.789 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-02-18T00:46:06.789 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:04.970992+0000 mgr.y (mgr.24454) 685 : audit [DBG] from='client.41855 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:05.312222+0000 osd.3 (osd.3) 402 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:05.313213+0000 osd.3 (osd.3) 403 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:05.479174+0000 osd.5 (osd.5) 344 : cluster [DBG] 6.d scrub starts 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:05.482106+0000 osd.5 (osd.5) 345 : cluster [DBG] 6.d scrub ok 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:05.713888+0000 osd.1 (osd.1) 371 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:05.716812+0000 osd.1 (osd.1) 372 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:05.837080+0000 mon.a (mon.0) 3733 : cluster [DBG] osdmap e797: 8 total, 8 up, 7 in 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:04.970992+0000 mgr.y (mgr.24454) 685 : audit [DBG] from='client.41855 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:05.312222+0000 osd.3 (osd.3) 402 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:05.313213+0000 osd.3 (osd.3) 403 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:46:06.790 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:05.479174+0000 osd.5 (osd.5) 344 : cluster [DBG] 6.d scrub starts 2024-02-18T00:46:06.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:05.482106+0000 osd.5 (osd.5) 345 : cluster [DBG] 6.d scrub ok 2024-02-18T00:46:06.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:05.713888+0000 osd.1 (osd.1) 371 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:46:06.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:05.716812+0000 osd.1 (osd.1) 372 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:46:06.791 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:05.837080+0000 mon.a (mon.0) 3733 : cluster [DBG] osdmap e797: 8 total, 8 up, 7 in 2024-02-18T00:46:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:07.138 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:07.150 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:46:07.156 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:07.157 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:46:07.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:46:07.288 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:07.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:06.030946+0000 mgr.y (mgr.24454) 686 : cluster [DBG] pgmap v1228: 182 pgs: 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:06.280386+0000 osd.3 (osd.3) 404 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:06.281421+0000 osd.3 (osd.3) 405 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:06.500308+0000 osd.5 (osd.5) 346 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:06.501210+0000 osd.5 (osd.5) 347 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:06.841498+0000 mon.a (mon.0) 3734 : cluster [DBG] osdmap e798: 8 total, 8 up, 7 in 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:06.843265+0000 mon.a (mon.0) 3735 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:06.030946+0000 mgr.y (mgr.24454) 686 : cluster [DBG] pgmap v1228: 182 pgs: 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:46:07.475 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:06.280386+0000 osd.3 (osd.3) 404 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:46:07.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:06.281421+0000 osd.3 (osd.3) 405 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:46:07.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:06.500308+0000 osd.5 (osd.5) 346 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:46:07.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:06.501210+0000 osd.5 (osd.5) 347 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:46:07.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:06.841498+0000 mon.a (mon.0) 3734 : cluster [DBG] osdmap e798: 8 total, 8 up, 7 in 2024-02-18T00:46:07.476 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:06.843265+0000 mon.a (mon.0) 3735 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:06.030946+0000 mgr.y (mgr.24454) 686 : cluster [DBG] pgmap v1228: 182 pgs: 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:46:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:06.280386+0000 osd.3 (osd.3) 404 : cluster [DBG] 5.17 scrub starts 2024-02-18T00:46:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:06.281421+0000 osd.3 (osd.3) 405 : cluster [DBG] 5.17 scrub ok 2024-02-18T00:46:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:06.500308+0000 osd.5 (osd.5) 346 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:46:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:06.501210+0000 osd.5 (osd.5) 347 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:46:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:06.841498+0000 mon.a (mon.0) 3734 : cluster [DBG] osdmap e798: 8 total, 8 up, 7 in 2024-02-18T00:46:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:06.843265+0000 mon.a (mon.0) 3735 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T00:46:07.604 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_ops_in_flight 2024-02-18T00:46:07.637 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:46:07.640 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:07.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:46:07.742 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3 2024-02-18T00:46:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:07.912 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:46:08.122 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:46:08.125 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:08.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3' 2024-02-18T00:46:08.226 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1 2024-02-18T00:46:08.257 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:07.122868+0000 osd.4 (osd.4) 415 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:46:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:07.125874+0000 osd.4 (osd.4) 416 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:46:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:07.456832+0000 osd.5 (osd.5) 348 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:46:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:07.458116+0000 osd.5 (osd.5) 349 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:46:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:07.751673+0000 osd.2 (osd.2) 311 : cluster [DBG] 5.b scrub starts 2024-02-18T00:46:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:07.752615+0000 osd.2 (osd.2) 312 : cluster [DBG] 5.b scrub ok 2024-02-18T00:46:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:07.845007+0000 mon.a (mon.0) 3736 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:07.845193+0000 mon.a (mon.0) 3737 : cluster [DBG] osdmap e799: 8 total, 8 up, 7 in 2024-02-18T00:46:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:07.855071+0000 mon.a (mon.0) 3738 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-101","var": "dedup_tier","val": "test-rados-api-smithi059-82957-144"}]: dispatch 2024-02-18T00:46:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:07.882813+0000 mon.a (mon.0) 3739 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:07.883196+0000 mon.a (mon.0) 3740 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-144"}]: dispatch 2024-02-18T00:46:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:08.225527+0000 mon.c (mon.2) 581 : audit [DBG] from='client.? 172.21.15.59:0/3632245751' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:46:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.280459+0000 osd.3 (osd.3) 406 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:46:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.281748+0000 osd.3 (osd.3) 407 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:46:08.391 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:07.122868+0000 osd.4 (osd.4) 415 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:07.125874+0000 osd.4 (osd.4) 416 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:07.456832+0000 osd.5 (osd.5) 348 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:07.458116+0000 osd.5 (osd.5) 349 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:07.751673+0000 osd.2 (osd.2) 311 : cluster [DBG] 5.b scrub starts 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:07.752615+0000 osd.2 (osd.2) 312 : cluster [DBG] 5.b scrub ok 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:07.845007+0000 mon.a (mon.0) 3736 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:07.845193+0000 mon.a (mon.0) 3737 : cluster [DBG] osdmap e799: 8 total, 8 up, 7 in 2024-02-18T00:46:08.392 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:07.855071+0000 mon.a (mon.0) 3738 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-101","var": "dedup_tier","val": "test-rados-api-smithi059-82957-144"}]: dispatch 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:07.882813+0000 mon.a (mon.0) 3739 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:07.883196+0000 mon.a (mon.0) 3740 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-144"}]: dispatch 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:08.225527+0000 mon.c (mon.2) 581 : audit [DBG] from='client.? 172.21.15.59:0/3632245751' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.280459+0000 osd.3 (osd.3) 406 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.281748+0000 osd.3 (osd.3) 407 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:07.122868+0000 osd.4 (osd.4) 415 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:07.125874+0000 osd.4 (osd.4) 416 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:07.456832+0000 osd.5 (osd.5) 348 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:07.458116+0000 osd.5 (osd.5) 349 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:07.751673+0000 osd.2 (osd.2) 311 : cluster [DBG] 5.b scrub starts 2024-02-18T00:46:08.393 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:07.752615+0000 osd.2 (osd.2) 312 : cluster [DBG] 5.b scrub ok 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:07.845007+0000 mon.a (mon.0) 3736 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi059-82957-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:07.845193+0000 mon.a (mon.0) 3737 : cluster [DBG] osdmap e799: 8 total, 8 up, 7 in 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:07.855071+0000 mon.a (mon.0) 3738 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi059-82957-101","var": "dedup_tier","val": "test-rados-api-smithi059-82957-144"}]: dispatch 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:07.882813+0000 mon.a (mon.0) 3739 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:07.883196+0000 mon.a (mon.0) 3740 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi059-82957-101", "tierpool": "test-rados-api-smithi059-82957-144"}]: dispatch 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:08.225527+0000 mon.c (mon.2) 581 : audit [DBG] from='client.? 172.21.15.59:0/3632245751' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.280459+0000 osd.3 (osd.3) 406 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:46:08.394 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.281748+0000 osd.3 (osd.3) 407 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:46:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:46:08.508 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:08.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1' 2024-02-18T00:46:08.610 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:46:08.638 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:08.710 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:46:08.712 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:08.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:46:08.814 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:46:08.840 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:08.915 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:46:08.926 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:08.927 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:46:09.028 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4 2024-02-18T00:46:09.054 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:09.602 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:46:09.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.031632+0000 mgr.y (mgr.24454) 687 : cluster [DBG] pgmap v1231: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:46:09.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.506376+0000 osd.5 (osd.5) 350 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:46:09.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.507226+0000 osd.5 (osd.5) 351 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:46:09.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.647763+0000 osd.1 (osd.1) 373 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:46:09.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.650652+0000 osd.1 (osd.1) 374 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:46:09.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.801303+0000 osd.2 (osd.2) 313 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:46:09.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.802041+0000 osd.2 (osd.2) 314 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:46:09.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:08.850714+0000 mon.a (mon.0) 3741 : cluster [DBG] osdmap e800: 8 total, 8 up, 7 in 2024-02-18T00:46:09.605 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:09.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4' 2024-02-18T00:46:09.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 -- 172.21.15.59:0/2549440429 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 msgr2=0x7fd51c0f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 --2- 172.21.15.59:0/2549440429 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c0f9090 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 rx=0x7fd50c004870 tx=0x7fd50c009cb0).stop 2024-02-18T00:46:09.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 -- 172.21.15.59:0/2549440429 shutdown_connections 2024-02-18T00:46:09.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 --2- 172.21.15.59:0/2549440429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd51c102210 0x7fd51c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 --2- 172.21.15.59:0/2549440429 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c0f9090 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 --2- 172.21.15.59:0/2549440429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd51c0f9610 0x7fd51c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 -- 172.21.15.59:0/2549440429 >> 172.21.15.59:0/2549440429 conn(0x7fd51c0f2820 msgr2=0x7fd51c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 -- 172.21.15.59:0/2549440429 shutdown_connections 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 -- 172.21.15.59:0/2549440429 wait complete. 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.628+0000 7fd521502700 1 librados: starting msgr at 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 librados: starting objecter 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 Processor -- start 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 -- start start 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 librados: setting wanted keys 2024-02-18T00:46:09.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 librados: calling monclient init 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c10e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd51c0f9610 0x7fd51c10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd51c102210 0x7fd51c1083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c108890 con 0x7fd51c102210 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c108a00 con 0x7fd51c0f9610 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd521502700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c108bc0 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd51affd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c10e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd51affd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c10e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49248/0 (socket says 172.21.15.59:49248) 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd51affd700 1 -- 172.21.15.59:0/3135176221 learned_addr learned my addr 172.21.15.59:0/3135176221 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:09.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.629+0000 7fd51b7fe700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd51c102210 0x7fd51c1083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd51a7fc700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd51c0f9610 0x7fd51c10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd51affd700 1 -- 172.21.15.59:0/3135176221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd51c0f9610 msgr2=0x7fd51c10e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd51affd700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd51c0f9610 0x7fd51c10e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd51affd700 1 -- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd51c102210 msgr2=0x7fd51c1083a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd51affd700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd51c102210 0x7fd51c1083a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd51affd700 1 -- 172.21.15.59:0/3135176221 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c005040 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd51affd700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c10e200 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7fd50c009eb0 tx=0x7fd50c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd513fff700 10 client.?.objecter ms_handle_connect 0x7fd51c0f8cb0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd513fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd513fff700 1 -- 172.21.15.59:0/3135176221 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00e070 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c108e40 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd521502700 10 client.41867.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c115030 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd513fff700 1 -- 172.21.15.59:0/3135176221 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd50c013da0 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd521502700 1 librados: init done 2024-02-18T00:46:09.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.630+0000 7fd513fff700 1 -- 172.21.15.59:0/3135176221 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c019070 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.631+0000 7fd513fff700 1 -- 172.21.15.59:0/3135176221 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd50c0273f0 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.631+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc004f50 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd513fff700 1 --2- 172.21.15.59:0/3135176221 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd508062c90 0x7fd508065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd513fff700 1 -- 172.21.15.59:0/3135176221 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(800..800 src has 418..800) v4 ==== 8404+0+0 (secure 0 0 0) 0x7fd50c07ebc0 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd513fff700 10 client.41867.objecter ms_dispatch 0x7fd51c001550 osd_map(800..800 src has 418..800) v4 2024-02-18T00:46:09.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd513fff700 3 client.41867.objecter handle_osd_map got epochs [800,800] > 0 2024-02-18T00:46:09.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd513fff700 3 client.41867.objecter handle_osd_map decoding full epoch 800 2024-02-18T00:46:09.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd51a7fc700 1 --2- 172.21.15.59:0/3135176221 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd508062c90 0x7fd508065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd513fff700 20 client.41867.objecter dump_active .. 0 homeless 2024-02-18T00:46:09.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd513fff700 10 client.41867.objecter ms_handle_connect 0x7fd508062c90 2024-02-18T00:46:09.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.632+0000 7fd51a7fc700 1 --2- 172.21.15.59:0/3135176221 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd508062c90 0x7fd508065150 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 rx=0x7fd51c109b40 tx=0x7fd50400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.636+0000 7fd513fff700 1 -- 172.21.15.59:0/3135176221 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd50c051f50 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:46:09.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.031632+0000 mgr.y (mgr.24454) 687 : cluster [DBG] pgmap v1231: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:46:09.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.506376+0000 osd.5 (osd.5) 350 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:46:09.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.507226+0000 osd.5 (osd.5) 351 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:46:09.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.647763+0000 osd.1 (osd.1) 373 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:46:09.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.650652+0000 osd.1 (osd.1) 374 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.801303+0000 osd.2 (osd.2) 313 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.802041+0000 osd.2 (osd.2) 314 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:08.850714+0000 mon.a (mon.0) 3741 : cluster [DBG] osdmap e800: 8 total, 8 up, 7 in 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.031632+0000 mgr.y (mgr.24454) 687 : cluster [DBG] pgmap v1231: 214 pgs: 32 unknown, 1 active+remapped+backfill_toofull, 181 active+clean; 3.8 MiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s; 1/810 objects misplaced (0.123%) 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.506376+0000 osd.5 (osd.5) 350 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.507226+0000 osd.5 (osd.5) 351 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.647763+0000 osd.1 (osd.1) 373 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.650652+0000 osd.1 (osd.1) 374 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.801303+0000 osd.2 (osd.2) 313 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.802041+0000 osd.2 (osd.2) 314 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:46:09.734 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:08.850714+0000 mon.a (mon.0) 3741 : cluster [DBG] osdmap e800: 8 total, 8 up, 7 in 2024-02-18T00:46:09.736 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.220+0000 7f00277e2700 1 -- 172.21.15.59:0/3791436819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fcbb0 msgr2=0x7f00200fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.220+0000 7f00277e2700 1 --2- 172.21.15.59:0/3791436819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fcbb0 0x7f00200fcf90 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 rx=0x7f001c0047d0 tx=0x7f001c009cb0).stop 2024-02-18T00:46:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.220+0000 7f00277e2700 1 -- 172.21.15.59:0/3791436819 shutdown_connections 2024-02-18T00:46:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.220+0000 7f00277e2700 1 --2- 172.21.15.59:0/3791436819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00200f7400 0x7f0020103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.220+0000 7f00277e2700 1 --2- 172.21.15.59:0/3791436819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fcbb0 0x7f00200fcf90 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.220+0000 7f00277e2700 1 --2- 172.21.15.59:0/3791436819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200fd510 0x7f00200f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.220+0000 7f00277e2700 1 -- 172.21.15.59:0/3791436819 >> 172.21.15.59:0/3791436819 conn(0x7f00200f2820 msgr2=0x7f00200f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:09.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 -- 172.21.15.59:0/3791436819 shutdown_connections 2024-02-18T00:46:09.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 -- 172.21.15.59:0/3791436819 wait complete. 2024-02-18T00:46:09.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 librados: starting msgr at 2024-02-18T00:46:09.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 librados: starting objecter 2024-02-18T00:46:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 Processor -- start 2024-02-18T00:46:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 -- start start 2024-02-18T00:46:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 librados: setting wanted keys 2024-02-18T00:46:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 librados: calling monclient init 2024-02-18T00:46:09.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00200f7400 0x7f0020110430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200fcbb0 0x7f0020110920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fd510 0x7f002010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002010aac0 con 0x7f00200fcbb0 2024-02-18T00:46:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002010ac00 con 0x7f00200f7400 2024-02-18T00:46:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.221+0000 7f00277e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002010adc0 con 0x7f00200fd510 2024-02-18T00:46:09.809 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0024d7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200fcbb0 0x7f0020110920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0025d7f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fd510 0x7f002010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0024d7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200fcbb0 0x7f0020110920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46330/0 (socket says 172.21.15.59:46330) 2024-02-18T00:46:09.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0024d7d700 1 -- 172.21.15.59:0/3632245751 learned_addr learned my addr 172.21.15.59:0/3632245751 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:09.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00200f7400 0x7f0020110430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0025d7f700 1 -- 172.21.15.59:0/3632245751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00200f7400 msgr2=0x7f0020110430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0025d7f700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00200f7400 0x7f0020110430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0025d7f700 1 -- 172.21.15.59:0/3632245751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200fcbb0 msgr2=0x7f0020110920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0025d7f700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200fcbb0 0x7f0020110920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0025d7f700 1 -- 172.21.15.59:0/3632245751 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014005040 con 0x7f00200fd510 2024-02-18T00:46:09.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0024d7d700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00200fcbb0 0x7f0020110920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:09.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00200f7400 0x7f0020110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:09.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f0025d7f700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fd510 0x7f002010a5d0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 rx=0x7f001800eb70 tx=0x7f001800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00127fc700 10 client.?.objecter ms_handle_connect 0x7f00200fd510 2024-02-18T00:46:09.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00127fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:09.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018009870 con 0x7f00200fd510 2024-02-18T00:46:09.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f001800f040 con 0x7f00200fd510 2024-02-18T00:46:09.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c005040 con 0x7f00200fd510 2024-02-18T00:46:09.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018007da0 con 0x7f00200fd510 2024-02-18T00:46:09.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00277e2700 10 client.41861.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:09.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002010b3f0 con 0x7f00200fd510 2024-02-18T00:46:09.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.222+0000 7f00277e2700 1 librados: init done 2024-02-18T00:46:09.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.223+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd ls"} v 0) v1 -- 0x7f002005fa60 con 0x7f00200fd510 2024-02-18T00:46:09.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f00180040b0 con 0x7f00200fd510 2024-02-18T00:46:09.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6800/3721891797 conn(0x7f000c062c90 0x7f000c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(799..799 src has 418..799) v4 ==== 8790+0+0 (secure 0 0 0) 0x7f001807d180 con 0x7f00200fd510 2024-02-18T00:46:09.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 osd_map(799..799 src has 418..799) v4 2024-02-18T00:46:09.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 3 client.41861.objecter handle_osd_map got epochs [799,799] > 0 2024-02-18T00:46:09.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 3 client.41861.objecter handle_osd_map decoding full epoch 799 2024-02-18T00:46:09.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6800/3721891797 conn(0x7f000c062c90 0x7f000c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 20 client.41861.objecter dump_active .. 0 homeless 2024-02-18T00:46:09.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.224+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v799) v1 ==== 54+0+16 (secure 0 0 0) 0x7f0018084070 con 0x7f00200fd510 2024-02-18T00:46:09.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f000c062c90 2024-02-18T00:46:09.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6800/3721891797 conn(0x7f000c062c90 0x7f000c065150 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 rx=0x7f001c009da0 tx=0x7f001c009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 10 client.41861.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 1 2024-02-18T00:46:09.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6801/3700238977 conn(0x7f00200fdea0 0x7f00200fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 1 2024-02-18T00:46:09.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 0 1 2024-02-18T00:46:09.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020110100 osd=0 4 2024-02-18T00:46:09.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 10 client.41861.objecter _send_command 1 2024-02-18T00:46:09.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0020105070 con 0x7f00200fdea0 2024-02-18T00:46:09.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f0024d7d700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6801/3700238977 conn(0x7f00200fdea0 0x7f00200fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f0024d7d700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6801/3700238977 conn(0x7f00200fdea0 0x7f00200fe280 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.225+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f00200fdea0 2024-02-18T00:46:09.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.226+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0020105070 con 0x7f00200fdea0 2024-02-18T00:46:09.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.226+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:46:09.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.226+0000 7f00127fc700 10 client.41861.objecter _finish_command 1 = system:0 2024-02-18T00:46:09.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.226+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f0020110100 osd=0 4 2024-02-18T00:46:09.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.226+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 0 1 2024-02-18T00:46:09.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 10 client.41861.objecter _submit_command 2 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.819 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 2 2024-02-18T00:46:09.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 2 2024-02-18T00:46:09.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f0020110100 osd=0 3 2024-02-18T00:46:09.820 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 0 2 2024-02-18T00:46:09.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020110100 osd=0 4 2024-02-18T00:46:09.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 10 client.41861.objecter _send_command 2 2024-02-18T00:46:09.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.240+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f00200fe9b0 con 0x7f00200fdea0 2024-02-18T00:46:09.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f001400f040 con 0x7f00200fdea0 2024-02-18T00:46:09.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 2: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00127fc700 10 client.41861.objecter _finish_command 2 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f0020110100 osd=0 4 2024-02-18T00:46:09.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 0 2 2024-02-18T00:46:09.823 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0: {} 2024-02-18T00:46:09.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 10 client.41861.objecter _submit_command 3 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 3 2024-02-18T00:46:09.823 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6805/26279110 conn(0x7f0020028540 0x7f00200598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 3 2024-02-18T00:46:09.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 1 3 2024-02-18T00:46:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002005ef10 osd=1 4 2024-02-18T00:46:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 10 client.41861.objecter _send_command 3 2024-02-18T00:46:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.242+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6805/26279110 -- command(tid 3: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0020059ee0 con 0x7f0020028540 2024-02-18T00:46:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.243+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6805/26279110 conn(0x7f0020028540 0x7f00200598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.243+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f0020028540 2024-02-18T00:46:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.243+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6805/26279110 conn(0x7f0020028540 0x7f00200598f0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.244+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 3: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0020059ee0 con 0x7f0020028540 2024-02-18T00:46:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.244+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 3: 0 ) v1 2024-02-18T00:46:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.244+0000 7f00127fc700 10 client.41861.objecter _finish_command 3 = system:0 2024-02-18T00:46:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.244+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002005ef10 osd=1 4 2024-02-18T00:46:09.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.244+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 1 3 2024-02-18T00:46:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 10 client.41861.objecter _submit_command 4 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 4 2024-02-18T00:46:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 4 2024-02-18T00:46:09.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002005ef10 osd=1 3 2024-02-18T00:46:09.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 1 4 2024-02-18T00:46:09.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002005ef10 osd=1 4 2024-02-18T00:46:09.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 10 client.41861.objecter _send_command 4 2024-02-18T00:46:09.828 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.259+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6805/26279110 -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0020059360 con 0x7f0020028540 2024-02-18T00:46:09.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0020059360 con 0x7f0020028540 2024-02-18T00:46:09.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 4: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00127fc700 10 client.41861.objecter _finish_command 4 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.829 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002005ef10 osd=1 4 2024-02-18T00:46:09.830 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1: {} 2024-02-18T00:46:09.830 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2: {} 2024-02-18T00:46:09.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 1 4 2024-02-18T00:46:09.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 10 client.41861.objecter _submit_command 5 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 5 2024-02-18T00:46:09.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6809/1153193637 conn(0x7f00200280a0 0x7f002011e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 5 2024-02-18T00:46:09.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 2 5 2024-02-18T00:46:09.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020059770 osd=2 4 2024-02-18T00:46:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 10 client.41861.objecter _send_command 5 2024-02-18T00:46:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.261+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6809/1153193637 -- command(tid 5: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0020121150 con 0x7f00200280a0 2024-02-18T00:46:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f0024d7d700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6809/1153193637 conn(0x7f00200280a0 0x7f002011e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f00200280a0 2024-02-18T00:46:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f0024d7d700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6809/1153193637 conn(0x7f00200280a0 0x7f002011e440 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.833 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 5: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f0020121150 con 0x7f00200280a0 2024-02-18T00:46:09.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 5: 0 ) v1 2024-02-18T00:46:09.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f00127fc700 10 client.41861.objecter _finish_command 5 = system:0 2024-02-18T00:46:09.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f0020059770 osd=2 4 2024-02-18T00:46:09.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.262+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 2 5 2024-02-18T00:46:09.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 10 client.41861.objecter _submit_command 6 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.834 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 6 2024-02-18T00:46:09.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 6 2024-02-18T00:46:09.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f0020059770 osd=2 3 2024-02-18T00:46:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 2 6 2024-02-18T00:46:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020059770 osd=2 4 2024-02-18T00:46:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 10 client.41861.objecter _send_command 6 2024-02-18T00:46:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.278+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6809/1153193637 -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0020027600 con 0x7f00200280a0 2024-02-18T00:46:09.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0020027600 con 0x7f00200280a0 2024-02-18T00:46:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 6: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00127fc700 10 client.41861.objecter _finish_command 6 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f0020059770 osd=2 4 2024-02-18T00:46:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 2 6 2024-02-18T00:46:09.837 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 10 client.41861.objecter _submit_command 7 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 7 2024-02-18T00:46:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6813/1805700964 conn(0x7f002011bf90 0x7f002011fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 7 2024-02-18T00:46:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 3 7 2024-02-18T00:46:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020027a10 osd=3 4 2024-02-18T00:46:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 10 client.41861.objecter _send_command 7 2024-02-18T00:46:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.280+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6813/1805700964 -- command(tid 7: {"prefix": "get_command_descriptions"}) v1 -- 0x7f002011ffe0 con 0x7f002011bf90 2024-02-18T00:46:09.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f0025d7f700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6813/1805700964 conn(0x7f002011bf90 0x7f002011fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f002011bf90 2024-02-18T00:46:09.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f0025d7f700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6813/1805700964 conn(0x7f002011bf90 0x7f002011fab0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 7: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f002011ffe0 con 0x7f002011bf90 2024-02-18T00:46:09.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 7: 0 ) v1 2024-02-18T00:46:09.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f00127fc700 10 client.41861.objecter _finish_command 7 = system:0 2024-02-18T00:46:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f0020027a10 osd=3 4 2024-02-18T00:46:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.281+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 3 7 2024-02-18T00:46:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 10 client.41861.objecter _submit_command 8 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 8 2024-02-18T00:46:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 8 2024-02-18T00:46:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f0020027a10 osd=3 3 2024-02-18T00:46:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 3 8 2024-02-18T00:46:09.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020027a10 osd=3 4 2024-02-18T00:46:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 10 client.41861.objecter _send_command 8 2024-02-18T00:46:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.296+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.59:6813/1805700964 -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f002011b4f0 con 0x7f002011bf90 2024-02-18T00:46:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.297+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f002011b4f0 con 0x7f002011bf90 2024-02-18T00:46:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.297+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 8: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.297+0000 7f00127fc700 10 client.41861.objecter _finish_command 8 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.297+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f0020027a10 osd=3 4 2024-02-18T00:46:09.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.297+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 3 8 2024-02-18T00:46:09.844 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3: {} 2024-02-18T00:46:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 10 client.41861.objecter _submit_command 9 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 9 2024-02-18T00:46:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0020123380 0x7f0020125770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 9 2024-02-18T00:46:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 4 9 2024-02-18T00:46:09.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b4f0 osd=4 4 2024-02-18T00:46:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 10 client.41861.objecter _send_command 9 2024-02-18T00:46:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6800/2044387056 -- command(tid 9: {"prefix": "get_command_descriptions"}) v1 -- 0x7f002011b730 con 0x7f0020123380 2024-02-18T00:46:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0020123380 0x7f0020125770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0020123380 0x7f0020125770 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.298+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f0020123380 2024-02-18T00:46:09.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.299+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 9: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f002011b730 con 0x7f0020123380 2024-02-18T00:46:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.299+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 9: 0 ) v1 2024-02-18T00:46:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.299+0000 7f00127fc700 10 client.41861.objecter _finish_command 9 = system:0 2024-02-18T00:46:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.299+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002011b4f0 osd=4 4 2024-02-18T00:46:09.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.299+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 4 9 2024-02-18T00:46:09.849 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4: {} 2024-02-18T00:46:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 10 client.41861.objecter _submit_command 10 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 10 2024-02-18T00:46:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 10 2024-02-18T00:46:09.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002011b4f0 osd=4 3 2024-02-18T00:46:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 4 10 2024-02-18T00:46:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b4f0 osd=4 4 2024-02-18T00:46:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 10 client.41861.objecter _send_command 10 2024-02-18T00:46:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.314+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6800/2044387056 -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f002011b730 con 0x7f0020123380 2024-02-18T00:46:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f002011b730 con 0x7f0020123380 2024-02-18T00:46:09.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 10: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00127fc700 10 client.41861.objecter _finish_command 10 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002011b4f0 osd=4 4 2024-02-18T00:46:09.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 4 10 2024-02-18T00:46:09.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 10 client.41861.objecter _submit_command 11 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 11 2024-02-18T00:46:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0020126170 0x7f0020128560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 11 2024-02-18T00:46:09.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 5 11 2024-02-18T00:46:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b730 osd=5 4 2024-02-18T00:46:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 10 client.41861.objecter _send_command 11 2024-02-18T00:46:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6804/2170432263 -- command(tid 11: {"prefix": "get_command_descriptions"}) v1 -- 0x7f002012b220 con 0x7f0020126170 2024-02-18T00:46:09.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.316+0000 7f0024d7d700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0020126170 0x7f0020128560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.855 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5: {} 2024-02-18T00:46:09.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.317+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f0020126170 2024-02-18T00:46:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.317+0000 7f0024d7d700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0020126170 0x7f0020128560 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.317+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 11: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f002012b220 con 0x7f0020126170 2024-02-18T00:46:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.317+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 11: 0 ) v1 2024-02-18T00:46:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.317+0000 7f00127fc700 10 client.41861.objecter _finish_command 11 = system:0 2024-02-18T00:46:09.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.317+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002011b730 osd=5 4 2024-02-18T00:46:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.317+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 5 11 2024-02-18T00:46:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 10 client.41861.objecter _submit_command 12 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 12 2024-02-18T00:46:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 12 2024-02-18T00:46:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002011b730 osd=5 3 2024-02-18T00:46:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 5 12 2024-02-18T00:46:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b730 osd=5 4 2024-02-18T00:46:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 10 client.41861.objecter _send_command 12 2024-02-18T00:46:09.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.332+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6804/2170432263 -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f002012b220 con 0x7f0020126170 2024-02-18T00:46:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f002012b220 con 0x7f0020126170 2024-02-18T00:46:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 12: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00127fc700 10 client.41861.objecter _finish_command 12 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002011b730 osd=5 4 2024-02-18T00:46:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 5 12 2024-02-18T00:46:09.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 10 client.41861.objecter _submit_command 13 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 13 2024-02-18T00:46:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0020128f60 0x7f00201293c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 13 2024-02-18T00:46:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 6 13 2024-02-18T00:46:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012b220 osd=6 4 2024-02-18T00:46:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 10 client.41861.objecter _send_command 13 2024-02-18T00:46:09.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6808/3356009390 -- command(tid 13: {"prefix": "get_command_descriptions"}) v1 -- 0x7f002012afc0 con 0x7f0020128f60 2024-02-18T00:46:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f0025d7f700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0020128f60 0x7f00201293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f0025d7f700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0020128f60 0x7f00201293c0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.334+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f0020128f60 2024-02-18T00:46:09.863 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6: {} 2024-02-18T00:46:09.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.335+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 13: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f002012afc0 con 0x7f0020128f60 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.335+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 13: 0 ) v1 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.335+0000 7f00127fc700 10 client.41861.objecter _finish_command 13 = system:0 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.335+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002012b220 osd=6 4 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.335+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 6 13 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 10 client.41861.objecter _submit_command 14 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 14 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 14 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002012b220 osd=6 3 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 6 14 2024-02-18T00:46:09.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012b220 osd=6 4 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 10 client.41861.objecter _send_command 14 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.349+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6808/3356009390 -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f002012afc0 con 0x7f0020128f60 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f002012afc0 con 0x7f0020128f60 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 14: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00127fc700 10 client.41861.objecter _finish_command 14 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002012b220 osd=6 4 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 6 14 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 10 client.41861.objecter _submit_command 15 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 15 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6812/53025171 conn(0x7f002012da20 0x7f002012de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 15 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 7 15 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012afc0 osd=7 4 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 10 client.41861.objecter _send_command 15 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.351+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6812/53025171 -- command(tid 15: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00201317c0 con 0x7f002012da20 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.352+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6812/53025171 conn(0x7f002012da20 0x7f002012de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.352+0000 7f002557e700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6812/53025171 conn(0x7f002012da20 0x7f002012de80 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.352+0000 7f00127fc700 10 client.41861.objecter ms_handle_connect 0x7f002012da20 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.353+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 15: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f00201317c0 con 0x7f002012da20 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.353+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 15: 0 ) v1 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.353+0000 7f00127fc700 10 client.41861.objecter _finish_command 15 = system:0 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.353+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002012afc0 osd=7 4 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.353+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 7 15 2024-02-18T00:46:09.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 10 client.41861.objecter _submit_command 16 [{"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}] 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign -1 16 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 20 client.41861.objecter _get_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 15 client.41861.objecter _session_command_op_remove -1 16 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 20 client.41861.objecter get_session s=0x7f002012afc0 osd=7 3 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 15 client.41861.objecter _session_command_op_assign 7 16 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012afc0 osd=7 4 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 10 client.41861.objecter _send_command 16 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.368+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 --> v2:172.21.15.137:6812/53025171 -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) v1 -- 0x7f0020129970 con 0x7f002012da20 2024-02-18T00:46:09.867 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7: {} 2024-02-18T00:46:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00127fc700 1 -- 172.21.15.59:0/3632245751 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 ==== 40+0+3 (crc 0 0 0) 0x7f0020129970 con 0x7f002012da20 2024-02-18T00:46:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00127fc700 10 client.41861.objecter ms_dispatch 0x7f0020001550 command_reply(tid 16: 0 osd_enable_op_tracker = 'false' ) v1 2024-02-18T00:46:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00127fc700 10 client.41861.objecter _finish_command 16 = system:0 osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00127fc700 20 client.41861.objecter put_session s=0x7f002012afc0 osd=7 4 2024-02-18T00:46:09.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00127fc700 15 client.41861.objecter _session_command_op_remove 7 16 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 librados: watch_flush enter 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 librados: watch_flush exit 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.0 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6801/3700238977 conn(0x7f00200fdea0 msgr2=0x7f00200fe280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6801/3700238977 conn(0x7f00200fdea0 0x7f00200fe280 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020110100 osd=0 1 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.1 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6805/26279110 conn(0x7f0020028540 msgr2=0x7f00200598f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6805/26279110 conn(0x7f0020028540 0x7f00200598f0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002005ef10 osd=1 1 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.2 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6809/1153193637 conn(0x7f00200280a0 msgr2=0x7f002011e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6809/1153193637 conn(0x7f00200280a0 0x7f002011e440 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020059770 osd=2 1 2024-02-18T00:46:09.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.3 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6813/1805700964 conn(0x7f002011bf90 msgr2=0x7f002011fab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6813/1805700964 conn(0x7f002011bf90 0x7f002011fab0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f0020027a10 osd=3 1 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.4 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0020123380 msgr2=0x7f0020125770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6800/2044387056 conn(0x7f0020123380 0x7f0020125770 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b4f0 osd=4 1 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.5 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0020126170 msgr2=0x7f0020128560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6804/2170432263 conn(0x7f0020126170 0x7f0020128560 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f0025d7f700 1 -- 172.21.15.59:0/3632245751 reap_dead start 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002011b730 osd=5 1 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.6 2024-02-18T00:46:09.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0020128f60 msgr2=0x7f00201293c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6808/3356009390 conn(0x7f0020128f60 0x7f00201293c0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f0025d7f700 1 -- 172.21.15.59:0/3632245751 reap_dead start 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f0025d7f700 1 -- 172.21.15.59:0/3632245751 reap_dead start 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012b220 osd=6 1 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter close_session for osd.7 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6812/53025171 conn(0x7f002012da20 msgr2=0x7f002012de80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6812/53025171 conn(0x7f002012da20 0x7f002012de80 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f0025d7f700 1 -- 172.21.15.59:0/3632245751 reap_dead start 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter put_session s=0x7f002012afc0 osd=7 1 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 20 client.41861.objecter shutdown clearing up homeless session... 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 10 client.41861.objecter successfully canceled tick 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6800/3721891797 conn(0x7f000c062c90 msgr2=0x7f000c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6800/3721891797 conn(0x7f000c062c90 0x7f000c065150 secure :-1 s=READY pgs=3240 cs=0 l=1 rev1=1 rx=0x7f001c009da0 tx=0x7f001c009cb0).stop 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fd510 msgr2=0x7f002010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fd510 0x7f002010a5d0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 rx=0x7f001800eb70 tx=0x7f001800b040).stop 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 shutdown_connections 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.137:6812/53025171 conn(0x7f002012da20 0x7f002012de80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00200fd510 0x7f002010a5d0 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 --2- 172.21.15.59:0/3632245751 >> v2:172.21.15.59:6800/3721891797 conn(0x7f000c062c90 0x7f000c065150 unknown :-1 s=CLOSED pgs=3240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 >> 172.21.15.59:0/3632245751 conn(0x7f00200f2820 msgr2=0x7f00200fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 shutdown_connections 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 -- 172.21.15.59:0/3632245751 wait complete. 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:08.370+0000 7f00277e2700 1 librados: shutdown 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:osd.0: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:osd.1: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:osd.2: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:osd.3: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:osd.4: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.872 INFO:teuthology.orchestra.run.smithi059.stderr:osd.5: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:osd.6: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.873 INFO:teuthology.orchestra.run.smithi059.stderr:osd.7: osd_enable_op_tracker = 'false' 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.827+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]} v 0) v1 -- 0x7fd4fc005780 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.854+0000 7fd513fff700 1 -- 172.21.15.59:0/3135176221 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]=0 set 4.16 pg_upmap_items mapping to [7->2,6->3,1->0] v801) v1 ==== 159+0+0 (secure 0 0 0) 0x7fd50c023210 con 0x7fd51c0f8cb0 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 10 librados: watch_flush enter 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 10 librados: watch_flush exit 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 20 client.41867.objecter shutdown clearing up homeless session... 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 10 client.41867.objecter successfully canceled tick 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd508062c90 msgr2=0x7fd508065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 --2- 172.21.15.59:0/3135176221 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd508062c90 0x7fd508065150 secure :-1 s=READY pgs=3241 cs=0 l=1 rev1=1 rx=0x7fd51c109b40 tx=0x7fd50400b040).stop 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 msgr2=0x7fd51c10e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c10e200 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 rx=0x7fd50c009eb0 tx=0x7fd50c00b040).stop 2024-02-18T00:46:09.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 shutdown_connections 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd51c102210 0x7fd51c1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd51c0f8cb0 0x7fd51c10e200 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 --2- 172.21.15.59:0/3135176221 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd508062c90 0x7fd508065150 unknown :-1 s=CLOSED pgs=3241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 --2- 172.21.15.59:0/3135176221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd51c0f9610 0x7fd51c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 >> 172.21.15.59:0/3135176221 conn(0x7fd51c0f2820 msgr2=0x7fd51c0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 shutdown_connections 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 -- 172.21.15.59:0/3135176221 wait complete. 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:09.857+0000 7fd521502700 1 librados: shutdown 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi059.stderr:set 4.16 pg_upmap_items mapping to [7->2,6->3,1->0] 2024-02-18T00:46:09.875 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:46:09.876 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:46:10.086 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:46:10.088 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:10.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:46:10.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0 2024-02-18T00:46:10.220 INFO:teuthology.orchestra.run.smithi059.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:10.475 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:09.614686+0000 osd.1 (osd.1) 375 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:46:10.475 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:09.615982+0000 osd.1 (osd.1) 376 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:46:10.475 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:09.828775+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.59:0/3135176221' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]: dispatch 2024-02-18T00:46:10.476 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:09.829098+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]: dispatch 2024-02-18T00:46:10.476 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:09.853773+0000 mon.a (mon.0) 3743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]': finished 2024-02-18T00:46:10.476 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:09.853838+0000 mon.a (mon.0) 3744 : cluster [DBG] osdmap e801: 8 total, 8 up, 7 in 2024-02-18T00:46:10.476 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:09.857465+0000 mon.a (mon.0) 3745 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:10.477 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_blocked_ops 2024-02-18T00:46:10.666 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:09.614686+0000 osd.1 (osd.1) 375 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:46:10.666 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:09.615982+0000 osd.1 (osd.1) 376 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:46:10.666 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:09.828775+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.59:0/3135176221' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]: dispatch 2024-02-18T00:46:10.666 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:09.829098+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]: dispatch 2024-02-18T00:46:10.666 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:09.853773+0000 mon.a (mon.0) 3743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]': finished 2024-02-18T00:46:10.666 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:09.853838+0000 mon.a (mon.0) 3744 : cluster [DBG] osdmap e801: 8 total, 8 up, 7 in 2024-02-18T00:46:10.666 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:09.857465+0000 mon.a (mon.0) 3745 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:10.667 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:09.614686+0000 osd.1 (osd.1) 375 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:46:10.667 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:09.615982+0000 osd.1 (osd.1) 376 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:46:10.667 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:09.828775+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.59:0/3135176221' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]: dispatch 2024-02-18T00:46:10.667 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:09.829098+0000 mon.a (mon.0) 3742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]: dispatch 2024-02-18T00:46:10.667 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:09.853773+0000 mon.a (mon.0) 3743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.16", "id": [7, 2, 6, 3, 1, 0]}]': finished 2024-02-18T00:46:10.667 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:09.853838+0000 mon.a (mon.0) 3744 : cluster [DBG] osdmap e801: 8 total, 8 up, 7 in 2024-02-18T00:46:10.667 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:09.857465+0000 mon.a (mon.0) 3745 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:10.745 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:46:10.970 INFO:teuthology.orchestra.run.smithi059.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:46:10.973 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:10.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi059 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0' 2024-02-18T00:46:11.075 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7 2024-02-18T00:46:11.102 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:11.200 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:46:11.202 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:11.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7' 2024-02-18T00:46:11.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:46:11.304 DEBUG:teuthology.orchestra.run.smithi137:> sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6 2024-02-18T00:46:11.331 INFO:teuthology.orchestra.run.smithi137.stderr:Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. 2024-02-18T00:46:11.602 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:10.032257+0000 mgr.y (mgr.24454) 688 : cluster [DBG] pgmap v1234: 174 pgs: 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/678 objects misplaced (0.147%) 2024-02-18T00:46:11.602 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:10.848043+0000 mon.a (mon.0) 3746 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:46:11.602 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:10.857613+0000 mon.a (mon.0) 3747 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:46:11.602 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:10.857707+0000 mon.a (mon.0) 3748 : cluster [DBG] osdmap e802: 8 total, 8 up, 7 in 2024-02-18T00:46:11.602 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:10.860379+0000 mon.a (mon.0) 3749 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:11.602 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:11.216332+0000 osd.3 (osd.3) 408 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:46:11.602 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:11.217437+0000 osd.3 (osd.3) 409 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:46:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:10.032257+0000 mgr.y (mgr.24454) 688 : cluster [DBG] pgmap v1234: 174 pgs: 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/678 objects misplaced (0.147%) 2024-02-18T00:46:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:10.848043+0000 mon.a (mon.0) 3746 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:46:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:10.857613+0000 mon.a (mon.0) 3747 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:46:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:10.857707+0000 mon.a (mon.0) 3748 : cluster [DBG] osdmap e802: 8 total, 8 up, 7 in 2024-02-18T00:46:11.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:10.860379+0000 mon.a (mon.0) 3749 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:11.216332+0000 osd.3 (osd.3) 408 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:46:11.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:11.217437+0000 osd.3 (osd.3) 409 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:46:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:10.032257+0000 mgr.y (mgr.24454) 688 : cluster [DBG] pgmap v1234: 174 pgs: 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1/678 objects misplaced (0.147%) 2024-02-18T00:46:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:10.848043+0000 mon.a (mon.0) 3746 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T00:46:11.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:10.857613+0000 mon.a (mon.0) 3747 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:46:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:10.857707+0000 mon.a (mon.0) 3748 : cluster [DBG] osdmap e802: 8 total, 8 up, 7 in 2024-02-18T00:46:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:10.860379+0000 mon.a (mon.0) 3749 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]: dispatch 2024-02-18T00:46:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:11.216332+0000 osd.3 (osd.3) 408 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:46:11.754 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:11.217437+0000 osd.3 (osd.3) 409 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:46:11.865 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (8052 ms) 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: first is 1708217088 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,1708217093,1708217094,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,1708217093,1708217094,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,1708217093,1708217094,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,1708217093,1708217094,1708217096,1708217097,0 2024-02-18T00:46:11.866 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,1708217093,1708217094,1708217096,1708217097,0 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217088,1708217090,1708217091,1708217093,1708217094,1708217096,1708217097,0 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: got ls 1708217091,1708217093,1708217094,1708217096,1708217097,1708217099,1708217100,0 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: first now 1708217091, trimmed 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20145 ms) 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: foo0 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (13845 ms) 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17194 ms) 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22725 ms) 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-18T00:46:11.867 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5035 ms) 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3012 ms) 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (2951 ms) 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3013 ms) 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (230098 ms total) 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [==========] 72 tests from 4 test suites ran. (800931 ms total) 2024-02-18T00:46:11.868 INFO:tasks.workunit.client.0.smithi059.stdout: api_tier_pp: [ PASSED ] 72 tests. 2024-02-18T00:46:11.869 INFO:tasks.workunit.client.0.smithi059.stderr:+ exit 0 2024-02-18T00:46:11.869 INFO:tasks.workunit.client.0.smithi059.stderr:+ cleanup 2024-02-18T00:46:11.869 INFO:tasks.workunit.client.0.smithi059.stderr:+ pkill -P 82867 2024-02-18T00:46:11.909 INFO:tasks.workunit.client.0.smithi059.stderr:+ true 2024-02-18T00:46:11.910 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-18T00:46:11.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-18T00:46:11.940 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2024-02-18T00:46:11.940 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-18T00:46:12.000 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:46:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:46:12.251 DEBUG:teuthology.parallel:result is None 2024-02-18T00:46:12.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T00:46:12.280 INFO:teuthology.orchestra.run.smithi137.stderr:Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:46:12.281 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T00:46:12.282 DEBUG:teuthology.orchestra.run.smithi059:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-18T00:46:12.284 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-02-18T00:46:12.284 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi137 with status 125: 'sudo docker kill -s 1 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6' 2024-02-18T00:46:12.368 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T00:46:12.371 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-02-18T00:46:12.384 INFO:tasks.thrashosds:joining thrashosds 2024-02-18T00:46:12.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:12] "GET /metrics HTTP/1.1" 200 187497 "" "Prometheus/2.33.4" 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:10.552674+0000 osd.5 (osd.5) 352 : cluster [DBG] 6.a scrub starts 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:10.553567+0000 osd.5 (osd.5) 353 : cluster [DBG] 6.a scrub ok 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:11.494617+0000 mon.a (mon.0) 3750 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:11.506255+0000 osd.5 (osd.5) 354 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:11.510142+0000 osd.5 (osd.5) 355 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:11.597533+0000 osd.1 (osd.1) 377 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:11.599412+0000 osd.1 (osd.1) 378 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:46:12.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:11.858233+0000 mon.a (mon.0) 3751 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:11.858301+0000 mon.a (mon.0) 3752 : cluster [DBG] osdmap e803: 8 total, 8 up, 7 in 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:12.070955+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:12.353148+0000 mon.a (mon.0) 3754 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:12.353823+0000 mon.a (mon.0) 3755 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:12.357954+0000 mon.a (mon.0) 3756 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:10.552674+0000 osd.5 (osd.5) 352 : cluster [DBG] 6.a scrub starts 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:10.553567+0000 osd.5 (osd.5) 353 : cluster [DBG] 6.a scrub ok 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:11.494617+0000 mon.a (mon.0) 3750 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:11.506255+0000 osd.5 (osd.5) 354 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:11.510142+0000 osd.5 (osd.5) 355 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:46:12.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:11.597533+0000 osd.1 (osd.1) 377 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:46:12.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:11.599412+0000 osd.1 (osd.1) 378 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:46:12.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:11.858233+0000 mon.a (mon.0) 3751 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:46:12.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:11.858301+0000 mon.a (mon.0) 3752 : cluster [DBG] osdmap e803: 8 total, 8 up, 7 in 2024-02-18T00:46:12.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:12.070955+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:12.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:12.353148+0000 mon.a (mon.0) 3754 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:46:12.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:12.353823+0000 mon.a (mon.0) 3755 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:46:12.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:12.357954+0000 mon.a (mon.0) 3756 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:12.523 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:46:12.523 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:10.552674+0000 osd.5 (osd.5) 352 : cluster [DBG] 6.a scrub starts 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:10.553567+0000 osd.5 (osd.5) 353 : cluster [DBG] 6.a scrub ok 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:11.494617+0000 mon.a (mon.0) 3750 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:11.506255+0000 osd.5 (osd.5) 354 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:11.510142+0000 osd.5 (osd.5) 355 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:11.597533+0000 osd.1 (osd.1) 377 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:11.599412+0000 osd.1 (osd.1) 378 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:11.858233+0000 mon.a (mon.0) 3751 : audit [INF] from='client.? 172.21.15.59:0/337723220' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi059-82957-101"}]': finished 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:11.858301+0000 mon.a (mon.0) 3752 : cluster [DBG] osdmap e803: 8 total, 8 up, 7 in 2024-02-18T00:46:12.542 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:12.070955+0000 mon.a (mon.0) 3753 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:12.543 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:12.353148+0000 mon.a (mon.0) 3754 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:46:12.543 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:12.353823+0000 mon.a (mon.0) 3755 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:46:12.543 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:12.357954+0000 mon.a (mon.0) 3756 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 -- 172.21.15.59:0/3381569159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 msgr2=0x7fd7b80fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 --2- 172.21.15.59:0/3381569159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b80fdf90 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 rx=0x7fd7b40047b0 tx=0x7fd7b4009cb0).stop 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 -- 172.21.15.59:0/3381569159 shutdown_connections 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 --2- 172.21.15.59:0/3381569159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7b80f8420 0x7fd7b8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 --2- 172.21.15.59:0/3381569159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b80fdf90 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 --2- 172.21.15.59:0/3381569159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b80fe510 0x7fd7b80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 -- 172.21.15.59:0/3381569159 >> 172.21.15.59:0/3381569159 conn(0x7fd7b80f3800 msgr2=0x7fd7b80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.410+0000 7fd7c0529700 1 -- 172.21.15.59:0/3381569159 shutdown_connections 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 -- 172.21.15.59:0/3381569159 wait complete. 2024-02-18T00:46:13.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 librados: starting msgr at 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 librados: starting objecter 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 Processor -- start 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 -- start start 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 librados: setting wanted keys 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 librados: calling monclient init 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7b80f8420 0x7fd7b810e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b810e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b80fe510 0x7fd7b8108270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8108760 con 0x7fd7b80fe510 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81088d0 con 0x7fd7b80f8420 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7c0529700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8108a90 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7bdac4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b810e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:13.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7beac6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b80fe510 0x7fd7b8108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7beac6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b80fe510 0x7fd7b8108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36956/0 (socket says 172.21.15.59:36956) 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.411+0000 7fd7bdac4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b810e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49258/0 (socket says 172.21.15.59:49258) 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7beac6700 1 -- 172.21.15.59:0/1354954888 learned_addr learned my addr 172.21.15.59:0/1354954888 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7be2c5700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7b80f8420 0x7fd7b810e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7bdac4700 1 -- 172.21.15.59:0/1354954888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7b80f8420 msgr2=0x7fd7b810e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7bdac4700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7b80f8420 0x7fd7b810e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7bdac4700 1 -- 172.21.15.59:0/1354954888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b80fe510 msgr2=0x7fd7b8108270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7bdac4700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b80fe510 0x7fd7b8108270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7bdac4700 1 -- 172.21.15.59:0/1354954888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac005040 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7bdac4700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b810e5c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7fd7ac0091f0 tx=0x7fd7ac009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7ab7fe700 10 client.?.objecter ms_handle_connect 0x7fd7b80fdbb0 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7ab7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:13.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7ab7fe700 1 -- 172.21.15.59:0/1354954888 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00e030 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b4005040 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7ab7fe700 1 -- 172.21.15.59:0/1354954888 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7ac00f040 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7ab7fe700 1 -- 172.21.15.59:0/1354954888 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0147e0 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7c0529700 10 client.41873.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:13.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b81090d0 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.412+0000 7fd7c0529700 1 librados: init done 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.413+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b805ee80 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 1 -- 172.21.15.59:0/1354954888 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd7ac0104f0 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 1 --2- 172.21.15.59:0/1354954888 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd7a4062c90 0x7fd7a4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 1 -- 172.21.15.59:0/1354954888 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(803..803 src has 418..803) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fd7ac07d070 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 10 client.41873.objecter ms_dispatch 0x7fd7b8001550 osd_map(803..803 src has 418..803) v4 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 3 client.41873.objecter handle_osd_map got epochs [803,803] > 0 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 3 client.41873.objecter handle_osd_map decoding full epoch 803 2024-02-18T00:46:13.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7be2c5700 1 --2- 172.21.15.59:0/1354954888 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd7a4062c90 0x7fd7a4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:13.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 20 client.41873.objecter dump_active .. 0 homeless 2024-02-18T00:46:13.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7ab7fe700 10 client.41873.objecter ms_handle_connect 0x7fd7a4062c90 2024-02-18T00:46:13.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.414+0000 7fd7be2c5700 1 --2- 172.21.15.59:0/1354954888 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd7a4062c90 0x7fd7a4065150 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 rx=0x7fd7b4005010 tx=0x7fd7b4007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:13.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.418+0000 7fd7ab7fe700 1 -- 172.21.15.59:0/1354954888 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd7ac050740 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:11.651835+0000 mgr.y (mgr.24454) 689 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:13.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:12.032593+0000 mgr.y (mgr.24454) 690 : cluster [DBG] pgmap v1237: 174 pgs: 1 peering, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:12.080503+0000 osd.4 (osd.4) 417 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:12.083791+0000 osd.4 (osd.4) 418 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:12.855113+0000 mon.a (mon.0) 3757 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:11.651835+0000 mgr.y (mgr.24454) 689 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:12.032593+0000 mgr.y (mgr.24454) 690 : cluster [DBG] pgmap v1237: 174 pgs: 1 peering, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:12.080503+0000 osd.4 (osd.4) 417 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:12.083791+0000 osd.4 (osd.4) 418 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:46:13.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:12.855113+0000 mon.a (mon.0) 3757 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-18T00:46:13.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:13.602+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fd7b805fa60 con 0x7fd7b80fdbb0 2024-02-18T00:46:13.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:11.651835+0000 mgr.y (mgr.24454) 689 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:13.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:12.032593+0000 mgr.y (mgr.24454) 690 : cluster [DBG] pgmap v1237: 174 pgs: 1 peering, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:46:13.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:12.080503+0000 osd.4 (osd.4) 417 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:46:13.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:12.083791+0000 osd.4 (osd.4) 418 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:46:13.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:12.855113+0000 mon.a (mon.0) 3757 : cluster [INF] Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-02-18T00:46:13.739 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph daemon osd.7 dump_historic_ops 2024-02-18T00:46:14.019 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.b/config 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.376+0000 7fd7ab7fe700 1 -- 172.21.15.59:0/1354954888 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v804) v1 ==== 101+0+0 (secure 0 0 0) 0x7fd7ac050330 con 0x7fd7b80fdbb0 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.378+0000 7fd7c0529700 10 librados: watch_flush enter 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.378+0000 7fd7c0529700 10 librados: watch_flush exit 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.378+0000 7fd7c0529700 20 client.41873.objecter shutdown clearing up homeless session... 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.378+0000 7fd7c0529700 10 client.41873.objecter successfully canceled tick 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd7a4062c90 msgr2=0x7fd7a4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 --2- 172.21.15.59:0/1354954888 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd7a4062c90 0x7fd7a4065150 secure :-1 s=READY pgs=3242 cs=0 l=1 rev1=1 rx=0x7fd7b4005010 tx=0x7fd7b4007f90).stop 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 msgr2=0x7fd7b810e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b810e5c0 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 rx=0x7fd7ac0091f0 tx=0x7fd7ac009cb0).stop 2024-02-18T00:46:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 shutdown_connections 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 --2- 172.21.15.59:0/1354954888 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd7a4062c90 0x7fd7a4065150 unknown :-1 s=CLOSED pgs=3242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7b80f8420 0x7fd7b810e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b80fe510 0x7fd7b8108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 --2- 172.21.15.59:0/1354954888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b80fdbb0 0x7fd7b810e5c0 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 >> 172.21.15.59:0/1354954888 conn(0x7fd7b80f3800 msgr2=0x7fd7b80f5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 shutdown_connections 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 -- 172.21.15.59:0/1354954888 wait complete. 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:14.379+0000 7fd7c0529700 1 librados: shutdown 2024-02-18T00:46:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:46:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:13.042467+0000 osd.7 (osd.7) 243 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:13.043587+0000 osd.7 (osd.7) 244 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:13.603570+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.59:0/1354954888' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:13.603858+0000 mon.a (mon.0) 3758 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:13.619112+0000 osd.1 (osd.1) 379 : cluster [DBG] 2.d scrub starts 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:13.619976+0000 osd.1 (osd.1) 380 : cluster [DBG] 2.d scrub ok 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:13.823477+0000 osd.2 (osd.2) 315 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:13.824342+0000 osd.2 (osd.2) 316 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.183331+0000 osd.3 (osd.3) 410 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:46:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.184256+0000 osd.3 (osd.3) 411 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:46:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:13.042467+0000 osd.7 (osd.7) 243 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:46:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:13.043587+0000 osd.7 (osd.7) 244 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:46:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:13.603570+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.59:0/1354954888' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:13.603858+0000 mon.a (mon.0) 3758 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:14.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:13.619112+0000 osd.1 (osd.1) 379 : cluster [DBG] 2.d scrub starts 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:13.619976+0000 osd.1 (osd.1) 380 : cluster [DBG] 2.d scrub ok 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:13.823477+0000 osd.2 (osd.2) 315 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:13.824342+0000 osd.2 (osd.2) 316 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.183331+0000 osd.3 (osd.3) 410 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.184256+0000 osd.3 (osd.3) 411 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:13.042467+0000 osd.7 (osd.7) 243 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:13.043587+0000 osd.7 (osd.7) 244 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:13.603570+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.59:0/1354954888' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:13.603858+0000 mon.a (mon.0) 3758 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:13.619112+0000 osd.1 (osd.1) 379 : cluster [DBG] 2.d scrub starts 2024-02-18T00:46:14.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:13.619976+0000 osd.1 (osd.1) 380 : cluster [DBG] 2.d scrub ok 2024-02-18T00:46:14.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:13.823477+0000 osd.2 (osd.2) 315 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:46:14.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:13.824342+0000 osd.2 (osd.2) 316 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:46:14.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.183331+0000 osd.3 (osd.3) 410 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:46:14.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.184256+0000 osd.3 (osd.3) 411 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:46:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.033039+0000 mgr.y (mgr.24454) 691 : cluster [DBG] pgmap v1238: 174 pgs: 1 peering, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:46:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.114667+0000 osd.4 (osd.4) 419 : cluster [DBG] 4.d scrub starts 2024-02-18T00:46:15.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.118134+0000 osd.4 (osd.4) 420 : cluster [DBG] 4.d scrub ok 2024-02-18T00:46:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.126709+0000 osd.0 (osd.0) 346 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:46:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.127735+0000 osd.0 (osd.0) 347 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:46:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.369027+0000 mon.a (mon.0) 3759 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:46:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:14.377371+0000 mon.a (mon.0) 3760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:46:15.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:14.377471+0000 mon.a (mon.0) 3761 : cluster [DBG] osdmap e804: 8 total, 8 up, 7 in 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.033039+0000 mgr.y (mgr.24454) 691 : cluster [DBG] pgmap v1238: 174 pgs: 1 peering, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.114667+0000 osd.4 (osd.4) 419 : cluster [DBG] 4.d scrub starts 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.118134+0000 osd.4 (osd.4) 420 : cluster [DBG] 4.d scrub ok 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.126709+0000 osd.0 (osd.0) 346 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.127735+0000 osd.0 (osd.0) 347 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.369027+0000 mon.a (mon.0) 3759 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:14.377371+0000 mon.a (mon.0) 3760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:14.377471+0000 mon.a (mon.0) 3761 : cluster [DBG] osdmap e804: 8 total, 8 up, 7 in 2024-02-18T00:46:15.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.033039+0000 mgr.y (mgr.24454) 691 : cluster [DBG] pgmap v1238: 174 pgs: 1 peering, 173 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T00:46:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.114667+0000 osd.4 (osd.4) 419 : cluster [DBG] 4.d scrub starts 2024-02-18T00:46:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.118134+0000 osd.4 (osd.4) 420 : cluster [DBG] 4.d scrub ok 2024-02-18T00:46:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.126709+0000 osd.0 (osd.0) 346 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:46:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.127735+0000 osd.0 (osd.0) 347 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:46:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.369027+0000 mon.a (mon.0) 3759 : cluster [INF] Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-02-18T00:46:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:14.377371+0000 mon.a (mon.0) 3760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:46:15.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:14.377471+0000 mon.a (mon.0) 3761 : cluster [DBG] osdmap e804: 8 total, 8 up, 7 in 2024-02-18T00:46:15.815 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-02-18T00:46:15.816 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-02-18T00:46:16.488 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:15.590093+0000 osd.1 (osd.1) 381 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:46:16.488 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:15.593583+0000 osd.1 (osd.1) 382 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:46:16.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:15.590093+0000 osd.1 (osd.1) 381 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:46:16.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:15.593583+0000 osd.1 (osd.1) 382 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:46:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:15.590093+0000 osd.1 (osd.1) 381 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:46:16.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:15.593583+0000 osd.1 (osd.1) 382 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:46:17.567 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset noscrub 2024-02-18T00:46:17.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:16.033874+0000 mgr.y (mgr.24454) 692 : cluster [DBG] pgmap v1240: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 79 B/s, 1 objects/s recovering 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:16.071780+0000 osd.7 (osd.7) 245 : cluster [DBG] 25.3 deep-scrub starts 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:16.073555+0000 osd.7 (osd.7) 246 : cluster [DBG] 25.3 deep-scrub ok 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:16.184646+0000 osd.0 (osd.0) 348 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:16.188103+0000 osd.0 (osd.0) 349 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:16.527910+0000 osd.5 (osd.5) 356 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.153086+0000 osd.3 (osd.3) 412 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.154054+0000 osd.3 (osd.3) 413 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:16.033874+0000 mgr.y (mgr.24454) 692 : cluster [DBG] pgmap v1240: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 79 B/s, 1 objects/s recovering 2024-02-18T00:46:17.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:16.071780+0000 osd.7 (osd.7) 245 : cluster [DBG] 25.3 deep-scrub starts 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:16.073555+0000 osd.7 (osd.7) 246 : cluster [DBG] 25.3 deep-scrub ok 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:16.184646+0000 osd.0 (osd.0) 348 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:16.188103+0000 osd.0 (osd.0) 349 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:16.527910+0000 osd.5 (osd.5) 356 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.153086+0000 osd.3 (osd.3) 412 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.154054+0000 osd.3 (osd.3) 413 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:16.033874+0000 mgr.y (mgr.24454) 692 : cluster [DBG] pgmap v1240: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.7 KiB/s rd, 1 op/s; 79 B/s, 1 objects/s recovering 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:16.071780+0000 osd.7 (osd.7) 245 : cluster [DBG] 25.3 deep-scrub starts 2024-02-18T00:46:17.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:16.073555+0000 osd.7 (osd.7) 246 : cluster [DBG] 25.3 deep-scrub ok 2024-02-18T00:46:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:16.184646+0000 osd.0 (osd.0) 348 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:46:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:16.188103+0000 osd.0 (osd.0) 349 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:46:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:16.527910+0000 osd.5 (osd.5) 356 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:46:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.153086+0000 osd.3 (osd.3) 412 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:46:17.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.154054+0000 osd.3 (osd.3) 413 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:46:17.863 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:18.239 INFO:tasks.thrashosds.thrasher:inning osd 2024-02-18T00:46:18.239 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-02-18T00:46:18.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd in 6 2024-02-18T00:46:18.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:16.531433+0000 osd.5 (osd.5) 357 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.067187+0000 osd.7 (osd.7) 247 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.068589+0000 osd.7 (osd.7) 248 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.195969+0000 osd.0 (osd.0) 350 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.199422+0000 osd.0 (osd.0) 351 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.513163+0000 osd.5 (osd.5) 358 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.516414+0000 osd.5 (osd.5) 359 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.771058+0000 osd.2 (osd.2) 317 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:17.772847+0000 osd.2 (osd.2) 318 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:16.531433+0000 osd.5 (osd.5) 357 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.067187+0000 osd.7 (osd.7) 247 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.068589+0000 osd.7 (osd.7) 248 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.195969+0000 osd.0 (osd.0) 350 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:46:18.446 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.199422+0000 osd.0 (osd.0) 351 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:46:18.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.513163+0000 osd.5 (osd.5) 358 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:46:18.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.516414+0000 osd.5 (osd.5) 359 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:46:18.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.771058+0000 osd.2 (osd.2) 317 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:46:18.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:17.772847+0000 osd.2 (osd.2) 318 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:46:18.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:16.531433+0000 osd.5 (osd.5) 357 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.067187+0000 osd.7 (osd.7) 247 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.068589+0000 osd.7 (osd.7) 248 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.195969+0000 osd.0 (osd.0) 350 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.199422+0000 osd.0 (osd.0) 351 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.513163+0000 osd.5 (osd.5) 358 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.516414+0000 osd.5 (osd.5) 359 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.771058+0000 osd.2 (osd.2) 317 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:46:18.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:17.772847+0000 osd.2 (osd.2) 318 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:46:19.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.028466+0000 osd.7 (osd.7) 249 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:46:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.029399+0000 osd.7 (osd.7) 250 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:46:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.034351+0000 mgr.y (mgr.24454) 693 : cluster [DBG] pgmap v1241: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 713 B/s rd, 0 op/s; 15 B/s, 0 objects/s recovering 2024-02-18T00:46:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.773084+0000 osd.2 (osd.2) 319 : cluster [DBG] 5.d scrub starts 2024-02-18T00:46:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.774816+0000 osd.2 (osd.2) 320 : cluster [DBG] 5.d scrub ok 2024-02-18T00:46:19.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.837666+0000 mon.a (mon.0) 3762 : cluster [DBG] osdmap e805: 8 total, 8 up, 7 in 2024-02-18T00:46:19.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.028466+0000 osd.7 (osd.7) 249 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.029399+0000 osd.7 (osd.7) 250 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.034351+0000 mgr.y (mgr.24454) 693 : cluster [DBG] pgmap v1241: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 713 B/s rd, 0 op/s; 15 B/s, 0 objects/s recovering 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.773084+0000 osd.2 (osd.2) 319 : cluster [DBG] 5.d scrub starts 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.774816+0000 osd.2 (osd.2) 320 : cluster [DBG] 5.d scrub ok 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.837666+0000 mon.a (mon.0) 3762 : cluster [DBG] osdmap e805: 8 total, 8 up, 7 in 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.028466+0000 osd.7 (osd.7) 249 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.029399+0000 osd.7 (osd.7) 250 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:46:19.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.034351+0000 mgr.y (mgr.24454) 693 : cluster [DBG] pgmap v1241: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 713 B/s rd, 0 op/s; 15 B/s, 0 objects/s recovering 2024-02-18T00:46:19.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.773084+0000 osd.2 (osd.2) 319 : cluster [DBG] 5.d scrub starts 2024-02-18T00:46:19.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.774816+0000 osd.2 (osd.2) 320 : cluster [DBG] 5.d scrub ok 2024-02-18T00:46:19.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.837666+0000 mon.a (mon.0) 3762 : cluster [DBG] osdmap e805: 8 total, 8 up, 7 in 2024-02-18T00:46:20.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 -- 172.21.15.59:0/2250930473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 msgr2=0x7fa104104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:20.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/2250930473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa104104a20 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 rx=0x7fa0fc009be0 tx=0x7fa0fc004d10).stop 2024-02-18T00:46:20.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 -- 172.21.15.59:0/2250930473 shutdown_connections 2024-02-18T00:46:20.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/2250930473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa104104a20 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/2250930473 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1040fdbb0 0x7fa1040fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/2250930473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1040fe510 0x7fa1040f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 -- 172.21.15.59:0/2250930473 >> 172.21.15.59:0/2250930473 conn(0x7fa1040f3800 msgr2=0x7fa1040f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 -- 172.21.15.59:0/2250930473 shutdown_connections 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 -- 172.21.15.59:0/2250930473 wait complete. 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 librados: starting msgr at 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.069+0000 7fa10b7ef700 1 librados: starting objecter 2024-02-18T00:46:20.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 Processor -- start 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 -- start start 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 librados: setting wanted keys 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 librados: calling monclient init 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa10406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1040fdbb0 0x7fa10406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1040fe510 0x7fa10406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104116300 con 0x7fa1040f8420 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104116470 con 0x7fa1040fdbb0 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10b7ef700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa104116630 con 0x7fa1040fe510 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa10406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa10406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56518/0 (socket says 172.21.15.59:56518) 2024-02-18T00:46:20.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 -- 172.21.15.59:0/49322503 learned_addr learned my addr 172.21.15.59:0/49322503 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa109d8c700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1040fe510 0x7fa10406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa108d8a700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1040fdbb0 0x7fa10406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 -- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1040fe510 msgr2=0x7fa10406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1040fe510 0x7fa10406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 -- 172.21.15.59:0/49322503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1040fdbb0 msgr2=0x7fa10406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1040fdbb0 0x7fa10406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.070+0000 7fa10958b700 1 -- 172.21.15.59:0/49322503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4005040 con 0x7fa1040f8420 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa10958b700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa10406b0d0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 rx=0x7fa0f40048f0 tx=0x7fa0f4009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa0fa7fc700 10 client.?.objecter ms_handle_connect 0x7fa1040f8420 2024-02-18T00:46:20.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa0fa7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa0fa7fc700 1 -- 172.21.15.59:0/49322503 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4007b00 con 0x7fa1040f8420 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc0097c0 con 0x7fa1040f8420 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa0fa7fc700 1 -- 172.21.15.59:0/49322503 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0f400f040 con 0x7fa1040f8420 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa0fa7fc700 1 -- 172.21.15.59:0/49322503 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4014990 con 0x7fa1040f8420 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa10b7ef700 10 client.32076.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa104116cf0 con 0x7fa1040f8420 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.071+0000 7fa10b7ef700 1 librados: init done 2024-02-18T00:46:20.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.072+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10405ee80 con 0x7fa1040f8420 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.072+0000 7fa0fa7fc700 1 -- 172.21.15.59:0/49322503 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa0f4010500 con 0x7fa1040f8420 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa0fa7fc700 1 --2- 172.21.15.59:0/49322503 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0f0062c90 0x7fa0f0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa108d8a700 1 --2- 172.21.15.59:0/49322503 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0f0062c90 0x7fa0f0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa0fa7fc700 1 -- 172.21.15.59:0/49322503 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(805..805 src has 418..805) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fa0f407cb20 con 0x7fa1040f8420 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa0fa7fc700 10 client.32076.objecter ms_dispatch 0x7fa104001550 osd_map(805..805 src has 418..805) v4 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa0fa7fc700 3 client.32076.objecter handle_osd_map got epochs [805,805] > 0 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa0fa7fc700 3 client.32076.objecter handle_osd_map decoding full epoch 805 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa0fa7fc700 20 client.32076.objecter dump_active .. 0 homeless 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa0fa7fc700 10 client.32076.objecter ms_handle_connect 0x7fa0f0062c90 2024-02-18T00:46:20.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.073+0000 7fa108d8a700 1 --2- 172.21.15.59:0/49322503 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0f0062c90 0x7fa0f0065150 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 rx=0x7fa10406c870 tx=0x7fa100009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:20.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.077+0000 7fa0fa7fc700 1 -- 172.21.15.59:0/49322503 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa0f4049f60 con 0x7fa1040f8420 2024-02-18T00:46:20.263 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:20.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.267+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) v1 -- 0x7fa104107110 con 0x7fa1040f8420 2024-02-18T00:46:20.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.395+0000 7fa0fa7fc700 1 -- 172.21.15.59:0/49322503 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v806) v1 ==== 91+0+0 (secure 0 0 0) 0x7fa0f40501f0 con 0x7fa1040f8420 2024-02-18T00:46:20.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 10 librados: watch_flush enter 2024-02-18T00:46:20.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 10 librados: watch_flush exit 2024-02-18T00:46:20.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 20 client.32076.objecter shutdown clearing up homeless session... 2024-02-18T00:46:20.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 10 client.32076.objecter successfully canceled tick 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0f0062c90 msgr2=0x7fa0f0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/49322503 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0f0062c90 0x7fa0f0065150 secure :-1 s=READY pgs=3243 cs=0 l=1 rev1=1 rx=0x7fa10406c870 tx=0x7fa100009040).stop 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 msgr2=0x7fa10406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa10406b0d0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 rx=0x7fa0f40048f0 tx=0x7fa0f4009cb0).stop 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 shutdown_connections 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1040fe510 0x7fa10406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1040f8420 0x7fa10406b0d0 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/49322503 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa0f0062c90 0x7fa0f0065150 unknown :-1 s=CLOSED pgs=3243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 --2- 172.21.15.59:0/49322503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1040fdbb0 0x7fa10406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.397+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 >> 172.21.15.59:0/49322503 conn(0x7fa1040f3800 msgr2=0x7fa1040f4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.398+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 shutdown_connections 2024-02-18T00:46:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.398+0000 7fa10b7ef700 1 -- 172.21.15.59:0/49322503 wait complete. 2024-02-18T00:46:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:20.398+0000 7fa10b7ef700 1 librados: shutdown 2024-02-18T00:46:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:noscrub is unset 2024-02-18T00:46:20.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.560311+0000 osd.5 (osd.5) 360 : cluster [DBG] 25.f scrub starts 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:18.564288+0000 osd.5 (osd.5) 361 : cluster [DBG] 25.f scrub ok 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:19.646717+0000 osd.1 (osd.1) 383 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:19.647788+0000 osd.1 (osd.1) 384 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:20.269059+0000 mon.a (mon.0) 3763 : audit [INF] from='client.? 172.21.15.59:0/49322503' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.344374+0000 osd.3 (osd.3) 414 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.344622+0000 osd.3 (osd.3) 415 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.560311+0000 osd.5 (osd.5) 360 : cluster [DBG] 25.f scrub starts 2024-02-18T00:46:20.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:18.564288+0000 osd.5 (osd.5) 361 : cluster [DBG] 25.f scrub ok 2024-02-18T00:46:20.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:19.646717+0000 osd.1 (osd.1) 383 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:46:20.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:19.647788+0000 osd.1 (osd.1) 384 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:46:20.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:20.269059+0000 mon.a (mon.0) 3763 : audit [INF] from='client.? 172.21.15.59:0/49322503' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:46:20.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.344374+0000 osd.3 (osd.3) 414 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:20.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.344622+0000 osd.3 (osd.3) 415 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:20.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.560311+0000 osd.5 (osd.5) 360 : cluster [DBG] 25.f scrub starts 2024-02-18T00:46:20.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:18.564288+0000 osd.5 (osd.5) 361 : cluster [DBG] 25.f scrub ok 2024-02-18T00:46:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:19.646717+0000 osd.1 (osd.1) 383 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:46:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:19.647788+0000 osd.1 (osd.1) 384 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:46:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:20.269059+0000 mon.a (mon.0) 3763 : audit [INF] from='client.? 172.21.15.59:0/49322503' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-02-18T00:46:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.344374+0000 osd.3 (osd.3) 414 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:20.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.344622+0000 osd.3 (osd.3) 415 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:21.397 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:46:21.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.034722+0000 mgr.y (mgr.24454) 694 : cluster [DBG] pgmap v1243: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 13 B/s, 0 objects/s recovering 2024-02-18T00:46:21.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.081082+0000 osd.7 (osd.7) 251 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:46:21.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.082047+0000 osd.7 (osd.7) 252 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:46:21.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.083792+0000 osd.4 (osd.4) 421 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:46:21.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.087264+0000 osd.4 (osd.4) 422 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:46:21.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:20.395841+0000 mon.a (mon.0) 3764 : audit [INF] from='client.? 172.21.15.59:0/49322503' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:46:21.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.395904+0000 mon.a (mon.0) 3765 : cluster [DBG] osdmap e806: 8 total, 8 up, 7 in 2024-02-18T00:46:21.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.527035+0000 osd.5 (osd.5) 362 : cluster [DBG] 5.f scrub starts 2024-02-18T00:46:21.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.530144+0000 osd.5 (osd.5) 363 : cluster [DBG] 5.f scrub ok 2024-02-18T00:46:21.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.636491+0000 osd.1 (osd.1) 385 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:46:21.664 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:20.637855+0000 osd.1 (osd.1) 386 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:46:21.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.034722+0000 mgr.y (mgr.24454) 694 : cluster [DBG] pgmap v1243: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 13 B/s, 0 objects/s recovering 2024-02-18T00:46:21.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.081082+0000 osd.7 (osd.7) 251 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:46:21.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.082047+0000 osd.7 (osd.7) 252 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:46:21.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.083792+0000 osd.4 (osd.4) 421 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:46:21.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.087264+0000 osd.4 (osd.4) 422 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:46:21.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:20.395841+0000 mon.a (mon.0) 3764 : audit [INF] from='client.? 172.21.15.59:0/49322503' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:46:21.727 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.395904+0000 mon.a (mon.0) 3765 : cluster [DBG] osdmap e806: 8 total, 8 up, 7 in 2024-02-18T00:46:21.728 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.527035+0000 osd.5 (osd.5) 362 : cluster [DBG] 5.f scrub starts 2024-02-18T00:46:21.728 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.530144+0000 osd.5 (osd.5) 363 : cluster [DBG] 5.f scrub ok 2024-02-18T00:46:21.728 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.636491+0000 osd.1 (osd.1) 385 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:46:21.728 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:20.637855+0000 osd.1 (osd.1) 386 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:46:21.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.034722+0000 mgr.y (mgr.24454) 694 : cluster [DBG] pgmap v1243: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 op/s; 13 B/s, 0 objects/s recovering 2024-02-18T00:46:21.728 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.081082+0000 osd.7 (osd.7) 251 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.082047+0000 osd.7 (osd.7) 252 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.083792+0000 osd.4 (osd.4) 421 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.087264+0000 osd.4 (osd.4) 422 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:20.395841+0000 mon.a (mon.0) 3764 : audit [INF] from='client.? 172.21.15.59:0/49322503' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.395904+0000 mon.a (mon.0) 3765 : cluster [DBG] osdmap e806: 8 total, 8 up, 7 in 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.527035+0000 osd.5 (osd.5) 362 : cluster [DBG] 5.f scrub starts 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.530144+0000 osd.5 (osd.5) 363 : cluster [DBG] 5.f scrub ok 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.636491+0000 osd.1 (osd.1) 385 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:46:21.729 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:20.637855+0000 osd.1 (osd.1) 386 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:46:22.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:46:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:46:22.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:21.084997+0000 osd.7 (osd.7) 253 : cluster [DBG] 5.10 deep-scrub starts 2024-02-18T00:46:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:21.085980+0000 osd.7 (osd.7) 254 : cluster [DBG] 5.10 deep-scrub ok 2024-02-18T00:46:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:21.204176+0000 osd.0 (osd.0) 352 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:46:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:21.206604+0000 osd.0 (osd.0) 353 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:46:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:21.768374+0000 osd.2 (osd.2) 321 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:46:22.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:21.770318+0000 osd.2 (osd.2) 322 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:46:22.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:22] "GET /metrics HTTP/1.1" 200 187497 "" "Prometheus/2.33.4" 2024-02-18T00:46:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:21.084997+0000 osd.7 (osd.7) 253 : cluster [DBG] 5.10 deep-scrub starts 2024-02-18T00:46:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:21.085980+0000 osd.7 (osd.7) 254 : cluster [DBG] 5.10 deep-scrub ok 2024-02-18T00:46:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:21.204176+0000 osd.0 (osd.0) 352 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:46:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:21.206604+0000 osd.0 (osd.0) 353 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:46:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:21.768374+0000 osd.2 (osd.2) 321 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:46:22.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:21.770318+0000 osd.2 (osd.2) 322 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:46:22.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd unset nodeep-scrub 2024-02-18T00:46:22.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:21.084997+0000 osd.7 (osd.7) 253 : cluster [DBG] 5.10 deep-scrub starts 2024-02-18T00:46:22.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:21.085980+0000 osd.7 (osd.7) 254 : cluster [DBG] 5.10 deep-scrub ok 2024-02-18T00:46:22.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:21.204176+0000 osd.0 (osd.0) 352 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:46:22.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:21.206604+0000 osd.0 (osd.0) 353 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:46:22.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:21.768374+0000 osd.2 (osd.2) 321 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:46:22.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:21.770318+0000 osd.2 (osd.2) 322 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:46:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:21.662483+0000 mgr.y (mgr.24454) 695 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:22.035533+0000 mgr.y (mgr.24454) 696 : cluster [DBG] pgmap v1245: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-02-18T00:46:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:22.656534+0000 osd.1 (osd.1) 387 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:46:23.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:22.659810+0000 osd.1 (osd.1) 388 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:46:23.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:21.662483+0000 mgr.y (mgr.24454) 695 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:23.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:22.035533+0000 mgr.y (mgr.24454) 696 : cluster [DBG] pgmap v1245: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-02-18T00:46:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:22.656534+0000 osd.1 (osd.1) 387 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:46:23.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:22.659810+0000 osd.1 (osd.1) 388 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:46:23.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:21.662483+0000 mgr.y (mgr.24454) 695 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:23.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:22.035533+0000 mgr.y (mgr.24454) 696 : cluster [DBG] pgmap v1245: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 1.3 KiB/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-02-18T00:46:23.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:22.656534+0000 osd.1 (osd.1) 387 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:46:23.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:22.659810+0000 osd.1 (osd.1) 388 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:46:23.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.833+0000 7f353c07f700 1 -- 172.21.15.59:0/317206485 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340f7400 msgr2=0x7f3534103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:23.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.833+0000 7f353c07f700 1 --2- 172.21.15.59:0/317206485 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340f7400 0x7f3534103a20 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 rx=0x7f352c009c30 tx=0x7f352c004c50).stop 2024-02-18T00:46:23.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.833+0000 7f353c07f700 1 -- 172.21.15.59:0/317206485 shutdown_connections 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.833+0000 7f353c07f700 1 --2- 172.21.15.59:0/317206485 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340f7400 0x7f3534103a20 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.833+0000 7f353c07f700 1 --2- 172.21.15.59:0/317206485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340fcbb0 0x7f35340fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.833+0000 7f353c07f700 1 --2- 172.21.15.59:0/317206485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35340fd510 0x7f35340f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.833+0000 7f353c07f700 1 -- 172.21.15.59:0/317206485 >> 172.21.15.59:0/317206485 conn(0x7f35340f2820 msgr2=0x7f35340f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 -- 172.21.15.59:0/317206485 shutdown_connections 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 -- 172.21.15.59:0/317206485 wait complete. 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 librados: starting msgr at 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 librados: starting objecter 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 Processor -- start 2024-02-18T00:46:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 -- start start 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 librados: setting wanted keys 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.834+0000 7f353c07f700 1 librados: calling monclient init 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340f7400 0x7f3534110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35340fcbb0 0x7f35341106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340fd510 0x7f353410a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353410a890 con 0x7f35340f7400 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353410aa00 con 0x7f35340fcbb0 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353410abc0 con 0x7f35340fd510 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340f7400 0x7f3534110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353a61c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340fd510 0x7f353410a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340f7400 0x7f3534110200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56530/0 (socket says 172.21.15.59:56530) 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 -- 172.21.15.59:0/1074452816 learned_addr learned my addr 172.21.15.59:0/1074452816 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353961a700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35340fcbb0 0x7f35341106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 -- 172.21.15.59:0/1074452816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340fd510 msgr2=0x7f353410a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340fd510 0x7f353410a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 -- 172.21.15.59:0/1074452816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35340fcbb0 msgr2=0x7f35341106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35340fcbb0 0x7f35341106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 -- 172.21.15.59:0/1074452816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3528005040 con 0x7f35340f7400 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3526ffd700 10 client.?.objecter ms_handle_connect 0x7f35340f7400 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3539e1b700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340f7400 0x7f3534110200 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 rx=0x7f3528009400 tx=0x7f3528009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3526ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3526ffd700 1 -- 172.21.15.59:0/1074452816 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528007a40 con 0x7f35340f7400 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c0097c0 con 0x7f35340f7400 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3526ffd700 1 -- 172.21.15.59:0/1074452816 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f352800f040 con 0x7f35340f7400 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f3526ffd700 1 -- 172.21.15.59:0/1074452816 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528014a70 con 0x7f35340f7400 2024-02-18T00:46:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 10 client.32082.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3534117030 con 0x7f35340f7400 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.835+0000 7f353c07f700 1 librados: init done 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.836+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353405fa60 con 0x7f35340f7400 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f3526ffd700 1 -- 172.21.15.59:0/1074452816 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f35280133c0 con 0x7f35340f7400 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f3526ffd700 1 --2- 172.21.15.59:0/1074452816 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3520062c90 0x7f3520065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f3526ffd700 1 -- 172.21.15.59:0/1074452816 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(806..806 src has 418..806) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f352807cc80 con 0x7f35340f7400 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f3526ffd700 10 client.32082.objecter ms_dispatch 0x7f3534001550 osd_map(806..806 src has 418..806) v4 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f3526ffd700 3 client.32082.objecter handle_osd_map got epochs [806,806] > 0 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f3526ffd700 3 client.32082.objecter handle_osd_map decoding full epoch 806 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f3526ffd700 20 client.32082.objecter dump_active .. 0 homeless 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.837+0000 7f353961a700 1 --2- 172.21.15.59:0/1074452816 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3520062c90 0x7f3520065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.838+0000 7f353961a700 1 --2- 172.21.15.59:0/1074452816 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3520062c90 0x7f3520065150 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 rx=0x7f353410bb40 tx=0x7f353000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.838+0000 7f3526ffd700 10 client.32082.objecter ms_handle_connect 0x7f3520062c90 2024-02-18T00:46:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:23.846+0000 7f3526ffd700 1 -- 172.21.15.59:0/1074452816 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3528049f60 con 0x7f35340f7400 2024-02-18T00:46:23.926 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:24.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.037+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd in", "ids": ["6"]} v 0) v1 -- 0x7f35341060e0 con 0x7f35340f7400 2024-02-18T00:46:24.406 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:46:24.037+0000 7f6151a5e700 -1 mon.a@0(leader).osd e807 definitely_dead 0 2024-02-18T00:46:24.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:22.532906+0000 osd.5 (osd.5) 364 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:46:24.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:22.536360+0000 osd.5 (osd.5) 365 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:23.625896+0000 osd.1 (osd.1) 389 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:23.628869+0000 osd.1 (osd.1) 390 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:23.841216+0000 mon.a (mon.0) 3766 : cluster [DBG] osdmap e807: 8 total, 8 up, 7 in 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.035887+0000 mgr.y (mgr.24454) 697 : cluster [DBG] pgmap v1247: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:24.038862+0000 mon.a (mon.0) 3767 : audit [DBG] from='client.? 172.21.15.59:0/1074452816' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:22.532906+0000 osd.5 (osd.5) 364 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:22.536360+0000 osd.5 (osd.5) 365 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:23.625896+0000 osd.1 (osd.1) 389 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:23.628869+0000 osd.1 (osd.1) 390 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:23.841216+0000 mon.a (mon.0) 3766 : cluster [DBG] osdmap e807: 8 total, 8 up, 7 in 2024-02-18T00:46:24.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.035887+0000 mgr.y (mgr.24454) 697 : cluster [DBG] pgmap v1247: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:46:24.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:24.038862+0000 mon.a (mon.0) 3767 : audit [DBG] from='client.? 172.21.15.59:0/1074452816' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-02-18T00:46:24.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:22.532906+0000 osd.5 (osd.5) 364 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:46:24.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:22.536360+0000 osd.5 (osd.5) 365 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:46:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:23.625896+0000 osd.1 (osd.1) 389 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:46:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:23.628869+0000 osd.1 (osd.1) 390 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:46:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:23.841216+0000 mon.a (mon.0) 3766 : cluster [DBG] osdmap e807: 8 total, 8 up, 7 in 2024-02-18T00:46:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.035887+0000 mgr.y (mgr.24454) 697 : cluster [DBG] pgmap v1247: 174 pgs: 174 active+clean; 5.6 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:46:24.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:24.038862+0000 mon.a (mon.0) 3767 : audit [DBG] from='client.? 172.21.15.59:0/1074452816' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-02-18T00:46:24.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.855+0000 7f3526ffd700 1 -- 172.21.15.59:0/1074452816 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["6"]}]=0 marked in osd.6. v808) v1 ==== 85+0+0 (secure 0 0 0) 0x7f3528050440 con 0x7f35340f7400 2024-02-18T00:46:24.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.858+0000 7f353c07f700 10 librados: watch_flush enter 2024-02-18T00:46:24.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.858+0000 7f353c07f700 10 librados: watch_flush exit 2024-02-18T00:46:24.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.858+0000 7f353c07f700 20 client.32082.objecter shutdown clearing up homeless session... 2024-02-18T00:46:24.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.858+0000 7f353c07f700 10 client.32082.objecter successfully canceled tick 2024-02-18T00:46:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.863+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3520062c90 msgr2=0x7f3520065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.863+0000 7f353c07f700 1 --2- 172.21.15.59:0/1074452816 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3520062c90 0x7f3520065150 secure :-1 s=READY pgs=3244 cs=0 l=1 rev1=1 rx=0x7f353410bb40 tx=0x7f353000b040).stop 2024-02-18T00:46:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.863+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340f7400 msgr2=0x7f3534110200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:24.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.863+0000 7f353c07f700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340f7400 0x7f3534110200 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 rx=0x7f3528009400 tx=0x7f3528009cb0).stop 2024-02-18T00:46:24.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.871+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 shutdown_connections 2024-02-18T00:46:24.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.871+0000 7f353c07f700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f35340fd510 0x7f353410a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:24.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.871+0000 7f353c07f700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35340f7400 0x7f3534110200 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:24.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.871+0000 7f353c07f700 1 --2- 172.21.15.59:0/1074452816 >> v2:172.21.15.59:6800/3721891797 conn(0x7f3520062c90 0x7f3520065150 unknown :-1 s=CLOSED pgs=3244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:24.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.871+0000 7f353c07f700 1 --2- 172.21.15.59:0/1074452816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35340fcbb0 0x7f35341106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:24.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.871+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 >> 172.21.15.59:0/1074452816 conn(0x7f35340f2820 msgr2=0x7f3534060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:24.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.874+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 shutdown_connections 2024-02-18T00:46:24.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.874+0000 7f353c07f700 1 -- 172.21.15.59:0/1074452816 wait complete. 2024-02-18T00:46:24.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:24.875+0000 7f353c07f700 1 librados: shutdown 2024-02-18T00:46:24.879 INFO:teuthology.orchestra.run.smithi059.stderr:marked in osd.6. 2024-02-18T00:46:25.676 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.040647+0000 osd.7 (osd.7) 255 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:46:25.676 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.041504+0000 osd.7 (osd.7) 256 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:46:25.676 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.098772+0000 osd.4 (osd.4) 423 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:46:25.676 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.102093+0000 osd.4 (osd.4) 424 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:46:25.676 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.599512+0000 osd.1 (osd.1) 391 : cluster [DBG] 4.a scrub starts 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.602967+0000 osd.1 (osd.1) 392 : cluster [DBG] 4.a scrub ok 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:24.853274+0000 mon.a (mon.0) 3768 : audit [INF] from='client.? 172.21.15.59:0/1074452816' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.853343+0000 mon.a (mon.0) 3769 : cluster [DBG] osdmap e808: 8 total, 8 up, 8 in 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:24.910510+0000 mon.a (mon.0) 3770 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:25.337491+0000 mon.a (mon.0) 3771 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.040647+0000 osd.7 (osd.7) 255 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.041504+0000 osd.7 (osd.7) 256 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.098772+0000 osd.4 (osd.4) 423 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.102093+0000 osd.4 (osd.4) 424 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.599512+0000 osd.1 (osd.1) 391 : cluster [DBG] 4.a scrub starts 2024-02-18T00:46:25.677 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.602967+0000 osd.1 (osd.1) 392 : cluster [DBG] 4.a scrub ok 2024-02-18T00:46:25.678 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:24.853274+0000 mon.a (mon.0) 3768 : audit [INF] from='client.? 172.21.15.59:0/1074452816' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-02-18T00:46:25.678 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.853343+0000 mon.a (mon.0) 3769 : cluster [DBG] osdmap e808: 8 total, 8 up, 8 in 2024-02-18T00:46:25.678 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:24.910510+0000 mon.a (mon.0) 3770 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:25.678 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:25.337491+0000 mon.a (mon.0) 3771 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:25.698 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-02-18T00:46:25.699 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.meta 2024-02-18T00:46:25.699 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool set default.rgw.meta pgp_num 22 2024-02-18T00:46:25.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.040647+0000 osd.7 (osd.7) 255 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.041504+0000 osd.7 (osd.7) 256 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.098772+0000 osd.4 (osd.4) 423 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.102093+0000 osd.4 (osd.4) 424 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.599512+0000 osd.1 (osd.1) 391 : cluster [DBG] 4.a scrub starts 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.602967+0000 osd.1 (osd.1) 392 : cluster [DBG] 4.a scrub ok 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:24.853274+0000 mon.a (mon.0) 3768 : audit [INF] from='client.? 172.21.15.59:0/1074452816' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.853343+0000 mon.a (mon.0) 3769 : cluster [DBG] osdmap e808: 8 total, 8 up, 8 in 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:24.910510+0000 mon.a (mon.0) 3770 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:25.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:25.337491+0000 mon.a (mon.0) 3771 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:26.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 -- 172.21.15.59:0/3687504299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400f7400 msgr2=0x7fd740103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3687504299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400f7400 0x7fd740103a20 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 rx=0x7fd738009be0 tx=0x7fd738004d10).stop 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 -- 172.21.15.59:0/3687504299 shutdown_connections 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3687504299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7400fcbb0 0x7fd7400fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3687504299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400f7400 0x7fd740103a20 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3687504299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7400fd510 0x7fd7400f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 -- 172.21.15.59:0/3687504299 >> 172.21.15.59:0/3687504299 conn(0x7fd7400f2820 msgr2=0x7fd7400f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 -- 172.21.15.59:0/3687504299 shutdown_connections 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 -- 172.21.15.59:0/3687504299 wait complete. 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 librados: starting msgr at 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 librados: starting objecter 2024-02-18T00:46:26.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.636+0000 7fd746ad1700 1 Processor -- start 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 -- start start 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 librados: setting wanted keys 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 librados: calling monclient init 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7400f7400 0x7fd74010e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400fcbb0 0x7fd74010e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7400fd510 0x7fd740108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd740108730 con 0x7fd7400fcbb0 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7401088a0 con 0x7fd7400f7400 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd746ad1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd740108a60 con 0x7fd7400fd510 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400fcbb0 0x7fd74010e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400fcbb0 0x7fd74010e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56550/0 (socket says 172.21.15.59:56550) 2024-02-18T00:46:26.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd74506e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7400fd510 0x7fd740108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 -- 172.21.15.59:0/3596281725 learned_addr learned my addr 172.21.15.59:0/3596281725 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd74486d700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7400f7400 0x7fd74010e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 -- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7400fd510 msgr2=0x7fd740108240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7400fd510 0x7fd740108240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 -- 172.21.15.59:0/3596281725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7400f7400 msgr2=0x7fd74010e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7400f7400 0x7fd74010e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 -- 172.21.15.59:0/3596281725 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380097c0 con 0x7fd7400fcbb0 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd74506e700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7400fd510 0x7fd740108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73ffff700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400fcbb0 0x7fd74010e540 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 rx=0x7fd72c0041d0 tx=0x7fd72c009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73df76700 10 client.?.objecter ms_handle_connect 0x7fd7400fcbb0 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73df76700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.637+0000 7fd73df76700 1 -- 172.21.15.59:0/3596281725 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c007b40 con 0x7fd7400fcbb0 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.638+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd740108d40 con 0x7fd7400fcbb0 2024-02-18T00:46:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.638+0000 7fd73df76700 1 -- 172.21.15.59:0/3596281725 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd72c00f040 con 0x7fd7400fcbb0 2024-02-18T00:46:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.638+0000 7fd73df76700 1 -- 172.21.15.59:0/3596281725 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c014730 con 0x7fd7400fcbb0 2024-02-18T00:46:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.638+0000 7fd746ad1700 10 client.32088.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.638+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7401090c0 con 0x7fd7400fcbb0 2024-02-18T00:46:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.638+0000 7fd746ad1700 1 librados: init done 2024-02-18T00:46:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74005fa60 con 0x7fd7400fcbb0 2024-02-18T00:46:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd73df76700 1 -- 172.21.15.59:0/3596281725 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd72c009110 con 0x7fd7400fcbb0 2024-02-18T00:46:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd73df76700 1 --2- 172.21.15.59:0/3596281725 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd730062c90 0x7fd730065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd73df76700 1 -- 172.21.15.59:0/3596281725 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(809..809 src has 418..809) v4 ==== 7646+0+0 (secure 0 0 0) 0x7fd72c07d050 con 0x7fd7400fcbb0 2024-02-18T00:46:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd73df76700 10 client.32088.objecter ms_dispatch 0x7fd740001550 osd_map(809..809 src has 418..809) v4 2024-02-18T00:46:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd73df76700 3 client.32088.objecter handle_osd_map got epochs [809,809] > 0 2024-02-18T00:46:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd73df76700 3 client.32088.objecter handle_osd_map decoding full epoch 809 2024-02-18T00:46:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd74486d700 1 --2- 172.21.15.59:0/3596281725 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd730062c90 0x7fd730065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:26.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.639+0000 7fd73df76700 20 client.32088.objecter dump_active .. 0 homeless 2024-02-18T00:46:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.640+0000 7fd73df76700 10 client.32088.objecter ms_handle_connect 0x7fd730062c90 2024-02-18T00:46:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.640+0000 7fd74486d700 1 --2- 172.21.15.59:0/3596281725 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd730062c90 0x7fd730065150 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 rx=0x7fd7340048b0 tx=0x7fd73400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:26.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.645+0000 7fd73df76700 1 -- 172.21.15.59:0/3596281725 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd72c050700 con 0x7fd7400fcbb0 2024-02-18T00:46:26.651 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:26.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.706829+0000 osd.5 (osd.5) 366 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:24.707012+0000 osd.5 (osd.5) 367 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:25.263944+0000 osd.0 (osd.0) 354 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:25.265143+0000 osd.0 (osd.0) 355 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:25.594791+0000 osd.1 (osd.1) 393 : cluster [DBG] 2.a scrub starts 2024-02-18T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:25.597994+0000 osd.1 (osd.1) 394 : cluster [DBG] 2.a scrub ok 2024-02-18T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:25.853640+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e809: 8 total, 8 up, 8 in 2024-02-18T00:46:26.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:26.036323+0000 mgr.y (mgr.24454) 698 : cluster [DBG] pgmap v1250: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.706829+0000 osd.5 (osd.5) 366 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:24.707012+0000 osd.5 (osd.5) 367 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:25.263944+0000 osd.0 (osd.0) 354 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:25.265143+0000 osd.0 (osd.0) 355 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:25.594791+0000 osd.1 (osd.1) 393 : cluster [DBG] 2.a scrub starts 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:25.597994+0000 osd.1 (osd.1) 394 : cluster [DBG] 2.a scrub ok 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:25.853640+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e809: 8 total, 8 up, 8 in 2024-02-18T00:46:26.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:26.036323+0000 mgr.y (mgr.24454) 698 : cluster [DBG] pgmap v1250: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.706829+0000 osd.5 (osd.5) 366 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:24.707012+0000 osd.5 (osd.5) 367 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:25.263944+0000 osd.0 (osd.0) 354 : cluster [DBG] 4.6 scrub starts 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:25.265143+0000 osd.0 (osd.0) 355 : cluster [DBG] 4.6 scrub ok 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:25.594791+0000 osd.1 (osd.1) 393 : cluster [DBG] 2.a scrub starts 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:25.597994+0000 osd.1 (osd.1) 394 : cluster [DBG] 2.a scrub ok 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:25.853640+0000 mon.a (mon.0) 3772 : cluster [DBG] osdmap e809: 8 total, 8 up, 8 in 2024-02-18T00:46:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:26.036323+0000 mgr.y (mgr.24454) 698 : cluster [DBG] pgmap v1250: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:26.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.838+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) v1 -- 0x7fd740106120 con 0x7fd7400fcbb0 2024-02-18T00:46:26.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.867+0000 7fd73df76700 1 -- 172.21.15.59:0/3596281725 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v810) v1 ==== 101+0+0 (secure 0 0 0) 0x7fd72c0502f0 con 0x7fd7400fcbb0 2024-02-18T00:46:26.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.869+0000 7fd746ad1700 10 librados: watch_flush enter 2024-02-18T00:46:26.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.869+0000 7fd746ad1700 10 librados: watch_flush exit 2024-02-18T00:46:26.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.869+0000 7fd746ad1700 20 client.32088.objecter shutdown clearing up homeless session... 2024-02-18T00:46:26.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.869+0000 7fd746ad1700 10 client.32088.objecter successfully canceled tick 2024-02-18T00:46:26.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd730062c90 msgr2=0x7fd730065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:26.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3596281725 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd730062c90 0x7fd730065150 secure :-1 s=READY pgs=3245 cs=0 l=1 rev1=1 rx=0x7fd7340048b0 tx=0x7fd73400b040).stop 2024-02-18T00:46:26.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400fcbb0 msgr2=0x7fd74010e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:26.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400fcbb0 0x7fd74010e540 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 rx=0x7fd72c0041d0 tx=0x7fd72c009f60).stop 2024-02-18T00:46:26.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 shutdown_connections 2024-02-18T00:46:26.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3596281725 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd730062c90 0x7fd730065150 unknown :-1 s=CLOSED pgs=3245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7400f7400 0x7fd74010e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7400fd510 0x7fd740108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 --2- 172.21.15.59:0/3596281725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7400fcbb0 0x7fd74010e540 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:26.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 >> 172.21.15.59:0/3596281725 conn(0x7fd7400f2820 msgr2=0x7fd740060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:26.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 shutdown_connections 2024-02-18T00:46:26.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 -- 172.21.15.59:0/3596281725 wait complete. 2024-02-18T00:46:26.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:26.870+0000 7fd746ad1700 1 librados: shutdown 2024-02-18T00:46:26.875 INFO:teuthology.orchestra.run.smithi059.stderr:nodeep-scrub is unset 2024-02-18T00:46:27.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:26.839620+0000 mon.a (mon.0) 3773 : audit [INF] from='client.? 172.21.15.59:0/3596281725' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:26.867154+0000 mon.a (mon.0) 3774 : audit [INF] from='client.? 172.21.15.59:0/3596281725' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:26.867223+0000 mon.a (mon.0) 3775 : cluster [DBG] osdmap e810: 8 total, 8 up, 8 in 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:26.870013+0000 osd.3 (osd.3) 416 : cluster [DBG] 6.c starting backfill to osd.6 from (0'0,0'0] MAX to 64'1 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:27.235961+0000 osd.6 (osd.6) 345 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:27.236737+0000 osd.6 (osd.6) 346 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:26.839620+0000 mon.a (mon.0) 3773 : audit [INF] from='client.? 172.21.15.59:0/3596281725' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:26.867154+0000 mon.a (mon.0) 3774 : audit [INF] from='client.? 172.21.15.59:0/3596281725' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:26.867223+0000 mon.a (mon.0) 3775 : cluster [DBG] osdmap e810: 8 total, 8 up, 8 in 2024-02-18T00:46:27.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:26.870013+0000 osd.3 (osd.3) 416 : cluster [DBG] 6.c starting backfill to osd.6 from (0'0,0'0] MAX to 64'1 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:27.235961+0000 osd.6 (osd.6) 345 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:27.236737+0000 osd.6 (osd.6) 346 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:26.839620+0000 mon.a (mon.0) 3773 : audit [INF] from='client.? 172.21.15.59:0/3596281725' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:26.867154+0000 mon.a (mon.0) 3774 : audit [INF] from='client.? 172.21.15.59:0/3596281725' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:26.867223+0000 mon.a (mon.0) 3775 : cluster [DBG] osdmap e810: 8 total, 8 up, 8 in 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:26.870013+0000 osd.3 (osd.3) 416 : cluster [DBG] 6.c starting backfill to osd.6 from (0'0,0'0] MAX to 64'1 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:27.235961+0000 osd.6 (osd.6) 345 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:46:27.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:27.236737+0000 osd.6 (osd.6) 346 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:27.066709+0000 osd.4 (osd.4) 425 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:27.068077+0000 osd.4 (osd.4) 426 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:27.777172+0000 osd.2 (osd.2) 323 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:27.778052+0000 osd.2 (osd.2) 324 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:27.867703+0000 mon.a (mon.0) 3776 : cluster [DBG] osdmap e811: 8 total, 8 up, 8 in 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:28.036722+0000 mgr.y (mgr.24454) 699 : cluster [DBG] pgmap v1253: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:27.066709+0000 osd.4 (osd.4) 425 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:46:28.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:27.068077+0000 osd.4 (osd.4) 426 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:27.777172+0000 osd.2 (osd.2) 323 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:27.778052+0000 osd.2 (osd.2) 324 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:27.867703+0000 mon.a (mon.0) 3776 : cluster [DBG] osdmap e811: 8 total, 8 up, 8 in 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:28.036722+0000 mgr.y (mgr.24454) 699 : cluster [DBG] pgmap v1253: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:27.066709+0000 osd.4 (osd.4) 425 : cluster [DBG] 4.18 scrub starts 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:27.068077+0000 osd.4 (osd.4) 426 : cluster [DBG] 4.18 scrub ok 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:27.777172+0000 osd.2 (osd.2) 323 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:46:28.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:27.778052+0000 osd.2 (osd.2) 324 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:46:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:27.867703+0000 mon.a (mon.0) 3776 : cluster [DBG] osdmap e811: 8 total, 8 up, 8 in 2024-02-18T00:46:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:28.036722+0000 mgr.y (mgr.24454) 699 : cluster [DBG] pgmap v1253: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 -- 172.21.15.59:0/4061532816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fcbb0 msgr2=0x7ff21c0fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 --2- 172.21.15.59:0/4061532816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fcbb0 0x7ff21c0fcf90 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 rx=0x7ff2100047b0 tx=0x7ff210009cb0).stop 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 -- 172.21.15.59:0/4061532816 shutdown_connections 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 --2- 172.21.15.59:0/4061532816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff21c0f7400 0x7ff21c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 --2- 172.21.15.59:0/4061532816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fcbb0 0x7ff21c0fcf90 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 --2- 172.21.15.59:0/4061532816 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fd510 0x7ff21c0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 -- 172.21.15.59:0/4061532816 >> 172.21.15.59:0/4061532816 conn(0x7ff21c0f2820 msgr2=0x7ff21c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 -- 172.21.15.59:0/4061532816 shutdown_connections 2024-02-18T00:46:29.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.453+0000 7ff222aa2700 1 -- 172.21.15.59:0/4061532816 wait complete. 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 librados: starting msgr at 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 librados: starting objecter 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 Processor -- start 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 -- start start 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 librados: setting wanted keys 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 librados: calling monclient init 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff21c0f7400 0x7ff21c10d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fcbb0 0x7ff21c10d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fd510 0x7ff21c107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c107780 con 0x7ff21c0fd510 2024-02-18T00:46:29.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c1078f0 con 0x7ff21c0f7400 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff222aa2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c107ab0 con 0x7ff21c0fcbb0 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fcbb0 0x7ff21c10d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff22103f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fd510 0x7ff21c107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff21bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fcbb0 0x7ff21c10d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56108/0 (socket says 172.21.15.59:56108) 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.454+0000 7ff21bfff700 1 -- 172.21.15.59:0/488577864 learned_addr learned my addr 172.21.15.59:0/488577864 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff22083e700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff21c0f7400 0x7ff21c10d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff22103f700 1 -- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fcbb0 msgr2=0x7ff21c10d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff22103f700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fcbb0 0x7ff21c10d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff22103f700 1 -- 172.21.15.59:0/488577864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff21c0f7400 msgr2=0x7ff21c10d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff22103f700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff21c0f7400 0x7ff21c10d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff22103f700 1 -- 172.21.15.59:0/488577864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff208005040 con 0x7ff21c0fd510 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff21bfff700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fcbb0 0x7ff21c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff22103f700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fd510 0x7ff21c107290 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 rx=0x7ff21400c7a0 tx=0x7ff214007d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:29.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff219f76700 10 client.?.objecter ms_handle_connect 0x7ff21c0fd510 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff219f76700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff219f76700 1 -- 172.21.15.59:0/488577864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21400e4a0 con 0x7ff21c0fd510 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff219f76700 1 -- 172.21.15.59:0/488577864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff21400ead0 con 0x7ff21c0fd510 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff210005040 con 0x7ff21c0fd510 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff219f76700 1 -- 172.21.15.59:0/488577864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214012ab0 con 0x7ff21c0fd510 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff222aa2700 10 client.32094.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c1080f0 con 0x7ff21c0fd510 2024-02-18T00:46:29.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.455+0000 7ff222aa2700 1 librados: init done 2024-02-18T00:46:29.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.456+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c05fa60 con 0x7ff21c0fd510 2024-02-18T00:46:29.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 1 -- 172.21.15.59:0/488577864 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ff21400e600 con 0x7ff21c0fd510 2024-02-18T00:46:29.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 1 --2- 172.21.15.59:0/488577864 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff20c062c90 0x7ff20c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:29.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 1 -- 172.21.15.59:0/488577864 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(812..812 src has 418..812) v4 ==== 7613+0+0 (secure 0 0 0) 0x7ff21407c910 con 0x7ff21c0fd510 2024-02-18T00:46:29.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 10 client.32094.objecter ms_dispatch 0x7ff21c001550 osd_map(812..812 src has 418..812) v4 2024-02-18T00:46:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 3 client.32094.objecter handle_osd_map got epochs [812,812] > 0 2024-02-18T00:46:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 3 client.32094.objecter handle_osd_map decoding full epoch 812 2024-02-18T00:46:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff22083e700 1 --2- 172.21.15.59:0/488577864 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff20c062c90 0x7ff20c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 20 client.32094.objecter dump_active .. 0 homeless 2024-02-18T00:46:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff22083e700 1 --2- 172.21.15.59:0/488577864 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff20c062c90 0x7ff20c065150 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 rx=0x7ff210005010 tx=0x7ff210009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.457+0000 7ff219f76700 10 client.32094.objecter ms_handle_connect 0x7ff20c062c90 2024-02-18T00:46:29.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.461+0000 7ff219f76700 1 -- 172.21.15.59:0/488577864 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff214048f60 con 0x7ff21c0fd510 2024-02-18T00:46:29.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:28.521692+0000 osd.5 (osd.5) 368 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:46:29.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:28.523449+0000 osd.5 (osd.5) 369 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:46:29.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:28.872121+0000 mon.a (mon.0) 3777 : cluster [DBG] osdmap e812: 8 total, 8 up, 8 in 2024-02-18T00:46:29.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:29.107441+0000 osd.3 (osd.3) 417 : cluster [DBG] 6.c scrub starts 2024-02-18T00:46:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:29.109424+0000 osd.3 (osd.3) 418 : cluster [DBG] 6.c scrub ok 2024-02-18T00:46:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:28.521692+0000 osd.5 (osd.5) 368 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:46:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:28.523449+0000 osd.5 (osd.5) 369 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:46:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:28.872121+0000 mon.a (mon.0) 3777 : cluster [DBG] osdmap e812: 8 total, 8 up, 8 in 2024-02-18T00:46:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:29.107441+0000 osd.3 (osd.3) 417 : cluster [DBG] 6.c scrub starts 2024-02-18T00:46:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:29.109424+0000 osd.3 (osd.3) 418 : cluster [DBG] 6.c scrub ok 2024-02-18T00:46:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:29.633+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"} v 0) v1 -- 0x7ff21c1060e0 con 0x7ff21c0fd510 2024-02-18T00:46:29.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:28.521692+0000 osd.5 (osd.5) 368 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:46:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:28.523449+0000 osd.5 (osd.5) 369 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:46:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:28.872121+0000 mon.a (mon.0) 3777 : cluster [DBG] osdmap e812: 8 total, 8 up, 8 in 2024-02-18T00:46:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:29.107441+0000 osd.3 (osd.3) 417 : cluster [DBG] 6.c scrub starts 2024-02-18T00:46:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:29.109424+0000 osd.3 (osd.3) 418 : cluster [DBG] 6.c scrub ok 2024-02-18T00:46:30.426 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.425+0000 7ff219f76700 1 -- 172.21.15.59:0/488577864 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]=0 set pool 6 pgp_num to 22 v813) v1 ==== 143+0+0 (secure 0 0 0) 0x7ff2140170b0 con 0x7ff21c0fd510 2024-02-18T00:46:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 10 librados: watch_flush enter 2024-02-18T00:46:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 10 librados: watch_flush exit 2024-02-18T00:46:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 20 client.32094.objecter shutdown clearing up homeless session... 2024-02-18T00:46:30.429 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 10 client.32094.objecter successfully canceled tick 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff20c062c90 msgr2=0x7ff20c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 1 --2- 172.21.15.59:0/488577864 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff20c062c90 0x7ff20c065150 secure :-1 s=READY pgs=3246 cs=0 l=1 rev1=1 rx=0x7ff210005010 tx=0x7ff210009cb0).stop 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fd510 msgr2=0x7ff21c107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fd510 0x7ff21c107290 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 rx=0x7ff21400c7a0 tx=0x7ff214007d10).stop 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 shutdown_connections 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 1 --2- 172.21.15.59:0/488577864 >> v2:172.21.15.59:6800/3721891797 conn(0x7ff20c062c90 0x7ff20c065150 unknown :-1 s=CLOSED pgs=3246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.427+0000 7ff222aa2700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff21c0f7400 0x7ff21c10d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.428+0000 7ff222aa2700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff21c0fd510 0x7ff21c107290 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.428+0000 7ff222aa2700 1 --2- 172.21.15.59:0/488577864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff21c0fcbb0 0x7ff21c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.428+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 >> 172.21.15.59:0/488577864 conn(0x7ff21c0f2820 msgr2=0x7ff21c0fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.428+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 shutdown_connections 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.428+0000 7ff222aa2700 1 -- 172.21.15.59:0/488577864 wait complete. 2024-02-18T00:46:30.430 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:30.428+0000 7ff222aa2700 1 librados: shutdown 2024-02-18T00:46:30.431 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 6 pgp_num to 22 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:29.317458+0000 osd.0 (osd.0) 356 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:29.321433+0000 osd.0 (osd.0) 357 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:29.634502+0000 mon.a (mon.0) 3778 : audit [INF] from='client.? 172.21.15.59:0/488577864' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:30.037365+0000 mgr.y (mgr.24454) 700 : cluster [DBG] pgmap v1255: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:29.317458+0000 osd.0 (osd.0) 356 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:29.321433+0000 osd.0 (osd.0) 357 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:29.634502+0000 mon.a (mon.0) 3778 : audit [INF] from='client.? 172.21.15.59:0/488577864' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-18T00:46:30.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:30.037365+0000 mgr.y (mgr.24454) 700 : cluster [DBG] pgmap v1255: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:30.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:29.317458+0000 osd.0 (osd.0) 356 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:46:30.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:29.321433+0000 osd.0 (osd.0) 357 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:46:30.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:29.634502+0000 mon.a (mon.0) 3778 : audit [INF] from='client.? 172.21.15.59:0/488577864' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]: dispatch 2024-02-18T00:46:30.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:30.037365+0000 mgr.y (mgr.24454) 700 : cluster [DBG] pgmap v1255: 174 pgs: 62 peering, 112 active+clean; 5.2 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:31.349 INFO:tasks.thrashosds.thrasher:fixing pg num pool default.rgw.log 2024-02-18T00:46:31.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd pool set default.rgw.log pgp_num 42 2024-02-18T00:46:31.424 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:46:31.639 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:30.425416+0000 mon.a (mon.0) 3779 : audit [INF] from='client.? 172.21.15.59:0/488577864' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]': finished 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:30.425506+0000 mon.a (mon.0) 3780 : cluster [DBG] osdmap e813: 8 total, 8 up, 8 in 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:30.816406+0000 osd.2 (osd.2) 325 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:30.817588+0000 osd.2 (osd.2) 326 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.037838+0000 osd.3 (osd.3) 419 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.040958+0000 osd.3 (osd.3) 420 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.178272+0000 osd.6 (osd.6) 347 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:46:31.674 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.179428+0000 osd.6 (osd.6) 348 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:46:31.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:30.425416+0000 mon.a (mon.0) 3779 : audit [INF] from='client.? 172.21.15.59:0/488577864' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]': finished 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:30.425506+0000 mon.a (mon.0) 3780 : cluster [DBG] osdmap e813: 8 total, 8 up, 8 in 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:30.816406+0000 osd.2 (osd.2) 325 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:30.817588+0000 osd.2 (osd.2) 326 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.037838+0000 osd.3 (osd.3) 419 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.040958+0000 osd.3 (osd.3) 420 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.178272+0000 osd.6 (osd.6) 347 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.179428+0000 osd.6 (osd.6) 348 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:30.425416+0000 mon.a (mon.0) 3779 : audit [INF] from='client.? 172.21.15.59:0/488577864' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pgp_num", "val": "22"}]': finished 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:30.425506+0000 mon.a (mon.0) 3780 : cluster [DBG] osdmap e813: 8 total, 8 up, 8 in 2024-02-18T00:46:31.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:30.816406+0000 osd.2 (osd.2) 325 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:46:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:30.817588+0000 osd.2 (osd.2) 326 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:46:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.037838+0000 osd.3 (osd.3) 419 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:46:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.040958+0000 osd.3 (osd.3) 420 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:46:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.178272+0000 osd.6 (osd.6) 347 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:46:31.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.179428+0000 osd.6 (osd.6) 348 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:46:32.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:46:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:46:32.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:32] "GET /metrics HTTP/1.1" 200 187514 "" "Prometheus/2.33.4" 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.112556+0000 osd.4 (osd.4) 427 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.115993+0000 osd.4 (osd.4) 428 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.318850+0000 osd.0 (osd.0) 358 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.319999+0000 osd.0 (osd.0) 359 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:31.673282+0000 mgr.y (mgr.24454) 701 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.690181+0000 osd.1 (osd.1) 395 : cluster [DBG] 6.1 deep-scrub starts 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.693365+0000 osd.1 (osd.1) 396 : cluster [DBG] 6.1 deep-scrub ok 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.818973+0000 osd.2 (osd.2) 327 : cluster [DBG] 6.f deep-scrub starts 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:31.822086+0000 osd.2 (osd.2) 328 : cluster [DBG] 6.f deep-scrub ok 2024-02-18T00:46:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.038071+0000 osd.3 (osd.3) 421 : cluster [DBG] 4.1d deep-scrub starts 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.038457+0000 mgr.y (mgr.24454) 702 : cluster [DBG] pgmap v1257: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 593 B/s wr, 53 op/s; 730 B/s, 22 objects/s recovering 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.042761+0000 osd.3 (osd.3) 422 : cluster [DBG] 4.1d deep-scrub ok 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.112556+0000 osd.4 (osd.4) 427 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.115993+0000 osd.4 (osd.4) 428 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.318850+0000 osd.0 (osd.0) 358 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.319999+0000 osd.0 (osd.0) 359 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:31.673282+0000 mgr.y (mgr.24454) 701 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.690181+0000 osd.1 (osd.1) 395 : cluster [DBG] 6.1 deep-scrub starts 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.693365+0000 osd.1 (osd.1) 396 : cluster [DBG] 6.1 deep-scrub ok 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.818973+0000 osd.2 (osd.2) 327 : cluster [DBG] 6.f deep-scrub starts 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:31.822086+0000 osd.2 (osd.2) 328 : cluster [DBG] 6.f deep-scrub ok 2024-02-18T00:46:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.038071+0000 osd.3 (osd.3) 421 : cluster [DBG] 4.1d deep-scrub starts 2024-02-18T00:46:32.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.038457+0000 mgr.y (mgr.24454) 702 : cluster [DBG] pgmap v1257: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 593 B/s wr, 53 op/s; 730 B/s, 22 objects/s recovering 2024-02-18T00:46:32.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.042761+0000 osd.3 (osd.3) 422 : cluster [DBG] 4.1d deep-scrub ok 2024-02-18T00:46:32.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.112556+0000 osd.4 (osd.4) 427 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.115993+0000 osd.4 (osd.4) 428 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.318850+0000 osd.0 (osd.0) 358 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.319999+0000 osd.0 (osd.0) 359 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:31.673282+0000 mgr.y (mgr.24454) 701 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.690181+0000 osd.1 (osd.1) 395 : cluster [DBG] 6.1 deep-scrub starts 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.693365+0000 osd.1 (osd.1) 396 : cluster [DBG] 6.1 deep-scrub ok 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.818973+0000 osd.2 (osd.2) 327 : cluster [DBG] 6.f deep-scrub starts 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:31.822086+0000 osd.2 (osd.2) 328 : cluster [DBG] 6.f deep-scrub ok 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.038071+0000 osd.3 (osd.3) 421 : cluster [DBG] 4.1d deep-scrub starts 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.038457+0000 mgr.y (mgr.24454) 702 : cluster [DBG] pgmap v1257: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 593 B/s wr, 53 op/s; 730 B/s, 22 objects/s recovering 2024-02-18T00:46:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.042761+0000 osd.3 (osd.3) 422 : cluster [DBG] 4.1d deep-scrub ok 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.071946+0000 osd.7 (osd.7) 257 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.072925+0000 osd.7 (osd.7) 258 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.139007+0000 osd.4 (osd.4) 429 : cluster [DBG] 25.b scrub starts 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.142835+0000 osd.4 (osd.4) 430 : cluster [DBG] 25.b scrub ok 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.217827+0000 osd.2 (osd.2) 329 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.218111+0000 osd.2 (osd.2) 330 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.276801+0000 osd.0 (osd.0) 360 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.278083+0000 osd.0 (osd.0) 361 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.434527+0000 mon.a (mon.0) 3781 : cluster [DBG] osdmap e814: 8 total, 8 up, 8 in 2024-02-18T00:46:33.435 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.993920+0000 osd.3 (osd.3) 423 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:46:33.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.995060+0000 osd.3 (osd.3) 424 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:46:33.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:33.213678+0000 osd.6 (osd.6) 349 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:33.214613+0000 osd.6 (osd.6) 350 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.071946+0000 osd.7 (osd.7) 257 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.072925+0000 osd.7 (osd.7) 258 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.139007+0000 osd.4 (osd.4) 429 : cluster [DBG] 25.b scrub starts 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.142835+0000 osd.4 (osd.4) 430 : cluster [DBG] 25.b scrub ok 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.217827+0000 osd.2 (osd.2) 329 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.218111+0000 osd.2 (osd.2) 330 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.276801+0000 osd.0 (osd.0) 360 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.278083+0000 osd.0 (osd.0) 361 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.434527+0000 mon.a (mon.0) 3781 : cluster [DBG] osdmap e814: 8 total, 8 up, 8 in 2024-02-18T00:46:33.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.993920+0000 osd.3 (osd.3) 423 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:46:33.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.995060+0000 osd.3 (osd.3) 424 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:46:33.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:33.213678+0000 osd.6 (osd.6) 349 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:46:33.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:33.214613+0000 osd.6 (osd.6) 350 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:46:33.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.071946+0000 osd.7 (osd.7) 257 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:46:33.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.072925+0000 osd.7 (osd.7) 258 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:46:33.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.139007+0000 osd.4 (osd.4) 429 : cluster [DBG] 25.b scrub starts 2024-02-18T00:46:33.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.142835+0000 osd.4 (osd.4) 430 : cluster [DBG] 25.b scrub ok 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.217827+0000 osd.2 (osd.2) 329 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.218111+0000 osd.2 (osd.2) 330 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.276801+0000 osd.0 (osd.0) 360 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.278083+0000 osd.0 (osd.0) 361 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.434527+0000 mon.a (mon.0) 3781 : cluster [DBG] osdmap e814: 8 total, 8 up, 8 in 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.993920+0000 osd.3 (osd.3) 423 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.995060+0000 osd.3 (osd.3) 424 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:33.213678+0000 osd.6 (osd.6) 349 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:46:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:33.214613+0000 osd.6 (osd.6) 350 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:46:33.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 -- 172.21.15.59:0/2561606333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 msgr2=0x7fe460103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:33.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 --2- 172.21.15.59:0/2561606333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 0x7fe460103a20 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 rx=0x7fe458009c30 tx=0x7fe458004c50).stop 2024-02-18T00:46:33.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 -- 172.21.15.59:0/2561606333 shutdown_connections 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 --2- 172.21.15.59:0/2561606333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4600fcbb0 0x7fe4600fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 --2- 172.21.15.59:0/2561606333 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 0x7fe460103a20 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 --2- 172.21.15.59:0/2561606333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fd510 0x7fe4600f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 -- 172.21.15.59:0/2561606333 >> 172.21.15.59:0/2561606333 conn(0x7fe4600f2820 msgr2=0x7fe4600f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 -- 172.21.15.59:0/2561606333 shutdown_connections 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.790+0000 7fe466d32700 1 -- 172.21.15.59:0/2561606333 wait complete. 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 librados: starting msgr at 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 librados: starting objecter 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 Processor -- start 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 -- start start 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 librados: setting wanted keys 2024-02-18T00:46:33.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 librados: calling monclient init 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 0x7fe46010e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fcbb0 0x7fe46010e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4600fd510 0x7fe4601081f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601086e0 con 0x7fe4600fcbb0 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460108850 con 0x7fe4600fd510 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe466d32700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe460108a10 con 0x7fe4600f7400 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe45ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fcbb0 0x7fe46010e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:33.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe464ace700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 0x7fe46010e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe45ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fcbb0 0x7fe46010e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47888/0 (socket says 172.21.15.59:47888) 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.791+0000 7fe45ffff700 1 -- 172.21.15.59:0/583351126 learned_addr learned my addr 172.21.15.59:0/583351126 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe4652cf700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4600fd510 0x7fe4601081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe464ace700 1 -- 172.21.15.59:0/583351126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4600fd510 msgr2=0x7fe4601081f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe464ace700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4600fd510 0x7fe4601081f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe464ace700 1 -- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fcbb0 msgr2=0x7fe46010e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe464ace700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fcbb0 0x7fe46010e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe464ace700 1 -- 172.21.15.59:0/583351126 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c005040 con 0x7fe4600f7400 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe45ffff700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fcbb0 0x7fe46010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe464ace700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 0x7fe46010e050 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 rx=0x7fe454002780 tx=0x7fe454009e40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe45dffb700 10 client.?.objecter ms_handle_connect 0x7fe4600f7400 2024-02-18T00:46:33.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe45dffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe45dffb700 1 -- 172.21.15.59:0/583351126 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454007b40 con 0x7fe4600f7400 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4580097c0 con 0x7fe4600f7400 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe45dffb700 1 -- 172.21.15.59:0/583351126 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe45400f040 con 0x7fe4600f7400 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe45dffb700 1 -- 172.21.15.59:0/583351126 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454014650 con 0x7fe4600f7400 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe466d32700 10 client.41903.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe460114e80 con 0x7fe4600f7400 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.792+0000 7fe466d32700 1 librados: init done 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.793+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46005fa60 con 0x7fe4600f7400 2024-02-18T00:46:33.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45dffb700 1 -- 172.21.15.59:0/583351126 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe454007ca0 con 0x7fe4600f7400 2024-02-18T00:46:33.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45dffb700 1 --2- 172.21.15.59:0/583351126 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe450062c90 0x7fe450065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:33.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45dffb700 1 -- 172.21.15.59:0/583351126 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(814..814 src has 418..814) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fe45407d000 con 0x7fe4600f7400 2024-02-18T00:46:33.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45dffb700 10 client.41903.objecter ms_dispatch 0x7fe460001550 osd_map(814..814 src has 418..814) v4 2024-02-18T00:46:33.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45dffb700 3 client.41903.objecter handle_osd_map got epochs [814,814] > 0 2024-02-18T00:46:33.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45dffb700 3 client.41903.objecter handle_osd_map decoding full epoch 814 2024-02-18T00:46:33.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45ffff700 1 --2- 172.21.15.59:0/583351126 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe450062c90 0x7fe450065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:33.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.794+0000 7fe45dffb700 20 client.41903.objecter dump_active .. 0 homeless 2024-02-18T00:46:33.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.795+0000 7fe45dffb700 10 client.41903.objecter ms_handle_connect 0x7fe450062c90 2024-02-18T00:46:33.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.795+0000 7fe45ffff700 1 --2- 172.21.15.59:0/583351126 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe450062c90 0x7fe450065150 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 rx=0x7fe460109990 tx=0x7fe44c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:33.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.798+0000 7fe45dffb700 1 -- 172.21.15.59:0/583351126 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe4540506d0 con 0x7fe4600f7400 2024-02-18T00:46:33.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:33.982+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"} v 0) v1 -- 0x7fe4601096d0 con 0x7fe4600f7400 2024-02-18T00:46:34.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.856+0000 7fe45dffb700 1 -- 172.21.15.59:0/583351126 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]=0 set pool 4 pgp_num to 42 v816) v1 ==== 142+0+0 (secure 0 0 0) 0x7fe4540502c0 con 0x7fe4600f7400 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 10 librados: watch_flush enter 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 10 librados: watch_flush exit 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 20 client.41903.objecter shutdown clearing up homeless session... 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 10 client.41903.objecter successfully canceled tick 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe450062c90 msgr2=0x7fe450065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 --2- 172.21.15.59:0/583351126 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe450062c90 0x7fe450065150 secure :-1 s=READY pgs=3247 cs=0 l=1 rev1=1 rx=0x7fe460109990 tx=0x7fe44c00b040).stop 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 msgr2=0x7fe46010e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:34.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 0x7fe46010e050 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 rx=0x7fe454002780 tx=0x7fe454009e40).stop 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 shutdown_connections 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 --2- 172.21.15.59:0/583351126 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe450062c90 0x7fe450065150 unknown :-1 s=CLOSED pgs=3247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4600fd510 0x7fe4601081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4600f7400 0x7fe46010e050 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 --2- 172.21.15.59:0/583351126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4600fcbb0 0x7fe46010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.858+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 >> 172.21.15.59:0/583351126 conn(0x7fe4600f2820 msgr2=0x7fe460060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.859+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 shutdown_connections 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.859+0000 7fe466d32700 1 -- 172.21.15.59:0/583351126 wait complete. 2024-02-18T00:46:34.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:34.859+0000 7fe466d32700 1 librados: shutdown 2024-02-18T00:46:34.862 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 4 pgp_num to 42 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.859128+0000 osd.2 (osd.2) 331 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:32.861023+0000 osd.2 (osd.2) 332 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:33.844384+0000 mon.a (mon.0) 3782 : cluster [DBG] osdmap e815: 8 total, 8 up, 8 in 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:33.983537+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.59:0/583351126' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:33.983864+0000 mon.a (mon.0) 3783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:34.038916+0000 mgr.y (mgr.24454) 703 : cluster [DBG] pgmap v1260: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 594 B/s wr, 53 op/s; 730 B/s, 22 objects/s recovering 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.859128+0000 osd.2 (osd.2) 331 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:46:35.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:32.861023+0000 osd.2 (osd.2) 332 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:33.844384+0000 mon.a (mon.0) 3782 : cluster [DBG] osdmap e815: 8 total, 8 up, 8 in 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:33.983537+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.59:0/583351126' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:33.983864+0000 mon.a (mon.0) 3783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:34.038916+0000 mgr.y (mgr.24454) 703 : cluster [DBG] pgmap v1260: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 594 B/s wr, 53 op/s; 730 B/s, 22 objects/s recovering 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.859128+0000 osd.2 (osd.2) 331 : cluster [DBG] 4.2 scrub starts 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:32.861023+0000 osd.2 (osd.2) 332 : cluster [DBG] 4.2 scrub ok 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:33.844384+0000 mon.a (mon.0) 3782 : cluster [DBG] osdmap e815: 8 total, 8 up, 8 in 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:33.983537+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.59:0/583351126' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:46:35.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:33.983864+0000 mon.a (mon.0) 3783 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]: dispatch 2024-02-18T00:46:35.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:34.038916+0000 mgr.y (mgr.24454) 703 : cluster [DBG] pgmap v1260: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 594 B/s wr, 53 op/s; 730 B/s, 22 objects/s recovering 2024-02-18T00:46:35.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-02-18T00:46:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:33.865000+0000 osd.2 (osd.2) 333 : cluster [DBG] 2.0 deep-scrub starts 2024-02-18T00:46:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:33.866658+0000 osd.2 (osd.2) 334 : cluster [DBG] 2.0 deep-scrub ok 2024-02-18T00:46:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:34.572105+0000 osd.5 (osd.5) 370 : cluster [DBG] 5.7 deep-scrub starts 2024-02-18T00:46:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:34.575170+0000 osd.5 (osd.5) 371 : cluster [DBG] 5.7 deep-scrub ok 2024-02-18T00:46:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:34.855903+0000 mon.a (mon.0) 3784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]': finished 2024-02-18T00:46:36.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:34.856034+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e816: 8 total, 8 up, 8 in 2024-02-18T00:46:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:33.865000+0000 osd.2 (osd.2) 333 : cluster [DBG] 2.0 deep-scrub starts 2024-02-18T00:46:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:33.866658+0000 osd.2 (osd.2) 334 : cluster [DBG] 2.0 deep-scrub ok 2024-02-18T00:46:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:34.572105+0000 osd.5 (osd.5) 370 : cluster [DBG] 5.7 deep-scrub starts 2024-02-18T00:46:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:34.575170+0000 osd.5 (osd.5) 371 : cluster [DBG] 5.7 deep-scrub ok 2024-02-18T00:46:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:34.855903+0000 mon.a (mon.0) 3784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]': finished 2024-02-18T00:46:36.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:34.856034+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e816: 8 total, 8 up, 8 in 2024-02-18T00:46:36.020 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:33.865000+0000 osd.2 (osd.2) 333 : cluster [DBG] 2.0 deep-scrub starts 2024-02-18T00:46:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:33.866658+0000 osd.2 (osd.2) 334 : cluster [DBG] 2.0 deep-scrub ok 2024-02-18T00:46:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:34.572105+0000 osd.5 (osd.5) 370 : cluster [DBG] 5.7 deep-scrub starts 2024-02-18T00:46:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:34.575170+0000 osd.5 (osd.5) 371 : cluster [DBG] 5.7 deep-scrub ok 2024-02-18T00:46:36.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:34.855903+0000 mon.a (mon.0) 3784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.log", "var": "pgp_num", "val": "42"}]': finished 2024-02-18T00:46:36.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:34.856034+0000 mon.a (mon.0) 3785 : cluster [DBG] osdmap e816: 8 total, 8 up, 8 in 2024-02-18T00:46:37.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:35.710734+0000 osd.1 (osd.1) 397 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:35.714056+0000 osd.1 (osd.1) 398 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:35.817364+0000 osd.2 (osd.2) 335 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:35.819366+0000 osd.2 (osd.2) 336 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:35.966863+0000 osd.3 (osd.3) 425 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:35.968211+0000 osd.3 (osd.3) 426 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:36.039758+0000 mgr.y (mgr.24454) 704 : cluster [DBG] pgmap v1262: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 546 B/s wr, 50 op/s; 672 B/s, 20 objects/s recovering 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:35.710734+0000 osd.1 (osd.1) 397 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:35.714056+0000 osd.1 (osd.1) 398 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:46:37.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:35.817364+0000 osd.2 (osd.2) 335 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:46:37.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:35.819366+0000 osd.2 (osd.2) 336 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:46:37.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:35.966863+0000 osd.3 (osd.3) 425 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:46:37.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:35.968211+0000 osd.3 (osd.3) 426 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:46:37.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:36.039758+0000 mgr.y (mgr.24454) 704 : cluster [DBG] pgmap v1262: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 546 B/s wr, 50 op/s; 672 B/s, 20 objects/s recovering 2024-02-18T00:46:37.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:35.710734+0000 osd.1 (osd.1) 397 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:46:37.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:35.714056+0000 osd.1 (osd.1) 398 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:46:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:35.817364+0000 osd.2 (osd.2) 335 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:46:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:35.819366+0000 osd.2 (osd.2) 336 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:46:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:35.966863+0000 osd.3 (osd.3) 425 : cluster [DBG] 25.4 scrub starts 2024-02-18T00:46:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:35.968211+0000 osd.3 (osd.3) 426 : cluster [DBG] 25.4 scrub ok 2024-02-18T00:46:37.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:36.039758+0000 mgr.y (mgr.24454) 704 : cluster [DBG] pgmap v1262: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 546 B/s wr, 50 op/s; 672 B/s, 20 objects/s recovering 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 -- 172.21.15.59:0/715755452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f89a0 msgr2=0x7fa3d00f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/715755452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f89a0 0x7fa3d00f8d80 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 rx=0x7fa3c0004870 tx=0x7fa3c0009cb0).stop 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 -- 172.21.15.59:0/715755452 shutdown_connections 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/715755452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00f7330 0x7fa3d0103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/715755452 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f89a0 0x7fa3d00f8d80 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/715755452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00f9270 0x7fa3d00f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 -- 172.21.15.59:0/715755452 >> 172.21.15.59:0/715755452 conn(0x7fa3d00f26f0 msgr2=0x7fa3d00f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:38.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 -- 172.21.15.59:0/715755452 shutdown_connections 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.216+0000 7fa3d5d35700 1 -- 172.21.15.59:0/715755452 wait complete. 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 librados: starting msgr at 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 librados: starting objecter 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 Processor -- start 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 -- start start 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 librados: setting wanted keys 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 librados: calling monclient init 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f7330 0x7fa3d010d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00f89a0 0x7fa3d010d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00f9270 0x7fa3d01072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d01077e0 con 0x7fa3d00f9270 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0107950 con 0x7fa3d00f89a0 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3d5d35700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0107b10 con 0x7fa3d00f7330 2024-02-18T00:46:38.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f7330 0x7fa3d010d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3cf7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f7330 0x7fa3d010d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:56140/0 (socket says 172.21.15.59:56140) 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3cffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00f9270 0x7fa3d01072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.217+0000 7fa3cf7fe700 1 -- 172.21.15.59:0/2883720778 learned_addr learned my addr 172.21.15.59:0/2883720778 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3ceffd700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00f89a0 0x7fa3d010d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3cf7fe700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00f89a0 msgr2=0x7fa3d010d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3cf7fe700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00f89a0 0x7fa3d010d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3cf7fe700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00f9270 msgr2=0x7fa3d01072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3cf7fe700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d00f9270 0x7fa3d01072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3cf7fe700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b8005040 con 0x7fa3d00f7330 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3cf7fe700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f7330 0x7fa3d010d150 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7fa3c0009eb0 tx=0x7fa3c000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3ccff9700 10 client.?.objecter ms_handle_connect 0x7fa3d00f7330 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3ccff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000e070 con 0x7fa3d00f7330 2024-02-18T00:46:38.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3c0013da0 con 0x7fa3d00f7330 2024-02-18T00:46:38.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c0005040 con 0x7fa3d00f7330 2024-02-18T00:46:38.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0019070 con 0x7fa3d00f7330 2024-02-18T00:46:38.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3d5d35700 10 client.41909.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:38.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d0108000 con 0x7fa3d00f7330 2024-02-18T00:46:38.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.218+0000 7fa3d5d35700 1 librados: init done 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.219+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa3d005ee80 con 0x7fa3d00f7330 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.219+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa3c001e420 con 0x7fa3d00f7330 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa3bc062c90 0x7fa3bc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(816..816 src has 418..816) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fa3c007da90 con 0x7fa3d00f7330 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 10 client.41909.objecter ms_dispatch 0x7fa3d0001550 osd_map(816..816 src has 418..816) v4 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 3 client.41909.objecter handle_osd_map got epochs [816,816] > 0 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 3 client.41909.objecter handle_osd_map decoding full epoch 816 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ceffd700 1 --2- 172.21.15.59:0/2883720778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa3bc062c90 0x7fa3bc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 20 client.41909.objecter dump_active .. 0 homeless 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1259 (secure 0 0 0) 0x7fa3c0082220 con 0x7fa3d00f7330 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ccff9700 10 client.41909.objecter ms_handle_connect 0x7fa3bc062c90 2024-02-18T00:46:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3ceffd700 1 --2- 172.21.15.59:0/2883720778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa3bc062c90 0x7fa3bc065150 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 rx=0x7fa3d0108a90 tx=0x7fa3b8009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0028540 0x7fa3d00598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.220+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0108220 con 0x7fa3d0028540 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0028540 0x7fa3d00598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3d0106080 con 0x7fa3d0028540 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 10 client.41909.objecter ms_handle_connect 0x7fa3d0028540 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 10 client.41909.objecter resend_mon_ops 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0028540 0x7fa3d00598f0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 rx=0x7fa3c8009fa0 tx=0x7fa3c8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800ed80 con 0x7fa3d0028540 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7fa3c800f040 con 0x7fa3d0028540 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0028540 msgr2=0x7fa3d00598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0028540 0x7fa3d00598f0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 rx=0x7fa3c8009fa0 tx=0x7fa3c8009780).stop 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0028540 msgr2=0x7fa3d00598f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:46:38.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.221+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0028540 0x7fa3d00598f0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.228+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0059de0 0x7fa3d0117fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.228+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d00fee70 con 0x7fa3d0059de0 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.228+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0059de0 0x7fa3d0117fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.228+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) v1 -- 0x7fa3d00fefe0 con 0x7fa3d0059de0 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.229+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0059de0 0x7fa3d0117fa0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 rx=0x7fa3c8009a70 tx=0x7fa3c8019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.229+0000 7fa3ccff9700 10 client.41909.objecter ms_handle_connect 0x7fa3d0059de0 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.229+0000 7fa3ccff9700 10 client.41909.objecter resend_mon_ops 2024-02-18T00:46:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.229+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8025070 con 0x7fa3d0059de0 2024-02-18T00:46:38.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '600' (not observed, change may require restart) ) v1 ==== 85+0+3 (secure 0 0 0) 0x7fa3c8014e80 con 0x7fa3d0059de0 2024-02-18T00:46:38.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0059de0 msgr2=0x7fa3d0117fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0059de0 0x7fa3d0117fa0 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 rx=0x7fa3c8009a70 tx=0x7fa3c8019000).stop 2024-02-18T00:46:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0059de0 msgr2=0x7fa3d0117fa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:46:38.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3d0059de0 0x7fa3d0117fa0 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.232 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a: {} 2024-02-18T00:46:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00fd890 0x7fa3d00fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d00fe190 con 0x7fa3d00fd890 2024-02-18T00:46:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.230+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00fd890 0x7fa3d00fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3d00fe300 con 0x7fa3d00fd890 2024-02-18T00:46:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 10 client.41909.objecter ms_handle_connect 0x7fa3d00fd890 2024-02-18T00:46:38.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 10 client.41909.objecter resend_mon_ops 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00fd890 0x7fa3d00fdca0 secure :-1 s=READY pgs=4066 cs=0 l=1 rev1=1 rx=0x7fa3c80097c0 tx=0x7fa3c8021000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c800e040 con 0x7fa3d00fd890 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.1 v2:172.21.15.137:3300/0 2 ==== command_reply(tid 4: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7fa3c8030070 con 0x7fa3d00fd890 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00fd890 msgr2=0x7fa3d00fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00fd890 0x7fa3d00fdca0 secure :-1 s=READY pgs=4066 cs=0 l=1 rev1=1 rx=0x7fa3c80097c0 tx=0x7fa3c8021000).stop 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00fd890 msgr2=0x7fa3d00fdca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d00fd890 0x7fa3d00fdca0 secure :-1 s=CLOSED pgs=4066 cs=0 l=1 rev1=1 rx=0x7fa3c80097c0 tx=0x7fa3c8021000).stop 2024-02-18T00:46:38.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.231+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 reap_dead start 2024-02-18T00:46:38.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.238+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d01072f0 0x7fa3d00fe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.238+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d010d7f0 con 0x7fa3d01072f0 2024-02-18T00:46:38.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.238+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d01072f0 0x7fa3d00fe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.239+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) v1 -- 0x7fa3d010d930 con 0x7fa3d01072f0 2024-02-18T00:46:38.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.239+0000 7fa3ccff9700 10 client.41909.objecter ms_handle_connect 0x7fa3d01072f0 2024-02-18T00:46:38.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.239+0000 7fa3ccff9700 10 client.41909.objecter resend_mon_ops 2024-02-18T00:46:38.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.239+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d01072f0 0x7fa3d00fe4d0 secure :-1 s=READY pgs=4067 cs=0 l=1 rev1=1 rx=0x7fa3d00fe2d0 tx=0x7fa3c802e070).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.239+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8021950 con 0x7fa3d01072f0 2024-02-18T00:46:38.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.240+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.1 v2:172.21.15.137:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '600' (not observed, change may require restart) ) v1 ==== 85+0+3 (secure 0 0 0) 0x7fa3c803e070 con 0x7fa3d01072f0 2024-02-18T00:46:38.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.240+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d01072f0 msgr2=0x7fa3d00fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.240+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d01072f0 0x7fa3d00fe4d0 secure :-1 s=READY pgs=4067 cs=0 l=1 rev1=1 rx=0x7fa3d00fe2d0 tx=0x7fa3c802e070).stop 2024-02-18T00:46:38.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.240+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d01072f0 msgr2=0x7fa3d00fe4d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:46:38.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.240+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3d01072f0 0x7fa3d00fe4d0 unknown :-1 s=CLOSED pgs=4067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.242 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b: {} 2024-02-18T00:46:38.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00fe9c0 0x7fa3d011a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d011d0f0 con 0x7fa3d00fe9c0 2024-02-18T00:46:38.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00fe9c0 0x7fa3d011a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3d011d230 con 0x7fa3d00fe9c0 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00fe9c0 0x7fa3d011a4a0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7fa3c8014df0 tx=0x7fa3c803c070).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3ccff9700 10 client.41909.objecter ms_handle_connect 0x7fa3d00fe9c0 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3ccff9700 10 client.41909.objecter resend_mon_ops 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.241+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8021c00 con 0x7fa3d00fe9c0 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.242+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 2 ==== command_reply(tid 6: 0 ) v1 ==== 8+0+8167 (secure 0 0 0) 0x7fa3c80136a0 con 0x7fa3d00fe9c0 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.242+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00fe9c0 msgr2=0x7fa3d011a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.242+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00fe9c0 0x7fa3d011a4a0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 rx=0x7fa3c8014df0 tx=0x7fa3c803c070).stop 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.242+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00fe9c0 msgr2=0x7fa3d011a4a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:46:38.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.242+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00fe9c0 0x7fa3d011a4a0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.248+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d011d4f0 0x7fa3d011c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.248+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d011ce90 con 0x7fa3d011d4f0 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.248+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d011d4f0 0x7fa3d011c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.248+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) v1 -- 0x7fa3d011eeb0 con 0x7fa3d011d4f0 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.248+0000 7fa3ccff9700 10 client.41909.objecter ms_handle_connect 0x7fa3d011d4f0 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.248+0000 7fa3ccff9700 10 client.41909.objecter resend_mon_ops 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.248+0000 7fa3cffff700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d011d4f0 0x7fa3d011c9a0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 rx=0x7fa3d00f48d0 tx=0x7fa3c8046000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:38.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.249+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8045e40 con 0x7fa3d011d4f0 2024-02-18T00:46:38.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:36.269843+0000 osd.6 (osd.6) 351 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:46:38.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:36.270796+0000 osd.6 (osd.6) 352 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:46:38.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.032154+0000 osd.7 (osd.7) 259 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:46:38.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.033119+0000 osd.7 (osd.7) 260 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:46:38.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.514702+0000 osd.5 (osd.5) 372 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:46:38.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.515706+0000 osd.5 (osd.5) 373 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:46:38.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:36.269843+0000 osd.6 (osd.6) 351 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:46:38.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:36.270796+0000 osd.6 (osd.6) 352 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:46:38.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.032154+0000 osd.7 (osd.7) 259 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:46:38.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.033119+0000 osd.7 (osd.7) 260 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:46:38.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.514702+0000 osd.5 (osd.5) 372 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:46:38.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.515706+0000 osd.5 (osd.5) 373 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:46:38.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 <== mon.2 v2:172.21.15.59:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '600' (not observed, change may require restart) ) v1 ==== 85+0+3 (secure 0 0 0) 0x7fa3c8057070 con 0x7fa3d011d4f0 2024-02-18T00:46:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d011d4f0 msgr2=0x7fa3d011c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d011d4f0 0x7fa3d011c9a0 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 rx=0x7fa3d00f48d0 tx=0x7fa3c8046000).stop 2024-02-18T00:46:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3ccff9700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d011d4f0 msgr2=0x7fa3d011c9a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-02-18T00:46:38.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3ccff9700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d011d4f0 0x7fa3d011c9a0 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:38.253 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c: {} 2024-02-18T00:46:38.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 10 librados: watch_flush enter 2024-02-18T00:46:38.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 10 librados: watch_flush exit 2024-02-18T00:46:38.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 20 client.41909.objecter shutdown clearing up homeless session... 2024-02-18T00:46:38.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 10 client.41909.objecter successfully canceled tick 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa3bc062c90 msgr2=0x7fa3bc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa3bc062c90 0x7fa3bc065150 secure :-1 s=READY pgs=3248 cs=0 l=1 rev1=1 rx=0x7fa3d0108a90 tx=0x7fa3b8009cb0).stop 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f7330 msgr2=0x7fa3d010d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 1 --2- 172.21.15.59:0/2883720778 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3d00f7330 0x7fa3d010d150 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 rx=0x7fa3c0009eb0 tx=0x7fa3c000b040).stop 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3cffff700 1 -- 172.21.15.59:0/2883720778 reap_dead start 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 shutdown_connections 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.250+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 >> 172.21.15.59:0/2883720778 conn(0x7fa3d00f26f0 msgr2=0x7fa3d00fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.251+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 shutdown_connections 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.251+0000 7fa3d5d35700 1 -- 172.21.15.59:0/2883720778 wait complete. 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:38.251+0000 7fa3d5d35700 1 librados: shutdown 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:mon.a: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:mon.b: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-18T00:46:38.255 INFO:teuthology.orchestra.run.smithi059.stderr:mon.c: mon_osd_down_out_interval = '600' (not observed, change may require restart) 2024-02-18T00:46:38.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:36.269843+0000 osd.6 (osd.6) 351 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:46:38.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:36.270796+0000 osd.6 (osd.6) 352 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:46:38.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.032154+0000 osd.7 (osd.7) 259 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:46:38.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.033119+0000 osd.7 (osd.7) 260 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:46:38.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.514702+0000 osd.5 (osd.5) 372 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:46:38.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.515706+0000 osd.5 (osd.5) 373 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.284990+0000 osd.6 (osd.6) 353 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.286174+0000 osd.6 (osd.6) 354 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.645309+0000 osd.1 (osd.1) 399 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.646859+0000 osd.1 (osd.1) 400 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.997983+0000 osd.3 (osd.3) 427 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:37.999522+0000 osd.3 (osd.3) 428 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.040326+0000 mgr.y (mgr.24454) 705 : cluster [DBG] pgmap v1263: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.048880+0000 osd.4 (osd.4) 431 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.050159+0000 osd.4 (osd.4) 432 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:38.221238+0000 mon.c (mon.2) 585 : audit [DBG] from='client.? 172.21.15.59:0/2883720778' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:46:39.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.848961+0000 mon.a (mon.0) 3786 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.284990+0000 osd.6 (osd.6) 353 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.286174+0000 osd.6 (osd.6) 354 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.645309+0000 osd.1 (osd.1) 399 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.646859+0000 osd.1 (osd.1) 400 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.997983+0000 osd.3 (osd.3) 427 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:37.999522+0000 osd.3 (osd.3) 428 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.040326+0000 mgr.y (mgr.24454) 705 : cluster [DBG] pgmap v1263: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.048880+0000 osd.4 (osd.4) 431 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.050159+0000 osd.4 (osd.4) 432 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:38.221238+0000 mon.c (mon.2) 585 : audit [DBG] from='client.? 172.21.15.59:0/2883720778' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:46:39.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.848961+0000 mon.a (mon.0) 3786 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-02-18T00:46:39.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 0 1 2024-02-18T00:46:39.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.284990+0000 osd.6 (osd.6) 353 : cluster [DBG] 25.8 scrub starts 2024-02-18T00:46:39.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.286174+0000 osd.6 (osd.6) 354 : cluster [DBG] 25.8 scrub ok 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.645309+0000 osd.1 (osd.1) 399 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.646859+0000 osd.1 (osd.1) 400 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.997983+0000 osd.3 (osd.3) 427 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:37.999522+0000 osd.3 (osd.3) 428 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.040326+0000 mgr.y (mgr.24454) 705 : cluster [DBG] pgmap v1263: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.048880+0000 osd.4 (osd.4) 431 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.050159+0000 osd.4 (osd.4) 432 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:38.221238+0000 mon.c (mon.2) 585 : audit [DBG] from='client.? 172.21.15.59:0/2883720778' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T00:46:39.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.848961+0000 mon.a (mon.0) 3786 : cluster [DBG] osdmap e817: 8 total, 8 up, 8 in 2024-02-18T00:46:39.361 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:39.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.275484+0000 osd.0 (osd.0) 362 : cluster [DBG] 25.a scrub starts 2024-02-18T00:46:39.990 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.278477+0000 osd.0 (osd.0) 363 : cluster [DBG] 25.a scrub ok 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.560141+0000 osd.5 (osd.5) 374 : cluster [DBG] 25.1d deep-scrub starts 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.563629+0000 osd.5 (osd.5) 375 : cluster [DBG] 25.1d deep-scrub ok 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.669456+0000 osd.1 (osd.1) 401 : cluster [DBG] 5.c scrub starts 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:38.672363+0000 osd.1 (osd.1) 402 : cluster [DBG] 5.c scrub ok 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.275484+0000 osd.0 (osd.0) 362 : cluster [DBG] 25.a scrub starts 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.278477+0000 osd.0 (osd.0) 363 : cluster [DBG] 25.a scrub ok 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.560141+0000 osd.5 (osd.5) 374 : cluster [DBG] 25.1d deep-scrub starts 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.563629+0000 osd.5 (osd.5) 375 : cluster [DBG] 25.1d deep-scrub ok 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.669456+0000 osd.1 (osd.1) 401 : cluster [DBG] 5.c scrub starts 2024-02-18T00:46:39.991 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:38.672363+0000 osd.1 (osd.1) 402 : cluster [DBG] 5.c scrub ok 2024-02-18T00:46:40.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.275484+0000 osd.0 (osd.0) 362 : cluster [DBG] 25.a scrub starts 2024-02-18T00:46:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.278477+0000 osd.0 (osd.0) 363 : cluster [DBG] 25.a scrub ok 2024-02-18T00:46:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.560141+0000 osd.5 (osd.5) 374 : cluster [DBG] 25.1d deep-scrub starts 2024-02-18T00:46:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.563629+0000 osd.5 (osd.5) 375 : cluster [DBG] 25.1d deep-scrub ok 2024-02-18T00:46:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.669456+0000 osd.1 (osd.1) 401 : cluster [DBG] 5.c scrub starts 2024-02-18T00:46:40.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:38.672363+0000 osd.1 (osd.1) 402 : cluster [DBG] 5.c scrub ok 2024-02-18T00:46:41.362 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:46:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 -- 172.21.15.59:0/2477960919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 msgr2=0x7f90780f9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2477960919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f90780f9090 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 rx=0x7f9068009c30 tx=0x7f9068004c50).stop 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 -- 172.21.15.59:0/2477960919 shutdown_connections 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2477960919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9078102210 0x7f9078104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2477960919 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f90780f9090 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2477960919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90780f9610 0x7f9078101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 -- 172.21.15.59:0/2477960919 >> 172.21.15.59:0/2477960919 conn(0x7f90780f2820 msgr2=0x7f90780f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 -- 172.21.15.59:0/2477960919 shutdown_connections 2024-02-18T00:46:41.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 -- 172.21.15.59:0/2477960919 wait complete. 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 librados: starting msgr at 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 librados: starting objecter 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.624+0000 7f907ddfb700 1 Processor -- start 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 -- start start 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 librados: setting wanted keys 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 librados: calling monclient init 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f907810e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90780f9610 0x7f907810e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9078102210 0x7f90781083a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9078108890 con 0x7f9078102210 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9078108a00 con 0x7f90780f9610 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f907ddfb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9078108bc0 con 0x7f90780f8cb0 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f90777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f907810e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:41.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f90777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f907810e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51000/0 (socket says 172.21.15.59:51000) 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f90777fe700 1 -- 172.21.15.59:0/2167285862 learned_addr learned my addr 172.21.15.59:0/2167285862 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9077fff700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9078102210 0x7f90781083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9076ffd700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90780f9610 0x7f907810e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9077fff700 1 -- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 msgr2=0x7f907810e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9077fff700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f907810e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9077fff700 1 -- 172.21.15.59:0/2167285862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90780f9610 msgr2=0x7f907810e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9077fff700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90780f9610 0x7f907810e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9077fff700 1 -- 172.21.15.59:0/2167285862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060005040 con 0x7f9078102210 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f90777fe700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f907810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9077fff700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9078102210 0x7f90781083a0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 rx=0x7f9070002750 tx=0x7f9070009e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9074ff9700 10 client.?.objecter ms_handle_connect 0x7f9078102210 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.625+0000 7f9074ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.626+0000 7f9074ff9700 1 -- 172.21.15.59:0/2167285862 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070007b40 con 0x7f9078102210 2024-02-18T00:46:41.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.626+0000 7f9074ff9700 1 -- 172.21.15.59:0/2167285862 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f907000f040 con 0x7f9078102210 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.626+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90680097c0 con 0x7f9078102210 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.626+0000 7f9074ff9700 1 -- 172.21.15.59:0/2167285862 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070013660 con 0x7f9078102210 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.626+0000 7f907ddfb700 10 client.32118.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.626+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9078109200 con 0x7f9078102210 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.626+0000 7f907ddfb700 1 librados: init done 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.627+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907805fa60 con 0x7f9078102210 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.627+0000 7f9074ff9700 1 -- 172.21.15.59:0/2167285862 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9070013a90 con 0x7f9078102210 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f9074ff9700 1 --2- 172.21.15.59:0/2167285862 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9064062c90 0x7f9064065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f90777fe700 1 --2- 172.21.15.59:0/2167285862 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9064062c90 0x7f9064065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f9074ff9700 1 -- 172.21.15.59:0/2167285862 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(817..817 src has 418..817) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f907007ca30 con 0x7f9078102210 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f9074ff9700 10 client.32118.objecter ms_dispatch 0x7f9078001550 osd_map(817..817 src has 418..817) v4 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f9074ff9700 3 client.32118.objecter handle_osd_map got epochs [817,817] > 0 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f9074ff9700 3 client.32118.objecter handle_osd_map decoding full epoch 817 2024-02-18T00:46:41.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f9074ff9700 20 client.32118.objecter dump_active .. 0 homeless 2024-02-18T00:46:41.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f9074ff9700 10 client.32118.objecter ms_handle_connect 0x7f9064062c90 2024-02-18T00:46:41.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.628+0000 7f90777fe700 1 --2- 172.21.15.59:0/2167285862 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9064062c90 0x7f9064065150 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 rx=0x7f9068004d40 tx=0x7f9068004c50).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:41.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.632+0000 7f9074ff9700 1 -- 172.21.15.59:0/2167285862 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9070050100 con 0x7f9078102210 2024-02-18T00:46:41.685 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:40.027371+0000 osd.7 (osd.7) 261 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:46:41.685 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:40.028405+0000 osd.7 (osd.7) 262 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:46:41.685 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:40.040812+0000 mgr.y (mgr.24454) 706 : cluster [DBG] pgmap v1265: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:46:41.685 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:40.361829+0000 mon.a (mon.0) 3787 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:41.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:40.027371+0000 osd.7 (osd.7) 261 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:46:41.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:40.028405+0000 osd.7 (osd.7) 262 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:46:41.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:40.040812+0000 mgr.y (mgr.24454) 706 : cluster [DBG] pgmap v1265: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:46:41.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:40.361829+0000 mon.a (mon.0) 3787 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:40.027371+0000 osd.7 (osd.7) 261 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:46:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:40.028405+0000 osd.7 (osd.7) 262 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:46:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:40.040812+0000 mgr.y (mgr.24454) 706 : cluster [DBG] pgmap v1265: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:46:41.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:40.361829+0000 mon.a (mon.0) 3787 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:46:41.818 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:41.816+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) v1 -- 0x7f90781072f0 con 0x7f9078102210 2024-02-18T00:46:42.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:46:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:46:42.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.367+0000 7f9074ff9700 1 -- 172.21.15.59:0/2167285862 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v818) v1 ==== 113+0+0 (secure 0 0 0) 0x7f9070019090 con 0x7f9078102210 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 10 librados: watch_flush enter 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 10 librados: watch_flush exit 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 20 client.32118.objecter shutdown clearing up homeless session... 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 10 client.32118.objecter successfully canceled tick 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9064062c90 msgr2=0x7f9064065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2167285862 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9064062c90 0x7f9064065150 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 rx=0x7f9068004d40 tx=0x7f9068004c50).stop 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9078102210 msgr2=0x7f90781083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:42.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9078102210 0x7f90781083a0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 rx=0x7f9070002750 tx=0x7f9070009e80).stop 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 shutdown_connections 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9078102210 0x7f90781083a0 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90780f8cb0 0x7f907810e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2167285862 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9064062c90 0x7f9064065150 unknown :-1 s=CLOSED pgs=3249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 --2- 172.21.15.59:0/2167285862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90780f9610 0x7f907810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 >> 172.21.15.59:0/2167285862 conn(0x7f90780f2820 msgr2=0x7f90780f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 shutdown_connections 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 -- 172.21.15.59:0/2167285862 wait complete. 2024-02-18T00:46:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:42.370+0000 7f907ddfb700 1 librados: shutdown 2024-02-18T00:46:42.375 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.0 to 1 (10000) 2024-02-18T00:46:42.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:42] "GET /metrics HTTP/1.1" 200 187513 "" "Prometheus/2.33.4" 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:40.934453+0000 osd.2 (osd.2) 337 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:40.935564+0000 osd.2 (osd.2) 338 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:41.051495+0000 osd.4 (osd.4) 433 : cluster [DBG] 4.22 scrub starts 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:41.053340+0000 osd.4 (osd.4) 434 : cluster [DBG] 4.22 scrub ok 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:41.749318+0000 osd.1 (osd.1) 403 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:41.752359+0000 osd.1 (osd.1) 404 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:41.818465+0000 mon.a (mon.0) 3788 : audit [INF] from='client.? 172.21.15.59:0/2167285862' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:46:42.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:41.941929+0000 osd.3 (osd.3) 429 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:41.943521+0000 osd.3 (osd.3) 430 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:40.934453+0000 osd.2 (osd.2) 337 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:40.935564+0000 osd.2 (osd.2) 338 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:41.051495+0000 osd.4 (osd.4) 433 : cluster [DBG] 4.22 scrub starts 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:41.053340+0000 osd.4 (osd.4) 434 : cluster [DBG] 4.22 scrub ok 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:41.749318+0000 osd.1 (osd.1) 403 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:41.752359+0000 osd.1 (osd.1) 404 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:41.818465+0000 mon.a (mon.0) 3788 : audit [INF] from='client.? 172.21.15.59:0/2167285862' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:41.941929+0000 osd.3 (osd.3) 429 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:46:42.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:41.943521+0000 osd.3 (osd.3) 430 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:46:42.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:40.934453+0000 osd.2 (osd.2) 337 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:40.935564+0000 osd.2 (osd.2) 338 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:41.051495+0000 osd.4 (osd.4) 433 : cluster [DBG] 4.22 scrub starts 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:41.053340+0000 osd.4 (osd.4) 434 : cluster [DBG] 4.22 scrub ok 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:41.749318+0000 osd.1 (osd.1) 403 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:41.752359+0000 osd.1 (osd.1) 404 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:41.818465+0000 mon.a (mon.0) 3788 : audit [INF] from='client.? 172.21.15.59:0/2167285862' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:41.941929+0000 osd.3 (osd.3) 429 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:46:42.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:41.943521+0000 osd.3 (osd.3) 430 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:46:43.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 0 1 2024-02-18T00:46:43.583 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:43.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:41.684225+0000 mgr.y (mgr.24454) 707 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.002664+0000 osd.4 (osd.4) 435 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.004682+0000 osd.4 (osd.4) 436 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.041734+0000 mgr.y (mgr.24454) 708 : cluster [DBG] pgmap v1266: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.087779+0000 osd.7 (osd.7) 263 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.088933+0000 osd.7 (osd.7) 264 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:42.367045+0000 mon.a (mon.0) 3789 : audit [INF] from='client.? 172.21.15.59:0/2167285862' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.367140+0000 mon.a (mon.0) 3790 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:41.684225+0000 mgr.y (mgr.24454) 707 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:43.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.002664+0000 osd.4 (osd.4) 435 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.004682+0000 osd.4 (osd.4) 436 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.041734+0000 mgr.y (mgr.24454) 708 : cluster [DBG] pgmap v1266: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.087779+0000 osd.7 (osd.7) 263 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.088933+0000 osd.7 (osd.7) 264 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:42.367045+0000 mon.a (mon.0) 3789 : audit [INF] from='client.? 172.21.15.59:0/2167285862' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.367140+0000 mon.a (mon.0) 3790 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:41.684225+0000 mgr.y (mgr.24454) 707 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:43.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.002664+0000 osd.4 (osd.4) 435 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:46:43.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.004682+0000 osd.4 (osd.4) 436 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:46:43.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.041734+0000 mgr.y (mgr.24454) 708 : cluster [DBG] pgmap v1266: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:46:43.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.087779+0000 osd.7 (osd.7) 263 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:46:43.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.088933+0000 osd.7 (osd.7) 264 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:46:43.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:42.367045+0000 mon.a (mon.0) 3789 : audit [INF] from='client.? 172.21.15.59:0/2167285862' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:46:43.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.367140+0000 mon.a (mon.0) 3790 : cluster [DBG] osdmap e818: 8 total, 8 up, 8 in 2024-02-18T00:46:44.651 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.640668+0000 osd.5 (osd.5) 376 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:46:44.651 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:42.642578+0000 osd.5 (osd.5) 377 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:46:44.651 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:43.286494+0000 osd.0 (osd.0) 364 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:43.288014+0000 osd.0 (osd.0) 365 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:43.437316+0000 osd.6 (osd.6) 355 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:43.438348+0000 osd.6 (osd.6) 356 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:43.845767+0000 mon.a (mon.0) 3791 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:43.863827+0000 osd.1 (osd.1) 405 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:43.864027+0000 osd.1 (osd.1) 406 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.640668+0000 osd.5 (osd.5) 376 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:42.642578+0000 osd.5 (osd.5) 377 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:43.286494+0000 osd.0 (osd.0) 364 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:46:44.652 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:43.288014+0000 osd.0 (osd.0) 365 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:46:44.653 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:43.437316+0000 osd.6 (osd.6) 355 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:46:44.653 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:43.438348+0000 osd.6 (osd.6) 356 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:46:44.653 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:43.845767+0000 mon.a (mon.0) 3791 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-02-18T00:46:44.653 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:43.863827+0000 osd.1 (osd.1) 405 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:44.653 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:43.864027+0000 osd.1 (osd.1) 406 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.640668+0000 osd.5 (osd.5) 376 : cluster [DBG] 5.1b scrub starts 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:42.642578+0000 osd.5 (osd.5) 377 : cluster [DBG] 5.1b scrub ok 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:43.286494+0000 osd.0 (osd.0) 364 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:43.288014+0000 osd.0 (osd.0) 365 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:43.437316+0000 osd.6 (osd.6) 355 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:43.438348+0000 osd.6 (osd.6) 356 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:43.845767+0000 mon.a (mon.0) 3791 : cluster [DBG] osdmap e819: 8 total, 8 up, 8 in 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:43.863827+0000 osd.1 (osd.1) 405 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:46:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:43.864027+0000 osd.1 (osd.1) 406 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 -- 172.21.15.59:0/3174659243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980f7400 msgr2=0x7f1898103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 --2- 172.21.15.59:0/3174659243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980f7400 0x7f1898103a20 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 rx=0x7f1890009c30 tx=0x7f1890004c50).stop 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 -- 172.21.15.59:0/3174659243 shutdown_connections 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 --2- 172.21.15.59:0/3174659243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18980fcbb0 0x7f18980fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 --2- 172.21.15.59:0/3174659243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980f7400 0x7f1898103a20 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 --2- 172.21.15.59:0/3174659243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18980fd510 0x7f18980f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 -- 172.21.15.59:0/3174659243 >> 172.21.15.59:0/3174659243 conn(0x7f18980f2820 msgr2=0x7f18980f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:45.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 -- 172.21.15.59:0/3174659243 shutdown_connections 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 -- 172.21.15.59:0/3174659243 wait complete. 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 librados: starting msgr at 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.601+0000 7f189fbe5700 1 librados: starting objecter 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 Processor -- start 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 -- start start 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 librados: setting wanted keys 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 librados: calling monclient init 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18980f7400 0x7f18980f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980fcbb0 0x7f18980f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18980fd510 0x7f18980f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981163e0 con 0x7f18980fcbb0 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1898116550 con 0x7f18980fd510 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189fbe5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1898116710 con 0x7f18980f7400 2024-02-18T00:46:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980fcbb0 0x7f18980f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d981700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18980f7400 0x7f18980f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980fcbb0 0x7f18980f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49824/0 (socket says 172.21.15.59:49824) 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 -- 172.21.15.59:0/164862601 learned_addr learned my addr 172.21.15.59:0/164862601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 -- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18980f7400 msgr2=0x7f18980f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18980f7400 0x7f18980f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 -- 172.21.15.59:0/164862601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18980fd510 msgr2=0x7f18980f9390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18980fd510 0x7f18980f9390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 -- 172.21.15.59:0/164862601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1888005040 con 0x7f18980fcbb0 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d981700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18980f7400 0x7f18980f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f189d180700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980fcbb0 0x7f18980f8ea0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 rx=0x7f1888009400 tx=0x7f1888009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f188effd700 10 client.?.objecter ms_handle_connect 0x7f18980fcbb0 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f188effd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.602+0000 7f188effd700 1 -- 172.21.15.59:0/164862601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888007a40 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.603+0000 7f188effd700 1 -- 172.21.15.59:0/164862601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188800f040 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.603+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18900097c0 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.603+0000 7f188effd700 1 -- 172.21.15.59:0/164862601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888014aa0 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.603+0000 7f189fbe5700 10 client.32124.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.603+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1898116da0 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.603+0000 7f189fbe5700 1 librados: init done 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.604+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189805fa60 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f188effd700 1 -- 172.21.15.59:0/164862601 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f188800e630 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f188effd700 1 --2- 172.21.15.59:0/164862601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1884062c90 0x7f1884065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f188effd700 1 -- 172.21.15.59:0/164862601 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(820..820 src has 418..820) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f188807d0a0 con 0x7f18980fcbb0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f188effd700 10 client.32124.objecter ms_dispatch 0x7f1898001550 osd_map(820..820 src has 418..820) v4 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f188effd700 3 client.32124.objecter handle_osd_map got epochs [820,820] > 0 2024-02-18T00:46:45.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f188effd700 3 client.32124.objecter handle_osd_map decoding full epoch 820 2024-02-18T00:46:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f189d981700 1 --2- 172.21.15.59:0/164862601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1884062c90 0x7f1884065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.605+0000 7f188effd700 20 client.32124.objecter dump_active .. 0 homeless 2024-02-18T00:46:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.606+0000 7f189d981700 1 --2- 172.21.15.59:0/164862601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1884062c90 0x7f1884065150 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 rx=0x7f18940103c0 tx=0x7f189400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.606+0000 7f188effd700 10 client.32124.objecter ms_handle_connect 0x7f1884062c90 2024-02-18T00:46:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.609+0000 7f188effd700 1 -- 172.21.15.59:0/164862601 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1888050810 con 0x7f18980fcbb0 2024-02-18T00:46:45.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:44.042093+0000 mgr.y (mgr.24454) 709 : cluster [DBG] pgmap v1269: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:46:45.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:44.860147+0000 mon.a (mon.0) 3792 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-02-18T00:46:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:44.866569+0000 osd.3 (osd.3) 431 : cluster [DBG] 25.5 deep-scrub starts 2024-02-18T00:46:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:44.868407+0000 osd.3 (osd.3) 432 : cluster [DBG] 25.5 deep-scrub ok 2024-02-18T00:46:45.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:44.042093+0000 mgr.y (mgr.24454) 709 : cluster [DBG] pgmap v1269: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:46:45.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:44.860147+0000 mon.a (mon.0) 3792 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-02-18T00:46:45.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:44.866569+0000 osd.3 (osd.3) 431 : cluster [DBG] 25.5 deep-scrub starts 2024-02-18T00:46:45.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:44.868407+0000 osd.3 (osd.3) 432 : cluster [DBG] 25.5 deep-scrub ok 2024-02-18T00:46:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:44.042093+0000 mgr.y (mgr.24454) 709 : cluster [DBG] pgmap v1269: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:46:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:44.860147+0000 mon.a (mon.0) 3792 : cluster [DBG] osdmap e820: 8 total, 8 up, 8 in 2024-02-18T00:46:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:44.866569+0000 osd.3 (osd.3) 431 : cluster [DBG] 25.5 deep-scrub starts 2024-02-18T00:46:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:44.868407+0000 osd.3 (osd.3) 432 : cluster [DBG] 25.5 deep-scrub ok 2024-02-18T00:46:45.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:45.794+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) v1 -- 0x7f18980f9e90 con 0x7f18980fcbb0 2024-02-18T00:46:46.382 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.382+0000 7f188effd700 1 -- 172.21.15.59:0/164862601 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v821) v1 ==== 133+0+0 (secure 0 0 0) 0x7f18880220a0 con 0x7f18980fcbb0 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.384+0000 7f189fbe5700 10 librados: watch_flush enter 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.384+0000 7f189fbe5700 10 librados: watch_flush exit 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.384+0000 7f189fbe5700 20 client.32124.objecter shutdown clearing up homeless session... 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.384+0000 7f189fbe5700 10 client.32124.objecter successfully canceled tick 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1884062c90 msgr2=0x7f1884065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 --2- 172.21.15.59:0/164862601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1884062c90 0x7f1884065150 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 rx=0x7f18940103c0 tx=0x7f189400b040).stop 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980fcbb0 msgr2=0x7f18980f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980fcbb0 0x7f18980f8ea0 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 rx=0x7f1888009400 tx=0x7f1888009cb0).stop 2024-02-18T00:46:46.385 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 shutdown_connections 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 --2- 172.21.15.59:0/164862601 >> v2:172.21.15.59:6800/3721891797 conn(0x7f1884062c90 0x7f1884065150 secure :-1 s=CLOSED pgs=3250 cs=0 l=1 rev1=1 rx=0x7f18940103c0 tx=0x7f189400b040).stop 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18980fd510 0x7f18980f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f18980f7400 0x7f18980f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 --2- 172.21.15.59:0/164862601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18980fcbb0 0x7f18980f8ea0 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.385+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 >> 172.21.15.59:0/164862601 conn(0x7f18980f2820 msgr2=0x7f18980f4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.386+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 shutdown_connections 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.386+0000 7f189fbe5700 1 -- 172.21.15.59:0/164862601 wait complete. 2024-02-18T00:46:46.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:46.386+0000 7f189fbe5700 1 librados: shutdown 2024-02-18T00:46:46.387 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-02-18T00:46:46.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.003792+0000 osd.4 (osd.4) 437 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:46:46.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.006206+0000 osd.4 (osd.4) 438 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.113352+0000 osd.7 (osd.7) 265 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.114293+0000 osd.7 (osd.7) 266 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.422694+0000 osd.6 (osd.6) 357 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.423922+0000 osd.6 (osd.6) 358 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:45.794622+0000 mon.a (mon.0) 3793 : audit [INF] from='client.? 172.21.15.59:0/164862601' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.893974+0000 osd.3 (osd.3) 433 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.895433+0000 osd.3 (osd.3) 434 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.003792+0000 osd.4 (osd.4) 437 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.006206+0000 osd.4 (osd.4) 438 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.113352+0000 osd.7 (osd.7) 265 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:46:46.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.114293+0000 osd.7 (osd.7) 266 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.422694+0000 osd.6 (osd.6) 357 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.423922+0000 osd.6 (osd.6) 358 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:45.794622+0000 mon.a (mon.0) 3793 : audit [INF] from='client.? 172.21.15.59:0/164862601' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.893974+0000 osd.3 (osd.3) 433 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.895433+0000 osd.3 (osd.3) 434 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.003792+0000 osd.4 (osd.4) 437 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.006206+0000 osd.4 (osd.4) 438 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.113352+0000 osd.7 (osd.7) 265 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.114293+0000 osd.7 (osd.7) 266 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:46:46.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.422694+0000 osd.6 (osd.6) 357 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:46:46.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.423922+0000 osd.6 (osd.6) 358 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:46:46.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:45.794622+0000 mon.a (mon.0) 3793 : audit [INF] from='client.? 172.21.15.59:0/164862601' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-02-18T00:46:46.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.893974+0000 osd.3 (osd.3) 433 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:46:46.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.895433+0000 osd.3 (osd.3) 434 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:46:47.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 1 1 2024-02-18T00:46:47.490 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:47.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.612139+0000 osd.5 (osd.5) 378 : cluster [DBG] 25.c scrub starts 2024-02-18T00:46:47.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.615046+0000 osd.5 (osd.5) 379 : cluster [DBG] 25.c scrub ok 2024-02-18T00:46:47.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.973203+0000 osd.4 (osd.4) 439 : cluster [DBG] 5.1 deep-scrub starts 2024-02-18T00:46:47.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:45.974330+0000 osd.4 (osd.4) 440 : cluster [DBG] 5.1 deep-scrub ok 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.036057+0000 osd.2 (osd.2) 339 : cluster [DBG] 25.6 deep-scrub starts 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.037546+0000 osd.2 (osd.2) 340 : cluster [DBG] 25.6 deep-scrub ok 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.043020+0000 mgr.y (mgr.24454) 710 : cluster [DBG] pgmap v1271: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.153261+0000 osd.7 (osd.7) 267 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.154471+0000 osd.7 (osd.7) 268 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.341466+0000 osd.0 (osd.0) 366 : cluster [DBG] 25.e scrub starts 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.342733+0000 osd.0 (osd.0) 367 : cluster [DBG] 25.e scrub ok 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:46.380405+0000 mon.a (mon.0) 3794 : audit [INF] from='client.? 172.21.15.59:0/164862601' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.380500+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.612139+0000 osd.5 (osd.5) 378 : cluster [DBG] 25.c scrub starts 2024-02-18T00:46:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.615046+0000 osd.5 (osd.5) 379 : cluster [DBG] 25.c scrub ok 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.973203+0000 osd.4 (osd.4) 439 : cluster [DBG] 5.1 deep-scrub starts 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:45.974330+0000 osd.4 (osd.4) 440 : cluster [DBG] 5.1 deep-scrub ok 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.036057+0000 osd.2 (osd.2) 339 : cluster [DBG] 25.6 deep-scrub starts 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.037546+0000 osd.2 (osd.2) 340 : cluster [DBG] 25.6 deep-scrub ok 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.043020+0000 mgr.y (mgr.24454) 710 : cluster [DBG] pgmap v1271: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.153261+0000 osd.7 (osd.7) 267 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.154471+0000 osd.7 (osd.7) 268 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.341466+0000 osd.0 (osd.0) 366 : cluster [DBG] 25.e scrub starts 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.342733+0000 osd.0 (osd.0) 367 : cluster [DBG] 25.e scrub ok 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:46.380405+0000 mon.a (mon.0) 3794 : audit [INF] from='client.? 172.21.15.59:0/164862601' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:46:47.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.380500+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-02-18T00:46:47.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.612139+0000 osd.5 (osd.5) 378 : cluster [DBG] 25.c scrub starts 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.615046+0000 osd.5 (osd.5) 379 : cluster [DBG] 25.c scrub ok 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.973203+0000 osd.4 (osd.4) 439 : cluster [DBG] 5.1 deep-scrub starts 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:45.974330+0000 osd.4 (osd.4) 440 : cluster [DBG] 5.1 deep-scrub ok 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.036057+0000 osd.2 (osd.2) 339 : cluster [DBG] 25.6 deep-scrub starts 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.037546+0000 osd.2 (osd.2) 340 : cluster [DBG] 25.6 deep-scrub ok 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.043020+0000 mgr.y (mgr.24454) 710 : cluster [DBG] pgmap v1271: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.153261+0000 osd.7 (osd.7) 267 : cluster [DBG] 6.15 scrub starts 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.154471+0000 osd.7 (osd.7) 268 : cluster [DBG] 6.15 scrub ok 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.341466+0000 osd.0 (osd.0) 366 : cluster [DBG] 25.e scrub starts 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.342733+0000 osd.0 (osd.0) 367 : cluster [DBG] 25.e scrub ok 2024-02-18T00:46:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:46.380405+0000 mon.a (mon.0) 3794 : audit [INF] from='client.? 172.21.15.59:0/164862601' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-02-18T00:46:47.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.380500+0000 mon.a (mon.0) 3795 : cluster [DBG] osdmap e821: 8 total, 8 up, 8 in 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.995452+0000 osd.2 (osd.2) 341 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:46.996658+0000 osd.2 (osd.2) 342 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:47.186881+0000 osd.7 (osd.7) 269 : cluster [DBG] 2.1c deep-scrub starts 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:47.187808+0000 osd.7 (osd.7) 270 : cluster [DBG] 2.1c deep-scrub ok 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.995452+0000 osd.2 (osd.2) 341 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:46.996658+0000 osd.2 (osd.2) 342 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:47.186881+0000 osd.7 (osd.7) 269 : cluster [DBG] 2.1c deep-scrub starts 2024-02-18T00:46:48.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:47.187808+0000 osd.7 (osd.7) 270 : cluster [DBG] 2.1c deep-scrub ok 2024-02-18T00:46:48.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.995452+0000 osd.2 (osd.2) 341 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:46:48.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:46.996658+0000 osd.2 (osd.2) 342 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:46:48.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:47.186881+0000 osd.7 (osd.7) 269 : cluster [DBG] 2.1c deep-scrub starts 2024-02-18T00:46:48.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:47.187808+0000 osd.7 (osd.7) 270 : cluster [DBG] 2.1c deep-scrub ok 2024-02-18T00:46:49.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.043625+0000 mgr.y (mgr.24454) 711 : cluster [DBG] pgmap v1273: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.219243+0000 osd.7 (osd.7) 271 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.220896+0000 osd.7 (osd.7) 272 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.465030+0000 osd.6 (osd.6) 359 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.465976+0000 osd.6 (osd.6) 360 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.043625+0000 mgr.y (mgr.24454) 711 : cluster [DBG] pgmap v1273: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.219243+0000 osd.7 (osd.7) 271 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.220896+0000 osd.7 (osd.7) 272 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:46:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.465030+0000 osd.6 (osd.6) 359 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:46:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.465976+0000 osd.6 (osd.6) 360 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:46:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.043625+0000 mgr.y (mgr.24454) 711 : cluster [DBG] pgmap v1273: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-02-18T00:46:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.219243+0000 osd.7 (osd.7) 271 : cluster [DBG] 6.5 deep-scrub starts 2024-02-18T00:46:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.220896+0000 osd.7 (osd.7) 272 : cluster [DBG] 6.5 deep-scrub ok 2024-02-18T00:46:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.465030+0000 osd.6 (osd.6) 359 : cluster [DBG] 5.2 scrub starts 2024-02-18T00:46:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.465976+0000 osd.6 (osd.6) 360 : cluster [DBG] 5.2 scrub ok 2024-02-18T00:46:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 -- 172.21.15.59:0/1426338160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fdbb0 msgr2=0x7f68300fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 --2- 172.21.15.59:0/1426338160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fdbb0 0x7f68300fdf90 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 rx=0x7f68180047b0 tx=0x7f6818009cb0).stop 2024-02-18T00:46:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 -- 172.21.15.59:0/1426338160 shutdown_connections 2024-02-18T00:46:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 --2- 172.21.15.59:0/1426338160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68300f8420 0x7f6830104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 --2- 172.21.15.59:0/1426338160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fdbb0 0x7f68300fdf90 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:49.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 --2- 172.21.15.59:0/1426338160 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68300fe510 0x7f68300f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 -- 172.21.15.59:0/1426338160 >> 172.21.15.59:0/1426338160 conn(0x7f68300f3800 msgr2=0x7f68300f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.758+0000 7f683508c700 1 -- 172.21.15.59:0/1426338160 shutdown_connections 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.759+0000 7f683508c700 1 -- 172.21.15.59:0/1426338160 wait complete. 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.759+0000 7f683508c700 1 librados: starting msgr at 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.759+0000 7f683508c700 1 librados: starting objecter 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.759+0000 7f683508c700 1 Processor -- start 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 -- start start 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 librados: setting wanted keys 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 librados: calling monclient init 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68300f8420 0x7f683006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:49.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68300fdbb0 0x7f683006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fe510 0x7f683006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830116580 con 0x7f68300fe510 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301166f0 con 0x7f68300fdbb0 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f683508c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68301168b0 con 0x7f68300f8420 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fe510 0x7f683006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682ed9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68300f8420 0x7f683006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fe510 0x7f683006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46806/0 (socket says 172.21.15.59:46806) 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 -- 172.21.15.59:0/837127001 learned_addr learned my addr 172.21.15.59:0/837127001 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682e59c700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68300fdbb0 0x7f683006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:49.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 -- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68300f8420 msgr2=0x7f683006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:49.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68300f8420 0x7f683006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:49.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 -- 172.21.15.59:0/837127001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68300fdbb0 msgr2=0x7f683006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:49.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68300fdbb0 0x7f683006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:49.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 -- 172.21.15.59:0/837127001 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6818005040 con 0x7f68300fe510 2024-02-18T00:46:49.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682ed9d700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68300f8420 0x7f683006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682e59c700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68300fdbb0 0x7f683006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:46:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f682f59e700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fe510 0x7f683006bab0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 rx=0x7f682800ec20 tx=0x7f682800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f6827fff700 10 client.?.objecter ms_handle_connect 0x7f68300fe510 2024-02-18T00:46:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.760+0000 7f6827fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:49.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.761+0000 7f6827fff700 1 -- 172.21.15.59:0/837127001 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828009950 con 0x7f68300fe510 2024-02-18T00:46:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.761+0000 7f6827fff700 1 -- 172.21.15.59:0/837127001 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f682800f040 con 0x7f68300fe510 2024-02-18T00:46:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.761+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6830116b30 con 0x7f68300fe510 2024-02-18T00:46:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.761+0000 7f6827fff700 1 -- 172.21.15.59:0/837127001 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800c6e0 con 0x7f68300fe510 2024-02-18T00:46:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.761+0000 7f683508c700 10 client.32130.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.761+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6830117050 con 0x7f68300fe510 2024-02-18T00:46:49.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.761+0000 7f683508c700 1 librados: init done 2024-02-18T00:46:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.762+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68301070d0 con 0x7f68300fe510 2024-02-18T00:46:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.762+0000 7f6827fff700 1 -- 172.21.15.59:0/837127001 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f68280092e0 con 0x7f68300fe510 2024-02-18T00:46:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f6827fff700 1 --2- 172.21.15.59:0/837127001 >> v2:172.21.15.59:6800/3721891797 conn(0x7f681c062c90 0x7f681c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f682ed9d700 1 --2- 172.21.15.59:0/837127001 >> v2:172.21.15.59:6800/3721891797 conn(0x7f681c062c90 0x7f681c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:49.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f6827fff700 1 -- 172.21.15.59:0/837127001 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(821..821 src has 418..821) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f682807d1f0 con 0x7f68300fe510 2024-02-18T00:46:49.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f6827fff700 10 client.32130.objecter ms_dispatch 0x7f6830001550 osd_map(821..821 src has 418..821) v4 2024-02-18T00:46:49.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f6827fff700 3 client.32130.objecter handle_osd_map got epochs [821,821] > 0 2024-02-18T00:46:49.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f6827fff700 3 client.32130.objecter handle_osd_map decoding full epoch 821 2024-02-18T00:46:49.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f6827fff700 20 client.32130.objecter dump_active .. 0 homeless 2024-02-18T00:46:49.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f6827fff700 10 client.32130.objecter ms_handle_connect 0x7f681c062c90 2024-02-18T00:46:49.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.763+0000 7f682ed9d700 1 --2- 172.21.15.59:0/837127001 >> v2:172.21.15.59:6800/3721891797 conn(0x7f681c062c90 0x7f681c065150 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 rx=0x7f6818005010 tx=0x7f6818007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:49.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.767+0000 7f6827fff700 1 -- 172.21.15.59:0/837127001 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6828050960 con 0x7f68300fe510 2024-02-18T00:46:49.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:49.954+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f683006c5b0 con 0x7f68300fe510 2024-02-18T00:46:50.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.396+0000 7f6827fff700 1 -- 172.21.15.59:0/837127001 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v822) v1 ==== 113+0+0 (secure 0 0 0) 0x7f6828022030 con 0x7f68300fe510 2024-02-18T00:46:50.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.398+0000 7f683508c700 10 librados: watch_flush enter 2024-02-18T00:46:50.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.398+0000 7f683508c700 10 librados: watch_flush exit 2024-02-18T00:46:50.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.398+0000 7f683508c700 20 client.32130.objecter shutdown clearing up homeless session... 2024-02-18T00:46:50.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.398+0000 7f683508c700 10 client.32130.objecter successfully canceled tick 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.398+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 >> v2:172.21.15.59:6800/3721891797 conn(0x7f681c062c90 msgr2=0x7f681c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.398+0000 7f683508c700 1 --2- 172.21.15.59:0/837127001 >> v2:172.21.15.59:6800/3721891797 conn(0x7f681c062c90 0x7f681c065150 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 rx=0x7f6818005010 tx=0x7f6818007f90).stop 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fe510 msgr2=0x7f683006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fe510 0x7f683006bab0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 rx=0x7f682800ec20 tx=0x7f682800b040).stop 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 shutdown_connections 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68300fe510 0x7f683006bab0 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68300f8420 0x7f683006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 --2- 172.21.15.59:0/837127001 >> v2:172.21.15.59:6800/3721891797 conn(0x7f681c062c90 0x7f681c065150 unknown :-1 s=CLOSED pgs=3251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 --2- 172.21.15.59:0/837127001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68300fdbb0 0x7f683006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 >> 172.21.15.59:0/837127001 conn(0x7f68300f3800 msgr2=0x7f68300f5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 shutdown_connections 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 -- 172.21.15.59:0/837127001 wait complete. 2024-02-18T00:46:50.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:50.399+0000 7f683508c700 1 librados: shutdown 2024-02-18T00:46:50.401 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.1 to 1 (10000) 2024-02-18T00:46:50.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.549469+0000 osd.5 (osd.5) 380 : cluster [DBG] 4.f scrub starts 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.552967+0000 osd.5 (osd.5) 381 : cluster [DBG] 4.f scrub ok 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.914308+0000 osd.2 (osd.2) 343 : cluster [DBG] 2.c scrub starts 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:48.915241+0000 osd.2 (osd.2) 344 : cluster [DBG] 2.c scrub ok 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:49.954880+0000 mon.a (mon.0) 3796 : audit [INF] from='client.? 172.21.15.59:0/837127001' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.549469+0000 osd.5 (osd.5) 380 : cluster [DBG] 4.f scrub starts 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.552967+0000 osd.5 (osd.5) 381 : cluster [DBG] 4.f scrub ok 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.914308+0000 osd.2 (osd.2) 343 : cluster [DBG] 2.c scrub starts 2024-02-18T00:46:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:48.915241+0000 osd.2 (osd.2) 344 : cluster [DBG] 2.c scrub ok 2024-02-18T00:46:50.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:49.954880+0000 mon.a (mon.0) 3796 : audit [INF] from='client.? 172.21.15.59:0/837127001' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:50.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.549469+0000 osd.5 (osd.5) 380 : cluster [DBG] 4.f scrub starts 2024-02-18T00:46:50.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.552967+0000 osd.5 (osd.5) 381 : cluster [DBG] 4.f scrub ok 2024-02-18T00:46:50.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.914308+0000 osd.2 (osd.2) 343 : cluster [DBG] 2.c scrub starts 2024-02-18T00:46:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:48.915241+0000 osd.2 (osd.2) 344 : cluster [DBG] 2.c scrub ok 2024-02-18T00:46:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:49.954880+0000 mon.a (mon.0) 3796 : audit [INF] from='client.? 172.21.15.59:0/837127001' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:51.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 1 1 2024-02-18T00:46:51.403 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:46:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:46:51.650 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:51.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:50.044280+0000 mgr.y (mgr.24454) 712 : cluster [DBG] pgmap v1274: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:46:51.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:50.395974+0000 mon.a (mon.0) 3797 : audit [INF] from='client.? 172.21.15.59:0/837127001' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:46:51.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:50.396059+0000 mon.a (mon.0) 3798 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-02-18T00:46:51.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:50.530578+0000 osd.5 (osd.5) 382 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:46:51.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:50.531687+0000 osd.5 (osd.5) 383 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:46:51.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:50.044280+0000 mgr.y (mgr.24454) 712 : cluster [DBG] pgmap v1274: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:50.395974+0000 mon.a (mon.0) 3797 : audit [INF] from='client.? 172.21.15.59:0/837127001' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:50.396059+0000 mon.a (mon.0) 3798 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:50.530578+0000 osd.5 (osd.5) 382 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:50.531687+0000 osd.5 (osd.5) 383 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:50.044280+0000 mgr.y (mgr.24454) 712 : cluster [DBG] pgmap v1274: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:50.395974+0000 mon.a (mon.0) 3797 : audit [INF] from='client.? 172.21.15.59:0/837127001' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:50.396059+0000 mon.a (mon.0) 3798 : cluster [DBG] osdmap e822: 8 total, 8 up, 8 in 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:50.530578+0000 osd.5 (osd.5) 382 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:46:51.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:50.531687+0000 osd.5 (osd.5) 383 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:46:52.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:46:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:46:52.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:46:52] "GET /metrics HTTP/1.1" 200 187513 "" "Prometheus/2.33.4" 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:50.884137+0000 osd.2 (osd.2) 345 : cluster [DBG] 5.a scrub starts 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:50.885255+0000 osd.2 (osd.2) 346 : cluster [DBG] 5.a scrub ok 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.376293+0000 osd.0 (osd.0) 368 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.377314+0000 osd.0 (osd.0) 369 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.402622+0000 osd.6 (osd.6) 361 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.404606+0000 osd.6 (osd.6) 362 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.945460+0000 osd.3 (osd.3) 435 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.947668+0000 osd.3 (osd.3) 436 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:46:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:50.884137+0000 osd.2 (osd.2) 345 : cluster [DBG] 5.a scrub starts 2024-02-18T00:46:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:50.885255+0000 osd.2 (osd.2) 346 : cluster [DBG] 5.a scrub ok 2024-02-18T00:46:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.376293+0000 osd.0 (osd.0) 368 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:46:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.377314+0000 osd.0 (osd.0) 369 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:46:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.402622+0000 osd.6 (osd.6) 361 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:46:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.404606+0000 osd.6 (osd.6) 362 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:46:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.945460+0000 osd.3 (osd.3) 435 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:46:52.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.947668+0000 osd.3 (osd.3) 436 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:46:52.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:50.884137+0000 osd.2 (osd.2) 345 : cluster [DBG] 5.a scrub starts 2024-02-18T00:46:52.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:50.885255+0000 osd.2 (osd.2) 346 : cluster [DBG] 5.a scrub ok 2024-02-18T00:46:52.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.376293+0000 osd.0 (osd.0) 368 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:46:52.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.377314+0000 osd.0 (osd.0) 369 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:46:52.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.402622+0000 osd.6 (osd.6) 361 : cluster [DBG] 4.5 scrub starts 2024-02-18T00:46:52.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.404606+0000 osd.6 (osd.6) 362 : cluster [DBG] 4.5 scrub ok 2024-02-18T00:46:52.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.945460+0000 osd.3 (osd.3) 435 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:46:52.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.947668+0000 osd.3 (osd.3) 436 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:51.689316+0000 mgr.y (mgr.24454) 713 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.911347+0000 osd.4 (osd.4) 441 : cluster [DBG] 4.e scrub starts 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:51.914808+0000 osd.4 (osd.4) 442 : cluster [DBG] 4.e scrub ok 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:52.044978+0000 mgr.y (mgr.24454) 714 : cluster [DBG] pgmap v1276: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:52.382397+0000 osd.6 (osd.6) 363 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:52.384248+0000 osd.6 (osd.6) 364 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:52.394210+0000 osd.0 (osd.0) 370 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:52.395347+0000 osd.0 (osd.0) 371 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:46:53.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:51.689316+0000 mgr.y (mgr.24454) 713 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.911347+0000 osd.4 (osd.4) 441 : cluster [DBG] 4.e scrub starts 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:51.914808+0000 osd.4 (osd.4) 442 : cluster [DBG] 4.e scrub ok 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:52.044978+0000 mgr.y (mgr.24454) 714 : cluster [DBG] pgmap v1276: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:52.382397+0000 osd.6 (osd.6) 363 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:52.384248+0000 osd.6 (osd.6) 364 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:52.394210+0000 osd.0 (osd.0) 370 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:52.395347+0000 osd.0 (osd.0) 371 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:51.689316+0000 mgr.y (mgr.24454) 713 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.911347+0000 osd.4 (osd.4) 441 : cluster [DBG] 4.e scrub starts 2024-02-18T00:46:53.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:51.914808+0000 osd.4 (osd.4) 442 : cluster [DBG] 4.e scrub ok 2024-02-18T00:46:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:52.044978+0000 mgr.y (mgr.24454) 714 : cluster [DBG] pgmap v1276: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:46:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:52.382397+0000 osd.6 (osd.6) 363 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:46:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:52.384248+0000 osd.6 (osd.6) 364 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:46:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:52.394210+0000 osd.0 (osd.0) 370 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:46:53.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:52.395347+0000 osd.0 (osd.0) 371 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:46:54.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 -- 172.21.15.59:0/1691645748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700fe510 msgr2=0x7f03700f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:54.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 --2- 172.21.15.59:0/1691645748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700fe510 0x7f03700f7d40 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 rx=0x7f0360004870 tx=0x7f0360009cb0).stop 2024-02-18T00:46:54.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 -- 172.21.15.59:0/1691645748 shutdown_connections 2024-02-18T00:46:54.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 --2- 172.21.15.59:0/1691645748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03700fdbb0 0x7f03700fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:54.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 --2- 172.21.15.59:0/1691645748 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700f8420 0x7f0370104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:54.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 --2- 172.21.15.59:0/1691645748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700fe510 0x7f03700f7d40 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:54.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 -- 172.21.15.59:0/1691645748 >> 172.21.15.59:0/1691645748 conn(0x7f03700f3800 msgr2=0x7f03700f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:54.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 -- 172.21.15.59:0/1691645748 shutdown_connections 2024-02-18T00:46:54.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.388+0000 7f037489a700 1 -- 172.21.15.59:0/1691645748 wait complete. 2024-02-18T00:46:54.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.389+0000 7f037489a700 1 librados: starting msgr at 2024-02-18T00:46:54.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.389+0000 7f037489a700 1 librados: starting objecter 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.389+0000 7f037489a700 1 Processor -- start 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.389+0000 7f037489a700 1 -- start start 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.389+0000 7f037489a700 1 librados: setting wanted keys 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.389+0000 7f037489a700 1 librados: calling monclient init 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f037489a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700f8420 0x7f037010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f037489a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700fdbb0 0x7f037010e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f037489a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03700fe510 0x7f0370108150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f037489a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0370108640 con 0x7f03700f8420 2024-02-18T00:46:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f037489a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03701087b0 con 0x7f03700fe510 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f037489a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0370108970 con 0x7f03700fdbb0 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700fdbb0 0x7f037010e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700fdbb0 0x7f037010e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37294/0 (socket says 172.21.15.59:37294) 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 -- 172.21.15.59:0/2031921845 learned_addr learned my addr 172.21.15.59:0/2031921845 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036e59c700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700f8420 0x7f037010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036ed9d700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03700fe510 0x7f0370108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 -- 172.21.15.59:0/2031921845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03700fe510 msgr2=0x7f0370108150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03700fe510 0x7f0370108150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 -- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700f8420 msgr2=0x7f037010dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700f8420 0x7f037010dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 -- 172.21.15.59:0/2031921845 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680097c0 con 0x7f03700fdbb0 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036e59c700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700f8420 0x7f037010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f036dd9b700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700fdbb0 0x7f037010e4a0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 rx=0x7f0360009eb0 tx=0x7f036000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f035f7fe700 10 client.?.objecter ms_handle_connect 0x7f03700fdbb0 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.390+0000 7f035f7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.391+0000 7f035f7fe700 1 -- 172.21.15.59:0/2031921845 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036000e070 con 0x7f03700fdbb0 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.391+0000 7f035f7fe700 1 -- 172.21.15.59:0/2031921845 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0360013da0 con 0x7f03700fdbb0 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.391+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0360005040 con 0x7f03700fdbb0 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.391+0000 7f035f7fe700 1 -- 172.21.15.59:0/2031921845 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360019070 con 0x7f03700fdbb0 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.391+0000 7f037489a700 10 client.41939.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.391+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0370108ee0 con 0x7f03700fdbb0 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.391+0000 7f037489a700 1 librados: init done 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.392+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037005ee80 con 0x7f03700fdbb0 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.392+0000 7f035f7fe700 1 -- 172.21.15.59:0/2031921845 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f036001e420 con 0x7f03700fdbb0 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.392+0000 7f035f7fe700 1 --2- 172.21.15.59:0/2031921845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0358062c90 0x7f0358065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f035f7fe700 1 -- 172.21.15.59:0/2031921845 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(823..823 src has 418..823) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f036007dc20 con 0x7f03700fdbb0 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f035f7fe700 10 client.41939.objecter ms_dispatch 0x7f0370001550 osd_map(823..823 src has 418..823) v4 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f035f7fe700 3 client.41939.objecter handle_osd_map got epochs [823,823] > 0 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f035f7fe700 3 client.41939.objecter handle_osd_map decoding full epoch 823 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f036e59c700 1 --2- 172.21.15.59:0/2031921845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0358062c90 0x7f0358065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f035f7fe700 20 client.41939.objecter dump_active .. 0 homeless 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f035f7fe700 10 client.41939.objecter ms_handle_connect 0x7f0358062c90 2024-02-18T00:46:54.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.393+0000 7f036e59c700 1 --2- 172.21.15.59:0/2031921845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0358062c90 0x7f0358065150 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 rx=0x7f03640041d0 tx=0x7f0364009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:54.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.397+0000 7f035f7fe700 1 -- 172.21.15.59:0/2031921845 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f03600512f0 con 0x7f03700fdbb0 2024-02-18T00:46:54.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:53.159930+0000 osd.7 (osd.7) 273 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:46:54.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:53.161856+0000 osd.7 (osd.7) 274 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:46:54.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:53.397663+0000 osd.6 (osd.6) 365 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:46:54.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:53.398644+0000 osd.6 (osd.6) 366 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:46:54.411 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:53.850414+0000 mon.a (mon.0) 3799 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-02-18T00:46:54.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:54.586+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) v1 -- 0x7f0370109630 con 0x7f03700fdbb0 2024-02-18T00:46:54.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:53.159930+0000 osd.7 (osd.7) 273 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:46:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:53.161856+0000 osd.7 (osd.7) 274 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:46:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:53.397663+0000 osd.6 (osd.6) 365 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:46:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:53.398644+0000 osd.6 (osd.6) 366 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:46:54.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:53.850414+0000 mon.a (mon.0) 3799 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-02-18T00:46:54.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:53.159930+0000 osd.7 (osd.7) 273 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:46:54.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:53.161856+0000 osd.7 (osd.7) 274 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:46:54.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:53.397663+0000 osd.6 (osd.6) 365 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:46:54.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:53.398644+0000 osd.6 (osd.6) 366 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:46:54.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:53.850414+0000 mon.a (mon.0) 3799 : cluster [DBG] osdmap e823: 8 total, 8 up, 8 in 2024-02-18T00:46:55.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.415+0000 7f035f7fe700 1 -- 172.21.15.59:0/2031921845 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v824) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0360023020 con 0x7f03700fdbb0 2024-02-18T00:46:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.418+0000 7f037489a700 10 librados: watch_flush enter 2024-02-18T00:46:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.418+0000 7f037489a700 10 librados: watch_flush exit 2024-02-18T00:46:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.418+0000 7f037489a700 20 client.41939.objecter shutdown clearing up homeless session... 2024-02-18T00:46:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.418+0000 7f037489a700 10 client.41939.objecter successfully canceled tick 2024-02-18T00:46:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.419+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0358062c90 msgr2=0x7f0358065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.419+0000 7f037489a700 1 --2- 172.21.15.59:0/2031921845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0358062c90 0x7f0358065150 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 rx=0x7f03640041d0 tx=0x7f0364009040).stop 2024-02-18T00:46:55.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.419+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700fdbb0 msgr2=0x7f037010e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.419+0000 7f037489a700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700fdbb0 0x7f037010e4a0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 rx=0x7f0360009eb0 tx=0x7f036000b040).stop 2024-02-18T00:46:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 shutdown_connections 2024-02-18T00:46:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 --2- 172.21.15.59:0/2031921845 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0358062c90 0x7f0358065150 unknown :-1 s=CLOSED pgs=3252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:55.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03700fe510 0x7f0370108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03700f8420 0x7f037010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 --2- 172.21.15.59:0/2031921845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03700fdbb0 0x7f037010e4a0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 >> 172.21.15.59:0/2031921845 conn(0x7f03700f3800 msgr2=0x7f03700fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 shutdown_connections 2024-02-18T00:46:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 -- 172.21.15.59:0/2031921845 wait complete. 2024-02-18T00:46:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:55.422+0000 7f037489a700 1 librados: shutdown 2024-02-18T00:46:55.423 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-02-18T00:46:55.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:53.867235+0000 osd.4 (osd.4) 443 : cluster [DBG] 2.f scrub starts 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:53.870676+0000 osd.4 (osd.4) 444 : cluster [DBG] 2.f scrub ok 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.045455+0000 mgr.y (mgr.24454) 715 : cluster [DBG] pgmap v1278: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 667 B/s rd, 0 op/s 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.133675+0000 osd.7 (osd.7) 275 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.134667+0000 osd.7 (osd.7) 276 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.318313+0000 osd.0 (osd.0) 372 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.319897+0000 osd.0 (osd.0) 373 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.376369+0000 osd.6 (osd.6) 367 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.377271+0000 osd.6 (osd.6) 368 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:46:55.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:54.586666+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.59:0/2031921845' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:54.586992+0000 mon.a (mon.0) 3800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.760129+0000 osd.1 (osd.1) 407 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.761396+0000 osd.1 (osd.1) 408 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:53.867235+0000 osd.4 (osd.4) 443 : cluster [DBG] 2.f scrub starts 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:53.870676+0000 osd.4 (osd.4) 444 : cluster [DBG] 2.f scrub ok 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.045455+0000 mgr.y (mgr.24454) 715 : cluster [DBG] pgmap v1278: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 667 B/s rd, 0 op/s 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.133675+0000 osd.7 (osd.7) 275 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.134667+0000 osd.7 (osd.7) 276 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.318313+0000 osd.0 (osd.0) 372 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:46:55.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.319897+0000 osd.0 (osd.0) 373 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.376369+0000 osd.6 (osd.6) 367 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.377271+0000 osd.6 (osd.6) 368 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:54.586666+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.59:0/2031921845' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:54.586992+0000 mon.a (mon.0) 3800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.760129+0000 osd.1 (osd.1) 407 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.761396+0000 osd.1 (osd.1) 408 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:53.867235+0000 osd.4 (osd.4) 443 : cluster [DBG] 2.f scrub starts 2024-02-18T00:46:55.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:53.870676+0000 osd.4 (osd.4) 444 : cluster [DBG] 2.f scrub ok 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.045455+0000 mgr.y (mgr.24454) 715 : cluster [DBG] pgmap v1278: 174 pgs: 174 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 667 B/s rd, 0 op/s 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.133675+0000 osd.7 (osd.7) 275 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.134667+0000 osd.7 (osd.7) 276 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.318313+0000 osd.0 (osd.0) 372 : cluster [DBG] 4.16 scrub starts 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.319897+0000 osd.0 (osd.0) 373 : cluster [DBG] 4.16 scrub ok 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.376369+0000 osd.6 (osd.6) 367 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.377271+0000 osd.6 (osd.6) 368 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:54.586666+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.59:0/2031921845' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:54.586992+0000 mon.a (mon.0) 3800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.760129+0000 osd.1 (osd.1) 407 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:46:55.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.761396+0000 osd.1 (osd.1) 408 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:46:56.280 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 2 1 2024-02-18T00:46:56.568 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.549467+0000 osd.5 (osd.5) 384 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:54.552939+0000 osd.5 (osd.5) 385 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:55.125661+0000 osd.7 (osd.7) 277 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:55.126664+0000 osd.7 (osd.7) 278 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:55.414980+0000 mon.a (mon.0) 3801 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:55.415039+0000 mon.a (mon.0) 3802 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.046337+0000 mgr.y (mgr.24454) 716 : cluster [DBG] pgmap v1280: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:56.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.549467+0000 osd.5 (osd.5) 384 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:54.552939+0000 osd.5 (osd.5) 385 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:55.125661+0000 osd.7 (osd.7) 277 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:55.126664+0000 osd.7 (osd.7) 278 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:55.414980+0000 mon.a (mon.0) 3801 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:55.415039+0000 mon.a (mon.0) 3802 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.046337+0000 mgr.y (mgr.24454) 716 : cluster [DBG] pgmap v1280: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.549467+0000 osd.5 (osd.5) 384 : cluster [DBG] 4.15 scrub starts 2024-02-18T00:46:56.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:54.552939+0000 osd.5 (osd.5) 385 : cluster [DBG] 4.15 scrub ok 2024-02-18T00:46:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:55.125661+0000 osd.7 (osd.7) 277 : cluster [DBG] 2.26 scrub starts 2024-02-18T00:46:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:55.126664+0000 osd.7 (osd.7) 278 : cluster [DBG] 2.26 scrub ok 2024-02-18T00:46:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:55.414980+0000 mon.a (mon.0) 3801 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-02-18T00:46:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:55.415039+0000 mon.a (mon.0) 3802 : cluster [DBG] osdmap e824: 8 total, 8 up, 8 in 2024-02-18T00:46:56.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.046337+0000 mgr.y (mgr.24454) 716 : cluster [DBG] pgmap v1280: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:57.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:55.880721+0000 osd.4 (osd.4) 445 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:55.884495+0000 osd.4 (osd.4) 446 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.370036+0000 osd.6 (osd.6) 369 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.371033+0000 osd.6 (osd.6) 370 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.758541+0000 osd.1 (osd.1) 409 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.759361+0000 osd.1 (osd.1) 410 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:55.880721+0000 osd.4 (osd.4) 445 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:55.884495+0000 osd.4 (osd.4) 446 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.370036+0000 osd.6 (osd.6) 369 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:46:57.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.371033+0000 osd.6 (osd.6) 370 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.758541+0000 osd.1 (osd.1) 409 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.759361+0000 osd.1 (osd.1) 410 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:55.880721+0000 osd.4 (osd.4) 445 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:55.884495+0000 osd.4 (osd.4) 446 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.370036+0000 osd.6 (osd.6) 369 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.371033+0000 osd.6 (osd.6) 370 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.758541+0000 osd.1 (osd.1) 409 : cluster [DBG] 2.15 scrub starts 2024-02-18T00:46:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.759361+0000 osd.1 (osd.1) 410 : cluster [DBG] 2.15 scrub ok 2024-02-18T00:46:58.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 -- 172.21.15.59:0/962911293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980fe510 msgr2=0x7f0f980f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 --2- 172.21.15.59:0/962911293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980fe510 0x7f0f980f7d40 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 rx=0x7f0f8c004870 tx=0x7f0f8c009cb0).stop 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 -- 172.21.15.59:0/962911293 shutdown_connections 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 --2- 172.21.15.59:0/962911293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f980f8420 0x7f0f98104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 --2- 172.21.15.59:0/962911293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 0x7f0f980fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 --2- 172.21.15.59:0/962911293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980fe510 0x7f0f980f7d40 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 -- 172.21.15.59:0/962911293 >> 172.21.15.59:0/962911293 conn(0x7f0f980f3800 msgr2=0x7f0f980f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 -- 172.21.15.59:0/962911293 shutdown_connections 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 -- 172.21.15.59:0/962911293 wait complete. 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 librados: starting msgr at 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.696+0000 7f0f9c935700 1 librados: starting objecter 2024-02-18T00:46:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 Processor -- start 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 -- start start 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 librados: setting wanted keys 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 librados: calling monclient init 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980f8420 0x7f0f9806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 0x7f0f9806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f980fe510 0x7f0f9806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98116530 con 0x7f0f980f8420 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f981166a0 con 0x7f0f980fe510 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9c935700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98116860 con 0x7f0f980fdbb0 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f95d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 0x7f0f9806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f95d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 0x7f0f9806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37298/0 (socket says 172.21.15.59:37298) 2024-02-18T00:46:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f95d9b700 1 -- 172.21.15.59:0/604066533 learned_addr learned my addr 172.21.15.59:0/604066533 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:46:58.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.697+0000 7f0f9659c700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980f8420 0x7f0f9806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:58.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f96d9d700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f980fe510 0x7f0f9806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:58.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f95d9b700 1 -- 172.21.15.59:0/604066533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f980fe510 msgr2=0x7f0f9806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:58.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f95d9b700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f980fe510 0x7f0f9806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f95d9b700 1 -- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980f8420 msgr2=0x7f0f9806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f95d9b700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980f8420 0x7f0f9806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f95d9b700 1 -- 172.21.15.59:0/604066533 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c005040 con 0x7f0f980fdbb0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f96d9d700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f980fe510 0x7f0f9806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f95d9b700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 0x7f0f9806b5c0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7f0f8c009eb0 tx=0x7f0f8c00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f877fe700 10 client.?.objecter ms_handle_connect 0x7f0f980fdbb0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f877fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f877fe700 1 -- 172.21.15.59:0/604066533 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00e030 con 0x7f0f980fdbb0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f98116ae0 con 0x7f0f980fdbb0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f877fe700 1 -- 172.21.15.59:0/604066533 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f8c014e80 con 0x7f0f980fdbb0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f877fe700 1 -- 172.21.15.59:0/604066533 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c019070 con 0x7f0f980fdbb0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f9c935700 10 client.41945.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f98116fc0 con 0x7f0f980fdbb0 2024-02-18T00:46:58.703 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.698+0000 7f0f9c935700 1 librados: init done 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.699+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9805ee80 con 0x7f0f980fdbb0 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 1 -- 172.21.15.59:0/604066533 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0f8c01e420 con 0x7f0f980fdbb0 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 1 --2- 172.21.15.59:0/604066533 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f80062c90 0x7f0f80065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 1 -- 172.21.15.59:0/604066533 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(824..824 src has 418..824) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f0f8c07e280 con 0x7f0f980fdbb0 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 10 client.41945.objecter ms_dispatch 0x7f0f98001550 osd_map(824..824 src has 418..824) v4 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 3 client.41945.objecter handle_osd_map got epochs [824,824] > 0 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 3 client.41945.objecter handle_osd_map decoding full epoch 824 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f9659c700 1 --2- 172.21.15.59:0/604066533 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f80062c90 0x7f0f80065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 20 client.41945.objecter dump_active .. 0 homeless 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f877fe700 10 client.41945.objecter ms_handle_connect 0x7f0f80062c90 2024-02-18T00:46:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.700+0000 7f0f9659c700 1 --2- 172.21.15.59:0/604066533 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f80062c90 0x7f0f80065150 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 rx=0x7f0f880092f0 tx=0x7f0f8800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:46:58.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.704+0000 7f0f877fe700 1 -- 172.21.15.59:0/604066533 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0f8c051950 con 0x7f0f980fdbb0 2024-02-18T00:46:58.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.845878+0000 osd.4 (osd.4) 447 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:46:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.847007+0000 osd.4 (osd.4) 448 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:46:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.951458+0000 osd.2 (osd.2) 347 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:46:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:56.952583+0000 osd.2 (osd.2) 348 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:46:58.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.046918+0000 mgr.y (mgr.24454) 717 : cluster [DBG] pgmap v1281: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:58.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.845878+0000 osd.4 (osd.4) 447 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:46:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.847007+0000 osd.4 (osd.4) 448 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:46:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.951458+0000 osd.2 (osd.2) 347 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:46:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:56.952583+0000 osd.2 (osd.2) 348 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:46:58.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.046918+0000 mgr.y (mgr.24454) 717 : cluster [DBG] pgmap v1281: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:58.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.845878+0000 osd.4 (osd.4) 447 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:46:58.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.847007+0000 osd.4 (osd.4) 448 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:46:58.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.951458+0000 osd.2 (osd.2) 347 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:46:58.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:56.952583+0000 osd.2 (osd.2) 348 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:46:58.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.046918+0000 mgr.y (mgr.24454) 717 : cluster [DBG] pgmap v1281: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:46:58.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:58.894+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) v1 -- 0x7f0f9805fa60 con 0x7f0f980fdbb0 2024-02-18T00:46:59.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.858+0000 7f0f877fe700 1 -- 172.21.15.59:0/604066533 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v826) v1 ==== 113+0+0 (secure 0 0 0) 0x7f0f8c051540 con 0x7f0f980fdbb0 2024-02-18T00:46:59.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 10 librados: watch_flush enter 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 10 librados: watch_flush exit 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 20 client.41945.objecter shutdown clearing up homeless session... 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 10 client.41945.objecter successfully canceled tick 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f80062c90 msgr2=0x7f0f80065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 --2- 172.21.15.59:0/604066533 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f80062c90 0x7f0f80065150 secure :-1 s=READY pgs=3253 cs=0 l=1 rev1=1 rx=0x7f0f880092f0 tx=0x7f0f8800b040).stop 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 msgr2=0x7f0f9806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 0x7f0f9806b5c0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 rx=0x7f0f8c009eb0 tx=0x7f0f8c00b040).stop 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 shutdown_connections 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 --2- 172.21.15.59:0/604066533 >> v2:172.21.15.59:6800/3721891797 conn(0x7f0f80062c90 0x7f0f80065150 unknown :-1 s=CLOSED pgs=3253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f980fe510 0x7f0f9806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f980f8420 0x7f0f9806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 --2- 172.21.15.59:0/604066533 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f980fdbb0 0x7f0f9806b5c0 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:46:59.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 >> 172.21.15.59:0/604066533 conn(0x7f0f980f3800 msgr2=0x7f0f980fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:46:59.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 shutdown_connections 2024-02-18T00:46:59.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 -- 172.21.15.59:0/604066533 wait complete. 2024-02-18T00:46:59.864 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:46:59.861+0000 7f0f9c935700 1 librados: shutdown 2024-02-18T00:46:59.864 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.2 to 1 (10000) 2024-02-18T00:47:00.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.224574+0000 osd.0 (osd.0) 374 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.225561+0000 osd.0 (osd.0) 375 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.347390+0000 osd.6 (osd.6) 371 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.348489+0000 osd.6 (osd.6) 372 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.770180+0000 osd.1 (osd.1) 411 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.773639+0000 osd.1 (osd.1) 412 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.850171+0000 mon.a (mon.0) 3803 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:58.896331+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.59:0/604066533' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:58.896635+0000 mon.a (mon.0) 3804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:59.749658+0000 osd.4 (osd.4) 449 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:47:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:59.753241+0000 osd.4 (osd.4) 450 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.224574+0000 osd.0 (osd.0) 374 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.225561+0000 osd.0 (osd.0) 375 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.347390+0000 osd.6 (osd.6) 371 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.348489+0000 osd.6 (osd.6) 372 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.770180+0000 osd.1 (osd.1) 411 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.773639+0000 osd.1 (osd.1) 412 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.850171+0000 mon.a (mon.0) 3803 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:58.896331+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.59:0/604066533' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:58.896635+0000 mon.a (mon.0) 3804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:59.749658+0000 osd.4 (osd.4) 449 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:47:00.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:46:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:59.753241+0000 osd.4 (osd.4) 450 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.224574+0000 osd.0 (osd.0) 374 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.225561+0000 osd.0 (osd.0) 375 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.347390+0000 osd.6 (osd.6) 371 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.348489+0000 osd.6 (osd.6) 372 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.770180+0000 osd.1 (osd.1) 411 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.773639+0000 osd.1 (osd.1) 412 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.850171+0000 mon.a (mon.0) 3803 : cluster [DBG] osdmap e825: 8 total, 8 up, 8 in 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:58.896331+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.59:0/604066533' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:58.896635+0000 mon.a (mon.0) 3804 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:59.749658+0000 osd.4 (osd.4) 449 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:47:00.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:46:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:59.753241+0000 osd.4 (osd.4) 450 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:47:00.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 2 1 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.960835+0000 osd.2 (osd.2) 349 : cluster [DBG] 2.e scrub starts 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:58.962340+0000 osd.2 (osd.2) 350 : cluster [DBG] 2.e scrub ok 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:46:59.859019+0000 mon.a (mon.0) 3805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:59.859123+0000 mon.a (mon.0) 3806 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:59.976960+0000 osd.3 (osd.3) 437 : cluster [DBG] 4.23 scrub starts 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:46:59.979164+0000 osd.3 (osd.3) 438 : cluster [DBG] 4.23 scrub ok 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:00.047366+0000 mgr.y (mgr.24454) 718 : cluster [DBG] pgmap v1284: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:47:01.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.960835+0000 osd.2 (osd.2) 349 : cluster [DBG] 2.e scrub starts 2024-02-18T00:47:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:58.962340+0000 osd.2 (osd.2) 350 : cluster [DBG] 2.e scrub ok 2024-02-18T00:47:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:46:59.859019+0000 mon.a (mon.0) 3805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:47:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:59.859123+0000 mon.a (mon.0) 3806 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-02-18T00:47:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:59.976960+0000 osd.3 (osd.3) 437 : cluster [DBG] 4.23 scrub starts 2024-02-18T00:47:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:46:59.979164+0000 osd.3 (osd.3) 438 : cluster [DBG] 4.23 scrub ok 2024-02-18T00:47:01.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:00.047366+0000 mgr.y (mgr.24454) 718 : cluster [DBG] pgmap v1284: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:47:01.024 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:01.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:47:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.960835+0000 osd.2 (osd.2) 349 : cluster [DBG] 2.e scrub starts 2024-02-18T00:47:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:58.962340+0000 osd.2 (osd.2) 350 : cluster [DBG] 2.e scrub ok 2024-02-18T00:47:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:46:59.859019+0000 mon.a (mon.0) 3805 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:47:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:59.859123+0000 mon.a (mon.0) 3806 : cluster [DBG] osdmap e826: 8 total, 8 up, 8 in 2024-02-18T00:47:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:59.976960+0000 osd.3 (osd.3) 437 : cluster [DBG] 4.23 scrub starts 2024-02-18T00:47:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:46:59.979164+0000 osd.3 (osd.3) 438 : cluster [DBG] 4.23 scrub ok 2024-02-18T00:47:01.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:00.047366+0000 mgr.y (mgr.24454) 718 : cluster [DBG] pgmap v1284: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:47:02.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:47:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:00.284359+0000 osd.6 (osd.6) 373 : cluster [DBG] 25.d scrub starts 2024-02-18T00:47:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:00.286059+0000 osd.6 (osd.6) 374 : cluster [DBG] 25.d scrub ok 2024-02-18T00:47:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:00.959617+0000 osd.3 (osd.3) 439 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:47:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:00.962097+0000 osd.3 (osd.3) 440 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:47:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:00.986170+0000 osd.2 (osd.2) 351 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:47:02.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:00.987068+0000 osd.2 (osd.2) 352 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:47:02.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:00.284359+0000 osd.6 (osd.6) 373 : cluster [DBG] 25.d scrub starts 2024-02-18T00:47:02.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:00.286059+0000 osd.6 (osd.6) 374 : cluster [DBG] 25.d scrub ok 2024-02-18T00:47:02.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:00.959617+0000 osd.3 (osd.3) 439 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:47:02.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:00.962097+0000 osd.3 (osd.3) 440 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:47:02.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:00.986170+0000 osd.2 (osd.2) 351 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:47:02.146 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:00.987068+0000 osd.2 (osd.2) 352 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:47:02.146 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:00.284359+0000 osd.6 (osd.6) 373 : cluster [DBG] 25.d scrub starts 2024-02-18T00:47:02.146 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:00.286059+0000 osd.6 (osd.6) 374 : cluster [DBG] 25.d scrub ok 2024-02-18T00:47:02.146 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:00.959617+0000 osd.3 (osd.3) 439 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:47:02.146 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:00.962097+0000 osd.3 (osd.3) 440 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:47:02.146 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:00.986170+0000 osd.2 (osd.2) 351 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:47:02.146 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:00.987068+0000 osd.2 (osd.2) 352 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:47:02.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:02] "GET /metrics HTTP/1.1" 200 187520 "" "Prometheus/2.33.4" 2024-02-18T00:47:03.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 -- 172.21.15.59:0/3505122784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec102270 msgr2=0x7f49ec104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 --2- 172.21.15.59:0/3505122784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec102270 0x7f49ec104660 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 rx=0x7f49e4009be0 tx=0x7f49e4004d10).stop 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 -- 172.21.15.59:0/3505122784 shutdown_connections 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 --2- 172.21.15.59:0/3505122784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec0f8d10 0x7f49ec0f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 --2- 172.21.15.59:0/3505122784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec102270 0x7f49ec104660 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 --2- 172.21.15.59:0/3505122784 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49ec0f9670 0x7f49ec101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 -- 172.21.15.59:0/3505122784 >> 172.21.15.59:0/3505122784 conn(0x7f49ec0f2820 msgr2=0x7f49ec0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 -- 172.21.15.59:0/3505122784 shutdown_connections 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 -- 172.21.15.59:0/3505122784 wait complete. 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 librados: starting msgr at 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 librados: starting objecter 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.173+0000 7f49f44a1700 1 Processor -- start 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 -- start start 2024-02-18T00:47:03.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 librados: setting wanted keys 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 librados: calling monclient init 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec0f8d10 0x7f49ec10dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49ec0f9670 0x7f49ec10e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec102270 0x7f49ec108140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec108630 con 0x7f49ec0f8d10 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec1087a0 con 0x7f49ec102270 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f44a1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec108960 con 0x7f49ec0f9670 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec0f8d10 0x7f49ec10dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f1a3c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49ec0f9670 0x7f49ec10e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec0f8d10 0x7f49ec10dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48518/0 (socket says 172.21.15.59:48518) 2024-02-18T00:47:03.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 -- 172.21.15.59:0/2203879172 learned_addr learned my addr 172.21.15.59:0/2203879172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f2a3e700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec102270 0x7f49ec108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 -- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49ec0f9670 msgr2=0x7f49ec10e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49ec0f9670 0x7f49ec10e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 -- 172.21.15.59:0/2203879172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec102270 msgr2=0x7f49ec108140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec102270 0x7f49ec108140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 -- 172.21.15.59:0/2203879172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc005040 con 0x7f49ec0f8d10 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f2a3e700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec102270 0x7f49ec108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f1a3c700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49ec0f9670 0x7f49ec10e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49f223d700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec0f8d10 0x7f49ec10dfa0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 rx=0x7f49e80041d0 tx=0x7f49e8009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49e37fe700 10 client.?.objecter ms_handle_connect 0x7f49ec0f8d10 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49e37fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:03.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.174+0000 7f49e37fe700 1 -- 172.21.15.59:0/2203879172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8007b40 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.175+0000 7f49e37fe700 1 -- 172.21.15.59:0/2203879172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49e800f040 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.175+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e40097c0 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.175+0000 7f49e37fe700 1 -- 172.21.15.59:0/2203879172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8014810 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.175+0000 7f49f44a1700 10 client.32148.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.175+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec108fa0 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.175+0000 7f49f44a1700 1 librados: init done 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.176+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec05fa60 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.176+0000 7f49e37fe700 1 -- 172.21.15.59:0/2203879172 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f49e8009110 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.176+0000 7f49e37fe700 1 --2- 172.21.15.59:0/2203879172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f49d8062c90 0x7f49d8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.176+0000 7f49e37fe700 1 -- 172.21.15.59:0/2203879172 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(826..826 src has 418..826) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f49e807d0c0 con 0x7f49ec0f8d10 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.177+0000 7f49e37fe700 10 client.32148.objecter ms_dispatch 0x7f49ec001550 osd_map(826..826 src has 418..826) v4 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.177+0000 7f49e37fe700 3 client.32148.objecter handle_osd_map got epochs [826,826] > 0 2024-02-18T00:47:03.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.177+0000 7f49e37fe700 3 client.32148.objecter handle_osd_map decoding full epoch 826 2024-02-18T00:47:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.177+0000 7f49f1a3c700 1 --2- 172.21.15.59:0/2203879172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f49d8062c90 0x7f49d8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.177+0000 7f49e37fe700 20 client.32148.objecter dump_active .. 0 homeless 2024-02-18T00:47:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.177+0000 7f49e37fe700 10 client.32148.objecter ms_handle_connect 0x7f49d8062c90 2024-02-18T00:47:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.177+0000 7f49f1a3c700 1 --2- 172.21.15.59:0/2203879172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f49d8062c90 0x7f49d8065150 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 rx=0x7f49ec1098e0 tx=0x7f49dc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:03.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.181+0000 7f49e37fe700 1 -- 172.21.15.59:0/2203879172 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f49e8050790 con 0x7f49ec0f8d10 2024-02-18T00:47:03.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:01.266841+0000 osd.0 (osd.0) 376 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:47:03.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:01.270146+0000 osd.0 (osd.0) 377 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:47:03.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:01.568649+0000 osd.5 (osd.5) 386 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:47:03.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:01.571534+0000 osd.5 (osd.5) 387 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:47:03.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:01.700269+0000 mgr.y (mgr.24454) 719 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:03.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:02.048173+0000 mgr.y (mgr.24454) 720 : cluster [DBG] pgmap v1285: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:47:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:01.266841+0000 osd.0 (osd.0) 376 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:47:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:01.270146+0000 osd.0 (osd.0) 377 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:47:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:01.568649+0000 osd.5 (osd.5) 386 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:47:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:01.571534+0000 osd.5 (osd.5) 387 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:47:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:01.700269+0000 mgr.y (mgr.24454) 719 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:03.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:02.048173+0000 mgr.y (mgr.24454) 720 : cluster [DBG] pgmap v1285: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:47:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:01.266841+0000 osd.0 (osd.0) 376 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:47:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:01.270146+0000 osd.0 (osd.0) 377 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:47:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:01.568649+0000 osd.5 (osd.5) 386 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:47:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:01.571534+0000 osd.5 (osd.5) 387 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:47:03.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:01.700269+0000 mgr.y (mgr.24454) 719 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:02.048173+0000 mgr.y (mgr.24454) 720 : cluster [DBG] pgmap v1285: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T00:47:03.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.367+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) v1 -- 0x7f49ec109620 con 0x7f49ec0f8d10 2024-02-18T00:47:03.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.852+0000 7f49e37fe700 1 -- 172.21.15.59:0/2203879172 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v827) v1 ==== 133+0+0 (secure 0 0 0) 0x7f49e8050380 con 0x7f49ec0f8d10 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 10 librados: watch_flush enter 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 10 librados: watch_flush exit 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 20 client.32148.objecter shutdown clearing up homeless session... 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 10 client.32148.objecter successfully canceled tick 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f49d8062c90 msgr2=0x7f49d8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 --2- 172.21.15.59:0/2203879172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f49d8062c90 0x7f49d8065150 secure :-1 s=READY pgs=3254 cs=0 l=1 rev1=1 rx=0x7f49ec1098e0 tx=0x7f49dc00b040).stop 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec0f8d10 msgr2=0x7f49ec10dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec0f8d10 0x7f49ec10dfa0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 rx=0x7f49e80041d0 tx=0x7f49e8009f60).stop 2024-02-18T00:47:03.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 shutdown_connections 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 --2- 172.21.15.59:0/2203879172 >> v2:172.21.15.59:6800/3721891797 conn(0x7f49d8062c90 0x7f49d8065150 unknown :-1 s=CLOSED pgs=3254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49ec102270 0x7f49ec108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ec0f8d10 0x7f49ec10dfa0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 --2- 172.21.15.59:0/2203879172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49ec0f9670 0x7f49ec10e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 >> 172.21.15.59:0/2203879172 conn(0x7f49ec0f2820 msgr2=0x7f49ec0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 shutdown_connections 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.855+0000 7f49f44a1700 1 -- 172.21.15.59:0/2203879172 wait complete. 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:03.856+0000 7f49f44a1700 1 librados: shutdown 2024-02-18T00:47:03.858 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-02-18T00:47:04.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:02.345029+0000 osd.6 (osd.6) 375 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:02.346833+0000 osd.6 (osd.6) 376 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:02.904489+0000 osd.3 (osd.3) 441 : cluster [DBG] 4.29 deep-scrub starts 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:02.906194+0000 osd.3 (osd.3) 442 : cluster [DBG] 4.29 deep-scrub ok 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.145208+0000 osd.7 (osd.7) 279 : cluster [DBG] 2.b scrub starts 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.146156+0000 osd.7 (osd.7) 280 : cluster [DBG] 2.b scrub ok 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:03.368891+0000 mon.a (mon.0) 3807 : audit [INF] from='client.? 172.21.15.59:0/2203879172' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.669250+0000 osd.4 (osd.4) 451 : cluster [DBG] 5.1c deep-scrub starts 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.672459+0000 osd.4 (osd.4) 452 : cluster [DBG] 5.1c deep-scrub ok 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:03.853445+0000 mon.a (mon.0) 3808 : audit [INF] from='client.? 172.21.15.59:0/2203879172' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:47:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.853533+0000 mon.a (mon.0) 3809 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:02.345029+0000 osd.6 (osd.6) 375 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:02.346833+0000 osd.6 (osd.6) 376 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:02.904489+0000 osd.3 (osd.3) 441 : cluster [DBG] 4.29 deep-scrub starts 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:02.906194+0000 osd.3 (osd.3) 442 : cluster [DBG] 4.29 deep-scrub ok 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.145208+0000 osd.7 (osd.7) 279 : cluster [DBG] 2.b scrub starts 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.146156+0000 osd.7 (osd.7) 280 : cluster [DBG] 2.b scrub ok 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:03.368891+0000 mon.a (mon.0) 3807 : audit [INF] from='client.? 172.21.15.59:0/2203879172' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.669250+0000 osd.4 (osd.4) 451 : cluster [DBG] 5.1c deep-scrub starts 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.672459+0000 osd.4 (osd.4) 452 : cluster [DBG] 5.1c deep-scrub ok 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:03.853445+0000 mon.a (mon.0) 3808 : audit [INF] from='client.? 172.21.15.59:0/2203879172' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:47:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.853533+0000 mon.a (mon.0) 3809 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:02.345029+0000 osd.6 (osd.6) 375 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:02.346833+0000 osd.6 (osd.6) 376 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:02.904489+0000 osd.3 (osd.3) 441 : cluster [DBG] 4.29 deep-scrub starts 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:02.906194+0000 osd.3 (osd.3) 442 : cluster [DBG] 4.29 deep-scrub ok 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.145208+0000 osd.7 (osd.7) 279 : cluster [DBG] 2.b scrub starts 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.146156+0000 osd.7 (osd.7) 280 : cluster [DBG] 2.b scrub ok 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:03.368891+0000 mon.a (mon.0) 3807 : audit [INF] from='client.? 172.21.15.59:0/2203879172' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.669250+0000 osd.4 (osd.4) 451 : cluster [DBG] 5.1c deep-scrub starts 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.672459+0000 osd.4 (osd.4) 452 : cluster [DBG] 5.1c deep-scrub ok 2024-02-18T00:47:04.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:03.853445+0000 mon.a (mon.0) 3808 : audit [INF] from='client.? 172.21.15.59:0/2203879172' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-02-18T00:47:04.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.853533+0000 mon.a (mon.0) 3809 : cluster [DBG] osdmap e827: 8 total, 8 up, 8 in 2024-02-18T00:47:04.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 3 1 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.293197+0000 osd.0 (osd.0) 378 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.296325+0000 osd.0 (osd.0) 379 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.543155+0000 osd.5 (osd.5) 388 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.544026+0000 osd.5 (osd.5) 389 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.602403+0000 osd.6 (osd.6) 377 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.602766+0000 osd.6 (osd.6) 378 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.930969+0000 osd.2 (osd.2) 353 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:47:05.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:03.932023+0000 osd.2 (osd.2) 354 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:04.048734+0000 mgr.y (mgr.24454) 721 : cluster [DBG] pgmap v1287: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:04.666765+0000 osd.7 (osd.7) 281 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:04.666896+0000 osd.7 (osd.7) 282 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.293197+0000 osd.0 (osd.0) 378 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.296325+0000 osd.0 (osd.0) 379 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.543155+0000 osd.5 (osd.5) 388 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.544026+0000 osd.5 (osd.5) 389 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.602403+0000 osd.6 (osd.6) 377 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:05.003 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.602766+0000 osd.6 (osd.6) 378 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:05.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.930969+0000 osd.2 (osd.2) 353 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:47:05.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:03.932023+0000 osd.2 (osd.2) 354 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:47:05.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:04.048734+0000 mgr.y (mgr.24454) 721 : cluster [DBG] pgmap v1287: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:05.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:04.666765+0000 osd.7 (osd.7) 281 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:05.004 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:04.666896+0000 osd.7 (osd.7) 282 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:05.043 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:05.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.293197+0000 osd.0 (osd.0) 378 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.296325+0000 osd.0 (osd.0) 379 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.543155+0000 osd.5 (osd.5) 388 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.544026+0000 osd.5 (osd.5) 389 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.602403+0000 osd.6 (osd.6) 377 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.602766+0000 osd.6 (osd.6) 378 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.930969+0000 osd.2 (osd.2) 353 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:03.932023+0000 osd.2 (osd.2) 354 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:04.048734+0000 mgr.y (mgr.24454) 721 : cluster [DBG] pgmap v1287: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:04.666765+0000 osd.7 (osd.7) 281 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:05.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:04.666896+0000 osd.7 (osd.7) 282 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:06.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:04.872529+0000 mon.a (mon.0) 3810 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-02-18T00:47:06.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:05.711460+0000 osd.4 (osd.4) 453 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:06.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:05.711796+0000 osd.4 (osd.4) 454 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:06.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:04.872529+0000 mon.a (mon.0) 3810 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-02-18T00:47:06.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:05.711460+0000 osd.4 (osd.4) 453 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:06.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:05.711796+0000 osd.4 (osd.4) 454 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:06.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:04.872529+0000 mon.a (mon.0) 3810 : cluster [DBG] osdmap e828: 8 total, 8 up, 8 in 2024-02-18T00:47:06.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:05.711460+0000 osd.4 (osd.4) 453 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:06.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:05.711796+0000 osd.4 (osd.4) 454 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:06.888 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:05.885884+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-02-18T00:47:06.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:05.942320+0000 osd.3 (osd.3) 443 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:47:06.889 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:05.944091+0000 osd.3 (osd.3) 444 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:47:07.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 -- 172.21.15.59:0/1120287849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80f8420 msgr2=0x7f6ed8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 --2- 172.21.15.59:0/1120287849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80f8420 0x7f6ed8104a20 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 rx=0x7f6ed0009c30 tx=0x7f6ed0004c50).stop 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 -- 172.21.15.59:0/1120287849 shutdown_connections 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 --2- 172.21.15.59:0/1120287849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80f8420 0x7f6ed8104a20 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 --2- 172.21.15.59:0/1120287849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 0x7f6ed80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 --2- 172.21.15.59:0/1120287849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ed80fe510 0x7f6ed80f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 -- 172.21.15.59:0/1120287849 >> 172.21.15.59:0/1120287849 conn(0x7f6ed80f3800 msgr2=0x7f6ed80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 -- 172.21.15.59:0/1120287849 shutdown_connections 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.164+0000 7f6edd258700 1 -- 172.21.15.59:0/1120287849 wait complete. 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 librados: starting msgr at 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 librados: starting objecter 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 Processor -- start 2024-02-18T00:47:07.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 -- start start 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 librados: setting wanted keys 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 librados: calling monclient init 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ed80f8420 0x7f6ed806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.165+0000 7f6edd258700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 0x7f6ed806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80fe510 0x7f6ed806bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8116400 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8116570 con 0x7f6ed80f8420 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8116730 con 0x7f6ed80fe510 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 0x7f6ed806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed759e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80fe510 0x7f6ed806bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 0x7f6ed806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48542/0 (socket says 172.21.15.59:48542) 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 -- 172.21.15.59:0/1253303873 learned_addr learned my addr 172.21.15.59:0/1253303873 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:07.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed759e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80fe510 0x7f6ed806bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:35550/0 (socket says 172.21.15.59:35550) 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed6d9d700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ed80f8420 0x7f6ed806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 -- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80fe510 msgr2=0x7f6ed806bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80fe510 0x7f6ed806bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 -- 172.21.15.59:0/1253303873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ed80f8420 msgr2=0x7f6ed806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ed80f8420 0x7f6ed806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 -- 172.21.15.59:0/1253303873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0005040 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed6d9d700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ed80f8420 0x7f6ed806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed759e700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80fe510 0x7f6ed806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ed659c700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 0x7f6ed806b5c0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 rx=0x7f6ec800b530 tx=0x7f6ec80090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ecffff700 10 client.?.objecter ms_handle_connect 0x7f6ed80fdbb0 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ecffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ecffff700 1 -- 172.21.15.59:0/1253303873 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8002320 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed00097c0 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ecffff700 1 -- 172.21.15.59:0/1253303873 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ec8002950 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6ecffff700 1 -- 172.21.15.59:0/1253303873 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80129a0 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 10 client.32154.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed8116e40 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.166+0000 7f6edd258700 1 librados: init done 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.167+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed8107110 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.168+0000 7f6ecffff700 1 -- 172.21.15.59:0/1253303873 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f6ec8002480 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.168+0000 7f6ecffff700 1 --2- 172.21.15.59:0/1253303873 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6ec4062c90 0x7f6ec4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.168+0000 7f6ecffff700 1 -- 172.21.15.59:0/1253303873 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(829..829 src has 418..829) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f6ec807d060 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.168+0000 7f6ecffff700 10 client.32154.objecter ms_dispatch 0x7f6ed8001550 osd_map(829..829 src has 418..829) v4 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.168+0000 7f6ecffff700 3 client.32154.objecter handle_osd_map got epochs [829,829] > 0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.168+0000 7f6ecffff700 3 client.32154.objecter handle_osd_map decoding full epoch 829 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.168+0000 7f6ed6d9d700 1 --2- 172.21.15.59:0/1253303873 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6ec4062c90 0x7f6ec4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.169+0000 7f6ecffff700 20 client.32154.objecter dump_active .. 0 homeless 2024-02-18T00:47:07.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.169+0000 7f6ecffff700 10 client.32154.objecter ms_handle_connect 0x7f6ec4062c90 2024-02-18T00:47:07.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.169+0000 7f6ed6d9d700 1 --2- 172.21.15.59:0/1253303873 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6ec4062c90 0x7f6ec4065150 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 rx=0x7f6ec0009f20 tx=0x7f6ec0009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:07.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.173+0000 7f6ecffff700 1 -- 172.21.15.59:0/1253303873 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6ec80506d0 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:05.885884+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-02-18T00:47:07.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:05.942320+0000 osd.3 (osd.3) 443 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:47:07.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:05.944091+0000 osd.3 (osd.3) 444 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:47:07.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:06.049316+0000 mgr.y (mgr.24454) 722 : cluster [DBG] pgmap v1290: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:47:07.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:06.049316+0000 mgr.y (mgr.24454) 722 : cluster [DBG] pgmap v1290: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:47:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:05.885884+0000 mon.a (mon.0) 3811 : cluster [DBG] osdmap e829: 8 total, 8 up, 8 in 2024-02-18T00:47:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:05.942320+0000 osd.3 (osd.3) 443 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:47:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:05.944091+0000 osd.3 (osd.3) 444 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:47:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:06.049316+0000 mgr.y (mgr.24454) 722 : cluster [DBG] pgmap v1290: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:47:07.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.356+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f6ed806c5b0 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.895+0000 7f6ecffff700 1 -- 172.21.15.59:0/1253303873 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v830) v1 ==== 113+0+0 (secure 0 0 0) 0x7f6ec80502c0 con 0x7f6ed80fdbb0 2024-02-18T00:47:07.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.897+0000 7f6edd258700 10 librados: watch_flush enter 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.897+0000 7f6edd258700 10 librados: watch_flush exit 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.897+0000 7f6edd258700 20 client.32154.objecter shutdown clearing up homeless session... 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.897+0000 7f6edd258700 10 client.32154.objecter successfully canceled tick 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6ec4062c90 msgr2=0x7f6ec4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 --2- 172.21.15.59:0/1253303873 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6ec4062c90 0x7f6ec4065150 secure :-1 s=READY pgs=3255 cs=0 l=1 rev1=1 rx=0x7f6ec0009f20 tx=0x7f6ec0009cb0).stop 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 msgr2=0x7f6ed806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 0x7f6ed806b5c0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 rx=0x7f6ec800b530 tx=0x7f6ec80090f0).stop 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 shutdown_connections 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 --2- 172.21.15.59:0/1253303873 >> v2:172.21.15.59:6800/3721891797 conn(0x7f6ec4062c90 0x7f6ec4065150 unknown :-1 s=CLOSED pgs=3255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ed80f8420 0x7f6ed806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6ed80fe510 0x7f6ed806bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 --2- 172.21.15.59:0/1253303873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ed80fdbb0 0x7f6ed806b5c0 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 >> 172.21.15.59:0/1253303873 conn(0x7f6ed80f3800 msgr2=0x7f6ed80f50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 shutdown_connections 2024-02-18T00:47:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 -- 172.21.15.59:0/1253303873 wait complete. 2024-02-18T00:47:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:07.898+0000 7f6edd258700 1 librados: shutdown 2024-02-18T00:47:07.901 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.3 to 1 (10000) 2024-02-18T00:47:08.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:07.358250+0000 mon.a (mon.0) 3812 : audit [INF] from='client.? 172.21.15.59:0/1253303873' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:47:08.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:07.648077+0000 osd.4 (osd.4) 455 : cluster [DBG] 4.c scrub starts 2024-02-18T00:47:08.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:07.651600+0000 osd.4 (osd.4) 456 : cluster [DBG] 4.c scrub ok 2024-02-18T00:47:08.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:07.358250+0000 mon.a (mon.0) 3812 : audit [INF] from='client.? 172.21.15.59:0/1253303873' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:47:08.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:07.648077+0000 osd.4 (osd.4) 455 : cluster [DBG] 4.c scrub starts 2024-02-18T00:47:08.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:07.651600+0000 osd.4 (osd.4) 456 : cluster [DBG] 4.c scrub ok 2024-02-18T00:47:08.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:07.358250+0000 mon.a (mon.0) 3812 : audit [INF] from='client.? 172.21.15.59:0/1253303873' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:47:08.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:07.648077+0000 osd.4 (osd.4) 455 : cluster [DBG] 4.c scrub starts 2024-02-18T00:47:08.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:07.651600+0000 osd.4 (osd.4) 456 : cluster [DBG] 4.c scrub ok 2024-02-18T00:47:08.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 3 1 2024-02-18T00:47:08.903 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:07.269250+0000 osd.0 (osd.0) 380 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:47:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:07.270335+0000 osd.0 (osd.0) 381 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:47:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:07.895476+0000 mon.a (mon.0) 3813 : audit [INF] from='client.? 172.21.15.59:0/1253303873' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:47:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:07.895571+0000 mon.a (mon.0) 3814 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-02-18T00:47:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:08.049711+0000 mgr.y (mgr.24454) 723 : cluster [DBG] pgmap v1292: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:08.904 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:08.853733+0000 mon.a (mon.0) 3815 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-02-18T00:47:08.937 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:09.180 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:07.269250+0000 osd.0 (osd.0) 380 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:47:09.180 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:07.270335+0000 osd.0 (osd.0) 381 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:47:09.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:07.895476+0000 mon.a (mon.0) 3813 : audit [INF] from='client.? 172.21.15.59:0/1253303873' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:47:09.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:07.895571+0000 mon.a (mon.0) 3814 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-02-18T00:47:09.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:08.049711+0000 mgr.y (mgr.24454) 723 : cluster [DBG] pgmap v1292: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:09.181 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:08.853733+0000 mon.a (mon.0) 3815 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-02-18T00:47:09.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:07.269250+0000 osd.0 (osd.0) 380 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:47:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:07.270335+0000 osd.0 (osd.0) 381 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:47:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:07.895476+0000 mon.a (mon.0) 3813 : audit [INF] from='client.? 172.21.15.59:0/1253303873' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:47:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:07.895571+0000 mon.a (mon.0) 3814 : cluster [DBG] osdmap e830: 8 total, 8 up, 8 in 2024-02-18T00:47:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:08.049711+0000 mgr.y (mgr.24454) 723 : cluster [DBG] pgmap v1292: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:09.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:08.853733+0000 mon.a (mon.0) 3815 : cluster [DBG] osdmap e831: 8 total, 8 up, 8 in 2024-02-18T00:47:10.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:09.643928+0000 osd.4 (osd.4) 457 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:47:10.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:09.645922+0000 osd.4 (osd.4) 458 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:47:10.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:09.643928+0000 osd.4 (osd.4) 457 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:47:10.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:09.645922+0000 osd.4 (osd.4) 458 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:47:10.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:09.643928+0000 osd.4 (osd.4) 457 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:47:10.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:09.645922+0000 osd.4 (osd.4) 458 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:47:11.001 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:10.050279+0000 mgr.y (mgr.24454) 724 : cluster [DBG] pgmap v1294: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-02-18T00:47:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:10.597427+0000 osd.4 (osd.4) 459 : cluster [DBG] 5.e scrub starts 2024-02-18T00:47:11.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:10.600892+0000 osd.4 (osd.4) 460 : cluster [DBG] 5.e scrub ok 2024-02-18T00:47:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:10.050279+0000 mgr.y (mgr.24454) 724 : cluster [DBG] pgmap v1294: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-02-18T00:47:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:10.597427+0000 osd.4 (osd.4) 459 : cluster [DBG] 5.e scrub starts 2024-02-18T00:47:11.002 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:10.600892+0000 osd.4 (osd.4) 460 : cluster [DBG] 5.e scrub ok 2024-02-18T00:47:11.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 -- 172.21.15.59:0/1828908651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f8d10 msgr2=0x7f8a080f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1828908651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f8d10 0x7f8a080f90f0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 rx=0x7f89f8004870 tx=0x7f89f8009cb0).stop 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 -- 172.21.15.59:0/1828908651 shutdown_connections 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1828908651 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a08102270 0x7f8a08104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1828908651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f8d10 0x7f8a080f90f0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1828908651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a080f9670 0x7f8a08101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 -- 172.21.15.59:0/1828908651 >> 172.21.15.59:0/1828908651 conn(0x7f8a080f2820 msgr2=0x7f8a080f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 -- 172.21.15.59:0/1828908651 shutdown_connections 2024-02-18T00:47:11.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.036+0000 7f8a101cb700 1 -- 172.21.15.59:0/1828908651 wait complete. 2024-02-18T00:47:11.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 librados: starting msgr at 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 librados: starting objecter 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 Processor -- start 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 -- start start 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 librados: setting wanted keys 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 librados: calling monclient init 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a080f8d10 0x7f8a0810e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f9670 0x7f8a0810e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08102270 0x7f8a081082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:11.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08108790 con 0x7f8a080f9670 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08108900 con 0x7f8a08102270 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a101cb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08108ac0 con 0x7f8a080f8d10 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a0df67700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a080f8d10 0x7f8a0810e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a0df67700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a080f8d10 0x7f8a0810e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49932/0 (socket says 172.21.15.59:49932) 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a0df67700 1 -- 172.21.15.59:0/1987848454 learned_addr learned my addr 172.21.15.59:0/1987848454 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.037+0000 7f8a0d766700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f9670 0x7f8a0810e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0e768700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08102270 0x7f8a081082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0d766700 1 -- 172.21.15.59:0/1987848454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a080f8d10 msgr2=0x7f8a0810e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0d766700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a080f8d10 0x7f8a0810e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0d766700 1 -- 172.21.15.59:0/1987848454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08102270 msgr2=0x7f8a081082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0d766700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08102270 0x7f8a081082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0d766700 1 -- 172.21.15.59:0/1987848454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a000097c0 con 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0e768700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08102270 0x7f8a081082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a0d766700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f9670 0x7f8a0810e5f0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 rx=0x7f8a0400c810 tx=0x7f8a04007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f89feffd700 10 client.?.objecter ms_handle_connect 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f89feffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f89feffd700 1 -- 172.21.15.59:0/1987848454 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400e4a0 con 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f89feffd700 1 -- 172.21.15.59:0/1987848454 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0400ead0 con 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f8005040 con 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f89feffd700 1 -- 172.21.15.59:0/1987848454 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04012a40 con 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a101cb700 10 client.32160.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a08114f30 con 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.038+0000 7f8a101cb700 1 librados: init done 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.039+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0805fa60 con 0x7f8a080f9670 2024-02-18T00:47:11.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.039+0000 7f89feffd700 1 -- 172.21.15.59:0/1987848454 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f8a0400ec40 con 0x7f8a080f9670 2024-02-18T00:47:11.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f89feffd700 1 --2- 172.21.15.59:0/1987848454 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89f4062c90 0x7f89f4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:11.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f89feffd700 1 -- 172.21.15.59:0/1987848454 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(831..831 src has 418..831) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f8a0407cba0 con 0x7f8a080f9670 2024-02-18T00:47:11.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f89feffd700 10 client.32160.objecter ms_dispatch 0x7f8a08001550 osd_map(831..831 src has 418..831) v4 2024-02-18T00:47:11.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f89feffd700 3 client.32160.objecter handle_osd_map got epochs [831,831] > 0 2024-02-18T00:47:11.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f89feffd700 3 client.32160.objecter handle_osd_map decoding full epoch 831 2024-02-18T00:47:11.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f8a0df67700 1 --2- 172.21.15.59:0/1987848454 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89f4062c90 0x7f89f4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:11.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f89feffd700 20 client.32160.objecter dump_active .. 0 homeless 2024-02-18T00:47:11.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f89feffd700 10 client.32160.objecter ms_handle_connect 0x7f89f4062c90 2024-02-18T00:47:11.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.040+0000 7f8a0df67700 1 --2- 172.21.15.59:0/1987848454 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89f4062c90 0x7f89f4065150 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 rx=0x7f89f8009040 tx=0x7f89f800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:11.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.044+0000 7f89feffd700 1 -- 172.21.15.59:0/1987848454 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8a04050270 con 0x7f8a080f9670 2024-02-18T00:47:11.234 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.232+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) v1 -- 0x7f8a0805eeb0 con 0x7f8a080f9670 2024-02-18T00:47:11.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:47:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:47:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:10.050279+0000 mgr.y (mgr.24454) 724 : cluster [DBG] pgmap v1294: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-02-18T00:47:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:10.597427+0000 osd.4 (osd.4) 459 : cluster [DBG] 5.e scrub starts 2024-02-18T00:47:11.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:10.600892+0000 osd.4 (osd.4) 460 : cluster [DBG] 5.e scrub ok 2024-02-18T00:47:11.922 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.920+0000 7f89feffd700 1 -- 172.21.15.59:0/1987848454 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v832) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8a040180a0 con 0x7f8a080f9670 2024-02-18T00:47:11.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 10 librados: watch_flush enter 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 10 librados: watch_flush exit 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 20 client.32160.objecter shutdown clearing up homeless session... 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 10 client.32160.objecter successfully canceled tick 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89f4062c90 msgr2=0x7f89f4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1987848454 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89f4062c90 0x7f89f4065150 secure :-1 s=READY pgs=3256 cs=0 l=1 rev1=1 rx=0x7f89f8009040 tx=0x7f89f800b040).stop 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f9670 msgr2=0x7f8a0810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f9670 0x7f8a0810e5f0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 rx=0x7f8a0400c810 tx=0x7f8a04007e30).stop 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 shutdown_connections 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1987848454 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89f4062c90 0x7f89f4065150 unknown :-1 s=CLOSED pgs=3256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a08102270 0x7f8a081082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a080f8d10 0x7f8a0810e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 --2- 172.21.15.59:0/1987848454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a080f9670 0x7f8a0810e5f0 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 >> 172.21.15.59:0/1987848454 conn(0x7f8a080f2820 msgr2=0x7f8a080f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 shutdown_connections 2024-02-18T00:47:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 -- 172.21.15.59:0/1987848454 wait complete. 2024-02-18T00:47:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:11.922+0000 7f8a101cb700 1 librados: shutdown 2024-02-18T00:47:11.925 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-02-18T00:47:12.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:47:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:47:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:10.849841+0000 osd.3 (osd.3) 445 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:47:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:10.851026+0000 osd.3 (osd.3) 446 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:47:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:11.234385+0000 mon.a (mon.0) 3816 : audit [INF] from='client.? 172.21.15.59:0/1987848454' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:47:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:11.573120+0000 osd.4 (osd.4) 461 : cluster [DBG] 6.e scrub starts 2024-02-18T00:47:12.004 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:11.576153+0000 osd.4 (osd.4) 462 : cluster [DBG] 6.e scrub ok 2024-02-18T00:47:12.211 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:10.849841+0000 osd.3 (osd.3) 445 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:10.851026+0000 osd.3 (osd.3) 446 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:11.234385+0000 mon.a (mon.0) 3816 : audit [INF] from='client.? 172.21.15.59:0/1987848454' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:11.573120+0000 osd.4 (osd.4) 461 : cluster [DBG] 6.e scrub starts 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:11.576153+0000 osd.4 (osd.4) 462 : cluster [DBG] 6.e scrub ok 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:10.849841+0000 osd.3 (osd.3) 445 : cluster [DBG] 3.2 scrub starts 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:10.851026+0000 osd.3 (osd.3) 446 : cluster [DBG] 3.2 scrub ok 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:11.234385+0000 mon.a (mon.0) 3816 : audit [INF] from='client.? 172.21.15.59:0/1987848454' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:11.573120+0000 osd.4 (osd.4) 461 : cluster [DBG] 6.e scrub starts 2024-02-18T00:47:12.212 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:11.576153+0000 osd.4 (osd.4) 462 : cluster [DBG] 6.e scrub ok 2024-02-18T00:47:12.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:12] "GET /metrics HTTP/1.1" 200 187521 "" "Prometheus/2.33.4" 2024-02-18T00:47:12.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 4 1 2024-02-18T00:47:13.165 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:11.710290+0000 mgr.y (mgr.24454) 725 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:11.918874+0000 mon.a (mon.0) 3817 : audit [INF] from='client.? 172.21.15.59:0/1987848454' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:11.918948+0000 mon.a (mon.0) 3818 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:12.050720+0000 mgr.y (mgr.24454) 726 : cluster [DBG] pgmap v1296: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:12.362711+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:12.522346+0000 osd.5 (osd.5) 390 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:12.523160+0000 osd.5 (osd.5) 391 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:12.530545+0000 osd.4 (osd.4) 463 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:47:13.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:12.531324+0000 osd.4 (osd.4) 464 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:11.710290+0000 mgr.y (mgr.24454) 725 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:11.918874+0000 mon.a (mon.0) 3817 : audit [INF] from='client.? 172.21.15.59:0/1987848454' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:11.918948+0000 mon.a (mon.0) 3818 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:12.050720+0000 mgr.y (mgr.24454) 726 : cluster [DBG] pgmap v1296: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:12.362711+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:12.522346+0000 osd.5 (osd.5) 390 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:12.523160+0000 osd.5 (osd.5) 391 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:12.530545+0000 osd.4 (osd.4) 463 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:47:13.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:12.531324+0000 osd.4 (osd.4) 464 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:11.710290+0000 mgr.y (mgr.24454) 725 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:11.918874+0000 mon.a (mon.0) 3817 : audit [INF] from='client.? 172.21.15.59:0/1987848454' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:11.918948+0000 mon.a (mon.0) 3818 : cluster [DBG] osdmap e832: 8 total, 8 up, 8 in 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:12.050720+0000 mgr.y (mgr.24454) 726 : cluster [DBG] pgmap v1296: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:12.362711+0000 mon.a (mon.0) 3819 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:12.522346+0000 osd.5 (osd.5) 390 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:12.523160+0000 osd.5 (osd.5) 391 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:12.530545+0000 osd.4 (osd.4) 463 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:47:13.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:12.531324+0000 osd.4 (osd.4) 464 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:12.130659+0000 osd.0 (osd.0) 382 : cluster [DBG] 4.b scrub starts 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:12.134186+0000 osd.0 (osd.0) 383 : cluster [DBG] 4.b scrub ok 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:13.010205+0000 mon.a (mon.0) 3820 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:13.010928+0000 mon.a (mon.0) 3821 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:13.015077+0000 mon.a (mon.0) 3822 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.487718+0000 osd.4 (osd.4) 465 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.490669+0000 osd.4 (osd.4) 466 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:47:14.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.855684+0000 mon.a (mon.0) 3823 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:12.130659+0000 osd.0 (osd.0) 382 : cluster [DBG] 4.b scrub starts 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:12.134186+0000 osd.0 (osd.0) 383 : cluster [DBG] 4.b scrub ok 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:13.010205+0000 mon.a (mon.0) 3820 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:13.010928+0000 mon.a (mon.0) 3821 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:13.015077+0000 mon.a (mon.0) 3822 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.487718+0000 osd.4 (osd.4) 465 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.490669+0000 osd.4 (osd.4) 466 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:47:14.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.855684+0000 mon.a (mon.0) 3823 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-02-18T00:47:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:12.130659+0000 osd.0 (osd.0) 382 : cluster [DBG] 4.b scrub starts 2024-02-18T00:47:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:12.134186+0000 osd.0 (osd.0) 383 : cluster [DBG] 4.b scrub ok 2024-02-18T00:47:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:13.010205+0000 mon.a (mon.0) 3820 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:47:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:13.010928+0000 mon.a (mon.0) 3821 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:47:14.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:13.015077+0000 mon.a (mon.0) 3822 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:47:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.487718+0000 osd.4 (osd.4) 465 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:47:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.490669+0000 osd.4 (osd.4) 466 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:47:14.255 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.855684+0000 mon.a (mon.0) 3823 : cluster [DBG] osdmap e833: 8 total, 8 up, 8 in 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.549202+0000 osd.5 (osd.5) 392 : cluster [DBG] 6.d scrub starts 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.552643+0000 osd.5 (osd.5) 393 : cluster [DBG] 6.d scrub ok 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.847301+0000 osd.3 (osd.3) 447 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.848904+0000 osd.3 (osd.3) 448 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.856683+0000 osd.2 (osd.2) 355 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:13.858020+0000 osd.2 (osd.2) 356 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:14.051086+0000 mgr.y (mgr.24454) 727 : cluster [DBG] pgmap v1298: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:14.444388+0000 osd.4 (osd.4) 467 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:47:15.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:14.447411+0000 osd.4 (osd.4) 468 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.549202+0000 osd.5 (osd.5) 392 : cluster [DBG] 6.d scrub starts 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.552643+0000 osd.5 (osd.5) 393 : cluster [DBG] 6.d scrub ok 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.847301+0000 osd.3 (osd.3) 447 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.848904+0000 osd.3 (osd.3) 448 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.856683+0000 osd.2 (osd.2) 355 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:13.858020+0000 osd.2 (osd.2) 356 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:14.051086+0000 mgr.y (mgr.24454) 727 : cluster [DBG] pgmap v1298: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:14.444388+0000 osd.4 (osd.4) 467 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:47:15.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:14.447411+0000 osd.4 (osd.4) 468 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.549202+0000 osd.5 (osd.5) 392 : cluster [DBG] 6.d scrub starts 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.552643+0000 osd.5 (osd.5) 393 : cluster [DBG] 6.d scrub ok 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.847301+0000 osd.3 (osd.3) 447 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.848904+0000 osd.3 (osd.3) 448 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.856683+0000 osd.2 (osd.2) 355 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:13.858020+0000 osd.2 (osd.2) 356 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:14.051086+0000 mgr.y (mgr.24454) 727 : cluster [DBG] pgmap v1298: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:14.444388+0000 osd.4 (osd.4) 467 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:47:15.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:14.447411+0000 osd.4 (osd.4) 468 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:47:15.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 -- 172.21.15.59:0/2892005289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640fd510 msgr2=0x7fe0640f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/2892005289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640fd510 0x7fe0640f6d40 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 rx=0x7fe0580047b0 tx=0x7fe058009cb0).stop 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 -- 172.21.15.59:0/2892005289 shutdown_connections 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/2892005289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0640fcbb0 0x7fe0640fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/2892005289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640f7400 0x7fe064103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/2892005289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640fd510 0x7fe0640f6d40 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 -- 172.21.15.59:0/2892005289 >> 172.21.15.59:0/2892005289 conn(0x7fe0640f2820 msgr2=0x7fe0640f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 -- 172.21.15.59:0/2892005289 shutdown_connections 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.229+0000 7fe06bdd5700 1 -- 172.21.15.59:0/2892005289 wait complete. 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 librados: starting msgr at 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 librados: starting objecter 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 Processor -- start 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 -- start start 2024-02-18T00:47:15.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 librados: setting wanted keys 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 librados: calling monclient init 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640f7400 0x7fe064109260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0640fcbb0 0x7fe064109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640fd510 0x7fe064109d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe064114440 con 0x7fe0640fd510 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0641145b0 con 0x7fe0640fcbb0 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06bdd5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe064114770 con 0x7fe0640f7400 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06a372700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640fd510 0x7fe064109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe069b71700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640f7400 0x7fe064109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06a372700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640fd510 0x7fe064109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46560/0 (socket says 172.21.15.59:46560) 2024-02-18T00:47:15.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe069b71700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640f7400 0x7fe064109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:49960/0 (socket says 172.21.15.59:49960) 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.230+0000 7fe06a372700 1 -- 172.21.15.59:0/1725740087 learned_addr learned my addr 172.21.15.59:0/1725740087 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe069370700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0640fcbb0 0x7fe064109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06a372700 1 -- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640f7400 msgr2=0x7fe064109260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06a372700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640f7400 0x7fe064109260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06a372700 1 -- 172.21.15.59:0/1725740087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0640fcbb0 msgr2=0x7fe064109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06a372700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0640fcbb0 0x7fe064109750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06a372700 1 -- 172.21.15.59:0/1725740087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c0097c0 con 0x7fe0640fd510 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe069b71700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640f7400 0x7fe064109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06a372700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640fd510 0x7fe064109d30 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 rx=0x7fe05c001df0 tx=0x7fe05c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe056ffd700 10 client.?.objecter ms_handle_connect 0x7fe0640fd510 2024-02-18T00:47:15.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe056ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe056ffd700 1 -- 172.21.15.59:0/1725740087 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c009a10 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe056ffd700 1 -- 172.21.15.59:0/1725740087 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe05c0090d0 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe058005040 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe056ffd700 1 -- 172.21.15.59:0/1725740087 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c007d30 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06bdd5700 10 client.32166.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe064114d60 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.231+0000 7fe06bdd5700 1 librados: init done 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.232+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06405fa60 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe056ffd700 1 -- 172.21.15.59:0/1725740087 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe05c00fb30 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe056ffd700 1 --2- 172.21.15.59:0/1725740087 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe050062c90 0x7fe050065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe056ffd700 1 -- 172.21.15.59:0/1725740087 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(833..833 src has 418..833) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fe05c07daf0 con 0x7fe0640fd510 2024-02-18T00:47:15.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe056ffd700 10 client.32166.objecter ms_dispatch 0x7fe064001550 osd_map(833..833 src has 418..833) v4 2024-02-18T00:47:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe069b71700 1 --2- 172.21.15.59:0/1725740087 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe050062c90 0x7fe050065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe056ffd700 3 client.32166.objecter handle_osd_map got epochs [833,833] > 0 2024-02-18T00:47:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe056ffd700 3 client.32166.objecter handle_osd_map decoding full epoch 833 2024-02-18T00:47:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.233+0000 7fe056ffd700 20 client.32166.objecter dump_active .. 0 homeless 2024-02-18T00:47:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.234+0000 7fe056ffd700 10 client.32166.objecter ms_handle_connect 0x7fe050062c90 2024-02-18T00:47:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.234+0000 7fe069b71700 1 --2- 172.21.15.59:0/1725740087 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe050062c90 0x7fe050065150 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 rx=0x7fe0600103c0 tx=0x7fe06000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:15.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.237+0000 7fe056ffd700 1 -- 172.21.15.59:0/1725740087 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe05c051260 con 0x7fe0640fd510 2024-02-18T00:47:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.422+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) v1 -- 0x7fe064061440 con 0x7fe0640fd510 2024-02-18T00:47:15.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.961+0000 7fe056ffd700 1 -- 172.21.15.59:0/1725740087 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v834) v1 ==== 113+0+0 (secure 0 0 0) 0x7fe05c050e50 con 0x7fe0640fd510 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 10 librados: watch_flush enter 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 10 librados: watch_flush exit 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 20 client.32166.objecter shutdown clearing up homeless session... 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 10 client.32166.objecter successfully canceled tick 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe050062c90 msgr2=0x7fe050065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/1725740087 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe050062c90 0x7fe050065150 secure :-1 s=READY pgs=3257 cs=0 l=1 rev1=1 rx=0x7fe0600103c0 tx=0x7fe06000b040).stop 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640fd510 msgr2=0x7fe064109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640fd510 0x7fe064109d30 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 rx=0x7fe05c001df0 tx=0x7fe05c004d10).stop 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 shutdown_connections 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0640fd510 0x7fe064109d30 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0640f7400 0x7fe064109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/1725740087 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe050062c90 0x7fe050065150 unknown :-1 s=CLOSED pgs=3257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 --2- 172.21.15.59:0/1725740087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0640fcbb0 0x7fe064109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:15.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 >> 172.21.15.59:0/1725740087 conn(0x7fe0640f2820 msgr2=0x7fe0640f4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:15.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 shutdown_connections 2024-02-18T00:47:15.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 -- 172.21.15.59:0/1725740087 wait complete. 2024-02-18T00:47:15.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:15.964+0000 7fe06bdd5700 1 librados: shutdown 2024-02-18T00:47:15.967 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.4 to 1 (10000) 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:14.151930+0000 osd.0 (osd.0) 384 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:14.154329+0000 osd.0 (osd.0) 385 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:15.423910+0000 mon.a (mon.0) 3824 : audit [INF] from='client.? 172.21.15.59:0/1725740087' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:15.525970+0000 osd.5 (osd.5) 394 : cluster [DBG] 2.2 deep-scrub starts 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:15.527231+0000 osd.5 (osd.5) 395 : cluster [DBG] 2.2 deep-scrub ok 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:15.793927+0000 osd.2 (osd.2) 357 : cluster [DBG] 5.b scrub starts 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:15.794786+0000 osd.2 (osd.2) 358 : cluster [DBG] 5.b scrub ok 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:14.151930+0000 osd.0 (osd.0) 384 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:47:16.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:14.154329+0000 osd.0 (osd.0) 385 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:15.423910+0000 mon.a (mon.0) 3824 : audit [INF] from='client.? 172.21.15.59:0/1725740087' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:15.525970+0000 osd.5 (osd.5) 394 : cluster [DBG] 2.2 deep-scrub starts 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:15.527231+0000 osd.5 (osd.5) 395 : cluster [DBG] 2.2 deep-scrub ok 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:15.793927+0000 osd.2 (osd.2) 357 : cluster [DBG] 5.b scrub starts 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:15.794786+0000 osd.2 (osd.2) 358 : cluster [DBG] 5.b scrub ok 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:14.151930+0000 osd.0 (osd.0) 384 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:14.154329+0000 osd.0 (osd.0) 385 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:15.423910+0000 mon.a (mon.0) 3824 : audit [INF] from='client.? 172.21.15.59:0/1725740087' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:16.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:15.525970+0000 osd.5 (osd.5) 394 : cluster [DBG] 2.2 deep-scrub starts 2024-02-18T00:47:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:15.527231+0000 osd.5 (osd.5) 395 : cluster [DBG] 2.2 deep-scrub ok 2024-02-18T00:47:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:15.793927+0000 osd.2 (osd.2) 357 : cluster [DBG] 5.b scrub starts 2024-02-18T00:47:16.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:15.794786+0000 osd.2 (osd.2) 358 : cluster [DBG] 5.b scrub ok 2024-02-18T00:47:16.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 4 1 2024-02-18T00:47:17.237 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:17.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:15.961877+0000 mon.a (mon.0) 3825 : audit [INF] from='client.? 172.21.15.59:0/1725740087' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:47:17.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:15.961992+0000 mon.a (mon.0) 3826 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-02-18T00:47:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:16.051488+0000 mgr.y (mgr.24454) 728 : cluster [DBG] pgmap v1300: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:47:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:15.961877+0000 mon.a (mon.0) 3825 : audit [INF] from='client.? 172.21.15.59:0/1725740087' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:47:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:15.961992+0000 mon.a (mon.0) 3826 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-02-18T00:47:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:16.051488+0000 mgr.y (mgr.24454) 728 : cluster [DBG] pgmap v1300: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:47:17.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:15.961877+0000 mon.a (mon.0) 3825 : audit [INF] from='client.? 172.21.15.59:0/1725740087' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:47:17.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:15.961992+0000 mon.a (mon.0) 3826 : cluster [DBG] osdmap e834: 8 total, 8 up, 8 in 2024-02-18T00:47:17.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:16.051488+0000 mgr.y (mgr.24454) 728 : cluster [DBG] pgmap v1300: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T00:47:18.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:16.112804+0000 osd.0 (osd.0) 386 : cluster [DBG] 2.1f deep-scrub starts 2024-02-18T00:47:18.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:16.113776+0000 osd.0 (osd.0) 387 : cluster [DBG] 2.1f deep-scrub ok 2024-02-18T00:47:18.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:16.185666+0000 osd.0 (osd.0) 388 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:18.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:16.185970+0000 osd.0 (osd.0) 389 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:18.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:16.975501+0000 mon.a (mon.0) 3827 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-02-18T00:47:18.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:16.112804+0000 osd.0 (osd.0) 386 : cluster [DBG] 2.1f deep-scrub starts 2024-02-18T00:47:18.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:16.113776+0000 osd.0 (osd.0) 387 : cluster [DBG] 2.1f deep-scrub ok 2024-02-18T00:47:18.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:16.185666+0000 osd.0 (osd.0) 388 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:18.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:16.185970+0000 osd.0 (osd.0) 389 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:18.245 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:16.975501+0000 mon.a (mon.0) 3827 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-02-18T00:47:18.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:16.112804+0000 osd.0 (osd.0) 386 : cluster [DBG] 2.1f deep-scrub starts 2024-02-18T00:47:18.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:16.113776+0000 osd.0 (osd.0) 387 : cluster [DBG] 2.1f deep-scrub ok 2024-02-18T00:47:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:16.185666+0000 osd.0 (osd.0) 388 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:16.185970+0000 osd.0 (osd.0) 389 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:18.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:16.975501+0000 mon.a (mon.0) 3827 : cluster [DBG] osdmap e835: 8 total, 8 up, 8 in 2024-02-18T00:47:18.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.051976+0000 mgr.y (mgr.24454) 729 : cluster [DBG] pgmap v1302: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:18.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.201080+0000 osd.7 (osd.7) 283 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:47:18.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.202303+0000 osd.7 (osd.7) 284 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:47:18.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.805388+0000 osd.2 (osd.2) 359 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:47:18.984 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.806575+0000 osd.2 (osd.2) 360 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:47:18.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.051976+0000 mgr.y (mgr.24454) 729 : cluster [DBG] pgmap v1302: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:18.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.201080+0000 osd.7 (osd.7) 283 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:47:18.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.202303+0000 osd.7 (osd.7) 284 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:47:18.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.805388+0000 osd.2 (osd.2) 359 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:47:18.985 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.806575+0000 osd.2 (osd.2) 360 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:47:19.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.051976+0000 mgr.y (mgr.24454) 729 : cluster [DBG] pgmap v1302: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:19.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.201080+0000 osd.7 (osd.7) 283 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:47:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.202303+0000 osd.7 (osd.7) 284 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:47:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.805388+0000 osd.2 (osd.2) 359 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:47:19.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.806575+0000 osd.2 (osd.2) 360 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:47:19.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 -- 172.21.15.59:0/2741446230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c1021c0 msgr2=0x7f426c1045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:19.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 --2- 172.21.15.59:0/2741446230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c1021c0 0x7f426c1045b0 secure :-1 s=READY pgs=2672 cs=0 l=1 rev1=1 rx=0x7f4264009c30 tx=0x7f4264004c50).stop 2024-02-18T00:47:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 -- 172.21.15.59:0/2741446230 shutdown_connections 2024-02-18T00:47:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 --2- 172.21.15.59:0/2741446230 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c1021c0 0x7f426c1045b0 unknown :-1 s=CLOSED pgs=2672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 --2- 172.21.15.59:0/2741446230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 0x7f426c0f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 --2- 172.21.15.59:0/2741446230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426c0f96f0 0x7f426c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 -- 172.21.15.59:0/2741446230 >> 172.21.15.59:0/2741446230 conn(0x7f426c0f26f0 msgr2=0x7f426c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 -- 172.21.15.59:0/2741446230 shutdown_connections 2024-02-18T00:47:19.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.398+0000 7f42737b3700 1 -- 172.21.15.59:0/2741446230 wait complete. 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 librados: starting msgr at 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 librados: starting objecter 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 Processor -- start 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 -- start start 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 librados: setting wanted keys 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 librados: calling monclient init 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 0x7f426c0f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:19.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c0f96f0 0x7f426c0f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426c1021c0 0x7f426c0f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c1163f0 con 0x7f426c0f8e20 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c116560 con 0x7f426c1021c0 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f42737b3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c116720 con 0x7f426c0f96f0 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f4270d4e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c0f96f0 0x7f426c0f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f427154f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 0x7f426c0f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f427154f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 0x7f426c0f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46568/0 (socket says 172.21.15.59:46568) 2024-02-18T00:47:19.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f427154f700 1 -- 172.21.15.59:0/2933199031 learned_addr learned my addr 172.21.15.59:0/2933199031 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f4271d50700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426c1021c0 0x7f426c0f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f4270d4e700 1 -- 172.21.15.59:0/2933199031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426c1021c0 msgr2=0x7f426c0f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.399+0000 7f4270d4e700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426c1021c0 0x7f426c0f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f4270d4e700 1 -- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 msgr2=0x7f426c0f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f4270d4e700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 0x7f426c0f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f4270d4e700 1 -- 172.21.15.59:0/2933199031 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c005040 con 0x7f426c0f96f0 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f427154f700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 0x7f426c0f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:19.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f4270d4e700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c0f96f0 0x7f426c0f7100 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7f426800c810 tx=0x7f4268007e30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42627fc700 10 client.?.objecter ms_handle_connect 0x7f426c0f96f0 2024-02-18T00:47:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42627fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42627fc700 1 -- 172.21.15.59:0/2933199031 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426800e4a0 con 0x7f426c0f96f0 2024-02-18T00:47:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42640097c0 con 0x7f426c0f96f0 2024-02-18T00:47:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42627fc700 1 -- 172.21.15.59:0/2933199031 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426800ead0 con 0x7f426c0f96f0 2024-02-18T00:47:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42627fc700 1 -- 172.21.15.59:0/2933199031 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42680129f0 con 0x7f426c0f96f0 2024-02-18T00:47:19.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42737b3700 10 client.41975.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c116db0 con 0x7f426c0f96f0 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.400+0000 7f42737b3700 1 librados: init done 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.401+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c05ee80 con 0x7f426c0f96f0 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.401+0000 7f42627fc700 1 -- 172.21.15.59:0/2933199031 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f426800ec40 con 0x7f426c0f96f0 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f42627fc700 1 --2- 172.21.15.59:0/2933199031 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4258062c90 0x7f4258065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f42627fc700 1 -- 172.21.15.59:0/2933199031 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(835..835 src has 418..835) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f426807cba0 con 0x7f426c0f96f0 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f42627fc700 10 client.41975.objecter ms_dispatch 0x7f426c001550 osd_map(835..835 src has 418..835) v4 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f42627fc700 3 client.41975.objecter handle_osd_map got epochs [835,835] > 0 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f42627fc700 3 client.41975.objecter handle_osd_map decoding full epoch 835 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f427154f700 1 --2- 172.21.15.59:0/2933199031 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4258062c90 0x7f4258065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:19.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f42627fc700 20 client.41975.objecter dump_active .. 0 homeless 2024-02-18T00:47:19.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f42627fc700 10 client.41975.objecter ms_handle_connect 0x7f4258062c90 2024-02-18T00:47:19.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.402+0000 7f427154f700 1 --2- 172.21.15.59:0/2933199031 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4258062c90 0x7f4258065150 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 rx=0x7f425c004d40 tx=0x7f425c00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:19.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.406+0000 7f42627fc700 1 -- 172.21.15.59:0/2933199031 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4268050270 con 0x7f426c0f96f0 2024-02-18T00:47:19.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.590+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) v1 -- 0x7f426c1071f0 con 0x7f426c0f96f0 2024-02-18T00:47:19.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.988+0000 7f42627fc700 1 -- 172.21.15.59:0/2933199031 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v836) v1 ==== 133+0+0 (secure 0 0 0) 0x7f426c1071f0 con 0x7f426c0f96f0 2024-02-18T00:47:19.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 10 librados: watch_flush enter 2024-02-18T00:47:19.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 10 librados: watch_flush exit 2024-02-18T00:47:19.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 20 client.41975.objecter shutdown clearing up homeless session... 2024-02-18T00:47:19.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 10 client.41975.objecter successfully canceled tick 2024-02-18T00:47:19.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4258062c90 msgr2=0x7f4258065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:19.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 1 --2- 172.21.15.59:0/2933199031 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4258062c90 0x7f4258065150 secure :-1 s=READY pgs=3258 cs=0 l=1 rev1=1 rx=0x7f425c004d40 tx=0x7f425c00b040).stop 2024-02-18T00:47:19.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c0f96f0 msgr2=0x7f426c0f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:19.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.992+0000 7f42737b3700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c0f96f0 0x7f426c0f7100 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 rx=0x7f426800c810 tx=0x7f4268007e30).stop 2024-02-18T00:47:19.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 shutdown_connections 2024-02-18T00:47:19.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 --2- 172.21.15.59:0/2933199031 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4258062c90 0x7f4258065150 unknown :-1 s=CLOSED pgs=3258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f426c1021c0 0x7f426c0f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c0f8e20 0x7f426c0f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 --2- 172.21.15.59:0/2933199031 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f426c0f96f0 0x7f426c0f7100 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:19.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 >> 172.21.15.59:0/2933199031 conn(0x7f426c0f26f0 msgr2=0x7f426c0f32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:19.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 shutdown_connections 2024-02-18T00:47:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 -- 172.21.15.59:0/2933199031 wait complete. 2024-02-18T00:47:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:19.993+0000 7f42737b3700 1 librados: shutdown 2024-02-18T00:47:19.998 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.587318+0000 osd.5 (osd.5) 396 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.590435+0000 osd.5 (osd.5) 397 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.814104+0000 osd.1 (osd.1) 413 : cluster [DBG] 4.7 deep-scrub starts 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:18.816168+0000 osd.1 (osd.1) 414 : cluster [DBG] 4.7 deep-scrub ok 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:19.592367+0000 mon.c (mon.2) 588 : audit [INF] from='client.? 172.21.15.59:0/2933199031' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:19.592737+0000 mon.a (mon.0) 3828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.587318+0000 osd.5 (osd.5) 396 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.590435+0000 osd.5 (osd.5) 397 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:47:20.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.814104+0000 osd.1 (osd.1) 413 : cluster [DBG] 4.7 deep-scrub starts 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:18.816168+0000 osd.1 (osd.1) 414 : cluster [DBG] 4.7 deep-scrub ok 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:19.592367+0000 mon.c (mon.2) 588 : audit [INF] from='client.? 172.21.15.59:0/2933199031' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:19.592737+0000 mon.a (mon.0) 3828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.587318+0000 osd.5 (osd.5) 396 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.590435+0000 osd.5 (osd.5) 397 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.814104+0000 osd.1 (osd.1) 413 : cluster [DBG] 4.7 deep-scrub starts 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:18.816168+0000 osd.1 (osd.1) 414 : cluster [DBG] 4.7 deep-scrub ok 2024-02-18T00:47:20.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:19.592367+0000 mon.c (mon.2) 588 : audit [INF] from='client.? 172.21.15.59:0/2933199031' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:20.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:19.592737+0000 mon.a (mon.0) 3828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-02-18T00:47:20.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 5 1 2024-02-18T00:47:20.999 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:19.986558+0000 mon.a (mon.0) 3829 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:47:21.017 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:21.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:19.986558+0000 mon.a (mon.0) 3829 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:47:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:19.986645+0000 mon.a (mon.0) 3830 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-02-18T00:47:21.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:20.052453+0000 mgr.y (mgr.24454) 730 : cluster [DBG] pgmap v1304: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:19.986645+0000 mon.a (mon.0) 3830 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-02-18T00:47:21.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:20.052453+0000 mgr.y (mgr.24454) 730 : cluster [DBG] pgmap v1304: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:21.253 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:47:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:47:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:19.986558+0000 mon.a (mon.0) 3829 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-02-18T00:47:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:19.986645+0000 mon.a (mon.0) 3830 : cluster [DBG] osdmap e836: 8 total, 8 up, 8 in 2024-02-18T00:47:21.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:20.052453+0000 mgr.y (mgr.24454) 730 : cluster [DBG] pgmap v1304: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:22.000 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:47:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:47:22.251 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:22] "GET /metrics HTTP/1.1" 200 187521 "" "Prometheus/2.33.4" 2024-02-18T00:47:22.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:20.576856+0000 osd.5 (osd.5) 398 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:47:22.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:20.580298+0000 osd.5 (osd.5) 399 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:47:22.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:20.760363+0000 osd.3 (osd.3) 449 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:47:22.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:20.761429+0000 osd.3 (osd.3) 450 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:47:22.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:21.196826+0000 osd.7 (osd.7) 285 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:47:22.255 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:21.198785+0000 osd.7 (osd.7) 286 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:47:22.255 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:20.576856+0000 osd.5 (osd.5) 398 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:20.580298+0000 osd.5 (osd.5) 399 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:20.760363+0000 osd.3 (osd.3) 449 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:20.761429+0000 osd.3 (osd.3) 450 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:21.196826+0000 osd.7 (osd.7) 285 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:21.198785+0000 osd.7 (osd.7) 286 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:20.576856+0000 osd.5 (osd.5) 398 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:20.580298+0000 osd.5 (osd.5) 399 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:47:22.256 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:20.760363+0000 osd.3 (osd.3) 449 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:47:22.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:20.761429+0000 osd.3 (osd.3) 450 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:47:22.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:21.196826+0000 osd.7 (osd.7) 285 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:47:22.257 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:21.198785+0000 osd.7 (osd.7) 286 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:47:23.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 -- 172.21.15.59:0/748706889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 msgr2=0x7f4b0c0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 --2- 172.21.15.59:0/748706889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c0f6d40 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 rx=0x7f4af80047b0 tx=0x7f4af8009cb0).stop 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 -- 172.21.15.59:0/748706889 shutdown_connections 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 --2- 172.21.15.59:0/748706889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b0c0f7400 0x7f4b0c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 --2- 172.21.15.59:0/748706889 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 0x7f4b0c0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 --2- 172.21.15.59:0/748706889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c0f6d40 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 -- 172.21.15.59:0/748706889 >> 172.21.15.59:0/748706889 conn(0x7f4b0c0f2820 msgr2=0x7f4b0c0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 -- 172.21.15.59:0/748706889 shutdown_connections 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 -- 172.21.15.59:0/748706889 wait complete. 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 librados: starting msgr at 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 librados: starting objecter 2024-02-18T00:47:23.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 Processor -- start 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 -- start start 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 librados: setting wanted keys 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.199+0000 7f4b12c86700 1 librados: calling monclient init 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b12c86700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b0c0f7400 0x7f4b0c10d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b12c86700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 0x7f4b0c10d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b12c86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c107360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b12c86700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c107850 con 0x7f4b0c0fd510 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b12c86700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c1079c0 con 0x7f4b0c0f7400 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b12c86700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c107b80 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 0x7f4b0c10d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b11223700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 0x7f4b0c10d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:32994/0 (socket says 172.21.15.59:32994) 2024-02-18T00:47:23.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b11223700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c107360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43248/0 (socket says 172.21.15.59:43248) 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 -- 172.21.15.59:0/578997663 learned_addr learned my addr 172.21.15.59:0/578997663 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b10a22700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b0c0f7400 0x7f4b0c10d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 -- 172.21.15.59:0/578997663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b0c0f7400 msgr2=0x7f4b0c10d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b0c0f7400 0x7f4b0c10d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 -- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 msgr2=0x7f4b0c107360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c107360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 -- 172.21.15.59:0/578997663 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00005040 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b11223700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b10a22700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b0c0f7400 0x7f4b0c10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b0bfff700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 0x7f4b0c10d6b0 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7f4af8009b10 tx=0x7f4af8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b09ffb700 10 client.?.objecter ms_handle_connect 0x7f4b0c0fcbb0 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b09ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b09ffb700 1 -- 172.21.15.59:0/578997663 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8018070 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af8005040 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b09ffb700 1 -- 172.21.15.59:0/578997663 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af8013e70 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.200+0000 7f4b09ffb700 1 -- 172.21.15.59:0/578997663 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af801ddb0 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.201+0000 7f4b12c86700 10 client.41981.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.201+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c108020 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.201+0000 7f4b12c86700 1 librados: init done 2024-02-18T00:47:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c108210 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b09ffb700 1 -- 172.21.15.59:0/578997663 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4af8009070 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b09ffb700 1 --2- 172.21.15.59:0/578997663 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4afc062c90 0x7f4afc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b09ffb700 1 -- 172.21.15.59:0/578997663 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(836..836 src has 418..836) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f4af807e450 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b09ffb700 10 client.41981.objecter ms_dispatch 0x7f4b0c001550 osd_map(836..836 src has 418..836) v4 2024-02-18T00:47:23.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b09ffb700 3 client.41981.objecter handle_osd_map got epochs [836,836] > 0 2024-02-18T00:47:23.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b09ffb700 3 client.41981.objecter handle_osd_map decoding full epoch 836 2024-02-18T00:47:23.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.202+0000 7f4b10a22700 1 --2- 172.21.15.59:0/578997663 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4afc062c90 0x7f4afc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:23.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.203+0000 7f4b09ffb700 20 client.41981.objecter dump_active .. 0 homeless 2024-02-18T00:47:23.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.203+0000 7f4b09ffb700 10 client.41981.objecter ms_handle_connect 0x7f4afc062c90 2024-02-18T00:47:23.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.203+0000 7f4b10a22700 1 --2- 172.21.15.59:0/578997663 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4afc062c90 0x7f4afc065150 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 rx=0x7f4b00004830 tx=0x7f4b00009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:23.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.207+0000 7f4b09ffb700 1 -- 172.21.15.59:0/578997663 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4af8051b20 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:21.719398+0000 mgr.y (mgr.24454) 731 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:21.741833+0000 osd.3 (osd.3) 451 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:21.743059+0000 osd.3 (osd.3) 452 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:21.836172+0000 osd.1 (osd.1) 415 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:21.839226+0000 osd.1 (osd.1) 416 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:22.053448+0000 mgr.y (mgr.24454) 732 : cluster [DBG] pgmap v1305: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 840 B/s rd, 0 op/s 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:21.719398+0000 mgr.y (mgr.24454) 731 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:21.741833+0000 osd.3 (osd.3) 451 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:21.743059+0000 osd.3 (osd.3) 452 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:21.836172+0000 osd.1 (osd.1) 415 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:47:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:21.839226+0000 osd.1 (osd.1) 416 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:47:23.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:22.053448+0000 mgr.y (mgr.24454) 732 : cluster [DBG] pgmap v1305: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 840 B/s rd, 0 op/s 2024-02-18T00:47:23.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.393+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f4b0c05ee80 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:21.719398+0000 mgr.y (mgr.24454) 731 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:23.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:21.741833+0000 osd.3 (osd.3) 451 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:47:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:21.743059+0000 osd.3 (osd.3) 452 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:47:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:21.836172+0000 osd.1 (osd.1) 415 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:47:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:21.839226+0000 osd.1 (osd.1) 416 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:47:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:22.053448+0000 mgr.y (mgr.24454) 732 : cluster [DBG] pgmap v1305: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 840 B/s rd, 0 op/s 2024-02-18T00:47:23.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.865+0000 7f4b09ffb700 1 -- 172.21.15.59:0/578997663 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v837) v1 ==== 113+0+0 (secure 0 0 0) 0x7f4af8051710 con 0x7f4b0c0fcbb0 2024-02-18T00:47:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.868+0000 7f4b12c86700 10 librados: watch_flush enter 2024-02-18T00:47:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.868+0000 7f4b12c86700 10 librados: watch_flush exit 2024-02-18T00:47:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.868+0000 7f4b12c86700 20 client.41981.objecter shutdown clearing up homeless session... 2024-02-18T00:47:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.868+0000 7f4b12c86700 10 client.41981.objecter successfully canceled tick 2024-02-18T00:47:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4afc062c90 msgr2=0x7f4afc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 --2- 172.21.15.59:0/578997663 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4afc062c90 0x7f4afc065150 secure :-1 s=READY pgs=3259 cs=0 l=1 rev1=1 rx=0x7f4b00004830 tx=0x7f4b00009cb0).stop 2024-02-18T00:47:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 msgr2=0x7f4b0c10d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 0x7f4b0c10d6b0 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 rx=0x7f4af8009b10 tx=0x7f4af8009cb0).stop 2024-02-18T00:47:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 shutdown_connections 2024-02-18T00:47:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 --2- 172.21.15.59:0/578997663 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4afc062c90 0x7f4afc065150 unknown :-1 s=CLOSED pgs=3259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b0c0f7400 0x7f4b0c10d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b0c0fd510 0x7f4b0c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 --2- 172.21.15.59:0/578997663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4b0c0fcbb0 0x7f4b0c10d6b0 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 >> 172.21.15.59:0/578997663 conn(0x7f4b0c0f2820 msgr2=0x7f4b0c0f4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 shutdown_connections 2024-02-18T00:47:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 -- 172.21.15.59:0/578997663 wait complete. 2024-02-18T00:47:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:23.869+0000 7f4b12c86700 1 librados: shutdown 2024-02-18T00:47:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.5 to 1 (10000) 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:22.778994+0000 osd.3 (osd.3) 453 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:22.780184+0000 osd.3 (osd.3) 454 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:22.796951+0000 osd.1 (osd.1) 417 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:22.798452+0000 osd.1 (osd.1) 418 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:23.394440+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.59:0/578997663' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:23.394713+0000 mon.a (mon.0) 3831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:23.863492+0000 mon.a (mon.0) 3832 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:47:24.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:23.863593+0000 mon.a (mon.0) 3833 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:22.778994+0000 osd.3 (osd.3) 453 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:22.780184+0000 osd.3 (osd.3) 454 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:22.796951+0000 osd.1 (osd.1) 417 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:22.798452+0000 osd.1 (osd.1) 418 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:23.394440+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.59:0/578997663' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:23.394713+0000 mon.a (mon.0) 3831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:23.863492+0000 mon.a (mon.0) 3832 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:47:24.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:23.863593+0000 mon.a (mon.0) 3833 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:22.778994+0000 osd.3 (osd.3) 453 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:22.780184+0000 osd.3 (osd.3) 454 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:22.796951+0000 osd.1 (osd.1) 417 : cluster [DBG] 4.26 scrub starts 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:22.798452+0000 osd.1 (osd.1) 418 : cluster [DBG] 4.26 scrub ok 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:23.394440+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.59:0/578997663' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:23.394713+0000 mon.a (mon.0) 3831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:23.863492+0000 mon.a (mon.0) 3832 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:47:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:23.863593+0000 mon.a (mon.0) 3833 : cluster [DBG] osdmap e837: 8 total, 8 up, 8 in 2024-02-18T00:47:24.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 5 1 2024-02-18T00:47:25.142 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:25.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:24.053794+0000 mgr.y (mgr.24454) 733 : cluster [DBG] pgmap v1307: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-02-18T00:47:25.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:24.053794+0000 mgr.y (mgr.24454) 733 : cluster [DBG] pgmap v1307: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-02-18T00:47:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:24.053794+0000 mgr.y (mgr.24454) 733 : cluster [DBG] pgmap v1307: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 723 B/s rd, 0 op/s 2024-02-18T00:47:26.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:24.573067+0000 osd.5 (osd.5) 400 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:24.576593+0000 osd.5 (osd.5) 401 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:24.848365+0000 osd.1 (osd.1) 419 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:24.851580+0000 osd.1 (osd.1) 420 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.435354+0000 osd.5 (osd.5) 402 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.435515+0000 osd.5 (osd.5) 403 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.558900+0000 osd.4 (osd.4) 469 : cluster [DBG] 4.d scrub starts 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.562804+0000 osd.4 (osd.4) 470 : cluster [DBG] 4.d scrub ok 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.705414+0000 osd.2 (osd.2) 361 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:47:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.707324+0000 osd.2 (osd.2) 362 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:24.573067+0000 osd.5 (osd.5) 400 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:24.576593+0000 osd.5 (osd.5) 401 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:24.848365+0000 osd.1 (osd.1) 419 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:24.851580+0000 osd.1 (osd.1) 420 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.435354+0000 osd.5 (osd.5) 402 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.435515+0000 osd.5 (osd.5) 403 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.558900+0000 osd.4 (osd.4) 469 : cluster [DBG] 4.d scrub starts 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.562804+0000 osd.4 (osd.4) 470 : cluster [DBG] 4.d scrub ok 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.705414+0000 osd.2 (osd.2) 361 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:47:26.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.707324+0000 osd.2 (osd.2) 362 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:24.573067+0000 osd.5 (osd.5) 400 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:24.576593+0000 osd.5 (osd.5) 401 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:24.848365+0000 osd.1 (osd.1) 419 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:24.851580+0000 osd.1 (osd.1) 420 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.435354+0000 osd.5 (osd.5) 402 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.435515+0000 osd.5 (osd.5) 403 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.558900+0000 osd.4 (osd.4) 469 : cluster [DBG] 4.d scrub starts 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.562804+0000 osd.4 (osd.4) 470 : cluster [DBG] 4.d scrub ok 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.705414+0000 osd.2 (osd.2) 361 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:47:26.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.707324+0000 osd.2 (osd.2) 362 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:47:27.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 -- 172.21.15.59:0/62836472 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 msgr2=0x7f4fe00f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 --2- 172.21.15.59:0/62836472 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 0x7f4fe00f7d40 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 rx=0x7f4fd00047b0 tx=0x7f4fd0009cb0).stop 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 -- 172.21.15.59:0/62836472 shutdown_connections 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 --2- 172.21.15.59:0/62836472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe00f8420 0x7f4fe0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 --2- 172.21.15.59:0/62836472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00fdbb0 0x7f4fe00fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 --2- 172.21.15.59:0/62836472 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 0x7f4fe00f7d40 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 -- 172.21.15.59:0/62836472 >> 172.21.15.59:0/62836472 conn(0x7f4fe00f3800 msgr2=0x7f4fe00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 -- 172.21.15.59:0/62836472 shutdown_connections 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 -- 172.21.15.59:0/62836472 wait complete. 2024-02-18T00:47:27.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 librados: starting msgr at 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 librados: starting objecter 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 Processor -- start 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 -- start start 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 librados: setting wanted keys 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.112+0000 7f4fe841a700 1 librados: calling monclient init 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00f8420 0x7f4fe006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe00fdbb0 0x7f4fe006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 0x7f4fe006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01165b0 con 0x7f4fe00f8420 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0116720 con 0x7f4fe00fdbb0 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01168e0 con 0x7f4fe00fe510 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe61b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00f8420 0x7f4fe006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe69b7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 0x7f4fe006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:27.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe61b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00f8420 0x7f4fe006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43254/0 (socket says 172.21.15.59:43254) 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe61b6700 1 -- 172.21.15.59:0/3969172542 learned_addr learned my addr 172.21.15.59:0/3969172542 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe59b5700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe00fdbb0 0x7f4fe006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe69b7700 1 -- 172.21.15.59:0/3969172542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe00fdbb0 msgr2=0x7f4fe006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe69b7700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe00fdbb0 0x7f4fe006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe69b7700 1 -- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00f8420 msgr2=0x7f4fe006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe69b7700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00f8420 0x7f4fe006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe69b7700 1 -- 172.21.15.59:0/3969172542 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc005040 con 0x7f4fe00fe510 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe59b5700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe00fdbb0 0x7f4fe006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe61b6700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00f8420 0x7f4fe006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe69b7700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 0x7f4fe006bab0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7f4fd800c7a0 tx=0x7f4fd8007d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fd77fe700 10 client.?.objecter ms_handle_connect 0x7f4fe00fe510 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fd77fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:27.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fd77fe700 1 -- 172.21.15.59:0/3969172542 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8010510 con 0x7f4fe00fe510 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fd77fe700 1 -- 172.21.15.59:0/3969172542 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd8010b40 con 0x7f4fe00fe510 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd0005040 con 0x7f4fe00fe510 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fd77fe700 1 -- 172.21.15.59:0/3969172542 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8012b10 con 0x7f4fe00fe510 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 10 client.41987.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe0116fa0 con 0x7f4fe00fe510 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.113+0000 7f4fe841a700 1 librados: init done 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.114+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe0107110 con 0x7f4fe00fe510 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.115+0000 7f4fd77fe700 1 -- 172.21.15.59:0/3969172542 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f4fd8010670 con 0x7f4fe00fe510 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.115+0000 7f4fd77fe700 1 --2- 172.21.15.59:0/3969172542 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fcc062c90 0x7f4fcc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.115+0000 7f4fe61b6700 1 --2- 172.21.15.59:0/3969172542 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fcc062c90 0x7f4fcc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:27.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.115+0000 7f4fd77fe700 1 -- 172.21.15.59:0/3969172542 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(838..838 src has 418..838) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f4fd807d0b0 con 0x7f4fe00fe510 2024-02-18T00:47:27.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.115+0000 7f4fd77fe700 10 client.41987.objecter ms_dispatch 0x7f4fe0001550 osd_map(838..838 src has 418..838) v4 2024-02-18T00:47:27.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.115+0000 7f4fd77fe700 3 client.41987.objecter handle_osd_map got epochs [838,838] > 0 2024-02-18T00:47:27.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.115+0000 7f4fd77fe700 3 client.41987.objecter handle_osd_map decoding full epoch 838 2024-02-18T00:47:27.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.116+0000 7f4fd77fe700 20 client.41987.objecter dump_active .. 0 homeless 2024-02-18T00:47:27.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.116+0000 7f4fd77fe700 10 client.41987.objecter ms_handle_connect 0x7f4fcc062c90 2024-02-18T00:47:27.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.116+0000 7f4fe61b6700 1 --2- 172.21.15.59:0/3969172542 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fcc062c90 0x7f4fcc065150 secure :-1 s=READY pgs=3260 cs=0 l=1 rev1=1 rx=0x7f4fdc004d40 tx=0x7f4fdc00b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:27.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.120+0000 7f4fd77fe700 1 -- 172.21.15.59:0/3969172542 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4fd8050780 con 0x7f4fe00fe510 2024-02-18T00:47:27.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.112003+0000 osd.0 (osd.0) 390 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.113233+0000 osd.0 (osd.0) 391 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.609884+0000 osd.5 (osd.5) 404 : cluster [DBG] 5.f scrub starts 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.612890+0000 osd.5 (osd.5) 405 : cluster [DBG] 5.f scrub ok 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.750458+0000 osd.3 (osd.3) 455 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:25.751586+0000 osd.3 (osd.3) 456 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:26.019013+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:26.054400+0000 mgr.y (mgr.24454) 734 : cluster [DBG] pgmap v1309: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-18T00:47:27.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.112003+0000 osd.0 (osd.0) 390 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:47:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.113233+0000 osd.0 (osd.0) 391 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:47:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.609884+0000 osd.5 (osd.5) 404 : cluster [DBG] 5.f scrub starts 2024-02-18T00:47:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.612890+0000 osd.5 (osd.5) 405 : cluster [DBG] 5.f scrub ok 2024-02-18T00:47:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.750458+0000 osd.3 (osd.3) 455 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:47:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:25.751586+0000 osd.3 (osd.3) 456 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:47:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:26.019013+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-02-18T00:47:27.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:26.054400+0000 mgr.y (mgr.24454) 734 : cluster [DBG] pgmap v1309: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-18T00:47:27.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:27.307+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) v1 -- 0x7f4fe005fa60 con 0x7f4fe00fe510 2024-02-18T00:47:27.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.112003+0000 osd.0 (osd.0) 390 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.113233+0000 osd.0 (osd.0) 391 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.609884+0000 osd.5 (osd.5) 404 : cluster [DBG] 5.f scrub starts 2024-02-18T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.612890+0000 osd.5 (osd.5) 405 : cluster [DBG] 5.f scrub ok 2024-02-18T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.750458+0000 osd.3 (osd.3) 455 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:25.751586+0000 osd.3 (osd.3) 456 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:26.019013+0000 mon.a (mon.0) 3834 : cluster [DBG] osdmap e838: 8 total, 8 up, 8 in 2024-02-18T00:47:27.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:26.054400+0000 mgr.y (mgr.24454) 734 : cluster [DBG] pgmap v1309: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-18T00:47:28.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.038+0000 7f4fd77fe700 1 -- 172.21.15.59:0/3969172542 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v839) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4fd8050370 con 0x7f4fe00fe510 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 10 librados: watch_flush enter 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 10 librados: watch_flush exit 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 20 client.41987.objecter shutdown clearing up homeless session... 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 10 client.41987.objecter successfully canceled tick 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fcc062c90 msgr2=0x7f4fcc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 --2- 172.21.15.59:0/3969172542 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fcc062c90 0x7f4fcc065150 secure :-1 s=READY pgs=3260 cs=0 l=1 rev1=1 rx=0x7f4fdc004d40 tx=0x7f4fdc00b040).stop 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 msgr2=0x7f4fe006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 0x7f4fe006bab0 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 rx=0x7f4fd800c7a0 tx=0x7f4fd8007d10).stop 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 shutdown_connections 2024-02-18T00:47:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4fe00fe510 0x7f4fe006bab0 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4fe00f8420 0x7f4fe006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 --2- 172.21.15.59:0/3969172542 >> v2:172.21.15.59:6800/3721891797 conn(0x7f4fcc062c90 0x7f4fcc065150 unknown :-1 s=CLOSED pgs=3260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 --2- 172.21.15.59:0/3969172542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe00fdbb0 0x7f4fe006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 >> 172.21.15.59:0/3969172542 conn(0x7f4fe00f3800 msgr2=0x7f4fe00f5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 shutdown_connections 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 -- 172.21.15.59:0/3969172542 wait complete. 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:28.041+0000 7f4fe841a700 1 librados: shutdown 2024-02-18T00:47:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-02-18T00:47:28.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:26.709882+0000 osd.3 (osd.3) 457 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:47:28.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:26.710727+0000 osd.3 (osd.3) 458 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:47:28.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:27.308372+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.59:0/3969172542' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:28.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:27.308640+0000 mon.a (mon.0) 3835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:28.481 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:26.709882+0000 osd.3 (osd.3) 457 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:47:28.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:26.710727+0000 osd.3 (osd.3) 458 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:47:28.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:27.308372+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.59:0/3969172542' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:28.482 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:27.308640+0000 mon.a (mon.0) 3835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:28.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:26.709882+0000 osd.3 (osd.3) 457 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:47:28.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:26.710727+0000 osd.3 (osd.3) 458 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:47:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:27.308372+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.59:0/3969172542' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:28.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:27.308640+0000 mon.a (mon.0) 3835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-02-18T00:47:28.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 6 1 2024-02-18T00:47:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:27.584125+0000 osd.5 (osd.5) 406 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:47:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:27.585655+0000 osd.5 (osd.5) 407 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:47:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:28.036015+0000 mon.a (mon.0) 3836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:47:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:28.036107+0000 mon.a (mon.0) 3837 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-02-18T00:47:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:28.054819+0000 mgr.y (mgr.24454) 735 : cluster [DBG] pgmap v1311: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:28.065687+0000 osd.7 (osd.7) 287 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:47:29.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:28.067574+0000 osd.7 (osd.7) 288 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:47:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:27.584125+0000 osd.5 (osd.5) 406 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:47:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:27.585655+0000 osd.5 (osd.5) 407 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:47:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:28.036015+0000 mon.a (mon.0) 3836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:47:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:28.036107+0000 mon.a (mon.0) 3837 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-02-18T00:47:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:28.054819+0000 mgr.y (mgr.24454) 735 : cluster [DBG] pgmap v1311: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:28.065687+0000 osd.7 (osd.7) 287 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:47:29.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:28.067574+0000 osd.7 (osd.7) 288 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:47:29.277 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:27.584125+0000 osd.5 (osd.5) 406 : cluster [DBG] 4.8 scrub starts 2024-02-18T00:47:29.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:27.585655+0000 osd.5 (osd.5) 407 : cluster [DBG] 4.8 scrub ok 2024-02-18T00:47:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:28.036015+0000 mon.a (mon.0) 3836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-02-18T00:47:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:28.036107+0000 mon.a (mon.0) 3837 : cluster [DBG] osdmap e839: 8 total, 8 up, 8 in 2024-02-18T00:47:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:28.054819+0000 mgr.y (mgr.24454) 735 : cluster [DBG] pgmap v1311: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:28.065687+0000 osd.7 (osd.7) 287 : cluster [DBG] 5.1a scrub starts 2024-02-18T00:47:29.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:28.067574+0000 osd.7 (osd.7) 288 : cluster [DBG] 5.1a scrub ok 2024-02-18T00:47:30.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:29.692046+0000 osd.2 (osd.2) 363 : cluster [DBG] 5.d scrub starts 2024-02-18T00:47:30.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:29.693492+0000 osd.2 (osd.2) 364 : cluster [DBG] 5.d scrub ok 2024-02-18T00:47:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:29.692046+0000 osd.2 (osd.2) 363 : cluster [DBG] 5.d scrub starts 2024-02-18T00:47:30.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:29.693492+0000 osd.2 (osd.2) 364 : cluster [DBG] 5.d scrub ok 2024-02-18T00:47:30.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:29.692046+0000 osd.2 (osd.2) 363 : cluster [DBG] 5.d scrub starts 2024-02-18T00:47:30.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:29.693492+0000 osd.2 (osd.2) 364 : cluster [DBG] 5.d scrub ok 2024-02-18T00:47:31.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:30.055456+0000 mgr.y (mgr.24454) 736 : cluster [DBG] pgmap v1312: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:31.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:30.055456+0000 mgr.y (mgr.24454) 736 : cluster [DBG] pgmap v1312: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:31.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:47:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:47:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:30.055456+0000 mgr.y (mgr.24454) 736 : cluster [DBG] pgmap v1312: 174 pgs: 174 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:31.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 -- 172.21.15.59:0/3227862079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fe510 msgr2=0x7f89b80f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:31.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/3227862079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fe510 0x7f89b80f7d40 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 rx=0x7f89a40047b0 tx=0x7f89a4009cb0).stop 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 -- 172.21.15.59:0/3227862079 shutdown_connections 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/3227862079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b80fdbb0 0x7f89b80fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/3227862079 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89b80f8420 0x7f89b8104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/3227862079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fe510 0x7f89b80f7d40 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 -- 172.21.15.59:0/3227862079 >> 172.21.15.59:0/3227862079 conn(0x7f89b80f3800 msgr2=0x7f89b80f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 -- 172.21.15.59:0/3227862079 shutdown_connections 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.541+0000 7f89bf0f5700 1 -- 172.21.15.59:0/3227862079 wait complete. 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 librados: starting msgr at 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 librados: starting objecter 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 Processor -- start 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 -- start start 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 librados: setting wanted keys 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 librados: calling monclient init 2024-02-18T00:47:31.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b80f8420 0x7f89b806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fdbb0 0x7f89b806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89b80fe510 0x7f89b806bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81165d0 con 0x7f89b80fdbb0 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b8116740 con 0x7f89b80f8420 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bf0f5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b8116900 con 0x7f89b80fe510 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fdbb0 0x7f89b806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89bd692700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89b80fe510 0x7f89b806bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89b7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fdbb0 0x7f89b806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51058/0 (socket says 172.21.15.59:51058) 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.542+0000 7f89b7fff700 1 -- 172.21.15.59:0/1964276546 learned_addr learned my addr 172.21.15.59:0/1964276546 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89bce91700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b80f8420 0x7f89b806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b7fff700 1 -- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89b80fe510 msgr2=0x7f89b806bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b7fff700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89b80fe510 0x7f89b806bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:31.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b7fff700 1 -- 172.21.15.59:0/1964276546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b80f8420 msgr2=0x7f89b806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b7fff700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b80f8420 0x7f89b806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b7fff700 1 -- 172.21.15.59:0/1964276546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00097c0 con 0x7f89b80fdbb0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89bd692700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89b80fe510 0x7f89b806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89bce91700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b80f8420 0x7f89b806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b7fff700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fdbb0 0x7f89b806b5c0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 rx=0x7f89a40044e0 tx=0x7f89a4007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b5ffb700 10 client.?.objecter ms_handle_connect 0x7f89b80fdbb0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b5ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b5ffb700 1 -- 172.21.15.59:0/1964276546 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4018070 con 0x7f89b80fdbb0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b5ffb700 1 -- 172.21.15.59:0/1964276546 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89a40093b0 con 0x7f89b80fdbb0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89b5ffb700 1 -- 172.21.15.59:0/1964276546 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a401de40 con 0x7f89b80fdbb0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a4005040 con 0x7f89b80fdbb0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89bf0f5700 10 client.32190.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b8116da0 con 0x7f89b80fdbb0 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.543+0000 7f89bf0f5700 1 librados: init done 2024-02-18T00:47:31.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.544+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b805ee80 con 0x7f89b80fdbb0 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.545+0000 7f89b5ffb700 1 -- 172.21.15.59:0/1964276546 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f89a4013ad0 con 0x7f89b80fdbb0 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89b5ffb700 1 --2- 172.21.15.59:0/1964276546 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89a8062c90 0x7f89a8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89b5ffb700 1 -- 172.21.15.59:0/1964276546 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(839..839 src has 418..839) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f89a407e9b0 con 0x7f89b80fdbb0 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89b5ffb700 10 client.32190.objecter ms_dispatch 0x7f89b8001550 osd_map(839..839 src has 418..839) v4 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89b5ffb700 3 client.32190.objecter handle_osd_map got epochs [839,839] > 0 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89b5ffb700 3 client.32190.objecter handle_osd_map decoding full epoch 839 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89bce91700 1 --2- 172.21.15.59:0/1964276546 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89a8062c90 0x7f89a8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89b5ffb700 20 client.32190.objecter dump_active .. 0 homeless 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89b5ffb700 10 client.32190.objecter ms_handle_connect 0x7f89a8062c90 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.546+0000 7f89bce91700 1 --2- 172.21.15.59:0/1964276546 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89a8062c90 0x7f89a8065150 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 rx=0x7f89ac007fd0 tx=0x7f89ac009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:31.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.550+0000 7f89b5ffb700 1 -- 172.21.15.59:0/1964276546 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f89a4051fe0 con 0x7f89b80fdbb0 2024-02-18T00:47:31.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:31.736+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) v1 -- 0x7f89b805fa60 con 0x7f89b80fdbb0 2024-02-18T00:47:32.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:47:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:47:32.056 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.053+0000 7f89b5ffb700 1 -- 172.21.15.59:0/1964276546 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v840) v1 ==== 113+0+0 (secure 0 0 0) 0x7f89a4051bd0 con 0x7f89b80fdbb0 2024-02-18T00:47:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 10 librados: watch_flush enter 2024-02-18T00:47:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 10 librados: watch_flush exit 2024-02-18T00:47:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 20 client.32190.objecter shutdown clearing up homeless session... 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 10 client.32190.objecter successfully canceled tick 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89a8062c90 msgr2=0x7f89a8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/1964276546 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89a8062c90 0x7f89a8065150 secure :-1 s=READY pgs=3261 cs=0 l=1 rev1=1 rx=0x7f89ac007fd0 tx=0x7f89ac009040).stop 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fdbb0 msgr2=0x7f89b806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fdbb0 0x7f89b806b5c0 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 rx=0x7f89a40044e0 tx=0x7f89a4007f90).stop 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 shutdown_connections 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/1964276546 >> v2:172.21.15.59:6800/3721891797 conn(0x7f89a8062c90 0x7f89a8065150 unknown :-1 s=CLOSED pgs=3261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b80f8420 0x7f89b806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89b80fe510 0x7f89b806bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 --2- 172.21.15.59:0/1964276546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89b80fdbb0 0x7f89b806b5c0 unknown :-1 s=CLOSED pgs=2730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 >> 172.21.15.59:0/1964276546 conn(0x7f89b80f3800 msgr2=0x7f89b80f5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 shutdown_connections 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 -- 172.21.15.59:0/1964276546 wait complete. 2024-02-18T00:47:32.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:32.056+0000 7f89bf0f5700 1 librados: shutdown 2024-02-18T00:47:32.059 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.6 to 1 (10000) 2024-02-18T00:47:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:30.648193+0000 osd.3 (osd.3) 459 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:47:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:30.649514+0000 osd.3 (osd.3) 460 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:47:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:30.835574+0000 osd.1 (osd.1) 421 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:47:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:30.836604+0000 osd.1 (osd.1) 422 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:47:32.404 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:31.737749+0000 mon.a (mon.0) 3838 : audit [INF] from='client.? 172.21.15.59:0/1964276546' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:32.404 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:30.648193+0000 osd.3 (osd.3) 459 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:47:32.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:30.649514+0000 osd.3 (osd.3) 460 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:47:32.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:30.835574+0000 osd.1 (osd.1) 421 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:47:32.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:30.836604+0000 osd.1 (osd.1) 422 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:47:32.405 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:31.737749+0000 mon.a (mon.0) 3838 : audit [INF] from='client.? 172.21.15.59:0/1964276546' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:32.405 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:47:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:32] "GET /metrics HTTP/1.1" 200 187529 "" "Prometheus/2.33.4" 2024-02-18T00:47:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:30.648193+0000 osd.3 (osd.3) 459 : cluster [DBG] 4.10 scrub starts 2024-02-18T00:47:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:30.649514+0000 osd.3 (osd.3) 460 : cluster [DBG] 4.10 scrub ok 2024-02-18T00:47:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:30.835574+0000 osd.1 (osd.1) 421 : cluster [DBG] 2.6 scrub starts 2024-02-18T00:47:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:30.836604+0000 osd.1 (osd.1) 422 : cluster [DBG] 2.6 scrub ok 2024-02-18T00:47:32.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:31.737749+0000 mon.a (mon.0) 3838 : audit [INF] from='client.? 172.21.15.59:0/1964276546' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:32.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 6 1 2024-02-18T00:47:33.204 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:33.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:31.730354+0000 mgr.y (mgr.24454) 737 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:32.054089+0000 mon.a (mon.0) 3839 : audit [INF] from='client.? 172.21.15.59:0/1964276546' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:47:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:32.054162+0000 mon.a (mon.0) 3840 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-02-18T00:47:33.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:32.055850+0000 mgr.y (mgr.24454) 738 : cluster [DBG] pgmap v1314: 174 pgs: 1 active+clean+scrubbing, 173 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 847 B/s rd, 0 op/s 2024-02-18T00:47:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:31.730354+0000 mgr.y (mgr.24454) 737 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:32.054089+0000 mon.a (mon.0) 3839 : audit [INF] from='client.? 172.21.15.59:0/1964276546' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:47:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:32.054162+0000 mon.a (mon.0) 3840 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-02-18T00:47:33.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:32.055850+0000 mgr.y (mgr.24454) 738 : cluster [DBG] pgmap v1314: 174 pgs: 1 active+clean+scrubbing, 173 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 847 B/s rd, 0 op/s 2024-02-18T00:47:33.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:31.730354+0000 mgr.y (mgr.24454) 737 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:33.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:32.054089+0000 mon.a (mon.0) 3839 : audit [INF] from='client.? 172.21.15.59:0/1964276546' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:47:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:32.054162+0000 mon.a (mon.0) 3840 : cluster [DBG] osdmap e840: 8 total, 8 up, 8 in 2024-02-18T00:47:33.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:32.055850+0000 mgr.y (mgr.24454) 738 : cluster [DBG] pgmap v1314: 174 pgs: 1 active+clean+scrubbing, 173 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 847 B/s rd, 0 op/s 2024-02-18T00:47:34.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:32.434442+0000 osd.6 (osd.6) 379 : cluster [DBG] 4.a deep-scrub starts 2024-02-18T00:47:34.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:32.441690+0000 osd.6 (osd.6) 380 : cluster [DBG] 4.a deep-scrub ok 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:32.565780+0000 osd.5 (osd.5) 408 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:32.568655+0000 osd.5 (osd.5) 409 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:33.863531+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:32.434442+0000 osd.6 (osd.6) 379 : cluster [DBG] 4.a deep-scrub starts 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:32.441690+0000 osd.6 (osd.6) 380 : cluster [DBG] 4.a deep-scrub ok 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:32.565780+0000 osd.5 (osd.5) 408 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:32.568655+0000 osd.5 (osd.5) 409 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:47:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:33.863531+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-02-18T00:47:34.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:32.434442+0000 osd.6 (osd.6) 379 : cluster [DBG] 4.a deep-scrub starts 2024-02-18T00:47:34.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:32.441690+0000 osd.6 (osd.6) 380 : cluster [DBG] 4.a deep-scrub ok 2024-02-18T00:47:34.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:32.565780+0000 osd.5 (osd.5) 408 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:47:34.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:32.568655+0000 osd.5 (osd.5) 409 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:47:34.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:33.863531+0000 mon.a (mon.0) 3841 : cluster [DBG] osdmap e841: 8 total, 8 up, 8 in 2024-02-18T00:47:35.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:34.056472+0000 mgr.y (mgr.24454) 739 : cluster [DBG] pgmap v1316: 174 pgs: 1 active+clean+scrubbing, 173 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 850 B/s rd, 0 op/s 2024-02-18T00:47:35.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:34.056472+0000 mgr.y (mgr.24454) 739 : cluster [DBG] pgmap v1316: 174 pgs: 1 active+clean+scrubbing, 173 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 850 B/s rd, 0 op/s 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 -- 172.21.15.59:0/2447861621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fd510 msgr2=0x7fafac0f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 --2- 172.21.15.59:0/2447861621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fd510 0x7fafac0f6d40 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 rx=0x7faf9c0047b0 tx=0x7faf9c009cb0).stop 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 -- 172.21.15.59:0/2447861621 shutdown_connections 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 --2- 172.21.15.59:0/2447861621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac0f7400 0x7fafac103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 --2- 172.21.15.59:0/2447861621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafac0fcbb0 0x7fafac0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 --2- 172.21.15.59:0/2447861621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fd510 0x7fafac0f6d40 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 -- 172.21.15.59:0/2447861621 >> 172.21.15.59:0/2447861621 conn(0x7fafac0f2820 msgr2=0x7fafac0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 -- 172.21.15.59:0/2447861621 shutdown_connections 2024-02-18T00:47:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 -- 172.21.15.59:0/2447861621 wait complete. 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 librados: starting msgr at 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.219+0000 7fafb3feb700 1 librados: starting objecter 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 Processor -- start 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 -- start start 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 librados: setting wanted keys 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 librados: calling monclient init 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafac0f7400 0x7fafac109300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fcbb0 0x7fafac1097f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac0fd510 0x7fafac109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac1144e0 con 0x7fafac0f7400 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac114650 con 0x7fafac0fd510 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb3feb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac114810 con 0x7fafac0fcbb0 2024-02-18T00:47:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fcbb0 0x7fafac1097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fcbb0 0x7fafac1097f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:33436/0 (socket says 172.21.15.59:33436) 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 -- 172.21.15.59:0/3297083316 learned_addr learned my addr 172.21.15.59:0/3297083316 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1d87700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafac0f7400 0x7fafac109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb2588700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac0fd510 0x7fafac109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 -- 172.21.15.59:0/3297083316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac0fd510 msgr2=0x7fafac109dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac0fd510 0x7fafac109dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 -- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafac0f7400 msgr2=0x7fafac109300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafac0f7400 0x7fafac109300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 -- 172.21.15.59:0/3297083316 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa8005040 con 0x7fafac0fcbb0 2024-02-18T00:47:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafb1586700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fcbb0 0x7fafac1097f0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 rx=0x7faf9c009b10 tx=0x7faf9c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafa2ffd700 10 client.?.objecter ms_handle_connect 0x7fafac0fcbb0 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.220+0000 7fafa2ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafa2ffd700 1 -- 172.21.15.59:0/3297083316 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c018070 con 0x7fafac0fcbb0 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c005040 con 0x7fafac0fcbb0 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafa2ffd700 1 -- 172.21.15.59:0/3297083316 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf9c013e70 con 0x7fafac0fcbb0 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafa2ffd700 1 -- 172.21.15.59:0/3297083316 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c01ddb0 con 0x7fafac0fcbb0 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafb3feb700 10 client.41999.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac114d00 con 0x7fafac0fcbb0 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafb3feb700 1 librados: init done 2024-02-18T00:47:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.221+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac05fa60 con 0x7fafac0fcbb0 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.222+0000 7fafa2ffd700 1 -- 172.21.15.59:0/3297083316 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7faf9c01d420 con 0x7fafac0fcbb0 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.222+0000 7fafa2ffd700 1 --2- 172.21.15.59:0/3297083316 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf98062c90 0x7faf98065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.222+0000 7fafa2ffd700 1 -- 172.21.15.59:0/3297083316 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(841..841 src has 418..841) v4 ==== 7613+0+0 (secure 0 0 0) 0x7faf9c07e480 con 0x7fafac0fcbb0 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.222+0000 7fafa2ffd700 10 client.41999.objecter ms_dispatch 0x7fafac001550 osd_map(841..841 src has 418..841) v4 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.222+0000 7fafa2ffd700 3 client.41999.objecter handle_osd_map got epochs [841,841] > 0 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.222+0000 7fafb1d87700 1 --2- 172.21.15.59:0/3297083316 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf98062c90 0x7faf98065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.222+0000 7fafa2ffd700 3 client.41999.objecter handle_osd_map decoding full epoch 841 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.223+0000 7fafa2ffd700 20 client.41999.objecter dump_active .. 0 homeless 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.223+0000 7fafa2ffd700 10 client.41999.objecter ms_handle_connect 0x7faf98062c90 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.223+0000 7fafb1d87700 1 --2- 172.21.15.59:0/3297083316 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf98062c90 0x7faf98065150 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 rx=0x7fafa8004870 tx=0x7fafa800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:35.270 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.226+0000 7fafa2ffd700 1 -- 172.21.15.59:0/3297083316 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faf9c051b50 con 0x7fafac0fcbb0 2024-02-18T00:47:35.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:35.411+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) v1 -- 0x7fafac1060e0 con 0x7fafac0fcbb0 2024-02-18T00:47:35.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:34.056472+0000 mgr.y (mgr.24454) 739 : cluster [DBG] pgmap v1316: 174 pgs: 1 active+clean+scrubbing, 173 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 850 B/s rd, 0 op/s 2024-02-18T00:47:36.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.071+0000 7fafa2ffd700 1 -- 172.21.15.59:0/3297083316 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v842) v1 ==== 133+0+0 (secure 0 0 0) 0x7faf9c051740 con 0x7fafac0fcbb0 2024-02-18T00:47:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.074+0000 7fafb3feb700 10 librados: watch_flush enter 2024-02-18T00:47:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.074+0000 7fafb3feb700 10 librados: watch_flush exit 2024-02-18T00:47:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.074+0000 7fafb3feb700 20 client.41999.objecter shutdown clearing up homeless session... 2024-02-18T00:47:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.074+0000 7fafb3feb700 10 client.41999.objecter successfully canceled tick 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf98062c90 msgr2=0x7faf98065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 --2- 172.21.15.59:0/3297083316 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf98062c90 0x7faf98065150 secure :-1 s=READY pgs=3262 cs=0 l=1 rev1=1 rx=0x7fafa8004870 tx=0x7fafa800b040).stop 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fcbb0 msgr2=0x7fafac1097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fcbb0 0x7fafac1097f0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 rx=0x7faf9c009b10 tx=0x7faf9c009cb0).stop 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 shutdown_connections 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 --2- 172.21.15.59:0/3297083316 >> v2:172.21.15.59:6800/3721891797 conn(0x7faf98062c90 0x7faf98065150 unknown :-1 s=CLOSED pgs=3262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac0fd510 0x7fafac109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafac0f7400 0x7fafac109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 --2- 172.21.15.59:0/3297083316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafac0fcbb0 0x7fafac1097f0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 >> 172.21.15.59:0/3297083316 conn(0x7fafac0f2820 msgr2=0x7fafac0f4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 shutdown_connections 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 -- 172.21.15.59:0/3297083316 wait complete. 2024-02-18T00:47:36.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:36.075+0000 7fafb3feb700 1 librados: shutdown 2024-02-18T00:47:36.078 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-02-18T00:47:36.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:34.139417+0000 osd.0 (osd.0) 392 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:47:36.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:34.142429+0000 osd.0 (osd.0) 393 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:47:36.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:35.412578+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.59:0/3297083316' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:36.453 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:35.412914+0000 mon.a (mon.0) 3842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:36.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:34.139417+0000 osd.0 (osd.0) 392 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:47:36.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:34.142429+0000 osd.0 (osd.0) 393 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:47:36.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:35.412578+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.59:0/3297083316' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:36.454 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:35.412914+0000 mon.a (mon.0) 3842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:36.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:34.139417+0000 osd.0 (osd.0) 392 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:47:36.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:34.142429+0000 osd.0 (osd.0) 393 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:47:36.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:35.412578+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.59:0/3297083316' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:36.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:35.412914+0000 mon.a (mon.0) 3842 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-02-18T00:47:36.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd reweight 7 1 2024-02-18T00:47:37.229 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:37.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.138722+0000 osd.0 (osd.0) 394 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:47:37.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.142209+0000 osd.0 (osd.0) 395 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.622574+0000 osd.5 (osd.5) 410 : cluster [DBG] 25.f scrub starts 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.626686+0000 osd.5 (osd.5) 411 : cluster [DBG] 25.f scrub ok 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.663442+0000 osd.3 (osd.3) 461 : cluster [DBG] 5.17 deep-scrub starts 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.664367+0000 osd.3 (osd.3) 462 : cluster [DBG] 5.17 deep-scrub ok 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.732688+0000 osd.1 (osd.1) 423 : cluster [DBG] 2.d scrub starts 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:35.733486+0000 osd.1 (osd.1) 424 : cluster [DBG] 2.d scrub ok 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:36.057354+0000 mgr.y (mgr.24454) 740 : cluster [DBG] pgmap v1317: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:36.070990+0000 mon.a (mon.0) 3843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:36.071052+0000 mon.a (mon.0) 3844 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-02-18T00:47:37.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.138722+0000 osd.0 (osd.0) 394 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.142209+0000 osd.0 (osd.0) 395 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.622574+0000 osd.5 (osd.5) 410 : cluster [DBG] 25.f scrub starts 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.626686+0000 osd.5 (osd.5) 411 : cluster [DBG] 25.f scrub ok 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.663442+0000 osd.3 (osd.3) 461 : cluster [DBG] 5.17 deep-scrub starts 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.664367+0000 osd.3 (osd.3) 462 : cluster [DBG] 5.17 deep-scrub ok 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.732688+0000 osd.1 (osd.1) 423 : cluster [DBG] 2.d scrub starts 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:35.733486+0000 osd.1 (osd.1) 424 : cluster [DBG] 2.d scrub ok 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:36.057354+0000 mgr.y (mgr.24454) 740 : cluster [DBG] pgmap v1317: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:36.070990+0000 mon.a (mon.0) 3843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:47:37.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:36.071052+0000 mon.a (mon.0) 3844 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-02-18T00:47:37.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.138722+0000 osd.0 (osd.0) 394 : cluster [DBG] 5.3 scrub starts 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.142209+0000 osd.0 (osd.0) 395 : cluster [DBG] 5.3 scrub ok 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.622574+0000 osd.5 (osd.5) 410 : cluster [DBG] 25.f scrub starts 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.626686+0000 osd.5 (osd.5) 411 : cluster [DBG] 25.f scrub ok 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.663442+0000 osd.3 (osd.3) 461 : cluster [DBG] 5.17 deep-scrub starts 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.664367+0000 osd.3 (osd.3) 462 : cluster [DBG] 5.17 deep-scrub ok 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.732688+0000 osd.1 (osd.1) 423 : cluster [DBG] 2.d scrub starts 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:35.733486+0000 osd.1 (osd.1) 424 : cluster [DBG] 2.d scrub ok 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:36.057354+0000 mgr.y (mgr.24454) 740 : cluster [DBG] pgmap v1317: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:36.070990+0000 mon.a (mon.0) 3843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-02-18T00:47:37.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:36.071052+0000 mon.a (mon.0) 3844 : cluster [DBG] osdmap e842: 8 total, 8 up, 8 in 2024-02-18T00:47:38.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:36.654930+0000 osd.5 (osd.5) 412 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:36.658395+0000 osd.5 (osd.5) 413 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:36.701382+0000 osd.3 (osd.3) 463 : cluster [DBG] 6.b scrub starts 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:36.702808+0000 osd.3 (osd.3) 464 : cluster [DBG] 6.b scrub ok 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:37.473324+0000 osd.4 (osd.4) 471 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:37.477039+0000 osd.4 (osd.4) 472 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:36.654930+0000 osd.5 (osd.5) 412 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:36.658395+0000 osd.5 (osd.5) 413 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:47:38.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:36.701382+0000 osd.3 (osd.3) 463 : cluster [DBG] 6.b scrub starts 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:36.702808+0000 osd.3 (osd.3) 464 : cluster [DBG] 6.b scrub ok 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:37.473324+0000 osd.4 (osd.4) 471 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:37.477039+0000 osd.4 (osd.4) 472 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:36.654930+0000 osd.5 (osd.5) 412 : cluster [DBG] 5.15 scrub starts 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:36.658395+0000 osd.5 (osd.5) 413 : cluster [DBG] 5.15 scrub ok 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:36.701382+0000 osd.3 (osd.3) 463 : cluster [DBG] 6.b scrub starts 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:36.702808+0000 osd.3 (osd.3) 464 : cluster [DBG] 6.b scrub ok 2024-02-18T00:47:38.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:37.473324+0000 osd.4 (osd.4) 471 : cluster [DBG] 25.13 scrub starts 2024-02-18T00:47:38.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:37.477039+0000 osd.4 (osd.4) 472 : cluster [DBG] 25.13 scrub ok 2024-02-18T00:47:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 -- 172.21.15.59:0/580858826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 msgr2=0x7f80d4104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 --2- 172.21.15.59:0/580858826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d4104b10 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 rx=0x7f80cc009c30 tx=0x7f80cc004c50).stop 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 -- 172.21.15.59:0/580858826 shutdown_connections 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 --2- 172.21.15.59:0/580858826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80d40f9ab0 0x7f80d40f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 --2- 172.21.15.59:0/580858826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d4104b10 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 --2- 172.21.15.59:0/580858826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80d40fa380 0x7f80d40f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 -- 172.21.15.59:0/580858826 >> 172.21.15.59:0/580858826 conn(0x7f80d40f3800 msgr2=0x7f80d40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 -- 172.21.15.59:0/580858826 shutdown_connections 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 -- 172.21.15.59:0/580858826 wait complete. 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 librados: starting msgr at 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 librados: starting objecter 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 Processor -- start 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 -- start start 2024-02-18T00:47:39.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 librados: setting wanted keys 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.204+0000 7f80d93ad700 1 librados: calling monclient init 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80d40f9ab0 0x7f80d406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80d40fa380 0x7f80d406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d41163e0 con 0x7f80d40f8440 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4116550 con 0x7f80d40f9ab0 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d4116710 con 0x7f80d40fa380 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d406b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51102/0 (socket says 172.21.15.59:51102) 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 -- 172.21.15.59:0/3942622810 learned_addr learned my addr 172.21.15.59:0/3942622810 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d37fe700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80d40fa380 0x7f80d406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d27fc700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80d40f9ab0 0x7f80d406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 -- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80d40fa380 msgr2=0x7f80d406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80d40fa380 0x7f80d406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 -- 172.21.15.59:0/3942622810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80d40f9ab0 msgr2=0x7f80d406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80d40f9ab0 0x7f80d406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 -- 172.21.15.59:0/3942622810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c4005040 con 0x7f80d40f8440 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d27fc700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80d40f9ab0 0x7f80d406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d37fe700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80d40fa380 0x7f80d406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d2ffd700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d406b0d0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 rx=0x7f80bc0041d0 tx=0x7f80bc009f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80cbfff700 10 client.?.objecter ms_handle_connect 0x7f80d40f8440 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80cbfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80cbfff700 1 -- 172.21.15.59:0/3942622810 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc007b40 con 0x7f80d40f8440 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80cbfff700 1 -- 172.21.15.59:0/3942622810 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80bc00f040 con 0x7f80d40f8440 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc0097c0 con 0x7f80d40f8440 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80cbfff700 1 -- 172.21.15.59:0/3942622810 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc014810 con 0x7f80d40f8440 2024-02-18T00:47:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 10 client.32202.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d4116dd0 con 0x7f80d40f8440 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.205+0000 7f80d93ad700 1 librados: init done 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.206+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d405ee80 con 0x7f80d40f8440 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.207+0000 7f80cbfff700 1 -- 172.21.15.59:0/3942622810 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f80bc009110 con 0x7f80d40f8440 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.207+0000 7f80cbfff700 1 --2- 172.21.15.59:0/3942622810 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80c0062c90 0x7f80c0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.207+0000 7f80d27fc700 1 --2- 172.21.15.59:0/3942622810 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80c0062c90 0x7f80c0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.207+0000 7f80cbfff700 1 -- 172.21.15.59:0/3942622810 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(843..843 src has 418..843) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f80bc07d0c0 con 0x7f80d40f8440 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.207+0000 7f80cbfff700 10 client.32202.objecter ms_dispatch 0x7f80d4001550 osd_map(843..843 src has 418..843) v4 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.207+0000 7f80cbfff700 3 client.32202.objecter handle_osd_map got epochs [843,843] > 0 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.207+0000 7f80cbfff700 3 client.32202.objecter handle_osd_map decoding full epoch 843 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.208+0000 7f80cbfff700 20 client.32202.objecter dump_active .. 0 homeless 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.208+0000 7f80cbfff700 10 client.32202.objecter ms_handle_connect 0x7f80c0062c90 2024-02-18T00:47:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.208+0000 7f80d27fc700 1 --2- 172.21.15.59:0/3942622810 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80c0062c90 0x7f80c0065150 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 rx=0x7f80c4009f20 tx=0x7f80c4009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.212+0000 7f80cbfff700 1 -- 172.21.15.59:0/3942622810 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f80bc050790 con 0x7f80d40f8440 2024-02-18T00:47:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:37.694321+0000 osd.3 (osd.3) 465 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:47:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:37.695521+0000 osd.3 (osd.3) 466 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:47:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:37.741588+0000 osd.1 (osd.1) 425 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:47:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:37.745129+0000 osd.1 (osd.1) 426 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:47:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:38.057911+0000 mgr.y (mgr.24454) 741 : cluster [DBG] pgmap v1319: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T00:47:39.253 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:38.866890+0000 mon.a (mon.0) 3845 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-02-18T00:47:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:37.694321+0000 osd.3 (osd.3) 465 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:47:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:37.695521+0000 osd.3 (osd.3) 466 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:47:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:37.741588+0000 osd.1 (osd.1) 425 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:47:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:37.745129+0000 osd.1 (osd.1) 426 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:47:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:38.057911+0000 mgr.y (mgr.24454) 741 : cluster [DBG] pgmap v1319: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T00:47:39.254 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:38.866890+0000 mon.a (mon.0) 3845 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-02-18T00:47:39.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:39.396+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) v1 -- 0x7f80d406c5b0 con 0x7f80d40f8440 2024-02-18T00:47:39.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:37.694321+0000 osd.3 (osd.3) 465 : cluster [DBG] 6.13 scrub starts 2024-02-18T00:47:39.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:37.695521+0000 osd.3 (osd.3) 466 : cluster [DBG] 6.13 scrub ok 2024-02-18T00:47:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:37.741588+0000 osd.1 (osd.1) 425 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:47:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:37.745129+0000 osd.1 (osd.1) 426 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:47:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:38.057911+0000 mgr.y (mgr.24454) 741 : cluster [DBG] pgmap v1319: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T00:47:39.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:38.866890+0000 mon.a (mon.0) 3845 : cluster [DBG] osdmap e843: 8 total, 8 up, 8 in 2024-02-18T00:47:40.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.084+0000 7f80cbfff700 1 -- 172.21.15.59:0/3942622810 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v844) v1 ==== 113+0+0 (secure 0 0 0) 0x7f80bc050380 con 0x7f80d40f8440 2024-02-18T00:47:40.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 10 librados: watch_flush enter 2024-02-18T00:47:40.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 10 librados: watch_flush exit 2024-02-18T00:47:40.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 20 client.32202.objecter shutdown clearing up homeless session... 2024-02-18T00:47:40.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 10 client.32202.objecter successfully canceled tick 2024-02-18T00:47:40.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80c0062c90 msgr2=0x7f80c0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:40.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 1 --2- 172.21.15.59:0/3942622810 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80c0062c90 0x7f80c0065150 secure :-1 s=READY pgs=3263 cs=0 l=1 rev1=1 rx=0x7f80c4009f20 tx=0x7f80c4009cb0).stop 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 msgr2=0x7f80d406b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.087+0000 7f80d93ad700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d406b0d0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 rx=0x7f80bc0041d0 tx=0x7f80bc009f60).stop 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 shutdown_connections 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80d40fa380 0x7f80d406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80d40f8440 0x7f80d406b0d0 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 --2- 172.21.15.59:0/3942622810 >> v2:172.21.15.59:6800/3721891797 conn(0x7f80c0062c90 0x7f80c0065150 unknown :-1 s=CLOSED pgs=3263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 --2- 172.21.15.59:0/3942622810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80d40f9ab0 0x7f80d406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 >> 172.21.15.59:0/3942622810 conn(0x7f80d40f3800 msgr2=0x7f80d40fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 shutdown_connections 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 -- 172.21.15.59:0/3942622810 wait complete. 2024-02-18T00:47:40.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:40.088+0000 7f80d93ad700 1 librados: shutdown 2024-02-18T00:47:40.091 INFO:teuthology.orchestra.run.smithi059.stderr:reweighted osd.7 to 1 (10000) 2024-02-18T00:47:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:38.679620+0000 osd.3 (osd.3) 467 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:47:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:38.681362+0000 osd.3 (osd.3) 468 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:47:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:38.763749+0000 osd.1 (osd.1) 427 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:47:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:38.765165+0000 osd.1 (osd.1) 428 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:47:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:39.398104+0000 mon.a (mon.0) 3846 : audit [INF] from='client.? 172.21.15.59:0/3942622810' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:47:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:38.679620+0000 osd.3 (osd.3) 467 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:47:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:38.681362+0000 osd.3 (osd.3) 468 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:38.763749+0000 osd.1 (osd.1) 427 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:38.765165+0000 osd.1 (osd.1) 428 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:39.398104+0000 mon.a (mon.0) 3846 : audit [INF] from='client.? 172.21.15.59:0/3942622810' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:38.679620+0000 osd.3 (osd.3) 467 : cluster [DBG] 5.14 scrub starts 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:38.681362+0000 osd.3 (osd.3) 468 : cluster [DBG] 5.14 scrub ok 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:38.763749+0000 osd.1 (osd.1) 427 : cluster [DBG] 4.27 scrub starts 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:38.765165+0000 osd.1 (osd.1) 428 : cluster [DBG] 4.27 scrub ok 2024-02-18T00:47:40.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:39.398104+0000 mon.a (mon.0) 3846 : audit [INF] from='client.? 172.21.15.59:0/3942622810' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:47:41.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd primary-affinity 7 1 2024-02-18T00:47:41.316 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:41.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.614167+0000 osd.6 (osd.6) 381 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.615288+0000 osd.6 (osd.6) 382 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.651867+0000 osd.3 (osd.3) 469 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.653580+0000 osd.3 (osd.3) 470 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.734547+0000 osd.5 (osd.5) 414 : cluster [DBG] 6.a scrub starts 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.735672+0000 osd.5 (osd.5) 415 : cluster [DBG] 6.a scrub ok 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.767960+0000 osd.1 (osd.1) 429 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:39.769073+0000 osd.1 (osd.1) 430 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.058404+0000 mgr.y (mgr.24454) 742 : cluster [DBG] pgmap v1321: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:40.085691+0000 mon.a (mon.0) 3847 : audit [INF] from='client.? 172.21.15.59:0/3942622810' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:47:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.085784+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.614167+0000 osd.6 (osd.6) 381 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.615288+0000 osd.6 (osd.6) 382 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.651867+0000 osd.3 (osd.3) 469 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.653580+0000 osd.3 (osd.3) 470 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.734547+0000 osd.5 (osd.5) 414 : cluster [DBG] 6.a scrub starts 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.735672+0000 osd.5 (osd.5) 415 : cluster [DBG] 6.a scrub ok 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.767960+0000 osd.1 (osd.1) 429 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:39.769073+0000 osd.1 (osd.1) 430 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.058404+0000 mgr.y (mgr.24454) 742 : cluster [DBG] pgmap v1321: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:40.085691+0000 mon.a (mon.0) 3847 : audit [INF] from='client.? 172.21.15.59:0/3942622810' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:47:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.085784+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-02-18T00:47:41.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:47:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.614167+0000 osd.6 (osd.6) 381 : cluster [DBG] 2.1a scrub starts 2024-02-18T00:47:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.615288+0000 osd.6 (osd.6) 382 : cluster [DBG] 2.1a scrub ok 2024-02-18T00:47:41.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.651867+0000 osd.3 (osd.3) 469 : cluster [DBG] 5.0 scrub starts 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.653580+0000 osd.3 (osd.3) 470 : cluster [DBG] 5.0 scrub ok 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.734547+0000 osd.5 (osd.5) 414 : cluster [DBG] 6.a scrub starts 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.735672+0000 osd.5 (osd.5) 415 : cluster [DBG] 6.a scrub ok 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.767960+0000 osd.1 (osd.1) 429 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:39.769073+0000 osd.1 (osd.1) 430 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.058404+0000 mgr.y (mgr.24454) 742 : cluster [DBG] pgmap v1321: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:40.085691+0000 mon.a (mon.0) 3847 : audit [INF] from='client.? 172.21.15.59:0/3942622810' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:47:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.085784+0000 mon.a (mon.0) 3848 : cluster [DBG] osdmap e844: 8 total, 8 up, 8 in 2024-02-18T00:47:42.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:47:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:47:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.632139+0000 osd.3 (osd.3) 471 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:47:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.635217+0000 osd.3 (osd.3) 472 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:47:42.209 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.762644+0000 osd.5 (osd.5) 416 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.765951+0000 osd.5 (osd.5) 417 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.773703+0000 osd.1 (osd.1) 431 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:40.776647+0000 osd.1 (osd.1) 432 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:41.100097+0000 osd.0 (osd.0) 396 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:41.101958+0000 osd.0 (osd.0) 397 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.632139+0000 osd.3 (osd.3) 471 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.635217+0000 osd.3 (osd.3) 472 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.762644+0000 osd.5 (osd.5) 416 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.765951+0000 osd.5 (osd.5) 417 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.773703+0000 osd.1 (osd.1) 431 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:47:42.210 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:40.776647+0000 osd.1 (osd.1) 432 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:47:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:41.100097+0000 osd.0 (osd.0) 396 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:47:42.211 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:41.101958+0000 osd.0 (osd.0) 397 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:47:42.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:47:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:42] "GET /metrics HTTP/1.1" 200 187523 "" "Prometheus/2.33.4" 2024-02-18T00:47:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.632139+0000 osd.3 (osd.3) 471 : cluster [DBG] 4.14 scrub starts 2024-02-18T00:47:42.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.635217+0000 osd.3 (osd.3) 472 : cluster [DBG] 4.14 scrub ok 2024-02-18T00:47:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.762644+0000 osd.5 (osd.5) 416 : cluster [DBG] 5.7 scrub starts 2024-02-18T00:47:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.765951+0000 osd.5 (osd.5) 417 : cluster [DBG] 5.7 scrub ok 2024-02-18T00:47:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.773703+0000 osd.1 (osd.1) 431 : cluster [DBG] 6.1 scrub starts 2024-02-18T00:47:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:40.776647+0000 osd.1 (osd.1) 432 : cluster [DBG] 6.1 scrub ok 2024-02-18T00:47:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:41.100097+0000 osd.0 (osd.0) 396 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:47:42.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:41.101958+0000 osd.0 (osd.0) 397 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:47:43.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:41.732062+0000 mgr.y (mgr.24454) 743 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:43.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:42.059420+0000 mgr.y (mgr.24454) 744 : cluster [DBG] pgmap v1323: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:43.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:42.084204+0000 osd.7 (osd.7) 289 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:47:43.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:42.085582+0000 osd.7 (osd.7) 290 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:47:43.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:42.093147+0000 osd.0 (osd.0) 398 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:47:43.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:42.096651+0000 osd.0 (osd.0) 399 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:47:43.178 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:41.732062+0000 mgr.y (mgr.24454) 743 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:43.178 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:42.059420+0000 mgr.y (mgr.24454) 744 : cluster [DBG] pgmap v1323: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:43.178 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:42.084204+0000 osd.7 (osd.7) 289 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:47:43.178 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:42.085582+0000 osd.7 (osd.7) 290 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:47:43.178 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:42.093147+0000 osd.0 (osd.0) 398 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:47:43.178 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:42.096651+0000 osd.0 (osd.0) 399 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 -- 172.21.15.59:0/2510879780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780f8420 msgr2=0x7f2078104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 --2- 172.21.15.59:0/2510879780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780f8420 0x7f2078104a20 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 rx=0x7f2070009be0 tx=0x7f2070004d10).stop 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 -- 172.21.15.59:0/2510879780 shutdown_connections 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 --2- 172.21.15.59:0/2510879780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780f8420 0x7f2078104a20 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 --2- 172.21.15.59:0/2510879780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780fdbb0 0x7f20780fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 --2- 172.21.15.59:0/2510879780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20780fe510 0x7f20780f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 -- 172.21.15.59:0/2510879780 >> 172.21.15.59:0/2510879780 conn(0x7f20780f3800 msgr2=0x7f20780f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 -- 172.21.15.59:0/2510879780 shutdown_connections 2024-02-18T00:47:43.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.440+0000 7f207e94c700 1 -- 172.21.15.59:0/2510879780 wait complete. 2024-02-18T00:47:43.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.441+0000 7f207e94c700 1 librados: starting msgr at 2024-02-18T00:47:43.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.441+0000 7f207e94c700 1 librados: starting objecter 2024-02-18T00:47:43.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.441+0000 7f207e94c700 1 Processor -- start 2024-02-18T00:47:43.444 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.441+0000 7f207e94c700 1 -- start start 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.441+0000 7f207e94c700 1 librados: setting wanted keys 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.441+0000 7f207e94c700 1 librados: calling monclient init 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780f8420 0x7f207810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780fdbb0 0x7f207810e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20780fe510 0x7f2078108050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078108540 con 0x7f20780f8420 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20781086b0 con 0x7f20780fe510 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2078108870 con 0x7f20780fdbb0 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780f8420 0x7f207810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780fdbb0 0x7f207810e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780f8420 0x7f207810deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56184/0 (socket says 172.21.15.59:56184) 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 -- 172.21.15.59:0/1296738525 learned_addr learned my addr 172.21.15.59:0/1296738525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20777fe700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780fdbb0 0x7f207810e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44218/0 (socket says 172.21.15.59:44218) 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207cee9700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20780fe510 0x7f2078108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 -- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780fdbb0 msgr2=0x7f207810e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780fdbb0 0x7f207810e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 -- 172.21.15.59:0/1296738525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20780fe510 msgr2=0x7f2078108050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20780fe510 0x7f2078108050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 -- 172.21.15.59:0/1296738525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2068005040 con 0x7f20780f8420 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20777fe700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780fdbb0 0x7f207810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207cee9700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20780fe510 0x7f2078108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f2077fff700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780f8420 0x7f207810deb0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 rx=0x7f20680048f0 tx=0x7f2068009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20757fa700 10 client.?.objecter ms_handle_connect 0x7f20780f8420 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20757fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20757fa700 1 -- 172.21.15.59:0/1296738525 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068007b00 con 0x7f20780f8420 2024-02-18T00:47:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20700097c0 con 0x7f20780f8420 2024-02-18T00:47:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20757fa700 1 -- 172.21.15.59:0/1296738525 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f206800f040 con 0x7f20780f8420 2024-02-18T00:47:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f20757fa700 1 -- 172.21.15.59:0/1296738525 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068014a00 con 0x7f20780f8420 2024-02-18T00:47:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 10 client.32208.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2078108eb0 con 0x7f20780f8420 2024-02-18T00:47:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.442+0000 7f207e94c700 1 librados: init done 2024-02-18T00:47:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.443+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078107110 con 0x7f20780f8420 2024-02-18T00:47:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 1 -- 172.21.15.59:0/1296738525 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f206800e580 con 0x7f20780f8420 2024-02-18T00:47:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 1 --2- 172.21.15.59:0/1296738525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2064062c90 0x7f2064065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 1 -- 172.21.15.59:0/1296738525 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(844..844 src has 418..844) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f206807cbc0 con 0x7f20780f8420 2024-02-18T00:47:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 10 client.32208.objecter ms_dispatch 0x7f2078001550 osd_map(844..844 src has 418..844) v4 2024-02-18T00:47:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 3 client.32208.objecter handle_osd_map got epochs [844,844] > 0 2024-02-18T00:47:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20777fe700 1 --2- 172.21.15.59:0/1296738525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2064062c90 0x7f2064065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 3 client.32208.objecter handle_osd_map decoding full epoch 844 2024-02-18T00:47:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 20 client.32208.objecter dump_active .. 0 homeless 2024-02-18T00:47:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20757fa700 10 client.32208.objecter ms_handle_connect 0x7f2064062c90 2024-02-18T00:47:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.444+0000 7f20777fe700 1 --2- 172.21.15.59:0/1296738525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2064062c90 0x7f2064065150 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 rx=0x7f20781097f0 tx=0x7f206000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.448+0000 7f20757fa700 1 -- 172.21.15.59:0/1296738525 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2068049f60 con 0x7f20780f8420 2024-02-18T00:47:43.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:41.732062+0000 mgr.y (mgr.24454) 743 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:42.059420+0000 mgr.y (mgr.24454) 744 : cluster [DBG] pgmap v1323: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:42.084204+0000 osd.7 (osd.7) 289 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:47:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:42.085582+0000 osd.7 (osd.7) 290 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:47:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:42.093147+0000 osd.0 (osd.0) 398 : cluster [DBG] 4.3 scrub starts 2024-02-18T00:47:43.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:42.096651+0000 osd.0 (osd.0) 399 : cluster [DBG] 4.3 scrub ok 2024-02-18T00:47:43.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.633+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) v1 -- 0x7f2078109530 con 0x7f20780f8420 2024-02-18T00:47:43.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.865+0000 7f20757fa700 1 -- 172.21.15.59:0/1296738525 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v845) v1 ==== 133+0+0 (secure 0 0 0) 0x7f2068050290 con 0x7f20780f8420 2024-02-18T00:47:43.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 10 librados: watch_flush enter 2024-02-18T00:47:43.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 10 librados: watch_flush exit 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 20 client.32208.objecter shutdown clearing up homeless session... 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 10 client.32208.objecter successfully canceled tick 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2064062c90 msgr2=0x7f2064065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 --2- 172.21.15.59:0/1296738525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2064062c90 0x7f2064065150 secure :-1 s=READY pgs=3264 cs=0 l=1 rev1=1 rx=0x7f20781097f0 tx=0x7f206000b040).stop 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780f8420 msgr2=0x7f207810deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780f8420 0x7f207810deb0 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 rx=0x7f20680048f0 tx=0x7f2068009cb0).stop 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 shutdown_connections 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 --2- 172.21.15.59:0/1296738525 >> v2:172.21.15.59:6800/3721891797 conn(0x7f2064062c90 0x7f2064065150 secure :-1 s=CLOSED pgs=3264 cs=0 l=1 rev1=1 rx=0x7f20781097f0 tx=0x7f206000b040).stop 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20780fe510 0x7f2078108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20780f8420 0x7f207810deb0 secure :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 rx=0x7f20680048f0 tx=0x7f2068009cb0).stop 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 --2- 172.21.15.59:0/1296738525 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f20780fdbb0 0x7f207810e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:43.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.868+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 >> 172.21.15.59:0/1296738525 conn(0x7f20780f3800 msgr2=0x7f20780f50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:43.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.869+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 shutdown_connections 2024-02-18T00:47:43.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.869+0000 7f207e94c700 1 -- 172.21.15.59:0/1296738525 wait complete. 2024-02-18T00:47:43.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:43.870+0000 7f207e94c700 1 librados: shutdown 2024-02-18T00:47:43.872 INFO:teuthology.orchestra.run.smithi059.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-02-18T00:47:44.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:42.683474+0000 osd.5 (osd.5) 418 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:42.685292+0000 osd.5 (osd.5) 419 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:43.130997+0000 osd.0 (osd.0) 400 : cluster [DBG] 25.a scrub starts 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:43.134116+0000 osd.0 (osd.0) 401 : cluster [DBG] 25.a scrub ok 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:43.634430+0000 mon.a (mon.0) 3849 : audit [INF] from='client.? 172.21.15.59:0/1296738525' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:43.866248+0000 mon.a (mon.0) 3850 : audit [INF] from='client.? 172.21.15.59:0/1296738525' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:43.866319+0000 mon.a (mon.0) 3851 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:42.683474+0000 osd.5 (osd.5) 418 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:42.685292+0000 osd.5 (osd.5) 419 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:47:44.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:43.130997+0000 osd.0 (osd.0) 400 : cluster [DBG] 25.a scrub starts 2024-02-18T00:47:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:43.134116+0000 osd.0 (osd.0) 401 : cluster [DBG] 25.a scrub ok 2024-02-18T00:47:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:43.634430+0000 mon.a (mon.0) 3849 : audit [INF] from='client.? 172.21.15.59:0/1296738525' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:47:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:43.866248+0000 mon.a (mon.0) 3850 : audit [INF] from='client.? 172.21.15.59:0/1296738525' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:47:44.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:43.866319+0000 mon.a (mon.0) 3851 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-02-18T00:47:44.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:42.683474+0000 osd.5 (osd.5) 418 : cluster [DBG] 4.21 scrub starts 2024-02-18T00:47:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:42.685292+0000 osd.5 (osd.5) 419 : cluster [DBG] 4.21 scrub ok 2024-02-18T00:47:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:43.130997+0000 osd.0 (osd.0) 400 : cluster [DBG] 25.a scrub starts 2024-02-18T00:47:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:43.134116+0000 osd.0 (osd.0) 401 : cluster [DBG] 25.a scrub ok 2024-02-18T00:47:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:43.634430+0000 mon.a (mon.0) 3849 : audit [INF] from='client.? 172.21.15.59:0/1296738525' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-02-18T00:47:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:43.866248+0000 mon.a (mon.0) 3850 : audit [INF] from='client.? 172.21.15.59:0/1296738525' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-02-18T00:47:44.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:43.866319+0000 mon.a (mon.0) 3851 : cluster [DBG] osdmap e845: 8 total, 8 up, 8 in 2024-02-18T00:47:44.702 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-02-18T00:47:44.702 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-02-18T00:47:44.702 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-02-18T00:47:44.702 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-02-18T00:47:44.702 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-18T00:47:44.702 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T00:47:44.989 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:45.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:44.059999+0000 mgr.y (mgr.24454) 745 : cluster [DBG] pgmap v1325: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:45.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:44.059999+0000 mgr.y (mgr.24454) 745 : cluster [DBG] pgmap v1325: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:45.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:44.059999+0000 mgr.y (mgr.24454) 745 : cluster [DBG] pgmap v1325: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:47:46.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:44.713568+0000 osd.6 (osd.6) 383 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:47:46.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:44.714587+0000 osd.6 (osd.6) 384 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:45.497221+0000 osd.4 (osd.4) 473 : cluster [DBG] 25.b scrub starts 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:45.500951+0000 osd.4 (osd.4) 474 : cluster [DBG] 25.b scrub ok 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:45.733480+0000 osd.2 (osd.2) 365 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:45.735262+0000 osd.2 (osd.2) 366 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:44.713568+0000 osd.6 (osd.6) 383 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:44.714587+0000 osd.6 (osd.6) 384 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:45.497221+0000 osd.4 (osd.4) 473 : cluster [DBG] 25.b scrub starts 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:45.500951+0000 osd.4 (osd.4) 474 : cluster [DBG] 25.b scrub ok 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:45.733480+0000 osd.2 (osd.2) 365 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:47:46.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:45.735262+0000 osd.2 (osd.2) 366 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:47:46.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:44.713568+0000 osd.6 (osd.6) 383 : cluster [DBG] 2.29 scrub starts 2024-02-18T00:47:46.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:44.714587+0000 osd.6 (osd.6) 384 : cluster [DBG] 2.29 scrub ok 2024-02-18T00:47:46.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:45.497221+0000 osd.4 (osd.4) 473 : cluster [DBG] 25.b scrub starts 2024-02-18T00:47:46.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:45.500951+0000 osd.4 (osd.4) 474 : cluster [DBG] 25.b scrub ok 2024-02-18T00:47:46.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:45.733480+0000 osd.2 (osd.2) 365 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:47:46.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:45.735262+0000 osd.2 (osd.2) 366 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:47:47.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 -- 172.21.15.59:0/1441901451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f9670 msgr2=0x7f08cc101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:47.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 --2- 172.21.15.59:0/1441901451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f9670 0x7f08cc101d80 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 rx=0x7f08b4004870 tx=0x7f08b4009cb0).stop 2024-02-18T00:47:47.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 -- 172.21.15.59:0/1441901451 shutdown_connections 2024-02-18T00:47:47.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 --2- 172.21.15.59:0/1441901451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08cc0f8d10 0x7f08cc0f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 --2- 172.21.15.59:0/1441901451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08cc102270 0x7f08cc104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 --2- 172.21.15.59:0/1441901451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f9670 0x7f08cc101d80 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 -- 172.21.15.59:0/1441901451 >> 172.21.15.59:0/1441901451 conn(0x7f08cc0f2820 msgr2=0x7f08cc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 -- 172.21.15.59:0/1441901451 shutdown_connections 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.195+0000 7f08d10c3700 1 -- 172.21.15.59:0/1441901451 wait complete. 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 librados: starting msgr at 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 librados: starting objecter 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 Processor -- start 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 -- start start 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 librados: setting wanted keys 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 librados: calling monclient init 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f8d10 0x7f08cc10e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08cc0f9670 0x7f08cc10e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08cc102270 0x7f08cc108320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc108810 con 0x7f08cc0f9670 2024-02-18T00:47:47.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc108980 con 0x7f08cc102270 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08d10c3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc108b40 con 0x7f08cc0f8d10 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.196+0000 7f08cad9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f8d10 0x7f08cc10e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08ca59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08cc0f9670 0x7f08cc10e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f8d10 0x7f08cc10e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44232/0 (socket says 172.21.15.59:44232) 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 -- 172.21.15.59:0/3604152530 learned_addr learned my addr 172.21.15.59:0/3604152530 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cb59e700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08cc102270 0x7f08cc108320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 -- 172.21.15.59:0/3604152530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08cc102270 msgr2=0x7f08cc108320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08cc102270 0x7f08cc108320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 -- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08cc0f9670 msgr2=0x7f08cc10e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08cc0f9670 0x7f08cc10e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 -- 172.21.15.59:0/3604152530 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c40097c0 con 0x7f08cc0f8d10 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08ca59c700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08cc0f9670 0x7f08cc10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:47.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08cad9d700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f8d10 0x7f08cc10e180 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f08bc00c810 tx=0x7f08bc004330).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08c3fff700 10 client.?.objecter ms_handle_connect 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08c3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08c3fff700 1 -- 172.21.15.59:0/3604152530 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0043a0 con 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08c3fff700 1 -- 172.21.15.59:0/3604152530 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08bc0109e0 con 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b4005040 con 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08c3fff700 1 -- 172.21.15.59:0/3604152530 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc012ab0 con 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08d10c3700 10 client.42017.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc109180 con 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.197+0000 7f08d10c3700 1 librados: init done 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.198+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc05fa60 con 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.198+0000 7f08c3fff700 1 -- 172.21.15.59:0/3604152530 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f08bc010b50 con 0x7f08cc0f8d10 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08c3fff700 1 --2- 172.21.15.59:0/3604152530 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08b8062c90 0x7f08b8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:47.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08c3fff700 1 -- 172.21.15.59:0/3604152530 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(846..846 src has 418..846) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f08bc07cf10 con 0x7f08cc0f8d10 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08c3fff700 10 client.42017.objecter ms_dispatch 0x7f08cc001550 osd_map(846..846 src has 418..846) v4 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08c3fff700 3 client.42017.objecter handle_osd_map got epochs [846,846] > 0 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08c3fff700 3 client.42017.objecter handle_osd_map decoding full epoch 846 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08ca59c700 1 --2- 172.21.15.59:0/3604152530 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08b8062c90 0x7f08b8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08c3fff700 20 client.42017.objecter dump_active .. 0 homeless 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08c3fff700 10 client.42017.objecter ms_handle_connect 0x7f08b8062c90 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.199+0000 7f08ca59c700 1 --2- 172.21.15.59:0/3604152530 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08b8062c90 0x7f08b8065150 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 rx=0x7f08b40043d0 tx=0x7f08b400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.203+0000 7f08c3fff700 1 -- 172.21.15.59:0/3604152530 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f08bc0505e0 con 0x7f08cc0f8d10 2024-02-18T00:47:47.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:45.768534+0000 osd.1 (osd.1) 433 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:45.771503+0000 osd.1 (osd.1) 434 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.060863+0000 mgr.y (mgr.24454) 746 : cluster [DBG] pgmap v1326: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.083627+0000 osd.0 (osd.0) 402 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.084754+0000 osd.0 (osd.0) 403 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.607244+0000 osd.2 (osd.2) 367 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.607627+0000 osd.2 (osd.2) 368 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:45.768534+0000 osd.1 (osd.1) 433 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:45.771503+0000 osd.1 (osd.1) 434 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:47:47.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.060863+0000 mgr.y (mgr.24454) 746 : cluster [DBG] pgmap v1326: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:47:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.083627+0000 osd.0 (osd.0) 402 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:47:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.084754+0000 osd.0 (osd.0) 403 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:47:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.607244+0000 osd.2 (osd.2) 367 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:47.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.607627+0000 osd.2 (osd.2) 368 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:47.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.374+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08cc109800 con 0x7f08cc0f8d10 2024-02-18T00:47:47.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.375+0000 7f08c3fff700 1 -- 172.21.15.59:0/3604152530 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v846) v1 ==== 74+0+22405 (secure 0 0 0) 0x7f08bc0501d0 con 0x7f08cc0f8d10 2024-02-18T00:47:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:47:47.377 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":846,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","created":"2024-02-18T00:23:11.390869+0000","modified":"2024-02-18T00:47:47.111459+0000","last_up_change":"2024-02-18T00:29:06.503960+0000","last_in_change":"2024-02-18T00:46:24.847638+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":105,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":256,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T00:27:02.909667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-18T00:29:13.697458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":42,"pg_placement_num":42,"pg_placement_num_target":42,"pg_num_target":42,"pg_num_pending":42,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"107","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-18T00:29:14.132992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T00:29:15.191206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":42,"pg_placement_num":42,"pg_placement_num_target":42,"pg_num_target":42,"pg_num_pending":42,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"816","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"641","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T00:29:17.526023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T00:29:19.600478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":22,"pg_placement_num":22,"pg_placement_num_target":22,"pg_num_target":22,"pg_num_pending":22,"last_pg_merge_meta":{"source_pgid":"6.16","ready_epoch":587,"last_epoch_started":586,"last_epoch_clean":586,"source_version":"0'0","target_version":"0'0"},"last_change":"813","last_force_op_resend":"0","last_force_op_resend_prenautilus":"587","last_force_op_resend_preluminous":"585","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}},{"pool":25,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-02-18T00:32:59.546190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"6e729102-80e2-4a4e-92ca-600ee8b1ed85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":3700238977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3700238977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3700238977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6803","nonce":3700238977}]},"public_addr":"172.21.15.59:6801/3700238977","cluster_addr":"172.21.15.59:6802/3700238977","heartbeat_back_addr":"172.21.15.59:6804/3700238977","heartbeat_front_addr":"172.21.15.59:6803/3700238977","state":["exists","up"]},{"osd":1,"uuid":"f67a7206-6d31-497f-97e0-672d134b13d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":26279110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":26279110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":26279110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6807","nonce":26279110}]},"public_addr":"172.21.15.59:6805/26279110","cluster_addr":"172.21.15.59:6806/26279110","heartbeat_back_addr":"172.21.15.59:6808/26279110","heartbeat_front_addr":"172.21.15.59:6807/26279110","state":["exists","up"]},{"osd":2,"uuid":"d6223dbd-e539-4770-9e5a-7b756304f8f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1153193637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1153193637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1153193637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6811","nonce":1153193637}]},"public_addr":"172.21.15.59:6809/1153193637","cluster_addr":"172.21.15.59:6810/1153193637","heartbeat_back_addr":"172.21.15.59:6812/1153193637","heartbeat_front_addr":"172.21.15.59:6811/1153193637","state":["exists","up"]},{"osd":3,"uuid":"fff275df-95e0-41f6-96c3-e531f05de32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":811,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1805700964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1805700964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1805700964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6815","nonce":1805700964}]},"public_addr":"172.21.15.59:6813/1805700964","cluster_addr":"172.21.15.59:6814/1805700964","heartbeat_back_addr":"172.21.15.59:6816/1805700964","heartbeat_front_addr":"172.21.15.59:6815/1805700964","state":["exists","up"]},{"osd":4,"uuid":"a0bae50b-4f29-4310-ac5f-bc7285464231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2044387056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":2044387056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":2044387056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2044387056}]},"public_addr":"172.21.15.137:6800/2044387056","cluster_addr":"172.21.15.137:6801/2044387056","heartbeat_back_addr":"172.21.15.137:6803/2044387056","heartbeat_front_addr":"172.21.15.137:6802/2044387056","state":["exists","up"]},{"osd":5,"uuid":"513a4192-45fb-4444-a1f9-4bd2f2a5fa01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2170432263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":2170432263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":2170432263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2170432263}]},"public_addr":"172.21.15.137:6804/2170432263","cluster_addr":"172.21.15.137:6805/2170432263","heartbeat_back_addr":"172.21.15.137:6807/2170432263","heartbeat_front_addr":"172.21.15.137:6806/2170432263","state":["exists","up"]},{"osd":6,"uuid":"5bbb7683-ea5a-490c-8c8b-67b53c45406f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3356009390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3356009390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6811","nonce":3356009390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3356009390}]},"public_addr":"172.21.15.137:6808/3356009390","cluster_addr":"172.21.15.137:6809/3356009390","heartbeat_back_addr":"172.21.15.137:6811/3356009390","heartbeat_front_addr":"172.21.15.137:6810/3356009390","state":["exists","up"]},{"osd":7,"uuid":"951d317b-7a34-41a0-b1a1-a2851b2070bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":53025171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":53025171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6815","nonce":53025171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":53025171}]},"public_addr":"172.21.15.137:6812/53025171","cluster_addr":"172.21.15.137:6813/53025171","heartbeat_back_addr":"172.21.15.137:6815/53025171","heartbeat_front_addr":"172.21.15.137:6814/53025171","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:16.185973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:46.838061+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:46.607632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:47.022392+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:05.711798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:25.435517+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:03.602769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:04.666898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":2}]},{"pgid":"2.3","mappings":[{"from":7,"to":1}]},{"pgid":"2.5","mappings":[{"from":7,"to":2}]},{"pgid":"2.6","mappings":[{"from":4,"to":1}]},{"pgid":"2.8","mappings":[{"from":7,"to":2}]},{"pgid":"2.18","mappings":[{"from":7,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5},{"from":3,"to":0}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":3,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":0}]},{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":7,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.21","mappings":[{"from":3,"to":5}]},{"pgid":"6.9","mappings":[{"from":0,"to":7},{"from":2,"to":1}]},{"pgid":"25.9","mappings":[{"from":0,"to":4}]},{"pgid":"25.18","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1243102711":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/817089002":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/3174805070":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/3506000666":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/4167740619":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/999852549":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/3575161033":"2024-02-19T00:23:45.307159+0000","172.21.15.59:0/2403979355":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/642821245":"2024-02-19T00:23:45.307159+0000","172.21.15.59:6800/1656814063":"2024-02-19T00:23:45.307159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":14,"snaps":[{"begin":9,"length":2}]},{"pool":60,"snaps":[{"begin":6,"length":2}]},{"pool":78,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":86,"snaps":[{"begin":2,"length":1}]},{"pool":94,"snaps":[{"begin":6,"length":2}]},{"pool":97,"snaps":[{"begin":4,"length":4}]},{"pool":108,"snaps":[{"begin":2,"length":1}]},{"pool":114,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":128,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":134,"snaps":[{"begin":12,"length":1}]},{"pool":154,"snaps":[{"begin":14,"length":1}]},{"pool":166,"snaps":[{"begin":17,"length":1}]},{"pool":177,"snaps":[{"begin":20,"length":1}]},{"pool":237,"snaps":[{"begin":2,"length":1}]},{"pool":238,"snaps":[{"begin":4,"length":1}]},{"pool":241,"snaps":[{"begin":6,"length":1}]},{"pool":245,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T00:47:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 10 librados: watch_flush enter 2024-02-18T00:47:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 10 librados: watch_flush exit 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 20 client.42017.objecter shutdown clearing up homeless session... 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 10 client.42017.objecter successfully canceled tick 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08b8062c90 msgr2=0x7f08b8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 --2- 172.21.15.59:0/3604152530 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08b8062c90 0x7f08b8065150 secure :-1 s=READY pgs=3265 cs=0 l=1 rev1=1 rx=0x7f08b40043d0 tx=0x7f08b400b040).stop 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f8d10 msgr2=0x7f08cc10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f8d10 0x7f08cc10e180 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 rx=0x7f08bc00c810 tx=0x7f08bc004330).stop 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 shutdown_connections 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 --2- 172.21.15.59:0/3604152530 >> v2:172.21.15.59:6800/3721891797 conn(0x7f08b8062c90 0x7f08b8065150 unknown :-1 s=CLOSED pgs=3265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08cc102270 0x7f08cc108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f08cc0f8d10 0x7f08cc10e180 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 --2- 172.21.15.59:0/3604152530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08cc0f9670 0x7f08cc10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 >> 172.21.15.59:0/3604152530 conn(0x7f08cc0f2820 msgr2=0x7f08cc0f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 shutdown_connections 2024-02-18T00:47:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 -- 172.21.15.59:0/3604152530 wait complete. 2024-02-18T00:47:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:47.377+0000 7f08d10c3700 1 librados: shutdown 2024-02-18T00:47:47.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:45.768534+0000 osd.1 (osd.1) 433 : cluster [DBG] 2.4 scrub starts 2024-02-18T00:47:47.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:45.771503+0000 osd.1 (osd.1) 434 : cluster [DBG] 2.4 scrub ok 2024-02-18T00:47:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.060863+0000 mgr.y (mgr.24454) 746 : cluster [DBG] pgmap v1326: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:47:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.083627+0000 osd.0 (osd.0) 402 : cluster [DBG] 6.10 scrub starts 2024-02-18T00:47:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.084754+0000 osd.0 (osd.0) 403 : cluster [DBG] 6.10 scrub ok 2024-02-18T00:47:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.607244+0000 osd.2 (osd.2) 367 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:47.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.607627+0000 osd.2 (osd.2) 368 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.686114+0000 osd.3 (osd.3) 473 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.687414+0000 osd.3 (osd.3) 474 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.837786+0000 osd.1 (osd.1) 435 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:46.838059+0000 osd.1 (osd.1) 436 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:47.022108+0000 osd.3 (osd.3) 475 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:47.022389+0000 osd.3 (osd.3) 476 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:47.124618+0000 mon.a (mon.0) 3852 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-02-18T00:47:48.126 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:47.375677+0000 mon.c (mon.2) 592 : audit [DBG] from='client.? 172.21.15.59:0/3604152530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:47:48.318 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-18T00:47:48.319 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T00:47:48.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.686114+0000 osd.3 (osd.3) 473 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:47:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.687414+0000 osd.3 (osd.3) 474 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:47:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.837786+0000 osd.1 (osd.1) 435 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:46.838059+0000 osd.1 (osd.1) 436 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:47.022108+0000 osd.3 (osd.3) 475 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:47.022389+0000 osd.3 (osd.3) 476 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:47.124618+0000 mon.a (mon.0) 3852 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-02-18T00:47:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:47.375677+0000 mon.c (mon.2) 592 : audit [DBG] from='client.? 172.21.15.59:0/3604152530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:47:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.686114+0000 osd.3 (osd.3) 473 : cluster [DBG] 2.28 scrub starts 2024-02-18T00:47:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.687414+0000 osd.3 (osd.3) 474 : cluster [DBG] 2.28 scrub ok 2024-02-18T00:47:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.837786+0000 osd.1 (osd.1) 435 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:46.838059+0000 osd.1 (osd.1) 436 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:47.022108+0000 osd.3 (osd.3) 475 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:47:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:47.022389+0000 osd.3 (osd.3) 476 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:47:48.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:47.124618+0000 mon.a (mon.0) 3852 : cluster [DBG] osdmap e846: 8 total, 8 up, 8 in 2024-02-18T00:47:48.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:47.375677+0000 mon.c (mon.2) 592 : audit [DBG] from='client.? 172.21.15.59:0/3604152530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:47:48.606 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:49.127 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:47.673501+0000 osd.3 (osd.3) 477 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:47:49.127 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:47.674562+0000 osd.3 (osd.3) 478 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:47:49.128 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:48.061394+0000 mgr.y (mgr.24454) 747 : cluster [DBG] pgmap v1328: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:47:49.128 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:48.867528+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-02-18T00:47:49.128 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:47.673501+0000 osd.3 (osd.3) 477 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:47:49.128 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:47.674562+0000 osd.3 (osd.3) 478 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:47:49.128 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:48.061394+0000 mgr.y (mgr.24454) 747 : cluster [DBG] pgmap v1328: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:47:49.128 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:48.867528+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-02-18T00:47:49.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:47.673501+0000 osd.3 (osd.3) 477 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:47:49.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:47.674562+0000 osd.3 (osd.3) 478 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:47:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:48.061394+0000 mgr.y (mgr.24454) 747 : cluster [DBG] pgmap v1328: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:47:49.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:48.867528+0000 mon.a (mon.0) 3853 : cluster [DBG] osdmap e847: 8 total, 8 up, 8 in 2024-02-18T00:47:50.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:48.552980+0000 osd.6 (osd.6) 385 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:47:50.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:48.553996+0000 osd.6 (osd.6) 386 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:47:50.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:48.712119+0000 osd.3 (osd.3) 479 : cluster [DBG] 6.c scrub starts 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:48.713564+0000 osd.3 (osd.3) 480 : cluster [DBG] 6.c scrub ok 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:49.141600+0000 osd.7 (osd.7) 291 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:49.142603+0000 osd.7 (osd.7) 292 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:49.526811+0000 osd.4 (osd.4) 475 : cluster [DBG] 4.18 deep-scrub starts 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:49.529311+0000 osd.4 (osd.4) 476 : cluster [DBG] 4.18 deep-scrub ok 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:48.552980+0000 osd.6 (osd.6) 385 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:48.553996+0000 osd.6 (osd.6) 386 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:48.712119+0000 osd.3 (osd.3) 479 : cluster [DBG] 6.c scrub starts 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:48.713564+0000 osd.3 (osd.3) 480 : cluster [DBG] 6.c scrub ok 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:49.141600+0000 osd.7 (osd.7) 291 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:47:50.469 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:49.142603+0000 osd.7 (osd.7) 292 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:47:50.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:49.526811+0000 osd.4 (osd.4) 475 : cluster [DBG] 4.18 deep-scrub starts 2024-02-18T00:47:50.470 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:49.529311+0000 osd.4 (osd.4) 476 : cluster [DBG] 4.18 deep-scrub ok 2024-02-18T00:47:50.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:48.552980+0000 osd.6 (osd.6) 385 : cluster [DBG] 5.11 scrub starts 2024-02-18T00:47:50.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:48.553996+0000 osd.6 (osd.6) 386 : cluster [DBG] 5.11 scrub ok 2024-02-18T00:47:50.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:48.712119+0000 osd.3 (osd.3) 479 : cluster [DBG] 6.c scrub starts 2024-02-18T00:47:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:48.713564+0000 osd.3 (osd.3) 480 : cluster [DBG] 6.c scrub ok 2024-02-18T00:47:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:49.141600+0000 osd.7 (osd.7) 291 : cluster [DBG] 2.1d scrub starts 2024-02-18T00:47:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:49.142603+0000 osd.7 (osd.7) 292 : cluster [DBG] 2.1d scrub ok 2024-02-18T00:47:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:49.526811+0000 osd.4 (osd.4) 475 : cluster [DBG] 4.18 deep-scrub starts 2024-02-18T00:47:50.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:49.529311+0000 osd.4 (osd.4) 476 : cluster [DBG] 4.18 deep-scrub ok 2024-02-18T00:47:50.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2983671945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f96e0 msgr2=0x7fe2e4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:50.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2983671945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f96e0 0x7fe2e4101c30 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 rx=0x7fe2d80047b0 tx=0x7fe2d8009cb0).stop 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2983671945 shutdown_connections 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2983671945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2e40f8d80 0x7fe2e40f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2983671945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 0x7fe2e4104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2983671945 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f96e0 0x7fe2e4101c30 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2983671945 >> 172.21.15.59:0/2983671945 conn(0x7fe2e40f26f0 msgr2=0x7fe2e40f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2983671945 shutdown_connections 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.797+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2983671945 wait complete. 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 librados: starting msgr at 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 librados: starting objecter 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 Processor -- start 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 -- start start 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 librados: setting wanted keys 2024-02-18T00:47:50.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 librados: calling monclient init 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f8d80 0x7fe2e410e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2e40f96e0 0x7fe2e410e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 0x7fe2e4108470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4108960 con 0x7fe2e4102250 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4108ad0 con 0x7fe2e40f96e0 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.798+0000 7fe2ea0e3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4108c90 con 0x7fe2e40f8d80 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 0x7fe2e4108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f8d80 0x7fe2e410e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 0x7fe2e4108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36154/0 (socket says 172.21.15.59:36154) 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 -- 172.21.15.59:0/2751278498 learned_addr learned my addr 172.21.15.59:0/2751278498 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e2ffd700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2e40f96e0 0x7fe2e410e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 -- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f8d80 msgr2=0x7fe2e410e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:50.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f8d80 0x7fe2e410e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 -- 172.21.15.59:0/2751278498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2e40f96e0 msgr2=0x7fe2e410e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2e40f96e0 0x7fe2e410e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 -- 172.21.15.59:0/2751278498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0097c0 con 0x7fe2e4102250 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e37fe700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f8d80 0x7fe2e410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e2ffd700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2e40f96e0 0x7fe2e410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e3fff700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 0x7fe2e4108470 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 rx=0x7fe2dc001df0 tx=0x7fe2dc004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e0ff9700 10 client.?.objecter ms_handle_connect 0x7fe2e4102250 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e0ff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e0ff9700 1 -- 172.21.15.59:0/2751278498 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc009a10 con 0x7fe2e4102250 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d8005040 con 0x7fe2e4102250 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e0ff9700 1 -- 172.21.15.59:0/2751278498 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2dc0095a0 con 0x7fe2e4102250 2024-02-18T00:47:50.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2e0ff9700 1 -- 172.21.15.59:0/2751278498 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc007e30 con 0x7fe2e4102250 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2ea0e3700 10 client.32220.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e4109280 con 0x7fe2e4102250 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.799+0000 7fe2ea0e3700 1 librados: init done 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e405ee80 con 0x7fe2e4102250 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e0ff9700 1 -- 172.21.15.59:0/2751278498 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe2dc0090d0 con 0x7fe2e4102250 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e0ff9700 1 --2- 172.21.15.59:0/2751278498 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe2cc062c90 0x7fe2cc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e0ff9700 1 -- 172.21.15.59:0/2751278498 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(847..847 src has 418..847) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fe2dc07d900 con 0x7fe2e4102250 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e0ff9700 10 client.32220.objecter ms_dispatch 0x7fe2e4001550 osd_map(847..847 src has 418..847) v4 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e0ff9700 3 client.32220.objecter handle_osd_map got epochs [847,847] > 0 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e0ff9700 3 client.32220.objecter handle_osd_map decoding full epoch 847 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e37fe700 1 --2- 172.21.15.59:0/2751278498 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe2cc062c90 0x7fe2cc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:50.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.801+0000 7fe2e0ff9700 20 client.32220.objecter dump_active .. 0 homeless 2024-02-18T00:47:50.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.802+0000 7fe2e0ff9700 10 client.32220.objecter ms_handle_connect 0x7fe2cc062c90 2024-02-18T00:47:50.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.802+0000 7fe2e37fe700 1 --2- 172.21.15.59:0/2751278498 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe2cc062c90 0x7fe2cc065150 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 rx=0x7fe2d4006fd0 tx=0x7fe2d4009060).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:50.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.806+0000 7fe2e0ff9700 1 -- 172.21.15.59:0/2751278498 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe2dc050fd0 con 0x7fe2e4102250 2024-02-18T00:47:50.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.979+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2e4109950 con 0x7fe2e4102250 2024-02-18T00:47:50.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.980+0000 7fe2e0ff9700 1 -- 172.21.15.59:0/2751278498 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v847) v1 ==== 74+0+22405 (secure 0 0 0) 0x7fe2dc0191b0 con 0x7fe2e4102250 2024-02-18T00:47:50.982 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:47:50.982 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":847,"fsid":"d129b4f8-cdf3-11ee-95ba-87774f69a715","created":"2024-02-18T00:23:11.390869+0000","modified":"2024-02-18T00:47:48.863132+0000","last_up_change":"2024-02-18T00:29:06.503960+0000","last_in_change":"2024-02-18T00:46:24.847638+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":105,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":256,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T00:27:02.909667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-18T00:29:13.697458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":42,"pg_placement_num":42,"pg_placement_num_target":42,"pg_num_target":42,"pg_num_pending":42,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"107","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"iscsi","create_time":"2024-02-18T00:29:14.132992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T00:29:15.191206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":42,"pg_placement_num":42,"pg_placement_num_target":42,"pg_num_target":42,"pg_num_pending":42,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"816","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"641","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T00:29:17.526023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T00:29:19.600478+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":22,"pg_placement_num":22,"pg_placement_num_target":22,"pg_num_target":22,"pg_num_pending":22,"last_pg_merge_meta":{"source_pgid":"6.16","ready_epoch":587,"last_epoch_started":586,"last_epoch_clean":586,"source_version":"0'0","target_version":"0'0"},"last_change":"813","last_force_op_resend":"0","last_force_op_resend_prenautilus":"587","last_force_op_resend_preluminous":"585","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}},{"pool":25,"pool_name":"ceph_test_rados_api_asio","create_time":"2024-02-18T00:32:59.546190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"6e729102-80e2-4a4e-92ca-600ee8b1ed85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":3700238977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3700238977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3700238977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6803","nonce":3700238977}]},"public_addr":"172.21.15.59:6801/3700238977","cluster_addr":"172.21.15.59:6802/3700238977","heartbeat_back_addr":"172.21.15.59:6804/3700238977","heartbeat_front_addr":"172.21.15.59:6803/3700238977","state":["exists","up"]},{"osd":1,"uuid":"f67a7206-6d31-497f-97e0-672d134b13d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":26279110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":26279110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":26279110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6807","nonce":26279110}]},"public_addr":"172.21.15.59:6805/26279110","cluster_addr":"172.21.15.59:6806/26279110","heartbeat_back_addr":"172.21.15.59:6808/26279110","heartbeat_front_addr":"172.21.15.59:6807/26279110","state":["exists","up"]},{"osd":2,"uuid":"d6223dbd-e539-4770-9e5a-7b756304f8f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":1153193637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1153193637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1153193637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6811","nonce":1153193637}]},"public_addr":"172.21.15.59:6809/1153193637","cluster_addr":"172.21.15.59:6810/1153193637","heartbeat_back_addr":"172.21.15.59:6812/1153193637","heartbeat_front_addr":"172.21.15.59:6811/1153193637","state":["exists","up"]},{"osd":3,"uuid":"fff275df-95e0-41f6-96c3-e531f05de32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":811,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":1805700964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1805700964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1805700964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6815","nonce":1805700964}]},"public_addr":"172.21.15.59:6813/1805700964","cluster_addr":"172.21.15.59:6814/1805700964","heartbeat_back_addr":"172.21.15.59:6816/1805700964","heartbeat_front_addr":"172.21.15.59:6815/1805700964","state":["exists","up"]},{"osd":4,"uuid":"a0bae50b-4f29-4310-ac5f-bc7285464231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":2044387056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":2044387056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6803","nonce":2044387056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":2044387056}]},"public_addr":"172.21.15.137:6800/2044387056","cluster_addr":"172.21.15.137:6801/2044387056","heartbeat_back_addr":"172.21.15.137:6803/2044387056","heartbeat_front_addr":"172.21.15.137:6802/2044387056","state":["exists","up"]},{"osd":5,"uuid":"513a4192-45fb-4444-a1f9-4bd2f2a5fa01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":2170432263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":2170432263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6807","nonce":2170432263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":2170432263}]},"public_addr":"172.21.15.137:6804/2170432263","cluster_addr":"172.21.15.137:6805/2170432263","heartbeat_back_addr":"172.21.15.137:6807/2170432263","heartbeat_front_addr":"172.21.15.137:6806/2170432263","state":["exists","up"]},{"osd":6,"uuid":"5bbb7683-ea5a-490c-8c8b-67b53c45406f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":3356009390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":3356009390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6811","nonce":3356009390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3356009390}]},"public_addr":"172.21.15.137:6808/3356009390","cluster_addr":"172.21.15.137:6809/3356009390","heartbeat_back_addr":"172.21.15.137:6811/3356009390","heartbeat_front_addr":"172.21.15.137:6810/3356009390","state":["exists","up"]},{"osd":7,"uuid":"951d317b-7a34-41a0-b1a1-a2851b2070bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":808,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":53025171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":53025171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6815","nonce":53025171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":53025171}]},"public_addr":"172.21.15.137:6812/53025171","cluster_addr":"172.21.15.137:6813/53025171","heartbeat_back_addr":"172.21.15.137:6815/53025171","heartbeat_front_addr":"172.21.15.137:6814/53025171","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:16.185973+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:46.838061+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:46.607632+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:47.022392+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:05.711798+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:25.435517+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:03.602769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T00:47:04.666898+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":2}]},{"pgid":"2.3","mappings":[{"from":7,"to":1}]},{"pgid":"2.5","mappings":[{"from":7,"to":2}]},{"pgid":"2.6","mappings":[{"from":4,"to":1}]},{"pgid":"2.8","mappings":[{"from":7,"to":2}]},{"pgid":"2.18","mappings":[{"from":7,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.4","mappings":[{"from":1,"to":5},{"from":3,"to":0}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":3,"to":5}]},{"pgid":"4.e","mappings":[{"from":1,"to":0}]},{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.16","mappings":[{"from":7,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"4.21","mappings":[{"from":3,"to":5}]},{"pgid":"6.9","mappings":[{"from":0,"to":7},{"from":2,"to":1}]},{"pgid":"25.9","mappings":[{"from":0,"to":4}]},{"pgid":"25.18","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1243102711":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/817089002":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/3174805070":"2024-02-19T00:29:39.703735+0000","172.21.15.59:6800/3506000666":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/4167740619":"2024-02-19T00:29:39.703735+0000","172.21.15.59:0/999852549":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/3575161033":"2024-02-19T00:23:45.307159+0000","172.21.15.59:0/2403979355":"2024-02-19T00:24:10.195218+0000","172.21.15.59:0/642821245":"2024-02-19T00:23:45.307159+0000","172.21.15.59:6800/1656814063":"2024-02-19T00:23:45.307159+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":14,"snaps":[{"begin":9,"length":2}]},{"pool":60,"snaps":[{"begin":6,"length":2}]},{"pool":78,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":86,"snaps":[{"begin":2,"length":1}]},{"pool":94,"snaps":[{"begin":6,"length":2}]},{"pool":97,"snaps":[{"begin":4,"length":4}]},{"pool":108,"snaps":[{"begin":2,"length":1}]},{"pool":114,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1}]},{"pool":128,"snaps":[{"begin":2,"length":2},{"begin":6,"length":2},{"begin":10,"length":2}]},{"pool":134,"snaps":[{"begin":12,"length":1}]},{"pool":154,"snaps":[{"begin":14,"length":1}]},{"pool":166,"snaps":[{"begin":17,"length":1}]},{"pool":177,"snaps":[{"begin":20,"length":1}]},{"pool":237,"snaps":[{"begin":2,"length":1}]},{"pool":238,"snaps":[{"begin":4,"length":1}]},{"pool":241,"snaps":[{"begin":6,"length":1}]},{"pool":245,"snaps":[{"begin":9,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T00:47:50.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 10 librados: watch_flush enter 2024-02-18T00:47:50.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 10 librados: watch_flush exit 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 20 client.32220.objecter shutdown clearing up homeless session... 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 10 client.32220.objecter successfully canceled tick 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe2cc062c90 msgr2=0x7fe2cc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2751278498 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe2cc062c90 0x7fe2cc065150 secure :-1 s=READY pgs=3266 cs=0 l=1 rev1=1 rx=0x7fe2d4006fd0 tx=0x7fe2d4009060).stop 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 msgr2=0x7fe2e4108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 0x7fe2e4108470 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 rx=0x7fe2dc001df0 tx=0x7fe2dc004d10).stop 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 shutdown_connections 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2e4102250 0x7fe2e4108470 unknown :-1 s=CLOSED pgs=2734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2e40f8d80 0x7fe2e410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2751278498 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe2cc062c90 0x7fe2cc065150 unknown :-1 s=CLOSED pgs=3266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 --2- 172.21.15.59:0/2751278498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2e40f96e0 0x7fe2e410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 >> 172.21.15.59:0/2751278498 conn(0x7fe2e40f26f0 msgr2=0x7fe2e40f3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:50.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 shutdown_connections 2024-02-18T00:47:50.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 -- 172.21.15.59:0/2751278498 wait complete. 2024-02-18T00:47:50.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:50.982+0000 7fe2ea0e3700 1 librados: shutdown 2024-02-18T00:47:51.377 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:49.755711+0000 osd.3 (osd.3) 481 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:47:51.377 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:49.756792+0000 osd.3 (osd.3) 482 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:50.061966+0000 mgr.y (mgr.24454) 748 : cluster [DBG] pgmap v1330: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:50.738002+0000 osd.2 (osd.2) 369 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:50.738941+0000 osd.2 (osd.2) 370 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:50.980882+0000 mon.a (mon.0) 3854 : audit [DBG] from='client.? 172.21.15.59:0/2751278498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:49.755711+0000 osd.3 (osd.3) 481 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:49.756792+0000 osd.3 (osd.3) 482 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:50.061966+0000 mgr.y (mgr.24454) 748 : cluster [DBG] pgmap v1330: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:50.738002+0000 osd.2 (osd.2) 369 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:50.738941+0000 osd.2 (osd.2) 370 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:47:51.378 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:50.980882+0000 mon.a (mon.0) 3854 : audit [DBG] from='client.? 172.21.15.59:0/2751278498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:47:51.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:47:51.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:49.755711+0000 osd.3 (osd.3) 481 : cluster [DBG] 2.1e scrub starts 2024-02-18T00:47:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:49.756792+0000 osd.3 (osd.3) 482 : cluster [DBG] 2.1e scrub ok 2024-02-18T00:47:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:50.061966+0000 mgr.y (mgr.24454) 748 : cluster [DBG] pgmap v1330: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 826 B/s rd, 0 op/s 2024-02-18T00:47:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:50.738002+0000 osd.2 (osd.2) 369 : cluster [DBG] 2.10 scrub starts 2024-02-18T00:47:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:50.738941+0000 osd.2 (osd.2) 370 : cluster [DBG] 2.10 scrub ok 2024-02-18T00:47:51.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:50.980882+0000 mon.a (mon.0) 3854 : audit [DBG] from='client.? 172.21.15.59:0/2751278498' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T00:47:51.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-18T00:47:52.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:47:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:47:52.052 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.023174+0000 osd.0 (osd.0) 404 : cluster [DBG] 25.e deep-scrub starts 2024-02-18T00:47:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.024512+0000 osd.0 (osd.0) 405 : cluster [DBG] 25.e deep-scrub ok 2024-02-18T00:47:52.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:47:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:47:52] "GET /metrics HTTP/1.1" 200 187523 "" "Prometheus/2.33.4" 2024-02-18T00:47:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.023174+0000 osd.0 (osd.0) 404 : cluster [DBG] 25.e deep-scrub starts 2024-02-18T00:47:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.024512+0000 osd.0 (osd.0) 405 : cluster [DBG] 25.e deep-scrub ok 2024-02-18T00:47:52.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.023174+0000 osd.0 (osd.0) 404 : cluster [DBG] 25.e deep-scrub starts 2024-02-18T00:47:52.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.024512+0000 osd.0 (osd.0) 405 : cluster [DBG] 25.e deep-scrub ok 2024-02-18T00:47:53.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:47:51.738877+0000 mgr.y (mgr.24454) 749 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:53.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.062927+0000 mgr.y (mgr.24454) 750 : cluster [DBG] pgmap v1331: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:53.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:47:51.738877+0000 mgr.y (mgr.24454) 749 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:53.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.062927+0000 mgr.y (mgr.24454) 750 : cluster [DBG] pgmap v1331: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:47:51.738877+0000 mgr.y (mgr.24454) 749 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:47:53.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.062927+0000 mgr.y (mgr.24454) 750 : cluster [DBG] pgmap v1331: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:54.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 -- 172.21.15.59:0/351669851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 msgr2=0x7fcc600fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 --2- 172.21.15.59:0/351669851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 0x7fcc600fdf90 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 rx=0x7fcc58009be0 tx=0x7fcc58004d10).stop 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 -- 172.21.15.59:0/351669851 shutdown_connections 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 --2- 172.21.15.59:0/351669851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc600f8420 0x7fcc60104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 --2- 172.21.15.59:0/351669851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 0x7fcc600fdf90 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 --2- 172.21.15.59:0/351669851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc600fe510 0x7fcc600f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 -- 172.21.15.59:0/351669851 >> 172.21.15.59:0/351669851 conn(0x7fcc600f3800 msgr2=0x7fcc600f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.095+0000 7fcc65cec700 1 -- 172.21.15.59:0/351669851 shutdown_connections 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.096+0000 7fcc65cec700 1 -- 172.21.15.59:0/351669851 wait complete. 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.096+0000 7fcc65cec700 1 librados: starting msgr at 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.096+0000 7fcc65cec700 1 librados: starting objecter 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.096+0000 7fcc65cec700 1 Processor -- start 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.096+0000 7fcc65cec700 1 -- start start 2024-02-18T00:47:54.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.096+0000 7fcc65cec700 1 librados: setting wanted keys 2024-02-18T00:47:54.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.096+0000 7fcc65cec700 1 librados: calling monclient init 2024-02-18T00:47:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc600f8420 0x7fcc6006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 0x7fcc6006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc600fe510 0x7fcc6006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60116600 con 0x7fcc600fe510 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60116770 con 0x7fcc600f8420 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60116930 con 0x7fcc600fdbb0 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc600fe510 0x7fcc6006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc600fe510 0x7fcc6006bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36180/0 (socket says 172.21.15.59:36180) 2024-02-18T00:47:54.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 -- 172.21.15.59:0/1154277618 learned_addr learned my addr 172.21.15.59:0/1154277618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5effd700 1 --2- 172.21.15.59:0/1154277618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 0x7fcc6006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5f7fe700 1 --2- 172.21.15.59:0/1154277618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc600f8420 0x7fcc6006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 -- 172.21.15.59:0/1154277618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 msgr2=0x7fcc6006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 --2- 172.21.15.59:0/1154277618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 0x7fcc6006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 -- 172.21.15.59:0/1154277618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc600f8420 msgr2=0x7fcc6006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 --2- 172.21.15.59:0/1154277618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc600f8420 0x7fcc6006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 -- 172.21.15.59:0/1154277618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50005040 con 0x7fcc600fe510 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5effd700 1 --2- 172.21.15.59:0/1154277618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcc600fdbb0 0x7fcc6006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5ffff700 1 --2- 172.21.15.59:0/1154277618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc600fe510 0x7fcc6006bab0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 rx=0x7fcc50004920 tx=0x7fcc50009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5cff9700 10 client.?.objecter ms_handle_connect 0x7fcc600fe510 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5cff9700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:54.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50007b20 con 0x7fcc600fe510 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc580097c0 con 0x7fcc600fe510 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5000f040 con 0x7fcc600fe510 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc500148e0 con 0x7fcc600fe510 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 10 client.32226.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc60116fa0 con 0x7fcc600fe510 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.097+0000 7fcc65cec700 1 librados: init done 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.098+0000 7fcc65cec700 10 client.32226.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.098+0000 7fcc65cec700 15 client.32226.objecter _session_command_op_assign -1 1 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.098+0000 7fcc65cec700 20 client.32226.objecter _get_session osd=-1 returning homeless 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.098+0000 7fcc65cec700 10 client.32226.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.098+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc6005ee80 con 0x7fcc600fe510 2024-02-18T00:47:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.099+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fcc50014d10 con 0x7fcc600fe510 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc4c062c90 0x7fcc4c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(847..847 src has 418..847) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fcc5007d550 con 0x7fcc600fe510 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 10 client.32226.objecter ms_dispatch 0x7fcc60001550 osd_map(847..847 src has 418..847) v4 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 3 client.32226.objecter handle_osd_map got epochs [847,847] > 0 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 3 client.32226.objecter handle_osd_map decoding full epoch 847 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5f7fe700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc4c062c90 0x7fcc4c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 10 client.32226.objecter checking command 1 2024-02-18T00:47:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc4c06e6d0 0x7fcc4c070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 20 client.32226.objecter _get_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 20 client.32226.objecter put_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 15 client.32226.objecter _session_command_op_remove -1 1 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 20 client.32226.objecter _get_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 20 client.32226.objecter get_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 15 client.32226.objecter _session_command_op_assign 0 1 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 20 client.32226.objecter put_session s=0x7fcc4c06e020 osd=0 4 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 10 client.32226.objecter _send_command 1 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 --> v2:172.21.15.59:6801/3700238977 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc4c0711b0 con 0x7fcc4c06e6d0 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 20 client.32226.objecter dump_active .. 0 homeless 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=847) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc5007d900 con 0x7fcc600fe510 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5effd700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc4c06e6d0 0x7fcc4c070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:54.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5f7fe700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc4c062c90 0x7fcc4c065150 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 rx=0x7fcc5800b3b0 tx=0x7fcc58004d10).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 10 client.32226.objecter ms_handle_connect 0x7fcc4c062c90 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5cff9700 10 client.32226.objecter ms_handle_connect 0x7fcc4c06e6d0 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.100+0000 7fcc5effd700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc4c06e6d0 0x7fcc4c070b70 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.101+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== osd.0 v2:172.21.15.59:6801/3700238977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcc4c0711b0 con 0x7fcc4c06e6d0 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.101+0000 7fcc5cff9700 10 client.32226.objecter ms_dispatch 0x7fcc60001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.101+0000 7fcc5cff9700 10 client.32226.objecter _finish_command 1 = system:0 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.101+0000 7fcc5cff9700 20 client.32226.objecter put_session s=0x7fcc4c06e020 osd=0 4 2024-02-18T00:47:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.101+0000 7fcc5cff9700 15 client.32226.objecter _session_command_op_remove 0 1 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 10 client.32226.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 15 client.32226.objecter _session_command_op_assign -1 2 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 20 client.32226.objecter _get_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 20 client.32226.objecter put_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 20 client.32226.objecter _get_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 15 client.32226.objecter _session_command_op_remove -1 2 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 20 client.32226.objecter get_session s=0x7fcc4c06e020 osd=0 3 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 15 client.32226.objecter _session_command_op_assign 0 2 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 20 client.32226.objecter put_session s=0x7fcc4c06e020 osd=0 4 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 10 client.32226.objecter _send_command 2 2024-02-18T00:47:54.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 --> v2:172.21.15.59:6801/3700238977 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc60107110 con 0x7fcc4c06e6d0 2024-02-18T00:47:54.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc5cff9700 1 -- 172.21.15.59:0/1154277618 <== osd.0 v2:172.21.15.59:6801/3700238977 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcc60107110 con 0x7fcc4c06e6d0 2024-02-18T00:47:54.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc5cff9700 10 client.32226.objecter ms_dispatch 0x7fcc60001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:47:54.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc5cff9700 10 client.32226.objecter _finish_command 2 = system:0 2024-02-18T00:47:54.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc5cff9700 20 client.32226.objecter put_session s=0x7fcc4c06e020 osd=0 4 2024-02-18T00:47:54.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.115+0000 7fcc5cff9700 15 client.32226.objecter _session_command_op_remove 0 2 2024-02-18T00:47:54.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 10 librados: watch_flush enter 2024-02-18T00:47:54.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 10 librados: watch_flush exit 2024-02-18T00:47:54.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 10 client.32226.objecter close_session for osd.0 2024-02-18T00:47:54.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc4c06e6d0 msgr2=0x7fcc4c070b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6801/3700238977 conn(0x7fcc4c06e6d0 0x7fcc4c070b70 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 20 client.32226.objecter put_session s=0x7fcc4c06e020 osd=0 1 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 20 client.32226.objecter shutdown clearing up homeless session... 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 10 client.32226.objecter successfully canceled tick 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc4c062c90 msgr2=0x7fcc4c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 --2- 172.21.15.59:0/1154277618 >> v2:172.21.15.59:6800/3721891797 conn(0x7fcc4c062c90 0x7fcc4c065150 secure :-1 s=READY pgs=3267 cs=0 l=1 rev1=1 rx=0x7fcc5800b3b0 tx=0x7fcc58004d10).stop 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc600fe510 msgr2=0x7fcc6006bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 --2- 172.21.15.59:0/1154277618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc600fe510 0x7fcc6006bab0 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 rx=0x7fcc50004920 tx=0x7fcc50009cb0).stop 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc5ffff700 1 -- 172.21.15.59:0/1154277618 reap_dead start 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 shutdown_connections 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 >> 172.21.15.59:0/1154277618 conn(0x7fcc600f3800 msgr2=0x7fcc600f53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 shutdown_connections 2024-02-18T00:47:54.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 -- 172.21.15.59:0/1154277618 wait complete. 2024-02-18T00:47:54.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:54.116+0000 7fcc65cec700 1 librados: shutdown 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.508859+0000 osd.6 (osd.6) 387 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.510502+0000 osd.6 (osd.6) 388 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.686066+0000 osd.3 (osd.3) 483 : cluster [DBG] 25.4 deep-scrub starts 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.687442+0000 osd.3 (osd.3) 484 : cluster [DBG] 25.4 deep-scrub ok 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.804540+0000 osd.1 (osd.1) 437 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:52.807799+0000 osd.1 (osd.1) 438 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:53.413461+0000 osd.4 (osd.4) 477 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:47:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:53.415035+0000 osd.4 (osd.4) 478 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.508859+0000 osd.6 (osd.6) 387 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.510502+0000 osd.6 (osd.6) 388 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.686066+0000 osd.3 (osd.3) 483 : cluster [DBG] 25.4 deep-scrub starts 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.687442+0000 osd.3 (osd.3) 484 : cluster [DBG] 25.4 deep-scrub ok 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.804540+0000 osd.1 (osd.1) 437 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:52.807799+0000 osd.1 (osd.1) 438 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:53.413461+0000 osd.4 (osd.4) 477 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:47:54.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:53.415035+0000 osd.4 (osd.4) 478 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.508859+0000 osd.6 (osd.6) 387 : cluster [DBG] 5.1f scrub starts 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.510502+0000 osd.6 (osd.6) 388 : cluster [DBG] 5.1f scrub ok 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.686066+0000 osd.3 (osd.3) 483 : cluster [DBG] 25.4 deep-scrub starts 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.687442+0000 osd.3 (osd.3) 484 : cluster [DBG] 25.4 deep-scrub ok 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.804540+0000 osd.1 (osd.1) 437 : cluster [DBG] 2.9 scrub starts 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:52.807799+0000 osd.1 (osd.1) 438 : cluster [DBG] 2.9 scrub ok 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:53.413461+0000 osd.4 (osd.4) 477 : cluster [DBG] 25.19 scrub starts 2024-02-18T00:47:54.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:53.415035+0000 osd.4 (osd.4) 478 : cluster [DBG] 25.19 scrub ok 2024-02-18T00:47:55.451 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:53.851882+0000 osd.1 (osd.1) 439 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:47:55.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:53.855217+0000 osd.1 (osd.1) 440 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:47:55.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:54.063279+0000 mgr.y (mgr.24454) 751 : cluster [DBG] pgmap v1332: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:47:55.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:54.379641+0000 osd.4 (osd.4) 479 : cluster [DBG] 4.22 deep-scrub starts 2024-02-18T00:47:55.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:54.381649+0000 osd.4 (osd.4) 480 : cluster [DBG] 4.22 deep-scrub ok 2024-02-18T00:47:55.452 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:53.851882+0000 osd.1 (osd.1) 439 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:47:55.453 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:53.855217+0000 osd.1 (osd.1) 440 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:47:55.453 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:54.063279+0000 mgr.y (mgr.24454) 751 : cluster [DBG] pgmap v1332: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:47:55.453 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:54.379641+0000 osd.4 (osd.4) 479 : cluster [DBG] 4.22 deep-scrub starts 2024-02-18T00:47:55.453 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:54.381649+0000 osd.4 (osd.4) 480 : cluster [DBG] 4.22 deep-scrub ok 2024-02-18T00:47:55.476 INFO:teuthology.orchestra.run.smithi059.stdout:34359738631 2024-02-18T00:47:55.476 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-18T00:47:55.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:53.851882+0000 osd.1 (osd.1) 439 : cluster [DBG] 25.12 scrub starts 2024-02-18T00:47:55.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:53.855217+0000 osd.1 (osd.1) 440 : cluster [DBG] 25.12 scrub ok 2024-02-18T00:47:55.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:54.063279+0000 mgr.y (mgr.24454) 751 : cluster [DBG] pgmap v1332: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:47:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:54.379641+0000 osd.4 (osd.4) 479 : cluster [DBG] 4.22 deep-scrub starts 2024-02-18T00:47:55.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:54.381649+0000 osd.4 (osd.4) 480 : cluster [DBG] 4.22 deep-scrub ok 2024-02-18T00:47:55.761 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:56.294 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:54.665585+0000 osd.3 (osd.3) 485 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:47:56.294 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:54.667363+0000 osd.3 (osd.3) 486 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:47:56.294 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:54.862033+0000 osd.1 (osd.1) 441 : cluster [DBG] 2.a scrub starts 2024-02-18T00:47:56.294 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:54.865330+0000 osd.1 (osd.1) 442 : cluster [DBG] 2.a scrub ok 2024-02-18T00:47:56.295 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:54.665585+0000 osd.3 (osd.3) 485 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:47:56.295 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:54.667363+0000 osd.3 (osd.3) 486 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:47:56.295 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:54.862033+0000 osd.1 (osd.1) 441 : cluster [DBG] 2.a scrub starts 2024-02-18T00:47:56.295 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:54.865330+0000 osd.1 (osd.1) 442 : cluster [DBG] 2.a scrub ok 2024-02-18T00:47:56.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:54.665585+0000 osd.3 (osd.3) 485 : cluster [DBG] 4.11 scrub starts 2024-02-18T00:47:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:54.667363+0000 osd.3 (osd.3) 486 : cluster [DBG] 4.11 scrub ok 2024-02-18T00:47:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:54.862033+0000 osd.1 (osd.1) 441 : cluster [DBG] 2.a scrub starts 2024-02-18T00:47:56.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:54.865330+0000 osd.1 (osd.1) 442 : cluster [DBG] 2.a scrub ok 2024-02-18T00:47:57.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:56.063976+0000 mgr.y (mgr.24454) 752 : cluster [DBG] pgmap v1333: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:47:57.467 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:56.063976+0000 mgr.y (mgr.24454) 752 : cluster [DBG] pgmap v1333: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:47:57.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:56.063976+0000 mgr.y (mgr.24454) 752 : cluster [DBG] pgmap v1333: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:47:57.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 -- 172.21.15.59:0/3316242307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200fd510 msgr2=0x7f61200f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:57.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 --2- 172.21.15.59:0/3316242307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200fd510 0x7f61200f6d40 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 rx=0x7f6110004870 tx=0x7f6110009cb0).stop 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 -- 172.21.15.59:0/3316242307 shutdown_connections 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 --2- 172.21.15.59:0/3316242307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61200f7400 0x7f6120103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 --2- 172.21.15.59:0/3316242307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61200fcbb0 0x7f61200fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 --2- 172.21.15.59:0/3316242307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200fd510 0x7f61200f6d40 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 -- 172.21.15.59:0/3316242307 >> 172.21.15.59:0/3316242307 conn(0x7f61200f2820 msgr2=0x7f61200f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 -- 172.21.15.59:0/3316242307 shutdown_connections 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 -- 172.21.15.59:0/3316242307 wait complete. 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 librados: starting msgr at 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 librados: starting objecter 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 Processor -- start 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 -- start start 2024-02-18T00:47:57.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 librados: setting wanted keys 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.760+0000 7f6128276700 1 librados: calling monclient init 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6128276700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200f7400 0x7f61201092e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6128276700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61200fcbb0 0x7f61201097d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6128276700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61200fd510 0x7f6120109db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6128276700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61201144c0 con 0x7f61200f7400 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6128276700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6120114630 con 0x7f61200fd510 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6128276700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61201147f0 con 0x7f61200fcbb0 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200f7400 0x7f61201092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6125811700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61200fcbb0 0x7f61201097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200f7400 0x7f61201092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36212/0 (socket says 172.21.15.59:36212) 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6125811700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61200fcbb0 0x7f61201097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51864/0 (socket says 172.21.15.59:51864) 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 -- 172.21.15.59:0/2777008488 learned_addr learned my addr 172.21.15.59:0/2777008488 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:47:57.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126813700 1 --2- 172.21.15.59:0/2777008488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61200fd510 0x7f6120109db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 -- 172.21.15.59:0/2777008488 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61200fcbb0 msgr2=0x7f61201097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 --2- 172.21.15.59:0/2777008488 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61200fcbb0 0x7f61201097d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 -- 172.21.15.59:0/2777008488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61200fd510 msgr2=0x7f6120109db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 --2- 172.21.15.59:0/2777008488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61200fd510 0x7f6120109db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 -- 172.21.15.59:0/2777008488 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c005040 con 0x7f61200f7400 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6125811700 1 --2- 172.21.15.59:0/2777008488 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f61200fcbb0 0x7f61201097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126813700 1 --2- 172.21.15.59:0/2777008488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61200fd510 0x7f6120109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6126012700 1 --2- 172.21.15.59:0/2777008488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200f7400 0x7f61201092e0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 rx=0x7f611c0048f0 tx=0x7f611c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6117779700 10 client.?.objecter ms_handle_connect 0x7f61200f7400 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6117779700 10 client.?.objecter resend_mon_ops 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c007b00 con 0x7f61200f7400 2024-02-18T00:47:57.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f611c00f040 con 0x7f61200f7400 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6110005040 con 0x7f61200f7400 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.761+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c014a00 con 0x7f61200f7400 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 10 client.32232.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6120114e30 con 0x7f61200f7400 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 1 librados: init done 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 10 client.32232.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 15 client.32232.objecter _session_command_op_assign -1 1 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 20 client.32232.objecter _get_session osd=-1 returning homeless 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 10 client.32232.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:47:57.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.762+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f612005fa60 con 0x7f61200f7400 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.763+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f611c00e580 con 0x7f61200f7400 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.763+0000 7f6117779700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6800/3721891797 conn(0x7f610c062c90 0x7f610c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.763+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(847..847 src has 418..847) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f611c07cca0 con 0x7f61200f7400 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.763+0000 7f6117779700 10 client.32232.objecter ms_dispatch 0x7f6120001550 osd_map(847..847 src has 418..847) v4 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.763+0000 7f6117779700 3 client.32232.objecter handle_osd_map got epochs [847,847] > 0 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.763+0000 7f6117779700 3 client.32232.objecter handle_osd_map decoding full epoch 847 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.763+0000 7f6125811700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6800/3721891797 conn(0x7f610c062c90 0x7f610c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 10 client.32232.objecter checking command 1 2024-02-18T00:47:57.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6805/26279110 conn(0x7f610c06e6e0 0x7f610c070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 20 client.32232.objecter _get_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 20 client.32232.objecter put_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 15 client.32232.objecter _session_command_op_remove -1 1 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 20 client.32232.objecter _get_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 20 client.32232.objecter get_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 15 client.32232.objecter _session_command_op_assign 1 1 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 20 client.32232.objecter put_session s=0x7f610c06e030 osd=1 4 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 10 client.32232.objecter _send_command 1 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 --> v2:172.21.15.59:6805/26279110 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f610c0711c0 con 0x7f610c06e6e0 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 20 client.32232.objecter dump_active .. 0 homeless 2024-02-18T00:47:57.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=847) v2 ==== 24+0+0 (secure 0 0 0) 0x7f611c081220 con 0x7f61200f7400 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6126813700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6805/26279110 conn(0x7f610c06e6e0 0x7f610c070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 10 client.32232.objecter ms_handle_connect 0x7f610c062c90 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6125811700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6800/3721891797 conn(0x7f610c062c90 0x7f610c065150 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 rx=0x7f6110004ee0 tx=0x7f611000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6126813700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6805/26279110 conn(0x7f610c06e6e0 0x7f610c070b80 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.764+0000 7f6117779700 10 client.32232.objecter ms_handle_connect 0x7f610c06e6e0 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.765+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== osd.1 v2:172.21.15.59:6805/26279110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f610c0711c0 con 0x7f610c06e6e0 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.765+0000 7f6117779700 10 client.32232.objecter ms_dispatch 0x7f6120001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.765+0000 7f6117779700 10 client.32232.objecter _finish_command 1 = system:0 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.765+0000 7f6117779700 20 client.32232.objecter put_session s=0x7f610c06e030 osd=1 4 2024-02-18T00:47:57.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.765+0000 7f6117779700 15 client.32232.objecter _session_command_op_remove 1 1 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 10 client.32232.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 15 client.32232.objecter _session_command_op_assign -1 2 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 20 client.32232.objecter _get_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 20 client.32232.objecter put_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 20 client.32232.objecter _get_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 15 client.32232.objecter _session_command_op_remove -1 2 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 20 client.32232.objecter get_session s=0x7f610c06e030 osd=1 3 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 15 client.32232.objecter _session_command_op_assign 1 2 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 20 client.32232.objecter put_session s=0x7f610c06e030 osd=1 4 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 10 client.32232.objecter _send_command 2 2024-02-18T00:47:57.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 --> v2:172.21.15.59:6805/26279110 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f612005ee80 con 0x7f610c06e6e0 2024-02-18T00:47:57.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6117779700 1 -- 172.21.15.59:0/2777008488 <== osd.1 v2:172.21.15.59:6805/26279110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f612005ee80 con 0x7f610c06e6e0 2024-02-18T00:47:57.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6117779700 10 client.32232.objecter ms_dispatch 0x7f6120001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:47:57.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6117779700 10 client.32232.objecter _finish_command 2 = system:0 2024-02-18T00:47:57.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6117779700 20 client.32232.objecter put_session s=0x7f610c06e030 osd=1 4 2024-02-18T00:47:57.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.779+0000 7f6117779700 15 client.32232.objecter _session_command_op_remove 1 2 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 10 librados: watch_flush enter 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 10 librados: watch_flush exit 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 10 client.32232.objecter close_session for osd.1 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6805/26279110 conn(0x7f610c06e6e0 msgr2=0x7f610c070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6805/26279110 conn(0x7f610c06e6e0 0x7f610c070b80 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 20 client.32232.objecter put_session s=0x7f610c06e030 osd=1 1 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 20 client.32232.objecter shutdown clearing up homeless session... 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 10 client.32232.objecter successfully canceled tick 2024-02-18T00:47:57.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6800/3721891797 conn(0x7f610c062c90 msgr2=0x7f610c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 --2- 172.21.15.59:0/2777008488 >> v2:172.21.15.59:6800/3721891797 conn(0x7f610c062c90 0x7f610c065150 secure :-1 s=READY pgs=3268 cs=0 l=1 rev1=1 rx=0x7f6110004ee0 tx=0x7f611000b040).stop 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200f7400 msgr2=0x7f61201092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 --2- 172.21.15.59:0/2777008488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61200f7400 0x7f61201092e0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 rx=0x7f611c0048f0 tx=0x7f611c009cb0).stop 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6126813700 1 -- 172.21.15.59:0/2777008488 reap_dead start 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 shutdown_connections 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 >> 172.21.15.59:0/2777008488 conn(0x7f61200f2820 msgr2=0x7f61200f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 shutdown_connections 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 -- 172.21.15.59:0/2777008488 wait complete. 2024-02-18T00:47:57.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:47:57.780+0000 7f6128276700 1 librados: shutdown 2024-02-18T00:47:58.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:57.183630+0000 osd.7 (osd.7) 293 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:47:58.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:57.184672+0000 osd.7 (osd.7) 294 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:47:58.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:57.183630+0000 osd.7 (osd.7) 293 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:47:58.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:57.184672+0000 osd.7 (osd.7) 294 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:47:58.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:57.183630+0000 osd.7 (osd.7) 293 : cluster [DBG] 6.8 scrub starts 2024-02-18T00:47:58.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:57.184672+0000 osd.7 (osd.7) 294 : cluster [DBG] 6.8 scrub ok 2024-02-18T00:47:58.682 INFO:teuthology.orchestra.run.smithi059.stdout:55834575106 2024-02-18T00:47:58.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-18T00:47:58.969 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:47:59.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:57.807456+0000 osd.1 (osd.1) 443 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:47:59.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:57.808613+0000 osd.1 (osd.1) 444 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:47:59.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:58.064566+0000 mgr.y (mgr.24454) 753 : cluster [DBG] pgmap v1334: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:59.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:58.146336+0000 osd.7 (osd.7) 295 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:47:59.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:58.147303+0000 osd.7 (osd.7) 296 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:47:59.444 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:57.807456+0000 osd.1 (osd.1) 443 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:47:59.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:57.808613+0000 osd.1 (osd.1) 444 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:47:59.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:58.064566+0000 mgr.y (mgr.24454) 753 : cluster [DBG] pgmap v1334: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:59.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:58.146336+0000 osd.7 (osd.7) 295 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:47:59.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:47:59 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:58.147303+0000 osd.7 (osd.7) 296 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:47:59.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:57.807456+0000 osd.1 (osd.1) 443 : cluster [DBG] 3.0 scrub starts 2024-02-18T00:47:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:57.808613+0000 osd.1 (osd.1) 444 : cluster [DBG] 3.0 scrub ok 2024-02-18T00:47:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:58.064566+0000 mgr.y (mgr.24454) 753 : cluster [DBG] pgmap v1334: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:47:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:58.146336+0000 osd.7 (osd.7) 295 : cluster [DBG] 1.0 scrub starts 2024-02-18T00:47:59.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:47:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:58.147303+0000 osd.7 (osd.7) 296 : cluster [DBG] 1.0 scrub ok 2024-02-18T00:48:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:58.455534+0000 osd.6 (osd.6) 389 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:48:00.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:58.456467+0000 osd.6 (osd.6) 390 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:48:00.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:58.455534+0000 osd.6 (osd.6) 389 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:48:00.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:58.456467+0000 osd.6 (osd.6) 390 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:48:00.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:58.455534+0000 osd.6 (osd.6) 389 : cluster [DBG] 2.11 scrub starts 2024-02-18T00:48:00.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:58.456467+0000 osd.6 (osd.6) 390 : cluster [DBG] 2.11 scrub ok 2024-02-18T00:48:01.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 -- 172.21.15.59:0/2506672376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fdbb0 msgr2=0x7fb3f00fdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:01.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 --2- 172.21.15.59:0/2506672376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fdbb0 0x7fb3f00fdf90 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 rx=0x7fb3e4004870 tx=0x7fb3e4009cb0).stop 2024-02-18T00:48:01.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 -- 172.21.15.59:0/2506672376 shutdown_connections 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 --2- 172.21.15.59:0/2506672376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3f00f8420 0x7fb3f0104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 --2- 172.21.15.59:0/2506672376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fdbb0 0x7fb3f00fdf90 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 --2- 172.21.15.59:0/2506672376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3f00fe510 0x7fb3f00f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 -- 172.21.15.59:0/2506672376 >> 172.21.15.59:0/2506672376 conn(0x7fb3f00f3800 msgr2=0x7fb3f00f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 -- 172.21.15.59:0/2506672376 shutdown_connections 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.080+0000 7fb3f5283700 1 -- 172.21.15.59:0/2506672376 wait complete. 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 librados: starting msgr at 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 librados: starting objecter 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 Processor -- start 2024-02-18T00:48:01.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 -- start start 2024-02-18T00:48:01.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 librados: setting wanted keys 2024-02-18T00:48:01.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 librados: calling monclient init 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3f00f8420 0x7fb3f010dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3f00fdbb0 0x7fb3f010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fe510 0x7fb3f0108180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0108670 con 0x7fb3f00fe510 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01087e0 con 0x7fb3f00f8420 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3f5283700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01089a0 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3ef7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fe510 0x7fb3f0108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3ee7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3f00fdbb0 0x7fb3f010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:01.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3ee7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3f00fdbb0 0x7fb3f010e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51204/0 (socket says 172.21.15.59:51204) 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3ef7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fe510 0x7fb3f0108180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37278/0 (socket says 172.21.15.59:37278) 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3ee7fc700 1 -- 172.21.15.59:0/3803703659 learned_addr learned my addr 172.21.15.59:0/3803703659 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.081+0000 7fb3eeffd700 1 --2- 172.21.15.59:0/3803703659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3f00f8420 0x7fb3f010dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3ee7fc700 1 -- 172.21.15.59:0/3803703659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3f00f8420 msgr2=0x7fb3f010dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3ee7fc700 1 --2- 172.21.15.59:0/3803703659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3f00f8420 0x7fb3f010dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3ee7fc700 1 -- 172.21.15.59:0/3803703659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fe510 msgr2=0x7fb3f0108180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3ee7fc700 1 --2- 172.21.15.59:0/3803703659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fe510 0x7fb3f0108180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3ee7fc700 1 -- 172.21.15.59:0/3803703659 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4005040 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3ef7fe700 1 --2- 172.21.15.59:0/3803703659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3f00fe510 0x7fb3f0108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3ee7fc700 1 --2- 172.21.15.59:0/3803703659 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3f00fdbb0 0x7fb3f010e4d0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7fb3e000c810 tx=0x7fb3e0004330).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3d7fff700 10 client.?.objecter ms_handle_connect 0x7fb3f00fdbb0 2024-02-18T00:48:01.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3d7fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e00043a0 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e000e970 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f0108c80 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0012990 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3f5283700 10 client.42041.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f0109000 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.082+0000 7fb3f5283700 1 librados: init done 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.083+0000 7fb3f5283700 10 client.42041.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.083+0000 7fb3f5283700 15 client.42041.objecter _session_command_op_assign -1 1 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.083+0000 7fb3f5283700 20 client.42041.objecter _get_session osd=-1 returning homeless 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.083+0000 7fb3f5283700 10 client.42041.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.083+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3f005ee80 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.083+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb3e000eae0 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3d8062c90 0x7fb3d8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:01.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3eeffd700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3d8062c90 0x7fb3d8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(847..847 src has 418..847) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fb3e007c7c0 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 10 client.42041.objecter ms_dispatch 0x7fb3f0001550 osd_map(847..847 src has 418..847) v4 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 3 client.42041.objecter handle_osd_map got epochs [847,847] > 0 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 3 client.42041.objecter handle_osd_map decoding full epoch 847 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 10 client.42041.objecter checking command 1 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb3d806e6e0 0x7fb3d8070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 20 client.42041.objecter _get_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 20 client.42041.objecter put_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 15 client.42041.objecter _session_command_op_remove -1 1 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 20 client.42041.objecter _get_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 20 client.42041.objecter get_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 15 client.42041.objecter _session_command_op_assign 2 1 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 20 client.42041.objecter put_session s=0x7fb3d806e030 osd=2 4 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 10 client.42041.objecter _send_command 1 2024-02-18T00:48:01.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 --> v2:172.21.15.59:6809/1153193637 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3d80711c0 con 0x7fb3d806e6e0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 20 client.42041.objecter dump_active .. 0 homeless 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=847) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3e007ed80 con 0x7fb3f00fdbb0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3ef7fe700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb3d806e6e0 0x7fb3d8070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3d7fff700 10 client.42041.objecter ms_handle_connect 0x7fb3d8062c90 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.084+0000 7fb3eeffd700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3d8062c90 0x7fb3d8065150 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 rx=0x7fb3e40043d0 tx=0x7fb3e400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.085+0000 7fb3d7fff700 10 client.42041.objecter ms_handle_connect 0x7fb3d806e6e0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.085+0000 7fb3ef7fe700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb3d806e6e0 0x7fb3d8070b80 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.085+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== osd.2 v2:172.21.15.59:6809/1153193637 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb3d80711c0 con 0x7fb3d806e6e0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.085+0000 7fb3d7fff700 10 client.42041.objecter ms_dispatch 0x7fb3f0001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.085+0000 7fb3d7fff700 10 client.42041.objecter _finish_command 1 = system:0 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.085+0000 7fb3d7fff700 20 client.42041.objecter put_session s=0x7fb3d806e030 osd=2 4 2024-02-18T00:48:01.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.085+0000 7fb3d7fff700 15 client.42041.objecter _session_command_op_remove 2 1 2024-02-18T00:48:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 10 client.42041.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:48:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 15 client.42041.objecter _session_command_op_assign -1 2 2024-02-18T00:48:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 20 client.42041.objecter _get_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 20 client.42041.objecter put_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 20 client.42041.objecter _get_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 15 client.42041.objecter _session_command_op_remove -1 2 2024-02-18T00:48:01.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 20 client.42041.objecter get_session s=0x7fb3d806e030 osd=2 3 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 15 client.42041.objecter _session_command_op_assign 2 2 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 20 client.42041.objecter put_session s=0x7fb3d806e030 osd=2 4 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 10 client.42041.objecter _send_command 2 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.099+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 --> v2:172.21.15.59:6809/1153193637 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb3f01070a0 con 0x7fb3d806e6e0 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3d7fff700 1 -- 172.21.15.59:0/3803703659 <== osd.2 v2:172.21.15.59:6809/1153193637 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb3f01070a0 con 0x7fb3d806e6e0 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3d7fff700 10 client.42041.objecter ms_dispatch 0x7fb3f0001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3d7fff700 10 client.42041.objecter _finish_command 2 = system:0 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3d7fff700 20 client.42041.objecter put_session s=0x7fb3d806e030 osd=2 4 2024-02-18T00:48:01.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3d7fff700 15 client.42041.objecter _session_command_op_remove 2 2 2024-02-18T00:48:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 10 librados: watch_flush enter 2024-02-18T00:48:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 10 librados: watch_flush exit 2024-02-18T00:48:01.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 10 client.42041.objecter close_session for osd.2 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb3d806e6e0 msgr2=0x7fb3d8070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6809/1153193637 conn(0x7fb3d806e6e0 0x7fb3d8070b80 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 20 client.42041.objecter put_session s=0x7fb3d806e030 osd=2 1 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 20 client.42041.objecter shutdown clearing up homeless session... 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 10 client.42041.objecter successfully canceled tick 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3d8062c90 msgr2=0x7fb3d8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 --2- 172.21.15.59:0/3803703659 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3d8062c90 0x7fb3d8065150 secure :-1 s=READY pgs=3269 cs=0 l=1 rev1=1 rx=0x7fb3e40043d0 tx=0x7fb3e400b040).stop 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3f00fdbb0 msgr2=0x7fb3f010e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 --2- 172.21.15.59:0/3803703659 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3f00fdbb0 0x7fb3f010e4d0 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 rx=0x7fb3e000c810 tx=0x7fb3e0004330).stop 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3ef7fe700 1 -- 172.21.15.59:0/3803703659 reap_dead start 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 shutdown_connections 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 >> 172.21.15.59:0/3803703659 conn(0x7fb3f00f3800 msgr2=0x7fb3f00f52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:01.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 shutdown_connections 2024-02-18T00:48:01.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.100+0000 7fb3f5283700 1 -- 172.21.15.59:0/3803703659 wait complete. 2024-02-18T00:48:01.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:01.101+0000 7fb3f5283700 1 librados: shutdown 2024-02-18T00:48:01.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:59.706279+0000 osd.3 (osd.3) 487 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:47:59.708557+0000 osd.3 (osd.3) 488 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:00.065223+0000 mgr.y (mgr.24454) 754 : cluster [DBG] pgmap v1335: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:00.116083+0000 osd.7 (osd.7) 297 : cluster [DBG] 5.10 deep-scrub starts 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:00.116938+0000 osd.7 (osd.7) 298 : cluster [DBG] 5.10 deep-scrub ok 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:00.329659+0000 osd.4 (osd.4) 481 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:00.330681+0000 osd.4 (osd.4) 482 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:00.848409+0000 osd.2 (osd.2) 371 : cluster [DBG] 6.f scrub starts 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:00.851785+0000 osd.2 (osd.2) 372 : cluster [DBG] 6.f scrub ok 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.028982+0000 osd.0 (osd.0) 406 : cluster [DBG] 4.6 deep-scrub starts 2024-02-18T00:48:01.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.032004+0000 osd.0 (osd.0) 407 : cluster [DBG] 4.6 deep-scrub ok 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:59.706279+0000 osd.3 (osd.3) 487 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:47:59.708557+0000 osd.3 (osd.3) 488 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:00.065223+0000 mgr.y (mgr.24454) 754 : cluster [DBG] pgmap v1335: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:00.116083+0000 osd.7 (osd.7) 297 : cluster [DBG] 5.10 deep-scrub starts 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:00.116938+0000 osd.7 (osd.7) 298 : cluster [DBG] 5.10 deep-scrub ok 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:00.329659+0000 osd.4 (osd.4) 481 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:00.330681+0000 osd.4 (osd.4) 482 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:00.848409+0000 osd.2 (osd.2) 371 : cluster [DBG] 6.f scrub starts 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:00.851785+0000 osd.2 (osd.2) 372 : cluster [DBG] 6.f scrub ok 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.028982+0000 osd.0 (osd.0) 406 : cluster [DBG] 4.6 deep-scrub starts 2024-02-18T00:48:01.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:01 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.032004+0000 osd.0 (osd.0) 407 : cluster [DBG] 4.6 deep-scrub ok 2024-02-18T00:48:01.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:48:01.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:59.706279+0000 osd.3 (osd.3) 487 : cluster [DBG] 25.17 scrub starts 2024-02-18T00:48:01.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:47:59.708557+0000 osd.3 (osd.3) 488 : cluster [DBG] 25.17 scrub ok 2024-02-18T00:48:01.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:00.065223+0000 mgr.y (mgr.24454) 754 : cluster [DBG] pgmap v1335: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:01.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:00.116083+0000 osd.7 (osd.7) 297 : cluster [DBG] 5.10 deep-scrub starts 2024-02-18T00:48:01.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:00.116938+0000 osd.7 (osd.7) 298 : cluster [DBG] 5.10 deep-scrub ok 2024-02-18T00:48:01.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:00.329659+0000 osd.4 (osd.4) 481 : cluster [DBG] 5.1 scrub starts 2024-02-18T00:48:01.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:00.330681+0000 osd.4 (osd.4) 482 : cluster [DBG] 5.1 scrub ok 2024-02-18T00:48:01.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:00.848409+0000 osd.2 (osd.2) 371 : cluster [DBG] 6.f scrub starts 2024-02-18T00:48:01.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:00.851785+0000 osd.2 (osd.2) 372 : cluster [DBG] 6.f scrub ok 2024-02-18T00:48:01.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.028982+0000 osd.0 (osd.0) 406 : cluster [DBG] 4.6 deep-scrub starts 2024-02-18T00:48:01.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.032004+0000 osd.0 (osd.0) 407 : cluster [DBG] 4.6 deep-scrub ok 2024-02-18T00:48:02.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:48:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:48:02.130 INFO:teuthology.orchestra.run.smithi059.stdout:77309411584 2024-02-18T00:48:02.131 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-18T00:48:02.416 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:02.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:02] "GET /metrics HTTP/1.1" 200 187531 "" "Prometheus/2.33.4" 2024-02-18T00:48:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.101910+0000 osd.7 (osd.7) 299 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:48:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.102861+0000 osd.7 (osd.7) 300 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:48:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.329653+0000 osd.4 (osd.4) 483 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:48:02.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.333163+0000 osd.4 (osd.4) 484 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.101910+0000 osd.7 (osd.7) 299 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.102861+0000 osd.7 (osd.7) 300 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.329653+0000 osd.4 (osd.4) 483 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.333163+0000 osd.4 (osd.4) 484 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.101910+0000 osd.7 (osd.7) 299 : cluster [DBG] 5.4 scrub starts 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.102861+0000 osd.7 (osd.7) 300 : cluster [DBG] 5.4 scrub ok 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.329653+0000 osd.4 (osd.4) 483 : cluster [DBG] 2.25 scrub starts 2024-02-18T00:48:02.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.333163+0000 osd.4 (osd.4) 484 : cluster [DBG] 2.25 scrub ok 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.722604+0000 osd.1 (osd.1) 445 : cluster [DBG] 5.c scrub starts 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:01.725568+0000 osd.1 (osd.1) 446 : cluster [DBG] 5.c scrub ok 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:01.750076+0000 mgr.y (mgr.24454) 755 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:02.066039+0000 mgr.y (mgr.24454) 756 : cluster [DBG] pgmap v1336: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:02.799500+0000 osd.2 (osd.2) 373 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:02.800347+0000 osd.2 (osd.2) 374 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.722604+0000 osd.1 (osd.1) 445 : cluster [DBG] 5.c scrub starts 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:01.725568+0000 osd.1 (osd.1) 446 : cluster [DBG] 5.c scrub ok 2024-02-18T00:48:03.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:01.750076+0000 mgr.y (mgr.24454) 755 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:02.066039+0000 mgr.y (mgr.24454) 756 : cluster [DBG] pgmap v1336: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:02.799500+0000 osd.2 (osd.2) 373 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:02.800347+0000 osd.2 (osd.2) 374 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.722604+0000 osd.1 (osd.1) 445 : cluster [DBG] 5.c scrub starts 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:01.725568+0000 osd.1 (osd.1) 446 : cluster [DBG] 5.c scrub ok 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:01.750076+0000 mgr.y (mgr.24454) 755 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:02.066039+0000 mgr.y (mgr.24454) 756 : cluster [DBG] pgmap v1336: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:03.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:02.799500+0000 osd.2 (osd.2) 373 : cluster [DBG] 2.0 scrub starts 2024-02-18T00:48:03.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:02.800347+0000 osd.2 (osd.2) 374 : cluster [DBG] 2.0 scrub ok 2024-02-18T00:48:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:03.144042+0000 osd.7 (osd.7) 301 : cluster [DBG] 6.15 deep-scrub starts 2024-02-18T00:48:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:03.145322+0000 osd.7 (osd.7) 302 : cluster [DBG] 6.15 deep-scrub ok 2024-02-18T00:48:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:03.300143+0000 osd.4 (osd.4) 485 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:48:04.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:03.301551+0000 osd.4 (osd.4) 486 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:48:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:03.144042+0000 osd.7 (osd.7) 301 : cluster [DBG] 6.15 deep-scrub starts 2024-02-18T00:48:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:03.145322+0000 osd.7 (osd.7) 302 : cluster [DBG] 6.15 deep-scrub ok 2024-02-18T00:48:04.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:03.300143+0000 osd.4 (osd.4) 485 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:48:04.253 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:04 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:03.301551+0000 osd.4 (osd.4) 486 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:48:04.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:03.144042+0000 osd.7 (osd.7) 301 : cluster [DBG] 6.15 deep-scrub starts 2024-02-18T00:48:04.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:03.145322+0000 osd.7 (osd.7) 302 : cluster [DBG] 6.15 deep-scrub ok 2024-02-18T00:48:04.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:03.300143+0000 osd.4 (osd.4) 485 : cluster [DBG] 4.1 scrub starts 2024-02-18T00:48:04.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:03.301551+0000 osd.4 (osd.4) 486 : cluster [DBG] 4.1 scrub ok 2024-02-18T00:48:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 -- 172.21.15.59:0/780763590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80f7400 msgr2=0x7fd3f8103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 --2- 172.21.15.59:0/780763590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80f7400 0x7fd3f8103a20 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 rx=0x7fd3f0004870 tx=0x7fd3f0009cb0).stop 2024-02-18T00:48:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 -- 172.21.15.59:0/780763590 shutdown_connections 2024-02-18T00:48:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 --2- 172.21.15.59:0/780763590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3f80fcbb0 0x7fd3f80fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 --2- 172.21.15.59:0/780763590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80f7400 0x7fd3f8103a20 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:04.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 --2- 172.21.15.59:0/780763590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3f80fd510 0x7fd3f80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 -- 172.21.15.59:0/780763590 >> 172.21.15.59:0/780763590 conn(0x7fd3f80f2820 msgr2=0x7fd3f80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 -- 172.21.15.59:0/780763590 shutdown_connections 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.625+0000 7fd3fd345700 1 -- 172.21.15.59:0/780763590 wait complete. 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 librados: starting msgr at 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 librados: starting objecter 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 Processor -- start 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 -- start start 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 librados: setting wanted keys 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 librados: calling monclient init 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3f80f7400 0x7fd3f810ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80fcbb0 0x7fd3f810d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3f80fd510 0x7fd3f8107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:04.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8107510 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8107680 con 0x7fd3f80fd510 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3fd345700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8107840 con 0x7fd3f80f7400 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3f6ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3f80f7400 0x7fd3f810ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3f67fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80fcbb0 0x7fd3f810d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3f6ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3f80f7400 0x7fd3f810ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51220/0 (socket says 172.21.15.59:51220) 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3f6ffd700 1 -- 172.21.15.59:0/3154417640 learned_addr learned my addr 172.21.15.59:0/3154417640 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3f67fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80fcbb0 0x7fd3f810d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37294/0 (socket says 172.21.15.59:37294) 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.626+0000 7fd3f77fe700 1 --2- 172.21.15.59:0/3154417640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3f80fd510 0x7fd3f8107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3f67fc700 1 -- 172.21.15.59:0/3154417640 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3f80f7400 msgr2=0x7fd3f810ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3f67fc700 1 --2- 172.21.15.59:0/3154417640 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3f80f7400 0x7fd3f810ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3f67fc700 1 -- 172.21.15.59:0/3154417640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3f80fd510 msgr2=0x7fd3f8107020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3f67fc700 1 --2- 172.21.15.59:0/3154417640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3f80fd510 0x7fd3f8107020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:04.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3f67fc700 1 -- 172.21.15.59:0/3154417640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec005040 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3f6ffd700 1 --2- 172.21.15.59:0/3154417640 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3f80f7400 0x7fd3f810ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3f67fc700 1 --2- 172.21.15.59:0/3154417640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80fcbb0 0x7fd3f810d370 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 rx=0x7fd3ec004920 tx=0x7fd3ec009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3dffff700 10 client.?.objecter ms_handle_connect 0x7fd3f80fcbb0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3dffff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00e030 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f0005040 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3ec00f040 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec014850 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3fd345700 10 client.32244.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f8107e70 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.627+0000 7fd3fd345700 1 librados: init done 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.628+0000 7fd3fd345700 10 client.32244.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:48:04.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.628+0000 7fd3fd345700 15 client.32244.objecter _session_command_op_assign -1 1 2024-02-18T00:48:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.628+0000 7fd3fd345700 20 client.32244.objecter _get_session osd=-1 returning homeless 2024-02-18T00:48:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.628+0000 7fd3fd345700 10 client.32244.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.628+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3f805fa60 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.628+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fd3ec010500 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd3e0062c90 0x7fd3e0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(847..847 src has 418..847) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fd3ec07d0f0 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 10 client.32244.objecter ms_dispatch 0x7fd3f8001550 osd_map(847..847 src has 418..847) v4 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 3 client.32244.objecter handle_osd_map got epochs [847,847] > 0 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 3 client.32244.objecter handle_osd_map decoding full epoch 847 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3f6ffd700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd3e0062c90 0x7fd3e0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 10 client.32244.objecter checking command 1 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd3e006e6e0 0x7fd3e0070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 20 client.32244.objecter _get_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 20 client.32244.objecter put_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 15 client.32244.objecter _session_command_op_remove -1 1 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 20 client.32244.objecter _get_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 20 client.32244.objecter get_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 15 client.32244.objecter _session_command_op_assign 3 1 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 20 client.32244.objecter put_session s=0x7fd3e006e030 osd=3 4 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 10 client.32244.objecter _send_command 1 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 --> v2:172.21.15.59:6813/1805700964 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3e00711c0 con 0x7fd3e006e6e0 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 20 client.32244.objecter dump_active .. 0 homeless 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=847) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3ec07f710 con 0x7fd3f80fcbb0 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3f77fe700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd3e006e6e0 0x7fd3e0070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 10 client.32244.objecter ms_handle_connect 0x7fd3e0062c90 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3f6ffd700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd3e0062c90 0x7fd3e0065150 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 rx=0x7fd3e8004550 tx=0x7fd3e8009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3dffff700 10 client.32244.objecter ms_handle_connect 0x7fd3e006e6e0 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.629+0000 7fd3f77fe700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd3e006e6e0 0x7fd3e0070b80 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.630+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== osd.3 v2:172.21.15.59:6813/1805700964 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fd3e00711c0 con 0x7fd3e006e6e0 2024-02-18T00:48:04.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.630+0000 7fd3dffff700 10 client.32244.objecter ms_dispatch 0x7fd3f8001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.630+0000 7fd3dffff700 10 client.32244.objecter _finish_command 1 = system:0 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.630+0000 7fd3dffff700 20 client.32244.objecter put_session s=0x7fd3e006e030 osd=3 4 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.630+0000 7fd3dffff700 15 client.32244.objecter _session_command_op_remove 3 1 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 10 client.32244.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 15 client.32244.objecter _session_command_op_assign -1 2 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 20 client.32244.objecter _get_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 20 client.32244.objecter put_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 20 client.32244.objecter _get_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 15 client.32244.objecter _session_command_op_remove -1 2 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 20 client.32244.objecter get_session s=0x7fd3e006e030 osd=3 3 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 15 client.32244.objecter _session_command_op_assign 3 2 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 20 client.32244.objecter put_session s=0x7fd3e006e030 osd=3 4 2024-02-18T00:48:04.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 10 client.32244.objecter _send_command 2 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 --> v2:172.21.15.59:6813/1805700964 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd3f805ee80 con 0x7fd3e006e6e0 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3dffff700 1 -- 172.21.15.59:0/3154417640 <== osd.3 v2:172.21.15.59:6813/1805700964 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd3f805ee80 con 0x7fd3e006e6e0 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3dffff700 10 client.32244.objecter ms_dispatch 0x7fd3f8001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3dffff700 10 client.32244.objecter _finish_command 2 = system:0 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3dffff700 20 client.32244.objecter put_session s=0x7fd3e006e030 osd=3 4 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.644+0000 7fd3dffff700 15 client.32244.objecter _session_command_op_remove 3 2 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 10 librados: watch_flush enter 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 10 librados: watch_flush exit 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 10 client.32244.objecter close_session for osd.3 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd3e006e6e0 msgr2=0x7fd3e0070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6813/1805700964 conn(0x7fd3e006e6e0 0x7fd3e0070b80 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 20 client.32244.objecter put_session s=0x7fd3e006e030 osd=3 1 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 20 client.32244.objecter shutdown clearing up homeless session... 2024-02-18T00:48:04.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 10 client.32244.objecter successfully canceled tick 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd3e0062c90 msgr2=0x7fd3e0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 --2- 172.21.15.59:0/3154417640 >> v2:172.21.15.59:6800/3721891797 conn(0x7fd3e0062c90 0x7fd3e0065150 secure :-1 s=READY pgs=3270 cs=0 l=1 rev1=1 rx=0x7fd3e8004550 tx=0x7fd3e8009040).stop 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80fcbb0 msgr2=0x7fd3f810d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 --2- 172.21.15.59:0/3154417640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80fcbb0 0x7fd3f810d370 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 rx=0x7fd3ec004920 tx=0x7fd3ec009cb0).stop 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3f77fe700 1 -- 172.21.15.59:0/3154417640 reap_dead start 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 shutdown_connections 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 >> 172.21.15.59:0/3154417640 conn(0x7fd3f80f2820 msgr2=0x7fd3f80f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 shutdown_connections 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 -- 172.21.15.59:0/3154417640 wait complete. 2024-02-18T00:48:04.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:04.645+0000 7fd3fd345700 1 librados: shutdown 2024-02-18T00:48:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:03.687433+0000 osd.3 (osd.3) 489 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:48:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:03.689290+0000 osd.3 (osd.3) 490 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:48:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:04.066415+0000 mgr.y (mgr.24454) 757 : cluster [DBG] pgmap v1337: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:04.171777+0000 osd.7 (osd.7) 303 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:48:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:04.172795+0000 osd.7 (osd.7) 304 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:48:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:04.782407+0000 osd.2 (osd.2) 375 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:48:05.482 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:04.783522+0000 osd.2 (osd.2) 376 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:48:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:03.687433+0000 osd.3 (osd.3) 489 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:48:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:03.689290+0000 osd.3 (osd.3) 490 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:48:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:04.066415+0000 mgr.y (mgr.24454) 757 : cluster [DBG] pgmap v1337: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:04.171777+0000 osd.7 (osd.7) 303 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:48:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:04.172795+0000 osd.7 (osd.7) 304 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:48:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:04.782407+0000 osd.2 (osd.2) 375 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:48:05.483 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:04.783522+0000 osd.2 (osd.2) 376 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:03.687433+0000 osd.3 (osd.3) 489 : cluster [DBG] 4.0 scrub starts 2024-02-18T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:03.689290+0000 osd.3 (osd.3) 490 : cluster [DBG] 4.0 scrub ok 2024-02-18T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:04.066415+0000 mgr.y (mgr.24454) 757 : cluster [DBG] pgmap v1337: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:04.171777+0000 osd.7 (osd.7) 303 : cluster [DBG] 6.12 scrub starts 2024-02-18T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:04.172795+0000 osd.7 (osd.7) 304 : cluster [DBG] 6.12 scrub ok 2024-02-18T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:04.782407+0000 osd.2 (osd.2) 375 : cluster [DBG] 5.6 scrub starts 2024-02-18T00:48:05.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:04.783522+0000 osd.2 (osd.2) 376 : cluster [DBG] 5.6 scrub ok 2024-02-18T00:48:05.506 INFO:teuthology.orchestra.run.smithi059.stdout:111669149946 2024-02-18T00:48:05.506 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-18T00:48:05.791 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:06.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:06.044484+0000 osd.0 (osd.0) 408 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:48:06.366 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:06.045272+0000 osd.0 (osd.0) 409 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:48:06.366 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:06.044484+0000 osd.0 (osd.0) 408 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:48:06.366 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:06 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:06.045272+0000 osd.0 (osd.0) 409 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:48:06.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:06.044484+0000 osd.0 (osd.0) 408 : cluster [DBG] 5.1e scrub starts 2024-02-18T00:48:06.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:06.045272+0000 osd.0 (osd.0) 409 : cluster [DBG] 5.1e scrub ok 2024-02-18T00:48:07.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:05.659087+0000 osd.3 (osd.3) 491 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:48:07.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:05.661096+0000 osd.3 (osd.3) 492 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:48:07.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:06.067140+0000 mgr.y (mgr.24454) 758 : cluster [DBG] pgmap v1338: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:07.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:06.404488+0000 osd.4 (osd.4) 487 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:48:07.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:06.405485+0000 osd.4 (osd.4) 488 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:48:07.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:06.765115+0000 osd.2 (osd.2) 377 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:48:07.183 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:06.766023+0000 osd.2 (osd.2) 378 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:48:07.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:05.659087+0000 osd.3 (osd.3) 491 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:48:07.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:05.661096+0000 osd.3 (osd.3) 492 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:48:07.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:06.067140+0000 mgr.y (mgr.24454) 758 : cluster [DBG] pgmap v1338: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:06.404488+0000 osd.4 (osd.4) 487 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:48:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:06.405485+0000 osd.4 (osd.4) 488 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:48:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:06.765115+0000 osd.2 (osd.2) 377 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:48:07.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:07 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:06.766023+0000 osd.2 (osd.2) 378 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:48:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:05.659087+0000 osd.3 (osd.3) 491 : cluster [DBG] 4.1d scrub starts 2024-02-18T00:48:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:05.661096+0000 osd.3 (osd.3) 492 : cluster [DBG] 4.1d scrub ok 2024-02-18T00:48:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:06.067140+0000 mgr.y (mgr.24454) 758 : cluster [DBG] pgmap v1338: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:06.404488+0000 osd.4 (osd.4) 487 : cluster [DBG] 5.18 scrub starts 2024-02-18T00:48:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:06.405485+0000 osd.4 (osd.4) 488 : cluster [DBG] 5.18 scrub ok 2024-02-18T00:48:07.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:06.765115+0000 osd.2 (osd.2) 377 : cluster [DBG] 5.8 scrub starts 2024-02-18T00:48:07.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:07 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:06.766023+0000 osd.2 (osd.2) 378 : cluster [DBG] 5.8 scrub ok 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.013+0000 7f5c1162a700 1 -- 172.21.15.59:0/1531962026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f89a0 msgr2=0x7f5c0c0f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.013+0000 7f5c1162a700 1 --2- 172.21.15.59:0/1531962026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f89a0 0x7f5c0c0f8d80 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 rx=0x7f5c000047b0 tx=0x7f5c00009cb0).stop 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.013+0000 7f5c1162a700 1 -- 172.21.15.59:0/1531962026 shutdown_connections 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.013+0000 7f5c1162a700 1 --2- 172.21.15.59:0/1531962026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c0c0f7330 0x7f5c0c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.013+0000 7f5c1162a700 1 --2- 172.21.15.59:0/1531962026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f89a0 0x7f5c0c0f8d80 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.013+0000 7f5c1162a700 1 --2- 172.21.15.59:0/1531962026 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c0c0f9270 0x7f5c0c0f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.013+0000 7f5c1162a700 1 -- 172.21.15.59:0/1531962026 >> 172.21.15.59:0/1531962026 conn(0x7f5c0c0f26f0 msgr2=0x7f5c0c0f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 -- 172.21.15.59:0/1531962026 shutdown_connections 2024-02-18T00:48:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 -- 172.21.15.59:0/1531962026 wait complete. 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 librados: starting msgr at 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 librados: starting objecter 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 Processor -- start 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 -- start start 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 librados: setting wanted keys 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.014+0000 7f5c1162a700 1 librados: calling monclient init 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c0c0f7330 0x7f5c0c109220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c0c0f89a0 0x7f5c0c109710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f9270 0x7f5c0c109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c114400 con 0x7f5c0c0f9270 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c114570 con 0x7f5c0c0f7330 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c114730 con 0x7f5c0c0f89a0 2024-02-18T00:48:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c0c0f89a0 0x7f5c0c109710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f9270 0x7f5c0c109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0a7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c0c0f89a0 0x7f5c0c109710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:51240/0 (socket says 172.21.15.59:51240) 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0a7fc700 1 -- 172.21.15.59:0/903792549 learned_addr learned my addr 172.21.15.59:0/903792549 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0affd700 1 --2- 172.21.15.59:0/903792549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c0c0f7330 0x7f5c0c109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0b7fe700 1 -- 172.21.15.59:0/903792549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c0c0f89a0 msgr2=0x7f5c0c109710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0b7fe700 1 --2- 172.21.15.59:0/903792549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c0c0f89a0 0x7f5c0c109710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0b7fe700 1 -- 172.21.15.59:0/903792549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c0c0f7330 msgr2=0x7f5c0c109220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0b7fe700 1 --2- 172.21.15.59:0/903792549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c0c0f7330 0x7f5c0c109220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0b7fe700 1 -- 172.21.15.59:0/903792549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc005040 con 0x7f5c0c0f9270 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0affd700 1 --2- 172.21.15.59:0/903792549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c0c0f7330 0x7f5c0c109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0a7fc700 1 --2- 172.21.15.59:0/903792549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c0c0f89a0 0x7f5c0c109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c0b7fe700 1 --2- 172.21.15.59:0/903792549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f9270 0x7f5c0c109cf0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 rx=0x7f5c0400d8d0 tx=0x7f5c04004db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:08.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5bf3fff700 10 client.?.objecter ms_handle_connect 0x7f5c0c0f9270 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5bf3fff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04004020 con 0x7f5c0c0f9270 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c00005040 con 0x7f5c0c0f9270 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c0400f040 con 0x7f5c0c0f9270 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040036c0 con 0x7f5c0c0f9270 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 10 client.32250.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c114df0 con 0x7f5c0c0f9270 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.015+0000 7f5c1162a700 1 librados: init done 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.016+0000 7f5c1162a700 10 client.32250.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.016+0000 7f5c1162a700 15 client.32250.objecter _session_command_op_assign -1 1 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.016+0000 7f5c1162a700 20 client.32250.objecter _get_session osd=-1 returning homeless 2024-02-18T00:48:08.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.016+0000 7f5c1162a700 10 client.32250.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.016+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c0c05ee80 con 0x7f5c0c0f9270 2024-02-18T00:48:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f5c04003af0 con 0x7f5c0c0f9270 2024-02-18T00:48:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bf4062c90 0x7f5bf4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5c0affd700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bf4062c90 0x7f5bf4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(847..847 src has 418..847) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f5c0407cb20 con 0x7f5c0c0f9270 2024-02-18T00:48:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 10 client.32250.objecter ms_dispatch 0x7f5c0c001550 osd_map(847..847 src has 418..847) v4 2024-02-18T00:48:08.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 3 client.32250.objecter handle_osd_map got epochs [847,847] > 0 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 3 client.32250.objecter handle_osd_map decoding full epoch 847 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 10 client.32250.objecter checking command 1 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bf406e6e0 0x7f5bf4070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 20 client.32250.objecter _get_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 20 client.32250.objecter put_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 15 client.32250.objecter _session_command_op_remove -1 1 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 20 client.32250.objecter _get_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 20 client.32250.objecter get_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 15 client.32250.objecter _session_command_op_assign 4 1 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 20 client.32250.objecter put_session s=0x7f5bf406e030 osd=4 4 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 10 client.32250.objecter _send_command 1 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 --> v2:172.21.15.137:6800/2044387056 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bf40711c0 con 0x7f5bf406e6e0 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 20 client.32250.objecter dump_active .. 0 homeless 2024-02-18T00:48:08.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.017+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=847) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c04081220 con 0x7f5c0c0f9270 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.018+0000 7f5c0affd700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bf4062c90 0x7f5bf4065150 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 rx=0x7f5c00005010 tx=0x7f5c00007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.018+0000 7f5bf3fff700 10 client.32250.objecter ms_handle_connect 0x7f5bf4062c90 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.018+0000 7f5c0a7fc700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bf406e6e0 0x7f5bf4070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.018+0000 7f5bf3fff700 10 client.32250.objecter ms_handle_connect 0x7f5bf406e6e0 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.018+0000 7f5c0a7fc700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bf406e6e0 0x7f5bf4070b80 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.019+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== osd.4 v2:172.21.15.137:6800/2044387056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f5bf40711c0 con 0x7f5bf406e6e0 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.019+0000 7f5bf3fff700 10 client.32250.objecter ms_dispatch 0x7f5c0c001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.019+0000 7f5bf3fff700 10 client.32250.objecter _finish_command 1 = system:0 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.019+0000 7f5bf3fff700 20 client.32250.objecter put_session s=0x7f5bf406e030 osd=4 4 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.019+0000 7f5bf3fff700 15 client.32250.objecter _session_command_op_remove 4 1 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 10 client.32250.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 15 client.32250.objecter _session_command_op_assign -1 2 2024-02-18T00:48:08.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 20 client.32250.objecter _get_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 20 client.32250.objecter put_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 20 client.32250.objecter _get_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 15 client.32250.objecter _session_command_op_remove -1 2 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 20 client.32250.objecter get_session s=0x7f5bf406e030 osd=4 3 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 15 client.32250.objecter _session_command_op_assign 4 2 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 20 client.32250.objecter put_session s=0x7f5bf406e030 osd=4 4 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 10 client.32250.objecter _send_command 2 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.033+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 --> v2:172.21.15.137:6800/2044387056 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5c0c1060b0 con 0x7f5bf406e6e0 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5bf3fff700 1 -- 172.21.15.59:0/903792549 <== osd.4 v2:172.21.15.137:6800/2044387056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5c0c1060b0 con 0x7f5bf406e6e0 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5bf3fff700 10 client.32250.objecter ms_dispatch 0x7f5c0c001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5bf3fff700 10 client.32250.objecter _finish_command 2 = system:0 2024-02-18T00:48:08.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5bf3fff700 20 client.32250.objecter put_session s=0x7f5bf406e030 osd=4 4 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5bf3fff700 15 client.32250.objecter _session_command_op_remove 4 2 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 10 librados: watch_flush enter 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 10 librados: watch_flush exit 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 10 client.32250.objecter close_session for osd.4 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bf406e6e0 msgr2=0x7f5bf4070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.137:6800/2044387056 conn(0x7f5bf406e6e0 0x7f5bf4070b80 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 20 client.32250.objecter put_session s=0x7f5bf406e030 osd=4 1 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 20 client.32250.objecter shutdown clearing up homeless session... 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 10 client.32250.objecter successfully canceled tick 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bf4062c90 msgr2=0x7f5bf4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 --2- 172.21.15.59:0/903792549 >> v2:172.21.15.59:6800/3721891797 conn(0x7f5bf4062c90 0x7f5bf4065150 secure :-1 s=READY pgs=3271 cs=0 l=1 rev1=1 rx=0x7f5c00005010 tx=0x7f5c00007f90).stop 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f9270 msgr2=0x7f5c0c109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 --2- 172.21.15.59:0/903792549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c0c0f9270 0x7f5c0c109cf0 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 rx=0x7f5c0400d8d0 tx=0x7f5c04004db0).stop 2024-02-18T00:48:08.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c0b7fe700 1 -- 172.21.15.59:0/903792549 reap_dead start 2024-02-18T00:48:08.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 shutdown_connections 2024-02-18T00:48:08.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 >> 172.21.15.59:0/903792549 conn(0x7f5c0c0f26f0 msgr2=0x7f5c0c0f4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:08.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 shutdown_connections 2024-02-18T00:48:08.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 -- 172.21.15.59:0/903792549 wait complete. 2024-02-18T00:48:08.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:08.034+0000 7f5c1162a700 1 librados: shutdown 2024-02-18T00:48:08.188 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:07.438688+0000 osd.4 (osd.4) 489 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:48:08.415 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:07.440570+0000 osd.4 (osd.4) 490 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:48:08.415 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:07.438688+0000 osd.4 (osd.4) 489 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:48:08.415 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:07.440570+0000 osd.4 (osd.4) 490 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:48:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:07.438688+0000 osd.4 (osd.4) 489 : cluster [DBG] 4.9 scrub starts 2024-02-18T00:48:08.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:08 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:07.440570+0000 osd.4 (osd.4) 490 : cluster [DBG] 4.9 scrub ok 2024-02-18T00:48:08.891 INFO:teuthology.orchestra.run.smithi059.stdout:133143986422 2024-02-18T00:48:08.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-18T00:48:09.180 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:09.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:08.067801+0000 mgr.y (mgr.24454) 759 : cluster [DBG] pgmap v1339: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:09.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.030841+0000 osd.0 (osd.0) 410 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:48:09.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.032331+0000 osd.0 (osd.0) 411 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:48:09.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:08.067801+0000 mgr.y (mgr.24454) 759 : cluster [DBG] pgmap v1339: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:09.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.030841+0000 osd.0 (osd.0) 410 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:48:09.495 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:09 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.032331+0000 osd.0 (osd.0) 411 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:48:09.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:08.067801+0000 mgr.y (mgr.24454) 759 : cluster [DBG] pgmap v1339: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:09.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.030841+0000 osd.0 (osd.0) 410 : cluster [DBG] 25.11 scrub starts 2024-02-18T00:48:09.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:09 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.032331+0000 osd.0 (osd.0) 411 : cluster [DBG] 25.11 scrub ok 2024-02-18T00:48:10.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:08.438991+0000 osd.6 (osd.6) 391 : cluster [DBG] 25.8 deep-scrub starts 2024-02-18T00:48:10.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:08.440181+0000 osd.6 (osd.6) 392 : cluster [DBG] 25.8 deep-scrub ok 2024-02-18T00:48:10.194 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:08.734378+0000 osd.1 (osd.1) 447 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:48:10.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:08.737290+0000 osd.1 (osd.1) 448 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:48:10.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.251337+0000 osd.7 (osd.7) 305 : cluster [DBG] 2.1c deep-scrub starts 2024-02-18T00:48:10.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.252331+0000 osd.7 (osd.7) 306 : cluster [DBG] 2.1c deep-scrub ok 2024-02-18T00:48:10.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.663680+0000 osd.2 (osd.2) 379 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:48:10.195 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.665580+0000 osd.2 (osd.2) 380 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:48:10.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:08.438991+0000 osd.6 (osd.6) 391 : cluster [DBG] 25.8 deep-scrub starts 2024-02-18T00:48:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:08.440181+0000 osd.6 (osd.6) 392 : cluster [DBG] 25.8 deep-scrub ok 2024-02-18T00:48:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:08.734378+0000 osd.1 (osd.1) 447 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:48:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:08.737290+0000 osd.1 (osd.1) 448 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:48:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.251337+0000 osd.7 (osd.7) 305 : cluster [DBG] 2.1c deep-scrub starts 2024-02-18T00:48:10.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.252331+0000 osd.7 (osd.7) 306 : cluster [DBG] 2.1c deep-scrub ok 2024-02-18T00:48:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.663680+0000 osd.2 (osd.2) 379 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:48:10.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.665580+0000 osd.2 (osd.2) 380 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:48:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:08.438991+0000 osd.6 (osd.6) 391 : cluster [DBG] 25.8 deep-scrub starts 2024-02-18T00:48:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:08.440181+0000 osd.6 (osd.6) 392 : cluster [DBG] 25.8 deep-scrub ok 2024-02-18T00:48:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:08.734378+0000 osd.1 (osd.1) 447 : cluster [DBG] 5.19 scrub starts 2024-02-18T00:48:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:08.737290+0000 osd.1 (osd.1) 448 : cluster [DBG] 5.19 scrub ok 2024-02-18T00:48:10.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.251337+0000 osd.7 (osd.7) 305 : cluster [DBG] 2.1c deep-scrub starts 2024-02-18T00:48:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.252331+0000 osd.7 (osd.7) 306 : cluster [DBG] 2.1c deep-scrub ok 2024-02-18T00:48:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.663680+0000 osd.2 (osd.2) 379 : cluster [DBG] 25.9 scrub starts 2024-02-18T00:48:10.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:10 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.665580+0000 osd.2 (osd.2) 380 : cluster [DBG] 25.9 scrub ok 2024-02-18T00:48:11.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 -- 172.21.15.59:0/982919166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fcbb0 msgr2=0x7f9e400fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 --2- 172.21.15.59:0/982919166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fcbb0 0x7f9e400fcf90 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 rx=0x7f9e340047b0 tx=0x7f9e34009cb0).stop 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 -- 172.21.15.59:0/982919166 shutdown_connections 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 --2- 172.21.15.59:0/982919166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400f7400 0x7f9e40103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 --2- 172.21.15.59:0/982919166 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fcbb0 0x7f9e400fcf90 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 --2- 172.21.15.59:0/982919166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e400fd510 0x7f9e400f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 -- 172.21.15.59:0/982919166 >> 172.21.15.59:0/982919166 conn(0x7f9e400f2820 msgr2=0x7f9e400f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 -- 172.21.15.59:0/982919166 shutdown_connections 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 -- 172.21.15.59:0/982919166 wait complete. 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 librados: starting msgr at 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 librados: starting objecter 2024-02-18T00:48:11.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.394+0000 7f9e47041700 1 Processor -- start 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 -- start start 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 librados: setting wanted keys 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 librados: calling monclient init 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e400f7400 0x7f9e400f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400fcbb0 0x7f9e400f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fd510 0x7f9e400f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e401165f0 con 0x7f9e400fcbb0 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40116760 con 0x7f9e400f7400 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e47041700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e40116920 con 0x7f9e400fd510 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fd510 0x7f9e400f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e3ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400fcbb0 0x7f9e400f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fd510 0x7f9e400f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37434/0 (socket says 172.21.15.59:37434) 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 -- 172.21.15.59:0/3400478986 learned_addr learned my addr 172.21.15.59:0/3400478986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:11.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e3ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400fcbb0 0x7f9e400f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50676/0 (socket says 172.21.15.59:50676) 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e44ddd700 1 --2- 172.21.15.59:0/3400478986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e400f7400 0x7f9e400f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 -- 172.21.15.59:0/3400478986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e400f7400 msgr2=0x7f9e400f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 --2- 172.21.15.59:0/3400478986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e400f7400 0x7f9e400f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 -- 172.21.15.59:0/3400478986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400fcbb0 msgr2=0x7f9e400f8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 --2- 172.21.15.59:0/3400478986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400fcbb0 0x7f9e400f8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 -- 172.21.15.59:0/3400478986 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e380097c0 con 0x7f9e400fd510 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e3ffff700 1 --2- 172.21.15.59:0/3400478986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e400fcbb0 0x7f9e400f8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e455de700 1 --2- 172.21.15.59:0/3400478986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fd510 0x7f9e400f9390 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 rx=0x7f9e38001df0 tx=0x7f9e38004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e3dffb700 10 client.?.objecter ms_handle_connect 0x7f9e400fd510 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.395+0000 7f9e3dffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.396+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38009a10 con 0x7f9e400fd510 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.396+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e380090d0 con 0x7f9e400fd510 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.396+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38007d30 con 0x7f9e400fd510 2024-02-18T00:48:11.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.396+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e34005040 con 0x7f9e400fd510 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.396+0000 7f9e47041700 10 client.42059.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.396+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e40116f10 con 0x7f9e400fd510 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.396+0000 7f9e47041700 1 librados: init done 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e47041700 10 client.42059.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e47041700 15 client.42059.objecter _session_command_op_assign -1 1 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e47041700 20 client.42059.objecter _get_session osd=-1 returning homeless 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e47041700 10 client.42059.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e4005fa60 con 0x7f9e400fd510 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9e3800fb30 con 0x7f9e400fd510 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e3dffb700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9e30062c90 0x7f9e30065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(847..847 src has 418..847) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f9e3807d810 con 0x7f9e400fd510 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e3dffb700 10 client.42059.objecter ms_dispatch 0x7f9e40001550 osd_map(847..847 src has 418..847) v4 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e3dffb700 3 client.42059.objecter handle_osd_map got epochs [847,847] > 0 2024-02-18T00:48:11.415 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e3dffb700 3 client.42059.objecter handle_osd_map decoding full epoch 847 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.397+0000 7f9e44ddd700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9e30062c90 0x7f9e30065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 10 client.42059.objecter checking command 1 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.137:6804/2170432263 conn(0x7f9e3006e6e0 0x7f9e30070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 20 client.42059.objecter _get_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 20 client.42059.objecter put_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 15 client.42059.objecter _session_command_op_remove -1 1 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 20 client.42059.objecter _get_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 20 client.42059.objecter get_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 15 client.42059.objecter _session_command_op_assign 5 1 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 20 client.42059.objecter put_session s=0x7f9e3006e030 osd=5 4 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 10 client.42059.objecter _send_command 1 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 --> v2:172.21.15.137:6804/2170432263 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e300711c0 con 0x7f9e3006e6e0 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 20 client.42059.objecter dump_active .. 0 homeless 2024-02-18T00:48:11.416 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=847) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e38082220 con 0x7f9e400fd510 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 10 client.42059.objecter ms_handle_connect 0x7f9e30062c90 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e44ddd700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9e30062c90 0x7f9e30065150 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 rx=0x7f9e34005010 tx=0x7f9e34009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3ffff700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.137:6804/2170432263 conn(0x7f9e3006e6e0 0x7f9e30070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3dffb700 10 client.42059.objecter ms_handle_connect 0x7f9e3006e6e0 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.398+0000 7f9e3ffff700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.137:6804/2170432263 conn(0x7f9e3006e6e0 0x7f9e30070b80 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.399+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== osd.5 v2:172.21.15.137:6804/2170432263 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f9e300711c0 con 0x7f9e3006e6e0 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.399+0000 7f9e3dffb700 10 client.42059.objecter ms_dispatch 0x7f9e40001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.399+0000 7f9e3dffb700 10 client.42059.objecter _finish_command 1 = system:0 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.399+0000 7f9e3dffb700 20 client.42059.objecter put_session s=0x7f9e3006e030 osd=5 4 2024-02-18T00:48:11.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.399+0000 7f9e3dffb700 15 client.42059.objecter _session_command_op_remove 5 1 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 10 client.42059.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 15 client.42059.objecter _session_command_op_assign -1 2 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 20 client.42059.objecter _get_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 20 client.42059.objecter put_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 20 client.42059.objecter _get_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 15 client.42059.objecter _session_command_op_remove -1 2 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 20 client.42059.objecter get_session s=0x7f9e3006e030 osd=5 3 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 15 client.42059.objecter _session_command_op_assign 5 2 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 20 client.42059.objecter put_session s=0x7f9e3006e030 osd=5 4 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 10 client.42059.objecter _send_command 2 2024-02-18T00:48:11.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.413+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 --> v2:172.21.15.137:6804/2170432263 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e4005ee80 con 0x7f9e3006e6e0 2024-02-18T00:48:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e3dffb700 1 -- 172.21.15.59:0/3400478986 <== osd.5 v2:172.21.15.137:6804/2170432263 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9e4005ee80 con 0x7f9e3006e6e0 2024-02-18T00:48:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e3dffb700 10 client.42059.objecter ms_dispatch 0x7f9e40001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:48:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e3dffb700 10 client.42059.objecter _finish_command 2 = system:0 2024-02-18T00:48:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e3dffb700 20 client.42059.objecter put_session s=0x7f9e3006e030 osd=5 4 2024-02-18T00:48:11.419 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e3dffb700 15 client.42059.objecter _session_command_op_remove 5 2 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 10 librados: watch_flush enter 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 10 librados: watch_flush exit 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 10 client.42059.objecter close_session for osd.5 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 >> v2:172.21.15.137:6804/2170432263 conn(0x7f9e3006e6e0 msgr2=0x7f9e30070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.137:6804/2170432263 conn(0x7f9e3006e6e0 0x7f9e30070b80 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 20 client.42059.objecter put_session s=0x7f9e3006e030 osd=5 1 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 20 client.42059.objecter shutdown clearing up homeless session... 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 10 client.42059.objecter successfully canceled tick 2024-02-18T00:48:11.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9e30062c90 msgr2=0x7f9e30065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 --2- 172.21.15.59:0/3400478986 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9e30062c90 0x7f9e30065150 secure :-1 s=READY pgs=3272 cs=0 l=1 rev1=1 rx=0x7f9e34005010 tx=0x7f9e34009cb0).stop 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fd510 msgr2=0x7f9e400f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 --2- 172.21.15.59:0/3400478986 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9e400fd510 0x7f9e400f9390 secure :-1 s=READY pgs=2686 cs=0 l=1 rev1=1 rx=0x7f9e38001df0 tx=0x7f9e38004d10).stop 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e455de700 1 -- 172.21.15.59:0/3400478986 reap_dead start 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 shutdown_connections 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.414+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 >> 172.21.15.59:0/3400478986 conn(0x7f9e400f2820 msgr2=0x7f9e40060df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.415+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 shutdown_connections 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.415+0000 7f9e47041700 1 -- 172.21.15.59:0/3400478986 wait complete. 2024-02-18T00:48:11.421 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:11.415+0000 7f9e47041700 1 librados: shutdown 2024-02-18T00:48:11.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.829765+0000 osd.5 (osd.5) 420 : cluster [DBG] 25.c scrub starts 2024-02-18T00:48:11.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:09.832628+0000 osd.5 (osd.5) 421 : cluster [DBG] 25.c scrub ok 2024-02-18T00:48:11.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:10.068326+0000 mgr.y (mgr.24454) 760 : cluster [DBG] pgmap v1340: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T00:48:11.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:10.206114+0000 osd.7 (osd.7) 307 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:48:11.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:10.207157+0000 osd.7 (osd.7) 308 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:48:11.442 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:10.630950+0000 osd.2 (osd.2) 381 : cluster [DBG] 2.c scrub starts 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:10.632279+0000 osd.2 (osd.2) 382 : cluster [DBG] 2.c scrub ok 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.829765+0000 osd.5 (osd.5) 420 : cluster [DBG] 25.c scrub starts 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:09.832628+0000 osd.5 (osd.5) 421 : cluster [DBG] 25.c scrub ok 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:10.068326+0000 mgr.y (mgr.24454) 760 : cluster [DBG] pgmap v1340: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:10.206114+0000 osd.7 (osd.7) 307 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:10.207157+0000 osd.7 (osd.7) 308 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:10.630950+0000 osd.2 (osd.2) 381 : cluster [DBG] 2.c scrub starts 2024-02-18T00:48:11.443 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:11 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:10.632279+0000 osd.2 (osd.2) 382 : cluster [DBG] 2.c scrub ok 2024-02-18T00:48:11.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:48:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.829765+0000 osd.5 (osd.5) 420 : cluster [DBG] 25.c scrub starts 2024-02-18T00:48:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:09.832628+0000 osd.5 (osd.5) 421 : cluster [DBG] 25.c scrub ok 2024-02-18T00:48:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:10.068326+0000 mgr.y (mgr.24454) 760 : cluster [DBG] pgmap v1340: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T00:48:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:10.206114+0000 osd.7 (osd.7) 307 : cluster [DBG] 2.27 scrub starts 2024-02-18T00:48:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:10.207157+0000 osd.7 (osd.7) 308 : cluster [DBG] 2.27 scrub ok 2024-02-18T00:48:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:10.630950+0000 osd.2 (osd.2) 381 : cluster [DBG] 2.c scrub starts 2024-02-18T00:48:11.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:10.632279+0000 osd.2 (osd.2) 382 : cluster [DBG] 2.c scrub ok 2024-02-18T00:48:12.003 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:48:11 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:48:12.215 INFO:teuthology.orchestra.run.smithi059.stdout:167503724786 2024-02-18T00:48:12.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-18T00:48:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:12.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:10.655532+0000 osd.3 (osd.3) 493 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:48:12.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:10.657078+0000 osd.3 (osd.3) 494 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:48:12.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.197634+0000 osd.7 (osd.7) 309 : cluster [DBG] 2.26 deep-scrub starts 2024-02-18T00:48:12.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.198668+0000 osd.7 (osd.7) 310 : cluster [DBG] 2.26 deep-scrub ok 2024-02-18T00:48:12.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.386274+0000 osd.4 (osd.4) 491 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:48:12.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.389787+0000 osd.4 (osd.4) 492 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:48:12.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:12] "GET /metrics HTTP/1.1" 200 187536 "" "Prometheus/2.33.4" 2024-02-18T00:48:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:10.655532+0000 osd.3 (osd.3) 493 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:48:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:10.657078+0000 osd.3 (osd.3) 494 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:48:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.197634+0000 osd.7 (osd.7) 309 : cluster [DBG] 2.26 deep-scrub starts 2024-02-18T00:48:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.198668+0000 osd.7 (osd.7) 310 : cluster [DBG] 2.26 deep-scrub ok 2024-02-18T00:48:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.386274+0000 osd.4 (osd.4) 491 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:48:12.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:12 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.389787+0000 osd.4 (osd.4) 492 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:48:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:10.655532+0000 osd.3 (osd.3) 493 : cluster [DBG] 25.1e scrub starts 2024-02-18T00:48:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:10.657078+0000 osd.3 (osd.3) 494 : cluster [DBG] 25.1e scrub ok 2024-02-18T00:48:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.197634+0000 osd.7 (osd.7) 309 : cluster [DBG] 2.26 deep-scrub starts 2024-02-18T00:48:12.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.198668+0000 osd.7 (osd.7) 310 : cluster [DBG] 2.26 deep-scrub ok 2024-02-18T00:48:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.386274+0000 osd.4 (osd.4) 491 : cluster [DBG] 4.13 scrub starts 2024-02-18T00:48:12.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:12 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.389787+0000 osd.4 (osd.4) 492 : cluster [DBG] 4.13 scrub ok 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.458421+0000 osd.6 (osd.6) 393 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.459757+0000 osd.6 (osd.6) 394 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:11.751946+0000 mgr.y (mgr.24454) 761 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.770291+0000 osd.5 (osd.5) 422 : cluster [DBG] 5.1b deep-scrub starts 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:11.771378+0000 osd.5 (osd.5) 423 : cluster [DBG] 5.1b deep-scrub ok 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.069389+0000 mgr.y (mgr.24454) 762 : cluster [DBG] pgmap v1341: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.085237+0000 osd.0 (osd.0) 412 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.086736+0000 osd.0 (osd.0) 413 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.215737+0000 osd.7 (osd.7) 311 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:48:13.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.216940+0000 osd.7 (osd.7) 312 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.267371+0000 osd.7 (osd.7) 313 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.267472+0000 osd.7 (osd.7) 314 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.687735+0000 osd.2 (osd.2) 383 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.688993+0000 osd.2 (osd.2) 384 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:13.019525+0000 mon.a (mon.0) 3855 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.458421+0000 osd.6 (osd.6) 393 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.459757+0000 osd.6 (osd.6) 394 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:11.751946+0000 mgr.y (mgr.24454) 761 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.770291+0000 osd.5 (osd.5) 422 : cluster [DBG] 5.1b deep-scrub starts 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:11.771378+0000 osd.5 (osd.5) 423 : cluster [DBG] 5.1b deep-scrub ok 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.069389+0000 mgr.y (mgr.24454) 762 : cluster [DBG] pgmap v1341: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:13.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.085237+0000 osd.0 (osd.0) 412 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.086736+0000 osd.0 (osd.0) 413 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.215737+0000 osd.7 (osd.7) 311 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.216940+0000 osd.7 (osd.7) 312 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.267371+0000 osd.7 (osd.7) 313 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.267472+0000 osd.7 (osd.7) 314 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.687735+0000 osd.2 (osd.2) 383 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.688993+0000 osd.2 (osd.2) 384 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:13.019525+0000 mon.a (mon.0) 3855 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:48:13.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.458421+0000 osd.6 (osd.6) 393 : cluster [DBG] 25.0 scrub starts 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.459757+0000 osd.6 (osd.6) 394 : cluster [DBG] 25.0 scrub ok 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:11.751946+0000 mgr.y (mgr.24454) 761 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.770291+0000 osd.5 (osd.5) 422 : cluster [DBG] 5.1b deep-scrub starts 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:11.771378+0000 osd.5 (osd.5) 423 : cluster [DBG] 5.1b deep-scrub ok 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.069389+0000 mgr.y (mgr.24454) 762 : cluster [DBG] pgmap v1341: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.085237+0000 osd.0 (osd.0) 412 : cluster [DBG] 4.1e scrub starts 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.086736+0000 osd.0 (osd.0) 413 : cluster [DBG] 4.1e scrub ok 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.215737+0000 osd.7 (osd.7) 311 : cluster [DBG] 5.9 scrub starts 2024-02-18T00:48:13.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.216940+0000 osd.7 (osd.7) 312 : cluster [DBG] 5.9 scrub ok 2024-02-18T00:48:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.267371+0000 osd.7 (osd.7) 313 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.267472+0000 osd.7 (osd.7) 314 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:13.506 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.687735+0000 osd.2 (osd.2) 383 : cluster [DBG] 4.2 deep-scrub starts 2024-02-18T00:48:13.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.688993+0000 osd.2 (osd.2) 384 : cluster [DBG] 4.2 deep-scrub ok 2024-02-18T00:48:13.507 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:13 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:13.019525+0000 mon.a (mon.0) 3855 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:48:14.436 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.668977+0000 osd.3 (osd.3) 495 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:48:14.436 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:12.670623+0000 osd.3 (osd.3) 496 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:48:14.436 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:13.207349+0000 mon.a (mon.0) 3856 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-02-18T00:48:14.436 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:13.226039+0000 osd.7 (osd.7) 315 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:13.226971+0000 osd.7 (osd.7) 316 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:13.558776+0000 mon.a (mon.0) 3857 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:13.559513+0000 mon.a (mon.0) 3858 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:13.563659+0000 mon.a (mon.0) 3859 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:13.876341+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.668977+0000 osd.3 (osd.3) 495 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:12.670623+0000 osd.3 (osd.3) 496 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:13.207349+0000 mon.a (mon.0) 3856 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:13.226039+0000 osd.7 (osd.7) 315 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:13.226971+0000 osd.7 (osd.7) 316 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:48:14.437 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:13.558776+0000 mon.a (mon.0) 3857 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:48:14.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:13.559513+0000 mon.a (mon.0) 3858 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:48:14.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:13.563659+0000 mon.a (mon.0) 3859 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:48:14.438 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:13.876341+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-02-18T00:48:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.668977+0000 osd.3 (osd.3) 495 : cluster [DBG] 25.5 scrub starts 2024-02-18T00:48:14.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:12.670623+0000 osd.3 (osd.3) 496 : cluster [DBG] 25.5 scrub ok 2024-02-18T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:13.207349+0000 mon.a (mon.0) 3856 : cluster [DBG] osdmap e848: 8 total, 8 up, 8 in 2024-02-18T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:13.226039+0000 osd.7 (osd.7) 315 : cluster [DBG] 2.20 scrub starts 2024-02-18T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:13.226971+0000 osd.7 (osd.7) 316 : cluster [DBG] 2.20 scrub ok 2024-02-18T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:13.558776+0000 mon.a (mon.0) 3857 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:13.559513+0000 mon.a (mon.0) 3858 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:13.563659+0000 mon.a (mon.0) 3859 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:48:14.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:14 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:13.876341+0000 mon.a (mon.0) 3860 : cluster [DBG] osdmap e849: 8 total, 8 up, 8 in 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 -- 172.21.15.59:0/1837951619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 msgr2=0x7fb2e40f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 --2- 172.21.15.59:0/1837951619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 0x7fb2e40f6c50 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 rx=0x7fb2d8004870 tx=0x7fb2d8009cb0).stop 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 -- 172.21.15.59:0/1837951619 shutdown_connections 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 --2- 172.21.15.59:0/1837951619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e40f7330 0x7fb2e4103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 --2- 172.21.15.59:0/1837951619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2e40fcac0 0x7fb2e40fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 --2- 172.21.15.59:0/1837951619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 0x7fb2e40f6c50 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 -- 172.21.15.59:0/1837951619 >> 172.21.15.59:0/1837951619 conn(0x7fb2e40f26f0 msgr2=0x7fb2e40f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 -- 172.21.15.59:0/1837951619 shutdown_connections 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 -- 172.21.15.59:0/1837951619 wait complete. 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 librados: starting msgr at 2024-02-18T00:48:14.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 librados: starting objecter 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.727+0000 7fb2e9490700 1 Processor -- start 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 -- start start 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 librados: setting wanted keys 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 librados: calling monclient init 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e40f7330 0x7fb2e40f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2e40fcac0 0x7fb2e40f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 0x7fb2e40f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4116630 con 0x7fb2e40fcac0 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41167a0 con 0x7fb2e40f7330 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e9490700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4116960 con 0x7fb2e40fd420 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 0x7fb2e40f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2e40fcac0 0x7fb2e40f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:14.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 0x7fb2e40f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37454/0 (socket says 172.21.15.59:37454) 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2e40fcac0 0x7fb2e40f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50680/0 (socket says 172.21.15.59:50680) 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 -- 172.21.15.59:0/1497493460 learned_addr learned my addr 172.21.15.59:0/1497493460 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e2ffd700 1 --2- 172.21.15.59:0/1497493460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e40f7330 0x7fb2e40f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 -- 172.21.15.59:0/1497493460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e40f7330 msgr2=0x7fb2e40f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 --2- 172.21.15.59:0/1497493460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2e40f7330 0x7fb2e40f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 -- 172.21.15.59:0/1497493460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2e40fcac0 msgr2=0x7fb2e40f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 --2- 172.21.15.59:0/1497493460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2e40fcac0 0x7fb2e40f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 -- 172.21.15.59:0/1497493460 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8005040 con 0x7fb2e40fd420 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e27fc700 1 --2- 172.21.15.59:0/1497493460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2e40fcac0 0x7fb2e40f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2e37fe700 1 --2- 172.21.15.59:0/1497493460 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 0x7fb2e40f92a0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7fb2dc00d970 tx=0x7fb2dc004e60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2cbfff700 10 client.?.objecter ms_handle_connect 0x7fb2e40fd420 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.728+0000 7fb2cbfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:14.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc004020 con 0x7fb2e40fd420 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2dc00f040 con 0x7fb2e40fd420 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e4116c40 con 0x7fb2e40fd420 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0036c0 con 0x7fb2e40fd420 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2e9490700 10 client.42065.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4117130 con 0x7fb2e40fd420 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2e9490700 1 librados: init done 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.729+0000 7fb2e9490700 10 client.42065.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2e9490700 15 client.42065.objecter _session_command_op_assign -1 1 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2e9490700 20 client.42065.objecter _get_session osd=-1 returning homeless 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2e9490700 10 client.42065.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2e40606d0 con 0x7fb2e40fd420 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb2dc003af0 con 0x7fb2e40fd420 2024-02-18T00:48:14.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2cbfff700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2cc062c90 0x7fb2cc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(849..849 src has 418..849) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fb2dc07ca00 con 0x7fb2e40fd420 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2cbfff700 10 client.42065.objecter ms_dispatch 0x7fb2e4001550 osd_map(849..849 src has 418..849) v4 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2cbfff700 3 client.42065.objecter handle_osd_map got epochs [849,849] > 0 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2cbfff700 3 client.42065.objecter handle_osd_map decoding full epoch 849 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.730+0000 7fb2e2ffd700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2cc062c90 0x7fb2cc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 10 client.42065.objecter checking command 1 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2cc06e6e0 0x7fb2cc070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 20 client.42065.objecter _get_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 20 client.42065.objecter put_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 15 client.42065.objecter _session_command_op_remove -1 1 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 20 client.42065.objecter _get_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 20 client.42065.objecter get_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 15 client.42065.objecter _session_command_op_assign 6 1 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 20 client.42065.objecter put_session s=0x7fb2cc06e030 osd=6 4 2024-02-18T00:48:14.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 10 client.42065.objecter _send_command 1 2024-02-18T00:48:14.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 --> v2:172.21.15.137:6808/3356009390 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2cc0711c0 con 0x7fb2cc06e6e0 2024-02-18T00:48:14.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 20 client.42065.objecter dump_active .. 0 homeless 2024-02-18T00:48:14.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=849) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2dc081220 con 0x7fb2e40fd420 2024-02-18T00:48:14.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 10 client.42065.objecter ms_handle_connect 0x7fb2cc062c90 2024-02-18T00:48:14.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2e2ffd700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2cc062c90 0x7fb2cc065150 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 rx=0x7fb2d40048b0 tx=0x7fb2d400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:14.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2e27fc700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2cc06e6e0 0x7fb2cc070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2e27fc700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2cc06e6e0 0x7fb2cc070b80 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.731+0000 7fb2cbfff700 10 client.42065.objecter ms_handle_connect 0x7fb2cc06e6e0 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.732+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== osd.6 v2:172.21.15.137:6808/3356009390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb2cc0711c0 con 0x7fb2cc06e6e0 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.732+0000 7fb2cbfff700 10 client.42065.objecter ms_dispatch 0x7fb2e4001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.732+0000 7fb2cbfff700 10 client.42065.objecter _finish_command 1 = system:0 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.732+0000 7fb2cbfff700 20 client.42065.objecter put_session s=0x7fb2cc06e030 osd=6 4 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.732+0000 7fb2cbfff700 15 client.42065.objecter _session_command_op_remove 6 1 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 10 client.42065.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 15 client.42065.objecter _session_command_op_assign -1 2 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 20 client.42065.objecter _get_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 20 client.42065.objecter put_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 20 client.42065.objecter _get_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 15 client.42065.objecter _session_command_op_remove -1 2 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 20 client.42065.objecter get_session s=0x7fb2cc06e030 osd=6 3 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 15 client.42065.objecter _session_command_op_assign 6 2 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 20 client.42065.objecter put_session s=0x7fb2cc06e030 osd=6 4 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 10 client.42065.objecter _send_command 2 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.746+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 --> v2:172.21.15.137:6808/3356009390 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2e405ee80 con 0x7fb2cc06e6e0 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2cbfff700 1 -- 172.21.15.59:0/1497493460 <== osd.6 v2:172.21.15.137:6808/3356009390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb2e405ee80 con 0x7fb2cc06e6e0 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2cbfff700 10 client.42065.objecter ms_dispatch 0x7fb2e4001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2cbfff700 10 client.42065.objecter _finish_command 2 = system:0 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2cbfff700 20 client.42065.objecter put_session s=0x7fb2cc06e030 osd=6 4 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2cbfff700 15 client.42065.objecter _session_command_op_remove 6 2 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 10 librados: watch_flush enter 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 10 librados: watch_flush exit 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 10 client.42065.objecter close_session for osd.6 2024-02-18T00:48:14.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2cc06e6e0 msgr2=0x7fb2cc070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.137:6808/3356009390 conn(0x7fb2cc06e6e0 0x7fb2cc070b80 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 20 client.42065.objecter put_session s=0x7fb2cc06e030 osd=6 1 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 20 client.42065.objecter shutdown clearing up homeless session... 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.747+0000 7fb2e9490700 10 client.42065.objecter successfully canceled tick 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2cc062c90 msgr2=0x7fb2cc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 --2- 172.21.15.59:0/1497493460 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb2cc062c90 0x7fb2cc065150 secure :-1 s=READY pgs=3273 cs=0 l=1 rev1=1 rx=0x7fb2d40048b0 tx=0x7fb2d400b040).stop 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 msgr2=0x7fb2e40f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 --2- 172.21.15.59:0/1497493460 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2e40fd420 0x7fb2e40f92a0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 rx=0x7fb2dc00d970 tx=0x7fb2dc004e60).stop 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e37fe700 1 -- 172.21.15.59:0/1497493460 reap_dead start 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 shutdown_connections 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 >> 172.21.15.59:0/1497493460 conn(0x7fb2e40f26f0 msgr2=0x7fb2e40f3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 shutdown_connections 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 -- 172.21.15.59:0/1497493460 wait complete. 2024-02-18T00:48:14.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:14.748+0000 7fb2e9490700 1 librados: shutdown 2024-02-18T00:48:15.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:14.069825+0000 mgr.y (mgr.24454) 763 : cluster [DBG] pgmap v1344: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:15.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:15 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:14.069825+0000 mgr.y (mgr.24454) 763 : cluster [DBG] pgmap v1344: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:15.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:15 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:14.069825+0000 mgr.y (mgr.24454) 763 : cluster [DBG] pgmap v1344: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:15.611 INFO:teuthology.orchestra.run.smithi059.stdout:201863463150 2024-02-18T00:48:15.612 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-18T00:48:15.898 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:16.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:14.487658+0000 osd.6 (osd.6) 395 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:48:16.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:14.488594+0000 osd.6 (osd.6) 396 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:48:16.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:16.063773+0000 osd.0 (osd.0) 414 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:48:16.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:16.064741+0000 osd.0 (osd.0) 415 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:48:16.464 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:14.487658+0000 osd.6 (osd.6) 395 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:48:16.464 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:14.488594+0000 osd.6 (osd.6) 396 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:48:16.464 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:16.063773+0000 osd.0 (osd.0) 414 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:48:16.464 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:16 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:16.064741+0000 osd.0 (osd.0) 415 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:48:16.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:14.487658+0000 osd.6 (osd.6) 395 : cluster [DBG] 2.14 scrub starts 2024-02-18T00:48:16.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:14.488594+0000 osd.6 (osd.6) 396 : cluster [DBG] 2.14 scrub ok 2024-02-18T00:48:16.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:16.063773+0000 osd.0 (osd.0) 414 : cluster [DBG] 3.1 scrub starts 2024-02-18T00:48:16.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:16 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:16.064741+0000 osd.0 (osd.0) 415 : cluster [DBG] 3.1 scrub ok 2024-02-18T00:48:17.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:15.735482+0000 osd.5 (osd.5) 424 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:48:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:15.739048+0000 osd.5 (osd.5) 425 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:48:17.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:16.070636+0000 mgr.y (mgr.24454) 764 : cluster [DBG] pgmap v1345: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:15.735482+0000 osd.5 (osd.5) 424 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:48:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:15.739048+0000 osd.5 (osd.5) 425 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:48:17.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:17 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:16.070636+0000 mgr.y (mgr.24454) 764 : cluster [DBG] pgmap v1345: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:17.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:15.735482+0000 osd.5 (osd.5) 424 : cluster [DBG] 25.1d scrub starts 2024-02-18T00:48:17.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:15.739048+0000 osd.5 (osd.5) 425 : cluster [DBG] 25.1d scrub ok 2024-02-18T00:48:17.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:17 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:16.070636+0000 mgr.y (mgr.24454) 764 : cluster [DBG] pgmap v1345: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:18.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 -- 172.21.15.59:0/214548603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740fe510 msgr2=0x7ffa740f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:18.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 --2- 172.21.15.59:0/214548603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740fe510 0x7ffa740f7d40 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 rx=0x7ffa5c0047b0 tx=0x7ffa5c009cb0).stop 2024-02-18T00:48:18.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 -- 172.21.15.59:0/214548603 shutdown_connections 2024-02-18T00:48:18.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 --2- 172.21.15.59:0/214548603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa740f8420 0x7ffa74104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:18.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 --2- 172.21.15.59:0/214548603 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa740fdbb0 0x7ffa740fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:18.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 --2- 172.21.15.59:0/214548603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740fe510 0x7ffa740f7d40 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 -- 172.21.15.59:0/214548603 >> 172.21.15.59:0/214548603 conn(0x7ffa740f3800 msgr2=0x7ffa740f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.068+0000 7ffa7ac60700 1 -- 172.21.15.59:0/214548603 shutdown_connections 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 -- 172.21.15.59:0/214548603 wait complete. 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 librados: starting msgr at 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 librados: starting objecter 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 Processor -- start 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 -- start start 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 librados: setting wanted keys 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 librados: calling monclient init 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740f8420 0x7ffa7410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:18.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa740fe510 0x7ffa741081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa741086b0 con 0x7ffa740f8420 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa74108820 con 0x7ffa740fdbb0 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa7ac60700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa741089e0 con 0x7ffa740fe510 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa789fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740f8420 0x7ffa7410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.069+0000 7ffa791fd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa740fe510 0x7ffa741081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa791fd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa740fe510 0x7ffa741081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:37474/0 (socket says 172.21.15.59:37474) 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa791fd700 1 -- 172.21.15.59:0/3212337842 learned_addr learned my addr 172.21.15.59:0/3212337842 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa73fff700 1 --2- 172.21.15.59:0/3212337842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa789fc700 1 -- 172.21.15.59:0/3212337842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa740fe510 msgr2=0x7ffa741081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa789fc700 1 --2- 172.21.15.59:0/3212337842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa740fe510 0x7ffa741081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa789fc700 1 -- 172.21.15.59:0/3212337842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa740fdbb0 msgr2=0x7ffa7410e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:18.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa789fc700 1 --2- 172.21.15.59:0/3212337842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa789fc700 1 -- 172.21.15.59:0/3212337842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c0097c0 con 0x7ffa740f8420 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa791fd700 1 --2- 172.21.15.59:0/3212337842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa740fe510 0x7ffa741081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa73fff700 1 --2- 172.21.15.59:0/3212337842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa740fdbb0 0x7ffa7410e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa71ffb700 10 client.?.objecter ms_handle_connect 0x7ffa740f8420 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa71ffb700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa789fc700 1 --2- 172.21.15.59:0/3212337842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740f8420 0x7ffa7410e020 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 rx=0x7ffa6c001e40 tx=0x7ffa6c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c009aa0 con 0x7ffa740f8420 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa6c0095a0 con 0x7ffa740f8420 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c005040 con 0x7ffa740f8420 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.070+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c007df0 con 0x7ffa740f8420 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 10 client.32268.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa74109020 con 0x7ffa740f8420 2024-02-18T00:48:18.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 1 librados: init done 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 10 client.32268.objecter _submit_command 1 [{"prefix": "get_command_descriptions"}] 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 15 client.32268.objecter _session_command_op_assign -1 1 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 20 client.32268.objecter _get_session osd=-1 returning homeless 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 10 client.32268.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.071+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa74060280 con 0x7ffa740f8420 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7ffa6c00fbe0 con 0x7ffa740f8420 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.59:6800/3721891797 conn(0x7ffa60062c90 0x7ffa60065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(849..849 src has 418..849) v4 ==== 7613+0+0 (secure 0 0 0) 0x7ffa6c07d960 con 0x7ffa740f8420 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 10 client.32268.objecter ms_dispatch 0x7ffa74001550 osd_map(849..849 src has 418..849) v4 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 3 client.32268.objecter handle_osd_map got epochs [849,849] > 0 2024-02-18T00:48:18.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 3 client.32268.objecter handle_osd_map decoding full epoch 849 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa73fff700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.59:6800/3721891797 conn(0x7ffa60062c90 0x7ffa60065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 10 client.32268.objecter checking command 1 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.137:6812/53025171 conn(0x7ffa6006e570 0x7ffa60070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 20 client.32268.objecter _get_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 20 client.32268.objecter put_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 15 client.32268.objecter _session_command_op_remove -1 1 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 20 client.32268.objecter _get_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 20 client.32268.objecter get_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 15 client.32268.objecter _session_command_op_assign 7 1 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 20 client.32268.objecter put_session s=0x7ffa6c009aa0 osd=7 4 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 10 client.32268.objecter _send_command 1 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 --> v2:172.21.15.137:6812/53025171 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa60071050 con 0x7ffa6006e570 2024-02-18T00:48:18.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 20 client.32268.objecter dump_active .. 0 homeless 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.072+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=849) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa6c082220 con 0x7ffa740f8420 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.073+0000 7ffa71ffb700 10 client.32268.objecter ms_handle_connect 0x7ffa60062c90 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.073+0000 7ffa73fff700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.59:6800/3721891797 conn(0x7ffa60062c90 0x7ffa60065150 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 rx=0x7ffa5c004e20 tx=0x7ffa5c007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.073+0000 7ffa791fd700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.137:6812/53025171 conn(0x7ffa6006e570 0x7ffa60070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.073+0000 7ffa71ffb700 10 client.32268.objecter ms_handle_connect 0x7ffa6006e570 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.073+0000 7ffa791fd700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.137:6812/53025171 conn(0x7ffa6006e570 0x7ffa60070a10 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.074+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== osd.7 v2:172.21.15.137:6812/53025171 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffa60071050 con 0x7ffa6006e570 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.074+0000 7ffa71ffb700 10 client.32268.objecter ms_dispatch 0x7ffa74001550 command_reply(tid 1: 0 ) v1 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.074+0000 7ffa71ffb700 10 client.32268.objecter _finish_command 1 = system:0 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.074+0000 7ffa71ffb700 20 client.32268.objecter put_session s=0x7ffa6c009aa0 osd=7 4 2024-02-18T00:48:18.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.074+0000 7ffa71ffb700 15 client.32268.objecter _session_command_op_remove 7 1 2024-02-18T00:48:18.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 10 client.32268.objecter _submit_command 2 [{"prefix": "flush_pg_stats"}] 2024-02-18T00:48:18.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 15 client.32268.objecter _session_command_op_assign -1 2 2024-02-18T00:48:18.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 20 client.32268.objecter _get_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 20 client.32268.objecter put_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 20 client.32268.objecter _get_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 15 client.32268.objecter _session_command_op_remove -1 2 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 20 client.32268.objecter get_session s=0x7ffa6c009aa0 osd=7 3 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 15 client.32268.objecter _session_command_op_assign 7 2 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 20 client.32268.objecter put_session s=0x7ffa6c009aa0 osd=7 4 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 10 client.32268.objecter _send_command 2 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.088+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 --> v2:172.21.15.137:6812/53025171 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa7405ee80 con 0x7ffa6006e570 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa71ffb700 1 -- 172.21.15.59:0/3212337842 <== osd.7 v2:172.21.15.137:6812/53025171 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffa7405ee80 con 0x7ffa6006e570 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa71ffb700 10 client.32268.objecter ms_dispatch 0x7ffa74001550 command_reply(tid 2: 0 ) v1 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa71ffb700 10 client.32268.objecter _finish_command 2 = system:0 2024-02-18T00:48:18.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa71ffb700 20 client.32268.objecter put_session s=0x7ffa6c009aa0 osd=7 4 2024-02-18T00:48:18.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa71ffb700 15 client.32268.objecter _session_command_op_remove 7 2 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 10 librados: watch_flush enter 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 10 librados: watch_flush exit 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 10 client.32268.objecter close_session for osd.7 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 >> v2:172.21.15.137:6812/53025171 conn(0x7ffa6006e570 msgr2=0x7ffa60070a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.137:6812/53025171 conn(0x7ffa6006e570 0x7ffa60070a10 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 20 client.32268.objecter put_session s=0x7ffa6c009aa0 osd=7 1 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 20 client.32268.objecter shutdown clearing up homeless session... 2024-02-18T00:48:18.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 10 client.32268.objecter successfully canceled tick 2024-02-18T00:48:18.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 >> v2:172.21.15.59:6800/3721891797 conn(0x7ffa60062c90 msgr2=0x7ffa60065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:18.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 --2- 172.21.15.59:0/3212337842 >> v2:172.21.15.59:6800/3721891797 conn(0x7ffa60062c90 0x7ffa60065150 secure :-1 s=READY pgs=3274 cs=0 l=1 rev1=1 rx=0x7ffa5c004e20 tx=0x7ffa5c007f90).stop 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740f8420 msgr2=0x7ffa7410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 --2- 172.21.15.59:0/3212337842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa740f8420 0x7ffa7410e020 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 rx=0x7ffa6c001e40 tx=0x7ffa6c004d10).stop 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa791fd700 1 -- 172.21.15.59:0/3212337842 reap_dead start 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 shutdown_connections 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.089+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 >> 172.21.15.59:0/3212337842 conn(0x7ffa740f3800 msgr2=0x7ffa740f5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.090+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 shutdown_connections 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.090+0000 7ffa7ac60700 1 -- 172.21.15.59:0/3212337842 wait complete. 2024-02-18T00:48:18.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:18.090+0000 7ffa7ac60700 1 librados: shutdown 2024-02-18T00:48:18.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:16.537916+0000 osd.6 (osd.6) 397 : cluster [DBG] 5.2 deep-scrub starts 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:16.538890+0000 osd.6 (osd.6) 398 : cluster [DBG] 5.2 deep-scrub ok 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:16.744087+0000 osd.5 (osd.5) 426 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:16.745014+0000 osd.5 (osd.5) 427 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:17.160902+0000 osd.7 (osd.7) 317 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:17.162452+0000 osd.7 (osd.7) 318 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:16.537916+0000 osd.6 (osd.6) 397 : cluster [DBG] 5.2 deep-scrub starts 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:16.538890+0000 osd.6 (osd.6) 398 : cluster [DBG] 5.2 deep-scrub ok 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:16.744087+0000 osd.5 (osd.5) 426 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:16.745014+0000 osd.5 (osd.5) 427 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:17.160902+0000 osd.7 (osd.7) 317 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:48:18.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:18 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:17.162452+0000 osd.7 (osd.7) 318 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:48:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:16.537916+0000 osd.6 (osd.6) 397 : cluster [DBG] 5.2 deep-scrub starts 2024-02-18T00:48:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:16.538890+0000 osd.6 (osd.6) 398 : cluster [DBG] 5.2 deep-scrub ok 2024-02-18T00:48:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:16.744087+0000 osd.5 (osd.5) 426 : cluster [DBG] 2.12 scrub starts 2024-02-18T00:48:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:16.745014+0000 osd.5 (osd.5) 427 : cluster [DBG] 2.12 scrub ok 2024-02-18T00:48:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:17.160902+0000 osd.7 (osd.7) 317 : cluster [DBG] 6.5 scrub starts 2024-02-18T00:48:18.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:18 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:17.162452+0000 osd.7 (osd.7) 318 : cluster [DBG] 6.5 scrub ok 2024-02-18T00:48:18.977 INFO:teuthology.orchestra.run.smithi059.stdout:227633266921 2024-02-18T00:48:18.977 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-18T00:48:19.229 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:17.495801+0000 osd.6 (osd.6) 399 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:17.497204+0000 osd.6 (osd.6) 400 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.071026+0000 mgr.y (mgr.24454) 765 : cluster [DBG] pgmap v1346: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.122004+0000 osd.0 (osd.0) 416 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.125314+0000 osd.0 (osd.0) 417 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.275771+0000 osd.4 (osd.4) 493 : cluster [DBG] 4.c scrub starts 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.279243+0000 osd.4 (osd.4) 494 : cluster [DBG] 4.c scrub ok 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.727698+0000 osd.2 (osd.2) 385 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.729246+0000 osd.2 (osd.2) 386 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:17.495801+0000 osd.6 (osd.6) 399 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:17.497204+0000 osd.6 (osd.6) 400 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:48:19.230 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.071026+0000 mgr.y (mgr.24454) 765 : cluster [DBG] pgmap v1346: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:19.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.122004+0000 osd.0 (osd.0) 416 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:48:19.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.125314+0000 osd.0 (osd.0) 417 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:48:19.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.275771+0000 osd.4 (osd.4) 493 : cluster [DBG] 4.c scrub starts 2024-02-18T00:48:19.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.279243+0000 osd.4 (osd.4) 494 : cluster [DBG] 4.c scrub ok 2024-02-18T00:48:19.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.727698+0000 osd.2 (osd.2) 385 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:48:19.231 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:19 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.729246+0000 osd.2 (osd.2) 386 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:48:19.263 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:19.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:17.495801+0000 osd.6 (osd.6) 399 : cluster [DBG] 4.20 scrub starts 2024-02-18T00:48:19.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:17.497204+0000 osd.6 (osd.6) 400 : cluster [DBG] 4.20 scrub ok 2024-02-18T00:48:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.071026+0000 mgr.y (mgr.24454) 765 : cluster [DBG] pgmap v1346: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.122004+0000 osd.0 (osd.0) 416 : cluster [DBG] 25.1a scrub starts 2024-02-18T00:48:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.125314+0000 osd.0 (osd.0) 417 : cluster [DBG] 25.1a scrub ok 2024-02-18T00:48:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.275771+0000 osd.4 (osd.4) 493 : cluster [DBG] 4.c scrub starts 2024-02-18T00:48:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.279243+0000 osd.4 (osd.4) 494 : cluster [DBG] 4.c scrub ok 2024-02-18T00:48:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.727698+0000 osd.2 (osd.2) 385 : cluster [DBG] 25.6 scrub starts 2024-02-18T00:48:19.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:19 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.729246+0000 osd.2 (osd.2) 386 : cluster [DBG] 25.6 scrub ok 2024-02-18T00:48:20.234 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.799457+0000 osd.1 (osd.1) 449 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:48:20.235 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.800822+0000 osd.1 (osd.1) 450 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:48:20.235 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.819492+0000 osd.5 (osd.5) 428 : cluster [DBG] 4.f scrub starts 2024-02-18T00:48:20.235 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:18.823232+0000 osd.5 (osd.5) 429 : cluster [DBG] 4.f scrub ok 2024-02-18T00:48:20.235 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.144081+0000 osd.7 (osd.7) 319 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:48:20.235 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.145384+0000 osd.7 (osd.7) 320 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:48:20.235 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.748307+0000 osd.2 (osd.2) 387 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:48:20.235 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.749498+0000 osd.2 (osd.2) 388 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:48:20.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.799457+0000 osd.1 (osd.1) 449 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:48:20.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.800822+0000 osd.1 (osd.1) 450 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:48:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.819492+0000 osd.5 (osd.5) 428 : cluster [DBG] 4.f scrub starts 2024-02-18T00:48:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:18.823232+0000 osd.5 (osd.5) 429 : cluster [DBG] 4.f scrub ok 2024-02-18T00:48:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.144081+0000 osd.7 (osd.7) 319 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:48:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.145384+0000 osd.7 (osd.7) 320 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:48:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.748307+0000 osd.2 (osd.2) 387 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:48:20.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.749498+0000 osd.2 (osd.2) 388 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:48:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.799457+0000 osd.1 (osd.1) 449 : cluster [DBG] 25.16 scrub starts 2024-02-18T00:48:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.800822+0000 osd.1 (osd.1) 450 : cluster [DBG] 25.16 scrub ok 2024-02-18T00:48:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.819492+0000 osd.5 (osd.5) 428 : cluster [DBG] 4.f scrub starts 2024-02-18T00:48:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:18.823232+0000 osd.5 (osd.5) 429 : cluster [DBG] 4.f scrub ok 2024-02-18T00:48:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.144081+0000 osd.7 (osd.7) 319 : cluster [DBG] 4.28 scrub starts 2024-02-18T00:48:20.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.145384+0000 osd.7 (osd.7) 320 : cluster [DBG] 4.28 scrub ok 2024-02-18T00:48:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.748307+0000 osd.2 (osd.2) 387 : cluster [DBG] 6.14 deep-scrub starts 2024-02-18T00:48:20.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.749498+0000 osd.2 (osd.2) 388 : cluster [DBG] 6.14 deep-scrub ok 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.660306+0000 osd.3 (osd.3) 497 : cluster [DBG] 4.29 scrub starts 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.661776+0000 osd.3 (osd.3) 498 : cluster [DBG] 4.29 scrub ok 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.860660+0000 osd.5 (osd.5) 430 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:19.861633+0000 osd.5 (osd.5) 431 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:20.071626+0000 mgr.y (mgr.24454) 766 : cluster [DBG] pgmap v1347: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.660306+0000 osd.3 (osd.3) 497 : cluster [DBG] 4.29 scrub starts 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.661776+0000 osd.3 (osd.3) 498 : cluster [DBG] 4.29 scrub ok 2024-02-18T00:48:21.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.860660+0000 osd.5 (osd.5) 430 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:48:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:19.861633+0000 osd.5 (osd.5) 431 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:48:21.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:20.071626+0000 mgr.y (mgr.24454) 766 : cluster [DBG] pgmap v1347: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:21.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:48:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:48:21.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.660306+0000 osd.3 (osd.3) 497 : cluster [DBG] 4.29 scrub starts 2024-02-18T00:48:21.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.661776+0000 osd.3 (osd.3) 498 : cluster [DBG] 4.29 scrub ok 2024-02-18T00:48:21.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.860660+0000 osd.5 (osd.5) 430 : cluster [DBG] 25.10 scrub starts 2024-02-18T00:48:21.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:19.861633+0000 osd.5 (osd.5) 431 : cluster [DBG] 25.10 scrub ok 2024-02-18T00:48:21.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:20.071626+0000 mgr.y (mgr.24454) 766 : cluster [DBG] pgmap v1347: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:21.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.510+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/3167741990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 msgr2=0x7fdb04103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:21.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.510+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/3167741990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 0x7fdb04103a20 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 rx=0x7fdafc009be0 tx=0x7fdafc004d10).stop 2024-02-18T00:48:21.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/3167741990 shutdown_connections 2024-02-18T00:48:21.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/3167741990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb040fcbb0 0x7fdb040fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/3167741990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 0x7fdb04103a20 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/3167741990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fd510 0x7fdb040f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/3167741990 >> 172.21.15.59:0/3167741990 conn(0x7fdb040f2820 msgr2=0x7fdb040f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/3167741990 shutdown_connections 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/3167741990 wait complete. 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 librados: starting msgr at 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 librados: starting objecter 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 Processor -- start 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 -- start start 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 librados: setting wanted keys 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.511+0000 7fdb0c7bb700 1 librados: calling monclient init 2024-02-18T00:48:21.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 0x7fdb0410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fcbb0 0x7fdb0410e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb040fd510 0x7fdb041081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041086b0 con 0x7fdb040f7400 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb04108820 con 0x7fdb040fd510 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041089e0 con 0x7fdb040fcbb0 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fcbb0 0x7fdb0410e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fcbb0 0x7fdb0410e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:43364/0 (socket says 172.21.15.59:43364) 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 -- 172.21.15.59:0/1921999099 learned_addr learned my addr 172.21.15.59:0/1921999099 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0a557700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 0x7fdb0410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0ad58700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb040fd510 0x7fdb041081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 -- 172.21.15.59:0/1921999099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb040fd510 msgr2=0x7fdb041081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:21.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb040fd510 0x7fdb041081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 -- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 msgr2=0x7fdb0410e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 0x7fdb0410e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 -- 172.21.15.59:0/1921999099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4005040 con 0x7fdb040fcbb0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0ad58700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb040fd510 0x7fdb041081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb09d56700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fcbb0 0x7fdb0410e510 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 rx=0x7fdaf40048f0 tx=0x7fdaf4009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdafb7fe700 10 client.?.objecter ms_handle_connect 0x7fdb040fcbb0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdafb7fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdafb7fe700 1 -- 172.21.15.59:0/1921999099 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4007b00 con 0x7fdb040fcbb0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdafb7fe700 1 -- 172.21.15.59:0/1921999099 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaf400f040 con 0x7fdb040fcbb0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc0097c0 con 0x7fdb040fcbb0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdafb7fe700 1 -- 172.21.15.59:0/1921999099 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4014a00 con 0x7fdb040fcbb0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.512+0000 7fdb0c7bb700 10 client.42077.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.513+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb04114e50 con 0x7fdb040fcbb0 2024-02-18T00:48:21.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.513+0000 7fdb0c7bb700 1 librados: init done 2024-02-18T00:48:21.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.513+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0405fa60 con 0x7fdb040fcbb0 2024-02-18T00:48:21.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.514+0000 7fdafb7fe700 1 -- 172.21.15.59:0/1921999099 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fdaf400e580 con 0x7fdb040fcbb0 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.514+0000 7fdafb7fe700 1 --2- 172.21.15.59:0/1921999099 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaf0062c90 0x7fdaf0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.514+0000 7fdafb7fe700 1 -- 172.21.15.59:0/1921999099 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(849..849 src has 418..849) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fdaf407ca70 con 0x7fdb040fcbb0 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.514+0000 7fdafb7fe700 10 client.42077.objecter ms_dispatch 0x7fdb04001550 osd_map(849..849 src has 418..849) v4 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.514+0000 7fdafb7fe700 3 client.42077.objecter handle_osd_map got epochs [849,849] > 0 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.514+0000 7fdafb7fe700 3 client.42077.objecter handle_osd_map decoding full epoch 849 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.514+0000 7fdb0a557700 1 --2- 172.21.15.59:0/1921999099 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaf0062c90 0x7fdaf0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.515+0000 7fdafb7fe700 20 client.42077.objecter dump_active .. 0 homeless 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.515+0000 7fdb0a557700 1 --2- 172.21.15.59:0/1921999099 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaf0062c90 0x7fdaf0065150 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 rx=0x7fdb000048b0 tx=0x7fdb0000b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:21.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.515+0000 7fdafb7fe700 10 client.42077.objecter ms_handle_connect 0x7fdaf0062c90 2024-02-18T00:48:21.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.519+0000 7fdafb7fe700 1 -- 172.21.15.59:0/1921999099 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdaf4049f60 con 0x7fdb040fcbb0 2024-02-18T00:48:21.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.691+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdb04106120 con 0x7fdb040fcbb0 2024-02-18T00:48:21.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.691+0000 7fdafb7fe700 1 -- 172.21.15.59:0/1921999099 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdaf40501e0 con 0x7fdb040fcbb0 2024-02-18T00:48:21.693 INFO:teuthology.orchestra.run.smithi059.stdout:34359738636 2024-02-18T00:48:21.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 10 librados: watch_flush enter 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 10 librados: watch_flush exit 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 20 client.42077.objecter shutdown clearing up homeless session... 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 10 client.42077.objecter successfully canceled tick 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaf0062c90 msgr2=0x7fdaf0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/1921999099 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaf0062c90 0x7fdaf0065150 secure :-1 s=READY pgs=3275 cs=0 l=1 rev1=1 rx=0x7fdb000048b0 tx=0x7fdb0000b040).stop 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fcbb0 msgr2=0x7fdb0410e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fcbb0 0x7fdb0410e510 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 rx=0x7fdaf40048f0 tx=0x7fdaf4009cb0).stop 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 shutdown_connections 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/1921999099 >> v2:172.21.15.59:6800/3721891797 conn(0x7fdaf0062c90 0x7fdaf0065150 unknown :-1 s=CLOSED pgs=3275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb040fd510 0x7fdb041081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb040f7400 0x7fdb0410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 --2- 172.21.15.59:0/1921999099 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb040fcbb0 0x7fdb0410e510 unknown :-1 s=CLOSED pgs=2689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:21.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 >> 172.21.15.59:0/1921999099 conn(0x7fdb040f2820 msgr2=0x7fdb040fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:21.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 shutdown_connections 2024-02-18T00:48:21.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 -- 172.21.15.59:0/1921999099 wait complete. 2024-02-18T00:48:21.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:21.693+0000 7fdb0c7bb700 1 librados: shutdown 2024-02-18T00:48:22.237 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:48:21 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:48:22.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:20.850542+0000 osd.5 (osd.5) 432 : cluster [DBG] 4.15 deep-scrub starts 2024-02-18T00:48:22.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:20.855829+0000 osd.5 (osd.5) 433 : cluster [DBG] 4.15 deep-scrub ok 2024-02-18T00:48:22.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:21.692617+0000 mon.c (mon.2) 593 : audit [DBG] from='client.? 172.21.15.59:0/1921999099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T00:48:22.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:20.850542+0000 osd.5 (osd.5) 432 : cluster [DBG] 4.15 deep-scrub starts 2024-02-18T00:48:22.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:20.855829+0000 osd.5 (osd.5) 433 : cluster [DBG] 4.15 deep-scrub ok 2024-02-18T00:48:22.460 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:21.692617+0000 mon.c (mon.2) 593 : audit [DBG] from='client.? 172.21.15.59:0/1921999099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T00:48:22.461 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:22] "GET /metrics HTTP/1.1" 200 187536 "" "Prometheus/2.33.4" 2024-02-18T00:48:22.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738631 got 34359738636 for osd.0 2024-02-18T00:48:22.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-18T00:48:22.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:20.850542+0000 osd.5 (osd.5) 432 : cluster [DBG] 4.15 deep-scrub starts 2024-02-18T00:48:22.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:20.855829+0000 osd.5 (osd.5) 433 : cluster [DBG] 4.15 deep-scrub ok 2024-02-18T00:48:22.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:22 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:21.692617+0000 mon.c (mon.2) 593 : audit [DBG] from='client.? 172.21.15.59:0/1921999099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T00:48:22.760 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:23.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:21.762929+0000 mgr.y (mgr.24454) 767 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:23.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:22.072374+0000 mgr.y (mgr.24454) 768 : cluster [DBG] pgmap v1348: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:21.762929+0000 mgr.y (mgr.24454) 767 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:23.252 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:22.072374+0000 mgr.y (mgr.24454) 768 : cluster [DBG] pgmap v1348: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:21.762929+0000 mgr.y (mgr.24454) 767 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:23.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:23 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:22.072374+0000 mgr.y (mgr.24454) 768 : cluster [DBG] pgmap v1348: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T00:48:24.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:22.554020+0000 osd.3 (osd.3) 499 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:48:24.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:22.556414+0000 osd.3 (osd.3) 500 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:48:24.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:22.880874+0000 osd.5 (osd.5) 434 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:48:24.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:22.881864+0000 osd.5 (osd.5) 435 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:48:24.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:23.383241+0000 osd.4 (osd.4) 495 : cluster [DBG] 4.e deep-scrub starts 2024-02-18T00:48:24.444 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:23.389246+0000 osd.4 (osd.4) 496 : cluster [DBG] 4.e deep-scrub ok 2024-02-18T00:48:24.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:22.554020+0000 osd.3 (osd.3) 499 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:48:24.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:22.556414+0000 osd.3 (osd.3) 500 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:48:24.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:22.880874+0000 osd.5 (osd.5) 434 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:48:24.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:22.881864+0000 osd.5 (osd.5) 435 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:48:24.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:23.383241+0000 osd.4 (osd.4) 495 : cluster [DBG] 4.e deep-scrub starts 2024-02-18T00:48:24.445 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:24 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:23.389246+0000 osd.4 (osd.4) 496 : cluster [DBG] 4.e deep-scrub ok 2024-02-18T00:48:24.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:22.554020+0000 osd.3 (osd.3) 499 : cluster [DBG] 25.1b scrub starts 2024-02-18T00:48:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:22.556414+0000 osd.3 (osd.3) 500 : cluster [DBG] 25.1b scrub ok 2024-02-18T00:48:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:22.880874+0000 osd.5 (osd.5) 434 : cluster [DBG] 2.3 scrub starts 2024-02-18T00:48:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:22.881864+0000 osd.5 (osd.5) 435 : cluster [DBG] 2.3 scrub ok 2024-02-18T00:48:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:23.383241+0000 osd.4 (osd.4) 495 : cluster [DBG] 4.e deep-scrub starts 2024-02-18T00:48:24.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:24 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:23.389246+0000 osd.4 (osd.4) 496 : cluster [DBG] 4.e deep-scrub ok 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 -- 172.21.15.59:0/3257525337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 msgr2=0x7f09840f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 --2- 172.21.15.59:0/3257525337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 0x7f09840f6d40 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 rx=0x7f09740047b0 tx=0x7f0974009cb0).stop 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 -- 172.21.15.59:0/3257525337 shutdown_connections 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 --2- 172.21.15.59:0/3257525337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09840f7400 0x7f0984103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 --2- 172.21.15.59:0/3257525337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840fcbb0 0x7f09840fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 --2- 172.21.15.59:0/3257525337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 0x7f09840f6d40 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 -- 172.21.15.59:0/3257525337 >> 172.21.15.59:0/3257525337 conn(0x7f09840f2820 msgr2=0x7f09840f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 -- 172.21.15.59:0/3257525337 shutdown_connections 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 -- 172.21.15.59:0/3257525337 wait complete. 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 librados: starting msgr at 2024-02-18T00:48:24.745 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.716+0000 7f098996c700 1 librados: starting objecter 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 Processor -- start 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 -- start start 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 librados: setting wanted keys 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 librados: calling monclient init 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840f7400 0x7f098410e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09840fcbb0 0x7f098410e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 0x7f0984108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0984108800 con 0x7f09840f7400 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0984108970 con 0x7f09840fcbb0 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f098996c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0984108b30 con 0x7f09840fd510 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f0982ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840f7400 0x7f098410e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f09837fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 0x7f0984108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f0982ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840f7400 0x7f098410e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37186/0 (socket says 172.21.15.59:37186) 2024-02-18T00:48:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f0982ffd700 1 -- 172.21.15.59:0/1669725705 learned_addr learned my addr 172.21.15.59:0/1669725705 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f09827fc700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09840fcbb0 0x7f098410e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f0982ffd700 1 -- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 msgr2=0x7f0984108310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f0982ffd700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 0x7f0984108310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f0982ffd700 1 -- 172.21.15.59:0/1669725705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09840fcbb0 msgr2=0x7f098410e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.717+0000 7f0982ffd700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09840fcbb0 0x7f098410e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f0982ffd700 1 -- 172.21.15.59:0/1669725705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0978005040 con 0x7f09840f7400 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f09837fe700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 0x7f0984108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f0982ffd700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840f7400 0x7f098410e170 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 rx=0x7f0978009400 tx=0x7f0978009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f096bfff700 10 client.?.objecter ms_handle_connect 0x7f09840f7400 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f096bfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f096bfff700 1 -- 172.21.15.59:0/1669725705 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978007a40 con 0x7f09840f7400 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0974005040 con 0x7f09840f7400 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f096bfff700 1 -- 172.21.15.59:0/1669725705 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f097800f040 con 0x7f09840f7400 2024-02-18T00:48:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f096bfff700 1 -- 172.21.15.59:0/1669725705 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978014a70 con 0x7f09840f7400 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f098996c700 10 client.32280.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0984109170 con 0x7f09840f7400 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.718+0000 7f098996c700 1 librados: init done 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.719+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098405fa60 con 0x7f09840f7400 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.719+0000 7f096bfff700 1 -- 172.21.15.59:0/1669725705 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f0978010530 con 0x7f09840f7400 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f096bfff700 1 --2- 172.21.15.59:0/1669725705 >> v2:172.21.15.59:6800/3721891797 conn(0x7f096c062c90 0x7f096c065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f096bfff700 1 -- 172.21.15.59:0/1669725705 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(849..849 src has 418..849) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f097807cbf0 con 0x7f09840f7400 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f096bfff700 10 client.32280.objecter ms_dispatch 0x7f0984001550 osd_map(849..849 src has 418..849) v4 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f09827fc700 1 --2- 172.21.15.59:0/1669725705 >> v2:172.21.15.59:6800/3721891797 conn(0x7f096c062c90 0x7f096c065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f096bfff700 3 client.32280.objecter handle_osd_map got epochs [849,849] > 0 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f096bfff700 3 client.32280.objecter handle_osd_map decoding full epoch 849 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f096bfff700 20 client.32280.objecter dump_active .. 0 homeless 2024-02-18T00:48:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f096bfff700 10 client.32280.objecter ms_handle_connect 0x7f096c062c90 2024-02-18T00:48:24.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.720+0000 7f09827fc700 1 --2- 172.21.15.59:0/1669725705 >> v2:172.21.15.59:6800/3721891797 conn(0x7f096c062c90 0x7f096c065150 secure :-1 s=READY pgs=3276 cs=0 l=1 rev1=1 rx=0x7f0974005010 tx=0x7f0974009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:24.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.724+0000 7f096bfff700 1 -- 172.21.15.59:0/1669725705 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0978049f60 con 0x7f09840f7400 2024-02-18T00:48:24.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.899+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f098405ee80 con 0x7f09840f7400 2024-02-18T00:48:24.901 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.899+0000 7f096bfff700 1 -- 172.21.15.59:0/1669725705 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f098405ee80 con 0x7f09840f7400 2024-02-18T00:48:24.901 INFO:teuthology.orchestra.run.smithi059.stdout:55834575111 2024-02-18T00:48:24.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 10 librados: watch_flush enter 2024-02-18T00:48:24.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 10 librados: watch_flush exit 2024-02-18T00:48:24.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 20 client.32280.objecter shutdown clearing up homeless session... 2024-02-18T00:48:24.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 10 client.32280.objecter successfully canceled tick 2024-02-18T00:48:24.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 >> v2:172.21.15.59:6800/3721891797 conn(0x7f096c062c90 msgr2=0x7f096c065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:24.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 --2- 172.21.15.59:0/1669725705 >> v2:172.21.15.59:6800/3721891797 conn(0x7f096c062c90 0x7f096c065150 secure :-1 s=READY pgs=3276 cs=0 l=1 rev1=1 rx=0x7f0974005010 tx=0x7f0974009cb0).stop 2024-02-18T00:48:24.904 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840f7400 msgr2=0x7f098410e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840f7400 0x7f098410e170 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 rx=0x7f0978009400 tx=0x7f0978009cb0).stop 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 shutdown_connections 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09840fd510 0x7f0984108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09840f7400 0x7f098410e170 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 --2- 172.21.15.59:0/1669725705 >> v2:172.21.15.59:6800/3721891797 conn(0x7f096c062c90 0x7f096c065150 unknown :-1 s=CLOSED pgs=3276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 --2- 172.21.15.59:0/1669725705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09840fcbb0 0x7f098410e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 >> 172.21.15.59:0/1669725705 conn(0x7f09840f2820 msgr2=0x7f0984060db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 shutdown_connections 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 -- 172.21.15.59:0/1669725705 wait complete. 2024-02-18T00:48:24.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:24.901+0000 7f098996c700 1 librados: shutdown 2024-02-18T00:48:25.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:23.563967+0000 osd.3 (osd.3) 501 : cluster [DBG] 4.23 scrub starts 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:23.565283+0000 osd.3 (osd.3) 502 : cluster [DBG] 4.23 scrub ok 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:24.072973+0000 mgr.y (mgr.24454) 769 : cluster [DBG] pgmap v1349: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:24.378017+0000 osd.4 (osd.4) 497 : cluster [DBG] 2.f scrub starts 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:24.381474+0000 osd.4 (osd.4) 498 : cluster [DBG] 2.f scrub ok 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:24.900851+0000 mon.a (mon.0) 3861 : audit [DBG] from='client.? 172.21.15.59:0/1669725705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:23.563967+0000 osd.3 (osd.3) 501 : cluster [DBG] 4.23 scrub starts 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:23.565283+0000 osd.3 (osd.3) 502 : cluster [DBG] 4.23 scrub ok 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:24.072973+0000 mgr.y (mgr.24454) 769 : cluster [DBG] pgmap v1349: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-18T00:48:25.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:24.378017+0000 osd.4 (osd.4) 497 : cluster [DBG] 2.f scrub starts 2024-02-18T00:48:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:24.381474+0000 osd.4 (osd.4) 498 : cluster [DBG] 2.f scrub ok 2024-02-18T00:48:25.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:25 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:24.900851+0000 mon.a (mon.0) 3861 : audit [DBG] from='client.? 172.21.15.59:0/1669725705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T00:48:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:23.563967+0000 osd.3 (osd.3) 501 : cluster [DBG] 4.23 scrub starts 2024-02-18T00:48:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:23.565283+0000 osd.3 (osd.3) 502 : cluster [DBG] 4.23 scrub ok 2024-02-18T00:48:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:24.072973+0000 mgr.y (mgr.24454) 769 : cluster [DBG] pgmap v1349: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1004 B/s rd, 0 op/s 2024-02-18T00:48:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:24.378017+0000 osd.4 (osd.4) 497 : cluster [DBG] 2.f scrub starts 2024-02-18T00:48:25.503 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:24.381474+0000 osd.4 (osd.4) 498 : cluster [DBG] 2.f scrub ok 2024-02-18T00:48:25.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:25 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:24.900851+0000 mon.a (mon.0) 3861 : audit [DBG] from='client.? 172.21.15.59:0/1669725705' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T00:48:26.204 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575106 got 55834575111 for osd.1 2024-02-18T00:48:26.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-18T00:48:26.491 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:26.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:24.652283+0000 osd.6 (osd.6) 401 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:48:26.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:24.653222+0000 osd.6 (osd.6) 402 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:48:26.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:24.780833+0000 osd.1 (osd.1) 451 : cluster [DBG] 2.15 deep-scrub starts 2024-02-18T00:48:26.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:24.781640+0000 osd.1 (osd.1) 452 : cluster [DBG] 2.15 deep-scrub ok 2024-02-18T00:48:26.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:25.758494+0000 osd.2 (osd.2) 389 : cluster [DBG] 5.a scrub starts 2024-02-18T00:48:26.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:25.759781+0000 osd.2 (osd.2) 390 : cluster [DBG] 5.a scrub ok 2024-02-18T00:48:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:24.652283+0000 osd.6 (osd.6) 401 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:48:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:24.653222+0000 osd.6 (osd.6) 402 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:48:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:24.780833+0000 osd.1 (osd.1) 451 : cluster [DBG] 2.15 deep-scrub starts 2024-02-18T00:48:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:24.781640+0000 osd.1 (osd.1) 452 : cluster [DBG] 2.15 deep-scrub ok 2024-02-18T00:48:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:25.758494+0000 osd.2 (osd.2) 389 : cluster [DBG] 5.a scrub starts 2024-02-18T00:48:26.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:26 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:25.759781+0000 osd.2 (osd.2) 390 : cluster [DBG] 5.a scrub ok 2024-02-18T00:48:26.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:24.652283+0000 osd.6 (osd.6) 401 : cluster [DBG] 2.17 scrub starts 2024-02-18T00:48:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:24.653222+0000 osd.6 (osd.6) 402 : cluster [DBG] 2.17 scrub ok 2024-02-18T00:48:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:24.780833+0000 osd.1 (osd.1) 451 : cluster [DBG] 2.15 deep-scrub starts 2024-02-18T00:48:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:24.781640+0000 osd.1 (osd.1) 452 : cluster [DBG] 2.15 deep-scrub ok 2024-02-18T00:48:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:25.758494+0000 osd.2 (osd.2) 389 : cluster [DBG] 5.a scrub starts 2024-02-18T00:48:26.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:25.759781+0000 osd.2 (osd.2) 390 : cluster [DBG] 5.a scrub ok 2024-02-18T00:48:27.527 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.073866+0000 mgr.y (mgr.24454) 770 : cluster [DBG] pgmap v1350: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:27.527 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.127831+0000 osd.0 (osd.0) 418 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:48:27.527 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.129046+0000 osd.0 (osd.0) 419 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:48:27.528 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.073866+0000 mgr.y (mgr.24454) 770 : cluster [DBG] pgmap v1350: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:27.528 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.127831+0000 osd.0 (osd.0) 418 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:48:27.528 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:27 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.129046+0000 osd.0 (osd.0) 419 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:48:27.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.073866+0000 mgr.y (mgr.24454) 770 : cluster [DBG] pgmap v1350: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:27.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.127831+0000 osd.0 (osd.0) 418 : cluster [DBG] 5.5 scrub starts 2024-02-18T00:48:27.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:27 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.129046+0000 osd.0 (osd.0) 419 : cluster [DBG] 5.5 scrub ok 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.841646+0000 osd.6 (osd.6) 403 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.841903+0000 osd.6 (osd.6) 404 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.842802+0000 osd.1 (osd.1) 453 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.846274+0000 osd.1 (osd.1) 454 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.854803+0000 osd.5 (osd.5) 436 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:26.857818+0000 osd.5 (osd.5) 437 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:27.255681+0000 mon.a (mon.0) 3862 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:27.754819+0000 osd.2 (osd.2) 391 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:48:28.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:27.755863+0000 osd.2 (osd.2) 392 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.841646+0000 osd.6 (osd.6) 403 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.841903+0000 osd.6 (osd.6) 404 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.842802+0000 osd.1 (osd.1) 453 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.846274+0000 osd.1 (osd.1) 454 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.854803+0000 osd.5 (osd.5) 436 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:26.857818+0000 osd.5 (osd.5) 437 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:27.255681+0000 mon.a (mon.0) 3862 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:27.754819+0000 osd.2 (osd.2) 391 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:48:28.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:28 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:27.755863+0000 osd.2 (osd.2) 392 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:48:28.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.489+0000 7fe503c23700 1 -- 172.21.15.59:0/1731599678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 msgr2=0x7fe4fc103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.489+0000 7fe503c23700 1 --2- 172.21.15.59:0/1731599678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc103a20 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 rx=0x7fe4f4009be0 tx=0x7fe4f4004d10).stop 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.489+0000 7fe503c23700 1 -- 172.21.15.59:0/1731599678 shutdown_connections 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.489+0000 7fe503c23700 1 --2- 172.21.15.59:0/1731599678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc0fcbb0 0x7fe4fc0fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.489+0000 7fe503c23700 1 --2- 172.21.15.59:0/1731599678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc103a20 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.489+0000 7fe503c23700 1 --2- 172.21.15.59:0/1731599678 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4fc0fd510 0x7fe4fc0f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.489+0000 7fe503c23700 1 -- 172.21.15.59:0/1731599678 >> 172.21.15.59:0/1731599678 conn(0x7fe4fc0f2820 msgr2=0x7fe4fc0f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 -- 172.21.15.59:0/1731599678 shutdown_connections 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 -- 172.21.15.59:0/1731599678 wait complete. 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 librados: starting msgr at 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 librados: starting objecter 2024-02-18T00:48:28.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 Processor -- start 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 -- start start 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 librados: setting wanted keys 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 librados: calling monclient init 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc109100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc0fcbb0 0x7fe4fc1095f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4fc0fd510 0x7fe4fc109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.490+0000 7fe503c23700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1142e0 con 0x7fe4fc0f7400 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe503c23700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc114450 con 0x7fe4fc0fcbb0 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe503c23700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc114610 con 0x7fe4fc0fd510 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5019bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5021c0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4fc0fd510 0x7fe4fc109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5019bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37206/0 (socket says 172.21.15.59:37206) 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5019bf700 1 -- 172.21.15.59:0/2195782949 learned_addr learned my addr 172.21.15.59:0/2195782949 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:28.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5011be700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc0fcbb0 0x7fe4fc1095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5021c0700 1 -- 172.21.15.59:0/2195782949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc0fcbb0 msgr2=0x7fe4fc1095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5021c0700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc0fcbb0 0x7fe4fc1095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5021c0700 1 -- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 msgr2=0x7fe4fc109100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5021c0700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc109100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5021c0700 1 -- 172.21.15.59:0/2195782949 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec005040 con 0x7fe4fc0fd510 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5019bf700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe5021c0700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4fc0fd510 0x7fe4fc109bd0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 rx=0x7fe4f4004e00 tx=0x7fe4f4004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe4f2ffd700 10 client.?.objecter ms_handle_connect 0x7fe4fc0fd510 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe4f2ffd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe4f2ffd700 1 -- 172.21.15.59:0/2195782949 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f400e040 con 0x7fe4fc0fd510 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe4f2ffd700 1 -- 172.21.15.59:0/2195782949 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4f4018040 con 0x7fe4fc0fd510 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f40097c0 con 0x7fe4fc0fd510 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe4f2ffd700 1 -- 172.21.15.59:0/2195782949 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4013cf0 con 0x7fe4fc0fd510 2024-02-18T00:48:28.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe503c23700 10 client.42089.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc114b00 con 0x7fe4fc0fd510 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.491+0000 7fe503c23700 1 librados: init done 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc05fa60 con 0x7fe4fc0fd510 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe4f2ffd700 1 -- 172.21.15.59:0/2195782949 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe4f4013420 con 0x7fe4fc0fd510 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe4f2ffd700 1 --2- 172.21.15.59:0/2195782949 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe4e8062c90 0x7fe4e8065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe4f2ffd700 1 -- 172.21.15.59:0/2195782949 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(850..850 src has 418..850) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fe4f407db90 con 0x7fe4fc0fd510 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe4f2ffd700 10 client.42089.objecter ms_dispatch 0x7fe4fc001550 osd_map(850..850 src has 418..850) v4 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe4f2ffd700 3 client.42089.objecter handle_osd_map got epochs [850,850] > 0 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe4f2ffd700 3 client.42089.objecter handle_osd_map decoding full epoch 850 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.493+0000 7fe5019bf700 1 --2- 172.21.15.59:0/2195782949 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe4e8062c90 0x7fe4e8065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.494+0000 7fe4f2ffd700 20 client.42089.objecter dump_active .. 0 homeless 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.494+0000 7fe4f2ffd700 10 client.42089.objecter ms_handle_connect 0x7fe4e8062c90 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.494+0000 7fe5019bf700 1 --2- 172.21.15.59:0/2195782949 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe4e8062c90 0x7fe4e8065150 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 rx=0x7fe4f8007d30 tx=0x7fe4f800b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:28.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.498+0000 7fe4f2ffd700 1 -- 172.21.15.59:0/2195782949 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe4f4051310 con 0x7fe4fc0fd510 2024-02-18T00:48:28.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.669+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe4fc061440 con 0x7fe4fc0fd510 2024-02-18T00:48:28.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.670+0000 7fe4f2ffd700 1 -- 172.21.15.59:0/2195782949 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4f4021020 con 0x7fe4fc0fd510 2024-02-18T00:48:28.671 INFO:teuthology.orchestra.run.smithi059.stdout:77309411589 2024-02-18T00:48:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 10 librados: watch_flush enter 2024-02-18T00:48:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 10 librados: watch_flush exit 2024-02-18T00:48:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 20 client.42089.objecter shutdown clearing up homeless session... 2024-02-18T00:48:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 10 client.42089.objecter successfully canceled tick 2024-02-18T00:48:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe4e8062c90 msgr2=0x7fe4e8065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 1 --2- 172.21.15.59:0/2195782949 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe4e8062c90 0x7fe4e8065150 secure :-1 s=READY pgs=3277 cs=0 l=1 rev1=1 rx=0x7fe4f8007d30 tx=0x7fe4f800b040).stop 2024-02-18T00:48:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4fc0fd510 msgr2=0x7fe4fc109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4fc0fd510 0x7fe4fc109bd0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 rx=0x7fe4f4004e00 tx=0x7fe4f4004d10).stop 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 shutdown_connections 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4fc0fd510 0x7fe4fc109bd0 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.671+0000 7fe503c23700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4fc0f7400 0x7fe4fc109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.672+0000 7fe503c23700 1 --2- 172.21.15.59:0/2195782949 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe4e8062c90 0x7fe4e8065150 unknown :-1 s=CLOSED pgs=3277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.672+0000 7fe503c23700 1 --2- 172.21.15.59:0/2195782949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4fc0fcbb0 0x7fe4fc1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.672+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 >> 172.21.15.59:0/2195782949 conn(0x7fe4fc0f2820 msgr2=0x7fe4fc0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.672+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 shutdown_connections 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.672+0000 7fe503c23700 1 -- 172.21.15.59:0/2195782949 wait complete. 2024-02-18T00:48:28.674 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:28.672+0000 7fe503c23700 1 librados: shutdown 2024-02-18T00:48:28.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.841646+0000 osd.6 (osd.6) 403 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:28.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.841903+0000 osd.6 (osd.6) 404 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.842802+0000 osd.1 (osd.1) 453 : cluster [DBG] 4.12 scrub starts 2024-02-18T00:48:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.846274+0000 osd.1 (osd.1) 454 : cluster [DBG] 4.12 scrub ok 2024-02-18T00:48:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.854803+0000 osd.5 (osd.5) 436 : cluster [DBG] 2.16 scrub starts 2024-02-18T00:48:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:26.857818+0000 osd.5 (osd.5) 437 : cluster [DBG] 2.16 scrub ok 2024-02-18T00:48:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:27.255681+0000 mon.a (mon.0) 3862 : cluster [DBG] osdmap e850: 8 total, 8 up, 8 in 2024-02-18T00:48:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:27.754819+0000 osd.2 (osd.2) 391 : cluster [DBG] 2.21 scrub starts 2024-02-18T00:48:28.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:28 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:27.755863+0000 osd.2 (osd.2) 392 : cluster [DBG] 2.21 scrub ok 2024-02-18T00:48:29.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:28.074421+0000 mgr.y (mgr.24454) 771 : cluster [DBG] pgmap v1352: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:28.488983+0000 osd.4 (osd.4) 499 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:28.492229+0000 osd.4 (osd.4) 500 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:28.671100+0000 mon.c (mon.2) 594 : audit [DBG] from='client.? 172.21.15.59:0/2195782949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:28.774789+0000 osd.4 (osd.4) 501 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:28.775061+0000 osd.4 (osd.4) 502 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:28.881254+0000 mon.a (mon.0) 3863 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:28.074421+0000 mgr.y (mgr.24454) 771 : cluster [DBG] pgmap v1352: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:28.488983+0000 osd.4 (osd.4) 499 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:28.492229+0000 osd.4 (osd.4) 500 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:48:29.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:28.671100+0000 mon.c (mon.2) 594 : audit [DBG] from='client.? 172.21.15.59:0/2195782949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T00:48:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:28.774789+0000 osd.4 (osd.4) 501 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:28.775061+0000 osd.4 (osd.4) 502 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:29.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:29 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:28.881254+0000 mon.a (mon.0) 3863 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-02-18T00:48:29.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411584 got 77309411589 for osd.2 2024-02-18T00:48:29.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-18T00:48:29.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:28.074421+0000 mgr.y (mgr.24454) 771 : cluster [DBG] pgmap v1352: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:29.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:28.488983+0000 osd.4 (osd.4) 499 : cluster [DBG] 5.1c scrub starts 2024-02-18T00:48:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:28.492229+0000 osd.4 (osd.4) 500 : cluster [DBG] 5.1c scrub ok 2024-02-18T00:48:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:28.671100+0000 mon.c (mon.2) 594 : audit [DBG] from='client.? 172.21.15.59:0/2195782949' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T00:48:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:28.774789+0000 osd.4 (osd.4) 501 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:28.775061+0000 osd.4 (osd.4) 502 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:29.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:29 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:28.881254+0000 mon.a (mon.0) 3863 : cluster [DBG] osdmap e851: 8 total, 8 up, 8 in 2024-02-18T00:48:29.896 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:30.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.333125+0000 osd.0 (osd.0) 420 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:30.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.333309+0000 osd.0 (osd.0) 421 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:30.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.751637+0000 osd.2 (osd.2) 393 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:48:30.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.752473+0000 osd.2 (osd.2) 394 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:48:30.466 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.333125+0000 osd.0 (osd.0) 420 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:30.466 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.333309+0000 osd.0 (osd.0) 421 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:30.466 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.751637+0000 osd.2 (osd.2) 393 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:48:30.466 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.752473+0000 osd.2 (osd.2) 394 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:48:30.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.333125+0000 osd.0 (osd.0) 420 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.333309+0000 osd.0 (osd.0) 421 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.751637+0000 osd.2 (osd.2) 393 : cluster [DBG] 2.8 scrub starts 2024-02-18T00:48:30.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:30 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.752473+0000 osd.2 (osd.2) 394 : cluster [DBG] 2.8 scrub ok 2024-02-18T00:48:31.503 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.587028+0000 osd.3 (osd.3) 503 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.588374+0000 osd.3 (osd.3) 504 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.904221+0000 osd.5 (osd.5) 438 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:29.905146+0000 osd.5 (osd.5) 439 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.074866+0000 mgr.y (mgr.24454) 772 : cluster [DBG] pgmap v1354: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.276586+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.460305+0000 osd.4 (osd.4) 503 : cluster [DBG] 5.e scrub starts 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.463979+0000 osd.4 (osd.4) 504 : cluster [DBG] 5.e scrub ok 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.800333+0000 osd.2 (osd.2) 395 : cluster [DBG] 2.e scrub starts 2024-02-18T00:48:31.504 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.801287+0000 osd.2 (osd.2) 396 : cluster [DBG] 2.e scrub ok 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.587028+0000 osd.3 (osd.3) 503 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.588374+0000 osd.3 (osd.3) 504 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.904221+0000 osd.5 (osd.5) 438 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:29.905146+0000 osd.5 (osd.5) 439 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.074866+0000 mgr.y (mgr.24454) 772 : cluster [DBG] pgmap v1354: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.276586+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.460305+0000 osd.4 (osd.4) 503 : cluster [DBG] 5.e scrub starts 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.463979+0000 osd.4 (osd.4) 504 : cluster [DBG] 5.e scrub ok 2024-02-18T00:48:31.699 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.800333+0000 osd.2 (osd.2) 395 : cluster [DBG] 2.e scrub starts 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.801287+0000 osd.2 (osd.2) 396 : cluster [DBG] 2.e scrub ok 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.587028+0000 osd.3 (osd.3) 503 : cluster [DBG] 6.6 scrub starts 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.588374+0000 osd.3 (osd.3) 504 : cluster [DBG] 6.6 scrub ok 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.904221+0000 osd.5 (osd.5) 438 : cluster [DBG] 2.18 scrub starts 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:29.905146+0000 osd.5 (osd.5) 439 : cluster [DBG] 2.18 scrub ok 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.074866+0000 mgr.y (mgr.24454) 772 : cluster [DBG] pgmap v1354: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.276586+0000 mon.a (mon.0) 3864 : cluster [DBG] osdmap e852: 8 total, 8 up, 8 in 2024-02-18T00:48:31.700 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.460305+0000 osd.4 (osd.4) 503 : cluster [DBG] 5.e scrub starts 2024-02-18T00:48:31.701 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.463979+0000 osd.4 (osd.4) 504 : cluster [DBG] 5.e scrub ok 2024-02-18T00:48:31.701 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.800333+0000 osd.2 (osd.2) 395 : cluster [DBG] 2.e scrub starts 2024-02-18T00:48:31.701 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:31 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.801287+0000 osd.2 (osd.2) 396 : cluster [DBG] 2.e scrub ok 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.043+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/706907902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40f8420 msgr2=0x7fb3c4104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.043+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/706907902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40f8420 0x7fb3c4104a20 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 rx=0x7fb3bc009be0 tx=0x7fb3bc004d10).stop 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/706907902 shutdown_connections 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/706907902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c40fdbb0 0x7fb3c40fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/706907902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40f8420 0x7fb3c4104a20 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/706907902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3c40fe510 0x7fb3c40f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/706907902 >> 172.21.15.59:0/706907902 conn(0x7fb3c40f3800 msgr2=0x7fb3c40f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/706907902 shutdown_connections 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/706907902 wait complete. 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 librados: starting msgr at 2024-02-18T00:48:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 librados: starting objecter 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 Processor -- start 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 -- start start 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 librados: setting wanted keys 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 librados: calling monclient init 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3c40f8420 0x7fb3c406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40fdbb0 0x7fb3c406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.044+0000 7fb3cc5eb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c40fe510 0x7fb3c406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cc5eb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4116330 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cc5eb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c41164a0 con 0x7fb3c40fe510 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cc5eb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4116660 con 0x7fb3c40f8420 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40fdbb0 0x7fb3c406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40fdbb0 0x7fb3c406b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51492/0 (socket says 172.21.15.59:51492) 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 -- 172.21.15.59:0/3310806326 learned_addr learned my addr 172.21.15.59:0/3310806326 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3ca387700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3c40f8420 0x7fb3c406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cab88700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c40fe510 0x7fb3c406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 -- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3c40f8420 msgr2=0x7fb3c406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3c40f8420 0x7fb3c406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 -- 172.21.15.59:0/3310806326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c40fe510 msgr2=0x7fb3c406bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c40fe510 0x7fb3c406bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 -- 172.21.15.59:0/3310806326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b8005040 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3c9b86700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40fdbb0 0x7fb3c406b5c0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 rx=0x7fb3b80048f0 tx=0x7fb3b8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3b77fe700 10 client.?.objecter ms_handle_connect 0x7fb3c40fdbb0 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3b77fe700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3b77fe700 1 -- 172.21.15.59:0/3310806326 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8007b00 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3b77fe700 1 -- 172.21.15.59:0/3310806326 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3b800f040 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3bc0097c0 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3b77fe700 1 -- 172.21.15.59:0/3310806326 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8014990 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cc5eb700 10 client.32292.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c4116d20 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.045+0000 7fb3cc5eb700 1 librados: init done 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.046+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c4107110 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 1 -- 172.21.15.59:0/3310806326 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fb3b8010500 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 1 --2- 172.21.15.59:0/3310806326 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3b0062c90 0x7fb3b0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 1 -- 172.21.15.59:0/3310806326 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(852..852 src has 418..852) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fb3b807cb90 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 10 client.32292.objecter ms_dispatch 0x7fb3c4001550 osd_map(852..852 src has 418..852) v4 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 3 client.32292.objecter handle_osd_map got epochs [852,852] > 0 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 3 client.32292.objecter handle_osd_map decoding full epoch 852 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3ca387700 1 --2- 172.21.15.59:0/3310806326 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3b0062c90 0x7fb3b0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 20 client.32292.objecter dump_active .. 0 homeless 2024-02-18T00:48:32.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3b77fe700 10 client.32292.objecter ms_handle_connect 0x7fb3b0062c90 2024-02-18T00:48:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.047+0000 7fb3ca387700 1 --2- 172.21.15.59:0/3310806326 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3b0062c90 0x7fb3b0065150 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 rx=0x7fb3c00041d0 tx=0x7fb3c0009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:32.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.051+0000 7fb3b77fe700 1 -- 172.21.15.59:0/3310806326 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb3b8049f60 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.225+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb3c405fa90 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.226+0000 7fb3b77fe700 1 -- 172.21.15.59:0/3310806326 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3b8050260 con 0x7fb3c40fdbb0 2024-02-18T00:48:32.227 INFO:teuthology.orchestra.run.smithi059.stdout:111669149951 2024-02-18T00:48:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 10 librados: watch_flush enter 2024-02-18T00:48:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 10 librados: watch_flush exit 2024-02-18T00:48:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 20 client.32292.objecter shutdown clearing up homeless session... 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 10 client.32292.objecter successfully canceled tick 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3b0062c90 msgr2=0x7fb3b0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/3310806326 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3b0062c90 0x7fb3b0065150 secure :-1 s=READY pgs=3278 cs=0 l=1 rev1=1 rx=0x7fb3c00041d0 tx=0x7fb3c0009040).stop 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40fdbb0 msgr2=0x7fb3c406b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.227+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40fdbb0 0x7fb3c406b5c0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 rx=0x7fb3b80048f0 tx=0x7fb3b8009cb0).stop 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 shutdown_connections 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/3310806326 >> v2:172.21.15.59:6800/3721891797 conn(0x7fb3b0062c90 0x7fb3b0065150 unknown :-1 s=CLOSED pgs=3278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3c40fe510 0x7fb3c406bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3c40f8420 0x7fb3c406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 --2- 172.21.15.59:0/3310806326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3c40fdbb0 0x7fb3c406b5c0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 >> 172.21.15.59:0/3310806326 conn(0x7fb3c40f3800 msgr2=0x7fb3c40f5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 shutdown_connections 2024-02-18T00:48:32.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 -- 172.21.15.59:0/3310806326 wait complete. 2024-02-18T00:48:32.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:32.228+0000 7fb3cc5eb700 1 librados: shutdown 2024-02-18T00:48:32.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:48:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:48:32.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:32] "GET /metrics HTTP/1.1" 200 187526 "" "Prometheus/2.33.4" 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.730883+0000 osd.1 (osd.1) 455 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:30.734511+0000 osd.1 (osd.1) 456 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:31.611044+0000 osd.6 (osd.6) 405 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:31.612036+0000 osd.6 (osd.6) 406 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.027395+0000 osd.0 (osd.0) 422 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.028255+0000 osd.0 (osd.0) 423 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:32.227257+0000 mon.a (mon.0) 3865 : audit [DBG] from='client.? 172.21.15.59:0/3310806326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.730883+0000 osd.1 (osd.1) 455 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:30.734511+0000 osd.1 (osd.1) 456 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:31.611044+0000 osd.6 (osd.6) 405 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:48:32.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:31.612036+0000 osd.6 (osd.6) 406 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:48:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.027395+0000 osd.0 (osd.0) 422 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:48:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.028255+0000 osd.0 (osd.0) 423 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:48:32.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:32 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:32.227257+0000 mon.a (mon.0) 3865 : audit [DBG] from='client.? 172.21.15.59:0/3310806326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T00:48:32.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.730883+0000 osd.1 (osd.1) 455 : cluster [DBG] 6.4 scrub starts 2024-02-18T00:48:32.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:30.734511+0000 osd.1 (osd.1) 456 : cluster [DBG] 6.4 scrub ok 2024-02-18T00:48:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:31.611044+0000 osd.6 (osd.6) 405 : cluster [DBG] 2.7 scrub starts 2024-02-18T00:48:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:31.612036+0000 osd.6 (osd.6) 406 : cluster [DBG] 2.7 scrub ok 2024-02-18T00:48:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.027395+0000 osd.0 (osd.0) 422 : cluster [DBG] 6.0 scrub starts 2024-02-18T00:48:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.028255+0000 osd.0 (osd.0) 423 : cluster [DBG] 6.0 scrub ok 2024-02-18T00:48:32.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:32.227257+0000 mon.a (mon.0) 3865 : audit [DBG] from='client.? 172.21.15.59:0/3310806326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T00:48:33.232 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149946 got 111669149951 for osd.3 2024-02-18T00:48:33.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-18T00:48:33.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:31.763964+0000 mgr.y (mgr.24454) 773 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:33.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.075812+0000 mgr.y (mgr.24454) 774 : cluster [DBG] pgmap v1356: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:33.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.795563+0000 osd.2 (osd.2) 397 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.796322+0000 osd.2 (osd.2) 398 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:33.067223+0000 osd.0 (osd.0) 424 : cluster [DBG] 4.16 deep-scrub starts 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:33.068978+0000 osd.0 (osd.0) 425 : cluster [DBG] 4.16 deep-scrub ok 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:31.763964+0000 mgr.y (mgr.24454) 773 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.075812+0000 mgr.y (mgr.24454) 774 : cluster [DBG] pgmap v1356: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.795563+0000 osd.2 (osd.2) 397 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.796322+0000 osd.2 (osd.2) 398 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:33.067223+0000 osd.0 (osd.0) 424 : cluster [DBG] 4.16 deep-scrub starts 2024-02-18T00:48:33.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:33.068978+0000 osd.0 (osd.0) 425 : cluster [DBG] 4.16 deep-scrub ok 2024-02-18T00:48:33.518 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:33.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:31.763964+0000 mgr.y (mgr.24454) 773 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:33.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.075812+0000 mgr.y (mgr.24454) 774 : cluster [DBG] pgmap v1356: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:33.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.795563+0000 osd.2 (osd.2) 397 : cluster [DBG] 2.1 scrub starts 2024-02-18T00:48:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.796322+0000 osd.2 (osd.2) 398 : cluster [DBG] 2.1 scrub ok 2024-02-18T00:48:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:33.067223+0000 osd.0 (osd.0) 424 : cluster [DBG] 4.16 deep-scrub starts 2024-02-18T00:48:33.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:33.068978+0000 osd.0 (osd.0) 425 : cluster [DBG] 4.16 deep-scrub ok 2024-02-18T00:48:34.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.767887+0000 osd.1 (osd.1) 457 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.770764+0000 osd.1 (osd.1) 458 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.896811+0000 osd.5 (osd.5) 440 : cluster [DBG] 6.d scrub starts 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:32.899938+0000 osd.5 (osd.5) 441 : cluster [DBG] 6.d scrub ok 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:33.473741+0000 osd.4 (osd.4) 505 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:33.474940+0000 osd.4 (osd.4) 506 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:33.880301+0000 mon.a (mon.0) 3866 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.767887+0000 osd.1 (osd.1) 457 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.770764+0000 osd.1 (osd.1) 458 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.896811+0000 osd.5 (osd.5) 440 : cluster [DBG] 6.d scrub starts 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:32.899938+0000 osd.5 (osd.5) 441 : cluster [DBG] 6.d scrub ok 2024-02-18T00:48:34.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:33.473741+0000 osd.4 (osd.4) 505 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:48:34.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:33.474940+0000 osd.4 (osd.4) 506 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:48:34.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:33.880301+0000 mon.a (mon.0) 3866 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-02-18T00:48:34.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.767887+0000 osd.1 (osd.1) 457 : cluster [DBG] 5.12 scrub starts 2024-02-18T00:48:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.770764+0000 osd.1 (osd.1) 458 : cluster [DBG] 5.12 scrub ok 2024-02-18T00:48:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.896811+0000 osd.5 (osd.5) 440 : cluster [DBG] 6.d scrub starts 2024-02-18T00:48:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:32.899938+0000 osd.5 (osd.5) 441 : cluster [DBG] 6.d scrub ok 2024-02-18T00:48:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:33.473741+0000 osd.4 (osd.4) 505 : cluster [DBG] 6.2 scrub starts 2024-02-18T00:48:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:33.474940+0000 osd.4 (osd.4) 506 : cluster [DBG] 6.2 scrub ok 2024-02-18T00:48:34.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:33.880301+0000 mon.a (mon.0) 3866 : cluster [DBG] osdmap e853: 8 total, 8 up, 8 in 2024-02-18T00:48:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 -- 172.21.15.59:0/2362289064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440fe510 msgr2=0x7fa4440f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 --2- 172.21.15.59:0/2362289064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440fe510 0x7fa4440f7d40 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 rx=0x7fa4400047b0 tx=0x7fa440009cb0).stop 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 -- 172.21.15.59:0/2362289064 shutdown_connections 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 --2- 172.21.15.59:0/2362289064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4440f8420 0x7fa444104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 --2- 172.21.15.59:0/2362289064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4440fdbb0 0x7fa4440fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 --2- 172.21.15.59:0/2362289064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440fe510 0x7fa4440f7d40 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 -- 172.21.15.59:0/2362289064 >> 172.21.15.59:0/2362289064 conn(0x7fa4440f3800 msgr2=0x7fa4440f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 -- 172.21.15.59:0/2362289064 shutdown_connections 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.564+0000 7fa44c057700 1 -- 172.21.15.59:0/2362289064 wait complete. 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 librados: starting msgr at 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 librados: starting objecter 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 Processor -- start 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 -- start start 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 librados: setting wanted keys 2024-02-18T00:48:35.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 librados: calling monclient init 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440f8420 0x7fa44410e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.565+0000 7fa44c057700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4440fdbb0 0x7fa44410e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4440fe510 0x7fa444108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4441086f0 con 0x7fa4440f8420 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa444108860 con 0x7fa4440fdbb0 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa444108a20 con 0x7fa4440fe510 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440f8420 0x7fa44410e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440f8420 0x7fa44410e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51498/0 (socket says 172.21.15.59:51498) 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 -- 172.21.15.59:0/69334118 learned_addr learned my addr 172.21.15.59:0/69334118 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44a5f4700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4440fe510 0x7fa444108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa4495f2700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4440fdbb0 0x7fa44410e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 -- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4440fe510 msgr2=0x7fa444108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:35.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4440fe510 0x7fa444108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 -- 172.21.15.59:0/69334118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4440fdbb0 msgr2=0x7fa44410e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4440fdbb0 0x7fa44410e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 -- 172.21.15.59:0/69334118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434005040 con 0x7fa4440f8420 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa4495f2700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4440fdbb0 0x7fa44410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa449df3700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440f8420 0x7fa44410e060 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 rx=0x7fa434009400 tx=0x7fa434009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa43affd700 10 client.?.objecter ms_handle_connect 0x7fa4440f8420 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa43affd700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa43affd700 1 -- 172.21.15.59:0/69334118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434007a40 con 0x7fa4440f8420 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa440005040 con 0x7fa4440f8420 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa43affd700 1 -- 172.21.15.59:0/69334118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa43400f040 con 0x7fa4440f8420 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa43affd700 1 -- 172.21.15.59:0/69334118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434014aa0 con 0x7fa4440f8420 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 10 client.32298.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:35.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa444114e90 con 0x7fa4440f8420 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.566+0000 7fa44c057700 1 librados: init done 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.567+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44405ee80 con 0x7fa4440f8420 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 1 -- 172.21.15.59:0/69334118 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fa43400e630 con 0x7fa4440f8420 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 1 --2- 172.21.15.59:0/69334118 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa430062c90 0x7fa430065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 1 -- 172.21.15.59:0/69334118 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(853..853 src has 418..853) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fa43407cc70 con 0x7fa4440f8420 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 10 client.32298.objecter ms_dispatch 0x7fa444001550 osd_map(853..853 src has 418..853) v4 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 3 client.32298.objecter handle_osd_map got epochs [853,853] > 0 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 3 client.32298.objecter handle_osd_map decoding full epoch 853 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa4495f2700 1 --2- 172.21.15.59:0/69334118 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa430062c90 0x7fa430065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 20 client.32298.objecter dump_active .. 0 homeless 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa43affd700 10 client.32298.objecter ms_handle_connect 0x7fa430062c90 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.568+0000 7fa4495f2700 1 --2- 172.21.15.59:0/69334118 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa430062c90 0x7fa430065150 secure :-1 s=READY pgs=3279 cs=0 l=1 rev1=1 rx=0x7fa440009bc0 tx=0x7fa440007f90).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:35.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.572+0000 7fa43affd700 1 -- 172.21.15.59:0/69334118 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa434049f60 con 0x7fa4440f8420 2024-02-18T00:48:35.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.746+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa4441097f0 con 0x7fa4440f8420 2024-02-18T00:48:35.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.746+0000 7fa43affd700 1 -- 172.21.15.59:0/69334118 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa434050340 con 0x7fa4440f8420 2024-02-18T00:48:35.748 INFO:teuthology.orchestra.run.smithi059.stdout:133143986427 2024-02-18T00:48:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 10 librados: watch_flush enter 2024-02-18T00:48:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 10 librados: watch_flush exit 2024-02-18T00:48:35.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 20 client.32298.objecter shutdown clearing up homeless session... 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 10 client.32298.objecter successfully canceled tick 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa430062c90 msgr2=0x7fa430065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 --2- 172.21.15.59:0/69334118 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa430062c90 0x7fa430065150 secure :-1 s=READY pgs=3279 cs=0 l=1 rev1=1 rx=0x7fa440009bc0 tx=0x7fa440007f90).stop 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440f8420 msgr2=0x7fa44410e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440f8420 0x7fa44410e060 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 rx=0x7fa434009400 tx=0x7fa434009cb0).stop 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 shutdown_connections 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa4440fe510 0x7fa444108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4440f8420 0x7fa44410e060 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 --2- 172.21.15.59:0/69334118 >> v2:172.21.15.59:6800/3721891797 conn(0x7fa430062c90 0x7fa430065150 unknown :-1 s=CLOSED pgs=3279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 --2- 172.21.15.59:0/69334118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4440fdbb0 0x7fa44410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 >> 172.21.15.59:0/69334118 conn(0x7fa4440f3800 msgr2=0x7fa4440fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 shutdown_connections 2024-02-18T00:48:35.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 -- 172.21.15.59:0/69334118 wait complete. 2024-02-18T00:48:35.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:35.748+0000 7fa44c057700 1 librados: shutdown 2024-02-18T00:48:35.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:34.076196+0000 mgr.y (mgr.24454) 775 : cluster [DBG] pgmap v1358: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:35.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:34.513762+0000 osd.6 (osd.6) 407 : cluster [DBG] 4.5 deep-scrub starts 2024-02-18T00:48:35.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:34.518342+0000 osd.6 (osd.6) 408 : cluster [DBG] 4.5 deep-scrub ok 2024-02-18T00:48:35.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:34.076196+0000 mgr.y (mgr.24454) 775 : cluster [DBG] pgmap v1358: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:35.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:34.513762+0000 osd.6 (osd.6) 407 : cluster [DBG] 4.5 deep-scrub starts 2024-02-18T00:48:35.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:34.518342+0000 osd.6 (osd.6) 408 : cluster [DBG] 4.5 deep-scrub ok 2024-02-18T00:48:35.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:34.076196+0000 mgr.y (mgr.24454) 775 : cluster [DBG] pgmap v1358: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:35.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:34.513762+0000 osd.6 (osd.6) 407 : cluster [DBG] 4.5 deep-scrub starts 2024-02-18T00:48:35.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:34.518342+0000 osd.6 (osd.6) 408 : cluster [DBG] 4.5 deep-scrub ok 2024-02-18T00:48:36.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:34.898616+0000 osd.5 (osd.5) 442 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:34.902133+0000 osd.5 (osd.5) 443 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.508660+0000 osd.4 (osd.4) 507 : cluster [DBG] 6.e scrub starts 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.511679+0000 osd.4 (osd.4) 508 : cluster [DBG] 6.e scrub ok 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.536826+0000 osd.6 (osd.6) 409 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.538657+0000 osd.6 (osd.6) 410 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:35.747730+0000 mon.a (mon.0) 3867 : audit [DBG] from='client.? 172.21.15.59:0/69334118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:34.898616+0000 osd.5 (osd.5) 442 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:34.902133+0000 osd.5 (osd.5) 443 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.508660+0000 osd.4 (osd.4) 507 : cluster [DBG] 6.e scrub starts 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.511679+0000 osd.4 (osd.4) 508 : cluster [DBG] 6.e scrub ok 2024-02-18T00:48:36.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.536826+0000 osd.6 (osd.6) 409 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:48:36.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.538657+0000 osd.6 (osd.6) 410 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:48:36.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:36 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:35.747730+0000 mon.a (mon.0) 3867 : audit [DBG] from='client.? 172.21.15.59:0/69334118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T00:48:36.706 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986422 got 133143986427 for osd.4 2024-02-18T00:48:36.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-18T00:48:36.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:34.898616+0000 osd.5 (osd.5) 442 : cluster [DBG] 4.4 scrub starts 2024-02-18T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:34.902133+0000 osd.5 (osd.5) 443 : cluster [DBG] 4.4 scrub ok 2024-02-18T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.508660+0000 osd.4 (osd.4) 507 : cluster [DBG] 6.e scrub starts 2024-02-18T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.511679+0000 osd.4 (osd.4) 508 : cluster [DBG] 6.e scrub ok 2024-02-18T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.536826+0000 osd.6 (osd.6) 409 : cluster [DBG] 4.1f scrub starts 2024-02-18T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.538657+0000 osd.6 (osd.6) 410 : cluster [DBG] 4.1f scrub ok 2024-02-18T00:48:36.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:35.747730+0000 mon.a (mon.0) 3867 : audit [DBG] from='client.? 172.21.15.59:0/69334118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T00:48:36.993 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:37.533 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.496074+0000 osd.3 (osd.3) 505 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:48:37.533 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.497929+0000 osd.3 (osd.3) 506 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:48:37.533 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.577769+0000 osd.5 (osd.5) 444 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:37.533 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:35.577947+0000 osd.5 (osd.5) 445 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:37.533 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.049667+0000 osd.7 (osd.7) 321 : cluster [DBG] 2.b scrub starts 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.050596+0000 osd.7 (osd.7) 322 : cluster [DBG] 2.b scrub ok 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.076924+0000 mgr.y (mgr.24454) 776 : cluster [DBG] pgmap v1359: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.293088+0000 mon.a (mon.0) 3868 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.496074+0000 osd.3 (osd.3) 505 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.497929+0000 osd.3 (osd.3) 506 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.577769+0000 osd.5 (osd.5) 444 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:35.577947+0000 osd.5 (osd.5) 445 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.049667+0000 osd.7 (osd.7) 321 : cluster [DBG] 2.b scrub starts 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.050596+0000 osd.7 (osd.7) 322 : cluster [DBG] 2.b scrub ok 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.076924+0000 mgr.y (mgr.24454) 776 : cluster [DBG] pgmap v1359: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:48:37.534 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:37 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.293088+0000 mon.a (mon.0) 3868 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-02-18T00:48:37.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.496074+0000 osd.3 (osd.3) 505 : cluster [DBG] 4.19 scrub starts 2024-02-18T00:48:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.497929+0000 osd.3 (osd.3) 506 : cluster [DBG] 4.19 scrub ok 2024-02-18T00:48:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.577769+0000 osd.5 (osd.5) 444 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:35.577947+0000 osd.5 (osd.5) 445 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.049667+0000 osd.7 (osd.7) 321 : cluster [DBG] 2.b scrub starts 2024-02-18T00:48:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.050596+0000 osd.7 (osd.7) 322 : cluster [DBG] 2.b scrub ok 2024-02-18T00:48:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.076924+0000 mgr.y (mgr.24454) 776 : cluster [DBG] pgmap v1359: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T00:48:37.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.293088+0000 mon.a (mon.0) 3868 : cluster [DBG] osdmap e854: 8 total, 8 up, 8 in 2024-02-18T00:48:38.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.887569+0000 osd.5 (osd.5) 446 : cluster [DBG] 5.f scrub starts 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.890527+0000 osd.5 (osd.5) 447 : cluster [DBG] 5.f scrub ok 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.912456+0000 osd.2 (osd.2) 399 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:36.913841+0000 osd.2 (osd.2) 400 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:37.595627+0000 osd.6 (osd.6) 411 : cluster [DBG] 25.d deep-scrub starts 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:37.597367+0000 osd.6 (osd.6) 412 : cluster [DBG] 25.d deep-scrub ok 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.887569+0000 osd.5 (osd.5) 446 : cluster [DBG] 5.f scrub starts 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.890527+0000 osd.5 (osd.5) 447 : cluster [DBG] 5.f scrub ok 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.912456+0000 osd.2 (osd.2) 399 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:48:38.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:36.913841+0000 osd.2 (osd.2) 400 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:48:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:37.595627+0000 osd.6 (osd.6) 411 : cluster [DBG] 25.d deep-scrub starts 2024-02-18T00:48:38.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:38 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:37.597367+0000 osd.6 (osd.6) 412 : cluster [DBG] 25.d deep-scrub ok 2024-02-18T00:48:38.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.887569+0000 osd.5 (osd.5) 446 : cluster [DBG] 5.f scrub starts 2024-02-18T00:48:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.890527+0000 osd.5 (osd.5) 447 : cluster [DBG] 5.f scrub ok 2024-02-18T00:48:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.912456+0000 osd.2 (osd.2) 399 : cluster [DBG] 25.18 scrub starts 2024-02-18T00:48:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:36.913841+0000 osd.2 (osd.2) 400 : cluster [DBG] 25.18 scrub ok 2024-02-18T00:48:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:37.595627+0000 osd.6 (osd.6) 411 : cluster [DBG] 25.d deep-scrub starts 2024-02-18T00:48:38.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:37.597367+0000 osd.6 (osd.6) 412 : cluster [DBG] 25.d deep-scrub ok 2024-02-18T00:48:39.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 -- 172.21.15.59:0/2311184077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fd510 msgr2=0x7efbf40f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:39.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 --2- 172.21.15.59:0/2311184077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fd510 0x7efbf40f6d40 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 rx=0x7efbdc004870 tx=0x7efbdc009cb0).stop 2024-02-18T00:48:39.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 -- 172.21.15.59:0/2311184077 shutdown_connections 2024-02-18T00:48:39.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 --2- 172.21.15.59:0/2311184077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf40fcbb0 0x7efbf40fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 --2- 172.21.15.59:0/2311184077 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efbf40f7400 0x7efbf4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 --2- 172.21.15.59:0/2311184077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fd510 0x7efbf40f6d40 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 -- 172.21.15.59:0/2311184077 >> 172.21.15.59:0/2311184077 conn(0x7efbf40f2820 msgr2=0x7efbf40f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:39.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 -- 172.21.15.59:0/2311184077 shutdown_connections 2024-02-18T00:48:39.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.133+0000 7efbf9087700 1 -- 172.21.15.59:0/2311184077 wait complete. 2024-02-18T00:48:39.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.134+0000 7efbf9087700 1 librados: starting msgr at 2024-02-18T00:48:39.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.134+0000 7efbf9087700 1 librados: starting objecter 2024-02-18T00:48:39.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.134+0000 7efbf9087700 1 Processor -- start 2024-02-18T00:48:39.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.134+0000 7efbf9087700 1 -- start start 2024-02-18T00:48:39.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.134+0000 7efbf9087700 1 librados: setting wanted keys 2024-02-18T00:48:39.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.134+0000 7efbf9087700 1 librados: calling monclient init 2024-02-18T00:48:39.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf9087700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efbf40f7400 0x7efbf40f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:39.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf9087700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fcbb0 0x7efbf40f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf9087700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf40fd510 0x7efbf40f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf9087700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf41166b0 con 0x7efbf40fcbb0 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf9087700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4116820 con 0x7efbf40fd510 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf9087700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf41169e0 con 0x7efbf40f7400 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fcbb0 0x7efbf40f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf2d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efbf40f7400 0x7efbf40f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fcbb0 0x7efbf40f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51514/0 (socket says 172.21.15.59:51514) 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 -- 172.21.15.59:0/3430738834 learned_addr learned my addr 172.21.15.59:0/3430738834 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf359e700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf40fd510 0x7efbf40f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 -- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efbf40f7400 msgr2=0x7efbf40f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efbf40f7400 0x7efbf40f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 -- 172.21.15.59:0/3430738834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf40fd510 msgr2=0x7efbf40f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf40fd510 0x7efbf40f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 -- 172.21.15.59:0/3430738834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec0097c0 con 0x7efbf40fcbb0 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf2d9d700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efbf40f7400 0x7efbf40f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf259c700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fcbb0 0x7efbf40f8ea0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 rx=0x7efbdc0043d0 tx=0x7efbdc009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbebfff700 10 client.?.objecter ms_handle_connect 0x7efbf40fcbb0 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbebfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbebfff700 1 -- 172.21.15.59:0/3430738834 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbdc00e030 con 0x7efbf40fcbb0 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbdc005040 con 0x7efbf40fcbb0 2024-02-18T00:48:39.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbebfff700 1 -- 172.21.15.59:0/3430738834 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbdc013da0 con 0x7efbf40fcbb0 2024-02-18T00:48:39.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.135+0000 7efbebfff700 1 -- 172.21.15.59:0/3430738834 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbdc018070 con 0x7efbf40fcbb0 2024-02-18T00:48:39.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.136+0000 7efbf9087700 10 client.32304.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:39.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.136+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf4116e80 con 0x7efbf40fcbb0 2024-02-18T00:48:39.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.136+0000 7efbf9087700 1 librados: init done 2024-02-18T00:48:39.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf405fa60 con 0x7efbf40fcbb0 2024-02-18T00:48:39.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbebfff700 1 -- 172.21.15.59:0/3430738834 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7efbdc01d420 con 0x7efbf40fcbb0 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbebfff700 1 --2- 172.21.15.59:0/3430738834 >> v2:172.21.15.59:6800/3721891797 conn(0x7efbe0062ce0 0x7efbe00651a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbebfff700 1 -- 172.21.15.59:0/3430738834 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(855..855 src has 418..855) v4 ==== 7613+0+0 (secure 0 0 0) 0x7efbdc07e2e0 con 0x7efbf40fcbb0 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbebfff700 10 client.32304.objecter ms_dispatch 0x7efbf4001550 osd_map(855..855 src has 418..855) v4 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbebfff700 3 client.32304.objecter handle_osd_map got epochs [855,855] > 0 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbebfff700 3 client.32304.objecter handle_osd_map decoding full epoch 855 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.137+0000 7efbf2d9d700 1 --2- 172.21.15.59:0/3430738834 >> v2:172.21.15.59:6800/3721891797 conn(0x7efbe0062ce0 0x7efbe00651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.138+0000 7efbebfff700 20 client.32304.objecter dump_active .. 0 homeless 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.138+0000 7efbebfff700 10 client.32304.objecter ms_handle_connect 0x7efbe0062ce0 2024-02-18T00:48:39.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.138+0000 7efbf2d9d700 1 --2- 172.21.15.59:0/3430738834 >> v2:172.21.15.59:6800/3721891797 conn(0x7efbe0062ce0 0x7efbe00651a0 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 rx=0x7efbe40103c0 tx=0x7efbe400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:39.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.142+0000 7efbebfff700 1 -- 172.21.15.59:0/3430738834 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efbdc0519b0 con 0x7efbf40fcbb0 2024-02-18T00:48:39.319 INFO:teuthology.orchestra.run.smithi059.stdout:167503724791 2024-02-18T00:48:39.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.317+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efbf40f9e90 con 0x7efbf40fcbb0 2024-02-18T00:48:39.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.318+0000 7efbebfff700 1 -- 172.21.15.59:0/3430738834 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efbdc0515a0 con 0x7efbf40fcbb0 2024-02-18T00:48:39.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.319+0000 7efbf9087700 10 librados: watch_flush enter 2024-02-18T00:48:39.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.319+0000 7efbf9087700 10 librados: watch_flush exit 2024-02-18T00:48:39.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.319+0000 7efbf9087700 20 client.32304.objecter shutdown clearing up homeless session... 2024-02-18T00:48:39.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.319+0000 7efbf9087700 10 client.32304.objecter successfully canceled tick 2024-02-18T00:48:39.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.319+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 >> v2:172.21.15.59:6800/3721891797 conn(0x7efbe0062ce0 msgr2=0x7efbe00651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.319+0000 7efbf9087700 1 --2- 172.21.15.59:0/3430738834 >> v2:172.21.15.59:6800/3721891797 conn(0x7efbe0062ce0 0x7efbe00651a0 secure :-1 s=READY pgs=3280 cs=0 l=1 rev1=1 rx=0x7efbe40103c0 tx=0x7efbe400b040).stop 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fcbb0 msgr2=0x7efbf40f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fcbb0 0x7efbf40f8ea0 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 rx=0x7efbdc0043d0 tx=0x7efbdc009cb0).stop 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 shutdown_connections 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 --2- 172.21.15.59:0/3430738834 >> v2:172.21.15.59:6800/3721891797 conn(0x7efbe0062ce0 0x7efbe00651a0 unknown :-1 s=CLOSED pgs=3280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efbf40fd510 0x7efbf40f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efbf40f7400 0x7efbf40f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 --2- 172.21.15.59:0/3430738834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efbf40fcbb0 0x7efbf40f8ea0 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 >> 172.21.15.59:0/3430738834 conn(0x7efbf40f2820 msgr2=0x7efbf4060dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:39.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 shutdown_connections 2024-02-18T00:48:39.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 -- 172.21.15.59:0/3430738834 wait complete. 2024-02-18T00:48:39.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:39.320+0000 7efbf9087700 1 librados: shutdown 2024-02-18T00:48:39.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:37.959001+0000 osd.2 (osd.2) 401 : cluster [DBG] 5.b scrub starts 2024-02-18T00:48:39.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:37.960316+0000 osd.2 (osd.2) 402 : cluster [DBG] 5.b scrub ok 2024-02-18T00:48:39.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.077472+0000 mgr.y (mgr.24454) 777 : cluster [DBG] pgmap v1361: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:48:39.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.553574+0000 osd.4 (osd.4) 509 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:48:39.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.557058+0000 osd.4 (osd.4) 510 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:48:39.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.883172+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-02-18T00:48:39.712 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:37.959001+0000 osd.2 (osd.2) 401 : cluster [DBG] 5.b scrub starts 2024-02-18T00:48:39.712 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:37.960316+0000 osd.2 (osd.2) 402 : cluster [DBG] 5.b scrub ok 2024-02-18T00:48:39.712 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.077472+0000 mgr.y (mgr.24454) 777 : cluster [DBG] pgmap v1361: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:48:39.712 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.553574+0000 osd.4 (osd.4) 509 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:48:39.712 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.557058+0000 osd.4 (osd.4) 510 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:48:39.712 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:39 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.883172+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-02-18T00:48:39.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:37.959001+0000 osd.2 (osd.2) 401 : cluster [DBG] 5.b scrub starts 2024-02-18T00:48:39.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:37.960316+0000 osd.2 (osd.2) 402 : cluster [DBG] 5.b scrub ok 2024-02-18T00:48:39.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.077472+0000 mgr.y (mgr.24454) 777 : cluster [DBG] pgmap v1361: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T00:48:39.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.553574+0000 osd.4 (osd.4) 509 : cluster [DBG] 4.1b scrub starts 2024-02-18T00:48:39.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.557058+0000 osd.4 (osd.4) 510 : cluster [DBG] 4.1b scrub ok 2024-02-18T00:48:39.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.883172+0000 mon.a (mon.0) 3869 : cluster [DBG] osdmap e855: 8 total, 8 up, 8 in 2024-02-18T00:48:40.206 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724786 got 167503724791 for osd.5 2024-02-18T00:48:40.206 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-18T00:48:40.489 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.434481+0000 osd.3 (osd.3) 507 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:48:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.435615+0000 osd.3 (osd.3) 508 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:48:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.987574+0000 osd.2 (osd.2) 403 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:48:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:38.989805+0000 osd.2 (osd.2) 404 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:48:40.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.003011+0000 osd.7 (osd.7) 323 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.005346+0000 osd.7 (osd.7) 324 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:39.319314+0000 mon.a (mon.0) 3870 : audit [DBG] from='client.? 172.21.15.59:0/3430738834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.434481+0000 osd.3 (osd.3) 507 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.435615+0000 osd.3 (osd.3) 508 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.987574+0000 osd.2 (osd.2) 403 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:38.989805+0000 osd.2 (osd.2) 404 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.003011+0000 osd.7 (osd.7) 323 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.005346+0000 osd.7 (osd.7) 324 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:48:40.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:39.319314+0000 mon.a (mon.0) 3870 : audit [DBG] from='client.? 172.21.15.59:0/3430738834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T00:48:40.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.434481+0000 osd.3 (osd.3) 507 : cluster [DBG] 5.13 scrub starts 2024-02-18T00:48:40.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.435615+0000 osd.3 (osd.3) 508 : cluster [DBG] 5.13 scrub ok 2024-02-18T00:48:40.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.987574+0000 osd.2 (osd.2) 403 : cluster [DBG] 25.1 scrub starts 2024-02-18T00:48:40.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:38.989805+0000 osd.2 (osd.2) 404 : cluster [DBG] 25.1 scrub ok 2024-02-18T00:48:40.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.003011+0000 osd.7 (osd.7) 323 : cluster [DBG] 25.3 scrub starts 2024-02-18T00:48:40.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.005346+0000 osd.7 (osd.7) 324 : cluster [DBG] 25.3 scrub ok 2024-02-18T00:48:40.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:39.319314+0000 mon.a (mon.0) 3870 : audit [DBG] from='client.? 172.21.15.59:0/3430738834' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.455294+0000 osd.3 (osd.3) 509 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.456487+0000 osd.3 (osd.3) 510 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.617505+0000 osd.1 (osd.1) 459 : cluster [DBG] 4.26 deep-scrub starts 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.619971+0000 osd.1 (osd.1) 460 : cluster [DBG] 4.26 deep-scrub ok 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.969525+0000 osd.5 (osd.5) 448 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.972605+0000 osd.5 (osd.5) 449 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.979517+0000 osd.2 (osd.2) 405 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:39.980388+0000 osd.2 (osd.2) 406 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:40.077811+0000 mgr.y (mgr.24454) 778 : cluster [DBG] pgmap v1363: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:40.137250+0000 osd.0 (osd.0) 426 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:48:41.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:40.140214+0000 osd.0 (osd.0) 427 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:40.617895+0000 osd.4 (osd.4) 511 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:40.620852+0000 osd.4 (osd.4) 512 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.455294+0000 osd.3 (osd.3) 509 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.456487+0000 osd.3 (osd.3) 510 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.617505+0000 osd.1 (osd.1) 459 : cluster [DBG] 4.26 deep-scrub starts 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.619971+0000 osd.1 (osd.1) 460 : cluster [DBG] 4.26 deep-scrub ok 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.969525+0000 osd.5 (osd.5) 448 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.972605+0000 osd.5 (osd.5) 449 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.979517+0000 osd.2 (osd.2) 405 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:48:41.503 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:39.980388+0000 osd.2 (osd.2) 406 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:48:41.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:40.077811+0000 mgr.y (mgr.24454) 778 : cluster [DBG] pgmap v1363: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:48:41.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:40.137250+0000 osd.0 (osd.0) 426 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:48:41.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:40.140214+0000 osd.0 (osd.0) 427 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:48:41.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:40.617895+0000 osd.4 (osd.4) 511 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:48:41.504 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:41 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:40.620852+0000 osd.4 (osd.4) 512 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:48:41.504 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.455294+0000 osd.3 (osd.3) 509 : cluster [DBG] 2.23 scrub starts 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.456487+0000 osd.3 (osd.3) 510 : cluster [DBG] 2.23 scrub ok 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.617505+0000 osd.1 (osd.1) 459 : cluster [DBG] 4.26 deep-scrub starts 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.619971+0000 osd.1 (osd.1) 460 : cluster [DBG] 4.26 deep-scrub ok 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.969525+0000 osd.5 (osd.5) 448 : cluster [DBG] 5.16 scrub starts 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.972605+0000 osd.5 (osd.5) 449 : cluster [DBG] 5.16 scrub ok 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.979517+0000 osd.2 (osd.2) 405 : cluster [DBG] 2.5 scrub starts 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:39.980388+0000 osd.2 (osd.2) 406 : cluster [DBG] 2.5 scrub ok 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:40.077811+0000 mgr.y (mgr.24454) 778 : cluster [DBG] pgmap v1363: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 op/s 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:40.137250+0000 osd.0 (osd.0) 426 : cluster [DBG] 2.13 scrub starts 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:40.140214+0000 osd.0 (osd.0) 427 : cluster [DBG] 2.13 scrub ok 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:40.617895+0000 osd.4 (osd.4) 511 : cluster [DBG] 25.14 scrub starts 2024-02-18T00:48:41.505 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:40.620852+0000 osd.4 (osd.4) 512 : cluster [DBG] 25.14 scrub ok 2024-02-18T00:48:42.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:48:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:48:42.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:41.637850+0000 osd.4 (osd.4) 513 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:48:42.411 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:41.639815+0000 osd.4 (osd.4) 514 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:48:42.412 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:42] "GET /metrics HTTP/1.1" 200 187530 "" "Prometheus/2.33.4" 2024-02-18T00:48:42.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:41.637850+0000 osd.4 (osd.4) 513 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:48:42.412 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:42 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:41.639815+0000 osd.4 (osd.4) 514 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:48:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 -- 172.21.15.59:0/3660605176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f7330 msgr2=0x7f9ce4103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/3660605176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f7330 0x7f9ce4103a00 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 rx=0x7f9cdc009be0 tx=0x7f9cdc004d10).stop 2024-02-18T00:48:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 -- 172.21.15.59:0/3660605176 shutdown_connections 2024-02-18T00:48:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/3660605176 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f7330 0x7f9ce4103a00 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/3660605176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ce40f89a0 0x7f9ce40f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/3660605176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ce40f9270 0x7f9ce40f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 -- 172.21.15.59:0/3660605176 >> 172.21.15.59:0/3660605176 conn(0x7f9ce40f26f0 msgr2=0x7f9ce40f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 -- 172.21.15.59:0/3660605176 shutdown_connections 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 -- 172.21.15.59:0/3660605176 wait complete. 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 librados: starting msgr at 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 librados: starting objecter 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.677+0000 7f9ce97b3700 1 Processor -- start 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 -- start start 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 librados: setting wanted keys 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 librados: calling monclient init 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ce40f7330 0x7f9ce4106df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ce40f89a0 0x7f9ce41072e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f9270 0x7f9ce41077d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:42.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4116470 con 0x7f9ce40f7330 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce41165b0 con 0x7f9ce40f89a0 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce97b3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4116720 con 0x7f9ce40f9270 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f9270 0x7f9ce41077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ce40f7330 0x7f9ce4106df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f9270 0x7f9ce41077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38886/0 (socket says 172.21.15.59:38886) 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 -- 172.21.15.59:0/2423709683 learned_addr learned my addr 172.21.15.59:0/2423709683 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce27fc700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ce40f89a0 0x7f9ce41072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 -- 172.21.15.59:0/2423709683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ce40f89a0 msgr2=0x7f9ce41072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ce40f89a0 0x7f9ce41072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 -- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ce40f7330 msgr2=0x7f9ce4106df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ce40f7330 0x7f9ce4106df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.678+0000 7f9ce37fe700 1 -- 172.21.15.59:0/2423709683 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd4005040 con 0x7f9ce40f9270 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9ce2ffd700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ce40f7330 0x7f9ce4106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:42.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9ce37fe700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f9270 0x7f9ce41077d0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 rx=0x7f9cdc004a30 tx=0x7f9cdc004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9cdbfff700 10 client.?.objecter ms_handle_connect 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9cdbfff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9cdbfff700 1 -- 172.21.15.59:0/2423709683 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00e040 con 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9cdbfff700 1 -- 172.21.15.59:0/2423709683 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc018040 con 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc0097c0 con 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9cdbfff700 1 -- 172.21.15.59:0/2423709683 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc01c3f0 con 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9ce97b3700 10 client.42113.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce4116c90 con 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.679+0000 7f9ce97b3700 1 librados: init done 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.680+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce405ee80 con 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.680+0000 7f9cdbfff700 1 -- 172.21.15.59:0/2423709683 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f9cdc013420 con 0x7f9ce40f9270 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9cdbfff700 1 --2- 172.21.15.59:0/2423709683 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9cd0062c90 0x7f9cd0065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:42.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9cdbfff700 1 -- 172.21.15.59:0/2423709683 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(855..855 src has 418..855) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f9cdc07e120 con 0x7f9ce40f9270 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9cdbfff700 10 client.42113.objecter ms_dispatch 0x7f9ce4001550 osd_map(855..855 src has 418..855) v4 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9ce2ffd700 1 --2- 172.21.15.59:0/2423709683 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9cd0062c90 0x7f9cd0065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9cdbfff700 3 client.42113.objecter handle_osd_map got epochs [855,855] > 0 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9cdbfff700 3 client.42113.objecter handle_osd_map decoding full epoch 855 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9cdbfff700 20 client.42113.objecter dump_active .. 0 homeless 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9cdbfff700 10 client.42113.objecter ms_handle_connect 0x7f9cd0062c90 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.681+0000 7f9ce2ffd700 1 --2- 172.21.15.59:0/2423709683 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9cd0062c90 0x7f9cd0065150 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 rx=0x7f9cd4009760 tx=0x7f9cd400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:42.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.685+0000 7f9cdbfff700 1 -- 172.21.15.59:0/2423709683 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9cdc0518a0 con 0x7f9ce40f9270 2024-02-18T00:48:42.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:41.637850+0000 osd.4 (osd.4) 513 : cluster [DBG] 4.1a scrub starts 2024-02-18T00:48:42.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:41.639815+0000 osd.4 (osd.4) 514 : cluster [DBG] 4.1a scrub ok 2024-02-18T00:48:42.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.858+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9ce41060f0 con 0x7f9ce40f9270 2024-02-18T00:48:42.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.858+0000 7f9cdbfff700 1 -- 172.21.15.59:0/2423709683 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9cdc021070 con 0x7f9ce40f9270 2024-02-18T00:48:42.860 INFO:teuthology.orchestra.run.smithi059.stdout:201863463156 2024-02-18T00:48:42.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 10 librados: watch_flush enter 2024-02-18T00:48:42.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 10 librados: watch_flush exit 2024-02-18T00:48:42.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 20 client.42113.objecter shutdown clearing up homeless session... 2024-02-18T00:48:42.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 10 client.42113.objecter successfully canceled tick 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9cd0062c90 msgr2=0x7f9cd0065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/2423709683 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9cd0062c90 0x7f9cd0065150 secure :-1 s=READY pgs=3281 cs=0 l=1 rev1=1 rx=0x7f9cd4009760 tx=0x7f9cd400b040).stop 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f9270 msgr2=0x7f9ce41077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f9270 0x7f9ce41077d0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 rx=0x7f9cdc004a30 tx=0x7f9cdc004d10).stop 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 shutdown_connections 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ce40f9270 0x7f9ce41077d0 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ce40f7330 0x7f9ce4106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/2423709683 >> v2:172.21.15.59:6800/3721891797 conn(0x7f9cd0062c90 0x7f9cd0065150 unknown :-1 s=CLOSED pgs=3281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 --2- 172.21.15.59:0/2423709683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ce40f89a0 0x7f9ce41072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 >> 172.21.15.59:0/2423709683 conn(0x7f9ce40f26f0 msgr2=0x7f9ce40f2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 shutdown_connections 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 -- 172.21.15.59:0/2423709683 wait complete. 2024-02-18T00:48:42.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:42.860+0000 7f9ce97b3700 1 librados: shutdown 2024-02-18T00:48:43.714 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:41.769845+0000 mgr.y (mgr.24454) 779 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:43.714 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:42.078699+0000 mgr.y (mgr.24454) 780 : cluster [DBG] pgmap v1364: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:43.714 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:42.561516+0000 osd.6 (osd.6) 413 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:48:43.714 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:42.562467+0000 osd.6 (osd.6) 414 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:48:43.714 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:42.859602+0000 mon.c (mon.2) 595 : audit [DBG] from='client.? 172.21.15.59:0/2423709683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T00:48:43.715 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:41.769845+0000 mgr.y (mgr.24454) 779 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:43.715 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:42.078699+0000 mgr.y (mgr.24454) 780 : cluster [DBG] pgmap v1364: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:43.715 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:42.561516+0000 osd.6 (osd.6) 413 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:48:43.715 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:42.562467+0000 osd.6 (osd.6) 414 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:48:43.715 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:42.859602+0000 mon.c (mon.2) 595 : audit [DBG] from='client.? 172.21.15.59:0/2423709683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T00:48:43.737 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863463150 got 201863463156 for osd.6 2024-02-18T00:48:43.737 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-18T00:48:43.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:41.769845+0000 mgr.y (mgr.24454) 779 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:43.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:42.078699+0000 mgr.y (mgr.24454) 780 : cluster [DBG] pgmap v1364: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:43.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:42.561516+0000 osd.6 (osd.6) 413 : cluster [DBG] 2.24 scrub starts 2024-02-18T00:48:43.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:42.562467+0000 osd.6 (osd.6) 414 : cluster [DBG] 2.24 scrub ok 2024-02-18T00:48:43.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:42.859602+0000 mon.c (mon.2) 595 : audit [DBG] from='client.? 172.21.15.59:0/2423709683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T00:48:44.039 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:42.420049+0000 osd.3 (osd.3) 511 : cluster [DBG] 4.10 deep-scrub starts 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:42.424530+0000 osd.3 (osd.3) 512 : cluster [DBG] 4.10 deep-scrub ok 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:42.935057+0000 osd.5 (osd.5) 450 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:42.938536+0000 osd.5 (osd.5) 451 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.058130+0000 osd.7 (osd.7) 325 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.059051+0000 osd.7 (osd.7) 326 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.134785+0000 osd.0 (osd.0) 428 : cluster [DBG] 4.b scrub starts 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.138465+0000 osd.0 (osd.0) 429 : cluster [DBG] 4.b scrub ok 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.371039+0000 osd.3 (osd.3) 513 : cluster [DBG] 3.2 deep-scrub starts 2024-02-18T00:48:44.594 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.372057+0000 osd.3 (osd.3) 514 : cluster [DBG] 3.2 deep-scrub ok 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:42.420049+0000 osd.3 (osd.3) 511 : cluster [DBG] 4.10 deep-scrub starts 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:42.424530+0000 osd.3 (osd.3) 512 : cluster [DBG] 4.10 deep-scrub ok 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:42.935057+0000 osd.5 (osd.5) 450 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:42.938536+0000 osd.5 (osd.5) 451 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.058130+0000 osd.7 (osd.7) 325 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.059051+0000 osd.7 (osd.7) 326 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.134785+0000 osd.0 (osd.0) 428 : cluster [DBG] 4.b scrub starts 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.138465+0000 osd.0 (osd.0) 429 : cluster [DBG] 4.b scrub ok 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.371039+0000 osd.3 (osd.3) 513 : cluster [DBG] 3.2 deep-scrub starts 2024-02-18T00:48:44.595 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:44 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.372057+0000 osd.3 (osd.3) 514 : cluster [DBG] 3.2 deep-scrub ok 2024-02-18T00:48:44.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:42.420049+0000 osd.3 (osd.3) 511 : cluster [DBG] 4.10 deep-scrub starts 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:42.424530+0000 osd.3 (osd.3) 512 : cluster [DBG] 4.10 deep-scrub ok 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:42.935057+0000 osd.5 (osd.5) 450 : cluster [DBG] 5.15 deep-scrub starts 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:42.938536+0000 osd.5 (osd.5) 451 : cluster [DBG] 5.15 deep-scrub ok 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.058130+0000 osd.7 (osd.7) 325 : cluster [DBG] 5.1a deep-scrub starts 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.059051+0000 osd.7 (osd.7) 326 : cluster [DBG] 5.1a deep-scrub ok 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.134785+0000 osd.0 (osd.0) 428 : cluster [DBG] 4.b scrub starts 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.138465+0000 osd.0 (osd.0) 429 : cluster [DBG] 4.b scrub ok 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.371039+0000 osd.3 (osd.3) 513 : cluster [DBG] 3.2 deep-scrub starts 2024-02-18T00:48:44.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.372057+0000 osd.3 (osd.3) 514 : cluster [DBG] 3.2 deep-scrub ok 2024-02-18T00:48:45.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.980184+0000 osd.5 (osd.5) 452 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:48:45.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:43.981076+0000 osd.5 (osd.5) 453 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:48:45.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:44.079171+0000 mgr.y (mgr.24454) 781 : cluster [DBG] pgmap v1365: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:45.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.980184+0000 osd.5 (osd.5) 452 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:48:45.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:43.981076+0000 osd.5 (osd.5) 453 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:48:45.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:45 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:44.079171+0000 mgr.y (mgr.24454) 781 : cluster [DBG] pgmap v1365: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.980184+0000 osd.5 (osd.5) 452 : cluster [DBG] 2.2 scrub starts 2024-02-18T00:48:45.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:43.981076+0000 osd.5 (osd.5) 453 : cluster [DBG] 2.2 scrub ok 2024-02-18T00:48:45.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:44.079171+0000 mgr.y (mgr.24454) 781 : cluster [DBG] pgmap v1365: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T00:48:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 -- 172.21.15.59:0/2029860208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 msgr2=0x7f70780f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 --2- 172.21.15.59:0/2029860208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f70780f7d40 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 rx=0x7f7068004870 tx=0x7f7068009cb0).stop 2024-02-18T00:48:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 -- 172.21.15.59:0/2029860208 shutdown_connections 2024-02-18T00:48:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 --2- 172.21.15.59:0/2029860208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70780fdbb0 0x7f70780fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 --2- 172.21.15.59:0/2029860208 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780f8420 0x7f7078104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 --2- 172.21.15.59:0/2029860208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f70780f7d40 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 -- 172.21.15.59:0/2029860208 >> 172.21.15.59:0/2029860208 conn(0x7f70780f3800 msgr2=0x7f70780f5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 -- 172.21.15.59:0/2029860208 shutdown_connections 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.207+0000 7f707e319700 1 -- 172.21.15.59:0/2029860208 wait complete. 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 librados: starting msgr at 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 librados: starting objecter 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 Processor -- start 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 -- start start 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 librados: setting wanted keys 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 librados: calling monclient init 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70780f8420 0x7f707810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780fdbb0 0x7f707810e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:46.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f7078108150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078108640 con 0x7f70780fe510 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70781087b0 con 0x7f70780f8420 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.208+0000 7f707e319700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7078108970 con 0x7f70780fdbb0 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f7078108150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780fdbb0 0x7f707810e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f7078108150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39110/0 (socket says 172.21.15.59:39110) 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 -- 172.21.15.59:0/2073409463 learned_addr learned my addr 172.21.15.59:0/2073409463 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70777fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780fdbb0 0x7f707810e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:38900/0 (socket says 172.21.15.59:38900) 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f7077fff700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70780f8420 0x7f707810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 -- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780fdbb0 msgr2=0x7f707810e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780fdbb0 0x7f707810e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 -- 172.21.15.59:0/2073409463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70780f8420 msgr2=0x7f707810dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70780f8420 0x7f707810dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 -- 172.21.15.59:0/2073409463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068005040 con 0x7f70780fe510 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70777fe700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780fdbb0 0x7f707810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f7077fff700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70780f8420 0x7f707810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707c8b6700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f7078108150 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 rx=0x7f707000ec20 tx=0x7f707000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70757fa700 10 client.?.objecter ms_handle_connect 0x7f70780fe510 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70757fa700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70757fa700 1 -- 172.21.15.59:0/2073409463 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070009950 con 0x7f70780fe510 2024-02-18T00:48:46.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7078108bf0 con 0x7f70780fe510 2024-02-18T00:48:46.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70757fa700 1 -- 172.21.15.59:0/2073409463 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f707000f040 con 0x7f70780fe510 2024-02-18T00:48:46.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f70757fa700 1 -- 172.21.15.59:0/2073409463 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707000c6e0 con 0x7f70780fe510 2024-02-18T00:48:46.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707e319700 10 client.32316.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:46.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7078114de0 con 0x7f70780fe510 2024-02-18T00:48:46.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.209+0000 7f707e319700 1 librados: init done 2024-02-18T00:48:46.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707805ee80 con 0x7f70780fe510 2024-02-18T00:48:46.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f70757fa700 1 -- 172.21.15.59:0/2073409463 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f70700092e0 con 0x7f70780fe510 2024-02-18T00:48:46.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f70757fa700 1 --2- 172.21.15.59:0/2073409463 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7064062c90 0x7f7064065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:46.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f70757fa700 1 -- 172.21.15.59:0/2073409463 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(855..855 src has 418..855) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f707007ccd0 con 0x7f70780fe510 2024-02-18T00:48:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f70757fa700 10 client.32316.objecter ms_dispatch 0x7f7078001550 osd_map(855..855 src has 418..855) v4 2024-02-18T00:48:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f70757fa700 3 client.32316.objecter handle_osd_map got epochs [855,855] > 0 2024-02-18T00:48:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f70757fa700 3 client.32316.objecter handle_osd_map decoding full epoch 855 2024-02-18T00:48:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f7077fff700 1 --2- 172.21.15.59:0/2073409463 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7064062c90 0x7f7064065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.211+0000 7f70757fa700 20 client.32316.objecter dump_active .. 0 homeless 2024-02-18T00:48:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.212+0000 7f70757fa700 10 client.32316.objecter ms_handle_connect 0x7f7064062c90 2024-02-18T00:48:46.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.212+0000 7f7077fff700 1 --2- 172.21.15.59:0/2073409463 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7064062c90 0x7f7064065150 secure :-1 s=READY pgs=3282 cs=0 l=1 rev1=1 rx=0x7f7060007fd0 tx=0x7f7060009040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:46.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.216+0000 7f70757fa700 1 -- 172.21.15.59:0/2073409463 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7070049f60 con 0x7f70780fe510 2024-02-18T00:48:46.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.391+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7078109750 con 0x7f70780fe510 2024-02-18T00:48:46.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.391+0000 7f70757fa700 1 -- 172.21.15.59:0/2073409463 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7078109750 con 0x7f70780fe510 2024-02-18T00:48:46.394 INFO:teuthology.orchestra.run.smithi059.stdout:227633266927 2024-02-18T00:48:46.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 10 librados: watch_flush enter 2024-02-18T00:48:46.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 10 librados: watch_flush exit 2024-02-18T00:48:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 20 client.32316.objecter shutdown clearing up homeless session... 2024-02-18T00:48:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 10 client.32316.objecter successfully canceled tick 2024-02-18T00:48:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7064062c90 msgr2=0x7f7064065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 --2- 172.21.15.59:0/2073409463 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7064062c90 0x7f7064065150 secure :-1 s=READY pgs=3282 cs=0 l=1 rev1=1 rx=0x7f7060007fd0 tx=0x7f7060009040).stop 2024-02-18T00:48:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 msgr2=0x7f7078108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f7078108150 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 rx=0x7f707000ec20 tx=0x7f707000b040).stop 2024-02-18T00:48:46.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 shutdown_connections 2024-02-18T00:48:46.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 --2- 172.21.15.59:0/2073409463 >> v2:172.21.15.59:6800/3721891797 conn(0x7f7064062c90 0x7f7064065150 unknown :-1 s=CLOSED pgs=3282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70780f8420 0x7f707810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70780fe510 0x7f7078108150 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 --2- 172.21.15.59:0/2073409463 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70780fdbb0 0x7f707810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:46.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 >> 172.21.15.59:0/2073409463 conn(0x7f70780f3800 msgr2=0x7f70780fb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:46.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 shutdown_connections 2024-02-18T00:48:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.393+0000 7f707e319700 1 -- 172.21.15.59:0/2073409463 wait complete. 2024-02-18T00:48:46.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:46.394+0000 7f707e319700 1 librados: shutdown 2024-02-18T00:48:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:44.387922+0000 osd.3 (osd.3) 515 : cluster [DBG] 6.b deep-scrub starts 2024-02-18T00:48:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:44.389206+0000 osd.3 (osd.3) 516 : cluster [DBG] 6.b deep-scrub ok 2024-02-18T00:48:46.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:44.993819+0000 osd.5 (osd.5) 454 : cluster [DBG] 25.f scrub starts 2024-02-18T00:48:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:44.997847+0000 osd.5 (osd.5) 455 : cluster [DBG] 25.f scrub ok 2024-02-18T00:48:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:45.805123+0000 osd.4 (osd.4) 515 : cluster [DBG] 4.d scrub starts 2024-02-18T00:48:46.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:45.808582+0000 osd.4 (osd.4) 516 : cluster [DBG] 4.d scrub ok 2024-02-18T00:48:46.755 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:44.387922+0000 osd.3 (osd.3) 515 : cluster [DBG] 6.b deep-scrub starts 2024-02-18T00:48:46.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:44.389206+0000 osd.3 (osd.3) 516 : cluster [DBG] 6.b deep-scrub ok 2024-02-18T00:48:46.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:44.993819+0000 osd.5 (osd.5) 454 : cluster [DBG] 25.f scrub starts 2024-02-18T00:48:46.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:44.997847+0000 osd.5 (osd.5) 455 : cluster [DBG] 25.f scrub ok 2024-02-18T00:48:46.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:45.805123+0000 osd.4 (osd.4) 515 : cluster [DBG] 4.d scrub starts 2024-02-18T00:48:46.756 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:46 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:45.808582+0000 osd.4 (osd.4) 516 : cluster [DBG] 4.d scrub ok 2024-02-18T00:48:46.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:44.387922+0000 osd.3 (osd.3) 515 : cluster [DBG] 6.b deep-scrub starts 2024-02-18T00:48:46.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:44.389206+0000 osd.3 (osd.3) 516 : cluster [DBG] 6.b deep-scrub ok 2024-02-18T00:48:46.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:44.993819+0000 osd.5 (osd.5) 454 : cluster [DBG] 25.f scrub starts 2024-02-18T00:48:46.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:44.997847+0000 osd.5 (osd.5) 455 : cluster [DBG] 25.f scrub ok 2024-02-18T00:48:46.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:45.805123+0000 osd.4 (osd.4) 515 : cluster [DBG] 4.d scrub starts 2024-02-18T00:48:46.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:45.808582+0000 osd.4 (osd.4) 516 : cluster [DBG] 4.d scrub ok 2024-02-18T00:48:47.198 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266921 got 227633266927 for osd.7 2024-02-18T00:48:47.198 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-02-18T00:48:47.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:48:47.483 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:47.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:46.080131+0000 mgr.y (mgr.24454) 782 : cluster [DBG] pgmap v1366: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:48:47.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:46.356239+0000 osd.3 (osd.3) 517 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:48:47.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:46.357824+0000 osd.3 (osd.3) 518 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:46.392995+0000 mon.a (mon.0) 3871 : audit [DBG] from='client.? 172.21.15.59:0/2073409463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:46.656134+0000 osd.6 (osd.6) 415 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:46.657410+0000 osd.6 (osd.6) 416 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:46.080131+0000 mgr.y (mgr.24454) 782 : cluster [DBG] pgmap v1366: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:46.356239+0000 osd.3 (osd.3) 517 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:46.357824+0000 osd.3 (osd.3) 518 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:46.392995+0000 mon.a (mon.0) 3871 : audit [DBG] from='client.? 172.21.15.59:0/2073409463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:46.656134+0000 osd.6 (osd.6) 415 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:48:47.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:47 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:46.657410+0000 osd.6 (osd.6) 416 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:48:47.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:46.080131+0000 mgr.y (mgr.24454) 782 : cluster [DBG] pgmap v1366: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T00:48:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:46.356239+0000 osd.3 (osd.3) 517 : cluster [DBG] 4.17 scrub starts 2024-02-18T00:48:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:46.357824+0000 osd.3 (osd.3) 518 : cluster [DBG] 4.17 scrub ok 2024-02-18T00:48:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:46.392995+0000 mon.a (mon.0) 3871 : audit [DBG] from='client.? 172.21.15.59:0/2073409463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T00:48:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:46.656134+0000 osd.6 (osd.6) 415 : cluster [DBG] 2.22 scrub starts 2024-02-18T00:48:47.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:46.657410+0000 osd.6 (osd.6) 416 : cluster [DBG] 2.22 scrub ok 2024-02-18T00:48:48.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:46.689818+0000 osd.1 (osd.1) 461 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:48:48.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:46.693287+0000 osd.1 (osd.1) 462 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.684672+0000 osd.6 (osd.6) 417 : cluster [DBG] 4.a scrub starts 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.687162+0000 osd.6 (osd.6) 418 : cluster [DBG] 4.a scrub ok 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.807544+0000 osd.4 (osd.4) 517 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.810891+0000 osd.4 (osd.4) 518 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:46.689818+0000 osd.1 (osd.1) 461 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:46.693287+0000 osd.1 (osd.1) 462 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.684672+0000 osd.6 (osd.6) 417 : cluster [DBG] 4.a scrub starts 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.687162+0000 osd.6 (osd.6) 418 : cluster [DBG] 4.a scrub ok 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.807544+0000 osd.4 (osd.4) 517 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:48:48.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:48 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.810891+0000 osd.4 (osd.4) 518 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:48:48.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:46.689818+0000 osd.1 (osd.1) 461 : cluster [DBG] 4.24 scrub starts 2024-02-18T00:48:48.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:46.693287+0000 osd.1 (osd.1) 462 : cluster [DBG] 4.24 scrub ok 2024-02-18T00:48:48.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.684672+0000 osd.6 (osd.6) 417 : cluster [DBG] 4.a scrub starts 2024-02-18T00:48:48.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.687162+0000 osd.6 (osd.6) 418 : cluster [DBG] 4.a scrub ok 2024-02-18T00:48:48.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.807544+0000 osd.4 (osd.4) 517 : cluster [DBG] 25.1c scrub starts 2024-02-18T00:48:48.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.810891+0000 osd.4 (osd.4) 518 : cluster [DBG] 25.1c scrub ok 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 -- 172.21.15.59:0/3375693495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80fcbb0 msgr2=0x7f91b80fcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 --2- 172.21.15.59:0/3375693495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80fcbb0 0x7f91b80fcf90 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 rx=0x7f91a80047b0 tx=0x7f91a8009cb0).stop 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 -- 172.21.15.59:0/3375693495 shutdown_connections 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 --2- 172.21.15.59:0/3375693495 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91b80f7400 0x7f91b8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 --2- 172.21.15.59:0/3375693495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80fcbb0 0x7f91b80fcf90 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 --2- 172.21.15.59:0/3375693495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b80fd510 0x7f91b80f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 -- 172.21.15.59:0/3375693495 >> 172.21.15.59:0/3375693495 conn(0x7f91b80f2820 msgr2=0x7f91b80f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 -- 172.21.15.59:0/3375693495 shutdown_connections 2024-02-18T00:48:49.730 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.723+0000 7f91bd71c700 1 -- 172.21.15.59:0/3375693495 wait complete. 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 librados: starting msgr at 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 librados: starting objecter 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 Processor -- start 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 -- start start 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 librados: setting wanted keys 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 librados: calling monclient init 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80f7400 0x7f91b8109260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b80fcbb0 0x7f91b8109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91b80fd510 0x7f91b8109d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8114490 con 0x7f91b80f7400 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8114600 con 0x7f91b80fcbb0 2024-02-18T00:48:49.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.724+0000 7f91bd71c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b81147c0 con 0x7f91b80fd510 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80f7400 0x7f91b8109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80f7400 0x7f91b8109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55062/0 (socket says 172.21.15.59:55062) 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 -- 172.21.15.59:0/759581932 learned_addr learned my addr 172.21.15.59:0/759581932 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b77fe700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91b80fd510 0x7f91b8109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b67fc700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b80fcbb0 0x7f91b8109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 -- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91b80fd510 msgr2=0x7f91b8109d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91b80fd510 0x7f91b8109d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 -- 172.21.15.59:0/759581932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b80fcbb0 msgr2=0x7f91b8109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b80fcbb0 0x7f91b8109750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 -- 172.21.15.59:0/759581932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00097c0 con 0x7f91b80f7400 2024-02-18T00:48:49.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91b6ffd700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80f7400 0x7f91b8109260 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 rx=0x7f91a8009b10 tx=0x7f91a8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91affff700 10 client.?.objecter ms_handle_connect 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91affff700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91affff700 1 -- 172.21.15.59:0/759581932 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8018070 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91affff700 1 -- 172.21.15.59:0/759581932 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a8013e70 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91affff700 1 -- 172.21.15.59:0/759581932 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a801ddb0 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a8005040 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91bd71c700 10 client.32322.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b8114c60 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.725+0000 7f91bd71c700 1 librados: init done 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.726+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b805fa60 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91affff700 1 -- 172.21.15.59:0/759581932 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7f91a801d420 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91affff700 1 --2- 172.21.15.59:0/759581932 >> v2:172.21.15.59:6800/3721891797 conn(0x7f91a4062c90 0x7f91a4065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91affff700 1 -- 172.21.15.59:0/759581932 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(855..855 src has 418..855) v4 ==== 7613+0+0 (secure 0 0 0) 0x7f91a807e4b0 con 0x7f91b80f7400 2024-02-18T00:48:49.733 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91affff700 10 client.32322.objecter ms_dispatch 0x7f91b8001550 osd_map(855..855 src has 418..855) v4 2024-02-18T00:48:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91affff700 3 client.32322.objecter handle_osd_map got epochs [855,855] > 0 2024-02-18T00:48:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91affff700 3 client.32322.objecter handle_osd_map decoding full epoch 855 2024-02-18T00:48:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91b67fc700 1 --2- 172.21.15.59:0/759581932 >> v2:172.21.15.59:6800/3721891797 conn(0x7f91a4062c90 0x7f91a4065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.727+0000 7f91affff700 20 client.32322.objecter dump_active .. 0 homeless 2024-02-18T00:48:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.728+0000 7f91affff700 10 client.32322.objecter ms_handle_connect 0x7f91a4062c90 2024-02-18T00:48:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.728+0000 7f91b67fc700 1 --2- 172.21.15.59:0/759581932 >> v2:172.21.15.59:6800/3721891797 conn(0x7f91a4062c90 0x7f91a4065150 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 rx=0x7f91b8061760 tx=0x7f91a0009d60).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:49.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.731+0000 7f91affff700 1 -- 172.21.15.59:0/759581932 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f91a8051bc0 con 0x7f91b80f7400 2024-02-18T00:48:49.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.896505+0000 osd.2 (osd.2) 407 : cluster [DBG] 5.d scrub starts 2024-02-18T00:48:49.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.898123+0000 osd.2 (osd.2) 408 : cluster [DBG] 5.d scrub ok 2024-02-18T00:48:49.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.989736+0000 osd.7 (osd.7) 327 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:48:49.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:47.990755+0000 osd.7 (osd.7) 328 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:48:49.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.080478+0000 mgr.y (mgr.24454) 783 : cluster [DBG] pgmap v1367: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.198022+0000 osd.0 (osd.0) 430 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.199465+0000 osd.0 (osd.0) 431 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.765868+0000 osd.4 (osd.4) 519 : cluster [DBG] 25.b scrub starts 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.770075+0000 osd.4 (osd.4) 520 : cluster [DBG] 25.b scrub ok 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.896505+0000 osd.2 (osd.2) 407 : cluster [DBG] 5.d scrub starts 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.898123+0000 osd.2 (osd.2) 408 : cluster [DBG] 5.d scrub ok 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.989736+0000 osd.7 (osd.7) 327 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:47.990755+0000 osd.7 (osd.7) 328 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.080478+0000 mgr.y (mgr.24454) 783 : cluster [DBG] pgmap v1367: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.198022+0000 osd.0 (osd.0) 430 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:48:49.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.199465+0000 osd.0 (osd.0) 431 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.765868+0000 osd.4 (osd.4) 519 : cluster [DBG] 25.b scrub starts 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:49 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.770075+0000 osd.4 (osd.4) 520 : cluster [DBG] 25.b scrub ok 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.896505+0000 osd.2 (osd.2) 407 : cluster [DBG] 5.d scrub starts 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.898123+0000 osd.2 (osd.2) 408 : cluster [DBG] 5.d scrub ok 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.989736+0000 osd.7 (osd.7) 327 : cluster [DBG] 25.2 scrub starts 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:47.990755+0000 osd.7 (osd.7) 328 : cluster [DBG] 25.2 scrub ok 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.080478+0000 mgr.y (mgr.24454) 783 : cluster [DBG] pgmap v1367: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:49.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.198022+0000 osd.0 (osd.0) 430 : cluster [DBG] 25.15 scrub starts 2024-02-18T00:48:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.199465+0000 osd.0 (osd.0) 431 : cluster [DBG] 25.15 scrub ok 2024-02-18T00:48:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.765868+0000 osd.4 (osd.4) 519 : cluster [DBG] 25.b scrub starts 2024-02-18T00:48:49.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.770075+0000 osd.4 (osd.4) 520 : cluster [DBG] 25.b scrub ok 2024-02-18T00:48:49.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.905+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91b8106170 con 0x7f91a4062c90 2024-02-18T00:48:49.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.918+0000 7f91affff700 1 -- 172.21.15.59:0/759581932 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+454612 (secure 0 0 0) 0x7f91b8106170 con 0x7f91a4062c90 2024-02-18T00:48:49.926 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 10 librados: watch_flush enter 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 10 librados: watch_flush exit 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 20 client.32322.objecter shutdown clearing up homeless session... 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 10 client.32322.objecter successfully canceled tick 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 >> v2:172.21.15.59:6800/3721891797 conn(0x7f91a4062c90 msgr2=0x7f91a4065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 --2- 172.21.15.59:0/759581932 >> v2:172.21.15.59:6800/3721891797 conn(0x7f91a4062c90 0x7f91a4065150 secure :-1 s=READY pgs=3283 cs=0 l=1 rev1=1 rx=0x7f91b8061760 tx=0x7f91a0009d60).stop 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80f7400 msgr2=0x7f91b8109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80f7400 0x7f91b8109260 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 rx=0x7f91a8009b10 tx=0x7f91a8009cb0).stop 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 shutdown_connections 2024-02-18T00:48:49.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91b80fd510 0x7f91b8109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91b80f7400 0x7f91b8109260 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 --2- 172.21.15.59:0/759581932 >> v2:172.21.15.59:6800/3721891797 conn(0x7f91a4062c90 0x7f91a4065150 unknown :-1 s=CLOSED pgs=3283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 --2- 172.21.15.59:0/759581932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91b80fcbb0 0x7f91b8109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.921+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 >> 172.21.15.59:0/759581932 conn(0x7f91b80f2820 msgr2=0x7f91b80fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.922+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 shutdown_connections 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.922+0000 7f91bd71c700 1 -- 172.21.15.59:0/759581932 wait complete. 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:49.922+0000 7f91bd71c700 1 librados: shutdown 2024-02-18T00:48:49.932 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:48:50.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.406289+0000 osd.3 (osd.3) 519 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:48:50.751 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.407777+0000 osd.3 (osd.3) 520 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.989584+0000 osd.7 (osd.7) 329 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:48.990926+0000 osd.7 (osd.7) 330 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:49.187089+0000 osd.0 (osd.0) 432 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:49.189596+0000 osd.0 (osd.0) 433 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.406289+0000 osd.3 (osd.3) 519 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.407777+0000 osd.3 (osd.3) 520 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.989584+0000 osd.7 (osd.7) 329 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:48.990926+0000 osd.7 (osd.7) 330 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:49.187089+0000 osd.0 (osd.0) 432 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:48:50.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:49.189596+0000 osd.0 (osd.0) 433 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:48:50.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.406289+0000 osd.3 (osd.3) 519 : cluster [DBG] 2.1b scrub starts 2024-02-18T00:48:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.407777+0000 osd.3 (osd.3) 520 : cluster [DBG] 2.1b scrub ok 2024-02-18T00:48:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.989584+0000 osd.7 (osd.7) 329 : cluster [DBG] 6.3 scrub starts 2024-02-18T00:48:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:48.990926+0000 osd.7 (osd.7) 330 : cluster [DBG] 6.3 scrub ok 2024-02-18T00:48:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:49.187089+0000 osd.0 (osd.0) 432 : cluster [DBG] 25.1f scrub starts 2024-02-18T00:48:50.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:49.189596+0000 osd.0 (osd.0) 433 : cluster [DBG] 25.1f scrub ok 2024-02-18T00:48:50.844 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":1367,"stamp":"2024-02-18T00:48:48.080273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2120,"num_read_kb":1949,"num_write":579,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":8376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1425,"ondisk_log_size":1425,"up":522,"acting":522,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":522,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2472568,"kb_used_data":57912,"kb_used_omap":0,"kb_used_meta":2414592,"kb_avail":747488648,"statfs":{"total":767960285184,"available":765428375552,"internally_reserved":0,"allocated":59301888,"data_stored":32264243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2472542208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003132"},"pg_stats":[{"pgid":"4.29","version":"731'34","reported_seq":740,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.376870+0000","last_change":"2024-02-18T00:48:19.661865+0000","last_active":"2024-02-18T00:48:36.376870+0000","last_peered":"2024-02-18T00:48:36.376870+0000","last_clean":"2024-02-18T00:48:36.376870+0000","last_became_active":"2024-02-18T00:43:09.798744+0000","last_became_peered":"2024-02-18T00:43:09.798744+0000","last_unstale":"2024-02-18T00:48:36.376870+0000","last_undegraded":"2024-02-18T00:48:36.376870+0000","last_fullsized":"2024-02-18T00:48:36.376870+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"731'34","last_scrub_stamp":"2024-02-18T00:48:19.661781+0000","last_deep_scrub":"731'34","last_deep_scrub_stamp":"2024-02-18T00:47:02.906198+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.661781+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"457'39","reported_seq":777,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291279+0000","last_change":"2024-02-18T00:48:19.145476+0000","last_active":"2024-02-18T00:48:36.291279+0000","last_peered":"2024-02-18T00:48:36.291279+0000","last_clean":"2024-02-18T00:48:36.291279+0000","last_became_active":"2024-02-18T00:43:09.791383+0000","last_became_peered":"2024-02-18T00:43:09.791383+0000","last_unstale":"2024-02-18T00:48:36.291279+0000","last_undegraded":"2024-02-18T00:48:36.291279+0000","last_fullsized":"2024-02-18T00:48:36.291279+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:48:19.145389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.145389+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795447+0000","last_change":"2024-02-18T00:47:44.714703+0000","last_active":"2024-02-18T00:48:36.795447+0000","last_peered":"2024-02-18T00:48:36.795447+0000","last_clean":"2024-02-18T00:48:36.795447+0000","last_became_active":"2024-02-18T00:46:25.889860+0000","last_became_peered":"2024-02-18T00:46:25.889860+0000","last_unstale":"2024-02-18T00:48:36.795447+0000","last_undegraded":"2024-02-18T00:48:36.795447+0000","last_fullsized":"2024-02-18T00:48:36.795447+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:44.714596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:47:44.714596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":790,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.376839+0000","last_change":"2024-02-18T00:47:46.687503+0000","last_active":"2024-02-18T00:48:36.376839+0000","last_peered":"2024-02-18T00:48:36.376839+0000","last_clean":"2024-02-18T00:48:36.376839+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:48:36.376839+0000","last_undegraded":"2024-02-18T00:48:36.376839+0000","last_fullsized":"2024-02-18T00:48:36.376839+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:46.687418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:46.687418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.21","version":"457'34","reported_seq":211,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794006+0000","last_change":"2024-02-18T00:47:42.685385+0000","last_active":"2024-02-18T00:48:36.794006+0000","last_peered":"2024-02-18T00:48:36.794006+0000","last_clean":"2024-02-18T00:48:36.794006+0000","last_became_active":"2024-02-18T00:43:44.817896+0000","last_became_peered":"2024-02-18T00:43:44.817896+0000","last_unstale":"2024-02-18T00:48:36.794006+0000","last_undegraded":"2024-02-18T00:48:36.794006+0000","last_fullsized":"2024-02-18T00:48:36.794006+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:47:42.685299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:42.685299+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292786+0000","last_change":"2024-02-18T00:48:10.207239+0000","last_active":"2024-02-18T00:48:36.292786+0000","last_peered":"2024-02-18T00:48:36.292786+0000","last_clean":"2024-02-18T00:48:36.292786+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:48:36.292786+0000","last_undegraded":"2024-02-18T00:48:36.292786+0000","last_fullsized":"2024-02-18T00:48:36.292786+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:10.207161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:48:10.207161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.20","version":"457'52","reported_seq":76,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794465+0000","last_change":"2024-02-18T00:48:17.497291+0000","last_active":"2024-02-18T00:48:36.794465+0000","last_peered":"2024-02-18T00:48:36.794465+0000","last_clean":"2024-02-18T00:48:36.794465+0000","last_became_active":"2024-02-18T00:46:25.883556+0000","last_became_peered":"2024-02-18T00:46:25.883556+0000","last_unstale":"2024-02-18T00:48:36.794465+0000","last_undegraded":"2024-02-18T00:48:36.794465+0000","last_fullsized":"2024-02-18T00:48:36.794465+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:48:17.497210+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:48:17.497210+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292682+0000","last_change":"2024-02-18T00:48:11.198777+0000","last_active":"2024-02-18T00:48:36.292682+0000","last_peered":"2024-02-18T00:48:36.292682+0000","last_clean":"2024-02-18T00:48:36.292682+0000","last_became_active":"2024-02-18T00:46:25.880548+0000","last_became_peered":"2024-02-18T00:46:25.880548+0000","last_unstale":"2024-02-18T00:48:36.292682+0000","last_undegraded":"2024-02-18T00:48:36.292682+0000","last_fullsized":"2024-02-18T00:48:36.292682+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:11.198674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:11.198674+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.198674+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.23","version":"734'38","reported_seq":248,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377089+0000","last_change":"2024-02-18T00:48:23.565361+0000","last_active":"2024-02-18T00:48:36.377089+0000","last_peered":"2024-02-18T00:48:36.377089+0000","last_clean":"2024-02-18T00:48:36.377089+0000","last_became_active":"2024-02-18T00:43:09.796063+0000","last_became_peered":"2024-02-18T00:43:09.796063+0000","last_unstale":"2024-02-18T00:48:36.377089+0000","last_undegraded":"2024-02-18T00:48:36.377089+0000","last_fullsized":"2024-02-18T00:48:36.377089+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"734'38","last_scrub_stamp":"2024-02-18T00:48:23.565288+0000","last_deep_scrub":"734'38","last_deep_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_clean_scrub_stamp":"2024-02-18T00:48:23.565288+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":824,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407981+0000","last_change":"2024-02-18T00:48:01.333278+0000","last_active":"2024-02-18T00:48:36.407981+0000","last_peered":"2024-02-18T00:48:36.407981+0000","last_clean":"2024-02-18T00:48:36.407981+0000","last_became_active":"2024-02-18T00:46:25.870529+0000","last_became_peered":"2024-02-18T00:46:25.870529+0000","last_unstale":"2024-02-18T00:48:36.407981+0000","last_undegraded":"2024-02-18T00:48:36.407981+0000","last_fullsized":"2024-02-18T00:48:36.407981+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:48:01.333171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.333171+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1746,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.22","version":"457'32","reported_seq":735,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406594+0000","last_change":"2024-02-18T00:47:54.381756+0000","last_active":"2024-02-18T00:48:36.406594+0000","last_peered":"2024-02-18T00:48:36.406594+0000","last_clean":"2024-02-18T00:48:36.406594+0000","last_became_active":"2024-02-18T00:46:25.878558+0000","last_became_peered":"2024-02-18T00:46:25.878558+0000","last_unstale":"2024-02-18T00:48:36.406594+0000","last_undegraded":"2024-02-18T00:48:36.406594+0000","last_fullsized":"2024-02-18T00:48:36.406594+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:47:54.381655+0000","last_deep_scrub":"457'32","last_deep_scrub_stamp":"2024-02-18T00:47:54.381655+0000","last_clean_scrub_stamp":"2024-02-18T00:47:54.381655+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":66,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:42.562546+0000","last_change":"2024-02-18T00:48:42.562546+0000","last_active":"2024-02-18T00:48:42.562546+0000","last_peered":"2024-02-18T00:48:42.562546+0000","last_clean":"2024-02-18T00:48:42.562546+0000","last_became_active":"2024-02-18T00:46:25.888934+0000","last_became_peered":"2024-02-18T00:46:25.888934+0000","last_unstale":"2024-02-18T00:48:42.562546+0000","last_undegraded":"2024-02-18T00:48:42.562546+0000","last_fullsized":"2024-02-18T00:48:42.562546+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:42.562471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:42.562471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.25","version":"457'26","reported_seq":249,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794404+0000","last_change":"2024-02-18T00:47:20.580387+0000","last_active":"2024-02-18T00:48:36.794404+0000","last_peered":"2024-02-18T00:48:36.794404+0000","last_clean":"2024-02-18T00:48:36.794404+0000","last_became_active":"2024-02-18T00:43:09.791838+0000","last_became_peered":"2024-02-18T00:43:09.791838+0000","last_unstale":"2024-02-18T00:48:36.794404+0000","last_undegraded":"2024-02-18T00:48:36.794404+0000","last_fullsized":"2024-02-18T00:48:36.794404+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:47:20.580303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:20.580303+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":795,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.456562+0000","last_change":"2024-02-18T00:48:39.456562+0000","last_active":"2024-02-18T00:48:39.456562+0000","last_peered":"2024-02-18T00:48:39.456562+0000","last_clean":"2024-02-18T00:48:39.456562+0000","last_became_active":"2024-02-18T00:46:25.878862+0000","last_became_peered":"2024-02-18T00:46:25.878862+0000","last_unstale":"2024-02-18T00:48:39.456562+0000","last_undegraded":"2024-02-18T00:48:39.456562+0000","last_fullsized":"2024-02-18T00:48:39.456562+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:39.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.456491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"457'47","reported_seq":284,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293966+0000","last_change":"2024-02-18T00:47:37.745207+0000","last_active":"2024-02-18T00:48:36.293966+0000","last_peered":"2024-02-18T00:48:36.293966+0000","last_clean":"2024-02-18T00:48:36.293966+0000","last_became_active":"2024-02-18T00:43:09.789985+0000","last_became_peered":"2024-02-18T00:43:09.789985+0000","last_unstale":"2024-02-18T00:48:36.293966+0000","last_undegraded":"2024-02-18T00:48:36.293966+0000","last_fullsized":"2024-02-18T00:48:36.293966+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:47:37.745133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:37.745133+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":61,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795072+0000","last_change":"2024-02-18T00:46:53.398729+0000","last_active":"2024-02-18T00:48:36.795072+0000","last_peered":"2024-02-18T00:48:36.795072+0000","last_clean":"2024-02-18T00:48:36.795072+0000","last_became_active":"2024-02-18T00:46:25.893401+0000","last_became_peered":"2024-02-18T00:46:25.893401+0000","last_unstale":"2024-02-18T00:48:36.795072+0000","last_undegraded":"2024-02-18T00:48:36.795072+0000","last_fullsized":"2024-02-18T00:48:36.795072+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:46:53.398649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:46:53.398649+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"457'23","reported_seq":950,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292719+0000","last_change":"2024-02-18T00:47:38.765285+0000","last_active":"2024-02-18T00:48:36.292719+0000","last_peered":"2024-02-18T00:48:36.292719+0000","last_clean":"2024-02-18T00:48:36.292719+0000","last_became_active":"2024-02-18T00:46:25.877765+0000","last_became_peered":"2024-02-18T00:46:25.877765+0000","last_unstale":"2024-02-18T00:48:36.292719+0000","last_undegraded":"2024-02-18T00:48:36.292719+0000","last_fullsized":"2024-02-18T00:48:36.292719+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:47:38.765173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:38.765173+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":875,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292347+0000","last_change":"2024-02-18T00:48:27.755955+0000","last_active":"2024-02-18T00:48:36.292347+0000","last_peered":"2024-02-18T00:48:36.292347+0000","last_clean":"2024-02-18T00:48:36.292347+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:48:36.292347+0000","last_undegraded":"2024-02-18T00:48:36.292347+0000","last_fullsized":"2024-02-18T00:48:36.292347+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:27.755869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_clean_scrub_stamp":"2024-02-18T00:48:27.755869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"855'27","reported_seq":742,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.620048+0000","last_change":"2024-02-18T00:48:39.620048+0000","last_active":"2024-02-18T00:48:39.620048+0000","last_peered":"2024-02-18T00:48:39.620048+0000","last_clean":"2024-02-18T00:48:39.620048+0000","last_became_active":"2024-02-18T00:46:25.878258+0000","last_became_peered":"2024-02-18T00:46:25.878258+0000","last_unstale":"2024-02-18T00:48:39.620048+0000","last_undegraded":"2024-02-18T00:48:39.620048+0000","last_fullsized":"2024-02-18T00:48:39.620048+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"855'27","last_scrub_stamp":"2024-02-18T00:48:39.619977+0000","last_deep_scrub":"855'27","last_deep_scrub_stamp":"2024-02-18T00:48:39.619977+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.619977+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291527+0000","last_change":"2024-02-18T00:48:13.227039+0000","last_active":"2024-02-18T00:48:36.291527+0000","last_peered":"2024-02-18T00:48:36.291527+0000","last_clean":"2024-02-18T00:48:36.291527+0000","last_became_active":"2024-02-18T00:46:25.886018+0000","last_became_peered":"2024-02-18T00:46:25.886018+0000","last_unstale":"2024-02-18T00:48:36.291527+0000","last_undegraded":"2024-02-18T00:48:36.291527+0000","last_fullsized":"2024-02-18T00:48:36.291527+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:13.226976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:48:13.226976+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":800,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793950+0000","last_change":"2024-02-18T00:47:14.154392+0000","last_active":"2024-02-18T00:48:36.793950+0000","last_peered":"2024-02-18T00:48:36.793950+0000","last_clean":"2024-02-18T00:48:36.793950+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:48:36.793950+0000","last_undegraded":"2024-02-18T00:48:36.793950+0000","last_fullsized":"2024-02-18T00:48:36.793950+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:14.154332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_clean_scrub_stamp":"2024-02-18T00:47:14.154332+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"811'13","reported_seq":549,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.004619+0000","last_change":"2024-02-18T00:48:12.217031+0000","last_active":"2024-02-18T00:48:39.004619+0000","last_peered":"2024-02-18T00:48:39.004619+0000","last_clean":"2024-02-18T00:48:39.004619+0000","last_became_active":"2024-02-18T00:46:25.884361+0000","last_became_peered":"2024-02-18T00:46:25.884361+0000","last_unstale":"2024-02-18T00:48:39.004619+0000","last_undegraded":"2024-02-18T00:48:39.004619+0000","last_fullsized":"2024-02-18T00:48:39.004619+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"811'13","last_scrub_stamp":"2024-02-18T00:48:12.216946+0000","last_deep_scrub":"736'11","last_deep_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.216946+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":1016,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793941+0000","last_change":"2024-02-18T00:47:27.585765+0000","last_active":"2024-02-18T00:48:36.793941+0000","last_peered":"2024-02-18T00:48:36.793941+0000","last_clean":"2024-02-18T00:48:36.793941+0000","last_became_active":"2024-02-18T00:46:25.878115+0000","last_became_peered":"2024-02-18T00:46:25.878115+0000","last_unstale":"2024-02-18T00:48:36.793941+0000","last_undegraded":"2024-02-18T00:48:36.793941+0000","last_fullsized":"2024-02-18T00:48:36.793941+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:47:27.585659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:27.585659+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":868,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293409+0000","last_change":"2024-02-18T00:48:30.801362+0000","last_active":"2024-02-18T00:48:36.293409+0000","last_peered":"2024-02-18T00:48:36.293409+0000","last_clean":"2024-02-18T00:48:36.293409+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:48:36.293409+0000","last_undegraded":"2024-02-18T00:48:36.293409+0000","last_fullsized":"2024-02-18T00:48:36.293409+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:30.801291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:30.801291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":870,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794849+0000","last_change":"2024-02-18T00:47:39.735766+0000","last_active":"2024-02-18T00:48:36.794849+0000","last_peered":"2024-02-18T00:48:36.794849+0000","last_clean":"2024-02-18T00:48:36.794849+0000","last_became_active":"2024-02-18T00:46:25.879142+0000","last_became_peered":"2024-02-18T00:46:25.879142+0000","last_unstale":"2024-02-18T00:48:36.794849+0000","last_undegraded":"2024-02-18T00:48:36.794849+0000","last_fullsized":"2024-02-18T00:48:36.794849+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:39.735677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:39.735677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":790,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292210+0000","last_change":"2024-02-18T00:48:18.800902+0000","last_active":"2024-02-18T00:48:36.292210+0000","last_peered":"2024-02-18T00:48:36.292210+0000","last_clean":"2024-02-18T00:48:36.292210+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:48:36.292210+0000","last_undegraded":"2024-02-18T00:48:36.292210+0000","last_fullsized":"2024-02-18T00:48:36.292210+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:18.800826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.800826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":903,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.138531+0000","last_change":"2024-02-18T00:48:43.138531+0000","last_active":"2024-02-18T00:48:43.138531+0000","last_peered":"2024-02-18T00:48:43.138531+0000","last_clean":"2024-02-18T00:48:43.138531+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:48:43.138531+0000","last_undegraded":"2024-02-18T00:48:43.138531+0000","last_fullsized":"2024-02-18T00:48:43.138531+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:48:43.138469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.138469+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292928+0000","last_change":"2024-02-18T00:47:35.733560+0000","last_active":"2024-02-18T00:48:36.292928+0000","last_peered":"2024-02-18T00:48:36.292928+0000","last_clean":"2024-02-18T00:48:36.292928+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:48:36.292928+0000","last_undegraded":"2024-02-18T00:48:36.292928+0000","last_fullsized":"2024-02-18T00:48:36.292928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:35.733490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:47:35.733490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":844,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293480+0000","last_change":"2024-02-18T00:48:25.759864+0000","last_active":"2024-02-18T00:48:36.293480+0000","last_peered":"2024-02-18T00:48:36.293480+0000","last_clean":"2024-02-18T00:48:36.293480+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:48:36.293480+0000","last_undegraded":"2024-02-18T00:48:36.293480+0000","last_fullsized":"2024-02-18T00:48:36.293480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:25.759786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:25.759786+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":889,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793669+0000","last_change":"2024-02-18T00:47:25.113306+0000","last_active":"2024-02-18T00:48:36.793669+0000","last_peered":"2024-02-18T00:48:36.793669+0000","last_clean":"2024-02-18T00:48:36.793669+0000","last_became_active":"2024-02-18T00:43:35.060523+0000","last_became_peered":"2024-02-18T00:43:35.060523+0000","last_unstale":"2024-02-18T00:48:36.793669+0000","last_undegraded":"2024-02-18T00:48:36.793669+0000","last_fullsized":"2024-02-18T00:48:36.793669+0000","mapping_epoch":668,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":669,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:25.113237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:25.113237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":792,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.378015+0000","last_change":"2024-02-18T00:47:59.708632+0000","last_active":"2024-02-18T00:48:36.378015+0000","last_peered":"2024-02-18T00:48:36.378015+0000","last_clean":"2024-02-18T00:48:36.378015+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:48:36.378015+0000","last_undegraded":"2024-02-18T00:48:36.378015+0000","last_fullsized":"2024-02-18T00:48:36.378015+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:59.708561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:47:59.708561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"840'52","reported_seq":93,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795251+0000","last_change":"2024-02-18T00:47:32.441781+0000","last_active":"2024-02-18T00:48:36.795251+0000","last_peered":"2024-02-18T00:48:36.795251+0000","last_clean":"2024-02-18T00:48:36.795251+0000","last_became_active":"2024-02-18T00:46:25.888711+0000","last_became_peered":"2024-02-18T00:46:25.888711+0000","last_unstale":"2024-02-18T00:48:36.795251+0000","last_undegraded":"2024-02-18T00:48:36.795251+0000","last_fullsized":"2024-02-18T00:48:36.795251+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"840'52","last_scrub_stamp":"2024-02-18T00:47:32.441696+0000","last_deep_scrub":"840'52","last_deep_scrub_stamp":"2024-02-18T00:47:32.441696+0000","last_clean_scrub_stamp":"2024-02-18T00:47:32.441696+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292773+0000","last_change":"2024-02-18T00:48:10.632366+0000","last_active":"2024-02-18T00:48:36.292773+0000","last_peered":"2024-02-18T00:48:36.292773+0000","last_clean":"2024-02-18T00:48:36.292773+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:48:36.292773+0000","last_undegraded":"2024-02-18T00:48:36.292773+0000","last_fullsized":"2024-02-18T00:48:36.292773+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:10.632284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:48:10.632284+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":844,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:37.960417+0000","last_change":"2024-02-18T00:48:37.960387+0000","last_active":"2024-02-18T00:48:37.960417+0000","last_peered":"2024-02-18T00:48:37.960417+0000","last_clean":"2024-02-18T00:48:37.960417+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:48:37.960417+0000","last_undegraded":"2024-02-18T00:48:37.960417+0000","last_fullsized":"2024-02-18T00:48:37.960417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:37.960319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:37.960319+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291697+0000","last_change":"2024-02-18T00:47:57.184768+0000","last_active":"2024-02-18T00:48:36.291697+0000","last_peered":"2024-02-18T00:48:36.291697+0000","last_clean":"2024-02-18T00:48:36.291697+0000","last_became_active":"2024-02-18T00:41:53.837106+0000","last_became_peered":"2024-02-18T00:41:53.837106+0000","last_unstale":"2024-02-18T00:48:36.291697+0000","last_undegraded":"2024-02-18T00:48:36.291697+0000","last_fullsized":"2024-02-18T00:48:36.291697+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:57.184677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:57.184677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794054+0000","last_change":"2024-02-18T00:48:19.861719+0000","last_active":"2024-02-18T00:48:36.794054+0000","last_peered":"2024-02-18T00:48:36.794054+0000","last_clean":"2024-02-18T00:48:36.794054+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:48:36.794054+0000","last_undegraded":"2024-02-18T00:48:36.794054+0000","last_fullsized":"2024-02-18T00:48:36.794054+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:19.861637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.861637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":907,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:45.808687+0000","last_change":"2024-02-18T00:48:45.808687+0000","last_active":"2024-02-18T00:48:45.808687+0000","last_peered":"2024-02-18T00:48:45.808687+0000","last_clean":"2024-02-18T00:48:45.808687+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:48:45.808687+0000","last_undegraded":"2024-02-18T00:48:45.808687+0000","last_fullsized":"2024-02-18T00:48:45.808687+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:48:45.808587+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:48:45.808587+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291974+0000","last_change":"2024-02-18T00:48:36.050679+0000","last_active":"2024-02-18T00:48:36.291974+0000","last_peered":"2024-02-18T00:48:36.291974+0000","last_clean":"2024-02-18T00:48:36.291974+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:48:36.291974+0000","last_undegraded":"2024-02-18T00:48:36.291974+0000","last_fullsized":"2024-02-18T00:48:36.291974+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:36.050601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:48:36.050601+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":844,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.294159+0000","last_change":"2024-02-18T00:48:01.725656+0000","last_active":"2024-02-18T00:48:36.294159+0000","last_peered":"2024-02-18T00:48:36.294159+0000","last_clean":"2024-02-18T00:48:36.294159+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:48:36.294159+0000","last_undegraded":"2024-02-18T00:48:36.294159+0000","last_fullsized":"2024-02-18T00:48:36.294159+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:01.725572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.725572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":855,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293265+0000","last_change":"2024-02-18T00:48:00.851863+0000","last_active":"2024-02-18T00:48:36.293265+0000","last_peered":"2024-02-18T00:48:36.293265+0000","last_clean":"2024-02-18T00:48:36.293265+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:48:36.293265+0000","last_undegraded":"2024-02-18T00:48:36.293265+0000","last_fullsized":"2024-02-18T00:48:36.293265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:00.851789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:31.822090+0000","last_clean_scrub_stamp":"2024-02-18T00:48:00.851789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":254,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793443+0000","last_change":"2024-02-18T00:48:09.032408+0000","last_active":"2024-02-18T00:48:36.793443+0000","last_peered":"2024-02-18T00:48:36.793443+0000","last_clean":"2024-02-18T00:48:36.793443+0000","last_became_active":"2024-02-18T00:42:38.553174+0000","last_became_peered":"2024-02-18T00:42:38.553174+0000","last_unstale":"2024-02-18T00:48:36.793443+0000","last_undegraded":"2024-02-18T00:48:36.793443+0000","last_fullsized":"2024-02-18T00:48:36.793443+0000","mapping_epoch":623,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.032334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.032334+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":936,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407475+0000","last_change":"2024-02-18T00:48:18.279326+0000","last_active":"2024-02-18T00:48:36.407475+0000","last_peered":"2024-02-18T00:48:36.407475+0000","last_clean":"2024-02-18T00:48:36.407475+0000","last_became_active":"2024-02-18T00:46:25.870141+0000","last_became_peered":"2024-02-18T00:46:25.870141+0000","last_unstale":"2024-02-18T00:48:36.407475+0000","last_undegraded":"2024-02-18T00:48:36.407475+0000","last_fullsized":"2024-02-18T00:48:36.407475+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:48:18.279247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.279247+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":868,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293534+0000","last_change":"2024-02-18T00:47:54.865456+0000","last_active":"2024-02-18T00:48:36.293534+0000","last_peered":"2024-02-18T00:48:36.293534+0000","last_clean":"2024-02-18T00:48:36.293534+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:48:36.293534+0000","last_undegraded":"2024-02-18T00:48:36.293534+0000","last_fullsized":"2024-02-18T00:48:36.293534+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:54.865338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:47:54.865338+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":1112,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.083237+0000","last_change":"2024-02-18T00:47:29.693566+0000","last_active":"2024-02-18T00:48:39.083237+0000","last_peered":"2024-02-18T00:48:39.083237+0000","last_clean":"2024-02-18T00:48:39.083237+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:48:39.083237+0000","last_undegraded":"2024-02-18T00:48:39.083237+0000","last_fullsized":"2024-02-18T00:48:39.083237+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:47:29.693497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:47:29.693497+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":857,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407835+0000","last_change":"2024-02-18T00:48:35.511782+0000","last_active":"2024-02-18T00:48:36.407835+0000","last_peered":"2024-02-18T00:48:36.407835+0000","last_clean":"2024-02-18T00:48:36.407835+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:48:36.407835+0000","last_undegraded":"2024-02-18T00:48:36.407835+0000","last_fullsized":"2024-02-18T00:48:36.407835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:35.511683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:35.511683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":806,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293842+0000","last_change":"2024-02-18T00:47:53.855296+0000","last_active":"2024-02-18T00:48:36.293842+0000","last_peered":"2024-02-18T00:48:36.293842+0000","last_clean":"2024-02-18T00:48:36.293842+0000","last_became_active":"2024-02-18T00:46:25.878475+0000","last_became_peered":"2024-02-18T00:46:25.878475+0000","last_unstale":"2024-02-18T00:48:36.293842+0000","last_undegraded":"2024-02-18T00:48:36.293842+0000","last_fullsized":"2024-02-18T00:48:36.293842+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:53.855222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:47:53.855222+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":864,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795289+0000","last_change":"2024-02-18T00:48:18.823350+0000","last_active":"2024-02-18T00:48:36.795289+0000","last_peered":"2024-02-18T00:48:36.795289+0000","last_clean":"2024-02-18T00:48:36.795289+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:48:36.795289+0000","last_undegraded":"2024-02-18T00:48:36.795289+0000","last_fullsized":"2024-02-18T00:48:36.795289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:48:18.823241+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.823241+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":875,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293481+0000","last_change":"2024-02-18T00:47:52.807880+0000","last_active":"2024-02-18T00:48:36.293481+0000","last_peered":"2024-02-18T00:48:36.293481+0000","last_clean":"2024-02-18T00:48:36.293481+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:48:36.293481+0000","last_undegraded":"2024-02-18T00:48:36.293481+0000","last_fullsized":"2024-02-18T00:48:36.293481+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:52.807803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.807803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":1083,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.883678+0000","last_change":"2024-02-18T00:48:30.464109+0000","last_active":"2024-02-18T00:48:38.883678+0000","last_peered":"2024-02-18T00:48:38.883678+0000","last_clean":"2024-02-18T00:48:38.883678+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:48:38.883678+0000","last_undegraded":"2024-02-18T00:48:38.883678+0000","last_fullsized":"2024-02-18T00:48:38.883678+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:48:30.463987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:30.463987+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":852,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795233+0000","last_change":"2024-02-18T00:48:32.900019+0000","last_active":"2024-02-18T00:48:36.795233+0000","last_peered":"2024-02-18T00:48:36.795233+0000","last_clean":"2024-02-18T00:48:36.795233+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:48:36.795233+0000","last_undegraded":"2024-02-18T00:48:36.795233+0000","last_fullsized":"2024-02-18T00:48:36.795233+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.899942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.899942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406804+0000","last_change":"2024-02-18T00:47:37.477135+0000","last_active":"2024-02-18T00:48:36.406804+0000","last_peered":"2024-02-18T00:48:36.406804+0000","last_clean":"2024-02-18T00:48:36.406804+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:48:36.406804+0000","last_undegraded":"2024-02-18T00:48:36.406804+0000","last_fullsized":"2024-02-18T00:48:36.406804+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:37.477044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:47:37.477044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"849'31","reported_seq":964,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407752+0000","last_change":"2024-02-18T00:48:23.389365+0000","last_active":"2024-02-18T00:48:36.407752+0000","last_peered":"2024-02-18T00:48:36.407752+0000","last_clean":"2024-02-18T00:48:36.407752+0000","last_became_active":"2024-02-18T00:46:25.878579+0000","last_became_peered":"2024-02-18T00:46:25.878579+0000","last_unstale":"2024-02-18T00:48:36.407752+0000","last_undegraded":"2024-02-18T00:48:36.407752+0000","last_fullsized":"2024-02-18T00:48:36.407752+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"849'31","last_scrub_stamp":"2024-02-18T00:48:23.389255+0000","last_deep_scrub":"849'31","last_deep_scrub_stamp":"2024-02-18T00:48:23.389255+0000","last_clean_scrub_stamp":"2024-02-18T00:48:23.389255+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":416,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291705+0000","last_change":"2024-02-18T00:48:29.752540+0000","last_active":"2024-02-18T00:48:36.291705+0000","last_peered":"2024-02-18T00:48:36.291705+0000","last_clean":"2024-02-18T00:48:36.291705+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:48:36.291705+0000","last_undegraded":"2024-02-18T00:48:36.291705+0000","last_fullsized":"2024-02-18T00:48:36.291705+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:29.752478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:29.752478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":859,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.890654+0000","last_change":"2024-02-18T00:48:36.890622+0000","last_active":"2024-02-18T00:48:36.890654+0000","last_peered":"2024-02-18T00:48:36.890654+0000","last_clean":"2024-02-18T00:48:36.890654+0000","last_became_active":"2024-02-18T00:46:25.878159+0000","last_became_peered":"2024-02-18T00:46:25.878159+0000","last_unstale":"2024-02-18T00:48:36.890654+0000","last_undegraded":"2024-02-18T00:48:36.890654+0000","last_fullsized":"2024-02-18T00:48:36.890654+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:36.890531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:48:36.890531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":900,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377262+0000","last_change":"2024-02-18T00:47:48.713645+0000","last_active":"2024-02-18T00:48:36.377262+0000","last_peered":"2024-02-18T00:48:36.377262+0000","last_clean":"2024-02-18T00:48:36.377262+0000","last_became_active":"2024-02-18T00:46:28.878485+0000","last_became_peered":"2024-02-18T00:46:28.878485+0000","last_unstale":"2024-02-18T00:48:36.377262+0000","last_undegraded":"2024-02-18T00:48:36.377262+0000","last_fullsized":"2024-02-18T00:48:36.377262+0000","mapping_epoch":811,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":812,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:47:48.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:48.713569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407615+0000","last_change":"2024-02-18T00:47:13.490760+0000","last_active":"2024-02-18T00:48:36.407615+0000","last_peered":"2024-02-18T00:48:36.407615+0000","last_clean":"2024-02-18T00:48:36.407615+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:48:36.407615+0000","last_undegraded":"2024-02-18T00:48:36.407615+0000","last_fullsized":"2024-02-18T00:48:36.407615+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:13.490673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:47:13.490673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":1073,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.884791+0000","last_change":"2024-02-18T00:47:39.653669+0000","last_active":"2024-02-18T00:48:38.884791+0000","last_peered":"2024-02-18T00:48:38.884791+0000","last_clean":"2024-02-18T00:48:38.884791+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:48:38.884791+0000","last_undegraded":"2024-02-18T00:48:38.884791+0000","last_fullsized":"2024-02-18T00:48:38.884791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:47:39.653587+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_clean_scrub_stamp":"2024-02-18T00:47:39.653587+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":948,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406628+0000","last_change":"2024-02-18T00:48:03.301641+0000","last_active":"2024-02-18T00:48:36.406628+0000","last_peered":"2024-02-18T00:48:36.406628+0000","last_clean":"2024-02-18T00:48:36.406628+0000","last_became_active":"2024-02-18T00:46:25.870910+0000","last_became_peered":"2024-02-18T00:46:25.870910+0000","last_unstale":"2024-02-18T00:48:36.406628+0000","last_undegraded":"2024-02-18T00:48:36.406628+0000","last_fullsized":"2024-02-18T00:48:36.406628+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:48:03.301556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:03.301556+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795084+0000","last_change":"2024-02-18T00:48:31.612121+0000","last_active":"2024-02-18T00:48:36.795084+0000","last_peered":"2024-02-18T00:48:36.795084+0000","last_clean":"2024-02-18T00:48:36.795084+0000","last_became_active":"2024-02-18T00:46:25.889388+0000","last_became_peered":"2024-02-18T00:46:25.889388+0000","last_unstale":"2024-02-18T00:48:36.795084+0000","last_undegraded":"2024-02-18T00:48:36.795084+0000","last_fullsized":"2024-02-18T00:48:36.795084+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:31.612041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:48:31.612041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292030+0000","last_change":"2024-02-18T00:47:42.085673+0000","last_active":"2024-02-18T00:48:36.292030+0000","last_peered":"2024-02-18T00:48:36.292030+0000","last_clean":"2024-02-18T00:48:36.292030+0000","last_became_active":"2024-02-18T00:46:25.877376+0000","last_became_peered":"2024-02-18T00:46:25.877376+0000","last_unstale":"2024-02-18T00:48:36.292030+0000","last_undegraded":"2024-02-18T00:48:36.292030+0000","last_fullsized":"2024-02-18T00:48:36.292030+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:42.085586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:42.085586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794493+0000","last_change":"2024-02-18T00:48:15.739183+0000","last_active":"2024-02-18T00:48:36.794493+0000","last_peered":"2024-02-18T00:48:36.794493+0000","last_clean":"2024-02-18T00:48:36.794493+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:48:36.794493+0000","last_undegraded":"2024-02-18T00:48:36.794493+0000","last_fullsized":"2024-02-18T00:48:36.794493+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:15.739057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:38.563638+0000","last_clean_scrub_stamp":"2024-02-18T00:48:15.739057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":859,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406851+0000","last_change":"2024-02-18T00:48:00.330785+0000","last_active":"2024-02-18T00:48:36.406851+0000","last_peered":"2024-02-18T00:48:36.406851+0000","last_clean":"2024-02-18T00:48:36.406851+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:48:36.406851+0000","last_undegraded":"2024-02-18T00:48:36.406851+0000","last_fullsized":"2024-02-18T00:48:36.406851+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:00.330686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:45.974334+0000","last_clean_scrub_stamp":"2024-02-18T00:48:00.330686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":999,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377028+0000","last_change":"2024-02-18T00:48:03.689371+0000","last_active":"2024-02-18T00:48:36.377028+0000","last_peered":"2024-02-18T00:48:36.377028+0000","last_clean":"2024-02-18T00:48:36.377028+0000","last_became_active":"2024-02-18T00:43:07.799144+0000","last_became_peered":"2024-02-18T00:43:07.799144+0000","last_unstale":"2024-02-18T00:48:36.377028+0000","last_undegraded":"2024-02-18T00:48:36.377028+0000","last_fullsized":"2024-02-18T00:48:36.377028+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:48:03.689295+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:48:03.689295+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":893,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292292+0000","last_change":"2024-02-18T00:47:30.836684+0000","last_active":"2024-02-18T00:48:36.292292+0000","last_peered":"2024-02-18T00:48:36.292292+0000","last_clean":"2024-02-18T00:48:36.292292+0000","last_became_active":"2024-02-18T00:46:25.877757+0000","last_became_peered":"2024-02-18T00:46:25.877757+0000","last_unstale":"2024-02-18T00:48:36.292292+0000","last_undegraded":"2024-02-18T00:48:36.292292+0000","last_fullsized":"2024-02-18T00:48:36.292292+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:30.836608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:47:30.836608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":839,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407059+0000","last_change":"2024-02-18T00:48:33.475054+0000","last_active":"2024-02-18T00:48:36.407059+0000","last_peered":"2024-02-18T00:48:36.407059+0000","last_clean":"2024-02-18T00:48:36.407059+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:48:36.407059+0000","last_undegraded":"2024-02-18T00:48:36.407059+0000","last_fullsized":"2024-02-18T00:48:36.407059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:33.474946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:33.474946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":774,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377924+0000","last_change":"2024-02-18T00:48:10.657179+0000","last_active":"2024-02-18T00:48:36.377924+0000","last_peered":"2024-02-18T00:48:36.377924+0000","last_clean":"2024-02-18T00:48:36.377924+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:48:36.377924+0000","last_undegraded":"2024-02-18T00:48:36.377924+0000","last_fullsized":"2024-02-18T00:48:36.377924+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:10.657084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:10.657084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795154+0000","last_change":"2024-02-18T00:48:16.538979+0000","last_active":"2024-02-18T00:48:36.795154+0000","last_peered":"2024-02-18T00:48:36.795154+0000","last_clean":"2024-02-18T00:48:36.795154+0000","last_became_active":"2024-02-18T00:46:25.887171+0000","last_became_peered":"2024-02-18T00:46:25.887171+0000","last_unstale":"2024-02-18T00:48:36.795154+0000","last_undegraded":"2024-02-18T00:48:36.795154+0000","last_fullsized":"2024-02-18T00:48:36.795154+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:16.538896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:16.538896+0000","last_clean_scrub_stamp":"2024-02-18T00:48:16.538896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":976,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793840+0000","last_change":"2024-02-18T00:47:42.096731+0000","last_active":"2024-02-18T00:48:36.793840+0000","last_peered":"2024-02-18T00:48:36.793840+0000","last_clean":"2024-02-18T00:48:36.793840+0000","last_became_active":"2024-02-18T00:43:07.797505+0000","last_became_peered":"2024-02-18T00:43:07.797505+0000","last_unstale":"2024-02-18T00:48:36.793840+0000","last_undegraded":"2024-02-18T00:48:36.793840+0000","last_fullsized":"2024-02-18T00:48:36.793840+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:47:42.096655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:42.096655+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":416,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.980461+0000","last_change":"2024-02-18T00:48:39.980461+0000","last_active":"2024-02-18T00:48:39.980461+0000","last_peered":"2024-02-18T00:48:39.980461+0000","last_clean":"2024-02-18T00:48:39.980461+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:48:39.980461+0000","last_undegraded":"2024-02-18T00:48:39.980461+0000","last_fullsized":"2024-02-18T00:48:39.980461+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:48:39.980392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.980392+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":860,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293561+0000","last_change":"2024-02-18T00:47:40.776739+0000","last_active":"2024-02-18T00:48:36.293561+0000","last_peered":"2024-02-18T00:48:36.293561+0000","last_clean":"2024-02-18T00:48:36.293561+0000","last_became_active":"2024-02-18T00:46:25.878733+0000","last_became_peered":"2024-02-18T00:46:25.878733+0000","last_unstale":"2024-02-18T00:48:36.293561+0000","last_undegraded":"2024-02-18T00:48:36.293561+0000","last_fullsized":"2024-02-18T00:48:36.293561+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:40.776651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:31.693369+0000","last_clean_scrub_stamp":"2024-02-18T00:47:40.776651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":774,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377670+0000","last_change":"2024-02-18T00:48:22.556490+0000","last_active":"2024-02-18T00:48:36.377670+0000","last_peered":"2024-02-18T00:48:36.377670+0000","last_clean":"2024-02-18T00:48:36.377670+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:48:36.377670+0000","last_undegraded":"2024-02-18T00:48:36.377670+0000","last_fullsized":"2024-02-18T00:48:36.377670+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:22.556417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:22.556417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":836,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795580+0000","last_change":"2024-02-18T00:47:40.766038+0000","last_active":"2024-02-18T00:48:36.795580+0000","last_peered":"2024-02-18T00:48:36.795580+0000","last_clean":"2024-02-18T00:48:36.795580+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:48:36.795580+0000","last_undegraded":"2024-02-18T00:48:36.795580+0000","last_fullsized":"2024-02-18T00:48:36.795580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:40.765955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:34.575174+0000","last_clean_scrub_stamp":"2024-02-18T00:47:40.765955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"847'27","reported_seq":891,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794160+0000","last_change":"2024-02-18T00:48:01.032079+0000","last_active":"2024-02-18T00:48:36.794160+0000","last_peered":"2024-02-18T00:48:36.794160+0000","last_clean":"2024-02-18T00:48:36.794160+0000","last_became_active":"2024-02-18T00:43:07.799985+0000","last_became_peered":"2024-02-18T00:43:07.799985+0000","last_unstale":"2024-02-18T00:48:36.794160+0000","last_undegraded":"2024-02-18T00:48:36.794160+0000","last_fullsized":"2024-02-18T00:48:36.794160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"847'27","last_scrub_stamp":"2024-02-18T00:48:01.032008+0000","last_deep_scrub":"847'27","last_deep_scrub_stamp":"2024-02-18T00:48:01.032008+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.032008+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":411,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291711+0000","last_change":"2024-02-18T00:48:02.800420+0000","last_active":"2024-02-18T00:48:36.291711+0000","last_peered":"2024-02-18T00:48:36.291711+0000","last_clean":"2024-02-18T00:48:36.291711+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:48:36.291711+0000","last_undegraded":"2024-02-18T00:48:36.291711+0000","last_fullsized":"2024-02-18T00:48:36.291711+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:02.800351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:33.866668+0000","last_clean_scrub_stamp":"2024-02-18T00:48:02.800351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":843,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793553+0000","last_change":"2024-02-18T00:48:16.064808+0000","last_active":"2024-02-18T00:48:36.793553+0000","last_peered":"2024-02-18T00:48:36.793553+0000","last_clean":"2024-02-18T00:48:36.793553+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:48:36.793553+0000","last_undegraded":"2024-02-18T00:48:36.793553+0000","last_fullsized":"2024-02-18T00:48:36.793553+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:16.064744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:48:16.064744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":842,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.294206+0000","last_change":"2024-02-18T00:48:30.734581+0000","last_active":"2024-02-18T00:48:36.294206+0000","last_peered":"2024-02-18T00:48:36.294206+0000","last_clean":"2024-02-18T00:48:36.294206+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:48:36.294206+0000","last_undegraded":"2024-02-18T00:48:36.294206+0000","last_fullsized":"2024-02-18T00:48:36.294206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:30.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_clean_scrub_stamp":"2024-02-18T00:48:30.734515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":273,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.913999+0000","last_change":"2024-02-18T00:48:36.913957+0000","last_active":"2024-02-18T00:48:36.913999+0000","last_peered":"2024-02-18T00:48:36.913999+0000","last_clean":"2024-02-18T00:48:36.913999+0000","last_became_active":"2024-02-18T00:46:25.876777+0000","last_became_peered":"2024-02-18T00:46:25.876777+0000","last_unstale":"2024-02-18T00:48:36.913999+0000","last_undegraded":"2024-02-18T00:48:36.913999+0000","last_fullsized":"2024-02-18T00:48:36.913999+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:36.913850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:36.913850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292392+0000","last_change":"2024-02-18T00:48:01.102950+0000","last_active":"2024-02-18T00:48:36.292392+0000","last_peered":"2024-02-18T00:48:36.292392+0000","last_clean":"2024-02-18T00:48:36.292392+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:48:36.292392+0000","last_undegraded":"2024-02-18T00:48:36.292392+0000","last_fullsized":"2024-02-18T00:48:36.292392+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:01.102866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.102866+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"853'29","reported_seq":77,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794611+0000","last_change":"2024-02-18T00:48:34.518432+0000","last_active":"2024-02-18T00:48:36.794611+0000","last_peered":"2024-02-18T00:48:36.794611+0000","last_clean":"2024-02-18T00:48:36.794611+0000","last_became_active":"2024-02-18T00:46:25.887464+0000","last_became_peered":"2024-02-18T00:46:25.887464+0000","last_unstale":"2024-02-18T00:48:36.794611+0000","last_undegraded":"2024-02-18T00:48:36.794611+0000","last_fullsized":"2024-02-18T00:48:36.794611+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"853'29","last_scrub_stamp":"2024-02-18T00:48:34.518347+0000","last_deep_scrub":"853'29","last_deep_scrub_stamp":"2024-02-18T00:48:34.518347+0000","last_clean_scrub_stamp":"2024-02-18T00:48:34.518347+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291550+0000","last_change":"2024-02-18T00:47:58.147390+0000","last_active":"2024-02-18T00:48:36.291550+0000","last_peered":"2024-02-18T00:48:36.291550+0000","last_clean":"2024-02-18T00:48:36.291550+0000","last_became_active":"2024-02-18T00:46:25.884860+0000","last_became_peered":"2024-02-18T00:46:25.884860+0000","last_unstale":"2024-02-18T00:48:36.291550+0000","last_undegraded":"2024-02-18T00:48:36.291550+0000","last_fullsized":"2024-02-18T00:48:36.291550+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:58.147307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_clean_scrub_stamp":"2024-02-18T00:47:58.147307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":881,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795539+0000","last_change":"2024-02-18T00:48:22.881947+0000","last_active":"2024-02-18T00:48:36.795539+0000","last_peered":"2024-02-18T00:48:36.795539+0000","last_clean":"2024-02-18T00:48:36.795539+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:48:36.795539+0000","last_undegraded":"2024-02-18T00:48:36.795539+0000","last_fullsized":"2024-02-18T00:48:36.795539+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:22.881869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:22.881869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":851,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.372131+0000","last_change":"2024-02-18T00:48:43.372131+0000","last_active":"2024-02-18T00:48:43.372131+0000","last_peered":"2024-02-18T00:48:43.372131+0000","last_clean":"2024-02-18T00:48:43.372131+0000","last_became_active":"2024-02-18T00:46:25.885092+0000","last_became_peered":"2024-02-18T00:46:25.885092+0000","last_unstale":"2024-02-18T00:48:43.372131+0000","last_undegraded":"2024-02-18T00:48:43.372131+0000","last_fullsized":"2024-02-18T00:48:43.372131+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:43.372061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:43.372061+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.372061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":852,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795594+0000","last_change":"2024-02-18T00:47:18.590533+0000","last_active":"2024-02-18T00:48:36.795594+0000","last_peered":"2024-02-18T00:48:36.795594+0000","last_clean":"2024-02-18T00:48:36.795594+0000","last_became_active":"2024-02-18T00:41:57.791956+0000","last_became_peered":"2024-02-18T00:41:57.791956+0000","last_unstale":"2024-02-18T00:48:36.795594+0000","last_undegraded":"2024-02-18T00:48:36.795594+0000","last_fullsized":"2024-02-18T00:48:36.795594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":584,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:18.590439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:47:18.590439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":785,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793677+0000","last_change":"2024-02-18T00:48:18.125386+0000","last_active":"2024-02-18T00:48:36.793677+0000","last_peered":"2024-02-18T00:48:36.793677+0000","last_clean":"2024-02-18T00:48:36.793677+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:48:36.793677+0000","last_undegraded":"2024-02-18T00:48:36.793677+0000","last_fullsized":"2024-02-18T00:48:36.793677+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:18.125320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.125320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":862,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293535+0000","last_change":"2024-02-18T00:48:04.783601+0000","last_active":"2024-02-18T00:48:36.293535+0000","last_peered":"2024-02-18T00:48:36.293535+0000","last_clean":"2024-02-18T00:48:36.293535+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:48:36.293535+0000","last_undegraded":"2024-02-18T00:48:36.293535+0000","last_fullsized":"2024-02-18T00:48:36.293535+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:04.783526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:04.783526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"680'27","reported_seq":910,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292920+0000","last_change":"2024-02-18T00:47:18.816250+0000","last_active":"2024-02-18T00:48:36.292920+0000","last_peered":"2024-02-18T00:48:36.292920+0000","last_clean":"2024-02-18T00:48:36.292920+0000","last_became_active":"2024-02-18T00:43:07.801362+0000","last_became_peered":"2024-02-18T00:43:07.801362+0000","last_unstale":"2024-02-18T00:48:36.292920+0000","last_undegraded":"2024-02-18T00:48:36.292920+0000","last_fullsized":"2024-02-18T00:48:36.292920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"680'27","last_scrub_stamp":"2024-02-18T00:47:18.816173+0000","last_deep_scrub":"680'27","last_deep_scrub_stamp":"2024-02-18T00:47:18.816173+0000","last_clean_scrub_stamp":"2024-02-18T00:47:18.816173+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":857,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292025+0000","last_change":"2024-02-18T00:48:32.796392+0000","last_active":"2024-02-18T00:48:36.292025+0000","last_peered":"2024-02-18T00:48:36.292025+0000","last_clean":"2024-02-18T00:48:36.292025+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:48:36.292025+0000","last_undegraded":"2024-02-18T00:48:36.292025+0000","last_fullsized":"2024-02-18T00:48:36.292025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.796325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.796325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":2041,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:44.122935+0000","last_change":"2024-02-18T00:47:57.808688+0000","last_active":"2024-02-18T00:48:44.122935+0000","last_peered":"2024-02-18T00:48:44.122935+0000","last_clean":"2024-02-18T00:48:44.122935+0000","last_became_active":"2024-02-18T00:46:25.873170+0000","last_became_peered":"2024-02-18T00:46:25.873170+0000","last_unstale":"2024-02-18T00:48:44.122935+0000","last_undegraded":"2024-02-18T00:48:44.122935+0000","last_fullsized":"2024-02-18T00:48:44.122935+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:47:57.808617+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:47:57.808617+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1158,"num_read_kb":1153,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":438,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292558+0000","last_change":"2024-02-18T00:48:17.162540+0000","last_active":"2024-02-18T00:48:36.292558+0000","last_peered":"2024-02-18T00:48:36.292558+0000","last_clean":"2024-02-18T00:48:36.292558+0000","last_became_active":"2024-02-18T00:46:25.885434+0000","last_became_peered":"2024-02-18T00:46:25.885434+0000","last_unstale":"2024-02-18T00:48:36.292558+0000","last_undegraded":"2024-02-18T00:48:36.292558+0000","last_fullsized":"2024-02-18T00:48:36.292558+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:48:17.162456+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:46:48.220900+0000","last_clean_scrub_stamp":"2024-02-18T00:48:17.162456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":798,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406867+0000","last_change":"2024-02-18T00:47:53.415120+0000","last_active":"2024-02-18T00:48:36.406867+0000","last_peered":"2024-02-18T00:48:36.406867+0000","last_clean":"2024-02-18T00:48:36.406867+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:48:36.406867+0000","last_undegraded":"2024-02-18T00:48:36.406867+0000","last_fullsized":"2024-02-18T00:48:36.406867+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:53.415039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_clean_scrub_stamp":"2024-02-18T00:47:53.415039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":840,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794173+0000","last_change":"2024-02-18T00:48:26.129134+0000","last_active":"2024-02-18T00:48:36.794173+0000","last_peered":"2024-02-18T00:48:36.794173+0000","last_clean":"2024-02-18T00:48:36.794173+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:48:36.794173+0000","last_undegraded":"2024-02-18T00:48:36.794173+0000","last_fullsized":"2024-02-18T00:48:36.794173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:26.129050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:26.129050+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"682'49","reported_seq":850,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.082102+0000","last_change":"2024-02-18T00:48:34.902228+0000","last_active":"2024-02-18T00:48:39.082102+0000","last_peered":"2024-02-18T00:48:39.082102+0000","last_clean":"2024-02-18T00:48:39.082102+0000","last_became_active":"2024-02-18T00:43:44.816299+0000","last_became_peered":"2024-02-18T00:43:44.816299+0000","last_unstale":"2024-02-18T00:48:39.082102+0000","last_undegraded":"2024-02-18T00:48:39.082102+0000","last_fullsized":"2024-02-18T00:48:39.082102+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":0,"last_scrub":"682'49","last_scrub_stamp":"2024-02-18T00:48:34.902137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:34.902137+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":248,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":872,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.981185+0000","last_change":"2024-02-18T00:48:43.981185+0000","last_active":"2024-02-18T00:48:43.981185+0000","last_peered":"2024-02-18T00:48:43.981185+0000","last_clean":"2024-02-18T00:48:43.981185+0000","last_became_active":"2024-02-18T00:46:25.879623+0000","last_became_peered":"2024-02-18T00:46:25.879623+0000","last_unstale":"2024-02-18T00:48:43.981185+0000","last_undegraded":"2024-02-18T00:48:43.981185+0000","last_fullsized":"2024-02-18T00:48:43.981185+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:43.981081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:15.527235+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.981081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":864,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377504+0000","last_change":"2024-02-18T00:48:29.588456+0000","last_active":"2024-02-18T00:48:36.377504+0000","last_peered":"2024-02-18T00:48:36.377504+0000","last_clean":"2024-02-18T00:48:36.377504+0000","last_became_active":"2024-02-18T00:42:02.125704+0000","last_became_peered":"2024-02-18T00:42:02.125704+0000","last_unstale":"2024-02-18T00:48:36.377504+0000","last_undegraded":"2024-02-18T00:48:36.377504+0000","last_fullsized":"2024-02-18T00:48:36.377504+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:29.588378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:48:29.588378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":798,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794379+0000","last_change":"2024-02-18T00:47:34.142510+0000","last_active":"2024-02-18T00:48:36.794379+0000","last_peered":"2024-02-18T00:48:36.794379+0000","last_clean":"2024-02-18T00:48:36.794379+0000","last_became_active":"2024-02-18T00:46:25.893568+0000","last_became_peered":"2024-02-18T00:46:25.893568+0000","last_unstale":"2024-02-18T00:48:36.794379+0000","last_undegraded":"2024-02-18T00:48:36.794379+0000","last_fullsized":"2024-02-18T00:48:36.794379+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:34.142434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:47:34.142434+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"811'10","reported_seq":1103,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.884565+0000","last_change":"2024-02-18T00:47:35.142340+0000","last_active":"2024-02-18T00:48:38.884565+0000","last_peered":"2024-02-18T00:48:38.884565+0000","last_clean":"2024-02-18T00:48:38.884565+0000","last_became_active":"2024-02-18T00:46:25.893478+0000","last_became_peered":"2024-02-18T00:46:25.893478+0000","last_unstale":"2024-02-18T00:48:38.884565+0000","last_undegraded":"2024-02-18T00:48:38.884565+0000","last_fullsized":"2024-02-18T00:48:38.884565+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"811'10","last_scrub_stamp":"2024-02-18T00:47:35.142218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:47:35.142218+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":867,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293692+0000","last_change":"2024-02-18T00:48:12.689067+0000","last_active":"2024-02-18T00:48:36.293692+0000","last_peered":"2024-02-18T00:48:36.293692+0000","last_clean":"2024-02-18T00:48:36.293692+0000","last_became_active":"2024-02-18T00:43:07.795433+0000","last_became_peered":"2024-02-18T00:43:07.795433+0000","last_unstale":"2024-02-18T00:48:36.293692+0000","last_undegraded":"2024-02-18T00:48:36.293692+0000","last_fullsized":"2024-02-18T00:48:36.293692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:48:12.688998+0000","last_deep_scrub":"457'32","last_deep_scrub_stamp":"2024-02-18T00:48:12.688998+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.688998+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":880,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.294075+0000","last_change":"2024-02-18T00:47:45.771572+0000","last_active":"2024-02-18T00:48:36.294075+0000","last_peered":"2024-02-18T00:48:36.294075+0000","last_clean":"2024-02-18T00:48:36.294075+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:48:36.294075+0000","last_undegraded":"2024-02-18T00:48:36.294075+0000","last_fullsized":"2024-02-18T00:48:36.294075+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:45.771506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_clean_scrub_stamp":"2024-02-18T00:47:45.771506+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":833,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793328+0000","last_change":"2024-02-18T00:48:32.028325+0000","last_active":"2024-02-18T00:48:36.793328+0000","last_peered":"2024-02-18T00:48:36.793328+0000","last_clean":"2024-02-18T00:48:36.793328+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:48:36.793328+0000","last_undegraded":"2024-02-18T00:48:36.793328+0000","last_fullsized":"2024-02-18T00:48:36.793328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.028259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.028259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":785,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:40.620935+0000","last_change":"2024-02-18T00:48:40.620935+0000","last_active":"2024-02-18T00:48:40.620935+0000","last_peered":"2024-02-18T00:48:40.620935+0000","last_clean":"2024-02-18T00:48:40.620935+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:48:40.620935+0000","last_undegraded":"2024-02-18T00:48:40.620935+0000","last_fullsized":"2024-02-18T00:48:40.620935+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:40.620856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:40.620856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":839,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291950+0000","last_change":"2024-02-18T00:48:06.766100+0000","last_active":"2024-02-18T00:48:36.291950+0000","last_peered":"2024-02-18T00:48:36.291950+0000","last_clean":"2024-02-18T00:48:36.291950+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:48:36.291950+0000","last_undegraded":"2024-02-18T00:48:36.291950+0000","last_fullsized":"2024-02-18T00:48:36.291950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:06.766026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:48:06.766026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":915,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406658+0000","last_change":"2024-02-18T00:48:07.440661+0000","last_active":"2024-02-18T00:48:36.406658+0000","last_peered":"2024-02-18T00:48:36.406658+0000","last_clean":"2024-02-18T00:48:36.406658+0000","last_became_active":"2024-02-18T00:43:07.795322+0000","last_became_peered":"2024-02-18T00:43:07.795322+0000","last_unstale":"2024-02-18T00:48:36.406658+0000","last_undegraded":"2024-02-18T00:48:36.406658+0000","last_fullsized":"2024-02-18T00:48:36.406658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:48:07.440575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:07.440575+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":893,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407865+0000","last_change":"2024-02-18T00:48:24.381563+0000","last_active":"2024-02-18T00:48:36.407865+0000","last_peered":"2024-02-18T00:48:36.407865+0000","last_clean":"2024-02-18T00:48:36.407865+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:48:36.407865+0000","last_undegraded":"2024-02-18T00:48:36.407865+0000","last_fullsized":"2024-02-18T00:48:36.407865+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:48:24.381480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:24.381480+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":872,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:44.389289+0000","last_change":"2024-02-18T00:48:44.389289+0000","last_active":"2024-02-18T00:48:44.389289+0000","last_peered":"2024-02-18T00:48:44.389289+0000","last_clean":"2024-02-18T00:48:44.389289+0000","last_became_active":"2024-02-18T00:41:42.776144+0000","last_became_peered":"2024-02-18T00:41:42.776144+0000","last_unstale":"2024-02-18T00:48:44.389289+0000","last_undegraded":"2024-02-18T00:48:44.389289+0000","last_fullsized":"2024-02-18T00:48:44.389289+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:44.389213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:44.389213+0000","last_clean_scrub_stamp":"2024-02-18T00:48:44.389213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406825+0000","last_change":"2024-02-18T00:47:45.501077+0000","last_active":"2024-02-18T00:48:36.406825+0000","last_peered":"2024-02-18T00:48:36.406825+0000","last_clean":"2024-02-18T00:48:36.406825+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:48:36.406825+0000","last_undegraded":"2024-02-18T00:48:36.406825+0000","last_fullsized":"2024-02-18T00:48:36.406825+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:45.500956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:47:45.500956+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"774'33","reported_seq":972,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794208+0000","last_change":"2024-02-18T00:48:33.069065+0000","last_active":"2024-02-18T00:48:36.794208+0000","last_peered":"2024-02-18T00:48:36.794208+0000","last_clean":"2024-02-18T00:48:36.794208+0000","last_became_active":"2024-02-18T00:46:10.866407+0000","last_became_peered":"2024-02-18T00:46:10.866407+0000","last_unstale":"2024-02-18T00:48:36.794208+0000","last_undegraded":"2024-02-18T00:48:36.794208+0000","last_fullsized":"2024-02-18T00:48:36.794208+0000","mapping_epoch":801,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":802,"parent":"0.0","parent_split_bits":0,"last_scrub":"774'33","last_scrub_stamp":"2024-02-18T00:48:33.068984+0000","last_deep_scrub":"774'33","last_deep_scrub_stamp":"2024-02-18T00:48:33.068984+0000","last_clean_scrub_stamp":"2024-02-18T00:48:33.068984+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":843,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291868+0000","last_change":"2024-02-18T00:47:50.739024+0000","last_active":"2024-02-18T00:48:36.291868+0000","last_peered":"2024-02-18T00:48:36.291868+0000","last_clean":"2024-02-18T00:48:36.291868+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:48:36.291868+0000","last_undegraded":"2024-02-18T00:48:36.291868+0000","last_fullsized":"2024-02-18T00:48:36.291868+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:50.738946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:50.738946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377679+0000","last_change":"2024-02-18T00:47:35.664446+0000","last_active":"2024-02-18T00:48:36.377679+0000","last_peered":"2024-02-18T00:48:36.377679+0000","last_clean":"2024-02-18T00:48:36.377679+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:48:36.377679+0000","last_undegraded":"2024-02-18T00:48:36.377679+0000","last_fullsized":"2024-02-18T00:48:36.377679+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:35.664372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:35.664372+0000","last_clean_scrub_stamp":"2024-02-18T00:47:35.664372+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":857,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292519+0000","last_change":"2024-02-18T00:48:19.749565+0000","last_active":"2024-02-18T00:48:36.292519+0000","last_peered":"2024-02-18T00:48:36.292519+0000","last_clean":"2024-02-18T00:48:36.292519+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:48:36.292519+0000","last_undegraded":"2024-02-18T00:48:36.292519+0000","last_fullsized":"2024-02-18T00:48:36.292519+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:19.749502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:19.749502+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.749502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793622+0000","last_change":"2024-02-18T00:47:43.134205+0000","last_active":"2024-02-18T00:48:36.793622+0000","last_peered":"2024-02-18T00:48:36.793622+0000","last_clean":"2024-02-18T00:48:36.793622+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:48:36.793622+0000","last_undegraded":"2024-02-18T00:48:36.793622+0000","last_fullsized":"2024-02-18T00:48:36.793622+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:43.134120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:47:43.134120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":925,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377480+0000","last_change":"2024-02-18T00:47:13.848991+0000","last_active":"2024-02-18T00:48:36.377480+0000","last_peered":"2024-02-18T00:48:36.377480+0000","last_clean":"2024-02-18T00:48:36.377480+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:48:36.377480+0000","last_undegraded":"2024-02-18T00:48:36.377480+0000","last_fullsized":"2024-02-18T00:48:36.377480+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'18","last_scrub_stamp":"2024-02-18T00:47:13.848910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:13.848910+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794004+0000","last_change":"2024-02-18T00:47:58.456547+0000","last_active":"2024-02-18T00:48:36.794004+0000","last_peered":"2024-02-18T00:48:36.794004+0000","last_clean":"2024-02-18T00:48:36.794004+0000","last_became_active":"2024-02-18T00:46:25.883636+0000","last_became_peered":"2024-02-18T00:46:25.883636+0000","last_unstale":"2024-02-18T00:48:36.794004+0000","last_undegraded":"2024-02-18T00:48:36.794004+0000","last_fullsized":"2024-02-18T00:48:36.794004+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:58.456471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:58.456471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":841,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.972701+0000","last_change":"2024-02-18T00:48:39.972701+0000","last_active":"2024-02-18T00:48:39.972701+0000","last_peered":"2024-02-18T00:48:39.972701+0000","last_clean":"2024-02-18T00:48:39.972701+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:48:39.972701+0000","last_undegraded":"2024-02-18T00:48:39.972701+0000","last_fullsized":"2024-02-18T00:48:39.972701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:39.972610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.972610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292586+0000","last_change":"2024-02-18T00:48:03.145476+0000","last_active":"2024-02-18T00:48:36.292586+0000","last_peered":"2024-02-18T00:48:36.292586+0000","last_clean":"2024-02-18T00:48:36.292586+0000","last_became_active":"2024-02-18T00:46:25.884359+0000","last_became_peered":"2024-02-18T00:46:25.884359+0000","last_unstale":"2024-02-18T00:48:36.292586+0000","last_undegraded":"2024-02-18T00:48:36.292586+0000","last_fullsized":"2024-02-18T00:48:36.292586+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:03.145333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:03.145333+0000","last_clean_scrub_stamp":"2024-02-18T00:48:03.145333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":792,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292268+0000","last_change":"2024-02-18T00:48:09.665657+0000","last_active":"2024-02-18T00:48:36.292268+0000","last_peered":"2024-02-18T00:48:36.292268+0000","last_clean":"2024-02-18T00:48:36.292268+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:48:36.292268+0000","last_undegraded":"2024-02-18T00:48:36.292268+0000","last_fullsized":"2024-02-18T00:48:36.292268+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.665586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.665586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":942,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377676+0000","last_change":"2024-02-18T00:47:40.635304+0000","last_active":"2024-02-18T00:48:36.377676+0000","last_peered":"2024-02-18T00:48:36.377676+0000","last_clean":"2024-02-18T00:48:36.377676+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:48:36.377676+0000","last_undegraded":"2024-02-18T00:48:36.377676+0000","last_fullsized":"2024-02-18T00:48:36.377676+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:47:40.635222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:40.635222+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":867,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794931+0000","last_change":"2024-02-18T00:48:16.745096+0000","last_active":"2024-02-18T00:48:36.794931+0000","last_peered":"2024-02-18T00:48:36.794931+0000","last_clean":"2024-02-18T00:48:36.794931+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:48:36.794931+0000","last_undegraded":"2024-02-18T00:48:36.794931+0000","last_fullsized":"2024-02-18T00:48:36.794931+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:16.745018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:16.745018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":1084,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:42.938710+0000","last_change":"2024-02-18T00:48:42.938666+0000","last_active":"2024-02-18T00:48:42.938710+0000","last_peered":"2024-02-18T00:48:42.938710+0000","last_clean":"2024-02-18T00:48:42.938710+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:48:42.938710+0000","last_undegraded":"2024-02-18T00:48:42.938710+0000","last_fullsized":"2024-02-18T00:48:42.938710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:48:42.938544+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:48:42.938544+0000","last_clean_scrub_stamp":"2024-02-18T00:48:42.938544+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794198+0000","last_change":"2024-02-18T00:48:08.440272+0000","last_active":"2024-02-18T00:48:36.794198+0000","last_peered":"2024-02-18T00:48:36.794198+0000","last_clean":"2024-02-18T00:48:36.794198+0000","last_became_active":"2024-02-18T00:46:25.887695+0000","last_became_peered":"2024-02-18T00:46:25.887695+0000","last_unstale":"2024-02-18T00:48:36.794198+0000","last_undegraded":"2024-02-18T00:48:36.794198+0000","last_fullsized":"2024-02-18T00:48:36.794198+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:08.440187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:08.440187+0000","last_clean_scrub_stamp":"2024-02-18T00:48:08.440187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"849'26","reported_seq":925,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794589+0000","last_change":"2024-02-18T00:48:20.855911+0000","last_active":"2024-02-18T00:48:36.794589+0000","last_peered":"2024-02-18T00:48:36.794589+0000","last_clean":"2024-02-18T00:48:36.794589+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:48:36.794589+0000","last_undegraded":"2024-02-18T00:48:36.794589+0000","last_fullsized":"2024-02-18T00:48:36.794589+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"849'26","last_scrub_stamp":"2024-02-18T00:48:20.855835+0000","last_deep_scrub":"849'26","last_deep_scrub_stamp":"2024-02-18T00:48:20.855835+0000","last_clean_scrub_stamp":"2024-02-18T00:48:20.855835+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":844,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:40.140287+0000","last_change":"2024-02-18T00:48:40.140287+0000","last_active":"2024-02-18T00:48:40.140287+0000","last_peered":"2024-02-18T00:48:40.140287+0000","last_clean":"2024-02-18T00:48:40.140287+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:48:40.140287+0000","last_undegraded":"2024-02-18T00:48:40.140287+0000","last_fullsized":"2024-02-18T00:48:40.140287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:40.140220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:48:40.140220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":1099,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.885795+0000","last_change":"2024-02-18T00:47:38.681440+0000","last_active":"2024-02-18T00:48:38.885795+0000","last_peered":"2024-02-18T00:48:38.885795+0000","last_clean":"2024-02-18T00:48:38.885795+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:48:38.885795+0000","last_undegraded":"2024-02-18T00:48:38.885795+0000","last_fullsized":"2024-02-18T00:48:38.885795+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:47:38.681367+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:47:38.681367+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":803,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:44.997940+0000","last_change":"2024-02-18T00:48:44.997940+0000","last_active":"2024-02-18T00:48:44.997940+0000","last_peered":"2024-02-18T00:48:44.997940+0000","last_clean":"2024-02-18T00:48:44.997940+0000","last_became_active":"2024-02-18T00:46:25.878776+0000","last_became_peered":"2024-02-18T00:46:25.878776+0000","last_unstale":"2024-02-18T00:48:44.997940+0000","last_undegraded":"2024-02-18T00:48:44.997940+0000","last_fullsized":"2024-02-18T00:48:44.997940+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:44.997853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:48:44.997853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":911,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293626+0000","last_change":"2024-02-18T00:48:26.846345+0000","last_active":"2024-02-18T00:48:36.293626+0000","last_peered":"2024-02-18T00:48:36.293626+0000","last_clean":"2024-02-18T00:48:36.293626+0000","last_became_active":"2024-02-18T00:46:25.884390+0000","last_became_peered":"2024-02-18T00:46:25.884390+0000","last_unstale":"2024-02-18T00:48:36.293626+0000","last_undegraded":"2024-02-18T00:48:36.293626+0000","last_fullsized":"2024-02-18T00:48:36.293626+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:48:26.846279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:26.846279+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794349+0000","last_change":"2024-02-18T00:48:14.488680+0000","last_active":"2024-02-18T00:48:36.794349+0000","last_peered":"2024-02-18T00:48:36.794349+0000","last_clean":"2024-02-18T00:48:36.794349+0000","last_became_active":"2024-02-18T00:46:25.887011+0000","last_became_peered":"2024-02-18T00:46:25.887011+0000","last_unstale":"2024-02-18T00:48:36.794349+0000","last_undegraded":"2024-02-18T00:48:36.794349+0000","last_fullsized":"2024-02-18T00:48:36.794349+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:14.488598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:14.488598+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":835,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.435734+0000","last_change":"2024-02-18T00:48:38.435700+0000","last_active":"2024-02-18T00:48:38.435734+0000","last_peered":"2024-02-18T00:48:38.435734+0000","last_clean":"2024-02-18T00:48:38.435734+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:48:38.435734+0000","last_undegraded":"2024-02-18T00:48:38.435734+0000","last_fullsized":"2024-02-18T00:48:38.435734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:38.435618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:38.435618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":838,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793266+0000","last_change":"2024-02-18T00:47:46.084843+0000","last_active":"2024-02-18T00:48:36.793266+0000","last_peered":"2024-02-18T00:48:36.793266+0000","last_clean":"2024-02-18T00:48:36.793266+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:48:36.793266+0000","last_undegraded":"2024-02-18T00:48:36.793266+0000","last_fullsized":"2024-02-18T00:48:36.793266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:46.084758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:46.084758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":782,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794077+0000","last_change":"2024-02-18T00:47:52.024589+0000","last_active":"2024-02-18T00:48:36.794077+0000","last_peered":"2024-02-18T00:48:36.794077+0000","last_clean":"2024-02-18T00:48:36.794077+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:48:36.794077+0000","last_undegraded":"2024-02-18T00:48:36.794077+0000","last_fullsized":"2024-02-18T00:48:36.794077+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:52.024516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:52.024516+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.024516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":916,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407018+0000","last_change":"2024-02-18T00:48:11.389874+0000","last_active":"2024-02-18T00:48:36.407018+0000","last_peered":"2024-02-18T00:48:36.407018+0000","last_clean":"2024-02-18T00:48:36.407018+0000","last_became_active":"2024-02-18T00:46:25.878984+0000","last_became_peered":"2024-02-18T00:46:25.878984+0000","last_unstale":"2024-02-18T00:48:36.407018+0000","last_undegraded":"2024-02-18T00:48:36.407018+0000","last_fullsized":"2024-02-18T00:48:36.407018+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:48:11.389795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.389795+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292779+0000","last_change":"2024-02-18T00:48:24.781709+0000","last_active":"2024-02-18T00:48:36.292779+0000","last_peered":"2024-02-18T00:48:36.292779+0000","last_clean":"2024-02-18T00:48:36.292779+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:48:36.292779+0000","last_undegraded":"2024-02-18T00:48:36.292779+0000","last_fullsized":"2024-02-18T00:48:36.292779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:24.781645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:24.781645+0000","last_clean_scrub_stamp":"2024-02-18T00:48:24.781645+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":849,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293706+0000","last_change":"2024-02-18T00:48:32.770838+0000","last_active":"2024-02-18T00:48:36.293706+0000","last_peered":"2024-02-18T00:48:36.293706+0000","last_clean":"2024-02-18T00:48:36.293706+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:48:36.293706+0000","last_undegraded":"2024-02-18T00:48:36.293706+0000","last_fullsized":"2024-02-18T00:48:36.293706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.770769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.770769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":828,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.376954+0000","last_change":"2024-02-18T00:47:47.674631+0000","last_active":"2024-02-18T00:48:36.376954+0000","last_peered":"2024-02-18T00:48:36.376954+0000","last_clean":"2024-02-18T00:48:36.376954+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:48:36.376954+0000","last_undegraded":"2024-02-18T00:48:36.376954+0000","last_fullsized":"2024-02-18T00:48:36.376954+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:47.674565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:47.674565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:37.597512+0000","last_change":"2024-02-18T00:48:37.597465+0000","last_active":"2024-02-18T00:48:37.597512+0000","last_peered":"2024-02-18T00:48:37.597512+0000","last_clean":"2024-02-18T00:48:37.597512+0000","last_became_active":"2024-02-18T00:46:25.877319+0000","last_became_peered":"2024-02-18T00:46:25.877319+0000","last_unstale":"2024-02-18T00:48:37.597512+0000","last_undegraded":"2024-02-18T00:48:37.597512+0000","last_fullsized":"2024-02-18T00:48:37.597512+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:37.597372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:37.597372+0000","last_clean_scrub_stamp":"2024-02-18T00:48:37.597372+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"855'10","reported_seq":881,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:42.424614+0000","last_change":"2024-02-18T00:48:42.424614+0000","last_active":"2024-02-18T00:48:42.424614+0000","last_peered":"2024-02-18T00:48:42.424614+0000","last_clean":"2024-02-18T00:48:42.424614+0000","last_became_active":"2024-02-18T00:46:25.877523+0000","last_became_peered":"2024-02-18T00:46:25.877523+0000","last_unstale":"2024-02-18T00:48:42.424614+0000","last_undegraded":"2024-02-18T00:48:42.424614+0000","last_fullsized":"2024-02-18T00:48:42.424614+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"855'10","last_scrub_stamp":"2024-02-18T00:48:42.424538+0000","last_deep_scrub":"855'10","last_deep_scrub_stamp":"2024-02-18T00:48:42.424538+0000","last_clean_scrub_stamp":"2024-02-18T00:48:42.424538+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":863,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794961+0000","last_change":"2024-02-18T00:48:26.857897+0000","last_active":"2024-02-18T00:48:36.794961+0000","last_peered":"2024-02-18T00:48:36.794961+0000","last_clean":"2024-02-18T00:48:36.794961+0000","last_became_active":"2024-02-18T00:46:25.877697+0000","last_became_peered":"2024-02-18T00:46:25.877697+0000","last_unstale":"2024-02-18T00:48:36.794961+0000","last_undegraded":"2024-02-18T00:48:36.794961+0000","last_fullsized":"2024-02-18T00:48:36.794961+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:26.857823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_clean_scrub_stamp":"2024-02-18T00:48:26.857823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795441+0000","last_change":"2024-02-18T00:47:48.554083+0000","last_active":"2024-02-18T00:48:36.795441+0000","last_peered":"2024-02-18T00:48:36.795441+0000","last_clean":"2024-02-18T00:48:36.795441+0000","last_became_active":"2024-02-18T00:46:25.889345+0000","last_became_peered":"2024-02-18T00:46:25.889345+0000","last_unstale":"2024-02-18T00:48:36.795441+0000","last_undegraded":"2024-02-18T00:48:36.795441+0000","last_fullsized":"2024-02-18T00:48:36.795441+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:48.554001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:47:48.554001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291810+0000","last_change":"2024-02-18T00:48:04.172891+0000","last_active":"2024-02-18T00:48:36.291810+0000","last_peered":"2024-02-18T00:48:36.291810+0000","last_clean":"2024-02-18T00:48:36.291810+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:48:36.291810+0000","last_undegraded":"2024-02-18T00:48:36.291810+0000","last_fullsized":"2024-02-18T00:48:36.291810+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:04.172800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:04.172800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":798,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795313+0000","last_change":"2024-02-18T00:48:09.832713+0000","last_active":"2024-02-18T00:48:36.795313+0000","last_peered":"2024-02-18T00:48:36.795313+0000","last_clean":"2024-02-18T00:48:36.795313+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:48:36.795313+0000","last_undegraded":"2024-02-18T00:48:36.795313+0000","last_fullsized":"2024-02-18T00:48:36.795313+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.832634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.832634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":970,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377477+0000","last_change":"2024-02-18T00:47:54.667460+0000","last_active":"2024-02-18T00:48:36.377477+0000","last_peered":"2024-02-18T00:48:36.377477+0000","last_clean":"2024-02-18T00:48:36.377477+0000","last_became_active":"2024-02-18T00:46:25.877948+0000","last_became_peered":"2024-02-18T00:46:25.877948+0000","last_unstale":"2024-02-18T00:48:36.377477+0000","last_undegraded":"2024-02-18T00:48:36.377477+0000","last_fullsized":"2024-02-18T00:48:36.377477+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:47:54.667369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:54.667369+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794929+0000","last_change":"2024-02-18T00:48:24.653307+0000","last_active":"2024-02-18T00:48:36.794929+0000","last_peered":"2024-02-18T00:48:36.794929+0000","last_clean":"2024-02-18T00:48:36.794929+0000","last_became_active":"2024-02-18T00:46:25.887695+0000","last_became_peered":"2024-02-18T00:46:25.887695+0000","last_unstale":"2024-02-18T00:48:36.794929+0000","last_undegraded":"2024-02-18T00:48:36.794929+0000","last_fullsized":"2024-02-18T00:48:36.794929+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:24.653226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_clean_scrub_stamp":"2024-02-18T00:48:24.653226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":421,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291833+0000","last_change":"2024-02-18T00:48:00.117023+0000","last_active":"2024-02-18T00:48:36.291833+0000","last_peered":"2024-02-18T00:48:36.291833+0000","last_clean":"2024-02-18T00:48:36.291833+0000","last_became_active":"2024-02-18T00:46:25.881315+0000","last_became_peered":"2024-02-18T00:46:25.881315+0000","last_unstale":"2024-02-18T00:48:36.291833+0000","last_undegraded":"2024-02-18T00:48:36.291833+0000","last_fullsized":"2024-02-18T00:48:36.291833+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:00.116944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:00.116944+0000","last_clean_scrub_stamp":"2024-02-18T00:48:00.116944+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":846,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377619+0000","last_change":"2024-02-18T00:47:37.695596+0000","last_active":"2024-02-18T00:48:36.377619+0000","last_peered":"2024-02-18T00:48:36.377619+0000","last_clean":"2024-02-18T00:48:36.377619+0000","last_became_active":"2024-02-18T00:46:25.884576+0000","last_became_peered":"2024-02-18T00:46:25.884576+0000","last_unstale":"2024-02-18T00:48:36.377619+0000","last_undegraded":"2024-02-18T00:48:36.377619+0000","last_fullsized":"2024-02-18T00:48:36.377619+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:37.695526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:37.695526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":413,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.005444+0000","last_change":"2024-02-18T00:48:39.005444+0000","last_active":"2024-02-18T00:48:39.005444+0000","last_peered":"2024-02-18T00:48:39.005444+0000","last_clean":"2024-02-18T00:48:39.005444+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:48:39.005444+0000","last_undegraded":"2024-02-18T00:48:39.005444+0000","last_fullsized":"2024-02-18T00:48:39.005444+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:39.005355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:16.073560+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.005355+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":939,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794331+0000","last_change":"2024-02-18T00:48:12.086826+0000","last_active":"2024-02-18T00:48:36.794331+0000","last_peered":"2024-02-18T00:48:36.794331+0000","last_clean":"2024-02-18T00:48:36.794331+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:48:36.794331+0000","last_undegraded":"2024-02-18T00:48:36.794331+0000","last_fullsized":"2024-02-18T00:48:36.794331+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:48:12.086743+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.086743+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":867,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793936+0000","last_change":"2024-02-18T00:48:29.905227+0000","last_active":"2024-02-18T00:48:36.793936+0000","last_peered":"2024-02-18T00:48:36.793936+0000","last_clean":"2024-02-18T00:48:36.793936+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:48:36.793936+0000","last_undegraded":"2024-02-18T00:48:36.793936+0000","last_fullsized":"2024-02-18T00:48:36.793936+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:29.905152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:29.905152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"811'12","reported_seq":93,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.084693+0000","last_change":"2024-02-18T00:47:52.510602+0000","last_active":"2024-02-18T00:48:39.084693+0000","last_peered":"2024-02-18T00:48:39.084693+0000","last_clean":"2024-02-18T00:48:39.084693+0000","last_became_active":"2024-02-18T00:46:25.888871+0000","last_became_peered":"2024-02-18T00:46:25.888871+0000","last_unstale":"2024-02-18T00:48:39.084693+0000","last_undegraded":"2024-02-18T00:48:39.084693+0000","last_fullsized":"2024-02-18T00:48:39.084693+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"811'12","last_scrub_stamp":"2024-02-18T00:47:52.510507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.510507+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":410,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291753+0000","last_change":"2024-02-18T00:47:18.202393+0000","last_active":"2024-02-18T00:48:36.291753+0000","last_peered":"2024-02-18T00:48:36.291753+0000","last_clean":"2024-02-18T00:48:36.291753+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:48:36.291753+0000","last_undegraded":"2024-02-18T00:48:36.291753+0000","last_fullsized":"2024-02-18T00:48:36.291753+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:18.202309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:47:18.202309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":80,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794992+0000","last_change":"2024-02-18T00:48:35.538755+0000","last_active":"2024-02-18T00:48:36.794992+0000","last_peered":"2024-02-18T00:48:36.794992+0000","last_clean":"2024-02-18T00:48:36.794992+0000","last_became_active":"2024-02-18T00:46:25.888212+0000","last_became_peered":"2024-02-18T00:46:25.888212+0000","last_unstale":"2024-02-18T00:48:36.794992+0000","last_undegraded":"2024-02-18T00:48:36.794992+0000","last_fullsized":"2024-02-18T00:48:36.794992+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:48:35.538662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:35.538662+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":35,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":852,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377065+0000","last_change":"2024-02-18T00:47:22.780265+0000","last_active":"2024-02-18T00:48:36.377065+0000","last_peered":"2024-02-18T00:48:36.377065+0000","last_clean":"2024-02-18T00:48:36.377065+0000","last_became_active":"2024-02-18T00:46:25.884395+0000","last_became_peered":"2024-02-18T00:46:25.884395+0000","last_unstale":"2024-02-18T00:48:36.377065+0000","last_undegraded":"2024-02-18T00:48:36.377065+0000","last_fullsized":"2024-02-18T00:48:36.377065+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:22.780187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:38.322790+0000","last_clean_scrub_stamp":"2024-02-18T00:47:22.780187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":853,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794345+0000","last_change":"2024-02-18T00:48:06.045344+0000","last_active":"2024-02-18T00:48:36.794345+0000","last_peered":"2024-02-18T00:48:36.794345+0000","last_clean":"2024-02-18T00:48:36.794345+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:48:36.794345+0000","last_undegraded":"2024-02-18T00:48:36.794345+0000","last_fullsized":"2024-02-18T00:48:36.794345+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:06.045277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_clean_scrub_stamp":"2024-02-18T00:48:06.045277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":827,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.989922+0000","last_change":"2024-02-18T00:48:38.989890+0000","last_active":"2024-02-18T00:48:38.989922+0000","last_peered":"2024-02-18T00:48:38.989922+0000","last_clean":"2024-02-18T00:48:38.989922+0000","last_became_active":"2024-02-18T00:46:25.876848+0000","last_became_peered":"2024-02-18T00:46:25.876848+0000","last_unstale":"2024-02-18T00:48:38.989922+0000","last_undegraded":"2024-02-18T00:48:38.989922+0000","last_fullsized":"2024-02-18T00:48:38.989922+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:48:38.989811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:38.989811+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":10,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":889,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292868+0000","last_change":"2024-02-18T00:47:45.735338+0000","last_active":"2024-02-18T00:48:36.292868+0000","last_peered":"2024-02-18T00:48:36.292868+0000","last_clean":"2024-02-18T00:48:36.292868+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:48:36.292868+0000","last_undegraded":"2024-02-18T00:48:36.292868+0000","last_fullsized":"2024-02-18T00:48:36.292868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:47:45.735267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:45.735267+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794971+0000","last_change":"2024-02-18T00:47:39.615410+0000","last_active":"2024-02-18T00:48:36.794971+0000","last_peered":"2024-02-18T00:48:36.794971+0000","last_clean":"2024-02-18T00:48:36.794971+0000","last_became_active":"2024-02-18T00:46:25.888425+0000","last_became_peered":"2024-02-18T00:46:25.888425+0000","last_unstale":"2024-02-18T00:48:36.794971+0000","last_undegraded":"2024-02-18T00:48:36.794971+0000","last_fullsized":"2024-02-18T00:48:36.794971+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:39.615298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:39.615298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":844,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293076+0000","last_change":"2024-02-18T00:47:39.769162+0000","last_active":"2024-02-18T00:48:36.293076+0000","last_peered":"2024-02-18T00:48:36.293076+0000","last_clean":"2024-02-18T00:48:36.293076+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:48:36.293076+0000","last_undegraded":"2024-02-18T00:48:36.293076+0000","last_fullsized":"2024-02-18T00:48:36.293076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:39.769080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:47:39.769080+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795391+0000","last_change":"2024-02-18T00:48:11.459844+0000","last_active":"2024-02-18T00:48:36.795391+0000","last_peered":"2024-02-18T00:48:36.795391+0000","last_clean":"2024-02-18T00:48:36.795391+0000","last_became_active":"2024-02-18T00:46:25.887945+0000","last_became_peered":"2024-02-18T00:46:25.887945+0000","last_unstale":"2024-02-18T00:48:36.795391+0000","last_undegraded":"2024-02-18T00:48:36.795391+0000","last_fullsized":"2024-02-18T00:48:36.795391+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:11.459762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.459762+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"813'30","reported_seq":885,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377686+0000","last_change":"2024-02-18T00:48:05.661177+0000","last_active":"2024-02-18T00:48:36.377686+0000","last_peered":"2024-02-18T00:48:36.377686+0000","last_clean":"2024-02-18T00:48:36.377686+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:48:36.377686+0000","last_undegraded":"2024-02-18T00:48:36.377686+0000","last_fullsized":"2024-02-18T00:48:36.377686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"813'30","last_scrub_stamp":"2024-02-18T00:48:05.661100+0000","last_deep_scrub":"813'30","last_deep_scrub_stamp":"2024-02-18T00:46:32.042766+0000","last_clean_scrub_stamp":"2024-02-18T00:48:05.661100+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":893,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377383+0000","last_change":"2024-02-18T00:47:21.743133+0000","last_active":"2024-02-18T00:48:36.377383+0000","last_peered":"2024-02-18T00:48:36.377383+0000","last_clean":"2024-02-18T00:48:36.377383+0000","last_became_active":"2024-02-18T00:46:25.877602+0000","last_became_peered":"2024-02-18T00:46:25.877602+0000","last_unstale":"2024-02-18T00:48:36.377383+0000","last_undegraded":"2024-02-18T00:48:36.377383+0000","last_fullsized":"2024-02-18T00:48:36.377383+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:47:21.743064+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:47:21.743064+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":841,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407520+0000","last_change":"2024-02-18T00:48:28.492357+0000","last_active":"2024-02-18T00:48:36.407520+0000","last_peered":"2024-02-18T00:48:36.407520+0000","last_clean":"2024-02-18T00:48:36.407520+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:48:36.407520+0000","last_undegraded":"2024-02-18T00:48:36.407520+0000","last_fullsized":"2024-02-18T00:48:36.407520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:28.492238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:03.672462+0000","last_clean_scrub_stamp":"2024-02-18T00:48:28.492238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":816,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794406+0000","last_change":"2024-02-18T00:47:41.102035+0000","last_active":"2024-02-18T00:48:36.794406+0000","last_peered":"2024-02-18T00:48:36.794406+0000","last_clean":"2024-02-18T00:48:36.794406+0000","last_became_active":"2024-02-18T00:46:25.877473+0000","last_became_peered":"2024-02-18T00:46:25.877473+0000","last_unstale":"2024-02-18T00:48:36.794406+0000","last_undegraded":"2024-02-18T00:48:36.794406+0000","last_fullsized":"2024-02-18T00:48:36.794406+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:41.101964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:41.101964+0000","last_clean_scrub_stamp":"2024-02-18T00:47:41.101964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":912,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:41.639902+0000","last_change":"2024-02-18T00:48:41.639902+0000","last_active":"2024-02-18T00:48:41.639902+0000","last_peered":"2024-02-18T00:48:41.639902+0000","last_clean":"2024-02-18T00:48:41.639902+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:48:41.639902+0000","last_undegraded":"2024-02-18T00:48:41.639902+0000","last_fullsized":"2024-02-18T00:48:41.639902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:48:41.639819+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:48:41.639819+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292227+0000","last_change":"2024-02-18T00:48:09.252416+0000","last_active":"2024-02-18T00:48:36.292227+0000","last_peered":"2024-02-18T00:48:36.292227+0000","last_clean":"2024-02-18T00:48:36.292227+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:48:36.292227+0000","last_undegraded":"2024-02-18T00:48:36.292227+0000","last_fullsized":"2024-02-18T00:48:36.292227+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.252337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:09.252337+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.252337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":877,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795446+0000","last_change":"2024-02-18T00:48:11.771480+0000","last_active":"2024-02-18T00:48:36.795446+0000","last_peered":"2024-02-18T00:48:36.795446+0000","last_clean":"2024-02-18T00:48:36.795446+0000","last_became_active":"2024-02-18T00:45:35.150420+0000","last_became_peered":"2024-02-18T00:45:35.150420+0000","last_unstale":"2024-02-18T00:48:36.795446+0000","last_undegraded":"2024-02-18T00:48:36.795446+0000","last_fullsized":"2024-02-18T00:48:36.795446+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:11.771383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:11.771383+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.771383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":785,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293259+0000","last_change":"2024-02-18T00:48:18.729333+0000","last_active":"2024-02-18T00:48:36.293259+0000","last_peered":"2024-02-18T00:48:36.293259+0000","last_clean":"2024-02-18T00:48:36.293259+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:48:36.293259+0000","last_undegraded":"2024-02-18T00:48:36.293259+0000","last_fullsized":"2024-02-18T00:48:36.293259+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:18.729250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:46.037549+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.729250+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":874,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.557180+0000","last_change":"2024-02-18T00:48:38.557148+0000","last_active":"2024-02-18T00:48:38.557180+0000","last_peered":"2024-02-18T00:48:38.557180+0000","last_clean":"2024-02-18T00:48:38.557180+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:48:38.557180+0000","last_undegraded":"2024-02-18T00:48:38.557180+0000","last_fullsized":"2024-02-18T00:48:38.557180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:48:38.557063+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:48:38.557063+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292418+0000","last_change":"2024-02-18T00:47:49.142720+0000","last_active":"2024-02-18T00:48:36.292418+0000","last_peered":"2024-02-18T00:48:36.292418+0000","last_clean":"2024-02-18T00:48:36.292418+0000","last_became_active":"2024-02-18T00:46:25.884881+0000","last_became_peered":"2024-02-18T00:46:25.884881+0000","last_unstale":"2024-02-18T00:48:36.292418+0000","last_undegraded":"2024-02-18T00:48:36.292418+0000","last_fullsized":"2024-02-18T00:48:36.292418+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:49.142613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:47:49.142613+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":408,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.059132+0000","last_change":"2024-02-18T00:48:43.059132+0000","last_active":"2024-02-18T00:48:43.059132+0000","last_peered":"2024-02-18T00:48:43.059132+0000","last_clean":"2024-02-18T00:48:43.059132+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:48:43.059132+0000","last_undegraded":"2024-02-18T00:48:43.059132+0000","last_fullsized":"2024-02-18T00:48:43.059132+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:43.059055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:43.059055+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.059055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":792,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377972+0000","last_change":"2024-02-18T00:48:12.670696+0000","last_active":"2024-02-18T00:48:36.377972+0000","last_peered":"2024-02-18T00:48:36.377972+0000","last_clean":"2024-02-18T00:48:36.377972+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:48:36.377972+0000","last_undegraded":"2024-02-18T00:48:36.377972+0000","last_fullsized":"2024-02-18T00:48:36.377972+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:12.670627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:44.868415+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.670627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":925,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406429+0000","last_change":"2024-02-18T00:47:49.529429+0000","last_active":"2024-02-18T00:48:36.406429+0000","last_peered":"2024-02-18T00:48:36.406429+0000","last_clean":"2024-02-18T00:48:36.406429+0000","last_became_active":"2024-02-18T00:46:25.869773+0000","last_became_peered":"2024-02-18T00:46:25.869773+0000","last_unstale":"2024-02-18T00:48:36.406429+0000","last_undegraded":"2024-02-18T00:48:36.406429+0000","last_fullsized":"2024-02-18T00:48:36.406429+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:47:49.529319+0000","last_deep_scrub":"457'26","last_deep_scrub_stamp":"2024-02-18T00:47:49.529319+0000","last_clean_scrub_stamp":"2024-02-18T00:47:49.529319+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":834,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377830+0000","last_change":"2024-02-18T00:47:49.756873+0000","last_active":"2024-02-18T00:48:36.377830+0000","last_peered":"2024-02-18T00:48:36.377830+0000","last_clean":"2024-02-18T00:48:36.377830+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:48:36.377830+0000","last_undegraded":"2024-02-18T00:48:36.377830+0000","last_fullsized":"2024-02-18T00:48:36.377830+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:49.756796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:47:49.756796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":862,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293399+0000","last_change":"2024-02-18T00:48:08.737377+0000","last_active":"2024-02-18T00:48:36.293399+0000","last_peered":"2024-02-18T00:48:36.293399+0000","last_clean":"2024-02-18T00:48:36.293399+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:48:36.293399+0000","last_undegraded":"2024-02-18T00:48:36.293399+0000","last_fullsized":"2024-02-18T00:48:36.293399+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:08.737294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:08.737294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":794,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377992+0000","last_change":"2024-02-18T00:47:52.687520+0000","last_active":"2024-02-18T00:48:36.377992+0000","last_peered":"2024-02-18T00:48:36.377992+0000","last_clean":"2024-02-18T00:48:36.377992+0000","last_became_active":"2024-02-18T00:46:25.877328+0000","last_became_peered":"2024-02-18T00:46:25.877328+0000","last_unstale":"2024-02-18T00:48:36.377992+0000","last_undegraded":"2024-02-18T00:48:36.377992+0000","last_fullsized":"2024-02-18T00:48:36.377992+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:52.687448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:52.687448+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.687448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":937,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377375+0000","last_change":"2024-02-18T00:48:35.498016+0000","last_active":"2024-02-18T00:48:36.377375+0000","last_peered":"2024-02-18T00:48:36.377375+0000","last_clean":"2024-02-18T00:48:36.377375+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:48:36.377375+0000","last_undegraded":"2024-02-18T00:48:36.377375+0000","last_fullsized":"2024-02-18T00:48:36.377375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-18T00:48:35.497936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:35.497936+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":859,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794262+0000","last_change":"2024-02-18T00:47:16.113861+0000","last_active":"2024-02-18T00:48:36.794262+0000","last_peered":"2024-02-18T00:48:36.794262+0000","last_clean":"2024-02-18T00:48:36.794262+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:48:36.794262+0000","last_undegraded":"2024-02-18T00:48:36.794262+0000","last_fullsized":"2024-02-18T00:48:36.794262+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:16.113781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:16.113781+0000","last_clean_scrub_stamp":"2024-02-18T00:47:16.113781+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":859,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406610+0000","last_change":"2024-02-18T00:48:06.405599+0000","last_active":"2024-02-18T00:48:36.406610+0000","last_peered":"2024-02-18T00:48:36.406610+0000","last_clean":"2024-02-18T00:48:36.406610+0000","last_became_active":"2024-02-18T00:46:25.869741+0000","last_became_peered":"2024-02-18T00:46:25.869741+0000","last_unstale":"2024-02-18T00:48:36.406610+0000","last_undegraded":"2024-02-18T00:48:36.406610+0000","last_fullsized":"2024-02-18T00:48:36.406610+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:06.405490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:06.405490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":10,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":42,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":940,"num_read_kb":774,"num_write":562,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":3080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1322,"ondisk_log_size":1322,"up":126,"acting":126,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1158,"num_read_kb":1153,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266927,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304464,"kb_used_data":7240,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440688,"statfs":{"total":95995035648,"available":95683264512,"internally_reserved":0,"allocated":7413760,"data_stored":4032682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.502,"15min":0.57399999999999995},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.16900000000000001},"max":{"1min":0.71499999999999997,"5min":0.96399999999999997,"15min":6.8700000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.60899999999999999},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.218},"max":{"1min":0.77800000000000002,"5min":1.24,"15min":11.288},"last":0.46800000000000003}]},{"osd":1,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.63300000000000001},"min":{"1min":0.40699999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.748,"5min":1.7330000000000001,"15min":7.4729999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.65200000000000002},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":1.7150000000000001,"15min":7.7869999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.23300000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":6.7380000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52100000000000002,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.247},"max":{"1min":0.752,"5min":1.3029999999999999,"15min":12.861000000000001},"last":0.56000000000000005}]},{"osd":3,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.58899999999999997},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.26300000000000001},"max":{"1min":0.73799999999999999,"5min":0.94099999999999995,"15min":7.1360000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":1.2549999999999999,"15min":3.7989999999999999},"last":0.501}]},{"osd":4,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.434,"15min":0.435},"min":{"1min":0.30199999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":1.1779999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.246},"max":{"1min":0.70299999999999996,"5min":0.84099999999999997,"15min":4.7859999999999996},"last":0.34699999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.746,"5min":0.95299999999999996,"15min":1.877},"last":0.379},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47999999999999998,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.26600000000000001},"max":{"1min":0.83099999999999996,"5min":0.85899999999999999,"15min":4.9130000000000003},"last":0.39100000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.9019999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.309,"15min":0.28100000000000003},"max":{"1min":0.77400000000000002,"5min":1.4279999999999999,"15min":5.3860000000000001},"last":0.54200000000000004}]}]},{"osd":6,"up_from":47,"seq":201863463157,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312660,"kb_used_data":7244,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93432492,"statfs":{"total":95995035648,"available":95674871808,"internally_reserved":0,"allocated":7417856,"data_stored":4034116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53100000000000003,"15min":1.272},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.22500000000000001},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":69.194999999999993},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.75},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":1.179,"15min":10.433},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":1.355},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.27900000000000003},"max":{"1min":0.79500000000000004,"5min":1.9610000000000001,"15min":76.108999999999995},"last":0.436},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57999999999999996,"15min":1.3500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70399999999999996,"5min":3.665,"15min":71.872},"last":0.35999999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.83299999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":3.702,"15min":39.939999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":1.276,"15min":39.673000000000002},"last":0.51000000000000001}]},{"osd":3,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.84999999999999998},"min":{"1min":0.38100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70299999999999996,"5min":0.81899999999999995,"15min":21.219000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61399999999999999,"15min":0.81200000000000006},"min":{"1min":0.41499999999999998,"5min":0.33300000000000002,"15min":0.29299999999999998},"max":{"1min":0.81799999999999995,"5min":3.7280000000000002,"15min":14.007999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.52600000000000002},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73599999999999999,"5min":1.2170000000000001,"15min":8.1479999999999997},"last":0.378},{"interface":"front","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.72699999999999998},"min":{"1min":0.34899999999999998,"5min":0.27100000000000002,"15min":0.26100000000000001},"max":{"1min":0.81399999999999995,"5min":4.1310000000000002,"15min":35.18},"last":0.34999999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.55700000000000005},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.67900000000000005,"5min":1.3069999999999999,"15min":10.606},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.753},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":3.8439999999999999,"15min":35.119},"last":0.40100000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:47:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.55600000000000005},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.29399999999999998},"max":{"1min":0.68500000000000005,"5min":1.2470000000000001,"15min":5.4770000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.73399999999999999},"min":{"1min":0.35799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":1.7629999999999999,"15min":32.985999999999997},"last":0.55300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575116,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308564,"kb_used_data":7244,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436588,"statfs":{"total":95995035648,"available":95679066112,"internally_reserved":0,"allocated":7417856,"data_stored":4032899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.63300000000000001},"min":{"1min":0.22,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":0.55400000000000005,"5min":0.64900000000000002,"15min":32.106999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36699999999999999,"15min":0.70299999999999996},"min":{"1min":0.248,"5min":0.247,"15min":0.20899999999999999},"max":{"1min":0.67900000000000005,"5min":0.83499999999999996,"15min":43.137},"last":0.34599999999999997}]},{"osd":2,"last update":"Sun Feb 18 00:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.36199999999999999,"15min":0.752},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.19900000000000001},"max":{"1min":0.51200000000000001,"5min":1.139,"15min":49.201000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39400000000000002,"15min":0.58399999999999996},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.59199999999999997,"5min":0.85899999999999999,"15min":17.596},"last":0.33300000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.71099999999999997},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26200000000000001},"max":{"1min":0.60099999999999998,"5min":0.78700000000000003,"15min":37.524000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.73499999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.26400000000000001},"max":{"1min":0.69499999999999995,"5min":0.97599999999999998,"15min":32.036000000000001},"last":0.38}]},{"osd":4,"last update":"Sun Feb 18 00:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.751},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.29499999999999998},"max":{"1min":0.84299999999999997,"5min":0.91900000000000004,"15min":17.486999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.76400000000000001},"min":{"1min":0.39100000000000001,"5min":0.33500000000000002,"15min":0.32500000000000001},"max":{"1min":0.91000000000000003,"5min":1.222,"15min":12.75},"last":0.72799999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.75800000000000001},"min":{"1min":0.499,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":1.0109999999999999,"15min":12.611000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.61499999999999999,"15min":0.78000000000000003},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.33100000000000002},"max":{"1min":0.92600000000000005,"5min":1.141,"15min":17.417000000000002},"last":0.77300000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.77700000000000002},"min":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":1.1839999999999999,"15min":12.834},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.61199999999999999,"15min":0.77700000000000002},"min":{"1min":0.48899999999999999,"5min":0.37,"15min":0.35499999999999998},"max":{"1min":0.90800000000000003,"5min":1.046,"15min":12.651999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63700000000000001,"15min":0.79700000000000004},"min":{"1min":0.49099999999999999,"5min":0.40500000000000003,"15min":0.38700000000000001},"max":{"1min":0.89600000000000002,"5min":1.175,"15min":17.452000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.79200000000000004},"min":{"1min":0.51400000000000001,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":1.149,"15min":12.805},"last":0.75900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738642,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308560,"kb_used_data":7240,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436592,"statfs":{"total":95995035648,"available":95679070208,"internally_reserved":0,"allocated":7413760,"data_stored":4032405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.372,"15min":0.97899999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.18099999999999999},"max":{"1min":0.503,"5min":1.1619999999999999,"15min":133.38999999999999},"last":0.433},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41799999999999998,"15min":0.79400000000000004},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59699999999999998,"5min":2.6560000000000001,"15min":90.623000000000005},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.438,"15min":0.60599999999999998},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.20200000000000001},"max":{"1min":0.52300000000000002,"5min":2.915,"15min":21.388999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39600000000000002,"15min":0.55800000000000005},"min":{"1min":0.25600000000000001,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":0.52100000000000002,"5min":3.3130000000000002,"15min":23.388999999999999},"last":0.378}]},{"osd":3,"last update":"Sun Feb 18 00:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40799999999999997,"15min":0.63700000000000001},"min":{"1min":0.27200000000000002,"5min":0.23899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54400000000000004,"5min":2.8170000000000002,"15min":19.006},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.443,"5min":0.49399999999999999,"15min":1.0600000000000001},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.23999999999999999},"max":{"1min":0.621,"5min":4.2030000000000003,"15min":146.71899999999999},"last":0.48099999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.68400000000000005},"min":{"1min":0.39300000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":1.885,"15min":12.988},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63100000000000001,"15min":0.70499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.30399999999999999},"max":{"1min":0.83599999999999997,"5min":2.8690000000000002,"15min":11.551},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.64000000000000001,"15min":0.96899999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32000000000000001},"max":{"1min":0.78500000000000003,"5min":4.5499999999999998,"15min":90.049000000000007},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.995},"min":{"1min":0.44,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96699999999999997,"5min":1.853,"15min":57.991},"last":0.73399999999999999}]},{"osd":6,"last update":"Sun Feb 18 00:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.81000000000000005},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.35999999999999999},"max":{"1min":0.94599999999999995,"5min":1.194,"15min":19.039000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66600000000000004,"15min":1.0029999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.33500000000000002},"max":{"1min":0.89700000000000002,"5min":3.3540000000000001,"15min":89.989999999999995},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68999999999999995,"15min":0.79000000000000004},"min":{"1min":0.46500000000000002,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.76200000000000001,"5min":2.9449999999999998,"15min":13.24},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63800000000000001,"15min":0.75600000000000001},"min":{"1min":0.47199999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81799999999999995,"5min":1.911,"15min":13.019},"last":0.67600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411593,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308556,"kb_used_data":7236,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436596,"statfs":{"total":95995035648,"available":95679074304,"internally_reserved":0,"allocated":7409664,"data_stored":4032597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.23899999999999999,"5min":0.20200000000000001,"15min":0.192},"max":{"1min":0.46300000000000002,"5min":8.6489999999999991,"15min":10.087999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.372,"15min":0.55100000000000005},"min":{"1min":0.26300000000000001,"5min":0.23899999999999999,"15min":0.19800000000000001},"max":{"1min":0.50600000000000001,"5min":0.59099999999999997,"15min":18.891999999999999},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38400000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.61199999999999999,"5min":0.85199999999999998,"15min":10.138999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.41599999999999998,"15min":0.52100000000000002},"min":{"1min":0.26600000000000001,"5min":0.246,"15min":0.20399999999999999},"max":{"1min":0.53400000000000003,"5min":1.7869999999999999,"15min":18.919},"last":0.35499999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.50600000000000001,"15min":0.60399999999999998},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.24099999999999999},"max":{"1min":0.60999999999999999,"5min":8.5640000000000001,"15min":22.712},"last":0.315},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.23899999999999999},"max":{"1min":0.61799999999999999,"5min":1.0229999999999999,"15min":22.359000000000002},"last":0.32200000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.67900000000000005},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.77200000000000002,"5min":0.90600000000000003,"15min":18.812000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61099999999999999,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.28699999999999998},"max":{"1min":0.79500000000000004,"5min":1.002,"15min":10.016},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Feb 18 00:48:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.72099999999999997,"15min":0.68799999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.32800000000000001},"max":{"1min":0.81100000000000005,"5min":11.805,"15min":11.805},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.317},"max":{"1min":0.81200000000000006,"5min":8.6059999999999999,"15min":10.114000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Feb 18 00:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.73599999999999999,"15min":0.69599999999999995},"min":{"1min":0.46400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":11.680999999999999,"15min":11.680999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.69799999999999995},"min":{"1min":0.47599999999999998,"5min":0.379,"15min":0.34100000000000003},"max":{"1min":0.77200000000000002,"5min":0.871,"15min":18.861000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.73999999999999999,"15min":0.70199999999999996},"min":{"1min":0.439,"5min":0.39000000000000001,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":11.746,"15min":11.746},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.70899999999999996},"min":{"1min":0.51400000000000001,"5min":0.42599999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":1.115,"15min":10.058999999999999},"last":0.628}]}]},{"osd":3,"up_from":26,"seq":111669149955,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304480,"kb_used_data":7256,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440672,"statfs":{"total":95995035648,"available":95683248128,"internally_reserved":0,"allocated":7430144,"data_stored":4033465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34699999999999998,"15min":0.45400000000000001},"min":{"1min":0.23100000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.495,"5min":0.59299999999999997,"15min":25.082999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.39100000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":0.53900000000000003,"5min":1.8080000000000001,"15min":18.911000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Feb 18 00:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.378,"15min":0.45200000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.495,"5min":0.78400000000000003,"15min":6.8479999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40699999999999997,"15min":0.68700000000000006},"min":{"1min":0.247,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51500000000000001,"5min":2.8570000000000002,"15min":23.606999999999999},"last":0.379}]},{"osd":2,"last update":"Sun Feb 18 00:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.65100000000000002},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.23799999999999999},"max":{"1min":0.53500000000000003,"5min":2.899,"15min":23.405000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.45200000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.54600000000000004,"5min":1.6279999999999999,"15min":5.3479999999999999},"last":0.32600000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":0.68799999999999994},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.33600000000000002},"max":{"1min":0.81899999999999995,"5min":2.9529999999999998,"15min":10.131},"last":0.443},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.61699999999999999},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.32700000000000001},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":5.3049999999999997},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:48:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.32800000000000001},"max":{"1min":0.70899999999999996,"5min":0.92700000000000005,"15min":7.4249999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.43099999999999999,"5min":0.34200000000000003,"15min":0.32400000000000001},"max":{"1min":0.75800000000000001,"5min":0.98999999999999999,"15min":6.7750000000000004},"last":0.39400000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":2.9830000000000001,"15min":10.189},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.39000000000000001},"max":{"1min":0.81399999999999995,"5min":0.97799999999999998,"15min":6.4279999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.95399999999999996,"15min":6.8220000000000001},"last":0.433},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65300000000000002,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88600000000000001,"5min":2.9300000000000002,"15min":10.164999999999999},"last":0.499}]}]},{"osd":4,"up_from":31,"seq":133143986430,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312652,"kb_used_data":7236,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93432500,"statfs":{"total":95995035648,"available":95674880000,"internally_reserved":0,"allocated":7409664,"data_stored":4033184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54600000000000004,"15min":0.84399999999999997},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.247},"max":{"1min":0.76500000000000001,"5min":2.2959999999999998,"15min":77.581999999999994},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.30399999999999999,"15min":0.26100000000000001},"max":{"1min":0.66900000000000004,"5min":0.875,"15min":78.063999999999993},"last":0.42699999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56000000000000005,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.27300000000000002},"max":{"1min":0.76200000000000001,"5min":1.3879999999999999,"15min":9.3160000000000007},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54100000000000004,"15min":0.874},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.29699999999999999},"max":{"1min":0.82099999999999995,"5min":0.97799999999999998,"15min":81.242000000000004},"last":0.45600000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.61399999999999999},"min":{"1min":0.38100000000000001,"5min":0.28599999999999998,"15min":0.27900000000000003},"max":{"1min":0.79800000000000004,"5min":0.94799999999999995,"15min":11.691000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.64300000000000002,"5min":0.90700000000000003,"15min":17.77},"last":0.55500000000000005}]},{"osd":3,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.29599999999999999},"max":{"1min":0.69299999999999995,"5min":0.97999999999999998,"15min":7.218},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.72999999999999998},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.309},"max":{"1min":0.85799999999999998,"5min":1.3600000000000001,"15min":33.530999999999999},"last":0.47499999999999998}]},{"osd":5,"last update":"Sun Feb 18 00:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.45100000000000001},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.23799999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.621},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.46000000000000002,"15min":0.45300000000000001},"min":{"1min":0.33100000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.67400000000000004,"5min":1.45,"15min":1.45},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.45400000000000001},"min":{"1min":0.35599999999999998,"5min":0.245,"15min":0.221},"max":{"1min":0.68500000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.505},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.503},"min":{"1min":0.376,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68999999999999995,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Feb 18 00:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.40500000000000003,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":0.72499999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.374,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.81100000000000005},"last":0.61199999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724793,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312632,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93432520,"statfs":{"total":95995035648,"available":95674900480,"internally_reserved":0,"allocated":7389184,"data_stored":4032895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.50700000000000001,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.27300000000000002,"15min":0.217},"max":{"1min":0.90200000000000002,"5min":1.3380000000000001,"15min":6.3150000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.60699999999999998},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.26000000000000001},"max":{"1min":0.90900000000000003,"5min":1.123,"15min":12.744},"last":0.51400000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56200000000000006,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.27400000000000002},"max":{"1min":1.0029999999999999,"5min":1.2010000000000001,"15min":13.25},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.54700000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.311,"15min":0.30099999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":13.262},"last":0.56699999999999995}]},{"osd":2,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.58899999999999997},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.251},"max":{"1min":0.94899999999999995,"5min":1.4510000000000001,"15min":6.1680000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.377,"15min":0.308},"max":{"1min":0.88800000000000001,"5min":1.4370000000000001,"15min":6.1879999999999997},"last":0.61399999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52300000000000002,"15min":0.61899999999999999},"min":{"1min":0.47899999999999998,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":7.641},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.60399999999999998},"min":{"1min":0.46700000000000003,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.83699999999999997,"5min":1.22,"15min":8.8650000000000002},"last":0.64100000000000001}]},{"osd":4,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.497},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.23599999999999999},"max":{"1min":0.77300000000000002,"5min":0.79400000000000004,"15min":11.818},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.23300000000000001},"max":{"1min":0.68899999999999995,"5min":0.80700000000000005,"15min":4.274},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Feb 18 00:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47499999999999998,"15min":0.56799999999999995},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.26300000000000001},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":18.605},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.51600000000000001},"min":{"1min":0.35599999999999998,"5min":0.27200000000000002,"15min":0.26500000000000001},"max":{"1min":0.75600000000000001,"5min":1.0629999999999999,"15min":4.2089999999999996},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Feb 18 00:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35399999999999998,"5min":0.311,"15min":0.26600000000000001},"max":{"1min":0.73799999999999999,"5min":1.2370000000000001,"15min":2.0739999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.28299999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.649},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:48:50.853 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T00:48:51.142 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:51.342 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:48:51.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:49.456260+0000 osd.3 (osd.3) 521 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:49.457306+0000 osd.3 (osd.3) 522 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:49.657089+0000 osd.1 (osd.1) 463 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:49.658997+0000 osd.1 (osd.1) 464 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:49.906988+0000 mgr.y (mgr.24454) 784 : audit [DBG] from='client.32322 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:50.080940+0000 mgr.y (mgr.24454) 785 : cluster [DBG] pgmap v1368: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 914 B/s rd, 0 op/s 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:50.140096+0000 osd.0 (osd.0) 434 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:50.141240+0000 osd.0 (osd.0) 435 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:50.656645+0000 osd.1 (osd.1) 465 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:50.657644+0000 osd.1 (osd.1) 466 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:48:51.697 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:49.456260+0000 osd.3 (osd.3) 521 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:49.457306+0000 osd.3 (osd.3) 522 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:49.657089+0000 osd.1 (osd.1) 463 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:49.658997+0000 osd.1 (osd.1) 464 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:49.906988+0000 mgr.y (mgr.24454) 784 : audit [DBG] from='client.32322 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:50.080940+0000 mgr.y (mgr.24454) 785 : cluster [DBG] pgmap v1368: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 914 B/s rd, 0 op/s 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:50.140096+0000 osd.0 (osd.0) 434 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:50.141240+0000 osd.0 (osd.0) 435 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:50.656645+0000 osd.1 (osd.1) 465 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:48:51.698 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:51 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:50.657644+0000 osd.1 (osd.1) 466 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:48:51.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:49.456260+0000 osd.3 (osd.3) 521 : cluster [DBG] 2.19 scrub starts 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:49.457306+0000 osd.3 (osd.3) 522 : cluster [DBG] 2.19 scrub ok 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:49.657089+0000 osd.1 (osd.1) 463 : cluster [DBG] 4.7 scrub starts 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:49.658997+0000 osd.1 (osd.1) 464 : cluster [DBG] 4.7 scrub ok 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:49.906988+0000 mgr.y (mgr.24454) 784 : audit [DBG] from='client.32322 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:50.080940+0000 mgr.y (mgr.24454) 785 : cluster [DBG] pgmap v1368: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 914 B/s rd, 0 op/s 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:50.140096+0000 osd.0 (osd.0) 434 : cluster [DBG] 2.1f scrub starts 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:50.141240+0000 osd.0 (osd.0) 435 : cluster [DBG] 2.1f scrub ok 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:50.656645+0000 osd.1 (osd.1) 465 : cluster [DBG] 5.1d scrub starts 2024-02-18T00:48:51.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:50.657644+0000 osd.1 (osd.1) 466 : cluster [DBG] 5.1d scrub ok 2024-02-18T00:48:52.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:48:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:48:52.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:51.117680+0000 osd.5 (osd.5) 456 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:48:52.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:51.121151+0000 osd.5 (osd.5) 457 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:48:52.502 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:48:52] "GET /metrics HTTP/1.1" 200 187530 "" "Prometheus/2.33.4" 2024-02-18T00:48:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:51.117680+0000 osd.5 (osd.5) 456 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:48:52.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:52 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:51.121151+0000 osd.5 (osd.5) 457 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:48:52.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:51.117680+0000 osd.5 (osd.5) 456 : cluster [DBG] 4.25 scrub starts 2024-02-18T00:48:52.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:51.121151+0000 osd.5 (osd.5) 457 : cluster [DBG] 4.25 scrub ok 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.204+0000 7feed7661700 1 -- 172.21.15.59:0/206192630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 msgr2=0x7feed0103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.204+0000 7feed7661700 1 --2- 172.21.15.59:0/206192630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed0103a20 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 rx=0x7feec8009c30 tx=0x7feec8004c50).stop 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.204+0000 7feed7661700 1 -- 172.21.15.59:0/206192630 shutdown_connections 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.204+0000 7feed7661700 1 --2- 172.21.15.59:0/206192630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed00fcbb0 0x7feed00fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.204+0000 7feed7661700 1 --2- 172.21.15.59:0/206192630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed0103a20 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.204+0000 7feed7661700 1 --2- 172.21.15.59:0/206192630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feed00fd510 0x7feed00f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.204+0000 7feed7661700 1 -- 172.21.15.59:0/206192630 >> 172.21.15.59:0/206192630 conn(0x7feed00f2820 msgr2=0x7feed00f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 -- 172.21.15.59:0/206192630 shutdown_connections 2024-02-18T00:48:53.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 -- 172.21.15.59:0/206192630 wait complete. 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 librados: starting msgr at 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 librados: starting objecter 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 Processor -- start 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 -- start start 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 librados: setting wanted keys 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 librados: calling monclient init 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed010df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed00fcbb0 0x7feed010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feed00fd510 0x7feed01080a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0108590 con 0x7feed00f7400 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0108700 con 0x7feed00fcbb0 2024-02-18T00:48:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed7661700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed01088c0 con 0x7feed00fd510 2024-02-18T00:48:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.205+0000 7feed53fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed010df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed010df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55072/0 (socket says 172.21.15.59:55072) 2024-02-18T00:48:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 -- 172.21.15.59:0/2296385990 learned_addr learned my addr 172.21.15.59:0/2296385990 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed4bfc700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed00fcbb0 0x7feed010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed5bfe700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feed00fd510 0x7feed01080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 -- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feed00fd510 msgr2=0x7feed01080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feed00fd510 0x7feed01080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 -- 172.21.15.59:0/2296385990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed00fcbb0 msgr2=0x7feed010e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed00fcbb0 0x7feed010e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 -- 172.21.15.59:0/2296385990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80097c0 con 0x7feed00f7400 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed53fd700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed010df00 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 rx=0x7feecc00c7c0 tx=0x7feecc007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feec27fc700 10 client.?.objecter ms_handle_connect 0x7feed00f7400 2024-02-18T00:48:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feec27fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feec27fc700 1 -- 172.21.15.59:0/2296385990 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc004430 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feec27fc700 1 -- 172.21.15.59:0/2296385990 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feecc00e970 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed0108ba0 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feec27fc700 1 -- 172.21.15.59:0/2296385990 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc013a60 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed7661700 10 client.32328.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed0108f20 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.206+0000 7feed7661700 1 librados: init done 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.207+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed005fa60 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 1 -- 172.21.15.59:0/2296385990 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7feecc00eae0 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 1 --2- 172.21.15.59:0/2296385990 >> v2:172.21.15.59:6800/3721891797 conn(0x7feebc062c90 0x7feebc065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 1 -- 172.21.15.59:0/2296385990 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(855..855 src has 418..855) v4 ==== 7613+0+0 (secure 0 0 0) 0x7feecc07cc70 con 0x7feed00f7400 2024-02-18T00:48:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 10 client.32328.objecter ms_dispatch 0x7feed0001550 osd_map(855..855 src has 418..855) v4 2024-02-18T00:48:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 3 client.32328.objecter handle_osd_map got epochs [855,855] > 0 2024-02-18T00:48:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 3 client.32328.objecter handle_osd_map decoding full epoch 855 2024-02-18T00:48:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feed4bfc700 1 --2- 172.21.15.59:0/2296385990 >> v2:172.21.15.59:6800/3721891797 conn(0x7feebc062c90 0x7feebc065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 20 client.32328.objecter dump_active .. 0 homeless 2024-02-18T00:48:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feec27fc700 10 client.32328.objecter ms_handle_connect 0x7feebc062c90 2024-02-18T00:48:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.208+0000 7feed4bfc700 1 --2- 172.21.15.59:0/2296385990 >> v2:172.21.15.59:6800/3721891797 conn(0x7feebc062c90 0x7feebc065150 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 rx=0x7feed0109840 tx=0x7feec400b040).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.212+0000 7feec27fc700 1 -- 172.21.15.59:0/2296385990 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7feecc050340 con 0x7feed00f7400 2024-02-18T00:48:53.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.382+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 --> v2:172.21.15.59:6800/3721891797 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feed0109580 con 0x7feebc062c90 2024-02-18T00:48:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.394+0000 7feec27fc700 1 -- 172.21.15.59:0/2296385990 <== mgr.24454 v2:172.21.15.59:6800/3721891797 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+454628 (secure 0 0 0) 0x7feed0109580 con 0x7feebc062c90 2024-02-18T00:48:53.402 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-02-18T00:48:53.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 10 librados: watch_flush enter 2024-02-18T00:48:53.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 10 librados: watch_flush exit 2024-02-18T00:48:53.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 20 client.32328.objecter shutdown clearing up homeless session... 2024-02-18T00:48:53.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 10 client.32328.objecter successfully canceled tick 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 >> v2:172.21.15.59:6800/3721891797 conn(0x7feebc062c90 msgr2=0x7feebc065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 --2- 172.21.15.59:0/2296385990 >> v2:172.21.15.59:6800/3721891797 conn(0x7feebc062c90 0x7feebc065150 secure :-1 s=READY pgs=3284 cs=0 l=1 rev1=1 rx=0x7feed0109840 tx=0x7feec400b040).stop 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 msgr2=0x7feed010df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed010df00 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 rx=0x7feecc00c7c0 tx=0x7feecc007ef0).stop 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 shutdown_connections 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feed00fd510 0x7feed01080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feed00f7400 0x7feed010df00 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 --2- 172.21.15.59:0/2296385990 >> v2:172.21.15.59:6800/3721891797 conn(0x7feebc062c90 0x7feebc065150 unknown :-1 s=CLOSED pgs=3284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 --2- 172.21.15.59:0/2296385990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feed00fcbb0 0x7feed010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:53.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 >> 172.21.15.59:0/2296385990 conn(0x7feed00f2820 msgr2=0x7feed00f44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:53.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 shutdown_connections 2024-02-18T00:48:53.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 -- 172.21.15.59:0/2296385990 wait complete. 2024-02-18T00:48:53.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:53.398+0000 7feed7661700 1 librados: shutdown 2024-02-18T00:48:53.408 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-02-18T00:48:53.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:51.780877+0000 mgr.y (mgr.24454) 786 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:53.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:52.081760+0000 mgr.y (mgr.24454) 787 : cluster [DBG] pgmap v1369: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:53.751 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:52.131907+0000 osd.0 (osd.0) 436 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:48:53.752 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:52.133750+0000 osd.0 (osd.0) 437 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:48:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:51.780877+0000 mgr.y (mgr.24454) 786 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:52.081760+0000 mgr.y (mgr.24454) 787 : cluster [DBG] pgmap v1369: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:52.131907+0000 osd.0 (osd.0) 436 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:48:53.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:52.133750+0000 osd.0 (osd.0) 437 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:48:53.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:51.780877+0000 mgr.y (mgr.24454) 786 : audit [DBG] from='client.24430 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T00:48:53.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:52.081760+0000 mgr.y (mgr.24454) 787 : cluster [DBG] pgmap v1369: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T00:48:53.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:52.131907+0000 osd.0 (osd.0) 436 : cluster [DBG] 25.7 deep-scrub starts 2024-02-18T00:48:53.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:52.133750+0000 osd.0 (osd.0) 437 : cluster [DBG] 25.7 deep-scrub ok 2024-02-18T00:48:54.177 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":1369,"stamp":"2024-02-18T00:48:52.081175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5801,"num_objects":225,"num_object_clones":0,"num_object_copies":675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":225,"num_whiteouts":0,"num_read":2125,"num_read_kb":1954,"num_write":579,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":8376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1425,"ondisk_log_size":1425,"up":522,"acting":522,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":522,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2472568,"kb_used_data":57912,"kb_used_omap":0,"kb_used_meta":2414592,"kb_avail":747488648,"statfs":{"total":767960285184,"available":765428375552,"internally_reserved":0,"allocated":59301888,"data_stored":32264243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2472542208},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003561"},"pg_stats":[{"pgid":"4.29","version":"731'34","reported_seq":740,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.376870+0000","last_change":"2024-02-18T00:48:19.661865+0000","last_active":"2024-02-18T00:48:36.376870+0000","last_peered":"2024-02-18T00:48:36.376870+0000","last_clean":"2024-02-18T00:48:36.376870+0000","last_became_active":"2024-02-18T00:43:09.798744+0000","last_became_peered":"2024-02-18T00:43:09.798744+0000","last_unstale":"2024-02-18T00:48:36.376870+0000","last_undegraded":"2024-02-18T00:48:36.376870+0000","last_fullsized":"2024-02-18T00:48:36.376870+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"731'34","last_scrub_stamp":"2024-02-18T00:48:19.661781+0000","last_deep_scrub":"731'34","last_deep_scrub_stamp":"2024-02-18T00:47:02.906198+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.661781+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"457'39","reported_seq":777,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291279+0000","last_change":"2024-02-18T00:48:19.145476+0000","last_active":"2024-02-18T00:48:36.291279+0000","last_peered":"2024-02-18T00:48:36.291279+0000","last_clean":"2024-02-18T00:48:36.291279+0000","last_became_active":"2024-02-18T00:43:09.791383+0000","last_became_peered":"2024-02-18T00:43:09.791383+0000","last_unstale":"2024-02-18T00:48:36.291279+0000","last_undegraded":"2024-02-18T00:48:36.291279+0000","last_fullsized":"2024-02-18T00:48:36.291279+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:48:19.145389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.145389+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.29","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795447+0000","last_change":"2024-02-18T00:47:44.714703+0000","last_active":"2024-02-18T00:48:36.795447+0000","last_peered":"2024-02-18T00:48:36.795447+0000","last_clean":"2024-02-18T00:48:36.795447+0000","last_became_active":"2024-02-18T00:46:25.889860+0000","last_became_peered":"2024-02-18T00:46:25.889860+0000","last_unstale":"2024-02-18T00:48:36.795447+0000","last_undegraded":"2024-02-18T00:48:36.795447+0000","last_fullsized":"2024-02-18T00:48:36.795447+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:44.714596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:20.213170+0000","last_clean_scrub_stamp":"2024-02-18T00:47:44.714596+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.28","version":"0'0","reported_seq":790,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.376839+0000","last_change":"2024-02-18T00:47:46.687503+0000","last_active":"2024-02-18T00:48:36.376839+0000","last_peered":"2024-02-18T00:48:36.376839+0000","last_clean":"2024-02-18T00:48:36.376839+0000","last_became_active":"2024-02-18T00:39:36.765689+0000","last_became_peered":"2024-02-18T00:39:36.765689+0000","last_unstale":"2024-02-18T00:48:36.376839+0000","last_undegraded":"2024-02-18T00:48:36.376839+0000","last_fullsized":"2024-02-18T00:48:36.376839+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:46.687418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:46.687418+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.21","version":"457'34","reported_seq":211,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794006+0000","last_change":"2024-02-18T00:47:42.685385+0000","last_active":"2024-02-18T00:48:36.794006+0000","last_peered":"2024-02-18T00:48:36.794006+0000","last_clean":"2024-02-18T00:48:36.794006+0000","last_became_active":"2024-02-18T00:43:44.817896+0000","last_became_peered":"2024-02-18T00:43:44.817896+0000","last_unstale":"2024-02-18T00:48:36.794006+0000","last_undegraded":"2024-02-18T00:48:36.794006+0000","last_fullsized":"2024-02-18T00:48:36.794006+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:47:42.685299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:42.685299+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.27","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292786+0000","last_change":"2024-02-18T00:48:10.207239+0000","last_active":"2024-02-18T00:48:36.292786+0000","last_peered":"2024-02-18T00:48:36.292786+0000","last_clean":"2024-02-18T00:48:36.292786+0000","last_became_active":"2024-02-18T00:39:36.760155+0000","last_became_peered":"2024-02-18T00:39:36.760155+0000","last_unstale":"2024-02-18T00:48:36.292786+0000","last_undegraded":"2024-02-18T00:48:36.292786+0000","last_fullsized":"2024-02-18T00:48:36.292786+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:10.207161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.593401+0000","last_clean_scrub_stamp":"2024-02-18T00:48:10.207161+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.20","version":"457'52","reported_seq":76,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794465+0000","last_change":"2024-02-18T00:48:17.497291+0000","last_active":"2024-02-18T00:48:36.794465+0000","last_peered":"2024-02-18T00:48:36.794465+0000","last_clean":"2024-02-18T00:48:36.794465+0000","last_became_active":"2024-02-18T00:46:25.883556+0000","last_became_peered":"2024-02-18T00:46:25.883556+0000","last_unstale":"2024-02-18T00:48:36.794465+0000","last_undegraded":"2024-02-18T00:48:36.794465+0000","last_fullsized":"2024-02-18T00:48:36.794465+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:48:17.497210+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:48:17.497210+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.26","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292682+0000","last_change":"2024-02-18T00:48:11.198777+0000","last_active":"2024-02-18T00:48:36.292682+0000","last_peered":"2024-02-18T00:48:36.292682+0000","last_clean":"2024-02-18T00:48:36.292682+0000","last_became_active":"2024-02-18T00:46:25.880548+0000","last_became_peered":"2024-02-18T00:46:25.880548+0000","last_unstale":"2024-02-18T00:48:36.292682+0000","last_undegraded":"2024-02-18T00:48:36.292682+0000","last_fullsized":"2024-02-18T00:48:36.292682+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:11.198674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:11.198674+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.198674+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.23","version":"734'38","reported_seq":248,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377089+0000","last_change":"2024-02-18T00:48:23.565361+0000","last_active":"2024-02-18T00:48:36.377089+0000","last_peered":"2024-02-18T00:48:36.377089+0000","last_clean":"2024-02-18T00:48:36.377089+0000","last_became_active":"2024-02-18T00:43:09.796063+0000","last_became_peered":"2024-02-18T00:43:09.796063+0000","last_unstale":"2024-02-18T00:48:36.377089+0000","last_undegraded":"2024-02-18T00:48:36.377089+0000","last_fullsized":"2024-02-18T00:48:36.377089+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"734'38","last_scrub_stamp":"2024-02-18T00:48:23.565288+0000","last_deep_scrub":"734'38","last_deep_scrub_stamp":"2024-02-18T00:44:43.235973+0000","last_clean_scrub_stamp":"2024-02-18T00:48:23.565288+0000","log_size":38,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.25","version":"57'1","reported_seq":824,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407981+0000","last_change":"2024-02-18T00:48:01.333278+0000","last_active":"2024-02-18T00:48:36.407981+0000","last_peered":"2024-02-18T00:48:36.407981+0000","last_clean":"2024-02-18T00:48:36.407981+0000","last_became_active":"2024-02-18T00:46:25.870529+0000","last_became_peered":"2024-02-18T00:46:25.870529+0000","last_unstale":"2024-02-18T00:48:36.407981+0000","last_undegraded":"2024-02-18T00:48:36.407981+0000","last_fullsized":"2024-02-18T00:48:36.407981+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:48:01.333171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.333171+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1746,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.22","version":"457'32","reported_seq":735,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406594+0000","last_change":"2024-02-18T00:47:54.381756+0000","last_active":"2024-02-18T00:48:36.406594+0000","last_peered":"2024-02-18T00:48:36.406594+0000","last_clean":"2024-02-18T00:48:36.406594+0000","last_became_active":"2024-02-18T00:46:25.878558+0000","last_became_peered":"2024-02-18T00:46:25.878558+0000","last_unstale":"2024-02-18T00:48:36.406594+0000","last_undegraded":"2024-02-18T00:48:36.406594+0000","last_fullsized":"2024-02-18T00:48:36.406594+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:47:54.381655+0000","last_deep_scrub":"457'32","last_deep_scrub_stamp":"2024-02-18T00:47:54.381655+0000","last_clean_scrub_stamp":"2024-02-18T00:47:54.381655+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.24","version":"0'0","reported_seq":66,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:42.562546+0000","last_change":"2024-02-18T00:48:42.562546+0000","last_active":"2024-02-18T00:48:42.562546+0000","last_peered":"2024-02-18T00:48:42.562546+0000","last_clean":"2024-02-18T00:48:42.562546+0000","last_became_active":"2024-02-18T00:46:25.888934+0000","last_became_peered":"2024-02-18T00:46:25.888934+0000","last_unstale":"2024-02-18T00:48:42.562546+0000","last_undegraded":"2024-02-18T00:48:42.562546+0000","last_fullsized":"2024-02-18T00:48:42.562546+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:42.562471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:42.562471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.25","version":"457'26","reported_seq":249,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794404+0000","last_change":"2024-02-18T00:47:20.580387+0000","last_active":"2024-02-18T00:48:36.794404+0000","last_peered":"2024-02-18T00:48:36.794404+0000","last_clean":"2024-02-18T00:48:36.794404+0000","last_became_active":"2024-02-18T00:43:09.791838+0000","last_became_peered":"2024-02-18T00:43:09.791838+0000","last_unstale":"2024-02-18T00:48:36.794404+0000","last_undegraded":"2024-02-18T00:48:36.794404+0000","last_fullsized":"2024-02-18T00:48:36.794404+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:47:20.580303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:20.580303+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.23","version":"0'0","reported_seq":795,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.456562+0000","last_change":"2024-02-18T00:48:39.456562+0000","last_active":"2024-02-18T00:48:39.456562+0000","last_peered":"2024-02-18T00:48:39.456562+0000","last_clean":"2024-02-18T00:48:39.456562+0000","last_became_active":"2024-02-18T00:46:25.878862+0000","last_became_peered":"2024-02-18T00:46:25.878862+0000","last_unstale":"2024-02-18T00:48:39.456562+0000","last_undegraded":"2024-02-18T00:48:39.456562+0000","last_fullsized":"2024-02-18T00:48:39.456562+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:39.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.456491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"457'47","reported_seq":289,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:46.693362+0000","last_change":"2024-02-18T00:48:46.693362+0000","last_active":"2024-02-18T00:48:46.693362+0000","last_peered":"2024-02-18T00:48:46.693362+0000","last_clean":"2024-02-18T00:48:46.693362+0000","last_became_active":"2024-02-18T00:43:09.789985+0000","last_became_peered":"2024-02-18T00:43:09.789985+0000","last_unstale":"2024-02-18T00:48:46.693362+0000","last_undegraded":"2024-02-18T00:48:46.693362+0000","last_fullsized":"2024-02-18T00:48:46.693362+0000","mapping_epoch":643,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":644,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'47","last_scrub_stamp":"2024-02-18T00:48:46.693291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:46.693291+0000","log_size":47,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.22","version":"0'0","reported_seq":66,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:46.657542+0000","last_change":"2024-02-18T00:48:46.657542+0000","last_active":"2024-02-18T00:48:46.657542+0000","last_peered":"2024-02-18T00:48:46.657542+0000","last_clean":"2024-02-18T00:48:46.657542+0000","last_became_active":"2024-02-18T00:46:25.893401+0000","last_became_peered":"2024-02-18T00:46:25.893401+0000","last_unstale":"2024-02-18T00:48:46.657542+0000","last_undegraded":"2024-02-18T00:48:46.657542+0000","last_fullsized":"2024-02-18T00:48:46.657542+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:46.657420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:07.532776+0000","last_clean_scrub_stamp":"2024-02-18T00:48:46.657420+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.27","version":"457'23","reported_seq":950,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292719+0000","last_change":"2024-02-18T00:47:38.765285+0000","last_active":"2024-02-18T00:48:36.292719+0000","last_peered":"2024-02-18T00:48:36.292719+0000","last_clean":"2024-02-18T00:48:36.292719+0000","last_became_active":"2024-02-18T00:46:25.877765+0000","last_became_peered":"2024-02-18T00:46:25.877765+0000","last_unstale":"2024-02-18T00:48:36.292719+0000","last_undegraded":"2024-02-18T00:48:36.292719+0000","last_fullsized":"2024-02-18T00:48:36.292719+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"457'23","last_scrub_stamp":"2024-02-18T00:47:38.765173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:38.765173+0000","log_size":23,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.21","version":"0'0","reported_seq":875,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292347+0000","last_change":"2024-02-18T00:48:27.755955+0000","last_active":"2024-02-18T00:48:36.292347+0000","last_peered":"2024-02-18T00:48:36.292347+0000","last_clean":"2024-02-18T00:48:36.292347+0000","last_became_active":"2024-02-18T00:39:36.763203+0000","last_became_peered":"2024-02-18T00:39:36.763203+0000","last_unstale":"2024-02-18T00:48:36.292347+0000","last_undegraded":"2024-02-18T00:48:36.292347+0000","last_fullsized":"2024-02-18T00:48:36.292347+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:27.755869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:25.770128+0000","last_clean_scrub_stamp":"2024-02-18T00:48:27.755869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"855'27","reported_seq":742,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.620048+0000","last_change":"2024-02-18T00:48:39.620048+0000","last_active":"2024-02-18T00:48:39.620048+0000","last_peered":"2024-02-18T00:48:39.620048+0000","last_clean":"2024-02-18T00:48:39.620048+0000","last_became_active":"2024-02-18T00:46:25.878258+0000","last_became_peered":"2024-02-18T00:46:25.878258+0000","last_unstale":"2024-02-18T00:48:39.620048+0000","last_undegraded":"2024-02-18T00:48:39.620048+0000","last_fullsized":"2024-02-18T00:48:39.620048+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":641,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"855'27","last_scrub_stamp":"2024-02-18T00:48:39.619977+0000","last_deep_scrub":"855'27","last_deep_scrub_stamp":"2024-02-18T00:48:39.619977+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.619977+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.20","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291527+0000","last_change":"2024-02-18T00:48:13.227039+0000","last_active":"2024-02-18T00:48:36.291527+0000","last_peered":"2024-02-18T00:48:36.291527+0000","last_clean":"2024-02-18T00:48:36.291527+0000","last_became_active":"2024-02-18T00:46:25.886018+0000","last_became_peered":"2024-02-18T00:46:25.886018+0000","last_unstale":"2024-02-18T00:48:36.291527+0000","last_undegraded":"2024-02-18T00:48:36.291527+0000","last_fullsized":"2024-02-18T00:48:36.291527+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":6,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:13.226976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:51.582542+0000","last_clean_scrub_stamp":"2024-02-18T00:48:13.226976+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":805,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:48.199539+0000","last_change":"2024-02-18T00:48:48.199539+0000","last_active":"2024-02-18T00:48:48.199539+0000","last_peered":"2024-02-18T00:48:48.199539+0000","last_clean":"2024-02-18T00:48:48.199539+0000","last_became_active":"2024-02-18T00:39:36.770927+0000","last_became_peered":"2024-02-18T00:39:36.770927+0000","last_unstale":"2024-02-18T00:48:48.199539+0000","last_undegraded":"2024-02-18T00:48:48.199539+0000","last_fullsized":"2024-02-18T00:48:48.199539+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:48.199469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:11.626025+0000","last_clean_scrub_stamp":"2024-02-18T00:48:48.199469+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"811'13","reported_seq":550,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.497096+0000","last_change":"2024-02-18T00:48:12.217031+0000","last_active":"2024-02-18T00:48:43.497096+0000","last_peered":"2024-02-18T00:48:43.497096+0000","last_clean":"2024-02-18T00:48:43.497096+0000","last_became_active":"2024-02-18T00:46:25.884361+0000","last_became_peered":"2024-02-18T00:46:25.884361+0000","last_unstale":"2024-02-18T00:48:43.497096+0000","last_undegraded":"2024-02-18T00:48:43.497096+0000","last_fullsized":"2024-02-18T00:48:43.497096+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"811'13","last_scrub_stamp":"2024-02-18T00:48:12.216946+0000","last_deep_scrub":"736'11","last_deep_scrub_stamp":"2024-02-18T00:44:45.265972+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.216946+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"457'39","reported_seq":1016,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793941+0000","last_change":"2024-02-18T00:47:27.585765+0000","last_active":"2024-02-18T00:48:36.793941+0000","last_peered":"2024-02-18T00:48:36.793941+0000","last_clean":"2024-02-18T00:48:36.793941+0000","last_became_active":"2024-02-18T00:46:25.878115+0000","last_became_peered":"2024-02-18T00:46:25.878115+0000","last_unstale":"2024-02-18T00:48:36.793941+0000","last_undegraded":"2024-02-18T00:48:36.793941+0000","last_fullsized":"2024-02-18T00:48:36.793941+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'39","last_scrub_stamp":"2024-02-18T00:47:27.585659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:27.585659+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":868,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293409+0000","last_change":"2024-02-18T00:48:30.801362+0000","last_active":"2024-02-18T00:48:36.293409+0000","last_peered":"2024-02-18T00:48:36.293409+0000","last_clean":"2024-02-18T00:48:36.293409+0000","last_became_active":"2024-02-18T00:39:36.758885+0000","last_became_peered":"2024-02-18T00:39:36.758885+0000","last_unstale":"2024-02-18T00:48:36.293409+0000","last_undegraded":"2024-02-18T00:48:36.293409+0000","last_fullsized":"2024-02-18T00:48:36.293409+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:30.801291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:30.801291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":870,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794849+0000","last_change":"2024-02-18T00:47:39.735766+0000","last_active":"2024-02-18T00:48:36.794849+0000","last_peered":"2024-02-18T00:48:36.794849+0000","last_clean":"2024-02-18T00:48:36.794849+0000","last_became_active":"2024-02-18T00:46:25.879142+0000","last_became_peered":"2024-02-18T00:46:25.879142+0000","last_unstale":"2024-02-18T00:48:36.794849+0000","last_undegraded":"2024-02-18T00:48:36.794849+0000","last_fullsized":"2024-02-18T00:48:36.794849+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:39.735677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:39.735677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":790,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292210+0000","last_change":"2024-02-18T00:48:18.800902+0000","last_active":"2024-02-18T00:48:36.292210+0000","last_peered":"2024-02-18T00:48:36.292210+0000","last_clean":"2024-02-18T00:48:36.292210+0000","last_became_active":"2024-02-18T00:33:02.534427+0000","last_became_peered":"2024-02-18T00:33:02.534427+0000","last_unstale":"2024-02-18T00:48:36.292210+0000","last_undegraded":"2024-02-18T00:48:36.292210+0000","last_fullsized":"2024-02-18T00:48:36.292210+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:18.800826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.800826+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"457'21","reported_seq":903,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.138531+0000","last_change":"2024-02-18T00:48:43.138531+0000","last_active":"2024-02-18T00:48:43.138531+0000","last_peered":"2024-02-18T00:48:43.138531+0000","last_clean":"2024-02-18T00:48:43.138531+0000","last_became_active":"2024-02-18T00:35:42.679215+0000","last_became_peered":"2024-02-18T00:35:42.679215+0000","last_unstale":"2024-02-18T00:48:43.138531+0000","last_undegraded":"2024-02-18T00:48:43.138531+0000","last_fullsized":"2024-02-18T00:48:43.138531+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'21","last_scrub_stamp":"2024-02-18T00:48:43.138469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.138469+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292928+0000","last_change":"2024-02-18T00:47:35.733560+0000","last_active":"2024-02-18T00:48:36.292928+0000","last_peered":"2024-02-18T00:48:36.292928+0000","last_clean":"2024-02-18T00:48:36.292928+0000","last_became_active":"2024-02-18T00:29:14.714677+0000","last_became_peered":"2024-02-18T00:29:14.714677+0000","last_unstale":"2024-02-18T00:48:36.292928+0000","last_undegraded":"2024-02-18T00:48:36.292928+0000","last_fullsized":"2024-02-18T00:48:36.292928+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:35.733490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:03.930967+0000","last_clean_scrub_stamp":"2024-02-18T00:47:35.733490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":844,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293480+0000","last_change":"2024-02-18T00:48:25.759864+0000","last_active":"2024-02-18T00:48:36.293480+0000","last_peered":"2024-02-18T00:48:36.293480+0000","last_clean":"2024-02-18T00:48:36.293480+0000","last_became_active":"2024-02-18T00:29:19.098313+0000","last_became_peered":"2024-02-18T00:29:19.098313+0000","last_unstale":"2024-02-18T00:48:36.293480+0000","last_undegraded":"2024-02-18T00:48:36.293480+0000","last_fullsized":"2024-02-18T00:48:36.293480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:25.759786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:25.759786+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":889,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793669+0000","last_change":"2024-02-18T00:47:25.113306+0000","last_active":"2024-02-18T00:48:36.793669+0000","last_peered":"2024-02-18T00:48:36.793669+0000","last_clean":"2024-02-18T00:48:36.793669+0000","last_became_active":"2024-02-18T00:43:35.060523+0000","last_became_peered":"2024-02-18T00:43:35.060523+0000","last_unstale":"2024-02-18T00:48:36.793669+0000","last_undegraded":"2024-02-18T00:48:36.793669+0000","last_fullsized":"2024-02-18T00:48:36.793669+0000","mapping_epoch":668,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":669,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:25.113237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:25.113237+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":792,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.378015+0000","last_change":"2024-02-18T00:47:59.708632+0000","last_active":"2024-02-18T00:48:36.378015+0000","last_peered":"2024-02-18T00:48:36.378015+0000","last_clean":"2024-02-18T00:48:36.378015+0000","last_became_active":"2024-02-18T00:39:36.762796+0000","last_became_peered":"2024-02-18T00:39:36.762796+0000","last_unstale":"2024-02-18T00:48:36.378015+0000","last_undegraded":"2024-02-18T00:48:36.378015+0000","last_fullsized":"2024-02-18T00:48:36.378015+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:59.708561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:41:36.841306+0000","last_clean_scrub_stamp":"2024-02-18T00:47:59.708561+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"840'52","reported_seq":98,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:47.687254+0000","last_change":"2024-02-18T00:48:47.687254+0000","last_active":"2024-02-18T00:48:47.687254+0000","last_peered":"2024-02-18T00:48:47.687254+0000","last_clean":"2024-02-18T00:48:47.687254+0000","last_became_active":"2024-02-18T00:46:25.888711+0000","last_became_peered":"2024-02-18T00:46:25.888711+0000","last_unstale":"2024-02-18T00:48:47.687254+0000","last_undegraded":"2024-02-18T00:48:47.687254+0000","last_fullsized":"2024-02-18T00:48:47.687254+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"840'52","last_scrub_stamp":"2024-02-18T00:48:47.687167+0000","last_deep_scrub":"840'52","last_deep_scrub_stamp":"2024-02-18T00:47:32.441696+0000","last_clean_scrub_stamp":"2024-02-18T00:48:47.687167+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292773+0000","last_change":"2024-02-18T00:48:10.632366+0000","last_active":"2024-02-18T00:48:36.292773+0000","last_peered":"2024-02-18T00:48:36.292773+0000","last_clean":"2024-02-18T00:48:36.292773+0000","last_became_active":"2024-02-18T00:29:14.718013+0000","last_became_peered":"2024-02-18T00:29:14.718013+0000","last_unstale":"2024-02-18T00:48:36.292773+0000","last_undegraded":"2024-02-18T00:48:36.292773+0000","last_fullsized":"2024-02-18T00:48:36.292773+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:10.632284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:49.680414+0000","last_clean_scrub_stamp":"2024-02-18T00:48:10.632284+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":844,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:37.960417+0000","last_change":"2024-02-18T00:48:37.960387+0000","last_active":"2024-02-18T00:48:37.960417+0000","last_peered":"2024-02-18T00:48:37.960417+0000","last_clean":"2024-02-18T00:48:37.960417+0000","last_became_active":"2024-02-18T00:29:19.094765+0000","last_became_peered":"2024-02-18T00:29:19.094765+0000","last_unstale":"2024-02-18T00:48:37.960417+0000","last_undegraded":"2024-02-18T00:48:37.960417+0000","last_fullsized":"2024-02-18T00:48:37.960417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:37.960319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:37.960319+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291697+0000","last_change":"2024-02-18T00:47:57.184768+0000","last_active":"2024-02-18T00:48:36.291697+0000","last_peered":"2024-02-18T00:48:36.291697+0000","last_clean":"2024-02-18T00:48:36.291697+0000","last_became_active":"2024-02-18T00:41:53.837106+0000","last_became_peered":"2024-02-18T00:41:53.837106+0000","last_unstale":"2024-02-18T00:48:36.291697+0000","last_undegraded":"2024-02-18T00:48:36.291697+0000","last_fullsized":"2024-02-18T00:48:36.291697+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":580,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:57.184677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:57.184677+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794054+0000","last_change":"2024-02-18T00:48:19.861719+0000","last_active":"2024-02-18T00:48:36.794054+0000","last_peered":"2024-02-18T00:48:36.794054+0000","last_clean":"2024-02-18T00:48:36.794054+0000","last_became_active":"2024-02-18T00:33:02.523313+0000","last_became_peered":"2024-02-18T00:33:02.523313+0000","last_unstale":"2024-02-18T00:48:36.794054+0000","last_undegraded":"2024-02-18T00:48:36.794054+0000","last_fullsized":"2024-02-18T00:48:36.794054+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:19.861637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.861637+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"555'39","reported_seq":907,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:45.808687+0000","last_change":"2024-02-18T00:48:45.808687+0000","last_active":"2024-02-18T00:48:45.808687+0000","last_peered":"2024-02-18T00:48:45.808687+0000","last_clean":"2024-02-18T00:48:45.808687+0000","last_became_active":"2024-02-18T00:29:17.089468+0000","last_became_peered":"2024-02-18T00:29:17.089468+0000","last_unstale":"2024-02-18T00:48:45.808687+0000","last_undegraded":"2024-02-18T00:48:45.808687+0000","last_fullsized":"2024-02-18T00:48:45.808687+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"555'39","last_scrub_stamp":"2024-02-18T00:48:45.808587+0000","last_deep_scrub":"555'39","last_deep_scrub_stamp":"2024-02-18T00:41:29.051225+0000","last_clean_scrub_stamp":"2024-02-18T00:48:45.808587+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291974+0000","last_change":"2024-02-18T00:48:36.050679+0000","last_active":"2024-02-18T00:48:36.291974+0000","last_peered":"2024-02-18T00:48:36.291974+0000","last_clean":"2024-02-18T00:48:36.291974+0000","last_became_active":"2024-02-18T00:39:36.757840+0000","last_became_peered":"2024-02-18T00:39:36.757840+0000","last_unstale":"2024-02-18T00:48:36.291974+0000","last_undegraded":"2024-02-18T00:48:36.291974+0000","last_fullsized":"2024-02-18T00:48:36.291974+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:36.050601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:19.319010+0000","last_clean_scrub_stamp":"2024-02-18T00:48:36.050601+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":844,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.294159+0000","last_change":"2024-02-18T00:48:01.725656+0000","last_active":"2024-02-18T00:48:36.294159+0000","last_peered":"2024-02-18T00:48:36.294159+0000","last_clean":"2024-02-18T00:48:36.294159+0000","last_became_active":"2024-02-18T00:29:19.099344+0000","last_became_peered":"2024-02-18T00:29:19.099344+0000","last_unstale":"2024-02-18T00:48:36.294159+0000","last_undegraded":"2024-02-18T00:48:36.294159+0000","last_fullsized":"2024-02-18T00:48:36.294159+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:01.725572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.725572+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":855,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293265+0000","last_change":"2024-02-18T00:48:00.851863+0000","last_active":"2024-02-18T00:48:36.293265+0000","last_peered":"2024-02-18T00:48:36.293265+0000","last_clean":"2024-02-18T00:48:36.293265+0000","last_became_active":"2024-02-18T00:41:25.756461+0000","last_became_peered":"2024-02-18T00:41:25.756461+0000","last_unstale":"2024-02-18T00:48:36.293265+0000","last_undegraded":"2024-02-18T00:48:36.293265+0000","last_fullsized":"2024-02-18T00:48:36.293265+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:00.851789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:31.822090+0000","last_clean_scrub_stamp":"2024-02-18T00:48:00.851789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":254,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793443+0000","last_change":"2024-02-18T00:48:09.032408+0000","last_active":"2024-02-18T00:48:36.793443+0000","last_peered":"2024-02-18T00:48:36.793443+0000","last_clean":"2024-02-18T00:48:36.793443+0000","last_became_active":"2024-02-18T00:42:38.553174+0000","last_became_peered":"2024-02-18T00:42:38.553174+0000","last_unstale":"2024-02-18T00:48:36.793443+0000","last_undegraded":"2024-02-18T00:48:36.793443+0000","last_fullsized":"2024-02-18T00:48:36.793443+0000","mapping_epoch":623,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":624,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.032334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.032334+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"457'20","reported_seq":936,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407475+0000","last_change":"2024-02-18T00:48:18.279326+0000","last_active":"2024-02-18T00:48:36.407475+0000","last_peered":"2024-02-18T00:48:36.407475+0000","last_clean":"2024-02-18T00:48:36.407475+0000","last_became_active":"2024-02-18T00:46:25.870141+0000","last_became_peered":"2024-02-18T00:46:25.870141+0000","last_unstale":"2024-02-18T00:48:36.407475+0000","last_undegraded":"2024-02-18T00:48:36.407475+0000","last_fullsized":"2024-02-18T00:48:36.407475+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'20","last_scrub_stamp":"2024-02-18T00:48:18.279247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.279247+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":868,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293534+0000","last_change":"2024-02-18T00:47:54.865456+0000","last_active":"2024-02-18T00:48:36.293534+0000","last_peered":"2024-02-18T00:48:36.293534+0000","last_clean":"2024-02-18T00:48:36.293534+0000","last_became_active":"2024-02-18T00:39:36.738200+0000","last_became_peered":"2024-02-18T00:39:36.738200+0000","last_unstale":"2024-02-18T00:48:36.293534+0000","last_undegraded":"2024-02-18T00:48:36.293534+0000","last_fullsized":"2024-02-18T00:48:36.293534+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:54.865338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:50.988429+0000","last_clean_scrub_stamp":"2024-02-18T00:47:54.865338+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"466'10","reported_seq":1118,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:47.898239+0000","last_change":"2024-02-18T00:48:47.898212+0000","last_active":"2024-02-18T00:48:47.898239+0000","last_peered":"2024-02-18T00:48:47.898239+0000","last_clean":"2024-02-18T00:48:47.898239+0000","last_became_active":"2024-02-18T00:39:36.762482+0000","last_became_peered":"2024-02-18T00:39:36.762482+0000","last_unstale":"2024-02-18T00:48:47.898239+0000","last_undegraded":"2024-02-18T00:48:47.898239+0000","last_fullsized":"2024-02-18T00:48:47.898239+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'10","last_scrub_stamp":"2024-02-18T00:48:47.898127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:47.898127+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":857,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407835+0000","last_change":"2024-02-18T00:48:35.511782+0000","last_active":"2024-02-18T00:48:36.407835+0000","last_peered":"2024-02-18T00:48:36.407835+0000","last_clean":"2024-02-18T00:48:36.407835+0000","last_became_active":"2024-02-18T00:41:29.740529+0000","last_became_peered":"2024-02-18T00:41:29.740529+0000","last_unstale":"2024-02-18T00:48:36.407835+0000","last_undegraded":"2024-02-18T00:48:36.407835+0000","last_fullsized":"2024-02-18T00:48:36.407835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":556,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:35.511683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:35.511683+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":806,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293842+0000","last_change":"2024-02-18T00:47:53.855296+0000","last_active":"2024-02-18T00:48:36.293842+0000","last_peered":"2024-02-18T00:48:36.293842+0000","last_clean":"2024-02-18T00:48:36.293842+0000","last_became_active":"2024-02-18T00:46:25.878475+0000","last_became_peered":"2024-02-18T00:46:25.878475+0000","last_unstale":"2024-02-18T00:48:36.293842+0000","last_undegraded":"2024-02-18T00:48:36.293842+0000","last_fullsized":"2024-02-18T00:48:36.293842+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:53.855222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:47:53.855222+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"457'34","reported_seq":864,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795289+0000","last_change":"2024-02-18T00:48:18.823350+0000","last_active":"2024-02-18T00:48:36.795289+0000","last_peered":"2024-02-18T00:48:36.795289+0000","last_clean":"2024-02-18T00:48:36.795289+0000","last_became_active":"2024-02-18T00:30:41.642148+0000","last_became_peered":"2024-02-18T00:30:41.642148+0000","last_unstale":"2024-02-18T00:48:36.795289+0000","last_undegraded":"2024-02-18T00:48:36.795289+0000","last_fullsized":"2024-02-18T00:48:36.795289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:48:18.823241+0000","last_deep_scrub":"82'22","last_deep_scrub_stamp":"2024-02-18T00:31:50.843994+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.823241+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":875,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293481+0000","last_change":"2024-02-18T00:47:52.807880+0000","last_active":"2024-02-18T00:48:36.293481+0000","last_peered":"2024-02-18T00:48:36.293481+0000","last_clean":"2024-02-18T00:48:36.293481+0000","last_became_active":"2024-02-18T00:39:36.745995+0000","last_became_peered":"2024-02-18T00:39:36.745995+0000","last_unstale":"2024-02-18T00:48:36.293481+0000","last_undegraded":"2024-02-18T00:48:36.293481+0000","last_fullsized":"2024-02-18T00:48:36.293481+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:52.807803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:47.199009+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.807803+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"64'8","reported_seq":1084,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.497084+0000","last_change":"2024-02-18T00:48:30.464109+0000","last_active":"2024-02-18T00:48:43.497084+0000","last_peered":"2024-02-18T00:48:43.497084+0000","last_clean":"2024-02-18T00:48:43.497084+0000","last_became_active":"2024-02-18T00:29:19.097127+0000","last_became_peered":"2024-02-18T00:29:19.097127+0000","last_unstale":"2024-02-18T00:48:43.497084+0000","last_undegraded":"2024-02-18T00:48:43.497084+0000","last_fullsized":"2024-02-18T00:48:43.497084+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'8","last_scrub_stamp":"2024-02-18T00:48:30.463987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:30.463987+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":852,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795233+0000","last_change":"2024-02-18T00:48:32.900019+0000","last_active":"2024-02-18T00:48:36.795233+0000","last_peered":"2024-02-18T00:48:36.795233+0000","last_clean":"2024-02-18T00:48:36.795233+0000","last_became_active":"2024-02-18T00:41:33.827923+0000","last_became_peered":"2024-02-18T00:41:33.827923+0000","last_unstale":"2024-02-18T00:48:36.795233+0000","last_undegraded":"2024-02-18T00:48:36.795233+0000","last_fullsized":"2024-02-18T00:48:36.795233+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":560,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.899942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:08.054205+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.899942+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406804+0000","last_change":"2024-02-18T00:47:37.477135+0000","last_active":"2024-02-18T00:48:36.406804+0000","last_peered":"2024-02-18T00:48:36.406804+0000","last_clean":"2024-02-18T00:48:36.406804+0000","last_became_active":"2024-02-18T00:33:02.533457+0000","last_became_peered":"2024-02-18T00:33:02.533457+0000","last_unstale":"2024-02-18T00:48:36.406804+0000","last_undegraded":"2024-02-18T00:48:36.406804+0000","last_fullsized":"2024-02-18T00:48:36.406804+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:37.477044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:38:26.149169+0000","last_clean_scrub_stamp":"2024-02-18T00:47:37.477044+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"849'31","reported_seq":964,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407752+0000","last_change":"2024-02-18T00:48:23.389365+0000","last_active":"2024-02-18T00:48:36.407752+0000","last_peered":"2024-02-18T00:48:36.407752+0000","last_clean":"2024-02-18T00:48:36.407752+0000","last_became_active":"2024-02-18T00:46:25.878579+0000","last_became_peered":"2024-02-18T00:46:25.878579+0000","last_unstale":"2024-02-18T00:48:36.407752+0000","last_undegraded":"2024-02-18T00:48:36.407752+0000","last_fullsized":"2024-02-18T00:48:36.407752+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"849'31","last_scrub_stamp":"2024-02-18T00:48:23.389255+0000","last_deep_scrub":"849'31","last_deep_scrub_stamp":"2024-02-18T00:48:23.389255+0000","last_clean_scrub_stamp":"2024-02-18T00:48:23.389255+0000","log_size":31,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,0],"acting":[4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":416,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291705+0000","last_change":"2024-02-18T00:48:29.752540+0000","last_active":"2024-02-18T00:48:36.291705+0000","last_peered":"2024-02-18T00:48:36.291705+0000","last_clean":"2024-02-18T00:48:36.291705+0000","last_became_active":"2024-02-18T00:39:43.732210+0000","last_became_peered":"2024-02-18T00:39:43.732210+0000","last_unstale":"2024-02-18T00:48:36.291705+0000","last_undegraded":"2024-02-18T00:48:36.291705+0000","last_fullsized":"2024-02-18T00:48:36.291705+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:29.752478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:29.752478+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":859,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.890654+0000","last_change":"2024-02-18T00:48:36.890622+0000","last_active":"2024-02-18T00:48:36.890654+0000","last_peered":"2024-02-18T00:48:36.890654+0000","last_clean":"2024-02-18T00:48:36.890654+0000","last_became_active":"2024-02-18T00:46:25.878159+0000","last_became_peered":"2024-02-18T00:46:25.878159+0000","last_unstale":"2024-02-18T00:48:36.890654+0000","last_undegraded":"2024-02-18T00:48:36.890654+0000","last_fullsized":"2024-02-18T00:48:36.890654+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:36.890531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:32:47.915092+0000","last_clean_scrub_stamp":"2024-02-18T00:48:36.890531+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"64'1","reported_seq":900,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377262+0000","last_change":"2024-02-18T00:47:48.713645+0000","last_active":"2024-02-18T00:48:36.377262+0000","last_peered":"2024-02-18T00:48:36.377262+0000","last_clean":"2024-02-18T00:48:36.377262+0000","last_became_active":"2024-02-18T00:46:28.878485+0000","last_became_peered":"2024-02-18T00:46:28.878485+0000","last_unstale":"2024-02-18T00:48:36.377262+0000","last_undegraded":"2024-02-18T00:48:36.377262+0000","last_fullsized":"2024-02-18T00:48:36.377262+0000","mapping_epoch":811,"log_start":"64'1","ondisk_log_start":"64'1","created":62,"last_epoch_clean":812,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-02-18T00:47:48.713569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:48.713569+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":785,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:47.810999+0000","last_change":"2024-02-18T00:48:47.810999+0000","last_active":"2024-02-18T00:48:47.810999+0000","last_peered":"2024-02-18T00:48:47.810999+0000","last_clean":"2024-02-18T00:48:47.810999+0000","last_became_active":"2024-02-18T00:33:02.481965+0000","last_became_peered":"2024-02-18T00:33:02.481965+0000","last_unstale":"2024-02-18T00:48:47.810999+0000","last_undegraded":"2024-02-18T00:48:47.810999+0000","last_fullsized":"2024-02-18T00:48:47.810999+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:47.810900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.199947+0000","last_clean_scrub_stamp":"2024-02-18T00:48:47.810900+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"74'9","reported_seq":1074,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.496698+0000","last_change":"2024-02-18T00:47:39.653669+0000","last_active":"2024-02-18T00:48:43.496698+0000","last_peered":"2024-02-18T00:48:43.496698+0000","last_clean":"2024-02-18T00:48:43.496698+0000","last_became_active":"2024-02-18T00:29:19.100159+0000","last_became_peered":"2024-02-18T00:29:19.100159+0000","last_unstale":"2024-02-18T00:48:43.496698+0000","last_undegraded":"2024-02-18T00:48:43.496698+0000","last_fullsized":"2024-02-18T00:48:43.496698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'9","last_scrub_stamp":"2024-02-18T00:47:39.653587+0000","last_deep_scrub":"74'9","last_deep_scrub_stamp":"2024-02-18T00:43:05.211604+0000","last_clean_scrub_stamp":"2024-02-18T00:47:39.653587+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"457'34","reported_seq":948,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406628+0000","last_change":"2024-02-18T00:48:03.301641+0000","last_active":"2024-02-18T00:48:36.406628+0000","last_peered":"2024-02-18T00:48:36.406628+0000","last_clean":"2024-02-18T00:48:36.406628+0000","last_became_active":"2024-02-18T00:46:25.870910+0000","last_became_peered":"2024-02-18T00:46:25.870910+0000","last_unstale":"2024-02-18T00:48:36.406628+0000","last_undegraded":"2024-02-18T00:48:36.406628+0000","last_fullsized":"2024-02-18T00:48:36.406628+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:48:03.301556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:03.301556+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795084+0000","last_change":"2024-02-18T00:48:31.612121+0000","last_active":"2024-02-18T00:48:36.795084+0000","last_peered":"2024-02-18T00:48:36.795084+0000","last_clean":"2024-02-18T00:48:36.795084+0000","last_became_active":"2024-02-18T00:46:25.889388+0000","last_became_peered":"2024-02-18T00:46:25.889388+0000","last_unstale":"2024-02-18T00:48:36.795084+0000","last_undegraded":"2024-02-18T00:48:36.795084+0000","last_fullsized":"2024-02-18T00:48:36.795084+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:31.612041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:52.642073+0000","last_clean_scrub_stamp":"2024-02-18T00:48:31.612041+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":431,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:48.991008+0000","last_change":"2024-02-18T00:48:48.991008+0000","last_active":"2024-02-18T00:48:48.991008+0000","last_peered":"2024-02-18T00:48:48.991008+0000","last_clean":"2024-02-18T00:48:48.991008+0000","last_became_active":"2024-02-18T00:46:25.877376+0000","last_became_peered":"2024-02-18T00:46:25.877376+0000","last_unstale":"2024-02-18T00:48:48.991008+0000","last_undegraded":"2024-02-18T00:48:48.991008+0000","last_fullsized":"2024-02-18T00:48:48.991008+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:48.990932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:48.990932+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794493+0000","last_change":"2024-02-18T00:48:15.739183+0000","last_active":"2024-02-18T00:48:36.794493+0000","last_peered":"2024-02-18T00:48:36.794493+0000","last_clean":"2024-02-18T00:48:36.794493+0000","last_became_active":"2024-02-18T00:33:02.476904+0000","last_became_peered":"2024-02-18T00:33:02.476904+0000","last_unstale":"2024-02-18T00:48:36.794493+0000","last_undegraded":"2024-02-18T00:48:36.794493+0000","last_fullsized":"2024-02-18T00:48:36.794493+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:15.739057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:38.563638+0000","last_clean_scrub_stamp":"2024-02-18T00:48:15.739057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":859,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406851+0000","last_change":"2024-02-18T00:48:00.330785+0000","last_active":"2024-02-18T00:48:36.406851+0000","last_peered":"2024-02-18T00:48:36.406851+0000","last_clean":"2024-02-18T00:48:36.406851+0000","last_became_active":"2024-02-18T00:39:36.742797+0000","last_became_peered":"2024-02-18T00:39:36.742797+0000","last_unstale":"2024-02-18T00:48:36.406851+0000","last_undegraded":"2024-02-18T00:48:36.406851+0000","last_fullsized":"2024-02-18T00:48:36.406851+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:00.330686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:45.974334+0000","last_clean_scrub_stamp":"2024-02-18T00:48:00.330686+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"457'52","reported_seq":999,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377028+0000","last_change":"2024-02-18T00:48:03.689371+0000","last_active":"2024-02-18T00:48:36.377028+0000","last_peered":"2024-02-18T00:48:36.377028+0000","last_clean":"2024-02-18T00:48:36.377028+0000","last_became_active":"2024-02-18T00:43:07.799144+0000","last_became_peered":"2024-02-18T00:43:07.799144+0000","last_unstale":"2024-02-18T00:48:36.377028+0000","last_undegraded":"2024-02-18T00:48:36.377028+0000","last_fullsized":"2024-02-18T00:48:36.377028+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'52","last_scrub_stamp":"2024-02-18T00:48:03.689295+0000","last_deep_scrub":"94'40","last_deep_scrub_stamp":"2024-02-18T00:32:05.656344+0000","last_clean_scrub_stamp":"2024-02-18T00:48:03.689295+0000","log_size":52,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":893,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292292+0000","last_change":"2024-02-18T00:47:30.836684+0000","last_active":"2024-02-18T00:48:36.292292+0000","last_peered":"2024-02-18T00:48:36.292292+0000","last_clean":"2024-02-18T00:48:36.292292+0000","last_became_active":"2024-02-18T00:46:25.877757+0000","last_became_peered":"2024-02-18T00:46:25.877757+0000","last_unstale":"2024-02-18T00:48:36.292292+0000","last_undegraded":"2024-02-18T00:48:36.292292+0000","last_fullsized":"2024-02-18T00:48:36.292292+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:30.836608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:36:09.647928+0000","last_clean_scrub_stamp":"2024-02-18T00:47:30.836608+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":839,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407059+0000","last_change":"2024-02-18T00:48:33.475054+0000","last_active":"2024-02-18T00:48:36.407059+0000","last_peered":"2024-02-18T00:48:36.407059+0000","last_clean":"2024-02-18T00:48:36.407059+0000","last_became_active":"2024-02-18T00:29:21.101529+0000","last_became_peered":"2024-02-18T00:29:21.101529+0000","last_unstale":"2024-02-18T00:48:36.407059+0000","last_undegraded":"2024-02-18T00:48:36.407059+0000","last_fullsized":"2024-02-18T00:48:36.407059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:33.474946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:33.474946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":774,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377924+0000","last_change":"2024-02-18T00:48:10.657179+0000","last_active":"2024-02-18T00:48:36.377924+0000","last_peered":"2024-02-18T00:48:36.377924+0000","last_clean":"2024-02-18T00:48:36.377924+0000","last_became_active":"2024-02-18T00:33:04.650420+0000","last_became_peered":"2024-02-18T00:33:04.650420+0000","last_unstale":"2024-02-18T00:48:36.377924+0000","last_undegraded":"2024-02-18T00:48:36.377924+0000","last_fullsized":"2024-02-18T00:48:36.377924+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:10.657084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:10.657084+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795154+0000","last_change":"2024-02-18T00:48:16.538979+0000","last_active":"2024-02-18T00:48:36.795154+0000","last_peered":"2024-02-18T00:48:36.795154+0000","last_clean":"2024-02-18T00:48:36.795154+0000","last_became_active":"2024-02-18T00:46:25.887171+0000","last_became_peered":"2024-02-18T00:46:25.887171+0000","last_unstale":"2024-02-18T00:48:36.795154+0000","last_undegraded":"2024-02-18T00:48:36.795154+0000","last_fullsized":"2024-02-18T00:48:36.795154+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:16.538896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:16.538896+0000","last_clean_scrub_stamp":"2024-02-18T00:48:16.538896+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"457'35","reported_seq":976,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793840+0000","last_change":"2024-02-18T00:47:42.096731+0000","last_active":"2024-02-18T00:48:36.793840+0000","last_peered":"2024-02-18T00:48:36.793840+0000","last_clean":"2024-02-18T00:48:36.793840+0000","last_became_active":"2024-02-18T00:43:07.797505+0000","last_became_peered":"2024-02-18T00:43:07.797505+0000","last_unstale":"2024-02-18T00:48:36.793840+0000","last_undegraded":"2024-02-18T00:48:36.793840+0000","last_fullsized":"2024-02-18T00:48:36.793840+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'35","last_scrub_stamp":"2024-02-18T00:47:42.096655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:42.096655+0000","log_size":35,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"57'1","reported_seq":416,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.980461+0000","last_change":"2024-02-18T00:48:39.980461+0000","last_active":"2024-02-18T00:48:39.980461+0000","last_peered":"2024-02-18T00:48:39.980461+0000","last_clean":"2024-02-18T00:48:39.980461+0000","last_became_active":"2024-02-18T00:39:43.723568+0000","last_became_peered":"2024-02-18T00:39:43.723568+0000","last_unstale":"2024-02-18T00:48:39.980461+0000","last_undegraded":"2024-02-18T00:48:39.980461+0000","last_fullsized":"2024-02-18T00:48:39.980461+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'1","last_scrub_stamp":"2024-02-18T00:48:39.980392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.980392+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,4],"acting":[2,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":860,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293561+0000","last_change":"2024-02-18T00:47:40.776739+0000","last_active":"2024-02-18T00:48:36.293561+0000","last_peered":"2024-02-18T00:48:36.293561+0000","last_clean":"2024-02-18T00:48:36.293561+0000","last_became_active":"2024-02-18T00:46:25.878733+0000","last_became_peered":"2024-02-18T00:46:25.878733+0000","last_unstale":"2024-02-18T00:48:36.293561+0000","last_undegraded":"2024-02-18T00:48:36.293561+0000","last_fullsized":"2024-02-18T00:48:36.293561+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:40.776651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:31.693369+0000","last_clean_scrub_stamp":"2024-02-18T00:47:40.776651+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":774,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377670+0000","last_change":"2024-02-18T00:48:22.556490+0000","last_active":"2024-02-18T00:48:36.377670+0000","last_peered":"2024-02-18T00:48:36.377670+0000","last_clean":"2024-02-18T00:48:36.377670+0000","last_became_active":"2024-02-18T00:33:04.585787+0000","last_became_peered":"2024-02-18T00:33:04.585787+0000","last_unstale":"2024-02-18T00:48:36.377670+0000","last_undegraded":"2024-02-18T00:48:36.377670+0000","last_fullsized":"2024-02-18T00:48:36.377670+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:22.556417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:22.556417+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0],"acting":[3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":836,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795580+0000","last_change":"2024-02-18T00:47:40.766038+0000","last_active":"2024-02-18T00:48:36.795580+0000","last_peered":"2024-02-18T00:48:36.795580+0000","last_clean":"2024-02-18T00:48:36.795580+0000","last_became_active":"2024-02-18T00:29:19.100772+0000","last_became_peered":"2024-02-18T00:29:19.100772+0000","last_unstale":"2024-02-18T00:48:36.795580+0000","last_undegraded":"2024-02-18T00:48:36.795580+0000","last_fullsized":"2024-02-18T00:48:36.795580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:40.765955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:34.575174+0000","last_clean_scrub_stamp":"2024-02-18T00:47:40.765955+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"847'27","reported_seq":891,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794160+0000","last_change":"2024-02-18T00:48:01.032079+0000","last_active":"2024-02-18T00:48:36.794160+0000","last_peered":"2024-02-18T00:48:36.794160+0000","last_clean":"2024-02-18T00:48:36.794160+0000","last_became_active":"2024-02-18T00:43:07.799985+0000","last_became_peered":"2024-02-18T00:43:07.799985+0000","last_unstale":"2024-02-18T00:48:36.794160+0000","last_undegraded":"2024-02-18T00:48:36.794160+0000","last_fullsized":"2024-02-18T00:48:36.794160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"847'27","last_scrub_stamp":"2024-02-18T00:48:01.032008+0000","last_deep_scrub":"847'27","last_deep_scrub_stamp":"2024-02-18T00:48:01.032008+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.032008+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":411,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291711+0000","last_change":"2024-02-18T00:48:02.800420+0000","last_active":"2024-02-18T00:48:36.291711+0000","last_peered":"2024-02-18T00:48:36.291711+0000","last_clean":"2024-02-18T00:48:36.291711+0000","last_became_active":"2024-02-18T00:39:43.732640+0000","last_became_peered":"2024-02-18T00:39:43.732640+0000","last_unstale":"2024-02-18T00:48:36.291711+0000","last_undegraded":"2024-02-18T00:48:36.291711+0000","last_fullsized":"2024-02-18T00:48:36.291711+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:02.800351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:33.866668+0000","last_clean_scrub_stamp":"2024-02-18T00:48:02.800351+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":843,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793553+0000","last_change":"2024-02-18T00:48:16.064808+0000","last_active":"2024-02-18T00:48:36.793553+0000","last_peered":"2024-02-18T00:48:36.793553+0000","last_clean":"2024-02-18T00:48:36.793553+0000","last_became_active":"2024-02-18T00:29:16.076890+0000","last_became_peered":"2024-02-18T00:29:16.076890+0000","last_unstale":"2024-02-18T00:48:36.793553+0000","last_undegraded":"2024-02-18T00:48:36.793553+0000","last_fullsized":"2024-02-18T00:48:36.793553+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:16.064744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:14.699326+0000","last_clean_scrub_stamp":"2024-02-18T00:48:16.064744+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":842,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.294206+0000","last_change":"2024-02-18T00:48:30.734581+0000","last_active":"2024-02-18T00:48:36.294206+0000","last_peered":"2024-02-18T00:48:36.294206+0000","last_clean":"2024-02-18T00:48:36.294206+0000","last_became_active":"2024-02-18T00:29:21.104558+0000","last_became_peered":"2024-02-18T00:29:21.104558+0000","last_unstale":"2024-02-18T00:48:36.294206+0000","last_undegraded":"2024-02-18T00:48:36.294206+0000","last_fullsized":"2024-02-18T00:48:36.294206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:30.734515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:41.934482+0000","last_clean_scrub_stamp":"2024-02-18T00:48:30.734515+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":273,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.913999+0000","last_change":"2024-02-18T00:48:36.913957+0000","last_active":"2024-02-18T00:48:36.913999+0000","last_peered":"2024-02-18T00:48:36.913999+0000","last_clean":"2024-02-18T00:48:36.913999+0000","last_became_active":"2024-02-18T00:46:25.876777+0000","last_became_peered":"2024-02-18T00:46:25.876777+0000","last_unstale":"2024-02-18T00:48:36.913999+0000","last_undegraded":"2024-02-18T00:48:36.913999+0000","last_fullsized":"2024-02-18T00:48:36.913999+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:36.913850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:36.913850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,7],"acting":[2,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292392+0000","last_change":"2024-02-18T00:48:01.102950+0000","last_active":"2024-02-18T00:48:36.292392+0000","last_peered":"2024-02-18T00:48:36.292392+0000","last_clean":"2024-02-18T00:48:36.292392+0000","last_became_active":"2024-02-18T00:39:36.763302+0000","last_became_peered":"2024-02-18T00:39:36.763302+0000","last_unstale":"2024-02-18T00:48:36.292392+0000","last_undegraded":"2024-02-18T00:48:36.292392+0000","last_fullsized":"2024-02-18T00:48:36.292392+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:01.102866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:35.259766+0000","last_clean_scrub_stamp":"2024-02-18T00:48:01.102866+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"853'29","reported_seq":77,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794611+0000","last_change":"2024-02-18T00:48:34.518432+0000","last_active":"2024-02-18T00:48:36.794611+0000","last_peered":"2024-02-18T00:48:36.794611+0000","last_clean":"2024-02-18T00:48:36.794611+0000","last_became_active":"2024-02-18T00:46:25.887464+0000","last_became_peered":"2024-02-18T00:46:25.887464+0000","last_unstale":"2024-02-18T00:48:36.794611+0000","last_undegraded":"2024-02-18T00:48:36.794611+0000","last_fullsized":"2024-02-18T00:48:36.794611+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"853'29","last_scrub_stamp":"2024-02-18T00:48:34.518347+0000","last_deep_scrub":"853'29","last_deep_scrub_stamp":"2024-02-18T00:48:34.518347+0000","last_clean_scrub_stamp":"2024-02-18T00:48:34.518347+0000","log_size":29,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291550+0000","last_change":"2024-02-18T00:47:58.147390+0000","last_active":"2024-02-18T00:48:36.291550+0000","last_peered":"2024-02-18T00:48:36.291550+0000","last_clean":"2024-02-18T00:48:36.291550+0000","last_became_active":"2024-02-18T00:46:25.884860+0000","last_became_peered":"2024-02-18T00:46:25.884860+0000","last_unstale":"2024-02-18T00:48:36.291550+0000","last_undegraded":"2024-02-18T00:48:36.291550+0000","last_fullsized":"2024-02-18T00:48:36.291550+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:58.147307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:42.067810+0000","last_clean_scrub_stamp":"2024-02-18T00:47:58.147307+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":881,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795539+0000","last_change":"2024-02-18T00:48:22.881947+0000","last_active":"2024-02-18T00:48:36.795539+0000","last_peered":"2024-02-18T00:48:36.795539+0000","last_clean":"2024-02-18T00:48:36.795539+0000","last_became_active":"2024-02-18T00:39:43.726900+0000","last_became_peered":"2024-02-18T00:39:43.726900+0000","last_unstale":"2024-02-18T00:48:36.795539+0000","last_undegraded":"2024-02-18T00:48:36.795539+0000","last_fullsized":"2024-02-18T00:48:36.795539+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:22.881869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:22.881869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":851,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.372131+0000","last_change":"2024-02-18T00:48:43.372131+0000","last_active":"2024-02-18T00:48:43.372131+0000","last_peered":"2024-02-18T00:48:43.372131+0000","last_clean":"2024-02-18T00:48:43.372131+0000","last_became_active":"2024-02-18T00:46:25.885092+0000","last_became_peered":"2024-02-18T00:46:25.885092+0000","last_unstale":"2024-02-18T00:48:43.372131+0000","last_undegraded":"2024-02-18T00:48:43.372131+0000","last_fullsized":"2024-02-18T00:48:43.372131+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:43.372061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:43.372061+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.372061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":852,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795594+0000","last_change":"2024-02-18T00:47:18.590533+0000","last_active":"2024-02-18T00:48:36.795594+0000","last_peered":"2024-02-18T00:48:36.795594+0000","last_clean":"2024-02-18T00:48:36.795594+0000","last_became_active":"2024-02-18T00:41:57.791956+0000","last_became_peered":"2024-02-18T00:41:57.791956+0000","last_unstale":"2024-02-18T00:48:36.795594+0000","last_undegraded":"2024-02-18T00:48:36.795594+0000","last_fullsized":"2024-02-18T00:48:36.795594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":584,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:18.590439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:46.851573+0000","last_clean_scrub_stamp":"2024-02-18T00:47:18.590439+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":785,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793677+0000","last_change":"2024-02-18T00:48:18.125386+0000","last_active":"2024-02-18T00:48:36.793677+0000","last_peered":"2024-02-18T00:48:36.793677+0000","last_clean":"2024-02-18T00:48:36.793677+0000","last_became_active":"2024-02-18T00:33:04.648698+0000","last_became_peered":"2024-02-18T00:33:04.648698+0000","last_unstale":"2024-02-18T00:48:36.793677+0000","last_undegraded":"2024-02-18T00:48:36.793677+0000","last_fullsized":"2024-02-18T00:48:36.793677+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:18.125320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:05.469634+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.125320+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":862,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293535+0000","last_change":"2024-02-18T00:48:04.783601+0000","last_active":"2024-02-18T00:48:36.293535+0000","last_peered":"2024-02-18T00:48:36.293535+0000","last_clean":"2024-02-18T00:48:36.293535+0000","last_became_active":"2024-02-18T00:39:36.759505+0000","last_became_peered":"2024-02-18T00:39:36.759505+0000","last_unstale":"2024-02-18T00:48:36.293535+0000","last_undegraded":"2024-02-18T00:48:36.293535+0000","last_fullsized":"2024-02-18T00:48:36.293535+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:04.783526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:04.783526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"680'27","reported_seq":915,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:49.659129+0000","last_change":"2024-02-18T00:48:49.659129+0000","last_active":"2024-02-18T00:48:49.659129+0000","last_peered":"2024-02-18T00:48:49.659129+0000","last_clean":"2024-02-18T00:48:49.659129+0000","last_became_active":"2024-02-18T00:43:07.801362+0000","last_became_peered":"2024-02-18T00:43:07.801362+0000","last_unstale":"2024-02-18T00:48:49.659129+0000","last_undegraded":"2024-02-18T00:48:49.659129+0000","last_fullsized":"2024-02-18T00:48:49.659129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"680'27","last_scrub_stamp":"2024-02-18T00:48:49.659007+0000","last_deep_scrub":"680'27","last_deep_scrub_stamp":"2024-02-18T00:47:18.816173+0000","last_clean_scrub_stamp":"2024-02-18T00:48:49.659007+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":857,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292025+0000","last_change":"2024-02-18T00:48:32.796392+0000","last_active":"2024-02-18T00:48:36.292025+0000","last_peered":"2024-02-18T00:48:36.292025+0000","last_clean":"2024-02-18T00:48:36.292025+0000","last_became_active":"2024-02-18T00:33:04.664893+0000","last_became_peered":"2024-02-18T00:33:04.664893+0000","last_unstale":"2024-02-18T00:48:36.292025+0000","last_undegraded":"2024-02-18T00:48:36.292025+0000","last_fullsized":"2024-02-18T00:48:36.292025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.796325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.796325+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'6","reported_seq":2046,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:49.131229+0000","last_change":"2024-02-18T00:47:57.808688+0000","last_active":"2024-02-18T00:48:49.131229+0000","last_peered":"2024-02-18T00:48:49.131229+0000","last_clean":"2024-02-18T00:48:49.131229+0000","last_became_active":"2024-02-18T00:46:25.873170+0000","last_became_peered":"2024-02-18T00:46:25.873170+0000","last_unstale":"2024-02-18T00:48:49.131229+0000","last_undegraded":"2024-02-18T00:48:49.131229+0000","last_fullsized":"2024-02-18T00:48:49.131229+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-02-18T00:47:57.808617+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-02-18T00:30:32.271183+0000","last_clean_scrub_stamp":"2024-02-18T00:47:57.808617+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1163,"num_read_kb":1158,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"85'2","reported_seq":438,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292558+0000","last_change":"2024-02-18T00:48:17.162540+0000","last_active":"2024-02-18T00:48:36.292558+0000","last_peered":"2024-02-18T00:48:36.292558+0000","last_clean":"2024-02-18T00:48:36.292558+0000","last_became_active":"2024-02-18T00:46:25.885434+0000","last_became_peered":"2024-02-18T00:46:25.885434+0000","last_unstale":"2024-02-18T00:48:36.292558+0000","last_undegraded":"2024-02-18T00:48:36.292558+0000","last_fullsized":"2024-02-18T00:48:36.292558+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"85'2","last_scrub_stamp":"2024-02-18T00:48:17.162456+0000","last_deep_scrub":"85'2","last_deep_scrub_stamp":"2024-02-18T00:46:48.220900+0000","last_clean_scrub_stamp":"2024-02-18T00:48:17.162456+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":39,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":798,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406867+0000","last_change":"2024-02-18T00:47:53.415120+0000","last_active":"2024-02-18T00:48:36.406867+0000","last_peered":"2024-02-18T00:48:36.406867+0000","last_clean":"2024-02-18T00:48:36.406867+0000","last_became_active":"2024-02-18T00:39:36.742411+0000","last_became_peered":"2024-02-18T00:39:36.742411+0000","last_unstale":"2024-02-18T00:48:36.406867+0000","last_undegraded":"2024-02-18T00:48:36.406867+0000","last_fullsized":"2024-02-18T00:48:36.406867+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:53.415039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:52.430010+0000","last_clean_scrub_stamp":"2024-02-18T00:47:53.415039+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":840,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794173+0000","last_change":"2024-02-18T00:48:26.129134+0000","last_active":"2024-02-18T00:48:36.794173+0000","last_peered":"2024-02-18T00:48:36.794173+0000","last_clean":"2024-02-18T00:48:36.794173+0000","last_became_active":"2024-02-18T00:29:19.090419+0000","last_became_peered":"2024-02-18T00:29:19.090419+0000","last_unstale":"2024-02-18T00:48:36.794173+0000","last_undegraded":"2024-02-18T00:48:36.794173+0000","last_fullsized":"2024-02-18T00:48:36.794173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:26.129050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:26.129050+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"682'49","reported_seq":851,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.497299+0000","last_change":"2024-02-18T00:48:34.902228+0000","last_active":"2024-02-18T00:48:43.497299+0000","last_peered":"2024-02-18T00:48:43.497299+0000","last_clean":"2024-02-18T00:48:43.497299+0000","last_became_active":"2024-02-18T00:43:44.816299+0000","last_became_peered":"2024-02-18T00:43:44.816299+0000","last_unstale":"2024-02-18T00:48:43.497299+0000","last_undegraded":"2024-02-18T00:48:43.497299+0000","last_fullsized":"2024-02-18T00:48:43.497299+0000","mapping_epoch":678,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":679,"parent":"0.0","parent_split_bits":0,"last_scrub":"682'49","last_scrub_stamp":"2024-02-18T00:48:34.902137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:34.902137+0000","log_size":49,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":248,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,0],"acting":[5,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":872,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.981185+0000","last_change":"2024-02-18T00:48:43.981185+0000","last_active":"2024-02-18T00:48:43.981185+0000","last_peered":"2024-02-18T00:48:43.981185+0000","last_clean":"2024-02-18T00:48:43.981185+0000","last_became_active":"2024-02-18T00:46:25.879623+0000","last_became_peered":"2024-02-18T00:46:25.879623+0000","last_unstale":"2024-02-18T00:48:43.981185+0000","last_undegraded":"2024-02-18T00:48:43.981185+0000","last_fullsized":"2024-02-18T00:48:43.981185+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:43.981081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:15.527235+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.981081+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":864,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377504+0000","last_change":"2024-02-18T00:48:29.588456+0000","last_active":"2024-02-18T00:48:36.377504+0000","last_peered":"2024-02-18T00:48:36.377504+0000","last_clean":"2024-02-18T00:48:36.377504+0000","last_became_active":"2024-02-18T00:42:02.125704+0000","last_became_peered":"2024-02-18T00:42:02.125704+0000","last_unstale":"2024-02-18T00:48:36.377504+0000","last_undegraded":"2024-02-18T00:48:36.377504+0000","last_fullsized":"2024-02-18T00:48:36.377504+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:29.588378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:39:14.570919+0000","last_clean_scrub_stamp":"2024-02-18T00:48:29.588378+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":803,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:49.189677+0000","last_change":"2024-02-18T00:48:49.189677+0000","last_active":"2024-02-18T00:48:49.189677+0000","last_peered":"2024-02-18T00:48:49.189677+0000","last_clean":"2024-02-18T00:48:49.189677+0000","last_became_active":"2024-02-18T00:46:25.893568+0000","last_became_peered":"2024-02-18T00:46:25.893568+0000","last_unstale":"2024-02-18T00:48:49.189677+0000","last_undegraded":"2024-02-18T00:48:49.189677+0000","last_fullsized":"2024-02-18T00:48:49.189677+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:49.189600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:49.189600+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,6],"acting":[0,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"811'10","reported_seq":1104,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.496678+0000","last_change":"2024-02-18T00:47:35.142340+0000","last_active":"2024-02-18T00:48:43.496678+0000","last_peered":"2024-02-18T00:48:43.496678+0000","last_clean":"2024-02-18T00:48:43.496678+0000","last_became_active":"2024-02-18T00:46:25.893478+0000","last_became_peered":"2024-02-18T00:46:25.893478+0000","last_unstale":"2024-02-18T00:48:43.496678+0000","last_undegraded":"2024-02-18T00:48:43.496678+0000","last_fullsized":"2024-02-18T00:48:43.496678+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"811'10","last_scrub_stamp":"2024-02-18T00:47:35.142218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:47:35.142218+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"457'32","reported_seq":867,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293692+0000","last_change":"2024-02-18T00:48:12.689067+0000","last_active":"2024-02-18T00:48:36.293692+0000","last_peered":"2024-02-18T00:48:36.293692+0000","last_clean":"2024-02-18T00:48:36.293692+0000","last_became_active":"2024-02-18T00:43:07.795433+0000","last_became_peered":"2024-02-18T00:43:07.795433+0000","last_unstale":"2024-02-18T00:48:36.293692+0000","last_undegraded":"2024-02-18T00:48:36.293692+0000","last_fullsized":"2024-02-18T00:48:36.293692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:48:12.688998+0000","last_deep_scrub":"457'32","last_deep_scrub_stamp":"2024-02-18T00:48:12.688998+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.688998+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":880,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.294075+0000","last_change":"2024-02-18T00:47:45.771572+0000","last_active":"2024-02-18T00:48:36.294075+0000","last_peered":"2024-02-18T00:48:36.294075+0000","last_clean":"2024-02-18T00:48:36.294075+0000","last_became_active":"2024-02-18T00:39:36.745291+0000","last_became_peered":"2024-02-18T00:39:36.745291+0000","last_unstale":"2024-02-18T00:48:36.294075+0000","last_undegraded":"2024-02-18T00:48:36.294075+0000","last_fullsized":"2024-02-18T00:48:36.294075+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:45.771506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:43.113292+0000","last_clean_scrub_stamp":"2024-02-18T00:47:45.771506+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":833,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793328+0000","last_change":"2024-02-18T00:48:32.028325+0000","last_active":"2024-02-18T00:48:36.793328+0000","last_peered":"2024-02-18T00:48:36.793328+0000","last_clean":"2024-02-18T00:48:36.793328+0000","last_became_active":"2024-02-18T00:29:21.102711+0000","last_became_peered":"2024-02-18T00:29:21.102711+0000","last_unstale":"2024-02-18T00:48:36.793328+0000","last_undegraded":"2024-02-18T00:48:36.793328+0000","last_fullsized":"2024-02-18T00:48:36.793328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.028259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.028259+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":785,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:40.620935+0000","last_change":"2024-02-18T00:48:40.620935+0000","last_active":"2024-02-18T00:48:40.620935+0000","last_peered":"2024-02-18T00:48:40.620935+0000","last_clean":"2024-02-18T00:48:40.620935+0000","last_became_active":"2024-02-18T00:33:02.483107+0000","last_became_peered":"2024-02-18T00:33:02.483107+0000","last_unstale":"2024-02-18T00:48:40.620935+0000","last_undegraded":"2024-02-18T00:48:40.620935+0000","last_fullsized":"2024-02-18T00:48:40.620935+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:40.620856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:40.620856+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,3],"acting":[4,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":839,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291950+0000","last_change":"2024-02-18T00:48:06.766100+0000","last_active":"2024-02-18T00:48:36.291950+0000","last_peered":"2024-02-18T00:48:36.291950+0000","last_clean":"2024-02-18T00:48:36.291950+0000","last_became_active":"2024-02-18T00:29:19.096643+0000","last_became_peered":"2024-02-18T00:29:19.096643+0000","last_unstale":"2024-02-18T00:48:36.291950+0000","last_undegraded":"2024-02-18T00:48:36.291950+0000","last_fullsized":"2024-02-18T00:48:36.291950+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:06.766026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:37:44.335390+0000","last_clean_scrub_stamp":"2024-02-18T00:48:06.766026+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"457'30","reported_seq":915,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406658+0000","last_change":"2024-02-18T00:48:07.440661+0000","last_active":"2024-02-18T00:48:36.406658+0000","last_peered":"2024-02-18T00:48:36.406658+0000","last_clean":"2024-02-18T00:48:36.406658+0000","last_became_active":"2024-02-18T00:43:07.795322+0000","last_became_peered":"2024-02-18T00:43:07.795322+0000","last_unstale":"2024-02-18T00:48:36.406658+0000","last_undegraded":"2024-02-18T00:48:36.406658+0000","last_fullsized":"2024-02-18T00:48:36.406658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":642,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'30","last_scrub_stamp":"2024-02-18T00:48:07.440575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:07.440575+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"57'2","reported_seq":893,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407865+0000","last_change":"2024-02-18T00:48:24.381563+0000","last_active":"2024-02-18T00:48:36.407865+0000","last_peered":"2024-02-18T00:48:36.407865+0000","last_clean":"2024-02-18T00:48:36.407865+0000","last_became_active":"2024-02-18T00:39:36.750714+0000","last_became_peered":"2024-02-18T00:39:36.750714+0000","last_unstale":"2024-02-18T00:48:36.407865+0000","last_undegraded":"2024-02-18T00:48:36.407865+0000","last_fullsized":"2024-02-18T00:48:36.407865+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"57'2","last_scrub_stamp":"2024-02-18T00:48:24.381480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:24.381480+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":872,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:44.389289+0000","last_change":"2024-02-18T00:48:44.389289+0000","last_active":"2024-02-18T00:48:44.389289+0000","last_peered":"2024-02-18T00:48:44.389289+0000","last_clean":"2024-02-18T00:48:44.389289+0000","last_became_active":"2024-02-18T00:41:42.776144+0000","last_became_peered":"2024-02-18T00:41:42.776144+0000","last_unstale":"2024-02-18T00:48:44.389289+0000","last_undegraded":"2024-02-18T00:48:44.389289+0000","last_fullsized":"2024-02-18T00:48:44.389289+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:44.389213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:44.389213+0000","last_clean_scrub_stamp":"2024-02-18T00:48:44.389213+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":785,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:48.770163+0000","last_change":"2024-02-18T00:48:48.770163+0000","last_active":"2024-02-18T00:48:48.770163+0000","last_peered":"2024-02-18T00:48:48.770163+0000","last_clean":"2024-02-18T00:48:48.770163+0000","last_became_active":"2024-02-18T00:33:02.533212+0000","last_became_peered":"2024-02-18T00:33:02.533212+0000","last_unstale":"2024-02-18T00:48:48.770163+0000","last_undegraded":"2024-02-18T00:48:48.770163+0000","last_fullsized":"2024-02-18T00:48:48.770163+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:48.770079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:48.770079+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"774'33","reported_seq":972,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794208+0000","last_change":"2024-02-18T00:48:33.069065+0000","last_active":"2024-02-18T00:48:36.794208+0000","last_peered":"2024-02-18T00:48:36.794208+0000","last_clean":"2024-02-18T00:48:36.794208+0000","last_became_active":"2024-02-18T00:46:10.866407+0000","last_became_peered":"2024-02-18T00:46:10.866407+0000","last_unstale":"2024-02-18T00:48:36.794208+0000","last_undegraded":"2024-02-18T00:48:36.794208+0000","last_fullsized":"2024-02-18T00:48:36.794208+0000","mapping_epoch":801,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":802,"parent":"0.0","parent_split_bits":0,"last_scrub":"774'33","last_scrub_stamp":"2024-02-18T00:48:33.068984+0000","last_deep_scrub":"774'33","last_deep_scrub_stamp":"2024-02-18T00:48:33.068984+0000","last_clean_scrub_stamp":"2024-02-18T00:48:33.068984+0000","log_size":33,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":843,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291868+0000","last_change":"2024-02-18T00:47:50.739024+0000","last_active":"2024-02-18T00:48:36.291868+0000","last_peered":"2024-02-18T00:48:36.291868+0000","last_clean":"2024-02-18T00:48:36.291868+0000","last_became_active":"2024-02-18T00:29:14.711564+0000","last_became_peered":"2024-02-18T00:29:14.711564+0000","last_unstale":"2024-02-18T00:48:36.291868+0000","last_undegraded":"2024-02-18T00:48:36.291868+0000","last_fullsized":"2024-02-18T00:48:36.291868+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:50.738946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:50.738946+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377679+0000","last_change":"2024-02-18T00:47:35.664446+0000","last_active":"2024-02-18T00:48:36.377679+0000","last_peered":"2024-02-18T00:48:36.377679+0000","last_clean":"2024-02-18T00:48:36.377679+0000","last_became_active":"2024-02-18T00:39:36.763197+0000","last_became_peered":"2024-02-18T00:39:36.763197+0000","last_unstale":"2024-02-18T00:48:36.377679+0000","last_undegraded":"2024-02-18T00:48:36.377679+0000","last_fullsized":"2024-02-18T00:48:36.377679+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:35.664372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:35.664372+0000","last_clean_scrub_stamp":"2024-02-18T00:47:35.664372+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":857,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292519+0000","last_change":"2024-02-18T00:48:19.749565+0000","last_active":"2024-02-18T00:48:36.292519+0000","last_peered":"2024-02-18T00:48:36.292519+0000","last_clean":"2024-02-18T00:48:36.292519+0000","last_became_active":"2024-02-18T00:39:36.757564+0000","last_became_peered":"2024-02-18T00:39:36.757564+0000","last_unstale":"2024-02-18T00:48:36.292519+0000","last_undegraded":"2024-02-18T00:48:36.292519+0000","last_fullsized":"2024-02-18T00:48:36.292519+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:19.749502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:19.749502+0000","last_clean_scrub_stamp":"2024-02-18T00:48:19.749502+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":780,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793622+0000","last_change":"2024-02-18T00:47:43.134205+0000","last_active":"2024-02-18T00:48:36.793622+0000","last_peered":"2024-02-18T00:48:36.793622+0000","last_clean":"2024-02-18T00:48:36.793622+0000","last_became_active":"2024-02-18T00:33:04.694370+0000","last_became_peered":"2024-02-18T00:33:04.694370+0000","last_unstale":"2024-02-18T00:48:36.793622+0000","last_undegraded":"2024-02-18T00:48:36.793622+0000","last_fullsized":"2024-02-18T00:48:36.793622+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:43.134120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:47:43.134120+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"457'18","reported_seq":930,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:46.357916+0000","last_change":"2024-02-18T00:48:46.357916+0000","last_active":"2024-02-18T00:48:46.357916+0000","last_peered":"2024-02-18T00:48:46.357916+0000","last_clean":"2024-02-18T00:48:46.357916+0000","last_became_active":"2024-02-18T00:39:36.757265+0000","last_became_peered":"2024-02-18T00:39:36.757265+0000","last_unstale":"2024-02-18T00:48:46.357916+0000","last_undegraded":"2024-02-18T00:48:46.357916+0000","last_fullsized":"2024-02-18T00:48:46.357916+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'18","last_scrub_stamp":"2024-02-18T00:48:46.357830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:46.357830+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794004+0000","last_change":"2024-02-18T00:47:58.456547+0000","last_active":"2024-02-18T00:48:36.794004+0000","last_peered":"2024-02-18T00:48:36.794004+0000","last_clean":"2024-02-18T00:48:36.794004+0000","last_became_active":"2024-02-18T00:46:25.883636+0000","last_became_peered":"2024-02-18T00:46:25.883636+0000","last_unstale":"2024-02-18T00:48:36.794004+0000","last_undegraded":"2024-02-18T00:48:36.794004+0000","last_fullsized":"2024-02-18T00:48:36.794004+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:58.456471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:58.456471+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":841,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.972701+0000","last_change":"2024-02-18T00:48:39.972701+0000","last_active":"2024-02-18T00:48:39.972701+0000","last_peered":"2024-02-18T00:48:39.972701+0000","last_clean":"2024-02-18T00:48:39.972701+0000","last_became_active":"2024-02-18T00:29:19.100715+0000","last_became_peered":"2024-02-18T00:29:19.100715+0000","last_unstale":"2024-02-18T00:48:39.972701+0000","last_undegraded":"2024-02-18T00:48:39.972701+0000","last_fullsized":"2024-02-18T00:48:39.972701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:39.972610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.972610+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292586+0000","last_change":"2024-02-18T00:48:03.145476+0000","last_active":"2024-02-18T00:48:36.292586+0000","last_peered":"2024-02-18T00:48:36.292586+0000","last_clean":"2024-02-18T00:48:36.292586+0000","last_became_active":"2024-02-18T00:46:25.884359+0000","last_became_peered":"2024-02-18T00:46:25.884359+0000","last_unstale":"2024-02-18T00:48:36.292586+0000","last_undegraded":"2024-02-18T00:48:36.292586+0000","last_fullsized":"2024-02-18T00:48:36.292586+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:03.145333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:03.145333+0000","last_clean_scrub_stamp":"2024-02-18T00:48:03.145333+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":792,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292268+0000","last_change":"2024-02-18T00:48:09.665657+0000","last_active":"2024-02-18T00:48:36.292268+0000","last_peered":"2024-02-18T00:48:36.292268+0000","last_clean":"2024-02-18T00:48:36.292268+0000","last_became_active":"2024-02-18T00:35:42.681789+0000","last_became_peered":"2024-02-18T00:35:42.681789+0000","last_unstale":"2024-02-18T00:48:36.292268+0000","last_undegraded":"2024-02-18T00:48:36.292268+0000","last_fullsized":"2024-02-18T00:48:36.292268+0000","mapping_epoch":263,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":264,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.665586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.665586+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"457'32","reported_seq":942,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377676+0000","last_change":"2024-02-18T00:47:40.635304+0000","last_active":"2024-02-18T00:48:36.377676+0000","last_peered":"2024-02-18T00:48:36.377676+0000","last_clean":"2024-02-18T00:48:36.377676+0000","last_became_active":"2024-02-18T00:39:36.758540+0000","last_became_peered":"2024-02-18T00:39:36.758540+0000","last_unstale":"2024-02-18T00:48:36.377676+0000","last_undegraded":"2024-02-18T00:48:36.377676+0000","last_fullsized":"2024-02-18T00:48:36.377676+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:47:40.635222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:40.635222+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":54,"num_read_kb":47,"num_write":32,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":867,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794931+0000","last_change":"2024-02-18T00:48:16.745096+0000","last_active":"2024-02-18T00:48:36.794931+0000","last_peered":"2024-02-18T00:48:36.794931+0000","last_clean":"2024-02-18T00:48:36.794931+0000","last_became_active":"2024-02-18T00:39:36.751224+0000","last_became_peered":"2024-02-18T00:39:36.751224+0000","last_unstale":"2024-02-18T00:48:36.794931+0000","last_undegraded":"2024-02-18T00:48:36.794931+0000","last_fullsized":"2024-02-18T00:48:36.794931+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:16.745018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:16.745018+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"487'9","reported_seq":1085,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.497258+0000","last_change":"2024-02-18T00:48:42.938666+0000","last_active":"2024-02-18T00:48:43.497258+0000","last_peered":"2024-02-18T00:48:43.497258+0000","last_clean":"2024-02-18T00:48:43.497258+0000","last_became_active":"2024-02-18T00:29:19.100809+0000","last_became_peered":"2024-02-18T00:29:19.100809+0000","last_unstale":"2024-02-18T00:48:43.497258+0000","last_undegraded":"2024-02-18T00:48:43.497258+0000","last_fullsized":"2024-02-18T00:48:43.497258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"487'9","last_scrub_stamp":"2024-02-18T00:48:42.938544+0000","last_deep_scrub":"487'9","last_deep_scrub_stamp":"2024-02-18T00:48:42.938544+0000","last_clean_scrub_stamp":"2024-02-18T00:48:42.938544+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794198+0000","last_change":"2024-02-18T00:48:08.440272+0000","last_active":"2024-02-18T00:48:36.794198+0000","last_peered":"2024-02-18T00:48:36.794198+0000","last_clean":"2024-02-18T00:48:36.794198+0000","last_became_active":"2024-02-18T00:46:25.887695+0000","last_became_peered":"2024-02-18T00:46:25.887695+0000","last_unstale":"2024-02-18T00:48:36.794198+0000","last_undegraded":"2024-02-18T00:48:36.794198+0000","last_fullsized":"2024-02-18T00:48:36.794198+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:08.440187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:08.440187+0000","last_clean_scrub_stamp":"2024-02-18T00:48:08.440187+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.15","version":"849'26","reported_seq":925,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794589+0000","last_change":"2024-02-18T00:48:20.855911+0000","last_active":"2024-02-18T00:48:36.794589+0000","last_peered":"2024-02-18T00:48:36.794589+0000","last_clean":"2024-02-18T00:48:36.794589+0000","last_became_active":"2024-02-18T00:39:36.755784+0000","last_became_peered":"2024-02-18T00:39:36.755784+0000","last_unstale":"2024-02-18T00:48:36.794589+0000","last_undegraded":"2024-02-18T00:48:36.794589+0000","last_fullsized":"2024-02-18T00:48:36.794589+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"849'26","last_scrub_stamp":"2024-02-18T00:48:20.855835+0000","last_deep_scrub":"849'26","last_deep_scrub_stamp":"2024-02-18T00:48:20.855835+0000","last_clean_scrub_stamp":"2024-02-18T00:48:20.855835+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":844,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:40.140287+0000","last_change":"2024-02-18T00:48:40.140287+0000","last_active":"2024-02-18T00:48:40.140287+0000","last_peered":"2024-02-18T00:48:40.140287+0000","last_clean":"2024-02-18T00:48:40.140287+0000","last_became_active":"2024-02-18T00:29:14.715400+0000","last_became_peered":"2024-02-18T00:29:14.715400+0000","last_unstale":"2024-02-18T00:48:40.140287+0000","last_undegraded":"2024-02-18T00:48:40.140287+0000","last_fullsized":"2024-02-18T00:48:40.140287+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:40.140220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:21.674903+0000","last_clean_scrub_stamp":"2024-02-18T00:48:40.140220+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"466'11","reported_seq":1100,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.496674+0000","last_change":"2024-02-18T00:47:38.681440+0000","last_active":"2024-02-18T00:48:43.496674+0000","last_peered":"2024-02-18T00:48:43.496674+0000","last_clean":"2024-02-18T00:48:43.496674+0000","last_became_active":"2024-02-18T00:39:36.758042+0000","last_became_peered":"2024-02-18T00:39:36.758042+0000","last_unstale":"2024-02-18T00:48:43.496674+0000","last_undegraded":"2024-02-18T00:48:43.496674+0000","last_fullsized":"2024-02-18T00:48:43.496674+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'11","last_scrub_stamp":"2024-02-18T00:47:38.681367+0000","last_deep_scrub":"466'11","last_deep_scrub_stamp":"2024-02-18T00:40:06.960509+0000","last_clean_scrub_stamp":"2024-02-18T00:47:38.681367+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":803,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:44.997940+0000","last_change":"2024-02-18T00:48:44.997940+0000","last_active":"2024-02-18T00:48:44.997940+0000","last_peered":"2024-02-18T00:48:44.997940+0000","last_clean":"2024-02-18T00:48:44.997940+0000","last_became_active":"2024-02-18T00:46:25.878776+0000","last_became_peered":"2024-02-18T00:46:25.878776+0000","last_unstale":"2024-02-18T00:48:44.997940+0000","last_undegraded":"2024-02-18T00:48:44.997940+0000","last_fullsized":"2024-02-18T00:48:44.997940+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:44.997853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:40:06.573143+0000","last_clean_scrub_stamp":"2024-02-18T00:48:44.997853+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"457'15","reported_seq":911,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293626+0000","last_change":"2024-02-18T00:48:26.846345+0000","last_active":"2024-02-18T00:48:36.293626+0000","last_peered":"2024-02-18T00:48:36.293626+0000","last_clean":"2024-02-18T00:48:36.293626+0000","last_became_active":"2024-02-18T00:46:25.884390+0000","last_became_peered":"2024-02-18T00:46:25.884390+0000","last_unstale":"2024-02-18T00:48:36.293626+0000","last_undegraded":"2024-02-18T00:48:36.293626+0000","last_fullsized":"2024-02-18T00:48:36.293626+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'15","last_scrub_stamp":"2024-02-18T00:48:26.846279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:26.846279+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794349+0000","last_change":"2024-02-18T00:48:14.488680+0000","last_active":"2024-02-18T00:48:36.794349+0000","last_peered":"2024-02-18T00:48:36.794349+0000","last_clean":"2024-02-18T00:48:36.794349+0000","last_became_active":"2024-02-18T00:46:25.887011+0000","last_became_peered":"2024-02-18T00:46:25.887011+0000","last_unstale":"2024-02-18T00:48:36.794349+0000","last_undegraded":"2024-02-18T00:48:36.794349+0000","last_fullsized":"2024-02-18T00:48:36.794349+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:14.488598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:14.488598+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":835,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.435734+0000","last_change":"2024-02-18T00:48:38.435700+0000","last_active":"2024-02-18T00:48:38.435734+0000","last_peered":"2024-02-18T00:48:38.435734+0000","last_clean":"2024-02-18T00:48:38.435734+0000","last_became_active":"2024-02-18T00:29:19.099446+0000","last_became_peered":"2024-02-18T00:29:19.099446+0000","last_unstale":"2024-02-18T00:48:38.435734+0000","last_undegraded":"2024-02-18T00:48:38.435734+0000","last_fullsized":"2024-02-18T00:48:38.435734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:38.435618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:38.435618+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":838,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793266+0000","last_change":"2024-02-18T00:47:46.084843+0000","last_active":"2024-02-18T00:48:36.793266+0000","last_peered":"2024-02-18T00:48:36.793266+0000","last_clean":"2024-02-18T00:48:36.793266+0000","last_became_active":"2024-02-18T00:29:21.101125+0000","last_became_peered":"2024-02-18T00:29:21.101125+0000","last_unstale":"2024-02-18T00:48:36.793266+0000","last_undegraded":"2024-02-18T00:48:36.793266+0000","last_fullsized":"2024-02-18T00:48:36.793266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:46.084758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:46.084758+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":782,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794077+0000","last_change":"2024-02-18T00:47:52.024589+0000","last_active":"2024-02-18T00:48:36.794077+0000","last_peered":"2024-02-18T00:48:36.794077+0000","last_clean":"2024-02-18T00:48:36.794077+0000","last_became_active":"2024-02-18T00:33:04.653802+0000","last_became_peered":"2024-02-18T00:33:04.653802+0000","last_unstale":"2024-02-18T00:48:36.794077+0000","last_undegraded":"2024-02-18T00:48:36.794077+0000","last_fullsized":"2024-02-18T00:48:36.794077+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:52.024516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:52.024516+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.024516+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"457'19","reported_seq":916,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407018+0000","last_change":"2024-02-18T00:48:11.389874+0000","last_active":"2024-02-18T00:48:36.407018+0000","last_peered":"2024-02-18T00:48:36.407018+0000","last_clean":"2024-02-18T00:48:36.407018+0000","last_became_active":"2024-02-18T00:46:25.878984+0000","last_became_peered":"2024-02-18T00:46:25.878984+0000","last_unstale":"2024-02-18T00:48:36.407018+0000","last_undegraded":"2024-02-18T00:48:36.407018+0000","last_fullsized":"2024-02-18T00:48:36.407018+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'19","last_scrub_stamp":"2024-02-18T00:48:11.389795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.389795+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":848,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292779+0000","last_change":"2024-02-18T00:48:24.781709+0000","last_active":"2024-02-18T00:48:36.292779+0000","last_peered":"2024-02-18T00:48:36.292779+0000","last_clean":"2024-02-18T00:48:36.292779+0000","last_became_active":"2024-02-18T00:29:14.716625+0000","last_became_peered":"2024-02-18T00:29:14.716625+0000","last_unstale":"2024-02-18T00:48:36.292779+0000","last_undegraded":"2024-02-18T00:48:36.292779+0000","last_fullsized":"2024-02-18T00:48:36.292779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:24.781645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:24.781645+0000","last_clean_scrub_stamp":"2024-02-18T00:48:24.781645+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":849,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293706+0000","last_change":"2024-02-18T00:48:32.770838+0000","last_active":"2024-02-18T00:48:36.293706+0000","last_peered":"2024-02-18T00:48:36.293706+0000","last_clean":"2024-02-18T00:48:36.293706+0000","last_became_active":"2024-02-18T00:29:19.098369+0000","last_became_peered":"2024-02-18T00:29:19.098369+0000","last_unstale":"2024-02-18T00:48:36.293706+0000","last_undegraded":"2024-02-18T00:48:36.293706+0000","last_fullsized":"2024-02-18T00:48:36.293706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:32.770769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:39.958201+0000","last_clean_scrub_stamp":"2024-02-18T00:48:32.770769+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":828,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.376954+0000","last_change":"2024-02-18T00:47:47.674631+0000","last_active":"2024-02-18T00:48:36.376954+0000","last_peered":"2024-02-18T00:48:36.376954+0000","last_clean":"2024-02-18T00:48:36.376954+0000","last_became_active":"2024-02-18T00:29:21.101349+0000","last_became_peered":"2024-02-18T00:29:21.101349+0000","last_unstale":"2024-02-18T00:48:36.376954+0000","last_undegraded":"2024-02-18T00:48:36.376954+0000","last_fullsized":"2024-02-18T00:48:36.376954+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:47.674565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:47.674565+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:37.597512+0000","last_change":"2024-02-18T00:48:37.597465+0000","last_active":"2024-02-18T00:48:37.597512+0000","last_peered":"2024-02-18T00:48:37.597512+0000","last_clean":"2024-02-18T00:48:37.597512+0000","last_became_active":"2024-02-18T00:46:25.877319+0000","last_became_peered":"2024-02-18T00:46:25.877319+0000","last_unstale":"2024-02-18T00:48:37.597512+0000","last_undegraded":"2024-02-18T00:48:37.597512+0000","last_fullsized":"2024-02-18T00:48:37.597512+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:37.597372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:37.597372+0000","last_clean_scrub_stamp":"2024-02-18T00:48:37.597372+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"855'10","reported_seq":881,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:42.424614+0000","last_change":"2024-02-18T00:48:42.424614+0000","last_active":"2024-02-18T00:48:42.424614+0000","last_peered":"2024-02-18T00:48:42.424614+0000","last_clean":"2024-02-18T00:48:42.424614+0000","last_became_active":"2024-02-18T00:46:25.877523+0000","last_became_peered":"2024-02-18T00:46:25.877523+0000","last_unstale":"2024-02-18T00:48:42.424614+0000","last_undegraded":"2024-02-18T00:48:42.424614+0000","last_fullsized":"2024-02-18T00:48:42.424614+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"855'10","last_scrub_stamp":"2024-02-18T00:48:42.424538+0000","last_deep_scrub":"855'10","last_deep_scrub_stamp":"2024-02-18T00:48:42.424538+0000","last_clean_scrub_stamp":"2024-02-18T00:48:42.424538+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":863,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794961+0000","last_change":"2024-02-18T00:48:26.857897+0000","last_active":"2024-02-18T00:48:36.794961+0000","last_peered":"2024-02-18T00:48:36.794961+0000","last_clean":"2024-02-18T00:48:36.794961+0000","last_became_active":"2024-02-18T00:46:25.877697+0000","last_became_peered":"2024-02-18T00:46:25.877697+0000","last_unstale":"2024-02-18T00:48:36.794961+0000","last_undegraded":"2024-02-18T00:48:36.794961+0000","last_fullsized":"2024-02-18T00:48:36.794961+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:26.857823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:07.782283+0000","last_clean_scrub_stamp":"2024-02-18T00:48:26.857823+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795441+0000","last_change":"2024-02-18T00:47:48.554083+0000","last_active":"2024-02-18T00:48:36.795441+0000","last_peered":"2024-02-18T00:48:36.795441+0000","last_clean":"2024-02-18T00:48:36.795441+0000","last_became_active":"2024-02-18T00:46:25.889345+0000","last_became_peered":"2024-02-18T00:46:25.889345+0000","last_unstale":"2024-02-18T00:48:36.795441+0000","last_undegraded":"2024-02-18T00:48:36.795441+0000","last_fullsized":"2024-02-18T00:48:36.795441+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:48.554001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:47:48.554001+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291810+0000","last_change":"2024-02-18T00:48:04.172891+0000","last_active":"2024-02-18T00:48:36.291810+0000","last_peered":"2024-02-18T00:48:36.291810+0000","last_clean":"2024-02-18T00:48:36.291810+0000","last_became_active":"2024-02-18T00:39:36.767204+0000","last_became_peered":"2024-02-18T00:39:36.767204+0000","last_unstale":"2024-02-18T00:48:36.291810+0000","last_undegraded":"2024-02-18T00:48:36.291810+0000","last_fullsized":"2024-02-18T00:48:36.291810+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:04.172800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:48:04.172800+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":798,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795313+0000","last_change":"2024-02-18T00:48:09.832713+0000","last_active":"2024-02-18T00:48:36.795313+0000","last_peered":"2024-02-18T00:48:36.795313+0000","last_clean":"2024-02-18T00:48:36.795313+0000","last_became_active":"2024-02-18T00:39:36.755111+0000","last_became_peered":"2024-02-18T00:39:36.755111+0000","last_unstale":"2024-02-18T00:48:36.795313+0000","last_undegraded":"2024-02-18T00:48:36.795313+0000","last_fullsized":"2024-02-18T00:48:36.795313+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.832634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.832634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"457'25","reported_seq":970,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377477+0000","last_change":"2024-02-18T00:47:54.667460+0000","last_active":"2024-02-18T00:48:36.377477+0000","last_peered":"2024-02-18T00:48:36.377477+0000","last_clean":"2024-02-18T00:48:36.377477+0000","last_became_active":"2024-02-18T00:46:25.877948+0000","last_became_peered":"2024-02-18T00:46:25.877948+0000","last_unstale":"2024-02-18T00:48:36.377477+0000","last_undegraded":"2024-02-18T00:48:36.377477+0000","last_fullsized":"2024-02-18T00:48:36.377477+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:47:54.667369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:54.667369+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":44,"num_read_kb":36,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794929+0000","last_change":"2024-02-18T00:48:24.653307+0000","last_active":"2024-02-18T00:48:36.794929+0000","last_peered":"2024-02-18T00:48:36.794929+0000","last_clean":"2024-02-18T00:48:36.794929+0000","last_became_active":"2024-02-18T00:46:25.887695+0000","last_became_peered":"2024-02-18T00:46:25.887695+0000","last_unstale":"2024-02-18T00:48:36.794929+0000","last_undegraded":"2024-02-18T00:48:36.794929+0000","last_fullsized":"2024-02-18T00:48:36.794929+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:24.653226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:43:03.852193+0000","last_clean_scrub_stamp":"2024-02-18T00:48:24.653226+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":421,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.291833+0000","last_change":"2024-02-18T00:48:00.117023+0000","last_active":"2024-02-18T00:48:36.291833+0000","last_peered":"2024-02-18T00:48:36.291833+0000","last_clean":"2024-02-18T00:48:36.291833+0000","last_became_active":"2024-02-18T00:46:25.881315+0000","last_became_peered":"2024-02-18T00:46:25.881315+0000","last_unstale":"2024-02-18T00:48:36.291833+0000","last_undegraded":"2024-02-18T00:48:36.291833+0000","last_fullsized":"2024-02-18T00:48:36.291833+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:00.116944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:00.116944+0000","last_clean_scrub_stamp":"2024-02-18T00:48:00.116944+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":846,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377619+0000","last_change":"2024-02-18T00:47:37.695596+0000","last_active":"2024-02-18T00:48:36.377619+0000","last_peered":"2024-02-18T00:48:36.377619+0000","last_clean":"2024-02-18T00:48:36.377619+0000","last_became_active":"2024-02-18T00:46:25.884576+0000","last_became_peered":"2024-02-18T00:46:25.884576+0000","last_unstale":"2024-02-18T00:48:36.377619+0000","last_undegraded":"2024-02-18T00:48:36.377619+0000","last_fullsized":"2024-02-18T00:48:36.377619+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:37.695526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:20.081568+0000","last_clean_scrub_stamp":"2024-02-18T00:47:37.695526+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":413,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:39.005444+0000","last_change":"2024-02-18T00:48:39.005444+0000","last_active":"2024-02-18T00:48:39.005444+0000","last_peered":"2024-02-18T00:48:39.005444+0000","last_clean":"2024-02-18T00:48:39.005444+0000","last_became_active":"2024-02-18T00:39:36.757406+0000","last_became_peered":"2024-02-18T00:39:36.757406+0000","last_unstale":"2024-02-18T00:48:39.005444+0000","last_undegraded":"2024-02-18T00:48:39.005444+0000","last_fullsized":"2024-02-18T00:48:39.005444+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:39.005355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:16.073560+0000","last_clean_scrub_stamp":"2024-02-18T00:48:39.005355+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"457'32","reported_seq":939,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794331+0000","last_change":"2024-02-18T00:48:12.086826+0000","last_active":"2024-02-18T00:48:36.794331+0000","last_peered":"2024-02-18T00:48:36.794331+0000","last_clean":"2024-02-18T00:48:36.794331+0000","last_became_active":"2024-02-18T00:39:36.771510+0000","last_became_peered":"2024-02-18T00:39:36.771510+0000","last_unstale":"2024-02-18T00:48:36.794331+0000","last_undegraded":"2024-02-18T00:48:36.794331+0000","last_fullsized":"2024-02-18T00:48:36.794331+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":466,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'32","last_scrub_stamp":"2024-02-18T00:48:12.086743+0000","last_deep_scrub":"157'22","last_deep_scrub_stamp":"2024-02-18T00:33:56.121872+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.086743+0000","log_size":32,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":36,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":867,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.793936+0000","last_change":"2024-02-18T00:48:29.905227+0000","last_active":"2024-02-18T00:48:36.793936+0000","last_peered":"2024-02-18T00:48:36.793936+0000","last_clean":"2024-02-18T00:48:36.793936+0000","last_became_active":"2024-02-18T00:39:43.716806+0000","last_became_peered":"2024-02-18T00:39:43.716806+0000","last_unstale":"2024-02-18T00:48:36.793936+0000","last_undegraded":"2024-02-18T00:48:36.793936+0000","last_fullsized":"2024-02-18T00:48:36.793936+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:29.905152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:48:29.905152+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1f","version":"811'12","reported_seq":94,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.497382+0000","last_change":"2024-02-18T00:47:52.510602+0000","last_active":"2024-02-18T00:48:43.497382+0000","last_peered":"2024-02-18T00:48:43.497382+0000","last_clean":"2024-02-18T00:48:43.497382+0000","last_became_active":"2024-02-18T00:46:25.888871+0000","last_became_peered":"2024-02-18T00:46:25.888871+0000","last_unstale":"2024-02-18T00:48:43.497382+0000","last_undegraded":"2024-02-18T00:48:43.497382+0000","last_fullsized":"2024-02-18T00:48:43.497382+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"811'12","last_scrub_stamp":"2024-02-18T00:47:52.510507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.510507+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":415,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:47.990833+0000","last_change":"2024-02-18T00:48:47.990833+0000","last_active":"2024-02-18T00:48:47.990833+0000","last_peered":"2024-02-18T00:48:47.990833+0000","last_clean":"2024-02-18T00:48:47.990833+0000","last_became_active":"2024-02-18T00:39:36.763986+0000","last_became_peered":"2024-02-18T00:39:36.763986+0000","last_unstale":"2024-02-18T00:48:47.990833+0000","last_undegraded":"2024-02-18T00:48:47.990833+0000","last_fullsized":"2024-02-18T00:48:47.990833+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:47.990761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:47.990761+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"457'25","reported_seq":80,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794992+0000","last_change":"2024-02-18T00:48:35.538755+0000","last_active":"2024-02-18T00:48:36.794992+0000","last_peered":"2024-02-18T00:48:36.794992+0000","last_clean":"2024-02-18T00:48:36.794992+0000","last_became_active":"2024-02-18T00:46:25.888212+0000","last_became_peered":"2024-02-18T00:46:25.888212+0000","last_unstale":"2024-02-18T00:48:36.794992+0000","last_undegraded":"2024-02-18T00:48:36.794992+0000","last_fullsized":"2024-02-18T00:48:36.794992+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'25","last_scrub_stamp":"2024-02-18T00:48:35.538662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:35.538662+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":35,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":857,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:49.457388+0000","last_change":"2024-02-18T00:48:49.457388+0000","last_active":"2024-02-18T00:48:49.457388+0000","last_peered":"2024-02-18T00:48:49.457388+0000","last_clean":"2024-02-18T00:48:49.457388+0000","last_became_active":"2024-02-18T00:46:25.884395+0000","last_became_peered":"2024-02-18T00:46:25.884395+0000","last_unstale":"2024-02-18T00:48:49.457388+0000","last_undegraded":"2024-02-18T00:48:49.457388+0000","last_fullsized":"2024-02-18T00:48:49.457388+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:49.457311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:45:38.322790+0000","last_clean_scrub_stamp":"2024-02-18T00:48:49.457311+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":853,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794345+0000","last_change":"2024-02-18T00:48:06.045344+0000","last_active":"2024-02-18T00:48:36.794345+0000","last_peered":"2024-02-18T00:48:36.794345+0000","last_clean":"2024-02-18T00:48:36.794345+0000","last_became_active":"2024-02-18T00:39:36.771004+0000","last_became_peered":"2024-02-18T00:39:36.771004+0000","last_unstale":"2024-02-18T00:48:36.794345+0000","last_undegraded":"2024-02-18T00:48:36.794345+0000","last_fullsized":"2024-02-18T00:48:36.794345+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:06.045277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:44:46.487955+0000","last_clean_scrub_stamp":"2024-02-18T00:48:06.045277+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1","version":"109'7","reported_seq":827,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.989922+0000","last_change":"2024-02-18T00:48:38.989890+0000","last_active":"2024-02-18T00:48:38.989922+0000","last_peered":"2024-02-18T00:48:38.989922+0000","last_clean":"2024-02-18T00:48:38.989922+0000","last_became_active":"2024-02-18T00:46:25.876848+0000","last_became_peered":"2024-02-18T00:46:25.876848+0000","last_unstale":"2024-02-18T00:48:38.989922+0000","last_undegraded":"2024-02-18T00:48:38.989922+0000","last_fullsized":"2024-02-18T00:48:38.989922+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'7","last_scrub_stamp":"2024-02-18T00:48:38.989811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:33:01.316512+0000","last_clean_scrub_stamp":"2024-02-18T00:48:38.989811+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":10,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1c","version":"457'27","reported_seq":889,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292868+0000","last_change":"2024-02-18T00:47:45.735338+0000","last_active":"2024-02-18T00:48:36.292868+0000","last_peered":"2024-02-18T00:48:36.292868+0000","last_clean":"2024-02-18T00:48:36.292868+0000","last_became_active":"2024-02-18T00:29:17.095084+0000","last_became_peered":"2024-02-18T00:29:17.095084+0000","last_unstale":"2024-02-18T00:48:36.292868+0000","last_undegraded":"2024-02-18T00:48:36.292868+0000","last_fullsized":"2024-02-18T00:48:36.292868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'27","last_scrub_stamp":"2024-02-18T00:47:45.735267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:47:45.735267+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794971+0000","last_change":"2024-02-18T00:47:39.615410+0000","last_active":"2024-02-18T00:48:36.794971+0000","last_peered":"2024-02-18T00:48:36.794971+0000","last_clean":"2024-02-18T00:48:36.794971+0000","last_became_active":"2024-02-18T00:46:25.888425+0000","last_became_peered":"2024-02-18T00:46:25.888425+0000","last_unstale":"2024-02-18T00:48:36.794971+0000","last_undegraded":"2024-02-18T00:48:36.794971+0000","last_fullsized":"2024-02-18T00:48:36.794971+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:39.615298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:13.697546+0000","last_clean_scrub_stamp":"2024-02-18T00:47:39.615298+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":849,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:50.657761+0000","last_change":"2024-02-18T00:48:50.657761+0000","last_active":"2024-02-18T00:48:50.657761+0000","last_peered":"2024-02-18T00:48:50.657761+0000","last_clean":"2024-02-18T00:48:50.657761+0000","last_became_active":"2024-02-18T00:29:19.090285+0000","last_became_peered":"2024-02-18T00:29:19.090285+0000","last_unstale":"2024-02-18T00:48:50.657761+0000","last_undegraded":"2024-02-18T00:48:50.657761+0000","last_fullsized":"2024-02-18T00:48:50.657761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:50.657653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:31:02.145253+0000","last_clean_scrub_stamp":"2024-02-18T00:48:50.657653+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":66,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795391+0000","last_change":"2024-02-18T00:48:11.459844+0000","last_active":"2024-02-18T00:48:36.795391+0000","last_peered":"2024-02-18T00:48:36.795391+0000","last_clean":"2024-02-18T00:48:36.795391+0000","last_became_active":"2024-02-18T00:46:25.887945+0000","last_became_peered":"2024-02-18T00:46:25.887945+0000","last_unstale":"2024-02-18T00:48:36.795391+0000","last_undegraded":"2024-02-18T00:48:36.795391+0000","last_fullsized":"2024-02-18T00:48:36.795391+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:11.459762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:42:16.894524+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.459762+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"813'30","reported_seq":885,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377686+0000","last_change":"2024-02-18T00:48:05.661177+0000","last_active":"2024-02-18T00:48:36.377686+0000","last_peered":"2024-02-18T00:48:36.377686+0000","last_clean":"2024-02-18T00:48:36.377686+0000","last_became_active":"2024-02-18T00:29:17.093213+0000","last_became_peered":"2024-02-18T00:29:17.093213+0000","last_unstale":"2024-02-18T00:48:36.377686+0000","last_undegraded":"2024-02-18T00:48:36.377686+0000","last_fullsized":"2024-02-18T00:48:36.377686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"813'30","last_scrub_stamp":"2024-02-18T00:48:05.661100+0000","last_deep_scrub":"813'30","last_deep_scrub_stamp":"2024-02-18T00:46:32.042766+0000","last_clean_scrub_stamp":"2024-02-18T00:48:05.661100+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":36,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"74'2","reported_seq":898,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:48.407865+0000","last_change":"2024-02-18T00:48:48.407865+0000","last_active":"2024-02-18T00:48:48.407865+0000","last_peered":"2024-02-18T00:48:48.407865+0000","last_clean":"2024-02-18T00:48:48.407865+0000","last_became_active":"2024-02-18T00:46:25.877602+0000","last_became_peered":"2024-02-18T00:46:25.877602+0000","last_unstale":"2024-02-18T00:48:48.407865+0000","last_undegraded":"2024-02-18T00:48:48.407865+0000","last_fullsized":"2024-02-18T00:48:48.407865+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'2","last_scrub_stamp":"2024-02-18T00:48:48.407781+0000","last_deep_scrub":"74'2","last_deep_scrub_stamp":"2024-02-18T00:30:25.803723+0000","last_clean_scrub_stamp":"2024-02-18T00:48:48.407781+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":841,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.407520+0000","last_change":"2024-02-18T00:48:28.492357+0000","last_active":"2024-02-18T00:48:36.407520+0000","last_peered":"2024-02-18T00:48:36.407520+0000","last_clean":"2024-02-18T00:48:36.407520+0000","last_became_active":"2024-02-18T00:29:19.101492+0000","last_became_peered":"2024-02-18T00:29:19.101492+0000","last_unstale":"2024-02-18T00:48:36.407520+0000","last_undegraded":"2024-02-18T00:48:36.407520+0000","last_fullsized":"2024-02-18T00:48:36.407520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:28.492238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:03.672462+0000","last_clean_scrub_stamp":"2024-02-18T00:48:28.492238+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":816,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.794406+0000","last_change":"2024-02-18T00:47:41.102035+0000","last_active":"2024-02-18T00:48:36.794406+0000","last_peered":"2024-02-18T00:48:36.794406+0000","last_clean":"2024-02-18T00:48:36.794406+0000","last_became_active":"2024-02-18T00:46:25.877473+0000","last_became_peered":"2024-02-18T00:46:25.877473+0000","last_unstale":"2024-02-18T00:48:36.794406+0000","last_undegraded":"2024-02-18T00:48:36.794406+0000","last_fullsized":"2024-02-18T00:48:36.794406+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:41.101964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:41.101964+0000","last_clean_scrub_stamp":"2024-02-18T00:47:41.101964+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"457'34","reported_seq":912,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:41.639902+0000","last_change":"2024-02-18T00:48:41.639902+0000","last_active":"2024-02-18T00:48:41.639902+0000","last_peered":"2024-02-18T00:48:41.639902+0000","last_clean":"2024-02-18T00:48:41.639902+0000","last_became_active":"2024-02-18T00:29:17.086020+0000","last_became_peered":"2024-02-18T00:29:17.086020+0000","last_unstale":"2024-02-18T00:48:41.639902+0000","last_undegraded":"2024-02-18T00:48:41.639902+0000","last_fullsized":"2024-02-18T00:48:41.639902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'34","last_scrub_stamp":"2024-02-18T00:48:41.639819+0000","last_deep_scrub":"105'28","last_deep_scrub_stamp":"2024-02-18T00:35:19.264580+0000","last_clean_scrub_stamp":"2024-02-18T00:48:41.639819+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":42,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":408,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292227+0000","last_change":"2024-02-18T00:48:09.252416+0000","last_active":"2024-02-18T00:48:36.292227+0000","last_peered":"2024-02-18T00:48:36.292227+0000","last_clean":"2024-02-18T00:48:36.292227+0000","last_became_active":"2024-02-18T00:39:36.769227+0000","last_became_peered":"2024-02-18T00:39:36.769227+0000","last_unstale":"2024-02-18T00:48:36.292227+0000","last_undegraded":"2024-02-18T00:48:36.292227+0000","last_fullsized":"2024-02-18T00:48:36.292227+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:09.252337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:09.252337+0000","last_clean_scrub_stamp":"2024-02-18T00:48:09.252337+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":877,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.795446+0000","last_change":"2024-02-18T00:48:11.771480+0000","last_active":"2024-02-18T00:48:36.795446+0000","last_peered":"2024-02-18T00:48:36.795446+0000","last_clean":"2024-02-18T00:48:36.795446+0000","last_became_active":"2024-02-18T00:45:35.150420+0000","last_became_peered":"2024-02-18T00:45:35.150420+0000","last_unstale":"2024-02-18T00:48:36.795446+0000","last_undegraded":"2024-02-18T00:48:36.795446+0000","last_fullsized":"2024-02-18T00:48:36.795446+0000","mapping_epoch":770,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":771,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:11.771383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:11.771383+0000","last_clean_scrub_stamp":"2024-02-18T00:48:11.771383+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":785,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293259+0000","last_change":"2024-02-18T00:48:18.729333+0000","last_active":"2024-02-18T00:48:36.293259+0000","last_peered":"2024-02-18T00:48:36.293259+0000","last_clean":"2024-02-18T00:48:36.293259+0000","last_became_active":"2024-02-18T00:33:04.669516+0000","last_became_peered":"2024-02-18T00:33:04.669516+0000","last_unstale":"2024-02-18T00:48:36.293259+0000","last_undegraded":"2024-02-18T00:48:36.293259+0000","last_fullsized":"2024-02-18T00:48:36.293259+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:18.729250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:46.037549+0000","last_clean_scrub_stamp":"2024-02-18T00:48:18.729250+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"457'16","reported_seq":874,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:38.557180+0000","last_change":"2024-02-18T00:48:38.557148+0000","last_active":"2024-02-18T00:48:38.557180+0000","last_peered":"2024-02-18T00:48:38.557180+0000","last_clean":"2024-02-18T00:48:38.557180+0000","last_became_active":"2024-02-18T00:29:17.086330+0000","last_became_peered":"2024-02-18T00:29:17.086330+0000","last_unstale":"2024-02-18T00:48:38.557180+0000","last_undegraded":"2024-02-18T00:48:38.557180+0000","last_fullsized":"2024-02-18T00:48:38.557180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'16","last_scrub_stamp":"2024-02-18T00:48:38.557063+0000","last_deep_scrub":"82'14","last_deep_scrub_stamp":"2024-02-18T00:30:46.066555+0000","last_clean_scrub_stamp":"2024-02-18T00:48:38.557063+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":21,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":426,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.292418+0000","last_change":"2024-02-18T00:47:49.142720+0000","last_active":"2024-02-18T00:48:36.292418+0000","last_peered":"2024-02-18T00:48:36.292418+0000","last_clean":"2024-02-18T00:48:36.292418+0000","last_became_active":"2024-02-18T00:46:25.884881+0000","last_became_peered":"2024-02-18T00:46:25.884881+0000","last_unstale":"2024-02-18T00:48:36.292418+0000","last_undegraded":"2024-02-18T00:48:36.292418+0000","last_fullsized":"2024-02-18T00:48:36.292418+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:49.142613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:34:31.241146+0000","last_clean_scrub_stamp":"2024-02-18T00:47:49.142613+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":408,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:43.059132+0000","last_change":"2024-02-18T00:48:43.059132+0000","last_active":"2024-02-18T00:48:43.059132+0000","last_peered":"2024-02-18T00:48:43.059132+0000","last_clean":"2024-02-18T00:48:43.059132+0000","last_became_active":"2024-02-18T00:39:36.758977+0000","last_became_peered":"2024-02-18T00:39:36.758977+0000","last_unstale":"2024-02-18T00:48:43.059132+0000","last_undegraded":"2024-02-18T00:48:43.059132+0000","last_fullsized":"2024-02-18T00:48:43.059132+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:43.059055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:48:43.059055+0000","last_clean_scrub_stamp":"2024-02-18T00:48:43.059055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":792,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377972+0000","last_change":"2024-02-18T00:48:12.670696+0000","last_active":"2024-02-18T00:48:36.377972+0000","last_peered":"2024-02-18T00:48:36.377972+0000","last_clean":"2024-02-18T00:48:36.377972+0000","last_became_active":"2024-02-18T00:39:36.766447+0000","last_became_peered":"2024-02-18T00:39:36.766447+0000","last_unstale":"2024-02-18T00:48:36.377972+0000","last_undegraded":"2024-02-18T00:48:36.377972+0000","last_fullsized":"2024-02-18T00:48:36.377972+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:12.670627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:46:44.868415+0000","last_clean_scrub_stamp":"2024-02-18T00:48:12.670627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"457'26","reported_seq":925,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406429+0000","last_change":"2024-02-18T00:47:49.529429+0000","last_active":"2024-02-18T00:48:36.406429+0000","last_peered":"2024-02-18T00:48:36.406429+0000","last_clean":"2024-02-18T00:48:36.406429+0000","last_became_active":"2024-02-18T00:46:25.869773+0000","last_became_peered":"2024-02-18T00:46:25.869773+0000","last_unstale":"2024-02-18T00:48:36.406429+0000","last_undegraded":"2024-02-18T00:48:36.406429+0000","last_fullsized":"2024-02-18T00:48:36.406429+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'26","last_scrub_stamp":"2024-02-18T00:47:49.529319+0000","last_deep_scrub":"457'26","last_deep_scrub_stamp":"2024-02-18T00:47:49.529319+0000","last_clean_scrub_stamp":"2024-02-18T00:47:49.529319+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":834,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377830+0000","last_change":"2024-02-18T00:47:49.756873+0000","last_active":"2024-02-18T00:48:36.377830+0000","last_peered":"2024-02-18T00:48:36.377830+0000","last_clean":"2024-02-18T00:48:36.377830+0000","last_became_active":"2024-02-18T00:29:14.716719+0000","last_became_peered":"2024-02-18T00:29:14.716719+0000","last_unstale":"2024-02-18T00:48:36.377830+0000","last_undegraded":"2024-02-18T00:48:36.377830+0000","last_fullsized":"2024-02-18T00:48:36.377830+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:49.756796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:30:48.890677+0000","last_clean_scrub_stamp":"2024-02-18T00:47:49.756796+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":862,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.293399+0000","last_change":"2024-02-18T00:48:08.737377+0000","last_active":"2024-02-18T00:48:36.293399+0000","last_peered":"2024-02-18T00:48:36.293399+0000","last_clean":"2024-02-18T00:48:36.293399+0000","last_became_active":"2024-02-18T00:39:36.745388+0000","last_became_peered":"2024-02-18T00:39:36.745388+0000","last_unstale":"2024-02-18T00:48:36.293399+0000","last_undegraded":"2024-02-18T00:48:36.293399+0000","last_fullsized":"2024-02-18T00:48:36.293399+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:08.737294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:08.737294+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":794,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377992+0000","last_change":"2024-02-18T00:47:52.687520+0000","last_active":"2024-02-18T00:48:36.377992+0000","last_peered":"2024-02-18T00:48:36.377992+0000","last_clean":"2024-02-18T00:48:36.377992+0000","last_became_active":"2024-02-18T00:46:25.877328+0000","last_became_peered":"2024-02-18T00:46:25.877328+0000","last_unstale":"2024-02-18T00:48:36.377992+0000","last_undegraded":"2024-02-18T00:48:36.377992+0000","last_fullsized":"2024-02-18T00:48:36.377992+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:47:52.687448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:52.687448+0000","last_clean_scrub_stamp":"2024-02-18T00:47:52.687448+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"457'51","reported_seq":937,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.377375+0000","last_change":"2024-02-18T00:48:35.498016+0000","last_active":"2024-02-18T00:48:36.377375+0000","last_peered":"2024-02-18T00:48:36.377375+0000","last_clean":"2024-02-18T00:48:36.377375+0000","last_became_active":"2024-02-18T00:29:17.091215+0000","last_became_peered":"2024-02-18T00:29:17.091215+0000","last_unstale":"2024-02-18T00:48:36.377375+0000","last_undegraded":"2024-02-18T00:48:36.377375+0000","last_fullsized":"2024-02-18T00:48:36.377375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"457'51","last_scrub_stamp":"2024-02-18T00:48:35.497936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:16.066615+0000","last_clean_scrub_stamp":"2024-02-18T00:48:35.497936+0000","log_size":51,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":92,"num_read_kb":79,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":864,"reported_epoch":855,"state":"active+clean","last_fresh":"2024-02-18T00:48:50.141308+0000","last_change":"2024-02-18T00:48:50.141308+0000","last_active":"2024-02-18T00:48:50.141308+0000","last_peered":"2024-02-18T00:48:50.141308+0000","last_clean":"2024-02-18T00:48:50.141308+0000","last_became_active":"2024-02-18T00:39:36.772254+0000","last_became_peered":"2024-02-18T00:39:36.772254+0000","last_unstale":"2024-02-18T00:48:50.141308+0000","last_undegraded":"2024-02-18T00:48:50.141308+0000","last_fullsized":"2024-02-18T00:48:50.141308+0000","mapping_epoch":464,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":465,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:50.141244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:47:16.113781+0000","last_clean_scrub_stamp":"2024-02-18T00:48:50.141244+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":859,"reported_epoch":854,"state":"active+clean","last_fresh":"2024-02-18T00:48:36.406610+0000","last_change":"2024-02-18T00:48:06.405599+0000","last_active":"2024-02-18T00:48:36.406610+0000","last_peered":"2024-02-18T00:48:36.406610+0000","last_clean":"2024-02-18T00:48:36.406610+0000","last_became_active":"2024-02-18T00:46:25.869741+0000","last_became_peered":"2024-02-18T00:46:25.869741+0000","last_unstale":"2024-02-18T00:48:36.406610+0000","last_undegraded":"2024-02-18T00:48:36.406610+0000","last_fullsized":"2024-02-18T00:48:36.406610+0000","mapping_epoch":808,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":809,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T00:48:06.405490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T00:29:18.073185+0000","last_clean_scrub_stamp":"2024-02-18T00:48:06.405490+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":5,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":10,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":15,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":96,"acting":96,"num_store_stats":8},{"poolid":6,"num_pg":22,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":66,"acting":66,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":42,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":940,"num_read_kb":774,"num_write":562,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":3080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1322,"ondisk_log_size":1322,"up":126,"acting":126,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1163,"num_read_kb":1158,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4},{"poolid":2,"num_pg":42,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3362,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":126,"acting":126,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266928,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304464,"kb_used_data":7240,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440688,"statfs":{"total":95995035648,"available":95683264512,"internally_reserved":0,"allocated":7413760,"data_stored":4032682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.502,"15min":0.57399999999999995},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.16900000000000001},"max":{"1min":0.71499999999999997,"5min":0.96399999999999997,"15min":6.8700000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54300000000000004,"15min":0.60899999999999999},"min":{"1min":0.34499999999999997,"5min":0.33300000000000002,"15min":0.218},"max":{"1min":0.77800000000000002,"5min":1.24,"15min":11.288},"last":0.64500000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.63300000000000001},"min":{"1min":0.40699999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.748,"5min":1.7330000000000001,"15min":7.4729999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57599999999999996,"15min":0.65200000000000002},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":1.7150000000000001,"15min":7.7869999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.23300000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":6.7380000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52100000000000002,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.247},"max":{"1min":0.752,"5min":1.3029999999999999,"15min":12.861000000000001},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.58899999999999997},"min":{"1min":0.35999999999999999,"5min":0.34799999999999998,"15min":0.26300000000000001},"max":{"1min":0.73799999999999999,"5min":0.94099999999999995,"15min":7.1360000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53200000000000003,"15min":0.57399999999999995},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.30199999999999999},"max":{"1min":0.76300000000000001,"5min":1.2549999999999999,"15min":3.7989999999999999},"last":0.54300000000000004}]},{"osd":4,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.434,"15min":0.435},"min":{"1min":0.30199999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":1.1779999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.45200000000000001,"15min":0.48899999999999999},"min":{"1min":0.34699999999999998,"5min":0.28799999999999998,"15min":0.246},"max":{"1min":0.70299999999999996,"5min":0.84099999999999997,"15min":4.7859999999999996},"last":0.48399999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.746,"5min":0.95299999999999996,"15min":1.877},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47999999999999998,"15min":0.51100000000000001},"min":{"1min":0.373,"5min":0.27500000000000002,"15min":0.26600000000000001},"max":{"1min":0.83099999999999996,"5min":0.85899999999999999,"15min":4.9130000000000003},"last":0.45300000000000001}]},{"osd":6,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.9019999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55000000000000004,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.309,"15min":0.28100000000000003},"max":{"1min":0.77400000000000002,"5min":1.4279999999999999,"15min":5.3860000000000001},"last":0.66800000000000004}]}]},{"osd":6,"up_from":47,"seq":201863463158,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312660,"kb_used_data":7244,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93432492,"statfs":{"total":95995035648,"available":95674871808,"internally_reserved":0,"allocated":7417856,"data_stored":4034116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53100000000000003,"15min":1.272},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.22500000000000001},"max":{"1min":0.73899999999999999,"5min":0.83299999999999996,"15min":69.194999999999993},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.75},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":1.179,"15min":10.433},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":1.355},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.27900000000000003},"max":{"1min":0.79500000000000004,"5min":1.9610000000000001,"15min":76.108999999999995},"last":0.626},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57999999999999996,"15min":1.3500000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70399999999999996,"5min":3.665,"15min":71.872},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57699999999999996,"15min":0.83299999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31},"max":{"1min":0.77900000000000003,"5min":3.702,"15min":39.939999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.32800000000000001},"max":{"1min":0.747,"5min":1.276,"15min":39.673000000000002},"last":0.52600000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.84999999999999998},"min":{"1min":0.38100000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70299999999999996,"5min":0.81899999999999995,"15min":21.219000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.61399999999999999,"15min":0.81200000000000006},"min":{"1min":0.41499999999999998,"5min":0.33300000000000002,"15min":0.29299999999999998},"max":{"1min":0.81799999999999995,"5min":3.7280000000000002,"15min":14.007999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.52600000000000002},"min":{"1min":0.27600000000000002,"5min":0.25800000000000001,"15min":0.22800000000000001},"max":{"1min":0.73599999999999999,"5min":1.2170000000000001,"15min":8.1479999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.53000000000000003,"15min":0.72699999999999998},"min":{"1min":0.34899999999999998,"5min":0.27100000000000002,"15min":0.26100000000000001},"max":{"1min":0.81399999999999995,"5min":4.1310000000000002,"15min":35.18},"last":0.39700000000000002}]},{"osd":5,"last update":"Sun Feb 18 00:48:13 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.55700000000000005},"min":{"1min":0.314,"5min":0.251,"15min":0.251},"max":{"1min":0.67900000000000005,"5min":1.3069999999999999,"15min":10.606},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54700000000000004,"15min":0.753},"min":{"1min":0.32400000000000001,"5min":0.30199999999999999,"15min":0.28199999999999997},"max":{"1min":0.78200000000000003,"5min":3.8439999999999999,"15min":35.119},"last":0.435}]},{"osd":7,"last update":"Sun Feb 18 00:48:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.41199999999999998,"5min":0.33500000000000002,"15min":0.29399999999999998},"max":{"1min":0.74399999999999999,"5min":1.2470000000000001,"15min":5.4770000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.36099999999999999,"5min":0.33500000000000002,"15min":0.318},"max":{"1min":0.72999999999999998,"5min":1.7629999999999999,"15min":5.7030000000000003},"last":0.46100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575117,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308564,"kb_used_data":7244,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436588,"statfs":{"total":95995035648,"available":95679066112,"internally_reserved":0,"allocated":7417856,"data_stored":4032899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33800000000000002,"15min":0.63300000000000001},"min":{"1min":0.22,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":0.55400000000000005,"5min":0.64900000000000002,"15min":32.106999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36699999999999999,"15min":0.70299999999999996},"min":{"1min":0.248,"5min":0.247,"15min":0.20899999999999999},"max":{"1min":0.67900000000000005,"5min":0.83499999999999996,"15min":43.137},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun Feb 18 00:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.36199999999999999,"15min":0.752},"min":{"1min":0.246,"5min":0.21099999999999999,"15min":0.19900000000000001},"max":{"1min":0.51200000000000001,"5min":1.139,"15min":49.201000000000001},"last":0.376},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.39400000000000002,"15min":0.58399999999999996},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.59199999999999997,"5min":0.85899999999999999,"15min":17.596},"last":0.48199999999999998}]},{"osd":3,"last update":"Sun Feb 18 00:48:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.39400000000000002,"15min":0.71099999999999997},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26200000000000001},"max":{"1min":0.60099999999999998,"5min":0.78700000000000003,"15min":37.524000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.73499999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.26400000000000001},"max":{"1min":0.69499999999999995,"5min":0.97599999999999998,"15min":32.036000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Sun Feb 18 00:48:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.751},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.29499999999999998},"max":{"1min":0.84299999999999997,"5min":0.91900000000000004,"15min":17.486999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.76400000000000001},"min":{"1min":0.39100000000000001,"5min":0.33500000000000002,"15min":0.32500000000000001},"max":{"1min":0.91000000000000003,"5min":1.222,"15min":12.75},"last":0.57699999999999996}]},{"osd":5,"last update":"Sun Feb 18 00:48:01 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57299999999999995,"15min":0.75800000000000001},"min":{"1min":0.499,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":1.0109999999999999,"15min":12.611000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.61499999999999999,"15min":0.78000000000000003},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.33100000000000002},"max":{"1min":0.92600000000000005,"5min":1.141,"15min":17.417000000000002},"last":0.60499999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.77700000000000002},"min":{"1min":0.45100000000000001,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.79800000000000004,"5min":1.1839999999999999,"15min":12.834},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.61199999999999999,"15min":0.77700000000000002},"min":{"1min":0.48899999999999999,"5min":0.37,"15min":0.35499999999999998},"max":{"1min":0.90800000000000003,"5min":1.046,"15min":12.651999999999999},"last":0.63}]},{"osd":7,"last update":"Sun Feb 18 00:47:47 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63700000000000001,"15min":0.79700000000000004},"min":{"1min":0.49099999999999999,"5min":0.40500000000000003,"15min":0.38700000000000001},"max":{"1min":0.89600000000000002,"5min":1.175,"15min":17.452000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.79200000000000004},"min":{"1min":0.51400000000000001,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":1.149,"15min":12.805},"last":0.61799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738643,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308560,"kb_used_data":7240,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436592,"statfs":{"total":95995035648,"available":95679070208,"internally_reserved":0,"allocated":7413760,"data_stored":4032405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 00:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.372,"15min":0.97899999999999998},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.18099999999999999},"max":{"1min":0.503,"5min":1.1619999999999999,"15min":133.38999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41799999999999998,"15min":0.79400000000000004},"min":{"1min":0.23100000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59699999999999998,"5min":2.6560000000000001,"15min":90.623000000000005},"last":0.378}]},{"osd":2,"last update":"Sun Feb 18 00:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.438,"15min":0.60599999999999998},"min":{"1min":0.27000000000000002,"5min":0.22700000000000001,"15min":0.20200000000000001},"max":{"1min":0.52300000000000002,"5min":2.915,"15min":21.388999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.39600000000000002,"15min":0.55800000000000005},"min":{"1min":0.25600000000000001,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":0.52100000000000002,"5min":3.3130000000000002,"15min":23.388999999999999},"last":0.27200000000000002}]},{"osd":3,"last update":"Sun Feb 18 00:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40799999999999997,"15min":0.63700000000000001},"min":{"1min":0.27200000000000002,"5min":0.23899999999999999,"15min":0.20899999999999999},"max":{"1min":0.54400000000000004,"5min":2.8170000000000002,"15min":19.006},"last":0.434},{"interface":"front","average":{"1min":0.443,"5min":0.49399999999999999,"15min":1.0600000000000001},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.23999999999999999},"max":{"1min":0.621,"5min":4.2030000000000003,"15min":146.71899999999999},"last":0.41999999999999998}]},{"osd":4,"last update":"Sun Feb 18 00:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.68400000000000005},"min":{"1min":0.39300000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.82099999999999995,"5min":1.885,"15min":12.988},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63100000000000001,"15min":0.70499999999999996},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.30399999999999999},"max":{"1min":0.83599999999999997,"5min":2.8690000000000002,"15min":11.551},"last":0.56299999999999994}]},{"osd":5,"last update":"Sun Feb 18 00:47:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.64000000000000001,"15min":0.96899999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.32000000000000001},"max":{"1min":0.78500000000000003,"5min":4.5499999999999998,"15min":90.049000000000007},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.995},"min":{"1min":0.44,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96699999999999997,"5min":1.853,"15min":57.991},"last":0.52200000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:48:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.623,"15min":0.81000000000000005},"min":{"1min":0.49199999999999999,"5min":0.377,"15min":0.35999999999999999},"max":{"1min":0.94599999999999995,"5min":1.194,"15min":19.039000000000001},"last":0.503},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66600000000000004,"15min":1.0029999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.33500000000000002},"max":{"1min":0.89700000000000002,"5min":3.3540000000000001,"15min":89.989999999999995},"last":0.61099999999999999}]},{"osd":7,"last update":"Sun Feb 18 00:48:45 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68999999999999995,"15min":0.79000000000000004},"min":{"1min":0.46500000000000002,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.76200000000000001,"5min":2.9449999999999998,"15min":13.24},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63800000000000001,"15min":0.75600000000000001},"min":{"1min":0.47199999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81799999999999995,"5min":1.911,"15min":13.019},"last":0.55700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411594,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":308556,"kb_used_data":7236,"kb_used_omap":0,"kb_used_meta":301312,"kb_avail":93436596,"statfs":{"total":95995035648,"available":95679074304,"internally_reserved":0,"allocated":7409664,"data_stored":4032597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308543488},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.44900000000000001,"15min":0.45900000000000002},"min":{"1min":0.23899999999999999,"5min":0.20200000000000001,"15min":0.192},"max":{"1min":0.46300000000000002,"5min":8.6489999999999991,"15min":10.087999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.372,"15min":0.55100000000000005},"min":{"1min":0.26300000000000001,"5min":0.23899999999999999,"15min":0.19800000000000001},"max":{"1min":0.50600000000000001,"5min":0.59099999999999997,"15min":18.891999999999999},"last":0.30299999999999999}]},{"osd":1,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38400000000000001,"15min":0.45100000000000001},"min":{"1min":0.26100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.61199999999999999,"5min":0.85199999999999998,"15min":10.138999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.41599999999999998,"15min":0.52100000000000002},"min":{"1min":0.26600000000000001,"5min":0.246,"15min":0.20399999999999999},"max":{"1min":0.53400000000000003,"5min":1.7869999999999999,"15min":18.919},"last":0.23899999999999999}]},{"osd":3,"last update":"Sun Feb 18 00:48:06 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.50600000000000001,"15min":0.60399999999999998},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.24099999999999999},"max":{"1min":0.60999999999999999,"5min":8.5640000000000001,"15min":22.712},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.433,"15min":0.57399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.23899999999999999},"max":{"1min":0.61799999999999999,"5min":1.0229999999999999,"15min":22.359000000000002},"last":0.36599999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58099999999999996,"15min":0.67900000000000005},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.77200000000000002,"5min":0.90600000000000003,"15min":18.812000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.61099999999999999,"15min":0.67600000000000005},"min":{"1min":0.41799999999999998,"5min":0.371,"15min":0.28699999999999998},"max":{"1min":0.79500000000000004,"5min":1.002,"15min":10.016},"last":0.54500000000000004}]},{"osd":5,"last update":"Sun Feb 18 00:48:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.72099999999999997,"15min":0.68799999999999994},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.32800000000000001},"max":{"1min":0.81100000000000005,"5min":11.805,"15min":11.805},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.70799999999999996,"15min":0.71899999999999997},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.317},"max":{"1min":0.81200000000000006,"5min":8.6059999999999999,"15min":10.114000000000001},"last":0.505}]},{"osd":6,"last update":"Sun Feb 18 00:48:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.73599999999999999,"15min":0.69599999999999995},"min":{"1min":0.46400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78900000000000003,"5min":11.680999999999999,"15min":11.680999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.69799999999999995},"min":{"1min":0.47599999999999998,"5min":0.379,"15min":0.34100000000000003},"max":{"1min":0.77200000000000002,"5min":0.871,"15min":18.861000000000001},"last":0.57299999999999995}]},{"osd":7,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.73999999999999999,"15min":0.70199999999999996},"min":{"1min":0.439,"5min":0.39000000000000001,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":11.746,"15min":11.746},"last":0.628},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.67400000000000004,"15min":0.70899999999999996},"min":{"1min":0.51400000000000001,"5min":0.42599999999999999,"15min":0.42299999999999999},"max":{"1min":0.84799999999999998,"5min":1.115,"15min":10.058999999999999},"last":0.61399999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149956,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":304480,"kb_used_data":7256,"kb_used_omap":0,"kb_used_meta":297216,"kb_avail":93440672,"statfs":{"total":95995035648,"available":95683248128,"internally_reserved":0,"allocated":7430144,"data_stored":4033465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304349184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34699999999999998,"15min":0.45400000000000001},"min":{"1min":0.23100000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.495,"5min":0.59299999999999997,"15min":25.082999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.39100000000000001,"15min":0.51200000000000001},"min":{"1min":0.27100000000000002,"5min":0.21099999999999999,"15min":0.20599999999999999},"max":{"1min":0.53900000000000003,"5min":1.8080000000000001,"15min":18.911000000000001},"last":0.27000000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.378,"15min":0.45200000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.223},"max":{"1min":0.495,"5min":0.78400000000000003,"15min":6.8479999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40699999999999997,"15min":0.68700000000000006},"min":{"1min":0.247,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51500000000000001,"5min":2.8570000000000002,"15min":23.606999999999999},"last":0.32100000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:48:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.65100000000000002},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.23799999999999999},"max":{"1min":0.53500000000000003,"5min":2.899,"15min":23.405000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.40999999999999998,"15min":0.45200000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.54600000000000004,"5min":1.6279999999999999,"15min":5.3479999999999999},"last":0.30399999999999999}]},{"osd":4,"last update":"Sun Feb 18 00:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":0.68799999999999994},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.33600000000000002},"max":{"1min":0.81899999999999995,"5min":2.9529999999999998,"15min":10.131},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.61699999999999999},"min":{"1min":0.39900000000000002,"5min":0.35299999999999998,"15min":0.32700000000000001},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":5.3049999999999997},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Feb 18 00:48:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.32800000000000001},"max":{"1min":0.70899999999999996,"5min":0.92700000000000005,"15min":7.4249999999999998},"last":0.499},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60399999999999998,"15min":0.67500000000000004},"min":{"1min":0.43099999999999999,"5min":0.34200000000000003,"15min":0.32400000000000001},"max":{"1min":0.75800000000000001,"5min":0.98999999999999999,"15min":6.7750000000000004},"last":0.60099999999999998}]},{"osd":6,"last update":"Sun Feb 18 00:48:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63100000000000001,"15min":0.75700000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":2.9830000000000001,"15min":10.189},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.67000000000000004},"min":{"1min":0.48399999999999999,"5min":0.40999999999999998,"15min":0.39000000000000001},"max":{"1min":0.81399999999999995,"5min":0.97799999999999998,"15min":6.4279999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Sun Feb 18 00:48:44 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.95399999999999996,"15min":6.8220000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.65300000000000002,"15min":0.72899999999999998},"min":{"1min":0.48399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88600000000000001,"5min":2.9300000000000002,"15min":10.164999999999999},"last":0.63200000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986431,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312652,"kb_used_data":7236,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93432500,"statfs":{"total":95995035648,"available":95674880000,"internally_reserved":0,"allocated":7409664,"data_stored":4033184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54600000000000004,"15min":0.84399999999999997},"min":{"1min":0.38400000000000001,"5min":0.33300000000000002,"15min":0.247},"max":{"1min":0.76500000000000001,"5min":2.2959999999999998,"15min":77.581999999999994},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.82699999999999996},"min":{"1min":0.38200000000000001,"5min":0.30399999999999999,"15min":0.26100000000000001},"max":{"1min":0.66900000000000004,"5min":0.875,"15min":78.063999999999993},"last":0.39300000000000002}]},{"osd":1,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56000000000000005,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.27300000000000002},"max":{"1min":0.76200000000000001,"5min":1.3879999999999999,"15min":9.3160000000000007},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54100000000000004,"15min":0.874},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.29699999999999999},"max":{"1min":0.82099999999999995,"5min":0.97799999999999998,"15min":81.242000000000004},"last":0.51500000000000001}]},{"osd":2,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.61399999999999999},"min":{"1min":0.38100000000000001,"5min":0.28599999999999998,"15min":0.27900000000000003},"max":{"1min":0.79800000000000004,"5min":0.94799999999999995,"15min":11.691000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.66300000000000003},"min":{"1min":0.45300000000000001,"5min":0.32000000000000001,"15min":0.27200000000000002},"max":{"1min":0.64300000000000002,"5min":0.90700000000000003,"15min":17.77},"last":0.56999999999999995}]},{"osd":3,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.33700000000000002,"15min":0.29599999999999999},"max":{"1min":0.69299999999999995,"5min":0.97999999999999998,"15min":7.218},"last":0.502},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.72999999999999998},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.309},"max":{"1min":0.85799999999999998,"5min":1.3600000000000001,"15min":33.530999999999999},"last":0.49199999999999999}]},{"osd":5,"last update":"Sun Feb 18 00:48:49 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48499999999999999,"15min":0.45700000000000002},"min":{"1min":0.35699999999999998,"5min":0.28899999999999998,"15min":0.23799999999999999},"max":{"1min":0.68899999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.47199999999999998,"15min":0.45700000000000002},"min":{"1min":0.34100000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.70899999999999996,"5min":1.45,"15min":1.45},"last":0.33500000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:48:41 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46500000000000002,"15min":0.45400000000000001},"min":{"1min":0.35599999999999998,"5min":0.245,"15min":0.221},"max":{"1min":0.68500000000000005,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.503},"min":{"1min":0.376,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.68999999999999995,"5min":3.2970000000000002,"15min":3.2970000000000002},"last":0.434}]},{"osd":7,"last update":"Sun Feb 18 00:48:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53500000000000003,"15min":0.51700000000000002},"min":{"1min":0.40500000000000003,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":0.72499999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.51700000000000002,"15min":0.50800000000000001},"min":{"1min":0.374,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70199999999999996,"5min":0.70599999999999996,"15min":0.81100000000000005},"last":0.53700000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724794,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312632,"kb_used_data":7216,"kb_used_omap":0,"kb_used_meta":305408,"kb_avail":93432520,"statfs":{"total":95995035648,"available":95674900480,"internally_reserved":0,"allocated":7389184,"data_stored":4032895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312737792},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.50700000000000001,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.27300000000000002,"15min":0.217},"max":{"1min":0.90200000000000002,"5min":1.3380000000000001,"15min":6.3150000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.50900000000000001,"15min":0.60699999999999998},"min":{"1min":0.34599999999999997,"5min":0.28799999999999998,"15min":0.26000000000000001},"max":{"1min":0.90900000000000003,"5min":1.123,"15min":12.744},"last":0.94499999999999995}]},{"osd":1,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56200000000000006,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.30099999999999999,"15min":0.27400000000000002},"max":{"1min":1.0029999999999999,"5min":1.2010000000000001,"15min":13.25},"last":1.054},{"interface":"front","average":{"1min":0.621,"5min":0.54700000000000004,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.311,"15min":0.30099999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":13.262},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55900000000000005,"15min":0.58899999999999997},"min":{"1min":0.36899999999999999,"5min":0.30399999999999999,"15min":0.251},"max":{"1min":0.94899999999999995,"5min":1.4510000000000001,"15min":6.1680000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.377,"15min":0.308},"max":{"1min":0.88800000000000001,"5min":1.4370000000000001,"15min":6.1879999999999997},"last":1.073}]},{"osd":3,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52300000000000002,"15min":0.61899999999999999},"min":{"1min":0.47899999999999998,"5min":0.29999999999999999,"15min":0.28499999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":7.641},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.60399999999999998},"min":{"1min":0.46700000000000003,"5min":0.34499999999999997,"15min":0.27600000000000002},"max":{"1min":0.83699999999999997,"5min":1.22,"15min":8.8650000000000002},"last":1.032}]},{"osd":4,"last update":"Sun Feb 18 00:47:52 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.497},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.23599999999999999},"max":{"1min":0.77300000000000002,"5min":0.79400000000000004,"15min":11.818},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48199999999999998,"15min":0.49399999999999999},"min":{"1min":0.35099999999999998,"5min":0.29099999999999998,"15min":0.23300000000000001},"max":{"1min":0.68899999999999995,"5min":0.80700000000000005,"15min":4.274},"last":0.77700000000000002}]},{"osd":6,"last update":"Sun Feb 18 00:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47499999999999998,"15min":0.56799999999999995},"min":{"1min":0.34499999999999997,"5min":0.30499999999999999,"15min":0.26300000000000001},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":18.605},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.51600000000000001},"min":{"1min":0.35599999999999998,"5min":0.27200000000000002,"15min":0.26500000000000001},"max":{"1min":0.75600000000000001,"5min":1.0629999999999999,"15min":4.2089999999999996},"last":0.747}]},{"osd":7,"last update":"Sun Feb 18 00:48:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52200000000000002,"15min":0.51100000000000001},"min":{"1min":0.35399999999999998,"5min":0.311,"15min":0.26600000000000001},"max":{"1min":0.73799999999999999,"5min":1.2370000000000001,"15min":2.0739999999999998},"last":1.093},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52300000000000002,"15min":0.52400000000000002},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.28299999999999997},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":4.649},"last":0.53700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":5,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T00:48:54.186 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-02-18T00:48:54.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-02-18T00:48:54.471 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T00:48:54.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:52.988372+0000 osd.2 (osd.2) 409 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:48:54.501 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:52.990171+0000 osd.2 (osd.2) 410 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:48:54.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:53.383411+0000 mgr.y (mgr.24454) 788 : audit [DBG] from='client.32328 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:48:54.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:53.637994+0000 osd.1 (osd.1) 467 : cluster [DBG] 2.d scrub starts 2024-02-18T00:48:54.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:53.639178+0000 osd.1 (osd.1) 468 : cluster [DBG] 2.d scrub ok 2024-02-18T00:48:54.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:52.988372+0000 osd.2 (osd.2) 409 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:48:54.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:52.990171+0000 osd.2 (osd.2) 410 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:48:54.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:53.383411+0000 mgr.y (mgr.24454) 788 : audit [DBG] from='client.32328 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:48:54.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:53.637994+0000 osd.1 (osd.1) 467 : cluster [DBG] 2.d scrub starts 2024-02-18T00:48:54.503 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:54 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:53.639178+0000 osd.1 (osd.1) 468 : cluster [DBG] 2.d scrub ok 2024-02-18T00:48:54.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:52.988372+0000 osd.2 (osd.2) 409 : cluster [DBG] 4.1c scrub starts 2024-02-18T00:48:54.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:52.990171+0000 osd.2 (osd.2) 410 : cluster [DBG] 4.1c scrub ok 2024-02-18T00:48:54.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:53.383411+0000 mgr.y (mgr.24454) 788 : audit [DBG] from='client.32328 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T00:48:54.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:53.637994+0000 osd.1 (osd.1) 467 : cluster [DBG] 2.d scrub starts 2024-02-18T00:48:54.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:53.639178+0000 osd.1 (osd.1) 468 : cluster [DBG] 2.d scrub ok 2024-02-18T00:48:55.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:53.429468+0000 osd.3 (osd.3) 523 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:55.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:53.429693+0000 osd.3 (osd.3) 524 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:54.082364+0000 mgr.y (mgr.24454) 789 : cluster [DBG] pgmap v1370: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:54.214459+0000 osd.0 (osd.0) 438 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:54.215582+0000 osd.0 (osd.0) 439 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:54.353219+0000 mon.a (mon.0) 3872 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:53.429468+0000 osd.3 (osd.3) 523 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:53.429693+0000 osd.3 (osd.3) 524 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:54.082364+0000 mgr.y (mgr.24454) 789 : cluster [DBG] pgmap v1370: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:54.214459+0000 osd.0 (osd.0) 438 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:54.215582+0000 osd.0 (osd.0) 439 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:48:55.502 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:55 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:54.353219+0000 mon.a (mon.0) 3872 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-02-18T00:48:55.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:53.429468+0000 osd.3 (osd.3) 523 : cluster [DBG] purged_snaps scrub starts 2024-02-18T00:48:55.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:53.429693+0000 osd.3 (osd.3) 524 : cluster [DBG] purged_snaps scrub ok 2024-02-18T00:48:55.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:54.082364+0000 mgr.y (mgr.24454) 789 : cluster [DBG] pgmap v1370: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T00:48:55.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:54.214459+0000 osd.0 (osd.0) 438 : cluster [DBG] 6.9 scrub starts 2024-02-18T00:48:55.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:54.215582+0000 osd.0 (osd.0) 439 : cluster [DBG] 6.9 scrub ok 2024-02-18T00:48:55.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:54.353219+0000 mon.a (mon.0) 3872 : cluster [DBG] osdmap e856: 8 total, 8 up, 8 in 2024-02-18T00:48:56.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 -- 172.21.15.59:0/2743989391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3280f9670 msgr2=0x7fe328101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/2743989391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3280f9670 0x7fe328101d80 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 rx=0x7fe3180047b0 tx=0x7fe318009cb0).stop 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 -- 172.21.15.59:0/2743989391 shutdown_connections 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/2743989391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe328102270 0x7fe328104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/2743989391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 0x7fe3280f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/2743989391 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3280f9670 0x7fe328101d80 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 -- 172.21.15.59:0/2743989391 >> 172.21.15.59:0/2743989391 conn(0x7fe3280f2820 msgr2=0x7fe3280f4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 -- 172.21.15.59:0/2743989391 shutdown_connections 2024-02-18T00:48:56.709 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.707+0000 7fe32f9b1700 1 -- 172.21.15.59:0/2743989391 wait complete. 2024-02-18T00:48:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 librados: starting msgr at 2024-02-18T00:48:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 librados: starting objecter 2024-02-18T00:48:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 Processor -- start 2024-02-18T00:48:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 -- start start 2024-02-18T00:48:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 librados: setting wanted keys 2024-02-18T00:48:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 librados: calling monclient init 2024-02-18T00:48:56.711 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 0x7fe32810e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3280f9670 0x7fe32810e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe328102270 0x7fe328108380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe328108870 con 0x7fe3280f8d10 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281089e0 con 0x7fe3280f9670 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32f9b1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe328108ba0 con 0x7fe328102270 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32d74d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 0x7fe32810e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32d74d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 0x7fe32810e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55100/0 (socket says 172.21.15.59:55100) 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32d74d700 1 -- 172.21.15.59:0/1282199680 learned_addr learned my addr 172.21.15.59:0/1282199680 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32df4e700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe328102270 0x7fe328108380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.708+0000 7fe32cf4c700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3280f9670 0x7fe32810e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32d74d700 1 -- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe328102270 msgr2=0x7fe328108380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32d74d700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe328102270 0x7fe328108380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.712 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32d74d700 1 -- 172.21.15.59:0/1282199680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3280f9670 msgr2=0x7fe32810e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32d74d700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3280f9670 0x7fe32810e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32d74d700 1 -- 172.21.15.59:0/1282199680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe324005040 con 0x7fe3280f8d10 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32d74d700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 0x7fe32810e1e0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 rx=0x7fe324009400 tx=0x7fe324009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe31e7fc700 10 client.?.objecter ms_handle_connect 0x7fe3280f8d10 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe31e7fc700 10 client.?.objecter resend_mon_ops 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe31e7fc700 1 -- 172.21.15.59:0/1282199680 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324007a40 con 0x7fe3280f8d10 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe318005040 con 0x7fe3280f8d10 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe31e7fc700 1 -- 172.21.15.59:0/1282199680 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe32400f040 con 0x7fe3280f8d10 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe31e7fc700 1 -- 172.21.15.59:0/1282199680 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324014a70 con 0x7fe3280f8d10 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32f9b1700 10 client.32334.objecter _maybe_request_map subscribing (onetime) to next osd map 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3281091e0 con 0x7fe3280f8d10 2024-02-18T00:48:56.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.709+0000 7fe32f9b1700 1 librados: init done 2024-02-18T00:48:56.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.710+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32805fa60 con 0x7fe3280f8d10 2024-02-18T00:48:56.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 1 -- 172.21.15.59:0/1282199680 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 20) v1 ==== 77086+0+0 (secure 0 0 0) 0x7fe324010530 con 0x7fe3280f8d10 2024-02-18T00:48:56.714 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 1 --2- 172.21.15.59:0/1282199680 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe314062c90 0x7fe314065150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 1 -- 172.21.15.59:0/1282199680 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(856..856 src has 418..856) v4 ==== 7613+0+0 (secure 0 0 0) 0x7fe32407cbf0 con 0x7fe3280f8d10 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 10 client.32334.objecter ms_dispatch 0x7fe328001550 osd_map(856..856 src has 418..856) v4 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 3 client.32334.objecter handle_osd_map got epochs [856,856] > 0 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 3 client.32334.objecter handle_osd_map decoding full epoch 856 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe32cf4c700 1 --2- 172.21.15.59:0/1282199680 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe314062c90 0x7fe314065150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 20 client.32334.objecter dump_active .. 0 homeless 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe31e7fc700 10 client.32334.objecter ms_handle_connect 0x7fe314062c90 2024-02-18T00:48:56.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.711+0000 7fe32cf4c700 1 --2- 172.21.15.59:0/1282199680 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe314062c90 0x7fe314065150 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 rx=0x7fe318005010 tx=0x7fe318009cb0).ready entity=mgr.24454 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T00:48:56.716 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.715+0000 7fe31e7fc700 1 -- 172.21.15.59:0/1282199680 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe324049f60 con 0x7fe3280f8d10 2024-02-18T00:48:56.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.888+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) v1 -- 0x7fe32805ee80 con 0x7fe3280f8d10 2024-02-18T00:48:56.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.893+0000 7fe31e7fc700 1 -- 172.21.15.59:0/1282199680 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe32805ee80 con 0x7fe3280f8d10 2024-02-18T00:48:56.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.894+0000 7fe32f9b1700 10 librados: watch_flush enter 2024-02-18T00:48:56.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 10 librados: watch_flush exit 2024-02-18T00:48:56.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 20 client.32334.objecter shutdown clearing up homeless session... 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 10 client.32334.objecter successfully canceled tick 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe314062c90 msgr2=0x7fe314065150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/1282199680 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe314062c90 0x7fe314065150 secure :-1 s=READY pgs=3285 cs=0 l=1 rev1=1 rx=0x7fe318005010 tx=0x7fe318009cb0).stop 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 msgr2=0x7fe32810e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 0x7fe32810e1e0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 rx=0x7fe324009400 tx=0x7fe324009cb0).stop 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 shutdown_connections 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe328102270 0x7fe328108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3280f8d10 0x7fe32810e1e0 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/1282199680 >> v2:172.21.15.59:6800/3721891797 conn(0x7fe314062c90 0x7fe314065150 unknown :-1 s=CLOSED pgs=3285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 --2- 172.21.15.59:0/1282199680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3280f9670 0x7fe32810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 >> 172.21.15.59:0/1282199680 conn(0x7fe3280f2820 msgr2=0x7fe3280f3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 shutdown_connections 2024-02-18T00:48:56.897 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 -- 172.21.15.59:0/1282199680 wait complete. 2024-02-18T00:48:56.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-02-18T00:48:56.895+0000 7fe32f9b1700 1 librados: shutdown 2024-02-18T00:48:57.251 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:56] ENGINE Bus STOPPING 2024-02-18T00:48:57.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:48:57.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:56] ENGINE Bus STOPPED 2024-02-18T00:48:57.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:56 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:56] ENGINE Bus STARTING 2024-02-18T00:48:57.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:57] ENGINE Serving on http://:::9283 2024-02-18T00:48:57.252 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:57] ENGINE Bus STARTED 2024-02-18T00:48:57.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:55.383959+0000 osd.3 (osd.3) 525 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:48:57.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:55.385110+0000 osd.3 (osd.3) 526 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:48:57.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:56.083139+0000 mgr.y (mgr.24454) 790 : cluster [DBG] pgmap v1372: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:57.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:56.090115+0000 osd.5 (osd.5) 458 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:48:57.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: cluster 2024-02-18T00:48:56.093177+0000 osd.5 (osd.5) 459 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:48:57.635 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:56.890041+0000 mon.a (mon.0) 3873 : audit [INF] from='client.? 172.21.15.59:0/1282199680' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:56.894051+0000 mon.a (mon.0) 3874 : audit [INF] from='client.? 172.21.15.59:0/1282199680' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:56.924779+0000 mon.a (mon.0) 3875 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:56.926159+0000 mon.a (mon.0) 3876 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:56.926715+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: audit 2024-02-18T00:48:56.931298+0000 mon.a (mon.0) 3878 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:55.383959+0000 osd.3 (osd.3) 525 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:55.385110+0000 osd.3 (osd.3) 526 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:56.083139+0000 mgr.y (mgr.24454) 790 : cluster [DBG] pgmap v1372: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:57.636 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:56.090115+0000 osd.5 (osd.5) 458 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:48:57.637 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: cluster 2024-02-18T00:48:56.093177+0000 osd.5 (osd.5) 459 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:48:57.637 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:56.890041+0000 mon.a (mon.0) 3873 : audit [INF] from='client.? 172.21.15.59:0/1282199680' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-18T00:48:57.637 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:56.894051+0000 mon.a (mon.0) 3874 : audit [INF] from='client.? 172.21.15.59:0/1282199680' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-18T00:48:57.637 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:56.924779+0000 mon.a (mon.0) 3875 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:48:57.637 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:56.926159+0000 mon.a (mon.0) 3876 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:48:57.637 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:56.926715+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:48:57.637 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:48:57 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: audit 2024-02-18T00:48:56.931298+0000 mon.a (mon.0) 3878 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:48:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:55.383959+0000 osd.3 (osd.3) 525 : cluster [DBG] 6.11 scrub starts 2024-02-18T00:48:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:55.385110+0000 osd.3 (osd.3) 526 : cluster [DBG] 6.11 scrub ok 2024-02-18T00:48:57.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:56.083139+0000 mgr.y (mgr.24454) 790 : cluster [DBG] pgmap v1372: 174 pgs: 174 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:56.090115+0000 osd.5 (osd.5) 458 : cluster [DBG] 6.7 scrub starts 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: cluster 2024-02-18T00:48:56.093177+0000 osd.5 (osd.5) 459 : cluster [DBG] 6.7 scrub ok 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:56.890041+0000 mon.a (mon.0) 3873 : audit [INF] from='client.? 172.21.15.59:0/1282199680' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:56.894051+0000 mon.a (mon.0) 3874 : audit [INF] from='client.? 172.21.15.59:0/1282199680' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:56.924779+0000 mon.a (mon.0) 3875 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:56.926159+0000 mon.a (mon.0) 3876 : audit [DBG] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:56.926715+0000 mon.a (mon.0) 3877 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T00:48:57.754 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:48:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: audit 2024-02-18T00:48:56.931298+0000 mon.a (mon.0) 3878 : audit [INF] from='mgr.24454 172.21.15.59:0/2539449959' entity='mgr.y' 2024-02-18T00:48:58.107 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-18T00:48:58.117 INFO:tasks.cephadm:Teardown begin 2024-02-18T00:48:58.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T00:48:58.153 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T00:48:58.191 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-18T00:48:58.191 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-18T00:48:58.214 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-18T00:48:58.250 INFO:tasks.cephadm:Stopping all daemons... 2024-02-18T00:48:58.251 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-18T00:48:58.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.a 2024-02-18T00:48:58.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:58 smithi059 systemd[1]: Stopping Ceph mon.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:48:58.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:58 smithi059 bash[250508]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon.a" found: no such container 2024-02-18T00:48:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:48:58.596+0000 7f615826b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T00:48:59.002 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a[38271]: debug 2024-02-18T00:48:58.596+0000 7f615826b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-18T00:48:59.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:58] ENGINE Bus STOPPING 2024-02-18T00:48:59.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:48:59.002 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:58] ENGINE Bus STOPPED 2024-02-18T00:48:59.003 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:58] ENGINE Bus STARTING 2024-02-18T00:48:59.003 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:58] ENGINE Serving on http://:::9283 2024-02-18T00:48:59.003 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:48:58 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:48:58] ENGINE Bus STARTED 2024-02-18T00:48:59.501 INFO:journalctl@ceph.mon.a.smithi059.stdout:Feb 18 00:48:59 smithi059 bash[250547]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-a 2024-02-18T00:48:59.891 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.a.service' 2024-02-18T00:48:59.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:48:59.932 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-18T00:48:59.932 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-18T00:48:59.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.c 2024-02-18T00:49:00.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:49:00 smithi059 systemd[1]: Stopping Ceph mon.c for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:00.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:49:00 smithi059 bash[250679]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon.c" found: no such container 2024-02-18T00:49:00.653 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:49:00] ENGINE Bus STOPPING 2024-02-18T00:49:00.653 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:49:00] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T00:49:00.653 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:49:00] ENGINE Bus STOPPED 2024-02-18T00:49:00.654 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:49:00] ENGINE Bus STARTING 2024-02-18T00:49:00.654 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:49:00] ENGINE Serving on http://:::9283 2024-02-18T00:49:00.654 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: [18/Feb/2024:00:49:00] ENGINE Bus STARTED 2024-02-18T00:49:00.654 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:49:00.324+0000 7f94682a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T00:49:00.654 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:49:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c[45117]: debug 2024-02-18T00:49:00.324+0000 7f94682a7700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-18T00:49:01.251 INFO:journalctl@ceph.mon.c.smithi059.stdout:Feb 18 00:49:00 smithi059 bash[250718]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-c 2024-02-18T00:49:01.429 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.c.service' 2024-02-18T00:49:01.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:01.467 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-18T00:49:01.467 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-18T00:49:01.467 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.b 2024-02-18T00:49:01.498 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:49:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x[40982]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:49:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T00:49:01.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:49:01 smithi137 systemd[1]: Stopping Ceph mon.b for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:01.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:49:01 smithi137 bash[117691]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon.b" found: no such container 2024-02-18T00:49:02.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:49:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug there is no tcmu-runner data available 2024-02-18T00:49:02.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:49:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:49:01.776+0000 7f702cac1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T00:49:02.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:49:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b[39511]: debug 2024-02-18T00:49:01.776+0000 7f702cac1700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-18T00:49:02.501 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:02 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: ::ffff:172.21.15.137 - - [18/Feb/2024:00:49:02] "GET /metrics HTTP/1.1" 200 187526 "" "Prometheus/2.33.4" 2024-02-18T00:49:02.753 INFO:journalctl@ceph.mon.b.smithi137.stdout:Feb 18 00:49:02 smithi137 bash[117731]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mon-b 2024-02-18T00:49:02.908 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mon.b.service' 2024-02-18T00:49:03.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:03.003 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-18T00:49:03.003 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-18T00:49:03.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.y 2024-02-18T00:49:03.372 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:03 smithi059 systemd[1]: Stopping Ceph mgr.y for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:03.373 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:03 smithi059 bash[250848]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr.y" found: no such container 2024-02-18T00:49:03.750 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:49:03.369+0000 7f11a813e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:03.750 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y[38549]: debug 2024-02-18T00:49:03.369+0000 7f11a813e700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-18T00:49:04.001 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Feb 18 00:49:03 smithi059 bash[250889]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-y 2024-02-18T00:49:04.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.y.service' 2024-02-18T00:49:04.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:04.391 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-18T00:49:04.392 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-18T00:49:04.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.x 2024-02-18T00:49:04.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:49:04 smithi137 systemd[1]: Stopping Ceph mgr.x for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:04.754 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:49:04 smithi137 bash[117853]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr.x" found: no such container 2024-02-18T00:49:05.403 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Feb 18 00:49:05 smithi137 bash[117894]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-mgr-x 2024-02-18T00:49:05.513 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@mgr.x.service' 2024-02-18T00:49:05.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:05.549 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-18T00:49:05.549 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-18T00:49:05.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.0 2024-02-18T00:49:05.877 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:05 smithi059 systemd[1]: Stopping Ceph osd.0 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:05.877 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:05 smithi059 bash[251008]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-18T00:49:06.251 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-0[48611]: debug 2024-02-18T00:49:05.875+0000 7f26b67b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:06.251 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-0[48611]: debug 2024-02-18T00:49:05.875+0000 7f26b67b6700 -1 osd.0 856 *** Got signal Terminated *** 2024-02-18T00:49:06.251 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:05 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-0[48611]: debug 2024-02-18T00:49:05.875+0000 7f26b67b6700 -1 osd.0 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:49:11.751 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:11 smithi059 bash[251046]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-0 2024-02-18T00:49:13.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.0.service' 2024-02-18T00:49:13.807 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:13 smithi059 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.0.service: Succeeded. 2024-02-18T00:49:13.807 INFO:journalctl@ceph.osd.0.smithi059.stdout:Feb 18 00:49:13 smithi059 systemd[1]: Stopped Ceph osd.0 for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:49:13.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:13.823 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-18T00:49:13.823 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-18T00:49:13.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.1 2024-02-18T00:49:14.164 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:49:13 smithi059 systemd[1]: Stopping Ceph osd.1 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:14.165 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:49:13 smithi059 bash[251287]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-18T00:49:14.501 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:49:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-1[53219]: debug 2024-02-18T00:49:14.162+0000 7f146fa2c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:14.501 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:49:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-1[53219]: debug 2024-02-18T00:49:14.162+0000 7f146fa2c700 -1 osd.1 856 *** Got signal Terminated *** 2024-02-18T00:49:14.501 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:49:14 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-1[53219]: debug 2024-02-18T00:49:14.162+0000 7f146fa2c700 -1 osd.1 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:49:20.002 INFO:journalctl@ceph.osd.1.smithi059.stdout:Feb 18 00:49:19 smithi059 bash[251325]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-1 2024-02-18T00:49:20.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:20.145Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:20.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:20.145Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:22.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.1.service' 2024-02-18T00:49:22.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:22.096 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-18T00:49:22.096 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-18T00:49:22.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.2 2024-02-18T00:49:22.378 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:49:22 smithi059 systemd[1]: Stopping Ceph osd.2 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:22.378 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:49:22 smithi059 bash[251562]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-18T00:49:22.751 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:49:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2[57675]: debug 2024-02-18T00:49:22.376+0000 7f0e707f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:22.751 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:49:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2[57675]: debug 2024-02-18T00:49:22.376+0000 7f0e707f8700 -1 osd.2 856 *** Got signal Terminated *** 2024-02-18T00:49:22.751 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:49:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2[57675]: debug 2024-02-18T00:49:22.376+0000 7f0e707f8700 -1 osd.2 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:49:24.001 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:23.620Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:24.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:23 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:23.620Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:28.368 INFO:journalctl@ceph.osd.2.smithi059.stdout:Feb 18 00:49:28 smithi059 bash[251603]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-2 2024-02-18T00:49:30.437 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:49:30.145Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:30.437 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:30.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:30.438 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:30.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:30.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.2.service' 2024-02-18T00:49:30.492 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:30.492 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-18T00:49:30.493 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-18T00:49:30.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.3 2024-02-18T00:49:30.752 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:30 smithi059 systemd[1]: Stopping Ceph osd.3 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:30.752 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:30 smithi059 bash[251842]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-18T00:49:31.251 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:49:30.866+0000 7fe34cade700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:31.251 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:49:30.866+0000 7fe34cade700 -1 osd.3 856 *** Got signal Terminated *** 2024-02-18T00:49:31.252 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:30 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:49:30.866+0000 7fe34cade700 -1 osd.3 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:49:32.117 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:31 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:31.808+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:32.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:32.116+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:33.144 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:32.827+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:33.144 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:32.837+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:33.145 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:32 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:32.833+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:33.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:33.143+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:33.621 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:49:33.317+0000 7fe349a5b700 -1 osd.3 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.279705+0000 front 2024-02-18T00:49:07.279771+0000 (oldest deadline 2024-02-18T00:49:33.179922+0000) 2024-02-18T00:49:34.001 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:49:33.620Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=5 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:34.001 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:33.621Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:34.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:33 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:33.621Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:34.097 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:33.806+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:34.097 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:33.845+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:34.097 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:33 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:33.841+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:34.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:34.096+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:34.751 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:34 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:49:34.342+0000 7fe349a5b700 -1 osd.3 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.279705+0000 front 2024-02-18T00:49:07.279771+0000 (oldest deadline 2024-02-18T00:49:33.179922+0000) 2024-02-18T00:49:35.122 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:34.812+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:35.122 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:34.865+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:35.122 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:34 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:34.883+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:35.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:35.121+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:35.751 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:35 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3[62160]: debug 2024-02-18T00:49:35.305+0000 7fe349a5b700 -1 osd.3 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.279705+0000 front 2024-02-18T00:49:07.279771+0000 (oldest deadline 2024-02-18T00:49:33.179922+0000) 2024-02-18T00:49:36.159 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:35.788+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:36.159 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:35.904+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:36.159 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:35 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:35.857+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:36.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:36.158+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:36.751 INFO:journalctl@ceph.osd.3.smithi059.stdout:Feb 18 00:49:36 smithi059 bash[251882]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-3 2024-02-18T00:49:37.121 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:36.820+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:37.122 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:36.883+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:37.122 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:36 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:36.893+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:37.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:37.121+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:38.117 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:37.864+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:38.118 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:37.862+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:38.118 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:37 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:37.848+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:38.462 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.3.service' 2024-02-18T00:49:38.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:38.117+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:38.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:38.509 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-18T00:49:38.509 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-18T00:49:38.510 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.4 2024-02-18T00:49:38.806 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:38 smithi137 systemd[1]: Stopping Ceph osd.4 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:38.806 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:38 smithi137 bash[118028]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-18T00:49:39.136 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:38.815+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:39.137 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:38.817+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:39.137 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:38.805+0000 7fc4f7f96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:39.137 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:38.805+0000 7fc4f7f96700 -1 osd.4 856 *** Got signal Terminated *** 2024-02-18T00:49:39.137 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:38.805+0000 7fc4f7f96700 -1 osd.4 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:49:39.137 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:38 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:38.868+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:39.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:39.136+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:40.137 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:39.849+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:40.137 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:39.858+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:40.138 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:39.797+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:40.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:49:40.146Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:40.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:40.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:40.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:40 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:40.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:40.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:40.136+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:41.138 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:40.885+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:41.139 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:40.882+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:41.139 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:40.882+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:41.139 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:40 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:40.794+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:41.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:41.137+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:42.152 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:41.926+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:42.152 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:41.926+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:17.310700+0000 front 2024-02-18T00:49:17.310567+0000 (oldest deadline 2024-02-18T00:49:41.410480+0000) 2024-02-18T00:49:42.152 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:41.930+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:42.152 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:41.930+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:42.153 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:41 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:41.842+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:42.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:42.149+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:43.177 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:42.948+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:06.109507+0000 front 2024-02-18T00:49:06.109443+0000 (oldest deadline 2024-02-18T00:49:32.009515+0000) 2024-02-18T00:49:43.177 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4[43735]: debug 2024-02-18T00:49:42.948+0000 7fc4f270e700 -1 osd.4 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:17.310700+0000 front 2024-02-18T00:49:17.310567+0000 (oldest deadline 2024-02-18T00:49:41.410480+0000) 2024-02-18T00:49:43.178 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:42.895+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:43.178 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:42.895+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:43.178 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:42 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:42.855+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:43.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:43.175+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:43.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:43.175+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:44.001 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:49:43.621Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=5 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:43.621Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:44.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:43 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:43.622Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:44.149 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:43.921+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:44.149 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:43.921+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:44.150 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:43 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:43.848+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:44.504 INFO:journalctl@ceph.osd.4.smithi137.stdout:Feb 18 00:49:44 smithi137 bash[118068]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-4 2024-02-18T00:49:44.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:44.153+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:44.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:44.153+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:45.204 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:44.879+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:45.204 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:44.879+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:45.205 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:44.863+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:45.205 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:44 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:44.863+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:45.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:45.202+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:45.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:45.202+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:45.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.4.service' 2024-02-18T00:49:45.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:45.873 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-18T00:49:45.874 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-18T00:49:45.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.5 2024-02-18T00:49:46.161 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:45.818+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:46.161 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:45.818+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:46.162 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:45.919+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:46.162 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:45 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:45.919+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:46.162 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:45 smithi137 systemd[1]: Stopping Ceph osd.5 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:46.162 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:45 smithi137 bash[118307]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-18T00:49:46.503 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:46.195+0000 7fb79628c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:46.504 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:46.195+0000 7fb79628c700 -1 osd.5 856 *** Got signal Terminated *** 2024-02-18T00:49:46.504 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:46.195+0000 7fb79628c700 -1 osd.5 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:49:46.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:46.159+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:46.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:46.159+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:47.172 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:46.889+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:47.173 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:46.889+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:47.173 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:46.850+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:47.173 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:46 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:46.850+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:47.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:47.167+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:47.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:47.167+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:48.208 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:47.899+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:48.208 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:47.899+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:48.208 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:47.888+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:48.208 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:47 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:47.888+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:48.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:48.206+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:48.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:48.206+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:49.228 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:48.913+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:49.228 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:48.913+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:49.228 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:48.926+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:49.228 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:48 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:48.926+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:49.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:49.226+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:49.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:49.226+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:50.205 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:49.957+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:50.205 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:49.957+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:50.205 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:49.927+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:50.205 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:49 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:49.927+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:50.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:49:50.147Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:50.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:50.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:50.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:50 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:50.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:50.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:50.202+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:50.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:50.202+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:51.179 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:50.951+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.485899+0000 front 2024-02-18T00:49:10.485981+0000 (oldest deadline 2024-02-18T00:49:30.985862+0000) 2024-02-18T00:49:51.179 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:50.951+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:14.987481+0000 front 2024-02-18T00:49:14.987594+0000 (oldest deadline 2024-02-18T00:49:40.287499+0000) 2024-02-18T00:49:51.180 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5[48239]: debug 2024-02-18T00:49:50.951+0000 7fb791205700 -1 osd.5 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:26.688684+0000 front 2024-02-18T00:49:26.688629+0000 (oldest deadline 2024-02-18T00:49:50.188557+0000) 2024-02-18T00:49:51.180 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:50.892+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:51.180 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:50 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:50.892+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:51.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:51.178+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:51.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:51.178+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:51.852 INFO:journalctl@ceph.osd.5.smithi137.stdout:Feb 18 00:49:51 smithi137 bash[118346]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-5 2024-02-18T00:49:52.205 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:51.850+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:52.205 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:51.850+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:52.205 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:51 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:51.850+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.365362+0000 front 2024-02-18T00:49:25.365391+0000 (oldest deadline 2024-02-18T00:49:51.265300+0000) 2024-02-18T00:49:52.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:52.203+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:52.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:52.203+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:52.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:52.203+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:49:53.075 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:52.829+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:53.075 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:52.829+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:53.075 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:52 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:52.829+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.365362+0000 front 2024-02-18T00:49:25.365391+0000 (oldest deadline 2024-02-18T00:49:51.265300+0000) 2024-02-18T00:49:53.378 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:53.254+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:53.378 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:53.254+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:53.378 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:53.254+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:49:53.385 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.5.service' 2024-02-18T00:49:53.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:49:53.426 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-18T00:49:53.426 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-18T00:49:53.427 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.6 2024-02-18T00:49:53.727 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 systemd[1]: Stopping Ceph osd.6 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:49:53.727 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 bash[118576]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.6" found: no such container 2024-02-18T00:49:54.001 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:49:53.622Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=5 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:54.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:53.622Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:49:54.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:49:53 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:49:53.623Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:49:54.003 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:53.724+0000 7fd7a0593700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:49:54.003 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:53.724+0000 7fd7a0593700 -1 osd.6 856 *** Got signal Terminated *** 2024-02-18T00:49:54.003 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:53.724+0000 7fd7a0593700 -1 osd.6 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:49:54.003 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:53.878+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:54.003 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:53.878+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:54.004 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:53 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:53.878+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.365362+0000 front 2024-02-18T00:49:25.365391+0000 (oldest deadline 2024-02-18T00:49:51.265300+0000) 2024-02-18T00:49:54.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:54.221+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:54.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:54.221+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:54.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:54.221+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:49:55.194 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:54.831+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:55.194 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:54.831+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:55.194 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:54 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:54.831+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.365362+0000 front 2024-02-18T00:49:25.365391+0000 (oldest deadline 2024-02-18T00:49:51.265300+0000) 2024-02-18T00:49:55.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:55.193+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:55.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:55.193+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:55.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:55.193+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:49:56.150 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:55.846+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:56.150 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:55.846+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:56.150 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:55 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:55.846+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.365362+0000 front 2024-02-18T00:49:25.365391+0000 (oldest deadline 2024-02-18T00:49:51.265300+0000) 2024-02-18T00:49:56.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:56.148+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:56.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:56.148+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:56.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:56.148+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:49:57.105 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:56.816+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:57.106 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:56.816+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:57.106 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:56 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:56.816+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.365362+0000 front 2024-02-18T00:49:25.365391+0000 (oldest deadline 2024-02-18T00:49:51.265300+0000) 2024-02-18T00:49:57.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:57.104+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:57.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:57.104+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:57.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:57.104+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:49:58.112 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:57.809+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:07.163459+0000 front 2024-02-18T00:49:07.163603+0000 (oldest deadline 2024-02-18T00:49:32.463449+0000) 2024-02-18T00:49:58.112 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:57.809+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.364696+0000 front 2024-02-18T00:49:18.364626+0000 (oldest deadline 2024-02-18T00:49:44.264498+0000) 2024-02-18T00:49:58.112 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:57.809+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.365362+0000 front 2024-02-18T00:49:25.365391+0000 (oldest deadline 2024-02-18T00:49:51.265300+0000) 2024-02-18T00:49:58.113 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:57 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6[52815]: debug 2024-02-18T00:49:57.809+0000 7fd79d510700 -1 osd.6 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:32.966254+0000 front 2024-02-18T00:49:32.966195+0000 (oldest deadline 2024-02-18T00:49:57.066183+0000) 2024-02-18T00:49:58.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:58.111+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:58.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:58.111+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:58.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:58 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:58.111+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:49:59.503 INFO:journalctl@ceph.osd.6.smithi137.stdout:Feb 18 00:49:59 smithi137 bash[118616]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-6 2024-02-18T00:49:59.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:59.115+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:49:59.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:59.115+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:49:59.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:49:59 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:49:59.115+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:00.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:50:00.148Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:00.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:00.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:00.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:00 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:00.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:00.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:00.119+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:50:00.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:00.119+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:50:00.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:00.119+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:00.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:00 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:00.119+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:34.689834+0000 front 2024-02-18T00:49:34.689850+0000 (oldest deadline 2024-02-18T00:49:59.989832+0000) 2024-02-18T00:50:00.903 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.6.service' 2024-02-18T00:50:00.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:50:00.943 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-18T00:50:00.943 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-18T00:50:00.944 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.7 2024-02-18T00:50:01.156 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 systemd[1]: Stopping Ceph osd.7 for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:01.156 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 bash[118854]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd.7" found: no such container 2024-02-18T00:50:01.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:01.154+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:50:01.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:01.154+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:50:01.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:01.154+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:01.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:01.154+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:34.689834+0000 front 2024-02-18T00:49:34.689850+0000 (oldest deadline 2024-02-18T00:49:59.989832+0000) 2024-02-18T00:50:01.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:01.302+0000 7ff5afc4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:50:01.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:01.302+0000 7ff5afc4e700 -1 osd.7 856 *** Got signal Terminated *** 2024-02-18T00:50:01.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:01 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:01.302+0000 7ff5afc4e700 -1 osd.7 856 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T00:50:02.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:02.204+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:50:02.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:02.204+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:50:02.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:02.204+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:02.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:02 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:02.204+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:34.689834+0000 front 2024-02-18T00:49:34.689850+0000 (oldest deadline 2024-02-18T00:49:59.989832+0000) 2024-02-18T00:50:03.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:03.239+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:50:03.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:03.239+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:50:03.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:03.239+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:03.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:03 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:03.239+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:34.689834+0000 front 2024-02-18T00:49:34.689850+0000 (oldest deadline 2024-02-18T00:49:59.989832+0000) 2024-02-18T00:50:04.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:50:03.623Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:04.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:03.623Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:04.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:03 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:03.624Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:04.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:04.216+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:50:04.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:04.216+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:50:04.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:04.216+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:04.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:04 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:04.216+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:34.689834+0000 front 2024-02-18T00:49:34.689850+0000 (oldest deadline 2024-02-18T00:49:59.989832+0000) 2024-02-18T00:50:05.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:05.210+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:50:05.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:05.210+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:50:05.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:05.210+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:05.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:05 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:05.210+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:34.689834+0000 front 2024-02-18T00:49:34.689850+0000 (oldest deadline 2024-02-18T00:49:59.989832+0000) 2024-02-18T00:50:06.503 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:06.163+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6803 osd.0 since back 2024-02-18T00:49:10.186793+0000 front 2024-02-18T00:49:10.186667+0000 (oldest deadline 2024-02-18T00:49:31.886599+0000) 2024-02-18T00:50:06.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:06.163+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6807 osd.1 since back 2024-02-18T00:49:18.288067+0000 front 2024-02-18T00:49:18.288060+0000 (oldest deadline 2024-02-18T00:49:42.387968+0000) 2024-02-18T00:50:06.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:06.163+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6811 osd.2 since back 2024-02-18T00:49:25.889023+0000 front 2024-02-18T00:49:25.889075+0000 (oldest deadline 2024-02-18T00:49:51.188859+0000) 2024-02-18T00:50:06.504 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:06 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7[57263]: debug 2024-02-18T00:50:06.163+0000 7ff5acbcb700 -1 osd.7 856 heartbeat_check: no reply from 172.21.15.59:6815 osd.3 since back 2024-02-18T00:49:34.689834+0000 front 2024-02-18T00:49:34.689850+0000 (oldest deadline 2024-02-18T00:49:59.989832+0000) 2024-02-18T00:50:07.003 INFO:journalctl@ceph.osd.7.smithi137.stdout:Feb 18 00:50:06 smithi137 bash[118894]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-osd-7 2024-02-18T00:50:08.447 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@osd.7.service' 2024-02-18T00:50:08.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:50:08.491 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-18T00:50:08.491 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-18T00:50:08.491 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@rgw.foo.a 2024-02-18T00:50:08.802 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:50:08 smithi059 systemd[1]: Stopping Ceph rgw.foo.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:08.802 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:50:08 smithi059 bash[252128]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-rgw.foo.a" found: no such container 2024-02-18T00:50:09.251 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:50:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-rgw-foo-a[64804]: debug 2024-02-18T00:50:08.799+0000 7f63006bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T00:50:09.251 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:50:08 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-rgw-foo-a[64804]: debug 2024-02-18T00:50:08.799+0000 7f6317785880 -1 shutting down 2024-02-18T00:50:10.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:50:10.149Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:10.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:10.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:10.502 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:10 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:10.150Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:14.001 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:50:13.624Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=6 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:14.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:13.624Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:14.002 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:13 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:13.625Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:19.177 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:50:18 smithi059 bash[252166]: time="2024-02-18T00:50:18Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-18T00:50:19.501 INFO:journalctl@ceph.rgw.foo.a.smithi059.stdout:Feb 18 00:50:19 smithi059 bash[252166]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-rgw-foo-a 2024-02-18T00:50:19.820 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@rgw.foo.a.service' 2024-02-18T00:50:19.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:50:19.859 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-18T00:50:19.859 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-18T00:50:19.859 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@prometheus.a 2024-02-18T00:50:20.149 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:19 smithi137 systemd[1]: Stopping Ceph prometheus.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:20.149 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:19 smithi137 bash[119131]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus.a" found: no such container 2024-02-18T00:50:20.478 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.148Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.149Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.149Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-18T00:50:20.479 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a[63144]: ts=2024-02-18T00:50:20.149Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-18T00:50:20.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=error ts=2024-02-18T00:50:20.150Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:20.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:20.150Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:20.501 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:20 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:20.151Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:20.753 INFO:journalctl@ceph.prometheus.a.smithi137.stdout:Feb 18 00:50:20 smithi137 bash[119171]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-prometheus-a 2024-02-18T00:50:20.823 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@prometheus.a.service' 2024-02-18T00:50:20.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T00:50:20.859 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-18T00:50:20.859 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-18T00:50:21.854 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:21.624Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi059.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.59:8443: connect: connection refused" 2024-02-18T00:50:21.854 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:21 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=warn ts=2024-02-18T00:50:21.625Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi137.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.137:8443: connect: connection refused" 2024-02-18T00:50:22.251 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:22 smithi059 systemd[1]: Stopping Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:22.251 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:22 smithi059 bash[252587]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager.a" found: no such container 2024-02-18T00:50:22.708 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:22 smithi059 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a[71175]: level=info ts=2024-02-18T00:50:22.362Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-18T00:50:23.001 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:22 smithi059 bash[252626]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-alertmanager-a 2024-02-18T00:50:23.751 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:23 smithi059 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-18T00:50:23.751 INFO:journalctl@ceph.alertmanager.a.smithi059.stdout:Feb 18 00:50:23 smithi059 systemd[1]: Stopped Ceph alertmanager.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:50:23.752 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:50:23 smithi059 systemd[1]: Stopping Ceph node-exporter.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:23.752 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:50:23 smithi059 bash[252731]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter.a" found: no such container 2024-02-18T00:50:24.501 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:50:24 smithi059 bash[252772]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-a 2024-02-18T00:50:24.906 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:50:24 smithi059 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-18T00:50:24.906 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:50:24 smithi059 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-18T00:50:24.906 INFO:journalctl@ceph.node-exporter.a.smithi059.stdout:Feb 18 00:50:24 smithi059 systemd[1]: Stopped Ceph node-exporter.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:50:25.184 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-18T00:50:26.442 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:26 smithi137 systemd[1]: Stopping Ceph iscsi.iscsi.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:26.442 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:26 smithi137 bash[119524]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi.iscsi.a" found: no such container 2024-02-18T00:50:26.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:26 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a[60983]: debug Shutdown received 2024-02-18T00:50:36.753 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:36 smithi137 bash[119564]: time="2024-02-18T00:50:36Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-18T00:50:37.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:36 smithi137 bash[119564]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a 2024-02-18T00:50:37.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:37 smithi137 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-18T00:50:38.253 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:37 smithi137 bash[119637]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-iscsi-iscsi-a-tcmu 2024-02-18T00:50:38.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:37 smithi137 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-18T00:50:38.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi137.stdout:Feb 18 00:50:37 smithi137 systemd[1]: Stopped Ceph iscsi.iscsi.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:50:38.254 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:50:37 smithi137 systemd[1]: Stopping Ceph node-exporter.b for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:38.254 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:50:38 smithi137 bash[119720]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter.b" found: no such container 2024-02-18T00:50:38.860 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:50:38 smithi137 bash[119760]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-node-exporter-b 2024-02-18T00:50:39.142 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:50:38 smithi137 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-18T00:50:39.142 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:50:39 smithi137 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-18T00:50:39.142 INFO:journalctl@ceph.node-exporter.b.smithi137.stdout:Feb 18 00:50:39 smithi137 systemd[1]: Stopped Ceph node-exporter.b for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:50:39.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:50:39 smithi137 systemd[1]: Stopping Ceph grafana.a for d129b4f8-cdf3-11ee-95ba-87774f69a715... 2024-02-18T00:50:39.492 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:50:39 smithi137 bash[119894]: Error: no container with name or ID "ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana.a" found: no such container 2024-02-18T00:50:39.753 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:50:39 smithi137 ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a[63504]: t=2024-02-18T00:50:39+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-02-18T00:50:40.254 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:50:39 smithi137 bash[119932]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715-grafana-a 2024-02-18T00:50:40.579 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:50:40 smithi137 systemd[1]: ceph-d129b4f8-cdf3-11ee-95ba-87774f69a715@grafana.a.service: Succeeded. 2024-02-18T00:50:40.579 INFO:journalctl@ceph.grafana.a.smithi137.stdout:Feb 18 00:50:40 smithi137 systemd[1]: Stopped Ceph grafana.a for d129b4f8-cdf3-11ee-95ba-87774f69a715. 2024-02-18T00:50:40.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T00:50:40.722 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T00:50:40.752 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-18T00:50:40.754 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957/remote/ubuntu@smithi059.front.sepia.ceph.com/crash 2024-02-18T00:50:40.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/crash -- . 2024-02-18T00:50:40.809 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-18T00:50:40.809 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2024-02-18T00:50:40.810 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957/remote/ubuntu@smithi137.front.sepia.ceph.com/crash 2024-02-18T00:50:40.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/crash -- . 2024-02-18T00:50:40.844 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/d129b4f8-cdf3-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-18T00:50:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-02-18T00:50:40.846 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-18T00:50:40.846 INFO:tasks.cephadm:Removing cluster... 2024-02-18T00:50:40.846 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 --force 2024-02-18T00:50:41.754 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d129b4f8-cdf3-11ee-95ba-87774f69a715 --force 2024-02-18T00:50:42.535 INFO:tasks.cephadm:Removing cephadm ... 2024-02-18T00:50:42.536 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-18T00:50:42.558 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-18T00:50:42.577 INFO:tasks.cephadm:Teardown complete 2024-02-18T00:50:42.577 ERROR:teuthology.run_tasks:Manager failed: cephadm Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 1508, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 1376, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 238, in ceph_log if first_in_ceph_log('\[ERR\]|\[WRN\]|\[SEC\]', File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 229, in first_in_ceph_log r = ctx.ceph[cluster_name].bootstrap_remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 447, in run r = RemoteProcess(client, args, check_status=check_status, hostname=name, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 65, in __init__ self.command = quote(args) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 257, in quote return ' '.join(_quote(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 255, in _quote yield pipes.quote(a) File "/usr/lib/python3.8/shlex.py", line 325, in quote if _find_unsafe(s) is None: TypeError: expected string or bytes-like object 2024-02-18T00:50:42.579 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-18T00:50:42.591 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 1508, in task log.info('Teardown begin') File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 1376, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 238, in ceph_log if first_in_ceph_log('\[ERR\]|\[WRN\]|\[SEC\]', File "/home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks/cephadm.py", line 229, in first_in_ceph_log r = ctx.ceph[cluster_name].bootstrap_remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 447, in run r = RemoteProcess(client, args, check_status=check_status, hostname=name, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 65, in __init__ self.command = quote(args) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 257, in quote return ' '.join(_quote(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 255, in _quote yield pipes.quote(a) File "/usr/lib/python3.8/shlex.py", line 325, in quote if _find_unsafe(s) is None: TypeError: expected string or bytes-like object 2024-02-18T00:50:42.592 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-18T00:50:42.592 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-18T00:50:42.600 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-18T00:50:42.689 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-18T00:50:42.698 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-18T00:50:42.699 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T00:50:42.701 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T00:50:42.718 INFO:teuthology.orchestra.run.smithi137.stderr:bash: ntpq: command not found 2024-02-18T00:50:42.720 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2024-02-18T00:50:42.723 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T00:50:42.723 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-02-18T00:50:42.723 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +3085us[+3120us] +/- 120ms 2024-02-18T00:50:42.723 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -1248us[-1212us] +/- 59ms 2024-02-18T00:50:42.723 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -436us[ -401us] +/- 90ms 2024-02-18T00:50:42.723 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:50:42.724 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T00:50:42.724 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-02-18T00:50:42.724 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +3165us[+3171us] +/- 120ms 2024-02-18T00:50:42.724 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -1177us[-1170us] +/- 59ms 2024-02-18T00:50:42.725 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -386us[ -380us] +/- 90ms 2024-02-18T00:50:42.725 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T00:50:42.725 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-18T00:50:42.736 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-18T00:50:42.736 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-18T00:50:42.803 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-18T00:50:42.904 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-18T00:50:42.985 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T00:50:43.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:50:43.020 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T00:50:43.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:50:43.052 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-18T00:50:43.062 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-18T00:50:43.072 INFO:teuthology.task.internal:Duration was 2273.046125 seconds 2024-02-18T00:50:43.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-18T00:50:43.081 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-18T00:50:43.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-18T00:50:43.084 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-18T00:50:43.131 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T00:50:43.137 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T00:50:43.512 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-18T00:50:43.512 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-02-18T00:50:43.513 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-18T00:50:43.538 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-02-18T00:50:43.539 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-18T00:50:43.562 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-18T00:50:43.562 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-18T00:50:43.581 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-18T00:50:43.702 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-18T00:50:43.702 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-18T00:50:43.705 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-18T00:50:44.010 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-18T00:50:44.020 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-18T00:50:44.021 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-18T00:50:44.049 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-18T00:50:44.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-18T00:50:44.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-18T00:50:44.091 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-18T00:50:44.116 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-02-18T00:50:44.143 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-02-18T00:50:44.165 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-18T00:50:44.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:50:44.186 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-18T00:50:44.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:50:44.222 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-18T00:50:44.235 INFO:teuthology.task.internal:Transferring archived files... 2024-02-18T00:50:44.236 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957/remote/smithi059 2024-02-18T00:50:44.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-18T00:50:44.301 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957/remote/smithi137 2024-02-18T00:50:44.302 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-18T00:50:44.351 INFO:teuthology.task.internal:Removing archive directory... 2024-02-18T00:50:44.352 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-18T00:50:44.354 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-18T00:50:44.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-18T00:50:44.413 INFO:teuthology.task.internal:Not uploading archives. 2024-02-18T00:50:44.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-18T00:50:44.437 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-18T00:50:44.438 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-18T00:50:44.440 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-18T00:50:44.456 INFO:teuthology.orchestra.run.smithi059.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 18 00:50 /home/ubuntu/cephtest 2024-02-18T00:50:44.457 INFO:teuthology.orchestra.run.smithi137.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 18 00:50 /home/ubuntu/cephtest 2024-02-18T00:50:44.458 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-18T00:50:44.471 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-18T00:50:44.493 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: TypeError: expected string or bytes-like object 2024-02-18T00:50:44.493 INFO:teuthology.nuke:Checking targets against current locks 2024-02-18T00:50:44.522 DEBUG:teuthology.nuke:shortname: smithi059 2024-02-18T00:50:44.522 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T00:50:44.544 DEBUG:teuthology.nuke:shortname: smithi137 2024-02-18T00:50:44.544 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T00:50:44.550 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:04:05.334488', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPgEiIgn9ZRpFMBfeYM7W7ByetwlioK7CSZpkeBUEhmvUvm0L/RU4GpmH1aDZHPQdzqWONLS4ciTMHBjYvM3y8A='} 2024-02-18T00:50:44.566 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7564957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:04:05.333105', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK3u1MOUZPUsflX1PKVxUVIu5WPzW4XRo8+yeR5RZVHxAE5Gyaz5Jesqc7uJXNjywTxonkQbwCeRga6+De6GT5g='} 2024-02-18T00:50:44.601 INFO:teuthology.orchestra.console.smithi059:Power off 2024-02-18T00:50:44.601 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T00:50:44.619 INFO:teuthology.orchestra.console.smithi137:Power off 2024-02-18T00:50:44.620 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T00:50:44.630 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-02-18T00:50:44.630 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T00:50:44.643 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-02-18T00:50:44.643 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T00:50:44.660 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-02-18T00:50:44.668 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-02-18T00:50:48.661 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T00:50:48.670 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T00:50:48.773 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-02-18T00:50:48.783 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-02-18T00:50:52.776 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T00:50:52.784 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T00:50:52.887 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-02-18T00:50:52.894 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-02-18T00:50:52.995 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-02-18T00:50:56.892 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T00:50:57.002 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-02-18T00:50:57.103 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-02-18T00:50:57.207 INFO:teuthology.run:Summary data: description: rados/cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} duration: 2273.0461246967316 failure_reason: expected string or bytes-like object flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-02-18T00:50:57.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T00:50:57.302 INFO:teuthology.run:FAIL